Introduction to Arcadia

Arcadia is an asynchronous calculation framework inspired by a discussion by Tobias Gedell on Eden YouTube video and an article series by Daniel Earwicker on his Eventless library link.

The main points of Eden that stuck with me were :

Laziness and partial recalc
Caching
Asynchronous result production
Automatic parallelization
Optional manual calculation
Cancellation

Currently I have implemented the above plus basic error handling (changes the node with error to an Error status, no logging of error currently.)

TO DO LIST
logging
redo/undo
serialization/persistense of CalculationEngine to database

Arcadia is implemented using .Net generics so calculation "nodes" do not need to implement just a single numberic value. Inputs/Outputs can be any POCO/recordset/struct that you want.

Node Dependency Graph

Here is a dependency graph with input nodes (green) and output nodes (blue). We will use this as an illustration of the dependency tree that we will now try to replicate using simple integer based nodes.

F# Example - simple integers

First lets define some simple functions to represent some slow running functions.

1: 
2: 
3: 
4: 
5: 
6: 
7: 
8: 
9:

open System.Threading

let add2 (x1,x2) = 
    Thread.Sleep 500
    x1 + x2

let add3 (x1,x2,x3) =
    Thread.Sleep 1000
    x1 + x2 + x3

Now lets create a calculation engine that does simple addition at nodes based on the dependency graph we saw earlier. An optional custom ID can be assigned to a node. If no node ID is given then Setables will be named in0, in1, in2, ... and Getables will be named out0, out1, out2, ...
For F# there is an additional operator that allows in0.Value to be replaced with !!in0. For C# there is an implicit coversion of Getable<T> in0 to T in0.Value.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
21: 
22: 
23: 
24: 
25: 
26: 
27: 
28: 
29: 
30: 
31: 
32: 
33: 
34: 
35: 
36: 
37: 
38:

open Arcadia
open Arcadia.FSharp

type SimpleCalculationEngine() as x =
    inherit CalculationEngine()

    do        
        // input nodes
        let in0 = x.Setable 1
        let in1 = x.Setable 1
        let in2 = x.Setable 1
        let in3 = x.Setable 1
        let in4 = x.Setable 1
        let in5 = x.Setable 1
        let in6 = x.Setable 1
        let in7 = x.Setable 1
        let in8 = x.Setable 1
        let in9 = x.Setable 1
        let in10 = x.Setable 1
        let in11 = x.Setable 1
        let in12 = x.Setable 1
        let in13 = x.Setable 1

        // main calculation chain
        let out0 = x.Computed(fun () -> add2 !!in0 !!in1)
        let out1 = x.Computed(fun () -> add2 !!in2 !!in3)
        let out2 = x.Computed(fun () -> add3 !!in4 !!in5 !!in6)
        let out3 = x.Computed(fun () -> add2 !!in7 !!in8)
        let out4 = x.Computed(fun () -> add2 !!out1 !!out2)
        let out5 = x.Computed(fun () -> add2 !!out0 !!out3)
        let out6 = x.Computed(fun () -> add2 !!in9 !!in10)
        let out7 = x.Computed(fun () -> add2 !!in11 !!in12)
        let out8 = x.Computed(fun () -> add2 !!out4 !!out6)
        let out9 = x.Computed(fun () -> add3 !!out5 !!out7 !!out8)

        // secondary calculation chain
        let out10 = x.Computed(fun () -> add2 !!out0 !!out5)
        let out11 = x.Computed(fun () -> !!in13)

Test out our Calculation Engine

Create an instance of the calculation engine and turn on automatic calculations. Run the following a statement at a time and see how it works.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13:

let ce = SimpleCalcEngine()

/// print out the status and value of a given node.
let nodeValue(nodeId) = 
    let n = ce.Node<int>(nodeId)
    printfn "%s status:%A value:%i" (n.Id) (n.Status) (n.Value)

nodeValue "out9" // returns "out9 status:Dirty value:0"

ce.Calculation.Automatic <- true

// check again (will need to wait a few seconds while calculations complete)
nodeValue "out9" // returns "out9 status:Valid value:13"

You can also do manual calculations if you didn't want to have everything calculating automatically.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16:

// set calculations back to manual
ce.Calculation.Automatic <- false

// set the value of in1 to 3 
ce.Node("in1").Value <- 3

// check the value of nodes dependent on in1
nodeValue "out9" // returns out9 status: Dirty value:13
nodeValue "out10" // returns out10 status: Dirty value: 6

// if we want to get the updated value we can request an update
ce.Node<int>("out9").AsyncCalculate()

// wait a couple of seconds (or not and see a Dirty result for out9)
nodeValue "out9" // returns out9 status: Valid value:15
nodeValue "out10" // returns out10 status: Dirty value: 6

Since out9 does not depend on out10 it did not recalculate (point 1 from our starting list).

Here is the above example implemented in C#.

An example of how this can be implemented in an MVVM application can be found on the GitHub site in the src/Samples folder.

namespace System

namespace System.Threading

val add2 : x1:obj * x2:obj -> (int -> obj * obj)

Full name: Introduction.add2

val x1 : obj

val x2 : obj

Multiple items
type Thread =
  inherit CriticalFinalizerObject
  new : start:ThreadStart -> Thread + 3 overloads
  member Abort : unit -> unit + 1 overload
  member ApartmentState : ApartmentState with get, set
  member CurrentCulture : CultureInfo with get, set
  member CurrentUICulture : CultureInfo with get, set
  member DisableComObjectEagerCleanup : unit -> unit
  member ExecutionContext : ExecutionContext
  member GetApartmentState : unit -> ApartmentState
  member GetCompressedStack : unit -> CompressedStack
  member GetHashCode : unit -> int
  ...

Full name: System.Threading.Thread

--------------------
Thread(start: ThreadStart) : unit
Thread(start: ParameterizedThreadStart) : unit
Thread(start: ThreadStart, maxStackSize: int) : unit
Thread(start: ParameterizedThreadStart, maxStackSize: int) : unit

Thread.Sleep(timeout: System.TimeSpan) : unit
Thread.Sleep(millisecondsTimeout: int) : unit

val add3 : x1:'a * x2:'b * x3:'d -> (int -> int -> int) (requires member ( + ) and member ( + ))

Full name: Introduction.add3

val x1 : 'a (requires member ( + ) and member ( + ))

val x2 : 'a (requires member ( + ) and member ( + ))

val x3 : 'a (requires member ( + ) and member ( + ))

namespace Arcadia

module FSharp

from Arcadia

Multiple items
type SimpleCalculationEngine =
inherit CalculationEngine
new : unit -> SimpleCalculationEngine

Full name: Introduction.SimpleCalculationEngine

--------------------
new : unit -> SimpleCalculationEngine

val x : SimpleCalculationEngine

Multiple items
type CalculationEngine =
  interface ICalculationEngine
  new : unit -> CalculationEngine
  new : calculationHandler:ICalculationHandler -> CalculationEngine
  abstract member OnPropertyChanged : string -> unit
  member Computed : nodeFunction:Func<'U> -> Computed<'U>
  member Computed : nodeFunction:Func<'U> * throttle:int -> Computed<'U>
  member Computed : nodeFunction:Func<'U> * nodeId:string -> Computed<'U>
  member Computed : nodeFunction:Func<'U> * nodeId:string * throttle:int -> Computed<'U>
  member Node : nodeId:string -> INode<'U>
  override OnPropertyChanged : string -> unit
  ...

Full name: Arcadia.CalculationEngine

--------------------
new : unit -> CalculationEngine
new : calculationHandler:ICalculationHandler -> CalculationEngine

val in0 : Setable<int>

member CalculationEngine.Setable : value:'U -> Setable<'U>
member CalculationEngine.Setable : value:'U * nodeId:string -> Setable<'U>

val in1 : Setable<int>

val in2 : Setable<int>

val in3 : Setable<int>

val in4 : Setable<int>

val in5 : Setable<int>

val in6 : Setable<int>

val in7 : Setable<int>

val in8 : Setable<int>

val in9 : Setable<int>

val in10 : Setable<int>

val in11 : Setable<int>

val in12 : Setable<int>

val in13 : Setable<int>

val out0 : Computed<obj * obj>

member CalculationEngine.Computed : nodeFunction:System.Func<'U> -> Computed<'U>
member CalculationEngine.Computed : nodeFunction:System.Func<'U> * throttle:int -> Computed<'U>
member CalculationEngine.Computed : nodeFunction:System.Func<'U> * nodeId:string -> Computed<'U>
member CalculationEngine.Computed : nodeFunction:System.Func<'U> * nodeId:string * throttle:int -> Computed<'U>

val out1 : Computed<obj * obj>

val out2 : Computed<int>

val out3 : Computed<obj * obj>

val ce : 'a

Full name: Introduction.ce

val nodeValue : nodeId:'a -> unit

Full name: Introduction.nodeValue

print out the status and value of a given node.

val nodeId : 'a

val n : 'a

Multiple items
val int : value:'T -> int (requires member op_Explicit)

Full name: Microsoft.FSharp.Core.Operators.int

--------------------
type int = int32

Full name: Microsoft.FSharp.Core.int

--------------------
type int<'Measure> = int

Full name: Microsoft.FSharp.Core.int<_>

val printfn : format:Printf.TextWriterFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.printfn