State

State is the idea that boggles almost everyone eventually, and the reason is simple: we cram four different things into one word and one memory cell. Pull them apart and the fog lifts.

Start where you already think: dynamical systems

You are at home here. A dynamical system has a state — the snapshot of everything you need to know right now to compute what comes next. Time advances and a transition rule carries one state to the next: $x_{t+1} = f(x_t)$. The trajectory is the whole succession of states; the system persists through all of them, while any single state is just one frame.

Hold that picture, because programming’s version is identical — we just usually hide it.

The tangle

In most languages a variable is a place in memory you overwrite, and that one place silently fuses four ideas:

• a value — something that never changes; 3 is always 3, a list is just those elements;
• an identity — a logical thing we track over time (“my account”, “the cursor”);
• a state — the value an identity has at one moment;
• time — the succession of those states.

Overwrite the place and the old value is gone. Let two threads touch it at once and you get a half-updated read — hence locks, races, and the bugs that only show up on a customer’s machine. This is why state is the hardest part of concurrency.

The people who saw it

Heraclitus, twenty-five centuries ago, in the one line of his everyone remembers: you cannot step in the same river twice. Panta rhei — everything flows. The river is an identity; the water is an endless succession of values. He had the whole model in hand before there were any computers to muddle it.

Rich Hickey made it operational. He was in his mid-forties, with no employer and no client, living off savings he had set aside on purpose so he could think without anyone asking when it would ship. He had spent years watching capable teams drown in the same bug: shared data changing under them. What he reached was not a feature but a distinction the field had blurred for decades: the account is not the balance. The identity persists; the value is only a snapshot; time is the succession of snapshots. His word for pulling those apart is decomplect, and once you have seen the seam you cannot unsee it.

Side by side

• C — the place model. A variable is memory you mutate; identity, value and time collapse into one overwritten cell. Fast, and a minefield to share.
• Erlang — no shared state at all. A process is an identity; its state is a plain argument to a loop, evolved by receiving a message and recursing with a new value. State as a parameter, never a place.
• Clojure — values are immutable; an identity is a reference (an atom) you swap atomically from one value to the next. You can keep any past value forever, and nobody ever sees a half-change.

Why it endures

Once state is “a value at a time”, a great deal falls out for free: undo and time-travel, event sourcing, audit logs, git (a commit is a value; a branch is an identity), and Datomic (a database as a value that accretes over time). Concurrency stops being terrifying, because immutable values are safe to share and time is made explicit instead of raced over. It is the same move dynamical systems made long ago: keep the state separate from the thing that has it. (Its twin idea is Immutability, which is what makes the swap from one value to the next safe.)

Try it

Clojure — one identity, a frozen snapshot, an atomic change. Watch the old snapshot stay put while the account moves on:

;; A value never changes. An identity is a succession of values over time.
(def account (atom {:balance 100}))   ; identity: a reference
(def snapshot @account)               ; a value, frozen forever
(swap! account update :balance + 50)  ; swap to a new state, atomically
(println "now:" @account)             ; {:balance 150}
(println "snapshot still:" snapshot)  ; {:balance 100}, unchanged

Erlang — state with no mutation at all, living inside a process as a loop argument:

%%% State without mutation: a process holds its state as a loop argument.
main(_) ->
    C = spawn(fun() -> loop(0) end),
    C ! {add, 5},
    C ! {add, 3},
    C ! {get, self()},
    receive {value, V} -> io:format("counter = ~w~n", [V]) end.

loop(N) ->
    receive
        {add, X}    -> loop(N + X);
        {get, From} -> From ! {value, N}, loop(N)
    end.

[!recall] Identity, value, and state are three different things. How do they relate? A value is immutable (3 is always 3; a list never changes). An identity is a logical entity we follow over time (an account, a river). State is the value an identity has at a particular moment. Mutable-place languages fuse all three into one overwritable cell; separating them — immutable values, identities as references to a succession of values, and time made explicit — is exactly what makes concurrency and reasoning about change tractable.

Sources

• Hickey, “The Value of Values” (2012 talk) — values, time, and information that accretes rather than overwrites.
• Hickey, “Simple Made Easy” (2011 talk) — the decomplecting argument in full.

See also: Clojure, Erlang, C, Lisp.