Phase 2 of the per-cluster mesh program: move the three central->node command
channels (deployments, driver-control, alarm-commands) and the deployment-acks
reply channel off DistributedPubSub onto Akka ClusterClient, behind a config
flag.
Recon of both repos turned up three corrections to the design doc's account of
the sister project, all folded into the plan:
- Design doc S2 claims "every unhandled message replies with a typed failure".
ScadaBridge has no ReceiveAny/Unhandled override in either comm actor; the
typed-failure idiom fires only for a MISSING REGISTRATION, per message type.
- "No central buffering" was never a setting they wrote. They have zero
akka.cluster.client HOCON and run the defaults (buffer-size 1000,
reconnect-timeout off). We choose buffer-size = 0 deliberately.
- Publish -> Send is the wrong substitution. Today's deploy notify is a
broadcast every DriverHostActor receives (no ClusterId filter exists on the
node side), so it must become SendToAll. Send would deploy to exactly one
node of the fleet and seal green on partial acks.
Also records the single-mesh duplicate-delivery trap (one ClusterClient in
Phase 2, not one per Cluster -- a receptionist serves its whole mesh) and one
deviation from the program plan's exit gate: there is no cross-boundary Ask in
Phase 2, because every migrated command is fire-and-forget with a local reply,
so "an Ask timing out cleanly" cannot be run as written.
Marks the program tracking tables current: prereq + Phase 1 are DONE (they
still read "not executed"), Phase 2 in progress.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
docs/Redundancy.md's bootstrap section is rewritten around the mechanism that is
actually in force: which of Akka's two bootstrap processes runs is decided by
whether seed-nodes[0] is this node's own address, so the ORDER of Cluster:SeedNodes
is the fix, not a timer. Adds the per-process table, the shipped self-first
example, why the validator is conditional (site nodes are not seeds) and why it
matches PublicHostname rather than the 0.0.0.0 bind address.
The retirement is documented rather than erased: a new subsection explains that a
watchdog outside the join handshake cannot tell "no seed answered" from "a seed
answered and the join is in flight", and the ea45ace1 live-gate finding is kept
verbatim as the evidence that produced that conclusion (InitJoinAck at 10:49:05,
old incarnation retired 10:49:06, watchdog fired 10:49:15, second cluster). The
watchdog's own earlier PASS is kept too — it did pass what it was gated on.
Also: a live gate for the NEW mechanism is registered as outstanding (not
re-drilled on the docker-dev rig since the swap), and the plan doc that shipped
the watchdog gets a superseded banner rather than an edit, so the execution record
and its Task 8 finding stay readable.
Ripple: CLAUDE.md cluster section, docs/Configuration.md + docs/v2/Cluster.md +
docs/ServiceHosting.md SeedNodes entries, the per-cluster-mesh design doc's two
"CLOSED by SelfFormAfter" notes, and mesh-program Phase 6 (which had this
convergence queued — now marked done early) + Phase 7's live-gate name.
Joseph Doherty