docs(mesh): MeshTransport config, Phase 2 as shipped, design-doc correction
Rig: all six docker-dev nodes carry MeshTransport__Mode plus both central
contact points. Mode is "${OTOPCUA_MESH_MODE:-Dps}", so the rig still comes up
on the transport it has always used and the live gate can flip the whole fleet
at `docker compose up` without a compose edit or a rebuild.
Docs: a MeshTransport section in Configuration.md (including why contact points
are addresses only, why the contact set does not scope delivery, and why
buffer-size=0 is a behaviour decision rather than a tuning knob), a "Command
transport" section in Redundancy.md contrasting the two modes, and a summary
beside the Redundancy notes in CLAUDE.md.
Corrects the design doc's claim that ScadaBridge replies with a typed failure to
every unhandled message. Neither of their comm actors has a ReceiveAny or an
Unhandled override; the idiom is per message type and fires on a missing
registration, and an unknown type dead-letters there as it does anywhere else.
Records the exit-gate deviation in the program plan: "an Ask timing out cleanly
against a stopped node" cannot be run, because no Ask crosses this boundary. The
honest equivalents are a stopped node failing the deploy at the apply deadline
and an unreachable contact dropping the command with a Warning.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
@@ -228,7 +228,44 @@ The server supports non-transparent warm/hot redundancy via the `Redundancy` sec
|
||||
|
||||
Assert the **effective** cluster config off a running `ActorSystem` (as `SplitBrainResolverActivationTests` does), never the typed options or the akka.conf text — several HOCON fragments compete and the losing one still reads correctly in the file.
|
||||
|
||||
**Still per-Akka-cluster, not per application `Cluster`.** One Primary is elected across the whole Akka mesh, so a fleet running several application clusters in one mesh (what docker-dev models) gets one Primary among all driver nodes. The fix is the per-cluster mesh workstream: `docs/plans/2026-07-21-per-cluster-mesh-design.md` (Phases 0a/0b/1 done; 2–7 not started).
|
||||
**Still per-Akka-cluster, not per application `Cluster`.** One Primary is elected across the whole Akka mesh, so a fleet running several application clusters in one mesh (what docker-dev models) gets one Primary among all driver nodes. The fix is the per-cluster mesh workstream: `docs/plans/2026-07-21-per-cluster-mesh-design.md` (Phases 0a/0b/1 done; 2 code-complete behind a dark switch; 3–7 not started).
|
||||
|
||||
## Mesh command transport (`MeshTransport`)
|
||||
|
||||
Per-cluster mesh **Phase 2** added a second transport for the three central→node command channels
|
||||
(`deployments`, `driver-control`, `alarm-commands`) and the `deployment-acks` reply, selected by
|
||||
`MeshTransport:Mode`: `Dps` (the DistributedPubSub path this repo has always used, **the default**)
|
||||
or `ClusterClient`, which does **not** require central and the node to share a gossip ring — the
|
||||
prerequisite for splitting the fleet into one mesh per application `Cluster`. Both `CentralCommunicationActor`
|
||||
(`/user/central-communication`) and `NodeCommunicationActor` (`/user/node-communication`) are spawned
|
||||
and registered with the `ClusterClientReceptionist` in **both** modes, so flipping the flag is a config
|
||||
change, not a redeploy. Things worth knowing before touching it:
|
||||
|
||||
- **`SendToAll`, never `Send`.** Today's DPS publish reaches every `DriverHostActor`; there is no
|
||||
ClusterId or node filter on the node side at all (scoping happens later, inside the artifact via
|
||||
`DeploymentArtifact.ResolveClusterScope`). `ClusterClient.Send` delivers to exactly **one** registered
|
||||
actor — it would deploy to a single node while every other node silently kept its old configuration,
|
||||
and the deployment could still seal green.
|
||||
- **Exactly ONE fleet-wide ClusterClient while the fleet is one mesh.** A receptionist serves its whole
|
||||
cluster, so `SendToAll` reaches every registered node-comm actor regardless of which node's address was
|
||||
dialled. One client per application `Cluster` — the obviously-right-looking shape, and what Phase 6
|
||||
wants — would fan each command out once per cluster (N× duplicate dispatch *and* N× duplicate ack).
|
||||
Marked `TODO(Phase 6)` in `CentralCommunicationActor.RebuildClient`. **Corollary:** the contact set does
|
||||
not scope delivery; excluding a `MaintenanceMode` node from the contacts does not stop it receiving.
|
||||
- **Inbound commands re-emit on the node's `EventStream`, not on their DPS topic** — DPS is mesh-wide and
|
||||
central `SendToAll`s to every node, so a DPS re-publish would deliver N copies to every subscriber.
|
||||
Node-side subscribers (`DriverHostActor`, `ScriptedAlarmHostActor`) accept both.
|
||||
- **Comm actors are per-node, NOT singletons.** A client rotates across contact points and must find a live
|
||||
comm actor at whichever node answers; `akka.cluster.client.receptionist.role` is empty for the same reason.
|
||||
- **`buffer-size = 0` is a behaviour decision, not a tuning knob.** Akka's default buffers 1000 and replays
|
||||
on reconnect, which would apply a `DispatchDeployment` for a deployment the coordinator already sealed
|
||||
`TimedOut` — the node would run a config the DB records as failed. Do not raise it to quiet a flaky test.
|
||||
|
||||
Contact points are node **addresses only** (`akka.tcp://<system>@<host>:<port>`); `/system/receptionist` is
|
||||
appended at construction and `MeshTransportOptionsValidator` refuses to start a node whose contact carries an
|
||||
actor-path suffix. `redundancy-state` stays on DPS in both modes (bidirectional, and built from
|
||||
`Cluster.State`). See `docs/Configuration.md` §`MeshTransport`, `docs/Redundancy.md` §"Command transport",
|
||||
and `docs/plans/2026-07-22-mesh-phase2-clusterclient-transport.md`.
|
||||
|
||||
**Phase 1 landed 2026-07-22 and changed a deploy-path behaviour.** `ConfigPublishCoordinator` sources its expected-ack set from **enabled `ClusterNode` rows**, not from `Akka.Cluster.State.Members` filtered by the `driver` role — central must be able to name a deployment's nodes without sharing a gossip ring with them. Consequences worth knowing before touching deploys:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user