dohertj2/lmxopcua
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(mesh): MeshTransport config, Phase 2 as shipped, design-doc correction

Browse Source 
Rig: all six docker-dev nodes carry MeshTransport__Mode plus both central
contact points. Mode is "${OTOPCUA_MESH_MODE:-Dps}", so the rig still comes up
on the transport it has always used and the live gate can flip the whole fleet
at `docker compose up` without a compose edit or a rebuild.

Docs: a MeshTransport section in Configuration.md (including why contact points
are addresses only, why the contact set does not scope delivery, and why
buffer-size=0 is a behaviour decision rather than a tuning knob), a "Command
transport" section in Redundancy.md contrasting the two modes, and a summary
beside the Redundancy notes in CLAUDE.md.

Corrects the design doc's claim that ScadaBridge replies with a typed failure to
every unhandled message. Neither of their comm actors has a ReceiveAny or an
Unhandled override; the idiom is per message type and fires on a missing
registration, and an unknown type dead-letters there as it does anywhere else.

Records the exit-gate deviation in the program plan: "an Ask timing out cleanly
against a stopped node" cannot be run, because no Ask crosses this boundary. The
honest equivalents are a stopped node failing the deploy at the apply deadline
and an unreachable contact dropping the command with a Warning.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
Joseph Doherty
2026-07-22 17:46:46 -04:00
parent 4eca87f04d
commit e4cb9a08e0
6 changed files with 202 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/2026-07-21-per-cluster-mesh-design.md
+8 -2
View File
@@ -68,8 +68,14 @@ Other load-bearing details:
caller's Ask times out. *"It keeps the central coordinator stateless with respect to site
availability."* A `ConnectionStateChanged` mechanism was built for this and **deleted as dead
code**.
- **Every unhandled message replies with a typed failure rather than dropping**, so a central Ask
can never stall on a missing handler.
- ~~**Every unhandled message replies with a typed failure rather than dropping**, so a central Ask
can never stall on a missing handler.~~ **CORRECTED 2026-07-22 (Phase 2 recon).** This overstates
what ScadaBridge does. Neither of their comm actors has a `ReceiveAny` or an `Unhandled` override:
the typed-failure idiom is **per message type**, and it fires only when a *registration* is missing
(no ClusterClient for the site yet, no handler registered for that type). A genuinely unknown
message type still dead-letters exactly as it would anywhere else. Do not build a catch-all failure
reply believing you are copying them. OtOpcUa's Phase 2 comm actors list each type explicitly and
deliberately let an unknown type dead-letter loudly rather than republish it blind.
- **Sender preservation is the whole Ask idiom**: `_client.Tell(new ClusterClient.Send(path, msg),
Sender)` routes the reply straight back to the waiting Ask, bypassing the comm actor.
docs/plans/2026-07-22-per-cluster-mesh-program.md
+26 -2
View File
@@ -99,7 +99,31 @@ this removes the coordinator's one genuinely mesh-bound dependency).
set proven DB-sourced (test: a `ClusterNode` row present but node down → deploy reports that node
missing; a node up but row absent → its ack is not expected).
### Phase 2 — Comm actors + ClusterClient transport
### Phase 2 — Comm actors + ClusterClient transport — **CODE COMPLETE 2026-07-22, live gate pending**
Shipped per [`2026-07-22-mesh-phase2-clusterclient-transport.md`](2026-07-22-mesh-phase2-clusterclient-transport.md)
as a **dark switch**: `MeshTransport:Mode` defaults to `Dps`, so nothing changes on any deployment
until the flag flips. Both comm actors are spawned and receptionist-registered in **both** modes, so
the flip is a config change rather than a redeploy. Three scope corrections landed against the
design doc (that plan's header carries them in full):
- the node path is `/user/node-communication`, not `/user/cluster-communication` (there is no
per-cluster scoping until Phase 6);
- the substitution for `Publish` is **`SendToAll`**, not `Send` — `Send` reaches exactly one node and
would deploy to one member of the fleet while the rest silently kept their old config;
- central creates **exactly one fleet-wide ClusterClient**. On a single mesh a receptionist serves
its whole cluster, so a client per application `Cluster` would fan every command out once per
cluster. Per-cluster clients become correct only when Phase 6 splits the meshes (`TODO(Phase 6)`
in `CentralCommunicationActor.RebuildClient`).
**Exit-gate deviation — "an Ask timing out cleanly against a stopped node" cannot be run as written.**
There is no cross-boundary Ask in Phase 2: every migrated command is fire-and-forget on the wire.
`DispatchDeployment` is a `Tell` whose ack returns later as an independent `ApplyAck`;
`RestartDriver` / `ReconnectDriver` / `AlarmCommand` reply `Ok = true` from the admin node the moment
they are dispatched (`Ok` has always meant *dispatched*, not *applied* — the AdminUI string is
literally "Restart dispatched"). The honest equivalents, both in the live gate: a **stopped node**
fails the deploy at the apply deadline naming it (Phase 1 semantics), and **no reachable contact**
drops the command with a Warning and never buffers it.
**Scope:** one receptionist-registered actor per side (`/user/central-communication`,
`/user/cluster-communication`), registered **per node, not as a singleton** (contact rotation);
ClusterClient central → cluster carrying deploy notify + acks + driver-control; the ScadaBridge
@@ -200,7 +224,7 @@ resource sizing on the site VMs should be checked once both products run the ful
| 0b oldest-Up election | DONE 2026-07-21 |
| Prereq: selfform-fallback + manual-failover plan | **DONE 2026-07-22**, merged `a78425ea`. Shipped *not* as the planned `SelfFormAfter` watchdog — the live gate caught that islanding a manually-failed-over node — but as self-first seed ordering + `AkkaClusterOptionsValidator`. Phase 6's seed-ordering scope is therefore already discharged. |
| 1 ClusterNode columns + DB ack set | **DONE 2026-07-22**, merged `7654f24d`. Deploy-seal semantics changed as flagged, but the escape hatch is **`ClusterNode.MaintenanceMode`**, not `Enabled=0` — the live gate found `Enabled=0` is rejected by `DraftValidator.ValidateClusterTopology` on any 2-node pair. Gate record: `2026-07-22-mesh-phase1-live-gate.md`. |
| 2 ClusterClient transport | **IN PROGRESS** — plan being written 2026-07-22 |
| 2 ClusterClient transport | **CODE COMPLETE 2026-07-22, live gate pending** — shipped dark (`MeshTransport:Mode=Dps` default), both comm actors registered in both modes. Corrections: `SendToAll` not `Send`, one fleet-wide client not one per cluster, `/user/node-communication` not `/user/cluster-communication`. The gate's "Ask timing out" item does not exist to test — no cross-boundary Ask in this phase; see the phase section. |
| 3 fetch-and-cache | not started |
| 4 cut driver ConfigDb | not started |
| 5 gRPC telemetry | not started |