docs(mesh): Phase 2 live-gate record + correct the buffer-size rationale

Flipped the docker-dev rig end-to-end to MeshTransport:Mode=ClusterClient
(OTOPCUA_MESH_MODE, no rebuild) and ran the gate. PASSED for the transport:
deploy seals on ClusterClient, a stopped node fails the deploy naming it, the
MaintenanceMode hatch shrinks the DB-sourced contact set live (6->5->6 on both
centrals independently), and the frame-size canary stayed silent.

Four findings, recorded in 2026-07-22-mesh-phase2-live-gate.md:

  A. A 10-30s post-restart delivery window exists, but the DPS control fails
     identically -- it is a cluster-convergence property, not a Phase 2
     regression. Running the control is the only reason this is a note and not
     a false attribution.

  B. buffer-size=0's stated rationale was OVERSTATED. A node applies a TimedOut
     deployment anyway -- live-observed at +44s -- via DriverHostActor.Bootstrap
     replaying the orphan Applying row on restart. Zero buffering removes the
     silent, timer-driven replay, not the boot-time DB one. Corrected in the
     plan decisions table, Configuration.md, and CLAUDE.md; the decision stands,
     the justification narrowed.

  C. Stopping BOTH centrals strands the non-seed nodes at the Akka membership
     layer (frozen mid-Exiting, not Unreachable, so auto-down does not help).
     Transport-independent; a Phase 7 downing/rejoin drill item.

  D. The AdminUI driver Restart/Reconnect buttons (DriverStatusPanel) are
     rendered by no page -- orphaned since v3 Batch 2 retired /clusters/{id}/
     drivers. So gate steps 6/7 could not run; the driver-control node-side leg
     is proven live only because DispatchDeployment shares its EventStream path.

Step 8's planned log-signal does not exist (HandleApplyAck is silent on
success); proven instead by both centrals independently logging per-instance
ClusterClient creation, which a singleton cannot.

Also adds a repo-root .dockerignore: docker-dev/Dockerfile does COPY . ., which
was pulling .claude/worktrees (19GB of agent checkouts with their own bin/obj)
into the build context and exhausting the Docker VM disk mid-COPY.

Rig restored to Mode=Dps, six members, no leftover maintenance flags.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
Joseph Doherty
2026-07-22 18:40:42 -04:00
parent 986b640eea
commit 32055c2238
7 changed files with 274 additions and 8 deletions
+3 -1
View File
@@ -127,7 +127,9 @@ The checked-in `appsettings*.json` files are deliberately thin: they carry only
**Central's contact set comes from the enabled, non-maintenance `ClusterNode` rows** — the same source Phase 1 gave the deploy ack set. A node in `MaintenanceMode` is not dialled, but note that **the contact set does not scope delivery**: a receptionist serves its whole cluster, so `SendToAll` still reaches every registered node exactly as today's DPS broadcast does. The filter is about which receptionists are worth dialling, not about who receives.
**Buffering is off by design** (`akka.cluster.client.buffer-size = 0`, set in the shipped `akka.conf`). If no receptionist is known, the command is dropped with a Warning rather than queued. Buffering would replay a `DispatchDeployment` on reconnect and apply a deployment the coordinator had already sealed as `TimedOut`, leaving the node running a configuration the database records as failed. Do not "fix" this back to Akka's default of 1000 to quiet a flaky test.
**Buffering is off by design** (`akka.cluster.client.buffer-size = 0`, set in the shipped `akka.conf`). If no receptionist is known, the command is dropped rather than queued. Akka's default of 1000 would replay a `DispatchDeployment` on reconnect — silently, on the client's own timer, at an unbounded remove from the dispatch. Do not "fix" it back to quiet a flaky test.
> **What this does *not* buy you.** It does not guarantee a node never applies a deployment the DB records as `TimedOut`. A node that missed a dispatch keeps the orphan `Applying` row the coordinator pre-created, and `DriverHostActor.Bootstrap()` replays that row on the node's next restart — live-observed applying a deployment 44 s after it was sealed `TimedOut` (see the [Phase 2 live gate](plans/2026-07-22-mesh-phase2-live-gate.md), finding B). The boot replay is the better-behaved of the two — deliberate, Warning-logged, and bounded to a restart — but it is still a replay. `buffer-size = 0` removes the *silent, timer-driven* one only.
On the docker-dev rig every node carries these keys already; flip the whole rig with `OTOPCUA_MESH_MODE=ClusterClient` at `docker compose up`.