Re-ran the 1A gate on the docker-dev rig at e54883a8 after the defect-4 boot
fix. Image rebuilt (otopcua-host:dev -> 1d8452a29018); six host nodes plus
both one-shots recreated; run 1's preserved image was stale and deliberately
not reused. VERDICT: PASS.
Boot: 6/6 nodes healthy INCLUDING the central pair, 0 Hosting failed to start,
/health/ready 200 on every node, MAIN mesh 2/2.
Check 1 PASS: one DispatchDeployment, one correlation id, accepted by all six
nodes; central logs 6 accepted / 0 failed; zero ClusterClient lines on either
central node with Akka.Cluster.Tools.Client at Debug.
Check 2 PASS: deployment Sealed with all six NodeDeploymentState rows Applied
inside ~90 ms, while the four site nodes hold receptionist associations — the
only ack path they have in 1A. Mixed transport proven.
Check 4 PASS: Ack clicked on the real /alerts page against a live ACTIVATED
row; central fanned one AlarmCommand to all six over gRPC and the condition
acknowledged 0.54 s later carrying the operator identity.
Check 7 PASS: with site-b-2's GrpcPort NULLed, both central nodes warned once
and the next dispatch reached exactly five nodes; the deployment then honestly
ended TimedOut with site-b-2 alone unapplied. Row restored, 6/6 sealed after.
Check 8 PASS and no longer vacuous: 0 denials on all four interceptors, and
this time config-serve (28 KB verified artifact fetches from :4055) and
telemetry (streams open on all six, live driver health on /hosts) both carried
real traffic.
Check 3 closed by composition, with the reason now established rather than
assumed: <DriverStatusPanel has 0 usages repo-wide, no rendered page carries a
Restart/Reconnect control, and the only non-script-analysis POST is
/api/deployments. Run 1 drove the node-side leg for both commands; check 4
drives the identical AdminOperationsActor.Dispatch -> DispatchOverGrpc origin
path from the AdminUI.
Checks 5 + 6 stand from run 1 — nothing in the fix touches the interceptor.
New finding 5, recorded because it is load-bearing for the fix's design: the
PreStart resolve LOSES its race on real boots — 2 of 3 observed admin boots
logged Failed to resolve for both dependencies with the same
AuditWriterActorKey cause. The memoize-on-first-use hardening is therefore not
theoretical, and its heal was proven live by moving the coordinator singleton
onto the node that lost the race (stop central-2) and watching central-1 fan a
dispatch to all five live nodes with no dialer error. Residual is cosmetic and
carried to Phase 2: the Error text reads as permanent, and it will trip a soak
error-grep on a healthy node.
Defect 4 marked CLOSED; run 1's record kept in full underneath the combined
table. Rig restored: flip reverted, six nodes healthy on all-ClusterClient on
the committed image, three mesh ClusterClients back on each central node.
Joseph Doherty
e3a0ddbc84