Overview Phase 4 — the two OtOpcUa-blocked §9 checks, re-run 2026-11-12 (scadaproj#14)
Design §9 (docs/plans/2026-07-22-overview-dashboard-design.md), plan task 20. On 2026-07-24 two of the nine checks
passed for ScadaBridge but were blocked for OtOpcUa (pre-Health-0.2.0 build: every node Active, no data, no leader
chip). Both are re-run here against a current OtOpcUa rig. Result: both PASS.
Setup
| OtOpcUa | docker-dev rig built from master 412da27c (Telemetry 0.3.0; pins ZB.MOM.WW.Health 0.3.0), run as its own compose project otopcua-g761 (remapped host ports, image otopcua-host:g761) so it did not touch the shared otopcua-dev rig. Three independent 2-node meshes: MAIN central-1/2, SITE-A site-a-1/2, SITE-B site-b-1/2. |
| Overview | scadaproj main d93ca7b (Overview last changed 6123d03), published framework-dependent and run in an aspnet:10.0 container joined to the rig network (otopcua-g761_default), so every node is addressed by container DNS name. Leader resolution is per group and host-only, so a host-run dashboard on localhost:* URLs could not tell pair members apart. Registry: six instances, three groups (MAIN on :9000, SITE-A/SITE-B on :8080), HasActiveRole: true, poll 5 s. |
| Clock | All times UTC 2026-11-13 02:49–02:55 (= 2026-11-12 evening US Eastern). |
Check 1 — every Active/Standby pair shows exactly one Active — PASS
GET /health/active on all six nodes (raw bodies: active-baseline.txt):
| node | HTTP | akka memberCount |
akka leader |
cluster-primary |
|---|---|---|---|---|
| central-1 | 503 | 2 | central-1 | Standby: active node for role 'admin' is central-2 |
| central-2 | 200 | 2 | central-1 | Active node for role 'admin' (oldest Up member) |
| site-a-1 | 200 | 2 | site-a-1 | Active node for role 'driver' (oldest Up member) |
| site-a-2 | 503 | 2 | site-a-1 | Standby: active node for role 'driver' is site-a-1 |
| site-b-1 | 200 | 2 | site-b-1 | Active node for role 'driver' (oldest Up member) |
| site-b-2 | 503 | 2 | site-b-1 | Standby: active node for role 'driver' is site-b-1 |
Exactly one 200 per mesh. The dashboard renders one ACTIVE + one STANDBY in all three groups
(14-check1-baseline.png). MAIN shows the documented leader/active divergence: Akka leader
central-1 (lowest address) carries LEADER + STANDBY, and central-2 (oldest Up admin member, the singleton host)
is ACTIVE.
Check 2 — leader chip, and a split-brain chip that appears and clears — PASS
- Leader chip. Every group shows
LEADER · <node>from the akka entry'sdata.leader(MAINcentral-1, SITE-Asite-a-1, SITE-Bsite-b-1) —14-check1-baseline.png. - Split brain forced. At 02:49:45
site-b-2was disconnected from the rig network (docker network disconnect), after first being attached to a private networkg761-probe(aliassite-b-2) shared only with the dashboard, so the pair lost each other but the dashboard kept reaching both. Within ~25 s each side downed the other (auto-down, the accepted availability-over-partition-safety trade): both became 1-member clusters, both answered/health/active200 and each named itself leader (split-timeline.txt). The dashboard showedSPLIT BRAIN · AKKA.TCP://OTOPCUA@SITE-B-1:4053 VS AKKA.TCP://OTOPCUA@SITE-B-2:4053on SITE-B with both cards ACTIVE (14-check2-splitbrain.png). - Clears. Reconnecting the network (02:51:41) does not end the split. Two formed Akka clusters never merge,
and both nodes stayed 1-member and Active for the next 70 s. This is the documented recovery in OtOpcUa
docs/Redundancy.md("after the link heals, restart one side"), not a dashboard or product defect. Afterdocker restartofsite-b-2(02:53:10) it rejoined; at 02:53:25 site-b-1 was 200 and site-b-2 503, bothmemberCount2, both namingsite-b-1leader, and the chip was gone (14-check2-cleared.png; rawactive-after-rejoin.txt: three 200s, one per mesh). - Leader chip moves (the §9 wording). Stopping SITE-A's leader
site-a-1(02:53:44) moved the chip toLEADER · SITE-A-2, withsite-a-2ACTIVE andsite-a-1UNREACHABLE (14-check2-leader-moved.png).site-a-1was restarted at 02:54:24 and rejoined.
Observations (none is a defect)
- The group header's "N members · M unreachable" counts registered instances (
Overview.razorgroup.Count()), not Akka membership, so it read "2 members · 0 unreachable" during the split while each node reportedmemberCount1. Akka's own count is on each card's check detail. Working as designed. - Only the resolved group leader gets a LEADER badge; during the split
site-b-2(which named itself leader) had none, and the disagreement is carried by the split-brain chip instead.
Cleanup
The otopcua-g761 rig, the g761-probe network and the dashboard container were removed after the
lmxopcua#761 gate run that shared the rig.