Files
scadaproj/docs/plans/evidence/2026-11-12-overview-phase4/split-timeline.txt
T
Joseph Doherty 4909946104 docs(overview): Phase 4 live acceptance complete - OtOpcUa one-Active + leader/split-brain chips pass (#14)
Re-ran the two design 9 checks that were blocked for OtOpcUa on 2026-07-24, against a
docker-dev rig from OtOpcUa master 412da27c (Health 0.3.0) with the dashboard in a
container on the rig network:

1. one Active per pair: exactly one /health/active 200 per mesh (MAIN central-2,
   SITE-A site-a-1, SITE-B site-b-1), dashboard shows one ACTIVE + one STANDBY each;
2. leader chip on every group; network-partitioning site-b-2 from site-b-1 produced
   two 1-member clusters, both Active, and the SPLIT BRAIN chip, which cleared once
   site-b-2 was restarted (healing the link alone does not merge two Akka clusters,
   the documented OtOpcUa recovery); stopping site-a-1 moved the SITE-A chip.

Evidence (screenshots, /health/active bodies, timeline) under
docs/plans/evidence/2026-11-12-overview-phase4/. Ledger task 20 -> completed (20/20).

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 22:47:13 -05:00

41 lines
3.4 KiB
Plaintext

# All times UTC 2026-11-13 (= 2026-11-12 evening US Eastern). Probe: GET /health/active from inside the rig
# network (site-b-2 via the g761-probe network while partitioned). Format: node HTTP members unreachable leader | cluster-primary.
02:49:45 site-b-2 disconnected from otopcua-g761_default (stays reachable to the dashboard via g761-probe)
site-b-1 200 members 2 unreach 0 leader akka.tcp://otopcua@site-b-1:4053 | Active node for role 'driver' (oldest Up member).
site-b-2 503 members 2 unreach 0 leader akka.tcp://otopcua@site-b-1:4053 | Standby: active node for role 'driver' is akka.tcp://otopcua@site-b-1:4053.
02:49:54
site-b-1 200 members 2 unreach 1 leader akka.tcp://otopcua@site-b-1:4053 | Active node for role 'driver' (oldest Up member).
site-b-2 503 members 2 unreach 1 leader akka.tcp://otopcua@site-b-2:4053 | Standby: active node for role 'driver' is akka.tcp://otopcua@site-b-1:4053.
02:50:05
site-b-1 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-1:4053 | Active node for role 'driver' (oldest Up member).
site-b-2 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-2:4053 | Active node for role 'driver' (oldest Up member).
02:50:18 (unchanged at 02:50:30, 02:50:41, 02:50:53; screenshot 14-check2-splitbrain.png at 02:51:10)
healed network at 02:51:41
site-b-1 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-1:4053 | Active node for role 'driver' (oldest Up member).
site-b-2 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-2:4053 | Active node for role 'driver' (oldest Up member).
02:51:43
site-b-1 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-1:4053 | Active node for role 'driver' (oldest Up member).
site-b-2 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-2:4053 | Active node for role 'driver' (oldest Up member).
02:51:54
site-b-1 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-1:4053 | Active node for role 'driver' (oldest Up member).
site-b-2 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-2:4053 | Active node for role 'driver' (oldest Up member).
02:52:05
site-b-1 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-1:4053 | Active node for role 'driver' (oldest Up member).
site-b-2 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-2:4053 | Active node for role 'driver' (oldest Up member).
02:52:16
site-b-1 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-1:4053 | Active node for role 'driver' (oldest Up member).
site-b-2 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-2:4053 | Active node for role 'driver' (oldest Up member).
02:52:27
site-b-1 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-1:4053 | Active node for role 'driver' (oldest Up member).
site-b-2 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-2:4053 | Active node for role 'driver' (oldest Up member).
02:52:38
site-b-1 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-1:4053 | Active node for role 'driver' (oldest Up member).
site-b-2 200 members 1 unreach 0 leader akka.tcp://otopcua@site-b-2:4053 | Active node for role 'driver' (oldest Up member).
02:52:50
restarted site-b-2 at 02:53:10
site-b-1 200 members 2 unreach 0 leader akka.tcp://otopcua@site-b-1:4053 | Active node for role 'driver' (oldest Up member).
site-b-2 503 members 2 unreach 0 leader akka.tcp://otopcua@site-b-1:4053 | Standby: active node for role 'driver' is akka.tcp://otopcua@site-b-1:4053.
02:53:25
stop site-a-1 at 02:53:44
start site-a-1 at 02:54:24