Joseph Doherty
a78425ea8f
Corrects the plan's behavior table (the mid-join-handshake row was FALSE) and notes the mesh-program Phase 6 convergence on ScadaBridge's self-first seed ordering, which retires the watchdog + guard entirely. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
15 lines
3.1 KiB
JSON
15 lines
3.1 KiB
JSON
{
|
|
"planPath": "docs/plans/2026-07-22-selfform-fallback-and-manual-failover.md",
|
|
"tasks": [
|
|
{"id": 0, "subject": "Task 1: SelfFormAfter on AkkaClusterOptions", "status": "completed"},
|
|
{"id": 1, "subject": "Task 2: ClusterBootstrapFallback + AddStartup wiring", "status": "completed", "blockedBy": [0]},
|
|
{"id": 2, "subject": "Task 3: SelfFormBootstrapTests — 4 scenarios incl. site-node island guard", "status": "completed", "blockedBy": [1], "notes": "Teeth proven: scenario 2 run with selfFormAfter:null failed (red) before restore. A 5th test was added by Task 8."},
|
|
{"id": 3, "subject": "Task 4: Docs — Redundancy.md bootstrap section + mesh design note", "status": "completed", "blockedBy": [1]},
|
|
{"id": 4, "subject": "Task 5: Full verification + optional docker-dev live check", "status": "completed", "blockedBy": [2, 3], "notes": "Live gate PASSED for the cold-start case (defect observed on old image, fix on new, island guard on a real site node, one six-node cluster after recovery)."},
|
|
{"id": 5, "subject": "Task 6: Manual failover service + cluster-level test", "status": "completed", "blockedBy": [4], "notes": "4/4. Teeth proven: sabotaging FailOverCore to address order turned the target + parity tests red."},
|
|
{"id": 6, "subject": "Task 7: ClusterRedundancy live panel + failover button + tests", "status": "completed", "blockedBy": [5], "notes": "PLAN DEVIATION: plan said bUnit; repo has no bUnit (PageAuthorizationGuardTests documents itself as the substitute). Adapted to a pure ManualFailoverPageModel + 9 unit tests, razor as thin shell. Policy is FleetAdmin, not the ConfigEditor the neighbouring cluster pages use (this restarts a node; ConfigEditor admits Designer). Live-verified: panel, confirm dialog, execution, singleton handover, audit row."},
|
|
{"id": 7, "subject": "Task 8 (NEW, from live gate): reachability guard — don't self-form while a seed peer is reachable", "status": "completed", "blockedBy": [6], "notes": "DEFECT: the Task 7 live drill caught the self-form fallback islanding the node the failover bounced. central-1 restarted, got InitJoinAck from central-2 at 10:49:05 (join in flight, healthy) but no Welcome inside the window (peer's ring still held the old incarnation until 10:49:06); at 10:49:15 the fallback fired and formed a SECOND cluster. The plan's 'Window expires mid-join-handshake -> benign, Akka ignores Join once joined' row is FALSE — the node had not joined, and Join(self) wins. Manual failover deliberately produces that restart, so the two halves of the plan collided. FIX (user chose the reachability guard): TCP-probe the other seed addresses before self-forming; a reachable peer means wait another window and re-arm. Proven three ways: (1) new regression test whose 'peer' is a bare TcpListener speaking no Akka — RED without the guard; (2) live, a throwaway node seeded at sql:1433 (TCP-reachable, not Akka) logged the guard every 10s and never self-formed; (3) positive control, the same node seeded at a dead port self-formed at +10s. Failover re-drill on the rig: no island, all six nodes one cluster."}
|
|
],
|
|
"lastUpdated": "2026-07-22T00:00:00Z"
|
|
}
|