R2-01: Cluster, Host & Failover round-2 fixes (T1,T3-doc,T4-T11; stacked on r2-plan02) #12

2026-07-13T11:02:48-04:00

dohertj2 commented

2026-07-13 11:02:48 -04:00

Round-2 architecture-review fixes — PLAN-R2-01 (Cluster, Host & Failover). Source: archreview/01-cluster-host-failover.md.

Stacked on r2-plan02 (shares Host/Actors/AkkaHostedService.cs + IntegrationTests/Cluster/TwoNodeClusterFixture.cs). Base = r2-plan02; retarget to main after r2-plan02 merges. TDD per task; per-task pathspec commits; solution builds clean.

T1 badc97af failover drill: kills STANDBY by default; active mode measures the registered keep-oldest outage instead of pretending recovery
T2 d5364506 per-direction drill docs (README + Component-ClusterInfrastructure)
T3 30356803 (live drill run DEFERRED — results table marked PENDING to avoid rebuilding/killing the running docker cluster; script + PASS criteria are in place)
T4 65e0f770 wire FailoverTimingTests to TwoNodeClusterFixture — measured failover envelope 33.7s (in 15–40s budget) — closes NF2
T7 e4765b2e correct master-tracker deferral record
T8 11efe4be FirstSeenAt metrics-staleness anchor
T9 c2707fdc purge stale "cluster leader" narration
T10 80e5b368 CentralSingletonRegistrar → role-aware SingletonRegistrar
T11 56d39b5f route site singletons (deployment-manager, event-log-handler) through the registrar for PhaseClusterLeave drains — Host.Tests 261/261

On-disk only (gitignored deploy/ artifact, not committed): T5 appsettings overlay (NodeName central-a/node-a, AllowSingleNodeCluster:true, phantom-seed removal), T6 install.ps1 sc.exe failureflag recovery + RUNBOOK. Owner action: sync to host + restart services.

Part of the round-2 initiative (8 plans / 86 tasks).

Round-2 architecture-review fixes — **PLAN-R2-01 (Cluster, Host & Failover)**. Source: `archreview/01-cluster-host-failover.md`. **Stacked on `r2-plan02`** (shares `Host/Actors/AkkaHostedService.cs` + `IntegrationTests/Cluster/TwoNodeClusterFixture.cs`). Base = r2-plan02; retarget to main after r2-plan02 merges. TDD per task; per-task pathspec commits; solution builds clean. - **T1** `badc97af` failover drill: kills STANDBY by default; `active` mode measures the registered keep-oldest outage instead of pretending recovery - **T2** `d5364506` per-direction drill docs (README + Component-ClusterInfrastructure) - **T3** `30356803` *(live drill run DEFERRED — results table marked PENDING to avoid rebuilding/killing the running docker cluster; script + PASS criteria are in place)* - **T4** `65e0f770` wire `FailoverTimingTests` to `TwoNodeClusterFixture` — **measured failover envelope 33.7s** (in 15–40s budget) — closes NF2 - **T7** `e4765b2e` correct master-tracker deferral record - **T8** `11efe4be` `FirstSeenAt` metrics-staleness anchor - **T9** `c2707fdc` purge stale "cluster leader" narration - **T10** `80e5b368` `CentralSingletonRegistrar` → role-aware `SingletonRegistrar` - **T11** `56d39b5f` route site singletons (deployment-manager, event-log-handler) through the registrar for `PhaseClusterLeave` drains — Host.Tests 261/261 **On-disk only (gitignored `deploy/` artifact, not committed):** T5 `appsettings` overlay (NodeName central-a/node-a, AllowSingleNodeCluster:true, phantom-seed removal), T6 `install.ps1` `sc.exe failureflag` recovery + RUNBOOK. Owner action: sync to host + restart services. Part of the round-2 initiative (8 plans / 86 tasks).

dohertj2 added 9 commits 2026-07-13 11:02:49 -04:00

fix(docker): failover drill kills the STANDBY by default; active mode measures the registered keep-oldest outage instead of pretending recovery (plan R2-01 T1) badc97afe8

docs(cluster): per-direction failover truth — first-seed bootstrap constraint + operator recovery actions; drop the undeliverable ~25s active-crash promise (plan R2-01 T2) d536450611

docs(docker): mark failover-drill results table PENDING — live drill run (plan R2-01 T3) deferred to avoid disrupting the running docker cluster 30356803e7

test(failover): unskip FailoverTimingTests on the two-node rig at production timings — measured 33.7s vs ~25s design envelope (plan R2-01 T4; covers report-08 NF2) 65e0f770d1

docs(tracker): correct the wonder-app-vd03 deferral record — artifact is present (gitignored); overlay edits applied on-disk by R2-01 T5/T6 (plan R2-01 T7) e4765b2eed

fix(health): flag metrics-stale for never-reported sites via FirstSeenAt anchor — null LastReportReceivedAt no longer skips the check (plan R2-01 T8) 11efe4be05

docs(health): CentralHealthReportLoop comments say oldest-member, not cluster leader — stop re-teaching the S2 conflation (plan R2-01 T9) c2707fdcf8

refactor(host): CentralSingletonRegistrar -> SingletonRegistrar with optional role scope — unblocks site-singleton drains (plan R2-01 T10) 80e5b36852

fix(host): site singletons (deployment-manager, event-log-handler) via SingletonRegistrar — PhaseClusterLeave drains for site SQLite writes (plan R2-01 T11) 56d39b5fce

dohertj2 commented

2026-07-13 11:48:38 -04:00

Merged to main via the r2-integration fast-forward (main @ 1930f19b, pushed to origin). All commits from this branch are in main; the two integration-surfaced test fixes (IntegrationTests NodeName host-boot config, compile-cache assertion relax) landed on main as well. Closing — content fully integrated.

Merged to main via the r2-integration fast-forward (main @ 1930f19b, pushed to origin). All commits from this branch are in main; the two integration-surfaced test fixes (IntegrationTests NodeName host-boot config, compile-cache assertion relax) landed on main as well. Closing — content fully integrated.

dohertj2 closed this pull request

2026-07-13 11:48:39 -04:00

Pull request closed

This pull request cannot be reopened because the branch was deleted.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dohertj2/ScadaBridge#12