dohertj2/ScadaBridge
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(archreview): PLAN-01 complete (23/23) — sync tracker + tasks.json; log deploy-artifact deferral

Browse Source
This commit is contained in:
Joseph Doherty
2026-07-08 17:12:46 -04:00
parent dc7a613dae
commit ec47cb5612
2 changed files with 22 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
archreview/plans/00-MASTER-TRACKER.md
+6 -2
View File
@@ -10,7 +10,7 @@ Every plan follows the TDD bite-sized-task format and ships a co-located `.tasks
| Plan | Domain | Tasks | Done | Status | Findings coverage |
|------|--------|------:|-----:|--------|-------------------|
| [PLAN-01](PLAN-01-cluster-host-failover.md) | Cluster, Host & Failover | 23 | 12 | 🟨 In progress (P0: T1,T3,T4; Wave 1: T2,T5–T12) | 28 findings: 20 fixed, 3 → other plans, 3 accepted, 2 roadmap-deferred |
| [PLAN-01](PLAN-01-cluster-host-failover.md) | Cluster, Host & Failover | 23 | 23 | ✅ Complete (all 23 tasks; branch not yet merged) | 28 findings: 20 fixed, 3 → other plans, 3 accepted, 2 roadmap-deferred |
| [PLAN-02](PLAN-02-communication-store-and-forward.md) | Communication & S&F | 24 | 2 | 🟨 In progress (P0: T1, T2) | 28 findings: 25 fixed, 1 → plan 01, 1 deferred (serializer swap), 1 won't-fix |
| [PLAN-03](PLAN-03-site-runtime-dcl.md) | Site Runtime & DCL | 26 | 0 | ⬜ Not started | 28 actionable: 25 fixed, 3 deferred with rationale |
| [PLAN-04](PLAN-04-data-audit-backbone.md) | Data & Audit Backbone | 26 | 0 | ⬜ Not started | 27 items: 22 fixed, 4 deferred, 1 won't-fix (incl. NodeB reconcile from report 08) |
@@ -23,7 +23,7 @@ Status values: ⬜ Not started · 🟨 In progress · ✅ Complete · ⏸ Blocke
**Progress:** P0 (Wave 0) complete — **10 of 192 tasks done** (2026-07-08): PLAN-01 T1/T3/T4, PLAN-02 T1/T2, PLAN-05 T1, PLAN-07 T1/T2/T3, PLAN-08 T1. Merged to `main` (`3f8e18c6`). One follow-up surfaced during execution — see the [active-node-crash SBR gap](#follow-up-discovered-during-p0-execution-2026-07-08) in the deferred registry.
**Wave 1 in progress — PLAN-01 (2026-07-08):** T2, T5–T12 delivered on worktree branch `worktree-archreview-wave1-plan01` (NOT yet merged/pushed). Active-node semantics unified on oldest-member (`ClusterActivityEvaluator`; T5–T8, incl. partition-safe `/health/active` + graceful-leave divergence proof), all seven central singletons routed through `CentralSingletonRegistrar` with drains (T2, T9, T10), and end-to-end health-report acking (`SiteHealthReportAck`; T11–T12). PLAN-01 now **12 / 23**. Remaining PLAN-01: T13–T23. Initiative total: **19 of 192**. Each task TDD + pathspec-committed; full solution builds clean.
**Wave 1 — PLAN-01 COMPLETE (2026-07-08):** all 23 tasks delivered on worktree branch `worktree-archreview-wave1-plan01` (**NOT yet merged/pushed**). Beyond the P0/early-Wave-1 spine (SBR downing + two-node rig T1/T3/T4; oldest-member active-node unification via `ClusterActivityEvaluator` T5–T8; `CentralSingletonRegistrar` drains T2/T9/T10; health-report acking T11–T12), Wave 1 also landed: metrics-staleness + status-transition state machine and its dashboard surface (T13/T14), deleted-site eviction from the aggregator (T18), seed-vs-metrics-port validator + `AllowSingleNodeCluster` + docker `stop_grace_period` (T15–T17), rate-limited dead-letter warnings (T19), the down-if-alone process-exit recovery loop (T20), the `failover-drill.sh` SIGKILL drill (T21), and the REQ-HOST-6 + Traefik/Host doc reconciliation with unused-Akka.Hosting-package removal (T22/T23). PLAN-01 now **23 / 23**. Initiative total: **30 of 192**. Each task TDD + pathspec-committed; full solution builds clean (0 warnings). Three deploy-artifact sub-edits (`deploy/wonder-app-vd03/` appsettings + install.ps1, Tasks 16/20/23) were deferred because that production artifact is not tracked in this repo — logged in the deferred register below.
## P0 — Do These First (regardless of plan order)
@@ -90,3 +90,7 @@ Consolidated in **PLAN-08 Task 11** → `docs/plans/2026-07-08-deferred-work-reg
### Follow-up discovered during P0 execution (2026-07-08)
- **Active(oldest)-node hard crash is NOT covered by two-node `keep-oldest` — total central outage until restart.** PLAN-01 Task 1's `downing-provider-class` fix is verified active (a hard-crashed node now transitions `Unreachable → Down → Removed`, impossible under the pre-fix NoDowning default). **But** while building PLAN-01 Task 4's behavioral proof, the live two-node rig showed (Akka 1.5.62) that `keep-oldest` downs the partition *without* the oldest: crashing the **younger** node is survivable (oldest keeps its singletons), but crashing the **oldest** node makes the younger survivor **down itself** (`Down` ~8s, terminated ~10s) → the whole central cluster is lost until the crashed node restarts. `down-if-alone = on` does not help on a hard crash (the alone-oldest is dead and cannot down itself). Because ScadaBridge treats **oldest = active node** (PLAN-01 Task 5 `ClusterActivityEvaluator`), this means a crash of the *active* central node does not fail over to the standby — the scenario operators care most about. Task 4's test was rewritten to prove the achievable guarantee (younger-crash → SBR downs it → oldest keeps singleton) and documents this asymmetry in its XML doc. **Remedy is a topology/strategy decision (deferred, owner: user):** e.g. `keep-majority` + a 3rd/lighthouse seed node, `static-quorum`, or an explicit accepted-risk note. Not fixed in P0 to avoid silently changing the production SBR strategy.
### Deferred during Wave 1 PLAN-01 execution (2026-07-08)
- **`deploy/wonder-app-vd03/` overlay edits not applied — that production deploy artifact is not tracked in this repo.** PLAN-01 Tasks 16, 20, and 23 each specified a small edit to the on-host deployment overlay (`appsettings.Central.json`: add `AllowSingleNodeCluster: true` + drop the phantom seed, add `NodeName: central-a`; `install.ps1`: add `sc.exe failure` service-recovery actions for the down-if-alone restart contract). Only the *deployment-record* doc (`docs/deployment/deployment-records/2026-06-27-wonder-app-vd03.md`) is version-controlled; the `deploy/wonder-app-vd03/` artifact directory itself is not present in the repo (production config kept out of source control). The **code/validator/doc** halves of all three tasks shipped and are what make the features usable (`AllowSingleNodeCluster` flag + validator, the `IHostApplicationLifetime` process-exit watchdog, the oldest-member Traefik/Host doc reconciliation + TLS roadmap note). **Owner: whoever maintains the on-host artifact** — apply the three overlay edits out-of-band on `wonder-app-vd03` (see the referenced task steps); without `NodeName` every audit row from that deployment stamps a NULL `SourceNode`, and without the service-recovery actions a self-downed node will not auto-restart.