docs(archreview): PLAN-01 complete (23/23) — sync tracker + tasks.json; log deploy-artifact deferral
This commit is contained in:
Joseph Doherty
@@ -10,7 +10,7 @@ Every plan follows the TDD bite-sized-task format and ships a co-located `.tasks
|
||||
|
||||
| Plan | Domain | Tasks | Done | Status | Findings coverage |
|
||||
|------|--------|------:|-----:|--------|-------------------|
|
||||
| [PLAN-01](PLAN-01-cluster-host-failover.md) | Cluster, Host & Failover | 23 | 12 | 🟨 In progress (P0: T1,T3,T4; Wave 1: T2,T5–T12) | 28 findings: 20 fixed, 3 → other plans, 3 accepted, 2 roadmap-deferred |
|
||||
| [PLAN-01](PLAN-01-cluster-host-failover.md) | Cluster, Host & Failover | 23 | 23 | ✅ Complete (all 23 tasks; branch not yet merged) | 28 findings: 20 fixed, 3 → other plans, 3 accepted, 2 roadmap-deferred |
|
||||
| [PLAN-02](PLAN-02-communication-store-and-forward.md) | Communication & S&F | 24 | 2 | 🟨 In progress (P0: T1, T2) | 28 findings: 25 fixed, 1 → plan 01, 1 deferred (serializer swap), 1 won't-fix |
|
||||
| [PLAN-03](PLAN-03-site-runtime-dcl.md) | Site Runtime & DCL | 26 | 0 | ⬜ Not started | 28 actionable: 25 fixed, 3 deferred with rationale |
|
||||
| [PLAN-04](PLAN-04-data-audit-backbone.md) | Data & Audit Backbone | 26 | 0 | ⬜ Not started | 27 items: 22 fixed, 4 deferred, 1 won't-fix (incl. NodeB reconcile from report 08) |
|
||||
@@ -23,7 +23,7 @@ Status values: ⬜ Not started · 🟨 In progress · ✅ Complete · ⏸ Blocke
|
||||
|
||||
**Progress:** P0 (Wave 0) complete — **10 of 192 tasks done** (2026-07-08): PLAN-01 T1/T3/T4, PLAN-02 T1/T2, PLAN-05 T1, PLAN-07 T1/T2/T3, PLAN-08 T1. Merged to `main` (`3f8e18c6`). One follow-up surfaced during execution — see the [active-node-crash SBR gap](#follow-up-discovered-during-p0-execution-2026-07-08) in the deferred registry.
|
||||
|
||||
**Wave 1 in progress — PLAN-01 (2026-07-08):** T2, T5–T12 delivered on worktree branch `worktree-archreview-wave1-plan01` (NOT yet merged/pushed). Active-node semantics unified on oldest-member (`ClusterActivityEvaluator`; T5–T8, incl. partition-safe `/health/active` + graceful-leave divergence proof), all seven central singletons routed through `CentralSingletonRegistrar` with drains (T2, T9, T10), and end-to-end health-report acking (`SiteHealthReportAck`; T11–T12). PLAN-01 now **12 / 23**. Remaining PLAN-01: T13–T23. Initiative total: **19 of 192**. Each task TDD + pathspec-committed; full solution builds clean.
|
||||
**Wave 1 — PLAN-01 COMPLETE (2026-07-08):** all 23 tasks delivered on worktree branch `worktree-archreview-wave1-plan01` (**NOT yet merged/pushed**). Beyond the P0/early-Wave-1 spine (SBR downing + two-node rig T1/T3/T4; oldest-member active-node unification via `ClusterActivityEvaluator` T5–T8; `CentralSingletonRegistrar` drains T2/T9/T10; health-report acking T11–T12), Wave 1 also landed: metrics-staleness + status-transition state machine and its dashboard surface (T13/T14), deleted-site eviction from the aggregator (T18), seed-vs-metrics-port validator + `AllowSingleNodeCluster` + docker `stop_grace_period` (T15–T17), rate-limited dead-letter warnings (T19), the down-if-alone process-exit recovery loop (T20), the `failover-drill.sh` SIGKILL drill (T21), and the REQ-HOST-6 + Traefik/Host doc reconciliation with unused-Akka.Hosting-package removal (T22/T23). PLAN-01 now **23 / 23**. Initiative total: **30 of 192**. Each task TDD + pathspec-committed; full solution builds clean (0 warnings). Three deploy-artifact sub-edits (`deploy/wonder-app-vd03/` appsettings + install.ps1, Tasks 16/20/23) were deferred because that production artifact is not tracked in this repo — logged in the deferred register below.
|
||||
|
||||
## P0 — Do These First (regardless of plan order)
|
||||
|
||||
@@ -90,3 +90,7 @@ Consolidated in **PLAN-08 Task 11** → `docs/plans/2026-07-08-deferred-work-reg
|
||||
### Follow-up discovered during P0 execution (2026-07-08)
|
||||
|
||||
- **Active(oldest)-node hard crash is NOT covered by two-node `keep-oldest` — total central outage until restart.** PLAN-01 Task 1's `downing-provider-class` fix is verified active (a hard-crashed node now transitions `Unreachable → Down → Removed`, impossible under the pre-fix NoDowning default). **But** while building PLAN-01 Task 4's behavioral proof, the live two-node rig showed (Akka 1.5.62) that `keep-oldest` downs the partition *without* the oldest: crashing the **younger** node is survivable (oldest keeps its singletons), but crashing the **oldest** node makes the younger survivor **down itself** (`Down` ~8s, terminated ~10s) → the whole central cluster is lost until the crashed node restarts. `down-if-alone = on` does not help on a hard crash (the alone-oldest is dead and cannot down itself). Because ScadaBridge treats **oldest = active node** (PLAN-01 Task 5 `ClusterActivityEvaluator`), this means a crash of the *active* central node does not fail over to the standby — the scenario operators care most about. Task 4's test was rewritten to prove the achievable guarantee (younger-crash → SBR downs it → oldest keeps singleton) and documents this asymmetry in its XML doc. **Remedy is a topology/strategy decision (deferred, owner: user):** e.g. `keep-majority` + a 3rd/lighthouse seed node, `static-quorum`, or an explicit accepted-risk note. Not fixed in P0 to avoid silently changing the production SBR strategy.
|
||||
|
||||
### Deferred during Wave 1 PLAN-01 execution (2026-07-08)
|
||||
|
||||
- **`deploy/wonder-app-vd03/` overlay edits not applied — that production deploy artifact is not tracked in this repo.** PLAN-01 Tasks 16, 20, and 23 each specified a small edit to the on-host deployment overlay (`appsettings.Central.json`: add `AllowSingleNodeCluster: true` + drop the phantom seed, add `NodeName: central-a`; `install.ps1`: add `sc.exe failure` service-recovery actions for the down-if-alone restart contract). Only the *deployment-record* doc (`docs/deployment/deployment-records/2026-06-27-wonder-app-vd03.md`) is version-controlled; the `deploy/wonder-app-vd03/` artifact directory itself is not present in the repo (production config kept out of source control). The **code/validator/doc** halves of all three tasks shipped and are what make the features usable (`AllowSingleNodeCluster` flag + validator, the `IHostApplicationLifetime` process-exit watchdog, the oldest-member Traefik/Host doc reconciliation + TLS roadmap note). **Owner: whoever maintains the on-host artifact** — apply the three overlay edits out-of-band on `wonder-app-vd03` (see the referenced task steps); without `NodeName` every audit row from that deployment stamps a NULL `SourceNode`, and without the service-recovery actions a self-downed node will not auto-restart.
|
||||
|
||||
@@ -25,7 +25,7 @@
|
||||
},
|
||||
{
|
||||
"id": 4,
|
||||
"subject": "Task 4: SBR behavioral proof — DELIVERED as younger-node-crash (SBR downs it, oldest keeps singleton); the planned oldest-crash 'singleton migration' is impossible under 2-node keep-oldest — see master-tracker follow-up",
|
||||
"subject": "Task 4: SBR behavioral proof \u2014 DELIVERED as younger-node-crash (SBR downs it, oldest keeps singleton); the planned oldest-crash 'singleton migration' is impossible under 2-node keep-oldest \u2014 see master-tracker follow-up",
|
||||
"status": "completed",
|
||||
"blockedBy": [
|
||||
3
|
||||
@@ -33,7 +33,7 @@
|
||||
},
|
||||
{
|
||||
"id": 5,
|
||||
"subject": "Task 5: ClusterActivityEvaluator — single oldest-member \"active node\" definition",
|
||||
"subject": "Task 5: ClusterActivityEvaluator \u2014 single oldest-member \"active node\" definition",
|
||||
"status": "completed",
|
||||
"blockedBy": []
|
||||
},
|
||||
@@ -55,7 +55,7 @@
|
||||
},
|
||||
{
|
||||
"id": 8,
|
||||
"subject": "Task 8: Divergence proof — restarted first node must NOT become active again",
|
||||
"subject": "Task 8: Divergence proof \u2014 restarted first node must NOT become active again",
|
||||
"status": "completed",
|
||||
"blockedBy": [
|
||||
3,
|
||||
@@ -95,13 +95,13 @@
|
||||
{
|
||||
"id": 13,
|
||||
"subject": "Task 13: Metrics-staleness signal + status-transition timestamps in the aggregator",
|
||||
"status": "pending",
|
||||
"status": "completed",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
"id": 14,
|
||||
"subject": "Task 14: Surface metrics-stale + offline-since on the Health dashboard",
|
||||
"status": "pending",
|
||||
"status": "completed",
|
||||
"blockedBy": [
|
||||
13
|
||||
]
|
||||
@@ -109,25 +109,25 @@
|
||||
{
|
||||
"id": 15,
|
||||
"subject": "Task 15: Fix the metrics-port seed in appsettings.Site.json + close the validator gap",
|
||||
"status": "pending",
|
||||
"status": "completed",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
"id": 16,
|
||||
"subject": "Task 16: Single-node installs — explicit AllowSingleNodeCluster instead of phantom seeds",
|
||||
"status": "pending",
|
||||
"subject": "Task 16: Single-node installs \u2014 explicit AllowSingleNodeCluster instead of phantom seeds",
|
||||
"status": "completed",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
"id": 17,
|
||||
"subject": "Task 17: stop_grace_period for all app containers (both topologies)",
|
||||
"status": "pending",
|
||||
"status": "completed",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
"id": 18,
|
||||
"subject": "Task 18: Evict deleted sites from the health aggregator",
|
||||
"status": "pending",
|
||||
"status": "completed",
|
||||
"blockedBy": [
|
||||
13
|
||||
]
|
||||
@@ -135,13 +135,13 @@
|
||||
{
|
||||
"id": 19,
|
||||
"subject": "Task 19: Rate-limit dead-letter warnings",
|
||||
"status": "pending",
|
||||
"status": "completed",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
"id": 20,
|
||||
"subject": "Task 20: Down-if-alone recovery — exit on unexpected ActorSystem termination + service restart actions",
|
||||
"status": "pending",
|
||||
"subject": "Task 20: Down-if-alone recovery \u2014 exit on unexpected ActorSystem termination + service restart actions",
|
||||
"status": "completed",
|
||||
"blockedBy": [
|
||||
10
|
||||
]
|
||||
@@ -149,7 +149,7 @@
|
||||
{
|
||||
"id": 21,
|
||||
"subject": "Task 21: Docker failover drill script (manual/CI kill test against the real topology)",
|
||||
"status": "pending",
|
||||
"status": "completed",
|
||||
"blockedBy": [
|
||||
1,
|
||||
17
|
||||
@@ -158,7 +158,7 @@
|
||||
{
|
||||
"id": 22,
|
||||
"subject": "Task 22: Reconcile REQ-HOST-6 (Akka.Hosting) with the hand-rolled bootstrap",
|
||||
"status": "pending",
|
||||
"status": "completed",
|
||||
"blockedBy": [
|
||||
1
|
||||
]
|
||||
@@ -166,7 +166,7 @@
|
||||
{
|
||||
"id": 23,
|
||||
"subject": "Task 23: Docs & deploy-config cleanup batch (Traefik doc, NodeName overlays, TLS roadmap note)",
|
||||
"status": "pending",
|
||||
"status": "completed",
|
||||
"blockedBy": [
|
||||
7,
|
||||
16
|
||||
|
||||
Reference in New Issue
Block a user