Files
ScadaBridge/archreview/plans/00-MASTER-TRACKER.md
T

209 lines
43 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Architecture Review Fix Plans — Master Tracker
**Created:** 2026-07-08
**Source:** [`archreview/00-OVERALL.md`](../00-OVERALL.md) and domain reports 0108
**Plans:** 8 implementation plans, **192 tasks** (191 net of one cross-plan duplicate — see Dedupe Rulings)
Every plan follows the TDD bite-sized-task format and ships a co-located `.tasks.json` manifest. Execute a plan with the `superpowers-extended-cc:executing-plans` skill (pass the plan path); it reads/updates the `.tasks.json`. **When a plan's status changes, update the table below** — this file is the single source of truth for initiative progress.
---
# ROUND 2 (2026-07-12)
**Source:** the round-2 re-review (all 8 `archreview/0X-*.md` reports rewritten 2026-07-12 at HEAD `8c888f13`): every round-1 finding source-verified (168 fixed, 0 regressions, 0 false claims), plus **56 new findings** (1 Critical / 4 High / 15 Medium / 36 Low) concentrated in post-baseline code. Round-2 plans: **8 plans, 86 tasks.**
## Round-2 Status Board
| Plan | Domain | Tasks | Done | Status | Findings coverage |
|------|--------|------:|-----:|--------|-------------------|
| [PLAN-R2-01](PLAN-R2-01-cluster-host-failover.md) | Cluster, Host & Failover | 11 | 0 | ⬜ Not started | N1→T1T4 (incl. live drill run + envelope measurement, covers R2-08's NF2); N2→T5T7 (`needs-user`-adjacent: deploy overlay edits, no git add); N3→T8; N4→T9; N5→T10/T11; N6→R2-08 |
| [PLAN-R2-02](PLAN-R2-02-communication-store-and-forward.md) | Communication & S&F + live alarm stream | 15 | 0 | ⬜ Not started | **N1 Critical→T1T4** (shared oldest-Up predicate, failing-first repro); **N2 High→T5T7** (chunked resync protocol); N3→T8; N4→T9; N5→T6; N6→T10 (MUTEX w/ R2-07); N7→T11/T12; N8→T13/T14; N9→T15 |
| [PLAN-R2-03](PLAN-R2-03-site-runtime-dcl.md) | Site Runtime & DCL | 7 | 0 | ⬜ Not started | N1→T1; N2→T2/T3; N3→T4; N4→T5/T6 (full compile-cache adoption); N5/N6→T7 |
| [PLAN-R2-04](PLAN-R2-04-data-audit-backbone.md) | Data & Audit Backbone + KPI rollups | 13 | 0 | ⬜ Not started | **R1 High→T2T4** (sliced backfill + watermark fast-path); R2→T1; R3→T5T7; R4→T8; R5→T9; R6→T10/T11 (1 EF migration, build-first gotcha noted); R7→T12; final verify T13 |
| [PLAN-R2-05](PLAN-R2-05-templates-deployment-transport.md) | Templates, Deployment & Transport | 9 | 0 | ⬜ Not started | N1→T2T4 (verdict-cache globals-surface keying); N2→T1; N3→T5 (publisher; subscriber = R2-06); N4→T7; N5→T6 (trust-gate extension, hard-error); N6→T8; docs T9 |
| [PLAN-R2-06](PLAN-R2-06-edge-integrations.md) | Edge Integrations | 6 | 0 | ⬜ Not started | N1→T1/T2 (**wires the change-bus subscriber**, closes round-1 U2 partial); N2→T3; N3→T5 (typed SITE_UNREACHABLE); N4→T4; N5→T6 |
| [PLAN-R2-07](PLAN-R2-07-ui-management-security.md) | UI, Management & Security | 12 | 0 | ⬜ Not started | N1→T1 (hub throttle); N2→T2/T3 (ForwardedHeaders); N3→T4T6 (secured-write site scope, ManagementActor lane); N4→T7; N5→T8; N6→T10 (MUTEX w/ R2-02); N7→T9; N8→T11; N9→T12 |
| [PLAN-R2-08](PLAN-R2-08-conventions-hygiene.md) | Conventions & Hygiene | 13 | 0 | ⬜ Not started | **NF1 High P0→T1T3** (`needs-user`: key rotation + secret-file deletion — ldap/sql login files hold LIVE secrets too); NF3→T4; NF4→T5T8 (T8 = possible live `IOperationTrackingStore` DI gap, verify-then-fix); NF5→T10/T11; NF6→T12; NF7→T13; NF8→T9 (Host duplicates are the LIVE bindings — canonicalize SCE sections first) |
**Round-2 progress: 0 of 86 tasks.**
## Round-2 P0 (do first, any order)
1. **R2-08 T1/T2** — rotate the exposed wonder-app-vd03 API key; delete `test.txt` + the three root credential files (ALL contain live secrets incl. a production sysadmin password). `needs-user`.
2. **R2-02 T1T4** — the Critical: unify the active-node predicate; failing-first buffer-wipe repro then fix.
3. **R2-02 T5T7** — the resync frame-size High (chunked protocol).
4. **R2-04 T2T4** — bound the rollup backfill before the next central failover at data volume.
## Round-2 Cross-Plan Mutexes & Dedupe Rulings
| Surface / finding | Ruling |
|---|---|
| `SiteAlarmLiveCacheService.cs` | R2-02 T10 (coalescing) and R2-07 T10 (aggregator deathwatch/sticky IsLive) must NOT run concurrently — serialize, either order. R2-02 T12/T15 also touch adjacent files; keep the whole live-alarm slice single-lane across plans. |
| `ManagementActor.cs` | Single-writer lane (round-1 rule stands): R2-07 T4/T6 and R2-02 T13 (`HandleDeleteSite`) serialize against each other. |
| `IScriptArtifactChangeBus` | Publisher Add-fix = R2-05 T5; subscriber wiring + Host comment = R2-06 T1/T2. Independent, no cross-plan blockedBy. |
| Failover envelope measurement (report-08 NF2) | Owned by R2-01 T4; R2-08 T10 only repoints the register row. |
| Deferral-register consolidation (reports 01 N6 / 08 NF5) | Owned by R2-08 T10/T11. |
| Options-validation tail | Split by owner: SiteRuntime knobs = R2-03 T4; Transport knob = R2-05 T8; S&F sweep knobs = R2-02 T9; AuditLog/Host = R2-08 T5T8. |
| SBR keep-oldest active-crash topology gap | Still a registered deferred USER decision — no round-2 plan fixes it; R2-01 documents/measures it, R2-08 T10 moves it into the canonical register. |
## Round-2 Recommended Waves
- **Wave 0 (P0):** R2-08 T1T3 (`needs-user`) ∥ R2-02 T1T7 ∥ R2-04 T1T4.
- **Wave 1:** rest of R2-02 ∥ R2-03 ∥ R2-05 ∥ R2-06 (all file-disjoint domains); R2-01 T1T4 (drill + measurement, needs live docker cluster).
- **Wave 2:** R2-07 (after R2-02's live-alarm slice lands, honoring the mutex) ∥ rest of R2-04 ∥ rest of R2-01 ∥ rest of R2-08.
- Full-suite sweep + docker redeploy at the end, per round-1 practice (watch for stale enum-count lock-in tests — bit PLAN-04 and PLAN-07).
---
# ROUND 1 (2026-07-08) — COMPLETE
## Status Board
| Plan | Domain | Tasks | Done | Status | Findings coverage |
|------|--------|------:|-----:|--------|-------------------|
| [PLAN-01](PLAN-01-cluster-host-failover.md) | Cluster, Host & Failover | 23 | 23 | ✅ Complete (all 23 tasks; merged to `main` `ec47cb56`) | 28 findings: 20 fixed, 3 → other plans, 3 accepted, 2 roadmap-deferred |
| [PLAN-02](PLAN-02-communication-store-and-forward.md) | Communication & S&F | 24 | 24 | ✅ Complete (all 24 tasks; merged to `main`) | 28 findings: 25 fixed, 1 → plan 01, 1 deferred (serializer swap), 1 won't-fix |
| [PLAN-03](PLAN-03-site-runtime-dcl.md) | Site Runtime & DCL | 26 | 26 | ✅ Complete (all 26 tasks; merged to `main` `c7a0c922`) | 28 actionable: 25 fixed, 3 deferred with rationale |
| [PLAN-04](PLAN-04-data-audit-backbone.md) | Data & Audit Backbone | 26 | 26 | ✅ Complete (all 26 tasks; merged to `main` `dbd3ed2f` via fast-forward; branch removed) | 27 items: 22 fixed, 4 deferred, 1 won't-fix (incl. NodeB reconcile from report 08) |
| [PLAN-05](PLAN-05-templates-deployment-transport.md) | Templates, Deployment & Transport | 25 | 25 | ✅ Complete (all 25 tasks; merged to `main` `ec23db35` via fast-forward; worktree removed, branch deleted) | 33 line-items: 27 fixed, 6 deferred with rationale |
| [PLAN-06](PLAN-06-edge-integrations.md) | Edge Integrations | 24 | 24 | ✅ Complete (all 24 tasks; merged to `main` `3e964ac2` via fast-forward; worktree removed, branch deleted) | 24 findings: 21 fixed, 3 deferred/won't-fix |
| [PLAN-07](PLAN-07-ui-management-security.md) | UI, Management & Security | 33 | 32 | ✅ Complete (all 32 active tasks; T33 `skipped-duplicate` — CLI.Tests already in slnx via PLAN-08 T1; merged to `main` via fast-forward; worktree removed, branch deleted) | 24 items: 20 fixed, 3 partial (deferrals documented), 1 won't-fix |
| [PLAN-08](PLAN-08-conventions-tests.md) | Conventions & Tests | 11 | 11 | ✅ Complete (all 11 tasks; T1 in P0, T2T11 in Wave 6; merged to `main` via fast-forward; worktrees removed, branches deleted) | 13 covered, 4 owned by other plans, ~20 triaged into deferred-work register |
Status values: ⬜ Not started · 🟨 In progress · ✅ Complete · ⏸ Blocked
**Progress:** P0 (Wave 0) complete — **10 of 192 tasks done** (2026-07-08): PLAN-01 T1/T3/T4, PLAN-02 T1/T2, PLAN-05 T1, PLAN-07 T1/T2/T3, PLAN-08 T1. Merged to `main` (`3f8e18c6`). One follow-up surfaced during execution — see the [active-node-crash SBR gap](#follow-up-discovered-during-p0-execution-2026-07-08) in the deferred registry.
**Wave 1 — PLAN-02 COMPLETE (2026-07-08):** all 24 tasks delivered and **merged to `main`** (`ce239f6f`). Store-and-Forward: active-node delivery gate (`Func<bool>` seam), bounded/short-circuiting retry sweep, upsert-based standby applies + anti-entropy peer resync, WAL+busy_timeout, epoch-ms due-check, ordered single-reader audit-observer channel; `Notify.Send` enqueue-only (`deferToSweep`, `maxRetries:0`, corrupt-payload parks not discards); gRPC channel keying + DropOldest debug-stream eviction counting + 5s application heartbeat; oldest-parked-age health field. Docker cluster rebuilt on merged image (Healthy, 5 singletons, corrected active-node semantics) and 3 sites seeded. Initiative total: **52 of 192** (sum of the Done column above).
**Wave 1 — PLAN-01 COMPLETE (2026-07-08):** all 23 tasks delivered on worktree branch `worktree-archreview-wave1-plan01` (**merged to `main` `ec47cb56` 2026-07-09; branch since removed**). Beyond the P0/early-Wave-1 spine (SBR downing + two-node rig T1/T3/T4; oldest-member active-node unification via `ClusterActivityEvaluator` T5T8; `CentralSingletonRegistrar` drains T2/T9/T10; health-report acking T11T12), Wave 1 also landed: metrics-staleness + status-transition state machine and its dashboard surface (T13/T14), deleted-site eviction from the aggregator (T18), seed-vs-metrics-port validator + `AllowSingleNodeCluster` + docker `stop_grace_period` (T15T17), rate-limited dead-letter warnings (T19), the down-if-alone process-exit recovery loop (T20), the `failover-drill.sh` SIGKILL drill (T21), and the REQ-HOST-6 + Traefik/Host doc reconciliation with unused-Akka.Hosting-package removal (T22/T23). PLAN-01 now **23 / 23**. Initiative total: **30 of 192**. Each task TDD + pathspec-committed; full solution builds clean (0 warnings). Three deploy-artifact sub-edits (`deploy/wonder-app-vd03/` appsettings + install.ps1, Tasks 16/20/23) were deferred because that production artifact is not tracked in this repo — logged in the deferred register below.
**Wave 1 — PLAN-03 COMPLETE (2026-07-09):** all 26 tasks delivered and **merged to `main`** (`c7a0c922`). Site Runtime & DCL hardening across nine TDD batches: reconnect/backoff leak fixes and the `DeployCompileValidator` compile gate, DCL adapter lifecycle + alarm-filter overwrite warning (S10) + observed adapter dispose on stop (S9), site-node cert reconcile (CertStoreActor trusted-store export → `DeploymentManagerActor` MemberUp-driven additive push), replicated config-fetch retry (`ConfigFetchRetryCount`), invariant-culture conditional-trigger fallback (C6), bounded/DropOldest event-channel comment fix (C2), and the SiteRuntime/DCL requirements-doc sync. Three low-severity items deferred with rationale (see PLAN-03 coverage table). Full CentralUI/SiteRuntime/DCL suites green. Initiative total: **78 of 192** (sum of the Done column above). Two out-of-band fixes also landed post-merge: PLAN-07's SecuredWrites `IBrowseService` bUnit-harness gap (resolved 2026-07-09, `9049690b`) — a pre-existing failure surfaced during PLAN-03's full-suite run, not a plan task.
**Wave 2 — PLAN-04 COMPLETE (2026-07-09):** all 26 tasks delivered on worktree branch `worktree-archreview-wave2-plan04` (Batches 13 fast-forwarded to `main`+`origin`; Batches 48 + T24 + the T26 fix still on the branch, pending `finishing-a-development-branch`). Site SQLite retention chain + partition-switch hardening + purger role-grants/index migrations (T1T9), backlog KPI + SiteCalls freshness upsert + EF `EnableRetryOnFailure` (T10T12), operator-identity audit on Retry/Discard relays + reconciliation composite-keyset cursor/abandoned-row audit/NodeB failover (T13T18), time-sliced KpiSample purge + single-query NotificationOutbox KPI + SiteCalls `OPTION (RECOMPILE)` (T19T21), short-first-tick purge timers + data-layer cleanups + bounded parallel notification delivery (T22, T23, T25), the design-doc reconciliation pass (T24). **T26 (final `dotnet build`/`dotnet test` sweep):** build 0 warn/0 err; all 30 unit + integration test assemblies green after fixing one stale enum-count lock-in test (`22d52390` — Task 17 added `AuditKind.ReconciliationAbandoned` but left `AuditEnumTests` asserting 14 members; the per-task filtered runs never hit that Commons guard, so the full sweep caught it). The Playwright E2E residue (89 fails) is environmental — a 17h-old docker cluster whose site containers fail `instance deploy` with `SQLite Error 14: unable to open database file`; the failure set is non-deterministic and PLAN-04 changed zero files in that project or the deployment-DB path. Initiative total: **104 of 192** (sum of the Done column).
**Wave 3 — PLAN-05 COMPLETE (2026-07-10):** all 25 tasks delivered on worktree branch `worktree-archreview-wave3-plan05` and **merged to `main` `ec23db35` via fast-forward** (main was still at PLAN-04's `dbd3ed2f`, zero divergence; worktree removed, branch deleted). Templates/Deployment/Transport hardening across seven report-and-wait waves: import correctness spine (BundleImporter consumes `BaseTemplateName`/composition/inheritance edges + import-time acyclicity, `MinTimeBetweenRuns`/`ExecutionTimeoutSeconds`/`LockedInDerived`/native-alarm-sources/real `AreaName` carried through — T1T7), a reflection round-trip equivalence guard that surfaced and fixed 5 silent-data-loss holes (ES retry+methods-on-Add, notification recipients, folder parent edges — T8), flattener/collision-detector repeated-composition fixes (T9/T10), revision-hash + DiffService native-source coverage (T11/T12), native-source lock enforcement (T13), the `IScriptArtifactChangeBus` invalidation contract + Host in-process bus + Transport post-commit publisher (T14, plan-06 handoff), compile-verdict cache + read-only-path compile skip + post-success audit isolation + central NotDeployed-delete (T15T18), and the closing security/quality chain: **import blocker severity split** (template-script findings advisory, ApiMethod hard — T19), **the script trust gate on bundle import** (5th `ScriptTrustValidator` call site, over script bodies *and* Expression-trigger bodies — T20, given a code-reviewer security pass that caught + fixed 2 Important gaps: expression-trigger coverage and per-origin local-declaration scoping), ScriptTrustPolicy deny-list hardening (T21), LineDiffer input cap (T22), ArtifactDiff placeholder-identity resolution (T23), the low-severity batch (session cap / full error lists / leaf-fallback warning — T24), and the design-doc + CLAUDE.md sweep (T25). **Terminal verification:** full slnx build 0 warn/0 err; the entire `dotnet test` sweep green (~5,900 unit + integration tests across every suite) plus a direct `CLI.Tests` run (341). Only residue: the same environmental 8 `CentralUI.PlaywrightTests` Deployment E2E failures (`SQLite Error 14` on `instance deploy` — needs a live docker cluster; PLAN-05 touched zero files in that project's deployment-DB path). Initiative total: **128 of 192** (sum of the Done column).
**Wave 4 — PLAN-06 COMPLETE (2026-07-10):** all 24 tasks delivered and **merged to `main` `3e964ac2` via fast-forward** (main was still at PLAN-05's `9b0cf44c`, zero divergence; worktree removed, branch deleted). Edge Integrations hardening across six report-and-wait waves of file-disjoint parallel subagents: the **InboundAPI** spine — revision-checked compiled-handler cache (heals failover / direct-SQL / known-bad staleness, DB row now authoritative — T1/T2), `InvalidateMethod` seam consuming PLAN-05's `IScriptArtifactChangeBus` contract (T3), deferred DI-scope disposal + orphan-execution telemetry (T4), machine-readable error `code`s incl. `SITE_UNREACHABLE` + 415-on-non-JSON (T5), `IOptionsMonitor` hot-reload (T6), parallel startup compile (T7); the **ESG** per-system `TimeoutSeconds` full vertical slice (entity→EF migration→artifact→site SQLite→client honor→mgmt→CLI→UI→doc — T8T12) plus `{param}` path-template substitution (T13), bounded outbound response buffering (T14), structured JSON `AuthConfiguration` with legacy-colon fallback (T15), cancellation-aware `IsTransient` (T16); the **Notification** batch — SMS first-transient short-circuit (T17), wired the dead `SmsConfiguration.MaxRetries/RetryDelay` per-type retry policy (T18), configurable OAuth2 authority/scope (entity→EF migration→token service→mgmt/CLI/UI surfaces — T19/T20), deterministic lowest-Id SMTP selection + atomic OAuth2 token cache (T21), Twilio `AccountSid` format validation (T22); and the docs (DelmiaNotifier at-least-once semantics — T23; dual-key API-key rotation procedure — T24). Two EF migrations added (`AddExternalSystemTimeoutSeconds` T8, `AddSmtpOAuth2AuthorityScope` T19, serialized on the shared snapshot). **Terminal verification:** full slnx build 0 warn/0 err; every unit + integration suite green (InboundAPI 253, ESG 132, NotificationOutbox 132, NotificationService 57, ManagementService 245, CLI 346, CentralUI 893, SiteRuntime 494, Commons 597, ConfigDB 338, Host 260, + all others). Only residue: the same environmental 8 `CentralUI.PlaywrightTests` Deployment E2E failures (`SQLite Error 14` on `instance deploy` — needs a live docker cluster; PLAN-06 touched zero files in that project's deployment-DB path). Execution note: `isolation:"worktree"` subagents proved non-isolating here (shared index → commit races); the working model was pre-created dedicated `git worktree`s per task + cherry-pick onto an integration branch + per-wave combined build/impacted-suite barrier. Initiative total: **152 of 192** (sum of the Done column).
**Wave 5 — PLAN-07 COMPLETE (2026-07-10):** all 32 active tasks (T4T32; T1T3 landed in P0; **T33 `skipped-duplicate`** — CLI.Tests already in the slnx via PLAN-08 T1) delivered on worktree branch `plan07` and **merged to `main` via fast-forward** (main was still at PLAN-06's `4fb754f9`, zero divergence; all worktrees removed, branch deleted). UI/Management/Security hardening across six report-and-wait waves of file-disjoint parallel subagents against the dominant **ManagementActor single-writer lane** (15 tasks serialized in one integration worktree; the file-disjoint remainder ran in dedicated per-task worktrees, cherry-picked in with a per-wave combined build + impacted-suite barrier): **C1** auth-disabled artifact killed + `DisableLoginGuard` fail-fast (T1T3, P0); **C2** `DeployToSiteAsync` single-site deploy + fleet-wide site-scope guard (T4/T5); **C3** connection/external-system/DB-connection secret elision via `ConfigSecretScrubber` + sentinel-preserving update (T6T9); **UA1/C6/C5** any-of role gates + `ListSecuredWrites` gating + area-gate reconciliation (Designer|Deployer) + role-casing canonicalization (T10T12); **UA2** the frozen `GetRequiredRoles` authorization matrix over all **138** registered commands + dispatch-coverage guard (T13/T24); **S2/P3** real server-side secured-write TTL + expiry-at-approve/reject + opportunistic list sweep + list paging + UI history pager & submission-age dialog (T14T18); **S1** 5-min transport/deploy Ask timeout + shipped config (T19/T20); **C4** additive `SiteIdentifier` on browse/verify/cert commands + actor dispatch (Designer browse/verify, Admin cert-trust) + dispatch-coverage + CLI parity (T21T25); **P1/UA5** `LoginThrottle` per-username+IP LDAP-bind lockout wired into every bind surface (T26/T27); **P2** additive Skip/Take paging on template/instance lists (T28); **S3** `PollGate` reentrancy guard on Health/AlarmSummary timers (T29); **S4** transport single-flight + export base64-copy elimination (T30); **S5/C7/C8/P4/UA6** debug-stream fault logging + event-log-filter doc fix + AlarmSummary memoization/a11y + the stale CLAUDE.md `SourceNode` note corrected (C7/C8 both verified true — T31/T32). Two message-contract additions (`AuditKind.SecuredWriteExpire`, secured-write list paging params) landed additively; no EF migrations needed. **Terminal verification:** full slnx build 0 warn/0 err; 28 of 30 test assemblies green (ManagementService 448 incl. the frozen matrix + dispatch-coverage, CentralUI 898, CLI 359, Security 175, Commons 609, Communication 263, Transport 138, ConfigDB 341, SiteRuntime 494, Host 260, + all others), after fixing one stale enum-count lock-in test (`AuditEnumTests` 15→16 for T15's additive `SecuredWriteExpire` — the per-task filtered runs never hit that Commons guard, so the full sweep caught it, exactly as in PLAN-04). Only residue: the same environmental 8 `CentralUI.PlaywrightTests` Deployment E2E failures (`SQLite Error 14` on `instance deploy` — needs a live docker cluster; PLAN-07 touched zero files in that project's deployment-DB path). Initiative total: **181 of 192** (sum of the Done column; PLAN-07's max is 32/33 with T33 a documented skip).
**Wave 6 — PLAN-08 COMPLETE (2026-07-10): INITIATIVE COMPLETE.** All 10 remaining tasks (T2T11; T1 landed in P0) delivered on integration branch `plan08` and **merged to `main` via fast-forward** (main was still at PLAN-07's `187fbd7c`, zero divergence; all worktrees removed, branches deleted). Conventions/tests/underdeveloped-areas hardening across **two** report-and-wait waves of file-disjoint parallel subagents — the initiative's single highest-parallelism slice (8 concurrent implementers in Wave 1): **§1.5 eager options validation** — new `OptionsValidatorBase<T>` validators wired via `.ValidateOnStart()` + `TryAddEnumerable` across **12 components** in four tasks: Communication + DataConnectionLayer (T2, + 3 extra TimeSpan fields beyond the plan; string-only global-options classes correctly skipped), the site pipeline SiteRuntime/StoreAndForward/SiteEventLog bound in Host (T3), edge/management InboundAPI/ESG/Notification/ManagementService (T4, correctly leaving `SecuredWritePendingTtl`≤0 and empty `ApiKeyPepper` valid as legitimate disable/dev defaults), central Transport/SiteCallAudit/DeploymentManager (T5, incl. the security-sensitive `Pbkdf2Iterations >= 100_000` floor; validating base not `Resolved*` properties); **§1.3/#23** excised the vestigial site-side notification-list surface — deleted `SiteNotificationRepository`, its DI registration, and the dead `StoreNotificationListAsync`/`StoreSmtpConfigurationAsync` write paths, keeping `PurgeCentralOnlyNotificationConfigAsync` as the security cleanup (T6, with a `Site_NotificationRepository_IsNotRegistered` design-invariant lock replacing the old registration test); **§2.3** the first real perf-envelope test — site-wide stream throughput (100k events, ≥10k/s + ≥90% delivery assertions — T7) plus a skipped failover-timing harness documenting the 25s-envelope protocol pending PLAN-01's two-node rig (T8); **§1.8** contract-lock tests for the 7 highest-traffic ClusterClient records enforcing additive-only evolution under version skew (T9, which surfaced a real wire-behavior finding: Newtonsoft ignores C# ctor optional-param defaults, so a missing trailing field deserializes to `default(T)` not the C# default — the test now locks that reality); **§1.1/§1.3/§1.4/§4** the docs-drift batch — Commons `Observability/` folder, the site-repo `NotSupportedException` exception, the `..\``../` csproj-separator fix, the scoped "24 of 27 reference docs" README claim, and the DelmiaNotifier convention-exception bullet in CLAUDE.md (T10); and **§3** the deferred-work register triaging the 23-item inventory into fix-now (plan-owned) vs deferred-with-rationale+revisit-trigger (T11). No EF migrations, no wire-contract changes, no new runtime components — all convention-hardening around existing seams. **Terminal verification:** full slnx build 0 warn/0 err; the entire `dotnet test` sweep green across all 30 test assemblies — now including **CLI.Tests (359) at the solution level** for the first time (T1's slnx addition) — except two documented non-PLAN-08 residues: (1) a **pre-existing** `Communication.Tests` gRPC observable-gauge timing flake (`SiteStreamGrpcServerTests.SiteConnectionUpGauge_*`, proven to flake 3/6 on base `main` `187fbd7c`; PLAN-08 touched zero `Grpc/` files), and (2) the same environmental 89 `CentralUI.PlaywrightTests` Deployment E2E failures (`SQLite Error 14` on `instance deploy` — needs a live docker cluster). Initiative total: **191 of 192** (sum of the Done column) — the single remaining gap is PLAN-07 T33, a documented `skipped-duplicate` (CLI.Tests slnx addition, delivered instead by PLAN-08 T1). **All 8 archreview plans complete.**
## Parallelization Map — Incomplete Tasks (2026-07-09)
Covers the plans with open tasks: **none — all 8 plans complete as of 2026-07-10 (Wave 6, PLAN-08).** PLAN-01/02/03/04/05/06/07/08 are complete; the section below is retained for its cross-plan mutex analysis and the record of how PLAN-07's ManagementActor lane and PLAN-08's highest-parallelism validator slice were run. **"Ready"** = every `blockedBy` entry is already `completed` in the plan's `.tasks.json`, so the task can start now. Ready sets are recomputed below from the live manifests; re-derive after each merge.
### Cross-plan serialization mutexes (a task holding one must be the ONLY task touching that surface, anywhere)
| Shared surface | Tasks that touch it (P0n = PLAN-0n) | Rule |
|---|---|---|
| `ManagementService/ManagementActor.cs` | P05 T13; P06 T2, T3, T11, T20, T22; P07 T5, T7, T8, T9, T10, T11, T12, T15, T16, T17, T22, T23, T28, T30, T31 | **Global single-writer.** At most one ManagementActor task in flight across ALL plans. PLAN-07 already serializes its 15 internally — extend the same mutex to P05 T13 and the five P06 tasks. |
| EF migration model-snapshot (`ConfigurationDatabase/Migrations/*` + `*ModelSnapshot`) | P06 T8 (ready), T19 | **Never two migration tasks concurrently.** Build first, never `--no-build` (empty-scaffold gotcha). Only P06 T8 is ready now. |
| `Host/Program.cs` | P06 T7; P08 T4 | Single-writer — don't run P06 T7 and P08 T4 at the same time. |
| `DeploymentManager/ArtifactDeploymentService.cs` | P06 T10; P07 T4 | Single-writer (P06 T10 not yet ready; P07 T4 is). |
| `SiteRuntime/Persistence/SiteStorageService.cs` | P06 T10; P08 T6 | Single-writer (neither ready yet). |
| `NotificationOutbox/NotificationOutboxActor.cs` | P06 T18, T21 | Intra-PLAN-06 (PLAN-04's delivery work is done and no longer touches it). |
| `CentralUI/.../Monitoring/AlarmSummary.razor` | P07 T29, T32 | Intra-PLAN-07 — serialize. |
| Docs (low-risk, coordinate — pathspec-commit + last-writer reconcile): `CLAUDE.md` (P05 T25, P06 T18, P07 T15/T31/T33, P08 T10), `Component-ManagementService.md`, `Component-InboundAPI.md`, `Component-ExternalSystemGateway.md`, `Component-NotificationService.md` | multiple | Not a hard blocker; do not co-edit the same file in the same wall-clock window. |
**Dedupe reminder:** **P07 T33 is a skip** — CLI.Tests is already in the slnx via P08 T1 (done). Mark it `skipped-duplicate` in `PLAN-07-…tasks.json`; do not execute it.
### Ready set + concurrent lanes per plan
- **PLAN-04** — ✅ COMPLETE (2026-07-09); all 26 tasks done, merged to `main` `dbd3ed2f`. No longer a parallelization lane.
- **PLAN-05** — ✅ COMPLETE (2026-07-10); all 25 tasks done, merged to `main` `ec23db35` (fast-forward). No longer a parallelization lane. (Historical: the BundleImporter serial chain T1→2→…→6→19→20→23 ran in-worktree; T14's `IScriptArtifactChangeBus` contract is shipped for PLAN-06 to consume.)
- **PLAN-06** — ✅ COMPLETE (2026-07-10); all 24 tasks done, merged to `main` `3e964ac2` (fast-forward). No longer a parallelization lane. (Historical: ran as 6 waves — W1{1,6,7,8,17,23} W2{2,9,10,12,18,19} W3{3,13,21} W4{4,11,14} W5{5,15,20} W6{16,22,24}; the ExternalSystemClient chain T9→13→14→15→16 and inbound-executor chain T1→3→4→5 were the two serial spines.)
- **PLAN-07** — ✅ COMPLETE (2026-07-10); all 32 active tasks done, merged to `main` (fast-forward); T33 `skipped-duplicate`. No longer a parallelization lane. (Historical: ran as 6 waves — W1{4,6,10,14,19,21,26,29} W2{7,8,9,27,32} W3{11,20,22,23} W4{13,15,16,17,24} W5{5,12,18,25,28} W6{30,31}; Lane M = the ManagementActor single-writer chain T10→7→8→9→11→22→23→15→16→17→5→12→28→30→31 driven serially in the integration worktree, with the file-disjoint remainder in dedicated per-task worktrees cherry-picked in per wave.)
- **PLAN-08** — ready: **{2, 3, 4, 5, 7, 8, 9, 11}** — each is a new per-component validator file or an isolated test/doc, so **mutually parallel**, with one caveat: **T4 shares `Host/Program.cs` with P06 T7** (sequence them). This is the initiative's single highest-parallelism slice (independent validator files). T6, T10 stay blocked.
### Recommended concurrent execution
Run up to **five plan-lane executor sessions concurrently** (one each for PLAN-04/05/06/07/08 — PLAN-01/02/03 are done), with these coordination rules layered on top:
1. **One ManagementActor task at a time, initiative-wide** — route P05 T13 and every ready P07 ManagementActor task ({8,9,10,12,28,30,31}) through a single serialized "Lane M"; start with **P07 T10**.
2. **One migration at a time** — only **P06 T8** is a ready migration; hold the snapshot until it lands.
3. **`Host/Program.cs` mutex** — **P06 T7 ⟂ P08 T4**.
4. Inside each session, dispatch that plan's ready, file-disjoint tasks as concurrent implementers per the lanes above; use **pathspec-scoped commits and verify each landed on HEAD** (known repo failure mode — see Execution hygiene).
5. Highest-value immediate fan-out: **PLAN-08 T2/T3/T5/T7/T8/T9/T11** (independent validator + test + doc files) and **PLAN-07's non-ManagementActor octet** can run as many concurrent implementers today; the throughput ceiling is Lane M (ManagementActor) and the single migration lane.
## P0 — Do These First (regardless of plan order)
**✅ All six delivered 2026-07-08** on branch `archreview-p0` (worktree). TDD throughout; full solution builds clean; touched suites green (Host, Communication 244, Security 163, Transport 127+63, IntegrationTests cluster rig). One finding surfaced — see the active-node-crash SBR gap in the deferred registry below.
1.**PLAN-01 Task 1** — enable the SBR downing provider (the system-wide Critical; one line + parsed-config test). *Provider verified active via the live two-node rig.*
2.**PLAN-01 Tasks 34** — the two-node kill-test rig (`TwoNodeClusterFixture`); integration-proves Task 1 and later PLAN-02 work. *Task 4 rewritten to the achievable keep-oldest guarantee (younger-crash → SBR downs it → oldest keeps singleton); oldest-crash gap registered as follow-up.*
3.**PLAN-07 Tasks 13**`DisableLoginGuard` + scrub `deploy/wonder-app-vd03` (Critical; unauthenticated prod surface). *Guard + Host wiring committed; `deploy/` is gitignored so its scrub is on-disk-only (not version-controlled), effective on next redeploy.*
4.**PLAN-02 Tasks 12** — ClusterClient generation-suffixed names (Critical crash-loop dropping routing to all sites).
5.**PLAN-05 Task 1** — bundle import consumes `BaseTemplateName` (Critical silent data loss).
6.**PLAN-08 Task 1** — CLI.Tests into the slnx (protects every subsequent plan's test runs). *341 CLI tests now run at the solution level.*
## Cross-Plan Dependency Map
**Hard-ish sequencing (execute earlier):**
- **PLAN-01 Tasks 57 → PLAN-02 Tasks 34.** PLAN-02's standby S&F delivery gate is a `Func<bool>` seam with marked swap points; it works standalone but its final wiring consumes PLAN-01's `ClusterActivityEvaluator` / `SelfIsPrimary` (singleton-host semantics).
- **PLAN-01 Tasks 34 → PLAN-02 integration proof.** The two-node rig is how PLAN-02's Critical fixes get integration-verified.
- **PLAN-04 Task 13 (additive `RequestedBy` message change) before any PLAN-07 sender refactors** touching the same messages.
**Soft seams (either order works, coordinate):**
- **PLAN-05 Task 14 (`IScriptArtifactChangeBus`) ↔ PLAN-06 Task 3 (`InboundScriptExecutor.InvalidateMethod`).** PLAN-06 Task 1's per-request revision check is the designed correctness fallback, so there is no hard sequencing; when both land, wire the bus to the invalidate seam.
- **PLAN-03's `DeployCompileValidator` should adopt PLAN-05's compile-cache abstraction** once it exists (later refactor, not a blocker).
- **PLAN-03 Task 8's stuck-thread watchdog pattern is reusable by PLAN-06 Task 4** (inbound orphan counter).
**EF Core migrations — serialize globally.** Migration-adding tasks exist in PLAN-04 (Tasks 6, 8), PLAN-06 (Tasks 8, 19), and possibly PLAN-07 (secured-write expiry). All share one model snapshot: **never run two migration tasks concurrently**, and always build first (never `--no-build` on `dotnet ef migrations add` — known empty-scaffold gotcha).
**Shared-file merge surfaces (don't run these plans' overlapping tasks concurrently):**
| File / area | Plans touching it |
|---|---|
| `ManagementActor.cs` | PLAN-07 (15 tasks, serialized within plan), PLAN-05 (management-path publish), PLAN-06 (method-update paths) |
| `deploy/wonder-app-vd03/appsettings.Central.json` | PLAN-07 (auth), PLAN-01 (Cluster/Node sections) |
| `NotificationOutboxActor.cs` | PLAN-04, PLAN-06 (SMS retry policy) |
| `StoreAndForwardService.cs` / site tracking store | PLAN-02 (primary owner), PLAN-04 Task 15 (keyset query) |
| `Host/Program.cs` + service registration | PLAN-01, PLAN-08 (options validation) |
| `SiteHealthReport` (Commons record, additive fields) | PLAN-03 Task 6, PLAN-04 health-event fields |
| Transport `BundleImporter.cs` | PLAN-05 (primary, serialized chain), PLAN-07 (import idempotency note) |
## Dedupe Rulings (one owner per finding)
| Finding | Owner | Others must skip |
|---|---|---|
| CLI.Tests into slnx | **PLAN-08 Task 1** (also retires CLAUDE.md gotcha + memory note) | PLAN-07 Task 33 — **skip as duplicate**; mark `skipped-duplicate` in its tasks.json |
| `DisableLogin: true` artifact | **PLAN-07 Tasks 13** | PLAN-01 references only |
| AuditLog reconciliation NodeB failover | **PLAN-04 Task 18** | PLAN-08 pointer only |
| `SmsConfiguration.MaxRetryCount` unused | **PLAN-06 Task 18** | PLAN-08 pointer only |
| Transport `AreaName: null` | **PLAN-05 Task 7** | PLAN-08 pointer only |
| Role-casing normalization | **PLAN-07 Task 10 chain** | PLAN-08 pointer only |
| Two-node failover rig | **PLAN-01 Tasks 34** | PLAN-08 Task 8 is a documented placeholder; PLAN-02 consumes |
| Script-artifact invalidation contract | **PLAN-05 Task 14** (design + publisher) | PLAN-06 implements the inbound consumer |
## Recommended Execution Waves
- **Wave 0 (P0 list above).** Independent files; all six items can run in parallel. ~1 focused session.
- **Wave 1 — stability spine.** Rest of PLAN-01 (active-node semantics, drains); PLAN-02 Tasks 39 (standby gate, gRPC channel keying, sweep bounding); PLAN-03 Tasks 14 (reconnect leak, compile gate); PLAN-04 Tasks 15 (purge chains); PLAN-05 import chain Tasks 26; PLAN-06 Tasks 14 + 8; PLAN-07 Tasks 49 (deploy scoping, secret elision).
- **Wave 2 — remainder.** All other tasks per intra-plan `blockedBy` edges; migration tasks serialized globally; docs-update tasks land with their code tasks (repo rule: doc + code + tests together).
- **Wave 3 — closeout.** PLAN-08 Task 11 (deferred-work register), each plan's doc-sweep task, then a full `dotnet build ZB.MOM.WW.ScadaBridge.slnx` + `dotnet test` (which now includes CLI.Tests) + `bash docker/deploy.sh` + the two-node drill from PLAN-01.
**Execution hygiene:** work in a dedicated worktree (not the primary checkout); one plan per executor session where possible; for parallel implementers use pathspec-scoped commits and verify every commit landed on HEAD afterward (both are known failure modes in this repo's history). Re-verify a finding's line numbers before editing — reports and plans cite the code as of 2026-07-08.
## Deferred / Won't-Fix Registry (initiative-level)
> 2026-07-12 (plan R2-08 T10): the two live items below (SBR oldest-crash gap, wonder-app-vd03 overlay) now have canonical rows in `docs/plans/2026-07-08-deferred-work-register.md` — that register is the single tracking place; the subsections below remain as the historical narrative.
Consolidated in **PLAN-08 Task 11**`docs/plans/2026-07-08-deferred-work-register.md` (triages the 23-item inventory from report 08 plus each plan's coverage-table deferrals: serializer swap, hash-chain/Parquet, aggregated live alarm stream, central cert-trust persistence, runtime script sandbox, clustered-GUID benchmark, TLS profile, Dockerfile HEALTHCHECK, etc.). Nothing from the review is silently dropped: every report finding is either a task, an ownership pointer, or a registered deferral with rationale.
### Follow-up discovered during P0 execution (2026-07-08)
- **Active(oldest)-node hard crash is NOT covered by two-node `keep-oldest` — total central outage until restart.** PLAN-01 Task 1's `downing-provider-class` fix is verified active (a hard-crashed node now transitions `Unreachable → Down → Removed`, impossible under the pre-fix NoDowning default). **But** while building PLAN-01 Task 4's behavioral proof, the live two-node rig showed (Akka 1.5.62) that `keep-oldest` downs the partition *without* the oldest: crashing the **younger** node is survivable (oldest keeps its singletons), but crashing the **oldest** node makes the younger survivor **down itself** (`Down` ~8s, terminated ~10s) → the whole central cluster is lost until the crashed node restarts. `down-if-alone = on` does not help on a hard crash (the alone-oldest is dead and cannot down itself). Because ScadaBridge treats **oldest = active node** (PLAN-01 Task 5 `ClusterActivityEvaluator`), this means a crash of the *active* central node does not fail over to the standby — the scenario operators care most about. Task 4's test was rewritten to prove the achievable guarantee (younger-crash → SBR downs it → oldest keeps singleton) and documents this asymmetry in its XML doc. **Remedy is a topology/strategy decision (deferred, owner: user):** e.g. `keep-majority` + a 3rd/lighthouse seed node, `static-quorum`, or an explicit accepted-risk note. Not fixed in P0 to avoid silently changing the production SBR strategy.
### Deferred during Wave 1 PLAN-01 execution (2026-07-08)
- **`deploy/wonder-app-vd03/` overlay edits not applied — that production deploy artifact is not tracked in this repo.** PLAN-01 Tasks 16, 20, and 23 each specified a small edit to the on-host deployment overlay (`appsettings.Central.json`: add `AllowSingleNodeCluster: true` + drop the phantom seed, add `NodeName: central-a`; `install.ps1`: add `sc.exe failure` service-recovery actions for the down-if-alone restart contract). Only the *deployment-record* doc (`docs/deployment/deployment-records/2026-06-27-wonder-app-vd03.md`) is version-controlled; the `deploy/wonder-app-vd03/` artifact directory itself is not present in the repo (production config kept out of source control). The **code/validator/doc** halves of all three tasks shipped and are what make the features usable (`AllowSingleNodeCluster` flag + validator, the `IHostApplicationLifetime` process-exit watchdog, the oldest-member Traefik/Host doc reconciliation + TLS roadmap note). **Owner: whoever maintains the on-host artifact** — apply the three overlay edits out-of-band on `wonder-app-vd03` (see the referenced task steps); without `NodeName` every audit row from that deployment stamps a NULL `SourceNode`, and without the service-recovery actions a self-downed node will not auto-restart.
### Plan deviations during Wave 1 PLAN-03 execution (2026-07-09)
- **Task 15 (S6) — deploy-success reporting redesigned: readiness handshake, not swallow-only.** The plan's Task 15 relied on the `Terminated` signal beating the persistence result, so a stale persistence `Success` could simply be swallowed once the init-failure path had already replied `Failed`. **In practice the store commits FIRST** (the optimistic `StoreDeployedConfigAsync` completes before the child's async `PreStart` throws), so the swallow-only guard still reported `Success` for an actor that then died — the exact S6 bug. Fix as shipped makes the join deterministic: `InstanceActor.PreStart` now sends a new `InstanceActorInitialized` readiness message to the DeploymentManager as its final init step, and deployment `Success` requires **both** the persistence commit **and** that signal (`InFlightDeploy.Persisted && Initialized`); an actor that dies during init never signals readiness, so the `Terminated` fallback fails the deployment and rolls back the optimistic state. A second, related bug was fixed at the same time: the optimistic `Store…` and the rollback `Remove…` were independent `Task.Run`s with no ordering guarantee, so the persisted-row rollback is now **deferred until the store commits** (`_initFailedPendingRowRemoval`) and can never race ahead of it. New public surface: `Messages/InstanceActorInitialized.cs` (additive). **Follow-up owner: PLAN-03 Task 26** — the design-doc updates must describe the two-signal deploy-success join + the readiness message (alongside Task 4's already-flagged synchronous-compile-gate deviation). No further code work outstanding; this is a doc-capture follow-up.
### Pre-existing failure discovered during Wave 1 PLAN-03 execution — owner PLAN-07 (2026-07-09)
- **✅ RESOLVED 2026-07-09 — 3 `CentralUI.Tests / SecuredWritesTests` render tests fixed by registering a stub `IBrowseService` in the bUnit harness; suite now 7/7 green. Original report retained below for history.** ~~`IBrowseService` not registered in the page's bUnit harness (PLAN-07 / Secured Writes, NOT PLAN-03).~~ PLAN-03's final full-solution `dotnet test` surfaced 3 failures in `SecuredWritesTests` (`Operator_SeesSubmitForm`, `Operator_Submit_CallsServiceWithEnteredValues`, `Verifier_OwnSubmission_HasApproveAndRejectDisabled`), all `InvalidOperationException: Cannot provide a value for property 'BrowseService' on type 'NodeBrowserDialog' — no registered service of type 'IBrowseService'`. Root cause: the Secured Writes page (`Components/Pages/Operations/SecuredWrites.razor`) renders `Components/Dialogs/NodeBrowserDialog.razor`, which `@inject`s `IBrowseService`; the test fixture (`SecuredWritesTests`) registers `_service`/`_siteRepo`/`_dialog`/auth but **not** a fake `IBrowseService`, so rendering the child dialog throws. **Confirmed pre-existing and out of PLAN-03 scope** — the failing files were last touched by commits outside the PLAN-03 session (`1a7e7351` page/test, `9cff87fe` dialog), and PLAN-03 never touches Secured Writes / CentralUI. Merged to `main` (`c7a0c922`) with this red present because it is not a regression from this branch. **Owner: PLAN-07** — register a stub `IBrowseService` in the `SecuredWritesTests` `Services` collection before `RenderPage()` (or in the shared bUnit fixture). See PLAN-07 "Post-plan known issues".