docs: truth sweep — retire stale registers, reconcile ledgers with shipped state

- deferred.md: DELETED (git rm) — stale 2026-07-10 duplicate of the canonical
  deferred-work register; this completes archreview R2-08 T11 (the file was
  tracked, not untracked as the task assumed)
- ScadaBridge-docs-issues.md, ScadaBridge-docs-fixed.md: DELETED (git rm) —
  generated 2026-07-10 CommentChecker reports, already consumed; completes
  R2-08 T13 (also tracked, not untracked)
- stillpending.md: prepended historical-snapshot banner (2026-06-15 audit;
  Tier-1 table is not current open work)
- docs/plans/phase-8-checklist.md: replaced the unevidenced 'Complete / All
  passing' stub with the honest state per register row 25 (WP-4 target-scale
  load test never run)
- archreview/plans/00-MASTER-TRACKER.md: R2-01 T2 live failover drill
  annotated RESOLVED 2026-08-01 (PLAN-R2-01 T4 + docker/failover-drill.sh +
  SbrFailoverTests); R2-08 T11/T13 recorded completed by this sweep
- docs/plans/2026-07-22-clusterclient-to-grpc-plan.md: P3 deferred-RPCs note
  updated (all four live-proven 2026-08-01, 1c99d6fa); ClusterClientSiteAuditClient
  naming follow-up marked DONE (63c16d69)
- docs/plans/2026-05-28-opcua-tag-browser.md.tasks.json: Task 19 flipped to
  completed (manual smoke PASS 2026-08-01, 6dc5d94c)
- archreview/plans/PLAN-R2-0[1-8]*.tasks.json: all-pending manifests reconciled
  with the authoritative tracker (round 2 merged @ 1930f19b) — flipped to
  completed except R2-08 T1/T2 which remain pending needs-user
- docs/operations/2026-07-16-secrets-clustered-master-key.md: correction banner
  (SQL-hub replication shipped 8e12f994; KEK-rotation + clustered-secrets
  runbooks ship with ZB.MOM.WW.Secrets)
- docs/plans/2026-07-19-localdb-phase2-live-gate.md: external-system-delete
  observation annotated RESOLVED (2d03f2d5 reconciles deletions incl.
  external_systems)

Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
This commit is contained in:
Joseph Doherty
2026-08-07 01:56:33 -04:00
parent 7caa8bfd99
commit 4df3a55824
18 changed files with 151 additions and 4553 deletions
@@ -19,7 +19,7 @@
{"id": 85, "subject": "Task 16: Tree rendering + lazy load + selection in dialog", "status": "completed", "blockedBy": [83, 84]},
{"id": 86, "subject": "Task 17: Error banner mapping + polish on dialog", "status": "completed", "blockedBy": [85]},
{"id": 87, "subject": "Task 18: Add Override column + Browse button to InstanceConfigure.razor", "status": "completed", "blockedBy": [83, 86]},
{"id": 88, "subject": "Task 19: End-to-end manual smoke (online + offline)", "status": "pending", "blockedBy": [70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87], "notes": "needs live rig — tracked in the 2026-08-01 pending-work task list"},
{"id": 88, "subject": "Task 19: End-to-end manual smoke (online + offline)", "status": "completed", "blockedBy": [70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87], "notes": "Completed 2026-08-01: manual smoke PASS, online + offline paths verified live (commit 6dc5d94c). Status flipped from stale 'pending' by the 2026-08-07 truth sweep."},
{"id": 89, "subject": "Task 20: Update Component-DataConnectionLayer.md", "status": "completed", "blockedBy": [81]},
{"id": 90, "subject": "Task 21: Update Component-TemplateEngine.md", "status": "completed", "blockedBy": [81]},
{"id": 91, "subject": "Task 22: Update Component-CentralUI.md", "status": "completed", "blockedBy": [87]}
@@ -192,6 +192,14 @@ was down and back to 0 after rejoin.
`GateDeadTarget` centrally and re-running `deploy artifacts`, both site nodes still carry it.
Artifact application is an upsert with no reconciliation of removals. Pre-existing behaviour in
the artifact pipeline, unrelated to LocalDb — but it means site config tables accumulate orphans.
**Update 2026-08-07 (truth sweep): RESOLVED** by `2d03f2d5` ("fix(site-runtime): reconcile
artifact deletions on apply", 2026-08-01) — the artifact apply is now upsert-then-reconcile:
after storing the incoming complete set, `SiteStorageService.DeleteRowsExceptAsync` removes any
stored row absent from it, per artifact table, explicitly including **external systems**
(`DeleteExternalSystemsExceptAsync` on `external_systems`) plus shared scripts, database
connections and data-connection definitions; a null (not-shipped) list still touches nothing,
and runtime cleanup rides along (shared-script unregistration, DCL eviction of the removed
connection).
- **Deleting an instance orphans its buffered messages.** Removing the `soakgen-*` instances left
their 11,804 `sf_messages` with no tracking snapshot, producing a continuous
`Cached-telemetry drain: no tracking snapshot for …` warning flood. Also pre-existing, and worth
@@ -324,11 +324,11 @@ Critical path ≈ 1B: **~46 weeks total**, matching the design estimate.
**Phase 2 ∥ 3 — cutover + soak**
- [x] P2 All sites `CentralTransport=Grpc`; central-kill S&F soak (no loss/dupes), failback observed, health sequences clean — **PASS 2026-07-23** (live gate: single-node active-kill failover 29s + full-outage ~59s both-down drain; every 5s bucket = exactly 3 through both outages, 216 total == 216 distinct; 0 auth failures / 0 seq regressions; whole control plane — heartbeat/health/notification/audit — on gRPC `CentralControlService`)
- [x] P3 Central `SiteTransport=Grpc` all sites; full UI command matrix per site; site-kill mid-command clean; no PSK noise; zero ClusterClient log activity on flipped paths — **PASS 2026-07-23** (live gate: `ExecuteQuery`/`ExecuteParked` all 3 sites + `ExecuteLifecycle` disable/enable on site-a, all 200 over `SiteCommandService`; active-site-node hard-kill mid-command → clean `TIMEOUT` at the 30s deadline, next call failed over to site-a-b automatically; 0 PermissionDenied / 0 ClusterClient-to-site on both central. `ExecuteOpcUa`/`ExecuteRoute`/`TriggerFailover` deferred — no OPC-bound instance / no CLI verb, unit-proven)
- [x] P3 Central `SiteTransport=Grpc` all sites; full UI command matrix per site; site-kill mid-command clean; no PSK noise; zero ClusterClient log activity on flipped paths — **PASS 2026-07-23** (live gate: `ExecuteQuery`/`ExecuteParked` all 3 sites + `ExecuteLifecycle` disable/enable on site-a, all 200 over `SiteCommandService`; active-site-node hard-kill mid-command → clean `TIMEOUT` at the 30s deadline, next call failed over to site-a-b automatically; 0 PermissionDenied / 0 ClusterClient-to-site on both central. `ExecuteOpcUa`/`ExecuteRoute`/`TriggerFailover` deferred — no OPC-bound instance / no CLI verb, unit-proven) **Update 2026-08-07 (truth sweep): no longer deferred — `ExecuteOpcUa`/`ExecuteRoute`/parked-retry/`TriggerSiteFailover` were ALL LIVE-PROVEN 2026-08-01 (`1c99d6fa`); see the Phase-5 closure note below.**
**Phase 4 — deletion**
- [x] gRPC made the only transport (flags DELETED, not flipped — end state is identical) + deleted `AkkaCentralTransport`/`AkkaSiteTransport`, ClusterClient creation (`AkkaHostedService`), `DefaultSiteClientFactory`+`ISiteClientFactory`, both receptionist registrations (`:436`/`:1001`), `CentralContactPoints` + the `CentralTransport`/`SiteTransport` flags + `CentralTransportMode`/`SiteTransportKind` enums. `NoOpCentralTransport` added as the fail-loud null-default; `CentralGrpcEndpoints` now unconditional (StartupValidator requires ≥1 on Site). Kept `Akka.Cluster.Tools` (ClusterSingleton). Rig configs (docker ×6, docker-env2 ×2, Host default, wonder-app-vd03) moved `CentralContactPoints`→`CentralGrpcEndpoints`. **Full solution build 0/0; Communication.Tests 640, Host.Tests + StartupValidator + SiteActorPath green, audit-push integration green** (2026-07-23)
- [x] Grep-gates pass (`CentralContactPoints` → only "replaces the former" doc refs + plan trackers; deleted symbols → 0 live refs; remaining `clusterclient` in src = the misleadingly-named `ClusterClientSiteAuditClient` [transport-agnostic, works unchanged] + stale inline doc-comments, noted as follow-up)
- [x] Grep-gates pass (`CentralContactPoints` → only "replaces the former" doc refs + plan trackers; deleted symbols → 0 live refs; remaining `clusterclient` in src = the misleadingly-named `ClusterClientSiteAuditClient` [transport-agnostic, works unchanged] + stale inline doc-comments, noted as follow-up) **Update 2026-08-07 (truth sweep): follow-up DONE — commit `63c16d69` (2026-07-27, "retire ClusterClient naming after the gRPC cutover") renamed it to `SiteCommunicationAuditClient` and rewrote the stale inline comments; grep confirms 0 `ClusterClientSiteAuditClient` references remain in `src/`/`tests/`.**
- [x] Docs updated: `Component-Communication.md`, `components/Communication.md`, `Component-Host.md`, `Component-StoreAndForward.md`, `topology-guide.md`, `grpc_streams.md`, known-issues frame-size amendment, **CLAUDE.md** transport decisions
**Phase 5 — live gate** (record in `2026-07-22-clusterclient-to-grpc-live-gate.md`) — **PASS 2026-07-23** (deletion build, `main` @ `7fd5cb2b`)
+21 -3
View File
@@ -1,5 +1,23 @@
# Phase 8 Execution Checklist
**Status**: Complete
**Tests**: All passing
**Build**: 0 errors, 0 warnings
**Update 2026-08-07 (truth sweep):** This checklist previously read
"Status: Complete / Tests: All passing / Build: 0 errors, 0 warnings" — a
107-byte stub with no per-work-package results and no linked run. That claim
was **unevidenced** and has been retired.
Honest state, per the deferred-work register
(`2026-07-08-deferred-work-register.md`, row 25):
- The **Phase-8 WP-4 target-scale load test** (10 sites × 500 instances ×
75 tags = 37,500 subscriptions/site, ~375,000 total) has **never been run**.
- Nearest real coverage is arithmetic/aggregation only —
`PerformanceTests/StaggeredStartupTests.cs`
(`TagCapacity_75TagsPer500Machines_37500Total`) and
`HealthAggregationTests` — plus a single-subscriber 100k-event
`Streaming/SiteStreamThroughputTests.cs`. No sustained multi-site run
exists anywhere in `tests/` or `docker/`.
- Revisit trigger: before any production go-live at target scale, or the
first site approaching ~500 instances / ~37.5k subscriptions.
See `phase-8-production-readiness.md:152-170` (WP-4) and `:314-320`
(test protocol) for the original scope.