docs(localdb): phase 2 implementation plan (21 tasks, full scope)
Moves scadabridge.db's 9 config tables + sf_messages into the consolidated LocalDb file and deletes SiteReplicationActor + StoreAndForward's ReplicationService. Resolves the phase 2 gate's five open questions from code recon (D1-D5): D1 notify-and-fetch is DELETED, not preserved. It exists only because the config blob exceeds Akka's 128KB frame; LocalDb sync is gRPC. The deployed_at version guard protected against a stale fetch racing, so it dies with the fetch. Do not reproduce it on LWW - different clocks. D2 ReplaceAllAsync is deleted and the N1 directional guard becomes unnecessary: LocalDb's snapshot resync merges per-row LWW and never deletes (SnapshotApplier has no DELETE; LwwApplier.cs:69-78 discards a lower-HLC incoming row). Semantic change - the standby is convergent, no longer byte-identical. D3 The SMTP purge (plaintext passwords) rides the replication path and is re-homed to the active node BEFORE any deletion. D4 native_alarm_state volume is measured by a rig soak, not assumed. Task 1 gates the plan and stops it if the oplog cannot absorb the churn. D5 No dual-mechanism period forecloses rolling site upgrades - both nodes must stop and start together. Recon also found the gate doc's "two test files are the spec" undercounts: the real specification is five files, including the N1 Critical regression test and the only Requeue coverage. Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
This commit is contained in:
Joseph Doherty
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,89 @@
|
||||
{
|
||||
"planPath": "docs/plans/2026-07-19-localdb-adoption-phase2.md",
|
||||
"execution": {
|
||||
"mode": "parallel-waves",
|
||||
"implementerModel": "opus",
|
||||
"isolation": "worktree",
|
||||
"branch": "feat/localdb-phase2",
|
||||
"baseBranch": "feat/localdb-phase1",
|
||||
"note": "Phase 1's branch is NOT merged/pushed, so phase 2 branches from it. Dispatch every unblocked task concurrently per the wave table in the plan. Parallel implementers MUST use worktree isolation - concurrent git in one worktree races destructively."
|
||||
},
|
||||
"scopeDecision": {
|
||||
"date": "2026-07-19",
|
||||
"by": "user",
|
||||
"choice": "Full scope as designed",
|
||||
"note": "User chose both surfaces (config tables + sf_messages) in one phase, deleting both bespoke mechanisms together, over the recommended split of S&F-first. The four open design questions are therefore resolved INSIDE this plan as D1-D5 rather than deferred."
|
||||
},
|
||||
"decisions": [
|
||||
{
|
||||
"id": "D1",
|
||||
"subject": "Config moves to CDC; notify-and-fetch is DELETED, not preserved",
|
||||
"evidence": "SiteReplicationActor sends id+fetch-coords only because the config blob exceeds Akka's 128KB frame (docs/known-issues/2026-06-26-deploy-config-exceeds-akka-frame-size.md). LocalDb sync is gRPC - no such limit. StoreDeployedConfigIfNewerAsync's guard (SiteStorageService.cs:325, `WHERE excluded.deployed_at > deployed_configurations.deployed_at`) protects only against a stale FETCH racing; with no fetch there is no race. Do NOT reproduce the guard on top of LWW - deployed_at and HLC are different clocks and mixing them is non-convergent."
|
||||
},
|
||||
{
|
||||
"id": "D2",
|
||||
"subject": "ReplaceAllAsync deleted; the N1 directional guard becomes unnecessary",
|
||||
"evidence": "LocalDb's snapshot resync MERGES per-row LWW and never deletes: SnapshotApplier.OnBeginAsync (SnapshotStreamer.cs:163-170) resets counters only, no DELETE anywhere in the class; OnBatchAsync (:172-186) routes snapshot rows through the same LwwApplier as deltas; LwwApplier.cs:69-78 discards an incoming row whose HLC is lower. So a stale peer cannot wipe a live buffer - structurally impossible, not guarded. SEMANTIC CHANGE: the standby is convergent, no longer byte-identical."
|
||||
},
|
||||
{
|
||||
"id": "D3",
|
||||
"subject": "The SMTP purge is re-homed BEFORE anything is deleted",
|
||||
"evidence": "SiteReplicationActor.HandleApplyArtifacts calls PurgeCentralOnlyNotificationConfigAsync (SiteStorageService.cs:811-821), deleting notification_lists + smtp_configurations incl. plaintext SMTP passwords from pre-2026-07-10 deployments. A security cleanup riding the replication path; CDC will not reproduce it. Task 12 moves it to the active node's deploy path before any deletion, so it never lapses."
|
||||
},
|
||||
{
|
||||
"id": "D4",
|
||||
"subject": "native_alarm_state volume is MEASURED, not assumed",
|
||||
"evidence": "scadabridge.db is not only config - native_alarm_state mirrors live A&C conditions (NativeAlarmActor.cs:504) and is the highest-volume table in either DB. sf_messages worst case ~50 row-writes/sec. Task 1 is a rig soak that measures both and sets MaxOplogRows/MaxOplogAge. If backlog grows monotonically, STOP: the design doc's keyed-instances escape hatch (line 140) is a scadaproj library effort that would suspend this plan."
|
||||
},
|
||||
{
|
||||
"id": "D5",
|
||||
"subject": "Cutover forecloses rolling site upgrades",
|
||||
"evidence": "SiteReplicationActor retains a legacy monolithic SfBufferSnapshot handler for rolling upgrades. With no dual-mechanism period, one node would speak a protocol the other no longer implements. Both nodes of a site must be stopped and started together. Task 21 puts this in the deployment runbook."
|
||||
}
|
||||
],
|
||||
"reconFindings": [
|
||||
"The gate doc named 2 test files as the specification; the real spec is 5 - it missed StoreAndForwardReplicationTests.cs (incl. the only Requeue coverage), ReplicationWireSerializationPinTests.cs, ResyncWireSerializationPinTests.cs, and SfBufferResyncPredicateTests.cs (the N1 Critical regression test).",
|
||||
"sf_messages has NO version column today - ON CONFLICT(id) DO UPDATE has no comparison predicate (StoreAndForwardStorage.cs:331-345). 'Newest wins' is bare arrival order. LWW-by-HLC is an IMPROVEMENT here, not a regression.",
|
||||
"No autoincrement-integer PKs exist anywhere in Phase 2 scope - all 9 config tables use natural TEXT or composite TEXT keys, sf_messages is TEXT. Phase 1's site_events GUID conversion has no Phase 2 analogue.",
|
||||
"SiteStorageService has NO foreign keys. RemoveDeployedConfigAsync (:343-376) is a manual 3-statement cascade in one transaction. Under CDC these become three independent delete streams that LWW may reorder - the most likely real defect in the plan (Task 18 scenario 2).",
|
||||
"DeploymentManagerActor's replicationActor is an OPTIONAL POSITIONAL parameter (:184). Removing it silently shifts every argument after it. Compilers will not catch a shift between two same-typed optional params - check every call site by hand (Task 16).",
|
||||
"ActiveNodeEvaluator must NOT be deleted - the S&F delivery gate still uses it. Only its doc comment mentions replication.",
|
||||
"ConfigFetchRetryCount may not be fully dead: IDeploymentConfigFetcher serves BOTH the standby replication path and the active singleton's RefreshDeploymentCommand path. Verify before removing (Task 13 step 2).",
|
||||
"notification_lists and smtp_configurations are deliberately NOT migrated (Task 9) - migrating them would resurrect plaintext SMTP passwords from a pre-fix legacy file into a replicated table."
|
||||
],
|
||||
"tasks": [
|
||||
{"id": 1, "subject": "Task 1: Rig soak - measure oplog growth under real write rates", "status": "pending", "classification": "high-risk", "note": "GATES EVERYTHING AFTER TASK 2. Stop the plan if oplog backlog grows monotonically."},
|
||||
{"id": 2, "subject": "Task 2: Decision record - close the phase 2 gate", "status": "pending", "classification": "trivial", "blockedBy": [1]},
|
||||
{"id": 3, "subject": "Task 3: Extract StoreAndForwardSchema.Apply", "status": "pending", "classification": "small", "blockedBy": [2]},
|
||||
{"id": 4, "subject": "Task 4: Extract SiteStorageSchema.Apply", "status": "pending", "classification": "small", "blockedBy": [2]},
|
||||
{"id": 5, "subject": "Task 5: Rewire StoreAndForwardStorage onto ILocalDb", "status": "pending", "classification": "high-risk", "blockedBy": [3]},
|
||||
{"id": 6, "subject": "Task 6: Rewire SiteStorageService onto ILocalDb", "status": "pending", "classification": "high-risk", "blockedBy": [4]},
|
||||
{"id": 7, "subject": "Task 7: Extend SiteLocalDbSetup with the new DDL (not yet registered)", "status": "pending", "classification": "standard", "blockedBy": [5, 6]},
|
||||
{"id": 8, "subject": "Task 8: Extend the legacy migrator for sf_messages", "status": "pending", "classification": "high-risk", "blockedBy": [7]},
|
||||
{"id": 9, "subject": "Task 9: Extend the legacy migrator for the 9 config tables", "status": "pending", "classification": "high-risk", "blockedBy": [7]},
|
||||
{"id": 10, "subject": "Task 10: Port the S&F replication test intents as CDC specs", "status": "pending", "classification": "standard", "blockedBy": [8]},
|
||||
{"id": 11, "subject": "Task 11: Port the resync + directional-authority tests", "status": "pending", "classification": "standard", "blockedBy": [9]},
|
||||
{"id": 12, "subject": "Task 12: Re-home the SMTP purge off the replication path", "status": "pending", "classification": "high-risk", "blockedBy": [10, 11], "note": "MUST land before any deletion so the security purge never lapses."},
|
||||
{"id": 13, "subject": "Task 13: Delete the notify-and-fetch config path", "status": "pending", "classification": "high-risk", "blockedBy": [12]},
|
||||
{"id": 14, "subject": "Task 14: Register the Phase 2 tables and delete ReplicationService", "status": "pending", "classification": "high-risk", "blockedBy": [13], "note": "THE CUTOVER. Registration + bespoke deletion in ONE commit so both mechanisms never run together."},
|
||||
{"id": 15, "subject": "Task 15: Delete SiteReplicationActor and its messages", "status": "pending", "classification": "high-risk", "blockedBy": [14]},
|
||||
{"id": 16, "subject": "Task 16: Clean up DeploymentManagerActor and AkkaHostedService", "status": "pending", "classification": "standard", "blockedBy": [15]},
|
||||
{"id": 17, "subject": "Task 17: Config-key cleanup", "status": "pending", "classification": "standard", "blockedBy": [16]},
|
||||
{"id": 18, "subject": "Task 18: Two-node convergence suite for the Phase 2 tables", "status": "pending", "classification": "high-risk", "blockedBy": [17]},
|
||||
{"id": 19, "subject": "Task 19: Rig configuration", "status": "pending", "classification": "small", "blockedBy": [17]},
|
||||
{"id": 20, "subject": "Task 20: Live gate on the docker rig", "status": "pending", "classification": "high-risk", "blockedBy": [18, 19]},
|
||||
{"id": 21, "subject": "Task 21: Documentation truth pass", "status": "pending", "classification": "standard", "blockedBy": [20]}
|
||||
],
|
||||
"knownFlakes": [
|
||||
{
|
||||
"test": "SiteRuntime.Tests InstanceActorChildAttributeRaceTests.ChildActors_AreSeededFromAnIsolatedCopy_NotTheLiveAttributesDictionary",
|
||||
"note": "Intermittent ActorNotFoundException under full-suite load; passes in isolation. Pre-existing, carried over from Phase 1."
|
||||
},
|
||||
{
|
||||
"test": "AuditLog.Tests ParentExecutionIdCorrelationTests.InboundRoutedRun_AllRoutedRows_CarryInboundExecutionId_AsParentExecutionId",
|
||||
"note": "Cold-MSSQL-fixture timing: ~91s and AwaitAssert-times-out cold, ~1s warm. Re-run before investigating."
|
||||
}
|
||||
],
|
||||
"lastUpdated": "2026-07-19",
|
||||
"phase2Status": "PLANNED - not started. Task 1 (rig soak) gates the rest."
|
||||
}
|
||||
Reference in New Issue
Block a user