{ "planPath": "docs/plans/2026-07-19-localdb-adoption-phase2.md", "execution": { "mode": "parallel-waves", "implementerModel": "opus", "isolation": "worktree", "branch": "feat/localdb-phase2", "baseBranch": "feat/localdb-phase1", "note": "Phase 1's branch is NOT merged/pushed, so phase 2 branches from it. Dispatch every unblocked task concurrently per the wave table in the plan. Parallel implementers MUST use worktree isolation - concurrent git in one worktree races destructively." }, "scopeDecision": { "date": "2026-07-19", "by": "user", "choice": "Full scope as designed", "note": "User chose both surfaces (config tables + sf_messages) in one phase, deleting both bespoke mechanisms together, over the recommended split of S&F-first. The four open design questions are therefore resolved INSIDE this plan as D1-D5 rather than deferred." }, "decisions": [ { "id": "D1", "subject": "Config moves to CDC; notify-and-fetch is DELETED, not preserved", "evidence": "SiteReplicationActor sends id+fetch-coords only because the config blob exceeds Akka's 128KB frame (docs/known-issues/2026-06-26-deploy-config-exceeds-akka-frame-size.md). LocalDb sync is gRPC - no such limit. StoreDeployedConfigIfNewerAsync's guard (SiteStorageService.cs:325, `WHERE excluded.deployed_at > deployed_configurations.deployed_at`) protects only against a stale FETCH racing; with no fetch there is no race. Do NOT reproduce the guard on top of LWW - deployed_at and HLC are different clocks and mixing them is non-convergent." }, { "id": "D2", "subject": "ReplaceAllAsync deleted; the N1 directional guard becomes unnecessary", "evidence": "LocalDb's snapshot resync MERGES per-row LWW and never deletes: SnapshotApplier.OnBeginAsync (SnapshotStreamer.cs:163-170) resets counters only, no DELETE anywhere in the class; OnBatchAsync (:172-186) routes snapshot rows through the same LwwApplier as deltas; LwwApplier.cs:69-78 discards an incoming row whose HLC is lower. So a stale peer cannot wipe a live buffer - structurally impossible, not guarded. SEMANTIC CHANGE: the standby is convergent, no longer byte-identical." }, { "id": "D3", "subject": "The SMTP purge is re-homed BEFORE anything is deleted", "evidence": "SiteReplicationActor.HandleApplyArtifacts calls PurgeCentralOnlyNotificationConfigAsync (SiteStorageService.cs:811-821), deleting notification_lists + smtp_configurations incl. plaintext SMTP passwords from pre-2026-07-10 deployments. A security cleanup riding the replication path; CDC will not reproduce it. Task 12 moves it to the active node's deploy path before any deletion, so it never lapses." }, { "id": "D4", "subject": "native_alarm_state volume is MEASURED, not assumed", "evidence": "scadabridge.db is not only config - native_alarm_state mirrors live A&C conditions (NativeAlarmActor.cs:504) and is the highest-volume table in either DB. sf_messages worst case ~50 row-writes/sec. Task 1 is a rig soak that measures both and sets MaxOplogRows/MaxOplogAge. If backlog grows monotonically, STOP: the design doc's keyed-instances escape hatch (line 140) is a scadaproj library effort that would suspend this plan." }, { "id": "D5", "subject": "Cutover forecloses rolling site upgrades", "evidence": "SiteReplicationActor retains a legacy monolithic SfBufferSnapshot handler for rolling upgrades. With no dual-mechanism period, one node would speak a protocol the other no longer implements. Both nodes of a site must be stopped and started together. Task 21 puts this in the deployment runbook." } ], "reconFindings": [ "The gate doc named 2 test files as the specification; the real spec is 5 - it missed StoreAndForwardReplicationTests.cs (incl. the only Requeue coverage), ReplicationWireSerializationPinTests.cs, ResyncWireSerializationPinTests.cs, and SfBufferResyncPredicateTests.cs (the N1 Critical regression test).", "sf_messages has NO version column today - ON CONFLICT(id) DO UPDATE has no comparison predicate (StoreAndForwardStorage.cs:331-345). 'Newest wins' is bare arrival order. LWW-by-HLC is an IMPROVEMENT here, not a regression.", "No autoincrement-integer PKs exist anywhere in Phase 2 scope - all 9 config tables use natural TEXT or composite TEXT keys, sf_messages is TEXT. Phase 1's site_events GUID conversion has no Phase 2 analogue.", "SiteStorageService has NO foreign keys. RemoveDeployedConfigAsync (:343-376) is a manual 3-statement cascade in one transaction. Under CDC these become three independent delete streams that LWW may reorder - the most likely real defect in the plan (Task 18 scenario 2).", "DeploymentManagerActor's replicationActor is an OPTIONAL POSITIONAL parameter (:184). Removing it silently shifts every argument after it. Compilers will not catch a shift between two same-typed optional params - check every call site by hand (Task 16).", "ActiveNodeEvaluator must NOT be deleted - the S&F delivery gate still uses it. Only its doc comment mentions replication.", "ConfigFetchRetryCount may not be fully dead: IDeploymentConfigFetcher serves BOTH the standby replication path and the active singleton's RefreshDeploymentCommand path. Verify before removing (Task 13 step 2).", "notification_lists and smtp_configurations are deliberately NOT migrated (Task 9) - migrating them would resurrect plaintext SMTP passwords from a pre-fix legacy file into a replicated table." ], "tasks": [ {"id": 1, "subject": "Task 1: Rig soak - measure oplog growth under real write rates", "status": "pending", "classification": "high-risk", "note": "GATES EVERYTHING AFTER TASK 2. Stop the plan if oplog backlog grows monotonically."}, {"id": 2, "subject": "Task 2: Decision record - close the phase 2 gate", "status": "pending", "classification": "trivial", "blockedBy": [1]}, {"id": 3, "subject": "Task 3: Extract StoreAndForwardSchema.Apply", "status": "pending", "classification": "small", "blockedBy": [2]}, {"id": 4, "subject": "Task 4: Extract SiteStorageSchema.Apply", "status": "pending", "classification": "small", "blockedBy": [2]}, {"id": 5, "subject": "Task 5: Rewire StoreAndForwardStorage onto ILocalDb", "status": "pending", "classification": "high-risk", "blockedBy": [3]}, {"id": 6, "subject": "Task 6: Rewire SiteStorageService onto ILocalDb", "status": "pending", "classification": "high-risk", "blockedBy": [4]}, {"id": 7, "subject": "Task 7: Extend SiteLocalDbSetup with the new DDL (not yet registered)", "status": "pending", "classification": "standard", "blockedBy": [5, 6]}, {"id": 8, "subject": "Task 8: Extend the legacy migrator for sf_messages", "status": "pending", "classification": "high-risk", "blockedBy": [7]}, {"id": 9, "subject": "Task 9: Extend the legacy migrator for the 9 config tables", "status": "pending", "classification": "high-risk", "blockedBy": [7]}, {"id": 10, "subject": "Task 10: Port the S&F replication test intents as CDC specs", "status": "pending", "classification": "standard", "blockedBy": [8]}, {"id": 11, "subject": "Task 11: Port the resync + directional-authority tests", "status": "pending", "classification": "standard", "blockedBy": [9]}, {"id": 12, "subject": "Task 12: Re-home the SMTP purge off the replication path", "status": "pending", "classification": "high-risk", "blockedBy": [10, 11], "note": "MUST land before any deletion so the security purge never lapses."}, {"id": 13, "subject": "Task 13: Delete the notify-and-fetch config path", "status": "pending", "classification": "high-risk", "blockedBy": [12]}, {"id": 14, "subject": "Task 14: Register the Phase 2 tables and delete ReplicationService", "status": "pending", "classification": "high-risk", "blockedBy": [13], "note": "THE CUTOVER. Registration + bespoke deletion in ONE commit so both mechanisms never run together."}, {"id": 15, "subject": "Task 15: Delete SiteReplicationActor and its messages", "status": "pending", "classification": "high-risk", "blockedBy": [14]}, {"id": 16, "subject": "Task 16: Clean up DeploymentManagerActor and AkkaHostedService", "status": "pending", "classification": "standard", "blockedBy": [15]}, {"id": 17, "subject": "Task 17: Config-key cleanup", "status": "pending", "classification": "standard", "blockedBy": [16]}, {"id": 18, "subject": "Task 18: Two-node convergence suite for the Phase 2 tables", "status": "pending", "classification": "high-risk", "blockedBy": [17]}, {"id": 19, "subject": "Task 19: Rig configuration", "status": "pending", "classification": "small", "blockedBy": [17]}, {"id": 20, "subject": "Task 20: Live gate on the docker rig", "status": "pending", "classification": "high-risk", "blockedBy": [18, 19]}, {"id": 21, "subject": "Task 21: Documentation truth pass", "status": "pending", "classification": "standard", "blockedBy": [20]} ], "knownFlakes": [ { "test": "SiteRuntime.Tests InstanceActorChildAttributeRaceTests.ChildActors_AreSeededFromAnIsolatedCopy_NotTheLiveAttributesDictionary", "note": "Intermittent ActorNotFoundException under full-suite load; passes in isolation. Pre-existing, carried over from Phase 1." }, { "test": "AuditLog.Tests ParentExecutionIdCorrelationTests.InboundRoutedRun_AllRoutedRows_CarryInboundExecutionId_AsParentExecutionId", "note": "Cold-MSSQL-fixture timing: ~91s and AwaitAssert-times-out cold, ~1s warm. Re-run before investigating." } ], "lastUpdated": "2026-07-19", "phase2Status": "PLANNED - not started. Task 1 (rig soak) gates the rest." }