perf(host): install CDC capture only when replication is configured

SiteLocalDbSetup.OnReady registered all ten replicated tables
unconditionally, so a deliberately unreplicated site node (site-b and
site-c on the rig) carried the full 30-trigger CDC set forever. Every
write to those tables paid two extra INSERTs plus a json_object
serialization of the whole row, inside the caller's own transaction, and
appended to an oplog nothing ever drains. Arch-review finding #5 (High),
repo half; the library half — trigger cleanup API and O(1) backlog — is
WP3.3.

The ten RegisterReplicated calls are now behind a guard on whether the
node has LocalDb:Replication:PeerAddress OR LocalDb:Replication:ApiKey.
Either key counts, and the OR is load-bearing rather than defensive:
replication is one bidirectional stream that exactly one side dials, so
only the initiator sets PeerAddress. Verified against the rig — site-a
node-a has PeerAddress + ApiKey, site-a node-b (passive) has ApiKey
alone, site-b/site-c have no Replication section at all. Keying on
PeerAddress alone would have stripped capture from every passive node and
silently made each pair converge in one direction only.

The load-bearing ordering documented in the file is preserved: DDL still
precedes registration, and the legacy migrator still runs unconditionally
after it — an unreplicated node must still absorb its pre-Phase-1 files,
and it has no peer for those rows to be invisible to.

Known residual, documented in-file and in the topology guide: a database
file first created by an older build keeps its stale __localdb_* triggers.
The guard decides whether triggers are installed, not whether existing
ones are removed, and the library has no removal API until WP3.3. Moot on
the docker rig, where a schema-change redeploy recreates the volumes.
The inverse is also now documented: enabling replication on a site that
has run without it does not baseline existing rows, since CDC never
recorded them in __localdb_row_version and the snapshot resync streams
from that ledger.

Tests: new SiteLocalDbCdcRegistrationTests asserts trigger presence and
absence via sqlite_master across all four config shapes (none, ApiKey
only, PeerAddress + ApiKey, and the notification-table exclusion), plus
DDL-still-runs and migrator-still-runs on the unreplicated branch.
SiteLocalDbWiringTests and the integration site-pair harness now
configure an ApiKey — mirroring the rig's passive node — so their
registration and convergence assertions still describe a replicating
node. 483/483 Host.Tests pass; the 20 offline LocalDb convergence tests
still pass.
This commit is contained in:
Joseph Doherty
2026-08-14 19:59:53 -04:00
parent ee193cd2bb
commit 7ebdcd370a
5 changed files with 358 additions and 33 deletions
+20
View File
@@ -157,6 +157,26 @@ Each site has its own two-node cluster:
- SQLite persistence: each node owns its own consolidated LocalDb database, kept in step by
asynchronous CDC replication over a gRPC sync stream (LocalDb Phase 1 + 2). The nodes do NOT
share a SQLite file.
- CDC capture triggers are installed **only on a node that has replication configured**
`LocalDb:Replication:PeerAddress` *or* `LocalDb:Replication:ApiKey`. Either key counts, because
only the initiating half of a pair sets `PeerAddress` (one bidirectional stream, dialled by one
side); the passive half carries the key alone. A deliberately unreplicated node — site-b and
site-c on the rig — runs with no triggers at all and stops paying the per-write capture cost.
#### Turning replication ON for a site that has been running without it
Set the keys on **both** nodes and restart both. Two things to know before you do:
- **Existing rows are not baselined.** Capture is change-data-capture: rows written while the node
had no triggers were never recorded in `__localdb_row_version`, and LocalDb's snapshot resync
streams from that ledger, so it will not ship them. The pair converges on everything written
*after* the restart and stays silently divergent on everything before it. Start from a copy of one
node's database on both sides, or accept that only new writes converge.
- **A node upgraded in place keeps stale triggers.** The guard decides whether triggers are
*installed*, not whether existing ones are removed, and the library has no removal API yet. A
database file first created by a build that always registered keeps capturing until that lands.
Recreating the node's data volume clears it — which is what a schema-change redeploy does on the
docker rig, so the rig is unaffected.
### Site Pair Upgrades — stop and start BOTH nodes together