dohertj2/ScadaBridge
1
0
Fork 0
Code Issues 2 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

chore(deps): LocalDb 0.2.0 — dereg cleanup, late-opt-in baselining, byte-budget replication

Browse Source
This commit is contained in:
Joseph Doherty
2026-08-14 22:22:18 -04:00
parent 312216ff2b
commit cca7f1786d
9 changed files with 288 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/ZB.MOM.WW.ScadaBridge.Host/SiteLocalDbSetup.cs
+98 -36
View File
@@ -29,6 +29,13 @@ namespace ZB.MOM.WW.ScadaBridge.Host;
/// (its rows simply do not replicate, which is what "unreplicated node" means).
/// </para>
/// <para>
/// <b>The unreplicated branch actively cleans up</b> rather than merely abstaining:
/// <c>DeregisterReplicated</c> (LocalDb 0.2.0) drops any capture triggers a previous build
/// left in the file and prunes that table's oplog and row-version rows. Skipping
/// registration alone would have left a file registered under an older build capturing
/// forever, which is exactly the node that has no peer to capture for.
/// </para>
/// <para>
/// Runs inside the <c>AddZbLocalDb</c> onReady callback, once, before any caller
/// receives the <see cref="ILocalDb"/> singleton. onReady is a synchronous
/// <c>Action&lt;ILocalDb&gt;</c>, hence the direct <c>CreateConnection()</c> rather than
@@ -39,6 +46,30 @@ namespace ZB.MOM.WW.ScadaBridge.Host;
/// </remarks>
public static class SiteLocalDbSetup
{
/// <summary>
/// The ten tables that participate in replication — registered when this node has a peer,
/// and deregistered (stale triggers dropped) when it does not.
/// </summary>
/// <remarks>
/// <c>notification_lists</c> and <c>smtp_configurations</c> are created by the schema but
/// deliberately absent here; see the rationale in <see cref="OnReady"/>. The list is shared by
/// both branches on purpose: a table added to one and not the other would either register
/// without ever being cleaned up, or be cleaned up on a node that never registered it.
/// </remarks>
private static readonly string[] ReplicatedTables =
[
"OperationTracking",
"site_events",
"sf_messages",
"deployed_configurations",
"static_attribute_overrides",
"shared_scripts",
"external_systems",
"database_connections",
"data_connection_definitions",
"native_alarm_state",
];
/// <summary>
/// Creates the site node's tables, opts them into change capture when this node
/// replicates, and migrates any pre-Phase-1 databases in.
@@ -66,36 +97,70 @@ public static class SiteLocalDbSetup
if (ReplicationIsConfigured(config))
{
// Both tables qualify: each has an explicit primary key (RegisterReplicated
// Every table below qualifies: each has an explicit primary key (RegisterReplicated
// rejects tables without one) and no BLOB columns (which json_object cannot
// capture). Registration is idempotent and installs the capture triggers.
db.RegisterReplicated("OperationTracking");
db.RegisterReplicated("site_events");
// Phase 2: the store-and-forward buffer and the seven site configuration tables.
// These replaced the bespoke SiteReplicationActor and StoreAndForward
// capture); the two composite-PK tables are fine, since RegisterReplicated orders
// multi-column PKs by ordinal. Registration is idempotent and installs the capture
// triggers.
//
// The Phase 2 members of the list — the store-and-forward buffer and the seven site
// configuration tables — replaced the bespoke SiteReplicationActor and StoreAndForward
// ReplicationService, which shipped hand-written Add/Remove/Park/Requeue operations
// over Akka; both were deleted in the same commit that added these lines, so the
// two mechanisms never ran at once.
//
// Both composite-PK tables are fine: RegisterReplicated orders multi-column PKs by
// ordinal. No Phase 2 table has a BLOB column, which it would reject.
db.RegisterReplicated("sf_messages");
db.RegisterReplicated("deployed_configurations");
db.RegisterReplicated("static_attribute_overrides");
db.RegisterReplicated("shared_scripts");
db.RegisterReplicated("external_systems");
db.RegisterReplicated("database_connections");
db.RegisterReplicated("data_connection_definitions");
db.RegisterReplicated("native_alarm_state");
// baselineExistingRows: true is what makes turning replication ON for a site that has
// been running WITHOUT it actually converge. Capture is change-data-capture and the
// snapshot streamer pages from __localdb_row_version, so rows written before the first
// registration exist in neither — a late opt-in used to replicate only writes made
// after the restart, silently and forever. Baselining seeds the ledger at the LWW
// floor (HLC 0, this node's id) and flags a snapshot resync, so a baselined row loses
// to any genuine remote write of the same key and wins only where the peer has no
// version at all. It is idempotent (ON CONFLICT DO NOTHING) and therefore free on
// every boot after the first, which is why it is unconditional rather than a flag.
foreach (var table in ReplicatedTables)
db.RegisterReplicated(table, baselineExistingRows: true);
// notification_lists and smtp_configurations are created but deliberately NOT
// registered. They are permanently empty by design — the site-side write paths were
// removed on 2026-07-10, the legacy migrator skips them, and the active node's
// artifact apply purges them on every deploy. Registering them would open a standing
// replication channel whose only historical payload was plaintext SMTP passwords, in
// exchange for replicating nothing. Anyone adding them here should first establish
// that a site has a legitimate reason to hold SMTP credentials at all.
// registered — see ReplicatedTables. They are permanently empty by design — the
// site-side write paths were removed on 2026-07-10, the legacy migrator skips them,
// and the active node's artifact apply purges them on every deploy. Registering them
// would open a standing replication channel whose only historical payload was
// plaintext SMTP passwords, in exchange for replicating nothing. Anyone adding them
// here should first establish that a site has a legitimate reason to hold SMTP
// credentials at all.
}
else
{
// Not merely "do not register": actively remove capture this node must not pay for.
// The file may have been registered by an EARLIER build (every build before WP1.3
// registered unconditionally), in which case its triggers are still there, still
// running two INSERTs plus a json_object of the full row inside every write
// transaction, still appending to an oplog with no reader. DeregisterReplicated is
// idempotent and deliberately usable on a table this process never registered, so a
// never-replicated file simply reports nothing to clean.
//
// Symmetry is not a concern on this branch: this node has no peer, so there is no
// handshake digest for a one-sided deregistration to fail. And re-enabling later is
// safe despite the ledger prune, because the registration branch above baselines.
var cleaned = 0;
foreach (var table in ReplicatedTables)
{
if (db.DeregisterReplicated(table)) cleaned++;
}
if (cleaned > 0)
{
// Serilog's static logger, as in SecretsRegistration: this runs inside the
// AddZbLocalDb singleton factory, where no ILogger can be resolved without
// nesting a service resolution inside a singleton construction. Program.cs
// configures Log.Logger before the service graph is built.
Serilog.Log.Information(
"LocalDb: removed stale change-capture from {TableCount} table(s) — this node has no "
+ "replication peer configured, so the triggers a previous build installed have been "
+ "dropped and their oplog/row-version rows pruned.",
cleaned);
}
}
// AFTER registration, so migrated rows enter the oplog and reach the peer like
@@ -130,19 +195,16 @@ public static class SiteLocalDbSetup
/// replicates.
/// </para>
/// <para>
/// <b>Known residual:</b> a database file that was registered by an OLDER build keeps
/// its stale <c>__localdb_*</c> triggers — this only decides whether new ones are
/// installed, and the library has no removal API yet (it arrives with the WP3.3 library
/// work, which will also drop them on an unconfigured node). On the docker rig this is
/// moot: a schema change recreates the volumes. On a long-lived unreplicated node
/// upgraded in place, capture continues until that lands.
/// </para>
/// <para>
/// <b>The other direction has a consequence too:</b> turning replication ON for a site
/// that has been running without it does NOT baseline the rows already in the file.
/// Capture never recorded them in <c>__localdb_row_version</c>, and the snapshot resync
/// streams from that ledger, so the pair converges only on writes made after the
/// restart. Seed both nodes from one node's database if the existing rows matter. See
/// <b>Both directions of a change to this predicate are now handled at boot</b> (LocalDb
/// 0.2.0). Flipping it to false deregisters, so a file registered by an older build stops
/// capturing on the next start instead of paying for triggers forever; flipping it to true
/// baselines, so a site that has been running unreplicated converges on the rows already in
/// its file and not merely on writes made after the restart. Seeding one node from the
/// other's database beforehand is no longer required — only still advisable where BOTH
/// files hold their own version of the same key, since two baselined rows both sit at HLC
/// 0 and the node-id tie-break then decides arbitrarily which content survives. Either
/// way both nodes must be changed together: deregistration is only symmetric — and the
/// handshake digest only agrees — if both sides do it. See
/// <c>docs/deployment/topology-guide.md</c>.
/// </para>
/// </remarks>