chore(deps): LocalDb 0.3.4 -> 0.3.7 (#130, scadaproj#53)

Pins ZB.MOM.WW.LocalDb, .Replication and .Contracts at 0.3.7 (published
2026-11-12). Wire-compatible; no config or code change needed.

- 0.3.6 (scadaproj#36): the library sync service enforces
  LocalDb:Replication:ApiKey itself. LocalDbSyncAuthInterceptor is kept for
  its refuse-every-stream-when-blank rule; its doc comment is updated. The
  rig's site-a pair already carries one identical key on both nodes.
- 0.3.7 (scadaproj#16): a tombstone is held until the peer has it, so a
  node offline past TombstoneRetention no longer resurrects deleted rows.
  The survivor's ledger (sf_messages / site_events deletes) grows for the
  length of the outage; optional TombstoneHardCap bounds it.

Deploy notes added to the topology + installation guides, the S&F
requirement and CLAUDE.md: stop/start the pair together, same ApiKey on
both nodes, ledger growth while a peer is down.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
2026-11-12 13:44:53 -05:00
parent 25015442e5
commit 12fa7affc3
6 changed files with 43 additions and 18 deletions
+10 -2
View File
@@ -191,8 +191,16 @@ Ensure bidirectional TCP connectivity between all Akka.NET cluster peers. The re
**Stop both nodes of a site pair, upgrade both, then start both.** Rolling one node at a time is
not supported as of LocalDb Phase 2 — the legacy snapshot-compatibility handler that made a
mixed-version pair converge was deleted with the bespoke replicator, and a mixed pair now diverges
silently. See `docs/deployment/topology-guide.md` for the reasoning and for the related
`TombstoneRetention` bound on how long one node may stay offline.
silently. See `docs/deployment/topology-guide.md` for the reasoning.
For a replicated pair (LocalDb 0.3.7):
- `LocalDb:Replication:ApiKey` must be set, byte-identical, on **both** nodes (a `${secret:...}`
reference or an environment variable — never a literal in a shipped config). Both the library
and ScadaBridge's `LocalDbSyncAuthInterceptor` refuse the sync stream without it.
- While one node is down, the survivor's delete tombstones are held until the partner returns and
acks them, so its LocalDb file grows for the length of the outage. Deleted rows no longer
resurrect on rejoin, however long the outage (scadaproj#16).
## Post-Installation Verification
+22 -7
View File
@@ -365,13 +365,28 @@ same boot, so a node picks up the new columns and the matching triggers in one s
(Most recent instance: `external_systems` gained `max_retries` / `retry_delay_ms` for the
per-system store-and-forward retry policy.)
**Related bound — do not leave one node of a pair offline for long.** A node absent for longer than
`LocalDb:Replication:TombstoneRetention` (default **7 days**) can **resurrect deleted rows** when
it rejoins: deletes replicate as HLC-ordered tombstones, and once a tombstone is pruned there is
nothing left to suppress the stale row the returning node still holds. Within the retention window
a rejoin is safe and self-correcting (verified live: a node stopped and restarted mid-load rejoined
with both nodes byte-identical and zero duplicates). Beyond it, rebuild the returning node's
database from its peer rather than letting it rejoin.
**Related bound — a node offline for long (changed in LocalDb 0.3.7, scadaproj#16).** Through
0.3.6 a node absent for longer than `LocalDb:Replication:TombstoneRetention` (default **7 days**)
could **resurrect deleted rows** when it rejoined, because its partner pruned the tombstones that
would have suppressed them. **As of 0.3.7 a tombstone is pruned only once it is past retention AND
the peer holds it** — an unacked local delete and any outstanding snapshot debt hold it, and
retention restarts from the moment of delivery. A rejoin after any length of outage is therefore
safe again. **The cost is ledger growth while the peer is down:** every delete made on the
surviving node (on a site, mostly `sf_messages` delivery deletes and the daily `site_events` purge)
keeps its tombstone in `__localdb_row_version` until the partner returns and acks it, so the site
LocalDb file grows for the length of the outage and shrinks only after the drain. The optional
`LocalDb:Replication:TombstoneHardCap` (default unset = off, must be ≥ `TombstoneRetention`) bounds
that growth — but past the cap the old resurrection risk returns, so leave it unset unless disk is
the binding constraint, and in that case rebuild the returning node from its peer instead of
letting it rejoin. A peer restored from an old backup of its own file is still not covered.
**The sync endpoint key is enforced by the library as of LocalDb 0.3.6 (scadaproj#36).**
`LocalDb:Replication:ApiKey` must be **byte-identical on both nodes** of a replicated pair.
ScadaBridge's own `LocalDbSyncAuthInterceptor` already refused a missing or mismatched key, so the
pin bump changes nothing for a correctly configured pair; it is the second, library-side check of
the same key. Supply it as a `${secret:...}` reference (resolved pre-host) or an environment
variable, never a literal in a shipped config. A mismatch shows as `LocalDbReplicationConnected =
false` with an `Unauthenticated` sync failure in the dialling node's log.
### Site LocalDb files created before 2026-08-20 need a one-off offline VACUUM