chore(deps): LocalDb 0.3.4 -> 0.3.7 (#889, scadaproj#53)
v2-ci / unit-tests (push) Waiting to run

Pins ZB.MOM.WW.LocalDb and .Replication at 0.3.7 (published 2026-11-12).
Wire-compatible; no code change needed.

- 0.3.6 (scadaproj#36): the library sync service enforces
  LocalDb:Replication:ApiKey itself. LocalDbSyncAuthInterceptor is kept for
  its refuse-every-stream-when-blank rule; doc comment updated. docker-dev
  site-a pair: both nodes now read one variable,
  OTOPCUA_SITE_A_LOCALDB_SYNC_KEY (same dev default as before), so an
  override can never split the pair.
- 0.3.7 (scadaproj#16): a tombstone is held until the peer has it, so a
  node offline past TombstoneRetention no longer brings a pruned
  deployment back. The ledger grows while a peer is down; optional
  TombstoneHardCap (documented in Configuration.md) bounds it.

Runbook, Configuration.md and the cluster-topology rule updated.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
2026-11-12 13:44:55 -05:00
parent 2e9cccbdef
commit e303165444
6 changed files with 40 additions and 23 deletions
+3 -2
View File
@@ -494,7 +494,8 @@ replication stays inert until the sync port + peer are set.
| `LocalDb:Path` | `./data/otopcua-localdb.db` | SQLite file path. `ValidateOnStart`-required once `AddZbLocalDb` runs (driver nodes only). In containers put it on a durable volume. |
| `LocalDb:SyncListenPort` | `0` | `0` = replication off (no sync listener). A non-zero port binds a dedicated **h2c** sync listener; set the **same** port on both pair nodes. Setting it makes the Host add an explicit Kestrel `Listen*` and re-bind the primary HTTP port (an explicit `Listen*` otherwise discards `ASPNETCORE_URLS`). |
| `LocalDb:Replication:PeerAddress` | `""` | The peer this node **dials** (`http://<peer>:<port>`). Set on the initiator only; leave empty on the passive node. The stream is bidirectional. |
| `LocalDb:Replication:ApiKey` | `""` | Bearer token for the sync stream. **Fail-closed:** no key ⇒ the passive endpoint refuses everything; a mismatch ⇒ the pair silently stops converging. **Must be byte-identical on both nodes.** Supply via `${secret:...}` / env — never a cleartext literal in production. |
| `LocalDb:Replication:ApiKey` | `""` | Bearer token for the sync stream. **Fail-closed:** no key ⇒ the passive endpoint refuses everything; a mismatch ⇒ the pair silently stops converging. **Must be byte-identical on both nodes.** Enforced by the host interceptor and, since LocalDb 0.3.6, by the library's sync service too. Supply via `${secret:...}` / env — never a cleartext literal in production. |
| `LocalDb:Replication:TombstoneHardCap` | unset (off) | LocalDb 0.3.7. A tombstone is kept until it is past `TombstoneRetention` (7 days) **and** the peer holds it, so the ledger grows while a peer is down. This optional cap (≥ `TombstoneRetention`) bounds an undelivered tombstone's life; past it a returning node can resurrect deleted rows again. Leave unset unless disk is the constraint. |
| `LocalDb:Replication:MaxBatchSize` | `500` (library) | **Secondary** row-count cap on a batch — *and* the DB read-page size. `16` on the replicating pair. Since LocalDb 0.2.0 the wire message is bounded by `MaxBatchBytes`, not by this, so it is no longer the hand-tuned byte proxy the 4 MB cap once forced. It still matters because `OplogStore.ReadBatchAboveAsync`/`SnapshotStreamer` materialise a whole read page **before** the byte-budget split runs — `16` bounds that transient allocation at ≈2.7 MB for ≈171 KB artifact chunk rows. |
| `LocalDb:Replication:MaxBatchBytes` | `2 MB` (library) | **Primary** cap on one outbound delta/snapshot message: the summed serialized size of its entries, sized under gRPC's 4 MB receive limit. Left unset across the fleet. A single row larger than the budget is sent **alone** rather than stalling the stream. Added in LocalDb 0.2.0; needs no wire change, so a node on this version interoperates with any earlier peer in both directions. |
@@ -510,7 +511,7 @@ turning replication on converges the rows **already in the file** and not merely
unconfigured node **deregisters** them at boot, dropping capture triggers an earlier build installed. ⚠️ The
sync handshake compares registered-table digests **fail-closed**, so flipping either key is a
both-nodes-together change: set it on both and restart both. As of LocalDb 0.3.0 (this repo pins
0.3.4) baselining flags a **per-table** snapshot resync — enabling replication on one table no
0.3.7) baselining flags a **per-table** snapshot resync — enabling replication on one table no
longer forces a full-database resync of every registered table in both directions.
Replication is **default-OFF** across the fleet; it is enabled per-pair as an opt-in. See
@@ -77,9 +77,10 @@ Replication has two knobs, both under `LocalDb`:
| `LocalDb:Replication:MaxBatchSize` | secondary row cap **and** the DB read-page size | `16` for the artifact cache. **Since LocalDb 0.2.0 this no longer bounds the wire message** — `MaxBatchBytes` does — so it is not the byte proxy the note here used to describe. Keep it at `16` anyway: `OplogStore`/`SnapshotStreamer` materialise a whole read page in memory *before* the byte-budget split runs, and 16 × ≈171 KB artifact chunk rows ≈ 2.7 MB bounds that transient allocation. Raising it no longer risks tripping the gRPC cap, only the allocation. |
| `LocalDb:Replication:MaxBatchBytes` | primary cap on one outbound message (bytes) | Library default **2 MB**, sized under gRPC's 4 MB receive limit. Left unset on this rig. A single row over budget is sent alone rather than stalling the stream. |
> **Library version.** This repo pins **`ZB.MOM.WW.LocalDb` / `.Replication` 0.3.4**
> (`Directory.Packages.props`; bumped from `0.2.1` on 2026-09-03, itself bumped from `0.1.3` on
> 2026-08-15). Every move in that chain needed no config or code change here: the wire protocol is
> **Library version.** This repo pins **`ZB.MOM.WW.LocalDb` / `.Replication` 0.3.7**
> (`Directory.Packages.props`; bumped from `0.3.4` on 2026-11-12 under scadaproj#53, from `0.2.1`
> on 2026-09-03, and from `0.1.3` on 2026-08-15). 0.3.5–0.3.7 are wire-compatible too; what they
> change for an operator is the ApiKey rule and the tombstone rule below. Every move in that chain needed no config or code change here: the wire protocol is
> unchanged end to end (0.1.x/0.2.x/0.3.x peers all still sync), and the on-disk bookkeeping schema
> upgrades in place on open (now v3) with no operator action on an existing node's database file.
> Behaviours worth knowing at the current pin: batching is byte-budgeted (the two rows above);
@@ -133,8 +134,10 @@ site-a-2: LocalDb__SyncListenPort=9001
### The ApiKey rule (fail-closed)
The replication library's passive endpoint verifies **no** authentication — the host
`LocalDbSyncAuthInterceptor` is the only gate, and it is **fail-closed**:
Two gates check the same key. Since LocalDb 0.3.6 (scadaproj#36) the library's passive
`LocalDbSyncService` enforces `LocalDb:Replication:ApiKey` itself (Bearer, constant-time,
`Unauthenticated` on a mismatch). The host `LocalDbSyncAuthInterceptor` runs first and is kept for
its stricter rule — it is **fail-closed**:
- **No key configured ⇒ every sync call is refused** (`PermissionDenied`). "No key" is never
"no auth required".
@@ -148,11 +151,16 @@ The replication library's passive endpoint verifies **no** authentication — th
while its peer is down; the outage is not a data-loss event — the surviving node accumulates
writes and the peer catches up on rejoin. But a long-lived solo node drifts further from its
peer, so avoid leaving a pair split for extended periods.
- **Tombstone-retention resurrection window.** Retention prunes to the newest 2 deployments and
replicates the prune as tombstones. Tombstones are themselves retained only for a bounded window.
If a node is offline **longer than the tombstone-retention window**, a delete that happened during
its outage may no longer be expressible as a tombstone on rejoin — a pruned deployment could
briefly reappear until the next deploy re-prunes it. Keep pair outages well inside that window.
- **Tombstones are held until the peer has them (LocalDb 0.3.7, scadaproj#16).** Retention prunes
to the newest 2 deployments and replicates the prune as tombstones. Through 0.3.6 a tombstone was
dropped after `LocalDb:Replication:TombstoneRetention` (7 days) whether or not the peer had it, so
a node offline longer than that could bring a pruned deployment back on rejoin. As of 0.3.7 a
tombstone is pruned only when it is past retention **and** delivered (retention restarts at
delivery), so a rejoin after any length of outage is safe. **The cost is ledger growth while the
peer is down:** the survivor keeps every undelivered tombstone (deployment prunes, drained
`alarm_sf_events` rows) until the partner returns. Optional `LocalDb:Replication:TombstoneHardCap`
(default unset = off, must be ≥ `TombstoneRetention`) bounds it, and past the cap the old risk
returns. A node restored from an old backup of its own file is still not covered.
- **A rebuilt node is back-filled by its peer.** If a node loses its LocalDb file — a wiped volume,
a re-imaged host, a fresh container — it rejoins with an empty database and its peer snapshots the
cached configuration back to it, with no new deploy and no central SQL. Two things are worth