chore(deps): LocalDb 0.3.4 -> 0.3.7 (#889, scadaproj#53)
v2-ci / unit-tests (push) Waiting to run
v2-ci / unit-tests (push) Waiting to run
Pins ZB.MOM.WW.LocalDb and .Replication at 0.3.7 (published 2026-11-12). Wire-compatible; no code change needed. - 0.3.6 (scadaproj#36): the library sync service enforces LocalDb:Replication:ApiKey itself. LocalDbSyncAuthInterceptor is kept for its refuse-every-stream-when-blank rule; doc comment updated. docker-dev site-a pair: both nodes now read one variable, OTOPCUA_SITE_A_LOCALDB_SYNC_KEY (same dev default as before), so an override can never split the pair. - 0.3.7 (scadaproj#16): a tombstone is held until the peer has it, so a node offline past TombstoneRetention no longer brings a pruned deployment back. The ledger grows while a peer is down; optional TombstoneHardCap (documented in Configuration.md) bounds it. Runbook, Configuration.md and the cluster-topology rule updated. Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
@@ -309,8 +309,12 @@ same event accepted on both nodes — which happens in every boot window, since
|
||||
docker-dev rig the site-a pair and site-b both run `AlarmHistorian__Enabled=true` against a
|
||||
deliberately unresolvable gateway endpoint, so the buffer is always a live, non-draining queue.
|
||||
|
||||
**The library pin is `0.3.4`** (`Directory.Packages.props`, both `ZB.MOM.WW.LocalDb` and
|
||||
`.Replication`), bumped from `0.2.1` on 2026-09-03 (was itself bumped from `0.1.3` on 2026-08-15).
|
||||
**The library pin is `0.3.7`** (`Directory.Packages.props`, both `ZB.MOM.WW.LocalDb` and
|
||||
`.Replication`; bumped from `0.3.4` on 2026-11-12, scadaproj#53 — the library's sync service now
|
||||
enforces `LocalDb:Replication:ApiKey` itself (0.3.6, scadaproj#36; `LocalDbSyncAuthInterceptor` kept
|
||||
for its refuse-when-blank rule), `RegisterReplicated` refuses `AUTOINCREMENT` keys (none here), and
|
||||
a tombstone is held until the peer has it (0.3.7, scadaproj#16 — no resurrection after a long outage,
|
||||
ledger grows while the peer is down; optional `TombstoneHardCap`)). Earlier: bumped from `0.2.1` on 2026-09-03 (was itself bumped from `0.1.3` on 2026-08-15).
|
||||
The 0.2.1→0.3.4 bump is wire-compatible with 0.2.x, and the on-disk bookkeeping schema v2 → v3
|
||||
upgrades in place on open, so no rig or volume action was needed. `RegisterReplicated(string)` is
|
||||
unchanged and everything 0.2.x/0.3.x added is additive, **except** that `RegisterReplicated` (0.3.3)
|
||||
@@ -358,7 +362,7 @@ things are worth knowing:
|
||||
used to replicate only writes made after the restart. It seeds the ledger at the LWW floor
|
||||
(HLC `0`, this node's id) and flags a resync — a baselined row loses to any genuine remote
|
||||
write of the same key and wins only where the peer has no version at all. It is idempotent, so
|
||||
it is free on every boot after the first. As of `0.3.0` (this repo pins `0.3.4`) the snapshot
|
||||
it is free on every boot after the first. As of `0.3.0` (this repo pins `0.3.7`) the snapshot
|
||||
flag is **per-table**, not per-database — baselining one table no longer streams every
|
||||
registered table in both directions. (Prior to `0.3.0`, on the `0.2.1` pin this repo carried
|
||||
until 2026-09-03, the flag was per-database and baselining anything streamed *every*
|
||||
|
||||
@@ -224,8 +224,8 @@
|
||||
package floors the native SQLitePCLRaw.lib.e_sqlite3 at 2.1.12, so consumers inherit the
|
||||
CVE-2025-6965-fixed native binary without needing the surgical pin below.
|
||||
-->
|
||||
<PackageVersion Include="ZB.MOM.WW.LocalDb" Version="0.3.4" />
|
||||
<PackageVersion Include="ZB.MOM.WW.LocalDb.Replication" Version="0.3.4" />
|
||||
<PackageVersion Include="ZB.MOM.WW.LocalDb" Version="0.3.7" />
|
||||
<PackageVersion Include="ZB.MOM.WW.LocalDb.Replication" Version="0.3.7" />
|
||||
<PackageVersion Include="ZB.MOM.WW.Health.Akka" Version="0.3.0" />
|
||||
<PackageVersion Include="ZB.MOM.WW.Health.EntityFrameworkCore" Version="0.3.0" />
|
||||
<PackageVersion Include="ZB.MOM.WW.Telemetry" Version="0.1.0" />
|
||||
|
||||
@@ -729,7 +729,9 @@ services:
|
||||
# DEV-ONLY committed key (rig philosophy — like the SQL password + secrets KEK above). It
|
||||
# MUST be byte-identical on both nodes: the interceptor is fail-closed, so any mismatch
|
||||
# silently stops the pair converging. NEVER reuse this key outside this local dev rig.
|
||||
LocalDb__Replication__ApiKey: "dev-site-a-localdb-sync-key"
|
||||
# Both site-a nodes read the SAME variable, so an override can never split the pair; since
|
||||
# LocalDb 0.3.6 the library's sync service checks this key too (scadaproj#36).
|
||||
LocalDb__Replication__ApiKey: "${OTOPCUA_SITE_A_LOCALDB_SYNC_KEY:-dev-site-a-localdb-sync-key}"
|
||||
# Since LocalDb 0.2.0 the WIRE message is bounded by MaxBatchBytes (2 MB default, left
|
||||
# unset), not by this. MaxBatchSize is now the secondary row cap AND the DB read-page size,
|
||||
# and the read page is materialised in memory before the byte-budget split runs — so 16 is
|
||||
@@ -838,7 +840,7 @@ services:
|
||||
ServerHistorian__UseTls: "false"
|
||||
LocalDb__Path: "/app/data/otopcua-localdb.db"
|
||||
LocalDb__SyncListenPort: "9001"
|
||||
LocalDb__Replication__ApiKey: "dev-site-a-localdb-sync-key"
|
||||
LocalDb__Replication__ApiKey: "${OTOPCUA_SITE_A_LOCALDB_SYNC_KEY:-dev-site-a-localdb-sync-key}"
|
||||
LocalDb__Replication__MaxBatchSize: "16"
|
||||
ports:
|
||||
- "4843:4840"
|
||||
|
||||
@@ -494,7 +494,8 @@ replication stays inert until the sync port + peer are set.
|
||||
| `LocalDb:Path` | `./data/otopcua-localdb.db` | SQLite file path. `ValidateOnStart`-required once `AddZbLocalDb` runs (driver nodes only). In containers put it on a durable volume. |
|
||||
| `LocalDb:SyncListenPort` | `0` | `0` = replication off (no sync listener). A non-zero port binds a dedicated **h2c** sync listener; set the **same** port on both pair nodes. Setting it makes the Host add an explicit Kestrel `Listen*` and re-bind the primary HTTP port (an explicit `Listen*` otherwise discards `ASPNETCORE_URLS`). |
|
||||
| `LocalDb:Replication:PeerAddress` | `""` | The peer this node **dials** (`http://<peer>:<port>`). Set on the initiator only; leave empty on the passive node. The stream is bidirectional. |
|
||||
| `LocalDb:Replication:ApiKey` | `""` | Bearer token for the sync stream. **Fail-closed:** no key ⇒ the passive endpoint refuses everything; a mismatch ⇒ the pair silently stops converging. **Must be byte-identical on both nodes.** Supply via `${secret:...}` / env — never a cleartext literal in production. |
|
||||
| `LocalDb:Replication:ApiKey` | `""` | Bearer token for the sync stream. **Fail-closed:** no key ⇒ the passive endpoint refuses everything; a mismatch ⇒ the pair silently stops converging. **Must be byte-identical on both nodes.** Enforced by the host interceptor and, since LocalDb 0.3.6, by the library's sync service too. Supply via `${secret:...}` / env — never a cleartext literal in production. |
|
||||
| `LocalDb:Replication:TombstoneHardCap` | unset (off) | LocalDb 0.3.7. A tombstone is kept until it is past `TombstoneRetention` (7 days) **and** the peer holds it, so the ledger grows while a peer is down. This optional cap (≥ `TombstoneRetention`) bounds an undelivered tombstone's life; past it a returning node can resurrect deleted rows again. Leave unset unless disk is the constraint. |
|
||||
| `LocalDb:Replication:MaxBatchSize` | `500` (library) | **Secondary** row-count cap on a batch — *and* the DB read-page size. `16` on the replicating pair. Since LocalDb 0.2.0 the wire message is bounded by `MaxBatchBytes`, not by this, so it is no longer the hand-tuned byte proxy the 4 MB cap once forced. It still matters because `OplogStore.ReadBatchAboveAsync`/`SnapshotStreamer` materialise a whole read page **before** the byte-budget split runs — `16` bounds that transient allocation at ≈2.7 MB for ≈171 KB artifact chunk rows. |
|
||||
| `LocalDb:Replication:MaxBatchBytes` | `2 MB` (library) | **Primary** cap on one outbound delta/snapshot message: the summed serialized size of its entries, sized under gRPC's 4 MB receive limit. Left unset across the fleet. A single row larger than the budget is sent **alone** rather than stalling the stream. Added in LocalDb 0.2.0; needs no wire change, so a node on this version interoperates with any earlier peer in both directions. |
|
||||
|
||||
@@ -510,7 +511,7 @@ turning replication on converges the rows **already in the file** and not merely
|
||||
unconfigured node **deregisters** them at boot, dropping capture triggers an earlier build installed. ⚠️ The
|
||||
sync handshake compares registered-table digests **fail-closed**, so flipping either key is a
|
||||
both-nodes-together change: set it on both and restart both. As of LocalDb 0.3.0 (this repo pins
|
||||
0.3.4) baselining flags a **per-table** snapshot resync — enabling replication on one table no
|
||||
0.3.7) baselining flags a **per-table** snapshot resync — enabling replication on one table no
|
||||
longer forces a full-database resync of every registered table in both directions.
|
||||
|
||||
Replication is **default-OFF** across the fleet; it is enabled per-pair as an opt-in. See
|
||||
|
||||
@@ -77,9 +77,10 @@ Replication has two knobs, both under `LocalDb`:
|
||||
| `LocalDb:Replication:MaxBatchSize` | secondary row cap **and** the DB read-page size | `16` for the artifact cache. **Since LocalDb 0.2.0 this no longer bounds the wire message** — `MaxBatchBytes` does — so it is not the byte proxy the note here used to describe. Keep it at `16` anyway: `OplogStore`/`SnapshotStreamer` materialise a whole read page in memory *before* the byte-budget split runs, and 16 × ≈171 KB artifact chunk rows ≈ 2.7 MB bounds that transient allocation. Raising it no longer risks tripping the gRPC cap, only the allocation. |
|
||||
| `LocalDb:Replication:MaxBatchBytes` | primary cap on one outbound message (bytes) | Library default **2 MB**, sized under gRPC's 4 MB receive limit. Left unset on this rig. A single row over budget is sent alone rather than stalling the stream. |
|
||||
|
||||
> **Library version.** This repo pins **`ZB.MOM.WW.LocalDb` / `.Replication` 0.3.4**
|
||||
> (`Directory.Packages.props`; bumped from `0.2.1` on 2026-09-03, itself bumped from `0.1.3` on
|
||||
> 2026-08-15). Every move in that chain needed no config or code change here: the wire protocol is
|
||||
> **Library version.** This repo pins **`ZB.MOM.WW.LocalDb` / `.Replication` 0.3.7**
|
||||
> (`Directory.Packages.props`; bumped from `0.3.4` on 2026-11-12 under scadaproj#53, from `0.2.1`
|
||||
> on 2026-09-03, and from `0.1.3` on 2026-08-15). 0.3.5–0.3.7 are wire-compatible too; what they
|
||||
> change for an operator is the ApiKey rule and the tombstone rule below. Every move in that chain needed no config or code change here: the wire protocol is
|
||||
> unchanged end to end (0.1.x/0.2.x/0.3.x peers all still sync), and the on-disk bookkeeping schema
|
||||
> upgrades in place on open (now v3) with no operator action on an existing node's database file.
|
||||
> Behaviours worth knowing at the current pin: batching is byte-budgeted (the two rows above);
|
||||
@@ -133,8 +134,10 @@ site-a-2: LocalDb__SyncListenPort=9001
|
||||
|
||||
### The ApiKey rule (fail-closed)
|
||||
|
||||
The replication library's passive endpoint verifies **no** authentication — the host
|
||||
`LocalDbSyncAuthInterceptor` is the only gate, and it is **fail-closed**:
|
||||
Two gates check the same key. Since LocalDb 0.3.6 (scadaproj#36) the library's passive
|
||||
`LocalDbSyncService` enforces `LocalDb:Replication:ApiKey` itself (Bearer, constant-time,
|
||||
`Unauthenticated` on a mismatch). The host `LocalDbSyncAuthInterceptor` runs first and is kept for
|
||||
its stricter rule — it is **fail-closed**:
|
||||
|
||||
- **No key configured ⇒ every sync call is refused** (`PermissionDenied`). "No key" is never
|
||||
"no auth required".
|
||||
@@ -148,11 +151,16 @@ The replication library's passive endpoint verifies **no** authentication — th
|
||||
while its peer is down; the outage is not a data-loss event — the surviving node accumulates
|
||||
writes and the peer catches up on rejoin. But a long-lived solo node drifts further from its
|
||||
peer, so avoid leaving a pair split for extended periods.
|
||||
- **Tombstone-retention resurrection window.** Retention prunes to the newest 2 deployments and
|
||||
replicates the prune as tombstones. Tombstones are themselves retained only for a bounded window.
|
||||
If a node is offline **longer than the tombstone-retention window**, a delete that happened during
|
||||
its outage may no longer be expressible as a tombstone on rejoin — a pruned deployment could
|
||||
briefly reappear until the next deploy re-prunes it. Keep pair outages well inside that window.
|
||||
- **Tombstones are held until the peer has them (LocalDb 0.3.7, scadaproj#16).** Retention prunes
|
||||
to the newest 2 deployments and replicates the prune as tombstones. Through 0.3.6 a tombstone was
|
||||
dropped after `LocalDb:Replication:TombstoneRetention` (7 days) whether or not the peer had it, so
|
||||
a node offline longer than that could bring a pruned deployment back on rejoin. As of 0.3.7 a
|
||||
tombstone is pruned only when it is past retention **and** delivered (retention restarts at
|
||||
delivery), so a rejoin after any length of outage is safe. **The cost is ledger growth while the
|
||||
peer is down:** the survivor keeps every undelivered tombstone (deployment prunes, drained
|
||||
`alarm_sf_events` rows) until the partner returns. Optional `LocalDb:Replication:TombstoneHardCap`
|
||||
(default unset = off, must be ≥ `TombstoneRetention`) bounds it, and past the cap the old risk
|
||||
returns. A node restored from an old backup of its own file is still not covered.
|
||||
- **A rebuilt node is back-filled by its peer.** If a node loses its LocalDb file — a wiped volume,
|
||||
a re-imaged host, a fresh container — it rejoins with an empty database and its peer snapshots the
|
||||
cached configuration back to it, with no new deploy and no central SQL. Two things are worth
|
||||
|
||||
@@ -9,10 +9,12 @@ using ZB.MOM.WW.LocalDb.Replication;
|
||||
namespace ZB.MOM.WW.OtOpcUa.Host.Configuration;
|
||||
|
||||
/// <summary>
|
||||
/// Gates the LocalDb passive sync endpoint. The replication library deliberately leaves inbound
|
||||
/// authentication to the host — its <c>LocalDbSyncService</c> verifies nothing — so without this
|
||||
/// interceptor anything that can reach a driver node's sync port could stream arbitrary rows
|
||||
/// into that node's deployment-artifact cache.
|
||||
/// Gates the LocalDb passive sync endpoint. Through LocalDb 0.3.5 the replication library left
|
||||
/// inbound authentication to the host — its <c>LocalDbSyncService</c> verified nothing — so
|
||||
/// without this interceptor anything that could reach a driver node's sync port could stream
|
||||
/// arbitrary rows into that node's deployment-artifact cache. Since 0.3.6 (scadaproj#36) the
|
||||
/// library checks the same key and header itself; this interceptor is kept for the one rule the
|
||||
/// library does not apply — refusing every stream when no key is configured.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
|
||||
Reference in New Issue
Block a user