413bc5c04fb3398b93d231d4e00932d763ef718e
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joseph Doherty
|
b8b4c18f54 |
chore(deps): bump Auth 0.2.1->0.3.0, LocalDb 0.2.1->0.3.4, MxGateway.Client/Contracts 0.2.0->0.2.1
Additive/wire-compatible releases across all three families; Directory.Packages.props is the only pin surface (no csproj-level overrides, no lock files). Auth 0.3.0 adds IApiKeyStoreMigrator and the optional Auth.ApiKeys.SqlServer package, neither needed here. LocalDb 0.3.4's on-disk bookkeeping schema goes v2->v3 (upgrades in place on open); 0.3.0 made the snapshot-resync flag per-table instead of per-database; 0.3.2 added validated Replication:KeepAlivePingDelay/KeepAlivePingTimeout config (defaults 60s/20s, left unset here) and a public LocalDbOplogGapException; 0.3.3 makes RegisterReplicated throw LocalDbRegistrationException for a table that is the parent of a row-mutating ON DELETE FK, which does not apply to this repo's four replicated tables (no FKs). MxGateway.Client/Contracts 0.2.1 publishes the alarm snapshot_status truncation frame and live alarm watch-list refresh; vendored the new nupkgs into nuget-packages/ (the local-mxgw source MxGateway.* is exclusively package-source-mapped to), matching the existing pattern from the 0.1.0->0.1.1 and 0.1.1->0.2.0 bumps. Full solution builds clean (0 errors, 908 pre-existing analyzer warnings, unchanged count). Full test suite: one failure, ClusterRoleInfoRoleLeaderCaseTests. RoleLeaderChanged_for_driver_then_Driver_keeps_distinct_entries, reproduces identically on the unbumped 0.2.1/0.2.1/0.2.0 tree under the same full-suite parallel run (and passes standalone in isolation in 2s on the bumped tree) - a pre-existing timing/parallelism flake, not a bump regression, so left as-is. Updated CLAUDE.md and two docs (Configuration.md, operations/2026-07-20-localdb-pair- replication.md) where they stated the old pin numbers and the now-superseded per-database snapshot-resync behavior as current fact; dated historical records (archreview snapshots, dated plan docs) were left untouched. Claude-Session: https://claude.ai/code/session_01Wzg6NUCWHipSexLs2cgRFv |
||
|
Joseph Doherty
|
3e62f1617b |
fix(localdb): registration follows replication config — baseline on enable, deregister self-heal on disable (pair-symmetric)
LocalDbSetup.OnReady registered all four replicated tables unconditionally on
every driver node, replicating or not. That was not merely wasteful (an
unreplicated node paid two INSERTs plus a json_object of the whole row inside
every write transaction, for an oplog nothing ever drained) — it made
baselineExistingRows unusable, so enabling replication on a long-running
default-OFF node converged on SUBSEQUENT WRITES ONLY and the rows already in the
file stayed invisible to the peer, silently and forever.
Registration now follows one predicate, LocalDbSetup.ReplicationIsConfigured:
LocalDb:Replication:PeerAddress OR :ApiKey non-blank. The OR is load-bearing —
only the dialling half of a pair sets PeerAddress, so keying on it alone would
strip capture from every passive node and the pair would converge in one
direction only. SyncListenPort deliberately does not join it: the passive
endpoint is fail-closed on the key, so a port with no ApiKey syncs with nobody.
configured => RegisterReplicated(table, baselineExistingRows: true), per
table, in the unchanged load-bearing DDL->register order
unconfigured => DeregisterReplicated(table), an idempotent boot self-heal that
drops the capture triggers an earlier build left behind and
prunes that table's oplog/row-version rows. One Information
line, only when it actually cleaned something.
PAIR SYMMETRY: the sync handshake compares registered-table digests fail-closed,
so flipping either key is a both-nodes-together change — set it on both and
restart both. Documented in CLAUDE.md, docs/Configuration.md and the replication
runbook (which claimed "this repo never calls DeregisterReplicated").
Verified against the pinned LocalDb 0.2.1: RegisterReplicated(string, bool) and
DeregisterReplicated(string) -> bool both exist; baselining seeds
__localdb_row_version at HLC 0 and the deregister return value distinguishes
"cleaned" from "nothing to clean". No pin bump needed.
Tests: LocalDbSetupTests gains pins for both branches (configured via
PeerAddress alone and via ApiKey alone; unconfigured => no tables, zero
triggers; late-enable => an HLC-0 baseline ledger entry; the log line fires once
and only when something was cleaned) — four went red against the old
unconditional code, and all four go red again under a constant-true predicate.
LocalDbWiringTests splits its exact-set pin into a replicating graph and a
default-OFF one. LocalDbPairHarness and AlarmSfLegacyMigratorTests built their
database from LocalDb:Path alone while configuring the replication engine from a
second config — production reads one configuration, so the two are now aligned.
Host.IntegrationTests: 224 passed, 0 failed, 5 skipped.
|
||
|
Joseph Doherty
|
a1a258d85d |
chore(deps): bump ZB.MOM.WW.LocalDb 0.1.3 -> 0.2.1, correct the MaxBatchSize rationale
Closes the pin skew against ScadaBridge (which already pins 0.2.1). Pure pin
bump — no code change was required:
- `RegisterReplicated(string)` still delegates to `baselineExistingRows: false`,
so `LocalDbSetup.OnReady` is byte-identical in behaviour.
- Everything 0.2.x added is additive (`DeregisterReplicated`, `MaxBatchBytes`,
the `baselineExistingRows` overload); none of it is called here.
- The wire protocol is unchanged (0.1.x peers still sync) and the on-disk
bookkeeping schema v1 -> v2 upgrades in place on open, so no rig, volume or
operator action is needed.
The one thing that DID need reconciling is documentation. 0.2.0 made batching
byte-budgeted: `MaxBatchBytes` (2 MB default, under the 4 MB gRPC cap) now
bounds the wire message, and `MaxBatchSize` is demoted to a secondary row cap
plus the DB read-page size. The rig's `MaxBatchSize = 16` was chosen as a
hand-computed byte proxy ("row-count-only against gRPC's 4 MB cap"), and that
rationale is now wrong everywhere it appeared — appsettings, Configuration.md,
the pair-replication runbook, and both docker-dev site-a services.
The VALUE stays 16, for a different and still-real reason: OplogStore /
SnapshotStreamer materialise a whole read page in memory *before* the
byte-budget split runs, so 16 x ~171 KB artifact chunk rows bounds that
transient allocation at ~2.7 MB. Documented as such, with `MaxBatchBytes`
added to both config tables.
CLAUDE.md records the pin and two deliberate non-adoptions: this repo does not
take ScadaBridge's conditional-registration + boot-time `DeregisterReplicated`
self-heal (OnReady registers all four tables unconditionally, and
LocalDbSetupTests pins that set), and consequently does not use
`baselineExistingRows` either — so enabling replication on a long-running
default-OFF node converges on subsequent writes, not on rows already in the
file. Both are behaviour changes, out of scope for a pin bump.
Verification: solution builds clean; full suite run (57 projects). All 66
LocalDb tests in Host.IntegrationTests green. One unrelated failure,
RoslynVirtualTagEvaluatorTests.Evaluate_racing_ClearCompiledScripts_never_
fails_with_disposed — a Roslyn evaluator disposal race with no LocalDb in its
path, whose own comment documents that its "at most one clear per evaluation"
assumption depends on thread-pool pressure. It passed 3/3 re-runs in isolation
and fails only under full-solution parallel load; pre-existing test-design
fragility, not a regression from this bump.
|
||
|
Joseph Doherty
|
f76d1f91e5 |
docs+chore(localdb): phase-2 rig config + docs
Enables the alarm store-and-forward sink on all four driver nodes of the docker-dev rig -- the replicating site-a pair and the default-OFF site-b pin -- so the live gate has a real buffer to watch converge, and can see that site-b's sink works as a plain node-local queue with no peer traffic. Two rig details worth stating rather than rediscovering. The endpoint is deliberately unresolvable: there is no HistorianGateway here, so every drain attempt fails, which is exactly the historian-outage state the buffer exists for. But it still has to be a syntactically valid absolute http(s) URI or the host refuses to start, because ServerHistorianOptionsValidator is consumer-gated -- AlarmHistorian:Enabled=true makes the endpoint required even while ServerHistorian:Enabled=false. And MaxAttempts is raised far above the production default of 10, which against a permanently unreachable gateway would dead-letter the entire queue about five minutes in, turning a buffering test into a dead-letter test. Docs record what an operator now has to know: that a rising queue depth on a Secondary is correct and on BOTH nodes is not (that shape is a redundancy snapshot naming neither node -- check node identity before suspecting the sink), that delivery is at-least-once across a failover by design with no dedup layer, and that AlarmHistorian:DatabasePath is removed but should be left in place through the upgrade because the migrator still reads it to find the file to copy. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
f587e7972e |
docs(localdb): live-gate the two Phase 1 follow-ups; pin LocalDb 0.1.3
The gate closed both follow-ups on the docker-dev rig and found a third defect that offline tests could not have found first: it lived behind the one being fixed. Once 0.1.2 made back-fill work, a rebuilt node could be observed writing for the first time — and its writes went nowhere, because the healthy node kept the old peer's seq watermark. Library 0.1.1 -> 0.1.2 -> 0.1.3 over the course of the gate; both consumers now pinned to 0.1.3. Check 4 (was PARTIAL, now PASS): with SQL stopped and site-a-2's LocalDb volume destroyed, a-1 logged "Snapshot sent (as_of_seq 0, 4 rows)" — a line that could not appear on 0.1.1 — and a-2 came back with a row_version dump byte-identical to a-1's, carrying a-1's origin node ids and its ORIGINAL applied_at_utc. It then booted from cache and served 17 ns=2 nodes, diff-identical to its peer, from a configuration it never applied and could not have fetched. Check 8 (PASS): two restarts with an unchanged artifact left the oplog at 7 and the pointer timestamp frozen — the re-cache was skipped outright. Positive control: a real config change still wrote, 7 -> 10 -> 13. Also records one unrelated pre-existing finding the gate's SQL flapping surfaced: a transient ConfigDb error during artifact load empties the served address space while logging "rebuild becomes no-op". The cache layer correctly refused to store the bad artifact and the node recovered on restart, but the two logs disagree and it is the same class as the Phase 1 gate's check-3 defect. Untouched by this work; wants its own issue. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
232bff5df7 |
fix(localdb): close both Phase 1 live-gate follow-ups
1. Wiped-node back-fill (live gate check 4). Fixed upstream in ZB.MOM.WW.LocalDb 0.1.2 and pinned here. The gate's diagnosis was slightly off: the oplog cap was not the gate. Snapshot detection measured the peer's gap against the oldest surviving oplog row and read an empty oplog as "no gap possible" — and an empty oplog is the steady state of a converged pair, since ack-pruning deletes everything the peer confirmed. The healthy state was the one state that could not heal a wiped peer. Now measured against last_acked_seq when the oplog is empty. Pinned at this level too, not just the library's: the pair harness grows a WipePassiveAsync (a NEW database — a rebuilt node comes back with a new node id and a zero watermark), and the new scenario asserts the emptied oplog as its precondition before wiping, then requires the deployment artifact to come back byte-identical with no new deploy. Verified RED against the pinned 0.1.1 (times out waiting for back-fill) and green on 0.1.2. 2. Oplog growth on default-OFF nodes (live gate check 8), fixed at the source: StoreAsync now skips entirely when the pointer already names this deployment/revision, the SHA matches the bytes, and the expected chunk count is present. Re-caching an artifact the node already holds — every restart's boot-from-cache, every RestoreApplied — writes nothing and mints no oplog rows, where before it cost a delete plus an insert per chunk plus a pointer update, all identical to what was already there. Identity is over the bytes and the chunks are counted, both deliberately: a re-composed artifact can carry the same ids with different bytes, and a pointer can name a deployment whose chunks are missing, where skipping would make an unreadable cache permanent. Both guards verified RED against a naive pointer-only skip. The gate's stated bound was also wrong and is corrected in the doc: growth was never headed for the 1M row cap. AddZbLocalDbReplication is registered unconditionally, so MaintenanceBackgroundService runs on default-OFF nodes and the 7-day MaxOplogAge cap prunes. Runtime.Tests 412/0/31, Host.IntegrationTests LocalDb 45/45. Pre-existing and unrelated: AbCip_Green_AgainstSim (needs the AB CIP docker fixture, red on a clean master too) and a flaky Roslyn race test (green 2/2 in isolation). Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
c834d89ae7 |
docs(localdb): phase-1 runbook + redundancy/config docs
New operations runbook (enable/disable, fail-closed ApiKey rule, stop/start- together, tombstone-retention window, MaxBatchSize row-count-vs-4MB, never sqlite3-a-live-WAL-DB cp-triplet recipe). Configuration.md gains a LocalDb section; Redundancy.md gains a pair-local config cache section (what boot-from- cache does and does not cover); CLAUDE.md gains the Phase 1 scope paragraph. |
||
|
Joseph Doherty
|
ce383df39a |
docs(secrets): G-5 clustered master-key posture runbook (Task 6)
OtOpcUa is Akka-clustered and AddZbSecrets is registered unconditionally, so every node (admin, driver, fused) resolves secrets and needs the SAME KEK + same store rows. Ship G-5 as a production-posture runbook rather than hardcoding Source=File + shared paths into the committed role-overlay appsettings (which dev + TwoNodeClusterHarness also consume — that would break every dev/CI boot). Base appsettings stays on Source=Environment. Documents the interim File-KEK + shared-SQLite posture and the G-7 hand-off (ConfigDb-backed ISecretStore mirroring the existing DP PersistKeysToDbContext<OtOpcUaConfigDbContext> key-ring sharing). Cross-linked from docs/security.md. Mirrors the ScadaBridge G-5 resolution. |