dohertj2/scadaproj
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(localdb): snapshot as_of_seq floors at last_acked_seq so a rebuilt peer's watermark is lifted — ends the full-snapshot-on-every-reconnect loop (0.3.4, ScadaBridge register row 106)

Browse Source 
SnapshotStreamer.SendAsync stamped as_of_seq = MAX(seq), which reads 0 once ack-pruning has
emptied a converged sender's oplog. A receiver whose watermark genuinely is 0 (a rebuilt node —
ScadaBridge's site-a-a since its 2026-08-15 rebuild) was healed by the snapshot but never lifted
by it (the receiver's watermark update is correctly MAX-guarded), so the 0.1.2 empty-oplog
horizon rule saw 0 < last_acked_seq as a gap on every subsequent handshake and re-streamed the
whole database on every reconnect until the sender wrote again. Observed live: the same 127k-row
full snapshot on two consecutive restarts of the pair.

Fix: as_of_seq = max(MAX(seq), last_acked_seq). Every delta pruned below last_acked_seq is
already folded into the rows the snapshot carries, so that is the floor it may claim; the
receiver's MAX guard then lifts the watermark and the next handshake finds no gap. Sender-side
only; wire-compatible; no schema change.

Test WipedPeer_SnapshotLiftsItsWatermark_SoTheNextReconnectSendsNoSecondSnapshot fails on the
unfixed code exactly as the rig does (watermark 0, expected 3) and passes with the fix: the
second episode exchanges only a delta, and a capturing logger on the snapshot streamer proves
no snapshot was sent. Suite 216/216 on 8 of 9 runs; one unnamed single-test failure on the first
run did not recur in eight further runs and could not be attributed.
This commit is contained in:
Joseph Doherty
2026-08-20 09:52:19 -04:00
parent 5e02c1c38f
commit 75c1897522
4 changed files with 93 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ZB.MOM.WW.LocalDb/README.md
+12 -1
View File
@@ -10,7 +10,7 @@ Change capture uses SQLite triggers writing a single HLC-stamped oplog inside th
consumer's own write transaction (crash-consistent, non-bypassable). Convergence is
last-writer-wins over a hybrid logical clock. All timestamps are UTC.
## Packages (all `0.3.3`)
## Packages (all `0.3.4`)
| Package | Contents |
|---|---|
@@ -20,6 +20,17 @@ last-writer-wins over a hybrid logical clock. All timestamps are UTC.
> Status: **published to the Gitea feed; adopted by ScadaBridge (Phases 1+2) and OtOpcUa (Phase 1).**
>
> - `0.3.4` (2026-08-20) — **a converged pair no longer re-streams a full snapshot on every
> reconnect once the sender's oplog has been ack-pruned to empty.** Found on the ScadaBridge
> docker rig (its register row 106): a rebuilt node (watermark 0) was healed by a snapshot
> stamped `as_of_seq = MAX(seq)` — which is 0 on an empty oplog — so its MAX-guarded watermark
> stayed 0, and the healthy side's empty-oplog horizon rule (`0.1.2`: streamable from
> `last_acked_seq + 1`) saw a gap on every later handshake and re-sent the whole database
> (127k rows per restart on the rig) until it happened to write a new row. The as-of is now
> `max(MAX(seq), last_acked_seq)`: every delta pruned below `last_acked_seq` is already folded
> into the rows the snapshot carries, so that is the floor it may claim. Sender-side only,
> wire-compatible, no schema change; pinned by
> `WipedPeer_SnapshotLiftsItsWatermark_SoTheNextReconnectSendsNoSecondSnapshot`.
> - `0.3.3` (2026-08-20) — two latent apply-path hazards from the ScadaBridge adversarial
> architecture review (P4-7). Neither had a live trigger in any current consumer; both are the
> kind that surface as silent divergence, not as an error. Wire-compatible: no proto change, no