fix(localdb): snapshot as_of_seq floors at last_acked_seq so a rebuilt peer's watermark is lifted — ends the full-snapshot-on-every-reconnect loop (0.3.4, ScadaBridge register row 106)

SnapshotStreamer.SendAsync stamped as_of_seq = MAX(seq), which reads 0 once ack-pruning has emptied a converged sender's oplog. A receiver whose watermark genuinely is 0 (a rebuilt node — ScadaBridge's site-a-a since its 2026-08-15 rebuild) was healed by the snapshot but never lifted by it (the receiver's watermark update is correctly MAX-guarded), so the 0.1.2 empty-oplog horizon rule saw 0 < last_acked_seq as a gap on every subsequent handshake and re-streamed the whole database on every reconnect until the sender wrote again. Observed live: the same 127k-row full snapshot on two consecutive restarts of the pair. Fix: as_of_seq = max(MAX(seq), last_acked_seq). Every delta pruned below last_acked_seq is already folded into the rows the snapshot carries, so that is the floor it may claim; the receiver's MAX guard then lifts the watermark and the next handshake finds no gap. Sender-side only; wire-compatible; no schema change. Test WipedPeer_SnapshotLiftsItsWatermark_SoTheNextReconnectSendsNoSecondSnapshot fails on the unfixed code exactly as the rig does (watermark 0, expected 3) and passes with the fix: the second episode exchanges only a delta, and a capturing logger on the snapshot streamer proves no snapshot was sent. Suite 216/216 on 8 of 9 runs; one unnamed single-test failure on the first run did not recur in eight further runs and could not be attributed.
2026-08-20 09:52:19 -04:00
parent 5e02c1c38f
commit 75c1897522
4 changed files with 93 additions and 5 deletions
@@ -61,7 +61,16 @@ internal sealed class SnapshotStreamer(SqliteLocalDb db, OplogStore store, Repli
    /// </summary>
    public async Task<long> SendAsync(SnapshotScope scope, Func<SyncMessage, CancellationToken, Task> send, CancellationToken ct)
    {
-        var asOfSeq = await store.GetMaxSeqAsync(ct);
+        // The as-of is the highest seq this snapshot can stand in for. MAX(seq) alone is wrong once
+        // ack-pruning has emptied the oplog: it reads 0, and a receiver whose watermark genuinely IS
+        // 0 (a rebuilt node) is then never lifted by the snapshot — its MAX-guarded watermark stays
+        // 0, the empty-oplog horizon rule (streamable from last_acked_seq + 1) sees a gap on the
+        // next handshake, and the whole database is re-streamed on every reconnect until this node
+        // happens to write again. Every delta pruned below last_acked_seq is already folded into the
+        // rows this snapshot carries, so last_acked_seq is the floor the as-of must carry.
+        var asOfSeq = Math.Max(
+            await store.GetMaxSeqAsync(ct),
+            (await store.GetPeerStateAsync(ct)).LastAckedSeq);
        var begin = new SnapshotBegin { AsOfSeq = asOfSeq };
        begin.Tables.AddRange(scope.Tables);
        await send(new SyncMessage { SnapshotBegin = begin }, ct);