feat(localdb): per-table needs_snapshot (0.3.0, on-disk schema v3)
Baselining one table used to set a single per-DATABASE needs_snapshot bit, and a snapshot streams every registered table in full — so opting one table into replication late re-sent all of them, in both directions at once on a pair enabling replication late. On a database holding a high-volume table (an event log at its retention cap) that is the largest transfer this library performs. 0.2.1 documented the cost and deferred the fix as needing an on-disk change. Schema v3 adds __localdb_snapshot_state (table_name PK, needs_snapshot), created by the shared DDL with IF NOT EXISTS so an older file gains it on open; the v2->v3 transition is a version stamp only. RegisterReplicated(..., baselineExistingRows: true) now flags that table alone. The database-wide flag survives unchanged for the paths that genuinely reach every table — a cap prune or an impossible peer ack loses delta continuity for the oplog as a whole. DeregisterReplicated drops a table's pending flag with its ledger rows, since nothing would ever stream (and so clear) it afterwards. Wire compatibility is additive and negotiated by CAPABILITY, not version: lib_schema_version stays 1 because ValidateHandshake compares it fail-closed for equality, so bumping it would stop 0.1.x/0.2.x pairs syncing outright. Handshake gains supports_partial_snapshot (field 5) and SnapshotBegin gains a tables scope (field 2) where EMPTY means all tables — precisely what a pre-0.3.0 sender emits and what a pre-0.3.0 receiver assumes. A peer that does not advertise the capability is served a full snapshot: a scoped one would make it advance last_applied_remote_seq to the snapshot's as-of and silently discard every delta below it for the tables left out. For the same reason a partial snapshot moves no watermark on either side. The receiver keeps last_applied_remote_seq (carrying only the HLC across, which is MAX-guarded), and the sender keeps _sentThruSeq, so the deltas the snapshot overlaps are still pumped and merged by LWW. Crash safety: snapshot debt now clears on the receiver's new SnapshotAck rather than on enqueue, so an ungraceful exit part-way through re-streams the same scope on reconnect. The ack rides the unbounded CONTROL lane like DeltaAck, so it composes with 0.2.1's bounded 64-message inbox and its out-of-handshake snapshot. Against a pre-0.3.0 peer the 0.2.x clear-on-enqueue behaviour is kept — there is no ack to wait for — with the handshake's horizon gap check as the net. Tests 199/199 (was 188): single-table baseline streams only that table, mixed- version interop (peer without the capability gets a full snapshot), debt standing until SnapshotAck, partial snapshot not advancing the receiver's watermark, no SnapshotAck emitted to an old peer, end-to-end over two real databases, plus v2->v3 upgrade and per-table flag isolation. README, ILocalDb remarks (which named this as the follow-up) and the umbrella CLAUDE.md index updated.
This commit is contained in:
@@ -233,10 +233,61 @@ internal sealed class OplogStore(ILocalDb db, ReplicationOptions options, Func<D
|
||||
db.ExecuteAsync(
|
||||
"UPDATE __localdb_peer_state SET last_applied_remote_seq = 0, last_seen_hlc = 0 WHERE id = 1", null, ct);
|
||||
|
||||
/// <summary>Sets or clears the snapshot-resync-required flag.</summary>
|
||||
/// <summary>
|
||||
/// Sets or clears the DATABASE-WIDE snapshot-resync flag — the one that means "every registered
|
||||
/// table". Set by the oplog-horizon paths (cap prune, impossible peer ack), which invalidate
|
||||
/// delta continuity for all tables at once. Per-table flags live in
|
||||
/// <c>__localdb_snapshot_state</c>; see <see cref="GetTablesNeedingSnapshotAsync"/>.
|
||||
/// </summary>
|
||||
public Task SetNeedsSnapshotAsync(bool value, CancellationToken ct = default) =>
|
||||
db.ExecuteAsync("UPDATE __localdb_peer_state SET needs_snapshot = @v WHERE id = 1", new { v = value ? 1 : 0 }, ct);
|
||||
|
||||
/// <summary>
|
||||
/// The tables individually flagged as owing the peer a snapshot (schema v3), in ordinal name
|
||||
/// order. Independent of the database-wide flag: baselining one table flags only that table.
|
||||
/// </summary>
|
||||
public Task<IReadOnlyList<string>> GetTablesNeedingSnapshotAsync(CancellationToken ct = default) =>
|
||||
db.QueryAsync(
|
||||
"SELECT table_name FROM __localdb_snapshot_state WHERE needs_snapshot <> 0 ORDER BY table_name",
|
||||
static r => r.GetString(0), null, ct);
|
||||
|
||||
/// <summary>
|
||||
/// Clears the snapshot debt a completed snapshot discharged: the named tables when
|
||||
/// <paramref name="tables"/> is non-empty, otherwise <b>everything</b> — the database-wide flag
|
||||
/// and every per-table row, because a full snapshot subsumes each of them.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// Rows are deleted rather than zeroed: absence is the resting state, and keeping a row per
|
||||
/// table that has ever been baselined would accumulate bookkeeping that means nothing. Runs in
|
||||
/// one transaction so a full snapshot's two clears cannot half-apply and leave the pair
|
||||
/// re-snapshotting on every reconnect.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// A scoped clear deliberately does NOT touch the database-wide flag. If the cap enforcer
|
||||
/// flagged a whole-database resync while this snapshot was in flight, that debt is real and
|
||||
/// unrelated — the snapshot only carried some of the tables — so it must survive. The full
|
||||
/// clear may take it down because a full snapshot carries every row state as of its as-of seq,
|
||||
/// which is exactly what the pruned deltas would have delivered.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
public async Task ClearSnapshotDebtAsync(IReadOnlyList<string> tables, CancellationToken ct = default)
|
||||
{
|
||||
await using var tx = await db.BeginTransactionAsync(ct);
|
||||
if (tables.Count == 0)
|
||||
{
|
||||
await tx.ExecuteAsync("UPDATE __localdb_peer_state SET needs_snapshot = 0 WHERE id = 1", null, ct);
|
||||
await tx.ExecuteAsync("DELETE FROM __localdb_snapshot_state", null, ct);
|
||||
}
|
||||
else
|
||||
{
|
||||
foreach (var table in tables)
|
||||
await tx.ExecuteAsync(
|
||||
"DELETE FROM __localdb_snapshot_state WHERE table_name = @t", new { t = table }, ct);
|
||||
}
|
||||
await tx.CommitAsync(ct);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Records the highest remote seq applied locally plus the peer HLC observed with it; also stamps
|
||||
/// <c>last_sync_utc</c>. Both watermarks are MAX-guarded (monotonic): a snapshot completing with a
|
||||
|
||||
Reference in New Issue
Block a user