feat(localdb): per-table needs_snapshot (0.3.0, on-disk schema v3)
Baselining one table used to set a single per-DATABASE needs_snapshot bit, and a snapshot streams every registered table in full — so opting one table into replication late re-sent all of them, in both directions at once on a pair enabling replication late. On a database holding a high-volume table (an event log at its retention cap) that is the largest transfer this library performs. 0.2.1 documented the cost and deferred the fix as needing an on-disk change. Schema v3 adds __localdb_snapshot_state (table_name PK, needs_snapshot), created by the shared DDL with IF NOT EXISTS so an older file gains it on open; the v2->v3 transition is a version stamp only. RegisterReplicated(..., baselineExistingRows: true) now flags that table alone. The database-wide flag survives unchanged for the paths that genuinely reach every table — a cap prune or an impossible peer ack loses delta continuity for the oplog as a whole. DeregisterReplicated drops a table's pending flag with its ledger rows, since nothing would ever stream (and so clear) it afterwards. Wire compatibility is additive and negotiated by CAPABILITY, not version: lib_schema_version stays 1 because ValidateHandshake compares it fail-closed for equality, so bumping it would stop 0.1.x/0.2.x pairs syncing outright. Handshake gains supports_partial_snapshot (field 5) and SnapshotBegin gains a tables scope (field 2) where EMPTY means all tables — precisely what a pre-0.3.0 sender emits and what a pre-0.3.0 receiver assumes. A peer that does not advertise the capability is served a full snapshot: a scoped one would make it advance last_applied_remote_seq to the snapshot's as-of and silently discard every delta below it for the tables left out. For the same reason a partial snapshot moves no watermark on either side. The receiver keeps last_applied_remote_seq (carrying only the HLC across, which is MAX-guarded), and the sender keeps _sentThruSeq, so the deltas the snapshot overlaps are still pumped and merged by LWW. Crash safety: snapshot debt now clears on the receiver's new SnapshotAck rather than on enqueue, so an ungraceful exit part-way through re-streams the same scope on reconnect. The ack rides the unbounded CONTROL lane like DeltaAck, so it composes with 0.2.1's bounded 64-message inbox and its out-of-handshake snapshot. Against a pre-0.3.0 peer the 0.2.x clear-on-enqueue behaviour is kept — there is no ack to wait for — with the handshake's horizon gap check as the net. Tests 199/199 (was 188): single-table baseline streams only that table, mixed- version interop (peer without the capability gets a full snapshot), debt standing until SnapshotAck, partial snapshot not advancing the receiver's watermark, no SnapshotAck emitted to an old peer, end-to-end over two real databases, plus v2->v3 upgrade and per-table flag isolation. README, ILocalDb remarks (which named this as the follow-up) and the umbrella CLAUDE.md index updated.
This commit is contained in:
@@ -291,6 +291,45 @@ public sealed class RegistrationTests : IDisposable
|
||||
// Without the flag the seeding would be inert: no gap exists at the handshake, so the peer
|
||||
// would be told it needs no snapshot and the baselined rows would never travel.
|
||||
Assert.True(await NeedsSnapshot(db));
|
||||
// ...and it is flagged for THIS TABLE ONLY. Through 0.2.x this set the one database-wide
|
||||
// bit, so baselining `orders` re-streamed every other registered table with it.
|
||||
Assert.False(await NeedsFullSnapshot(db));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Register_WithBaseline_FlagsOnlyTheBaselinedTable()
|
||||
{
|
||||
// The per-table flag's reason to exist: opting one table into replication late must not
|
||||
// schedule a full re-stream of its unrelated neighbours.
|
||||
using var db = Create();
|
||||
await CreateOrders(db);
|
||||
await db.ExecuteAsync("CREATE TABLE widgets (id INTEGER PRIMARY KEY, name TEXT)");
|
||||
await db.ExecuteAsync("INSERT INTO orders (id, sku, qty) VALUES (1,'A',1)");
|
||||
await db.ExecuteAsync("INSERT INTO widgets (id, name) VALUES (1,'W')");
|
||||
|
||||
db.RegisterReplicated("widgets");
|
||||
db.RegisterReplicated("orders", baselineExistingRows: true);
|
||||
|
||||
Assert.True(await NeedsSnapshot(db, "orders"));
|
||||
Assert.False(await NeedsSnapshot(db, "widgets"));
|
||||
Assert.False(await NeedsFullSnapshot(db));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Deregister_DropsThePendingSnapshotFlag()
|
||||
{
|
||||
// A flag naming a table nothing streams any more can never clear itself — the streamer skips
|
||||
// unregistered tables — so it would sit in the file and re-appear as unasked-for debt if the
|
||||
// table were registered again.
|
||||
using var db = Create();
|
||||
await CreateOrders(db);
|
||||
await db.ExecuteAsync("INSERT INTO orders (id, sku, qty) VALUES (1,'A',1)");
|
||||
db.RegisterReplicated("orders", baselineExistingRows: true);
|
||||
Assert.True(await NeedsSnapshot(db));
|
||||
|
||||
db.DeregisterReplicated("orders");
|
||||
|
||||
Assert.False(await NeedsSnapshot(db));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -301,7 +340,8 @@ public sealed class RegistrationTests : IDisposable
|
||||
await db.ExecuteAsync("INSERT INTO orders (id, sku, qty) VALUES (1,'A',1)");
|
||||
|
||||
db.RegisterReplicated("orders", baselineExistingRows: true);
|
||||
await db.ExecuteAsync("UPDATE __localdb_peer_state SET needs_snapshot = 0 WHERE id = 1");
|
||||
// Stand in for the snapshot having been delivered and acknowledged.
|
||||
await db.ExecuteAsync("DELETE FROM __localdb_snapshot_state");
|
||||
|
||||
db.RegisterReplicated("orders", baselineExistingRows: true);
|
||||
|
||||
@@ -480,7 +520,17 @@ public sealed class RegistrationTests : IDisposable
|
||||
private static async Task<long> CountAsync(SqliteLocalDb db, string sql) =>
|
||||
(await db.QueryAsync(sql, r => r.GetInt64(0)))[0];
|
||||
|
||||
private static async Task<bool> NeedsSnapshot(SqliteLocalDb db) =>
|
||||
/// <summary>Whether <paramref name="table"/> individually owes the peer a snapshot (schema v3).</summary>
|
||||
private static async Task<bool> NeedsSnapshot(SqliteLocalDb db, string table = "orders") =>
|
||||
(await db.QueryAsync(
|
||||
"SELECT COUNT(*) FROM __localdb_snapshot_state WHERE table_name = @t AND needs_snapshot <> 0",
|
||||
r => r.GetInt64(0), new { t = table }))[0] != 0;
|
||||
|
||||
/// <summary>
|
||||
/// The DATABASE-WIDE flag — "every registered table". Baselining must never set it: that is the
|
||||
/// whole point of the per-table flags, and setting it would restore the 0.2.x behaviour silently.
|
||||
/// </summary>
|
||||
private static async Task<bool> NeedsFullSnapshot(SqliteLocalDb db) =>
|
||||
(await db.QueryAsync(
|
||||
"SELECT needs_snapshot FROM __localdb_peer_state WHERE id = 1", r => r.GetInt64(0)))[0] != 0;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user