5 Commits

Author	SHA1	Message	Date
Joseph Doherty	9663279594	fix(localdb): preserve the poison cause when dead-lettering itself fails; pin self-referencing cascade (0.3.3 review follow-ups) ... Three review follow-ups on 122940c. Still 0.3.3 — unpublished, so no further version bump. No behaviour change to any succeeding path. - The per-entry poison handler's recovery work (ROLLBACK TO + the dead-letter INSERT) was unguarded: a wedged connection, an I/O error, or a dead-letter INSERT that faults would propagate the SECOND exception and silently discard the FIRST — the one that says why the entry was poison at all. The pair is now wrapped; a recovery failure throws an AggregateException whose message names the entry (table + seq) and the recovery fault, and whose InnerException is the ORIGINAL entry fault, with the recovery fault carried alongside it at InnerExceptions[1] with its own stack. The outcome is unchanged and still fail-closed: nothing commits, and disposing the batch transaction unwinds it. OperationCanceledException is deliberately excluded from the wrap so a shutdown still surfaces as a cancellation rather than as a fault. No upstream caller catches SqliteException from the apply path (checked), so widening the escaping type is safe. Injected cheaply after all: a RAISE(ABORT) trigger on __localdb_dead_letter, stacked on the existing row-version poison trigger, produces exactly the two-fault case with no production test seam. The test asserts both causes are carried AND that the batch committed nothing — the entry that applied cleanly before the poison one is gone and the peer watermark never moved. - Register_SelfReferencingCascade_Throws pins the README's "or from itself" claim: a table that is its own cascading parent (parent_id REFERENCES nodes(id) ON DELETE CASCADE) is refused. The guard needed no special case — it reads every foreign key in the database — but the promise is now tested. - Comment on why EnterEntryAsync sits outside the try: a SAVEPOINT that failed opened nothing, so a handler ROLLBACK TO would name a savepoint that does not exist and fault in turn. Both new tests were run against 122940c first and fail there: the self-reference case throws nothing, and the stacked-fault case escapes as a bare SqliteException("dead-letter write failed") with the original cause gone. Suite: 215/215 green, build 0 warnings / 0 errors.	2026-08-20 08:49:11 -04:00
Joseph Doherty	122940c0e4	fix(localdb): refuse cascading-FK parents at registration; make an entry's row+ledger writes atomic (0.3.3, ScadaBridge arch-review P4-7) ... Two latent apply-path hazards from the ScadaBridge adversarial architecture review. Neither has a live trigger in any current consumer — ScadaBridge's ten replicated tables carry no foreign keys at all (verified against the running rig's site-localdb.db) and OtOpcUa's LocalDb tables carry none either — but both are the kind that surface as silent divergence rather than as an error, so they are closed before a consumer's schema grows into them. Wire-compatible: no proto change, no lib_schema_version bump, no on-disk schema change. No consumer code change. - RegisterReplicated now refuses a table that is the PARENT of a foreign key with a row-mutating ON DELETE action (CASCADE / SET NULL / SET DEFAULT), naming the offending child column and action. The apply path writes remote rows with INSERT OR REPLACE, whose PK-conflict resolution DELETES the existing row first; connections run PRAGMA foreign_keys=ON, so that implicit delete fires the child's ON DELETE action while the capture guard is raised. The child rows would change on the applying node only, produce no oplog entry, and diverge the pair with nothing able to detect it — LWW compares only rows it was told about. RESTRICT / NO ACTION stay allowed: they abort the statement instead of mutating rows behind capture's back, which the apply path already handles as per-entry poison. The check inspects every foreign key in the database and runs at registration; SQLite cannot add a foreign key to an existing table, so a child table created later is caught at the next startup's registration, not at the DDL. Documented on ILocalDb.RegisterReplicated and in the README's limitations list. - An entry's row write and its ledger write are now atomic with each other. They are two statements and SQLite aborts only the failing one, so a statement-level fault between them — the case the per-entry poison catch exists for — committed a half-applied entry: a tombstone whose table row was gone while __localdb_row_version still called it live (SnapshotStreamer can only warn and skip that, so the peer never learns the row died), or an upsert whose new row carried the OLD version and therefore lost the next comparison it should have won. The pair now runs inside a per-entry SAVEPOINT, rolled back (and released, so nothing nests) before dead-lettering, leaving both writes or neither while the rest of the batch proceeds and commits. Every new test was run against the unfixed code first and fails there with exactly the divergence shape described: the tombstone case reads back null instead of the live row, the upsert case reads back the remote row against an unchanged ledger, and the registration cases throw nothing. Suite: 213/213 green, build 0 warnings / 0 errors.	2026-08-20 08:40:14 -04:00
Joseph Doherty	9377fa1678	perf(localdb): dereg API, O(1) backlog depth, byte-budget batching, prepare-once apply (0.2.0) ... WP3.3 of the ScadaBridge architectural-review remediation. Six changes, all wire-compatible with 0.1.x peers — no proto change, no lib_schema_version bump. The on-disk bookkeeping schema goes to v2 and is upgraded in place when a file is opened. DeregisterReplicated(table). There was no way to stop capturing once a file had been registered, so a node that does not replicate kept paying three triggers inside every write transaction, appending to an oplog nothing would ever drain. Drops the triggers and prunes the table's oplog + row-version rows in one transaction; idempotent, and deliberately usable on a table this process never registered, since stale triggers left by an EARLIER build are the primary case. Deregistration must be symmetric across a pair, and already is enforced: the handshake compares registered-table digests fail-closed, so a node that drops a table its peer still replicates stops syncing loudly rather than diverging. RegisterReplicated(table, baselineExistingRows: true). Closes the late-opt-in hole: the snapshot streamer pages from __localdb_row_version, which only triggers populate, so a node that started replicating LATE never sent its pre-existing rows — silently, forever. Seeding alone would not have fixed it, because a node whose oplog starts at seq 1 presents no gap at the handshake and is told it needs no snapshot; the seed therefore also flags needs_snapshot, and the existing resync machinery carries the rows. The floor is HLC 0 stamped with this node's id: every genuine HLC is a UTC millisecond shifted left 16 bits and so is strictly greater, which makes a baselined row lose every comparison against a real remote write and win only where the peer has no version of that key at all. Two nodes baselining the same key both hold 0 and resolve by the node-id tie-break — convergent, and nothing better is available for two rows of unordered antiquity. Opt-in, so the documented CDC contract is unchanged unless asked. Pruning the ledger on deregister is safe precisely because re-registration can re-baseline; the two APIs are a pair. O(1) backlog depth. GetOplogDepth ran COUNT(*) over the entire unacked backlog, once a minute from maintenance AND on every metrics scrape — a full scan of a million rows exactly when a node is already behind. Now MAX(seq) - MAX(last_acked_seq, MIN(seq) - 1), floored at 0: three b-tree seeks, measured at 58 VM steps against 100,020 for the count on a 50k backlog. Exact, because seq is AUTOINCREMENT and every prune path removes a prefix (ack pruning by watermark, the row cap by count, the age cap by an HLC cutoff that rises with seq). The MIN clamp is what keeps it exact after a cap prune has moved the horizon above a stalled watermark — without it the caps would re-flag on every tick forever. Byte-budget batching. MaxBatchSize was the only cap on an outbound message, which made it a proxy for a byte budget that consumers had to tune against their widest row: ScadaBridge pinned it to 16 because 70 KB config_json times the 500 default is ~35 MB against a 4 MB gRPC limit. MaxBatchBytes (default 2 MB) now bounds delta and snapshot messages by summed serialized size, with the row count demoted to a secondary cap. A row larger than the budget is sent alone rather than stalling the stream on it. Since more, smaller messages would otherwise mean more apply transactions and more acks, inbound delta batches and acks ALREADY queued are coalesced — one transaction and one ack per burst, bounded by the sender's own caps, and a no-op under a trickle. LwwApplier prepares each statement once. It went through the generic transaction façade, which builds a fresh SqliteCommand per call: three SQL parses per applied row, on the path that drains a reconnect backlog. Statements are now held for the batch and rebound. Dropped __localdb_oplog_hlc. It indexed (table_name, pk_json, hlc), which no query has ever used — every oplog access path is by seq or a full hlc-predicate delete — while costing a b-tree insert inside every captured consumer write. 184/184 tests pass (was 149). New coverage: schema upgrade drops the index and preserves identity/watermarks/data, deregistration idempotency and trigger absence and cross-process cleanup, baseline seeding at the floor with LWW safety proved through the real applier, depth asserted equal to the counting query across every prune scenario plus a constant-time assertion over a 200k backlog, byte-budget chunking including the one-giant-row edge on both the delta and snapshot paths, prepare-once statement counting, and coalescing that must not swallow the message that ends its run. Version 0.2.0.	2026-08-14 22:08:43 -04:00
Joseph Doherty	b223507070	fix(localdb): applier review fixes — null-row_json guard, clock recovery via last_seen_hlc, fidelity + guard tests ... Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts	2026-07-17 22:15:51 -04:00
Joseph Doherty	d7cbdce65c	feat(localdb): LWW applier (hlc+node_id tie-break, applying guard, dead-letter, atomic watermark) ... Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts	2026-07-17 22:03:24 -04:00