fix(localdb): snapshot delivers FK-linked rows in any order (scadaproj#37)

The snapshot streamed tables by name and rows by pk_json text, so a child
(or a self-reference whose parent's key sorts later) reached the receiver
first, failed its immediate FK check and was dead-lettered - permanently,
since the snapshot's watermark and ack covered it (LocalDb-002).

Sender: stream referenced tables first (Kahn order, name tie-break).
Receiver: SnapshotApplier defers rows that fail only
SQLITE_CONSTRAINT_FOREIGNKEY, retries them as rows land, and dead-letters
true orphans at SnapshotComplete. Delta path unchanged; wire unchanged.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 14:52:35 -05:00
parent 48813bba6f
commit 4af0ea8152
7 changed files with 351 additions and 18 deletions
+22 -3
View File
@@ -10,7 +10,7 @@
| Last reviewed | 2026-11-09 |
| Reviewer | Claude |
| Commit reviewed | `91fb2b4` |
| Open findings | 6 |
| Open findings | 5 |
## Summary
@@ -150,7 +150,7 @@ non-displacement cases fail.
| Severity | Medium |
| Priority | P2 |
| Category | Correctness & logic bugs |
| Status | Open |
| Status | Resolved |
| Location | `ZB.MOM.WW.LocalDb/src/ZB.MOM.WW.LocalDb.Replication/Internal/SnapshotStreamer.cs:79` |
**Description**
@@ -203,7 +203,26 @@ Also add a snapshot test with a non-cascading FK, and consider not dead-letterin
**Resolution**
_Unresolved._
Resolved 2026-11-09 (scadaproj#37, LocalDb `0.3.6`, unpublished). Re-verified with a 2-node test: a wiped peer
received `a_child` before `z_parent` and `node {"id":10}` before `node {"id":2}`. Both children were dead-lettered
(`SQLite Error 19: 'FOREIGN KEY constraint failed'`) and the watermark advanced past them. Fixed on both sides, and
the wire is unchanged:
- **Sender.** `SnapshotStreamer.ParentsFirstAsync` orders the tables in scope topologically by
`pragma_foreign_key_list` (Kahn's algorithm, ordinal name tie-break). Self-references are ignored, and a cycle
falls back to name order.
- **Receiver.** `LwwApplier.ApplyBatchAsync` gained `deferForeignKeyViolations`, used only by `SnapshotApplier`. An
entry that fails with SQLITE_CONSTRAINT_FOREIGNKEY (787) is rolled back to its savepoint and returned as deferred,
not dead-lettered. The applier retries deferred rows after every batch that applied something, until a pass makes
no progress. At `SnapshotComplete` it makes one final pass with deferral off, which dead-letters the true orphans
(with a warning). The delta path is unchanged, because commit order already puts parents first.
Tests (`SnapshotForeignKeyTests`, all 4 fail pre-fix):
- 2-node wiped-peer snapshot with `MaxBatchSize = 1`, cross-table and self-referencing;
- streamer table order;
- a receiver fed child-first in the old sender order;
- a true orphan dead-lettered only at complete.
README §Limitations documents the mixed-version caveat: a pre-0.3.6 receiver still loses child-first self-references.
### LocalDb-003 — `INSERT OR REPLACE` apply silently deletes a different row on a secondary `UNIQUE` conflict, so the pair diverges