fix(localdb): snapshot delivers FK-linked rows in any order (scadaproj#37)
The snapshot streamed tables by name and rows by pk_json text, so a child (or a self-reference whose parent's key sorts later) reached the receiver first, failed its immediate FK check and was dead-lettered - permanently, since the snapshot's watermark and ack covered it (LocalDb-002). Sender: stream referenced tables first (Kahn order, name tie-break). Receiver: SnapshotApplier defers rows that fail only SQLITE_CONSTRAINT_FOREIGNKEY, retries them as rows land, and dead-letters true orphans at SnapshotComplete. Delta path unchanged; wire unchanged. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
@@ -10,7 +10,7 @@
|
||||
| Last reviewed | 2026-11-09 |
|
||||
| Reviewer | Claude |
|
||||
| Commit reviewed | `91fb2b4` |
|
||||
| Open findings | 6 |
|
||||
| Open findings | 5 |
|
||||
|
||||
## Summary
|
||||
|
||||
@@ -150,7 +150,7 @@ non-displacement cases fail.
|
||||
| Severity | Medium |
|
||||
| Priority | P2 |
|
||||
| Category | Correctness & logic bugs |
|
||||
| Status | Open |
|
||||
| Status | Resolved |
|
||||
| Location | `ZB.MOM.WW.LocalDb/src/ZB.MOM.WW.LocalDb.Replication/Internal/SnapshotStreamer.cs:79` |
|
||||
|
||||
**Description**
|
||||
@@ -203,7 +203,26 @@ Also add a snapshot test with a non-cascading FK, and consider not dead-letterin
|
||||
|
||||
**Resolution**
|
||||
|
||||
_Unresolved._
|
||||
Resolved 2026-11-09 (scadaproj#37, LocalDb `0.3.6`, unpublished). Re-verified with a 2-node test: a wiped peer
|
||||
received `a_child` before `z_parent` and `node {"id":10}` before `node {"id":2}`. Both children were dead-lettered
|
||||
(`SQLite Error 19: 'FOREIGN KEY constraint failed'`) and the watermark advanced past them. Fixed on both sides, and
|
||||
the wire is unchanged:
|
||||
- **Sender.** `SnapshotStreamer.ParentsFirstAsync` orders the tables in scope topologically by
|
||||
`pragma_foreign_key_list` (Kahn's algorithm, ordinal name tie-break). Self-references are ignored, and a cycle
|
||||
falls back to name order.
|
||||
- **Receiver.** `LwwApplier.ApplyBatchAsync` gained `deferForeignKeyViolations`, used only by `SnapshotApplier`. An
|
||||
entry that fails with SQLITE_CONSTRAINT_FOREIGNKEY (787) is rolled back to its savepoint and returned as deferred,
|
||||
not dead-lettered. The applier retries deferred rows after every batch that applied something, until a pass makes
|
||||
no progress. At `SnapshotComplete` it makes one final pass with deferral off, which dead-letters the true orphans
|
||||
(with a warning). The delta path is unchanged, because commit order already puts parents first.
|
||||
|
||||
Tests (`SnapshotForeignKeyTests`, all 4 fail pre-fix):
|
||||
- 2-node wiped-peer snapshot with `MaxBatchSize = 1`, cross-table and self-referencing;
|
||||
- streamer table order;
|
||||
- a receiver fed child-first in the old sender order;
|
||||
- a true orphan dead-lettered only at complete.
|
||||
|
||||
README §Limitations documents the mixed-version caveat: a pre-0.3.6 receiver still loses child-first self-references.
|
||||
|
||||
### LocalDb-003 — `INSERT OR REPLACE` apply silently deletes a different row on a secondary `UNIQUE` conflict, so the pair diverges
|
||||
|
||||
|
||||
@@ -40,7 +40,7 @@ code-reviews/
|
||||
|
||||
## Summary
|
||||
|
||||
10 of 10 libraries reviewed. 22 pending findings across all libraries.
|
||||
10 of 10 libraries reviewed. 21 pending findings across all libraries.
|
||||
|
||||
Coverage: Secrets, LocalDb, GalaxyRepository and Overview first reviewed, and Auth, Health,
|
||||
Theme, Audit, Configuration and Telemetry re-reviewed, on 2026-11-09 (#15). Each finding also
|
||||
@@ -50,9 +50,9 @@ carries a P1/P2/P3 priority (P1 = Critical/High, P2 = Medium, P3 = Low).
|
||||
|----------|---------------|
|
||||
| Critical | 0 |
|
||||
| High | 0 |
|
||||
| Medium | 3 |
|
||||
| Medium | 2 |
|
||||
| Low | 19 |
|
||||
| **Total** | **22** |
|
||||
| **Total** | **21** |
|
||||
|
||||
## Library Status
|
||||
|
||||
@@ -63,7 +63,7 @@ carries a P1/P2/P3 priority (P1 = Critical/High, P2 = Medium, P3 = Low).
|
||||
| [Configuration](Configuration/findings.md) | 2026-11-09 | `91fb2b4` | 0/0/0/0 | 0 | 4 |
|
||||
| [GalaxyRepository](GalaxyRepository/findings.md) | 2026-11-09 | `91fb2b4` | 0/0/0/0 | 0 | 8 |
|
||||
| [Health](Health/findings.md) | 2026-11-09 | `91fb2b4` | 0/0/0/0 | 0 | 7 |
|
||||
| [LocalDb](LocalDb/findings.md) | 2026-11-09 | `91fb2b4` | 0/0/3/3 | 6 | 7 |
|
||||
| [LocalDb](LocalDb/findings.md) | 2026-11-09 | `91fb2b4` | 0/0/2/3 | 5 | 7 |
|
||||
| [Overview](Overview/findings.md) | 2026-11-09 | `91fb2b4` | 0/0/0/0 | 0 | 5 |
|
||||
| [Secrets](Secrets/findings.md) | 2026-11-09 | `91fb2b4` | 0/0/0/13 | 13 | 19 |
|
||||
| [Telemetry](Telemetry/findings.md) | 2026-11-09 | `91fb2b4` | 0/0/0/0 | 0 | 11 |
|
||||
@@ -84,11 +84,10 @@ _None open._
|
||||
|
||||
_None open._
|
||||
|
||||
### Medium (3)
|
||||
### Medium (2)
|
||||
|
||||
| ID | Library | Title |
|
||||
|----|---------|-------|
|
||||
| LocalDb-002 | [LocalDb](LocalDb/findings.md) | Snapshot streams tables by name and rows by `pk_json` text, so FK children land before parents and are dead-lettered for good |
|
||||
| LocalDb-003 | [LocalDb](LocalDb/findings.md) | `INSERT OR REPLACE` apply silently deletes a different row on a secondary `UNIQUE` conflict, so the pair diverges |
|
||||
| LocalDb-004 | [LocalDb](LocalDb/findings.md) | `FailClosedOnDrift` is bypassed on the snapshot path, so a future-stamped snapshot row moves the local HLC forward permanently |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user