fix(localdb): refuse AUTOINCREMENT keys at RegisterReplicated (LocalDb-005)

Each node assigns its own AUTOINCREMENT keys, so two different rows inserted
on the two nodes share a pk_json and LWW silently keeps one on both nodes.
RegisterReplicated now refuses such a table (detected from SQLite column
metadata, before any trigger is installed). A plain INTEGER PRIMARY KEY is
still accepted (application-assigned ids are legitimate); the requirement
that replicated keys be unique across both nodes is documented in the
RegisterReplicated remarks and README Limitations. No ScadaBridge or OtOpcUa
replicated table is AUTOINCREMENT.

(scadaproj#46)

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 15:09:45 -05:00
parent 76e9c355b4
commit fd46999946
5 changed files with 142 additions and 6 deletions
+10
View File
@@ -174,6 +174,8 @@ builder.Services.AddZbLocalDb(builder.Configuration, db =>
"CREATE TABLE IF NOT EXISTS widget (id INTEGER PRIMARY KEY, name TEXT, qty INTEGER)") "CREATE TABLE IF NOT EXISTS widget (id INTEGER PRIMARY KEY, name TEXT, qty INTEGER)")
.GetAwaiter().GetResult(); .GetAwaiter().GetResult();
db.RegisterReplicated("widget"); // PK required; installs capture triggers db.RegisterReplicated("widget"); // PK required; installs capture triggers
// The application assigns widget ids. A key SQLite assigns per node (AUTOINCREMENT, or a NULL
// rowid) collides across a pair: AUTOINCREMENT is refused, prefer TEXT GUIDs. See Limitations.
}); });
// Consume ILocalDb anywhere: // Consume ILocalDb anywhere:
@@ -295,6 +297,14 @@ All options are validated at startup (`ZB.MOM.WW.Configuration` validators).
- **Exactly two nodes.** No mesh / N-node topologies. - **Exactly two nodes.** No mesh / N-node topologies.
- **LWW only.** No per-table pluggable conflict resolvers. - **LWW only.** No per-table pluggable conflict resolvers.
- **Replicated primary keys must be unique across both nodes.** LWW keys on the primary key,
so two different rows inserted on the two nodes under the same key are read as two versions
of one row, and one is lost on **both** nodes with no dead-letter and no metric. A key each
node assigns on its own cannot replicate: `RegisterReplicated` **refuses an `AUTOINCREMENT`
key** (`0.3.6`), and a plain `INTEGER PRIMARY KEY` is accepted only because the application
may assign it from a node-scoped range — left `NULL` on insert it loses rows the same way.
Use a TEXT GUID minted by the writer (what ScadaBridge's `site_events` and OtOpcUa's alarm
store-and-forward did when they moved onto LocalDb).
- **BLOB columns are rejected at registration** (a column declared `BLOB` fails - **BLOB columns are rejected at registration** (a column declared `BLOB` fails
`RegisterReplicated`, because `json_object` cannot capture it). A *typeless* column can `RegisterReplicated`, because `json_object` cannot capture it). A *typeless* column can
still hold a BLOB value at runtime, which then surfaces as a write-time error, not a still hold a BLOB value at runtime, which then surfaces as a write-time error, not a
@@ -38,11 +38,22 @@ public interface ILocalDb
/// schema digest, and installs the three AFTER triggers that capture every change into the /// schema digest, and installs the three AFTER triggers that capture every change into the
/// oplog + row-version tables within the same transaction as the consumer's write. Idempotent /// oplog + row-version tables within the same transaction as the consumer's write. Idempotent
/// (re-registering regenerates the triggers). Throws <see cref="LocalDbRegistrationException"/> /// (re-registering regenerates the triggers). Throws <see cref="LocalDbRegistrationException"/>
/// if the name is unsafe, the table is missing, it has no primary key, or it is the parent of a /// if the name is unsafe, the table is missing, it has no primary key, its primary key is
/// foreign key whose <c>ON DELETE</c> action mutates the child (<c>CASCADE</c> / <c>SET NULL</c> / /// <c>AUTOINCREMENT</c> (0.3.6), or it is the parent of a foreign key whose <c>ON DELETE</c> action
/// <c>SET DEFAULT</c>) — see the remarks for why that last one cannot be replicated. /// mutates the child (<c>CASCADE</c> / <c>SET NULL</c> / <c>SET DEFAULT</c>) — see the remarks for why
/// the last two cannot be replicated.
/// </summary> /// </summary>
/// <remarks> /// <remarks>
/// <para>
/// <b>Replicated primary keys must be unique across BOTH nodes.</b> Last-writer-wins keys on the
/// primary key, so two different rows that two nodes insert under the same key are read as two
/// versions of ONE row and one of them is silently lost on both nodes. A key each node assigns on
/// its own therefore cannot replicate: an <c>AUTOINCREMENT</c> key is refused outright, and a plain
/// <c>INTEGER PRIMARY KEY</c> (a rowid alias) is accepted only because the application may assign
/// it from a node-scoped range — left NULL on insert it loses rows the same way. Prefer a TEXT GUID
/// minted by the writer.
/// </para>
/// <para>
/// <b>Replicated tables may not be the parent of a cascading foreign key.</b> The apply path /// <b>Replicated tables may not be the parent of a cascading foreign key.</b> The apply path
/// deletes rows for remote tombstones (and, before 0.3.6, wrote remote rows with /// deletes rows for remote tombstones (and, before 0.3.6, wrote remote rows with
/// <c>INSERT OR REPLACE</c>, which deleted the conflicting row first); with /// <c>INSERT OR REPLACE</c>, which deleted the conflicting row first); with
@@ -52,6 +63,7 @@ public interface ILocalDb
/// runs at registration and inspects the whole database's foreign keys; SQLite cannot add a /// runs at registration and inspects the whole database's foreign keys; SQLite cannot add a
/// foreign key to an existing table, so a later-created child table is caught on the next /// foreign key to an existing table, so a later-created child table is caught on the next
/// registration (i.e. the next startup), not at the moment of the DDL. /// registration (i.e. the next startup), not at the moment of the DDL.
/// </para>
/// <para> /// <para>
/// <b>Secondary <c>UNIQUE</c> keys must be unique across BOTH nodes.</b> Since 0.3.6 a remote row /// <b>Secondary <c>UNIQUE</c> keys must be unique across BOTH nodes.</b> Since 0.3.6 a remote row
/// is upserted on its primary key only; if it collides with a different local row on another /// is upserted on its primary key only; if it collides with a different local row on another
@@ -199,6 +199,7 @@ internal sealed class SqliteLocalDb : ILocalDb, IDisposable
throw new LocalDbRegistrationException( throw new LocalDbRegistrationException(
$"Cannot register table '{tableName}': replication requires an explicit primary key."); $"Cannot register table '{tableName}': replication requires an explicit primary key.");
EnsureNoAutoincrementKey(tableName);
EnsureNoInboundActionOnDelete(tableName); EnsureNoInboundActionOnDelete(tableName);
var allColumns = columns.Select(c => (c.Name, c.Type)).ToList(); var allColumns = columns.Select(c => (c.Name, c.Type)).ToList();
@@ -295,6 +296,46 @@ internal sealed class SqliteLocalDb : ILocalDb, IDisposable
} }
} }
/// <summary>
/// Refuses to register a table whose primary key is declared <c>AUTOINCREMENT</c> (LocalDb-005).
/// </summary>
/// <remarks>
/// <para>
/// <b>Why.</b> LWW keys on <c>pk_json</c>. <c>AUTOINCREMENT</c> exists to have SQLite assign the
/// key, and each node assigns its own 1, 2, 3 … — so two DIFFERENT rows inserted on the two nodes
/// (both nodes of a pair logging during a split, say) carry the same <c>pk_json</c>, LWW reads them
/// as two versions of one row, and one row is lost on BOTH nodes with no dead-letter and no metric.
/// That is never what a replicated table wants, so it is refused rather than warned about.
/// </para>
/// <para>
/// <b>A plain <c>INTEGER PRIMARY KEY</c> is still allowed</b>, because the application may assign
/// it itself from a globally unique or node-scoped range. Left NULL on insert it has exactly the
/// same hazard (it is the same rowid alias, minus the never-reuse guarantee); the remarks on
/// <see cref="ILocalDb.RegisterReplicated(string)"/> say so, and the safe choice is a TEXT GUID.
/// </para>
/// <para>
/// Detection reads SQLite's own column metadata (<c>sqlite3_table_column_metadata</c>, surfaced as
/// <c>IsAutoIncrement</c> in the reader's schema table) rather than parsing the <c>CREATE</c> text,
/// so a column named or defaulted to the word is not mistaken for the keyword.
/// </para>
/// </remarks>
private void EnsureNoAutoincrementKey(string tableName)
{
using var cmd = _master.CreateCommand();
cmd.CommandText = $"SELECT * FROM {TriggerSqlGenerator.Ident(tableName)} LIMIT 0";
using var reader = cmd.ExecuteReader();
var schema = reader.GetSchemaTable();
foreach (System.Data.DataRow row in schema.Rows)
{
if (row["IsAutoIncrement"] is true)
throw new LocalDbRegistrationException(
$"Cannot register table '{tableName}': its primary key '{row["BaseColumnName"]}' is declared " +
"AUTOINCREMENT. Each node would assign the same keys to different rows, and last-writer-wins " +
"would silently keep only one of them on both nodes. Use a globally unique key instead " +
"(a TEXT GUID, or an INTEGER the application assigns from a node-scoped range).");
}
}
/// <summary> /// <summary>
/// Refuses to register a table that is the PARENT of a foreign key whose <c>ON DELETE</c> action /// Refuses to register a table that is the PARENT of a foreign key whose <c>ON DELETE</c> action
/// mutates the child (<c>CASCADE</c>, <c>SET NULL</c>, <c>SET DEFAULT</c>). /// mutates the child (<c>CASCADE</c>, <c>SET NULL</c>, <c>SET DEFAULT</c>).
@@ -70,6 +70,58 @@ public sealed class RegistrationTests : IDisposable
Assert.Contains("BLOB", ex.Message); Assert.Contains("BLOB", ex.Message);
} }
// ---- LocalDb-005 (scadaproj#46): AUTOINCREMENT keys are assigned independently on each node ----
[Fact]
public async Task Register_AutoincrementKey_Throws()
{
// Each node hands out 1, 2, 3 … on its own, so two DIFFERENT rows inserted on the two nodes
// share a pk_json; LWW reads them as two versions of one row and one is lost on both nodes.
using var db = Create();
await db.ExecuteAsync("CREATE TABLE ev (id INTEGER PRIMARY KEY AUTOINCREMENT, msg TEXT)");
var ex = Assert.Throws<LocalDbRegistrationException>(() => db.RegisterReplicated("ev"));
Assert.Contains("ev", ex.Message);
Assert.Contains("AUTOINCREMENT", ex.Message);
// Refused before anything was installed.
Assert.Empty(await db.QueryAsync(
"SELECT name FROM sqlite_master WHERE type='trigger' AND tbl_name='ev'", r => r.GetString(0)));
}
[Fact]
public async Task Register_AutoincrementKey_Throws_WithBaselineToo()
{
using var db = Create();
await db.ExecuteAsync("CREATE TABLE ev (id INTEGER PRIMARY KEY AUTOINCREMENT, msg TEXT)");
Assert.Throws<LocalDbRegistrationException>(() => db.RegisterReplicated("ev", baselineExistingRows: true));
}
[Fact]
public async Task Register_PlainIntegerPrimaryKey_IsStillAllowed()
{
// Compatibility guard: an INTEGER PRIMARY KEY the application assigns itself (node-scoped or
// otherwise globally unique ids) is legitimate, and refusing it would break existing schemas.
using var db = Create();
await CreateOrders(db);
db.RegisterReplicated("orders");
Assert.True(db.ReplicatedTables.ContainsKey("orders"));
}
[Fact]
public async Task Register_TheWordAutoincrementOutsideTheKey_IsNotMistakenForIt()
{
// Detection reads SQLite's own column metadata, not the CREATE text, so a column NAMED
// autoincrement or a default value spelling it does not trip the check.
using var db = Create();
await db.ExecuteAsync(
"CREATE TABLE notes (id TEXT PRIMARY KEY, \"autoincrement\" INTEGER, note TEXT DEFAULT 'AUTOINCREMENT')");
db.RegisterReplicated("notes");
Assert.True(db.ReplicatedTables.ContainsKey("notes"));
}
[Fact] [Fact]
public async Task Register_CascadeDeleteParent_Throws() public async Task Register_CascadeDeleteParent_Throws()
{ {
+24 -3
View File
@@ -10,7 +10,7 @@
| Last reviewed | 2026-11-09 | | Last reviewed | 2026-11-09 |
| Reviewer | Claude | | Reviewer | Claude |
| Commit reviewed | `91fb2b4` | | Commit reviewed | `91fb2b4` |
| Open findings | 2 | | Open findings | 1 |
## Summary ## Summary
@@ -362,7 +362,7 @@ applied and that no dead-letter is written. README §Semantics is updated.
| Severity | Low | | Severity | Low |
| Priority | P3 | | Priority | P3 |
| Category | Documentation & XML docs | | Category | Documentation & XML docs |
| Status | Open | | Status | Resolved |
| Location | `ZB.MOM.WW.LocalDb/src/ZB.MOM.WW.LocalDb/Internal/SqliteLocalDb.cs:197` | | Location | `ZB.MOM.WW.LocalDb/src/ZB.MOM.WW.LocalDb/Internal/SqliteLocalDb.cs:197` |
**Description** **Description**
@@ -393,7 +393,28 @@ at registration when the sole PK column is an `INTEGER PRIMARY KEY` rowid alias.
**Resolution** **Resolution**
_Unresolved._ Resolved 2026-11-09 (scadaproj#46, LocalDb `0.3.6`, unpublished). Re-verified with a throwaway 2-node probe:
`ev(id INTEGER PRIMARY KEY AUTOINCREMENT, msg TEXT)`, `'from A'` inserted on A and `'from B'` on B, both nodes ended
with `1:from B` only and no dead-letter. Fix, taking the review's safer option: `RegisterReplicated` (both overloads)
now **refuses** a table whose primary key is `AUTOINCREMENT` with a `LocalDbRegistrationException` that names the
column and the remedy, before any trigger is installed (`SqliteLocalDb.EnsureNoAutoincrementKey`). Detection reads
SQLite's column metadata (`IsAutoIncrement` from `sqlite3_table_column_metadata`), not the `CREATE` text. A refusal
rather than a warning, because `AUTOINCREMENT` only exists to have SQLite assign the key, which is never safe on a
replicated table, and `SqliteLocalDb` has no logger to warn through.
A plain `INTEGER PRIMARY KEY` is still accepted, because an application may assign it from a node-scoped range;
refusing it would break schemas that are correct today. Its hazard when left `NULL` is documented instead: the
`RegisterReplicated` XML remarks and README §Limitations now say replicated primary keys must be unique across both
nodes and recommend writer-minted TEXT GUIDs, and the quickstart notes it.
Compatibility: checked read-only on 2026-11-09. ScadaBridge's 12 replicated tables (`SiteLocalDbSetup.ReplicatedTables`)
and OtOpcUa's 5 (`LocalDbSetup.ReplicatedTables`: the two deployment-cache tables, the alarm S&F events table, and
the alarm condition state and comments tables) all use TEXT or composite keys, and none is `AUTOINCREMENT`. Both apps had already replaced their historical
`INTEGER PRIMARY KEY AUTOINCREMENT` keys with GUIDs for this reason, so neither boot is affected.
Tests (`RegistrationTests`): `Register_AutoincrementKey_Throws` and `..._WithBaselineToo` fail pre-fix. Two guards pass
both ways: a plain `INTEGER PRIMARY KEY` still registers, and a column named `autoincrement` with a `DEFAULT
'AUTOINCREMENT'` is not mistaken for the keyword.
### LocalDb-006 — Column names containing `'` are captured but break the apply and snapshot SQL ### LocalDb-006 — Column names containing `'` are captured but break the apply and snapshot SQL