dohertj2/ScadaBridge
1
0
Fork 0
Code Issues 2 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat(localdb): move site_events to application-minted GUID ids

Browse Source 
Tasks 2 + 5a-writer + 5b of the LocalDb Phase 1 plan, landed together because
they are one indivisible change: the schema, the writer that fills it, and every
consumer that assumed the old id semantics.

WHY the id changes. Site pairs will replicate site_events with last-writer-wins on
the primary key. With INTEGER PRIMARY KEY AUTOINCREMENT both nodes independently
mint 1, 2, 3... for unrelated events, so sync would treat them as the same row and
silently overwrite. A GUID makes the event log a pure union across the pair.

Task 2 - schema extracted so the Host's AddZbLocalDb onReady can create the tables
before RegisterReplicated installs capture triggers (pre-registration rows are
never captured):
  - new OperationTrackingSchema.Apply / SiteEventLogSchema.Apply, plain
    Microsoft.Data.Sqlite, no LocalDb dependency
  - both stores delegate their InitializeSchema to them (idempotent, so a
    directly-constructed store still works)
  - OperationTracking is unchanged - it already had a TEXT PK and replicates as-is

Task 5a - SiteEventLogger mints Guid.NewGuid("N") per event and inserts it.

Task 5b - three consumers assumed a monotonic integer id. All three move to
timestamp ordering; leaving any one behind would be a live bug:
  - EventLogQueryService: "id > $afterId" would return an ARBITRARY subset of a
    GUID-keyed table and SILENTLY DROP ROWS from page-through. Now a composite
    (timestamp, id) keyset cursor with an opaque string token; timestamps are not
    unique, so id is the tie-break that guarantees exactly-once paging.
  - EventLogPurgeService: "ORDER BY id ASC LIMIT 1000" would delete a RANDOM batch
    instead of the oldest. Now orders by timestamp.
  - EventLogEntry.Id and both ContinuationTokens: long -> string.

WIRE COMPATIBILITY. Those DTOs cross the site<->central Akka boundary
(SiteCommunicationActor -> CommunicationService -> ManagementActor / CentralUI).
No rolling-upgrade shim is needed because both sides ship in the same deployable
and the rig redeploys as a unit. Checked: no Akka serializer binding pins these
types by name. A stale numeric token degrades to "start from the beginning"
(a visible repeat) rather than throwing or losing rows.

Tests: the two that encoded the old semantics were rewritten to guard the new
invariant rather than deleted - uniqueness instead of monotonicity, and
oldest-purged-first keyed on timestamp. That second test also exposed a latent
weakness: the bulk seed stamped every row with the same UtcNow, so "oldest" was
never actually well-defined; rows now get distinct increasing timestamps, kept
inside the retention window so the retention purge does not eat them first.

Verified:
  dotnet build ZB.MOM.WW.ScadaBridge.slnx -> 0 Error(s), 0 Warning(s)
  SiteEventLogging.Tests -> 70 passed (12 new schema tests, red-first)
  CentralUI.Tests        -> 925 passed
  Commons.Tests          -> 684 passed
  SiteRuntime.Tests      -> 529 passed, 1 pre-existing flaky failure
                            (InstanceActorChildAttributeRaceTests - passes 3/3 in
                            isolation with and without this change)

Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
This commit is contained in:
Joseph Doherty
2026-07-19 03:15:17 -04:00
parent f056b67e9a
commit 727fa48cba
14 changed files with 659 additions and 147 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Tracking/OperationTrackingStore.cs
+6 -70
View File
@@ -69,76 +69,12 @@ public class OperationTrackingStore : IOperationTrackingStore, IAsyncDisposable,
InitializeSchema();
}
private void InitializeSchema()
{
using var cmd = _writeConnection.CreateCommand();
cmd.CommandText = """
CREATE TABLE IF NOT EXISTS OperationTracking (
TrackedOperationId TEXT NOT NULL PRIMARY KEY,
Kind TEXT NOT NULL,
TargetSummary TEXT NULL,
Status TEXT NOT NULL,
RetryCount INTEGER NOT NULL DEFAULT 0,
LastError TEXT NULL,
HttpStatus INTEGER NULL,
CreatedAtUtc TEXT NOT NULL,
UpdatedAtUtc TEXT NOT NULL,
TerminalAtUtc TEXT NULL,
SourceInstanceId TEXT NULL,
SourceScript TEXT NULL,
SourceNode TEXT NULL
);
CREATE INDEX IF NOT EXISTS IX_OperationTracking_Status_Updated
ON OperationTracking (Status, UpdatedAtUtc);
CREATE INDEX IF NOT EXISTS IX_OperationTracking_UpdatedAt
ON OperationTracking (UpdatedAtUtc);
""";
cmd.ExecuteNonQuery();
// SourceNode stamping: additively add the SourceNode column.
// CREATE TABLE IF NOT EXISTS above does NOT add columns to an
// OperationTracking table that already exists from a pre-SourceNode
// build, so a tracking.db created by an older build needs the column
// ALTER-ed in. The file is durable across restart/failover by design
// (retention window default 7 days), so without this step every
// RecordEnqueueAsync on an upgraded deployment would bind $sourceNode
// against a missing column and the write would fail.
// SQLite has no "ADD COLUMN IF NOT EXISTS"; the column presence is
// probed first and the ALTER skipped when already there. The column is
// nullable with no default, so any row written before this migration
// reads back SourceNode = null (back-compat).
//
// NOTE: This is the FIRST idempotent column-upgrade in
// OperationTrackingStore — prior schema changes pre-dated any
// production rollout and relied solely on CREATE TABLE IF NOT EXISTS.
// The helper mirrors the SqliteAuditWriter precedent.
AddColumnIfMissing("SourceNode", "TEXT NULL");
}
/// <summary>
/// Additively adds a column to <c>OperationTracking</c> only when it is not
/// already present. SQLite lacks <c>ADD COLUMN IF NOT EXISTS</c>, so the
/// schema is probed via <c>PRAGMA table_info</c> first. Idempotent — safe
/// to run on every <see cref="InitializeSchema"/>. Mirrors the
/// <c>SqliteAuditWriter.AddColumnIfMissing</c> precedent.
/// </summary>
private void AddColumnIfMissing(string columnName, string columnDefinition)
{
using var probe = _writeConnection.CreateCommand();
probe.CommandText = "SELECT COUNT(*) FROM pragma_table_info('OperationTracking') WHERE name = $name";
probe.Parameters.AddWithValue("$name", columnName);
var exists = Convert.ToInt32(probe.ExecuteScalar()) > 0;
if (exists)
{
return;
}
using var alter = _writeConnection.CreateCommand();
// Column name + definition are caller-controlled constants, never user
// input — safe to interpolate (parameters are not permitted in DDL).
alter.CommandText = $"ALTER TABLE OperationTracking ADD COLUMN {columnName} {columnDefinition}";
alter.ExecuteNonQuery();
}
// Schema lives in OperationTrackingSchema so the Host's AddZbLocalDb onReady
// callback can create this table in the consolidated site database before
// RegisterReplicated installs its capture triggers. Applying it here too keeps a
// directly-constructed store (tests, tooling) self-sufficient; the DDL is
// idempotent, so running it from both places is harmless.
private void InitializeSchema() => OperationTrackingSchema.Apply(_writeConnection);
/// <inheritdoc/>
public async Task RecordEnqueueAsync(