846c8d6335
Every Part 9 transition used to re-serialize the alarm's whole comment trail into alarm_condition_state.comments_json and replicate it to the pair peer: O(n) per transition, O(n^2) over an alarm's life (measured: one ack at 10k comments shipped 1,080,553 oplog bytes; now ~1.6 KB, the same as at 10). - New replicated table alarm_condition_comment, PK (scripted_alarm_id, seq), seq = the entry's ordinal in the trail. Ordinal (not content) keying makes the two pair nodes' copies of one logical entry (each stamped by its own clock, since both nodes apply every command) collapse to one row under LWW. - LocalDbAlarmConditionStateStore.SaveAsync appends only entries past the stored MAX(seq) (INSERT OR IGNORE, append-only) and upserts a fixed-size condition row with comments_json = "[]", in one transaction. Remove drops the trail with the row. Load merges the table with any inline trail. - comments_json column kept (always "[]") so the condition table's shape and replication digest are unchanged and old-shape rows still load exactly. - AlarmConditionStateSchema.MigrateInlineComments: idempotent per-row move of inline trails, run by LocalDbSetup.OnReady after RegisterReplicated so the moved entries replicate. Comment table joins ReplicatedTables (now five). Tests: constant transition payload at 10 vs 10k comments (+ instrument positive control), old-shape back-compat load, migration idempotency / partial re-run / lazy move, OnReady boot migration captured in the oplog, and two loopback pair tests (A->B and B->A trail convergence; the shared entry converges to one copy, with a distinct-alarm positive control). Refs #748 Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
271 lines
14 KiB
C#
271 lines
14 KiB
C#
using System.Globalization;
|
|
using System.Text.Json;
|
|
using Microsoft.Data.Sqlite;
|
|
|
|
namespace ZB.MOM.WW.OtOpcUa.Core.AlarmHistorian;
|
|
|
|
/// <summary>
|
|
/// DDL for the Part 9 scripted-alarm condition state: one fixed-size row per alarm identity
|
|
/// holding the operator-supplied state (Enabled / Acked / Confirmed / Shelving) plus the
|
|
/// ack/confirm audit fields, and a separate append-only table holding each alarm's comment trail.
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// <para>
|
|
/// Replaces the central config DB's <c>ScriptedAlarmState</c> table as the node-local home
|
|
/// for this state (per-cluster mesh Phase 4, which cuts the ConfigDb from driver-only
|
|
/// nodes). Living in the consolidated LocalDb file is what lets the state replicate to the
|
|
/// redundant pair peer, exactly like the alarm store-and-forward buffer — see
|
|
/// <see cref="AlarmSfSchema"/>, which this mirrors.
|
|
/// </para>
|
|
/// <para>
|
|
/// Deliberately depends on nothing but <see cref="SqliteConnection"/> so it can be applied
|
|
/// to any connection — the host's <c>LocalDbSetup.OnReady</c> in production, and a bare
|
|
/// connection in a test — without dragging the DI graph along.
|
|
/// </para>
|
|
/// <para>
|
|
/// <b>The primary key is the alarm identity, and Save is an unconditional upsert onto it.</b>
|
|
/// Convergence across the pair is last-writer-wins over the primary key, handled by the
|
|
/// LocalDb replication HLC — so the store stacks no application-level last-write-wins on top,
|
|
/// the same discipline the alarm-sf sink and the deployment pointer follow.
|
|
/// </para>
|
|
/// <para>
|
|
/// <b>ActiveState has no column</b> — it is re-derived from the live predicate on startup,
|
|
/// so persisting it would only risk operators seeing a stale Active on restart. Likewise
|
|
/// <c>LastActiveUtc</c> / <c>LastClearedUtc</c> re-derive alongside it and get no columns.
|
|
/// <c>LastTransitionUtc</c> is carried by <c>updated_at_utc</c> — the row-write timestamp is
|
|
/// the last transition. Timestamps are round-trip ("O") TEXT so lexicographic order is
|
|
/// chronological; nullable timestamps and audit fields are stored as NULL.
|
|
/// </para>
|
|
/// <para>
|
|
/// <b>The comment trail is NOT on the condition row.</b> It used to be serialized whole into
|
|
/// <c>comments_json</c>, so every transition rewrote — and replicated — the alarm's entire
|
|
/// history: O(n) per transition, O(n²) over the alarm's life (review R05:4). It now lives in
|
|
/// <see cref="CommentsTable"/>, one row per entry keyed by <c>(scripted_alarm_id, seq)</c>,
|
|
/// where <c>seq</c> is the entry's ordinal in the alarm's trail. A transition inserts only the
|
|
/// entries it appended, so the condition row stays a fixed size and a transition's write is
|
|
/// O(1) however long the history is. <c>comments_json</c> is kept, always written as
|
|
/// <c>"[]"</c>, so the condition table's shape — and with it the pair's replication digest
|
|
/// for that table — does not change and an old-shape row stays readable;
|
|
/// <see cref="MigrateInlineComments"/> moves any inline trail across.
|
|
/// </para>
|
|
/// <para>
|
|
/// <b>Why the ordinal, and not a content hash, is the key.</b> Both halves of a redundant pair
|
|
/// run the engine and both apply every operator command, each stamping the entry with its own
|
|
/// clock. A content-derived id would keep TWO rows for every logical entry once the pair
|
|
/// converged. Keyed by ordinal, the two nodes' copies of entry <i>n</i> share one primary key
|
|
/// and LWW collapses them to one row — the convergence the whole-row LWW gave before, now per
|
|
/// entry. Entries go in with <c>INSERT OR IGNORE</c> (<see cref="InsertCommentSql"/>), so an
|
|
/// entry already held — written locally, or arrived from the peer — is never rewritten.
|
|
/// </para>
|
|
/// </remarks>
|
|
public static class AlarmConditionStateSchema
|
|
{
|
|
/// <summary>Table holding one condition-state row per scripted-alarm identity.</summary>
|
|
public const string StateTable = "alarm_condition_state";
|
|
|
|
/// <summary>Append-only table holding one row per comment-trail entry, keyed by alarm id + ordinal.</summary>
|
|
public const string CommentsTable = "alarm_condition_comment";
|
|
|
|
/// <summary>What <c>comments_json</c> holds once the trail lives in <see cref="CommentsTable"/>.</summary>
|
|
public const string EmptyInlineComments = "[]";
|
|
|
|
/// <summary>
|
|
/// The one append statement for <see cref="CommentsTable"/>, shared by the migration and the
|
|
/// store. <c>OR IGNORE</c> is what makes the trail append-only: an ordinal already held is
|
|
/// never rewritten. Parameters: <c>@Id @Seq @Ts @User @Kind @Text</c>.
|
|
/// </summary>
|
|
public const string InsertCommentSql =
|
|
"""
|
|
INSERT OR IGNORE INTO alarm_condition_comment
|
|
(scripted_alarm_id, seq, timestamp_utc, user_name, kind, text)
|
|
VALUES (@Id, @Seq, @Ts, @User, @Kind, @Text)
|
|
""";
|
|
|
|
private static readonly JsonSerializerOptions InlineJsonOptions = new(JsonSerializerDefaults.Web);
|
|
|
|
/// <summary>
|
|
/// Creates the state table and the comment table if they do not already exist. Idempotent.
|
|
/// </summary>
|
|
/// <param name="connection">
|
|
/// An already-open connection. <c>ILocalDb.CreateConnection()</c> hands out open,
|
|
/// pragma-configured connections — do not call <c>Open()</c> on one.
|
|
/// </param>
|
|
public static void Apply(SqliteConnection connection)
|
|
{
|
|
ArgumentNullException.ThrowIfNull(connection);
|
|
|
|
using var cmd = connection.CreateCommand();
|
|
|
|
// The four *_state columns are always written to a mapped, non-null string (Enabled/Disabled,
|
|
// Acknowledged/Unacknowledged, Confirmed/Unconfirmed, Unshelved/OneShotShelved/TimedShelved),
|
|
// so they are NOT NULL. comments_json is never null either — it is the literal "[]" since the
|
|
// trail moved to alarm_condition_comment (an old-shape row may still carry an inline trail
|
|
// until MigrateInlineComments or the next save moves it). The audit user/comment/utc columns
|
|
// and shelving_expires_utc are NULL when the alarm has no such history yet. updated_at_utc
|
|
// carries LastTransitionUtc (there is no dedicated transition column).
|
|
cmd.CommandText = """
|
|
CREATE TABLE IF NOT EXISTS alarm_condition_state (
|
|
scripted_alarm_id TEXT NOT NULL PRIMARY KEY,
|
|
enabled_state TEXT NOT NULL,
|
|
acked_state TEXT NOT NULL,
|
|
confirmed_state TEXT NOT NULL,
|
|
shelving_state TEXT NOT NULL,
|
|
shelving_expires_utc TEXT NULL,
|
|
last_ack_user TEXT NULL,
|
|
last_ack_comment TEXT NULL,
|
|
last_ack_utc TEXT NULL,
|
|
last_confirm_user TEXT NULL,
|
|
last_confirm_comment TEXT NULL,
|
|
last_confirm_utc TEXT NULL,
|
|
comments_json TEXT NOT NULL,
|
|
updated_at_utc TEXT NOT NULL
|
|
);
|
|
|
|
CREATE TABLE IF NOT EXISTS alarm_condition_comment (
|
|
scripted_alarm_id TEXT NOT NULL,
|
|
seq INTEGER NOT NULL,
|
|
timestamp_utc TEXT NOT NULL,
|
|
user_name TEXT NOT NULL,
|
|
kind TEXT NOT NULL,
|
|
text TEXT NOT NULL,
|
|
PRIMARY KEY (scripted_alarm_id, seq)
|
|
);
|
|
""";
|
|
cmd.ExecuteNonQuery();
|
|
}
|
|
|
|
/// <summary>
|
|
/// Moves every condition row's inline <c>comments_json</c> trail into
|
|
/// <see cref="CommentsTable"/> and resets the column to <c>"[]"</c>. Idempotent.
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// <para>
|
|
/// <b>Run it AFTER the tables are registered for replication</b> — it writes rows, and a
|
|
/// row written before the capture triggers exist never reaches the peer (the same
|
|
/// DDL → register → writes rule <c>AlarmSfLegacyMigrator</c> follows).
|
|
/// </para>
|
|
/// <para>
|
|
/// Each entry lands at its ordinal in the inline array via <see cref="InsertCommentSql"/>,
|
|
/// so a re-run — or a row whose trail was already (partly) moved — neither duplicates nor
|
|
/// reorders anything, and a second run finds no non-empty inline column and writes
|
|
/// nothing. Each row moves in its own transaction together with its column reset, so a
|
|
/// crash leaves every row either fully moved or untouched. A row whose inline JSON does
|
|
/// not parse is left exactly as it is (the pre-split store could not load it either)
|
|
/// rather than failing host start.
|
|
/// </para>
|
|
/// </remarks>
|
|
/// <param name="connection">An already-open connection from <c>ILocalDb.CreateConnection()</c>.</param>
|
|
/// <returns>The number of condition rows whose inline trail was moved.</returns>
|
|
public static int MigrateInlineComments(SqliteConnection connection)
|
|
{
|
|
ArgumentNullException.ThrowIfNull(connection);
|
|
|
|
var pending = new List<(string AlarmId, string Json)>();
|
|
using (var select = connection.CreateCommand())
|
|
{
|
|
select.CommandText =
|
|
"SELECT scripted_alarm_id, comments_json FROM alarm_condition_state WHERE comments_json <> @Empty";
|
|
select.Parameters.AddWithValue("@Empty", EmptyInlineComments);
|
|
using var reader = select.ExecuteReader();
|
|
while (reader.Read())
|
|
pending.Add((reader.GetString(0), reader.GetString(1)));
|
|
}
|
|
|
|
var migrated = 0;
|
|
foreach (var (alarmId, json) in pending)
|
|
{
|
|
IReadOnlyList<InlineComment> entries;
|
|
try
|
|
{
|
|
entries = ParseInlineComments(json);
|
|
}
|
|
catch (JsonException)
|
|
{
|
|
continue;
|
|
}
|
|
|
|
using var tx = connection.BeginTransaction();
|
|
for (var seq = 0; seq < entries.Count; seq++)
|
|
{
|
|
using var insert = connection.CreateCommand();
|
|
insert.Transaction = tx;
|
|
insert.CommandText = InsertCommentSql;
|
|
insert.Parameters.AddWithValue("@Id", alarmId);
|
|
insert.Parameters.AddWithValue("@Seq", seq);
|
|
insert.Parameters.AddWithValue("@Ts", FormatTimestamp(entries[seq].TimestampUtc));
|
|
insert.Parameters.AddWithValue("@User", entries[seq].User);
|
|
insert.Parameters.AddWithValue("@Kind", entries[seq].Kind);
|
|
insert.Parameters.AddWithValue("@Text", entries[seq].Text);
|
|
insert.ExecuteNonQuery();
|
|
}
|
|
|
|
using (var clear = connection.CreateCommand())
|
|
{
|
|
clear.Transaction = tx;
|
|
clear.CommandText =
|
|
"UPDATE alarm_condition_state SET comments_json = @Empty WHERE scripted_alarm_id = @Id";
|
|
clear.Parameters.AddWithValue("@Empty", EmptyInlineComments);
|
|
clear.Parameters.AddWithValue("@Id", alarmId);
|
|
clear.ExecuteNonQuery();
|
|
}
|
|
|
|
tx.Commit();
|
|
migrated++;
|
|
}
|
|
|
|
return migrated;
|
|
}
|
|
|
|
/// <summary>Formats a comment timestamp for <see cref="CommentsTable"/> (round-trip "O" text).</summary>
|
|
/// <param name="value">The timestamp.</param>
|
|
/// <returns>The round-trip text form.</returns>
|
|
public static string FormatTimestamp(DateTime value) => value.ToString("O", CultureInfo.InvariantCulture);
|
|
|
|
/// <summary>Parses a <see cref="CommentsTable"/> timestamp, preserving its <see cref="DateTimeKind"/>.</summary>
|
|
/// <param name="value">The round-trip text form.</param>
|
|
/// <returns>The timestamp.</returns>
|
|
public static DateTime ParseTimestamp(string value) =>
|
|
DateTime.Parse(value, CultureInfo.InvariantCulture, DateTimeStyles.RoundtripKind);
|
|
|
|
/// <summary>
|
|
/// Parses a legacy inline <c>comments_json</c> trail exactly as the pre-split store did
|
|
/// (<see cref="JsonSerializerDefaults.Web"/>; null strings become empty).
|
|
/// </summary>
|
|
/// <param name="json">The column value.</param>
|
|
/// <returns>The entries in trail order; empty for a blank or <c>"[]"</c> column.</returns>
|
|
/// <exception cref="JsonException">The column is not a valid trail.</exception>
|
|
public static IReadOnlyList<InlineComment> ParseInlineComments(string? json)
|
|
{
|
|
if (string.IsNullOrWhiteSpace(json)) return [];
|
|
var dtos = JsonSerializer.Deserialize<List<InlineCommentDto>>(json, InlineJsonOptions);
|
|
if (dtos is null || dtos.Count == 0) return [];
|
|
return dtos
|
|
.Select(d => new InlineComment(
|
|
d.TimestampUtc, d.User ?? string.Empty, d.Kind ?? string.Empty, d.Text ?? string.Empty))
|
|
.ToList();
|
|
}
|
|
|
|
/// <summary>One comment-trail entry as stored — the persistence-side mirror of the engine's <c>AlarmComment</c>.</summary>
|
|
/// <param name="TimestampUtc">When the action happened.</param>
|
|
/// <param name="User">Identity of the actor.</param>
|
|
/// <param name="Kind">Classification (Acknowledge, Confirm, AddComment, …).</param>
|
|
/// <param name="Text">Operator-supplied or engine-generated text.</param>
|
|
public sealed record InlineComment(DateTime TimestampUtc, string User, string Kind, string Text);
|
|
|
|
/// <summary>The pre-split on-disk shape of one entry in <c>comments_json</c>.</summary>
|
|
private sealed class InlineCommentDto
|
|
{
|
|
/// <summary>When the comment was recorded.</summary>
|
|
public DateTime TimestampUtc { get; set; }
|
|
|
|
/// <summary>Identity of the actor.</summary>
|
|
public string? User { get; set; }
|
|
|
|
/// <summary>Classification of the entry.</summary>
|
|
public string? Kind { get; set; }
|
|
|
|
/// <summary>Entry text.</summary>
|
|
public string? Text { get; set; }
|
|
}
|
|
}
|