ScadaBridge/src/ZB.MOM.WW.ScadaBridge.Host/SiteLocalDbSetup.cs

using Microsoft.Extensions.Configuration;
using ZB.MOM.WW.LocalDb;
using ZB.MOM.WW.ScadaBridge.SiteEventLogging;
using ZB.MOM.WW.ScadaBridge.SiteRuntime.Persistence;
using ZB.MOM.WW.ScadaBridge.SiteRuntime.Tracking;
using ZB.MOM.WW.ScadaBridge.StoreAndForward;

namespace ZB.MOM.WW.ScadaBridge.Host;

/// <summary>
/// Initializes the consolidated site database — the single <c>ZB.MOM.WW.LocalDb</c>-managed
/// file that holds the site node's replicated state.
/// </summary>
/// <remarks>
/// <para>
/// <b>Ordering is load-bearing.</b> Table DDL must run BEFORE
/// <c>RegisterReplicated</c>, and every row that should replicate must be written
/// AFTER it. Capture is trigger-based change-data-capture: the library neither
/// captures nor snapshots rows that already existed when the triggers were installed,
/// so a row written before registration is invisible to the peer forever — silently,
/// with no error anywhere.
/// </para>
/// <para>
/// <b>Registration is conditional</b> on the node actually having a replication peer —
/// see <see cref="ReplicationIsConfigured"/>. An unreplicated node used to install the
/// full CDC trigger set anyway and then pay for it on every single write, forever, for an
/// oplog nothing ever drains. The ordering rule above is unaffected: when registration is
/// skipped there is no capture to be too late for, and the legacy migrator still runs
/// (its rows simply do not replicate, which is what "unreplicated node" means).
/// </para>
/// <para>
/// <b>The unreplicated branch actively cleans up</b> rather than merely abstaining:
/// <c>DeregisterReplicated</c> (LocalDb 0.2.0) drops any capture triggers a previous build
/// left in the file and prunes that table's oplog and row-version rows. Skipping
/// registration alone would have left a file registered under an older build capturing
/// forever, which is exactly the node that has no peer to capture for.
/// </para>
/// <para>
/// Runs inside the <c>AddZbLocalDb</c> onReady callback, once, before any caller
/// receives the <see cref="ILocalDb"/> singleton. onReady is a synchronous
/// <c>Action&lt;ILocalDb&gt;</c>, hence the direct <c>CreateConnection()</c> rather than
/// blocking on the async execute API. A throw here propagates out of the first
/// <c>GetRequiredService&lt;ILocalDb&gt;()</c> and fails host startup, which is the
/// intent: a site node that cannot establish its schema must not come up half-working.
/// </para>
/// </remarks>
public static class SiteLocalDbSetup
{
    /// <summary>
    /// The ten tables that participate in replication — registered when this node has a peer,
    /// and deregistered (stale triggers dropped) when it does not.
    /// </summary>
    /// <remarks>
    /// <c>notification_lists</c> and <c>smtp_configurations</c> are created by the schema but
    /// deliberately absent here; see the rationale in <see cref="OnReady"/>. The list is shared by
    /// both branches on purpose: a table added to one and not the other would either register
    /// without ever being cleaned up, or be cleaned up on a node that never registered it.
    /// </remarks>
    private static readonly string[] ReplicatedTables =
    [
        "OperationTracking",
        "site_events",
        "sf_messages",
        "deployed_configurations",
        "static_attribute_overrides",
        "shared_scripts",
        "external_systems",
        "database_connections",
        "data_connection_definitions",
        "native_alarm_state",
    ];

    /// <summary>
    /// Creates the site node's tables, opts them into change capture when this node
    /// replicates, and migrates any pre-Phase-1 databases in.
    /// </summary>
    /// <param name="db">The LocalDb instance being initialized.</param>
    /// <param name="config">
    /// Configuration, for the replication predicate, the legacy migrator's old path keys,
    /// and the node name.
    /// </param>
    public static void OnReady(ILocalDb db, IConfiguration config)
    {
        ArgumentNullException.ThrowIfNull(db);
        ArgumentNullException.ThrowIfNull(config);

        using (var connection = db.CreateConnection())
        {
            OperationTrackingSchema.Apply(connection);
            SiteEventLogSchema.Apply(connection);

            // Phase 2: the site's configuration tables and the store-and-forward buffer
            // now live in this file too.
            SiteStorageSchema.Apply(connection);
            StoreAndForwardSchema.Apply(connection);
        }

        if (ReplicationIsConfigured(config))
        {
            // Every table below qualifies: each has an explicit primary key (RegisterReplicated
            // rejects tables without one) and no BLOB columns (which json_object cannot
            // capture); the two composite-PK tables are fine, since RegisterReplicated orders
            // multi-column PKs by ordinal. Registration is idempotent and installs the capture
            // triggers.
            //
            // The Phase 2 members of the list — the store-and-forward buffer and the seven site
            // configuration tables — replaced the bespoke SiteReplicationActor and StoreAndForward
            // ReplicationService, which shipped hand-written Add/Remove/Park/Requeue operations
            // over Akka; both were deleted in the same commit that added these lines, so the
            // two mechanisms never ran at once.
            //
            // baselineExistingRows: true is what makes turning replication ON for a site that has
            // been running WITHOUT it actually converge. Capture is change-data-capture and the
            // snapshot streamer pages from __localdb_row_version, so rows written before the first
            // registration exist in neither — a late opt-in used to replicate only writes made
            // after the restart, silently and forever. Baselining seeds the ledger at the LWW
            // floor (HLC 0, this node's id) and flags a snapshot resync, so a baselined row loses
            // to any genuine remote write of the same key and wins only where the peer has no
            // version at all. It is idempotent (ON CONFLICT DO NOTHING) and therefore free on
            // every boot after the first, which is why it is unconditional rather than a flag.
            foreach (var table in ReplicatedTables)
                db.RegisterReplicated(table, baselineExistingRows: true);

            // notification_lists and smtp_configurations are created but deliberately NOT
            // registered — see ReplicatedTables. They are permanently empty by design — the
            // site-side write paths were removed on 2026-07-10, the legacy migrator skips them,
            // and the active node's artifact apply purges them on every deploy. Registering them
            // would open a standing replication channel whose only historical payload was
            // plaintext SMTP passwords, in exchange for replicating nothing. Anyone adding them
            // here should first establish that a site has a legitimate reason to hold SMTP
            // credentials at all.
        }
        else
        {
            // Not merely "do not register": actively remove capture this node must not pay for.
            // The file may have been registered by an EARLIER build (every build before WP1.3
            // registered unconditionally), in which case its triggers are still there, still
            // running two INSERTs plus a json_object of the full row inside every write
            // transaction, still appending to an oplog with no reader. DeregisterReplicated is
            // idempotent and deliberately usable on a table this process never registered, so a
            // never-replicated file simply reports nothing to clean.
            //
            // Symmetry is not a concern on this branch: this node has no peer, so there is no
            // handshake digest for a one-sided deregistration to fail. And re-enabling later is
            // safe despite the ledger prune, because the registration branch above baselines.
            var cleaned = 0;
            foreach (var table in ReplicatedTables)
            {
                if (db.DeregisterReplicated(table)) cleaned++;
            }

            if (cleaned > 0)
            {
                // Serilog's static logger, as in SecretsRegistration: this runs inside the
                // AddZbLocalDb singleton factory, where no ILogger can be resolved without
                // nesting a service resolution inside a singleton construction. Program.cs
                // configures Log.Logger before the service graph is built.
                Serilog.Log.Information(
                    "LocalDb: removed stale change-capture from {TableCount} table(s) — this node has no "
                    + "replication peer configured, so the triggers a previous build installed have been "
                    + "dropped and their oplog/row-version rows pruned.",
                    cleaned);
            }
        }

        // AFTER registration, so migrated rows enter the oplog and reach the peer like
        // any other write. Before it, they would be invisible to replication forever.
        // Deliberately OUTSIDE the guard: an unreplicated node still has to absorb its
        // pre-Phase-1 files, and there is no peer for its rows to be invisible to.
        SiteLocalDbLegacyMigrator.Migrate(db, config);
    }

    /// <summary>
    /// Whether this node participates in LocalDb replication, and therefore needs the CDC
    /// capture triggers at all.
    /// </summary>
    /// <remarks>
    /// <para>
    /// <b>Either key counts, and that is not redundancy — it is the only predicate that is
    /// true on BOTH halves of a replicated pair.</b> Replication is one bidirectional
    /// stream that exactly one side dials, so only the initiator sets
    /// <c>LocalDb:Replication:PeerAddress</c>; the passive node has an <c>ApiKey</c> and
    /// nothing else. (The rig is the reference: site-a node-a carries PeerAddress + ApiKey,
    /// site-a node-b carries ApiKey alone, and site-b/site-c carry no <c>Replication</c>
    /// section at all.) Keying on PeerAddress alone would strip capture from every passive
    /// node — its local writes would stop reaching the initiator, and the pair would
    /// silently converge in one direction only.
    /// </para>
    /// <para>
    /// The keys are read straight from configuration rather than through
    /// <c>IOptions&lt;ReplicationOptions&gt;</c> because this runs inside the
    /// <c>AddZbLocalDb</c> factory, where resolving another option would nest a service
    /// resolution inside a singleton construction. The section name matches what
    /// <c>AddZbLocalDbReplication</c> binds, so the two cannot disagree about which node
    /// replicates.
    /// </para>
    /// <para>
    /// <b><c>internal</c>, not private, so there is exactly one copy of this rule.</b>
    /// <c>SiteServiceRegistration</c> reuses it to supply SiteEventLogging's
    /// <c>SiteEventLogReplicationCheck</c> — the predicate that decides whether the daily
    /// <c>site_events</c> purge adds its "expect a transient oplog backlog" operator note.
    /// A second hand-rolled PeerAddress-OR-ApiKey test would be free to drift out of step
    /// with the one that actually installs the triggers.
    /// </para>
    /// <para>
    /// <b>Both directions of a change to this predicate are now handled at boot</b> (LocalDb
    /// 0.2.0). Flipping it to false deregisters, so a file registered by an older build stops
    /// capturing on the next start instead of paying for triggers forever; flipping it to true
    /// baselines, so a site that has been running unreplicated converges on the rows already in
    /// its file and not merely on writes made after the restart. Seeding one node from the
    /// other's database beforehand is no longer required — only still advisable where BOTH
    /// files hold their own version of the same key, since two baselined rows both sit at HLC
    /// 0 and the node-id tie-break then decides arbitrarily which content survives. Either
    /// way both nodes must be changed together: deregistration is only symmetric — and the
    /// handshake digest only agrees — if both sides do it. See
    /// <c>docs/deployment/topology-guide.md</c>.
    /// </para>
    /// </remarks>
    internal static bool ReplicationIsConfigured(IConfiguration config)
    {
        var section = config.GetSection("LocalDb:Replication");

        return !string.IsNullOrWhiteSpace(section["PeerAddress"])
            || !string.IsNullOrWhiteSpace(section["ApiKey"]);
    }
}