fix(site-secrets): report every site node's secret mirror failures on the health report (M1)

A DeployedSecretsMirror failure on the STANDBY node (e.g. a master-key mismatch
that stops a deployed secret decrypting there) was only logged on that node:
site health reports are sent by the active node only, so central never saw it.

- New 12th LocalDb-replicated table secret_mirror_status (node_name PK,
  failures_json of {SecretName, Message}, checked_at; no FKs). Each node's
  mirror writes only its own row, and only when its failure set changes plus
  once per start-up (every write replicates via CDC). Best-effort: a failed
  write is a Warning and is retried next pass. Node key = INodeIdentityProvider
  NodeName, falling back to the machine name.
- SecretMirrorStatusReporter (SiteRuntime, every node, 30 s) reads every row
  into ISiteHealthCollector.SetSecretMirrorStatuses -> additive nullable
  SiteHealthReport.SecretMirrorStatuses (null = no data).
- Wire: SiteHealthReportDto.secret_mirror_statuses = 33 (presence wrapper so
  null != empty); vendored CentralControl.cs regenerated; mapper both ways.
- Central UI Health card: 'Secret mirror failing' badge + compact per-node
  node / secret / message list.
- Tests: mirror status write-on-change/first-pass/failure isolation, storage
  round-trip, reporter/collector, DTO mapper round-trip incl. null/empty,
  bUnit, replicated-table set (twelve) + FK guard wording.
This commit is contained in:
Joseph Doherty
2026-09-28 21:33:10 -04:00
parent 23a139a46c
commit 0289d51137
24 changed files with 1718 additions and 100 deletions
@@ -48,7 +48,7 @@ namespace ZB.MOM.WW.ScadaBridge.Host;
public static class SiteLocalDbSetup
{
/// <summary>
/// The eleven tables that participate in replication — registered when this node has a peer,
/// The twelve tables that participate in replication — registered when this node has a peer,
/// and deregistered (stale triggers dropped) when it does not.
/// </summary>
/// <remarks>
@@ -69,6 +69,13 @@ public static class SiteLocalDbSetup
/// <c>deployed_secrets</c> row, and replicating the ledger would make one node's progress
/// masquerade as the other's.
/// </para>
/// <para>
/// <c>secret_mirror_status</c> is the twelfth entry (review finding M1). It is per-node
/// DATA — one row per node, keyed by <c>node_name</c>, each written only by its own node —
/// but it is replicated precisely so the active node can read the standby's row and put
/// it on the site health report (only the active node sends one). Explicit primary key,
/// no FK, no BLOB columns.
/// </para>
/// </remarks>
internal static readonly string[] ReplicatedTables =
[
@@ -83,6 +90,7 @@ public static class SiteLocalDbSetup
"data_connection_definitions",
"native_alarm_state",
"deployed_secrets",
"secret_mirror_status",
];
/// <summary>
@@ -140,8 +148,9 @@ public static class SiteLocalDbSetup
// ELEVENTH entry to ReplicatedTables on an already-replicating site streams a
// snapshot of only that one table, where through 0.2.x the flag was a single
// per-database bit and the same edit would have re-streamed all eleven in full, in
// both directions. Nothing about the first boot changes: enabling replication on a
// site that has been running without it seeds all eleven at once, the flagged set is
// both directions (the same holds for secret_mirror_status, the twelfth, added by
// review finding M1). Nothing about the first boot changes: enabling replication on a
// site that has been running without it seeds all twelve at once, the flagged set is
// therefore every registered table, and the library sends that as an ordinary full
// snapshot. A hypothetical "register the whole list in one call" API would silently
// give the coarse behaviour back.