docs: site secrets ride the artifact deploy; gRPC hub retired in ScadaBridge

This commit is contained in:
Joseph Doherty
2026-09-28 20:22:45 -04:00
parent bc29dc412b
commit 0ccf68951d
8 changed files with 259 additions and 6 deletions
+28 -1
View File
@@ -162,6 +162,12 @@ flowchart TD
- Recompiles shared scripts and makes updated code available to all Script Actors.
- **Reconciles deletions**: each non-null artifact list is the complete set for its class, so after upserting, any stored row absent from the set is deleted (the artifact was deleted centrally — there is no per-artifact delete command). A removed shared script is also unregistered from the compiled script library, and a removed data connection's live DCL connection actor is stopped. A `null` list means "field not shipped" and reconciles nothing.
### Deployed Secrets (2026-09-28)
- `HandleDeployArtifacts` additionally accepts the command's `Secrets` list (`SecretArtifact`, ciphertext only) and writes the full shipped set to the `deployed_secrets` table via `SiteStorageService.UpsertDeployedSecretsAsync` — upsert every shipped name, soft-remove (`is_removed = 1`) any stored name absent from the set. A `null` `Secrets` field (pre-feature central) touches nothing and answers with no `SecretsApplied`. Storage detail: *Deployed Secrets (Site SQLite / LocalDb)* below.
- After writing the table, the active node runs `DeployedSecretsMirror.ReconcileAsync` **synchronously**, before replying — so the deploy response's `SecretsApplied` count and any apply/verify error reflect this deploy's own outcome, not a value left over from the mirror's background schedule. `SecretsApplied` counts only the names *this deploy shipped* that applied and verified cleanly; a mirror failure for an unrelated, already-broken name (a different deploy's leftover, or a stale removal) is logged at Warning and excluded — it neither fails this deploy nor reduces its `SecretsApplied`.
- No `DeployedSecretsMirror` registered (secrets deployment not composed on this node) with a non-empty `Secrets` list fails the deploy with `Secret deployment is not enabled on this site node`; with an empty `Secrets` list it still succeeds (`SecretsApplied = 0`).
- See *Deployed Secrets (Site SQLite / LocalDb)* below for the mirror itself.
### Instance Lifecycle Commands
- **Disable**: Stops the Instance Actor and its children. Retains the deployed configuration in SQLite so the instance can be re-enabled without redeployment.
- **Enable**: Creates a new Instance Actor from the stored configuration (same as startup).
@@ -424,6 +430,26 @@ The Instance Actor owns native-alarm setup alongside its computed Script and Ala
---
## Deployed Secrets (Site SQLite / LocalDb)
Site secrets ride the artifact deploy (design `docs/plans/2026-09-28-site-secrets-artifact-deploy-design.md`; see that doc's *Implementation notes (as built)* for deviations from the original design). Two tables, two different replication postures:
- **`deployed_secrets`** — the **11th LocalDb-replicated table** (`SiteLocalDbSetup.ReplicatedTables`, up from ten). Columns: `name` (PK), `row_json` (the shipped `SecretArtifact` — ciphertext, nonce, tag, wrapped DEK, `KekId`, revision; never plaintext), `revision`, `updated_utc`, `is_removed` (soft-remove flag, default 0), `deployment_id`, `applied_at`. No FKs (the standing FK guard — `SiteLocalDbReplicatedTableFkGuardTests` — applies to it like every other replicated table). `UpsertDeployedSecretsAsync` is full-set upsert-then-soft-remove, never a hard `DELETE`, so a removal is itself a replicated row change the standby node can react to.
- **`mirrored_secrets`** — a **node-local, unreplicated** ledger (`name` PK, `revision`, `mirrored_at`) recording which names *this specific node* has applied into its own `ISecretStore`. Deliberately not registered with LocalDb: it is a per-node fact (which names this node mirrored), not site-wide state, and replicating it would make one node's mirroring progress masquerade as the other's.
**`DeployedSecretsMirror`** (`SiteRuntime/Secrets/DeployedSecretsMirror.cs`) is a per-node `BackgroundService` — registered on **every** site node, not a singleton — that reconciles `deployed_secrets` into the node's own local `ISecretStore` (site nodes never run a replication-decorated store; see Component-Host.md → *Secrets*):
- **Schedule**: runs once immediately at start, then every `SiteRuntime:DeployedSecretsMirrorInterval` (default 15s, validated `> 0`), and synchronously on demand right after an active-node artifact apply (see *Deployed Secrets* under Deployment Manager Singleton above). This 15s interval is the **standby node's own convergence path** — it never receives the active node's synchronous call, only the replicated `deployed_secrets` rows via LocalDb, so it picks them up on its own next tick.
- **Apply is confirm-then-ledger, not blind.** `ApplyReplicatedAsync` is strict last-writer-wins on `(UpdatedUtc, Revision)` (a tie is a no-op) and `DeleteAsync` tombstones rather than hard-deletes — so the mirror always reads the row back after applying and only ledgers it in `mirrored_secrets` when the local ciphertext matches what was shipped. A tombstoned or clock-skewed local row that the ledger already owns is safely re-stamped (`UpdatedUtc`/`Revision` bumped past the local value) and re-applied; a live row with **different** ciphertext that the ledger does **not** already own is left completely untouched — not overwritten, not ledgered, not verified — and reported as an out-of-band conflict (`"exists on this node with a newer local value; the deployed revision was not applied"`), so an operator's genuinely local secret of the same name is never silently destroyed by a later central deploy or removal.
- **Ownership, not tombstone-state, decides re-stamping.** Once a name is ledgered in `mirrored_secrets` at any revision, this mirror treats that name as its own going forward — including re-stamping a mirror-owned name that drifted (e.g. an operator edited it locally) back to the deployed value on the next pass. Central is authoritative for every name it deploys.
- **Removal** deletes from the local store **only if the name was previously ledgered** (present in `mirrored_secrets`) — an out-of-band local secret sharing a deployed name that was never mirrored is never touched by a central-side removal.
- **Verify.** Every pass trial-resolves every live row through `ISecretResolver` (even one not re-applied this pass, so a later KEK change is still caught) — a decrypt failure names the secret and its `KekId` and is reported as a distinct error (de-spammed: Error the first time a given `(name, revision, KekId)` fails, Debug on identical repeats, Error again after an intervening success).
- **Isolation.** One row failing to apply/verify does not block any other row in the same pass.
- **Cache invalidation.** A successful apply or delete invalidates the shared `ISecretResolver` cache before verifying, so the verify step (and any script/connection reading the secret) sees the fresh value immediately rather than waiting out the resolver's cache TTL.
- **Never logs a secret value** — log lines and the reported `MirrorResult` carry only names, revisions, and `KekId`s.
---
## Enriched `AlarmStateChanged` Message
The `AlarmStateChanged` message published by both Alarm Actors and Native Alarm Actors was extended **additively** (existing consumers keep working with computed defaults):
@@ -642,10 +668,11 @@ Per Akka.NET best practices, internal actor communication uses **Tell** (fire-an
- **Site Event Logging**: Records script executions, alarm events, deployment events, instance lifecycle events.
- **Health Monitoring**: Reports script error rates and alarm evaluation error rates.
- **Local SQLite**: Persists deployed configurations, system-wide artifacts (external system definitions, database connection definitions, data connection definitions). Sites also maintain peer SQLite stores for the Store-and-Forward buffer, the site event log, the operation tracking table, and the site-local `AuditLog` (see [Component-AuditLog.md](Component-AuditLog.md)). The `AuditLog` file is purged on the same daily cadence as the others but respects the hard `ForwardState` invariant — rows still `Pending` forward are never purged, regardless of age.
- **`ZB.MOM.WW.Secrets.Abstractions`** (`ISecretStore`, `ISecretResolver`, `ISecretCacheInvalidator`): `DeployedSecretsMirror` applies deployed secrets into, and verifies against, the node's local secret store (composed by the Host — see Component-Host.md → *Secrets*). The Site Runtime depends only on the abstractions package, never the concrete store implementation.
## Interactions
- **Deployment Manager (central)**: Receives flattened configurations, system-wide artifact updates, and instance lifecycle commands.
- **Deployment Manager (central)**: Receives flattened configurations, system-wide artifact updates (including a site's referenced secrets, ciphertext only), and instance lifecycle commands.
- **Data Connection Layer**: Bidirectional — receives value updates, sends write-back commands.
- **Communication Layer**: Receives commands from central, sends deployment results, serves debug view data.
- **Store-and-Forward Engine**: Scripts route cached writes, notifications, and external system calls here.