# Site Secrets via Artifact Deploy — Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:subagent-driven-development (recommended) or superpowers-extended-cc:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Ship each site exactly the secrets its data connections reference as part of the on-demand artifact deploy, count them in the site's artifact staleness hash, log and audit every secret deployment, and retire the gRPC secrets hub for ScadaBridge sites. **Architecture:** Central scans the site's data-connection configs for `secret:` references, reads those rows (ciphertext, never plaintext) from one shared SQL-Server secret store, and adds them to `DeployArtifactsCommand` and to `ArtifactRevisionHashService`. The site's active node writes them into a new LocalDb-replicated `deployed_secrets` table. A per-node `DeployedSecretsMirror` then applies them into that node's own `secrets.db` with `ApplyReplicatedAsync` and checks each one decrypts. The standby converges through LocalDb replication plus its own mirror. **Tech Stack:** .NET 10, Akka.NET, gRPC (vendored generated code), SQLite via ZB.MOM.WW.LocalDb 0.3.4, ZB.MOM.WW.Secrets 0.6.2 → 0.6.3, xUnit, docker rig. **Spec:** `docs/plans/2026-09-28-site-secrets-artifact-deploy-design.md` (read it first — every section number cited below refers to it). ## Global Constraints - **Never plaintext on the wire, in LocalDb, in logs or in audit rows.** Only `StoredSecret` ciphertext fields cross central→site; logs/audit carry names, revisions, KekIds, counts. - **Additive contracts only:** new proto fields `DeployArtifactsCommandDto.secrets = 9`, `ArtifactDeploymentResponseDto.secrets_applied` (next free field number, `google.protobuf.Int32Value`); new record parameters appended with `= null` defaults. Vendored generated C# regenerated with `bash docker/regen-proto.sh sitecommand`; an active `` item must never be committed. - **Hash determinism:** every `Hashable*` record declares properties alphabetically (guarded by `ArtifactRevisionHashServiceTests.HashableRecords_PropertiesDeclaredAlphabetically`); lists sorted by name ordinal; null-when-empty. - **Bootstrap secrets are never deployed** (gRPC PSK `SB-GRPC-PSK-*`, LocalDb replication key, master key) — design §3.1. - **Fail closed:** a referenced secret missing/tombstoned at central fails that site's deploy before sending; a secret that fails to apply or decrypt on the active site node fails the deploy response. - **Never `await` an audit write on a script path** (not touched here, but do not introduce one). - **Do not use `git stash`** (shared across worktrees). Commit with pathspecs. - Build `dotnet build ZB.MOM.WW.ScadaBridge.slnx`; per-task run the filtered tests named in each task; full solution test + rig rebuild only in Task 13. **User decisions (already made):** - "Which secrets should a site receive with its artifacts?" → **Only referenced** (derived from the site's data-connection `secret:` refs). - "How should this relate to the existing gRPC hub pull?" → **Replace it.** - "What should the logging cover?" → **Deploy log lines, Audit rows, Hub sweep logging.** - "Central needs one authoritative secret store to deploy from" → **Shared SQL store.** --- ## File Structure | File | Responsibility | |---|---| | `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.Grpc/GrpcSecretSyncService.cs` (+ new `ISecretsHubSyncStatus.cs`, `SecretsHubSyncStatus.cs`) | Library 0.6.3: sweep outcome logging + public status | | `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/SecretArtifact.cs` (new) | Wire DTO for one ciphertext secret row | | `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/DeployArtifactsCommand.cs`, `ArtifactDeploymentResponse.cs` | Additive `Secrets` / `SecretsApplied` | | `src/ZB.MOM.WW.ScadaBridge.Communication/Protos/site_command.proto`, `Communication/SiteCommandGrpc/*` (vendored), `Communication/Grpc/SiteCommandDtoMapper.cs` | Wire mapping | | `src/ZB.MOM.WW.ScadaBridge.Commons/Types/Secrets/SecretReferenceScanner.cs` (new) | Derive referenced secret names from data-connection JSON | | `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactRevisionHashService.cs` | Secrets in the staleness hash | | `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/SiteSecretsCollector.cs` (new) | Central: scan + load rows from `ISecretStore` + map to `SecretArtifact` | | `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactDeploymentService.cs` | Build/fail-closed/stamp gating/audit/logging | | `src/ZB.MOM.WW.ScadaBridge.Host/SecretsRegistration.cs`, `StartupValidator.cs` | `Secrets:Store:Provider`, hub retirement, warnings | | `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Persistence/SiteStorageSchema.cs`, `SiteStorageService.cs` | `deployed_secrets` + `mirrored_secrets` tables and accessors | | `src/ZB.MOM.WW.ScadaBridge.Host/SiteLocalDbSetup.cs` | Register `deployed_secrets` as replicated table 11 | | `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Secrets/DeployedSecretsMirror.cs` (new) | Per-node reconcile `deployed_secrets` → `secrets.db` + verify | | `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/DeploymentManagerActor.cs` | Apply secrets in `HandleDeployArtifacts` | | `docker/docker-compose.yml`, `docker/verify-site-secrets.sh` (new) | Rig switch-over + live gate | | docs (`Component-*.md`, CLAUDE.md, register, umbrella index) | Spec sync | --- ### Task 1: Secrets library 0.6.3 — hub sweep logging + status **Goal:** Every hub sweep outcome is observable (Debug in-sync + hourly Information heartbeat + recovery line) and a public `ISecretsHubSyncStatus` exposes last sweep/success/failures; published as 0.6.3 and pinned in ScadaBridge. **Files:** - Modify: `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.Grpc/GrpcSecretSyncService.cs` - Create: `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.Grpc/ISecretsHubSyncStatus.cs` - Modify: `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.Grpc/SecretsGrpcHubClientOptions.cs` (add `InSyncLogInterval`, default 1 h, validated > 0) - Modify: the Grpc hub client DI extension (register `ISecretsHubSyncStatus` singleton) - Modify: `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/Directory.Build.props` (`0.6.3`), `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/CLAUDE.md` (version narrative) - Test: `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Replicator.Grpc.Tests/GrpcSecretSyncServiceLoggingTests.cs` (new) - Modify: `Directory.Packages.props` (ScadaBridge — all `ZB.MOM.WW.Secrets*` to 0.6.3) **Acceptance Criteria:** - [ ] An in-sync sweep (0 pulled) logs EventId 4 at Debug every time and EventId 5 `Secrets hub in sync ({Count} secret(s)); last change pulled {LastPullUtc}` at Information at most once per `InSyncLogInterval`. - [ ] The first successful sweep after ≥ 1 failure logs EventId 6 `Secrets hub sync recovered after {Failures} failed sweep(s)` at Information. - [ ] `ISecretsHubSyncStatus` returns `LastSweepUtc`, `LastSuccessUtc`, `ConsecutiveFailures`, `LastError` (message only, never a secret value) reflecting the last sweep. - [ ] Existing EventIds 1–3 unchanged; library tests green; package 0.6.3 on the Gitea feed; ScadaBridge builds on 0.6.3. **Verify:** `cd ~/Desktop/scadaproj/ZB.MOM.WW.Secrets && dotnet test --filter FullyQualifiedName~GrpcSecretSyncServiceLoggingTests` → all pass; `cd ~/Desktop/ScadaBridge && dotnet build ZB.MOM.WW.ScadaBridge.slnx` → 0 errors. **Steps:** - [ ] **Step 1: Write the failing tests** (fake hub reader returning an unchanged manifest; `FakeTimeProvider`; capture logs with a list logger): ```csharp [Fact] public async Task InSyncSweep_LogsDebugEveryTime_AndInformationHeartbeatOncePerInterval() { var time = new FakeTimeProvider(DateTimeOffset.Parse("2026-09-28T00:00:00Z")); var (svc, logs, _) = Harness.Create(time, inSyncLogInterval: TimeSpan.FromHours(1), manifestUnchanged: true); await svc.SweepForTestAsync(CancellationToken.None); time.Advance(TimeSpan.FromMinutes(30)); await svc.SweepForTestAsync(CancellationToken.None); time.Advance(TimeSpan.FromMinutes(31)); await svc.SweepForTestAsync(CancellationToken.None); Assert.Equal(3, logs.Count(l => l.EventId.Id == 4 && l.Level == LogLevel.Debug)); Assert.Equal(2, logs.Count(l => l.EventId.Id == 5 && l.Level == LogLevel.Information)); } [Fact] public async Task FirstSuccessAfterFailures_LogsRecovered_AndStatusResets() { var time = new FakeTimeProvider(); var (svc, logs, status) = Harness.Create(time, failFirst: 2); await svc.SweepForTestAsync(default); await svc.SweepForTestAsync(default); Assert.Equal(2, status.ConsecutiveFailures); await svc.SweepForTestAsync(default); Assert.Contains(logs, l => l.EventId.Id == 6 && l.Message.Contains("after 2 failed")); Assert.Equal(0, status.ConsecutiveFailures); Assert.NotNull(status.LastSuccessUtc); } ``` (`SweepForTestAsync` is an `internal` wrapper over the existing `SweepAsync`; add `InternalsVisibleTo` for the test project if missing.) - [ ] **Step 2: Run to verify they fail** — `dotnet test --filter FullyQualifiedName~GrpcSecretSyncServiceLoggingTests` → FAIL (EventIds 4–6 / status type missing). - [ ] **Step 3: Implement.** Add the status type: ```csharp namespace ZB.MOM.WW.Secrets.Replicator.Grpc; /// Point-in-time outcome of the hub follower's sweeps. Never carries secret material. public interface ISecretsHubSyncStatus { DateTimeOffset? LastSweepUtc { get; } DateTimeOffset? LastSuccessUtc { get; } int ConsecutiveFailures { get; } string? LastError { get; } } internal sealed class SecretsHubSyncStatus : ISecretsHubSyncStatus { private readonly object _gate = new(); public DateTimeOffset? LastSweepUtc { get; private set; } public DateTimeOffset? LastSuccessUtc { get; private set; } public int ConsecutiveFailures { get; private set; } public string? LastError { get; private set; } internal int RecordSuccess(DateTimeOffset now) { lock (_gate) { var prior = ConsecutiveFailures; LastSweepUtc = LastSuccessUtc = now; ConsecutiveFailures = 0; LastError = null; return prior; } } internal void RecordFailure(DateTimeOffset now, Exception ex) { lock (_gate) { LastSweepUtc = now; ConsecutiveFailures++; LastError = ex.GetType().Name + ": " + ex.Message; } } } ``` In `GrpcSecretSyncService.SweepAsync`, after the reconcile succeeds: `var prior = _status.RecordSuccess(now)`; if `prior > 0` → `LogSweepRecovered(prior)`; if `pulled > 0` → existing `LogSweepConverged(pulled)`, record `_lastPullUtc = now`; else `LogSweepInSync(count)` (Debug) and, when `now - _lastInSyncInfo >= _options.InSyncLogInterval` or it has never logged, `LogSweepInSyncHeartbeat(count, _lastPullUtc)` and set `_lastInSyncInfo = now`. In the catch: `_status.RecordFailure(now, ex)` then the existing `LogSweepFailed(ex)`. New `[LoggerMessage]` EventIds 4 (Debug), 5 (Information), 6 (Information). Register `SecretsHubSyncStatus` as a singleton for both the concrete type and `ISecretsHubSyncStatus` in the hub-client DI extension, and inject it into the service. - [ ] **Step 4: Run tests** → PASS; run the whole library suite `dotnet test` → PASS. - [ ] **Step 5: Version, pack, push, pin.** Bump `` to 0.6.3, add the 0.6.3 entry to `ZB.MOM.WW.Secrets/CLAUDE.md`, run the repo's existing pack/push script for Secrets (see `ZB.MOM.WW.Secrets/CLAUDE.md` → publishing), commit in scadaproj. In ScadaBridge set every `ZB.MOM.WW.Secrets*` `PackageVersion` in `Directory.Packages.props` to `0.6.3`, `dotnet build ZB.MOM.WW.ScadaBridge.slnx`. ```bash git -C ~/Desktop/scadaproj add ZB.MOM.WW.Secrets && git -C ~/Desktop/scadaproj commit -m "feat(secrets): 0.6.3 — hub sweep outcome logging (in-sync debug + hourly heartbeat + recovered) and ISecretsHubSyncStatus" git add Directory.Packages.props && git commit -m "build: pin ZB.MOM.WW.Secrets 0.6.3 (hub sweep logging)" ``` ```json:metadata {"files": ["~/Desktop/scadaproj/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.Grpc/GrpcSecretSyncService.cs", "~/Desktop/scadaproj/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.Grpc/ISecretsHubSyncStatus.cs", "Directory.Packages.props"], "verifyCommand": "cd ~/Desktop/scadaproj/ZB.MOM.WW.Secrets && dotnet test --filter FullyQualifiedName~GrpcSecretSyncServiceLoggingTests", "acceptanceCriteria": ["EventId 4 Debug per in-sync sweep, EventId 5 Information at most once per InSyncLogInterval", "EventId 6 recovered line after failures", "ISecretsHubSyncStatus reflects last sweep", "0.6.3 published and pinned; ScadaBridge builds"], "modelTier": "standard"} ``` --- ### Task 2: Wire contract — `SecretArtifact`, `Secrets`, `SecretsApplied` **Goal:** `DeployArtifactsCommand` carries an optional ciphertext secret list and `ArtifactDeploymentResponse` an optional applied count, end to end over gRPC, backward compatible. **Files:** - Create: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/SecretArtifact.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/DeployArtifactsCommand.cs`, `ArtifactDeploymentResponse.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.Communication/Protos/site_command.proto` (fields per Global Constraints), vendored `src/ZB.MOM.WW.ScadaBridge.Communication/SiteCommandGrpc/*` via `docker/regen-proto.sh sitecommand` - Modify: `src/ZB.MOM.WW.ScadaBridge.Communication/Grpc/SiteCommandDtoMapper.cs` (`ToProto`/`FromProto` for command and response) - Test: `tests/ZB.MOM.WW.ScadaBridge.Communication.Tests/Grpc/SiteCommandDtoMapperSecretsTests.cs` (new) **Acceptance Criteria:** - [ ] Command round-trip preserves every `SecretArtifact` field byte-for-byte; `Secrets == null` round-trips as null (field absent), empty list as empty. - [ ] Response round-trip: `SecretsApplied` null ⇄ absent wrapper, `3` ⇄ `3`. - [ ] A `DeployArtifactsCommandDto` serialized WITHOUT field 9 (old central) maps to `Secrets == null`. - [ ] No `` item left active in the csproj. **Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.Communication.Tests --filter FullyQualifiedName~SiteCommandDtoMapperSecretsTests` → all pass. **Steps:** - [ ] **Step 1: Contracts.** ```csharp namespace ZB.MOM.WW.ScadaBridge.Commons.Messages.Artifacts; /// /// One secret shipped to a site with its artifacts: the ZB.MOM.WW.Secrets StoredSecret row /// VERBATIM — ciphertext under the fleet's shared KEK, never plaintext. The site applies it /// with ISecretStore.ApplyReplicatedAsync. Design: docs/plans/2026-09-28-site-secrets-artifact-deploy-design.md §3.3. /// public sealed record SecretArtifact( string Name, string ContentType, byte[] Ciphertext, byte[] Nonce, byte[] Tag, byte[] WrappedDek, byte[] WrapNonce, byte[] WrapTag, string KekId, long Revision, DateTimeOffset CreatedUtc, DateTimeOffset UpdatedUtc, string? Description = null); ``` Append to `DeployArtifactsCommand`: `IReadOnlyList? Secrets = null` (after `Timestamp`). Append to `ArtifactDeploymentResponse`: `int? SecretsApplied = null`. - [ ] **Step 2: Proto.** Add `message SecretArtifactDto { string name = 1; string content_type = 2; bytes ciphertext = 3; bytes nonce = 4; bytes tag = 5; bytes wrapped_dek = 6; bytes wrap_nonce = 7; bytes wrap_tag = 8; string kek_id = 9; int64 revision = 10; google.protobuf.Timestamp created_utc = 11; google.protobuf.Timestamp updated_utc = 12; google.protobuf.StringValue description = 13; }`, `message SecretArtifactListDto { repeated SecretArtifactDto items = 1; }`, `SecretArtifactListDto secrets = 9; // absent => null` on `DeployArtifactsCommandDto`, and `google.protobuf.Int32Value secrets_applied = ;` on `ArtifactDeploymentResponseDto`. Run `bash docker/regen-proto.sh sitecommand`; confirm `git diff --stat` shows only the vendored files + proto and the csproj unchanged. - [ ] **Step 3: Failing tests** in `SiteCommandDtoMapperSecretsTests`: ```csharp private static SecretArtifact Sample(string name) => new(name, "text/plain", [1, 2, 3], [4], [5], [6, 7], [8], [9], "kek-A", 7, DateTimeOffset.Parse("2026-09-01T00:00:00Z"), DateTimeOffset.Parse("2026-09-02T00:00:00Z"), "d"); [Fact] public void DeployArtifacts_Secrets_RoundTrip_Verbatim() { var cmd = new DeployArtifactsCommand("dep", null, null, null, null, null, null, DateTimeOffset.UtcNow, [Sample("A"), Sample("B")]); var back = (DeployArtifactsCommand)SiteCommandDtoMapper.FromProto(SiteCommandDtoMapper.ToProto(cmd)); Assert.Equal(2, back.Secrets!.Count); Assert.Equal(cmd.Secrets![0].Ciphertext, back.Secrets[0].Ciphertext); Assert.Equal("kek-A", back.Secrets[1].KekId); Assert.Equal(7, back.Secrets[1].Revision); } [Fact] public void DeployArtifacts_NullSecrets_StaysNull() { var cmd = new DeployArtifactsCommand("dep", null, null, null, null, null, null, DateTimeOffset.UtcNow); var dto = SiteCommandDtoMapper.ToProto(cmd); Assert.Null(dto.Secrets); Assert.Null(((DeployArtifactsCommand)SiteCommandDtoMapper.FromProto(dto)).Secrets); } [Theory, InlineData(null), InlineData(3)] public void ArtifactResponse_SecretsApplied_RoundTrips(int? applied) { var r = new ArtifactDeploymentResponse("dep", "site-a", true, null, DateTimeOffset.UtcNow, applied); Assert.Equal(applied, SiteCommandDtoMapper.FromProto(SiteCommandDtoMapper.ToProto(r)).SecretsApplied); } ``` (Adapt the `FromProto`/`ToProto` call shapes to the mapper's actual overloads for the lifecycle request and the response DTO — the existing `DeployArtifacts` mapping at `SiteCommandDtoMapper.cs:395` is the pattern.) - [ ] **Step 4: Run → FAIL, implement mapper branches, run → PASS.** Byte fields via `ByteString.CopyFrom`/`.ToByteArray()`, timestamps via `Timestamp.FromDateTimeOffset`/`.ToDateTimeOffset()`, null list ⇒ leave field unset. - [ ] **Step 5: Commit** — `git add src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts src/ZB.MOM.WW.ScadaBridge.Communication tests/ZB.MOM.WW.ScadaBridge.Communication.Tests && git commit -m "feat(artifacts): additive Secrets list + SecretsApplied on the artifact deploy contract (proto field 9)"` ```json:metadata {"files": ["src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/SecretArtifact.cs", "src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/DeployArtifactsCommand.cs", "src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/ArtifactDeploymentResponse.cs", "src/ZB.MOM.WW.ScadaBridge.Communication/Protos/site_command.proto", "src/ZB.MOM.WW.ScadaBridge.Communication/Grpc/SiteCommandDtoMapper.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.Communication.Tests --filter FullyQualifiedName~SiteCommandDtoMapperSecretsTests", "acceptanceCriteria": ["SecretArtifact round-trips byte-for-byte", "null Secrets stays null / absent", "SecretsApplied null and 3 round-trip", "no active Protobuf item committed"], "modelTier": "standard"} ``` --- ### Task 3: `SecretReferenceScanner` **Goal:** A pure function returning, per site, the distinct `secret:` references found anywhere in its data connections' primary/backup JSON, each with the referencing connection names. **Files:** - Create: `src/ZB.MOM.WW.ScadaBridge.Commons/Types/Secrets/SecretReferenceScanner.cs` - Test: `tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Types/Secrets/SecretReferenceScannerTests.cs` **Acceptance Criteria:** - [ ] Finds `secret:` string values at any depth (objects and arrays), prefix case-insensitive, name trimmed. - [ ] Ignores non-string values, strings merely containing `secret:` mid-text, `${secret:X}` (node-local config form), empty names, and unparseable JSON (no throw). - [ ] Output ordinal-sorted by name; each entry lists referencing connections ordinal-sorted and de-duplicated. - [ ] Bootstrap names (`SB-GRPC-PSK-` prefix) are excluded. **Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.Commons.Tests --filter FullyQualifiedName~SecretReferenceScannerTests` → all pass. **Steps:** - [ ] **Step 1: Failing tests.** ```csharp private static DataConnectionArtifact Dc(string name, string? primary, string? backup = null) => new(name, "MxGateway", primary, backup, 3); [Fact] public void FindsNestedRefs_CaseInsensitive_AcrossPrimaryAndBackup() { var refs = SecretReferenceScanner.Scan([ Dc("GW1", """{"Endpoint":"http://gw:5000","ApiKey":"SECRET: mxgw-key "}"""), Dc("GW2", """{"a":{"b":["x","secret:mxgw-key"]}}""", """{"ApiKey":"secret:backup-key"}"""), ]); Assert.Equal(["backup-key", "mxgw-key"], refs.Select(r => r.Name)); Assert.Equal(["GW1", "GW2"], refs.Single(r => r.Name == "mxgw-key").ReferencedBy); } [Theory] [InlineData("""{"ApiKey":"${secret:node-local}"}""")] [InlineData("""{"Note":"see secret:x in docs"}""")] [InlineData("""{"ApiKey":"secret:"}""")] [InlineData("""{"Port":5000}""")] [InlineData("not json")] [InlineData("""{"Psk":"secret:SB-GRPC-PSK-site-a"}""")] public void IgnoresNonReferences(string json) => Assert.Empty(SecretReferenceScanner.Scan([Dc("C", json)])); ``` - [ ] **Step 2: Run → FAIL.** - [ ] **Step 3: Implement.** ```csharp using System.Text.Json; using ZB.MOM.WW.ScadaBridge.Commons.Messages.Artifacts; namespace ZB.MOM.WW.ScadaBridge.Commons.Types.Secrets; /// One secret a site needs, and which of its data connections reference it. public sealed record SecretReference(string Name, IReadOnlyList ReferencedBy); /// /// Derives the secrets a site needs from its data-connection configs: every JSON string VALUE of the /// exact form secret:<name>, at any depth, in primary or backup configuration. Protocol-agnostic /// so a new adapter adopting secret: refs needs no change here. Bootstrap secrets are excluded /// (design §3.1). Unparseable JSON yields nothing rather than throwing — the deploy gate owns config validity. /// public static class SecretReferenceScanner { public const string Prefix = "secret:"; private static readonly string[] BootstrapPrefixes = ["SB-GRPC-PSK-"]; public static IReadOnlyList Scan(IEnumerable connections) { var found = new SortedDictionary>(StringComparer.Ordinal); foreach (var dc in connections) foreach (var json in new[] { dc.PrimaryConfigurationJson, dc.BackupConfigurationJson }) foreach (var name in NamesIn(json)) { if (!found.TryGetValue(name, out var by)) found[name] = by = new SortedSet(StringComparer.Ordinal); by.Add(dc.Name); } return found.Select(kv => new SecretReference(kv.Key, kv.Value.ToList())).ToList(); } private static IEnumerable NamesIn(string? json) { if (string.IsNullOrWhiteSpace(json)) yield break; JsonDocument doc; try { doc = JsonDocument.Parse(json); } catch (JsonException) { yield break; } using (doc) { var stack = new Stack(); stack.Push(doc.RootElement); while (stack.Count > 0) { var e = stack.Pop(); switch (e.ValueKind) { case JsonValueKind.Object: foreach (var p in e.EnumerateObject()) stack.Push(p.Value); break; case JsonValueKind.Array: foreach (var i in e.EnumerateArray()) stack.Push(i); break; case JsonValueKind.String: var s = e.GetString()!.Trim(); if (!s.StartsWith(Prefix, StringComparison.OrdinalIgnoreCase)) break; var name = s[Prefix.Length..].Trim(); if (name.Length == 0 || BootstrapPrefixes.Any(b => name.StartsWith(b, StringComparison.OrdinalIgnoreCase))) break; yield return name; break; } } } } } ``` (`DataConnectionArtifact` property names: confirm `PrimaryConfigurationJson`/`BackupConfigurationJson` in `Commons/Messages/Artifacts/DataConnectionArtifact.cs` and adjust.) - [ ] **Step 4: Run → PASS.** **Step 5: Commit** — `git add src/ZB.MOM.WW.ScadaBridge.Commons/Types/Secrets tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Types/Secrets && git commit -m "feat(secrets): SecretReferenceScanner derives a site's secret refs from its data-connection configs"` ```json:metadata {"files": ["src/ZB.MOM.WW.ScadaBridge.Commons/Types/Secrets/SecretReferenceScanner.cs", "tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Types/Secrets/SecretReferenceScannerTests.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.Commons.Tests --filter FullyQualifiedName~SecretReferenceScannerTests", "acceptanceCriteria": ["nested/case-insensitive refs found in primary+backup", "non-refs, ${secret:}, bad JSON, bootstrap names ignored", "sorted, de-duplicated output"], "modelTier": "mechanical"} ``` --- ### Task 4: Secrets in the artifact revision hash **Goal:** `ArtifactRevisionHashService` folds the command's secrets (and missing-secret markers) into the hash; sites with no secret refs keep their existing hash. **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactRevisionHashService.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/DeployArtifactsCommand.cs` (append `IReadOnlyList? MissingSecrets = null` — central-only, never mapped to proto; document that on the parameter) - Test: `tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests/ArtifactRevisionHashServiceTests.cs` **Acceptance Criteria:** - [ ] A command with `Secrets == null` and `MissingSecrets == null` hashes identically to the pre-change golden value (add a golden-hash test computed on `main` BEFORE the change). - [ ] Changing a secret's revision, ciphertext, wrapped DEK or KekId changes the hash; reordering secrets does not. - [ ] A missing name hashes differently from the same name present. - [ ] `HashableRecords_PropertiesDeclaredAlphabetically` covers `HashableSecret` and passes. **Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests --filter FullyQualifiedName~ArtifactRevisionHashServiceTests` → all pass. **Steps:** - [ ] **Step 1: Golden test first (on unchanged code):** build a fixed command with one of each existing artifact kind, `Assert.Equal("", svc.ComputeHash(cmd))`; run once to capture the value, paste it in, commit the test alone (`test: pin pre-secrets artifact hash`). - [ ] **Step 2: Failing tests.** ```csharp [Fact] public void SecretChanges_MoveTheHash_ButOrderDoesNot() { var svc = new ArtifactRevisionHashService(); var a = Secret("A", rev: 1); var b = Secret("B", rev: 1); var baseline = svc.ComputeHash(Cmd(secrets: [a, b])); Assert.Equal(baseline, svc.ComputeHash(Cmd(secrets: [b, a]))); Assert.NotEqual(baseline, svc.ComputeHash(Cmd(secrets: [a with { Revision = 2 }, b]))); Assert.NotEqual(baseline, svc.ComputeHash(Cmd(secrets: [a with { WrappedDek = [42] }, b]))); Assert.NotEqual(baseline, svc.ComputeHash(Cmd(secrets: [a with { KekId = "kek-B" }, b]))); } [Fact] public void MissingSecret_HashesDifferentlyFromPresent() { var svc = new ArtifactRevisionHashService(); Assert.NotEqual(svc.ComputeHash(Cmd(secrets: [Secret("A", 1)])), svc.ComputeHash(Cmd(missing: ["A"]))); } ``` - [ ] **Step 3: Implement.** Add to `HashableArtifactSet` (keep alphabetical: `DataConnections, DatabaseConnections, ExternalSystems, Secrets, SharedScripts`): ```csharp Secrets = BuildHashableSecrets(command), ... private static List? BuildHashableSecrets(DeployArtifactsCommand c) { var list = new List(); foreach (var s in c.Secrets ?? []) list.Add(new HashableSecret { ContentDigest = Convert.ToHexStringLower(SHA256.HashData([.. s.Ciphertext, .. s.Nonce, .. s.Tag, .. s.WrappedDek, .. s.WrapNonce, .. s.WrapTag])), KekId = s.KekId, Missing = null, Name = s.Name, Revision = s.Revision }); foreach (var m in c.MissingSecrets ?? []) list.Add(new HashableSecret { Missing = true, Name = m }); return list.Count == 0 ? null : list.OrderBy(x => x.Name, StringComparer.Ordinal).ThenBy(x => x.Missing == true).ToList(); } private sealed record HashableSecret { public string? ContentDigest { get; init; } public string? KekId { get; init; } public bool? Missing { get; init; } public required string Name { get; init; } public long? Revision { get; init; } } ``` Update the class doc's EXCLUDED/INCLUDED paragraphs (secrets included as a ciphertext digest; `MissingSecrets` included as markers). - [ ] **Step 4: Run → PASS (incl. golden).** **Step 5: Commit** — `git commit -m "feat(artifacts): secrets and missing-secret markers in the artifact revision hash" -- src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactRevisionHashService.cs src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/DeployArtifactsCommand.cs tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests/ArtifactRevisionHashServiceTests.cs` ```json:metadata {"files": ["src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactRevisionHashService.cs", "tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests/ArtifactRevisionHashServiceTests.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests --filter FullyQualifiedName~ArtifactRevisionHashServiceTests", "acceptanceCriteria": ["golden pre-change hash unchanged when no secrets", "revision/ciphertext/wrapped DEK/KekId changes move the hash; order does not", "missing marker differs from present", "alphabetical-properties test passes"], "modelTier": "standard"} ``` --- ### Task 5: Central shared secret store — `Secrets:Store:Provider` **Goal:** A Central node can use the shared SQL-Server secret store without the hub; a clustered Central on `Sqlite` warns. **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.Host/SecretsRegistration.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.Host/StartupValidator.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.Host/appsettings.json` (`"Store": { "Provider": "Sqlite" }` under `Secrets`, with an `_comment`) - Test: `tests/ZB.MOM.WW.ScadaBridge.Host.Tests/SecretsReplicationWiringTests.cs` (extend), `tests/ZB.MOM.WW.ScadaBridge.Host.Tests/StartupValidatorTests.cs` (extend) **Acceptance Criteria:** - [ ] Central + `Provider=SqlServer` + connection string → `ISecretStore` resolves to the library's SQL-Server store; no hub endpoint mapped. - [ ] Central + `Provider=SqlServer` + blank connection string → startup failure naming `Secrets:SqlServer:ConnectionString`; a `${secret:` value is rejected (bootstrap rule, same message style as the existing Grpc-mode check). - [ ] Site role ignores `Secrets:Store:Provider` (always local SQLite). - [ ] Clustered Central (seed-nodes > 1) with `Provider=Sqlite` logs one Warning naming the per-node divergence risk; unknown provider value fails validation. **Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests --filter "FullyQualifiedName~SecretsReplicationWiringTests|FullyQualifiedName~StartupValidatorTests"` → all pass. **Steps:** - [ ] **Step 1: Failing tests** modelled on the existing Grpc-mode central tests in `SecretsReplicationWiringTests` (they already build a service collection with `AddScadaBridgeSecrets(config, SecretsNodeRole.Central)` and assert the resolved `ISecretStore` type) — add `Central_StoreProviderSqlServer_UsesSharedSqlStore_WithoutHub`, `Central_StoreProviderSqlServer_BlankConnectionString_Throws`, `Central_StoreProviderSqlServer_SecretReferenceConnectionString_Throws`, `Site_IgnoresStoreProvider`. In `StartupValidatorTests` add `ClusteredCentral_SqliteSecretStore_Warns` and `UnknownSecretStoreProvider_Fails`. - [ ] **Step 2: Run → FAIL.** - [ ] **Step 3: Implement.** In `SecretsRegistration`: add `public const string StoreProviderKey = "Secrets:Store:Provider";` and, in `AddScadaBridgeSecrets`, BEFORE the replication branch: when `role == SecretsNodeRole.Central` and the provider equals `SqlServer` (ordinal-ignore-case) and replication is not enabled, validate the connection string with the same helper the Grpc-central branch uses (blank / `${secret:` rejection), then call `services.AddZbSecretsSqlServerStore(...)` with the same arguments the Grpc-central branch passes, then `services.AddZbSecrets(...)` exactly as that branch does, and return. Keep the Grpc branch untouched here (Task 11 retires it). In `StartupValidator`: validate the provider is `Sqlite|SqlServer`; for a Central with > 1 seed node and `Sqlite`, log the Warning `Secrets use a per-node SQLite store on a clustered central — each node would deploy its own secret set to sites; set Secrets:Store:Provider=SqlServer`. - [ ] **Step 4: Run → PASS.** **Step 5: Commit** — `git commit -m "feat(secrets): Secrets:Store:Provider=SqlServer — central pair shares one SQL secret store without the hub" -- src/ZB.MOM.WW.ScadaBridge.Host tests/ZB.MOM.WW.ScadaBridge.Host.Tests` ```json:metadata {"files": ["src/ZB.MOM.WW.ScadaBridge.Host/SecretsRegistration.cs", "src/ZB.MOM.WW.ScadaBridge.Host/StartupValidator.cs", "src/ZB.MOM.WW.ScadaBridge.Host/appsettings.json"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests --filter \"FullyQualifiedName~SecretsReplicationWiringTests|FullyQualifiedName~StartupValidatorTests\"", "acceptanceCriteria": ["Central+SqlServer resolves shared SQL store, no hub", "blank or ${secret:} connection string fails naming the key", "Site ignores provider", "clustered Central on Sqlite warns; unknown provider fails"], "modelTier": "standard"} ``` --- ### Task 6: Central — collect a site's secrets, fail closed, log **Goal:** `ArtifactDeploymentService` adds each site's referenced secrets (ciphertext rows from `ISecretStore`) to its command, fails that site closed on a missing secret, logs names/revisions, and `ComputeCurrentArtifactHashesAsync` includes missing markers. **Files:** - Create: `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/SiteSecretsCollector.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactDeploymentService.cs` (`BuildDeployArtifactsCommandAsync`, `DeployCoreAsync`, `RetryForSiteAsync`, `ComputeCurrentArtifactHashesAsync`) - Modify: `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ZB.MOM.WW.ScadaBridge.DeploymentManager.csproj` (``), DeploymentManager DI registration (`SiteSecretsCollector` singleton) - Test: `tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests/SiteSecretsCollectorTests.cs` (new), `ArtifactDeploymentServiceTests.cs` (extend) **Acceptance Criteria:** - [ ] A site whose data connections reference `A` and `B` gets a command whose `Secrets` are exactly A and B's `StoredSecret` rows mapped field-for-field; other central secrets are not included. - [ ] A referenced name that is absent or `IsDeleted` → that site's `SiteArtifactResult` fails with `Secret 'X' referenced by data connection(s) C1, C2 is not in the central secret store`, nothing is sent to that site, other sites in the batch still deploy. - [ ] `ComputeCurrentArtifactHashesAsync` puts missing names in `MissingSecrets` instead of failing, so such a site reads *Out of date*. - [ ] Central logs Information `Deploying {Count} secret(s) to site {SiteId}: {Secrets}` (formatted `name@rN`) and Error for missing; a test asserts no log/exception text contains the secret's plaintext or base64 ciphertext. **Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests --filter "FullyQualifiedName~SiteSecretsCollectorTests|FullyQualifiedName~ArtifactDeploymentServiceTests"` → all pass. **Steps:** - [ ] **Step 1: Failing tests** for the collector with an in-memory fake `ISecretStore` (implement only `GetAsync`; other members throw `NotSupportedException`): ```csharp [Fact] public async Task Collect_ReturnsOnlyReferencedRows_AndNamesMissing() { var store = new FakeSecretStore(Row("A", 3), Row("B", 1, deleted: true), Row("OTHER", 1)); var result = await new SiteSecretsCollector(store).CollectAsync( [Dc("GW1", """{"ApiKey":"secret:A"}"""), Dc("GW2", """{"ApiKey":"secret:B"}""")], default); Assert.Equal(["A"], result.Secrets.Select(s => s.Name)); Assert.Equal(3, result.Secrets[0].Revision); Assert.Equal(["B"], result.Missing.Select(m => m.Name)); Assert.Equal(["GW2"], result.Missing[0].ReferencedBy); } ``` And in `ArtifactDeploymentServiceTests`: `DeployToAllSites_SiteWithMissingSecret_FailsThatSiteOnly_AndSendsNothingToIt` (assert the communication fake received no command for that site and the other site succeeded), `ComputeCurrentHashes_MissingSecret_UsesMissingMarker` (hash differs from the one computed once the secret exists), `DeployLogs_NeverContainSecretMaterial`. - [ ] **Step 2: Run → FAIL.** - [ ] **Step 3: Implement the collector.** ```csharp using ZB.MOM.WW.ScadaBridge.Commons.Messages.Artifacts; using ZB.MOM.WW.ScadaBridge.Commons.Types.Secrets; using ZB.MOM.WW.Secrets.Abstractions; namespace ZB.MOM.WW.ScadaBridge.DeploymentManager; public sealed record SiteSecretsCollection(IReadOnlyList Secrets, IReadOnlyList Missing); /// Central half of design §3.1/§3.3: which secrets a site needs and their ciphertext rows. Never decrypts. public sealed class SiteSecretsCollector(ISecretStore store) { public async Task CollectAsync(IEnumerable connections, CancellationToken ct) { var secrets = new List(); var missing = new List(); foreach (var reference in SecretReferenceScanner.Scan(connections)) { var row = await store.GetAsync(new SecretName(reference.Name), ct); if (row is null || row.IsDeleted) { missing.Add(reference); continue; } secrets.Add(new SecretArtifact(row.Name.Value, row.ContentType.ToString(), row.Ciphertext, row.Nonce, row.Tag, row.WrappedDek, row.WrapNonce, row.WrapTag, row.KekId, row.Revision, row.CreatedUtc, row.UpdatedUtc, row.Description)); } return new SiteSecretsCollection(secrets, missing); } } ``` (`SecretName` is a `readonly record struct` that normalizes its value at construction — always ship `row.Name.Value`, the normalized form. `SecretContentType` is an enum; ship `ToString()`, parsed back in Task 9.) In `BuildDeployArtifactsCommandAsync`, after mapping data connections: `var collected = await _secretsCollector.CollectAsync(dataConnectionArtifacts, ct);` and pass `Secrets: collected.Secrets.Count > 0 ? collected.Secrets : null, MissingSecrets: collected.Missing.Count > 0 ? collected.Missing.Select(m => m.Name).ToList() : null`. Return the missing list alongside the command (change the private method to return `(DeployArtifactsCommand Command, IReadOnlyList Missing)`). In `DeployCoreAsync` and `RetryForSiteAsync`: if `Missing.Count > 0`, log Error and produce a failed `SiteArtifactResult` with the message from the acceptance criteria WITHOUT calling `_communicationService`; else log the Information line (only when secrets were present) and send. `ComputeCurrentArtifactHashesAsync` keeps using the command (missing markers included). - [ ] **Step 4: Run → PASS.** **Step 5: Commit** — `git commit -m "feat(artifacts): central ships each site's referenced secrets (ciphertext) and fails closed on a missing one" -- src/ZB.MOM.WW.ScadaBridge.DeploymentManager tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests` ```json:metadata {"files": ["src/ZB.MOM.WW.ScadaBridge.DeploymentManager/SiteSecretsCollector.cs", "src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactDeploymentService.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests --filter \"FullyQualifiedName~SiteSecretsCollectorTests|FullyQualifiedName~ArtifactDeploymentServiceTests\"", "acceptanceCriteria": ["only referenced rows shipped, field-for-field", "missing/tombstoned ref fails only that site with the named message, nothing sent", "current-hash uses missing markers", "logs name@revision, never material"], "modelTier": "standard"} ``` --- ### Task 7: Central — stamp gating for old sites + audit row **Goal:** Central never stamps a site *Current* unless the site confirmed applying every shipped secret, and every secret deployment writes one audit row. **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactDeploymentService.cs` - Modify: `SiteArtifactResult` record (same file or its Commons location) — append `int? SecretsDeployed = null` - Modify: `src/ZB.MOM.WW.ScadaBridge.CLI/Commands/DeployCommands.cs` only if its table output enumerates columns (JSON output carries the new field automatically) - Test: `tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests/ArtifactDeploymentServiceTests.cs` **Acceptance Criteria:** - [ ] Command with ≥ 1 secret + response `Success=true, SecretsApplied=null` → site result is a failure `Site build predates secret deployment — upgrade the site before deploying secrets`; hash NOT stamped. - [ ] `SecretsApplied` ≠ shipped count → failure `Site applied {n} of {m} secrets`; not stamped. - [ ] Command with no secrets → old-site response (`SecretsApplied=null`) still succeeds and stamps (unchanged behaviour). - [ ] One `IAuditService.LogAsync(user, "DeploySiteSecrets", "SiteSecrets", siteIdentifier, siteName, { deploymentId, secrets:[{name,revision}], missing:[name], success, error })` per site whose command carried or would have carried secrets (incl. the fail-closed case); none for sites with no refs. Audit has no material. **Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests --filter FullyQualifiedName~ArtifactDeploymentServiceTests` → all pass. **Steps:** - [ ] **Step 1: Failing tests** — `OldSite_NullSecretsApplied_WithSecrets_FailsAndDoesNotStamp`, `PartialApply_FailsAndDoesNotStamp`, `OldSite_NoSecrets_StillStamps`, `SecretDeploy_WritesOneAuditRowPerSite_WithoutMaterial` (capture `IAuditService` calls in the existing test fake; serialize `afterState` and assert it contains the names/revisions and not the base64 ciphertext). - [ ] **Step 2: Run → FAIL.** - [ ] **Step 3: Implement** a private `static SiteArtifactResult ReconcileSecretsOutcome(DeployArtifactsCommand cmd, ArtifactDeploymentResponse r, Site site)` applied right after each `DeployArtifactsAsync` response in both paths: ```csharp var shipped = cmd.Secrets?.Count ?? 0; if (r.Success && shipped > 0 && r.SecretsApplied is null) return new(site.SiteIdentifier, site.Name, false, "Site build predates secret deployment — upgrade the site before deploying secrets", null); if (r.Success && shipped > 0 && r.SecretsApplied != shipped) return new(site.SiteIdentifier, site.Name, false, $"Site applied {r.SecretsApplied} of {shipped} secrets", r.SecretsApplied); return new(site.SiteIdentifier, site.Name, r.Success, r.ErrorMessage, shipped > 0 ? r.SecretsApplied : null); ``` Stamping already keys off `Success`, so a reconciled failure is not stamped. Add the audit call after the per-site results are known (after the existing `DeployArtifacts` audit row, same ordering rule: audit first, best-effort stamp after). - [ ] **Step 4: Run → PASS.** **Step 5: Commit** — `git commit -m "feat(artifacts): never stamp a site current unless it applied every secret; DeploySiteSecrets audit row" -- src/ZB.MOM.WW.ScadaBridge.DeploymentManager src/ZB.MOM.WW.ScadaBridge.CLI tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests` ```json:metadata {"files": ["src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactDeploymentService.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests --filter FullyQualifiedName~ArtifactDeploymentServiceTests", "acceptanceCriteria": ["old site + secrets -> failure, not stamped", "partial apply -> failure, not stamped", "no secrets -> unchanged behaviour", "one DeploySiteSecrets audit row per secret-bearing site, no material"], "modelTier": "standard"} ``` --- ### Task 8: Site storage — `deployed_secrets` (replicated) + `mirrored_secrets` (local) **Goal:** The site database has a replicated ciphertext table for deployed secrets and a node-local ledger of mirrored names, with full-set accessors. **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Persistence/SiteStorageSchema.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Persistence/SiteStorageService.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.Host/SiteLocalDbSetup.cs` (`"deployed_secrets"` appended to `ReplicatedTables`; update the "ten tables" comments to eleven; `mirrored_secrets` deliberately NOT registered, with a comment why) - Test: `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Persistence/DeployedSecretsStorageTests.cs` (new); existing `SiteLocalDbReplicatedTableFkGuardTests` must still pass **Acceptance Criteria:** - [ ] `UpsertDeployedSecretsAsync(IReadOnlyList, deploymentId)` upserts rows (`is_removed = 0`) and marks every other row `is_removed = 1`; returns `(applied names, newly removed names)`. - [ ] `GetDeployedSecretsAsync()` returns all rows incl. removed with their `SecretArtifact` round-tripped from `row_json`. - [ ] `mirrored_secrets` get/add/remove work; it is not in `ReplicatedTables`. - [ ] A security test asserts `row_json` holds only base64 ciphertext fields (deserializes to `SecretArtifact`, no extra properties). - [ ] FK guard test green (no FKs on either table). **Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests --filter FullyQualifiedName~DeployedSecretsStorageTests` and `dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests --filter FullyQualifiedName~SiteLocalDbReplicatedTableFkGuardTests` → all pass. **Steps:** - [ ] **Step 1: Schema** (append to the `CREATE TABLE IF NOT EXISTS` script): ```sql CREATE TABLE IF NOT EXISTS deployed_secrets ( name TEXT PRIMARY KEY, row_json TEXT NOT NULL, revision INTEGER NOT NULL, updated_utc TEXT NOT NULL, is_removed INTEGER NOT NULL DEFAULT 0, deployment_id TEXT NOT NULL, applied_at TEXT NOT NULL ); CREATE TABLE IF NOT EXISTS mirrored_secrets ( name TEXT PRIMARY KEY, revision INTEGER NOT NULL, mirrored_at TEXT NOT NULL ); ``` - [ ] **Step 2: Failing tests** against a temp-file `SiteStorageService` (follow the existing SiteStorageService test fixture in `SiteRuntime.Tests/Persistence`): ```csharp [Fact] public async Task Upsert_IsFullSet_MarksAbsentRowsRemoved() { var s = await Fixture.CreateAsync(); await s.UpsertDeployedSecretsAsync([Art("A", 1), Art("B", 1)], "d1"); var (applied, removed) = await s.UpsertDeployedSecretsAsync([Art("A", 2)], "d2"); Assert.Equal(["A"], applied); Assert.Equal(["B"], removed); var rows = await s.GetDeployedSecretsAsync(); Assert.Equal(2, rows.Single(r => r.Name == "A").Secret.Revision); Assert.True(rows.Single(r => r.Name == "B").IsRemoved); } ``` - [ ] **Step 3: Implement** `DeployedSecretRow(string Name, SecretArtifact Secret, bool IsRemoved, string DeploymentId)` and the accessors (single transaction; `row_json` via `JsonSerializer.Serialize(secretArtifact)` — byte[] serializes as base64). Add `GetMirroredSecretsAsync`, `SetMirroredSecretAsync(name, revision)`, `RemoveMirroredSecretAsync(name)`. Register `deployed_secrets` in `ReplicatedTables`. - [ ] **Step 4: Run → PASS.** **Step 5: Commit** — `git commit -m "feat(site): deployed_secrets (LocalDb-replicated ciphertext) + node-local mirrored_secrets ledger" -- src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Persistence src/ZB.MOM.WW.ScadaBridge.Host/SiteLocalDbSetup.cs tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Persistence` ```json:metadata {"files": ["src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Persistence/SiteStorageSchema.cs", "src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Persistence/SiteStorageService.cs", "src/ZB.MOM.WW.ScadaBridge.Host/SiteLocalDbSetup.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests --filter FullyQualifiedName~DeployedSecretsStorageTests", "acceptanceCriteria": ["full-set upsert marks absent rows removed", "rows round-trip SecretArtifact", "mirrored_secrets local only", "row_json ciphertext-only security test", "FK guard green"], "modelTier": "standard"} ``` --- ### Task 9: `DeployedSecretsMirror` — per-node apply + verify **Goal:** Every site node reconciles `deployed_secrets` into its own `secrets.db` (apply live rows, delete removed rows it previously mirrored), verifies each deployed secret decrypts, logs changes, and can be invoked synchronously. **Files:** - Create: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Secrets/DeployedSecretsMirror.cs` (service + `IHostedService` loop) - Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/SiteRuntimeOptions.cs` (+ `DeployedSecretsMirrorInterval`, default 15 s) and `SiteRuntimeOptionsValidator.cs` (> 0) - Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/ZB.MOM.WW.ScadaBridge.SiteRuntime.csproj` (`ZB.MOM.WW.Secrets.Abstractions`), SiteRuntime DI (`AddSingleton()` + `AddHostedService(sp => sp.GetRequiredService())`) - Test: `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Secrets/DeployedSecretsMirrorTests.cs` **Acceptance Criteria:** - [ ] `ReconcileAsync()` applies each live row with `ISecretStore.ApplyReplicatedAsync` (mapped back to `StoredSecret`), records it in `mirrored_secrets`, and returns a `MirrorResult(Applied, Removed, Failed)`. - [ ] A removed row is deleted from the store only if its name is in `mirrored_secrets`; a same-named out-of-band secret never mirrored is left alone. - [ ] Each live secret is trial-resolved via `ISecretResolver`; a failure (null or exception) is a `Failed` entry `Secret '{Name}' (kek {KekId}) did not decrypt on this node — the node's master key differs from central's` and is logged at Error. - [ ] Logging: Information per applied/removed secret and one summary when anything changed; Debug when in sync; never values. - [ ] The hosted loop runs once at start and every `DeployedSecretsMirrorInterval`; a reconcile exception is logged and the loop continues. **Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests --filter FullyQualifiedName~DeployedSecretsMirrorTests` → all pass. **Steps:** - [ ] **Step 1: Failing tests** using a real temp `SiteStorageService`, an in-memory fake `ISecretStore` recording `ApplyReplicatedAsync`/`DeleteAsync`, and a fake `ISecretResolver`: ```csharp [Fact] public async Task Reconcile_Applies_Verifies_AndOnlyDeletesWhatItMirrored() { var (mirror, storage, store, resolver) = Harness.Create(); store.Seed("OUT-OF-BAND"); await storage.UpsertDeployedSecretsAsync([Art("A", 1), Art("OUT-OF-BAND", 1)], "d1"); var r1 = await mirror.ReconcileAsync(default); Assert.Equal(["A", "OUT-OF-BAND"], r1.Applied.OrderBy(x => x)); await storage.UpsertDeployedSecretsAsync([], "d2"); await storage.RemoveMirroredSecretAsync("OUT-OF-BAND"); // simulate: never mirrored by us var r2 = await mirror.ReconcileAsync(default); Assert.Equal(["A"], r2.Removed); Assert.True(store.Contains("OUT-OF-BAND")); } [Fact] public async Task Reconcile_DecryptFailure_IsReportedWithKekId() { var (mirror, storage, _, resolver) = Harness.Create(); resolver.FailFor("A"); await storage.UpsertDeployedSecretsAsync([Art("A", 1, kek: "kek-central")], "d1"); var r = await mirror.ReconcileAsync(default); Assert.Contains("kek-central", Assert.Single(r.Failed)); } ``` - [ ] **Step 2: Run → FAIL.** - [ ] **Step 3: Implement.** ```csharp public sealed record MirrorResult(IReadOnlyList Applied, IReadOnlyList Removed, IReadOnlyList Failed) { public bool Changed => Applied.Count > 0 || Removed.Count > 0; } /// Design §3.5 step 2: per-node reconcile of the replicated deployed_secrets table into this node's secret store. public sealed class DeployedSecretsMirror( SiteStorageService storage, ISecretStore store, ISecretResolver resolver, IOptions options, ILogger logger, TimeProvider time) : BackgroundService { private readonly SemaphoreSlim _gate = new(1, 1); public async Task ReconcileAsync(CancellationToken ct) { await _gate.WaitAsync(ct); try { var rows = await storage.GetDeployedSecretsAsync(); var mirrored = (await storage.GetMirroredSecretsAsync()).ToDictionary(m => m.Name, m => m.Revision, StringComparer.Ordinal); List applied = [], removed = [], failed = []; foreach (var row in rows) { if (row.IsRemoved) { if (!mirrored.ContainsKey(row.Name)) continue; await store.DeleteAsync(new SecretName(row.Name), "artifact-deploy", ct); await storage.RemoveMirroredSecretAsync(row.Name); removed.Add(row.Name); logger.LogInformation("Removed deployed secret {SecretName} (no longer referenced by this site)", row.Name); continue; } if (!mirrored.TryGetValue(row.Name, out var rev) || rev != row.Secret.Revision) { await store.ApplyReplicatedAsync(ToStored(row.Secret), ct); await storage.SetMirroredSecretAsync(row.Name, row.Secret.Revision); applied.Add(row.Name); logger.LogInformation("Applied deployed secret {SecretName} r{Revision} (kek {KekId})", row.Name, row.Secret.Revision, row.Secret.KekId); } if (!await Resolves(row.Name, ct)) { var msg = $"Secret '{row.Name}' (kek {row.Secret.KekId}) did not decrypt on this node — the node's master key differs from central's"; failed.Add(msg); logger.LogError("{Failure}", msg); } } var result = new MirrorResult(applied, removed, failed); if (result.Changed) logger.LogInformation("Deployed secrets mirror: {Applied} applied, {Removed} removed, {Failed} failed", applied.Count, removed.Count, failed.Count); else logger.LogDebug("Deployed secrets mirror in sync ({Count} secret(s))", rows.Count(r => !r.IsRemoved)); return result; } finally { _gate.Release(); } } private async Task Resolves(string name, CancellationToken ct) { try { return await resolver.GetAsync(new SecretName(name), ct) is not null; } catch (Exception) { return false; } } protected override async Task ExecuteAsync(CancellationToken stoppingToken) { using var timer = new PeriodicTimer(options.Value.DeployedSecretsMirrorInterval, time); do { try { await ReconcileAsync(stoppingToken); } catch (Exception ex) when (ex is not OperationCanceledException) { logger.LogWarning(ex, "Deployed secrets mirror pass failed; retrying next interval"); } } while (await timer.WaitForNextTickAsync(stoppingToken)); } private static StoredSecret ToStored(SecretArtifact s) => new() { Name = new SecretName(s.Name), Description = s.Description, ContentType = Enum.Parse(s.ContentType, ignoreCase: false), Ciphertext = s.Ciphertext, Nonce = s.Nonce, Tag = s.Tag, WrappedDek = s.WrappedDek, WrapNonce = s.WrapNonce, WrapTag = s.WrapTag, KekId = s.KekId, Revision = s.Revision, CreatedUtc = s.CreatedUtc, UpdatedUtc = s.UpdatedUtc }; } ``` `ContentType` is the `SecretContentType` enum (`Text|ConnectionString|Json|BinaryBase64`): Task 6 writes `row.ContentType.ToString()`, this method parses it back with `Enum.Parse` — add a round-trip unit test over all four values. Note: the resolver caches (`ResolveCacheTtl` 30 s); if a trial-resolve after a rotation returns the cached old value that is still a successful decrypt of *a* value — acceptable for the verify's purpose (KEK check), document it in the class remarks. - [ ] **Step 4: Run → PASS.** **Step 5: Commit** — `git commit -m "feat(site): DeployedSecretsMirror — per-node apply + decrypt-verify of deployed secrets" -- src/ZB.MOM.WW.ScadaBridge.SiteRuntime tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Secrets` ```json:metadata {"files": ["src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Secrets/DeployedSecretsMirror.cs", "src/ZB.MOM.WW.ScadaBridge.SiteRuntime/SiteRuntimeOptions.cs", "src/ZB.MOM.WW.ScadaBridge.SiteRuntime/SiteRuntimeOptionsValidator.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests --filter FullyQualifiedName~DeployedSecretsMirrorTests", "acceptanceCriteria": ["live rows applied via ApplyReplicatedAsync and ledgered", "removed rows deleted only if previously mirrored", "decrypt failure reported with KekId at Error", "change/in-sync logging without values", "hosted loop at start + interval, survives exceptions"], "modelTier": "standard"} ``` --- ### Task 10: Site apply — secrets in `HandleDeployArtifacts` **Goal:** The active site node writes shipped secrets to `deployed_secrets`, runs the mirror synchronously, and answers with `SecretsApplied` or a failure. **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/DeploymentManagerActor.cs` (`HandleDeployArtifacts`, constructor gets `DeployedSecretsMirror?`) - Modify: the actor's Props/DI construction site to pass the mirror - Test: `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Actors/DeploymentManagerActorArtifactSecretsTests.cs` (new; follow the existing artifact-apply actor tests) **Acceptance Criteria:** - [ ] `Secrets == null` → `deployed_secrets` untouched, response `SecretsApplied = null` (old-central compatible). - [ ] `Secrets` present (incl. empty) → full-set write, synchronous `ReconcileAsync`, response `Success=true, SecretsApplied = shipped count` when the mirror reports no failures. - [ ] Any mirror failure → `Success=false`, `ErrorMessage` = the failure lines joined with `; `, `SecretsApplied` = count applied without failure. - [ ] Site Information log `Applying {Count} deployed secret(s), deploymentId={DeploymentId}` + the mirror's per-secret lines. **Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests --filter FullyQualifiedName~DeploymentManagerActorArtifactSecretsTests` → all pass. **Steps:** - [ ] **Step 1: Failing tests** (TestKit; send `DeployArtifactsCommand` with 2 secrets, expect `ArtifactDeploymentResponse` with `SecretsApplied == 2`; a resolver fake failing one → `Success == false` and the KekId in the message; `Secrets == null` → `SecretsApplied == null` and table empty). - [ ] **Step 2: Run → FAIL.** - [ ] **Step 3: Implement** inside the existing `Task.Run` block, AFTER data connections are stored (so the DCL can resolve by the time it reconnects) and BEFORE the success reply: ```csharp int? secretsApplied = null; string? secretError = null; if (command.Secrets != null && _secretsMirror != null) { _logger.LogInformation("Applying {Count} deployed secret(s), deploymentId={DeploymentId}", command.Secrets.Count, command.DeploymentId); await _storage.UpsertDeployedSecretsAsync(command.Secrets, command.DeploymentId); var mirror = await _secretsMirror.ReconcileAsync(CancellationToken.None); var failedNames = mirror.Failed.Count; secretsApplied = command.Secrets.Count - failedNames; if (failedNames > 0) secretError = string.Join("; ", mirror.Failed); } ``` and build the reply with `Success = secretError is null && `, `ErrorMessage = secretError ?? `, `SecretsApplied = secretsApplied`. If the mirror is not registered (null) but secrets were shipped, reply failure `Secret deployment is not enabled on this site node`. - [ ] **Step 4: Run → PASS**, plus the existing artifact-apply tests. **Step 5: Commit** — `git commit -m "feat(site): artifact apply writes deployed secrets and answers SecretsApplied" -- src/ZB.MOM.WW.ScadaBridge.SiteRuntime tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Actors` ```json:metadata {"files": ["src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/DeploymentManagerActor.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests --filter FullyQualifiedName~DeploymentManagerActorArtifactSecretsTests", "acceptanceCriteria": ["null Secrets leaves table and SecretsApplied null", "secrets -> full-set write + sync mirror + SecretsApplied=count", "mirror failure -> Success=false with KekId message", "apply logging"], "modelTier": "standard"} ``` --- ### Task 11: Retire the gRPC hub for ScadaBridge; switch the rig **Goal:** ScadaBridge no longer composes the secrets hub (client or server); config asking for it fails fast; the docker rig's central pair uses `Secrets:Store:Provider=SqlServer` and site-a gets secrets only via artifact deploy. **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.Host/SecretsRegistration.cs` (delete the Grpc hub/hub-client and SqlServer-replication branches; keep local SQLite + Task 5's shared store), `Program.cs` (remove `MapScadaBridgeSecretsHub`), `StartupValidator.cs` (reject `Secrets:Replication:Enabled=true`: `Secrets replication (the gRPC hub) is retired in ScadaBridge — sites receive secrets with Deploy Artifacts; central uses Secrets:Store:Provider=SqlServer`) - Modify: `src/ZB.MOM.WW.ScadaBridge.Host/appsettings.json` (remove `Replication`/`GrpcHub` blocks; keep `SqlServer.ConnectionString`/`SchemaName`; update comments) - Modify: `src/ZB.MOM.WW.ScadaBridge.Host/ZB.MOM.WW.ScadaBridge.Host.csproj` (drop `ZB.MOM.WW.Secrets.Replicator.Grpc` reference) and `Directory.Packages.props` if nothing else uses it - Modify: `tests/ZB.MOM.WW.ScadaBridge.Host.Tests/SecretsReplicationWiringTests.cs`, `MappedGrpcAuthenticationCoverageTests.cs` (hub route gone) - Modify: `docker/docker-compose.yml` (delete `x-secrets-hub-env` + `x-secrets-hub-site-env` anchors and their merges; central-a/b get `Secrets__Store__Provider: "SqlServer"` + the existing `x-secrets-hub-central-env` connection string renamed `x-secrets-central-store-env`; the rig KEK `ZB_SECRETS_MASTER_KEY` goes on central-a/b AND site-a-a/b — every node that handles deployed secrets must share it; site-b/site-c keep a DIFFERENT dev KEK so the Task 13 mismatch negative is real) **Acceptance Criteria:** - [ ] No ScadaBridge code references `AddZbSecretsGrpcHub`, `AddZbSecretsGrpcHubClient` or `MapScadaBridgeSecretsHub`. - [ ] `Secrets:Replication:Enabled=true` on either role → startup failure with the message above. - [ ] Wiring tests cover: Central Sqlite, Central SqlServer store, Site local — all green; hub-route auth coverage test updated. - [ ] `docker compose config` renders; central-a/b carry `Secrets__Store__Provider=SqlServer`; no node carries `Secrets__Replication__*` or `Secrets__GrpcHub__*`. **Verify:** `dotnet build ZB.MOM.WW.ScadaBridge.slnx` → 0 errors; `dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests` → all pass; `docker compose -f docker/docker-compose.yml config | grep -c Secrets__Replication` → `0`. **Steps:** - [ ] **Step 1:** Update tests first (replace hub-mode wiring tests with `ReplicationEnabled_IsRejected_BothRoles`), run → FAIL. - [ ] **Step 2:** Remove the hub branches/mapping/package ref, add the validator rule, run → PASS. - [ ] **Step 3:** Edit `docker-compose.yml` as listed; `docker compose -f docker/docker-compose.yml config > /dev/null` → exit 0. - [ ] **Step 4: Commit** — `git commit -m "feat(secrets)!: retire the gRPC secrets hub in ScadaBridge — sites get secrets via Deploy Artifacts; rig central on the shared SQL store" -- src/ZB.MOM.WW.ScadaBridge.Host tests/ZB.MOM.WW.ScadaBridge.Host.Tests docker/docker-compose.yml Directory.Packages.props` ```json:metadata {"files": ["src/ZB.MOM.WW.ScadaBridge.Host/SecretsRegistration.cs", "src/ZB.MOM.WW.ScadaBridge.Host/Program.cs", "src/ZB.MOM.WW.ScadaBridge.Host/StartupValidator.cs", "docker/docker-compose.yml"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests", "acceptanceCriteria": ["no hub wiring left in ScadaBridge", "Replication:Enabled=true fails startup with the retirement message", "wiring tests green", "compose renders; centrals SqlServer store; no hub env anywhere"], "modelTier": "standard"} ``` --- ### Task 12: Documentation sync **Goal:** The spec docs, CLAUDE.md, the deferred-work register and the umbrella index describe secrets-via-artifacts and the hub retirement. **Files:** - Modify: `docs/requirements/Component-DeploymentManager.md` (artifact set + secrets, fail-closed, hash, audit), `Component-SiteRuntime.md` (deployed_secrets, mirror, options), `Component-Host.md` (Secrets section: providers, hub retired, bootstrap secrets out-of-band), `Component-CentralUI.md` (Sites page: secrets make a site out of date; failure messages) - Modify: `docs/deployment/topology-guide.md` (KEK must match on every central + site node; standby convergence interval) - Modify: `CLAUDE.md` (replace the secrets/hub facts with one bullet under *Templates & Deployment*; update the LocalDb bullet "ten tables" → eleven incl. `deployed_secrets`) - Modify: `docs/plans/2026-07-08-deferred-work-register.md` (new row: vd03 + QA rollout — zbsecrets store, `Secrets:Store:Provider=SqlServer` on both centrals, KEK alignment check, recreate hand-seeded site secrets centrally, Deploy Artifacts) - Modify: `../scadaproj/CLAUDE.md` (ScadaBridge row: short line; Secrets 0.6.3 note in the libraries table) and `../scadaproj/ZB.MOM.WW.Secrets/CLAUDE.md` (ScadaBridge no longer a hub consumer) **Acceptance Criteria:** - [ ] `grep -rn "GrpcHub\|secrets hub" docs/requirements CLAUDE.md` returns only historical/retired mentions. - [ ] Register row lists the four rollout prerequisites and the trigger ("user asks for the vd03/QA secrets window"). - [ ] Umbrella index row stays a few lines. **Verify:** `grep -rn "deployed_secrets" docs/requirements/Component-SiteRuntime.md CLAUDE.md` → matches in both; `git diff --stat` shows only doc files. **Steps:** - [ ] **Step 1:** Edit the docs listed (quote the design doc sections rather than restating detail). - [ ] **Step 2:** `git diff` review. **Step 3: Commit** in each repo — `git commit -m "docs: site secrets ride the artifact deploy; gRPC hub retired in ScadaBridge" -- docs CLAUDE.md` and `git -C ../scadaproj commit -m "index: ScadaBridge secrets via artifact deploy; Secrets 0.6.3" -- CLAUDE.md ZB.MOM.WW.Secrets/CLAUDE.md` ```json:metadata {"files": ["docs/requirements/Component-DeploymentManager.md", "docs/requirements/Component-SiteRuntime.md", "docs/requirements/Component-Host.md", "CLAUDE.md", "docs/plans/2026-07-08-deferred-work-register.md"], "verifyCommand": "grep -rn deployed_secrets docs/requirements/Component-SiteRuntime.md CLAUDE.md", "acceptanceCriteria": ["component docs + topology guide updated", "CLAUDE.md bullet + LocalDb eleven tables", "register rollout row with 4 prerequisites", "umbrella + Secrets CLAUDE.md updated"], "modelTier": "mechanical"} ``` --- ### Task 13: Full test run, rig rebuild, live gate **Goal:** The whole solution is green and the rebuilt docker rig demonstrates secrets deploying with artifacts end to end, including standby convergence, staleness, removal, fail-closed and KEK mismatch. **Files:** - Create: `docker/verify-site-secrets.sh` (idempotent gate script, prints PASS/FAIL per check, exits non-zero on any FAIL) - Create: `docs/plans/2026-09-28-site-secrets-artifact-deploy-live-gate.md` (results record) **Acceptance Criteria:** - [ ] `dotnet test ZB.MOM.WW.ScadaBridge.slnx` green except the known env-only Playwright residue (list any failure explicitly). - [ ] `bash docker/deploy.sh` succeeds; all 8 nodes `/health/ready` 200. - [ ] Gate checks (each PASS): 1. Create secret `sb-gate-mxgw` at central (Secrets CLI or `/admin/secrets`) and an unbound MxGateway data connection on site-a with `"ApiKey":"secret:sb-gate-mxgw"` → Sites page/`site list` shows site-a **Out of date**, site-b/site-c unchanged. 2. `deploy artifacts --site-id ` → success, `secretsDeployed: 1`; site-a **Current**. 3. Both site-a nodes' `secrets.db` contain `sb-gate-mxgw` at the central revision (`docker exec` + `sqlite3` on the library table, ciphertext compare by SHA-256) — the standby within 30 s. 4. Rotate the secret at central → site-a **Out of date** → deploy → both nodes at the new revision. 5. Remove the `secret:` reference → deploy → both nodes no longer hold the secret; an out-of-band secret seeded on site-a-a beforehand is still present. 6. Reference a non-existent `sb-gate-missing` → deploy site-a fails with the named message; site-a stays **Out of date**; nothing changed on the site. 7. Same connection on site-b (different dev KEK) → deploy fails with the KEK-mismatch message naming the KekId. 8. Audit log has one `DeploySiteSecrets` row per secret-bearing deploy; `grep` of all 8 nodes' logs for the secret's plaintext and base64 ciphertext → 0 hits. - [ ] Gate script and results record committed; gate objects cleaned up by the script's teardown. **Verify:** `bash docker/verify-site-secrets.sh` → `PASS 8/8`. **Steps:** - [ ] **Step 1:** `dotnet build ZB.MOM.WW.ScadaBridge.slnx && dotnet test ZB.MOM.WW.ScadaBridge.slnx` — record counts. - [ ] **Step 2:** `bash docker/deploy.sh` (from the primary checkout, not a worktree — see memory: deploy.sh from a worktree rehomes mounts); wait for readiness. - [ ] **Step 3:** Write `docker/verify-site-secrets.sh` implementing checks 1–8 with the CLI (`src/ZB.MOM.WW.ScadaBridge.CLI/bin/Debug/net10.0/scadabridge --url http://localhost:9000 --username multi-role --password password`), `docker exec`, and a trap-based teardown; run it until PASS 8/8, fixing product defects in their owning task's files (commit separately with the task tag). - [ ] **Step 4:** Write the results record (method, per-check evidence, timings incl. standby convergence), commit — `git commit -m "test(rig): site secrets artifact-deploy live gate PASS 8/8" -- docker/verify-site-secrets.sh docs/plans/2026-09-28-site-secrets-artifact-deploy-live-gate.md` ```json:metadata {"files": ["docker/verify-site-secrets.sh", "docs/plans/2026-09-28-site-secrets-artifact-deploy-live-gate.md"], "verifyCommand": "bash docker/verify-site-secrets.sh", "acceptanceCriteria": ["solution tests green (env-only residue listed)", "rig rebuilt, 8 nodes ready", "gate checks 1-8 PASS incl. standby convergence, removal, fail-closed, KEK mismatch, audit + no-material grep", "gate script + results record committed"], "modelTier": "standard"} ``` --- ## Dependencies - Task 1 is independent (library) — run first so the rig transition in Task 11/13 is observable. - Task 2 → Tasks 4, 6, 7, 9, 10. Task 3 → Task 6. Task 4 → Task 6. Task 5 → Task 6 (central store DI) and Task 11. - Task 6 → Task 7. Task 8 → Task 9 → Task 10. - Tasks 7 and 10 → Task 11 → Task 12 → Task 13.