Joseph Doherty
990 lines
70 KiB
Markdown
990 lines
70 KiB
Markdown
# Site Secrets via Artifact Deploy — Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:subagent-driven-development (recommended) or superpowers-extended-cc:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** Ship each site exactly the secrets its data connections reference as part of the on-demand artifact deploy, count them in the site's artifact staleness hash, log and audit every secret deployment, and retire the gRPC secrets hub for ScadaBridge sites.
|
||
|
||
**Architecture:** Central scans the site's data-connection configs for `secret:<name>` references, reads those rows (ciphertext, never plaintext) from one shared SQL-Server secret store, and adds them to `DeployArtifactsCommand` and to `ArtifactRevisionHashService`. The site's active node writes them into a new LocalDb-replicated `deployed_secrets` table. A per-node `DeployedSecretsMirror` then applies them into that node's own `secrets.db` with `ApplyReplicatedAsync` and checks each one decrypts. The standby converges through LocalDb replication plus its own mirror.
|
||
|
||
**Tech Stack:** .NET 10, Akka.NET, gRPC (vendored generated code), SQLite via ZB.MOM.WW.LocalDb 0.3.4, ZB.MOM.WW.Secrets 0.6.2 → 0.6.3, xUnit, docker rig.
|
||
|
||
**Spec:** `docs/plans/2026-09-28-site-secrets-artifact-deploy-design.md` (read it first — every section number cited below refers to it).
|
||
|
||
## Global Constraints
|
||
|
||
- **Never plaintext on the wire, in LocalDb, in logs or in audit rows.** Only `StoredSecret` ciphertext fields cross central→site; logs/audit carry names, revisions, KekIds, counts.
|
||
- **Additive contracts only:** new proto fields `DeployArtifactsCommandDto.secrets = 9`, `ArtifactDeploymentResponseDto.secrets_applied` (next free field number, `google.protobuf.Int32Value`); new record parameters appended with `= null` defaults. Vendored generated C# regenerated with `bash docker/regen-proto.sh sitecommand`; an active `<Protobuf>` item must never be committed.
|
||
- **Hash determinism:** every `Hashable*` record declares properties alphabetically (guarded by `ArtifactRevisionHashServiceTests.HashableRecords_PropertiesDeclaredAlphabetically`); lists sorted by name ordinal; null-when-empty.
|
||
- **Bootstrap secrets are never deployed** (gRPC PSK `SB-GRPC-PSK-*`, LocalDb replication key, master key) — design §3.1.
|
||
- **Fail closed:** a referenced secret missing/tombstoned at central fails that site's deploy before sending; a secret that fails to apply or decrypt on the active site node fails the deploy response.
|
||
- **Never `await` an audit write on a script path** (not touched here, but do not introduce one).
|
||
- **Do not use `git stash`** (shared across worktrees). Commit with pathspecs.
|
||
- Build `dotnet build ZB.MOM.WW.ScadaBridge.slnx`; per-task run the filtered tests named in each task; full solution test + rig rebuild only in Task 13.
|
||
|
||
**User decisions (already made):**
|
||
- "Which secrets should a site receive with its artifacts?" → **Only referenced** (derived from the site's data-connection `secret:` refs).
|
||
- "How should this relate to the existing gRPC hub pull?" → **Replace it.**
|
||
- "What should the logging cover?" → **Deploy log lines, Audit rows, Hub sweep logging.**
|
||
- "Central needs one authoritative secret store to deploy from" → **Shared SQL store.**
|
||
|
||
---
|
||
|
||
## File Structure
|
||
|
||
| File | Responsibility |
|
||
|---|---|
|
||
| `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.Grpc/GrpcSecretSyncService.cs` (+ new `ISecretsHubSyncStatus.cs`, `SecretsHubSyncStatus.cs`) | Library 0.6.3: sweep outcome logging + public status |
|
||
| `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/SecretArtifact.cs` (new) | Wire DTO for one ciphertext secret row |
|
||
| `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/DeployArtifactsCommand.cs`, `ArtifactDeploymentResponse.cs` | Additive `Secrets` / `SecretsApplied` |
|
||
| `src/ZB.MOM.WW.ScadaBridge.Communication/Protos/site_command.proto`, `Communication/SiteCommandGrpc/*` (vendored), `Communication/Grpc/SiteCommandDtoMapper.cs` | Wire mapping |
|
||
| `src/ZB.MOM.WW.ScadaBridge.Commons/Types/Secrets/SecretReferenceScanner.cs` (new) | Derive referenced secret names from data-connection JSON |
|
||
| `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactRevisionHashService.cs` | Secrets in the staleness hash |
|
||
| `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/SiteSecretsCollector.cs` (new) | Central: scan + load rows from `ISecretStore` + map to `SecretArtifact` |
|
||
| `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactDeploymentService.cs` | Build/fail-closed/stamp gating/audit/logging |
|
||
| `src/ZB.MOM.WW.ScadaBridge.Host/SecretsRegistration.cs`, `StartupValidator.cs` | `Secrets:Store:Provider`, hub retirement, warnings |
|
||
| `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Persistence/SiteStorageSchema.cs`, `SiteStorageService.cs` | `deployed_secrets` + `mirrored_secrets` tables and accessors |
|
||
| `src/ZB.MOM.WW.ScadaBridge.Host/SiteLocalDbSetup.cs` | Register `deployed_secrets` as replicated table 11 |
|
||
| `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Secrets/DeployedSecretsMirror.cs` (new) | Per-node reconcile `deployed_secrets` → `secrets.db` + verify |
|
||
| `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/DeploymentManagerActor.cs` | Apply secrets in `HandleDeployArtifacts` |
|
||
| `docker/docker-compose.yml`, `docker/verify-site-secrets.sh` (new) | Rig switch-over + live gate |
|
||
| docs (`Component-*.md`, CLAUDE.md, register, umbrella index) | Spec sync |
|
||
|
||
---
|
||
|
||
### Task 1: Secrets library 0.6.3 — hub sweep logging + status
|
||
|
||
**Goal:** Every hub sweep outcome is observable (Debug in-sync + hourly Information heartbeat + recovery line) and a public `ISecretsHubSyncStatus` exposes last sweep/success/failures; published as 0.6.3 and pinned in ScadaBridge.
|
||
|
||
**Files:**
|
||
- Modify: `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.Grpc/GrpcSecretSyncService.cs`
|
||
- Create: `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.Grpc/ISecretsHubSyncStatus.cs`
|
||
- Modify: `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.Grpc/SecretsGrpcHubClientOptions.cs` (add `InSyncLogInterval`, default 1 h, validated > 0)
|
||
- Modify: the Grpc hub client DI extension (register `ISecretsHubSyncStatus` singleton)
|
||
- Modify: `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/Directory.Build.props` (`<Version>0.6.3</Version>`), `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/CLAUDE.md` (version narrative)
|
||
- Test: `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Replicator.Grpc.Tests/GrpcSecretSyncServiceLoggingTests.cs` (new)
|
||
- Modify: `Directory.Packages.props` (ScadaBridge — all `ZB.MOM.WW.Secrets*` to 0.6.3)
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] An in-sync sweep (0 pulled) logs EventId 4 at Debug every time and EventId 5 `Secrets hub in sync ({Count} secret(s)); last change pulled {LastPullUtc}` at Information at most once per `InSyncLogInterval`.
|
||
- [ ] The first successful sweep after ≥ 1 failure logs EventId 6 `Secrets hub sync recovered after {Failures} failed sweep(s)` at Information.
|
||
- [ ] `ISecretsHubSyncStatus` returns `LastSweepUtc`, `LastSuccessUtc`, `ConsecutiveFailures`, `LastError` (message only, never a secret value) reflecting the last sweep.
|
||
- [ ] Existing EventIds 1–3 unchanged; library tests green; package 0.6.3 on the Gitea feed; ScadaBridge builds on 0.6.3.
|
||
|
||
**Verify:** `cd ~/Desktop/scadaproj/ZB.MOM.WW.Secrets && dotnet test --filter FullyQualifiedName~GrpcSecretSyncServiceLoggingTests` → all pass; `cd ~/Desktop/ScadaBridge && dotnet build ZB.MOM.WW.ScadaBridge.slnx` → 0 errors.
|
||
|
||
**Steps:**
|
||
|
||
- [ ] **Step 1: Write the failing tests** (fake hub reader returning an unchanged manifest; `FakeTimeProvider`; capture logs with a list logger):
|
||
|
||
```csharp
|
||
[Fact]
|
||
public async Task InSyncSweep_LogsDebugEveryTime_AndInformationHeartbeatOncePerInterval()
|
||
{
|
||
var time = new FakeTimeProvider(DateTimeOffset.Parse("2026-09-28T00:00:00Z"));
|
||
var (svc, logs, _) = Harness.Create(time, inSyncLogInterval: TimeSpan.FromHours(1), manifestUnchanged: true);
|
||
await svc.SweepForTestAsync(CancellationToken.None);
|
||
time.Advance(TimeSpan.FromMinutes(30));
|
||
await svc.SweepForTestAsync(CancellationToken.None);
|
||
time.Advance(TimeSpan.FromMinutes(31));
|
||
await svc.SweepForTestAsync(CancellationToken.None);
|
||
Assert.Equal(3, logs.Count(l => l.EventId.Id == 4 && l.Level == LogLevel.Debug));
|
||
Assert.Equal(2, logs.Count(l => l.EventId.Id == 5 && l.Level == LogLevel.Information));
|
||
}
|
||
|
||
[Fact]
|
||
public async Task FirstSuccessAfterFailures_LogsRecovered_AndStatusResets()
|
||
{
|
||
var time = new FakeTimeProvider();
|
||
var (svc, logs, status) = Harness.Create(time, failFirst: 2);
|
||
await svc.SweepForTestAsync(default); await svc.SweepForTestAsync(default);
|
||
Assert.Equal(2, status.ConsecutiveFailures);
|
||
await svc.SweepForTestAsync(default);
|
||
Assert.Contains(logs, l => l.EventId.Id == 6 && l.Message.Contains("after 2 failed"));
|
||
Assert.Equal(0, status.ConsecutiveFailures);
|
||
Assert.NotNull(status.LastSuccessUtc);
|
||
}
|
||
```
|
||
|
||
(`SweepForTestAsync` is an `internal` wrapper over the existing `SweepAsync`; add `InternalsVisibleTo` for the test project if missing.)
|
||
|
||
- [ ] **Step 2: Run to verify they fail** — `dotnet test --filter FullyQualifiedName~GrpcSecretSyncServiceLoggingTests` → FAIL (EventIds 4–6 / status type missing).
|
||
|
||
- [ ] **Step 3: Implement.** Add the status type:
|
||
|
||
```csharp
|
||
namespace ZB.MOM.WW.Secrets.Replicator.Grpc;
|
||
|
||
/// <summary>Point-in-time outcome of the hub follower's sweeps. Never carries secret material.</summary>
|
||
public interface ISecretsHubSyncStatus
|
||
{
|
||
DateTimeOffset? LastSweepUtc { get; }
|
||
DateTimeOffset? LastSuccessUtc { get; }
|
||
int ConsecutiveFailures { get; }
|
||
string? LastError { get; }
|
||
}
|
||
|
||
internal sealed class SecretsHubSyncStatus : ISecretsHubSyncStatus
|
||
{
|
||
private readonly object _gate = new();
|
||
public DateTimeOffset? LastSweepUtc { get; private set; }
|
||
public DateTimeOffset? LastSuccessUtc { get; private set; }
|
||
public int ConsecutiveFailures { get; private set; }
|
||
public string? LastError { get; private set; }
|
||
|
||
internal int RecordSuccess(DateTimeOffset now)
|
||
{
|
||
lock (_gate) { var prior = ConsecutiveFailures; LastSweepUtc = LastSuccessUtc = now; ConsecutiveFailures = 0; LastError = null; return prior; }
|
||
}
|
||
|
||
internal void RecordFailure(DateTimeOffset now, Exception ex)
|
||
{
|
||
lock (_gate) { LastSweepUtc = now; ConsecutiveFailures++; LastError = ex.GetType().Name + ": " + ex.Message; }
|
||
}
|
||
}
|
||
```
|
||
|
||
In `GrpcSecretSyncService.SweepAsync`, after the reconcile succeeds: `var prior = _status.RecordSuccess(now)`; if `prior > 0` → `LogSweepRecovered(prior)`; if `pulled > 0` → existing `LogSweepConverged(pulled)`, record `_lastPullUtc = now`; else `LogSweepInSync(count)` (Debug) and, when `now - _lastInSyncInfo >= _options.InSyncLogInterval` or it has never logged, `LogSweepInSyncHeartbeat(count, _lastPullUtc)` and set `_lastInSyncInfo = now`. In the catch: `_status.RecordFailure(now, ex)` then the existing `LogSweepFailed(ex)`. New `[LoggerMessage]` EventIds 4 (Debug), 5 (Information), 6 (Information). Register `SecretsHubSyncStatus` as a singleton for both the concrete type and `ISecretsHubSyncStatus` in the hub-client DI extension, and inject it into the service.
|
||
|
||
- [ ] **Step 4: Run tests** → PASS; run the whole library suite `dotnet test` → PASS.
|
||
|
||
- [ ] **Step 5: Version, pack, push, pin.** Bump `<Version>` to 0.6.3, add the 0.6.3 entry to `ZB.MOM.WW.Secrets/CLAUDE.md`, run the repo's existing pack/push script for Secrets (see `ZB.MOM.WW.Secrets/CLAUDE.md` → publishing), commit in scadaproj. In ScadaBridge set every `ZB.MOM.WW.Secrets*` `PackageVersion` in `Directory.Packages.props` to `0.6.3`, `dotnet build ZB.MOM.WW.ScadaBridge.slnx`.
|
||
|
||
```bash
|
||
git -C ~/Desktop/scadaproj add ZB.MOM.WW.Secrets && git -C ~/Desktop/scadaproj commit -m "feat(secrets): 0.6.3 — hub sweep outcome logging (in-sync debug + hourly heartbeat + recovered) and ISecretsHubSyncStatus"
|
||
git add Directory.Packages.props && git commit -m "build: pin ZB.MOM.WW.Secrets 0.6.3 (hub sweep logging)"
|
||
```
|
||
|
||
```json:metadata
|
||
{"files": ["~/Desktop/scadaproj/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.Grpc/GrpcSecretSyncService.cs", "~/Desktop/scadaproj/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.Grpc/ISecretsHubSyncStatus.cs", "Directory.Packages.props"], "verifyCommand": "cd ~/Desktop/scadaproj/ZB.MOM.WW.Secrets && dotnet test --filter FullyQualifiedName~GrpcSecretSyncServiceLoggingTests", "acceptanceCriteria": ["EventId 4 Debug per in-sync sweep, EventId 5 Information at most once per InSyncLogInterval", "EventId 6 recovered line after failures", "ISecretsHubSyncStatus reflects last sweep", "0.6.3 published and pinned; ScadaBridge builds"], "modelTier": "standard"}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 2: Wire contract — `SecretArtifact`, `Secrets`, `SecretsApplied`
|
||
|
||
**Goal:** `DeployArtifactsCommand` carries an optional ciphertext secret list and `ArtifactDeploymentResponse` an optional applied count, end to end over gRPC, backward compatible.
|
||
|
||
**Files:**
|
||
- Create: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/SecretArtifact.cs`
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/DeployArtifactsCommand.cs`, `ArtifactDeploymentResponse.cs`
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Communication/Protos/site_command.proto` (fields per Global Constraints), vendored `src/ZB.MOM.WW.ScadaBridge.Communication/SiteCommandGrpc/*` via `docker/regen-proto.sh sitecommand`
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Communication/Grpc/SiteCommandDtoMapper.cs` (`ToProto`/`FromProto` for command and response)
|
||
- Test: `tests/ZB.MOM.WW.ScadaBridge.Communication.Tests/Grpc/SiteCommandDtoMapperSecretsTests.cs` (new)
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] Command round-trip preserves every `SecretArtifact` field byte-for-byte; `Secrets == null` round-trips as null (field absent), empty list as empty.
|
||
- [ ] Response round-trip: `SecretsApplied` null ⇄ absent wrapper, `3` ⇄ `3`.
|
||
- [ ] A `DeployArtifactsCommandDto` serialized WITHOUT field 9 (old central) maps to `Secrets == null`.
|
||
- [ ] No `<Protobuf>` item left active in the csproj.
|
||
|
||
**Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.Communication.Tests --filter FullyQualifiedName~SiteCommandDtoMapperSecretsTests` → all pass.
|
||
|
||
**Steps:**
|
||
|
||
- [ ] **Step 1: Contracts.**
|
||
|
||
```csharp
|
||
namespace ZB.MOM.WW.ScadaBridge.Commons.Messages.Artifacts;
|
||
|
||
/// <summary>
|
||
/// One secret shipped to a site with its artifacts: the ZB.MOM.WW.Secrets StoredSecret row
|
||
/// VERBATIM — ciphertext under the fleet's shared KEK, never plaintext. The site applies it
|
||
/// with ISecretStore.ApplyReplicatedAsync. Design: docs/plans/2026-09-28-site-secrets-artifact-deploy-design.md §3.3.
|
||
/// </summary>
|
||
public sealed record SecretArtifact(
|
||
string Name,
|
||
string ContentType,
|
||
byte[] Ciphertext,
|
||
byte[] Nonce,
|
||
byte[] Tag,
|
||
byte[] WrappedDek,
|
||
byte[] WrapNonce,
|
||
byte[] WrapTag,
|
||
string KekId,
|
||
long Revision,
|
||
DateTimeOffset CreatedUtc,
|
||
DateTimeOffset UpdatedUtc,
|
||
string? Description = null);
|
||
```
|
||
|
||
Append to `DeployArtifactsCommand`: `IReadOnlyList<SecretArtifact>? Secrets = null` (after `Timestamp`). Append to `ArtifactDeploymentResponse`: `int? SecretsApplied = null`.
|
||
|
||
- [ ] **Step 2: Proto.** Add `message SecretArtifactDto { string name = 1; string content_type = 2; bytes ciphertext = 3; bytes nonce = 4; bytes tag = 5; bytes wrapped_dek = 6; bytes wrap_nonce = 7; bytes wrap_tag = 8; string kek_id = 9; int64 revision = 10; google.protobuf.Timestamp created_utc = 11; google.protobuf.Timestamp updated_utc = 12; google.protobuf.StringValue description = 13; }`, `message SecretArtifactListDto { repeated SecretArtifactDto items = 1; }`, `SecretArtifactListDto secrets = 9; // absent => null` on `DeployArtifactsCommandDto`, and `google.protobuf.Int32Value secrets_applied = <next free>;` on `ArtifactDeploymentResponseDto`. Run `bash docker/regen-proto.sh sitecommand`; confirm `git diff --stat` shows only the vendored files + proto and the csproj unchanged.
|
||
|
||
- [ ] **Step 3: Failing tests** in `SiteCommandDtoMapperSecretsTests`:
|
||
|
||
```csharp
|
||
private static SecretArtifact Sample(string name) => new(name, "text/plain",
|
||
[1, 2, 3], [4], [5], [6, 7], [8], [9], "kek-A", 7,
|
||
DateTimeOffset.Parse("2026-09-01T00:00:00Z"), DateTimeOffset.Parse("2026-09-02T00:00:00Z"), "d");
|
||
|
||
[Fact]
|
||
public void DeployArtifacts_Secrets_RoundTrip_Verbatim()
|
||
{
|
||
var cmd = new DeployArtifactsCommand("dep", null, null, null, null, null, null,
|
||
DateTimeOffset.UtcNow, [Sample("A"), Sample("B")]);
|
||
var back = (DeployArtifactsCommand)SiteCommandDtoMapper.FromProto(SiteCommandDtoMapper.ToProto(cmd));
|
||
Assert.Equal(2, back.Secrets!.Count);
|
||
Assert.Equal(cmd.Secrets![0].Ciphertext, back.Secrets[0].Ciphertext);
|
||
Assert.Equal("kek-A", back.Secrets[1].KekId);
|
||
Assert.Equal(7, back.Secrets[1].Revision);
|
||
}
|
||
|
||
[Fact]
|
||
public void DeployArtifacts_NullSecrets_StaysNull()
|
||
{
|
||
var cmd = new DeployArtifactsCommand("dep", null, null, null, null, null, null, DateTimeOffset.UtcNow);
|
||
var dto = SiteCommandDtoMapper.ToProto(cmd);
|
||
Assert.Null(dto.Secrets);
|
||
Assert.Null(((DeployArtifactsCommand)SiteCommandDtoMapper.FromProto(dto)).Secrets);
|
||
}
|
||
|
||
[Theory, InlineData(null), InlineData(3)]
|
||
public void ArtifactResponse_SecretsApplied_RoundTrips(int? applied)
|
||
{
|
||
var r = new ArtifactDeploymentResponse("dep", "site-a", true, null, DateTimeOffset.UtcNow, applied);
|
||
Assert.Equal(applied, SiteCommandDtoMapper.FromProto(SiteCommandDtoMapper.ToProto(r)).SecretsApplied);
|
||
}
|
||
```
|
||
|
||
(Adapt the `FromProto`/`ToProto` call shapes to the mapper's actual overloads for the lifecycle request and the response DTO — the existing `DeployArtifacts` mapping at `SiteCommandDtoMapper.cs:395` is the pattern.)
|
||
|
||
- [ ] **Step 4: Run → FAIL, implement mapper branches, run → PASS.** Byte fields via `ByteString.CopyFrom`/`.ToByteArray()`, timestamps via `Timestamp.FromDateTimeOffset`/`.ToDateTimeOffset()`, null list ⇒ leave field unset.
|
||
|
||
- [ ] **Step 5: Commit** — `git add src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts src/ZB.MOM.WW.ScadaBridge.Communication tests/ZB.MOM.WW.ScadaBridge.Communication.Tests && git commit -m "feat(artifacts): additive Secrets list + SecretsApplied on the artifact deploy contract (proto field 9)"`
|
||
|
||
```json:metadata
|
||
{"files": ["src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/SecretArtifact.cs", "src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/DeployArtifactsCommand.cs", "src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/ArtifactDeploymentResponse.cs", "src/ZB.MOM.WW.ScadaBridge.Communication/Protos/site_command.proto", "src/ZB.MOM.WW.ScadaBridge.Communication/Grpc/SiteCommandDtoMapper.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.Communication.Tests --filter FullyQualifiedName~SiteCommandDtoMapperSecretsTests", "acceptanceCriteria": ["SecretArtifact round-trips byte-for-byte", "null Secrets stays null / absent", "SecretsApplied null and 3 round-trip", "no active Protobuf item committed"], "modelTier": "standard"}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 3: `SecretReferenceScanner`
|
||
|
||
**Goal:** A pure function returning, per site, the distinct `secret:<name>` references found anywhere in its data connections' primary/backup JSON, each with the referencing connection names.
|
||
|
||
**Files:**
|
||
- Create: `src/ZB.MOM.WW.ScadaBridge.Commons/Types/Secrets/SecretReferenceScanner.cs`
|
||
- Test: `tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Types/Secrets/SecretReferenceScannerTests.cs`
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] Finds `secret:` string values at any depth (objects and arrays), prefix case-insensitive, name trimmed.
|
||
- [ ] Ignores non-string values, strings merely containing `secret:` mid-text, `${secret:X}` (node-local config form), empty names, and unparseable JSON (no throw).
|
||
- [ ] Output ordinal-sorted by name; each entry lists referencing connections ordinal-sorted and de-duplicated.
|
||
- [ ] Bootstrap names (`SB-GRPC-PSK-` prefix) are excluded.
|
||
|
||
**Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.Commons.Tests --filter FullyQualifiedName~SecretReferenceScannerTests` → all pass.
|
||
|
||
**Steps:**
|
||
|
||
- [ ] **Step 1: Failing tests.**
|
||
|
||
```csharp
|
||
private static DataConnectionArtifact Dc(string name, string? primary, string? backup = null)
|
||
=> new(name, "MxGateway", primary, backup, 3);
|
||
|
||
[Fact]
|
||
public void FindsNestedRefs_CaseInsensitive_AcrossPrimaryAndBackup()
|
||
{
|
||
var refs = SecretReferenceScanner.Scan([
|
||
Dc("GW1", """{"Endpoint":"http://gw:5000","ApiKey":"SECRET: mxgw-key "}"""),
|
||
Dc("GW2", """{"a":{"b":["x","secret:mxgw-key"]}}""", """{"ApiKey":"secret:backup-key"}"""),
|
||
]);
|
||
Assert.Equal(["backup-key", "mxgw-key"], refs.Select(r => r.Name));
|
||
Assert.Equal(["GW1", "GW2"], refs.Single(r => r.Name == "mxgw-key").ReferencedBy);
|
||
}
|
||
|
||
[Theory]
|
||
[InlineData("""{"ApiKey":"${secret:node-local}"}""")]
|
||
[InlineData("""{"Note":"see secret:x in docs"}""")]
|
||
[InlineData("""{"ApiKey":"secret:"}""")]
|
||
[InlineData("""{"Port":5000}""")]
|
||
[InlineData("not json")]
|
||
[InlineData("""{"Psk":"secret:SB-GRPC-PSK-site-a"}""")]
|
||
public void IgnoresNonReferences(string json)
|
||
=> Assert.Empty(SecretReferenceScanner.Scan([Dc("C", json)]));
|
||
```
|
||
|
||
- [ ] **Step 2: Run → FAIL.**
|
||
|
||
- [ ] **Step 3: Implement.**
|
||
|
||
```csharp
|
||
using System.Text.Json;
|
||
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Artifacts;
|
||
|
||
namespace ZB.MOM.WW.ScadaBridge.Commons.Types.Secrets;
|
||
|
||
/// <summary>One secret a site needs, and which of its data connections reference it.</summary>
|
||
public sealed record SecretReference(string Name, IReadOnlyList<string> ReferencedBy);
|
||
|
||
/// <summary>
|
||
/// Derives the secrets a site needs from its data-connection configs: every JSON string VALUE of the
|
||
/// exact form <c>secret:<name></c>, at any depth, in primary or backup configuration. Protocol-agnostic
|
||
/// so a new adapter adopting <c>secret:</c> refs needs no change here. Bootstrap secrets are excluded
|
||
/// (design §3.1). Unparseable JSON yields nothing rather than throwing — the deploy gate owns config validity.
|
||
/// </summary>
|
||
public static class SecretReferenceScanner
|
||
{
|
||
public const string Prefix = "secret:";
|
||
private static readonly string[] BootstrapPrefixes = ["SB-GRPC-PSK-"];
|
||
|
||
public static IReadOnlyList<SecretReference> Scan(IEnumerable<DataConnectionArtifact> connections)
|
||
{
|
||
var found = new SortedDictionary<string, SortedSet<string>>(StringComparer.Ordinal);
|
||
foreach (var dc in connections)
|
||
foreach (var json in new[] { dc.PrimaryConfigurationJson, dc.BackupConfigurationJson })
|
||
foreach (var name in NamesIn(json))
|
||
{
|
||
if (!found.TryGetValue(name, out var by)) found[name] = by = new SortedSet<string>(StringComparer.Ordinal);
|
||
by.Add(dc.Name);
|
||
}
|
||
return found.Select(kv => new SecretReference(kv.Key, kv.Value.ToList())).ToList();
|
||
}
|
||
|
||
private static IEnumerable<string> NamesIn(string? json)
|
||
{
|
||
if (string.IsNullOrWhiteSpace(json)) yield break;
|
||
JsonDocument doc;
|
||
try { doc = JsonDocument.Parse(json); } catch (JsonException) { yield break; }
|
||
using (doc)
|
||
{
|
||
var stack = new Stack<JsonElement>(); stack.Push(doc.RootElement);
|
||
while (stack.Count > 0)
|
||
{
|
||
var e = stack.Pop();
|
||
switch (e.ValueKind)
|
||
{
|
||
case JsonValueKind.Object: foreach (var p in e.EnumerateObject()) stack.Push(p.Value); break;
|
||
case JsonValueKind.Array: foreach (var i in e.EnumerateArray()) stack.Push(i); break;
|
||
case JsonValueKind.String:
|
||
var s = e.GetString()!.Trim();
|
||
if (!s.StartsWith(Prefix, StringComparison.OrdinalIgnoreCase)) break;
|
||
var name = s[Prefix.Length..].Trim();
|
||
if (name.Length == 0 || BootstrapPrefixes.Any(b => name.StartsWith(b, StringComparison.OrdinalIgnoreCase))) break;
|
||
yield return name;
|
||
break;
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
(`DataConnectionArtifact` property names: confirm `PrimaryConfigurationJson`/`BackupConfigurationJson` in `Commons/Messages/Artifacts/DataConnectionArtifact.cs` and adjust.)
|
||
|
||
- [ ] **Step 4: Run → PASS.** **Step 5: Commit** — `git add src/ZB.MOM.WW.ScadaBridge.Commons/Types/Secrets tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Types/Secrets && git commit -m "feat(secrets): SecretReferenceScanner derives a site's secret refs from its data-connection configs"`
|
||
|
||
```json:metadata
|
||
{"files": ["src/ZB.MOM.WW.ScadaBridge.Commons/Types/Secrets/SecretReferenceScanner.cs", "tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Types/Secrets/SecretReferenceScannerTests.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.Commons.Tests --filter FullyQualifiedName~SecretReferenceScannerTests", "acceptanceCriteria": ["nested/case-insensitive refs found in primary+backup", "non-refs, ${secret:}, bad JSON, bootstrap names ignored", "sorted, de-duplicated output"], "modelTier": "mechanical"}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 4: Secrets in the artifact revision hash
|
||
|
||
**Goal:** `ArtifactRevisionHashService` folds the command's secrets (and missing-secret markers) into the hash; sites with no secret refs keep their existing hash.
|
||
|
||
**Files:**
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactRevisionHashService.cs`
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/DeployArtifactsCommand.cs` (append `IReadOnlyList<string>? MissingSecrets = null` — central-only, never mapped to proto; document that on the parameter)
|
||
- Test: `tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests/ArtifactRevisionHashServiceTests.cs`
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] A command with `Secrets == null` and `MissingSecrets == null` hashes identically to the pre-change golden value (add a golden-hash test computed on `main` BEFORE the change).
|
||
- [ ] Changing a secret's revision, ciphertext, wrapped DEK or KekId changes the hash; reordering secrets does not.
|
||
- [ ] A missing name hashes differently from the same name present.
|
||
- [ ] `HashableRecords_PropertiesDeclaredAlphabetically` covers `HashableSecret` and passes.
|
||
|
||
**Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests --filter FullyQualifiedName~ArtifactRevisionHashServiceTests` → all pass.
|
||
|
||
**Steps:**
|
||
|
||
- [ ] **Step 1: Golden test first (on unchanged code):** build a fixed command with one of each existing artifact kind, `Assert.Equal("<value printed on first run>", svc.ComputeHash(cmd))`; run once to capture the value, paste it in, commit the test alone (`test: pin pre-secrets artifact hash`).
|
||
|
||
- [ ] **Step 2: Failing tests.**
|
||
|
||
```csharp
|
||
[Fact]
|
||
public void SecretChanges_MoveTheHash_ButOrderDoesNot()
|
||
{
|
||
var svc = new ArtifactRevisionHashService();
|
||
var a = Secret("A", rev: 1); var b = Secret("B", rev: 1);
|
||
var baseline = svc.ComputeHash(Cmd(secrets: [a, b]));
|
||
Assert.Equal(baseline, svc.ComputeHash(Cmd(secrets: [b, a])));
|
||
Assert.NotEqual(baseline, svc.ComputeHash(Cmd(secrets: [a with { Revision = 2 }, b])));
|
||
Assert.NotEqual(baseline, svc.ComputeHash(Cmd(secrets: [a with { WrappedDek = [42] }, b])));
|
||
Assert.NotEqual(baseline, svc.ComputeHash(Cmd(secrets: [a with { KekId = "kek-B" }, b])));
|
||
}
|
||
|
||
[Fact]
|
||
public void MissingSecret_HashesDifferentlyFromPresent()
|
||
{
|
||
var svc = new ArtifactRevisionHashService();
|
||
Assert.NotEqual(svc.ComputeHash(Cmd(secrets: [Secret("A", 1)])), svc.ComputeHash(Cmd(missing: ["A"])));
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 3: Implement.** Add to `HashableArtifactSet` (keep alphabetical: `DataConnections, DatabaseConnections, ExternalSystems, Secrets, SharedScripts`):
|
||
|
||
```csharp
|
||
Secrets = BuildHashableSecrets(command),
|
||
...
|
||
private static List<HashableSecret>? BuildHashableSecrets(DeployArtifactsCommand c)
|
||
{
|
||
var list = new List<HashableSecret>();
|
||
foreach (var s in c.Secrets ?? [])
|
||
list.Add(new HashableSecret
|
||
{
|
||
ContentDigest = Convert.ToHexStringLower(SHA256.HashData([.. s.Ciphertext, .. s.Nonce, .. s.Tag, .. s.WrappedDek, .. s.WrapNonce, .. s.WrapTag])),
|
||
KekId = s.KekId, Missing = null, Name = s.Name, Revision = s.Revision
|
||
});
|
||
foreach (var m in c.MissingSecrets ?? [])
|
||
list.Add(new HashableSecret { Missing = true, Name = m });
|
||
return list.Count == 0 ? null : list.OrderBy(x => x.Name, StringComparer.Ordinal).ThenBy(x => x.Missing == true).ToList();
|
||
}
|
||
|
||
private sealed record HashableSecret
|
||
{
|
||
public string? ContentDigest { get; init; }
|
||
public string? KekId { get; init; }
|
||
public bool? Missing { get; init; }
|
||
public required string Name { get; init; }
|
||
public long? Revision { get; init; }
|
||
}
|
||
```
|
||
|
||
Update the class doc's EXCLUDED/INCLUDED paragraphs (secrets included as a ciphertext digest; `MissingSecrets` included as markers).
|
||
|
||
- [ ] **Step 4: Run → PASS (incl. golden).** **Step 5: Commit** — `git commit -m "feat(artifacts): secrets and missing-secret markers in the artifact revision hash" -- src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactRevisionHashService.cs src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/DeployArtifactsCommand.cs tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests/ArtifactRevisionHashServiceTests.cs`
|
||
|
||
```json:metadata
|
||
{"files": ["src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactRevisionHashService.cs", "tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests/ArtifactRevisionHashServiceTests.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests --filter FullyQualifiedName~ArtifactRevisionHashServiceTests", "acceptanceCriteria": ["golden pre-change hash unchanged when no secrets", "revision/ciphertext/wrapped DEK/KekId changes move the hash; order does not", "missing marker differs from present", "alphabetical-properties test passes"], "modelTier": "standard"}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 5: Central shared secret store — `Secrets:Store:Provider`
|
||
|
||
**Goal:** A Central node can use the shared SQL-Server secret store without the hub; a clustered Central on `Sqlite` warns.
|
||
|
||
**Files:**
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Host/SecretsRegistration.cs`
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Host/StartupValidator.cs`
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Host/appsettings.json` (`"Store": { "Provider": "Sqlite" }` under `Secrets`, with an `_comment`)
|
||
- Test: `tests/ZB.MOM.WW.ScadaBridge.Host.Tests/SecretsReplicationWiringTests.cs` (extend), `tests/ZB.MOM.WW.ScadaBridge.Host.Tests/StartupValidatorTests.cs` (extend)
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] Central + `Provider=SqlServer` + connection string → `ISecretStore` resolves to the library's SQL-Server store; no hub endpoint mapped.
|
||
- [ ] Central + `Provider=SqlServer` + blank connection string → startup failure naming `Secrets:SqlServer:ConnectionString`; a `${secret:` value is rejected (bootstrap rule, same message style as the existing Grpc-mode check).
|
||
- [ ] Site role ignores `Secrets:Store:Provider` (always local SQLite).
|
||
- [ ] Clustered Central (seed-nodes > 1) with `Provider=Sqlite` logs one Warning naming the per-node divergence risk; unknown provider value fails validation.
|
||
|
||
**Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests --filter "FullyQualifiedName~SecretsReplicationWiringTests|FullyQualifiedName~StartupValidatorTests"` → all pass.
|
||
|
||
**Steps:**
|
||
|
||
- [ ] **Step 1: Failing tests** modelled on the existing Grpc-mode central tests in `SecretsReplicationWiringTests` (they already build a service collection with `AddScadaBridgeSecrets(config, SecretsNodeRole.Central)` and assert the resolved `ISecretStore` type) — add `Central_StoreProviderSqlServer_UsesSharedSqlStore_WithoutHub`, `Central_StoreProviderSqlServer_BlankConnectionString_Throws`, `Central_StoreProviderSqlServer_SecretReferenceConnectionString_Throws`, `Site_IgnoresStoreProvider`. In `StartupValidatorTests` add `ClusteredCentral_SqliteSecretStore_Warns` and `UnknownSecretStoreProvider_Fails`.
|
||
|
||
- [ ] **Step 2: Run → FAIL.**
|
||
|
||
- [ ] **Step 3: Implement.** In `SecretsRegistration`: add `public const string StoreProviderKey = "Secrets:Store:Provider";` and, in `AddScadaBridgeSecrets`, BEFORE the replication branch: when `role == SecretsNodeRole.Central` and the provider equals `SqlServer` (ordinal-ignore-case) and replication is not enabled, validate the connection string with the same helper the Grpc-central branch uses (blank / `${secret:` rejection), then call `services.AddZbSecretsSqlServerStore(...)` with the same arguments the Grpc-central branch passes, then `services.AddZbSecrets(...)` exactly as that branch does, and return. Keep the Grpc branch untouched here (Task 11 retires it). In `StartupValidator`: validate the provider is `Sqlite|SqlServer`; for a Central with > 1 seed node and `Sqlite`, log the Warning `Secrets use a per-node SQLite store on a clustered central — each node would deploy its own secret set to sites; set Secrets:Store:Provider=SqlServer`.
|
||
|
||
- [ ] **Step 4: Run → PASS.** **Step 5: Commit** — `git commit -m "feat(secrets): Secrets:Store:Provider=SqlServer — central pair shares one SQL secret store without the hub" -- src/ZB.MOM.WW.ScadaBridge.Host tests/ZB.MOM.WW.ScadaBridge.Host.Tests`
|
||
|
||
```json:metadata
|
||
{"files": ["src/ZB.MOM.WW.ScadaBridge.Host/SecretsRegistration.cs", "src/ZB.MOM.WW.ScadaBridge.Host/StartupValidator.cs", "src/ZB.MOM.WW.ScadaBridge.Host/appsettings.json"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests --filter \"FullyQualifiedName~SecretsReplicationWiringTests|FullyQualifiedName~StartupValidatorTests\"", "acceptanceCriteria": ["Central+SqlServer resolves shared SQL store, no hub", "blank or ${secret:} connection string fails naming the key", "Site ignores provider", "clustered Central on Sqlite warns; unknown provider fails"], "modelTier": "standard"}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 6: Central — collect a site's secrets, fail closed, log
|
||
|
||
**Goal:** `ArtifactDeploymentService` adds each site's referenced secrets (ciphertext rows from `ISecretStore`) to its command, fails that site closed on a missing secret, logs names/revisions, and `ComputeCurrentArtifactHashesAsync` includes missing markers.
|
||
|
||
**Files:**
|
||
- Create: `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/SiteSecretsCollector.cs`
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactDeploymentService.cs` (`BuildDeployArtifactsCommandAsync`, `DeployCoreAsync`, `RetryForSiteAsync`, `ComputeCurrentArtifactHashesAsync`)
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ZB.MOM.WW.ScadaBridge.DeploymentManager.csproj` (`<PackageReference Include="ZB.MOM.WW.Secrets.Abstractions" />`), DeploymentManager DI registration (`SiteSecretsCollector` singleton)
|
||
- Test: `tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests/SiteSecretsCollectorTests.cs` (new), `ArtifactDeploymentServiceTests.cs` (extend)
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] A site whose data connections reference `A` and `B` gets a command whose `Secrets` are exactly A and B's `StoredSecret` rows mapped field-for-field; other central secrets are not included.
|
||
- [ ] A referenced name that is absent or `IsDeleted` → that site's `SiteArtifactResult` fails with `Secret 'X' referenced by data connection(s) C1, C2 is not in the central secret store`, nothing is sent to that site, other sites in the batch still deploy.
|
||
- [ ] `ComputeCurrentArtifactHashesAsync` puts missing names in `MissingSecrets` instead of failing, so such a site reads *Out of date*.
|
||
- [ ] Central logs Information `Deploying {Count} secret(s) to site {SiteId}: {Secrets}` (formatted `name@rN`) and Error for missing; a test asserts no log/exception text contains the secret's plaintext or base64 ciphertext.
|
||
|
||
**Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests --filter "FullyQualifiedName~SiteSecretsCollectorTests|FullyQualifiedName~ArtifactDeploymentServiceTests"` → all pass.
|
||
|
||
**Steps:**
|
||
|
||
- [ ] **Step 1: Failing tests** for the collector with an in-memory fake `ISecretStore` (implement only `GetAsync`; other members throw `NotSupportedException`):
|
||
|
||
```csharp
|
||
[Fact]
|
||
public async Task Collect_ReturnsOnlyReferencedRows_AndNamesMissing()
|
||
{
|
||
var store = new FakeSecretStore(Row("A", 3), Row("B", 1, deleted: true), Row("OTHER", 1));
|
||
var result = await new SiteSecretsCollector(store).CollectAsync(
|
||
[Dc("GW1", """{"ApiKey":"secret:A"}"""), Dc("GW2", """{"ApiKey":"secret:B"}""")], default);
|
||
Assert.Equal(["A"], result.Secrets.Select(s => s.Name));
|
||
Assert.Equal(3, result.Secrets[0].Revision);
|
||
Assert.Equal(["B"], result.Missing.Select(m => m.Name));
|
||
Assert.Equal(["GW2"], result.Missing[0].ReferencedBy);
|
||
}
|
||
```
|
||
|
||
And in `ArtifactDeploymentServiceTests`: `DeployToAllSites_SiteWithMissingSecret_FailsThatSiteOnly_AndSendsNothingToIt` (assert the communication fake received no command for that site and the other site succeeded), `ComputeCurrentHashes_MissingSecret_UsesMissingMarker` (hash differs from the one computed once the secret exists), `DeployLogs_NeverContainSecretMaterial`.
|
||
|
||
- [ ] **Step 2: Run → FAIL.**
|
||
|
||
- [ ] **Step 3: Implement the collector.**
|
||
|
||
```csharp
|
||
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Artifacts;
|
||
using ZB.MOM.WW.ScadaBridge.Commons.Types.Secrets;
|
||
using ZB.MOM.WW.Secrets.Abstractions;
|
||
|
||
namespace ZB.MOM.WW.ScadaBridge.DeploymentManager;
|
||
|
||
public sealed record SiteSecretsCollection(IReadOnlyList<SecretArtifact> Secrets, IReadOnlyList<SecretReference> Missing);
|
||
|
||
/// <summary>Central half of design §3.1/§3.3: which secrets a site needs and their ciphertext rows. Never decrypts.</summary>
|
||
public sealed class SiteSecretsCollector(ISecretStore store)
|
||
{
|
||
public async Task<SiteSecretsCollection> CollectAsync(IEnumerable<DataConnectionArtifact> connections, CancellationToken ct)
|
||
{
|
||
var secrets = new List<SecretArtifact>(); var missing = new List<SecretReference>();
|
||
foreach (var reference in SecretReferenceScanner.Scan(connections))
|
||
{
|
||
var row = await store.GetAsync(new SecretName(reference.Name), ct);
|
||
if (row is null || row.IsDeleted) { missing.Add(reference); continue; }
|
||
secrets.Add(new SecretArtifact(row.Name.Value, row.ContentType.ToString(), row.Ciphertext, row.Nonce, row.Tag,
|
||
row.WrappedDek, row.WrapNonce, row.WrapTag, row.KekId, row.Revision, row.CreatedUtc, row.UpdatedUtc, row.Description));
|
||
}
|
||
return new SiteSecretsCollection(secrets, missing);
|
||
}
|
||
}
|
||
```
|
||
|
||
(`SecretName` is a `readonly record struct` that normalizes its value at construction — always ship `row.Name.Value`, the normalized form. `SecretContentType` is an enum; ship `ToString()`, parsed back in Task 9.)
|
||
|
||
In `BuildDeployArtifactsCommandAsync`, after mapping data connections: `var collected = await _secretsCollector.CollectAsync(dataConnectionArtifacts, ct);` and pass `Secrets: collected.Secrets.Count > 0 ? collected.Secrets : null, MissingSecrets: collected.Missing.Count > 0 ? collected.Missing.Select(m => m.Name).ToList() : null`. Return the missing list alongside the command (change the private method to return `(DeployArtifactsCommand Command, IReadOnlyList<SecretReference> Missing)`). In `DeployCoreAsync` and `RetryForSiteAsync`: if `Missing.Count > 0`, log Error and produce a failed `SiteArtifactResult` with the message from the acceptance criteria WITHOUT calling `_communicationService`; else log the Information line (only when secrets were present) and send. `ComputeCurrentArtifactHashesAsync` keeps using the command (missing markers included).
|
||
|
||
- [ ] **Step 4: Run → PASS.** **Step 5: Commit** — `git commit -m "feat(artifacts): central ships each site's referenced secrets (ciphertext) and fails closed on a missing one" -- src/ZB.MOM.WW.ScadaBridge.DeploymentManager tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests`
|
||
|
||
```json:metadata
|
||
{"files": ["src/ZB.MOM.WW.ScadaBridge.DeploymentManager/SiteSecretsCollector.cs", "src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactDeploymentService.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests --filter \"FullyQualifiedName~SiteSecretsCollectorTests|FullyQualifiedName~ArtifactDeploymentServiceTests\"", "acceptanceCriteria": ["only referenced rows shipped, field-for-field", "missing/tombstoned ref fails only that site with the named message, nothing sent", "current-hash uses missing markers", "logs name@revision, never material"], "modelTier": "standard"}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 7: Central — stamp gating for old sites + audit row
|
||
|
||
**Goal:** Central never stamps a site *Current* unless the site confirmed applying every shipped secret, and every secret deployment writes one audit row.
|
||
|
||
**Files:**
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactDeploymentService.cs`
|
||
- Modify: `SiteArtifactResult` record (same file or its Commons location) — append `int? SecretsDeployed = null`
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.CLI/Commands/DeployCommands.cs` only if its table output enumerates columns (JSON output carries the new field automatically)
|
||
- Test: `tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests/ArtifactDeploymentServiceTests.cs`
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] Command with ≥ 1 secret + response `Success=true, SecretsApplied=null` → site result is a failure `Site build predates secret deployment — upgrade the site before deploying secrets`; hash NOT stamped.
|
||
- [ ] `SecretsApplied` ≠ shipped count → failure `Site applied {n} of {m} secrets`; not stamped.
|
||
- [ ] Command with no secrets → old-site response (`SecretsApplied=null`) still succeeds and stamps (unchanged behaviour).
|
||
- [ ] One `IAuditService.LogAsync(user, "DeploySiteSecrets", "SiteSecrets", siteIdentifier, siteName, { deploymentId, secrets:[{name,revision}], missing:[name], success, error })` per site whose command carried or would have carried secrets (incl. the fail-closed case); none for sites with no refs. Audit has no material.
|
||
|
||
**Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests --filter FullyQualifiedName~ArtifactDeploymentServiceTests` → all pass.
|
||
|
||
**Steps:**
|
||
|
||
- [ ] **Step 1: Failing tests** — `OldSite_NullSecretsApplied_WithSecrets_FailsAndDoesNotStamp`, `PartialApply_FailsAndDoesNotStamp`, `OldSite_NoSecrets_StillStamps`, `SecretDeploy_WritesOneAuditRowPerSite_WithoutMaterial` (capture `IAuditService` calls in the existing test fake; serialize `afterState` and assert it contains the names/revisions and not the base64 ciphertext).
|
||
|
||
- [ ] **Step 2: Run → FAIL.**
|
||
|
||
- [ ] **Step 3: Implement** a private `static SiteArtifactResult ReconcileSecretsOutcome(DeployArtifactsCommand cmd, ArtifactDeploymentResponse r, Site site)` applied right after each `DeployArtifactsAsync` response in both paths:
|
||
|
||
```csharp
|
||
var shipped = cmd.Secrets?.Count ?? 0;
|
||
if (r.Success && shipped > 0 && r.SecretsApplied is null)
|
||
return new(site.SiteIdentifier, site.Name, false, "Site build predates secret deployment — upgrade the site before deploying secrets", null);
|
||
if (r.Success && shipped > 0 && r.SecretsApplied != shipped)
|
||
return new(site.SiteIdentifier, site.Name, false, $"Site applied {r.SecretsApplied} of {shipped} secrets", r.SecretsApplied);
|
||
return new(site.SiteIdentifier, site.Name, r.Success, r.ErrorMessage, shipped > 0 ? r.SecretsApplied : null);
|
||
```
|
||
|
||
Stamping already keys off `Success`, so a reconciled failure is not stamped. Add the audit call after the per-site results are known (after the existing `DeployArtifacts` audit row, same ordering rule: audit first, best-effort stamp after).
|
||
|
||
- [ ] **Step 4: Run → PASS.** **Step 5: Commit** — `git commit -m "feat(artifacts): never stamp a site current unless it applied every secret; DeploySiteSecrets audit row" -- src/ZB.MOM.WW.ScadaBridge.DeploymentManager src/ZB.MOM.WW.ScadaBridge.CLI tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests`
|
||
|
||
```json:metadata
|
||
{"files": ["src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactDeploymentService.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests --filter FullyQualifiedName~ArtifactDeploymentServiceTests", "acceptanceCriteria": ["old site + secrets -> failure, not stamped", "partial apply -> failure, not stamped", "no secrets -> unchanged behaviour", "one DeploySiteSecrets audit row per secret-bearing site, no material"], "modelTier": "standard"}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 8: Site storage — `deployed_secrets` (replicated) + `mirrored_secrets` (local)
|
||
|
||
**Goal:** The site database has a replicated ciphertext table for deployed secrets and a node-local ledger of mirrored names, with full-set accessors.
|
||
|
||
**Files:**
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Persistence/SiteStorageSchema.cs`
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Persistence/SiteStorageService.cs`
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Host/SiteLocalDbSetup.cs` (`"deployed_secrets"` appended to `ReplicatedTables`; update the "ten tables" comments to eleven; `mirrored_secrets` deliberately NOT registered, with a comment why)
|
||
- Test: `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Persistence/DeployedSecretsStorageTests.cs` (new); existing `SiteLocalDbReplicatedTableFkGuardTests` must still pass
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] `UpsertDeployedSecretsAsync(IReadOnlyList<SecretArtifact>, deploymentId)` upserts rows (`is_removed = 0`) and marks every other row `is_removed = 1`; returns `(applied names, newly removed names)`.
|
||
- [ ] `GetDeployedSecretsAsync()` returns all rows incl. removed with their `SecretArtifact` round-tripped from `row_json`.
|
||
- [ ] `mirrored_secrets` get/add/remove work; it is not in `ReplicatedTables`.
|
||
- [ ] A security test asserts `row_json` holds only base64 ciphertext fields (deserializes to `SecretArtifact`, no extra properties).
|
||
- [ ] FK guard test green (no FKs on either table).
|
||
|
||
**Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests --filter FullyQualifiedName~DeployedSecretsStorageTests` and `dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests --filter FullyQualifiedName~SiteLocalDbReplicatedTableFkGuardTests` → all pass.
|
||
|
||
**Steps:**
|
||
|
||
- [ ] **Step 1: Schema** (append to the `CREATE TABLE IF NOT EXISTS` script):
|
||
|
||
```sql
|
||
CREATE TABLE IF NOT EXISTS deployed_secrets (
|
||
name TEXT PRIMARY KEY,
|
||
row_json TEXT NOT NULL,
|
||
revision INTEGER NOT NULL,
|
||
updated_utc TEXT NOT NULL,
|
||
is_removed INTEGER NOT NULL DEFAULT 0,
|
||
deployment_id TEXT NOT NULL,
|
||
applied_at TEXT NOT NULL
|
||
);
|
||
|
||
CREATE TABLE IF NOT EXISTS mirrored_secrets (
|
||
name TEXT PRIMARY KEY,
|
||
revision INTEGER NOT NULL,
|
||
mirrored_at TEXT NOT NULL
|
||
);
|
||
```
|
||
|
||
- [ ] **Step 2: Failing tests** against a temp-file `SiteStorageService` (follow the existing SiteStorageService test fixture in `SiteRuntime.Tests/Persistence`):
|
||
|
||
```csharp
|
||
[Fact]
|
||
public async Task Upsert_IsFullSet_MarksAbsentRowsRemoved()
|
||
{
|
||
var s = await Fixture.CreateAsync();
|
||
await s.UpsertDeployedSecretsAsync([Art("A", 1), Art("B", 1)], "d1");
|
||
var (applied, removed) = await s.UpsertDeployedSecretsAsync([Art("A", 2)], "d2");
|
||
Assert.Equal(["A"], applied); Assert.Equal(["B"], removed);
|
||
var rows = await s.GetDeployedSecretsAsync();
|
||
Assert.Equal(2, rows.Single(r => r.Name == "A").Secret.Revision);
|
||
Assert.True(rows.Single(r => r.Name == "B").IsRemoved);
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 3: Implement** `DeployedSecretRow(string Name, SecretArtifact Secret, bool IsRemoved, string DeploymentId)` and the accessors (single transaction; `row_json` via `JsonSerializer.Serialize(secretArtifact)` — byte[] serializes as base64). Add `GetMirroredSecretsAsync`, `SetMirroredSecretAsync(name, revision)`, `RemoveMirroredSecretAsync(name)`. Register `deployed_secrets` in `ReplicatedTables`.
|
||
|
||
- [ ] **Step 4: Run → PASS.** **Step 5: Commit** — `git commit -m "feat(site): deployed_secrets (LocalDb-replicated ciphertext) + node-local mirrored_secrets ledger" -- src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Persistence src/ZB.MOM.WW.ScadaBridge.Host/SiteLocalDbSetup.cs tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Persistence`
|
||
|
||
```json:metadata
|
||
{"files": ["src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Persistence/SiteStorageSchema.cs", "src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Persistence/SiteStorageService.cs", "src/ZB.MOM.WW.ScadaBridge.Host/SiteLocalDbSetup.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests --filter FullyQualifiedName~DeployedSecretsStorageTests", "acceptanceCriteria": ["full-set upsert marks absent rows removed", "rows round-trip SecretArtifact", "mirrored_secrets local only", "row_json ciphertext-only security test", "FK guard green"], "modelTier": "standard"}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 9: `DeployedSecretsMirror` — per-node apply + verify
|
||
|
||
**Goal:** Every site node reconciles `deployed_secrets` into its own `secrets.db` (apply live rows, delete removed rows it previously mirrored), verifies each deployed secret decrypts, logs changes, and can be invoked synchronously.
|
||
|
||
**Files:**
|
||
- Create: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Secrets/DeployedSecretsMirror.cs` (service + `IHostedService` loop)
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/SiteRuntimeOptions.cs` (+ `DeployedSecretsMirrorInterval`, default 15 s) and `SiteRuntimeOptionsValidator.cs` (> 0)
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/ZB.MOM.WW.ScadaBridge.SiteRuntime.csproj` (`ZB.MOM.WW.Secrets.Abstractions`), SiteRuntime DI (`AddSingleton<DeployedSecretsMirror>()` + `AddHostedService(sp => sp.GetRequiredService<DeployedSecretsMirror>())`)
|
||
- Test: `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Secrets/DeployedSecretsMirrorTests.cs`
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] `ReconcileAsync()` applies each live row with `ISecretStore.ApplyReplicatedAsync` (mapped back to `StoredSecret`), records it in `mirrored_secrets`, and returns a `MirrorResult(Applied, Removed, Failed)`.
|
||
- [ ] A removed row is deleted from the store only if its name is in `mirrored_secrets`; a same-named out-of-band secret never mirrored is left alone.
|
||
- [ ] Each live secret is trial-resolved via `ISecretResolver`; a failure (null or exception) is a `Failed` entry `Secret '{Name}' (kek {KekId}) did not decrypt on this node — the node's master key differs from central's` and is logged at Error.
|
||
- [ ] Logging: Information per applied/removed secret and one summary when anything changed; Debug when in sync; never values.
|
||
- [ ] The hosted loop runs once at start and every `DeployedSecretsMirrorInterval`; a reconcile exception is logged and the loop continues.
|
||
|
||
**Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests --filter FullyQualifiedName~DeployedSecretsMirrorTests` → all pass.
|
||
|
||
**Steps:**
|
||
|
||
- [ ] **Step 1: Failing tests** using a real temp `SiteStorageService`, an in-memory fake `ISecretStore` recording `ApplyReplicatedAsync`/`DeleteAsync`, and a fake `ISecretResolver`:
|
||
|
||
```csharp
|
||
[Fact]
|
||
public async Task Reconcile_Applies_Verifies_AndOnlyDeletesWhatItMirrored()
|
||
{
|
||
var (mirror, storage, store, resolver) = Harness.Create();
|
||
store.Seed("OUT-OF-BAND");
|
||
await storage.UpsertDeployedSecretsAsync([Art("A", 1), Art("OUT-OF-BAND", 1)], "d1");
|
||
var r1 = await mirror.ReconcileAsync(default);
|
||
Assert.Equal(["A", "OUT-OF-BAND"], r1.Applied.OrderBy(x => x));
|
||
await storage.UpsertDeployedSecretsAsync([], "d2");
|
||
await storage.RemoveMirroredSecretAsync("OUT-OF-BAND"); // simulate: never mirrored by us
|
||
var r2 = await mirror.ReconcileAsync(default);
|
||
Assert.Equal(["A"], r2.Removed);
|
||
Assert.True(store.Contains("OUT-OF-BAND"));
|
||
}
|
||
|
||
[Fact]
|
||
public async Task Reconcile_DecryptFailure_IsReportedWithKekId()
|
||
{
|
||
var (mirror, storage, _, resolver) = Harness.Create();
|
||
resolver.FailFor("A");
|
||
await storage.UpsertDeployedSecretsAsync([Art("A", 1, kek: "kek-central")], "d1");
|
||
var r = await mirror.ReconcileAsync(default);
|
||
Assert.Contains("kek-central", Assert.Single(r.Failed));
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Run → FAIL.**
|
||
|
||
- [ ] **Step 3: Implement.**
|
||
|
||
```csharp
|
||
public sealed record MirrorResult(IReadOnlyList<string> Applied, IReadOnlyList<string> Removed, IReadOnlyList<string> Failed)
|
||
{
|
||
public bool Changed => Applied.Count > 0 || Removed.Count > 0;
|
||
}
|
||
|
||
/// <summary>Design §3.5 step 2: per-node reconcile of the replicated deployed_secrets table into this node's secret store.</summary>
|
||
public sealed class DeployedSecretsMirror(
|
||
SiteStorageService storage, ISecretStore store, ISecretResolver resolver,
|
||
IOptions<SiteRuntimeOptions> options, ILogger<DeployedSecretsMirror> logger, TimeProvider time) : BackgroundService
|
||
{
|
||
private readonly SemaphoreSlim _gate = new(1, 1);
|
||
|
||
public async Task<MirrorResult> ReconcileAsync(CancellationToken ct)
|
||
{
|
||
await _gate.WaitAsync(ct);
|
||
try
|
||
{
|
||
var rows = await storage.GetDeployedSecretsAsync();
|
||
var mirrored = (await storage.GetMirroredSecretsAsync()).ToDictionary(m => m.Name, m => m.Revision, StringComparer.Ordinal);
|
||
List<string> applied = [], removed = [], failed = [];
|
||
foreach (var row in rows)
|
||
{
|
||
if (row.IsRemoved)
|
||
{
|
||
if (!mirrored.ContainsKey(row.Name)) continue;
|
||
await store.DeleteAsync(new SecretName(row.Name), "artifact-deploy", ct);
|
||
await storage.RemoveMirroredSecretAsync(row.Name);
|
||
removed.Add(row.Name);
|
||
logger.LogInformation("Removed deployed secret {SecretName} (no longer referenced by this site)", row.Name);
|
||
continue;
|
||
}
|
||
if (!mirrored.TryGetValue(row.Name, out var rev) || rev != row.Secret.Revision)
|
||
{
|
||
await store.ApplyReplicatedAsync(ToStored(row.Secret), ct);
|
||
await storage.SetMirroredSecretAsync(row.Name, row.Secret.Revision);
|
||
applied.Add(row.Name);
|
||
logger.LogInformation("Applied deployed secret {SecretName} r{Revision} (kek {KekId})", row.Name, row.Secret.Revision, row.Secret.KekId);
|
||
}
|
||
if (!await Resolves(row.Name, ct))
|
||
{
|
||
var msg = $"Secret '{row.Name}' (kek {row.Secret.KekId}) did not decrypt on this node — the node's master key differs from central's";
|
||
failed.Add(msg);
|
||
logger.LogError("{Failure}", msg);
|
||
}
|
||
}
|
||
var result = new MirrorResult(applied, removed, failed);
|
||
if (result.Changed) logger.LogInformation("Deployed secrets mirror: {Applied} applied, {Removed} removed, {Failed} failed", applied.Count, removed.Count, failed.Count);
|
||
else logger.LogDebug("Deployed secrets mirror in sync ({Count} secret(s))", rows.Count(r => !r.IsRemoved));
|
||
return result;
|
||
}
|
||
finally { _gate.Release(); }
|
||
}
|
||
|
||
private async Task<bool> Resolves(string name, CancellationToken ct)
|
||
{
|
||
try { return await resolver.GetAsync(new SecretName(name), ct) is not null; }
|
||
catch (Exception) { return false; }
|
||
}
|
||
|
||
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
|
||
{
|
||
using var timer = new PeriodicTimer(options.Value.DeployedSecretsMirrorInterval, time);
|
||
do
|
||
{
|
||
try { await ReconcileAsync(stoppingToken); }
|
||
catch (Exception ex) when (ex is not OperationCanceledException) { logger.LogWarning(ex, "Deployed secrets mirror pass failed; retrying next interval"); }
|
||
} while (await timer.WaitForNextTickAsync(stoppingToken));
|
||
}
|
||
|
||
private static StoredSecret ToStored(SecretArtifact s) => new()
|
||
{
|
||
Name = new SecretName(s.Name), Description = s.Description, ContentType = Enum.Parse<SecretContentType>(s.ContentType, ignoreCase: false),
|
||
Ciphertext = s.Ciphertext, Nonce = s.Nonce, Tag = s.Tag, WrappedDek = s.WrappedDek, WrapNonce = s.WrapNonce,
|
||
WrapTag = s.WrapTag, KekId = s.KekId, Revision = s.Revision, CreatedUtc = s.CreatedUtc, UpdatedUtc = s.UpdatedUtc
|
||
};
|
||
}
|
||
```
|
||
|
||
`ContentType` is the `SecretContentType` enum (`Text|ConnectionString|Json|BinaryBase64`): Task 6 writes `row.ContentType.ToString()`, this method parses it back with `Enum.Parse<SecretContentType>` — add a round-trip unit test over all four values. Note: the resolver caches (`ResolveCacheTtl` 30 s); if a trial-resolve after a rotation returns the cached old value that is still a successful decrypt of *a* value — acceptable for the verify's purpose (KEK check), document it in the class remarks.
|
||
|
||
- [ ] **Step 4: Run → PASS.** **Step 5: Commit** — `git commit -m "feat(site): DeployedSecretsMirror — per-node apply + decrypt-verify of deployed secrets" -- src/ZB.MOM.WW.ScadaBridge.SiteRuntime tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Secrets`
|
||
|
||
```json:metadata
|
||
{"files": ["src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Secrets/DeployedSecretsMirror.cs", "src/ZB.MOM.WW.ScadaBridge.SiteRuntime/SiteRuntimeOptions.cs", "src/ZB.MOM.WW.ScadaBridge.SiteRuntime/SiteRuntimeOptionsValidator.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests --filter FullyQualifiedName~DeployedSecretsMirrorTests", "acceptanceCriteria": ["live rows applied via ApplyReplicatedAsync and ledgered", "removed rows deleted only if previously mirrored", "decrypt failure reported with KekId at Error", "change/in-sync logging without values", "hosted loop at start + interval, survives exceptions"], "modelTier": "standard"}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 10: Site apply — secrets in `HandleDeployArtifacts`
|
||
|
||
**Goal:** The active site node writes shipped secrets to `deployed_secrets`, runs the mirror synchronously, and answers with `SecretsApplied` or a failure.
|
||
|
||
**Files:**
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/DeploymentManagerActor.cs` (`HandleDeployArtifacts`, constructor gets `DeployedSecretsMirror?`)
|
||
- Modify: the actor's Props/DI construction site to pass the mirror
|
||
- Test: `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Actors/DeploymentManagerActorArtifactSecretsTests.cs` (new; follow the existing artifact-apply actor tests)
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] `Secrets == null` → `deployed_secrets` untouched, response `SecretsApplied = null` (old-central compatible).
|
||
- [ ] `Secrets` present (incl. empty) → full-set write, synchronous `ReconcileAsync`, response `Success=true, SecretsApplied = shipped count` when the mirror reports no failures.
|
||
- [ ] Any mirror failure → `Success=false`, `ErrorMessage` = the failure lines joined with `; `, `SecretsApplied` = count applied without failure.
|
||
- [ ] Site Information log `Applying {Count} deployed secret(s), deploymentId={DeploymentId}` + the mirror's per-secret lines.
|
||
|
||
**Verify:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests --filter FullyQualifiedName~DeploymentManagerActorArtifactSecretsTests` → all pass.
|
||
|
||
**Steps:**
|
||
|
||
- [ ] **Step 1: Failing tests** (TestKit; send `DeployArtifactsCommand` with 2 secrets, expect `ArtifactDeploymentResponse` with `SecretsApplied == 2`; a resolver fake failing one → `Success == false` and the KekId in the message; `Secrets == null` → `SecretsApplied == null` and table empty).
|
||
|
||
- [ ] **Step 2: Run → FAIL.**
|
||
|
||
- [ ] **Step 3: Implement** inside the existing `Task.Run` block, AFTER data connections are stored (so the DCL can resolve by the time it reconnects) and BEFORE the success reply:
|
||
|
||
```csharp
|
||
int? secretsApplied = null;
|
||
string? secretError = null;
|
||
if (command.Secrets != null && _secretsMirror != null)
|
||
{
|
||
_logger.LogInformation("Applying {Count} deployed secret(s), deploymentId={DeploymentId}", command.Secrets.Count, command.DeploymentId);
|
||
await _storage.UpsertDeployedSecretsAsync(command.Secrets, command.DeploymentId);
|
||
var mirror = await _secretsMirror.ReconcileAsync(CancellationToken.None);
|
||
var failedNames = mirror.Failed.Count;
|
||
secretsApplied = command.Secrets.Count - failedNames;
|
||
if (failedNames > 0) secretError = string.Join("; ", mirror.Failed);
|
||
}
|
||
```
|
||
|
||
and build the reply with `Success = secretError is null && <existing success>`, `ErrorMessage = secretError ?? <existing>`, `SecretsApplied = secretsApplied`. If the mirror is not registered (null) but secrets were shipped, reply failure `Secret deployment is not enabled on this site node`.
|
||
|
||
- [ ] **Step 4: Run → PASS**, plus the existing artifact-apply tests. **Step 5: Commit** — `git commit -m "feat(site): artifact apply writes deployed secrets and answers SecretsApplied" -- src/ZB.MOM.WW.ScadaBridge.SiteRuntime tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Actors`
|
||
|
||
```json:metadata
|
||
{"files": ["src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/DeploymentManagerActor.cs"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests --filter FullyQualifiedName~DeploymentManagerActorArtifactSecretsTests", "acceptanceCriteria": ["null Secrets leaves table and SecretsApplied null", "secrets -> full-set write + sync mirror + SecretsApplied=count", "mirror failure -> Success=false with KekId message", "apply logging"], "modelTier": "standard"}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 11: Retire the gRPC hub for ScadaBridge; switch the rig
|
||
|
||
**Goal:** ScadaBridge no longer composes the secrets hub (client or server); config asking for it fails fast; the docker rig's central pair uses `Secrets:Store:Provider=SqlServer` and site-a gets secrets only via artifact deploy.
|
||
|
||
**Files:**
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Host/SecretsRegistration.cs` (delete the Grpc hub/hub-client and SqlServer-replication branches; keep local SQLite + Task 5's shared store), `Program.cs` (remove `MapScadaBridgeSecretsHub`), `StartupValidator.cs` (reject `Secrets:Replication:Enabled=true`: `Secrets replication (the gRPC hub) is retired in ScadaBridge — sites receive secrets with Deploy Artifacts; central uses Secrets:Store:Provider=SqlServer`)
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Host/appsettings.json` (remove `Replication`/`GrpcHub` blocks; keep `SqlServer.ConnectionString`/`SchemaName`; update comments)
|
||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Host/ZB.MOM.WW.ScadaBridge.Host.csproj` (drop `ZB.MOM.WW.Secrets.Replicator.Grpc` reference) and `Directory.Packages.props` if nothing else uses it
|
||
- Modify: `tests/ZB.MOM.WW.ScadaBridge.Host.Tests/SecretsReplicationWiringTests.cs`, `MappedGrpcAuthenticationCoverageTests.cs` (hub route gone)
|
||
- Modify: `docker/docker-compose.yml` (delete `x-secrets-hub-env` + `x-secrets-hub-site-env` anchors and their merges; central-a/b get `Secrets__Store__Provider: "SqlServer"` + the existing `x-secrets-hub-central-env` connection string renamed `x-secrets-central-store-env`; the rig KEK `ZB_SECRETS_MASTER_KEY` goes on central-a/b AND site-a-a/b — every node that handles deployed secrets must share it; site-b/site-c keep a DIFFERENT dev KEK so the Task 13 mismatch negative is real)
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] No ScadaBridge code references `AddZbSecretsGrpcHub`, `AddZbSecretsGrpcHubClient` or `MapScadaBridgeSecretsHub`.
|
||
- [ ] `Secrets:Replication:Enabled=true` on either role → startup failure with the message above.
|
||
- [ ] Wiring tests cover: Central Sqlite, Central SqlServer store, Site local — all green; hub-route auth coverage test updated.
|
||
- [ ] `docker compose config` renders; central-a/b carry `Secrets__Store__Provider=SqlServer`; no node carries `Secrets__Replication__*` or `Secrets__GrpcHub__*`.
|
||
|
||
**Verify:** `dotnet build ZB.MOM.WW.ScadaBridge.slnx` → 0 errors; `dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests` → all pass; `docker compose -f docker/docker-compose.yml config | grep -c Secrets__Replication` → `0`.
|
||
|
||
**Steps:**
|
||
|
||
- [ ] **Step 1:** Update tests first (replace hub-mode wiring tests with `ReplicationEnabled_IsRejected_BothRoles`), run → FAIL.
|
||
- [ ] **Step 2:** Remove the hub branches/mapping/package ref, add the validator rule, run → PASS.
|
||
- [ ] **Step 3:** Edit `docker-compose.yml` as listed; `docker compose -f docker/docker-compose.yml config > /dev/null` → exit 0.
|
||
- [ ] **Step 4: Commit** — `git commit -m "feat(secrets)!: retire the gRPC secrets hub in ScadaBridge — sites get secrets via Deploy Artifacts; rig central on the shared SQL store" -- src/ZB.MOM.WW.ScadaBridge.Host tests/ZB.MOM.WW.ScadaBridge.Host.Tests docker/docker-compose.yml Directory.Packages.props`
|
||
|
||
```json:metadata
|
||
{"files": ["src/ZB.MOM.WW.ScadaBridge.Host/SecretsRegistration.cs", "src/ZB.MOM.WW.ScadaBridge.Host/Program.cs", "src/ZB.MOM.WW.ScadaBridge.Host/StartupValidator.cs", "docker/docker-compose.yml"], "verifyCommand": "dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests", "acceptanceCriteria": ["no hub wiring left in ScadaBridge", "Replication:Enabled=true fails startup with the retirement message", "wiring tests green", "compose renders; centrals SqlServer store; no hub env anywhere"], "modelTier": "standard"}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 12: Documentation sync
|
||
|
||
**Goal:** The spec docs, CLAUDE.md, the deferred-work register and the umbrella index describe secrets-via-artifacts and the hub retirement.
|
||
|
||
**Files:**
|
||
- Modify: `docs/requirements/Component-DeploymentManager.md` (artifact set + secrets, fail-closed, hash, audit), `Component-SiteRuntime.md` (deployed_secrets, mirror, options), `Component-Host.md` (Secrets section: providers, hub retired, bootstrap secrets out-of-band), `Component-CentralUI.md` (Sites page: secrets make a site out of date; failure messages)
|
||
- Modify: `docs/deployment/topology-guide.md` (KEK must match on every central + site node; standby convergence interval)
|
||
- Modify: `CLAUDE.md` (replace the secrets/hub facts with one bullet under *Templates & Deployment*; update the LocalDb bullet "ten tables" → eleven incl. `deployed_secrets`)
|
||
- Modify: `docs/plans/2026-07-08-deferred-work-register.md` (new row: vd03 + QA rollout — zbsecrets store, `Secrets:Store:Provider=SqlServer` on both centrals, KEK alignment check, recreate hand-seeded site secrets centrally, Deploy Artifacts)
|
||
- Modify: `../scadaproj/CLAUDE.md` (ScadaBridge row: short line; Secrets 0.6.3 note in the libraries table) and `../scadaproj/ZB.MOM.WW.Secrets/CLAUDE.md` (ScadaBridge no longer a hub consumer)
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] `grep -rn "GrpcHub\|secrets hub" docs/requirements CLAUDE.md` returns only historical/retired mentions.
|
||
- [ ] Register row lists the four rollout prerequisites and the trigger ("user asks for the vd03/QA secrets window").
|
||
- [ ] Umbrella index row stays a few lines.
|
||
|
||
**Verify:** `grep -rn "deployed_secrets" docs/requirements/Component-SiteRuntime.md CLAUDE.md` → matches in both; `git diff --stat` shows only doc files.
|
||
|
||
**Steps:**
|
||
|
||
- [ ] **Step 1:** Edit the docs listed (quote the design doc sections rather than restating detail).
|
||
- [ ] **Step 2:** `git diff` review. **Step 3: Commit** in each repo — `git commit -m "docs: site secrets ride the artifact deploy; gRPC hub retired in ScadaBridge" -- docs CLAUDE.md` and `git -C ../scadaproj commit -m "index: ScadaBridge secrets via artifact deploy; Secrets 0.6.3" -- CLAUDE.md ZB.MOM.WW.Secrets/CLAUDE.md`
|
||
|
||
```json:metadata
|
||
{"files": ["docs/requirements/Component-DeploymentManager.md", "docs/requirements/Component-SiteRuntime.md", "docs/requirements/Component-Host.md", "CLAUDE.md", "docs/plans/2026-07-08-deferred-work-register.md"], "verifyCommand": "grep -rn deployed_secrets docs/requirements/Component-SiteRuntime.md CLAUDE.md", "acceptanceCriteria": ["component docs + topology guide updated", "CLAUDE.md bullet + LocalDb eleven tables", "register rollout row with 4 prerequisites", "umbrella + Secrets CLAUDE.md updated"], "modelTier": "mechanical"}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 13: Full test run, rig rebuild, live gate
|
||
|
||
**Goal:** The whole solution is green and the rebuilt docker rig demonstrates secrets deploying with artifacts end to end, including standby convergence, staleness, removal, fail-closed and KEK mismatch.
|
||
|
||
**Files:**
|
||
- Create: `docker/verify-site-secrets.sh` (idempotent gate script, prints PASS/FAIL per check, exits non-zero on any FAIL)
|
||
- Create: `docs/plans/2026-09-28-site-secrets-artifact-deploy-live-gate.md` (results record)
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] `dotnet test ZB.MOM.WW.ScadaBridge.slnx` green except the known env-only Playwright residue (list any failure explicitly).
|
||
- [ ] `bash docker/deploy.sh` succeeds; all 8 nodes `/health/ready` 200.
|
||
- [ ] Gate checks (each PASS):
|
||
1. Create secret `sb-gate-mxgw` at central (Secrets CLI or `/admin/secrets`) and an unbound MxGateway data connection on site-a with `"ApiKey":"secret:sb-gate-mxgw"` → Sites page/`site list` shows site-a **Out of date**, site-b/site-c unchanged.
|
||
2. `deploy artifacts --site-id <site-a>` → success, `secretsDeployed: 1`; site-a **Current**.
|
||
3. Both site-a nodes' `secrets.db` contain `sb-gate-mxgw` at the central revision (`docker exec` + `sqlite3` on the library table, ciphertext compare by SHA-256) — the standby within 30 s.
|
||
4. Rotate the secret at central → site-a **Out of date** → deploy → both nodes at the new revision.
|
||
5. Remove the `secret:` reference → deploy → both nodes no longer hold the secret; an out-of-band secret seeded on site-a-a beforehand is still present.
|
||
6. Reference a non-existent `sb-gate-missing` → deploy site-a fails with the named message; site-a stays **Out of date**; nothing changed on the site.
|
||
7. Same connection on site-b (different dev KEK) → deploy fails with the KEK-mismatch message naming the KekId.
|
||
8. Audit log has one `DeploySiteSecrets` row per secret-bearing deploy; `grep` of all 8 nodes' logs for the secret's plaintext and base64 ciphertext → 0 hits.
|
||
- [ ] Gate script and results record committed; gate objects cleaned up by the script's teardown.
|
||
|
||
**Verify:** `bash docker/verify-site-secrets.sh` → `PASS 8/8`.
|
||
|
||
**Steps:**
|
||
|
||
- [ ] **Step 1:** `dotnet build ZB.MOM.WW.ScadaBridge.slnx && dotnet test ZB.MOM.WW.ScadaBridge.slnx` — record counts.
|
||
- [ ] **Step 2:** `bash docker/deploy.sh` (from the primary checkout, not a worktree — see memory: deploy.sh from a worktree rehomes mounts); wait for readiness.
|
||
- [ ] **Step 3:** Write `docker/verify-site-secrets.sh` implementing checks 1–8 with the CLI (`src/ZB.MOM.WW.ScadaBridge.CLI/bin/Debug/net10.0/scadabridge --url http://localhost:9000 --username multi-role --password password`), `docker exec`, and a trap-based teardown; run it until PASS 8/8, fixing product defects in their owning task's files (commit separately with the task tag).
|
||
- [ ] **Step 4:** Write the results record (method, per-check evidence, timings incl. standby convergence), commit — `git commit -m "test(rig): site secrets artifact-deploy live gate PASS 8/8" -- docker/verify-site-secrets.sh docs/plans/2026-09-28-site-secrets-artifact-deploy-live-gate.md`
|
||
|
||
```json:metadata
|
||
{"files": ["docker/verify-site-secrets.sh", "docs/plans/2026-09-28-site-secrets-artifact-deploy-live-gate.md"], "verifyCommand": "bash docker/verify-site-secrets.sh", "acceptanceCriteria": ["solution tests green (env-only residue listed)", "rig rebuilt, 8 nodes ready", "gate checks 1-8 PASS incl. standby convergence, removal, fail-closed, KEK mismatch, audit + no-material grep", "gate script + results record committed"], "modelTier": "standard"}
|
||
```
|
||
|
||
---
|
||
|
||
## Dependencies
|
||
|
||
- Task 1 is independent (library) — run first so the rig transition in Task 11/13 is observable.
|
||
- Task 2 → Tasks 4, 6, 7, 9, 10. Task 3 → Task 6. Task 4 → Task 6. Task 5 → Task 6 (central store DI) and Task 11.
|
||
- Task 6 → Task 7. Task 8 → Task 9 → Task 10.
|
||
- Tasks 7 and 10 → Task 11 → Task 12 → Task 13.
|