From 9df4f9bb842a31f6a2cde47ee3c99ece87c495e0 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Wed, 15 Jul 2026 16:14:10 -0400 Subject: [PATCH] docs(plans): ZB.MOM.WW.Secrets encrypted secrets manager design Envelope-encryption (AES-256-GCM) secrets lib + Theme RCL UI + CLI for the ZB.MOM.WW.* family. Pluggable IMasterKeyProvider (env/file/DPAPI), SQLite behind an ISecretStore seam, ISecretResolver + ${secret:} config expander, HistorianGateway as reference consumer. Cluster-sync seam + schema now, Akka replicator deferred. --- .../2026-07-15-secrets-manager-design.md | 261 ++++++++++++++++++ 1 file changed, 261 insertions(+) create mode 100644 docs/plans/2026-07-15-secrets-manager-design.md diff --git a/docs/plans/2026-07-15-secrets-manager-design.md b/docs/plans/2026-07-15-secrets-manager-design.md new file mode 100644 index 0000000..f3c7efe --- /dev/null +++ b/docs/plans/2026-07-15-secrets-manager-design.md @@ -0,0 +1,261 @@ +# ZB.MOM.WW.Secrets — reusable encrypted secrets manager (design) + +**Date:** 2026-07-15 +**Status:** Design approved (brainstorming), ready for implementation planning +**Author:** Joseph Doherty + Claude + +## 1. Purpose + +A new shared library in the `ZB.MOM.WW.*` family that stores secrets **encrypted at +rest** and returns plaintext **on demand**, plus a Blazor UI to manage them. It normalizes +the ad-hoc secret handling scattered across the family today: ScadaBridge's +Data-Protection-encrypted connection strings, peppers/passwords riding in environment +variables, and LDAP passwords sitting in `appsettings`. + +Canonical use case: *"add a SQL login password, then get the value later, but keep it +encrypted for storage."* + +Lives as **plain files under `scadaproj/`** (like every other `ZB.MOM.WW.*` lib) — it is +**not** a nested git repo. (`CLAUDE.md` mis-states the nested-repo convention; do not +`git init` it.) + +## 2. Decisions (from brainstorming) + +| Axis | Decision | +|---|---| +| Crypto | **AES-256-GCM envelope encryption** — per-secret DEK wrapped by a master KEK | +| Master key | **Pluggable `IMasterKeyProvider` seam** with built-in env / file / DPAPI providers | +| Storage | **SQLite default behind an `ISecretStore` seam** (SQL Server swappable later) | +| UI | **`ZB.MOM.WW.Secrets.Ui` RCL** on the Theme kit, embedded in each app's dashboard | +| Consumption | **Both** — `ISecretResolver` for app runtime **and** audited human reveal; `${secret:name}` config expander | +| Versioning | **Overwrite-in-place** — no value history (tombstones for deletes only, see §8) | +| Scope | Build lib + UI + CLI, wire **HistorianGateway** as the reference consumer | +| Cluster sync | **Seam + schema now, Akka replicator deferred** until a clustered app adopts | + +## 3. Package decomposition (mirrors `ZB.MOM.WW.Auth`'s shape) + +| Package | Role | Depends on | +|---|---|---| +| `ZB.MOM.WW.Secrets.Abstractions` | Contracts only: `ISecretStore`, `ISecretResolver`, `IMasterKeyProvider`, `ISecretCipher`, `ISecretReplicator`, records / enums / exceptions | BCL + `DI.Abstractions` | +| `ZB.MOM.WW.Secrets` | Impl: `AesGcmEnvelopeCipher`, `SqliteSecretStore` + migrator, `DefaultSecretResolver`, master-key providers, `${secret:…}` config expander, `AddZbSecrets(…)` | Abstractions, `Microsoft.Data.Sqlite`, `ZB.MOM.WW.Audit` | +| `ZB.MOM.WW.Secrets.Ui` | Blazor RCL on `ZB.MOM.WW.Theme`: list / add / rotate / delete / gated reveal | Abstractions, `ZB.MOM.WW.Theme`, `Auth.AspNetCore` | +| `ZB.MOM.WW.Secrets.Cli` *(tool, not packed)* | `secret set / get / list / rm / rotate`, like the `apikey` CLI | Secrets | +| `ZB.MOM.WW.Secrets.Akka` *(deferred)* | Cluster replication of encrypted rows — **design only in this cut**, built when a clustered app adopts | Secrets, Akka.Cluster | + +## 4. Crypto — envelope encryption + +- **Master KEK** (32 bytes, AES-256) resolved from `IMasterKeyProvider`; **never** persisted + to the secrets DB. +- Every write generates a fresh **per-write DEK** (32 bytes). Plaintext → AES-256-GCM under + the DEK → `(ciphertext, nonce, tag)`. The DEK is then **wrapped** by the KEK + (AES-256-GCM) → `(wrapped_dek, wrap_nonce, wrap_tag)`. +- **AAD binding**: the secret body is authenticated against `name` (the AAD) so a ciphertext + row cannot be swapped under a different secret name. +- **KEK rotation is cheap**: re-wrap each DEK, never re-encrypt bodies. A `kek_id` column + records which master key wrapped each row; an admin op `RewrapAll(oldKek, newKek)` walks + the table. A row whose `kek_id` is unknown to the current provider fails closed on resolve. +- All primitives from `System.Security.Cryptography.AesGcm` and `RandomNumberGenerator` + (BCL, cross-platform, no native deps). 12-byte random nonces, 16-byte tags. + +## 5. Master-key providers (`IMasterKeyProvider` seam) + +```csharp +public interface IMasterKeyProvider +{ + // 32-byte AES-256 key material; null/throw => fail closed at construction. + ReadOnlyMemory GetMasterKey(); + string KekId { get; } // stable identifier of the currently-active KEK +} +``` + +Built-in implementations, selected by config (`Secrets:MasterKey:Source`): + +- `EnvironmentMasterKeyProvider` — base64 32 bytes from an env var (default + `ZB_SECRETS_MASTER_KEY`). Container/CI friendly; matches how peppers already ride in env. +- `FileMasterKeyProvider` — reads a key file (path from config), mountable as a Docker/K8s + secret. **This is the provider used to satisfy the shared-KEK requirement across a + clustered pair** (same file mounted on both nodes). +- `DpapiMasterKeyProvider` — Windows DPAPI-protected key. **Windows-only**, guarded behind an + OS check; skipped on macOS/Docker Linux. + +Mirrors the family's existing `IApiKeyPepperProvider` / `IGroupRoleMapper` seam +pattern. Fail-closed: a missing/invalid key throws `MasterKeyUnavailableException` at +construction so the app won't boot half-blind (same posture as `ApiKeyVerifier` on a missing +pepper). + +## 6. Storage (`ISecretStore` seam, SQLite default) + +```csharp +public interface ISecretStore +{ + Task GetAsync(SecretName name, CancellationToken ct); + Task UpsertAsync(StoredSecret row, CancellationToken ct); // overwrite-in-place + Task DeleteAsync(SecretName name, CancellationToken ct); // soft delete (tombstone) + Task> ListAsync(CancellationToken ct); // metadata only, no ciphertext + // Replication support (see §8): + Task> GetManifestAsync(CancellationToken ct); + Task ApplyReplicatedAsync(StoredSecret row, CancellationToken ct); // LWW upsert +} +``` + +`SqliteSecretStore` with a schema-versioned, idempotent migrator (exactly like +`SqliteAuthStoreMigrator` / `SqliteAuthSchema`), schema **v1**. + +**Table `secret`** (one row per secret — overwrite-in-place, no value-history table): + +| Column | Notes | +|---|---| +| `name` | TEXT PK, canonical namespaced key e.g. `sql/historiangw/historian-password` | +| `description` | TEXT | +| `content_type` | TEXT — `text` \| `connection-string` \| `json` \| `binary-base64` | +| `ciphertext`, `nonce`, `tag` | BLOB — AES-GCM under the DEK | +| `wrapped_dek`, `wrap_nonce`, `wrap_tag` | BLOB — DEK wrapped by the KEK | +| `kek_id` | TEXT — which master key wrapped this row | +| `revision` | INTEGER — per-row monotonic counter (bumped every write); conflict resolution | +| `is_deleted`, `deleted_utc` | tombstone (soft delete) — propagates deletes across a cluster | +| `created_utc`, `updated_utc`, `created_by`, `updated_by` | audit columns | + +`schema_version` table/pragma drives the idempotent migrator. + +> The `revision` / `updated_utc` / `is_deleted` columns exist **now** even though the Akka +> replicator is deferred — they're the cheap future-proofing so cluster sync needs no schema +> migration later (§8). They are **not** value-history: values are still overwritten in +> place; only a delete-*marker* survives. + +## 7. Consumption + +### 7a. App runtime — `ISecretResolver` + +```csharp +public interface ISecretResolver +{ + Task GetAsync(SecretName name, CancellationToken ct); + Task TryGetAsync(SecretName name, out string value, CancellationToken ct); +} +``` + +`DefaultSecretResolver` decrypts via the cipher + KEK, with a small in-memory cache +invalidated on rotate/delete. **Every resolve writes a `ZB.MOM.WW.Audit` event** (actor + +secret name + outcome — **never the value**). + +### 7b. Config integration — `${secret:name}` expander + +A `SecretReferenceExpander` runs **after** configuration load and **before** options +validation, expanding `${secret:name}` tokens in bound config: + +```json +"ConnectionStrings:Historian": "Server=wonder-sql-vd03,32565;...;Password=${secret:sql/historiangw/historian-password}" +``` + +Keeps plaintext out of `appsettings`. **Fail-closed**: a referenced-but-missing secret stops +startup with a clear error naming the token. + +### 7c. Human reveal + +The UI "reveal" action returns the current plaintext, **gated behind a dedicated +`secrets:reveal` authz policy** (strictly more privileged than `secrets:manage`), and writes +an audit event with the actor. Reveal is **opt-in per deployment**. + +## 8. Clustered pairs — syncing local secrets (design; impl deferred) + +Applies to the Akka-clustered apps (**ScadaBridge**, **OtOpcUa**) — **not** the reference +consumer HistorianGateway (single-process sidecar). With per-node local SQLite, a secret +added/rotated on node A leaves node B stale. + +**Enabler:** because each row is ciphertext + a KEK-wrapped DEK, whole rows can be shipped +over the cluster wire **without exposing plaintext or the KEK** — the transport need not be +trusted with secrets. + +**Hard operational constraint:** every node in a pair MUST resolve the **same master KEK** +(same env var / same mounted key file). A replicated row whose `kek_id` doesn't match the +local provider fails closed on resolve ("row wrapped by unknown KEK") rather than returning +garbage. Documented as a deployment requirement for clustered adopters; `FileMasterKeyProvider` +with a shared mounted key is the intended path. + +**Mechanism** — optional `ZB.MOM.WW.Secrets.Akka` package (core stays Akka-free, exactly +like `ZB.MOM.WW.Health.Akka`), driving an `ISecretReplicator` seam defined in Abstractions +(no-op default in core): + +- **On write** (any node): bump `revision` + `updated_utc`, broadcast a + `SecretReplicationEnvelope` (the **encrypted** row only) over Akka `DistributedPubSub`; + peers `ApplyReplicatedAsync`. +- **Conflict resolution:** last-writer-wins by `(updated_utc, revision, node-id tiebreak)` — + deterministic on every node. No singleton writer bottleneck. +- **Anti-entropy resync** on node join / rejoin / cold start: exchange a compact + `(name → revision)` manifest with a peer, pull rows where the peer is ahead, push where + local is ahead. Heals partitions and freshly-recreated nodes. +- **Deletes = tombstones.** A hard delete would be resurrected from a peer on next resync, so + deletes are soft (`is_deleted` + `deleted_utc`), replicated, and reaped after a TTL. + +**Simpler alternative available via the seam:** a clustered app may instead point +`ISecretStore` at a **shared SQL Server store** — one source of truth, no replication (the +shared DB still holds only ciphertext, since the KEK stays per-node). This is the usual +best-practice answer for clustered state; app-level replication is for when a shared +datastore isn't available. + +**First-cut scope:** ship the `revision` / `updated_utc` / `is_deleted` columns + +`ISecretReplicator` no-op seam + manifest/apply store methods **now**; build the real +`ZB.MOM.WW.Secrets.Akka` replicator when a clustered app adopts (it's dead code for +HistorianGateway). + +## 9. UI (RCL on Theme) + +Routable `/admin/secrets` page the host mounts into its dashboard, styled with Theme tokens / +side-rail. Features: + +- **List** — name, description, content-type, `kek_id`, `created/updated_utc/_by`. + **Metadata only; never the value.** +- **Add** — name + description + content-type + value (masked input). +- **Rotate** — enter a new value → overwrite-in-place (bumps `revision`). +- **Delete** — in-page **Theme modal** confirm (no JS `confirm`/`alert` — browser-dialog + hazard). +- **Reveal** — opt-in, `secrets:reveal`-gated, audited. + +Two permissions wired through `Auth.AspNetCore` claims: `secrets:manage` (CRUD) vs +`secrets:reveal` (see plaintext). + +## 10. Error handling — fail closed everywhere + +- Missing/invalid master key → `MasterKeyUnavailableException` at construction (app won't + boot half-blind). +- GCM tag mismatch on decrypt → `SecretDecryptionException` (tamper / wrong KEK) — never + returns garbage. +- Unknown secret on resolve → `SecretNotFoundException` / `TryGet` false; `${secret:}` + expansion fails startup naming the token. +- Row wrapped by an unknown `kek_id` → fail-closed `SecretDecryptionException`. +- Nothing logs plaintext; failures audit as `Outcome=Failure`. + +## 11. Testing + +- **Crypto/unit (xUnit):** cipher round-trip; tamper detection (flip a byte → throws); AAD + name-binding (swap rows → throws); KEK re-wrap; provider selection (env/file, DPAPI + Windows-guarded); resolver cache invalidation on rotate/delete; `${secret:}` expansion + + fail-closed. +- **Store:** SQLite migrator idempotency; schema v1 CRUD; soft-delete tombstone; manifest / + `ApplyReplicated` LWW semantics. +- **UI (bUnit):** list renders metadata-only (asserts value never appears in markup); reveal + gated by policy; add / rotate / delete flows. +- All offline / cross-platform (macOS / Windows / Docker) — `AesGcm` + Sqlite are portable; + DPAPI provider guarded behind an OS check. + +## 12. Reference consumer — HistorianGateway + +Wire HistorianGateway to source its two SQL passwords (historian `wonder-sql-vd03` + Galaxy +SQL) via `${secret:…}`, seeded through the UI/CLI — proving both the runtime-resolve path and +the UI end-to-end in one place. Retains a plain-text fallback path only for local dev where no +master key is provisioned (fail-closed otherwise). + +## 13. Out of scope (this cut) + +- The `ZB.MOM.WW.Secrets.Akka` cluster replicator implementation (design only; §8). +- SQL Server `ISecretStore` provider (seam supports it; build when a consumer needs it). +- Full `components/secrets/` normalization (spec / current-state ×3 / GAPS) and all-app + adoption — a tracked follow-on, like Health/Telemetry were "Built" before "Adopted". +- The secure-by-default / KEK-rotation runbook automation beyond the `RewrapAll` primitive. + +## 14. Component-normalization follow-on + +After the lib + reference consumer land, add `components/secrets/` (spec, shared-contract, +current-state per app documenting today's ad-hoc handling, GAPS backlog) and adopt across +ScadaBridge / OtOpcUa / mxaccessgw — at which point the `ZB.MOM.WW.Secrets.Akka` replicator +gets built for the clustered adopters.