docs(plans): ZB.MOM.WW.Secrets encrypted secrets manager design
Envelope-encryption (AES-256-GCM) secrets lib + Theme RCL UI + CLI for the
ZB.MOM.WW.* family. Pluggable IMasterKeyProvider (env/file/DPAPI), SQLite behind
an ISecretStore seam, ISecretResolver + ${secret:} config expander, HistorianGateway
as reference consumer. Cluster-sync seam + schema now, Akka replicator deferred.
This commit is contained in:
Joseph Doherty
@@ -0,0 +1,261 @@
|
||||
# ZB.MOM.WW.Secrets — reusable encrypted secrets manager (design)
|
||||
|
||||
**Date:** 2026-07-15
|
||||
**Status:** Design approved (brainstorming), ready for implementation planning
|
||||
**Author:** Joseph Doherty + Claude
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
A new shared library in the `ZB.MOM.WW.*` family that stores secrets **encrypted at
|
||||
rest** and returns plaintext **on demand**, plus a Blazor UI to manage them. It normalizes
|
||||
the ad-hoc secret handling scattered across the family today: ScadaBridge's
|
||||
Data-Protection-encrypted connection strings, peppers/passwords riding in environment
|
||||
variables, and LDAP passwords sitting in `appsettings`.
|
||||
|
||||
Canonical use case: *"add a SQL login password, then get the value later, but keep it
|
||||
encrypted for storage."*
|
||||
|
||||
Lives as **plain files under `scadaproj/`** (like every other `ZB.MOM.WW.*` lib) — it is
|
||||
**not** a nested git repo. (`CLAUDE.md` mis-states the nested-repo convention; do not
|
||||
`git init` it.)
|
||||
|
||||
## 2. Decisions (from brainstorming)
|
||||
|
||||
| Axis | Decision |
|
||||
|---|---|
|
||||
| Crypto | **AES-256-GCM envelope encryption** — per-secret DEK wrapped by a master KEK |
|
||||
| Master key | **Pluggable `IMasterKeyProvider` seam** with built-in env / file / DPAPI providers |
|
||||
| Storage | **SQLite default behind an `ISecretStore` seam** (SQL Server swappable later) |
|
||||
| UI | **`ZB.MOM.WW.Secrets.Ui` RCL** on the Theme kit, embedded in each app's dashboard |
|
||||
| Consumption | **Both** — `ISecretResolver` for app runtime **and** audited human reveal; `${secret:name}` config expander |
|
||||
| Versioning | **Overwrite-in-place** — no value history (tombstones for deletes only, see §8) |
|
||||
| Scope | Build lib + UI + CLI, wire **HistorianGateway** as the reference consumer |
|
||||
| Cluster sync | **Seam + schema now, Akka replicator deferred** until a clustered app adopts |
|
||||
|
||||
## 3. Package decomposition (mirrors `ZB.MOM.WW.Auth`'s shape)
|
||||
|
||||
| Package | Role | Depends on |
|
||||
|---|---|---|
|
||||
| `ZB.MOM.WW.Secrets.Abstractions` | Contracts only: `ISecretStore`, `ISecretResolver`, `IMasterKeyProvider`, `ISecretCipher`, `ISecretReplicator`, records / enums / exceptions | BCL + `DI.Abstractions` |
|
||||
| `ZB.MOM.WW.Secrets` | Impl: `AesGcmEnvelopeCipher`, `SqliteSecretStore` + migrator, `DefaultSecretResolver`, master-key providers, `${secret:…}` config expander, `AddZbSecrets(…)` | Abstractions, `Microsoft.Data.Sqlite`, `ZB.MOM.WW.Audit` |
|
||||
| `ZB.MOM.WW.Secrets.Ui` | Blazor RCL on `ZB.MOM.WW.Theme`: list / add / rotate / delete / gated reveal | Abstractions, `ZB.MOM.WW.Theme`, `Auth.AspNetCore` |
|
||||
| `ZB.MOM.WW.Secrets.Cli` *(tool, not packed)* | `secret set / get / list / rm / rotate`, like the `apikey` CLI | Secrets |
|
||||
| `ZB.MOM.WW.Secrets.Akka` *(deferred)* | Cluster replication of encrypted rows — **design only in this cut**, built when a clustered app adopts | Secrets, Akka.Cluster |
|
||||
|
||||
## 4. Crypto — envelope encryption
|
||||
|
||||
- **Master KEK** (32 bytes, AES-256) resolved from `IMasterKeyProvider`; **never** persisted
|
||||
to the secrets DB.
|
||||
- Every write generates a fresh **per-write DEK** (32 bytes). Plaintext → AES-256-GCM under
|
||||
the DEK → `(ciphertext, nonce, tag)`. The DEK is then **wrapped** by the KEK
|
||||
(AES-256-GCM) → `(wrapped_dek, wrap_nonce, wrap_tag)`.
|
||||
- **AAD binding**: the secret body is authenticated against `name` (the AAD) so a ciphertext
|
||||
row cannot be swapped under a different secret name.
|
||||
- **KEK rotation is cheap**: re-wrap each DEK, never re-encrypt bodies. A `kek_id` column
|
||||
records which master key wrapped each row; an admin op `RewrapAll(oldKek, newKek)` walks
|
||||
the table. A row whose `kek_id` is unknown to the current provider fails closed on resolve.
|
||||
- All primitives from `System.Security.Cryptography.AesGcm` and `RandomNumberGenerator`
|
||||
(BCL, cross-platform, no native deps). 12-byte random nonces, 16-byte tags.
|
||||
|
||||
## 5. Master-key providers (`IMasterKeyProvider` seam)
|
||||
|
||||
```csharp
|
||||
public interface IMasterKeyProvider
|
||||
{
|
||||
// 32-byte AES-256 key material; null/throw => fail closed at construction.
|
||||
ReadOnlyMemory<byte> GetMasterKey();
|
||||
string KekId { get; } // stable identifier of the currently-active KEK
|
||||
}
|
||||
```
|
||||
|
||||
Built-in implementations, selected by config (`Secrets:MasterKey:Source`):
|
||||
|
||||
- `EnvironmentMasterKeyProvider` — base64 32 bytes from an env var (default
|
||||
`ZB_SECRETS_MASTER_KEY`). Container/CI friendly; matches how peppers already ride in env.
|
||||
- `FileMasterKeyProvider` — reads a key file (path from config), mountable as a Docker/K8s
|
||||
secret. **This is the provider used to satisfy the shared-KEK requirement across a
|
||||
clustered pair** (same file mounted on both nodes).
|
||||
- `DpapiMasterKeyProvider` — Windows DPAPI-protected key. **Windows-only**, guarded behind an
|
||||
OS check; skipped on macOS/Docker Linux.
|
||||
|
||||
Mirrors the family's existing `IApiKeyPepperProvider` / `IGroupRoleMapper<TRole>` seam
|
||||
pattern. Fail-closed: a missing/invalid key throws `MasterKeyUnavailableException` at
|
||||
construction so the app won't boot half-blind (same posture as `ApiKeyVerifier` on a missing
|
||||
pepper).
|
||||
|
||||
## 6. Storage (`ISecretStore` seam, SQLite default)
|
||||
|
||||
```csharp
|
||||
public interface ISecretStore
|
||||
{
|
||||
Task<StoredSecret?> GetAsync(SecretName name, CancellationToken ct);
|
||||
Task UpsertAsync(StoredSecret row, CancellationToken ct); // overwrite-in-place
|
||||
Task<bool> DeleteAsync(SecretName name, CancellationToken ct); // soft delete (tombstone)
|
||||
Task<IReadOnlyList<SecretMetadata>> ListAsync(CancellationToken ct); // metadata only, no ciphertext
|
||||
// Replication support (see §8):
|
||||
Task<IReadOnlyList<SecretManifestEntry>> GetManifestAsync(CancellationToken ct);
|
||||
Task ApplyReplicatedAsync(StoredSecret row, CancellationToken ct); // LWW upsert
|
||||
}
|
||||
```
|
||||
|
||||
`SqliteSecretStore` with a schema-versioned, idempotent migrator (exactly like
|
||||
`SqliteAuthStoreMigrator` / `SqliteAuthSchema`), schema **v1**.
|
||||
|
||||
**Table `secret`** (one row per secret — overwrite-in-place, no value-history table):
|
||||
|
||||
| Column | Notes |
|
||||
|---|---|
|
||||
| `name` | TEXT PK, canonical namespaced key e.g. `sql/historiangw/historian-password` |
|
||||
| `description` | TEXT |
|
||||
| `content_type` | TEXT — `text` \| `connection-string` \| `json` \| `binary-base64` |
|
||||
| `ciphertext`, `nonce`, `tag` | BLOB — AES-GCM under the DEK |
|
||||
| `wrapped_dek`, `wrap_nonce`, `wrap_tag` | BLOB — DEK wrapped by the KEK |
|
||||
| `kek_id` | TEXT — which master key wrapped this row |
|
||||
| `revision` | INTEGER — per-row monotonic counter (bumped every write); conflict resolution |
|
||||
| `is_deleted`, `deleted_utc` | tombstone (soft delete) — propagates deletes across a cluster |
|
||||
| `created_utc`, `updated_utc`, `created_by`, `updated_by` | audit columns |
|
||||
|
||||
`schema_version` table/pragma drives the idempotent migrator.
|
||||
|
||||
> The `revision` / `updated_utc` / `is_deleted` columns exist **now** even though the Akka
|
||||
> replicator is deferred — they're the cheap future-proofing so cluster sync needs no schema
|
||||
> migration later (§8). They are **not** value-history: values are still overwritten in
|
||||
> place; only a delete-*marker* survives.
|
||||
|
||||
## 7. Consumption
|
||||
|
||||
### 7a. App runtime — `ISecretResolver`
|
||||
|
||||
```csharp
|
||||
public interface ISecretResolver
|
||||
{
|
||||
Task<string?> GetAsync(SecretName name, CancellationToken ct);
|
||||
Task<bool> TryGetAsync(SecretName name, out string value, CancellationToken ct);
|
||||
}
|
||||
```
|
||||
|
||||
`DefaultSecretResolver` decrypts via the cipher + KEK, with a small in-memory cache
|
||||
invalidated on rotate/delete. **Every resolve writes a `ZB.MOM.WW.Audit` event** (actor +
|
||||
secret name + outcome — **never the value**).
|
||||
|
||||
### 7b. Config integration — `${secret:name}` expander
|
||||
|
||||
A `SecretReferenceExpander` runs **after** configuration load and **before** options
|
||||
validation, expanding `${secret:name}` tokens in bound config:
|
||||
|
||||
```json
|
||||
"ConnectionStrings:Historian": "Server=wonder-sql-vd03,32565;...;Password=${secret:sql/historiangw/historian-password}"
|
||||
```
|
||||
|
||||
Keeps plaintext out of `appsettings`. **Fail-closed**: a referenced-but-missing secret stops
|
||||
startup with a clear error naming the token.
|
||||
|
||||
### 7c. Human reveal
|
||||
|
||||
The UI "reveal" action returns the current plaintext, **gated behind a dedicated
|
||||
`secrets:reveal` authz policy** (strictly more privileged than `secrets:manage`), and writes
|
||||
an audit event with the actor. Reveal is **opt-in per deployment**.
|
||||
|
||||
## 8. Clustered pairs — syncing local secrets (design; impl deferred)
|
||||
|
||||
Applies to the Akka-clustered apps (**ScadaBridge**, **OtOpcUa**) — **not** the reference
|
||||
consumer HistorianGateway (single-process sidecar). With per-node local SQLite, a secret
|
||||
added/rotated on node A leaves node B stale.
|
||||
|
||||
**Enabler:** because each row is ciphertext + a KEK-wrapped DEK, whole rows can be shipped
|
||||
over the cluster wire **without exposing plaintext or the KEK** — the transport need not be
|
||||
trusted with secrets.
|
||||
|
||||
**Hard operational constraint:** every node in a pair MUST resolve the **same master KEK**
|
||||
(same env var / same mounted key file). A replicated row whose `kek_id` doesn't match the
|
||||
local provider fails closed on resolve ("row wrapped by unknown KEK") rather than returning
|
||||
garbage. Documented as a deployment requirement for clustered adopters; `FileMasterKeyProvider`
|
||||
with a shared mounted key is the intended path.
|
||||
|
||||
**Mechanism** — optional `ZB.MOM.WW.Secrets.Akka` package (core stays Akka-free, exactly
|
||||
like `ZB.MOM.WW.Health.Akka`), driving an `ISecretReplicator` seam defined in Abstractions
|
||||
(no-op default in core):
|
||||
|
||||
- **On write** (any node): bump `revision` + `updated_utc`, broadcast a
|
||||
`SecretReplicationEnvelope` (the **encrypted** row only) over Akka `DistributedPubSub`;
|
||||
peers `ApplyReplicatedAsync`.
|
||||
- **Conflict resolution:** last-writer-wins by `(updated_utc, revision, node-id tiebreak)` —
|
||||
deterministic on every node. No singleton writer bottleneck.
|
||||
- **Anti-entropy resync** on node join / rejoin / cold start: exchange a compact
|
||||
`(name → revision)` manifest with a peer, pull rows where the peer is ahead, push where
|
||||
local is ahead. Heals partitions and freshly-recreated nodes.
|
||||
- **Deletes = tombstones.** A hard delete would be resurrected from a peer on next resync, so
|
||||
deletes are soft (`is_deleted` + `deleted_utc`), replicated, and reaped after a TTL.
|
||||
|
||||
**Simpler alternative available via the seam:** a clustered app may instead point
|
||||
`ISecretStore` at a **shared SQL Server store** — one source of truth, no replication (the
|
||||
shared DB still holds only ciphertext, since the KEK stays per-node). This is the usual
|
||||
best-practice answer for clustered state; app-level replication is for when a shared
|
||||
datastore isn't available.
|
||||
|
||||
**First-cut scope:** ship the `revision` / `updated_utc` / `is_deleted` columns +
|
||||
`ISecretReplicator` no-op seam + manifest/apply store methods **now**; build the real
|
||||
`ZB.MOM.WW.Secrets.Akka` replicator when a clustered app adopts (it's dead code for
|
||||
HistorianGateway).
|
||||
|
||||
## 9. UI (RCL on Theme)
|
||||
|
||||
Routable `/admin/secrets` page the host mounts into its dashboard, styled with Theme tokens /
|
||||
side-rail. Features:
|
||||
|
||||
- **List** — name, description, content-type, `kek_id`, `created/updated_utc/_by`.
|
||||
**Metadata only; never the value.**
|
||||
- **Add** — name + description + content-type + value (masked input).
|
||||
- **Rotate** — enter a new value → overwrite-in-place (bumps `revision`).
|
||||
- **Delete** — in-page **Theme modal** confirm (no JS `confirm`/`alert` — browser-dialog
|
||||
hazard).
|
||||
- **Reveal** — opt-in, `secrets:reveal`-gated, audited.
|
||||
|
||||
Two permissions wired through `Auth.AspNetCore` claims: `secrets:manage` (CRUD) vs
|
||||
`secrets:reveal` (see plaintext).
|
||||
|
||||
## 10. Error handling — fail closed everywhere
|
||||
|
||||
- Missing/invalid master key → `MasterKeyUnavailableException` at construction (app won't
|
||||
boot half-blind).
|
||||
- GCM tag mismatch on decrypt → `SecretDecryptionException` (tamper / wrong KEK) — never
|
||||
returns garbage.
|
||||
- Unknown secret on resolve → `SecretNotFoundException` / `TryGet` false; `${secret:}`
|
||||
expansion fails startup naming the token.
|
||||
- Row wrapped by an unknown `kek_id` → fail-closed `SecretDecryptionException`.
|
||||
- Nothing logs plaintext; failures audit as `Outcome=Failure`.
|
||||
|
||||
## 11. Testing
|
||||
|
||||
- **Crypto/unit (xUnit):** cipher round-trip; tamper detection (flip a byte → throws); AAD
|
||||
name-binding (swap rows → throws); KEK re-wrap; provider selection (env/file, DPAPI
|
||||
Windows-guarded); resolver cache invalidation on rotate/delete; `${secret:}` expansion +
|
||||
fail-closed.
|
||||
- **Store:** SQLite migrator idempotency; schema v1 CRUD; soft-delete tombstone; manifest /
|
||||
`ApplyReplicated` LWW semantics.
|
||||
- **UI (bUnit):** list renders metadata-only (asserts value never appears in markup); reveal
|
||||
gated by policy; add / rotate / delete flows.
|
||||
- All offline / cross-platform (macOS / Windows / Docker) — `AesGcm` + Sqlite are portable;
|
||||
DPAPI provider guarded behind an OS check.
|
||||
|
||||
## 12. Reference consumer — HistorianGateway
|
||||
|
||||
Wire HistorianGateway to source its two SQL passwords (historian `wonder-sql-vd03` + Galaxy
|
||||
SQL) via `${secret:…}`, seeded through the UI/CLI — proving both the runtime-resolve path and
|
||||
the UI end-to-end in one place. Retains a plain-text fallback path only for local dev where no
|
||||
master key is provisioned (fail-closed otherwise).
|
||||
|
||||
## 13. Out of scope (this cut)
|
||||
|
||||
- The `ZB.MOM.WW.Secrets.Akka` cluster replicator implementation (design only; §8).
|
||||
- SQL Server `ISecretStore` provider (seam supports it; build when a consumer needs it).
|
||||
- Full `components/secrets/` normalization (spec / current-state ×3 / GAPS) and all-app
|
||||
adoption — a tracked follow-on, like Health/Telemetry were "Built" before "Adopted".
|
||||
- The secure-by-default / KEK-rotation runbook automation beyond the `RewrapAll` primitive.
|
||||
|
||||
## 14. Component-normalization follow-on
|
||||
|
||||
After the lib + reference consumer land, add `components/secrets/` (spec, shared-contract,
|
||||
current-state per app documenting today's ad-hoc handling, GAPS backlog) and adopt across
|
||||
ScadaBridge / OtOpcUa / mxaccessgw — at which point the `ZB.MOM.WW.Secrets.Akka` replicator
|
||||
gets built for the clustered adopters.
|
||||
Reference in New Issue
Block a user