diff --git a/CLAUDE.md b/CLAUDE.md
index d10aadd..27b6e7f 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -143,7 +143,7 @@ each project's **code-verified current state**, and the **gaps** between. See
| Config + validation (options / startup validation) | Adopted (lib `0.1.0`; all 3 apps, local) | Shared `ZB.MOM.WW.Configuration` lib | [`components/configuration/`](components/configuration/) | [`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) |
| Audit (event model + writer seam) | Adopted (lib `0.1.0`; all 3 apps, merged to **local default** main/master + **pushed to origin** (gitea)) | Shared `ZB.MOM.WW.Audit` lib | [`components/audit/`](components/audit/) | [`ZB.MOM.WW.Audit/`](ZB.MOM.WW.Audit/) |
| Galaxy Repository (object-hierarchy SQL browse + gRPC service) | Built (lib `0.1.0`, **published to the Gitea feed**; consumed by HistorianGateway as a feed `PackageReference`) | Shared `ZB.MOM.WW.GalaxyRepository` lib | _(design in histsdk + design doc 2026-06-23)_ | [`ZB.MOM.WW.GalaxyRepository/`](ZB.MOM.WW.GalaxyRepository/) |
-| Secrets (encrypted store + `${secret:}` resolution) | Built (libs `0.1.2`, **published to the Gitea feed**; **HistorianGateway adopted + live-proven** 2026-07-16; **mxaccessgw adopted G-4/G-5/G-6 + merged to `origin/main` @ `e088dfa`** 2026-07-16, box-verified fail-closed + encrypt-at-rest; **ScadaBridge adopted G-3/G-4/G-5/G-6 + merged to `origin/main` @ `128f1596`** 2026-07-16, **G-3 live-proven vs the real production MxGateway gateway** `wonder-app-vd03:5120` (secret-ref ApiKey → Connected + browsed real Galaxy); **OtOpcUa adopted G-2/G-4/G-5/G-6 + merged to `origin/master` @ `872cf7e3`** (lmxopcua) 2026-07-16 — the last app, so **all four now adopted**; Layer-B driver secrets (Galaxy API key + OpcUaClient `Password`/`UserCertificatePassword`) resolve fail-closed at driver session-open, `AddZbSecrets` registered unconditionally so driver-only nodes work; **G-2 live-proven vs the real production MxGateway** `wonder-app-vd03:5120` — a `secret:`-ref Galaxy ApiKey resolved through OtOpcUa's real `GalaxyDriverBrowser` → dummy key `MxGatewayAuthenticationException`, real key → browsed the real Galaxy root (10 nodes); temp key minted+revoked+deleted) | Shared `ZB.MOM.WW.Secrets` lib (3 packages + CLI) | [`components/secrets/`](components/secrets/) | [`ZB.MOM.WW.Secrets/`](ZB.MOM.WW.Secrets/) |
+| Secrets (encrypted store + `${secret:}` resolution) | Built (libs `0.1.2`, **published to the Gitea feed**; **HistorianGateway adopted + live-proven** 2026-07-16; **mxaccessgw adopted G-4/G-5/G-6 + merged to `origin/main` @ `e088dfa`** 2026-07-16, box-verified fail-closed + encrypt-at-rest; **ScadaBridge adopted G-3/G-4/G-5/G-6 + merged to `origin/main` @ `128f1596`** 2026-07-16, **G-3 live-proven vs the real production MxGateway gateway** `wonder-app-vd03:5120` (secret-ref ApiKey → Connected + browsed real Galaxy); **OtOpcUa adopted G-2/G-4/G-5/G-6 + merged to `origin/master` @ `872cf7e3`** (lmxopcua) 2026-07-16 — the last app, so **all four now adopted**; Layer-B driver secrets (Galaxy API key + OpcUaClient `Password`/`UserCertificatePassword`) resolve fail-closed at driver session-open, `AddZbSecrets` registered unconditionally so driver-only nodes work; **G-2 live-proven vs the real production MxGateway** `wonder-app-vd03:5120` — a `secret:`-ref Galaxy ApiKey resolved through OtOpcUa's real `GalaxyDriverBrowser` → dummy key `MxGatewayAuthenticationException`, real key → browsed the real Galaxy root (10 nodes); temp key minted+revoked+deleted); **G-8 KEK-rotation BUILT (lib bumped `0.1.2`→`0.1.3`, 2026-07-17):** `Rewrap` DEK primitive + CAS-guarded `ApplyRewrapAsync` + `KekRotationService.RewrapAllAsync` + `secret rewrap-all` CLI + operator runbook (`ZB.MOM.WW.Secrets/docs/operations/kek-rotation.md`), full suite green + CLI smoke + adversarial crypto review PASS (TOCTOU closed via compare-and-swap); republish to feed is opt-in. **G-7 clustered replication DESIGNED + PLANNED** — fork resolved to **build the shared SQL-Server `ISecretStore`** (Akka replicator deferred phase-2), executable plan `docs/plans/2026-07-17-secrets-g7-*`. Both not yet committed/pushed (awaiting go-ahead) | Shared `ZB.MOM.WW.Secrets` lib (3 packages + CLI) | [`components/secrets/`](components/secrets/) | [`ZB.MOM.WW.Secrets/`](ZB.MOM.WW.Secrets/) |
The auth component is fully populated: a normalized [`spec`](components/auth/spec/SPEC.md), a
proposed [`shared-contract`](components/auth/shared-contract/ZB.MOM.WW.Auth.md), three
diff --git a/ZB.MOM.WW.Secrets/Directory.Build.props b/ZB.MOM.WW.Secrets/Directory.Build.props
index 974d6cf..0b2b340 100644
--- a/ZB.MOM.WW.Secrets/Directory.Build.props
+++ b/ZB.MOM.WW.Secrets/Directory.Build.props
@@ -5,7 +5,7 @@
enable
enable
latest
- 0.1.2
+ 0.1.3
true
README.md
diff --git a/ZB.MOM.WW.Secrets/README.md b/ZB.MOM.WW.Secrets/README.md
index 4a7ec36..e3f6e8c 100644
--- a/ZB.MOM.WW.Secrets/README.md
+++ b/ZB.MOM.WW.Secrets/README.md
@@ -12,7 +12,7 @@ plaintext back on demand — from application code, from configuration, or from
| `ZB.MOM.WW.Secrets` | Implementation: AES-256-GCM envelope cipher, env/file/DPAPI master-key providers, SQLite store, TTL resolver (audited), `${secret:}` config expander, `AddZbSecrets`. |
| `ZB.MOM.WW.Secrets.Ui` | Blazor RCL on `ZB.MOM.WW.Theme`: list / add / rotate / delete + policy-gated, audited reveal. |
-A `secret` CLI (`set` / `get` / `list` / `rm` / `rotate`) ships in the repo (not packed).
+A `secret` CLI (`set` / `get` / `list` / `rm` / `rotate` / `rewrap-all`) ships in the repo (not packed).
## How it protects secrets
@@ -50,6 +50,24 @@ await expander.ExpandConfigurationAsync((IConfigurationRoot)builder.Configuratio
The master key is 32 bytes, provided base64 in `ZB_SECRETS_MASTER_KEY` (Environment source).
A missing/invalid key fails closed at startup (`MasterKeyUnavailableException`).
+## Rotating the master KEK
+
+Because each row is a body sealed under a per-secret DEK that is *wrapped* by the KEK, rotating
+the master key only re-wraps DEKs — bodies are never re-encrypted and no value history changes.
+The `secret rewrap-all` CLI verb (backed by `KekRotationService`) migrates every row from the
+old KEK to the new one; it is idempotent and safe to re-run:
+
+```bash
+# New KEK defaults to the configured Secrets:MasterKey; supply the OLD key by env-var name or path.
+ZB_SECRETS_MASTER_KEY= ZB_SECRETS_OLD_KEY= \
+ secret rewrap-all --old-key-env ZB_SECRETS_OLD_KEY
+# → {"action":"rewrap-all","total":N,"rewrapped":N,"alreadyCurrent":0}
+```
+
+Run it with resolve traffic quiesced and once per independent store (once for a shared
+SQL-Server store; once per node for per-node SQLite on a shared KEK). See the operator runbook:
+[`docs/operations/kek-rotation.md`](docs/operations/kek-rotation.md).
+
## Runtime + human access
- **App code:** inject `ISecretResolver` and call `GetAsync(name, ct)`. Every resolve is
diff --git a/ZB.MOM.WW.Secrets/docs/operations/kek-rotation.md b/ZB.MOM.WW.Secrets/docs/operations/kek-rotation.md
new file mode 100644
index 0000000..b32f76d
--- /dev/null
+++ b/ZB.MOM.WW.Secrets/docs/operations/kek-rotation.md
@@ -0,0 +1,120 @@
+# Operator runbook — rotating the master KEK (`ZB.MOM.WW.Secrets`)
+
+**Audience:** operators rotating the key-encryption key (KEK) that protects a secrets store.
+**Primitive:** `secret rewrap-all` (CLI) → `KekRotationService.RewrapAllAsync` → per-row
+`ISecretCipher.Rewrap` + `ISecretStore.ApplyRewrapAsync`.
+**Status:** G-8 (KEK-rotation `RewrapAll` + runbook). Companion to the deferred G-7 clustered
+replication (`ZB.MOM.WW.Secrets.Akka`).
+
+---
+
+## What rotation does (and does not) touch
+
+Each secret is stored as a body sealed under a fresh per-secret **data-encryption key (DEK)**;
+the DEK is then **wrapped** by the master **KEK**. Rotation **re-wraps the DEK only**:
+
+- **Changed:** `wrapped_dek`, `wrap_nonce`, `wrap_tag`, `kek_id` — the KEK-wrap envelope.
+- **Untouched:** the sealed body (`ciphertext`/`nonce`/`tag`), `revision`, `updated_utc`,
+ `created_*`/`updated_by`, the tombstone flag. Plaintext is never decrypted to disk or logged.
+
+Because `revision`/`updated_utc` are preserved, a re-wrap is **invisible to cluster
+last-writer-wins** anti-entropy — so it never churns replication, and it must be run **per
+independent store** (see step 5).
+
+## Properties you can rely on
+
+- **Idempotent / resumable.** A row already on the new KEK is skipped (`alreadyCurrent`). If a
+ run is interrupted, just run it again — it resumes from where it stopped.
+- **Fail-closed.** A row wrapped by a KEK that is *neither* the old nor the new one **aborts**
+ the pass (rows already re-wrapped stay persisted). A wrong old-key or tampered wrap aborts with
+ `SecretDecryptionException`. An identical old/new KEK id is rejected up front.
+- **No plaintext exposure.** The report is counts only: `{total, rewrapped, alreadyCurrent}`.
+
+## KEK identity — the one thing to get right
+
+The old KEK you supply **must resolve to the same `kek_id` that is stored on the rows.** The
+provider derives `kek_id` as `sha256:<12 hex>` of the key bytes **unless** an explicit
+`Secrets:MasterKey:KekId` was configured.
+
+- **Derived id (default):** supplying the same old key **bytes** reproduces the same id — nothing
+ extra to pass.
+- **Explicit id:** the app set `Secrets:MasterKey:KekId`. Pass the matching id with
+ `--old-key-id ` (and `--new-key-id ` if the new KEK also uses an explicit id).
+
+A mismatch fails closed (`Cannot rewrap … not the supplied old KEK`) — it never corrupts a row.
+
+> **The new KEK id MUST differ from the old one.** With derived ids this is automatic (new key
+> bytes → new `sha256:` id). With **explicit** ids you must assign a new `--new-key-id`; reusing the
+> old id for new key material is rejected up front (`old and new KEK identifiers are identical;
+> there is nothing to rotate`) because `kek_id` is the only per-row rotation discriminator.
+
+## Key material handling
+
+- Key **values are never passed as command-line arguments** (they would be echoed/logged). Supply
+ the old (and optionally new) key by **env-var NAME** (`--old-key-env VAR`) or **file PATH**
+ (`--old-key-file /path`). The value lives only in that env var / file.
+- Never print `ZB_SECRETS_MASTER_KEY` or the old key to a terminal, log, or ticket.
+- The new KEK **defaults to the app's configured `Secrets:MasterKey` provider**, so you usually
+ only supply the *old* key.
+
+---
+
+## Procedure
+
+Rotation is an offline, store-local operation. Do it once per store.
+
+**1. Back up the store.** Copy the SQLite file (or snapshot the SQL-Server DB). This is the
+undo — if anything is wrong, restore and retry.
+
+**2. Quiesce resolve traffic.** Stop (or drain) the app(s) reading this store. Rotation preserves
+`updated_utc`/`revision`, so a running node on the *old* KEK would keep working until you flip its
+key — but running rotation against a store that other writers are mutating risks racing a fresh
+write onto the old KEK. Simplest correct posture: app down.
+
+**3. Stage the new KEK.** Generate a fresh 32-byte key, base64-encode it, and place it where the
+CLI (and, after cutover, the app) will read it — e.g. `ZB_SECRETS_MASTER_KEY` for the Environment
+provider, or the mounted key file for the File provider.
+
+**4. Run `rewrap-all`.** Point the CLI at the store's `appsettings.json` (so `Secrets:SqlitePath`
++ the new `Secrets:MasterKey` resolve), and supply the old key:
+
+```bash
+# Old key delivered via a throwaway env var (name only on the command line):
+ZB_SECRETS_OLD_KEY='' \
+ secret rewrap-all --old-key-env ZB_SECRETS_OLD_KEY
+# Explicit-id deployment:
+# secret rewrap-all --old-key-env ZB_SECRETS_OLD_KEY --old-key-id sha256:abc… --new-key-id sha256:def…
+# Override the new key explicitly instead of the configured provider:
+# secret rewrap-all --old-key-env ZB_SECRETS_OLD_KEY --new-key-file /run/secrets/new_kek
+```
+
+Expected: `{"action":"rewrap-all","total":N,"rewrapped":N,"alreadyCurrent":0}` and exit 0. On a
+re-run everything reports `alreadyCurrent` and `rewrapped:0`.
+
+**5. For clustered / multi-store deployments, repeat per store.**
+
+- **Shared SQL-Server store (the G-7 primary path):** one store → run rotation **once**.
+- **Per-node SQLite on a shared KEK:** each node has its own store file → run rotation **on each
+ node** (all with the same old→new keys). Re-wrap does not replicate (revision preserved), so
+ every store must be migrated independently.
+
+**6. Cut the app over to the new KEK.** Ensure `Secrets:MasterKey` (and every node's KEK source)
+now resolves the **new** key, then start the app(s). Confirm a resolve succeeds
+(e.g. `secret get ` or an app readiness probe that exercises a `${secret:}` value).
+
+**7. Destroy the old key.** Remove the old key material from env/files/secret managers once every
+store is confirmed on the new KEK and resolves are healthy.
+
+## Rollback
+
+If step 6/7 surfaces a problem, restore the pre-rotation store backup (step 1) and revert the app
+to the old KEK. Because bodies were never re-encrypted, the backup is a complete, consistent undo.
+
+## Troubleshooting
+
+| Symptom | Cause | Action |
+|---|---|---|
+| `error: … old and new KEK identifiers are identical` | New KEK resolves to the same id as the old | You supplied the wrong new key, or forgot to stage it. Verify `Secrets:MasterKey`. |
+| `error: Secret '…' is wrapped by KEK 'X', which is neither the old nor the new KEK` | A row is on a third KEK (a prior partial/aborted rotation, or a mis-set id) | Investigate that row (`secret list` shows each row's `kekId`); re-point old/new keys or fix the row, then re-run (completed rows are skipped). |
+| `error: … DEK could not be unwrapped under the supplied old KEK` | Right `kek_id` but wrong key **bytes**, or a tampered row | Confirm you staged the correct old key bytes. If the row is genuinely tampered, restore from backup. |
+| Post-cutover resolves fail closed (`Row wrapped by unknown KEK`) | App still on the old KEK, or a store/node was missed in step 5 | Ensure every node's `Secrets:MasterKey` resolves the new key and every store was rewrapped. |
diff --git a/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Abstractions/ISecretCipher.cs b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Abstractions/ISecretCipher.cs
index 6fe9876..94a8461 100644
--- a/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Abstractions/ISecretCipher.cs
+++ b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Abstractions/ISecretCipher.cs
@@ -32,4 +32,34 @@ public interface ISecretCipher
/// The authentication tag fails to verify, or the row references an unknown or unavailable KEK.
///
string Decrypt(StoredSecret secret);
+
+ ///
+ /// Re-wraps the data-encryption key (DEK) of an existing row from to
+ /// — the KEK-rotation primitive. The DEK is unwrapped under the old
+ /// master key and re-wrapped under the new one; the sealed body
+ /// (//)
+ /// is never re-encrypted, and , the timestamps, the
+ /// tombstone flag, and every other field are preserved verbatim. Only
+ /// //
+ /// and change. Plaintext is never exposed.
+ ///
+ ///
+ /// This operation is provider-explicit: it uses only the two supplied providers and does
+ /// not consult whatever KEK the cipher instance was constructed with, because a rotation spans
+ /// two keys. The caller must have already established that is wrapped
+ /// by (the row's equals
+ /// of ); a mismatch fails closed.
+ ///
+ /// The stored row whose DEK should be re-wrapped.
+ /// The provider for the KEK that currently wraps the row's DEK.
+ /// The provider for the KEK the DEK should be re-wrapped under.
+ ///
+ /// A copy of with the DEK re-wrapped under
+ /// and set to the new KEK id; all other fields unchanged.
+ ///
+ ///
+ /// The row is not wrapped by , or the DEK fails to unwrap (wrong old
+ /// key material or a tampered wrap).
+ ///
+ StoredSecret Rewrap(StoredSecret secret, IMasterKeyProvider oldKek, IMasterKeyProvider newKek);
}
diff --git a/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Abstractions/ISecretStore.cs b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Abstractions/ISecretStore.cs
index 2662f3a..1a6a0fa 100644
--- a/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Abstractions/ISecretStore.cs
+++ b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Abstractions/ISecretStore.cs
@@ -63,4 +63,42 @@ public interface ISecretStore
/// A token to cancel the operation.
/// A task that completes when the row has been applied or ignored.
Task ApplyReplicatedAsync(StoredSecret row, CancellationToken ct);
+
+ ///
+ /// Applies a KEK re-wrap in place: overwrites only the KEK-wrap envelope columns
+ /// (//)
+ /// and for the row named by ,
+ /// leaving the sealed body, revision, timestamps, tombstone flag, and audit stamps untouched.
+ ///
+ ///
+ ///
+ /// This is the persistence half of KEK rotation. It deliberately does not bump the
+ /// revision or refresh updated_utc: a re-wrap does not change the secret's plaintext, so
+ /// it must stay invisible to the (updated_utc, revision) last-writer-wins ordering used by
+ /// cluster anti-entropy — otherwise every node's independent re-wrap would churn replication.
+ ///
+ ///
+ /// The update is a compare-and-swap on : the
+ /// row is updated only if its current still equals the DEK
+ /// wrap that was unwrapped to produce . A concurrent write to the
+ /// same secret (a set/rotate that generated a new DEK) changes that wrap, so the
+ /// CAS matches 0 rows and the row is left untouched — rather than corrupting it by pairing a
+ /// stale re-wrap with a newer body. The caller re-processes a 0-row result (fetch → re-wrap).
+ ///
+ ///
+ ///
+ /// The row carrying the new wrap envelope + ; matched by
+ /// . Only the four wrap-related columns are read from it.
+ ///
+ ///
+ /// The the row must still carry for the swap to apply — the
+ /// wrap that was unwrapped to produce .
+ ///
+ /// A token to cancel the operation.
+ ///
+ /// true if the row was updated; false if no row matched (absent, or its wrapped DEK
+ /// changed under a concurrent write).
+ ///
+ Task ApplyRewrapAsync(
+ StoredSecret rewrappedRow, byte[] expectedCurrentWrappedDek, CancellationToken ct);
}
diff --git a/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Cli/Program.cs b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Cli/Program.cs
index 465639d..e0ab45e 100644
--- a/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Cli/Program.cs
+++ b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Cli/Program.cs
@@ -1,9 +1,11 @@
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
+using Microsoft.Extensions.DependencyInjection;
using ZB.MOM.WW.Secrets.Abstractions;
using ZB.MOM.WW.Secrets.Cli;
using ZB.MOM.WW.Secrets.DependencyInjection;
+using ZB.MOM.WW.Secrets.MasterKey;
using ZB.MOM.WW.Secrets.Sqlite;
// Headless `secret` CLI: set / get / list / rm / rotate over the ZB.MOM.WW secrets store.
@@ -77,6 +79,41 @@ try
return await commands.RemoveAsync(args[1], actor, ct);
}
+ case "rewrap-all":
+ {
+ // Rotate the master KEK by re-wrapping every row's DEK. Key material is supplied ONLY by
+ // env-var NAME or file PATH — never as a literal argument (which would be echoed/logged).
+ string? oldKind = null, oldValue = null, oldKekId = null;
+ string? newKind = null, newValue = null, newKekId = null;
+
+ for (int i = 1; i < args.Length; i++)
+ {
+ switch (args[i])
+ {
+ case "--old-key-env": (oldKind, oldValue) = ("env", RequireValue(args, ref i)); break;
+ case "--old-key-file": (oldKind, oldValue) = ("file", RequireValue(args, ref i)); break;
+ case "--old-key-id": oldKekId = RequireValue(args, ref i); break;
+ case "--new-key-env": (newKind, newValue) = ("env", RequireValue(args, ref i)); break;
+ case "--new-key-file": (newKind, newValue) = ("file", RequireValue(args, ref i)); break;
+ case "--new-key-id": newKekId = RequireValue(args, ref i); break;
+ default: return Usage($"Unknown option '{args[i]}'.");
+ }
+ }
+
+ if (oldKind is null)
+ return Usage("rewrap-all requires --old-key-env or --old-key-file .");
+ if (newKind is null && newKekId is not null)
+ return Usage("--new-key-id requires --new-key-env or --new-key-file .");
+
+ IMasterKeyProvider oldKek = BuildProvider(oldKind, oldValue!, oldKekId);
+ // Default the new KEK to the app's configured provider (Secrets:MasterKey); override with flags.
+ IMasterKeyProvider newKek = newKind is null
+ ? host.Services.GetRequiredService()
+ : BuildProvider(newKind, newValue!, newKekId);
+
+ return await commands.RewrapAllAsync(oldKek, newKek, ct);
+ }
+
default:
return Usage($"Unknown command '{args[0]}'.");
}
@@ -132,6 +169,26 @@ static SecretContentType ParseContentType(string value) => value switch
$"Unknown --content-type '{value}'; expected text|connection-string|json|binary-base64."),
};
+// Reads the value that must follow an option flag at args[i], advancing i past it.
+static string RequireValue(string[] args, ref int i)
+{
+ if (i + 1 >= args.Length)
+ throw new ArgumentException($"{args[i]} requires a value.");
+ return args[++i];
+}
+
+// Builds a master-key provider from a source kind ("env" | "file") + its env-var name or file path,
+// with an optional explicit KEK id (needed when the app configured Secrets:MasterKey:KekId rather
+// than the derived sha256: id). The key VALUE itself is only ever read from the env var / file.
+static IMasterKeyProvider BuildProvider(string kind, string value, string? kekId) => kind switch
+{
+ "env" => MasterKeyProviderFactory.Create(
+ new MasterKeyOptions { Source = MasterKeySource.Environment, EnvVarName = value, KekId = kekId }),
+ "file" => MasterKeyProviderFactory.Create(
+ new MasterKeyOptions { Source = MasterKeySource.File, FilePath = value, KekId = kekId }),
+ _ => throw new ArgumentException($"Unknown key source '{kind}'."),
+};
+
// Prints usage (optionally preceded by an error line) and returns the standard usage exit code.
int Usage(string? error = null)
{
@@ -147,6 +204,14 @@ int Usage(string? error = null)
get
list [--include-deleted]
rm
+ rewrap-all --old-key-env |--old-key-file [--old-key-id ]
+ [--new-key-env |--new-key-file ] [--new-key-id ]
+
+ rewrap-all rotates the master KEK: it re-wraps every stored secret's data key from the old
+ KEK to the new one (bodies are never re-encrypted, no plaintext is exposed). The new KEK
+ defaults to the configured Secrets:MasterKey provider. Supply key material only by env-var
+ NAME or file PATH — never as a literal. Run it with resolve traffic quiesced; it is
+ idempotent and safe to re-run.
""");
return 2;
}
diff --git a/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Cli/SecretCommands.cs b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Cli/SecretCommands.cs
index 5a6ede4..25e9914 100644
--- a/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Cli/SecretCommands.cs
+++ b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Cli/SecretCommands.cs
@@ -1,5 +1,6 @@
using System.Text.Json;
using ZB.MOM.WW.Secrets.Abstractions;
+using ZB.MOM.WW.Secrets.Rotation;
namespace ZB.MOM.WW.Secrets.Cli;
@@ -156,6 +157,47 @@ public sealed class SecretCommands
return removed ? 0 : 1;
}
+ ///
+ /// rewrap-all: rotates the master KEK by re-wrapping every row's DEK from
+ /// onto — bodies are never re-encrypted and no plaintext is exposed.
+ /// Prints a JSON report of the counts (total / rewrapped / already-current) and NEVER any key
+ /// material. Idempotent + resumable; fails closed (non-zero) on an identical-KEK mistake or a row
+ /// wrapped by an unrecognized KEK.
+ ///
+ /// The provider for the KEK rows are currently wrapped under.
+ /// The provider for the KEK rows should be re-wrapped onto.
+ /// A token to cancel the operation.
+ /// 0 on success; non-zero if the rotation was rejected or aborted.
+ public async Task RewrapAllAsync(
+ IMasterKeyProvider oldKek, IMasterKeyProvider newKek, CancellationToken ct)
+ {
+ var service = new KekRotationService(_store, _cipher);
+ try
+ {
+ RewrapReport report = await service.RewrapAllAsync(oldKek, newKek, ct).ConfigureAwait(false);
+ WriteJson(new
+ {
+ action = "rewrap-all",
+ total = report.Total,
+ rewrapped = report.Rewrapped,
+ alreadyCurrent = report.AlreadyCurrent,
+ });
+ return 0;
+ }
+ catch (ArgumentException ex)
+ {
+ // Identical old/new KEK ids — nothing to rotate.
+ WriteJson(new { action = "rewrap-all", error = ex.Message });
+ return 1;
+ }
+ catch (SecretDecryptionException ex)
+ {
+ // A row on an unrecognized KEK aborted the pass, or a DEK failed to unwrap.
+ WriteJson(new { action = "rewrap-all", error = ex.Message });
+ return 1;
+ }
+ }
+
/// Seals a value under the cipher and upserts it, carrying description + actor stamps.
private async Task UpsertSealedAsync(
SecretName name, string value, SecretContentType contentType, string? description, string? actor, CancellationToken ct)
diff --git a/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Crypto/AesGcmEnvelopeCipher.cs b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Crypto/AesGcmEnvelopeCipher.cs
index c5d580c..e1de368 100644
--- a/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Crypto/AesGcmEnvelopeCipher.cs
+++ b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Crypto/AesGcmEnvelopeCipher.cs
@@ -153,4 +153,75 @@ public sealed class AesGcmEnvelopeCipher : ISecretCipher
CryptographicOperations.ZeroMemory(dek);
}
}
+
+ ///
+ public StoredSecret Rewrap(StoredSecret secret, IMasterKeyProvider oldKek, IMasterKeyProvider newKek)
+ {
+ ArgumentNullException.ThrowIfNull(secret);
+ ArgumentNullException.ThrowIfNull(oldKek);
+ ArgumentNullException.ThrowIfNull(newKek);
+
+ // Fail closed if the row is not actually wrapped by the supplied old KEK — a caller must
+ // not silently "rewrap" a row that is already on a different key.
+ if (!string.Equals(secret.KekId, oldKek.KekId, StringComparison.Ordinal))
+ {
+ throw new SecretDecryptionException(
+ $"Cannot rewrap secret '{secret.Name.Value}': row is wrapped by KEK '{secret.KekId}', " +
+ $"not the supplied old KEK '{oldKek.KekId}'.");
+ }
+
+ ReadOnlySpan oldMasterKey = oldKek.GetMasterKey().Span;
+ ReadOnlySpan newMasterKey = newKek.GetMasterKey().Span;
+ string newKekId = newKek.KekId;
+
+ byte[] dek = new byte[KeySizeBytes];
+ try
+ {
+ // Unwrap the DEK under the OLD master key, verifying the old KEK-id AAD binding. A wrong
+ // old key or tampered wrap surfaces as SecretDecryptionException (never a raw crypto leak).
+ byte[] oldKekAad = Encoding.UTF8.GetBytes(secret.KekId);
+ try
+ {
+ using var unwrap = new AesGcm(oldMasterKey, TagSizeBytes);
+ unwrap.Decrypt(secret.WrapNonce, secret.WrappedDek, secret.WrapTag, dek, oldKekAad);
+ }
+ // CryptographicException = wrong key / tampered wrap; ArgumentException = a malformed
+ // (wrong-length) wrap nonce/tag on a corrupt row. Both fail closed as a decryption error
+ // rather than leaking a raw crypto/arg exception (which the caller would misclassify).
+ catch (Exception ex) when (ex is CryptographicException or ArgumentException)
+ {
+ throw new SecretDecryptionException(
+ $"Failed to rewrap secret '{secret.Name.Value}': the DEK could not be unwrapped " +
+ "under the supplied old KEK.", ex);
+ }
+
+ // Re-wrap the SAME DEK under the NEW master key with a fresh nonce, binding the new KEK id
+ // as AAD. The body ciphertext is left untouched — rotation re-wraps DEKs only.
+ byte[] wrapNonce = new byte[NonceSizeBytes];
+ byte[] wrapTag = new byte[TagSizeBytes];
+ byte[] wrappedDek = new byte[KeySizeBytes];
+ byte[] newKekAad = Encoding.UTF8.GetBytes(newKekId);
+
+ RandomNumberGenerator.Fill(wrapNonce);
+ using (var wrapGcm = new AesGcm(newMasterKey, TagSizeBytes))
+ {
+ wrapGcm.Encrypt(wrapNonce, dek, wrappedDek, wrapTag, newKekAad);
+ }
+
+ // Only the KEK-wrap envelope + kek_id change; body, revision, timestamps, tombstone,
+ // content-type, and every other field are preserved so a rewrap is invisible to the
+ // (updated_utc, revision) last-writer-wins ordering used by cluster anti-entropy.
+ return secret with
+ {
+ WrappedDek = wrappedDek,
+ WrapNonce = wrapNonce,
+ WrapTag = wrapTag,
+ KekId = newKekId,
+ };
+ }
+ finally
+ {
+ CryptographicOperations.ZeroMemory(dek);
+ }
+ }
}
diff --git a/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Rotation/KekRotationService.cs b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Rotation/KekRotationService.cs
new file mode 100644
index 0000000..c4db89a
--- /dev/null
+++ b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Rotation/KekRotationService.cs
@@ -0,0 +1,155 @@
+using ZB.MOM.WW.Secrets.Abstractions;
+
+namespace ZB.MOM.WW.Secrets.Rotation;
+
+///
+/// Rotates the master key-encryption key (KEK) across every stored secret by re-wrapping each
+/// row's data-encryption key (DEK) from the old KEK to the new one. Because the envelope model
+/// wraps a per-secret DEK under the KEK, rotation never re-encrypts secret bodies — only the DEK
+/// wrap changes — so plaintext is never exposed and no value history is touched.
+///
+///
+///
+/// The pass is idempotent and resumable: a row already wrapped by the new KEK is skipped
+/// (counted in ), so a run interrupted part-way can simply
+/// be re-run to finish. It is fail-closed: a row wrapped by neither the old nor the new KEK
+/// aborts the pass (rather than silently leaving an un-migratable row that would become
+/// undecryptable once the old KEK is retired); rows already re-wrapped in the aborted run stay
+/// persisted, so a re-run resumes cleanly after the anomaly is investigated.
+///
+///
+/// Rotation is a per-store operation and must be run once per independent store (once for a shared
+/// SQL-Server store; once per node for a per-node SQLite store on a shared KEK). It should be run
+/// with resolve traffic quiesced: the persistence path deliberately does not bump revision or
+/// updated_utc, so a re-wrap is invisible to cluster last-writer-wins reconciliation.
+///
+///
+public sealed class KekRotationService
+{
+ private readonly ISecretStore _store;
+ private readonly ISecretCipher _cipher;
+
+ /// Creates the rotation service over the encrypted store and envelope cipher.
+ /// The store to enumerate and re-wrap in place.
+ /// The envelope cipher providing the DEK re-wrap primitive.
+ public KekRotationService(ISecretStore store, ISecretCipher cipher)
+ {
+ _store = store ?? throw new ArgumentNullException(nameof(store));
+ _cipher = cipher ?? throw new ArgumentNullException(nameof(cipher));
+ }
+
+ ///
+ /// Re-wraps every row from onto .
+ ///
+ /// The provider for the KEK rows are currently wrapped under.
+ /// The provider for the KEK rows should be re-wrapped onto.
+ /// A token to cancel the pass (already-persisted re-wraps are retained).
+ /// A summarizing the pass (no secret material).
+ ///
+ /// The old and new KEK identifiers are the same — there is nothing to rotate.
+ ///
+ ///
+ /// A row is wrapped by a KEK that is neither the old nor the new one (the pass aborts), or a
+ /// row's DEK fails to unwrap under the supplied old KEK.
+ ///
+ public async Task RewrapAllAsync(
+ IMasterKeyProvider oldKek, IMasterKeyProvider newKek, CancellationToken ct)
+ {
+ ArgumentNullException.ThrowIfNull(oldKek);
+ ArgumentNullException.ThrowIfNull(newKek);
+
+ if (string.Equals(oldKek.KekId, newKek.KekId, StringComparison.Ordinal))
+ {
+ throw new ArgumentException(
+ $"The old and new KEK identifiers are identical ('{oldKek.KekId}'); there is nothing to rotate.",
+ nameof(newKek));
+ }
+
+ // Enumerate metadata (never ciphertext) for ALL rows including tombstones — a tombstoned row
+ // still carries a KEK-wrapped DEK and would become un-migratable if left on a retired KEK.
+ IReadOnlyList all =
+ await _store.ListAsync(includeDeleted: true, ct).ConfigureAwait(false);
+
+ int rewrapped = 0;
+ int alreadyCurrent = 0;
+
+ foreach (SecretMetadata meta in all)
+ {
+ ct.ThrowIfCancellationRequested();
+
+ // Fast-path skip on the metadata kek_id (no ciphertext fetch): a row already on the new
+ // KEK is a completed migration — this is what makes the pass idempotent / resumable.
+ if (string.Equals(meta.KekId, newKek.KekId, StringComparison.Ordinal))
+ {
+ alreadyCurrent++;
+ continue;
+ }
+
+ switch (await RewrapOneAsync(meta.Name, oldKek, newKek, rewrapped, ct).ConfigureAwait(false))
+ {
+ case RowOutcome.Rewrapped:
+ rewrapped++;
+ break;
+ case RowOutcome.AlreadyCurrent:
+ alreadyCurrent++;
+ break;
+ case RowOutcome.Vanished:
+ break;
+ }
+ }
+
+ return new RewrapReport { Total = all.Count, Rewrapped = rewrapped, AlreadyCurrent = alreadyCurrent };
+ }
+
+ private enum RowOutcome { Rewrapped, AlreadyCurrent, Vanished }
+
+ // Re-wraps a single row with a bounded fetch→rewrap→compare-and-swap retry so a rare concurrent
+ // write (the pass should run with writes quiesced) is retried rather than corrupting the row.
+ private async Task RewrapOneAsync(
+ SecretName name, IMasterKeyProvider oldKek, IMasterKeyProvider newKek, int rewrappedSoFar, CancellationToken ct)
+ {
+ const int maxAttempts = 5;
+ for (int attempt = 1; ; attempt++)
+ {
+ ct.ThrowIfCancellationRequested();
+
+ StoredSecret? row = await _store.GetAsync(name, ct).ConfigureAwait(false);
+ if (row is null)
+ {
+ // Vanished between the list and the fetch (a concurrent hard-absence).
+ return RowOutcome.Vanished;
+ }
+
+ // Re-evaluate against the FRESH row: a concurrent write may have already moved it.
+ if (string.Equals(row.KekId, newKek.KekId, StringComparison.Ordinal))
+ {
+ return RowOutcome.AlreadyCurrent;
+ }
+
+ if (!string.Equals(row.KekId, oldKek.KekId, StringComparison.Ordinal))
+ {
+ throw new SecretDecryptionException(
+ $"Secret '{name.Value}' is wrapped by KEK '{row.KekId}', which is neither the old " +
+ $"('{oldKek.KekId}') nor the new ('{newKek.KekId}') KEK. Rotation aborted after re-wrapping " +
+ $"{rewrappedSoFar} row(s); investigate the anomalous row and re-run to resume.");
+ }
+
+ // Rewrap re-verifies row.KekId == oldKek.KekId and fails closed on a wrong old key.
+ StoredSecret rewrappedRow = _cipher.Rewrap(row, oldKek, newKek);
+
+ // Compare-and-swap on the wrap we just unwrapped: a concurrent set/rotate changed it →
+ // 0 rows → retry (fetch the newer row and re-evaluate).
+ if (await _store.ApplyRewrapAsync(rewrappedRow, row.WrappedDek, ct).ConfigureAwait(false))
+ {
+ return RowOutcome.Rewrapped;
+ }
+
+ if (attempt >= maxAttempts)
+ {
+ throw new SecretDecryptionException(
+ $"Secret '{name.Value}' is being modified concurrently; rotation could not converge after " +
+ $"{maxAttempts} attempts. Re-run with secret writes quiesced.");
+ }
+ }
+ }
+}
diff --git a/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Rotation/RewrapReport.cs b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Rotation/RewrapReport.cs
new file mode 100644
index 0000000..ab1f736
--- /dev/null
+++ b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Rotation/RewrapReport.cs
@@ -0,0 +1,21 @@
+namespace ZB.MOM.WW.Secrets.Rotation;
+
+///
+/// The outcome of a pass: how many rows were
+/// re-wrapped onto the new KEK, how many were already on it (skipped), and the total scanned.
+/// Carries no secret material.
+///
+public sealed record RewrapReport
+{
+ /// Total number of rows scanned (includes tombstoned rows).
+ public required int Total { get; init; }
+
+ /// Number of rows re-wrapped from the old KEK onto the new KEK in this pass.
+ public required int Rewrapped { get; init; }
+
+ ///
+ /// Number of rows already wrapped by the new KEK and therefore skipped — the count that makes a
+ /// re-run idempotent (a completed rotation reports 0 and all rows here).
+ ///
+ public required int AlreadyCurrent { get; init; }
+}
diff --git a/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Sqlite/SqliteSecretStore.cs b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Sqlite/SqliteSecretStore.cs
index 4b5afd5..64a2f44 100644
--- a/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Sqlite/SqliteSecretStore.cs
+++ b/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Sqlite/SqliteSecretStore.cs
@@ -260,6 +260,43 @@ public sealed class SqliteSecretStore(SecretsSqliteConnectionFactory connectionF
await transaction.CommitAsync(ct).ConfigureAwait(false);
}
+ ///
+ public async Task ApplyRewrapAsync(
+ StoredSecret rewrappedRow, byte[] expectedCurrentWrappedDek, CancellationToken ct)
+ {
+ ArgumentNullException.ThrowIfNull(rewrappedRow);
+ ArgumentNullException.ThrowIfNull(expectedCurrentWrappedDek);
+
+ await using SqliteConnection connection =
+ await connectionFactory.OpenConnectionAsync(ct).ConfigureAwait(false);
+
+ // Overwrite ONLY the KEK-wrap envelope + kek_id. Revision, updated_utc/by, the sealed body,
+ // and the tombstone state are deliberately untouched — a re-wrap is not a logical change, so
+ // it must not bump the revision or updated timestamp (see ISecretStore.ApplyRewrapAsync).
+ //
+ // Compare-and-swap on wrapped_dek: a concurrent set/rotate generates a fresh (random) DEK
+ // wrap, so this matches 0 rows instead of pairing this stale re-wrap with a newer body —
+ // which would leave the row permanently undecryptable. The caller re-processes a 0-row miss.
+ await using SqliteCommand command = connection.CreateCommand();
+ command.CommandText = """
+ UPDATE secret SET
+ wrapped_dek = $wrapped_dek,
+ wrap_nonce = $wrap_nonce,
+ wrap_tag = $wrap_tag,
+ kek_id = $kek_id
+ WHERE name = $name AND wrapped_dek = $expected_wrapped_dek;
+ """;
+ command.Parameters.AddWithValue("$wrapped_dek", rewrappedRow.WrappedDek);
+ command.Parameters.AddWithValue("$wrap_nonce", rewrappedRow.WrapNonce);
+ command.Parameters.AddWithValue("$wrap_tag", rewrappedRow.WrapTag);
+ command.Parameters.AddWithValue("$kek_id", rewrappedRow.KekId);
+ command.Parameters.AddWithValue("$name", rewrappedRow.Name.Value);
+ command.Parameters.AddWithValue("$expected_wrapped_dek", expectedCurrentWrappedDek);
+
+ int rowsAffected = await command.ExecuteNonQueryAsync(ct).ConfigureAwait(false);
+ return rowsAffected > 0;
+ }
+
// Binds the identity, description, content-type, KEK id, and all six crypto BLOB columns
// shared by every insert path.
private static void BindCryptoColumns(SqliteCommand command, StoredSecret row)
diff --git a/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Cli/SecretCommandsTests.cs b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Cli/SecretCommandsTests.cs
index 5285464..78028b5 100644
--- a/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Cli/SecretCommandsTests.cs
+++ b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Cli/SecretCommandsTests.cs
@@ -19,6 +19,7 @@ public sealed class SecretCommandsTests : IAsyncLifetime, IDisposable
private readonly SecretsSqliteConnectionFactory _factory;
private readonly SqliteSecretStore _store;
+ private readonly FakeMasterKeyProvider _oldKek = new("kek-cli");
private readonly AesGcmEnvelopeCipher _cipher;
private readonly DefaultSecretResolver _resolver;
@@ -26,7 +27,7 @@ public sealed class SecretCommandsTests : IAsyncLifetime, IDisposable
{
_factory = new SecretsSqliteConnectionFactory(_dbPath);
_store = new SqliteSecretStore(_factory);
- _cipher = new AesGcmEnvelopeCipher(new FakeMasterKeyProvider("kek-cli"));
+ _cipher = new AesGcmEnvelopeCipher(_oldKek);
// A near-zero TTL keeps the resolver from serving a stale plaintext across a rotate/remove.
_resolver = new DefaultSecretResolver(
_store, _cipher, NoOpAuditWriter.Instance, TimeSpan.Zero);
@@ -53,6 +54,45 @@ public sealed class SecretCommandsTests : IAsyncLifetime, IDisposable
private static CancellationToken Ct => CancellationToken.None;
+ [Fact]
+ public async Task RewrapAll_MigratesRowsToNewKek_ReportsCountsWithoutLeakingValues()
+ {
+ var (seed, _) = NewSut();
+ await seed.SetAsync("sql/a", "value-a", SecretContentType.Text, null, "alice", Ct);
+ await seed.SetAsync("sql/b", "value-b", SecretContentType.Text, null, "alice", Ct);
+
+ var newKek = new FakeMasterKeyProvider("kek-new");
+ var (cmd, output) = NewSut();
+ int rc = await cmd.RewrapAllAsync(_oldKek, newKek, Ct);
+
+ Assert.Equal(0, rc);
+ string outStr = output.ToString();
+ Assert.Contains("\"action\":\"rewrap-all\"", outStr);
+ Assert.Contains("\"total\":2", outStr);
+ Assert.Contains("\"rewrapped\":2", outStr);
+ Assert.Contains("\"alreadyCurrent\":0", outStr);
+ // The report carries counts only — never a secret value.
+ Assert.DoesNotContain("value-a", outStr);
+ Assert.DoesNotContain("value-b", outStr);
+
+ // The rows now decrypt under the NEW KEK (proof the rewrap actually re-keyed them).
+ var newCipher = new AesGcmEnvelopeCipher(newKek);
+ Assert.Equal("value-a", newCipher.Decrypt((await _store.GetAsync(new SecretName("sql/a"), Ct))!));
+ }
+
+ [Fact]
+ public async Task RewrapAll_IdenticalKek_ReturnsErrorExitCode()
+ {
+ var (seed, _) = NewSut();
+ await seed.SetAsync("sql/a", "value-a", SecretContentType.Text, null, "alice", Ct);
+
+ var (cmd, output) = NewSut();
+ int rc = await cmd.RewrapAllAsync(_oldKek, new FakeMasterKeyProvider("kek-cli"), Ct);
+
+ Assert.Equal(1, rc);
+ Assert.Contains("\"error\"", output.ToString());
+ }
+
[Fact]
public async Task Set_Then_Get_RoundTrips()
{
diff --git a/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Crypto/AesGcmEnvelopeCipherTests.cs b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Crypto/AesGcmEnvelopeCipherTests.cs
index b7f8eb1..238a3e5 100644
--- a/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Crypto/AesGcmEnvelopeCipherTests.cs
+++ b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Crypto/AesGcmEnvelopeCipherTests.cs
@@ -95,6 +95,97 @@ public class AesGcmEnvelopeCipherTests
Assert.Contains("unknown KEK", ex.Message, StringComparison.Ordinal);
}
+ [Fact]
+ public void Rewrap_RewrapsDek_BodyStillDecryptsUnderNewKek()
+ {
+ var oldKek = new FakeMasterKeyProvider("kek-old");
+ var newKek = new FakeMasterKeyProvider("kek-new");
+ var oldCipher = new AesGcmEnvelopeCipher(oldKek);
+ var name = new SecretName("sql/foo");
+
+ StoredSecret row = oldCipher.Encrypt(name, "hunter2", SecretContentType.Text);
+ Assert.Equal("kek-old", row.KekId);
+
+ // Rewrap is provider-explicit — it can be driven through any cipher instance.
+ StoredSecret rewrapped = oldCipher.Rewrap(row, oldKek, newKek);
+
+ Assert.Equal("kek-new", rewrapped.KekId);
+ // The DEK-sealed body is untouched; only the KEK-wrap envelope changed.
+ Assert.True(rewrapped.Ciphertext.AsSpan().SequenceEqual(row.Ciphertext));
+ Assert.True(rewrapped.Nonce.AsSpan().SequenceEqual(row.Nonce));
+ Assert.True(rewrapped.Tag.AsSpan().SequenceEqual(row.Tag));
+ Assert.False(rewrapped.WrappedDek.AsSpan().SequenceEqual(row.WrappedDek));
+
+ // The rewrapped row decrypts under the NEW KEK...
+ var newCipher = new AesGcmEnvelopeCipher(newKek);
+ Assert.Equal("hunter2", newCipher.Decrypt(rewrapped));
+ // ...and no longer under the OLD KEK (kek_id now names the new KEK; old cipher fails closed).
+ Assert.Throws(() => oldCipher.Decrypt(rewrapped));
+ }
+
+ [Fact]
+ public void Rewrap_PreservesRevisionTimestampsTombstoneAndMetadata()
+ {
+ var oldKek = new FakeMasterKeyProvider("kek-old");
+ var newKek = new FakeMasterKeyProvider("kek-new");
+ var cipher = new AesGcmEnvelopeCipher(oldKek);
+
+ StoredSecret row = cipher.Encrypt(new SecretName("sql/foo"), "hunter2", SecretContentType.Text) with
+ {
+ Revision = 7,
+ IsDeleted = true,
+ DeletedUtc = DateTimeOffset.UnixEpoch,
+ CreatedUtc = DateTimeOffset.UnixEpoch,
+ UpdatedUtc = DateTimeOffset.UnixEpoch.AddMinutes(5),
+ CreatedBy = "alice",
+ UpdatedBy = "bob",
+ Description = "the foo password",
+ };
+
+ StoredSecret rewrapped = cipher.Rewrap(row, oldKek, newKek);
+
+ // Only the wrap envelope + kek_id change — a rewrap must be invisible to (updated_utc, revision) LWW.
+ Assert.Equal(7, rewrapped.Revision);
+ Assert.True(rewrapped.IsDeleted);
+ Assert.Equal(row.DeletedUtc, rewrapped.DeletedUtc);
+ Assert.Equal(row.CreatedUtc, rewrapped.CreatedUtc);
+ Assert.Equal(row.UpdatedUtc, rewrapped.UpdatedUtc);
+ Assert.Equal("alice", rewrapped.CreatedBy);
+ Assert.Equal("bob", rewrapped.UpdatedBy);
+ Assert.Equal("the foo password", rewrapped.Description);
+ Assert.Equal(SecretContentType.Text, rewrapped.ContentType);
+ }
+
+ [Fact]
+ public void Rewrap_WrongOldKekId_FailsClosed()
+ {
+ var actualKek = new FakeMasterKeyProvider("kek-old");
+ var wrongOld = new FakeMasterKeyProvider("kek-somethingelse");
+ var newKek = new FakeMasterKeyProvider("kek-new");
+ var cipher = new AesGcmEnvelopeCipher(actualKek);
+
+ StoredSecret row = cipher.Encrypt(new SecretName("sql/foo"), "hunter2", SecretContentType.Text);
+
+ // Row is wrapped by "kek-old" but the caller claims the old KEK is "kek-somethingelse".
+ var ex = Assert.Throws(() => cipher.Rewrap(row, wrongOld, newKek));
+ Assert.Contains("kek-old", ex.Message, StringComparison.Ordinal);
+ }
+
+ [Fact]
+ public void Rewrap_CorrectKekIdButWrongKeyMaterial_FailsClosed()
+ {
+ // Same KEK-id string, different key bytes → the unwrap must fail closed (tag mismatch),
+ // never emitting a garbage DEK.
+ var realOld = new FakeMasterKeyProvider("kek-old");
+ var imposterOld = new FakeMasterKeyProvider("kek-old"); // same id, fresh random bytes
+ var newKek = new FakeMasterKeyProvider("kek-new");
+ var cipher = new AesGcmEnvelopeCipher(realOld);
+
+ StoredSecret row = cipher.Encrypt(new SecretName("sql/foo"), "hunter2", SecretContentType.Text);
+
+ Assert.Throws(() => cipher.Rewrap(row, imposterOld, newKek));
+ }
+
private static bool Contains(byte[] haystack, byte[] needle)
{
if (needle.Length == 0 || haystack.Length < needle.Length)
diff --git a/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Fakes/CountingSecretStore.cs b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Fakes/CountingSecretStore.cs
index 1ffd006..d1bc10e 100644
--- a/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Fakes/CountingSecretStore.cs
+++ b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Fakes/CountingSecretStore.cs
@@ -48,4 +48,9 @@ public sealed class CountingSecretStore : ISecretStore
///
public Task ApplyReplicatedAsync(StoredSecret row, CancellationToken ct) =>
throw new NotSupportedException();
+
+ ///
+ public Task ApplyRewrapAsync(
+ StoredSecret rewrappedRow, byte[] expectedCurrentWrappedDek, CancellationToken ct) =>
+ throw new NotSupportedException();
}
diff --git a/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Rotation/KekRotationServiceTests.cs b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Rotation/KekRotationServiceTests.cs
new file mode 100644
index 0000000..b35e933
--- /dev/null
+++ b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Rotation/KekRotationServiceTests.cs
@@ -0,0 +1,163 @@
+using Microsoft.Data.Sqlite;
+using ZB.MOM.WW.Secrets.Abstractions;
+using ZB.MOM.WW.Secrets.Crypto;
+using ZB.MOM.WW.Secrets.Rotation;
+using ZB.MOM.WW.Secrets.Sqlite;
+using ZB.MOM.WW.Secrets.Tests.Fakes;
+
+namespace ZB.MOM.WW.Secrets.Tests.Rotation;
+
+///
+/// End-to-end KEK-rotation tests against the REAL SQLite store and REAL AES-256-GCM cipher — the
+/// combination that actually proves a re-wrapped row decrypts under the new KEK (and no longer
+/// under the old one).
+///
+public sealed class KekRotationServiceTests : IAsyncLifetime, IDisposable
+{
+ private readonly string _dbPath =
+ Path.Combine(Path.GetTempPath(), $"zb-secrets-rotate-{Guid.NewGuid():N}.db");
+
+ private readonly SecretsSqliteConnectionFactory _factory;
+ private readonly SqliteSecretStore _store;
+
+ private readonly FakeMasterKeyProvider _oldKek = new("kek-old");
+ private readonly FakeMasterKeyProvider _newKek = new("kek-new");
+
+ public KekRotationServiceTests()
+ {
+ _factory = new SecretsSqliteConnectionFactory(_dbPath);
+ _store = new SqliteSecretStore(_factory);
+ }
+
+ public async Task InitializeAsync() =>
+ await new SqliteSecretsStoreMigrator(_factory).MigrateAsync(CancellationToken.None);
+
+ public Task DisposeAsync() => Task.CompletedTask;
+
+ // Seeds a secret sealed under a given KEK provider (defaults to the old KEK).
+ private async Task SeedAsync(string name, string value, IMasterKeyProvider? kek = null)
+ {
+ var cipher = new AesGcmEnvelopeCipher(kek ?? _oldKek);
+ StoredSecret row = cipher.Encrypt(new SecretName(name), value, SecretContentType.Text);
+ await _store.UpsertAsync(row, CancellationToken.None);
+ }
+
+ private KekRotationService Service() => new(_store, new AesGcmEnvelopeCipher(_oldKek));
+
+ [Fact]
+ public async Task RewrapAll_MigratesEveryRow_DecryptsUnderNewKekOnly()
+ {
+ await SeedAsync("sql/a", "value-a");
+ await SeedAsync("sql/b", "value-b");
+ await SeedAsync("ldap/c", "value-c");
+
+ RewrapReport report = await Service().RewrapAllAsync(_oldKek, _newKek, CancellationToken.None);
+
+ Assert.Equal(3, report.Total);
+ Assert.Equal(3, report.Rewrapped);
+ Assert.Equal(0, report.AlreadyCurrent);
+
+ var newCipher = new AesGcmEnvelopeCipher(_newKek);
+ var oldCipher = new AesGcmEnvelopeCipher(_oldKek);
+ foreach ((string name, string value) in new[] { ("sql/a", "value-a"), ("sql/b", "value-b"), ("ldap/c", "value-c") })
+ {
+ StoredSecret row = (await _store.GetAsync(new SecretName(name), CancellationToken.None))!;
+ Assert.Equal("kek-new", row.KekId);
+ // Decrypts cleanly under the NEW KEK...
+ Assert.Equal(value, newCipher.Decrypt(row));
+ // ...and fails closed under the OLD KEK (kek_id no longer matches).
+ Assert.Throws(() => oldCipher.Decrypt(row));
+ }
+ }
+
+ [Fact]
+ public async Task RewrapAll_IsIdempotent_SecondRunSkipsEverything()
+ {
+ await SeedAsync("sql/a", "value-a");
+ await SeedAsync("sql/b", "value-b");
+
+ RewrapReport first = await Service().RewrapAllAsync(_oldKek, _newKek, CancellationToken.None);
+ Assert.Equal(2, first.Rewrapped);
+
+ RewrapReport second = await Service().RewrapAllAsync(_oldKek, _newKek, CancellationToken.None);
+ Assert.Equal(2, second.Total);
+ Assert.Equal(0, second.Rewrapped);
+ Assert.Equal(2, second.AlreadyCurrent);
+ }
+
+ [Fact]
+ public async Task RewrapAll_RewrapsTombstonedRows()
+ {
+ await SeedAsync("sql/live", "live");
+ await SeedAsync("sql/dead", "dead");
+ await _store.DeleteAsync(new SecretName("sql/dead"), "carol", CancellationToken.None);
+
+ RewrapReport report = await Service().RewrapAllAsync(_oldKek, _newKek, CancellationToken.None);
+
+ Assert.Equal(2, report.Rewrapped);
+ StoredSecret dead = (await _store.GetAsync(new SecretName("sql/dead"), CancellationToken.None))!;
+ Assert.Equal("kek-new", dead.KekId);
+ Assert.True(dead.IsDeleted);
+ }
+
+ [Fact]
+ public async Task RewrapAll_DoesNotBumpRevisionOrUpdatedUtc()
+ {
+ await SeedAsync("sql/a", "value-a");
+ StoredSecret before = (await _store.GetAsync(new SecretName("sql/a"), CancellationToken.None))!;
+
+ await Service().RewrapAllAsync(_oldKek, _newKek, CancellationToken.None);
+
+ StoredSecret after = (await _store.GetAsync(new SecretName("sql/a"), CancellationToken.None))!;
+ Assert.Equal(before.Revision, after.Revision);
+ Assert.Equal(before.UpdatedUtc, after.UpdatedUtc);
+ }
+
+ [Fact]
+ public async Task RewrapAll_IdenticalKekIds_Throws()
+ {
+ await SeedAsync("sql/a", "value-a");
+ var same = new FakeMasterKeyProvider("kek-old");
+
+ await Assert.ThrowsAsync(
+ () => Service().RewrapAllAsync(_oldKek, same, CancellationToken.None));
+ }
+
+ [Fact]
+ public async Task RewrapAll_RowOnUnknownKek_Aborts_ButKeepsPriorProgress()
+ {
+ // Names are scanned in ORDER BY name: "a-old" and "b-old" migrate, then "z-third" (a KEK that
+ // is neither old nor new) aborts the pass.
+ await SeedAsync("a-old", "va");
+ await SeedAsync("b-old", "vb");
+ await SeedAsync("z-third", "vz", new FakeMasterKeyProvider("kek-third"));
+
+ await Assert.ThrowsAsync(
+ () => Service().RewrapAllAsync(_oldKek, _newKek, CancellationToken.None));
+
+ // The two old-KEK rows scanned before the anomaly are already persisted on the new KEK,
+ // so a re-run (after removing the anomaly) resumes cleanly.
+ Assert.Equal("kek-new", (await _store.GetAsync(new SecretName("a-old"), CancellationToken.None))!.KekId);
+ Assert.Equal("kek-new", (await _store.GetAsync(new SecretName("b-old"), CancellationToken.None))!.KekId);
+ Assert.Equal("kek-third", (await _store.GetAsync(new SecretName("z-third"), CancellationToken.None))!.KekId);
+ }
+
+ public void Dispose()
+ {
+ SqliteConnection.ClearAllPools();
+ foreach (string path in new[] { _dbPath, _dbPath + "-wal", _dbPath + "-shm" })
+ {
+ try
+ {
+ if (File.Exists(path))
+ {
+ File.Delete(path);
+ }
+ }
+ catch (IOException)
+ {
+ // Best-effort temp cleanup.
+ }
+ }
+ }
+}
diff --git a/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Sqlite/SqliteSecretStoreTests.cs b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Sqlite/SqliteSecretStoreTests.cs
index 16bd68a..eba6e24 100644
--- a/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Sqlite/SqliteSecretStoreTests.cs
+++ b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Tests/Sqlite/SqliteSecretStoreTests.cs
@@ -204,6 +204,101 @@ public sealed class SqliteSecretStoreTests : IAsyncLifetime, IDisposable
Assert.Equal(new byte[] { 6, 6, 6 }, afterStale.Ciphertext);
}
+ [Fact]
+ public async Task ApplyRewrap_ChangesOnlyWrapEnvelopeAndKekId_PreservingRevisionBodyAndTimestamps()
+ {
+ DateTimeOffset created = new(2026, 1, 1, 0, 0, 0, TimeSpan.Zero);
+ StoredSecret seed = MakeSecret("app/rewrap", ciphertext: [1, 2, 3], createdUtc: created);
+ await _store.UpsertAsync(seed, CancellationToken.None);
+ // Bump the revision once so we can prove ApplyRewrap does NOT change it.
+ await _store.UpsertAsync(MakeSecret("app/rewrap", ciphertext: [1, 2, 3]), CancellationToken.None);
+
+ StoredSecret before = (await _store.GetAsync(new SecretName("app/rewrap"), CancellationToken.None))!;
+ Assert.Equal(1, before.Revision);
+
+ // Re-wrap: only the four wrap-related fields differ; everything else is carried from `before`.
+ StoredSecret rewrapped = before with
+ {
+ WrappedDek = [90, 91, 92],
+ WrapNonce = [93, 94, 95],
+ WrapTag = [96, 97, 98],
+ KekId = "kek-2",
+ };
+ bool applied = await _store.ApplyRewrapAsync(rewrapped, before.WrappedDek, CancellationToken.None);
+ Assert.True(applied);
+
+ StoredSecret after = (await _store.GetAsync(new SecretName("app/rewrap"), CancellationToken.None))!;
+ // Wrap envelope + kek_id changed...
+ Assert.Equal(new byte[] { 90, 91, 92 }, after.WrappedDek);
+ Assert.Equal(new byte[] { 93, 94, 95 }, after.WrapNonce);
+ Assert.Equal(new byte[] { 96, 97, 98 }, after.WrapTag);
+ Assert.Equal("kek-2", after.KekId);
+ // ...but revision, updated timestamp, the sealed body, and audit stamps are untouched.
+ Assert.Equal(before.Revision, after.Revision);
+ Assert.Equal(before.UpdatedUtc, after.UpdatedUtc);
+ Assert.Equal(before.CreatedUtc, after.CreatedUtc);
+ Assert.Equal(before.UpdatedBy, after.UpdatedBy);
+ Assert.Equal(new byte[] { 1, 2, 3 }, after.Ciphertext);
+ Assert.Equal(before.Nonce, after.Nonce);
+ Assert.Equal(before.Tag, after.Tag);
+ }
+
+ [Fact]
+ public async Task ApplyRewrap_RewrapsTombstonedRows()
+ {
+ await _store.UpsertAsync(MakeSecret("app/dead"), CancellationToken.None);
+ await _store.DeleteAsync(new SecretName("app/dead"), "carol", CancellationToken.None);
+
+ StoredSecret tomb = (await _store.GetAsync(new SecretName("app/dead"), CancellationToken.None))!;
+ Assert.True(tomb.IsDeleted);
+
+ bool applied = await _store.ApplyRewrapAsync(
+ tomb with { WrappedDek = [1], WrapNonce = [2], WrapTag = [3], KekId = "kek-2" },
+ tomb.WrappedDek,
+ CancellationToken.None);
+
+ Assert.True(applied);
+ StoredSecret after = (await _store.GetAsync(new SecretName("app/dead"), CancellationToken.None))!;
+ Assert.Equal("kek-2", after.KekId);
+ // The tombstone survives the rewrap.
+ Assert.True(after.IsDeleted);
+ Assert.Equal(tomb.Revision, after.Revision);
+ }
+
+ [Fact]
+ public async Task ApplyRewrap_ReturnsFalse_WhenRowAbsent()
+ {
+ StoredSecret ghost = MakeSecret("never-existed") with { KekId = "kek-2" };
+ bool applied = await _store.ApplyRewrapAsync(ghost, ghost.WrappedDek, CancellationToken.None);
+ Assert.False(applied);
+ }
+
+ [Fact]
+ public async Task ApplyRewrap_ReturnsFalse_WhenCurrentWrapChanged_UnderConcurrentWrite()
+ {
+ StoredSecret original = MakeSecret("app/cas", ciphertext: [1, 2, 3]);
+ await _store.UpsertAsync(original, CancellationToken.None);
+ StoredSecret before = (await _store.GetAsync(new SecretName("app/cas"), CancellationToken.None))!;
+
+ // Simulate a concurrent set/rotate: a fresh write changes the wrapped DEK out from under us.
+ await _store.UpsertAsync(
+ MakeSecret("app/cas", ciphertext: [9, 9, 9]) with { WrappedDek = [77, 78, 79] },
+ CancellationToken.None);
+
+ // A rewrap that still expects the ORIGINAL wrap must NOT apply (0 rows) — it would otherwise
+ // pair a stale re-wrap with the newer body and corrupt the row.
+ bool applied = await _store.ApplyRewrapAsync(
+ before with { WrappedDek = [1], WrapNonce = [2], WrapTag = [3], KekId = "kek-2" },
+ before.WrappedDek,
+ CancellationToken.None);
+
+ Assert.False(applied);
+ StoredSecret after = (await _store.GetAsync(new SecretName("app/cas"), CancellationToken.None))!;
+ // The concurrent write's wrap survived untouched; no stale re-wrap was applied.
+ Assert.Equal(new byte[] { 77, 78, 79 }, after.WrappedDek);
+ Assert.Equal("kek-1", after.KekId);
+ }
+
public void Dispose()
{
// Drop pooled connections so the WAL/-shm/-wal sidecars release before we delete.
diff --git a/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Ui.Tests/Fakes/FakeCipher.cs b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Ui.Tests/Fakes/FakeCipher.cs
index fb13a8d..32cb712 100644
--- a/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Ui.Tests/Fakes/FakeCipher.cs
+++ b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Ui.Tests/Fakes/FakeCipher.cs
@@ -42,4 +42,8 @@ public sealed class FakeCipher : ISecretCipher
return Encoding.UTF8.GetString(secret.Ciphertext);
}
+
+ ///
+ public StoredSecret Rewrap(StoredSecret secret, IMasterKeyProvider oldKek, IMasterKeyProvider newKek)
+ => secret with { KekId = newKek.KekId };
}
diff --git a/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Ui.Tests/Fakes/RecordingSecretStore.cs b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Ui.Tests/Fakes/RecordingSecretStore.cs
index 3f274b1..5af397b 100644
--- a/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Ui.Tests/Fakes/RecordingSecretStore.cs
+++ b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Ui.Tests/Fakes/RecordingSecretStore.cs
@@ -88,4 +88,9 @@ public sealed class RecordingSecretStore : ISecretStore
///
public Task ApplyReplicatedAsync(StoredSecret row, CancellationToken ct)
=> throw new NotSupportedException();
+
+ ///
+ public Task ApplyRewrapAsync(
+ StoredSecret rewrappedRow, byte[] expectedCurrentWrappedDek, CancellationToken ct)
+ => throw new NotSupportedException();
}
diff --git a/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Ui.Tests/SecretsListTests.cs b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Ui.Tests/SecretsListTests.cs
index 673123e..fbe8513 100644
--- a/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Ui.Tests/SecretsListTests.cs
+++ b/ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Ui.Tests/SecretsListTests.cs
@@ -124,5 +124,9 @@ public class SecretsListTests : TestContext
public Task ApplyReplicatedAsync(StoredSecret row, CancellationToken ct)
=> throw new NotSupportedException();
+
+ public Task ApplyRewrapAsync(
+ StoredSecret rewrappedRow, byte[] expectedCurrentWrappedDek, CancellationToken ct)
+ => throw new NotSupportedException();
}
}
diff --git a/components/secrets/GAPS.md b/components/secrets/GAPS.md
index 1f3b9c9..9068a55 100644
--- a/components/secrets/GAPS.md
+++ b/components/secrets/GAPS.md
@@ -39,6 +39,14 @@ construction).
parallelization for Host.Tests).
- **OtOpcUa — G-2 + G-4 + G-5 + G-6 DONE (the LAST app — all three now adopted).** Executed 2026-07-16 via the plan below (10 tasks, 2 slices, subagent-driven, classification-driven reviews), **FF-merged + pushed to `origin/master` (lmxopcua @ `872cf7e3`, `ec6598ce..872cf7e3`)**. Clean build (0 errors). The full offline suite caught **one real regression** — `AddOtOpcUaDriverFactories`' registration now resolves `ISecretResolver` (Galaxy/OpcUaClient secret: refs), so two `ResilienceInvokerFactoryRegistrationTests` that built a minimal container threw `No service for ISecretResolver`; fixed test-only by registering a stub resolver (production always has it via the unconditional `AddZbSecrets`, and `GetRequiredService` correctly fails-fast for a genuinely misconfigured host). Touched-area suites green (Galaxy 338, OpcUaClient 142, AdminUI 662); remaining non-passes are unrelated (flaky Akka/Roslyn timing tests pass in isolation; the environmental `AbCip_Green_AgainstSim` sim-fixture test — zero AbCip code touched). **Three deliberate deviations from the plan's literal text** (verified corrections, same class as the CPM miss the plan itself caught): (1) **G-4 does NOT commit a `${secret:}` token into the default appsettings** — the expander is fail-closed AND section-agnostic and the only committed secret-shaped value is `ServerHistorian:ApiKey=""` (empty, disabled-by-default), so a committed token would break every dev/CI/`TwoNodeClusterHarness` boot for zero at-rest benefit; instead documented the token-delivery convention (`docs/security.md`) + updated the ServerHistorian comments, and proved fail-closed live (unseeded → `SecretNotFoundException` at pre-host expansion; seeded → resolves). (2) **G-5 shipped docs-only** (`docs/operations/2026-07-16-secrets-clustered-master-key.md`) — hardcoding `Source=File`+shared paths into the committed role-overlay appsettings (also consumed by dev + integration) would break those boots; base stays `Source=Environment`. Mirrors the ScadaBridge G-5 resolution. (3) **Task 8 resolves in the factory/driver only, NOT the probe** — the v3 `OpcUaClientDriverProbe` is unauthenticated `GetEndpoints`-only and never reads `Password`/`UserCertificatePassword`, so probe resolution would be dead code. Layer-B resolution mirrors Galaxy: the sync driver-registry factory means resolution happens lazily at the driver's async session-open (`GalaxyDriver.BuildClientOptionsAsync` / `OpcUaClientDriver.InitializeAsync`), threaded via ctor + `DriverFactoryBootstrap` (real resolver from DI; `NullSecretResolver` null-object backs test/parse paths only, fail-closed on `secret:`). `OpcUaClientDriverOptions` was converted `sealed class` → `sealed record` for the `with`-expression (no positional params → identical JSON; verified no reference-equality/dict-key/ToString-log usages → no leak/behavior change). AdminUI G-2c is structurally leak-proof: `DriverConfig` persists as an opaque JSON string via `RawTreeService`, which has NO resolver dependency. **Live wonder gate PROVEN 2026-07-16 (VPN up):** a throwaway harness drove OtOpcUa's real `GalaxyDriverBrowser` (resolves the `secret:` `ApiKeySecretRef` via `ISecretResolver` → `GalaxySecretRef.ResolveApiKeyAsync` → connects) against the REAL production MxGateway `wonder-app-vd03:5120`. **Dummy `secret:` ref → `MxGatewayAuthenticationException` (proves the resolved value is transmitted, not fail-closed-first); real `metadata:read` key → CONNECTED + browsed the real Galaxy root (10 nodes: ControlSystem/EUR/KPI_CNC/NAM/_TestArea/MESReceiver…).** Temp key minted via the gateway dashboard, then **revoked (functionally proven — the same key that browsed 10 nodes then returned Unauthenticated) + deleted** (row gone; no prod key touched; the dashboard key-list is stale-cached so revoke/delete were verified FUNCTIONALLY, not by the list UI). Harness + scratch stores destroyed. **All of G-2/G-4/G-5/G-6 now DONE + live-proven across all three apps + mxaccessgw.**
**Plan re-verified 2026-07-16 against `origin/master` @ `ec6598ce`** (the merged **v3.0 dual-namespace** rewrite, PR #472). The v3.0 rewrite did NOT touch the driver-secret flow, so the approach held; the plan was corrected for: OtOpcUa **is** on Central Package Management (was mis-stated as inline-versioned), `OpcUaClientDriverOptions` lives in `…Driver.OpcUaClient.Contracts` + `GalaxyDriverBrowser` in `…Driver.Galaxy.Browser`, and `DriverHostActor.cs` line shifts (1704/1756/1790 → 1766/1818/1852). Program.cs / Security / AdminUI anchors survived essentially unchanged.
+- **G-8 (KEK rotation) — BUILT; G-7 (clustered replication) — DESIGNED + PLANNED (2026-07-17).**
+ G-8 shipped into `ZB.MOM.WW.Secrets` (lib `0.1.3`): the `Rewrap` DEK primitive, CAS-guarded
+ `ApplyRewrapAsync`, `KekRotationService.RewrapAllAsync`, the `secret rewrap-all` CLI verb, and an
+ operator runbook — full suite green (82+15, 0 regressions) + end-to-end CLI smoke, adversarial
+ crypto review PASS with a review-caught TOCTOU closed via compare-and-swap. G-7's SQL-store-vs-Akka
+ fork is resolved (build the shared SQL-Server `ISecretStore`; Akka replicator deferred phase-2)
+ with a design doc + executable plan + `.tasks.json`, **no code built**. See the G-7/G-8 backlog
+ entries below. Not yet committed/pushed (awaiting go-ahead).
## Design + implementation plans (2026-07-16)
@@ -55,7 +63,7 @@ code-verified anchors, co-located `.tasks.json`):
- [`docs/plans/2026-07-16-secrets-adoption-mxaccessgw.md`](../../docs/plans/2026-07-16-secrets-adoption-mxaccessgw.md)
— **G-4 + G-5 + G-6** (8 tasks; single-box, no Layer B).
-G-7/G-8 remain design-only/deferred (below).
+**G-8 (KEK rotation) is now BUILT; G-7 (clustered replication) is now DESIGNED + PLANNED** (2026-07-17, below).
## Cross-cutting observations
@@ -109,14 +117,35 @@ mounted key)** for clustered pairs (hard requirement — same KEK on every node)
Add the RCL page to each dashboard (OtOpcUa AdminUI, MxGateway Server, ScadaBridge CentralUI)
and map each app's admin role onto `secrets:manage` / `secrets:reveal`.
-### G-7 — Clustered replication (ScadaBridge, OtOpcUa) — build `ZB.MOM.WW.Secrets.Akka`
-Deferred until G-2…G-6 land for a clustered app. Options per SPEC: shared SQL-Server
-`ISecretStore` (simplest — mirrors the shared Data-Protection key ring) **or** the Akka
-replicator (LWW, anti-entropy resync, tombstones). Requires a shared KEK.
+### G-7 — Clustered replication (ScadaBridge, OtOpcUa) — DESIGNED + PLANNED (2026-07-17)
+The SPEC's "shared SQL-Server store **vs** Akka replicator" fork is **resolved: build Option A —
+a shared SQL-Server `ISecretStore`** (both clustered apps already run shared SQL + a shared
+Data-Protection key ring, so it delivers cluster-wide secrets with zero distributed-systems failure
+modes); the Akka replicator (`ZB.MOM.WW.Secrets.Akka`, LWW/anti-entropy/tombstones) is a **deferred
+phase-2** to build only on a stated partition-tolerance requirement. Both require a shared KEK; the
+`ISecretReplicator` seam stays in place so Option B remains a drop-in later. Design + executable
+plan (+ `.tasks.json`): [`docs/plans/2026-07-17-secrets-g7-clustered-replication-design.md`](../../docs/plans/2026-07-17-secrets-g7-clustered-replication-design.md)
++ [`docs/plans/2026-07-17-secrets-g7-sqlserver-store.md`](../../docs/plans/2026-07-17-secrets-g7-sqlserver-store.md). **No code built yet — plan is ready to execute.**
-### G-8 — KEK-rotation runbook + `RewrapAll`
-The `RewrapAll(oldKek, newKek)` admin primitive + an operator runbook. Deferred (design-only
-in this cut).
+### G-8 — KEK-rotation `RewrapAll` + runbook ✅ BUILT (2026-07-17, lib `0.1.3`)
+The `RewrapAll(oldKek, newKek)` admin primitive + operator runbook are **built + fully tested** in
+`ZB.MOM.WW.Secrets`. Because the envelope wraps a per-secret DEK under the KEK, rotation **re-wraps
+DEKs only** — bodies are never re-encrypted, revision/timestamps are preserved (invisible to cluster
+LWW). Shipped: `ISecretCipher.Rewrap(row, oldKek, newKek)` (fail-closed on wrong old KEK; DEK zeroed
+on all paths; also catches malformed-length wraps → `SecretDecryptionException`),
+`ISecretStore.ApplyRewrapAsync(rewrappedRow, expectedCurrentWrappedDek)` (updates only the 4 wrap
+columns + `kek_id`, **compare-and-swap on the current wrapped DEK** so a concurrent write can't
+corrupt a row — review-caught TOCTOU), `KekRotationService.RewrapAllAsync` + `RewrapReport`
+(enumerates all rows incl. tombstones, idempotent/resumable skip-already-current, bounded CAS-retry,
+fail-closed on unknown/identical KEK), the `secret rewrap-all` CLI verb (key material only via
+env-var-name/file-path — never a literal), README section, and the operator runbook
+[`ZB.MOM.WW.Secrets/docs/operations/kek-rotation.md`](../../ZB.MOM.WW.Secrets/docs/operations/kek-rotation.md).
+**Verified:** full offline suite green (82 core + 15 UI, 0 regressions) + an end-to-end CLI smoke
+(migrate → decrypts under new KEK, fail-closed under old → idempotent re-run → identical-KEK guard).
+Version bumped `0.1.2`→`0.1.3` (additive interface members); **republish to the Gitea feed is a
+separate opt-in step (the apps don't need rewrap — it's a library/CLI admin primitive).** Reviewed
+by an adversarial crypto/correctness pass (all 7 categories PASS); the one real finding (concurrent-
+write TOCTOU) is fixed by the CAS above, plus two low/nit fixes folded in.
## Out of scope (this component)
diff --git a/docs/plans/2026-07-17-secrets-g7-clustered-replication-design.md b/docs/plans/2026-07-17-secrets-g7-clustered-replication-design.md
new file mode 100644
index 0000000..42a31ba
--- /dev/null
+++ b/docs/plans/2026-07-17-secrets-g7-clustered-replication-design.md
@@ -0,0 +1,102 @@
+# G-7 — Clustered secret replication: design & fork resolution
+
+> **Status:** design-only (G-7 in [`components/secrets/GAPS.md`](../../components/secrets/GAPS.md)).
+> Companion executable plan: [`2026-07-17-secrets-g7-sqlserver-store.md`](2026-07-17-secrets-g7-sqlserver-store.md).
+> Prerequisite G-2…G-6 have landed for both clustered apps (ScadaBridge, OtOpcUa); G-8 (KEK
+> rotation) is built. This resolves the SPEC's "shared SQL store **vs** Akka replicator" fork and
+> specifies the recommended build.
+
+## Problem
+
+`ZB.MOM.WW.Secrets` stores secrets in a **local SQLite** file by default. The two Akka-clustered
+apps run **multiple nodes** (ScadaBridge: central pair + N site pairs; OtOpcUa: admin/driver/dev
+roles). A secret written on one node must be resolvable on every node that needs it, **without**
+exposing plaintext or the KEK over the wire, and **without** each node holding a divergent copy.
+
+The library was built anticipating this: the schema already carries `revision` / `updated_utc` /
+`is_deleted`; `ISecretStore` exposes `GetManifestAsync` + `ApplyReplicatedAsync` (last-writer-wins);
+`ISecretReplicator` is a no-op seam; `SecretManifestEntry` supports anti-entropy. Nothing needs a
+migration to turn replication on.
+
+## The fork (from SPEC §"Clustered pairs")
+
+Two ways to make one logical secret set visible cluster-wide. **Both require a shared KEK** — every
+node must resolve the *same* master key (shared mounted key file or shared env), because a row whose
+`kek_id` doesn't match the local provider fails closed on resolve. Ciphertext can cross the wire /
+sit in a shared DB safely because the KEK never does.
+
+### Option A — Shared SQL-Server `ISecretStore` (one source of truth)
+
+Point every node's `ISecretStore` at **one shared SQL-Server database** instead of a per-node
+SQLite file. There is exactly one copy of each row; no replication, no reconciliation. The store
+still holds **ciphertext only** (the KEK stays per-node/out-of-DB). Build a
+`SqlServerSecretStore : ISecretStore` behind the existing seam (the SPEC already scoped "SQL-Server
+`ISecretStore` provider construction — build with G-7").
+
+- **Pro:** zero distributed-systems code or failure modes (no split-brain, no anti-entropy lag, no
+ tombstone GC, no LWW clock-skew). ~1 new file mirroring `SqliteSecretStore`'s SQL + a migrator.
+- **Pro:** both apps **already run shared SQL Server** (ScadaBridge ConfigDb + central `dbo.*`;
+ OtOpcUa central config DB) and already share their Data-Protection key ring through SQL — a shared
+ secret store is operationally identical to what ops already run and back up.
+- **Pro:** rotation (G-8 `rewrap-all`) runs **once** against the one store.
+- **Con:** availability is coupled to the shared DB. A brief outage is bridged by the resolver's
+ in-memory TTL cache; but a node that must keep resolving secrets while *partitioned from* the
+ shared DB is not served by this option (see Option B).
+
+### Option B — Akka replicator `ZB.MOM.WW.Secrets.Akka` (each node keeps a local store)
+
+Each node keeps its **local SQLite** store; a real `ISecretReplicator` broadcasts each newly-written
+encrypted row to peers, and a cluster anti-entropy actor periodically exchanges manifests
+(`GetManifestAsync`) and pulls missing/newer rows (`ApplyReplicatedAsync`, LWW). Tombstones
+propagate deletes.
+
+- **Pro:** each node resolves from local state, so a node **survives partition** from its peers /
+ from any central DB — the right answer for air-gapped or intermittently-connected sites.
+- **Con:** a whole new package + actor lifecycle (a cluster-singleton reconciler or per-node gossip),
+ message serialization for `StoredSecret`, LWW edge cases (clock skew, tombstone retention), and —
+ critically — it can only be *proven* against a live multi-node cluster, a G-2-class live-validation
+ effort per app.
+
+## Decision
+
+**Build Option A (shared SQL-Server `ISecretStore`) as G-7.** Specify Option B as a **deferred
+phase-2**, to be built only when an app declares a concrete requirement to resolve secrets while
+partitioned from the shared store (an availability SLA neither app states today).
+
+Rationale: Option A delivers cluster-wide secrets for both apps with the least new code and **no new
+failure modes**, reusing the shared-SQL operational posture both apps already depend on for config,
+audit, and the DP key ring. The clustered-secrets requirement today is "every node sees the same
+secrets," which a shared store satisfies exactly. Partition-tolerance (Option B's only unique
+benefit) is a stronger, unstated requirement; paying its distributed-systems complexity now would be
+speculative (YAGNI). The `ISecretReplicator` seam stays in place, so Option B remains a drop-in later
+with **no change** to consumers or the wire/store contract.
+
+## Option A — build shape (detail in the executable plan)
+
+1. **`SqlServerSecretStore : ISecretStore`** — mirrors `SqliteSecretStore` in T-SQL: `GetAsync`,
+ `UpsertAsync` (revision bump via `MERGE`/`UPDATE`), `DeleteAsync` (tombstone), `ListAsync`
+ (metadata projection — never ciphertext), `GetManifestAsync`, `ApplyReplicatedAsync` (LWW under a
+ serializable transaction), and **`ApplyRewrapAsync`** (G-8; UPDATE the 4 wrap columns only).
+ `Microsoft.Data.SqlClient`, fully parameterized.
+2. **`SqlServerSecretsStoreMigrator`** — schema-versioned, idempotent (`IF NOT EXISTS`), same column
+ set/semantics as SQLite; `varbinary(max)` for the crypto BLOBs, `datetimeoffset` (or ISO-8601
+ text, to match the SQLite round-trip exactly) for timestamps.
+3. **DI selection** — add `SecretsOptions.Store` (`Sqlite` default | `SqlServer`) +
+ `SqlServerConnectionString`; `AddZbSecrets` binds the store + migrator from it. SQLite stays the
+ default so single-process consumers (HistorianGateway, mxaccessgw) are unaffected.
+4. **Adoption (ScadaBridge, OtOpcUa)** — set `Secrets:Store = SqlServer` + a connection string
+ (delivered via `${secret:}` / a `secret:` ref like every other connstr), and ensure **the same
+ KEK** on every node (shared key file or shared env). Run G-8 `rewrap-all` once against the shared
+ store when the KEK rotates.
+
+## Hard constraints (both options)
+
+- **Same KEK on every node.** Non-negotiable — a mismatched `kek_id` fails closed on resolve.
+- **Ciphertext only crosses trust boundaries.** The store/wire never carries plaintext or the KEK.
+- **Rotation is per independent store.** G-8 `rewrap-all`: once for the shared SQL store (Option A);
+ once per node for per-node SQLite (Option B).
+
+## Out of scope
+
+- Akka remoting auth/TLS hardening (a separate concern from secret storage).
+- Option B's actual construction — captured as the deferred phase-2 in the executable plan.
diff --git a/docs/plans/2026-07-17-secrets-g7-sqlserver-store.md b/docs/plans/2026-07-17-secrets-g7-sqlserver-store.md
new file mode 100644
index 0000000..7c8cebc
--- /dev/null
+++ b/docs/plans/2026-07-17-secrets-g7-sqlserver-store.md
@@ -0,0 +1,122 @@
+# G-7 (Option A) — Shared SQL-Server `ISecretStore` Implementation Plan
+
+> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans (or
+> subagent-driven-development) to implement this plan task-by-task.
+
+**Goal:** Add a shared SQL-Server `ISecretStore` provider to `ZB.MOM.WW.Secrets` so the Akka-clustered
+apps (ScadaBridge, OtOpcUa) can back their secrets with one shared, ciphertext-only database — every
+node sees the same secrets, no replication code.
+
+**Architecture:** Mirror the existing `SqliteSecretStore` + `SqliteSecretsStoreMigrator` in T-SQL
+behind the unchanged `ISecretStore` seam. Select the store in `AddZbSecrets` from a new
+`SecretsOptions.Store` enum (SQLite stays the default). The KEK stays per-node/out-of-DB, so the DB
+holds only ciphertext. See the design + fork rationale in
+[`2026-07-17-secrets-g7-clustered-replication-design.md`](2026-07-17-secrets-g7-clustered-replication-design.md).
+
+**Tech Stack:** .NET 10, `Microsoft.Data.SqlClient`, xUnit + Shouldly, env-gated live SQL tests
+(LocalDB or a `mcr.microsoft.com/mssql/server` container), matching the app live-test idiom.
+
+**Precondition:** G-8 is merged — `ISecretStore.ApplyRewrapAsync` exists and the SQL-Server store
+must implement it. `rewrap-all` then runs once against the shared store.
+
+---
+
+### Task 1: SQL-Server schema + migrator
+
+**Classification:** standard
+**Estimated implement time:** ~5 min
+**Parallelizable with:** none (Task 2 depends on it)
+
+**Files:**
+- Create: `src/ZB.MOM.WW.Secrets/SqlServer/SqlServerSecretsSchema.cs`
+- Create: `src/ZB.MOM.WW.Secrets/SqlServer/SqlServerSecretsStoreMigrator.cs`
+- Create: `src/ZB.MOM.WW.Secrets/SqlServer/SecretsSqlServerConnectionFactory.cs`
+- Test: `tests/ZB.MOM.WW.Secrets.Tests/SqlServer/SqlServerSecretsStoreMigratorTests.cs` (env-gated)
+
+**Notes:**
+- Mirror `SqliteSecretsSchema` / `SqliteSecretsStoreMigrator` semantics: `schema_version` single-row
+ table + `secret` table, `CurrentVersion = 1`, idempotent `IF NOT EXISTS` DDL, refuse a newer
+ on-disk version. Column set identical to SQLite; use `varbinary(max)` for the 6 crypto BLOBs,
+ `nvarchar` for text, and store timestamps as **ISO-8601 (`"O"`) text** so `StoredSecret`
+ round-trips byte-identically to the SQLite path (avoids `datetimeoffset` precision drift in tests).
+- Add `Microsoft.Data.SqlClient` to `Directory.Packages.props` + the core csproj.
+- Gate live tests on an env var (e.g. `SECRETS_SQLSERVER_CONNSTR`); skip cleanly when unset.
+
+### Task 2: `SqlServerSecretStore : ISecretStore`
+
+**Classification:** high-risk
+**Estimated implement time:** ~5 min
+**Parallelizable with:** none (depends on Task 1)
+
+**Files:**
+- Create: `src/ZB.MOM.WW.Secrets/SqlServer/SqlServerSecretStore.cs`
+- Test: `tests/ZB.MOM.WW.Secrets.Tests/SqlServer/SqlServerSecretStoreTests.cs` (env-gated; port the
+ `SqliteSecretStoreTests` cases 1:1, incl. the G-8 `ApplyRewrap*` cases)
+
+**Notes:**
+- Implement all 7 members: `GetAsync`, `UpsertAsync` (revision bump — `MERGE` or
+ insert/`ON CONFLICT`-equivalent via `UPDATE`+`INSERT` under a transaction), `DeleteAsync`
+ (tombstone + revision bump), `ListAsync` (metadata projection — **never** the crypto columns),
+ `GetManifestAsync`, `ApplyReplicatedAsync` (LWW under `IsolationLevel.Serializable`, applied
+ verbatim, no revision bump), `ApplyRewrapAsync` (UPDATE the 4 wrap columns + `kek_id` only; no
+ revision/updated_utc change).
+- Fully parameterized commands; same "created_utc/created_by preserved on overwrite" semantics as
+ SQLite. This is high-risk because the LWW + rewrap semantics MUST match SQLite exactly (a cluster
+ reconciles across both if a node ever migrates).
+
+### Task 3: DI store-selection wiring
+
+**Classification:** standard
+**Estimated implement time:** ~4 min
+**Parallelizable with:** none (depends on Task 2)
+
+**Files:**
+- Modify: `src/ZB.MOM.WW.Secrets/SecretsOptions.cs` (add `SecretsStoreKind Store` = `Sqlite` default;
+ add `string? SqlServerConnectionString`)
+- Create: `src/ZB.MOM.WW.Secrets/SecretsStoreKind.cs` (enum `Sqlite | SqlServer`)
+- Modify: `src/ZB.MOM.WW.Secrets/DependencyInjection/SecretsServiceCollectionExtensions.cs`
+ (branch the `ISecretStore` + migrator registration on `Store`; keep SQLite the default so
+ HistorianGateway/mxaccessgw are unaffected)
+- Modify: `src/ZB.MOM.WW.Secrets/DependencyInjection/SecretsMigrationHostedService.cs` if it hard-refs
+ the SQLite migrator type (abstract behind a shared migrator seam or a switch)
+- Test: `tests/ZB.MOM.WW.Secrets.Tests/DependencyInjection/AddZbSecretsTests.cs` (assert the correct
+ store type is resolved for each `Store` value)
+
+**Notes:**
+- Introduce a tiny `ISecretsStoreMigrator` seam (both migrators implement it) so the hosted service +
+ CLI stay store-agnostic, OR switch on `Store` at registration. Prefer the seam — cleaner.
+- The connection string should itself be deliverable via `${secret:}` / a `secret:` ref (consumers
+ already do this for other connstrings).
+
+### Task 4: Docs + adoption notes
+
+**Classification:** small
+**Estimated implement time:** ~3 min
+**Parallelizable with:** Task 3
+
+**Files:**
+- Modify: `README.md` (Clustered deployments: document `Secrets:Store = SqlServer` + shared-KEK
+ requirement + "run `rewrap-all` once against the shared store")
+- Create: `docs/operations/shared-sql-store.md` (operator setup: provision the DB, grant, set the
+ connstr via a secret ref, confirm the same KEK on every node)
+- Modify: `components/secrets/GAPS.md` (G-7 → "Option A built"; note Option B deferred)
+
+**Notes:** No app changes here — adoption in ScadaBridge/OtOpcUa is a follow-on (flip `Secrets:Store`,
+supply the connstr + shared KEK). Capture that as a checklist, not code.
+
+---
+
+## Deferred phase-2 (NOT built here): Option B — `ZB.MOM.WW.Secrets.Akka`
+
+Build only when an app declares it must resolve secrets while **partitioned** from the shared store.
+Shape (for when it's warranted):
+- New package `ZB.MOM.WW.Secrets.Akka` (net10; refs Akka.Cluster + Abstractions).
+- `AkkaSecretReplicator : ISecretReplicator` — `PublishAsync` broadcasts the encrypted `StoredSecret`
+ to peers (a cluster-aware router / distributed pub-sub topic).
+- An anti-entropy actor (cluster singleton or per-node) that periodically exchanges
+ `GetManifestAsync` digests and pulls newer/missing rows via `ApplyReplicatedAsync` (LWW);
+ tombstones propagate deletes; a retention window bounds tombstone GC.
+- `StoredSecret` serialization (ciphertext only — never the KEK) + an `ISecretActorAccessor` wiring.
+- **Validation:** a live 2-node ScadaBridge/OtOpcUa rig proving write-on-A → resolve-on-B, delete
+ propagation, and partition-heal resync — a G-2-class live gate.
+- KEK constraint identical to Option A: same master KEK on every node; `rewrap-all` per node.
diff --git a/docs/plans/2026-07-17-secrets-g7-sqlserver-store.md.tasks.json b/docs/plans/2026-07-17-secrets-g7-sqlserver-store.md.tasks.json
new file mode 100644
index 0000000..e77a427
--- /dev/null
+++ b/docs/plans/2026-07-17-secrets-g7-sqlserver-store.md.tasks.json
@@ -0,0 +1,11 @@
+{
+ "planPath": "docs/plans/2026-07-17-secrets-g7-sqlserver-store.md",
+ "tasks": [
+ {"id": 0, "subject": "Task 1: SQL-Server schema + migrator", "status": "pending"},
+ {"id": 1, "subject": "Task 2: SqlServerSecretStore : ISecretStore", "status": "pending", "blockedBy": [0]},
+ {"id": 2, "subject": "Task 3: DI store-selection wiring", "status": "pending", "blockedBy": [1]},
+ {"id": 3, "subject": "Task 4: Docs + adoption notes", "status": "pending", "blockedBy": [1]}
+ ],
+ "deferredPhase2": "Option B — ZB.MOM.WW.Secrets.Akka replicator (build only on a stated partition-tolerance requirement); see plan doc.",
+ "lastUpdated": "2026-07-17"
+}