diff --git a/ZB.MOM.WW.Secrets/Directory.Build.props b/ZB.MOM.WW.Secrets/Directory.Build.props index c915ed0..c610048 100644 --- a/ZB.MOM.WW.Secrets/Directory.Build.props +++ b/ZB.MOM.WW.Secrets/Directory.Build.props @@ -5,7 +5,7 @@ enable enable latest - 0.2.3 + 0.3.0 true README.md diff --git a/ZB.MOM.WW.Secrets/README.md b/ZB.MOM.WW.Secrets/README.md index 1e74986..ba3431c 100644 --- a/ZB.MOM.WW.Secrets/README.md +++ b/ZB.MOM.WW.Secrets/README.md @@ -16,6 +16,45 @@ plaintext back on demand — from application code, from configuration, or from A `secret` CLI (`set` / `get` / `list` / `rm` / `rotate` / `rewrap-all`) ships in the repo (not packed). +### Interactive console + +Run `secret` with **no arguments** in a real terminal and it opens an interactive console +instead of printing usage — a menu-driven session for an operator sitting at a deployment, +rather than a scripted one-shot verb. The launch condition is exact: no verb **and** neither +stdin nor stdout is redirected. Any redirection (a pipe, `| cat`, CI, a script) falls straight +through to the headless usage/exit-2 behavior above, so nothing scripted ever hangs on a +prompt. + +The first screen picks a **store target** — recent targets (persisted to +`~/.zb-secrets/recent-targets.json`, names + paths only, never key material), an app's +`appsettings.json` (the console composes base + `appsettings..json` + environment +variables and binds `Secrets` exactly like the app itself, so every operation hits precisely +the store the app reads), or manual entry (a raw SQLite path + a master-key environment +variable name). If the configured KEK cannot be resolved in the operator's shell, the session +still opens — **degraded**, metadata-only — and any flow that needs the key prompts for one +(masked paste or a key file) to upgrade it in place. + +Once a target is open, the menu offers: + +- **List secrets** — metadata table (name, content type, KEK id, revision, updated); no KEK needed. +- **Set / rotate a secret** — masked value prompt, seals a fresh row under the session KEK. +- **Get (reveal) a secret** — confirm-gated; prints the plaintext once, never cached or logged. +- **Delete a secret** — confirm-gated tombstone; no KEK needed. +- **Reference audit & seed** — scans the target's composed configuration for `${secret:NAME}` + tokens, classifies each Ok / Missing / Tombstoned / Undecryptable / InvalidName, and walks the + gaps through a seed prompt. This is *the* deployment-setup flow. +- **KEK doctor (lockout recovery)** — per-row diagnosis (Ok / WrongKek / Corrupt, with the + foreign KEK id) plus a guided remedy: rewrap every row from an old KEK, or re-set individual + rows when the old KEK is unrecoverable. +- **Export bundle (ciphertext-only)** / **Import bundle** — move rows between stores. Bundles + carry name + metadata + wrapped DEK + ciphertext, never plaintext; importing across KEKs + verifies the pasted source key against the bundle's KEK id before re-wrapping, conflicts + resolve last-writer-wins (with an optional per-row prompt), and the format is versioned. + +There is no login — the console runs locally with direct file/DB access, the same trust model +as any other admin CLI. See the operator runbook for the full walkthrough: +[`docs/operations/interactive-console.md`](docs/operations/interactive-console.md). + ## How it protects secrets Each value is encrypted with a fresh **data key (DEK)** using AES-256-GCM; the DEK is then @@ -149,4 +188,6 @@ node can always reach a shared database, because you pay real distributed-system stays invisible to last-writer-wins. Run `rewrap-all` against **each** independent store. See [`docs/operations/clustered-secrets.md`](docs/operations/clustered-secrets.md) for operator -setup, and [`docs/operations/kek-rotation.md`](docs/operations/kek-rotation.md) for rotation. +setup, [`docs/operations/kek-rotation.md`](docs/operations/kek-rotation.md) for rotation, and +[`docs/operations/interactive-console.md`](docs/operations/interactive-console.md) for the +`secret` interactive console (deployment seeding, lockout recovery, bundles). diff --git a/ZB.MOM.WW.Secrets/docs/operations/interactive-console.md b/ZB.MOM.WW.Secrets/docs/operations/interactive-console.md new file mode 100644 index 0000000..40e28c9 --- /dev/null +++ b/ZB.MOM.WW.Secrets/docs/operations/interactive-console.md @@ -0,0 +1,256 @@ +# Operator runbook — the `secret` interactive console (`ZB.MOM.WW.Secrets`) + +**Audience:** operators running the `secret` CLI at a terminal against a deployment's store. +**Primitive:** a bare `secret` invocation (no verb) on a real TTY → `InteractiveEntry` → +`InteractiveShell` → pluggable `IInteractiveFlow`s over an opened `SecretsSession`. +**Status:** ships alongside the headless verbs (`set` / `get` / `list` / `rm` / `rotate` / +`rewrap-all`), which are unaffected. Companion to +[`kek-rotation.md`](kek-rotation.md) (the `rewrap-all` primitive the KEK-doctor remedy uses +under the hood) and [`clustered-secrets.md`](clustered-secrets.md) (replication topologies the +console does not manage). + +--- + +## What the console is (and is not) + +Every flow in the console is a thin UI wrapper over the same store/cipher primitives the +headless CLI and the app itself use — `ISecretStore`, `ISecretCipher`, `KekRotationService`, +`SecretReferenceExpander`'s token pattern. Nothing about the store, the wrap scheme, or the +crypto changes. What the console adds is: + +- **Target discovery** — pick a deployment's store the same way its app would, instead of + hand-assembling a connection string. +- **A guided menu** in place of memorized verb syntax, with confirm-gates and masked prompts + where a headless verb would take a positional argument. +- **Two flows with no headless equivalent**: the reference audit & seed (deployment setup) and + the KEK doctor (lockout triage + guided remedy). + +It is **not** a login surface, a replication controller, or a replacement for `rewrap-all` — +it drives that primitive for one convenience case (rewrap onto the *session's* KEK) and defers +to the headless verb (and [`kek-rotation.md`](kek-rotation.md)) for a general old→new rotation. + +## Launching + +```bash +secret +``` + +- **Real terminal, no args:** opens the console. The check is exact — + `args.Length == 0 && !Console.IsInputRedirected && !Console.IsOutputRedirected` — so any + redirection at all (a pipe, `secret | cat`, a CI runner, a script capturing output) falls + through to the ordinary headless path instead of hanging on a prompt. +- **Any verb, or a redirected stream:** unchanged headless behavior — `set` / `get` / `list` / + `rm` / `rotate` / `rewrap-all` dispatch exactly as before; a bare `secret` with a redirected + stream (or an unrecognized verb) prints usage and exits **2**, same as pre-console builds. +- **Exit codes:** the console itself returns **0** on a clean quit (menu "Quit", or Ctrl-C at + the target picker or the action menu) and never returns a non-zero code on its own — a + failed action is caught, rendered as an error panel, and returns you to the menu rather than + exiting the process. Ctrl-C mid-flow abandons just that action. +- **Ctrl-C behavior:** cooperative cancellation is checked between actions and at the target + picker; Spectre's blocking prompts cannot observe the token mid-prompt, but the surrounding + loop always resolves to a clean exit rather than a stack trace. + +## Targeting a deployment + +The first screen is always the target picker: + +1. **Recent targets** — up to 10 entries from `~/.zb-secrets/recent-targets.json`, newest + first, labeled ` — `. Selecting one re-reads that path (config may + have changed since last time) and re-stamps it as most-recent. +2. **Enter an appsettings.json path…** — point at any app's base `appsettings.json`. +3. **Manual entry (SQLite db + KEK env var)…** — a raw SQLite file path plus a master-key + environment variable name (default `ZB_SECRETS_MASTER_KEY`), for a store with no + accompanying appsettings file. Manual targets are never added to recents (there is no + appsettings path to remember). + +**The honest-store guarantee.** Option 2 composes configuration *exactly the way the target +app would at startup*: `appsettings.json` as the base, an optional `appsettings..json` +overlay, then environment variables — layered in that order via the same +`ConfigurationBuilder` pattern the app's own `Host.CreateApplicationBuilder` uses — and binds +the `Secrets` section (and, if present, `Secrets:SqlServer`) out of the result. There is no +separate "CLI view" of the config that could drift from what the app actually reads: if the app +resolves its SQLite path or its SQL-Server hub connection string a particular way, the console +resolves it the same way, against the same files, from the same working directory. This is +also why `list`/`get`/`set` here can be trusted as evidence of what the running app sees. + +**Store selection mirrors the app's DI wiring**: a configured `Secrets:SqlServer` connection +string wins over local SQLite — *unless* that connection string is itself an unresolved +`${secret:...}` reference (the app resolves that from its own bootstrapped store at startup; +the console has no such bootstrap step), in which case the console falls back to the local +SQLite store and prints a warning. In hub-replication mode, editing the SQL-Server hub through +the console is the correct target — the change converges to every node's local store on the +next sync sweep, rather than racing one node's SQLite against the hub. + +**Recents contain no secret material** — only a display name and the appsettings path, ever. +See the security notes below. + +## Degraded sessions and the KEK prompt + +Opening a session never fails outright on a bad or missing KEK. If the configured master key +cannot be resolved — unset/empty env var, missing key file, a malformed `MasterKey:Source`, +DPAPI off Windows — the session opens anyway, **degraded**: metadata-only operations (List, +Delete — tombstoning touches no ciphertext) work immediately, and the header banner shows +`KEK UNAVAILABLE (degraded)` in red instead of `KEK OK` in green. + +The moment the operator picks a flow that needs plaintext access (Set/rotate, Get, reference +audit & seed, KEK doctor, bundle export/import — anything whose `RequiresKek` is true), the +shell intercepts the dispatch and prompts: + +- **Paste base64 key** — a masked `TextPrompt` (nothing echoed) fed straight to an in-memory + provider; nothing is written to disk. +- **Key file path** — points at a file the configured `File` provider can read. + +A successful upgrade re-opens the session with that key and continues into the flow the +operator originally chose. A key that fails to resolve, or resolves but still doesn't unlock +the session (wrong key), reports the failure and returns to the menu — the degraded session is +never silently discarded. + +## Deployment seeding walk-through: Reference audit & seed + +This is the flow to run **the first time you point the console at a fresh deployment**, or +whenever an app fails to start because a `${secret:NAME}` reference can't resolve. + +1. The auditor walks every value in the target's *composed* configuration (same honest-store + composition as targeting, above) with the identical `\$\{secret:([^}]+)\}` token pattern + `SecretReferenceExpander` uses at app startup — so what you see here is exactly what the app + would try to resolve, including config keys documented via a `"_comment": "...${secret:x}..."` + convention, which are skipped the same way the expander skips them. +2. Each distinct referenced name is looked up against the store and classified: + - **Ok** — exists and decrypts under the session KEK. Nothing to do. + - **Missing** — no row at all. A fail-closed expansion would throw at app startup. + - **Tombstoned** — a soft-deleted row. Also unresolvable to the app. + - **Undecryptable** — the row exists but does not decrypt under the session KEK (wrong or + rotated key, or damage). *This is often the first sign a KEK doctor pass is needed.* + - **InvalidName** — the token text isn't a legal secret name (illegal characters, an + unfilled placeholder like `REPLACE_ME`). Can't be looked up or seeded — fix the token in + the config file itself. +3. A color-coded table renders every finding with the config key paths that reference it (so + you know exactly which setting(s) are affected), then the flow walks each seedable gap + (Missing / Tombstoned / Undecryptable) in turn, asking to seed it now. Accepting prompts for + a masked value, a content type, and an optional description, then seals a fresh row under + the session KEK — **overwriting** an Undecryptable row and **reviving** (un-tombstoning) a + Tombstoned one. +4. A closing re-audit prints a one-line count-by-status summary so you can confirm the gap + closed without re-reading the whole table. + +Run this again any time after adding a new `${secret:}` reference to a config file, or after +deploying to a fresh environment — it is the fast path from "app won't start, references don't +resolve" to "every reference is Ok." + +## Lockout playbook: KEK doctor + +Run this when an app (or the console itself) reports decrypt failures it didn't before — +typically after a KEK rotation was only partially applied, a config value pointing at the +wrong key source, or a store copied from another environment. + +**Diagnosis** probes *every* row (including tombstones, since they still carry a KEK-wrapped +DEK and block a clean `rewrap-all`) against the session's KEK: + +- **Ok** — the row's `kek_id` matches the session KEK and it decrypts cleanly. +- **WrongKek** — the row's `kek_id` differs from the session KEK. Not decrypted (no point — + the wrap alone tells you it won't work); reported with the row's *own* `kek_id` so you know + which key to go find. The summary line also lists the distinct **foreign KEK ids** seen + across all wrong-KEK rows — usually there is exactly one (the store's actual current key), + which tells you the session simply has the wrong key configured. +- **Corrupt** — the `kek_id` matches (so the wrap envelope claims to be under the right key), + but the decrypt still fails closed. This means the wrap envelope or the sealed body itself is + damaged — a **different problem than WrongKek**, and *not* fixable by rewrapping: rewrapping + only re-wraps a DEK it can first unwrap, and a Corrupt row can't be unwrapped at all. + +If every row reports Ok, the doctor says so and there is nothing to remedy — the lockout is +elsewhere (check the app's actual configured KEK source against what you used to open this +session). + +**Two remedies**, offered only when something is unhealthy: + +- **Rewrap from old KEK** — the non-destructive path, and the *only* correct remedy for + WrongKek rows. You supply the old KEK (env var, key file, or a masked paste) the affected + rows are actually wrapped under; the flow shows exactly how many rows will move and from + which KEK id, then asks for an explicit confirmation (default **No** — nothing happens on + Enter) before calling the same `KekRotationService.RewrapAllAsync` primitive + [`kek-rotation.md`](kek-rotation.md) documents for the headless `rewrap-all` verb. Secret + bodies are never touched — only the DEK wrap moves onto the session KEK. A wrong old key + surfaces as a decryption failure and aborts (rows already rewrapped before the anomaly stay + persisted, so a retry after fixing the key resumes cleanly). +- **Old KEK is lost — re-set affected secrets** — the destructive last resort, and the *only* + remedy for a Corrupt row (which cannot be rewrapped because it can't be unwrapped in the + first place). For every WrongKek or Corrupt row still affected, the flow offers a masked + re-set: type a fresh value, and it's sealed fresh under the session KEK, overwriting the + unrecoverable row. Each row is confirmed individually (default **No**) so you can skip rows + you'd rather chase down the old key for instead. + +A closing re-diagnosis prints a one-line count-by-status summary so you can see the remedy's +effect immediately, in the same session, without re-running the flow. + +## Bundles: export / import + +Bundles move secret rows **between deployment stores** — cloning a deployment, staging a +recovery, seeding a new environment from a known-good one — without ever touching plaintext. + +**Export bundle (ciphertext-only)** writes every live row (or, if you opt in, tombstones too) +to a JSON file: for each row, the six crypto fields (`ciphertext`/`nonce`/`tag` for the body, +`wrapped_dek`/`wrap_nonce`/`wrap_tag` for the DEK wrap) ride as base64, plus the row's +`kek_id`, revision, and timestamps. The bundle also records the exporting session's KEK id +(`SourceKekId`) and a `FormatVersion`. A default filename +(`secrets-bundle--.json`) is offered but can be overridden. The bundle is +useless without the source KEK — there is no plaintext anywhere in it — so it is safe to copy, +attach to a ticket, or commit to a secured artifact store, but treat it with the same care as +any other encrypted-at-rest artifact (it does reveal *which* names and content types exist). + +**Import bundle** reads and parses the file *before* touching the store or prompting for +anything, so a malformed file or unsupported `FormatVersion` (this build supports **version +1** only) is caught up front. Then, per row: + +- **Same KEK as the session** — imported directly. +- **Different KEK (`SourceKekId` ≠ the session's)** — the flow tells you both KEK ids and asks + for the bundle's source key (paste / env var / key file) so it can re-wrap each row onto the + session KEK. The supplied key's own derived id is checked against the bundle's declared + `SourceKekId` *before* anything is imported — a valid-but-wrong key is rejected with both ids + shown, rather than silently landing every row in the "skipped foreign KEK" bucket. Declining + to supply a key aborts the whole import (no key means no row can be re-wrapped). +- **Name collision (a row with that name already exists locally)** — resolved by **last-writer- + wins** (the shared `SecretLastWriterWins` tie-break used by every replication topology: + later `updated_utc`, then higher revision, wins) unless you opt into **per-conflict + prompting**, in which case each collision shows both rows' timestamp and revision and asks + whether to take the bundle's row (default **No** — keep local). A winning row lands via the + store's replicated-apply path so its own revision/timestamps are preserved verbatim (the same + path cluster anti-entropy uses); a row you force *against* LWW ordering is instead written as + a fresh local upsert so the override actually sticks rather than being reverted by the next + reconciliation. + +A closing tally table (Imported / Skipped-older / Skipped-foreign-KEK / Conflicts) reports the +outcome; a non-zero "skipped foreign KEK" count is annotated with the bundle's source KEK id so +a wrong-key paste is diagnosable from the report alone without re-running the import. + +## Security notes + +- **No login.** The console is a local admin tool: whoever can run it already has direct + filesystem/DB access to the target store (they could `sqlite3` the file open themselves), so + the console adds no new trust boundary — it does not check identity, does not audit its own + actions distinctly from the store-level audit the underlying `ISecretResolver`/store calls + already perform, and should be restricted the same way you'd restrict shell access to the + host or the deployment's config files. +- **Values are never echoed.** Every prompt for a secret value or key material uses Spectre's + masked `TextPrompt(...).Secret()` — nothing typed is drawn to the terminal, and nothing typed + is written to a log. +- **Values are never cached.** Get (reveal) decrypts and writes the plaintext exactly once, with + no resolver, no in-memory cache, and no re-use across the session — asking again re-decrypts + from the row. +- **Recents contain no key material.** `~/.zb-secrets/recent-targets.json` stores only a + display name, an appsettings path, and a timestamp — never a KEK, a secret value, or a + connection string. It is safe to inspect or delete. +- **Bundles are ciphertext-only and safe at rest.** A bundle file contains no plaintext under + any circumstance — export cannot produce one and import cannot be pointed at one that + "leaks" plaintext through the format; the only way to recover a value from a bundle is to + import it into a session holding the matching KEK and then explicitly reveal it. + +## Troubleshooting + +| Symptom | Cause | Action | +|---|---|---| +| Bare `secret` prints usage instead of opening the console | stdin or stdout is redirected (a pipe, a script, CI) | Expected — run it directly at an interactive terminal with no redirection. | +| Header shows `KEK UNAVAILABLE (degraded)` | The target's configured KEK source didn't resolve in this shell (unset env var, missing file, DPAPI off Windows) | List/Delete still work. Pick a KEK-requiring flow to get the upgrade prompt, or fix the env var/file and re-select the target. | +| Reference audit shows `Undecryptable` for a name that used to resolve | The row is wrapped under a KEK the session doesn't have — often a partially-applied rotation | Run the KEK doctor; if it reports `WrongKek`, rewrap from the old KEK. | +| KEK doctor reports `Corrupt`, not `WrongKek` | `kek_id` matches but the wrap/body itself is damaged | Rewrap cannot fix this (nothing to unwrap first) — use "Old KEK is lost" to re-set the row, or restore the store from a backup. | +| Bundle import reports a large `Skipped (foreign KEK)` count | Declined (or gave the wrong) source key when prompted | Re-run the import and supply the bundle's actual source key — the report shows the expected `SourceKekId`. | +| `secret rewrap-all`-style behavior needed across a whole fleet, not just this session's KEK | The KEK doctor's rewrap remedy always targets the *session's* KEK, for lockout recovery on one store | For a general old→new rotation across every store, use the headless `secret rewrap-all` verb per [`kek-rotation.md`](kek-rotation.md). |