docs(secrets): interactive console runbook + 0.3.0 version bump
This commit is contained in:
Joseph Doherty
@@ -5,7 +5,7 @@
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<LangVersion>latest</LangVersion>
|
||||
<Version>0.2.3</Version>
|
||||
<Version>0.3.0</Version>
|
||||
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
|
||||
<PackageReadmeFile>README.md</PackageReadmeFile>
|
||||
</PropertyGroup>
|
||||
|
||||
@@ -16,6 +16,45 @@ plaintext back on demand — from application code, from configuration, or from
|
||||
|
||||
A `secret` CLI (`set` / `get` / `list` / `rm` / `rotate` / `rewrap-all`) ships in the repo (not packed).
|
||||
|
||||
### Interactive console
|
||||
|
||||
Run `secret` with **no arguments** in a real terminal and it opens an interactive console
|
||||
instead of printing usage — a menu-driven session for an operator sitting at a deployment,
|
||||
rather than a scripted one-shot verb. The launch condition is exact: no verb **and** neither
|
||||
stdin nor stdout is redirected. Any redirection (a pipe, `| cat`, CI, a script) falls straight
|
||||
through to the headless usage/exit-2 behavior above, so nothing scripted ever hangs on a
|
||||
prompt.
|
||||
|
||||
The first screen picks a **store target** — recent targets (persisted to
|
||||
`~/.zb-secrets/recent-targets.json`, names + paths only, never key material), an app's
|
||||
`appsettings.json` (the console composes base + `appsettings.<env>.json` + environment
|
||||
variables and binds `Secrets` exactly like the app itself, so every operation hits precisely
|
||||
the store the app reads), or manual entry (a raw SQLite path + a master-key environment
|
||||
variable name). If the configured KEK cannot be resolved in the operator's shell, the session
|
||||
still opens — **degraded**, metadata-only — and any flow that needs the key prompts for one
|
||||
(masked paste or a key file) to upgrade it in place.
|
||||
|
||||
Once a target is open, the menu offers:
|
||||
|
||||
- **List secrets** — metadata table (name, content type, KEK id, revision, updated); no KEK needed.
|
||||
- **Set / rotate a secret** — masked value prompt, seals a fresh row under the session KEK.
|
||||
- **Get (reveal) a secret** — confirm-gated; prints the plaintext once, never cached or logged.
|
||||
- **Delete a secret** — confirm-gated tombstone; no KEK needed.
|
||||
- **Reference audit & seed** — scans the target's composed configuration for `${secret:NAME}`
|
||||
tokens, classifies each Ok / Missing / Tombstoned / Undecryptable / InvalidName, and walks the
|
||||
gaps through a seed prompt. This is *the* deployment-setup flow.
|
||||
- **KEK doctor (lockout recovery)** — per-row diagnosis (Ok / WrongKek / Corrupt, with the
|
||||
foreign KEK id) plus a guided remedy: rewrap every row from an old KEK, or re-set individual
|
||||
rows when the old KEK is unrecoverable.
|
||||
- **Export bundle (ciphertext-only)** / **Import bundle** — move rows between stores. Bundles
|
||||
carry name + metadata + wrapped DEK + ciphertext, never plaintext; importing across KEKs
|
||||
verifies the pasted source key against the bundle's KEK id before re-wrapping, conflicts
|
||||
resolve last-writer-wins (with an optional per-row prompt), and the format is versioned.
|
||||
|
||||
There is no login — the console runs locally with direct file/DB access, the same trust model
|
||||
as any other admin CLI. See the operator runbook for the full walkthrough:
|
||||
[`docs/operations/interactive-console.md`](docs/operations/interactive-console.md).
|
||||
|
||||
## How it protects secrets
|
||||
|
||||
Each value is encrypted with a fresh **data key (DEK)** using AES-256-GCM; the DEK is then
|
||||
@@ -149,4 +188,6 @@ node can always reach a shared database, because you pay real distributed-system
|
||||
stays invisible to last-writer-wins. Run `rewrap-all` against **each** independent store.
|
||||
|
||||
See [`docs/operations/clustered-secrets.md`](docs/operations/clustered-secrets.md) for operator
|
||||
setup, and [`docs/operations/kek-rotation.md`](docs/operations/kek-rotation.md) for rotation.
|
||||
setup, [`docs/operations/kek-rotation.md`](docs/operations/kek-rotation.md) for rotation, and
|
||||
[`docs/operations/interactive-console.md`](docs/operations/interactive-console.md) for the
|
||||
`secret` interactive console (deployment seeding, lockout recovery, bundles).
|
||||
|
||||
@@ -0,0 +1,256 @@
|
||||
# Operator runbook — the `secret` interactive console (`ZB.MOM.WW.Secrets`)
|
||||
|
||||
**Audience:** operators running the `secret` CLI at a terminal against a deployment's store.
|
||||
**Primitive:** a bare `secret` invocation (no verb) on a real TTY → `InteractiveEntry` →
|
||||
`InteractiveShell` → pluggable `IInteractiveFlow`s over an opened `SecretsSession`.
|
||||
**Status:** ships alongside the headless verbs (`set` / `get` / `list` / `rm` / `rotate` /
|
||||
`rewrap-all`), which are unaffected. Companion to
|
||||
[`kek-rotation.md`](kek-rotation.md) (the `rewrap-all` primitive the KEK-doctor remedy uses
|
||||
under the hood) and [`clustered-secrets.md`](clustered-secrets.md) (replication topologies the
|
||||
console does not manage).
|
||||
|
||||
---
|
||||
|
||||
## What the console is (and is not)
|
||||
|
||||
Every flow in the console is a thin UI wrapper over the same store/cipher primitives the
|
||||
headless CLI and the app itself use — `ISecretStore`, `ISecretCipher`, `KekRotationService`,
|
||||
`SecretReferenceExpander`'s token pattern. Nothing about the store, the wrap scheme, or the
|
||||
crypto changes. What the console adds is:
|
||||
|
||||
- **Target discovery** — pick a deployment's store the same way its app would, instead of
|
||||
hand-assembling a connection string.
|
||||
- **A guided menu** in place of memorized verb syntax, with confirm-gates and masked prompts
|
||||
where a headless verb would take a positional argument.
|
||||
- **Two flows with no headless equivalent**: the reference audit & seed (deployment setup) and
|
||||
the KEK doctor (lockout triage + guided remedy).
|
||||
|
||||
It is **not** a login surface, a replication controller, or a replacement for `rewrap-all` —
|
||||
it drives that primitive for one convenience case (rewrap onto the *session's* KEK) and defers
|
||||
to the headless verb (and [`kek-rotation.md`](kek-rotation.md)) for a general old→new rotation.
|
||||
|
||||
## Launching
|
||||
|
||||
```bash
|
||||
secret
|
||||
```
|
||||
|
||||
- **Real terminal, no args:** opens the console. The check is exact —
|
||||
`args.Length == 0 && !Console.IsInputRedirected && !Console.IsOutputRedirected` — so any
|
||||
redirection at all (a pipe, `secret | cat`, a CI runner, a script capturing output) falls
|
||||
through to the ordinary headless path instead of hanging on a prompt.
|
||||
- **Any verb, or a redirected stream:** unchanged headless behavior — `set` / `get` / `list` /
|
||||
`rm` / `rotate` / `rewrap-all` dispatch exactly as before; a bare `secret` with a redirected
|
||||
stream (or an unrecognized verb) prints usage and exits **2**, same as pre-console builds.
|
||||
- **Exit codes:** the console itself returns **0** on a clean quit (menu "Quit", or Ctrl-C at
|
||||
the target picker or the action menu) and never returns a non-zero code on its own — a
|
||||
failed action is caught, rendered as an error panel, and returns you to the menu rather than
|
||||
exiting the process. Ctrl-C mid-flow abandons just that action.
|
||||
- **Ctrl-C behavior:** cooperative cancellation is checked between actions and at the target
|
||||
picker; Spectre's blocking prompts cannot observe the token mid-prompt, but the surrounding
|
||||
loop always resolves to a clean exit rather than a stack trace.
|
||||
|
||||
## Targeting a deployment
|
||||
|
||||
The first screen is always the target picker:
|
||||
|
||||
1. **Recent targets** — up to 10 entries from `~/.zb-secrets/recent-targets.json`, newest
|
||||
first, labeled `<name> — <appsettings path>`. Selecting one re-reads that path (config may
|
||||
have changed since last time) and re-stamps it as most-recent.
|
||||
2. **Enter an appsettings.json path…** — point at any app's base `appsettings.json`.
|
||||
3. **Manual entry (SQLite db + KEK env var)…** — a raw SQLite file path plus a master-key
|
||||
environment variable name (default `ZB_SECRETS_MASTER_KEY`), for a store with no
|
||||
accompanying appsettings file. Manual targets are never added to recents (there is no
|
||||
appsettings path to remember).
|
||||
|
||||
**The honest-store guarantee.** Option 2 composes configuration *exactly the way the target
|
||||
app would at startup*: `appsettings.json` as the base, an optional `appsettings.<env>.json`
|
||||
overlay, then environment variables — layered in that order via the same
|
||||
`ConfigurationBuilder` pattern the app's own `Host.CreateApplicationBuilder` uses — and binds
|
||||
the `Secrets` section (and, if present, `Secrets:SqlServer`) out of the result. There is no
|
||||
separate "CLI view" of the config that could drift from what the app actually reads: if the app
|
||||
resolves its SQLite path or its SQL-Server hub connection string a particular way, the console
|
||||
resolves it the same way, against the same files, from the same working directory. This is
|
||||
also why `list`/`get`/`set` here can be trusted as evidence of what the running app sees.
|
||||
|
||||
**Store selection mirrors the app's DI wiring**: a configured `Secrets:SqlServer` connection
|
||||
string wins over local SQLite — *unless* that connection string is itself an unresolved
|
||||
`${secret:...}` reference (the app resolves that from its own bootstrapped store at startup;
|
||||
the console has no such bootstrap step), in which case the console falls back to the local
|
||||
SQLite store and prints a warning. In hub-replication mode, editing the SQL-Server hub through
|
||||
the console is the correct target — the change converges to every node's local store on the
|
||||
next sync sweep, rather than racing one node's SQLite against the hub.
|
||||
|
||||
**Recents contain no secret material** — only a display name and the appsettings path, ever.
|
||||
See the security notes below.
|
||||
|
||||
## Degraded sessions and the KEK prompt
|
||||
|
||||
Opening a session never fails outright on a bad or missing KEK. If the configured master key
|
||||
cannot be resolved — unset/empty env var, missing key file, a malformed `MasterKey:Source`,
|
||||
DPAPI off Windows — the session opens anyway, **degraded**: metadata-only operations (List,
|
||||
Delete — tombstoning touches no ciphertext) work immediately, and the header banner shows
|
||||
`KEK UNAVAILABLE (degraded)` in red instead of `KEK OK` in green.
|
||||
|
||||
The moment the operator picks a flow that needs plaintext access (Set/rotate, Get, reference
|
||||
audit & seed, KEK doctor, bundle export/import — anything whose `RequiresKek` is true), the
|
||||
shell intercepts the dispatch and prompts:
|
||||
|
||||
- **Paste base64 key** — a masked `TextPrompt` (nothing echoed) fed straight to an in-memory
|
||||
provider; nothing is written to disk.
|
||||
- **Key file path** — points at a file the configured `File` provider can read.
|
||||
|
||||
A successful upgrade re-opens the session with that key and continues into the flow the
|
||||
operator originally chose. A key that fails to resolve, or resolves but still doesn't unlock
|
||||
the session (wrong key), reports the failure and returns to the menu — the degraded session is
|
||||
never silently discarded.
|
||||
|
||||
## Deployment seeding walk-through: Reference audit & seed
|
||||
|
||||
This is the flow to run **the first time you point the console at a fresh deployment**, or
|
||||
whenever an app fails to start because a `${secret:NAME}` reference can't resolve.
|
||||
|
||||
1. The auditor walks every value in the target's *composed* configuration (same honest-store
|
||||
composition as targeting, above) with the identical `\$\{secret:([^}]+)\}` token pattern
|
||||
`SecretReferenceExpander` uses at app startup — so what you see here is exactly what the app
|
||||
would try to resolve, including config keys documented via a `"_comment": "...${secret:x}..."`
|
||||
convention, which are skipped the same way the expander skips them.
|
||||
2. Each distinct referenced name is looked up against the store and classified:
|
||||
- **Ok** — exists and decrypts under the session KEK. Nothing to do.
|
||||
- **Missing** — no row at all. A fail-closed expansion would throw at app startup.
|
||||
- **Tombstoned** — a soft-deleted row. Also unresolvable to the app.
|
||||
- **Undecryptable** — the row exists but does not decrypt under the session KEK (wrong or
|
||||
rotated key, or damage). *This is often the first sign a KEK doctor pass is needed.*
|
||||
- **InvalidName** — the token text isn't a legal secret name (illegal characters, an
|
||||
unfilled placeholder like `REPLACE_ME`). Can't be looked up or seeded — fix the token in
|
||||
the config file itself.
|
||||
3. A color-coded table renders every finding with the config key paths that reference it (so
|
||||
you know exactly which setting(s) are affected), then the flow walks each seedable gap
|
||||
(Missing / Tombstoned / Undecryptable) in turn, asking to seed it now. Accepting prompts for
|
||||
a masked value, a content type, and an optional description, then seals a fresh row under
|
||||
the session KEK — **overwriting** an Undecryptable row and **reviving** (un-tombstoning) a
|
||||
Tombstoned one.
|
||||
4. A closing re-audit prints a one-line count-by-status summary so you can confirm the gap
|
||||
closed without re-reading the whole table.
|
||||
|
||||
Run this again any time after adding a new `${secret:}` reference to a config file, or after
|
||||
deploying to a fresh environment — it is the fast path from "app won't start, references don't
|
||||
resolve" to "every reference is Ok."
|
||||
|
||||
## Lockout playbook: KEK doctor
|
||||
|
||||
Run this when an app (or the console itself) reports decrypt failures it didn't before —
|
||||
typically after a KEK rotation was only partially applied, a config value pointing at the
|
||||
wrong key source, or a store copied from another environment.
|
||||
|
||||
**Diagnosis** probes *every* row (including tombstones, since they still carry a KEK-wrapped
|
||||
DEK and block a clean `rewrap-all`) against the session's KEK:
|
||||
|
||||
- **Ok** — the row's `kek_id` matches the session KEK and it decrypts cleanly.
|
||||
- **WrongKek** — the row's `kek_id` differs from the session KEK. Not decrypted (no point —
|
||||
the wrap alone tells you it won't work); reported with the row's *own* `kek_id` so you know
|
||||
which key to go find. The summary line also lists the distinct **foreign KEK ids** seen
|
||||
across all wrong-KEK rows — usually there is exactly one (the store's actual current key),
|
||||
which tells you the session simply has the wrong key configured.
|
||||
- **Corrupt** — the `kek_id` matches (so the wrap envelope claims to be under the right key),
|
||||
but the decrypt still fails closed. This means the wrap envelope or the sealed body itself is
|
||||
damaged — a **different problem than WrongKek**, and *not* fixable by rewrapping: rewrapping
|
||||
only re-wraps a DEK it can first unwrap, and a Corrupt row can't be unwrapped at all.
|
||||
|
||||
If every row reports Ok, the doctor says so and there is nothing to remedy — the lockout is
|
||||
elsewhere (check the app's actual configured KEK source against what you used to open this
|
||||
session).
|
||||
|
||||
**Two remedies**, offered only when something is unhealthy:
|
||||
|
||||
- **Rewrap from old KEK** — the non-destructive path, and the *only* correct remedy for
|
||||
WrongKek rows. You supply the old KEK (env var, key file, or a masked paste) the affected
|
||||
rows are actually wrapped under; the flow shows exactly how many rows will move and from
|
||||
which KEK id, then asks for an explicit confirmation (default **No** — nothing happens on
|
||||
Enter) before calling the same `KekRotationService.RewrapAllAsync` primitive
|
||||
[`kek-rotation.md`](kek-rotation.md) documents for the headless `rewrap-all` verb. Secret
|
||||
bodies are never touched — only the DEK wrap moves onto the session KEK. A wrong old key
|
||||
surfaces as a decryption failure and aborts (rows already rewrapped before the anomaly stay
|
||||
persisted, so a retry after fixing the key resumes cleanly).
|
||||
- **Old KEK is lost — re-set affected secrets** — the destructive last resort, and the *only*
|
||||
remedy for a Corrupt row (which cannot be rewrapped because it can't be unwrapped in the
|
||||
first place). For every WrongKek or Corrupt row still affected, the flow offers a masked
|
||||
re-set: type a fresh value, and it's sealed fresh under the session KEK, overwriting the
|
||||
unrecoverable row. Each row is confirmed individually (default **No**) so you can skip rows
|
||||
you'd rather chase down the old key for instead.
|
||||
|
||||
A closing re-diagnosis prints a one-line count-by-status summary so you can see the remedy's
|
||||
effect immediately, in the same session, without re-running the flow.
|
||||
|
||||
## Bundles: export / import
|
||||
|
||||
Bundles move secret rows **between deployment stores** — cloning a deployment, staging a
|
||||
recovery, seeding a new environment from a known-good one — without ever touching plaintext.
|
||||
|
||||
**Export bundle (ciphertext-only)** writes every live row (or, if you opt in, tombstones too)
|
||||
to a JSON file: for each row, the six crypto fields (`ciphertext`/`nonce`/`tag` for the body,
|
||||
`wrapped_dek`/`wrap_nonce`/`wrap_tag` for the DEK wrap) ride as base64, plus the row's
|
||||
`kek_id`, revision, and timestamps. The bundle also records the exporting session's KEK id
|
||||
(`SourceKekId`) and a `FormatVersion`. A default filename
|
||||
(`secrets-bundle-<target>-<yyyyMMdd>.json`) is offered but can be overridden. The bundle is
|
||||
useless without the source KEK — there is no plaintext anywhere in it — so it is safe to copy,
|
||||
attach to a ticket, or commit to a secured artifact store, but treat it with the same care as
|
||||
any other encrypted-at-rest artifact (it does reveal *which* names and content types exist).
|
||||
|
||||
**Import bundle** reads and parses the file *before* touching the store or prompting for
|
||||
anything, so a malformed file or unsupported `FormatVersion` (this build supports **version
|
||||
1** only) is caught up front. Then, per row:
|
||||
|
||||
- **Same KEK as the session** — imported directly.
|
||||
- **Different KEK (`SourceKekId` ≠ the session's)** — the flow tells you both KEK ids and asks
|
||||
for the bundle's source key (paste / env var / key file) so it can re-wrap each row onto the
|
||||
session KEK. The supplied key's own derived id is checked against the bundle's declared
|
||||
`SourceKekId` *before* anything is imported — a valid-but-wrong key is rejected with both ids
|
||||
shown, rather than silently landing every row in the "skipped foreign KEK" bucket. Declining
|
||||
to supply a key aborts the whole import (no key means no row can be re-wrapped).
|
||||
- **Name collision (a row with that name already exists locally)** — resolved by **last-writer-
|
||||
wins** (the shared `SecretLastWriterWins` tie-break used by every replication topology:
|
||||
later `updated_utc`, then higher revision, wins) unless you opt into **per-conflict
|
||||
prompting**, in which case each collision shows both rows' timestamp and revision and asks
|
||||
whether to take the bundle's row (default **No** — keep local). A winning row lands via the
|
||||
store's replicated-apply path so its own revision/timestamps are preserved verbatim (the same
|
||||
path cluster anti-entropy uses); a row you force *against* LWW ordering is instead written as
|
||||
a fresh local upsert so the override actually sticks rather than being reverted by the next
|
||||
reconciliation.
|
||||
|
||||
A closing tally table (Imported / Skipped-older / Skipped-foreign-KEK / Conflicts) reports the
|
||||
outcome; a non-zero "skipped foreign KEK" count is annotated with the bundle's source KEK id so
|
||||
a wrong-key paste is diagnosable from the report alone without re-running the import.
|
||||
|
||||
## Security notes
|
||||
|
||||
- **No login.** The console is a local admin tool: whoever can run it already has direct
|
||||
filesystem/DB access to the target store (they could `sqlite3` the file open themselves), so
|
||||
the console adds no new trust boundary — it does not check identity, does not audit its own
|
||||
actions distinctly from the store-level audit the underlying `ISecretResolver`/store calls
|
||||
already perform, and should be restricted the same way you'd restrict shell access to the
|
||||
host or the deployment's config files.
|
||||
- **Values are never echoed.** Every prompt for a secret value or key material uses Spectre's
|
||||
masked `TextPrompt(...).Secret()` — nothing typed is drawn to the terminal, and nothing typed
|
||||
is written to a log.
|
||||
- **Values are never cached.** Get (reveal) decrypts and writes the plaintext exactly once, with
|
||||
no resolver, no in-memory cache, and no re-use across the session — asking again re-decrypts
|
||||
from the row.
|
||||
- **Recents contain no key material.** `~/.zb-secrets/recent-targets.json` stores only a
|
||||
display name, an appsettings path, and a timestamp — never a KEK, a secret value, or a
|
||||
connection string. It is safe to inspect or delete.
|
||||
- **Bundles are ciphertext-only and safe at rest.** A bundle file contains no plaintext under
|
||||
any circumstance — export cannot produce one and import cannot be pointed at one that
|
||||
"leaks" plaintext through the format; the only way to recover a value from a bundle is to
|
||||
import it into a session holding the matching KEK and then explicitly reveal it.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Action |
|
||||
|---|---|---|
|
||||
| Bare `secret` prints usage instead of opening the console | stdin or stdout is redirected (a pipe, a script, CI) | Expected — run it directly at an interactive terminal with no redirection. |
|
||||
| Header shows `KEK UNAVAILABLE (degraded)` | The target's configured KEK source didn't resolve in this shell (unset env var, missing file, DPAPI off Windows) | List/Delete still work. Pick a KEK-requiring flow to get the upgrade prompt, or fix the env var/file and re-select the target. |
|
||||
| Reference audit shows `Undecryptable` for a name that used to resolve | The row is wrapped under a KEK the session doesn't have — often a partially-applied rotation | Run the KEK doctor; if it reports `WrongKek`, rewrap from the old KEK. |
|
||||
| KEK doctor reports `Corrupt`, not `WrongKek` | `kek_id` matches but the wrap/body itself is damaged | Rewrap cannot fix this (nothing to unwrap first) — use "Old KEK is lost" to re-set the row, or restore the store from a backup. |
|
||||
| Bundle import reports a large `Skipped (foreign KEK)` count | Declined (or gave the wrong) source key when prompted | Re-run the import and supply the bundle's actual source key — the report shows the expected `SourceKekId`. |
|
||||
| `secret rewrap-all`-style behavior needed across a whole fleet, not just this session's KEK | The KEK doctor's rewrap remedy always targets the *session's* KEK, for lockout recovery on one store | For a general old→new rotation across every store, use the headless `secret rewrap-all` verb per [`kek-rotation.md`](kek-rotation.md). |
|
||||
Reference in New Issue
Block a user