scadaproj/ZB.MOM.WW.Secrets/docs/interactive-console-quickstart.md

Quickstart — the secret interactive console

Two 5-minute jobs, one per section: seeding a fresh deployment whose ${secret:} references won't resolve, and recovering from a KEK/decrypt lockout. This is the task-oriented companion to the full reference, docs/operations/interactive-console.md — read that for the complete flow catalogue, exit-code/Ctrl-C semantics, and security model.

The console ships in the secret CLI tool (0.3.0+). You need a terminal on the box that runs the target app, with read/write access to the app's appsettings.json and its secrets store — the same access the app itself has. There is no login.

Launch it

secret

Bare secret, no verb, at a real terminal (neither stdin nor stdout redirected) opens the console. Any verb (set / get / list / rm / rotate / rewrap-all), or a redirected stream (a pipe, a script, CI), falls straight through to the headless CLI — a redirected no-verb invocation prints usage and exits 2, so nothing scripted ever hangs on a prompt.

---

Scenario A — "the app won't start, ${secret:} references don't resolve"

You just deployed an app (or added a new ${secret:NAME} reference to its config) and it fails at startup because a referenced secret has no value in the store. Goal: get every reference to Ok without hand-writing secret set commands.

1. Point the console at the app. The first screen is the target picker. Choose Enter an appsettings.json path… and give the app's base appsettings.json. The console composes config exactly like the app (base + appsettings.<env>.json + environment variables, binding the Secrets section), so it operates on precisely the store the app reads.

Select a store target
> MyApp — /opt/myapp/appsettings.json        (a recent target, if you've been here before)
  Enter an appsettings.json path…
  Manual entry (SQLite db + KEK env var)…

appsettings.json path: /opt/myapp/appsettings.json

(Illustrative transcript — actual menu rendering varies.)

2. Confirm the KEK banner. If the master key resolves in your shell, the header shows KEK OK (green). If it doesn't (env var unset, key file not mounted), it shows KEK UNAVAILABLE (degraded) (red) — that's fine; the seed flow will prompt you for the key when it needs it (paste base64 or a key-file path).

3. Run Reference audit & seed. Pick it from the action menu. It scans the composed config for every ${secret:NAME} token and classifies each name against the store:

Action
  List secrets
  Set / rotate a secret
  Get (reveal) a secret
  Delete a secret
> Reference audit & seed
  KEK doctor (lockout recovery)
  Export bundle (ciphertext-only)
  Import bundle
  Switch store target
  Quit

┌───────────────────────────┬──────────────┬───────────────────────────────────────┐
│ Secret                    │ Status       │ Config paths                          │
├───────────────────────────┼──────────────┼───────────────────────────────────────┤
│ sql/myapp/db-password     │ Missing      │ ConnectionStrings:Main                │
│ api/myapp/upstream-token  │ Missing      │ Upstream:ApiKey                       │
│ sql/myapp/reporting-pw    │ Ok           │ ConnectionStrings:Reporting           │
└───────────────────────────┴──────────────┴───────────────────────────────────────┘

The status column tells you exactly what's wrong per reference:

Status	Meaning	Seedable here?
Ok	Exists and decrypts under the session KEK.	No — nothing to do.
Missing	No row at all; the app throws at startup.	Yes.
Tombstoned	Soft-deleted row; the resolver treats it as absent.	Yes — seeding revives it.
Undecryptable	Row exists but won't decrypt (wrong/rotated KEK, or damage).	Yes — seeding overwrites it (but consider the KEK doctor first — see Scenario B).
InvalidName	The token text isn't a legal secret name (illegal chars, an unfilled REPLACE_ME).	No — fix the ${secret:…} token in the config file.

(A sixth status, PresentUnverified, only appears if you audit from a degraded session elsewhere; the seed flow always holds a KEK, so you won't see it here.)

4. Seed each gap. For every seedable gap the flow prompts in turn — a masked value, a content type, and an optional description — and seals a fresh row under the session KEK:

Seed 'sql/myapp/db-password' now? [Y/n] y
  Value for sql/myapp/db-password: ********************   (masked; nothing echoed)
  Content type: Text
  Description (optional): production DB login
Sealed 'sql/myapp/db-password'.

A closing re-audit prints a one-line count so you can confirm the gaps closed:

Audit complete: 3 Ok.

5. Restart the app. Every reference is now Ok. Re-run this flow any time you add a new ${secret:} reference or deploy to a fresh environment.

---

Scenario B — "2am: the app is logging KEK / decrypt failures"

The app was fine; now it can't decrypt its secrets — typically after a KEK rotation was only partially applied, a config value points at the wrong key source, or a store was copied from another environment. Goal: find out which KEK each row is actually under and recover.

1. Open the console on the affected store (target picker, as above). You need the KEK you believe is current; if the session opens degraded, pick the KEK doctor and paste/point at the key when prompted.

2. Run KEK doctor (lockout recovery). It probes every row (including tombstones) against the session KEK and reports a verdict per row:

Session KEK sha256:1a2b… — 4 row(s) scanned.
2 Ok, 2 wrong-KEK, 0 corrupt.
Foreign KEK id(s) seen: sha256:9f8e…

• Ok — the row's kek_id matches and it decrypts.
• WrongKek — the row is under a different KEK; reported with that KEK's own id (the "Foreign KEK id" line) so you know which key to go find. Usually there's exactly one — the store's real current key — which means the session simply has the wrong key configured.
• Corrupt — kek_id matches but the decrypt still fails: the wrap or body is damaged. Not fixable by rewrapping (there's nothing to unwrap first).

If every row is Ok, the doctor says so — the lockout is elsewhere (check the app's actual configured KEK source against the key you opened this session with).

3a. The common case — WrongKek → rewrap from the old KEK (non-destructive). Choose Rewrap from old KEK, supply the KEK the affected rows are actually under (env var, key file, or a masked paste), and confirm the exactly-scoped prompt (defaults to No):

Choose a remedy
> Rewrap from old KEK
  Old KEK is lost — re-set affected secrets
  Back

Where is the old KEK?
> Environment variable
  Key file
  Paste base64 key

  Environment variable name: ZB_SECRETS_OLD_KEY
Re-wrap 2 row(s) from KEK 'sha256:9f8e…' → session KEK 'sha256:1a2b…'? [y/N] y
Re-wrapped 2 row(s) onto the session KEK (0 already current) of 4 total.
Re-diagnosis: 4 Ok.

Secret bodies are never touched — only the DEK wrap moves onto the session KEK. A wrong old key surfaces as a decryption failure and aborts (rows already moved stay put; fix the key and re-run).

3b. The last resort — the old KEK is truly lost, or a row is Corrupt. Choose Old KEK is lost — re-set affected secrets. For each WrongKek/Corrupt row (confirmed individually, default No), type a fresh value; it's sealed under the session KEK, overwriting the unrecoverable row. This is the only remedy for a Corrupt row — but you're re-entering the real value from scratch, so use it only when the old key genuinely can't be recovered.

4. Confirm. The closing re-diagnosis line (e.g. 4 Ok) shows the store's post-remedy state in the same session.

---

When not to use the console

• A whole-fleet KEK rotation (old → new across every store): the doctor's rewrap always targets the session's KEK for one-store lockout recovery. For a planned rotation use the headless secret rewrap-all verb — see docs/operations/kek-rotation.md.
• Scripted / automated secret writes: use the headless verbs (secret set / get / list / rm / rotate). The console is deliberately interactive-only.
• Moving rows between stores (clone a deployment, stage a recovery): use Export bundle (ciphertext-only) / Import bundle — covered in the runbook.