feat(transport): production-boundary signing advisory, secret-key proof, runbook (#110)

Owner decisions on #110: one signing key per environment; TrustedKeys stay hand-edited;
Signing:ProductionBoundary (new, default off) logs a startup warning and reports a Degraded
bundle-signing-policy /health/ready entry while RequireSignedBundles is off; the private key
is read via ${secret:} (proved end to end, never logged); cluster pull decided not to be
built; differential deletions stay deferred. Adds docs/operations/bundle-signing-runbook.md.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 22:50:18 -05:00
parent a4806576af
commit 559e02f5c8
11 changed files with 421 additions and 10 deletions
+93
View File
@@ -0,0 +1,93 @@
# Transport bundle signing — operator runbook (ScadaBridge#45 / #110)
Decisions (owner, 2026-11-09): **one signing key per environment** (DEV, QA, PROD each have their own);
trust is the **hand-edited `ScadaBridge:Transport:Signing:TrustedKeys`** list (no Admin UI trust list);
the private key lives in the **Secrets store** and is read through `${secret:}` config expansion;
`RequireSignedBundles` is turned **on at the production boundary** once every exporting cluster signs.
Reference: "Bundle Signing" in `docs/requirements/Component-Transport.md`.
A bundle names its signer by key fingerprint (`sha256:<hex of SubjectPublicKeyInfo>`). The importing
environment trusts a bundle only if that fingerprint matches a public key in its `TrustedKeys`.
There is no key-generation CLI; use `openssl`.
## 1. Generate a key for an environment
Run once per environment, on a trusted workstation (not on the application node):
```bash
ENV=dev # dev | qa | prod
openssl ecparam -name prime256v1 -genkey -noout | openssl pkcs8 -topk8 -nocrypt -out "bundle-signing-$ENV.pem"
openssl ec -in "bundle-signing-$ENV.pem" -pubout -out "bundle-signing-$ENV.pub.pem"
```
`bundle-signing-$ENV.pem` is the **private** key (PKCS#8 PEM); `*.pub.pem` is the public key to distribute.
The key must be ECDSA P-256 — anything else fails boot.
## 2. Store the private key as a secret
On the exporting environment, create a secret (for example `SB-BUNDLE-SIGNING-KEY`) whose value is the full
contents of the private PEM, via `/admin/secrets` on a central node (or the `ZB.MOM.WW.Secrets` CLI). Both
central nodes share the secret store when `Secrets:Store:Provider=SqlServer`, and either node may run an export.
Then reference it from config — never paste the PEM into `appsettings`:
```json
"ScadaBridge": { "Transport": { "Signing": {
"Enabled": true,
"PrivateKey": "${secret:SB-BUNDLE-SIGNING-KEY}"
} } }
```
The reference is expanded before startup validation (Program.cs, pre-host); a key that is missing or does not
parse as P-256 fails boot, it never silently ships unsigned bundles. The key is not logged: validation messages
name only the config key, and the signature entry carries only the public fingerprint. Destroy the local
`*.pem` private file after storing it (keep an offline backup per your key-custody policy).
Restart both central nodes together after changing config.
## 3. Distribute the public key into each importing environment
On every environment that will **import** bundles from this one, add the public key to `TrustedKeys`
(hand-edited config; restart both central nodes together):
```json
"Signing": { "TrustedKeys": [
{ "Name": "dev", "PublicKey": "-----BEGIN PUBLIC KEY-----\nMFkw...\n-----END PUBLIC KEY-----" }
] }
```
`Name` is shown on a verified import and recorded on the `BundleImported` audit row. Public keys are not
secret, so they may sit in `appsettings`. After this, a bundle exported from DEV shows **Verified (dev)** in
the QA import wizard; before `RequireSignedBundles` is on, an untrusted or unsigned bundle still loads and
shows Unsigned / UntrustedKey.
## 4. Rotation
1. Generate the new key pair (step 1) and store the new private key as a **new** secret name.
2. Add the new public key to `TrustedKeys` in **every importing environment** (keep the old one). Restart.
3. On the exporting environment, switch `PrivateKey` to the new `${secret:}` reference. Restart. New exports
are signed with the new key and verify everywhere.
4. Once no outstanding bundle signed with the old key still needs importing, remove the old public key from every
importing environment's `TrustedKeys`, and delete the old secret.
Never remove the old trusted key before step 3 is live everywhere (it would refuse in-flight bundles when
`RequireSignedBundles` is on). A compromised key: skip the grace period — do steps 1-3, then remove the old key
immediately.
## 5. Turning on `RequireSignedBundles` (order matters)
The code default stays **off**. For the production boundary:
1. Every exporting environment has a key (step 1-2) and `Signing:Enabled=true`; confirm an export shows
`Signed` on its `BundleExported` audit row.
2. Every key is in the production environment's `TrustedKeys` (step 3); confirm a real bundle from each
shows **Verified** in the import wizard preview.
3. On the production environment set `ScadaBridge:Transport:Signing:ProductionBoundary=true`. While
`RequireSignedBundles` is still false the node logs a startup **WARNING** ("ProductionBoundary is true but
RequireSignedBundles is false") and `/health/ready` shows the `bundle-signing-policy` entry as **Degraded**
(HTTP 200 — visible on Overview, not out of rotation). That is the reminder to finish the flip.
4. Set `ScadaBridge:Transport:Signing:RequireSignedBundles=true` and restart both central nodes together.
Unsigned, untrusted-key and badly-signed bundles are now refused at upload; the warning and Degraded entry clear.
5. Rollback: set `RequireSignedBundles=false` (the warning returns). Pre-signing bundles remain importable
whenever it is off.
A signature that is present, names a trusted key and fails to verify is refused regardless of the flag.