Joseph Doherty
559e02f5c8
Owner decisions on #110: one signing key per environment; TrustedKeys stay hand-edited; Signing:ProductionBoundary (new, default off) logs a startup warning and reports a Degraded bundle-signing-policy /health/ready entry while RequireSignedBundles is off; the private key is read via ${secret:} (proved end to end, never logged); cluster pull decided not to be built; differential deletions stay deferred. Adds docs/operations/bundle-signing-runbook.md. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
94 lines
5.1 KiB
Markdown
94 lines
5.1 KiB
Markdown
# Transport bundle signing — operator runbook (ScadaBridge#45 / #110)
|
|
|
|
Decisions (owner, 2026-11-09): **one signing key per environment** (DEV, QA, PROD each have their own);
|
|
trust is the **hand-edited `ScadaBridge:Transport:Signing:TrustedKeys`** list (no Admin UI trust list);
|
|
the private key lives in the **Secrets store** and is read through `${secret:}` config expansion;
|
|
`RequireSignedBundles` is turned **on at the production boundary** once every exporting cluster signs.
|
|
Reference: "Bundle Signing" in `docs/requirements/Component-Transport.md`.
|
|
|
|
A bundle names its signer by key fingerprint (`sha256:<hex of SubjectPublicKeyInfo>`). The importing
|
|
environment trusts a bundle only if that fingerprint matches a public key in its `TrustedKeys`.
|
|
There is no key-generation CLI; use `openssl`.
|
|
|
|
## 1. Generate a key for an environment
|
|
|
|
Run once per environment, on a trusted workstation (not on the application node):
|
|
|
|
```bash
|
|
ENV=dev # dev | qa | prod
|
|
openssl ecparam -name prime256v1 -genkey -noout | openssl pkcs8 -topk8 -nocrypt -out "bundle-signing-$ENV.pem"
|
|
openssl ec -in "bundle-signing-$ENV.pem" -pubout -out "bundle-signing-$ENV.pub.pem"
|
|
```
|
|
|
|
`bundle-signing-$ENV.pem` is the **private** key (PKCS#8 PEM); `*.pub.pem` is the public key to distribute.
|
|
The key must be ECDSA P-256 — anything else fails boot.
|
|
|
|
## 2. Store the private key as a secret
|
|
|
|
On the exporting environment, create a secret (for example `SB-BUNDLE-SIGNING-KEY`) whose value is the full
|
|
contents of the private PEM, via `/admin/secrets` on a central node (or the `ZB.MOM.WW.Secrets` CLI). Both
|
|
central nodes share the secret store when `Secrets:Store:Provider=SqlServer`, and either node may run an export.
|
|
Then reference it from config — never paste the PEM into `appsettings`:
|
|
|
|
```json
|
|
"ScadaBridge": { "Transport": { "Signing": {
|
|
"Enabled": true,
|
|
"PrivateKey": "${secret:SB-BUNDLE-SIGNING-KEY}"
|
|
} } }
|
|
```
|
|
|
|
The reference is expanded before startup validation (Program.cs, pre-host); a key that is missing or does not
|
|
parse as P-256 fails boot, it never silently ships unsigned bundles. The key is not logged: validation messages
|
|
name only the config key, and the signature entry carries only the public fingerprint. Destroy the local
|
|
`*.pem` private file after storing it (keep an offline backup per your key-custody policy).
|
|
|
|
Restart both central nodes together after changing config.
|
|
|
|
## 3. Distribute the public key into each importing environment
|
|
|
|
On every environment that will **import** bundles from this one, add the public key to `TrustedKeys`
|
|
(hand-edited config; restart both central nodes together):
|
|
|
|
```json
|
|
"Signing": { "TrustedKeys": [
|
|
{ "Name": "dev", "PublicKey": "-----BEGIN PUBLIC KEY-----\nMFkw...\n-----END PUBLIC KEY-----" }
|
|
] }
|
|
```
|
|
|
|
`Name` is shown on a verified import and recorded on the `BundleImported` audit row. Public keys are not
|
|
secret, so they may sit in `appsettings`. After this, a bundle exported from DEV shows **Verified (dev)** in
|
|
the QA import wizard; before `RequireSignedBundles` is on, an untrusted or unsigned bundle still loads and
|
|
shows Unsigned / UntrustedKey.
|
|
|
|
## 4. Rotation
|
|
|
|
1. Generate the new key pair (step 1) and store the new private key as a **new** secret name.
|
|
2. Add the new public key to `TrustedKeys` in **every importing environment** (keep the old one). Restart.
|
|
3. On the exporting environment, switch `PrivateKey` to the new `${secret:}` reference. Restart. New exports
|
|
are signed with the new key and verify everywhere.
|
|
4. Once no outstanding bundle signed with the old key still needs importing, remove the old public key from every
|
|
importing environment's `TrustedKeys`, and delete the old secret.
|
|
|
|
Never remove the old trusted key before step 3 is live everywhere (it would refuse in-flight bundles when
|
|
`RequireSignedBundles` is on). A compromised key: skip the grace period — do steps 1-3, then remove the old key
|
|
immediately.
|
|
|
|
## 5. Turning on `RequireSignedBundles` (order matters)
|
|
|
|
The code default stays **off**. For the production boundary:
|
|
|
|
1. Every exporting environment has a key (step 1-2) and `Signing:Enabled=true`; confirm an export shows
|
|
`Signed` on its `BundleExported` audit row.
|
|
2. Every key is in the production environment's `TrustedKeys` (step 3); confirm a real bundle from each
|
|
shows **Verified** in the import wizard preview.
|
|
3. On the production environment set `ScadaBridge:Transport:Signing:ProductionBoundary=true`. While
|
|
`RequireSignedBundles` is still false the node logs a startup **WARNING** ("ProductionBoundary is true but
|
|
RequireSignedBundles is false") and `/health/ready` shows the `bundle-signing-policy` entry as **Degraded**
|
|
(HTTP 200 — visible on Overview, not out of rotation). That is the reminder to finish the flip.
|
|
4. Set `ScadaBridge:Transport:Signing:RequireSignedBundles=true` and restart both central nodes together.
|
|
Unsigned, untrusted-key and badly-signed bundles are now refused at upload; the warning and Degraded entry clear.
|
|
5. Rollback: set `RequireSignedBundles=false` (the warning returns). Pre-signing bundles remain importable
|
|
whenever it is off.
|
|
|
|
A signature that is present, names a trusted key and fails to verify is refused regardless of the flag.
|