feat(transport): production-boundary signing advisory, secret-key proof, runbook (#110)
Owner decisions on #110: one signing key per environment; TrustedKeys stay hand-edited; Signing:ProductionBoundary (new, default off) logs a startup warning and reports a Degraded bundle-signing-policy /health/ready entry while RequireSignedBundles is off; the private key is read via ${secret:} (proved end to end, never logged); cluster pull decided not to be built; differential deletions stay deferred. Adds docs/operations/bundle-signing-runbook.md. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
@@ -0,0 +1,93 @@
|
||||
# Transport bundle signing — operator runbook (ScadaBridge#45 / #110)
|
||||
|
||||
Decisions (owner, 2026-11-09): **one signing key per environment** (DEV, QA, PROD each have their own);
|
||||
trust is the **hand-edited `ScadaBridge:Transport:Signing:TrustedKeys`** list (no Admin UI trust list);
|
||||
the private key lives in the **Secrets store** and is read through `${secret:}` config expansion;
|
||||
`RequireSignedBundles` is turned **on at the production boundary** once every exporting cluster signs.
|
||||
Reference: "Bundle Signing" in `docs/requirements/Component-Transport.md`.
|
||||
|
||||
A bundle names its signer by key fingerprint (`sha256:<hex of SubjectPublicKeyInfo>`). The importing
|
||||
environment trusts a bundle only if that fingerprint matches a public key in its `TrustedKeys`.
|
||||
There is no key-generation CLI; use `openssl`.
|
||||
|
||||
## 1. Generate a key for an environment
|
||||
|
||||
Run once per environment, on a trusted workstation (not on the application node):
|
||||
|
||||
```bash
|
||||
ENV=dev # dev | qa | prod
|
||||
openssl ecparam -name prime256v1 -genkey -noout | openssl pkcs8 -topk8 -nocrypt -out "bundle-signing-$ENV.pem"
|
||||
openssl ec -in "bundle-signing-$ENV.pem" -pubout -out "bundle-signing-$ENV.pub.pem"
|
||||
```
|
||||
|
||||
`bundle-signing-$ENV.pem` is the **private** key (PKCS#8 PEM); `*.pub.pem` is the public key to distribute.
|
||||
The key must be ECDSA P-256 — anything else fails boot.
|
||||
|
||||
## 2. Store the private key as a secret
|
||||
|
||||
On the exporting environment, create a secret (for example `SB-BUNDLE-SIGNING-KEY`) whose value is the full
|
||||
contents of the private PEM, via `/admin/secrets` on a central node (or the `ZB.MOM.WW.Secrets` CLI). Both
|
||||
central nodes share the secret store when `Secrets:Store:Provider=SqlServer`, and either node may run an export.
|
||||
Then reference it from config — never paste the PEM into `appsettings`:
|
||||
|
||||
```json
|
||||
"ScadaBridge": { "Transport": { "Signing": {
|
||||
"Enabled": true,
|
||||
"PrivateKey": "${secret:SB-BUNDLE-SIGNING-KEY}"
|
||||
} } }
|
||||
```
|
||||
|
||||
The reference is expanded before startup validation (Program.cs, pre-host); a key that is missing or does not
|
||||
parse as P-256 fails boot, it never silently ships unsigned bundles. The key is not logged: validation messages
|
||||
name only the config key, and the signature entry carries only the public fingerprint. Destroy the local
|
||||
`*.pem` private file after storing it (keep an offline backup per your key-custody policy).
|
||||
|
||||
Restart both central nodes together after changing config.
|
||||
|
||||
## 3. Distribute the public key into each importing environment
|
||||
|
||||
On every environment that will **import** bundles from this one, add the public key to `TrustedKeys`
|
||||
(hand-edited config; restart both central nodes together):
|
||||
|
||||
```json
|
||||
"Signing": { "TrustedKeys": [
|
||||
{ "Name": "dev", "PublicKey": "-----BEGIN PUBLIC KEY-----\nMFkw...\n-----END PUBLIC KEY-----" }
|
||||
] }
|
||||
```
|
||||
|
||||
`Name` is shown on a verified import and recorded on the `BundleImported` audit row. Public keys are not
|
||||
secret, so they may sit in `appsettings`. After this, a bundle exported from DEV shows **Verified (dev)** in
|
||||
the QA import wizard; before `RequireSignedBundles` is on, an untrusted or unsigned bundle still loads and
|
||||
shows Unsigned / UntrustedKey.
|
||||
|
||||
## 4. Rotation
|
||||
|
||||
1. Generate the new key pair (step 1) and store the new private key as a **new** secret name.
|
||||
2. Add the new public key to `TrustedKeys` in **every importing environment** (keep the old one). Restart.
|
||||
3. On the exporting environment, switch `PrivateKey` to the new `${secret:}` reference. Restart. New exports
|
||||
are signed with the new key and verify everywhere.
|
||||
4. Once no outstanding bundle signed with the old key still needs importing, remove the old public key from every
|
||||
importing environment's `TrustedKeys`, and delete the old secret.
|
||||
|
||||
Never remove the old trusted key before step 3 is live everywhere (it would refuse in-flight bundles when
|
||||
`RequireSignedBundles` is on). A compromised key: skip the grace period — do steps 1-3, then remove the old key
|
||||
immediately.
|
||||
|
||||
## 5. Turning on `RequireSignedBundles` (order matters)
|
||||
|
||||
The code default stays **off**. For the production boundary:
|
||||
|
||||
1. Every exporting environment has a key (step 1-2) and `Signing:Enabled=true`; confirm an export shows
|
||||
`Signed` on its `BundleExported` audit row.
|
||||
2. Every key is in the production environment's `TrustedKeys` (step 3); confirm a real bundle from each
|
||||
shows **Verified** in the import wizard preview.
|
||||
3. On the production environment set `ScadaBridge:Transport:Signing:ProductionBoundary=true`. While
|
||||
`RequireSignedBundles` is still false the node logs a startup **WARNING** ("ProductionBoundary is true but
|
||||
RequireSignedBundles is false") and `/health/ready` shows the `bundle-signing-policy` entry as **Degraded**
|
||||
(HTTP 200 — visible on Overview, not out of rotation). That is the reminder to finish the flip.
|
||||
4. Set `ScadaBridge:Transport:Signing:RequireSignedBundles=true` and restart both central nodes together.
|
||||
Unsigned, untrusted-key and badly-signed bundles are now refused at upload; the warning and Degraded entry clear.
|
||||
5. Rollback: set `RequireSignedBundles=false` (the warning returns). Pre-signing bundles remain importable
|
||||
whenever it is off.
|
||||
|
||||
A signature that is present, names a trusted key and fails to verify is refused regardless of the flag.
|
||||
@@ -1,6 +1,6 @@
|
||||
# Transport addendum — cluster-to-cluster pull and differential bundles (ScadaBridge#45)
|
||||
|
||||
**Date:** 2026-11-09 · **Status:** design addendum. Not implemented, and not scheduled.
|
||||
**Date:** 2026-11-09 · **Status:** design addendum. **Owner decisions 2026-11-09 (ScadaBridge#110): pull — decided NOT to build; differential deletions — deferred.** Neither is implemented.
|
||||
**Origin:** `2026-05-24-transport-design.md` §18 and deferred-work register row 19 (now Gitea #45).
|
||||
The third deferred item, bundle signing, **is** implemented. See "Bundle Signing" in
|
||||
`docs/requirements/Component-Transport.md`.
|
||||
@@ -20,6 +20,8 @@ recommendation, so either feature can be picked up when there is a concrete need
|
||||
|
||||
## 1. Cluster-to-cluster pull
|
||||
|
||||
> **DECIDED — NOT TO BUILD (owner, 2026-11-09, ScadaBridge#110).** No cluster-to-cluster pull, no cross-environment network path, no stored cross-environment credential. The CLI export → import recipe below (Option A) is the supported path, with the receiving side requiring signed bundles (`docs/operations/bundle-signing-runbook.md`). The questions and options that follow are kept as a historical record only; do not build B or C without a new owner decision.
|
||||
|
||||
### What exists today
|
||||
|
||||
A pull can already be **orchestrated** without any new server code. The CLI speaks to any
|
||||
@@ -74,6 +76,8 @@ design session before writing any of it.
|
||||
|
||||
## 2. Differential bundles
|
||||
|
||||
> **DEFERRED (owner, 2026-11-09, ScadaBridge#110).** Differential deletions (D2) stay out of scope: the importer never deletes and there are no tombstones. Import-side Identical auto-skip already gives the differential effect. The hash-only variant below remains an option only if file size or preview length becomes a real complaint.
|
||||
|
||||
### What exists today
|
||||
|
||||
Every export is a full snapshot of the selected artifacts plus, optionally, their dependency
|
||||
@@ -114,11 +118,9 @@ unless D2 is designed on its own.
|
||||
|
||||
---
|
||||
|
||||
## Decisions this addendum leaves open
|
||||
## Decisions (resolved 2026-11-09, ScadaBridge#110)
|
||||
|
||||
1. Pull: is any cross-environment network path acceptable to change control (P3)? Nothing else
|
||||
can be decided until this is.
|
||||
2. Pull: which machine identity, a stored service account or a new export-scoped credential (P1/P2)?
|
||||
3. Differential: is propagating deletions in scope at all (D2)?
|
||||
4. Signing, which the pull depends on: see the open decisions in Component-Transport.md
|
||||
"Bundle Signing".
|
||||
1. Pull: **not built.** No cross-environment network path (P3) and no machine identity (P1/P2) will be introduced.
|
||||
2. Differential: deletions (D2) **stay deferred**; no tombstones.
|
||||
3. Signing: one key per environment, hand-edited `TrustedKeys`, `${secret:}` key source, enforcement on at the
|
||||
production boundary once every exporter signs — see `docs/operations/bundle-signing-runbook.md`.
|
||||
|
||||
@@ -384,10 +384,10 @@ Signing adds **non-repudiation** on top of the passphrase encryption — the pas
|
||||
|
||||
Generate a key pair with `openssl ecparam -name prime256v1 -genkey -noout | openssl pkcs8 -topk8 -nocrypt -out bundle-signing.pem` and `openssl ec -in bundle-signing.pem -pubout`. The private key belongs in the encrypted secret store, referenced as `${secret:}` (expanded pre-host like every other config secret); both central nodes of a cluster share it, since either may run the export.
|
||||
- **Pluggable key material.** `IBundleSigningKeySource` (export key) and `IBundleTrustStore` (fingerprint → trusted public key) are `TryAdd`-registered; a host can register its own (HSM/KMS signing, a shared trust registry) before `AddTransport()`.
|
||||
- **Open decisions** (not settled by the original design, recorded here rather than guessed in code): (1) whether each environment gets its own signing key or one key per organisation; (2) how trust is distributed and rotated beyond hand-edited `TrustedKeys` (a trust bundle in the config DB editable from the Admin UI? key expiry/revocation?); (3) whether `RequireSignedBundles` should become the default at the production boundary once every exporting cluster signs; (4) whether the default key source should read the Secrets store directly (by secret name) instead of through `${secret:}` config expansion.
|
||||
- **Decisions** (owner, 2026-11-09, ScadaBridge#110): (1) **one signing key per environment**; (2) trust stays the hand-edited `TrustedKeys` (no Admin UI trust list, no expiry/revocation store); (3) `RequireSignedBundles` is to be **on at the production boundary** once every exporting cluster signs — the code default stays off, and a cluster marked `Signing:ProductionBoundary=true` that still has it off logs a startup Warning and reports a Degraded `bundle-signing-policy` entry on `/health/ready` (HTTP 200); (4) the private key is read through `${secret:}` config expansion from the Secrets store (verified end to end by `TransportSigningSecretExpansionTests`; the key is never logged), not by a direct Secrets-store key source. Operator procedure (key generation, storage, distribution, rotation, the order for turning enforcement on): `docs/operations/bundle-signing-runbook.md`.
|
||||
- **Compatibility proof.** `Transport.IntegrationTests/Fixtures/pre-signing-*.scadabundle` were exported by the pre-#45 code and are imported end to end by `LegacyBundleCompatibilityTests` (plaintext and encrypted, signing off and on); `SignedBundleLoadTests.Signed_bundle_is_readable_by_pre_signing_read_path` shows the reverse direction.
|
||||
|
||||
Cluster-to-cluster pull and differential bundles (the other two items deferred with signing) are **not implemented**; their design questions are recorded in `docs/plans/2026-11-09-transport-pull-and-differential-addendum.md`.
|
||||
Cluster-to-cluster pull is **decided not to be built** (CLI export → import covers it) and differential-bundle deletions stay **deferred**; the rationale is recorded in `docs/plans/2026-11-09-transport-pull-and-differential-addendum.md`.
|
||||
|
||||
## Configuration Audit Trail
|
||||
|
||||
|
||||
Reference in New Issue
Block a user