diff --git a/docs/plans/2026-11-09-transport-pull-and-differential-addendum.md b/docs/plans/2026-11-09-transport-pull-and-differential-addendum.md new file mode 100644 index 00000000..89bc7982 --- /dev/null +++ b/docs/plans/2026-11-09-transport-pull-and-differential-addendum.md @@ -0,0 +1,124 @@ +# Transport addendum — cluster-to-cluster pull and differential bundles (ScadaBridge#45) + +**Date:** 2026-11-09 · **Status:** design addendum. Not implemented, and not scheduled. +**Origin:** `2026-05-24-transport-design.md` §18 and deferred-work register row 19 (now Gitea #45). +The third deferred item, bundle signing, **is** implemented. See "Bundle Signing" in +`docs/requirements/Component-Transport.md`. + +#45 grouped three deferred Transport features. Signing could be built safely because it is +off by default and adds only an optional zip entry. The other two are not specified well +enough to build: + +- **Pull** opens a new network surface whose authentication and network model nobody has + decided. +- **Differential** has no defined baseline and no defined deletion semantics. + +This addendum records the questions each one has to answer, the options, and a +recommendation, so either feature can be picked up when there is a concrete need for it. + +--- + +## 1. Cluster-to-cluster pull + +### What exists today + +A pull can already be **orchestrated** without any new server code. The CLI speaks to any +central cluster's `POST /management` endpoint. It authenticates with a bearer token from +`/auth/token`, and the server does the LDAP bind and role check. So an operator or a CI job +can run: + +```bash +scadabridge --url https://central-dev bundle export --all --passphrase "$P" --output b.scadabundle +scadabridge --url https://central-prod bundle import --input b.scadabundle --passphrase "$P" --on-conflict skip +``` + +Each leg runs under its own cluster's role gate (`RequireDesign` to export, `RequireAdmin` to +import) with a human's or service account's credentials, and each leg is audited. With +signing on, the target can also require that the bundle came from a trusted source key. +None of this adds a server-to-server path. + +### What a server-side pull would add, and the questions it raises + +A server-side pull means the target central calls the source central itself, for example +from an Import wizard option such as "Pull from environment X". + +| # | Question | Why it matters | +|---|---|---| +| P1 | **Credential.** What identity does the target present to the source? | Management auth is LDAP-backed and per-user. A pull would need either (a) a stored service-account credential for env A kept in env B's secret store, or (b) a new machine credential such as an export-scoped API key, mTLS client certificate, or PSK. Inbound-API keys (`ZB.MOM.WW.Auth.ApiKeys`) are scoped to inbound API *methods* and are deliberately non-transportable, so they cannot be reused as they are. | +| P2 | **Authorisation on the source.** Which role does a remote pull run under, and can it be limited to a selection? | `RequireDesign` is a human authoring role. A machine pull that holds it could export everything. It would need either a narrower `RequireExport` permission or a source-side allow-list of the environments that may pull. | +| P3 | **Network direction.** Which side opens the connection? | The environments are usually segmented, for example DEV vd03 and QA vq03/vq04. Production → dev egress, or dev → production ingress, is a firewall and change-control decision, not a code decision. A *push* from source to target has the mirror-image problem. | +| P4 | **Secrets in flight.** Where does the passphrase come from? | Bundles carry secrets: DataConnection credentials, SMTP and SMS tokens. An automated pull needs a passphrase that both sides know, which becomes yet another shared secret. The alternative is per-pull ECDH key agreement, which is a new crypto protocol. TLS alone protects the transport but not the bundle at rest in the target's staging directory. | +| P5 | **Provenance.** How does the target know the bundle came from env A? | Answered by #45 signing. A pull should **require** a Verified signature from a key pinned to that source environment. | +| P6 | **Review.** Does a pull skip the diff/confirm gate? | It must not. The import wizard's preview, Map step and environment-name confirmation are the safety gate at the production boundary. So a pull would only replace the file upload; it would not be "sync". | + +### Options + +- **A — Keep orchestration in the CLI (recommended).** Document the two-command recipe above, + and ask operators to enable signing with `RequireSignedBundles` on the receiving side. + This needs no new surface, no stored cross-environment credential and no firewall change. + The cost is that automation needs credentials for both clusters, which a CI system holds anyway. +- **B — Target-initiated pull over the existing `/management` endpoint.** The target stores an + env-A service account in its secret store (P1a), calls `ExportBundleCommand` remotely, and + feeds the result into the normal wizard at the Upload step, so P6 is kept. This needs + decisions on P1–P4, a `Transport:PullSources` configuration (URL, credential secret name, + pinned signing key), a source-side allow-list (P2) and a firewall rule (P3). +- **C — A dedicated gRPC export service with mTLS.** This is the most robust machine-to-machine + option and the most work: new contract, certificate management and a new port. It is + justified only if pulls become frequent and unattended. + +**Recommendation:** A, and **no code now**. Revisit B when a concrete requirement names a pair +of environments and a network path that change control has approved. Settle P1–P4 in a +design session before writing any of it. + +--- + +## 2. Differential bundles + +### What exists today + +Every export is a full snapshot of the selected artifacts plus, optionally, their dependency +closure. On import, every artifact that matches the target field for field is classified +**Identical** and auto-skipped. A full bundle therefore already *applies* differentially. Only +the changed and new rows are written, audited, and make instances stale. A differential bundle +would gain only: + +- a smaller file; +- a shorter preview for the operator to review. + +### Open questions + +| # | Question | Notes | +|---|---|---| +| D1 | **Baseline.** "Changed since *what*?" | The candidates are a previous bundle file, a timestamp, or the target's actual state. Entities carry no reliable last-modified stamp to diff against. The manifest's per-entry `version` is a constant `1` today: it is reserved, not populated. | +| D2 | **Deletions.** How does an artifact deleted on the source reach the target? | The importer never deletes, and the bundle has no tombstone concept. A differential bundle without tombstones silently diverges. Adding tombstones would make import destructive, which needs its own review UX and its own audit. | +| D3 | **Dependency closure.** Should an unchanged dependency be left out? | If it is, the target must already hold it. The importer's blocker scan already accepts a reference the target satisfies, so this works as long as the baseline really was applied to that target. If it was not, the result is a blocker the operator cannot fix without a full export. | +| D4 | **Baseline drift.** What if the target was edited after the baseline was applied? | A differential bundle computed against the *source's* last export cannot see target-side edits. The full-bundle diff can. | + +### Smallest sound design, if one is ever needed + +1. Make the manifest additive: give each `contents[]` entry a `contentHash` (SHA-256 of that + artifact's canonical DTO JSON). Older importers ignore unknown members, so this is not a + format break. +2. Export: add `--since ` in the CLI and wizard. Read the baseline's + manifest, which needs no passphrase because the manifest is plaintext, and leave out every + selected artifact whose hash is unchanged. Keep dependencies by reference only. Do not add + tombstones. +3. Manifest: record `baselineContentHash`, so the import preview can warn when the target never + imported that baseline (D4). + +**Recommendation: do not build it yet (YAGNI stands).** The import-side Identical auto-skip +already gives operators the differential *effect*. The remaining benefits, file size and +review noise, have not been reported as a problem. Revisit only if bundle size or preview +length turns into a real complaint. Build the hash-only variant above, never tombstones, +unless D2 is designed on its own. + +--- + +## Decisions this addendum leaves open + +1. Pull: is any cross-environment network path acceptable to change control (P3)? Nothing else + can be decided until this is. +2. Pull: which machine identity, a stored service account or a new export-scoped credential (P1/P2)? +3. Differential: is propagating deletions in scope at all (D2)? +4. Signing, which the pull depends on: see the open decisions in Component-Transport.md + "Bundle Signing".