ScadaBridge/docs/plans/2026-11-09-transport-pull-and-differential-addendum.md

# Transport addendum — cluster-to-cluster pull and differential bundles (ScadaBridge#45)

**Date:** 2026-11-09 · **Status:** design addendum. **Owner decisions 2026-11-09 (ScadaBridge#110): pull — decided NOT to build; differential deletions — deferred.** Neither is implemented.
**Origin:** `2026-05-24-transport-design.md` §18 and deferred-work register row 19 (now Gitea #45).
The third deferred item, bundle signing, **is** implemented. See "Bundle Signing" in
`docs/requirements/Component-Transport.md`.

#45 grouped three deferred Transport features. Signing could be built safely because it is
off by default and adds only an optional zip entry. The other two are not specified well
enough to build:

- **Pull** opens a new network surface whose authentication and network model nobody has
  decided.
- **Differential** has no defined baseline and no defined deletion semantics.

This addendum records the questions each one has to answer, the options, and a
recommendation, so either feature can be picked up when there is a concrete need for it.

---

## 1. Cluster-to-cluster pull

> **DECIDED — NOT TO BUILD (owner, 2026-11-09, ScadaBridge#110).** No cluster-to-cluster pull, no cross-environment network path, no stored cross-environment credential. The CLI export → import recipe below (Option A) is the supported path, with the receiving side requiring signed bundles (`docs/operations/bundle-signing-runbook.md`). The questions and options that follow are kept as a historical record only; do not build B or C without a new owner decision.

### What exists today

A pull can already be **orchestrated** without any new server code. The CLI speaks to any
central cluster's `POST /management` endpoint. It authenticates with a bearer token from
`/auth/token`, and the server does the LDAP bind and role check. So an operator or a CI job
can run:

```bash
scadabridge --url https://central-dev  bundle export --all --passphrase "$P" --output b.scadabundle
scadabridge --url https://central-prod bundle import --input b.scadabundle --passphrase "$P" --on-conflict skip
```

Each leg runs under its own cluster's role gate (`RequireDesign` to export, `RequireAdmin` to
import) with a human's or service account's credentials, and each leg is audited. With
signing on, the target can also require that the bundle came from a trusted source key.
None of this adds a server-to-server path.

### What a server-side pull would add, and the questions it raises

A server-side pull means the target central calls the source central itself, for example
from an Import wizard option such as "Pull from environment X".

| # | Question | Why it matters |
|---|---|---|
| P1 | **Credential.** What identity does the target present to the source? | Management auth is LDAP-backed and per-user. A pull would need either (a) a stored service-account credential for env A kept in env B's secret store, or (b) a new machine credential such as an export-scoped API key, mTLS client certificate, or PSK. Inbound-API keys (`ZB.MOM.WW.Auth.ApiKeys`) are scoped to inbound API *methods* and are deliberately non-transportable, so they cannot be reused as they are. |
| P2 | **Authorisation on the source.** Which role does a remote pull run under, and can it be limited to a selection? | `RequireDesign` is a human authoring role. A machine pull that holds it could export everything. It would need either a narrower `RequireExport` permission or a source-side allow-list of the environments that may pull. |
| P3 | **Network direction.** Which side opens the connection? | The environments are usually segmented, for example DEV vd03 and QA vq03/vq04. Production → dev egress, or dev → production ingress, is a firewall and change-control decision, not a code decision. A *push* from source to target has the mirror-image problem. |
| P4 | **Secrets in flight.** Where does the passphrase come from? | Bundles carry secrets: DataConnection credentials, SMTP and SMS tokens. An automated pull needs a passphrase that both sides know, which becomes yet another shared secret. The alternative is per-pull ECDH key agreement, which is a new crypto protocol. TLS alone protects the transport but not the bundle at rest in the target's staging directory. |
| P5 | **Provenance.** How does the target know the bundle came from env A? | Answered by #45 signing. A pull should **require** a Verified signature from a key pinned to that source environment. |
| P6 | **Review.** Does a pull skip the diff/confirm gate? | It must not. The import wizard's preview, Map step and environment-name confirmation are the safety gate at the production boundary. So a pull would only replace the file upload; it would not be "sync". |

### Options

- **A — Keep orchestration in the CLI (recommended).** Document the two-command recipe above,
  and ask operators to enable signing with `RequireSignedBundles` on the receiving side.
  This needs no new surface, no stored cross-environment credential and no firewall change.
  The cost is that automation needs credentials for both clusters, which a CI system holds anyway.
- **B — Target-initiated pull over the existing `/management` endpoint.** The target stores an
  env-A service account in its secret store (P1a), calls `ExportBundleCommand` remotely, and
  feeds the result into the normal wizard at the Upload step, so P6 is kept. This needs
  decisions on P1–P4, a `Transport:PullSources` configuration (URL, credential secret name,
  pinned signing key), a source-side allow-list (P2) and a firewall rule (P3).
- **C — A dedicated gRPC export service with mTLS.** This is the most robust machine-to-machine
  option and the most work: new contract, certificate management and a new port. It is
  justified only if pulls become frequent and unattended.

**Recommendation:** A, and **no code now**. Revisit B when a concrete requirement names a pair
of environments and a network path that change control has approved. Settle P1–P4 in a
design session before writing any of it.

---

## 2. Differential bundles

> **DEFERRED (owner, 2026-11-09, ScadaBridge#110).** Differential deletions (D2) stay out of scope: the importer never deletes and there are no tombstones. Import-side Identical auto-skip already gives the differential effect. The hash-only variant below remains an option only if file size or preview length becomes a real complaint.

### What exists today

Every export is a full snapshot of the selected artifacts plus, optionally, their dependency
closure. On import, every artifact that matches the target field for field is classified
**Identical** and auto-skipped. A full bundle therefore already *applies* differentially. Only
the changed and new rows are written, audited, and make instances stale. A differential bundle
would gain only:

- a smaller file;
- a shorter preview for the operator to review.

### Open questions

| # | Question | Notes |
|---|---|---|
| D1 | **Baseline.** "Changed since *what*?" | The candidates are a previous bundle file, a timestamp, or the target's actual state. Entities carry no reliable last-modified stamp to diff against. The manifest's per-entry `version` is a constant `1` today: it is reserved, not populated. |
| D2 | **Deletions.** How does an artifact deleted on the source reach the target? | The importer never deletes, and the bundle has no tombstone concept. A differential bundle without tombstones silently diverges. Adding tombstones would make import destructive, which needs its own review UX and its own audit. |
| D3 | **Dependency closure.** Should an unchanged dependency be left out? | If it is, the target must already hold it. The importer's blocker scan already accepts a reference the target satisfies, so this works as long as the baseline really was applied to that target. If it was not, the result is a blocker the operator cannot fix without a full export. |
| D4 | **Baseline drift.** What if the target was edited after the baseline was applied? | A differential bundle computed against the *source's* last export cannot see target-side edits. The full-bundle diff can. |

### Smallest sound design, if one is ever needed

1. Make the manifest additive: give each `contents[]` entry a `contentHash` (SHA-256 of that
   artifact's canonical DTO JSON). Older importers ignore unknown members, so this is not a
   format break.
2. Export: add `--since <baseline.scadabundle>` in the CLI and wizard. Read the baseline's
   manifest, which needs no passphrase because the manifest is plaintext, and leave out every
   selected artifact whose hash is unchanged. Keep dependencies by reference only. Do not add
   tombstones.
3. Manifest: record `baselineContentHash`, so the import preview can warn when the target never
   imported that baseline (D4).

**Recommendation: do not build it yet (YAGNI stands).** The import-side Identical auto-skip
already gives operators the differential *effect*. The remaining benefits, file size and
review noise, have not been reported as a problem. Revisit only if bundle size or preview
length turns into a real complaint. Build the hash-only variant above, never tombstones,
unless D2 is designed on its own.

---

## Decisions (resolved 2026-11-09, ScadaBridge#110)

1. Pull: **not built.** No cross-environment network path (P3) and no machine identity (P1/P2) will be introduced.
2. Differential: deletions (D2) **stay deferred**; no tombstones.
3. Signing: one key per environment, hand-edited `TrustedKeys`, `${secret:}` key source, enforcement on at the
   production boundary once every exporter signs — see `docs/operations/bundle-signing-runbook.md`.