docs(secrets): reconcile 0.2.1 adoption - what was proven vs merely built
- All 4 apps on 0.2.1 (local branches, unpushed). - Records the corrected security story: the version bump closed NO advisory (all four repos already resolved patched 2.1.12); the real live vulnerability was in ScadaBridge, masked by a NuGetAuditSuppress, and was fixed by separate work. - Records the upstream 0.2.0 inert-Akka-replicator defect, its root cause (DI extensions with no container-building test - third instance of that class), and the 0.2.1 fix. - Marks clustered topology WIRED-but-default-OFF and explicitly NOT live-validated; Task 9 remains open and the topology is not 'adopted' until it passes.
This commit is contained in:
Joseph Doherty
@@ -148,7 +148,7 @@ each project's **code-verified current state**, and the **gaps** between. See
|
||||
| Config + validation (options / startup validation) | Adopted (lib `0.1.0`, on the feed; **all 4 apps**, pushed — the "local only / not yet pushed" caveat was stale) | Shared `ZB.MOM.WW.Configuration` lib | [`components/configuration/`](components/configuration/) | [`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) |
|
||||
| Audit (event model + writer seam) | Adopted (lib `0.1.0`, on the feed; **all 4 apps**, pushed to origin) | Shared `ZB.MOM.WW.Audit` lib | [`components/audit/`](components/audit/) | [`ZB.MOM.WW.Audit/`](ZB.MOM.WW.Audit/) |
|
||||
| Galaxy Repository (object-hierarchy SQL browse + gRPC service) | **Adopted** (feed `0.1.0` + `0.2.0`; consumed at `0.2.0` by HistorianGateway **and mxaccessgw**, whose Server wires `AddZbGalaxyRepository` — verified 2026-07-18; the prior "mxaccessgw adoption is a follow-on" claim was stale) | Shared `ZB.MOM.WW.GalaxyRepository` lib | _(design in histsdk + design doc 2026-06-23)_ | [`ZB.MOM.WW.GalaxyRepository/`](ZB.MOM.WW.GalaxyRepository/) |
|
||||
| Secrets (encrypted store + `${secret:}` resolution) | Built (libs **`0.1.3`**, **published to the Gitea feed**; **HistorianGateway adopted + live-proven** 2026-07-16; **mxaccessgw adopted G-4/G-5/G-6 + merged to `origin/main` @ `e088dfa`** 2026-07-16, box-verified fail-closed + encrypt-at-rest; **ScadaBridge adopted G-3/G-4/G-5/G-6 + merged to `origin/main` @ `128f1596`** 2026-07-16, **G-3 live-proven vs the real production MxGateway gateway** `wonder-app-vd03:5120` (secret-ref ApiKey → Connected + browsed real Galaxy); **OtOpcUa adopted G-2/G-4/G-5/G-6 + merged to `origin/master` @ `872cf7e3`** (lmxopcua) 2026-07-16 — the last app, so **all four now adopted**; Layer-B driver secrets (Galaxy API key + OpcUaClient `Password`/`UserCertificatePassword`) resolve fail-closed at driver session-open, `AddZbSecrets` registered unconditionally so driver-only nodes work; **G-2 live-proven vs the real production MxGateway** `wonder-app-vd03:5120` — a `secret:`-ref Galaxy ApiKey resolved through OtOpcUa's real `GalaxyDriverBrowser` → dummy key `MxGatewayAuthenticationException`, real key → browsed the real Galaxy root (10 nodes); temp key minted+revoked+deleted); **G-8 KEK-rotation BUILT (lib bumped `0.1.2`→`0.1.3`, 2026-07-17):** `Rewrap` DEK primitive + CAS-guarded `ApplyRewrapAsync` + `KekRotationService.RewrapAllAsync` + `secret rewrap-all` CLI + operator runbook (`ZB.MOM.WW.Secrets/docs/operations/kek-rotation.md`), full suite green + CLI smoke + adversarial crypto review PASS (TOCTOU closed via compare-and-swap). **G-7 clustered replication BUILT 2026-07-18 (lib `0.2.0`) — BOTH fork options, packaged as two new libs:** **`ZB.MOM.WW.Secrets.Replicator.SqlServer`** (shared SQL-Server `ISecretStore` = the plan's Option A, *plus* a local-store-with-hub mode) and **`ZB.MOM.WW.Secrets.Replicator.AkkaDotNet`** (peer-to-peer over distributed pub/sub = Option B, which the plan had deferred). Core gained `ISecretsStoreMigrator`, `SecretLastWriterWins` (one shared LWW tie-break so stores cannot diverge), `SecretReplicationReconciler`, and `ReplicatingSecretStore` — the last of which fixes a real gap: **nothing had ever called `ISecretReplicator.PublishAsync`**, so the seam was inert. Verified 170 offline / 184 with the live SQL suite / 1 skip / 0 warnings, including **15 live tests vs a real SQL Server 2022** (the SQLite suite ported case-for-case) and a **9-test in-process 2-node Akka cluster** (real remoting; write→peer, delete propagation, both anti-entropy directions, late-joiner catch-up). Hard constraints across all topologies: same KEK on every node, ciphertext-only across trust boundaries, and re-wraps deliberately do NOT replicate (`rewrap-all` runs once per independent store). Operator runbook `ZB.MOM.WW.Secrets/docs/operations/clustered-secrets.md`. **Corrected 2026-07-18:** G-8 + the G-7 plan **ARE committed and pushed** on `main` (`d82d345`) — the "not yet committed/pushed" note was stale. **`0.1.3` PUBLISHED to the feed 2026-07-18** (restore-verified), carrying a **transitive security pin** (`SQLitePCLRaw.lib.e_sqlite3` → `2.1.12`, advisory GHSA-2m69-gcr7-jv3q) that `0.1.2` lacked. All 4 apps still pinned at `0.1.2` → bump to consume KEK rotation + the security fix. **`0.2.0` PUBLISHED to the feed 2026-07-18** (5 packages, restore-verified from a scratch consumer; merged to `main` @ `15ef1f3`) — no app has adopted a clustered topology yet | Shared `ZB.MOM.WW.Secrets` lib (5 packages + CLI) | [`components/secrets/`](components/secrets/) | [`ZB.MOM.WW.Secrets/`](ZB.MOM.WW.Secrets/) |
|
||||
| Secrets (encrypted store + `${secret:}` resolution) | Built (libs **`0.1.3`**, **published to the Gitea feed**; **HistorianGateway adopted + live-proven** 2026-07-16; **mxaccessgw adopted G-4/G-5/G-6 + merged to `origin/main` @ `e088dfa`** 2026-07-16, box-verified fail-closed + encrypt-at-rest; **ScadaBridge adopted G-3/G-4/G-5/G-6 + merged to `origin/main` @ `128f1596`** 2026-07-16, **G-3 live-proven vs the real production MxGateway gateway** `wonder-app-vd03:5120` (secret-ref ApiKey → Connected + browsed real Galaxy); **OtOpcUa adopted G-2/G-4/G-5/G-6 + merged to `origin/master` @ `872cf7e3`** (lmxopcua) 2026-07-16 — the last app, so **all four now adopted**; Layer-B driver secrets (Galaxy API key + OpcUaClient `Password`/`UserCertificatePassword`) resolve fail-closed at driver session-open, `AddZbSecrets` registered unconditionally so driver-only nodes work; **G-2 live-proven vs the real production MxGateway** `wonder-app-vd03:5120` — a `secret:`-ref Galaxy ApiKey resolved through OtOpcUa's real `GalaxyDriverBrowser` → dummy key `MxGatewayAuthenticationException`, real key → browsed the real Galaxy root (10 nodes); temp key minted+revoked+deleted); **G-8 KEK-rotation BUILT (lib bumped `0.1.2`→`0.1.3`, 2026-07-17):** `Rewrap` DEK primitive + CAS-guarded `ApplyRewrapAsync` + `KekRotationService.RewrapAllAsync` + `secret rewrap-all` CLI + operator runbook (`ZB.MOM.WW.Secrets/docs/operations/kek-rotation.md`), full suite green + CLI smoke + adversarial crypto review PASS (TOCTOU closed via compare-and-swap). **G-7 clustered replication BUILT 2026-07-18 (lib `0.2.0`) — BOTH fork options, packaged as two new libs:** **`ZB.MOM.WW.Secrets.Replicator.SqlServer`** (shared SQL-Server `ISecretStore` = the plan's Option A, *plus* a local-store-with-hub mode) and **`ZB.MOM.WW.Secrets.Replicator.AkkaDotNet`** (peer-to-peer over distributed pub/sub = Option B, which the plan had deferred). Core gained `ISecretsStoreMigrator`, `SecretLastWriterWins` (one shared LWW tie-break so stores cannot diverge), `SecretReplicationReconciler`, and `ReplicatingSecretStore` — the last of which fixes a real gap: **nothing had ever called `ISecretReplicator.PublishAsync`**, so the seam was inert. Verified 170 offline / 184 with the live SQL suite / 1 skip / 0 warnings, including **15 live tests vs a real SQL Server 2022** (the SQLite suite ported case-for-case) and a **9-test in-process 2-node Akka cluster** (real remoting; write→peer, delete propagation, both anti-entropy directions, late-joiner catch-up). Hard constraints across all topologies: same KEK on every node, ciphertext-only across trust boundaries, and re-wraps deliberately do NOT replicate (`rewrap-all` runs once per independent store). Operator runbook `ZB.MOM.WW.Secrets/docs/operations/clustered-secrets.md`. **Corrected 2026-07-18:** G-8 + the G-7 plan **ARE committed and pushed** on `main` (`d82d345`) — the "not yet committed/pushed" note was stale. **`0.1.3` PUBLISHED to the feed 2026-07-18** (restore-verified), carrying a **transitive security pin** (`SQLitePCLRaw.lib.e_sqlite3` → `2.1.12`, advisory GHSA-2m69-gcr7-jv3q) that `0.1.2` lacked. **All 4 apps bumped to `0.2.1`** 2026-07-18 (local `feat/secrets-0.2.0` branches, NOT pushed). `0.2.1` fixes a critical defect: `0.2.0`'s Akka replicator was **silently inert** (TryAdd registration order — it published into a no-op sink and never spawned an actor). SQL-Server replication was never affected. Clustered topology is **wired but default-OFF and NOT live-validated** — OtOpcUa on Akka peer-to-peer, ScadaBridge on SQL-hub mode (its central and site clusters are separate Akka clusters, so pub/sub cannot cross them). Separately, a **live** SQLitePCLRaw vulnerability was found and fixed in ScadaBridge, where a NuGetAuditSuppress had been masking vulnerable 2.1.11 in 15 projects. **`0.2.0` PUBLISHED to the feed 2026-07-18** (5 packages, restore-verified from a scratch consumer; merged to `main` @ `15ef1f3`) — no app has adopted a clustered topology yet | Shared `ZB.MOM.WW.Secrets` lib (5 packages + CLI) | [`components/secrets/`](components/secrets/) | [`ZB.MOM.WW.Secrets/`](ZB.MOM.WW.Secrets/) |
|
||||
| LocalDb (embedded cache + 2-node sync) | Built (3 pkgs `0.1.0`, **published to the Gitea feed** 2026-07-18, restore-verified from a scratch consumer; no app adoption yet) | Shared `ZB.MOM.WW.LocalDb` lib | [`docs/plans/2026-07-17-localdb-design.md`](docs/plans/2026-07-17-localdb-design.md) | [`ZB.MOM.WW.LocalDb/`](ZB.MOM.WW.LocalDb/) |
|
||||
|
||||
The auth component is fully populated: a normalized [`spec`](components/auth/spec/SPEC.md), a
|
||||
|
||||
@@ -49,11 +49,64 @@ construction).
|
||||
|
||||
### Open
|
||||
|
||||
- ⬜ Bump the four apps `0.1.2` → `0.1.3`/`0.2.0` to pick up KEK rotation + the SQLitePCLRaw security fix.
|
||||
- ✅ **All four apps bumped `0.1.2` → `0.2.1`** (2026-07-18), each on a local `feat/secrets-0.2.0`
|
||||
branch, **not pushed**. HistorianGateway `bb9d923`, MxAccessGateway `b021e29`, OtOpcUa `1ccc237c`,
|
||||
ScadaBridge `0083ce35`.
|
||||
- **Correction:** the bump was originally justified as closing GHSA-2m69-gcr7-jv3q in all four
|
||||
apps. Measured against the `0.1.2` baseline in every repo, **all four already resolved the
|
||||
patched `SQLitePCLRaw.lib.e_sqlite3 2.1.12`** — two via their own pins, two transitively via
|
||||
`Auth.ApiKeys 0.1.5`. The bump delivers **no security change**; its real value is version
|
||||
currency, `secret rewrap-all` access, and being the precondition for the replicator packages.
|
||||
- ✅ **A real SQLite vulnerability WAS found and fixed — but by separate work, not the bump.**
|
||||
ScadaBridge's `NuGetAuditSuppress` was masking a **live** high-severity finding: only the Host
|
||||
project resolved a patched `2.1.12`; `AuditLog`, `SiteRuntime`, `StoreAndForward`,
|
||||
`SiteEventLogging` and 11 test projects still resolved the vulnerable **`2.1.11`**. Fixed with an
|
||||
explicit `2.1.12` pin per SQLite-consuming project and the suppression deleted (`2a0faeab`).
|
||||
MxAccessGateway's obsolete suppression removed likewise (`2f0cfe3`). Both suppressions rested on
|
||||
false premises (*"no patched release exists"* / *"only the 3.x line is patched"*) — `2.1.12`
|
||||
patches it within 2.1.x, verified by forced restore. HistorianGateway and OtOpcUa needed nothing.
|
||||
- Rejected `CentralPackageTransitivePinningEnabled`: it clears the advisory in one line but makes
|
||||
every central version a transitive ceiling, demanding `Google.Protobuf`, `Grpc.Net.Client`,
|
||||
`Microsoft.Data.SqlClient` and `Newtonsoft.Json` bumps. That is a gRPC/data-access change to a
|
||||
production SCADA platform and deserves its own reviewed commit. **Still open as a follow-up** —
|
||||
it is the better long-term posture.
|
||||
- ✅ `0.2.0` **PUBLISHED to the Gitea feed 2026-07-18** (all 5 packages; restore-verified from a
|
||||
scratch consumer — resolves `SQLitePCLRaw.lib.e_sqlite3 2.1.12`, `Microsoft.Data.SqlClient 6.0.2`,
|
||||
`Akka.Cluster.Tools 1.5.62`, and scans clean). Merged to `main` @ `15ef1f3`.
|
||||
- ⬜ Adopt a topology in ScadaBridge / OtOpcUa (config + shared KEK; no code change needed).
|
||||
- 🟡 **Topology WIRED in both clustered apps, default-OFF, NOT live-validated.**
|
||||
**OtOpcUa → Akka peer-to-peer** (one cluster; `DistributedPubSub` already in product use;
|
||||
`ActorSystem` in DI via `Akka.Hosting`; Akka 1.5.62 matches the package exactly).
|
||||
**ScadaBridge → SQL-Server hub mode**, because central and each site are *separate* Akka clusters
|
||||
— `DistributedPubSub` cannot cross that boundary, so an Akka replicator there would produce
|
||||
**secret islands that never reconcile while every node reports healthy**.
|
||||
- Both gate on `Secrets:Replication:Enabled`, **default false**, so current behaviour is
|
||||
unchanged. ScadaBridge additionally requires a non-blank hub connection string — its options
|
||||
`Validate()` eagerly, so unconditional wiring would have broken startup on every node.
|
||||
- Bootstrap constraint recorded: the hub connstr **cannot come from the hub**; it must be seeded
|
||||
in each node's local store or supplied via env.
|
||||
- ⬜ **Task 9, the G-2-class live gate, is NOT done** — write→resolve across nodes, tombstone
|
||||
propagation (no resurrection), partition survival, and the negative-KEK fail-closed control.
|
||||
Needs a 2-node OtOpcUa rig and a ScadaBridge central-pair + site + SQL hub. **The topology is
|
||||
not "adopted" until this passes.** Do not enable `Secrets:Replication:Enabled` anywhere first.
|
||||
|
||||
### Upstream defect found during adoption — `0.2.0` Akka replication was inert
|
||||
|
||||
- ✅ **Fixed in `0.2.1`, published to the feed 2026-07-18** (all 5 packages; all four apps consume it).
|
||||
`AddZbSecretsAkkaReplication` called `AddZbSecrets` **first**, which does
|
||||
`TryAddSingleton<ISecretReplicator, NoOpSecretReplicator>()`, so the package's own `TryAdd` was
|
||||
silently discarded. Replication published into a no-op sink and **no actor was ever spawned** —
|
||||
no exception, no log line. The SQL-Server replicator always had the correct order.
|
||||
- **Why it escaped:** the SQL-Server package had a DI test asserting its replicator type; the Akka
|
||||
package had none. Every Akka test exercised the actor, serializer and reconciler in isolation —
|
||||
**nothing built a container**. This is the **third** instance of this defect class in this library
|
||||
(inert `ISecretReplicator` seam → unregistered concrete `SqliteSecretStore` → this), all with one
|
||||
cause: DI extensions with no test that constructs the graph. `0.2.1` adds 5 such tests, verified
|
||||
to discriminate (2 fail on the old order).
|
||||
- **Testing note:** `Akka.TestKit` is the family convention for non-end-to-end Akka tests, but
|
||||
`Akka.TestKit.Xunit2` is **xunit-v2-only** and OtOpcUa's `Host.IntegrationTests` is on xunit.v3
|
||||
(`CS0433` if added). OtOpcUa's guards therefore assert on `ServiceCollection` descriptors — which
|
||||
is exactly where this defect lives — while actor creation and convergence stay covered by the
|
||||
library's own 2-node cluster suite.
|
||||
- ⬜ Per-app live wonder gates for the G-8 rotation path.
|
||||
- ⬜ Akka self-echo forwarding is only observable with **three** nodes; the 2-node rig covers the
|
||||
drift symptom, not the forwarding filter itself (noted in the test).
|
||||
|
||||
@@ -1,5 +1,14 @@
|
||||
# Secrets Adoption (G-2 … G-6) — Shared Design
|
||||
|
||||
> **✅ EXECUTED 2026-07-16 at lib `0.1.2` — all four apps adopted.** Version numbers and the
|
||||
> Layer-A/Layer-B wiring template below are the **historical record of that adoption** and are
|
||||
> left as-written; do not read `0.1.2` here as current. **Current version is `0.2.0`**, and the
|
||||
> clustering guidance in §5 has been superseded — see
|
||||
> [`2026-07-18-secrets-0.2.0-upgrade-and-clustering.md`](2026-07-18-secrets-0.2.0-upgrade-and-clustering.md)
|
||||
> for the version bump and the per-app clustered topology. Everything else here (API surface §1,
|
||||
> the two resolution layers §2, the wiring template §3, the gotchas §8) remains accurate and is
|
||||
> still the reference for any new consumer.
|
||||
|
||||
> **For Claude:** This is the shared design that the three per-repo implementation plans
|
||||
> depend on. It is NOT itself executed. The executable plans are:
|
||||
> - [`2026-07-16-secrets-adoption-otopcua.md`](2026-07-16-secrets-adoption-otopcua.md) — G-2, G-4, G-5, G-6
|
||||
@@ -178,28 +187,40 @@ Both apps resolve secrets on multiple nodes (OtOpcUa driver-role nodes resolve L
|
||||
secrets; both central nodes of a ScadaBridge pair resolve Layer-A config secrets at boot). Two
|
||||
constraints follow: (1) identical KEK on every node, (2) identical store contents on every node.
|
||||
|
||||
The library today ships a **SQLite-only store** with a **`NoOpSecretReplicator`** — there is no
|
||||
built-in shared-SQL store and no cross-node replication. So the near-term (G-5) posture is:
|
||||
> **⚠️ UPDATED 2026-07-18 — the interim posture below is superseded.** This section originally
|
||||
> said the library "ships a SQLite-only store with a `NoOpSecretReplicator` — there is no built-in
|
||||
> shared-SQL store and no cross-node replication," and recommended a shared-mounted-SQLite
|
||||
> workaround. **That is no longer true.** G-7 shipped in `0.2.0` with two real replication
|
||||
> packages. The KEK guidance below still stands verbatim; the *store* guidance does not.
|
||||
> The shared-SQLite-on-a-network-share workaround should **not** be adopted — it was always a
|
||||
> caveat-laden stopgap, and there is now a supported alternative.
|
||||
>
|
||||
> Current topology options and the per-app recommendation live in
|
||||
> [`2026-07-18-secrets-0.2.0-upgrade-and-clustering.md`](2026-07-18-secrets-0.2.0-upgrade-and-clustering.md) §2.
|
||||
> In brief: **OtOpcUa → Akka peer-to-peer** (one cluster, pub/sub already in product use);
|
||||
> **ScadaBridge → SQL-Server hub mode** (central and each site are *separate* Akka clusters, so
|
||||
> `DistributedPubSub` cannot cross that boundary — an Akka replicator there would create secret
|
||||
> islands that never reconcile while every node reports healthy).
|
||||
|
||||
- **Recommended interim: `File` provider with a shared mounted key** (`MasterKey.Source=File`,
|
||||
`FilePath` = a read-only mounted secret file identical on every node) **+ a single shared
|
||||
SQLite store on a shared/replicated volume** (`SqlitePath` → the shared mount). Secrets are
|
||||
written rarely, from one node's UI/CLI; every node reads the same file.
|
||||
*Best-practice fit:* satisfies the library's documented hard requirement (same KEK per node)
|
||||
with zero new code; acceptable because secret **writes** are rare and human-driven while
|
||||
**reads** dominate. *Caveat:* SQLite over a network share has known multi-writer locking
|
||||
limits — fine for read-mostly, single-writer; not a substitute for real replication.
|
||||
- **Alternative (more integrated, more work): a ConfigDb-backed `ISecretStore`** mirroring each
|
||||
app's existing `AddDataProtection().PersistKeysToDbContext<…>()` pattern (OtOpcUa persists its
|
||||
DP key ring to ConfigDb precisely to share key material across nodes). This is the "right"
|
||||
long-term shape but requires building a custom `ISecretStore` — it **overlaps G-7** and is
|
||||
deferred there, not attempted in this cut.
|
||||
**KEK requirement (unchanged, and independent of store choice):** every node must resolve the
|
||||
**same** master key. Deliver it via `MasterKey.Source=File` with a read-only mounted key file
|
||||
identical on every node, or the same `ZB_SECRETS_MASTER_KEY` env value. A node with a different
|
||||
KEK stores replicated rows fine and then fails closed on every resolve with a `kek_id` mismatch —
|
||||
it reads like data corruption but is a deployment error.
|
||||
|
||||
> **Decision to confirm at execution time:** for the two clustered apps, take the interim
|
||||
> File-KEK + shared-SQLite posture now (ships G-5 with no new library code) and track the
|
||||
> ConfigDb/`ISecretStore` integration under G-7 — **unless** you want the integrated store
|
||||
> built first, which enlarges G-5 into library work. Each clustered plan is written for the
|
||||
> interim posture and flags the G-7 hand-off.
|
||||
**Store options as of `0.2.0`** (all three preserve ciphertext-only-at-rest):
|
||||
|
||||
- **Shared SQL store** (`AddZbSecretsSqlServerStore`) — one database, one copy of each row, no
|
||||
replication and no reconciliation. Simplest; the trade is that a node partitioned from the DB
|
||||
falls back only to the resolver's short TTL cache.
|
||||
- **Local store + SQL hub** (`AddZbSecretsSqlServerReplication`) — each node keeps a local store
|
||||
and syncs bidirectionally with a central hub. Survives partition; converges eventually.
|
||||
- **Akka peer-to-peer** (`AddZbSecretsAkkaReplication`) — no shared database at all; live
|
||||
broadcast plus periodic manifest anti-entropy **within a single Akka cluster**.
|
||||
|
||||
**Rotation is per independent store.** G-8 `rewrap-all` runs once for a shared store; once per
|
||||
node *and* against the hub in hub mode; once per node in Akka mode. Re-wraps deliberately do not
|
||||
replicate (they leave `revision`/`updated_utc` untouched so LWW ignores them).
|
||||
|
||||
---
|
||||
|
||||
@@ -315,6 +336,9 @@ default and push to Gitea.
|
||||
|
||||
## 11. Task tracking
|
||||
|
||||
Each per-repo plan co-locates its `*.tasks.json` beside the `.md`. Deferred items (G-7 Akka
|
||||
replicator / shared-SQL `ISecretStore`, G-8 `RewrapAll` KEK-rotation) stay in
|
||||
[`components/secrets/GAPS.md`](../../components/secrets/GAPS.md) and are out of scope here.
|
||||
Each per-repo plan co-locates its `*.tasks.json` beside the `.md`. The items this design deferred
|
||||
— G-7 (replication) and G-8 (`RewrapAll` KEK rotation) — were **both built and shipped in `0.2.0`
|
||||
on 2026-07-18**, G-7 as *two* packages (`.Replicator.SqlServer` and `.Replicator.AkkaDotNet`)
|
||||
rather than the single deferred Akka replicator this section anticipated. Adopting them in the
|
||||
apps is [`2026-07-18-secrets-0.2.0-upgrade-and-clustering.md`](2026-07-18-secrets-0.2.0-upgrade-and-clustering.md);
|
||||
remaining backlog stays in [`components/secrets/GAPS.md`](../../components/secrets/GAPS.md).
|
||||
|
||||
@@ -1,4 +1,15 @@
|
||||
# MxAccessGateway — Secrets Adoption (G-4, G-5, G-6) Implementation Plan
|
||||
|
||||
> **✅ EXECUTED 2026-07-16** — merged to `origin/main` @ `e088dfa`, box-verified fail-closed and
|
||||
> encrypt-at-rest. This document is the **historical record**; the `0.1.2` pins are what was
|
||||
> landed then, **not** what to use now.
|
||||
>
|
||||
> **Follow-on (not yet executed):** a **version bump only** —
|
||||
> [`2026-07-18-secrets-0.2.0-upgrade-and-clustering.md`](2026-07-18-secrets-0.2.0-upgrade-and-clustering.md)
|
||||
> Task 2. This app is a single Windows/NSSM box with no Akka clustering, so **no replication
|
||||
> package and no topology apply**. The bump still matters: `0.2.0` carries a high-severity
|
||||
> transitive `SQLitePCLRaw` advisory fix that `0.1.2` lacks.
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task. Shared context: docs/plans/2026-07-16-secrets-adoption-design.md.
|
||||
|
||||
**Goal:** Adopt the already-built, already-published `ZB.MOM.WW.Secrets` library (`0.1.2`, Gitea feed, reference-consumer-proven in HistorianGateway) into **MxAccessGateway** — retiring the plaintext LDAP bind password (and any password-bearing config) at rest behind pre-host `${secret:}` expansion, adding a runtime `ISecretResolver`, and mounting the `/admin/secrets` Blazor UI. Covers gaps **G-4** (pre-host config-secret expansion), **G-5** (master-key provider), and **G-6** (mount UI). There is **no G-2/G-3** for this app — it has no database-resident driver/connection secrets.
|
||||
|
||||
@@ -1,4 +1,16 @@
|
||||
# OtOpcUa — Secrets Adoption (G-2, G-4, G-5, G-6) Implementation Plan
|
||||
|
||||
> **✅ EXECUTED 2026-07-16** — merged to `origin/master` @ `872cf7e3`, G-2 live-proven against the
|
||||
> real production MxGateway. This document is the **historical record** of that adoption; the
|
||||
> `0.1.2` pins throughout are what was landed then, **not** what to use now.
|
||||
>
|
||||
> **Follow-on (not yet executed):** bump to `0.2.0` and adopt clustered replication —
|
||||
> [`2026-07-18-secrets-0.2.0-upgrade-and-clustering.md`](2026-07-18-secrets-0.2.0-upgrade-and-clustering.md),
|
||||
> Tasks 3, 5, 6, 9. OtOpcUa's recommended topology is **Akka peer-to-peer**
|
||||
> (`ZB.MOM.WW.Secrets.Replicator.AkkaDotNet`): it is a single Akka cluster on 1.5.62, already uses
|
||||
> `DistributedPubSub` in product code, registers its `ActorSystem` in DI via `Akka.Hosting`, and
|
||||
> its driver-role nodes benefit from resolving Layer-B driver secrets while partitioned.
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task. Shared context: docs/plans/2026-07-16-secrets-adoption-design.md.
|
||||
|
||||
**Goal:** Adopt the already-built, already-published `ZB.MOM.WW.Secrets` library (@ `0.1.2`, Gitea feed, reference-consumer-proven in HistorianGateway) into **OtOpcUa** — retiring plaintext credentials at rest. Covers **G-4** (pre-host `${secret:}` config-secret expansion), **G-6** (mount `/admin/secrets` UI), **G-5** (clustered master-key posture), and the highest-value gap **G-2** (retire cleartext-in-DB driver secrets via runtime `ISecretResolver`).
|
||||
|
||||
@@ -1,5 +1,19 @@
|
||||
# ScadaBridge — Secrets Adoption (G-3, G-4, G-5, G-6) Implementation Plan
|
||||
|
||||
> **✅ EXECUTED 2026-07-16** — merged to `origin/main` @ `128f1596`, G-3 live-proven against the
|
||||
> real production MxGateway gateway. This document is the **historical record**; the `0.1.2` pins
|
||||
> are what was landed then, **not** what to use now.
|
||||
>
|
||||
> **Follow-on (not yet executed):** bump to `0.2.0` and adopt clustered replication —
|
||||
> [`2026-07-18-secrets-0.2.0-upgrade-and-clustering.md`](2026-07-18-secrets-0.2.0-upgrade-and-clustering.md),
|
||||
> Tasks 4, 7, 8, 9. ScadaBridge's recommended topology is **SQL-Server hub mode**
|
||||
> (`ZB.MOM.WW.Secrets.Replicator.SqlServer`, `AddZbSecretsSqlServerReplication`), **not** Akka.
|
||||
> Reason: central and each site are **separate Akka clusters** with their own `seed-nodes` lists,
|
||||
> so `DistributedPubSub` cannot cross the central↔site boundary — an Akka replicator here would
|
||||
> converge each cluster internally and leave **secret islands that never reconcile**, while every
|
||||
> node reports healthy. Hub mode gives sites local reads through a WAN outage, which is the
|
||||
> premise of the hub-and-spoke architecture.
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task. Shared context: [`docs/plans/2026-07-16-secrets-adoption-design.md`](2026-07-16-secrets-adoption-design.md) — read it first; this plan **references** that design (library API §1, two layers §2, wiring template §3, G-4 §4, G-5 §5, G-6 §6 incl. the ScadaBridge claim-type caveat, G-3 §7, gotchas §8) rather than re-deriving it.
|
||||
|
||||
**Goal:** Adopt the already-built, already-published `ZB.MOM.WW.Secrets` library (`0.1.2`, Gitea feed, reference-consumer-proven in HistorianGateway) into **ScadaBridge** — retiring plaintext credentials at rest. Concretely: **G-3** (retire the MxGateway per-endpoint `ApiKey` plaintext-in-ConfigDb via a runtime `ISecretResolver` `secret:` ref), **G-4** (pre-host `${secret:}` config-secret expansion — ScadaBridge is the heaviest config-secret surface in the family), **G-5** (clustered master-key posture for the central pair + site nodes), **G-6** (mount `/admin/secrets` in CentralUI, with a mandatory claim-type verification).
|
||||
|
||||
@@ -0,0 +1,402 @@
|
||||
# Secrets `0.2.0` Upgrade + Clustered Topology Adoption — Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Move all four family apps from `ZB.MOM.WW.Secrets` `0.1.2` to `0.2.0`, and adopt a **clustered secrets topology** in the two Akka-clustered apps (OtOpcUa, ScadaBridge) so a secret written on one node is resolvable on every node that needs it.
|
||||
|
||||
**Architecture:** No library construction — `0.2.0` is built, published, and restore-verified on the Gitea feed (5 packages). Two distinct workstreams: (A) a **pure version bump** for all four apps, which is behaviour-preserving and independently valuable (it picks up the G-8 KEK-rotation surface and a **high-severity transitive security fix**); (B) **topology adoption** for the two clustered apps, which is *configuration + one DI call*, no product-code change. The topology choice is **not the same for both apps** — see §2, this is the load-bearing design point.
|
||||
|
||||
**Tech Stack:** .NET 10, C#, `ZB.MOM.WW.Secrets{,.Abstractions,.Ui}` @ `0.2.0`, plus `ZB.MOM.WW.Secrets.Replicator.AkkaDotNet` (OtOpcUa) / `ZB.MOM.WW.Secrets.Replicator.SqlServer` (ScadaBridge), all from the `dohertj2-gitea` feed. Akka.NET **1.5.62** in both clustered apps — an exact match with the replicator package's Akka dependency.
|
||||
|
||||
**Predecessors:** [`2026-07-16-secrets-adoption-design.md`](2026-07-16-secrets-adoption-design.md) (the adoption template — §5 rewritten by this work), the three executed per-app adoption plans, and [`2026-07-17-secrets-g7-clustered-replication-design.md`](2026-07-17-secrets-g7-clustered-replication-design.md) (the fork analysis, now a *deployment* decision rather than a build one).
|
||||
|
||||
---
|
||||
|
||||
## 1. Why bump at all
|
||||
|
||||
> **⚠️ CORRECTED TWICE during execution, 2026-07-18 — read this before citing the bump's value.**
|
||||
> The original text claimed the bump "closes a high-severity advisory in all four apps." A first
|
||||
> correction narrowed that to "two apps." **Both were wrong.** Measured by A/B against the `0.1.2`
|
||||
> baseline in every repo: **all four already resolved the patched `SQLitePCLRaw.lib.e_sqlite3
|
||||
> 2.1.12` before this bump.** The Secrets `0.2.0` upgrade delivers **no security change to any
|
||||
> app.** The lesson — the same one G-7's post-build review taught — is that a plausible mechanism
|
||||
> is not evidence. Both wrong claims came from reasoning about what `0.2.0`'s nuspec *should*
|
||||
> deliver instead of measuring what each repo *already resolved*.
|
||||
|
||||
`0.2.0` does declare a direct dependency on the patched `SQLitePCLRaw.lib.e_sqlite3 2.1.12`
|
||||
(GHSA-2m69-gcr7-jv3q, high). That is real — it is simply **redundant everywhere**, because each
|
||||
repo had already arrived at ≥`2.1.12` by another route:
|
||||
|
||||
| App | How it already resolved ≥ 2.1.12 before the bump | Security delta from this bump |
|
||||
|---|---|---|
|
||||
| HistorianGateway | Direct pin `SQLitePCLRaw.lib.e_sqlite3 3.50.3` (cites a *different* CVE, CVE-2025-6965) | **None** |
|
||||
| OtOpcUa | `SQLitePCLRaw.bundle_e_sqlite3` pinned `2.1.12` in `Directory.Packages.props` | **None** |
|
||||
| MxAccessGateway | Transitively from the pre-existing `ZB.MOM.WW.Auth.ApiKeys 0.1.5` reference | **None** (A/B-verified on the 0.1.2 baseline) |
|
||||
| ScadaBridge | Transitively from `ZB.MOM.WW.Auth.ApiKeys 0.1.5` (commit `50d79ed1`) | **None** (A/B-verified) |
|
||||
|
||||
**The honest justification for Tasks 1–4 is therefore version hygiene plus a precondition:** the
|
||||
replication packages in Tasks 5–9 are `0.2.0` artifacts and cannot be consumed on `0.1.2`, and
|
||||
`0.2.0` exposes the G-8 KEK-rotation surface (`secret rewrap-all`) the runbooks document but no app
|
||||
can currently run. That is a legitimate reason to land them — but it is **weaker than originally
|
||||
stated**, and it means Tasks 1–4 are *not* independently valuable in the way this plan first
|
||||
claimed. If the clustering work is abandoned, the bumps buy only currency and rotation access.
|
||||
|
||||
### The real SQLite finding — and it was a live vulnerability
|
||||
|
||||
Chasing the suppressions turned up something the version bump had nothing to do with. **Fixed
|
||||
2026-07-18**, in its own commits, separate from the Secrets bump:
|
||||
|
||||
**ScadaBridge was shipping a live high-severity vulnerability, masked by its own suppression.**
|
||||
The earlier A/B only checked the *Host* project, which resolved a patched `2.1.12` transitively via
|
||||
`Auth.ApiKeys`. Every *other* SQLite consumer — `AuditLog`, `SiteRuntime`, `StoreAndForward`,
|
||||
`SiteEventLogging` and 11 test projects — still resolved the vulnerable **`2.1.11`**, and the
|
||||
`NuGetAuditSuppress` hid NU1903 for all of them. Proven by removing the suppression and forcing a
|
||||
restore: 12+ projects immediately reported NU1903.
|
||||
|
||||
Both suppressions rested on false premises: mxaccessgw's said *"no patched e_sqlite3 release exists
|
||||
yet"*; ScadaBridge's said *"the only patched native lib is the SQLitePCLRaw 3.x line."* **`2.1.12`
|
||||
patches this advisory within the 2.1.x line** — verified empirically (suppression removed + pin in
|
||||
place → forced restore reports no NU1903). HistorianGateway's separate `3.50.3` pin cites
|
||||
**CVE-2025-6965**, a *different* advisory; that is the likely source of the "only 3.x" confusion.
|
||||
|
||||
**Fix applied:** explicit `PackageReference` to `SQLitePCLRaw.lib.e_sqlite3 2.1.12` in every
|
||||
SQLite-consuming project, suppressions deleted in both repos (ScadaBridge `2a0faeab`, mxaccessgw
|
||||
`2f0cfe3`). HistorianGateway and OtOpcUa needed nothing — genuinely fixed already, no suppressions.
|
||||
|
||||
**Rejected alternative:** `CentralPackageTransitivePinningEnabled`, which clears it in one line but
|
||||
makes every central version a ceiling for transitive resolution too — immediately demanding bumps
|
||||
to `Google.Protobuf`, `Grpc.Net.Client`, `Microsoft.Data.SqlClient` and `Newtonsoft.Json`. That is
|
||||
a gRPC/data-access change to a production SCADA platform and belongs in its own reviewed commit,
|
||||
not smuggled in under a security fix. **Recorded as a genuine follow-up** — it is the better
|
||||
long-term posture.
|
||||
|
||||
Verified: ScadaBridge 55 projects build 0 warn / 0 error, **1108 tests pass across the SQLite
|
||||
layer**; mxaccessgw 0 warn / 0 error with test results identical to baseline.
|
||||
|
||||
Tasks 1–4 remain four independent, single-file, behaviour-preserving edits in four separate repos
|
||||
— **fully parallelizable**.
|
||||
|
||||
### Build/test expectations — corrected against reality
|
||||
|
||||
The original text expected "0 warnings" everywhere. Measured, on the `0.1.2` baseline:
|
||||
|
||||
- **OtOpcUa** emits **861** pre-existing warnings (`OTOPCUA0001` / xUnit analyzers). Identical count before and after the bump. Do not expect 0.
|
||||
- **HistorianGateway**'s full-solution restore **fails** on an AngleSharp `NU1902` advisory promoted to an error by `TreatWarningsAsErrors` — pre-existing, reproduced on the unmodified base branch.
|
||||
- **MxAccessGateway** has no `src/MxGateway.sln`; the solutions are `src/ZB.MOM.WW.MxGateway.slnx` and `src/ZB.MOM.WW.MxGateway.NonWindows.slnx` (the macOS-safe subset omitting the x86 net48 worker). **`CLAUDE.md`'s recorded build command is stale** and should be corrected.
|
||||
- Several suites have pre-existing environment-gated failures on macOS (live-historian DNS, Unix-domain-socket path length, shared Docker-fixture contention). **A/B against the baseline before attributing any failure to the bump.**
|
||||
|
||||
## 2. The topology fork — and why the two clustered apps resolve it differently
|
||||
|
||||
Verified against both repos on 2026-07-18. This is the single most important section; getting it wrong produces a cluster that looks healthy and silently fails to converge.
|
||||
|
||||
### OtOpcUa → **Akka peer-to-peer** (recommended)
|
||||
|
||||
- OtOpcUa is **one Akka cluster**. Every node shares a single `seed-nodes` list, so `DistributedPubSub` reaches every member. The app **already uses `DistributedPubSub` heavily** in product code (`DriverHostActor.cs:419` caches the mediator; driver-health, alarm, and deployment-ack topics all ride it), so the transport is proven in-app, not newly introduced.
|
||||
- OtOpcUa registers its `ActorSystem` **in DI via `Akka.Hosting`** — which is exactly what `SecretReplicationActorProvider` requires.
|
||||
- **Driver-role nodes resolve Layer-B driver secrets** (Galaxy API key, OpcUaClient passwords) at driver-session-open. Those nodes should keep starting drivers when central SQL is unreachable; peer-to-peer replication serves reads from local state, so a partitioned driver node keeps working on last-known-good and re-converges on heal.
|
||||
- *Best-practice fit:* matches the app's existing cluster-communication idiom, adds no new infrastructure dependency, and preserves partition tolerance for the node role that most needs it.
|
||||
- **Alternative:** shared SQL store against the existing ConfigDb (`AddZbSecretsSqlServerStore`) — simpler (zero distributed failure modes, one `rewrap-all`), but couples every driver node's secret resolution to ConfigDb reachability. Prefer this **only** if you decide driver nodes should hard-fail rather than run on cached secrets during a ConfigDb outage.
|
||||
|
||||
### ScadaBridge → **SQL-Server hub mode** (recommended) — Akka *cannot* do the job here
|
||||
|
||||
- ScadaBridge is **hub-and-spoke: one central cluster + N site clusters, each a SEPARATE Akka cluster** with its own `seed-nodes` list (`AkkaHostedService.BuildHocon`, `seed-nodes = [...]` per node). Central↔site traffic goes through `CentralCommunicationActor`, not cluster membership.
|
||||
- **Therefore `DistributedPubSub` is cluster-scoped and does not cross the central↔site boundary.** An Akka replicator would converge the central pair among themselves and each site pair among themselves, producing **independent secret islands that never reconcile** — while every node reports healthy. This is the failure mode to avoid, and it is why "integrate with Akka" is the wrong answer for this app.
|
||||
- Hub mode (`AddZbSecretsSqlServerReplication`) fits the topology exactly: each node keeps a **local** store and syncs bidirectionally with a shared central SQL hub. Site nodes keep resolving secrets **through a WAN outage** — which is the entire point of the hub-and-spoke design — and re-converge when the link returns.
|
||||
- *Best-practice fit:* mirrors how ScadaBridge already treats central SQL (config, audit forwarding) — authoritative when reachable, locally buffered when not.
|
||||
- **Alternative:** shared SQL store (`AddZbSecretsSqlServerStore`) — one store, no reconciliation, but a site that loses the WAN loses secret resolution past the resolver's short TTL cache. **Not recommended**: it contradicts the site-autonomy premise of the architecture.
|
||||
|
||||
### HistorianGateway, MxAccessGateway → **version bump only**
|
||||
|
||||
Both are single-box, non-clustered. No replication package, no topology. Tasks 1–2.
|
||||
|
||||
## 3. Constraints that apply to every topology
|
||||
|
||||
1. **Same KEK on every node — non-negotiable.** Rows are ciphertext-only; a node whose master key differs stores replicated rows fine and then **fails closed on every resolve** with a `kek_id` mismatch. It reads like data corruption but is a deployment error. Verify with the digest check in [`clustered-secrets.md`](../../ZB.MOM.WW.Secrets/docs/operations/clustered-secrets.md) §0 **before** enabling replication.
|
||||
2. **KEK rotation does not replicate.** A re-wrap deliberately leaves `revision`/`updated_utc` untouched so it stays invisible to LWW. `rewrap-all` runs **once per independent store** — once for a shared store; once per node *and* against the hub in hub mode; once per node in Akka mode.
|
||||
3. **`nuget.config` needs no change.** Both clustered repos already map `ZB.MOM.WW.Secrets.*` to `dohertj2-gitea` (OtOpcUa `NuGet.config:29`, ScadaBridge `nuget.config:29`), and that pattern already covers `ZB.MOM.WW.Secrets.Replicator.SqlServer` / `.Replicator.AkkaDotNet`. **Verified** — do not add redundant patterns.
|
||||
4. **Do not reconfigure Data Protection.** Unchanged from the original adoption: secrets envelope crypto is independent of DP, and touching DP risks invalidating cookies/hub tokens.
|
||||
|
||||
---
|
||||
|
||||
## Task 1 — HistorianGateway: bump `0.1.2` → `0.2.0`
|
||||
|
||||
**Classification:** trivial
|
||||
**Estimated implement time:** ~2 min
|
||||
**Parallelizable with:** Task 2, Task 3, Task 4 (different repos entirely)
|
||||
|
||||
**Files:**
|
||||
- Modify: `~/Desktop/HistorianGateway/src/ZB.MOM.WW.HistorianGateway.Server/ZB.MOM.WW.HistorianGateway.Server.csproj:37-38`
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. This repo does **not** use CPM for these — versions are inline on the `PackageReference`. Change both:
|
||||
```xml
|
||||
<PackageReference Include="ZB.MOM.WW.Secrets" Version="0.2.0" />
|
||||
<PackageReference Include="ZB.MOM.WW.Secrets.Ui" Version="0.2.0" />
|
||||
```
|
||||
2. Restore + build:
|
||||
```bash
|
||||
cd ~/Desktop/HistorianGateway && dotnet build ZB.MOM.WW.HistorianGateway.slnx
|
||||
```
|
||||
**Expected:** succeeds, 0 warnings. `0.2.0` is API-compatible with `0.1.2` for everything this app uses.
|
||||
3. Confirm the security fix actually landed — this is the *point* of the task, so verify it rather than assuming:
|
||||
```bash
|
||||
dotnet list ZB.MOM.WW.HistorianGateway.slnx package --include-transitive | grep -i sqlitepclraw
|
||||
```
|
||||
**Expected:** `SQLitePCLRaw.lib.e_sqlite3` resolves to `2.1.12` (not `2.1.11`).
|
||||
4. Run the offline suite: `dotnet test ZB.MOM.WW.HistorianGateway.slnx`
|
||||
**Expected:** green. **Known pre-existing:** an AngleSharp `NU1902` advisory warning in the test project — unrelated to this change; do not chase it, but confirm it is the *only* warning and that it predates the bump (`git stash` + rebuild to compare if unsure).
|
||||
5. Commit:
|
||||
```bash
|
||||
git add -A && git commit -m "chore(secrets): bump ZB.MOM.WW.Secrets 0.1.2 -> 0.2.0 (SQLitePCLRaw advisory fix + KEK rotation)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2 — MxAccessGateway: bump `0.1.2` → `0.2.0`
|
||||
|
||||
**Classification:** trivial
|
||||
**Estimated implement time:** ~2 min
|
||||
**Parallelizable with:** Task 1, Task 3, Task 4
|
||||
|
||||
**Files:**
|
||||
- Modify: `~/Desktop/MxAccessGateway/src/ZB.MOM.WW.MxGateway.Server/ZB.MOM.WW.MxGateway.Server.csproj:20-22`
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Set all three inline versions to `0.2.0` (`ZB.MOM.WW.Secrets`, `.Abstractions`, `.Ui`).
|
||||
2. Build: `cd ~/Desktop/MxAccessGateway && dotnet build src/MxGateway.sln` → **Expected:** 0 warnings.
|
||||
3. Verify the transitive pin as in Task 1 step 3.
|
||||
4. Test: `dotnet test src/MxGateway.Tests/MxGateway.Tests.csproj` → **Expected:** green (no MXAccess needed — fake worker).
|
||||
5. Commit with the same message shape as Task 1.
|
||||
|
||||
---
|
||||
|
||||
## Task 3 — OtOpcUa: bump `0.1.2` → `0.2.0` (CPM)
|
||||
|
||||
**Classification:** trivial
|
||||
**Estimated implement time:** ~2 min
|
||||
**Parallelizable with:** Task 1, Task 2, Task 4
|
||||
|
||||
**Files:**
|
||||
- Modify: `~/Desktop/OtOpcUa/Directory.Packages.props:135-137`
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. OtOpcUa **is** on Central Package Management — edit the three `<PackageVersion>` rows to `0.2.0`. Consuming csprojs carry bare `<PackageReference>` with no `Version`; **do not** add versions there.
|
||||
2. Build: `cd ~/Desktop/OtOpcUa && dotnet build ZB.MOM.WW.OtOpcUa.slnx` → **Expected:** 0 warnings (repo builds under `TreatWarningsAsErrors`).
|
||||
3. Verify the transitive pin as in Task 1 step 3.
|
||||
4. Test: `dotnet test ZB.MOM.WW.OtOpcUa.slnx` → **Expected:** green.
|
||||
5. Commit. Branch: `feat/secrets-0.2.0` off `origin/master` (**OtOpcUa's default is `master`**).
|
||||
|
||||
---
|
||||
|
||||
## Task 4 — ScadaBridge: bump `0.1.2` → `0.2.0` (CPM)
|
||||
|
||||
**Classification:** trivial
|
||||
**Estimated implement time:** ~2 min
|
||||
**Parallelizable with:** Task 1, Task 2, Task 3
|
||||
|
||||
**Files:**
|
||||
- Modify: `~/Desktop/ScadaBridge/Directory.Packages.props:89-91`
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Edit the three `<PackageVersion>` rows to `0.2.0`.
|
||||
2. Build: `cd ~/Desktop/ScadaBridge && dotnet build ZB.MOM.WW.ScadaBridge.slnx` → **Expected:** 0 warnings.
|
||||
3. Verify the transitive pin as in Task 1 step 3.
|
||||
4. Test: `dotnet test ZB.MOM.WW.ScadaBridge.slnx`
|
||||
**Expected:** green except the **known pre-existing** AngleSharp `NU1902` break in `CentralUI.Tests`. Confirm it predates the bump; do not fix it here.
|
||||
5. Commit. Branch: `feat/secrets-0.2.0` off `origin/main`.
|
||||
|
||||
---
|
||||
|
||||
> **Checkpoint — review after Tasks 1–4.** All four apps are on `0.2.0` and the advisory is closed family-wide. This is a coherent, independently shippable slice: FF-merge and push each repo before starting Task 5. If the topology work is deferred, the plan still delivered its security value.
|
||||
|
||||
---
|
||||
|
||||
## Task 5 — OtOpcUa: add the Akka replicator package
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~3 min
|
||||
**Parallelizable with:** Task 7 (different repo)
|
||||
**Depends on:** Task 3
|
||||
|
||||
**Files:**
|
||||
- Modify: `~/Desktop/OtOpcUa/Directory.Packages.props` (ZB.MOM block, beside the rows edited in Task 3)
|
||||
- Modify: `~/Desktop/OtOpcUa/src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj`
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Add the CPM version row:
|
||||
```xml
|
||||
<PackageVersion Include="ZB.MOM.WW.Secrets.Replicator.AkkaDotNet" Version="0.2.0" />
|
||||
```
|
||||
2. Add a bare reference in the **Host** csproj only (the Host owns the actor system and the secrets registration; driver projects keep referencing only `.Abstractions`):
|
||||
```xml
|
||||
<PackageReference Include="ZB.MOM.WW.Secrets.Replicator.AkkaDotNet" />
|
||||
```
|
||||
3. Restore: `dotnet restore ZB.MOM.WW.OtOpcUa.slnx`
|
||||
**Expected:** resolves from `dohertj2-gitea` via the existing `ZB.MOM.WW.Secrets.*` mapping. It pulls `Akka.Cluster.Tools 1.5.62` — **the exact version OtOpcUa already pins**, so expect no downgrade/conflict. If NU1109 appears, stop: that means the Akka pins have drifted and the plan's premise needs rechecking.
|
||||
4. Build → **Expected:** 0 warnings, nothing consumes the package yet.
|
||||
5. Commit.
|
||||
|
||||
---
|
||||
|
||||
## Task 6 — OtOpcUa: wire Akka replication + shared-KEK config
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none
|
||||
**Depends on:** Task 5
|
||||
|
||||
> **High-risk because** it changes which `ISecretStore` implementation the container resolves on **every** node, including driver-role nodes that have no auth/AdminUI. A mistake here surfaces as drivers failing to open sessions, not as a test failure.
|
||||
|
||||
**Files:**
|
||||
- Modify: `~/Desktop/OtOpcUa/src/Server/ZB.MOM.WW.OtOpcUa.Host/Program.cs` (the existing unconditional `AddZbSecrets` call site added by the G-2 adoption)
|
||||
- Modify: `~/Desktop/OtOpcUa/src/Server/ZB.MOM.WW.OtOpcUa.Host/appsettings.json`
|
||||
- Test: `~/Desktop/OtOpcUa/tests/.../Host.Tests/` — DI-resolution test (see step 4)
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. **Replace** the runtime registration — `AddZbSecretsAkkaReplication` calls `AddZbSecrets` internally, so calling both would double-register:
|
||||
```csharp
|
||||
// was: builder.Services.AddZbSecrets(builder.Configuration, "Secrets");
|
||||
builder.Services.AddZbSecretsAkkaReplication(builder.Configuration, "Secrets", "Secrets:Replication");
|
||||
```
|
||||
Using: `ZB.MOM.WW.Secrets.Replicator.AkkaDotNet.DependencyInjection`.
|
||||
**Leave the pre-host expander untouched** — it builds a throwaway container that must stay a plain local SQLite store. Replication is a *runtime* concern; the bootstrap provider runs before the actor system exists.
|
||||
2. Merge the serializer HOCON into the actor-system config so replication messages use the package's `SerializerWithStringManifest` rather than falling back to a default serializer:
|
||||
```csharp
|
||||
// where the Akka HOCON is assembled:
|
||||
config = config.WithFallback(AkkaSecretsReplication.SerializationConfig);
|
||||
```
|
||||
3. Add the config block to `appsettings.json` beside the existing `Secrets` section:
|
||||
```json
|
||||
"Secrets": {
|
||||
"Replication": { "AnnounceInterval": "00:00:30", "ActorName": "zb-secret-replication" }
|
||||
}
|
||||
```
|
||||
4. **Write a DI-resolution test before wiring** (this is the class of defect that shipped in the library itself — every unit test passed because nothing ever built a container):
|
||||
```csharp
|
||||
[Fact]
|
||||
public void Secrets_store_resolves_as_the_replicating_decorator()
|
||||
{
|
||||
// Build the Host's service collection, then:
|
||||
var store = provider.GetRequiredService<ISecretStore>();
|
||||
Assert.IsType<ReplicatingSecretStore>(store);
|
||||
// And the undecorated concrete store must still resolve — the decorator depends on it:
|
||||
Assert.NotNull(provider.GetRequiredService<SqliteSecretStore>());
|
||||
}
|
||||
```
|
||||
Run it first and confirm it **fails** before step 1's edit, passes after.
|
||||
5. Build + full suite → **Expected:** 0 warnings, green.
|
||||
6. **Verify the same KEK is on every node** before any multi-node run — per §3.1, use the digest check in `clustered-secrets.md` §0. Do not skip: a mismatch is silent until first resolve.
|
||||
7. Commit.
|
||||
|
||||
---
|
||||
|
||||
## Task 7 — ScadaBridge: add the SQL-Server replicator package + hub schema
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** Task 5 (different repo)
|
||||
**Depends on:** Task 4
|
||||
|
||||
**Files:**
|
||||
- Modify: `~/Desktop/ScadaBridge/Directory.Packages.props`
|
||||
- Modify: `~/Desktop/ScadaBridge/src/ZB.MOM.WW.ScadaBridge.Host/ZB.MOM.WW.ScadaBridge.Host.csproj`
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Add CPM row + bare reference for `ZB.MOM.WW.Secrets.Replicator.SqlServer` @ `0.2.0`.
|
||||
2. Restore + build → **Expected:** resolves via the existing mapping; pulls `Microsoft.Data.SqlClient 6.0.2`. ScadaBridge already uses SqlClient — **if NU1109 appears, the repo pins an older SqlClient and you must reconcile the pin before continuing.** Report rather than force.
|
||||
3. **Do not hand-provision the hub schema.** `AddZbSecretsSqlServerReplication` registers `SqlServerHubMigrationHostedService`, which creates it idempotently at startup. The DB and a login with `CREATE TABLE`/`CREATE SCHEMA` rights must exist — see `clustered-secrets.md` for the provisioning SQL.
|
||||
4. Commit.
|
||||
|
||||
---
|
||||
|
||||
## Task 8 — ScadaBridge: wire hub replication + per-node config
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none
|
||||
**Depends on:** Task 7
|
||||
|
||||
> **High-risk because** it introduces a background sweep that writes to a shared central database from every site node, and because a wrong `Secrets:SqlServer:ConnectionString` on a site node fails at startup (eager `Validate()`), taking the node down.
|
||||
|
||||
**Files:**
|
||||
- Modify: `~/Desktop/ScadaBridge/src/ZB.MOM.WW.ScadaBridge.Host/Program.cs` (existing `AddZbSecrets` call site)
|
||||
- Modify: `~/Desktop/ScadaBridge/src/ZB.MOM.WW.ScadaBridge.Host/appsettings.json`
|
||||
- Modify: the per-node docker appsettings under `~/Desktop/ScadaBridge/docker/`
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Replace the runtime registration (it calls `AddZbSecrets` internally — do not call both):
|
||||
```csharp
|
||||
builder.Services.AddZbSecretsSqlServerReplication(
|
||||
builder.Configuration, "Secrets", "Secrets:SqlServer");
|
||||
```
|
||||
2. Config block — the connection string is itself a secret, so deliver it the way every other ScadaBridge connstr is delivered (a `${secret:}` token resolved by the pre-host expander, which runs *before* this registration binds):
|
||||
```json
|
||||
"Secrets": {
|
||||
"SqlServer": {
|
||||
"ConnectionString": "${secret:sql/scadabridge/secrets-hub}",
|
||||
"SchemaName": "zbsecrets",
|
||||
"SyncInterval": "00:00:30",
|
||||
"SyncOnStartup": true
|
||||
}
|
||||
}
|
||||
```
|
||||
**Ordering gotcha:** seed `sql/scadabridge/secrets-hub` in the **local** store on every node first. The hub connstr cannot itself come from the hub — that is a bootstrap cycle. This is the one config value that must stay locally seeded.
|
||||
3. Add the same DI-resolution test as Task 6 step 4, asserting `ReplicatingSecretStore`.
|
||||
4. Build + suite → **Expected:** 0 warnings; green modulo the known AngleSharp break.
|
||||
5. Verify the shared KEK across central + site nodes (§3.1).
|
||||
6. Commit.
|
||||
|
||||
---
|
||||
|
||||
## Task 9 — Live convergence gate (both clustered apps)
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~5 min (excluding rig setup)
|
||||
**Parallelizable with:** none
|
||||
**Depends on:** Task 6, Task 8
|
||||
|
||||
> This is the **G-2-class live gate** the G-7 design called for. The library's offline suite proves the algorithms (2-node in-process Akka cluster, 15 live SQL-Server tests); it cannot prove *this app's* wiring. Do not mark the topology adopted without it.
|
||||
|
||||
**Steps — run per app:**
|
||||
|
||||
1. Stand up 2 nodes (OtOpcUa: docker-dev rig; ScadaBridge: central pair, then a site node against the central hub).
|
||||
2. **Write on node A**, via the `/admin/secrets` UI or the `secret` CLI.
|
||||
3. **Resolve on node B.** Expected: the value appears within one sweep/announce interval (~30 s).
|
||||
4. **Delete on A → confirm the tombstone propagates to B** and B no longer resolves it. (Resurrection is the classic anti-entropy bug; this is the case that catches it.)
|
||||
5. **Partition test** — the reason each topology was chosen:
|
||||
- OtOpcUa: stop node B's cluster link; confirm B still resolves its Layer-B driver secrets from local state; heal; confirm re-convergence.
|
||||
- ScadaBridge: stop the site node's route to the central hub; confirm it still resolves; restore; confirm the sweep re-converges both directions.
|
||||
6. **Negative KEK control** (proves fail-closed rather than silent-wrong): point one node at a *different* KEK, confirm it fails closed with a `kek_id` mismatch rather than returning a wrong plaintext. Revert.
|
||||
7. Record results in [`components/secrets/GAPS.md`](../../components/secrets/GAPS.md) — including anything that did **not** pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 10 — Docs reconcile
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~3 min
|
||||
**Parallelizable with:** none
|
||||
**Depends on:** Task 9
|
||||
|
||||
**Files:**
|
||||
- Modify: `components/secrets/GAPS.md` (close the two Open items: version bump; topology adoption)
|
||||
- Modify: `CLAUDE.md` (Secrets row: consumer pins `0.1.2` → `0.2.0`; record the adopted topology per app)
|
||||
- Modify: `components/secrets/current-state/*/CURRENT-STATE.md` (per-app store/topology)
|
||||
|
||||
**Steps:** State what was **proven live** vs what was only built. If Task 9's partition or KEK-negative steps were skipped for lack of a rig, say so explicitly — an unproven claim in these docs is the exact failure mode [`component-status-claims-are-optimistic`](../../components/secrets/GAPS.md) already warns about.
|
||||
|
||||
---
|
||||
|
||||
## Rollback
|
||||
|
||||
Every task is a version/config change with a clean revert:
|
||||
- Tasks 1–4: revert the version rows; `0.1.2` is still on the feed.
|
||||
- Tasks 5–8: revert the `AddZbSecrets*Replication` call to plain `AddZbSecrets(config, "Secrets")` and drop the package reference. Local SQLite stores are untouched by replication — rows written while replication was on remain readable, since the KEK and schema are unchanged.
|
||||
@@ -0,0 +1,128 @@
|
||||
{
|
||||
"planPath": "docs/plans/2026-07-18-secrets-0.2.0-upgrade-and-clustering.md",
|
||||
"tasks": [
|
||||
{
|
||||
"id": 1,
|
||||
"nativeId": "7",
|
||||
"subject": "Task 1: HistorianGateway bump 0.1.2 -> 0.2.0",
|
||||
"status": "completed",
|
||||
"classification": "trivial",
|
||||
"parallelizableWith": [
|
||||
2,
|
||||
3,
|
||||
4
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"nativeId": "8",
|
||||
"subject": "Task 2: MxAccessGateway bump 0.1.2 -> 0.2.0",
|
||||
"status": "completed",
|
||||
"classification": "trivial",
|
||||
"parallelizableWith": [
|
||||
1,
|
||||
3,
|
||||
4
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"nativeId": "9",
|
||||
"subject": "Task 3: OtOpcUa bump 0.1.2 -> 0.2.0 (CPM)",
|
||||
"status": "completed",
|
||||
"classification": "trivial",
|
||||
"parallelizableWith": [
|
||||
1,
|
||||
2,
|
||||
4
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 4,
|
||||
"nativeId": "10",
|
||||
"subject": "Task 4: ScadaBridge bump 0.1.2 -> 0.2.0 (CPM)",
|
||||
"status": "completed",
|
||||
"classification": "trivial",
|
||||
"parallelizableWith": [
|
||||
1,
|
||||
2,
|
||||
3
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 5,
|
||||
"nativeId": "11",
|
||||
"subject": "Task 5: OtOpcUa add Akka replicator package",
|
||||
"status": "completed",
|
||||
"classification": "small",
|
||||
"blockedBy": [
|
||||
3
|
||||
],
|
||||
"parallelizableWith": [
|
||||
7
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 6,
|
||||
"nativeId": "12",
|
||||
"subject": "Task 6: OtOpcUa wire Akka replication + shared KEK",
|
||||
"status": "completed",
|
||||
"classification": "high-risk",
|
||||
"blockedBy": [
|
||||
5
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 7,
|
||||
"nativeId": "13",
|
||||
"subject": "Task 7: ScadaBridge add SqlServer replicator package",
|
||||
"status": "completed",
|
||||
"classification": "standard",
|
||||
"blockedBy": [
|
||||
4
|
||||
],
|
||||
"parallelizableWith": [
|
||||
5
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 8,
|
||||
"nativeId": "14",
|
||||
"subject": "Task 8: ScadaBridge wire hub replication + per-node config",
|
||||
"status": "completed",
|
||||
"classification": "high-risk",
|
||||
"blockedBy": [
|
||||
7
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 9,
|
||||
"nativeId": "15",
|
||||
"subject": "Task 9: Live convergence gate (both clustered apps)",
|
||||
"status": "pending",
|
||||
"classification": "high-risk",
|
||||
"blockedBy": [
|
||||
6,
|
||||
8
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 10,
|
||||
"nativeId": "16",
|
||||
"subject": "Task 10: Docs reconcile",
|
||||
"status": "pending",
|
||||
"classification": "small",
|
||||
"blockedBy": [
|
||||
9
|
||||
]
|
||||
}
|
||||
],
|
||||
"checkpoints": [
|
||||
{
|
||||
"afterTask": 4,
|
||||
"note": "All four apps on 0.2.0, advisory closed family-wide. Independently shippable \u2014 FF-merge and push each repo before Task 5."
|
||||
}
|
||||
],
|
||||
"lastUpdated": "2026-07-18",
|
||||
"notes": "Tasks 1-8 complete on local feat/secrets-0.2.0 branches in 4 repos (not pushed). Apps ended on Secrets 0.2.1, not 0.2.0: adoption uncovered that 0.2.0's Akka replicator was silently inert (TryAdd order); fixed and republished. Task 9 (live gate) NOT done - needs 2-node rigs. Replication is default-OFF everywhere, so nothing ships enabled."
|
||||
}
|
||||
Reference in New Issue
Block a user