From 8d9bcedaf33a48f02149adde9496a8a1f6ba56c5 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Sat, 18 Jul 2026 15:04:17 -0400 Subject: [PATCH] docs(secrets): OtOpcUa on 0.2.2 - Akka gate re-run vs feed packages 6/6 Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts --- CLAUDE.md | 2 +- components/secrets/GAPS.md | 13 +++++++++---- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 9cd328b..d677371 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -148,7 +148,7 @@ each project's **code-verified current state**, and the **gaps** between. See | Config + validation (options / startup validation) | Adopted (lib `0.1.0`, on the feed; **all 4 apps**, pushed — the "local only / not yet pushed" caveat was stale) | Shared `ZB.MOM.WW.Configuration` lib | [`components/configuration/`](components/configuration/) | [`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) | | Audit (event model + writer seam) | Adopted (lib `0.1.0`, on the feed; **all 4 apps**, pushed to origin) | Shared `ZB.MOM.WW.Audit` lib | [`components/audit/`](components/audit/) | [`ZB.MOM.WW.Audit/`](ZB.MOM.WW.Audit/) | | Galaxy Repository (object-hierarchy SQL browse + gRPC service) | **Adopted** (feed `0.1.0` + `0.2.0`; consumed at `0.2.0` by HistorianGateway **and mxaccessgw**, whose Server wires `AddZbGalaxyRepository` — verified 2026-07-18; the prior "mxaccessgw adoption is a follow-on" claim was stale) | Shared `ZB.MOM.WW.GalaxyRepository` lib | _(design in histsdk + design doc 2026-06-23)_ | [`ZB.MOM.WW.GalaxyRepository/`](ZB.MOM.WW.GalaxyRepository/) | -| Secrets (encrypted store + `${secret:}` resolution) | Built (libs **`0.1.3`**, **published to the Gitea feed**; **HistorianGateway adopted + live-proven** 2026-07-16; **mxaccessgw adopted G-4/G-5/G-6 + merged to `origin/main` @ `e088dfa`** 2026-07-16, box-verified fail-closed + encrypt-at-rest; **ScadaBridge adopted G-3/G-4/G-5/G-6 + merged to `origin/main` @ `128f1596`** 2026-07-16, **G-3 live-proven vs the real production MxGateway gateway** `wonder-app-vd03:5120` (secret-ref ApiKey → Connected + browsed real Galaxy); **OtOpcUa adopted G-2/G-4/G-5/G-6 + merged to `origin/master` @ `872cf7e3`** (lmxopcua) 2026-07-16 — the last app, so **all four now adopted**; Layer-B driver secrets (Galaxy API key + OpcUaClient `Password`/`UserCertificatePassword`) resolve fail-closed at driver session-open, `AddZbSecrets` registered unconditionally so driver-only nodes work; **G-2 live-proven vs the real production MxGateway** `wonder-app-vd03:5120` — a `secret:`-ref Galaxy ApiKey resolved through OtOpcUa's real `GalaxyDriverBrowser` → dummy key `MxGatewayAuthenticationException`, real key → browsed the real Galaxy root (10 nodes); temp key minted+revoked+deleted); **G-8 KEK-rotation BUILT (lib bumped `0.1.2`→`0.1.3`, 2026-07-17):** `Rewrap` DEK primitive + CAS-guarded `ApplyRewrapAsync` + `KekRotationService.RewrapAllAsync` + `secret rewrap-all` CLI + operator runbook (`ZB.MOM.WW.Secrets/docs/operations/kek-rotation.md`), full suite green + CLI smoke + adversarial crypto review PASS (TOCTOU closed via compare-and-swap). **G-7 clustered replication BUILT 2026-07-18 (lib `0.2.0`) — BOTH fork options, packaged as two new libs:** **`ZB.MOM.WW.Secrets.Replicator.SqlServer`** (shared SQL-Server `ISecretStore` = the plan's Option A, *plus* a local-store-with-hub mode) and **`ZB.MOM.WW.Secrets.Replicator.AkkaDotNet`** (peer-to-peer over distributed pub/sub = Option B, which the plan had deferred). Core gained `ISecretsStoreMigrator`, `SecretLastWriterWins` (one shared LWW tie-break so stores cannot diverge), `SecretReplicationReconciler`, and `ReplicatingSecretStore` — the last of which fixes a real gap: **nothing had ever called `ISecretReplicator.PublishAsync`**, so the seam was inert. Verified 170 offline / 184 with the live SQL suite / 1 skip / 0 warnings, including **15 live tests vs a real SQL Server 2022** (the SQLite suite ported case-for-case) and a **9-test in-process 2-node Akka cluster** (real remoting; write→peer, delete propagation, both anti-entropy directions, late-joiner catch-up). Hard constraints across all topologies: same KEK on every node, ciphertext-only across trust boundaries, and re-wraps deliberately do NOT replicate (`rewrap-all` runs once per independent store). Operator runbook `ZB.MOM.WW.Secrets/docs/operations/clustered-secrets.md`. **Corrected 2026-07-18:** G-8 + the G-7 plan **ARE committed and pushed** on `main` (`d82d345`) — the "not yet committed/pushed" note was stale. **`0.1.3` PUBLISHED to the feed 2026-07-18** (restore-verified), carrying a **transitive security pin** (`SQLitePCLRaw.lib.e_sqlite3` → `2.1.12`, advisory GHSA-2m69-gcr7-jv3q) that `0.1.2` lacked. **All 4 apps bumped to `0.2.1`** 2026-07-18 (local `feat/secrets-0.2.0` branches, NOT pushed). `0.2.1` fixes a critical defect: `0.2.0`'s Akka replicator was **silently inert** (TryAdd registration order — it published into a no-op sink and never spawned an actor). SQL-Server replication was never affected. Clustered topology is **wired but default-OFF and NOT live-validated** — OtOpcUa on Akka peer-to-peer, ScadaBridge on SQL-hub mode (its central and site clusters are separate Akka clusters, so pub/sub cannot cross them). Separately, a **live** SQLitePCLRaw vulnerability was found and fixed in ScadaBridge, where a NuGetAuditSuppress had been masking vulnerable 2.1.11 in 15 projects. **`0.2.0` PUBLISHED to the feed 2026-07-18** (5 packages, restore-verified from a scratch consumer; merged to `main` @ `15ef1f3`) — no app has adopted a clustered topology yet. **`0.2.2` (2026-07-18, [scadaproj#1](https://gitea.dohertylan.com/dohertj2/scadaproj/issues/1) FIXED): the Akka replicator's hosted-process DI deadlock root-caused** — a circular singleton dependency hidden in factory lambdas (`ISecretStore` decorator → `ISecretReplicator` → actor provider → `ISecretCacheInvalidator` → `DefaultSecretResolver` → `ISecretStore`), which MS.DI's `StackGuard` turns into a silent cross-thread call-site-lock deadlock instead of a stack overflow; NOT `DistributedPubSub`/HOCON (both disproven — the actor was never constructed). Fixed by deferring the one cycle edge (`DeferredSecretCacheInvalidator`, resolves on first eviction); **Akka live gate now 6/6** (was: hang), regression coverage `HostedProcessResolutionTests` (container-built graph + hosted services; red on 0.2.1), suite 180/0/15. Fourth defect class only visible with the DI graph built in a real host. **`0.2.2` PUBLISHED to the feed 2026-07-18** (5 packages, restore-verified from a scratch consumer); `lmxopcua#482` (bump + re-gate + enable) is unblocked | Shared `ZB.MOM.WW.Secrets` lib (5 packages + CLI) | [`components/secrets/`](components/secrets/) | [`ZB.MOM.WW.Secrets/`](ZB.MOM.WW.Secrets/) | +| Secrets (encrypted store + `${secret:}` resolution) | Built (libs **`0.1.3`**, **published to the Gitea feed**; **HistorianGateway adopted + live-proven** 2026-07-16; **mxaccessgw adopted G-4/G-5/G-6 + merged to `origin/main` @ `e088dfa`** 2026-07-16, box-verified fail-closed + encrypt-at-rest; **ScadaBridge adopted G-3/G-4/G-5/G-6 + merged to `origin/main` @ `128f1596`** 2026-07-16, **G-3 live-proven vs the real production MxGateway gateway** `wonder-app-vd03:5120` (secret-ref ApiKey → Connected + browsed real Galaxy); **OtOpcUa adopted G-2/G-4/G-5/G-6 + merged to `origin/master` @ `872cf7e3`** (lmxopcua) 2026-07-16 — the last app, so **all four now adopted**; Layer-B driver secrets (Galaxy API key + OpcUaClient `Password`/`UserCertificatePassword`) resolve fail-closed at driver session-open, `AddZbSecrets` registered unconditionally so driver-only nodes work; **G-2 live-proven vs the real production MxGateway** `wonder-app-vd03:5120` — a `secret:`-ref Galaxy ApiKey resolved through OtOpcUa's real `GalaxyDriverBrowser` → dummy key `MxGatewayAuthenticationException`, real key → browsed the real Galaxy root (10 nodes); temp key minted+revoked+deleted); **G-8 KEK-rotation BUILT (lib bumped `0.1.2`→`0.1.3`, 2026-07-17):** `Rewrap` DEK primitive + CAS-guarded `ApplyRewrapAsync` + `KekRotationService.RewrapAllAsync` + `secret rewrap-all` CLI + operator runbook (`ZB.MOM.WW.Secrets/docs/operations/kek-rotation.md`), full suite green + CLI smoke + adversarial crypto review PASS (TOCTOU closed via compare-and-swap). **G-7 clustered replication BUILT 2026-07-18 (lib `0.2.0`) — BOTH fork options, packaged as two new libs:** **`ZB.MOM.WW.Secrets.Replicator.SqlServer`** (shared SQL-Server `ISecretStore` = the plan's Option A, *plus* a local-store-with-hub mode) and **`ZB.MOM.WW.Secrets.Replicator.AkkaDotNet`** (peer-to-peer over distributed pub/sub = Option B, which the plan had deferred). Core gained `ISecretsStoreMigrator`, `SecretLastWriterWins` (one shared LWW tie-break so stores cannot diverge), `SecretReplicationReconciler`, and `ReplicatingSecretStore` — the last of which fixes a real gap: **nothing had ever called `ISecretReplicator.PublishAsync`**, so the seam was inert. Verified 170 offline / 184 with the live SQL suite / 1 skip / 0 warnings, including **15 live tests vs a real SQL Server 2022** (the SQLite suite ported case-for-case) and a **9-test in-process 2-node Akka cluster** (real remoting; write→peer, delete propagation, both anti-entropy directions, late-joiner catch-up). Hard constraints across all topologies: same KEK on every node, ciphertext-only across trust boundaries, and re-wraps deliberately do NOT replicate (`rewrap-all` runs once per independent store). Operator runbook `ZB.MOM.WW.Secrets/docs/operations/clustered-secrets.md`. **Corrected 2026-07-18:** G-8 + the G-7 plan **ARE committed and pushed** on `main` (`d82d345`) — the "not yet committed/pushed" note was stale. **`0.1.3` PUBLISHED to the feed 2026-07-18** (restore-verified), carrying a **transitive security pin** (`SQLitePCLRaw.lib.e_sqlite3` → `2.1.12`, advisory GHSA-2m69-gcr7-jv3q) that `0.1.2` lacked. **All 4 apps bumped to `0.2.1`** 2026-07-18 (local `feat/secrets-0.2.0` branches, NOT pushed). `0.2.1` fixes a critical defect: `0.2.0`'s Akka replicator was **silently inert** (TryAdd registration order — it published into a no-op sink and never spawned an actor). SQL-Server replication was never affected. Clustered topology is **wired but default-OFF and NOT live-validated** — OtOpcUa on Akka peer-to-peer, ScadaBridge on SQL-hub mode (its central and site clusters are separate Akka clusters, so pub/sub cannot cross them). Separately, a **live** SQLitePCLRaw vulnerability was found and fixed in ScadaBridge, where a NuGetAuditSuppress had been masking vulnerable 2.1.11 in 15 projects. **`0.2.0` PUBLISHED to the feed 2026-07-18** (5 packages, restore-verified from a scratch consumer; merged to `main` @ `15ef1f3`) — no app has adopted a clustered topology yet. **`0.2.2` (2026-07-18, [scadaproj#1](https://gitea.dohertylan.com/dohertj2/scadaproj/issues/1) FIXED): the Akka replicator's hosted-process DI deadlock root-caused** — a circular singleton dependency hidden in factory lambdas (`ISecretStore` decorator → `ISecretReplicator` → actor provider → `ISecretCacheInvalidator` → `DefaultSecretResolver` → `ISecretStore`), which MS.DI's `StackGuard` turns into a silent cross-thread call-site-lock deadlock instead of a stack overflow; NOT `DistributedPubSub`/HOCON (both disproven — the actor was never constructed). Fixed by deferring the one cycle edge (`DeferredSecretCacheInvalidator`, resolves on first eviction); **Akka live gate now 6/6** (was: hang), regression coverage `HostedProcessResolutionTests` (container-built graph + hosted services; red on 0.2.1), suite 180/0/15. Fourth defect class only visible with the DI graph built in a real host. **`0.2.2` PUBLISHED to the feed 2026-07-18** (5 packages, restore-verified from a scratch consumer). **OtOpcUa bumped to `0.2.2` same day (`lmxopcua master @ 2254ae3d`): Akka gate re-run vs the FEED packages 6/6**, plus a new provider-based `The_startup_hook_actually_creates_the_replication_actor` test of OtOpcUa's own wiring (8/8); `lmxopcua#482` stays open only for the per-environment enable decision (`Secrets:Replication:Enabled` remains default-false; same-KEK prerequisite) | Shared `ZB.MOM.WW.Secrets` lib (5 packages + CLI) | [`components/secrets/`](components/secrets/) | [`ZB.MOM.WW.Secrets/`](ZB.MOM.WW.Secrets/) | | LocalDb (embedded cache + 2-node sync) | Built (3 pkgs `0.1.0`, **published to the Gitea feed** 2026-07-18, restore-verified from a scratch consumer; no app adoption yet) | Shared `ZB.MOM.WW.LocalDb` lib | [`docs/plans/2026-07-17-localdb-design.md`](docs/plans/2026-07-17-localdb-design.md) | [`ZB.MOM.WW.LocalDb/`](ZB.MOM.WW.LocalDb/) | The auth component is fully populated: a normalized [`spec`](components/auth/spec/SPEC.md), a diff --git a/components/secrets/GAPS.md b/components/secrets/GAPS.md index dbda5db..10359af 100644 --- a/components/secrets/GAPS.md +++ b/components/secrets/GAPS.md @@ -141,11 +141,16 @@ project 38/38 green. scratch consumer — 5 pinned refs, clean restore + build). Also folds in the deterministic rewrite of the `ActorContextAfterAwaitTests` pin (its old pool-thread formulation asserted a non-guaranteed property and flaked under parallel load). -- ⬜ OtOpcUa may now adopt the Akka topology: bump to `0.2.2`, re-run its gate, - then consider enabling `Secrets:Replication:Enabled`. +- ✅ **OtOpcUa bumped to `0.2.2` (2026-07-18, `lmxopcua master @ 2254ae3d`)** and the **gate re-run + against the published feed packages passes 6/6**. The long-promised + `The_startup_hook_actually_creates_the_replication_actor` test now exists for real: a + provider-based resolve of OtOpcUa's own `AddOtOpcUaSecrets` wiring on a self-joined single-node + cluster, watchdogged at the exact point 0.2.1 deadlocked (`SecretsReplicationRegistrationTests` + 8/8; plain `ActorSystem`, no TestKit — xunit.v3-compatible). **Tracked:** [`scadaproj#1`](https://gitea.dohertylan.com/dohertj2/scadaproj/issues/1) (the defect — - fixed) and [`lmxopcua#482`](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/482) - (the consumer-side tracker: bump the package, re-run the gate, then consider enabling). + fixed, CLOSED) and [`lmxopcua#482`](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/482) + (open only for the per-environment decision to enable `Secrets:Replication:Enabled`, which stays + default-false and requires the same KEK on every node first). - ✅ ScadaBridge's hub topology **is** live-validated and safe to enable (config + a shared KEK). Harness: `scratchpad/livegate/` (two scenarios, `akka` / `sql`; not committed).