Options investigation (Aspire standalone/HealthChecks.UI/Gatus/Homepage/
Grafana vs custom) → decision: custom ZB.MOM.WW.Overview Blazor app in the
family look. Design: anonymous read-only single pane, appsettings registry
of all four apps' instances, cache-first poller on a configurable timer,
configurable staleness timeout, Active/Standby from /health/active, and
per-cluster Akka leader chips (needs Health 0.2.0 optional per-entry data).
Impl plan: phased + code-verified — Phase 0 Health 0.2.0 (writer data field
+ AkkaClusterHealthCheck cluster view), Phase 1 ScadaBridge site-node
MapZbHealth PR (:8084; role-scoped active check), Phase 2 OtOpcUa bump,
Phase 3 the app (:5320), Phase 4 live rig acceptance. Mockup is the visual
reference: real Theme tokens + embedded IBM Plex, all card states incl.
stale + split-brain. Ready to execute in a fresh session.
The #485/#486 fixes changed nothing structural about OtOpcUa — same drivers,
same dual-namespace scheme, same Galaxy flow — so the row is mostly untouched.
What DID change is a fact any OPC UA consumer depends on, and ScadaBridge is
mid-cutover onto exactly this address space: a transient ConfigDb error used to
empty the served address space (a PureRemove of every node, observed live and it
stayed empty), stop every driver and clear every subscription, while still
reporting the deploy Applied. A consumer would have seen all bindings go bad
with nothing upstream admitting a problem. Recorded on the OtOpcUa row.
Also fixes real drift found while checking: the OtOpcUa LocalDb adoption design
was sitting UNTRACKED here and unreferenced by the index, even though the
LocalDb row claims the adoption designs live in this repo. Now committed,
referenced alongside the ScadaBridge one, and its stale header corrected —
it still said Phase 1 was "not yet executed" against lib 0.1.1, when Phase 1
shipped and merged against 0.1.3. Phase 2 designed but not executed, on both
the doc and the row.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
SQL-Server hub topology (ScadaBridge) is live-validated against a real SQL Server
2022 with two nodes: write->resolve via hub, tombstone propagation without
resurrection, bidirectional sweep, partitioned-node local resolution (site
autonomy - the reason hub mode was chosen), and wrong-KEK fail-closed via
SecretDecryptionException.
Akka peer-to-peer topology (OtOpcUa) FAILS the gate: resolving ISecretReplicator
hangs indefinitely in a real clustered process. Reproduced twice on a healthy
2-node cluster; isolated to constructing AkkaSecretReplicator -> ActorRef ->
system.ActorOf, NOT nested DI resolution. The library's own 2-node suite passes,
so the trigger is creation through DI in a hosted process. Root cause not yet
identified.
Impact: OtOpcUa would hang at startup if replication were enabled, since the
startup hook resolves ISecretStore. Harmless today only because the flag defaults
false. Do NOT enable Secrets:Replication:Enabled in OtOpcUa.
The gate did its job - it caught a blocking defect that every offline suite,
including the library's own real-cluster tests, had passed over.
- All 4 apps on 0.2.1 (local branches, unpushed).
- Records the corrected security story: the version bump closed NO advisory (all four
repos already resolved patched 2.1.12); the real live vulnerability was in ScadaBridge,
masked by a NuGetAuditSuppress, and was fixed by separate work.
- Records the upstream 0.2.0 inert-Akka-replicator defect, its root cause (DI extensions
with no container-building test - third instance of that class), and the 0.2.1 fix.
- Marks clustered topology WIRED-but-default-OFF and explicitly NOT live-validated;
Task 9 remains open and the topology is not 'adopted' until it passes.
Secrets were per-node SQLite, so a secret written on one node was invisible to
the rest of a cluster. G-7's design resolved the "shared SQL store vs Akka
replicator" fork to build only the former; both are built here so the choice is
a deployment decision (availability vs partition tolerance) rather than a
library limitation.
Two new packages — ZB.MOM.WW.Secrets.Replicator.SqlServer (shared store, plus a
local-store-with-hub mode) and .Replicator.AkkaDotNet (peer-to-peer over
distributed pub/sub). Core gains ISecretsStoreMigrator, one shared
SecretLastWriterWins predicate so no two stores can disagree on a tie, the
transport-agnostic reconciler, and ReplicatingSecretStore — which closes a real
gap: nothing had ever called ISecretReplicator.PublishAsync, so the seam was
inert and local writes would not have propagated at all.
Verified 182 pass / 1 skip / 0 warnings, including 15 live tests against a real
SQL Server 2022 (the SQLite suite ported case-for-case, so any behavioural
divergence between the stores fails) and a 9-test in-process 2-node Akka
cluster over real remoting. A post-build review caught six defects, all fixed
and now covered: both replication modes could not resolve from the container
(no test had built one), an unbounded fetch that broke past SQL Server's
2100-parameter cap, a poison row that aborted the rest of its batch forever,
Enum.Parse on peer input that could restart the actor in a loop, null crypto
blobs crossing the trust boundary, and a silently dropped pull-read failure.
Packed at 0.2.0 and vulnerability-scanned clean; not yet published to the feed.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
G-8 (KEK rotation) — built in ZB.MOM.WW.Secrets, lib 0.1.2->0.1.3:
- ISecretCipher.Rewrap(row, oldKek, newKek): re-wraps the per-secret DEK only
(bodies never re-encrypted; revision/timestamps preserved -> invisible to
cluster LWW). Fail-closed on wrong old-KEK id, wrong key bytes, and malformed
wraps; DEK zeroed on all paths.
- ISecretStore.ApplyRewrapAsync(rewrappedRow, expectedCurrentWrappedDek):
updates only the 4 wrap columns + kek_id, compare-and-swap on the current
wrapped DEK so a concurrent set/rotate cannot corrupt a row (closes a
review-caught TOCTOU).
- KekRotationService.RewrapAllAsync + RewrapReport: enumerate all rows incl.
tombstones, idempotent/resumable skip-already-current, bounded CAS-retry,
fail-closed on unknown/identical KEK.
- `secret rewrap-all` CLI verb: key material only via env-var name / file path,
JSON counts report; README section + operator runbook.
Verified: full offline suite green (82 core + 15 UI, 0 regressions) + end-to-end
CLI smoke + adversarial crypto review (all 7 categories PASS; TOCTOU fixed).
G-7 (clustered replication) — designed + planned, no code:
- Fork resolved to build Option A (shared SQL-Server ISecretStore); Akka
replicator ZB.MOM.WW.Secrets.Akka is a deferred phase-2. Design doc +
executable plan + .tasks.json under docs/plans/2026-07-17-secrets-g7-*.
Tracking: components/secrets/GAPS.md + CLAUDE.md secrets row updated.
The v3.0 dual-namespace rewrite (PR #472) moved anchors and reshaped the
build, so the plan is refreshed to stay executable: CPM correction (OtOpcUa
is on Central Package Management, not inline versions), OpcUaClient options
in .Contracts + GalaxyDriverBrowser in .Browser, DriverHostActor line
shifts, and appsettings/owner precision. Driver-secret flow itself was
untouched by v3.0, so the two-layer approach holds.
Phase 0 of the shared-GLAuth standardization. config.toml = merged dc=zb,dc=local
directory (15 groups in partitioned 55xx/56xx/57xx families, 14 users incl.
multi-role spanning all groups, serviceaccount search account). compose runs one
glauth/glauth:latest on :3893. README is the deploy/verify runbook. Code-reviewed;
fixed scp -r idempotency in the deploy command (README + plan Task 4).
Approved design: consolidate OtOpcUa, MxAccessGateway, ScadaBridge dev/test auth
onto one shared GLAuth at 10.100.0.35:3893 (dc=zb,dc=local, plaintext). App-neutral
source of truth in scadaproj/infra/glauth/; merged directory with gid families
partitioned 55xx/56xx/57xx + multi-role/admin/serviceaccount; per-app Server
repoints; incremental rollout keeping old glauths until verified.
Joseph Doherty