I dismissed this finding from the code review as a false positive, reasoning
that Akka's ActorBase caches Self in a field and that three passing
anti-entropy tests traverse the path. Both premises were wrong. Self resolves
through Context, which is [ThreadStatic], and throws NotSupportedException once
a continuation resumes on a thread-pool thread.
The tests passed because a local SQLite store usually completes await
SYNCHRONOUSLY, so the continuation stayed on the mailbox thread and the context
was still intact. Correctness therefore depended on store latency and
thread-pool timing: green here, broken under a slower or contended store, with
the only symptom a per-peer warning every announce interval while nodes
silently stopped converging. The live-broadcast fast path masked it further —
only the anti-entropy repair path was dead.
Captures self on the actor thread and passes it in. Adds
GenuinelyAsyncSecretStore to force the async path, a regression test that fails
on the unfixed code (20s timeout) and passes in 3s after, and
ActorContextAfterAwaitTests pinning the underlying Akka behaviour so the wrong
assumption cannot be made again. Audited every remaining Self/Sender access in
the actor.
170 pass offline / 184 with the live SQL suite / 1 skip / 0 warnings.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
Secrets were per-node SQLite, so a secret written on one node was invisible to
the rest of a cluster. G-7's design resolved the "shared SQL store vs Akka
replicator" fork to build only the former; both are built here so the choice is
a deployment decision (availability vs partition tolerance) rather than a
library limitation.
Two new packages — ZB.MOM.WW.Secrets.Replicator.SqlServer (shared store, plus a
local-store-with-hub mode) and .Replicator.AkkaDotNet (peer-to-peer over
distributed pub/sub). Core gains ISecretsStoreMigrator, one shared
SecretLastWriterWins predicate so no two stores can disagree on a tie, the
transport-agnostic reconciler, and ReplicatingSecretStore — which closes a real
gap: nothing had ever called ISecretReplicator.PublishAsync, so the seam was
inert and local writes would not have propagated at all.
Verified 182 pass / 1 skip / 0 warnings, including 15 live tests against a real
SQL Server 2022 (the SQLite suite ported case-for-case, so any behavioural
divergence between the stores fails) and a 9-test in-process 2-node Akka
cluster over real remoting. A post-build review caught six defects, all fixed
and now covered: both replication modes could not resolve from the container
(no test had built one), an unbounded fetch that broke past SQL Server's
2100-parameter cap, a poison row that aborted the rest of its batch forever,
Enum.Parse on peer input that could restart the actor in a loop, null crypto
blobs crossing the trust boundary, and a silently dropped pull-read failure.
Packed at 0.2.0 and vulnerability-scanned clean; not yet published to the feed.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
Closes out the security-fix rollout: mxaccessgw, ScadaBridge and HistorianGateway
are all on Auth 0.1.5 and pushed. Exposure was checked per repo rather than assumed --
only mxaccessgw and ScadaBridge actually resolved the vulnerable 2.1.11;
HistorianGateway already resolved 3.50.3, and OtOpcUa takes no ApiKeys/SQLite
dependency at all.
Also records two things worth not rediscovering: HistorianGateway's bump is
unverified by tests, and both it and ScadaBridge have a pre-existing AngleSharp
NU1902 error failing full-solution builds under TreatWarningsAsErrors.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
Packing Secrets 0.1.3 surfaced NU1903: Microsoft.Data.Sqlite 10.0.7 pulls
SQLitePCLRaw.lib.e_sqlite3 2.1.11, which carries high-severity advisory
GHSA-2m69-gcr7-jv3q. Auth.ApiKeys had the same exposure and was already shipped
at 0.1.4 to three consumers, so both libs are fixed rather than only the one
being published.
Fixed the way ZB.MOM.WW.LocalDb already had it: CentralPackageTransitivePinningEnabled
plus a pin to the patched 2.1.12. Bumping Microsoft.Data.Sqlite does not help --
even 10.0.10 still resolves 2.1.11 -- so the pin is the actual fix. The pin reaches
consumers: both nuspecs now declare SQLitePCLRaw.lib.e_sqlite3 >= 2.1.12 as a direct
dependency, verified by restoring the published packages into a scratch project,
which resolves 2.1.12 and scans clean.
Auth goes 0.1.4 -> 0.1.5 with no API change (0.1.4 plus the pin). Suites re-run
after the native-lib swap with identical counts, so no behavioral regression:
Secrets 97 pass/1 skip, Auth 215 pass/1 skip (both skips are Windows-only DPAPI
and opt-in LDAP).
Consumers are NOT yet bumped and remain on vulnerable versions: mxaccessgw 0.1.4,
HistorianGateway 0.1.4, ScadaBridge 0.1.3. Secrets consumers all sit at 0.1.2 and
also lack KEK rotation.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
Publishes the three ZB.MOM.WW.LocalDb packages to the Gitea feed (restore-verified
from a scratch consumer) and adds the build/push.sh the other shared libs already have.
The status prose across CLAUDE.md, README.md and components/*/GAPS.md had drifted from
reality, so it was re-derived from the feed listing and the actual PackageReferences +
registration calls on each consumer's default branch rather than from prior claims.
Five claims were false: Health "not yet adopted" (all four apps wire MapZbHealth),
GalaxyRepository's mxaccessgw adoption "a follow-on" (its Server wires
AddZbGalaxyRepository), Configuration "not yet pushed", Secrets G-8 "not yet
committed", and Theme pinned at 0.2.0 (all four are on 0.3.1). Every doc also said
"three apps" while HistorianGateway is a fourth consumer of seven libs, and all
eight libraries' test counts were stale (re-ran each suite; all green).
Surfaces one previously unrecorded gap: Secrets source is at 0.1.3 with KEK rotation
committed, but the feed tops out at 0.1.2, so no app can consume rotation until it
is published.
Health and observability divergence tables are labelled historical, not re-verified —
the libraries are proven wired, but per-app probe coverage vs spec was not re-walked.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
G-8 (KEK rotation) — built in ZB.MOM.WW.Secrets, lib 0.1.2->0.1.3:
- ISecretCipher.Rewrap(row, oldKek, newKek): re-wraps the per-secret DEK only
(bodies never re-encrypted; revision/timestamps preserved -> invisible to
cluster LWW). Fail-closed on wrong old-KEK id, wrong key bytes, and malformed
wraps; DEK zeroed on all paths.
- ISecretStore.ApplyRewrapAsync(rewrappedRow, expectedCurrentWrappedDek):
updates only the 4 wrap columns + kek_id, compare-and-swap on the current
wrapped DEK so a concurrent set/rotate cannot corrupt a row (closes a
review-caught TOCTOU).
- KekRotationService.RewrapAllAsync + RewrapReport: enumerate all rows incl.
tombstones, idempotent/resumable skip-already-current, bounded CAS-retry,
fail-closed on unknown/identical KEK.
- `secret rewrap-all` CLI verb: key material only via env-var name / file path,
JSON counts report; README section + operator runbook.
Verified: full offline suite green (82 core + 15 UI, 0 regressions) + end-to-end
CLI smoke + adversarial crypto review (all 7 categories PASS; TOCTOU fixed).
G-7 (clustered replication) — designed + planned, no code:
- Fork resolved to build Option A (shared SQL-Server ISecretStore); Akka
replicator ZB.MOM.WW.Secrets.Akka is a deferred phase-2. Design doc +
executable plan + .tasks.json under docs/plans/2026-07-17-secrets-g7-*.
Tracking: components/secrets/GAPS.md + CLAUDE.md secrets row updated.
Record the OtOpcUa live wonder gate (item 2). A throwaway harness drove OtOpcUa's
real GalaxyDriverBrowser (secret: ApiKeySecretRef resolved via ISecretResolver) against
the real production MxGateway: dummy secret: ref -> MxGatewayAuthenticationException,
real metadata:read key -> CONNECTED + browsed the real Galaxy root (10 nodes). Temp key
minted via the dashboard, then revoked (functionally proven) + deleted; no prod key
touched; harness + scratch stores destroyed. All four apps now G-2..G-6 done + live-proven.
Record the OtOpcUa secrets adoption (G-2/G-4/G-5/G-6, the last app): FF-merged +
pushed to lmxopcua origin/master @872cf7e3. Update the components/secrets/GAPS.md
OtOpcUa bullet (execution record + the three verified plan deviations + the regression
the gate caught) and the CLAUDE.md secrets-component-table row. All four apps
(HistorianGateway, mxaccessgw, ScadaBridge, OtOpcUa) now adopt ZB.MOM.WW.Secrets.
The v3.0 dual-namespace rewrite (PR #472) moved anchors and reshaped the
build, so the plan is refreshed to stay executable: CPM correction (OtOpcUa
is on Central Package Management, not inline versions), OpcUaClient options
in .Contracts + GalaxyDriverBrowser in .Browser, DriverHostActor line
shifts, and appsettings/owner precision. Driver-secret flow itself was
untouched by v3.0, so the two-layer approach holds.
Update the OtOpcUa umbrella entry for v3.0: the single .../ns namespace is
replaced by .../raw (s=<RawPath>) + .../uns (s=<Area>/<Line>/<Equipment>/<Eff>),
single-source fan-out to both NodeIds, Organizes UNS->Raw, writes/HistoryRead via
either NodeId, multi-notifier alarms, EquipmentNodeIds retired. Flags the wire-
contract change as a pending ScadaBridge Data-Connection-Layer cutover.
Claude-Session: https://claude.ai/code/session_01LVneM3eh1UtJxEisFXgmox
Add components/secrets/ (SPEC, realized shared-contract, code-verified
current-state for OtOpcUa/mxaccessgw/ScadaBridge, GAPS adoption backlog) and
register the Secrets row in CLAUDE.md + components/README.md. The
ZB.MOM.WW.Secrets lib is built + published 0.1.2 + reference-consumer-proven
(HistorianGateway, live vs the wonder historian); per-app adoption is the
tracked follow-on.
7th shared ZB.MOM.WW.* library: AES-256-GCM envelope-encrypted secret store
(ISecretStore/SQLite), pluggable IMasterKeyProvider, ${secret:name} config
resolution, ISecretResolver runtime cache, Blazor /admin/secrets management UI
on the Theme kit, and a headless 'secret' CLI. Published 0.1.2 to the Gitea feed;
HistorianGateway adopted as reference consumer (live-proven end-to-end against
the real wonder historian). 80 tests pass.
Live HistorianGateway boot caught that the expander scanned a '_secretsComment' appsettings
value documenting the ${secret:...} syntax, tried to resolve the literal example token, and
crashed boot. The family uses _*Comment keys pervasively, so skip any config key whose leaf
segment starts with '_' (the _comment convention). Functional keys never lead with '_'.
Joseph Doherty