AddZbSecretsAkkaReplication called AddZbSecrets FIRST, which does
TryAddSingleton<ISecretReplicator, NoOpSecretReplicator>(). The package's own
TryAddSingleton<ISecretReplicator> therefore found a descriptor already present
and was silently discarded.
Consequence: ISecretReplicator resolved to the no-op sink, so every write
published into nothing; and because SecretReplicationActor is only spawned as a
side effect of constructing AkkaSecretReplicator, no actor was ever created
either. No exception, no log line - a cluster that reports healthy and silently
never converges. The worst available failure mode for a secrets store.
Fix: register ISecretReplicator BEFORE AddZbSecrets, matching what the SQL-Server
package already did. Found during OtOpcUa adoption (Task 6), which is the first
code that ever built a container around this extension.
Root cause of the gap: the SQL-Server package had a DI test asserting its
replicator type (AddZbSecretsSqlServerTests:58) and the correct order; the Akka
package had neither. Every Akka test exercised the actor, serializer and
reconciler in isolation - none built a container, so nothing could see it. This
is the third instance of the same defect class in this library (the inert
ISecretReplicator seam, the unregistered concrete SqliteSecretStore, and now
this), all of which share one cause: unit tests that never construct the DI graph.
Adds AddZbSecretsAkkaReplicationTests (5 tests) asserting registration at the
ServiceCollection level. Verified to discriminate: with the 0.2.0 order restored,
2 of the 5 fail; with the fix, all 5 pass. They assert descriptors rather than
resolving from a provider on purpose - resolving ISecretReplicator eagerly spawns
the actor, whose PreStart needs DistributedPubSub and therefore a joined cluster,
which would make the test hang rather than fail.
Full suite Release-green: 175 passed, 15 skipped, no new warnings.
I dismissed this finding from the code review as a false positive, reasoning
that Akka's ActorBase caches Self in a field and that three passing
anti-entropy tests traverse the path. Both premises were wrong. Self resolves
through Context, which is [ThreadStatic], and throws NotSupportedException once
a continuation resumes on a thread-pool thread.
The tests passed because a local SQLite store usually completes await
SYNCHRONOUSLY, so the continuation stayed on the mailbox thread and the context
was still intact. Correctness therefore depended on store latency and
thread-pool timing: green here, broken under a slower or contended store, with
the only symptom a per-peer warning every announce interval while nodes
silently stopped converging. The live-broadcast fast path masked it further —
only the anti-entropy repair path was dead.
Captures self on the actor thread and passes it in. Adds
GenuinelyAsyncSecretStore to force the async path, a regression test that fails
on the unfixed code (20s timeout) and passes in 3s after, and
ActorContextAfterAwaitTests pinning the underlying Akka behaviour so the wrong
assumption cannot be made again. Audited every remaining Self/Sender access in
the actor.
170 pass offline / 184 with the live SQL suite / 1 skip / 0 warnings.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
Secrets were per-node SQLite, so a secret written on one node was invisible to
the rest of a cluster. G-7's design resolved the "shared SQL store vs Akka
replicator" fork to build only the former; both are built here so the choice is
a deployment decision (availability vs partition tolerance) rather than a
library limitation.
Two new packages — ZB.MOM.WW.Secrets.Replicator.SqlServer (shared store, plus a
local-store-with-hub mode) and .Replicator.AkkaDotNet (peer-to-peer over
distributed pub/sub). Core gains ISecretsStoreMigrator, one shared
SecretLastWriterWins predicate so no two stores can disagree on a tie, the
transport-agnostic reconciler, and ReplicatingSecretStore — which closes a real
gap: nothing had ever called ISecretReplicator.PublishAsync, so the seam was
inert and local writes would not have propagated at all.
Verified 182 pass / 1 skip / 0 warnings, including 15 live tests against a real
SQL Server 2022 (the SQLite suite ported case-for-case, so any behavioural
divergence between the stores fails) and a 9-test in-process 2-node Akka
cluster over real remoting. A post-build review caught six defects, all fixed
and now covered: both replication modes could not resolve from the container
(no test had built one), an unbounded fetch that broke past SQL Server's
2100-parameter cap, a poison row that aborted the rest of its batch forever,
Enum.Parse on peer input that could restart the actor in a loop, null crypto
blobs crossing the trust boundary, and a silently dropped pull-read failure.
Packed at 0.2.0 and vulnerability-scanned clean; not yet published to the feed.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
Joseph Doherty