feat(secrets): opt-in Akka cluster secret replication (default OFF; upstream blocker documented)

Routes the host's secrets registration through a new AddOtOpcUaSecrets extension
that gates the ISecretStore implementation on Secrets:Replication:Enabled.

Opt-in gate (default FALSE)
  This call decides which ISecretStore every node resolves — including driver-role
  nodes with no auth/AdminUI, where a wrong store surfaces as drivers failing to
  open sessions rather than as a failing test. With the flag false the wiring is
  the pre-existing AddZbSecrets(config, "Secrets") call, unchanged, so current
  behavior is byte-identical. With it true, AddZbSecretsAkkaReplication replaces
  that call (it invokes AddZbSecrets internally; calling both would double-register).

  Extracted to a named extension specifically so the registration is testable:
  Program.cs is top-level statements and cannot be exercised by a container test,
  which is how a "registered but never resolvable" defect ships unnoticed.

Serializer HOCON
  AkkaSecretsReplication.SerializationConfig is merged into the ActorSystem config
  inside the AddAkka configurator, conditionally on the same gate — a non-replicating
  node carries no bindings for messages it will never see. Merged via
  AddHocon(..., HoconAddMode.Append), Akka.Hosting's fallback merge and the same mode
  the existing base-config merge uses; a raw Config.WithFallback would fight the
  builder's own assembly.

Lazy-actor mitigation
  The replication actor is created lazily on first ISecretStore resolution, so a node
  that never touches a secret would never announce a manifest and would silently never
  converge. SecretReplicationStarter (IHostedService) resolves the store once at
  startup to make participation unconditional.

KNOWN BLOCKER — replication is currently NON-FUNCTIONAL; do not enable
  ZB.MOM.WW.Secrets.Replicator.AkkaDotNet 0.2.0 never binds its own ISecretReplicator.
  AddZbSecretsAkkaReplication calls AddZbSecrets FIRST, which does
  TryAddSingleton<ISecretReplicator, NoOpSecretReplicator>(); the package's own
  TryAddSingleton<ISecretReplicator>(AkkaSecretReplicator) that follows is therefore
  a no-op. Verified empirically in a built container: with Enabled=true,
  ISecretReplicator resolves to NoOpSecretReplicator, so ReplicatingSecretStore
  publishes into a sink and no actor is ever spawned.

  Consequence: the startup hook cannot create the actor, and the test asserting it
  does is committed Skipped with the evidence. Not worked around here — the fix
  belongs upstream (AddSingleton, or register before calling AddZbSecrets).
  Because the flag defaults false, this commit is inert in production.

Tests: SecretsReplicationRegistrationTests (new) — disabled path resolves plain
SqliteSecretStore and needs no ActorSystem; enabled path resolves
ReplicatingSecretStore AND the undecorated concrete SqliteSecretStore the decorator
is built from (the exact registration gap that shipped once); startup hook registered
only when enabled. Red before wiring (4 assertion failures), green after: 6 pass,
1 skipped (blocker above).

Build: 861 warnings / 0 errors, unchanged from baseline (full --no-incremental A/B).
Host.IntegrationTests: 123 pass, 6 skip, 1 fail — AbCip_Green_AgainstSim, verified
pre-existing on the stashed tree (fixture-gated).

Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
This commit is contained in:
Joseph Doherty
2026-07-18 11:15:03 -04:00
parent e27c19c49d
commit 3336ec08c7
5 changed files with 323 additions and 2 deletions
+11 -1
View File
@@ -41,6 +41,7 @@ using Microsoft.Extensions.DependencyInjection;
using ZB.MOM.WW.Secrets.Abstractions;
using ZB.MOM.WW.Secrets.Configuration;
using ZB.MOM.WW.Secrets.DependencyInjection;
using ZB.MOM.WW.Secrets.Replicator.AkkaDotNet;
using ZB.MOM.WW.Secrets.Sqlite;
// Roles drive the entire conditional wiring below — see ZB.MOM.WW.OtOpcUa.Cluster.RoleParser.
@@ -313,6 +314,14 @@ if (hasDriver)
builder.Services.AddAkka("otopcua", (ab, sp) =>
{
ab.WithOtOpcUaClusterBootstrap(sp);
// Secret-replication wire protocol → its own serializer, merged only when replication is on so a
// non-replicating node carries no serializer bindings for messages it will never see. Appended
// (Akka.Hosting's fallback merge — the same mode WithOtOpcUaClusterBootstrap uses for the base
// config) rather than a raw Config.WithFallback, which would fight the builder's own assembly.
// Without this the protocol DTOs would round-trip on Akka's default JSON serializer, leaving the
// serializer that carries secret ciphertext an inherited default rather than an explicit choice.
if (SecretsRegistration.IsReplicationEnabled(builder.Configuration))
ab.AddHocon(AkkaSecretsReplication.SerializationConfig, HoconAddMode.Append);
if (hasAdmin)
{
ab.WithOtOpcUaControlPlaneSingletons();
@@ -350,7 +359,8 @@ if (hasAdmin)
}
// Registered unconditionally: driver-role nodes resolve Layer-B DriverConfig secrets and have no auth/DP/AdminUI.
builder.Services.AddZbSecrets(builder.Configuration, "Secrets");
// Cluster replication is opt-in behind Secrets:Replication:Enabled (default false) — see SecretsRegistration.
builder.Services.AddOtOpcUaSecrets(builder.Configuration);
builder.Services.AddOtOpcUaHealth();
builder.Services.AddOtOpcUaObservability(builder.Configuration);