chore(secrets): adopt ZB.MOM.WW.Secrets 0.6.2 — fail-closed store-path validation

Four-minor jump 0.2.3 → 0.6.2. The release line exists because a deploy
renaming the app directory took the MxGateway credential store with it
(2026-08-09, two silent days): Secrets:SqlitePath must now be absolute AND
outside the content root, rejected at boot rather than relocated. 0.6.2 over
0.6.1 is message-only (the rejection text now teaches the mounted-volume
case and forecloses the override hunt) — the live rig evidence below was
gathered at 0.6.1 and the validator behavior is identical.

Adoption changes beyond the pins:

- appsettings.json drops the relative "otopcua-secrets.db" (now boot-fatal);
  dev gets the library's absolute per-user default, deployments supply their
  own (wonder already pins an absolute out-of-root path in the service env).
- Program.cs's pre-host ${secret:} expander now calls the FOUR-argument
  AddZbSecrets with builder.Environment.ContentRootPath. This call site is
  the actual fix, not the pin: the throwaway container has no
  IHostEnvironment, and without the explicit root the under-content-root
  rule is skipped at exactly the moment the migrator creates the store file
  (the 0.6.0 defect). Do NOT simplify back to the 3-argument overload — it
  compiles and silently reopens the gap. Pinned by a Program.cs source-scan
  test AND a runtime falsification control that RUNS the migration and
  demonstrably creates the store at the rejected path under the 3-arg form.
- docker-dev: Secrets__SqlitePath moves /app/data → /data via a SECOND mount
  of the same per-node volume (same file at the volume root, so seeded
  stores carry over) because /app is the container's content root and the
  new rule correctly rejects it. LocalDb stays on /app/data (secrets-only
  rule).
- SecretsStorePathValidationTests: positive-control probes, artifact-first
  (relative and in-content-root paths must FAIL leaving no file; outside-
  root passes; the no-explicit-root control proves the file assertions can
  fail) — a clean boot is equally consistent with the validator never being
  wired.

Live-verified on the six-node docker-dev rig (at 0.6.1): all nodes boot
under the new validation with stores at /data; an AdminUI-written secret
converged into BOTH central nodes' independent stores; site-a held only its
own pair's secrets (Phase 6 pair-local isolation intact). Replicator wire
behavior unchanged 0.3.0–0.6.x per CHANGELOG; registration-graph pins and
the full suite pass.

Directory.Packages.props

@@ -187,10 +187,10 @@
     <PackageVersion Include="ZB.MOM.WW.Auth.Abstractions" Version="0.1.5" />
     <PackageVersion Include="ZB.MOM.WW.Auth.Ldap" Version="0.1.5" />
     <PackageVersion Include="ZB.MOM.WW.Auth.AspNetCore" Version="0.1.5" />
-    <PackageVersion Include="ZB.MOM.WW.Secrets" Version="0.2.3" />
-    <PackageVersion Include="ZB.MOM.WW.Secrets.Abstractions" Version="0.2.3" />
-    <PackageVersion Include="ZB.MOM.WW.Secrets.Ui" Version="0.2.3" />
-    <PackageVersion Include="ZB.MOM.WW.Secrets.Replicator.AkkaDotNet" Version="0.2.3" />
+    <PackageVersion Include="ZB.MOM.WW.Secrets" Version="0.6.2" />
+    <PackageVersion Include="ZB.MOM.WW.Secrets.Abstractions" Version="0.6.2" />
+    <PackageVersion Include="ZB.MOM.WW.Secrets.Ui" Version="0.6.2" />
+    <PackageVersion Include="ZB.MOM.WW.Secrets.Replicator.AkkaDotNet" Version="0.6.2" />
     <PackageVersion Include="ZB.MOM.WW.Audit" Version="0.1.0" />
     <PackageVersion Include="ZB.MOM.WW.Theme" Version="0.4.1" />
     <PackageVersion Include="ZB.MOM.WW.HistorianGateway.Client" Version="0.3.0" />

docker-dev/docker-compose.yml

@@ -118,16 +118,22 @@ name: otopcua-dev
 # observable quickly during dev exercise. Each node keeps its OWN local SQLite store;
 # replication syncs them.
 #
-# SqlitePath is overridden onto the node's existing per-node `/app/data` volume (the one
-# LocalDb already uses) instead of the appsettings default `otopcua-secrets.db`, which
-# resolves to /app — inside the image's writable container layer. Two things that costs,
-# both of which bit the 2026-08-07 pair-local gate: the store is DESTROYED by any
-# `up -d --build` container recreate (so a seeded row cannot outlive a rig rebuild), and
-# it is unreachable from the host, so seeding or inspecting a node's store meant stopping
-# the container and `docker cp`-ing the file out. On the volume, each node still has a
-# private store (one volume per node — this is NOT shared storage, and replication is
-# still what syncs them), but it survives recreate and a helper container can mount it
-# read/write while the node keeps running.
+# SqlitePath is overridden onto the node's existing per-node volume instead of the
+# in-container default. Two things a container-layer path costs, both of which bit the
+# 2026-08-07 pair-local gate: the store is DESTROYED by any `up -d --build` container
+# recreate (so a seeded row cannot outlive a rig rebuild), and it is unreachable from the
+# host, so seeding or inspecting a node's store meant stopping the container and
+# `docker cp`-ing the file out. On the volume, each node still has a private store (one
+# volume per node — this is NOT shared storage, and replication is still what syncs
+# them), but it survives recreate and a helper container can mount it read/write while
+# the node keeps running.
+#
+# The path is /data (a SECOND mount of the same per-node volume), not the volume's
+# /app/data mount: Secrets 0.6.0 fail-closed validation rejects a store inside the
+# application content root (/app) — the rule that exists because a deploy renaming the
+# app directory took the MxGateway credential store with it (2026-08-09). Same volume,
+# same file at the volume root, different mount point — existing seeded stores carry
+# over untouched. LocalDb deliberately stays on /app/data (the rule is secrets-only).
 #
 # Per-cluster mesh Phase 6: this DPS topic now rides each pair's OWN 2-node mesh, not one
 # shared six-node mesh — so secret replication is PAIR-LOCAL. central-1/central-2 replicate
@@ -138,7 +144,7 @@ name: otopcua-dev
 # was needed for this — Enabled=true + the same KEK on every node is still correct; it just now
 # converges within three independent 2-node groups instead of one six-node group.
 x-secrets-env: &secrets-env
-  Secrets__SqlitePath: "/app/data/otopcua-secrets.db"
+  Secrets__SqlitePath: "/data/otopcua-secrets.db"
   Secrets__Replication__Enabled: "true"
   Secrets__Replication__AnnounceInterval: "00:00:05"
   ZB_SECRETS_MASTER_KEY: "${OTOPCUA_SECRETS_KEK:-ZYGhIX0luS/XsevpCB2W18jYHMcqO6AjM9oXy+T6Zp4=}"
@@ -429,6 +435,7 @@ services:
       - "4840:4840"
     volumes:
       - otopcua-localdb-central-1:/app/data
+      - otopcua-localdb-central-1:/data
 
   central-2:
     <<: *otopcua-host
@@ -538,6 +545,7 @@ services:
       - "4841:4840"
     volumes:
       - otopcua-localdb-central-2:/app/data
+      - otopcua-localdb-central-2:/data
 
   # ── Site A cluster (2-node driver-only) ─────────────────────────────────────
   # Per-cluster mesh Phase 6: site-a-1 and site-a-2 form their OWN 2-node "SITE-A" Akka mesh,
@@ -659,6 +667,7 @@ services:
       - "4842:4840"
     volumes:
       - otopcua-localdb-site-a-1:/app/data
+      - otopcua-localdb-site-a-1:/data
 
   site-a-2:
     <<: *otopcua-host
@@ -753,6 +762,7 @@ services:
       - "4843:4840"
     volumes:
       - otopcua-localdb-site-a-2:/app/data
+      - otopcua-localdb-site-a-2:/data
 
   # ── Site B cluster (2-node driver-only) ─────────────────────────────────────
   # Per-cluster mesh Phase 6: site-b-1 and site-b-2 form their OWN 2-node "SITE-B" Akka mesh,
@@ -824,6 +834,7 @@ services:
       - "4844:4840"
     volumes:
       - otopcua-localdb-site-b-1:/app/data
+      - otopcua-localdb-site-b-1:/data
 
   site-b-2:
     <<: *otopcua-host
@@ -891,6 +902,7 @@ services:
       - "4845:4840"
     volumes:
       - otopcua-localdb-site-b-2:/app/data
+      - otopcua-localdb-site-b-2:/data
 
   traefik:
     image: traefik:v3.1

src/Server/ZB.MOM.WW.OtOpcUa.Host/Program.cs

@@ -100,9 +100,14 @@ if (roleSuffix is not null)
 // it also migrates the secrets SQLite store on startup.
 // ASP0000: DELIBERATE throwaway container — expansion must run before builder.Build(), so it
 // cannot use the app's DI. Fully disposed here; shares no singletons with the host container.
+// The content root is passed EXPLICITLY (Secrets 0.6.1 four-argument overload): this throwaway
+// container has no IHostEnvironment, and without the explicit root the under-content-root store
+// rule is skipped at exactly the moment the migrator creates the store file — a rejected path
+// would get an empty database written to it here before the real host aborts a second later
+// (the 0.6.0 defect HistorianGateway's adoption found). Pinned by a Program.cs source-scan test.
 #pragma warning disable ASP0000
 await using (var secretsProvider = new ServiceCollection()
-    .AddZbSecrets(builder.Configuration, "Secrets")
+    .AddZbSecrets(builder.Configuration, "Secrets", builder.Environment.ContentRootPath)
     .BuildServiceProvider())
 #pragma warning restore ASP0000
 {

src/Server/ZB.MOM.WW.OtOpcUa.Host/appsettings.json

@@ -12,7 +12,6 @@
     }
   },
   "Secrets": {
-    "SqlitePath": "otopcua-secrets.db",
     "MasterKey": {
       "Source": "Environment",
       "EnvVarName": "ZB_SECRETS_MASTER_KEY"

tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/SecretsStorePathValidationTests.cs

@@ -0,0 +1,152 @@
+using Microsoft.Extensions.Configuration;
+using Microsoft.Extensions.DependencyInjection;
+using Microsoft.Extensions.FileProviders;
+using Microsoft.Extensions.Hosting;
+using Microsoft.Extensions.Options;
+using Shouldly;
+using Xunit;
+using ZB.MOM.WW.OtOpcUa.Host.Configuration;
+using ZB.MOM.WW.Secrets;
+using ZB.MOM.WW.Secrets.DependencyInjection;
+using ZB.MOM.WW.Secrets.Sqlite;
+
+namespace ZB.MOM.WW.OtOpcUa.Host.IntegrationTests;
+
+/// <summary>
+///     Positive-control probes of the Secrets 0.6.x fail-closed store-path validation as composed by
+///     THIS host — a clean boot is equally consistent with the validator never being wired, so the
+///     load-bearing assertions here are the rejections (the approach that caught the library's own
+///     0.6.0 pre-host hole during the family adoption). Covers both compositions the host has:
+///     <list type="number">
+///         <item>in-host via <c>AddOtOpcUaSecrets</c> (an <see cref="IHostEnvironment"/> supplies the
+///         content root);</item>
+///         <item>the pre-host <c>${secret:}</c> expander shape from <c>Program.cs</c> — a bare
+///         <see cref="ServiceCollection"/> with NO <see cref="IHostEnvironment"/>, where the content
+///         root must be passed explicitly or the under-content-root rule silently vanishes at exactly
+///         the moment the migrator creates the store file (the 0.6.0 defect).</item>
+///     </list>
+/// </summary>
+public sealed class SecretsStorePathValidationTests : IDisposable
+{
+    private readonly string _contentRoot =
+        Directory.CreateTempSubdirectory("otopcua-secrets-croot-").FullName;
+
+    public void Dispose()
+    {
+        try { Directory.Delete(_contentRoot, recursive: true); } catch { /* best effort */ }
+    }
+
+    private static IConfiguration ConfigWithPath(string sqlitePath) =>
+        new ConfigurationBuilder().AddInMemoryCollection(new Dictionary<string, string?>
+        {
+            ["Secrets:SqlitePath"] = sqlitePath,
+            ["Secrets:MasterKey:Source"] = "Environment",
+            ["Secrets:MasterKey:EnvVarName"] = "ZB_SECRETS_MASTER_KEY",
+            ["Secrets:RunMigrationsOnStartup"] = "false",
+        }).Build();
+
+    private ServiceProvider BuildHostShapedProvider(string sqlitePath)
+    {
+        var services = new ServiceCollection();
+        services.AddLogging();
+        services.AddSingleton<IHostEnvironment>(new FakeHostEnvironment(_contentRoot));
+        services.AddOtOpcUaSecrets(ConfigWithPath(sqlitePath));
+        return services.BuildServiceProvider();
+    }
+
+    [Fact]
+    public void A_relative_store_path_is_rejected()
+    {
+        using var sp = BuildHostShapedProvider("otopcua-secrets.db");
+
+        var ex = Should.Throw<OptionsValidationException>(
+            () => sp.GetRequiredService<IOptions<SecretsOptions>>().Value);
+
+        ex.Message.ShouldContain("absolute", Case.Insensitive,
+            "the rejection must name the rule, or the operator who caused it cannot act on it");
+    }
+
+    [Fact]
+    public void An_absolute_path_inside_the_content_root_is_rejected()
+    {
+        // Absolute does NOT imply safe — this is the MxGateway outage shape: the path that lost
+        // the credential store WAS absolute; it sat in the directory the deploy renames.
+        var inside = Path.Combine(_contentRoot, "data", "otopcua-secrets.db");
+        using var sp = BuildHostShapedProvider(inside);
+
+        var ex = Should.Throw<OptionsValidationException>(
+            () => sp.GetRequiredService<IOptions<SecretsOptions>>().Value);
+
+        ex.Message.ShouldContain("content root", Case.Insensitive);
+        File.Exists(inside).ShouldBeFalse("a rejected path must never gain a store file");
+    }
+
+    [Fact]
+    public void An_absolute_path_outside_the_content_root_passes()
+    {
+        // The control for the two rejections above: proves the failures fail because of the RULES,
+        // not because the whole subsystem is unresolvable in this composition.
+        var outside = Path.Combine(Path.GetTempPath(), $"otopcua-secrets-{Guid.NewGuid():N}.db");
+        using var sp = BuildHostShapedProvider(outside);
+
+        sp.GetRequiredService<IOptions<SecretsOptions>>().Value.SqlitePath.ShouldBe(outside);
+    }
+
+    [Fact]
+    public async Task The_prehost_expander_rejects_an_in_content_root_path_before_creating_the_store()
+    {
+        // Mirrors Program.cs's ${secret:} expansion container byte-for-byte: bare ServiceCollection,
+        // no IHostEnvironment, content root passed EXPLICITLY (the 0.6.1 four-argument overload),
+        // then RUN the migration — resolving the migrator alone never touches the filesystem, so a
+        // resolve-only probe leaves the file assertions vacuous. The rejection must land BEFORE
+        // creation: an empty store with -wal/-shm at the rejected path is the artifact that made
+        // the MxGateway outage read as "the database is there, it's just empty".
+        var inside = Path.Combine(_contentRoot, "data", "otopcua-secrets.db");
+        await using var sp = new ServiceCollection()
+            .AddZbSecrets(ConfigWithPath(inside), "Secrets", _contentRoot)
+            .BuildServiceProvider();
+
+        // Record rather than Should.Throw, so the ARTIFACT is asserted first: a regression that
+        // stops the throw but still writes the store must fail on the file — which is the defect —
+        // not on a missing exception that never mentions the database at the rejected path.
+        var ex = await Record.ExceptionAsync(() =>
+            sp.GetRequiredService<SqliteSecretsStoreMigrator>()
+                .MigrateAsync(TestContext.Current.CancellationToken));
+
+        File.Exists(inside).ShouldBeFalse("a rejected path must not be left holding a store file");
+        File.Exists(inside + "-wal").ShouldBeFalse();
+        File.Exists(inside + "-shm").ShouldBeFalse();
+        ex.ShouldBeOfType<OptionsValidationException>();
+    }
+
+    [Fact]
+    public async Task Without_the_explicit_content_root_the_prehost_hole_reopens()
+    {
+        // Falsification control for the test above, and the reason Program.cs's four-argument call
+        // is load-bearing: the same composition WITHOUT the explicit root has no IHostEnvironment
+        // to consult, the under-content-root rule is skipped, and the migration WRITES the store at
+        // the rejected path. Running the migration (not just resolving) is what makes this a true
+        // control — it proves the file assertions above are capable of failing. If the library ever
+        // closes this differently (making this test fail), the source-scan pin on Program.cs can be
+        // retired with it.
+        var inside = Path.Combine(_contentRoot, "data", "otopcua-secrets.db");
+        await using var sp = new ServiceCollection()
+            .AddZbSecrets(ConfigWithPath(inside), "Secrets")
+            .BuildServiceProvider();
+
+        await sp.GetRequiredService<SqliteSecretsStoreMigrator>()
+            .MigrateAsync(TestContext.Current.CancellationToken);
+
+        File.Exists(inside).ShouldBeTrue(
+            "the 3-argument overload with no IHostEnvironment must create the store at the "
+            + "in-content-root path — the exact hole the explicit content root closes");
+    }
+
+    private sealed class FakeHostEnvironment(string contentRoot) : IHostEnvironment
+    {
+        public string EnvironmentName { get; set; } = "Production";
+        public string ApplicationName { get; set; } = "OtOpcUa.Tests";
+        public string ContentRootPath { get; set; } = contentRoot;
+        public IFileProvider ContentRootFileProvider { get; set; } = new NullFileProvider();
+    }
+}

tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/Configuration/SecretsPreHostContentRootSourceTests.cs

@@ -0,0 +1,48 @@
+using System.Text.RegularExpressions;
+using Shouldly;
+using Xunit;
+
+namespace ZB.MOM.WW.OtOpcUa.Host.Tests.Configuration;
+
+/// <summary>
+///     Pins the one <c>AddZbSecrets</c> call that no executable test can reach: the pre-host
+///     <c>${secret:}</c> expander in <c>Program.cs</c> (top-level statements). That call composes the
+///     secrets subsystem in a throwaway <c>ServiceCollection</c> with no <c>IHostEnvironment</c>, so
+///     it MUST pass the content root explicitly (the Secrets 0.6.1 four-argument overload) — dropped
+///     back to the three-argument form, the under-content-root store rule silently vanishes at
+///     exactly the moment the migrator creates the store file, and a rejected path gains an empty
+///     database before the real host aborts (the 0.6.0 defect; the runtime half of this pin is
+///     <c>SecretsStorePathValidationTests.Without_the_explicit_content_root_the_prehost_hole_reopens</c>
+///     in Host.IntegrationTests).
+/// </summary>
+public sealed class SecretsPreHostContentRootSourceTests
+{
+    [Fact]
+    public void The_prehost_expander_passes_the_content_root_explicitly()
+    {
+        var source = ReadProgramSource();
+
+        // The throwaway container's AddZbSecrets call, allowing for formatting drift but requiring
+        // the third argument to be the builder's content root.
+        Regex.IsMatch(
+                source,
+                @"new\s+ServiceCollection\(\)\s*\.\s*AddZbSecrets\(\s*builder\.Configuration\s*,\s*""Secrets""\s*,\s*builder\.Environment\.ContentRootPath\s*\)",
+                RegexOptions.Singleline)
+            .ShouldBeTrue(
+                "Program.cs's pre-host secrets expander must call the four-argument AddZbSecrets "
+                + "overload with builder.Environment.ContentRootPath — the throwaway container has no "
+                + "IHostEnvironment, and without the explicit root the under-content-root store rule "
+                + "is skipped while the migrator creates the store file");
+    }
+
+    private static string ReadProgramSource()
+    {
+        var dir = new DirectoryInfo(AppContext.BaseDirectory);
+        while (dir is not null && !File.Exists(Path.Combine(dir.FullName, "ZB.MOM.WW.OtOpcUa.slnx")))
+            dir = dir.Parent!;
+        dir.ShouldNotBeNull();
+
+        return File.ReadAllText(Path.Combine(
+            dir.FullName, "src", "Server", "ZB.MOM.WW.OtOpcUa.Host", "Program.cs"));
+    }
+}