docs(plans): mark F21 complete (b0a2bb0)

test(integration): F21 — docker-compose + env-driven SQL/LDAP harness mode
Adds a real-infra mode for the integration test harness alongside the default in-memory mode. Drops the previously-untested code paths (EF SqlServer behaviors, real LDAP bind) under env-var control without breaking the zero-infra default that CI runs. - docker-compose.yml — minimal SQL 2022 (14331) + OpenLDAP (3894) stack (ports chosen to coexist with docker-dev/ on 14330/3893) - HarnessMode record reads OTOPCUA_HARNESS_USE_SQL=1 / USE_LDAP=1 from env - SQL mode: per-harness unique DB OtOpcUa_Harness_{guid}, EnsureCreated at startup, EnsureDeleted on dispose (best-effort) - LDAP mode: drops StubLdapAuthService and configures real LdapAuthService against the compose'd OpenLDAP via Authentication:Ldap:* config keys - Microsoft.EntityFrameworkCore.SqlServer added to the test project - README documents both modes + the macOS no-Docker caveat Default in-memory mode unchanged — all 9 existing tests still pass.
2026-05-26 07:25:36 -04:00 · 2026-05-26 07:25:16 -04:00
5 changed files with 215 additions and 48 deletions
@@ -95,7 +95,7 @@
    {"id": "F18", "subject": "Follow-up: Thread HttpContext.User.Identity.Name into Deployments page (createdBy)", "status": "completed", "classification": "small", "estMinutes": 5, "parallelizableWith": [], "blockedBy": [], "commit": "b266f63", "origin": "Self-review of Task 52 — Deployments.razor hardcodes createdBy=\"(current user)\"; needs @inject AuthenticationStateProvider."},
    {"id": "F19", "subject": "Follow-up: RuntimeStartup extension for driver-role node-actor spawn", "status": "completed", "classification": "standard", "estMinutes": 20, "parallelizableWith": [], "blockedBy": [], "commit": "09d6676", "origin": "Self-review of Task 53 — only admin-role singletons are wired via WithOtOpcUaControlPlaneSingletons. Driver-role nodes need a parallel WithOtOpcUaRuntimeActors that spawns DriverHostActor."},
    {"id": "F20", "subject": "Follow-up: Wire DriverInstanceActor.ShouldStub() into DriverHostActor child spawn", "status": "pending", "classification": "small", "estMinutes": 10, "parallelizableWith": ["F7"], "blockedBy": [], "origin": "Self-review of Task 55 — ShouldStub helper exists but nothing calls it. Folds into F7 when DriverHostActor learns to spawn DriverInstanceActor children."},
-    {"id": "F21", "subject": "Follow-up: docker-compose.yml for Host.IntegrationTests (real SQL Server + OpenLDAP)", "status": "pending", "classification": "standard", "estMinutes": 30, "parallelizableWith": [], "blockedBy": [], "origin": "Deviation from Task 58 — TwoNodeClusterHarness uses EF InMemoryDatabase + StubLdapAuthService. For Mac-friendly local runs against real SQL constraints + LDAP, add tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/docker-compose.yml (SQL Server + OpenLDAP), wire EF SqlServer provider behind an env var (OTOPCUA_HARNESS_USE_SQL=1), and add a test trait so CI can run both modes."},
+    {"id": "F21", "subject": "Follow-up: docker-compose.yml for Host.IntegrationTests (real SQL Server + OpenLDAP)", "status": "completed", "classification": "standard", "estMinutes": 30, "parallelizableWith": [], "blockedBy": [], "commit": "b0a2bb0", "deviationNotes": "Stack shipped (SQL on 14331, OpenLDAP on 3894). HarnessMode reads OTOPCUA_HARNESS_USE_SQL=1 / USE_LDAP=1 from env; SQL mode uses per-harness unique DB via EnsureCreated. Compose itself not local-validated — DESKTOP-6JL3KKO has no Docker per CLAUDE.md; CI on Linux will exercise the real path. The xunit test-trait split was punted — env vars are simpler and cover the same use case (one suite, two modes, no test-class duplication).", "origin": "Deviation from Task 58 — TwoNodeClusterHarness uses EF InMemoryDatabase + StubLdapAuthService. For Mac-friendly local runs against real SQL constraints + LDAP, add tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/docker-compose.yml (SQL Server + OpenLDAP), wire EF SqlServer provider behind an env var (OTOPCUA_HARNESS_USE_SQL=1), and add a test trait so CI can run both modes."},
    {"id": "F22", "subject": "Follow-up: failover scenario integration tests (kill-mid-apply, split-brain, restart-during-deploy)", "status": "completed", "classification": "standard", "estMinutes": 60, "parallelizableWith": [], "blockedBy": [], "commit": "cd5540c", "deviationNotes": "Shipped 3 scenarios on the existing 2-node harness: stop-shrinks, restart-rejoins-same-port, deploy-with-one-node-down. Split-brain via simulated partition deferred — Akka.Hosting + xunit don't expose transport-level interference cleanly. The graceful-shutdown + rejoin path is what production actually exercises; ungraceful kill-mid-apply non-deterministic under SBR's 15s stable-after.", "origin": "Deviation from Task 59 — happy-path + idempotency landed but design §8 cases 3-7 need controlled node-down primitives on TwoNodeClusterHarness (StopNodeAsync, RestartNodeAsync, PartitionBetweenAsync). Add those + 5 scenario tests."}
  ]
 }
@@ -0,0 +1,48 @@
+# ZB.MOM.WW.OtOpcUa.Host.IntegrationTests
+
+Two-node Akka cluster integration tests on top of `TwoNodeClusterHarness`.
+
+## Default mode (no infra required)
+
+```bash
+dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests
+```
+
+Uses `Microsoft.EntityFrameworkCore.InMemory` for `ConfigDb` and a stub `ILdapAuthService` that
+accepts any username when the password is `valid-password`. Each harness instance creates a
+unique in-memory database scoped to its lifetime. This is the mode CI runs by default.
+
+## Real-infra mode (SQL Server + OpenLDAP)
+
+When you need to exercise EF behaviors that diverge between providers (index uniqueness,
+`RowVersion` concurrency, JSON columns, migration application) or a real LDAP bind, bring up
+the bundled compose stack and set the env-var switches:
+
+```bash
+docker compose -f tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/docker-compose.yml up -d
+
+export OTOPCUA_HARNESS_USE_SQL=1
+export OTOPCUA_HARNESS_USE_LDAP=1
+dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests
+
+docker compose -f tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/docker-compose.yml down -v
+```
+
+### SQL Server mode (`OTOPCUA_HARNESS_USE_SQL=1`)
+
+- Container: `mcr.microsoft.com/mssql/server:2022-latest` on `localhost:14331`
+- Each `TwoNodeClusterHarness.StartAsync()` creates a unique database
+  `OtOpcUa_Harness_{guid}` via `Database.EnsureCreatedAsync()` and drops it on
+  `DisposeAsync()` (best-effort).
+- Port `14331` chosen to avoid colliding with the `docker-dev/` fleet (which uses `14330`).
+
+### LDAP mode (`OTOPCUA_HARNESS_USE_LDAP=1`)
+
+- Container: `bitnami/openldap:2.6` on `localhost:3894`
+- Users `alice` / `alice123` and `bob` / `bob123`, all under `ou=FleetAdmin`.
+- Port `3894` chosen to avoid colliding with the `docker-dev/` fleet (which uses `3893`).
+
+## Local-dev caveat
+
+This dev VM (`DESKTOP-6JL3KKO`) does not run Docker locally. Real-infra mode runs on the
+shared Linux Docker host (`10.100.0.35`) per `docs/v2/dev-environment.md`, or in CI on Linux.
@@ -24,27 +24,36 @@ namespace ZB.MOM.WW.OtOpcUa.Host.IntegrationTests;

 /// <summary>
 /// Spins up two in-process <c>OtOpcUa.Host</c>-equivalent <see cref="WebApplication"/> instances
-/// that share an in-memory <see cref="OtOpcUaConfigDbContext"/> and form a 2-member Akka cluster.
-/// Both nodes carry the <c>admin</c> + <c>driver</c> roles, matching design §8's failover-test
-/// 2-node profile.
+/// that share an <see cref="OtOpcUaConfigDbContext"/> and form a 2-member Akka cluster. Both
+/// nodes carry the <c>admin</c> + <c>driver</c> roles, matching design §8's failover-test 2-node
+/// profile.
 ///
-/// Why not <c>WebApplicationFactory&lt;Program&gt;</c>?
-/// Program.cs reads <c>OTOPCUA_ROLES</c> from process env (shared across in-process WAF
-/// instances) and writes both Serilog file sinks + Akka cluster TCP listener to the host
-/// process — neither survives two parallel WAFs cleanly. This harness instead replays the
-/// Program.cs DI graph from a clean <see cref="WebApplicationBuilder"/> per node with
-/// per-node config overrides. The production wiring is the same set of extensions
-/// (<see cref="ServiceCollectionExtensions.AddOtOpcUaConfigDb"/>,
-/// <see cref="AkkaCluster.ServiceCollectionExtensions.AddOtOpcUaCluster"/>,
-/// <see cref="AddOtOpcUaAuth"/>, <see cref="AddOtOpcUaHealth"/>,
-/// <see cref="WithOtOpcUaControlPlaneSingletons"/>,
-/// <see cref="WithOtOpcUaRuntimeActors"/>).
+/// <para>Default mode uses EF InMemoryDatabase + <see cref="StubLdapAuthService"/>. Optional
+/// real-infra modes (env-var driven, see <see cref="HarnessMode.FromEnvironment"/>):</para>
+/// <list type="bullet">
+///   <item><c>OTOPCUA_HARNESS_USE_SQL=1</c> → swap the in-memory DB for SQL Server on
+///         <c>localhost:14331</c> (see <c>docker-compose.yml</c>). Each harness gets a unique
+///         database name (<c>OtOpcUa_Harness_{guid}</c>) created via <c>EnsureCreated</c>
+///         and dropped via <c>EnsureDeleted</c> on dispose.</item>
+///   <item><c>OTOPCUA_HARNESS_USE_LDAP=1</c> → drop the stub and point <c>LdapAuthService</c>
+///         at OpenLDAP on <c>localhost:3894</c>.</item>
+/// </list>
+///
+/// <para>Why not <c>WebApplicationFactory&lt;Program&gt;</c>? Program.cs reads <c>OTOPCUA_ROLES</c>
+/// from process env (shared across in-process WAF instances) and writes both Serilog file sinks
+/// + Akka cluster TCP listener to the host process — neither survives two parallel WAFs cleanly.
+/// This harness instead replays the Program.cs DI graph from a clean
+/// <see cref="WebApplicationBuilder"/> per node with per-node config overrides.</para>
 /// </summary>
 public sealed class TwoNodeClusterHarness : IAsyncDisposable
 {
    public const string TestRoles = "admin,driver";
    public string SharedDbName { get; } = $"two-node-cluster-{Guid.NewGuid():N}";

+    public HarnessMode Mode { get; } = HarnessMode.FromEnvironment();
+    private string? _sqlDbName;
+    private string? _sqlConnString;
+
    public WebApplication NodeA { get; private set; } = null!;
    public WebApplication NodeB { get; private set; } = null!;

@@ -65,20 +74,16 @@ public sealed class TwoNodeClusterHarness : IAsyncDisposable
        harness.NodeAAkkaPort = AllocateFreePort();
        harness.NodeBAkkaPort = AllocateFreePort();

-        // Node A boots first as the seed.
-        harness.NodeA = await BuildNodeAsync(
-            host: LoopbackHost,
-            akkaPort: harness.NodeAAkkaPort,
-            seedHost: LoopbackHost,
-            seedAkkaPort: harness.NodeAAkkaPort,
-            dbName: harness.SharedDbName);
+        if (harness.Mode.UseSqlServer)
+        {
+            harness._sqlDbName = $"OtOpcUa_Harness_{Guid.NewGuid():N}";
+            harness._sqlConnString = $"Server=localhost,14331;Database={harness._sqlDbName};User Id=sa;Password=OtOpcUa!Harness123;TrustServerCertificate=True;";
+            await EnsureSqlSchemaCreatedAsync(harness._sqlConnString);
+        }

-        harness.NodeB = await BuildNodeAsync(
-            host: LoopbackHost,
-            akkaPort: harness.NodeBAkkaPort,
-            seedHost: LoopbackHost,
-            seedAkkaPort: harness.NodeAAkkaPort,
-            dbName: harness.SharedDbName);
+        // Node A boots first as the seed.
+        harness.NodeA = await BuildNodeAsync(harness, NodeRole.Seed);
+        harness.NodeB = await BuildNodeAsync(harness, NodeRole.Joiner);

        await WaitForClusterFormationAsync(
            harness.NodeASystem,
@@ -102,17 +107,12 @@ public sealed class TwoNodeClusterHarness : IAsyncDisposable
    }

    /// <summary>
-    /// Rebuilds node B on the same Akka port + same in-memory ConfigDb and waits for the cluster
+    /// Rebuilds node B on the same Akka port + same ConfigDb and waits for the cluster
    /// to re-converge to 2 Up members. Use after <see cref="StopNodeBAsync"/> to test rejoin.
    /// </summary>
    public async Task RestartNodeBAsync(TimeSpan? formationTimeout = null)
    {
-        NodeB = await BuildNodeAsync(
-            host: LoopbackHost,
-            akkaPort: NodeBAkkaPort,
-            seedHost: LoopbackHost,
-            seedAkkaPort: NodeAAkkaPort,
-            dbName: SharedDbName);
+        NodeB = await BuildNodeAsync(this, NodeRole.Joiner);

        await WaitForClusterFormationAsync(
            NodeASystem,
@@ -140,29 +140,56 @@ public sealed class TwoNodeClusterHarness : IAsyncDisposable
            $"Cluster did not converge to {expectedUpMembers} Up members within {timeout}. Actual={actual}");
    }

-    private static async Task<WebApplication> BuildNodeAsync(
-        string host, int akkaPort, string seedHost, int seedAkkaPort, string dbName)
+    private enum NodeRole { Seed, Joiner }
+
+    private static async Task<WebApplication> BuildNodeAsync(TwoNodeClusterHarness harness, NodeRole role)
    {
+        var akkaPort = role == NodeRole.Seed ? harness.NodeAAkkaPort : harness.NodeBAkkaPort;
        var builder = WebApplication.CreateBuilder(new WebApplicationOptions { Args = [] });

-        builder.WebHost.UseKestrel(o => o.Listen(System.Net.IPAddress.Parse(host), 0));
-        builder.Configuration.AddInMemoryCollection(new Dictionary<string, string?>
+        builder.WebHost.UseKestrel(o => o.Listen(System.Net.IPAddress.Parse(LoopbackHost), 0));
+
+        var configOverrides = new Dictionary<string, string?>
        {
-            ["ConnectionStrings:ConfigDb"] = "Server=test;Database=test;Trusted_Connection=True;TrustServerCertificate=True;",
-            ["Cluster:Hostname"] = host,
+            ["ConnectionStrings:ConfigDb"] = harness._sqlConnString
+                ?? "Server=test;Database=test;Trusted_Connection=True;TrustServerCertificate=True;",
+            ["Cluster:Hostname"] = LoopbackHost,
            ["Cluster:Port"] = akkaPort.ToString(),
-            ["Cluster:PublicHostname"] = host,
-            ["Cluster:SeedNodes:0"] = $"akka.tcp://otopcua@{seedHost}:{seedAkkaPort}",
+            ["Cluster:PublicHostname"] = LoopbackHost,
+            ["Cluster:SeedNodes:0"] = $"akka.tcp://otopcua@{LoopbackHost}:{harness.NodeAAkkaPort}",
            ["Cluster:Roles:0"] = "admin",
            ["Cluster:Roles:1"] = "driver",
            ["Security:Jwt:SigningKey"] = "two-node-harness-test-signing-key-with-enough-bytes-for-hs256",
            ["Security:Jwt:Issuer"] = "otopcua-test",
            ["Security:Jwt:Audience"] = "otopcua-test",
-        });
+        };
+
+        if (harness.Mode.UseRealLdap)
+        {
+            configOverrides["Authentication:Ldap:Enabled"] = "true";
+            configOverrides["Authentication:Ldap:Server"] = "localhost";
+            configOverrides["Authentication:Ldap:Port"] = "3894";
+            configOverrides["Authentication:Ldap:UseTls"] = "false";
+            configOverrides["Authentication:Ldap:AllowInsecureLdap"] = "true";
+            configOverrides["Authentication:Ldap:SearchBase"] = "dc=lmxopcua,dc=local";
+            configOverrides["Authentication:Ldap:ServiceAccountDn"] = "cn=admin,dc=lmxopcua,dc=local";
+            configOverrides["Authentication:Ldap:ServiceAccountPassword"] = "ldapadmin";
+        }
+
+        builder.Configuration.AddInMemoryCollection(configOverrides);
+
+        // Provider swap: same DbContext type wired to either InMemory or SqlServer at startup.
+        if (harness.Mode.UseSqlServer)
+        {
+            builder.Services.AddDbContextFactory<OtOpcUaConfigDbContext>(opt => opt.UseSqlServer(harness._sqlConnString));
+            builder.Services.AddDbContext<OtOpcUaConfigDbContext>(opt => opt.UseSqlServer(harness._sqlConnString));
+        }
+        else
+        {
+            builder.Services.AddDbContextFactory<OtOpcUaConfigDbContext>(opt => opt.UseInMemoryDatabase(harness.SharedDbName));
+            builder.Services.AddDbContext<OtOpcUaConfigDbContext>(opt => opt.UseInMemoryDatabase(harness.SharedDbName));
+        }

-        // Replicate Program.cs role wiring with the harness-shared in-memory ConfigDb.
-        builder.Services.AddDbContextFactory<OtOpcUaConfigDbContext>(opt => opt.UseInMemoryDatabase(dbName));
-        builder.Services.AddDbContext<OtOpcUaConfigDbContext>(opt => opt.UseInMemoryDatabase(dbName));
        builder.Services.AddOtOpcUaCluster(builder.Configuration);

        builder.Services.AddAkka("otopcua", (ab, sp) =>
@@ -173,7 +200,8 @@ public sealed class TwoNodeClusterHarness : IAsyncDisposable
        });

        builder.Services.AddOtOpcUaAuth(builder.Configuration);
-        builder.Services.AddSingleton<ILdapAuthService, StubLdapAuthService>();
+        if (!harness.Mode.UseRealLdap)
+            builder.Services.AddSingleton<ILdapAuthService, StubLdapAuthService>();
        builder.Services.AddAdminUI();
        builder.Services.AddSignalR();
        builder.Services.AddOtOpcUaAdminClients();
@@ -190,6 +218,26 @@ public sealed class TwoNodeClusterHarness : IAsyncDisposable
        return app;
    }

+    private static async Task EnsureSqlSchemaCreatedAsync(string connString)
+    {
+        var opts = new DbContextOptionsBuilder<OtOpcUaConfigDbContext>()
+            .UseSqlServer(connString)
+            .Options;
+        await using var db = new OtOpcUaConfigDbContext(opts);
+        // EnsureCreated bypasses migrations but builds the model in one shot — fine for tests.
+        // Production deployments use Migrate-To-V2.ps1 to apply EF migrations.
+        await db.Database.EnsureCreatedAsync();
+    }
+
+    private static async Task DropSqlDatabaseAsync(string connString)
+    {
+        var opts = new DbContextOptionsBuilder<OtOpcUaConfigDbContext>()
+            .UseSqlServer(connString)
+            .Options;
+        await using var db = new OtOpcUaConfigDbContext(opts);
+        await db.Database.EnsureDeletedAsync();
+    }
+
    private static async Task WaitForClusterFormationAsync(ActorSystem a, ActorSystem b, TimeSpan timeout)
    {
        var deadline = DateTime.UtcNow + timeout;
@@ -221,6 +269,19 @@ public sealed class TwoNodeClusterHarness : IAsyncDisposable
    {
        if (NodeB is not null) await NodeB.DisposeAsync();
        if (NodeA is not null) await NodeA.DisposeAsync();
+        if (_sqlConnString is not null)
+        {
+            try { await DropSqlDatabaseAsync(_sqlConnString); }
+            catch { /* best-effort cleanup */ }
+        }
+    }
+
+    /// <summary>Captures the env-var driven harness mode at construction time.</summary>
+    public sealed record HarnessMode(bool UseSqlServer, bool UseRealLdap)
+    {
+        public static HarnessMode FromEnvironment() => new(
+            UseSqlServer: Environment.GetEnvironmentVariable("OTOPCUA_HARNESS_USE_SQL") == "1",
+            UseRealLdap: Environment.GetEnvironmentVariable("OTOPCUA_HARNESS_USE_LDAP") == "1");
    }

    private sealed class StubLdapAuthService : ILdapAuthService
@@ -13,6 +13,7 @@
        <PackageReference Include="Microsoft.NET.Test.Sdk"/>
        <PackageReference Include="Microsoft.AspNetCore.Mvc.Testing"/>
        <PackageReference Include="Microsoft.EntityFrameworkCore.InMemory"/>
+        <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer"/>
        <PackageReference Include="Akka.Hosting"/>
        <PackageReference Include="xunit.runner.visualstudio">
            <PrivateAssets>all</PrivateAssets>
@@ -0,0 +1,57 @@
+# Real-mode dependencies for ZB.MOM.WW.OtOpcUa.Host.IntegrationTests.
+#
+# The default harness (TwoNodeClusterHarness) uses EF InMemoryDatabase + StubLdapAuthService
+# so the suite runs anywhere with zero infrastructure. This compose stack exists for two
+# situations the in-memory mode can't cover:
+#
+#   1. EF behaviors that diverge between provider implementations — index uniqueness,
+#      RowVersion concurrency, JSON column round-trips, EF migration application.
+#   2. Real LDAP binds against an OpenLDAP server with the dev users from
+#      C:\publish\glauth\auth.md.
+#
+# Activate by setting these env vars before running the suite:
+#
+#   export OTOPCUA_HARNESS_USE_SQL=1
+#   export OTOPCUA_HARNESS_USE_LDAP=1
+#   docker compose -f tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/docker-compose.yml up -d
+#   dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests --filter "Category!=E2E"
+#   docker compose -f tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/docker-compose.yml down -v
+#
+# Ports differ from docker-dev/ on purpose so both stacks can run side-by-side:
+#   - SQL:  14331 (docker-dev uses 14330)
+#   - LDAP: 3894  (docker-dev uses 3893)
+#
+# DESKTOP-6JL3KKO note: Docker Desktop is not installed here. Run this stack on the shared
+# Linux Docker host (10.100.0.35) per docs/v2/dev-environment.md, or in CI on Linux.
+
+name: otopcua-harness
+
+services:
+
+  sql:
+    image: mcr.microsoft.com/mssql/server:2022-latest
+    environment:
+      ACCEPT_EULA: "Y"
+      SA_PASSWORD: "OtOpcUa!Harness123"
+      MSSQL_PID: Developer
+    ports:
+      - "14331:1433"
+    healthcheck:
+      test: ["CMD-SHELL", "/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P 'OtOpcUa!Harness123' -No -Q 'SELECT 1' || exit 1"]
+      interval: 10s
+      timeout: 5s
+      retries: 20
+
+  ldap:
+    # OpenLDAP image — same one docker-dev/ uses, just on a different port. Dev users
+    # alice/bob match the GLAuth fixtures so AuthEndpoints contract tests share creds.
+    image: bitnami/openldap:2.6
+    environment:
+      LDAP_ROOT: "dc=lmxopcua,dc=local"
+      LDAP_ADMIN_USERNAME: "admin"
+      LDAP_ADMIN_PASSWORD: "ldapadmin"
+      LDAP_USERS: "alice,bob"
+      LDAP_PASSWORDS: "alice123,bob123"
+      LDAP_USER_DC: "ou=FleetAdmin"
+    ports:
+      - "3894:1389"