docs(claude): record the two redundancy corrections in the project instructions

Both change how someone should reason about this area, and the second changes a public message contract, so they belong in CLAUDE.md rather than only in docs/Redundancy.md. Includes the rule the work established: assert the effective cluster config off a running ActorSystem, never the typed options or the akka.conf text, because several HOCON fragments compete and the losing one still reads correctly in the file. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
fix(deps): pin System.Security.Cryptography.Xml 10.0.10 — fresh restores were red
2026-07-21 19:05:32 -04:00 · 2026-07-21 18:59:48 -04:00 · 2026-07-21 18:46:10 -04:00 · 2026-07-21 18:42:34 -04:00 · 2026-07-21 18:39:07 -04:00 · 2026-07-21 18:32:42 -04:00
29 changed files with 1055 additions and 176 deletions
@@ -219,6 +219,15 @@ The server supports configurable OPC UA transport security via the `OpcUa:Enable

 The server supports non-transparent warm/hot redundancy via the `Redundancy` section in `appsettings.json`. Two instances share the same Galaxy DB and the same mxaccessgw (under distinct `MxAccess.ClientName` values) but have unique `ApplicationUri` values. Each exposes `RedundancySupport`, `ServerUriArray`, and a dynamic `ServiceLevel` based on role and runtime health. The primary advertises a higher ServiceLevel than the secondary. See `docs/Redundancy.md` for the full guide.

+**Two corrections landed 2026-07-21 — both were total-outage or wrong-node defects, and both are worth knowing before touching this area.**
+
+- **Downing is `auto-down`, not `keep-oldest`.** `Cluster:SplitBrainResolverStrategy` (default `auto-down`) selects the provider via `ServiceCollectionExtensions.BuildDowningHocon`; an unknown value fails host start. A two-node pair running the previous `keep-oldest` + `down-if-alone` **could not survive a crash of the oldest node**: Akka's `KeepOldest.OldestDecision` only lets `down-if-alone` rescue a side holding ≥ 2 members, so the 1-vs-1 survivor downs *itself* and exits. `down-if-alone` is a 3+-node feature. The accepted trade for `auto-down` is dual-active during a genuine network partition. **Its live gate is still open** — the pathology only appears 1-vs-1 and docker-dev is a single six-node mesh, so the crash-the-oldest drill is deferred to the per-cluster mesh work.
+- **The Primary is the oldest Up `driver` member, not the role leader.** `RedundancyStateActor` previously used `ClusterState.RoleLeader("driver")` (lowest *address*), while `ClusterSingletonManager` places singletons on the *oldest*. They agree only on a freshly-formed cluster and diverge after any restart, so the Primary-gated data plane (writes, alarm acks, alerts emit, alarm-history drain) could enable on a node not hosting the singletons. `NodeRedundancyState.IsRoleLeaderForDriver` was renamed **`IsDriverPrimary`**.
+
+Assert the **effective** cluster config off a running `ActorSystem` (as `SplitBrainResolverActivationTests` does), never the typed options or the akka.conf text — several HOCON fragments compete and the losing one still reads correctly in the file.
+
+**Still per-Akka-cluster, not per application `Cluster`.** One Primary is elected across the whole Akka mesh, so a fleet running several application clusters in one mesh (what docker-dev models) gets one Primary among all driver nodes. The fix is the per-cluster mesh workstream: `docs/plans/2026-07-21-per-cluster-mesh-design.md` (Phases 0a/0b done; 1–7 not started, Phase 1 planned in `…-phase1.md`).
+
 ## LocalDb pair-local store (Phases 1 + 2)

 Every **driver-role** node keeps a consolidated `ZB.MOM.WW.LocalDb` SQLite database (the retired
@@ -0,0 +1,12 @@
+# MSBuild auto-response file, applied to every build run from this repo root.
+#
+# Disable the MSBuild node pool. By default MSBuild leaves worker nodes resident
+# after a build so the next one starts warm. Across a session of repeated
+# solution-wide builds and test runs this repo accumulated 55 idle nodes holding
+# ~6 GB, which is not merely untidy: the integration suites assert on timeouts,
+# and memory pressure turns those into failures indistinguishable from real
+# regressions (see the Workstation-GC note in Host.IntegrationTests.csproj for
+# the same failure mode from a different cause).
+#
+# The cost is a small startup penalty per build. That is a good trade here.
+-nodeReuse:false
@@ -104,6 +104,23 @@
      clearing the advisory without CentralPackageTransitivePinningEnabled (which breaks the Roslyn split).
    -->
    <PackageVersion Include="SQLitePCLRaw.bundle_e_sqlite3" Version="2.1.12" />
+    <!--
+      Surgical transitive pin #2. Four NU1903 high-severity advisories (GHSA-23rf-6693-g89p,
+      GHSA-8q5v-6pqq-x66h, GHSA-cvvh-rhrc-wg4q, GHSA-g8r8-53c2-pm3f) landed in the NuGet audit
+      data against System.Security.Cryptography.Xml 10.0.7, which Microsoft.AspNetCore.DataProtection
+      10.0.7 pulls in transitively for its key storage. Under TreatWarningsAsErrors every FRESH
+      restore goes red — 204 NU1903 errors across the solution — while machines with cached audit
+      data keep building, so this surfaces first on a clean clone or a docker image build, not
+      locally. Verified 2026-07-21 by restoring a detached worktree.
+
+      10.0.10 is the patched version, pinned by direct PackageReference in Core.Configuration (the
+      one project where the DataProtection chain enters). Bumping the DataProtection parent to
+      10.0.10 instead was rejected in the sister repo for a reason that applies here too: it floors
+      Microsoft.Extensions.* and, through the EFCore adapter, Microsoft.EntityFrameworkCore at
+      10.0.10, forcing a family-wide servicing bump (NU1605 downgrade cascade) that deserves its own
+      reviewed change.
+    -->
+    <PackageVersion Include="System.Security.Cryptography.Xml" Version="10.0.10" />
    <PackageVersion Include="System.CommandLine" Version="2.0.5" />
    <PackageVersion Include="System.Data.SqlClient" Version="4.9.0" />
    <PackageVersion Include="System.IdentityModel.Tokens.Jwt" Version="8.11.0" />
@@ -6,7 +6,7 @@ OtOpcUa supports OPC UA **non-transparent** warm/hot redundancy. Two or more `Ot

 > **Discovery surface.** The `ServerArray` path on the `Server` object is what each node populates with self + peer `ApplicationUri`s — see `OpcUaApplicationHost.PopulateServerArray` and the per-node `PeerApplicationUris` option below. The redundancy-object-type `ServerUriArray` proper (a child of `Server.ServerRedundancy`) remains deferred pending an SDK object-type upgrade; clients should read `Server.ServerArray` for peer discovery today.

-> **v2 change.** v1's operator-managed `ClusterNode.RedundancyRole` column + `RedundancyCoordinator` / `ApplyLeaseRegistry` / `PeerHttpProbeLoop` are gone. Primary/secondary is now derived from **Akka cluster role-leader** for the `driver` role. The operator no longer writes a role into the DB; cluster topology (specifically the `driver` role-leader) drives ServiceLevel automatically.
+> **v2 change.** v1's operator-managed `ClusterNode.RedundancyRole` column + `RedundancyCoordinator` / `ApplyLeaseRegistry` / `PeerHttpProbeLoop` are gone. Primary/secondary is now derived from **Akka cluster membership** — the oldest Up member carrying the `driver` role. The operator no longer writes a role into the DB; cluster topology drives ServiceLevel automatically.

 The runtime pieces live in:

@@ -19,7 +19,7 @@ The runtime pieces live in:
 | `DbHealthProbeActor` | `OtOpcUa.Runtime.Health` | Per-node; runs `SELECT 1` against ConfigDb every 5s. Read by the health endpoint AND by `OpcUaPublishActor` (the `DbReachable` ServiceLevel input). |
 | `PeerProbeSupervisor` | `OtOpcUa.Runtime.Health` | Per-node; subscribes to the `redundancy-state` topic and maintains one `PeerOpcUaProbeActor` child per OTHER driver-role peer (spawn on join, stop on departure), so every node is continuously probed by its peers. |
 | `PeerOpcUaProbeActor` | `OtOpcUa.Runtime.Health` | Spawned by `PeerProbeSupervisor`; pings a peer `opc.tcp://peer:4840` with a TCP connect (2s timeout) and publishes `OpcUaProbeResult` on the `redundancy-state` topic. A full secure-channel Hello handshake is a possible future upgrade; the TCP connect is the current real probe. |
-| `ClusterRoleInfo` | `OtOpcUa.Cluster` | Live view of cluster membership + role-leader; exposes `IClusterRoleInfo` to the rest of the host. |
+| `ClusterRoleInfo` | `OtOpcUa.Cluster` | Live view of cluster membership + role-leader; exposes `IClusterRoleInfo` to the rest of the host. (Akka's role-leader is *not* what elects the redundancy Primary — see below.) |

 ## ServiceLevel tiers

@@ -38,7 +38,7 @@ The four inputs are sourced locally per driver node:
 | `DbReachable` | Local `DbHealthProbeActor` — `OpcUaPublishActor` Asks it on each `HealthTick`; an Ask timeout is treated as `Reachable=false`. |
 | `OpcUaProbeOk` | Result of a peer probing THIS node's OPC UA endpoint: `PeerProbeSupervisor` spawns one `PeerOpcUaProbeActor` per OTHER driver-role peer; each probe publishes `OpcUaProbeResult(probed-node, ok)` on the `redundancy-state` topic; the publish actor consumes only results whose target is itself. Freshness-debounced: absent or stale (>30 s) → `true` (benefit of the doubt — single-node clusters and a departed peer never demote); only an actively-observed RECENT `false` demotes. |
 | `Stale` (derived) | `!DbReachable \|\| (now − lastDbHealth.AsOfUtc) > 30 s \|\| (now − snapshotEntry.AsOfUtc) > 30 s`. |
-| `IsDriverRoleLeader` | The local node's entry in the `RedundancyStateChanged` snapshot from `RedundancyStateActor`. |
+| `IsDriverPrimary` | The local node's entry in the `RedundancyStateChanged` snapshot from `RedundancyStateActor`. |

 The resulting truth table (all tiers are now reachable at runtime):

@@ -47,8 +47,8 @@ The resulting truth table (all tiers are now reachable at runtime):
 | Down / Detached | 0 | Member status is not `Up` or `Joining` (leaving, removed, exiting), OR node has no `driver` role (Detached). Published immediately — a starting or detached node never leaves the SDK default 255. |
 | Critically degraded | 100 | ConfigDb unreachable AND data is stale. |
 | Stale | 200 | Data stale but ConfigDb reachable. |
-| Healthy follower | 240 | DB reachable + OPC UA probe ok + not stale + not role-leader. |
-| Healthy leader | 250 | Same as healthy follower + this node is the `driver` role-leader (+10 bonus). |
+| Healthy follower | 240 | DB reachable + OPC UA probe ok + not stale + not the driver Primary. |
+| Healthy primary | 250 | Same as healthy follower + this node is the driver Primary (+10 bonus). |

 > **Secondary 100 → 240 (behavior change).** Previously a healthy Secondary
 > published 100 (coarse role-only mapping). It now publishes **240** — both
@@ -66,11 +66,23 @@ ServiceLevel (even 0) is always published so no node lingers at the SDK default
 255.

 Roles come from `RedundancyStateActor.BuildSnapshot`: a node with the `driver`
-role is `Primary` when it holds the `driver` role-leader lease, otherwise
+role is `Primary` when it is the **oldest Up member carrying the `driver` role**, otherwise
 `Secondary`; a node without the `driver` role is `Detached`.

+**Oldest, not role leader (changed 2026-07-21).** The election previously used
+`ClusterState.RoleLeader("driver")` — the *lowest-addressed* Up member with the role. Akka offers
+both, and they name different nodes: role leader is ordered by address (host, then port), oldest by
+up-number. They agree on a freshly-formed cluster, and diverge after any restart — a restarted node
+re-joins as the *youngest* while keeping its address, so if it holds the lower address it becomes
+role leader. Meanwhile `ClusterSingletonManager` places singletons on the **oldest**, so the old
+derivation could name a Primary that was not hosting the singletons, enabling the Primary-gated data
+plane (writes, alarm acks, the alerts emit, the alarm-history drain) on the wrong node. Pinned by
+`RedundancyPrimaryElectionTests`, which forms a real two-node cluster where the oldest node holds the
+*higher* address so the two orderings genuinely disagree. `NodeRedundancyState.IsRoleLeaderForDriver`
+was renamed `IsDriverPrimary` to stop the name asserting the old derivation.
+
 > **KNOWN LIMITATION — the election is per *Akka* cluster, not per application `Cluster`.**
-> `RoleLeader("driver")` yields exactly **one** Primary across the whole Akka cluster. A fleet that
+> The election yields exactly **one** Primary across the whole Akka cluster. A fleet that
 > runs several application clusters (each with its own redundant pair) inside one Akka cluster —
 > which is what `docker-dev/docker-compose.yml` models, with MAIN, SITE-A and SITE-B — therefore has
 > one Primary among *all* its driver nodes, not one per pair. Every consumer of the role inherits
@@ -149,7 +161,7 @@ OTOPCUA_ROLES=admin,driver

 Both nodes share the same `ConfigDb` connection string; `Cluster.PublicHostname` + `Roles` are what makes them distinct in cluster gossip. The first node bootstraps the cluster (its address goes in `SeedNodes`); the second node joins via the same `SeedNodes` list.

-There is no longer a `Node:NodeId` setting and no `ClusterNode.RedundancyRole` column (the V2 migration dropped it — primary/secondary is now derived from cluster role-leadership). NodeId is derived as `host:port` of the cluster `PublicHostname` (see `ClusterRoleInfo.LocalNode` for the formula).
+There is no longer a `Node:NodeId` setting and no `ClusterNode.RedundancyRole` column (the V2 migration dropped it — primary/secondary is now derived from cluster membership age). NodeId is derived as `host:port` of the cluster `PublicHostname` (see `ClusterRoleInfo.LocalNode` for the formula).

 > **`RedundancyStateActor` NodeId consistency (fixed).** `RedundancyStateActor` now keys each node's `NodeRedundancyState` entry by the canonical `host:port` node id (via a `ToNodeId(Address)` helper mirroring `ClusterRoleInfo.ToNodeId`). Previously it keyed by `member.Address.Host` (host-only, e.g. `central-2`); since every subscriber matches by the canonical `host:port` form, the mismatch silently meant no node ever matched its own entry — all nodes stayed at the default ServiceLevel 255 and never learned their role. This fix makes `RedundancyStateActor` consistent with the stated contract above. Additionally, `RedundancyStateActor` now **re-publishes the current snapshot on a periodic heartbeat (default 10 s)** so any node that subscribes after the last topology-change publish converges within the interval (DistributedPubSub does not replay to late subscribers).

@@ -169,55 +181,94 @@ Each node advertises its partner via `OpcUaApplicationHostOptions.PeerApplicatio

 Node A lists Node B's `ApplicationUri` and vice-versa. Validated by `DualEndpointTests` in `tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/` — boots two `OpcUaApplicationHost` instances on loopback, asserts a real OPCFoundation client `Session` reading `Server.ServerArray` from Node A sees both URIs.

-## Split-brain
+## Split-brain / downing

-The split-brain resolver is **active by default**: Akka.Cluster.Hosting's `WithClustering` enables an SBR
-downing provider whenever `ClusterOptions.SplitBrainResolver` is null (it applies
-`SplitBrainResolverOption.Default`, which registers `Akka.Cluster.SBR.SplitBrainResolverProvider`), and that
-provider reads the `split-brain-resolver` HOCON block in `akka.conf`. On top of that,
-`ServiceCollectionExtensions.BuildClusterOptions` sets the typed `ClusterOptions.SplitBrainResolver`
-(`KeepOldestOption { DownIfAlone = true }`) to make the strategy **explicit in code** rather than relying on
-the framework default — it is reinforcing, not the sole activator, and yields the same effective behavior. So
-the cluster is **not** running `NoDowning`; hard-crashed nodes are downed and the cluster recovers (how it
-recovers depends on *which* node was lost — see the table below). (Only an *explicit* `NoDowning`, e.g.
-`akka.cluster.downing-provider-class = ""`, would leave both redundancy sides at ServiceLevel 240
-indefinitely.) The HOCON block carries the tuning: `active-strategy = keep-oldest`, `stable-after = 15s`,
-`keep-oldest.down-if-alone = on`, `failure-detector.threshold = 10.0` (its `active-strategy` + `down-if-alone`
-must stay consistent with the typed option; `stable-after` lives only in HOCON because the typed option can't
-express it).
+**The default downing strategy is `auto-down` (changed 2026-07-21).** It is selected by
+`Cluster:SplitBrainResolverStrategy`, and `ServiceCollectionExtensions.BuildDowningHocon` turns that
+into the `akka.cluster.downing-provider-class` actually installed:

-`keep-oldest` is the correct strategy for a 2-node warm-redundancy pair (`keep-majority`/`static-quorum`
-are wrong for two nodes — no majority in a 1-1 split), but its two loss cases recover **differently**, and
-this is inherent to any 2-node cluster:
-
-| Node lost | What keep-oldest does | Recovery |
+| `Cluster:SplitBrainResolverStrategy` | Provider installed | Posture |
 |---|---|---|
-| **Younger** node (crash or partition) | The oldest is on the surviving side → it stays up and downs the younger side (`DownUnreachable`). | **In-place, fast.** The oldest keeps its singletons + `driver` role-leadership; `RedundancyStateActor` re-computes from the post-loss `Cluster.State`. |
-| **Oldest** node (crash or partition) | The lone survivor is "the side **without** the oldest" → keep-oldest downs the **survivor itself** (`DownReachable "including myself"`), and `run-coordinated-shutdown-when-down = on` terminates it. `down-if-alone` does **not** rescue a lone survivor (its rescue branch needs ≥ 2 surviving members, so it is a 3+-node feature). | **Exit-and-rejoin.** The self-downed survivor **and** the crashed oldest are both restarted by the service supervisor and re-form / rejoin. There is a brief restart-window outage; there is **no** in-place takeover. |
+| `auto-down` (**default**) | `Akka.Cluster.AutoDowning` with `auto-down-unreachable-after = 15s` | **Availability.** The leader among the *reachable* members downs the unreachable peer, so a crash of **either** node — oldest included — fails over in place. |
+| `keep-oldest` | `Akka.Cluster.SBR.SplitBrainResolverProvider` reading the `split-brain-resolver` block in `akka.conf` | **Partition-safety.** A partition can never run dual-active. **Not safe for a 2-node pair** — see below. |

-**Recovery therefore depends on three things being in place** (the [ScadaBridge](../../ScadaBridge/CLAUDE.md)
-sister project runs the same pattern):
+Any other value **fails the host at startup** rather than falling through to a default.
+
+### Why keep-oldest is no longer the default
+
+The previous documentation in this section — and the code comments behind it — asserted that
+`keep-oldest` with `down-if-alone = on` was "the correct strategy for a 2-node warm-redundancy pair."
+**That was wrong, and the error was not conservative: it described a total-outage configuration as the
+recommended one.**
+
+In Akka.NET 1.5.62's `KeepOldest.OldestDecision`, the `down-if-alone` rescue branch requires the
+*surviving* side to hold **≥ 2 members**. In a two-node cluster a lost peer is always a 1-vs-1 split, so
+the branch never fires and the lone survivor falls through to `DownReachable` — downing **itself** — and
+`run-coordinated-shutdown-when-down = on` then terminates it. `down-if-alone` is a 3+-node feature; it
+does not do what its name suggests here. Live-proven on the sister project's rig, whose survivor logged
+`SBR took decision …DownReachable and is downing [self] including myself, [1] unreachable of [2] members`.
+See [`../../ScadaBridge/docs/plans/2026-07-21-auto-down-availability-decision.md`](../../ScadaBridge/docs/plans/2026-07-21-auto-down-availability-decision.md)
+for the upstream decision record, which OtOpcUa follows.
+
+The alternatives do not help a pair: `static-quorum` with quorum 1 hits Akka's `IsTooManyMembers` guard
+and returns `DownAll`; with quorum 2 the survivor downs itself; `keep-majority` keeps the lowest-address
+side, which just moves the fatal crash from "the oldest node" to "the other node"; `lease-majority` needs
+a shared lease store both nodes can reach.
+
+### The accepted trade
+
+Under `auto-down`, a **genuine network partition** (both nodes alive, link cut) leaves each side downing
+the other and continuing alone: **both run active**, both hold singletons, and both advertise the primary
+`ServiceLevel`. Recovery is operator-driven — after the link heals, restart **one** side; it rejoins as a
+fresh incarnation and settles as secondary. The two sides do not merge on their own, because the mutual
+downing quarantines the association.
+
+This is a deliberate choice of availability over partition-safety, matching ScadaBridge: the pairs run one
+node per VM with no external arbiter available, so a crash is a far more likely event than a partition, and
+a crash under `keep-oldest` was unrecoverable without operator action anyway.
+
+Deployments that would rather take an outage than ever run dual-active can set
+`Cluster:SplitBrainResolverStrategy: "keep-oldest"` — but should read the section above first, because for a
+two-node pair that choice means *any* crash of the oldest node is a full outage.
+
+### Recovery still depends on supervision

 1. **A service supervisor that restarts the process** on exit — production `Install-Services.ps1` sets
   `sc.exe failure OtOpcUaHost … actions= restart/5000/restart/30000/restart/60000`; the docker-dev rig sets
-   `restart: unless-stopped`. Without it a downed node stays down and an oldest-crash looks like a total outage.
-2. **The recovery watchdog** `ActorSystemTerminationWatchdog` (registered in `Program.cs` after `AddAkka`) — it
-   watches `ActorSystem.WhenTerminated` and, on an unexpected self-down, calls
-   `IHostApplicationLifetime.StopApplication()` so the process exits (and the supervisor restarts it) instead of
-   idling forever with a dead actor system.
-3. **Both peers listed in `SeedNodes`** on every node, so a restarted node can re-join the mesh via either peer.
+   `restart: unless-stopped`.
+2. **The recovery watchdog** `ActorSystemTerminationWatchdog` (registered in `Program.cs` after `AddAkka`) —
+   it watches `ActorSystem.WhenTerminated` and, on an unexpected self-down, calls
+   `IHostApplicationLifetime.StopApplication()` so the process exits and the supervisor restarts it instead
+   of idling forever with a dead actor system. Under `auto-down` the *survivor* no longer needs this
+   (it is never the one downed); the **downed** node still does.
+3. **Both peers listed in `SeedNodes`** on every node, so a restarted node can re-join via either peer.
+   Note the residual bootstrap constraint: only the **first** seed may form a cluster alone, so a node
+   cold-starting while its peer is dead still waits in `InitJoin`. Auto-down removes the crash outage, not
+   this one.

 > **On `HardKillFailoverTests`:** that in-process test hard-kills the oldest and asserts the survivor becomes
-> sole `driver` role-leader, and it passes — but it simulates the crash with `provider.Transport.Shutdown()`,
-> which leaves node A's `ActorSystem` **alive** (a transport partition with the oldest still running), not a
-> real process death. It is therefore **not** representative of an oldest-process crash; a real 2-container
-> `docker kill` of the oldest downs the survivor (verified 2026-07-15, see
-> `archreview/plans/artifacts/459-oldest-crash-live-finding-2026-07-15.md`). Treat the recovery guarantee for
-> the oldest as "exit-and-rejoin under supervision," not "in-place failover."
+> sole `driver` role-leader. It passed even under `keep-oldest` — because it simulates the crash with
+> `provider.Transport.Shutdown()`, which leaves node A's `ActorSystem` **alive** (a transport partition with
+> the oldest still running), not a real process death. A real 2-container `docker kill` of the oldest downed
+> the survivor (verified 2026-07-15, `archreview/plans/artifacts/459-oldest-crash-live-finding-2026-07-15.md`).
+> **It is a standing example of a green test over a fatal defect**: keep the distinction between a transport
+> partition and a process death in mind when reading it.

-There is no operator-driven role swap during a partition. Failover / recovery is what the cluster + supervisor
-do automatically. **Instant in-place takeover on *any* single-node loss requires 3+ members** (an odd cluster
-or a lightweight witness) — a deliberate future option, not the current 2-node posture.
+`SplitBrainResolverActivationTests` pins the strategy by starting a real host through the production
+bootstrap and reading `akka.cluster.downing-provider-class` back off the running `ActorSystem` — asserting
+the typed option or the akka.conf text is not sufficient, because several HOCON fragments compete and the
+losing one still looks correct in the file.
+
+There is no operator-driven role swap during a partition; failover is what the cluster and supervisor do
+automatically.
+
+> **Live gate outstanding.** The auto-down change is verified by unit tests against the effective
+> configuration, and by the sister project's live drill on an equivalent Akka version and topology. It has
+> **not** yet been drilled on an OtOpcUa two-node rig — the docker-dev rig runs a single six-node mesh, where
+> the 1-vs-1 pathology cannot occur. The drill (kill the oldest container of a genuine two-node cluster;
+> assert the survivor stays up, takes the singletons, and reports the primary `ServiceLevel`) should run
+> alongside the per-cluster mesh work, which makes every mesh exactly two nodes. See
+> `docs/plans/2026-07-21-per-cluster-mesh-design.md` §6.2 / Phase 0a.

 ## Primary data-plane gate (writes, acks, alerts emit)

@@ -1,9 +1,10 @@
 # Per-Cluster Akka Mesh — design

-> **Status:** design for review. No implementation planned yet.
+> **Status:** design for review. **Phases 0a and 0b are DONE and merged** (2026-07-21); Phases 1–7
+> are not started and each needs its own plan.
 > **Decisions settled 2026-07-21** — see §6: one central DB with driver nodes disconnected from it,
-> the two-node keep-oldest gap CLOSED upstream and still live here (§6.2 — most urgent item), and
-> their auth posture matched for now.
+> the two-node keep-oldest gap closed upstream and now closed here too (§6.2), and their auth posture
+> matched for now.

 ## 1. What this changes

@@ -142,6 +143,12 @@ the dual-primary shape archreview 03/S4 exists to prevent.
 and should not wait for this design.** It also happens to be exactly what the separate-mesh model
 needs, so it is not throwaway work.

+**FIXED 2026-07-21 (Phase 0b).** `RedundancyStateActor.BuildSnapshot` selects the oldest Up member
+carrying the `driver` role, matching singleton placement; `NodeRedundancyState.IsRoleLeaderForDriver`
+was renamed `IsDriverPrimary` so the contract stops asserting the old derivation. Pinned by
+`RedundancyPrimaryElectionTests`, which builds a real two-node cluster in which the oldest node holds
+the *higher* address, so the two orderings genuinely disagree.
+
 ## 5. Target architecture

 Mirror ScadaBridge's shape, adapted to OtOpcUa's existing substrate:
@@ -231,11 +238,20 @@ Its commit message carries the analysis, live-proven on their rig:
 `keep-oldest { down-if-alone = on }`, `stable-after = 15s`, with
 `BuildClusterOptions` setting the matching typed `KeepOldestOption { DownIfAlone = true }`.

-So a two-node OtOpcUa pair **cannot fail over when the oldest node crashes**: the survivor downs
-itself. That is a total-outage defect in the redundancy feature itself, it is independent of this
-design, and the sister project has already done the analysis and shipped the fix. **It should be
-Phase 0 alongside the oldest-Up role derivation**, and it needs its own live gate — a
-crash-the-oldest drill, since the failure only appears in the 1-vs-1 case.
+So a two-node OtOpcUa pair **could not fail over when the oldest node crashed**: the survivor
+downed itself. That was a total-outage defect in the redundancy feature itself, independent of this
+design.
+
+**FIXED 2026-07-21 (Phase 0a).** `Cluster:SplitBrainResolverStrategy` now selects the downing
+provider and defaults to `auto-down` — Akka's `AutoDowning` with a 15s window — so the leader among
+the reachable members downs the unreachable peer and a crash of either node fails over. `keep-oldest`
+remains selectable for deployments that prefer partition-safety. Pinned by
+`SplitBrainResolverActivationTests`, which asserts the provider read back off a running ActorSystem
+rather than the configured one. See `docs/Redundancy.md`.
+
+**Its live gate is still outstanding**, and cannot run on today's rig: the failure only appears in a
+1-vs-1 split, and `docker-dev` is a single six-node mesh. The crash-the-oldest drill therefore lands
+with Phase 6/7, which makes every mesh exactly two nodes.

 ### 6.3 DECIDED — match ScadaBridge's auth posture for now

@@ -250,8 +266,8 @@ Deliberately not a task plan — per-phase plans follow, one at a time.

 | Phase | Content | Independent of the split? |
 |---|---|---|
-| 0a | Downing strategy: keep-oldest cannot survive an oldest-node crash in a 2-node cluster (§6.2) | **Yes** — most urgent |
-| 0b | Oldest-Up role derivation (§4) | **Yes** |
+| 0a | Downing strategy: keep-oldest cannot survive an oldest-node crash in a 2-node cluster (§6.2) | **Yes** — **DONE 2026-07-21**, live gate deferred to Phase 7 |
+| 0b | Oldest-Up role derivation (§4) | **Yes** — **DONE 2026-07-21** |
 | 1 | `ClusterNode` gains Akka + gRPC address columns; coordinator sources its expected-ack set from the DB | Yes |
 | 2 | Comm actors + receptionist registration; ClusterClient transport; deploy notify + acks across the boundary | No |
 | 3 | Config fetch-and-cache: artifact served by central, driver nodes read LocalDb (§6.1) | No |
@@ -0,0 +1,191 @@
+# Per-Cluster Mesh — Phase 1 plan
+
+> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task.
+
+**Parent design:** [`2026-07-21-per-cluster-mesh-design.md`](2026-07-21-per-cluster-mesh-design.md).
+Phases 0a and 0b are done and merged; this is the next phase.
+
+**Goal:** Give `ClusterNode` the transport addresses central will need to reach each cluster
+directly, and cut `ConfigPublishCoordinator`'s last dependency on shared cluster membership by
+sourcing its expected-ack set from those rows.
+
+**Architecture:** Additive schema change plus one substitution inside the coordinator. Nothing else
+moves; the deploy channel stays on DistributedPubSub until Phase 2.
+
+**Status:** NOT STARTED. Read §"The decision this phase forces" before executing — it changes
+deploy-seal semantics and should be confirmed, not assumed.
+
+---
+
+## Why this is smaller than the design implies
+
+The design says "`ClusterNode` gains Akka + gRPC address columns; coordinator sources its
+expected-ack set from the DB", which reads as though the two are one change. They are not, and the
+second does not depend on the first.
+
+`ConfigPublishCoordinator.DiscoverDriverNodes` builds `NodeId.Parse($"{host}:{port}")` from
+`Cluster.State.Members`. `ClusterNode.NodeId` is **already in that identifier space** — the rig seeds
+`central-1:4053`, `site-a-1:4053` and so on (`docker-dev/seed/seed-clusters.sql:65`), and its comment
+says so explicitly, because `NodeDeploymentState.NodeId` is FK-bound to `ClusterNode.NodeId` and a
+mismatch throws FK 547 on deploy. The coordinator already writes `NodeDeploymentState` rows keyed by
+the membership-derived id and they already satisfy that FK — which is standing proof the two sets
+agree today.
+
+So the coordinator switch is a substitution inside one private method, and the address columns are
+purely forward-looking groundwork for Phase 2's ClusterClient and Phase 5's gRPC dial.
+
+## The decision this phase forces
+
+**Membership-derived and DB-derived expected-ack sets are not the same set, and the difference is a
+behaviour change on the deploy path.**
+
+| Situation | Today (membership) | After (DB rows) |
+|---|---|---|
+| Configured node is running | expected, acks, seals | same |
+| Configured node is **stopped** | not expected — deploy seals **successfully** without it | expected — deploy waits, then **fails at the apply deadline** |
+| Node running but not in `ClusterNode` | expected (and then FK 547 on the state row) | not expected; its ack is discarded |
+
+The middle row is the one that matters. Today a deployment can seal green while a node that should
+have received it was down and did not — the operator is told the fleet is deployed when it is not.
+Failing instead is arguably the correct behaviour and is the reason the design moves this to the DB
+(central cannot see cluster membership once the meshes split).
+
+But it is a behaviour change, and it needs an operator escape hatch or a node taken down for
+maintenance blocks every deployment. `ClusterNode.Enabled` already exists and is exactly that hatch,
+so the query filters on it: **`Enabled = 1` rows are expected to ack; disabled rows are not.**
+
+**Confirm this before executing Task 3.** If the preference is to keep sealing green past a down
+node, the alternative is to intersect the DB set with current membership, which preserves today's
+behaviour but does not survive the mesh split — it would have to be revisited in Phase 2 anyway.
+
+---
+
+## Task 1: Address columns on `ClusterNode`
+
+**Classification:** small
+**Estimated implement time:** ~4 min
+**Parallelizable with:** none (Task 2 migrates what this declares)
+
+**Files:**
+- Modify: `src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Entities/ClusterNode.cs`
+- Modify: the `ClusterNode` entity configuration in `src/Core/ZB.MOM.WW.OtOpcUa.Configuration/` (find the `IEntityTypeConfiguration<ClusterNode>` or the `OnModelCreating` block)
+- Test: `tests/Core/ZB.MOM.WW.OtOpcUa.Configuration.Tests/` (mirror an existing entity-mapping test)
+
+Add two nullable columns beside the existing `OpcUaPort` / `DashboardPort`:
+
+- `int AkkaPort` — default `4053`. The remoting port central will use to build the ClusterClient
+  contact point. Non-nullable with a default, because every node has one.
+- `int? GrpcPort` — nullable, no default. The Phase 5 telemetry-stream port. Nullable is deliberate:
+  it does not exist yet, and a non-null default would assert a port nothing listens on.
+
+Document on each property that these are **central's dial targets**, not the node's own binding
+configuration — the node binds from `Cluster:Port` in its own appsettings, and these columns
+duplicate that value for a reader that cannot see the node's config. That duplication is the reason
+Task 4 exists.
+
+**Step 1:** Write a mapping test asserting a round-tripped `ClusterNode` preserves both values and
+that `AkkaPort` defaults to 4053.
+**Step 2:** Run it; expect failure to compile.
+**Step 3:** Add the properties + mapping.
+**Step 4:** Run; expect pass.
+**Step 5:** Commit.
+
+## Task 2: EF migration
+
+**Classification:** small
+**Estimated implement time:** ~3 min
+**Parallelizable with:** none
+
+**Files:**
+- Create: a new migration under the Configuration project's `Migrations/` folder
+
+```bash
+dotnet ef migrations add AddClusterNodeTransportPorts \
+  --project src/Core/ZB.MOM.WW.OtOpcUa.Configuration
+```
+
+Verify the generated `Up` adds `AkkaPort` with `defaultValue: 4053` and `GrpcPort` as nullable, and
+that `Down` drops both. **Check the migration is additive only** — an `AlterColumn` on anything else
+means the model had drifted from the last migration and that drift must be understood before this
+ships, not carried along by it.
+
+## Task 3: Coordinator sources the expected-ack set from `ClusterNode`
+
+**Classification:** high-risk — this is the deploy path
+**Estimated implement time:** ~5 min
+**Parallelizable with:** none
+
+**Files:**
+- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.ControlPlane/Coordinators/ConfigPublishCoordinator.cs` (`DiscoverDriverNodes`, ~line 262, and the class doc comment at line 21 which states the membership rule)
+- Test: `tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests/ConfigPublishCoordinatorTests.cs`
+
+Replace the `Cluster.State.Members` scan with a query over enabled `ClusterNode` rows. Keep
+`NodeIdComparer` — the case-insensitivity rationale in its comment applies unchanged, and SQL
+collation makes it more relevant, not less.
+
+Points to get right:
+
+- **Filter on `Enabled`.** See "The decision this phase forces".
+- **Scope to the deployment's cluster if the deployment is cluster-scoped.** `DeploymentArtifact.ResolveClusterScope`
+  already defines that scoping; a fleet-wide deployment expects every enabled node, a cluster-scoped
+  one expects only that cluster's. Getting this wrong makes every cluster-scoped deploy hang on nodes
+  that were never sent it. **Read `ResolveClusterScope` before writing the query.**
+- **The empty-set branch still matters.** `HandleDispatch` seals immediately when the set is empty,
+  and its log message says "no driver-role members in cluster" — reword it, because the reason is now
+  "no enabled ClusterNode rows", which is a configuration error rather than a topology observation.
+- The DB is read inside `HandleDispatch`, which already opens a context; reuse it rather than opening
+  a second.
+
+**Tests to add:** enabled-only filtering; cluster-scope filtering; empty set seals with the new
+reason; an ack from a node absent from `ClusterNode` is discarded. **Positive control on each:**
+delete the filter and confirm the test goes red — the existing suite passed throughout the
+membership era, so a test that also passes against the old derivation is testing nothing.
+
+## Task 4: Reconcile the duplicated port with the node's own config
+
+**Classification:** standard
+**Estimated implement time:** ~4 min
+**Parallelizable with:** none
+
+`ClusterNode.AkkaPort` and the node's own `Cluster:Port` are the same fact in two places, and nothing
+makes them agree. A node that binds 4054 while its row says 4053 is unreachable from central in Phase
+2, and the symptom will be a silent absence of acks rather than an error.
+
+Options, in preference order:
+
+1. **Have the node assert its row on startup** — on join, compare `Cluster:Port` /
+   `Cluster:PublicHostname` to its own `ClusterNode` row and log an Error (not a Warning) on
+   mismatch. Cheap, no schema change, no write path from driver nodes to the ConfigDb (which Phase 4
+   removes anyway — so put this check on an **admin** node, reading the membership it can already see).
+2. Deploy-time validation in `DraftValidator`, alongside the existing gates.
+
+Pick one and implement it. Do not skip this task: an unenforced duplicated address is the exact shape
+of the `Modbus`/`ModbusTcp` drift and the `TwinCat`/`Focas` drift already recorded in this repo.
+
+## Task 5: Rig seed + docs
+
+**Classification:** small
+**Estimated implement time:** ~3 min
+
+**Files:**
+- Modify: `docker-dev/seed/seed-clusters.sql` — add `AkkaPort` to the six INSERTs (all 4053) so a
+  freshly-seeded rig matches a migrated one. Leave `GrpcPort` null.
+- Modify: `docs/Configuration.md` — document both columns.
+- Modify: `docs/plans/2026-07-21-per-cluster-mesh-design.md` — mark Phase 1 done in §7.
+
+## Task 6: Live gate
+
+**Classification:** high-risk
+**Estimated implement time:** n/a — operator-driven
+
+Offline tests cannot cover the substitution's real risk, which is that the DB set and the live set
+disagree on the rig. Run on `docker-dev`:
+
+1. Deploy with all six nodes up → seals green, six `NodeDeploymentState` rows, no FK 547.
+2. Deploy a **cluster-scoped** config → expected set is that cluster's nodes only.
+3. `docker stop` one node, deploy → **fails at the apply deadline** naming the missing node. This is
+   the new behaviour; confirm the failure is legible, because an operator will meet it.
+4. Set that node's `Enabled = 0`, deploy again → seals green without it. This proves the escape
+   hatch, and is the check that makes step 3 acceptable to ship.
+
+Record results in a gate doc beside this plan, per the Phase 1/Phase 2 precedent.
@@ -29,4 +29,30 @@ public sealed class AkkaClusterOptions
    /// <c>admin</c>, <c>driver</c>, <c>dev</c>.
    /// </summary>
    public string[] Roles { get; set; } = Array.Empty<string>();
+
+    /// <summary>
+    /// How the cluster decides to down a node it can no longer reach. One of <c>auto-down</c>
+    /// (default) or <c>keep-oldest</c>; any other value fails the host at startup.
+    /// </summary>
+    /// <remarks>
+    ///     <para>
+    ///         <b><c>auto-down</c> — availability.</b> The leader among the <i>reachable</i> members
+    ///         downs the unreachable peer after
+    ///         <see cref="ServiceCollectionExtensions.DowningStableAfter"/>. A hard crash of either
+    ///         node — including the oldest — fails over to the survivor with no operator action.
+    ///         The trade is that a genuine network partition (both nodes alive, link cut) leaves
+    ///         both sides running active until an operator restarts one.
+    ///     </para>
+    ///     <para>
+    ///         <b><c>keep-oldest</c> — partition-safety.</b> The SBR resolver sacrifices the younger
+    ///         side of a split, so a partition can never run dual-active. <b>The cost is severe for a
+    ///         two-node pair:</b> it cannot survive a crash of the oldest node at all. Akka.NET's
+    ///         <c>KeepOldest.OldestDecision</c> only lets <c>down-if-alone</c> rescue a side holding
+    ///         &gt;= 2 members, so the 1-vs-1 survivor downs <i>itself</i> and shuts down — the
+    ///         redundancy pair turns a single-node crash into a total outage. Choose this only for
+    ///         clusters of three or more nodes, or where dual-active is genuinely worse than an
+    ///         outage.
+    ///     </para>
+    /// </remarks>
+    public string SplitBrainResolverStrategy { get; set; } = "auto-down";
 }
@@ -7,7 +7,7 @@ public readonly record struct NodeHealthInputs(
    bool DbReachable,
    bool OpcUaProbeOk,
    bool Stale,
-    bool IsDriverRoleLeader);
+    bool IsDriverPrimary);

 /// <summary>
 /// Pure ServiceLevel computation per design §6. Output range 0–255, where higher = "more
@@ -18,7 +18,9 @@ public readonly record struct NodeHealthInputs(
 /// - Member not Up/Joining: 0 (cluster cannot trust this node).
 /// - DB reachable + OPC UA probe ok + not stale: 240 (full service).
 /// - Stale config (DB reachable or not, OPC UA probe state ignored): 100 or 200 depending on DB.
-/// - +10 bonus when this node holds the role-leader lease for the "driver" role.
+/// - +10 bonus when this node is the driver Primary (the oldest Up member with the "driver" role —
+///   where the cluster singletons live). Previously this keyed off Akka's role *leader*, the
+///   lowest-addressed member, which names a different node once any node has restarted.
 /// </summary>
 public static class ServiceLevelCalculator
 {
@@ -38,6 +40,6 @@ public static class ServiceLevelCalculator
            _ => 0,
        };

-        return (byte)Math.Clamp(basis + (h.IsDriverRoleLeader ? 10 : 0), 0, 255);
+        return (byte)Math.Clamp(basis + (h.IsDriverPrimary ? 10 : 0), 0, 255);
    }
 }
@@ -37,14 +37,26 @@ akka {
        roles = []
        min-nr-of-members = 1

-        # Split-brain resolver (arch-review 03/S1). This block IS active: Akka.Cluster.Hosting's
-        # WithClustering enables an SBR downing provider by default (it applies
-        # SplitBrainResolverOption.Default when ClusterOptions.SplitBrainResolver is null), and that
-        # provider reads this block. ServiceCollectionExtensions.BuildClusterOptions additionally sets
-        # the typed KeepOldestOption { DownIfAlone = true } to make the strategy EXPLICIT in code —
-        # its active-strategy + keep-oldest.down-if-alone MUST match this block. (The cluster is NOT
-        # NoDowning; only an explicit downing-provider-class = "" would disable failover.)
-        # stable-after must stay >= failure-detector.acceptable-heartbeat-pause (10s) + margin.
+        # DOWNING. The provider is NOT selected here — this file is added with HoconAddMode.Append,
+        # which loses to what Akka.Hosting's WithClustering emits, so a downing-provider-class set
+        # here would be silently overridden. Selection lives in
+        # ServiceCollectionExtensions.BuildDowningHocon (Prepended, which outranks WithClustering),
+        # driven by Cluster:SplitBrainResolverStrategy.
+        #
+        # The DEFAULT strategy is "auto-down", under which this whole block is inert: Akka's
+        # AutoDowning provider is installed with auto-down-unreachable-after = 15s, and the leader
+        # among the reachable members downs the unreachable peer — so a hard crash of EITHER node,
+        # oldest included, fails over. The trade is dual-active during a real network partition.
+        #
+        # The block below applies only when an operator selects "keep-oldest". Note that keep-oldest
+        # is NOT safe for a two-node pair: Akka.NET's KeepOldest.OldestDecision only lets
+        # down-if-alone rescue a side holding >= 2 members, so a 1-vs-1 survivor downs ITSELF and
+        # (with run-coordinated-shutdown-when-down) exits — a crash of the oldest node becomes a
+        # total outage. See docs/Redundancy.md.
+        #
+        # stable-after must stay >= failure-detector.acceptable-heartbeat-pause (10s) + margin, and
+        # must equal ServiceCollectionExtensions.DowningStableAfter so both strategies fail over at
+        # the same latency. Both constraints are pinned by SplitBrainResolverActivationTests.
        split-brain-resolver {
            active-strategy = "keep-oldest"
            stable-after = 15s
@@ -80,40 +80,125 @@ public static class ServiceCollectionExtensions

        builder.WithClustering(BuildClusterOptions(options));

+        // Must come AFTER WithClustering, which always emits a downing-provider-class of its own:
+        // Akka.Cluster.Hosting applies SplitBrainResolverOption.Default whenever the typed
+        // ClusterOptions.SplitBrainResolver is null, so there is always a competing value to beat.
+        // Prepend is the highest-precedence mode, which makes this block win irrespective of how
+        // many fragments are added or in what order.
+        //
+        // Do not take this comment's word for it. Which fragment actually wins is not obvious from
+        // the mode names — measured, Append also wins here, purely because it happens to be added
+        // last. SplitBrainResolverActivationTests reads the provider back off a *running*
+        // ActorSystem for exactly that reason: the effective value is the only one worth asserting.
+        builder.AddHocon(BuildDowningHocon(options), HoconAddMode.Prepend);
+
        return builder;
    }

    /// <summary>
-    /// Builds the <see cref="ClusterOptions"/> for the fused-host cluster, setting the split-brain-resolver
-    /// strategy <b>explicitly in code</b>.
-    ///
-    /// <b>Activation note (arch-review 03/S1, corrected):</b> Akka.Cluster.Hosting's <c>WithClustering</c>
-    /// already enables an SBR downing provider <b>by default</b> — when <see cref="ClusterOptions.SplitBrainResolver"/>
-    /// is <c>null</c> it applies <c>SplitBrainResolverOption.Default</c>, which registers
-    /// <c>Akka.Cluster.SBR.SplitBrainResolverProvider</c> and reads the <c>split-brain-resolver</c> HOCON block in
-    /// <c>Resources/akka.conf</c>. So the cluster was <b>not</b> running <c>NoDowning</c> before this option was
-    /// set: the pre-existing akka.conf <c>keep-oldest</c> block was already active, and hard-crashed nodes already
-    /// failed over. Setting this typed option makes the strategy <b>explicit in code</b> (independent of the
-    /// framework default) rather than being the sole activator — it is reinforcing/belt-and-suspenders and
-    /// produces the same effective behavior. (Only an <i>explicit</i> <c>NoDowning</c>, e.g.
-    /// <c>downing-provider-class = ""</c>, would disable failover.)
-    ///
-    /// <see cref="KeepOldestOption"/> with <c>DownIfAlone=true</c> mirrors the HOCON intent and is the
-    /// correct strategy for a 2-node warm-redundancy pair: on an even split the oldest member (typically
-    /// the long-running primary) survives, and <c>down-if-alone</c> downs a singleton that loses its
-    /// peer. <c>keep-majority</c>/<c>static-quorum</c> are wrong for two nodes. The strategy + down-if-alone
-    /// here MUST stay consistent with the HOCON block; <c>stable-after</c> lives only in HOCON because the
-    /// typed option cannot express it (it must stay ≥ <c>failure-detector.acceptable-heartbeat-pause</c>).
+    /// How long a member must stay unreachable before it is downed. Shared by both strategies
+    /// (<c>auto-down-unreachable-after</c> and the SBR resolver's <c>stable-after</c>) so the two
+    /// have the same failover latency. Must stay above
+    /// <c>akka.cluster.failure-detector.acceptable-heartbeat-pause</c> or a merely-slow node gets
+    /// downed; both constraints are pinned by tests.
    /// </summary>
-    /// <param name="options">The bound cluster options carrying seed nodes and roles.</param>
-    /// <returns>The cluster options with seed nodes, roles, and the explicit split-brain-resolver strategy.</returns>
+    public static readonly TimeSpan DowningStableAfter = TimeSpan.FromSeconds(15);
+
+    /// <summary>
+    /// Builds the HOCON that selects the downing provider, per
+    /// <see cref="AkkaClusterOptions.SplitBrainResolverStrategy"/>.
+    /// </summary>
+    /// <param name="options">The bound cluster options carrying the strategy.</param>
+    /// <returns>
+    /// For <c>auto-down</c>, a block installing Akka's <c>AutoDowning</c> provider with a downing
+    /// window of <see cref="DowningStableAfter"/>. For <c>keep-oldest</c>, an empty string — the
+    /// typed <see cref="KeepOldestOption"/> from <see cref="BuildClusterOptions"/> already installs
+    /// the SBR provider, and the resolver's own settings live in <c>Resources/akka.conf</c>.
+    /// </returns>
+    /// <exception cref="ArgumentOutOfRangeException">The strategy is not a recognised value.</exception>
+    public static string BuildDowningHocon(AkkaClusterOptions options)
+    {
+        ArgumentNullException.ThrowIfNull(options);
+
+        if (IsAutoDown(options))
+        {
+            // auto-down-unreachable-after is load-bearing: it defaults to `off`, under which
+            // AutoDowning is installed but never downs anything — failover silently disabled.
+            return $$"""
+                akka.cluster {
+                    downing-provider-class = "Akka.Cluster.AutoDowning, Akka.Cluster"
+                    auto-down-unreachable-after = {{(int)DowningStableAfter.TotalMilliseconds}}ms
+                }
+                """;
+        }
+
+        if (IsKeepOldest(options))
+        {
+            return string.Empty;
+        }
+
+        throw new ArgumentOutOfRangeException(
+            nameof(options),
+            options.SplitBrainResolverStrategy,
+            $"Unknown {AkkaClusterOptions.SectionName}:{nameof(AkkaClusterOptions.SplitBrainResolverStrategy)}. "
+            + "Expected 'auto-down' (default; survives a crash of either node) or 'keep-oldest' "
+            + "(partition-safe, but a two-node pair cannot survive a crash of the oldest node).");
+    }
+
+    private static bool IsAutoDown(AkkaClusterOptions options) =>
+        string.Equals(options.SplitBrainResolverStrategy, "auto-down", StringComparison.OrdinalIgnoreCase);
+
+    private static bool IsKeepOldest(AkkaClusterOptions options) =>
+        string.Equals(options.SplitBrainResolverStrategy, "keep-oldest", StringComparison.OrdinalIgnoreCase);
+
+    /// <summary>
+    /// Builds the <see cref="ClusterOptions"/> for the fused-host cluster.
+    /// </summary>
+    /// <remarks>
+    ///     <para>
+    ///         <b>Activation note (arch-review 03/S1, corrected twice).</b> Akka.Cluster.Hosting's
+    ///         <c>WithClustering</c> always installs a downing provider: when
+    ///         <see cref="ClusterOptions.SplitBrainResolver"/> is <c>null</c> it applies
+    ///         <c>SplitBrainResolverOption.Default</c>, which registers
+    ///         <c>Akka.Cluster.SBR.SplitBrainResolverProvider</c> and reads the
+    ///         <c>split-brain-resolver</c> block in <c>Resources/akka.conf</c>. The cluster is
+    ///         therefore never <c>NoDowning</c>. What this method chooses is only <i>which</i>
+    ///         provider — and under the default <c>auto-down</c> strategy the choice is not made
+    ///         here at all but by the Prepended HOCON in
+    ///         <see cref="WithOtOpcUaClusterBootstrap"/>, which outranks whatever
+    ///         <c>WithClustering</c> emits.
+    ///     </para>
+    ///     <para>
+    ///         <b>Why keep-oldest is no longer the default (2026-07-21).</b> The previous
+    ///         documentation here asserted that <see cref="KeepOldestOption"/> with
+    ///         <c>DownIfAlone=true</c> was "the correct strategy for a 2-node warm-redundancy pair"
+    ///         because <c>down-if-alone</c> would down a node that lost its peer. That is the
+    ///         opposite of what Akka.NET does. In <c>KeepOldest.OldestDecision</c> the
+    ///         <c>down-if-alone</c> branch requires the surviving side to hold &gt;= 2 members, so
+    ///         in a 1-vs-1 split the survivor falls through to <c>DownReachable</c> and downs
+    ///         <i>itself</i>; with <c>run-coordinated-shutdown-when-down = on</c> it then exits. A
+    ///         two-node pair running keep-oldest converts a crash of the oldest node into a total
+    ///         outage — precisely the failure redundancy exists to absorb. Live-proven on the sister
+    ///         project's rig; see <c>docs/Redundancy.md</c> and
+    ///         <see cref="AkkaClusterOptions.SplitBrainResolverStrategy"/> for the trade.
+    ///     </para>
+    /// </remarks>
+    /// <param name="options">The bound cluster options carrying seed nodes, roles and the strategy.</param>
+    /// <returns>
+    /// The cluster options with seed nodes and roles. <see cref="ClusterOptions.SplitBrainResolver"/>
+    /// is set only under <c>keep-oldest</c>, where the SBR provider is the one wanted; under
+    /// <c>auto-down</c> it is left null because the Prepended HOCON replaces the provider anyway,
+    /// and naming a resolver that is not in force would be misleading.
+    /// </returns>
    public static ClusterOptions BuildClusterOptions(AkkaClusterOptions options)
    {
+        ArgumentNullException.ThrowIfNull(options);
+
        return new ClusterOptions
        {
            SeedNodes = options.SeedNodes,
            Roles = options.Roles,
-            SplitBrainResolver = new KeepOldestOption { DownIfAlone = true },
+            SplitBrainResolver = IsKeepOldest(options) ? new KeepOldestOption { DownIfAlone = true } : null,
        };
    }
 }
@@ -8,9 +8,20 @@ public enum RedundancyRole { Primary, Secondary, Detached }
 /// Snapshot of a single node's redundancy state. Aggregated by <c>RedundancyStateActor</c>
 /// to compute fleet-wide ServiceLevel.
 /// </summary>
+/// <param name="NodeId">Canonical <c>host:port</c> id of the node this entry describes.</param>
+/// <param name="Role">The node's redundancy role in the current snapshot.</param>
+/// <param name="IsClusterLeader">Whether the node is the Akka cluster leader.</param>
+/// <param name="IsDriverPrimary">
+/// Whether the node is the driver Primary — the <b>oldest</b> Up member carrying the <c>driver</c>
+/// role, which is where <c>ClusterSingletonManager</c> places singletons. Renamed from
+/// <c>IsRoleLeaderForDriver</c>: it was derived from <c>ClusterState.RoleLeader("driver")</c>, the
+/// lowest-<i>addressed</i> member, which diverges from the oldest after any node restart. The name
+/// now describes what the value means rather than how it used to be computed.
+/// </param>
+/// <param name="AsOfUtc">When the snapshot was computed.</param>
 public sealed record NodeRedundancyState(
    NodeId NodeId,
    RedundancyRole Role,
    bool IsClusterLeader,
-    bool IsRoleLeaderForDriver,
+    bool IsDriverPrimary,
    DateTime AsOfUtc);
@@ -24,6 +24,10 @@
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
        </PackageReference>
        <PackageReference Include="Microsoft.AspNetCore.DataProtection.EntityFrameworkCore"/>
+        <!-- Surgical transitive pin — see the System.Security.Cryptography.Xml block in
+             Directory.Packages.props. This is the one project where the DataProtection chain
+             enters the repo; every other consumer reaches it through a ProjectReference here. -->
+        <PackageReference Include="System.Security.Cryptography.Xml"/>
        <PackageReference Include="Microsoft.Extensions.Configuration.Abstractions"/>
        <PackageReference Include="Microsoft.Extensions.Logging.Abstractions"/>
        <PackageReference Include="Polly.Core"/>
@@ -111,9 +111,49 @@ public sealed class RedundancyStateActor : ReceiveActor, IWithTimers
        _log.Debug("Published RedundancyStateChanged with {Count} nodes", snapshot.Count);
    }

+    /// <summary>The cluster role that marks a node as carrying the driver runtime.</summary>
+    public const string DriverRole = "driver";
+
+    /// <summary>
+    /// Selects the driver Primary: the <b>oldest</b> Up member carrying the <see cref="DriverRole"/>.
+    /// </summary>
+    /// <remarks>
+    ///     <para>
+    ///         This deliberately does <b>not</b> use <c>ClusterState.RoleLeader(role)</c>, which it
+    ///         previously did. Role leader is the <i>lowest-addressed</i> Up member with the role —
+    ///         address order (host, then port), with no relationship to time. Oldest is the member
+    ///         with the lowest up-number, and it is what <c>ClusterSingletonManager</c> uses to place
+    ///         singletons.
+    ///     </para>
+    ///     <para>
+    ///         On a freshly-formed cluster the two agree, which is why the discrepancy was invisible.
+    ///         They diverge after any restart: the restarted node re-joins as the youngest but keeps
+    ///         its address, so if it holds the lower address it becomes role leader while the
+    ///         singletons — and all the work they own — stay on the other node. Electing it Primary
+    ///         would enable the Primary-gated data plane (inbound writes, alarm acks, the fleet-wide
+    ///         alerts emit, the alarm-history drain) on the node that is not hosting the work.
+    ///     </para>
+    ///     <para>
+    ///         Only <see cref="MemberStatus.Up"/> members are eligible, matching singleton placement:
+    ///         a <c>Leaving</c> node is handing its singletons over and must not be named Primary.
+    ///     </para>
+    /// </remarks>
+    /// <param name="members">The current cluster members, in any order.</param>
+    /// <returns>The oldest Up driver member's address, or <c>null</c> when there is none.</returns>
+    public static Address? SelectDriverPrimary(IEnumerable<Member> members)
+    {
+        ArgumentNullException.ThrowIfNull(members);
+
+        return members
+            .Where(m => m.Status == MemberStatus.Up && m.Roles.Contains(DriverRole))
+            .OrderBy(m => m, Member.AgeOrdering)
+            .FirstOrDefault()
+            ?.Address;
+    }
+
    private IReadOnlyList<NodeRedundancyState> BuildSnapshot()
    {
-        var driverLeader = _cluster.State.RoleLeader("driver");
+        var driverPrimary = SelectDriverPrimary(_cluster.State.Members);
        var clusterLeader = _cluster.State.Leader;
        var now = DateTime.UtcNow;

@@ -123,15 +163,16 @@ public sealed class RedundancyStateActor : ReceiveActor, IWithTimers
            var host = member.Address.Host;
            if (string.IsNullOrWhiteSpace(host)) continue;

-            var role = member.Roles.Contains("driver")
-                ? (driverLeader == member.Address ? CommonsRedundancyRole.Primary : CommonsRedundancyRole.Secondary)
+            var isPrimary = driverPrimary is not null && driverPrimary == member.Address;
+            var role = member.Roles.Contains(DriverRole)
+                ? (isPrimary ? CommonsRedundancyRole.Primary : CommonsRedundancyRole.Secondary)
                : CommonsRedundancyRole.Detached;

            list.Add(new NodeRedundancyState(
                ToNodeId(member.Address),
                role,
                IsClusterLeader: clusterLeader == member.Address,
-                IsRoleLeaderForDriver: driverLeader == member.Address,
+                IsDriverPrimary: isPrimary,
                AsOfUtc: now));
        }
        return list;
@@ -652,7 +652,7 @@ public sealed class OpcUaPublishActor : ReceiveActor, IWithTimers
            Stale: !_lastDbHealth.Reachable
                   || (now - _lastDbHealth.AsOfUtc) > _staleWindow
                   || (now - entry.AsOfUtc) > _staleWindow,
-            IsDriverRoleLeader: entry.IsRoleLeaderForDriver);
+            IsDriverPrimary: entry.IsDriverPrimary);

        Self.Tell(new ServiceLevelChanged(ServiceLevelCalculator.Compute(inputs)));
    }
@@ -661,7 +661,7 @@ public sealed class OpcUaPublishActor : ReceiveActor, IWithTimers
    /// secondary → 100, _ → 0). Preserved as the back-compat / bootstrap seam.</summary>
    private static byte LegacyRoleOnly(NodeRedundancyState entry) => entry.Role switch
    {
-        RedundancyRole.Primary when entry.IsRoleLeaderForDriver => 240,
+        RedundancyRole.Primary when entry.IsDriverPrimary => 240,
        RedundancyRole.Primary => 200,
        RedundancyRole.Secondary => 100,
        _ => 0,
@@ -21,7 +21,7 @@ public sealed class ServiceLevelCalculatorTests
    public void NotUp_returns_zero(MemberStatus status)
    {
        var sl = ServiceLevelCalculator.Compute(new(status,
-            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverRoleLeader: true));
+            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverPrimary: true));
        sl.ShouldBe((byte)0);
    }

@@ -30,7 +30,7 @@ public sealed class ServiceLevelCalculatorTests
    public void Fully_healthy_non_leader_returns_240()
    {
        var sl = ServiceLevelCalculator.Compute(new(MemberStatus.Up,
-            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverRoleLeader: false));
+            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverPrimary: false));
        sl.ShouldBe((byte)240);
    }

@@ -39,7 +39,7 @@ public sealed class ServiceLevelCalculatorTests
    public void Fully_healthy_role_leader_returns_250()
    {
        var sl = ServiceLevelCalculator.Compute(new(MemberStatus.Up,
-            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverRoleLeader: true));
+            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverPrimary: true));
        sl.ShouldBe((byte)250);
    }

@@ -48,7 +48,7 @@ public sealed class ServiceLevelCalculatorTests
    public void Db_reachable_but_stale_returns_200()
    {
        var sl = ServiceLevelCalculator.Compute(new(MemberStatus.Up,
-            DbReachable: true, OpcUaProbeOk: true, Stale: true, IsDriverRoleLeader: false));
+            DbReachable: true, OpcUaProbeOk: true, Stale: true, IsDriverPrimary: false));
        sl.ShouldBe((byte)200);
    }

@@ -57,7 +57,7 @@ public sealed class ServiceLevelCalculatorTests
    public void Db_unreachable_and_stale_returns_100()
    {
        var sl = ServiceLevelCalculator.Compute(new(MemberStatus.Up,
-            DbReachable: false, OpcUaProbeOk: false, Stale: true, IsDriverRoleLeader: false));
+            DbReachable: false, OpcUaProbeOk: false, Stale: true, IsDriverPrimary: false));
        sl.ShouldBe((byte)100);
    }

@@ -66,7 +66,7 @@ public sealed class ServiceLevelCalculatorTests
    public void Opcua_probe_fail_when_not_stale_returns_zero()
    {
        var sl = ServiceLevelCalculator.Compute(new(MemberStatus.Up,
-            DbReachable: true, OpcUaProbeOk: false, Stale: false, IsDriverRoleLeader: false));
+            DbReachable: true, OpcUaProbeOk: false, Stale: false, IsDriverPrimary: false));
        sl.ShouldBe((byte)0);
    }

@@ -75,7 +75,7 @@ public sealed class ServiceLevelCalculatorTests
    public void Joining_member_is_treated_like_Up()
    {
        var sl = ServiceLevelCalculator.Compute(new(MemberStatus.Joining,
-            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverRoleLeader: false));
+            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverPrimary: false));
        sl.ShouldBe((byte)240);
    }

@@ -85,7 +85,7 @@ public sealed class ServiceLevelCalculatorTests
    {
        // basis 240 + 10 = 250, already within byte range; confirms Clamp is in path
        var sl = ServiceLevelCalculator.Compute(new(MemberStatus.Up,
-            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverRoleLeader: true));
+            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverPrimary: true));
        ((int)sl).ShouldBeLessThanOrEqualTo(255);
    }
 }
@@ -1,47 +1,215 @@
+using Akka.Actor;
 using Akka.Cluster.Hosting;
 using Akka.Cluster.Hosting.SBR;
+using Akka.Configuration;
+using Akka.Hosting;
+using Microsoft.Extensions.DependencyInjection;
+using Microsoft.Extensions.Hosting;
+using Microsoft.Extensions.Options;
 using Shouldly;
 using Xunit;

 namespace ZB.MOM.WW.OtOpcUa.Cluster.Tests;

 /// <summary>
-/// Guards that the split-brain-resolver strategy stays set <b>explicitly in code</b> (arch-review 03/S1,
-/// premise corrected — see #11). Akka.Cluster.Hosting already enables an SBR downing provider by default
-/// (it applies <c>SplitBrainResolverOption.Default</c> when <see cref="ClusterOptions.SplitBrainResolver"/>
-/// is null, which reads the akka.conf <c>keep-oldest</c> block), so the cluster is not NoDowning even
-/// without the typed option. These tests assert
-/// <see cref="ServiceCollectionExtensions.BuildClusterOptions"/> sets the typed
-/// <c>KeepOldestOption { DownIfAlone = true }</c> so the strategy is explicit in code rather than relying on
-/// the framework default — a future refactor cannot silently drop that explicitness. (Failover behaviour is
-/// separately verified by <c>HardKillFailoverTests</c>, which passes regardless of activation path.)
+/// Guards the downing strategy — the mechanism that decides whether a node that loses its peer
+/// keeps running or shuts itself down.
 /// </summary>
+/// <remarks>
+///     <para>
+///         <b>Why these tests assert the EFFECTIVE config, not the typed option.</b> The previous
+///         version of this file asserted only that
+///         <see cref="ServiceCollectionExtensions.BuildClusterOptions"/> returned a
+///         <see cref="KeepOldestOption"/>. That is a different question from "what does the running
+///         ActorSystem actually do". Akka.Hosting merges several HOCON fragments — the akka.conf
+///         resource, whatever <c>WithClustering</c> emits, and anything added explicitly — and which
+///         one wins is not readable off the <see cref="HoconAddMode"/> names alone. A downing
+///         setting can therefore be right in the resource file, right in the typed options object,
+///         and still not be the one in force. These tests start a real host through the production
+///         bootstrap and read <c>akka.cluster.downing-provider-class</c> back off
+///         <see cref="ActorSystem.Settings"/>, which is the only assertion precedence cannot fool.
+///     </para>
+///     <para>
+///         <b>Why the default is auto-down.</b> A two-node cluster running the SBR
+///         <c>keep-oldest</c> resolver cannot survive a crash of the oldest node. Akka.NET 1.5.62's
+///         <c>KeepOldest.OldestDecision</c> only lets <c>down-if-alone</c> rescue a side holding
+///         &gt;= 2 members, so in a 1-vs-1 split the lone survivor takes <c>DownReachable</c> and
+///         downs <i>itself</i> — then <c>run-coordinated-shutdown-when-down</c> terminates it. A
+///         crash of one node becomes a total outage: exactly the failure the redundancy pair exists
+///         to absorb. Proven live on the sister project's rig (see
+///         <c>../ScadaBridge/docs/plans/2026-07-21-auto-down-availability-decision.md</c>) and
+///         adopted here for the same reason. The accepted trade is dual-active during a genuine
+///         network partition; see <c>docs/Redundancy.md</c>.
+///     </para>
+/// </remarks>
 public sealed class SplitBrainResolverActivationTests
 {
-    private static AkkaClusterOptions SampleOptions() => new()
+    private static AkkaClusterOptions SampleOptions(string? strategy = null)
    {
-        SeedNodes = new[] { "akka.tcp://otopcua@127.0.0.1:4053" },
-        Roles = new[] { "admin", "driver" },
-    };
+        var options = new AkkaClusterOptions
+        {
+            SeedNodes = new[] { "akka.tcp://otopcua@127.0.0.1:4053" },
+            Roles = new[] { "admin", "driver" },
+        };
+        if (strategy is not null)
+        {
+            options.SplitBrainResolverStrategy = strategy;
+        }

-    /// <summary>Verifies the cluster options carry an explicit split-brain resolver strategy.</summary>
-    [Fact]
-    public void BuildClusterOptions_sets_an_explicit_split_brain_resolver()
-    {
-        var options = ServiceCollectionExtensions.BuildClusterOptions(SampleOptions());
-
-        options.SplitBrainResolver.ShouldNotBeNull(
-            "SplitBrainResolver must be set explicitly in code — keeps the strategy independent of Akka.Cluster.Hosting's default provider rather than relying on it.");
+        return options;
    }

-    /// <summary>Verifies the resolver is keep-oldest with down-if-alone (correct for a 2-node pair).</summary>
-    [Fact]
-    public void BuildClusterOptions_uses_keep_oldest_with_down_if_alone()
+    /// <summary>
+    /// Starts a real host through <see cref="ServiceCollectionExtensions.WithOtOpcUaClusterBootstrap"/>
+    /// — the production entry point, not a re-creation of it — and returns the ActorSystem's
+    /// effective configuration.
+    /// </summary>
+    /// <remarks>
+    /// Calling the production bootstrap rather than inlining the calls it makes is deliberate: an
+    /// earlier draft of this helper rebuilt the same three steps itself, which pinned the
+    /// <i>test's</i> wiring instead of the shipped one and would have stayed green through a change
+    /// to production's HOCON precedence.
+    /// </remarks>
+    private static async Task<Config> EffectiveConfigAsync(AkkaClusterOptions options)
    {
-        var options = ServiceCollectionExtensions.BuildClusterOptions(SampleOptions());
+        // Port 0 so concurrently-running test hosts cannot collide on 4053, and no seed nodes so the
+        // system never tries to join a cluster.
+        options.Port = 0;
+        options.Hostname = "127.0.0.1";
+        options.PublicHostname = "127.0.0.1";
+        options.SeedNodes = Array.Empty<string>();

-        var keepOldest = options.SplitBrainResolver.ShouldBeOfType<KeepOldestOption>();
-        keepOldest.DownIfAlone.ShouldBe(true);
+        var builder = Host.CreateDefaultBuilder();
+        builder.ConfigureServices(services =>
+        {
+            services.AddSingleton<IOptions<AkkaClusterOptions>>(Options.Create(options));
+            services.AddAkka("otopcua-sbr-test", (ab, sp) => ab.WithOtOpcUaClusterBootstrap(sp));
+        });
+
+        using var host = builder.Build();
+        await host.StartAsync();
+        try
+        {
+            return host.Services.GetRequiredService<ActorSystem>().Settings.Config;
+        }
+        finally
+        {
+            await host.StopAsync();
+        }
+    }
+
+    /// <summary>
+    /// The default strategy installs Akka's AutoDowning provider — the leader among the
+    /// <i>reachable</i> members downs the unreachable peer, so a crash of either node fails over.
+    /// </summary>
+    [Fact]
+    public async Task Default_strategy_installs_the_auto_downing_provider()
+    {
+        var config = await EffectiveConfigAsync(SampleOptions());
+
+        config.GetString("akka.cluster.downing-provider-class")
+            .ShouldStartWith(
+                "Akka.Cluster.AutoDowning, Akka.Cluster",
+                Case.Sensitive,
+                "the effective downing provider — not merely the configured one — must be AutoDowning, or a crash of the oldest node is a total outage");
+    }
+
+    /// <summary>
+    /// AutoDowning is inert without a downing window: <c>auto-down-unreachable-after</c> defaults to
+    /// <c>off</c>, under which the provider is installed but never downs anything. Selecting the
+    /// provider and forgetting the window would look correct and fail over never.
+    /// </summary>
+    [Fact]
+    public async Task Default_strategy_sets_the_auto_down_window()
+    {
+        var config = await EffectiveConfigAsync(SampleOptions());
+
+        config.GetTimeSpan("akka.cluster.auto-down-unreachable-after")
+            .ShouldBe(ServiceCollectionExtensions.DowningStableAfter);
+    }
+
+    /// <summary>
+    /// The opt-out is real: an operator who chooses partition-safety over availability gets the SBR
+    /// keep-oldest resolver back, in force rather than merely configured.
+    /// </summary>
+    [Fact]
+    public async Task Keep_oldest_strategy_installs_the_sbr_provider()
+    {
+        var config = await EffectiveConfigAsync(SampleOptions("keep-oldest"));
+
+        config.GetString("akka.cluster.downing-provider-class")
+            .ShouldStartWith("Akka.Cluster.SBR.SplitBrainResolverProvider, Akka.Cluster");
+        config.GetString("akka.cluster.split-brain-resolver.active-strategy").ShouldBe("keep-oldest");
+        config.GetBoolean("akka.cluster.split-brain-resolver.keep-oldest.down-if-alone").ShouldBeTrue();
+    }
+
+    /// <summary>The default is auto-down, so an unconfigured deployment gets the survivable one.</summary>
+    [Fact]
+    public void Strategy_defaults_to_auto_down()
+    {
+        new AkkaClusterOptions().SplitBrainResolverStrategy.ShouldBe("auto-down");
+    }
+
+    /// <summary>
+    /// A typo in the strategy must fail the host at startup, not silently fall through to whichever
+    /// provider Akka.Hosting would have installed by default — which is the fatal one.
+    /// </summary>
+    [Fact]
+    public void Unknown_strategy_throws_rather_than_silently_falling_back()
+    {
+        var ex = Should.Throw<ArgumentOutOfRangeException>(
+            () => ServiceCollectionExtensions.BuildDowningHocon(SampleOptions("keep-majority")));
+
+        ex.Message.ShouldContain("keep-majority");
+        ex.Message.ShouldContain("auto-down");
+    }
+
+    /// <summary>Strategy comparison is case-insensitive — config casing is not a failure mode.</summary>
+    [Fact]
+    public void Strategy_matching_is_case_insensitive()
+    {
+        Should.NotThrow(() => ServiceCollectionExtensions.BuildDowningHocon(SampleOptions("Auto-Down")));
+        Should.NotThrow(() => ServiceCollectionExtensions.BuildDowningHocon(SampleOptions("Keep-Oldest")));
+    }
+
+    /// <summary>
+    /// Under keep-oldest the SBR resolver is still set as a typed option, so the strategy stays
+    /// explicit in code rather than depending on Akka.Cluster.Hosting's default.
+    /// </summary>
+    [Fact]
+    public void Keep_oldest_sets_the_typed_resolver_option()
+    {
+        var options = ServiceCollectionExtensions.BuildClusterOptions(SampleOptions("keep-oldest"));
+
+        options.SplitBrainResolver.ShouldBeOfType<KeepOldestOption>().DownIfAlone.ShouldBe(true);
+    }
+
+    /// <summary>
+    /// Pins the two stability windows together. <c>auto-down-unreachable-after</c> comes from
+    /// <see cref="ServiceCollectionExtensions.DowningStableAfter"/> while the SBR resolver reads
+    /// <c>stable-after</c> out of akka.conf; changing one and missing the other would leave the two
+    /// strategies failing over at different latencies for no stated reason.
+    /// </summary>
+    [Fact]
+    public void Downing_window_matches_the_akka_conf_stable_after()
+    {
+        var akkaConf = ConfigurationFactory.ParseString(HoconLoader.LoadBaseConfig());
+
+        akkaConf.GetTimeSpan("akka.cluster.split-brain-resolver.stable-after")
+            .ShouldBe(ServiceCollectionExtensions.DowningStableAfter);
+    }
+
+    /// <summary>
+    /// The stability window must stay above the failure detector's tolerance, or a node gets downed
+    /// while its peer is merely slow.
+    /// </summary>
+    [Fact]
+    public void Downing_window_exceeds_the_acceptable_heartbeat_pause()
+    {
+        var akkaConf = ConfigurationFactory.ParseString(HoconLoader.LoadBaseConfig());
+
+        ServiceCollectionExtensions.DowningStableAfter
+            .ShouldBeGreaterThan(akkaConf.GetTimeSpan("akka.cluster.failure-detector.acceptable-heartbeat-pause"));
    }

    /// <summary>Verifies seed nodes and roles are still threaded through unchanged.</summary>
@@ -3,6 +3,7 @@ using Shouldly;
 using Xunit;
 using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
 using ZB.MOM.WW.OtOpcUa.Core.Hosting;
+using ZB.MOM.WW.Secrets.Abstractions;

 namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests;

@@ -23,13 +24,40 @@ public sealed class DriverTypeNamesGuardTests
    private static readonly string[] NonFactoryDriverAssemblySuffixes =
        { ".Contracts", ".Addressing", ".Cli" };

+    /// <summary>
+    ///     A stand-in <see cref="ISecretResolver"/> for factories that require one.
+    /// </summary>
+    /// <remarks>
+    ///     Every secret resolves to absent. That is deliberate and safe here: the test only needs the
+    ///     type-name key each <c>Register</c> call installs, and never invokes the registered factory
+    ///     func, so no secret is ever requested. Resolving to absent rather than to a fake value means
+    ///     that if this assumption is ever broken, the driver fails closed.
+    /// </remarks>
+    private sealed class AbsentSecretResolver : ISecretResolver
+    {
+        public static readonly AbsentSecretResolver Instance = new();
+
+        public Task<string?> GetAsync(SecretName name, CancellationToken ct) =>
+            Task.FromResult<string?>(null);
+    }
+
    /// <summary>
    ///     Build the authoritative registered-factory set by discovering every
    ///     <c>*DriverFactoryExtensions.Register</c> in the deployed driver assemblies (by convention) and
-    ///     invoking it against a fresh registry. The factory func is never invoked — we only need the
-    ///     type-name key each <c>Register</c> call installs — so <see langword="null"/> is passed for the
-    ///     optional <c>loggerFactory</c>/other parameters.
+    ///     invoking it against a fresh registry. The factory func is never invoked — only the type-name
+    ///     key each <c>Register</c> call installs is needed.
    /// </summary>
+    /// <remarks>
+    ///     Argument construction is by parameter <i>type</i>, not by position. It used to pass
+    ///     <see langword="null"/> for everything after the registry on the assumption that the
+    ///     remaining parameters were all optional. They are not: <c>GalaxyDriverFactoryExtensions.Register</c>
+    ///     takes a <b>required</b> <see cref="ISecretResolver"/> and guards it with
+    ///     <c>ArgumentNullException.ThrowIfNull</c>, so the reflective call threw and all three parity
+    ///     facts failed — for a reason that had nothing to do with the drift they exist to catch.
+    ///     (<c>OpcUaClient</c>'s equivalent parameter is optional, which is why only Galaxy tripped it.)
+    ///     A parameter this method cannot satisfy now fails with a message naming the factory and the
+    ///     parameter, rather than surfacing as an opaque <see cref="ArgumentNullException"/>.
+    /// </remarks>
    private static IReadOnlyCollection<string> RegisteredDriverTypeNames()
    {
        var registry = new DriverFactoryRegistry();
@@ -62,8 +90,7 @@ public sealed class DriverTypeNamesGuardTests
                    || ps[0].ParameterType != typeof(DriverFactoryRegistry))
                    continue;

-                var args = new object?[ps.Length];
-                args[0] = registry; // remaining params (optional loggerFactory, etc.) default to null
+                var args = ps.Select(p => ArgumentFor(p, registry, type.Name)).ToArray();
                register.Invoke(null, args);
                invoked++;
            }
@@ -78,6 +105,29 @@ public sealed class DriverTypeNamesGuardTests
        return registry.RegisteredTypes;
    }

+    /// <summary>
+    ///     Supplies one argument for a discovered <c>Register</c> parameter, by type.
+    /// </summary>
+    /// <param name="parameter">The parameter to supply.</param>
+    /// <param name="registry">The registry the factories register into.</param>
+    /// <param name="factoryTypeName">The declaring factory type, for the failure message.</param>
+    /// <returns>The argument value to pass.</returns>
+    private static object? ArgumentFor(ParameterInfo parameter, DriverFactoryRegistry registry, string factoryTypeName)
+    {
+        if (parameter.ParameterType == typeof(DriverFactoryRegistry)) return registry;
+        if (parameter.ParameterType == typeof(ISecretResolver)) return AbsentSecretResolver.Instance;
+
+        // Optional or nullable-reference parameters are safe to omit; the factory is documented to
+        // cope. Anything else is a required dependency this method does not know how to build, and
+        // guessing null would resurface as an opaque ArgumentNullException from inside the factory.
+        if (parameter.IsOptional || !parameter.ParameterType.IsValueType) return null;
+
+        throw new InvalidOperationException(
+            $"{factoryTypeName}.Register requires parameter '{parameter.Name}' of type "
+            + $"{parameter.ParameterType.Name}, which the DriverTypeNames guard does not know how to "
+            + "supply. Add a case to ArgumentFor, or give the parameter a default on the factory.");
+    }
+
    /// <summary>Reflect every <c>public const string</c> declared on <see cref="DriverTypeNames"/>.</summary>
    private static IReadOnlyCollection<string> DeclaredConstantValues() =>
        typeof(DriverTypeNames)
@@ -0,0 +1,184 @@
+using Akka.Actor;
+using Akka.Cluster;
+using Akka.Configuration;
+using Shouldly;
+using Xunit;
+using ZB.MOM.WW.OtOpcUa.Commons.Messages.Redundancy;
+using ZB.MOM.WW.OtOpcUa.ControlPlane.Redundancy;
+
+namespace ZB.MOM.WW.OtOpcUa.ControlPlane.Tests;
+
+/// <summary>
+/// Pins <b>which</b> node <see cref="RedundancyStateActor"/> elects Primary, using a real two-node
+/// cluster deliberately built so that the oldest member is <i>not</i> the lowest-addressed one.
+/// </summary>
+/// <remarks>
+///     <para>
+///         The distinction is the whole point. Akka offers two different "first" members and they
+///         disagree after any restart:
+///     </para>
+///     <list type="bullet">
+///         <item>
+///             <description>
+///                 <c>ClusterState.RoleLeader(role)</c> — the <b>lowest-addressed</b> Up member with
+///                 that role. Address order is host, then port; it has nothing to do with time.
+///             </description>
+///         </item>
+///         <item>
+///             <description>
+///                 <b>Oldest</b> — the Up member with the lowest up-number, i.e. the one that has been
+///                 in the cluster longest. This is what <c>ClusterSingletonManager</c> uses to place
+///                 singletons, and (under <c>keep-oldest</c>) what the split-brain resolver keys on.
+///             </description>
+///         </item>
+///     </list>
+///     <para>
+///         They coincide on a freshly-formed cluster, which is why single-node and
+///         happy-path tests never caught the difference. They diverge the moment a node restarts: the
+///         restarted node becomes the <i>youngest</i> while keeping its address, so if it happens to
+///         hold the lower address it becomes role leader — and the actor would have named it Primary
+///         while the cluster singletons, and every piece of work they own, stayed on the other node.
+///         The Primary-gated data plane (writes, alarm acks, alerts emit, alarm-history drain) would
+///         then be enabled on the node that is not hosting the work.
+///     </para>
+///     <para>
+///         This test reproduces that divergence directly rather than waiting for a restart: node A
+///         binds the <b>higher</b> port and joins first (so it is oldest), node B binds the
+///         <b>lower</b> port and joins second (so it is role leader). The correct answer is A.
+///     </para>
+/// </remarks>
+public sealed class RedundancyPrimaryElectionTests : IAsyncLifetime
+{
+    // Fixed, unusual ports: the test needs a deterministic address ordering, which port 0 cannot give.
+    private const int OldestPort = 19_531;   // joins FIRST  -> oldest, but the HIGHER address
+    private const int YoungestPort = 19_530; // joins SECOND -> role leader, the LOWER address
+
+    private ActorSystem? _oldest;
+    private ActorSystem? _youngest;
+
+    private static Config NodeConfig(int port) => ConfigurationFactory.ParseString($$"""
+        akka {
+            loglevel = "WARNING"
+            actor.provider = "Akka.Cluster.ClusterActorRefProvider, Akka.Cluster"
+            remote.dot-netty.tcp {
+                hostname = "127.0.0.1"
+                port = {{port}}
+            }
+            cluster {
+                seed-nodes = ["akka.tcp://redundancy-election@127.0.0.1:{{OldestPort}}"]
+                roles = ["admin", "driver"]
+                min-nr-of-members = 1
+                run-coordinated-shutdown-when-down = off
+                downing-provider-class = ""
+            }
+        }
+        """);
+
+    public async Task InitializeAsync()
+    {
+        // Order matters: the seed forms the cluster and is therefore the oldest member.
+        _oldest = ActorSystem.Create("redundancy-election", NodeConfig(OldestPort));
+        await WaitForUpAsync(_oldest, expectedMembers: 1);
+
+        _youngest = ActorSystem.Create("redundancy-election", NodeConfig(YoungestPort));
+        await WaitForUpAsync(_oldest, expectedMembers: 2);
+        await WaitForUpAsync(_youngest, expectedMembers: 2);
+    }
+
+    public async Task DisposeAsync()
+    {
+        if (_youngest is not null) await _youngest.Terminate();
+        if (_oldest is not null) await _oldest.Terminate();
+    }
+
+    private static async Task WaitForUpAsync(ActorSystem system, int expectedMembers)
+    {
+        var cluster = Akka.Cluster.Cluster.Get(system);
+        var deadline = DateTime.UtcNow + TimeSpan.FromSeconds(30);
+        while (DateTime.UtcNow < deadline)
+        {
+            var up = cluster.State.Members.Count(m => m.Status == MemberStatus.Up);
+            if (up >= expectedMembers) return;
+            await Task.Delay(100);
+        }
+
+        throw new TimeoutException(
+            $"cluster on port {cluster.SelfAddress.Port} never saw {expectedMembers} Up members "
+            + $"(saw {cluster.State.Members.Count(m => m.Status == MemberStatus.Up)})");
+    }
+
+    /// <summary>
+    /// Sanity check on the fixture itself: if the two orderings did not actually diverge, the real
+    /// assertion below would pass for the wrong reason. This asserts the setup produced the
+    /// divergence it claims to — role leader is the younger node, not the oldest one.
+    /// </summary>
+    [Fact]
+    public void Fixture_actually_produces_divergent_age_and_address_ordering()
+    {
+        var state = Akka.Cluster.Cluster.Get(_oldest!).State;
+
+        state.RoleLeader("driver")!.Port.ShouldBe(
+            YoungestPort,
+            "the fixture is only meaningful if the lowest-addressed node is the YOUNGER one");
+    }
+
+    /// <summary>
+    /// The elected Primary is the oldest driver member — the node that hosts the cluster singletons —
+    /// not the lowest-addressed one.
+    /// </summary>
+    [Fact]
+    public async Task Primary_is_the_oldest_driver_member_not_the_role_leader()
+    {
+        var snapshot = await CaptureSnapshotAsync();
+
+        var primaries = snapshot.Where(n => n.Role == RedundancyRole.Primary).ToList();
+        primaries.Count.ShouldBe(1, "exactly one node may be Primary");
+        primaries[0].NodeId.Value.ShouldBe(
+            $"127.0.0.1:{OldestPort}",
+            "Primary must follow cluster-singleton placement (oldest Up member), or the Primary-gated data plane is enabled on a node that is not hosting the work");
+    }
+
+    /// <summary>The younger node is Secondary, not a second Primary and not Detached.</summary>
+    [Fact]
+    public async Task Younger_driver_member_is_secondary()
+    {
+        var snapshot = await CaptureSnapshotAsync();
+
+        snapshot.Single(n => n.NodeId.Value == $"127.0.0.1:{YoungestPort}")
+            .Role.ShouldBe(RedundancyRole.Secondary);
+    }
+
+    /// <summary>
+    /// <c>IsDriverPrimary</c> agrees with <c>Role</c>. They feed different consumers — the OPC UA
+    /// ServiceLevel calculation reads the flag while the data-plane gates read the role — so a
+    /// disagreement would advertise one node as authoritative while gating writes on the other.
+    /// </summary>
+    [Fact]
+    public async Task IsDriverPrimary_flag_agrees_with_the_role()
+    {
+        var snapshot = await CaptureSnapshotAsync();
+
+        foreach (var node in snapshot)
+        {
+            node.IsDriverPrimary.ShouldBe(
+                node.Role == RedundancyRole.Primary,
+                $"{node.NodeId.Value} disagrees between Role and IsDriverPrimary");
+        }
+    }
+
+    private async Task<IReadOnlyList<NodeRedundancyState>> CaptureSnapshotAsync()
+    {
+        var tcs = new TaskCompletionSource<RedundancyStateChanged>(
+            TaskCreationOptions.RunContinuationsAsynchronously);
+
+        _oldest!.ActorOf(
+            RedundancyStateActor.Props(broadcast: msg =>
+            {
+                if (msg is RedundancyStateChanged changed) tcs.TrySetResult(changed);
+            }),
+            $"redundancy-{Guid.NewGuid():N}");
+
+        var published = await tcs.Task.WaitAsync(TimeSpan.FromSeconds(15));
+        return published.Nodes;
+    }
+}
@@ -17,7 +17,7 @@ public sealed class ServiceLevelCalculatorTests
    public void NotUp_returns_zero(MemberStatus status)
    {
        var sl = ServiceLevelCalculator.Compute(new(status,
-            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverRoleLeader: true));
+            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverPrimary: true));
        sl.ShouldBe((byte)0);
    }

@@ -26,7 +26,7 @@ public sealed class ServiceLevelCalculatorTests
    public void Fully_healthy_non_leader_returns_240()
    {
        var sl = ServiceLevelCalculator.Compute(new(MemberStatus.Up,
-            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverRoleLeader: false));
+            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverPrimary: false));
        sl.ShouldBe((byte)240);
    }

@@ -35,7 +35,7 @@ public sealed class ServiceLevelCalculatorTests
    public void Fully_healthy_role_leader_returns_250()
    {
        var sl = ServiceLevelCalculator.Compute(new(MemberStatus.Up,
-            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverRoleLeader: true));
+            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverPrimary: true));
        sl.ShouldBe((byte)250);
    }

@@ -44,7 +44,7 @@ public sealed class ServiceLevelCalculatorTests
    public void Db_reachable_but_stale_returns_200()
    {
        var sl = ServiceLevelCalculator.Compute(new(MemberStatus.Up,
-            DbReachable: true, OpcUaProbeOk: true, Stale: true, IsDriverRoleLeader: false));
+            DbReachable: true, OpcUaProbeOk: true, Stale: true, IsDriverPrimary: false));
        sl.ShouldBe((byte)200);
    }

@@ -53,7 +53,7 @@ public sealed class ServiceLevelCalculatorTests
    public void Db_unreachable_and_stale_returns_100()
    {
        var sl = ServiceLevelCalculator.Compute(new(MemberStatus.Up,
-            DbReachable: false, OpcUaProbeOk: false, Stale: true, IsDriverRoleLeader: false));
+            DbReachable: false, OpcUaProbeOk: false, Stale: true, IsDriverPrimary: false));
        sl.ShouldBe((byte)100);
    }

@@ -63,7 +63,7 @@ public sealed class ServiceLevelCalculatorTests
    {
        // (DbReachable=true, OpcUaProbeOk=false, Stale=false) falls through to the catch-all 0.
        var sl = ServiceLevelCalculator.Compute(new(MemberStatus.Up,
-            DbReachable: true, OpcUaProbeOk: false, Stale: false, IsDriverRoleLeader: false));
+            DbReachable: true, OpcUaProbeOk: false, Stale: false, IsDriverPrimary: false));
        sl.ShouldBe((byte)0);
    }

@@ -72,7 +72,7 @@ public sealed class ServiceLevelCalculatorTests
    public void Joining_member_is_treated_like_Up_for_grading()
    {
        var sl = ServiceLevelCalculator.Compute(new(MemberStatus.Joining,
-            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverRoleLeader: false));
+            DbReachable: true, OpcUaProbeOk: true, Stale: false, IsDriverPrimary: false));
        sl.ShouldBe((byte)240);
    }
 }
@@ -106,7 +106,7 @@ public sealed class DriverHostActorNativeAlarmAckRoutingTests : RuntimeActorTest
            new[]
            {
                new NodeRedundancyState(TestNode, RedundancyRole.Secondary,
-                    IsClusterLeader: false, IsRoleLeaderForDriver: false, AsOfUtc: DateTime.UtcNow),
+                    IsClusterLeader: false, IsDriverPrimary: false, AsOfUtc: DateTime.UtcNow),
            },
            CorrelationId.NewId()));

@@ -330,7 +330,7 @@ public sealed class DriverHostActorNativeAlarmTests : RuntimeActorTestBase
                    NodeId: TestNode,
                    Role: role,
                    IsClusterLeader: role == RedundancyRole.Primary,
-                    IsRoleLeaderForDriver: role == RedundancyRole.Primary,
+                    IsDriverPrimary: role == RedundancyRole.Primary,
                    AsOfUtc: DateTime.UtcNow),
            },
            CorrelationId.NewId()));
@@ -216,7 +216,7 @@ public sealed class DriverHostActorPrimaryGateTests : RuntimeActorTestBase
            {
                new NodeRedundancyState(TestNode, role,
                    IsClusterLeader: role == RedundancyRole.Primary,
-                    IsRoleLeaderForDriver: role == RedundancyRole.Primary,
+                    IsDriverPrimary: role == RedundancyRole.Primary,
                    AsOfUtc: DateTime.UtcNow),
            },
            CorrelationId.NewId()));
@@ -66,10 +66,10 @@ public sealed class DriverHostActorRoleViewTests : RuntimeActorTestBase
        SpawnHost(view, driverMemberCount: 4).Tell(new RedundancyStateChanged(
            [
                new NodeRedundancyState(TestNode, RedundancyRole.Secondary,
-                    IsClusterLeader: false, IsRoleLeaderForDriver: false, AsOfUtc: DateTime.UtcNow),
+                    IsClusterLeader: false, IsDriverPrimary: false, AsOfUtc: DateTime.UtcNow),
                // A Primary DOES exist — in another pair. It is not this node's replication peer.
                new NodeRedundancyState(NodeId.Parse("some-other-pair:4053"), RedundancyRole.Primary,
-                    IsClusterLeader: true, IsRoleLeaderForDriver: true, AsOfUtc: DateTime.UtcNow),
+                    IsClusterLeader: true, IsDriverPrimary: true, AsOfUtc: DateTime.UtcNow),
            ],
            CorrelationId.NewId()));

@@ -198,13 +198,13 @@ public sealed class DriverHostActorRoleViewTests : RuntimeActorTestBase
            [
                new NodeRedundancyState(node, role,
                    IsClusterLeader: role == RedundancyRole.Primary,
-                    IsRoleLeaderForDriver: role == RedundancyRole.Primary,
+                    IsDriverPrimary: role == RedundancyRole.Primary,
                    AsOfUtc: DateTime.UtcNow),
                new NodeRedundancyState(
                    NodeId.Parse($"{PeerHost}:4053"),
                    role == RedundancyRole.Primary ? RedundancyRole.Secondary : RedundancyRole.Primary,
                    IsClusterLeader: role != RedundancyRole.Primary,
-                    IsRoleLeaderForDriver: role != RedundancyRole.Primary,
+                    IsDriverPrimary: role != RedundancyRole.Primary,
                    AsOfUtc: DateTime.UtcNow),
            ],
            CorrelationId.NewId());
@@ -106,7 +106,7 @@ public sealed class DriverHostActorWriteRoutingTests : RuntimeActorTestBase
            new[]
            {
                new NodeRedundancyState(TestNode, RedundancyRole.Secondary,
-                    IsClusterLeader: false, IsRoleLeaderForDriver: false, AsOfUtc: DateTime.UtcNow),
+                    IsClusterLeader: false, IsDriverPrimary: false, AsOfUtc: DateTime.UtcNow),
            },
            CorrelationId.NewId()));

@@ -38,7 +38,7 @@ public sealed class PeerProbeSupervisorTests : RuntimeActorTestBase
        Akka.Actor.Props.Create(() => new RecordingNoopActor(spawned));

    private static NodeRedundancyState State(NodeId id, RedundancyRole role) =>
-        new(id, role, IsClusterLeader: false, IsRoleLeaderForDriver: false, DateTime.UtcNow);
+        new(id, role, IsClusterLeader: false, IsDriverPrimary: false, DateTime.UtcNow);

    private static RedundancyStateChanged Snapshot(params NodeRedundancyState[] nodes) =>
        new(nodes, CorrelationId.NewId());
@@ -85,7 +85,7 @@ public sealed class HistorianAdapterActorTests : RuntimeActorTestBase
                    NodeId: LocalNode,
                    Role: role,
                    IsClusterLeader: role == RedundancyRole.Primary,
-                    IsRoleLeaderForDriver: role == RedundancyRole.Primary,
+                    IsDriverPrimary: role == RedundancyRole.Primary,
                    AsOfUtc: DateTime.UtcNow),
            },
            CorrelationId.NewId()));
@@ -166,7 +166,7 @@ public sealed class HistorianAdapterActorTests : RuntimeActorTestBase
                    NodeId: new NodeId("some-other-node"),
                    Role: RedundancyRole.Secondary,
                    IsClusterLeader: false,
-                    IsRoleLeaderForDriver: false,
+                    IsDriverPrimary: false,
                    AsOfUtc: DateTime.UtcNow),
            },
            CorrelationId.NewId()));
@@ -181,7 +181,7 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
            Nodes: new[]
            {
                new NodeRedundancyState(local, RedundancyRole.Detached,
-                    IsClusterLeader: false, IsRoleLeaderForDriver: false, DateTime.UtcNow),
+                    IsClusterLeader: false, IsDriverPrimary: false, DateTime.UtcNow),
            },
            CorrelationId.NewId()));

@@ -202,9 +202,9 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
            Nodes: new[]
            {
                new NodeRedundancyState(local, RedundancyRole.Primary,
-                    IsClusterLeader: true, IsRoleLeaderForDriver: true, DateTime.UtcNow),
+                    IsClusterLeader: true, IsDriverPrimary: true, DateTime.UtcNow),
                new NodeRedundancyState(NodeId.Parse("other-node"), RedundancyRole.Secondary,
-                    IsClusterLeader: false, IsRoleLeaderForDriver: false, DateTime.UtcNow),
+                    IsClusterLeader: false, IsDriverPrimary: false, DateTime.UtcNow),
            },
            CorrelationId.NewId());
        actor.Tell(snapshot);
@@ -225,7 +225,7 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
            Nodes: new[]
            {
                new NodeRedundancyState(local, RedundancyRole.Secondary,
-                    IsClusterLeader: false, IsRoleLeaderForDriver: false, DateTime.UtcNow),
+                    IsClusterLeader: false, IsDriverPrimary: false, DateTime.UtcNow),
            },
            CorrelationId.NewId());
        actor.Tell(snapshot);
@@ -251,7 +251,7 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
            Nodes: new[]
            {
                new NodeRedundancyState(local, RedundancyRole.Primary,
-                    IsClusterLeader: true, IsRoleLeaderForDriver: true, DateTime.UtcNow),
+                    IsClusterLeader: true, IsDriverPrimary: true, DateTime.UtcNow),
            },
            CorrelationId.NewId()));

@@ -277,7 +277,7 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
            Nodes: new[]
            {
                new NodeRedundancyState(local, RedundancyRole.Secondary,
-                    IsClusterLeader: false, IsRoleLeaderForDriver: false, DateTime.UtcNow),
+                    IsClusterLeader: false, IsDriverPrimary: false, DateTime.UtcNow),
            },
            CorrelationId.NewId()));

@@ -302,7 +302,7 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
            Nodes: new[]
            {
                new NodeRedundancyState(local, RedundancyRole.Secondary,
-                    IsClusterLeader: false, IsRoleLeaderForDriver: false, DateTime.UtcNow),
+                    IsClusterLeader: false, IsDriverPrimary: false, DateTime.UtcNow),
            },
            CorrelationId.NewId()));

@@ -329,7 +329,7 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
            Nodes: new[]
            {
                new NodeRedundancyState(local, RedundancyRole.Secondary,
-                    IsClusterLeader: false, IsRoleLeaderForDriver: false,
+                    IsClusterLeader: false, IsDriverPrimary: false,
                    DateTime.UtcNow - TimeSpan.FromMinutes(1)),
            },
            CorrelationId.NewId()));
@@ -358,7 +358,7 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
            Nodes: new[]
            {
                new NodeRedundancyState(local, RedundancyRole.Primary,
-                    IsClusterLeader: true, IsRoleLeaderForDriver: true, DateTime.UtcNow),
+                    IsClusterLeader: true, IsDriverPrimary: true, DateTime.UtcNow),
            },
            CorrelationId.NewId()));
        AwaitAssert(() => publisher.Levels.ShouldContain((byte)250),
@@ -369,7 +369,7 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
            Nodes: new[]
            {
                new NodeRedundancyState(local, RedundancyRole.Detached,
-                    IsClusterLeader: false, IsRoleLeaderForDriver: false, DateTime.UtcNow),
+                    IsClusterLeader: false, IsDriverPrimary: false, DateTime.UtcNow),
            },
            CorrelationId.NewId()));

@@ -398,7 +398,7 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
            Nodes: new[]
            {
                new NodeRedundancyState(local, RedundancyRole.Secondary,
-                    IsClusterLeader: false, IsRoleLeaderForDriver: false, DateTime.UtcNow),
+                    IsClusterLeader: false, IsDriverPrimary: false, DateTime.UtcNow),
            },
            CorrelationId.NewId()));

@@ -441,7 +441,7 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
            Nodes: new[]
            {
                new NodeRedundancyState(local, RedundancyRole.Primary,
-                    IsClusterLeader: true, IsRoleLeaderForDriver: true, DateTime.UtcNow),
+                    IsClusterLeader: true, IsDriverPrimary: true, DateTime.UtcNow),
            },
            CorrelationId.NewId()));

@@ -469,7 +469,7 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
            Nodes: new[]
            {
                new NodeRedundancyState(local, RedundancyRole.Primary,
-                    IsClusterLeader: true, IsRoleLeaderForDriver: true, DateTime.UtcNow),
+                    IsClusterLeader: true, IsDriverPrimary: true, DateTime.UtcNow),
            },
            CorrelationId.NewId()));

@@ -498,7 +498,7 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
            Nodes: new[]
            {
                new NodeRedundancyState(local, RedundancyRole.Secondary,
-                    IsClusterLeader: false, IsRoleLeaderForDriver: false, DateTime.UtcNow),
+                    IsClusterLeader: false, IsDriverPrimary: false, DateTime.UtcNow),
            },
            CorrelationId.NewId()));

@@ -526,7 +526,7 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
            Nodes: new[]
            {
                new NodeRedundancyState(local, RedundancyRole.Primary,
-                    IsClusterLeader: true, IsRoleLeaderForDriver: true, DateTime.UtcNow),
+                    IsClusterLeader: true, IsDriverPrimary: true, DateTime.UtcNow),
            },
            CorrelationId.NewId()));

@@ -548,7 +548,7 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
            Nodes: new[]
            {
                new NodeRedundancyState(local, RedundancyRole.Primary,
-                    IsClusterLeader: true, IsRoleLeaderForDriver: true, DateTime.UtcNow),
+                    IsClusterLeader: true, IsDriverPrimary: true, DateTime.UtcNow),
            },
            CorrelationId.NewId()));

@@ -574,7 +574,7 @@ public sealed class OpcUaPublishActorTests : RuntimeActorTestBase
        actor.Tell(new RedundancyStateChanged(new[]
        {
            new NodeRedundancyState(local, RedundancyRole.Primary,
-                IsClusterLeader: true, IsRoleLeaderForDriver: true, DateTime.UtcNow),
+                IsClusterLeader: true, IsDriverPrimary: true, DateTime.UtcNow),
        }, CorrelationId.NewId()));
        // No direct DbHealthStatus Tell — the periodic HealthTick Ask must populate it → 250.
        AwaitAssert(() => publisher.Levels.ShouldContain((byte)250), TimeSpan.FromSeconds(2));
@@ -51,7 +51,7 @@ public sealed class ServiceLevelEndToEndTests : RuntimeActorTestBase
            actor.Tell(new RedundancyStateChanged(
                Nodes: new[]
                {
-                    new NodeRedundancyState(localNode, RedundancyRole.Primary, IsClusterLeader: true, IsRoleLeaderForDriver: true, AsOfUtc: DateTime.UtcNow),
+                    new NodeRedundancyState(localNode, RedundancyRole.Primary, IsClusterLeader: true, IsDriverPrimary: true, AsOfUtc: DateTime.UtcNow),
                },
                CorrelationId: CorrelationId.NewId()));

@@ -91,7 +91,7 @@ public sealed class ServiceLevelEndToEndTests : RuntimeActorTestBase
            actor.Tell(new RedundancyStateChanged(
                Nodes: new[]
                {
-                    new NodeRedundancyState(localNode, RedundancyRole.Secondary, IsClusterLeader: false, IsRoleLeaderForDriver: false, AsOfUtc: DateTime.UtcNow),
+                    new NodeRedundancyState(localNode, RedundancyRole.Secondary, IsClusterLeader: false, IsDriverPrimary: false, AsOfUtc: DateTime.UtcNow),
                },
                CorrelationId: CorrelationId.NewId()));

@@ -102,7 +102,7 @@ public sealed class ScriptedAlarmHostActorTests : RuntimeActorTestBase
                    NodeId: id,
                    Role: role,
                    IsClusterLeader: role == RedundancyRole.Primary,
-                    IsRoleLeaderForDriver: role == RedundancyRole.Primary,
+                    IsDriverPrimary: role == RedundancyRole.Primary,
                    AsOfUtc: DateTime.UtcNow),
            },
            CorrelationId.NewId()));
Author	SHA1	Message	Date
Joseph Doherty	8e195263f6	docs(claude): record the two redundancy corrections in the project instructions v2-ci / build (push) Successful in 3m44s Details v2-ci / unit-tests (push) Failing after 9m55s Details Both change how someone should reason about this area, and the second changes a public message contract, so they belong in CLAUDE.md rather than only in docs/Redundancy.md. Includes the rule the work established: assert the effective cluster config off a running ActorSystem, never the typed options or the akka.conf text, because several HOCON fragments compete and the losing one still reads correctly in the file. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW	2026-07-21 19:05:32 -04:00
Joseph Doherty	0de1352e55	fix(deps): pin System.Security.Cryptography.Xml 10.0.10 — fresh restores were red Four NU1903 high-severity advisories (GHSA-23rf-6693-g89p, GHSA-8q5v-6pqq-x66h, GHSA-cvvh-rhrc-wg4q, GHSA-g8r8-53c2-pm3f) landed in the NuGet audit data against System.Security.Cryptography.Xml 10.0.7, which Microsoft.AspNetCore.DataProtection 10.0.7 pulls in transitively for key storage. Under TreatWarningsAsErrors a fresh restore produces 204 NU1903 errors and the solution does not build at all. This was invisible locally: machines with cached audit data keep building, so the break only shows on a clean clone or a docker image build. It surfaced here by accident — a detached worktree at the pre-change baseline, created to check whether a flaky test predated today's work, could not restore. Worth noting that the diagnostic path was the accident, not the intent: nothing in the normal build or test loop would have reported this before someone else hit it. Same surgical-pin pattern as the SQLitePCLRaw entry directly above it: a direct PackageReference at the one project where the chain enters the repo (Core.Configuration), rather than CentralPackageTransitivePinningEnabled, which this repo already documents as breaking the Roslyn version split. Bumping the DataProtection parent to 10.0.10 instead was rejected for the reason the sister repo recorded when it hit the same advisories: 10.0.10 floors Microsoft.Extensions.* and, via the EFCore adapter, Microsoft.EntityFrameworkCore at 10.0.10, forcing a family-wide servicing bump and an NU1605 downgrade cascade that deserves its own reviewed change. Verified the way the defect demanded — in a fresh worktree, not this one: restore clean (0 errors, down from 204) and full solution build clean. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW	2026-07-21 18:59:48 -04:00
Joseph Doherty	84c6524edc	docs(mesh): record phases 0a/0b as done and plan phase 1 The design doc still described the downing gap and the RoleLeader derivation as open. Both shipped earlier today, so 6.2 and 4 now carry the fix and its pin instead of the diagnosis, and 7 marks 0a/0b done. 0a's live gate stays open and is explicitly deferred to phase 7: the failure only appears in a 1-vs-1 split and docker-dev is a single six-node mesh, so there is no rig on which to drill it until the meshes are actually split. The phase 1 plan lands with one finding that changes its shape. The design pairs "ClusterNode gains address columns" with "coordinator sources its expected-ack set from the DB" as though they were one change; they are independent, and the second is far smaller than it reads. ClusterNode.NodeId is ALREADY in the coordinator's identifier space — the rig seeds central-1:4053 and so on, because NodeDeploymentState.NodeId is FK-bound to it and the coordinator's membership-derived ids already satisfy that FK. That the deploy path works today is standing proof the two sets agree. What is not small is the semantics. Membership-derived and DB-derived sets differ on a configured-but-stopped node: today the deployment seals green without it — telling the operator the fleet is deployed when it is not — and afterwards it would fail at the apply deadline. Failing is the better behaviour and is required once central cannot see cluster membership, but it needs an escape hatch or a node down for maintenance blocks every deploy. ClusterNode.Enabled already is that hatch, so the plan filters on it and gates the phase on a live check that proves both halves: a stopped node fails the deploy legibly, and disabling it lets the deploy seal. The plan also adds a task the design does not have: AkkaPort duplicates the node's own Cluster:Port with nothing making them agree, and an unenforced duplicated address is the same shape as the Modbus/ModbusTcp and TwinCat/Focas drifts already recorded here. Phases 2-7 remain unstarted. They change the running data path and rewrite the rig, and each needs its own plan and live gate. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW	2026-07-21 18:46:10 -04:00
Joseph Doherty	b2d19a1730	fix(tests): repair the DriverTypeNames guard, and stop MSBuild hoarding worker nodes Two independent housekeeping defects. DriverTypeNamesGuardTests (3 failures, pre-existing). The guard discovers each driver's *DriverFactoryExtensions.Register by reflection and invokes it, then asserts the registered type-name set matches the DriverTypeNames constants. It built the argument array positionally — registry first, null for everything after — on the assumption that the remaining parameters were optional. Galaxy's are not: its Register takes a REQUIRED ISecretResolver and guards it with ArgumentNullException.ThrowIfNull, so the reflective call threw and all three parity facts failed. OpcUaClient's equivalent parameter is optional, which is why only Galaxy tripped it. Worth stating plainly: the guard was not partially broken, it was completely inert. The throw happened while building the registered set, so NO driver's parity was ever checked — a genuine drift in any of the nine would have been invisible behind the same three red tests. Arguments are now supplied by parameter type, and a parameter the helper cannot satisfy fails with a message naming the factory and the parameter instead of an opaque ArgumentNullException. The stand-in resolver reports every secret absent, so if the "factory func is never invoked" assumption is ever broken, the driver fails closed. Core.Abstractions.Tests: 138 passed, 0 failed. MSBuild node reuse. MSBuild leaves worker nodes resident between builds so the next one starts warm; across this session's repeated solution builds and test runs that reached 55 idle processes holding ~6 GB. That is not just untidy — the integration suites assert on timeouts, so memory pressure produces failures indistinguishable from real regressions, the same failure mode the Workstation-GC fix addressed from a different direction. Directory.Build.rsp now passes -nodeReuse:false. Measured on the same incremental solution build: 15 residual nodes without it, 2 with, and elapsed time 6.12s vs 6.11s — no cost worth the memory on this machine. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW	2026-07-21 18:42:34 -04:00
Joseph Doherty	50b55ee4b9	fix(redundancy): elect the Primary by cluster age, not by address RedundancyStateActor derived the driver Primary from ClusterState.RoleLeader("driver"). Akka offers two different notions of a "first" member and they are not the same one: role leader is the lowest-ADDRESSED Up member (host, then port), while ClusterSingletonManager places singletons on the OLDEST — lowest up-number. They agree on a freshly-formed cluster, which is why every existing test passed. They diverge after any restart: the restarted node re-joins as the youngest while keeping its address, so if it holds the lower address it becomes role leader while the singletons stay put. The snapshot would then name a Primary that is not hosting the work, and every Primary-gated surface follows it — inbound device writes, native-alarm acks, the fleet-wide alerts emit, and the alarm-history drain would all enable on the wrong node while the node actually running the singletons stayed gated off. BuildSnapshot now selects the oldest Up member carrying the driver role, matching singleton placement. Leaving members are excluded: a node handing its singletons over must not be named Primary. NodeRedundancyState.IsRoleLeaderForDriver is renamed IsDriverPrimary, and NodeHealthInputs.IsDriverRoleLeader likewise. Keeping the old names would have left the wire contract asserting a derivation the code no longer uses — the same drift that made this defect invisible. Proven by a real two-node cluster rather than a mock. RedundancyPrimaryElectionTests binds the first-joining node to the HIGHER port, so oldest and lowest-address name different nodes, and includes a fixture assertion that the divergence actually occurred — without it the real assertion could pass for the wrong reason. Positive control: restoring the RoleLeader derivation turns exactly the two election tests red while the fixture check stays green. Runtime.Tests 440 passed, ControlPlane.Tests 82, Cluster.Tests 36. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW	2026-07-21 18:39:07 -04:00
Joseph Doherty	2964361a6f	fix(cluster): auto-down downing strategy — a two-node pair could not survive an oldest-node crash Ports the sister project's live-proven fix (ScadaBridge cf3bd52f). OtOpcUa ran the identical configuration it indicts: keep-oldest with down-if-alone = on. Akka.NET 1.5.62's KeepOldest.OldestDecision only lets the down-if-alone branch rescue a side holding >= 2 members. A two-node cluster losing a peer is always a 1-vs-1 split, so the branch never fires, the lone survivor falls through to DownReachable and downs ITSELF, and run-coordinated-shutdown-when-down then terminates it. down-if-alone is a 3+-node feature and does not do what its name suggests for a pair: a crash of the oldest node is a total outage, which is the exact failure the redundancy pair exists to absorb. Cluster:SplitBrainResolverStrategy now selects the provider, defaulting to auto-down: Akka's AutoDowning with auto-down-unreachable-after = 15s, so the leader among the reachable members downs the unreachable peer and a crash of either node fails over in place. keep-oldest remains available for deployments that would rather take an outage than ever run dual-active during a real partition. An unrecognised value fails the host at startup rather than falling through to the fatal default. The tests assert the EFFECTIVE configuration — they start a real host through WithOtOpcUaClusterBootstrap and read akka.cluster.downing-provider-class back off the running ActorSystem. This is not incidental. The first draft inlined the three calls the bootstrap makes instead of calling it, which pinned the test's own wiring: sabotaging production's HOCON precedence left all 36 green. The prior file had the same shape at a smaller scale, asserting only that BuildClusterOptions returned a KeepOldestOption — true, and true of a configuration that cannot fail over. Positive control: making BuildDowningHocon emit nothing for auto-down turns exactly the two effective-config guards red. Measured while verifying rather than assumed: HoconAddMode.Append also wins here, purely because it is added last, so the mode name is not the guarantee. The comment now says so instead of asserting a precedence rule that does not hold. Not yet live-drilled on OtOpcUa. The docker-dev rig is a single six-node mesh where the 1-vs-1 pathology cannot occur; the kill-the-oldest drill belongs with the per-cluster mesh work that makes every mesh exactly two nodes (design doc 6.2 / Phase 0a). Recorded as an outstanding gate in docs/Redundancy.md. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW	2026-07-21 18:32:42 -04:00