fix(cluster): auto-down downing strategy — a two-node pair could not survive an oldest-node crash
Ports the sister project's live-proven fix (ScadaBridge cf3bd52f). OtOpcUa ran the identical configuration it indicts: keep-oldest with down-if-alone = on. Akka.NET 1.5.62's KeepOldest.OldestDecision only lets the down-if-alone branch rescue a side holding >= 2 members. A two-node cluster losing a peer is always a 1-vs-1 split, so the branch never fires, the lone survivor falls through to DownReachable and downs ITSELF, and run-coordinated-shutdown-when-down then terminates it. down-if-alone is a 3+-node feature and does not do what its name suggests for a pair: a crash of the oldest node is a total outage, which is the exact failure the redundancy pair exists to absorb. Cluster:SplitBrainResolverStrategy now selects the provider, defaulting to auto-down: Akka's AutoDowning with auto-down-unreachable-after = 15s, so the leader among the reachable members downs the unreachable peer and a crash of either node fails over in place. keep-oldest remains available for deployments that would rather take an outage than ever run dual-active during a real partition. An unrecognised value fails the host at startup rather than falling through to the fatal default. The tests assert the EFFECTIVE configuration — they start a real host through WithOtOpcUaClusterBootstrap and read akka.cluster.downing-provider-class back off the running ActorSystem. This is not incidental. The first draft inlined the three calls the bootstrap makes instead of calling it, which pinned the test's own wiring: sabotaging production's HOCON precedence left all 36 green. The prior file had the same shape at a smaller scale, asserting only that BuildClusterOptions returned a KeepOldestOption — true, and true of a configuration that cannot fail over. Positive control: making BuildDowningHocon emit nothing for auto-down turns exactly the two effective-config guards red. Measured while verifying rather than assumed: HoconAddMode.Append also wins here, purely because it is added last, so the mode name is not the guarantee. The comment now says so instead of asserting a precedence rule that does not hold. Not yet live-drilled on OtOpcUa. The docker-dev rig is a single six-node mesh where the 1-vs-1 pathology cannot occur; the kill-the-oldest drill belongs with the per-cluster mesh work that makes every mesh exactly two nodes (design doc 6.2 / Phase 0a). Recorded as an outstanding gate in docs/Redundancy.md. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
@@ -29,4 +29,30 @@ public sealed class AkkaClusterOptions
|
||||
/// <c>admin</c>, <c>driver</c>, <c>dev</c>.
|
||||
/// </summary>
|
||||
public string[] Roles { get; set; } = Array.Empty<string>();
|
||||
|
||||
/// <summary>
|
||||
/// How the cluster decides to down a node it can no longer reach. One of <c>auto-down</c>
|
||||
/// (default) or <c>keep-oldest</c>; any other value fails the host at startup.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// <b><c>auto-down</c> — availability.</b> The leader among the <i>reachable</i> members
|
||||
/// downs the unreachable peer after
|
||||
/// <see cref="ServiceCollectionExtensions.DowningStableAfter"/>. A hard crash of either
|
||||
/// node — including the oldest — fails over to the survivor with no operator action.
|
||||
/// The trade is that a genuine network partition (both nodes alive, link cut) leaves
|
||||
/// both sides running active until an operator restarts one.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b><c>keep-oldest</c> — partition-safety.</b> The SBR resolver sacrifices the younger
|
||||
/// side of a split, so a partition can never run dual-active. <b>The cost is severe for a
|
||||
/// two-node pair:</b> it cannot survive a crash of the oldest node at all. Akka.NET's
|
||||
/// <c>KeepOldest.OldestDecision</c> only lets <c>down-if-alone</c> rescue a side holding
|
||||
/// >= 2 members, so the 1-vs-1 survivor downs <i>itself</i> and shuts down — the
|
||||
/// redundancy pair turns a single-node crash into a total outage. Choose this only for
|
||||
/// clusters of three or more nodes, or where dual-active is genuinely worse than an
|
||||
/// outage.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
public string SplitBrainResolverStrategy { get; set; } = "auto-down";
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user