feat(cluster): auto-down downing strategy — either-node crash now fails over (owner decision 2026-07-21: availability over partition-safety)
Two-node keep-oldest could NEVER survive a crash of the oldest/active node:
Akka.NET 1.5.62 KeepOldest.OldestDecision only lets down-if-alone rescue a
side with >= 2 members, so the 1-vs-1 survivor takes DownReachable and downs
ITSELF — proven live on the rig ('SBR took decision ... including myself')
before this change. static-quorum(1) is worse (IsTooManyMembers -> DownAll);
keep-majority just re-keys the fatal crash to the lowest address.
SplitBrainResolverStrategy gains 'auto-down' (new default): BuildHocon emits
Akka's AutoDowning provider with auto-down-unreachable-after = StableAfter.
The leader among the REACHABLE members downs the unreachable peer, so the
survivor takes over singletons and /health/active in ~25s regardless of which
node died. Accepted trade (explicit owner decision): a real network partition
runs dual-active until an operator restarts one side. keep-oldest remains
supported; DownIfAlone validation is now scoped to it.
Live drill on the rebuilt rig: active-crash TAKEOVER in 28s (victim still
down; all 7 singletons Younger->Oldest), standby-crash removal 27s with 0
routing blips; victims rejoin as standby in 2s. New real-cluster tests pin
both directions (SbrFailoverTests.AutoDown_*); TwoNodeClusterFixture gains a
strategy knob. All 16 appsettings flipped (src, docker, docker-env2, and the
gitignored wonder-app-vd03 overlay on disk — owner must sync to the host).
Docs: decision record docs/plans/2026-07-21-auto-down-availability-decision.md,
Component-ClusterInfrastructure downing section rewritten, drill + README
reworked (active mode now asserts takeover), deferred-work SBR row resolved.
This commit is contained in:
+30
-1
@@ -153,9 +153,10 @@ public class ClusterOptionsValidatorTests
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DownIfAloneFalse_FailsValidation()
|
||||
public void DownIfAloneFalse_UnderKeepOldest_FailsValidation()
|
||||
{
|
||||
var options = ValidOptions();
|
||||
options.SplitBrainResolverStrategy = "keep-oldest";
|
||||
options.DownIfAlone = false;
|
||||
|
||||
var result = new ClusterOptionsValidator().Validate(null, options);
|
||||
@@ -164,6 +165,34 @@ public class ClusterOptionsValidatorTests
|
||||
Assert.Contains("DownIfAlone", result.FailureMessage);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void AutoDownStrategy_Passes()
|
||||
{
|
||||
// Decision 2026-07-21: 'auto-down' is the supported availability-first
|
||||
// posture — either-node crash fails over; dual-active on a real partition
|
||||
// is the accepted trade.
|
||||
var options = ValidOptions();
|
||||
options.SplitBrainResolverStrategy = "auto-down";
|
||||
|
||||
var result = new ClusterOptionsValidator().Validate(null, options);
|
||||
|
||||
Assert.True(result.Succeeded, result.FailureMessage);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DownIfAloneFalse_UnderAutoDown_Passes()
|
||||
{
|
||||
// DownIfAlone is a keep-oldest knob; under auto-down it is inert and must
|
||||
// not block startup.
|
||||
var options = ValidOptions();
|
||||
options.SplitBrainResolverStrategy = "auto-down";
|
||||
options.DownIfAlone = false;
|
||||
|
||||
var result = new ClusterOptionsValidator().Validate(null, options);
|
||||
|
||||
Assert.True(result.Succeeded, result.FailureMessage);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Validate_AccumulatesAllFailures()
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user