dohertj2/lmxopcua
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(cluster): activate split-brain resolver (arch-review 03/S1)

Browse Source 
The split-brain-resolver HOCON block in akka.conf was inert — nothing
registered a downing provider, so the cluster ran Akka's default NoDowning:
a hard-crashed node was never downed, cluster singletons + the driver
role-leader never failed over, and a partition left both redundancy sides
at ServiceLevel 240 indefinitely.

Activate the resolver via the typed ClusterOptions.SplitBrainResolver
(KeepOldestOption { DownIfAlone = true }) in a new testable
BuildClusterOptions helper. keep-oldest + down-if-alone is correct for a
2-node warm-redundancy pair. HOCON keeps stable-after (typed option can't
express it) + a cross-reference comment. Deterministic unit guard added;
end-to-end verified the resolved ActorSystem config carries the provider
and stable-after survives the HOCON/Hosting merge. Redundancy.md corrected.

Hard-kill failover integration test tracked as a follow-up (effort-M harness).
This commit is contained in:
Joseph Doherty
2026-07-08 16:14:26 -04:00
parent 9cad9ed0fc
commit a81dea102c
4 changed files with 112 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests/SplitBrainResolverActivationTests.cs
+55
View File
@@ -0,0 +1,55 @@
using Akka.Cluster.Hosting;
using Akka.Cluster.Hosting.SBR;
using Shouldly;
using Xunit;
namespace ZB.MOM.WW.OtOpcUa.Cluster.Tests;
/// <summary>
/// Guards the split-brain-resolver ACTIVATION (arch-review 03/S1). The <c>split-brain-resolver</c>
/// HOCON block is inert on its own; the resolver only runs when the typed
/// <see cref="ClusterOptions.SplitBrainResolver"/> is set. These tests assert
/// <see cref="ServiceCollectionExtensions.BuildClusterOptions"/> registers it, so a future refactor
/// cannot silently drop back to Akka's default NoDowning (the exact "built-but-never-wired" failure
/// mode this finding described).
/// </summary>
public sealed class SplitBrainResolverActivationTests
{
private static AkkaClusterOptions SampleOptions() => new()
{
SeedNodes = new[] { "akka.tcp://otopcua@127.0.0.1:4053" },
Roles = new[] { "admin", "driver" },
};
/// <summary>Verifies the cluster options carry a split-brain resolver (provider gets activated).</summary>
[Fact]
public void BuildClusterOptions_activates_a_split_brain_resolver()
{
var options = ServiceCollectionExtensions.BuildClusterOptions(SampleOptions());
options.SplitBrainResolver.ShouldNotBeNull(
"SplitBrainResolver must be set — without it the cluster runs default NoDowning and never fails over on a hard crash.");
}
/// <summary>Verifies the resolver is keep-oldest with down-if-alone (correct for a 2-node pair).</summary>
[Fact]
public void BuildClusterOptions_uses_keep_oldest_with_down_if_alone()
{
var options = ServiceCollectionExtensions.BuildClusterOptions(SampleOptions());
var keepOldest = options.SplitBrainResolver.ShouldBeOfType<KeepOldestOption>();
keepOldest.DownIfAlone.ShouldBe(true);
}
/// <summary>Verifies seed nodes and roles are still threaded through unchanged.</summary>
[Fact]
public void BuildClusterOptions_preserves_seed_nodes_and_roles()
{
var input = SampleOptions();
var options = ServiceCollectionExtensions.BuildClusterOptions(input);
options.SeedNodes.ShouldBe(input.SeedNodes);
options.Roles.ShouldBe(input.Roles);
}
}