Files
scadaproj/ZB.MOM.WW.Health/tests/ZB.MOM.WW.Health.Akka.Tests/ActiveNodeStartupSafetyTests.cs
T
Joseph Doherty a61e041e58 feat(health)!: active node is the oldest Up member, not the leader (0.3.0)
Promotes the two apps' private replacements for ActiveNodeHealthCheck into one
shared primitive, and retires the leader/RoleLeader selection they were written
to avoid.

Both Akka consumers had already hand-rolled a replacement rather than use this
package (ScadaBridge OldestNodeActiveHealthCheck, OtOpcUa
ClusterPrimaryHealthCheck), so the entire active-node surface here —
ActiveNodeHealthCheck, AkkaActiveNodeGate — had ZERO consumers. It was not
merely unused: it was avoided, twice, for the same reason.

Leadership is address-ordered (host, then port) and has no relationship to time;
singleton placement is age-ordered. The two agree on a freshly-formed cluster,
which is why single-node and happy-path tests never caught it. They diverge
permanently after any restart: the restarted node rejoins as the youngest but
keeps its address, so if it holds the lower address it becomes leader while the
singletons — and all the work they own — stay on the other node.

New ClusterActiveNode is the single implementation: oldest Up member, optional
role scope, plus role-preference resolution for a fused node that must answer
for the role its singletons are pinned to. ActiveNodeHealthCheck and
AkkaActiveNodeGate both delegate to it, so an endpoint gate and the
/health/active probe an orchestrator routes by cannot disagree.

BREAKING (behaviour, not signature):
- Selection is by age, not leadership.
- The role-filtered mode no longer reports Healthy for a node LACKING the role.
  That "not applicable => Healthy" made the tier answer 200 on every node and
  silently broke leader-pinning (lmxopcua#494); it is now Unhealthy by default,
  overridable via NoActiveRoleStatus. This case is reachable, not defensive —
  OtOpcUa's RoleParser admits dev-only and cluster-role-only nodes.
- Identity compares UniqueAddress, so a node restarted on the same host:port is
  correctly a different member during the overlap.

Results now carry activeRole/selfAddress/activeNode in the 0.2.0 per-entry data
object, so a standby reports WHO is active and a dashboard can render a pair
from either node's payload alone.

Tests: 45 Akka (was 39), 76 total. The age-vs-address divergence is pinned
against a real two-node cluster built so the oldest member is not the
lowest-addressed one, with a fixture sanity check so it cannot pass for the
wrong reason.
2026-07-24 13:07:56 -04:00

114 lines
4.0 KiB
C#

using Akka.Actor;
using Akka.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Diagnostics.HealthChecks;
using ZB.MOM.WW.Health.Akka;
namespace ZB.MOM.WW.Health.Akka.Tests;
/// <summary>
/// The states a node passes through on the way up, where the active tier must not report a booting
/// node as a failed standby — now that the tier is a real 503, an Unhealthy here would pull the node
/// out of an orchestrator's pool while it starts.
/// </summary>
public sealed class ActiveNodeStartupSafetyTests
{
[Fact]
public async Task NoActorSystem_IsDegraded()
{
var check = new ActiveNodeHealthCheck(new ServiceCollection().BuildServiceProvider());
var result = await check.CheckHealthAsync(NewContext(check));
Assert.Equal(HealthStatus.Degraded, result.Status);
// Description-only: an empty data dictionary is what makes the writer omit "data" entirely.
Assert.Empty(result.Data);
}
[Fact]
public async Task NoActorSystem_RoleScoped_IsDegraded()
{
var check = new ActiveNodeHealthCheck(new ServiceCollection().BuildServiceProvider(), "admin");
Assert.Equal(HealthStatus.Degraded, (await check.CheckHealthAsync(NewContext(check))).Status);
}
[Fact]
public void Gate_NoActorSystem_IsNotActive()
{
Assert.False(new AkkaActiveNodeGate(new ServiceCollection().BuildServiceProvider()).IsActiveNode);
}
[Fact]
public async Task ClusterNotConfigured_IsDegraded()
{
// ActorSystem present but Akka.Cluster not configured → Cluster.Get throws. The check must
// return Degraded (the spec's startup-safety rule), not let the exception escape (→ Unhealthy).
using var system = ActorSystem.Create("active-node-no-cluster");
try
{
var provider = new ServiceCollection().AddSingleton(system).BuildServiceProvider();
var check = new ActiveNodeHealthCheck(provider);
Assert.Equal(HealthStatus.Degraded, (await check.CheckHealthAsync(NewContext(check))).Status);
}
finally
{
await system.Terminate();
}
}
[Fact]
public async Task Gate_ClusterNotConfigured_IsNotActive()
{
using var system = ActorSystem.Create("active-node-gate-no-cluster");
try
{
var provider = new ServiceCollection().AddSingleton(system).BuildServiceProvider();
Assert.False(new AkkaActiveNodeGate(provider).IsActiveNode);
}
finally
{
await system.Terminate();
}
}
[Fact]
public async Task ClusteredButNeverJoined_IsUnhealthy_NobodyIsActive()
{
// A cold start: the cluster provider is up but no member has reached Up. Nobody is active, so
// nobody may claim to be — this is the one case that must NOT be Degraded, because a cluster
// that never forms would otherwise sit at 200 forever.
var config = ConfigurationFactory.ParseString("""
akka {
loglevel = "WARNING"
actor.provider = cluster
remote.dot-netty.tcp { hostname = "127.0.0.1", port = 0 }
cluster.seed-nodes = []
}
""");
var system = ActorSystem.Create("active-node-never-joined", config);
try
{
var provider = new ServiceCollection().AddSingleton(system).BuildServiceProvider();
var check = new ActiveNodeHealthCheck(provider);
var result = await check.CheckHealthAsync(NewContext(check));
Assert.Equal(HealthStatus.Unhealthy, result.Status);
Assert.Contains("No Up member", result.Description);
Assert.False(new AkkaActiveNodeGate(provider).IsActiveNode);
}
finally
{
await system.Terminate();
}
}
private static HealthCheckContext NewContext(IHealthCheck check) => new()
{
Registration = new HealthCheckRegistration("active-node", check, HealthStatus.Unhealthy, tags: null),
};
}