dohertj2/lmxopcua
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(mesh): one ClusterClient per Cluster in CentralCommunicationActor (Phase 6)

Browse Source 
After the fleet splits into one Akka mesh per application Cluster, a
ClusterClientReceptionist serves only its own mesh — so the single
fleet-wide client's SendToAll reached only the mesh whose receptionist
answered, leaving every other cluster silently on its old configuration.

Central now holds one ClusterClient per ClusterId and fans SendToAll
across all of them. LoadContactsFromDb selects ClusterId and groups the
receptionist contacts per cluster (keeping the per-row TryParse guard and
the enabled/non-maintenance filter); ContactsLoaded carries
ContactsByCluster; HandleContactsLoaded diffs per cluster (rebuild changed,
stop+drop vanished, warn-don't-create on empty); RebuildClient builds a
per-cluster client under a clusterId-derived actor name. IMeshClusterClientFactory.Create
gained a clusterId parameter so the per-cluster clients get distinct,
diagnosable names. ApplyAck handling and the DPS branch are unchanged; the
deploy path stays payload-free.

Tests: focused unit coverage of the grouping + fan-out (one-client-per-cluster,
fan-out SendToAll to every cluster client) via the recording factory double,
plus the real two-mesh boundary test extended to central + two separate site
meshes proving one dispatch reaches both and a client scoped to one cluster
never crosses into another. Red-before-green verified: crippling the fan-out
to a single client fails the reaches-both-meshes assertion.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
Joseph Doherty
2026-07-24 01:53:20 -04:00
parent 2535df019b
commit 6531ec1984
5 changed files with 358 additions and 75 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests/Communication/CentralCommunicationActorTests.cs
+47 -9
View File
@@ -31,13 +31,20 @@ public class CentralCommunicationActorTests : ControlPlaneActorTestBase
public List<ImmutableHashSet<ActorPath>> Calls { get; } = [];
public List<string> ClusterIds { get; } = [];
public List<TestProbe> Probes { get; } = [];
public IActorRef Create(ActorSystem system, ImmutableHashSet<ActorPath> contacts)
/// <summary>Probe created for a given cluster id, for per-cluster fan-out assertions.</summary>
public Dictionary<string, TestProbe> ProbeByCluster { get; } = [];
public IActorRef Create(ActorSystem system, string clusterId, ImmutableHashSet<ActorPath> contacts)
{
Calls.Add(contacts);
ClusterIds.Add(clusterId);
var probe = _newProbe();
Probes.Add(probe);
ProbeByCluster[clusterId] = probe;
return probe.Ref;
}
}
@@ -138,23 +145,54 @@ public class CentralCommunicationActorTests : ControlPlaneActorTestBase
}
[Fact]
public void Exactly_one_client_is_created_for_a_multi_cluster_fleet()
public void One_client_per_cluster_is_created_for_a_multi_cluster_fleet()
{
// A receptionist serves its whole mesh, so one client per Cluster would fan each command out
// once per cluster while the fleet is still a single mesh — N x duplicate deployments.
// THE Phase 6 inversion. Each application Cluster is now its own Akka mesh, and a ClusterClient
// reaches only the mesh whose receptionist it dialled. Central must therefore build one client
// per cluster, each dialling only that cluster's nodes.
var factory = NewInMemoryDbFactory();
SeedCluster(factory, "C1", ("host-a:4053", "host-a", 4053, true, false));
SeedCluster(factory, "C2", ("host-b:4053", "host-b", 4053, true, false));
var clients = new RecordingClientFactory(() => CreateTestProbe());
Spawn(factory, ClusterClientMode(), clients);
AwaitCondition(() => clients.Calls.Count == 1, TimeSpan.FromSeconds(5));
AwaitCondition(() => clients.Calls.Count == 2, TimeSpan.FromSeconds(5));
// ...and that one client dials BOTH clusters' nodes.
clients.Calls[0].Count.ShouldBe(2);
// Give a second refresh a chance to (wrongly) create another.
// Give a second refresh a chance to (wrongly) create more.
ExpectNoMsg(TimeSpan.FromMilliseconds(300));
clients.Calls.Count.ShouldBe(1);
clients.Calls.Count.ShouldBe(2);
clients.ClusterIds.ShouldBe(new[] { "C1", "C2" }, ignoreOrder: true);
// Each client dials ONLY its own cluster's node.
clients.Calls[clients.ClusterIds.IndexOf("C1")].Single().ToString().ShouldContain("host-a");
clients.Calls[clients.ClusterIds.IndexOf("C2")].Single().ToString().ShouldContain("host-b");
}
[Fact]
public void A_command_fans_out_a_send_to_all_to_every_cluster_client()
{
// The other half of the inversion: one MeshCommand must reach EVERY cluster's mesh, so central
// fans SendToAll across all per-cluster clients. A single Send/one-client shape would leave
// every cluster but one silently on its old configuration.
var factory = NewInMemoryDbFactory();
SeedCluster(factory, "C1", ("host-a:4053", "host-a", 4053, true, false));
SeedCluster(factory, "C2", ("host-b:4053", "host-b", 4053, true, false));
var clients = new RecordingClientFactory(() => CreateTestProbe());
var actor = Spawn(factory, ClusterClientMode(), clients);
AwaitCondition(() => clients.Probes.Count == 2, TimeSpan.FromSeconds(5));
var dispatch = new DispatchDeployment(
new DeploymentId(Guid.NewGuid()), new RevisionHash("abc"), new CorrelationId(Guid.NewGuid()));
actor.Tell(new MeshCommand(DeploymentsTopic, dispatch));
foreach (var (clusterId, probe) in clients.ProbeByCluster)
{
var sent = probe.ExpectMsg<ClusterClient.SendToAll>(
TimeSpan.FromSeconds(5), $"cluster {clusterId} must receive the fanned SendToAll");
sent.Path.ShouldBe(MeshPaths.NodeCommunication);
sent.Message.ShouldBe(dispatch);
}
}
[Fact]