refactor(mesh): step 6 — delete the ClusterClient mode value; split topology now requires Grpc

MeshTransportOptions.ModeClusterClient is gone. MeshTransportOptionsValidator accepts Dps or
Grpc and nothing else, so a node left carrying the string "ClusterClient" is now REFUSED at
boot naming MeshTransport:Mode and both remaining values — deliberately fail-closed rather
than falling back to Dps, which on a split-topology node would deliver nothing at all. A new
fact pins exactly that.

SplitTopologyTransportValidator tightens from an allow-list of two to Grpc only, as its own
comment said it would once ClusterClient was deleted. Dps on a cluster-role node stays
refused; the rule is still an explicit allow-list, so a typo'd mode fails too.

Tests: the two ClusterClient-mode facts in MeshTransportOptionsValidatorTests are deleted
(their Dps siblings already assert the same rule for the one remaining non-Grpc mode), the
SplitTopology fixtures move to Grpc, and the three-mode Theories in
ControlPlaneStarterOrderTests / CentralCommunicationActorTests become two-mode.
This commit is contained in:
Joseph Doherty
2026-08-06 15:49:42 -04:00
parent 85ca86e955
commit af3b0c3146
7 changed files with 74 additions and 82 deletions
@@ -6,9 +6,11 @@ namespace ZB.MOM.WW.OtOpcUa.Cluster;
/// </summary>
/// <remarks>
/// <para>
/// <see cref="Mode"/> is a deliberate <b>dark switch</b>: the transports live in the tree
/// side by side so the docker-dev rig can run the same deployment over each and compare, and
/// so a bad rollout rolls back with a config change rather than a revert.
/// <see cref="Mode"/> selects between a <b>pair-local</b> transport and a <b>cross-mesh</b>
/// one. <see cref="ModeDps"/> is the default and cannot cross an application-<c>Cluster</c>
/// boundary; <see cref="ModeGrpc"/> is the only mode that can, and
/// <see cref="SplitTopologyTransportValidator"/> requires it on any node carrying a
/// <c>cluster-{ClusterId}</c> role.
/// </para>
/// <para>
/// <b>The DistributedPubSub branch is retained, not scheduled for deletion.</b> An earlier
@@ -16,15 +18,15 @@ namespace ZB.MOM.WW.OtOpcUa.Cluster;
/// deliberately reversed — see <c>docs/Configuration.md</c> § "The compiled transport-mode
/// defaults were deliberately NOT flipped". DPS remains the correct and default transport for
/// a single-mesh topology, which every integration test and every profile that leaves this
/// key unset still runs. What is being replaced is <see cref="ModeClusterClient"/>, by
/// <see cref="ModeGrpc"/>.
/// key unset still runs.
/// </para>
/// <para>
/// <b>Discovery is deliberately asymmetric</b>, matching the sister project. Central
/// discovers nodes <i>from the database</i> — enabled, non-maintenance <c>ClusterNode</c>
/// rows, rebuilt every <see cref="ContactRefreshSeconds"/>. Nodes discover central <i>from
/// this section</i>: static, restart to change. The asymmetry follows the rate of change —
/// operators add and retire nodes, but central's own address is part of the deployment.
/// this section</i> (<see cref="CentralGrpcEndpoints"/>): static, restart to change. The
/// asymmetry follows the rate of change — operators add and retire nodes, but central's own
/// address is part of the deployment.
/// </para>
/// </remarks>
public sealed class MeshTransportOptions
@@ -32,26 +34,26 @@ public sealed class MeshTransportOptions
/// <summary>Configuration section name.</summary>
public const string SectionName = "MeshTransport";
/// <summary>Every command stays on DistributedPubSub — the pre-Phase-2 behaviour.</summary>
/// <summary>
/// Every command stays on DistributedPubSub — pair-local, and the default. Cannot cross an
/// application-<c>Cluster</c> boundary.
/// </summary>
public const string ModeDps = "Dps";
/// <summary>Commands cross the ClusterClient receptionist boundary.</summary>
public const string ModeClusterClient = "ClusterClient";
/// <summary>
/// Commands cross the mesh boundary as unary gRPC calls on the <c>mesh_transport.v1</c>
/// contract — central dials each node's <c>NodeCommandService</c>, a node posts its
/// deployment ack to central's <c>CentralCommandService</c>. The replacement for
/// <see cref="ModeClusterClient"/>; unlike it, the command plane is authenticated
/// (<see cref="GrpcPsk"/>).
/// deployment ack to central's <c>CentralCommandService</c>. The command plane is
/// authenticated per application <c>Cluster</c> (<see cref="GrpcPsk"/>).
/// </summary>
public const string ModeGrpc = "Grpc";
/// <summary>
/// <see cref="ModeDps"/> (default), <see cref="ModeClusterClient"/> or
/// <see cref="ModeGrpc"/>. Any other value fails the host at startup rather than silently
/// falling back — a transport that is none of these delivers nothing, and the symptom would
/// be commands that vanish.
/// <see cref="ModeDps"/> (default) or <see cref="ModeGrpc"/>. Any other value fails the host
/// at startup rather than silently falling back — a transport that is neither delivers
/// nothing, and the symptom would be commands that vanish. That includes the retired
/// <c>ClusterClient</c> value: a node still carrying it is refused at boot rather than
/// quietly demoted to DPS, which on a split-topology node would deliver nothing at all.
/// </summary>
public string Mode { get; set; } = ModeDps;
@@ -40,16 +40,18 @@ public sealed class MeshTransportOptionsValidator : IValidateOptions<MeshTranspo
var errors = new List<string>();
var isDps = string.Equals(options.Mode, MeshTransportOptions.ModeDps, StringComparison.OrdinalIgnoreCase);
var isClusterClient = string.Equals(
options.Mode, MeshTransportOptions.ModeClusterClient, StringComparison.OrdinalIgnoreCase);
var isGrpc = string.Equals(options.Mode, MeshTransportOptions.ModeGrpc, StringComparison.OrdinalIgnoreCase);
if (!isDps && !isClusterClient && !isGrpc)
// An unrecognised mode is refused, never defaulted. The retired 'ClusterClient' value lands
// here: a node left carrying it fails to start with the two remaining values named, rather
// than silently falling back to Dps — which on a split-topology node cannot cross the mesh
// boundary and would deliver nothing at all.
if (!isDps && !isGrpc)
{
errors.Add(
$"{MeshTransportOptions.SectionName}:{nameof(MeshTransportOptions.Mode)} is "
+ $"'{options.Mode}'. Expected '{MeshTransportOptions.ModeDps}', "
+ $"'{MeshTransportOptions.ModeClusterClient}' or '{MeshTransportOptions.ModeGrpc}'.");
+ $"'{options.Mode}'. Expected '{MeshTransportOptions.ModeDps}' or "
+ $"'{MeshTransportOptions.ModeGrpc}'.");
}
if (options.ContactRefreshSeconds <= 0)
@@ -64,6 +66,7 @@ public sealed class MeshTransportOptionsValidator : IValidateOptions<MeshTranspo
// The first Grpc block below is node-side: the PSK a driver presents (and requires), the
// central endpoints it posts its ack to, and the two call deadlines (the admin-side rule that
// follows it is a separate scope). Scoped to the driver role because an admin-only central node
// dials nodes rather than being dialled, and resolves per-cluster keys at runtime through
// MeshTransportOptions.ClusterPsks / the Secrets store, so requiring a single GrpcPsk and an
// endpoint list of it would demand configuration it never reads. Roles come from
// configuration, not IClusterRoleInfo (see the ctor remark), and through
@@ -140,7 +143,7 @@ public sealed class MeshTransportOptionsValidator : IValidateOptions<MeshTranspo
// every node's deployment ack has nowhere to land, and nothing errors: the deployment simply
// times out naming nodes that applied perfectly. Owned here rather than by ConfigServe's own
// validation because the requirement comes from MeshTransport:Mode, not from config-serve — a
// Dps or ClusterClient central node is legitimately portless.
// Dps central node is legitimately portless.
if (isGrpc && isAdmin)
{
// Read as a string, the way SplitTopologyTransportValidator reads the modes:
@@ -98,22 +98,19 @@ public sealed class SplitTopologyTransportValidator : IValidateOptions<MeshTrans
var telemetryMode = _configuration["Telemetry:Mode"] ?? TelemetryOptions.ModeDps;
var telemetryDialMode = _configuration["TelemetryDial:Mode"] ?? TelemetryDialOptions.ModeDps;
// Two accepted values, not one, for the duration of the ClusterClient→gRPC migration: both
// cross a mesh boundary, which is the only property this rule cares about. Without the
// relaxation the very first rollout step — flipping a split-topology node to Grpc — would be a
// host-start failure. It is NOT widened to "anything but Dps": a typo'd mode delivers nothing
// just as surely, so the check stays an explicit allow-list and tightens to Grpc-only once
// ClusterClient is deleted.
if (!string.Equals(meshMode, MeshTransportOptions.ModeClusterClient, StringComparison.OrdinalIgnoreCase)
&& !string.Equals(meshMode, MeshTransportOptions.ModeGrpc, StringComparison.OrdinalIgnoreCase))
// Exactly ONE accepted value now that the Akka ClusterClient transport is deleted: Grpc is
// the only mode that crosses a mesh boundary, which is the only property this rule cares
// about. Deliberately an explicit allow-list rather than "anything but Dps" — a typo'd mode
// delivers nothing just as surely as DPS does, and the retired 'ClusterClient' value is
// caught here as well as by MeshTransportOptionsValidator's own unknown-mode rule.
if (!string.Equals(meshMode, MeshTransportOptions.ModeGrpc, StringComparison.OrdinalIgnoreCase))
{
errors.Add(
$"Cluster:Roles declares a cluster-{clusterId} role (split topology) but "
+ $"{MeshTransportOptions.SectionName}:{nameof(MeshTransportOptions.Mode)} is "
+ $"'{meshMode}'; a split mesh cannot use DistributedPubSub — set "
+ $"{MeshTransportOptions.SectionName}:{nameof(MeshTransportOptions.Mode)}="
+ $"{MeshTransportOptions.ModeGrpc} (or {MeshTransportOptions.ModeClusterClient}, the "
+ "transport it replaces).");
+ $"{MeshTransportOptions.ModeGrpc}.");
}
if (isDriver