test(mesh): step 7 — replace the boundary test with MeshGrpcBoundaryTests and sweep the comments
MeshGrpcBoundaryTests replaces the MeshClusterClientBoundaryTests deleted in step 1. Three
self-seeded single-member clusters sharing the ActorSystem name, joined ONLY by gRPC, driving
the REAL CentralCommunicationActor -> MeshGrpcCommandDialer -> Kestrel h2c ->
MeshCommandAuthInterceptor -> NodeCommandGrpcService -> NodeCommunicationActor -> EventStream:
1. No_cross_mesh_Akka_association_forms_between_central_and_either_site — the program's own
point, kept as a test rather than a live-gate note: each cluster stays single-member and
none ever holds another's address as a member.
2. One_dispatch_through_central_reaches_a_node_in_every_cluster_mesh — the fan, with each
site holding its OWN per-cluster PSK, so a dial that selected the wrong cluster's key
would be refused at the far side.
3. Falsifiability_control_a_dial_carrying_the_wrong_cluster_key_delivers_nothing — without
it, (2) cannot distinguish "the gRPC boundary carried it" from "something else in the
process did". The same node then accepts the right key, so the silence is the auth gate
and not a dead endpoint.
All three run fully offline — no Docker fixture — and pass.
Comment sweep across every remaining clusterclient|receptionist hit in src/ and tests/:
the program label becomes "mesh gRPC command plane", and every comment that described the
deleted transport as live now describes what actually runs (Dps = pair-local default,
Grpc = cross-mesh). mesh_transport.proto's header/service comments are reworded; no message,
field or enum is touched — the contract is locked. MeshCommActorPathTests is reshaped rather
than retired: the two paths still exist and still matter, but what a mismatch costs is now a
misleading Unavailable from the gRPC ingress rather than a silently dropped receptionist send.
The only surviving mentions of the old names are the ones that must survive: the fail-closed
rules that name "ClusterClient" as a retired value refused at boot, their tests, and this
test class's own account of what it replaces.
This commit is contained in:
@@ -17,10 +17,10 @@ using Status = Grpc.Core.Status;
|
||||
namespace ZB.MOM.WW.OtOpcUa.AdminUI.Grpc;
|
||||
|
||||
/// <summary>
|
||||
/// Central-side ingress for a node's deployment acknowledgment (ClusterClient→gRPC migration
|
||||
/// Central-side ingress for a node's deployment acknowledgment (mesh gRPC command plane
|
||||
/// Phase 1B). Decodes an <see cref="ApplyAckMessage"/> and hands it to the node-local
|
||||
/// <c>CentralCommunicationActor</c> — the same actor an Akka ClusterClient receptionist delivered
|
||||
/// to, whose forward-to-the-coordinator path is untouched. Registered only on admin-role nodes
|
||||
/// <c>CentralCommunicationActor</c> — the same actor the node-local DistributedPubSub path
|
||||
/// delivers to, whose forward-to-the-coordinator path is untouched. Registered only on admin-role nodes
|
||||
/// and only when <c>ConfigServe:GrpcListenPort > 0</c>.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
@@ -52,7 +52,7 @@ namespace ZB.MOM.WW.OtOpcUa.AdminUI.Grpc;
|
||||
/// <para>
|
||||
/// <b>One-way semantics, preserved.</b> <c>Tell</c>, not <c>Ask</c>: the returned
|
||||
/// <see cref="ReportAck"/> means "handed to the comm actor", never "the deployment is
|
||||
/// complete" — the same promise the receptionist made. The comm actor forwards to the
|
||||
/// complete" — the same promise the DistributedPubSub leg makes. The comm actor forwards to the
|
||||
/// coordinator singleton proxy, so an ack landing on either central node routes identically.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
|
||||
@@ -92,8 +92,7 @@ public sealed class AdminOperationsActor : ReceiveActor
|
||||
/// <b>retained</b>, not scheduled for deletion — an earlier note here promised Phase 6 would
|
||||
/// remove it, which was deliberately reversed (<c>docs/Configuration.md</c> § "The compiled
|
||||
/// transport-mode defaults were deliberately NOT flipped"): DPS is still the correct and
|
||||
/// default transport for a single-mesh topology. What is being replaced is the ClusterClient
|
||||
/// branch, by gRPC.
|
||||
/// default transport for a single-mesh topology; gRPC is what crosses a mesh boundary.
|
||||
/// </remarks>
|
||||
/// <param name="topic">The legacy DistributedPubSub topic.</param>
|
||||
/// <param name="message">The command.</param>
|
||||
|
||||
@@ -10,7 +10,7 @@ namespace ZB.MOM.WW.OtOpcUa.ControlPlane.Communication;
|
||||
|
||||
/// <summary>
|
||||
/// Places one central→node command call on one node's <c>NodeCommandService</c>
|
||||
/// (ClusterClient→gRPC migration Phase 1A). The seam <c>CentralCommunicationActor</c> fans
|
||||
/// (mesh gRPC command plane, Phase 1A). The seam <c>CentralCommunicationActor</c> fans
|
||||
/// across its target set, kept as an interface so the actor's tests never open a socket.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
|
||||
@@ -72,7 +72,7 @@ public sealed class ConfigPublishCoordinator : ReceiveActor, IWithTimers
|
||||
/// not scheduled for deletion — an earlier note here promised Phase 6 would remove it, which
|
||||
/// was deliberately reversed (<c>docs/Configuration.md</c> § "The compiled transport-mode
|
||||
/// defaults were deliberately NOT flipped"): DPS is still the correct and default transport
|
||||
/// for a single-mesh topology. What is being replaced is the ClusterClient branch, by gRPC.
|
||||
/// for a single-mesh topology; gRPC is what crosses a mesh boundary.
|
||||
/// </remarks>
|
||||
private readonly IActorRef? _meshRouter;
|
||||
private readonly ILoggingAdapter _log = Context.GetLogger();
|
||||
|
||||
@@ -19,7 +19,7 @@
|
||||
Google.Protobuf + Grpc.Core.Api flow transitively from Commons (the generated client). -->
|
||||
<PackageReference Include="Grpc.Net.Client"/>
|
||||
<PackageReference Include="ZB.MOM.WW.Audit"/>
|
||||
<!-- ClusterClient→gRPC: MeshPskProvider resolves OT-MESH-PSK-{clusterId} from the shared
|
||||
<!-- Mesh gRPC command plane: MeshPskProvider resolves OT-MESH-PSK-{clusterId} from the shared
|
||||
secret store when MeshTransport:ClusterPsks carries no override. Abstractions only —
|
||||
the concrete store is the host's choice (SecretsRegistration), not the control plane's. -->
|
||||
<PackageReference Include="ZB.MOM.WW.Secrets.Abstractions"/>
|
||||
|
||||
@@ -6,7 +6,7 @@ using ZB.MOM.WW.OtOpcUa.ControlPlane.Communication;
|
||||
namespace ZB.MOM.WW.OtOpcUa.Host.Configuration;
|
||||
|
||||
/// <summary>
|
||||
/// Gates the central-side command endpoint (ClusterClient→gRPC migration). The
|
||||
/// Gates the central-side command endpoint (mesh gRPC command plane). The
|
||||
/// <c>CentralCommandService.ReportApply</c> RPC is how a node tells central its deployment
|
||||
/// applied; the coordinator counts those acks, so anything able to reach central's command port
|
||||
/// could otherwise forge a fleet-wide "everything applied". This interceptor is the ONLY inbound
|
||||
|
||||
@@ -9,7 +9,7 @@ using ZB.MOM.WW.OtOpcUa.Cluster;
|
||||
namespace ZB.MOM.WW.OtOpcUa.Host.Configuration;
|
||||
|
||||
/// <summary>
|
||||
/// Gates the node-side command endpoint (ClusterClient→gRPC migration). The
|
||||
/// Gates the node-side command endpoint (mesh gRPC command plane). The
|
||||
/// <c>NodeCommandService.Execute</c> RPC is how central drives this node — restart a driver,
|
||||
/// acknowledge an alarm, dispatch a deployment — so anything able to reach a node's command port
|
||||
/// could otherwise operate the node. This interceptor is the ONLY inbound auth on that endpoint.
|
||||
|
||||
@@ -15,11 +15,11 @@ using Status = Grpc.Core.Status;
|
||||
namespace ZB.MOM.WW.OtOpcUa.Host.Grpc;
|
||||
|
||||
/// <summary>
|
||||
/// Node-side command ingress (ClusterClient→gRPC migration Phase 1). Central dials a driver node
|
||||
/// Node-side command ingress (mesh gRPC command plane, Phase 1). Central dials a driver node
|
||||
/// and calls <see cref="Execute"/>; the envelope is decoded back into the same CLR record the
|
||||
/// ClusterClient receptionist used to deliver and handed to this node's
|
||||
/// node-local DistributedPubSub path delivers and handed to this node's
|
||||
/// <c>NodeCommunicationActor</c>, which re-emits it on the node's <c>EventStream</c> exactly as
|
||||
/// before. Nothing downstream of that actor can tell which transport carried the command.
|
||||
/// it would otherwise. Nothing downstream of that actor can tell which transport carried it.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
|
||||
@@ -527,20 +527,19 @@ if (hasAdmin)
|
||||
// hosted service has built the system).
|
||||
builder.Services.TryAddSingleton<Func<ActorSystem>>(sp => () => sp.GetRequiredService<ActorSystem>());
|
||||
builder.Services.AddSingleton<IManualFailoverService, ManualFailoverService>();
|
||||
// Per-cluster mesh PSKs (ClusterClient→gRPC). Admin-only, and singleton because its whole job is
|
||||
// Per-cluster mesh PSKs (mesh gRPC command plane). Admin-only, and singleton because its whole job is
|
||||
// to hold a cache: central talks to N clusters and would otherwise re-read the secret store on
|
||||
// every inbound ack. A node needs nothing of this — it holds exactly its own MeshTransport:GrpcPsk
|
||||
// — so registering it fleet-wide would give a driver-only node a lookup it can never use.
|
||||
// Depends on ISecretResolver, which AddOtOpcUaSecrets registers unconditionally below (DI
|
||||
// registration order is irrelevant; resolution order is not).
|
||||
builder.Services.AddSingleton<IMeshPskProvider, MeshPskProvider>();
|
||||
// Central's outbound command dial (ClusterClient→gRPC Phase 1A). Admin-only for the same reason
|
||||
// Central's outbound command dial (mesh gRPC command plane, Phase 1A). Admin-only for the same reason
|
||||
// as the provider above — a node never dials another node — and singleton because it owns a
|
||||
// GrpcChannel per node endpoint; per-call channels would leak sockets across a fleet-wide fan.
|
||||
// Registered in every mode: CentralCommunicationActor only reaches it under Mode=Grpc, and an
|
||||
// idle registration costs nothing, whereas gating it here would mean flipping the mode on a
|
||||
// running fleet needs a central restart before any command lands — the same reasoning that
|
||||
// registers the ClusterClient receptionist unconditionally.
|
||||
// running fleet needs a central restart before any command lands.
|
||||
builder.Services.AddSingleton<IMeshCommandDialer, MeshGrpcCommandDialer>();
|
||||
}
|
||||
|
||||
@@ -564,7 +563,7 @@ builder.Services.AddOtOpcUaObservability(builder.Configuration);
|
||||
// Registered whenever either role is present so an admin-only node still serves config and a
|
||||
// driver-only node still gates sync + telemetry.
|
||||
//
|
||||
// The two mesh-command interceptors (ClusterClient→gRPC migration) join on the same terms:
|
||||
// The two mesh-command interceptors (mesh gRPC command plane) join on the same terms:
|
||||
// MeshCommandAuthInterceptor gates the node-side NodeCommandService (driver-role, one key — this
|
||||
// node's own), CentralCommandAuthInterceptor gates the central-side CentralCommandService
|
||||
// (admin-role, one key per cluster via IMeshPskProvider). Both are registered NOW and are pure
|
||||
@@ -745,7 +744,7 @@ if (hasAdmin && configServeGrpcPort > 0)
|
||||
{
|
||||
app.MapGrpcService<DeploymentArtifactGrpcService>();
|
||||
|
||||
// Central command ingress (ClusterClient→gRPC migration): the node→central deployment ack. It
|
||||
// Central command ingress (mesh gRPC command plane): the node→central deployment ack. It
|
||||
// rides the SAME listener rather than claiming a port of its own — a second cleartext h2c port
|
||||
// would buy nothing but another firewall rule, and both endpoints are admin-role and dialled by
|
||||
// the same driver nodes. Deliberately NOT gated on MeshTransport:Mode: mapping it under Dps
|
||||
@@ -759,7 +758,7 @@ if (hasAdmin && configServeGrpcPort > 0)
|
||||
// Driver-node live-telemetry stream (per-cluster mesh Phase 5). Central dials in; gated by
|
||||
// TelemetryStreamAuthInterceptor (fail-closed on empty Telemetry:ApiKey).
|
||||
//
|
||||
// The node-side command endpoint (ClusterClient→gRPC migration Phase 1) rides the SAME driver-role
|
||||
// The node-side command endpoint (mesh gRPC command plane, Phase 1) rides the SAME driver-role
|
||||
// h2c listener rather than claiming a fourth port: central already dials this node here, both
|
||||
// directions are per-cluster-keyed, and a separate port would be one more thing to open on every
|
||||
// site firewall for no isolation gain. Mapping it is what makes MeshCommandAuthInterceptor live —
|
||||
|
||||
@@ -12,14 +12,14 @@ namespace ZB.MOM.WW.OtOpcUa.Runtime.Communication;
|
||||
|
||||
/// <summary>
|
||||
/// Posts this node's <see cref="ApplyAck"/> to central's <c>CentralCommandService</c> as a unary
|
||||
/// gRPC call (ClusterClient→gRPC migration Task 1B.2), the <c>MeshTransport:Mode = Grpc</c>
|
||||
/// replacement for the ClusterClient <c>Send</c>.
|
||||
/// gRPC call (mesh gRPC command plane, Task 1B.2), the <c>MeshTransport:Mode = Grpc</c>
|
||||
/// cross-mesh ack transport.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// <b>One ordered pass, stopping at the first success.</b>
|
||||
/// <see cref="MeshTransportOptions.CentralGrpcEndpoints"/> is an availability list, not a
|
||||
/// fan-out list — the semantic of the <c>ClusterClient.Send</c> being replaced (<c>Send</c>,
|
||||
/// fan-out list — the single-target semantic this leg owes (one ack per node per dispatch,
|
||||
/// deliberately not <c>SendToAll</c>).
|
||||
/// </para>
|
||||
/// <para>
|
||||
@@ -35,7 +35,7 @@ namespace ZB.MOM.WW.OtOpcUa.Runtime.Communication;
|
||||
/// depth and a bound on wasted work, not a correctness dependency.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>No retry beyond that single pass, and no queue.</b> The receptionist buffered nothing
|
||||
/// <b>No retry beyond that single pass, and no queue.</b> The transport buffers nothing
|
||||
/// and retried nothing, and this is a like-for-like replacement, not an upgrade to delivery
|
||||
/// guarantees. The deploy path's own apply deadline stays the only recovery mechanism; a
|
||||
/// retry loop here would instead pile acks up behind an unreachable central and deliver a
|
||||
@@ -43,7 +43,7 @@ namespace ZB.MOM.WW.OtOpcUa.Runtime.Communication;
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>All endpoints failing is a DROP — but a loud one.</b> That is the same outcome the
|
||||
/// ClusterClient path had when it could not reach central; what changes is that the operator
|
||||
/// node has when it cannot reach central; what changes is that the operator
|
||||
/// now gets the node, the deployment, the correlation id and the status each endpoint
|
||||
/// answered with, which is what distinguishes "central is down" from "central rejected this
|
||||
/// node's key".
|
||||
|
||||
@@ -4,15 +4,15 @@ namespace ZB.MOM.WW.OtOpcUa.Runtime.Communication;
|
||||
|
||||
/// <summary>
|
||||
/// Posts this node's deployment acknowledgment to central over the <c>mesh_transport.v1</c>
|
||||
/// command plane (ClusterClient→gRPC migration Phase 1B). The seam
|
||||
/// command plane (mesh gRPC command plane, Phase 1B). The seam
|
||||
/// <see cref="NodeCommunicationActor"/> uses under <c>MeshTransport:Mode = Grpc</c>, in place of
|
||||
/// the <c>ClusterClient.Send</c> it uses under <c>ClusterClient</c>.
|
||||
/// the node-local DistributedPubSub publish it uses under <c>Dps</c>.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// <b>One ack per node per dispatch.</b> Whatever the implementation does about redundancy, it
|
||||
/// posts to <i>one</i> central endpoint at a time rather than fanning out — the semantic of the
|
||||
/// <c>ClusterClient.Send</c> this replaces (<c>Send</c>, deliberately not <c>SendToAll</c>).
|
||||
/// one-ack-per-node-per-dispatch semantic the deploy coordinator counts on.
|
||||
/// This is <b>defence in depth, not a correctness dependency</b>: the coordinator counts acks
|
||||
/// in a <c>NodeId</c>-keyed dictionary behind an expected-set membership guard
|
||||
/// (<c>ConfigPublishCoordinator.HandleAck</c>), so a duplicate from one node overwrites its own
|
||||
|
||||
@@ -374,9 +374,8 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers
|
||||
/// <param name="dbFactory">Database context factory for configuration database access.</param>
|
||||
/// <param name="localNode">The local cluster node identifier.</param>
|
||||
/// <param name="ackRouter">Where ApplyAcks are sent. A direct coordinator handle in tests; under
|
||||
/// <c>MeshTransport:Mode = ClusterClient</c> <b>and</b> <c>= Grpc</c> the node's
|
||||
/// <c>NodeCommunicationActor</c>, which relays across the boundary over whichever transport it was
|
||||
/// built with. Null publishes on the <c>deployment-acks</c> DPS topic.</param>
|
||||
/// <c>MeshTransport:Mode = Grpc</c> the node's <c>NodeCommunicationActor</c>, which relays the ack
|
||||
/// across the mesh boundary. Null publishes on the <c>deployment-acks</c> DPS topic.</param>
|
||||
/// <param name="driverFactory">Optional driver factory; defaults to null factory if not provided.</param>
|
||||
/// <param name="localRoles">Optional set of roles assigned to the local node.</param>
|
||||
/// <param name="dependencyMux">Optional actor reference for dependency multiplexing.</param>
|
||||
@@ -459,9 +458,8 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers
|
||||
/// <param name="dbFactory">Database context factory for configuration database access.</param>
|
||||
/// <param name="localNode">The local cluster node identifier.</param>
|
||||
/// <param name="ackRouter">Where ApplyAcks are sent. A direct coordinator handle in tests; under
|
||||
/// <c>MeshTransport:Mode = ClusterClient</c> <b>and</b> <c>= Grpc</c> the node's
|
||||
/// <c>NodeCommunicationActor</c>, which relays across the boundary over whichever transport it was
|
||||
/// built with. Null publishes on the <c>deployment-acks</c> DPS topic.</param>
|
||||
/// <c>MeshTransport:Mode = Grpc</c> the node's <c>NodeCommunicationActor</c>, which relays the ack
|
||||
/// across the mesh boundary. Null publishes on the <c>deployment-acks</c> DPS topic.</param>
|
||||
/// <param name="driverFactory">Optional driver factory; defaults to null factory if not provided.</param>
|
||||
/// <param name="localRoles">Optional set of roles assigned to the local node.</param>
|
||||
/// <param name="dependencyMux">Optional actor reference for dependency multiplexing.</param>
|
||||
@@ -584,11 +582,11 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers
|
||||
_mediator.Tell(
|
||||
new Subscribe(ZB.MOM.WW.OtOpcUa.Runtime.OpcUa.OpcUaPublishActor.RedundancyStateTopic, Self));
|
||||
|
||||
// Per-cluster mesh Phase 2 dark switch. Under MeshTransport:Mode = ClusterClient these same
|
||||
// commands arrive from NodeCommunicationActor on the node-local EventStream instead of the
|
||||
// mesh-wide DPS topic. Subscribing to BOTH unconditionally is deliberate: exactly one of the
|
||||
// two ever publishes, so the transport flag lives entirely on the publishing side and the
|
||||
// switch can be flipped (or reverted) without touching any subscriber.
|
||||
// Per-cluster mesh Phase 2. Under MeshTransport:Mode = Grpc these same commands arrive from
|
||||
// NodeCommunicationActor on the node-local EventStream instead of the mesh-wide DPS topic.
|
||||
// Subscribing to BOTH unconditionally is deliberate: exactly one of the two ever publishes, so
|
||||
// the transport choice lives entirely on the publishing side and the mode can be flipped (or
|
||||
// reverted) without touching any subscriber.
|
||||
Context.System.EventStream.Subscribe(Self, typeof(DispatchDeployment));
|
||||
Context.System.EventStream.Subscribe(Self, typeof(RestartDriver));
|
||||
Context.System.EventStream.Subscribe(Self, typeof(ReconnectDriver));
|
||||
@@ -2492,11 +2490,11 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers
|
||||
var ack = new ApplyAck(deploymentId, _localNode, outcome, failureReason, correlation);
|
||||
if (_ackRouter is not null)
|
||||
{
|
||||
// Either a direct coordinator handle (tests) or, under MeshTransport:Mode = ClusterClient
|
||||
// and = Grpc alike, the NodeCommunicationActor that relays this across the boundary (it
|
||||
// owns the choice of transport, not this actor). Renamed from _coordinatorOverride in
|
||||
// per-cluster mesh Phase 2: under those two modes this ref is emphatically NOT a
|
||||
// coordinator, and the old name would send the next reader looking for one.
|
||||
// Either a direct coordinator handle (tests) or, under MeshTransport:Mode = Grpc, the
|
||||
// NodeCommunicationActor that relays this across the mesh boundary (it owns the transport,
|
||||
// not this actor). Renamed from _coordinatorOverride in per-cluster mesh Phase 2: under
|
||||
// Grpc this ref is emphatically NOT a coordinator, and the old name would send the next
|
||||
// reader looking for one.
|
||||
_ackRouter.Tell(ack);
|
||||
}
|
||||
else
|
||||
|
||||
@@ -627,7 +627,7 @@ public sealed class ScriptedAlarmHostActor : ReceiveActor
|
||||
|
||||
// Per-cluster mesh Phase 2 dark switch. An AdminUI-originated ack/shelve arrives from
|
||||
// NodeCommunicationActor on the node-local EventStream under
|
||||
// MeshTransport:Mode = ClusterClient; the DPS subscribe above still carries the node-LOCAL
|
||||
// MeshTransport:Mode = Grpc; the DPS subscribe above still carries the node-LOCAL
|
||||
// publisher (OtOpcUaServerHostedService, the OPC UA Part 9 method calls), which never
|
||||
// crosses the boundary and stays on DPS in both modes. Subscribing to both is safe: exactly
|
||||
// one publisher feeds each path.
|
||||
|
||||
Reference in New Issue
Block a user