feat(comm): Phase 4 — delete Akka ClusterClient site↔central transport, gRPC-only
ClusterClient→gRPC migration Phase 4 (docs/plans/2026-07-22-clusterclient-to-grpc-plan.md). Phases 2/3 proved both directions on gRPC; this removes the Akka transport underneath. Deleted: - AkkaCentralTransport, AkkaSiteTransport (+ their dedicated tests) - ISiteClientFactory + DefaultSiteClientFactory; CentralCommunicationActor legacy ctor + SelectTransport (Host now builds GrpcSiteTransport and injects it) - ClusterClient creation + both ClusterClientReceptionist.RegisterService calls in AkkaHostedService; the RegisterCentralClient message + receive block - CommunicationOptions.CentralContactPoints; the CentralTransport/SiteTransport coexistence flags; the CentralTransportMode/SiteTransportKind enums gRPC is now the only site↔central transport (site→central CentralControlService via GrpcCentralTransport; central→site SiteCommandService via GrpcSiteTransport), both built unconditionally by the Host. NoOpCentralTransport is the fail-loud null-default so TestKit command-dispatch suites still construct the site actor without a wired transport; production always injects GrpcCentralTransport. Config: CentralGrpcEndpoints is now unconditional — CommunicationOptionsValidator rejects blank entries (role-agnostic), and StartupValidator requires a Site node to list >=1 endpoint (fail-fast, mirrors GrpcPsk). Rig configs moved CentralContactPoints -> CentralGrpcEndpoints (docker x6, docker-env2 x2, Host default, deploy/wonder-app-vd03). Kept Akka.Cluster.Tools (ClusterSingleton still used). Tests: build 0/0; Communication.Tests 640, Host.Tests 421 green. Removed the ClusterClient.Send per-site-routing tests (covered by the transport suites), swapped the ISiteClientFactory-based ctors to a substitute ISiteCommandTransport, converted the audit-push integration relay to an in-process bridge transport. Docs: Component-Communication/Host/StoreAndForward, components/Communication, topology-guide, grpc_streams (SUPERSEDED note), the frame-size known-issue (retired amendment), and CLAUDE.md transport decisions. Not included: the dead IntegrationCallRequest path (#32) is a separate user-owned behavioral decision — SiteEnvelope routing is transport-agnostic so it still compiles.
This commit is contained in:
Joseph Doherty
@@ -1,11 +1,8 @@
|
||||
using System.Collections.Immutable;
|
||||
using Akka.Actor;
|
||||
using Akka.Cluster.Tools.Client;
|
||||
using Akka.Cluster.Tools.PublishSubscribe;
|
||||
using Akka.Event;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Audit;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Communication;
|
||||
@@ -17,61 +14,10 @@ using ZB.MOM.WW.ScadaBridge.HealthMonitoring;
|
||||
namespace ZB.MOM.WW.ScadaBridge.Communication.Actors;
|
||||
|
||||
/// <summary>
|
||||
/// Abstraction for creating ClusterClient instances per site, enabling testability.
|
||||
/// </summary>
|
||||
public interface ISiteClientFactory
|
||||
{
|
||||
/// <summary>Creates a ClusterClient actor for the given site with the specified contact points.</summary>
|
||||
/// <param name="system">The actor system in which to create the client.</param>
|
||||
/// <param name="siteId">The site identifier, used to name the actor.</param>
|
||||
/// <param name="contacts">The set of receptionist actor paths to use as initial contacts.</param>
|
||||
/// <returns>An actor reference for the new ClusterClient.</returns>
|
||||
IActorRef Create(ActorSystem system, string siteId, ImmutableHashSet<ActorPath> contacts);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Default implementation that creates a real ClusterClient for each site.
|
||||
/// </summary>
|
||||
public class DefaultSiteClientFactory : ISiteClientFactory
|
||||
{
|
||||
/// <summary>
|
||||
/// Per-incarnation generation counter. Context.Stop of the previous client is
|
||||
/// asynchronous — its actor name stays reserved until termination completes —
|
||||
/// so a same-named recreate in the same message handling throws
|
||||
/// InvalidActorNameException. A generation suffix makes every incarnation's
|
||||
/// name unique by construction (mirrors SiteStreamGrpcServer._actorCounter).
|
||||
/// </summary>
|
||||
private long _generation;
|
||||
|
||||
/// <inheritdoc />
|
||||
public IActorRef Create(ActorSystem system, string siteId, ImmutableHashSet<ActorPath> contacts)
|
||||
{
|
||||
var settings = ClusterClientSettings.Create(system).WithInitialContacts(contacts);
|
||||
var name = $"site-client-{SanitizeForActorName(siteId)}-{System.Threading.Interlocked.Increment(ref _generation)}";
|
||||
return system.ActorOf(ClusterClient.Props(settings), name);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Maps an arbitrary SiteIdentifier onto a valid Akka actor-path element:
|
||||
/// letters/digits/'-'/'_' pass through, everything else becomes '_'. Uniqueness
|
||||
/// is NOT required here (the generation suffix guarantees it); only validity is.
|
||||
/// </summary>
|
||||
/// <param name="siteId">The SiteIdentifier to sanitize.</param>
|
||||
/// <returns>A valid Akka actor-path element derived from <paramref name="siteId"/>.</returns>
|
||||
internal static string SanitizeForActorName(string siteId)
|
||||
{
|
||||
if (string.IsNullOrEmpty(siteId)) return "site";
|
||||
var sanitized = new string(siteId
|
||||
.Select(c => char.IsAsciiLetterOrDigit(c) || c is '-' or '_' ? c : '_')
|
||||
.ToArray());
|
||||
return ActorPath.IsValidPathElement(sanitized) ? sanitized : "site";
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Central-side actor that routes messages from central to site clusters via ClusterClient.
|
||||
/// Resolves site addresses from the database on a periodic refresh cycle and manages
|
||||
/// per-site ClusterClient instances.
|
||||
/// Central-side actor that routes messages from central to site clusters over the gRPC
|
||||
/// <c>SiteCommandService</c> command plane (<see cref="Grpc.GrpcSiteTransport"/>). Resolves site
|
||||
/// addresses from the database on a periodic refresh cycle and reconciles the transport's per-site
|
||||
/// channel pairs.
|
||||
///
|
||||
/// All 8 message patterns routed through this actor.
|
||||
/// Ask timeout on connection drop (no central buffering). Debug streams killed on interruption.
|
||||
@@ -82,11 +28,10 @@ public class CentralCommunicationActor : ReceiveActor
|
||||
private readonly IServiceProvider _serviceProvider;
|
||||
|
||||
/// <summary>
|
||||
/// The active central→site command transport, chosen by
|
||||
/// <c>ScadaBridge:Communication:SiteTransport</c> (default <see cref="SiteTransportKind.Akka"/>).
|
||||
/// The <see cref="SiteEnvelope"/> handler delegates every send here, and each DB refresh tick
|
||||
/// reconciles its per-site resources (ClusterClients for Akka, channel pairs for gRPC). Assigned
|
||||
/// in the public constructor body before any message can arrive.
|
||||
/// The central→site command transport (the gRPC <see cref="Grpc.GrpcSiteTransport"/>, built by
|
||||
/// the Host and injected). The <see cref="SiteEnvelope"/> handler delegates every send here, and
|
||||
/// each DB refresh tick reconciles its per-site channel pairs. Assigned in the constructor body
|
||||
/// before any message can arrive.
|
||||
/// </summary>
|
||||
private ISiteCommandTransport _transport = null!;
|
||||
|
||||
@@ -160,32 +105,9 @@ public class CentralCommunicationActor : ReceiveActor
|
||||
private const string HealthReportTopic = "site-health-replica";
|
||||
|
||||
/// <summary>
|
||||
/// Legacy constructor: builds the transport by reading
|
||||
/// <c>ScadaBridge:Communication:SiteTransport</c> and wrapping <paramref name="siteClientFactory"/>
|
||||
/// in an <see cref="AkkaSiteTransport"/> (default) or resolving the gRPC transport from
|
||||
/// <paramref name="serviceProvider"/>. Kept so the Host and the existing TestKit suites construct
|
||||
/// the actor exactly as before (the factory is still the Akka seam).
|
||||
/// </summary>
|
||||
/// <param name="serviceProvider">DI service provider for scoped repository and aggregator access.</param>
|
||||
/// <param name="siteClientFactory">Factory used to create per-site ClusterClient actors (Akka transport).</param>
|
||||
/// <param name="auditIngestAskTimeout">
|
||||
/// Optional override for the audit-ingest Ask timeout; defaults to
|
||||
/// <see cref="Grpc.SiteStreamGrpcServer.AuditIngestAskTimeout"/> (30 s). Exists only so tests can
|
||||
/// exercise the timeout/fault path quickly — production always uses the default.
|
||||
/// </param>
|
||||
public CentralCommunicationActor(
|
||||
IServiceProvider serviceProvider,
|
||||
ISiteClientFactory siteClientFactory,
|
||||
TimeSpan? auditIngestAskTimeout = null)
|
||||
: this(serviceProvider, auditIngestAskTimeout)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(siteClientFactory);
|
||||
_transport = SelectTransport(serviceProvider, siteClientFactory);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Primary constructor: takes the already-selected <see cref="ISiteCommandTransport"/> directly.
|
||||
/// Used by tests that substitute the transport, and reachable via the legacy constructor.
|
||||
/// Constructs the actor over the given central→site command transport (production injects the
|
||||
/// gRPC <see cref="Grpc.GrpcSiteTransport"/>, built by the Host from the DI
|
||||
/// <see cref="Grpc.SitePairChannelProvider"/>; tests substitute a fake).
|
||||
/// </summary>
|
||||
/// <param name="serviceProvider">DI service provider for scoped repository and aggregator access.</param>
|
||||
/// <param name="transport">The central→site command transport to route every <see cref="SiteEnvelope"/> through.</param>
|
||||
@@ -223,7 +145,7 @@ public class CentralCommunicationActor : ReceiveActor
|
||||
// distinguish "no sites configured" from "database is down". Log at Warning.
|
||||
Receive<Status.Failure>(failure =>
|
||||
_log.Warning(failure.Cause,
|
||||
"Failed to load site addresses from the database; the site ClusterClient "
|
||||
"Failed to load site addresses from the database; the per-site gRPC channel "
|
||||
+ "cache was not refreshed and may be stale or empty"));
|
||||
|
||||
// Health monitoring: heartbeats and health reports from sites
|
||||
@@ -489,43 +411,12 @@ public class CentralCommunicationActor : ReceiveActor
|
||||
|
||||
private void HandleSiteEnvelope(SiteEnvelope envelope)
|
||||
{
|
||||
// Below-the-seam routing: the active transport (Akka ClusterClient or gRPC) owns the
|
||||
// "no route for this site ⇒ warn + drop, caller's Ask times out" contract. Sender is the
|
||||
// temporary Ask actor (or the debug-bridge actor) and is preserved for reply routing.
|
||||
// Below-the-seam routing: the gRPC transport owns the "no route for this site ⇒ warn +
|
||||
// drop, caller's Ask times out" contract. Sender is the temporary Ask actor (or the
|
||||
// debug-bridge actor) and is preserved for reply routing.
|
||||
_transport.Send(envelope, Sender);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Chooses the transport from <c>ScadaBridge:Communication:SiteTransport</c> (default
|
||||
/// <see cref="SiteTransportKind.Akka"/>). For Akka, wraps the injected
|
||||
/// <see cref="ISiteClientFactory"/> in an <see cref="AkkaSiteTransport"/> bound to this actor's
|
||||
/// context; for gRPC, resolves the shared <see cref="Grpc.SitePairChannelProvider"/> and options.
|
||||
/// Runs in the constructor body, where <see cref="ActorBase.Context"/> is available.
|
||||
/// </summary>
|
||||
/// <param name="serviceProvider">The DI provider carrying options and (for gRPC) the channel provider.</param>
|
||||
/// <param name="siteClientFactory">The ClusterClient factory used by the Akka transport.</param>
|
||||
/// <returns>The selected transport.</returns>
|
||||
private ISiteCommandTransport SelectTransport(
|
||||
IServiceProvider serviceProvider, ISiteClientFactory siteClientFactory)
|
||||
{
|
||||
var options = serviceProvider.GetService<IOptions<CommunicationOptions>>()?.Value;
|
||||
var kind = options?.SiteTransport ?? SiteTransportKind.Akka;
|
||||
|
||||
if (kind == SiteTransportKind.Grpc)
|
||||
{
|
||||
var channelProvider = serviceProvider.GetRequiredService<Grpc.SitePairChannelProvider>();
|
||||
var loggerFactory = serviceProvider.GetRequiredService<ILoggerFactory>();
|
||||
_log.Info("central→site command transport: gRPC (SiteCommandService)");
|
||||
return new Grpc.GrpcSiteTransport(
|
||||
channelProvider,
|
||||
options!,
|
||||
loggerFactory.CreateLogger<Grpc.GrpcSiteTransport>());
|
||||
}
|
||||
|
||||
_log.Info("central→site command transport: Akka ClusterClient");
|
||||
return new AkkaSiteTransport(siteClientFactory, Context, _log);
|
||||
}
|
||||
|
||||
private void LoadSiteAddressesFromDb()
|
||||
{
|
||||
var self = Self;
|
||||
|
||||
Reference in New Issue
Block a user