feat(comm): Phase 4 — delete Akka ClusterClient site↔central transport, gRPC-only

ClusterClient→gRPC migration Phase 4 (docs/plans/2026-07-22-clusterclient-to-grpc-plan.md).
Phases 2/3 proved both directions on gRPC; this removes the Akka transport underneath.

Deleted:
- AkkaCentralTransport, AkkaSiteTransport (+ their dedicated tests)
- ISiteClientFactory + DefaultSiteClientFactory; CentralCommunicationActor legacy
  ctor + SelectTransport (Host now builds GrpcSiteTransport and injects it)
- ClusterClient creation + both ClusterClientReceptionist.RegisterService calls in
  AkkaHostedService; the RegisterCentralClient message + receive block
- CommunicationOptions.CentralContactPoints; the CentralTransport/SiteTransport
  coexistence flags; the CentralTransportMode/SiteTransportKind enums

gRPC is now the only site↔central transport (site→central CentralControlService via
GrpcCentralTransport; central→site SiteCommandService via GrpcSiteTransport), both
built unconditionally by the Host. NoOpCentralTransport is the fail-loud null-default
so TestKit command-dispatch suites still construct the site actor without a wired
transport; production always injects GrpcCentralTransport.

Config: CentralGrpcEndpoints is now unconditional — CommunicationOptionsValidator
rejects blank entries (role-agnostic), and StartupValidator requires a Site node to
list >=1 endpoint (fail-fast, mirrors GrpcPsk). Rig configs moved
CentralContactPoints -> CentralGrpcEndpoints (docker x6, docker-env2 x2, Host default,
deploy/wonder-app-vd03). Kept Akka.Cluster.Tools (ClusterSingleton still used).

Tests: build 0/0; Communication.Tests 640, Host.Tests 421 green. Removed the
ClusterClient.Send per-site-routing tests (covered by the transport suites), swapped
the ISiteClientFactory-based ctors to a substitute ISiteCommandTransport, converted
the audit-push integration relay to an in-process bridge transport.

Docs: Component-Communication/Host/StoreAndForward, components/Communication,
topology-guide, grpc_streams (SUPERSEDED note), the frame-size known-issue (retired
amendment), and CLAUDE.md transport decisions.

Not included: the dead IntegrationCallRequest path (#32) is a separate user-owned
behavioral decision — SiteEnvelope routing is transport-agnostic so it still compiles.
This commit is contained in:
Joseph Doherty
2026-07-23 12:54:32 -04:00
parent 3a8ddb7087
commit 7fd5cb2b56
42 changed files with 479 additions and 1295 deletions
@@ -1,7 +1,5 @@
using System.Collections.Immutable;
using Akka.Actor;
using Akka.Cluster;
using Akka.Cluster.Tools.Client;
using Akka.Cluster.Tools.Singleton;
using Akka.Configuration;
using Microsoft.Extensions.Options;
@@ -428,15 +426,21 @@ akka {{
var centralHealthCollector = _serviceProvider.GetService<ZB.MOM.WW.ScadaBridge.HealthMonitoring.ISiteHealthCollector>();
centralHealthCollector?.SetNodeHostname(_nodeOptions.NodeHostname);
var siteClientFactory = new DefaultSiteClientFactory();
// Central→site command transport: gRPC SiteCommandService. This is the ONLY transport since
// the ClusterClient→gRPC migration's Phase 4 removed the Akka path; the actor no longer
// chooses a transport, the Host builds it. Constructed once from the DI
// SitePairChannelProvider and shared across CentralCommunicationActor incarnations (it holds
// only the shared provider + options); the actor reconciles its per-site channel pairs from
// the DB refresh loop.
var siteCommandTransport = new GrpcSiteTransport(
_serviceProvider.GetRequiredService<SitePairChannelProvider>(),
_communicationOptions,
_serviceProvider.GetRequiredService<ILoggerFactory>().CreateLogger<GrpcSiteTransport>());
_logger.LogInformation("central→site command transport: gRPC (SiteCommandService)");
var centralCommActor = _actorSystem!.ActorOf(
Props.Create(() => new CentralCommunicationActor(_serviceProvider, siteClientFactory)),
Props.Create(() => new CentralCommunicationActor(_serviceProvider, siteCommandTransport)),
"central-communication");
// Register CentralCommunicationActor with ClusterClientReceptionist so site ClusterClients can reach it
ClusterClientReceptionist.Get(_actorSystem).RegisterService(centralCommActor);
_logger.LogInformation("CentralCommunicationActor registered with ClusterClientReceptionist");
// Hand the same actor to the central-hosted gRPC control plane (T1A.2) and open its
// readiness gate — the gRPC face Asks this exact actor, so both transports resolve to
// one handler implementation. Mirrors SiteStreamGrpcServer.SetReady on the site side:
@@ -833,35 +837,25 @@ akka {{
_logger, role: siteRole);
var dmProxy = dm.Proxy;
// Select the site→central transport behind the coexistence flag (default Akka
// ClusterClient). When gRPC is chosen the site dials CentralControlService directly with
// a sticky-failover channel pair, presenting its own preshared key; the ClusterClient
// below is then not created at all.
ICentralTransport? centralTransport = null;
if (_communicationOptions.CentralTransport == CentralTransportMode.Grpc)
{
var loggerFactory = _serviceProvider.GetRequiredService<ILoggerFactory>();
var channelProvider = new CentralChannelProvider(
_communicationOptions.CentralGrpcEndpoints,
new StaticSitePskProvider(_communicationOptions.GrpcPsk),
_nodeOptions.SiteId!,
_communicationOptions,
loggerFactory.CreateLogger<CentralChannelProvider>());
_trackedDisposables.Add(channelProvider);
centralTransport = new GrpcCentralTransport(
channelProvider,
_communicationOptions,
loggerFactory.CreateLogger<GrpcCentralTransport>());
_logger.LogInformation(
"Site→central transport: gRPC to {Count} central endpoint(s) for site {SiteId}",
_communicationOptions.CentralGrpcEndpoints.Count, _nodeOptions.SiteId);
}
else
{
_logger.LogInformation(
"Site→central transport: Akka ClusterClient (default) for site {SiteId}",
_nodeOptions.SiteId);
}
// Site→central transport: a gRPC dial of CentralControlService with a sticky
// central-a→central-b channel pair, presenting this site's preshared key. This is the ONLY
// transport since the ClusterClient→gRPC migration's Phase 4 removed the Akka path;
// StartupValidator guarantees a Site node lists at least one CentralGrpcEndpoint.
var loggerFactory = _serviceProvider.GetRequiredService<ILoggerFactory>();
var channelProvider = new CentralChannelProvider(
_communicationOptions.CentralGrpcEndpoints,
new StaticSitePskProvider(_communicationOptions.GrpcPsk),
_nodeOptions.SiteId!,
_communicationOptions,
loggerFactory.CreateLogger<CentralChannelProvider>());
_trackedDisposables.Add(channelProvider);
ICentralTransport centralTransport = new GrpcCentralTransport(
channelProvider,
_communicationOptions,
loggerFactory.CreateLogger<GrpcCentralTransport>());
_logger.LogInformation(
"Site→central transport: gRPC to {Count} central endpoint(s) for site {SiteId}",
_communicationOptions.CentralGrpcEndpoints.Count, _nodeOptions.SiteId);
// The ONE routing table for central→site commands, shared by the Akka
// SiteCommunicationActor (below) and the gRPC SiteCommandGrpcService (SetReady at the end
@@ -997,36 +991,10 @@ akka {{
siteCommActor.Tell(new RegisterLocalHandler(LocalHandlerType.ParkedMessages, parkedMessageHandler));
}
// Register SiteCommunicationActor with ClusterClientReceptionist so central ClusterClients can reach it
ClusterClientReceptionist.Get(_actorSystem).RegisterService(siteCommActor);
_logger.LogInformation(
"Site actors registered. DeploymentManager singleton scoped to role={SiteRole}, SiteCommunicationActor created.",
siteRole);
// Create ClusterClient to central if contact points are configured — but only on the Akka
// transport. On the gRPC transport the SiteCommunicationActor already holds a
// GrpcCentralTransport and never receives RegisterCentralClient, so a ClusterClient here
// would be dead weight (and keep an unwanted cross-cluster Akka association alive).
if (_communicationOptions.CentralTransport == CentralTransportMode.Akka
&& _communicationOptions.CentralContactPoints.Count > 0)
{
var contacts = _communicationOptions.CentralContactPoints
.Select(cp => ActorPath.Parse($"{cp}/system/receptionist"))
.ToImmutableHashSet();
var clientSettings = ClusterClientSettings.Create(_actorSystem)
.WithInitialContacts(contacts);
var centralClient = _actorSystem.ActorOf(
ClusterClient.Props(clientSettings), "central-cluster-client");
var siteCommSelection = _actorSystem.ActorSelection("/user/site-communication");
siteCommSelection.Tell(new RegisterCentralClient(centralClient));
_logger.LogInformation(
"Created ClusterClient to central with {Count} contact point(s) for site {SiteId}",
contacts.Count, _nodeOptions.SiteId);
}
// Per-node startup reconciliation. Created on EVERY site node (NOT a
// singleton) so a standby that was DOWN during a deploy self-heals on its next
// restart: it reports its local deployed inventory to central via the