dohertj2/ScadaBridge
1
0
Fork 0
Code Issues 2 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat(comm): Phase 4 — delete Akka ClusterClient site↔central transport, gRPC-only

Browse Source 
ClusterClient→gRPC migration Phase 4 (docs/plans/2026-07-22-clusterclient-to-grpc-plan.md).
Phases 2/3 proved both directions on gRPC; this removes the Akka transport underneath.

Deleted:
- AkkaCentralTransport, AkkaSiteTransport (+ their dedicated tests)
- ISiteClientFactory + DefaultSiteClientFactory; CentralCommunicationActor legacy
  ctor + SelectTransport (Host now builds GrpcSiteTransport and injects it)
- ClusterClient creation + both ClusterClientReceptionist.RegisterService calls in
  AkkaHostedService; the RegisterCentralClient message + receive block
- CommunicationOptions.CentralContactPoints; the CentralTransport/SiteTransport
  coexistence flags; the CentralTransportMode/SiteTransportKind enums

gRPC is now the only site↔central transport (site→central CentralControlService via
GrpcCentralTransport; central→site SiteCommandService via GrpcSiteTransport), both
built unconditionally by the Host. NoOpCentralTransport is the fail-loud null-default
so TestKit command-dispatch suites still construct the site actor without a wired
transport; production always injects GrpcCentralTransport.

Config: CentralGrpcEndpoints is now unconditional — CommunicationOptionsValidator
rejects blank entries (role-agnostic), and StartupValidator requires a Site node to
list >=1 endpoint (fail-fast, mirrors GrpcPsk). Rig configs moved
CentralContactPoints -> CentralGrpcEndpoints (docker x6, docker-env2 x2, Host default,
deploy/wonder-app-vd03). Kept Akka.Cluster.Tools (ClusterSingleton still used).

Tests: build 0/0; Communication.Tests 640, Host.Tests 421 green. Removed the
ClusterClient.Send per-site-routing tests (covered by the transport suites), swapped
the ISiteClientFactory-based ctors to a substitute ISiteCommandTransport, converted
the audit-push integration relay to an in-process bridge transport.

Docs: Component-Communication/Host/StoreAndForward, components/Communication,
topology-guide, grpc_streams (SUPERSEDED note), the frame-size known-issue (retired
amendment), and CLAUDE.md transport decisions.

Not included: the dead IntegrationCallRequest path (#32) is a separate user-owned
behavioral decision — SiteEnvelope routing is transport-agnostic so it still compiles.
This commit is contained in:
Joseph Doherty
2026-07-23 12:54:32 -04:00
parent 3a8ddb7087
commit 7fd5cb2b56
42 changed files with 479 additions and 1295 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/ZB.MOM.WW.ScadaBridge.Communication/Actors/AkkaCentralTransport.cs
-185
View File
@@ -1,185 +0,0 @@
using Akka.Actor;
using Akka.Cluster.Tools.Client;
using Akka.Event;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Audit;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Deployment;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Health;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Notification;
namespace ZB.MOM.WW.ScadaBridge.Communication.Actors;
/// <summary>
/// The <see cref="ICentralTransport"/> that carries the seven site→central sends over Akka
/// <c>ClusterClient</c> — the transport in production today, and the default. Every method is a
/// verbatim lift of the corresponding <c>SiteCommunicationActor</c> send block: it forwards a
/// <see cref="ClusterClient.Send"/> to <c>/user/central-communication</c> with the
/// <paramref name="replyTo"/> as the send's sender, so central's reply routes straight back to the
/// waiting Ask rather than through the site communication actor.
/// </summary>
/// <remarks>
/// The <c>ClusterClient</c> reference arrives after construction via
/// <see cref="SetCentralClient"/> (the actor forwards the <c>RegisterCentralClient</c> message it
/// receives once the Host builds the client). Until then — and if central contact points are not
/// configured at all — the client is null and each method answers the same transient-failure reply
/// the old inline handlers did.
/// </remarks>
public sealed class AkkaCentralTransport : ICentralTransport
{
/// <summary>The receptionist-registered path of the central communication actor.</summary>
private const string CentralPath = "/user/central-communication";
private readonly ILoggingAdapter? _log;
private IActorRef? _centralClient;
/// <summary>Creates the transport with no logging adapter (behaviourally identical; warnings are dropped).</summary>
public AkkaCentralTransport()
{
}
/// <summary>Creates the transport bound to the site communication actor's logging adapter.</summary>
/// <param name="log">Logging adapter used for the "no ClusterClient registered" warnings.</param>
public AkkaCentralTransport(ILoggingAdapter log)
{
_log = log;
}
/// <summary>
/// Registers the central <c>ClusterClient</c> once the Host has built it. Called from the site
/// communication actor's <c>RegisterCentralClient</c> handler.
/// </summary>
/// <param name="centralClient">The ClusterClient reaching the central cluster.</param>
public void SetCentralClient(IActorRef centralClient)
{
_centralClient = centralClient;
_log?.Info("Registered central ClusterClient");
}
/// <inheritdoc />
public void SubmitNotification(NotificationSubmit message, IActorRef replyTo)
{
if (_centralClient == null)
{
// No ClusterClient registered yet (e.g. central contact points not
// configured, or registration not yet completed). A non-accepted ack
// makes the S&F forwarder treat this as transient and retry later.
_log?.Warning(
"Cannot forward NotificationSubmit {0} — no central ClusterClient registered",
message.NotificationId);
replyTo.Tell(new NotificationSubmitAck(
message.NotificationId, Accepted: false, Error: "Central ClusterClient not registered"));
return;
}
_log?.Debug("Forwarding NotificationSubmit {0} to central", message.NotificationId);
_centralClient.Tell(new ClusterClient.Send(CentralPath, message), replyTo);
}
/// <inheritdoc />
public void QueryNotificationStatus(NotificationStatusQuery message, IActorRef replyTo)
{
if (_centralClient == null)
{
// No ClusterClient registered yet. Reply Found: false so Notify.Status
// falls back to the site S&F buffer to decide Forwarding vs Unknown.
_log?.Warning(
"Cannot forward NotificationStatusQuery {0} — no central ClusterClient registered",
message.NotificationId);
replyTo.Tell(new NotificationStatusResponse(
message.CorrelationId, Found: false, Status: "Unknown",
RetryCount: 0, LastError: null, DeliveredAt: null));
return;
}
_log?.Debug("Forwarding NotificationStatusQuery {0} to central", message.NotificationId);
_centralClient.Tell(new ClusterClient.Send(CentralPath, message), replyTo);
}
/// <inheritdoc />
public void IngestAuditEvents(IngestAuditEventsCommand message, IActorRef replyTo)
{
if (_centralClient == null)
{
// No ClusterClient registered yet. Faulting the Ask makes the
// SiteAuditTelemetryActor drain loop treat this as transient and keep
// the rows Pending for the next tick.
_log?.Warning(
"Cannot forward IngestAuditEventsCommand ({0} events) — no central ClusterClient registered",
message.Events.Count);
replyTo.Tell(new Status.Failure(
new InvalidOperationException("Central ClusterClient not registered")));
return;
}
_log?.Debug("Forwarding IngestAuditEventsCommand ({0} events) to central", message.Events.Count);
_centralClient.Tell(new ClusterClient.Send(CentralPath, message), replyTo);
}
/// <inheritdoc />
public void IngestCachedTelemetry(IngestCachedTelemetryCommand message, IActorRef replyTo)
{
if (_centralClient == null)
{
_log?.Warning(
"Cannot forward IngestCachedTelemetryCommand ({0} entries) — no central ClusterClient registered",
message.Entries.Count);
replyTo.Tell(new Status.Failure(
new InvalidOperationException("Central ClusterClient not registered")));
return;
}
_log?.Debug("Forwarding IngestCachedTelemetryCommand ({0} entries) to central", message.Entries.Count);
_centralClient.Tell(new ClusterClient.Send(CentralPath, message), replyTo);
}
/// <inheritdoc />
public void ReconcileSite(ReconcileSiteRequest message, IActorRef replyTo)
{
if (_centralClient == null)
{
// No ClusterClient registered yet. Faulting the Ask makes the
// SiteReconciliationActor treat the pass as best-effort-failed; it
// logs a warning and retries reconcile on the next node startup.
_log?.Warning(
"Cannot forward ReconcileSiteRequest for site {0} node {1} — no central ClusterClient registered",
message.SiteIdentifier, message.NodeId);
replyTo.Tell(new Status.Failure(
new InvalidOperationException("Central ClusterClient not registered")));
return;
}
_log?.Debug(
"Forwarding ReconcileSiteRequest for site {0} node {1} ({2} local instance(s)) to central",
message.SiteIdentifier, message.NodeId, message.LocalNameToRevisionHash.Count);
_centralClient.Tell(new ClusterClient.Send(CentralPath, message), replyTo);
}
/// <inheritdoc />
public void ReportSiteHealth(SiteHealthReport message, IActorRef replyTo)
{
if (_centralClient == null)
{
// No ClusterClient registered yet. A non-accepted ack makes the
// sender's counter-restore path treat this tick as a loss.
_log?.Warning(
"Cannot forward SiteHealthReport #{0} — no central ClusterClient registered",
message.SequenceNumber);
replyTo.Tell(new SiteHealthReportAck(
message.SiteId, message.SequenceNumber, Accepted: false,
Error: "Central ClusterClient not registered"));
return;
}
_centralClient.Tell(new ClusterClient.Send(CentralPath, message), replyTo);
}
/// <inheritdoc />
public void SendHeartbeat(HeartbeatMessage message, IActorRef self)
{
if (_centralClient == null)
{
return;
}
_centralClient.Tell(new ClusterClient.Send(CentralPath, message), self);
}
}
src/ZB.MOM.WW.ScadaBridge.Communication/Actors/AkkaSiteTransport.cs
-141
View File
@@ -1,141 +0,0 @@
using System.Collections.Immutable;
using Akka.Actor;
using Akka.Cluster.Tools.Client;
using Akka.Event;
namespace ZB.MOM.WW.ScadaBridge.Communication.Actors;
/// <summary>
/// The default (and, until the Phase 1B cutover, the shipping) central→site transport: routes each
/// <see cref="SiteEnvelope"/> through a per-site Akka <see cref="ClusterClient"/>, exactly as
/// <c>CentralCommunicationActor</c> did inline before the seam was extracted.
/// </summary>
/// <remarks>
/// <para>
/// Behaviour is identical to the pre-seam code: the per-site client map is (re)built from the DB
/// refresh cache; a send to a site with no client is warned and dropped (the caller's Ask times
/// out — central never buffers); a send preserves the reply-to sender so the site's reply routes
/// straight back to the waiting Ask (or the debug-bridge actor).
/// </para>
/// <para>
/// Both members run on the owning actor's thread, so the client map needs no synchronisation and
/// the stored <see cref="IActorContext"/> (used for <c>Context.Stop</c> and <c>Context.System</c>)
/// is only ever touched there.
/// </para>
/// </remarks>
public sealed class AkkaSiteTransport : ISiteCommandTransport
{
private readonly ISiteClientFactory _siteClientFactory;
private readonly IActorContext _context;
private readonly ILoggingAdapter _log;
/// <summary>
/// Per-site ClusterClient instances and their contact addresses.
/// Maps SiteIdentifier → (ClusterClient actor, set of contact address strings).
/// Refreshed by <see cref="ReconcileSites"/>.
/// </summary>
private readonly Dictionary<string, (IActorRef Client, ImmutableHashSet<string> ContactAddresses)> _siteClients = new();
/// <summary>Creates the Akka transport bound to the owning actor's context.</summary>
/// <param name="siteClientFactory">Factory that creates a ClusterClient per site.</param>
/// <param name="context">The owning actor's context (for <c>Stop</c>/<c>System</c>); calls stay on the actor thread.</param>
/// <param name="log">The owning actor's logger, so warnings keep the actor's log source.</param>
public AkkaSiteTransport(ISiteClientFactory siteClientFactory, IActorContext context, ILoggingAdapter log)
{
_siteClientFactory = siteClientFactory ?? throw new ArgumentNullException(nameof(siteClientFactory));
_context = context ?? throw new ArgumentNullException(nameof(context));
_log = log ?? throw new ArgumentNullException(nameof(log));
}
/// <inheritdoc />
public void Send(SiteEnvelope envelope, IActorRef replyTo)
{
ArgumentNullException.ThrowIfNull(envelope);
if (!_siteClients.TryGetValue(envelope.SiteId, out var entry))
{
_log.Warning("No ClusterClient for site {0}, cannot route message {1}",
envelope.SiteId, envelope.Message.GetType().Name);
// The Ask will timeout on the caller side — no central buffering
return;
}
// Route via ClusterClient — replyTo is preserved for Ask response routing
entry.Client.Tell(
new ClusterClient.Send("/user/site-communication", envelope.Message),
replyTo);
}
/// <inheritdoc />
public void ReconcileSites(SiteAddressCacheLoaded cache)
{
ArgumentNullException.ThrowIfNull(cache);
var newSiteIds = cache.SiteContacts.Keys.ToHashSet();
var existingSiteIds = _siteClients.Keys.ToHashSet();
// Stop ClusterClients for removed sites
foreach (var removed in existingSiteIds.Except(newSiteIds))
{
_log.Info("Stopping ClusterClient for removed site {0}", removed);
_context.Stop(_siteClients[removed].Client);
_siteClients.Remove(removed);
}
// Add or update
foreach (var (siteId, addresses) in cache.SiteContacts)
{
// Parse all addresses up front inside a try/catch so a
// single malformed site row cannot abort the whole refresh loop and leave
// the cache half-updated. A bad site is logged and skipped; others proceed.
ImmutableHashSet<ActorPath> contactPaths;
try
{
contactPaths = addresses
.Select(a => ActorPath.Parse($"{a}/system/receptionist"))
.ToImmutableHashSet();
}
catch (Exception ex)
{
_log.Warning(ex,
"Malformed contact address for site {0}; skipping this site in the refresh "
+ "(other sites are unaffected)", siteId);
continue;
}
var contactStrings = addresses.ToImmutableHashSet();
// Skip if unchanged
if (_siteClients.TryGetValue(siteId, out var existing) && existing.ContactAddresses.SetEquals(contactStrings))
continue;
// Stop old client if addresses changed
if (_siteClients.ContainsKey(siteId))
{
_log.Info("Updating ClusterClient for site {0} (addresses changed)", siteId);
_context.Stop(_siteClients[siteId].Client);
// Remove now: if the replacement create below fails, a stale entry
// would route envelopes to a stopping actor.
_siteClients.Remove(siteId);
}
IActorRef client;
try
{
client = _siteClientFactory.Create(_context.System, siteId, contactPaths);
}
catch (Exception ex)
{
_log.Error(ex,
"Failed to create ClusterClient for site {0}; site is unroutable until the next refresh",
siteId);
continue;
}
_siteClients[siteId] = (client, contactStrings);
_log.Info("Created ClusterClient for site {0} with {1} contact(s)", siteId, addresses.Count);
}
_log.Info("Site ClusterClient cache refreshed with {0} site(s)", _siteClients.Count);
}
}
src/ZB.MOM.WW.ScadaBridge.Communication/Actors/CentralCommunicationActor.cs
+15 -124
View File
@@ -1,11 +1,8 @@
using System.Collections.Immutable;
using Akka.Actor;
using Akka.Cluster.Tools.Client;
using Akka.Cluster.Tools.PublishSubscribe;
using Akka.Event;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Audit;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Communication;
@@ -17,61 +14,10 @@ using ZB.MOM.WW.ScadaBridge.HealthMonitoring;
namespace ZB.MOM.WW.ScadaBridge.Communication.Actors;
/// <summary>
/// Abstraction for creating ClusterClient instances per site, enabling testability.
/// </summary>
public interface ISiteClientFactory
{
/// <summary>Creates a ClusterClient actor for the given site with the specified contact points.</summary>
/// <param name="system">The actor system in which to create the client.</param>
/// <param name="siteId">The site identifier, used to name the actor.</param>
/// <param name="contacts">The set of receptionist actor paths to use as initial contacts.</param>
/// <returns>An actor reference for the new ClusterClient.</returns>
IActorRef Create(ActorSystem system, string siteId, ImmutableHashSet<ActorPath> contacts);
}
/// <summary>
/// Default implementation that creates a real ClusterClient for each site.
/// </summary>
public class DefaultSiteClientFactory : ISiteClientFactory
{
/// <summary>
/// Per-incarnation generation counter. Context.Stop of the previous client is
/// asynchronous — its actor name stays reserved until termination completes —
/// so a same-named recreate in the same message handling throws
/// InvalidActorNameException. A generation suffix makes every incarnation's
/// name unique by construction (mirrors SiteStreamGrpcServer._actorCounter).
/// </summary>
private long _generation;
/// <inheritdoc />
public IActorRef Create(ActorSystem system, string siteId, ImmutableHashSet<ActorPath> contacts)
{
var settings = ClusterClientSettings.Create(system).WithInitialContacts(contacts);
var name = $"site-client-{SanitizeForActorName(siteId)}-{System.Threading.Interlocked.Increment(ref _generation)}";
return system.ActorOf(ClusterClient.Props(settings), name);
}
/// <summary>
/// Maps an arbitrary SiteIdentifier onto a valid Akka actor-path element:
/// letters/digits/'-'/'_' pass through, everything else becomes '_'. Uniqueness
/// is NOT required here (the generation suffix guarantees it); only validity is.
/// </summary>
/// <param name="siteId">The SiteIdentifier to sanitize.</param>
/// <returns>A valid Akka actor-path element derived from <paramref name="siteId"/>.</returns>
internal static string SanitizeForActorName(string siteId)
{
if (string.IsNullOrEmpty(siteId)) return "site";
var sanitized = new string(siteId
.Select(c => char.IsAsciiLetterOrDigit(c) || c is '-' or '_' ? c : '_')
.ToArray());
return ActorPath.IsValidPathElement(sanitized) ? sanitized : "site";
}
}
/// <summary>
/// Central-side actor that routes messages from central to site clusters via ClusterClient.
/// Resolves site addresses from the database on a periodic refresh cycle and manages
/// per-site ClusterClient instances.
/// Central-side actor that routes messages from central to site clusters over the gRPC
/// <c>SiteCommandService</c> command plane (<see cref="Grpc.GrpcSiteTransport"/>). Resolves site
/// addresses from the database on a periodic refresh cycle and reconciles the transport's per-site
/// channel pairs.
///
/// All 8 message patterns routed through this actor.
/// Ask timeout on connection drop (no central buffering). Debug streams killed on interruption.
@@ -82,11 +28,10 @@ public class CentralCommunicationActor : ReceiveActor
private readonly IServiceProvider _serviceProvider;
/// <summary>
/// The active central→site command transport, chosen by
/// <c>ScadaBridge:Communication:SiteTransport</c> (default <see cref="SiteTransportKind.Akka"/>).
/// The <see cref="SiteEnvelope"/> handler delegates every send here, and each DB refresh tick
/// reconciles its per-site resources (ClusterClients for Akka, channel pairs for gRPC). Assigned
/// in the public constructor body before any message can arrive.
/// The central→site command transport (the gRPC <see cref="Grpc.GrpcSiteTransport"/>, built by
/// the Host and injected). The <see cref="SiteEnvelope"/> handler delegates every send here, and
/// each DB refresh tick reconciles its per-site channel pairs. Assigned in the constructor body
/// before any message can arrive.
/// </summary>
private ISiteCommandTransport _transport = null!;
@@ -160,32 +105,9 @@ public class CentralCommunicationActor : ReceiveActor
private const string HealthReportTopic = "site-health-replica";
/// <summary>
/// Legacy constructor: builds the transport by reading
/// <c>ScadaBridge:Communication:SiteTransport</c> and wrapping <paramref name="siteClientFactory"/>
/// in an <see cref="AkkaSiteTransport"/> (default) or resolving the gRPC transport from
/// <paramref name="serviceProvider"/>. Kept so the Host and the existing TestKit suites construct
/// the actor exactly as before (the factory is still the Akka seam).
/// </summary>
/// <param name="serviceProvider">DI service provider for scoped repository and aggregator access.</param>
/// <param name="siteClientFactory">Factory used to create per-site ClusterClient actors (Akka transport).</param>
/// <param name="auditIngestAskTimeout">
/// Optional override for the audit-ingest Ask timeout; defaults to
/// <see cref="Grpc.SiteStreamGrpcServer.AuditIngestAskTimeout"/> (30 s). Exists only so tests can
/// exercise the timeout/fault path quickly — production always uses the default.
/// </param>
public CentralCommunicationActor(
IServiceProvider serviceProvider,
ISiteClientFactory siteClientFactory,
TimeSpan? auditIngestAskTimeout = null)
: this(serviceProvider, auditIngestAskTimeout)
{
ArgumentNullException.ThrowIfNull(siteClientFactory);
_transport = SelectTransport(serviceProvider, siteClientFactory);
}
/// <summary>
/// Primary constructor: takes the already-selected <see cref="ISiteCommandTransport"/> directly.
/// Used by tests that substitute the transport, and reachable via the legacy constructor.
/// Constructs the actor over the given central→site command transport (production injects the
/// gRPC <see cref="Grpc.GrpcSiteTransport"/>, built by the Host from the DI
/// <see cref="Grpc.SitePairChannelProvider"/>; tests substitute a fake).
/// </summary>
/// <param name="serviceProvider">DI service provider for scoped repository and aggregator access.</param>
/// <param name="transport">The central→site command transport to route every <see cref="SiteEnvelope"/> through.</param>
@@ -223,7 +145,7 @@ public class CentralCommunicationActor : ReceiveActor
// distinguish "no sites configured" from "database is down". Log at Warning.
Receive<Status.Failure>(failure =>
_log.Warning(failure.Cause,
"Failed to load site addresses from the database; the site ClusterClient "
"Failed to load site addresses from the database; the per-site gRPC channel "
+ "cache was not refreshed and may be stale or empty"));
// Health monitoring: heartbeats and health reports from sites
@@ -489,43 +411,12 @@ public class CentralCommunicationActor : ReceiveActor
private void HandleSiteEnvelope(SiteEnvelope envelope)
{
// Below-the-seam routing: the active transport (Akka ClusterClient or gRPC) owns the
// "no route for this site ⇒ warn + drop, caller's Ask times out" contract. Sender is the
// temporary Ask actor (or the debug-bridge actor) and is preserved for reply routing.
// Below-the-seam routing: the gRPC transport owns the "no route for this site ⇒ warn +
// drop, caller's Ask times out" contract. Sender is the temporary Ask actor (or the
// debug-bridge actor) and is preserved for reply routing.
_transport.Send(envelope, Sender);
}
/// <summary>
/// Chooses the transport from <c>ScadaBridge:Communication:SiteTransport</c> (default
/// <see cref="SiteTransportKind.Akka"/>). For Akka, wraps the injected
/// <see cref="ISiteClientFactory"/> in an <see cref="AkkaSiteTransport"/> bound to this actor's
/// context; for gRPC, resolves the shared <see cref="Grpc.SitePairChannelProvider"/> and options.
/// Runs in the constructor body, where <see cref="ActorBase.Context"/> is available.
/// </summary>
/// <param name="serviceProvider">The DI provider carrying options and (for gRPC) the channel provider.</param>
/// <param name="siteClientFactory">The ClusterClient factory used by the Akka transport.</param>
/// <returns>The selected transport.</returns>
private ISiteCommandTransport SelectTransport(
IServiceProvider serviceProvider, ISiteClientFactory siteClientFactory)
{
var options = serviceProvider.GetService<IOptions<CommunicationOptions>>()?.Value;
var kind = options?.SiteTransport ?? SiteTransportKind.Akka;
if (kind == SiteTransportKind.Grpc)
{
var channelProvider = serviceProvider.GetRequiredService<Grpc.SitePairChannelProvider>();
var loggerFactory = serviceProvider.GetRequiredService<ILoggerFactory>();
_log.Info("central→site command transport: gRPC (SiteCommandService)");
return new Grpc.GrpcSiteTransport(
channelProvider,
options!,
loggerFactory.CreateLogger<Grpc.GrpcSiteTransport>());
}
_log.Info("central→site command transport: Akka ClusterClient");
return new AkkaSiteTransport(siteClientFactory, Context, _log);
}
private void LoadSiteAddressesFromDb()
{
var self = Self;
src/ZB.MOM.WW.ScadaBridge.Communication/Actors/ICentralTransport.cs
+12 -13
View File
@@ -13,21 +13,20 @@ namespace ZB.MOM.WW.ScadaBridge.Communication.Actors;
/// <remarks>
/// <para>
/// The actor's receive handlers no longer own the wire plumbing — they capture the current
/// <c>Sender</c> and hand it to the transport as <paramref name="replyTo"/>. Two implementations
/// exist behind the <c>ScadaBridge:Communication:CentralTransport</c> flag: the default
/// <see cref="AkkaCentralTransport"/> (verbatim of the old <c>ClusterClient.Send</c> path,
/// including the exact sender-forwarding that routes central's reply straight back to the waiting
/// Ask) and <see cref="Grpc.GrpcCentralTransport"/> (a gRPC dial of <c>CentralControlService</c>).
/// <c>Sender</c> and hand it to the transport as <paramref name="replyTo"/>. The production
/// implementation is <see cref="Grpc.GrpcCentralTransport"/> (a gRPC dial of
/// <c>CentralControlService</c> with sticky central-a→central-b failover) — the only site→central
/// transport since the ClusterClient→gRPC migration removed the Akka path in Phase 4.
/// <see cref="NoOpCentralTransport"/> is the fail-loud placeholder used only when the Host injects
/// nothing (a wiring bug, and in TestKit suites that only exercise command dispatch).
/// </para>
/// <para>
/// <b>Reply/fault contract, identical on both transports.</b> Each Ask-returning method (all but
/// the heartbeat) guarantees exactly one reply eventually lands at <paramref name="replyTo"/>:
/// either the real reply type the caller Asks for, or a transient-failure signal. The Akka path
/// sends a not-accepted ack / <see cref="Status.Failure"/> when no ClusterClient is registered;
/// the gRPC path sends <see cref="Status.Failure"/> on any non-OK status (a timeout or an
/// <c>Unavailable</c> that could not be failed over). Both are what the S&amp;F / audit / health
/// layers above the seam already treat as transient — rows stay buffered, counters restore, the
/// pass re-runs.
/// <b>Reply/fault contract.</b> Each Ask-returning method (all but the heartbeat) guarantees exactly
/// one reply eventually lands at <paramref name="replyTo"/>: either the real reply type the caller
/// Asks for, or a transient-failure signal. The gRPC path sends <see cref="Status.Failure"/> on any
/// non-OK status (a timeout or an <c>Unavailable</c> that could not be failed over) — what the
/// S&amp;F / audit / health layers above the seam already treat as transient (rows stay buffered,
/// counters restore, the pass re-runs).
/// </para>
/// <para>
/// <b>The heartbeat stays fire-and-forget end-to-end.</b> <see cref="SendHeartbeat"/> takes no
src/ZB.MOM.WW.ScadaBridge.Communication/Actors/NoOpCentralTransport.cs
+62
View File
@@ -0,0 +1,62 @@
using Akka.Actor;
using Akka.Event;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Audit;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Deployment;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Health;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Notification;
namespace ZB.MOM.WW.ScadaBridge.Communication.Actors;
/// <summary>
/// The <see cref="ICentralTransport"/> used only when the Host injects none — a fail-loud placeholder,
/// not a working transport. Production always injects <see cref="Grpc.GrpcCentralTransport"/>; a null
/// here is a wiring bug, so every send warns and each Ask-returning method answers a
/// <see cref="Status.Failure"/> so the caller sees the same transient failure the old ClusterClient
/// path produced when no client was registered (rows stay buffered, the pass re-runs). The heartbeat
/// stays fire-and-forget: swallowed and logged, never faulting the heartbeat timer.
/// </summary>
/// <remarks>
/// This replaced the previous default (<c>AkkaCentralTransport</c> with no registered ClusterClient),
/// deleted when the site→central path became gRPC-only in the ClusterClient→gRPC migration's Phase 4.
/// It exists mostly so TestKit suites that only exercise central→site command dispatch — and never
/// inject a transport — construct the actor without wiring a real channel.
/// </remarks>
public sealed class NoOpCentralTransport : ICentralTransport
{
private readonly ILoggingAdapter _log;
/// <summary>Creates the placeholder transport with the owning actor's logging adapter.</summary>
/// <param name="log">The actor's logging adapter, used for the "no transport configured" warnings.</param>
public NoOpCentralTransport(ILoggingAdapter log) => _log = log;
private void Fail(string what, IActorRef replyTo)
{
_log.Warning(
"No site→central transport is configured; dropping {0} as a transient failure. This is a "
+ "wiring bug — the Host must inject a GrpcCentralTransport.", what);
replyTo.Tell(new Status.Failure(new InvalidOperationException(
$"No site→central transport configured (dropping {what}).")));
}
/// <inheritdoc />
public void SubmitNotification(NotificationSubmit message, IActorRef replyTo) => Fail(nameof(SubmitNotification), replyTo);
/// <inheritdoc />
public void QueryNotificationStatus(NotificationStatusQuery message, IActorRef replyTo) => Fail(nameof(QueryNotificationStatus), replyTo);
/// <inheritdoc />
public void IngestAuditEvents(IngestAuditEventsCommand message, IActorRef replyTo) => Fail(nameof(IngestAuditEvents), replyTo);
/// <inheritdoc />
public void IngestCachedTelemetry(IngestCachedTelemetryCommand message, IActorRef replyTo) => Fail(nameof(IngestCachedTelemetry), replyTo);
/// <inheritdoc />
public void ReconcileSite(ReconcileSiteRequest message, IActorRef replyTo) => Fail(nameof(ReconcileSite), replyTo);
/// <inheritdoc />
public void ReportSiteHealth(SiteHealthReport message, IActorRef replyTo) => Fail(nameof(ReportSiteHealth), replyTo);
/// <inheritdoc />
public void SendHeartbeat(HeartbeatMessage message, IActorRef self) =>
_log.Warning("No site→central transport is configured; heartbeat dropped (wiring bug).");
}
src/ZB.MOM.WW.ScadaBridge.Communication/Actors/SiteCommunicationActor.cs
+13 -25
View File
@@ -51,13 +51,14 @@ public class SiteCommunicationActor : ReceiveActor, IWithTimers
private readonly SiteCommandDispatcher _dispatcher;
/// <summary>
/// The site→central transport. Finalized in <see cref="PreStart"/> to the injected instance,
/// or a default <see cref="AkkaCentralTransport"/> (ClusterClient) when none is supplied — so
/// the seven site→central sends delegate here rather than owning the wire plumbing inline.
/// The site→central transport. Finalized in <see cref="PreStart"/> to the injected instance, or a
/// fail-loud <see cref="NoOpCentralTransport"/> when none is supplied — so the seven site→central
/// sends delegate here rather than owning the wire plumbing inline. Production injects the gRPC
/// <see cref="Grpc.GrpcCentralTransport"/>.
/// </summary>
private ICentralTransport _transport;
/// <summary>The transport supplied by the Host (null selects the default Akka transport).</summary>
/// <summary>The transport supplied by the Host (null falls back to <see cref="NoOpCentralTransport"/>).</summary>
private readonly ICentralTransport? _injectedTransport;
/// <summary>
@@ -81,10 +82,9 @@ public class SiteCommunicationActor : ReceiveActor, IWithTimers
/// ActorSystem.
/// </param>
/// <param name="transport">
/// The site→central transport. <c>null</c> (the default, used by every existing test and by
/// the Host's Akka path) selects an <see cref="AkkaCentralTransport"/> over ClusterClient; the
/// Host injects a <see cref="Grpc.GrpcCentralTransport"/> when
/// <c>ScadaBridge:Communication:CentralTransport</c> is <c>Grpc</c>.
/// The site→central transport. Production always passes the Host-built gRPC
/// <see cref="Grpc.GrpcCentralTransport"/>. <c>null</c> (used by TestKit suites that only
/// exercise command dispatch) falls back to a fail-loud <see cref="NoOpCentralTransport"/>.
/// </param>
/// <param name="dispatcher">
/// The shared <see cref="SiteCommandDispatcher"/> (production: created by the Host and also
@@ -121,13 +121,6 @@ public class SiteCommunicationActor : ReceiveActor, IWithTimers
ClusterState.ClusterFailoverCoordinator.FailOverOldest(system, role, dryRun)?.ToString();
_dispatcher = dispatcher ?? new SiteCommandDispatcher(siteId, deploymentManagerProxy, resolveFailover);
// Registration. Feeding the ClusterClient into the transport is a no-op unless the
// default/Akka transport is in use — the gRPC transport dials configured endpoints and
// never receives this message (the Host does not create a ClusterClient for it).
Receive<RegisterCentralClient>(msg =>
{
(_transport as AkkaCentralTransport)?.SetCentralClient(msg.Client);
});
Receive<RegisterLocalHandler>(HandleRegisterLocalHandler);
// ── The 27 migrated central→site commands (28th is failover, below) all route
@@ -234,11 +227,11 @@ public class SiteCommunicationActor : ReceiveActor, IWithTimers
/// <inheritdoc />
protected override void PreStart()
{
// Finalize the transport now that the actor context (and _log) exist. The default Akka
// transport is given this actor's logging adapter so the "no ClusterClient registered"
// warnings are preserved exactly. PreStart always runs before any message, so the Receive
// closures above see a non-null _transport.
_transport = _injectedTransport ?? new AkkaCentralTransport(_log);
// Finalize the transport now that the actor context (and _log) exist. When the Host injects
// none, fall back to a fail-loud NoOpCentralTransport (given this actor's logging adapter so
// "no transport configured" warnings surface). PreStart always runs before any message, so
// the Receive closures above see a non-null _transport.
_transport = _injectedTransport ?? new NoOpCentralTransport(_log);
_log.Info("SiteCommunicationActor started for site {0}", _siteId);
@@ -362,11 +355,6 @@ public class SiteCommunicationActor : ReceiveActor, IWithTimers
internal record SendHeartbeat;
}
/// <summary>
/// Command to register a ClusterClient for communicating with the central cluster.
/// </summary>
public record RegisterCentralClient(IActorRef Client);
/// <summary>
/// Command to register a local actor as a handler for a specific message pattern.
/// </summary>
src/ZB.MOM.WW.ScadaBridge.Communication/CommunicationOptions.cs
+2 -52
View File
@@ -1,51 +1,16 @@
namespace ZB.MOM.WW.ScadaBridge.Communication;
/// Which transport carries the seven site→central control messages. Selected per node by
/// <c>ScadaBridge:Communication:CentralTransport</c>; the migration ships with
/// <see cref="Akka"/> as the default so nothing flips until a node opts in.
/// </summary>
public enum CentralTransportMode
{
/// <summary>Akka <c>ClusterClient</c> — the transport in production today, and the default.</summary>
Akka = 0,
/// <summary>gRPC dial of the central <c>CentralControlService</c> (Phase 1A migration target).</summary>
Grpc = 1,
}
/// <summary>
/// Selects the transport the central→site command plane rides on. The Akka
/// per-site <c>ClusterClient</c> path is the default until the gRPC cutover
/// (ClusterClient→gRPC migration, Phase 1B); flipping to <see cref="Grpc"/> is the
/// rollback-by-flag switch.
/// </summary>
public enum SiteTransportKind
{
/// <summary>Route <c>SiteEnvelope</c>s through the per-site Akka <c>ClusterClient</c> (today's default).</summary>
Akka,
/// <summary>Route <c>SiteEnvelope</c>s over the site <c>SiteCommandService</c> gRPC plane.</summary>
Grpc
}
/// <summary>
/// Configuration options for central-site communication, including per-pattern
/// timeouts and transport heartbeat settings.
/// </summary>
public class CommunicationOptions
{
/// <summary>
/// Which transport carries the site→central control messages. Default <see cref="CentralTransportMode.Akka"/>
/// (ClusterClient) — coexistence rule: a node flips to gRPC only by setting this to <c>Grpc</c>,
/// and rollback is flipping it back. Selecting <c>Grpc</c> requires <see cref="CentralGrpcEndpoints"/>.
/// </summary>
public CentralTransportMode CentralTransport { get; set; } = CentralTransportMode.Akka;
/// <summary>
/// Central control-plane gRPC endpoints (preferred first), e.g.
/// <c>["http://scadabridge-central-a:8083", "http://scadabridge-central-b:8083"]</c>. Dialled by
/// <see cref="Grpc.CentralChannelProvider"/> with sticky failover/failback. Required when
/// <see cref="CentralTransport"/> is <see cref="CentralTransportMode.Grpc"/>, ignored otherwise.
/// <see cref="Grpc.CentralChannelProvider"/> with sticky failover/failback. Required on every site
/// node — gRPC (<c>CentralControlService</c>) is the only site→central transport.
/// </summary>
/// <remarks>
/// Sites reach central by container/host name, NOT via Traefik (which is HTTP/1 only; gRPC is
@@ -53,15 +18,6 @@ public class CommunicationOptions
/// </remarks>
public List<string> CentralGrpcEndpoints { get; set; } = new();
/// <summary>
/// Which transport the central→site command plane uses. Default <see cref="SiteTransportKind.Akka"/>
/// (the per-site ClusterClient path) — flipping to <see cref="SiteTransportKind.Grpc"/> moves
/// every <c>SiteEnvelope</c> onto the site <c>SiteCommandService</c> gRPC plane. Selected inside
/// <c>CentralCommunicationActor</c>; <c>CommunicationService</c> and <c>SiteCallAuditActor</c>
/// are unchanged either way. Rollback at any point = flip this back to <c>Akka</c>.
/// </summary>
public SiteTransportKind SiteTransport { get; set; } = SiteTransportKind.Akka;
/// <summary>Timeout for deployment commands (typically longest due to apply logic).</summary>
public TimeSpan DeploymentTimeout { get; set; } = TimeSpan.FromMinutes(2);
@@ -91,12 +47,6 @@ public class CommunicationOptions
/// </summary>
public TimeSpan NotificationForwardTimeout { get; set; } = TimeSpan.FromSeconds(30);
/// <summary>
/// Contact point addresses for the central cluster (e.g. "akka.tcp://scadabridge@central-a:8081").
/// Used by site nodes to create a ClusterClient for reaching central.
/// </summary>
public List<string> CentralContactPoints { get; set; } = new();
/// <summary>
/// Preshared key authenticating this node's gRPC control plane — the site↔central
/// boundary. On a site node this is the key its inbound gate
src/ZB.MOM.WW.ScadaBridge.Communication/CommunicationOptionsValidator.cs
+9 -12
View File
@@ -66,18 +66,15 @@ public sealed class CommunicationOptionsValidator : OptionsValidatorBase<Communi
builder.RequireThat(options.GrpcMaxConcurrentStreams > 0,
$"ScadaBridge:Communication:GrpcMaxConcurrentStreams must be positive (was {options.GrpcMaxConcurrentStreams}).");
// The gRPC site→central transport needs at least one central endpoint to dial. Only
// enforced when that transport is selected — the default Akka path ignores the list, so a
// node on ClusterClient must not be forced to declare gRPC endpoints it never uses.
if (options.CentralTransport == CentralTransportMode.Grpc)
{
builder.RequireThat(
options.CentralGrpcEndpoints.Count > 0
&& options.CentralGrpcEndpoints.All(e => !string.IsNullOrWhiteSpace(e)),
"ScadaBridge:Communication:CentralGrpcEndpoints must list at least one non-empty "
+ "central gRPC endpoint when CentralTransport is Grpc "
+ $"(was {options.CentralGrpcEndpoints.Count} entr{(options.CentralGrpcEndpoints.Count == 1 ? "y" : "ies")}).");
}
// The gRPC site→central transport needs at least one central endpoint to dial. gRPC is now
// the only site→central transport (ClusterClient was removed in the migration's Phase 4), so
// every site node must declare its central endpoints — there is no Akka fallback to ignore
// the list. Central nodes leave it empty (they host CentralControlService, they don't dial it).
builder.RequireThat(
options.CentralGrpcEndpoints.All(e => !string.IsNullOrWhiteSpace(e)),
"ScadaBridge:Communication:CentralGrpcEndpoints must not contain empty or whitespace "
+ "entries (a site node must list at least one central gRPC endpoint; central nodes "
+ "leave it empty).");
// ── Aggregated live alarm cache (plan #10, Task 6) ───────────────────────
// Linger drives a Timer dueTime; TimeSpan.Zero is valid (stop the aggregator
src/ZB.MOM.WW.ScadaBridge.Host/Actors/AkkaHostedService.cs
+31 -63
View File
@@ -1,7 +1,5 @@
using System.Collections.Immutable;
using Akka.Actor;
using Akka.Cluster;
using Akka.Cluster.Tools.Client;
using Akka.Cluster.Tools.Singleton;
using Akka.Configuration;
using Microsoft.Extensions.Options;
@@ -428,15 +426,21 @@ akka {{
var centralHealthCollector = _serviceProvider.GetService<ZB.MOM.WW.ScadaBridge.HealthMonitoring.ISiteHealthCollector>();
centralHealthCollector?.SetNodeHostname(_nodeOptions.NodeHostname);
var siteClientFactory = new DefaultSiteClientFactory();
// Central→site command transport: gRPC SiteCommandService. This is the ONLY transport since
// the ClusterClient→gRPC migration's Phase 4 removed the Akka path; the actor no longer
// chooses a transport, the Host builds it. Constructed once from the DI
// SitePairChannelProvider and shared across CentralCommunicationActor incarnations (it holds
// only the shared provider + options); the actor reconciles its per-site channel pairs from
// the DB refresh loop.
var siteCommandTransport = new GrpcSiteTransport(
_serviceProvider.GetRequiredService<SitePairChannelProvider>(),
_communicationOptions,
_serviceProvider.GetRequiredService<ILoggerFactory>().CreateLogger<GrpcSiteTransport>());
_logger.LogInformation("central→site command transport: gRPC (SiteCommandService)");
var centralCommActor = _actorSystem!.ActorOf(
Props.Create(() => new CentralCommunicationActor(_serviceProvider, siteClientFactory)),
Props.Create(() => new CentralCommunicationActor(_serviceProvider, siteCommandTransport)),
"central-communication");
// Register CentralCommunicationActor with ClusterClientReceptionist so site ClusterClients can reach it
ClusterClientReceptionist.Get(_actorSystem).RegisterService(centralCommActor);
_logger.LogInformation("CentralCommunicationActor registered with ClusterClientReceptionist");
// Hand the same actor to the central-hosted gRPC control plane (T1A.2) and open its
// readiness gate — the gRPC face Asks this exact actor, so both transports resolve to
// one handler implementation. Mirrors SiteStreamGrpcServer.SetReady on the site side:
@@ -833,35 +837,25 @@ akka {{
_logger, role: siteRole);
var dmProxy = dm.Proxy;
// Select the site→central transport behind the coexistence flag (default Akka
// ClusterClient). When gRPC is chosen the site dials CentralControlService directly with
// a sticky-failover channel pair, presenting its own preshared key; the ClusterClient
// below is then not created at all.
ICentralTransport? centralTransport = null;
if (_communicationOptions.CentralTransport == CentralTransportMode.Grpc)
{
var loggerFactory = _serviceProvider.GetRequiredService<ILoggerFactory>();
var channelProvider = new CentralChannelProvider(
_communicationOptions.CentralGrpcEndpoints,
new StaticSitePskProvider(_communicationOptions.GrpcPsk),
_nodeOptions.SiteId!,
_communicationOptions,
loggerFactory.CreateLogger<CentralChannelProvider>());
_trackedDisposables.Add(channelProvider);
centralTransport = new GrpcCentralTransport(
channelProvider,
_communicationOptions,
loggerFactory.CreateLogger<GrpcCentralTransport>());
_logger.LogInformation(
"Site→central transport: gRPC to {Count} central endpoint(s) for site {SiteId}",
_communicationOptions.CentralGrpcEndpoints.Count, _nodeOptions.SiteId);
}
else
{
_logger.LogInformation(
"Site→central transport: Akka ClusterClient (default) for site {SiteId}",
_nodeOptions.SiteId);
}
// Site→central transport: a gRPC dial of CentralControlService with a sticky
// central-a→central-b channel pair, presenting this site's preshared key. This is the ONLY
// transport since the ClusterClient→gRPC migration's Phase 4 removed the Akka path;
// StartupValidator guarantees a Site node lists at least one CentralGrpcEndpoint.
var loggerFactory = _serviceProvider.GetRequiredService<ILoggerFactory>();
var channelProvider = new CentralChannelProvider(
_communicationOptions.CentralGrpcEndpoints,
new StaticSitePskProvider(_communicationOptions.GrpcPsk),
_nodeOptions.SiteId!,
_communicationOptions,
loggerFactory.CreateLogger<CentralChannelProvider>());
_trackedDisposables.Add(channelProvider);
ICentralTransport centralTransport = new GrpcCentralTransport(
channelProvider,
_communicationOptions,
loggerFactory.CreateLogger<GrpcCentralTransport>());
_logger.LogInformation(
"Site→central transport: gRPC to {Count} central endpoint(s) for site {SiteId}",
_communicationOptions.CentralGrpcEndpoints.Count, _nodeOptions.SiteId);
// The ONE routing table for central→site commands, shared by the Akka
// SiteCommunicationActor (below) and the gRPC SiteCommandGrpcService (SetReady at the end
@@ -997,36 +991,10 @@ akka {{
siteCommActor.Tell(new RegisterLocalHandler(LocalHandlerType.ParkedMessages, parkedMessageHandler));
}
// Register SiteCommunicationActor with ClusterClientReceptionist so central ClusterClients can reach it
ClusterClientReceptionist.Get(_actorSystem).RegisterService(siteCommActor);
_logger.LogInformation(
"Site actors registered. DeploymentManager singleton scoped to role={SiteRole}, SiteCommunicationActor created.",
siteRole);
// Create ClusterClient to central if contact points are configured — but only on the Akka
// transport. On the gRPC transport the SiteCommunicationActor already holds a
// GrpcCentralTransport and never receives RegisterCentralClient, so a ClusterClient here
// would be dead weight (and keep an unwanted cross-cluster Akka association alive).
if (_communicationOptions.CentralTransport == CentralTransportMode.Akka
&& _communicationOptions.CentralContactPoints.Count > 0)
{
var contacts = _communicationOptions.CentralContactPoints
.Select(cp => ActorPath.Parse($"{cp}/system/receptionist"))
.ToImmutableHashSet();
var clientSettings = ClusterClientSettings.Create(_actorSystem)
.WithInitialContacts(contacts);
var centralClient = _actorSystem.ActorOf(
ClusterClient.Props(clientSettings), "central-cluster-client");
var siteCommSelection = _actorSystem.ActorSelection("/user/site-communication");
siteCommSelection.Tell(new RegisterCentralClient(centralClient));
_logger.LogInformation(
"Created ClusterClient to central with {Count} contact point(s) for site {SiteId}",
contacts.Count, _nodeOptions.SiteId);
}
// Per-node startup reconciliation. Created on EVERY site node (NOT a
// singleton) so a standby that was DOWN during a deploy self-heals on its next
// restart: it reports its local deployed inventory to central via the
src/ZB.MOM.WW.ScadaBridge.Host/StartupValidator.cs
+16
View File
@@ -143,6 +143,22 @@ public static class StartupValidator
+ "production as ${secret:SB-GRPC-PSK-<siteId>}) and under the secret "
+ "name SB-GRPC-PSK-<siteId> in central's secret store");
// gRPC (CentralControlService) is the only site→central transport after the
// ClusterClient→gRPC migration's Phase 4 — the Akka ClusterClient path and its
// CentralContactPoints option are gone. A site with no central gRPC endpoint has
// nothing to dial: heartbeats, health reports, notification forwards and audit
// ingest all silently fail. The shared CommunicationOptionsValidator only rejects
// BLANK entries (it is role-agnostic, and central nodes legitimately leave the list
// empty), so the "a Site must have at least one" rule lives here, where the role is
// known. The predicate reads index :0 directly, so its non-empty presence proves the
// list has a usable first endpoint.
p.Require("ScadaBridge:Communication:CentralGrpcEndpoints:0",
value => !string.IsNullOrWhiteSpace(value),
"is required for Site nodes: gRPC (CentralControlService) is the only "
+ "site→central transport, so each site must list at least one central gRPC "
+ "endpoint under ScadaBridge:Communication:CentralGrpcEndpoints "
+ "(e.g. http://scadabridge-central-a:8083). Central nodes leave it empty.");
// ScadaBridge:Database:SiteDbPath was required here until LocalDb
// Phase 2. The site's tables now live in the consolidated LocalDb
// database (LocalDb:Path, which SiteServiceRegistration requires),
src/ZB.MOM.WW.ScadaBridge.Host/appsettings.Site.json
+3 -3
View File
@@ -44,9 +44,9 @@
"Communication": {
"_grpcPsk": "REQUIRED on Site nodes (StartupValidator fails the boot without it). The preshared key the gRPC control plane authenticates with: ControlPlaneAuthInterceptor is fail-closed, so an unset key refuses every SiteStream call — live subscriptions, audit pulls, cached-telemetry ingest — while the node still reports healthy. Supply it as ${secret:SB-GRPC-PSK-<siteId>} so the plaintext never sits in this file, and seed the SAME value on central: either as the secret SB-GRPC-PSK-<siteId> in its store, or as ScadaBridge:Communication:SitePsks:<siteId>. BOTH nodes of the pair carry the same key. Distinct from LocalDb:Replication:ApiKey, which authenticates the pair partner, not central — never share the two.",
"GrpcPsk": "${secret:SB-GRPC-PSK-site-1}",
"_centralContactPoints": "Host-016: each entry MUST be a central node's remoting endpoint, NOT this site's own remoting port. The single dev-loopback default below points only at central-a (localhost:8081). In a multi-central deployment add the second central node here (e.g. 'akka.tcp://scadabridge@central-b-host:8081') so ClusterClient can fail over when central-a is down. The previous template listed localhost:8082 as the second contact — that is THIS site's own RemotingPort and is a permanent failure in the initial-contact rotation.",
"CentralContactPoints": [
"akka.tcp://scadabridge@localhost:8081"
"_centralGrpcEndpoints": "gRPC (CentralControlService) is the only site→central transport since the ClusterClient→gRPC migration's Phase 4. Each entry MUST be a central node's gRPC (h2c) endpoint on its CentralGrpcPort (default 8083) — NOT this site's own gRPC port, and NOT via Traefik (HTTP/1 only). The single dev-loopback default below points only at central-a (localhost:8083). In a multi-central deployment add the second central node here (e.g. 'http://central-b-host:8083') so the channel pair can fail over when central-a is down. StartupValidator requires a Site node to list at least one endpoint.",
"CentralGrpcEndpoints": [
"http://localhost:8083"
],
"DeploymentTimeout": "00:02:00",
"LifecycleTimeout": "00:00:30",