Files
ScadaBridge/src/ZB.MOM.WW.ScadaBridge.Communication/CommunicationOptions.cs
T
Joseph Doherty 7fd5cb2b56 feat(comm): Phase 4 — delete Akka ClusterClient site↔central transport, gRPC-only
ClusterClient→gRPC migration Phase 4 (docs/plans/2026-07-22-clusterclient-to-grpc-plan.md).
Phases 2/3 proved both directions on gRPC; this removes the Akka transport underneath.

Deleted:
- AkkaCentralTransport, AkkaSiteTransport (+ their dedicated tests)
- ISiteClientFactory + DefaultSiteClientFactory; CentralCommunicationActor legacy
  ctor + SelectTransport (Host now builds GrpcSiteTransport and injects it)
- ClusterClient creation + both ClusterClientReceptionist.RegisterService calls in
  AkkaHostedService; the RegisterCentralClient message + receive block
- CommunicationOptions.CentralContactPoints; the CentralTransport/SiteTransport
  coexistence flags; the CentralTransportMode/SiteTransportKind enums

gRPC is now the only site↔central transport (site→central CentralControlService via
GrpcCentralTransport; central→site SiteCommandService via GrpcSiteTransport), both
built unconditionally by the Host. NoOpCentralTransport is the fail-loud null-default
so TestKit command-dispatch suites still construct the site actor without a wired
transport; production always injects GrpcCentralTransport.

Config: CentralGrpcEndpoints is now unconditional — CommunicationOptionsValidator
rejects blank entries (role-agnostic), and StartupValidator requires a Site node to
list >=1 endpoint (fail-fast, mirrors GrpcPsk). Rig configs moved
CentralContactPoints -> CentralGrpcEndpoints (docker x6, docker-env2 x2, Host default,
deploy/wonder-app-vd03). Kept Akka.Cluster.Tools (ClusterSingleton still used).

Tests: build 0/0; Communication.Tests 640, Host.Tests 421 green. Removed the
ClusterClient.Send per-site-routing tests (covered by the transport suites), swapped
the ISiteClientFactory-based ctors to a substitute ISiteCommandTransport, converted
the audit-push integration relay to an in-process bridge transport.

Docs: Component-Communication/Host/StoreAndForward, components/Communication,
topology-guide, grpc_streams (SUPERSEDED note), the frame-size known-issue (retired
amendment), and CLAUDE.md transport decisions.

Not included: the dead IntegrationCallRequest path (#32) is a separate user-owned
behavioral decision — SiteEnvelope routing is transport-agnostic so it still compiles.
2026-07-23 12:54:32 -04:00

189 lines
10 KiB
C#

namespace ZB.MOM.WW.ScadaBridge.Communication;
/// <summary>
/// Configuration options for central-site communication, including per-pattern
/// timeouts and transport heartbeat settings.
/// </summary>
public class CommunicationOptions
{
/// <summary>
/// Central control-plane gRPC endpoints (preferred first), e.g.
/// <c>["http://scadabridge-central-a:8083", "http://scadabridge-central-b:8083"]</c>. Dialled by
/// <see cref="Grpc.CentralChannelProvider"/> with sticky failover/failback. Required on every site
/// node — gRPC (<c>CentralControlService</c>) is the only site→central transport.
/// </summary>
/// <remarks>
/// Sites reach central by container/host name, NOT via Traefik (which is HTTP/1 only; gRPC is
/// h2c on the central node's dedicated <c>CentralGrpcPort</c>).
/// </remarks>
public List<string> CentralGrpcEndpoints { get; set; } = new();
/// <summary>Timeout for deployment commands (typically longest due to apply logic).</summary>
public TimeSpan DeploymentTimeout { get; set; } = TimeSpan.FromMinutes(2);
/// <summary>Timeout for lifecycle commands (disable, enable, delete).</summary>
public TimeSpan LifecycleTimeout { get; set; } = TimeSpan.FromSeconds(30);
/// <summary>Timeout for artifact deployment commands.</summary>
public TimeSpan ArtifactDeploymentTimeout { get; set; } = TimeSpan.FromMinutes(1);
/// <summary>Timeout for remote query requests (event logs, parked messages).</summary>
public TimeSpan QueryTimeout { get; set; } = TimeSpan.FromSeconds(30);
/// <summary>Timeout for integration call routing.</summary>
public TimeSpan IntegrationTimeout { get; set; } = TimeSpan.FromSeconds(30);
/// <summary>Timeout for debug view subscribe/unsubscribe handshake.</summary>
public TimeSpan DebugViewTimeout { get; set; } = TimeSpan.FromSeconds(10);
/// <summary>Timeout for health report acknowledgement (fire-and-forget, but bounded).</summary>
public TimeSpan HealthReportTimeout { get; set; } = TimeSpan.FromSeconds(10);
/// <summary>
/// Notification Outbox: timeout for forwarding a buffered notification to central
/// and awaiting its <c>NotificationSubmitAck</c>. A timeout is treated as a
/// transient failure — the Store-and-Forward engine keeps the message buffered
/// and retries the forward at the fixed retry interval.
/// </summary>
public TimeSpan NotificationForwardTimeout { get; set; } = TimeSpan.FromSeconds(30);
/// <summary>
/// Preshared key authenticating this node's gRPC control plane — the site↔central
/// boundary. On a site node this is the key its inbound gate
/// (<c>ControlPlaneAuthInterceptor</c>) expects on every <c>SiteStreamService</c> call, and
/// which central must present; central resolves the matching value per site from its own
/// secret store under the name <c>SB-GRPC-PSK-{siteId}</c>.
/// </summary>
/// <remarks>
/// <para>
/// In production this is supplied as <c>${secret:SB-GRPC-PSK-&lt;siteId&gt;}</c> and expanded
/// out of the secrets store before the host is built, so the plaintext never sits in
/// appsettings. Development rigs set a literal, mirroring the LocalDb replication key.
/// </para>
/// <para>
/// <b>Empty means closed, not open.</b> With no key set the interceptor rejects every gated
/// call. This is not optional configuration: a node that ships without a key serves no
/// streams and no audit pulls.
/// </para>
/// <para>
/// Distinct from <c>LocalDb:Replication:ApiKey</c>, which authenticates the pair partner for
/// database replication over the same listener. The two are never shared.
/// </para>
/// </remarks>
public string GrpcPsk { get; set; } = "";
/// <summary>
/// Central-side per-site gRPC preshared keys, keyed by site identifier — the mirror image
/// of <see cref="GrpcPsk"/>, which is the single key a site node expects on its own inbound
/// gate. An entry here takes precedence over the secret store for that site.
/// </summary>
/// <remarks>
/// <para>
/// <b>Why both a config map and a secret store.</b> The store is the primary source and the
/// only one that works for the real case: sites are added at runtime from the Central UI, so
/// their keys cannot be enumerated in configuration at boot, and <c>SitePskProvider</c>
/// resolves <c>SB-GRPC-PSK-{siteId}</c> on demand. This map covers the cases the store
/// cannot or should not: a development rig that runs with no master key and injects every
/// credential as an environment override, and an operator pinning one site's key without
/// touching the store. Values may themselves be <c>${secret:…}</c> references, since a map
/// declared in configuration IS enumerable at boot.
/// </para>
/// <para>
/// Absence is not a fallback to "unauthenticated" in either source — a site with no key in
/// the map and none in the store cannot be dialed at all.
/// </para>
/// </remarks>
public Dictionary<string, string> SitePsks { get; set; } = new();
/// <summary>gRPC keepalive ping interval for streaming connections.</summary>
public TimeSpan GrpcKeepAlivePingDelay { get; set; } = TimeSpan.FromSeconds(15);
/// <summary>gRPC keepalive ping timeout — stream is considered dead if no response within this period.</summary>
public TimeSpan GrpcKeepAlivePingTimeout { get; set; } = TimeSpan.FromSeconds(10);
/// <summary>Maximum lifetime for a single gRPC stream before the server forces re-establishment.</summary>
public TimeSpan GrpcMaxStreamLifetime { get; set; } = TimeSpan.FromHours(4);
/// <summary>Maximum number of concurrent gRPC streaming subscriptions per site node.</summary>
public int GrpcMaxConcurrentStreams { get; set; } = 100;
/// <summary>Akka.Remote transport heartbeat interval.</summary>
public TimeSpan TransportHeartbeatInterval { get; set; } = TimeSpan.FromSeconds(5);
/// <summary>
/// Application-level site→central heartbeat cadence. Distinct from
/// <see cref="TransportHeartbeatInterval"/> (the Akka.Remote failure-detector
/// setting) — tuning the transport FD must not silently retune the health
/// heartbeat. Default equals the old effective value (5s) — no behavior change.
/// </summary>
public TimeSpan ApplicationHeartbeatInterval { get; set; } = TimeSpan.FromSeconds(5);
/// <summary>Akka.Remote transport failure detection threshold.</summary>
public TimeSpan TransportFailureThreshold { get; set; } = TimeSpan.FromSeconds(15);
/// <summary>
/// Base URL (Traefik/LB) the SITE uses to fetch deploy configs from central,
/// e.g. "https://central.example:9000". Carried in RefreshDeploymentCommand so
/// sites need no new standing config. Empty disables notify-and-fetch fallback.
/// </summary>
public string CentralFetchBaseUrl { get; set; } = "";
/// <summary>
/// How long a staged PendingDeployment (and its fetch token) stays valid. Must
/// comfortably cover both site nodes' fetches within one deploy window.
/// </summary>
public TimeSpan PendingDeploymentTtl { get; set; } = TimeSpan.FromMinutes(5);
/// <summary>
/// How often the central <c>PendingDeploymentPurgeActor</c> singleton reclaims
/// expired (TTL-elapsed) PendingDeployment staging rows. Best-effort hygiene only:
/// supersession bounds pending rows to ≤1 per instance and the config-fetch endpoint
/// already enforces the TTL, so this purge merely sweeps rows left behind by instances
/// that are deployed once and never re-deployed. Default 1 hour ≫ <see cref="PendingDeploymentTtl"/>.
/// </summary>
public TimeSpan PendingDeploymentPurgeInterval { get; set; } = TimeSpan.FromHours(1);
// ── Aggregated live alarm cache (plan #10, Task 4) ───────────────────────────
// Introduced by Task 4 with sane defaults; Task 6 formalizes eager validation
// (positive interval/linger, positive concurrency/subscriber cap) and telemetry.
/// <summary>
/// How long the shared per-site live alarm aggregator keeps its gRPC stream +
/// in-memory cache warm after its LAST Alarm Summary viewer leaves, before it is
/// torn down. A short linger avoids re-seed thrash when an operator navigates away
/// and straight back. Default 30s.
/// </summary>
public TimeSpan LiveAlarmCacheLinger { get; set; } = TimeSpan.FromSeconds(30);
/// <summary>
/// Cadence of the per-site aggregator's periodic reconcile snapshot fan-out. The
/// backstop that corrects instance-set drift (enable/disable/deploy/delete) and any
/// alarm delta missed by the live stream, since the aggregated stream never carries
/// an explicit "removed" event. Default 60s.
/// </summary>
public TimeSpan LiveAlarmCacheReconcileInterval { get; set; } = TimeSpan.FromSeconds(60);
/// <summary>
/// Max concurrent per-instance snapshot fetches during a live-cache seed/reconcile
/// fan-out — the aggregated analogue of the Alarm Summary poll's
/// <c>MaxConcurrentFetches</c>. Default 8.
/// </summary>
public int LiveAlarmCacheSeedConcurrency { get; set; } = 8;
/// <summary>
/// Upper bound on concurrent Alarm Summary viewers sharing one site's aggregator.
/// A defensive cap only — one gRPC stream per site is shared regardless of viewer
/// count; this just bounds the subscriber list. Default 200.
/// </summary>
public int LiveAlarmCacheMaxSubscribersPerSite { get; set; } = 200;
/// <summary>
/// Publish-coalescing window for live alarm deltas: an applied delta marks the cache
/// dirty and one publish (fresh snapshot + per-viewer onChanged fan-out) fires after
/// this window, batching an alarm storm into ~4 publishes/second instead of one per
/// transition (review 02 round 2, N6). Zero = publish per delta (legacy). Seed and
/// reconcile publishes are always immediate. Default 250 ms.
/// </summary>
public TimeSpan LiveAlarmCachePublishCoalesce { get; set; } = TimeSpan.FromMilliseconds(250);
}