7fd5cb2b56
ClusterClient→gRPC migration Phase 4 (docs/plans/2026-07-22-clusterclient-to-grpc-plan.md). Phases 2/3 proved both directions on gRPC; this removes the Akka transport underneath. Deleted: - AkkaCentralTransport, AkkaSiteTransport (+ their dedicated tests) - ISiteClientFactory + DefaultSiteClientFactory; CentralCommunicationActor legacy ctor + SelectTransport (Host now builds GrpcSiteTransport and injects it) - ClusterClient creation + both ClusterClientReceptionist.RegisterService calls in AkkaHostedService; the RegisterCentralClient message + receive block - CommunicationOptions.CentralContactPoints; the CentralTransport/SiteTransport coexistence flags; the CentralTransportMode/SiteTransportKind enums gRPC is now the only site↔central transport (site→central CentralControlService via GrpcCentralTransport; central→site SiteCommandService via GrpcSiteTransport), both built unconditionally by the Host. NoOpCentralTransport is the fail-loud null-default so TestKit command-dispatch suites still construct the site actor without a wired transport; production always injects GrpcCentralTransport. Config: CentralGrpcEndpoints is now unconditional — CommunicationOptionsValidator rejects blank entries (role-agnostic), and StartupValidator requires a Site node to list >=1 endpoint (fail-fast, mirrors GrpcPsk). Rig configs moved CentralContactPoints -> CentralGrpcEndpoints (docker x6, docker-env2 x2, Host default, deploy/wonder-app-vd03). Kept Akka.Cluster.Tools (ClusterSingleton still used). Tests: build 0/0; Communication.Tests 640, Host.Tests 421 green. Removed the ClusterClient.Send per-site-routing tests (covered by the transport suites), swapped the ISiteClientFactory-based ctors to a substitute ISiteCommandTransport, converted the audit-push integration relay to an in-process bridge transport. Docs: Component-Communication/Host/StoreAndForward, components/Communication, topology-guide, grpc_streams (SUPERSEDED note), the frame-size known-issue (retired amendment), and CLAUDE.md transport decisions. Not included: the dead IntegrationCallRequest path (#32) is a separate user-owned behavioral decision — SiteEnvelope routing is transport-agnostic so it still compiles.
189 lines
10 KiB
C#
189 lines
10 KiB
C#
namespace ZB.MOM.WW.ScadaBridge.Communication;
|
|
|
|
/// <summary>
|
|
/// Configuration options for central-site communication, including per-pattern
|
|
/// timeouts and transport heartbeat settings.
|
|
/// </summary>
|
|
public class CommunicationOptions
|
|
{
|
|
/// <summary>
|
|
/// Central control-plane gRPC endpoints (preferred first), e.g.
|
|
/// <c>["http://scadabridge-central-a:8083", "http://scadabridge-central-b:8083"]</c>. Dialled by
|
|
/// <see cref="Grpc.CentralChannelProvider"/> with sticky failover/failback. Required on every site
|
|
/// node — gRPC (<c>CentralControlService</c>) is the only site→central transport.
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// Sites reach central by container/host name, NOT via Traefik (which is HTTP/1 only; gRPC is
|
|
/// h2c on the central node's dedicated <c>CentralGrpcPort</c>).
|
|
/// </remarks>
|
|
public List<string> CentralGrpcEndpoints { get; set; } = new();
|
|
|
|
/// <summary>Timeout for deployment commands (typically longest due to apply logic).</summary>
|
|
public TimeSpan DeploymentTimeout { get; set; } = TimeSpan.FromMinutes(2);
|
|
|
|
/// <summary>Timeout for lifecycle commands (disable, enable, delete).</summary>
|
|
public TimeSpan LifecycleTimeout { get; set; } = TimeSpan.FromSeconds(30);
|
|
|
|
/// <summary>Timeout for artifact deployment commands.</summary>
|
|
public TimeSpan ArtifactDeploymentTimeout { get; set; } = TimeSpan.FromMinutes(1);
|
|
|
|
/// <summary>Timeout for remote query requests (event logs, parked messages).</summary>
|
|
public TimeSpan QueryTimeout { get; set; } = TimeSpan.FromSeconds(30);
|
|
|
|
/// <summary>Timeout for integration call routing.</summary>
|
|
public TimeSpan IntegrationTimeout { get; set; } = TimeSpan.FromSeconds(30);
|
|
|
|
/// <summary>Timeout for debug view subscribe/unsubscribe handshake.</summary>
|
|
public TimeSpan DebugViewTimeout { get; set; } = TimeSpan.FromSeconds(10);
|
|
|
|
/// <summary>Timeout for health report acknowledgement (fire-and-forget, but bounded).</summary>
|
|
public TimeSpan HealthReportTimeout { get; set; } = TimeSpan.FromSeconds(10);
|
|
|
|
/// <summary>
|
|
/// Notification Outbox: timeout for forwarding a buffered notification to central
|
|
/// and awaiting its <c>NotificationSubmitAck</c>. A timeout is treated as a
|
|
/// transient failure — the Store-and-Forward engine keeps the message buffered
|
|
/// and retries the forward at the fixed retry interval.
|
|
/// </summary>
|
|
public TimeSpan NotificationForwardTimeout { get; set; } = TimeSpan.FromSeconds(30);
|
|
|
|
/// <summary>
|
|
/// Preshared key authenticating this node's gRPC control plane — the site↔central
|
|
/// boundary. On a site node this is the key its inbound gate
|
|
/// (<c>ControlPlaneAuthInterceptor</c>) expects on every <c>SiteStreamService</c> call, and
|
|
/// which central must present; central resolves the matching value per site from its own
|
|
/// secret store under the name <c>SB-GRPC-PSK-{siteId}</c>.
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// <para>
|
|
/// In production this is supplied as <c>${secret:SB-GRPC-PSK-<siteId>}</c> and expanded
|
|
/// out of the secrets store before the host is built, so the plaintext never sits in
|
|
/// appsettings. Development rigs set a literal, mirroring the LocalDb replication key.
|
|
/// </para>
|
|
/// <para>
|
|
/// <b>Empty means closed, not open.</b> With no key set the interceptor rejects every gated
|
|
/// call. This is not optional configuration: a node that ships without a key serves no
|
|
/// streams and no audit pulls.
|
|
/// </para>
|
|
/// <para>
|
|
/// Distinct from <c>LocalDb:Replication:ApiKey</c>, which authenticates the pair partner for
|
|
/// database replication over the same listener. The two are never shared.
|
|
/// </para>
|
|
/// </remarks>
|
|
public string GrpcPsk { get; set; } = "";
|
|
|
|
/// <summary>
|
|
/// Central-side per-site gRPC preshared keys, keyed by site identifier — the mirror image
|
|
/// of <see cref="GrpcPsk"/>, which is the single key a site node expects on its own inbound
|
|
/// gate. An entry here takes precedence over the secret store for that site.
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// <para>
|
|
/// <b>Why both a config map and a secret store.</b> The store is the primary source and the
|
|
/// only one that works for the real case: sites are added at runtime from the Central UI, so
|
|
/// their keys cannot be enumerated in configuration at boot, and <c>SitePskProvider</c>
|
|
/// resolves <c>SB-GRPC-PSK-{siteId}</c> on demand. This map covers the cases the store
|
|
/// cannot or should not: a development rig that runs with no master key and injects every
|
|
/// credential as an environment override, and an operator pinning one site's key without
|
|
/// touching the store. Values may themselves be <c>${secret:…}</c> references, since a map
|
|
/// declared in configuration IS enumerable at boot.
|
|
/// </para>
|
|
/// <para>
|
|
/// Absence is not a fallback to "unauthenticated" in either source — a site with no key in
|
|
/// the map and none in the store cannot be dialed at all.
|
|
/// </para>
|
|
/// </remarks>
|
|
public Dictionary<string, string> SitePsks { get; set; } = new();
|
|
|
|
/// <summary>gRPC keepalive ping interval for streaming connections.</summary>
|
|
public TimeSpan GrpcKeepAlivePingDelay { get; set; } = TimeSpan.FromSeconds(15);
|
|
|
|
/// <summary>gRPC keepalive ping timeout — stream is considered dead if no response within this period.</summary>
|
|
public TimeSpan GrpcKeepAlivePingTimeout { get; set; } = TimeSpan.FromSeconds(10);
|
|
|
|
/// <summary>Maximum lifetime for a single gRPC stream before the server forces re-establishment.</summary>
|
|
public TimeSpan GrpcMaxStreamLifetime { get; set; } = TimeSpan.FromHours(4);
|
|
|
|
/// <summary>Maximum number of concurrent gRPC streaming subscriptions per site node.</summary>
|
|
public int GrpcMaxConcurrentStreams { get; set; } = 100;
|
|
|
|
/// <summary>Akka.Remote transport heartbeat interval.</summary>
|
|
public TimeSpan TransportHeartbeatInterval { get; set; } = TimeSpan.FromSeconds(5);
|
|
|
|
/// <summary>
|
|
/// Application-level site→central heartbeat cadence. Distinct from
|
|
/// <see cref="TransportHeartbeatInterval"/> (the Akka.Remote failure-detector
|
|
/// setting) — tuning the transport FD must not silently retune the health
|
|
/// heartbeat. Default equals the old effective value (5s) — no behavior change.
|
|
/// </summary>
|
|
public TimeSpan ApplicationHeartbeatInterval { get; set; } = TimeSpan.FromSeconds(5);
|
|
|
|
/// <summary>Akka.Remote transport failure detection threshold.</summary>
|
|
public TimeSpan TransportFailureThreshold { get; set; } = TimeSpan.FromSeconds(15);
|
|
|
|
/// <summary>
|
|
/// Base URL (Traefik/LB) the SITE uses to fetch deploy configs from central,
|
|
/// e.g. "https://central.example:9000". Carried in RefreshDeploymentCommand so
|
|
/// sites need no new standing config. Empty disables notify-and-fetch fallback.
|
|
/// </summary>
|
|
public string CentralFetchBaseUrl { get; set; } = "";
|
|
|
|
/// <summary>
|
|
/// How long a staged PendingDeployment (and its fetch token) stays valid. Must
|
|
/// comfortably cover both site nodes' fetches within one deploy window.
|
|
/// </summary>
|
|
public TimeSpan PendingDeploymentTtl { get; set; } = TimeSpan.FromMinutes(5);
|
|
|
|
/// <summary>
|
|
/// How often the central <c>PendingDeploymentPurgeActor</c> singleton reclaims
|
|
/// expired (TTL-elapsed) PendingDeployment staging rows. Best-effort hygiene only:
|
|
/// supersession bounds pending rows to ≤1 per instance and the config-fetch endpoint
|
|
/// already enforces the TTL, so this purge merely sweeps rows left behind by instances
|
|
/// that are deployed once and never re-deployed. Default 1 hour ≫ <see cref="PendingDeploymentTtl"/>.
|
|
/// </summary>
|
|
public TimeSpan PendingDeploymentPurgeInterval { get; set; } = TimeSpan.FromHours(1);
|
|
|
|
// ── Aggregated live alarm cache (plan #10, Task 4) ───────────────────────────
|
|
// Introduced by Task 4 with sane defaults; Task 6 formalizes eager validation
|
|
// (positive interval/linger, positive concurrency/subscriber cap) and telemetry.
|
|
|
|
/// <summary>
|
|
/// How long the shared per-site live alarm aggregator keeps its gRPC stream +
|
|
/// in-memory cache warm after its LAST Alarm Summary viewer leaves, before it is
|
|
/// torn down. A short linger avoids re-seed thrash when an operator navigates away
|
|
/// and straight back. Default 30s.
|
|
/// </summary>
|
|
public TimeSpan LiveAlarmCacheLinger { get; set; } = TimeSpan.FromSeconds(30);
|
|
|
|
/// <summary>
|
|
/// Cadence of the per-site aggregator's periodic reconcile snapshot fan-out. The
|
|
/// backstop that corrects instance-set drift (enable/disable/deploy/delete) and any
|
|
/// alarm delta missed by the live stream, since the aggregated stream never carries
|
|
/// an explicit "removed" event. Default 60s.
|
|
/// </summary>
|
|
public TimeSpan LiveAlarmCacheReconcileInterval { get; set; } = TimeSpan.FromSeconds(60);
|
|
|
|
/// <summary>
|
|
/// Max concurrent per-instance snapshot fetches during a live-cache seed/reconcile
|
|
/// fan-out — the aggregated analogue of the Alarm Summary poll's
|
|
/// <c>MaxConcurrentFetches</c>. Default 8.
|
|
/// </summary>
|
|
public int LiveAlarmCacheSeedConcurrency { get; set; } = 8;
|
|
|
|
/// <summary>
|
|
/// Upper bound on concurrent Alarm Summary viewers sharing one site's aggregator.
|
|
/// A defensive cap only — one gRPC stream per site is shared regardless of viewer
|
|
/// count; this just bounds the subscriber list. Default 200.
|
|
/// </summary>
|
|
public int LiveAlarmCacheMaxSubscribersPerSite { get; set; } = 200;
|
|
|
|
/// <summary>
|
|
/// Publish-coalescing window for live alarm deltas: an applied delta marks the cache
|
|
/// dirty and one publish (fresh snapshot + per-viewer onChanged fan-out) fires after
|
|
/// this window, batching an alarm storm into ~4 publishes/second instead of one per
|
|
/// transition (review 02 round 2, N6). Zero = publish per delta (legacy). Seed and
|
|
/// reconcile publishes are always immediate. Default 250 ms.
|
|
/// </summary>
|
|
public TimeSpan LiveAlarmCachePublishCoalesce { get; set; } = TimeSpan.FromMilliseconds(250);
|
|
}
|