feat(grpc): PSK-authenticate the site gRPC control plane; drop the vestigial management receptionist registration
Phase 0 of the ClusterClient→gRPC migration
(docs/plans/2026-07-22-clusterclient-to-grpc-plan.md). Standalone hardening: it
closes a gap that exists today and is a precondition for moving command/control
onto gRPC in later phases.
T0.1 — delete the ManagementActor ClusterClientReceptionist registration.
It was built for an out-of-cluster CLI that was never written: the shipped CLI
speaks HTTP Basic to /management, which asks the actor in-process through
ManagementActorHolder. Nothing in the repo ever sent to /user/management. The
actor still runs there; only the cross-boundary advertisement is gone. Six
documents claimed the CLI used ClusterClient — including the CLI's own README
"Architecture Notes" — and are corrected here rather than left to rot.
T0.2 — record, do not port, the dead integration-routing path.
IntegrationCallRequest is unwired at BOTH ends: RouteIntegrationCallAsync has
zero callers anywhere, and RegisterLocalHandler(Integration, …) appears only in
a test, so production always answers "Integration handler not available". It is
excluded from the gRPC contract (28 of 29 commands migrate) rather than
enshrined on an additive-only wire format, and deleting it during a
transport migration would mix a behavioural change into a change whose whole
value is that behaviour is identical. See
docs/known-issues/2026-07-22-integration-call-routing-is-dead-code.md.
T0.3 — preshared-key authentication on SiteStreamService.
The service shipped with no auth at all: plaintext h2c, no interceptor, so
anything that could reach a site node's :8083 could open a live data stream or
read audit rows back via PullAuditEvents/PullSiteCalls. ControlPlaneAuthInterceptor
now gates /sitestream.SiteStreamService/ — modeled on LocalDbSyncAuthInterceptor
(constant-time compare, fail-closed, PermissionDenied) but gating a SET of
service prefixes so phases 1A/1B add services rather than interceptors. LocalDb
sync keeps its own separate key: it authenticates the pair partner, not central,
and collapsing the two would make a site's central-facing key also admit writes
into its database.
Keys are per site (SB-GRPC-PSK-<siteId>), never fleet-wide, so a compromised
site yields only its own. Central attaches them through ControlPlaneCredentials,
which binds CallCredentials to the channel — covering unary and streaming
uniformly, and letting the key resolve asynchronously, which a client
interceptor could not do without blocking. All three central→site channel
creation sites go through it (SiteStreamGrpcClient and both audit pull invokers);
the pull invokers' channel caches are re-keyed by (site, endpoint) because
credentials are per-site and bound to the channel.
Two decisions beyond the plan:
* StartupValidator now requires GrpcPsk on Site nodes. The plan specified only
the runtime gate, but fail-closed with no boot check produces a node that
joins, answers heartbeats and reports healthy while refusing every stream,
audit pull and telemetry ingest — silent and total. Same reasoning as the
existing inbound API-key pepper rule.
* Added Communication:SitePsks as a central-side key map. The plan assumed
central would read the store, seeded via a dev KEK; the docker rig
deliberately boots with no master key, so store-only resolution would leave
it unable to dial its own sites. The store stays primary — it is the only
source that can serve a site added at runtime — with the map covering
key-less hosts and one-off pins. Neither source falling back to
"unauthenticated" is the invariant.
T0.4 — dev keys on both rigs and tests.
34 tests. The seven that matter most exercise a real in-process gRPC stack over
TestServer: the unit tests on either side of the wire would both stay green if
the halves disagreed, and gRPC refuses call credentials on a plaintext channel
by default — the UnsafeUseInsecureChannelCallCredentials opt-in is only provable
by making a real call. They confirm correct key passes on unary AND streaming,
wrong key and no-credentials both get PermissionDenied, and an unresolvable key
fails the call with nothing reaching the service.
OPERATIONAL: a site node upgraded to this build without a key will not boot.
That includes the gitignored deploy/wonder-app-vd03/ overlay.
This commit is contained in:
Joseph Doherty
@@ -450,16 +450,22 @@ akka {{
|
||||
siteAlarmLiveCache?.SetActorSystem(_actorSystem!);
|
||||
|
||||
|
||||
// Management Service — accessible via ClusterClient
|
||||
// Management Service — reached IN-PROCESS only, via ManagementActorHolder.
|
||||
//
|
||||
// This actor used to be registered with the ClusterClientReceptionist as well, for
|
||||
// an out-of-cluster CLI that was never built (REQ-HOST-6a). The shipped CLI speaks
|
||||
// HTTP Basic to /management, which asks this actor through the holder below
|
||||
// (ManagementEndpoints), so the registration had no sender anywhere in the repo —
|
||||
// it only advertised a management surface across the cluster-client boundary for
|
||||
// free. Removed 2026-07-22 (ClusterClient→gRPC migration, T0.1).
|
||||
var mgmtLogger = _serviceProvider.GetRequiredService<ILoggerFactory>()
|
||||
.CreateLogger<ZB.MOM.WW.ScadaBridge.ManagementService.ManagementActor>();
|
||||
var mgmtActor = _actorSystem!.ActorOf(
|
||||
Props.Create(() => new ZB.MOM.WW.ScadaBridge.ManagementService.ManagementActor(_serviceProvider, mgmtLogger)),
|
||||
"management");
|
||||
ClusterClientReceptionist.Get(_actorSystem).RegisterService(mgmtActor);
|
||||
var mgmtHolder = _serviceProvider.GetRequiredService<ZB.MOM.WW.ScadaBridge.ManagementService.ManagementActorHolder>();
|
||||
mgmtHolder.ActorRef = mgmtActor;
|
||||
_logger.LogInformation("ManagementActor registered with ClusterClientReceptionist");
|
||||
_logger.LogInformation("ManagementActor started at /user/management (in-process access via ManagementActorHolder)");
|
||||
|
||||
// Notification Outbox — cluster singleton so exactly one node owns ingest,
|
||||
// the dispatch sweep and the purge loop. Central actors run on the base
|
||||
|
||||
@@ -0,0 +1,211 @@
|
||||
using System.Security.Cryptography;
|
||||
using System.Text;
|
||||
using Grpc.Core;
|
||||
using Grpc.Core.Interceptors;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
using ZB.MOM.WW.ScadaBridge.Communication;
|
||||
using ZB.MOM.WW.ScadaBridge.Communication.Grpc;
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.Host;
|
||||
|
||||
/// <summary>
|
||||
/// Gates the site↔central gRPC control plane with a preshared key.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// <b>The gap this closes.</b> <c>SiteStreamService</c> shipped with no authentication at all —
|
||||
/// plaintext h2c, no interceptor. Anything that could reach a site node's gRPC port could open a
|
||||
/// live data stream or call <c>PullAuditEvents</c>/<c>PullSiteCalls</c> and read audit rows back.
|
||||
/// The only gated surface on that listener was LocalDb sync, and only for its own service. That
|
||||
/// gap exists independently of the ClusterClient→gRPC migration; it becomes indefensible once
|
||||
/// every command crosses this listener.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Modeled on <see cref="LocalDbSyncAuthInterceptor"/>,</b> deliberately: same four server
|
||||
/// handlers funnelling into one <c>Authorize</c>, same <c>authorization: Bearer</c> extraction,
|
||||
/// same <see cref="CryptographicOperations.FixedTimeEquals"/> comparison, same fail-closed
|
||||
/// posture, same <see cref="StatusCode.PermissionDenied"/> rejection. Two differences:
|
||||
/// </para>
|
||||
/// <list type="number">
|
||||
/// <item>It gates a <b>set</b> of service prefixes rather than one, so later phases can add the
|
||||
/// new command/control services without a second interceptor.</item>
|
||||
/// <item>Its expected key comes from <see cref="CommunicationOptions.GrpcPsk"/> — the site's own
|
||||
/// key, supplied in production as <c>${secret:SB-GRPC-PSK-<siteId>}</c> and expanded before
|
||||
/// the host is built.</item>
|
||||
/// </list>
|
||||
/// <para>
|
||||
/// <b>The two keys are separate on purpose.</b> LocalDb sync keeps its own
|
||||
/// <c>LocalDb:Replication:ApiKey</c>, which authenticates a different peer (the pair partner, not
|
||||
/// central) over a different trust relationship. Sharing one key would mean a site's central-facing
|
||||
/// key also admits writes into its database.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Fail-closed, and not optional.</b> With no <c>GrpcPsk</c> configured, every gated call is
|
||||
/// rejected — including the ones that work today. That is a deliberate break: LocalDb replication
|
||||
/// is an opt-in feature whose "off" state is "no peer", whereas streaming and audit pull are
|
||||
/// core paths, so "no key" must not silently mean "no authentication". Every environment must
|
||||
/// carry a key before upgrading to this build.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
public sealed class ControlPlaneAuthInterceptor : Interceptor
|
||||
{
|
||||
/// <summary>
|
||||
/// Service prefixes gated by default. Read from the generated <c>sitestream.proto</c>
|
||||
/// package/service names — <c>package sitestream; service SiteStreamService</c>.
|
||||
/// Later phases append their own services here.
|
||||
/// </summary>
|
||||
public static readonly IReadOnlyList<string> DefaultGatedPrefixes =
|
||||
new[] { "/sitestream.SiteStreamService/" };
|
||||
|
||||
private readonly IReadOnlyList<string> _gatedPrefixes;
|
||||
private readonly IOptions<CommunicationOptions> _options;
|
||||
private readonly ILogger<ControlPlaneAuthInterceptor> _logger;
|
||||
|
||||
/// <summary>Creates the interceptor gating <see cref="DefaultGatedPrefixes"/>.</summary>
|
||||
/// <param name="options">Communication options; <c>GrpcPsk</c> is the expected bearer token.</param>
|
||||
/// <param name="logger">Logger for denial diagnostics.</param>
|
||||
public ControlPlaneAuthInterceptor(
|
||||
IOptions<CommunicationOptions> options,
|
||||
ILogger<ControlPlaneAuthInterceptor> logger)
|
||||
: this(options, logger, DefaultGatedPrefixes)
|
||||
{
|
||||
}
|
||||
|
||||
/// <summary>Creates the interceptor gating an explicit set of service prefixes.</summary>
|
||||
/// <param name="options">Communication options; <c>GrpcPsk</c> is the expected bearer token.</param>
|
||||
/// <param name="logger">Logger for denial diagnostics.</param>
|
||||
/// <param name="gatedPrefixes">Method-path prefixes to gate, e.g. <c>/sitestream.SiteStreamService/</c>.</param>
|
||||
public ControlPlaneAuthInterceptor(
|
||||
IOptions<CommunicationOptions> options,
|
||||
ILogger<ControlPlaneAuthInterceptor> logger,
|
||||
IReadOnlyList<string> gatedPrefixes)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(options);
|
||||
ArgumentNullException.ThrowIfNull(logger);
|
||||
ArgumentNullException.ThrowIfNull(gatedPrefixes);
|
||||
|
||||
_options = options;
|
||||
_logger = logger;
|
||||
_gatedPrefixes = gatedPrefixes;
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public override Task<TResponse> UnaryServerHandler<TRequest, TResponse>(
|
||||
TRequest request,
|
||||
ServerCallContext context,
|
||||
UnaryServerMethod<TRequest, TResponse> continuation)
|
||||
{
|
||||
Authorize(context);
|
||||
return continuation(request, context);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public override Task DuplexStreamingServerHandler<TRequest, TResponse>(
|
||||
IAsyncStreamReader<TRequest> requestStream,
|
||||
IServerStreamWriter<TResponse> responseStream,
|
||||
ServerCallContext context,
|
||||
DuplexStreamingServerMethod<TRequest, TResponse> continuation)
|
||||
{
|
||||
Authorize(context);
|
||||
return continuation(requestStream, responseStream, context);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public override Task<TResponse> ClientStreamingServerHandler<TRequest, TResponse>(
|
||||
IAsyncStreamReader<TRequest> requestStream,
|
||||
ServerCallContext context,
|
||||
ClientStreamingServerMethod<TRequest, TResponse> continuation)
|
||||
{
|
||||
Authorize(context);
|
||||
return continuation(requestStream, context);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public override Task ServerStreamingServerHandler<TRequest, TResponse>(
|
||||
TRequest request,
|
||||
IServerStreamWriter<TResponse> responseStream,
|
||||
ServerCallContext context,
|
||||
ServerStreamingServerMethod<TRequest, TResponse> continuation)
|
||||
{
|
||||
Authorize(context);
|
||||
return continuation(request, responseStream, context);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Throws <see cref="RpcException"/> with <see cref="StatusCode.PermissionDenied"/> if this
|
||||
/// is a gated call that does not carry the configured bearer token. Calls to services
|
||||
/// outside <c>gatedPrefixes</c> — notably LocalDb sync, which has its own interceptor —
|
||||
/// return immediately.
|
||||
/// </summary>
|
||||
private void Authorize(ServerCallContext context)
|
||||
{
|
||||
if (!IsGated(context.Method))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
var expected = _options.Value.GrpcPsk;
|
||||
if (string.IsNullOrEmpty(expected))
|
||||
{
|
||||
_logger.LogWarning(
|
||||
"Rejected a control-plane call to {Method}: no ScadaBridge:Communication:GrpcPsk is "
|
||||
+ "configured, so the control plane is closed. Set the same key here (in production, "
|
||||
+ "as ${{secret:SB-GRPC-PSK-<siteId>}}) and in central's secret store.",
|
||||
context.Method);
|
||||
throw new RpcException(new Status(
|
||||
StatusCode.PermissionDenied,
|
||||
"Control plane is not accepting calls: no preshared key is configured on this node."));
|
||||
}
|
||||
|
||||
var presented = ExtractBearerToken(context.RequestHeaders);
|
||||
if (presented is null || !FixedTimeEquals(presented, expected))
|
||||
{
|
||||
_logger.LogWarning(
|
||||
"Rejected a control-plane call to {Method}: {Reason}.",
|
||||
context.Method,
|
||||
presented is null ? "no bearer token presented" : "bearer token did not match");
|
||||
throw new RpcException(new Status(
|
||||
StatusCode.PermissionDenied,
|
||||
"Control plane authentication failed."));
|
||||
}
|
||||
}
|
||||
|
||||
private bool IsGated(string method)
|
||||
{
|
||||
foreach (var prefix in _gatedPrefixes)
|
||||
{
|
||||
if (method.StartsWith(prefix, StringComparison.Ordinal))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private static string? ExtractBearerToken(Metadata headers)
|
||||
{
|
||||
// gRPC lowercases header keys on the wire; compare case-insensitively anyway so a
|
||||
// hand-built Metadata in a test behaves the same as a real request.
|
||||
foreach (var entry in headers)
|
||||
{
|
||||
if (!string.Equals(entry.Key, ControlPlaneCredentials.AuthorizationHeader,
|
||||
StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
var value = entry.Value;
|
||||
if (value is not null && value.StartsWith("Bearer ", StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
return value["Bearer ".Length..];
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
private static bool FixedTimeEquals(string presented, string expected)
|
||||
=> CryptographicOperations.FixedTimeEquals(
|
||||
Encoding.UTF8.GetBytes(presented), Encoding.UTF8.GetBytes(expected));
|
||||
}
|
||||
@@ -100,6 +100,14 @@ try
|
||||
// Shared components
|
||||
builder.Services.AddClusterInfrastructure();
|
||||
builder.Services.AddCommunication();
|
||||
|
||||
// Per-site gRPC preshared keys. Central-only: it is the side that dials sites, and
|
||||
// the only side whose key set is dynamic (sites come from the configuration
|
||||
// database, so there is no fixed list of ${secret:} references to expand at boot —
|
||||
// hence a runtime resolver rather than the pre-host SecretReferenceExpander a site
|
||||
// node uses for its single key). Registered before the clients that consume it.
|
||||
builder.Services.AddSingleton<
|
||||
ZB.MOM.WW.ScadaBridge.Communication.Grpc.ISitePskProvider, SitePskProvider>();
|
||||
builder.Services.AddHealthMonitoring();
|
||||
builder.Services.AddCentralHealthAggregation();
|
||||
builder.Services.AddExternalSystemGateway();
|
||||
@@ -519,12 +527,21 @@ try
|
||||
});
|
||||
});
|
||||
|
||||
// gRPC server registration
|
||||
// The interceptor gates ONLY /localdb_sync.v1.LocalDbSync/ — SiteStream calls on
|
||||
// this same pipeline pass through untouched. It is fail-closed: with no
|
||||
// LocalDb:Replication:ApiKey configured, no sync stream is accepted at all.
|
||||
// gRPC server registration. Two interceptors, two disjoint service prefixes, two
|
||||
// separate keys — neither one's absence weakens the other:
|
||||
//
|
||||
// LocalDbSyncAuthInterceptor gates /localdb_sync.v1.LocalDbSync/ (LocalDb:Replication:ApiKey)
|
||||
// ControlPlaneAuthInterceptor gates /sitestream.SiteStreamService/ (ScadaBridge:Communication:GrpcPsk)
|
||||
//
|
||||
// Both are fail-closed: an unset key closes that surface rather than opening it. For
|
||||
// LocalDb that means replication simply does not start; for the control plane it means
|
||||
// this node serves no streams and no audit pulls until a key is configured, which is
|
||||
// why every environment must carry one before running this build.
|
||||
builder.Services.AddGrpc(options =>
|
||||
options.Interceptors.Add<LocalDbSyncAuthInterceptor>());
|
||||
{
|
||||
options.Interceptors.Add<LocalDbSyncAuthInterceptor>();
|
||||
options.Interceptors.Add<ControlPlaneAuthInterceptor>();
|
||||
});
|
||||
builder.Services.AddSingleton<ZB.MOM.WW.ScadaBridge.Communication.Grpc.SiteStreamGrpcServer>();
|
||||
|
||||
// Existing site service registrations (this is also where LocalDb and its
|
||||
|
||||
@@ -0,0 +1,149 @@
|
||||
using System.Collections.Concurrent;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
using ZB.MOM.WW.ScadaBridge.Communication;
|
||||
using ZB.MOM.WW.ScadaBridge.Communication.Grpc;
|
||||
using ZB.MOM.WW.Secrets.Abstractions;
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.Host;
|
||||
|
||||
/// <summary>
|
||||
/// Central's <see cref="ISitePskProvider"/>: resolves each site's gRPC preshared key at
|
||||
/// channel-build time — from <c>ScadaBridge:Communication:SitePsks</c> if the site is listed
|
||||
/// there, otherwise from the secrets store under the name <c>SB-GRPC-PSK-{siteId}</c>.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// <b>Why resolved at runtime rather than expanded into config at boot.</b> Site nodes get their
|
||||
/// single key through the pre-host <c>SecretReferenceExpander</c> — a <c>${secret:…}</c> in
|
||||
/// appsettings, resolved once before the host is built. Central cannot do that: its set of sites
|
||||
/// comes from the configuration database and changes while the process runs, so there is no
|
||||
/// fixed list of references to expand at boot. This mirrors the pattern the MxGateway data
|
||||
/// connection already uses for its per-connection API keys.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Caching.</b> Successful resolves are cached indefinitely; a site's key changes only on
|
||||
/// rotation, and rotation restarts the pair. Failures are deliberately NOT cached, so a key
|
||||
/// seeded after the first (failed) dial is picked up on the next attempt instead of requiring a
|
||||
/// restart. <see cref="Invalidate"/> drops an entry when a site is removed or a key is rotated
|
||||
/// in place.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Fail-closed.</b> A key absent from BOTH sources throws, and the dial that needed it fails.
|
||||
/// It never degrades to an unauthenticated call. The message names the exact config key and the
|
||||
/// exact secret name, so the fix is one setting or one <c>secret</c> CLI seed.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
public sealed class SitePskProvider : ISitePskProvider
|
||||
{
|
||||
/// <summary>Prefix of the per-site secret name; the site identifier is appended verbatim.</summary>
|
||||
public const string SecretNamePrefix = "SB-GRPC-PSK-";
|
||||
|
||||
private readonly ConcurrentDictionary<string, string> _cache = new(StringComparer.Ordinal);
|
||||
private readonly ISecretResolver _resolver;
|
||||
private readonly IOptionsMonitor<CommunicationOptions> _options;
|
||||
private readonly ILogger<SitePskProvider> _logger;
|
||||
|
||||
/// <summary>Creates the provider over the host's secret resolver.</summary>
|
||||
/// <param name="resolver">Runtime secret resolver from the host container.</param>
|
||||
/// <param name="options">
|
||||
/// Communication options supplying the optional <c>SitePsks</c> map. Read through an
|
||||
/// <see cref="IOptionsMonitor{TOptions}"/> so a reloaded configuration is honoured on the
|
||||
/// next uncached resolve rather than pinned at construction.
|
||||
/// </param>
|
||||
/// <param name="logger">Logger for resolution diagnostics.</param>
|
||||
public SitePskProvider(
|
||||
ISecretResolver resolver,
|
||||
IOptionsMonitor<CommunicationOptions> options,
|
||||
ILogger<SitePskProvider> logger)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(resolver);
|
||||
ArgumentNullException.ThrowIfNull(options);
|
||||
ArgumentNullException.ThrowIfNull(logger);
|
||||
|
||||
_resolver = resolver;
|
||||
_options = options;
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
/// <summary>Builds the secret name for a site.</summary>
|
||||
/// <param name="siteId">Site identifier.</param>
|
||||
/// <returns>The secret name, e.g. <c>SB-GRPC-PSK-site-a</c>.</returns>
|
||||
public static string SecretNameFor(string siteId) => SecretNamePrefix + siteId;
|
||||
|
||||
/// <inheritdoc />
|
||||
public async ValueTask<string> GetAsync(string siteId, CancellationToken ct)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(siteId);
|
||||
|
||||
if (_cache.TryGetValue(siteId, out var cached))
|
||||
{
|
||||
return cached;
|
||||
}
|
||||
|
||||
// Configured map first: it is the explicit, operator-stated answer for this site, and
|
||||
// it is the only source available to a host running without a secrets master key.
|
||||
var configured = _options.CurrentValue.SitePsks;
|
||||
if (configured.TryGetValue(siteId, out var fromConfig) && !string.IsNullOrEmpty(fromConfig))
|
||||
{
|
||||
_cache[siteId] = fromConfig;
|
||||
return fromConfig;
|
||||
}
|
||||
|
||||
// Otherwise the store, which is the only source that can serve a site added at runtime.
|
||||
var secretName = SecretNameFor(siteId);
|
||||
var value = await ResolveFromStoreAsync(secretName, siteId, ct).ConfigureAwait(false);
|
||||
|
||||
if (string.IsNullOrEmpty(value))
|
||||
{
|
||||
_logger.LogError(
|
||||
"gRPC control-plane key for site {SiteId} could not be resolved: it is absent from "
|
||||
+ "ScadaBridge:Communication:SitePsks and secret '{SecretName}' is missing, empty or "
|
||||
+ "tombstoned. Calls to this site are refused until one of the two is set (the site "
|
||||
+ "node must carry the same value in ScadaBridge:Communication:GrpcPsk).",
|
||||
siteId, secretName);
|
||||
throw new InvalidOperationException(
|
||||
$"gRPC preshared key for site '{siteId}' could not be resolved from "
|
||||
+ $"ScadaBridge:Communication:SitePsks or secret '{secretName}'.");
|
||||
}
|
||||
|
||||
// Two concurrent first dials may both resolve; the read is idempotent and the values
|
||||
// identical, so the loser simply overwrites with the same string.
|
||||
_cache[siteId] = value;
|
||||
return value;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reads the key from the secrets store, treating a store fault as "not found" rather than
|
||||
/// letting it propagate. A host with no master key configured (the development rig) throws
|
||||
/// from the resolver; that must produce the same clear "key not configured" error as a
|
||||
/// missing secret, not an opaque cryptographic one.
|
||||
/// </summary>
|
||||
private async Task<string?> ResolveFromStoreAsync(string secretName, string siteId, CancellationToken ct)
|
||||
{
|
||||
try
|
||||
{
|
||||
return await _resolver.GetAsync(new SecretName(secretName), ct).ConfigureAwait(false);
|
||||
}
|
||||
catch (OperationCanceledException)
|
||||
{
|
||||
throw;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogDebug(ex,
|
||||
"Secret store lookup for '{SecretName}' (site {SiteId}) faulted; treating as not found.",
|
||||
secretName, siteId);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public void Invalidate(string siteId)
|
||||
{
|
||||
if (!string.IsNullOrWhiteSpace(siteId))
|
||||
{
|
||||
_cache.TryRemove(siteId, out _);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -126,6 +126,23 @@ public static class StartupValidator
|
||||
p.Require("ScadaBridge:Node:MetricsPort", _ => metricsPort != port, "must differ from RemotingPort");
|
||||
p.Require("ScadaBridge:Node:MetricsPort", _ => metricsPort != grpcPort, "must differ from GrpcPort");
|
||||
|
||||
// The gRPC control-plane preshared key. Same argument as the inbound
|
||||
// API-key pepper above: without it the node boots and looks healthy, but
|
||||
// ControlPlaneAuthInterceptor is fail-closed, so every SiteStream call —
|
||||
// live subscriptions, audit pulls, cached-telemetry ingest — is refused
|
||||
// with PermissionDenied. A silent, total loss of the site's central-facing
|
||||
// surface is far worse than a loud boot failure, so require it here.
|
||||
// Central holds the matching value per site in its secret store as
|
||||
// SB-GRPC-PSK-{SiteId}; production supplies this one as
|
||||
// ${secret:SB-GRPC-PSK-<siteId>}, expanded before the host is built.
|
||||
p.Require("ScadaBridge:Communication:GrpcPsk",
|
||||
value => !string.IsNullOrWhiteSpace(value),
|
||||
"is required for Site nodes: it is the preshared key the gRPC control "
|
||||
+ "plane authenticates with, and the interceptor is fail-closed, so an "
|
||||
+ "unset key refuses every SiteStream call. Set the same value here (in "
|
||||
+ "production as ${secret:SB-GRPC-PSK-<siteId>}) and under the secret "
|
||||
+ "name SB-GRPC-PSK-<siteId> in central's secret store");
|
||||
|
||||
// ScadaBridge:Database:SiteDbPath was required here until LocalDb
|
||||
// Phase 2. The site's tables now live in the consolidated LocalDb
|
||||
// database (LocalDb:Path, which SiteServiceRegistration requires),
|
||||
|
||||
@@ -42,6 +42,8 @@
|
||||
"SqliteDbPath": "./data/store-and-forward.db"
|
||||
},
|
||||
"Communication": {
|
||||
"_grpcPsk": "REQUIRED on Site nodes (StartupValidator fails the boot without it). The preshared key the gRPC control plane authenticates with: ControlPlaneAuthInterceptor is fail-closed, so an unset key refuses every SiteStream call — live subscriptions, audit pulls, cached-telemetry ingest — while the node still reports healthy. Supply it as ${secret:SB-GRPC-PSK-<siteId>} so the plaintext never sits in this file, and seed the SAME value on central: either as the secret SB-GRPC-PSK-<siteId> in its store, or as ScadaBridge:Communication:SitePsks:<siteId>. BOTH nodes of the pair carry the same key. Distinct from LocalDb:Replication:ApiKey, which authenticates the pair partner, not central — never share the two.",
|
||||
"GrpcPsk": "${secret:SB-GRPC-PSK-site-1}",
|
||||
"_centralContactPoints": "Host-016: each entry MUST be a central node's remoting endpoint, NOT this site's own remoting port. The single dev-loopback default below points only at central-a (localhost:8081). In a multi-central deployment add the second central node here (e.g. 'akka.tcp://scadabridge@central-b-host:8081') so ClusterClient can fail over when central-a is down. The previous template listed localhost:8082 as the second contact — that is THIS site's own RemotingPort and is a permanent failure in the initial-contact rotation.",
|
||||
"CentralContactPoints": [
|
||||
"akka.tcp://scadabridge@localhost:8081"
|
||||
|
||||
Reference in New Issue
Block a user