Files
ScadaBridge/src/ZB.MOM.WW.ScadaBridge.Host/SitePskProvider.cs
T
Joseph Doherty 2ee84af1c0 feat(grpc): PSK-authenticate the site gRPC control plane; drop the vestigial management receptionist registration
Phase 0 of the ClusterClient→gRPC migration
(docs/plans/2026-07-22-clusterclient-to-grpc-plan.md). Standalone hardening: it
closes a gap that exists today and is a precondition for moving command/control
onto gRPC in later phases.

T0.1 — delete the ManagementActor ClusterClientReceptionist registration.
It was built for an out-of-cluster CLI that was never written: the shipped CLI
speaks HTTP Basic to /management, which asks the actor in-process through
ManagementActorHolder. Nothing in the repo ever sent to /user/management. The
actor still runs there; only the cross-boundary advertisement is gone. Six
documents claimed the CLI used ClusterClient — including the CLI's own README
"Architecture Notes" — and are corrected here rather than left to rot.

T0.2 — record, do not port, the dead integration-routing path.
IntegrationCallRequest is unwired at BOTH ends: RouteIntegrationCallAsync has
zero callers anywhere, and RegisterLocalHandler(Integration, …) appears only in
a test, so production always answers "Integration handler not available". It is
excluded from the gRPC contract (28 of 29 commands migrate) rather than
enshrined on an additive-only wire format, and deleting it during a
transport migration would mix a behavioural change into a change whose whole
value is that behaviour is identical. See
docs/known-issues/2026-07-22-integration-call-routing-is-dead-code.md.

T0.3 — preshared-key authentication on SiteStreamService.
The service shipped with no auth at all: plaintext h2c, no interceptor, so
anything that could reach a site node's :8083 could open a live data stream or
read audit rows back via PullAuditEvents/PullSiteCalls. ControlPlaneAuthInterceptor
now gates /sitestream.SiteStreamService/ — modeled on LocalDbSyncAuthInterceptor
(constant-time compare, fail-closed, PermissionDenied) but gating a SET of
service prefixes so phases 1A/1B add services rather than interceptors. LocalDb
sync keeps its own separate key: it authenticates the pair partner, not central,
and collapsing the two would make a site's central-facing key also admit writes
into its database.

Keys are per site (SB-GRPC-PSK-<siteId>), never fleet-wide, so a compromised
site yields only its own. Central attaches them through ControlPlaneCredentials,
which binds CallCredentials to the channel — covering unary and streaming
uniformly, and letting the key resolve asynchronously, which a client
interceptor could not do without blocking. All three central→site channel
creation sites go through it (SiteStreamGrpcClient and both audit pull invokers);
the pull invokers' channel caches are re-keyed by (site, endpoint) because
credentials are per-site and bound to the channel.

Two decisions beyond the plan:

  * StartupValidator now requires GrpcPsk on Site nodes. The plan specified only
    the runtime gate, but fail-closed with no boot check produces a node that
    joins, answers heartbeats and reports healthy while refusing every stream,
    audit pull and telemetry ingest — silent and total. Same reasoning as the
    existing inbound API-key pepper rule.

  * Added Communication:SitePsks as a central-side key map. The plan assumed
    central would read the store, seeded via a dev KEK; the docker rig
    deliberately boots with no master key, so store-only resolution would leave
    it unable to dial its own sites. The store stays primary — it is the only
    source that can serve a site added at runtime — with the map covering
    key-less hosts and one-off pins. Neither source falling back to
    "unauthenticated" is the invariant.

T0.4 — dev keys on both rigs and tests.
34 tests. The seven that matter most exercise a real in-process gRPC stack over
TestServer: the unit tests on either side of the wire would both stay green if
the halves disagreed, and gRPC refuses call credentials on a plaintext channel
by default — the UnsafeUseInsecureChannelCallCredentials opt-in is only provable
by making a real call. They confirm correct key passes on unary AND streaming,
wrong key and no-credentials both get PermissionDenied, and an unresolvable key
fails the call with nothing reaching the service.

OPERATIONAL: a site node upgraded to this build without a key will not boot.
That includes the gitignored deploy/wonder-app-vd03/ overlay.
2026-07-22 17:51:09 -04:00

150 lines
6.5 KiB
C#

using System.Collections.Concurrent;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.ScadaBridge.Communication;
using ZB.MOM.WW.ScadaBridge.Communication.Grpc;
using ZB.MOM.WW.Secrets.Abstractions;
namespace ZB.MOM.WW.ScadaBridge.Host;
/// <summary>
/// Central's <see cref="ISitePskProvider"/>: resolves each site's gRPC preshared key at
/// channel-build time — from <c>ScadaBridge:Communication:SitePsks</c> if the site is listed
/// there, otherwise from the secrets store under the name <c>SB-GRPC-PSK-{siteId}</c>.
/// </summary>
/// <remarks>
/// <para>
/// <b>Why resolved at runtime rather than expanded into config at boot.</b> Site nodes get their
/// single key through the pre-host <c>SecretReferenceExpander</c> — a <c>${secret:…}</c> in
/// appsettings, resolved once before the host is built. Central cannot do that: its set of sites
/// comes from the configuration database and changes while the process runs, so there is no
/// fixed list of references to expand at boot. This mirrors the pattern the MxGateway data
/// connection already uses for its per-connection API keys.
/// </para>
/// <para>
/// <b>Caching.</b> Successful resolves are cached indefinitely; a site's key changes only on
/// rotation, and rotation restarts the pair. Failures are deliberately NOT cached, so a key
/// seeded after the first (failed) dial is picked up on the next attempt instead of requiring a
/// restart. <see cref="Invalidate"/> drops an entry when a site is removed or a key is rotated
/// in place.
/// </para>
/// <para>
/// <b>Fail-closed.</b> A key absent from BOTH sources throws, and the dial that needed it fails.
/// It never degrades to an unauthenticated call. The message names the exact config key and the
/// exact secret name, so the fix is one setting or one <c>secret</c> CLI seed.
/// </para>
/// </remarks>
public sealed class SitePskProvider : ISitePskProvider
{
/// <summary>Prefix of the per-site secret name; the site identifier is appended verbatim.</summary>
public const string SecretNamePrefix = "SB-GRPC-PSK-";
private readonly ConcurrentDictionary<string, string> _cache = new(StringComparer.Ordinal);
private readonly ISecretResolver _resolver;
private readonly IOptionsMonitor<CommunicationOptions> _options;
private readonly ILogger<SitePskProvider> _logger;
/// <summary>Creates the provider over the host's secret resolver.</summary>
/// <param name="resolver">Runtime secret resolver from the host container.</param>
/// <param name="options">
/// Communication options supplying the optional <c>SitePsks</c> map. Read through an
/// <see cref="IOptionsMonitor{TOptions}"/> so a reloaded configuration is honoured on the
/// next uncached resolve rather than pinned at construction.
/// </param>
/// <param name="logger">Logger for resolution diagnostics.</param>
public SitePskProvider(
ISecretResolver resolver,
IOptionsMonitor<CommunicationOptions> options,
ILogger<SitePskProvider> logger)
{
ArgumentNullException.ThrowIfNull(resolver);
ArgumentNullException.ThrowIfNull(options);
ArgumentNullException.ThrowIfNull(logger);
_resolver = resolver;
_options = options;
_logger = logger;
}
/// <summary>Builds the secret name for a site.</summary>
/// <param name="siteId">Site identifier.</param>
/// <returns>The secret name, e.g. <c>SB-GRPC-PSK-site-a</c>.</returns>
public static string SecretNameFor(string siteId) => SecretNamePrefix + siteId;
/// <inheritdoc />
public async ValueTask<string> GetAsync(string siteId, CancellationToken ct)
{
ArgumentException.ThrowIfNullOrWhiteSpace(siteId);
if (_cache.TryGetValue(siteId, out var cached))
{
return cached;
}
// Configured map first: it is the explicit, operator-stated answer for this site, and
// it is the only source available to a host running without a secrets master key.
var configured = _options.CurrentValue.SitePsks;
if (configured.TryGetValue(siteId, out var fromConfig) && !string.IsNullOrEmpty(fromConfig))
{
_cache[siteId] = fromConfig;
return fromConfig;
}
// Otherwise the store, which is the only source that can serve a site added at runtime.
var secretName = SecretNameFor(siteId);
var value = await ResolveFromStoreAsync(secretName, siteId, ct).ConfigureAwait(false);
if (string.IsNullOrEmpty(value))
{
_logger.LogError(
"gRPC control-plane key for site {SiteId} could not be resolved: it is absent from "
+ "ScadaBridge:Communication:SitePsks and secret '{SecretName}' is missing, empty or "
+ "tombstoned. Calls to this site are refused until one of the two is set (the site "
+ "node must carry the same value in ScadaBridge:Communication:GrpcPsk).",
siteId, secretName);
throw new InvalidOperationException(
$"gRPC preshared key for site '{siteId}' could not be resolved from "
+ $"ScadaBridge:Communication:SitePsks or secret '{secretName}'.");
}
// Two concurrent first dials may both resolve; the read is idempotent and the values
// identical, so the loser simply overwrites with the same string.
_cache[siteId] = value;
return value;
}
/// <summary>
/// Reads the key from the secrets store, treating a store fault as "not found" rather than
/// letting it propagate. A host with no master key configured (the development rig) throws
/// from the resolver; that must produce the same clear "key not configured" error as a
/// missing secret, not an opaque cryptographic one.
/// </summary>
private async Task<string?> ResolveFromStoreAsync(string secretName, string siteId, CancellationToken ct)
{
try
{
return await _resolver.GetAsync(new SecretName(secretName), ct).ConfigureAwait(false);
}
catch (OperationCanceledException)
{
throw;
}
catch (Exception ex)
{
_logger.LogDebug(ex,
"Secret store lookup for '{SecretName}' (site {SiteId}) faulted; treating as not found.",
secretName, siteId);
return null;
}
}
/// <inheritdoc />
public void Invalidate(string siteId)
{
if (!string.IsNullOrWhiteSpace(siteId))
{
_cache.TryRemove(siteId, out _);
}
}
}