dohertj2/ScadaBridge
1
0
Fork 0
Code Issues 2 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat(grpc): PSK-authenticate the site gRPC control plane; drop the vestigial management receptionist registration

Browse Source 
Phase 0 of the ClusterClient→gRPC migration
(docs/plans/2026-07-22-clusterclient-to-grpc-plan.md). Standalone hardening: it
closes a gap that exists today and is a precondition for moving command/control
onto gRPC in later phases.

T0.1 — delete the ManagementActor ClusterClientReceptionist registration.
It was built for an out-of-cluster CLI that was never written: the shipped CLI
speaks HTTP Basic to /management, which asks the actor in-process through
ManagementActorHolder. Nothing in the repo ever sent to /user/management. The
actor still runs there; only the cross-boundary advertisement is gone. Six
documents claimed the CLI used ClusterClient — including the CLI's own README
"Architecture Notes" — and are corrected here rather than left to rot.

T0.2 — record, do not port, the dead integration-routing path.
IntegrationCallRequest is unwired at BOTH ends: RouteIntegrationCallAsync has
zero callers anywhere, and RegisterLocalHandler(Integration, …) appears only in
a test, so production always answers "Integration handler not available". It is
excluded from the gRPC contract (28 of 29 commands migrate) rather than
enshrined on an additive-only wire format, and deleting it during a
transport migration would mix a behavioural change into a change whose whole
value is that behaviour is identical. See
docs/known-issues/2026-07-22-integration-call-routing-is-dead-code.md.

T0.3 — preshared-key authentication on SiteStreamService.
The service shipped with no auth at all: plaintext h2c, no interceptor, so
anything that could reach a site node's :8083 could open a live data stream or
read audit rows back via PullAuditEvents/PullSiteCalls. ControlPlaneAuthInterceptor
now gates /sitestream.SiteStreamService/ — modeled on LocalDbSyncAuthInterceptor
(constant-time compare, fail-closed, PermissionDenied) but gating a SET of
service prefixes so phases 1A/1B add services rather than interceptors. LocalDb
sync keeps its own separate key: it authenticates the pair partner, not central,
and collapsing the two would make a site's central-facing key also admit writes
into its database.

Keys are per site (SB-GRPC-PSK-<siteId>), never fleet-wide, so a compromised
site yields only its own. Central attaches them through ControlPlaneCredentials,
which binds CallCredentials to the channel — covering unary and streaming
uniformly, and letting the key resolve asynchronously, which a client
interceptor could not do without blocking. All three central→site channel
creation sites go through it (SiteStreamGrpcClient and both audit pull invokers);
the pull invokers' channel caches are re-keyed by (site, endpoint) because
credentials are per-site and bound to the channel.

Two decisions beyond the plan:

  * StartupValidator now requires GrpcPsk on Site nodes. The plan specified only
    the runtime gate, but fail-closed with no boot check produces a node that
    joins, answers heartbeats and reports healthy while refusing every stream,
    audit pull and telemetry ingest — silent and total. Same reasoning as the
    existing inbound API-key pepper rule.

  * Added Communication:SitePsks as a central-side key map. The plan assumed
    central would read the store, seeded via a dev KEK; the docker rig
    deliberately boots with no master key, so store-only resolution would leave
    it unable to dial its own sites. The store stays primary — it is the only
    source that can serve a site added at runtime — with the map covering
    key-less hosts and one-off pins. Neither source falling back to
    "unauthenticated" is the invariant.

T0.4 — dev keys on both rigs and tests.
34 tests. The seven that matter most exercise a real in-process gRPC stack over
TestServer: the unit tests on either side of the wire would both stay green if
the halves disagreed, and gRPC refuses call credentials on a plaintext channel
by default — the UnsafeUseInsecureChannelCallCredentials opt-in is only provable
by making a real call. They confirm correct key passes on unary AND streaming,
wrong key and no-credentials both get PermissionDenied, and an unresolvable key
fails the call with nothing reaching the service.

OPERATIONAL: a site node upgraded to this build without a key will not boot.
That includes the gitignored deploy/wonder-app-vd03/ overlay.
This commit is contained in:
Joseph Doherty
2026-07-22 17:51:09 -04:00
parent f1ad967083
commit 2ee84af1c0
45 changed files with 1913 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/GrpcPullSiteCallsClient.cs
+34 -11
View File
@@ -174,7 +174,7 @@ public sealed class GrpcPullSiteCallsClient : IPullSiteCallsClient
{
try
{
var reply = await _invoker.InvokeAsync(endpoint, request, ct).ConfigureAwait(false);
var reply = await _invoker.InvokeAsync(siteId, endpoint, request, ct).ConfigureAwait(false);
return (reply, false);
}
catch (RpcException ex) when (IsTolerable(ex.StatusCode))
@@ -254,11 +254,17 @@ public sealed class GrpcPullSiteCallsClient : IPullSiteCallsClient
/// May throw <see cref="RpcException"/> / <see cref="HttpRequestException"/>
/// on transport faults — the caller classifies and swallows tolerable ones.
/// </summary>
/// <param name="siteId">
/// The site being pulled from. Selects which preshared key the call presents —
/// <c>PullSiteCalls</c> is gated by the site's <c>ControlPlaneAuthInterceptor</c>, and
/// keys are per-site, so the endpoint alone is not enough to authenticate.
/// </param>
/// <param name="endpoint">The site gRPC authority (e.g. <c>http://site-a:8083</c>).</param>
/// <param name="request">The wire-format pull request.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>The wire-format pull response.</returns>
Task<ProtoPullResponse> InvokeAsync(string endpoint, ProtoPullRequest request, CancellationToken ct);
Task<ProtoPullResponse> InvokeAsync(
string siteId, string endpoint, ProtoPullRequest request, CancellationToken ct);
}
}
@@ -277,8 +283,9 @@ public sealed class GrpcPullSiteCallsClient : IPullSiteCallsClient
public sealed class GrpcPullSiteCallsInvoker
: GrpcPullSiteCallsClient.IPullSiteCallsInvoker, IDisposable
{
private readonly ConcurrentDictionary<string, GrpcChannel> _channels = new(StringComparer.Ordinal);
private readonly ConcurrentDictionary<(string Site, string Endpoint), GrpcChannel> _channels = new();
private readonly CommunicationOptions _options;
private readonly ISitePskProvider? _pskProvider;
/// <summary>Creates the invoker using default <see cref="CommunicationOptions"/>.</summary>
public GrpcPullSiteCallsInvoker()
@@ -292,15 +299,27 @@ public sealed class GrpcPullSiteCallsInvoker
/// </summary>
/// <param name="options">Communication options supplying gRPC keepalive timings.</param>
public GrpcPullSiteCallsInvoker(CommunicationOptions options)
: this(options, pskProvider: null)
{
}
/// <summary>
/// Creates the invoker with per-site call credentials, the production shape: the site's
/// <c>ControlPlaneAuthInterceptor</c> refuses an unauthenticated <c>PullSiteCalls</c>.
/// </summary>
/// <param name="options">Communication options supplying gRPC keepalive timings.</param>
/// <param name="pskProvider">Resolves each site's preshared key; null dials unauthenticated.</param>
public GrpcPullSiteCallsInvoker(CommunicationOptions options, ISitePskProvider? pskProvider)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
_pskProvider = pskProvider;
}
/// <inheritdoc />
public async Task<ProtoPullResponse> InvokeAsync(
string endpoint, ProtoPullRequest request, CancellationToken ct)
string siteId, string endpoint, ProtoPullRequest request, CancellationToken ct)
{
var channel = GetOrCreateChannel(endpoint);
var channel = GetOrCreateChannel(siteId, endpoint);
var client = new SiteStreamService.SiteStreamServiceClient(channel);
using var call = client.PullSiteCallsAsync(request, cancellationToken: ct);
return await call.ResponseAsync.ConfigureAwait(false);
@@ -310,12 +329,13 @@ public sealed class GrpcPullSiteCallsInvoker
// concurrent first dials of the same endpoint can both build a GrpcChannel;
// only the channel actually installed survives, the loser is disposed.
// Mirrors SiteStreamGrpcClientFactory / GrpcPullAuditEventsInvoker.
private GrpcChannel GetOrCreateChannel(string endpoint)
private GrpcChannel GetOrCreateChannel(string siteId, string endpoint)
{
if (!_channels.TryGetValue(endpoint, out var channel))
var key = (siteId, endpoint);
if (!_channels.TryGetValue(key, out var channel))
{
var created = CreateChannel(endpoint);
channel = _channels.GetOrAdd(endpoint, created);
var created = CreateChannel(siteId, endpoint);
channel = _channels.GetOrAdd(key, created);
if (!ReferenceEquals(channel, created))
{
created.Dispose();
@@ -324,7 +344,10 @@ public sealed class GrpcPullSiteCallsInvoker
return channel;
}
private GrpcChannel CreateChannel(string endpoint) =>
// Keyed by (site, endpoint) rather than endpoint alone: the call credentials are bound to
// the channel, and they are per-site, so two sites sharing an endpoint string would
// otherwise share one channel carrying the first site's key.
private GrpcChannel CreateChannel(string siteId, string endpoint) =>
GrpcChannel.ForAddress(endpoint, new GrpcChannelOptions
{
HttpHandler = new SocketsHttpHandler
@@ -333,7 +356,7 @@ public sealed class GrpcPullSiteCallsInvoker
KeepAlivePingTimeout = _options.GrpcKeepAlivePingTimeout,
KeepAlivePingPolicy = HttpKeepAlivePingPolicy.Always,
},
});
}.WithSiteCredentials(_pskProvider, siteId));
/// <summary>Disposes all cached channels.</summary>
public void Dispose()