feat(grpc): PSK-authenticate the site gRPC control plane; drop the vestigial management receptionist registration
Phase 0 of the ClusterClient→gRPC migration
(docs/plans/2026-07-22-clusterclient-to-grpc-plan.md). Standalone hardening: it
closes a gap that exists today and is a precondition for moving command/control
onto gRPC in later phases.
T0.1 — delete the ManagementActor ClusterClientReceptionist registration.
It was built for an out-of-cluster CLI that was never written: the shipped CLI
speaks HTTP Basic to /management, which asks the actor in-process through
ManagementActorHolder. Nothing in the repo ever sent to /user/management. The
actor still runs there; only the cross-boundary advertisement is gone. Six
documents claimed the CLI used ClusterClient — including the CLI's own README
"Architecture Notes" — and are corrected here rather than left to rot.
T0.2 — record, do not port, the dead integration-routing path.
IntegrationCallRequest is unwired at BOTH ends: RouteIntegrationCallAsync has
zero callers anywhere, and RegisterLocalHandler(Integration, …) appears only in
a test, so production always answers "Integration handler not available". It is
excluded from the gRPC contract (28 of 29 commands migrate) rather than
enshrined on an additive-only wire format, and deleting it during a
transport migration would mix a behavioural change into a change whose whole
value is that behaviour is identical. See
docs/known-issues/2026-07-22-integration-call-routing-is-dead-code.md.
T0.3 — preshared-key authentication on SiteStreamService.
The service shipped with no auth at all: plaintext h2c, no interceptor, so
anything that could reach a site node's :8083 could open a live data stream or
read audit rows back via PullAuditEvents/PullSiteCalls. ControlPlaneAuthInterceptor
now gates /sitestream.SiteStreamService/ — modeled on LocalDbSyncAuthInterceptor
(constant-time compare, fail-closed, PermissionDenied) but gating a SET of
service prefixes so phases 1A/1B add services rather than interceptors. LocalDb
sync keeps its own separate key: it authenticates the pair partner, not central,
and collapsing the two would make a site's central-facing key also admit writes
into its database.
Keys are per site (SB-GRPC-PSK-<siteId>), never fleet-wide, so a compromised
site yields only its own. Central attaches them through ControlPlaneCredentials,
which binds CallCredentials to the channel — covering unary and streaming
uniformly, and letting the key resolve asynchronously, which a client
interceptor could not do without blocking. All three central→site channel
creation sites go through it (SiteStreamGrpcClient and both audit pull invokers);
the pull invokers' channel caches are re-keyed by (site, endpoint) because
credentials are per-site and bound to the channel.
Two decisions beyond the plan:
* StartupValidator now requires GrpcPsk on Site nodes. The plan specified only
the runtime gate, but fail-closed with no boot check produces a node that
joins, answers heartbeats and reports healthy while refusing every stream,
audit pull and telemetry ingest — silent and total. Same reasoning as the
existing inbound API-key pepper rule.
* Added Communication:SitePsks as a central-side key map. The plan assumed
central would read the store, seeded via a dev KEK; the docker rig
deliberately boots with no master key, so store-only resolution would leave
it unable to dial its own sites. The store stays primary — it is the only
source that can serve a site added at runtime — with the map covering
key-less hosts and one-off pins. Neither source falling back to
"unauthenticated" is the invariant.
T0.4 — dev keys on both rigs and tests.
34 tests. The seven that matter most exercise a real in-process gRPC stack over
TestServer: the unit tests on either side of the wire would both stay green if
the halves disagreed, and gRPC refuses call credentials on a plaintext channel
by default — the UnsafeUseInsecureChannelCallCredentials opt-in is only provable
by making a real call. They confirm correct key passes on unary AND streaming,
wrong key and no-credentials both get PermissionDenied, and an unresolvable key
fails the call with nothing reaching the service.
OPERATIONAL: a site node upgraded to this build without a key will not boot.
That includes the gitignored deploy/wonder-app-vd03/ overlay.
This commit is contained in:
@@ -26,9 +26,11 @@ public class SiteStreamGrpcClientFactory : IAsyncDisposable, IDisposable
|
||||
private readonly ConcurrentDictionary<(string Site, string Endpoint), SiteStreamGrpcClient> _clients = new();
|
||||
private readonly ILoggerFactory _loggerFactory;
|
||||
private readonly CommunicationOptions _options;
|
||||
private readonly ISitePskProvider? _pskProvider;
|
||||
|
||||
/// <summary>
|
||||
/// Test/default constructor — uses default <see cref="CommunicationOptions"/>.
|
||||
/// Test/default constructor — uses default <see cref="CommunicationOptions"/> and creates
|
||||
/// unauthenticated channels.
|
||||
/// </summary>
|
||||
/// <param name="loggerFactory">Logger factory passed to created clients.</param>
|
||||
public SiteStreamGrpcClientFactory(ILoggerFactory loggerFactory)
|
||||
@@ -37,16 +39,36 @@ public class SiteStreamGrpcClientFactory : IAsyncDisposable, IDisposable
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// DI constructor — flows <see cref="CommunicationOptions"/> into every created
|
||||
/// <see cref="SiteStreamGrpcClient"/> so the configured gRPC keepalive settings
|
||||
/// are applied rather than hard-coded defaults.
|
||||
/// Constructor without a key provider — creates unauthenticated channels, which a gated
|
||||
/// site will refuse. Retained for tests and for hosts that never dial a site.
|
||||
/// </summary>
|
||||
/// <param name="loggerFactory">Logger factory passed to created clients.</param>
|
||||
/// <param name="options">Communication options applied to each created client.</param>
|
||||
public SiteStreamGrpcClientFactory(ILoggerFactory loggerFactory, IOptions<CommunicationOptions> options)
|
||||
: this(loggerFactory, options, pskProvider: null)
|
||||
{
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// DI constructor — flows <see cref="CommunicationOptions"/> into every created
|
||||
/// <see cref="SiteStreamGrpcClient"/> so the configured gRPC keepalive settings are applied
|
||||
/// rather than hard-coded defaults, and attaches the per-site preshared key that the site's
|
||||
/// <c>ControlPlaneAuthInterceptor</c> requires.
|
||||
/// </summary>
|
||||
/// <param name="loggerFactory">Logger factory passed to created clients.</param>
|
||||
/// <param name="options">Communication options applied to each created client.</param>
|
||||
/// <param name="pskProvider">
|
||||
/// Resolves each site's preshared key. Optional in DI so a host that registers no provider
|
||||
/// (a site node, which never dials another site) still resolves this factory.
|
||||
/// </param>
|
||||
public SiteStreamGrpcClientFactory(
|
||||
ILoggerFactory loggerFactory,
|
||||
IOptions<CommunicationOptions> options,
|
||||
ISitePskProvider? pskProvider)
|
||||
{
|
||||
_loggerFactory = loggerFactory;
|
||||
_options = options.Value;
|
||||
_pskProvider = pskProvider;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -59,7 +81,7 @@ public class SiteStreamGrpcClientFactory : IAsyncDisposable, IDisposable
|
||||
/// <param name="grpcEndpoint">gRPC endpoint (second half of the cache key) the client is bound to.</param>
|
||||
/// <returns>The cached or newly-created client bound to <paramref name="grpcEndpoint"/>.</returns>
|
||||
public virtual SiteStreamGrpcClient GetOrCreate(string siteIdentifier, string grpcEndpoint) =>
|
||||
_clients.GetOrAdd((siteIdentifier, grpcEndpoint), _ => CreateClient(grpcEndpoint));
|
||||
_clients.GetOrAdd((siteIdentifier, grpcEndpoint), key => CreateClient(key.Site, key.Endpoint));
|
||||
|
||||
/// <summary>
|
||||
/// Returns the cached client for <c>(site, endpoint)</c>, or <c>null</c> — never creates.
|
||||
@@ -77,12 +99,13 @@ public class SiteStreamGrpcClientFactory : IAsyncDisposable, IDisposable
|
||||
/// can substitute a tracking client while still exercising the factory's real
|
||||
/// caching and disposal machinery.
|
||||
/// </summary>
|
||||
/// <param name="siteIdentifier">Site the new client talks to; selects which preshared key it presents.</param>
|
||||
/// <param name="grpcEndpoint">gRPC endpoint the new client will connect to.</param>
|
||||
/// <returns>A new <see cref="SiteStreamGrpcClient"/> connected to <paramref name="grpcEndpoint"/>.</returns>
|
||||
protected virtual SiteStreamGrpcClient CreateClient(string grpcEndpoint)
|
||||
protected virtual SiteStreamGrpcClient CreateClient(string siteIdentifier, string grpcEndpoint)
|
||||
{
|
||||
var logger = _loggerFactory.CreateLogger<SiteStreamGrpcClient>();
|
||||
return new SiteStreamGrpcClient(grpcEndpoint, logger, _options);
|
||||
return new SiteStreamGrpcClient(grpcEndpoint, logger, _options, _pskProvider, siteIdentifier);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -99,6 +122,10 @@ public class SiteStreamGrpcClientFactory : IAsyncDisposable, IDisposable
|
||||
if (_clients.TryRemove(key, out var client))
|
||||
await client.DisposeAsync();
|
||||
}
|
||||
|
||||
// Drop the cached preshared key too, so a site removed and re-added under the same
|
||||
// identifier (with a rotated key) is not dialed with the stale one.
|
||||
_pskProvider?.Invalidate(siteIdentifier);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
|
||||
Reference in New Issue
Block a user