feat(grpc): PSK-authenticate the site gRPC control plane; drop the vestigial management receptionist registration
Phase 0 of the ClusterClient→gRPC migration
(docs/plans/2026-07-22-clusterclient-to-grpc-plan.md). Standalone hardening: it
closes a gap that exists today and is a precondition for moving command/control
onto gRPC in later phases.
T0.1 — delete the ManagementActor ClusterClientReceptionist registration.
It was built for an out-of-cluster CLI that was never written: the shipped CLI
speaks HTTP Basic to /management, which asks the actor in-process through
ManagementActorHolder. Nothing in the repo ever sent to /user/management. The
actor still runs there; only the cross-boundary advertisement is gone. Six
documents claimed the CLI used ClusterClient — including the CLI's own README
"Architecture Notes" — and are corrected here rather than left to rot.
T0.2 — record, do not port, the dead integration-routing path.
IntegrationCallRequest is unwired at BOTH ends: RouteIntegrationCallAsync has
zero callers anywhere, and RegisterLocalHandler(Integration, …) appears only in
a test, so production always answers "Integration handler not available". It is
excluded from the gRPC contract (28 of 29 commands migrate) rather than
enshrined on an additive-only wire format, and deleting it during a
transport migration would mix a behavioural change into a change whose whole
value is that behaviour is identical. See
docs/known-issues/2026-07-22-integration-call-routing-is-dead-code.md.
T0.3 — preshared-key authentication on SiteStreamService.
The service shipped with no auth at all: plaintext h2c, no interceptor, so
anything that could reach a site node's :8083 could open a live data stream or
read audit rows back via PullAuditEvents/PullSiteCalls. ControlPlaneAuthInterceptor
now gates /sitestream.SiteStreamService/ — modeled on LocalDbSyncAuthInterceptor
(constant-time compare, fail-closed, PermissionDenied) but gating a SET of
service prefixes so phases 1A/1B add services rather than interceptors. LocalDb
sync keeps its own separate key: it authenticates the pair partner, not central,
and collapsing the two would make a site's central-facing key also admit writes
into its database.
Keys are per site (SB-GRPC-PSK-<siteId>), never fleet-wide, so a compromised
site yields only its own. Central attaches them through ControlPlaneCredentials,
which binds CallCredentials to the channel — covering unary and streaming
uniformly, and letting the key resolve asynchronously, which a client
interceptor could not do without blocking. All three central→site channel
creation sites go through it (SiteStreamGrpcClient and both audit pull invokers);
the pull invokers' channel caches are re-keyed by (site, endpoint) because
credentials are per-site and bound to the channel.
Two decisions beyond the plan:
* StartupValidator now requires GrpcPsk on Site nodes. The plan specified only
the runtime gate, but fail-closed with no boot check produces a node that
joins, answers heartbeats and reports healthy while refusing every stream,
audit pull and telemetry ingest — silent and total. Same reasoning as the
existing inbound API-key pepper rule.
* Added Communication:SitePsks as a central-side key map. The plan assumed
central would read the store, seeded via a dev KEK; the docker rig
deliberately boots with no master key, so store-only resolution would leave
it unable to dial its own sites. The store stays primary — it is the only
source that can serve a site added at runtime — with the map covering
key-less hosts and one-off pins. Neither source falling back to
"unauthenticated" is the invariant.
T0.4 — dev keys on both rigs and tests.
34 tests. The seven that matter most exercise a real in-process gRPC stack over
TestServer: the unit tests on either side of the wire would both stay green if
the halves disagreed, and gRPC refuses call credentials on a plaintext channel
by default — the UnsafeUseInsecureChannelCallCredentials opt-in is only provable
by making a real call. They confirm correct key passes on unary AND streaming,
wrong key and no-credentials both get PermissionDenied, and an unresolvable key
fails the call with nothing reaching the service.
OPERATIONAL: a site node upgraded to this build without a key will not boot.
That includes the gitignored deploy/wonder-app-vd03/ overlay.
This commit is contained in:
@@ -178,9 +178,45 @@ database from its peer rather than letting it rejoin.
|
||||
|
||||
### Central-Site Communication
|
||||
|
||||
- Sites connect to central via Akka.NET remoting.
|
||||
- The `Communication:CentralSeedNode` setting in the site config points to one of the central nodes.
|
||||
- If that central node is down, the site's communication actor will retry until it connects to the active central node.
|
||||
Three transports cross the boundary, not one:
|
||||
|
||||
- **Akka ClusterClient** — command/control. Sites list every central node in
|
||||
`ScadaBridge:Communication:CentralContactPoints`; contact rotation reaches whichever node
|
||||
answers, so no "active central" needs to be identified. (There is no `Communication:CentralSeedNode`
|
||||
setting — earlier revisions of this guide named one that never existed in the code.)
|
||||
- **gRPC** — real-time data and audit pull. Note the direction is inverted from the data flow:
|
||||
each **site node hosts the gRPC server** on `GrpcPort` (default 8083, h2c) and central dials in.
|
||||
- **Plain HTTP** — the deploy config itself, fetched by the site with a per-deployment token.
|
||||
|
||||
#### gRPC control-plane preshared key (required)
|
||||
|
||||
Every site node must set `ScadaBridge:Communication:GrpcPsk`, and central must hold the same
|
||||
value for that site. **`StartupValidator` refuses to boot a site node without it**, deliberately:
|
||||
the gate is fail-closed, so an unset key would leave the node joined, healthy-looking and
|
||||
answering heartbeats while refusing every gRPC call — no live subscriptions, no audit pull, no
|
||||
cached-telemetry ingest.
|
||||
|
||||
| Side | Where the key lives |
|
||||
|---|---|
|
||||
| Site node (both nodes of the pair, identical) | `ScadaBridge:Communication:GrpcPsk`, in production `${secret:SB-GRPC-PSK-<siteId>}` |
|
||||
| Central | secret `SB-GRPC-PSK-<siteId>` in its store — **or** `ScadaBridge:Communication:SitePsks:<siteId>` |
|
||||
|
||||
The store is the source that matters in production, because sites are added at runtime and their
|
||||
keys cannot be enumerated in configuration at boot; `SitePsks` covers a host running without a
|
||||
master key (the docker rig) and one-off pins.
|
||||
|
||||
One key **per site**, never one for the fleet: a compromised site must not yield another site's
|
||||
key. And never share it with `LocalDb:Replication:ApiKey` — that authenticates the *pair partner*
|
||||
for database replication, a different trust relationship on the same listener.
|
||||
|
||||
**Rotation:** set the new value on both sides, then restart the pair (pairs restart together
|
||||
anyway — see above). **Upgrading to a build that has this gate requires seeding the key first**,
|
||||
including in the on-host `deploy/` overlays.
|
||||
|
||||
The key is a bearer token over plaintext h2c, so it is readable and replayable by anyone on the
|
||||
path. That is the accepted posture today — the same trusted-network assumption the boundary
|
||||
already made, now with authentication rather than none. TLS on these listeners is follow-on
|
||||
hardening and needs no change to the key design.
|
||||
|
||||
## Scaling Guidelines
|
||||
|
||||
|
||||
Reference in New Issue
Block a user