feat(grpc): PSK-authenticate the site gRPC control plane; drop the vestigial management receptionist registration
Phase 0 of the ClusterClient→gRPC migration
(docs/plans/2026-07-22-clusterclient-to-grpc-plan.md). Standalone hardening: it
closes a gap that exists today and is a precondition for moving command/control
onto gRPC in later phases.
T0.1 — delete the ManagementActor ClusterClientReceptionist registration.
It was built for an out-of-cluster CLI that was never written: the shipped CLI
speaks HTTP Basic to /management, which asks the actor in-process through
ManagementActorHolder. Nothing in the repo ever sent to /user/management. The
actor still runs there; only the cross-boundary advertisement is gone. Six
documents claimed the CLI used ClusterClient — including the CLI's own README
"Architecture Notes" — and are corrected here rather than left to rot.
T0.2 — record, do not port, the dead integration-routing path.
IntegrationCallRequest is unwired at BOTH ends: RouteIntegrationCallAsync has
zero callers anywhere, and RegisterLocalHandler(Integration, …) appears only in
a test, so production always answers "Integration handler not available". It is
excluded from the gRPC contract (28 of 29 commands migrate) rather than
enshrined on an additive-only wire format, and deleting it during a
transport migration would mix a behavioural change into a change whose whole
value is that behaviour is identical. See
docs/known-issues/2026-07-22-integration-call-routing-is-dead-code.md.
T0.3 — preshared-key authentication on SiteStreamService.
The service shipped with no auth at all: plaintext h2c, no interceptor, so
anything that could reach a site node's :8083 could open a live data stream or
read audit rows back via PullAuditEvents/PullSiteCalls. ControlPlaneAuthInterceptor
now gates /sitestream.SiteStreamService/ — modeled on LocalDbSyncAuthInterceptor
(constant-time compare, fail-closed, PermissionDenied) but gating a SET of
service prefixes so phases 1A/1B add services rather than interceptors. LocalDb
sync keeps its own separate key: it authenticates the pair partner, not central,
and collapsing the two would make a site's central-facing key also admit writes
into its database.
Keys are per site (SB-GRPC-PSK-<siteId>), never fleet-wide, so a compromised
site yields only its own. Central attaches them through ControlPlaneCredentials,
which binds CallCredentials to the channel — covering unary and streaming
uniformly, and letting the key resolve asynchronously, which a client
interceptor could not do without blocking. All three central→site channel
creation sites go through it (SiteStreamGrpcClient and both audit pull invokers);
the pull invokers' channel caches are re-keyed by (site, endpoint) because
credentials are per-site and bound to the channel.
Two decisions beyond the plan:
* StartupValidator now requires GrpcPsk on Site nodes. The plan specified only
the runtime gate, but fail-closed with no boot check produces a node that
joins, answers heartbeats and reports healthy while refusing every stream,
audit pull and telemetry ingest — silent and total. Same reasoning as the
existing inbound API-key pepper rule.
* Added Communication:SitePsks as a central-side key map. The plan assumed
central would read the store, seeded via a dev KEK; the docker rig
deliberately boots with no master key, so store-only resolution would leave
it unable to dial its own sites. The store stays primary — it is the only
source that can serve a site added at runtime — with the map covering
key-less hosts and one-off pins. Neither source falling back to
"unauthenticated" is the invariant.
T0.4 — dev keys on both rigs and tests.
34 tests. The seven that matter most exercise a real in-process gRPC stack over
TestServer: the unit tests on either side of the wire would both stay green if
the halves disagreed, and gRPC refuses call credentials on a plaintext channel
by default — the UnsafeUseInsecureChannelCallCredentials opt-in is only provable
by making a real call. They confirm correct key passes on unary AND streaming,
wrong key and no-credentials both get PermissionDenied, and an unresolvable key
fails the call with nothing reaching the service.
OPERATIONAL: a site node upgraded to this build without a key will not boot.
That includes the gitignored deploy/wonder-app-vd03/ overlay.
This commit is contained in:
@@ -109,6 +109,31 @@ The streaming protocol is defined in `sitestream.proto` (`src/ZB.MOM.WW.ScadaBri
|
||||
- The `oneof event` pattern is extensible — future event types (health metrics, connection state changes) are added as new fields without breaking existing consumers.
|
||||
- Proto field numbers are never reused; new RPCs and message fields are appended additively. Old clients ignore unknown `oneof` variants.
|
||||
|
||||
##### Authentication (preshared key, 2026-07-22)
|
||||
|
||||
Every RPC on this service — streaming and unary alike — requires a preshared key, presented as
|
||||
`authorization: Bearer <key>` metadata and verified by `ControlPlaneAuthInterceptor` on the site
|
||||
node. The gate is **fail-closed**: with no key configured, every call is refused with
|
||||
`PermissionDenied`, and `StartupValidator` refuses to boot a site node in that state (an unset key
|
||||
would otherwise leave the node joined and reporting healthy while serving nothing).
|
||||
|
||||
Keys are scoped **one per site** — secret `SB-GRPC-PSK-<siteId>`, so a compromised site never
|
||||
yields another site's key. The site node reads its own key from
|
||||
`ScadaBridge:Communication:GrpcPsk` (in production a `${secret:}` reference expanded before the
|
||||
host is built); central resolves each site's key at channel-build time via `SitePskProvider`,
|
||||
because sites are created at runtime and cannot be enumerated in configuration at boot.
|
||||
`ControlPlaneCredentials` binds the key to the channel as `CallCredentials`, so no individual call
|
||||
site can omit it.
|
||||
|
||||
This is distinct from `LocalDb:Replication:ApiKey`, which gates
|
||||
`/localdb_sync.v1.LocalDbSync/` on the same listener via its own interceptor: that key
|
||||
authenticates the *pair partner* for database replication, a different trust relationship, and the
|
||||
two are never shared.
|
||||
|
||||
The transport remains h2c, so the key is readable and replayable by anyone on the path — the
|
||||
boundary still assumes a trusted network, with the bar raised from "can reach the port" to "can
|
||||
read the traffic". TLS is follow-on hardening and does not change this design.
|
||||
|
||||
#### Enriched AlarmStateUpdate (Native Alarm Mirror)
|
||||
|
||||
`AlarmStateUpdate` carries the read-only native alarm mirror (Computed, native OPC UA, and native MxAccess Gateway alarms) to central over the **existing gRPC real-time stream** — no new transport, no command/control round-trip. The message was extended **additively**: existing fields 1–7 are unchanged, and fields 8–23 carry the enriched native-alarm state. Old clients that only read fields 1–7 continue to work; new fields are populated only where the source provides them.
|
||||
@@ -306,7 +331,9 @@ Akka.NET guarantees message ordering between a specific sender/receiver actor pa
|
||||
|
||||
## ManagementActor and ClusterClient
|
||||
|
||||
The ManagementActor is registered at the well-known path `/user/management` on central nodes and advertised via **ClusterClientReceptionist**. External tools (primarily the CLI) connect using Akka.NET ClusterClient, which contacts the receptionist to discover the ManagementActor. This is a separate ClusterClient usage from the inter-cluster ClusterClient connections used for central-site messaging — the CLI does not participate in cluster membership or affect the hub-and-spoke topology.
|
||||
The ManagementActor runs at the well-known path `/user/management` on central nodes. It is **not** advertised via ClusterClientReceptionist, and no ClusterClient reaches it.
|
||||
|
||||
That registration existed until 2026-07-22 for an out-of-cluster CLI that was never built. The shipped CLI speaks **HTTP Basic to the central `/management` endpoints**; those endpoints ask the ManagementActor **in-process** through `ManagementActorHolder` (`ManagementEndpoints.cs`). Nothing in the repo ever sent to `/user/management` across the boundary, so the registration was deleted — leaving exactly one receptionist registration per cluster role (`CentralCommunicationActor` on central, `SiteCommunicationActor` on sites), both of which serve inter-cluster central↔site messaging and are themselves scheduled for removal by the gRPC transport migration (`docs/plans/2026-07-22-clusterclient-to-grpc-plan.md`).
|
||||
|
||||
## Connection Failure Behavior
|
||||
|
||||
|
||||
@@ -125,7 +125,11 @@ The Host bootstraps the Akka.NET actor system from a **hand-assembled, injection
|
||||
|
||||
### REQ-HOST-6a: ClusterClientReceptionist (Central Only)
|
||||
|
||||
On central nodes, the Host must configure the Akka.NET **ClusterClientReceptionist** and register the ManagementActor with it. This allows external processes (e.g., the CLI) to discover and communicate with the ManagementActor via ClusterClient without joining the cluster as full members. The receptionist is started as part of the Akka.NET bootstrap (REQ-HOST-6) on central nodes only.
|
||||
On central nodes, the Host must configure the Akka.NET **ClusterClientReceptionist** and register the **CentralCommunicationActor** with it, so that site clusters' ClusterClients can reach the central command/control endpoint without joining the central cluster.
|
||||
|
||||
**The ManagementActor is NOT registered with the receptionist** (removed 2026-07-22). That registration was written for an out-of-cluster CLI that was never built: the shipped CLI speaks HTTP Basic to the central `/management` endpoints, which ask the ManagementActor **in-process** via `ManagementActorHolder` (`ManagementEndpoints.cs`). No sender to `/user/management` existed anywhere in the repo, so the registration only widened the cluster-client surface for nothing. The actor itself still runs at `/user/management`; only its cross-boundary advertisement is gone.
|
||||
|
||||
> **Migration note.** This receptionist registration — and the `CentralCommunicationActor` one that remains — are scheduled for deletion once the site↔central transport moves to gRPC. See `docs/plans/2026-07-22-clusterclient-to-grpc-plan.md`.
|
||||
|
||||
### REQ-HOST-7: ASP.NET Web Endpoints
|
||||
|
||||
|
||||
@@ -24,7 +24,7 @@ Central cluster only. The ManagementActor runs as a plain actor on **every** cen
|
||||
|
||||
### ManagementActor
|
||||
|
||||
The central actor that receives and processes all management commands. Registered at a well-known actor path (`/user/management`) and with ClusterClientReceptionist.
|
||||
The central actor that receives and processes all management commands. Created at the well-known actor path (`/user/management`) and handed to `ManagementActorHolder`, which is how every in-process caller (notably `ManagementEndpoints`) reaches it. It is **not** advertised via ClusterClientReceptionist — see the note under "ManagementActor and ClusterClient" in Component-Communication.
|
||||
|
||||
### ManagementEndpoints
|
||||
|
||||
|
||||
Reference in New Issue
Block a user