Joseph Doherty
2ee84af1c0
Phase 0 of the ClusterClient→gRPC migration
(docs/plans/2026-07-22-clusterclient-to-grpc-plan.md). Standalone hardening: it
closes a gap that exists today and is a precondition for moving command/control
onto gRPC in later phases.
T0.1 — delete the ManagementActor ClusterClientReceptionist registration.
It was built for an out-of-cluster CLI that was never written: the shipped CLI
speaks HTTP Basic to /management, which asks the actor in-process through
ManagementActorHolder. Nothing in the repo ever sent to /user/management. The
actor still runs there; only the cross-boundary advertisement is gone. Six
documents claimed the CLI used ClusterClient — including the CLI's own README
"Architecture Notes" — and are corrected here rather than left to rot.
T0.2 — record, do not port, the dead integration-routing path.
IntegrationCallRequest is unwired at BOTH ends: RouteIntegrationCallAsync has
zero callers anywhere, and RegisterLocalHandler(Integration, …) appears only in
a test, so production always answers "Integration handler not available". It is
excluded from the gRPC contract (28 of 29 commands migrate) rather than
enshrined on an additive-only wire format, and deleting it during a
transport migration would mix a behavioural change into a change whose whole
value is that behaviour is identical. See
docs/known-issues/2026-07-22-integration-call-routing-is-dead-code.md.
T0.3 — preshared-key authentication on SiteStreamService.
The service shipped with no auth at all: plaintext h2c, no interceptor, so
anything that could reach a site node's :8083 could open a live data stream or
read audit rows back via PullAuditEvents/PullSiteCalls. ControlPlaneAuthInterceptor
now gates /sitestream.SiteStreamService/ — modeled on LocalDbSyncAuthInterceptor
(constant-time compare, fail-closed, PermissionDenied) but gating a SET of
service prefixes so phases 1A/1B add services rather than interceptors. LocalDb
sync keeps its own separate key: it authenticates the pair partner, not central,
and collapsing the two would make a site's central-facing key also admit writes
into its database.
Keys are per site (SB-GRPC-PSK-<siteId>), never fleet-wide, so a compromised
site yields only its own. Central attaches them through ControlPlaneCredentials,
which binds CallCredentials to the channel — covering unary and streaming
uniformly, and letting the key resolve asynchronously, which a client
interceptor could not do without blocking. All three central→site channel
creation sites go through it (SiteStreamGrpcClient and both audit pull invokers);
the pull invokers' channel caches are re-keyed by (site, endpoint) because
credentials are per-site and bound to the channel.
Two decisions beyond the plan:
* StartupValidator now requires GrpcPsk on Site nodes. The plan specified only
the runtime gate, but fail-closed with no boot check produces a node that
joins, answers heartbeats and reports healthy while refusing every stream,
audit pull and telemetry ingest — silent and total. Same reasoning as the
existing inbound API-key pepper rule.
* Added Communication:SitePsks as a central-side key map. The plan assumed
central would read the store, seeded via a dev KEK; the docker rig
deliberately boots with no master key, so store-only resolution would leave
it unable to dial its own sites. The store stays primary — it is the only
source that can serve a site added at runtime — with the map covering
key-less hosts and one-off pins. Neither source falling back to
"unauthenticated" is the invariant.
T0.4 — dev keys on both rigs and tests.
34 tests. The seven that matter most exercise a real in-process gRPC stack over
TestServer: the unit tests on either side of the wire would both stay green if
the halves disagreed, and gRPC refuses call credentials on a plaintext channel
by default — the UnsafeUseInsecureChannelCallCredentials opt-in is only provable
by making a real call. They confirm correct key passes on unary AND streaming,
wrong key and no-credentials both get PermissionDenied, and an unresolvable key
fails the call with nothing reaching the service.
OPERATIONAL: a site node upgraded to this build without a key will not boot.
That includes the gitignored deploy/wonder-app-vd03/ overlay.
232 lines
11 KiB
YAML
232 lines
11 KiB
YAML
services:
|
|
central-a:
|
|
image: scadabridge:latest
|
|
# CoordinatedShutdown needs cluster-leave (15s budget) + cluster-exiting +
|
|
# actor-system-terminate + Serilog flush; the 10s SIGTERM default SIGKILLed
|
|
# mid-drain, turning every redeploy into the crash path (review 01 [Medium]).
|
|
stop_grace_period: 30s
|
|
container_name: scadabridge-central-a
|
|
environment:
|
|
SCADABRIDGE_CONFIG: Central
|
|
ASPNETCORE_ENVIRONMENT: Development
|
|
ASPNETCORE_URLS: "http://+:5000"
|
|
# DEV-ONLY local-cluster value — NOT a real secret. The Auth/Config normalization
|
|
# (2026-06-03) made ScadaBridge:InboundApi:ApiKeyPepper a hard Central startup
|
|
# requirement (>=16 chars, per-environment). Real deployments inject a true secret
|
|
# out-of-band (env/secret store), never from source control — see
|
|
# docs/operations/inbound-api-key-reissue.md. Both Central nodes share one pepper.
|
|
ScadaBridge__InboundApi__ApiKeyPepper: "dev-only-insecure-pepper-docker-cluster-0001"
|
|
# DEV-ONLY local-cluster secrets — NOT real secrets. Relocated out of the mounted
|
|
# appsettings.Central.json (G-4 T5 secrets cleanup) so that app-config file carries no
|
|
# plaintext credentials. Env overrides layer over JSON BEFORE the ${secret:} pre-host
|
|
# expander runs, so the dev cluster boots with no KEK/secret store. The SQL password is
|
|
# the same value already committed for the sibling scadabridge-mssql container
|
|
# (infra/docker-compose.yml MSSQL_SA_PASSWORD) — consolidation, not new exposure. Real
|
|
# deployments use ${secret:} tokens (src/.../appsettings.Central.json) + a seeded store.
|
|
ScadaBridge__Database__ConfigurationDb: "Server=scadabridge-mssql,1433;Database=ScadaBridgeConfig;User Id=scadabridge_app;Password=ScadaBridge_Dev1#;TrustServerCertificate=true"
|
|
ScadaBridge__Database__MachineDataDb: "Server=scadabridge-mssql,1433;Database=ScadaBridgeMachineData;User Id=scadabridge_app;Password=ScadaBridge_Dev1#;TrustServerCertificate=true"
|
|
ScadaBridge__Security__Ldap__ServiceAccountPassword: "serviceaccount123"
|
|
ScadaBridge__Security__JwtSigningKey: "scadabridge-dev-jwt-signing-key-must-be-at-least-32-characters-long"
|
|
# DEV-ONLY gRPC control-plane preshared keys, one per site — NOT real secrets.
|
|
# Central verifies/presents these; each site node carries the same value as
|
|
# ScadaBridge:Communication:GrpcPsk in its mounted appsettings.Site.json. Kept as
|
|
# env overrides (not in the mounted central appsettings) so that file stays free of
|
|
# plaintext credentials. Production instead seeds SB-GRPC-PSK-<siteId> into the
|
|
# secret store, which is also the only source that can serve a site added at runtime.
|
|
ScadaBridge__Communication__SitePsks__site-a: "dev-grpc-psk-docker-site-a"
|
|
ScadaBridge__Communication__SitePsks__site-b: "dev-grpc-psk-docker-site-b"
|
|
ScadaBridge__Communication__SitePsks__site-c: "dev-grpc-psk-docker-site-c"
|
|
ports:
|
|
- "9001:5000" # Web UI + Inbound API
|
|
- "9011:8081" # Akka remoting (host access for CLI/debugging)
|
|
volumes:
|
|
- ./central-node-a/appsettings.Central.json:/app/appsettings.Central.json:ro
|
|
- ./central-node-a/logs:/app/logs
|
|
networks:
|
|
- scadabridge-net
|
|
restart: unless-stopped
|
|
|
|
central-b:
|
|
image: scadabridge:latest
|
|
# CoordinatedShutdown needs cluster-leave (15s budget) + cluster-exiting +
|
|
# actor-system-terminate + Serilog flush; the 10s SIGTERM default SIGKILLed
|
|
# mid-drain, turning every redeploy into the crash path (review 01 [Medium]).
|
|
stop_grace_period: 30s
|
|
container_name: scadabridge-central-b
|
|
environment:
|
|
SCADABRIDGE_CONFIG: Central
|
|
ASPNETCORE_ENVIRONMENT: Development
|
|
ASPNETCORE_URLS: "http://+:5000"
|
|
# DEV-ONLY local-cluster value — NOT a real secret. The Auth/Config normalization
|
|
# (2026-06-03) made ScadaBridge:InboundApi:ApiKeyPepper a hard Central startup
|
|
# requirement (>=16 chars, per-environment). Real deployments inject a true secret
|
|
# out-of-band (env/secret store), never from source control — see
|
|
# docs/operations/inbound-api-key-reissue.md. Both Central nodes share one pepper.
|
|
ScadaBridge__InboundApi__ApiKeyPepper: "dev-only-insecure-pepper-docker-cluster-0001"
|
|
# DEV-ONLY local-cluster secrets — NOT real secrets. Relocated out of the mounted
|
|
# appsettings.Central.json (G-4 T5 secrets cleanup) so that app-config file carries no
|
|
# plaintext credentials. Env overrides layer over JSON BEFORE the ${secret:} pre-host
|
|
# expander runs, so the dev cluster boots with no KEK/secret store. The SQL password is
|
|
# the same value already committed for the sibling scadabridge-mssql container
|
|
# (infra/docker-compose.yml MSSQL_SA_PASSWORD) — consolidation, not new exposure. Real
|
|
# deployments use ${secret:} tokens (src/.../appsettings.Central.json) + a seeded store.
|
|
ScadaBridge__Database__ConfigurationDb: "Server=scadabridge-mssql,1433;Database=ScadaBridgeConfig;User Id=scadabridge_app;Password=ScadaBridge_Dev1#;TrustServerCertificate=true"
|
|
ScadaBridge__Database__MachineDataDb: "Server=scadabridge-mssql,1433;Database=ScadaBridgeMachineData;User Id=scadabridge_app;Password=ScadaBridge_Dev1#;TrustServerCertificate=true"
|
|
ScadaBridge__Security__Ldap__ServiceAccountPassword: "serviceaccount123"
|
|
ScadaBridge__Security__JwtSigningKey: "scadabridge-dev-jwt-signing-key-must-be-at-least-32-characters-long"
|
|
# DEV-ONLY gRPC control-plane preshared keys, one per site — NOT real secrets.
|
|
# Central verifies/presents these; each site node carries the same value as
|
|
# ScadaBridge:Communication:GrpcPsk in its mounted appsettings.Site.json. Kept as
|
|
# env overrides (not in the mounted central appsettings) so that file stays free of
|
|
# plaintext credentials. Production instead seeds SB-GRPC-PSK-<siteId> into the
|
|
# secret store, which is also the only source that can serve a site added at runtime.
|
|
ScadaBridge__Communication__SitePsks__site-a: "dev-grpc-psk-docker-site-a"
|
|
ScadaBridge__Communication__SitePsks__site-b: "dev-grpc-psk-docker-site-b"
|
|
ScadaBridge__Communication__SitePsks__site-c: "dev-grpc-psk-docker-site-c"
|
|
ports:
|
|
- "9002:5000" # Web UI + Inbound API
|
|
- "9012:8081" # Akka remoting
|
|
volumes:
|
|
- ./central-node-b/appsettings.Central.json:/app/appsettings.Central.json:ro
|
|
- ./central-node-b/logs:/app/logs
|
|
networks:
|
|
- scadabridge-net
|
|
restart: unless-stopped
|
|
|
|
site-a-a:
|
|
image: scadabridge:latest
|
|
# CoordinatedShutdown needs cluster-leave (15s budget) + cluster-exiting +
|
|
# actor-system-terminate + Serilog flush; the 10s SIGTERM default SIGKILLed
|
|
# mid-drain, turning every redeploy into the crash path (review 01 [Medium]).
|
|
stop_grace_period: 30s
|
|
container_name: scadabridge-site-a-a
|
|
environment:
|
|
SCADABRIDGE_CONFIG: Site
|
|
ports:
|
|
- "9021:8082" # Akka remoting (host access for debugging)
|
|
- "9023:8083" # gRPC streaming
|
|
volumes:
|
|
- ./site-a-node-a/appsettings.Site.json:/app/appsettings.Site.json:ro
|
|
- ./site-a-node-a/data:/app/data
|
|
- ./site-a-node-a/logs:/app/logs
|
|
networks:
|
|
- scadabridge-net
|
|
restart: unless-stopped
|
|
|
|
site-a-b:
|
|
image: scadabridge:latest
|
|
# CoordinatedShutdown needs cluster-leave (15s budget) + cluster-exiting +
|
|
# actor-system-terminate + Serilog flush; the 10s SIGTERM default SIGKILLed
|
|
# mid-drain, turning every redeploy into the crash path (review 01 [Medium]).
|
|
stop_grace_period: 30s
|
|
container_name: scadabridge-site-a-b
|
|
environment:
|
|
SCADABRIDGE_CONFIG: Site
|
|
ports:
|
|
- "9022:8082" # Akka remoting
|
|
- "9024:8083" # gRPC streaming
|
|
volumes:
|
|
- ./site-a-node-b/appsettings.Site.json:/app/appsettings.Site.json:ro
|
|
- ./site-a-node-b/data:/app/data
|
|
- ./site-a-node-b/logs:/app/logs
|
|
networks:
|
|
- scadabridge-net
|
|
restart: unless-stopped
|
|
|
|
site-b-a:
|
|
image: scadabridge:latest
|
|
# CoordinatedShutdown needs cluster-leave (15s budget) + cluster-exiting +
|
|
# actor-system-terminate + Serilog flush; the 10s SIGTERM default SIGKILLed
|
|
# mid-drain, turning every redeploy into the crash path (review 01 [Medium]).
|
|
stop_grace_period: 30s
|
|
container_name: scadabridge-site-b-a
|
|
environment:
|
|
SCADABRIDGE_CONFIG: Site
|
|
ports:
|
|
- "9031:8082" # Akka remoting
|
|
- "9033:8083" # gRPC streaming
|
|
volumes:
|
|
- ./site-b-node-a/appsettings.Site.json:/app/appsettings.Site.json:ro
|
|
- ./site-b-node-a/data:/app/data
|
|
- ./site-b-node-a/logs:/app/logs
|
|
networks:
|
|
- scadabridge-net
|
|
restart: unless-stopped
|
|
|
|
site-b-b:
|
|
image: scadabridge:latest
|
|
# CoordinatedShutdown needs cluster-leave (15s budget) + cluster-exiting +
|
|
# actor-system-terminate + Serilog flush; the 10s SIGTERM default SIGKILLed
|
|
# mid-drain, turning every redeploy into the crash path (review 01 [Medium]).
|
|
stop_grace_period: 30s
|
|
container_name: scadabridge-site-b-b
|
|
environment:
|
|
SCADABRIDGE_CONFIG: Site
|
|
ports:
|
|
- "9032:8082" # Akka remoting
|
|
- "9034:8083" # gRPC streaming
|
|
volumes:
|
|
- ./site-b-node-b/appsettings.Site.json:/app/appsettings.Site.json:ro
|
|
- ./site-b-node-b/data:/app/data
|
|
- ./site-b-node-b/logs:/app/logs
|
|
networks:
|
|
- scadabridge-net
|
|
restart: unless-stopped
|
|
|
|
site-c-a:
|
|
image: scadabridge:latest
|
|
# CoordinatedShutdown needs cluster-leave (15s budget) + cluster-exiting +
|
|
# actor-system-terminate + Serilog flush; the 10s SIGTERM default SIGKILLed
|
|
# mid-drain, turning every redeploy into the crash path (review 01 [Medium]).
|
|
stop_grace_period: 30s
|
|
container_name: scadabridge-site-c-a
|
|
environment:
|
|
SCADABRIDGE_CONFIG: Site
|
|
ports:
|
|
- "9041:8082" # Akka remoting
|
|
- "9043:8083" # gRPC streaming
|
|
volumes:
|
|
- ./site-c-node-a/appsettings.Site.json:/app/appsettings.Site.json:ro
|
|
- ./site-c-node-a/data:/app/data
|
|
- ./site-c-node-a/logs:/app/logs
|
|
networks:
|
|
- scadabridge-net
|
|
restart: unless-stopped
|
|
|
|
site-c-b:
|
|
image: scadabridge:latest
|
|
# CoordinatedShutdown needs cluster-leave (15s budget) + cluster-exiting +
|
|
# actor-system-terminate + Serilog flush; the 10s SIGTERM default SIGKILLed
|
|
# mid-drain, turning every redeploy into the crash path (review 01 [Medium]).
|
|
stop_grace_period: 30s
|
|
container_name: scadabridge-site-c-b
|
|
environment:
|
|
SCADABRIDGE_CONFIG: Site
|
|
ports:
|
|
- "9042:8082" # Akka remoting
|
|
- "9044:8083" # gRPC streaming
|
|
volumes:
|
|
- ./site-c-node-b/appsettings.Site.json:/app/appsettings.Site.json:ro
|
|
- ./site-c-node-b/data:/app/data
|
|
- ./site-c-node-b/logs:/app/logs
|
|
networks:
|
|
- scadabridge-net
|
|
restart: unless-stopped
|
|
|
|
traefik:
|
|
image: traefik:v3.4
|
|
container_name: scadabridge-traefik
|
|
ports:
|
|
- "9000:80" # Central load-balanced entrypoint
|
|
- "8180:8080" # Traefik dashboard
|
|
volumes:
|
|
- ./traefik/traefik.yml:/etc/traefik/traefik.yml:ro
|
|
- ./traefik/dynamic.yml:/etc/traefik/dynamic.yml:ro
|
|
networks:
|
|
- scadabridge-net
|
|
restart: unless-stopped
|
|
|
|
networks:
|
|
scadabridge-net:
|
|
external: true
|