fix(host): follow an in-place site PSK rotation without a central restart (ScadaBridge#96)

SitePskProvider cached each site's gRPC preshared key forever; only site removal
invalidated it, so rotating SB-GRPC-PSK-{site} broke the central<->site link on
both central nodes until they restarted. The cached key is now dropped when:

- the secret is written through /admin/secrets: central replaces the secrets
  library's ISecretCacheInvalidator with SitePskRotationInvalidator, which also
  publishes SitePskInvalidated on a new DistributedPubSub topic
  (site-psk-invalidation) so the peer central node drops its copy and its
  resolver cache entry too (SitePskInvalidationSubscriber);
- a site rejects a central->site call with PermissionDenied/Unauthenticated:
  WithSiteCredentials wraps every per-site channel's handler in
  PskRejectionRefreshHandler, so the next call re-reads the key (covers
  out-of-process writes such as the secret CLI);
- a site->central call presents a token that does not match the cached key:
  CentralControlAuthInterceptor invalidates and re-reads once before refusing;
- ScadaBridge:Communication:SitePsks reloads.

Invalidation is case-insensitive because SecretName lowercases names. Rotation
runbook (topology-guide), Component-Communication/Host and the #85 cache audit
(D2) updated.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 14:27:14 -05:00
parent 7f479f3a5c
commit 3a31dd2de7
15 changed files with 666 additions and 26 deletions
@@ -204,6 +204,18 @@ because sites are created at runtime and cannot be enumerated in configuration a
`ControlPlaneCredentials` binds the key to the channel as `CallCredentials`, so no individual call
site can omit it.
Central caches each resolved key, and drops the entry (Gitea #96) when: the site is removed; the
secret `SB-GRPC-PSK-<siteId>` is written through central's secrets write path (`/admin/secrets`) —
`SitePskRotationInvalidator` wraps the secrets library's `ISecretCacheInvalidator` and also
broadcasts a `SitePskInvalidated` on the central `DistributedPubSub` topic `site-psk-invalidation`
so the peer central node drops its copy too; the site rejects a call with `PermissionDenied` /
`Unauthenticated` (`PskRejectionRefreshHandler`, wrapped around every per-site channel by
`WithSiteCredentials`, so the next call re-reads the key — this covers keys rotated outside the
process, e.g. by the `secret` CLI, after at most the resolver's `Secrets:ResolveCacheTtl`); a
site's inbound call to central presents a token that does not match the cached key
(`CentralControlAuthInterceptor` drops the entry and re-reads once before refusing); or
`ScadaBridge:Communication:SitePsks` reloads. An in-place rotation therefore needs no central restart.
This is distinct from `LocalDb:Replication:ApiKey`, which gates
`/localdb_sync.v1.LocalDbSync/` on the same listener via its own interceptor: that key
authenticates the *pair partner* for database replication, a different trust relationship, and the
+1
View File
@@ -351,4 +351,5 @@ built)* section for where the shipped code deviates from the original design).
- **Port options come from the bound `NodeOptions`.** `Program.cs` binds `ScadaBridge:Node` once and reads `GrpcPort` / `CentralGrpcPort` / `MetricsPort` from it; the 8083/8084 defaults live only in `NodeOptions`. Central's dedicated metrics listener remains an explicit *presence* check on the raw key (a defaulted read would bind a new port on every existing central node).
- **Inbound method warm-up is a hosted service** (`InboundMethodWarmupService`, registered after `AkkaHostedService`), not a pre-`RunAsync` loop — see Component-InboundAPI.md.
- **`SitePskProvider` distinguishes a secret-store fault from a missing secret.** A store fault logs a Warning with the exception and throws `SecretStoreUnavailableException` (not cached); only a genuinely absent/empty/tombstoned secret produces the "seed `SB-GRPC-PSK-{site}`" message.
- **`SitePskProvider`'s cache follows an in-place rotation (Gitea #96).** Central replaces the secrets library's `ISecretCacheInvalidator` with `SitePskRotationInvalidator` (`AddSitePskRotationInvalidation`), which drops the site's cached key when `SB-GRPC-PSK-{site}` is written and publishes `SitePskInvalidated` on the `site-psk-invalidation` `DistributedPubSub` topic; `AkkaHostedService.RegisterCentralActors` starts the `SitePskInvalidationSubscriber` that applies the peer's broadcasts. A site's auth rejection and a `SitePsks` reload also drop the cache — see Component-Communication.md → *Authentication*.
- Central registers `ExternalSystemGatewayOptions.DefinitionCacheTtl = 10 s` ahead of `AddExternalSystemGateway()` so an explicit config value wins; sites leave it off.