fix(central): audit per-node central caches; management API drops ApiMethodCache entries (ScadaBridge#85)

Audit (docs/plans/2026-11-09-central-cache-staleness-audit.md) of every central in-process
cache holding configuration or secret state, with its staleness bound on the peer central
node. Every config cache with peer staleness is TTL-bounded (5-60 s) and Traefik routes to the
active node only, so no peer case was clearly a bug and no cross-node broadcast is added; the
doc names the existing DistributedPubSub mediator (health replication topic) as the mechanism
if one is wanted. Owner decisions listed: RoleMappingCache 30 s revocation lag on the peer,
SitePskProvider caching a PSK forever across an in-place rotation (both nodes), and the #83
ExternalDefinitionLookupCache 10 s bound.

Fixed: ManagementActor API-method create/update/delete invalidated only the compiled handler,
not ApiMethodCache, so the node that made the change kept serving the pre-change row for up to
the 5 s TTL (a deleted method stayed callable; a cached not-found hid a new one). The handlers
now drop the entry post-commit, like the role-mapping cache.

Tests: ApiMethodCacheInvalidationTests (create/update/delete fail without the fix; a
negative control checks other methods stay cached).

(ScadaBridge#85)

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 13:43:44 -05:00
parent 23c35df234
commit a16b7b4e65
6 changed files with 223 additions and 9 deletions
+1 -1
View File
@@ -175,7 +175,7 @@ The `sbk_<keyId>_<secret>` design already supports zero-downtime rotation — a
## Script Compilation & Hot-Reload
API method scripts are pre-compiled in the **background after startup** by `InboundMethodWarmupService` (a hosted service registered after `AkkaHostedService`, so it never delays the cluster join or `/health/*`): it waits for `ApplicationStarted`, then — only once this node is the active central node (`IActiveNodeGate`; a standby pays nothing until a failover promotes it) — loads all method definitions and compiles them into in-memory delegates. Lazy compile + the per-request revision check guarantee correctness regardless. Compiles are **single-flight per method name + script revision**: N concurrent first callers share one compile (and one non-collectible loaded assembly). `InboundScriptExecutor.CompileAndRegister(ApiMethod, out IReadOnlyList<string> errors)` compiles once and returns the diagnostics, so a save needs no separate validate pass; the management Create/Update path uses its sibling `CompileForPublish(method, out errors, out publish)`, which compiles once for the `compileWarning` and hands back a `publish` action queued as the post-commit registration — one compile (one pinned assembly) per save even inside a configuration transaction (review 2026-10-21 05/M1). `ApiMethodCache` prunes expired entries on a miss once large (scopes are exact method names — pinned by test — so growth is bounded).
API method scripts are pre-compiled in the **background after startup** by `InboundMethodWarmupService` (a hosted service registered after `AkkaHostedService`, so it never delays the cluster join or `/health/*`): it waits for `ApplicationStarted`, then — only once this node is the active central node (`IActiveNodeGate`; a standby pays nothing until a failover promotes it) — loads all method definitions and compiles them into in-memory delegates. Lazy compile + the per-request revision check guarantee correctness regardless. Compiles are **single-flight per method name + script revision**: N concurrent first callers share one compile (and one non-collectible loaded assembly). `InboundScriptExecutor.CompileAndRegister(ApiMethod, out IReadOnlyList<string> errors)` compiles once and returns the diagnostics, so a save needs no separate validate pass; the management Create/Update path uses its sibling `CompileForPublish(method, out errors, out publish)`, which compiles once for the `compileWarning` and hands back a `publish` action queued as the post-commit registration — one compile (one pinned assembly) per save even inside a configuration transaction (review 2026-10-21 05/M1). `ApiMethodCache` prunes expired entries on a miss once large (scopes are exact method names — pinned by test — so growth is bounded). A Management API/UI create, update or delete drops that method's `ApiMethodCache` entry post-commit on the node that made the change; the peer central node converges within `ApiMethodCacheTtl` (default 5 s) — Gitea #85, audit `docs/plans/2026-11-09-central-cache-staleness-audit.md`.
### Update Workflow