feat(sitestream): central transient per-site live alarm cache w/ seed-then-stream + failover (plan #10 T4)

Adds the active-central-node, in-memory, reference-counted per-site live alarm
cache backing the operator Alarm Summary page. No persisted central alarm store
([PERM]) — no EF entity/table/migration/DbSet.

- ISiteAlarmLiveCache (new): singleton seam — Subscribe(siteId, onChanged) ->
  IDisposable (ref-counted, linger stop on last-out), GetCurrentAlarms, IsLive.
- SiteAlarmAggregatorActor (new): one per site. Seed-then-stream ordering copied
  from DebugStreamBridgeActor — open the site-wide alarm-only gRPC stream first,
  buffer live deltas while the snapshot fan-out runs, flush with per-key dedup
  (AlarmKey = InstanceUniqueName|AlarmName|SourceReference), then live pass-through
  into an in-memory dict. Placeholder rows seeded from the snapshot and never
  expected on the stream; a real-alarm delta (distinct key) never wipes them.
  NodeA<->NodeB reconnect (retry budget + stability window), reconnect re-seeds,
  periodic reconcile (authoritative clear-and-rebuild) corrects instance-set drift,
  and a budget-exhausted stream self-heals on the next reconcile tick.
- SiteAlarmLiveCacheService (new): DI singleton facade — viewer reference-counting,
  linger-delayed last-out stop (version + TryRemove(ref) race guards), the
  snapshot fan-out seed (RequestDebugSnapshotAsync per Enabled instance, capped),
  bounded start-retry self-heal on transient start failure, and the immutable
  published-snapshot store the page reads. Cache mutated only on the actor thread;
  viewer callbacks invoked outside the lock.
- CommunicationOptions: LiveAlarmCacheLinger (30s), LiveAlarmCacheReconcileInterval
  (60s), LiveAlarmCacheSeedConcurrency (8), LiveAlarmCacheMaxSubscribersPerSite
  (200). Task 6 formalizes eager validation + telemetry.
- DI registration + AkkaHostedService SetActorSystem wiring on the active central node.
- Tests: 14 actor (seed/stream ordering, dedup, native-alarm parity, placeholder
  coherence, reconnect re-seed, reconcile replace, self-heal, stop) + 6 service
  (shared start, linger stop, resubscribe cancels stop, idempotent dispose, unknown
  site, transient-start self-heal). Code-reviewer pass: no persistence, no Critical;
  both Important findings addressed.

Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
This commit is contained in:
Joseph Doherty
2026-07-10 12:18:02 -04:00
parent 6d5ecc92a5
commit 696a4ffea2
8 changed files with 1739 additions and 0 deletions
@@ -22,6 +22,13 @@ public static class ServiceCollectionExtensions
services.AddSingleton<SiteStreamGrpcClientFactory>();
services.AddSingleton<DebugStreamService>();
// Aggregated live alarm cache (plan #10, Task 4): transient, in-memory, shared
// per-site aggregator backing the operator Alarm Summary page. Singleton so every
// Blazor circuit shares one aggregator per site; started/stopped by viewer
// reference-count. No persistence (locked [PERM]: no central alarm store).
services.AddSingleton<SiteAlarmLiveCacheService>();
services.AddSingleton<ISiteAlarmLiveCache>(sp => sp.GetRequiredService<SiteAlarmLiveCacheService>());
// Startup reconciliation handler — scoped (holds scoped repositories), resolved
// per-request by CentralCommunicationActor inside a DI scope. Harmless on site
// hosts: only the central actor ever resolves it.