feat(sitestream): validate live-alarm-cache options + active-aggregator/reconnect telemetry (plan #10 T6)

Extend CommunicationOptionsValidator with eager bounds for the four T4
live-alarm-cache options (linger >= 0, reconcile > 0, seed concurrency 1..64,
subscribers-per-site >= 1). Enforce the per-site viewer cap fail-safe in
SiteAlarmLiveCacheService.Subscribe (reject excess viewers with a no-op
disposable rather than growing the list or throwing into the Blazor render
path). Surface two telemetry instruments on the existing ScadaBridgeTelemetry
meter: an active-aggregator observable gauge and a reconnect counter, wired from
the aggregator actor's PreStart/PostStop and its NodeA<->NodeB flip /
reconcile-driven reopen.

Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
This commit is contained in:
Joseph Doherty
2026-07-10 12:33:40 -04:00
parent b91ed3c840
commit c4f97d0e87
8 changed files with 263 additions and 5 deletions
@@ -92,12 +92,18 @@ public sealed class SiteAlarmLiveCacheService : ISiteAlarmLiveCache
// A pending linger stop is now moot — a viewer arrived. Invalidate it.
entry.CancelLinger();
// Enforce the per-site viewer cap — fail SAFE: reject the excess viewer with a
// no-op disposable rather than throwing (this runs on a Blazor render path) or
// growing the subscriber list unbounded. The shared aggregator keeps serving the
// already-registered viewers; the rejected circuit just keeps its 15s poll
// fallback. Reaching the cap almost always means a Dispose leak or a runaway page.
if (entry.Subscribers.Count >= _options.LiveAlarmCacheMaxSubscribersPerSite)
{
_logger.LogWarning(
"Site {SiteId} live alarm cache already has {Count} viewers (cap {Cap}); " +
"new viewer still registered but this indicates a leak or a very busy page",
siteId, entry.Subscribers.Count, _options.LiveAlarmCacheMaxSubscribersPerSite);
"Site {SiteId} live alarm cache is at its viewer cap ({Cap}); rejecting the new " +
"viewer (it will keep polling). This indicates a subscription leak or a very busy page.",
siteId, _options.LiveAlarmCacheMaxSubscribersPerSite);
return NoOpSubscription.Instance;
}
entry.Subscribers.Add(subscription);
@@ -446,6 +452,18 @@ public sealed class SiteAlarmLiveCacheService : ISiteAlarmLiveCache
}
}
/// <summary>
/// The handle returned when a Subscribe is rejected at the per-site viewer cap. It was
/// never registered, so <see cref="Dispose"/> is a genuine no-op — safe to dispose any
/// number of times and it never touches the site's subscriber list.
/// </summary>
private sealed class NoOpSubscription : IDisposable
{
public static readonly NoOpSubscription Instance = new();
private NoOpSubscription() { }
public void Dispose() { }
}
/// <summary>The disposable handed to a viewer; idempotent <see cref="Dispose"/> unregisters it.</summary>
private sealed class Subscription : IDisposable
{