perf(misc): cached hot-path lookups, bounded observer queue, alarm-priority stream path

WP2.6 (arch-review remediation, cross-cutting misc):
- SiteExternalSystemRepository: name/ID-indexed ExternalSystemDefinitionCache replaces
  the fetch-all + reverse-map scan on every by-ID/method lookup; loaded once per
  redeploy, invalidated by DeploymentManagerActor after HandleDeployArtifacts applies
  external-system changes. Static JsonSerializerOptions for method-list parsing.
- Inbound API: short-TTL ApiMethodCache fronts the per-request ApiMethod repository
  fetch; invalidated by name via the existing ScriptArtifactChangeSubscriber/
  IScriptArtifactChangeBus pipeline, self-healing via TTL for changes the bus
  doesn't cover (e.g. Management API edits).
- StoreAndForward: the cached-call audit-observer queue — the one unbounded channel
  left in the system — is now bounded (ObserverQueueCapacity, default 10,000) with
  DropOldest overflow and a dropped-notification counter.
- SiteStreamManager: alarm state changes now travel a dedicated publish
  source/broadcast hub, isolated from the (far higher-volume) attribute path, so an
  attribute storm can no longer evict a pending alarm transition; the alarm hand-off
  queue is bounded with a drop counter surfaced on the site health report
  (SiteStreamAlarmDropCount via the new SiteStreamAlarmDropReporter), and publishing
  is skipped entirely at zero subscribers on either path.
- CLI ManagementHttpClient: explicit 30s HttpClient.Timeout on the shared
  construction (was the 100s framework default), overridable via
  SCADABRIDGE_HTTP_TIMEOUT_SECONDS.

Deviation: the failback-probe heartbeat item is NOT included — its only viable
surface (CentralChannelProvider.cs / heartbeat consumers) lives entirely in the
Communication project, explicitly off-limits to this work package this phase.

Tests: SiteRuntime.Tests (550), InboundAPI.Tests (278), StoreAndForward.Tests (133),
CLI.Tests (390), HealthMonitoring.Tests (97) — all green after full solution build.
This commit is contained in:
Joseph Doherty
2026-08-14 20:59:43 -04:00
parent ee193cd2bb
commit a212283104
32 changed files with 1361 additions and 111 deletions
@@ -9,13 +9,53 @@ public class ManagementHttpClient : IDisposable
private readonly HttpClient _httpClient;
/// <summary>
/// Initializes a new instance of the <see cref="ManagementHttpClient"/> class.
/// WP2.6e (arch-review misc — CLI HttpClient timeout): default overall
/// <see cref="HttpClient.Timeout"/> for the shared client construction (30 s). This
/// bounds a hung/black-holed connection — before this, the public constructor left
/// <see cref="HttpClient.Timeout"/> at its framework default (100 s), silently longer
/// than most CLI callers' own per-request <c>TimeSpan timeout</c> argument
/// (<see cref="SendCommandAsync"/>/<see cref="SendGetAsync"/>/<see cref="SendPostAsync"/>
/// already bound each call via their own <see cref="CancellationTokenSource"/>, but a
/// connection attempt that never completes at all — no response headers, ever — is
/// bounded by <see cref="HttpClient.Timeout"/> instead, since that governs the whole
/// request/response including connect). Config-overridable via the
/// <c>SCADABRIDGE_HTTP_TIMEOUT_SECONDS</c> environment variable, consistent with how
/// every other CLI setting is overridden (see <see cref="CliConfig"/>) — kept
/// self-contained here (no <see cref="CliConfig"/>/command-file plumbing) since CLI
/// commands are owned by a separate work package this phase.
/// </summary>
public static readonly TimeSpan DefaultTimeout = TimeSpan.FromSeconds(30);
/// <summary>Test seam (WP2.6e) — the effective <see cref="HttpClient.Timeout"/> this instance was constructed with.</summary>
internal TimeSpan EffectiveTimeout { get; }
/// <summary>
/// Resolves the effective default timeout: the <c>SCADABRIDGE_HTTP_TIMEOUT_SECONDS</c>
/// environment variable when set to a positive integer, otherwise <see cref="DefaultTimeout"/>.
/// </summary>
private static TimeSpan ResolveDefaultTimeout()
{
var env = Environment.GetEnvironmentVariable("SCADABRIDGE_HTTP_TIMEOUT_SECONDS");
if (!string.IsNullOrWhiteSpace(env)
&& int.TryParse(env, out var seconds)
&& seconds > 0)
{
return TimeSpan.FromSeconds(seconds);
}
return DefaultTimeout;
}
/// <summary>
/// Initializes a new instance of the <see cref="ManagementHttpClient"/> class, with
/// <see cref="HttpClient.Timeout"/> set to <see cref="ResolveDefaultTimeout"/>
/// (30 s, or the <c>SCADABRIDGE_HTTP_TIMEOUT_SECONDS</c> override).
/// </summary>
/// <param name="baseUrl">The base URL for the management API.</param>
/// <param name="username">The username for HTTP Basic authentication.</param>
/// <param name="password">The password for HTTP Basic authentication.</param>
public ManagementHttpClient(string baseUrl, string username, string password)
: this(new HttpClient(), baseUrl, username, password)
: this(new HttpClient { Timeout = ResolveDefaultTimeout() }, baseUrl, username, password)
{
}
@@ -31,6 +71,9 @@ public class ManagementHttpClient : IDisposable
internal ManagementHttpClient(HttpClient httpClient, string baseUrl, string username, string password)
{
_httpClient = httpClient;
// Test seam (WP2.6e): exposes the constructed HttpClient's effective Timeout
// without requiring reflection.
EffectiveTimeout = httpClient.Timeout;
_httpClient.BaseAddress = new Uri(baseUrl.TrimEnd('/') + "/");
var credentials = Convert.ToBase64String(Encoding.UTF8.GetBytes($"{username}:{password}"));
_httpClient.DefaultRequestHeaders.Authorization =