Files
scadaproj/ZB.MOM.WW.Overview/tests/ZB.MOM.WW.Overview.Tests/BootTests.cs
T
Joseph Doherty 92997b40dc feat(overview): self-observability, config and the container image
Closes Phase 3 of the overview-dashboard plan (tasks 3.7-3.9).

Self-observability
- OverviewMetrics: an observable gauge read from the published snapshot at scrape
  time, so the metric and the page cannot disagree about what the dashboard
  believes, plus a sweep-duration histogram. Instrument names follow the family
  METRIC-CONVENTIONS spec (overview.instance.status / overview.poll.duration,
  unit s) rather than the plan's literal zb_ names; Prometheus renders them as
  overview_instance_status and overview_poll_duration_seconds. Labels are
  application/node/group - "node", not "instance", because Prometheus stamps its
  own instance label on every scraped series and a colliding metric label is
  silently renamed exported_instance. Instances never probed emit nothing: the
  enum's zero ordinal is Up, so a placeholder would publish the single most
  reassuring answer about a node nobody has heard from.
- Two readiness checks make the dashboard's own /health/ready mean something:
  registry-loaded (reports the counts it bound) and last-poll-cycle-completed,
  which catches the failure where Kestrel is fine, the page still renders, and
  every card on it is quietly frozen. The heartbeat is stamped by the sweep, not
  derived from the snapshot's timestamp - the poller publishes the registry
  before it probes anything, so a stuck poller would otherwise look healthy.
  No cycle yet is Degraded inside the grace window and Unhealthy past it, so a
  restart does not fail readiness for its first seconds and a poller that never
  started cannot hide behind "still starting".

Configuration
- appsettings.json ships tuning defaults with an EMPTY Applications list, so a
  deployment that supplies no registry fails at ConfigPreflight naming the key
  instead of booting into a dashboard of nothing.
- Three registries, because reachability genuinely differs: Development is the
  host-reachable subset (4 instances), Docker is the full 16-instance fleet by
  container DNS name. The rigs publish almost no per-node HTTP port to the host
  - OtOpcUa publishes none (only Traefik :9200, load-balanced across the central
  pair, so it cannot identify a node) and ScadaBridge keeps site :8084
  container-internal. Ports were established by probing the running rigs, not
  read off the compose files, because the two disagree: OtOpcUa site nodes serve
  health on :8080, not the :9000 their compose anchor implies.
- Real Serilog sinks, not the family's empty "Serilog": {}. AddZbSerilog drives
  sinks entirely from configuration and adds none of its own, so the empty
  section produces an app that logs NOWHERE - verified against the running
  HistorianGateway container, whose log stream is completely empty for exactly
  this reason. The HttpClient level override matters too: 16 instances on a 10s
  cadence emit several Information lines a second of pure probe noise.

Container
- HistorianGateway-pattern runtime-only image; the compose stack joins the three
  rig networks as external, so bringing the dashboard up or down cannot disturb
  a rig. The Dockerfile clears ASPNETCORE_HTTP_PORTS as well as ASPNETCORE_URLS:
  on .NET 8+ the base image declares its binding via the former, and clearing
  only the latter leaves Kestrel logging "Overriding address(es)" every boot.
  No healthcheck - aspnet:10.0 has neither curl nor wget, and adding a
  --healthcheck entry point to production code to satisfy Compose is not a
  trade worth making.

Tests
- BootTests boot the real Program.cs: every endpoint anonymous, the page
  rendering from cache while nothing it watches is reachable, the Meters
  allowlist proven by an end-to-end /metrics scrape, and both refuse-to-boot
  paths (missing registry, stale window inside the poll interval).
  The registry is supplied via environment variables, not
  ConfigureAppConfiguration, because ConfigPreflight runs before Build() - when
  the factory's callbacks are applied - and would not see them.

170 tests, 0 warnings. Verified live against the running rigs: 16 cards, 16
gauge series, page and gauge agreeing exactly.
2026-07-24 07:35:32 -04:00

194 lines
7.4 KiB
C#

using System.Net;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using ZB.MOM.WW.Overview.Observability;
using ZB.MOM.WW.Overview.Polling;
namespace ZB.MOM.WW.Overview.Tests;
/// <summary>
/// Boots the REAL <c>Program.cs</c> and asks the questions no in-process test can: does every
/// endpoint answer without a login, and does a bad registry stop the host rather than produce a
/// dashboard full of nothing.
/// </summary>
/// <remarks>
/// The registry is supplied through environment variables rather than
/// <c>ConfigureAppConfiguration</c>. <c>ConfigPreflight</c> runs against
/// <c>builder.Configuration</c> in the top-level statements, before <c>Build()</c> — which is when
/// the factory's configuration callbacks are applied — so a callback-supplied registry would arrive
/// too late for the very check these tests are about. Environment variables are read by
/// <c>CreateBuilder</c> itself, so they are in place from the first line.
///
/// <para>
/// Environment variables are process-global, so every test here lives in one class: xunit runs a
/// class's tests sequentially, which is what keeps two hosts from fighting over the same keys.
/// </para>
/// </remarks>
public sealed class BootTests : IDisposable
{
private readonly List<string> _keys = [];
private void Set(string key, string? value)
{
_keys.Add(key);
Environment.SetEnvironmentVariable(key, value);
}
/// <summary>Restores the environment so a later-running test class sees a clean process.</summary>
public void Dispose()
{
foreach (var key in _keys)
Environment.SetEnvironmentVariable(key, null);
}
/// <summary>
/// A registry with one instance pointing at a port nothing listens on. The dashboard must boot
/// and serve regardless of whether anything it watches is up — that is the entire premise of
/// rendering from cache.
/// </summary>
private void SetValidRegistry()
{
Set("ASPNETCORE_ENVIRONMENT", "Testing");
Set("Overview__PollIntervalSeconds", "10");
Set("Overview__TimeoutSeconds", "1");
Set("Overview__StaleAfterSeconds", "45");
Set("Overview__Applications__0__Name", "TestApp");
Set("Overview__Applications__0__ManagementLabel", "UI");
Set("Overview__Applications__0__Instances__0__Name", "node-a");
Set("Overview__Applications__0__Instances__0__BaseUrl", "http://127.0.0.1:1");
Set("Overview__Applications__0__Instances__0__HasActiveRole", "false");
}
private static WebApplicationFactory<Program> Factory() => new();
[Fact]
public async Task Page_IsServedAnonymously()
{
SetValidRegistry();
using var factory = Factory();
// AllowAutoRedirect off so an auth redirect would surface as a 302 here rather than being
// silently followed to a login page that then returns 200.
using var client = factory.CreateClient(new WebApplicationFactoryClientOptions
{
AllowAutoRedirect = false,
});
using var response = await client.GetAsync("/");
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
Assert.Contains("SCADA Overview", await response.Content.ReadAsStringAsync(), StringComparison.Ordinal);
}
[Fact]
public async Task Page_RendersEvenThoughNothingItWatchesIsReachable()
{
// Cache-first, stated as a test: the registry above points at a dead port, and the page
// must still paint its full layout instead of waiting on a sweep.
SetValidRegistry();
using var factory = Factory();
using var client = factory.CreateClient();
var html = await client.GetStringAsync("/");
Assert.Contains("data-testid=\"instance-card\"", html, StringComparison.Ordinal);
Assert.Contains("node-a", html, StringComparison.Ordinal);
}
[Theory]
[InlineData("/health/ready")]
[InlineData("/health/active")]
[InlineData("/healthz")]
[InlineData("/metrics")]
public async Task HealthAndMetrics_AreServedAnonymously(string path)
{
SetValidRegistry();
using var factory = Factory();
using var client = factory.CreateClient(new WebApplicationFactoryClientOptions
{
AllowAutoRedirect = false,
});
using var response = await client.GetAsync(path);
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
}
[Fact]
public async Task Ready_ReportsTheDashboardsOwnChecks()
{
SetValidRegistry();
using var factory = Factory();
using var client = factory.CreateClient();
var body = await client.GetStringAsync("/health/ready");
Assert.Contains(ObservabilityServiceCollectionExtensions.RegistryLoadedCheckName, body, StringComparison.Ordinal);
Assert.Contains(ObservabilityServiceCollectionExtensions.PollCycleCheckName, body, StringComparison.Ordinal);
}
[Fact]
public async Task Metrics_ExportsTheAppsOwnMeter()
{
// The Meters allowlist in AddZbTelemetry is silent when it is wrong: a missing name drops
// the app's whole instrument surface from /metrics with no warning anywhere. Only an
// end-to-end scrape catches that.
SetValidRegistry();
using var factory = Factory();
using var client = factory.CreateClient();
// Force a sweep so the observable gauge has a probed instance to report. The instance is
// unreachable, which is a perfectly good status to publish.
var poller = factory.Services.GetServices<IHostedService>().OfType<OverviewPollerService>().Single();
await poller.SweepAsync();
var body = await client.GetStringAsync("/metrics");
Assert.Contains("overview_instance_status", body, StringComparison.Ordinal);
Assert.Contains("node=\"node-a\"", body, StringComparison.Ordinal);
}
[Fact]
public async Task MissingRegistry_FailsToBootAndNamesTheKey()
{
// appsettings.json ships an EMPTY Applications list precisely so this is the outcome when a
// deployment forgets its registry — an empty dashboard that boots would look like a healthy
// fleet of nothing.
Set("ASPNETCORE_ENVIRONMENT", "Testing");
using var factory = Factory();
var error = await Record.ExceptionAsync(() => factory.CreateClient().GetAsync("/"));
Assert.NotNull(error);
Assert.Contains("Overview:Applications:0:Name", Flatten(error), StringComparison.Ordinal);
}
[Fact]
public async Task StaleWindowInsideThePollInterval_FailsToBootWithTheValidatorMessage()
{
// Past preflight and into the validator: a staleness window shorter than the poll interval
// would mark every instance Stale between two perfectly healthy polls.
SetValidRegistry();
Set("Overview__StaleAfterSeconds", "5");
using var factory = Factory();
var error = await Record.ExceptionAsync(() => factory.CreateClient().GetAsync("/"));
Assert.NotNull(error);
Assert.Contains("StaleAfterSeconds", Flatten(error), StringComparison.Ordinal);
}
private static string Flatten(Exception exception)
{
var text = new System.Text.StringBuilder();
for (var current = exception; current is not null; current = current.InnerException)
text.AppendLine(current.Message);
return text.ToString();
}
}