fix(adminui): Faulted-devices tile counts distinct Faulted devices across the cluster pair (#786)

Owner decision 2026-11-11: /fleet/diagnostics "Faulted devices" counts DISTINCT
devices across the nodes of the selected node's cluster, Faulted state only.

- DriverHostActor: FaultedDevices = HostState.Faulted only (Stopped no longer
  counted); new StoppedDevices + FaultedHostNames. Counting extracted to the
  testable DiagnosticsFromHealth.
- DriverInstanceDiagnostics: StoppedDevices (default 0) and FaultedHostNames
  (default null) are additive with defaults, so a pre-#786 sender's snapshot
  still deserializes over Akka's default JSON serializer (pinned by a test).
- FleetDiagnosticsAggregation (pure): identity key = (driver-instance string
  id, HostName), case-insensitive; a single-host driver / nameless pre-#786
  count is combined by MAX across nodes, never summed. Cluster peers come from
  enabled ClusterNode rows of the selected node's ClusterId.
- FleetDiagnostics.razor: tile fetches the cluster peers and shows the
  distinct count + "n of m nodes answered"; per-driver table gains a Stopped
  devices column.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
This commit is contained in:
Joseph Doherty
2026-11-11 11:58:35 -05:00
parent e857135e8b
commit 43bdeff2b6
6 changed files with 437 additions and 10 deletions
@@ -2755,9 +2755,10 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
/// One child's diagnostics row. <c>Name</c> carries the string driver-instance id — the identity every
/// other surface (/hosts, telemetry, the artifact) keys on — and <c>DriverInstanceId</c> the DB row Guid;
/// the wire record has no slot for the authored display name. Device counts come from the driver's host
/// statuses when it is an <c>IHostConnectivityProbe</c> (Running = connected; Stopped/Faulted = faulted;
/// Unknown = neither); a single-host driver counts as one device, connected while it reports Healthy or
/// Degraded and faulted while it reports Faulted. A child that has not published yet reports
/// statuses when it is an <c>IHostConnectivityProbe</c> (Running = connected; Faulted = faulted;
/// Stopped = stopped, counted separately since #786; Unknown = none) and the faulted hosts' names ride
/// along so the AdminUI can de-duplicate a device across a redundant pair; a single-host driver counts as
/// one device, connected while it reports Healthy or Degraded and faulted while it reports Faulted. A child that has not published yet reports
/// "Connecting" (or "Stubbed") with its spawn time as the last change.
/// </summary>
private static DriverInstanceDiagnostics ToDiagnostics(string driverInstanceId, ChildEntry entry, ChildHealthRecord? health)
@@ -2768,20 +2769,40 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
if (health is null)
return new DriverInstanceDiagnostics(entry.Spec.DriverInstanceRowId, name, "Connecting", 0, 0, entry.SpawnedUtc);
int connected, faulted;
return DiagnosticsFromHealth(entry.Spec.DriverInstanceRowId, name, health);
}
/// <summary>The device counts of a child that has published health (#786: Faulted and Stopped split).</summary>
/// <param name="rowId">The driver instance's ConfigDb row id.</param>
/// <param name="name">The string driver-instance id.</param>
/// <param name="health">The child's last-published health.</param>
/// <returns>The diagnostics row.</returns>
internal static DriverInstanceDiagnostics DiagnosticsFromHealth(Guid rowId, string name, ChildHealthRecord health)
{
int connected, faulted, stopped;
IReadOnlyList<string> faultedHostNames;
if (health.HostStatuses is { Count: > 0 } hosts)
{
connected = hosts.Count(h => h.State == HostState.Running);
faulted = hosts.Count(h => h.State is HostState.Stopped or HostState.Faulted);
// #786: Faulted only. A Stopped host (e.g. a Galaxy engine an operator took off scan) is a
// deliberate, separately-reported state, not a fault.
var faultedHosts = hosts.Where(h => h.State == HostState.Faulted).ToArray();
faulted = faultedHosts.Length;
stopped = hosts.Count(h => h.State == HostState.Stopped);
faultedHostNames = faultedHosts.Select(h => h.HostName).ToArray();
}
else
{
connected = health.State is DriverState.Healthy or DriverState.Degraded ? 1 : 0;
faulted = health.State == DriverState.Faulted ? 1 : 0;
stopped = 0;
// A single-host driver is its own device: no host name; the AdminUI keys it on the instance id.
faultedHostNames = Array.Empty<string>();
}
return new DriverInstanceDiagnostics(
entry.Spec.DriverInstanceRowId, name, health.State.ToString(), connected, faulted, health.PublishedUtc);
rowId, name, health.State.ToString(), connected, faulted, health.PublishedUtc,
StoppedDevices: stopped, FaultedHostNames: faultedHostNames);
}
private void HandleDispatchFromSteady(DispatchDeployment msg)