fix(adminui): Faulted-devices tile counts distinct Faulted devices across the cluster pair (#786)

Owner decision 2026-11-11: /fleet/diagnostics "Faulted devices" counts DISTINCT
devices across the nodes of the selected node's cluster, Faulted state only.

- DriverHostActor: FaultedDevices = HostState.Faulted only (Stopped no longer
  counted); new StoppedDevices + FaultedHostNames. Counting extracted to the
  testable DiagnosticsFromHealth.
- DriverInstanceDiagnostics: StoppedDevices (default 0) and FaultedHostNames
  (default null) are additive with defaults, so a pre-#786 sender's snapshot
  still deserializes over Akka's default JSON serializer (pinned by a test).
- FleetDiagnosticsAggregation (pure): identity key = (driver-instance string
  id, HostName), case-insensitive; a single-host driver / nameless pre-#786
  count is combined by MAX across nodes, never summed. Cluster peers come from
  enabled ClusterNode rows of the selected node's ClusterId.
- FleetDiagnostics.razor: tile fetches the cluster peers and shows the
  distinct count + "n of m nodes answered"; per-driver table gains a Stopped
  devices column.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
This commit is contained in:
Joseph Doherty
2026-11-11 11:58:35 -05:00
parent e857135e8b
commit 43bdeff2b6
6 changed files with 437 additions and 10 deletions
@@ -0,0 +1,117 @@
using Microsoft.EntityFrameworkCore;
using ZB.MOM.WW.OtOpcUa.Commons.Interfaces;
using ZB.MOM.WW.OtOpcUa.Configuration;
namespace ZB.MOM.WW.OtOpcUa.AdminUI.Clients;
/// <summary>
/// Cluster-wide counting for the <c>/fleet/diagnostics</c> "Faulted devices" tile (#786). Pure, so it is
/// unit-testable without bUnit.
/// </summary>
/// <remarks>
/// <para>
/// <b>Owner decision 2026-11-11:</b> the tile counts DISTINCT devices across the nodes of one application
/// cluster (a redundant pair), and only devices in <c>Faulted</c> — <c>Stopped</c> is a separate per-node
/// column. Both nodes of a pair run the same driver instances against the same devices, so summing the two
/// nodes' per-driver counts reported every faulted device twice.
/// </para>
/// <para>
/// <b>The device identity key is (driver-instance id, host name)</b>, both compared case-insensitively:
/// <list type="bullet">
/// <item>
/// The driver-instance id is <see cref="DriverInstanceDiagnostics.Name"/> — the string id from the
/// deployed artifact, identical on both nodes of a pair (the row Guid is identical too, but can be
/// <see cref="Guid.Empty"/> on a node with no ConfigDb row behind its spec, so the string id is the
/// sturdier key and the one every other surface uses).
/// </item>
/// <item>
/// The host name is the driver's own <c>HostConnectivityStatus.HostName</c> (e.g. <c>host:port</c>
/// for Modbus/S7, a Platform/Engine name for Galaxy) — driver-defined from the same authored config,
/// so the same device yields the same name on both nodes. It is scoped by the driver id because two
/// different drivers can legitimately name the same endpoint and are counted as two devices.
/// </item>
/// </list>
/// A single-host driver (no <c>IHostConnectivityProbe</c>) reports no host names: the driver instance itself
/// is the device. Its faulted figure (0 or 1) — and any count a pre-#786 sender reports without names —
/// is "unnamed", and unnamed counts for one driver are combined by MAX across nodes, never summed: the
/// same unnamed device on both nodes counts once, which is the most the snapshot can prove distinct.
/// </para>
/// <para>
/// A node that did not answer (the client degrades to an empty snapshot) contributes nothing, so the figure
/// is a lower bound over the nodes that did — the page shows how many answered.
/// </para>
/// </remarks>
public static class FleetDiagnosticsAggregation
{
/// <summary>Counts the distinct faulted devices across a set of node snapshots (normally one cluster's nodes).</summary>
/// <param name="snapshots">One snapshot per node; duplicates of a node are harmless (the count is a set union).</param>
/// <returns>The number of distinct (driver id, host name) devices faulted on at least one node.</returns>
public static int DistinctFaultedDevices(IEnumerable<NodeDiagnosticsSnapshot> snapshots)
{
ArgumentNullException.ThrowIfNull(snapshots);
var named = new HashSet<(string Driver, string Host)>(DeviceKeyComparer.Instance);
var unnamedMax = new Dictionary<string, int>(StringComparer.OrdinalIgnoreCase);
foreach (var snapshot in snapshots)
{
foreach (var driver in snapshot.Drivers)
{
var names = driver.FaultedHostNames ?? Array.Empty<string>();
var namedHere = 0;
foreach (var host in names.Where(n => !string.IsNullOrWhiteSpace(n)).Distinct(StringComparer.OrdinalIgnoreCase))
{
named.Add((driver.Name, host));
namedHere++;
}
var unnamed = Math.Max(0, driver.FaultedDevices - namedHere);
if (unnamed > 0)
unnamedMax[driver.Name] = Math.Max(unnamedMax.GetValueOrDefault(driver.Name), unnamed);
}
}
return named.Count + unnamedMax.Values.Sum();
}
/// <summary>
/// The node ids (<c>host:port</c>) of every enabled node in the same application cluster as
/// <paramref name="nodeId"/>, the node itself included. A node with no <c>ClusterNode</c> row (or an empty
/// ConfigDb) yields just itself, so the tile degrades to a single-node count rather than failing.
/// </summary>
/// <param name="db">The config DB context.</param>
/// <param name="nodeId">The selected node's id, as <c>/fleet</c> shows it.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>The cluster id (null when unknown) and its node ids, the selected node first.</returns>
public static async Task<(string? ClusterId, IReadOnlyList<string> NodeIds)> LoadClusterNodeIdsAsync(
OtOpcUaConfigDbContext db, string nodeId, CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(db);
var self = await db.ClusterNodes.AsNoTracking()
.Where(n => n.NodeId == nodeId)
.Select(n => n.ClusterId)
.FirstOrDefaultAsync(ct);
if (self is null) return (null, new[] { nodeId });
var peers = await db.ClusterNodes.AsNoTracking()
.Where(n => n.ClusterId == self && n.Enabled && n.NodeId != nodeId)
.Select(n => n.NodeId)
.ToListAsync(ct);
peers.Sort(StringComparer.OrdinalIgnoreCase);
return (self, new[] { nodeId }.Concat(peers).ToList());
}
private sealed class DeviceKeyComparer : IEqualityComparer<(string Driver, string Host)>
{
public static readonly DeviceKeyComparer Instance = new();
public bool Equals((string Driver, string Host) x, (string Driver, string Host) y) =>
StringComparer.OrdinalIgnoreCase.Equals(x.Driver, y.Driver)
&& StringComparer.OrdinalIgnoreCase.Equals(x.Host, y.Host);
public int GetHashCode((string Driver, string Host) key) =>
HashCode.Combine(
StringComparer.OrdinalIgnoreCase.GetHashCode(key.Driver),
StringComparer.OrdinalIgnoreCase.GetHashCode(key.Host));
}
}