feat(hosts): surface per-host connectivity on /hosts; drop the unwritable table (#521)

`IHostConnectivityProbe` was a dead surface: eleven drivers implement it, `GetHostStatuses()` had ZERO production call sites, and `OnHostStatusChanged` had no subscriber outside the Galaxy driver's own aggregator. Per-host connectivity was computed by every driver and read by nobody. The issue offered "build the publisher or delete it". The publisher as its entity doc described it — driver nodes upserting `DriverHostStatus` rows — is not buildable: per-cluster mesh Phase 4 gates `AddOtOpcUaConfigDb` on the `admin` role, so a driver-only node has no ConfigDb connection to write rows with. So the capability is kept and the transport changed. `DriverHealthChanged.HostStatuses` now carries the probe result to `/hosts` as a Hosts column. That channel already reached the page, already survives the mesh split via the Phase 5 gRPC telemetry stream, and already replays a last-value snapshot on re-subscribe — so per-host state re-primes after a reconnect without a durable store. Both halves of the interface finally do what they are for: the event triggers a prompt publish, the pull is the source of truth. The point of the column is the case the driver-level state chip structurally cannot express: a multi-device driver stays aggregate-Healthy while ONE of its devices is unreachable. Two traps, both pinned by tests that were falsified against the prod code: - The host digest MUST be in the publish fingerprint. On a single-host-down transition every other fingerprint component is unchanged, so the dedup would swallow exactly the publish carrying the news — the trap that already bit the rediscovery signal. Removing it turns the guard test red, verified. - null (no probe) must stay distinct from empty (probe with no hosts). proto3 cannot tell an absent repeated field from an empty one, hence the explicit `has_host_statuses` flag; collapsing them would render every probe-less driver as one whose devices are all fine. Dropped: the DriverHostStatus entity, enum, DbSet, model config and table (migration DropDriverHostStatusTable — empty on every deployment, so the scaffolder's data-loss warning is moot, and Down() recreates it exactly). Found en route, NOT fixed here: `DriverInstanceResilienceStatus` is the identical defect — no writer, no reader, only a DbSet declaration, while the live data rides the `driver-resilience-status` telemetry channel. Its doc-comment now states that rather than describing the sampler and AdminUI join that were never built. Filed as #524 rather than widening this schema change beyond what was asked. Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-07-30 04:21:08 -04:00
parent dc9d947bca
commit 30d0697c28
24 changed files with 2411 additions and 305 deletions
@@ -88,6 +88,38 @@ single stream per (central, driver-node) pair carries all four, rather than one
 Proto field evolution is additive-only (never renumber/reuse a tag), locked by a contract test that
 reflects over the `oneof` cases.
 ### Per-host connectivity rides `driver-health` (Gitea #521)
 `DriverHealthChanged.HostStatuses` carries each driver's `IHostConnectivityProbe.GetHostStatuses()`
 result, rendered as the **Hosts** column on `/hosts`. It exists to show the one thing the driver-level
 state chip structurally cannot: a **multi-device driver stays aggregate-`Healthy` while one of its
 devices is unreachable**. A FOCAS or AbLegacy instance owning several PLCs previously hid that
 entirely.
 Three things to know before touching it:
 - **This deliberately does NOT go through SQL.** A `DriverHostStatus` table existed, with an entity
  doc describing a publisher hosted service that upserted rows from each driver node. That publisher
  was never written, and **per-cluster mesh Phase 4 made it unbuildable as described** — `Program.cs`
  gates `AddOtOpcUaConfigDb` on the `admin` role, so a driver-only node has no ConfigDb connection to
  write rows with. The table was dropped (migration `DropDriverHostStatusTable`); it was empty on
  every deployment. This channel needs no DB and already survives the mesh split.
 - **null ≠ empty, on the wire too.** Null means "the driver has no probe"; an empty list means "it has
  one that currently knows no hosts". proto3 cannot distinguish an absent repeated field from an empty
  one, so `DriverHealth.has_host_statuses` carries that bit explicitly. Collapsing the two would render
  every probe-less driver as one whose devices are all fine.
 - **The host digest is part of the publish fingerprint, and must stay there.** `PublishHealthSnapshot`
  dedups on that fingerprint, and on a single-host-down transition *every other component is
  unchanged* — so without it the dedup swallows precisely the publish carrying the news. Same trap
  that bit the rediscovery signal; pinned by
  `DriverInstanceActorHostStatusTests.A_single_host_going_down_is_not_swallowed_by_the_unchanged_health_dedup`.
  The digest is a flattened, host-name-ordered **string** for the converse reason: a tuple holding an
  `IReadOnlyList` compares by reference and would re-publish on every 30 s heartbeat forever.
 ⚠️ **`DriverInstanceResilienceStatus` is the same defect, still open.** That table also has no writer
 and no reader — the only reference in the repo is its DbSet declaration — while the live data reaches
 the AdminUI over the `driver-resilience-status` channel above. Keep-or-delete is Gitea **#524**.
 ## The three deferred channels (NOT migrated in Phase 5 — do not read this as "seven done")
 The program sketch originally named seven observability topics for Phase 5. Three were scoped out,
@@ -1,3 +1,5 @@
 using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
 namespace ZB.MOM.WW.OtOpcUa.Commons.Messages.Drivers;
 /// <summary>
@@ -26,6 +28,22 @@ namespace ZB.MOM.WW.OtOpcUa.Commons.Messages.Drivers;
 ///     The driver-supplied reason string from the same event (e.g. <c>"deploy-time-changed"</c>), shown
 ///     to the operator alongside the prompt. Null when <paramref name="RediscoveryNeededUtc"/> is null.
 /// </param>
 /// <param name="HostStatuses">
 ///     Per-host connectivity as reported by <c>IHostConnectivityProbe.GetHostStatuses()</c>, or null when
 ///     the driver does not implement that capability. Empty (not null) when it does but knows no hosts yet.
 ///     <para><b>Why this rides the health snapshot rather than a table.</b> The <c>DriverHostStatus</c>
 ///     entity used to own this, with a doc-comment describing a publisher hosted service that upserted rows
 ///     from each driver node. That publisher was never built, and since per-cluster mesh Phase 4 it is
 ///     unbuildable as described: <c>Program.cs</c> gates <c>AddOtOpcUaConfigDb</c> on the <c>admin</c> role,
 ///     so a driver-only node has no ConfigDb connection to write rows to. This channel already reaches
 ///     <c>/hosts</c>, already survives the mesh split via the Phase 5 gRPC telemetry stream, and already
 ///     replays a last-value snapshot on every (re)subscribe — so per-host state re-primes after a
 ///     reconnect without a durable store. The table was dropped; see Gitea #521.</para>
 ///     <para><b>Value-equality caveat.</b> A record's generated <c>Equals</c> compares this list by
 ///     REFERENCE, so two <c>DriverHealthChanged</c> carrying equal-but-distinct lists are not equal.
 ///     Nothing dedups on record equality — <c>DriverInstanceActor</c> keeps its own flattened fingerprint
 ///     precisely because of this — but do not introduce such a comparison without fixing it here first.</para>
 /// </param>
 public sealed record DriverHealthChanged(
    string ClusterId,
    string DriverInstanceId,
@@ -35,7 +53,8 @@ public sealed record DriverHealthChanged(
    int ErrorCount5Min,
    DateTime PublishedUtc,
    DateTime? RediscoveryNeededUtc = null,
-    string? RediscoveryReason = null)
+    string? RediscoveryReason = null,
    IReadOnlyList<HostConnectivityStatus>? HostStatuses = null)
 {
    /// <summary>
    ///     DPS topic name. Both the runtime <c>AkkaDriverHealthPublisher</c> and the AdminUI
@@ -70,6 +70,19 @@ message DriverHealth {
  google.protobuf.Timestamp published_utc = 7;
  google.protobuf.Timestamp rediscovery_needed_utc = 8; // DateTime? — absent Timestamp encodes null
  optional string rediscovery_reason = 9;               // nullable in the record
  // Per-host connectivity (Gitea #521). proto3 cannot distinguish an absent repeated field from an
  // empty one, and the two mean different things here — "driver is not an IHostConnectivityProbe" vs
  // "it is one and currently knows no hosts" — so the presence flag carries that bit explicitly rather
  // than letting an empty list silently claim the driver has no probe.
  bool has_host_statuses = 10;
  repeated HostConnectivity host_statuses = 11;
 }
 // Mirrors ZB.MOM.WW.OtOpcUa.Core.Abstractions.HostConnectivityStatus.
 message HostConnectivity {
  string host_name = 1;
  string state = 2;                                       // HostState-as-string, matching DriverHealth.state
  google.protobuf.Timestamp last_changed_utc = 3;
 }
 // Mirrors ZB.MOM.WW.OtOpcUa.Commons.Messages.Drivers.DriverResilienceStatusChanged.
@@ -1,62 +0,0 @@
 using ZB.MOM.WW.OtOpcUa.Configuration.Enums;
 namespace ZB.MOM.WW.OtOpcUa.Configuration.Entities;
 /// <summary>
 ///     Per-host connectivity snapshot the Server publishes for each driver's
 ///     <c>IHostConnectivityProbe.GetHostStatuses</c> entry. One row per
 ///     (<see cref="NodeId"/>, <see cref="DriverInstanceId"/>, <see cref="HostName"/>) triple —
 ///     a redundant 2-node cluster with one Galaxy driver reporting 3 platforms produces 6
 ///     rows, not 3, because each server node owns its own runtime view.
 /// </summary>
 /// <remarks>
 ///     <para>
 ///         Supports the per-AppEngine Admin dashboard drill-down. The publisher hosted
 ///         service on the Server side subscribes to every
 ///         registered driver's <c>OnHostStatusChanged</c> and upserts rows on transitions +
 ///         periodic liveness heartbeats. <see cref="LastSeenUtc"/> advances on every
 ///         heartbeat so the Admin UI can flag stale rows from a crashed Server.
 ///     </para>
 ///     <para>
 ///         No foreign-key to <see cref="ClusterNode"/> — a Server may start reporting host
 ///         status before its ClusterNode row exists (e.g. first-boot bootstrap), and we'd
 ///         rather keep the status row than drop it. The Admin-side service left-joins on
 ///         NodeId when presenting rows.
 ///     </para>
 /// </remarks>
 public sealed class DriverHostStatus
 {
    /// <summary>Server node that's running the driver.</summary>
    public required string NodeId { get; set; }
    /// <summary>Driver instance's stable id (matches <c>IDriver.DriverInstanceId</c>).</summary>
    public required string DriverInstanceId { get; set; }
    /// <summary>
    ///     Driver-side host identifier — Galaxy Platform / AppEngine name, Modbus
    ///     <c>host:port</c>, whatever the probe returns. Opaque to the Admin UI except as
    ///     a display string.
    /// </summary>
    public required string HostName { get; set; }
    /// <summary>Gets or sets the current connectivity state of the host.</summary>
    public DriverHostState State { get; set; } = DriverHostState.Unknown;
    /// <summary>Timestamp of the last state transition (not of the most recent heartbeat).</summary>
    public DateTime StateChangedUtc { get; set; }
    /// <summary>
    ///     Advances on every publisher heartbeat — the Admin UI uses
    ///     <c>now - LastSeenUtc &gt; threshold</c> to flag rows whose owning Server has
    ///     stopped reporting (crashed, network-partitioned, etc.), independent of
    ///     <see cref="State"/>.
    /// </summary>
    public DateTime LastSeenUtc { get; set; }
    /// <summary>
    ///     Optional human-readable detail populated when <see cref="State"/> is
    ///     <see cref="DriverHostState.Faulted"/> — e.g. the exception message from the
    ///     driver's probe. Null for Running / Stopped / Unknown transitions.
    /// </summary>
    public string? Detail { get; set; }
 }
@@ -1,17 +1,26 @@
 namespace ZB.MOM.WW.OtOpcUa.Configuration.Entities;
 /// <summary>
-///     Runtime resilience counters the CapabilityInvoker + MemoryTracking + MemoryRecycle
+///     Runtime resilience counters per <c>(DriverInstanceId, HostName)</c> pair.
-///     surfaces for each <c>(DriverInstanceId, HostName)</c> pair. Separate from
+///     <para><b>⚠️ This table is DEAD: nothing writes it and nothing reads it.</b> The only reference in
-///     <see cref="DriverHostStatus"/> (which owns per-host <i>connectivity</i> state) so a
+///     the repo is the <c>DriverInstanceResilienceStatuses</c> DbSet declaration. Do not treat a query
-///     host that's Running but has tripped its breaker or is approaching its memory ceiling
+///     against it as a source of runtime state — it returns empty on every deployment.</para>
 ///     shows up distinctly on Admin <c>/hosts</c>.
 /// </summary>
 /// <remarks>
-///     Per <c>docs/v2/implementation/phase-6-1-resilience-and-observability.md</c> §Stream E.1.
+///     <para>
-///     The Admin UI left-joins this table on DriverHostStatus for display; rows are written
+///         The original design (<c>docs/v2/implementation/phase-6-1-resilience-and-observability.md</c>
-///     by the runtime via a HostedService that samples the tracker at a configurable
+///         §Stream E.1) called for a HostedService sampling the tracker every ~5 s into this table, and
-///     interval (default 5 s) — writes are non-critical, a missed sample is tolerated.
+///         an Admin UI that left-joined it on <c>DriverHostStatus</c> for display. Neither was built:
 ///         there is no sampler, the join exists in no razor file, and <c>DriverHostStatus</c> itself was
 ///         removed in Gitea #521.
 ///     </para>
 ///     <para>
 ///         The live data does exist — it just never goes through SQL. Resilience state reaches the
 ///         AdminUI as <c>DriverResilienceStatusChanged</c> over the Phase 5 telemetry stream into the
 ///         in-memory <c>IDriverResilienceStatusStore</c>, the same shape #521 adopted for per-host
 ///         connectivity, and for the same reason: per-cluster mesh Phase 4 leaves a driver-only node with
 ///         no ConfigDb connection to write rows with. Keep-or-delete is tracked as Gitea #524.
 ///     </para>
 /// </remarks>
 public sealed class DriverInstanceResilienceStatus
 {
@@ -1,21 +0,0 @@
 namespace ZB.MOM.WW.OtOpcUa.Configuration.Enums;
 /// <summary>
 ///     Persisted mirror of <c>Core.Abstractions.HostState</c> — the lifecycle state each
 ///     <c>IHostConnectivityProbe</c>-capable driver reports for its per-host topology
 ///     (Galaxy Platforms / AppEngines, Modbus PLC endpoints, future OPC UA gateway upstreams).
 ///     Defined here instead of re-using <c>Core.Abstractions.HostState</c> so the
 ///     Configuration project stays free of driver-runtime dependencies.
 /// </summary>
 /// <remarks>
 ///     The server-side publisher (follow-up PR) translates
 ///     <c>HostStatusChangedEventArgs.NewState</c> to this enum on every transition and
 ///     upserts into <see cref="Entities.DriverHostStatus"/>. Admin UI reads from the DB.
 /// </remarks>
 public enum DriverHostState
 {
    Unknown,
    Running,
    Stopped,
    Faulted,
 }
@@ -0,0 +1,56 @@
 using System;
 using Microsoft.EntityFrameworkCore.Migrations;
 #nullable disable
 namespace ZB.MOM.WW.OtOpcUa.Configuration.Migrations
 {
    /// <summary>
    ///     Drops the DriverHostStatus table (Gitea #521). The scaffolder warns this "may result in the loss
    ///     of data"; here it cannot. No code ever inserted a row — the publisher hosted service its entity
    ///     doc described was never written, and per-cluster mesh Phase 4 made it unbuildable as described,
    ///     since a driver-only node has no ConfigDb connection. Every deployment's copy of this table is
    ///     empty. Per-host connectivity now rides DriverHealthChanged.HostStatuses over the Phase 5
    ///     telemetry stream to /hosts. Down() recreates the table exactly, so the rollback is lossless too.
    /// </summary>
    public partial class DropDriverHostStatusTable : Migration
    {
        /// <inheritdoc />
        protected override void Up(MigrationBuilder migrationBuilder)
        {
            migrationBuilder.DropTable(
                name: "DriverHostStatus");
        }
        /// <inheritdoc />
        protected override void Down(MigrationBuilder migrationBuilder)
        {
            migrationBuilder.CreateTable(
                name: "DriverHostStatus",
                columns: table => new
                {
                    NodeId = table.Column<string>(type: "nvarchar(64)", maxLength: 64, nullable: false),
                    DriverInstanceId = table.Column<string>(type: "nvarchar(64)", maxLength: 64, nullable: false),
                    HostName = table.Column<string>(type: "nvarchar(256)", maxLength: 256, nullable: false),
                    Detail = table.Column<string>(type: "nvarchar(1024)", maxLength: 1024, nullable: true),
                    LastSeenUtc = table.Column<DateTime>(type: "datetime2(3)", nullable: false),
                    State = table.Column<string>(type: "nvarchar(16)", maxLength: 16, nullable: false),
                    StateChangedUtc = table.Column<DateTime>(type: "datetime2(3)", nullable: false)
                },
                constraints: table =>
                {
                    table.PrimaryKey("PK_DriverHostStatus", x => new { x.NodeId, x.DriverInstanceId, x.HostName });
                });
            migrationBuilder.CreateIndex(
                name: "IX_DriverHostStatus_LastSeen",
                table: "DriverHostStatus",
                column: "LastSeenUtc");
            migrationBuilder.CreateIndex(
                name: "IX_DriverHostStatus_Node",
                table: "DriverHostStatus",
                column: "NodeId");
        }
    }
 }
@@ -394,46 +394,6 @@ namespace ZB.MOM.WW.OtOpcUa.Configuration.Migrations
                        });
                });
            modelBuilder.Entity("ZB.MOM.WW.OtOpcUa.Configuration.Entities.DriverHostStatus", b =>
                {
                    b.Property<string>("NodeId")
                        .HasMaxLength(64)
                        .HasColumnType("nvarchar(64)");
                    b.Property<string>("DriverInstanceId")
                        .HasMaxLength(64)
                        .HasColumnType("nvarchar(64)");
                    b.Property<string>("HostName")
                        .HasMaxLength(256)
                        .HasColumnType("nvarchar(256)");
                    b.Property<string>("Detail")
                        .HasMaxLength(1024)
                        .HasColumnType("nvarchar(1024)");
                    b.Property<DateTime>("LastSeenUtc")
                        .HasColumnType("datetime2(3)");
                    b.Property<string>("State")
                        .IsRequired()
                        .HasMaxLength(16)
                        .HasColumnType("nvarchar(16)");
                    b.Property<DateTime>("StateChangedUtc")
                        .HasColumnType("datetime2(3)");
                    b.HasKey("NodeId", "DriverInstanceId", "HostName");
                    b.HasIndex("LastSeenUtc")
                        .HasDatabaseName("IX_DriverHostStatus_LastSeen");
                    b.HasIndex("NodeId")
                        .HasDatabaseName("IX_DriverHostStatus_Node");
                    b.ToTable("DriverHostStatus", (string)null);
                });
            modelBuilder.Entity("ZB.MOM.WW.OtOpcUa.Configuration.Entities.DriverInstance", b =>
                {
                    b.Property<Guid>("DriverInstanceRowId")
@@ -44,8 +44,6 @@ public sealed class OtOpcUaConfigDbContext(DbContextOptions<OtOpcUaConfigDbConte
    public DbSet<ConfigAuditLog> ConfigAuditLogs => Set<ConfigAuditLog>();
    /// <summary>Gets the DbSet of external ID reservations.</summary>
    public DbSet<ExternalIdReservation> ExternalIdReservations => Set<ExternalIdReservation>();
    /// <summary>Gets the DbSet of driver host statuses.</summary>
    public DbSet<DriverHostStatus> DriverHostStatuses => Set<DriverHostStatus>();
    /// <summary>Gets the DbSet of driver instance resilience statuses.</summary>
    public DbSet<DriverInstanceResilienceStatus> DriverInstanceResilienceStatuses => Set<DriverInstanceResilienceStatus>();
    /// <summary>Gets the DbSet of LDAP group role mappings.</summary>
@@ -90,7 +88,6 @@ public sealed class OtOpcUaConfigDbContext(DbContextOptions<OtOpcUaConfigDbConte
        ConfigureNodeAcl(modelBuilder);
        ConfigureConfigAuditLog(modelBuilder);
        ConfigureExternalIdReservation(modelBuilder);
        ConfigureDriverHostStatus(modelBuilder);
        ConfigureDriverInstanceResilienceStatus(modelBuilder);
        ConfigureLdapGroupRoleMapping(modelBuilder);
        ConfigureScript(modelBuilder);
@@ -542,31 +539,12 @@ public sealed class OtOpcUaConfigDbContext(DbContextOptions<OtOpcUaConfigDbConte
        });
    }
-    private static void ConfigureDriverHostStatus(ModelBuilder modelBuilder)
+    // ConfigureDriverHostStatus is GONE (Gitea #521). The DriverHostStatus table held per-host
-    {
+    // connectivity that a publisher hosted service was supposed to upsert from each driver node. That
-        modelBuilder.Entity<DriverHostStatus>(e =>
+    // publisher was never written, and per-cluster mesh Phase 4 made it unwritable as designed: ConfigDb
-        {
+    // is registered only on the admin role, so a driver-only node has no connection to write rows with.
-            e.ToTable("DriverHostStatus");
+    // Per-host connectivity now rides DriverHealthChanged.HostStatuses to /hosts over the Phase 5
-            // Composite key — one row per (server node, driver instance, probe-reported host).
+    // telemetry stream, which needs no DB and survives the mesh split.
            // A redundant 2-node cluster with one Galaxy driver reporting 3 platforms produces
            // 6 rows because each server node owns its own runtime view; the composite key is
            // what lets both views coexist without shadowing each other.
            e.HasKey(x => new { x.NodeId, x.DriverInstanceId, x.HostName });
            e.Property(x => x.NodeId).HasMaxLength(64);
            e.Property(x => x.DriverInstanceId).HasMaxLength(64);
            e.Property(x => x.HostName).HasMaxLength(256);
            e.Property(x => x.State).HasConversion<string>().HasMaxLength(16);
            e.Property(x => x.StateChangedUtc).HasColumnType("datetime2(3)");
            e.Property(x => x.LastSeenUtc).HasColumnType("datetime2(3)");
            e.Property(x => x.Detail).HasMaxLength(1024);
            // NodeId-only index drives the Admin UI's per-cluster drill-down (select all host
            // statuses for the nodes of a specific cluster via join on ClusterNode.ClusterId).
            e.HasIndex(x => x.NodeId).HasDatabaseName("IX_DriverHostStatus_Node");
            // LastSeenUtc index powers the Admin UI's stale-row query (now - LastSeen > N).
            e.HasIndex(x => x.LastSeenUtc).HasDatabaseName("IX_DriverHostStatus_LastSeen");
        });
    }
    private static void ConfigureDriverInstanceResilienceStatus(ModelBuilder modelBuilder)
    {
@@ -22,13 +22,20 @@ public interface IDriverHealthPublisher
    /// </param>
    /// <param name="rediscoveryReason">The driver-supplied reason from that event; null when
    ///     <paramref name="rediscoveryNeededUtc"/> is null.</param>
    /// <param name="hostStatuses">
    ///     Per-host connectivity from <see cref="IHostConnectivityProbe.GetHostStatuses"/>, or null when the
    ///     driver is not an <see cref="IHostConnectivityProbe"/>. Lets a multi-device driver (a FOCAS or
    ///     AbLegacy instance owning several PLCs) surface ONE unreachable device that would otherwise be
    ///     invisible behind an aggregate-Healthy driver row.
    /// </param>
    void Publish(
        string clusterId,
        string driverInstanceId,
        DriverHealth health,
        int errorCount5Min,
        DateTime? rediscoveryNeededUtc = null,
-        string? rediscoveryReason = null);
+        string? rediscoveryReason = null,
        IReadOnlyList<HostConnectivityStatus>? hostStatuses = null);
 }
 /// <summary>
@@ -49,6 +56,7 @@ public sealed class NullDriverHealthPublisher : IDriverHealthPublisher
        DriverHealth health,
        int errorCount5Min,
        DateTime? rediscoveryNeededUtc = null,
-        string? rediscoveryReason = null)
+        string? rediscoveryReason = null,
        IReadOnlyList<HostConnectivityStatus>? hostStatuses = null)
    { /* no-op */ }
 }
@@ -168,6 +168,7 @@ else
                                <th>Driver</th>
                                <th>Type</th>
                                <th>Status</th>
                                <th>Hosts</th>
                                <th>Last read</th>
                                <th>Errors/5 min</th>
                                <th>Last error</th>
@@ -177,7 +178,7 @@ else
                            @if (g.Drivers.Count == 0)
                            {
                                <tr>
-                                    <td colspan="6"><span class="text-muted">No drivers.</span></td>
+                                    <td colspan="7"><span class="text-muted">No drivers.</span></td>
                                </tr>
                            }
                            else
@@ -204,6 +205,28 @@ else
                                        </td>
                                        <td>@(d.DriverType ?? "—")</td>
                                        <td><span class="chip @DriverChipClass(d.State)">@d.State</span></td>
                                        <td>
                                            @* Per-host connectivity (Gitea #521). The point of this column is the case
                                               the driver-level Status chip cannot express: a multi-device driver stays
                                               Healthy in aggregate while ONE of its PLCs is unreachable. A driver with
                                               no probe shows "—" — deliberately distinct from a probe reporting zero
                                               hosts, which shows "0 hosts". *@
                                            @if (d.HostStatuses is null)
                                            {
                                                <span class="text-muted">—</span>
                                            }
                                            else if (d.DegradedHosts.Count == 0)
                                            {
                                                <span class="text-muted small">@d.HostStatuses.Count hosts</span>
                                            }
                                            else
                                            {
                                                <span class="chip chip-warn"
                                                      title="@DegradedHostTitle(d)">
                                                    @d.DegradedHosts.Count / @d.HostStatuses.Count down
                                                </span>
                                            }
                                        </td>
                                        <td>@(d.LastSuccessfulReadUtc?.ToString("HH:mm:ss 'UTC'") ?? "—")</td>
                                        <td class="numeric">@d.ErrorCount5Min</td>
                                        <td><span class="text-muted small">@(d.LastError ?? "—")</span></td>
@@ -355,6 +378,13 @@ else
        _              => "chip-idle",
    };
    // Tooltip listing each host that is not Running, with its state and when it last changed. Built in
    // C# rather than inline in the markup so the string is composed once per render, and so the row can
    // never be the thing that 500s the page — string.Join over an already-materialised list does no
    // indexing (cf. #504, where slicing a short DB string in Razor took down the whole page).
    private static string DegradedHostTitle(HostsDriverRow d) =>
        string.Join(" · ", d.DegradedHosts.Select(h => $"{h.HostName}: {h.State} since {h.LastChangedUtc:u}"));
    public async ValueTask DisposeAsync()
    {
        // Unsubscribe first so the singleton store can't invoke a handler on a disposed component.
@@ -1,6 +1,7 @@
 namespace ZB.MOM.WW.OtOpcUa.AdminUI.Hosts;
 using ZB.MOM.WW.OtOpcUa.Commons.Messages.Drivers;
 using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
 /// <summary>
 ///     One configured host node within a cluster, as the <c>/hosts</c> page needs it: the cluster
@@ -40,10 +41,26 @@ public sealed record HostsDriverInstanceInfo(string DriverInstanceId, string Clu
 ///     is unchanged and an operator must re-browse the device via <c>/raw</c> to pick anything up.</param>
 /// <param name="RediscoveryReason">The driver-supplied reason for that report; null when
 ///     <paramref name="RediscoveryNeededUtc"/> is null.</param>
 /// <param name="HostStatuses">Per-host connectivity for a multi-device driver, ordered by host name; null
 ///     when the driver reports no per-host detail. Lets one unreachable device show even while the driver
 ///     row itself is Healthy.</param>
 public sealed record HostsDriverRow(
    string DriverInstanceId, string? Name, string? DriverType, string State,
    DateTime? LastSuccessfulReadUtc, string? LastError, int ErrorCount5Min, DateTime PublishedUtc,
-    DateTime? RediscoveryNeededUtc = null, string? RediscoveryReason = null);
+    DateTime? RediscoveryNeededUtc = null, string? RediscoveryReason = null,
    IReadOnlyList<HostConnectivityStatus>? HostStatuses = null)
 {
    /// <summary>Hosts that are not <see cref="HostState.Running"/>, ordered by name — the ones worth an
    /// operator's attention. Empty when every host is fine or none are reported.
    /// <para><see cref="HostState.Unknown"/> counts as degraded on purpose: a probe that has not yet
    /// completed its first tick, or one a driver failed to start (AbCip logs exactly this case), reports
    /// Unknown — and silently rendering that as healthy is how the gap got missed the first time.</para></summary>
    public IReadOnlyList<HostConnectivityStatus> DegradedHosts { get; } =
        (HostStatuses ?? [])
            .Where(h => h.State != HostState.Running)
            .OrderBy(h => h.HostName, StringComparer.OrdinalIgnoreCase)
            .ToList();
 }
 /// <summary>
 ///     One cluster's section on the <c>/hosts</c> page: its configured nodes plus its enriched
@@ -118,7 +135,8 @@ public static class HostsDriverView
                        s.ErrorCount5Min,
                        s.PublishedUtc,
                        s.RediscoveryNeededUtc,
-                        s.RediscoveryReason);
+                        s.RediscoveryReason,
                        s.HostStatuses);
                })
                .OrderBy(d => d.Name ?? d.DriverInstanceId, StringComparer.OrdinalIgnoreCase)
                .ThenBy(d => d.DriverInstanceId, StringComparer.OrdinalIgnoreCase)
@@ -3,6 +3,10 @@ using ZB.MOM.WW.OtOpcUa.Commons.Messages.Alerts;
 using ZB.MOM.WW.OtOpcUa.Commons.Messages.Drivers;
 using ZB.MOM.WW.OtOpcUa.Commons.Messages.Logging;
 using ZB.MOM.WW.OtOpcUa.Commons.Protos.Telemetry.V1;
 // Aliased, not imported wholesale: Core.Abstractions also declares a DriverHealth, which would collide
 // with the proto DriverHealth this file maps.
 using HostConnectivityStatus = ZB.MOM.WW.OtOpcUa.Core.Abstractions.HostConnectivityStatus;
 using HostState = ZB.MOM.WW.OtOpcUa.Core.Abstractions.HostState;
 namespace ZB.MOM.WW.OtOpcUa.ControlPlane.Telemetry;
@@ -120,7 +124,32 @@ public static class TelemetryProtoMapCentral
            ErrorCount5Min: msg.ErrorCount5Min,
            PublishedUtc: Required(msg.PublishedUtc, "DriverHealth", "published_utc"),
            RediscoveryNeededUtc: msg.RediscoveryNeededUtc?.ToDateTime(),
-            RediscoveryReason: msg.HasRediscoveryReason ? msg.RediscoveryReason : null);
+            RediscoveryReason: msg.HasRediscoveryReason ? msg.RediscoveryReason : null,
            HostStatuses: ToHostStatuses(msg));
    }
    /// <summary>
    ///     Projects the repeated host-connectivity field, honouring the explicit presence flag: null when
    ///     the node said the driver has no probe, an empty list when it has one that knows no hosts.
    ///     <para>An unparseable state string degrades to <see cref="HostState.Unknown"/> rather than
    ///     throwing — a node running a newer build that added an enum member must not be able to kill
    ///     central's telemetry stream, which is observability and has no business failing closed.</para>
    /// </summary>
    private static IReadOnlyList<HostConnectivityStatus>? ToHostStatuses(DriverHealth msg)
    {
        if (!msg.HasHostStatuses) return null;
        var result = new List<HostConnectivityStatus>(msg.HostStatuses.Count);
        foreach (var h in msg.HostStatuses)
        {
            result.Add(new HostConnectivityStatus(
                h.HostName,
                // System.Enum qualified: Google.Protobuf.WellKnownTypes also declares an Enum type.
                System.Enum.TryParse<HostState>(h.State, ignoreCase: true, out var state) ? state : HostState.Unknown,
                h.LastChangedUtc?.ToDateTime() ?? default));
        }
        return result;
    }
    /// <summary>Projects a <see cref="DriverResilienceStatus"/> onto a <see cref="DriverResilienceStatusChanged"/>.</summary>
@@ -133,6 +133,21 @@ public static class TelemetryProtoMapNode
        if (e.RediscoveryReason is not null)
            msg.RediscoveryReason = e.RediscoveryReason;
        // Presence flag first — an empty repeated field cannot say whether the driver has a probe at all.
        if (e.HostStatuses is not null)
        {
            msg.HasHostStatuses = true;
            foreach (var h in e.HostStatuses)
            {
                msg.HostStatuses.Add(new HostConnectivity
                {
                    HostName = h.HostName ?? "",
                    State = h.State.ToString(),
                    LastChangedUtc = ToUtcTimestamp(h.LastChangedUtc),
                });
            }
        }
        return msg;
    }
@@ -36,7 +36,8 @@ public sealed class AkkaDriverHealthPublisher : IDriverHealthPublisher
        DriverHealth health,
        int errorCount5Min,
        DateTime? rediscoveryNeededUtc = null,
-        string? rediscoveryReason = null)
+        string? rediscoveryReason = null,
        IReadOnlyList<HostConnectivityStatus>? hostStatuses = null)
    {
        var msg = new DriverHealthChanged(
            clusterId,
@@ -47,7 +48,8 @@ public sealed class AkkaDriverHealthPublisher : IDriverHealthPublisher
            errorCount5Min,
            DateTime.UtcNow,
            rediscoveryNeededUtc,
-            rediscoveryReason);
+            rediscoveryReason,
            hostStatuses);
        DistributedPubSub.Get(_system).Mediator.Tell(new Publish(TopicName, msg));
        // Phase 5: fan the same snapshot into the node-local live-telemetry hub (no-op until a gRPC
        // client subscribes). The DPS publish above is unchanged — the hub is a strictly additive tap.
@@ -120,6 +120,14 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
    /// connection affinity (a Galaxy redeploy or a TwinCAT symbol-version bump can land while the driver is
    /// between connects), and dropping it in one state would lose the signal silently.</summary>
    private sealed record RediscoveryRaised(RediscoveryEventArgs Args);
    /// <summary>Self-sent when the wrapped driver raises <see cref="IHostConnectivityProbe.OnHostStatusChanged"/>
    /// — one of its hosts went Running ↔ Stopped ↔ Faulted. Marshals the event off the driver's probe thread
    /// onto the actor thread. Carries NO payload: the handler re-pulls
    /// <see cref="IHostConnectivityProbe.GetHostStatuses"/>, so the driver stays the single source of truth
    /// for the host set and a host that appeared since the last publish is picked up too. Handled in every
    /// behaviour for the same reason as <see cref="RediscoveryRaised"/> — a probe tick can land while the
    /// driver is between connects.</summary>
    private sealed record HostStatusRaised;
    public sealed class RetryConnect
    {
@@ -171,6 +179,7 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
    private EventHandler<DataChangeEventArgs>? _dataChangeHandler;
    private EventHandler<AlarmEventArgs>? _alarmEventHandler;
    private EventHandler<RediscoveryEventArgs>? _rediscoveryHandler;
    private EventHandler<HostStatusChangedEventArgs>? _hostStatusHandler;
    /// <summary>When the driver last raised <see cref="IRediscoverable.OnRediscoveryNeeded"/>, and the reason
    /// it gave. Null until the first raise. Carried on every subsequent health snapshot so the AdminUI can
@@ -306,6 +315,7 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
        // Attach the rediscovery signal before the first publish. Not per-connect: an IRediscoverable raise
        // has no connection affinity, and a driver can observe a remote change while disconnected.
        AttachRediscoverySource();
        AttachHostStatusSource();
        PublishHealthSnapshot();
        Timers.StartPeriodicTimer("health-poll", HealthPollTick.Instance, _healthPollInterval);
    }
@@ -325,6 +335,7 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
        // Stubbed drivers never enter Connected, so they never kick discovery; swallow defensively in case a
        // re-discovery self-tick is ever routed here so it doesn't surface as an Akka Unhandled message.
        Receive<RediscoveryRaised>(HandleRediscoveryRaised);
        Receive<HostStatusRaised>(_ => PublishHealthSnapshot());
        Receive<HealthPollTick>(_ => PublishHealthSnapshot());
    }
@@ -378,6 +389,7 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
        // this state; swallow it so it doesn't dead-letter — the next Connected entry re-subscribes.
        Receive<SubscribeAlarms>(_ => { });
        Receive<RediscoveryRaised>(HandleRediscoveryRaised);
        Receive<HostStatusRaised>(_ => PublishHealthSnapshot());
        Receive<HealthPollTick>(_ => PublishHealthSnapshot());
    }
@@ -438,6 +450,7 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
        Receive<SubscriptionFailed>(msg =>
            _log.Debug("DriverInstance {Id}: resubscribe reported failure: {Reason}", _driverInstanceId, msg.Reason));
        Receive<RediscoveryRaised>(HandleRediscoveryRaised);
        Receive<HostStatusRaised>(_ => PublishHealthSnapshot());
        Receive<HealthPollTick>(_ =>
        {
            PublishHealthSnapshot();
@@ -541,6 +554,7 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
        // this state; swallow it so it doesn't dead-letter — the next Connected entry re-subscribes.
        Receive<SubscribeAlarms>(_ => { });
        Receive<RediscoveryRaised>(HandleRediscoveryRaised);
        Receive<HostStatusRaised>(_ => PublishHealthSnapshot());
        Receive<HealthPollTick>(_ => PublishHealthSnapshot());
        Timers.StartPeriodicTimer("retry-connect", RetryConnect.Instance, _reconnectInterval);
    }
@@ -809,6 +823,48 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
        _rediscoveryHandler = null;
    }
    /// <summary>Subscribe the driver's <see cref="IHostConnectivityProbe.OnHostStatusChanged"/> (if it is
    /// one), marshaling each transition to the actor thread. Idempotent; mirrors
    /// <see cref="AttachRediscoverySource"/>, including the PreStart-not-per-connect placement — a probe
    /// loop can report a host down while the driver itself is between connects.</summary>
    private void AttachHostStatusSource()
    {
        if (_driver is not IHostConnectivityProbe src || _hostStatusHandler is not null) return;
        var self = Self;
        _hostStatusHandler = (_, _) => self.Tell(new HostStatusRaised());
        src.OnHostStatusChanged += _hostStatusHandler;
    }
    /// <summary>Symmetric teardown, called from PostStop — same leak argument as
    /// <see cref="DetachRediscoverySource"/>.</summary>
    private void DetachHostStatusSource()
    {
        if (_driver is IHostConnectivityProbe src && _hostStatusHandler is not null)
            src.OnHostStatusChanged -= _hostStatusHandler;
        _hostStatusHandler = null;
    }
    /// <summary>Current per-host connectivity, or null when the driver is not an
    /// <see cref="IHostConnectivityProbe"/>. Pulled fresh rather than cached: the driver owns the host set,
    /// and a stale local copy would be a second source of truth to keep in sync.
    /// <para><c>GetHostStatuses()</c> is called UNGUARDED (not through <see cref="_invoker"/>) on purpose —
    /// it is a pure in-memory snapshot with no I/O, which is exactly why
    /// <c>UnwrappedCapabilityCallAnalyzer</c> exempts it. A driver that makes it do I/O breaks that
    /// contract; the try/catch below keeps a misbehaving one from killing the health publish.</para></summary>
    private IReadOnlyList<HostConnectivityStatus>? CurrentHostStatuses()
    {
        if (_driver is not IHostConnectivityProbe probe) return null;
        try
        {
            return probe.GetHostStatuses();
        }
        catch (Exception ex)
        {
            _log.Warning(ex, "DriverInstance {Id}: GetHostStatuses threw during health publish; omitting host detail", _driverInstanceId);
            return null;
        }
    }
    /// <summary>Records the driver's rediscovery raise and re-publishes health so the signal reaches the
    /// AdminUI promptly rather than waiting for the next 30 s heartbeat.
    /// <para><b>Advisory only.</b> The served address space is deliberately NOT rebuilt: v3 authors raw tags
@@ -961,16 +1017,26 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
        {
            var health = _driver.GetHealth();
            var errorCount = ErrorCount5Min();
            var hostStatuses = CurrentHostStatuses();
            // _rediscoveryNeededUtc is PART OF THE FINGERPRINT on purpose. A rediscovery raise on an
            // otherwise-unchanged Healthy driver leaves (state, lastSuccess, lastError, errorCount)
            // identical, so without it the dedup below would swallow the very publish that carries the
            // signal and the operator would never see the prompt.
-            var fingerprint = (health.State, health.LastSuccessfulRead, health.LastError, errorCount, _rediscoveryNeededUtc);
+            //
            // The host-status digest is in the fingerprint for EXACTLY the same reason, and the failure
            // mode is the more likely one of the two: a multi-device driver stays aggregate-Healthy when a
            // single device drops, so every other fingerprint component is unchanged and the transition —
            // the whole point of this channel — would be deduped away. It is a flattened STRING because a
            // tuple holding IReadOnlyList compares by reference, which would never match and so would
            // defeat the dedup in the opposite direction (re-publishing every 30 s heartbeat).
            var fingerprint = (health.State, health.LastSuccessfulRead, health.LastError, errorCount,
                _rediscoveryNeededUtc, HostStatusDigest(hostStatuses));
            if (_lastPublishedFingerprint is { } prev && prev.Equals(fingerprint))
                return;
            _lastPublishedFingerprint = fingerprint;
            _healthPublisher.Publish(
-                _clusterId, _driverInstanceId, health, errorCount, _rediscoveryNeededUtc, _rediscoveryReason);
+                _clusterId, _driverInstanceId, health, errorCount, _rediscoveryNeededUtc, _rediscoveryReason,
                hostStatuses);
        }
        catch (Exception ex)
        {
@@ -978,8 +1044,22 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
        }
    }
    /// <summary>Order-insensitive value digest of a host-status list for the publish fingerprint. Null in,
    /// null out — so "driver is not a probe" and "probe reports no hosts" stay distinguishable rather than
    /// both collapsing to the empty string.
    /// <para>Ordered by host name because a driver is free to return its hosts in any order (several build
    /// the list from a <c>Dictionary</c>), and an order flip would otherwise read as a real transition and
    /// re-publish forever.</para></summary>
    private static string? HostStatusDigest(IReadOnlyList<HostConnectivityStatus>? statuses) =>
        statuses is null
            ? null
            : string.Join('|', statuses
                .OrderBy(s => s.HostName, StringComparer.Ordinal)
                .Select(s => $"{s.HostName}={s.State}@{s.LastChangedUtc:O}"));
    /// <summary>Fingerprint of the last <see cref="PublishHealthSnapshot"/> call; null until first publish.</summary>
-    private (DriverState State, DateTime? LastSuccess, string? LastError, int ErrorCount, DateTime? RediscoveryNeededUtc)? _lastPublishedFingerprint;
+    private (DriverState State, DateTime? LastSuccess, string? LastError, int ErrorCount,
        DateTime? RediscoveryNeededUtc, string? HostStatusDigest)? _lastPublishedFingerprint;
    /// <inheritdoc />
    protected override void PostStop()
@@ -988,6 +1068,7 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
        // MUST happen: the IDriver instance can outlive this actor (the host respawns a child around the
        // same driver object), so a missing unsubscribe accumulates a handler per respawn holding a dead Self.
        DetachRediscoverySource();
        DetachHostStatusSource();
        try { _driver.ShutdownAsync(CancellationToken.None).GetAwaiter().GetResult(); }
        catch (Exception ex) { _log.Warning(ex, "DriverInstance {Id}: ShutdownAsync threw on PostStop", _driverInstanceId); }
        OtOpcUaTelemetry.DriverInstanceLifecycle.Add(1,
@@ -1,131 +0,0 @@
 using Microsoft.EntityFrameworkCore;
 using Shouldly;
 using Xunit;
 using ZB.MOM.WW.OtOpcUa.Configuration.Entities;
 using ZB.MOM.WW.OtOpcUa.Configuration.Enums;
 namespace ZB.MOM.WW.OtOpcUa.Configuration.Tests;
 /// <summary>
 ///     End-to-end round-trip through the DB for the <see cref="DriverHostStatus"/> entity
 ///     added in PR 33 — exercises the composite primary key (NodeId, DriverInstanceId,
 ///     HostName), string-backed <c>DriverHostState</c> conversion, and the two indexes the
 ///     Admin UI's drill-down queries will scan (NodeId, LastSeenUtc).
 /// </summary>
 [Trait("Category", "SchemaCompliance")]
 [Collection(nameof(SchemaComplianceCollection))]
 public sealed class DriverHostStatusTests(SchemaComplianceFixture fixture)
 {
    /// <summary>Verifies that the composite key allows the same host across different nodes or drivers.</summary>
    [Fact]
    public async Task Composite_key_allows_same_host_across_different_nodes_or_drivers()
    {
        await using var ctx = NewContext();
        // Same HostName + DriverInstanceId across two different server nodes — classic 2-node
        // redundancy case. Both rows must be insertable because each server node owns its own
        // runtime view of the shared host.
        var now = DateTime.UtcNow;
        ctx.DriverHostStatuses.Add(new DriverHostStatus
        {
            NodeId = "node-a", DriverInstanceId = "galaxy-1", HostName = "GRPlatform",
            State = DriverHostState.Running,
            StateChangedUtc = now, LastSeenUtc = now,
        });
        ctx.DriverHostStatuses.Add(new DriverHostStatus
        {
            NodeId = "node-b", DriverInstanceId = "galaxy-1", HostName = "GRPlatform",
            State = DriverHostState.Stopped,
            StateChangedUtc = now, LastSeenUtc = now,
            Detail = "secondary hasn't taken over yet",
        });
        // Same server node + host, different driver instance — second driver doesn't clobber.
        ctx.DriverHostStatuses.Add(new DriverHostStatus
        {
            NodeId = "node-a", DriverInstanceId = "modbus-plc1", HostName = "GRPlatform",
            State = DriverHostState.Running,
            StateChangedUtc = now, LastSeenUtc = now,
        });
        await ctx.SaveChangesAsync();
        var rows = await ctx.DriverHostStatuses.AsNoTracking()
            .Where(r => r.HostName == "GRPlatform").ToListAsync();
        rows.Count.ShouldBe(3);
        rows.ShouldContain(r => r.NodeId == "node-a" && r.DriverInstanceId == "galaxy-1");
        rows.ShouldContain(r => r.NodeId == "node-b" && r.State == DriverHostState.Stopped && r.Detail == "secondary hasn't taken over yet");
        rows.ShouldContain(r => r.NodeId == "node-a" && r.DriverInstanceId == "modbus-plc1");
    }
    /// <summary>Verifies that the upsert pattern updates existing records in place.</summary>
    [Fact]
    public async Task Upsert_pattern_for_same_key_updates_in_place()
    {
        // The publisher hosted service (follow-up PR) upserts on every transition +
        // heartbeat. This test pins the two-step pattern it will use: check-then-add-or-update
        // keyed on the composite PK. If the composite key ever changes, this test breaks
        // loudly so the publisher gets a synchronized update.
        await using var ctx = NewContext();
        var t0 = DateTime.UtcNow;
        ctx.DriverHostStatuses.Add(new DriverHostStatus
        {
            NodeId = "upsert-node", DriverInstanceId = "upsert-driver", HostName = "upsert-host",
            State = DriverHostState.Running,
            StateChangedUtc = t0, LastSeenUtc = t0,
        });
        await ctx.SaveChangesAsync();
        var t1 = t0.AddSeconds(30);
        await using (var ctx2 = NewContext())
        {
            var existing = await ctx2.DriverHostStatuses.SingleAsync(r =>
                r.NodeId == "upsert-node" && r.DriverInstanceId == "upsert-driver" && r.HostName == "upsert-host");
            existing.State = DriverHostState.Faulted;
            existing.StateChangedUtc = t1;
            existing.LastSeenUtc = t1;
            existing.Detail = "transport reset by peer";
            await ctx2.SaveChangesAsync();
        }
        await using var ctx3 = NewContext();
        var final = await ctx3.DriverHostStatuses.AsNoTracking().SingleAsync(r =>
            r.NodeId == "upsert-node" && r.HostName == "upsert-host");
        final.State.ShouldBe(DriverHostState.Faulted);
        final.Detail.ShouldBe("transport reset by peer");
        // Only one row — a naive "always insert" would have created a duplicate PK and thrown.
        (await ctx3.DriverHostStatuses.CountAsync(r => r.NodeId == "upsert-node")).ShouldBe(1);
    }
    /// <summary>Verifies that the State enum is persisted as a string, not an integer.</summary>
    [Fact]
    public async Task Enum_persists_as_string_not_int()
    {
        // Fluent config sets HasConversion<string>() on State — the DB stores 'Running' /
        // 'Stopped' / 'Faulted' / 'Unknown' as nvarchar(16). Verify by reading the raw
        // string back via ADO; if someone drops the conversion the column will contain '1'
        // / '2' / '3' and this assertion fails. Matters because DBAs inspecting the table
        // directly should see readable state names, not enum ordinals.
        await using var ctx = NewContext();
        ctx.DriverHostStatuses.Add(new DriverHostStatus
        {
            NodeId = "enum-node", DriverInstanceId = "enum-driver", HostName = "enum-host",
            State = DriverHostState.Faulted,
            StateChangedUtc = DateTime.UtcNow, LastSeenUtc = DateTime.UtcNow,
        });
        await ctx.SaveChangesAsync();
        await using var conn = fixture.OpenConnection();
        using var cmd = conn.CreateCommand();
        cmd.CommandText = "SELECT [State] FROM DriverHostStatus WHERE NodeId = 'enum-node'";
        var rawValue = (string?)await cmd.ExecuteScalarAsync();
        rawValue.ShouldBe("Faulted");
    }
    private OtOpcUaConfigDbContext NewContext()
    {
        var options = new DbContextOptionsBuilder<OtOpcUaConfigDbContext>()
            .UseSqlServer(fixture.ConnectionString)
            .Options;
        return new OtOpcUaConfigDbContext(options);
    }
 }
@@ -2,6 +2,7 @@ using Shouldly;
 using Xunit;
 using ZB.MOM.WW.OtOpcUa.AdminUI.Hosts;
 using ZB.MOM.WW.OtOpcUa.Commons.Messages.Drivers;
 using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
 namespace ZB.MOM.WW.OtOpcUa.AdminUI.Tests.Hosts;
@@ -154,4 +155,62 @@ public sealed class HostsDriverViewTests
        groups.Select(g => g.ClusterId).ShouldBe(new[] { "Alpha", "Beta", "zeta" });
    }
    /// <summary>
    ///     Per-host connectivity flows through to the row (Gitea #521), and <c>DegradedHosts</c> picks out
    ///     exactly the hosts an operator needs to look at. This is the case the driver-level Status chip
    ///     cannot express: the driver is Healthy and one of its devices is not.
    /// </summary>
    [Fact]
    public void Build_carries_host_statuses_and_flags_only_the_degraded_ones()
    {
        var snapshot = Snap("MAIN", "drv-a") with
        {
            HostStatuses =
            [
                new HostConnectivityStatus("plc-a", HostState.Running, When),
                new HostConnectivityStatus("plc-b", HostState.Stopped, When),
                new HostConnectivityStatus("plc-c", HostState.Faulted, When),
            ],
        };
        var row = HostsDriverView.Build([snapshot], nodes: null, instances: null).Single().Drivers.Single();
        row.State.ShouldBe("Healthy");
        row.HostStatuses!.Count.ShouldBe(3);
        row.DegradedHosts.Select(h => h.HostName).ShouldBe(["plc-b", "plc-c"]);
    }
    /// <summary>
    ///     <see cref="HostState.Unknown"/> counts as degraded. A probe that has not completed its first tick
    ///     — or one a driver failed to start at all, which AbCip logs explicitly — reports Unknown, and
    ///     rendering that as healthy is how an unstarted probe stays invisible.
    /// </summary>
    [Fact]
    public void Unknown_host_state_counts_as_degraded()
    {
        var snapshot = Snap("MAIN", "drv-a") with
        {
            HostStatuses = [new HostConnectivityStatus("plc-a", HostState.Unknown, When)],
        };
        var row = HostsDriverView.Build([snapshot], nodes: null, instances: null).Single().Drivers.Single();
        row.DegradedHosts.ShouldHaveSingleItem().HostName.ShouldBe("plc-a");
    }
    /// <summary>
    ///     A driver with no probe keeps a null host list — distinct from a probe reporting zero hosts. The
    ///     /hosts column renders "—" for the former and "0 hosts" for the latter, and collapsing them would
    ///     claim every probe-less driver's devices are fine.
    /// </summary>
    [Fact]
    public void A_driver_without_host_statuses_keeps_null_and_reports_no_degraded_hosts()
    {
        var row = HostsDriverView.Build([Snap("MAIN", "drv-a")], nodes: null, instances: null)
            .Single().Drivers.Single();
        row.HostStatuses.ShouldBeNull();
        row.DegradedHosts.ShouldBeEmpty();
    }
 }
@@ -3,6 +3,9 @@ using Shouldly;
 using ZB.MOM.WW.OtOpcUa.Commons.Protos;
 using ZB.MOM.WW.OtOpcUa.Commons.Protos.Telemetry.V1;
 using ZB.MOM.WW.OtOpcUa.ControlPlane.Telemetry;
 // Aliased, not imported wholesale: Core.Abstractions also declares a DriverHealth, which collides with
 // the proto DriverHealth these tests construct.
 using HostState = ZB.MOM.WW.OtOpcUa.Core.Abstractions.HostState;
 using Xunit;
 namespace ZB.MOM.WW.OtOpcUa.ControlPlane.Tests.Telemetry;
@@ -168,6 +171,77 @@ public sealed class TelemetryProtoMapCentralTests
        e.PublishedUtc.Kind.ShouldBe(DateTimeKind.Utc);
    }
    /// <summary>
    ///     Per-host connectivity (Gitea #521) survives the wire with its tri-state intact:
    ///     <b>null</b> (driver has no probe) must stay distinguishable from an <b>empty list</b> (it has one
    ///     that knows no hosts). proto3 cannot tell an absent repeated field from an empty one, which is why
    ///     <c>has_host_statuses</c> exists — without it a driver with no probe would arrive looking like one
    ///     whose devices are all fine, and the /hosts column would render "0 hosts" for every driver in the
    ///     fleet.
    /// </summary>
    [Fact]
    public void ToHealth_host_statuses_round_trip_with_the_null_vs_empty_distinction_intact()
    {
        var populated = new DriverHealth
        {
            ClusterId = "c1", DriverInstanceId = "d1", State = "Healthy",
            PublishedUtc = Timestamp.FromDateTime(SampleUtc),
            HasHostStatuses = true,
            HostStatuses =
            {
                new HostConnectivity { HostName = "plc-a", State = "Running", LastChangedUtc = Timestamp.FromDateTime(OtherUtc) },
                new HostConnectivity { HostName = "plc-b", State = "Stopped", LastChangedUtc = Timestamp.FromDateTime(SampleUtc) },
            },
        };
        var mapped = TelemetryProtoMapCentral.ToHealth(populated).HostStatuses;
        mapped.ShouldNotBeNull();
        mapped!.Count.ShouldBe(2);
        mapped[0].HostName.ShouldBe("plc-a");
        mapped[0].State.ShouldBe(HostState.Running);
        mapped[0].LastChangedUtc.ShouldBe(OtherUtc);
        mapped[1].State.ShouldBe(HostState.Stopped);
        // A probe that currently knows no hosts: empty, NOT null.
        var empty = new DriverHealth
        {
            ClusterId = "c1", DriverInstanceId = "d1", State = "Healthy",
            PublishedUtc = Timestamp.FromDateTime(SampleUtc),
            HasHostStatuses = true,
        };
        TelemetryProtoMapCentral.ToHealth(empty).HostStatuses.ShouldBeEmpty();
        // No probe at all: null, NOT empty.
        var absent = new DriverHealth
        {
            ClusterId = "c1", DriverInstanceId = "d1", State = "Healthy",
            PublishedUtc = Timestamp.FromDateTime(SampleUtc),
        };
        TelemetryProtoMapCentral.ToHealth(absent).HostStatuses.ShouldBeNull();
    }
    /// <summary>
    ///     An unparseable host state degrades to <see cref="HostState.Unknown"/> rather than throwing. A node
    ///     running a newer build that added an enum member must not be able to kill central's telemetry
    ///     stream — this is observability, and it has no business failing closed.
    /// </summary>
    [Fact]
    public void ToHealth_unknown_host_state_string_degrades_instead_of_throwing()
    {
        var proto = new DriverHealth
        {
            ClusterId = "c1", DriverInstanceId = "d1", State = "Healthy",
            PublishedUtc = Timestamp.FromDateTime(SampleUtc),
            HasHostStatuses = true,
            HostStatuses = { new HostConnectivity { HostName = "plc-a", State = "Quiescing" } },
        };
        var mapped = TelemetryProtoMapCentral.ToHealth(proto).HostStatuses;
        mapped.ShouldNotBeNull();
        mapped![0].State.ShouldBe(HostState.Unknown);
    }
    [Fact]
    public void ToHealth_absent_nullable_timestamp_and_optional_string_map_to_null()
    {
@@ -8,6 +8,10 @@ using ZB.MOM.WW.OtOpcUa.Commons.Messages.Logging;
 using ZB.MOM.WW.OtOpcUa.Commons.Protos.Telemetry.V1;
 using ZB.MOM.WW.OtOpcUa.Host.Grpc;
 using ZB.MOM.WW.OtOpcUa.Runtime.Telemetry;
 // Aliased, not imported wholesale: Core.Abstractions also declares a DriverHealth, which would collide
 // with the proto DriverHealth these tests assert on.
 using HostConnectivityStatus = ZB.MOM.WW.OtOpcUa.Core.Abstractions.HostConnectivityStatus;
 using HostState = ZB.MOM.WW.OtOpcUa.Core.Abstractions.HostState;
 namespace ZB.MOM.WW.OtOpcUa.Host.Tests.Grpc;
@@ -210,6 +214,40 @@ public sealed class TelemetryStreamGrpcServiceTests
        evt.DriverHealth.LastSuccessfulReadUtc.ToDateTime().ShouldBe(expectedUtc);
    }
    /// <summary>
    ///     Node side of the #521 host-status carry: the presence flag must be written, so the tri-state
    ///     (no probe / probe with no hosts / probe with hosts) survives a wire proto3 cannot express on the
    ///     repeated field alone. Paired with the central-side decode in <c>TelemetryProtoMapCentralTests</c>.
    /// </summary>
    [Fact]
    public void ToProto_health_writes_the_host_status_presence_flag_and_entries()
    {
        var changedAt = new DateTime(2026, 7, 30, 11, 0, 0, DateTimeKind.Utc);
        var withHosts = new DriverHealthChanged(
            "cluster-a", "drv-1", "Healthy", null, null, 0, DateTime.UtcNow,
            HostStatuses: [new HostConnectivityStatus("plc-a", HostState.Stopped, changedAt)]);
        var evt = TelemetryProtoMapNode.ToProto(new TelemetryItem.Health(withHosts), "c");
        evt.DriverHealth.HasHostStatuses.ShouldBeTrue();
        evt.DriverHealth.HostStatuses.Count.ShouldBe(1);
        evt.DriverHealth.HostStatuses[0].HostName.ShouldBe("plc-a");
        evt.DriverHealth.HostStatuses[0].State.ShouldBe("Stopped");
        evt.DriverHealth.HostStatuses[0].LastChangedUtc.ToDateTime().ShouldBe(changedAt);
        // A probe reporting zero hosts still sets the flag — that is the whole point of having one.
        var emptyProbe = new DriverHealthChanged(
            "cluster-a", "drv-1", "Healthy", null, null, 0, DateTime.UtcNow, HostStatuses: []);
        var emptyEvt = TelemetryProtoMapNode.ToProto(new TelemetryItem.Health(emptyProbe), "c");
        emptyEvt.DriverHealth.HasHostStatuses.ShouldBeTrue();
        emptyEvt.DriverHealth.HostStatuses.ShouldBeEmpty();
        // No probe: flag clear.
        var noProbe = new DriverHealthChanged("cluster-a", "drv-1", "Healthy", null, null, 0, DateTime.UtcNow);
        TelemetryProtoMapNode.ToProto(new TelemetryItem.Health(noProbe), "c")
            .DriverHealth.HasHostStatuses.ShouldBeFalse();
    }
    [Fact]
    public async Task Client_disconnect_mid_stream_ends_cleanly_without_leaking_a_slot()
    {
@@ -0,0 +1,298 @@
 using Akka.Actor;
 using Shouldly;
 using Xunit;
 using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
 using ZB.MOM.WW.OtOpcUa.Runtime.Drivers;
 using ZB.MOM.WW.OtOpcUa.Runtime.Tests.Harness;
 namespace ZB.MOM.WW.OtOpcUa.Runtime.Tests.Drivers;
 /// <summary>
 ///     Covers the <see cref="IHostConnectivityProbe"/> consumer (Gitea #521). Eleven drivers implement the
 ///     capability; before this wiring <c>GetHostStatuses()</c> had ZERO production call sites and
 ///     <c>OnHostStatusChanged</c> had no subscriber outside the Galaxy driver's own aggregator, so per-host
 ///     connectivity was computed by every driver and read by nobody.
 ///     <para><b>Why it does not go to the DB.</b> The <c>DriverHostStatus</c> table's doc-comment described a
 ///     publisher hosted service that upserted rows from each driver node. It was never built, and
 ///     per-cluster mesh Phase 4 made it unbuildable as described — <c>AddOtOpcUaConfigDb</c> is gated on the
 ///     <c>admin</c> role, so a driver-only node has no ConfigDb connection. The table was dropped; the data
 ///     rides the driver-health snapshot instead.</para>
 /// </summary>
 [Trait("Category", "Unit")]
 public sealed class DriverInstanceActorHostStatusTests : RuntimeActorTestBase
 {
    /// <summary>The base case: a probe driver's hosts reach the health publisher.</summary>
    [Fact]
    public void Host_statuses_reach_the_health_publisher()
    {
        var driver = new ProbeStubDriver();
        driver.SetHost("plc-a", HostState.Running);
        driver.SetHost("plc-b", HostState.Running);
        var publisher = new RecordingHealthPublisher();
        var actor = SpawnDriverActor(driver, publisher);
        actor.Tell(new DriverInstanceActor.InitializeRequested("{}"));
        AwaitAssert(
            () =>
            {
                var latest = publisher.Published.LastOrDefault();
                latest.ShouldNotBeNull();
                latest!.HostStatuses.ShouldNotBeNull();
                latest.HostStatuses!.Select(h => h.HostName).OrderBy(n => n, StringComparer.Ordinal)
                    .ShouldBe(["plc-a", "plc-b"]);
            },
            TimeSpan.FromSeconds(3));
    }
    /// <summary>
    ///     <b>The load-bearing case, and the entire reason this channel exists.</b> A multi-device driver
    ///     stays aggregate-<c>Healthy</c> when ONE of its devices drops — the driver-level state chip cannot
    ///     express it. So the transition must reach the operator through the per-host detail.
    ///     <para>This is also the dedup trap that already bit the rediscovery signal once.
    ///     <c>PublishHealthSnapshot</c> suppresses a publish whose fingerprint repeats, and on this
    ///     transition (state, lastSuccessfulRead, lastError, errorCount) are ALL unchanged. Unless the
    ///     host-status digest is part of the fingerprint, the dedup swallows exactly the publish that
    ///     carries the news.</para>
    ///     <para><b>Falsifiability:</b> the assertion is that the publish count STRICTLY INCREASES across the
    ///     transition — an "eventually shows Stopped" assertion would be satisfied by the warm-up publish
    ///     plus a later 30 s heartbeat and would prove nothing. Drop <c>HostStatusDigest</c> from the
    ///     fingerprint tuple in <c>DriverInstanceActor</c> and this test must go red. Verified by doing so.</para>
    /// </summary>
    [Fact]
    public void A_single_host_going_down_is_not_swallowed_by_the_unchanged_health_dedup()
    {
        var driver = new ProbeStubDriver();
        driver.SetHost("plc-a", HostState.Running);
        driver.SetHost("plc-b", HostState.Running);
        var publisher = new RecordingHealthPublisher();
        var actor = SpawnDriverActor(driver, publisher);
        actor.Tell(new DriverInstanceActor.InitializeRequested("{}"));
        AwaitAssert(() => publisher.Published.Count.ShouldBeGreaterThan(0), TimeSpan.FromSeconds(3));
        // Settle, so the baseline is a quiet actor: from here only the host state changes. The driver's
        // OWN health stays Healthy throughout — that is the point.
        ExpectNoMsg(TimeSpan.FromMilliseconds(200));
        var before = publisher.Published.Count;
        driver.SetHost("plc-b", HostState.Stopped, raise: true);
        AwaitAssert(
            () => publisher.Published.Count.ShouldBeGreaterThan(before),
            TimeSpan.FromSeconds(3));
        var latest = publisher.Published[^1];
        latest.Health.State.ShouldBe(DriverState.Healthy, "the driver itself never faulted — only one of its devices did");
        latest.HostStatuses.ShouldNotBeNull();
        latest.HostStatuses!.Single(h => h.HostName == "plc-b").State.ShouldBe(HostState.Stopped);
        latest.HostStatuses.Single(h => h.HostName == "plc-a").State.ShouldBe(HostState.Running);
    }
    /// <summary>
    ///     The other half of the dedup contract: when nothing changes, the digest must NOT churn. A digest
    ///     built over an <c>IReadOnlyList</c> by reference, or one sensitive to the order a driver happens to
    ///     enumerate its hosts in, would differ on every call and re-publish on every 30 s heartbeat forever
    ///     — turning the dedup off without anyone noticing.
    /// </summary>
    [Fact]
    public void Unchanged_hosts_do_not_defeat_the_dedup()
    {
        var driver = new ProbeStubDriver();
        driver.SetHost("plc-a", HostState.Running);
        driver.SetHost("plc-b", HostState.Running);
        var publisher = new RecordingHealthPublisher();
        var actor = SpawnDriverActor(driver, publisher);
        actor.Tell(new DriverInstanceActor.InitializeRequested("{}"));
        AwaitAssert(() => publisher.Published.Count.ShouldBeGreaterThan(0), TimeSpan.FromSeconds(3));
        ExpectNoMsg(TimeSpan.FromMilliseconds(200));
        var before = publisher.Published.Count;
        // The driver re-shuffles its host order without changing any state. A real driver builds this list
        // from a Dictionary, so enumeration order is not guaranteed stable between calls.
        driver.ReverseHostOrder();
        // Poke the actor into re-publishing without changing anything material.
        driver.RaiseHostStatusChanged();
        ExpectNoMsg(TimeSpan.FromMilliseconds(500));
        publisher.Published.Count.ShouldBe(before, "an order flip with no state change must be deduped, not re-published");
    }
    /// <summary>
    ///     <b>Leak guard.</b> The <see cref="IDriver"/> can OUTLIVE the actor — the host respawns a child
    ///     around the same driver object — so a missing <c>-=</c> in <c>PostStop</c> accumulates one handler
    ///     per respawn, each holding a dead <c>Self</c>.
    /// </summary>
    [Fact]
    public void Stopping_the_actor_detaches_the_host_status_handler()
    {
        var driver = new ProbeStubDriver();
        var parent = CreateTestProbe();
        parent.IgnoreMessages(_ => true);
        var actor = parent.ChildActorOf(DriverInstanceActor.Props(driver));
        actor.Tell(new DriverInstanceActor.InitializeRequested("{}"));
        AwaitAssert(() => driver.SubscriberCount.ShouldBe(1), TimeSpan.FromSeconds(3));
        Watch(actor);
        actor.Tell(PoisonPill.Instance);
        ExpectTerminated(actor, TimeSpan.FromSeconds(3));
        driver.SubscriberCount.ShouldBe(0);
    }
    /// <summary>
    ///     A driver with no probe publishes null host statuses — NOT an empty list. The two mean different
    ///     things at the UI ("no per-host detail available" vs "a probe that currently knows no hosts") and
    ///     collapsing them would render a driver with no probe as one whose devices are all fine.
    /// </summary>
    [Fact]
    public void A_driver_without_a_probe_publishes_null_host_statuses()
    {
        var publisher = new RecordingHealthPublisher();
        var actor = SpawnDriverActor(new StubDriver(), publisher);
        actor.Tell(new DriverInstanceActor.InitializeRequested("{}"));
        AwaitAssert(() => publisher.Published.Count.ShouldBeGreaterThan(0), TimeSpan.FromSeconds(3));
        publisher.Published.ShouldAllBe(p => p.HostStatuses == null);
    }
    /// <summary>
    ///     A probe that throws must not take the health publish down with it. <c>GetHostStatuses()</c> is
    ///     documented as a pure in-memory snapshot (which is why the capability analyzer exempts it from
    ///     the guarded-call rule), but a driver is free to violate that, and losing the whole health channel
    ///     for one misbehaving probe would be a much worse outcome than losing the host detail.
    /// </summary>
    [Fact]
    public void A_throwing_probe_degrades_to_null_without_killing_the_health_publish()
    {
        var driver = new ProbeStubDriver { ThrowOnGetHostStatuses = true };
        var publisher = new RecordingHealthPublisher();
        var actor = SpawnDriverActor(driver, publisher);
        actor.Tell(new DriverInstanceActor.InitializeRequested("{}"));
        AwaitAssert(() => publisher.Published.Count.ShouldBeGreaterThan(0), TimeSpan.FromSeconds(3));
        publisher.Published[^1].Health.State.ShouldBe(DriverState.Healthy);
        publisher.Published[^1].HostStatuses.ShouldBeNull();
    }
    private IActorRef SpawnDriverActor(IDriver driver, IDriverHealthPublisher publisher)
    {
        var parent = CreateTestProbe();
        parent.IgnoreMessages(_ => true);
        return parent.ChildActorOf(DriverInstanceActor.Props(driver, healthPublisher: publisher));
    }
    /// <summary>Captures every health publish so a test can assert on the host-status field.</summary>
    private sealed record HealthPublish(
        DriverHealth Health,
        IReadOnlyList<HostConnectivityStatus>? HostStatuses);
    private sealed class RecordingHealthPublisher : IDriverHealthPublisher
    {
        private readonly List<HealthPublish> _published = [];
        /// <summary>Thread-safe snapshot — <c>Publish</c> runs on the actor thread while the test asserts
        /// from its own.</summary>
        public IReadOnlyList<HealthPublish> Published
        {
            get { lock (_published) return _published.ToArray(); }
        }
        /// <inheritdoc />
        public void Publish(
            string clusterId,
            string driverInstanceId,
            DriverHealth health,
            int errorCount5Min,
            DateTime? rediscoveryNeededUtc = null,
            string? rediscoveryReason = null,
            IReadOnlyList<HostConnectivityStatus>? hostStatuses = null)
        {
            lock (_published) _published.Add(new HealthPublish(health, hostStatuses));
        }
    }
    /// <summary>
    ///     A stub driver exposing <see cref="IHostConnectivityProbe"/>.
    ///     <para><b>Its own health is deliberately STABLE</b> (a fixed last-read timestamp), for the same
    ///     reason <c>RediscoverableStubDriver</c> is: the shared <c>StubDriver</c> returns
    ///     <c>DateTime.UtcNow</c> from <c>GetHealth()</c>, so its fingerprint differs on every call, the
    ///     dedup never engages, and any test built on it passes vacuously whether or not the fix is
    ///     present.</para>
    /// </summary>
    private sealed class ProbeStubDriver : IDriver, IHostConnectivityProbe
    {
        private static readonly DateTime FixedLastRead = new(2026, 7, 30, 12, 0, 0, DateTimeKind.Utc);
        private static readonly DateTime FixedChangedAt = new(2026, 7, 30, 11, 0, 0, DateTimeKind.Utc);
        private readonly List<HostConnectivityStatus> _hosts = [];
        /// <summary>When set, <see cref="GetHostStatuses"/> throws — the misbehaving-probe case.</summary>
        public bool ThrowOnGetHostStatuses { get; init; }
        /// <inheritdoc />
        public event EventHandler<HostStatusChangedEventArgs>? OnHostStatusChanged;
        /// <inheritdoc />
        public string DriverInstanceId => "probe-stub-1";
        /// <inheritdoc />
        public string DriverType => "Stub";
        /// <summary>Number of live subscribers on <see cref="OnHostStatusChanged"/>.</summary>
        public int SubscriberCount => OnHostStatusChanged?.GetInvocationList().Length ?? 0;
        /// <summary>Adds or updates a host, optionally raising the transition event as a real probe loop does.</summary>
        public void SetHost(string hostName, HostState state, bool raise = false)
        {
            lock (_hosts)
            {
                var index = _hosts.FindIndex(h => h.HostName == hostName);
                var entry = new HostConnectivityStatus(hostName, state, FixedChangedAt);
                if (index >= 0) _hosts[index] = entry;
                else _hosts.Add(entry);
            }
            if (raise) RaiseHostStatusChanged();
        }
        /// <summary>Flips enumeration order without changing any host's state.</summary>
        public void ReverseHostOrder()
        {
            lock (_hosts) _hosts.Reverse();
        }
        /// <summary>Raises the event exactly as a real probe loop does.</summary>
        public void RaiseHostStatusChanged()
            => OnHostStatusChanged?.Invoke(this, new HostStatusChangedEventArgs("plc-b", HostState.Running, HostState.Stopped));
        /// <inheritdoc />
        public IReadOnlyList<HostConnectivityStatus> GetHostStatuses()
        {
            if (ThrowOnGetHostStatuses) throw new InvalidOperationException("probe is broken");
            lock (_hosts) return _hosts.ToArray();
        }
        /// <inheritdoc />
        public Task InitializeAsync(string driverConfigJson, CancellationToken cancellationToken) => Task.CompletedTask;
        /// <inheritdoc />
        public Task ReinitializeAsync(string driverConfigJson, CancellationToken cancellationToken) => Task.CompletedTask;
        /// <inheritdoc />
        public Task ShutdownAsync(CancellationToken cancellationToken) => Task.CompletedTask;
        /// <inheritdoc />
        public DriverHealth GetHealth() => new(DriverState.Healthy, FixedLastRead, null);
        /// <inheritdoc />
        public long GetMemoryFootprint() => 0;
        /// <inheritdoc />
        public Task FlushOptionalCachesAsync(CancellationToken cancellationToken) => Task.CompletedTask;
    }
 }
@@ -185,7 +185,8 @@ public sealed class DriverInstanceActorRediscoverySignalTests : RuntimeActorTest
            DriverHealth health,
            int errorCount5Min,
            DateTime? rediscoveryNeededUtc = null,
-            string? rediscoveryReason = null)
+            string? rediscoveryReason = null,
            IReadOnlyList<HostConnectivityStatus>? hostStatuses = null)
        {
            lock (_published)
            {
@@ -122,7 +122,8 @@ public sealed class DriverInstanceActorSubscriptionReconcileTests : RuntimeActor
            DriverHealth health,
            int errorCount5Min,
            DateTime? rediscoveryNeededUtc = null,
-            string? rediscoveryReason = null)
+            string? rediscoveryReason = null,
            IReadOnlyList<HostConnectivityStatus>? hostStatuses = null)
            => Interlocked.Increment(ref _count);
    }
 }