lmxopcua/docs/plans/2026-06-18-driver-pages-reconnect-e2e-design.md

# Driver-pages Phase 10 — reconnect-transition E2E + plan close-out — Design

**Date:** 2026-06-18
**Status:** Approved
**Base:** master `08c7a2bd`
**Branch:** `feat/driver-pages-reconnect-e2e`

## Context

The `2026-05-28-adminui-driver-pages` plan is **fully shipped** (Phases 0–10), but its
`.tasks.json` is comprehensively stale (it still marks Phases 6–10 "pending" while the
code + tests all exist and are real). A brainstorming pass on 2026-06-18 verified, seam
by seam, that:

- **Phase 6** (live `DriverStatusPanel`) is built end-to-end: `DriverHealthChanged`
  contract, `AkkaDriverHealthPublisher` (DI-bound in prod, invoked at
  `DriverInstanceActor.PublishHealthSnapshot`), `DriverStatusSignalRBridge` (spawned
  admin-gated at `Program.cs:196`), the shared-singleton snapshot store, the hub
  (`MapHub<DriverStatusHub>`), and the panel wired into all 9 driver pages. Page
  `DriverInstanceId` and actor `_driverInstanceId` key on the same EF value — no mismatch.
- **Phase 8** (Reconnect/Restart) is built: messages, `AdminOperationsActor` +
  `DriverHostActor` handlers, DriverOperator-gated buttons in the panel.
- **Phase 10** automated E2E tests (`DriverTestConnectE2eTests`, `DriverReconnectE2eTests`,
  `DriverStatusHubE2eTests`) are real, skip-gated, and honest about their scope.

Two genuine remnants remain, both flagged by the `DriverReconnectE2eTests` /
`DriverStatusHubE2eTests` scope-notes:

1. **The one real coverage gap:** no test proves a *deployed* driver actually transitions
   **Healthy → Reconnecting → Healthy** in response to `ReconnectDriver`. The existing
   10.2 test only asserts command *ingestion* (the round-trip reply), not the resulting
   actor health transition.
2. **A harness-fidelity gap behind it:** `TwoNodeClusterHarness.BuildNodeAsync` calls
   `WithOtOpcUaRuntimeActors()` (the Akka actor spawn) but **not** `AddOtOpcUaRuntime()`
   (the DI registration that binds `IDriverHealthPublisher → AkkaDriverHealthPublisher`).
   Consequently, in *every* current `Host.IntegrationTests`, deployed driver actors fall
   back to `NullDriverHealthPublisher` and emit no health to the `driver-health` DPS topic.
   The harness also leaves `IDriverFactory` at `NullDriverFactory`, so deployed drivers
   reach `Stubbed`, never `Connected`.

The plan's stale trackers caused this item to be (wrongly) re-listed as OPEN in
`stillpending.md` §A.9. This phase closes both remnants and reconciles the trackers.

## Goal

Close the genuine `ReconnectDriver` health-transition E2E gap, fix the harness-fidelity gap
behind it, prove the full Phase 10 driver suite green, and reconcile the stale trackers so
this fully-shipped plan stops re-triggering as backlog.

## Design

### 1. Harness fidelity fix

In `TwoNodeClusterHarness.BuildNodeAsync`:

- Add `builder.Services.AddOtOpcUaRuntime()` **before** `AddAkka` (matching production
  `Program.cs:87`). This binds the real `AkkaDriverHealthPublisher` so deployed drivers
  publish health to the `driver-health` DPS topic. It also seeds the `Null*` runtime
  defaults (`IHistorianDataSource`, `IAlarmHistorianSink`, `IHistoryWriter`,
  `IDriverFactory`, …) — all harmless no-ops that don't change existing test behavior
  (nothing in the current suite subscribes to driver-health, and the Null sinks are inert).
- Add an **opt-in** seam to inject a test `IDriverFactory` for tests that need a connecting
  driver. Default (no factory supplied) leaves the existing behavior untouched. Mechanism:
  a `StartAsync` parameter (e.g. `IDriverFactory? driverFactory = null`) threaded into
  `BuildNodeAsync`; when supplied, register it as a singleton **after** `AddOtOpcUaRuntime`
  so it wins over the `Null` default (last-registration-wins / replace).

This change is fidelity-improving for the whole suite: the existing `DriverStatusHubE2eTests`
keeps spawning its own bridge, but real driver health now flows in tests that deploy drivers.

### 2. The reconnect-transition E2E test (the real gap)

A new test (in `DriverReconnectE2eTests.cs`, or a focused new file) that:

1. Starts the harness with a **controllable fake `IDriverFactory`** (see decision below).
2. Seeds a driver row + minimal equipment/tag using the existing
   `SeedDeploymentWithEquipmentTags` precedent (from `DriverHostActorLiveValueTests`),
   bound to `NodeANodeId`.
3. Triggers a deploy (`DispatchDeployment`, or `POST /api/deployments` with
   `HarnessDeployApiKey`).
4. Spawns the **real** `DriverStatusSignalRBridge` over the real DI snapshot store (the
   store is the observation surface; a mock `IHubContext` captures the hub push the same
   way the existing hub test does).
5. Waits (condition-poll, generous timeout) for the snapshot store to report the deployed
   instance as `Healthy`.
6. Dispatches `ReconnectDriver` via `IAdminOperationsClient` (the real cluster-singleton
   path the AdminUI button uses).
7. Asserts the store observes the transition **`Reconnecting`** and then returns to
   **`Healthy`** within a timeout — proving the full wiring:
   `ReconnectDriver → AdminOperationsActor → DriverHostActor.HandleReconnectDriver →
   DriverInstanceActor FSM (ForceReconnect → Become(Reconnecting) → Become(Connected)) →
   PublishHealthSnapshot → driver-health DPS topic → DriverStatusSignalRBridge → store`.

#### Decision: controllable fake driver factory (not the real Modbus sim)

**Recommended and approved:** observe the transition via a deterministic, controllable fake
`IDriver` / `IDriverFactory` test double rather than a real Modbus sim connection.

Rationale:

- **Determinism, no flakiness.** A fake driver whose connect succeeds drives the actor to
  `Connected` immediately; `ReconnectDriver` re-enters `Reconnecting` then `Connected`
  deterministically. No sim timing, no skip-gate, runs everywhere.
- **Smaller blast radius.** The real-sim path additionally needs the full driver-factory
  bootstrap (all 9 driver factories + deps) wired into the shared harness.
- **The wiring is what matters.** This gap is about the *health-transition + command
  wiring*, not the Modbus protocol. The real Modbus TCP connect/reconnect is already
  covered by the `Modbus.IntegrationTests` and the 10.1 `TestConnect` E2E (against the sim).

The fake `IDriver` exposes a minimal controllable surface (succeed-connect, optionally
signal a fault) sufficient to walk the FSM through `Connected → Reconnecting → Connected`.

### 3. Live suite run

Run the full `Host.IntegrationTests` driver E2E suite. Bring up the Modbus sim
(`lmxopcua-fix up modbus standard`, endpoint `10.100.0.35:5020` / `MODBUS_SIM_ENDPOINT`) so
the skip-gated 10.1 `DriverTestConnectE2eTests` actually execute green (not skipped),
alongside the new deterministic reconnect test (which runs regardless of the sim).

### 4. Reconcile the stale trackers

- `docs/plans/2026-05-28-adminui-driver-pages-plan.md.tasks.json`: mark Phases 6–10 tasks
  `completed` with their real commits / a "shipped — reconciled 2026-06-18" note; flip
  `executionState`/`lastUpdated`.
- `stillpending.md` §A.9: mark Phase 6/8/10 SHIPPED (note the new reconnect-transition test;
  keep the deferred full-stack hub test as a documented follow-up). *(Never staged — local
  working file.)*
- `docs/plans/2026-05-28-adminui-driver-pages-design.md` §8.3: fix the stale `ModbusTcp` →
  `Modbus` reference in the smoke checklist.
- Memory: update `project_stillpending_backlog.md` + `MEMORY.md`.

### 5. Explicitly deferred (documented follow-up)

The **full-stack WebSocket + JWT `DriverStatusHub` connection test** (a real `HubConnection`
to `/hubs/driverstatus` with a minted bearer token, `JoinDriver`, assert client receipt).
No repo precedent (no test mints a JWT or opens a real `HubConnection`), flaky-prone, and it
only re-covers transport the mock-hub test + the §8.3 manual runbook already handle.

## Out of scope

- The 10.4 manual browser smoke (driving the AdminUI on docker-dev). Foldable later; the
  automated reconnect test + green suite is the higher-value core.
- Any production code change. This phase is test + harness + docs only.

## Constraints

- xUnit + Shouldly. **No bUnit.**
- **No** EF migration, **no** Commons wire/proto contract change, **no**
  Core.Abstractions / interface contract change.
- Stage by explicit path; never `git add .`; never stage `sql_login.txt`,
  `src/Server/ZB.MOM.WW.OtOpcUa.Host/pki/`, `pending.md`, `current.md`,
  `docker-dev/docker-compose.yml`, `stillpending.md`.
- No force-push, no `--no-verify`.

## Finish

Merge to master + push.