docs(plans): R3-03 live gate — PASS on docker-dev (restart re-subscribe resumes values in 0.36 s; pre-fix control at 7c77a8c7 silent for 155 s; F1 widened alarm set re-subscribes 1→2; watchdog no re-trip post-reconnect)

archreview/plans/R3-03-restart-resubscribe-watchdog-plan.md

@@ -770,3 +770,92 @@ so "the second crash tripped supervision" is proven before the backoff is read.
 - T9 (live gate) still **pending** — the orchestrator runs it. Worth noting for that session: F1 is
   directly observable on the rig, by deploying a second alarm-bearing raw tag to a driver that already
   has one and confirming a transition on the NEW tag reaches `/alerts`.
+
+---
+
+## Live gate result (2026-08-20) — **PASS** on docker-dev (before-fix control reproduced the silence first)
+
+Rig: `docker-dev/docker-compose.yml` on OrbStack (Mac), the six host containers plus SQL + Traefik, AdminUI on
+`Security__Auth__DisableLogin=true` (auto-authenticated admin, no sign-in). Deploys via `./docker-dev/deploy.sh`
+(`POST /api/deployments`, `X-Api-Key: docker-dev-deploy-key`, `202` every time). Modbus fixture
+`otopcua-pymodbus-standard` on `10.100.0.35:5020` (left running throughout — R3-02's gate owns it).
+Client.CLI `subscribe … -i 500` against `opc.tcp://localhost:4840` (central-1). **The seed's `HR200` is a static
+scratch register, so a second raw tag `HR100` (`MAIN-modbus-hr100`, the fixture's auto-increment HR[100]) was
+added by SQL and deployed — a subscription on a register that never changes cannot distinguish "resumed" from
+"silent".** Both image swaps asserted parity on all six containers via `docker inspect --format '{{.Image}}'`.
+
+### After-fix run — image `otopcua-host:dev` = `b344565cad3d` (master `e280db37`, contains the five R3 fixes) ×6
+
+**S16 leg (operator Restart → values resume): PASS.**
+
+- Baseline: `HR100` streaming at 1/s (`38592 → 38762`, `16:48:22Z → 16:49:48.02Z`).
+- AdminUI `/hosts → pymodbus (MAIN-modbus) → Controls → Restart → Confirm restart`. Both centrals logged
+  `[16:49:48 INF] DriverHost central-N:4053: restarting driver MAIN-modbus by request of multi-role-test`,
+  then on each: `spawned Modbus driver MAIN-modbus` → `driver MAIN-modbus disconnected — 2 value node(s) marked
+  BadNoCommunication (value retained)` (the R3-02 mark) → `DriverInstance MAIN-modbus: connected` →
+  **`subscribed to 2 refs (poll-sub-1)`** — all in the same second. That last line is the T3 spawn-time desired-set
+  push; it is absent in the control below.
+- CLI: last pre-restart value `38762 @ 16:49:48.0216Z`, **first post-restart value `38764 @ 16:49:48.3780Z`** — a
+  0.36 s gap — then `38766, 38768, …` at 1/s with fresh server timestamps for the rest of the session (130 lines,
+  every one `0x00000000`). **Values RESUMED, not merely "driver reads Connected".**
+- The R3-02 `BadNoCommunication` flash (brief's note): the mark and the reconnect landed inside one second, and at
+  the CLI's 500 ms sampling no Bad-status notification was delivered — the client saw only the Good stream with a
+  0.36 s gap. Recorded as: no visible Bad flash at this cadence; the server-side mark is in the log.
+
+**S17 leg (watchdog grace after the restart, log evidence only — stop-the-fixture control deliberately NOT run, R3-02
+owns it): PASS as far as this leg can show.**
+
+- `grep -ciE "comms lost|DisconnectObserved|Reconnecting"` on both centrals over the window `16:49:48 → 16:53:42`
+  (> 2 × `DefaultCommsLossTimeout` after the reconnect): **0 and 0**. Health snapshots over the same window
+  (`DriverHealthChanged … State = Healthy, LastSuccessfulReadUtc = 16:50:47 / 16:51:17`) — reads were succeeding,
+  so no trip was expected and none happened. This proves the watchdog did not re-trip on a freshly-reconnected
+  driver; it does **not** exercise the `??`→max change itself (that needs a non-Healthy driver post-reconnect — the
+  churn/positive-control pair stays owned by R3-02's fixture-stop gate, and by the T1 unit pair).
+
+**F1 leg (a widened alarm set re-subscribes): PASS, on the establishment line; the transition-to-`/alerts` half could
+not be exercised on this rig.**
+
+- The rig has no native-alarm-capable driver (Modbus is not an `IAlarmSource`; the `IAlarmSource` drivers are
+  OpcUaClient / AbCip / FOCAS / Galaxy). An **OpcUaClient** driver `MAIN-opcplc` was authored by SQL against the
+  shared opc-plc fixture `opc.tcp://10.100.0.35:50000` (`TargetNamespaceKind=SystemPlatform`, `AutoAcceptCertificates`)
+  with one alarm-bearing raw tag (`TagConfig.alarm = {alarmType, severity}`), deployed (`c7267b77…`, Sealed):
+  `[16:47:46 INF] DriverInstance MAIN-opcplc: native-alarm subscription established for 1 alarm ref(s) (opcua-alarm-sub-1)`.
+- Second alarm-bearing tag added, deployed (`72a18d67…`, Sealed 16:53:50.513): both centrals
+  `[16:53:50 INF] DriverHost: stopped driver child MAIN-opcplc` → `spawned` → `connected` →
+  **`native-alarm subscription established for 2 alarm ref(s) (opcua-alarm-sub-1)`** — the new set, in one connect
+  cycle. Before F1 the second deploy would have left the handle on the 1-ref set.
+- The alarm-bearing delta forced a full rebuild, yet the Modbus `HR100` CLI stream ran through it unbroken
+  (`16:53:49.47Z → 16:53:50.50Z → …`, no gap, no Bad).
+- **Not shown:** a transition on the new tag reaching `/alerts`. The OpcUaClient driver's event handler filters on
+  `SourceNode` against the ref set it was handed, and opc-plc's `--alm` simulation raises from its own
+  `DeviceHealthAlarms` objects, so no upstream event matches an authored tag — nothing reached `/alerts` for either
+  tag, before or after the widening. That is a fixture/driver-filter mismatch, not an R3-03 regression; the
+  re-subscribe itself (F1's defect) is what the 1→2 line evidences. `A_deploy_that_widens_the_alarm_set_re_subscribes_the_new_refs`
+  covers the handle semantics offline.
+
+**Supervision leg:** not provokable from an operator surface — **not live-gated, by design**; covered by
+`Supervision_restart_reinitialises_and_re_subscribes_the_child` and the F2/F3 tests in `DriverHostActorRestartTests`
+(10 cases, green at merge).
+
+### Before-fix control — image `otopcua-host:prefix-r303` = `32a2dcee0285`, built from a scratch worktree at `7c77a8c7` (pre-tranche master) ×6 — **FAILED as required**
+
+Tagged over `otopcua-host:dev`, all six containers recreated, parity `32a2dcee0285` ×6. `site-b-2` was put in
+`ClusterNode.MaintenanceMode=1` for the control only (the R3-04 standing pin makes a pre-R3-04 build mount no driver
+stack on that node); the centrals replayed the last applied deployment (`72a18d67…`) from SQL on bootstrap
+(`recovered Applied state at rev 57251f4b…`, `SubscribeBulk pushed 2 references`), so no new deploy was needed.
+
+- Baseline on the control: `HR100` streaming (`39474 → 39596`, `16:56:11Z → 16:57:12.35Z`).
+- Same `/hosts` Restart at `16:57:12`. Both centrals: `restarting driver MAIN-modbus by request of multi-role-test` →
+  `spawned Modbus driver MAIN-modbus (stub=False)` → `DriverInstance MAIN-modbus: connected` — and **nothing else**:
+  no `subscribed to N refs`, no `SubscribeBulk` (count of both over the following 3 min: **0**).
+- CLI: **last value `39596 @ 16:57:12.3468Z`, then silence for the remaining 155 s of the session** (checked at
+  16:59:47) — no value, no Bad status (the pre-R3-02 build has no `BadNoCommunication` mark either, so the client held
+  a stale Good forever — the exact 03/S15+S16 shape). `/hosts` kept showing the driver Healthy throughout.
+
+### Restore
+
+`docker tag b344565cad3d otopcua-host:dev`, all six recreated → parity `b344565cad3d` ×6; `MaintenanceMode` cleared
+on `site-b-2:4053`; the scratch worktree removed; the `otopcua-host:prefix-r303` image tag left behind for re-runs.
+The gate's `MAIN-opcplc` driver + its two alarm tags were deleted again and a final deploy (`1c666be3…`, Sealed)
+returned the rig to the seed plus the one extra `HR100` raw tag (kept — it is the only changing Modbus register and
+the next gate will want it). Rig left UP on the master image with `DisableLogin=true`.

archreview/plans/R3-03-restart-resubscribe-watchdog-plan.md.tasks.json

@@ -63,7 +63,7 @@
     {
       "id": "T9",
       "subject": "LIVE GATE: docker-dev restart re-subscribe (before/after) + watchdog churn cadence with stop-the-fixture control",
-      "status": "pending",
+      "status": "done",
       "blockedBy": [
         "T3",
         "T5",