docs(deploy): record QA WarmCanary OPC UA connection (vq03/vq04 OtOpcUa) and the #138 live gate

Connection 2 / template 27 / instance 16 on site-a, bound to the OtOpcUa script-runner canaries; serviceLevelFailover at its defaults. Under Warm the connection moved off the Standby by ServiceLevel (connect-time preference, 0.1 s) and on the booting Standby (50); a graceful Primary stop still took the connectivity path (66 s, #141). Kept as monitor. Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 09:01:19 -05:00
parent 89d2becba9
commit 933da568e3
1 changed files with 56 additions and 0 deletions
@@ -0,0 +1,56 @@
+# 2026-11-13: QA (`site-a`) `WarmCanary` OPC UA connection and the #138 live gate
+
+**Owner-approved.** This adds a test OPC UA connection on the QA site so that #138 (ServiceLevel-aware failover) has
+something to act on. QA OtOpcUa was then switched to Warm redundancy (lmxopcua#907/#717). The full gate record, with
+every timing, is lmxopcua `docs/deploy/2026-11-13-qa-warm-717.md`.
+
+**Build:** unchanged, `8e32f353` on both QA nodes (`2026-11-13-wonder-app-vq03-vq04-8e32f353.md`).
+
+**Not touched:**
+- no ScadaBridge service was stopped or restarted, and no appsettings or env was changed;
+- the `MxGateway` connection (id 1), Z28061–Z28069 and ZIPSEN;
+- mxgw and HistorianGateway.
+
+Everything went through the management API (CLI from the Mac, the bundled QA GLAuth user, bearer token).
+
+## What was added (central DB `ScadaBridge` on wonder-sql-vq03)
+
+| Object | Id | Details |
+|---|---|---|
+| Data connection `WarmCanary` | 2 | Site 2 (`site-a`), protocol `OpcUa`, failover retry count 3. **Primary** `opc.tcp://wonder-app-vq03.zmr.zimmer.com:4840/OtOpcUa`, **backup** `opc.tcp://wonder-app-vq04.zmr.zimmer.com:4840/OtOpcUa`. Security None, anonymous, auto-accept. `serviceLevelFailover` not set, so the defaults apply (enabled, 10 s, 60 s). `subscriptionDisplayName` `WarmCanary` (set in step 4c to force an in-place reconfigure) |
+| Template `WarmCanary` | 27 | `CalcSeconds` (Double) → `ns=2;s=script-runner-canary/Device1/canary-calc-seconds`; `VtSecond` (Double) → `ns=3;s=EQ-ec3979550d90/canary-vt-second`. These are OtOpcUa's QA script-runner canaries, 1 s timers |
+| Instance `WarmCanary` | 16 | Site 2, both attributes bound to connection 2, deployed (deployment `990a693f…`, revision `sha256:d34e2726…`) |
+
+Site artifacts were deployed twice to `site-a`:
+- `414664c8…` (13:42:11Z), the create;
+- `627a4803…` (13:56:48Z), the 4c reconfigure.
+
+The `MxGateway` connection was not reconfigured: the hash guard made it a no-op, and there is no log line for it. Its
+site health stayed `Connected`, 2196/2196 Good.
+
+## #138 results (QA site node vq03 active)
+
+| Case | Result |
+|---|---|
+| Connect on the Healthy Primary (Unset, 250) | Good at 1/s. No ServiceLevel line, by design: Healthy samples log nothing |
+| Unset: stop the Primary OtOpcUa | Connectivity failover (3 retries) → Backup. Good again **67.1 s** after the stop |
+| Pair restart into Warm: the connection reached the **booting Standby** (50) | `will check the other endpoint now (connect-time preference)` → `Failing over from "Backup" to "Primary" (ServiceLevel 50 Degraded for 4s; "Primary" reports 250)`. Good 8.6 s after the Standby connect |
+| Warm: stop the Primary | ServiceLevel **0** seen at once (Degraded Warning), then cleared as unknown when the server closed. The connectivity failover → Backup (promoted, 250). Good **66.3 s** after the stop. **#141** filed for this case |
+| Old Primary back as Standby (90) | No failback. The connection stayed on the 250 node for 110 s with no Bad sample |
+| Reconfigure re-arms on the primary, which is now the **Standby at 90**, over a healthy session | `reports ServiceLevel 90 … connect-time preference` → `Failing over from "Primary" to "Backup" (ServiceLevel 90 Degraded for 0s; "Backup" reports 250)`. **0.1 s** from connect to Good, no disconnect |
+| 60 s dwell | Not exercisable live: a reconfigure resets it, and a role flip drops the session. No flapping was seen |
+
+**End state:** `WarmCanary` is on the backup endpoint (vq04, the Warm Primary at 250), 2 Good / 0 Bad, and is kept as the
+QA Warm monitor. Site health: 10 deployed instances, `MxGateway` 2196/2196 Good, LocalDb replication connected.
+
+## Removal (if the monitor is no longer wanted)
+
+```
+scadabridge instance delete --id 16
+scadabridge template delete --id 27
+scadabridge data-connection delete --id 2
+scadabridge deploy artifacts --site-id 2
+```
+
+The artifact deploy reconciles the site: the DCL actor for `WarmCanary` is stopped. If OtOpcUa goes back to `Mode`
+Unset, the connection can stay as it is, because 250/240 never starts a degraded streak.