Files
ScadaBridge/docs/deployment/deployment-records/2026-11-13-qa-warmcanary-opcua-connection.md
T
Joseph Doherty 933da568e3 docs(deploy): record QA WarmCanary OPC UA connection (vq03/vq04 OtOpcUa) and the #138 live gate
Connection 2 / template 27 / instance 16 on site-a, bound to the OtOpcUa script-runner
canaries; serviceLevelFailover at its defaults. Under Warm the connection moved off the
Standby by ServiceLevel (connect-time preference, 0.1 s) and on the booting Standby (50);
a graceful Primary stop still took the connectivity path (66 s, #141). Kept as monitor.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 09:01:19 -05:00

4.0 KiB
Raw Blame History

2026-11-13: QA (site-a) WarmCanary OPC UA connection and the #138 live gate

Owner-approved. This adds a test OPC UA connection on the QA site so that #138 (ServiceLevel-aware failover) has something to act on. QA OtOpcUa was then switched to Warm redundancy (lmxopcua#907/#717). The full gate record, with every timing, is lmxopcua docs/deploy/2026-11-13-qa-warm-717.md.

Build: unchanged, 8e32f353 on both QA nodes (2026-11-13-wonder-app-vq03-vq04-8e32f353.md).

Not touched:

  • no ScadaBridge service was stopped or restarted, and no appsettings or env was changed;
  • the MxGateway connection (id 1), Z28061–Z28069 and ZIPSEN;
  • mxgw and HistorianGateway.

Everything went through the management API (CLI from the Mac, the bundled QA GLAuth user, bearer token).

What was added (central DB ScadaBridge on wonder-sql-vq03)

Object Id Details
Data connection WarmCanary 2 Site 2 (site-a), protocol OpcUa, failover retry count 3. Primary opc.tcp://wonder-app-vq03.zmr.zimmer.com:4840/OtOpcUa, backup opc.tcp://wonder-app-vq04.zmr.zimmer.com:4840/OtOpcUa. Security None, anonymous, auto-accept. serviceLevelFailover not set, so the defaults apply (enabled, 10 s, 60 s). subscriptionDisplayName WarmCanary (set in step 4c to force an in-place reconfigure)
Template WarmCanary 27 CalcSeconds (Double) → ns=2;s=script-runner-canary/Device1/canary-calc-seconds; VtSecond (Double) → ns=3;s=EQ-ec3979550d90/canary-vt-second. These are OtOpcUa's QA script-runner canaries, 1 s timers
Instance WarmCanary 16 Site 2, both attributes bound to connection 2, deployed (deployment 990a693f…, revision sha256:d34e2726…)

Site artifacts were deployed twice to site-a:

  • 414664c8… (13:42:11Z), the create;
  • 627a4803… (13:56:48Z), the 4c reconfigure.

The MxGateway connection was not reconfigured: the hash guard made it a no-op, and there is no log line for it. Its site health stayed Connected, 2196/2196 Good.

#138 results (QA site node vq03 active)

Case Result
Connect on the Healthy Primary (Unset, 250) Good at 1/s. No ServiceLevel line, by design: Healthy samples log nothing
Unset: stop the Primary OtOpcUa Connectivity failover (3 retries) → Backup. Good again 67.1 s after the stop
Pair restart into Warm: the connection reached the booting Standby (50) will check the other endpoint now (connect-time preference) → Failing over from "Backup" to "Primary" (ServiceLevel 50 Degraded for 4s; "Primary" reports 250). Good 8.6 s after the Standby connect
Warm: stop the Primary ServiceLevel 0 seen at once (Degraded Warning), then cleared as unknown when the server closed. The connectivity failover → Backup (promoted, 250). Good 66.3 s after the stop. #141 filed for this case
Old Primary back as Standby (90) No failback. The connection stayed on the 250 node for 110 s with no Bad sample
Reconfigure re-arms on the primary, which is now the Standby at 90, over a healthy session reports ServiceLevel 90 … connect-time preference → Failing over from "Primary" to "Backup" (ServiceLevel 90 Degraded for 0s; "Backup" reports 250). 0.1 s from connect to Good, no disconnect
60 s dwell Not exercisable live: a reconfigure resets it, and a role flip drops the session. No flapping was seen

End state: WarmCanary is on the backup endpoint (vq04, the Warm Primary at 250), 2 Good / 0 Bad, and is kept as the QA Warm monitor. Site health: 10 deployed instances, MxGateway 2196/2196 Good, LocalDb replication connected.

Removal (if the monitor is no longer wanted)

scadabridge instance delete --id 16
scadabridge template delete --id 27
scadabridge data-connection delete --id 2
scadabridge deploy artifacts --site-id 2

The artifact deploy reconciles the site: the DCL actor for WarmCanary is stopped. If OtOpcUa goes back to Mode Unset, the connection can stay as it is, because 250/240 never starts a degraded streak.