dohertj2/lmxopcua
1
0
Fork 0
Code Issues 5 Pull Requests Actions Packages Projects Releases Wiki Activity

Warm redundancy tier flip blocked on ScadaBridge#138 (companion of #717) #907

New Issue
Closed
opened 2026-11-13 03:46:33 -05:00 by dohertj2 · 1 comment
dohertj2 commented 2026-11-13 03:46:33 -05:00
Owner
Copy Link
Copy Source

Companion of #717 (plan T11). Warm non-transparent redundancy is on master default-off (OpcUa:Redundancy:Mode=Unset). No tier may set Mode=Warm until dohertj2/ScadaBridge#138 (ServiceLevel-aware failover) is deployed wherever ScadaBridge reads OtOpcUa.

Why: ScadaBridge fails over only on repeated unstable disconnects. A Warm Standby keeps the session healthy and serves ordinary tags BadOutOfService, so a ScadaBridge connection that lands on the Standby would sit on Bad quality and never fail over (spec §5, docs/plans/2026-11-11-warm-redundancy-717-design.md).

Unblock when ScadaBridge#138 is deployed. Then: T13 docker-dev gate (docker-dev/docker-compose.warm.yml), then one non-production pair (rollout in the plan).

https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi

Companion of #717 (plan T11). Warm non-transparent redundancy is on `master` default-off (`OpcUa:Redundancy:Mode=Unset`). No tier may set `Mode=Warm` until **dohertj2/ScadaBridge#138** (ServiceLevel-aware failover) is deployed wherever ScadaBridge reads OtOpcUa. Why: ScadaBridge fails over only on repeated unstable disconnects. A Warm Standby keeps the session healthy and serves ordinary tags `BadOutOfService`, so a ScadaBridge connection that lands on the Standby would sit on Bad quality and never fail over (spec §5, `docs/plans/2026-11-11-warm-redundancy-717-design.md`). Unblock when ScadaBridge#138 is deployed. Then: T13 docker-dev gate (`docker-dev/docker-compose.warm.yml`), then one non-production pair (rollout in the plan). https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
dohertj2 added the type:deploy-gatepriority:p2blockedcross-repo labels 2026-11-13 03:46:33 -05:00
dohertj2 commented 2026-11-13 09:02:46 -05:00
Author
Owner
Copy Link
Copy Source

QA vq03/vq04 is on Mode=Warm since 2026-11-13 13:49Z. Every applicable gate passes, and this closes the tier flip. ScadaBridge#138 is deployed on QA (8e32f353) and was gated live against this pair in the same window.

Record: docs/deploy/2026-11-13-qa-warm-717.md (1862fab6). ScadaBridge side: docs/deployment/deployment-records/2026-11-13-qa-warmcanary-opcua-connection.md (933da568).

Change. Two service-env entries per node: OpcUa__Redundancy__Mode=Warm and OpcUa__PeerApplicationUris__0=<the other node's URI>. Hashes were checked against each node's OpcUa__ApplicationUri; no values were printed. The build is unchanged (76e73e7d).

  • The pair was stopped together (Secondary first, then a wait for the processes to exit, so #908 did not occur) and started vq03 first.
  • Pair outage: 22.8 s.
  • -Verify OK 21/21 after every start.
  • ConfigDb RedundancyMode is already Warm, so there is no mode_mismatch.
Gate Result
Bands Primary 250, Standby 90 (50 while booting). Rejoin reads 50 then 90
Metadata redundancy CLI: Mode Warm, ServerUriArray self-first on each node. Boot banner Mode="Warm" … peers=[urn:wonder-app-vq0N:OtOpcUa]
Standby data Both canaries read BadOutOfService/null on the Standby and are Good on the Primary
Standby write Not exercisable on QA. Both QA tags are read-only, so a typed Write returns BadNotWritable on both nodes, ahead of the Warm check. BadInvalidState stays proven by T13 L10. Part 9 refusal: QA has no condition
LocalDb sync connected, backlog 0, on both
Canary on the Primary Good, about 1/s; script runner Ready
Graceful Primary stop vq03 demotion … ServiceLevel=0 at +0.03 s; vq04 promotion to "Active" (reason=agreed) at +0.18 s (subscription 250 at +0.65 s)
Rejoin vq03 back as Standby at 90 in 11 s. No failback: vq04 stays 250 / warm_role 1

End state: Warm on both nodes. vq04 is Primary (250) and vq03 is Standby (90); the Primary moved during the gate and Warm does not fail back. DEV vd03 stays Unset.

Rollback, on each node:

  1. Run E:\psrem-stage\otopcua-76e73e7d\warm-env.ps1 -Action Unset.
  2. Stop both nodes, then start both.
  3. Check 250/240.

.reg backups are in …\backup\OtOpcUaHost-pre-warm-*.reg.

Follow-up from the gate: ScadaBridge#141. A graceful Primary stop still reaches ScadaBridge through the 66 s connectivity retries, because the 0 level lasts about 6 s, which is shorter than the 10 s window. It does not block Warm.

https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi

**QA vq03/vq04 is on `Mode=Warm` since 2026-11-13 13:49Z. Every applicable gate passes, and this closes the tier flip.** ScadaBridge#138 is deployed on QA (`8e32f353`) and was gated live against this pair in the same window. Record: `docs/deploy/2026-11-13-qa-warm-717.md` (`1862fab6`). ScadaBridge side: `docs/deployment/deployment-records/2026-11-13-qa-warmcanary-opcua-connection.md` (`933da568`). **Change.** Two service-env entries per node: `OpcUa__Redundancy__Mode=Warm` and `OpcUa__PeerApplicationUris__0=<the other node's URI>`. Hashes were checked against each node's `OpcUa__ApplicationUri`; no values were printed. The build is unchanged (`76e73e7d`). - The pair was stopped together (Secondary first, then a wait for the processes to exit, so #908 did not occur) and started vq03 first. - Pair outage: 22.8 s. - `-Verify` OK 21/21 after every start. - ConfigDb `RedundancyMode` is already Warm, so there is no `mode_mismatch`. | Gate | Result | |---|---| | Bands | Primary **250**, Standby **90** (50 while booting). Rejoin reads 50 then 90 | | Metadata | `redundancy` CLI: Mode **Warm**, ServerUriArray self-first on each node. Boot banner `Mode="Warm" … peers=[urn:wonder-app-vq0N:OtOpcUa]` | | Standby data | Both canaries read `BadOutOfService`/null on the Standby and are Good on the Primary | | Standby write | **Not exercisable on QA.** Both QA tags are read-only, so a typed Write returns `BadNotWritable` on **both** nodes, ahead of the Warm check. `BadInvalidState` stays proven by T13 L10. Part 9 refusal: QA has no condition | | LocalDb sync | connected, backlog 0, on both | | Canary on the Primary | Good, about 1/s; script runner Ready | | Graceful Primary stop | vq03 `demotion … ServiceLevel=0` at +0.03 s; vq04 `promotion to "Active" (reason=agreed)` at **+0.18 s** (subscription 250 at +0.65 s) | | Rejoin | vq03 back as Standby at 90 in 11 s. **No failback**: vq04 stays 250 / `warm_role` 1 | **End state:** Warm on both nodes. **vq04 is Primary (250) and vq03 is Standby (90)**; the Primary moved during the gate and Warm does not fail back. DEV vd03 stays Unset. **Rollback**, on each node: 1. Run `E:\psrem-stage\otopcua-76e73e7d\warm-env.ps1 -Action Unset`. 2. Stop both nodes, then start both. 3. Check 250/240. `.reg` backups are in `…\backup\OtOpcUaHost-pre-warm-*.reg`. Follow-up from the gate: ScadaBridge#141. A graceful Primary stop still reaches ScadaBridge through the 66 s connectivity retries, because the 0 level lasts about 6 s, which is shorter than the 10 s window. It does not block Warm. https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
dohertj2 removed the blocked label 2026-11-13 09:03:06 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
auto-managed
queue management
 blocked
Blocked by dependency or external hold
 cross-driver
queue management
 cross-repo
Needs coordinated change in another family repo (wire contract etc.)
 driver/abcip
queue management
 driver/ablegacy
queue management
 driver/focas
queue management
 driver/mqtt
MQTT / Sparkplug B broker-subscribe driver
 driver/opcuaclient
queue management
 driver/s7
queue management
 driver/twincat
queue management
 migrated
Migrated from a markdown backlog (2026-11 standardization)
 phase/1
queue management
 phase/2
queue management
 phase/3
queue management
 phase/4
queue management
 phase/5
queue management
 phase/6
queue management
 priority:p0
Must-have / blocking
 priority:p1
High priority
 priority:p2
Normal priority
 priority:p3
Low priority / nice to have
 queue/blocked
queue management
 queue/done
queue management
 queue/failed
queue management
 queue/in-progress
queue management
 queue/queued
queue management
 type:bug
Defect in shipped behaviour
 type:debt
Technical debt / hygiene
 type:deploy-gate
Rollout/deploy/migration step not yet done on an environment
 type:docs
Documentation work
 type:feature
Feature implementation
 type:followup
Follow-up deferred from a plan, PR or gate
 type:infra
Infrastructure and scaffolding
 type:review-finding
Unfixed finding from an architecture/code review
 type:test
Test work
No Label cross-repo priority:p2 type:deploy-gate
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
dohertj2 (dohertj2)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dohertj2/lmxopcua#907
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?