dohertj2/lmxopcua
1
0
Fork 0
Code Issues 12 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

fix(health): bump to ZB.MOM.WW.Health 0.2.1 — Traefik leader-pinning now works

Browse Source 
0.2.1 makes the shared ActiveNodeHealthCheck return Unhealthy (503), not
Degraded (200), for a node that carries the role but is not the role leader.
That is what the shared health spec always specified, what docs/ServiceHosting.md,
docs/v2/Architecture-v2.md and the 2026-05-26 alignment design all describe
("200 only on the Admin-role leader; 503 elsewhere"), and what
docker-dev/traefik-dynamic.yml + scripts/install/traefik-dynamic.yml assume when
they use /health/active as the load-balancer probe.

The implementation returned Degraded, which MapZbHealth maps to 200, so BOTH
central nodes always passed the probe and Traefik load-balanced the admin UI
across the pair instead of pinning it to the leader. The docs were right; the
code was wrong. Closes the Traefik half of #494.

No OtOpcUa code change — package pin only.

Live-verified on docker-dev, with the MAIN pair genuinely formed (memberCount 2,
both members naming one leader):

  central-1 (leader)  /health/active -> 200   traefik UP
  central-2 (standby) /health/active -> 503   traefik DOWN

Failover drill (README step 2), which had never actually exercised anything:
stopping central-1 flipped Traefik to central-2 within 20 s and
http://localhost:9200/ answered 200 continuously across the swap — no outage.
Restarting central-1 reclaimed the role-leader and Traefik flipped back.

README corrected on two counts: the drill now states that exactly ONE backend is
UP by design, and step 3 no longer claims central-2 keeps leadership — RoleLeader
is the lowest-address member, so central-1 deterministically reclaims it. Added
the cold-boot caveat that starting both nodes at once can leave each self-forming
a 1-member cluster, in which case both are their own leader and both answer 200.

STILL OPEN in #494: driver-only nodes. The check is scoped to the admin role and
returns Healthy for any node that lacks it, so all four site nodes answer 200 and
the real per-Cluster redundancy Primary (oldest Up driver member, IsDriverPrimary)
is still not what this tier reports.
This commit is contained in:
Joseph Doherty
2026-07-24 11:30:50 -04:00
parent 9ed2251f37
commit 8dd9da7d4d
2 changed files with 8 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker-dev/README.md
+5 -3
View File
@@ -100,9 +100,11 @@ The `-v` drops the SQL volume; remove it to keep ConfigDb state across restarts.
## Failover smoke
1. Watch the Traefik dashboard at `http://localhost:8089`. Both `central-1` and `central-2` should be listed as healthy in the `otopcua-admin` service.
2. `docker compose -f docker-dev/docker-compose.yml stop central-1` — `central-2` should pick up the admin role-leader within ~15 s (Akka split-brain stable-after). Traefik will route traffic to `central-2` once its `/health/active` returns 200.
3. `docker compose -f docker-dev/docker-compose.yml start central-1` — `central-1` rejoins as a follower; `central-2` keeps the leader role until something disturbs it.
1. Watch the Traefik dashboard at `http://localhost:8089`. Exactly ONE of `central-1` / `central-2` is UP in the `otopcua-admin` service — the admin role-leader. The standby is DOWN by design: `/health/active` answers 503 on it, which is how Traefik pins browser traffic to the leader. (Before `ZB.MOM.WW.Health` 0.2.1 the standby answered 200, so both showed UP and this pinning silently never worked — see `lmxopcua#494`.)
2. `docker compose -f docker-dev/docker-compose.yml stop central-1` — `central-2` picks up the admin role-leader within ~15 s (Akka split-brain stable-after) and Traefik flips it to UP. Verified 2026-07-24: the swap completed inside 20 s with **no gap** — `http://localhost:9200/` answered 200 continuously across the failover.
3. `docker compose -f docker-dev/docker-compose.yml start central-1` — `central-1` rejoins and **reclaims** the admin role-leader, and Traefik flips back. It does not stay with `central-2`: Akka's `RoleLeader` is the lowest-address member, so the outcome is deterministic rather than sticky.
> **Cold-boot caveat.** Starting both central nodes simultaneously from stopped can leave each self-forming its own 1-member cluster (self-first seed lists), in which case BOTH are their own role-leader and both answer `/health/active` 200. Check `entries["akka"].data.memberCount` on `/health/ready` — it should read 2. Restarting either node makes it join the survivor.
## Resource limits & dev logging