docs(health): oldest-Up-member is the active-node rule (0.3.0)

Rewrites the active-node sections of the spec and shared contract, which
described the leader-based design that both consumers rejected.

- SPEC §2.3 replaced: the rule, the ordered RolePreference, why a node holding
  none of the preferred roles is Unhealthy rather than Healthy, and the data
  payload. §4 (IActiveNodeGate) and the tier table follow.
- SPEC §6 marked historical: it still tells adopters to use
  ActiveNodeHealthCheck(role: "admin") and AkkaActiveNodeGate, which is the
  advice that produced the defect. Kept for provenance, annotated so nobody
  follows it again.
- GAPS: gaps L1/L2 recorded as closed the wrong way, reopened, then closed at
  0.3.0 — including the part worth remembering, that both apps adopted the
  shared check as specified, hit it live, and reverted to private copies, so the
  surface reached zero consumers by being avoided rather than unused.
- Index rows updated with the consumer-visible changes and both app branches.
This commit is contained in:
Joseph Doherty
2026-07-24 13:24:30 -04:00
parent a61e041e58
commit 6a4fa220de
5 changed files with 139 additions and 49 deletions
+1 -1
View File
@@ -143,7 +143,7 @@ each project's **code-verified current state**, and the **gaps** between. See
|---|---|---|---|---|
| Auth (login / identity / authz) | Adopted (lib `0.1.4`; all 3 apps, merged to **local default** main/master + **pushed to origin** (gitea)) | Shared `ZB.MOM.WW.Auth` lib | [`components/auth/`](components/auth/) | [`ZB.MOM.WW.Auth/`](ZB.MOM.WW.Auth/) |
| UI Theme (layout / tokens / components) | Adopted (feed `0.2.0``0.3.1`; **all 4 apps pinned `0.3.1`**, pushed to origin) | Shared `ZB.MOM.WW.Theme` RCL | [`components/ui-theme/`](components/ui-theme/) | [`ZB.MOM.WW.Theme/`](ZB.MOM.WW.Theme/) |
| Health (readiness / liveness / active-node) | **Adopted** (feed carries `0.1.0`/`0.2.0`/`0.2.1`; **all 4 apps** reference it + wire `MapZbHealth`). **`0.2.0` (2026-07-24)** added an optional per-entry `data` object to the canonical health JSON and made `AkkaClusterHealthCheck` publish its cluster view (`leader`/`selfAddress`/`selfRoles`/`memberCount`/`unreachableCount`) — additive, so pre-0.2.0 payloads still parse. **`0.2.1` (2026-07-24) is a behaviour fix with a live consumer impact:** the active tier now returns **Unhealthy (503)** on a role-member-but-not-leader node instead of `Degraded`, which the spec maps to 200. The old value made `/health/active` answer 200 on *every* node, so it could not distinguish active from standby over HTTP at all — and OtOpcUa's Traefik routes the admin UI by exactly this probe, so its leader-pinning had silently never worked ([`lmxopcua#494`](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/494)). **Only OtOpcUa is affected** — it is the sole user of the role-filtered mode; ScadaBridge deliberately uses its own `OldestNodeActiveHealthCheck` (leadership is address-ordered and diverges from the singleton host), and mxgw/HistorianGateway have no Akka cluster. | Shared `ZB.MOM.WW.Health` lib | [`components/health/`](components/health/) | [`ZB.MOM.WW.Health/`](ZB.MOM.WW.Health/) |
| Health (readiness / liveness / active-node) | **Adopted** (feed carries `0.1.0`/`0.2.0`/`0.2.1`; **all 4 apps** reference it + wire `MapZbHealth`). **`0.2.0` (2026-07-24)** added an optional per-entry `data` object to the canonical health JSON and made `AkkaClusterHealthCheck` publish its cluster view (`leader`/`selfAddress`/`selfRoles`/`memberCount`/`unreachableCount`) — additive, so pre-0.2.0 payloads still parse. **`0.2.1` (2026-07-24) is a behaviour fix with a live consumer impact:** the active tier now returns **Unhealthy (503)** on a role-member-but-not-leader node instead of `Degraded`, which the spec maps to 200. The old value made `/health/active` answer 200 on *every* node, so it could not distinguish active from standby over HTTP at all — and OtOpcUa's Traefik routes the admin UI by exactly this probe, so its leader-pinning had silently never worked ([`lmxopcua#494`](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/494)). **`0.3.0` (2026-07-24) finished that fix and ENDED a family-wide duplication.** 0.2.1 made the tier a real 503 but still selected by `RoleLeader`; on a rebuilt rig the two orderings diverge live (akka leader `central-1`, oldest admin `central-2` — the singleton host), so it would have pinned Traefik to the node *not* hosting the work. **Leadership is address-ordered (host, then port) and has no relationship to time; singleton placement is age-ordered.** They agree on a freshly-formed cluster — which is why single-node and happy-path tests never caught it — and diverge permanently after any restart. Both Akka apps had already discovered this independently and each written a private replacement rather than use the shared check (ScadaBridge `OldestNodeActiveHealthCheck`, OtOpcUa `ClusterPrimaryHealthCheck`), so the package's entire active-node surface had **zero consumers — not unused but avoided, twice, for the same reason.** 0.3.0 promotes those two into one `ClusterActiveNode` primitive (oldest `Up` member, optional role scope, plus an ordered role-preference so a fused node answers for the role its singletons are pinned to); `ActiveNodeHealthCheck` and `AkkaActiveNodeGate` both delegate to it, and results now carry `activeRole`/`selfAddress`/`activeNode` in the 0.2.0 `data` object so a **standby reports who IS active**. Two further behaviour changes: a node holding **none** of the preferred roles is now **Unhealthy**, not Healthy — the old "not applicable ⇒ Healthy" is exactly what made the tier answer 200 everywhere, and the case is reachable (OtOpcUa's `RoleParser` admits `dev`-only and `cluster-`only nodes); and identity compares `UniqueAddress`, so a node restarted on the same host:port is correctly a different member. **Both apps migrated and their private copies deleted** — OtOpcUa (`feat/health-0.3.0-active-node` @ `88adec04`) also routes `RedundancyStateActor.SelectOldestUpMemberOfRole` through the shared primitive, so the tier and the OPC UA `ServiceLevel` 250/240 split cannot drift; ScadaBridge (`feat/site-node-health` @ `5d4853a1`) routes `ActiveNodeEvaluator` through it, unifying the S&F delivery gate, the heartbeat `IsActive` stamp, the inbound-API gate and the health tier on one rule. ScadaBridge keeps `SitePairActiveNodeHealthCheck` deliberately — it is a thin adapter over the site's own role-scoped `IClusterNodeProvider` (itself now backed by the shared rule), not a second copy of it. mxgw/HistorianGateway have no Akka cluster and are unaffected. Both app branches are **committed but NOT pushed or merged**. | Shared `ZB.MOM.WW.Health` lib | [`components/health/`](components/health/) | [`ZB.MOM.WW.Health/`](ZB.MOM.WW.Health/) |
| Observability (metrics / traces / logs) | **Adopted** (libs `0.1.0`, on the feed; **all 4 apps** call `AddZbTelemetry`; `AddZbSerilog` in all but ScadaBridge, which keeps its own factory by design) | Shared `ZB.MOM.WW.Telemetry` lib + `.Serilog` | [`components/observability/`](components/observability/) | [`ZB.MOM.WW.Telemetry/`](ZB.MOM.WW.Telemetry/) |
| Config + validation (options / startup validation) | Adopted (lib `0.1.0`, on the feed; **all 4 apps**, pushed — the "local only / not yet pushed" caveat was stale) | Shared `ZB.MOM.WW.Configuration` lib | [`components/configuration/`](components/configuration/) | [`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) |
| Audit (event model + writer seam) | Adopted (lib `0.1.0`, on the feed; **all 4 apps**, pushed to origin) | Shared `ZB.MOM.WW.Audit` lib | [`components/audit/`](components/audit/) | [`ZB.MOM.WW.Audit/`](ZB.MOM.WW.Audit/) |