Joseph Doherty
9cbe65c844
Audited the registration and mapping code of each app at its origin default branch: OtOpcUa c904472a, MxAccessGateway de86f8b8, ScadaBridge c7d868c5, HistorianGateway 4d6a9ad (all on Health 0.3.0, all MapZbHealth with default paths: /healthz runs nothing, /health/ready and /health/active by tag). GAPS.md now carries the verified per-endpoint coverage matrix and a SPEC-by-app table; each current-state doc lists exactly what each endpoint runs, and HistorianGateway gets its first current-state doc. Recorded: mxaccessgw#206 (worker-executable + galaxy-sql readiness checks, on /health/ready) is merged; historiangw#87 (session-pool check latches 503 until restart) is still open at 4d6a9ad. Deferred P2 downstream probes are now per-app; ScadaBridge IActiveNodeGate unification stays deferred (shared rule already used, only the interface is local). Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
84 lines
4.7 KiB
Markdown
84 lines
4.7 KiB
Markdown
# Health (readiness / liveness / active-node)
|
|
|
|
Second normalized component under the operability cluster. **Goal: path to shared code** — converge
|
|
the three sister projects onto a common three-tier health endpoint convention and a set of shared
|
|
probe implementations, proposed as the `ZB.MOM.WW.Health` library set (3 packages), while each
|
|
project keeps its own probe registration and orchestrator wiring.
|
|
|
|
- The one target: [`spec/SPEC.md`](spec/SPEC.md)
|
|
- The proposed shared library: [`shared-contract/ZB.MOM.WW.Health.md`](shared-contract/ZB.MOM.WW.Health.md)
|
|
- Divergences + backlog: [`GAPS.md`](GAPS.md)
|
|
- Current state, per project: [`current-state/`](current-state/)
|
|
|
|
## Why health is a strong normalization candidate
|
|
|
|
Both OtOpcUa and ScadaBridge trace their health-check structure to the same "ScadaLink three-tier
|
|
pattern" (`HealthEndpoints.cs:13` says so explicitly) but have already diverged in probe logic,
|
|
status semantics, response writer, and endpoint registration style. MxAccessGateway has no shared
|
|
ancestry here — it has a single hardcoded `/health/live` endpoint with no real probes at all.
|
|
The common core (three tiers, database probe, Akka cluster probe, active-node probe) is
|
|
re-implemented twice and absent once. Shared probe implementations with configurable policies
|
|
close the gap without forcing identical behavior onto projects with legitimately different cluster
|
|
semantics.
|
|
|
|
## Status by project
|
|
|
|
**Adopted by all four apps at `0.3.0`; probe coverage re-verified 2026-11-09 (scadaproj#11).** Every app
|
|
calls `MapZbHealth()` (`/healthz`, `/health/ready`, `/health/active`, canonical JSON). The per-endpoint
|
|
coverage matrix, SPEC deviations and remaining gaps are in [`GAPS.md`](GAPS.md); code-verified detail per
|
|
app (OtOpcUa, MxAccessGateway, ScadaBridge, HistorianGateway) is in
|
|
[`current-state/<project>/CURRENT-STATE.md`](current-state/). The "Why" section above and the bullets
|
|
below describe the original 2026-06 design rationale.
|
|
|
|
## Normalized vs. left per-project
|
|
|
|
**Normalized (the shared target):**
|
|
|
|
- Three-tier endpoint convention: `/health/ready` (tag `ready`), `/health/active` (tag `active`),
|
|
`/healthz` (bare liveness). Mapped by `app.MapZbHealth()` from `ZB.MOM.WW.Health`.
|
|
- Canonical JSON response writer (lifted from `HealthChecks.UI.Client` style; no per-project
|
|
writer wiring needed).
|
|
- `IActiveNodeGate` seam — generalized from ScadaBridge's `ActiveNodeGate`; wired into `MapZbHealth`
|
|
for automatic active-tier response.
|
|
- `GrpcDependencyHealthCheck` — reachability probe for a downstream gRPC dependency (covers
|
|
OtOpcUa → MxAccessGateway channel and MxAccessGateway → worker IPC).
|
|
- `AkkaClusterHealthCheck` (in `ZB.MOM.WW.Health.Akka`) with a configurable status policy.
|
|
Default = ScadaBridge's three-way policy; `OtOpcUaCompat` preset preserves OtOpcUa's two-way
|
|
self-Up-among-members scan.
|
|
- `ActiveNodeHealthCheck` (in `ZB.MOM.WW.Health.Akka`) with an optional role filter. Role-less =
|
|
ScadaBridge's behavior (Up + cluster leader); role-filtered = OtOpcUa's `AdminRoleLeader`
|
|
behavior.
|
|
- `DatabaseHealthCheck<TContext>` (in `ZB.MOM.WW.Health.EntityFrameworkCore`) with default
|
|
`CanConnectAsync` and an optional `ProbeQuery` delegate.
|
|
- `AllowAnonymous` on all three tiers by default (consistent across all three projects today).
|
|
|
|
**Left per-project (not forced together):**
|
|
|
|
- Which probes each app registers, their names, and which tags they carry.
|
|
- Orchestrator / Traefik wiring (sidecars, route rules, upstreams).
|
|
- ScadaBridge's `HealthMonitoring/` distributed aggregation pipeline (`SiteHealthCollector`,
|
|
`CentralHealthAggregator`, `HealthReportSender`, etc.) — domain-specific, no shared-library
|
|
equivalent.
|
|
- MxAccessGateway's `GatewayHealthReply` metadata (`DefaultBackend`, `WorkerProtocolVersion`) —
|
|
keep as a bespoke `/info` endpoint.
|
|
- The x86 worker process — out of process and out of scope; the gateway-side
|
|
`GrpcDependencyHealthCheck` observes it indirectly.
|
|
|
|
## Package structure
|
|
|
|
`ZB.MOM.WW.Health` ships as three dependency-split packages:
|
|
|
|
| Package | Contents | Consumers |
|
|
|---|---|---|
|
|
| `ZB.MOM.WW.Health` | Core tiers, `MapZbHealth`, canonical writer, `IActiveNodeGate`, `GrpcDependencyHealthCheck` | All three |
|
|
| `ZB.MOM.WW.Health.Akka` | `AkkaClusterHealthCheck` + status presets, `ActiveNodeHealthCheck` + role filter | OtOpcUa, ScadaBridge |
|
|
| `ZB.MOM.WW.Health.EntityFrameworkCore` | `DatabaseHealthCheck<TContext>` + optional probe delegate | OtOpcUa, ScadaBridge |
|
|
|
|
MxAccessGateway consumes the core package only (no Akka, no EF). OtOpcUa and ScadaBridge consume
|
|
all three.
|
|
|
|
## Component status
|
|
|
|
**Status: Adopted — library at 0.3.0** ([`../../ZB.MOM.WW.Health/`](../../ZB.MOM.WW.Health/), 3 packages),
|
|
pinned by all four apps; coverage re-verified 2026-11-09. Remaining gaps: [`GAPS.md`](GAPS.md).
|