Files

T

Joseph Doherty 9cbe65c844 docs(health): re-verify per-app probe coverage against the SPEC; refresh GAPS and current-state (scadaproj#11)

Audited the registration and mapping code of each app at its origin default branch: OtOpcUa
c904472a, MxAccessGateway de86f8b8, ScadaBridge c7d868c5, HistorianGateway 4d6a9ad (all on Health
0.3.0, all MapZbHealth with default paths: /healthz runs nothing, /health/ready and /health/active
by tag). GAPS.md now carries the verified per-endpoint coverage matrix and a SPEC-by-app table;
each current-state doc lists exactly what each endpoint runs, and HistorianGateway gets its first
current-state doc.

Recorded: mxaccessgw#206 (worker-executable + galaxy-sql readiness checks, on /health/ready) is
merged; historiangw#87 (session-pool check latches 503 until restart) is still open at 4d6a9ad.
Deferred P2 downstream probes are now per-app; ScadaBridge IActiveNodeGate unification stays
deferred (shared rule already used, only the interface is local).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV

2026-11-09 14:00:44 -05:00

4.7 KiB

Raw Blame History

Health (readiness / liveness / active-node)

Second normalized component under the operability cluster. Goal: path to shared code — converge the three sister projects onto a common three-tier health endpoint convention and a set of shared probe implementations, proposed as the ZB.MOM.WW.Health library set (3 packages), while each project keeps its own probe registration and orchestrator wiring.

The one target: spec/SPEC.md
The proposed shared library: shared-contract/ZB.MOM.WW.Health.md
Divergences + backlog: GAPS.md
Current state, per project: current-state/

Why health is a strong normalization candidate

Both OtOpcUa and ScadaBridge trace their health-check structure to the same "ScadaLink three-tier pattern" (HealthEndpoints.cs:13 says so explicitly) but have already diverged in probe logic, status semantics, response writer, and endpoint registration style. MxAccessGateway has no shared ancestry here — it has a single hardcoded /health/live endpoint with no real probes at all. The common core (three tiers, database probe, Akka cluster probe, active-node probe) is re-implemented twice and absent once. Shared probe implementations with configurable policies close the gap without forcing identical behavior onto projects with legitimately different cluster semantics.

Status by project

Adopted by all four apps at 0.3.0; probe coverage re-verified 2026-11-09 (scadaproj#11). Every app calls MapZbHealth() (/healthz, /health/ready, /health/active, canonical JSON). The per-endpoint coverage matrix, SPEC deviations and remaining gaps are in GAPS.md; code-verified detail per app (OtOpcUa, MxAccessGateway, ScadaBridge, HistorianGateway) is in current-state/<project>/CURRENT-STATE.md. The "Why" section above and the bullets below describe the original 2026-06 design rationale.

Normalized vs. left per-project

Normalized (the shared target):

Three-tier endpoint convention: /health/ready (tag ready), /health/active (tag active), /healthz (bare liveness). Mapped by app.MapZbHealth() from ZB.MOM.WW.Health.
Canonical JSON response writer (lifted from HealthChecks.UI.Client style; no per-project writer wiring needed).
IActiveNodeGate seam — generalized from ScadaBridge's ActiveNodeGate; wired into MapZbHealth for automatic active-tier response.
GrpcDependencyHealthCheck — reachability probe for a downstream gRPC dependency (covers OtOpcUa → MxAccessGateway channel and MxAccessGateway → worker IPC).
AkkaClusterHealthCheck (in ZB.MOM.WW.Health.Akka) with a configurable status policy. Default = ScadaBridge's three-way policy; OtOpcUaCompat preset preserves OtOpcUa's two-way self-Up-among-members scan.
ActiveNodeHealthCheck (in ZB.MOM.WW.Health.Akka) with an optional role filter. Role-less = ScadaBridge's behavior (Up + cluster leader); role-filtered = OtOpcUa's AdminRoleLeader behavior.
DatabaseHealthCheck<TContext> (in ZB.MOM.WW.Health.EntityFrameworkCore) with default CanConnectAsync and an optional ProbeQuery delegate.
AllowAnonymous on all three tiers by default (consistent across all three projects today).

Left per-project (not forced together):

Which probes each app registers, their names, and which tags they carry.
Orchestrator / Traefik wiring (sidecars, route rules, upstreams).
ScadaBridge's HealthMonitoring/ distributed aggregation pipeline (SiteHealthCollector, CentralHealthAggregator, HealthReportSender, etc.) — domain-specific, no shared-library equivalent.
MxAccessGateway's GatewayHealthReply metadata (DefaultBackend, WorkerProtocolVersion) — keep as a bespoke /info endpoint.
The x86 worker process — out of process and out of scope; the gateway-side GrpcDependencyHealthCheck observes it indirectly.

Package structure

ZB.MOM.WW.Health ships as three dependency-split packages:

Package	Contents	Consumers
`ZB.MOM.WW.Health`	Core tiers, `MapZbHealth`, canonical writer, `IActiveNodeGate`, `GrpcDependencyHealthCheck`	All three
`ZB.MOM.WW.Health.Akka`	`AkkaClusterHealthCheck` + status presets, `ActiveNodeHealthCheck` + role filter	OtOpcUa, ScadaBridge
`ZB.MOM.WW.Health.EntityFrameworkCore`	`DatabaseHealthCheck<TContext>` + optional probe delegate	OtOpcUa, ScadaBridge

MxAccessGateway consumes the core package only (no Akka, no EF). OtOpcUa and ScadaBridge consume all three.

Component status

Status: Adopted — library at 0.3.0 (../../ZB.MOM.WW.Health/, 3 packages), pinned by all four apps; coverage re-verified 2026-11-09. Remaining gaps: GAPS.md.

4.7 KiB Raw Blame History