Files
scadaproj/components/health/current-state/scadabridge/CURRENT-STATE.md
T
Joseph Doherty 9cbe65c844 docs(health): re-verify per-app probe coverage against the SPEC; refresh GAPS and current-state (scadaproj#11)
Audited the registration and mapping code of each app at its origin default branch: OtOpcUa
c904472a, MxAccessGateway de86f8b8, ScadaBridge c7d868c5, HistorianGateway 4d6a9ad (all on Health
0.3.0, all MapZbHealth with default paths: /healthz runs nothing, /health/ready and /health/active
by tag). GAPS.md now carries the verified per-endpoint coverage matrix and a SPEC-by-app table;
each current-state doc lists exactly what each endpoint runs, and HistorianGateway gets its first
current-state doc.

Recorded: mxaccessgw#206 (worker-executable + galaxy-sql readiness checks, on /health/ready) is
merged; historiangw#87 (session-pool check latches 503 until restart) is still open at 4d6a9ad.
Deferred P2 downstream probes are now per-app; ScadaBridge IActiveNodeGate unification stays
deferred (shared rule already used, only the interface is local).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 14:00:44 -05:00

72 lines
4.8 KiB
Markdown

# Health — current state: ScadaBridge
**Verified 2026-11-09 at `ScadaBridge@c7d868c5` (`origin/main`)** for scadaproj#11. Read-only `git grep` /
`git show` against the origin default branch; all paths relative to the repo root
(`~/Desktop/ScadaBridge`). The pre-adoption state (verified 2026-06-01: name-predicate endpoints, no
`/healthz`, `HealthChecks.UI.Client` writer) is gone; see git history of this file.
Out of scope, as before: the `src/ZB.MOM.WW.ScadaBridge.HealthMonitoring/` domain-aggregation pipeline
(site → central health reports over Akka) is not an ASP.NET health probe.
- **Packages:** `ZB.MOM.WW.Health`, `.Akka`, `.EntityFrameworkCore` all **0.3.0** (`Directory.Packages.props:97-99`;
referenced by `src/ZB.MOM.WW.ScadaBridge.Host/ZB.MOM.WW.ScadaBridge.Host.csproj:32-34`; `.Akka` also by `Communication`).
- **Two composition roots**, both calling `app.MapZbHealth()` with defaults:
central (`src/ZB.MOM.WW.ScadaBridge.Host/Program.cs:544-574` register, `:741` map) and
site (`src/ZB.MOM.WW.ScadaBridge.Host/SiteServiceRegistration.cs:267-282` register, `Program.cs:861` map, on the
HTTP/1.1 metrics listener, default `:8084`).
- `CentralListenerIsolation.cs:14-15` admits `/healthz`, `/health/ready`, `/health/active` and `/metrics`
on a dedicated metrics port and 404s everything else there.
## Per-endpoint contents
| Endpoint | Central node | Site node |
|---|---|---|
| `/healthz` | none | none |
| `/health/ready` | `database`, `akka-cluster`, `required-singletons` | `akka-cluster`, `localdb` |
| `/health/active` | `database`, `active-node` | `active-node` |
## Checks
| Root | Name | Class | Tags | Probes | Failure |
|---|---|---|---|---|---|
| central | `database` | shared `DatabaseHealthCheck<ScadaBridgeDbContext>` (scoped context, no factory) | ready, active | `CanConnectAsync` | Unhealthy |
| central | `akka-cluster` | shared `AkkaClusterHealthCheck`, policy **`Default`** | ready | `SelfMember.Status` | Up/Joining Healthy; Leaving/Exiting Degraded; else Unhealthy; Degraded during startup |
| central | `required-singletons` | `RequiredSingletonsHealthCheck` (host) | ready | `Identify` (2 s, concurrent) on 5 singleton proxies (`notification-outbox`, `audit-log-ingest`, `site-call-audit`, `audit-log-purge`, `site-audit-reconciliation`) | Unhealthy if any unreachable or no ActorSystem |
| central | `active-node` | shared `ActiveNodeHealthCheck`, unscoped | active | oldest `Up` member of the central cluster | Unhealthy on standby; Degraded while cluster not yet available |
| site | `akka-cluster` | shared `AkkaClusterHealthCheck`, `Default` | ready | as above | as above |
| site | `localdb` | `SiteLocalDbHealthCheck` (host) | ready | `SELECT 1` on the site LocalDb; replication state in `data` only | Unhealthy if unreachable |
| site | `active-node` | `SitePairActiveNodeHealthCheck` (host) | active | `IClusterNodeProvider.SelfIsPrimary` → `ClusterActiveNode.SelfIsActive(cluster, siteRole)` | **Unhealthy** on standby **and while the ActorSystem is null** |
The `database` check on central's active tier is deliberate (review 01 [High]): a DB-dead active node drops
out of Traefik rotation (`Program.cs:534-537`).
## Active-node gating
- **Probes:** central `active-node` (shared check) and site `active-node` (thin adapter over the site's
role-scoped `IClusterNodeProvider`). Both route through the shared `ClusterActiveNode` via
`Communication.ClusterState.ActiveNodeEvaluator.SelfIsOldestUp` — the same rule as the S&F delivery
gate and the heartbeat `IsActive` stamp.
- **Request gate:** `ActiveNodeGate` (`Host/Health/ActiveNodeGate.cs`) registered as
**`ZB.MOM.WW.ScadaBridge.InboundAPI.IActiveNodeGate`** (`Program.cs:614`), *not* the shared
`ZB.MOM.WW.Health.IActiveNodeGate`. Consumed by `InboundApiEndpointFilter` (503 on standby),
`InboundMethodWarmupService`, `AuditWriteMiddleware` and `FlowDebugRunner`. Its logic is
`ClusterActivityEvaluator.SelfIsOldest(cluster)` → shared `ClusterActiveNode` — identical in behaviour to
`AkkaActiveNodeGate`; only the interface type is local. Central only; sites register no gate.
## Other endpoints
`MapZbMetrics()` on both roots; central gRPC control plane on its own h2c port.
## vs SPEC
| SPEC | Status | Note |
|---|---|---|
| §1 three tiers via `MapZbHealth`, canonical tags | ✅ | both roots |
| §1 nothing on `live` | ✅ | |
| §2.1 `DatabaseHealthCheck<T>` | ✅ central | site uses a custom LocalDb check (no EF on sites) |
| §2.2 / §7(b) Akka probe, Default policy | ✅ | |
| §2.3 `ActiveNodeHealthCheck` | ✅ central; 🟡 site | site adapter is Unhealthy (not Degraded) during startup and carries no `selfAddress`/`activeNode` `data` |
| §2.4 downstream probe | n/a | DCL connection health is reported through HealthMonitoring, not readiness |
| §3 canonical JSON writer | ✅ | |
| §4 / §7(e) `IActiveNodeGate` | 🟡 | same rule, local interface type (GAPS L2 deferral) |