9cbe65c844
Audited the registration and mapping code of each app at its origin default branch: OtOpcUa c904472a, MxAccessGateway de86f8b8, ScadaBridge c7d868c5, HistorianGateway 4d6a9ad (all on Health 0.3.0, all MapZbHealth with default paths: /healthz runs nothing, /health/ready and /health/active by tag). GAPS.md now carries the verified per-endpoint coverage matrix and a SPEC-by-app table; each current-state doc lists exactly what each endpoint runs, and HistorianGateway gets its first current-state doc. Recorded: mxaccessgw#206 (worker-executable + galaxy-sql readiness checks, on /health/ready) is merged; historiangw#87 (session-pool check latches 503 until restart) is still open at 4d6a9ad. Deferred P2 downstream probes are now per-app; ScadaBridge IActiveNodeGate unification stays deferred (shared rule already used, only the interface is local). Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
72 lines
4.8 KiB
Markdown
72 lines
4.8 KiB
Markdown
# Health — current state: ScadaBridge
|
|
|
|
**Verified 2026-11-09 at `ScadaBridge@c7d868c5` (`origin/main`)** for scadaproj#11. Read-only `git grep` /
|
|
`git show` against the origin default branch; all paths relative to the repo root
|
|
(`~/Desktop/ScadaBridge`). The pre-adoption state (verified 2026-06-01: name-predicate endpoints, no
|
|
`/healthz`, `HealthChecks.UI.Client` writer) is gone; see git history of this file.
|
|
|
|
Out of scope, as before: the `src/ZB.MOM.WW.ScadaBridge.HealthMonitoring/` domain-aggregation pipeline
|
|
(site → central health reports over Akka) is not an ASP.NET health probe.
|
|
|
|
- **Packages:** `ZB.MOM.WW.Health`, `.Akka`, `.EntityFrameworkCore` all **0.3.0** (`Directory.Packages.props:97-99`;
|
|
referenced by `src/ZB.MOM.WW.ScadaBridge.Host/ZB.MOM.WW.ScadaBridge.Host.csproj:32-34`; `.Akka` also by `Communication`).
|
|
- **Two composition roots**, both calling `app.MapZbHealth()` with defaults:
|
|
central (`src/ZB.MOM.WW.ScadaBridge.Host/Program.cs:544-574` register, `:741` map) and
|
|
site (`src/ZB.MOM.WW.ScadaBridge.Host/SiteServiceRegistration.cs:267-282` register, `Program.cs:861` map, on the
|
|
HTTP/1.1 metrics listener, default `:8084`).
|
|
- `CentralListenerIsolation.cs:14-15` admits `/healthz`, `/health/ready`, `/health/active` and `/metrics`
|
|
on a dedicated metrics port and 404s everything else there.
|
|
|
|
## Per-endpoint contents
|
|
|
|
| Endpoint | Central node | Site node |
|
|
|---|---|---|
|
|
| `/healthz` | none | none |
|
|
| `/health/ready` | `database`, `akka-cluster`, `required-singletons` | `akka-cluster`, `localdb` |
|
|
| `/health/active` | `database`, `active-node` | `active-node` |
|
|
|
|
## Checks
|
|
|
|
| Root | Name | Class | Tags | Probes | Failure |
|
|
|---|---|---|---|---|---|
|
|
| central | `database` | shared `DatabaseHealthCheck<ScadaBridgeDbContext>` (scoped context, no factory) | ready, active | `CanConnectAsync` | Unhealthy |
|
|
| central | `akka-cluster` | shared `AkkaClusterHealthCheck`, policy **`Default`** | ready | `SelfMember.Status` | Up/Joining Healthy; Leaving/Exiting Degraded; else Unhealthy; Degraded during startup |
|
|
| central | `required-singletons` | `RequiredSingletonsHealthCheck` (host) | ready | `Identify` (2 s, concurrent) on 5 singleton proxies (`notification-outbox`, `audit-log-ingest`, `site-call-audit`, `audit-log-purge`, `site-audit-reconciliation`) | Unhealthy if any unreachable or no ActorSystem |
|
|
| central | `active-node` | shared `ActiveNodeHealthCheck`, unscoped | active | oldest `Up` member of the central cluster | Unhealthy on standby; Degraded while cluster not yet available |
|
|
| site | `akka-cluster` | shared `AkkaClusterHealthCheck`, `Default` | ready | as above | as above |
|
|
| site | `localdb` | `SiteLocalDbHealthCheck` (host) | ready | `SELECT 1` on the site LocalDb; replication state in `data` only | Unhealthy if unreachable |
|
|
| site | `active-node` | `SitePairActiveNodeHealthCheck` (host) | active | `IClusterNodeProvider.SelfIsPrimary` → `ClusterActiveNode.SelfIsActive(cluster, siteRole)` | **Unhealthy** on standby **and while the ActorSystem is null** |
|
|
|
|
The `database` check on central's active tier is deliberate (review 01 [High]): a DB-dead active node drops
|
|
out of Traefik rotation (`Program.cs:534-537`).
|
|
|
|
## Active-node gating
|
|
|
|
- **Probes:** central `active-node` (shared check) and site `active-node` (thin adapter over the site's
|
|
role-scoped `IClusterNodeProvider`). Both route through the shared `ClusterActiveNode` via
|
|
`Communication.ClusterState.ActiveNodeEvaluator.SelfIsOldestUp` — the same rule as the S&F delivery
|
|
gate and the heartbeat `IsActive` stamp.
|
|
- **Request gate:** `ActiveNodeGate` (`Host/Health/ActiveNodeGate.cs`) registered as
|
|
**`ZB.MOM.WW.ScadaBridge.InboundAPI.IActiveNodeGate`** (`Program.cs:614`), *not* the shared
|
|
`ZB.MOM.WW.Health.IActiveNodeGate`. Consumed by `InboundApiEndpointFilter` (503 on standby),
|
|
`InboundMethodWarmupService`, `AuditWriteMiddleware` and `FlowDebugRunner`. Its logic is
|
|
`ClusterActivityEvaluator.SelfIsOldest(cluster)` → shared `ClusterActiveNode` — identical in behaviour to
|
|
`AkkaActiveNodeGate`; only the interface type is local. Central only; sites register no gate.
|
|
|
|
## Other endpoints
|
|
|
|
`MapZbMetrics()` on both roots; central gRPC control plane on its own h2c port.
|
|
|
|
## vs SPEC
|
|
|
|
| SPEC | Status | Note |
|
|
|---|---|---|
|
|
| §1 three tiers via `MapZbHealth`, canonical tags | ✅ | both roots |
|
|
| §1 nothing on `live` | ✅ | |
|
|
| §2.1 `DatabaseHealthCheck<T>` | ✅ central | site uses a custom LocalDb check (no EF on sites) |
|
|
| §2.2 / §7(b) Akka probe, Default policy | ✅ | |
|
|
| §2.3 `ActiveNodeHealthCheck` | ✅ central; 🟡 site | site adapter is Unhealthy (not Degraded) during startup and carries no `selfAddress`/`activeNode` `data` |
|
|
| §2.4 downstream probe | n/a | DCL connection health is reported through HealthMonitoring, not readiness |
|
|
| §3 canonical JSON writer | ✅ | |
|
|
| §4 / §7(e) `IActiveNodeGate` | 🟡 | same rule, local interface type (GAPS L2 deferral) |
|