Rewrites the active-node sections of the spec and shared contract, which
described the leader-based design that both consumers rejected.
- SPEC §2.3 replaced: the rule, the ordered RolePreference, why a node holding
none of the preferred roles is Unhealthy rather than Healthy, and the data
payload. §4 (IActiveNodeGate) and the tier table follow.
- SPEC §6 marked historical: it still tells adopters to use
ActiveNodeHealthCheck(role: "admin") and AkkaActiveNodeGate, which is the
advice that produced the defect. Kept for provenance, annotated so nobody
follows it again.
- GAPS: gaps L1/L2 recorded as closed the wrong way, reopened, then closed at
0.3.0 — including the part worth remembering, that both apps adopted the
shared check as specified, hit it live, and reverted to private copies, so the
surface reached zero consumers by being avoided rather than unused.
- Index rows updated with the consumer-visible changes and both app branches.
Promotes the two apps' private replacements for ActiveNodeHealthCheck into one
shared primitive, and retires the leader/RoleLeader selection they were written
to avoid.
Both Akka consumers had already hand-rolled a replacement rather than use this
package (ScadaBridge OldestNodeActiveHealthCheck, OtOpcUa
ClusterPrimaryHealthCheck), so the entire active-node surface here —
ActiveNodeHealthCheck, AkkaActiveNodeGate — had ZERO consumers. It was not
merely unused: it was avoided, twice, for the same reason.
Leadership is address-ordered (host, then port) and has no relationship to time;
singleton placement is age-ordered. The two agree on a freshly-formed cluster,
which is why single-node and happy-path tests never caught it. They diverge
permanently after any restart: the restarted node rejoins as the youngest but
keeps its address, so if it holds the lower address it becomes leader while the
singletons — and all the work they own — stay on the other node.
New ClusterActiveNode is the single implementation: oldest Up member, optional
role scope, plus role-preference resolution for a fused node that must answer
for the role its singletons are pinned to. ActiveNodeHealthCheck and
AkkaActiveNodeGate both delegate to it, so an endpoint gate and the
/health/active probe an orchestrator routes by cannot disagree.
BREAKING (behaviour, not signature):
- Selection is by age, not leadership.
- The role-filtered mode no longer reports Healthy for a node LACKING the role.
That "not applicable => Healthy" made the tier answer 200 on every node and
silently broke leader-pinning (lmxopcua#494); it is now Unhealthy by default,
overridable via NoActiveRoleStatus. This case is reachable, not defensive —
OtOpcUa's RoleParser admits dev-only and cluster-role-only nodes.
- Identity compares UniqueAddress, so a node restarted on the same host:port is
correctly a different member during the overlap.
Results now carry activeRole/selfAddress/activeNode in the 0.2.0 per-entry data
object, so a standby reports WHO is active and a dashboard can render a pair
from either node's payload alone.
Tests: 45 Akka (was 39), 76 total. The age-vs-address divergence is pinned
against a real two-node cluster built so the oldest member is not the
lowest-addressed one, with a fixture sanity check so it cannot pass for the
wrong reason.
The shared spec states the active tier "Fails (503) on a standby or
role-member-but-not-leader node", and the same spec maps Degraded to HTTP 200.
ActiveNodeDecision.Evaluate returned Degraded for exactly that case, so the tier
answered 200 on every node and could not distinguish an active node from a
standby over HTTP at all - only the response body differed.
The consequence was live, not theoretical: OtOpcUa's Traefik routes the admin UI
by this probe (docker-dev/traefik-dynamic.yml, scripts/install/traefik-dynamic.yml),
so both central nodes always passed and the leader-pinning the design called for
has silently never worked. Found while running the overview dashboard's live
acceptance, which reads the same tier and showed all six rig nodes as Active.
Filed as lmxopcua#494.
Role-member-but-not-leader now returns Unhealthy. The other two branches are
unchanged: a node that lacks the role stays Healthy (the probe is irrelevant to
it), and the startup-safety path still returns Degraded when the ActorSystem or
cluster is not yet available.
Blast radius is one consumer. OtOpcUa is the only user of the role-filtered mode;
ScadaBridge deliberately uses its own OldestNodeActiveHealthCheck (leadership is
address-ordered and diverges from the singleton host), and MxGateway and
HistorianGateway have no Akka cluster.
Version 0.2.1, three packages published to the Gitea feed and restore-verified.
70 tests pass.
Phase 0 of docs/plans/2026-07-22-overview-dashboard-impl-plan.md: give the
canonical health JSON a structured channel so the family overview dashboard can
read each Akka cluster's current leader.
- ZbHealthWriter: optional `"data": {...}` per entry, sourced from
HealthReportEntry.Data, emitted only when non-empty. Per-property JsonIgnore
(NOT a global DefaultIgnoreCondition) so `"description": null` still renders —
payloads from data-less checks stay byte-identical to 0.1.0.
- AkkaClusterHealthCheck: BuildClusterData publishes this node's own view —
leader (omitted while unknown), selfAddress, selfRoles (sorted), memberCount,
unreachableCount — on every result path. The startup-safety paths (no
ActorSystem / cluster inaccessible) stay description-only.
- Tests: writer data emit/omit (raw-JSON assert on the omit case), and a real
single-node self-joined cluster via Akka.TestKit.Xunit2 for the data values.
70 tests green (25/39/6).
- Version 0.1.0 -> 0.2.0; 3 packages published to the Gitea feed and
restore-verified from a scratch consumer, which serves data.leader live.
Resolves the 35 findings from the 2026-06-01 baseline (commit 26ba1c7),
test-first for every behavioral change. +51 tests (331 -> 382 passing, 0 failed).
- Telemetry-001 (HIGH): RedactionEnricher now honours property removal, so a
redactor that drops a key actually scrubs the secret from the event.
- Auth: LDAP validator ValidateOnStart; API-key verify no longer fails on a
best-effort MarkUsed write or a corrupt scopes column (fail-closed); LDAP cert
validation hook; KeyPrefix persistence aligned; README algorithm corrected.
- Health: Akka checks return Degraded (not throw) when the cluster isn't up yet;
GrpcDependencyHealthCheck catch-all; null 'description' rendered; composite
endpoint builder; XML docs shipped.
- Audit: CompositeAuditWriter no longer re-throws OperationCanceledException;
TruncatingAuditRedactor over-redact scrubs Target + safe negative max; options
record; XML docs shipped.
- Configuration: TryAddEnumerable idempotent registration; consistent port
quoting; strict invariant port parsing; XML docs + README packaged.
- Theme: mobile toggle is now CSS-only (no Bootstrap JS); token/CSS hygiene;
XML docs on the public parameter surface.
Shared-contract/spec docs updated where the code was the source of truth
(observability service.instance.id, MapZbMetrics, redactor reach). All changes
additive/back-compatible at v0.1.0. code-reviews bookkeeping follows separately.
- Added PackageTags to all 3 library csproj files (health-checks;aspnetcore/akka/efcore;scada;wonderware;zb-mom-ww)
- Full solution dotnet test: 58 tests green (32 Akka + 20 core + 6 EFCore)
- dotnet pack -c Release produces ZB.MOM.WW.Health.0.1.0.nupkg, ZB.MOM.WW.Health.Akka.0.1.0.nupkg, ZB.MOM.WW.Health.EntityFrameworkCore.0.1.0.nupkg; artifacts/ not committed
- ZB.MOM.WW.Health/README.md: overview, packages table, consumer matrix, versioning, build/test/pack instructions, status note
- components/README.md: Health row added to component registry
- CLAUDE.md: Health row in Component-normalization table + Health paragraph; intro updated from "two pieces" to "three pieces"
- upcoming.md: Health checks item checked off with pointer to components/health/ and ZB.MOM.WW.Health/
- components/health/README.md: status updated from "Draft / scaffolded / follow-on" to "Built @ 0.1.0"
- Eagerly call CancelAfter(InfiniteTimeSpan) after a successful probe so the pending OS
timer is released on the happy path rather than held for the full timeout window.
- Add ProbeTimeout_Unhealthy test: 50 ms timeout with an infinite-blocking probe delegate
asserts Unhealthy, covering the timeout code path.
- Fix ProbeQueryThrows_Unhealthy to use Task.FromException rather than a synchronous throw,
accurately modelling a faulted async delegate.
- Wrap all BuildServiceProvider() results in await using so ServiceProvider is disposed
after each test (no DI provider leak).
- Remove unused Microsoft.EntityFrameworkCore.InMemory package reference; tests use
SQLite only (InMemory CanConnect semantics differ and the package was not exercised).
- Add <remarks> to DatabaseHealthCheck<TContext> noting the scoped-resolution path is
safe for AddDbContextPool (scope dispose returns context to pool, not destroys it).
Joseph Doherty