The server half of the central-hosted pull-only secrets hub. SecretsHubService
adapts the node's own ISecretStore onto secrets_hub.v1 using exactly two read
members (GetManifestAsync, GetAsync) — the proto exposes no write RPC, so a
follower has no path to put a row into central whatever its credentials allow.
GetSecrets omits a name the hub does not hold and skips one that will not parse
as a SecretName; neither is an error, because both describe a follower acting on
a manifest that was true a moment ago. Failing the call would stall the sweep and
let one bad name deny the follower every other row in the batch. Names are
re-validated server-side (the wire type is a plain string) and de-duplicated on
the normalized value.
DomainToWire mirrors the Akka replicator's SecretRowDto discipline exactly:
round-trip "O" timestamps so a DateTimeOffset's offset and ticks survive the
last-writer-wins equality compare, the content type as its enum MEMBER NAME so an
unknown value is rejected rather than decoded to the zero member, and nullable
fields left absent rather than written as "" so null and empty stay distinct.
SecretsHubAuthInterceptor fails closed: with no token configured every call is
denied and one warning names the exact configuration key — never a token value,
presented or configured, at any level. Denial is uniform (one Status, one detail,
no "why") so a caller gets no oracle. Comparison is constant-time over UTF-8
bytes. All four handler shapes are gated, not just unary, so a future RPC cannot
arrive unauthenticated by omission; more than one Authorization header is refused
as a smuggling shape. The type keeps EXACTLY ONE public constructor — Grpc's
activator silently stops invoking an interceptor it cannot construct
unambiguously, and the failure mode is an auth gate that quietly does not run — so
a test pins the count.
Tests: 50, red-first (compile-red, then a fail-closed mutation verified 5 of them
bite). Full Secrets suite 328 passed / 0 failed / 15 live-gated skips, 0 new
warnings.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
Adds the third replicator sibling next to Replicator.SqlServer and
Replicator.AkkaDotNet: the central-hosted gRPC secrets hub. This commit is
scaffold only -- project, wire contract, solution wiring -- no service or
client implementation.
The proto mirrors exactly the read half of ISecretReplicationHub
(GetManifestAsync, GetManyAsync). ApplyReplicatedAsync is deliberately NOT
mirrored: in this topology every update originates at central and a site node
is a strict follower, so the narrow contract IS the enforcement -- with no
write RPC on the wire, a mis-wired or compromised site node has no way to push
a row into the hub regardless of what its authorization allows.
Messages are hand-written mirrors of StoredSecret and SecretManifestEntry
rather than serializations of the domain types, following the Akka
replicator's SecretRowDto precedent: the trust boundary stays structural, and
a field added to StoredSecret later cannot start crossing the network merely
by existing. Timestamps are ISO-8601 "O" strings (not google.protobuf.
Timestamp, which cannot round-trip a DateTimeOffset's offset) so a relayed row
is byte-identical to the one central holds; nullable .NET fields use proto3
explicit presence so null and empty string stay distinguishable.
Grpc 2.76.0 / Google.Protobuf 3.34.1 match what ScadaBridge, OtOpcUa and the
sibling ZB.MOM.WW.LocalDb libraries already resolve, so a consumer taking both
gets one Grpc rather than a downgrade warning.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
Design doc + runbook Production-enablement section + GAPS/index updates.
The previously-untracked production-KEK decision is now recorded: one
fleet-wide KEK via appsettings/env (never committed); ScadaBridge rejects
the SqlServer hub on topology grounds (sites talk to central, never the
DB) in favor of a new ZB.MOM.WW.Secrets.Replicator.Grpc package —
pull-only, updates originate at central only, by design.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
The InitJoin self-form watchdog (a timer calling Cluster.Join(SelfAddress) after a
window) was implemented and REJECTED after live testing: it sits outside Akka's join
handshake, so on a routine standby restart — peer alive, join stalled behind removal of
the node's own stale incarnation — it fires mid-join, abandons the in-flight join and
forms a permanent second cluster at the same address (measured: still split after 90s;
OtOpcUa hit it live in its manual-failover drill).
The safety argument 'a booting node only self-forms when no seed answers InitJoin' is
true of Akka's NATIVE first-seed rule (FirstSeedNodeProcess, inside the handshake) and
FALSE of the watchdog, which can fire while a seed is answering. That reasoning error is
now called out explicitly so no future plan re-derives the pattern.
- docs/plans/2026-07-22-initjoin-selfform-fallback.md: dated correction banner (history
preserved) + the two surviving false claims struck in place (behavior-spec row and the
Mechanism section's 'races are benign'); points at ScadaBridge
Component-ClusterInfrastructure.md as the surviving design.
- akka_failover.md 6.1: shipped design stated as self-first seed ordering plus the
opt-in, default-OFF simultaneous-cold-start bootstrap guard; added a
never-re-derive-the-watchdog note. 6.2: two items my 6.1 edit contradicted are marked
closed (OtOpcUa auto-down live gate, mesh election scope — both done 2026-07-24).
- CLAUDE.md: ScadaBridge issue #14 is CLOSED (2026-07-24), not open — v3 dual-namespace
live gates all passed, ScadaBridge consumes OtOpcUa v3 end-to-end.
#520 stage 6 (master 2026-08-01, live-gated). With Security:Acl:Enabled ON,
every data-plane operation needs a matching NodeAcl grant, not just writes /
alarm acks / HistoryRead. The stage-5 entry told ScadaBridge that a read-only
consumer was 'unaffected either way' — that now holds only while the flag is
off, which is the default.
Also records the two stage-6 behaviours a consumer would otherwise discover the
hard way: a deep Browse grant makes its ancestor folders browsable (so it
exposes their names), and revoking a grant stops a live subscription — one
BadUserAccessDenied then silence, resuming by itself if the grant returns, with
no reconnect needed.
Default is off, so a read-only address-space consumer is unaffected. With it on,
writes / alarm acks / HistoryRead need a matching grant for the principal the
session authenticates as — anonymous unless ScadaBridge is issued a key.
Reads, browses and subscriptions stay ungated in both modes.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Gitea #520 (OtOpcUa master, merge fbe7403c) replaced LdapOpcUaUserAuthenticator
with an API-key authenticator on the OPC UA data plane. The index still told a
consumer that enforcement was "LDAP roles plus a write gate", which is now the
wrong instruction for anyone wiring a client.
Records what a consumer actually needs: the UserName/`apikey`/`otua_...` token
shape, that LDAP survives for AdminUI login only, that anonymous still defaults
to admitted (so ScadaBridge is not broken today), and that revocation is
asymmetric because keys reach driver-only nodes inside the deployment artifact.
The "ACLs authored but not enforced" warning stands -- stages 3, 5 and 6 remove
it.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
All three live-gated and merged to OtOpcUa master. The index still said write-through was
deferred and every MQTT node read-only, which #508 changed on 2026-07-31; #511 and #515
landed in merge 857a10b4. Two of the three are consumer-visible to ScadaBridge: MQTT nodes
now materialize Operate rather than ViewOnly where a write destination is declared, and a
Sparkplug UDT surfaces as ordinary scalar nodes instead of being skipped.
Both were recorded as open keep-or-delete decisions; both closed on OtOpcUa master abda6f9e.
Neither changes the OPC UA wire contract, so neither is consumer-facing to ScadaBridge — but
the index asserted them as current state, so it would have drifted.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
The entry claimed the periodic tag-set drift check had shipped. It was taken back
out two days later and is not in master. Corrected in place rather than deleted,
so the record shows both what landed and what became of it.
Carries forward the consequence a reader needs: AbCip and FOCAS browse a live
backend but implement no IRediscoverable, so they now have no tag-change signal
at all. Galaxy/TwinCAT/MTConnect/Mqtt raise natively and are unaffected. Still
not consumer-facing to ScadaBridge -- no OPC UA wire-contract change either way,
and the IRediscoverable operator prompt is untouched.
Also records the rest of that merge: 13 stale skipped tests retired, a reflection
guard over the deploy-time TagConfig gate, three silently-failing OpcUaClient
integration tests fixed, and the two missing driver guides written.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
The index recorded this as unmerged on a feature branch; it landed on master
2026-07-28. Also records the two follow-ups the first pass had deliberately
left unbuilt and which have since shipped: Sql/FOCAS in-place config re-parse
(made atomic, so a config change cannot half-apply a new tag set against an
old connection or backend), and tag-set drift detection for the four
genuinely-browsable drivers.
Neither is consumer-facing to ScadaBridge — no OPC UA wire-contract change, and
the drift signal is advisory rather than an address-space rebuild. The
previously-noted operationally-visible item stands: a DriverConfig edit now
costs a driver reconnect.
Corrects a fact this index recorded two days ago as consumer-relevant to
ScadaBridge. "IRediscoverable is inert fleet-wide" no longer holds: it is
consumed as an operator re-browse prompt on /hosts, carried on
DriverHealthChanged. The consumer-facing conclusion is unchanged and worth
keeping explicit — the served address space still only changes on a config
redeploy, because the prompt is advisory by design.
Gitea #507 (discovered-node injection) is closed as superseded and its code
deleted; it read two structures that are structurally empty in v3, so it could
never have injected anything. #521 tracks the still-unconsumed
IHostConnectivityProbe.
Adds the remediation status for the deferment register (branch
feat/deferment-remediation, tip 88ce8df0, unmerged). One item there is
operationally visible beyond OtOpcUa: #516 makes a changed DriverConfig
stop+respawn the driver rather than an in-place delta, so a config edit now
costs a driver reconnect — previously five drivers ignored the edit entirely
while the deployment still sealed green.
Corrects four stale claims about OtOpcUa and adds two consumer-relevant facts,
all source-verified against the OtOpcUa repo (not its docs):
* "Phases 0a..6 done, 7 not started" -> PROGRAM COMPLETE. Phase 7 merged
c50ebcf7 (6/6 failover drills) + bootstrap guard d1dac87f, both on master
and pushed 2026-07-24.
* The auto-down 1-vs-1 crash-the-oldest live gate is no longer "still open" /
"deferred to Phase 7" — Phase 7 drill D4 closed it; every survivor stayed
Up without self-downing.
* Phase 6's live gate was NOT "deferred by owner choice" — it ran and caught
three fix-forward defects, including a boot-crash StackOverflow on every
node (IClusterRoleInfo resolved inside the AddAkka configurator while the
ActorSystem was still building).
* Records the MAIN-only manual-failover constraint and the default-OFF
bootstrap guard's founder-election shape.
Adds OtOpcUa/deferment.md (e08855fb) and its two ScadaBridge-relevant entries:
* Node ACLs are authored, shipped in every artifact, and NOT ENFORCED — the
evaluator subsystem has zero production consumers and OtOpcUaNodeManager
never references it. Reads are ungated; authored deny rules have no effect.
Do not assume OtOpcUa filters an address space per-user.
* IRediscoverable is inert fleet-wide (#518/#507) — nine drivers raise it,
nothing consumes it, so a Galaxy redeploy / PLC re-download / Agent restart
does not change the served address space without a config redeploy.
Keeps the umbrella index in step with the OtOpcUa driver set, per the
propagation rule in that repo's CLAUDE.md. Read-only Agent driver:
hand-rolled HTTP/XML client (probe/current + multipart /sample pump),
TrakHound dependency evaluated and dropped, browse free via the Wave-0
universal discovery browser, write-back deferred. Live-gated against a
real mtconnect/agent 2.7.0.12 fixture.
Sister projects are told to update their entry here whenever an indexed fact
changes, but that edit lands in this repo's own checkout — so a naive commit
writes it onto whatever branch scadaproj happens to be on. Observed 2026-07-27:
the OtOpcUa Sql driver's entry sat unpushed on feat/overview-dashboard for three
days, and the Mqtt entry was about to join it.
Records the procedure (stash, main, push, restore) and says to cherry-pick
stranded index commits rather than merge the branch they are stuck on.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
The OtOpcUa entry's driver set changed, so the index has to change with it
(OtOpcUa CLAUDE.md's cross-repo rule).
A read-only `Mqtt` broker-subscribe driver landed with two ingest shapes over
one MQTTnet-5 client: Plain binds a tag to a concrete topic, SparkplugB decodes
vendored Eclipse-Tahu protobuf under a single spBv1.0/{groupId}/# subscription
and binds by the group/edgeNode/device?/metric tuple (birth/alias/seq-gap/
rebirth state machine, death→STALE, a birth-driven browse picker with a scoped
Request-rebirth NCMD). Write-through is deferred — every MQTT node is read-only.
Live-gated on docker-dev against a Mosquitto TLS+auth broker plus a
project-owned C# Sparkplug edge-node simulator on 10.100.0.35:8883.
Recorded as **on branch feat/mqtt-sparkplug-driver, not yet merged to master**,
so the index does not overstate what is on OtOpcUa's default branch — unlike the
Sql DB-poll driver next to it, which is merged.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
OtOpcUa merged a read-only Sql Equipment-kind driver to master 2026-07-24
(polls SQL Server → OPC UA variables; connectionStringRef env resolution,
schema-walk address picker, live /run gate passed on docker-dev). Keep the
umbrella index's OtOpcUa driver enumeration in sync per the propagation rule.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
The OtOpcUa entry's driver set changed, so the index has to change with it
(OtOpcUa CLAUDE.md's cross-repo rule).
A read-only `Mqtt` broker-subscribe driver landed with two ingest shapes over
one MQTTnet-5 client: Plain binds a tag to a concrete topic, SparkplugB decodes
vendored Eclipse-Tahu protobuf under a single spBv1.0/{groupId}/# subscription
and binds by the group/edgeNode/device?/metric tuple (birth/alias/seq-gap/
rebirth state machine, death→STALE, a birth-driven browse picker with a scoped
Request-rebirth NCMD). Write-through is deferred — every MQTT node is read-only.
Live-gated on docker-dev against a Mosquitto TLS+auth broker plus a
project-owned C# Sparkplug edge-node simulator on 10.100.0.35:8883.
Recorded as **on branch feat/mqtt-sparkplug-driver, not yet merged to master**,
so the index does not overstate what is on OtOpcUa's default branch — unlike the
Sql DB-poll driver next to it, which is merged.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
OtOpcUa merged a read-only Sql Equipment-kind driver to master 2026-07-24
(polls SQL Server → OPC UA variables; connectionStringRef env resolution,
schema-walk address picker, live /run gate passed on docker-dev). Keep the
umbrella index's OtOpcUa driver enumeration in sync per the propagation rule.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
The dashboard derived every leader chip from AkkaClusterHealthCheck's `leader`
data — ClusterState.Leader, the LOWEST-ADDRESSED Up member. That is not the node
in charge. The node holding the singletons, gated for writes, and answering 200
on /health/active is the OLDEST Up member. The two agree only until some node
restarts, after which the restarted node rejoins youngest but keeps its address.
Observed live on the rebuilt OtOpcUa rig during the Health 0.3.0 work: leader
central-1, active node central-2. The dashboard would have put the Active chip on
central-1 while /health/active said central-2 — the chip contradicting the tier
it exists to visualise. The Phase 4 acceptance run missed it because the rig had
just been formed, the one state in which the two orderings agree.
Health 0.3.0's active tier now publishes `activeNode` on every member, including
standbys, so the fix is to prefer it and keep `leader` as the fallback for an
instance on an older Health or whose active tier was not probed.
The instance card now shows `active` and `leader` side by side rather than
swapping one for the other: they are legitimately different nodes, and showing
both turns a silent contradiction into something readable at 3am.
Tests: 177 (+3) — precedence when the two disagree, a standby naming the active
node rather than itself, and the fallback still working with no activeNode.
Rewrites the active-node sections of the spec and shared contract, which
described the leader-based design that both consumers rejected.
- SPEC §2.3 replaced: the rule, the ordered RolePreference, why a node holding
none of the preferred roles is Unhealthy rather than Healthy, and the data
payload. §4 (IActiveNodeGate) and the tier table follow.
- SPEC §6 marked historical: it still tells adopters to use
ActiveNodeHealthCheck(role: "admin") and AkkaActiveNodeGate, which is the
advice that produced the defect. Kept for provenance, annotated so nobody
follows it again.
- GAPS: gaps L1/L2 recorded as closed the wrong way, reopened, then closed at
0.3.0 — including the part worth remembering, that both apps adopted the
shared check as specified, hit it live, and reverted to private copies, so the
surface reached zero consumers by being avoided rather than unused.
- Index rows updated with the consumer-visible changes and both app branches.
Promotes the two apps' private replacements for ActiveNodeHealthCheck into one
shared primitive, and retires the leader/RoleLeader selection they were written
to avoid.
Both Akka consumers had already hand-rolled a replacement rather than use this
package (ScadaBridge OldestNodeActiveHealthCheck, OtOpcUa
ClusterPrimaryHealthCheck), so the entire active-node surface here —
ActiveNodeHealthCheck, AkkaActiveNodeGate — had ZERO consumers. It was not
merely unused: it was avoided, twice, for the same reason.
Leadership is address-ordered (host, then port) and has no relationship to time;
singleton placement is age-ordered. The two agree on a freshly-formed cluster,
which is why single-node and happy-path tests never caught it. They diverge
permanently after any restart: the restarted node rejoins as the youngest but
keeps its address, so if it holds the lower address it becomes leader while the
singletons — and all the work they own — stay on the other node.
New ClusterActiveNode is the single implementation: oldest Up member, optional
role scope, plus role-preference resolution for a fused node that must answer
for the role its singletons are pinned to. ActiveNodeHealthCheck and
AkkaActiveNodeGate both delegate to it, so an endpoint gate and the
/health/active probe an orchestrator routes by cannot disagree.
BREAKING (behaviour, not signature):
- Selection is by age, not leadership.
- The role-filtered mode no longer reports Healthy for a node LACKING the role.
That "not applicable => Healthy" made the tier answer 200 on every node and
silently broke leader-pinning (lmxopcua#494); it is now Unhealthy by default,
overridable via NoActiveRoleStatus. This case is reachable, not defensive —
OtOpcUa's RoleParser admits dev-only and cluster-role-only nodes.
- Identity compares UniqueAddress, so a node restarted on the same host:port is
correctly a different member during the overlap.
Results now carry activeRole/selfAddress/activeNode in the 0.2.0 per-entry data
object, so a standby reports WHO is active and a dashboard can render a pair
from either node's payload alone.
Tests: 45 Akka (was 39), 76 total. The age-vs-address divergence is pinned
against a real two-node cluster built so the oldest member is not the
lowest-addressed one, with a fixture sanity check so it cannot pass for the
wrong reason.
The shared spec states the active tier "Fails (503) on a standby or
role-member-but-not-leader node", and the same spec maps Degraded to HTTP 200.
ActiveNodeDecision.Evaluate returned Degraded for exactly that case, so the tier
answered 200 on every node and could not distinguish an active node from a
standby over HTTP at all - only the response body differed.
The consequence was live, not theoretical: OtOpcUa's Traefik routes the admin UI
by this probe (docker-dev/traefik-dynamic.yml, scripts/install/traefik-dynamic.yml),
so both central nodes always passed and the leader-pinning the design called for
has silently never worked. Found while running the overview dashboard's live
acceptance, which reads the same tier and showed all six rig nodes as Active.
Filed as lmxopcua#494.
Role-member-but-not-leader now returns Unhealthy. The other two branches are
unchanged: a node that lacks the role stays Healthy (the probe is irrelevant to
it), and the startup-safety path still returns Degraded when the ActorSystem or
cluster is not yet available.
Blast radius is one consumer. OtOpcUa is the only user of the role-filtered mode;
ScadaBridge deliberately uses its own OldestNodeActiveHealthCheck (leadership is
address-ordered and diverges from the singleton host), and MxGateway and
HistorianGateway have no Akka cluster.
Version 0.2.1, three packages published to the Gitea feed and restore-verified.
70 tests pass.
Found by the live rig, and invisible to every unit test that existed.
LeaderResolver and InstanceCard.ClusterDataLine both looked the cluster view up
under the check name "akka-cluster". ScadaBridge registers the shared
AkkaClusterHealthCheck under that name; OtOpcUa registers the SAME check as
"akka". So once the OtOpcUa rig was rebuilt on Health 0.2.0 and started
publishing data.leader, its groups still rendered with no leader chip and no
evidence line - and a group with no votes is a legitimate state, so nothing
looked wrong. ScadaBridge's four groups were fully populated the whole time,
which is what made it look like the feature worked.
Both now find the entry by the DATA KEY ("leader"), which is the part of the
contract the shared check actually owns rather than the part each consumer names
for itself.
Regression tests: the leader is found under akka-cluster, akka, and an arbitrary
future name; and an entry that publishes data WITHOUT a leader (ScadaBridge's
localdb check, whose data carries replication counters) does not get picked in
preference to the one that answers the question. Every prior fixture used
ScadaBridge's name, which is precisely why this got through.
After the fix all seven cluster groups render a leader chip with zero
split-brain flags - design section 9 check 2 now passes for BOTH products.
174 tests, 0 warnings.
Found during the live acceptance run. The host calls UseStaticWebAssets only when
the environment is literally "Development", so running the build output under any
other name - checking a staging registry locally, say - leaves the Theme RCL's
assets under _content/ unresolvable. The failure is silent and unhelpful: the page
returns 200 with its markup fully present and renders as a blank white screen,
because every stylesheet 404s. Calling it unconditionally is a no-op once
published (the manifest is gone and the assets are real files on disk), so the
only thing it changes is that failure. The container was never affected.
Phase 4 results recorded in the ledger: 7 of 9 design section 9 checks fully pass,
2 pass for ScadaBridge and are blocked for OtOpcUa pending a rig redeploy.
Notable live evidence:
- Leader chip moved from site-c-a to site-c-b when site-c-a was stopped, and the
split-brain chip appeared while the pair was self-formed then cleared once it
genuinely joined.
- Kill to Unreachable took two failing sweeps (the damping working as designed);
recovery landed on the first good poll.
- Five consecutive page loads in 0.00-0.02s all showed the same "last sweep"
timestamp, which is cache-first stated as a measurement.
- Freezing the probed endpoints while auto-refresh stayed RUNNING froze the poller
mid-sweep and the cards aged into STALE on their own - staleness is computed at
render, so a stuck poller needs no explicit stall detection.
The run also surfaced a pre-existing rig defect in both products: self-first Akka
seed lists mean a pair booted cold self-forms two 1-member clusters that never
join. After a --force-recreate every ScadaBridge node named ITSELF leader and every
node answered /health/active 200; restarting ONE node of each pair converged them
to one Leader plus one Active plus one Standby. That is the user's own open backlog
item (docs/plans/2026-07-22-initjoin-selfform-fallback.md, unexecuted), and the
dashboard made it visible fleet-wide in one screen.
170 tests, 0 warnings.
Closes Phase 3 of the overview-dashboard plan (tasks 3.7-3.9).
Self-observability
- OverviewMetrics: an observable gauge read from the published snapshot at scrape
time, so the metric and the page cannot disagree about what the dashboard
believes, plus a sweep-duration histogram. Instrument names follow the family
METRIC-CONVENTIONS spec (overview.instance.status / overview.poll.duration,
unit s) rather than the plan's literal zb_ names; Prometheus renders them as
overview_instance_status and overview_poll_duration_seconds. Labels are
application/node/group - "node", not "instance", because Prometheus stamps its
own instance label on every scraped series and a colliding metric label is
silently renamed exported_instance. Instances never probed emit nothing: the
enum's zero ordinal is Up, so a placeholder would publish the single most
reassuring answer about a node nobody has heard from.
- Two readiness checks make the dashboard's own /health/ready mean something:
registry-loaded (reports the counts it bound) and last-poll-cycle-completed,
which catches the failure where Kestrel is fine, the page still renders, and
every card on it is quietly frozen. The heartbeat is stamped by the sweep, not
derived from the snapshot's timestamp - the poller publishes the registry
before it probes anything, so a stuck poller would otherwise look healthy.
No cycle yet is Degraded inside the grace window and Unhealthy past it, so a
restart does not fail readiness for its first seconds and a poller that never
started cannot hide behind "still starting".
Configuration
- appsettings.json ships tuning defaults with an EMPTY Applications list, so a
deployment that supplies no registry fails at ConfigPreflight naming the key
instead of booting into a dashboard of nothing.
- Three registries, because reachability genuinely differs: Development is the
host-reachable subset (4 instances), Docker is the full 16-instance fleet by
container DNS name. The rigs publish almost no per-node HTTP port to the host
- OtOpcUa publishes none (only Traefik :9200, load-balanced across the central
pair, so it cannot identify a node) and ScadaBridge keeps site :8084
container-internal. Ports were established by probing the running rigs, not
read off the compose files, because the two disagree: OtOpcUa site nodes serve
health on :8080, not the :9000 their compose anchor implies.
- Real Serilog sinks, not the family's empty "Serilog": {}. AddZbSerilog drives
sinks entirely from configuration and adds none of its own, so the empty
section produces an app that logs NOWHERE - verified against the running
HistorianGateway container, whose log stream is completely empty for exactly
this reason. The HttpClient level override matters too: 16 instances on a 10s
cadence emit several Information lines a second of pure probe noise.
Container
- HistorianGateway-pattern runtime-only image; the compose stack joins the three
rig networks as external, so bringing the dashboard up or down cannot disturb
a rig. The Dockerfile clears ASPNETCORE_HTTP_PORTS as well as ASPNETCORE_URLS:
on .NET 8+ the base image declares its binding via the former, and clearing
only the latter leaves Kestrel logging "Overriding address(es)" every boot.
No healthcheck - aspnet:10.0 has neither curl nor wget, and adding a
--healthcheck entry point to production code to satisfy Compose is not a
trade worth making.
Tests
- BootTests boot the real Program.cs: every endpoint anonymous, the page
rendering from cache while nothing it watches is reachable, the Meters
allowlist proven by an end-to-end /metrics scrape, and both refuse-to-boot
paths (missing registry, stale window inside the poll interval).
The registry is supplied via environment variables, not
ConfigureAppConfiguration, because ConfigPreflight runs before Build() - when
the factory's callbacks are applied - and would not see them.
170 tests, 0 warnings. Verified live against the running rigs: 16 cards, 16
gauge series, page and gauge agreeing exactly.
Tasks 3.4-3.6 of the overview-dashboard plan.
Polling (3.4/3.5):
- OverviewPollerService: one timer at the fastest configured cadence, each
target polled when its own interval is due, fan-out per sweep, no retries
(the next tick is the retry, so damping stays the only place that decides
whether a failure is worth showing).
- OverviewSnapshotStore: atomic swap of a complete immutable graph plus a
change event. Pages read it and never probe — the cache-first requirement.
- LeaderResolver: per-group leader from akka-cluster data, with the
split-brain flag scoped to members that are actually answering, so an
ordinary failover is not misread as a disagreement.
UI (3.6): ThemeShell + StatusPill/TechCard composition against the mockup;
instance cards carry the status stripe (dashed for Unreachable), the check
detail list and the raw ready/active signal line.
Two defects found by verification rather than by review:
- The active tier answers with a status code and an EMPTY body, but the client
demanded parseable JSON on every probe — so a healthy pair rendered as
"role unknown" instead of Active/Standby. Split into ProbeAsync (ready tier,
body is the payload) and ProbeStatusAsync (active tier, code is the answer).
Caught by a live smoke run against two fake health endpoints; the poller
tests now serve an empty body so they hold the fix.
- SweepAsync published even when cancellation was already requested, leaving a
half-probed registry as the store's final state on shutdown. It now checks
the token itself rather than relying on the transport to throw.
Also: app.UseAntiforgery() is required despite the anonymous-by-design
pipeline — AddRazorComponents stamps antiforgery metadata unconditionally and
the endpoint middleware hard-fails without it (every page was 500). Chosen
over DisableAntiforgery() so a future form is protected by default.
147 tests, 0 warnings. Live-verified end to end against fake endpoints:
Up/Unreachable/Active/Standby, leader chip, cluster-data line, KPI counts.
Joseph Doherty