Owner decision 2026-11-11: /fleet/diagnostics "Faulted devices" counts DISTINCT
devices across the nodes of the selected node's cluster, Faulted state only.
- DriverHostActor: FaultedDevices = HostState.Faulted only (Stopped no longer
counted); new StoppedDevices + FaultedHostNames. Counting extracted to the
testable DiagnosticsFromHealth.
- DriverInstanceDiagnostics: StoppedDevices (default 0) and FaultedHostNames
(default null) are additive with defaults, so a pre-#786 sender's snapshot
still deserializes over Akka's default JSON serializer (pinned by a test).
- FleetDiagnosticsAggregation (pure): identity key = (driver-instance string
id, HostName), case-insensitive; a single-host driver / nameless pre-#786
count is combined by MAX across nodes, never summed. Cluster peers come from
enabled ClusterNode rows of the selected node's ClusterId.
- FleetDiagnostics.razor: tile fetches the cluster peers and shows the
distinct count + "n of m nodes answered"; per-driver table gains a Stopped
devices column.
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
User decision: save, Test Connect, live browse and the deploy gate keep
refusing out-of-namespace credential references, but an ALREADY-DEPLOYED
driver must not fault at the upgrade restart or (Galaxy) at the next
gateway reconnect, since ReopenAsync re-resolves Gateway.ApiKeySecretRef.
- LegacyCredentialReferenceTracker (Core.Abstractions): the one, deprecated,
call-site-chosen compatibility path. Running Galaxy (InitializeAsync,
ReopenAsync, ack fallback) and OpcUaClient (session open) pass it and keep
the pre-#815 semantics (case-insensitive prefixes, Galaxy file:, OpcUaClient
literal pass-through) with a rate-limited Warning (driver/field/allowed
form, never the value) and a credential-compat:<field> diagnostics entry
(both drivers now implement IDriverDiagnostics). Browsers and probes pass
none and stay hard-fail; Heidenhain has none.
- Deploy-gate message names the migration.
- Upgrade preflight: CredentialReferencePreflightService (admin nodes, once
after ApplicationStarted, ConfigDb only, resolves nothing) logs a Warning
per Draft row / Deployed-artifact finding; the /raw banner shows them plus
Running drivers reporting warn-only use (driver-health diagnostics).
- docs/security.md: exact replacement table (incl. the old form default
env:MX_API_KEY, file:, un-namespaced secret:, casing changes), the
ReopenAsync reconnect behaviour, the warn-only window and its removal.
Co-Authored-By: Claude <noreply@anthropic.com>
The shared RawConfirmDialog sat inside the Alarms tab branch, so Remove
reference (Tags) and Delete virtual tag (Virtual Tags) set it visible on
a page that did not render it. Pinned by brace depth in
DeleteConfirmationWiringTests.
The four Delete buttons deleted on the first click. They now open the shared
RawConfirmDialog naming the item; only Confirm deletes. Adds bUnit (test-only)
render tests: modal opens without deleting, Cancel keeps the item, Confirm deletes.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
mxaccessgw 488649c+ acknowledges by AuthenticateUser as the operator and
writing <alarm>.AckMsg, so AcknowledgeAlarmRequest now needs a real Galaxy
operator_user + operator_domain + operator_password. The OPC UA data plane is
API-key/anonymous and carries no Galaxy password, so the driver resolves the
account per ack:
1. the session's API key's Galaxy ack identity, set on AdminUI /apikeys and
held ONLY in the encrypted Secrets store
(otopcua/apikeys/<keyId>/galaxy-ack-{user,password}); a key with a user but
no password secret is refused, never silently downgraded;
2. else the driver instance's AlarmAck fallback account (DriverConfig
FallbackUser/FallbackDomain/FallbackPasswordSecretRef; secret:/${secret:}/
env:/file:/dev: only — a literal is refused);
3. else DriverOperationRefusedException: nothing is sent, never an empty
operator_user. Refusals are excluded from the resilience retry/breaker.
- Pins ZB.MOM.WW.MxGateway.Client/Contracts 0.2.1 -> 0.3.0 (vendored into
nuget-packages/ from the Gitea feed, per the local-mxgw source mapping).
- The session ApiKeyId (not a secret) rides NativeAlarmAck ->
RouteNativeAlarmAck -> RouteAlarmAck -> AlarmAcknowledgeRequest; the password
is resolved in the driver and never travels through actor messages.
- The OPC UA principal + key id are appended to the AckMsg comment
("[OPC UA: <principal>, key <id>]") and logged, so a shared fallback
account does not lose who acknowledged.
- Password never logged: GalaxyAckCredential.ToString omits it, the
acknowledger scrubs it from every gateway message it echoes and withholds
any exception that echoes it; audit rows carry only the key id.
- AdminUI: /apikeys "Ack identity" editor (set/remove; delete-key clears it),
Galaxy driver form gains the fallback section.
- Synchronous OPC UA status for a refusal is follow-up #760 (the native ack
path is fire-and-forget after the Part 9 method returns).
Tests: request mapping (password populated), password absent from logs and
exception text (mutation-checked), key-mapped / fallback / refused paths at
resolver and driver level, breaker non-trip, key-id propagation through node
manager and DriverHostActor, AdminUI service + form round-trip.
Closes#749
Staleness is not a liveness proof: DriverInstanceActor publishes on a 30 s
tick and dedups on a health fingerprint, so a running Faulted, idle or
tagless driver deleted in /raw (undeployed) went stale and was evicted.
DeployedDriverSet reads, per node, the latest and latest-Applied
NodeDeploymentState deployments plus the latest Sealed one, parses each
artifact's DriverInstances once (cached by deployment id, pruned to the live
set, only the blob column of an uncached row is read), and /hosts unions it
with the authored set. The staleness clause stays as a second guard. An
unreadable artifact or a failed read means 'unknown' and skips eviction.
The Hosts.razor comment claiming '~12 health publishes' is corrected.
N-new-2: the CSV download's Content-Disposition carries an RFC 5987
filename* via ContentDispositionHeaderValue.SetHttpFileName.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
I1: /hosts eviction keyed off the AUTHORED DriverInstances, so a driver
deleted in /raw but not yet deployed - still running, still publishing -
vanished every tick. The stores' sweep is now EvictUnconfiguredStale(set,
olderThanUtc): an entry goes only when unconfigured AND its PublishedUtc is
older than 60 s. Tests cover unconfigured+fresh kept (positive control),
unconfigured+stale evicted, configured+stale kept, and the deploy hand-off.
M1: Test Connect falls back to the driver's documented ProbeTimeoutSeconds
default (Galaxy 30, OpcUaClient 15, ...) when the key is absent.
M2: the budget never undercuts the config's root timeoutMs (+1 s).
M5: RawBrowseModal compares the open generation captured before the await,
so a close-then-reopen mid-open closes the stale session.
M6: /alarms-historian keeps a 30 s re-poll on 'not registered' (a driver
node may still be starting) instead of stopping for good.
M7: /scripts source fetch failure shows in the dialog, not the error boundary.
M8: the export modal maps only its preview rows; the endpoint writes via
Results.Text (no second byte[] copy).
M9: /hosts config queries are ordered, so SequenceEqual change detection holds.
N1 tooltip, N2 CRLF-safe regex, N3 throttled Warning for subscriber faults,
N5 equipment-import help text now describes RFC 4180 quoting.
H3: MonacoEditor documents that themes are page-global.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
/fleet/diagnostics was reachable only by URL: each driver-role row on /fleet now
links to it (FleetDiagnosticsLinks.For escapes the host:port node id). The
diagnostics table showed the string instance id because the wire carries no
display name; the page now joins the authored DriverInstances.Name from the
ConfigDb (enrichment only - a failed lookup falls back to the id) and keeps the
id beneath the name.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The dev-only demo page shipped routable in the production assembly. Its route
moves from @page to a code-behind [Route] under #if DEBUG, so a Release build
keeps the component type (nothing referencing it breaks) but has no /dev/*
route. The page-authorization guard classifies it under #if DEBUG, and
DevRouteGatingTests asserts each configuration's half (Debug: exactly the demo
route; Release: none) with a positive control. Both configurations pass.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The equipment page's Remove reference / Delete virtual tag / Delete alarm row
buttons and the ScriptEdit Delete ran on the first click. They now go through
RawConfirmDialog, the same confirm /raw uses. The ScriptEdit change is limited
to the delete button and its dialog.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
I-1 - a non-C# paste produced thousands of Roslyn markers. The Problems-list report sent
them all as one JS->.NET message, which could exceed the 1 MB hub limit and kill the
circuit. The JS now sends at most MaxReportedMarkers (200) markers, errors first, each
message cut to 300 chars, plus the total. ScriptProblemReport.FromEditor clamps again
on arrival, and the Problems list shows "showing N of Total" / "+N more". The diagnostics
endpoint also bounds its response: Distinct, errors first, MaxDiagnosticMarkers (1000),
messages cut to 1000 chars. An error is always kept, so editor-accepts == publish-accepts.
M-1 - drop the per-instance compilation lineage; analyses derive from BaseCompilation.
M-2 - the VirtualTag modal clears its problems (and editor ref) on close and on a script
reload.
M-5 - an over-limit text is announced at blur (OnSyncRefused), not only at Save.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
A bare catch {} turned every failure - including an admin-only node with no
historian actor - into 'Loading...' forever, and a tick queued after a sync
timer dispose still issued an Ask. HistorianSinkStatusReader classifies a poll
as Available / NotOnThisRole (permanent: shown once, polling stops) /
Unavailable (shown with the last good status and its read time). The page
follows Fleet.razor: IAsyncDisposable, _disposed, overlap guard, page CTS.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
R04:P1 - the editor no longer ships its full text JS->.NET on every keystroke. It reports
at blur, and ScriptEdit / the VirtualTag modal read the authoritative value with
MonacoEditor.GetValueAsync at Save (validation runs after the pull). The Blazor hub's
MaximumReceiveMessageSize is raised from 32 KB to 1 MB. The JS refuses a value that
would not fit instead of sending an oversize frame, and Save surfaces that refusal
rather than persisting stale source.
R04:P9 - diagnostics carry a generation counter and an AbortController. Every edit aborts
the in-flight request, and only the response for the current text is applied.
R04:S2 - the always-empty inlay-hints provider, endpoint, service method and contracts
are removed.
R04:S3 - MarkersChanged is typed (DiagnosticMarker) and feeds a new ScriptProblemsList.
Clicking a problem calls RevealLineAsync. Markers cross to .NET only when a list
subscribes.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The list loaded every script entity including SourceCode to render names and
hashes. ScriptListQueries projects {ScriptId, Name, Language, SourceHash} for
the list and fetches one script's source when its Source dialog opens, with a
generation guard against a superseded open.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Each received event used to Insert(0) into a List and render the full page.
LiveTailBuffer queues events from the bridge thread into a bounded newest-first
ring; CoalescingRenderScheduler (TimeProvider-driven, drained on dispose)
flushes and renders at most once per 200 ms window. ScriptLog's filtered rows
are recomputed on flush/filter change instead of three times per render.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
OnSnapshotChanged used to re-project every cluster group and re-render the
whole page for each driver publish, per open tab. It now only marks a
CoalescingDirtyFlag; the 5 s tick rebuilds when the flag is set or the config
load changed (or evicted) anything.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The driver-status and resilience stores had no removal path, so a deleted
driver stayed on /hosts as a frozen row until restart. Both stores gain
RetainOnly, which /hosts calls after each successful config load (never on a
failed one).
The stores and InProcessBroadcaster now raise their events one subscriber at a
time, so a throwing handler cannot starve later subscribers or escape onto the
bridge actor. The alert/script-log bridges publish inside their try, and the
resilience bridge catches a store failure instead of restarting.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- Writes: a second write to a tag already inside the driver waits in the
queue behind it (_tagsInFlight) while other tags still run under the
cap of 4; queued writes are stamped and failed BadTimeout at dequeue
once older than the 8 s host Ask budget (Props queuedWriteBudget)
instead of being written late.
- Initialise attempts are bounded pessimistically (initializeTimeout,
default 30 s = 2x Tier C) on top of the invoker's token-only timeout:
an overrun is logged as abandoned, reported InitializeFailed so the
retry loop continues, and its late result is dropped by generation.
Generations come from a process-wide Interlocked counter so a prior
incarnation's late result cannot collide (F6).
- PostStop shares one shutdownTimeout budget across shutdown + dispose.
- TelemetryDialSupervisor cancels the establishment deadline on
StreamStopped; FleetDiagnosticsClient builds the ActorSelection inside
the try (a malformed node degrades to "unavailable"); the diagnostics
page labels the column "Last health publish".
- Tests: same-tag serialisation with a distinct-tag positive control;
stale queued write BadTimeout; token-ignoring hung init -> later
attempt connects; stopwatch assertion demoted to a ceiling.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
DriverInstanceDiagnostics.Name carries the string driver-instance id
(what /hosts, telemetry and the artifact key on) and DriverInstanceId
the DB row Guid; the wire record has no slot for the authored display
name. The write-cap test asserts entry order per slot pair instead of
completion order of two concurrent writes.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
DriverHostActor wraps the health publisher it hands its children in a
ChildHealthLedger (last snapshot per driver id, forwarded to the real
publisher), so GetDiagnostics now answers with each child's row id,
authored name, last-published state, device counts (host statuses when
the driver probes hosts, else one device) and last-change time — instead
of Guid.Empty / "Spawned" / 0 / now. Entries are forgotten on StopChild
and operator Restart.
IFleetDiagnosticsClient gains its first consumer: the new
/fleet/diagnostics page (node selector + ?node= query, revision,
running-from-cache notice, per-driver table). FleetDiagnosticsClient
logs the degrade-to-empty path at Warning.
Review ref: 02/U1.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- OtOpcUaNodeManager overrides HistoryUpdateData / HistoryDeleteRawModified / HistoryDeleteAtTime,
block-bridging to the optional IHistorianDataUpdater under the shared HistoryRead batch limiter +
deadline. Insert/Replace/Update map to HistoryUpdateMode; Remove, IsDeleteModified, events and
structure data are BadHistoryOperationUnsupported. Per-item codes stitched back in request order
(MinValue SourceTimestamp => BadInvalidTimestamp, untypeable value => BadTypeMismatch); success
clears the base's pre-seeded Unsupported error.
- HistoryWrite AccessLevel bit OR-ed in for historized nodes when an updater is wired (raw + UNS);
it is dispatch-load-bearing in the SDK base. OtOpcUaSdkServer.SetHistorianDataUpdater; the hosted
service assigns it from an optional HistorianUpdaterHolder before SetSink, clears it on stop.
- AuthorizeHistoryUpdate: ACL on => scoped OpcUaOperation.HistoryUpdate decision (replaces the role
check); ACL off => #563 key re-check + new OpcUaDataPlaneRoles.HistoryUpdate role, conferred by
ApiKeyInterimRoleMapper only from a Cluster-scoped HistoryUpdate grant (WriteOperate does not).
- AdminUI ACL editor offers NodePermissions.HistoryUpdate.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
M1: PageErrorBoundary wraps the body of every InteractiveServer page; the inert layout boundary is removed.
m1: DisposeBridge cancels its CTS when the 5 s wait times out.
m2: a CSV import that commits after the modal closed still raises OnSaved.
m3: document SecurityMode=None no-fallback in Client.CLI.md / Client.UI.md.
m4: browsePageDelayMs written only when loaded or changed; negative flagged.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
The modal's existence was a function of _rows (FindSourceScript() as the
visibility gate), so any render where the row is momentarily unresolvable —
a future reload nulling or replacing the list — unmounts the subtree and
disposes the Close handler's binding id mid-click (GetRequiredEventBindingEntry).
Latent here (_rows is written once, in OnInitializedAsync) but the identical
shape bit ScadaBridge live the moment a page refresh landed, and it presents
as an unattributable intermittent once one does.
ScadaBridge's shape, ported by the family sweep: intent (_sourceScriptId)
gates existence, the resolve drives only content; an unresolvable script
renders a notice with the frame and Close still mounted, and closing remains
the operator's action rather than a side effect of list state.
The page ran members, the driver-instances heading, and every cluster group
together inside one panel; driver name and id slammed inline; one long driver
exception blew the table out sideways (Controls scrolled out of reach); and the
inline #546 expander row dumped a full-width panel into the table.
- One panel per cluster group, node chips in the panel head.
- Driver id renders on its own muted mono line under the name.
- Last error abbreviated to 60 chars (DisplayText.Abbreviate — #504 rule),
full text on the tooltip and in the Controls dialog.
- DriverStatusPanel moved from an inline expander row to a modal (one at a
time); the dialog re-resolves its row from the current groups each render so
the 5 s rebuild can't strand it, and the Enabled-in-key recycle discipline
carries over unchanged. HostsPanelWiringTests re-pinned to the dialog markup.
Live-verified on docker-dev: separated cards, in-viewport table, dialog opens
with live breaker/status data and grouped Reconnect|Restart, closes cleanly.
Group related action buttons into btn-group/btn-group-sm on Alerts, Certificates,
ApiKeys and ScriptEdit; promote the RoleGrants Delete from btn-link to
btn-outline-danger. Replace the two inline style="border-color:var(--alert)"
no-ops with the .notice-alert class, and the undefined --surface-2 token in
Scripts with the theme's shipped --zebra-bg.
Delete unconditional documentation prose now covered by docs/ (Redundancy,
Telemetry, AlarmTracking, security, Reservations, AlarmHistorian, ScriptEditor),
keeping state banners and live captions. Correct the LDAP config key on
RoleGrants to Security:Ldap:GroupToRole (LdapOptions.SectionName) and drop the
same wrong key from Account with its deleted block; reword the stale
"Engine emit (F8/F9) is pending" script-log empty state.
Task 1 of the AdminUI UI sweep. Behavior-preserving: only wrappers, classes
and prose change — every @onclick, disabled, href and arm->confirm flow is
untouched.
Buttons -> Bootstrap btn-group:
- ClusterOverview page-head Edit cluster + Deployments pair
- Edit-form feet (Save+Cancel grouped; ms-auto Delete stays outside):
ClusterEdit, NodeEdit, NewCluster, AclEdit
- ClusterRedundancy confirm-failover Confirm+Cancel (ms-2 dropped)
- AclEdit permission-bundle presets: btn-link + middot separators become an
outline-secondary btn-group
Inline style="border-color:var(--alert)" (a no-op — that token is defined
nowhere) replaced by the .notice-alert class: NodeEdit, AclEdit, ClusterEdit.
Prose (unconditional documentation deleted; state banners/legends kept):
- ClusterRedundancy: removed the stale RedundancyStateActor "on each admin
node" block (cluster-scoped singleton on driver nodes since mesh Phase 6)
which also hyperlinked an unserved repo path; trimmed the mesh-split
explanation to its data-provenance first sentence
- ClusterAcls: removed the Security:Acl:Enabled mode walkthrough and the
grant/principal explanation (both in docs/security.md); kept the
unresolved-chip decoder and trimmed the empty state to one sentence
- AclEdit: removed its near-duplicate of that walkthrough
- ClusterAudit: dropped the stale (F3) label and the EventId/CorrelationId
dedup sentence; kept the page-size + Details-JSON caption
- ClusterNamespaces / NamespaceEdit: retired-in-v3 placeholders reworded to
past tense — Batches 2/3 shipped, /raw and /uns are live
DriverStatusPanel decides whether to subscribe exactly once, in
OnInitializedAsync (`if (!Enabled) return;`), and exposes no OnParametersSet
hook. The detail row was keyed only by DriverInstanceId, so a live config edit
that flipped a row's Enabled true->false — surfaced on the page by the 5 s
LoadConfigAsync refresh — reused the same panel instance: the markup correctly
switched to "Disabled - not deployed" while the panel's two store SnapshotChanged
subscriptions and its 5 s staleness timer stayed wired underneath, until the
operator happened to collapse the row.
Including the flag in the detail row's key recycles the component instead. Both
directions verified against the panel's own lifecycle: true->false disposes the
old instance (unsubscribing both stores, draining both timers) and mounts one
that skips subscription setup entirely; false->true disposes an instance that
never subscribed (the unsubscribes are no-ops, both timers null) and mounts one
that subscribes and primes from the stores.
The data row's key is unchanged. No behaviour change when Enabled is stable.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
Joseph Doherty