docs(dashboard): describe the pages as shipped and the bounded hub/feed behaviour
Rewrite the Pages 'Show:' lists to what each page actually renders (no per-session subscriber/pending-command/queue counters, no worker memory/CPU, no per-family command counters, counters not rates). Document the per-session EventsHub pump, alarm-hub send bound and AlarmFeedReset marker, configuration sent once per snapshot-hub connection, health-derived gateway status, Browse search debounce/index, bounded disposal, 30s circuit retention with connection-gated polling, per-viewer advise-set share, and AlarmsPage memo.
This commit is contained in:
+151
-73
@@ -273,9 +273,14 @@ bearer). Each hub class is `[Authorize(Policy = HubClientsPolicy)]`.
|
||||
|
||||
| Hub | Path | Producer | Payload | Routing |
|
||||
|---|---|---|---|---|
|
||||
| `DashboardSnapshotHub` | `/hubs/snapshot` | `DashboardSnapshotPublisher` (BackgroundService consuming `IDashboardSnapshotService.WatchSnapshotsAsync`) | `DashboardSnapshot` | Sent to all connected clients on every snapshot tick, but only while at least one client is connected (see "Idle gating" below); new connections receive the current snapshot synchronously in `OnConnectedAsync`. |
|
||||
| `AlarmsHub` | `/hubs/alarms` | `AlarmsHubPublisher` (BackgroundService consuming `IGatewayAlarmService.StreamAsync(filter: null)`) | `AlarmFeedMessage` (`active_alarm` / `snapshot_complete` / `transition` / `provider_status` / `snapshot_status`) | Guarded by `MxGateway.Dashboard.Alarms`, not `HubClients`: a real authenticated dashboard role is required, so the `Authentication:Mode=Disabled` and anonymous-localhost bypasses are excluded (SEC-40). Connected clients auto-join `__alarms__`; all admitted clients receive every message — the feed is galaxy-wide and carries no session provenance, so no per-caller scoping applies (see "Alarm-feed visibility"). Publisher auto-reconnects every 5s on stream faults. The alarm value fields (`current_value` / `limit_value`) are stripped from a deep-cloned copy of the message when `Dashboard:ShowTagValues` is false (the default) — on both value-bearing payload arms, so neither a live transition nor a snapshot record leaks a process value; every other field still renders. The source message is never mutated: it fans out from the same feed to gRPC `StreamAlarms` subscribers, which this dashboard-display flag does not govern. Arms carrying no value are forwarded as-is, uncloned. |
|
||||
| `EventsHub` | `/hubs/events` | `DashboardEventBroadcaster` invoked by each session's internal dashboard-mirror subscriber on its `SessionEventDistributor` (registered when the session becomes Ready) | `MxEvent` | Clients call `SubscribeSession(sessionId)` to join `session:{id}`, which also registers them in `EventsHubViewerRegistry` — the mirror is gated on that registry, which counts hub and in-process viewers alike (see "Mirror gating" below). The dashboard is a first-class distributor subscriber, so it receives the session's events whether or not a gRPC client is streaming. It sees RAW session events — not the per-gRPC-subscriber `AfterWorkerSequence` filtering that `EventStreamService` applies at its own boundary — because the dashboard is a separate LDAP-authenticated monitoring view meant to show the session's full event activity. Tag values are stripped from the mirrored `MxEvent` copy by `DashboardEventBroadcaster` when `Dashboard:ShowTagValues` is false (the default) — event metadata (tag reference, quality, status, timestamps) still renders, but the value fields are blanked, so no value leaks through this seam. `SubscribeSession` is gated by `IDashboardSessionAcl` (SEC-25 / TST-15): a denied caller gets a `HubException`, is not joined to the group, and is not registered as a viewer, so the mirror stays off for a session nobody is legitimately watching. The same ACL gates the in-process seam the session-details page uses, so neither path is the weaker one. Value redaction remains an independent layer — it bounds what a *permitted* subscriber sees. |
|
||||
| `DashboardSnapshotHub` | `/hubs/snapshot` | `DashboardSnapshotPublisher` (BackgroundService consuming `IDashboardSnapshotService.WatchSnapshotsAsync`) | `SnapshotUpdated`: `DashboardSnapshot` **without** `configuration`; `ConfigurationUpdated`: `EffectiveGatewayConfiguration` | Ticks go to all connected clients on every snapshot interval, but only while at least one client is connected (see "Idle gating" below). The effective configuration is startup-static and the largest part of the record, so it is `[JsonIgnore]`d on `DashboardSnapshot` and sent once per connection instead: `OnConnectedAsync` pushes `ConfigurationUpdated` and then the current `SnapshotUpdated`, and the hub method `GetConfiguration()` returns it on demand. Each tick broadcast is bounded to 1 s; a send that misses the bound is abandoned, so one slow client cannot delay the next tick for everyone. |
|
||||
| `AlarmsHub` | `/hubs/alarms` | `AlarmsHubPublisher` (BackgroundService consuming `IGatewayAlarmService.StreamAsync(filter: null)`) | `AlarmFeedMessage` (`active_alarm` / `snapshot_complete` / `transition` / `provider_status` / `snapshot_status`) | Guarded by `MxGateway.Dashboard.Alarms`, not `HubClients`: a real authenticated dashboard role is required, so the `Authentication:Mode=Disabled` and anonymous-localhost bypasses are excluded (SEC-40). Connected clients auto-join `__alarms__`; all admitted clients receive every message — the feed is galaxy-wide and carries no session provenance, so no per-caller scoping applies (see "Alarm-feed visibility"). Publisher auto-reconnects every 5s on stream faults. Every subscription to the monitor — the first and each resubscribe — is preceded by an `AlarmFeedReset` frame (`{ generation }`, monotonic per process): the monitor primes each new subscription with a full `active_alarm` snapshot ending in `snapshot_complete`, so a client must discard its accumulated alarm state on the marker and rebuild from that snapshot rather than double-counting it. Each broadcast is bounded to 2 s and abandoned on timeout: awaiting it unbounded let one slow client stall the drain until the monitor dropped the publisher as a slow subscriber, and the resubscribe then replayed the whole snapshot to every client. The alarm value fields (`current_value` / `limit_value`) are stripped from a deep-cloned copy of the message when `Dashboard:ShowTagValues` is false (the default) — on both value-bearing payload arms, so neither a live transition nor a snapshot record leaks a process value; every other field still renders. The source message is never mutated: it fans out from the same feed to gRPC `StreamAlarms` subscribers, which this dashboard-display flag does not govern. Arms carrying no value are forwarded as-is, uncloned. |
|
||||
| `EventsHub` | `/hubs/events` | `DashboardEventBroadcaster` invoked by each session's internal dashboard-mirror subscriber on its `SessionEventDistributor` (registered when the session becomes Ready) | `MxEvent` | Clients call `SubscribeSession(sessionId)` to join `session:{id}`, which also registers them in `EventsHubViewerRegistry`. Delivery to the group goes through one bounded per-session pump (see "Mirror gating"), so a slow viewer loses its oldest events instead of accumulating pending sends — the mirror is gated on that registry, which counts hub and in-process viewers alike (see "Mirror gating" below). The dashboard is a first-class distributor subscriber, so it receives the session's events whether or not a gRPC client is streaming. It sees RAW session events — not the per-gRPC-subscriber `AfterWorkerSequence` filtering that `EventStreamService` applies at its own boundary — because the dashboard is a separate LDAP-authenticated monitoring view meant to show the session's full event activity. Tag values are stripped from the mirrored `MxEvent` copy by `DashboardEventBroadcaster` when `Dashboard:ShowTagValues` is false (the default) — event metadata (tag reference, quality, status, timestamps) still renders, but the value fields are blanked, so no value leaks through this seam. `SubscribeSession` is gated by `IDashboardSessionAcl` (SEC-25 / TST-15): a denied caller gets a `HubException`, is not joined to the group, and is not registered as a viewer, so the mirror stays off for a session nobody is legitimately watching. The same ACL gates the in-process seam the session-details page uses, so neither path is the weaker one. Value redaction remains an independent layer — it bounds what a *permitted* subscriber sees. |
|
||||
|
||||
No hub client ships in this repository: the server-rendered pages read the
|
||||
in-process seams above, and the hubs serve external consumers. Those consumers
|
||||
must honour `AlarmFeedReset` and keep the configuration from
|
||||
`ConfigurationUpdated`, since ticks no longer carry it.
|
||||
|
||||
### Default cadences
|
||||
|
||||
@@ -343,12 +348,17 @@ subscribers. Two races motivate the extra state:
|
||||
gate open, and cancelling on their behalf would stop a live pump that other
|
||||
viewers depend on.
|
||||
|
||||
Two per-tick costs inside the snapshot itself are bounded independently of either gate:
|
||||
Three per-tick costs inside the snapshot itself are bounded independently of either gate:
|
||||
|
||||
- the effective configuration (`EffectiveGatewayConfiguration`) is built once and
|
||||
cached. It is a projection of `IOptions<GatewayOptions>`, which the gateway binds
|
||||
at startup and never reloads, so rebuilding the whole option tree every tick
|
||||
produced an identical object;
|
||||
produced an identical object. For the same reason it is not serialized on the
|
||||
`/hubs/snapshot` tick (see the hub table above);
|
||||
- the gateway status is one `HealthCheckService.CheckHealthAsync` evaluation per
|
||||
tick, bounded to 2 s and behind a zero-wait gate so concurrent enumerations (the
|
||||
hub publisher and the page feed) do not both evaluate; `GetSnapshot()` reads the
|
||||
cached result;
|
||||
- the API key summaries are refreshed at most once every 15 seconds
|
||||
(`ApiKeySummaryRefreshInterval`) instead of on every tick. The list is a SQLite
|
||||
read whose content changes only when an operator creates, rotates, or revokes a
|
||||
@@ -359,7 +369,7 @@ Two per-tick costs inside the snapshot itself are bounded independently of eithe
|
||||
Avoid pushing every MXAccess data-change event into a wider broadcast group.
|
||||
Events are routed strictly per session (`session:{id}` groups on the hub,
|
||||
per-session subscriber lists in process); the snapshot seams continue to carry
|
||||
aggregate event counters and rates.
|
||||
aggregate event counters.
|
||||
|
||||
### Mirror gating
|
||||
|
||||
@@ -394,10 +404,21 @@ group. With `ShowTagValues` true there is no clone at all — the original `MxEv
|
||||
instance is handed to both audiences — so the "one clone per event" cost holds only
|
||||
in the redacting configuration, and in the value-showing one both audiences share a
|
||||
reference to the session pipeline's own event object. In-process delivery runs first
|
||||
and synchronously — it cannot
|
||||
throw, and it must not be skipped by the guard clause around the hub send — into
|
||||
per-subscriber bounded drop-oldest channels, so a page that falls behind loses its
|
||||
oldest queued events rather than blocking the session's event pipeline.
|
||||
and synchronously — it cannot throw — into per-subscriber bounded drop-oldest
|
||||
channels, so a page that falls behind loses its oldest queued events rather than
|
||||
blocking the session's event pipeline.
|
||||
|
||||
Hub delivery has the same shape. `EventsHubViewerRegistry` counts remote hub
|
||||
connections separately from `inproc-` subscribers (`HasHubViewers`), and while a
|
||||
session has at least one, `Publish` writes the event into that session's
|
||||
`HubPump`: a 256-deep drop-oldest channel drained by a single task that sends one
|
||||
event at a time to `session:{id}`, each send bounded to 5 s. A session watched only
|
||||
in process starts no pump. The pump starts on the first event published while a
|
||||
hub viewer is present and is torn down when the registry raises
|
||||
`HubViewersReleased` for the session's last hub viewer (unsubscribe or disconnect).
|
||||
Before this, `Publish` fired a group `SendAsync` per event and returned; a slow
|
||||
remote viewer made each send stay pending in the SignalR lifetime manager, and the
|
||||
pending sends — each holding a cloned event — grew without bound.
|
||||
|
||||
The mirror subscriber itself is still registered on the `SessionEventDistributor`
|
||||
for the session's whole lifetime; only the per-event work is gated. Starting and
|
||||
@@ -411,55 +432,67 @@ clone and send this gate already removes.
|
||||
|
||||
### Dashboard home
|
||||
|
||||
Show top-level status:
|
||||
The Overview page (`/`) shows, from the snapshot:
|
||||
|
||||
- gateway status,
|
||||
- gateway version,
|
||||
- uptime,
|
||||
- open sessions,
|
||||
- workers running,
|
||||
- sessions faulted,
|
||||
- command rate,
|
||||
- command failure count,
|
||||
- event rate,
|
||||
- event queue depth,
|
||||
- worker restart/kill count.
|
||||
- gateway status badge — the aggregate status of the registered ASP.NET Core
|
||||
health checks (`Healthy` / `Degraded` / `Unhealthy`), evaluated once per snapshot
|
||||
tick with a 2 s bound and cached for the synchronous `GetSnapshot()` seed path. It
|
||||
reads `Unknown` before the first evaluation, when an evaluation throws or times
|
||||
out, and when no `HealthCheckService` is registered — never a hardcoded `Healthy`
|
||||
(SEC-27);
|
||||
- the snapshot's generation time;
|
||||
- metric cards: uptime (with the gateway version as detail), open sessions, workers
|
||||
running, worker event queue depth, failed commands, events received, faults, and
|
||||
queue overflows — all cumulative counters or point-in-time gauges, not rates;
|
||||
- a Galaxy Repository section: cache status, last deploy time, object count (with
|
||||
area count), attribute, historized-attribute and alarm-attribute counts, and the
|
||||
last refresh time;
|
||||
- the recent faults list (`Dashboard:RecentFaultLimit`).
|
||||
|
||||
Use Bootstrap cards for individual metric summaries. Keep the layout compact
|
||||
and operational.
|
||||
There is no per-family command or event rate, no faulted-session counter, and no
|
||||
worker restart/kill card on this page; `mxgateway.workers.killed` and
|
||||
`mxgateway.workers.exited` are in the snapshot's metric list but not rendered.
|
||||
Metric summaries use Bootstrap cards; the layout stays compact and operational.
|
||||
|
||||
### Sessions page
|
||||
|
||||
Show active and recent sessions in a table:
|
||||
Shows the most recent sessions (`Dashboard:RecentSessionLimit`, newest first) in a
|
||||
table with these columns:
|
||||
|
||||
- session id,
|
||||
- client identity or API key display name,
|
||||
- state,
|
||||
- session id (links to the session details page),
|
||||
- session state,
|
||||
- client identity (redacted),
|
||||
- backend,
|
||||
- worker process id,
|
||||
- worker process id and worker state,
|
||||
- events received for the session (the `mxgateway.events.received` per-session
|
||||
counter),
|
||||
- open time,
|
||||
- last client activity,
|
||||
- last worker heartbeat,
|
||||
- active event subscribers,
|
||||
- pending commands,
|
||||
- event queue depth,
|
||||
- last fault summary.
|
||||
- last fault summary (abbreviated, full text on hover),
|
||||
- Close / Kill actions for Admin principals.
|
||||
|
||||
Rows should link to session details.
|
||||
Event subscriber counts, pending command counts, and per-session queue depth are
|
||||
not tracked per session and are not shown.
|
||||
|
||||
### Session details page
|
||||
|
||||
Show:
|
||||
Shows:
|
||||
|
||||
- session metadata,
|
||||
- worker metadata,
|
||||
- command counters by method,
|
||||
- event counters by family,
|
||||
- active server handles and item counts if gateway shadow state has them,
|
||||
- latest faults,
|
||||
- last heartbeat payload,
|
||||
- a session section: state badge, backend, client identity, client session name,
|
||||
client correlation id (all redacted), open time, last activity, lease expiry,
|
||||
events received, and last fault;
|
||||
- a worker section: process id, worker state, and last heartbeat time;
|
||||
- a "Recent events" table — the newest 50 events from the in-process mirror, with
|
||||
worker sequence, family, server handle, item handle, and status (no values), plus
|
||||
a live/offline pill; gated by the session ACL, so a caller who may not observe the
|
||||
session sees "Not authorized" instead;
|
||||
- admin Close session / Kill worker controls (Admin role only).
|
||||
|
||||
Per-method command counters, per-family event counters for the session, server
|
||||
handle / item counts, and the heartbeat payload are not shown — the gateway keeps
|
||||
no per-session shadow state for them.
|
||||
|
||||
The Sessions list, the Workers list, and this details page all render the same
|
||||
admin controls when the signed-in principal carries the `Admin` role; viewers
|
||||
and the localhost-anonymous bypass see no action affordances and the server
|
||||
@@ -484,33 +517,31 @@ management audit path.
|
||||
|
||||
### Workers page
|
||||
|
||||
Show:
|
||||
Shows one row per worker that is not closed, with:
|
||||
|
||||
- worker process id,
|
||||
- session id,
|
||||
- executable path/version,
|
||||
- state,
|
||||
- startup duration,
|
||||
- memory and CPU if available,
|
||||
- last heartbeat,
|
||||
- current command correlation id,
|
||||
- pending command count,
|
||||
- event queue depth,
|
||||
- restart/kill reason if terminal.
|
||||
- worker state,
|
||||
- owning session id (links to the session details page),
|
||||
- last heartbeat time,
|
||||
- last fault summary (the owning session's final fault),
|
||||
- a Kill worker action for Admin principals.
|
||||
|
||||
Executable path and version, startup duration, memory/CPU, in-flight command
|
||||
correlation, pending command count, and per-worker queue depth are not collected
|
||||
and are not shown; the worker executable path is on the Settings page.
|
||||
|
||||
### Events page
|
||||
|
||||
Show aggregate event diagnostics:
|
||||
Shows aggregate event diagnostics from the snapshot:
|
||||
|
||||
- event rate by session,
|
||||
- event rate by event family,
|
||||
- total events since start,
|
||||
- events received since start,
|
||||
- worker event queue depth and gRPC stream queue depth (gauges),
|
||||
- queue overflow count,
|
||||
- stream disconnect count,
|
||||
- recent terminal faults.
|
||||
- gRPC stream disconnect count,
|
||||
- an "Event Families" table: cumulative events received per event family.
|
||||
|
||||
Do not display full tag values by default. If value display is later added, make
|
||||
it opt-in and redacted.
|
||||
The figures are counters since start, not rates; per-session event counts are on
|
||||
the Sessions page. No tag values are shown here.
|
||||
|
||||
### Browse page
|
||||
|
||||
@@ -537,6 +568,33 @@ at the service, not in the page: one decision point, and the value never enters
|
||||
the render tree. Set the flag true to make the panel the explicit opt-in
|
||||
tag-value surface it was originally designed as.
|
||||
|
||||
The filter box is debounced: keystrokes within 150 ms coalesce into one search,
|
||||
which runs on the thread pool with a cancellation token that the next keystroke
|
||||
cancels, and only the latest query may publish its results. The search reads
|
||||
`DashboardBrowseSearchIndex`, a singleton holding every attribute's full reference
|
||||
and name lowercased once, keyed on the Galaxy cache `Sequence` and rebuilt only when
|
||||
a deploy moves it. The previous filter rescanned the whole cached Galaxy with
|
||||
case-insensitive comparisons, synchronously on the render dispatcher, once per
|
||||
keystroke per circuit.
|
||||
|
||||
The page's two background loops — the 2 s live-read poll and the Galaxy
|
||||
deploy-notification subscription — handle faults inside the loop and retry; only
|
||||
cancellation ends them. A poll fault surfaces in the "Live read failed" banner; a
|
||||
notifier fault surfaces as a "redeploy notifications failed (retrying)" warning and
|
||||
resubscribes after 5 s; the tree rebuild after a redeploy runs on the renderer
|
||||
dispatcher. `DisposeAsync` cancels the loops and any in-flight search, drains them
|
||||
concurrently with the same 5 s bound `DashboardPageBase` uses, and disposes its
|
||||
cancellation sources in a `finally`.
|
||||
|
||||
A closed or backgrounded tab leaves its circuit alive, disconnected, for
|
||||
`DisconnectedCircuitRetentionPeriod`, which the dashboard sets to **30 seconds**
|
||||
(`DashboardServiceCollectionExtensions.DisconnectedCircuitRetention`) instead of
|
||||
the framework's three minutes. Within that window the poll does not read: a scoped
|
||||
`DashboardCircuitConnectionTracker`, registered as the circuit's `CircuitHandler`,
|
||||
reports the browser connection down and the poll skips its ticks until it comes
|
||||
back. Every tick it does run is a `ReadBulk` against the shared dashboard worker,
|
||||
so a closed tab previously kept the worker busy for three minutes.
|
||||
|
||||
### Alarms page
|
||||
|
||||
`/dashboard/alarms` lists the alarms the gateway's central alarm monitor
|
||||
@@ -552,7 +610,10 @@ client, no loopback socket, and no hub token — and the same subscription carri
|
||||
`snapshot_status` frame behind the truncated-snapshot banner ("Alarm snapshot may be
|
||||
incomplete"), so a capped provider fetch is caveated the moment the monitor decides
|
||||
it. The alarm rows themselves still come from the three-second poll, which also
|
||||
re-asserts the truncation verdict as the reconcile baseline. If `MxGateway:Alarms:Enabled` is
|
||||
re-asserts the truncation verdict as the reconcile baseline. The filtered rows and
|
||||
the area list are memoized in `DashboardAlarmView` on the poll's alarm generation and
|
||||
the filter inputs, so the "Showing" card and the table share one filter-and-sort per
|
||||
change rather than recomputing it several times per render. If `MxGateway:Alarms:Enabled` is
|
||||
false the central monitor never starts, and the page says so instead of showing
|
||||
an empty list with no explanation.
|
||||
|
||||
@@ -599,6 +660,19 @@ overflow against the oversized set and evicts the whole excess in one pass (a
|
||||
subscribes nothing new evicts nothing, but neither can it grow the set. A browse
|
||||
page requests far fewer tags than the cap, so in practice the set settles at 256.
|
||||
|
||||
The 256 is shared between viewers rather than one global LRU. Each Browse page
|
||||
reads under its own viewer id, every advised tag records the viewer that read it
|
||||
last, and each active viewer (one that read within the last 30 s) owns a share of
|
||||
`256 / active viewers`, floored at 32. When a read needs room, eviction first takes
|
||||
the least-recently-read tags of *other* viewers that are over their share — an
|
||||
inactive viewer's share is zero, so a closed page's tags go first — and then the
|
||||
reader's own tags that this read did not touch. A viewer within its share is never
|
||||
evicted by someone else's read, so with several viewers the cap is soft: the set
|
||||
can sit above 256 rather than strip a neighbour. Under the global LRU, two circuits
|
||||
browsing different areas evicted each other's whole panels on every poll and
|
||||
re-advised them two seconds later. With a single viewer the behaviour is exactly
|
||||
the global LRU described above.
|
||||
|
||||
The Alarms page does **not** use the dashboard session: alarm data comes from
|
||||
the gateway's always-on central monitor. `QueryAlarmsAsync` reads
|
||||
`IGatewayAlarmService.CurrentAlarms` — the monitor's in-process cache — so the
|
||||
@@ -665,20 +739,24 @@ Secrets and pepper values are never logged.
|
||||
|
||||
### Settings page
|
||||
|
||||
Show read-only effective configuration:
|
||||
Shows the read-only effective configuration:
|
||||
|
||||
- worker executable path,
|
||||
- configured timeouts,
|
||||
- queue capacities,
|
||||
- auth mode,
|
||||
- SQLite auth database path with sensitive parts redacted if needed,
|
||||
- dashboard enabled state,
|
||||
- authentication: mode, SQLite auth database path, pepper secret name, and whether
|
||||
migrations run on startup;
|
||||
- LDAP: enabled, server and port, fallback servers, transport, search base,
|
||||
service account DN, service password (redacted), and the username and group
|
||||
attributes;
|
||||
- the LDAP group mappings that decide what a signed-in user may see —
|
||||
`Dashboard:GroupToRole` (group → `Administrator` / `Viewer`) and
|
||||
`Dashboard:GroupToTag` (group → the session tags it grants),
|
||||
- `Dashboard:UntaggedSessionVisibility`, which decides who sees a session whose
|
||||
owning API key carries no dashboard tags,
|
||||
- protocol version.
|
||||
`Dashboard:GroupToTag` (group → the session tags it grants);
|
||||
- worker: executable path, required architecture, and the startup, shutdown, and
|
||||
heartbeat-grace timeouts;
|
||||
- sessions: default command timeout and max sessions;
|
||||
- events: queue capacity and backpressure policy;
|
||||
- dashboard: enabled, anonymous localhost, snapshot interval, recent fault and
|
||||
session limits, `ShowTagValues`, and `Dashboard:UntaggedSessionVisibility`
|
||||
(who sees a session whose owning API key carries no dashboard tags);
|
||||
- the worker protocol version.
|
||||
|
||||
Do not show API key secrets or pepper values.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user