From bd7abf4e3580b53527649dbf4ac6ca4f508fff78 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 22 Oct 2026 06:32:17 -0400 Subject: [PATCH] fix(dashboard): re-check hub viewers under lock before starting a pump; document alarm send-timeout tradeoff --- docs/GatewayDashboardDesign.md | 2 +- .../Dashboard/Hubs/DashboardEventBroadcaster.cs | 5 ++++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/GatewayDashboardDesign.md b/docs/GatewayDashboardDesign.md index 1c01074..e6feaa7 100644 --- a/docs/GatewayDashboardDesign.md +++ b/docs/GatewayDashboardDesign.md @@ -274,7 +274,7 @@ bearer). Each hub class is `[Authorize(Policy = HubClientsPolicy)]`. | Hub | Path | Producer | Payload | Routing | |---|---|---|---|---| | `DashboardSnapshotHub` | `/hubs/snapshot` | `DashboardSnapshotPublisher` (BackgroundService consuming `IDashboardSnapshotService.WatchSnapshotsAsync`) | `SnapshotUpdated`: `DashboardSnapshot` **without** `configuration`; `ConfigurationUpdated`: `EffectiveGatewayConfiguration` | Ticks go to all connected clients on every snapshot interval, but only while at least one client is connected (see "Idle gating" below). The effective configuration is startup-static and the largest part of the record, so it is `[JsonIgnore]`d on `DashboardSnapshot` and sent once per connection instead: `OnConnectedAsync` pushes `ConfigurationUpdated` and then the current `SnapshotUpdated`, and the hub method `GetConfiguration()` returns it on demand. Each tick broadcast is bounded to 1 s; a send that misses the bound is abandoned, so one slow client cannot delay the next tick for everyone. | -| `AlarmsHub` | `/hubs/alarms` | `AlarmsHubPublisher` (BackgroundService consuming `IGatewayAlarmService.StreamAsync(filter: null)`) | `AlarmFeedMessage` (`active_alarm` / `snapshot_complete` / `transition` / `provider_status` / `snapshot_status`) | Guarded by `MxGateway.Dashboard.Alarms`, not `HubClients`: a real authenticated dashboard role is required, so the `Authentication:Mode=Disabled` and anonymous-localhost bypasses are excluded (SEC-40). Connected clients auto-join `__alarms__`; all admitted clients receive every message — the feed is galaxy-wide and carries no session provenance, so no per-caller scoping applies (see "Alarm-feed visibility"). Publisher auto-reconnects every 5s on stream faults. Every subscription to the monitor — the first and each resubscribe — is preceded by an `AlarmFeedReset` frame (`{ generation }`, monotonic per process): the monitor primes each new subscription with a full `active_alarm` snapshot ending in `snapshot_complete`, so a client must discard its accumulated alarm state on the marker and rebuild from that snapshot rather than double-counting it. Each broadcast is bounded to 2 s and abandoned on timeout: awaiting it unbounded let one slow client stall the drain until the monitor dropped the publisher as a slow subscriber, and the resubscribe then replayed the whole snapshot to every client. The alarm value fields (`current_value` / `limit_value`) are stripped from a deep-cloned copy of the message when `Dashboard:ShowTagValues` is false (the default) — on both value-bearing payload arms, so neither a live transition nor a snapshot record leaks a process value; every other field still renders. The source message is never mutated: it fans out from the same feed to gRPC `StreamAlarms` subscribers, which this dashboard-display flag does not govern. Arms carrying no value are forwarded as-is, uncloned. | +| `AlarmsHub` | `/hubs/alarms` | `AlarmsHubPublisher` (BackgroundService consuming `IGatewayAlarmService.StreamAsync(filter: null)`) | `AlarmFeedMessage` (`active_alarm` / `snapshot_complete` / `transition` / `provider_status` / `snapshot_status`) | Guarded by `MxGateway.Dashboard.Alarms`, not `HubClients`: a real authenticated dashboard role is required, so the `Authentication:Mode=Disabled` and anonymous-localhost bypasses are excluded (SEC-40). Connected clients auto-join `__alarms__`; all admitted clients receive every message — the feed is galaxy-wide and carries no session provenance, so no per-caller scoping applies (see "Alarm-feed visibility"). Publisher auto-reconnects every 5s on stream faults. Every subscription to the monitor — the first and each resubscribe — is preceded by an `AlarmFeedReset` frame (`{ generation }`, monotonic per process): the monitor primes each new subscription with a full `active_alarm` snapshot ending in `snapshot_complete`, so a client must discard its accumulated alarm state on the marker and rebuild from that snapshot rather than double-counting it. Each broadcast is bounded to 2 s and abandoned on timeout: awaiting it unbounded let one slow client stall the drain until the monitor dropped the publisher as a slow subscriber, and the resubscribe then replayed the whole snapshot to every client. The tradeoff: a client whose send is abandoned misses that one frame and receives no reset for it, so its alarm view can be stale until the next resubscribe (or until SignalR drops the stalled connection); there is no in-family consumer of `/hubs/alarms` today. The alarm value fields (`current_value` / `limit_value`) are stripped from a deep-cloned copy of the message when `Dashboard:ShowTagValues` is false (the default) — on both value-bearing payload arms, so neither a live transition nor a snapshot record leaks a process value; every other field still renders. The source message is never mutated: it fans out from the same feed to gRPC `StreamAlarms` subscribers, which this dashboard-display flag does not govern. Arms carrying no value are forwarded as-is, uncloned. | | `EventsHub` | `/hubs/events` | `DashboardEventBroadcaster` invoked by each session's internal dashboard-mirror subscriber on its `SessionEventDistributor` (registered when the session becomes Ready) | `MxEvent` | Clients call `SubscribeSession(sessionId)` to join `session:{id}`, which also registers them in `EventsHubViewerRegistry`. Delivery to the group goes through one bounded per-session pump (see "Mirror gating"), so a slow viewer loses its oldest events instead of accumulating pending sends — the mirror is gated on that registry, which counts hub and in-process viewers alike (see "Mirror gating" below). The dashboard is a first-class distributor subscriber, so it receives the session's events whether or not a gRPC client is streaming. It sees RAW session events — not the per-gRPC-subscriber `AfterWorkerSequence` filtering that `EventStreamService` applies at its own boundary — because the dashboard is a separate LDAP-authenticated monitoring view meant to show the session's full event activity. Tag values are stripped from the mirrored `MxEvent` copy by `DashboardEventBroadcaster` when `Dashboard:ShowTagValues` is false (the default) — event metadata (tag reference, quality, status, timestamps) still renders, but the value fields are blanked, so no value leaks through this seam. `SubscribeSession` is gated by `IDashboardSessionAcl` (SEC-25 / TST-15): a denied caller gets a `HubException`, is not joined to the group, and is not registered as a viewer, so the mirror stays off for a session nobody is legitimately watching. The same ACL gates the in-process seam the session-details page uses, so neither path is the weaker one. Value redaction remains an independent layer — it bounds what a *permitted* subscriber sees. | No hub client ships in this repository: the server-rendered pages read the diff --git a/src/ZB.MOM.WW.MxGateway.Server/Dashboard/Hubs/DashboardEventBroadcaster.cs b/src/ZB.MOM.WW.MxGateway.Server/Dashboard/Hubs/DashboardEventBroadcaster.cs index 5789d31..1287aaf 100644 --- a/src/ZB.MOM.WW.MxGateway.Server/Dashboard/Hubs/DashboardEventBroadcaster.cs +++ b/src/ZB.MOM.WW.MxGateway.Server/Dashboard/Hubs/DashboardEventBroadcaster.cs @@ -209,7 +209,10 @@ public sealed class DashboardEventBroadcaster : IDashboardEventBroadcaster, IDas { lock (_syncRoot) { - if (_disposed) + // Re-check under the lock: the last viewer may have left between the + // caller's unlocked check and here, and its release handler (which takes + // this lock) would then never see a pump created after it ran. + if (_disposed || !_viewerRegistry.HasHubViewers(sessionId)) { return; }