perf(dcl): coalesce MxGateway alarm stream reopens across a subscribe burst (review 2026-10-21 02/P3)

This commit is contained in:
Joseph Doherty
2026-10-22 06:44:13 -04:00
parent e6406c494c
commit d8d44e0b87
7 changed files with 189 additions and 36 deletions
@@ -216,6 +216,7 @@ These are configured via `DataConnectionOptions` in `appsettings.json`, not per-
| `SubscribeBatchDelay` | 50ms | Delay BETWEEN reconnect re-subscribe chunks. Not applied on the instance-driven subscribe path — the Deployment Manager already staggers instance startup there. |
| `QualityFlushInterval` | 1s | Coalescing window for pushing tag-quality counters to the health collector |
| `MxSupervisoryAdviseParallelism` | 16 | Max in-flight supervisory advise commands on the MxGateway bulk-subscribe path when the endpoint has no write-user context |
| `MxAlarmStreamReopenDebounce` | 200ms | Window over which MxGateway alarm subscribes are coalesced into ONE reopen of the shared alarm stream (0–5 s; `0` = reopen per subscribe, the pre-2026-10-21 behaviour). Every new source still receives a fresh standing-state snapshot. |
## Subscription Management
@@ -321,7 +322,7 @@ The registered `SourceReference` is a **NodeId** for OPC UA (the picker, CSV and
- **Reconnect handling**: on entering **Reconnecting**, the actor pushes a `NativeAlarmSourceUnavailable` to every alarm subscriber (consumers mark mirrored alarms uncertain rather than clearing them). On successful reconnection it re-subscribes the feed; the adapter re-emits a snapshot, reconciling state.
- **Shared condition filter (last-subscriber-wins)**: because the feed is opened once per source, it carries a single condition filter. A second subscriber that registers a *different* filter for the same source overwrites it (last writer wins) and the actor logs a warning — co-subscribers to one source are expected to agree on the filter.
- **Routing index**: subscribed sources are bucketed by the **first path segment** of their reference (everything before the first `.`), and a transition is matched only against the bucket for its own first segment, plus a residue list of sources that carry no separator at all (a prefix shorter than one segment, which can match other buckets). This is sound because a source reference containing a separator can only prefix a reference sharing its entire first segment. The per-source `StartsWith` test and the condition-type gate inside the bucket are unchanged, so routing decisions are identical to the previous linear scan over every subscribed source; snapshot boundaries bypass the condition filter, but OPC UA boundaries are sent only to the exact source binding. Connection-wide Mx boundaries broadcast to all subscribers while preserving their refresh epoch.
- **Gateway union filter (MxGateway)**: `StreamAlarms` carries a single `alarm_filter_prefix`, so the adapter opens the stream on the **longest common prefix** of the currently subscribed source references and restarts it for each new source, including one covered by the existing prefix, so every newly bound source receives standing state. Each transport refresh has connection-wide start/end markers and a distinct epoch. An unsubscribe never restarts the stream — the prefix it leaves behind is at worst too broad, which may increase bandwidth. Correctness also depends on the epoch-scoped snapshot cache merging concurrent live updates and rejecting stale snapshot ends (DC-5); a broad filter alone provides no such guarantee. The prefix is a bandwidth optimisation only; the actor's per-source + condition-type gate remains authoritative, mirroring the OPC UA server-side WhereClause stance.
- **Gateway union filter (MxGateway)**: `StreamAlarms` carries a single `alarm_filter_prefix`, so the adapter opens the stream on the **longest common prefix** of the currently subscribed source references and restarts it for each new source, including one covered by the existing prefix, so every newly bound source receives standing state. Restarts are **coalesced** (review 2026-10-21): subscribes arriving within `MxAlarmStreamReopenDebounce` (200 ms) of the first share ONE restart on the union prefix, so a staggered deploy batch of N alarm-bound instances costs one close/reopen and one connection-wide snapshot replay instead of N (each of which was broadcast to every already-subscribed source — O(N·M) boundary messages). A reconnect, teardown or last unsubscribe invalidates a pending restart. Each transport refresh has connection-wide start/end markers and a distinct epoch. An unsubscribe never restarts the stream — the prefix it leaves behind is at worst too broad, which may increase bandwidth. Correctness also depends on the epoch-scoped snapshot cache merging concurrent live updates and rejecting stale snapshot ends (DC-5); a broad filter alone provides no such guarantee. The prefix is a bandwidth optimisation only; the actor's per-source + condition-type gate remains authoritative, mirroring the OPC UA server-side WhereClause stance.
### Protocol-Neutral Types & Messages
@@ -468,4 +469,4 @@ The DCL reports the following metrics to the Health Monitoring component via the
OPC UA connection and endpoint verification select exactly the configured security mode. If a protected mode is unavailable, or discovery fails while protection was requested, the operation fails with the requested mode in its diagnostic; it never falls back to an unsecured session.
DCL retains each subscribed alarm source's current standing state and its in-progress snapshot. A new subscriber sharing a feed receives a scoped replay, or the already-received prefix of an in-progress frame followed by the remaining frame. Cache state is discarded on reconnect and last-owner unsubscribe. A new Mx source reopens the shared stream even when the existing prefix covers it, obtaining a fresh standing-state snapshot; callbacks from the canceled stream are ignored. Mx refresh boundaries remain connection-wide but share a distinct per-stream epoch. Both replay cache and runtime mirror ignore stale-epoch rows and apply newer live raises/clears over a concurrent refresh before committing it. The OPC UA client's per-condition `(active, acked)` memory used to derive transition kinds follows the same removal rule as the replay cache — a condition that returns to inactive + acknowledged (the default prior state) is forgotten — so it stays bounded on servers whose condition names carry a per-occurrence component (review 2026-10-21).
DCL retains each subscribed alarm source's current standing state and its in-progress snapshot. A new subscriber sharing a feed receives a scoped replay, or the already-received prefix of an in-progress frame followed by the remaining frame. Cache state is discarded on reconnect and last-owner unsubscribe. A new Mx source reopens the shared stream (coalesced with any other new sources inside the reopen window) even when the existing prefix covers it, obtaining a fresh standing-state snapshot; callbacks from the canceled stream are ignored. Mx refresh boundaries remain connection-wide but share a distinct per-stream epoch. Both replay cache and runtime mirror ignore stale-epoch rows and apply newer live raises/clears over a concurrent refresh before committing it. The OPC UA client's per-condition `(active, acked)` memory used to derive transition kinds follows the same removal rule as the replay cache — a condition that returns to inactive + acknowledged (the default prior state) is forgotten — so it stays bounded on servers whose condition names carry a per-occurrence component (review 2026-10-21).