fix(opcua): ConditionRefresh on a Server-object item is not overtaken by older deferred condition events (#854)
While a folder subscriber's SDK channel is full, a condition's later events wait in its #838 deferral queue and reach i=2253 items only when the drain gets to them, off Lock. A ConditionRefresh in that window replayed the current state, then the older deferred events followed RefreshEnd (stale state, or a condition the refresh had just dropped re-added until the newest drained). - Condition events are reported as ConditionEventSnapshot (an InstanceStateSnapshot carrying a report sequence, condition NodeId and Retain); events queued behind a deferred report are marked Deferred. - Server-object refresh (ConditionRefresh and ConditionRefresh2, flag on AND off) now runs in RefreshServerObjectItems: watermark, "anything deferred?", the replay and its queueing in ONE Lock hold, so the replay is exactly the state at the cut. Per item it queues the replay and installs a RefreshCut under the item's gate. - OnConditionReportEvent holds that gate across "superseded? -> QueueEvent" for deferred events only. Withheld from that item alone: events at/below the cut whose condition the replay re-sent, or Retain=true events of a condition it omitted; an older Retain=false event of an omitted condition is still delivered. Queues, other Server-object items and folder items are untouched; refresh never waits for a drain. - Lock order Lock -> item gate -> SDK item lock, same as a synchronous report. - Meter otopcua.opcua.condition_events.superseded. - Folder items keep a documented residual (SDK channel backlog + memoised per-item verdict leave no per-event hook). Tests: ConditionRefreshDeferralWireTests (deterministic overload by holding the folder item's SDK lock; 6 variants fail before the fix; probe-driven gate test fails with the gate removed; concurrent reports/refreshes/stall stress test fails before the fix).
This commit is contained in:
+27
-4
@@ -882,15 +882,38 @@ Behaviours worth knowing before enabling it:
|
||||
changes which folders a later hop reaches — and the liveness check covers removal); `MonitoredNode2.Dispose` completes
|
||||
its channel writer, which fails a pending `WriteAsync` with `ChannelClosedException` (swallowed by the SDK), so a
|
||||
disposed node cannot stall a condition's queue; and the delivery task catches per-event exceptions, so it keeps draining.
|
||||
- ⚠️ **Residual (documented, not fixed): `ConditionRefresh` does not flush deferrals.** While a notifier channel is
|
||||
overloaded, a client that refreshes may receive older deferred events of a condition after `RefreshEndEvent`. They
|
||||
are still in that condition's order and its newest event arrives last, so the state a client ends up holding is
|
||||
correct; making refresh wait for the queues would block a service call on the very drain the overload is stalling.
|
||||
- **`ConditionRefresh` is not overtaken by deferred events on Server-object items (Gitea #854).** A refresh of an
|
||||
`i=2253` item replays each condition's current state between RefreshStart and RefreshEnd; the condition's deferred
|
||||
events — all older — used to follow RefreshEnd, briefly showing a stale state or re-adding an alarm the refresh had
|
||||
just dropped (ScadaBridge's snapshot swap drops absent conditions, then a stale active event re-added it). Now every
|
||||
condition event carries its report sequence; the refresh takes its **cut** (the sequence, whether anything is
|
||||
deferred, the replay, and its queueing) in **one hold of `Lock`**, so the replay is exactly the state at the cut, and
|
||||
installs it per item under that item's gate, which a deferred delivery also holds across its check and
|
||||
`QueueEvent`. A deferred event at or below the cut is withheld **from that item only** when the refresh re-sent its
|
||||
condition, or when it is `Retain=true` for a condition the refresh omitted; an older `Retain=false` event of an
|
||||
omitted condition still arrives (it agrees with the omission, and a client that does not reconcile on RefreshEnd
|
||||
needs it). The deferral queues, every other Server-object item and every folder item are untouched; the refresh never
|
||||
waits for a drain. Meter: `otopcua.opcua.condition_events.superseded` (per item and event). Both ConditionRefresh and
|
||||
ConditionRefresh2, flag on and off.
|
||||
- ⚠️ **Residual: folder (notifier) items.** A refresh of an item on an equipment/device folder can still be followed
|
||||
by older events of a condition. Two sources, neither reachable per item: the SDK's own `MonitoredNode2` channel
|
||||
(up to 4096 events already accepted before the refresh — this exists in the stock SDK too, and in normal operation
|
||||
for the few events in transit), and our deferral queue, whose folder hop goes through that channel, where the SDK
|
||||
memoises the per-item permission verdict by (item, event type, source) so there is no per-event, per-item hook. The
|
||||
events stay in the condition's order and its newest arrives last, so the client converges. Discarding the queue
|
||||
globally on refresh would drop transitions for every non-refreshing subscriber while leaving the channel backlog;
|
||||
making refresh wait would block a service call on the stalled drain. ScadaBridge also ignores a transition older
|
||||
than the one it holds for a condition.
|
||||
|
||||
Pinned by `FolderEventFloodWireTests`: 6000 events under one `Lock` hold with publishing disabled (flag on and off)
|
||||
complete, drain to zero, coalesce, and the item delivers at most its revised `QueueSize` in increasing order ending
|
||||
with the last event; and a condition **removed** in the same `Lock` hold has none of its deferred events delivered to
|
||||
either the folder or the Server-object item (the flag-off variant fails without the liveness checks).
|
||||
`ConditionRefreshDeferralWireTests` (#854) stalls a folder item's SDK lock so its channel fills without `Lock` held,
|
||||
refreshes the Server-object item (ConditionRefresh and ConditionRefresh2, flag on/off, condition retained or cleared)
|
||||
and pins that nothing older than the replay follows RefreshEnd while the non-refreshing folder item still receives the
|
||||
whole tail; a probe-driven test pins that a refresh waits for a delivery already past its check (fails without the
|
||||
item gate); and a stress test runs reports, refreshes and an intermittent stall concurrently.
|
||||
|
||||
**Model-change and audit events on the Server object are gated too** (Gitea #588). The node manager's
|
||||
own server-wide events — `GeneralModelChangeEvent` (NodeAdded / NodeDeleted / DataTypeChanged) and the
|
||||
|
||||
Reference in New Issue
Block a user