fix(alarms): truncation-safe transitions; perf: single-pass alarm parse; configurable poll cadence

This commit is contained in:
Joseph Doherty
2026-08-15 17:04:06 -04:00
parent 94fdc18c3c
commit f3e1de5f37
10 changed files with 910 additions and 33 deletions
+50
View File
@@ -135,6 +135,56 @@ alarm state is gateway-wide, not session-scoped — every client wants the same
current set plus updates, and forcing each to own a worker would multiply AVEVA
polling load for no benefit.
### Alarms — a capped snapshot fetch never implies a clear
Decision (2026-08-15): when the worker's `GetXmlCurrentAlarms2` fetch comes back
holding exactly `MxGateway:Alarms:MaxAlarmsPerFetch` records, the worker treats
the snapshot as **truncated** and merges it into the retained snapshot instead
of replacing it. Alarms the capped reply did carry update normally; alarms it
had no room to mention are retained untouched.
The COM API caps its reply at `maxAlmCnt` and exposes no "more available" flag,
so a reply sitting exactly on the cap is indistinguishable from a galaxy that
happens to hold exactly that many active alarms. Both are treated as truncated,
because the two error directions are not symmetric.
Nothing in the worker emits a Clear transition. The clear is an **inference**:
`WnWrapAlarmConsumer.ComputeTransitions` produces no transition for an alarm
that disappears from the snapshot, and `GatewayAlarmMonitor.ApplyReconcile`
later diffs its cache against `SnapshotActiveAlarms()` and broadcasts a Clear
for every cached alarm the worker no longer reports. Before this decision, a
capped fetch shrank that snapshot, so every alarm past the cap was broadcast as
cleared while still standing — a silent, galaxy-wide false clear on exactly the
alarm floods where the cap is reached.
Consequences, and how this sits with the existing failover/reconcile design:
- **The suppression is an eviction guard, not a transition filter.** It lives in
the snapshot update inside `PollOnce`, not in `ComputeTransitions`, which was
never going to emit anything for a disappearance. The reconcile/dedup
machinery (`_clearedByReconcile` tombstones, the NEXT-03 duplicate-Clear
suppression) is untouched: it still sees the same shape of snapshot, only
with the truncated poll's unmentionable alarms still present.
- **It preserves at-least-once, idempotent application.** The failure mode
becomes bounded staleness — a genuinely cleared alarm can linger until the
first sub-cap fetch evicts it, and the reconcile then broadcasts its Clear
late. A late Clear is repaired by the next complete poll; a Clear that never
happened is broadcast to every `StreamAlarms` subscriber and cannot be taken
back. Consumers already apply transitions as "set this alarm to this state",
so a repeated or delayed Clear is absorbed.
- **It does not synthesize anything.** Suppressing an inference is the opposite
of inventing an event; no transition is fabricated on a truncated poll.
- **Failover is unaffected.** `FailoverAlarmConsumer` selects which
`IMxAccessAlarmConsumer` is live; the guard is internal to the wnwrap
consumer's own snapshot bookkeeping and changes neither the failure counting
that triggers failover nor the subtag standby's snapshot, which is built from
a bounded watch-list and has no per-fetch cap to hit.
- **Operators get told.** A truncated poll logs a rate-limited (once per
minute) `AlarmSnapshotTruncated` warning carrying the cap, the record counts,
and the running truncated-fetch total — identifiers and counts only, never
tag names, values, limits, or comments. A galaxy that truncates persistently
is a configuration problem: raise `MxGateway:Alarms:MaxAlarmsPerFetch`.
## Session-Resilience Epic Scope
Decision (2026-07-09, archreview TST-04; migrated here 2026-08-07 from the retired