feat(alarms): structural degraded-status signal for truncated alarm snapshots
The truncation-cliff fix made alarm transitions truncation-safe but silent:
when GetXmlCurrentAlarms2 returns exactly maxAlmCnt records the worker
suppresses absence-implies-Clear inference and says so only in a rate-limited
stderr warning. No client and no operator could tell a complete active set
from a capped one.
Two additive proto3 booleans carry the verdict out:
- QueryActiveAlarmsReplyPayload.snapshot_truncated = 2 (worker IPC reply)
- ActiveAlarmSnapshot.from_truncated_snapshot = 16 (per record)
The per-record field is not an aesthetic choice. QueryActiveAlarms returns a
bare `stream ActiveAlarmSnapshot` with no envelope, header, or trailer, so a
per-record boolean is the only carrier that stays wire-compatible; an envelope
message would change every existing client's stream element type. The reply
payload states it too because a prefix filter can leave zero records and a
truncated fetch with nothing to report still has to say so. The flag means
"this set may be incomplete", never "this record is unreliable" — it is
independent of the subtag-fallback `degraded` field.
Detection is deliberately UNCHANGED: IsTruncatedFetch remains
`fetchedRecordCount >= maxAlarmsPerFetch`. The live probe (docs/AlarmProbeFindings.md,
ce5d8ae) could not verify whether ALARM_RECORDS/@COUNT reports the total active
count or only the records in the reply, so @COUNT is not parsed for detection;
switching to it stays blocked on probe evidence. The probe's comment
annotations in WnWrapAlarmConsumer.cs are preserved.
Reset semantics: not latched. WnWrapAlarmConsumer.FoldFetch replaces the
verdict on every poll under the same lock as the snapshot merge, so the first
sub-cap fetch clears it; GatewayAlarmMonitor.ClearCache drops it with the cache
generation it describes. A caveat that never turns off is one operators learn
to ignore.
Flow: WnWrapAlarmConsumer.LastSnapshotTruncated -> AlarmDispatcher (stamps every
record) / IAlarmCommandHandler (payload) -> MxAccessCommandExecutor reply ->
GatewayAlarmMonitor._snapshotTruncated -> IGatewayAlarmService.SnapshotTruncated
-> DashboardAlarmQueryResult -> AlarmsPage warning banner (render-side only; the
poll loop and DisposeAsync drain are untouched). The public QueryActiveAlarms
RPC forwards worker snapshots unmodified, so the per-record flag needed no
mapper change — a test pins that.
Parity: this describes OUR fetch mechanics — additive gateway metadata — not
MXAccess provider behavior. No event is synthesized and no MXAccess-observable
semantics change, so it is not a parity deviation.
Tests: worker LastSnapshotTruncated set/reset/consecutive-burst (windev-run);
gateway end-to-end truncated reply -> monitor -> public stream, with the
complete-reply control as the load-bearing assertion; AlarmsPage banner
present/absent. Docs: gateway.md alarm surface, docs/DesignDecisions.md entry.
This commit is contained in:
+18
@@ -240,6 +240,24 @@ monitoring (forced)") when subtag mode is the configured `Fallback:Mode=ForceSub
|
||||
as a fault. Metrics: `mxgateway.alarms.provider_mode` gauge (1 = alarmmgr,
|
||||
2 = subtag) and `mxgateway.alarms.provider_switches` counter.
|
||||
|
||||
**Truncated-snapshot visibility:** `GetXmlCurrentAlarms2` caps its reply at
|
||||
`MxGateway:Alarms:MaxAlarmsPerFetch` and offers no confirmed "more available"
|
||||
flag, so a reply holding exactly the cap is treated as truncated. On such a
|
||||
fetch `WnWrapAlarmConsumer` merges rather than replaces its retained snapshot,
|
||||
which suppresses the absence-implies-Clear inference and keeps a capped poll
|
||||
from broadcasting Clears for alarms it simply had no room to mention. That
|
||||
suppression is reported structurally rather than only in a rate-limited worker
|
||||
warning: the `QueryActiveAlarms` reply payload carries `snapshot_truncated`,
|
||||
every `ActiveAlarmSnapshot` in it carries `from_truncated_snapshot`, and the
|
||||
dashboard Alarms tab shows a warning banner while the flag is set. The flag
|
||||
means "this active set may be incomplete", not "this record is unreliable" —
|
||||
it is independent of the subtag-fallback `degraded` field above. It is not
|
||||
latched: the first fetch that comes back under the cap is complete, restores
|
||||
absence authority, and clears it. Detection remains the record-count heuristic;
|
||||
the reply's `ALARM_RECORDS/@COUNT` attribute would make the test exact only if
|
||||
it reported the total active count rather than the records in the reply, which
|
||||
a live probe could not discriminate (see `docs/AlarmProbeFindings.md`).
|
||||
|
||||
Forced modes are available via `MxGateway:Alarms:Fallback:Mode`:
|
||||
`ForceAlarmManager` disables failover; `ForceSubtag` forces the standby
|
||||
on from startup; `Auto` (default) enables failover and failback. Watch-list
|
||||
|
||||
Reference in New Issue
Block a user