feat(alarms): structural degraded-status signal for truncated alarm snapshots
The truncation-cliff fix made alarm transitions truncation-safe but silent:
when GetXmlCurrentAlarms2 returns exactly maxAlmCnt records the worker
suppresses absence-implies-Clear inference and says so only in a rate-limited
stderr warning. No client and no operator could tell a complete active set
from a capped one.
Two additive proto3 booleans carry the verdict out:
- QueryActiveAlarmsReplyPayload.snapshot_truncated = 2 (worker IPC reply)
- ActiveAlarmSnapshot.from_truncated_snapshot = 16 (per record)
The per-record field is not an aesthetic choice. QueryActiveAlarms returns a
bare `stream ActiveAlarmSnapshot` with no envelope, header, or trailer, so a
per-record boolean is the only carrier that stays wire-compatible; an envelope
message would change every existing client's stream element type. The reply
payload states it too because a prefix filter can leave zero records and a
truncated fetch with nothing to report still has to say so. The flag means
"this set may be incomplete", never "this record is unreliable" — it is
independent of the subtag-fallback `degraded` field.
Detection is deliberately UNCHANGED: IsTruncatedFetch remains
`fetchedRecordCount >= maxAlarmsPerFetch`. The live probe (docs/AlarmProbeFindings.md,
ce5d8ae) could not verify whether ALARM_RECORDS/@COUNT reports the total active
count or only the records in the reply, so @COUNT is not parsed for detection;
switching to it stays blocked on probe evidence. The probe's comment
annotations in WnWrapAlarmConsumer.cs are preserved.
Reset semantics: not latched. WnWrapAlarmConsumer.FoldFetch replaces the
verdict on every poll under the same lock as the snapshot merge, so the first
sub-cap fetch clears it; GatewayAlarmMonitor.ClearCache drops it with the cache
generation it describes. A caveat that never turns off is one operators learn
to ignore.
Flow: WnWrapAlarmConsumer.LastSnapshotTruncated -> AlarmDispatcher (stamps every
record) / IAlarmCommandHandler (payload) -> MxAccessCommandExecutor reply ->
GatewayAlarmMonitor._snapshotTruncated -> IGatewayAlarmService.SnapshotTruncated
-> DashboardAlarmQueryResult -> AlarmsPage warning banner (render-side only; the
poll loop and DisposeAsync drain are untouched). The public QueryActiveAlarms
RPC forwards worker snapshots unmodified, so the per-record flag needed no
mapper change — a test pins that.
Parity: this describes OUR fetch mechanics — additive gateway metadata — not
MXAccess provider behavior. No event is synthesized and no MXAccess-observable
semantics change, so it is not a parity deviation.
Tests: worker LastSnapshotTruncated set/reset/consecutive-burst (windev-run);
gateway end-to-end truncated reply -> monitor -> public stream, with the
complete-reply control as the load-bearing assertion; AlarmsPage banner
present/absent. Docs: gateway.md alarm surface, docs/DesignDecisions.md entry.
This commit is contained in:
+42
-4
@@ -199,12 +199,50 @@ Consequences, and how this sits with the existing failover/reconcile design:
|
||||
goes to the worker's console/stderr, which is captured on dev hosts but is
|
||||
not a metric, not a dashboard tile, and not part of any session-status or
|
||||
alarm-feed payload, so a production deployment can truncate indefinitely
|
||||
without anyone noticing. Surfacing truncation as a **structural** degraded
|
||||
status (a field on the alarm-provider mode/status surface the dashboard and
|
||||
`StreamAlarms` consumers already read) is filed as a follow-up; until it
|
||||
lands, the log line is the only signal. A galaxy that truncates persistently
|
||||
without anyone noticing. The structural signal that fixes this landed
|
||||
separately — see the next decision. A galaxy that truncates persistently
|
||||
is a configuration problem: raise `MxGateway:Alarms:MaxAlarmsPerFetch`.
|
||||
|
||||
### Alarms — truncation is reported per record on the public snapshot stream
|
||||
|
||||
Decision (2026-08-17): the truncated-fetch verdict above is carried to clients as
|
||||
`QueryActiveAlarmsReplyPayload.snapshot_truncated` on the worker IPC reply and as
|
||||
`ActiveAlarmSnapshot.from_truncated_snapshot` on **every record** of the public
|
||||
`QueryActiveAlarms` stream, with a matching `IGatewayAlarmService.SnapshotTruncated`
|
||||
driving a dashboard banner. Both fields are additive proto3 booleans.
|
||||
|
||||
A per-record boolean is an odd shape for what is set-level status, so the reason
|
||||
matters: `rpc QueryActiveAlarms(QueryActiveAlarmsRequest) returns (stream
|
||||
ActiveAlarmSnapshot)` returns a *bare* message stream. There is no envelope, no
|
||||
header message, and no trailing summary to hang a set-level field off. Adding one
|
||||
would mean either a new wrapper message (breaking every existing client's stream
|
||||
element type) or a trailing metadata convention (invisible to clients that stop
|
||||
reading early). Stamping the flag identically on each record is the only carrier
|
||||
that is additive on the wire: clients that ignore the field deserialize exactly
|
||||
as before. Consumers should read it as "the set this record belongs to may be
|
||||
incomplete", never as a statement about the record's own fidelity — that is what
|
||||
`degraded` / `source_provider` mean, and the two are independent. The reply
|
||||
payload carries the flag as well because a prefix filter (or an empty galaxy) can
|
||||
leave zero records, and a truncated fetch with nothing to report still has to say
|
||||
so.
|
||||
|
||||
The **detection heuristic is unchanged**: `IsTruncatedFetch` remains
|
||||
`fetchedRecordCount >= maxAlarmsPerFetch`. The live probe run for this work could
|
||||
not verify whether `ALARM_RECORDS/@COUNT` reports the total active count or only
|
||||
the records in the reply (`docs/AlarmProbeFindings.md`), and an exact-looking
|
||||
signal derived from an unverified attribute is worse than an honest heuristic —
|
||||
it would read as precise while being wrong in the one direction that matters.
|
||||
Switching to `@COUNT` stays blocked on probe evidence.
|
||||
|
||||
The flag is **not latched**. It is replaced by each fetch's verdict, so the first
|
||||
sub-cap fetch clears it, and `GatewayAlarmMonitor.ClearCache` drops it with the
|
||||
cache generation it describes. A caveat that never turns off is a caveat
|
||||
operators learn to ignore.
|
||||
|
||||
This is gateway metadata about **our** fetch mechanics, not a claim about MXAccess
|
||||
behaviour, so it is not a parity deviation: no event is synthesized and no
|
||||
MXAccess-observable semantics change.
|
||||
|
||||
## Session-Resilience Epic Scope
|
||||
|
||||
Decision (2026-07-09, archreview TST-04; migrated here 2026-08-07 from the retired
|
||||
|
||||
Reference in New Issue
Block a user