feat(alarms): feed-level snapshot_status truncation frame on StreamAlarms

This commit is contained in:
Joseph Doherty
2026-08-17 07:16:51 -04:00
parent 094f2ffee4
commit fccf75324b
8 changed files with 920 additions and 197 deletions
+18 -1
View File
@@ -203,7 +203,7 @@ Consequences, and how this sits with the existing failover/reconcile design:
separately — see the next decision. A galaxy that truncates persistently
is a configuration problem: raise `MxGateway:Alarms:MaxAlarmsPerFetch`.
### Alarms — truncation is reported per record on the public snapshot stream
### Alarms — truncation is reported per record on the snapshot stream, as a status frame on the live feed
Decision (2026-08-17): the truncated-fetch verdict above is carried to clients as
`QueryActiveAlarmsReplyPayload.snapshot_truncated` on the worker IPC reply and as
@@ -239,6 +239,23 @@ sub-cap fetch clears it, and `GatewayAlarmMonitor.ClearCache` drops it with the
cache generation it describes. A caveat that never turns off is a caveat
operators learn to ignore.
The **live feed carries the verdict as set-level status**, not per record.
`StreamAlarms` has an envelope — `AlarmFeedMessage` — so the shape forced on
`QueryActiveAlarms` above is not forced here: the feed gets a fifth oneof case,
`snapshot_status` (`AlarmSnapshotStatus.truncated`), alongside `provider_status`.
The two carriers are therefore deliberately different shapes for the same verdict,
and each is the only additive option on its own surface. Emission is
**edge-triggered**, for the same reason the flag is not latched: a status frame
repeated on every reconcile is noise a consumer filters out, and a filtered-out
signal is no signal. The exception is the open-time frame, which is
unconditional — a late joiner cannot distinguish "not truncated" from "this
gateway does not send the frame" by silence, so it is told explicitly. It is
ordered after `provider_status` and before the cached `active_alarm` frames so a
consumer applying the snapshot as it streams holds the caveat while it applies
the records it qualifies. A monitor restart's `ClearCache` emits the clearing
frame as well: feed subscribers outlive the monitor's worker session, so a silent
re-seed would leave them caveating a set that is no longer truncated.
This is gateway metadata about **our** fetch mechanics, not a claim about MXAccess
behaviour, so it is not a parity deviation: no event is synthesized and no
MXAccess-observable semantics change.
+31 -1
View File
@@ -94,7 +94,7 @@ An accepted gRPC command payload can still be too large for the worker pipe: the
### `StreamAlarms`
`StreamAlarms` is a server-streaming, **session-less** RPC that attaches to the gateway's central alarm feed. The handler delegates to `IGatewayAlarmService.StreamAsync`. The stream opens with one `AlarmFeedMessage` carrying an `active_alarm` per currently-active alarm (the ConditionRefresh snapshot), then a single `snapshot_complete`, then a `transition` for every subsequent raise / acknowledge / clear. It is served by the always-on `GatewayAlarmMonitor`, which owns a single gateway-managed worker session and fans out to every attached client — clients no longer open a session of their own. `alarm_filter_prefix`, when set, scopes the stream to a sub-tree.
`StreamAlarms` is a server-streaming, **session-less** RPC that attaches to the gateway's central alarm feed. The handler delegates to `IGatewayAlarmService.StreamAsync`. The stream opens with a `provider_status` and a `snapshot_status` `AlarmFeedMessage` (the current provider mode and snapshot-completeness verdict), then one `AlarmFeedMessage` carrying an `active_alarm` per currently-active alarm (the ConditionRefresh snapshot), then a single `snapshot_complete`, then a `transition` for every subsequent raise / acknowledge / clear — interleaved with a further `provider_status` on each failover/failback and a further `snapshot_status` on each change of the truncation verdict. It is served by the always-on `GatewayAlarmMonitor`, which owns a single gateway-managed worker session and fans out to every attached client — clients no longer open a session of their own. `alarm_filter_prefix`, when set, scopes the stream to a sub-tree.
### `QueryActiveAlarms`
@@ -119,6 +119,36 @@ The gateway emits `provider_status` once when a client first subscribes
and again on every failover or failback. A late-joining client therefore
always learns the current provider mode without waiting for the next switch.
#### Snapshot completeness on the alarm feed
`AlarmFeedMessage` has a fifth `payload` case, `snapshot_status`, carrying
an `AlarmSnapshotStatus` message:
```protobuf
message AlarmSnapshotStatus {
bool truncated = 1; // the cached active set may be missing alarms
}
```
It is the feed-level twin of the per-record
`ActiveAlarmSnapshot.from_truncated_snapshot` flag: `truncated` is true while the
monitor's cached active-alarm set derives from a capped worker fetch. Read it as
"this set may be incomplete", never as a statement about record fidelity — that
is what `degraded` / `source_provider` mean, and the two are independent.
Emission mirrors `provider_status` but with one ordering rule of its own. The
gateway emits `snapshot_status` once when a client subscribes, **after** the
open-time `provider_status` and **before** the cached `active_alarm` frames, so a
consumer applying the snapshot as it streams has the completeness caveat in hand
while it applies it. The open-time frame is unconditional — an explicit
`truncated = false` is what distinguishes a complete set from a gateway that
never sends the frame. Afterwards it is emitted only on a *change* of verdict:
when a reconcile flips the verdict either way, and when a monitor restart drops a
truncated verdict with the cache generation it describes (feed subscribers
outlive the monitor's worker session, so they see that clear). Clients that do
not know the case see an unset `payload` oneof and ignore the message, exactly as
before.
`AlarmProviderMode` is an enum with three values:
| Value | Meaning |