Compare commits

...

33 Commits

Author SHA1 Message Date
Joseph Doherty 3c9becc8d6 docs(plan): mark all alarm-subtag-fallback tasks completed 2026-06-13 10:55:18 -04:00
Joseph Doherty ec88532fe4 alarms: propagate degraded/source_provider through snapshot + gateway cache paths (integration fix I1/I2) 2026-06-13 10:53:55 -04:00
Joseph Doherty 2f30f0c7c0 docs(alarms): document alarmmgr->subtag fallback (providers, failover, config, contract, parity) 2026-06-13 10:43:37 -04:00
Joseph Doherty 27f6c9e6b7 dashboard(alarms): provider-status badge (alarmmgr vs degraded subtag) 2026-06-13 10:37:37 -04:00
Joseph Doherty 29bd504a99 test(alarms): end-to-end provider failover/failback lifecycle through GatewayAlarmMonitor 2026-06-13 10:34:24 -04:00
Joseph Doherty e10b252e3a test(alarms): drop unsupported Assert.Equal message args in live subtag smoke test (xUnit) 2026-06-13 10:30:39 -04:00
Joseph Doherty bcc54ca56b server(alarms): provider-mode gauge startup baseline; reconcile-lock comment; de-flake monitor test 2026-06-13 10:29:13 -04:00
Joseph Doherty ee459f43e1 test(alarms): opt-in live subtag-fallback smoke test (Skip by default)
Adds AlarmSubtagLiveSmokeTests to validate the open design item from Task 17:
confirms that LmxSubtagAlarmSource (real MxAccessComObjectFactory) wired to
SubtagAlarmConsumer synthesizes degraded Raise transitions with stable synthetic
GUIDs from Galaxy alarm subtags, and that AcknowledgeByName writes the
ack-comment subtag (rc=0). PLACEHOLDER_* subtag addresses are best-guess and
must be verified against MXAccess-Public-API.md + live Galaxy before flipping Skip.
2026-06-13 10:26:28 -04:00
Joseph Doherty ebf1d95f72 server(alarms): monitor resolves watch-list, sends ForcedMode/failover, reflects provider mode into feed + metrics 2026-06-13 10:20:03 -04:00
Joseph Doherty 3ccf0b5f9e server(alarms): honor ExcludeAttributes GR-only contract; warn on empty config-only watch-list 2026-06-13 10:12:58 -04:00
Joseph Doherty f7ccfd678e server(alarms): watch-list resolver merging GR discovery + config override 2026-06-13 10:09:10 -04:00
Joseph Doherty 3f5e5fc0b3 worker(alarms): route ForcedMode/watch-list/failover via AlarmCommandHandler; emit provider-mode-changed event 2026-06-13 10:04:33 -04:00
Joseph Doherty 7241a4fb9c worker(alarms): net48 index fix; enforce ProbeIntervalSeconds; OOM-safe catch; reset-on-failure test 2026-06-13 09:55:07 -04:00
Joseph Doherty d6c0bb41ca worker(alarms): failback probe re-polls the still-subscribed primary (no re-Subscribe) 2026-06-13 09:49:38 -04:00
Joseph Doherty 0a54c0bc4b worker(alarms): FailoverAlarmConsumer auto-failover/failback state machine 2026-06-13 09:46:47 -04:00
Joseph Doherty fd64b9260c worker(alarms): exact-match ack resolution (no substring false-match) + ack-by-guid tests 2026-06-13 09:42:00 -04:00
Joseph Doherty 4bd757a136 worker(alarms): SubtagAlarmConsumer synthesizing degraded transitions; dispatcher propagates Degraded 2026-06-13 09:35:49 -04:00
Joseph Doherty 1e2ed6d1ea worker(alarms): WriteRecord as class not positional record (net48 has no IsExternalInit) 2026-06-13 09:30:52 -04:00
Joseph Doherty 5f6655de27 server(alarms): drop redundant null-coalesce; tidy validator tests (review fixes) 2026-06-13 09:27:37 -04:00
Joseph Doherty fbc9cf56df worker(alarms): SyntheticAlarmGuid internal + alarmmgr-parity assertion (review fixes) 2026-06-13 09:26:52 -04:00
Joseph Doherty 4c0e14fc5d worker(alarms): COM-backed LmxSubtagAlarmSource advising alarm subtags 2026-06-13 09:24:09 -04:00
Joseph Doherty c75920c620 docs(plan): correct alarm proto location to mxaccess_gateway.proto (Tasks 1-2) 2026-06-13 09:18:11 -04:00
Joseph Doherty a46ce90e6f server(metrics): alarm provider mode gauge + provider switch counter (Task 13) 2026-06-13 09:18:11 -04:00
Joseph Doherty f113ca53a1 server(galaxy): GetAlarmAttributesAsync discovery query + alarm-attribute row mapping (Task 11) 2026-06-13 09:18:11 -04:00
Joseph Doherty f3616cc7fa server(alarms): AlarmFallbackOptions + ForceSubtag/threshold validation (Task 10) 2026-06-13 09:18:11 -04:00
Joseph Doherty 57d5a8725f worker(alarms): synthetic GUID + degraded/source_provider on emitted transitions 2026-06-13 09:14:23 -04:00
Joseph Doherty 60d35a914f contracts: regenerate Generated/ for alarm provider mode + subtag types
Keeps committed generated C# in sync with the .proto change in 1d85db7
(AlarmProviderMode, AlarmSubtagTarget, AlarmFailoverConfig, AlarmProviderStatus,
OnAlarmProviderModeChangedEvent, degraded/source_provider fields).
2026-06-13 09:10:08 -04:00
Joseph Doherty b10e103bcf worker(alarms): fix net48 build (init->set, usings), token-boundary name parse, acked latch, dup-address guard, tests 2026-06-13 09:05:58 -04:00
Joseph Doherty 348ab16456 worker(alarms): subtag value-source seam + synthesis state machine 2026-06-13 08:57:28 -04:00
Joseph Doherty c16f016f0a test(contracts): round-trip provider status + degraded provenance 2026-06-13 08:56:13 -04:00
Joseph Doherty 1d85db7b4e contracts(gateway): AlarmProviderMode, subtag watch-list, provider status, degraded provenance, mode-changed event 2026-06-13 08:53:02 -04:00
Joseph Doherty 5ea5618315 docs: implementation plan for alarm subtag-monitoring fallback
18 TDD tasks across contracts, worker (SubtagAlarmConsumer + FailoverAlarmConsumer),
gateway (GR-SQL watch-list discovery, monitor mode reflection, metrics, dashboard),
and docs. Grounded in current signatures; parity-preserving (worker-side synthesis).
2026-06-13 08:44:42 -04:00
Joseph Doherty 38a0ad8ab4 docs: design for alarmmgr→subtag alarm-provider fallback
Auto-failover/failback between the wnwrap alarmmgr consumer and a new
worker-side SubtagAlarmConsumer that advises alarm subtags and synthesizes
transitions. GR-SQL+config watch-list discovery, ack via ack-comment write,
degraded state surfaced in the gRPC contract and dashboard/metrics.
2026-06-13 08:35:18 -04:00
60 changed files with 9988 additions and 450 deletions
+124
View File
@@ -790,3 +790,127 @@ Post-ack transition: kind=Clear …
10s cadence held throughout; full proto fields populated correctly;
ack registered server-side without errors.
## Subtag-monitoring fallback provider
When the wnwrap alarm-manager source fails, the gateway worker switches to
`SubtagAlarmConsumer` — a synthetic alarm source that advises each alarm
attribute's subtags via the existing MXAccess `AddItem`/`Advise` pipeline and
derives alarm transitions from the resulting value-change stream. This is a
non-parity, degraded-mode source; every transition and snapshot it produces
carries `degraded = true`.
### Watch-list discovery
`GatewayAlarmMonitor` resolves the subtag watch-list at subscribe time by
calling `IAlarmWatchListResolver.GetAlarmAttributesAsync`. The resolver merges:
1. Galaxy Repository SQL (`GetAlarmAttributesAsync`) — objects that have alarm
extensions in the configured area.
2. Config overrides — `IncludeAttributes` adds explicit entries;
`ExcludeAttributes` removes Repository-derived ones. The config list takes
effect even when `UseGalaxyRepository` is `false`.
The resolved list is a set of `AlarmSubtagTarget` messages sent to the worker
inside `SubscribeAlarmsCommand.watch_list`. Each target carries the composed
MXAccess item addresses for the `.active`, `.acked`, ack-comment, and priority
subtags. The gateway re-runs discovery on its reconcile cadence and pushes an
updated watch-list when the model changes.
### Subtag advise and `LmxSubtagAlarmSource`
`LmxSubtagAlarmSource` (implements `ISubtagAlarmSource`) owns a separate
`LMXProxyServerClass` instance on the worker STA — it does not share the
session's main MXAccess object. For each watch-list target it calls
`AddItem`/`Advise` on the configured subtag addresses. When a subtag value
changes, it raises `ValueChanged` on the STA and `SubtagAlarmConsumer`
forwards it to `SubtagAlarmStateMachine`.
`PollOnce()` on the subtag consumer is a no-op — the path is event-driven
through `Advise`, not poll-driven.
### Synthesis rules
`SubtagAlarmStateMachine` tracks `(active, acked)` per watch-list entry and
emits `MxAlarmTransitionEvent` records on change:
| Subtag change | Emitted transition | Notes |
|---|---|---|
| `active` false → true | Raise (`UNACK_ALM`) | `original_raise_timestamp` = first observed active time for this episode |
| `acked` false → true, while `active` | Acknowledge (`ACK_ALM`) | `AckedDuringEpisode` latch set |
| `active` true → false | Clear | `AckRtn` if `AckedDuringEpisode` is set, else `UnackRtn` |
| `acked` true → false, while `active` | (none) | Latch is NOT cleared; the episode retains its acknowledged status at clear |
The `AckedDuringEpisode` latch addresses out-of-order subtag delivery:
MXAccess does not guarantee the `acked = false` update arrives before the
`active = false` update. The latch ensures a clear always emits `ACK_RTN`
when the alarm was acknowledged at any point during the active episode.
`SnapshotActive()` returns one `MxAlarmSnapshotRecord` per currently-active
alarm. State mapping:
- `active && !acked` → `UNACK_ALM`
- `active && acked` → `ACK_ALM`
- `!active` → not included in the snapshot
### Synthetic GUID
The alarmmgr provider supplies a native GUID per alarm record. The subtag
provider has no native GUID. `SubtagAlarmConsumer` derives a deterministic
GUID by hashing `alarm_full_reference` (via `SyntheticAlarmGuid.ForReference`).
The same reference always produces the same GUID within a session, so
GUID-based ack routing resolves correctly. The GUID is not stable across
different alarm references or gateway restarts in the sense of matching any
AVEVA-internal GUID.
### Acknowledge in subtag mode
`AlarmDispatcher` routes ack calls by active provider mode:
- **Alarm-manager mode:** `AlarmAckByName` on `wwAlarmConsumerClass` (unchanged).
- **Subtag mode:** `SubtagAlarmConsumer.AcknowledgeByName` resolves the
watch-list entry's `ack_comment_subtag` and issues a `Write(comment)` on
the STA via `LmxSubtagAlarmSource`. The write performs the ack in AVEVA.
If the alarm has no writable ack-comment subtag (`AckComment` config key is
empty, or the entry's `ack_comment_subtag` field is empty), the ack call
returns a failure code that the gateway surfaces as `FailedPrecondition`.
`AcknowledgeByGuid` maps the synthetic GUID back to its reference via an
internal dictionary, then calls the same write path.
### Fidelity limitations
The following fields are not available or have lower quality in subtag mode:
| Field | Subtag-mode behavior |
|-------|---------------------|
| `alarm_guid` | Synthetic deterministic GUID from `alarm_full_reference`; not an AVEVA-native GUID |
| `original_raise_timestamp` | First observed `active = true` time; no AVEVA-native raise time |
| `transition_timestamp` | `OnDataChange` source timestamp from MXAccess |
| `severity` | From priority subtag if advised; 0 otherwise |
| `category` / `description` | Not populated (no subtag for these) |
| `current_value` / `limit_value` | Not populated unless corresponding subtags are in the watch-list |
| `alarm_type_name` | Not populated |
| `operator_user` / `operator_comment` | Not populated on synthesized raise/clear transitions |
| `retrigger` transition | Not synthesized (no re-alarm counter subtag is observed) |
Every transition and snapshot record carries `degraded = true` and
`source_provider = ALARM_PROVIDER_MODE_SUBTAG`. Clients that require full
fidelity must wait for failback to the alarm manager.
### Provider mode reflection
When `FailoverAlarmConsumer` switches between providers, it raises
`ProviderModeChanged`. `AlarmDispatcher` enqueues an
`OnAlarmProviderModeChangedEvent` (carried as an `MxEvent`), which the
gateway receives and reflects into:
- `AlarmFeedMessage.provider_status` emitted to every `StreamAlarms`
subscriber.
- The `/hubs/alarms` SignalR hub for the dashboard.
- Metrics: `mxgateway.alarms.provider_mode` gauge and
`mxgateway.alarms.provider_switches` counter.
On every switch `GatewayAlarmMonitor` also forces a reconcile
(`QueryActiveAlarms`) against the now-active provider so the gateway cache
reflects the post-switch state without a spurious raise/clear storm.
+52
View File
@@ -411,6 +411,58 @@ a per-channel skip-verify hook:
See [Gateway Configuration — Automatic self-signed certificate](./GatewayConfiguration.md#automatic-self-signed-certificate)
and the per-client READMEs for the as-built behavior.
## Alarm-Manager to Subtag Fallback
Decision: add a second alarm provider (subtag monitoring) that the worker
activates automatically when the native wnwrap alarm manager fails, and fails
back to automatically when the manager recovers.
### Worker-side synthesis
Synthesis of alarm transitions from subtag value changes happens entirely in
the worker (`SubtagAlarmConsumer` / `SubtagAlarmStateMachine`). The gateway
still forwards only events the worker emits and synthesizes nothing itself.
This satisfies the parity rule even though the subtag path is inherently
non-parity: the parity rule governs where synthesis lives, not whether
synthesis is permitted when the native source is unavailable.
### Degraded is explicit
Every subtag-mode transition carries `degraded = true` on the
`OnAlarmTransitionEvent` and `ActiveAlarmSnapshot` proto messages, and the
`AlarmFeedMessage` feed carries an `AlarmProviderStatus` payload on stream
open and on every switch. No client can mistake a subtag-mode alarm for an
authoritative alarmmgr record. Subtag mode has lower fidelity: synthetic
deterministic GUID (SHA-derived from the alarm reference), best-effort
original-raise timestamp, narrower field set. Clients that need full fidelity
must wait for failback.
### Failover trigger
The failover trigger is N consecutive wnwrap COM failures — a `COMException`
thrown by `Subscribe` or `PollOnce`, or a failure HRESULT from
`GetXmlCurrentAlarms2`. A single poll failure does not trigger a switch; the
threshold (default 3, floored at 1) guards against transient COM hiccups. The
counter resets on any clean poll so a flapping provider does not permanently
latch in subtag mode.
### Acknowledge via ack-comment write
In subtag mode, `AcknowledgeAlarm` writes the operator comment to the alarm
attribute's ack-comment subtag (`Fallback:Subtags:AckComment`). The write
performs the native ack in AVEVA. This differs from alarmmgr mode, where
`AlarmAckByName` on `wwAlarmConsumerClass` is called directly. The `AckComment`
subtag name is empty by default; configuring it is required for ack to work in
subtag mode. The exact AVEVA subtag names are not hard-coded — the `Subtags`
config block exists precisely so names are not guessed without validation
against the live MXAccess attribute set.
### Related documentation
- [Gateway Configuration — Alarm Fallback options](./GatewayConfiguration.md#alarm-fallback-options)
- [Alarm Client Discovery — Subtag provider](./AlarmClientDiscovery.md)
- [gRPC Contract — provider_status and degraded fields](./Grpc.md)
## Later Revisit Items
These are explicit post-v1 revisit items, not open blockers:
+68
View File
@@ -230,6 +230,74 @@ behavior.
The alarm monitor is independent of client sessions: `AcknowledgeAlarm` and
`StreamAlarms` are session-less RPCs served by the monitor.
### Alarm fallback options
The `Fallback` sub-section controls how the alarm feed selects between the
native wnwrap alarm-manager provider and the subtag-monitoring fallback.
| Option | Default | Description |
|--------|---------|-------------|
| `MxGateway:Alarms:Fallback:Mode` | `Auto` | Provider selection mode. `Auto` uses the alarm manager as primary and fails over to subtag monitoring after consecutive COM failures, then fails back automatically. `ForceAlarmManager` disables failover. `ForceSubtag` forces subtag monitoring on from startup. Values are case-insensitive. |
| `MxGateway:Alarms:Fallback:ConsecutiveFailureThreshold` | `3` | Number of consecutive wnwrap COM failures (`COMException` or failure HRESULT from `Subscribe` / `GetXmlCurrentAlarms2`) before the monitor switches to subtag mode. Floored at 1. |
| `MxGateway:Alarms:Fallback:FailbackProbeIntervalSeconds` | `30` | While in subtag mode, how often (in seconds) the monitor probes the wnwrap provider to detect recovery. Floored at 1. |
| `MxGateway:Alarms:Fallback:FailbackStableProbes` | `3` | Number of consecutive clean wnwrap probes required before the monitor switches back to the alarm manager. Floored at 1. |
| `MxGateway:Alarms:Fallback:Discovery:UseGalaxyRepository` | `true` | When `true`, the monitor queries the Galaxy Repository SQL database to build the subtag watch-list for the configured area. |
| `MxGateway:Alarms:Fallback:Discovery:Area` | _(empty)_ | Galaxy area to scope the Repository query to. Falls back to `MxGateway:Alarms:DefaultArea` when empty. Ignored when `UseGalaxyRepository` is `false`. |
| `MxGateway:Alarms:Fallback:Discovery:IncludeAttributes` | _(empty)_ | Explicit MXAccess attribute paths to add to the subtag watch-list, supplementing (or replacing, when `UseGalaxyRepository` is `false`) the Repository-derived list. |
| `MxGateway:Alarms:Fallback:Discovery:ExcludeAttributes` | _(empty)_ | Attribute paths to remove from the Repository-derived watch-list. Ignored when `UseGalaxyRepository` is `false`. |
| `MxGateway:Alarms:Fallback:Subtags:Active` | `active` | Subtag name for the in-alarm boolean. |
| `MxGateway:Alarms:Fallback:Subtags:Acked` | `acked` | Subtag name for the acknowledged boolean. |
| `MxGateway:Alarms:Fallback:Subtags:AckComment` | _(empty)_ | Subtag name for the acknowledgement comment attribute. When empty, writing an ack comment in subtag mode is disabled. Must be verified against the live MXAccess attribute set before use. |
| `MxGateway:Alarms:Fallback:Subtags:Priority` | `priority` | Subtag name for the alarm priority / severity value. |
Validation rules:
- `Mode` must be `Auto`, `ForceAlarmManager`, or `ForceSubtag` (case-insensitive).
- `Mode = ForceSubtag` with both `UseGalaxyRepository = false` and an empty
`IncludeAttributes` list produces a startup validation warning: the subtag
provider has no attributes to advise.
- `ConsecutiveFailureThreshold`, `FailbackProbeIntervalSeconds`, and
`FailbackStableProbes` are floored at 1 by `GatewayOptionsValidator`.
Full example with non-default fallback settings:
```json
{
"MxGateway": {
"Alarms": {
"Enabled": true,
"SubscriptionExpression": "\\\\SCADA01\\Galaxy!PlantArea",
"DefaultArea": "PlantArea",
"ReconcileIntervalSeconds": 30,
"Fallback": {
"Mode": "Auto",
"ConsecutiveFailureThreshold": 3,
"FailbackProbeIntervalSeconds": 30,
"FailbackStableProbes": 3,
"Discovery": {
"UseGalaxyRepository": true,
"Area": "",
"IncludeAttributes": [],
"ExcludeAttributes": []
},
"Subtags": {
"Active": "active",
"Acked": "acked",
"AckComment": "",
"Priority": "priority"
}
}
}
}
}
```
The exact AVEVA subtag names for `Active`, `Acked`, `AckComment`, and
`Priority` are not hard-coded. The `Subtags` block exists so names can be
confirmed against the live MXAccess attribute set and configured without a
code change. See `docs/AlarmClientDiscovery.md` for the synthesis rules that
depend on these names.
## Host Endpoints and Transport Security (Kestrel)
The listening endpoints are **not** part of the `MxGateway` section. The gateway
+67
View File
@@ -94,6 +94,73 @@ Carrying the enqueue timestamp into the worker layer is what lets queue-wait tim
`StreamAlarms` is a server-streaming, **session-less** RPC that attaches to the gateway's central alarm feed. The handler delegates to `IGatewayAlarmService.StreamAsync`. The stream opens with one `AlarmFeedMessage` carrying an `active_alarm` per currently-active alarm (the ConditionRefresh snapshot), then a single `snapshot_complete`, then a `transition` for every subsequent raise / acknowledge / clear. It is served by the always-on `GatewayAlarmMonitor`, which owns a single gateway-managed worker session and fans out to every attached client — clients no longer open a session of their own. `alarm_filter_prefix`, when set, scopes the stream to a sub-tree.
#### Provider status on the alarm feed
`AlarmFeedMessage` has a fourth `payload` case, `provider_status`, carrying
an `AlarmProviderStatus` message:
```protobuf
message AlarmProviderStatus {
AlarmProviderMode mode = 1;
bool degraded = 2; // true whenever mode == SUBTAG
string reason = 3; // human-readable switch reason
google.protobuf.Timestamp since = 4;
}
```
The gateway emits `provider_status` once when a client first subscribes
(immediately after the initial snapshot and before the first live transition)
and again on every failover or failback. A late-joining client therefore
always learns the current provider mode without waiting for the next switch.
`AlarmProviderMode` is an enum with three values:
| Value | Meaning |
|-------|---------|
| `ALARM_PROVIDER_MODE_UNSPECIFIED` (0) | Default / unset |
| `ALARM_PROVIDER_MODE_ALARMMGR` (1) | Native wnwrap alarm-manager source |
| `ALARM_PROVIDER_MODE_SUBTAG` (2) | Subtag-monitoring fallback (degraded) |
#### Degraded and source-provider fields on transitions and snapshots
`OnAlarmTransitionEvent` and `ActiveAlarmSnapshot` both carry two new fields:
- `bool degraded` (field 14) — `true` when the record came from the subtag
fallback, not the native alarmmgr.
- `AlarmProviderMode source_provider` (field 15) — which provider produced
this record (`ALARMMGR` or `SUBTAG`).
Both fields are proto3 defaults (`false` / `UNSPECIFIED`) in alarmmgr mode,
so existing clients that do not read them continue to function without change.
Clients that care about provenance — for example, an OPC UA server that
applies different quality flags to degraded alarms — should inspect `degraded`
before consuming the transition.
Subtag-mode records are a non-parity source. They carry synthetic GUIDs,
best-effort timestamps, and reduced field coverage. See
`docs/AlarmClientDiscovery.md` for the full fidelity table.
#### Provider-mode-changed event
The worker emits `OnAlarmProviderModeChangedEvent` (family
`MX_EVENT_FAMILY_ON_ALARM_PROVIDER_MODE_CHANGED`) on each switch between
providers:
```protobuf
message OnAlarmProviderModeChangedEvent {
AlarmProviderMode mode = 1;
string reason = 2;
int32 hresult = 3; // COM HRESULT that triggered failover; 0 on failback
google.protobuf.Timestamp at = 4;
}
```
This event arrives on the `StreamEvents` stream of the alarm monitor's
internal gateway session (not on client sessions). `GatewayAlarmMonitor`
consumes it and reflects the new mode into the `StreamAlarms` feed's
`provider_status`, the dashboard hub, and metrics. Client sessions do not
receive this event directly.
## Validation Rules
`MxAccessGrpcRequestValidator` rejects requests with `StatusCode.InvalidArgument` before any session work happens. The rules are intentionally narrow — anything that requires session state (for example, "session does not exist") is left for `ISessionManager` so the validator can stay synchronous and side-effect free.
@@ -0,0 +1,302 @@
# Alarm Subtag-Monitoring Fallback — Design
**Date:** 2026-06-13
**Status:** Approved (brainstorming), ready for implementation planning
**Branch:** `feat/alarm-subtag-fallback`
## Problem
The gateway's central alarm feed (`GatewayAlarmMonitor` → worker
`WnWrapAlarmConsumer`) depends on the AVEVA wnwrap COM consumer
(`WNWRAPCONSUMERLib.wwAlarmConsumerClass`), which polls `GetXmlCurrentAlarms2`
on the worker STA. That provider can fail at the COM boundary (the older
`aaAlarmManagedClient` crashed on FILETIME marshaling; wnwrap can still return
failure HRESULTs or throw `COMException`). When it does, the gateway loses all
alarm visibility.
This design adds a **second alarm source** — direct monitoring of each alarm
attribute's subtags (`.active`, `.acked`, …) via the existing MXAccess
`AddItem`/`Advise` pipeline — and **fails over to it automatically when the
wnwrap provider breaks, then fails back automatically when it recovers**. The
subtag source can also be forced on by config.
## Decisions (locked during brainstorming)
| Decision | Choice |
|---|---|
| Failover model | **Auto-failover + auto-failback** (both directions, runtime) |
| Watch-list source | **Galaxy Repository SQL discovery + config override** |
| Acknowledge in subtag mode | **Write the operator comment to the alarm's ack-comment subtag** (the write performs the ack) |
| Failure signal | **N consecutive wnwrap COM failures** (Subscribe / `GetXmlCurrentAlarms2` throws or returns a failure HRESULT) |
| Degraded-state visibility | **Both** — explicit field in the gRPC contract **and** dashboard + metrics |
| Synthesis location | **Worker-side** (`Approach A`) — keeps the parity rule "the gateway forwards only events the worker emits; it never synthesizes events" |
## Core principle
Subtag monitoring is, by definition, a **non-parity, lower-fidelity** alarm
source: it synthesizes alarm transitions from raw data changes, has no native
alarm GUID, no native original-raise timestamp, and a narrower field set. Per
`CLAUDE.md`, synthesizing events is allowed only as an explicit opt-in
non-parity mode. This design satisfies that by (a) doing the synthesis **inside
the worker** (so the gateway still only forwards worker-emitted events) and
(b) marking every degraded event and the whole feed as degraded so no client
mistakes it for the authoritative alarmmgr feed.
## Architecture
```
GATEWAY (.NET 10, x64)
┌─────────────────────────────────────────────────────────────────┐
│ GatewayAlarmMonitor (BackgroundService) │
│ • resolves watch-list: Galaxy Repository SQL + config override │
│ • arms the worker with the watch-list at subscribe time │
│ • consumes AlarmProviderModeChanged → reflects mode into feed, │
│ /hubs/alarms dashboard hub, and metrics │
│ • forces a cache reconcile (QueryActiveAlarms) on every switch │
└───────────────────────────────┬───────────────────────────────────┘
│ IPC (WorkerEnvelope frames)
│ · SubscribeAlarms{ watch_list, failover cfg }
│ · AlarmProviderModeChanged{ mode, reason, hresult }
│ · OnAlarmTransitionEvent (degraded flag set in subtag mode)
▼
WORKER (.NET FW 4.8, x86, STA)
┌─────────────────────────────────────────────────────────────────┐
│ AlarmDispatcher → FailoverAlarmConsumer : IMxAccessAlarmConsumer │
│ ├─ primary : WnWrapAlarmConsumer (wnwrap COM poll, unchanged) │
│ └─ standby : SubtagAlarmConsumer (AddItem/Advise on subtags) │
│ │
│ FailoverAlarmConsumer owns the state machine: │
│ PrimaryActive ──(N consecutive wnwrap COM failures)──▶ Degraded │
│ Degraded ──(M consecutive clean wnwrap probe polls)──▶ Primary │
│ on each switch: snapshot the now-active provider, hand off │
└─────────────────────────────────────────────────────────────────┘
```
The failover state machine lives **worker-local** so the switch is instant — no
IPC round-trip at the moment alarmmgr dies. The gateway *arms* the standby
consumer up front (passes the watch-list at subscribe time) so it is ready
before it is ever needed.
## Components
### Worker (`src/ZB.MOM.WW.MxGateway.Worker/MxAccess/`)
**`SubtagAlarmConsumer : IMxAccessAlarmConsumer` (new)** — the standby provider.
- On `Subscribe`, instead of wnwrap registration it `AddItem`/`Advise`s the
configured subtags for each watch-list entry on the existing STA (reuses the
worker's item-subscription machinery). Per attribute it advises at minimum
`.active` and `.acked`; optionally `.priority`/severity, `.descr`, value/limit
if present.
- Converts each `OnDataChange` into the same `MxAlarmTransitionEvent` the wnwrap
consumer emits, via the synthesis rules below, and raises
`AlarmTransitionEmitted`. Marks each as **degraded**.
- `SnapshotActiveAlarms()` returns the currently-active set computed from
last-known subtag values.
- `AcknowledgeByName(...)` resolves the watch-list entry's ack-comment subtag and
issues a `Write(comment)` on the STA. `AcknowledgeByGuid(...)` maps the
synthetic GUID (see below) back to a reference, then does the same. If the
attribute exposes no writable ack-comment subtag, returns a failure code that
the gateway surfaces as `FailedPrecondition`.
- `PollOnce()` is a no-op (subtag mode is event-driven via Advise).
**`FailoverAlarmConsumer : IMxAccessAlarmConsumer` (new)** — composite + state
machine. Owns the wnwrap consumer (primary) and the subtag consumer (standby),
forwards `AlarmTransitionEmitted` from whichever child is active, and raises a
new `ProviderModeChanged` event on every switch.
- **Failure counting:** wraps `Subscribe`/`PollOnce` on the primary; a thrown
`COMException` or a failure HRESULT increments a consecutive-failure counter,
reset to zero on any clean poll.
- **Failover** (`PrimaryActive → Degraded`): at `ConsecutiveFailureThreshold`
(default 3), ensures the standby is subscribed (it was armed at startup), sets
active = standby, snapshots the standby's active set for hand-off, and emits
`ProviderModeChanged(SUBTAG, reason, hresult)`.
- **Failback probe** (`Degraded → PrimaryActive`): while degraded, every
`FailbackProbeIntervalSeconds` (default 30) it re-attempts wnwrap
`Subscribe`+`PollOnce` on the STA. After `FailbackStableProbes` (default 3)
consecutive clean polls it switches active = primary, returns the standby to
standby, and emits `ProviderModeChanged(ALARMMGR, "recovered")`.
- **Hand-off:** on every switch it takes `SnapshotActiveAlarms()` from the
now-active provider so the gateway can reconcile and avoid spurious
raise/clear storms.
**`AlarmDispatcher` / `MxAccessAlarmEventSink` / `AlarmCommandHandler`
(changed, minimal)** — `AlarmDispatcher` holds a `FailoverAlarmConsumer` instead
of a bare `WnWrapAlarmConsumer`; it subscribes to `ProviderModeChanged` and
enqueues a mode-changed worker event. The ack path routes by active mode (native
wnwrap ack in alarmmgr mode; ack-comment write in subtag mode), but that routing
is entirely inside the consumer — the dispatcher just calls
`AcknowledgeByName`/`AcknowledgeByGuid`.
### Gateway (`src/ZB.MOM.WW.MxGateway.Server/`)
**Galaxy Repository discovery (new query)** — alongside the existing GR SQL
browse RPCs, a query "attributes that have alarms configured, with their
ack-comment subtag and area", scoped to the configured area. Merged with the
config override (explicit includes/excludes). Produces the watch-list of
`AlarmSubtagTarget`s.
**`GatewayAlarmMonitor` (changed)** — resolves the watch-list at subscribe time
and passes it to the worker; consumes `AlarmProviderModeChanged` and reflects
the current provider mode into (a) the `AlarmFeedMessage` provider-status,
(b) the `/hubs/alarms` dashboard hub, and (c) metrics; forces a reconcile
(`QueryActiveAlarms`) on every switch. Re-runs discovery on its existing
reconcile cadence and pushes an updated watch-list when the model changes.
**`AlarmsOptions` (extended)** — new `Fallback` sub-section (below).
### Contract (`src/ZB.MOM.WW.MxGateway.Contracts/Protos/`)
**`mxaccess_gateway.proto`:**
- `enum AlarmProviderMode { ALARM_PROVIDER_MODE_UNSPECIFIED = 0; ALARMMGR = 1; SUBTAG = 2; }`
- New `AlarmFeedMessage` oneof case `AlarmProviderStatus provider_status`,
carrying `{ AlarmProviderMode mode; bool degraded; string reason;
google.protobuf.Timestamp since; }`. Emitted on stream open and on every
change so a late-joining client immediately learns the mode.
- Add `bool degraded` + `AlarmProviderMode source_provider` to
`OnAlarmTransitionEvent` **and** `ActiveAlarmSnapshot`, so per-item provenance
is visible even mid-stream. All additions are new field numbers — backward
compatible; existing clients ignore them and keep seeing alarms.
**`mxaccess_worker.proto`:**
- Extend the alarm-subscribe command with: `AlarmProviderMode forced_mode`
(`UNSPECIFIED` = auto), `int32 consecutive_failure_threshold`,
`int32 failback_probe_interval_seconds`, `int32 failback_stable_probes`, and
`repeated AlarmSubtagTarget watch_list`, where `AlarmSubtagTarget =
{ string alarm_full_reference; string source_object_reference;
string active_subtag; string acked_subtag; string ack_comment_subtag;
string priority_subtag; }`.
- New worker→gateway event `AlarmProviderModeChanged { AlarmProviderMode mode;
string reason; int32 hresult; google.protobuf.Timestamp at; }`.
> Generated code under `Generated/` and `clients/*/generated*/` is rebuilt from
> these `.proto` files — never hand-edited. Every generated client touched by
> the contract is rebuilt per the source-update workflow.
## Data flow
### Subtag synthesis rules
`SubtagAlarmConsumer` keeps last-known `(active, acked)` per watch-list entry and
emits transitions on change:
| Subtag change | Emitted transition | Notes |
|---|---|---|
| `active` false → true | `RAISE` (state `UNACK_ALM`) | `original_raise_timestamp` = first-observed active time |
| `acked` false → true while `active` | `ACKNOWLEDGE` | `operator_user`/`operator_comment` from ack-comment subtag if advised |
| `active` true → false | `CLEAR` | maps to `AckRtn` if acked at clear, else `UnackRtn` |
| `active` stays true, re-alarm | `RETRIGGER` | **only** if a re-alarm counter subtag exists; otherwise not synthesized (documented limitation) |
Snapshot state mapping for `ActiveAlarmSnapshot.current_state`:
`active && !acked → ACTIVE`, `active && acked → ACTIVE_ACKED`,
`!active → INACTIVE`.
Field degradation in subtag mode:
- `alarm_full_reference` — from the watch-list entry (stable, drives ack-by-ref).
- Synthetic, deterministic GUID derived by hashing `alarm_full_reference` so
GUID-based ack still resolves; flagged `degraded = true`.
- `severity` — from the priority subtag if advised, else 0.
- `original_raise_timestamp` — first-observed active time (best effort).
- `transition_timestamp` — the `OnDataChange` timestamp.
- `category`/`description`/`current_value`/`limit_value` — populated only if the
corresponding subtag is advised; otherwise empty.
### Acknowledge
`AcknowledgeAlarm`/`AcknowledgeAlarmByName` are unchanged at the RPC surface.
`AlarmDispatcher` routes by active provider mode:
- **alarmmgr mode:** native wnwrap `AlarmAckByName`/`AlarmAckByGUID` (unchanged).
- **subtag mode:** resolve the target's `ack_comment_subtag`, `Write` the
operator comment via the existing worker write path on the STA. No writable
ack-comment subtag → `FailedPrecondition`.
### Provider-mode reflection
Worker `AlarmProviderModeChanged` → `GatewayAlarmMonitor` → (a) emit/refresh
`AlarmFeedMessage.provider_status` to every `StreamAlarms` subscriber, (b) push
to `/hubs/alarms`, (c) update metrics, (d) force a reconcile.
## Error handling
- **Both providers down** (subtag advise also failing): the monitor stays
faulted and keeps retrying both; acknowledge returns `Unavailable`. No silent
data loss — the feed reports degraded with reason.
- **Empty watch-list in subtag mode** (GR SQL unavailable, no config override):
log + metric `alarm_fallback_watchlist_empty`; the feed reports degraded +
empty; the gateway keeps re-running discovery on its reconcile cadence and
pushes an updated watch-list when one becomes available.
- **Switch hand-off:** every switch snapshots the now-active provider and
reconciles against the gateway cache to avoid a raise/clear storm.
- **STA affinity:** all subtag advise/write and wnwrap probe calls run on the
worker STA (reuse the existing affinity guard) to satisfy
`ThreadingModel=Apartment`.
### Metrics
- `mxgateway_alarm_provider_mode` (gauge: 1 = alarmmgr, 2 = subtag)
- `mxgateway_alarm_provider_switch_total{from,to,reason}` (counter)
- `mxgateway_alarm_fallback_watchlist_size` (gauge)
## Configuration
```jsonc
"MxGateway": {
"Alarms": {
"Enabled": true,
"SubscriptionExpression": "\\\\DESKTOP-6JL3KKO\\Galaxy!DEV",
"DefaultArea": "DEV",
"ReconcileIntervalSeconds": 30,
"Fallback": {
"Mode": "Auto", // Auto | ForceAlarmManager | ForceSubtag
"ConsecutiveFailureThreshold": 3,
"FailbackProbeIntervalSeconds": 30,
"FailbackStableProbes": 3,
"Discovery": {
"UseGalaxyRepository": true,
"Area": "", // defaults to Alarms.DefaultArea
"IncludeAttributes": [], // explicit additions
"ExcludeAttributes": []
},
"Subtags": {
"Active": "active",
"Acked": "acked",
"AckComment": "", // verified against MXAccess analysis
"Priority": "priority"
}
}
}
}
```
`GatewayOptionsValidator` additions: `Mode = ForceSubtag` with empty discovery
result and no explicit `IncludeAttributes` → startup validation warning;
threshold/interval/probe values floored at sane minimums.
## Open item to confirm during implementation
The exact AVEVA subtag names (`.active`, `.acked`, the ack-comment attribute,
priority) must be confirmed against the MXAccess analysis project
(`C:\Users\dohertj2\Desktop\mxaccess`, `docs/MXAccess-Public-API.md`) and the
live Galaxy before wiring `SubtagAlarmConsumer`. The config `Subtags` block
exists precisely so the resolved names are not hard-coded.
## Testing
| Layer | Tests |
|---|---|
| Worker unit (`MxGateway.Worker.Tests`, x86) | `SubtagAlarmConsumer` synthesis — feed `OnDataChange` sequences, assert raise/ack/clear transitions, snapshot states, degraded flag, synthetic-GUID stability, ack-comment write routing |
| Worker unit | `FailoverAlarmConsumer` state machine — fake wnwrap throwing after K polls: assert switch at threshold, failback after stable probes, `ProviderModeChanged` emitted, no duplicate transitions across switch (hand-off reconcile) |
| Gateway unit (`MxGateway.Tests`, fake worker) | discovery + config-override merge; `GatewayAlarmMonitor` reflects mode into feed + hub; metrics increment on switch |
| Contract | proto round-trip for new fields; existing alarm tests unchanged (alarmmgr-mode regression — parity preserved) |
| Live (opt-in, `MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`) | real subtag advise + ack-comment write against a live alarm; GR SQL discovery query against the `ZB` DB (gated like existing GR tests) |
## Docs to update in the same change
`gateway.md` (alarm provider section), `docs/DesignDecisions.md` (record the
fallback decision), `docs/GatewayConfiguration.md` (the `Fallback` block),
`docs/AlarmClientDiscovery.md` (subtag provider + synthesis rules),
`docs/Grpc.md` (the new `provider_status` / `degraded` fields), and any client
READMEs whose generated alarm types gain fields.
@@ -0,0 +1,860 @@
# Alarm Subtag-Monitoring Fallback — Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans (or subagent-driven-development) to implement this plan task-by-task.
**Goal:** Add a second alarm source — direct MXAccess subtag monitoring — that the gateway auto-fails-over to when the wnwrap alarmmgr provider breaks, auto-fails-back to when it recovers, and can be forced on by config.
**Architecture:** Worker-side synthesis (parity rule preserved). A new `SubtagAlarmConsumer` (own `LMXProxyServerClass`, `AddItem`/`Advise` on alarm subtags) and a `FailoverAlarmConsumer` composite (state machine over the wnwrap primary + subtag standby) both implement the existing `IMxAccessAlarmConsumer` seam. The gateway resolves the subtag watch-list (Galaxy Repository SQL + config override), arms the worker at subscribe time, and reflects the live provider mode into the gRPC alarm feed, the dashboard hub, and metrics.
**Tech Stack:** .NET 10 (gateway, x64) + .NET Framework 4.8 (worker, x86, STA), protobuf/gRPC, `Microsoft.Data.SqlClient` (Galaxy Repository), SignalR (dashboard), `System.Diagnostics.Metrics`, xUnit (plain `Assert`, no FluentAssertions).
**Design source:** `docs/plans/2026-06-13-alarm-subtag-fallback-design.md`
**Branch:** `feat/alarm-subtag-fallback` (already created)
---
## Conventions for every task
- **TDD:** write the failing test, run it red, implement, run it green, commit.
- **xUnit, plain `Assert.*`**, naming `Subject_Condition_Expected`. Worker fakes are sealed private nested classes that raise events.
- **Build/test commands:**
- Contracts regen: `dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj`
- Gateway: `dotnet build src/ZB.MOM.WW.MxGateway.Server` ; `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj`
- Worker (x86): `dotnet build src/ZB.MOM.WW.MxGateway.Worker/ZB.MOM.WW.MxGateway.Worker.csproj -p:Platform=x86` ; `dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86`
- Single test: append `--filter FullyQualifiedName~<ClassOrMethod>`
- **Build is strict:** `TreatWarningsAsErrors=true`, nullable enabled. Add XML doc comments on public members (the repo runs a doc checker).
- **Generated code** under `Generated/` is never hand-edited — rebuild the contracts project to regenerate.
- **Namespaces:** worker MxAccess types live in `ZB.MOM.WW.MxGateway.Worker.MxAccess`; proto C# types in `ZB.MOM.WW.MxGateway.Contracts.Proto`.
---
## Phase 0 — Contracts
### Task 1: Worker proto — subtag watch-list, failover config, provider-mode enum
**Classification:** high-risk
**Estimated implement time:** ~4 min
**Parallelizable with:** none (Task 2 imports these types)
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto` (real `SubscribeAlarmsCommand` at ~line 324; `MxCommand` references it at 123-125)
> **CORRECTION (execution):** The alarm command messages and `MxCommand` live in **`mxaccess_gateway.proto`**, not the worker proto. `mxaccess_worker.proto` *imports* the gateway proto (`WorkerCommand.command` is `mxaccess_gateway.v1.MxCommand`), so the gateway proto is the base and the worker proto needs **no** change. `AlarmProviderMode` and the new types are added to the gateway proto and are visible to worker code as `mxaccess_gateway.v1` types. Tasks 1 and 2 are executed as a single combined edit on this one file.
**Step 1: Add the enum and messages.** In `mxaccess_gateway.proto`, extend the existing `SubscribeAlarmsCommand` message (line 324) and add the new types after it:
```protobuf
// Provider selection / current provider for the alarm feed. Defined here in
// the worker contract because the worker SubscribeAlarmsCommand references it;
// mxaccess_gateway.proto imports this file and reuses the same enum.
enum AlarmProviderMode {
ALARM_PROVIDER_MODE_UNSPECIFIED = 0; // auto: alarmmgr primary, subtag fallback
ALARM_PROVIDER_MODE_ALARMMGR = 1;
ALARM_PROVIDER_MODE_SUBTAG = 2;
}
message SubscribeAlarmsCommand {
string subscription_expression = 1; // existing field — keep
// UNSPECIFIED = auto-failover/failback. ALARMMGR/SUBTAG force one provider.
AlarmProviderMode forced_mode = 2;
// Subtag watch-list resolved by the gateway (GR SQL + config). Empty in pure
// alarmmgr mode; in subtag mode it bounds what the consumer can observe.
repeated AlarmSubtagTarget watch_list = 3;
AlarmFailoverConfig failover = 4;
}
// One alarm attribute the subtag consumer advises. Addresses are full MXAccess
// item references the worker passes straight to AddItem.
message AlarmSubtagTarget {
string alarm_full_reference = 1; // e.g. "Galaxy!Area.Tank01.Level.HiHi"
string source_object_reference = 2; // e.g. "Tank01"
string active_subtag = 3; // item address of the in-alarm boolean
string acked_subtag = 4; // item address of the acknowledged boolean
string ack_comment_subtag = 5; // writable ack-comment attribute (ack write target)
string priority_subtag = 6; // optional severity source; empty if absent
}
message AlarmFailoverConfig {
int32 consecutive_failure_threshold = 1; // wnwrap COM failures before switching (>=1)
int32 failback_probe_interval_seconds = 2; // probe cadence while degraded (>=1)
int32 failback_stable_probes = 3; // clean probes before switching back (>=1)
}
```
`UnsubscribeAlarmsCommand` and `AcknowledgeAlarmCommand` are unchanged.
**Step 2: Regenerate & verify it compiles.**
Run: `dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj`
Expected: build succeeds; generated `AlarmProviderMode`, `AlarmSubtagTarget`, `AlarmFailoverConfig` types appear.
**Step 3: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_worker.proto
git commit -m "contracts(worker): subtag watch-list + failover config + AlarmProviderMode"
```
---
### Task 2: Gateway proto — provider status on the feed, degraded provenance, mode-changed event
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none (depends on Task 1; Task 3 tests both)
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto` (`OnAlarmTransitionEvent` ~719-771, `ActiveAlarmSnapshot` ~783-803, `AlarmFeedMessage` ~860-870, `MxEvent` family enum + body oneof, `MxEventFamily` enum)
**Step 1: Add degraded provenance to the two alarm payloads.** Append to `OnAlarmTransitionEvent` (next free field 14):
```protobuf
// True when this transition came from the subtag-monitoring fallback rather
// than the native alarmmgr provider — i.e. it was synthesized from data
// changes and carries reduced fidelity (synthetic GUID, no native raise time).
bool degraded = 14;
// Which provider produced this transition.
AlarmProviderMode source_provider = 15;
```
Append the identical two fields to `ActiveAlarmSnapshot` (next free field 14):
```protobuf
bool degraded = 14;
AlarmProviderMode source_provider = 15;
```
**Step 2: Add provider status to the feed oneof.** Add a new oneof case to `AlarmFeedMessage` (next free field 4) and a new message:
```protobuf
message AlarmFeedMessage {
oneof payload {
ActiveAlarmSnapshot active_alarm = 1;
bool snapshot_complete = 2;
OnAlarmTransitionEvent transition = 3;
// Provider-mode status. Emitted once on stream open and again on every
// failover/failback so late joiners learn the current mode immediately.
AlarmProviderStatus provider_status = 4;
}
}
message AlarmProviderStatus {
AlarmProviderMode mode = 1;
bool degraded = 2; // true whenever mode == SUBTAG
string reason = 3; // human-readable switch reason
google.protobuf.Timestamp since = 4;
}
```
**Step 3: Add the worker→gateway mode-changed event to `MxEvent`.** Find the `MxEventFamily` enum and the `MxEvent` body oneof. Add a family member and a body message + oneof case (use the next free family value and the next free `MxEvent` body field number — check the file):
```protobuf
// in MxEventFamily enum:
MX_EVENT_FAMILY_ON_ALARM_PROVIDER_MODE_CHANGED = <next>;
// new message near OnAlarmTransitionEvent:
message OnAlarmProviderModeChangedEvent {
AlarmProviderMode mode = 1;
string reason = 2;
int32 hresult = 3; // COM HRESULT that triggered failover; 0 on failback
google.protobuf.Timestamp at = 4;
}
// in MxEvent body oneof:
OnAlarmProviderModeChangedEvent on_alarm_provider_mode_changed = <next>;
```
`AlarmProviderMode` is defined in `mxaccess_worker.proto`; confirm `mxaccess_gateway.proto` already has `import "mxaccess_worker.proto";` (it references `SubscribeAlarmsCommand`, so it does) and reference the enum unqualified or via its package as the existing references do.
**Step 4: Regenerate & verify.**
Run: `dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj`
Expected: build succeeds.
**Step 5: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto
git commit -m "contracts(gateway): AlarmProviderStatus feed case, degraded provenance, mode-changed event"
```
---
### Task 3: Proto round-trip tests for the new alarm fields
**Classification:** small
**Estimated implement time:** ~3 min
**Parallelizable with:** none (depends on Tasks 1-2)
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Tests/Contracts/ProtobufContractRoundTripTests.cs`
**Step 1: Add tests** mirroring the existing `Event_RoundTripsOnAlarmTransitionWithFullPayload` style:
```csharp
[Fact]
public void Feed_RoundTripsProviderStatus()
{
var since = Timestamp.FromDateTime(new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc));
var original = new AlarmFeedMessage
{
ProviderStatus = new AlarmProviderStatus
{
Mode = AlarmProviderMode.Subtag,
Degraded = true,
Reason = "wnwrap poll failed 3x (HRESULT 0x80004005)",
Since = since,
},
};
var parsed = AlarmFeedMessage.Parser.ParseFrom(original.ToByteArray());
Assert.Equal(original, parsed);
Assert.Equal(AlarmFeedMessage.PayloadOneofCase.ProviderStatus, parsed.PayloadCase);
Assert.True(parsed.ProviderStatus.Degraded);
Assert.Equal(AlarmProviderMode.Subtag, parsed.ProviderStatus.Mode);
}
[Fact]
public void Transition_RoundTripsDegradedProvenance()
{
var t = new OnAlarmTransitionEvent
{
AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi",
TransitionKind = AlarmTransitionKind.Raise,
Degraded = true,
SourceProvider = AlarmProviderMode.Subtag,
};
var parsed = OnAlarmTransitionEvent.Parser.ParseFrom(t.ToByteArray());
Assert.True(parsed.Degraded);
Assert.Equal(AlarmProviderMode.Subtag, parsed.SourceProvider);
}
```
**Step 2: Run red→green.**
Run: `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter FullyQualifiedName~ProtobufContractRoundTripTests`
Expected: PASS.
**Step 3: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Tests/Contracts/ProtobufContractRoundTripTests.cs
git commit -m "test(contracts): round-trip provider status + degraded provenance"
```
---
## Phase 1 — Worker: subtag consumer + failover
### Task 4: Subtag value-source abstraction + synthesis state holder
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none (Task 5 builds on it)
A testable seam so synthesis logic is unit-tested without COM. The COM wiring lands in Task 6.
**Files:**
- Create: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/ISubtagAlarmSource.cs`
- Create: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/SubtagAlarmStateMachine.cs`
- Test: `src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/SubtagAlarmStateMachineTests.cs`
**Step 1: Define the source abstraction.** `ISubtagAlarmSource` advises subtag addresses and raises a normalized value-change callback on the STA:
```csharp
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>A change in one advised subtag value, normalized off the COM boundary.</summary>
public sealed class SubtagValueChange
{
/// <summary>The full item address that changed (matches an AlarmSubtagTarget subtag).</summary>
public string ItemAddress { get; init; } = string.Empty;
/// <summary>The new value (boolean for .active/.acked, numeric for priority).</summary>
public object? Value { get; init; }
/// <summary>The change timestamp in UTC.</summary>
public DateTime TimestampUtc { get; init; }
}
/// <summary>
/// Advises a set of MXAccess subtag addresses and surfaces value changes.
/// The production implementation (Task 6) owns its own LMXProxyServerClass;
/// tests substitute a fake that pushes <see cref="SubtagValueChange"/>s.
/// </summary>
public interface ISubtagAlarmSource : IDisposable
{
/// <summary>Raised on the STA when an advised subtag's value changes.</summary>
event EventHandler<SubtagValueChange>? ValueChanged;
/// <summary>Advises every subtag in the supplied addresses; idempotent per address.</summary>
void Advise(IReadOnlyCollection<string> itemAddresses);
/// <summary>Writes a value to an item address (used for the ack-comment write).</summary>
void Write(string itemAddress, object? value);
}
```
**Step 2: Write the state-machine tests first.** `SubtagAlarmStateMachine` maps `(active, acked)` changes per target to `MxAlarmTransitionEvent`s. Test the four core transitions:
```csharp
namespace ZB.MOM.WW.MxGateway.Worker.Tests.MxAccess;
public sealed class SubtagAlarmStateMachineTests
{
private static AlarmSubtagTarget Target() => new()
{
AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi",
SourceObjectReference = "Tank01",
ActiveSubtag = "Tank01.Level.HiHi.active",
AckedSubtag = "Tank01.Level.HiHi.acked",
AckCommentSubtag = "Tank01.Level.HiHi.ackmsg",
};
[Fact]
public void ActiveFalseToTrue_EmitsRaise_FlaggedDegraded()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
var events = sm.Apply("Tank01.Level.HiHi.active", true, ts);
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.UnackAlm, e.Record.State);
Assert.Equal(MxAlarmStateKind.Unspecified, e.PreviousState);
Assert.Equal("Tank01.Level.HiHi", e.Record.TagName); // reference minus provider/area
}
[Fact]
public void AckedTrueWhileActive_EmitsAckTransition()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
var events = sm.Apply("Tank01.Level.HiHi.acked", true, ts.AddSeconds(5));
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.AckAlm, e.Record.State);
Assert.Equal(MxAlarmStateKind.UnackAlm, e.PreviousState);
}
[Fact]
public void ActiveTrueToFalse_WhileUnacked_EmitsUnackRtn()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
var events = sm.Apply("Tank01.Level.HiHi.active", false, ts.AddSeconds(10));
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.UnackRtn, e.Record.State);
}
[Fact]
public void Snapshot_ReflectsActiveAndAckedState()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
sm.Apply("Tank01.Level.HiHi.acked", true, ts);
var snap = Assert.Single(sm.SnapshotActive());
Assert.Equal(MxAlarmStateKind.AckAlm, snap.State);
}
}
```
Run: `dotnet test ...Worker.Tests... -p:Platform=x86 --filter FullyQualifiedName~SubtagAlarmStateMachineTests` → FAIL (type missing).
**Step 3: Implement `SubtagAlarmStateMachine`.** Build an address→target index (active/acked/priority/comment addresses), hold per-reference `(bool active, bool acked, DateTime firstRaiseUtc, int priority)`, and emit on change:
- active `false→true` ⇒ `UnackAlm`, set `firstRaiseUtc`, `PreviousState` from prior state.
- acked `false→true` while active ⇒ `AckAlm`.
- active `true→false` ⇒ `AckRtn` if currently acked else `UnackRtn`; then reset acked.
- priority change ⇒ update stored priority, no transition.
- `TagName` = `alarm_full_reference` with any `Provider!Area.` prefix stripped (match `WnWrapAlarmConsumer`'s reference shape so `GatewayAlarmMonitor` keys align). Set `ProviderName`, `Group`, `Priority`, `AlarmComment` from the target/last values. Mark a `Degraded`/source flag (carried via a new field — see Task 5 wiring).
- `SnapshotActive()` returns `MxAlarmSnapshotRecord` for references whose active is true.
**Step 4: Run green.** Expected: PASS.
**Step 5: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Worker/MxAccess/ISubtagAlarmSource.cs \
src/ZB.MOM.WW.MxGateway.Worker/MxAccess/SubtagAlarmStateMachine.cs \
src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/SubtagAlarmStateMachineTests.cs
git commit -m "worker(alarms): subtag value-source seam + synthesis state machine"
```
---
### Task 5: `SubtagAlarmConsumer` over the source seam (no COM yet)
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none (depends on Task 4)
**Files:**
- Create: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/SubtagAlarmConsumer.cs`
- Test: `src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/SubtagAlarmConsumerTests.cs`
**Step 1: Test with a fake `ISubtagAlarmSource`.** Drive value changes through the source, assert `AlarmTransitionEmitted` fires with synthesized records and that ack writes the comment to the ack-comment subtag:
```csharp
public sealed class SubtagAlarmConsumerTests
{
private sealed class FakeSource : ISubtagAlarmSource
{
public event EventHandler<SubtagValueChange>? ValueChanged;
public List<string> Advised { get; } = new();
public (string Address, object? Value)? LastWrite { get; private set; }
public void Advise(IReadOnlyCollection<string> a) => Advised.AddRange(a);
public void Write(string a, object? v) => LastWrite = (a, v);
public void Raise(string addr, object? val, DateTime ts) =>
ValueChanged?.Invoke(this, new SubtagValueChange { ItemAddress = addr, Value = val, TimestampUtc = ts });
public void Dispose() { }
}
private static AlarmSubtagTarget Target() => new()
{
AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi",
ActiveSubtag = "Tank01.Level.HiHi.active",
AckedSubtag = "Tank01.Level.HiHi.acked",
AckCommentSubtag = "Tank01.Level.HiHi.ackmsg",
};
[Fact]
public void Subscribe_AdvisesAllSubtags()
{
var src = new FakeSource();
using var c = new SubtagAlarmConsumer(src, new[] { Target() });
c.Subscribe("ignored-in-subtag-mode");
Assert.Contains("Tank01.Level.HiHi.active", src.Advised);
Assert.Contains("Tank01.Level.HiHi.acked", src.Advised);
}
[Fact]
public void ValueChange_RaisesSynthesizedTransition()
{
var src = new FakeSource();
using var c = new SubtagAlarmConsumer(src, new[] { Target() });
c.Subscribe("x");
MxAlarmTransitionEvent? seen = null;
c.AlarmTransitionEmitted += (_, e) => seen = e;
src.Raise("Tank01.Level.HiHi.active", true, new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc));
Assert.NotNull(seen);
Assert.Equal(MxAlarmStateKind.UnackAlm, seen!.Record.State);
}
[Fact]
public void AcknowledgeByName_WritesCommentToAckCommentSubtag()
{
var src = new FakeSource();
using var c = new SubtagAlarmConsumer(src, new[] { Target() });
c.Subscribe("x");
int rc = c.AcknowledgeByName("Tank01.Level.HiHi", "Galaxy", "Area",
"ack from HMI", "op1", "node", "dom", "Op One");
Assert.Equal(0, rc);
Assert.Equal(("Tank01.Level.HiHi.ackmsg", (object?)"ack from HMI"), src.LastWrite);
}
}
```
**Step 2: Implement `SubtagAlarmConsumer : IMxAccessAlarmConsumer`.**
- Constructor `(ISubtagAlarmSource source, IReadOnlyList<AlarmSubtagTarget> watchList)`; build a `SubtagAlarmStateMachine`; index `alarm_full_reference`→target for ack routing.
- `Subscribe(_)`: call `source.Advise(<all subtag addresses>)`; subscribe to `source.ValueChanged`, feed each into the state machine, and re-raise each produced `MxAlarmTransitionEvent` via `AlarmTransitionEmitted` (mark degraded).
- `AcknowledgeByName(alarmName, …, comment, …)`: resolve the target by reference; if no `AckCommentSubtag`, return a non-zero failure code; else `source.Write(target.AckCommentSubtag, comment)` and return 0.
- `AcknowledgeByGuid(guid, …)`: map the synthetic GUID (deterministic hash of reference — see Task 8 helper, or a local copy) back to a reference, then delegate to the name path; unknown GUID ⇒ non-zero.
- `SnapshotActiveAlarms()`: from the state machine.
- `PollOnce()`: no-op.
- `Dispose()`: unsubscribe + dispose source.
**Step 3: Run green.** `dotnet test ...Worker.Tests... -p:Platform=x86 --filter FullyQualifiedName~SubtagAlarmConsumerTests`.
**Step 4: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Worker/MxAccess/SubtagAlarmConsumer.cs \
src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/SubtagAlarmConsumerTests.cs
git commit -m "worker(alarms): SubtagAlarmConsumer synthesizing transitions over the source seam"
```
---
### Task 6: COM-backed `LmxSubtagAlarmSource` (own LMXProxyServerClass)
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none
The only piece that touches live COM. Like `WnWrapAlarmConsumer`, it owns its own MXAccess server object so the subtag source is self-contained and isolated from the session's item pipeline. Logic stays thin (advise/write/marshal); real verification is the live smoke test in Task 17.
**Files:**
- Create: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/LmxSubtagAlarmSource.cs`
- Test: `src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/LmxSubtagAlarmSourceTests.cs` (constructor/guard tests only; COM path is live-gated)
**Step 1: Implement `LmxSubtagAlarmSource : ISubtagAlarmSource`.**
- Own an `LMXProxyServerClass` (reuse the worker's `IMxAccessServer`/`MxAccessComServer` wrapper + `IMxAccessComObjectFactory` so it is fakeable; constructor takes the factory).
- `Advise(addresses)`: `RegisterServer` (topic) once; per address `AddItem`→`itemHandle`, `Advise`, and record `itemHandle→address`. Subscribe to the proxy's `OnDataChange`; in the handler, look up the address by `phItemHandle`, normalize `pvItemValue` (VARIANT→bool/double) and `pftItemTimeStamp`→UTC, and raise `ValueChanged`. All calls run on the STA (the worker STA pumps messages, so `OnDataChange` delivers).
- `Write(address, value)`: resolve/create the item handle, `server.Write(serverHandle, itemHandle, value, userId: 0)`.
- `Dispose()`: `UnAdvise`/`RemoveItem`/`Unregister`/release COM.
**Step 2: Tests** — only the non-COM guards (null factory throws; `Write` before `Advise` resolves a handle or throws a clear error). Mark the COM round-trip `[LiveMxAccessFact]` and `Skip` per the `AlarmsLiveSmokeTests` precedent.
**Step 3: Build x86 + run unit tests.**
`dotnet build src/ZB.MOM.WW.MxGateway.Worker/ZB.MOM.WW.MxGateway.Worker.csproj -p:Platform=x86`
`dotnet test ...Worker.Tests... -p:Platform=x86 --filter FullyQualifiedName~LmxSubtagAlarmSourceTests`
**Step 4: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Worker/MxAccess/LmxSubtagAlarmSource.cs \
src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/LmxSubtagAlarmSourceTests.cs
git commit -m "worker(alarms): COM-backed LmxSubtagAlarmSource advising alarm subtags"
```
---
### Task 7: `FailoverAlarmConsumer` state machine
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none (depends on Task 5)
**Files:**
- Create: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/FailoverAlarmConsumer.cs`
- Create: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/AlarmProviderModeChange.cs` (small EventArgs)
- Test: `src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/FailoverAlarmConsumerTests.cs`
**Step 1: Test the switch/failback with a fake primary that throws.**
```csharp
public sealed class FailoverAlarmConsumerTests
{
private sealed class FlakyPrimary : IMxAccessAlarmConsumer
{
public event EventHandler<MxAlarmTransitionEvent>? AlarmTransitionEmitted;
public int PollsUntilHeal = int.MaxValue; // becomes healthy after N polls while degraded
public bool ThrowOnPoll = true;
private int _polls;
public void Subscribe(string s) { if (ThrowOnPoll) throw new COMException("boom", unchecked((int)0x80004005)); }
public void PollOnce()
{
_polls++;
if (ThrowOnPoll && _polls < PollsUntilHeal) throw new COMException("boom", unchecked((int)0x80004005));
}
public int AcknowledgeByGuid(Guid g, string c, string a, string b, string d, string e) => 0;
public int AcknowledgeByName(string n, string p, string gr, string c, string a, string b, string d, string e) => 0;
public IReadOnlyList<MxAlarmSnapshotRecord> SnapshotActiveAlarms() => Array.Empty<MxAlarmSnapshotRecord>();
public void Dispose() { }
}
private sealed class StubStandby : IMxAccessAlarmConsumer { /* records Subscribe, no-op rest */ }
[Fact]
public void Primary_FailsThresholdTimes_SwitchesToSubtagAndEmitsModeChange()
{
var primary = new FlakyPrimary();
var standby = new StubStandby();
using var c = new FailoverAlarmConsumer(primary, standby,
new FailoverSettings(threshold: 3, probeIntervalSeconds: 30, stableProbes: 3));
AlarmProviderModeChange? change = null;
c.ProviderModeChanged += (_, e) => change = e;
c.Subscribe("\\\\host\\Galaxy!Area"); // primary.Subscribe throws -> counts as failure 1
c.PollOnce(); // failure 2
c.PollOnce(); // failure 3 -> switch
Assert.NotNull(change);
Assert.Equal(AlarmProviderMode.Subtag, change!.Mode);
}
[Fact]
public void WhileDegraded_PrimaryHeals_FailsBackAfterStableProbes()
{
var primary = new FlakyPrimary { PollsUntilHeal = 0 }; // will heal once we stop throwing
var standby = new StubStandby();
using var c = new FailoverAlarmConsumer(primary, standby,
new FailoverSettings(threshold: 1, probeIntervalSeconds: 0, stableProbes: 2));
var modes = new List<AlarmProviderMode>();
c.ProviderModeChanged += (_, e) => modes.Add(e.Mode);
c.Subscribe("x"); // failure -> switch to subtag
primary.ThrowOnPoll = false;
c.ProbeOnce(); // clean probe 1
c.ProbeOnce(); // clean probe 2 -> failback
Assert.Equal(AlarmProviderMode.Subtag, modes[0]);
Assert.Equal(AlarmProviderMode.Alarmmgr, modes[^1]);
}
}
```
**Step 2: Implement.**
- `record FailoverSettings(int threshold, int probeIntervalSeconds, int stableProbes)`; `AlarmProviderModeChange : EventArgs { AlarmProviderMode Mode; string Reason; int HResult; DateTime AtUtc; }`.
- Constructor `(IMxAccessAlarmConsumer primary, IMxAccessAlarmConsumer standby, FailoverSettings settings)`; forced-mode variants handled in Task 9 wiring (forced ⇒ skip the other consumer).
- Forward `AlarmTransitionEmitted` from the **active** child only (swap the subscription on switch).
- Wrap `Subscribe`/`PollOnce` on the primary: on `COMException` (or a failure HRESULT) while `PrimaryActive`, increment a counter; at `threshold`, ensure standby `Subscribe`d, set active=standby, snapshot standby for hand-off, raise `ProviderModeChanged(Subtag, reason, hresult, now)`. Reset counter on any clean primary poll.
- `ProbeOnce()` (driven by the poll loop while degraded, gated by `probeIntervalSeconds`): try primary `Subscribe`+`PollOnce`; count consecutive clean probes; at `stableProbes`, set active=primary, return standby to standby, raise `ProviderModeChanged(Alarmmgr, "recovered", 0, now)`.
- `Acknowledge*` / `SnapshotActiveAlarms` delegate to the **active** child.
- `PollOnce()` drives the active child's poll, and—while degraded—also drives the failback probe cadence.
**Step 3: Run green** (x86 filter `FailoverAlarmConsumerTests`).
**Step 4: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Worker/MxAccess/FailoverAlarmConsumer.cs \
src/ZB.MOM.WW.MxGateway.Worker/MxAccess/AlarmProviderModeChange.cs \
src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/FailoverAlarmConsumerTests.cs
git commit -m "worker(alarms): FailoverAlarmConsumer auto-failover/failback state machine"
```
---
### Task 8: Synthetic-GUID helper + degraded flag on the event sink path
**Classification:** standard
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 9
Carry `degraded` + `source_provider` from the worker synthesis into the emitted `OnAlarmTransitionEvent`.
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAlarmSnapshot.cs` (add `bool Degraded`)
- Modify: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessAlarmEventSink.cs` (`EnqueueTransition` carries degraded)
- Modify: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessEventMapper.cs` (`CreateOnAlarmTransition` sets `Degraded`/`SourceProvider`)
- Create: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/SyntheticAlarmGuid.cs`
- Test: add cases to `src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/AlarmDispatcherTests.cs` and a new `SyntheticAlarmGuidTests.cs`
**Step 1: `SyntheticAlarmGuid.ForReference(string reference)`** — deterministic GUID from a stable hash (e.g. MD5 of the UTF-8 reference → `new Guid(bytes)`), so subtag-mode acks resolve by GUID. Test determinism + difference:
```csharp
[Fact] public void SameReference_SameGuid() =>
Assert.Equal(SyntheticAlarmGuid.ForReference("A.B.C"), SyntheticAlarmGuid.ForReference("A.B.C"));
[Fact] public void DifferentReference_DifferentGuid() =>
Assert.NotEqual(SyntheticAlarmGuid.ForReference("A.B.C"), SyntheticAlarmGuid.ForReference("A.B.D"));
```
**Step 2: Thread `degraded`** through `MxAlarmSnapshotRecord.Degraded`, `EnqueueTransition(... bool degraded)`, and `CreateOnAlarmTransition(... bool degraded, AlarmProviderMode sourceProvider)`. Default `degraded=false`, `sourceProvider=Alarmmgr` so the wnwrap path is unchanged (regression: existing `AlarmDispatcherTests` still pass with `Degraded=false`).
**Step 3: Tests** — extend `AlarmDispatcherTests` with a subtag-style transition asserting `body.Degraded == true` and `SourceProvider == Subtag`.
**Step 4: Build x86 + run** worker tests for `AlarmDispatcherTests`, `SyntheticAlarmGuidTests`.
**Step 5: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAlarmSnapshot.cs \
src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessAlarmEventSink.cs \
src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessEventMapper.cs \
src/ZB.MOM.WW.MxGateway.Worker/MxAccess/SyntheticAlarmGuid.cs \
src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/
git commit -m "worker(alarms): synthetic GUID + degraded provenance on emitted transitions"
```
---
### Task 9: Wire watch-list + failover config through `AlarmCommandHandler`; emit mode-changed event
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none (depends on Tasks 5, 7, 8)
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/AlarmCommandHandler.cs`
- Modify: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/IAlarmCommandHandler.cs`
- Modify: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessCommandExecutor.cs` (`ExecuteSubscribeAlarms`, ~lines 588-616)
- Modify: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessStaSession.cs` (consumer factory wiring; mode-change → event queue)
- Test: `src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/AlarmCommandHandlerTests.cs` (extend or create)
**Step 1: Carry the subscribe payload.** Change the alarm subscribe entry point from `Subscribe(string subscription)` to `Subscribe(SubscribeAlarmsCommand command)` (the command now has `ForcedMode`, `WatchList`, `Failover`). In `AlarmCommandHandler.Subscribe`:
- Build the active provider per `ForcedMode`:
- `ALARMMGR` ⇒ `WnWrapAlarmConsumer` only.
- `SUBTAG` ⇒ `SubtagAlarmConsumer(new LmxSubtagAlarmSource(factory), watchList)` only.
- `UNSPECIFIED` ⇒ `FailoverAlarmConsumer(primary: wnwrap, standby: subtag, settings-from-Failover)`.
- Use the existing `consumerFactory` seam but widen it to `Func<SubscribeAlarmsCommand, IMxAccessAlarmConsumer>` so tests inject fakes and production builds the failover composite. Subscribe to `FailoverAlarmConsumer.ProviderModeChanged` and enqueue an `OnAlarmProviderModeChangedEvent` MxEvent via the event queue (new mapper method `CreateOnAlarmProviderModeChanged`).
**Step 2: Executor + STA wiring.** `ExecuteSubscribeAlarms` passes the full `SubscribeAlarmsCommand` (not just the expression). In `MxAccessStaSession`, the `alarmCommandHandlerFactory` must give the handler access to the `IMxAccessComObjectFactory` so the subtag source can create its own proxy server on the STA; keep the `EnsureOnAlarmConsumerThread` affinity guard on every path.
**Step 3: Test** — fake consumer factory; assert that a `SUBTAG` forced command builds the subtag consumer and advises; that an auto command building a fake failover composite, when it raises `ProviderModeChanged`, enqueues an `OnAlarmProviderModeChangedEvent` on the queue.
**Step 4: Build x86 + worker tests.**
**Step 5: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Worker/MxAccess/ src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/
git commit -m "worker(alarms): route watch-list/failover config; emit provider-mode-changed event"
```
---
## Phase 2 — Gateway: discovery, options, monitor, metrics, dashboard
### Task 10: `AlarmsOptions.Fallback` + validation
**Classification:** standard
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 11, Task 13
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Configuration/AlarmsOptions.cs`
- Create: `src/ZB.MOM.WW.MxGateway.Server/Configuration/AlarmFallbackOptions.cs`
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Configuration/GatewayOptionsValidator.cs` (`ValidateAlarms`, ~lines 234-258)
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Configuration/GatewayOptionsValidatorTests.cs` (extend)
**Step 1:** Add `AlarmFallbackOptions Fallback { get; init; } = new();` to `AlarmsOptions`. `AlarmFallbackOptions`: `string Mode = "Auto"` (`Auto|ForceAlarmManager|ForceSubtag`), `int ConsecutiveFailureThreshold = 3`, `int FailbackProbeIntervalSeconds = 30`, `int FailbackStableProbes = 3`, a `Discovery` sub-object (`bool UseGalaxyRepository = true`, `string Area = ""`, `string[] IncludeAttributes = []`, `string[] ExcludeAttributes = []`), and a `Subtags` sub-object (`Active="active"`, `Acked="acked"`, `AckComment=""`, `Priority="priority"`).
**Step 2:** In `ValidateAlarms`, when `Enabled` and `Mode == "ForceSubtag"` and `Discovery.UseGalaxyRepository == false` and `IncludeAttributes` empty ⇒ add a validation error ("ForceSubtag requires Galaxy Repository discovery or an explicit IncludeAttributes list"). Floor the three numeric values at 1. Validate `Mode` is one of the three literals.
**Step 3-5:** Test the new validation cases (red→green), build the server, commit.
---
### Task 11: Galaxy Repository "alarm attributes" discovery query
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 10, Task 13
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Galaxy/GalaxyRepository.cs` (add `GetAlarmAttributesAsync` + SQL constant, following `GetAttributesAsync` ~lines 86-115 and `AttributesSql` ~line 176)
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Galaxy/IGalaxyRepository.cs`
- Create: `src/ZB.MOM.WW.MxGateway.Server/Galaxy/GalaxyAlarmAttributeRow.cs`
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Galaxy/` (projection unit test; live SQL gated)
**Step 1:** `GalaxyAlarmAttributeRow { string FullTagReference; string SourceObjectReference; string AckCommentSubtag; }` (and any priority subtag). `GetAlarmAttributesAsync` reuses the existing `is_alarm` detection (the `AlarmExtension` primitive join already in `AttributesSql`) filtered to `is_alarm = 1`, projecting the alarm reference + its ack-comment attribute. Follow the exact `SqlConnection`/`SqlCommand`/`SqlDataReader` pattern from `GetAttributesAsync`.
**Step 2:** Unit-test the row→`AlarmSubtagTarget` mapping (a pure mapper function); gate any live-DB test like the existing Galaxy live tests (or `Skip` with a note, matching `AlarmsLiveSmokeTests`).
**Step 3-5:** red→green, build server, commit.
---
### Task 12: Watch-list resolver (GR SQL + config override → `AlarmSubtagTarget[]`)
**Classification:** standard
**Estimated implement time:** ~4 min
**Parallelizable with:** none (depends on Tasks 10, 11)
**Files:**
- Create: `src/ZB.MOM.WW.MxGateway.Server/Alarms/AlarmWatchListResolver.cs`
- Create: `src/ZB.MOM.WW.MxGateway.Server/Alarms/IAlarmWatchListResolver.cs`
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Alarms/AlarmWatchListResolverTests.cs`
**Step 1: Test the merge** with a fake `IGalaxyRepository`:
- discovery rows + `IncludeAttributes` are unioned; `ExcludeAttributes` removed; each becomes an `AlarmSubtagTarget` with `.active`/`.acked`/`.ackmsg` addresses composed from the configured `Subtags` names (`<reference>.<Active>`, etc.); empty config subtag names fall back to defaults; GR unavailable + no includes ⇒ empty list + a logged warning flag.
**Step 2: Implement** `ResolveAsync(AlarmsOptions, CancellationToken) → IReadOnlyList<AlarmSubtagTarget>`.
**Step 3-5:** red→green, build, commit.
---
### Task 13: Gateway metrics — provider-mode gauge + switch counter
**Classification:** small
**Estimated implement time:** ~3 min
**Parallelizable with:** Task 10, Task 11
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Metrics/GatewayMetrics.cs` (ctor ~lines 55-79; add counter + observable gauge following the existing pattern)
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Metrics/GatewayMetricsTests.cs` (if present; else assert via a `MeterListener`)
**Step 1:** Add `mxgateway.alarms.provider_switches` counter (tagged `from`,`to`,`reason`) and `mxgateway.alarms.provider_mode` observable gauge (1=alarmmgr, 2=subtag), plus `AlarmProviderSwitched(int from, int to, string reason)` and a private `GetAlarmProviderMode()` (lock on `_syncRoot` like the others).
**Step 2-4:** test, build, commit.
---
### Task 14: `GatewayAlarmMonitor` — arm watch-list, reflect provider mode, reconcile on switch
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none (depends on Tasks 9, 12, 13)
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Alarms/GatewayAlarmMonitor.cs` (ctor ~41-49; `SubscribeAlarmsAsync` ~210-233; event-drain loop; `StreamAsync` ~386-434)
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Alarms/GatewayAlarmMonitorProviderModeTests.cs` (new, using `FakeWorkerHarness`)
**Step 1:** Inject `IAlarmWatchListResolver` and `GatewayMetrics`. In `SubscribeAlarmsAsync`, resolve the watch-list and build the `SubscribeAlarmsCommand` with `ForcedMode` (from `Fallback.Mode`), `WatchList`, and `Failover` populated from options — instead of the bare `{ SubscriptionExpression }`.
**Step 2:** In the worker-event drain path, handle `OnAlarmProviderModeChangedEvent`: update a `_providerStatus` field (mode/degraded/reason/since), `Broadcast(new AlarmFeedMessage { ProviderStatus = … })` to every subscriber, call `metrics.AlarmProviderSwitched(...)`, and force a `ReconcileAsync` so the cache re-seeds from the now-active provider (avoids raise/clear storms).
**Step 3:** In `StreamAsync`, emit the current `provider_status` as the **first** message (before the snapshot) so a late joiner immediately knows the mode.
**Step 4: Test** — stand up the monitor with `FakeWorkerHarness`; emit an `OnAlarmProviderModeChangedEvent(Subtag)`; assert a `StreamAsync` subscriber receives a `ProviderStatus{ Mode=Subtag, Degraded=true }` and that the switch counter incremented. Also assert a transition emitted in subtag mode flows through with `Degraded=true`.
**Step 5:** build server, run the new test, commit.
---
### Task 15: Dashboard — push provider status to `/hubs/alarms` + UI indicator
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none (depends on Task 14)
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Dashboard/Hubs/AlarmsHubPublisher.cs` (forward `ProviderStatus` messages — they already flow through `StreamAsync`, so confirm the existing `SendAsync(AlarmMessage, message)` carries them; add a dedicated `"ProviderModeChanged"` client method if the dashboard needs a distinct channel)
- Modify: the alarms dashboard page/component (Bootstrap-only badge: green "alarmmgr" / amber "degraded — subtag") — find under `src/ZB.MOM.WW.MxGateway.Server/Dashboard/`
- Test: `src/ZB.MOM.WW.MxGateway.Tests/` dashboard model test (e.g. a `DashboardAlarmProviderStatus.FromFeed` mapper, mirroring `DashboardActiveAlarm.FromSnapshot`)
**Constraint:** Bootstrap CSS/JS only — no MudBlazor/Radzen/FluentUI.
**Steps:** TDD the model mapper, wire the publisher + badge, build, commit.
---
## Phase 3 — Integration, docs, live smoke
### Task 16: End-to-end fake-worker failover test
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 18
**Files:**
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Alarms/AlarmFailoverEndToEndTests.cs`
Drive the full gateway path with `FakeWorkerHarness`: subscribe (assert the `SubscribeAlarmsCommand` carries a watch-list), emit a wnwrap-style transition (assert `Degraded=false`), emit `OnAlarmProviderModeChangedEvent(Subtag)`, emit a synthesized transition (assert `Degraded=true`, `SourceProvider=Subtag`), then `OnAlarmProviderModeChangedEvent(Alarmmgr)` and assert the feed reports recovery. Build, run, commit.
---
### Task 17: Live subtag smoke test (opt-in)
**Classification:** small
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 18
**Files:**
- Test: `src/ZB.MOM.WW.MxGateway.IntegrationTests/...AlarmSubtagLiveSmokeTests.cs` (or the worker live suite)
A `[LiveMxAccessFact]`, `Skip`-by-default test (per `AlarmsLiveSmokeTests` precedent) that, against a live Galaxy + alarm flip script: advises the real `.active`/`.acked` subtags via `LmxSubtagAlarmSource`, asserts a synthesized raise/clear, and performs an ack via the ack-comment write. Document the exact subtag names discovered (resolves the design's open item). Commit.
---
### Task 18: Documentation
**Classification:** trivial
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 16, Task 17
**Files:**
- Modify: `gateway.md` (alarm provider section: dual provider + auto-failover/failback)
- Modify: `docs/DesignDecisions.md` (record the fallback decision + parity rationale)
- Modify: `docs/GatewayConfiguration.md` (the `MxGateway:Alarms:Fallback` block)
- Modify: `docs/AlarmClientDiscovery.md` (subtag provider, synthesis rules, ack-comment write)
- Modify: `docs/Grpc.md` (new `provider_status` feed case + `degraded`/`source_provider` fields)
Follow `StyleGuide.md` (PascalCase filenames, present tense, explain *why*). No code; commit.
---
## Execution order & parallelism summary
- **Serial spine:** 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8/9 → 10/11 → 12 → 13 → 14 → 15 → 16 → 17/18.
- **Parallelizable clusters:** {8, 9 partially}, {10, 11, 13}, {16, 17, 18}.
- **High-risk tasks** (full review chain): 1, 2, 6, 7, 9, 14. **Standard:** 4, 5, 8, 10, 11, 12, 15, 16. **Small/trivial:** 3, 13, 17, 18.
## Risk notes for the executor
- **Field-number collisions:** Task 2 must read the live `MxEvent`/`MxEventFamily` numbers before adding — the agent map gave alarm-payload maxima but not `MxEvent`'s. Verify before editing.
- **STA discipline:** every COM call in `LmxSubtagAlarmSource` and every consumer swap runs on the worker STA; keep the `EnsureOnAlarmConsumerThread` guard. The worker STA already pumps Windows messages, which is required for the subtag `OnDataChange` to deliver.
- **Parity regression:** alarmmgr-mode output must be byte-for-byte unchanged. Existing `AlarmDispatcherTests` and `ProtobufContractRoundTripTests` are the guardrail — they must stay green with `Degraded=false` defaults.
- **Subtag names unverified:** the design leaves exact AVEVA subtag names (`.active`, `.acked`, ack-comment) to confirm against `C:\Users\dohertj2\Desktop\mxaccess` + a live Galaxy (Task 17). The config `Subtags` block exists so names are not hard-coded.
@@ -0,0 +1,147 @@
{
"planPath": "docs/plans/2026-06-13-alarm-subtag-fallback.md",
"tasks": [
{
"id": 54,
"subject": "Task 1: Worker proto \u2014 watch-list, failover config, AlarmProviderMode",
"status": "completed"
},
{
"id": 55,
"subject": "Task 2: Gateway proto \u2014 provider status, degraded provenance, mode-changed event",
"status": "completed",
"blockedBy": [
54
]
},
{
"id": 56,
"subject": "Task 3: Proto round-trip tests for new alarm fields",
"status": "completed",
"blockedBy": [
54,
55
]
},
{
"id": 57,
"subject": "Task 4: Subtag value-source abstraction + synthesis state machine",
"status": "completed",
"blockedBy": [
54
]
},
{
"id": 58,
"subject": "Task 5: SubtagAlarmConsumer over the source seam",
"status": "completed",
"blockedBy": [
57
]
},
{
"id": 59,
"subject": "Task 6: COM-backed LmxSubtagAlarmSource",
"status": "completed",
"blockedBy": [
57
]
},
{
"id": 60,
"subject": "Task 7: FailoverAlarmConsumer state machine",
"status": "completed",
"blockedBy": [
58
]
},
{
"id": 61,
"subject": "Task 8: Synthetic GUID + degraded flag on event sink path",
"status": "completed",
"blockedBy": [
55
]
},
{
"id": 62,
"subject": "Task 9: Wire watch-list/failover through AlarmCommandHandler; emit mode-changed",
"status": "completed",
"blockedBy": [
58,
60,
61
]
},
{
"id": 63,
"subject": "Task 10: AlarmsOptions.Fallback + validation",
"status": "completed"
},
{
"id": 64,
"subject": "Task 11: Galaxy Repository alarm-attributes discovery query",
"status": "completed"
},
{
"id": 65,
"subject": "Task 12: Watch-list resolver (GR SQL + config override)",
"status": "completed",
"blockedBy": [
54,
63,
64
]
},
{
"id": 66,
"subject": "Task 13: Metrics \u2014 provider-mode gauge + switch counter",
"status": "completed"
},
{
"id": 67,
"subject": "Task 14: GatewayAlarmMonitor \u2014 arm watch-list, reflect mode, reconcile on switch",
"status": "completed",
"blockedBy": [
55,
62,
65,
66
]
},
{
"id": 68,
"subject": "Task 15: Dashboard \u2014 push provider status + UI badge",
"status": "completed",
"blockedBy": [
67
]
},
{
"id": 69,
"subject": "Task 16: End-to-end fake-worker failover test",
"status": "completed",
"blockedBy": [
67
]
},
{
"id": 70,
"subject": "Task 17: Live subtag smoke test (opt-in)",
"status": "completed",
"blockedBy": [
59,
62
]
},
{
"id": 71,
"subject": "Task 18: Documentation",
"status": "completed",
"blockedBy": [
67
]
}
],
"lastUpdated": "2026-06-13T13:30:00Z"
}
+57
View File
@@ -143,6 +143,63 @@ session if the worker faults. Gated by `MxGateway:Alarms:Enabled` — see
`docs/DesignDecisions.md` for why this reverses the v1 single-subscriber rule
for the alarm subsystem.
### Alarm providers and failover
The alarm feed has two providers, both implemented worker-side:
- **Alarm manager (primary):** `WnWrapAlarmConsumer` polls
`wwAlarmConsumerClass.GetXmlCurrentAlarms2` on the worker STA. This is the
authoritative native source.
- **Subtag monitoring (standby):** `SubtagAlarmConsumer` advises each alarm
attribute's subtags (`.active`, `.acked`, optionally `.priority`) via the
existing `AddItem`/`Advise` pipeline through `LmxSubtagAlarmSource` and
synthesizes alarm transitions with `SubtagAlarmStateMachine`. This is a
non-parity, lower-fidelity source — synthetic GUIDs, no native raise
timestamps, narrower fields.
`FailoverAlarmConsumer` wraps both and owns the state machine:
- **Auto-failover:** after `ConsecutiveFailureThreshold` (default 3)
consecutive wnwrap COM failures — `Subscribe` or `PollOnce` throws or
returns a failure HRESULT — it activates the standby. The standby is armed
(subscribed and adviseing) from the start so its state is warm at the moment
of switch.
- **Auto-failback:** while degraded, every `FailbackProbeIntervalSeconds`
(default 30) it re-probes the still-subscribed primary. After
`FailbackStableProbes` (default 3) consecutive clean polls it switches back
to the alarm manager.
- **On every switch:** the consumer snapshots the now-active provider and
emits `OnAlarmProviderModeChangedEvent` so the gateway can reconcile its
cache without a raise/clear storm.
Synthesis is worker-side. This preserves the parity rule — the gateway
forwards only events the worker emits and never synthesizes transitions
itself. The synthesis rules are documented in
`docs/AlarmClientDiscovery.md`.
**Acknowledge in subtag mode:** the ack-by-name path writes the operator
comment to the alarm attribute's ack-comment subtag. The write performs the
ack. If the attribute has no writable ack-comment subtag configured, the RPC
returns `FailedPrecondition`. In alarm-manager mode, `AlarmAckByName` is
used as before.
**Degraded state visibility:** every subtag-mode transition carries
`degraded = true` and `source_provider = ALARM_PROVIDER_MODE_SUBTAG` on the
`OnAlarmTransitionEvent` and `ActiveAlarmSnapshot` proto fields. The
`AlarmFeedMessage` feed emits an `AlarmProviderStatus` message (the
`provider_status` oneof case) on stream open and on every switch. The
dashboard shows a Bootstrap badge (green for alarm manager, amber when
degraded). Metrics: `mxgateway.alarms.provider_mode` gauge (1 = alarmmgr,
2 = subtag) and `mxgateway.alarms.provider_switches` counter.
Forced modes are available via `MxGateway:Alarms:Fallback:Mode`:
`ForceAlarmManager` disables failover; `ForceSubtag` forces the standby
on from startup; `Auto` (default) enables failover and failback. Watch-list
discovery for the subtag provider uses Galaxy Repository SQL with config
overrides. See `docs/GatewayConfiguration.md` for the full `Fallback` option
block and `docs/AlarmClientDiscovery.md` for synthesis rules and fidelity
limitations.
Dashboard authentication is LDAP-backed (distinct from the API-key model on
the gRPC API). `/login` accepts username and password in a form body, binds
against `MxGateway:Ldap`, maps the user's LDAP groups to `Admin` or `Viewer`
File diff suppressed because it is too large Load Diff
@@ -315,6 +315,14 @@ message SubscribeBulkCommand {
repeated string tag_addresses = 2;
}
// Provider selection / current provider for the alarm feed. UNSPECIFIED on a
// SubscribeAlarmsCommand means auto: alarmmgr primary with subtag fallback.
enum AlarmProviderMode {
ALARM_PROVIDER_MODE_UNSPECIFIED = 0;
ALARM_PROVIDER_MODE_ALARMMGR = 1;
ALARM_PROVIDER_MODE_SUBTAG = 2;
}
// Subscribe the worker's alarm consumer to an AVEVA alarm provider.
// Subscription expression follows the canonical
// `\\<machine>\Galaxy!<area>` format (literal "Galaxy" provider). The
@@ -323,6 +331,12 @@ message SubscribeBulkCommand {
// SubscribeAlarms to reconfigure).
message SubscribeAlarmsCommand {
string subscription_expression = 1;
// UNSPECIFIED = auto-failover/failback. ALARMMGR/SUBTAG force one provider.
AlarmProviderMode forced_mode = 2;
// Subtag watch-list resolved by the gateway (GR SQL + config). Empty in pure
// alarmmgr mode; in subtag mode it bounds what the consumer can observe.
repeated AlarmSubtagTarget watch_list = 3;
AlarmFailoverConfig failover = 4;
}
// Tear down the worker's alarm consumer. No-op if no subscription is
@@ -330,6 +344,23 @@ message SubscribeAlarmsCommand {
message UnsubscribeAlarmsCommand {
}
// One alarm attribute the subtag fallback consumer advises. Addresses are full
// MXAccess item references the worker passes straight to AddItem.
message AlarmSubtagTarget {
string alarm_full_reference = 1; // e.g. "Galaxy!Area.Tank01.Level.HiHi"
string source_object_reference = 2; // e.g. "Tank01"
string active_subtag = 3; // item address of the in-alarm boolean
string acked_subtag = 4; // item address of the acknowledged boolean
string ack_comment_subtag = 5; // writable ack-comment attribute (ack write target)
string priority_subtag = 6; // optional severity source; empty if absent
}
message AlarmFailoverConfig {
int32 consecutive_failure_threshold = 1; // wnwrap COM failures before switching (>=1)
int32 failback_probe_interval_seconds = 2; // probe cadence while degraded (>=1)
int32 failback_stable_probes = 3; // clean probes before switching back (>=1)
}
// Acknowledge a single alarm by its GUID. Operator identity fields are
// recorded atomically with the ack transition in the alarm-history log.
// The reply's hresult / native_status surfaces AVEVA's
@@ -684,6 +715,7 @@ message MxEvent {
OperationCompleteEvent operation_complete = 22;
OnBufferedDataChangeEvent on_buffered_data_change = 23;
OnAlarmTransitionEvent on_alarm_transition = 24;
OnAlarmProviderModeChangedEvent on_alarm_provider_mode_changed = 25;
}
}
@@ -694,6 +726,7 @@ enum MxEventFamily {
MX_EVENT_FAMILY_OPERATION_COMPLETE = 3;
MX_EVENT_FAMILY_ON_BUFFERED_DATA_CHANGE = 4;
MX_EVENT_FAMILY_ON_ALARM_TRANSITION = 5;
MX_EVENT_FAMILY_ON_ALARM_PROVIDER_MODE_CHANGED = 6;
}
message OnDataChangeEvent {
@@ -768,6 +801,20 @@ message OnAlarmTransitionEvent {
// Limit/threshold value that triggered the transition for limit alarms.
// Optional; populated for AnalogLimitAlarm-family transitions.
MxValue limit_value = 13;
// True when this transition came from the subtag-monitoring fallback rather
// than the native alarmmgr provider — synthesized from data changes, reduced
// fidelity (synthetic GUID, no native raise time).
bool degraded = 14;
// Which provider produced this transition.
AlarmProviderMode source_provider = 15;
}
message OnAlarmProviderModeChangedEvent {
AlarmProviderMode mode = 1;
string reason = 2;
int32 hresult = 3; // COM HRESULT that triggered failover; 0 on failback
google.protobuf.Timestamp at = 4;
}
enum AlarmTransitionKind {
@@ -800,6 +847,8 @@ message ActiveAlarmSnapshot {
string operator_comment = 11;
MxValue current_value = 12;
MxValue limit_value = 13;
bool degraded = 14;
AlarmProviderMode source_provider = 15;
}
enum AlarmConditionState {
@@ -866,9 +915,19 @@ message AlarmFeedMessage {
bool snapshot_complete = 2;
// A live alarm state change (raise / acknowledge / clear).
OnAlarmTransitionEvent transition = 3;
// Provider-mode status. Emitted once on stream open and again on every
// failover/failback so late joiners learn the current mode immediately.
AlarmProviderStatus provider_status = 4;
}
}
message AlarmProviderStatus {
AlarmProviderMode mode = 1;
bool degraded = 2; // true whenever mode == SUBTAG
string reason = 3; // human-readable switch reason
google.protobuf.Timestamp since = 4;
}
message MxStatusProxy {
// Mirrors the `success` member of the MXAccess MXSTATUS_PROXY struct
// (a 16-bit signed value in the COM struct, widened to int32 on the
@@ -0,0 +1,168 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Configuration;
using ZB.MOM.WW.MxGateway.Server.Galaxy;
namespace ZB.MOM.WW.MxGateway.Server.Alarms;
/// <summary>
/// Default <see cref="IAlarmWatchListResolver"/>. Merges Galaxy Repository
/// alarm-attribute discovery with the configured include/exclude overrides
/// and composes the per-attribute subtag item addresses from the configured
/// subtag names.
/// </summary>
// NOTE: The exact subtag names and the canonical AlarmFullReference shape
// ("Galaxy!{area}.{reference}") are validated against a live Galaxy in the
// Task 17 live smoke test. The config Subtags block exists precisely so these
// names are not hard-coded here.
public sealed class AlarmWatchListResolver : IAlarmWatchListResolver
{
private const string ProviderLiteral = "Galaxy";
private const string DefaultActiveSubtag = "active";
private const string DefaultAckedSubtag = "acked";
private readonly IGalaxyRepository _repository;
private readonly ILogger<AlarmWatchListResolver> _logger;
/// <summary>Initializes the watch-list resolver.</summary>
/// <param name="repository">Galaxy Repository used for alarm-attribute discovery.</param>
/// <param name="logger">Diagnostic logger.</param>
public AlarmWatchListResolver(
IGalaxyRepository repository,
ILogger<AlarmWatchListResolver> logger)
{
_repository = repository ?? throw new ArgumentNullException(nameof(repository));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
/// <inheritdoc />
public async Task<IReadOnlyList<AlarmSubtagTarget>> ResolveAsync(
AlarmsOptions options,
CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(options);
AlarmDiscoveryOptions discovery = options.Fallback.Discovery;
// 1. Build the ordered, de-duplicated attribute reference set.
// Each entry carries the reference plus the source-object reference.
List<(string Reference, string SourceObject)> ordered = [];
HashSet<string> seen = new(StringComparer.OrdinalIgnoreCase);
if (discovery.UseGalaxyRepository)
{
List<GalaxyAlarmAttributeRow> rows;
try
{
rows = await _repository.GetAlarmAttributesAsync(cancellationToken).ConfigureAwait(false);
}
catch (Exception ex)
{
// Discovery being unavailable must not crash the resolver: log and
// continue with an empty discovery set. The caller decides what to
// do with the (possibly config-only) result.
_logger.LogWarning(
ex,
"Galaxy Repository alarm-attribute discovery failed; continuing with configuration-only watch-list.");
rows = [];
}
foreach (GalaxyAlarmAttributeRow row in rows)
{
if (string.IsNullOrEmpty(row.FullTagReference) || !seen.Add(row.FullTagReference))
{
continue;
}
ordered.Add((row.FullTagReference, row.SourceObjectReference));
}
}
foreach (string include in discovery.IncludeAttributes)
{
if (string.IsNullOrEmpty(include) || !seen.Add(include))
{
continue;
}
ordered.Add((include, DeriveSourceObject(include)));
}
// Remove excluded references (case-insensitive), but only when GR discovery
// is active. ExcludeAttributes is documented as "Ignored when
// UseGalaxyRepository is false" (AlarmDiscoveryOptions.ExcludeAttributes).
// Whitespace-only entries are skipped, consistent with the include guard above.
if (discovery.UseGalaxyRepository)
{
HashSet<string> excluded = new(
discovery.ExcludeAttributes.Where(e => !string.IsNullOrWhiteSpace(e)),
StringComparer.OrdinalIgnoreCase);
if (excluded.Count > 0)
{
ordered.RemoveAll(e => excluded.Contains(e.Reference));
}
}
// 2. Resolve subtag names with safe fallbacks.
string active = string.IsNullOrEmpty(options.Fallback.Subtags.Active)
? DefaultActiveSubtag
: options.Fallback.Subtags.Active;
string acked = string.IsNullOrEmpty(options.Fallback.Subtags.Acked)
? DefaultAckedSubtag
: options.Fallback.Subtags.Acked;
string priority = options.Fallback.Subtags.Priority;
string ackComment = options.Fallback.Subtags.AckComment;
// 3. Resolve the area: discovery area, else the default area (may be empty).
string area = string.IsNullOrEmpty(discovery.Area) ? options.DefaultArea : discovery.Area;
// 4. Compose one target per reference.
List<AlarmSubtagTarget> targets = new(ordered.Count);
foreach ((string reference, string sourceObject) in ordered)
{
targets.Add(new AlarmSubtagTarget
{
AlarmFullReference = ComposeFullReference(area, reference),
SourceObjectReference = sourceObject,
ActiveSubtag = $"{reference}.{active}",
AckedSubtag = $"{reference}.{acked}",
PrioritySubtag = string.IsNullOrEmpty(priority) ? string.Empty : $"{reference}.{priority}",
AckCommentSubtag = string.IsNullOrEmpty(ackComment) ? string.Empty : $"{reference}.{ackComment}",
});
}
// 5. Report the resolved count; warn when subtag mode was expected to cover
// something (GR enabled, or explicit includes were configured) but resolved
// to nothing. Only emit the Debug line when there is at least one target,
// to avoid a confusing "0 target(s)" noise line.
if (targets.Count == 0 && (discovery.UseGalaxyRepository || discovery.IncludeAttributes.Length > 0))
{
_logger.LogWarning(
"Alarm subtag watch-list resolved to zero targets; subtag-polling fallback will cover no alarms.");
}
else if (targets.Count > 0)
{
_logger.LogDebug("Resolved alarm subtag watch-list with {TargetCount} target(s).", targets.Count);
}
return targets;
}
/// <summary>
/// Derives the source-object reference for a configuration entry: the
/// substring before the first '.', or the whole string when there is no dot.
/// </summary>
private static string DeriveSourceObject(string reference)
{
int dot = reference.IndexOf('.', StringComparison.Ordinal);
return dot < 0 ? reference : reference[..dot];
}
/// <summary>
/// Composes the canonical alarm full reference: <c>Galaxy!{area}.{reference}</c>
/// when an area is set, otherwise <c>Galaxy!{reference}</c>.
/// </summary>
private static string ComposeFullReference(string area, string reference) =>
string.IsNullOrEmpty(area)
? $"{ProviderLiteral}!{reference}"
: $"{ProviderLiteral}!{area}.{reference}";
}
@@ -13,6 +13,7 @@ public static class AlarmsServiceCollectionExtensions
/// <returns>The service collection for chaining.</returns>
public static IServiceCollection AddGatewayAlarms(this IServiceCollection services)
{
services.AddSingleton<IAlarmWatchListResolver, AlarmWatchListResolver>();
services.AddSingleton<GatewayAlarmMonitor>();
services.AddSingleton<IGatewayAlarmService>(provider => provider.GetRequiredService<GatewayAlarmMonitor>());
services.AddHostedService(provider => provider.GetRequiredService<GatewayAlarmMonitor>());
@@ -1,7 +1,9 @@
using System.Threading.Channels;
using Google.Protobuf.WellKnownTypes;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Configuration;
using ZB.MOM.WW.MxGateway.Server.Metrics;
using ZB.MOM.WW.MxGateway.Server.Sessions;
namespace ZB.MOM.WW.MxGateway.Server.Alarms;
@@ -23,6 +25,8 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
private static readonly TimeSpan StartupGrace = TimeSpan.FromSeconds(2);
private readonly ISessionManager _sessionManager;
private readonly IAlarmWatchListResolver _watchListResolver;
private readonly GatewayMetrics _metrics;
private readonly AlarmsOptions _options;
private readonly ILogger<GatewayAlarmMonitor> _logger;
@@ -30,20 +34,34 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
private readonly Dictionary<string, ActiveAlarmSnapshot> _alarms = new(StringComparer.Ordinal);
private readonly List<Subscriber> _subscribers = [];
// Current provider status (mode + degraded + reason + since), guarded by _sync.
// Initialized to the alarm-manager, not-degraded baseline so a late joiner sees
// a sensible status even before any OnAlarmProviderModeChanged event arrives.
private AlarmProviderMode _providerMode = AlarmProviderMode.Alarmmgr;
private bool _providerDegraded;
private string _providerReason = string.Empty;
private DateTimeOffset _providerSince = DateTimeOffset.UtcNow;
private volatile GatewayAlarmMonitorState _state = GatewayAlarmMonitorState.Disabled;
private volatile string? _lastError;
private GatewaySession? _session;
/// <summary>Initializes the gateway alarm monitor.</summary>
/// <param name="sessionManager">Gateway session manager.</param>
/// <param name="watchListResolver">Resolver for the subtag-fallback watch-list.</param>
/// <param name="metrics">Gateway metrics sink.</param>
/// <param name="options">Gateway options carrying the alarm configuration.</param>
/// <param name="logger">Diagnostic logger.</param>
public GatewayAlarmMonitor(
ISessionManager sessionManager,
IAlarmWatchListResolver watchListResolver,
GatewayMetrics metrics,
IOptions<GatewayOptions> options,
ILogger<GatewayAlarmMonitor> logger)
{
_sessionManager = sessionManager ?? throw new ArgumentNullException(nameof(sessionManager));
_watchListResolver = watchListResolver ?? throw new ArgumentNullException(nameof(watchListResolver));
_metrics = metrics ?? throw new ArgumentNullException(nameof(metrics));
_options = (options ?? throw new ArgumentNullException(nameof(options))).Value.Alarms;
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
@@ -139,6 +157,20 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
private async Task RunMonitorAsync(string subscription, CancellationToken stoppingToken)
{
_state = GatewayAlarmMonitorState.Starting;
lock (_sync)
{
// Re-baseline the provider status for this lifecycle so a restarted
// monitor advertises alarm-manager/not-degraded until told otherwise.
_providerMode = AlarmProviderMode.Alarmmgr;
_providerDegraded = false;
_providerReason = string.Empty;
_providerSince = DateTimeOffset.UtcNow;
}
// Align the observable gauge with the Alarmmgr baseline without recording
// a switch — the gauge was 0 (unknown) from construction until now.
_metrics.SetAlarmProviderMode(ModeToInt(AlarmProviderMode.Alarmmgr));
GatewaySession session = await _sessionManager.OpenSessionAsync(
new SessionOpenRequest(BackendName, MonitorClientName, Guid.NewGuid().ToString("N"), CommandTimeout: null),
MonitorClientName,
@@ -173,6 +205,15 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
{
ApplyTransition(mxEvent.OnAlarmTransition);
}
else if (mxEvent is { BodyCase: MxEvent.BodyOneofCase.OnAlarmProviderModeChanged }
&& mxEvent.OnAlarmProviderModeChanged is not null)
{
await ApplyProviderModeChangeAsync(
session.SessionId,
mxEvent.OnAlarmProviderModeChanged,
linked.Token)
.ConfigureAwait(false);
}
}
}
finally
@@ -209,6 +250,29 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
private async Task SubscribeAlarmsAsync(string sessionId, string subscription, CancellationToken cancellationToken)
{
IReadOnlyList<AlarmSubtagTarget> watchList = await _watchListResolver
.ResolveAsync(_options, cancellationToken)
.ConfigureAwait(false);
AlarmProviderMode forcedMode = MapForcedMode(_options.Fallback.Mode);
// When the forced mode is Unspecified (the "Auto" case) and the resolved
// watch-list is empty — the common alarmmgr-only deployment — the command
// is identical-in-effect to the historical SubscribeAlarms (wnwrap only):
// the worker builds the wnwrap consumer and no subtag watch-list.
SubscribeAlarmsCommand command = new()
{
SubscriptionExpression = subscription,
ForcedMode = forcedMode,
Failover = new AlarmFailoverConfig
{
ConsecutiveFailureThreshold = _options.Fallback.ConsecutiveFailureThreshold,
FailbackProbeIntervalSeconds = _options.Fallback.FailbackProbeIntervalSeconds,
FailbackStableProbes = _options.Fallback.FailbackStableProbes,
},
};
command.WatchList.AddRange(watchList);
WorkerCommandReply reply = await _sessionManager.InvokeAsync(
sessionId,
new WorkerCommand
@@ -216,7 +280,7 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
Command = new MxCommand
{
Kind = MxCommandKind.SubscribeAlarms,
SubscribeAlarms = new SubscribeAlarmsCommand { SubscriptionExpression = subscription },
SubscribeAlarms = command,
},
},
cancellationToken)
@@ -310,6 +374,98 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
}
}
// Handles the worker's provider-mode-change event: updates the stored provider
// status, broadcasts it to every subscriber (provider status is global, not
// alarm-scoped), records the switch metric, and forces a cache reconcile so the
// active-alarm set reflects whatever the new mode reports.
private async Task ApplyProviderModeChangeAsync(
string sessionId,
OnAlarmProviderModeChangedEvent change,
CancellationToken cancellationToken)
{
AlarmProviderMode toMode = change.Mode;
string reason = change.Reason ?? string.Empty;
AlarmProviderStatus status;
int fromModeInt;
lock (_sync)
{
fromModeInt = ModeToInt(_providerMode);
_providerMode = toMode;
_providerDegraded = toMode == AlarmProviderMode.Subtag;
_providerReason = reason;
_providerSince = DateTimeOffset.UtcNow;
status = BuildProviderStatus();
BroadcastToAll(new AlarmFeedMessage { ProviderStatus = status });
}
_metrics.AlarmProviderSwitched(fromModeInt, ModeToInt(toMode), reason);
_logger.LogInformation(
"Alarm provider mode changed to {Mode} (degraded={Degraded}): {Reason}",
toMode,
status.Degraded,
reason);
try
{
// Intentionally awaited OUTSIDE _sync: ReconcileAsync acquires _sync itself,
// so holding it across the await here would deadlock. Subscribers therefore
// see the ProviderStatus push (above) slightly before the cache is re-seeded
// by the reconcile — an accepted brief inconsistency.
await ReconcileAsync(sessionId, cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException)
{
throw;
}
catch (Exception exception)
{
_logger.LogDebug(
exception,
"Reconcile after alarm provider mode change failed; keeping the current cache.");
}
}
// Caller holds _sync. Builds an AlarmProviderStatus snapshot of the current state.
private AlarmProviderStatus BuildProviderStatus()
{
return new AlarmProviderStatus
{
Mode = _providerMode,
Degraded = _providerDegraded,
Reason = _providerReason,
Since = Timestamp.FromDateTimeOffset(_providerSince),
};
}
// Maps the configured fallback mode string to the forced provider mode the
// worker honours. Case-insensitive; anything other than the two force values
// (including the default "Auto") yields Unspecified ("let the worker decide").
private static AlarmProviderMode MapForcedMode(string? mode)
{
if (string.Equals(mode, "ForceAlarmManager", StringComparison.OrdinalIgnoreCase))
{
return AlarmProviderMode.Alarmmgr;
}
if (string.Equals(mode, "ForceSubtag", StringComparison.OrdinalIgnoreCase))
{
return AlarmProviderMode.Subtag;
}
return AlarmProviderMode.Unspecified;
}
// Maps the provider-mode enum to the integer the metric expects
// (alarmmgr=1, subtag=2, unknown/unspecified=0).
private static int ModeToInt(AlarmProviderMode mode) => mode switch
{
AlarmProviderMode.Alarmmgr => 1,
AlarmProviderMode.Subtag => 2,
_ => 0,
};
// Replaces the cache with the worker's authoritative snapshot, broadcasting
// a synthetic transition for any alarm the live stream missed.
private void ApplyReconcile(IEnumerable<ActiveAlarmSnapshot> snapshots)
@@ -374,6 +530,23 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
}
}
// Caller holds _sync. Pushes a feed message to every subscriber regardless of
// its alarm-filter prefix. Used for provider-status messages, which are global
// rather than scoped to a single alarm reference.
private void BroadcastToAll(AlarmFeedMessage message)
{
for (int index = _subscribers.Count - 1; index >= 0; index--)
{
Subscriber subscriber = _subscribers[index];
if (!subscriber.Channel.Writer.TryWrite(message))
{
subscriber.Channel.Writer.TryComplete(new InvalidOperationException(
"Alarm feed subscriber fell behind and was dropped; reconnect to re-snapshot."));
_subscribers.RemoveAt(index);
}
}
}
private void ClearCache()
{
lock (_sync)
@@ -398,11 +571,14 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
Subscriber subscriber = new(channel, prefix);
ActiveAlarmSnapshot[] snapshot;
AlarmProviderStatus providerStatus;
lock (_sync)
{
// Register before snapshotting under the same lock so no transition
// can slip between the snapshot and the live stream.
// Register before snapshotting under the same lock so neither a
// transition nor a provider-mode change can slip between the snapshot
// and the live stream.
_subscribers.Add(subscriber);
providerStatus = BuildProviderStatus();
snapshot = _alarms.Values
.Where(alarm => prefix.Length == 0
|| alarm.AlarmFullReference.StartsWith(prefix, StringComparison.Ordinal))
@@ -412,6 +588,10 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
try
{
// Emit the current provider status first so a late joiner immediately
// learns the mode (and whether the feed is degraded) before any alarms.
yield return new AlarmFeedMessage { ProviderStatus = providerStatus };
foreach (ActiveAlarmSnapshot alarm in snapshot)
{
yield return new AlarmFeedMessage { ActiveAlarm = alarm };
@@ -624,6 +804,8 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
Description = transition.Description,
OperatorUser = transition.OperatorUser,
OperatorComment = transition.OperatorComment,
Degraded = transition.Degraded,
SourceProvider = transition.SourceProvider,
};
if (transition.OriginalRaiseTimestamp is not null)
{
@@ -660,6 +842,8 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
Description = snapshot.Description,
OperatorUser = snapshot.OperatorUser,
OperatorComment = snapshot.OperatorComment,
Degraded = snapshot.Degraded,
SourceProvider = snapshot.SourceProvider,
};
if (snapshot.OriginalRaiseTimestamp is not null)
{
@@ -0,0 +1,28 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Configuration;
namespace ZB.MOM.WW.MxGateway.Server.Alarms;
/// <summary>
/// Resolves the subtag watch-list the gateway sends to the worker when the
/// central alarm monitor operates in subtag-polling fallback mode. Merges
/// Galaxy Repository alarm-attribute discovery with the configured
/// include/exclude overrides and composes the per-attribute subtag item
/// addresses from the configured subtag names.
/// </summary>
public interface IAlarmWatchListResolver
{
/// <summary>
/// Builds the subtag watch-list for the supplied alarm configuration.
/// </summary>
/// <param name="options">Alarm configuration carrying discovery and subtag-name settings.</param>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
/// <returns>
/// The resolved <see cref="AlarmSubtagTarget"/> watch-list, possibly empty.
/// Discovery being unavailable never throws; the caller decides what to do
/// with an empty list.
/// </returns>
Task<IReadOnlyList<AlarmSubtagTarget>> ResolveAsync(
AlarmsOptions options,
CancellationToken cancellationToken = default);
}
@@ -0,0 +1,125 @@
namespace ZB.MOM.WW.MxGateway.Server.Configuration;
/// <summary>
/// Controls how the central alarm monitor selects between the MXAccess
/// alarm-manager subscription and the subtag-polling fallback, and
/// governs the failure-detection thresholds used when switching.
/// </summary>
public sealed class AlarmFallbackOptions
{
/// <summary>
/// Selects the operating mode for the alarm-manager ↔ subtag fallback
/// mechanism. Accepted values (case-insensitive):
/// <list type="bullet">
/// <item><c>Auto</c> — use the alarm manager; switch to subtag polling
/// automatically when <see cref="ConsecutiveFailureThreshold"/> failures
/// are detected, and probe for failback.</item>
/// <item><c>ForceAlarmManager</c> — always use the alarm manager;
/// never fall back.</item>
/// <item><c>ForceSubtag</c> — always use subtag polling;
/// never try the alarm manager.</item>
/// </list>
/// Default is <c>Auto</c>.
/// </summary>
public string Mode { get; init; } = "Auto";
/// <summary>
/// Number of consecutive alarm-manager failures before the monitor
/// switches to subtag-polling fallback. Must be at least 1. Default 3.
/// </summary>
public int ConsecutiveFailureThreshold { get; init; } = 3;
/// <summary>
/// How often (in seconds) the monitor sends a probe to the alarm manager
/// while operating in subtag-polling fallback mode, to detect recovery.
/// Must be at least 1. Default 30.
/// </summary>
public int FailbackProbeIntervalSeconds { get; init; } = 30;
/// <summary>
/// Number of consecutive successful probes required before the monitor
/// considers the alarm manager recovered and switches back. Must be at
/// least 1. Default 3.
/// </summary>
public int FailbackStableProbes { get; init; } = 3;
/// <summary>
/// Controls how the monitor discovers the set of objects to poll when
/// operating in subtag-polling fallback mode.
/// </summary>
public AlarmDiscoveryOptions Discovery { get; init; } = new();
/// <summary>
/// Configures the subtag names the monitor reads when polling alarm state
/// in subtag-fallback mode.
/// </summary>
public AlarmSubtagNameOptions Subtags { get; init; } = new();
}
/// <summary>
/// Governs how the alarm monitor discovers objects to include in subtag-polling
/// fallback mode. Either the Galaxy Repository query (when
/// <see cref="UseGalaxyRepository"/> is <c>true</c>) or an explicit
/// <see cref="IncludeAttributes"/> list must be supplied when
/// <c>MxGateway:Alarms:Fallback:Mode</c> is <c>ForceSubtag</c>.
/// </summary>
public sealed class AlarmDiscoveryOptions
{
/// <summary>
/// When <c>true</c> the monitor queries the Galaxy Repository SQL database
/// to enumerate alarm objects for the configured area. Default <c>true</c>.
/// </summary>
public bool UseGalaxyRepository { get; init; } = true;
/// <summary>
/// Galaxy area to scope the Repository query to. When empty the monitor
/// falls back to <see cref="AlarmsOptions.DefaultArea"/>. Ignored when
/// <see cref="UseGalaxyRepository"/> is <c>false</c>.
/// </summary>
public string Area { get; init; } = string.Empty;
/// <summary>
/// Explicit list of MXAccess attribute paths to include in subtag polling,
/// supplementing (or replacing, when <see cref="UseGalaxyRepository"/> is
/// <c>false</c>) the Repository-derived list. Default empty.
/// </summary>
public string[] IncludeAttributes { get; init; } = Array.Empty<string>();
/// <summary>
/// Attribute paths to exclude from the Repository-derived poll list.
/// Ignored when <see cref="UseGalaxyRepository"/> is <c>false</c>.
/// Default empty.
/// </summary>
public string[] ExcludeAttributes { get; init; } = Array.Empty<string>();
}
/// <summary>
/// Configures the subtag names read by the alarm monitor when it is operating
/// in subtag-polling fallback mode. Names are matched against MXAccess item
/// handles; validation against the live MXAccess attribute list occurs at
/// runtime, not at startup.
/// </summary>
public sealed class AlarmSubtagNameOptions
{
/// <summary>
/// Subtag name for the active-alarm flag. Default <c>active</c>.
/// </summary>
public string Active { get; init; } = "active";
/// <summary>
/// Subtag name for the acknowledged flag. Default <c>acked</c>.
/// </summary>
public string Acked { get; init; } = "acked";
/// <summary>
/// Optional subtag name for the acknowledgement comment field.
/// When empty the feature is disabled. Verified against MXAccess at
/// runtime before use. Default empty.
/// </summary>
public string AckComment { get; init; } = string.Empty;
/// <summary>
/// Subtag name for the alarm priority. Default <c>priority</c>.
/// </summary>
public string Priority { get; init; } = "priority";
}
@@ -45,4 +45,12 @@ public sealed class AlarmsOptions
/// the monitor floors it at 5 seconds.
/// </summary>
public int ReconcileIntervalSeconds { get; init; } = 30;
/// <summary>
/// Configuration for the alarm-manager ↔ subtag fallback mechanism:
/// operating mode, failure-detection thresholds, discovery, and subtag
/// names. Defaults (Mode = "Auto") preserve behaviour when the section is
/// omitted from configuration.
/// </summary>
public AlarmFallbackOptions Fallback { get; init; } = new();
}
@@ -231,6 +231,8 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
builder);
}
private static readonly string[] ValidAlarmFallbackModes = ["Auto", "ForceAlarmManager", "ForceSubtag"];
private static void ValidateAlarms(AlarmsOptions options, ValidationBuilder builder)
{
if (!options.Enabled)
@@ -255,6 +257,46 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
builder.Add(
@"MxGateway:Alarms:SubscriptionExpression must start with '\\' (canonical \\<host>\Galaxy!<area> shape).");
}
ValidateAlarmFallback(options.Fallback, builder);
}
private static void ValidateAlarmFallback(AlarmFallbackOptions fallback, ValidationBuilder builder)
{
// Validate Mode is one of the recognised values (case-insensitive).
bool modeValid = Array.Exists(
ValidAlarmFallbackModes,
m => string.Equals(m, fallback.Mode, StringComparison.OrdinalIgnoreCase));
if (!modeValid)
{
builder.Add(
$"MxGateway:Alarms:Fallback:Mode must be one of: {string.Join(", ", ValidAlarmFallbackModes)} (was '{fallback.Mode}').");
}
// ForceSubtag requires either Galaxy Repository discovery or an explicit IncludeAttributes list.
if (modeValid
&& string.Equals(fallback.Mode, "ForceSubtag", StringComparison.OrdinalIgnoreCase)
&& !fallback.Discovery.UseGalaxyRepository
&& fallback.Discovery.IncludeAttributes.Length == 0)
{
builder.Add(
"MxGateway:Alarms:Fallback ForceSubtag requires Galaxy Repository discovery or a non-empty Discovery:IncludeAttributes list.");
}
// Floor validation: numeric thresholds must be at least 1.
AddIfNotPositive(
fallback.ConsecutiveFailureThreshold,
"MxGateway:Alarms:Fallback:ConsecutiveFailureThreshold must be greater than zero.",
builder);
AddIfNotPositive(
fallback.FailbackProbeIntervalSeconds,
"MxGateway:Alarms:Fallback:FailbackProbeIntervalSeconds must be greater than zero.",
builder);
AddIfNotPositive(
fallback.FailbackStableProbes,
"MxGateway:Alarms:Fallback:FailbackStableProbes must be greater than zero.",
builder);
}
private const int MinimumCertValidityYears = 1;
@@ -1,7 +1,10 @@
@page "/alarms"
@implements IAsyncDisposable
@using Microsoft.AspNetCore.SignalR.Client
@using ZB.MOM.WW.MxGateway.Server.Dashboard.Hubs
@inject IDashboardLiveDataService LiveData
@inject IOptions<GatewayOptions> GatewayOptions
@inject DashboardHubConnectionFactory HubFactory
<PageTitle>Dashboard Alarms</PageTitle>
@@ -10,6 +13,12 @@
<h1>Alarms</h1>
<div class="text-secondary">@HeaderLine()</div>
</div>
<div class="d-flex align-items-center gap-2">
<span class="badge @_providerStatus.BadgeCssClass"
title="@ProviderStatusTitle()">
@_providerStatus.Label
</span>
</div>
</div>
@if (!GatewayOptions.Value.Alarms.Enabled)
@@ -163,10 +172,44 @@
private readonly CancellationTokenSource _cts = new();
private Task? _pollTask;
private DashboardAlarmProviderStatus _providerStatus = DashboardAlarmProviderStatus.Healthy;
private HubConnection? _alarmsHub;
/// <inheritdoc />
protected override void OnInitialized()
{
_pollTask = PollLoopAsync();
_ = AttachAlarmsHubAsync();
}
private string? ProviderStatusTitle()
{
return _providerStatus.IsDegraded && !string.IsNullOrWhiteSpace(_providerStatus.Reason)
? _providerStatus.Reason
: null;
}
private async Task AttachAlarmsHubAsync()
{
_alarmsHub = HubFactory.Create("/hubs/alarms");
_alarmsHub.On<AlarmFeedMessage>(AlarmsHub.AlarmMessage, async message =>
{
if (message.PayloadCase == AlarmFeedMessage.PayloadOneofCase.ProviderStatus)
{
_providerStatus = DashboardAlarmProviderStatus.FromFeed(message);
await InvokeAsync(StateHasChanged).ConfigureAwait(false);
}
});
try
{
await _alarmsHub.StartAsync(_cts.Token).ConfigureAwait(false);
}
catch
{
// The badge is best-effort; it stays at the healthy default until
// the hub reconnects and delivers a fresh provider-status message.
}
}
private string HeaderLine()
@@ -268,6 +311,19 @@
public async ValueTask DisposeAsync()
{
await _cts.CancelAsync();
if (_alarmsHub is not null)
{
try
{
await _alarmsHub.DisposeAsync();
}
catch
{
// Disposal-time errors are best-effort.
}
}
if (_pollTask is not null)
{
try
@@ -0,0 +1,78 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
/// <summary>
/// Dashboard projection of an <see cref="AlarmProviderStatus" /> message
/// carried on the alarm feed. Maps the protobuf provider mode / degraded
/// flag into Bootstrap-only display fields so the Alarms page can render a
/// status badge without touching protobuf types.
/// </summary>
public sealed record DashboardAlarmProviderStatus(
AlarmProviderMode Mode,
bool IsDegraded,
string Label,
string BadgeCssClass,
string Reason,
DateTimeOffset? SinceUtc)
{
/// <summary>Badge label shown when the alarm-manager provider is healthy.</summary>
public const string AlarmManagerLabel = "Alarm Manager";
/// <summary>Badge label shown when the feed has fallen back to subtag monitoring.</summary>
public const string DegradedLabel = "Subtag monitoring (degraded)";
private const string HealthyBadge = "bg-success";
private const string DegradedBadge = "bg-warning text-dark";
/// <summary>
/// The default status assumed before the first provider-status message
/// arrives: healthy alarm-manager mode.
/// </summary>
public static DashboardAlarmProviderStatus Healthy { get; } = new(
Mode: AlarmProviderMode.Alarmmgr,
IsDegraded: false,
Label: AlarmManagerLabel,
BadgeCssClass: HealthyBadge,
Reason: string.Empty,
SinceUtc: null);
/// <summary>Projects an alarm-feed provider-status payload into a dashboard badge model.</summary>
/// <param name="status">The provider-status payload from an <see cref="AlarmFeedMessage" />.</param>
/// <returns>The projected dashboard status.</returns>
public static DashboardAlarmProviderStatus FromProviderStatus(AlarmProviderStatus status)
{
ArgumentNullException.ThrowIfNull(status);
// Treat the explicit degraded flag and the SUBTAG mode as equivalent;
// the contract sets degraded=true whenever mode == SUBTAG, but guard
// against either being set independently.
bool degraded = status.Degraded || status.Mode == AlarmProviderMode.Subtag;
return new DashboardAlarmProviderStatus(
Mode: status.Mode,
IsDegraded: degraded,
Label: degraded ? DegradedLabel : AlarmManagerLabel,
BadgeCssClass: degraded ? DegradedBadge : HealthyBadge,
Reason: status.Reason ?? string.Empty,
SinceUtc: status.Since?.ToDateTimeOffset());
}
/// <summary>Projects an alarm-feed message into a dashboard badge model.</summary>
/// <param name="message">An alarm-feed message whose payload is a provider status.</param>
/// <returns>The projected dashboard status.</returns>
/// <exception cref="ArgumentException">The message does not carry a provider-status payload.</exception>
public static DashboardAlarmProviderStatus FromFeed(AlarmFeedMessage message)
{
ArgumentNullException.ThrowIfNull(message);
if (message.PayloadCase != AlarmFeedMessage.PayloadOneofCase.ProviderStatus)
{
throw new ArgumentException(
"Alarm-feed message does not carry a provider-status payload.",
nameof(message));
}
return FromProviderStatus(message.ProviderStatus);
}
}
@@ -0,0 +1,36 @@
namespace ZB.MOM.WW.MxGateway.Server.Galaxy;
/// <summary>
/// One alarm-bearing attribute discovered by
/// <see cref="GalaxyRepository.GetAlarmAttributesAsync"/>: an attribute whose owning
/// object configures an <c>AlarmExtension</c> primitive (the same
/// <c>is_alarm</c> detection used by <see cref="GalaxyRepository.GetAttributesAsync"/>).
/// Used to build the subtag-fallback watch-list.
/// </summary>
public sealed class GalaxyAlarmAttributeRow
{
/// <summary>
/// Gets the alarm-bearing attribute reference (e.g. <c>Tank01.Level.HiHi</c>),
/// matching the <c>full_tag_reference</c> projection of
/// <see cref="GalaxyRepository.GetAttributesAsync"/>.
/// </summary>
public string FullTagReference { get; init; } = string.Empty;
/// <summary>
/// Gets the owning object reference (e.g. <c>Tank01</c>). This is the Galaxy
/// <c>tag_name</c> — the segment that precedes the first attribute dot in
/// <see cref="FullTagReference"/>.
/// </summary>
public string SourceObjectReference { get; init; } = string.Empty;
/// <summary>
/// Gets the writable ack-comment attribute address.
/// <para>
/// The Galaxy Repository schema does not expose an ack-comment subtag address
/// directly, so this is always <see cref="string.Empty"/> here. The watch-list
/// resolver (a later task) composes the concrete address from configuration plus
/// <see cref="SourceObjectReference"/> / <see cref="FullTagReference"/>.
/// </para>
/// </summary>
public string AckCommentSubtag { get; init; } = string.Empty;
}
@@ -114,6 +114,56 @@ public sealed class GalaxyRepository(GalaxyRepositoryOptions options) : IGalaxyR
return rows;
}
/// <summary>
/// Retrieves only the alarm-bearing attributes for the subtag-fallback watch-list.
/// Alarm detection is identical to <see cref="GetAttributesAsync"/>: a row is
/// alarm-bearing when its owning object configures an <c>AlarmExtension</c>
/// primitive (the same <c>is_alarm</c> projection, here applied as a SQL filter).
/// </summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
public async Task<List<GalaxyAlarmAttributeRow>> GetAlarmAttributesAsync(CancellationToken ct = default)
{
List<GalaxyAlarmAttributeRow> rows = new();
using SqlConnection conn = new(options.ConnectionString);
await conn.OpenAsync(ct).ConfigureAwait(false);
using SqlCommand cmd = new(AlarmAttributesSql, conn) { CommandTimeout = options.CommandTimeoutSeconds };
using SqlDataReader reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
rows.Add(MapAlarmRow(
fullTagReference: reader.GetString(0),
sourceObjectReference: reader.GetString(1)));
}
return rows;
}
/// <summary>
/// Maps a raw alarm-attribute reader row to a <see cref="GalaxyAlarmAttributeRow"/>.
/// <para>
/// <paramref name="sourceObjectReference"/> is the Galaxy <c>tag_name</c> (the
/// owning object), and <paramref name="fullTagReference"/> is
/// <c>tag_name + '.' + attribute_name</c> — the same composition the
/// <c>full_tag_reference</c> projection of <see cref="AttributesSql"/> produces.
/// <see cref="GalaxyAlarmAttributeRow.AckCommentSubtag"/> is left empty here; the
/// schema does not expose an ack-comment address and the watch-list resolver
/// composes it later.
/// </para>
/// Exposed internally so the derivation can be unit-tested without a database.
/// </summary>
/// <param name="fullTagReference">The alarm-bearing attribute reference.</param>
/// <param name="sourceObjectReference">The owning object reference (tag name).</param>
internal static GalaxyAlarmAttributeRow MapAlarmRow(
string fullTagReference,
string sourceObjectReference) => new()
{
FullTagReference = fullTagReference,
SourceObjectReference = sourceObjectReference,
AckCommentSubtag = string.Empty,
};
private const string HierarchySql = @"
;WITH template_chain AS (
SELECT g.gobject_id AS instance_gobject_id, t.gobject_id AS template_gobject_id,
@@ -248,5 +298,71 @@ SELECT
FROM ranked r
LEFT JOIN data_type dt ON dt.mx_data_type = r.mx_data_type
WHERE r.rn = 1
ORDER BY r.tag_name, r.attribute_name";
// Alarm-only discovery for the subtag-fallback watch-list. This deliberately reuses the
// exact candidate/ranked CTE structure and the same `AlarmExtension`-based is_alarm
// detection as AttributesSql so the two queries cannot drift: a row qualifies only when
// its user attribute (src_pri 0) anchors an `AlarmExtension` primitive on the owning
// object. It projects just what the watch-list needs — full_tag_reference (tag_name +
// '.' + attribute_name, matching AttributesSql) and the owning object's tag_name as
// source_object_reference. The array `[]` suffix is intentionally omitted: an
// alarm-bearing attribute is a scalar anchor, not an array body.
private const string AlarmAttributesSql = @"
;WITH deployed_package_chain AS (
SELECT g.gobject_id, p.package_id, p.derived_from_package_id, 0 AS depth
FROM gobject g
INNER JOIN package p ON p.package_id = g.deployed_package_id
WHERE g.is_template = 0 AND g.deployed_package_id <> 0
UNION ALL
SELECT dpc.gobject_id, p.package_id, p.derived_from_package_id, dpc.depth + 1
FROM deployed_package_chain dpc
INNER JOIN package p ON p.package_id = dpc.derived_from_package_id
WHERE dpc.derived_from_package_id <> 0 AND dpc.depth < 10
),
candidate AS (
SELECT
dpc.gobject_id, g.tag_name, da.attribute_name, dpc.depth, 0 AS src_pri
FROM deployed_package_chain dpc
INNER JOIN dynamic_attribute da ON da.package_id = dpc.package_id
INNER JOIN gobject g ON g.gobject_id = dpc.gobject_id
INNER JOIN template_definition td ON td.template_definition_id = g.template_definition_id
WHERE td.category_id IN (1, 3, 4, 10, 11, 13, 17, 24, 26)
AND da.attribute_name NOT LIKE '[_]%'
AND da.attribute_name NOT LIKE '%.Description'
AND da.mx_attribute_category IN (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 24)
UNION ALL
SELECT
dpc.gobject_id, g.tag_name,
CASE WHEN pi.primitive_name IS NULL OR pi.primitive_name = ''
THEN ad.attribute_name
ELSE pi.primitive_name + '.' + ad.attribute_name END AS attribute_name,
dpc.depth, 1 AS src_pri
FROM deployed_package_chain dpc
INNER JOIN primitive_instance pi ON pi.package_id = dpc.package_id
INNER JOIN attribute_definition ad ON ad.primitive_definition_id = pi.primitive_definition_id
INNER JOIN gobject g ON g.gobject_id = dpc.gobject_id
INNER JOIN template_definition td ON td.template_definition_id = g.template_definition_id
WHERE td.category_id IN (1, 3, 4, 10, 11, 13, 17, 24, 26)
AND ad.attribute_name NOT LIKE '[_]%'
AND ad.attribute_name NOT LIKE '%.Description'
),
ranked AS (
SELECT c.*, ROW_NUMBER() OVER (
PARTITION BY c.gobject_id, c.attribute_name ORDER BY c.src_pri, c.depth) AS rn
FROM candidate c
)
SELECT
r.tag_name + '.' + r.attribute_name AS full_tag_reference,
r.tag_name AS source_object_reference
FROM ranked r
WHERE r.rn = 1
AND r.src_pri = 0
AND EXISTS (
SELECT 1 FROM deployed_package_chain dpc2
INNER JOIN primitive_instance pi ON pi.package_id = dpc2.package_id AND pi.primitive_name = r.attribute_name
INNER JOIN primitive_definition pd ON pd.primitive_definition_id = pi.primitive_definition_id AND pd.primitive_name = 'AlarmExtension'
WHERE dpc2.gobject_id = r.gobject_id
)
ORDER BY r.tag_name, r.attribute_name";
}
@@ -27,4 +27,12 @@ public interface IGalaxyRepository
/// <summary>Retrieves all attributes for Galaxy objects from the repository.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default);
/// <summary>
/// Retrieves only the alarm-bearing attributes (those whose owning object
/// configures an <c>AlarmExtension</c> primitive) for building the
/// subtag-fallback watch-list.
/// </summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
Task<List<GalaxyAlarmAttributeRow>> GetAlarmAttributesAsync(CancellationToken ct = default);
}
@@ -1,5 +1,6 @@
using System.Collections.Concurrent;
using System.Diagnostics.Metrics;
using System.Globalization;
namespace ZB.MOM.WW.MxGateway.Server.Metrics;
@@ -22,6 +23,7 @@ public sealed class GatewayMetrics : IDisposable
private readonly Counter<long> _heartbeatFailuresCounter;
private readonly Counter<long> _streamDisconnectsCounter;
private readonly Counter<long> _retryAttemptsCounter;
private readonly Counter<long> _alarmProviderSwitchesCounter;
private readonly Histogram<double> _workerStartupLatencyHistogram;
private readonly Histogram<double> _commandLatencyHistogram;
private readonly Histogram<double> _eventStreamSendLatencyHistogram;
@@ -34,6 +36,7 @@ public sealed class GatewayMetrics : IDisposable
private int _workersRunning;
private int _workerEventQueueDepth;
private int _grpcEventStreamQueueDepth;
private int _alarmProviderMode;
private long _sessionsOpened;
private long _sessionsClosed;
private long _commandsStarted;
@@ -68,6 +71,7 @@ public sealed class GatewayMetrics : IDisposable
_heartbeatFailuresCounter = _meter.CreateCounter<long>("mxgateway.heartbeats.failed");
_streamDisconnectsCounter = _meter.CreateCounter<long>("mxgateway.grpc.streams.disconnected");
_retryAttemptsCounter = _meter.CreateCounter<long>("mxgateway.retries.attempted");
_alarmProviderSwitchesCounter = _meter.CreateCounter<long>("mxgateway.alarms.provider_switches");
_workerStartupLatencyHistogram = _meter.CreateHistogram<double>("mxgateway.workers.startup.duration", "s");
_commandLatencyHistogram = _meter.CreateHistogram<double>("mxgateway.commands.duration", "s");
_eventStreamSendLatencyHistogram = _meter.CreateHistogram<double>("mxgateway.events.stream_send.duration", "s");
@@ -76,6 +80,7 @@ public sealed class GatewayMetrics : IDisposable
_meter.CreateObservableGauge("mxgateway.workers.running", GetWorkersRunning);
_meter.CreateObservableGauge("mxgateway.events.worker_queue.depth", GetWorkerEventQueueDepth);
_meter.CreateObservableGauge("mxgateway.events.grpc_stream_queue.depth", GetGrpcEventStreamQueueDepth);
_meter.CreateObservableGauge("mxgateway.alarms.provider_mode", GetAlarmProviderMode);
}
/// <summary>
@@ -377,6 +382,32 @@ public sealed class GatewayMetrics : IDisposable
_retryAttemptsCounter.Add(1, new KeyValuePair<string, object?>("area", area));
}
/// <summary>
/// Records that the alarm provider switched modes and updates the current provider mode gauge.
/// </summary>
/// <param name="fromMode">Provider mode before the switch (1=alarmmgr, 2=subtag, 0=unknown).</param>
/// <param name="toMode">Provider mode after the switch (1=alarmmgr, 2=subtag, 0=unknown).</param>
/// <param name="reason">Human-readable reason for the switch.</param>
public void AlarmProviderSwitched(int fromMode, int toMode, string reason)
{
lock (_syncRoot)
{
_alarmProviderMode = toMode;
}
_alarmProviderSwitchesCounter.Add(
1,
new KeyValuePair<string, object?>("from", fromMode.ToString(CultureInfo.InvariantCulture)),
new KeyValuePair<string, object?>("to", toMode.ToString(CultureInfo.InvariantCulture)),
new KeyValuePair<string, object?>("reason", reason));
}
/// <summary>Sets the current alarm provider-mode gauge without recording a switch (e.g. startup baseline).</summary>
public void SetAlarmProviderMode(int mode)
{
lock (_syncRoot) { _alarmProviderMode = mode; }
}
/// <summary>
/// Returns a snapshot of all current metric values.
/// </summary>
@@ -455,6 +486,14 @@ public sealed class GatewayMetrics : IDisposable
}
}
private int GetAlarmProviderMode()
{
lock (_syncRoot)
{
return _alarmProviderMode;
}
}
private static void Increment(Dictionary<string, long> values, string key)
{
values.TryGetValue(key, out long currentValue);
@@ -0,0 +1,476 @@
using System.Diagnostics.CodeAnalysis;
using System.Runtime.CompilerServices;
using System.Threading.Channels;
using Google.Protobuf.WellKnownTypes;
using Microsoft.Extensions.Logging.Abstractions;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Alarms;
using ZB.MOM.WW.MxGateway.Server.Configuration;
using ZB.MOM.WW.MxGateway.Server.Metrics;
using ZB.MOM.WW.MxGateway.Server.Sessions;
namespace ZB.MOM.WW.MxGateway.Tests.Alarms;
/// <summary>
/// Drives a single <see cref="GatewayAlarmMonitor"/> instance through the full
/// alarm-provider failover/failback lifecycle and asserts the live feed reflects
/// each stage. This complements the per-aspect tests in
/// <c>GatewayAlarmMonitorProviderModeTests</c> with one cohesive scenario:
/// subscribe (watch-list + forced mode) -> baseline alarmmgr status -> alarmmgr
/// transition -> failover to subtag (degraded) -> subtag transition -> failback
/// to alarmmgr (recovered).
/// </summary>
/// <remarks>
/// The minimal session-manager / watch-list-resolver harness here is replicated
/// (not shared) from the sibling <c>GatewayAlarmMonitorProviderModeTests</c>. The
/// sibling's harness is a private nested type, and the task forbids changing that
/// test's behaviour; replicating the few members this scenario needs keeps the
/// sibling completely untouched and this file self-contained, at the cost of a
/// small amount of duplication.
/// </remarks>
public sealed class AlarmFailoverEndToEndTests
{
private static readonly TimeSpan WaitTimeout = TimeSpan.FromSeconds(30);
[Fact]
public async Task ProviderFailoverAndFailback_FullLifecycle_ReflectedInFeed()
{
using GatewayMetrics metrics = new();
FakeSessionManager sessions = new();
// Watch-list-bearing, Auto config: a non-empty resolved watch-list with the
// default ("Auto" -> Unspecified) forced mode, so step 1 can assert both the
// resolved watch-list and the forced mode/failover the SubscribeAlarms carries.
StubWatchListResolver resolver = new(
[
new AlarmSubtagTarget { AlarmFullReference = "Galaxy!Area.Tank01.Hi", ActiveSubtag = "Tank01.Hi.active" },
new AlarmSubtagTarget { AlarmFullReference = "Galaxy!Area.Tank02.Lo", ActiveSubtag = "Tank02.Lo.active" },
]);
AlarmsOptions options = new()
{
Enabled = true,
SubscriptionExpression = @"\\NODE\Galaxy!Area",
Fallback = new AlarmFallbackOptions
{
Mode = "Auto",
ConsecutiveFailureThreshold = 3,
FailbackProbeIntervalSeconds = 9,
FailbackStableProbes = 2,
},
};
using GatewayAlarmMonitor monitor = new(
sessions,
resolver,
metrics,
Microsoft.Extensions.Options.Options.Create(new GatewayOptions { Alarms = options }),
NullLogger<GatewayAlarmMonitor>.Instance);
using CancellationTokenSource cts = new();
await monitor.StartAsync(cts.Token);
await sessions.WaitForSubscribeAsync(WaitTimeout);
// --- Step 1: SubscribeAlarms carries the resolved watch-list + forced mode/failover. ---
SubscribeAlarmsCommand sent = Assert.IsType<SubscribeAlarmsCommand>(sessions.LastSubscribeCommand);
Assert.Equal(AlarmProviderMode.Unspecified, sent.ForcedMode); // "Auto"
Assert.Equal(3, sent.Failover.ConsecutiveFailureThreshold);
Assert.Equal(9, sent.Failover.FailbackProbeIntervalSeconds);
Assert.Equal(2, sent.Failover.FailbackStableProbes);
Assert.NotEmpty(sent.WatchList);
Assert.Equal(2, sent.WatchList.Count);
Assert.Contains(sent.WatchList, t => t.AlarmFullReference == "Galaxy!Area.Tank01.Hi");
// Live feed reader collecting every message in order.
List<AlarmFeedMessage> received = [];
TaskCompletionSource baselineReceived = new(TaskCreationOptions.RunContinuationsAsynchronously);
using CancellationTokenSource streamCts = new();
Task reader = Task.Run(async () =>
{
try
{
await foreach (AlarmFeedMessage message in monitor.StreamAsync(null, streamCts.Token))
{
lock (received)
{
received.Add(message);
if (received.Count == 1)
{
// The first message is the baseline ProviderStatus.
baselineReceived.TrySetResult();
}
}
}
}
catch (OperationCanceledException)
{
// Expected when the test cancels the stream.
}
});
// --- Step 2: first message is ProviderStatus{Alarmmgr, Degraded=false}. ---
await baselineReceived.Task.WaitAsync(WaitTimeout);
AlarmFeedMessage baseline;
lock (received)
{
baseline = received[0];
}
Assert.Equal(AlarmFeedMessage.PayloadOneofCase.ProviderStatus, baseline.PayloadCase);
Assert.Equal(AlarmProviderMode.Alarmmgr, baseline.ProviderStatus.Mode);
Assert.False(baseline.ProviderStatus.Degraded);
// --- Step 3: alarmmgr-style transition (Degraded=false, SourceProvider=Alarmmgr, Raise). ---
sessions.EmitEvent(new MxEvent
{
OnAlarmTransition = new OnAlarmTransitionEvent
{
AlarmFullReference = "Galaxy!Area.Tank01.Hi",
SourceObjectReference = "Tank01",
AlarmTypeName = "AnalogLimitAlarm.Hi",
TransitionKind = AlarmTransitionKind.Raise,
Severity = 500,
Degraded = false,
SourceProvider = AlarmProviderMode.Alarmmgr,
TransitionTimestamp = Timestamp.FromDateTimeOffset(DateTimeOffset.UtcNow),
},
});
AlarmFeedMessage alarmmgrTransition = await WaitForAsync(
received,
m => m.PayloadCase == AlarmFeedMessage.PayloadOneofCase.Transition
&& m.Transition.AlarmFullReference == "Galaxy!Area.Tank01.Hi"
&& m.Transition.SourceProvider == AlarmProviderMode.Alarmmgr,
WaitTimeout);
Assert.Equal(AlarmTransitionKind.Raise, alarmmgrTransition.Transition.TransitionKind);
Assert.False(alarmmgrTransition.Transition.Degraded);
Assert.Equal(AlarmProviderMode.Alarmmgr, alarmmgrTransition.Transition.SourceProvider);
// --- Step 4: failover to subtag -> ProviderStatus{Subtag, Degraded=true}. ---
sessions.EmitEvent(new MxEvent
{
OnAlarmProviderModeChanged = new OnAlarmProviderModeChangedEvent
{
Mode = AlarmProviderMode.Subtag,
Reason = "alarmmgr failed",
Hresult = unchecked((int)0x80004005),
At = Timestamp.FromDateTimeOffset(DateTimeOffset.UtcNow),
},
});
AlarmFeedMessage degraded = await WaitForAsync(
received,
m => m.PayloadCase == AlarmFeedMessage.PayloadOneofCase.ProviderStatus
&& m.ProviderStatus.Mode == AlarmProviderMode.Subtag,
WaitTimeout);
Assert.Equal(AlarmProviderMode.Subtag, degraded.ProviderStatus.Mode);
Assert.True(degraded.ProviderStatus.Degraded);
Assert.Equal("alarmmgr failed", degraded.ProviderStatus.Reason);
// --- Step 5: subtag-style transition (Degraded=true, SourceProvider=Subtag, Raise on a different ref). ---
sessions.EmitEvent(new MxEvent
{
OnAlarmTransition = new OnAlarmTransitionEvent
{
AlarmFullReference = "Galaxy!Area.Tank02.Lo",
SourceObjectReference = "Tank02",
AlarmTypeName = "AnalogLimitAlarm.Lo",
TransitionKind = AlarmTransitionKind.Raise,
Severity = 250,
Degraded = true,
SourceProvider = AlarmProviderMode.Subtag,
TransitionTimestamp = Timestamp.FromDateTimeOffset(DateTimeOffset.UtcNow),
},
});
AlarmFeedMessage subtagTransition = await WaitForAsync(
received,
m => m.PayloadCase == AlarmFeedMessage.PayloadOneofCase.Transition
&& m.Transition.AlarmFullReference == "Galaxy!Area.Tank02.Lo"
&& m.Transition.SourceProvider == AlarmProviderMode.Subtag,
WaitTimeout);
Assert.Equal(AlarmTransitionKind.Raise, subtagTransition.Transition.TransitionKind);
Assert.True(subtagTransition.Transition.Degraded);
Assert.Equal(AlarmProviderMode.Subtag, subtagTransition.Transition.SourceProvider);
// --- Step 6: failback to alarmmgr -> ProviderStatus{Alarmmgr, Degraded=false} (recovery). ---
sessions.EmitEvent(new MxEvent
{
OnAlarmProviderModeChanged = new OnAlarmProviderModeChangedEvent
{
Mode = AlarmProviderMode.Alarmmgr,
Reason = "recovered",
Hresult = 0,
At = Timestamp.FromDateTimeOffset(DateTimeOffset.UtcNow),
},
});
// Match the recovery status specifically: an Alarmmgr ProviderStatus that
// carries the "recovered" reason, distinguishing it from the baseline at [0].
AlarmFeedMessage recovered = await WaitForAsync(
received,
m => m.PayloadCase == AlarmFeedMessage.PayloadOneofCase.ProviderStatus
&& m.ProviderStatus.Mode == AlarmProviderMode.Alarmmgr
&& m.ProviderStatus.Reason == "recovered",
WaitTimeout);
Assert.Equal(AlarmProviderMode.Alarmmgr, recovered.ProviderStatus.Mode);
Assert.False(recovered.ProviderStatus.Degraded);
Assert.Equal("recovered", recovered.ProviderStatus.Reason);
await streamCts.CancelAsync();
await reader;
await cts.CancelAsync();
await monitor.StopAsync(CancellationToken.None);
}
[Fact]
public async Task DegradedTransition_CachedThenReplayed_CarriesDegradedAndSourceProviderToNewSubscriber()
{
using GatewayMetrics metrics = new();
FakeSessionManager sessions = new();
StubWatchListResolver resolver = new([]);
AlarmsOptions options = new()
{
Enabled = true,
SubscriptionExpression = @"\\NODE\Galaxy!Area",
Fallback = new AlarmFallbackOptions
{
Mode = "Auto",
ConsecutiveFailureThreshold = 3,
FailbackProbeIntervalSeconds = 9,
FailbackStableProbes = 2,
},
};
using GatewayAlarmMonitor monitor = new(
sessions,
resolver,
metrics,
Microsoft.Extensions.Options.Options.Create(new GatewayOptions { Alarms = options }),
NullLogger<GatewayAlarmMonitor>.Instance);
using CancellationTokenSource cts = new();
await monitor.StartAsync(cts.Token);
await sessions.WaitForSubscribeAsync(WaitTimeout);
// First subscriber: drive the feed past the baseline ProviderStatus so we
// know the monitor's event loop is live before we emit the transition.
List<AlarmFeedMessage> firstReader = [];
TaskCompletionSource baselineReceived = new(TaskCreationOptions.RunContinuationsAsynchronously);
using CancellationTokenSource firstStreamCts = new();
Task reader = Task.Run(async () =>
{
try
{
await foreach (AlarmFeedMessage message in monitor.StreamAsync(null, firstStreamCts.Token))
{
lock (firstReader)
{
firstReader.Add(message);
if (message.PayloadCase == AlarmFeedMessage.PayloadOneofCase.ProviderStatus)
{
baselineReceived.TrySetResult();
}
}
}
}
catch (OperationCanceledException)
{
// Expected when the test cancels the stream.
}
});
await baselineReceived.Task.WaitAsync(WaitTimeout);
// Apply a degraded (subtag) transition. This lands in the monitor's cache
// via SnapshotFromTransition, which must preserve Degraded/SourceProvider.
sessions.EmitEvent(new MxEvent
{
OnAlarmTransition = new OnAlarmTransitionEvent
{
AlarmFullReference = "Galaxy!Area.Tank02.Lo",
SourceObjectReference = "Tank02",
AlarmTypeName = "AnalogLimitAlarm.Lo",
TransitionKind = AlarmTransitionKind.Raise,
Severity = 250,
Degraded = true,
SourceProvider = AlarmProviderMode.Subtag,
TransitionTimestamp = Timestamp.FromDateTimeOffset(DateTimeOffset.UtcNow),
},
});
// Wait for the live transition to be observed by the first subscriber so we
// know the cache has been updated before opening the new stream.
await WaitForAsync(
firstReader,
m => m.PayloadCase == AlarmFeedMessage.PayloadOneofCase.Transition
&& m.Transition.AlarmFullReference == "Galaxy!Area.Tank02.Lo",
WaitTimeout);
// New subscriber: its initial cache snapshot must carry the degraded flags.
using CancellationTokenSource newStreamCts = new();
ActiveAlarmSnapshot? initialActiveAlarm = null;
await foreach (AlarmFeedMessage message in monitor.StreamAsync(null, newStreamCts.Token))
{
if (message.PayloadCase == AlarmFeedMessage.PayloadOneofCase.ActiveAlarm
&& message.ActiveAlarm.AlarmFullReference == "Galaxy!Area.Tank02.Lo")
{
initialActiveAlarm = message.ActiveAlarm;
}
if (message.PayloadCase == AlarmFeedMessage.PayloadOneofCase.SnapshotComplete)
{
break;
}
}
Assert.NotNull(initialActiveAlarm);
Assert.True(initialActiveAlarm!.Degraded);
Assert.Equal(AlarmProviderMode.Subtag, initialActiveAlarm.SourceProvider);
await newStreamCts.CancelAsync();
await firstStreamCts.CancelAsync();
await reader;
await cts.CancelAsync();
await monitor.StopAsync(CancellationToken.None);
}
private static async Task<AlarmFeedMessage> WaitForAsync(
List<AlarmFeedMessage> received,
Func<AlarmFeedMessage, bool> predicate,
TimeSpan timeout)
{
DateTime deadline = DateTime.UtcNow + timeout;
while (DateTime.UtcNow < deadline)
{
lock (received)
{
AlarmFeedMessage? match = received.FirstOrDefault(predicate);
if (match is not null)
{
return match;
}
}
await Task.Delay(25);
}
throw new TimeoutException("No matching AlarmFeedMessage was received in time.");
}
/// <summary><see cref="IAlarmWatchListResolver"/> that returns a fixed watch-list.</summary>
private sealed class StubWatchListResolver(IReadOnlyList<AlarmSubtagTarget> targets) : IAlarmWatchListResolver
{
/// <inheritdoc />
public Task<IReadOnlyList<AlarmSubtagTarget>> ResolveAsync(
AlarmsOptions options,
CancellationToken cancellationToken = default) => Task.FromResult(targets);
}
/// <summary>
/// Minimal <see cref="ISessionManager"/> for driving the monitor: opens a
/// constructed session, records the SubscribeAlarms command, replies OK to
/// every command, and exposes a channel for pushing worker events.
/// </summary>
private sealed class FakeSessionManager : ISessionManager
{
private readonly Channel<WorkerEvent> _events = Channel.CreateUnbounded<WorkerEvent>();
private readonly TaskCompletionSource _subscribed =
new(TaskCreationOptions.RunContinuationsAsynchronously);
/// <summary>The most recent SubscribeAlarms command the monitor sent.</summary>
public SubscribeAlarmsCommand? LastSubscribeCommand { get; private set; }
/// <summary>Pushes a worker event onto the monitor's event stream.</summary>
public void EmitEvent(MxEvent mxEvent) =>
_events.Writer.TryWrite(new WorkerEvent { Event = mxEvent });
/// <summary>Completes once the monitor has issued its SubscribeAlarms command.</summary>
public Task WaitForSubscribeAsync(TimeSpan timeout) => _subscribed.Task.WaitAsync(timeout);
/// <inheritdoc />
public Task<GatewaySession> OpenSessionAsync(
SessionOpenRequest request,
string? clientIdentity,
CancellationToken cancellationToken)
{
GatewaySession session = new(
Guid.NewGuid().ToString("N"),
"Galaxy",
"pipe-test",
"nonce-test",
clientIdentity,
null,
null,
TimeSpan.FromSeconds(30),
TimeSpan.FromSeconds(30),
TimeSpan.FromSeconds(30),
DateTimeOffset.UtcNow);
return Task.FromResult(session);
}
/// <inheritdoc />
public Task<WorkerCommandReply> InvokeAsync(
string sessionId,
WorkerCommand command,
CancellationToken cancellationToken)
{
if (command.Command?.Kind == MxCommandKind.SubscribeAlarms)
{
LastSubscribeCommand = command.Command.SubscribeAlarms;
_subscribed.TrySetResult();
}
MxCommandReply reply = new()
{
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
};
if (command.Command?.Kind == MxCommandKind.QueryActiveAlarms)
{
reply.QueryActiveAlarms = new QueryActiveAlarmsReplyPayload();
}
return Task.FromResult(new WorkerCommandReply { Reply = reply });
}
/// <inheritdoc />
public async IAsyncEnumerable<WorkerEvent> ReadEventsAsync(
string sessionId,
[EnumeratorCancellation] CancellationToken cancellationToken)
{
await foreach (WorkerEvent workerEvent in _events.Reader.ReadAllAsync(cancellationToken))
{
yield return workerEvent;
}
}
/// <inheritdoc />
public bool TryGetSession(string sessionId, [MaybeNullWhen(false)] out GatewaySession session)
{
session = null;
return false;
}
/// <inheritdoc />
public Task<SessionCloseResult> CloseSessionAsync(string sessionId, CancellationToken cancellationToken)
{
_events.Writer.TryComplete();
return Task.FromResult(new SessionCloseResult(sessionId, SessionState.Closed, AlreadyClosed: false));
}
/// <inheritdoc />
public Task<SessionCloseResult> KillWorkerAsync(string sessionId, string reason, CancellationToken cancellationToken) =>
Task.FromResult(new SessionCloseResult(sessionId, SessionState.Closed, AlreadyClosed: false));
/// <inheritdoc />
public Task<int> CloseExpiredLeasesAsync(DateTimeOffset now, CancellationToken cancellationToken) =>
Task.FromResult(0);
/// <inheritdoc />
public Task ShutdownAsync(CancellationToken cancellationToken) => Task.CompletedTask;
}
}
@@ -0,0 +1,332 @@
using Microsoft.Extensions.Logging.Abstractions;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Alarms;
using ZB.MOM.WW.MxGateway.Server.Configuration;
using ZB.MOM.WW.MxGateway.Server.Galaxy;
namespace ZB.MOM.WW.MxGateway.Tests.Alarms;
/// <summary>
/// Unit tests for <see cref="AlarmWatchListResolver"/>: discovery/config merge,
/// subtag-address composition, canonical reference shaping, and the
/// unavailable-discovery code path.
/// </summary>
public sealed class AlarmWatchListResolverTests
{
private static AlarmWatchListResolver CreateResolver(IGalaxyRepository repository) =>
new(repository, NullLogger<AlarmWatchListResolver>.Instance);
private static AlarmsOptions Options(
bool useGalaxyRepository = true,
string area = "",
string defaultArea = "",
string[]? include = null,
string[]? exclude = null,
AlarmSubtagNameOptions? subtags = null) =>
new()
{
DefaultArea = defaultArea,
Fallback = new AlarmFallbackOptions
{
Discovery = new AlarmDiscoveryOptions
{
UseGalaxyRepository = useGalaxyRepository,
Area = area,
IncludeAttributes = include ?? [],
ExcludeAttributes = exclude ?? [],
},
Subtags = subtags ?? new AlarmSubtagNameOptions(),
},
};
[Fact]
public async Task ResolveAsync_UnionsGalaxyRowsAndIncludes_RemovesExcludes_AndDeduplicates()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
new GalaxyAlarmAttributeRow { FullTagReference = "Tank02.Level.HiHi", SourceObjectReference = "Tank02" },
// Duplicate of an include below (case-insensitive) — should appear once.
new GalaxyAlarmAttributeRow { FullTagReference = "Pump01.Fault", SourceObjectReference = "Pump01" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
include: ["pump01.fault", "Valve03.Position.Lo"],
exclude: ["Tank02.Level.HiHi"]));
Assert.Equal(
new[] { "Tank01.Level.HiHi", "Pump01.Fault", "Valve03.Position.Lo" },
result.Select(t => t.ActiveSubtag.Replace(".active", string.Empty, StringComparison.Ordinal)));
// De-dup preserved first (GR) occurrence; exclude removed Tank02.
Assert.Equal(3, result.Count);
}
[Fact]
public async Task ResolveAsync_ComposesSubtagAddressesFromConfigNames()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
subtags: new AlarmSubtagNameOptions
{
Active = "InAlarm",
Acked = "Ack",
Priority = "Pri",
AckComment = "AckCmt",
}));
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Tank01.Level.HiHi.InAlarm", target.ActiveSubtag);
Assert.Equal("Tank01.Level.HiHi.Ack", target.AckedSubtag);
Assert.Equal("Tank01.Level.HiHi.Pri", target.PrioritySubtag);
Assert.Equal("Tank01.Level.HiHi.AckCmt", target.AckCommentSubtag);
}
[Fact]
public async Task ResolveAsync_EmptyPriorityAndAckComment_LeaveThoseFieldsEmpty()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
// Default Priority is "priority"; force it empty alongside empty AckComment.
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
subtags: new AlarmSubtagNameOptions { Priority = string.Empty, AckComment = string.Empty }));
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Tank01.Level.HiHi.active", target.ActiveSubtag);
Assert.Equal("Tank01.Level.HiHi.acked", target.AckedSubtag);
Assert.Equal(string.Empty, target.PrioritySubtag);
Assert.Equal(string.Empty, target.AckCommentSubtag);
}
[Fact]
public async Task ResolveAsync_ComposesCanonicalFullReference_WithArea()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(area: "Site_A"));
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Galaxy!Site_A.Tank01.Level.HiHi", target.AlarmFullReference);
}
[Fact]
public async Task ResolveAsync_ComposesCanonicalFullReference_WithoutArea()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
// No discovery area and no default area.
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options());
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Galaxy!Tank01.Level.HiHi", target.AlarmFullReference);
}
[Fact]
public async Task ResolveAsync_FallsBackToDefaultArea_WhenDiscoveryAreaEmpty()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(area: "", defaultArea: "Plant"));
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Galaxy!Plant.Tank01.Level.HiHi", target.AlarmFullReference);
}
[Fact]
public async Task ResolveAsync_UseGalaxyRepositoryFalse_DoesNotCallRepository_UsesIncludesOnly()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "ShouldNotAppear.X", SourceObjectReference = "ShouldNotAppear" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
useGalaxyRepository: false,
include: ["Tank01.Level.HiHi"]));
Assert.Equal(0, repo.GetAlarmAttributesCount);
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Galaxy!Tank01.Level.HiHi", target.AlarmFullReference);
}
[Fact]
public async Task ResolveAsync_RepositoryThrows_LogsAndReturnsConfigOnlySet()
{
ThrowingGalaxyRepository repo = new(new InvalidOperationException("SQL unavailable"));
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
include: ["Tank01.Level.HiHi"]));
// Did not throw; discovery set empty, include retained.
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Galaxy!Tank01.Level.HiHi", target.AlarmFullReference);
}
[Fact]
public async Task ResolveAsync_DerivesSourceObjectForConfigEntry()
{
StubGalaxyRepository repo = new([]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
include: ["Tank01.Level.HiHi", "StandaloneTag"]));
Assert.Equal("Tank01", result[0].SourceObjectReference);
// No dot — whole string is the source object.
Assert.Equal("StandaloneTag", result[1].SourceObjectReference);
}
/// <summary>
/// Fix 1: ExcludeAttributes must be ignored when UseGalaxyRepository is false.
/// A config-only include must survive even when the same path appears in ExcludeAttributes.
/// </summary>
[Fact]
public async Task ResolveAsync_ExcludeIgnored_WhenGalaxyRepositoryDisabled()
{
// Repo is never consulted; only IncludeAttributes matters.
StubGalaxyRepository repo = new([]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
useGalaxyRepository: false,
include: ["Tank01.Level.HiHi"],
exclude: ["Tank01.Level.HiHi"]));
// ExcludeAttributes is ignored when GR is off — the include must be present.
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Galaxy!Tank01.Level.HiHi", target.AlarmFullReference);
}
/// <summary>
/// Fix 1 (GR-on path): ExcludeAttributes still prunes GR rows when
/// UseGalaxyRepository is true.
/// </summary>
[Fact]
public async Task ResolveAsync_ExcludeApplied_WhenGalaxyRepositoryEnabled()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
new GalaxyAlarmAttributeRow { FullTagReference = "Tank02.Level.HiHi", SourceObjectReference = "Tank02" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
useGalaxyRepository: true,
exclude: ["Tank02.Level.HiHi"]));
// Tank02 was excluded; only Tank01 remains.
AlarmSubtagTarget target = Assert.Single(result);
Assert.Contains("Tank01", target.ActiveSubtag, StringComparison.Ordinal);
}
/// <summary>
/// Fix 1: A whitespace-only ExcludeAttributes entry must be skipped and must
/// not accidentally exclude any real reference.
/// </summary>
[Fact]
public async Task ResolveAsync_WhitespaceOnlyExcludeEntry_IsSkipped()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
// The exclude array contains a whitespace-only string — should be a no-op.
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
useGalaxyRepository: true,
exclude: [" "]));
// Tank01 must not have been wrongly excluded.
AlarmSubtagTarget target = Assert.Single(result);
Assert.Contains("Tank01", target.ActiveSubtag, StringComparison.Ordinal);
}
/// <summary>In-memory <see cref="IGalaxyRepository"/> returning a fixed alarm rowset.</summary>
private sealed class StubGalaxyRepository(List<GalaxyAlarmAttributeRow> rows) : IGalaxyRepository
{
/// <summary>Gets the number of times <see cref="GetAlarmAttributesAsync"/> was called.</summary>
public int GetAlarmAttributesCount { get; private set; }
/// <inheritdoc />
public Task<bool> TestConnectionAsync(CancellationToken ct = default) => Task.FromResult(true);
/// <inheritdoc />
public Task<DateTime?> GetLastDeployTimeAsync(CancellationToken ct = default) =>
Task.FromResult<DateTime?>(null);
/// <inheritdoc />
public Task<List<GalaxyHierarchyRow>> GetHierarchyAsync(CancellationToken ct = default) =>
Task.FromResult(new List<GalaxyHierarchyRow>());
/// <inheritdoc />
public Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default) =>
Task.FromResult(new List<GalaxyAttributeRow>());
/// <inheritdoc />
public Task<List<GalaxyAlarmAttributeRow>> GetAlarmAttributesAsync(CancellationToken ct = default)
{
GetAlarmAttributesCount++;
return Task.FromResult(rows);
}
}
/// <summary><see cref="IGalaxyRepository"/> whose alarm-attribute query throws.</summary>
private sealed class ThrowingGalaxyRepository(Exception toThrow) : IGalaxyRepository
{
/// <inheritdoc />
public Task<bool> TestConnectionAsync(CancellationToken ct = default) => Task.FromResult(true);
/// <inheritdoc />
public Task<DateTime?> GetLastDeployTimeAsync(CancellationToken ct = default) =>
Task.FromResult<DateTime?>(null);
/// <inheritdoc />
public Task<List<GalaxyHierarchyRow>> GetHierarchyAsync(CancellationToken ct = default) =>
Task.FromResult(new List<GalaxyHierarchyRow>());
/// <inheritdoc />
public Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default) =>
Task.FromResult(new List<GalaxyAttributeRow>());
/// <inheritdoc />
public Task<List<GalaxyAlarmAttributeRow>> GetAlarmAttributesAsync(CancellationToken ct = default) =>
Task.FromException<List<GalaxyAlarmAttributeRow>>(toThrow);
}
}
@@ -0,0 +1,412 @@
using System.Diagnostics.CodeAnalysis;
using System.Diagnostics.Metrics;
using System.Runtime.CompilerServices;
using System.Threading.Channels;
using Google.Protobuf.WellKnownTypes;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Alarms;
using ZB.MOM.WW.MxGateway.Server.Configuration;
using ZB.MOM.WW.MxGateway.Server.Metrics;
using ZB.MOM.WW.MxGateway.Server.Sessions;
namespace ZB.MOM.WW.MxGateway.Tests.Alarms;
/// <summary>
/// Drives <see cref="GatewayAlarmMonitor"/> with a fake session manager to
/// verify it reflects the worker's <c>OnAlarmProviderModeChanged</c> event into
/// the alarm feed and the switch metric, and that a new subscriber receives the
/// provider status as its first message. Also covers the watch-list / forced-mode
/// wiring of the <c>SubscribeAlarms</c> command and the Mode→enum mapping.
/// </summary>
public sealed class GatewayAlarmMonitorProviderModeTests
{
private static readonly TimeSpan WaitTimeout = TimeSpan.FromSeconds(15);
[Fact]
public async Task ProviderModeChange_BroadcastsDegradedStatus_AndIncrementsSwitchMetric()
{
using GatewayMetrics metrics = new();
long switchCount = 0;
using MeterListener listener = new();
listener.InstrumentPublished = (instrument, meterListener) =>
{
if (ReferenceEquals(instrument.Meter, metrics.Meter)
&& instrument.Name == "mxgateway.alarms.provider_switches")
{
meterListener.EnableMeasurementEvents(instrument);
}
};
listener.SetMeasurementEventCallback<long>(
(instrument, measurement, _, _) =>
{
if (ReferenceEquals(instrument.Meter, metrics.Meter)
&& instrument.Name == "mxgateway.alarms.provider_switches")
{
Interlocked.Add(ref switchCount, measurement);
}
});
listener.Start();
FakeSessionManager sessions = new();
using GatewayAlarmMonitor monitor = CreateMonitor(sessions, metrics);
using CancellationTokenSource cts = new();
await monitor.StartAsync(cts.Token);
await sessions.WaitForSubscribeAsync(WaitTimeout);
// Subscribe a live feed reader. Gate emitting the mode-change event until the
// reader has consumed its baseline ProviderStatus message, avoiding a race where
// the event arrives before the subscriber is registered and draining its snapshot.
List<AlarmFeedMessage> received = [];
TaskCompletionSource baselineReceived = new(TaskCreationOptions.RunContinuationsAsynchronously);
using CancellationTokenSource streamCts = new();
Task reader = Task.Run(async () =>
{
try
{
await foreach (AlarmFeedMessage message in monitor.StreamAsync(null, streamCts.Token))
{
lock (received)
{
received.Add(message);
// Signal once the first message (baseline ProviderStatus) has arrived.
if (received.Count == 1)
{
baselineReceived.TrySetResult();
}
}
}
}
catch (OperationCanceledException)
{
// Expected when the test cancels the stream.
}
});
// Wait for the baseline ProviderStatus to arrive before emitting the mode change,
// so the subscriber is registered and the event is not dropped.
await baselineReceived.Task.WaitAsync(WaitTimeout);
// Emit the worker event that flips the provider into subtag mode.
sessions.EmitEvent(new MxEvent
{
OnAlarmProviderModeChanged = new OnAlarmProviderModeChangedEvent
{
Mode = AlarmProviderMode.Subtag,
Reason = "alarmmgr failed",
Hresult = unchecked((int)0x80004005),
At = Timestamp.FromDateTimeOffset(DateTimeOffset.UtcNow),
},
});
AlarmFeedMessage degraded = await WaitForAsync(
received,
m => m.PayloadCase == AlarmFeedMessage.PayloadOneofCase.ProviderStatus
&& m.ProviderStatus.Mode == AlarmProviderMode.Subtag,
WaitTimeout);
Assert.Equal(AlarmProviderMode.Subtag, degraded.ProviderStatus.Mode);
Assert.True(degraded.ProviderStatus.Degraded);
Assert.Equal("alarmmgr failed", degraded.ProviderStatus.Reason);
await WaitUntilAsync(() => Interlocked.Read(ref switchCount) >= 1, WaitTimeout);
Assert.Equal(1, Interlocked.Read(ref switchCount));
await streamCts.CancelAsync();
await reader;
await cts.CancelAsync();
await monitor.StopAsync(CancellationToken.None);
}
[Fact]
public async Task NewSubscriber_ReceivesProviderStatusAsFirstMessage()
{
using GatewayMetrics metrics = new();
FakeSessionManager sessions = new();
using GatewayAlarmMonitor monitor = CreateMonitor(sessions, metrics);
using CancellationTokenSource cts = new();
await monitor.StartAsync(cts.Token);
await sessions.WaitForSubscribeAsync(WaitTimeout);
using CancellationTokenSource streamCts = new();
AlarmFeedMessage? first = null;
Task reader = Task.Run(async () =>
{
await foreach (AlarmFeedMessage message in monitor.StreamAsync(null, streamCts.Token))
{
first = message;
break;
}
});
await WaitUntilAsync(() => first is not null, WaitTimeout);
Assert.NotNull(first);
Assert.Equal(AlarmFeedMessage.PayloadOneofCase.ProviderStatus, first!.PayloadCase);
// Baseline before any provider-mode event: alarm-manager, not degraded.
Assert.Equal(AlarmProviderMode.Alarmmgr, first.ProviderStatus.Mode);
Assert.False(first.ProviderStatus.Degraded);
await streamCts.CancelAsync();
await reader;
await cts.CancelAsync();
await monitor.StopAsync(CancellationToken.None);
}
[Fact]
public async Task SubscribeAlarms_SendsForcedModeAndWatchList_FromConfiguration()
{
using GatewayMetrics metrics = new();
FakeSessionManager sessions = new();
StubWatchListResolver resolver = new(
[
new AlarmSubtagTarget { AlarmFullReference = "Galaxy!Area.Tank01.Hi", ActiveSubtag = "Tank01.Hi.active" },
]);
AlarmsOptions options = new()
{
Enabled = true,
SubscriptionExpression = @"\\NODE\Galaxy!Area",
Fallback = new AlarmFallbackOptions
{
Mode = "ForceSubtag",
ConsecutiveFailureThreshold = 7,
FailbackProbeIntervalSeconds = 11,
FailbackStableProbes = 4,
},
};
using GatewayAlarmMonitor monitor = new(
sessions,
resolver,
metrics,
Microsoft.Extensions.Options.Options.Create(new GatewayOptions { Alarms = options }),
NullLogger<GatewayAlarmMonitor>.Instance);
using CancellationTokenSource cts = new();
await monitor.StartAsync(cts.Token);
await sessions.WaitForSubscribeAsync(WaitTimeout);
SubscribeAlarmsCommand sent = Assert.IsType<SubscribeAlarmsCommand>(sessions.LastSubscribeCommand);
Assert.Equal(AlarmProviderMode.Subtag, sent.ForcedMode);
Assert.Equal(7, sent.Failover.ConsecutiveFailureThreshold);
Assert.Equal(11, sent.Failover.FailbackProbeIntervalSeconds);
Assert.Equal(4, sent.Failover.FailbackStableProbes);
AlarmSubtagTarget target = Assert.Single(sent.WatchList);
Assert.Equal("Galaxy!Area.Tank01.Hi", target.AlarmFullReference);
await cts.CancelAsync();
await monitor.StopAsync(CancellationToken.None);
}
[Theory]
[InlineData("ForceAlarmManager", AlarmProviderMode.Alarmmgr)]
[InlineData("forcealarmmanager", AlarmProviderMode.Alarmmgr)]
[InlineData("ForceSubtag", AlarmProviderMode.Subtag)]
[InlineData("forcesubtag", AlarmProviderMode.Subtag)]
[InlineData("Auto", AlarmProviderMode.Unspecified)]
[InlineData("", AlarmProviderMode.Unspecified)]
[InlineData("nonsense", AlarmProviderMode.Unspecified)]
public async Task ModeString_MapsToForcedProviderMode(string mode, AlarmProviderMode expected)
{
using GatewayMetrics metrics = new();
FakeSessionManager sessions = new();
AlarmsOptions options = new()
{
Enabled = true,
SubscriptionExpression = @"\\NODE\Galaxy!Area",
Fallback = new AlarmFallbackOptions { Mode = mode },
};
using GatewayAlarmMonitor monitor = new(
sessions,
new StubWatchListResolver([]),
metrics,
Microsoft.Extensions.Options.Options.Create(new GatewayOptions { Alarms = options }),
NullLogger<GatewayAlarmMonitor>.Instance);
using CancellationTokenSource cts = new();
await monitor.StartAsync(cts.Token);
await sessions.WaitForSubscribeAsync(WaitTimeout);
Assert.Equal(expected, sessions.LastSubscribeCommand!.ForcedMode);
// Auto + empty watch-list preserves historical alarmmgr-only behaviour.
if (expected == AlarmProviderMode.Unspecified)
{
Assert.Empty(sessions.LastSubscribeCommand!.WatchList);
}
await cts.CancelAsync();
await monitor.StopAsync(CancellationToken.None);
}
private static GatewayAlarmMonitor CreateMonitor(FakeSessionManager sessions, GatewayMetrics metrics)
{
AlarmsOptions options = new()
{
Enabled = true,
SubscriptionExpression = @"\\NODE\Galaxy!Area",
};
return new GatewayAlarmMonitor(
sessions,
new StubWatchListResolver([]),
metrics,
Microsoft.Extensions.Options.Options.Create(new GatewayOptions { Alarms = options }),
NullLogger<GatewayAlarmMonitor>.Instance);
}
private static async Task<AlarmFeedMessage> WaitForAsync(
List<AlarmFeedMessage> received,
Func<AlarmFeedMessage, bool> predicate,
TimeSpan timeout)
{
DateTime deadline = DateTime.UtcNow + timeout;
while (DateTime.UtcNow < deadline)
{
lock (received)
{
AlarmFeedMessage? match = received.FirstOrDefault(predicate);
if (match is not null)
{
return match;
}
}
await Task.Delay(25);
}
throw new TimeoutException("No matching AlarmFeedMessage was received in time.");
}
private static async Task WaitUntilAsync(Func<bool> condition, TimeSpan timeout)
{
DateTime deadline = DateTime.UtcNow + timeout;
while (DateTime.UtcNow < deadline)
{
if (condition())
{
return;
}
await Task.Delay(25);
}
throw new TimeoutException("Condition was not met in time.");
}
/// <summary><see cref="IAlarmWatchListResolver"/> that returns a fixed watch-list.</summary>
private sealed class StubWatchListResolver(IReadOnlyList<AlarmSubtagTarget> targets) : IAlarmWatchListResolver
{
/// <inheritdoc />
public Task<IReadOnlyList<AlarmSubtagTarget>> ResolveAsync(
AlarmsOptions options,
CancellationToken cancellationToken = default) => Task.FromResult(targets);
}
/// <summary>
/// Minimal <see cref="ISessionManager"/> for driving the monitor: opens a
/// constructed session, records the SubscribeAlarms command, replies OK to
/// every command, and exposes a channel for pushing worker events.
/// </summary>
private sealed class FakeSessionManager : ISessionManager
{
private readonly Channel<WorkerEvent> _events = Channel.CreateUnbounded<WorkerEvent>();
private readonly TaskCompletionSource _subscribed =
new(TaskCreationOptions.RunContinuationsAsynchronously);
/// <summary>The most recent SubscribeAlarms command the monitor sent.</summary>
public SubscribeAlarmsCommand? LastSubscribeCommand { get; private set; }
/// <summary>Pushes a worker event onto the monitor's event stream.</summary>
public void EmitEvent(MxEvent mxEvent) =>
_events.Writer.TryWrite(new WorkerEvent { Event = mxEvent });
/// <summary>Completes once the monitor has issued its SubscribeAlarms command.</summary>
public Task WaitForSubscribeAsync(TimeSpan timeout) => _subscribed.Task.WaitAsync(timeout);
/// <inheritdoc />
public Task<GatewaySession> OpenSessionAsync(
SessionOpenRequest request,
string? clientIdentity,
CancellationToken cancellationToken)
{
GatewaySession session = new(
Guid.NewGuid().ToString("N"),
"Galaxy",
"pipe-test",
"nonce-test",
clientIdentity,
null,
null,
TimeSpan.FromSeconds(30),
TimeSpan.FromSeconds(30),
TimeSpan.FromSeconds(30),
DateTimeOffset.UtcNow);
return Task.FromResult(session);
}
/// <inheritdoc />
public Task<WorkerCommandReply> InvokeAsync(
string sessionId,
WorkerCommand command,
CancellationToken cancellationToken)
{
if (command.Command?.Kind == MxCommandKind.SubscribeAlarms)
{
LastSubscribeCommand = command.Command.SubscribeAlarms;
_subscribed.TrySetResult();
}
MxCommandReply reply = new()
{
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
};
if (command.Command?.Kind == MxCommandKind.QueryActiveAlarms)
{
reply.QueryActiveAlarms = new QueryActiveAlarmsReplyPayload();
}
return Task.FromResult(new WorkerCommandReply { Reply = reply });
}
/// <inheritdoc />
public async IAsyncEnumerable<WorkerEvent> ReadEventsAsync(
string sessionId,
[EnumeratorCancellation] CancellationToken cancellationToken)
{
await foreach (WorkerEvent workerEvent in _events.Reader.ReadAllAsync(cancellationToken))
{
yield return workerEvent;
}
}
/// <inheritdoc />
public bool TryGetSession(string sessionId, [MaybeNullWhen(false)] out GatewaySession session)
{
session = null;
return false;
}
/// <inheritdoc />
public Task<SessionCloseResult> CloseSessionAsync(string sessionId, CancellationToken cancellationToken)
{
_events.Writer.TryComplete();
return Task.FromResult(new SessionCloseResult(sessionId, SessionState.Closed, AlreadyClosed: false));
}
/// <inheritdoc />
public Task<SessionCloseResult> KillWorkerAsync(string sessionId, string reason, CancellationToken cancellationToken) =>
Task.FromResult(new SessionCloseResult(sessionId, SessionState.Closed, AlreadyClosed: false));
/// <inheritdoc />
public Task<int> CloseExpiredLeasesAsync(DateTimeOffset now, CancellationToken cancellationToken) =>
Task.FromResult(0);
/// <inheritdoc />
public Task ShutdownAsync(CancellationToken cancellationToken) => Task.CompletedTask;
}
}
@@ -118,4 +118,175 @@ public sealed class GatewayOptionsValidatorTests
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Succeeded);
}
// -------------------------------------------------------------------------
// AlarmFallbackOptions validation
// -------------------------------------------------------------------------
private static AlarmsOptions EnabledAlarmsWithFallback(AlarmFallbackOptions fallback) => new()
{
Enabled = true,
DefaultArea = "Galaxy",
Fallback = fallback,
};
private static GatewayOptions CloneWithAlarms(GatewayOptions source, AlarmsOptions alarms)
=> new()
{
Authentication = source.Authentication,
Ldap = source.Ldap,
Worker = source.Worker,
Sessions = source.Sessions,
Events = source.Events,
Dashboard = source.Dashboard,
Protocol = source.Protocol,
Alarms = alarms,
Tls = source.Tls,
};
[Fact]
public void Validate_Succeeds_WhenAlarmsDisabled_FallbackNotValidated()
{
// Even an invalid Mode is acceptable when Enabled = false.
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
new AlarmsOptions
{
Enabled = false,
Fallback = new AlarmFallbackOptions { Mode = "InvalidMode" },
});
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Succeeded);
}
[Fact]
public void Validate_Succeeds_WhenAlarmsEnabled_DefaultAutoConfig()
{
// Default AlarmFallbackOptions (Mode="Auto") must pass validation when alarms are enabled.
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions()));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Succeeded);
}
[Theory]
[InlineData("Auto")]
[InlineData("ForceAlarmManager")]
[InlineData("ForceSubtag")]
[InlineData("auto")]
[InlineData("FORCESUBTAG")]
public void Validate_Succeeds_WhenAlarmsEnabled_RecognisedMode(string mode)
{
AlarmsOptions alarms = EnabledAlarmsWithFallback(new AlarmFallbackOptions { Mode = mode });
GatewayOptions options = CloneWithAlarms(ValidOptions(), alarms);
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Succeeded);
}
[Fact]
public void Validate_Fails_WhenAlarmsEnabled_InvalidMode()
{
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions { Mode = "InvalidMode" }));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Failed);
Assert.Contains(result.Failures!, f => f.Contains("MxGateway:Alarms:Fallback") && f.Contains("Mode"));
}
[Fact]
public void Validate_Fails_WhenForceSubtag_NoGalaxyRepository_NoIncludes()
{
// ForceSubtag without galaxy repository and without IncludeAttributes must fail.
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions
{
Mode = "ForceSubtag",
Discovery = new AlarmDiscoveryOptions
{
UseGalaxyRepository = false,
IncludeAttributes = [],
},
}));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Failed);
Assert.Contains(
result.Failures!,
f => f.Contains("ForceSubtag") && f.Contains("Discovery"));
}
[Fact]
public void Validate_Succeeds_WhenForceSubtag_NoGalaxyRepository_WithIncludes()
{
// ForceSubtag without galaxy repository is allowed when IncludeAttributes is non-empty.
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions
{
Mode = "ForceSubtag",
Discovery = new AlarmDiscoveryOptions
{
UseGalaxyRepository = false,
IncludeAttributes = ["attr1"],
},
}));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Succeeded);
}
[Fact]
public void Validate_Succeeds_WhenForceSubtag_WithGalaxyRepository()
{
// ForceSubtag + UseGalaxyRepository=true (default) must pass even without IncludeAttributes.
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions
{
Mode = "ForceSubtag",
Discovery = new AlarmDiscoveryOptions { UseGalaxyRepository = true },
}));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Succeeded);
}
[Theory]
[InlineData(0, nameof(AlarmFallbackOptions.ConsecutiveFailureThreshold))]
[InlineData(-1, nameof(AlarmFallbackOptions.ConsecutiveFailureThreshold))]
public void Validate_Fails_WhenConsecutiveFailureThresholdBelowOne(int value, string keyPart)
{
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions { ConsecutiveFailureThreshold = value }));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Failed);
Assert.Contains(result.Failures!, f => f.Contains(keyPart));
}
[Theory]
[InlineData(0, nameof(AlarmFallbackOptions.FailbackProbeIntervalSeconds))]
[InlineData(-5, nameof(AlarmFallbackOptions.FailbackProbeIntervalSeconds))]
public void Validate_Fails_WhenFailbackProbeIntervalSecondsBelowOne(int value, string keyPart)
{
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions { FailbackProbeIntervalSeconds = value }));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Failed);
Assert.Contains(result.Failures!, f => f.Contains(keyPart));
}
[Theory]
[InlineData(0, nameof(AlarmFallbackOptions.FailbackStableProbes))]
[InlineData(-1, nameof(AlarmFallbackOptions.FailbackStableProbes))]
public void Validate_Fails_WhenFailbackStableProbesBelowOne(int value, string keyPart)
{
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions { FailbackStableProbes = value }));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Failed);
Assert.Contains(result.Failures!, f => f.Contains(keyPart));
}
}
@@ -1332,6 +1332,56 @@ public sealed class ProtobufContractRoundTripTests
Assert.Equal(kind, parsed.Kind);
}
/// <summary>
/// Verifies that an <see cref="AlarmFeedMessage"/> carrying the
/// <c>provider_status</c> payload case round-trips and resolves to
/// <see cref="AlarmFeedMessage.PayloadOneofCase.ProviderStatus"/>.
/// </summary>
[Fact]
public void Feed_RoundTripsProviderStatus()
{
var since = Timestamp.FromDateTime(new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc));
var original = new AlarmFeedMessage
{
ProviderStatus = new AlarmProviderStatus
{
Mode = AlarmProviderMode.Subtag,
Degraded = true,
Reason = "wnwrap poll failed 3x (HRESULT 0x80004005)",
Since = since,
},
};
var parsed = AlarmFeedMessage.Parser.ParseFrom(original.ToByteArray());
Assert.Equal(original, parsed);
Assert.Equal(AlarmFeedMessage.PayloadOneofCase.ProviderStatus, parsed.PayloadCase);
Assert.True(parsed.ProviderStatus.Degraded);
Assert.Equal(AlarmProviderMode.Subtag, parsed.ProviderStatus.Mode);
}
/// <summary>
/// Verifies that an <see cref="OnAlarmTransitionEvent"/> carrying the
/// new <c>degraded</c> and <c>source_provider</c> provenance fields
/// round-trips with their values preserved.
/// </summary>
[Fact]
public void Transition_RoundTripsDegradedProvenance()
{
var t = new OnAlarmTransitionEvent
{
AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi",
TransitionKind = AlarmTransitionKind.Raise,
Degraded = true,
SourceProvider = AlarmProviderMode.Subtag,
};
var parsed = OnAlarmTransitionEvent.Parser.ParseFrom(t.ToByteArray());
Assert.True(parsed.Degraded);
Assert.Equal(AlarmProviderMode.Subtag, parsed.SourceProvider);
}
/// <summary>
/// Verifies that an <see cref="MxCommandReply"/> with kind
/// <see cref="MxCommandKind.ReadBulk"/> and a populated
@@ -0,0 +1,64 @@
using ZB.MOM.WW.MxGateway.Server.Galaxy;
namespace ZB.MOM.WW.MxGateway.Tests.Galaxy;
/// <summary>
/// Pure mapper tests for <see cref="GalaxyRepository.MapAlarmRow"/>. These assert the
/// FullTagReference / SourceObjectReference derivation produced by
/// <c>AlarmAttributesSql</c> without touching a database: the SQL projects
/// <c>tag_name</c> as the source object and <c>tag_name + '.' + attribute_name</c> as
/// the full reference, exactly as <c>AttributesSql</c> does.
/// </summary>
public sealed class GalaxyAlarmAttributeMappingTests
{
/// <summary>Verifies the mapper copies both projected columns onto the row.</summary>
[Fact]
public void MapAlarmRow_CopiesProjectedColumns()
{
GalaxyAlarmAttributeRow row = GalaxyRepository.MapAlarmRow(
fullTagReference: "Tank01.Level.HiHi",
sourceObjectReference: "Tank01");
Assert.Equal("Tank01.Level.HiHi", row.FullTagReference);
Assert.Equal("Tank01", row.SourceObjectReference);
}
/// <summary>
/// Verifies <see cref="GalaxyAlarmAttributeRow.AckCommentSubtag"/> is always empty:
/// the schema does not expose an ack-comment address, so the watch-list resolver
/// composes it later from configuration.
/// </summary>
[Fact]
public void MapAlarmRow_LeavesAckCommentSubtagEmpty()
{
GalaxyAlarmAttributeRow row = GalaxyRepository.MapAlarmRow(
fullTagReference: "Tank01.Level.HiHi",
sourceObjectReference: "Tank01");
Assert.Equal(string.Empty, row.AckCommentSubtag);
}
/// <summary>
/// Verifies the SourceObjectReference is the owning object (the SQL <c>tag_name</c>),
/// i.e. the segment that precedes the first attribute dot in the full reference, even
/// when the attribute itself is a multi-segment extension path.
/// </summary>
[Theory]
[InlineData("Tank01", "Level.HiHi", "Tank01.Level.HiHi")]
[InlineData("Pump_001", "Speed", "Pump_001.Speed")]
[InlineData("TestAlarm001", "Alarm001", "TestAlarm001.Alarm001")]
public void MapAlarmRow_SourceObjectIsSegmentBeforeFirstAttributeDot(
string tagName,
string attributeName,
string expectedFullReference)
{
// Mirror the AlarmAttributesSql projection: full_tag_reference = tag_name + '.' + attribute_name.
string fullTagReference = tagName + "." + attributeName;
GalaxyAlarmAttributeRow row = GalaxyRepository.MapAlarmRow(fullTagReference, tagName);
Assert.Equal(expectedFullReference, row.FullTagReference);
Assert.Equal(tagName, row.SourceObjectReference);
Assert.Equal(row.FullTagReference, row.SourceObjectReference + "." + attributeName);
}
}
@@ -378,6 +378,10 @@ public sealed class GalaxyHierarchyCacheTests : IDisposable
/// <inheritdoc />
public Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default)
=> throw new InvalidOperationException("GetAttributesAsync should not be reached");
/// <inheritdoc />
public Task<List<GalaxyAlarmAttributeRow>> GetAlarmAttributesAsync(CancellationToken ct = default)
=> throw new InvalidOperationException("GetAlarmAttributesAsync should not be reached");
}
/// <summary>Snapshot store whose <see cref="SaveAsync"/> cancels the token mid-save.</summary>
@@ -465,6 +469,10 @@ public sealed class GalaxyHierarchyCacheTests : IDisposable
GetAttributesCount++;
return Task.FromResult(_attributes);
}
/// <inheritdoc />
public Task<List<GalaxyAlarmAttributeRow>> GetAlarmAttributesAsync(CancellationToken ct = default)
=> Task.FromResult(new List<GalaxyAlarmAttributeRow>());
}
/// <inheritdoc />
@@ -518,6 +526,10 @@ public sealed class GalaxyHierarchyCacheTests : IDisposable
GetAttributesCount++;
throw toThrow;
}
/// <inheritdoc />
public Task<List<GalaxyAlarmAttributeRow>> GetAlarmAttributesAsync(CancellationToken ct = default)
=> throw toThrow;
}
}
@@ -137,6 +137,41 @@ public sealed class DashboardBrowseAndAlarmModelTests
Assert.False(ackedRow.IsUnacknowledged);
}
/// <summary>Verifies that a healthy alarmmgr provider status maps to a green badge.</summary>
[Fact]
public void FromProviderStatus_Alarmmgr_NotDegraded_GreenBadge()
{
AlarmProviderStatus status = new()
{
Mode = AlarmProviderMode.Alarmmgr,
Degraded = false,
};
DashboardAlarmProviderStatus model = DashboardAlarmProviderStatus.FromProviderStatus(status);
Assert.False(model.IsDegraded);
Assert.Contains("bg-success", model.BadgeCssClass, StringComparison.Ordinal);
Assert.Equal(DashboardAlarmProviderStatus.AlarmManagerLabel, model.Label);
}
/// <summary>Verifies that a degraded subtag provider status maps to an amber warning badge.</summary>
[Fact]
public void FromProviderStatus_Subtag_Degraded_WarningBadge()
{
AlarmProviderStatus status = new()
{
Mode = AlarmProviderMode.Subtag,
Degraded = true,
Reason = "x",
};
DashboardAlarmProviderStatus model = DashboardAlarmProviderStatus.FromProviderStatus(status);
Assert.True(model.IsDegraded);
Assert.Contains("bg-warning", model.BadgeCssClass, StringComparison.Ordinal);
Assert.Equal("x", model.Reason);
}
/// <summary>Verifies that the formatter renders array elements and element type correctly.</summary>
[Fact]
public void FormatValue_AndDataType_RenderArrayElementsAndElementType()
@@ -1,3 +1,4 @@
using System.Diagnostics.Metrics;
using ZB.MOM.WW.MxGateway.Server.Metrics;
namespace ZB.MOM.WW.MxGateway.Tests.Metrics;
@@ -63,6 +64,98 @@ public sealed class GatewayMetricsTests
Assert.Equal("depth", exception.ParamName);
}
/// <summary>
/// Verifies that <see cref="GatewayMetrics.AlarmProviderSwitched"/> increments
/// <c>mxgateway.alarms.provider_switches</c> by one with the expected from/to/reason tags.
/// The listener filters by the specific <see cref="System.Diagnostics.Metrics.Meter"/> instance
/// to avoid cross-talk between parallel tests (Tests-027).
/// </summary>
[Fact]
public void AlarmProviderSwitched_IncrementsCounterWithExpectedTags()
{
using GatewayMetrics metrics = new();
using MeterListener listener = new();
long capturedValue = 0;
string? capturedFrom = null;
string? capturedTo = null;
string? capturedReason = null;
listener.InstrumentPublished = (instrument, meterListener) =>
{
if (ReferenceEquals(instrument.Meter, metrics.Meter)
&& instrument.Name == "mxgateway.alarms.provider_switches")
{
meterListener.EnableMeasurementEvents(instrument);
}
};
listener.SetMeasurementEventCallback<long>(
(instrument, measurement, tags, _) =>
{
if (!ReferenceEquals(instrument.Meter, metrics.Meter)
|| instrument.Name != "mxgateway.alarms.provider_switches")
{
return;
}
capturedValue += measurement;
foreach (KeyValuePair<string, object?> tag in tags)
{
switch (tag.Key)
{
case "from": capturedFrom = tag.Value as string; break;
case "to": capturedTo = tag.Value as string; break;
case "reason": capturedReason = tag.Value as string; break;
}
}
});
listener.Start();
metrics.AlarmProviderSwitched(1, 2, "test");
Assert.Equal(1, capturedValue);
Assert.Equal("1", capturedFrom);
Assert.Equal("2", capturedTo);
Assert.Equal("test", capturedReason);
}
/// <summary>
/// Verifies that <see cref="GatewayMetrics.AlarmProviderSwitched"/> updates the
/// <c>mxgateway.alarms.provider_mode</c> observable gauge to the <paramref name="toMode"/> value.
/// </summary>
[Fact]
public void AlarmProviderSwitched_UpdatesProviderModeGauge()
{
using GatewayMetrics metrics = new();
using MeterListener listener = new();
int? capturedMode = null;
listener.InstrumentPublished = (instrument, meterListener) =>
{
if (ReferenceEquals(instrument.Meter, metrics.Meter)
&& instrument.Name == "mxgateway.alarms.provider_mode")
{
meterListener.EnableMeasurementEvents(instrument);
}
};
listener.SetMeasurementEventCallback<int>(
(instrument, measurement, _, _) =>
{
if (ReferenceEquals(instrument.Meter, metrics.Meter)
&& instrument.Name == "mxgateway.alarms.provider_mode")
{
capturedMode = measurement;
}
});
listener.Start();
metrics.AlarmProviderSwitched(1, 2, "test");
listener.RecordObservableInstruments();
Assert.Equal(2, capturedMode);
}
/// <summary>Verifies that removing session events only affects that session.</summary>
[Fact]
public void RemoveSessionEvents_RemovesOnlyThatSession()
@@ -372,11 +372,11 @@ public sealed class AlarmCommandExecutorTests
public string? LastFilterPrefix { get; private set; }
/// <summary>Records a subscription.</summary>
/// <param name="subscription">The subscription expression.</param>
/// <param name="command">The subscribe-alarms command.</param>
/// <param name="sessionId">The session identifier.</param>
public void Subscribe(string subscription, string sessionId)
public void Subscribe(SubscribeAlarmsCommand command, string sessionId)
{
LastSubscription = subscription;
LastSubscription = command.SubscriptionExpression;
LastSessionId = sessionId;
}
@@ -21,7 +21,7 @@ public sealed class AlarmCommandHandlerTests
new MxAccessEventQueue(),
() => consumer);
handler.Subscribe(@"\\HOST\Galaxy!Area", "session-1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!Area" }, "session-1");
Assert.True(handler.IsSubscribed);
Assert.Equal(@"\\HOST\Galaxy!Area", consumer.LastSubscription);
@@ -36,9 +36,9 @@ public sealed class AlarmCommandHandlerTests
new MxAccessEventQueue(),
() => consumer);
handler.Subscribe(@"\\HOST\Galaxy!A", "s1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
Assert.Throws<InvalidOperationException>(
() => handler.Subscribe(@"\\HOST\Galaxy!B", "s1"));
() => handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!B" }, "s1"));
}
/// <summary>
@@ -63,7 +63,7 @@ public sealed class AlarmCommandHandlerTests
() => consumer);
InvalidOperationException exception = Assert.Throws<InvalidOperationException>(
() => handler.Subscribe(@"\\HOST\Galaxy!A", "s1"));
() => handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1"));
Assert.Contains("simulated wnwrap subscribe failure", exception.Message);
Assert.False(handler.IsSubscribed);
Assert.True(consumer.Disposed);
@@ -77,7 +77,7 @@ public sealed class AlarmCommandHandlerTests
AlarmCommandHandler handler = new AlarmCommandHandler(
new MxAccessEventQueue(),
() => consumer);
handler.Subscribe(@"\\HOST\Galaxy!A", "s1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
handler.Unsubscribe();
@@ -104,7 +104,7 @@ public sealed class AlarmCommandHandlerTests
AlarmCommandHandler handler = new AlarmCommandHandler(
new MxAccessEventQueue(),
() => consumer);
handler.Subscribe(@"\\HOST\Galaxy!A", "s1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
Guid g = Guid.NewGuid();
int rc = handler.Acknowledge(g, "c", "u", "n", "d", "F");
@@ -149,7 +149,7 @@ public sealed class AlarmCommandHandlerTests
AlarmCommandHandler handler = new AlarmCommandHandler(
new MxAccessEventQueue(),
() => consumer);
handler.Subscribe(@"\\HOST\Galaxy!A", "s1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
IReadOnlyList<ActiveAlarmSnapshot> snapshots = handler.QueryActive(null);
@@ -173,7 +173,7 @@ public sealed class AlarmCommandHandlerTests
AlarmCommandHandler handler = new AlarmCommandHandler(
new MxAccessEventQueue(),
() => consumer);
handler.Subscribe(@"\\HOST\Galaxy!A", "s1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
IReadOnlyList<ActiveAlarmSnapshot> filtered = handler.QueryActive("Galaxy!AreaA");
@@ -189,13 +189,13 @@ public sealed class AlarmCommandHandlerTests
AlarmCommandHandler handler = new AlarmCommandHandler(
new MxAccessEventQueue(),
() => consumer);
handler.Subscribe(@"\\HOST\Galaxy!A", "s1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
handler.Dispose();
Assert.True(consumer.Disposed);
Assert.Throws<ObjectDisposedException>(
() => handler.Subscribe("x", "y"));
() => handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = "x" }, "y"));
}
/// <summary>
@@ -218,7 +218,7 @@ public sealed class AlarmCommandHandlerTests
// factory is invoked. We tally invocation counts after each call so
// that a missed guard surfaces as the diagnostic count, not a generic
// "Subscribe should have failed".
handler.Subscribe(@"\\HOST\Galaxy!A", "s1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
Assert.Equal(1, guardInvocations);
handler.Acknowledge(Guid.NewGuid(), "c", "u", "n", "d", "F");
@@ -254,7 +254,7 @@ public sealed class AlarmCommandHandlerTests
// Subscribe: guard runs before the dispatcher is constructed.
Assert.Throws<InvalidOperationException>(
() => handler.Subscribe(@"\\HOST\Galaxy!A", "s1"));
() => handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1"));
// To exercise the other entry points we need a subscribed handler.
// Construct a parallel handler with a passing guard, then swap in a
@@ -273,6 +273,132 @@ public sealed class AlarmCommandHandlerTests
Assert.Throws<InvalidOperationException>(() => handler.Unsubscribe());
}
/// <summary>
/// Worker-9: ForcedMode=Subtag builds a subtag consumer (via the
/// injected standby factory) and advises it — the primary
/// (alarmmgr) consumer is NOT created.
/// </summary>
[Fact]
public void Subscribe_WithForcedSubtagMode_BuildsStandbyConsumerOnly()
{
FakeConsumer primary = new FakeConsumer();
FakeConsumer standby = new FakeConsumer();
IReadOnlyList<AlarmSubtagTarget>? capturedWatchList = null;
AlarmCommandHandler handler = new AlarmCommandHandler(
new MxAccessEventQueue(),
() => primary,
threadAffinityCheck: null,
comFactory: null,
standbyFactory: watch =>
{
capturedWatchList = watch;
return standby;
});
SubscribeAlarmsCommand command = new SubscribeAlarmsCommand
{
SubscriptionExpression = @"\\HOST\Galaxy!Area",
ForcedMode = AlarmProviderMode.Subtag,
};
command.WatchList.Add(new AlarmSubtagTarget { AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi" });
handler.Subscribe(command, "s1");
Assert.True(handler.IsSubscribed);
Assert.Equal(@"\\HOST\Galaxy!Area", standby.LastSubscription); // standby advised
Assert.Null(primary.LastSubscription); // primary never built
Assert.NotNull(capturedWatchList);
Assert.Single(capturedWatchList!);
}
/// <summary>
/// Worker-9: ForcedMode=Unspecified + a non-empty watch list builds a
/// failover composite (primary + subtag standby). Forcing the primary
/// to fail on subscribe with a threshold of 1 drives the composite to
/// switch to the subtag provider, which must enqueue an
/// OnAlarmProviderModeChanged event carrying mode=Subtag.
/// </summary>
[Fact]
public void Subscribe_AutoModeWithWatchList_FailoverModeChange_EnqueuesProviderModeChangedEvent()
{
FakeConsumer primary = new FakeConsumer { ThrowOnSubscribe = true };
FakeConsumer standby = new FakeConsumer();
MxAccessEventQueue queue = new MxAccessEventQueue();
AlarmCommandHandler handler = new AlarmCommandHandler(
queue,
() => primary,
threadAffinityCheck: null,
comFactory: null,
standbyFactory: _ => standby);
SubscribeAlarmsCommand command = new SubscribeAlarmsCommand
{
SubscriptionExpression = @"\\HOST\Galaxy!Area",
ForcedMode = AlarmProviderMode.Unspecified,
Failover = new AlarmFailoverConfig
{
ConsecutiveFailureThreshold = 1,
FailbackProbeIntervalSeconds = 1,
FailbackStableProbes = 1,
},
};
command.WatchList.Add(new AlarmSubtagTarget { AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi" });
// Subscribe: standby is armed cleanly; the primary subscribe throws and,
// at threshold 1, the failover composite switches to standby and raises
// ProviderModeChanged. The handler enqueues the proto event.
handler.Subscribe(command, "s1");
IReadOnlyList<WorkerEvent> drained = queue.Drain(0);
Assert.Single(drained);
MxEvent evt = drained[0].Event;
Assert.Equal(MxEventFamily.OnAlarmProviderModeChanged, evt.Family);
Assert.Equal("s1", evt.SessionId);
Assert.NotNull(evt.OnAlarmProviderModeChanged);
Assert.Equal(AlarmProviderMode.Subtag, evt.OnAlarmProviderModeChanged.Mode);
}
/// <summary>
/// Worker-9: a non-failover subscribe (alarmmgr-only) never enqueues a
/// provider-mode-changed event, and a subsequent Unsubscribe detaches
/// the handler so no event leaks.
/// </summary>
[Fact]
public void Subscribe_AlarmmgrOnly_DoesNotEnqueueProviderModeChangedEvent()
{
FakeConsumer consumer = new FakeConsumer();
MxAccessEventQueue queue = new MxAccessEventQueue();
AlarmCommandHandler handler = new AlarmCommandHandler(queue, () => consumer);
handler.Subscribe(
new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
handler.Unsubscribe();
Assert.Empty(queue.Drain(0));
}
/// <summary>
/// Worker-9: the mapper builds a well-formed OnAlarmProviderModeChanged
/// MxEvent — correct family and populated body fields.
/// </summary>
[Fact]
public void Mapper_CreateOnAlarmProviderModeChanged_PopulatesFamilyAndBody()
{
MxAccessEventMapper mapper = new MxAccessEventMapper();
DateTime at = new DateTime(2026, 6, 13, 10, 0, 0, DateTimeKind.Utc);
MxEvent evt = mapper.CreateOnAlarmProviderModeChanged(
"session-7", AlarmProviderMode.Subtag, "primary PollOnce failed", unchecked((int)0x80004005), at);
Assert.Equal(MxEventFamily.OnAlarmProviderModeChanged, evt.Family);
Assert.Equal("session-7", evt.SessionId);
Assert.NotNull(evt.OnAlarmProviderModeChanged);
Assert.Equal(AlarmProviderMode.Subtag, evt.OnAlarmProviderModeChanged.Mode);
Assert.Equal("primary PollOnce failed", evt.OnAlarmProviderModeChanged.Reason);
Assert.Equal(unchecked((int)0x80004005), evt.OnAlarmProviderModeChanged.Hresult);
Assert.Equal(at, evt.OnAlarmProviderModeChanged.At.ToDateTime());
}
private static MxAlarmSnapshotRecord NewRecord(string provider, string group, string tag)
{
return new MxAlarmSnapshotRecord
@@ -62,6 +62,43 @@ public sealed class AlarmDispatcherTests
Assert.Equal("TestArea", body.Category);
Assert.NotNull(body.TransitionTimestamp);
Assert.Equal(ts, body.TransitionTimestamp.ToDateTime());
Assert.False(body.Degraded);
Assert.Equal(AlarmProviderMode.Alarmmgr, body.SourceProvider);
}
/// <summary>
/// Verifies that a transition enqueued via the subtag fallback
/// (<c>degraded: true</c>) is marked <see cref="OnAlarmTransitionEvent.Degraded"/>
/// with <see cref="AlarmProviderMode.Subtag"/>, while the default path
/// stays on the alarmmgr parity contract.
/// </summary>
[Fact]
public void EnqueueTransition_WhenDegraded_MarksDegradedAndSubtagProvider()
{
MxAccessEventQueue queue = new MxAccessEventQueue();
MxAccessAlarmEventSink sink = new MxAccessAlarmEventSink(queue, new MxAccessEventMapper());
sink.Attach(new object(), SessionId);
DateTime ts = new DateTime(2026, 5, 1, 17, 26, 14, 709, DateTimeKind.Utc);
sink.EnqueueTransition(
alarmFullReference: "Galaxy!TestArea.TestMachine_001.TestAlarm001",
sourceObjectReference: "TestMachine_001.TestAlarm001",
alarmTypeName: "DSC",
transitionKind: AlarmTransitionKind.Raise,
severity: 500,
originalRaiseTimestampUtc: null,
transitionTimestampUtc: ts,
operatorUser: string.Empty,
operatorComment: string.Empty,
category: "TestArea",
description: string.Empty,
degraded: true);
Assert.Equal(1, queue.Count);
Assert.True(queue.TryDequeue(out WorkerEvent? workerEvent));
OnAlarmTransitionEvent body = workerEvent!.Event.OnAlarmTransition;
Assert.True(body.Degraded);
Assert.Equal(AlarmProviderMode.Subtag, body.SourceProvider);
}
/// <summary>Verifies that unchanged alarm states do not emit transitions.</summary>
@@ -242,6 +279,60 @@ public sealed class AlarmDispatcherTests
Assert.Equal(AlarmConditionState.ActiveAcked, snapshots[1].CurrentState);
}
/// <summary>
/// Verifies that the per-record subtag-fallback flag flows through the
/// snapshot path: a degraded record maps to an
/// <see cref="ActiveAlarmSnapshot"/> with <see cref="ActiveAlarmSnapshot.Degraded"/>
/// set and <see cref="AlarmProviderMode.Subtag"/>, while a non-degraded
/// record stays on the alarmmgr parity contract.
/// </summary>
[Fact]
public void SnapshotActiveAlarms_PropagatesDegradedAndSourceProvider()
{
FakeAlarmConsumer consumer = new FakeAlarmConsumer();
DateTime ts = new DateTime(2026, 5, 1, 17, 26, 14, 709, DateTimeKind.Utc);
consumer.SnapshotResult = new[]
{
new MxAlarmSnapshotRecord
{
AlarmGuid = Guid.NewGuid(),
ProviderName = "Galaxy",
Group = "TestArea",
TagName = "Tag1",
Type = "DSC",
Priority = 500,
State = MxAlarmStateKind.UnackAlm,
TransitionTimestampUtc = ts,
Degraded = true,
},
new MxAlarmSnapshotRecord
{
AlarmGuid = Guid.NewGuid(),
ProviderName = "Galaxy",
Group = "TestArea",
TagName = "Tag2",
Type = "ANL",
Priority = 100,
State = MxAlarmStateKind.UnackAlm,
TransitionTimestampUtc = ts,
Degraded = false,
},
};
using AlarmDispatcher dispatcher = new AlarmDispatcher(
consumer,
new MxAccessAlarmEventSink(new MxAccessEventQueue(), new MxAccessEventMapper()),
SessionId);
IReadOnlyList<ActiveAlarmSnapshot> snapshots = dispatcher.SnapshotActiveAlarms();
Assert.Equal(2, snapshots.Count);
Assert.True(snapshots[0].Degraded);
Assert.Equal(AlarmProviderMode.Subtag, snapshots[0].SourceProvider);
Assert.False(snapshots[1].Degraded);
Assert.Equal(AlarmProviderMode.Alarmmgr, snapshots[1].SourceProvider);
}
/// <summary>Verifies that dispose unsubscribes the handler and disposes the consumer.</summary>
[Fact]
public void Dispose_WhenSubscribed_UnsubscribesHandlerAndDisposesConsumer()
@@ -0,0 +1,294 @@
using System;
using System.Collections.Generic;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Worker.MxAccess;
using Xunit;
namespace ZB.MOM.WW.MxGateway.Worker.Tests.MxAccess;
/// <summary>
/// Unit tests for <see cref="FailoverAlarmConsumer"/>: prove the
/// auto-failover (consecutive primary COM failures → standby) and
/// auto-failback (consecutive clean probes → primary) state machine,
/// active-child transition forwarding, and active-child delegation of
/// acknowledgments. Fakes stand in for both children so this needs no
/// AVEVA install.
/// </summary>
public sealed class FailoverAlarmConsumerTests
{
/// <summary>
/// Primary fake whose Subscribe/PollOnce throw a COMException while
/// <see cref="ThrowOnPoll"/> is set, modeling a wnwrap consumer that
/// surfaces COM HRESULT failures. Can also re-raise a transition so
/// before-failover forwarding can be exercised.
/// </summary>
private sealed class FlakyPrimary : IMxAccessAlarmConsumer
{
public event EventHandler<MxAlarmTransitionEvent>? AlarmTransitionEmitted;
public bool ThrowOnPoll = true;
public int Polls;
/// <summary>
/// Number of times <see cref="Subscribe"/> has been called.
/// Incremented at entry, before any throw, so every attempt is
/// counted regardless of whether <see cref="ThrowOnPoll"/> is set.
/// </summary>
public int SubscribeCount;
public void Subscribe(string s)
{
SubscribeCount++;
if (ThrowOnPoll)
{
throw new System.Runtime.InteropServices.COMException("boom", unchecked((int)0x80004005));
}
}
public void PollOnce()
{
Polls++;
if (ThrowOnPoll)
{
throw new System.Runtime.InteropServices.COMException("boom", unchecked((int)0x80004005));
}
}
public int AcknowledgeByGuid(Guid g, string c, string a, string b, string d, string e) => 11;
public int AcknowledgeByName(string n, string p, string gr, string c, string a, string b, string d, string e) => 11;
public IReadOnlyList<MxAlarmSnapshotRecord> SnapshotActiveAlarms() => Array.Empty<MxAlarmSnapshotRecord>();
public void Dispose() { }
public void Raise(MxAlarmTransitionEvent e) => AlarmTransitionEmitted?.Invoke(this, e);
}
/// <summary>
/// Standby fake (subtag stand-in): never throws, records that it was
/// armed, and can re-raise a transition.
/// </summary>
private sealed class StubStandby : IMxAccessAlarmConsumer
{
public event EventHandler<MxAlarmTransitionEvent>? AlarmTransitionEmitted;
public bool Subscribed;
public void Subscribe(string s) => Subscribed = true;
public void PollOnce() { }
public int AcknowledgeByGuid(Guid g, string c, string a, string b, string d, string e) => 22;
public int AcknowledgeByName(string n, string p, string gr, string c, string a, string b, string d, string e) => 22;
public IReadOnlyList<MxAlarmSnapshotRecord> SnapshotActiveAlarms() => Array.Empty<MxAlarmSnapshotRecord>();
public void Dispose() { }
public void Raise(MxAlarmTransitionEvent e) => AlarmTransitionEmitted?.Invoke(this, e);
}
private static MxAlarmTransitionEvent SampleTransition() => new MxAlarmTransitionEvent
{
Record = new MxAlarmSnapshotRecord { AlarmGuid = Guid.NewGuid() },
PreviousState = MxAlarmStateKind.Unspecified,
};
[Fact]
public void Primary_FailsThresholdTimes_SwitchesToSubtag()
{
FlakyPrimary primary = new FlakyPrimary { ThrowOnPoll = true };
StubStandby standby = new StubStandby();
FailoverSettings settings = new FailoverSettings(threshold: 3, probeIntervalSeconds: 0, stableProbes: 1);
using FailoverAlarmConsumer sut = new FailoverAlarmConsumer(primary, standby, settings);
List<AlarmProviderModeChange> changes = new List<AlarmProviderModeChange>();
sut.ProviderModeChanged += (_, e) => changes.Add(e);
sut.Subscribe(@"\\HOST\Galaxy!Area"); // failure 1 (primary), standby armed
Assert.True(standby.Subscribed);
Assert.Empty(changes);
sut.PollOnce(); // failure 2
Assert.Empty(changes);
sut.PollOnce(); // failure 3 → switch
Assert.Single(changes);
Assert.Equal(AlarmProviderMode.Subtag, changes[0].Mode);
Assert.Equal(AlarmProviderMode.Subtag, sut.Mode);
Assert.Equal(unchecked((int)0x80004005), changes[0].HResult);
}
[Fact]
public void AfterSwitch_StandbyTransitionsAreForwarded()
{
FlakyPrimary primary = new FlakyPrimary { ThrowOnPoll = true };
StubStandby standby = new StubStandby();
FailoverSettings settings = new FailoverSettings(threshold: 1, probeIntervalSeconds: 0, stableProbes: 1);
using FailoverAlarmConsumer sut = new FailoverAlarmConsumer(primary, standby, settings);
MxAlarmTransitionEvent? forwarded = null;
sut.AlarmTransitionEmitted += (_, e) => forwarded = e;
sut.Subscribe(@"\\HOST\Galaxy!Area"); // threshold=1 → switch to Subtag immediately
Assert.Equal(AlarmProviderMode.Subtag, sut.Mode);
MxAlarmTransitionEvent transition = SampleTransition();
standby.Raise(transition);
Assert.Same(transition, forwarded);
}
[Fact]
public void WhileDegraded_PrimaryHeals_FailsBackAfterStableProbes()
{
// threshold=1 so the initial Subscribe failure (PollOnce path) immediately
// switches to Subtag. stableProbes=2 means two consecutive clean PollOnce
// calls are needed before failback. ProbeOnce must NOT call Subscribe —
// WnWrapAlarmConsumer is single-subscribe; re-calling would always throw.
FlakyPrimary primary = new FlakyPrimary { ThrowOnPoll = true };
StubStandby standby = new StubStandby();
FailoverSettings settings = new FailoverSettings(threshold: 1, probeIntervalSeconds: 0, stableProbes: 2);
using FailoverAlarmConsumer sut = new FailoverAlarmConsumer(primary, standby, settings);
List<AlarmProviderModeChange> changes = new List<AlarmProviderModeChange>();
sut.ProviderModeChanged += (_, e) => changes.Add(e);
sut.Subscribe(@"\\HOST\Galaxy!Area"); // threshold=1 → Subtag (mode change 1)
Assert.Single(changes);
Assert.Equal(AlarmProviderMode.Subtag, changes[changes.Count - 1].Mode);
// Primary heals: PollOnce stops throwing. ProbeOnce should call only
// PollOnce (not Subscribe) to detect recovery.
primary.ThrowOnPoll = false;
int subscribeCountAfterFailover = primary.SubscribeCount;
sut.ProbeOnce(); // cleanProbes=1 — not yet at stableProbes=2
Assert.Single(changes);
sut.ProbeOnce(); // cleanProbes=2 → failback to Alarmmgr (mode change 2)
Assert.Equal(2, changes.Count);
Assert.Equal(AlarmProviderMode.Alarmmgr, changes[changes.Count - 1].Mode);
Assert.Equal(AlarmProviderMode.Alarmmgr, sut.Mode);
Assert.Equal(0, changes[changes.Count - 1].HResult);
// ProbeOnce must not have called Subscribe at all during probing.
Assert.Equal(subscribeCountAfterFailover, primary.SubscribeCount);
}
[Fact]
public void BeforeFailover_PrimaryTransitionsAreForwarded()
{
FlakyPrimary primary = new FlakyPrimary { ThrowOnPoll = false }; // healthy, can Raise
StubStandby standby = new StubStandby();
FailoverSettings settings = new FailoverSettings(threshold: 3, probeIntervalSeconds: 0, stableProbes: 1);
using FailoverAlarmConsumer sut = new FailoverAlarmConsumer(primary, standby, settings);
List<MxAlarmTransitionEvent> forwarded = new List<MxAlarmTransitionEvent>();
sut.AlarmTransitionEmitted += (_, e) => forwarded.Add(e);
sut.Subscribe(@"\\HOST\Galaxy!Area");
Assert.Equal(AlarmProviderMode.Alarmmgr, sut.Mode);
MxAlarmTransitionEvent fromPrimary = SampleTransition();
primary.Raise(fromPrimary); // active=Primary → forwarded
Assert.Single(forwarded);
Assert.Same(fromPrimary, forwarded[0]);
standby.Raise(SampleTransition()); // standby not active → suppressed
Assert.Single(forwarded);
}
/// <summary>
/// Proves that <see cref="FailoverAlarmConsumer.ProbeOnce"/> never calls
/// <c>Subscribe</c> on the primary while degraded. The production primary
/// (<see cref="WnWrapAlarmConsumer"/>) is single-subscribe; a second
/// <c>Subscribe</c> call would always throw and make failback impossible.
/// The probe must re-poll the still-subscribed primary via
/// <c>PollOnce</c> only.
/// </summary>
[Fact]
public void ProbeOnce_DoesNotCallPrimarySubscribe()
{
// threshold=1 → first Subscribe failure immediately switches to Subtag.
FlakyPrimary primary = new FlakyPrimary { ThrowOnPoll = true };
StubStandby standby = new StubStandby();
FailoverSettings settings = new FailoverSettings(threshold: 1, probeIntervalSeconds: 0, stableProbes: 3);
using FailoverAlarmConsumer sut = new FailoverAlarmConsumer(primary, standby, settings);
sut.Subscribe(@"\\HOST\Galaxy!Area"); // Subscribe attempt #1 (throws) → Subtag
// Capture how many Subscribe calls the initial setup caused (exactly 1:
// the attempt that threw and triggered failover).
int subscribeCountAfterSetup = primary.SubscribeCount;
Assert.Equal(1, subscribeCountAfterSetup);
Assert.Equal(AlarmProviderMode.Subtag, sut.Mode);
// Let PollOnce succeed so ProbeOnce progresses without throwing.
primary.ThrowOnPoll = false;
// Drive several ProbeOnce calls — none should touch Subscribe.
sut.ProbeOnce();
sut.ProbeOnce();
sut.ProbeOnce(); // stableProbes=3 → failback on this call
Assert.Equal(AlarmProviderMode.Alarmmgr, sut.Mode);
Assert.Equal(subscribeCountAfterSetup, primary.SubscribeCount);
}
[Fact]
public void Acknowledge_DelegatesToActiveChild()
{
FlakyPrimary primary = new FlakyPrimary { ThrowOnPoll = false };
StubStandby standby = new StubStandby();
FailoverSettings settings = new FailoverSettings(threshold: 1, probeIntervalSeconds: 0, stableProbes: 1);
using FailoverAlarmConsumer sut = new FailoverAlarmConsumer(primary, standby, settings);
sut.Subscribe(@"\\HOST\Galaxy!Area");
// Active = Primary → primary's sentinel value (11).
Assert.Equal(11, sut.AcknowledgeByGuid(Guid.NewGuid(), "c", "n", "node", "dom", "full"));
Assert.Equal(11, sut.AcknowledgeByName("a", "p", "g", "c", "n", "node", "dom", "full"));
// Force a failover by failing the primary past threshold.
primary.ThrowOnPoll = true;
sut.PollOnce(); // threshold=1 → switch to Standby
Assert.Equal(AlarmProviderMode.Subtag, sut.Mode);
// Active = Standby → standby's sentinel value (22).
Assert.Equal(22, sut.AcknowledgeByGuid(Guid.NewGuid(), "c", "n", "node", "dom", "full"));
Assert.Equal(22, sut.AcknowledgeByName("a", "p", "g", "c", "n", "node", "dom", "full"));
}
/// <summary>
/// Proves that an intermittent failure during failback probing resets the
/// clean-probe counter to zero, requiring a fresh unbroken run of
/// <see cref="FailoverSettings.StableProbes"/> before failing back.
/// </summary>
[Fact]
public void FailbackProbe_IntermittentFailure_ResetsCleanCount()
{
var primary = new FlakyPrimary { ThrowOnPoll = true };
var standby = new StubStandby();
using var sut = new FailoverAlarmConsumer(primary, standby, new FailoverSettings(threshold: 1, probeIntervalSeconds: 0, stableProbes: 3));
sut.Subscribe(@"\\HOST\Galaxy!Area"); // threshold=1 → switch to Subtag
Assert.Equal(AlarmProviderMode.Subtag, sut.Mode);
primary.ThrowOnPoll = false;
sut.ProbeOnce(); // clean 1
sut.ProbeOnce(); // clean 2
primary.ThrowOnPoll = true;
sut.ProbeOnce(); // fails → reset to 0
Assert.Equal(AlarmProviderMode.Subtag, sut.Mode);
primary.ThrowOnPoll = false;
sut.ProbeOnce(); // clean 1
sut.ProbeOnce(); // clean 2
sut.ProbeOnce(); // clean 3 → failback
Assert.Equal(AlarmProviderMode.Alarmmgr, sut.Mode);
}
}
@@ -0,0 +1,226 @@
using System;
using System.Collections.Generic;
using ZB.MOM.WW.MxGateway.Worker.MxAccess;
namespace ZB.MOM.WW.MxGateway.Worker.Tests.MxAccess;
/// <summary>
/// Unit-test coverage for <see cref="LmxSubtagAlarmSource"/>'s advise/write
/// sequencing and its <c>OnDataChange</c> normalization. The actual
/// <c>LMXProxyServerClass</c> COM event subscription cannot be exercised
/// without a live MXAccess install, so these tests drive the source through
/// its internal <see cref="IMxAccessServer"/> seam and call
/// <c>HandleDataChange</c> directly to simulate a COM callback — exactly the
/// boundary <c>MxAccessBaseEventSink.OnDataChange</c> uses for the
/// per-session pipeline. End-to-end COM delivery is covered by the
/// Skip-gated alarm live smoke tests.
/// </summary>
public sealed class LmxSubtagAlarmSourceTests
{
private const int FakeServerHandle = 7;
/// <summary>Verifies the production constructor rejects a null factory.</summary>
[Fact]
public void Constructor_NullFactory_Throws()
{
Assert.Throws<ArgumentNullException>(() => new LmxSubtagAlarmSource(factory: null!));
}
/// <summary>
/// Verifies <see cref="LmxSubtagAlarmSource.Advise"/> calls AddItem then
/// Advise once per distinct address, and is idempotent on a repeated
/// address.
/// </summary>
[Fact]
public void Advise_AddsAndAdvisesEachAddressOnce()
{
var server = new RecordingMxAccessServer();
using var source = new LmxSubtagAlarmSource(server, FakeServerHandle);
source.Advise(new[] { "Tank1.Alarm.Subtag", "Tank2.Alarm.Subtag" });
// Re-advising an already-advised address is a no-op.
source.Advise(new[] { "Tank1.Alarm.Subtag" });
Assert.Equal(
new[] { "Tank1.Alarm.Subtag", "Tank2.Alarm.Subtag" },
server.AddedItems);
Assert.Equal(2, server.AdviseCount);
// Every advise targeted the supplied server handle.
Assert.All(server.AdvisedServerHandles, h => Assert.Equal(FakeServerHandle, h));
}
/// <summary>
/// Verifies a simulated <c>OnDataChange</c> for an advised item handle
/// raises <see cref="LmxSubtagAlarmSource.ValueChanged"/> with the
/// address that was advised and the delivered value.
/// </summary>
[Fact]
public void HandleDataChange_RaisesValueChangedWithAdvisedAddress()
{
var server = new RecordingMxAccessServer();
using var source = new LmxSubtagAlarmSource(server, FakeServerHandle);
source.Advise(new[] { "Tank1.Alarm.Subtag" });
int itemHandle = server.LastItemHandleFor("Tank1.Alarm.Subtag");
SubtagValueChange? received = null;
source.ValueChanged += (_, change) => received = change;
source.HandleDataChange(itemHandle, pvItemValue: 42, pftItemTimeStamp: null);
Assert.NotNull(received);
Assert.Equal("Tank1.Alarm.Subtag", received!.ItemAddress);
Assert.Equal(42, received.Value);
Assert.Equal(DateTimeKind.Utc, received.TimestampUtc.Kind);
}
/// <summary>
/// Verifies <c>OnDataChange</c> for an unknown item handle is ignored
/// (no <see cref="LmxSubtagAlarmSource.ValueChanged"/> raised).
/// </summary>
[Fact]
public void HandleDataChange_UnknownHandle_DoesNotRaise()
{
var server = new RecordingMxAccessServer();
using var source = new LmxSubtagAlarmSource(server, FakeServerHandle);
bool raised = false;
source.ValueChanged += (_, _) => raised = true;
source.HandleDataChange(phItemHandle: 999, pvItemValue: 1, pftItemTimeStamp: null);
Assert.False(raised);
}
/// <summary>
/// Verifies <see cref="LmxSubtagAlarmSource.Write"/> adds the item when
/// it was not previously advised and writes with user id 0.
/// </summary>
[Fact]
public void Write_AddsItemWhenUnknownAndWrites()
{
var server = new RecordingMxAccessServer();
using var source = new LmxSubtagAlarmSource(server, FakeServerHandle);
source.Write("Tank1.Alarm.AckComment", "acknowledged");
Assert.Contains("Tank1.Alarm.AckComment", server.AddedItems);
Assert.Single(server.Writes);
RecordingMxAccessServer.WriteRecord write = server.Writes[0];
Assert.Equal(FakeServerHandle, write.ServerHandle);
Assert.Equal("acknowledged", write.Value);
Assert.Equal(0, write.UserId);
}
/// <summary>
/// Verifies <see cref="LmxSubtagAlarmSource.Write"/> reuses an existing
/// item handle (no duplicate AddItem) when the address was already
/// advised.
/// </summary>
[Fact]
public void Write_ReusesHandleForAdvisedAddress()
{
var server = new RecordingMxAccessServer();
using var source = new LmxSubtagAlarmSource(server, FakeServerHandle);
source.Advise(new[] { "Tank1.Alarm.Subtag" });
int adviseAddCount = server.AddedItems.Count;
source.Write("Tank1.Alarm.Subtag", true);
// No second AddItem for the already-advised address.
Assert.Equal(adviseAddCount, server.AddedItems.Count);
Assert.Single(server.Writes);
}
/// <summary>
/// Recording <see cref="IMxAccessServer"/> test double that captures the
/// AddItem/Advise/Write/UnAdvise/RemoveItem/Unregister calls
/// <see cref="LmxSubtagAlarmSource"/> makes and hands out monotonically
/// increasing item handles.
/// </summary>
private sealed class RecordingMxAccessServer : IMxAccessServer
{
private readonly Dictionary<string, int> handlesByAddress = new(StringComparer.Ordinal);
private int nextItemHandle = 100;
public List<string> AddedItems { get; } = new();
public int AdviseCount { get; private set; }
public List<int> AdvisedServerHandles { get; } = new();
public List<WriteRecord> Writes { get; } = new();
public int LastItemHandleFor(string itemAddress) => handlesByAddress[itemAddress];
public int Register(string clientName) => FakeServerHandle;
public void Unregister(int serverHandle)
{
}
public int AddItem(int serverHandle, string itemDefinition)
{
AddedItems.Add(itemDefinition);
int handle = nextItemHandle++;
handlesByAddress[itemDefinition] = handle;
return handle;
}
public int AddItem2(int serverHandle, string itemDefinition, string itemContext)
=> AddItem(serverHandle, itemDefinition);
public void RemoveItem(int serverHandle, int itemHandle)
{
}
public void Advise(int serverHandle, int itemHandle)
{
AdviseCount++;
AdvisedServerHandles.Add(serverHandle);
}
public void UnAdvise(int serverHandle, int itemHandle)
{
}
public void AdviseSupervisory(int serverHandle, int itemHandle)
{
}
public void Write(int serverHandle, int itemHandle, object? value, int userId)
=> Writes.Add(new WriteRecord(serverHandle, itemHandle, value, userId));
public void Write2(int serverHandle, int itemHandle, object? value, object? timestamp, int userId)
{
}
public void WriteSecured(int serverHandle, int itemHandle, int currentUserId, int verifierUserId, object? value)
{
}
public void WriteSecured2(int serverHandle, int itemHandle, int currentUserId, int verifierUserId, object? value, object? timestamp)
{
}
internal sealed class WriteRecord
{
public WriteRecord(int serverHandle, int itemHandle, object? value, int userId)
{
ServerHandle = serverHandle;
ItemHandle = itemHandle;
Value = value;
UserId = userId;
}
public int ServerHandle { get; }
public int ItemHandle { get; }
public object? Value { get; }
public int UserId { get; }
}
}
}
@@ -200,7 +200,7 @@ public sealed class MxAccessStaSessionTests
factory,
eventSink,
new MxAccessEventQueue(),
(_eq, _affinity) => handler);
(_eq, _affinity, _comFactory) => handler);
await session.StartAsync("session-1", workerProcessId: 1);
@@ -279,7 +279,7 @@ public sealed class MxAccessStaSessionTests
factory,
eventSink,
new MxAccessEventQueue(),
(_eq, _affinity) => handler);
(_eq, _affinity, _comFactory) => handler);
await session.StartAsync("session-1", workerProcessId: 1);
@@ -320,7 +320,7 @@ public sealed class MxAccessStaSessionTests
factory,
eventSink,
new MxAccessEventQueue(),
(_eq, _affinity) => handler);
(_eq, _affinity, _comFactory) => handler);
await session.StartAsync("session-1", workerProcessId: 1);
@@ -369,7 +369,7 @@ public sealed class MxAccessStaSessionTests
factory,
eventSink,
eventQueue,
(_eq, _affinity) => handler);
(_eq, _affinity, _comFactory) => handler);
await session.StartAsync("session-1", workerProcessId: 1);
@@ -416,7 +416,7 @@ public sealed class MxAccessStaSessionTests
factory,
eventSink,
eventQueue,
(_eq, _affinity) => handler);
(_eq, _affinity, _comFactory) => handler);
await session.StartAsync("session-1", workerProcessId: 1);
@@ -496,12 +496,12 @@ public sealed class MxAccessStaSessionTests
}
/// <summary>Subscribes to alarm events.</summary>
/// <param name="subscription">The subscription descriptor.</param>
/// <param name="command">The subscribe-alarms command.</param>
/// <param name="sessionId">The session identifier.</param>
public void Subscribe(string subscription, string sessionId)
public void Subscribe(SubscribeAlarmsCommand command, string sessionId)
{
IsSubscribed = true;
LastSubscription = subscription;
LastSubscription = command.SubscriptionExpression;
}
/// <summary>Unsubscribes from alarm events.</summary>
@@ -0,0 +1,331 @@
using System;
using System.Collections.Generic;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Worker.MxAccess;
using Xunit;
namespace ZB.MOM.WW.MxGateway.Worker.Tests.MxAccess;
/// <summary>
/// Unit tests for <see cref="SubtagAlarmConsumer"/>: prove that the subtag
/// fallback advises the observable subtags, synthesizes degraded
/// transitions with stable synthetic GUIDs, routes acknowledgments to the
/// ack-comment subtag, and stamps snapshots. A <see cref="FakeSource"/>
/// stands in for the live MXAccess subtag source so this needs no AVEVA
/// install.
/// </summary>
public sealed class SubtagAlarmConsumerTests
{
private const string Reference = "Galaxy!TestArea.Tank01.Level.HiHi";
private const string ActiveSubtag = "Tank01.Level.HiHi.InAlarm";
private const string AckedSubtag = "Tank01.Level.HiHi.Acked";
private const string AckCommentSubtag = "Tank01.Level.HiHi.AckComment";
private const string PrioritySubtag = "Tank01.Level.HiHi.Priority";
private static AlarmSubtagTarget BuildTarget()
{
return new AlarmSubtagTarget
{
AlarmFullReference = Reference,
SourceObjectReference = "Tank01",
ActiveSubtag = ActiveSubtag,
AckedSubtag = AckedSubtag,
AckCommentSubtag = AckCommentSubtag,
PrioritySubtag = PrioritySubtag,
};
}
private static SubtagAlarmConsumer BuildConsumer(FakeSource source)
{
return new SubtagAlarmConsumer(source, new[] { BuildTarget() });
}
/// <summary>Verifies Subscribe advises the active, acked, and priority subtags but not the ack-comment subtag.</summary>
[Fact]
public void Subscribe_AdvisesActiveAndAckedSubtags()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
Assert.Contains(ActiveSubtag, source.Advised);
Assert.Contains(AckedSubtag, source.Advised);
Assert.Contains(PrioritySubtag, source.Advised);
Assert.DoesNotContain(AckCommentSubtag, source.Advised);
}
/// <summary>Verifies an active=true value change raises a degraded, GUID-stamped UNACK_ALM transition.</summary>
[Fact]
public void ValueChange_RaisesDegradedSynthesizedTransition()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
MxAlarmTransitionEvent? emitted = null;
consumer.AlarmTransitionEmitted += (_, e) => emitted = e;
source.Raise(ActiveSubtag, true, new DateTime(2026, 6, 13, 10, 0, 0, DateTimeKind.Utc));
Assert.NotNull(emitted);
Assert.Equal(MxAlarmStateKind.UnackAlm, emitted!.Record.State);
Assert.True(emitted.Record.Degraded);
Assert.NotEqual(Guid.Empty, emitted.Record.AlarmGuid);
}
/// <summary>Verifies AcknowledgeByName writes the comment to the ack-comment subtag and returns success.</summary>
[Fact]
public void AcknowledgeByName_WritesCommentToAckCommentSubtag()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
int rc = consumer.AcknowledgeByName(
alarmName: "Tank01.Level.HiHi",
providerName: "Galaxy",
groupName: "TestArea",
ackComment: "operator ack",
ackOperatorName: "alice",
ackOperatorNode: "WS01",
ackOperatorDomain: "CORP",
ackOperatorFullName: "Alice Smith");
Assert.Equal(0, rc);
Assert.NotNull(source.LastWrite);
Assert.Equal(AckCommentSubtag, source.LastWrite!.Value.Address);
Assert.Equal("operator ack", source.LastWrite!.Value.Value);
}
/// <summary>Verifies AcknowledgeByName returns non-zero when no target matches the supplied name.</summary>
[Fact]
public void AcknowledgeByName_UnknownAlarm_ReturnsNonZero()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
int rc = consumer.AcknowledgeByName(
alarmName: "DoesNotExist.NoSuchAlarm",
providerName: "Galaxy",
groupName: "TestArea",
ackComment: "operator ack",
ackOperatorName: "alice",
ackOperatorNode: "WS01",
ackOperatorDomain: "CORP",
ackOperatorFullName: "Alice Smith");
Assert.NotEqual(0, rc);
Assert.Null(source.LastWrite);
}
/// <summary>Verifies a snapshot of an active alarm stamps Degraded and a non-empty synthetic GUID.</summary>
[Fact]
public void SnapshotActiveAlarms_StampsDegradedAndGuid()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
source.Raise(ActiveSubtag, true, new DateTime(2026, 6, 13, 10, 0, 0, DateTimeKind.Utc));
IReadOnlyList<MxAlarmSnapshotRecord> snapshot = consumer.SnapshotActiveAlarms();
Assert.Single(snapshot);
Assert.True(snapshot[0].Degraded);
Assert.NotEqual(Guid.Empty, snapshot[0].AlarmGuid);
}
/// <summary>Verifies the synthetic GUID on the emitted transition equals the GUID in the snapshot for the same alarm.</summary>
[Fact]
public void SameReference_SyntheticGuidStableAcrossTransitionAndSnapshot()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
MxAlarmTransitionEvent? emitted = null;
consumer.AlarmTransitionEmitted += (_, e) => emitted = e;
source.Raise(ActiveSubtag, true, new DateTime(2026, 6, 13, 10, 0, 0, DateTimeKind.Utc));
IReadOnlyList<MxAlarmSnapshotRecord> snapshot = consumer.SnapshotActiveAlarms();
Assert.NotNull(emitted);
Assert.Single(snapshot);
Assert.Equal(emitted!.Record.AlarmGuid, snapshot[0].AlarmGuid);
Assert.NotEqual(Guid.Empty, emitted.Record.AlarmGuid);
}
/// <summary>
/// Verifies that when two alarm targets share a prefix (e.g. Level.Hi vs Level.HiHi),
/// AcknowledgeByName routes each ack to its own ack-comment subtag and never
/// conflates the shorter name with the longer one.
/// </summary>
[Fact]
public void AcknowledgeByName_PrefixNameDoesNotFalseMatch()
{
const string ReferenceHi = "Galaxy!Area.Tank01.Level.Hi";
const string ReferenceHiHi = "Galaxy!Area.Tank01.Level.HiHi";
const string AckCommentHi = "Tank01.Level.Hi.AckComment";
const string AckCommentHiHi = "Tank01.Level.HiHi.AckComment";
AlarmSubtagTarget targetHi = new AlarmSubtagTarget
{
AlarmFullReference = ReferenceHi,
SourceObjectReference = "Tank01",
ActiveSubtag = "Tank01.Level.Hi.InAlarm",
AckedSubtag = "Tank01.Level.Hi.Acked",
AckCommentSubtag = AckCommentHi,
PrioritySubtag = "Tank01.Level.Hi.Priority",
};
AlarmSubtagTarget targetHiHi = new AlarmSubtagTarget
{
AlarmFullReference = ReferenceHiHi,
SourceObjectReference = "Tank01",
ActiveSubtag = "Tank01.Level.HiHi.InAlarm",
AckedSubtag = "Tank01.Level.HiHi.Acked",
AckCommentSubtag = AckCommentHiHi,
PrioritySubtag = "Tank01.Level.HiHi.Priority",
};
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = new SubtagAlarmConsumer(
source, new[] { targetHi, targetHiHi });
consumer.Subscribe(@"\\HOST\Galaxy!Area");
// Ack the shorter name — must write to the shorter target's subtag only.
int rcHi = consumer.AcknowledgeByName(
alarmName: "Tank01.Level.Hi",
providerName: "Galaxy",
groupName: "Area",
ackComment: "ack hi",
ackOperatorName: "op", ackOperatorNode: "WS01",
ackOperatorDomain: "CORP", ackOperatorFullName: "Operator");
Assert.Equal(0, rcHi);
Assert.NotNull(source.LastWrite);
Assert.Equal(AckCommentHi, source.LastWrite!.Value.Address);
Assert.Equal("ack hi", source.LastWrite.Value.Value);
// Ack the longer name — must write to the longer target's subtag.
int rcHiHi = consumer.AcknowledgeByName(
alarmName: "Tank01.Level.HiHi",
providerName: "Galaxy",
groupName: "Area",
ackComment: "ack hihi",
ackOperatorName: "op", ackOperatorNode: "WS01",
ackOperatorDomain: "CORP", ackOperatorFullName: "Operator");
Assert.Equal(0, rcHiHi);
Assert.NotNull(source.LastWrite);
Assert.Equal(AckCommentHiHi, source.LastWrite!.Value.Address);
Assert.Equal("ack hihi", source.LastWrite.Value.Value);
}
/// <summary>
/// Verifies AcknowledgeByGuid resolves the synthetic GUID (computed from
/// the alarm's full reference) to the correct target and writes the comment
/// to that target's ack-comment subtag.
/// </summary>
[Fact]
public void AcknowledgeByGuid_WritesCommentToAckCommentSubtag()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
// Raise a transition so the state machine sees the alarm, then capture
// the GUID stamped on the emitted event.
MxAlarmTransitionEvent? emitted = null;
consumer.AlarmTransitionEmitted += (_, e) => emitted = e;
source.Raise(ActiveSubtag, true, new DateTime(2026, 6, 13, 10, 0, 0, DateTimeKind.Utc));
Assert.NotNull(emitted);
Guid syntheticGuid = emitted!.Record.AlarmGuid;
Assert.NotEqual(Guid.Empty, syntheticGuid);
int rc = consumer.AcknowledgeByGuid(
alarmGuid: syntheticGuid,
ackComment: "guid ack",
ackOperatorName: "op",
ackOperatorNode: "WS01",
ackOperatorDomain: "CORP",
ackOperatorFullName: "Operator");
Assert.Equal(0, rc);
Assert.NotNull(source.LastWrite);
Assert.Equal(AckCommentSubtag, source.LastWrite!.Value.Address);
Assert.Equal("guid ack", source.LastWrite.Value.Value);
}
/// <summary>
/// Verifies AcknowledgeByGuid returns non-zero and performs no write when
/// the supplied GUID is not known to the consumer.
/// </summary>
[Fact]
public void AcknowledgeByGuid_UnknownGuid_ReturnsNonZero()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
int rc = consumer.AcknowledgeByGuid(
alarmGuid: Guid.NewGuid(),
ackComment: "should not write",
ackOperatorName: "op",
ackOperatorNode: "WS01",
ackOperatorDomain: "CORP",
ackOperatorFullName: "Operator");
Assert.NotEqual(0, rc);
Assert.Null(source.LastWrite);
}
private sealed class FakeSource : ISubtagAlarmSource
{
/// <summary>Raised when an advised subtag reports a new value.</summary>
public event EventHandler<SubtagValueChange>? ValueChanged;
/// <summary>Gets the subtag addresses passed to <see cref="Advise"/>.</summary>
public List<string> Advised { get; } = new List<string>();
/// <summary>Gets the most recent (address, value) pair passed to <see cref="Write"/>.</summary>
public (string Address, object? Value)? LastWrite { get; private set; }
/// <summary>Records the advised subtag addresses.</summary>
/// <param name="itemAddresses">The subtag references to advise.</param>
public void Advise(IReadOnlyCollection<string> itemAddresses)
{
Advised.AddRange(itemAddresses);
}
/// <summary>Records the most recent write.</summary>
/// <param name="itemAddress">The subtag reference to write.</param>
/// <param name="value">The value to write.</param>
public void Write(string itemAddress, object? value)
{
LastWrite = (itemAddress, value);
}
/// <summary>Raises a <see cref="SubtagValueChange"/> for the given subtag.</summary>
/// <param name="address">The subtag address whose value changed.</param>
/// <param name="value">The new value.</param>
/// <param name="timestampUtc">The UTC timestamp of the change.</param>
public void Raise(string address, object? value, DateTime timestampUtc)
{
ValueChanged?.Invoke(this, new SubtagValueChange
{
ItemAddress = address,
Value = value,
TimestampUtc = timestampUtc,
});
}
/// <inheritdoc />
public void Dispose()
{
}
}
}
@@ -0,0 +1,143 @@
using System;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Worker.MxAccess;
using Xunit;
namespace ZB.MOM.WW.MxGateway.Worker.Tests.MxAccess;
/// <summary>
/// Unit tests for the subtag-fallback synthesis state machine. The machine
/// consumes normalized subtag value changes (active/acked/priority) and
/// emits <see cref="MxAlarmTransitionEvent"/> records mirroring the wnwrap
/// consumer's UNACK_ALM / ACK_ALM / UNACK_RTN / ACK_RTN transitions. No COM
/// or AVEVA install is required.
/// </summary>
public sealed class SubtagAlarmStateMachineTests
{
private static AlarmSubtagTarget Target() => new()
{
AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi",
SourceObjectReference = "Tank01",
ActiveSubtag = "Tank01.Level.HiHi.active",
AckedSubtag = "Tank01.Level.HiHi.acked",
AckCommentSubtag = "Tank01.Level.HiHi.ackmsg",
};
[Fact]
public void ActiveFalseToTrue_EmitsRaise()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
var events = sm.Apply("Tank01.Level.HiHi.active", true, ts);
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.UnackAlm, e.Record.State);
Assert.Equal(MxAlarmStateKind.Unspecified, e.PreviousState);
Assert.Equal("Tank01.Level.HiHi", e.Record.TagName);
Assert.Equal("Galaxy", e.Record.ProviderName);
Assert.Equal("Area", e.Record.Group);
}
[Fact]
public void ActiveFalseToTrue_NoProviderBang_UsesWholeReferenceAsTagName()
{
var target = new AlarmSubtagTarget
{
AlarmFullReference = "Tank01.Level.HiHi",
SourceObjectReference = string.Empty,
ActiveSubtag = "Tank01.Level.HiHi.active",
};
var sm = new SubtagAlarmStateMachine(new[] { target });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
var events = sm.Apply("Tank01.Level.HiHi.active", true, ts);
var e = Assert.Single(events);
Assert.Equal("Tank01.Level.HiHi", e.Record.TagName);
Assert.Equal(string.Empty, e.Record.ProviderName);
Assert.Equal(string.Empty, e.Record.Group);
}
[Fact]
public void OutOfOrderAckThenClear_StillEmitsAckRtn()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
sm.Apply("Tank01.Level.HiHi.acked", true, ts.AddSeconds(2));
// Out-of-order un-ack arrives before the active=false clear.
sm.Apply("Tank01.Level.HiHi.acked", false, ts.AddSeconds(3));
var events = sm.Apply("Tank01.Level.HiHi.active", false, ts.AddSeconds(10));
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.AckRtn, e.Record.State);
}
[Fact]
public void DuplicateActiveSubtag_Throws()
{
var first = new AlarmSubtagTarget
{
AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi",
SourceObjectReference = "Tank01",
ActiveSubtag = "Shared.active",
};
var second = new AlarmSubtagTarget
{
AlarmFullReference = "Galaxy!Area.Tank02.Level.HiHi",
SourceObjectReference = "Tank02",
ActiveSubtag = "Shared.active",
};
Assert.Throws<ArgumentException>(() => new SubtagAlarmStateMachine(new[] { first, second }));
}
[Fact]
public void AckedTrueWhileActive_EmitsAck()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
var events = sm.Apply("Tank01.Level.HiHi.acked", true, ts.AddSeconds(5));
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.AckAlm, e.Record.State);
Assert.Equal(MxAlarmStateKind.UnackAlm, e.PreviousState);
}
[Fact]
public void ActiveTrueToFalse_WhileUnacked_EmitsUnackRtn()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
var events = sm.Apply("Tank01.Level.HiHi.active", false, ts.AddSeconds(10));
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.UnackRtn, e.Record.State);
}
[Fact]
public void ActiveTrueToFalse_WhileAcked_EmitsAckRtn()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
sm.Apply("Tank01.Level.HiHi.acked", true, ts.AddSeconds(2));
var events = sm.Apply("Tank01.Level.HiHi.active", false, ts.AddSeconds(10));
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.AckRtn, e.Record.State);
}
[Fact]
public void Snapshot_ReflectsActiveAndAckedState()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
sm.Apply("Tank01.Level.HiHi.acked", true, ts);
var snap = Assert.Single(sm.SnapshotActive());
Assert.Equal(MxAlarmStateKind.AckAlm, snap.State);
}
[Fact]
public void UnknownAddress_NoEvents()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var events = sm.Apply("Some.Other.Tag.active", true, DateTime.UtcNow);
Assert.Empty(events);
}
}
@@ -0,0 +1,27 @@
using System;
using ZB.MOM.WW.MxGateway.Worker.MxAccess;
namespace ZB.MOM.WW.MxGateway.Worker.Tests.MxAccess;
/// <summary>
/// Unit tests for <see cref="SyntheticAlarmGuid"/>: the subtag-fallback
/// path derives a deterministic GUID from the alarm reference, so identical
/// references must collide and distinct references must not.
/// </summary>
public sealed class SyntheticAlarmGuidTests
{
/// <summary>Verifies the same reference yields the same GUID.</summary>
[Fact]
public void SameReference_SameGuid() =>
Assert.Equal(SyntheticAlarmGuid.ForReference("A.B.C"), SyntheticAlarmGuid.ForReference("A.B.C"));
/// <summary>Verifies distinct references yield distinct GUIDs.</summary>
[Fact]
public void DifferentReference_DifferentGuid() =>
Assert.NotEqual(SyntheticAlarmGuid.ForReference("A.B.C"), SyntheticAlarmGuid.ForReference("A.B.D"));
/// <summary>Verifies a reference produces a non-empty GUID.</summary>
[Fact]
public void Reference_ProducesNonEmptyGuid() =>
Assert.NotEqual(Guid.Empty, SyntheticAlarmGuid.ForReference("A.B.C"));
}
@@ -0,0 +1,435 @@
// AlarmSubtagLiveSmokeTests.cs
//
// Validates the open design item from Task 17: exact subtag names and the
// canonical AlarmSubtagTarget reference shape for a known Galaxy alarm.
//
// This test exercises the full subtag-fallback pipeline end-to-end against
// a real Galaxy + MXAccess install:
// LmxSubtagAlarmSource (own LMXProxyServerClass) ->
// SubtagAlarmConsumer (state machine + AcknowledgeByName write) ->
// synthesized MxAlarmTransitionEvent (Raise / Clear, Degraded=true, SyntheticGuid)
//
// HOW TO RUN:
// 1. On the dev rig with AVEVA System Platform installed and Galaxy running:
// $env:MXGATEWAY_RUN_LIVE_MXACCESS_TESTS = "1"
// 2. Remove (or set to null) the Skip parameter on the [Fact] below.
// 3. Run with an alarm flip script (same one used by AlarmsLiveSmokeTests)
// so that TestMachine_001.TestAlarm001 toggles its Active/Acked subtags
// on a ~10 s cadence.
// 4. VERIFY the placeholder subtag names (PLACEHOLDER_* constants below)
// against the live Galaxy before flipping Skip -- see VERIFY comment block.
//
// OPEN ITEM (Task 17): the exact subtag address strings are not yet confirmed
// from a live Galaxy or from C:\Users\dohertj2\Desktop\mxaccess
// docs/MXAccess-Public-API.md. Flip this test only after verifying them.
//
// net48/x86 constraints:
// - No init-only properties, records, index/range operators, C# 8+ pattern
// matches beyond what the existing Worker.Tests files use.
// - All Booleans from MXAccess arrive as boxed int (0 / non-zero); coerce
// with Convert.ToBoolean or cast to int first.
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Threading;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Worker.MxAccess;
using Xunit.Abstractions;
namespace ZB.MOM.WW.MxGateway.Worker.Tests.Probes;
/// <summary>
/// Live dev-rig smoke test for the subtag-fallback alarm pipeline.
/// Validates the open design item from Task 17: confirms that
/// <see cref="LmxSubtagAlarmSource"/> wired to a real
/// <c>LMXProxyServerClass</c> + <see cref="SubtagAlarmConsumer"/> can
/// synthesize <see cref="MxAlarmTransitionEvent"/> records from Galaxy
/// alarm subtags, and that <see cref="SubtagAlarmConsumer.AcknowledgeByName"/>
/// writes the ack-comment subtag successfully.
///
/// Skip-gated; flip <c>Skip=null</c> on the dev rig with the alarm flip
/// script running. VERIFY the PLACEHOLDER_* subtag names before running.
/// </summary>
public sealed class AlarmSubtagLiveSmokeTests
{
// -------------------------------------------------------------------------
// PLACEHOLDER references — VERIFY before flipping Skip.
//
// These values are the best-guess subtag addresses for TestMachine_001.TestAlarm001
// as used by the existing AlarmsLiveSmokeTests (which watches the same Galaxy
// provider). The exact attribute suffix strings (.InAlarm, .Acked,
// .AckComment, .Priority) MUST be confirmed against:
// - C:\Users\dohertj2\Desktop\mxaccess\docs\MXAccess-Public-API.md
// - A live Galaxy attribute browse or MXTraceHarness capture
// before this test is flipped on. The subtag address format is
// <ObjectTagName>.<AlarmAttributeName>
// where the alarm attribute names are Galaxy-attribute names surfaced
// through MXAccess. Known AVEVA attribute names are .InAlarm (bool),
// .Acked (bool), .Priority (int), and a writable comment attribute whose
// exact name varies by alarm template (commonly .AlarmComment or
// .AckComment).
// -------------------------------------------------------------------------
/// <summary>The Galaxy provider expression used by the existing live smoke tests.</summary>
private static readonly string Provider =
string.Format(@"\\{0}\Galaxy", Environment.MachineName);
/// <summary>The Galaxy group (provider sub-path) containing the test alarm.</summary>
private const string Group = "DEV";
/// <summary>The test alarm's tag name as the dispatcher composes it (Group.TagName).</summary>
private const string AlarmTagName = "TestMachine_001.TestAlarm001";
// PLACEHOLDER — VERIFY against MXAccess-Public-API.md and live Galaxy.
// Typical InProcess Platform alarm active subtag: "<TagName>.InAlarm"
private const string PlaceholderActiveSubtag = "TestMachine_001.TestAlarm001.InAlarm";
// PLACEHOLDER — VERIFY. Typical: "<TagName>.Acked"
private const string PlaceholderAckedSubtag = "TestMachine_001.TestAlarm001.Acked";
// PLACEHOLDER — VERIFY. The writable ack-comment attribute varies by alarm
// template; common names are .AlarmComment, .AckComment, or .OperComment.
// Use a live Galaxy attribute browse (InTouch AlarmDB or MXTraceHarness
// GetItemList) to confirm before flipping.
private const string PlaceholderAckCommentSubtag = "TestMachine_001.TestAlarm001.AlarmComment";
// PLACEHOLDER — VERIFY. Typically: "<TagName>.Priority"
private const string PlaceholderPrioritySubtag = "TestMachine_001.TestAlarm001.Priority";
// -------------------------------------------------------------------------
private static readonly TimeSpan RaiseWaitTimeout = TimeSpan.FromSeconds(30);
private static readonly TimeSpan ClearWaitTimeout = TimeSpan.FromSeconds(30);
private static readonly string AlarmFullReference =
AlarmRecordTransitionMapper.ComposeFullReference(Provider, Group, AlarmTagName);
private readonly ITestOutputHelper output;
private readonly Stopwatch elapsed = Stopwatch.StartNew();
private readonly ConcurrentQueue<string> log =
new ConcurrentQueue<string>();
/// <summary>Initializes a new instance of the AlarmSubtagLiveSmokeTests class.</summary>
/// <param name="output">Test output helper for logging.</param>
public AlarmSubtagLiveSmokeTests(ITestOutputHelper output)
{
this.output = output;
}
/// <summary>
/// Verifies the subtag-fallback pipeline: advises alarm subtags through a
/// real <c>LMXProxyServerClass</c>, collects a Raise then a Clear
/// transition synthesized by <see cref="SubtagAlarmConsumer"/>, confirms
/// the Degraded flag and synthetic GUID are stamped, then
/// AcknowledgeByName and verifies the ack-comment write returns 0.
/// </summary>
[Fact(Skip = "Live dev-rig smoke test — flip Skip=null with AVEVA + an alarm flip script running. Subtag fallback path. VERIFY PLACEHOLDER_* subtag names before enabling.")]
public void SubtagFallback_FullPipelineRoundTrip_SynthesizesRaiseAndAcknowledges()
{
Exception? threadException = null;
ManualResetEventSlim done = new ManualResetEventSlim(false);
Thread thread = new Thread(() =>
{
try { RunSmoke(); }
catch (Exception ex) { threadException = ex; }
finally { done.Set(); }
});
thread.IsBackground = false;
thread.SetApartmentState(ApartmentState.STA);
thread.Start();
done.Wait();
thread.Join();
output.WriteLine(string.Format("Captured {0} log line(s):", log.Count));
string? line;
while (log.TryDequeue(out line))
{
output.WriteLine(line);
}
if (threadException != null)
{
throw threadException;
}
}
private void RunSmoke()
{
Log(string.Format("AlarmFullReference: {0}", AlarmFullReference));
Log("VERIFY: PlaceholderActiveSubtag = " + PlaceholderActiveSubtag);
Log("VERIFY: PlaceholderAckedSubtag = " + PlaceholderAckedSubtag);
Log("VERIFY: PlaceholderAckCommentSubtag = " + PlaceholderAckCommentSubtag);
Log("VERIFY: PlaceholderPrioritySubtag = " + PlaceholderPrioritySubtag);
Log(string.Format("RaiseWaitTimeout={0}s ClearWaitTimeout={1}s",
RaiseWaitTimeout.TotalSeconds, ClearWaitTimeout.TotalSeconds));
// Build target with PLACEHOLDER subtag names.
// VERIFY these against MXAccess-Public-API.md and a live Galaxy
// attribute browse before flipping Skip=null.
AlarmSubtagTarget target = new AlarmSubtagTarget
{
AlarmFullReference = AlarmFullReference,
SourceObjectReference = AlarmTagName,
ActiveSubtag = PlaceholderActiveSubtag,
AckedSubtag = PlaceholderAckedSubtag,
AckCommentSubtag = PlaceholderAckCommentSubtag,
PrioritySubtag = PlaceholderPrioritySubtag,
};
List<AlarmSubtagTarget> watchList = new List<AlarmSubtagTarget> { target };
// Construct the real COM-backed subtag source using the production
// factory (MxAccessComObjectFactory -> new LMXProxyServerClass()).
// This is the same factory the worker uses in production; no test
// double is involved on this path.
MxAccessComObjectFactory factory = new MxAccessComObjectFactory();
LmxSubtagAlarmSource source = new LmxSubtagAlarmSource(factory, clientName: null);
// SubtagAlarmConsumer wraps the source and drives the state machine.
using SubtagAlarmConsumer consumer = new SubtagAlarmConsumer(source, watchList);
// Collect emitted transitions on the STA (handler fires on the same
// STA that services the COM OnDataChange callback).
ConcurrentQueue<MxAlarmTransitionEvent> transitions =
new ConcurrentQueue<MxAlarmTransitionEvent>();
consumer.AlarmTransitionEmitted += (_, e) =>
{
Log(string.Format("Transition emitted: {0}", DescribeTransition(e)));
transitions.Enqueue(e);
};
// Subscribe binds Advise on all observable subtags (Active, Acked,
// Priority). The subscription expression is unused in subtag mode; pass
// something recognizable for diagnostics.
string subscriptionExpression = string.Format(@"\\{0}\Galaxy!{1}", Environment.MachineName, Group);
Log(string.Format("Calling Subscribe({0}) ...", subscriptionExpression));
consumer.Subscribe(subscriptionExpression);
Log("Subscribe returned OK.");
// 1. Wait for a Raise transition. The alarm flip script (same one
// used by AlarmsLiveSmokeTests) writes the active subtag on a
// ~10 s cadence. LmxSubtagAlarmSource delivers OnDataChange via
// the Windows message pump on the STA, so we must pump messages
// here while we wait — mirroring how AlarmsLiveSmokeTests drives
// its WnWrapAlarmConsumer.PollOnce() from the STA in a tight loop.
Log(string.Format("Waiting up to {0}s for a Raise transition ...", RaiseWaitTimeout.TotalSeconds));
MxAlarmTransitionEvent? raiseEvent = WaitForTransitionKind(
transitions, AlarmTransitionKind.Raise, RaiseWaitTimeout, "Raise");
Assert.NotNull(raiseEvent);
Assert.True(raiseEvent!.Record.Degraded,
"Subtag-synthesized records must have Degraded=true.");
// SubtagAlarmConsumer must stamp a synthetic GUID on the transition.
Assert.NotEqual(Guid.Empty, raiseEvent.Record.AlarmGuid);
// A Raise transition must leave the record in UnackAlm state.
Assert.Equal(MxAlarmStateKind.UnackAlm, raiseEvent.Record.State);
Log(string.Format("Raise confirmed: AlarmGuid={0} Degraded={1} State={2}",
raiseEvent.Record.AlarmGuid, raiseEvent.Record.Degraded, raiseEvent.Record.State));
// 2. Snapshot active alarms and confirm the raised alarm is present.
IReadOnlyList<MxAlarmSnapshotRecord> snapshot = consumer.SnapshotActiveAlarms();
Log(string.Format("SnapshotActiveAlarms count={0}", snapshot.Count));
foreach (MxAlarmSnapshotRecord s in snapshot)
{
Log(string.Format(" snapshot: TagName='{0}' Group='{1}' State={2} Degraded={3} Guid={4}",
s.TagName, s.Group, s.State, s.Degraded, s.AlarmGuid));
}
bool foundInSnapshot = false;
foreach (MxAlarmSnapshotRecord s in snapshot)
{
if (string.Equals(
AlarmRecordTransitionMapper.ComposeFullReference(s.ProviderName, s.Group, s.TagName),
AlarmFullReference,
StringComparison.OrdinalIgnoreCase))
{
foundInSnapshot = true;
break;
}
}
Assert.True(foundInSnapshot,
string.Format("Active alarm snapshot must contain '{0}' after a Raise.", AlarmFullReference));
// 3. AcknowledgeByName — writes the ack-comment subtag.
// The dispatcher derives (alarmName, provider, group) via the same
// TryParseReference logic as AlarmsLiveSmokeTests.
Log("Calling AcknowledgeByName ...");
int rc = consumer.AcknowledgeByName(
alarmName: AlarmTagName,
providerName: Provider,
groupName: Group,
ackComment: "subtag-fallback-smoke ack",
ackOperatorName: Environment.UserName,
ackOperatorNode: Environment.MachineName,
ackOperatorDomain: Environment.UserDomainName ?? string.Empty,
ackOperatorFullName: Environment.UserName);
Log(string.Format("AcknowledgeByName -> rc={0}", rc));
Assert.Equal(0, rc);
// 4. Wait for a Clear or Acknowledge transition to confirm the state
// machine continues tracking after the ack write.
Log(string.Format("Waiting up to {0}s for a Clear or Acknowledge transition ...",
ClearWaitTimeout.TotalSeconds));
MxAlarmTransitionEvent? postAckEvent = WaitForAnyTransition(transitions, ClearWaitTimeout, "post-ack");
// A null here is not a hard failure: the test alarm may not have
// cleared within the window, and the ack-comment write already
// confirmed the subtag path is wired. Log and assert non-null
// only when the post-ack transition is expected.
if (postAckEvent != null)
{
AlarmTransitionKind postAckKind = AlarmRecordTransitionMapper.MapTransition(
postAckEvent.PreviousState, postAckEvent.Record.State);
Log(string.Format("Post-ack transition: {0}", DescribeTransition(postAckEvent)));
Assert.NotEqual(AlarmTransitionKind.Unspecified, postAckKind);
}
else
{
Log("No post-ack transition within timeout — ack-comment write succeeded; state machine still live.");
}
Log("Smoke test complete.");
}
private MxAlarmTransitionEvent? WaitForTransitionKind(
ConcurrentQueue<MxAlarmTransitionEvent> queue,
AlarmTransitionKind kind,
TimeSpan timeout,
string label)
{
DateTime deadline = DateTime.UtcNow + timeout;
while (DateTime.UtcNow < deadline)
{
// Pump the STA Windows message queue so COM OnDataChange callbacks
// can be delivered. LMXProxyServerClass is apartment-threaded and
// requires the STA to be pumping; a bare Thread.Sleep would stall
// the pump.
PumpMessages();
MxAlarmTransitionEvent? evt;
if (queue.TryDequeue(out evt) && evt != null)
{
AlarmTransitionKind evtKind = AlarmRecordTransitionMapper.MapTransition(
evt.PreviousState, evt.Record.State);
if (evtKind == kind)
{
return evt;
}
Log(string.Format("Skipped transition kind={0} while waiting for {1}.", evtKind, label));
// Can't re-enqueue; log and continue.
}
Thread.Sleep(250);
}
Log(string.Format("Timed out waiting for {0} transition after {1}s.", label, timeout.TotalSeconds));
return null;
}
private MxAlarmTransitionEvent? WaitForAnyTransition(
ConcurrentQueue<MxAlarmTransitionEvent> queue,
TimeSpan timeout,
string label)
{
DateTime deadline = DateTime.UtcNow + timeout;
while (DateTime.UtcNow < deadline)
{
PumpMessages();
MxAlarmTransitionEvent? evt;
if (queue.TryDequeue(out evt) && evt != null)
{
return evt;
}
Thread.Sleep(250);
}
Log(string.Format("Timed out waiting for any transition ({0}) after {1}s.", label, timeout.TotalSeconds));
return null;
}
/// <summary>
/// Runs a single pass of the Windows STA message pump using a
/// non-blocking PeekMessage/DispatchMessage loop so COM
/// <c>OnDataChange</c> callbacks from <c>LMXProxyServerClass</c>
/// (ThreadingModel=Apartment) can be delivered on this thread.
/// Mirrors the pump pattern documented in
/// <c>docs/MxAccessWorkerInstanceDesign.md</c>.
/// </summary>
private static void PumpMessages()
{
NativeMethods.MSG msg;
// Drain all currently posted messages; return as soon as the queue
// is empty.
while (NativeMethods.PeekMessage(out msg, IntPtr.Zero, 0, 0, NativeMethods.PM_REMOVE))
{
NativeMethods.TranslateMessage(ref msg);
NativeMethods.DispatchMessage(ref msg);
}
}
private static string DescribeTransition(MxAlarmTransitionEvent e)
{
AlarmTransitionKind kind = AlarmRecordTransitionMapper.MapTransition(
e.PreviousState, e.Record.State);
return string.Format(
"Kind={0} PreviousState={1} State={2} TagName='{3}' Group='{4}' Provider='{5}' Degraded={6} Guid={7}",
kind,
e.PreviousState,
e.Record.State,
e.Record.TagName,
e.Record.Group,
e.Record.ProviderName,
e.Record.Degraded,
e.Record.AlarmGuid);
}
private void Log(string line)
{
log.Enqueue(string.Format("[t={0:F3}s] {1}", elapsed.Elapsed.TotalSeconds, line));
}
// -------------------------------------------------------------------------
// Minimal P/Invoke shim so the STA pump can be driven without pulling in
// the full StaRuntime machinery from the Worker project. The signatures
// mirror those in MxAccessStaRuntime and are well-known Win32.
// -------------------------------------------------------------------------
private static class NativeMethods
{
internal const uint PM_REMOVE = 0x0001;
[StructLayout(LayoutKind.Sequential)]
internal struct MSG
{
internal IntPtr hwnd;
internal uint message;
internal IntPtr wParam;
internal IntPtr lParam;
internal uint time;
internal int ptX;
internal int ptY;
}
[DllImport("user32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
internal static extern bool PeekMessage(
out MSG lpMsg,
IntPtr hWnd,
uint wMsgFilterMin,
uint wMsgFilterMax,
uint wRemoveMsg);
[DllImport("user32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
internal static extern bool TranslateMessage(ref MSG lpMsg);
[DllImport("user32.dll")]
internal static extern IntPtr DispatchMessage(ref MSG lpmsg);
}
}
@@ -51,7 +51,7 @@ public sealed class WorkerPipeSession
options,
() => Process.GetCurrentProcess().Id,
new WorkerPipeSessionOptions(),
() => new MxAccessStaSession((eq, affinity) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity)),
() => new MxAccessStaSession((eq, affinity, comFactory) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null)),
logger)
{
}
@@ -72,7 +72,7 @@ public sealed class WorkerPipeSession
options,
processIdProvider,
new WorkerPipeSessionOptions(),
() => new MxAccessStaSession((eq, affinity) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity)),
() => new MxAccessStaSession((eq, affinity, comFactory) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null)),
logger: null)
{
}
@@ -867,7 +867,7 @@ public sealed class WorkerPipeSession
// parameterless CompleteStartupHandshakeAsync is used without a
// prior factory call.
_runtimeSession ??= new MxAccessStaSession(
(eq, affinity) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity));
(eq, affinity, comFactory) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null));
IWorkerRuntimeSession session = _runtimeSession;
try
{
@@ -37,8 +37,14 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
private readonly MxAccessEventQueue eventQueue;
private readonly Func<IMxAccessAlarmConsumer> consumerFactory;
private readonly Action? threadAffinityCheck;
private readonly IMxAccessComObjectFactory? comFactory;
private readonly Func<IReadOnlyList<AlarmSubtagTarget>, IMxAccessAlarmConsumer>? standbyFactory;
private readonly MxAccessEventMapper mapper = new MxAccessEventMapper();
private readonly object syncRoot = new object();
private AlarmDispatcher? dispatcher;
private FailoverAlarmConsumer? failoverConsumer;
private EventHandler<AlarmProviderModeChange>? providerModeChangedHandler;
private string subscribeSessionId = string.Empty;
private bool disposed;
/// <summary>Initializes a new alarm command handler with the given event queue.</summary>
@@ -79,10 +85,49 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
MxAccessEventQueue eventQueue,
Func<IMxAccessAlarmConsumer> consumerFactory,
Action? threadAffinityCheck)
: this(eventQueue, consumerFactory, threadAffinityCheck, comFactory: null, standbyFactory: null)
{
}
/// <summary>
/// Full constructor that also threads the MXAccess COM-object factory and
/// an optional standby-consumer seam so the subscribe path can build the
/// subtag / failover consumers required by
/// <see cref="SubscribeAlarmsCommand.ForcedMode"/> and
/// <see cref="SubscribeAlarmsCommand.WatchList"/>.
/// </summary>
/// <param name="eventQueue">The event queue.</param>
/// <param name="consumerFactory">
/// Factory for the PRIMARY (alarmmgr) consumer — the existing
/// wnwrap-backed source. Used alone in alarmmgr mode and as the primary
/// of the failover composite in auto mode.
/// </param>
/// <param name="threadAffinityCheck">Optional STA thread-affinity guard.</param>
/// <param name="comFactory">
/// The MXAccess COM-object factory used to build the
/// <see cref="LmxSubtagAlarmSource"/> backing the subtag consumer. May be
/// <see langword="null"/> when a <paramref name="standbyFactory"/> is
/// supplied (tests) or when only the alarmmgr path is ever exercised.
/// </param>
/// <param name="standbyFactory">
/// Optional seam that builds the STANDBY (subtag) consumer from a watch
/// list. Defaults to a <see cref="SubtagAlarmConsumer"/> over an
/// <see cref="LmxSubtagAlarmSource"/> built from
/// <paramref name="comFactory"/>. Tests inject a fake so they need no
/// live COM factory.
/// </param>
public AlarmCommandHandler(
MxAccessEventQueue eventQueue,
Func<IMxAccessAlarmConsumer> consumerFactory,
Action? threadAffinityCheck,
IMxAccessComObjectFactory? comFactory,
Func<IReadOnlyList<AlarmSubtagTarget>, IMxAccessAlarmConsumer>? standbyFactory)
{
this.eventQueue = eventQueue ?? throw new ArgumentNullException(nameof(eventQueue));
this.consumerFactory = consumerFactory ?? throw new ArgumentNullException(nameof(consumerFactory));
this.threadAffinityCheck = threadAffinityCheck;
this.comFactory = comFactory;
this.standbyFactory = standbyFactory;
}
/// <summary>Gets a value indicating whether the handler is subscribed.</summary>
@@ -92,10 +137,10 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
}
/// <inheritdoc />
public void Subscribe(string subscription, string sessionId)
public void Subscribe(SubscribeAlarmsCommand command, string sessionId)
{
if (disposed) throw new ObjectDisposedException(nameof(AlarmCommandHandler));
if (subscription is null) throw new ArgumentNullException(nameof(subscription));
if (command is null) throw new ArgumentNullException(nameof(command));
threadAffinityCheck?.Invoke();
lock (syncRoot)
@@ -106,17 +151,31 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
"AlarmCommandHandler already has an active subscription; " +
"call Unsubscribe before issuing another SubscribeAlarms command.");
}
IMxAccessAlarmConsumer consumer = consumerFactory()
subscribeSessionId = sessionId ?? string.Empty;
IMxAccessAlarmConsumer consumer = BuildConsumer(command)
?? throw new InvalidOperationException("Alarm consumer factory returned null.");
MxAccessAlarmEventSink sink = new MxAccessAlarmEventSink(
eventQueue, new MxAccessEventMapper());
dispatcher = new AlarmDispatcher(consumer, sink, sessionId ?? string.Empty);
// When the selected consumer is a failover composite, surface its
// provider switches onto the worker's event queue so connected
// gateway clients can observe degraded/recovered state. The handler
// is unsubscribed/disposed on Unsubscribe/Dispose below.
if (consumer is FailoverAlarmConsumer failover)
{
failoverConsumer = failover;
providerModeChangedHandler = OnProviderModeChanged;
failover.ProviderModeChanged += providerModeChangedHandler;
}
MxAccessAlarmEventSink sink = new MxAccessAlarmEventSink(eventQueue, mapper);
dispatcher = new AlarmDispatcher(consumer, sink, subscribeSessionId);
try
{
dispatcher.Subscribe(subscription);
dispatcher.Subscribe(command.SubscriptionExpression ?? string.Empty);
}
catch
{
DetachProviderModeChanged();
try { dispatcher.Dispose(); } catch { /* swallow */ }
dispatcher = null;
throw;
@@ -124,6 +183,89 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
}
}
/// <summary>
/// Selects and builds the alarm consumer from the command's
/// <see cref="SubscribeAlarmsCommand.ForcedMode"/> and
/// <see cref="SubscribeAlarmsCommand.WatchList"/>:
/// <list type="bullet">
/// <item><description>
/// <c>Alarmmgr</c>, or <c>Unspecified</c> with an empty watch
/// list: the existing primary (alarmmgr) consumer only —
/// today's behavior.
/// </description></item>
/// <item><description>
/// <c>Subtag</c>: a <see cref="SubtagAlarmConsumer"/> only.
/// </description></item>
/// <item><description>
/// <c>Unspecified</c> with a non-empty watch list (auto): a
/// <see cref="FailoverAlarmConsumer"/> over the primary and a
/// subtag standby.
/// </description></item>
/// </list>
/// </summary>
private IMxAccessAlarmConsumer BuildConsumer(SubscribeAlarmsCommand command)
{
List<AlarmSubtagTarget> watchList = new List<AlarmSubtagTarget>(command.WatchList);
if (command.ForcedMode == AlarmProviderMode.Subtag)
{
return BuildStandby(watchList);
}
if (command.ForcedMode == AlarmProviderMode.Unspecified && watchList.Count > 0)
{
IMxAccessAlarmConsumer primary = consumerFactory()
?? throw new InvalidOperationException("Alarm consumer factory returned null.");
IMxAccessAlarmConsumer standby = BuildStandby(watchList);
AlarmFailoverConfig? failoverConfig = command.Failover;
FailoverSettings settings = new FailoverSettings(
failoverConfig?.ConsecutiveFailureThreshold ?? 3,
failoverConfig?.FailbackProbeIntervalSeconds ?? 30,
failoverConfig?.FailbackStableProbes ?? 3);
return new FailoverAlarmConsumer(primary, standby, settings);
}
// Alarmmgr, or Unspecified with an empty watch list — primary only.
return consumerFactory()
?? throw new InvalidOperationException("Alarm consumer factory returned null.");
}
private IMxAccessAlarmConsumer BuildStandby(IReadOnlyList<AlarmSubtagTarget> watchList)
{
if (standbyFactory is not null)
{
return standbyFactory(watchList)
?? throw new InvalidOperationException("Standby alarm consumer factory returned null.");
}
if (comFactory is null)
{
throw new InvalidOperationException(
"Subtag alarm consumer requires an IMxAccessComObjectFactory; the alarm command "
+ "handler was constructed without one and no standby factory was supplied.");
}
return new SubtagAlarmConsumer(new LmxSubtagAlarmSource(comFactory), watchList);
}
private void OnProviderModeChanged(object? sender, AlarmProviderModeChange change)
{
if (change is null) return;
eventQueue.Enqueue(mapper.CreateOnAlarmProviderModeChanged(
subscribeSessionId, change.Mode, change.Reason, change.HResult, change.AtUtc));
}
private void DetachProviderModeChanged()
{
if (failoverConsumer is not null && providerModeChangedHandler is not null)
{
try { failoverConsumer.ProviderModeChanged -= providerModeChangedHandler; }
catch { /* swallow */ }
}
failoverConsumer = null;
providerModeChangedHandler = null;
}
/// <inheritdoc />
public void Unsubscribe()
{
@@ -131,6 +273,7 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
AlarmDispatcher? toDispose;
lock (syncRoot)
{
DetachProviderModeChanged();
toDispose = dispatcher;
dispatcher = null;
}
@@ -191,7 +191,8 @@ public sealed class AlarmDispatcher : IDisposable
operatorUser: record.OperatorName,
operatorComment: record.AlarmComment,
category: record.Group,
description: string.Empty);
description: string.Empty,
degraded: record.Degraded);
}
private static ActiveAlarmSnapshot MapToSnapshot(MxAlarmSnapshotRecord record)
@@ -208,6 +209,8 @@ public sealed class AlarmDispatcher : IDisposable
OperatorComment = record.AlarmComment,
Category = record.Group,
Description = string.Empty,
Degraded = record.Degraded,
SourceProvider = record.Degraded ? AlarmProviderMode.Subtag : AlarmProviderMode.Alarmmgr,
};
if (record.TransitionTimestampUtc != DateTime.MinValue)
{
@@ -0,0 +1,61 @@
using System;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Raised by <see cref="FailoverAlarmConsumer"/> every time the active
/// alarm source switches between the primary (alarmmgr) consumer and the
/// standby (subtag) consumer. The worker translates this into the proto
/// family <c>OnAlarmProviderModeChanged</c> so connected gateway clients
/// can surface the degraded/recovered state.
/// </summary>
/// <remarks>
/// <para>
/// Plain class with constructor-assigned get-only properties — not a
/// <c>record</c> or <c>init</c>-only type — because the worker
/// multi-targets .NET Framework 4.8, which lacks
/// <c>System.Runtime.CompilerServices.IsExternalInit</c> (CS0518).
/// </para>
/// </remarks>
public sealed class AlarmProviderModeChange : EventArgs
{
/// <summary>
/// Initializes the change event payload.
/// </summary>
/// <param name="mode">The provider mode now active after the switch.</param>
/// <param name="reason">Human-readable reason for the switch.</param>
/// <param name="hResult">
/// The COM HRESULT that triggered a failover, or 0 for a clean
/// failback / no associated HRESULT.
/// </param>
/// <param name="atUtc">The UTC instant the switch occurred.</param>
public AlarmProviderModeChange(AlarmProviderMode mode, string reason, int hResult, DateTime atUtc)
{
Mode = mode;
Reason = reason ?? string.Empty;
HResult = hResult;
AtUtc = atUtc;
}
/// <summary>
/// The provider mode now active after the switch.
/// </summary>
public AlarmProviderMode Mode { get; }
/// <summary>
/// Human-readable reason for the switch (e.g. the failing COM call, or
/// <c>"recovered"</c> for a failback).
/// </summary>
public string Reason { get; }
/// <summary>
/// The COM HRESULT that triggered a failover, or 0 when none applies.
/// </summary>
public int HResult { get; }
/// <summary>
/// The UTC instant the switch occurred.
/// </summary>
public DateTime AtUtc { get; }
}
@@ -0,0 +1,368 @@
using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Composite <see cref="IMxAccessAlarmConsumer"/> that owns a PRIMARY
/// consumer (the wnwrap <see cref="WnWrapAlarmConsumer"/> alarmmgr source)
/// and a STANDBY consumer (the <c>SubtagAlarmConsumer</c> subtag fallback),
/// and switches between them automatically:
/// <list type="bullet">
/// <item><description>
/// Auto-fails-over to standby after
/// <see cref="FailoverSettings.Threshold"/> consecutive COM
/// failures on the primary.
/// </description></item>
/// <item><description>
/// Auto-fails-back to primary after
/// <see cref="FailoverSettings.StableProbes"/> consecutive clean
/// failback probes against the recovering primary.
/// </description></item>
/// </list>
/// It re-raises <see cref="AlarmTransitionEmitted"/> from whichever child
/// is active and raises <see cref="ProviderModeChanged"/> on every switch.
/// </summary>
/// <remarks>
/// <para>
/// <strong>Active-child event forwarding.</strong> This type subscribes
/// to <em>both</em> children's <see cref="AlarmTransitionEmitted"/>
/// events up front and gates re-raising on identity: a child transition
/// is forwarded only when its <c>sender</c> is the currently active
/// child. The standby is armed (subscribed) from the start so its
/// snapshot is warm at the moment of failover, but its transitions stay
/// suppressed until it becomes active. Gating-by-active is simpler and
/// less error-prone than subscribe/unsubscribe churn on every switch,
/// and it avoids a race where a transition fires during the switch.
/// </para>
/// <para>
/// <strong>Threading.</strong> Like its children, this type is driven
/// entirely on the worker's STA: <see cref="Subscribe"/>,
/// <see cref="PollOnce"/>, <see cref="ProbeOnce"/>, and the
/// <c>AcknowledgeBy*</c> calls are all invoked from the apartment that
/// owns the underlying COM objects. It owns no locks of its own and no
/// internal timer; the worker drives <see cref="PollOnce"/> on a timer.
/// </para>
/// </remarks>
public sealed class FailoverAlarmConsumer : IMxAccessAlarmConsumer
{
private enum Active
{
Primary,
Standby,
}
private readonly IMxAccessAlarmConsumer primary;
private readonly IMxAccessAlarmConsumer standby;
private readonly FailoverSettings settings;
private Active active = Active.Primary;
private AlarmProviderMode mode = AlarmProviderMode.Alarmmgr;
private int consecutiveFailures;
private int cleanProbes;
private bool disposed;
private DateTime lastProbeAtUtc = DateTime.MinValue;
/// <summary>
/// The subscription expression passed to <see cref="Subscribe"/>.
/// Stored for documentation and potential future full re-subscribe
/// scenarios; the primary is NOT re-subscribed during probing.
/// </summary>
private string subscriptionExpression = string.Empty;
/// <summary>
/// Composes the failover consumer over its two children.
/// </summary>
/// <param name="primary">The PRIMARY (alarmmgr) consumer.</param>
/// <param name="standby">The STANDBY (subtag) consumer.</param>
/// <param name="settings">The failover/failback tunables.</param>
public FailoverAlarmConsumer(
IMxAccessAlarmConsumer primary,
IMxAccessAlarmConsumer standby,
FailoverSettings settings)
{
this.primary = primary ?? throw new ArgumentNullException(nameof(primary));
this.standby = standby ?? throw new ArgumentNullException(nameof(standby));
this.settings = settings ?? throw new ArgumentNullException(nameof(settings));
this.primary.AlarmTransitionEmitted += OnChildTransition;
this.standby.AlarmTransitionEmitted += OnChildTransition;
}
/// <inheritdoc />
public event EventHandler<MxAlarmTransitionEvent>? AlarmTransitionEmitted;
/// <summary>
/// Fires on every switch between primary and standby. Carries the new
/// <see cref="AlarmProviderMode"/>, the reason, the triggering HRESULT
/// (0 for a clean failback), and the UTC instant.
/// </summary>
public event EventHandler<AlarmProviderModeChange>? ProviderModeChanged;
/// <summary>
/// The provider mode currently active.
/// </summary>
public AlarmProviderMode Mode => mode;
/// <inheritdoc />
/// <remarks>
/// Arms BOTH children up front so the standby snapshot is warm at the
/// moment of failover. The standby is always subscribed even if the
/// primary's <c>Subscribe</c> throws; a standby subscribe failure is
/// surfaced (rethrown) but does not count toward primary failover. The
/// primary subscribe runs through the failure-counting wrapper so a
/// COM failure on subscribe contributes to the failover threshold.
/// </remarks>
public void Subscribe(string subscription)
{
if (disposed) throw new ObjectDisposedException(nameof(FailoverAlarmConsumer));
// Store for documentation; the primary is not torn down on failover
// and is therefore not re-subscribed during ProbeOnce.
subscriptionExpression = subscription;
// Arm the standby first so it is warm regardless of primary outcome.
// A standby subscribe failure is a hard fault (the fallback itself is
// broken) and is surfaced to the caller; it does not feed the primary
// failover counter.
standby.Subscribe(subscription);
// Drive the primary subscribe through the failure-counting wrapper so
// a COM failure here counts toward the failover threshold instead of
// escaping. Swallowing the exception is deliberate: the standby is
// already armed, so a failed primary subscribe just nudges the state
// machine toward (or into) standby rather than aborting startup.
RunPrimary(() => primary.Subscribe(subscription), "Subscribe");
}
/// <inheritdoc />
/// <remarks>
/// While the primary is active, drives <c>primary.PollOnce</c> through
/// the failure-counting wrapper. While degraded (standby active),
/// drives <c>standby.PollOnce</c> and then runs one failback probe per
/// call via <see cref="ProbeOnce"/> — the worker drives this on a
/// timer, so one degraded poll equals one probe tick.
/// </remarks>
public void PollOnce()
{
if (disposed) throw new ObjectDisposedException(nameof(FailoverAlarmConsumer));
if (active == Active.Primary)
{
RunPrimary(() => primary.PollOnce(), "PollOnce");
return;
}
// Degraded: pump the standby for live transitions, then probe the
// primary for recovery. Standby PollOnce is a no-op for the subtag
// consumer but kept for symmetry / future standby sources.
standby.PollOnce();
ProbeOnce();
}
/// <summary>
/// Runs one failback probe against the (presumed recovering) primary.
/// Only meaningful while the standby is active; a no-op otherwise.
/// </summary>
/// <remarks>
/// <para>
/// A clean probe (primary <c>PollOnce</c> succeeds without
/// throwing) increments the clean-probe counter and, once it reaches
/// <see cref="FailoverSettings.StableProbes"/>, fails back to the
/// primary. Any probe failure resets the clean-probe counter to 0 so
/// the consumer requires a fresh unbroken run before failing back.
/// Exposed publicly so tests (and any external scheduler honoring
/// <see cref="FailoverSettings.ProbeIntervalSeconds"/> cadence) can
/// drive it directly.
/// </para>
/// <para>
/// <strong>Probe throttle.</strong> When
/// <see cref="FailoverSettings.ProbeIntervalSeconds"/> is greater than
/// zero, successive calls to this method are throttled: a probe is
/// skipped unless at least that many seconds have elapsed since the
/// last probe that was actually executed. When
/// <see cref="FailoverSettings.ProbeIntervalSeconds"/> is zero, the
/// throttle is disabled and every call probes immediately (the default
/// used by unit tests).
/// </para>
/// <para>
/// <strong>Why PollOnce only — no re-Subscribe.</strong>
/// Failover does NOT tear down the primary's subscription;
/// <see cref="WnWrapAlarmConsumer"/> is single-subscribe and would
/// throw <see cref="InvalidOperationException"/> on a second call.
/// The probe therefore re-polls the still-subscribed primary:
/// when the underlying COM provider recovers, <c>PollOnce</c> stops
/// throwing and clean probes accumulate toward failback. This covers
/// the dominant failure mode (transient COM/provider fault after a
/// successful initial subscribe).
/// </para>
/// <para>
/// <strong>Known v1 limitation.</strong> If the <em>original</em>
/// <c>Subscribe</c> itself failed (i.e., the primary never reached a
/// subscribed state — only reachable when
/// <see cref="FailoverSettings.Threshold"/> is 1), polling alone
/// cannot re-establish the subscription. That edge case is accepted
/// for v1: the operator must restart the session to force a fresh
/// subscribe attempt.
/// </para>
/// </remarks>
public void ProbeOnce()
{
if (disposed) throw new ObjectDisposedException(nameof(FailoverAlarmConsumer));
if (active != Active.Standby) return;
// Throttle probes to the configured cadence. When ProbeIntervalSeconds
// is 0 the throttle is disabled and every call probes immediately.
if (settings.ProbeIntervalSeconds > 0
&& (DateTime.UtcNow - lastProbeAtUtc).TotalSeconds < settings.ProbeIntervalSeconds)
{
return;
}
lastProbeAtUtc = DateTime.UtcNow;
try
{
// Re-poll the still-subscribed primary. Do NOT call Subscribe —
// WnWrapAlarmConsumer is single-subscribe and the primary remains
// subscribed across the failover; calling Subscribe again would
// always throw InvalidOperationException and prevent failback.
primary.PollOnce();
}
catch (Exception)
{
// Probe failed — the primary is still unhealthy. Demand a fresh
// unbroken run of StableProbes clean polls before failing back.
cleanProbes = 0;
return;
}
cleanProbes++;
if (cleanProbes >= settings.StableProbes)
{
SwitchToPrimary("recovered", 0);
}
}
/// <inheritdoc />
public int AcknowledgeByGuid(
Guid alarmGuid,
string ackComment,
string ackOperatorName,
string ackOperatorNode,
string ackOperatorDomain,
string ackOperatorFullName)
{
if (disposed) throw new ObjectDisposedException(nameof(FailoverAlarmConsumer));
return ActiveChild.AcknowledgeByGuid(
alarmGuid, ackComment, ackOperatorName, ackOperatorNode, ackOperatorDomain, ackOperatorFullName);
}
/// <inheritdoc />
public int AcknowledgeByName(
string alarmName,
string providerName,
string groupName,
string ackComment,
string ackOperatorName,
string ackOperatorNode,
string ackOperatorDomain,
string ackOperatorFullName)
{
if (disposed) throw new ObjectDisposedException(nameof(FailoverAlarmConsumer));
return ActiveChild.AcknowledgeByName(
alarmName, providerName, groupName, ackComment,
ackOperatorName, ackOperatorNode, ackOperatorDomain, ackOperatorFullName);
}
/// <inheritdoc />
public IReadOnlyList<MxAlarmSnapshotRecord> SnapshotActiveAlarms()
{
if (disposed) throw new ObjectDisposedException(nameof(FailoverAlarmConsumer));
return ActiveChild.SnapshotActiveAlarms();
}
private IMxAccessAlarmConsumer ActiveChild => active == Active.Primary ? primary : standby;
/// <summary>
/// Runs a primary COM action, counting consecutive failures. A
/// <see cref="COMException"/> (or any exception, treated as a COM
/// failure) increments the failure counter and, at
/// <see cref="FailoverSettings.Threshold"/> while the primary is still
/// active, switches to the standby. A success resets the counter.
/// </summary>
private void RunPrimary(Action action, string operation)
{
try
{
action();
}
catch (Exception ex) when (ex is not OutOfMemoryException)
{
consecutiveFailures++;
int hresult = ex is COMException ? ex.HResult : 0;
if (active == Active.Primary && consecutiveFailures >= settings.Threshold)
{
SwitchToStandby($"primary {operation} failed", hresult);
}
return;
}
consecutiveFailures = 0;
}
private void SwitchToStandby(string reason, int hresult)
{
active = Active.Standby;
mode = AlarmProviderMode.Subtag;
consecutiveFailures = 0;
cleanProbes = 0;
// Warm the standby snapshot for the gateway hand-off. The gateway
// reconciles state from this snapshot, so the return value is not
// consumed here — the call exists for its priming side effect.
_ = standby.SnapshotActiveAlarms();
RaiseModeChanged(AlarmProviderMode.Subtag, reason, hresult);
}
private void SwitchToPrimary(string reason, int hresult)
{
active = Active.Primary;
mode = AlarmProviderMode.Alarmmgr;
consecutiveFailures = 0;
cleanProbes = 0;
RaiseModeChanged(AlarmProviderMode.Alarmmgr, reason, hresult);
}
private void RaiseModeChanged(AlarmProviderMode newMode, string reason, int hresult)
{
ProviderModeChanged?.Invoke(
this,
new AlarmProviderModeChange(newMode, reason, hresult, DateTime.UtcNow));
}
private void OnChildTransition(object? sender, MxAlarmTransitionEvent e)
{
// Gate by active child: forward only the active source's transitions.
if (ReferenceEquals(sender, ActiveChild))
{
AlarmTransitionEmitted?.Invoke(this, e);
}
}
/// <inheritdoc />
public void Dispose()
{
if (disposed) return;
disposed = true;
primary.AlarmTransitionEmitted -= OnChildTransition;
standby.AlarmTransitionEmitted -= OnChildTransition;
primary.Dispose();
standby.Dispose();
}
}
@@ -0,0 +1,57 @@
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Tunables for <see cref="FailoverAlarmConsumer"/>'s auto-failover /
/// auto-failback state machine. Constructor-clamped to safe minimums so a
/// misconfigured options bind can never produce a zero/negative threshold
/// that would either never fail over or fail over on the first hiccup
/// unintentionally.
/// </summary>
/// <remarks>
/// <para>
/// Plain class with constructor-assigned get-only properties — not a
/// <c>record</c> or <c>init</c>-only type — because the worker
/// multi-targets .NET Framework 4.8, which lacks
/// <c>System.Runtime.CompilerServices.IsExternalInit</c> and so cannot
/// compile <c>init</c> accessors or positional records (CS0518).
/// </para>
/// </remarks>
public sealed class FailoverSettings
{
/// <summary>
/// Initializes the settings, clamping each value to its safe minimum.
/// </summary>
/// <param name="threshold">
/// Consecutive primary COM failures that trigger a switch to standby.
/// Clamped to a minimum of 1.
/// </param>
/// <param name="probeIntervalSeconds">
/// Minimum spacing (seconds) between failback probes against the
/// recovering primary. Clamped to a minimum of 0 (probe every tick).
/// </param>
/// <param name="stableProbes">
/// Consecutive clean failback probes required before switching back to
/// the primary. Clamped to a minimum of 1.
/// </param>
public FailoverSettings(int threshold, int probeIntervalSeconds, int stableProbes)
{
Threshold = threshold < 1 ? 1 : threshold;
ProbeIntervalSeconds = probeIntervalSeconds < 0 ? 0 : probeIntervalSeconds;
StableProbes = stableProbes < 1 ? 1 : stableProbes;
}
/// <summary>
/// Consecutive primary COM failures that trigger a switch to standby.
/// </summary>
public int Threshold { get; }
/// <summary>
/// Minimum spacing, in seconds, between failback probes.
/// </summary>
public int ProbeIntervalSeconds { get; }
/// <summary>
/// Consecutive clean failback probes required before failing back.
/// </summary>
public int StableProbes { get; }
}
@@ -13,10 +13,16 @@ namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// </summary>
public interface IAlarmCommandHandler : IDisposable
{
/// <summary>Begin a subscription against the supplied AVEVA alarm-provider expression.</summary>
/// <param name="subscription">The AVEVA alarm-provider subscription expression.</param>
/// <summary>
/// Begin an alarm subscription from the supplied command. The command's
/// <see cref="SubscribeAlarmsCommand.ForcedMode"/> and
/// <see cref="SubscribeAlarmsCommand.WatchList"/> select the consumer:
/// alarmmgr-only (the default), subtag-only, or an auto-failover
/// composite over both.
/// </summary>
/// <param name="command">The full SubscribeAlarms command.</param>
/// <param name="sessionId">The session identifier.</param>
void Subscribe(string subscription, string sessionId);
void Subscribe(SubscribeAlarmsCommand command, string sessionId);
/// <summary>Tear down the active subscription. No-op if not subscribed.</summary>
void Unsubscribe();
@@ -0,0 +1,24 @@
using System;
using System.Collections.Generic;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Advises a set of MXAccess subtag addresses and surfaces value changes.
/// The production implementation owns its own LMXProxyServerClass (later task);
/// tests substitute a fake that pushes SubtagValueChange values.
/// </summary>
public interface ISubtagAlarmSource : IDisposable
{
/// <summary>Raised when an advised subtag reports a new value.</summary>
event EventHandler<SubtagValueChange>? ValueChanged;
/// <summary>Begins advising the supplied MXAccess subtag item addresses.</summary>
/// <param name="itemAddresses">The subtag references to advise.</param>
void Advise(IReadOnlyCollection<string> itemAddresses);
/// <summary>Writes a value to the named MXAccess subtag (e.g. an ack request).</summary>
/// <param name="itemAddress">The subtag reference to write.</param>
/// <param name="value">The value to write.</param>
void Write(string itemAddress, object? value);
}
@@ -0,0 +1,334 @@
using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using ArchestrA.MxAccess;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Production <see cref="ISubtagAlarmSource"/> backed by its own MXAccess
/// <c>LMXProxyServerClass</c> COM object. It advises a set of alarm subtag
/// item addresses, normalizes each <c>OnDataChange</c> COM callback into a
/// <see cref="SubtagValueChange"/>, and supports writing a value (e.g. an
/// ack-comment) to a subtag.
/// </summary>
/// <remarks>
/// <para>
/// Like <see cref="WnWrapAlarmConsumer"/>, this type owns its
/// <em>own</em> MXAccess proxy-server COM object rather than sharing the
/// per-session item pipeline's <see cref="MxAccessSession"/>. That keeps
/// the subtag-fallback subscription isolated from the session's own
/// <c>Register</c>/<c>AddItem</c>/<c>Advise</c> bookkeeping: advising or
/// tearing down alarm subtags never perturbs the client's subscriptions.
/// </para>
/// <para>
/// <strong>Threading.</strong> The <c>LMXProxyServerClass</c> CLSID is
/// apartment-threaded and delivers <c>OnDataChange</c> via the STA
/// message pump. Every method on this type — the lazy <see cref="Advise"/>
/// creation, <see cref="Write"/>, and <see cref="Dispose"/> — must be
/// invoked on the worker's STA that created the source, exactly as
/// <see cref="MxAccessSession"/> and <see cref="WnWrapAlarmConsumer"/>
/// require. The <c>OnDataChange</c> COM event also arrives on that STA.
/// Because every call is confined to the single owning STA, no lock is
/// taken around the COM object or the handle maps: a lock here would risk
/// deadlocking the STA against a re-entrant pump callback, and the STA
/// affinity already serializes access.
/// </para>
/// </remarks>
public sealed class LmxSubtagAlarmSource : ISubtagAlarmSource
{
private const string DefaultClientName = "OtOpcUa.ZB.MOM.WW.MxGateway.Worker.SubtagAlarms";
private readonly IMxAccessComObjectFactory? factory;
private readonly string clientName;
private readonly Dictionary<string, int> itemHandlesByAddress =
new Dictionary<string, int>(StringComparer.Ordinal);
private readonly Dictionary<int, string> addressesByItemHandle =
new Dictionary<int, string>();
private object? mxAccessComObject;
private IMxAccessServer? server;
private LMXProxyServerClass? comEventSource;
private int serverHandle;
private bool registered;
private bool disposed;
/// <summary>
/// Production constructor. The MXAccess COM object is created lazily on
/// the first <see cref="Advise"/> call (which must run on the worker's
/// STA) via the supplied <paramref name="factory"/>.
/// </summary>
/// <param name="factory">Factory that creates the MXAccess proxy-server COM object.</param>
/// <param name="clientName">
/// Optional MXAccess client/registration name used for
/// <see cref="IMxAccessServer.Register"/>; a worker-specific default is
/// used when null or whitespace. Mirrors the registration name
/// <see cref="MxAccessSession.Register"/> accepts.
/// </param>
/// <exception cref="ArgumentNullException"><paramref name="factory"/> is null.</exception>
public LmxSubtagAlarmSource(
IMxAccessComObjectFactory factory,
string? clientName = null)
{
this.factory = factory ?? throw new ArgumentNullException(nameof(factory));
this.clientName = string.IsNullOrWhiteSpace(clientName) ? DefaultClientName : clientName!;
}
/// <summary>
/// Test-only seam: constructs a source over a caller-supplied
/// <see cref="IMxAccessServer"/> with a pre-registered server handle,
/// bypassing the live COM factory. Exposed via
/// <c>InternalsVisibleTo("ZB.MOM.WW.MxGateway.Worker.Tests")</c> so tests can
/// exercise <see cref="Advise"/>/<see cref="Write"/> and
/// <see cref="HandleDataChange"/> without a live <c>LMXProxyServerClass</c>.
/// No COM event source is wired in this mode — tests drive
/// <see cref="HandleDataChange"/> directly to simulate an
/// <c>OnDataChange</c> callback.
/// </summary>
/// <param name="server">The server abstraction to drive.</param>
/// <param name="serverHandle">The already-registered server handle to use.</param>
internal LmxSubtagAlarmSource(
IMxAccessServer server,
int serverHandle)
{
this.factory = null;
this.clientName = DefaultClientName;
this.server = server ?? throw new ArgumentNullException(nameof(server));
this.serverHandle = serverHandle;
this.registered = true;
}
/// <inheritdoc />
public event EventHandler<SubtagValueChange>? ValueChanged;
/// <inheritdoc />
/// <remarks>
/// Idempotent per address: an address already advised is skipped. The
/// MXAccess COM object is created and the client registered on the first
/// call. Must be invoked on the worker's STA.
/// </remarks>
public void Advise(IReadOnlyCollection<string> itemAddresses)
{
if (itemAddresses is null)
{
throw new ArgumentNullException(nameof(itemAddresses));
}
ThrowIfDisposed();
EnsureRegistered();
IMxAccessServer mxServer = server!;
foreach (string? itemAddress in itemAddresses)
{
if (string.IsNullOrWhiteSpace(itemAddress))
{
continue;
}
if (itemHandlesByAddress.ContainsKey(itemAddress!))
{
continue;
}
int itemHandle = mxServer.AddItem(serverHandle, itemAddress!);
mxServer.Advise(serverHandle, itemHandle);
itemHandlesByAddress[itemAddress!] = itemHandle;
addressesByItemHandle[itemHandle] = itemAddress!;
}
}
/// <inheritdoc />
/// <remarks>
/// Adds the item if it has not been added yet — the write may target a
/// subtag (e.g. the ack-comment) that was never advised — then writes
/// with MXAccess user id 0 (unsecured). Must be invoked on the worker's
/// STA.
/// </remarks>
public void Write(string itemAddress, object? value)
{
if (itemAddress is null)
{
throw new ArgumentNullException(nameof(itemAddress));
}
ThrowIfDisposed();
EnsureRegistered();
IMxAccessServer mxServer = server!;
if (!itemHandlesByAddress.TryGetValue(itemAddress, out int itemHandle))
{
itemHandle = mxServer.AddItem(serverHandle, itemAddress);
itemHandlesByAddress[itemAddress] = itemHandle;
addressesByItemHandle[itemHandle] = itemAddress;
}
mxServer.Write(serverHandle, itemHandle, value, userId: 0);
}
/// <summary>
/// Normalizes an MXAccess <c>OnDataChange</c> callback into a
/// <see cref="SubtagValueChange"/> and raises <see cref="ValueChanged"/>
/// when the item handle maps to an advised address. Unknown handles are
/// ignored. Exposed <c>internal</c> as a unit-test seam so the
/// normalization can be exercised without a live <c>LMXProxyServerClass</c>,
/// mirroring <c>MxAccessBaseEventSink.OnDataChange</c>.
/// </summary>
/// <param name="phItemHandle">The MXAccess item handle that changed.</param>
/// <param name="pvItemValue">The new item value (left as a boxed COM variant; the state machine coerces it).</param>
/// <param name="pftItemTimeStamp">The item timestamp as delivered by MXAccess (a culture-formatted string).</param>
internal void HandleDataChange(
int phItemHandle,
object? pvItemValue,
object? pftItemTimeStamp)
{
if (!addressesByItemHandle.TryGetValue(phItemHandle, out string? itemAddress))
{
return;
}
// MXAccess delivers OnDataChange timestamps as a culture-formatted
// string (VT_BSTR), not a FILETIME or VT_DATE — see
// MxAccessEventMapper.TryParseSourceTimestamp, which is the single
// conversion helper the per-session pipeline already uses. Reuse it so
// the subtag path interprets the host-local string the same way; fall
// back to capture time when the string is absent or unparsable.
DateTime timestampUtc = ConvertTimestampUtc(pftItemTimeStamp);
EventHandler<SubtagValueChange>? handler = ValueChanged;
handler?.Invoke(this, new SubtagValueChange
{
ItemAddress = itemAddress,
Value = pvItemValue,
TimestampUtc = timestampUtc,
});
}
/// <inheritdoc />
public void Dispose()
{
if (disposed)
{
return;
}
disposed = true;
// Unsubscribe the COM event before tearing handles down so a late pump
// callback cannot re-enter HandleDataChange mid-teardown.
if (comEventSource is not null)
{
try { comEventSource.OnDataChange -= OnDataChange; } catch { /* swallow — best effort */ }
}
IMxAccessServer? mxServer = server;
if (mxServer is not null && registered)
{
foreach (KeyValuePair<int, string> entry in addressesByItemHandle)
{
try { mxServer.UnAdvise(serverHandle, entry.Key); } catch { /* swallow — best effort */ }
try { mxServer.RemoveItem(serverHandle, entry.Key); } catch { /* swallow — best effort */ }
}
try { mxServer.Unregister(serverHandle); } catch { /* swallow — best effort */ }
}
itemHandlesByAddress.Clear();
addressesByItemHandle.Clear();
object? comToRelease = mxAccessComObject;
mxAccessComObject = null;
comEventSource = null;
server = null;
registered = false;
if (comToRelease is not null && Marshal.IsComObject(comToRelease))
{
try { Marshal.FinalReleaseComObject(comToRelease); } catch { /* swallow — best effort */ }
}
}
private void EnsureRegistered()
{
if (registered)
{
return;
}
// factory is only null in the test-only constructor, which sets
// registered = true, so this path is never hit there.
object created = factory!.Create()
?? throw new InvalidOperationException("MXAccess COM factory returned null.");
try
{
mxAccessComObject = created;
// Wire OnDataChange the same way MxAccessBaseEventSink.Attach does:
// cast to the concrete LMXProxyServerClass RCW and subscribe. Test
// doubles cannot be cast to LMXProxyServerClass, so the COM event
// subscription is exercised only on the live path; unit tests drive
// HandleDataChange directly via the internal constructor seam.
if (created is LMXProxyServerClass comObject)
{
comEventSource = comObject;
comObject.OnDataChange += OnDataChange;
}
server = new MxAccessComServer(created);
serverHandle = server.Register(clientName);
registered = true;
}
catch
{
if (comEventSource is not null)
{
try { comEventSource.OnDataChange -= OnDataChange; } catch { /* swallow */ }
comEventSource = null;
}
if (Marshal.IsComObject(created))
{
try { Marshal.FinalReleaseComObject(created); } catch { /* swallow */ }
}
mxAccessComObject = null;
server = null;
throw;
}
}
/// <summary>
/// COM <c>OnDataChange</c> delegate target. Forwards to
/// <see cref="HandleDataChange"/>; the quality, status array, and server
/// handle carry no information the subtag fallback needs.
/// </summary>
private void OnDataChange(
int hLMXServerHandle,
int phItemHandle,
object pvItemValue,
int pwItemQuality,
object pftItemTimeStamp,
ref MXSTATUS_PROXY[] pVars)
{
HandleDataChange(phItemHandle, pvItemValue, pftItemTimeStamp);
}
private static DateTime ConvertTimestampUtc(object? pftItemTimeStamp)
{
if (pftItemTimeStamp is not null
&& MxAccessEventMapper.TryParseSourceTimestamp(pftItemTimeStamp.ToString(), out DateTime parsedUtc))
{
return parsedUtc;
}
return DateTime.UtcNow;
}
private void ThrowIfDisposed()
{
if (disposed)
{
throw new ObjectDisposedException(nameof(LmxSubtagAlarmSource));
}
}
}
@@ -86,6 +86,12 @@ public sealed class MxAccessAlarmEventSink : IMxAccessEventSink
/// <param name="operatorComment">The operator's comment, if any.</param>
/// <param name="category">The alarm category.</param>
/// <param name="description">The alarm description.</param>
/// <param name="degraded">
/// <see langword="true"/> when the transition was synthesized by the
/// subtag-provider fallback rather than the native alarmmgr path.
/// Defaults to <see langword="false"/> so existing alarmmgr callers
/// compile unchanged and stay on the parity (alarmmgr) path.
/// </param>
internal void EnqueueTransition(
string alarmFullReference,
string sourceObjectReference,
@@ -97,10 +103,16 @@ public sealed class MxAccessAlarmEventSink : IMxAccessEventSink
string operatorUser,
string operatorComment,
string category,
string description)
string description,
bool degraded = false)
{
try
{
// Degraded transitions come from the subtag fallback; the native
// alarmmgr (wnwrap) path stays degraded=false / ALARMMGR for parity.
AlarmProviderMode sourceProvider = degraded
? AlarmProviderMode.Subtag
: AlarmProviderMode.Alarmmgr;
MxEvent mxEvent = eventMapper.CreateOnAlarmTransition(
sessionId,
alarmFullReference,
@@ -114,7 +126,9 @@ public sealed class MxAccessAlarmEventSink : IMxAccessEventSink
operatorComment,
category,
description,
statuses: null);
statuses: null,
degraded: degraded,
sourceProvider: sourceProvider);
eventQueue.Enqueue(mxEvent);
}
catch (Exception exception)
@@ -598,7 +598,8 @@ public sealed class MxAccessCommandExecutor : IStaCommandExecutor
"SubscribeAlarms requires an alarm command handler; the worker was constructed without one.");
}
string subscription = command.Command.SubscribeAlarms.SubscriptionExpression ?? string.Empty;
SubscribeAlarmsCommand subscribeCommand = command.Command.SubscribeAlarms;
string subscription = subscribeCommand.SubscriptionExpression ?? string.Empty;
if (string.IsNullOrWhiteSpace(subscription))
{
return CreateInvalidRequestReply(command, "SubscribeAlarms.subscription_expression is required.");
@@ -606,7 +607,7 @@ public sealed class MxAccessCommandExecutor : IStaCommandExecutor
try
{
alarmCommandHandler.Subscribe(subscription, command.SessionId);
alarmCommandHandler.Subscribe(subscribeCommand, command.SessionId);
return CreateOkReply(command);
}
catch (Exception ex)
@@ -124,6 +124,15 @@ public sealed class MxAccessEventMapper
/// <param name="category">Alarm taxonomy bucket from the Galaxy template.</param>
/// <param name="description">Human-readable alarm description.</param>
/// <param name="statuses">Array of MxStatusProxy values from MXAccess.</param>
/// <param name="degraded">
/// <see langword="true"/> when this transition was synthesized by the
/// subtag-provider fallback rather than the native alarmmgr path.
/// Defaults to <see langword="false"/> to preserve alarmmgr parity.
/// </param>
/// <param name="sourceProvider">
/// The alarm provider that sourced this transition. Defaults to
/// <see cref="AlarmProviderMode.Alarmmgr"/> for the native path.
/// </param>
public MxEvent CreateOnAlarmTransition(
string sessionId,
string alarmFullReference,
@@ -137,7 +146,9 @@ public sealed class MxAccessEventMapper
string operatorComment,
string category,
string description,
Array? statuses)
Array? statuses,
bool degraded = false,
AlarmProviderMode sourceProvider = AlarmProviderMode.Alarmmgr)
{
MxEvent mxEvent = CreateBaseEvent(
MxEventFamily.OnAlarmTransition,
@@ -159,6 +170,8 @@ public sealed class MxAccessEventMapper
OperatorComment = operatorComment ?? string.Empty,
Category = category ?? string.Empty,
Description = description ?? string.Empty,
Degraded = degraded,
SourceProvider = sourceProvider,
};
if (originalRaiseTimestampUtc is { } orts)
{
@@ -169,6 +182,43 @@ public sealed class MxAccessEventMapper
return mxEvent;
}
/// <summary>
/// Creates an OnAlarmProviderModeChanged event from a
/// <see cref="FailoverAlarmConsumer"/> provider switch. The worker's
/// alarm path drives this when the failover composite switches between
/// the alarmmgr primary and the subtag standby, so connected gateway
/// clients can observe the degraded/recovered provider state.
/// </summary>
/// <param name="sessionId">Identifier of the session.</param>
/// <param name="mode">The provider mode now active after the switch.</param>
/// <param name="reason">Human-readable reason for the switch.</param>
/// <param name="hresult">The COM HRESULT that triggered a failover, or 0 for a clean failback.</param>
/// <param name="atUtc">The UTC instant the switch occurred.</param>
public MxEvent CreateOnAlarmProviderModeChanged(
string sessionId,
AlarmProviderMode mode,
string reason,
int hresult,
DateTime atUtc)
{
MxEvent mxEvent = CreateBaseEvent(
MxEventFamily.OnAlarmProviderModeChanged,
sessionId,
serverHandle: 0,
itemHandle: 0,
statuses: null);
mxEvent.OnAlarmProviderModeChanged = new OnAlarmProviderModeChangedEvent
{
Mode = mode,
Reason = reason ?? string.Empty,
Hresult = hresult,
At = Google.Protobuf.WellKnownTypes.Timestamp.FromDateTime(
DateTime.SpecifyKind(atUtc, DateTimeKind.Utc)),
};
return mxEvent;
}
/// <summary>Creates an OnBufferedDataChange event from MXAccess COM event arguments.</summary>
/// <param name="sessionId">Identifier of the session.</param>
/// <param name="serverHandle">Handle returned by the worker.</param>
@@ -23,7 +23,10 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
// then invokes the guard at the entry of every method that touches the
// wnwrap consumer, matching the STA-affinity invariant already enforced
// for the poll path via EnsureOnAlarmConsumerThread.
private readonly Func<MxAccessEventQueue, Action, IAlarmCommandHandler>? alarmCommandHandlerFactory;
// Worker-9: the third arg is the session's IMxAccessComObjectFactory, so
// the handler can build the subtag-fallback source's own proxy-server COM
// object on this STA when a subscribe selects the subtag / failover path.
private readonly Func<MxAccessEventQueue, Action, IMxAccessComObjectFactory, IAlarmCommandHandler>? alarmCommandHandlerFactory;
private StaCommandDispatcher? commandDispatcher;
private MxAccessSession? session;
private IAlarmCommandHandler? alarmCommandHandler;
@@ -51,7 +54,7 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
/// of alarm-side commands.
/// </summary>
/// <param name="alarmCommandHandlerFactory">Factory that constructs the alarm-command handler.</param>
internal MxAccessStaSession(Func<MxAccessEventQueue, Action, IAlarmCommandHandler>? alarmCommandHandlerFactory)
internal MxAccessStaSession(Func<MxAccessEventQueue, Action, IMxAccessComObjectFactory, IAlarmCommandHandler>? alarmCommandHandlerFactory)
: this(
new StaRuntime(),
new MxAccessComObjectFactory(),
@@ -103,7 +106,7 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
StaRuntime staRuntime,
IMxAccessComObjectFactory factory,
MxAccessEventQueue eventQueue,
Func<MxAccessEventQueue, Action, IAlarmCommandHandler>? alarmCommandHandlerFactory)
Func<MxAccessEventQueue, Action, IMxAccessComObjectFactory, IAlarmCommandHandler>? alarmCommandHandlerFactory)
: this(staRuntime, factory, new MxAccessBaseEventSink(eventQueue), eventQueue, alarmCommandHandlerFactory)
{
}
@@ -141,7 +144,7 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
IMxAccessComObjectFactory factory,
IMxAccessEventSink eventSink,
MxAccessEventQueue eventQueue,
Func<MxAccessEventQueue, Action, IAlarmCommandHandler>? alarmCommandHandlerFactory)
Func<MxAccessEventQueue, Action, IMxAccessComObjectFactory, IAlarmCommandHandler>? alarmCommandHandlerFactory)
{
this.staRuntime = staRuntime ?? throw new ArgumentNullException(nameof(staRuntime));
this.factory = factory ?? throw new ArgumentNullException(nameof(factory));
@@ -209,9 +212,16 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
// on convention alone; a future refactor that let a
// command run off-STA would silently deadlock on
// cross-apartment marshaling against the wnwrap consumer.
// Worker-9: the factory also receives the session's
// IMxAccessComObjectFactory so the subtag-fallback source
// (LmxSubtagAlarmSource) can create its OWN proxy-server COM
// object on this STA, isolated from the item pipeline's
// MxAccessSession. The factory call runs on the STA, so the
// resulting source is bound to the correct apartment.
alarmCommandHandler = alarmCommandHandlerFactory(
eventQueue,
EnsureOnAlarmConsumerThread);
EnsureOnAlarmConsumerThread,
factory);
}
commandDispatcher = new StaCommandDispatcher(
staRuntime,
@@ -6,6 +6,8 @@ namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// Single alarm record as emitted by the wnwrapConsumer XML stream.
/// Field names match the captured XML schema (see
/// <c>docs/AlarmClientDiscovery.md</c> "Option A — captured" section).
/// Records are also produced by the subtag synthesis path when
/// <see cref="Degraded"/> is set.
/// </summary>
public sealed class MxAlarmSnapshotRecord
{
@@ -37,4 +39,12 @@ public sealed class MxAlarmSnapshotRecord
public string OperatorName { get; set; } = string.Empty;
/// <summary>Gets or sets the alarm comment.</summary>
public string AlarmComment { get; set; } = string.Empty;
/// <summary>
/// Gets or sets a value indicating whether this record was synthesized
/// by the subtag-provider fallback rather than emitted by the native
/// alarmmgr (wnwrap) path. Default <see langword="false"/> preserves
/// parity for the alarmmgr path; the subtag fallback sets it to
/// <see langword="true"/>.
/// </summary>
public bool Degraded { get; set; }
}
@@ -0,0 +1,317 @@
using System;
using System.Collections.Generic;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Subtag-fallback implementation of <see cref="IMxAccessAlarmConsumer"/>.
/// Where <see cref="WnWrapAlarmConsumer"/> polls the native alarmmgr
/// (wnwrap) COM stream, this consumer advises a configured set of MXAccess
/// alarm subtags via an <see cref="ISubtagAlarmSource"/> and synthesizes
/// alarm transitions through a <see cref="SubtagAlarmStateMachine"/>. Every
/// emitted record is flagged <see cref="MxAlarmSnapshotRecord.Degraded"/>
/// and assigned a deterministic <see cref="SyntheticAlarmGuid"/> derived
/// from the alarm's full reference, since the subtag path has no
/// alarmmgr-supplied GUID.
/// </summary>
/// <remarks>
/// <para>
/// Threading: like <see cref="WnWrapAlarmConsumer"/>, this consumer is
/// driven on the worker's STA thread. <see cref="Subscribe"/> binds the
/// source's <c>ValueChanged</c> event; the source raises that event on
/// the same STA in production, so <see cref="AlarmTransitionEmitted"/>
/// fires on the STA and subscribers must marshal off it themselves if
/// they need another thread.
/// </para>
/// <para>
/// The subtag path is event-driven, so <see cref="PollOnce"/> is a
/// no-op — value changes arrive through the source's advise stream
/// rather than an explicit poll. Acknowledgment in subtag mode writes
/// the comment to the target's writable ack-comment subtag rather than
/// calling a native <c>AlarmAckByGUID</c> / <c>AlarmAckByName</c>.
/// </para>
/// </remarks>
public sealed class SubtagAlarmConsumer : IMxAccessAlarmConsumer
{
private readonly ISubtagAlarmSource source;
private readonly SubtagAlarmStateMachine stateMachine;
private readonly Dictionary<string, AlarmSubtagTarget> targetsByReference;
private readonly Dictionary<Guid, string> referencesBySyntheticGuid;
private readonly EventHandler<SubtagValueChange> valueChangedHandler;
private bool subscribed;
private bool disposed;
/// <summary>Fires once per synthesized alarm-state transition.</summary>
public event EventHandler<MxAlarmTransitionEvent>? AlarmTransitionEmitted;
/// <summary>
/// Initializes the consumer over a subtag source and a watch list of
/// alarm targets.
/// </summary>
/// <param name="source">The subtag value source to advise and write through.</param>
/// <param name="watchList">The alarm subtag targets to observe.</param>
/// <exception cref="ArgumentNullException">
/// Thrown when <paramref name="source"/> or <paramref name="watchList"/>
/// is <see langword="null"/>.
/// </exception>
public SubtagAlarmConsumer(ISubtagAlarmSource source, IReadOnlyList<AlarmSubtagTarget> watchList)
{
this.source = source ?? throw new ArgumentNullException(nameof(source));
if (watchList is null)
{
throw new ArgumentNullException(nameof(watchList));
}
this.stateMachine = new SubtagAlarmStateMachine(watchList);
this.targetsByReference = new Dictionary<string, AlarmSubtagTarget>(StringComparer.OrdinalIgnoreCase);
this.referencesBySyntheticGuid = new Dictionary<Guid, string>();
foreach (AlarmSubtagTarget target in watchList)
{
string reference = target.AlarmFullReference ?? string.Empty;
this.targetsByReference[reference] = target;
this.referencesBySyntheticGuid[SyntheticAlarmGuid.ForReference(reference)] = reference;
}
this.valueChangedHandler = OnValueChanged;
}
/// <summary>
/// Advises every observable alarm subtag (active / acked / priority —
/// the ack-comment subtag is a write-only target and is not advised)
/// and begins listening for value changes. The <paramref name="subscription"/>
/// expression is ignored: the subtag set is fixed by the watch list.
/// </summary>
/// <param name="subscription">The subscription expression (unused in subtag mode).</param>
public void Subscribe(string subscription)
{
if (disposed)
{
throw new ObjectDisposedException(nameof(SubtagAlarmConsumer));
}
List<string> addresses = new List<string>();
foreach (AlarmSubtagTarget target in targetsByReference.Values)
{
AddIfNotEmpty(addresses, target.ActiveSubtag);
AddIfNotEmpty(addresses, target.AckedSubtag);
AddIfNotEmpty(addresses, target.PrioritySubtag);
}
source.Advise(addresses);
if (!subscribed)
{
source.ValueChanged += valueChangedHandler;
subscribed = true;
}
}
/// <summary>
/// Acknowledges an alarm by its synthetic GUID. Resolves the GUID back
/// to its alarm full reference and delegates to the by-name write path.
/// </summary>
/// <param name="alarmGuid">The synthetic alarm GUID.</param>
/// <param name="ackComment">The acknowledgment comment.</param>
/// <param name="ackOperatorName">The operator name (unused in subtag mode).</param>
/// <param name="ackOperatorNode">The operator node (unused in subtag mode).</param>
/// <param name="ackOperatorDomain">The operator domain (unused in subtag mode).</param>
/// <param name="ackOperatorFullName">The operator full name (unused in subtag mode).</param>
/// <returns>0 on success; non-zero when the GUID or ack-comment subtag is unknown.</returns>
public int AcknowledgeByGuid(
Guid alarmGuid,
string ackComment,
string ackOperatorName,
string ackOperatorNode,
string ackOperatorDomain,
string ackOperatorFullName)
{
if (disposed)
{
throw new ObjectDisposedException(nameof(SubtagAlarmConsumer));
}
if (!referencesBySyntheticGuid.TryGetValue(alarmGuid, out string reference) ||
!targetsByReference.TryGetValue(reference, out AlarmSubtagTarget target))
{
return -1;
}
return WriteAckComment(target, ackComment);
}
/// <summary>
/// Acknowledges an alarm by its (name, provider, group) tuple. In subtag
/// mode the comment is written to the target's writable ack-comment
/// subtag; the operator-identity arguments are not surfaced through the
/// subtag write.
/// </summary>
/// <param name="alarmName">The alarm name (object-rooted tag name as the dispatcher derives it).</param>
/// <param name="providerName">The provider name used to recompose the full reference for lookup.</param>
/// <param name="groupName">The group name used to recompose the full reference for lookup.</param>
/// <param name="ackComment">The acknowledgment comment.</param>
/// <param name="ackOperatorName">The operator name (unused in subtag mode).</param>
/// <param name="ackOperatorNode">The operator node (unused in subtag mode).</param>
/// <param name="ackOperatorDomain">The operator domain (unused in subtag mode).</param>
/// <param name="ackOperatorFullName">The operator full name (unused in subtag mode).</param>
/// <returns>0 on success; non-zero when no target matches or it lacks an ack-comment subtag.</returns>
public int AcknowledgeByName(
string alarmName,
string providerName,
string groupName,
string ackComment,
string ackOperatorName,
string ackOperatorNode,
string ackOperatorDomain,
string ackOperatorFullName)
{
if (disposed)
{
throw new ObjectDisposedException(nameof(SubtagAlarmConsumer));
}
AlarmSubtagTarget? target = ResolveTargetByName(alarmName, providerName, groupName);
if (target is null)
{
return -1;
}
return WriteAckComment(target, ackComment);
}
/// <summary>
/// Returns the state machine's currently-active alarm snapshot, with
/// each record stamped <see cref="MxAlarmSnapshotRecord.Degraded"/> and
/// assigned its synthetic GUID.
/// </summary>
/// <returns>The active alarm snapshot records.</returns>
public IReadOnlyList<MxAlarmSnapshotRecord> SnapshotActiveAlarms()
{
IReadOnlyList<MxAlarmSnapshotRecord> records = stateMachine.SnapshotActive();
foreach (MxAlarmSnapshotRecord record in records)
{
StampSynthetic(record);
}
return records;
}
/// <summary>
/// No-op: the subtag path is event-driven and owns no poll cadence.
/// </summary>
public void PollOnce()
{
// Subtag mode is event-driven; value changes arrive via the source's
// advise stream, so there is nothing to poll.
}
/// <inheritdoc />
public void Dispose()
{
if (disposed)
{
return;
}
disposed = true;
if (subscribed)
{
try
{
source.ValueChanged -= valueChangedHandler;
}
catch
{
// swallow — best-effort detach during dispose
}
subscribed = false;
}
try
{
source.Dispose();
}
catch
{
// swallow — best-effort source dispose
}
}
private void OnValueChanged(object? sender, SubtagValueChange change)
{
if (disposed || change is null)
{
return;
}
IReadOnlyList<MxAlarmTransitionEvent> transitions =
stateMachine.Apply(change.ItemAddress, change.Value, change.TimestampUtc);
if (transitions.Count == 0)
{
return;
}
EventHandler<MxAlarmTransitionEvent>? handler = AlarmTransitionEmitted;
foreach (MxAlarmTransitionEvent transition in transitions)
{
StampSynthetic(transition.Record);
handler?.Invoke(this, transition);
}
}
/// <summary>
/// Stamps the degraded flag and synthetic GUID onto a synthesized
/// record. The state machine sets only the provider / group / tag-name
/// parts, so the alarm reference is recomposed exactly as
/// <see cref="AlarmDispatcher"/> composes it — keying the synthetic GUID
/// off that recomposed reference keeps it stable across transitions and
/// snapshots for the same alarm.
/// </summary>
private static void StampSynthetic(MxAlarmSnapshotRecord record)
{
record.Degraded = true;
string reference = AlarmRecordTransitionMapper.ComposeFullReference(
record.ProviderName, record.Group, record.TagName);
record.AlarmGuid = SyntheticAlarmGuid.ForReference(reference);
}
private AlarmSubtagTarget? ResolveTargetByName(string? alarmName, string? providerName, string? groupName)
{
if (string.IsNullOrEmpty(alarmName))
{
return null;
}
// Recompose the full reference from the same (provider, group, name)
// tuple the dispatcher derived, then do an exact OrdinalIgnoreCase
// lookup in targetsByReference. This avoids the false-positive that a
// substring scan would produce when one alarm name is a prefix of
// another (e.g. "Level.Hi" matching "Level.HiHi").
string composed = AlarmRecordTransitionMapper.ComposeFullReference(providerName, groupName, alarmName);
targetsByReference.TryGetValue(composed, out AlarmSubtagTarget? target);
return target;
}
private int WriteAckComment(AlarmSubtagTarget target, string ackComment)
{
string ackCommentSubtag = target.AckCommentSubtag ?? string.Empty;
if (string.IsNullOrEmpty(ackCommentSubtag))
{
return -1;
}
source.Write(ackCommentSubtag, ackComment);
return 0;
}
private static void AddIfNotEmpty(List<string> addresses, string? address)
{
if (!string.IsNullOrEmpty(address))
{
addresses.Add(address!);
}
}
}
@@ -0,0 +1,381 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Synthesizes alarm transitions from raw subtag value changes when the
/// native alarmmgr (wnwrap) consumer is unavailable. Each configured
/// <see cref="AlarmSubtagTarget"/> contributes an <c>.active</c>,
/// <c>.acked</c>, optional <c>.priority</c>, and optional ack-comment
/// subtag; the machine tracks per-alarm state and emits
/// <see cref="MxAlarmTransitionEvent"/> records mirroring the wnwrap
/// consumer's UNACK_ALM / ACK_ALM / UNACK_RTN / ACK_RTN transitions.
/// This type performs no COM work and is fully unit-testable.
/// </summary>
/// <remarks>
/// This type has STA affinity: it is mutated and read on the worker STA
/// thread only and is not thread-safe. The per-target
/// <see cref="AlarmSubtagTarget.AckCommentSubtag"/> is a write-only ack
/// target and is deliberately not bound for observation. The synthesized
/// <see cref="MxAlarmSnapshotRecord.AlarmGuid"/> is left at its default
/// value: in subtag mode acks are correlated by name/comment rather than
/// by GUID, and a synthetic GUID is assigned by a later component.
/// </remarks>
public sealed class SubtagAlarmStateMachine
{
private static readonly IReadOnlyList<MxAlarmTransitionEvent> NoEvents = Array.Empty<MxAlarmTransitionEvent>();
private readonly Dictionary<string, AlarmState> _statesByReference = new(StringComparer.OrdinalIgnoreCase);
private readonly Dictionary<string, SubtagBinding> _bindingsByAddress = new(StringComparer.OrdinalIgnoreCase);
/// <summary>Initializes the machine from the configured alarm subtag targets.</summary>
/// <param name="targets">The alarm targets whose subtags will be observed.</param>
public SubtagAlarmStateMachine(IEnumerable<AlarmSubtagTarget> targets)
{
if (targets is null)
{
throw new ArgumentNullException(nameof(targets));
}
foreach (AlarmSubtagTarget target in targets)
{
var state = new AlarmState(target);
_statesByReference[target.AlarmFullReference] = state;
Bind(target.ActiveSubtag, state, SubtagRole.Active);
Bind(target.AckedSubtag, state, SubtagRole.Acked);
Bind(target.PrioritySubtag, state, SubtagRole.Priority);
}
}
/// <summary>
/// Applies a single subtag value change and returns any synthesized
/// transitions. Unknown addresses and no-op changes return an empty list.
/// </summary>
/// <param name="itemAddress">The subtag address whose value changed.</param>
/// <param name="value">The new value (bool, numeric, or string).</param>
/// <param name="timestampUtc">The UTC timestamp of the change.</param>
/// <returns>The synthesized alarm transitions, if any.</returns>
public IReadOnlyList<MxAlarmTransitionEvent> Apply(string itemAddress, object? value, DateTime timestampUtc)
{
if (itemAddress is null || !_bindingsByAddress.TryGetValue(itemAddress, out SubtagBinding binding))
{
return NoEvents;
}
AlarmState state = binding.State;
switch (binding.Role)
{
case SubtagRole.Active:
return ApplyActive(state, CoerceBool(value), timestampUtc);
case SubtagRole.Acked:
return ApplyAcked(state, CoerceBool(value), timestampUtc);
case SubtagRole.Priority:
state.Priority = CoerceInt(value, state.Priority);
return NoEvents;
default:
return NoEvents;
}
}
/// <summary>
/// Returns one snapshot record per currently-active alarm, with state
/// reflecting whether the alarm has been acknowledged.
/// </summary>
/// <returns>The active alarm snapshot records.</returns>
public IReadOnlyList<MxAlarmSnapshotRecord> SnapshotActive()
{
var records = new List<MxAlarmSnapshotRecord>();
foreach (AlarmState state in _statesByReference.Values)
{
if (!state.Active)
{
continue;
}
MxAlarmStateKind kind = state.Acked ? MxAlarmStateKind.AckAlm : MxAlarmStateKind.UnackAlm;
records.Add(state.BuildRecord(kind, state.FirstRaiseUtc));
}
return records;
}
private IReadOnlyList<MxAlarmTransitionEvent> ApplyActive(AlarmState state, bool newActive, DateTime timestampUtc)
{
if (newActive == state.Active)
{
return NoEvents;
}
if (newActive)
{
// false -> true: alarm raised, initially unacknowledged.
MxAlarmStateKind previous = state.DerivedState();
state.Active = true;
state.Acked = false;
state.AckedDuringEpisode = false;
state.FirstRaiseUtc = timestampUtc;
return Single(state, MxAlarmStateKind.UnackAlm, previous, timestampUtc);
}
else
{
// true -> false: alarm returned to normal. The return kind reflects
// whether the alarm was acknowledged at any point during the active
// episode, not just its instantaneous Acked flag (see remarks).
MxAlarmStateKind previous = state.DerivedState();
MxAlarmStateKind kind = state.AckedDuringEpisode ? MxAlarmStateKind.AckRtn : MxAlarmStateKind.UnackRtn;
state.Active = false;
state.Acked = false;
state.AckedDuringEpisode = false;
return Single(state, kind, previous, timestampUtc);
}
}
/// <remarks>
/// <see cref="AlarmState.AckedDuringEpisode"/> latches that the alarm was
/// acknowledged at some point while active. MXAccess does not guarantee
/// that the acked subtag update arrives before the active=false update on
/// clear, so an out-of-order un-ack (acked true->false) must not lose the
/// fact that the episode was acknowledged; the latch ensures the clear
/// still emits ACK_RTN rather than UNACK_RTN.
/// </remarks>
private IReadOnlyList<MxAlarmTransitionEvent> ApplyAcked(AlarmState state, bool newAcked, DateTime timestampUtc)
{
if (newAcked == state.Acked)
{
return NoEvents;
}
state.Acked = newAcked;
// Only a true ack of a currently-active alarm produces an ACK_ALM
// transition; the latch records that the episode was acked at some point.
if (newAcked && state.Active)
{
state.AckedDuringEpisode = true;
return Single(state, MxAlarmStateKind.AckAlm, MxAlarmStateKind.UnackAlm, timestampUtc);
}
// acked true->false while active: do NOT clear the latch (the episode was
// already acked); emit nothing.
return NoEvents;
}
private static IReadOnlyList<MxAlarmTransitionEvent> Single(
AlarmState state,
MxAlarmStateKind kind,
MxAlarmStateKind previous,
DateTime timestampUtc)
{
return new[]
{
new MxAlarmTransitionEvent
{
Record = state.BuildRecord(kind, timestampUtc),
PreviousState = previous,
},
};
}
private void Bind(string? address, AlarmState state, SubtagRole role)
{
if (!string.IsNullOrEmpty(address))
{
if (_bindingsByAddress.ContainsKey(address!))
{
throw new ArgumentException(
$"Duplicate subtag item address '{address}' is bound to more than one alarm target.",
nameof(address));
}
_bindingsByAddress[address!] = new SubtagBinding(state, role);
}
}
private static bool CoerceBool(object? value)
{
switch (value)
{
case null:
return false;
case bool b:
return b;
case string s:
if (bool.TryParse(s, out bool parsedBool))
{
return parsedBool;
}
if (double.TryParse(s, NumberStyles.Any, CultureInfo.InvariantCulture, out double parsedNum))
{
return parsedNum != 0d;
}
return false;
default:
try
{
return Convert.ToDouble(value, CultureInfo.InvariantCulture) != 0d;
}
catch (Exception ex) when (ex is FormatException or InvalidCastException or OverflowException)
{
return false;
}
}
}
private static int CoerceInt(object? value, int fallback)
{
switch (value)
{
case null:
return fallback;
case int i:
return i;
case string s when int.TryParse(s, NumberStyles.Any, CultureInfo.InvariantCulture, out int parsed):
return parsed;
case string:
return fallback;
default:
try
{
return Convert.ToInt32(value, CultureInfo.InvariantCulture);
}
catch (Exception ex) when (ex is FormatException or InvalidCastException or OverflowException)
{
return fallback;
}
}
}
private enum SubtagRole
{
Active,
Acked,
Priority,
}
private readonly struct SubtagBinding
{
public SubtagBinding(AlarmState state, SubtagRole role)
{
State = state;
Role = role;
}
public AlarmState State { get; }
public SubtagRole Role { get; }
}
private sealed class AlarmState
{
private readonly string _providerName;
private readonly string _group;
private readonly string _tagName;
public AlarmState(AlarmSubtagTarget target)
{
(_providerName, _group, _tagName) = DeriveNameParts(target);
}
public bool Active { get; set; }
public bool Acked { get; set; }
/// <summary>
/// Latches that the alarm was acknowledged at some point during the
/// current active episode, so a clear emits ACK_RTN even when the
/// un-ack subtag arrives before the active=false update.
/// </summary>
public bool AckedDuringEpisode { get; set; }
public int Priority { get; set; }
public DateTime FirstRaiseUtc { get; set; }
/// <summary>Derives the current alarm state from the tracked flags, before a transition is applied.</summary>
public MxAlarmStateKind DerivedState()
{
if (!Active)
{
return MxAlarmStateKind.Unspecified;
}
return Acked ? MxAlarmStateKind.AckAlm : MxAlarmStateKind.UnackAlm;
}
public MxAlarmSnapshotRecord BuildRecord(MxAlarmStateKind kind, DateTime timestampUtc)
{
return new MxAlarmSnapshotRecord
{
ProviderName = _providerName,
Group = _group,
TagName = _tagName,
Priority = Priority,
State = kind,
TransitionTimestampUtc = timestampUtc,
};
}
/// <summary>
/// Splits an alarm full reference such as <c>Galaxy!Area.Tank01.Level.HiHi</c>
/// into provider (<c>Galaxy</c>), group/area (<c>Area</c>), and the object-rooted
/// tag name (<c>Tank01.Level.HiHi</c>). The tag name is anchored on the target's
/// <see cref="AlarmSubtagTarget.SourceObjectReference"/> when present, so any
/// leading provider and area segments are stripped.
/// </summary>
private static (string ProviderName, string Group, string TagName) DeriveNameParts(AlarmSubtagTarget target)
{
string reference = target.AlarmFullReference ?? string.Empty;
string providerName = string.Empty;
string afterBang = reference;
int bang = reference.IndexOf('!');
if (bang >= 0)
{
providerName = reference.Substring(0, bang);
afterBang = reference.Substring(bang + 1);
}
string tagName = afterBang;
string source = target.SourceObjectReference ?? string.Empty;
string group = string.Empty;
if (source.Length > 0)
{
// Match the source on dot-delimited segment boundaries so a source
// such as "Tank" does not match mid-token inside "Tank01" and a
// source "Area" does not match inside "TankArea".
string[] segments = afterBang.Split('.');
int segmentIndex = -1;
for (int i = 0; i < segments.Length; i++)
{
if (string.Equals(segments[i], source, StringComparison.OrdinalIgnoreCase))
{
segmentIndex = i;
break;
}
}
if (segmentIndex > 0)
{
// Everything before the object is the area/group; the object
// and everything after it is the object-rooted tag name.
group = string.Join(".", segments, 0, segmentIndex);
tagName = string.Join(".", segments, segmentIndex, segments.Length - segmentIndex);
}
else
{
// segmentIndex == 0 (source is the first segment) or not found:
// no leading area to strip, so the tag name is the whole afterBang.
tagName = afterBang;
}
}
return (providerName, group, tagName);
}
}
}
@@ -0,0 +1,16 @@
using System;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>A change in one advised subtag value, normalized off the COM boundary.</summary>
public sealed class SubtagValueChange
{
/// <summary>Gets or sets the MXAccess item address (subtag reference) whose value changed.</summary>
public string ItemAddress { get; set; } = string.Empty;
/// <summary>Gets or sets the new value, as delivered by MXAccess (boxed COM variant or test value).</summary>
public object? Value { get; set; }
/// <summary>Gets or sets the UTC timestamp the change was observed.</summary>
public DateTime TimestampUtc { get; set; }
}
@@ -0,0 +1,42 @@
using System;
using System.Security.Cryptography;
using System.Text;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Derives a deterministic synthetic <see cref="Guid"/> from an alarm
/// reference for the subtag-provider fallback path, which has no native
/// MxAccess alarm GUID. Hashing the reference yields a stable identity so
/// repeated transitions for the same alarm reference correlate downstream
/// (acknowledge, snapshot, OPC UA mapping) without an alarmmgr-supplied GUID.
/// </summary>
internal static class SyntheticAlarmGuid
{
/// <summary>
/// Produces a stable <see cref="Guid"/> for the given alarm reference.
/// The same reference always maps to the same GUID; distinct references
/// map to distinct GUIDs with overwhelming probability.
/// </summary>
/// <param name="reference">
/// The fully-qualified alarm reference (for example
/// <c>"Galaxy!Area.Tag.HiHi"</c>). Treated as UTF-8 bytes.
/// </param>
/// <returns>A deterministic, non-empty GUID derived from the reference.</returns>
/// <exception cref="ArgumentNullException">
/// Thrown when <paramref name="reference"/> is <see langword="null"/>.
/// </exception>
public static Guid ForReference(string reference)
{
if (reference is null) throw new ArgumentNullException(nameof(reference));
byte[] bytes = Encoding.UTF8.GetBytes(reference);
// MD5 is used purely for a stable, non-cryptographic identity mapping
// (reference -> 16-byte GUID), never for security. Its 128-bit output
// fits a GUID exactly, which is why it is preferred here.
using MD5 md5 = MD5.Create();
byte[] hash = md5.ComputeHash(bytes);
return new Guid(hash);
}
}