Compare commits

...

38 Commits

Author SHA1 Message Date
Joseph Doherty 3c9becc8d6 docs(plan): mark all alarm-subtag-fallback tasks completed 2026-06-13 10:55:18 -04:00
Joseph Doherty ec88532fe4 alarms: propagate degraded/source_provider through snapshot + gateway cache paths (integration fix I1/I2) 2026-06-13 10:53:55 -04:00
Joseph Doherty 2f30f0c7c0 docs(alarms): document alarmmgr->subtag fallback (providers, failover, config, contract, parity) 2026-06-13 10:43:37 -04:00
Joseph Doherty 27f6c9e6b7 dashboard(alarms): provider-status badge (alarmmgr vs degraded subtag) 2026-06-13 10:37:37 -04:00
Joseph Doherty 29bd504a99 test(alarms): end-to-end provider failover/failback lifecycle through GatewayAlarmMonitor 2026-06-13 10:34:24 -04:00
Joseph Doherty e10b252e3a test(alarms): drop unsupported Assert.Equal message args in live subtag smoke test (xUnit) 2026-06-13 10:30:39 -04:00
Joseph Doherty bcc54ca56b server(alarms): provider-mode gauge startup baseline; reconcile-lock comment; de-flake monitor test 2026-06-13 10:29:13 -04:00
Joseph Doherty ee459f43e1 test(alarms): opt-in live subtag-fallback smoke test (Skip by default)
Adds AlarmSubtagLiveSmokeTests to validate the open design item from Task 17:
confirms that LmxSubtagAlarmSource (real MxAccessComObjectFactory) wired to
SubtagAlarmConsumer synthesizes degraded Raise transitions with stable synthetic
GUIDs from Galaxy alarm subtags, and that AcknowledgeByName writes the
ack-comment subtag (rc=0). PLACEHOLDER_* subtag addresses are best-guess and
must be verified against MXAccess-Public-API.md + live Galaxy before flipping Skip.
2026-06-13 10:26:28 -04:00
Joseph Doherty ebf1d95f72 server(alarms): monitor resolves watch-list, sends ForcedMode/failover, reflects provider mode into feed + metrics 2026-06-13 10:20:03 -04:00
Joseph Doherty 3ccf0b5f9e server(alarms): honor ExcludeAttributes GR-only contract; warn on empty config-only watch-list 2026-06-13 10:12:58 -04:00
Joseph Doherty f7ccfd678e server(alarms): watch-list resolver merging GR discovery + config override 2026-06-13 10:09:10 -04:00
Joseph Doherty 3f5e5fc0b3 worker(alarms): route ForcedMode/watch-list/failover via AlarmCommandHandler; emit provider-mode-changed event 2026-06-13 10:04:33 -04:00
Joseph Doherty 7241a4fb9c worker(alarms): net48 index fix; enforce ProbeIntervalSeconds; OOM-safe catch; reset-on-failure test 2026-06-13 09:55:07 -04:00
Joseph Doherty d6c0bb41ca worker(alarms): failback probe re-polls the still-subscribed primary (no re-Subscribe) 2026-06-13 09:49:38 -04:00
Joseph Doherty 0a54c0bc4b worker(alarms): FailoverAlarmConsumer auto-failover/failback state machine 2026-06-13 09:46:47 -04:00
Joseph Doherty fd64b9260c worker(alarms): exact-match ack resolution (no substring false-match) + ack-by-guid tests 2026-06-13 09:42:00 -04:00
Joseph Doherty 4bd757a136 worker(alarms): SubtagAlarmConsumer synthesizing degraded transitions; dispatcher propagates Degraded 2026-06-13 09:35:49 -04:00
Joseph Doherty 1e2ed6d1ea worker(alarms): WriteRecord as class not positional record (net48 has no IsExternalInit) 2026-06-13 09:30:52 -04:00
Joseph Doherty 5f6655de27 server(alarms): drop redundant null-coalesce; tidy validator tests (review fixes) 2026-06-13 09:27:37 -04:00
Joseph Doherty fbc9cf56df worker(alarms): SyntheticAlarmGuid internal + alarmmgr-parity assertion (review fixes) 2026-06-13 09:26:52 -04:00
Joseph Doherty 4c0e14fc5d worker(alarms): COM-backed LmxSubtagAlarmSource advising alarm subtags 2026-06-13 09:24:09 -04:00
Joseph Doherty c75920c620 docs(plan): correct alarm proto location to mxaccess_gateway.proto (Tasks 1-2) 2026-06-13 09:18:11 -04:00
Joseph Doherty a46ce90e6f server(metrics): alarm provider mode gauge + provider switch counter (Task 13) 2026-06-13 09:18:11 -04:00
Joseph Doherty f113ca53a1 server(galaxy): GetAlarmAttributesAsync discovery query + alarm-attribute row mapping (Task 11) 2026-06-13 09:18:11 -04:00
Joseph Doherty f3616cc7fa server(alarms): AlarmFallbackOptions + ForceSubtag/threshold validation (Task 10) 2026-06-13 09:18:11 -04:00
Joseph Doherty 57d5a8725f worker(alarms): synthetic GUID + degraded/source_provider on emitted transitions 2026-06-13 09:14:23 -04:00
Joseph Doherty 60d35a914f contracts: regenerate Generated/ for alarm provider mode + subtag types
Keeps committed generated C# in sync with the .proto change in 1d85db7
(AlarmProviderMode, AlarmSubtagTarget, AlarmFailoverConfig, AlarmProviderStatus,
OnAlarmProviderModeChangedEvent, degraded/source_provider fields).
2026-06-13 09:10:08 -04:00
Joseph Doherty b10e103bcf worker(alarms): fix net48 build (init->set, usings), token-boundary name parse, acked latch, dup-address guard, tests 2026-06-13 09:05:58 -04:00
Joseph Doherty 348ab16456 worker(alarms): subtag value-source seam + synthesis state machine 2026-06-13 08:57:28 -04:00
Joseph Doherty c16f016f0a test(contracts): round-trip provider status + degraded provenance 2026-06-13 08:56:13 -04:00
Joseph Doherty 1d85db7b4e contracts(gateway): AlarmProviderMode, subtag watch-list, provider status, degraded provenance, mode-changed event 2026-06-13 08:53:02 -04:00
Joseph Doherty 5ea5618315 docs: implementation plan for alarm subtag-monitoring fallback
18 TDD tasks across contracts, worker (SubtagAlarmConsumer + FailoverAlarmConsumer),
gateway (GR-SQL watch-list discovery, monitor mode reflection, metrics, dashboard),
and docs. Grounded in current signatures; parity-preserving (worker-side synthesis).
2026-06-13 08:44:42 -04:00
Joseph Doherty 38a0ad8ab4 docs: design for alarmmgr→subtag alarm-provider fallback
Auto-failover/failback between the wnwrap alarmmgr consumer and a new
worker-side SubtagAlarmConsumer that advises alarm subtags and synthesizes
transitions. GR-SQL+config watch-list discovery, ack via ack-comment write,
degraded state surfaced in the gRPC contract and dashboard/metrics.
2026-06-13 08:35:18 -04:00
Joseph Doherty 5df2ef0d1e chore(theme): bump ZB.MOM.WW.Theme 0.3.0 -> 0.3.1 (interactive-render nav fix) 2026-06-05 07:19:11 -04:00
Joseph Doherty e5785fd769 chore(theme): consume ZB.MOM.WW.Theme 0.3.0 (nav/login kit fixes) 2026-06-05 05:13:06 -04:00
Joseph Doherty 22370ca4da docs(glauth): repoint glauth.md at the shared GLAuth on 10.100.0.35
No more per-box C:\publish\glauth NSSM service — dev/test LDAP is the shared
zb-shared-glauth on 10.100.0.35:3893 (dc=zb,dc=local). Provisioning now via
scadaproj/infra/glauth/config.toml. Old localhost/NSSM procedures kept as
retired reference; test users multi-role/gw-viewer.
2026-06-04 16:38:24 -04:00
Joseph Doherty e0a3fbf35b fix(dashboard)!: move login POST to /auth/login to resolve AmbiguousMatchException
The themed Blazor <LoginCard> page (Components/Pages/Login.razor, @page "/login")
registers a Razor Components endpoint that matches ALL HTTP methods. The credential
form POSTed to /login, where MapPost("/login") also matched — so every POST /login
threw Microsoft.AspNetCore.Routing.Matching.AmbiguousMatchException (HTTP 500),
breaking dashboard login for every user. It was latent because the dashboard was only
ever reached via the AllowAnonymousLocalhost bypass on the host box.

Move the credential POST to a distinct /auth/login route (mirroring ScadaBridge, which
never collided because it posts to /auth/login). GET /login stays the Blazor page; the
cookie LoginPath stays /login. Adds a registration assertion pinning DashboardLoginPost
to /auth/login as the regression guard.

Files: Login.razor (LoginCard Action), DashboardEndpointRouteBuilderExtensions (MapPost
route), GatewayApplicationTests (route assertion).
2026-06-04 14:01:05 -04:00
Joseph Doherty 161ed6f80d chore(theme): bump ZB.MOM.WW.Theme 0.2.0 -> 0.2.1 (desktop app-shell render fix) 2026-06-04 10:23:44 -04:00
66 changed files with 10096 additions and 500 deletions
+1 -1
View File
@@ -100,7 +100,7 @@ When source code changes, build and test the affected component before reporting
## Design Sources To Consult Before Non-Trivial Changes
- `gateway.md` — top-level architecture, command/event surface, IPC envelope, STA thread model, fault handling.
- `glauth.md` — local LDAP server (GLAuth on `localhost:3893`, base DN `dc=zb,dc=local`) used for dev authn. Pre-provisioned users (`admin/admin123`, `readonly/readonly123`, etc.) and the role→capability mapping live there.
- `glauth.md` — shared GLAuth LDAP server (`10.100.0.35:3893`, base DN `dc=zb,dc=local`, source of truth `scadaproj/infra/glauth/`) used for dev authn. Dashboard test users (`multi-role`/`password` = Administrator, `gw-viewer`/`password` = Viewer) and the role→capability mapping live there.
- `docs/DesignDecisions.md` — v1 choices (MXAccess COM target `LMXProxyServerClass` from `C:\Program Files (x86)\ArchestrA\Framework\Bin\ArchestrA.MXAccess.dll`, API-key-in-SQLite auth, fail-fast event backpressure, etc.).
- `docs/GatewayProcessDesign.md`, `docs/MxAccessWorkerInstanceDesign.md`, `docs/WorkerFrameProtocol.md`, `docs/WorkerProcessLauncher.md` — detailed component designs.
- `docs/GatewayConfiguration.md` — full `MxGateway:*` options bound by `GatewayOptions` and validated at startup by `GatewayOptionsValidator`.
+124
View File
@@ -790,3 +790,127 @@ Post-ack transition: kind=Clear …
10s cadence held throughout; full proto fields populated correctly;
ack registered server-side without errors.
## Subtag-monitoring fallback provider
When the wnwrap alarm-manager source fails, the gateway worker switches to
`SubtagAlarmConsumer` — a synthetic alarm source that advises each alarm
attribute's subtags via the existing MXAccess `AddItem`/`Advise` pipeline and
derives alarm transitions from the resulting value-change stream. This is a
non-parity, degraded-mode source; every transition and snapshot it produces
carries `degraded = true`.
### Watch-list discovery
`GatewayAlarmMonitor` resolves the subtag watch-list at subscribe time by
calling `IAlarmWatchListResolver.GetAlarmAttributesAsync`. The resolver merges:
1. Galaxy Repository SQL (`GetAlarmAttributesAsync`) — objects that have alarm
extensions in the configured area.
2. Config overrides — `IncludeAttributes` adds explicit entries;
`ExcludeAttributes` removes Repository-derived ones. The config list takes
effect even when `UseGalaxyRepository` is `false`.
The resolved list is a set of `AlarmSubtagTarget` messages sent to the worker
inside `SubscribeAlarmsCommand.watch_list`. Each target carries the composed
MXAccess item addresses for the `.active`, `.acked`, ack-comment, and priority
subtags. The gateway re-runs discovery on its reconcile cadence and pushes an
updated watch-list when the model changes.
### Subtag advise and `LmxSubtagAlarmSource`
`LmxSubtagAlarmSource` (implements `ISubtagAlarmSource`) owns a separate
`LMXProxyServerClass` instance on the worker STA — it does not share the
session's main MXAccess object. For each watch-list target it calls
`AddItem`/`Advise` on the configured subtag addresses. When a subtag value
changes, it raises `ValueChanged` on the STA and `SubtagAlarmConsumer`
forwards it to `SubtagAlarmStateMachine`.
`PollOnce()` on the subtag consumer is a no-op — the path is event-driven
through `Advise`, not poll-driven.
### Synthesis rules
`SubtagAlarmStateMachine` tracks `(active, acked)` per watch-list entry and
emits `MxAlarmTransitionEvent` records on change:
| Subtag change | Emitted transition | Notes |
|---|---|---|
| `active` false → true | Raise (`UNACK_ALM`) | `original_raise_timestamp` = first observed active time for this episode |
| `acked` false → true, while `active` | Acknowledge (`ACK_ALM`) | `AckedDuringEpisode` latch set |
| `active` true → false | Clear | `AckRtn` if `AckedDuringEpisode` is set, else `UnackRtn` |
| `acked` true → false, while `active` | (none) | Latch is NOT cleared; the episode retains its acknowledged status at clear |
The `AckedDuringEpisode` latch addresses out-of-order subtag delivery:
MXAccess does not guarantee the `acked = false` update arrives before the
`active = false` update. The latch ensures a clear always emits `ACK_RTN`
when the alarm was acknowledged at any point during the active episode.
`SnapshotActive()` returns one `MxAlarmSnapshotRecord` per currently-active
alarm. State mapping:
- `active && !acked` → `UNACK_ALM`
- `active && acked` → `ACK_ALM`
- `!active` → not included in the snapshot
### Synthetic GUID
The alarmmgr provider supplies a native GUID per alarm record. The subtag
provider has no native GUID. `SubtagAlarmConsumer` derives a deterministic
GUID by hashing `alarm_full_reference` (via `SyntheticAlarmGuid.ForReference`).
The same reference always produces the same GUID within a session, so
GUID-based ack routing resolves correctly. The GUID is not stable across
different alarm references or gateway restarts in the sense of matching any
AVEVA-internal GUID.
### Acknowledge in subtag mode
`AlarmDispatcher` routes ack calls by active provider mode:
- **Alarm-manager mode:** `AlarmAckByName` on `wwAlarmConsumerClass` (unchanged).
- **Subtag mode:** `SubtagAlarmConsumer.AcknowledgeByName` resolves the
watch-list entry's `ack_comment_subtag` and issues a `Write(comment)` on
the STA via `LmxSubtagAlarmSource`. The write performs the ack in AVEVA.
If the alarm has no writable ack-comment subtag (`AckComment` config key is
empty, or the entry's `ack_comment_subtag` field is empty), the ack call
returns a failure code that the gateway surfaces as `FailedPrecondition`.
`AcknowledgeByGuid` maps the synthetic GUID back to its reference via an
internal dictionary, then calls the same write path.
### Fidelity limitations
The following fields are not available or have lower quality in subtag mode:
| Field | Subtag-mode behavior |
|-------|---------------------|
| `alarm_guid` | Synthetic deterministic GUID from `alarm_full_reference`; not an AVEVA-native GUID |
| `original_raise_timestamp` | First observed `active = true` time; no AVEVA-native raise time |
| `transition_timestamp` | `OnDataChange` source timestamp from MXAccess |
| `severity` | From priority subtag if advised; 0 otherwise |
| `category` / `description` | Not populated (no subtag for these) |
| `current_value` / `limit_value` | Not populated unless corresponding subtags are in the watch-list |
| `alarm_type_name` | Not populated |
| `operator_user` / `operator_comment` | Not populated on synthesized raise/clear transitions |
| `retrigger` transition | Not synthesized (no re-alarm counter subtag is observed) |
Every transition and snapshot record carries `degraded = true` and
`source_provider = ALARM_PROVIDER_MODE_SUBTAG`. Clients that require full
fidelity must wait for failback to the alarm manager.
### Provider mode reflection
When `FailoverAlarmConsumer` switches between providers, it raises
`ProviderModeChanged`. `AlarmDispatcher` enqueues an
`OnAlarmProviderModeChangedEvent` (carried as an `MxEvent`), which the
gateway receives and reflects into:
- `AlarmFeedMessage.provider_status` emitted to every `StreamAlarms`
subscriber.
- The `/hubs/alarms` SignalR hub for the dashboard.
- Metrics: `mxgateway.alarms.provider_mode` gauge and
`mxgateway.alarms.provider_switches` counter.
On every switch `GatewayAlarmMonitor` also forces a reconcile
(`QueryActiveAlarms`) against the now-active provider so the gateway cache
reflects the post-switch state without a spurious raise/clear storm.
+52
View File
@@ -411,6 +411,58 @@ a per-channel skip-verify hook:
See [Gateway Configuration — Automatic self-signed certificate](./GatewayConfiguration.md#automatic-self-signed-certificate)
and the per-client READMEs for the as-built behavior.
## Alarm-Manager to Subtag Fallback
Decision: add a second alarm provider (subtag monitoring) that the worker
activates automatically when the native wnwrap alarm manager fails, and fails
back to automatically when the manager recovers.
### Worker-side synthesis
Synthesis of alarm transitions from subtag value changes happens entirely in
the worker (`SubtagAlarmConsumer` / `SubtagAlarmStateMachine`). The gateway
still forwards only events the worker emits and synthesizes nothing itself.
This satisfies the parity rule even though the subtag path is inherently
non-parity: the parity rule governs where synthesis lives, not whether
synthesis is permitted when the native source is unavailable.
### Degraded is explicit
Every subtag-mode transition carries `degraded = true` on the
`OnAlarmTransitionEvent` and `ActiveAlarmSnapshot` proto messages, and the
`AlarmFeedMessage` feed carries an `AlarmProviderStatus` payload on stream
open and on every switch. No client can mistake a subtag-mode alarm for an
authoritative alarmmgr record. Subtag mode has lower fidelity: synthetic
deterministic GUID (SHA-derived from the alarm reference), best-effort
original-raise timestamp, narrower field set. Clients that need full fidelity
must wait for failback.
### Failover trigger
The failover trigger is N consecutive wnwrap COM failures — a `COMException`
thrown by `Subscribe` or `PollOnce`, or a failure HRESULT from
`GetXmlCurrentAlarms2`. A single poll failure does not trigger a switch; the
threshold (default 3, floored at 1) guards against transient COM hiccups. The
counter resets on any clean poll so a flapping provider does not permanently
latch in subtag mode.
### Acknowledge via ack-comment write
In subtag mode, `AcknowledgeAlarm` writes the operator comment to the alarm
attribute's ack-comment subtag (`Fallback:Subtags:AckComment`). The write
performs the native ack in AVEVA. This differs from alarmmgr mode, where
`AlarmAckByName` on `wwAlarmConsumerClass` is called directly. The `AckComment`
subtag name is empty by default; configuring it is required for ack to work in
subtag mode. The exact AVEVA subtag names are not hard-coded — the `Subtags`
config block exists precisely so names are not guessed without validation
against the live MXAccess attribute set.
### Related documentation
- [Gateway Configuration — Alarm Fallback options](./GatewayConfiguration.md#alarm-fallback-options)
- [Alarm Client Discovery — Subtag provider](./AlarmClientDiscovery.md)
- [gRPC Contract — provider_status and degraded fields](./Grpc.md)
## Later Revisit Items
These are explicit post-v1 revisit items, not open blockers:
+68
View File
@@ -230,6 +230,74 @@ behavior.
The alarm monitor is independent of client sessions: `AcknowledgeAlarm` and
`StreamAlarms` are session-less RPCs served by the monitor.
### Alarm fallback options
The `Fallback` sub-section controls how the alarm feed selects between the
native wnwrap alarm-manager provider and the subtag-monitoring fallback.
| Option | Default | Description |
|--------|---------|-------------|
| `MxGateway:Alarms:Fallback:Mode` | `Auto` | Provider selection mode. `Auto` uses the alarm manager as primary and fails over to subtag monitoring after consecutive COM failures, then fails back automatically. `ForceAlarmManager` disables failover. `ForceSubtag` forces subtag monitoring on from startup. Values are case-insensitive. |
| `MxGateway:Alarms:Fallback:ConsecutiveFailureThreshold` | `3` | Number of consecutive wnwrap COM failures (`COMException` or failure HRESULT from `Subscribe` / `GetXmlCurrentAlarms2`) before the monitor switches to subtag mode. Floored at 1. |
| `MxGateway:Alarms:Fallback:FailbackProbeIntervalSeconds` | `30` | While in subtag mode, how often (in seconds) the monitor probes the wnwrap provider to detect recovery. Floored at 1. |
| `MxGateway:Alarms:Fallback:FailbackStableProbes` | `3` | Number of consecutive clean wnwrap probes required before the monitor switches back to the alarm manager. Floored at 1. |
| `MxGateway:Alarms:Fallback:Discovery:UseGalaxyRepository` | `true` | When `true`, the monitor queries the Galaxy Repository SQL database to build the subtag watch-list for the configured area. |
| `MxGateway:Alarms:Fallback:Discovery:Area` | _(empty)_ | Galaxy area to scope the Repository query to. Falls back to `MxGateway:Alarms:DefaultArea` when empty. Ignored when `UseGalaxyRepository` is `false`. |
| `MxGateway:Alarms:Fallback:Discovery:IncludeAttributes` | _(empty)_ | Explicit MXAccess attribute paths to add to the subtag watch-list, supplementing (or replacing, when `UseGalaxyRepository` is `false`) the Repository-derived list. |
| `MxGateway:Alarms:Fallback:Discovery:ExcludeAttributes` | _(empty)_ | Attribute paths to remove from the Repository-derived watch-list. Ignored when `UseGalaxyRepository` is `false`. |
| `MxGateway:Alarms:Fallback:Subtags:Active` | `active` | Subtag name for the in-alarm boolean. |
| `MxGateway:Alarms:Fallback:Subtags:Acked` | `acked` | Subtag name for the acknowledged boolean. |
| `MxGateway:Alarms:Fallback:Subtags:AckComment` | _(empty)_ | Subtag name for the acknowledgement comment attribute. When empty, writing an ack comment in subtag mode is disabled. Must be verified against the live MXAccess attribute set before use. |
| `MxGateway:Alarms:Fallback:Subtags:Priority` | `priority` | Subtag name for the alarm priority / severity value. |
Validation rules:
- `Mode` must be `Auto`, `ForceAlarmManager`, or `ForceSubtag` (case-insensitive).
- `Mode = ForceSubtag` with both `UseGalaxyRepository = false` and an empty
`IncludeAttributes` list produces a startup validation warning: the subtag
provider has no attributes to advise.
- `ConsecutiveFailureThreshold`, `FailbackProbeIntervalSeconds`, and
`FailbackStableProbes` are floored at 1 by `GatewayOptionsValidator`.
Full example with non-default fallback settings:
```json
{
"MxGateway": {
"Alarms": {
"Enabled": true,
"SubscriptionExpression": "\\\\SCADA01\\Galaxy!PlantArea",
"DefaultArea": "PlantArea",
"ReconcileIntervalSeconds": 30,
"Fallback": {
"Mode": "Auto",
"ConsecutiveFailureThreshold": 3,
"FailbackProbeIntervalSeconds": 30,
"FailbackStableProbes": 3,
"Discovery": {
"UseGalaxyRepository": true,
"Area": "",
"IncludeAttributes": [],
"ExcludeAttributes": []
},
"Subtags": {
"Active": "active",
"Acked": "acked",
"AckComment": "",
"Priority": "priority"
}
}
}
}
}
```
The exact AVEVA subtag names for `Active`, `Acked`, `AckComment`, and
`Priority` are not hard-coded. The `Subtags` block exists so names can be
confirmed against the live MXAccess attribute set and configured without a
code change. See `docs/AlarmClientDiscovery.md` for the synthesis rules that
depend on these names.
## Host Endpoints and Transport Security (Kestrel)
The listening endpoints are **not** part of the `MxGateway` section. The gateway
+67
View File
@@ -94,6 +94,73 @@ Carrying the enqueue timestamp into the worker layer is what lets queue-wait tim
`StreamAlarms` is a server-streaming, **session-less** RPC that attaches to the gateway's central alarm feed. The handler delegates to `IGatewayAlarmService.StreamAsync`. The stream opens with one `AlarmFeedMessage` carrying an `active_alarm` per currently-active alarm (the ConditionRefresh snapshot), then a single `snapshot_complete`, then a `transition` for every subsequent raise / acknowledge / clear. It is served by the always-on `GatewayAlarmMonitor`, which owns a single gateway-managed worker session and fans out to every attached client — clients no longer open a session of their own. `alarm_filter_prefix`, when set, scopes the stream to a sub-tree.
#### Provider status on the alarm feed
`AlarmFeedMessage` has a fourth `payload` case, `provider_status`, carrying
an `AlarmProviderStatus` message:
```protobuf
message AlarmProviderStatus {
AlarmProviderMode mode = 1;
bool degraded = 2; // true whenever mode == SUBTAG
string reason = 3; // human-readable switch reason
google.protobuf.Timestamp since = 4;
}
```
The gateway emits `provider_status` once when a client first subscribes
(immediately after the initial snapshot and before the first live transition)
and again on every failover or failback. A late-joining client therefore
always learns the current provider mode without waiting for the next switch.
`AlarmProviderMode` is an enum with three values:
| Value | Meaning |
|-------|---------|
| `ALARM_PROVIDER_MODE_UNSPECIFIED` (0) | Default / unset |
| `ALARM_PROVIDER_MODE_ALARMMGR` (1) | Native wnwrap alarm-manager source |
| `ALARM_PROVIDER_MODE_SUBTAG` (2) | Subtag-monitoring fallback (degraded) |
#### Degraded and source-provider fields on transitions and snapshots
`OnAlarmTransitionEvent` and `ActiveAlarmSnapshot` both carry two new fields:
- `bool degraded` (field 14) — `true` when the record came from the subtag
fallback, not the native alarmmgr.
- `AlarmProviderMode source_provider` (field 15) — which provider produced
this record (`ALARMMGR` or `SUBTAG`).
Both fields are proto3 defaults (`false` / `UNSPECIFIED`) in alarmmgr mode,
so existing clients that do not read them continue to function without change.
Clients that care about provenance — for example, an OPC UA server that
applies different quality flags to degraded alarms — should inspect `degraded`
before consuming the transition.
Subtag-mode records are a non-parity source. They carry synthetic GUIDs,
best-effort timestamps, and reduced field coverage. See
`docs/AlarmClientDiscovery.md` for the full fidelity table.
#### Provider-mode-changed event
The worker emits `OnAlarmProviderModeChangedEvent` (family
`MX_EVENT_FAMILY_ON_ALARM_PROVIDER_MODE_CHANGED`) on each switch between
providers:
```protobuf
message OnAlarmProviderModeChangedEvent {
AlarmProviderMode mode = 1;
string reason = 2;
int32 hresult = 3; // COM HRESULT that triggered failover; 0 on failback
google.protobuf.Timestamp at = 4;
}
```
This event arrives on the `StreamEvents` stream of the alarm monitor's
internal gateway session (not on client sessions). `GatewayAlarmMonitor`
consumes it and reflects the new mode into the `StreamAlarms` feed's
`provider_status`, the dashboard hub, and metrics. Client sessions do not
receive this event directly.
## Validation Rules
`MxAccessGrpcRequestValidator` rejects requests with `StatusCode.InvalidArgument` before any session work happens. The rules are intentionally narrow — anything that requires session state (for example, "session does not exist") is left for `ISessionManager` so the validator can stay synchronous and side-effect free.
@@ -0,0 +1,302 @@
# Alarm Subtag-Monitoring Fallback — Design
**Date:** 2026-06-13
**Status:** Approved (brainstorming), ready for implementation planning
**Branch:** `feat/alarm-subtag-fallback`
## Problem
The gateway's central alarm feed (`GatewayAlarmMonitor` → worker
`WnWrapAlarmConsumer`) depends on the AVEVA wnwrap COM consumer
(`WNWRAPCONSUMERLib.wwAlarmConsumerClass`), which polls `GetXmlCurrentAlarms2`
on the worker STA. That provider can fail at the COM boundary (the older
`aaAlarmManagedClient` crashed on FILETIME marshaling; wnwrap can still return
failure HRESULTs or throw `COMException`). When it does, the gateway loses all
alarm visibility.
This design adds a **second alarm source** — direct monitoring of each alarm
attribute's subtags (`.active`, `.acked`, …) via the existing MXAccess
`AddItem`/`Advise` pipeline — and **fails over to it automatically when the
wnwrap provider breaks, then fails back automatically when it recovers**. The
subtag source can also be forced on by config.
## Decisions (locked during brainstorming)
| Decision | Choice |
|---|---|
| Failover model | **Auto-failover + auto-failback** (both directions, runtime) |
| Watch-list source | **Galaxy Repository SQL discovery + config override** |
| Acknowledge in subtag mode | **Write the operator comment to the alarm's ack-comment subtag** (the write performs the ack) |
| Failure signal | **N consecutive wnwrap COM failures** (Subscribe / `GetXmlCurrentAlarms2` throws or returns a failure HRESULT) |
| Degraded-state visibility | **Both** — explicit field in the gRPC contract **and** dashboard + metrics |
| Synthesis location | **Worker-side** (`Approach A`) — keeps the parity rule "the gateway forwards only events the worker emits; it never synthesizes events" |
## Core principle
Subtag monitoring is, by definition, a **non-parity, lower-fidelity** alarm
source: it synthesizes alarm transitions from raw data changes, has no native
alarm GUID, no native original-raise timestamp, and a narrower field set. Per
`CLAUDE.md`, synthesizing events is allowed only as an explicit opt-in
non-parity mode. This design satisfies that by (a) doing the synthesis **inside
the worker** (so the gateway still only forwards worker-emitted events) and
(b) marking every degraded event and the whole feed as degraded so no client
mistakes it for the authoritative alarmmgr feed.
## Architecture
```
GATEWAY (.NET 10, x64)
┌─────────────────────────────────────────────────────────────────┐
│ GatewayAlarmMonitor (BackgroundService) │
│ • resolves watch-list: Galaxy Repository SQL + config override │
│ • arms the worker with the watch-list at subscribe time │
│ • consumes AlarmProviderModeChanged → reflects mode into feed, │
│ /hubs/alarms dashboard hub, and metrics │
│ • forces a cache reconcile (QueryActiveAlarms) on every switch │
└───────────────────────────────┬───────────────────────────────────┘
│ IPC (WorkerEnvelope frames)
│ · SubscribeAlarms{ watch_list, failover cfg }
│ · AlarmProviderModeChanged{ mode, reason, hresult }
│ · OnAlarmTransitionEvent (degraded flag set in subtag mode)
▼
WORKER (.NET FW 4.8, x86, STA)
┌─────────────────────────────────────────────────────────────────┐
│ AlarmDispatcher → FailoverAlarmConsumer : IMxAccessAlarmConsumer │
│ ├─ primary : WnWrapAlarmConsumer (wnwrap COM poll, unchanged) │
│ └─ standby : SubtagAlarmConsumer (AddItem/Advise on subtags) │
│ │
│ FailoverAlarmConsumer owns the state machine: │
│ PrimaryActive ──(N consecutive wnwrap COM failures)──▶ Degraded │
│ Degraded ──(M consecutive clean wnwrap probe polls)──▶ Primary │
│ on each switch: snapshot the now-active provider, hand off │
└─────────────────────────────────────────────────────────────────┘
```
The failover state machine lives **worker-local** so the switch is instant — no
IPC round-trip at the moment alarmmgr dies. The gateway *arms* the standby
consumer up front (passes the watch-list at subscribe time) so it is ready
before it is ever needed.
## Components
### Worker (`src/ZB.MOM.WW.MxGateway.Worker/MxAccess/`)
**`SubtagAlarmConsumer : IMxAccessAlarmConsumer` (new)** — the standby provider.
- On `Subscribe`, instead of wnwrap registration it `AddItem`/`Advise`s the
configured subtags for each watch-list entry on the existing STA (reuses the
worker's item-subscription machinery). Per attribute it advises at minimum
`.active` and `.acked`; optionally `.priority`/severity, `.descr`, value/limit
if present.
- Converts each `OnDataChange` into the same `MxAlarmTransitionEvent` the wnwrap
consumer emits, via the synthesis rules below, and raises
`AlarmTransitionEmitted`. Marks each as **degraded**.
- `SnapshotActiveAlarms()` returns the currently-active set computed from
last-known subtag values.
- `AcknowledgeByName(...)` resolves the watch-list entry's ack-comment subtag and
issues a `Write(comment)` on the STA. `AcknowledgeByGuid(...)` maps the
synthetic GUID (see below) back to a reference, then does the same. If the
attribute exposes no writable ack-comment subtag, returns a failure code that
the gateway surfaces as `FailedPrecondition`.
- `PollOnce()` is a no-op (subtag mode is event-driven via Advise).
**`FailoverAlarmConsumer : IMxAccessAlarmConsumer` (new)** — composite + state
machine. Owns the wnwrap consumer (primary) and the subtag consumer (standby),
forwards `AlarmTransitionEmitted` from whichever child is active, and raises a
new `ProviderModeChanged` event on every switch.
- **Failure counting:** wraps `Subscribe`/`PollOnce` on the primary; a thrown
`COMException` or a failure HRESULT increments a consecutive-failure counter,
reset to zero on any clean poll.
- **Failover** (`PrimaryActive → Degraded`): at `ConsecutiveFailureThreshold`
(default 3), ensures the standby is subscribed (it was armed at startup), sets
active = standby, snapshots the standby's active set for hand-off, and emits
`ProviderModeChanged(SUBTAG, reason, hresult)`.
- **Failback probe** (`Degraded → PrimaryActive`): while degraded, every
`FailbackProbeIntervalSeconds` (default 30) it re-attempts wnwrap
`Subscribe`+`PollOnce` on the STA. After `FailbackStableProbes` (default 3)
consecutive clean polls it switches active = primary, returns the standby to
standby, and emits `ProviderModeChanged(ALARMMGR, "recovered")`.
- **Hand-off:** on every switch it takes `SnapshotActiveAlarms()` from the
now-active provider so the gateway can reconcile and avoid spurious
raise/clear storms.
**`AlarmDispatcher` / `MxAccessAlarmEventSink` / `AlarmCommandHandler`
(changed, minimal)** — `AlarmDispatcher` holds a `FailoverAlarmConsumer` instead
of a bare `WnWrapAlarmConsumer`; it subscribes to `ProviderModeChanged` and
enqueues a mode-changed worker event. The ack path routes by active mode (native
wnwrap ack in alarmmgr mode; ack-comment write in subtag mode), but that routing
is entirely inside the consumer — the dispatcher just calls
`AcknowledgeByName`/`AcknowledgeByGuid`.
### Gateway (`src/ZB.MOM.WW.MxGateway.Server/`)
**Galaxy Repository discovery (new query)** — alongside the existing GR SQL
browse RPCs, a query "attributes that have alarms configured, with their
ack-comment subtag and area", scoped to the configured area. Merged with the
config override (explicit includes/excludes). Produces the watch-list of
`AlarmSubtagTarget`s.
**`GatewayAlarmMonitor` (changed)** — resolves the watch-list at subscribe time
and passes it to the worker; consumes `AlarmProviderModeChanged` and reflects
the current provider mode into (a) the `AlarmFeedMessage` provider-status,
(b) the `/hubs/alarms` dashboard hub, and (c) metrics; forces a reconcile
(`QueryActiveAlarms`) on every switch. Re-runs discovery on its existing
reconcile cadence and pushes an updated watch-list when the model changes.
**`AlarmsOptions` (extended)** — new `Fallback` sub-section (below).
### Contract (`src/ZB.MOM.WW.MxGateway.Contracts/Protos/`)
**`mxaccess_gateway.proto`:**
- `enum AlarmProviderMode { ALARM_PROVIDER_MODE_UNSPECIFIED = 0; ALARMMGR = 1; SUBTAG = 2; }`
- New `AlarmFeedMessage` oneof case `AlarmProviderStatus provider_status`,
carrying `{ AlarmProviderMode mode; bool degraded; string reason;
google.protobuf.Timestamp since; }`. Emitted on stream open and on every
change so a late-joining client immediately learns the mode.
- Add `bool degraded` + `AlarmProviderMode source_provider` to
`OnAlarmTransitionEvent` **and** `ActiveAlarmSnapshot`, so per-item provenance
is visible even mid-stream. All additions are new field numbers — backward
compatible; existing clients ignore them and keep seeing alarms.
**`mxaccess_worker.proto`:**
- Extend the alarm-subscribe command with: `AlarmProviderMode forced_mode`
(`UNSPECIFIED` = auto), `int32 consecutive_failure_threshold`,
`int32 failback_probe_interval_seconds`, `int32 failback_stable_probes`, and
`repeated AlarmSubtagTarget watch_list`, where `AlarmSubtagTarget =
{ string alarm_full_reference; string source_object_reference;
string active_subtag; string acked_subtag; string ack_comment_subtag;
string priority_subtag; }`.
- New worker→gateway event `AlarmProviderModeChanged { AlarmProviderMode mode;
string reason; int32 hresult; google.protobuf.Timestamp at; }`.
> Generated code under `Generated/` and `clients/*/generated*/` is rebuilt from
> these `.proto` files — never hand-edited. Every generated client touched by
> the contract is rebuilt per the source-update workflow.
## Data flow
### Subtag synthesis rules
`SubtagAlarmConsumer` keeps last-known `(active, acked)` per watch-list entry and
emits transitions on change:
| Subtag change | Emitted transition | Notes |
|---|---|---|
| `active` false → true | `RAISE` (state `UNACK_ALM`) | `original_raise_timestamp` = first-observed active time |
| `acked` false → true while `active` | `ACKNOWLEDGE` | `operator_user`/`operator_comment` from ack-comment subtag if advised |
| `active` true → false | `CLEAR` | maps to `AckRtn` if acked at clear, else `UnackRtn` |
| `active` stays true, re-alarm | `RETRIGGER` | **only** if a re-alarm counter subtag exists; otherwise not synthesized (documented limitation) |
Snapshot state mapping for `ActiveAlarmSnapshot.current_state`:
`active && !acked → ACTIVE`, `active && acked → ACTIVE_ACKED`,
`!active → INACTIVE`.
Field degradation in subtag mode:
- `alarm_full_reference` — from the watch-list entry (stable, drives ack-by-ref).
- Synthetic, deterministic GUID derived by hashing `alarm_full_reference` so
GUID-based ack still resolves; flagged `degraded = true`.
- `severity` — from the priority subtag if advised, else 0.
- `original_raise_timestamp` — first-observed active time (best effort).
- `transition_timestamp` — the `OnDataChange` timestamp.
- `category`/`description`/`current_value`/`limit_value` — populated only if the
corresponding subtag is advised; otherwise empty.
### Acknowledge
`AcknowledgeAlarm`/`AcknowledgeAlarmByName` are unchanged at the RPC surface.
`AlarmDispatcher` routes by active provider mode:
- **alarmmgr mode:** native wnwrap `AlarmAckByName`/`AlarmAckByGUID` (unchanged).
- **subtag mode:** resolve the target's `ack_comment_subtag`, `Write` the
operator comment via the existing worker write path on the STA. No writable
ack-comment subtag → `FailedPrecondition`.
### Provider-mode reflection
Worker `AlarmProviderModeChanged` → `GatewayAlarmMonitor` → (a) emit/refresh
`AlarmFeedMessage.provider_status` to every `StreamAlarms` subscriber, (b) push
to `/hubs/alarms`, (c) update metrics, (d) force a reconcile.
## Error handling
- **Both providers down** (subtag advise also failing): the monitor stays
faulted and keeps retrying both; acknowledge returns `Unavailable`. No silent
data loss — the feed reports degraded with reason.
- **Empty watch-list in subtag mode** (GR SQL unavailable, no config override):
log + metric `alarm_fallback_watchlist_empty`; the feed reports degraded +
empty; the gateway keeps re-running discovery on its reconcile cadence and
pushes an updated watch-list when one becomes available.
- **Switch hand-off:** every switch snapshots the now-active provider and
reconciles against the gateway cache to avoid a raise/clear storm.
- **STA affinity:** all subtag advise/write and wnwrap probe calls run on the
worker STA (reuse the existing affinity guard) to satisfy
`ThreadingModel=Apartment`.
### Metrics
- `mxgateway_alarm_provider_mode` (gauge: 1 = alarmmgr, 2 = subtag)
- `mxgateway_alarm_provider_switch_total{from,to,reason}` (counter)
- `mxgateway_alarm_fallback_watchlist_size` (gauge)
## Configuration
```jsonc
"MxGateway": {
"Alarms": {
"Enabled": true,
"SubscriptionExpression": "\\\\DESKTOP-6JL3KKO\\Galaxy!DEV",
"DefaultArea": "DEV",
"ReconcileIntervalSeconds": 30,
"Fallback": {
"Mode": "Auto", // Auto | ForceAlarmManager | ForceSubtag
"ConsecutiveFailureThreshold": 3,
"FailbackProbeIntervalSeconds": 30,
"FailbackStableProbes": 3,
"Discovery": {
"UseGalaxyRepository": true,
"Area": "", // defaults to Alarms.DefaultArea
"IncludeAttributes": [], // explicit additions
"ExcludeAttributes": []
},
"Subtags": {
"Active": "active",
"Acked": "acked",
"AckComment": "", // verified against MXAccess analysis
"Priority": "priority"
}
}
}
}
```
`GatewayOptionsValidator` additions: `Mode = ForceSubtag` with empty discovery
result and no explicit `IncludeAttributes` → startup validation warning;
threshold/interval/probe values floored at sane minimums.
## Open item to confirm during implementation
The exact AVEVA subtag names (`.active`, `.acked`, the ack-comment attribute,
priority) must be confirmed against the MXAccess analysis project
(`C:\Users\dohertj2\Desktop\mxaccess`, `docs/MXAccess-Public-API.md`) and the
live Galaxy before wiring `SubtagAlarmConsumer`. The config `Subtags` block
exists precisely so the resolved names are not hard-coded.
## Testing
| Layer | Tests |
|---|---|
| Worker unit (`MxGateway.Worker.Tests`, x86) | `SubtagAlarmConsumer` synthesis — feed `OnDataChange` sequences, assert raise/ack/clear transitions, snapshot states, degraded flag, synthetic-GUID stability, ack-comment write routing |
| Worker unit | `FailoverAlarmConsumer` state machine — fake wnwrap throwing after K polls: assert switch at threshold, failback after stable probes, `ProviderModeChanged` emitted, no duplicate transitions across switch (hand-off reconcile) |
| Gateway unit (`MxGateway.Tests`, fake worker) | discovery + config-override merge; `GatewayAlarmMonitor` reflects mode into feed + hub; metrics increment on switch |
| Contract | proto round-trip for new fields; existing alarm tests unchanged (alarmmgr-mode regression — parity preserved) |
| Live (opt-in, `MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`) | real subtag advise + ack-comment write against a live alarm; GR SQL discovery query against the `ZB` DB (gated like existing GR tests) |
## Docs to update in the same change
`gateway.md` (alarm provider section), `docs/DesignDecisions.md` (record the
fallback decision), `docs/GatewayConfiguration.md` (the `Fallback` block),
`docs/AlarmClientDiscovery.md` (subtag provider + synthesis rules),
`docs/Grpc.md` (the new `provider_status` / `degraded` fields), and any client
READMEs whose generated alarm types gain fields.
@@ -0,0 +1,860 @@
# Alarm Subtag-Monitoring Fallback — Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans (or subagent-driven-development) to implement this plan task-by-task.
**Goal:** Add a second alarm source — direct MXAccess subtag monitoring — that the gateway auto-fails-over to when the wnwrap alarmmgr provider breaks, auto-fails-back to when it recovers, and can be forced on by config.
**Architecture:** Worker-side synthesis (parity rule preserved). A new `SubtagAlarmConsumer` (own `LMXProxyServerClass`, `AddItem`/`Advise` on alarm subtags) and a `FailoverAlarmConsumer` composite (state machine over the wnwrap primary + subtag standby) both implement the existing `IMxAccessAlarmConsumer` seam. The gateway resolves the subtag watch-list (Galaxy Repository SQL + config override), arms the worker at subscribe time, and reflects the live provider mode into the gRPC alarm feed, the dashboard hub, and metrics.
**Tech Stack:** .NET 10 (gateway, x64) + .NET Framework 4.8 (worker, x86, STA), protobuf/gRPC, `Microsoft.Data.SqlClient` (Galaxy Repository), SignalR (dashboard), `System.Diagnostics.Metrics`, xUnit (plain `Assert`, no FluentAssertions).
**Design source:** `docs/plans/2026-06-13-alarm-subtag-fallback-design.md`
**Branch:** `feat/alarm-subtag-fallback` (already created)
---
## Conventions for every task
- **TDD:** write the failing test, run it red, implement, run it green, commit.
- **xUnit, plain `Assert.*`**, naming `Subject_Condition_Expected`. Worker fakes are sealed private nested classes that raise events.
- **Build/test commands:**
- Contracts regen: `dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj`
- Gateway: `dotnet build src/ZB.MOM.WW.MxGateway.Server` ; `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj`
- Worker (x86): `dotnet build src/ZB.MOM.WW.MxGateway.Worker/ZB.MOM.WW.MxGateway.Worker.csproj -p:Platform=x86` ; `dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86`
- Single test: append `--filter FullyQualifiedName~<ClassOrMethod>`
- **Build is strict:** `TreatWarningsAsErrors=true`, nullable enabled. Add XML doc comments on public members (the repo runs a doc checker).
- **Generated code** under `Generated/` is never hand-edited — rebuild the contracts project to regenerate.
- **Namespaces:** worker MxAccess types live in `ZB.MOM.WW.MxGateway.Worker.MxAccess`; proto C# types in `ZB.MOM.WW.MxGateway.Contracts.Proto`.
---
## Phase 0 — Contracts
### Task 1: Worker proto — subtag watch-list, failover config, provider-mode enum
**Classification:** high-risk
**Estimated implement time:** ~4 min
**Parallelizable with:** none (Task 2 imports these types)
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto` (real `SubscribeAlarmsCommand` at ~line 324; `MxCommand` references it at 123-125)
> **CORRECTION (execution):** The alarm command messages and `MxCommand` live in **`mxaccess_gateway.proto`**, not the worker proto. `mxaccess_worker.proto` *imports* the gateway proto (`WorkerCommand.command` is `mxaccess_gateway.v1.MxCommand`), so the gateway proto is the base and the worker proto needs **no** change. `AlarmProviderMode` and the new types are added to the gateway proto and are visible to worker code as `mxaccess_gateway.v1` types. Tasks 1 and 2 are executed as a single combined edit on this one file.
**Step 1: Add the enum and messages.** In `mxaccess_gateway.proto`, extend the existing `SubscribeAlarmsCommand` message (line 324) and add the new types after it:
```protobuf
// Provider selection / current provider for the alarm feed. Defined here in
// the worker contract because the worker SubscribeAlarmsCommand references it;
// mxaccess_gateway.proto imports this file and reuses the same enum.
enum AlarmProviderMode {
ALARM_PROVIDER_MODE_UNSPECIFIED = 0; // auto: alarmmgr primary, subtag fallback
ALARM_PROVIDER_MODE_ALARMMGR = 1;
ALARM_PROVIDER_MODE_SUBTAG = 2;
}
message SubscribeAlarmsCommand {
string subscription_expression = 1; // existing field — keep
// UNSPECIFIED = auto-failover/failback. ALARMMGR/SUBTAG force one provider.
AlarmProviderMode forced_mode = 2;
// Subtag watch-list resolved by the gateway (GR SQL + config). Empty in pure
// alarmmgr mode; in subtag mode it bounds what the consumer can observe.
repeated AlarmSubtagTarget watch_list = 3;
AlarmFailoverConfig failover = 4;
}
// One alarm attribute the subtag consumer advises. Addresses are full MXAccess
// item references the worker passes straight to AddItem.
message AlarmSubtagTarget {
string alarm_full_reference = 1; // e.g. "Galaxy!Area.Tank01.Level.HiHi"
string source_object_reference = 2; // e.g. "Tank01"
string active_subtag = 3; // item address of the in-alarm boolean
string acked_subtag = 4; // item address of the acknowledged boolean
string ack_comment_subtag = 5; // writable ack-comment attribute (ack write target)
string priority_subtag = 6; // optional severity source; empty if absent
}
message AlarmFailoverConfig {
int32 consecutive_failure_threshold = 1; // wnwrap COM failures before switching (>=1)
int32 failback_probe_interval_seconds = 2; // probe cadence while degraded (>=1)
int32 failback_stable_probes = 3; // clean probes before switching back (>=1)
}
```
`UnsubscribeAlarmsCommand` and `AcknowledgeAlarmCommand` are unchanged.
**Step 2: Regenerate & verify it compiles.**
Run: `dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj`
Expected: build succeeds; generated `AlarmProviderMode`, `AlarmSubtagTarget`, `AlarmFailoverConfig` types appear.
**Step 3: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_worker.proto
git commit -m "contracts(worker): subtag watch-list + failover config + AlarmProviderMode"
```
---
### Task 2: Gateway proto — provider status on the feed, degraded provenance, mode-changed event
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none (depends on Task 1; Task 3 tests both)
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto` (`OnAlarmTransitionEvent` ~719-771, `ActiveAlarmSnapshot` ~783-803, `AlarmFeedMessage` ~860-870, `MxEvent` family enum + body oneof, `MxEventFamily` enum)
**Step 1: Add degraded provenance to the two alarm payloads.** Append to `OnAlarmTransitionEvent` (next free field 14):
```protobuf
// True when this transition came from the subtag-monitoring fallback rather
// than the native alarmmgr provider — i.e. it was synthesized from data
// changes and carries reduced fidelity (synthetic GUID, no native raise time).
bool degraded = 14;
// Which provider produced this transition.
AlarmProviderMode source_provider = 15;
```
Append the identical two fields to `ActiveAlarmSnapshot` (next free field 14):
```protobuf
bool degraded = 14;
AlarmProviderMode source_provider = 15;
```
**Step 2: Add provider status to the feed oneof.** Add a new oneof case to `AlarmFeedMessage` (next free field 4) and a new message:
```protobuf
message AlarmFeedMessage {
oneof payload {
ActiveAlarmSnapshot active_alarm = 1;
bool snapshot_complete = 2;
OnAlarmTransitionEvent transition = 3;
// Provider-mode status. Emitted once on stream open and again on every
// failover/failback so late joiners learn the current mode immediately.
AlarmProviderStatus provider_status = 4;
}
}
message AlarmProviderStatus {
AlarmProviderMode mode = 1;
bool degraded = 2; // true whenever mode == SUBTAG
string reason = 3; // human-readable switch reason
google.protobuf.Timestamp since = 4;
}
```
**Step 3: Add the worker→gateway mode-changed event to `MxEvent`.** Find the `MxEventFamily` enum and the `MxEvent` body oneof. Add a family member and a body message + oneof case (use the next free family value and the next free `MxEvent` body field number — check the file):
```protobuf
// in MxEventFamily enum:
MX_EVENT_FAMILY_ON_ALARM_PROVIDER_MODE_CHANGED = <next>;
// new message near OnAlarmTransitionEvent:
message OnAlarmProviderModeChangedEvent {
AlarmProviderMode mode = 1;
string reason = 2;
int32 hresult = 3; // COM HRESULT that triggered failover; 0 on failback
google.protobuf.Timestamp at = 4;
}
// in MxEvent body oneof:
OnAlarmProviderModeChangedEvent on_alarm_provider_mode_changed = <next>;
```
`AlarmProviderMode` is defined in `mxaccess_worker.proto`; confirm `mxaccess_gateway.proto` already has `import "mxaccess_worker.proto";` (it references `SubscribeAlarmsCommand`, so it does) and reference the enum unqualified or via its package as the existing references do.
**Step 4: Regenerate & verify.**
Run: `dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj`
Expected: build succeeds.
**Step 5: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto
git commit -m "contracts(gateway): AlarmProviderStatus feed case, degraded provenance, mode-changed event"
```
---
### Task 3: Proto round-trip tests for the new alarm fields
**Classification:** small
**Estimated implement time:** ~3 min
**Parallelizable with:** none (depends on Tasks 1-2)
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Tests/Contracts/ProtobufContractRoundTripTests.cs`
**Step 1: Add tests** mirroring the existing `Event_RoundTripsOnAlarmTransitionWithFullPayload` style:
```csharp
[Fact]
public void Feed_RoundTripsProviderStatus()
{
var since = Timestamp.FromDateTime(new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc));
var original = new AlarmFeedMessage
{
ProviderStatus = new AlarmProviderStatus
{
Mode = AlarmProviderMode.Subtag,
Degraded = true,
Reason = "wnwrap poll failed 3x (HRESULT 0x80004005)",
Since = since,
},
};
var parsed = AlarmFeedMessage.Parser.ParseFrom(original.ToByteArray());
Assert.Equal(original, parsed);
Assert.Equal(AlarmFeedMessage.PayloadOneofCase.ProviderStatus, parsed.PayloadCase);
Assert.True(parsed.ProviderStatus.Degraded);
Assert.Equal(AlarmProviderMode.Subtag, parsed.ProviderStatus.Mode);
}
[Fact]
public void Transition_RoundTripsDegradedProvenance()
{
var t = new OnAlarmTransitionEvent
{
AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi",
TransitionKind = AlarmTransitionKind.Raise,
Degraded = true,
SourceProvider = AlarmProviderMode.Subtag,
};
var parsed = OnAlarmTransitionEvent.Parser.ParseFrom(t.ToByteArray());
Assert.True(parsed.Degraded);
Assert.Equal(AlarmProviderMode.Subtag, parsed.SourceProvider);
}
```
**Step 2: Run red→green.**
Run: `dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter FullyQualifiedName~ProtobufContractRoundTripTests`
Expected: PASS.
**Step 3: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Tests/Contracts/ProtobufContractRoundTripTests.cs
git commit -m "test(contracts): round-trip provider status + degraded provenance"
```
---
## Phase 1 — Worker: subtag consumer + failover
### Task 4: Subtag value-source abstraction + synthesis state holder
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none (Task 5 builds on it)
A testable seam so synthesis logic is unit-tested without COM. The COM wiring lands in Task 6.
**Files:**
- Create: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/ISubtagAlarmSource.cs`
- Create: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/SubtagAlarmStateMachine.cs`
- Test: `src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/SubtagAlarmStateMachineTests.cs`
**Step 1: Define the source abstraction.** `ISubtagAlarmSource` advises subtag addresses and raises a normalized value-change callback on the STA:
```csharp
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>A change in one advised subtag value, normalized off the COM boundary.</summary>
public sealed class SubtagValueChange
{
/// <summary>The full item address that changed (matches an AlarmSubtagTarget subtag).</summary>
public string ItemAddress { get; init; } = string.Empty;
/// <summary>The new value (boolean for .active/.acked, numeric for priority).</summary>
public object? Value { get; init; }
/// <summary>The change timestamp in UTC.</summary>
public DateTime TimestampUtc { get; init; }
}
/// <summary>
/// Advises a set of MXAccess subtag addresses and surfaces value changes.
/// The production implementation (Task 6) owns its own LMXProxyServerClass;
/// tests substitute a fake that pushes <see cref="SubtagValueChange"/>s.
/// </summary>
public interface ISubtagAlarmSource : IDisposable
{
/// <summary>Raised on the STA when an advised subtag's value changes.</summary>
event EventHandler<SubtagValueChange>? ValueChanged;
/// <summary>Advises every subtag in the supplied addresses; idempotent per address.</summary>
void Advise(IReadOnlyCollection<string> itemAddresses);
/// <summary>Writes a value to an item address (used for the ack-comment write).</summary>
void Write(string itemAddress, object? value);
}
```
**Step 2: Write the state-machine tests first.** `SubtagAlarmStateMachine` maps `(active, acked)` changes per target to `MxAlarmTransitionEvent`s. Test the four core transitions:
```csharp
namespace ZB.MOM.WW.MxGateway.Worker.Tests.MxAccess;
public sealed class SubtagAlarmStateMachineTests
{
private static AlarmSubtagTarget Target() => new()
{
AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi",
SourceObjectReference = "Tank01",
ActiveSubtag = "Tank01.Level.HiHi.active",
AckedSubtag = "Tank01.Level.HiHi.acked",
AckCommentSubtag = "Tank01.Level.HiHi.ackmsg",
};
[Fact]
public void ActiveFalseToTrue_EmitsRaise_FlaggedDegraded()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
var events = sm.Apply("Tank01.Level.HiHi.active", true, ts);
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.UnackAlm, e.Record.State);
Assert.Equal(MxAlarmStateKind.Unspecified, e.PreviousState);
Assert.Equal("Tank01.Level.HiHi", e.Record.TagName); // reference minus provider/area
}
[Fact]
public void AckedTrueWhileActive_EmitsAckTransition()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
var events = sm.Apply("Tank01.Level.HiHi.acked", true, ts.AddSeconds(5));
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.AckAlm, e.Record.State);
Assert.Equal(MxAlarmStateKind.UnackAlm, e.PreviousState);
}
[Fact]
public void ActiveTrueToFalse_WhileUnacked_EmitsUnackRtn()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
var events = sm.Apply("Tank01.Level.HiHi.active", false, ts.AddSeconds(10));
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.UnackRtn, e.Record.State);
}
[Fact]
public void Snapshot_ReflectsActiveAndAckedState()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
sm.Apply("Tank01.Level.HiHi.acked", true, ts);
var snap = Assert.Single(sm.SnapshotActive());
Assert.Equal(MxAlarmStateKind.AckAlm, snap.State);
}
}
```
Run: `dotnet test ...Worker.Tests... -p:Platform=x86 --filter FullyQualifiedName~SubtagAlarmStateMachineTests` → FAIL (type missing).
**Step 3: Implement `SubtagAlarmStateMachine`.** Build an address→target index (active/acked/priority/comment addresses), hold per-reference `(bool active, bool acked, DateTime firstRaiseUtc, int priority)`, and emit on change:
- active `false→true` ⇒ `UnackAlm`, set `firstRaiseUtc`, `PreviousState` from prior state.
- acked `false→true` while active ⇒ `AckAlm`.
- active `true→false` ⇒ `AckRtn` if currently acked else `UnackRtn`; then reset acked.
- priority change ⇒ update stored priority, no transition.
- `TagName` = `alarm_full_reference` with any `Provider!Area.` prefix stripped (match `WnWrapAlarmConsumer`'s reference shape so `GatewayAlarmMonitor` keys align). Set `ProviderName`, `Group`, `Priority`, `AlarmComment` from the target/last values. Mark a `Degraded`/source flag (carried via a new field — see Task 5 wiring).
- `SnapshotActive()` returns `MxAlarmSnapshotRecord` for references whose active is true.
**Step 4: Run green.** Expected: PASS.
**Step 5: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Worker/MxAccess/ISubtagAlarmSource.cs \
src/ZB.MOM.WW.MxGateway.Worker/MxAccess/SubtagAlarmStateMachine.cs \
src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/SubtagAlarmStateMachineTests.cs
git commit -m "worker(alarms): subtag value-source seam + synthesis state machine"
```
---
### Task 5: `SubtagAlarmConsumer` over the source seam (no COM yet)
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none (depends on Task 4)
**Files:**
- Create: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/SubtagAlarmConsumer.cs`
- Test: `src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/SubtagAlarmConsumerTests.cs`
**Step 1: Test with a fake `ISubtagAlarmSource`.** Drive value changes through the source, assert `AlarmTransitionEmitted` fires with synthesized records and that ack writes the comment to the ack-comment subtag:
```csharp
public sealed class SubtagAlarmConsumerTests
{
private sealed class FakeSource : ISubtagAlarmSource
{
public event EventHandler<SubtagValueChange>? ValueChanged;
public List<string> Advised { get; } = new();
public (string Address, object? Value)? LastWrite { get; private set; }
public void Advise(IReadOnlyCollection<string> a) => Advised.AddRange(a);
public void Write(string a, object? v) => LastWrite = (a, v);
public void Raise(string addr, object? val, DateTime ts) =>
ValueChanged?.Invoke(this, new SubtagValueChange { ItemAddress = addr, Value = val, TimestampUtc = ts });
public void Dispose() { }
}
private static AlarmSubtagTarget Target() => new()
{
AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi",
ActiveSubtag = "Tank01.Level.HiHi.active",
AckedSubtag = "Tank01.Level.HiHi.acked",
AckCommentSubtag = "Tank01.Level.HiHi.ackmsg",
};
[Fact]
public void Subscribe_AdvisesAllSubtags()
{
var src = new FakeSource();
using var c = new SubtagAlarmConsumer(src, new[] { Target() });
c.Subscribe("ignored-in-subtag-mode");
Assert.Contains("Tank01.Level.HiHi.active", src.Advised);
Assert.Contains("Tank01.Level.HiHi.acked", src.Advised);
}
[Fact]
public void ValueChange_RaisesSynthesizedTransition()
{
var src = new FakeSource();
using var c = new SubtagAlarmConsumer(src, new[] { Target() });
c.Subscribe("x");
MxAlarmTransitionEvent? seen = null;
c.AlarmTransitionEmitted += (_, e) => seen = e;
src.Raise("Tank01.Level.HiHi.active", true, new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc));
Assert.NotNull(seen);
Assert.Equal(MxAlarmStateKind.UnackAlm, seen!.Record.State);
}
[Fact]
public void AcknowledgeByName_WritesCommentToAckCommentSubtag()
{
var src = new FakeSource();
using var c = new SubtagAlarmConsumer(src, new[] { Target() });
c.Subscribe("x");
int rc = c.AcknowledgeByName("Tank01.Level.HiHi", "Galaxy", "Area",
"ack from HMI", "op1", "node", "dom", "Op One");
Assert.Equal(0, rc);
Assert.Equal(("Tank01.Level.HiHi.ackmsg", (object?)"ack from HMI"), src.LastWrite);
}
}
```
**Step 2: Implement `SubtagAlarmConsumer : IMxAccessAlarmConsumer`.**
- Constructor `(ISubtagAlarmSource source, IReadOnlyList<AlarmSubtagTarget> watchList)`; build a `SubtagAlarmStateMachine`; index `alarm_full_reference`→target for ack routing.
- `Subscribe(_)`: call `source.Advise(<all subtag addresses>)`; subscribe to `source.ValueChanged`, feed each into the state machine, and re-raise each produced `MxAlarmTransitionEvent` via `AlarmTransitionEmitted` (mark degraded).
- `AcknowledgeByName(alarmName, …, comment, …)`: resolve the target by reference; if no `AckCommentSubtag`, return a non-zero failure code; else `source.Write(target.AckCommentSubtag, comment)` and return 0.
- `AcknowledgeByGuid(guid, …)`: map the synthetic GUID (deterministic hash of reference — see Task 8 helper, or a local copy) back to a reference, then delegate to the name path; unknown GUID ⇒ non-zero.
- `SnapshotActiveAlarms()`: from the state machine.
- `PollOnce()`: no-op.
- `Dispose()`: unsubscribe + dispose source.
**Step 3: Run green.** `dotnet test ...Worker.Tests... -p:Platform=x86 --filter FullyQualifiedName~SubtagAlarmConsumerTests`.
**Step 4: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Worker/MxAccess/SubtagAlarmConsumer.cs \
src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/SubtagAlarmConsumerTests.cs
git commit -m "worker(alarms): SubtagAlarmConsumer synthesizing transitions over the source seam"
```
---
### Task 6: COM-backed `LmxSubtagAlarmSource` (own LMXProxyServerClass)
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none
The only piece that touches live COM. Like `WnWrapAlarmConsumer`, it owns its own MXAccess server object so the subtag source is self-contained and isolated from the session's item pipeline. Logic stays thin (advise/write/marshal); real verification is the live smoke test in Task 17.
**Files:**
- Create: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/LmxSubtagAlarmSource.cs`
- Test: `src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/LmxSubtagAlarmSourceTests.cs` (constructor/guard tests only; COM path is live-gated)
**Step 1: Implement `LmxSubtagAlarmSource : ISubtagAlarmSource`.**
- Own an `LMXProxyServerClass` (reuse the worker's `IMxAccessServer`/`MxAccessComServer` wrapper + `IMxAccessComObjectFactory` so it is fakeable; constructor takes the factory).
- `Advise(addresses)`: `RegisterServer` (topic) once; per address `AddItem`→`itemHandle`, `Advise`, and record `itemHandle→address`. Subscribe to the proxy's `OnDataChange`; in the handler, look up the address by `phItemHandle`, normalize `pvItemValue` (VARIANT→bool/double) and `pftItemTimeStamp`→UTC, and raise `ValueChanged`. All calls run on the STA (the worker STA pumps messages, so `OnDataChange` delivers).
- `Write(address, value)`: resolve/create the item handle, `server.Write(serverHandle, itemHandle, value, userId: 0)`.
- `Dispose()`: `UnAdvise`/`RemoveItem`/`Unregister`/release COM.
**Step 2: Tests** — only the non-COM guards (null factory throws; `Write` before `Advise` resolves a handle or throws a clear error). Mark the COM round-trip `[LiveMxAccessFact]` and `Skip` per the `AlarmsLiveSmokeTests` precedent.
**Step 3: Build x86 + run unit tests.**
`dotnet build src/ZB.MOM.WW.MxGateway.Worker/ZB.MOM.WW.MxGateway.Worker.csproj -p:Platform=x86`
`dotnet test ...Worker.Tests... -p:Platform=x86 --filter FullyQualifiedName~LmxSubtagAlarmSourceTests`
**Step 4: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Worker/MxAccess/LmxSubtagAlarmSource.cs \
src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/LmxSubtagAlarmSourceTests.cs
git commit -m "worker(alarms): COM-backed LmxSubtagAlarmSource advising alarm subtags"
```
---
### Task 7: `FailoverAlarmConsumer` state machine
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none (depends on Task 5)
**Files:**
- Create: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/FailoverAlarmConsumer.cs`
- Create: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/AlarmProviderModeChange.cs` (small EventArgs)
- Test: `src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/FailoverAlarmConsumerTests.cs`
**Step 1: Test the switch/failback with a fake primary that throws.**
```csharp
public sealed class FailoverAlarmConsumerTests
{
private sealed class FlakyPrimary : IMxAccessAlarmConsumer
{
public event EventHandler<MxAlarmTransitionEvent>? AlarmTransitionEmitted;
public int PollsUntilHeal = int.MaxValue; // becomes healthy after N polls while degraded
public bool ThrowOnPoll = true;
private int _polls;
public void Subscribe(string s) { if (ThrowOnPoll) throw new COMException("boom", unchecked((int)0x80004005)); }
public void PollOnce()
{
_polls++;
if (ThrowOnPoll && _polls < PollsUntilHeal) throw new COMException("boom", unchecked((int)0x80004005));
}
public int AcknowledgeByGuid(Guid g, string c, string a, string b, string d, string e) => 0;
public int AcknowledgeByName(string n, string p, string gr, string c, string a, string b, string d, string e) => 0;
public IReadOnlyList<MxAlarmSnapshotRecord> SnapshotActiveAlarms() => Array.Empty<MxAlarmSnapshotRecord>();
public void Dispose() { }
}
private sealed class StubStandby : IMxAccessAlarmConsumer { /* records Subscribe, no-op rest */ }
[Fact]
public void Primary_FailsThresholdTimes_SwitchesToSubtagAndEmitsModeChange()
{
var primary = new FlakyPrimary();
var standby = new StubStandby();
using var c = new FailoverAlarmConsumer(primary, standby,
new FailoverSettings(threshold: 3, probeIntervalSeconds: 30, stableProbes: 3));
AlarmProviderModeChange? change = null;
c.ProviderModeChanged += (_, e) => change = e;
c.Subscribe("\\\\host\\Galaxy!Area"); // primary.Subscribe throws -> counts as failure 1
c.PollOnce(); // failure 2
c.PollOnce(); // failure 3 -> switch
Assert.NotNull(change);
Assert.Equal(AlarmProviderMode.Subtag, change!.Mode);
}
[Fact]
public void WhileDegraded_PrimaryHeals_FailsBackAfterStableProbes()
{
var primary = new FlakyPrimary { PollsUntilHeal = 0 }; // will heal once we stop throwing
var standby = new StubStandby();
using var c = new FailoverAlarmConsumer(primary, standby,
new FailoverSettings(threshold: 1, probeIntervalSeconds: 0, stableProbes: 2));
var modes = new List<AlarmProviderMode>();
c.ProviderModeChanged += (_, e) => modes.Add(e.Mode);
c.Subscribe("x"); // failure -> switch to subtag
primary.ThrowOnPoll = false;
c.ProbeOnce(); // clean probe 1
c.ProbeOnce(); // clean probe 2 -> failback
Assert.Equal(AlarmProviderMode.Subtag, modes[0]);
Assert.Equal(AlarmProviderMode.Alarmmgr, modes[^1]);
}
}
```
**Step 2: Implement.**
- `record FailoverSettings(int threshold, int probeIntervalSeconds, int stableProbes)`; `AlarmProviderModeChange : EventArgs { AlarmProviderMode Mode; string Reason; int HResult; DateTime AtUtc; }`.
- Constructor `(IMxAccessAlarmConsumer primary, IMxAccessAlarmConsumer standby, FailoverSettings settings)`; forced-mode variants handled in Task 9 wiring (forced ⇒ skip the other consumer).
- Forward `AlarmTransitionEmitted` from the **active** child only (swap the subscription on switch).
- Wrap `Subscribe`/`PollOnce` on the primary: on `COMException` (or a failure HRESULT) while `PrimaryActive`, increment a counter; at `threshold`, ensure standby `Subscribe`d, set active=standby, snapshot standby for hand-off, raise `ProviderModeChanged(Subtag, reason, hresult, now)`. Reset counter on any clean primary poll.
- `ProbeOnce()` (driven by the poll loop while degraded, gated by `probeIntervalSeconds`): try primary `Subscribe`+`PollOnce`; count consecutive clean probes; at `stableProbes`, set active=primary, return standby to standby, raise `ProviderModeChanged(Alarmmgr, "recovered", 0, now)`.
- `Acknowledge*` / `SnapshotActiveAlarms` delegate to the **active** child.
- `PollOnce()` drives the active child's poll, and—while degraded—also drives the failback probe cadence.
**Step 3: Run green** (x86 filter `FailoverAlarmConsumerTests`).
**Step 4: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Worker/MxAccess/FailoverAlarmConsumer.cs \
src/ZB.MOM.WW.MxGateway.Worker/MxAccess/AlarmProviderModeChange.cs \
src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/FailoverAlarmConsumerTests.cs
git commit -m "worker(alarms): FailoverAlarmConsumer auto-failover/failback state machine"
```
---
### Task 8: Synthetic-GUID helper + degraded flag on the event sink path
**Classification:** standard
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 9
Carry `degraded` + `source_provider` from the worker synthesis into the emitted `OnAlarmTransitionEvent`.
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAlarmSnapshot.cs` (add `bool Degraded`)
- Modify: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessAlarmEventSink.cs` (`EnqueueTransition` carries degraded)
- Modify: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessEventMapper.cs` (`CreateOnAlarmTransition` sets `Degraded`/`SourceProvider`)
- Create: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/SyntheticAlarmGuid.cs`
- Test: add cases to `src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/AlarmDispatcherTests.cs` and a new `SyntheticAlarmGuidTests.cs`
**Step 1: `SyntheticAlarmGuid.ForReference(string reference)`** — deterministic GUID from a stable hash (e.g. MD5 of the UTF-8 reference → `new Guid(bytes)`), so subtag-mode acks resolve by GUID. Test determinism + difference:
```csharp
[Fact] public void SameReference_SameGuid() =>
Assert.Equal(SyntheticAlarmGuid.ForReference("A.B.C"), SyntheticAlarmGuid.ForReference("A.B.C"));
[Fact] public void DifferentReference_DifferentGuid() =>
Assert.NotEqual(SyntheticAlarmGuid.ForReference("A.B.C"), SyntheticAlarmGuid.ForReference("A.B.D"));
```
**Step 2: Thread `degraded`** through `MxAlarmSnapshotRecord.Degraded`, `EnqueueTransition(... bool degraded)`, and `CreateOnAlarmTransition(... bool degraded, AlarmProviderMode sourceProvider)`. Default `degraded=false`, `sourceProvider=Alarmmgr` so the wnwrap path is unchanged (regression: existing `AlarmDispatcherTests` still pass with `Degraded=false`).
**Step 3: Tests** — extend `AlarmDispatcherTests` with a subtag-style transition asserting `body.Degraded == true` and `SourceProvider == Subtag`.
**Step 4: Build x86 + run** worker tests for `AlarmDispatcherTests`, `SyntheticAlarmGuidTests`.
**Step 5: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAlarmSnapshot.cs \
src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessAlarmEventSink.cs \
src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessEventMapper.cs \
src/ZB.MOM.WW.MxGateway.Worker/MxAccess/SyntheticAlarmGuid.cs \
src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/
git commit -m "worker(alarms): synthetic GUID + degraded provenance on emitted transitions"
```
---
### Task 9: Wire watch-list + failover config through `AlarmCommandHandler`; emit mode-changed event
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none (depends on Tasks 5, 7, 8)
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/AlarmCommandHandler.cs`
- Modify: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/IAlarmCommandHandler.cs`
- Modify: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessCommandExecutor.cs` (`ExecuteSubscribeAlarms`, ~lines 588-616)
- Modify: `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessStaSession.cs` (consumer factory wiring; mode-change → event queue)
- Test: `src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/AlarmCommandHandlerTests.cs` (extend or create)
**Step 1: Carry the subscribe payload.** Change the alarm subscribe entry point from `Subscribe(string subscription)` to `Subscribe(SubscribeAlarmsCommand command)` (the command now has `ForcedMode`, `WatchList`, `Failover`). In `AlarmCommandHandler.Subscribe`:
- Build the active provider per `ForcedMode`:
- `ALARMMGR` ⇒ `WnWrapAlarmConsumer` only.
- `SUBTAG` ⇒ `SubtagAlarmConsumer(new LmxSubtagAlarmSource(factory), watchList)` only.
- `UNSPECIFIED` ⇒ `FailoverAlarmConsumer(primary: wnwrap, standby: subtag, settings-from-Failover)`.
- Use the existing `consumerFactory` seam but widen it to `Func<SubscribeAlarmsCommand, IMxAccessAlarmConsumer>` so tests inject fakes and production builds the failover composite. Subscribe to `FailoverAlarmConsumer.ProviderModeChanged` and enqueue an `OnAlarmProviderModeChangedEvent` MxEvent via the event queue (new mapper method `CreateOnAlarmProviderModeChanged`).
**Step 2: Executor + STA wiring.** `ExecuteSubscribeAlarms` passes the full `SubscribeAlarmsCommand` (not just the expression). In `MxAccessStaSession`, the `alarmCommandHandlerFactory` must give the handler access to the `IMxAccessComObjectFactory` so the subtag source can create its own proxy server on the STA; keep the `EnsureOnAlarmConsumerThread` affinity guard on every path.
**Step 3: Test** — fake consumer factory; assert that a `SUBTAG` forced command builds the subtag consumer and advises; that an auto command building a fake failover composite, when it raises `ProviderModeChanged`, enqueues an `OnAlarmProviderModeChangedEvent` on the queue.
**Step 4: Build x86 + worker tests.**
**Step 5: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Worker/MxAccess/ src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/
git commit -m "worker(alarms): route watch-list/failover config; emit provider-mode-changed event"
```
---
## Phase 2 — Gateway: discovery, options, monitor, metrics, dashboard
### Task 10: `AlarmsOptions.Fallback` + validation
**Classification:** standard
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 11, Task 13
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Configuration/AlarmsOptions.cs`
- Create: `src/ZB.MOM.WW.MxGateway.Server/Configuration/AlarmFallbackOptions.cs`
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Configuration/GatewayOptionsValidator.cs` (`ValidateAlarms`, ~lines 234-258)
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Configuration/GatewayOptionsValidatorTests.cs` (extend)
**Step 1:** Add `AlarmFallbackOptions Fallback { get; init; } = new();` to `AlarmsOptions`. `AlarmFallbackOptions`: `string Mode = "Auto"` (`Auto|ForceAlarmManager|ForceSubtag`), `int ConsecutiveFailureThreshold = 3`, `int FailbackProbeIntervalSeconds = 30`, `int FailbackStableProbes = 3`, a `Discovery` sub-object (`bool UseGalaxyRepository = true`, `string Area = ""`, `string[] IncludeAttributes = []`, `string[] ExcludeAttributes = []`), and a `Subtags` sub-object (`Active="active"`, `Acked="acked"`, `AckComment=""`, `Priority="priority"`).
**Step 2:** In `ValidateAlarms`, when `Enabled` and `Mode == "ForceSubtag"` and `Discovery.UseGalaxyRepository == false` and `IncludeAttributes` empty ⇒ add a validation error ("ForceSubtag requires Galaxy Repository discovery or an explicit IncludeAttributes list"). Floor the three numeric values at 1. Validate `Mode` is one of the three literals.
**Step 3-5:** Test the new validation cases (red→green), build the server, commit.
---
### Task 11: Galaxy Repository "alarm attributes" discovery query
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 10, Task 13
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Galaxy/GalaxyRepository.cs` (add `GetAlarmAttributesAsync` + SQL constant, following `GetAttributesAsync` ~lines 86-115 and `AttributesSql` ~line 176)
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Galaxy/IGalaxyRepository.cs`
- Create: `src/ZB.MOM.WW.MxGateway.Server/Galaxy/GalaxyAlarmAttributeRow.cs`
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Galaxy/` (projection unit test; live SQL gated)
**Step 1:** `GalaxyAlarmAttributeRow { string FullTagReference; string SourceObjectReference; string AckCommentSubtag; }` (and any priority subtag). `GetAlarmAttributesAsync` reuses the existing `is_alarm` detection (the `AlarmExtension` primitive join already in `AttributesSql`) filtered to `is_alarm = 1`, projecting the alarm reference + its ack-comment attribute. Follow the exact `SqlConnection`/`SqlCommand`/`SqlDataReader` pattern from `GetAttributesAsync`.
**Step 2:** Unit-test the row→`AlarmSubtagTarget` mapping (a pure mapper function); gate any live-DB test like the existing Galaxy live tests (or `Skip` with a note, matching `AlarmsLiveSmokeTests`).
**Step 3-5:** red→green, build server, commit.
---
### Task 12: Watch-list resolver (GR SQL + config override → `AlarmSubtagTarget[]`)
**Classification:** standard
**Estimated implement time:** ~4 min
**Parallelizable with:** none (depends on Tasks 10, 11)
**Files:**
- Create: `src/ZB.MOM.WW.MxGateway.Server/Alarms/AlarmWatchListResolver.cs`
- Create: `src/ZB.MOM.WW.MxGateway.Server/Alarms/IAlarmWatchListResolver.cs`
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Alarms/AlarmWatchListResolverTests.cs`
**Step 1: Test the merge** with a fake `IGalaxyRepository`:
- discovery rows + `IncludeAttributes` are unioned; `ExcludeAttributes` removed; each becomes an `AlarmSubtagTarget` with `.active`/`.acked`/`.ackmsg` addresses composed from the configured `Subtags` names (`<reference>.<Active>`, etc.); empty config subtag names fall back to defaults; GR unavailable + no includes ⇒ empty list + a logged warning flag.
**Step 2: Implement** `ResolveAsync(AlarmsOptions, CancellationToken) → IReadOnlyList<AlarmSubtagTarget>`.
**Step 3-5:** red→green, build, commit.
---
### Task 13: Gateway metrics — provider-mode gauge + switch counter
**Classification:** small
**Estimated implement time:** ~3 min
**Parallelizable with:** Task 10, Task 11
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Metrics/GatewayMetrics.cs` (ctor ~lines 55-79; add counter + observable gauge following the existing pattern)
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Metrics/GatewayMetricsTests.cs` (if present; else assert via a `MeterListener`)
**Step 1:** Add `mxgateway.alarms.provider_switches` counter (tagged `from`,`to`,`reason`) and `mxgateway.alarms.provider_mode` observable gauge (1=alarmmgr, 2=subtag), plus `AlarmProviderSwitched(int from, int to, string reason)` and a private `GetAlarmProviderMode()` (lock on `_syncRoot` like the others).
**Step 2-4:** test, build, commit.
---
### Task 14: `GatewayAlarmMonitor` — arm watch-list, reflect provider mode, reconcile on switch
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none (depends on Tasks 9, 12, 13)
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Alarms/GatewayAlarmMonitor.cs` (ctor ~41-49; `SubscribeAlarmsAsync` ~210-233; event-drain loop; `StreamAsync` ~386-434)
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Alarms/GatewayAlarmMonitorProviderModeTests.cs` (new, using `FakeWorkerHarness`)
**Step 1:** Inject `IAlarmWatchListResolver` and `GatewayMetrics`. In `SubscribeAlarmsAsync`, resolve the watch-list and build the `SubscribeAlarmsCommand` with `ForcedMode` (from `Fallback.Mode`), `WatchList`, and `Failover` populated from options — instead of the bare `{ SubscriptionExpression }`.
**Step 2:** In the worker-event drain path, handle `OnAlarmProviderModeChangedEvent`: update a `_providerStatus` field (mode/degraded/reason/since), `Broadcast(new AlarmFeedMessage { ProviderStatus = … })` to every subscriber, call `metrics.AlarmProviderSwitched(...)`, and force a `ReconcileAsync` so the cache re-seeds from the now-active provider (avoids raise/clear storms).
**Step 3:** In `StreamAsync`, emit the current `provider_status` as the **first** message (before the snapshot) so a late joiner immediately knows the mode.
**Step 4: Test** — stand up the monitor with `FakeWorkerHarness`; emit an `OnAlarmProviderModeChangedEvent(Subtag)`; assert a `StreamAsync` subscriber receives a `ProviderStatus{ Mode=Subtag, Degraded=true }` and that the switch counter incremented. Also assert a transition emitted in subtag mode flows through with `Degraded=true`.
**Step 5:** build server, run the new test, commit.
---
### Task 15: Dashboard — push provider status to `/hubs/alarms` + UI indicator
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none (depends on Task 14)
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Dashboard/Hubs/AlarmsHubPublisher.cs` (forward `ProviderStatus` messages — they already flow through `StreamAsync`, so confirm the existing `SendAsync(AlarmMessage, message)` carries them; add a dedicated `"ProviderModeChanged"` client method if the dashboard needs a distinct channel)
- Modify: the alarms dashboard page/component (Bootstrap-only badge: green "alarmmgr" / amber "degraded — subtag") — find under `src/ZB.MOM.WW.MxGateway.Server/Dashboard/`
- Test: `src/ZB.MOM.WW.MxGateway.Tests/` dashboard model test (e.g. a `DashboardAlarmProviderStatus.FromFeed` mapper, mirroring `DashboardActiveAlarm.FromSnapshot`)
**Constraint:** Bootstrap CSS/JS only — no MudBlazor/Radzen/FluentUI.
**Steps:** TDD the model mapper, wire the publisher + badge, build, commit.
---
## Phase 3 — Integration, docs, live smoke
### Task 16: End-to-end fake-worker failover test
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 18
**Files:**
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Alarms/AlarmFailoverEndToEndTests.cs`
Drive the full gateway path with `FakeWorkerHarness`: subscribe (assert the `SubscribeAlarmsCommand` carries a watch-list), emit a wnwrap-style transition (assert `Degraded=false`), emit `OnAlarmProviderModeChangedEvent(Subtag)`, emit a synthesized transition (assert `Degraded=true`, `SourceProvider=Subtag`), then `OnAlarmProviderModeChangedEvent(Alarmmgr)` and assert the feed reports recovery. Build, run, commit.
---
### Task 17: Live subtag smoke test (opt-in)
**Classification:** small
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 18
**Files:**
- Test: `src/ZB.MOM.WW.MxGateway.IntegrationTests/...AlarmSubtagLiveSmokeTests.cs` (or the worker live suite)
A `[LiveMxAccessFact]`, `Skip`-by-default test (per `AlarmsLiveSmokeTests` precedent) that, against a live Galaxy + alarm flip script: advises the real `.active`/`.acked` subtags via `LmxSubtagAlarmSource`, asserts a synthesized raise/clear, and performs an ack via the ack-comment write. Document the exact subtag names discovered (resolves the design's open item). Commit.
---
### Task 18: Documentation
**Classification:** trivial
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 16, Task 17
**Files:**
- Modify: `gateway.md` (alarm provider section: dual provider + auto-failover/failback)
- Modify: `docs/DesignDecisions.md` (record the fallback decision + parity rationale)
- Modify: `docs/GatewayConfiguration.md` (the `MxGateway:Alarms:Fallback` block)
- Modify: `docs/AlarmClientDiscovery.md` (subtag provider, synthesis rules, ack-comment write)
- Modify: `docs/Grpc.md` (new `provider_status` feed case + `degraded`/`source_provider` fields)
Follow `StyleGuide.md` (PascalCase filenames, present tense, explain *why*). No code; commit.
---
## Execution order & parallelism summary
- **Serial spine:** 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8/9 → 10/11 → 12 → 13 → 14 → 15 → 16 → 17/18.
- **Parallelizable clusters:** {8, 9 partially}, {10, 11, 13}, {16, 17, 18}.
- **High-risk tasks** (full review chain): 1, 2, 6, 7, 9, 14. **Standard:** 4, 5, 8, 10, 11, 12, 15, 16. **Small/trivial:** 3, 13, 17, 18.
## Risk notes for the executor
- **Field-number collisions:** Task 2 must read the live `MxEvent`/`MxEventFamily` numbers before adding — the agent map gave alarm-payload maxima but not `MxEvent`'s. Verify before editing.
- **STA discipline:** every COM call in `LmxSubtagAlarmSource` and every consumer swap runs on the worker STA; keep the `EnsureOnAlarmConsumerThread` guard. The worker STA already pumps Windows messages, which is required for the subtag `OnDataChange` to deliver.
- **Parity regression:** alarmmgr-mode output must be byte-for-byte unchanged. Existing `AlarmDispatcherTests` and `ProtobufContractRoundTripTests` are the guardrail — they must stay green with `Degraded=false` defaults.
- **Subtag names unverified:** the design leaves exact AVEVA subtag names (`.active`, `.acked`, ack-comment) to confirm against `C:\Users\dohertj2\Desktop\mxaccess` + a live Galaxy (Task 17). The config `Subtags` block exists so names are not hard-coded.
@@ -0,0 +1,147 @@
{
"planPath": "docs/plans/2026-06-13-alarm-subtag-fallback.md",
"tasks": [
{
"id": 54,
"subject": "Task 1: Worker proto \u2014 watch-list, failover config, AlarmProviderMode",
"status": "completed"
},
{
"id": 55,
"subject": "Task 2: Gateway proto \u2014 provider status, degraded provenance, mode-changed event",
"status": "completed",
"blockedBy": [
54
]
},
{
"id": 56,
"subject": "Task 3: Proto round-trip tests for new alarm fields",
"status": "completed",
"blockedBy": [
54,
55
]
},
{
"id": 57,
"subject": "Task 4: Subtag value-source abstraction + synthesis state machine",
"status": "completed",
"blockedBy": [
54
]
},
{
"id": 58,
"subject": "Task 5: SubtagAlarmConsumer over the source seam",
"status": "completed",
"blockedBy": [
57
]
},
{
"id": 59,
"subject": "Task 6: COM-backed LmxSubtagAlarmSource",
"status": "completed",
"blockedBy": [
57
]
},
{
"id": 60,
"subject": "Task 7: FailoverAlarmConsumer state machine",
"status": "completed",
"blockedBy": [
58
]
},
{
"id": 61,
"subject": "Task 8: Synthetic GUID + degraded flag on event sink path",
"status": "completed",
"blockedBy": [
55
]
},
{
"id": 62,
"subject": "Task 9: Wire watch-list/failover through AlarmCommandHandler; emit mode-changed",
"status": "completed",
"blockedBy": [
58,
60,
61
]
},
{
"id": 63,
"subject": "Task 10: AlarmsOptions.Fallback + validation",
"status": "completed"
},
{
"id": 64,
"subject": "Task 11: Galaxy Repository alarm-attributes discovery query",
"status": "completed"
},
{
"id": 65,
"subject": "Task 12: Watch-list resolver (GR SQL + config override)",
"status": "completed",
"blockedBy": [
54,
63,
64
]
},
{
"id": 66,
"subject": "Task 13: Metrics \u2014 provider-mode gauge + switch counter",
"status": "completed"
},
{
"id": 67,
"subject": "Task 14: GatewayAlarmMonitor \u2014 arm watch-list, reflect mode, reconcile on switch",
"status": "completed",
"blockedBy": [
55,
62,
65,
66
]
},
{
"id": 68,
"subject": "Task 15: Dashboard \u2014 push provider status + UI badge",
"status": "completed",
"blockedBy": [
67
]
},
{
"id": 69,
"subject": "Task 16: End-to-end fake-worker failover test",
"status": "completed",
"blockedBy": [
67
]
},
{
"id": 70,
"subject": "Task 17: Live subtag smoke test (opt-in)",
"status": "completed",
"blockedBy": [
59,
62
]
},
{
"id": 71,
"subject": "Task 18: Documentation",
"status": "completed",
"blockedBy": [
67
]
}
],
"lastUpdated": "2026-06-13T13:30:00Z"
}
+57
View File
@@ -143,6 +143,63 @@ session if the worker faults. Gated by `MxGateway:Alarms:Enabled` — see
`docs/DesignDecisions.md` for why this reverses the v1 single-subscriber rule
for the alarm subsystem.
### Alarm providers and failover
The alarm feed has two providers, both implemented worker-side:
- **Alarm manager (primary):** `WnWrapAlarmConsumer` polls
`wwAlarmConsumerClass.GetXmlCurrentAlarms2` on the worker STA. This is the
authoritative native source.
- **Subtag monitoring (standby):** `SubtagAlarmConsumer` advises each alarm
attribute's subtags (`.active`, `.acked`, optionally `.priority`) via the
existing `AddItem`/`Advise` pipeline through `LmxSubtagAlarmSource` and
synthesizes alarm transitions with `SubtagAlarmStateMachine`. This is a
non-parity, lower-fidelity source — synthetic GUIDs, no native raise
timestamps, narrower fields.
`FailoverAlarmConsumer` wraps both and owns the state machine:
- **Auto-failover:** after `ConsecutiveFailureThreshold` (default 3)
consecutive wnwrap COM failures — `Subscribe` or `PollOnce` throws or
returns a failure HRESULT — it activates the standby. The standby is armed
(subscribed and adviseing) from the start so its state is warm at the moment
of switch.
- **Auto-failback:** while degraded, every `FailbackProbeIntervalSeconds`
(default 30) it re-probes the still-subscribed primary. After
`FailbackStableProbes` (default 3) consecutive clean polls it switches back
to the alarm manager.
- **On every switch:** the consumer snapshots the now-active provider and
emits `OnAlarmProviderModeChangedEvent` so the gateway can reconcile its
cache without a raise/clear storm.
Synthesis is worker-side. This preserves the parity rule — the gateway
forwards only events the worker emits and never synthesizes transitions
itself. The synthesis rules are documented in
`docs/AlarmClientDiscovery.md`.
**Acknowledge in subtag mode:** the ack-by-name path writes the operator
comment to the alarm attribute's ack-comment subtag. The write performs the
ack. If the attribute has no writable ack-comment subtag configured, the RPC
returns `FailedPrecondition`. In alarm-manager mode, `AlarmAckByName` is
used as before.
**Degraded state visibility:** every subtag-mode transition carries
`degraded = true` and `source_provider = ALARM_PROVIDER_MODE_SUBTAG` on the
`OnAlarmTransitionEvent` and `ActiveAlarmSnapshot` proto fields. The
`AlarmFeedMessage` feed emits an `AlarmProviderStatus` message (the
`provider_status` oneof case) on stream open and on every switch. The
dashboard shows a Bootstrap badge (green for alarm manager, amber when
degraded). Metrics: `mxgateway.alarms.provider_mode` gauge (1 = alarmmgr,
2 = subtag) and `mxgateway.alarms.provider_switches` counter.
Forced modes are available via `MxGateway:Alarms:Fallback:Mode`:
`ForceAlarmManager` disables failover; `ForceSubtag` forces the standby
on from startup; `Auto` (default) enables failover and failback. Watch-list
discovery for the subtag provider uses Galaxy Repository SQL with config
overrides. See `docs/GatewayConfiguration.md` for the full `Fallback` option
block and `docs/AlarmClientDiscovery.md` for synthesis rules and fidelity
limitations.
Dashboard authentication is LDAP-backed (distinct from the API-key model on
the gRPC API). `/login` accepts username and password in a form body, binds
against `MxGateway:Ldap`, maps the user's LDAP groups to `Admin` or `Viewer`
+81 -40
View File
@@ -1,27 +1,36 @@
# GLAuth — LDAP authn reference for mxaccessgw
GLAuth is a lightweight LDAP server installed on this dev box at
`C:\publish\glauth\` and run as a Windows service via NSSM. It already
backs the LmxOpcUa OPC UA server's UserName-token authn and the LmxOpcUa
Admin UI's cookie login; this doc captures everything mxaccessgw needs
to consume the same directory so a single set of dev credentials covers
both stacks.
> **UPDATED 2026-06-04 — mxaccessgw no longer uses a per-box GLAuth at `C:\publish\glauth`.
> Dev/test LDAP is now the SHARED GLAuth on `10.100.0.35:3893` (`dc=zb,dc=local`);
> the single source of truth is `scadaproj/infra/glauth/` (`config.toml` + `README`).
> The localhost/NSSM/`glauth.cfg` procedures below are RETIRED, kept for reference/rollback.**
The authoritative copy of LmxOpcUa's reference lives at
`C:\publish\glauth\auth.md`. This doc is a redistilled view tailored to
mxaccessgw — what users + groups are already provisioned, how to bind
against them, and what's needed to add a gw-specific role.
GLAuth is a lightweight LDAP server. It already backs all three sister apps (MxAccessGateway,
OtOpcUa, ScadaBridge) through a **shared container** (`zb-shared-glauth`) running on the Linux
docker host at **`10.100.0.35:3893`**. This doc captures everything mxaccessgw needs to consume
that directory so a single set of dev credentials covers all stacks.
~~GLAuth is installed on this dev box at `C:\publish\glauth\` and run as a Windows service via
NSSM.~~ *(RETIRED — the per-box Windows service has been stopped and set to Manual startup;
kept only as a rollback option. Do not edit or restart it for new work.)*
The single source of truth for the shared GLAuth is
**`~/Desktop/scadaproj/infra/glauth/config.toml`** (deploy/verify runbook:
`scadaproj/infra/glauth/README.md`). This doc is a redistilled view tailored to mxaccessgw —
what users + groups are provisioned, how to bind against them, and what's needed to add a
gw-specific role.
## Connection details
| Setting | Value |
|---|---|
| Protocol | LDAP (unencrypted) |
| Host | `localhost` |
| Host | **`10.100.0.35`** (shared docker host — ~~`localhost`~~ retired) |
| Port | `3893` |
| LDAPS | disabled in dev (set `[ldaps]` block to enable) |
| LDAPS | disabled in dev (`Transport=None`, `AllowInsecure=true`) |
| Base DN | `dc=zb,dc=local` |
| Bind DN format | `cn={username},dc=zb,dc=local` |
| Service account DN | `cn=serviceaccount,dc=zb,dc=local` / `serviceaccount123` |
| Group OU | `ou=<groupname>,ou=groups,dc=zb,dc=local` |
| Failed-bind throttle | 3 fails → 10-minute IP lockout (per `[behaviors]`) |
@@ -59,13 +68,13 @@ For mxaccessgw dev, `admin` covers every gw-side capability test;
`readonly` is the right "negative" case for proving Browse-OK /
Write-denied.
The gateway dashboard adds one role beyond this LmxOpcUa taxonomy:
`GwAdmin`. `LdapOptions.RequiredGroup` defaults to `GwAdmin`, so the
dashboard login and `DashboardLdapLiveTests` require `admin` to be a
member of a `GwAdmin` group. `GwAdmin` is **not** in the baseline
GLAuth config — it must be provisioned before dashboard authn or the
LDAP live tests work. See [Provisioning the GwAdmin
group](#provisioning-the-gwadmin-group) below.
The gateway dashboard uses two gateway-specific groups beyond the LmxOpcUa taxonomy:
`GwAdmin` (gid 5610 → role `Administrator`) and `GwReader` (gid 5611 → role `Viewer`).
These are already provisioned in the shared `scadaproj/infra/glauth/config.toml`.
The dashboard test users are **`multi-role`/`password`** (Administrator) and
**`gw-viewer`/`password`** (Viewer). `LdapOptions.RequiredGroup` defaults to `GwAdmin`.
See [Provisioning the GwAdmin group](#provisioning-the-gwadmin-group) below for the
(now-retired) per-box procedure and for the shared-config equivalent.
> **Dashboard role value (Task 1.7):** the LDAP `GwAdmin` group now maps to
> the canonical dashboard role **`Administrator`** (was `Admin`); `GwReader`
@@ -118,7 +127,7 @@ record:
```yaml
ldap:
enabled: true
server: localhost
server: 10.100.0.35 # shared GLAuth on docker host (was localhost)
port: 3893
useTls: false
allowInsecureLdap: true # dev only
@@ -143,13 +152,29 @@ look that up in `groupToRole`.
## Provisioning the GwAdmin group
> **UPDATED 2026-06-04 — RETIRED per-box procedure.** `GwAdmin` (gid 5610) and `GwReader`
> (gid 5611) are already present in the shared GLAuth. To add or modify users/groups,
> edit **`~/Desktop/scadaproj/infra/glauth/config.toml`** on host `10.100.0.35` and run:
>
> ```bash
> cd ~/Desktop/scadaproj/infra/glauth
> docker compose up -d --force-recreate
> ```
>
> The per-box `C:\publish\glauth\glauth.cfg` + NSSM procedure below is kept for
> rollback reference only — do not use it for new provisioning.
`GwAdmin` is the gateway-specific dashboard-admin role. It is the
default `LdapOptions.RequiredGroup`, so the dashboard cookie login and
`DashboardLdapLiveTests` (`MXGATEWAY_RUN_LIVE_LDAP_TESTS=1`) reject
`admin` until a `GwAdmin` group exists and `admin` is a member.
GLAuth's baseline config ships only the five LmxOpcUa role groups, so
`GwAdmin` must be added to GLAuth rather than run from a separate LDAP
server:
logins unless the user is a member of `GwAdmin`.
The `GwAdmin` (gid 5610) and `GwReader` (gid 5611) groups already exist in the shared
config at `scadaproj/infra/glauth/config.toml`. Dashboard test users are
`multi-role`/`password` (Administrator) and `gw-viewer`/`password` (Viewer).
---
**RETIRED — per-box provisioning (reference/rollback only):**
1. Edit `C:\publish\glauth\glauth.cfg`
2. Append the group:
@@ -199,15 +224,16 @@ echo -n "yourpassword" | openssl dgst -sha256
## Quick verification
From mxaccessgw's dev box, prove the directory is reachable:
From mxaccessgw's dev box, prove the shared directory is reachable:
```powershell
# Plain bind via PowerShell + System.DirectoryServices.Protocols
$ldap = New-Object System.DirectoryServices.Protocols.LdapConnection("localhost:3893")
# (shared GLAuth on 10.100.0.35 — was localhost, now the docker host)
$ldap = New-Object System.DirectoryServices.Protocols.LdapConnection("10.100.0.35:3893")
$ldap.AuthType = [System.DirectoryServices.Protocols.AuthType]::Basic
$ldap.SessionOptions.ProtocolVersion = 3
$ldap.SessionOptions.SecureSocketLayer = $false
$cred = New-Object System.Net.NetworkCredential("cn=admin,dc=zb,dc=local","admin123")
$cred = New-Object System.Net.NetworkCredential("cn=multi-role,dc=zb,dc=local","password")
$ldap.Bind($cred)
"Bind OK"
```
@@ -215,17 +241,32 @@ $ldap.Bind($cred)
Or via `ldapsearch` if you have OpenLDAP CLI tools:
```bash
ldapsearch -x -H ldap://localhost:3893 \
-D "cn=admin,dc=zb,dc=local" -w admin123 \
-b "dc=zb,dc=local" "(uid=admin)"
ldapsearch -x -H ldap://10.100.0.35:3893 \
-D "cn=serviceaccount,dc=zb,dc=local" -w serviceaccount123 \
-b "dc=zb,dc=local" "(uid=multi-role)"
```
The response should list `admin`'s entry with `memberOf` populated for
all five role groups — plus `GwAdmin` once the gateway-specific group
is provisioned.
The response should list `multi-role`'s entry with `memberOf` including
`ou=GwAdmin,ou=groups,dc=zb,dc=local`.
## Service management
> **RETIRED — per-box NSSM service (reference/rollback only).** The shared GLAuth is
> managed via `docker compose` on `10.100.0.35` (`scadaproj/infra/glauth/`). The
> Windows NSSM `GLAuth` service on the dev box has been stopped and set to
> `StartupType=Manual`; only restart it if you need to roll back to a local directory.
>
> **Active (shared) management:**
> ```bash
> ssh 10.100.0.35
> cd ~/Desktop/scadaproj/infra/glauth
> docker compose ps # check container status
> docker compose up -d --force-recreate # apply config.toml changes
> docker compose logs -f # tail logs
> ```
**RETIRED — per-box NSSM commands (rollback reference):**
```powershell
# Status / start / stop / restart
nssm status GLAuth
@@ -259,7 +300,7 @@ applies to mxaccessgw verbatim. Keys that change:
| Field | GLAuth dev value | AD production value |
|---|---|---|
| `Server` | `localhost` | a domain controller FQDN, or the domain itself |
| `Server` | `10.100.0.35` (shared docker host) | a domain controller FQDN, or the domain itself |
| `Port` | `3893` | `636` (LDAPS) — AD increasingly rejects plain bind under LDAP-signing enforcement |
| `UseTls` | `false` | `true` |
| `AllowInsecureLdap` | `true` | `false` |
@@ -275,12 +316,12 @@ add a `tokenGroups` query as an enhancement.
## Security notes for production
- **Plaintext passwords in `glauth.cfg` are dev-only.** The config is
unencrypted on disk; anyone with read access to `C:\publish\glauth\`
can SHA256-rainbow-table the entries. Treat the dev creds as
throwaway. Production LDAP is Active Directory.
- **Plaintext passwords in `config.toml` are dev-only.** The shared config is in
`scadaproj/infra/glauth/config.toml` (unencrypted); restrict filesystem access on
`10.100.0.35` accordingly. Treat the dev creds as throwaway. Production LDAP is Active
Directory. *(The retired per-box `C:\publish\glauth\glauth.cfg` has the same caveat.)*
- The 3-fail / 10-minute lockout is per source IP, not per user — a
shared NAT can lock out a whole office. Tunable in `[behaviors]`.
- LDAPS isn't enabled in dev; binding sends passwords cleartext on the
wire. Fine for `localhost`, never expose port 3893 off-box without
enabling TLS first.
wire. The shared GLAuth listens only on the LAN (`10.100.0.35`); never
expose port 3893 externally without enabling TLS first.
File diff suppressed because it is too large Load Diff
@@ -315,6 +315,14 @@ message SubscribeBulkCommand {
repeated string tag_addresses = 2;
}
// Provider selection / current provider for the alarm feed. UNSPECIFIED on a
// SubscribeAlarmsCommand means auto: alarmmgr primary with subtag fallback.
enum AlarmProviderMode {
ALARM_PROVIDER_MODE_UNSPECIFIED = 0;
ALARM_PROVIDER_MODE_ALARMMGR = 1;
ALARM_PROVIDER_MODE_SUBTAG = 2;
}
// Subscribe the worker's alarm consumer to an AVEVA alarm provider.
// Subscription expression follows the canonical
// `\\<machine>\Galaxy!<area>` format (literal "Galaxy" provider). The
@@ -323,6 +331,12 @@ message SubscribeBulkCommand {
// SubscribeAlarms to reconfigure).
message SubscribeAlarmsCommand {
string subscription_expression = 1;
// UNSPECIFIED = auto-failover/failback. ALARMMGR/SUBTAG force one provider.
AlarmProviderMode forced_mode = 2;
// Subtag watch-list resolved by the gateway (GR SQL + config). Empty in pure
// alarmmgr mode; in subtag mode it bounds what the consumer can observe.
repeated AlarmSubtagTarget watch_list = 3;
AlarmFailoverConfig failover = 4;
}
// Tear down the worker's alarm consumer. No-op if no subscription is
@@ -330,6 +344,23 @@ message SubscribeAlarmsCommand {
message UnsubscribeAlarmsCommand {
}
// One alarm attribute the subtag fallback consumer advises. Addresses are full
// MXAccess item references the worker passes straight to AddItem.
message AlarmSubtagTarget {
string alarm_full_reference = 1; // e.g. "Galaxy!Area.Tank01.Level.HiHi"
string source_object_reference = 2; // e.g. "Tank01"
string active_subtag = 3; // item address of the in-alarm boolean
string acked_subtag = 4; // item address of the acknowledged boolean
string ack_comment_subtag = 5; // writable ack-comment attribute (ack write target)
string priority_subtag = 6; // optional severity source; empty if absent
}
message AlarmFailoverConfig {
int32 consecutive_failure_threshold = 1; // wnwrap COM failures before switching (>=1)
int32 failback_probe_interval_seconds = 2; // probe cadence while degraded (>=1)
int32 failback_stable_probes = 3; // clean probes before switching back (>=1)
}
// Acknowledge a single alarm by its GUID. Operator identity fields are
// recorded atomically with the ack transition in the alarm-history log.
// The reply's hresult / native_status surfaces AVEVA's
@@ -684,6 +715,7 @@ message MxEvent {
OperationCompleteEvent operation_complete = 22;
OnBufferedDataChangeEvent on_buffered_data_change = 23;
OnAlarmTransitionEvent on_alarm_transition = 24;
OnAlarmProviderModeChangedEvent on_alarm_provider_mode_changed = 25;
}
}
@@ -694,6 +726,7 @@ enum MxEventFamily {
MX_EVENT_FAMILY_OPERATION_COMPLETE = 3;
MX_EVENT_FAMILY_ON_BUFFERED_DATA_CHANGE = 4;
MX_EVENT_FAMILY_ON_ALARM_TRANSITION = 5;
MX_EVENT_FAMILY_ON_ALARM_PROVIDER_MODE_CHANGED = 6;
}
message OnDataChangeEvent {
@@ -768,6 +801,20 @@ message OnAlarmTransitionEvent {
// Limit/threshold value that triggered the transition for limit alarms.
// Optional; populated for AnalogLimitAlarm-family transitions.
MxValue limit_value = 13;
// True when this transition came from the subtag-monitoring fallback rather
// than the native alarmmgr provider — synthesized from data changes, reduced
// fidelity (synthetic GUID, no native raise time).
bool degraded = 14;
// Which provider produced this transition.
AlarmProviderMode source_provider = 15;
}
message OnAlarmProviderModeChangedEvent {
AlarmProviderMode mode = 1;
string reason = 2;
int32 hresult = 3; // COM HRESULT that triggered failover; 0 on failback
google.protobuf.Timestamp at = 4;
}
enum AlarmTransitionKind {
@@ -800,6 +847,8 @@ message ActiveAlarmSnapshot {
string operator_comment = 11;
MxValue current_value = 12;
MxValue limit_value = 13;
bool degraded = 14;
AlarmProviderMode source_provider = 15;
}
enum AlarmConditionState {
@@ -866,9 +915,19 @@ message AlarmFeedMessage {
bool snapshot_complete = 2;
// A live alarm state change (raise / acknowledge / clear).
OnAlarmTransitionEvent transition = 3;
// Provider-mode status. Emitted once on stream open and again on every
// failover/failback so late joiners learn the current mode immediately.
AlarmProviderStatus provider_status = 4;
}
}
message AlarmProviderStatus {
AlarmProviderMode mode = 1;
bool degraded = 2; // true whenever mode == SUBTAG
string reason = 3; // human-readable switch reason
google.protobuf.Timestamp since = 4;
}
message MxStatusProxy {
// Mirrors the `success` member of the MXAccess MXSTATUS_PROXY struct
// (a 16-bit signed value in the COM struct, widened to int32 on the
@@ -0,0 +1,168 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Configuration;
using ZB.MOM.WW.MxGateway.Server.Galaxy;
namespace ZB.MOM.WW.MxGateway.Server.Alarms;
/// <summary>
/// Default <see cref="IAlarmWatchListResolver"/>. Merges Galaxy Repository
/// alarm-attribute discovery with the configured include/exclude overrides
/// and composes the per-attribute subtag item addresses from the configured
/// subtag names.
/// </summary>
// NOTE: The exact subtag names and the canonical AlarmFullReference shape
// ("Galaxy!{area}.{reference}") are validated against a live Galaxy in the
// Task 17 live smoke test. The config Subtags block exists precisely so these
// names are not hard-coded here.
public sealed class AlarmWatchListResolver : IAlarmWatchListResolver
{
private const string ProviderLiteral = "Galaxy";
private const string DefaultActiveSubtag = "active";
private const string DefaultAckedSubtag = "acked";
private readonly IGalaxyRepository _repository;
private readonly ILogger<AlarmWatchListResolver> _logger;
/// <summary>Initializes the watch-list resolver.</summary>
/// <param name="repository">Galaxy Repository used for alarm-attribute discovery.</param>
/// <param name="logger">Diagnostic logger.</param>
public AlarmWatchListResolver(
IGalaxyRepository repository,
ILogger<AlarmWatchListResolver> logger)
{
_repository = repository ?? throw new ArgumentNullException(nameof(repository));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
/// <inheritdoc />
public async Task<IReadOnlyList<AlarmSubtagTarget>> ResolveAsync(
AlarmsOptions options,
CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(options);
AlarmDiscoveryOptions discovery = options.Fallback.Discovery;
// 1. Build the ordered, de-duplicated attribute reference set.
// Each entry carries the reference plus the source-object reference.
List<(string Reference, string SourceObject)> ordered = [];
HashSet<string> seen = new(StringComparer.OrdinalIgnoreCase);
if (discovery.UseGalaxyRepository)
{
List<GalaxyAlarmAttributeRow> rows;
try
{
rows = await _repository.GetAlarmAttributesAsync(cancellationToken).ConfigureAwait(false);
}
catch (Exception ex)
{
// Discovery being unavailable must not crash the resolver: log and
// continue with an empty discovery set. The caller decides what to
// do with the (possibly config-only) result.
_logger.LogWarning(
ex,
"Galaxy Repository alarm-attribute discovery failed; continuing with configuration-only watch-list.");
rows = [];
}
foreach (GalaxyAlarmAttributeRow row in rows)
{
if (string.IsNullOrEmpty(row.FullTagReference) || !seen.Add(row.FullTagReference))
{
continue;
}
ordered.Add((row.FullTagReference, row.SourceObjectReference));
}
}
foreach (string include in discovery.IncludeAttributes)
{
if (string.IsNullOrEmpty(include) || !seen.Add(include))
{
continue;
}
ordered.Add((include, DeriveSourceObject(include)));
}
// Remove excluded references (case-insensitive), but only when GR discovery
// is active. ExcludeAttributes is documented as "Ignored when
// UseGalaxyRepository is false" (AlarmDiscoveryOptions.ExcludeAttributes).
// Whitespace-only entries are skipped, consistent with the include guard above.
if (discovery.UseGalaxyRepository)
{
HashSet<string> excluded = new(
discovery.ExcludeAttributes.Where(e => !string.IsNullOrWhiteSpace(e)),
StringComparer.OrdinalIgnoreCase);
if (excluded.Count > 0)
{
ordered.RemoveAll(e => excluded.Contains(e.Reference));
}
}
// 2. Resolve subtag names with safe fallbacks.
string active = string.IsNullOrEmpty(options.Fallback.Subtags.Active)
? DefaultActiveSubtag
: options.Fallback.Subtags.Active;
string acked = string.IsNullOrEmpty(options.Fallback.Subtags.Acked)
? DefaultAckedSubtag
: options.Fallback.Subtags.Acked;
string priority = options.Fallback.Subtags.Priority;
string ackComment = options.Fallback.Subtags.AckComment;
// 3. Resolve the area: discovery area, else the default area (may be empty).
string area = string.IsNullOrEmpty(discovery.Area) ? options.DefaultArea : discovery.Area;
// 4. Compose one target per reference.
List<AlarmSubtagTarget> targets = new(ordered.Count);
foreach ((string reference, string sourceObject) in ordered)
{
targets.Add(new AlarmSubtagTarget
{
AlarmFullReference = ComposeFullReference(area, reference),
SourceObjectReference = sourceObject,
ActiveSubtag = $"{reference}.{active}",
AckedSubtag = $"{reference}.{acked}",
PrioritySubtag = string.IsNullOrEmpty(priority) ? string.Empty : $"{reference}.{priority}",
AckCommentSubtag = string.IsNullOrEmpty(ackComment) ? string.Empty : $"{reference}.{ackComment}",
});
}
// 5. Report the resolved count; warn when subtag mode was expected to cover
// something (GR enabled, or explicit includes were configured) but resolved
// to nothing. Only emit the Debug line when there is at least one target,
// to avoid a confusing "0 target(s)" noise line.
if (targets.Count == 0 && (discovery.UseGalaxyRepository || discovery.IncludeAttributes.Length > 0))
{
_logger.LogWarning(
"Alarm subtag watch-list resolved to zero targets; subtag-polling fallback will cover no alarms.");
}
else if (targets.Count > 0)
{
_logger.LogDebug("Resolved alarm subtag watch-list with {TargetCount} target(s).", targets.Count);
}
return targets;
}
/// <summary>
/// Derives the source-object reference for a configuration entry: the
/// substring before the first '.', or the whole string when there is no dot.
/// </summary>
private static string DeriveSourceObject(string reference)
{
int dot = reference.IndexOf('.', StringComparison.Ordinal);
return dot < 0 ? reference : reference[..dot];
}
/// <summary>
/// Composes the canonical alarm full reference: <c>Galaxy!{area}.{reference}</c>
/// when an area is set, otherwise <c>Galaxy!{reference}</c>.
/// </summary>
private static string ComposeFullReference(string area, string reference) =>
string.IsNullOrEmpty(area)
? $"{ProviderLiteral}!{reference}"
: $"{ProviderLiteral}!{area}.{reference}";
}
@@ -13,6 +13,7 @@ public static class AlarmsServiceCollectionExtensions
/// <returns>The service collection for chaining.</returns>
public static IServiceCollection AddGatewayAlarms(this IServiceCollection services)
{
services.AddSingleton<IAlarmWatchListResolver, AlarmWatchListResolver>();
services.AddSingleton<GatewayAlarmMonitor>();
services.AddSingleton<IGatewayAlarmService>(provider => provider.GetRequiredService<GatewayAlarmMonitor>());
services.AddHostedService(provider => provider.GetRequiredService<GatewayAlarmMonitor>());
@@ -1,7 +1,9 @@
using System.Threading.Channels;
using Google.Protobuf.WellKnownTypes;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Configuration;
using ZB.MOM.WW.MxGateway.Server.Metrics;
using ZB.MOM.WW.MxGateway.Server.Sessions;
namespace ZB.MOM.WW.MxGateway.Server.Alarms;
@@ -23,6 +25,8 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
private static readonly TimeSpan StartupGrace = TimeSpan.FromSeconds(2);
private readonly ISessionManager _sessionManager;
private readonly IAlarmWatchListResolver _watchListResolver;
private readonly GatewayMetrics _metrics;
private readonly AlarmsOptions _options;
private readonly ILogger<GatewayAlarmMonitor> _logger;
@@ -30,20 +34,34 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
private readonly Dictionary<string, ActiveAlarmSnapshot> _alarms = new(StringComparer.Ordinal);
private readonly List<Subscriber> _subscribers = [];
// Current provider status (mode + degraded + reason + since), guarded by _sync.
// Initialized to the alarm-manager, not-degraded baseline so a late joiner sees
// a sensible status even before any OnAlarmProviderModeChanged event arrives.
private AlarmProviderMode _providerMode = AlarmProviderMode.Alarmmgr;
private bool _providerDegraded;
private string _providerReason = string.Empty;
private DateTimeOffset _providerSince = DateTimeOffset.UtcNow;
private volatile GatewayAlarmMonitorState _state = GatewayAlarmMonitorState.Disabled;
private volatile string? _lastError;
private GatewaySession? _session;
/// <summary>Initializes the gateway alarm monitor.</summary>
/// <param name="sessionManager">Gateway session manager.</param>
/// <param name="watchListResolver">Resolver for the subtag-fallback watch-list.</param>
/// <param name="metrics">Gateway metrics sink.</param>
/// <param name="options">Gateway options carrying the alarm configuration.</param>
/// <param name="logger">Diagnostic logger.</param>
public GatewayAlarmMonitor(
ISessionManager sessionManager,
IAlarmWatchListResolver watchListResolver,
GatewayMetrics metrics,
IOptions<GatewayOptions> options,
ILogger<GatewayAlarmMonitor> logger)
{
_sessionManager = sessionManager ?? throw new ArgumentNullException(nameof(sessionManager));
_watchListResolver = watchListResolver ?? throw new ArgumentNullException(nameof(watchListResolver));
_metrics = metrics ?? throw new ArgumentNullException(nameof(metrics));
_options = (options ?? throw new ArgumentNullException(nameof(options))).Value.Alarms;
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
@@ -139,6 +157,20 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
private async Task RunMonitorAsync(string subscription, CancellationToken stoppingToken)
{
_state = GatewayAlarmMonitorState.Starting;
lock (_sync)
{
// Re-baseline the provider status for this lifecycle so a restarted
// monitor advertises alarm-manager/not-degraded until told otherwise.
_providerMode = AlarmProviderMode.Alarmmgr;
_providerDegraded = false;
_providerReason = string.Empty;
_providerSince = DateTimeOffset.UtcNow;
}
// Align the observable gauge with the Alarmmgr baseline without recording
// a switch — the gauge was 0 (unknown) from construction until now.
_metrics.SetAlarmProviderMode(ModeToInt(AlarmProviderMode.Alarmmgr));
GatewaySession session = await _sessionManager.OpenSessionAsync(
new SessionOpenRequest(BackendName, MonitorClientName, Guid.NewGuid().ToString("N"), CommandTimeout: null),
MonitorClientName,
@@ -173,6 +205,15 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
{
ApplyTransition(mxEvent.OnAlarmTransition);
}
else if (mxEvent is { BodyCase: MxEvent.BodyOneofCase.OnAlarmProviderModeChanged }
&& mxEvent.OnAlarmProviderModeChanged is not null)
{
await ApplyProviderModeChangeAsync(
session.SessionId,
mxEvent.OnAlarmProviderModeChanged,
linked.Token)
.ConfigureAwait(false);
}
}
}
finally
@@ -209,6 +250,29 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
private async Task SubscribeAlarmsAsync(string sessionId, string subscription, CancellationToken cancellationToken)
{
IReadOnlyList<AlarmSubtagTarget> watchList = await _watchListResolver
.ResolveAsync(_options, cancellationToken)
.ConfigureAwait(false);
AlarmProviderMode forcedMode = MapForcedMode(_options.Fallback.Mode);
// When the forced mode is Unspecified (the "Auto" case) and the resolved
// watch-list is empty — the common alarmmgr-only deployment — the command
// is identical-in-effect to the historical SubscribeAlarms (wnwrap only):
// the worker builds the wnwrap consumer and no subtag watch-list.
SubscribeAlarmsCommand command = new()
{
SubscriptionExpression = subscription,
ForcedMode = forcedMode,
Failover = new AlarmFailoverConfig
{
ConsecutiveFailureThreshold = _options.Fallback.ConsecutiveFailureThreshold,
FailbackProbeIntervalSeconds = _options.Fallback.FailbackProbeIntervalSeconds,
FailbackStableProbes = _options.Fallback.FailbackStableProbes,
},
};
command.WatchList.AddRange(watchList);
WorkerCommandReply reply = await _sessionManager.InvokeAsync(
sessionId,
new WorkerCommand
@@ -216,7 +280,7 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
Command = new MxCommand
{
Kind = MxCommandKind.SubscribeAlarms,
SubscribeAlarms = new SubscribeAlarmsCommand { SubscriptionExpression = subscription },
SubscribeAlarms = command,
},
},
cancellationToken)
@@ -310,6 +374,98 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
}
}
// Handles the worker's provider-mode-change event: updates the stored provider
// status, broadcasts it to every subscriber (provider status is global, not
// alarm-scoped), records the switch metric, and forces a cache reconcile so the
// active-alarm set reflects whatever the new mode reports.
private async Task ApplyProviderModeChangeAsync(
string sessionId,
OnAlarmProviderModeChangedEvent change,
CancellationToken cancellationToken)
{
AlarmProviderMode toMode = change.Mode;
string reason = change.Reason ?? string.Empty;
AlarmProviderStatus status;
int fromModeInt;
lock (_sync)
{
fromModeInt = ModeToInt(_providerMode);
_providerMode = toMode;
_providerDegraded = toMode == AlarmProviderMode.Subtag;
_providerReason = reason;
_providerSince = DateTimeOffset.UtcNow;
status = BuildProviderStatus();
BroadcastToAll(new AlarmFeedMessage { ProviderStatus = status });
}
_metrics.AlarmProviderSwitched(fromModeInt, ModeToInt(toMode), reason);
_logger.LogInformation(
"Alarm provider mode changed to {Mode} (degraded={Degraded}): {Reason}",
toMode,
status.Degraded,
reason);
try
{
// Intentionally awaited OUTSIDE _sync: ReconcileAsync acquires _sync itself,
// so holding it across the await here would deadlock. Subscribers therefore
// see the ProviderStatus push (above) slightly before the cache is re-seeded
// by the reconcile — an accepted brief inconsistency.
await ReconcileAsync(sessionId, cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException)
{
throw;
}
catch (Exception exception)
{
_logger.LogDebug(
exception,
"Reconcile after alarm provider mode change failed; keeping the current cache.");
}
}
// Caller holds _sync. Builds an AlarmProviderStatus snapshot of the current state.
private AlarmProviderStatus BuildProviderStatus()
{
return new AlarmProviderStatus
{
Mode = _providerMode,
Degraded = _providerDegraded,
Reason = _providerReason,
Since = Timestamp.FromDateTimeOffset(_providerSince),
};
}
// Maps the configured fallback mode string to the forced provider mode the
// worker honours. Case-insensitive; anything other than the two force values
// (including the default "Auto") yields Unspecified ("let the worker decide").
private static AlarmProviderMode MapForcedMode(string? mode)
{
if (string.Equals(mode, "ForceAlarmManager", StringComparison.OrdinalIgnoreCase))
{
return AlarmProviderMode.Alarmmgr;
}
if (string.Equals(mode, "ForceSubtag", StringComparison.OrdinalIgnoreCase))
{
return AlarmProviderMode.Subtag;
}
return AlarmProviderMode.Unspecified;
}
// Maps the provider-mode enum to the integer the metric expects
// (alarmmgr=1, subtag=2, unknown/unspecified=0).
private static int ModeToInt(AlarmProviderMode mode) => mode switch
{
AlarmProviderMode.Alarmmgr => 1,
AlarmProviderMode.Subtag => 2,
_ => 0,
};
// Replaces the cache with the worker's authoritative snapshot, broadcasting
// a synthetic transition for any alarm the live stream missed.
private void ApplyReconcile(IEnumerable<ActiveAlarmSnapshot> snapshots)
@@ -374,6 +530,23 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
}
}
// Caller holds _sync. Pushes a feed message to every subscriber regardless of
// its alarm-filter prefix. Used for provider-status messages, which are global
// rather than scoped to a single alarm reference.
private void BroadcastToAll(AlarmFeedMessage message)
{
for (int index = _subscribers.Count - 1; index >= 0; index--)
{
Subscriber subscriber = _subscribers[index];
if (!subscriber.Channel.Writer.TryWrite(message))
{
subscriber.Channel.Writer.TryComplete(new InvalidOperationException(
"Alarm feed subscriber fell behind and was dropped; reconnect to re-snapshot."));
_subscribers.RemoveAt(index);
}
}
}
private void ClearCache()
{
lock (_sync)
@@ -398,11 +571,14 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
Subscriber subscriber = new(channel, prefix);
ActiveAlarmSnapshot[] snapshot;
AlarmProviderStatus providerStatus;
lock (_sync)
{
// Register before snapshotting under the same lock so no transition
// can slip between the snapshot and the live stream.
// Register before snapshotting under the same lock so neither a
// transition nor a provider-mode change can slip between the snapshot
// and the live stream.
_subscribers.Add(subscriber);
providerStatus = BuildProviderStatus();
snapshot = _alarms.Values
.Where(alarm => prefix.Length == 0
|| alarm.AlarmFullReference.StartsWith(prefix, StringComparison.Ordinal))
@@ -412,6 +588,10 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
try
{
// Emit the current provider status first so a late joiner immediately
// learns the mode (and whether the feed is degraded) before any alarms.
yield return new AlarmFeedMessage { ProviderStatus = providerStatus };
foreach (ActiveAlarmSnapshot alarm in snapshot)
{
yield return new AlarmFeedMessage { ActiveAlarm = alarm };
@@ -624,6 +804,8 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
Description = transition.Description,
OperatorUser = transition.OperatorUser,
OperatorComment = transition.OperatorComment,
Degraded = transition.Degraded,
SourceProvider = transition.SourceProvider,
};
if (transition.OriginalRaiseTimestamp is not null)
{
@@ -660,6 +842,8 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
Description = snapshot.Description,
OperatorUser = snapshot.OperatorUser,
OperatorComment = snapshot.OperatorComment,
Degraded = snapshot.Degraded,
SourceProvider = snapshot.SourceProvider,
};
if (snapshot.OriginalRaiseTimestamp is not null)
{
@@ -0,0 +1,28 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Configuration;
namespace ZB.MOM.WW.MxGateway.Server.Alarms;
/// <summary>
/// Resolves the subtag watch-list the gateway sends to the worker when the
/// central alarm monitor operates in subtag-polling fallback mode. Merges
/// Galaxy Repository alarm-attribute discovery with the configured
/// include/exclude overrides and composes the per-attribute subtag item
/// addresses from the configured subtag names.
/// </summary>
public interface IAlarmWatchListResolver
{
/// <summary>
/// Builds the subtag watch-list for the supplied alarm configuration.
/// </summary>
/// <param name="options">Alarm configuration carrying discovery and subtag-name settings.</param>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
/// <returns>
/// The resolved <see cref="AlarmSubtagTarget"/> watch-list, possibly empty.
/// Discovery being unavailable never throws; the caller decides what to do
/// with an empty list.
/// </returns>
Task<IReadOnlyList<AlarmSubtagTarget>> ResolveAsync(
AlarmsOptions options,
CancellationToken cancellationToken = default);
}
@@ -0,0 +1,125 @@
namespace ZB.MOM.WW.MxGateway.Server.Configuration;
/// <summary>
/// Controls how the central alarm monitor selects between the MXAccess
/// alarm-manager subscription and the subtag-polling fallback, and
/// governs the failure-detection thresholds used when switching.
/// </summary>
public sealed class AlarmFallbackOptions
{
/// <summary>
/// Selects the operating mode for the alarm-manager ↔ subtag fallback
/// mechanism. Accepted values (case-insensitive):
/// <list type="bullet">
/// <item><c>Auto</c> — use the alarm manager; switch to subtag polling
/// automatically when <see cref="ConsecutiveFailureThreshold"/> failures
/// are detected, and probe for failback.</item>
/// <item><c>ForceAlarmManager</c> — always use the alarm manager;
/// never fall back.</item>
/// <item><c>ForceSubtag</c> — always use subtag polling;
/// never try the alarm manager.</item>
/// </list>
/// Default is <c>Auto</c>.
/// </summary>
public string Mode { get; init; } = "Auto";
/// <summary>
/// Number of consecutive alarm-manager failures before the monitor
/// switches to subtag-polling fallback. Must be at least 1. Default 3.
/// </summary>
public int ConsecutiveFailureThreshold { get; init; } = 3;
/// <summary>
/// How often (in seconds) the monitor sends a probe to the alarm manager
/// while operating in subtag-polling fallback mode, to detect recovery.
/// Must be at least 1. Default 30.
/// </summary>
public int FailbackProbeIntervalSeconds { get; init; } = 30;
/// <summary>
/// Number of consecutive successful probes required before the monitor
/// considers the alarm manager recovered and switches back. Must be at
/// least 1. Default 3.
/// </summary>
public int FailbackStableProbes { get; init; } = 3;
/// <summary>
/// Controls how the monitor discovers the set of objects to poll when
/// operating in subtag-polling fallback mode.
/// </summary>
public AlarmDiscoveryOptions Discovery { get; init; } = new();
/// <summary>
/// Configures the subtag names the monitor reads when polling alarm state
/// in subtag-fallback mode.
/// </summary>
public AlarmSubtagNameOptions Subtags { get; init; } = new();
}
/// <summary>
/// Governs how the alarm monitor discovers objects to include in subtag-polling
/// fallback mode. Either the Galaxy Repository query (when
/// <see cref="UseGalaxyRepository"/> is <c>true</c>) or an explicit
/// <see cref="IncludeAttributes"/> list must be supplied when
/// <c>MxGateway:Alarms:Fallback:Mode</c> is <c>ForceSubtag</c>.
/// </summary>
public sealed class AlarmDiscoveryOptions
{
/// <summary>
/// When <c>true</c> the monitor queries the Galaxy Repository SQL database
/// to enumerate alarm objects for the configured area. Default <c>true</c>.
/// </summary>
public bool UseGalaxyRepository { get; init; } = true;
/// <summary>
/// Galaxy area to scope the Repository query to. When empty the monitor
/// falls back to <see cref="AlarmsOptions.DefaultArea"/>. Ignored when
/// <see cref="UseGalaxyRepository"/> is <c>false</c>.
/// </summary>
public string Area { get; init; } = string.Empty;
/// <summary>
/// Explicit list of MXAccess attribute paths to include in subtag polling,
/// supplementing (or replacing, when <see cref="UseGalaxyRepository"/> is
/// <c>false</c>) the Repository-derived list. Default empty.
/// </summary>
public string[] IncludeAttributes { get; init; } = Array.Empty<string>();
/// <summary>
/// Attribute paths to exclude from the Repository-derived poll list.
/// Ignored when <see cref="UseGalaxyRepository"/> is <c>false</c>.
/// Default empty.
/// </summary>
public string[] ExcludeAttributes { get; init; } = Array.Empty<string>();
}
/// <summary>
/// Configures the subtag names read by the alarm monitor when it is operating
/// in subtag-polling fallback mode. Names are matched against MXAccess item
/// handles; validation against the live MXAccess attribute list occurs at
/// runtime, not at startup.
/// </summary>
public sealed class AlarmSubtagNameOptions
{
/// <summary>
/// Subtag name for the active-alarm flag. Default <c>active</c>.
/// </summary>
public string Active { get; init; } = "active";
/// <summary>
/// Subtag name for the acknowledged flag. Default <c>acked</c>.
/// </summary>
public string Acked { get; init; } = "acked";
/// <summary>
/// Optional subtag name for the acknowledgement comment field.
/// When empty the feature is disabled. Verified against MXAccess at
/// runtime before use. Default empty.
/// </summary>
public string AckComment { get; init; } = string.Empty;
/// <summary>
/// Subtag name for the alarm priority. Default <c>priority</c>.
/// </summary>
public string Priority { get; init; } = "priority";
}
@@ -45,4 +45,12 @@ public sealed class AlarmsOptions
/// the monitor floors it at 5 seconds.
/// </summary>
public int ReconcileIntervalSeconds { get; init; } = 30;
/// <summary>
/// Configuration for the alarm-manager ↔ subtag fallback mechanism:
/// operating mode, failure-detection thresholds, discovery, and subtag
/// names. Defaults (Mode = "Auto") preserve behaviour when the section is
/// omitted from configuration.
/// </summary>
public AlarmFallbackOptions Fallback { get; init; } = new();
}
@@ -231,6 +231,8 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
builder);
}
private static readonly string[] ValidAlarmFallbackModes = ["Auto", "ForceAlarmManager", "ForceSubtag"];
private static void ValidateAlarms(AlarmsOptions options, ValidationBuilder builder)
{
if (!options.Enabled)
@@ -255,6 +257,46 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
builder.Add(
@"MxGateway:Alarms:SubscriptionExpression must start with '\\' (canonical \\<host>\Galaxy!<area> shape).");
}
ValidateAlarmFallback(options.Fallback, builder);
}
private static void ValidateAlarmFallback(AlarmFallbackOptions fallback, ValidationBuilder builder)
{
// Validate Mode is one of the recognised values (case-insensitive).
bool modeValid = Array.Exists(
ValidAlarmFallbackModes,
m => string.Equals(m, fallback.Mode, StringComparison.OrdinalIgnoreCase));
if (!modeValid)
{
builder.Add(
$"MxGateway:Alarms:Fallback:Mode must be one of: {string.Join(", ", ValidAlarmFallbackModes)} (was '{fallback.Mode}').");
}
// ForceSubtag requires either Galaxy Repository discovery or an explicit IncludeAttributes list.
if (modeValid
&& string.Equals(fallback.Mode, "ForceSubtag", StringComparison.OrdinalIgnoreCase)
&& !fallback.Discovery.UseGalaxyRepository
&& fallback.Discovery.IncludeAttributes.Length == 0)
{
builder.Add(
"MxGateway:Alarms:Fallback ForceSubtag requires Galaxy Repository discovery or a non-empty Discovery:IncludeAttributes list.");
}
// Floor validation: numeric thresholds must be at least 1.
AddIfNotPositive(
fallback.ConsecutiveFailureThreshold,
"MxGateway:Alarms:Fallback:ConsecutiveFailureThreshold must be greater than zero.",
builder);
AddIfNotPositive(
fallback.FailbackProbeIntervalSeconds,
"MxGateway:Alarms:Fallback:FailbackProbeIntervalSeconds must be greater than zero.",
builder);
AddIfNotPositive(
fallback.FailbackStableProbes,
"MxGateway:Alarms:Fallback:FailbackStableProbes must be greater than zero.",
builder);
}
private const int MinimumCertValidityYears = 1;
@@ -1,7 +1,10 @@
@page "/alarms"
@implements IAsyncDisposable
@using Microsoft.AspNetCore.SignalR.Client
@using ZB.MOM.WW.MxGateway.Server.Dashboard.Hubs
@inject IDashboardLiveDataService LiveData
@inject IOptions<GatewayOptions> GatewayOptions
@inject DashboardHubConnectionFactory HubFactory
<PageTitle>Dashboard Alarms</PageTitle>
@@ -10,6 +13,12 @@
<h1>Alarms</h1>
<div class="text-secondary">@HeaderLine()</div>
</div>
<div class="d-flex align-items-center gap-2">
<span class="badge @_providerStatus.BadgeCssClass"
title="@ProviderStatusTitle()">
@_providerStatus.Label
</span>
</div>
</div>
@if (!GatewayOptions.Value.Alarms.Enabled)
@@ -163,10 +172,44 @@
private readonly CancellationTokenSource _cts = new();
private Task? _pollTask;
private DashboardAlarmProviderStatus _providerStatus = DashboardAlarmProviderStatus.Healthy;
private HubConnection? _alarmsHub;
/// <inheritdoc />
protected override void OnInitialized()
{
_pollTask = PollLoopAsync();
_ = AttachAlarmsHubAsync();
}
private string? ProviderStatusTitle()
{
return _providerStatus.IsDegraded && !string.IsNullOrWhiteSpace(_providerStatus.Reason)
? _providerStatus.Reason
: null;
}
private async Task AttachAlarmsHubAsync()
{
_alarmsHub = HubFactory.Create("/hubs/alarms");
_alarmsHub.On<AlarmFeedMessage>(AlarmsHub.AlarmMessage, async message =>
{
if (message.PayloadCase == AlarmFeedMessage.PayloadOneofCase.ProviderStatus)
{
_providerStatus = DashboardAlarmProviderStatus.FromFeed(message);
await InvokeAsync(StateHasChanged).ConfigureAwait(false);
}
});
try
{
await _alarmsHub.StartAsync(_cts.Token).ConfigureAwait(false);
}
catch
{
// The badge is best-effort; it stays at the healthy default until
// the hub reconnects and delivers a fresh provider-status message.
}
}
private string HeaderLine()
@@ -268,6 +311,19 @@
public async ValueTask DisposeAsync()
{
await _cts.CancelAsync();
if (_alarmsHub is not null)
{
try
{
await _alarmsHub.DisposeAsync();
}
catch
{
// Disposal-time errors are best-effort.
}
}
if (_pollTask is not null)
{
try
@@ -6,13 +6,19 @@
cookie scheme's LoginPath="/login" redirect lands here for unauthenticated users.
The card is the shared kit's <LoginCard>: it renders a NATIVE static
<form method="post" action="/login"> (username/password + hidden returnUrl). A native
form submit is not a Blazor event, so it reaches the minimal-API POST /login endpoint
<form method="post" action="/auth/login"> (username/password + hidden returnUrl). A native
form submit is not a Blazor event, so it reaches the minimal-API POST /auth/login endpoint
regardless of this app's InteractiveServer render mode. <AntiforgeryToken/> supplies the
token that PostLoginAsync's antiforgery.ValidateRequestAsync checks. *@
token that PostLoginAsync's antiforgery.ValidateRequestAsync checks.
NOTE: the POST target is /auth/login, NOT /login. This @page lives at "/login" and the
Razor Components endpoint matches ALL methods, so a POST to /login collided with the
minimal-API MapPost("/login") and threw AmbiguousMatchException (HTTP 500). Posting to a
distinct /auth/login path (mirroring ScadaBridge) keeps the GET page and POST handler from
sharing a route. *@
@attribute [AllowAnonymous]
<LoginCard Product="MXAccess Gateway" Action="/login" ReturnUrl="@ReturnUrl" Error="@Error">
<LoginCard Product="MXAccess Gateway" Action="/auth/login" ReturnUrl="@ReturnUrl" Error="@Error">
<AntiforgeryToken />
</LoginCard>
@@ -0,0 +1,78 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
/// <summary>
/// Dashboard projection of an <see cref="AlarmProviderStatus" /> message
/// carried on the alarm feed. Maps the protobuf provider mode / degraded
/// flag into Bootstrap-only display fields so the Alarms page can render a
/// status badge without touching protobuf types.
/// </summary>
public sealed record DashboardAlarmProviderStatus(
AlarmProviderMode Mode,
bool IsDegraded,
string Label,
string BadgeCssClass,
string Reason,
DateTimeOffset? SinceUtc)
{
/// <summary>Badge label shown when the alarm-manager provider is healthy.</summary>
public const string AlarmManagerLabel = "Alarm Manager";
/// <summary>Badge label shown when the feed has fallen back to subtag monitoring.</summary>
public const string DegradedLabel = "Subtag monitoring (degraded)";
private const string HealthyBadge = "bg-success";
private const string DegradedBadge = "bg-warning text-dark";
/// <summary>
/// The default status assumed before the first provider-status message
/// arrives: healthy alarm-manager mode.
/// </summary>
public static DashboardAlarmProviderStatus Healthy { get; } = new(
Mode: AlarmProviderMode.Alarmmgr,
IsDegraded: false,
Label: AlarmManagerLabel,
BadgeCssClass: HealthyBadge,
Reason: string.Empty,
SinceUtc: null);
/// <summary>Projects an alarm-feed provider-status payload into a dashboard badge model.</summary>
/// <param name="status">The provider-status payload from an <see cref="AlarmFeedMessage" />.</param>
/// <returns>The projected dashboard status.</returns>
public static DashboardAlarmProviderStatus FromProviderStatus(AlarmProviderStatus status)
{
ArgumentNullException.ThrowIfNull(status);
// Treat the explicit degraded flag and the SUBTAG mode as equivalent;
// the contract sets degraded=true whenever mode == SUBTAG, but guard
// against either being set independently.
bool degraded = status.Degraded || status.Mode == AlarmProviderMode.Subtag;
return new DashboardAlarmProviderStatus(
Mode: status.Mode,
IsDegraded: degraded,
Label: degraded ? DegradedLabel : AlarmManagerLabel,
BadgeCssClass: degraded ? DegradedBadge : HealthyBadge,
Reason: status.Reason ?? string.Empty,
SinceUtc: status.Since?.ToDateTimeOffset());
}
/// <summary>Projects an alarm-feed message into a dashboard badge model.</summary>
/// <param name="message">An alarm-feed message whose payload is a provider status.</param>
/// <returns>The projected dashboard status.</returns>
/// <exception cref="ArgumentException">The message does not carry a provider-status payload.</exception>
public static DashboardAlarmProviderStatus FromFeed(AlarmFeedMessage message)
{
ArgumentNullException.ThrowIfNull(message);
if (message.PayloadCase != AlarmFeedMessage.PayloadOneofCase.ProviderStatus)
{
throw new ArgumentException(
"Alarm-feed message does not carry a provider-status payload.",
nameof(message));
}
return FromProviderStatus(message.ProviderStatus);
}
}
@@ -29,8 +29,14 @@ public static class DashboardEndpointRouteBuilderExtensions
// kit's <LoginCard>. Its [AllowAnonymous] attribute overrides the
// RequireAuthorization(ViewerPolicy) that MapRazorComponents<App>() applies,
// so the cookie scheme's LoginPath="/login" redirect resolves for anonymous users.
//
// The credential POST is mapped to /auth/login, NOT /login. The @page "/login"
// Razor Components endpoint matches ALL HTTP methods, so a MapPost("/login") shared
// the "/login" route with it and every POST threw AmbiguousMatchException (HTTP 500).
// A distinct /auth/login path (as ScadaBridge does) keeps the GET page and the POST
// handler on separate routes. The <LoginCard Action="/auth/login"> form posts here.
endpoints.MapPost(
"/login",
"/auth/login",
(HttpContext httpContext, IAntiforgery antiforgery, IDashboardAuthenticator authenticator) =>
PostLoginAsync(httpContext, antiforgery, authenticator))
.AllowAnonymous()
@@ -0,0 +1,36 @@
namespace ZB.MOM.WW.MxGateway.Server.Galaxy;
/// <summary>
/// One alarm-bearing attribute discovered by
/// <see cref="GalaxyRepository.GetAlarmAttributesAsync"/>: an attribute whose owning
/// object configures an <c>AlarmExtension</c> primitive (the same
/// <c>is_alarm</c> detection used by <see cref="GalaxyRepository.GetAttributesAsync"/>).
/// Used to build the subtag-fallback watch-list.
/// </summary>
public sealed class GalaxyAlarmAttributeRow
{
/// <summary>
/// Gets the alarm-bearing attribute reference (e.g. <c>Tank01.Level.HiHi</c>),
/// matching the <c>full_tag_reference</c> projection of
/// <see cref="GalaxyRepository.GetAttributesAsync"/>.
/// </summary>
public string FullTagReference { get; init; } = string.Empty;
/// <summary>
/// Gets the owning object reference (e.g. <c>Tank01</c>). This is the Galaxy
/// <c>tag_name</c> — the segment that precedes the first attribute dot in
/// <see cref="FullTagReference"/>.
/// </summary>
public string SourceObjectReference { get; init; } = string.Empty;
/// <summary>
/// Gets the writable ack-comment attribute address.
/// <para>
/// The Galaxy Repository schema does not expose an ack-comment subtag address
/// directly, so this is always <see cref="string.Empty"/> here. The watch-list
/// resolver (a later task) composes the concrete address from configuration plus
/// <see cref="SourceObjectReference"/> / <see cref="FullTagReference"/>.
/// </para>
/// </summary>
public string AckCommentSubtag { get; init; } = string.Empty;
}
@@ -114,6 +114,56 @@ public sealed class GalaxyRepository(GalaxyRepositoryOptions options) : IGalaxyR
return rows;
}
/// <summary>
/// Retrieves only the alarm-bearing attributes for the subtag-fallback watch-list.
/// Alarm detection is identical to <see cref="GetAttributesAsync"/>: a row is
/// alarm-bearing when its owning object configures an <c>AlarmExtension</c>
/// primitive (the same <c>is_alarm</c> projection, here applied as a SQL filter).
/// </summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
public async Task<List<GalaxyAlarmAttributeRow>> GetAlarmAttributesAsync(CancellationToken ct = default)
{
List<GalaxyAlarmAttributeRow> rows = new();
using SqlConnection conn = new(options.ConnectionString);
await conn.OpenAsync(ct).ConfigureAwait(false);
using SqlCommand cmd = new(AlarmAttributesSql, conn) { CommandTimeout = options.CommandTimeoutSeconds };
using SqlDataReader reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
rows.Add(MapAlarmRow(
fullTagReference: reader.GetString(0),
sourceObjectReference: reader.GetString(1)));
}
return rows;
}
/// <summary>
/// Maps a raw alarm-attribute reader row to a <see cref="GalaxyAlarmAttributeRow"/>.
/// <para>
/// <paramref name="sourceObjectReference"/> is the Galaxy <c>tag_name</c> (the
/// owning object), and <paramref name="fullTagReference"/> is
/// <c>tag_name + '.' + attribute_name</c> — the same composition the
/// <c>full_tag_reference</c> projection of <see cref="AttributesSql"/> produces.
/// <see cref="GalaxyAlarmAttributeRow.AckCommentSubtag"/> is left empty here; the
/// schema does not expose an ack-comment address and the watch-list resolver
/// composes it later.
/// </para>
/// Exposed internally so the derivation can be unit-tested without a database.
/// </summary>
/// <param name="fullTagReference">The alarm-bearing attribute reference.</param>
/// <param name="sourceObjectReference">The owning object reference (tag name).</param>
internal static GalaxyAlarmAttributeRow MapAlarmRow(
string fullTagReference,
string sourceObjectReference) => new()
{
FullTagReference = fullTagReference,
SourceObjectReference = sourceObjectReference,
AckCommentSubtag = string.Empty,
};
private const string HierarchySql = @"
;WITH template_chain AS (
SELECT g.gobject_id AS instance_gobject_id, t.gobject_id AS template_gobject_id,
@@ -248,5 +298,71 @@ SELECT
FROM ranked r
LEFT JOIN data_type dt ON dt.mx_data_type = r.mx_data_type
WHERE r.rn = 1
ORDER BY r.tag_name, r.attribute_name";
// Alarm-only discovery for the subtag-fallback watch-list. This deliberately reuses the
// exact candidate/ranked CTE structure and the same `AlarmExtension`-based is_alarm
// detection as AttributesSql so the two queries cannot drift: a row qualifies only when
// its user attribute (src_pri 0) anchors an `AlarmExtension` primitive on the owning
// object. It projects just what the watch-list needs — full_tag_reference (tag_name +
// '.' + attribute_name, matching AttributesSql) and the owning object's tag_name as
// source_object_reference. The array `[]` suffix is intentionally omitted: an
// alarm-bearing attribute is a scalar anchor, not an array body.
private const string AlarmAttributesSql = @"
;WITH deployed_package_chain AS (
SELECT g.gobject_id, p.package_id, p.derived_from_package_id, 0 AS depth
FROM gobject g
INNER JOIN package p ON p.package_id = g.deployed_package_id
WHERE g.is_template = 0 AND g.deployed_package_id <> 0
UNION ALL
SELECT dpc.gobject_id, p.package_id, p.derived_from_package_id, dpc.depth + 1
FROM deployed_package_chain dpc
INNER JOIN package p ON p.package_id = dpc.derived_from_package_id
WHERE dpc.derived_from_package_id <> 0 AND dpc.depth < 10
),
candidate AS (
SELECT
dpc.gobject_id, g.tag_name, da.attribute_name, dpc.depth, 0 AS src_pri
FROM deployed_package_chain dpc
INNER JOIN dynamic_attribute da ON da.package_id = dpc.package_id
INNER JOIN gobject g ON g.gobject_id = dpc.gobject_id
INNER JOIN template_definition td ON td.template_definition_id = g.template_definition_id
WHERE td.category_id IN (1, 3, 4, 10, 11, 13, 17, 24, 26)
AND da.attribute_name NOT LIKE '[_]%'
AND da.attribute_name NOT LIKE '%.Description'
AND da.mx_attribute_category IN (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 24)
UNION ALL
SELECT
dpc.gobject_id, g.tag_name,
CASE WHEN pi.primitive_name IS NULL OR pi.primitive_name = ''
THEN ad.attribute_name
ELSE pi.primitive_name + '.' + ad.attribute_name END AS attribute_name,
dpc.depth, 1 AS src_pri
FROM deployed_package_chain dpc
INNER JOIN primitive_instance pi ON pi.package_id = dpc.package_id
INNER JOIN attribute_definition ad ON ad.primitive_definition_id = pi.primitive_definition_id
INNER JOIN gobject g ON g.gobject_id = dpc.gobject_id
INNER JOIN template_definition td ON td.template_definition_id = g.template_definition_id
WHERE td.category_id IN (1, 3, 4, 10, 11, 13, 17, 24, 26)
AND ad.attribute_name NOT LIKE '[_]%'
AND ad.attribute_name NOT LIKE '%.Description'
),
ranked AS (
SELECT c.*, ROW_NUMBER() OVER (
PARTITION BY c.gobject_id, c.attribute_name ORDER BY c.src_pri, c.depth) AS rn
FROM candidate c
)
SELECT
r.tag_name + '.' + r.attribute_name AS full_tag_reference,
r.tag_name AS source_object_reference
FROM ranked r
WHERE r.rn = 1
AND r.src_pri = 0
AND EXISTS (
SELECT 1 FROM deployed_package_chain dpc2
INNER JOIN primitive_instance pi ON pi.package_id = dpc2.package_id AND pi.primitive_name = r.attribute_name
INNER JOIN primitive_definition pd ON pd.primitive_definition_id = pi.primitive_definition_id AND pd.primitive_name = 'AlarmExtension'
WHERE dpc2.gobject_id = r.gobject_id
)
ORDER BY r.tag_name, r.attribute_name";
}
@@ -27,4 +27,12 @@ public interface IGalaxyRepository
/// <summary>Retrieves all attributes for Galaxy objects from the repository.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default);
/// <summary>
/// Retrieves only the alarm-bearing attributes (those whose owning object
/// configures an <c>AlarmExtension</c> primitive) for building the
/// subtag-fallback watch-list.
/// </summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
Task<List<GalaxyAlarmAttributeRow>> GetAlarmAttributesAsync(CancellationToken ct = default);
}
@@ -1,5 +1,6 @@
using System.Collections.Concurrent;
using System.Diagnostics.Metrics;
using System.Globalization;
namespace ZB.MOM.WW.MxGateway.Server.Metrics;
@@ -22,6 +23,7 @@ public sealed class GatewayMetrics : IDisposable
private readonly Counter<long> _heartbeatFailuresCounter;
private readonly Counter<long> _streamDisconnectsCounter;
private readonly Counter<long> _retryAttemptsCounter;
private readonly Counter<long> _alarmProviderSwitchesCounter;
private readonly Histogram<double> _workerStartupLatencyHistogram;
private readonly Histogram<double> _commandLatencyHistogram;
private readonly Histogram<double> _eventStreamSendLatencyHistogram;
@@ -34,6 +36,7 @@ public sealed class GatewayMetrics : IDisposable
private int _workersRunning;
private int _workerEventQueueDepth;
private int _grpcEventStreamQueueDepth;
private int _alarmProviderMode;
private long _sessionsOpened;
private long _sessionsClosed;
private long _commandsStarted;
@@ -68,6 +71,7 @@ public sealed class GatewayMetrics : IDisposable
_heartbeatFailuresCounter = _meter.CreateCounter<long>("mxgateway.heartbeats.failed");
_streamDisconnectsCounter = _meter.CreateCounter<long>("mxgateway.grpc.streams.disconnected");
_retryAttemptsCounter = _meter.CreateCounter<long>("mxgateway.retries.attempted");
_alarmProviderSwitchesCounter = _meter.CreateCounter<long>("mxgateway.alarms.provider_switches");
_workerStartupLatencyHistogram = _meter.CreateHistogram<double>("mxgateway.workers.startup.duration", "s");
_commandLatencyHistogram = _meter.CreateHistogram<double>("mxgateway.commands.duration", "s");
_eventStreamSendLatencyHistogram = _meter.CreateHistogram<double>("mxgateway.events.stream_send.duration", "s");
@@ -76,6 +80,7 @@ public sealed class GatewayMetrics : IDisposable
_meter.CreateObservableGauge("mxgateway.workers.running", GetWorkersRunning);
_meter.CreateObservableGauge("mxgateway.events.worker_queue.depth", GetWorkerEventQueueDepth);
_meter.CreateObservableGauge("mxgateway.events.grpc_stream_queue.depth", GetGrpcEventStreamQueueDepth);
_meter.CreateObservableGauge("mxgateway.alarms.provider_mode", GetAlarmProviderMode);
}
/// <summary>
@@ -377,6 +382,32 @@ public sealed class GatewayMetrics : IDisposable
_retryAttemptsCounter.Add(1, new KeyValuePair<string, object?>("area", area));
}
/// <summary>
/// Records that the alarm provider switched modes and updates the current provider mode gauge.
/// </summary>
/// <param name="fromMode">Provider mode before the switch (1=alarmmgr, 2=subtag, 0=unknown).</param>
/// <param name="toMode">Provider mode after the switch (1=alarmmgr, 2=subtag, 0=unknown).</param>
/// <param name="reason">Human-readable reason for the switch.</param>
public void AlarmProviderSwitched(int fromMode, int toMode, string reason)
{
lock (_syncRoot)
{
_alarmProviderMode = toMode;
}
_alarmProviderSwitchesCounter.Add(
1,
new KeyValuePair<string, object?>("from", fromMode.ToString(CultureInfo.InvariantCulture)),
new KeyValuePair<string, object?>("to", toMode.ToString(CultureInfo.InvariantCulture)),
new KeyValuePair<string, object?>("reason", reason));
}
/// <summary>Sets the current alarm provider-mode gauge without recording a switch (e.g. startup baseline).</summary>
public void SetAlarmProviderMode(int mode)
{
lock (_syncRoot) { _alarmProviderMode = mode; }
}
/// <summary>
/// Returns a snapshot of all current metric values.
/// </summary>
@@ -455,6 +486,14 @@ public sealed class GatewayMetrics : IDisposable
}
}
private int GetAlarmProviderMode()
{
lock (_syncRoot)
{
return _alarmProviderMode;
}
}
private static void Increment(Dictionary<string, long> values, string key)
{
values.TryGetValue(key, out long currentValue);
@@ -11,7 +11,7 @@
<PackageReference Include="ZB.MOM.WW.Auth.ApiKeys" Version="0.1.2" />
<PackageReference Include="ZB.MOM.WW.Auth.AspNetCore" Version="0.1.2" />
<PackageReference Include="ZB.MOM.WW.Audit" Version="0.1.0" />
<PackageReference Include="ZB.MOM.WW.Theme" Version="0.2.0" />
<PackageReference Include="ZB.MOM.WW.Theme" Version="0.3.1" />
<PackageReference Include="ZB.MOM.WW.Configuration" Version="0.1.0" />
<PackageReference Include="ZB.MOM.WW.Health" Version="0.1.0" />
<PackageReference Include="ZB.MOM.WW.Telemetry" Version="0.1.0" />
@@ -0,0 +1,476 @@
using System.Diagnostics.CodeAnalysis;
using System.Runtime.CompilerServices;
using System.Threading.Channels;
using Google.Protobuf.WellKnownTypes;
using Microsoft.Extensions.Logging.Abstractions;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Alarms;
using ZB.MOM.WW.MxGateway.Server.Configuration;
using ZB.MOM.WW.MxGateway.Server.Metrics;
using ZB.MOM.WW.MxGateway.Server.Sessions;
namespace ZB.MOM.WW.MxGateway.Tests.Alarms;
/// <summary>
/// Drives a single <see cref="GatewayAlarmMonitor"/> instance through the full
/// alarm-provider failover/failback lifecycle and asserts the live feed reflects
/// each stage. This complements the per-aspect tests in
/// <c>GatewayAlarmMonitorProviderModeTests</c> with one cohesive scenario:
/// subscribe (watch-list + forced mode) -> baseline alarmmgr status -> alarmmgr
/// transition -> failover to subtag (degraded) -> subtag transition -> failback
/// to alarmmgr (recovered).
/// </summary>
/// <remarks>
/// The minimal session-manager / watch-list-resolver harness here is replicated
/// (not shared) from the sibling <c>GatewayAlarmMonitorProviderModeTests</c>. The
/// sibling's harness is a private nested type, and the task forbids changing that
/// test's behaviour; replicating the few members this scenario needs keeps the
/// sibling completely untouched and this file self-contained, at the cost of a
/// small amount of duplication.
/// </remarks>
public sealed class AlarmFailoverEndToEndTests
{
private static readonly TimeSpan WaitTimeout = TimeSpan.FromSeconds(30);
[Fact]
public async Task ProviderFailoverAndFailback_FullLifecycle_ReflectedInFeed()
{
using GatewayMetrics metrics = new();
FakeSessionManager sessions = new();
// Watch-list-bearing, Auto config: a non-empty resolved watch-list with the
// default ("Auto" -> Unspecified) forced mode, so step 1 can assert both the
// resolved watch-list and the forced mode/failover the SubscribeAlarms carries.
StubWatchListResolver resolver = new(
[
new AlarmSubtagTarget { AlarmFullReference = "Galaxy!Area.Tank01.Hi", ActiveSubtag = "Tank01.Hi.active" },
new AlarmSubtagTarget { AlarmFullReference = "Galaxy!Area.Tank02.Lo", ActiveSubtag = "Tank02.Lo.active" },
]);
AlarmsOptions options = new()
{
Enabled = true,
SubscriptionExpression = @"\\NODE\Galaxy!Area",
Fallback = new AlarmFallbackOptions
{
Mode = "Auto",
ConsecutiveFailureThreshold = 3,
FailbackProbeIntervalSeconds = 9,
FailbackStableProbes = 2,
},
};
using GatewayAlarmMonitor monitor = new(
sessions,
resolver,
metrics,
Microsoft.Extensions.Options.Options.Create(new GatewayOptions { Alarms = options }),
NullLogger<GatewayAlarmMonitor>.Instance);
using CancellationTokenSource cts = new();
await monitor.StartAsync(cts.Token);
await sessions.WaitForSubscribeAsync(WaitTimeout);
// --- Step 1: SubscribeAlarms carries the resolved watch-list + forced mode/failover. ---
SubscribeAlarmsCommand sent = Assert.IsType<SubscribeAlarmsCommand>(sessions.LastSubscribeCommand);
Assert.Equal(AlarmProviderMode.Unspecified, sent.ForcedMode); // "Auto"
Assert.Equal(3, sent.Failover.ConsecutiveFailureThreshold);
Assert.Equal(9, sent.Failover.FailbackProbeIntervalSeconds);
Assert.Equal(2, sent.Failover.FailbackStableProbes);
Assert.NotEmpty(sent.WatchList);
Assert.Equal(2, sent.WatchList.Count);
Assert.Contains(sent.WatchList, t => t.AlarmFullReference == "Galaxy!Area.Tank01.Hi");
// Live feed reader collecting every message in order.
List<AlarmFeedMessage> received = [];
TaskCompletionSource baselineReceived = new(TaskCreationOptions.RunContinuationsAsynchronously);
using CancellationTokenSource streamCts = new();
Task reader = Task.Run(async () =>
{
try
{
await foreach (AlarmFeedMessage message in monitor.StreamAsync(null, streamCts.Token))
{
lock (received)
{
received.Add(message);
if (received.Count == 1)
{
// The first message is the baseline ProviderStatus.
baselineReceived.TrySetResult();
}
}
}
}
catch (OperationCanceledException)
{
// Expected when the test cancels the stream.
}
});
// --- Step 2: first message is ProviderStatus{Alarmmgr, Degraded=false}. ---
await baselineReceived.Task.WaitAsync(WaitTimeout);
AlarmFeedMessage baseline;
lock (received)
{
baseline = received[0];
}
Assert.Equal(AlarmFeedMessage.PayloadOneofCase.ProviderStatus, baseline.PayloadCase);
Assert.Equal(AlarmProviderMode.Alarmmgr, baseline.ProviderStatus.Mode);
Assert.False(baseline.ProviderStatus.Degraded);
// --- Step 3: alarmmgr-style transition (Degraded=false, SourceProvider=Alarmmgr, Raise). ---
sessions.EmitEvent(new MxEvent
{
OnAlarmTransition = new OnAlarmTransitionEvent
{
AlarmFullReference = "Galaxy!Area.Tank01.Hi",
SourceObjectReference = "Tank01",
AlarmTypeName = "AnalogLimitAlarm.Hi",
TransitionKind = AlarmTransitionKind.Raise,
Severity = 500,
Degraded = false,
SourceProvider = AlarmProviderMode.Alarmmgr,
TransitionTimestamp = Timestamp.FromDateTimeOffset(DateTimeOffset.UtcNow),
},
});
AlarmFeedMessage alarmmgrTransition = await WaitForAsync(
received,
m => m.PayloadCase == AlarmFeedMessage.PayloadOneofCase.Transition
&& m.Transition.AlarmFullReference == "Galaxy!Area.Tank01.Hi"
&& m.Transition.SourceProvider == AlarmProviderMode.Alarmmgr,
WaitTimeout);
Assert.Equal(AlarmTransitionKind.Raise, alarmmgrTransition.Transition.TransitionKind);
Assert.False(alarmmgrTransition.Transition.Degraded);
Assert.Equal(AlarmProviderMode.Alarmmgr, alarmmgrTransition.Transition.SourceProvider);
// --- Step 4: failover to subtag -> ProviderStatus{Subtag, Degraded=true}. ---
sessions.EmitEvent(new MxEvent
{
OnAlarmProviderModeChanged = new OnAlarmProviderModeChangedEvent
{
Mode = AlarmProviderMode.Subtag,
Reason = "alarmmgr failed",
Hresult = unchecked((int)0x80004005),
At = Timestamp.FromDateTimeOffset(DateTimeOffset.UtcNow),
},
});
AlarmFeedMessage degraded = await WaitForAsync(
received,
m => m.PayloadCase == AlarmFeedMessage.PayloadOneofCase.ProviderStatus
&& m.ProviderStatus.Mode == AlarmProviderMode.Subtag,
WaitTimeout);
Assert.Equal(AlarmProviderMode.Subtag, degraded.ProviderStatus.Mode);
Assert.True(degraded.ProviderStatus.Degraded);
Assert.Equal("alarmmgr failed", degraded.ProviderStatus.Reason);
// --- Step 5: subtag-style transition (Degraded=true, SourceProvider=Subtag, Raise on a different ref). ---
sessions.EmitEvent(new MxEvent
{
OnAlarmTransition = new OnAlarmTransitionEvent
{
AlarmFullReference = "Galaxy!Area.Tank02.Lo",
SourceObjectReference = "Tank02",
AlarmTypeName = "AnalogLimitAlarm.Lo",
TransitionKind = AlarmTransitionKind.Raise,
Severity = 250,
Degraded = true,
SourceProvider = AlarmProviderMode.Subtag,
TransitionTimestamp = Timestamp.FromDateTimeOffset(DateTimeOffset.UtcNow),
},
});
AlarmFeedMessage subtagTransition = await WaitForAsync(
received,
m => m.PayloadCase == AlarmFeedMessage.PayloadOneofCase.Transition
&& m.Transition.AlarmFullReference == "Galaxy!Area.Tank02.Lo"
&& m.Transition.SourceProvider == AlarmProviderMode.Subtag,
WaitTimeout);
Assert.Equal(AlarmTransitionKind.Raise, subtagTransition.Transition.TransitionKind);
Assert.True(subtagTransition.Transition.Degraded);
Assert.Equal(AlarmProviderMode.Subtag, subtagTransition.Transition.SourceProvider);
// --- Step 6: failback to alarmmgr -> ProviderStatus{Alarmmgr, Degraded=false} (recovery). ---
sessions.EmitEvent(new MxEvent
{
OnAlarmProviderModeChanged = new OnAlarmProviderModeChangedEvent
{
Mode = AlarmProviderMode.Alarmmgr,
Reason = "recovered",
Hresult = 0,
At = Timestamp.FromDateTimeOffset(DateTimeOffset.UtcNow),
},
});
// Match the recovery status specifically: an Alarmmgr ProviderStatus that
// carries the "recovered" reason, distinguishing it from the baseline at [0].
AlarmFeedMessage recovered = await WaitForAsync(
received,
m => m.PayloadCase == AlarmFeedMessage.PayloadOneofCase.ProviderStatus
&& m.ProviderStatus.Mode == AlarmProviderMode.Alarmmgr
&& m.ProviderStatus.Reason == "recovered",
WaitTimeout);
Assert.Equal(AlarmProviderMode.Alarmmgr, recovered.ProviderStatus.Mode);
Assert.False(recovered.ProviderStatus.Degraded);
Assert.Equal("recovered", recovered.ProviderStatus.Reason);
await streamCts.CancelAsync();
await reader;
await cts.CancelAsync();
await monitor.StopAsync(CancellationToken.None);
}
[Fact]
public async Task DegradedTransition_CachedThenReplayed_CarriesDegradedAndSourceProviderToNewSubscriber()
{
using GatewayMetrics metrics = new();
FakeSessionManager sessions = new();
StubWatchListResolver resolver = new([]);
AlarmsOptions options = new()
{
Enabled = true,
SubscriptionExpression = @"\\NODE\Galaxy!Area",
Fallback = new AlarmFallbackOptions
{
Mode = "Auto",
ConsecutiveFailureThreshold = 3,
FailbackProbeIntervalSeconds = 9,
FailbackStableProbes = 2,
},
};
using GatewayAlarmMonitor monitor = new(
sessions,
resolver,
metrics,
Microsoft.Extensions.Options.Options.Create(new GatewayOptions { Alarms = options }),
NullLogger<GatewayAlarmMonitor>.Instance);
using CancellationTokenSource cts = new();
await monitor.StartAsync(cts.Token);
await sessions.WaitForSubscribeAsync(WaitTimeout);
// First subscriber: drive the feed past the baseline ProviderStatus so we
// know the monitor's event loop is live before we emit the transition.
List<AlarmFeedMessage> firstReader = [];
TaskCompletionSource baselineReceived = new(TaskCreationOptions.RunContinuationsAsynchronously);
using CancellationTokenSource firstStreamCts = new();
Task reader = Task.Run(async () =>
{
try
{
await foreach (AlarmFeedMessage message in monitor.StreamAsync(null, firstStreamCts.Token))
{
lock (firstReader)
{
firstReader.Add(message);
if (message.PayloadCase == AlarmFeedMessage.PayloadOneofCase.ProviderStatus)
{
baselineReceived.TrySetResult();
}
}
}
}
catch (OperationCanceledException)
{
// Expected when the test cancels the stream.
}
});
await baselineReceived.Task.WaitAsync(WaitTimeout);
// Apply a degraded (subtag) transition. This lands in the monitor's cache
// via SnapshotFromTransition, which must preserve Degraded/SourceProvider.
sessions.EmitEvent(new MxEvent
{
OnAlarmTransition = new OnAlarmTransitionEvent
{
AlarmFullReference = "Galaxy!Area.Tank02.Lo",
SourceObjectReference = "Tank02",
AlarmTypeName = "AnalogLimitAlarm.Lo",
TransitionKind = AlarmTransitionKind.Raise,
Severity = 250,
Degraded = true,
SourceProvider = AlarmProviderMode.Subtag,
TransitionTimestamp = Timestamp.FromDateTimeOffset(DateTimeOffset.UtcNow),
},
});
// Wait for the live transition to be observed by the first subscriber so we
// know the cache has been updated before opening the new stream.
await WaitForAsync(
firstReader,
m => m.PayloadCase == AlarmFeedMessage.PayloadOneofCase.Transition
&& m.Transition.AlarmFullReference == "Galaxy!Area.Tank02.Lo",
WaitTimeout);
// New subscriber: its initial cache snapshot must carry the degraded flags.
using CancellationTokenSource newStreamCts = new();
ActiveAlarmSnapshot? initialActiveAlarm = null;
await foreach (AlarmFeedMessage message in monitor.StreamAsync(null, newStreamCts.Token))
{
if (message.PayloadCase == AlarmFeedMessage.PayloadOneofCase.ActiveAlarm
&& message.ActiveAlarm.AlarmFullReference == "Galaxy!Area.Tank02.Lo")
{
initialActiveAlarm = message.ActiveAlarm;
}
if (message.PayloadCase == AlarmFeedMessage.PayloadOneofCase.SnapshotComplete)
{
break;
}
}
Assert.NotNull(initialActiveAlarm);
Assert.True(initialActiveAlarm!.Degraded);
Assert.Equal(AlarmProviderMode.Subtag, initialActiveAlarm.SourceProvider);
await newStreamCts.CancelAsync();
await firstStreamCts.CancelAsync();
await reader;
await cts.CancelAsync();
await monitor.StopAsync(CancellationToken.None);
}
private static async Task<AlarmFeedMessage> WaitForAsync(
List<AlarmFeedMessage> received,
Func<AlarmFeedMessage, bool> predicate,
TimeSpan timeout)
{
DateTime deadline = DateTime.UtcNow + timeout;
while (DateTime.UtcNow < deadline)
{
lock (received)
{
AlarmFeedMessage? match = received.FirstOrDefault(predicate);
if (match is not null)
{
return match;
}
}
await Task.Delay(25);
}
throw new TimeoutException("No matching AlarmFeedMessage was received in time.");
}
/// <summary><see cref="IAlarmWatchListResolver"/> that returns a fixed watch-list.</summary>
private sealed class StubWatchListResolver(IReadOnlyList<AlarmSubtagTarget> targets) : IAlarmWatchListResolver
{
/// <inheritdoc />
public Task<IReadOnlyList<AlarmSubtagTarget>> ResolveAsync(
AlarmsOptions options,
CancellationToken cancellationToken = default) => Task.FromResult(targets);
}
/// <summary>
/// Minimal <see cref="ISessionManager"/> for driving the monitor: opens a
/// constructed session, records the SubscribeAlarms command, replies OK to
/// every command, and exposes a channel for pushing worker events.
/// </summary>
private sealed class FakeSessionManager : ISessionManager
{
private readonly Channel<WorkerEvent> _events = Channel.CreateUnbounded<WorkerEvent>();
private readonly TaskCompletionSource _subscribed =
new(TaskCreationOptions.RunContinuationsAsynchronously);
/// <summary>The most recent SubscribeAlarms command the monitor sent.</summary>
public SubscribeAlarmsCommand? LastSubscribeCommand { get; private set; }
/// <summary>Pushes a worker event onto the monitor's event stream.</summary>
public void EmitEvent(MxEvent mxEvent) =>
_events.Writer.TryWrite(new WorkerEvent { Event = mxEvent });
/// <summary>Completes once the monitor has issued its SubscribeAlarms command.</summary>
public Task WaitForSubscribeAsync(TimeSpan timeout) => _subscribed.Task.WaitAsync(timeout);
/// <inheritdoc />
public Task<GatewaySession> OpenSessionAsync(
SessionOpenRequest request,
string? clientIdentity,
CancellationToken cancellationToken)
{
GatewaySession session = new(
Guid.NewGuid().ToString("N"),
"Galaxy",
"pipe-test",
"nonce-test",
clientIdentity,
null,
null,
TimeSpan.FromSeconds(30),
TimeSpan.FromSeconds(30),
TimeSpan.FromSeconds(30),
DateTimeOffset.UtcNow);
return Task.FromResult(session);
}
/// <inheritdoc />
public Task<WorkerCommandReply> InvokeAsync(
string sessionId,
WorkerCommand command,
CancellationToken cancellationToken)
{
if (command.Command?.Kind == MxCommandKind.SubscribeAlarms)
{
LastSubscribeCommand = command.Command.SubscribeAlarms;
_subscribed.TrySetResult();
}
MxCommandReply reply = new()
{
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
};
if (command.Command?.Kind == MxCommandKind.QueryActiveAlarms)
{
reply.QueryActiveAlarms = new QueryActiveAlarmsReplyPayload();
}
return Task.FromResult(new WorkerCommandReply { Reply = reply });
}
/// <inheritdoc />
public async IAsyncEnumerable<WorkerEvent> ReadEventsAsync(
string sessionId,
[EnumeratorCancellation] CancellationToken cancellationToken)
{
await foreach (WorkerEvent workerEvent in _events.Reader.ReadAllAsync(cancellationToken))
{
yield return workerEvent;
}
}
/// <inheritdoc />
public bool TryGetSession(string sessionId, [MaybeNullWhen(false)] out GatewaySession session)
{
session = null;
return false;
}
/// <inheritdoc />
public Task<SessionCloseResult> CloseSessionAsync(string sessionId, CancellationToken cancellationToken)
{
_events.Writer.TryComplete();
return Task.FromResult(new SessionCloseResult(sessionId, SessionState.Closed, AlreadyClosed: false));
}
/// <inheritdoc />
public Task<SessionCloseResult> KillWorkerAsync(string sessionId, string reason, CancellationToken cancellationToken) =>
Task.FromResult(new SessionCloseResult(sessionId, SessionState.Closed, AlreadyClosed: false));
/// <inheritdoc />
public Task<int> CloseExpiredLeasesAsync(DateTimeOffset now, CancellationToken cancellationToken) =>
Task.FromResult(0);
/// <inheritdoc />
public Task ShutdownAsync(CancellationToken cancellationToken) => Task.CompletedTask;
}
}
@@ -0,0 +1,332 @@
using Microsoft.Extensions.Logging.Abstractions;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Alarms;
using ZB.MOM.WW.MxGateway.Server.Configuration;
using ZB.MOM.WW.MxGateway.Server.Galaxy;
namespace ZB.MOM.WW.MxGateway.Tests.Alarms;
/// <summary>
/// Unit tests for <see cref="AlarmWatchListResolver"/>: discovery/config merge,
/// subtag-address composition, canonical reference shaping, and the
/// unavailable-discovery code path.
/// </summary>
public sealed class AlarmWatchListResolverTests
{
private static AlarmWatchListResolver CreateResolver(IGalaxyRepository repository) =>
new(repository, NullLogger<AlarmWatchListResolver>.Instance);
private static AlarmsOptions Options(
bool useGalaxyRepository = true,
string area = "",
string defaultArea = "",
string[]? include = null,
string[]? exclude = null,
AlarmSubtagNameOptions? subtags = null) =>
new()
{
DefaultArea = defaultArea,
Fallback = new AlarmFallbackOptions
{
Discovery = new AlarmDiscoveryOptions
{
UseGalaxyRepository = useGalaxyRepository,
Area = area,
IncludeAttributes = include ?? [],
ExcludeAttributes = exclude ?? [],
},
Subtags = subtags ?? new AlarmSubtagNameOptions(),
},
};
[Fact]
public async Task ResolveAsync_UnionsGalaxyRowsAndIncludes_RemovesExcludes_AndDeduplicates()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
new GalaxyAlarmAttributeRow { FullTagReference = "Tank02.Level.HiHi", SourceObjectReference = "Tank02" },
// Duplicate of an include below (case-insensitive) — should appear once.
new GalaxyAlarmAttributeRow { FullTagReference = "Pump01.Fault", SourceObjectReference = "Pump01" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
include: ["pump01.fault", "Valve03.Position.Lo"],
exclude: ["Tank02.Level.HiHi"]));
Assert.Equal(
new[] { "Tank01.Level.HiHi", "Pump01.Fault", "Valve03.Position.Lo" },
result.Select(t => t.ActiveSubtag.Replace(".active", string.Empty, StringComparison.Ordinal)));
// De-dup preserved first (GR) occurrence; exclude removed Tank02.
Assert.Equal(3, result.Count);
}
[Fact]
public async Task ResolveAsync_ComposesSubtagAddressesFromConfigNames()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
subtags: new AlarmSubtagNameOptions
{
Active = "InAlarm",
Acked = "Ack",
Priority = "Pri",
AckComment = "AckCmt",
}));
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Tank01.Level.HiHi.InAlarm", target.ActiveSubtag);
Assert.Equal("Tank01.Level.HiHi.Ack", target.AckedSubtag);
Assert.Equal("Tank01.Level.HiHi.Pri", target.PrioritySubtag);
Assert.Equal("Tank01.Level.HiHi.AckCmt", target.AckCommentSubtag);
}
[Fact]
public async Task ResolveAsync_EmptyPriorityAndAckComment_LeaveThoseFieldsEmpty()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
// Default Priority is "priority"; force it empty alongside empty AckComment.
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
subtags: new AlarmSubtagNameOptions { Priority = string.Empty, AckComment = string.Empty }));
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Tank01.Level.HiHi.active", target.ActiveSubtag);
Assert.Equal("Tank01.Level.HiHi.acked", target.AckedSubtag);
Assert.Equal(string.Empty, target.PrioritySubtag);
Assert.Equal(string.Empty, target.AckCommentSubtag);
}
[Fact]
public async Task ResolveAsync_ComposesCanonicalFullReference_WithArea()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(area: "Site_A"));
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Galaxy!Site_A.Tank01.Level.HiHi", target.AlarmFullReference);
}
[Fact]
public async Task ResolveAsync_ComposesCanonicalFullReference_WithoutArea()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
// No discovery area and no default area.
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options());
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Galaxy!Tank01.Level.HiHi", target.AlarmFullReference);
}
[Fact]
public async Task ResolveAsync_FallsBackToDefaultArea_WhenDiscoveryAreaEmpty()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(area: "", defaultArea: "Plant"));
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Galaxy!Plant.Tank01.Level.HiHi", target.AlarmFullReference);
}
[Fact]
public async Task ResolveAsync_UseGalaxyRepositoryFalse_DoesNotCallRepository_UsesIncludesOnly()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "ShouldNotAppear.X", SourceObjectReference = "ShouldNotAppear" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
useGalaxyRepository: false,
include: ["Tank01.Level.HiHi"]));
Assert.Equal(0, repo.GetAlarmAttributesCount);
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Galaxy!Tank01.Level.HiHi", target.AlarmFullReference);
}
[Fact]
public async Task ResolveAsync_RepositoryThrows_LogsAndReturnsConfigOnlySet()
{
ThrowingGalaxyRepository repo = new(new InvalidOperationException("SQL unavailable"));
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
include: ["Tank01.Level.HiHi"]));
// Did not throw; discovery set empty, include retained.
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Galaxy!Tank01.Level.HiHi", target.AlarmFullReference);
}
[Fact]
public async Task ResolveAsync_DerivesSourceObjectForConfigEntry()
{
StubGalaxyRepository repo = new([]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
include: ["Tank01.Level.HiHi", "StandaloneTag"]));
Assert.Equal("Tank01", result[0].SourceObjectReference);
// No dot — whole string is the source object.
Assert.Equal("StandaloneTag", result[1].SourceObjectReference);
}
/// <summary>
/// Fix 1: ExcludeAttributes must be ignored when UseGalaxyRepository is false.
/// A config-only include must survive even when the same path appears in ExcludeAttributes.
/// </summary>
[Fact]
public async Task ResolveAsync_ExcludeIgnored_WhenGalaxyRepositoryDisabled()
{
// Repo is never consulted; only IncludeAttributes matters.
StubGalaxyRepository repo = new([]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
useGalaxyRepository: false,
include: ["Tank01.Level.HiHi"],
exclude: ["Tank01.Level.HiHi"]));
// ExcludeAttributes is ignored when GR is off — the include must be present.
AlarmSubtagTarget target = Assert.Single(result);
Assert.Equal("Galaxy!Tank01.Level.HiHi", target.AlarmFullReference);
}
/// <summary>
/// Fix 1 (GR-on path): ExcludeAttributes still prunes GR rows when
/// UseGalaxyRepository is true.
/// </summary>
[Fact]
public async Task ResolveAsync_ExcludeApplied_WhenGalaxyRepositoryEnabled()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
new GalaxyAlarmAttributeRow { FullTagReference = "Tank02.Level.HiHi", SourceObjectReference = "Tank02" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
useGalaxyRepository: true,
exclude: ["Tank02.Level.HiHi"]));
// Tank02 was excluded; only Tank01 remains.
AlarmSubtagTarget target = Assert.Single(result);
Assert.Contains("Tank01", target.ActiveSubtag, StringComparison.Ordinal);
}
/// <summary>
/// Fix 1: A whitespace-only ExcludeAttributes entry must be skipped and must
/// not accidentally exclude any real reference.
/// </summary>
[Fact]
public async Task ResolveAsync_WhitespaceOnlyExcludeEntry_IsSkipped()
{
StubGalaxyRepository repo = new(
[
new GalaxyAlarmAttributeRow { FullTagReference = "Tank01.Level.HiHi", SourceObjectReference = "Tank01" },
]);
AlarmWatchListResolver resolver = CreateResolver(repo);
// The exclude array contains a whitespace-only string — should be a no-op.
IReadOnlyList<AlarmSubtagTarget> result = await resolver.ResolveAsync(Options(
useGalaxyRepository: true,
exclude: [" "]));
// Tank01 must not have been wrongly excluded.
AlarmSubtagTarget target = Assert.Single(result);
Assert.Contains("Tank01", target.ActiveSubtag, StringComparison.Ordinal);
}
/// <summary>In-memory <see cref="IGalaxyRepository"/> returning a fixed alarm rowset.</summary>
private sealed class StubGalaxyRepository(List<GalaxyAlarmAttributeRow> rows) : IGalaxyRepository
{
/// <summary>Gets the number of times <see cref="GetAlarmAttributesAsync"/> was called.</summary>
public int GetAlarmAttributesCount { get; private set; }
/// <inheritdoc />
public Task<bool> TestConnectionAsync(CancellationToken ct = default) => Task.FromResult(true);
/// <inheritdoc />
public Task<DateTime?> GetLastDeployTimeAsync(CancellationToken ct = default) =>
Task.FromResult<DateTime?>(null);
/// <inheritdoc />
public Task<List<GalaxyHierarchyRow>> GetHierarchyAsync(CancellationToken ct = default) =>
Task.FromResult(new List<GalaxyHierarchyRow>());
/// <inheritdoc />
public Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default) =>
Task.FromResult(new List<GalaxyAttributeRow>());
/// <inheritdoc />
public Task<List<GalaxyAlarmAttributeRow>> GetAlarmAttributesAsync(CancellationToken ct = default)
{
GetAlarmAttributesCount++;
return Task.FromResult(rows);
}
}
/// <summary><see cref="IGalaxyRepository"/> whose alarm-attribute query throws.</summary>
private sealed class ThrowingGalaxyRepository(Exception toThrow) : IGalaxyRepository
{
/// <inheritdoc />
public Task<bool> TestConnectionAsync(CancellationToken ct = default) => Task.FromResult(true);
/// <inheritdoc />
public Task<DateTime?> GetLastDeployTimeAsync(CancellationToken ct = default) =>
Task.FromResult<DateTime?>(null);
/// <inheritdoc />
public Task<List<GalaxyHierarchyRow>> GetHierarchyAsync(CancellationToken ct = default) =>
Task.FromResult(new List<GalaxyHierarchyRow>());
/// <inheritdoc />
public Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default) =>
Task.FromResult(new List<GalaxyAttributeRow>());
/// <inheritdoc />
public Task<List<GalaxyAlarmAttributeRow>> GetAlarmAttributesAsync(CancellationToken ct = default) =>
Task.FromException<List<GalaxyAlarmAttributeRow>>(toThrow);
}
}
@@ -0,0 +1,412 @@
using System.Diagnostics.CodeAnalysis;
using System.Diagnostics.Metrics;
using System.Runtime.CompilerServices;
using System.Threading.Channels;
using Google.Protobuf.WellKnownTypes;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Alarms;
using ZB.MOM.WW.MxGateway.Server.Configuration;
using ZB.MOM.WW.MxGateway.Server.Metrics;
using ZB.MOM.WW.MxGateway.Server.Sessions;
namespace ZB.MOM.WW.MxGateway.Tests.Alarms;
/// <summary>
/// Drives <see cref="GatewayAlarmMonitor"/> with a fake session manager to
/// verify it reflects the worker's <c>OnAlarmProviderModeChanged</c> event into
/// the alarm feed and the switch metric, and that a new subscriber receives the
/// provider status as its first message. Also covers the watch-list / forced-mode
/// wiring of the <c>SubscribeAlarms</c> command and the Mode→enum mapping.
/// </summary>
public sealed class GatewayAlarmMonitorProviderModeTests
{
private static readonly TimeSpan WaitTimeout = TimeSpan.FromSeconds(15);
[Fact]
public async Task ProviderModeChange_BroadcastsDegradedStatus_AndIncrementsSwitchMetric()
{
using GatewayMetrics metrics = new();
long switchCount = 0;
using MeterListener listener = new();
listener.InstrumentPublished = (instrument, meterListener) =>
{
if (ReferenceEquals(instrument.Meter, metrics.Meter)
&& instrument.Name == "mxgateway.alarms.provider_switches")
{
meterListener.EnableMeasurementEvents(instrument);
}
};
listener.SetMeasurementEventCallback<long>(
(instrument, measurement, _, _) =>
{
if (ReferenceEquals(instrument.Meter, metrics.Meter)
&& instrument.Name == "mxgateway.alarms.provider_switches")
{
Interlocked.Add(ref switchCount, measurement);
}
});
listener.Start();
FakeSessionManager sessions = new();
using GatewayAlarmMonitor monitor = CreateMonitor(sessions, metrics);
using CancellationTokenSource cts = new();
await monitor.StartAsync(cts.Token);
await sessions.WaitForSubscribeAsync(WaitTimeout);
// Subscribe a live feed reader. Gate emitting the mode-change event until the
// reader has consumed its baseline ProviderStatus message, avoiding a race where
// the event arrives before the subscriber is registered and draining its snapshot.
List<AlarmFeedMessage> received = [];
TaskCompletionSource baselineReceived = new(TaskCreationOptions.RunContinuationsAsynchronously);
using CancellationTokenSource streamCts = new();
Task reader = Task.Run(async () =>
{
try
{
await foreach (AlarmFeedMessage message in monitor.StreamAsync(null, streamCts.Token))
{
lock (received)
{
received.Add(message);
// Signal once the first message (baseline ProviderStatus) has arrived.
if (received.Count == 1)
{
baselineReceived.TrySetResult();
}
}
}
}
catch (OperationCanceledException)
{
// Expected when the test cancels the stream.
}
});
// Wait for the baseline ProviderStatus to arrive before emitting the mode change,
// so the subscriber is registered and the event is not dropped.
await baselineReceived.Task.WaitAsync(WaitTimeout);
// Emit the worker event that flips the provider into subtag mode.
sessions.EmitEvent(new MxEvent
{
OnAlarmProviderModeChanged = new OnAlarmProviderModeChangedEvent
{
Mode = AlarmProviderMode.Subtag,
Reason = "alarmmgr failed",
Hresult = unchecked((int)0x80004005),
At = Timestamp.FromDateTimeOffset(DateTimeOffset.UtcNow),
},
});
AlarmFeedMessage degraded = await WaitForAsync(
received,
m => m.PayloadCase == AlarmFeedMessage.PayloadOneofCase.ProviderStatus
&& m.ProviderStatus.Mode == AlarmProviderMode.Subtag,
WaitTimeout);
Assert.Equal(AlarmProviderMode.Subtag, degraded.ProviderStatus.Mode);
Assert.True(degraded.ProviderStatus.Degraded);
Assert.Equal("alarmmgr failed", degraded.ProviderStatus.Reason);
await WaitUntilAsync(() => Interlocked.Read(ref switchCount) >= 1, WaitTimeout);
Assert.Equal(1, Interlocked.Read(ref switchCount));
await streamCts.CancelAsync();
await reader;
await cts.CancelAsync();
await monitor.StopAsync(CancellationToken.None);
}
[Fact]
public async Task NewSubscriber_ReceivesProviderStatusAsFirstMessage()
{
using GatewayMetrics metrics = new();
FakeSessionManager sessions = new();
using GatewayAlarmMonitor monitor = CreateMonitor(sessions, metrics);
using CancellationTokenSource cts = new();
await monitor.StartAsync(cts.Token);
await sessions.WaitForSubscribeAsync(WaitTimeout);
using CancellationTokenSource streamCts = new();
AlarmFeedMessage? first = null;
Task reader = Task.Run(async () =>
{
await foreach (AlarmFeedMessage message in monitor.StreamAsync(null, streamCts.Token))
{
first = message;
break;
}
});
await WaitUntilAsync(() => first is not null, WaitTimeout);
Assert.NotNull(first);
Assert.Equal(AlarmFeedMessage.PayloadOneofCase.ProviderStatus, first!.PayloadCase);
// Baseline before any provider-mode event: alarm-manager, not degraded.
Assert.Equal(AlarmProviderMode.Alarmmgr, first.ProviderStatus.Mode);
Assert.False(first.ProviderStatus.Degraded);
await streamCts.CancelAsync();
await reader;
await cts.CancelAsync();
await monitor.StopAsync(CancellationToken.None);
}
[Fact]
public async Task SubscribeAlarms_SendsForcedModeAndWatchList_FromConfiguration()
{
using GatewayMetrics metrics = new();
FakeSessionManager sessions = new();
StubWatchListResolver resolver = new(
[
new AlarmSubtagTarget { AlarmFullReference = "Galaxy!Area.Tank01.Hi", ActiveSubtag = "Tank01.Hi.active" },
]);
AlarmsOptions options = new()
{
Enabled = true,
SubscriptionExpression = @"\\NODE\Galaxy!Area",
Fallback = new AlarmFallbackOptions
{
Mode = "ForceSubtag",
ConsecutiveFailureThreshold = 7,
FailbackProbeIntervalSeconds = 11,
FailbackStableProbes = 4,
},
};
using GatewayAlarmMonitor monitor = new(
sessions,
resolver,
metrics,
Microsoft.Extensions.Options.Options.Create(new GatewayOptions { Alarms = options }),
NullLogger<GatewayAlarmMonitor>.Instance);
using CancellationTokenSource cts = new();
await monitor.StartAsync(cts.Token);
await sessions.WaitForSubscribeAsync(WaitTimeout);
SubscribeAlarmsCommand sent = Assert.IsType<SubscribeAlarmsCommand>(sessions.LastSubscribeCommand);
Assert.Equal(AlarmProviderMode.Subtag, sent.ForcedMode);
Assert.Equal(7, sent.Failover.ConsecutiveFailureThreshold);
Assert.Equal(11, sent.Failover.FailbackProbeIntervalSeconds);
Assert.Equal(4, sent.Failover.FailbackStableProbes);
AlarmSubtagTarget target = Assert.Single(sent.WatchList);
Assert.Equal("Galaxy!Area.Tank01.Hi", target.AlarmFullReference);
await cts.CancelAsync();
await monitor.StopAsync(CancellationToken.None);
}
[Theory]
[InlineData("ForceAlarmManager", AlarmProviderMode.Alarmmgr)]
[InlineData("forcealarmmanager", AlarmProviderMode.Alarmmgr)]
[InlineData("ForceSubtag", AlarmProviderMode.Subtag)]
[InlineData("forcesubtag", AlarmProviderMode.Subtag)]
[InlineData("Auto", AlarmProviderMode.Unspecified)]
[InlineData("", AlarmProviderMode.Unspecified)]
[InlineData("nonsense", AlarmProviderMode.Unspecified)]
public async Task ModeString_MapsToForcedProviderMode(string mode, AlarmProviderMode expected)
{
using GatewayMetrics metrics = new();
FakeSessionManager sessions = new();
AlarmsOptions options = new()
{
Enabled = true,
SubscriptionExpression = @"\\NODE\Galaxy!Area",
Fallback = new AlarmFallbackOptions { Mode = mode },
};
using GatewayAlarmMonitor monitor = new(
sessions,
new StubWatchListResolver([]),
metrics,
Microsoft.Extensions.Options.Options.Create(new GatewayOptions { Alarms = options }),
NullLogger<GatewayAlarmMonitor>.Instance);
using CancellationTokenSource cts = new();
await monitor.StartAsync(cts.Token);
await sessions.WaitForSubscribeAsync(WaitTimeout);
Assert.Equal(expected, sessions.LastSubscribeCommand!.ForcedMode);
// Auto + empty watch-list preserves historical alarmmgr-only behaviour.
if (expected == AlarmProviderMode.Unspecified)
{
Assert.Empty(sessions.LastSubscribeCommand!.WatchList);
}
await cts.CancelAsync();
await monitor.StopAsync(CancellationToken.None);
}
private static GatewayAlarmMonitor CreateMonitor(FakeSessionManager sessions, GatewayMetrics metrics)
{
AlarmsOptions options = new()
{
Enabled = true,
SubscriptionExpression = @"\\NODE\Galaxy!Area",
};
return new GatewayAlarmMonitor(
sessions,
new StubWatchListResolver([]),
metrics,
Microsoft.Extensions.Options.Options.Create(new GatewayOptions { Alarms = options }),
NullLogger<GatewayAlarmMonitor>.Instance);
}
private static async Task<AlarmFeedMessage> WaitForAsync(
List<AlarmFeedMessage> received,
Func<AlarmFeedMessage, bool> predicate,
TimeSpan timeout)
{
DateTime deadline = DateTime.UtcNow + timeout;
while (DateTime.UtcNow < deadline)
{
lock (received)
{
AlarmFeedMessage? match = received.FirstOrDefault(predicate);
if (match is not null)
{
return match;
}
}
await Task.Delay(25);
}
throw new TimeoutException("No matching AlarmFeedMessage was received in time.");
}
private static async Task WaitUntilAsync(Func<bool> condition, TimeSpan timeout)
{
DateTime deadline = DateTime.UtcNow + timeout;
while (DateTime.UtcNow < deadline)
{
if (condition())
{
return;
}
await Task.Delay(25);
}
throw new TimeoutException("Condition was not met in time.");
}
/// <summary><see cref="IAlarmWatchListResolver"/> that returns a fixed watch-list.</summary>
private sealed class StubWatchListResolver(IReadOnlyList<AlarmSubtagTarget> targets) : IAlarmWatchListResolver
{
/// <inheritdoc />
public Task<IReadOnlyList<AlarmSubtagTarget>> ResolveAsync(
AlarmsOptions options,
CancellationToken cancellationToken = default) => Task.FromResult(targets);
}
/// <summary>
/// Minimal <see cref="ISessionManager"/> for driving the monitor: opens a
/// constructed session, records the SubscribeAlarms command, replies OK to
/// every command, and exposes a channel for pushing worker events.
/// </summary>
private sealed class FakeSessionManager : ISessionManager
{
private readonly Channel<WorkerEvent> _events = Channel.CreateUnbounded<WorkerEvent>();
private readonly TaskCompletionSource _subscribed =
new(TaskCreationOptions.RunContinuationsAsynchronously);
/// <summary>The most recent SubscribeAlarms command the monitor sent.</summary>
public SubscribeAlarmsCommand? LastSubscribeCommand { get; private set; }
/// <summary>Pushes a worker event onto the monitor's event stream.</summary>
public void EmitEvent(MxEvent mxEvent) =>
_events.Writer.TryWrite(new WorkerEvent { Event = mxEvent });
/// <summary>Completes once the monitor has issued its SubscribeAlarms command.</summary>
public Task WaitForSubscribeAsync(TimeSpan timeout) => _subscribed.Task.WaitAsync(timeout);
/// <inheritdoc />
public Task<GatewaySession> OpenSessionAsync(
SessionOpenRequest request,
string? clientIdentity,
CancellationToken cancellationToken)
{
GatewaySession session = new(
Guid.NewGuid().ToString("N"),
"Galaxy",
"pipe-test",
"nonce-test",
clientIdentity,
null,
null,
TimeSpan.FromSeconds(30),
TimeSpan.FromSeconds(30),
TimeSpan.FromSeconds(30),
DateTimeOffset.UtcNow);
return Task.FromResult(session);
}
/// <inheritdoc />
public Task<WorkerCommandReply> InvokeAsync(
string sessionId,
WorkerCommand command,
CancellationToken cancellationToken)
{
if (command.Command?.Kind == MxCommandKind.SubscribeAlarms)
{
LastSubscribeCommand = command.Command.SubscribeAlarms;
_subscribed.TrySetResult();
}
MxCommandReply reply = new()
{
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
};
if (command.Command?.Kind == MxCommandKind.QueryActiveAlarms)
{
reply.QueryActiveAlarms = new QueryActiveAlarmsReplyPayload();
}
return Task.FromResult(new WorkerCommandReply { Reply = reply });
}
/// <inheritdoc />
public async IAsyncEnumerable<WorkerEvent> ReadEventsAsync(
string sessionId,
[EnumeratorCancellation] CancellationToken cancellationToken)
{
await foreach (WorkerEvent workerEvent in _events.Reader.ReadAllAsync(cancellationToken))
{
yield return workerEvent;
}
}
/// <inheritdoc />
public bool TryGetSession(string sessionId, [MaybeNullWhen(false)] out GatewaySession session)
{
session = null;
return false;
}
/// <inheritdoc />
public Task<SessionCloseResult> CloseSessionAsync(string sessionId, CancellationToken cancellationToken)
{
_events.Writer.TryComplete();
return Task.FromResult(new SessionCloseResult(sessionId, SessionState.Closed, AlreadyClosed: false));
}
/// <inheritdoc />
public Task<SessionCloseResult> KillWorkerAsync(string sessionId, string reason, CancellationToken cancellationToken) =>
Task.FromResult(new SessionCloseResult(sessionId, SessionState.Closed, AlreadyClosed: false));
/// <inheritdoc />
public Task<int> CloseExpiredLeasesAsync(DateTimeOffset now, CancellationToken cancellationToken) =>
Task.FromResult(0);
/// <inheritdoc />
public Task ShutdownAsync(CancellationToken cancellationToken) => Task.CompletedTask;
}
}
@@ -118,4 +118,175 @@ public sealed class GatewayOptionsValidatorTests
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Succeeded);
}
// -------------------------------------------------------------------------
// AlarmFallbackOptions validation
// -------------------------------------------------------------------------
private static AlarmsOptions EnabledAlarmsWithFallback(AlarmFallbackOptions fallback) => new()
{
Enabled = true,
DefaultArea = "Galaxy",
Fallback = fallback,
};
private static GatewayOptions CloneWithAlarms(GatewayOptions source, AlarmsOptions alarms)
=> new()
{
Authentication = source.Authentication,
Ldap = source.Ldap,
Worker = source.Worker,
Sessions = source.Sessions,
Events = source.Events,
Dashboard = source.Dashboard,
Protocol = source.Protocol,
Alarms = alarms,
Tls = source.Tls,
};
[Fact]
public void Validate_Succeeds_WhenAlarmsDisabled_FallbackNotValidated()
{
// Even an invalid Mode is acceptable when Enabled = false.
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
new AlarmsOptions
{
Enabled = false,
Fallback = new AlarmFallbackOptions { Mode = "InvalidMode" },
});
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Succeeded);
}
[Fact]
public void Validate_Succeeds_WhenAlarmsEnabled_DefaultAutoConfig()
{
// Default AlarmFallbackOptions (Mode="Auto") must pass validation when alarms are enabled.
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions()));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Succeeded);
}
[Theory]
[InlineData("Auto")]
[InlineData("ForceAlarmManager")]
[InlineData("ForceSubtag")]
[InlineData("auto")]
[InlineData("FORCESUBTAG")]
public void Validate_Succeeds_WhenAlarmsEnabled_RecognisedMode(string mode)
{
AlarmsOptions alarms = EnabledAlarmsWithFallback(new AlarmFallbackOptions { Mode = mode });
GatewayOptions options = CloneWithAlarms(ValidOptions(), alarms);
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Succeeded);
}
[Fact]
public void Validate_Fails_WhenAlarmsEnabled_InvalidMode()
{
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions { Mode = "InvalidMode" }));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Failed);
Assert.Contains(result.Failures!, f => f.Contains("MxGateway:Alarms:Fallback") && f.Contains("Mode"));
}
[Fact]
public void Validate_Fails_WhenForceSubtag_NoGalaxyRepository_NoIncludes()
{
// ForceSubtag without galaxy repository and without IncludeAttributes must fail.
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions
{
Mode = "ForceSubtag",
Discovery = new AlarmDiscoveryOptions
{
UseGalaxyRepository = false,
IncludeAttributes = [],
},
}));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Failed);
Assert.Contains(
result.Failures!,
f => f.Contains("ForceSubtag") && f.Contains("Discovery"));
}
[Fact]
public void Validate_Succeeds_WhenForceSubtag_NoGalaxyRepository_WithIncludes()
{
// ForceSubtag without galaxy repository is allowed when IncludeAttributes is non-empty.
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions
{
Mode = "ForceSubtag",
Discovery = new AlarmDiscoveryOptions
{
UseGalaxyRepository = false,
IncludeAttributes = ["attr1"],
},
}));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Succeeded);
}
[Fact]
public void Validate_Succeeds_WhenForceSubtag_WithGalaxyRepository()
{
// ForceSubtag + UseGalaxyRepository=true (default) must pass even without IncludeAttributes.
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions
{
Mode = "ForceSubtag",
Discovery = new AlarmDiscoveryOptions { UseGalaxyRepository = true },
}));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Succeeded);
}
[Theory]
[InlineData(0, nameof(AlarmFallbackOptions.ConsecutiveFailureThreshold))]
[InlineData(-1, nameof(AlarmFallbackOptions.ConsecutiveFailureThreshold))]
public void Validate_Fails_WhenConsecutiveFailureThresholdBelowOne(int value, string keyPart)
{
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions { ConsecutiveFailureThreshold = value }));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Failed);
Assert.Contains(result.Failures!, f => f.Contains(keyPart));
}
[Theory]
[InlineData(0, nameof(AlarmFallbackOptions.FailbackProbeIntervalSeconds))]
[InlineData(-5, nameof(AlarmFallbackOptions.FailbackProbeIntervalSeconds))]
public void Validate_Fails_WhenFailbackProbeIntervalSecondsBelowOne(int value, string keyPart)
{
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions { FailbackProbeIntervalSeconds = value }));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Failed);
Assert.Contains(result.Failures!, f => f.Contains(keyPart));
}
[Theory]
[InlineData(0, nameof(AlarmFallbackOptions.FailbackStableProbes))]
[InlineData(-1, nameof(AlarmFallbackOptions.FailbackStableProbes))]
public void Validate_Fails_WhenFailbackStableProbesBelowOne(int value, string keyPart)
{
GatewayOptions options = CloneWithAlarms(
ValidOptions(),
EnabledAlarmsWithFallback(new AlarmFallbackOptions { FailbackStableProbes = value }));
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Failed);
Assert.Contains(result.Failures!, f => f.Contains(keyPart));
}
}
@@ -1332,6 +1332,56 @@ public sealed class ProtobufContractRoundTripTests
Assert.Equal(kind, parsed.Kind);
}
/// <summary>
/// Verifies that an <see cref="AlarmFeedMessage"/> carrying the
/// <c>provider_status</c> payload case round-trips and resolves to
/// <see cref="AlarmFeedMessage.PayloadOneofCase.ProviderStatus"/>.
/// </summary>
[Fact]
public void Feed_RoundTripsProviderStatus()
{
var since = Timestamp.FromDateTime(new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc));
var original = new AlarmFeedMessage
{
ProviderStatus = new AlarmProviderStatus
{
Mode = AlarmProviderMode.Subtag,
Degraded = true,
Reason = "wnwrap poll failed 3x (HRESULT 0x80004005)",
Since = since,
},
};
var parsed = AlarmFeedMessage.Parser.ParseFrom(original.ToByteArray());
Assert.Equal(original, parsed);
Assert.Equal(AlarmFeedMessage.PayloadOneofCase.ProviderStatus, parsed.PayloadCase);
Assert.True(parsed.ProviderStatus.Degraded);
Assert.Equal(AlarmProviderMode.Subtag, parsed.ProviderStatus.Mode);
}
/// <summary>
/// Verifies that an <see cref="OnAlarmTransitionEvent"/> carrying the
/// new <c>degraded</c> and <c>source_provider</c> provenance fields
/// round-trips with their values preserved.
/// </summary>
[Fact]
public void Transition_RoundTripsDegradedProvenance()
{
var t = new OnAlarmTransitionEvent
{
AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi",
TransitionKind = AlarmTransitionKind.Raise,
Degraded = true,
SourceProvider = AlarmProviderMode.Subtag,
};
var parsed = OnAlarmTransitionEvent.Parser.ParseFrom(t.ToByteArray());
Assert.True(parsed.Degraded);
Assert.Equal(AlarmProviderMode.Subtag, parsed.SourceProvider);
}
/// <summary>
/// Verifies that an <see cref="MxCommandReply"/> with kind
/// <see cref="MxCommandKind.ReadBulk"/> and a populated
@@ -0,0 +1,64 @@
using ZB.MOM.WW.MxGateway.Server.Galaxy;
namespace ZB.MOM.WW.MxGateway.Tests.Galaxy;
/// <summary>
/// Pure mapper tests for <see cref="GalaxyRepository.MapAlarmRow"/>. These assert the
/// FullTagReference / SourceObjectReference derivation produced by
/// <c>AlarmAttributesSql</c> without touching a database: the SQL projects
/// <c>tag_name</c> as the source object and <c>tag_name + '.' + attribute_name</c> as
/// the full reference, exactly as <c>AttributesSql</c> does.
/// </summary>
public sealed class GalaxyAlarmAttributeMappingTests
{
/// <summary>Verifies the mapper copies both projected columns onto the row.</summary>
[Fact]
public void MapAlarmRow_CopiesProjectedColumns()
{
GalaxyAlarmAttributeRow row = GalaxyRepository.MapAlarmRow(
fullTagReference: "Tank01.Level.HiHi",
sourceObjectReference: "Tank01");
Assert.Equal("Tank01.Level.HiHi", row.FullTagReference);
Assert.Equal("Tank01", row.SourceObjectReference);
}
/// <summary>
/// Verifies <see cref="GalaxyAlarmAttributeRow.AckCommentSubtag"/> is always empty:
/// the schema does not expose an ack-comment address, so the watch-list resolver
/// composes it later from configuration.
/// </summary>
[Fact]
public void MapAlarmRow_LeavesAckCommentSubtagEmpty()
{
GalaxyAlarmAttributeRow row = GalaxyRepository.MapAlarmRow(
fullTagReference: "Tank01.Level.HiHi",
sourceObjectReference: "Tank01");
Assert.Equal(string.Empty, row.AckCommentSubtag);
}
/// <summary>
/// Verifies the SourceObjectReference is the owning object (the SQL <c>tag_name</c>),
/// i.e. the segment that precedes the first attribute dot in the full reference, even
/// when the attribute itself is a multi-segment extension path.
/// </summary>
[Theory]
[InlineData("Tank01", "Level.HiHi", "Tank01.Level.HiHi")]
[InlineData("Pump_001", "Speed", "Pump_001.Speed")]
[InlineData("TestAlarm001", "Alarm001", "TestAlarm001.Alarm001")]
public void MapAlarmRow_SourceObjectIsSegmentBeforeFirstAttributeDot(
string tagName,
string attributeName,
string expectedFullReference)
{
// Mirror the AlarmAttributesSql projection: full_tag_reference = tag_name + '.' + attribute_name.
string fullTagReference = tagName + "." + attributeName;
GalaxyAlarmAttributeRow row = GalaxyRepository.MapAlarmRow(fullTagReference, tagName);
Assert.Equal(expectedFullReference, row.FullTagReference);
Assert.Equal(tagName, row.SourceObjectReference);
Assert.Equal(row.FullTagReference, row.SourceObjectReference + "." + attributeName);
}
}
@@ -378,6 +378,10 @@ public sealed class GalaxyHierarchyCacheTests : IDisposable
/// <inheritdoc />
public Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default)
=> throw new InvalidOperationException("GetAttributesAsync should not be reached");
/// <inheritdoc />
public Task<List<GalaxyAlarmAttributeRow>> GetAlarmAttributesAsync(CancellationToken ct = default)
=> throw new InvalidOperationException("GetAlarmAttributesAsync should not be reached");
}
/// <summary>Snapshot store whose <see cref="SaveAsync"/> cancels the token mid-save.</summary>
@@ -465,6 +469,10 @@ public sealed class GalaxyHierarchyCacheTests : IDisposable
GetAttributesCount++;
return Task.FromResult(_attributes);
}
/// <inheritdoc />
public Task<List<GalaxyAlarmAttributeRow>> GetAlarmAttributesAsync(CancellationToken ct = default)
=> Task.FromResult(new List<GalaxyAlarmAttributeRow>());
}
/// <inheritdoc />
@@ -518,6 +526,10 @@ public sealed class GalaxyHierarchyCacheTests : IDisposable
GetAttributesCount++;
throw toThrow;
}
/// <inheritdoc />
public Task<List<GalaxyAlarmAttributeRow>> GetAlarmAttributesAsync(CancellationToken ct = default)
=> throw toThrow;
}
}
@@ -137,6 +137,41 @@ public sealed class DashboardBrowseAndAlarmModelTests
Assert.False(ackedRow.IsUnacknowledged);
}
/// <summary>Verifies that a healthy alarmmgr provider status maps to a green badge.</summary>
[Fact]
public void FromProviderStatus_Alarmmgr_NotDegraded_GreenBadge()
{
AlarmProviderStatus status = new()
{
Mode = AlarmProviderMode.Alarmmgr,
Degraded = false,
};
DashboardAlarmProviderStatus model = DashboardAlarmProviderStatus.FromProviderStatus(status);
Assert.False(model.IsDegraded);
Assert.Contains("bg-success", model.BadgeCssClass, StringComparison.Ordinal);
Assert.Equal(DashboardAlarmProviderStatus.AlarmManagerLabel, model.Label);
}
/// <summary>Verifies that a degraded subtag provider status maps to an amber warning badge.</summary>
[Fact]
public void FromProviderStatus_Subtag_Degraded_WarningBadge()
{
AlarmProviderStatus status = new()
{
Mode = AlarmProviderMode.Subtag,
Degraded = true,
Reason = "x",
};
DashboardAlarmProviderStatus model = DashboardAlarmProviderStatus.FromProviderStatus(status);
Assert.True(model.IsDegraded);
Assert.Contains("bg-warning", model.BadgeCssClass, StringComparison.Ordinal);
Assert.Equal("x", model.Reason);
}
/// <summary>Verifies that the formatter renders array elements and element type correctly.</summary>
[Fact]
public void FormatValue_AndDataType_RenderArrayElementsAndElementType()
@@ -87,13 +87,18 @@ public sealed class GatewayApplicationTests
Assert.Contains(endpoints, endpoint => endpoint.RoutePattern.RawText == "/events");
Assert.Contains(endpoints, endpoint => endpoint.RoutePattern.RawText == "/settings");
// GET /login is now served by the [AllowAnonymous] Blazor <Login> component
// GET /login is served by the [AllowAnonymous] Blazor <Login> component
// (Components/Pages/Login.razor → @page "/login"), not a named minimal-API
// endpoint. The form still POSTs to the minimal-API DashboardLoginPost endpoint.
// endpoint. The credential POST goes to the DashboardLoginPost endpoint at
// /auth/login — a DISTINCT route. The Blazor component endpoint matches all HTTP
// methods, so sharing the "/login" route with MapPost previously made POST /login
// ambiguous (AmbiguousMatchException → HTTP 500). Pinning the POST to /auth/login
// is the regression guard for that fix.
Assert.Contains(endpoints, endpoint => endpoint.RoutePattern.RawText == "/login"
&& endpoint.Metadata.GetMetadata<Microsoft.AspNetCore.Components.Endpoints.ComponentTypeMetadata>() is not null);
Assert.Contains(endpoints, endpoint =>
endpoint.Metadata.GetMetadata<IEndpointNameMetadata>()?.EndpointName == "DashboardLoginPost");
endpoint.Metadata.GetMetadata<IEndpointNameMetadata>()?.EndpointName == "DashboardLoginPost"
&& endpoint.RoutePattern.RawText == "/auth/login");
Assert.Contains(endpoints, endpoint =>
endpoint.Metadata.GetMetadata<IEndpointNameMetadata>()?.EndpointName == "DashboardLogout");
}
@@ -1,3 +1,4 @@
using System.Diagnostics.Metrics;
using ZB.MOM.WW.MxGateway.Server.Metrics;
namespace ZB.MOM.WW.MxGateway.Tests.Metrics;
@@ -63,6 +64,98 @@ public sealed class GatewayMetricsTests
Assert.Equal("depth", exception.ParamName);
}
/// <summary>
/// Verifies that <see cref="GatewayMetrics.AlarmProviderSwitched"/> increments
/// <c>mxgateway.alarms.provider_switches</c> by one with the expected from/to/reason tags.
/// The listener filters by the specific <see cref="System.Diagnostics.Metrics.Meter"/> instance
/// to avoid cross-talk between parallel tests (Tests-027).
/// </summary>
[Fact]
public void AlarmProviderSwitched_IncrementsCounterWithExpectedTags()
{
using GatewayMetrics metrics = new();
using MeterListener listener = new();
long capturedValue = 0;
string? capturedFrom = null;
string? capturedTo = null;
string? capturedReason = null;
listener.InstrumentPublished = (instrument, meterListener) =>
{
if (ReferenceEquals(instrument.Meter, metrics.Meter)
&& instrument.Name == "mxgateway.alarms.provider_switches")
{
meterListener.EnableMeasurementEvents(instrument);
}
};
listener.SetMeasurementEventCallback<long>(
(instrument, measurement, tags, _) =>
{
if (!ReferenceEquals(instrument.Meter, metrics.Meter)
|| instrument.Name != "mxgateway.alarms.provider_switches")
{
return;
}
capturedValue += measurement;
foreach (KeyValuePair<string, object?> tag in tags)
{
switch (tag.Key)
{
case "from": capturedFrom = tag.Value as string; break;
case "to": capturedTo = tag.Value as string; break;
case "reason": capturedReason = tag.Value as string; break;
}
}
});
listener.Start();
metrics.AlarmProviderSwitched(1, 2, "test");
Assert.Equal(1, capturedValue);
Assert.Equal("1", capturedFrom);
Assert.Equal("2", capturedTo);
Assert.Equal("test", capturedReason);
}
/// <summary>
/// Verifies that <see cref="GatewayMetrics.AlarmProviderSwitched"/> updates the
/// <c>mxgateway.alarms.provider_mode</c> observable gauge to the <paramref name="toMode"/> value.
/// </summary>
[Fact]
public void AlarmProviderSwitched_UpdatesProviderModeGauge()
{
using GatewayMetrics metrics = new();
using MeterListener listener = new();
int? capturedMode = null;
listener.InstrumentPublished = (instrument, meterListener) =>
{
if (ReferenceEquals(instrument.Meter, metrics.Meter)
&& instrument.Name == "mxgateway.alarms.provider_mode")
{
meterListener.EnableMeasurementEvents(instrument);
}
};
listener.SetMeasurementEventCallback<int>(
(instrument, measurement, _, _) =>
{
if (ReferenceEquals(instrument.Meter, metrics.Meter)
&& instrument.Name == "mxgateway.alarms.provider_mode")
{
capturedMode = measurement;
}
});
listener.Start();
metrics.AlarmProviderSwitched(1, 2, "test");
listener.RecordObservableInstruments();
Assert.Equal(2, capturedMode);
}
/// <summary>Verifies that removing session events only affects that session.</summary>
[Fact]
public void RemoveSessionEvents_RemovesOnlyThatSession()
@@ -372,11 +372,11 @@ public sealed class AlarmCommandExecutorTests
public string? LastFilterPrefix { get; private set; }
/// <summary>Records a subscription.</summary>
/// <param name="subscription">The subscription expression.</param>
/// <param name="command">The subscribe-alarms command.</param>
/// <param name="sessionId">The session identifier.</param>
public void Subscribe(string subscription, string sessionId)
public void Subscribe(SubscribeAlarmsCommand command, string sessionId)
{
LastSubscription = subscription;
LastSubscription = command.SubscriptionExpression;
LastSessionId = sessionId;
}
@@ -21,7 +21,7 @@ public sealed class AlarmCommandHandlerTests
new MxAccessEventQueue(),
() => consumer);
handler.Subscribe(@"\\HOST\Galaxy!Area", "session-1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!Area" }, "session-1");
Assert.True(handler.IsSubscribed);
Assert.Equal(@"\\HOST\Galaxy!Area", consumer.LastSubscription);
@@ -36,9 +36,9 @@ public sealed class AlarmCommandHandlerTests
new MxAccessEventQueue(),
() => consumer);
handler.Subscribe(@"\\HOST\Galaxy!A", "s1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
Assert.Throws<InvalidOperationException>(
() => handler.Subscribe(@"\\HOST\Galaxy!B", "s1"));
() => handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!B" }, "s1"));
}
/// <summary>
@@ -63,7 +63,7 @@ public sealed class AlarmCommandHandlerTests
() => consumer);
InvalidOperationException exception = Assert.Throws<InvalidOperationException>(
() => handler.Subscribe(@"\\HOST\Galaxy!A", "s1"));
() => handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1"));
Assert.Contains("simulated wnwrap subscribe failure", exception.Message);
Assert.False(handler.IsSubscribed);
Assert.True(consumer.Disposed);
@@ -77,7 +77,7 @@ public sealed class AlarmCommandHandlerTests
AlarmCommandHandler handler = new AlarmCommandHandler(
new MxAccessEventQueue(),
() => consumer);
handler.Subscribe(@"\\HOST\Galaxy!A", "s1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
handler.Unsubscribe();
@@ -104,7 +104,7 @@ public sealed class AlarmCommandHandlerTests
AlarmCommandHandler handler = new AlarmCommandHandler(
new MxAccessEventQueue(),
() => consumer);
handler.Subscribe(@"\\HOST\Galaxy!A", "s1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
Guid g = Guid.NewGuid();
int rc = handler.Acknowledge(g, "c", "u", "n", "d", "F");
@@ -149,7 +149,7 @@ public sealed class AlarmCommandHandlerTests
AlarmCommandHandler handler = new AlarmCommandHandler(
new MxAccessEventQueue(),
() => consumer);
handler.Subscribe(@"\\HOST\Galaxy!A", "s1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
IReadOnlyList<ActiveAlarmSnapshot> snapshots = handler.QueryActive(null);
@@ -173,7 +173,7 @@ public sealed class AlarmCommandHandlerTests
AlarmCommandHandler handler = new AlarmCommandHandler(
new MxAccessEventQueue(),
() => consumer);
handler.Subscribe(@"\\HOST\Galaxy!A", "s1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
IReadOnlyList<ActiveAlarmSnapshot> filtered = handler.QueryActive("Galaxy!AreaA");
@@ -189,13 +189,13 @@ public sealed class AlarmCommandHandlerTests
AlarmCommandHandler handler = new AlarmCommandHandler(
new MxAccessEventQueue(),
() => consumer);
handler.Subscribe(@"\\HOST\Galaxy!A", "s1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
handler.Dispose();
Assert.True(consumer.Disposed);
Assert.Throws<ObjectDisposedException>(
() => handler.Subscribe("x", "y"));
() => handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = "x" }, "y"));
}
/// <summary>
@@ -218,7 +218,7 @@ public sealed class AlarmCommandHandlerTests
// factory is invoked. We tally invocation counts after each call so
// that a missed guard surfaces as the diagnostic count, not a generic
// "Subscribe should have failed".
handler.Subscribe(@"\\HOST\Galaxy!A", "s1");
handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
Assert.Equal(1, guardInvocations);
handler.Acknowledge(Guid.NewGuid(), "c", "u", "n", "d", "F");
@@ -254,7 +254,7 @@ public sealed class AlarmCommandHandlerTests
// Subscribe: guard runs before the dispatcher is constructed.
Assert.Throws<InvalidOperationException>(
() => handler.Subscribe(@"\\HOST\Galaxy!A", "s1"));
() => handler.Subscribe(new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1"));
// To exercise the other entry points we need a subscribed handler.
// Construct a parallel handler with a passing guard, then swap in a
@@ -273,6 +273,132 @@ public sealed class AlarmCommandHandlerTests
Assert.Throws<InvalidOperationException>(() => handler.Unsubscribe());
}
/// <summary>
/// Worker-9: ForcedMode=Subtag builds a subtag consumer (via the
/// injected standby factory) and advises it — the primary
/// (alarmmgr) consumer is NOT created.
/// </summary>
[Fact]
public void Subscribe_WithForcedSubtagMode_BuildsStandbyConsumerOnly()
{
FakeConsumer primary = new FakeConsumer();
FakeConsumer standby = new FakeConsumer();
IReadOnlyList<AlarmSubtagTarget>? capturedWatchList = null;
AlarmCommandHandler handler = new AlarmCommandHandler(
new MxAccessEventQueue(),
() => primary,
threadAffinityCheck: null,
comFactory: null,
standbyFactory: watch =>
{
capturedWatchList = watch;
return standby;
});
SubscribeAlarmsCommand command = new SubscribeAlarmsCommand
{
SubscriptionExpression = @"\\HOST\Galaxy!Area",
ForcedMode = AlarmProviderMode.Subtag,
};
command.WatchList.Add(new AlarmSubtagTarget { AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi" });
handler.Subscribe(command, "s1");
Assert.True(handler.IsSubscribed);
Assert.Equal(@"\\HOST\Galaxy!Area", standby.LastSubscription); // standby advised
Assert.Null(primary.LastSubscription); // primary never built
Assert.NotNull(capturedWatchList);
Assert.Single(capturedWatchList!);
}
/// <summary>
/// Worker-9: ForcedMode=Unspecified + a non-empty watch list builds a
/// failover composite (primary + subtag standby). Forcing the primary
/// to fail on subscribe with a threshold of 1 drives the composite to
/// switch to the subtag provider, which must enqueue an
/// OnAlarmProviderModeChanged event carrying mode=Subtag.
/// </summary>
[Fact]
public void Subscribe_AutoModeWithWatchList_FailoverModeChange_EnqueuesProviderModeChangedEvent()
{
FakeConsumer primary = new FakeConsumer { ThrowOnSubscribe = true };
FakeConsumer standby = new FakeConsumer();
MxAccessEventQueue queue = new MxAccessEventQueue();
AlarmCommandHandler handler = new AlarmCommandHandler(
queue,
() => primary,
threadAffinityCheck: null,
comFactory: null,
standbyFactory: _ => standby);
SubscribeAlarmsCommand command = new SubscribeAlarmsCommand
{
SubscriptionExpression = @"\\HOST\Galaxy!Area",
ForcedMode = AlarmProviderMode.Unspecified,
Failover = new AlarmFailoverConfig
{
ConsecutiveFailureThreshold = 1,
FailbackProbeIntervalSeconds = 1,
FailbackStableProbes = 1,
},
};
command.WatchList.Add(new AlarmSubtagTarget { AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi" });
// Subscribe: standby is armed cleanly; the primary subscribe throws and,
// at threshold 1, the failover composite switches to standby and raises
// ProviderModeChanged. The handler enqueues the proto event.
handler.Subscribe(command, "s1");
IReadOnlyList<WorkerEvent> drained = queue.Drain(0);
Assert.Single(drained);
MxEvent evt = drained[0].Event;
Assert.Equal(MxEventFamily.OnAlarmProviderModeChanged, evt.Family);
Assert.Equal("s1", evt.SessionId);
Assert.NotNull(evt.OnAlarmProviderModeChanged);
Assert.Equal(AlarmProviderMode.Subtag, evt.OnAlarmProviderModeChanged.Mode);
}
/// <summary>
/// Worker-9: a non-failover subscribe (alarmmgr-only) never enqueues a
/// provider-mode-changed event, and a subsequent Unsubscribe detaches
/// the handler so no event leaks.
/// </summary>
[Fact]
public void Subscribe_AlarmmgrOnly_DoesNotEnqueueProviderModeChangedEvent()
{
FakeConsumer consumer = new FakeConsumer();
MxAccessEventQueue queue = new MxAccessEventQueue();
AlarmCommandHandler handler = new AlarmCommandHandler(queue, () => consumer);
handler.Subscribe(
new SubscribeAlarmsCommand { SubscriptionExpression = @"\\HOST\Galaxy!A" }, "s1");
handler.Unsubscribe();
Assert.Empty(queue.Drain(0));
}
/// <summary>
/// Worker-9: the mapper builds a well-formed OnAlarmProviderModeChanged
/// MxEvent — correct family and populated body fields.
/// </summary>
[Fact]
public void Mapper_CreateOnAlarmProviderModeChanged_PopulatesFamilyAndBody()
{
MxAccessEventMapper mapper = new MxAccessEventMapper();
DateTime at = new DateTime(2026, 6, 13, 10, 0, 0, DateTimeKind.Utc);
MxEvent evt = mapper.CreateOnAlarmProviderModeChanged(
"session-7", AlarmProviderMode.Subtag, "primary PollOnce failed", unchecked((int)0x80004005), at);
Assert.Equal(MxEventFamily.OnAlarmProviderModeChanged, evt.Family);
Assert.Equal("session-7", evt.SessionId);
Assert.NotNull(evt.OnAlarmProviderModeChanged);
Assert.Equal(AlarmProviderMode.Subtag, evt.OnAlarmProviderModeChanged.Mode);
Assert.Equal("primary PollOnce failed", evt.OnAlarmProviderModeChanged.Reason);
Assert.Equal(unchecked((int)0x80004005), evt.OnAlarmProviderModeChanged.Hresult);
Assert.Equal(at, evt.OnAlarmProviderModeChanged.At.ToDateTime());
}
private static MxAlarmSnapshotRecord NewRecord(string provider, string group, string tag)
{
return new MxAlarmSnapshotRecord
@@ -62,6 +62,43 @@ public sealed class AlarmDispatcherTests
Assert.Equal("TestArea", body.Category);
Assert.NotNull(body.TransitionTimestamp);
Assert.Equal(ts, body.TransitionTimestamp.ToDateTime());
Assert.False(body.Degraded);
Assert.Equal(AlarmProviderMode.Alarmmgr, body.SourceProvider);
}
/// <summary>
/// Verifies that a transition enqueued via the subtag fallback
/// (<c>degraded: true</c>) is marked <see cref="OnAlarmTransitionEvent.Degraded"/>
/// with <see cref="AlarmProviderMode.Subtag"/>, while the default path
/// stays on the alarmmgr parity contract.
/// </summary>
[Fact]
public void EnqueueTransition_WhenDegraded_MarksDegradedAndSubtagProvider()
{
MxAccessEventQueue queue = new MxAccessEventQueue();
MxAccessAlarmEventSink sink = new MxAccessAlarmEventSink(queue, new MxAccessEventMapper());
sink.Attach(new object(), SessionId);
DateTime ts = new DateTime(2026, 5, 1, 17, 26, 14, 709, DateTimeKind.Utc);
sink.EnqueueTransition(
alarmFullReference: "Galaxy!TestArea.TestMachine_001.TestAlarm001",
sourceObjectReference: "TestMachine_001.TestAlarm001",
alarmTypeName: "DSC",
transitionKind: AlarmTransitionKind.Raise,
severity: 500,
originalRaiseTimestampUtc: null,
transitionTimestampUtc: ts,
operatorUser: string.Empty,
operatorComment: string.Empty,
category: "TestArea",
description: string.Empty,
degraded: true);
Assert.Equal(1, queue.Count);
Assert.True(queue.TryDequeue(out WorkerEvent? workerEvent));
OnAlarmTransitionEvent body = workerEvent!.Event.OnAlarmTransition;
Assert.True(body.Degraded);
Assert.Equal(AlarmProviderMode.Subtag, body.SourceProvider);
}
/// <summary>Verifies that unchanged alarm states do not emit transitions.</summary>
@@ -242,6 +279,60 @@ public sealed class AlarmDispatcherTests
Assert.Equal(AlarmConditionState.ActiveAcked, snapshots[1].CurrentState);
}
/// <summary>
/// Verifies that the per-record subtag-fallback flag flows through the
/// snapshot path: a degraded record maps to an
/// <see cref="ActiveAlarmSnapshot"/> with <see cref="ActiveAlarmSnapshot.Degraded"/>
/// set and <see cref="AlarmProviderMode.Subtag"/>, while a non-degraded
/// record stays on the alarmmgr parity contract.
/// </summary>
[Fact]
public void SnapshotActiveAlarms_PropagatesDegradedAndSourceProvider()
{
FakeAlarmConsumer consumer = new FakeAlarmConsumer();
DateTime ts = new DateTime(2026, 5, 1, 17, 26, 14, 709, DateTimeKind.Utc);
consumer.SnapshotResult = new[]
{
new MxAlarmSnapshotRecord
{
AlarmGuid = Guid.NewGuid(),
ProviderName = "Galaxy",
Group = "TestArea",
TagName = "Tag1",
Type = "DSC",
Priority = 500,
State = MxAlarmStateKind.UnackAlm,
TransitionTimestampUtc = ts,
Degraded = true,
},
new MxAlarmSnapshotRecord
{
AlarmGuid = Guid.NewGuid(),
ProviderName = "Galaxy",
Group = "TestArea",
TagName = "Tag2",
Type = "ANL",
Priority = 100,
State = MxAlarmStateKind.UnackAlm,
TransitionTimestampUtc = ts,
Degraded = false,
},
};
using AlarmDispatcher dispatcher = new AlarmDispatcher(
consumer,
new MxAccessAlarmEventSink(new MxAccessEventQueue(), new MxAccessEventMapper()),
SessionId);
IReadOnlyList<ActiveAlarmSnapshot> snapshots = dispatcher.SnapshotActiveAlarms();
Assert.Equal(2, snapshots.Count);
Assert.True(snapshots[0].Degraded);
Assert.Equal(AlarmProviderMode.Subtag, snapshots[0].SourceProvider);
Assert.False(snapshots[1].Degraded);
Assert.Equal(AlarmProviderMode.Alarmmgr, snapshots[1].SourceProvider);
}
/// <summary>Verifies that dispose unsubscribes the handler and disposes the consumer.</summary>
[Fact]
public void Dispose_WhenSubscribed_UnsubscribesHandlerAndDisposesConsumer()
@@ -0,0 +1,294 @@
using System;
using System.Collections.Generic;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Worker.MxAccess;
using Xunit;
namespace ZB.MOM.WW.MxGateway.Worker.Tests.MxAccess;
/// <summary>
/// Unit tests for <see cref="FailoverAlarmConsumer"/>: prove the
/// auto-failover (consecutive primary COM failures → standby) and
/// auto-failback (consecutive clean probes → primary) state machine,
/// active-child transition forwarding, and active-child delegation of
/// acknowledgments. Fakes stand in for both children so this needs no
/// AVEVA install.
/// </summary>
public sealed class FailoverAlarmConsumerTests
{
/// <summary>
/// Primary fake whose Subscribe/PollOnce throw a COMException while
/// <see cref="ThrowOnPoll"/> is set, modeling a wnwrap consumer that
/// surfaces COM HRESULT failures. Can also re-raise a transition so
/// before-failover forwarding can be exercised.
/// </summary>
private sealed class FlakyPrimary : IMxAccessAlarmConsumer
{
public event EventHandler<MxAlarmTransitionEvent>? AlarmTransitionEmitted;
public bool ThrowOnPoll = true;
public int Polls;
/// <summary>
/// Number of times <see cref="Subscribe"/> has been called.
/// Incremented at entry, before any throw, so every attempt is
/// counted regardless of whether <see cref="ThrowOnPoll"/> is set.
/// </summary>
public int SubscribeCount;
public void Subscribe(string s)
{
SubscribeCount++;
if (ThrowOnPoll)
{
throw new System.Runtime.InteropServices.COMException("boom", unchecked((int)0x80004005));
}
}
public void PollOnce()
{
Polls++;
if (ThrowOnPoll)
{
throw new System.Runtime.InteropServices.COMException("boom", unchecked((int)0x80004005));
}
}
public int AcknowledgeByGuid(Guid g, string c, string a, string b, string d, string e) => 11;
public int AcknowledgeByName(string n, string p, string gr, string c, string a, string b, string d, string e) => 11;
public IReadOnlyList<MxAlarmSnapshotRecord> SnapshotActiveAlarms() => Array.Empty<MxAlarmSnapshotRecord>();
public void Dispose() { }
public void Raise(MxAlarmTransitionEvent e) => AlarmTransitionEmitted?.Invoke(this, e);
}
/// <summary>
/// Standby fake (subtag stand-in): never throws, records that it was
/// armed, and can re-raise a transition.
/// </summary>
private sealed class StubStandby : IMxAccessAlarmConsumer
{
public event EventHandler<MxAlarmTransitionEvent>? AlarmTransitionEmitted;
public bool Subscribed;
public void Subscribe(string s) => Subscribed = true;
public void PollOnce() { }
public int AcknowledgeByGuid(Guid g, string c, string a, string b, string d, string e) => 22;
public int AcknowledgeByName(string n, string p, string gr, string c, string a, string b, string d, string e) => 22;
public IReadOnlyList<MxAlarmSnapshotRecord> SnapshotActiveAlarms() => Array.Empty<MxAlarmSnapshotRecord>();
public void Dispose() { }
public void Raise(MxAlarmTransitionEvent e) => AlarmTransitionEmitted?.Invoke(this, e);
}
private static MxAlarmTransitionEvent SampleTransition() => new MxAlarmTransitionEvent
{
Record = new MxAlarmSnapshotRecord { AlarmGuid = Guid.NewGuid() },
PreviousState = MxAlarmStateKind.Unspecified,
};
[Fact]
public void Primary_FailsThresholdTimes_SwitchesToSubtag()
{
FlakyPrimary primary = new FlakyPrimary { ThrowOnPoll = true };
StubStandby standby = new StubStandby();
FailoverSettings settings = new FailoverSettings(threshold: 3, probeIntervalSeconds: 0, stableProbes: 1);
using FailoverAlarmConsumer sut = new FailoverAlarmConsumer(primary, standby, settings);
List<AlarmProviderModeChange> changes = new List<AlarmProviderModeChange>();
sut.ProviderModeChanged += (_, e) => changes.Add(e);
sut.Subscribe(@"\\HOST\Galaxy!Area"); // failure 1 (primary), standby armed
Assert.True(standby.Subscribed);
Assert.Empty(changes);
sut.PollOnce(); // failure 2
Assert.Empty(changes);
sut.PollOnce(); // failure 3 → switch
Assert.Single(changes);
Assert.Equal(AlarmProviderMode.Subtag, changes[0].Mode);
Assert.Equal(AlarmProviderMode.Subtag, sut.Mode);
Assert.Equal(unchecked((int)0x80004005), changes[0].HResult);
}
[Fact]
public void AfterSwitch_StandbyTransitionsAreForwarded()
{
FlakyPrimary primary = new FlakyPrimary { ThrowOnPoll = true };
StubStandby standby = new StubStandby();
FailoverSettings settings = new FailoverSettings(threshold: 1, probeIntervalSeconds: 0, stableProbes: 1);
using FailoverAlarmConsumer sut = new FailoverAlarmConsumer(primary, standby, settings);
MxAlarmTransitionEvent? forwarded = null;
sut.AlarmTransitionEmitted += (_, e) => forwarded = e;
sut.Subscribe(@"\\HOST\Galaxy!Area"); // threshold=1 → switch to Subtag immediately
Assert.Equal(AlarmProviderMode.Subtag, sut.Mode);
MxAlarmTransitionEvent transition = SampleTransition();
standby.Raise(transition);
Assert.Same(transition, forwarded);
}
[Fact]
public void WhileDegraded_PrimaryHeals_FailsBackAfterStableProbes()
{
// threshold=1 so the initial Subscribe failure (PollOnce path) immediately
// switches to Subtag. stableProbes=2 means two consecutive clean PollOnce
// calls are needed before failback. ProbeOnce must NOT call Subscribe —
// WnWrapAlarmConsumer is single-subscribe; re-calling would always throw.
FlakyPrimary primary = new FlakyPrimary { ThrowOnPoll = true };
StubStandby standby = new StubStandby();
FailoverSettings settings = new FailoverSettings(threshold: 1, probeIntervalSeconds: 0, stableProbes: 2);
using FailoverAlarmConsumer sut = new FailoverAlarmConsumer(primary, standby, settings);
List<AlarmProviderModeChange> changes = new List<AlarmProviderModeChange>();
sut.ProviderModeChanged += (_, e) => changes.Add(e);
sut.Subscribe(@"\\HOST\Galaxy!Area"); // threshold=1 → Subtag (mode change 1)
Assert.Single(changes);
Assert.Equal(AlarmProviderMode.Subtag, changes[changes.Count - 1].Mode);
// Primary heals: PollOnce stops throwing. ProbeOnce should call only
// PollOnce (not Subscribe) to detect recovery.
primary.ThrowOnPoll = false;
int subscribeCountAfterFailover = primary.SubscribeCount;
sut.ProbeOnce(); // cleanProbes=1 — not yet at stableProbes=2
Assert.Single(changes);
sut.ProbeOnce(); // cleanProbes=2 → failback to Alarmmgr (mode change 2)
Assert.Equal(2, changes.Count);
Assert.Equal(AlarmProviderMode.Alarmmgr, changes[changes.Count - 1].Mode);
Assert.Equal(AlarmProviderMode.Alarmmgr, sut.Mode);
Assert.Equal(0, changes[changes.Count - 1].HResult);
// ProbeOnce must not have called Subscribe at all during probing.
Assert.Equal(subscribeCountAfterFailover, primary.SubscribeCount);
}
[Fact]
public void BeforeFailover_PrimaryTransitionsAreForwarded()
{
FlakyPrimary primary = new FlakyPrimary { ThrowOnPoll = false }; // healthy, can Raise
StubStandby standby = new StubStandby();
FailoverSettings settings = new FailoverSettings(threshold: 3, probeIntervalSeconds: 0, stableProbes: 1);
using FailoverAlarmConsumer sut = new FailoverAlarmConsumer(primary, standby, settings);
List<MxAlarmTransitionEvent> forwarded = new List<MxAlarmTransitionEvent>();
sut.AlarmTransitionEmitted += (_, e) => forwarded.Add(e);
sut.Subscribe(@"\\HOST\Galaxy!Area");
Assert.Equal(AlarmProviderMode.Alarmmgr, sut.Mode);
MxAlarmTransitionEvent fromPrimary = SampleTransition();
primary.Raise(fromPrimary); // active=Primary → forwarded
Assert.Single(forwarded);
Assert.Same(fromPrimary, forwarded[0]);
standby.Raise(SampleTransition()); // standby not active → suppressed
Assert.Single(forwarded);
}
/// <summary>
/// Proves that <see cref="FailoverAlarmConsumer.ProbeOnce"/> never calls
/// <c>Subscribe</c> on the primary while degraded. The production primary
/// (<see cref="WnWrapAlarmConsumer"/>) is single-subscribe; a second
/// <c>Subscribe</c> call would always throw and make failback impossible.
/// The probe must re-poll the still-subscribed primary via
/// <c>PollOnce</c> only.
/// </summary>
[Fact]
public void ProbeOnce_DoesNotCallPrimarySubscribe()
{
// threshold=1 → first Subscribe failure immediately switches to Subtag.
FlakyPrimary primary = new FlakyPrimary { ThrowOnPoll = true };
StubStandby standby = new StubStandby();
FailoverSettings settings = new FailoverSettings(threshold: 1, probeIntervalSeconds: 0, stableProbes: 3);
using FailoverAlarmConsumer sut = new FailoverAlarmConsumer(primary, standby, settings);
sut.Subscribe(@"\\HOST\Galaxy!Area"); // Subscribe attempt #1 (throws) → Subtag
// Capture how many Subscribe calls the initial setup caused (exactly 1:
// the attempt that threw and triggered failover).
int subscribeCountAfterSetup = primary.SubscribeCount;
Assert.Equal(1, subscribeCountAfterSetup);
Assert.Equal(AlarmProviderMode.Subtag, sut.Mode);
// Let PollOnce succeed so ProbeOnce progresses without throwing.
primary.ThrowOnPoll = false;
// Drive several ProbeOnce calls — none should touch Subscribe.
sut.ProbeOnce();
sut.ProbeOnce();
sut.ProbeOnce(); // stableProbes=3 → failback on this call
Assert.Equal(AlarmProviderMode.Alarmmgr, sut.Mode);
Assert.Equal(subscribeCountAfterSetup, primary.SubscribeCount);
}
[Fact]
public void Acknowledge_DelegatesToActiveChild()
{
FlakyPrimary primary = new FlakyPrimary { ThrowOnPoll = false };
StubStandby standby = new StubStandby();
FailoverSettings settings = new FailoverSettings(threshold: 1, probeIntervalSeconds: 0, stableProbes: 1);
using FailoverAlarmConsumer sut = new FailoverAlarmConsumer(primary, standby, settings);
sut.Subscribe(@"\\HOST\Galaxy!Area");
// Active = Primary → primary's sentinel value (11).
Assert.Equal(11, sut.AcknowledgeByGuid(Guid.NewGuid(), "c", "n", "node", "dom", "full"));
Assert.Equal(11, sut.AcknowledgeByName("a", "p", "g", "c", "n", "node", "dom", "full"));
// Force a failover by failing the primary past threshold.
primary.ThrowOnPoll = true;
sut.PollOnce(); // threshold=1 → switch to Standby
Assert.Equal(AlarmProviderMode.Subtag, sut.Mode);
// Active = Standby → standby's sentinel value (22).
Assert.Equal(22, sut.AcknowledgeByGuid(Guid.NewGuid(), "c", "n", "node", "dom", "full"));
Assert.Equal(22, sut.AcknowledgeByName("a", "p", "g", "c", "n", "node", "dom", "full"));
}
/// <summary>
/// Proves that an intermittent failure during failback probing resets the
/// clean-probe counter to zero, requiring a fresh unbroken run of
/// <see cref="FailoverSettings.StableProbes"/> before failing back.
/// </summary>
[Fact]
public void FailbackProbe_IntermittentFailure_ResetsCleanCount()
{
var primary = new FlakyPrimary { ThrowOnPoll = true };
var standby = new StubStandby();
using var sut = new FailoverAlarmConsumer(primary, standby, new FailoverSettings(threshold: 1, probeIntervalSeconds: 0, stableProbes: 3));
sut.Subscribe(@"\\HOST\Galaxy!Area"); // threshold=1 → switch to Subtag
Assert.Equal(AlarmProviderMode.Subtag, sut.Mode);
primary.ThrowOnPoll = false;
sut.ProbeOnce(); // clean 1
sut.ProbeOnce(); // clean 2
primary.ThrowOnPoll = true;
sut.ProbeOnce(); // fails → reset to 0
Assert.Equal(AlarmProviderMode.Subtag, sut.Mode);
primary.ThrowOnPoll = false;
sut.ProbeOnce(); // clean 1
sut.ProbeOnce(); // clean 2
sut.ProbeOnce(); // clean 3 → failback
Assert.Equal(AlarmProviderMode.Alarmmgr, sut.Mode);
}
}
@@ -0,0 +1,226 @@
using System;
using System.Collections.Generic;
using ZB.MOM.WW.MxGateway.Worker.MxAccess;
namespace ZB.MOM.WW.MxGateway.Worker.Tests.MxAccess;
/// <summary>
/// Unit-test coverage for <see cref="LmxSubtagAlarmSource"/>'s advise/write
/// sequencing and its <c>OnDataChange</c> normalization. The actual
/// <c>LMXProxyServerClass</c> COM event subscription cannot be exercised
/// without a live MXAccess install, so these tests drive the source through
/// its internal <see cref="IMxAccessServer"/> seam and call
/// <c>HandleDataChange</c> directly to simulate a COM callback — exactly the
/// boundary <c>MxAccessBaseEventSink.OnDataChange</c> uses for the
/// per-session pipeline. End-to-end COM delivery is covered by the
/// Skip-gated alarm live smoke tests.
/// </summary>
public sealed class LmxSubtagAlarmSourceTests
{
private const int FakeServerHandle = 7;
/// <summary>Verifies the production constructor rejects a null factory.</summary>
[Fact]
public void Constructor_NullFactory_Throws()
{
Assert.Throws<ArgumentNullException>(() => new LmxSubtagAlarmSource(factory: null!));
}
/// <summary>
/// Verifies <see cref="LmxSubtagAlarmSource.Advise"/> calls AddItem then
/// Advise once per distinct address, and is idempotent on a repeated
/// address.
/// </summary>
[Fact]
public void Advise_AddsAndAdvisesEachAddressOnce()
{
var server = new RecordingMxAccessServer();
using var source = new LmxSubtagAlarmSource(server, FakeServerHandle);
source.Advise(new[] { "Tank1.Alarm.Subtag", "Tank2.Alarm.Subtag" });
// Re-advising an already-advised address is a no-op.
source.Advise(new[] { "Tank1.Alarm.Subtag" });
Assert.Equal(
new[] { "Tank1.Alarm.Subtag", "Tank2.Alarm.Subtag" },
server.AddedItems);
Assert.Equal(2, server.AdviseCount);
// Every advise targeted the supplied server handle.
Assert.All(server.AdvisedServerHandles, h => Assert.Equal(FakeServerHandle, h));
}
/// <summary>
/// Verifies a simulated <c>OnDataChange</c> for an advised item handle
/// raises <see cref="LmxSubtagAlarmSource.ValueChanged"/> with the
/// address that was advised and the delivered value.
/// </summary>
[Fact]
public void HandleDataChange_RaisesValueChangedWithAdvisedAddress()
{
var server = new RecordingMxAccessServer();
using var source = new LmxSubtagAlarmSource(server, FakeServerHandle);
source.Advise(new[] { "Tank1.Alarm.Subtag" });
int itemHandle = server.LastItemHandleFor("Tank1.Alarm.Subtag");
SubtagValueChange? received = null;
source.ValueChanged += (_, change) => received = change;
source.HandleDataChange(itemHandle, pvItemValue: 42, pftItemTimeStamp: null);
Assert.NotNull(received);
Assert.Equal("Tank1.Alarm.Subtag", received!.ItemAddress);
Assert.Equal(42, received.Value);
Assert.Equal(DateTimeKind.Utc, received.TimestampUtc.Kind);
}
/// <summary>
/// Verifies <c>OnDataChange</c> for an unknown item handle is ignored
/// (no <see cref="LmxSubtagAlarmSource.ValueChanged"/> raised).
/// </summary>
[Fact]
public void HandleDataChange_UnknownHandle_DoesNotRaise()
{
var server = new RecordingMxAccessServer();
using var source = new LmxSubtagAlarmSource(server, FakeServerHandle);
bool raised = false;
source.ValueChanged += (_, _) => raised = true;
source.HandleDataChange(phItemHandle: 999, pvItemValue: 1, pftItemTimeStamp: null);
Assert.False(raised);
}
/// <summary>
/// Verifies <see cref="LmxSubtagAlarmSource.Write"/> adds the item when
/// it was not previously advised and writes with user id 0.
/// </summary>
[Fact]
public void Write_AddsItemWhenUnknownAndWrites()
{
var server = new RecordingMxAccessServer();
using var source = new LmxSubtagAlarmSource(server, FakeServerHandle);
source.Write("Tank1.Alarm.AckComment", "acknowledged");
Assert.Contains("Tank1.Alarm.AckComment", server.AddedItems);
Assert.Single(server.Writes);
RecordingMxAccessServer.WriteRecord write = server.Writes[0];
Assert.Equal(FakeServerHandle, write.ServerHandle);
Assert.Equal("acknowledged", write.Value);
Assert.Equal(0, write.UserId);
}
/// <summary>
/// Verifies <see cref="LmxSubtagAlarmSource.Write"/> reuses an existing
/// item handle (no duplicate AddItem) when the address was already
/// advised.
/// </summary>
[Fact]
public void Write_ReusesHandleForAdvisedAddress()
{
var server = new RecordingMxAccessServer();
using var source = new LmxSubtagAlarmSource(server, FakeServerHandle);
source.Advise(new[] { "Tank1.Alarm.Subtag" });
int adviseAddCount = server.AddedItems.Count;
source.Write("Tank1.Alarm.Subtag", true);
// No second AddItem for the already-advised address.
Assert.Equal(adviseAddCount, server.AddedItems.Count);
Assert.Single(server.Writes);
}
/// <summary>
/// Recording <see cref="IMxAccessServer"/> test double that captures the
/// AddItem/Advise/Write/UnAdvise/RemoveItem/Unregister calls
/// <see cref="LmxSubtagAlarmSource"/> makes and hands out monotonically
/// increasing item handles.
/// </summary>
private sealed class RecordingMxAccessServer : IMxAccessServer
{
private readonly Dictionary<string, int> handlesByAddress = new(StringComparer.Ordinal);
private int nextItemHandle = 100;
public List<string> AddedItems { get; } = new();
public int AdviseCount { get; private set; }
public List<int> AdvisedServerHandles { get; } = new();
public List<WriteRecord> Writes { get; } = new();
public int LastItemHandleFor(string itemAddress) => handlesByAddress[itemAddress];
public int Register(string clientName) => FakeServerHandle;
public void Unregister(int serverHandle)
{
}
public int AddItem(int serverHandle, string itemDefinition)
{
AddedItems.Add(itemDefinition);
int handle = nextItemHandle++;
handlesByAddress[itemDefinition] = handle;
return handle;
}
public int AddItem2(int serverHandle, string itemDefinition, string itemContext)
=> AddItem(serverHandle, itemDefinition);
public void RemoveItem(int serverHandle, int itemHandle)
{
}
public void Advise(int serverHandle, int itemHandle)
{
AdviseCount++;
AdvisedServerHandles.Add(serverHandle);
}
public void UnAdvise(int serverHandle, int itemHandle)
{
}
public void AdviseSupervisory(int serverHandle, int itemHandle)
{
}
public void Write(int serverHandle, int itemHandle, object? value, int userId)
=> Writes.Add(new WriteRecord(serverHandle, itemHandle, value, userId));
public void Write2(int serverHandle, int itemHandle, object? value, object? timestamp, int userId)
{
}
public void WriteSecured(int serverHandle, int itemHandle, int currentUserId, int verifierUserId, object? value)
{
}
public void WriteSecured2(int serverHandle, int itemHandle, int currentUserId, int verifierUserId, object? value, object? timestamp)
{
}
internal sealed class WriteRecord
{
public WriteRecord(int serverHandle, int itemHandle, object? value, int userId)
{
ServerHandle = serverHandle;
ItemHandle = itemHandle;
Value = value;
UserId = userId;
}
public int ServerHandle { get; }
public int ItemHandle { get; }
public object? Value { get; }
public int UserId { get; }
}
}
}
@@ -200,7 +200,7 @@ public sealed class MxAccessStaSessionTests
factory,
eventSink,
new MxAccessEventQueue(),
(_eq, _affinity) => handler);
(_eq, _affinity, _comFactory) => handler);
await session.StartAsync("session-1", workerProcessId: 1);
@@ -279,7 +279,7 @@ public sealed class MxAccessStaSessionTests
factory,
eventSink,
new MxAccessEventQueue(),
(_eq, _affinity) => handler);
(_eq, _affinity, _comFactory) => handler);
await session.StartAsync("session-1", workerProcessId: 1);
@@ -320,7 +320,7 @@ public sealed class MxAccessStaSessionTests
factory,
eventSink,
new MxAccessEventQueue(),
(_eq, _affinity) => handler);
(_eq, _affinity, _comFactory) => handler);
await session.StartAsync("session-1", workerProcessId: 1);
@@ -369,7 +369,7 @@ public sealed class MxAccessStaSessionTests
factory,
eventSink,
eventQueue,
(_eq, _affinity) => handler);
(_eq, _affinity, _comFactory) => handler);
await session.StartAsync("session-1", workerProcessId: 1);
@@ -416,7 +416,7 @@ public sealed class MxAccessStaSessionTests
factory,
eventSink,
eventQueue,
(_eq, _affinity) => handler);
(_eq, _affinity, _comFactory) => handler);
await session.StartAsync("session-1", workerProcessId: 1);
@@ -496,12 +496,12 @@ public sealed class MxAccessStaSessionTests
}
/// <summary>Subscribes to alarm events.</summary>
/// <param name="subscription">The subscription descriptor.</param>
/// <param name="command">The subscribe-alarms command.</param>
/// <param name="sessionId">The session identifier.</param>
public void Subscribe(string subscription, string sessionId)
public void Subscribe(SubscribeAlarmsCommand command, string sessionId)
{
IsSubscribed = true;
LastSubscription = subscription;
LastSubscription = command.SubscriptionExpression;
}
/// <summary>Unsubscribes from alarm events.</summary>
@@ -0,0 +1,331 @@
using System;
using System.Collections.Generic;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Worker.MxAccess;
using Xunit;
namespace ZB.MOM.WW.MxGateway.Worker.Tests.MxAccess;
/// <summary>
/// Unit tests for <see cref="SubtagAlarmConsumer"/>: prove that the subtag
/// fallback advises the observable subtags, synthesizes degraded
/// transitions with stable synthetic GUIDs, routes acknowledgments to the
/// ack-comment subtag, and stamps snapshots. A <see cref="FakeSource"/>
/// stands in for the live MXAccess subtag source so this needs no AVEVA
/// install.
/// </summary>
public sealed class SubtagAlarmConsumerTests
{
private const string Reference = "Galaxy!TestArea.Tank01.Level.HiHi";
private const string ActiveSubtag = "Tank01.Level.HiHi.InAlarm";
private const string AckedSubtag = "Tank01.Level.HiHi.Acked";
private const string AckCommentSubtag = "Tank01.Level.HiHi.AckComment";
private const string PrioritySubtag = "Tank01.Level.HiHi.Priority";
private static AlarmSubtagTarget BuildTarget()
{
return new AlarmSubtagTarget
{
AlarmFullReference = Reference,
SourceObjectReference = "Tank01",
ActiveSubtag = ActiveSubtag,
AckedSubtag = AckedSubtag,
AckCommentSubtag = AckCommentSubtag,
PrioritySubtag = PrioritySubtag,
};
}
private static SubtagAlarmConsumer BuildConsumer(FakeSource source)
{
return new SubtagAlarmConsumer(source, new[] { BuildTarget() });
}
/// <summary>Verifies Subscribe advises the active, acked, and priority subtags but not the ack-comment subtag.</summary>
[Fact]
public void Subscribe_AdvisesActiveAndAckedSubtags()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
Assert.Contains(ActiveSubtag, source.Advised);
Assert.Contains(AckedSubtag, source.Advised);
Assert.Contains(PrioritySubtag, source.Advised);
Assert.DoesNotContain(AckCommentSubtag, source.Advised);
}
/// <summary>Verifies an active=true value change raises a degraded, GUID-stamped UNACK_ALM transition.</summary>
[Fact]
public void ValueChange_RaisesDegradedSynthesizedTransition()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
MxAlarmTransitionEvent? emitted = null;
consumer.AlarmTransitionEmitted += (_, e) => emitted = e;
source.Raise(ActiveSubtag, true, new DateTime(2026, 6, 13, 10, 0, 0, DateTimeKind.Utc));
Assert.NotNull(emitted);
Assert.Equal(MxAlarmStateKind.UnackAlm, emitted!.Record.State);
Assert.True(emitted.Record.Degraded);
Assert.NotEqual(Guid.Empty, emitted.Record.AlarmGuid);
}
/// <summary>Verifies AcknowledgeByName writes the comment to the ack-comment subtag and returns success.</summary>
[Fact]
public void AcknowledgeByName_WritesCommentToAckCommentSubtag()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
int rc = consumer.AcknowledgeByName(
alarmName: "Tank01.Level.HiHi",
providerName: "Galaxy",
groupName: "TestArea",
ackComment: "operator ack",
ackOperatorName: "alice",
ackOperatorNode: "WS01",
ackOperatorDomain: "CORP",
ackOperatorFullName: "Alice Smith");
Assert.Equal(0, rc);
Assert.NotNull(source.LastWrite);
Assert.Equal(AckCommentSubtag, source.LastWrite!.Value.Address);
Assert.Equal("operator ack", source.LastWrite!.Value.Value);
}
/// <summary>Verifies AcknowledgeByName returns non-zero when no target matches the supplied name.</summary>
[Fact]
public void AcknowledgeByName_UnknownAlarm_ReturnsNonZero()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
int rc = consumer.AcknowledgeByName(
alarmName: "DoesNotExist.NoSuchAlarm",
providerName: "Galaxy",
groupName: "TestArea",
ackComment: "operator ack",
ackOperatorName: "alice",
ackOperatorNode: "WS01",
ackOperatorDomain: "CORP",
ackOperatorFullName: "Alice Smith");
Assert.NotEqual(0, rc);
Assert.Null(source.LastWrite);
}
/// <summary>Verifies a snapshot of an active alarm stamps Degraded and a non-empty synthetic GUID.</summary>
[Fact]
public void SnapshotActiveAlarms_StampsDegradedAndGuid()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
source.Raise(ActiveSubtag, true, new DateTime(2026, 6, 13, 10, 0, 0, DateTimeKind.Utc));
IReadOnlyList<MxAlarmSnapshotRecord> snapshot = consumer.SnapshotActiveAlarms();
Assert.Single(snapshot);
Assert.True(snapshot[0].Degraded);
Assert.NotEqual(Guid.Empty, snapshot[0].AlarmGuid);
}
/// <summary>Verifies the synthetic GUID on the emitted transition equals the GUID in the snapshot for the same alarm.</summary>
[Fact]
public void SameReference_SyntheticGuidStableAcrossTransitionAndSnapshot()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
MxAlarmTransitionEvent? emitted = null;
consumer.AlarmTransitionEmitted += (_, e) => emitted = e;
source.Raise(ActiveSubtag, true, new DateTime(2026, 6, 13, 10, 0, 0, DateTimeKind.Utc));
IReadOnlyList<MxAlarmSnapshotRecord> snapshot = consumer.SnapshotActiveAlarms();
Assert.NotNull(emitted);
Assert.Single(snapshot);
Assert.Equal(emitted!.Record.AlarmGuid, snapshot[0].AlarmGuid);
Assert.NotEqual(Guid.Empty, emitted.Record.AlarmGuid);
}
/// <summary>
/// Verifies that when two alarm targets share a prefix (e.g. Level.Hi vs Level.HiHi),
/// AcknowledgeByName routes each ack to its own ack-comment subtag and never
/// conflates the shorter name with the longer one.
/// </summary>
[Fact]
public void AcknowledgeByName_PrefixNameDoesNotFalseMatch()
{
const string ReferenceHi = "Galaxy!Area.Tank01.Level.Hi";
const string ReferenceHiHi = "Galaxy!Area.Tank01.Level.HiHi";
const string AckCommentHi = "Tank01.Level.Hi.AckComment";
const string AckCommentHiHi = "Tank01.Level.HiHi.AckComment";
AlarmSubtagTarget targetHi = new AlarmSubtagTarget
{
AlarmFullReference = ReferenceHi,
SourceObjectReference = "Tank01",
ActiveSubtag = "Tank01.Level.Hi.InAlarm",
AckedSubtag = "Tank01.Level.Hi.Acked",
AckCommentSubtag = AckCommentHi,
PrioritySubtag = "Tank01.Level.Hi.Priority",
};
AlarmSubtagTarget targetHiHi = new AlarmSubtagTarget
{
AlarmFullReference = ReferenceHiHi,
SourceObjectReference = "Tank01",
ActiveSubtag = "Tank01.Level.HiHi.InAlarm",
AckedSubtag = "Tank01.Level.HiHi.Acked",
AckCommentSubtag = AckCommentHiHi,
PrioritySubtag = "Tank01.Level.HiHi.Priority",
};
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = new SubtagAlarmConsumer(
source, new[] { targetHi, targetHiHi });
consumer.Subscribe(@"\\HOST\Galaxy!Area");
// Ack the shorter name — must write to the shorter target's subtag only.
int rcHi = consumer.AcknowledgeByName(
alarmName: "Tank01.Level.Hi",
providerName: "Galaxy",
groupName: "Area",
ackComment: "ack hi",
ackOperatorName: "op", ackOperatorNode: "WS01",
ackOperatorDomain: "CORP", ackOperatorFullName: "Operator");
Assert.Equal(0, rcHi);
Assert.NotNull(source.LastWrite);
Assert.Equal(AckCommentHi, source.LastWrite!.Value.Address);
Assert.Equal("ack hi", source.LastWrite.Value.Value);
// Ack the longer name — must write to the longer target's subtag.
int rcHiHi = consumer.AcknowledgeByName(
alarmName: "Tank01.Level.HiHi",
providerName: "Galaxy",
groupName: "Area",
ackComment: "ack hihi",
ackOperatorName: "op", ackOperatorNode: "WS01",
ackOperatorDomain: "CORP", ackOperatorFullName: "Operator");
Assert.Equal(0, rcHiHi);
Assert.NotNull(source.LastWrite);
Assert.Equal(AckCommentHiHi, source.LastWrite!.Value.Address);
Assert.Equal("ack hihi", source.LastWrite.Value.Value);
}
/// <summary>
/// Verifies AcknowledgeByGuid resolves the synthetic GUID (computed from
/// the alarm's full reference) to the correct target and writes the comment
/// to that target's ack-comment subtag.
/// </summary>
[Fact]
public void AcknowledgeByGuid_WritesCommentToAckCommentSubtag()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
// Raise a transition so the state machine sees the alarm, then capture
// the GUID stamped on the emitted event.
MxAlarmTransitionEvent? emitted = null;
consumer.AlarmTransitionEmitted += (_, e) => emitted = e;
source.Raise(ActiveSubtag, true, new DateTime(2026, 6, 13, 10, 0, 0, DateTimeKind.Utc));
Assert.NotNull(emitted);
Guid syntheticGuid = emitted!.Record.AlarmGuid;
Assert.NotEqual(Guid.Empty, syntheticGuid);
int rc = consumer.AcknowledgeByGuid(
alarmGuid: syntheticGuid,
ackComment: "guid ack",
ackOperatorName: "op",
ackOperatorNode: "WS01",
ackOperatorDomain: "CORP",
ackOperatorFullName: "Operator");
Assert.Equal(0, rc);
Assert.NotNull(source.LastWrite);
Assert.Equal(AckCommentSubtag, source.LastWrite!.Value.Address);
Assert.Equal("guid ack", source.LastWrite.Value.Value);
}
/// <summary>
/// Verifies AcknowledgeByGuid returns non-zero and performs no write when
/// the supplied GUID is not known to the consumer.
/// </summary>
[Fact]
public void AcknowledgeByGuid_UnknownGuid_ReturnsNonZero()
{
FakeSource source = new FakeSource();
using SubtagAlarmConsumer consumer = BuildConsumer(source);
consumer.Subscribe(@"\\HOST\Galaxy!TestArea");
int rc = consumer.AcknowledgeByGuid(
alarmGuid: Guid.NewGuid(),
ackComment: "should not write",
ackOperatorName: "op",
ackOperatorNode: "WS01",
ackOperatorDomain: "CORP",
ackOperatorFullName: "Operator");
Assert.NotEqual(0, rc);
Assert.Null(source.LastWrite);
}
private sealed class FakeSource : ISubtagAlarmSource
{
/// <summary>Raised when an advised subtag reports a new value.</summary>
public event EventHandler<SubtagValueChange>? ValueChanged;
/// <summary>Gets the subtag addresses passed to <see cref="Advise"/>.</summary>
public List<string> Advised { get; } = new List<string>();
/// <summary>Gets the most recent (address, value) pair passed to <see cref="Write"/>.</summary>
public (string Address, object? Value)? LastWrite { get; private set; }
/// <summary>Records the advised subtag addresses.</summary>
/// <param name="itemAddresses">The subtag references to advise.</param>
public void Advise(IReadOnlyCollection<string> itemAddresses)
{
Advised.AddRange(itemAddresses);
}
/// <summary>Records the most recent write.</summary>
/// <param name="itemAddress">The subtag reference to write.</param>
/// <param name="value">The value to write.</param>
public void Write(string itemAddress, object? value)
{
LastWrite = (itemAddress, value);
}
/// <summary>Raises a <see cref="SubtagValueChange"/> for the given subtag.</summary>
/// <param name="address">The subtag address whose value changed.</param>
/// <param name="value">The new value.</param>
/// <param name="timestampUtc">The UTC timestamp of the change.</param>
public void Raise(string address, object? value, DateTime timestampUtc)
{
ValueChanged?.Invoke(this, new SubtagValueChange
{
ItemAddress = address,
Value = value,
TimestampUtc = timestampUtc,
});
}
/// <inheritdoc />
public void Dispose()
{
}
}
}
@@ -0,0 +1,143 @@
using System;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Worker.MxAccess;
using Xunit;
namespace ZB.MOM.WW.MxGateway.Worker.Tests.MxAccess;
/// <summary>
/// Unit tests for the subtag-fallback synthesis state machine. The machine
/// consumes normalized subtag value changes (active/acked/priority) and
/// emits <see cref="MxAlarmTransitionEvent"/> records mirroring the wnwrap
/// consumer's UNACK_ALM / ACK_ALM / UNACK_RTN / ACK_RTN transitions. No COM
/// or AVEVA install is required.
/// </summary>
public sealed class SubtagAlarmStateMachineTests
{
private static AlarmSubtagTarget Target() => new()
{
AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi",
SourceObjectReference = "Tank01",
ActiveSubtag = "Tank01.Level.HiHi.active",
AckedSubtag = "Tank01.Level.HiHi.acked",
AckCommentSubtag = "Tank01.Level.HiHi.ackmsg",
};
[Fact]
public void ActiveFalseToTrue_EmitsRaise()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
var events = sm.Apply("Tank01.Level.HiHi.active", true, ts);
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.UnackAlm, e.Record.State);
Assert.Equal(MxAlarmStateKind.Unspecified, e.PreviousState);
Assert.Equal("Tank01.Level.HiHi", e.Record.TagName);
Assert.Equal("Galaxy", e.Record.ProviderName);
Assert.Equal("Area", e.Record.Group);
}
[Fact]
public void ActiveFalseToTrue_NoProviderBang_UsesWholeReferenceAsTagName()
{
var target = new AlarmSubtagTarget
{
AlarmFullReference = "Tank01.Level.HiHi",
SourceObjectReference = string.Empty,
ActiveSubtag = "Tank01.Level.HiHi.active",
};
var sm = new SubtagAlarmStateMachine(new[] { target });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
var events = sm.Apply("Tank01.Level.HiHi.active", true, ts);
var e = Assert.Single(events);
Assert.Equal("Tank01.Level.HiHi", e.Record.TagName);
Assert.Equal(string.Empty, e.Record.ProviderName);
Assert.Equal(string.Empty, e.Record.Group);
}
[Fact]
public void OutOfOrderAckThenClear_StillEmitsAckRtn()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
sm.Apply("Tank01.Level.HiHi.acked", true, ts.AddSeconds(2));
// Out-of-order un-ack arrives before the active=false clear.
sm.Apply("Tank01.Level.HiHi.acked", false, ts.AddSeconds(3));
var events = sm.Apply("Tank01.Level.HiHi.active", false, ts.AddSeconds(10));
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.AckRtn, e.Record.State);
}
[Fact]
public void DuplicateActiveSubtag_Throws()
{
var first = new AlarmSubtagTarget
{
AlarmFullReference = "Galaxy!Area.Tank01.Level.HiHi",
SourceObjectReference = "Tank01",
ActiveSubtag = "Shared.active",
};
var second = new AlarmSubtagTarget
{
AlarmFullReference = "Galaxy!Area.Tank02.Level.HiHi",
SourceObjectReference = "Tank02",
ActiveSubtag = "Shared.active",
};
Assert.Throws<ArgumentException>(() => new SubtagAlarmStateMachine(new[] { first, second }));
}
[Fact]
public void AckedTrueWhileActive_EmitsAck()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
var events = sm.Apply("Tank01.Level.HiHi.acked", true, ts.AddSeconds(5));
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.AckAlm, e.Record.State);
Assert.Equal(MxAlarmStateKind.UnackAlm, e.PreviousState);
}
[Fact]
public void ActiveTrueToFalse_WhileUnacked_EmitsUnackRtn()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
var events = sm.Apply("Tank01.Level.HiHi.active", false, ts.AddSeconds(10));
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.UnackRtn, e.Record.State);
}
[Fact]
public void ActiveTrueToFalse_WhileAcked_EmitsAckRtn()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
sm.Apply("Tank01.Level.HiHi.acked", true, ts.AddSeconds(2));
var events = sm.Apply("Tank01.Level.HiHi.active", false, ts.AddSeconds(10));
var e = Assert.Single(events);
Assert.Equal(MxAlarmStateKind.AckRtn, e.Record.State);
}
[Fact]
public void Snapshot_ReflectsActiveAndAckedState()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var ts = new DateTime(2026, 6, 13, 9, 0, 0, DateTimeKind.Utc);
sm.Apply("Tank01.Level.HiHi.active", true, ts);
sm.Apply("Tank01.Level.HiHi.acked", true, ts);
var snap = Assert.Single(sm.SnapshotActive());
Assert.Equal(MxAlarmStateKind.AckAlm, snap.State);
}
[Fact]
public void UnknownAddress_NoEvents()
{
var sm = new SubtagAlarmStateMachine(new[] { Target() });
var events = sm.Apply("Some.Other.Tag.active", true, DateTime.UtcNow);
Assert.Empty(events);
}
}
@@ -0,0 +1,27 @@
using System;
using ZB.MOM.WW.MxGateway.Worker.MxAccess;
namespace ZB.MOM.WW.MxGateway.Worker.Tests.MxAccess;
/// <summary>
/// Unit tests for <see cref="SyntheticAlarmGuid"/>: the subtag-fallback
/// path derives a deterministic GUID from the alarm reference, so identical
/// references must collide and distinct references must not.
/// </summary>
public sealed class SyntheticAlarmGuidTests
{
/// <summary>Verifies the same reference yields the same GUID.</summary>
[Fact]
public void SameReference_SameGuid() =>
Assert.Equal(SyntheticAlarmGuid.ForReference("A.B.C"), SyntheticAlarmGuid.ForReference("A.B.C"));
/// <summary>Verifies distinct references yield distinct GUIDs.</summary>
[Fact]
public void DifferentReference_DifferentGuid() =>
Assert.NotEqual(SyntheticAlarmGuid.ForReference("A.B.C"), SyntheticAlarmGuid.ForReference("A.B.D"));
/// <summary>Verifies a reference produces a non-empty GUID.</summary>
[Fact]
public void Reference_ProducesNonEmptyGuid() =>
Assert.NotEqual(Guid.Empty, SyntheticAlarmGuid.ForReference("A.B.C"));
}
@@ -0,0 +1,435 @@
// AlarmSubtagLiveSmokeTests.cs
//
// Validates the open design item from Task 17: exact subtag names and the
// canonical AlarmSubtagTarget reference shape for a known Galaxy alarm.
//
// This test exercises the full subtag-fallback pipeline end-to-end against
// a real Galaxy + MXAccess install:
// LmxSubtagAlarmSource (own LMXProxyServerClass) ->
// SubtagAlarmConsumer (state machine + AcknowledgeByName write) ->
// synthesized MxAlarmTransitionEvent (Raise / Clear, Degraded=true, SyntheticGuid)
//
// HOW TO RUN:
// 1. On the dev rig with AVEVA System Platform installed and Galaxy running:
// $env:MXGATEWAY_RUN_LIVE_MXACCESS_TESTS = "1"
// 2. Remove (or set to null) the Skip parameter on the [Fact] below.
// 3. Run with an alarm flip script (same one used by AlarmsLiveSmokeTests)
// so that TestMachine_001.TestAlarm001 toggles its Active/Acked subtags
// on a ~10 s cadence.
// 4. VERIFY the placeholder subtag names (PLACEHOLDER_* constants below)
// against the live Galaxy before flipping Skip -- see VERIFY comment block.
//
// OPEN ITEM (Task 17): the exact subtag address strings are not yet confirmed
// from a live Galaxy or from C:\Users\dohertj2\Desktop\mxaccess
// docs/MXAccess-Public-API.md. Flip this test only after verifying them.
//
// net48/x86 constraints:
// - No init-only properties, records, index/range operators, C# 8+ pattern
// matches beyond what the existing Worker.Tests files use.
// - All Booleans from MXAccess arrive as boxed int (0 / non-zero); coerce
// with Convert.ToBoolean or cast to int first.
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Threading;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Worker.MxAccess;
using Xunit.Abstractions;
namespace ZB.MOM.WW.MxGateway.Worker.Tests.Probes;
/// <summary>
/// Live dev-rig smoke test for the subtag-fallback alarm pipeline.
/// Validates the open design item from Task 17: confirms that
/// <see cref="LmxSubtagAlarmSource"/> wired to a real
/// <c>LMXProxyServerClass</c> + <see cref="SubtagAlarmConsumer"/> can
/// synthesize <see cref="MxAlarmTransitionEvent"/> records from Galaxy
/// alarm subtags, and that <see cref="SubtagAlarmConsumer.AcknowledgeByName"/>
/// writes the ack-comment subtag successfully.
///
/// Skip-gated; flip <c>Skip=null</c> on the dev rig with the alarm flip
/// script running. VERIFY the PLACEHOLDER_* subtag names before running.
/// </summary>
public sealed class AlarmSubtagLiveSmokeTests
{
// -------------------------------------------------------------------------
// PLACEHOLDER references — VERIFY before flipping Skip.
//
// These values are the best-guess subtag addresses for TestMachine_001.TestAlarm001
// as used by the existing AlarmsLiveSmokeTests (which watches the same Galaxy
// provider). The exact attribute suffix strings (.InAlarm, .Acked,
// .AckComment, .Priority) MUST be confirmed against:
// - C:\Users\dohertj2\Desktop\mxaccess\docs\MXAccess-Public-API.md
// - A live Galaxy attribute browse or MXTraceHarness capture
// before this test is flipped on. The subtag address format is
// <ObjectTagName>.<AlarmAttributeName>
// where the alarm attribute names are Galaxy-attribute names surfaced
// through MXAccess. Known AVEVA attribute names are .InAlarm (bool),
// .Acked (bool), .Priority (int), and a writable comment attribute whose
// exact name varies by alarm template (commonly .AlarmComment or
// .AckComment).
// -------------------------------------------------------------------------
/// <summary>The Galaxy provider expression used by the existing live smoke tests.</summary>
private static readonly string Provider =
string.Format(@"\\{0}\Galaxy", Environment.MachineName);
/// <summary>The Galaxy group (provider sub-path) containing the test alarm.</summary>
private const string Group = "DEV";
/// <summary>The test alarm's tag name as the dispatcher composes it (Group.TagName).</summary>
private const string AlarmTagName = "TestMachine_001.TestAlarm001";
// PLACEHOLDER — VERIFY against MXAccess-Public-API.md and live Galaxy.
// Typical InProcess Platform alarm active subtag: "<TagName>.InAlarm"
private const string PlaceholderActiveSubtag = "TestMachine_001.TestAlarm001.InAlarm";
// PLACEHOLDER — VERIFY. Typical: "<TagName>.Acked"
private const string PlaceholderAckedSubtag = "TestMachine_001.TestAlarm001.Acked";
// PLACEHOLDER — VERIFY. The writable ack-comment attribute varies by alarm
// template; common names are .AlarmComment, .AckComment, or .OperComment.
// Use a live Galaxy attribute browse (InTouch AlarmDB or MXTraceHarness
// GetItemList) to confirm before flipping.
private const string PlaceholderAckCommentSubtag = "TestMachine_001.TestAlarm001.AlarmComment";
// PLACEHOLDER — VERIFY. Typically: "<TagName>.Priority"
private const string PlaceholderPrioritySubtag = "TestMachine_001.TestAlarm001.Priority";
// -------------------------------------------------------------------------
private static readonly TimeSpan RaiseWaitTimeout = TimeSpan.FromSeconds(30);
private static readonly TimeSpan ClearWaitTimeout = TimeSpan.FromSeconds(30);
private static readonly string AlarmFullReference =
AlarmRecordTransitionMapper.ComposeFullReference(Provider, Group, AlarmTagName);
private readonly ITestOutputHelper output;
private readonly Stopwatch elapsed = Stopwatch.StartNew();
private readonly ConcurrentQueue<string> log =
new ConcurrentQueue<string>();
/// <summary>Initializes a new instance of the AlarmSubtagLiveSmokeTests class.</summary>
/// <param name="output">Test output helper for logging.</param>
public AlarmSubtagLiveSmokeTests(ITestOutputHelper output)
{
this.output = output;
}
/// <summary>
/// Verifies the subtag-fallback pipeline: advises alarm subtags through a
/// real <c>LMXProxyServerClass</c>, collects a Raise then a Clear
/// transition synthesized by <see cref="SubtagAlarmConsumer"/>, confirms
/// the Degraded flag and synthetic GUID are stamped, then
/// AcknowledgeByName and verifies the ack-comment write returns 0.
/// </summary>
[Fact(Skip = "Live dev-rig smoke test — flip Skip=null with AVEVA + an alarm flip script running. Subtag fallback path. VERIFY PLACEHOLDER_* subtag names before enabling.")]
public void SubtagFallback_FullPipelineRoundTrip_SynthesizesRaiseAndAcknowledges()
{
Exception? threadException = null;
ManualResetEventSlim done = new ManualResetEventSlim(false);
Thread thread = new Thread(() =>
{
try { RunSmoke(); }
catch (Exception ex) { threadException = ex; }
finally { done.Set(); }
});
thread.IsBackground = false;
thread.SetApartmentState(ApartmentState.STA);
thread.Start();
done.Wait();
thread.Join();
output.WriteLine(string.Format("Captured {0} log line(s):", log.Count));
string? line;
while (log.TryDequeue(out line))
{
output.WriteLine(line);
}
if (threadException != null)
{
throw threadException;
}
}
private void RunSmoke()
{
Log(string.Format("AlarmFullReference: {0}", AlarmFullReference));
Log("VERIFY: PlaceholderActiveSubtag = " + PlaceholderActiveSubtag);
Log("VERIFY: PlaceholderAckedSubtag = " + PlaceholderAckedSubtag);
Log("VERIFY: PlaceholderAckCommentSubtag = " + PlaceholderAckCommentSubtag);
Log("VERIFY: PlaceholderPrioritySubtag = " + PlaceholderPrioritySubtag);
Log(string.Format("RaiseWaitTimeout={0}s ClearWaitTimeout={1}s",
RaiseWaitTimeout.TotalSeconds, ClearWaitTimeout.TotalSeconds));
// Build target with PLACEHOLDER subtag names.
// VERIFY these against MXAccess-Public-API.md and a live Galaxy
// attribute browse before flipping Skip=null.
AlarmSubtagTarget target = new AlarmSubtagTarget
{
AlarmFullReference = AlarmFullReference,
SourceObjectReference = AlarmTagName,
ActiveSubtag = PlaceholderActiveSubtag,
AckedSubtag = PlaceholderAckedSubtag,
AckCommentSubtag = PlaceholderAckCommentSubtag,
PrioritySubtag = PlaceholderPrioritySubtag,
};
List<AlarmSubtagTarget> watchList = new List<AlarmSubtagTarget> { target };
// Construct the real COM-backed subtag source using the production
// factory (MxAccessComObjectFactory -> new LMXProxyServerClass()).
// This is the same factory the worker uses in production; no test
// double is involved on this path.
MxAccessComObjectFactory factory = new MxAccessComObjectFactory();
LmxSubtagAlarmSource source = new LmxSubtagAlarmSource(factory, clientName: null);
// SubtagAlarmConsumer wraps the source and drives the state machine.
using SubtagAlarmConsumer consumer = new SubtagAlarmConsumer(source, watchList);
// Collect emitted transitions on the STA (handler fires on the same
// STA that services the COM OnDataChange callback).
ConcurrentQueue<MxAlarmTransitionEvent> transitions =
new ConcurrentQueue<MxAlarmTransitionEvent>();
consumer.AlarmTransitionEmitted += (_, e) =>
{
Log(string.Format("Transition emitted: {0}", DescribeTransition(e)));
transitions.Enqueue(e);
};
// Subscribe binds Advise on all observable subtags (Active, Acked,
// Priority). The subscription expression is unused in subtag mode; pass
// something recognizable for diagnostics.
string subscriptionExpression = string.Format(@"\\{0}\Galaxy!{1}", Environment.MachineName, Group);
Log(string.Format("Calling Subscribe({0}) ...", subscriptionExpression));
consumer.Subscribe(subscriptionExpression);
Log("Subscribe returned OK.");
// 1. Wait for a Raise transition. The alarm flip script (same one
// used by AlarmsLiveSmokeTests) writes the active subtag on a
// ~10 s cadence. LmxSubtagAlarmSource delivers OnDataChange via
// the Windows message pump on the STA, so we must pump messages
// here while we wait — mirroring how AlarmsLiveSmokeTests drives
// its WnWrapAlarmConsumer.PollOnce() from the STA in a tight loop.
Log(string.Format("Waiting up to {0}s for a Raise transition ...", RaiseWaitTimeout.TotalSeconds));
MxAlarmTransitionEvent? raiseEvent = WaitForTransitionKind(
transitions, AlarmTransitionKind.Raise, RaiseWaitTimeout, "Raise");
Assert.NotNull(raiseEvent);
Assert.True(raiseEvent!.Record.Degraded,
"Subtag-synthesized records must have Degraded=true.");
// SubtagAlarmConsumer must stamp a synthetic GUID on the transition.
Assert.NotEqual(Guid.Empty, raiseEvent.Record.AlarmGuid);
// A Raise transition must leave the record in UnackAlm state.
Assert.Equal(MxAlarmStateKind.UnackAlm, raiseEvent.Record.State);
Log(string.Format("Raise confirmed: AlarmGuid={0} Degraded={1} State={2}",
raiseEvent.Record.AlarmGuid, raiseEvent.Record.Degraded, raiseEvent.Record.State));
// 2. Snapshot active alarms and confirm the raised alarm is present.
IReadOnlyList<MxAlarmSnapshotRecord> snapshot = consumer.SnapshotActiveAlarms();
Log(string.Format("SnapshotActiveAlarms count={0}", snapshot.Count));
foreach (MxAlarmSnapshotRecord s in snapshot)
{
Log(string.Format(" snapshot: TagName='{0}' Group='{1}' State={2} Degraded={3} Guid={4}",
s.TagName, s.Group, s.State, s.Degraded, s.AlarmGuid));
}
bool foundInSnapshot = false;
foreach (MxAlarmSnapshotRecord s in snapshot)
{
if (string.Equals(
AlarmRecordTransitionMapper.ComposeFullReference(s.ProviderName, s.Group, s.TagName),
AlarmFullReference,
StringComparison.OrdinalIgnoreCase))
{
foundInSnapshot = true;
break;
}
}
Assert.True(foundInSnapshot,
string.Format("Active alarm snapshot must contain '{0}' after a Raise.", AlarmFullReference));
// 3. AcknowledgeByName — writes the ack-comment subtag.
// The dispatcher derives (alarmName, provider, group) via the same
// TryParseReference logic as AlarmsLiveSmokeTests.
Log("Calling AcknowledgeByName ...");
int rc = consumer.AcknowledgeByName(
alarmName: AlarmTagName,
providerName: Provider,
groupName: Group,
ackComment: "subtag-fallback-smoke ack",
ackOperatorName: Environment.UserName,
ackOperatorNode: Environment.MachineName,
ackOperatorDomain: Environment.UserDomainName ?? string.Empty,
ackOperatorFullName: Environment.UserName);
Log(string.Format("AcknowledgeByName -> rc={0}", rc));
Assert.Equal(0, rc);
// 4. Wait for a Clear or Acknowledge transition to confirm the state
// machine continues tracking after the ack write.
Log(string.Format("Waiting up to {0}s for a Clear or Acknowledge transition ...",
ClearWaitTimeout.TotalSeconds));
MxAlarmTransitionEvent? postAckEvent = WaitForAnyTransition(transitions, ClearWaitTimeout, "post-ack");
// A null here is not a hard failure: the test alarm may not have
// cleared within the window, and the ack-comment write already
// confirmed the subtag path is wired. Log and assert non-null
// only when the post-ack transition is expected.
if (postAckEvent != null)
{
AlarmTransitionKind postAckKind = AlarmRecordTransitionMapper.MapTransition(
postAckEvent.PreviousState, postAckEvent.Record.State);
Log(string.Format("Post-ack transition: {0}", DescribeTransition(postAckEvent)));
Assert.NotEqual(AlarmTransitionKind.Unspecified, postAckKind);
}
else
{
Log("No post-ack transition within timeout — ack-comment write succeeded; state machine still live.");
}
Log("Smoke test complete.");
}
private MxAlarmTransitionEvent? WaitForTransitionKind(
ConcurrentQueue<MxAlarmTransitionEvent> queue,
AlarmTransitionKind kind,
TimeSpan timeout,
string label)
{
DateTime deadline = DateTime.UtcNow + timeout;
while (DateTime.UtcNow < deadline)
{
// Pump the STA Windows message queue so COM OnDataChange callbacks
// can be delivered. LMXProxyServerClass is apartment-threaded and
// requires the STA to be pumping; a bare Thread.Sleep would stall
// the pump.
PumpMessages();
MxAlarmTransitionEvent? evt;
if (queue.TryDequeue(out evt) && evt != null)
{
AlarmTransitionKind evtKind = AlarmRecordTransitionMapper.MapTransition(
evt.PreviousState, evt.Record.State);
if (evtKind == kind)
{
return evt;
}
Log(string.Format("Skipped transition kind={0} while waiting for {1}.", evtKind, label));
// Can't re-enqueue; log and continue.
}
Thread.Sleep(250);
}
Log(string.Format("Timed out waiting for {0} transition after {1}s.", label, timeout.TotalSeconds));
return null;
}
private MxAlarmTransitionEvent? WaitForAnyTransition(
ConcurrentQueue<MxAlarmTransitionEvent> queue,
TimeSpan timeout,
string label)
{
DateTime deadline = DateTime.UtcNow + timeout;
while (DateTime.UtcNow < deadline)
{
PumpMessages();
MxAlarmTransitionEvent? evt;
if (queue.TryDequeue(out evt) && evt != null)
{
return evt;
}
Thread.Sleep(250);
}
Log(string.Format("Timed out waiting for any transition ({0}) after {1}s.", label, timeout.TotalSeconds));
return null;
}
/// <summary>
/// Runs a single pass of the Windows STA message pump using a
/// non-blocking PeekMessage/DispatchMessage loop so COM
/// <c>OnDataChange</c> callbacks from <c>LMXProxyServerClass</c>
/// (ThreadingModel=Apartment) can be delivered on this thread.
/// Mirrors the pump pattern documented in
/// <c>docs/MxAccessWorkerInstanceDesign.md</c>.
/// </summary>
private static void PumpMessages()
{
NativeMethods.MSG msg;
// Drain all currently posted messages; return as soon as the queue
// is empty.
while (NativeMethods.PeekMessage(out msg, IntPtr.Zero, 0, 0, NativeMethods.PM_REMOVE))
{
NativeMethods.TranslateMessage(ref msg);
NativeMethods.DispatchMessage(ref msg);
}
}
private static string DescribeTransition(MxAlarmTransitionEvent e)
{
AlarmTransitionKind kind = AlarmRecordTransitionMapper.MapTransition(
e.PreviousState, e.Record.State);
return string.Format(
"Kind={0} PreviousState={1} State={2} TagName='{3}' Group='{4}' Provider='{5}' Degraded={6} Guid={7}",
kind,
e.PreviousState,
e.Record.State,
e.Record.TagName,
e.Record.Group,
e.Record.ProviderName,
e.Record.Degraded,
e.Record.AlarmGuid);
}
private void Log(string line)
{
log.Enqueue(string.Format("[t={0:F3}s] {1}", elapsed.Elapsed.TotalSeconds, line));
}
// -------------------------------------------------------------------------
// Minimal P/Invoke shim so the STA pump can be driven without pulling in
// the full StaRuntime machinery from the Worker project. The signatures
// mirror those in MxAccessStaRuntime and are well-known Win32.
// -------------------------------------------------------------------------
private static class NativeMethods
{
internal const uint PM_REMOVE = 0x0001;
[StructLayout(LayoutKind.Sequential)]
internal struct MSG
{
internal IntPtr hwnd;
internal uint message;
internal IntPtr wParam;
internal IntPtr lParam;
internal uint time;
internal int ptX;
internal int ptY;
}
[DllImport("user32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
internal static extern bool PeekMessage(
out MSG lpMsg,
IntPtr hWnd,
uint wMsgFilterMin,
uint wMsgFilterMax,
uint wRemoveMsg);
[DllImport("user32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
internal static extern bool TranslateMessage(ref MSG lpMsg);
[DllImport("user32.dll")]
internal static extern IntPtr DispatchMessage(ref MSG lpmsg);
}
}
@@ -51,7 +51,7 @@ public sealed class WorkerPipeSession
options,
() => Process.GetCurrentProcess().Id,
new WorkerPipeSessionOptions(),
() => new MxAccessStaSession((eq, affinity) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity)),
() => new MxAccessStaSession((eq, affinity, comFactory) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null)),
logger)
{
}
@@ -72,7 +72,7 @@ public sealed class WorkerPipeSession
options,
processIdProvider,
new WorkerPipeSessionOptions(),
() => new MxAccessStaSession((eq, affinity) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity)),
() => new MxAccessStaSession((eq, affinity, comFactory) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null)),
logger: null)
{
}
@@ -867,7 +867,7 @@ public sealed class WorkerPipeSession
// parameterless CompleteStartupHandshakeAsync is used without a
// prior factory call.
_runtimeSession ??= new MxAccessStaSession(
(eq, affinity) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity));
(eq, affinity, comFactory) => new AlarmCommandHandler(eq, () => new WnWrapAlarmConsumer(), affinity, comFactory, standbyFactory: null));
IWorkerRuntimeSession session = _runtimeSession;
try
{
@@ -37,8 +37,14 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
private readonly MxAccessEventQueue eventQueue;
private readonly Func<IMxAccessAlarmConsumer> consumerFactory;
private readonly Action? threadAffinityCheck;
private readonly IMxAccessComObjectFactory? comFactory;
private readonly Func<IReadOnlyList<AlarmSubtagTarget>, IMxAccessAlarmConsumer>? standbyFactory;
private readonly MxAccessEventMapper mapper = new MxAccessEventMapper();
private readonly object syncRoot = new object();
private AlarmDispatcher? dispatcher;
private FailoverAlarmConsumer? failoverConsumer;
private EventHandler<AlarmProviderModeChange>? providerModeChangedHandler;
private string subscribeSessionId = string.Empty;
private bool disposed;
/// <summary>Initializes a new alarm command handler with the given event queue.</summary>
@@ -79,10 +85,49 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
MxAccessEventQueue eventQueue,
Func<IMxAccessAlarmConsumer> consumerFactory,
Action? threadAffinityCheck)
: this(eventQueue, consumerFactory, threadAffinityCheck, comFactory: null, standbyFactory: null)
{
}
/// <summary>
/// Full constructor that also threads the MXAccess COM-object factory and
/// an optional standby-consumer seam so the subscribe path can build the
/// subtag / failover consumers required by
/// <see cref="SubscribeAlarmsCommand.ForcedMode"/> and
/// <see cref="SubscribeAlarmsCommand.WatchList"/>.
/// </summary>
/// <param name="eventQueue">The event queue.</param>
/// <param name="consumerFactory">
/// Factory for the PRIMARY (alarmmgr) consumer — the existing
/// wnwrap-backed source. Used alone in alarmmgr mode and as the primary
/// of the failover composite in auto mode.
/// </param>
/// <param name="threadAffinityCheck">Optional STA thread-affinity guard.</param>
/// <param name="comFactory">
/// The MXAccess COM-object factory used to build the
/// <see cref="LmxSubtagAlarmSource"/> backing the subtag consumer. May be
/// <see langword="null"/> when a <paramref name="standbyFactory"/> is
/// supplied (tests) or when only the alarmmgr path is ever exercised.
/// </param>
/// <param name="standbyFactory">
/// Optional seam that builds the STANDBY (subtag) consumer from a watch
/// list. Defaults to a <see cref="SubtagAlarmConsumer"/> over an
/// <see cref="LmxSubtagAlarmSource"/> built from
/// <paramref name="comFactory"/>. Tests inject a fake so they need no
/// live COM factory.
/// </param>
public AlarmCommandHandler(
MxAccessEventQueue eventQueue,
Func<IMxAccessAlarmConsumer> consumerFactory,
Action? threadAffinityCheck,
IMxAccessComObjectFactory? comFactory,
Func<IReadOnlyList<AlarmSubtagTarget>, IMxAccessAlarmConsumer>? standbyFactory)
{
this.eventQueue = eventQueue ?? throw new ArgumentNullException(nameof(eventQueue));
this.consumerFactory = consumerFactory ?? throw new ArgumentNullException(nameof(consumerFactory));
this.threadAffinityCheck = threadAffinityCheck;
this.comFactory = comFactory;
this.standbyFactory = standbyFactory;
}
/// <summary>Gets a value indicating whether the handler is subscribed.</summary>
@@ -92,10 +137,10 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
}
/// <inheritdoc />
public void Subscribe(string subscription, string sessionId)
public void Subscribe(SubscribeAlarmsCommand command, string sessionId)
{
if (disposed) throw new ObjectDisposedException(nameof(AlarmCommandHandler));
if (subscription is null) throw new ArgumentNullException(nameof(subscription));
if (command is null) throw new ArgumentNullException(nameof(command));
threadAffinityCheck?.Invoke();
lock (syncRoot)
@@ -106,17 +151,31 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
"AlarmCommandHandler already has an active subscription; " +
"call Unsubscribe before issuing another SubscribeAlarms command.");
}
IMxAccessAlarmConsumer consumer = consumerFactory()
subscribeSessionId = sessionId ?? string.Empty;
IMxAccessAlarmConsumer consumer = BuildConsumer(command)
?? throw new InvalidOperationException("Alarm consumer factory returned null.");
MxAccessAlarmEventSink sink = new MxAccessAlarmEventSink(
eventQueue, new MxAccessEventMapper());
dispatcher = new AlarmDispatcher(consumer, sink, sessionId ?? string.Empty);
// When the selected consumer is a failover composite, surface its
// provider switches onto the worker's event queue so connected
// gateway clients can observe degraded/recovered state. The handler
// is unsubscribed/disposed on Unsubscribe/Dispose below.
if (consumer is FailoverAlarmConsumer failover)
{
failoverConsumer = failover;
providerModeChangedHandler = OnProviderModeChanged;
failover.ProviderModeChanged += providerModeChangedHandler;
}
MxAccessAlarmEventSink sink = new MxAccessAlarmEventSink(eventQueue, mapper);
dispatcher = new AlarmDispatcher(consumer, sink, subscribeSessionId);
try
{
dispatcher.Subscribe(subscription);
dispatcher.Subscribe(command.SubscriptionExpression ?? string.Empty);
}
catch
{
DetachProviderModeChanged();
try { dispatcher.Dispose(); } catch { /* swallow */ }
dispatcher = null;
throw;
@@ -124,6 +183,89 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
}
}
/// <summary>
/// Selects and builds the alarm consumer from the command's
/// <see cref="SubscribeAlarmsCommand.ForcedMode"/> and
/// <see cref="SubscribeAlarmsCommand.WatchList"/>:
/// <list type="bullet">
/// <item><description>
/// <c>Alarmmgr</c>, or <c>Unspecified</c> with an empty watch
/// list: the existing primary (alarmmgr) consumer only —
/// today's behavior.
/// </description></item>
/// <item><description>
/// <c>Subtag</c>: a <see cref="SubtagAlarmConsumer"/> only.
/// </description></item>
/// <item><description>
/// <c>Unspecified</c> with a non-empty watch list (auto): a
/// <see cref="FailoverAlarmConsumer"/> over the primary and a
/// subtag standby.
/// </description></item>
/// </list>
/// </summary>
private IMxAccessAlarmConsumer BuildConsumer(SubscribeAlarmsCommand command)
{
List<AlarmSubtagTarget> watchList = new List<AlarmSubtagTarget>(command.WatchList);
if (command.ForcedMode == AlarmProviderMode.Subtag)
{
return BuildStandby(watchList);
}
if (command.ForcedMode == AlarmProviderMode.Unspecified && watchList.Count > 0)
{
IMxAccessAlarmConsumer primary = consumerFactory()
?? throw new InvalidOperationException("Alarm consumer factory returned null.");
IMxAccessAlarmConsumer standby = BuildStandby(watchList);
AlarmFailoverConfig? failoverConfig = command.Failover;
FailoverSettings settings = new FailoverSettings(
failoverConfig?.ConsecutiveFailureThreshold ?? 3,
failoverConfig?.FailbackProbeIntervalSeconds ?? 30,
failoverConfig?.FailbackStableProbes ?? 3);
return new FailoverAlarmConsumer(primary, standby, settings);
}
// Alarmmgr, or Unspecified with an empty watch list — primary only.
return consumerFactory()
?? throw new InvalidOperationException("Alarm consumer factory returned null.");
}
private IMxAccessAlarmConsumer BuildStandby(IReadOnlyList<AlarmSubtagTarget> watchList)
{
if (standbyFactory is not null)
{
return standbyFactory(watchList)
?? throw new InvalidOperationException("Standby alarm consumer factory returned null.");
}
if (comFactory is null)
{
throw new InvalidOperationException(
"Subtag alarm consumer requires an IMxAccessComObjectFactory; the alarm command "
+ "handler was constructed without one and no standby factory was supplied.");
}
return new SubtagAlarmConsumer(new LmxSubtagAlarmSource(comFactory), watchList);
}
private void OnProviderModeChanged(object? sender, AlarmProviderModeChange change)
{
if (change is null) return;
eventQueue.Enqueue(mapper.CreateOnAlarmProviderModeChanged(
subscribeSessionId, change.Mode, change.Reason, change.HResult, change.AtUtc));
}
private void DetachProviderModeChanged()
{
if (failoverConsumer is not null && providerModeChangedHandler is not null)
{
try { failoverConsumer.ProviderModeChanged -= providerModeChangedHandler; }
catch { /* swallow */ }
}
failoverConsumer = null;
providerModeChangedHandler = null;
}
/// <inheritdoc />
public void Unsubscribe()
{
@@ -131,6 +273,7 @@ public sealed class AlarmCommandHandler : IAlarmCommandHandler
AlarmDispatcher? toDispose;
lock (syncRoot)
{
DetachProviderModeChanged();
toDispose = dispatcher;
dispatcher = null;
}
@@ -191,7 +191,8 @@ public sealed class AlarmDispatcher : IDisposable
operatorUser: record.OperatorName,
operatorComment: record.AlarmComment,
category: record.Group,
description: string.Empty);
description: string.Empty,
degraded: record.Degraded);
}
private static ActiveAlarmSnapshot MapToSnapshot(MxAlarmSnapshotRecord record)
@@ -208,6 +209,8 @@ public sealed class AlarmDispatcher : IDisposable
OperatorComment = record.AlarmComment,
Category = record.Group,
Description = string.Empty,
Degraded = record.Degraded,
SourceProvider = record.Degraded ? AlarmProviderMode.Subtag : AlarmProviderMode.Alarmmgr,
};
if (record.TransitionTimestampUtc != DateTime.MinValue)
{
@@ -0,0 +1,61 @@
using System;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Raised by <see cref="FailoverAlarmConsumer"/> every time the active
/// alarm source switches between the primary (alarmmgr) consumer and the
/// standby (subtag) consumer. The worker translates this into the proto
/// family <c>OnAlarmProviderModeChanged</c> so connected gateway clients
/// can surface the degraded/recovered state.
/// </summary>
/// <remarks>
/// <para>
/// Plain class with constructor-assigned get-only properties — not a
/// <c>record</c> or <c>init</c>-only type — because the worker
/// multi-targets .NET Framework 4.8, which lacks
/// <c>System.Runtime.CompilerServices.IsExternalInit</c> (CS0518).
/// </para>
/// </remarks>
public sealed class AlarmProviderModeChange : EventArgs
{
/// <summary>
/// Initializes the change event payload.
/// </summary>
/// <param name="mode">The provider mode now active after the switch.</param>
/// <param name="reason">Human-readable reason for the switch.</param>
/// <param name="hResult">
/// The COM HRESULT that triggered a failover, or 0 for a clean
/// failback / no associated HRESULT.
/// </param>
/// <param name="atUtc">The UTC instant the switch occurred.</param>
public AlarmProviderModeChange(AlarmProviderMode mode, string reason, int hResult, DateTime atUtc)
{
Mode = mode;
Reason = reason ?? string.Empty;
HResult = hResult;
AtUtc = atUtc;
}
/// <summary>
/// The provider mode now active after the switch.
/// </summary>
public AlarmProviderMode Mode { get; }
/// <summary>
/// Human-readable reason for the switch (e.g. the failing COM call, or
/// <c>"recovered"</c> for a failback).
/// </summary>
public string Reason { get; }
/// <summary>
/// The COM HRESULT that triggered a failover, or 0 when none applies.
/// </summary>
public int HResult { get; }
/// <summary>
/// The UTC instant the switch occurred.
/// </summary>
public DateTime AtUtc { get; }
}
@@ -0,0 +1,368 @@
using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Composite <see cref="IMxAccessAlarmConsumer"/> that owns a PRIMARY
/// consumer (the wnwrap <see cref="WnWrapAlarmConsumer"/> alarmmgr source)
/// and a STANDBY consumer (the <c>SubtagAlarmConsumer</c> subtag fallback),
/// and switches between them automatically:
/// <list type="bullet">
/// <item><description>
/// Auto-fails-over to standby after
/// <see cref="FailoverSettings.Threshold"/> consecutive COM
/// failures on the primary.
/// </description></item>
/// <item><description>
/// Auto-fails-back to primary after
/// <see cref="FailoverSettings.StableProbes"/> consecutive clean
/// failback probes against the recovering primary.
/// </description></item>
/// </list>
/// It re-raises <see cref="AlarmTransitionEmitted"/> from whichever child
/// is active and raises <see cref="ProviderModeChanged"/> on every switch.
/// </summary>
/// <remarks>
/// <para>
/// <strong>Active-child event forwarding.</strong> This type subscribes
/// to <em>both</em> children's <see cref="AlarmTransitionEmitted"/>
/// events up front and gates re-raising on identity: a child transition
/// is forwarded only when its <c>sender</c> is the currently active
/// child. The standby is armed (subscribed) from the start so its
/// snapshot is warm at the moment of failover, but its transitions stay
/// suppressed until it becomes active. Gating-by-active is simpler and
/// less error-prone than subscribe/unsubscribe churn on every switch,
/// and it avoids a race where a transition fires during the switch.
/// </para>
/// <para>
/// <strong>Threading.</strong> Like its children, this type is driven
/// entirely on the worker's STA: <see cref="Subscribe"/>,
/// <see cref="PollOnce"/>, <see cref="ProbeOnce"/>, and the
/// <c>AcknowledgeBy*</c> calls are all invoked from the apartment that
/// owns the underlying COM objects. It owns no locks of its own and no
/// internal timer; the worker drives <see cref="PollOnce"/> on a timer.
/// </para>
/// </remarks>
public sealed class FailoverAlarmConsumer : IMxAccessAlarmConsumer
{
private enum Active
{
Primary,
Standby,
}
private readonly IMxAccessAlarmConsumer primary;
private readonly IMxAccessAlarmConsumer standby;
private readonly FailoverSettings settings;
private Active active = Active.Primary;
private AlarmProviderMode mode = AlarmProviderMode.Alarmmgr;
private int consecutiveFailures;
private int cleanProbes;
private bool disposed;
private DateTime lastProbeAtUtc = DateTime.MinValue;
/// <summary>
/// The subscription expression passed to <see cref="Subscribe"/>.
/// Stored for documentation and potential future full re-subscribe
/// scenarios; the primary is NOT re-subscribed during probing.
/// </summary>
private string subscriptionExpression = string.Empty;
/// <summary>
/// Composes the failover consumer over its two children.
/// </summary>
/// <param name="primary">The PRIMARY (alarmmgr) consumer.</param>
/// <param name="standby">The STANDBY (subtag) consumer.</param>
/// <param name="settings">The failover/failback tunables.</param>
public FailoverAlarmConsumer(
IMxAccessAlarmConsumer primary,
IMxAccessAlarmConsumer standby,
FailoverSettings settings)
{
this.primary = primary ?? throw new ArgumentNullException(nameof(primary));
this.standby = standby ?? throw new ArgumentNullException(nameof(standby));
this.settings = settings ?? throw new ArgumentNullException(nameof(settings));
this.primary.AlarmTransitionEmitted += OnChildTransition;
this.standby.AlarmTransitionEmitted += OnChildTransition;
}
/// <inheritdoc />
public event EventHandler<MxAlarmTransitionEvent>? AlarmTransitionEmitted;
/// <summary>
/// Fires on every switch between primary and standby. Carries the new
/// <see cref="AlarmProviderMode"/>, the reason, the triggering HRESULT
/// (0 for a clean failback), and the UTC instant.
/// </summary>
public event EventHandler<AlarmProviderModeChange>? ProviderModeChanged;
/// <summary>
/// The provider mode currently active.
/// </summary>
public AlarmProviderMode Mode => mode;
/// <inheritdoc />
/// <remarks>
/// Arms BOTH children up front so the standby snapshot is warm at the
/// moment of failover. The standby is always subscribed even if the
/// primary's <c>Subscribe</c> throws; a standby subscribe failure is
/// surfaced (rethrown) but does not count toward primary failover. The
/// primary subscribe runs through the failure-counting wrapper so a
/// COM failure on subscribe contributes to the failover threshold.
/// </remarks>
public void Subscribe(string subscription)
{
if (disposed) throw new ObjectDisposedException(nameof(FailoverAlarmConsumer));
// Store for documentation; the primary is not torn down on failover
// and is therefore not re-subscribed during ProbeOnce.
subscriptionExpression = subscription;
// Arm the standby first so it is warm regardless of primary outcome.
// A standby subscribe failure is a hard fault (the fallback itself is
// broken) and is surfaced to the caller; it does not feed the primary
// failover counter.
standby.Subscribe(subscription);
// Drive the primary subscribe through the failure-counting wrapper so
// a COM failure here counts toward the failover threshold instead of
// escaping. Swallowing the exception is deliberate: the standby is
// already armed, so a failed primary subscribe just nudges the state
// machine toward (or into) standby rather than aborting startup.
RunPrimary(() => primary.Subscribe(subscription), "Subscribe");
}
/// <inheritdoc />
/// <remarks>
/// While the primary is active, drives <c>primary.PollOnce</c> through
/// the failure-counting wrapper. While degraded (standby active),
/// drives <c>standby.PollOnce</c> and then runs one failback probe per
/// call via <see cref="ProbeOnce"/> — the worker drives this on a
/// timer, so one degraded poll equals one probe tick.
/// </remarks>
public void PollOnce()
{
if (disposed) throw new ObjectDisposedException(nameof(FailoverAlarmConsumer));
if (active == Active.Primary)
{
RunPrimary(() => primary.PollOnce(), "PollOnce");
return;
}
// Degraded: pump the standby for live transitions, then probe the
// primary for recovery. Standby PollOnce is a no-op for the subtag
// consumer but kept for symmetry / future standby sources.
standby.PollOnce();
ProbeOnce();
}
/// <summary>
/// Runs one failback probe against the (presumed recovering) primary.
/// Only meaningful while the standby is active; a no-op otherwise.
/// </summary>
/// <remarks>
/// <para>
/// A clean probe (primary <c>PollOnce</c> succeeds without
/// throwing) increments the clean-probe counter and, once it reaches
/// <see cref="FailoverSettings.StableProbes"/>, fails back to the
/// primary. Any probe failure resets the clean-probe counter to 0 so
/// the consumer requires a fresh unbroken run before failing back.
/// Exposed publicly so tests (and any external scheduler honoring
/// <see cref="FailoverSettings.ProbeIntervalSeconds"/> cadence) can
/// drive it directly.
/// </para>
/// <para>
/// <strong>Probe throttle.</strong> When
/// <see cref="FailoverSettings.ProbeIntervalSeconds"/> is greater than
/// zero, successive calls to this method are throttled: a probe is
/// skipped unless at least that many seconds have elapsed since the
/// last probe that was actually executed. When
/// <see cref="FailoverSettings.ProbeIntervalSeconds"/> is zero, the
/// throttle is disabled and every call probes immediately (the default
/// used by unit tests).
/// </para>
/// <para>
/// <strong>Why PollOnce only — no re-Subscribe.</strong>
/// Failover does NOT tear down the primary's subscription;
/// <see cref="WnWrapAlarmConsumer"/> is single-subscribe and would
/// throw <see cref="InvalidOperationException"/> on a second call.
/// The probe therefore re-polls the still-subscribed primary:
/// when the underlying COM provider recovers, <c>PollOnce</c> stops
/// throwing and clean probes accumulate toward failback. This covers
/// the dominant failure mode (transient COM/provider fault after a
/// successful initial subscribe).
/// </para>
/// <para>
/// <strong>Known v1 limitation.</strong> If the <em>original</em>
/// <c>Subscribe</c> itself failed (i.e., the primary never reached a
/// subscribed state — only reachable when
/// <see cref="FailoverSettings.Threshold"/> is 1), polling alone
/// cannot re-establish the subscription. That edge case is accepted
/// for v1: the operator must restart the session to force a fresh
/// subscribe attempt.
/// </para>
/// </remarks>
public void ProbeOnce()
{
if (disposed) throw new ObjectDisposedException(nameof(FailoverAlarmConsumer));
if (active != Active.Standby) return;
// Throttle probes to the configured cadence. When ProbeIntervalSeconds
// is 0 the throttle is disabled and every call probes immediately.
if (settings.ProbeIntervalSeconds > 0
&& (DateTime.UtcNow - lastProbeAtUtc).TotalSeconds < settings.ProbeIntervalSeconds)
{
return;
}
lastProbeAtUtc = DateTime.UtcNow;
try
{
// Re-poll the still-subscribed primary. Do NOT call Subscribe —
// WnWrapAlarmConsumer is single-subscribe and the primary remains
// subscribed across the failover; calling Subscribe again would
// always throw InvalidOperationException and prevent failback.
primary.PollOnce();
}
catch (Exception)
{
// Probe failed — the primary is still unhealthy. Demand a fresh
// unbroken run of StableProbes clean polls before failing back.
cleanProbes = 0;
return;
}
cleanProbes++;
if (cleanProbes >= settings.StableProbes)
{
SwitchToPrimary("recovered", 0);
}
}
/// <inheritdoc />
public int AcknowledgeByGuid(
Guid alarmGuid,
string ackComment,
string ackOperatorName,
string ackOperatorNode,
string ackOperatorDomain,
string ackOperatorFullName)
{
if (disposed) throw new ObjectDisposedException(nameof(FailoverAlarmConsumer));
return ActiveChild.AcknowledgeByGuid(
alarmGuid, ackComment, ackOperatorName, ackOperatorNode, ackOperatorDomain, ackOperatorFullName);
}
/// <inheritdoc />
public int AcknowledgeByName(
string alarmName,
string providerName,
string groupName,
string ackComment,
string ackOperatorName,
string ackOperatorNode,
string ackOperatorDomain,
string ackOperatorFullName)
{
if (disposed) throw new ObjectDisposedException(nameof(FailoverAlarmConsumer));
return ActiveChild.AcknowledgeByName(
alarmName, providerName, groupName, ackComment,
ackOperatorName, ackOperatorNode, ackOperatorDomain, ackOperatorFullName);
}
/// <inheritdoc />
public IReadOnlyList<MxAlarmSnapshotRecord> SnapshotActiveAlarms()
{
if (disposed) throw new ObjectDisposedException(nameof(FailoverAlarmConsumer));
return ActiveChild.SnapshotActiveAlarms();
}
private IMxAccessAlarmConsumer ActiveChild => active == Active.Primary ? primary : standby;
/// <summary>
/// Runs a primary COM action, counting consecutive failures. A
/// <see cref="COMException"/> (or any exception, treated as a COM
/// failure) increments the failure counter and, at
/// <see cref="FailoverSettings.Threshold"/> while the primary is still
/// active, switches to the standby. A success resets the counter.
/// </summary>
private void RunPrimary(Action action, string operation)
{
try
{
action();
}
catch (Exception ex) when (ex is not OutOfMemoryException)
{
consecutiveFailures++;
int hresult = ex is COMException ? ex.HResult : 0;
if (active == Active.Primary && consecutiveFailures >= settings.Threshold)
{
SwitchToStandby($"primary {operation} failed", hresult);
}
return;
}
consecutiveFailures = 0;
}
private void SwitchToStandby(string reason, int hresult)
{
active = Active.Standby;
mode = AlarmProviderMode.Subtag;
consecutiveFailures = 0;
cleanProbes = 0;
// Warm the standby snapshot for the gateway hand-off. The gateway
// reconciles state from this snapshot, so the return value is not
// consumed here — the call exists for its priming side effect.
_ = standby.SnapshotActiveAlarms();
RaiseModeChanged(AlarmProviderMode.Subtag, reason, hresult);
}
private void SwitchToPrimary(string reason, int hresult)
{
active = Active.Primary;
mode = AlarmProviderMode.Alarmmgr;
consecutiveFailures = 0;
cleanProbes = 0;
RaiseModeChanged(AlarmProviderMode.Alarmmgr, reason, hresult);
}
private void RaiseModeChanged(AlarmProviderMode newMode, string reason, int hresult)
{
ProviderModeChanged?.Invoke(
this,
new AlarmProviderModeChange(newMode, reason, hresult, DateTime.UtcNow));
}
private void OnChildTransition(object? sender, MxAlarmTransitionEvent e)
{
// Gate by active child: forward only the active source's transitions.
if (ReferenceEquals(sender, ActiveChild))
{
AlarmTransitionEmitted?.Invoke(this, e);
}
}
/// <inheritdoc />
public void Dispose()
{
if (disposed) return;
disposed = true;
primary.AlarmTransitionEmitted -= OnChildTransition;
standby.AlarmTransitionEmitted -= OnChildTransition;
primary.Dispose();
standby.Dispose();
}
}
@@ -0,0 +1,57 @@
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Tunables for <see cref="FailoverAlarmConsumer"/>'s auto-failover /
/// auto-failback state machine. Constructor-clamped to safe minimums so a
/// misconfigured options bind can never produce a zero/negative threshold
/// that would either never fail over or fail over on the first hiccup
/// unintentionally.
/// </summary>
/// <remarks>
/// <para>
/// Plain class with constructor-assigned get-only properties — not a
/// <c>record</c> or <c>init</c>-only type — because the worker
/// multi-targets .NET Framework 4.8, which lacks
/// <c>System.Runtime.CompilerServices.IsExternalInit</c> and so cannot
/// compile <c>init</c> accessors or positional records (CS0518).
/// </para>
/// </remarks>
public sealed class FailoverSettings
{
/// <summary>
/// Initializes the settings, clamping each value to its safe minimum.
/// </summary>
/// <param name="threshold">
/// Consecutive primary COM failures that trigger a switch to standby.
/// Clamped to a minimum of 1.
/// </param>
/// <param name="probeIntervalSeconds">
/// Minimum spacing (seconds) between failback probes against the
/// recovering primary. Clamped to a minimum of 0 (probe every tick).
/// </param>
/// <param name="stableProbes">
/// Consecutive clean failback probes required before switching back to
/// the primary. Clamped to a minimum of 1.
/// </param>
public FailoverSettings(int threshold, int probeIntervalSeconds, int stableProbes)
{
Threshold = threshold < 1 ? 1 : threshold;
ProbeIntervalSeconds = probeIntervalSeconds < 0 ? 0 : probeIntervalSeconds;
StableProbes = stableProbes < 1 ? 1 : stableProbes;
}
/// <summary>
/// Consecutive primary COM failures that trigger a switch to standby.
/// </summary>
public int Threshold { get; }
/// <summary>
/// Minimum spacing, in seconds, between failback probes.
/// </summary>
public int ProbeIntervalSeconds { get; }
/// <summary>
/// Consecutive clean failback probes required before failing back.
/// </summary>
public int StableProbes { get; }
}
@@ -13,10 +13,16 @@ namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// </summary>
public interface IAlarmCommandHandler : IDisposable
{
/// <summary>Begin a subscription against the supplied AVEVA alarm-provider expression.</summary>
/// <param name="subscription">The AVEVA alarm-provider subscription expression.</param>
/// <summary>
/// Begin an alarm subscription from the supplied command. The command's
/// <see cref="SubscribeAlarmsCommand.ForcedMode"/> and
/// <see cref="SubscribeAlarmsCommand.WatchList"/> select the consumer:
/// alarmmgr-only (the default), subtag-only, or an auto-failover
/// composite over both.
/// </summary>
/// <param name="command">The full SubscribeAlarms command.</param>
/// <param name="sessionId">The session identifier.</param>
void Subscribe(string subscription, string sessionId);
void Subscribe(SubscribeAlarmsCommand command, string sessionId);
/// <summary>Tear down the active subscription. No-op if not subscribed.</summary>
void Unsubscribe();
@@ -0,0 +1,24 @@
using System;
using System.Collections.Generic;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Advises a set of MXAccess subtag addresses and surfaces value changes.
/// The production implementation owns its own LMXProxyServerClass (later task);
/// tests substitute a fake that pushes SubtagValueChange values.
/// </summary>
public interface ISubtagAlarmSource : IDisposable
{
/// <summary>Raised when an advised subtag reports a new value.</summary>
event EventHandler<SubtagValueChange>? ValueChanged;
/// <summary>Begins advising the supplied MXAccess subtag item addresses.</summary>
/// <param name="itemAddresses">The subtag references to advise.</param>
void Advise(IReadOnlyCollection<string> itemAddresses);
/// <summary>Writes a value to the named MXAccess subtag (e.g. an ack request).</summary>
/// <param name="itemAddress">The subtag reference to write.</param>
/// <param name="value">The value to write.</param>
void Write(string itemAddress, object? value);
}
@@ -0,0 +1,334 @@
using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using ArchestrA.MxAccess;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Production <see cref="ISubtagAlarmSource"/> backed by its own MXAccess
/// <c>LMXProxyServerClass</c> COM object. It advises a set of alarm subtag
/// item addresses, normalizes each <c>OnDataChange</c> COM callback into a
/// <see cref="SubtagValueChange"/>, and supports writing a value (e.g. an
/// ack-comment) to a subtag.
/// </summary>
/// <remarks>
/// <para>
/// Like <see cref="WnWrapAlarmConsumer"/>, this type owns its
/// <em>own</em> MXAccess proxy-server COM object rather than sharing the
/// per-session item pipeline's <see cref="MxAccessSession"/>. That keeps
/// the subtag-fallback subscription isolated from the session's own
/// <c>Register</c>/<c>AddItem</c>/<c>Advise</c> bookkeeping: advising or
/// tearing down alarm subtags never perturbs the client's subscriptions.
/// </para>
/// <para>
/// <strong>Threading.</strong> The <c>LMXProxyServerClass</c> CLSID is
/// apartment-threaded and delivers <c>OnDataChange</c> via the STA
/// message pump. Every method on this type — the lazy <see cref="Advise"/>
/// creation, <see cref="Write"/>, and <see cref="Dispose"/> — must be
/// invoked on the worker's STA that created the source, exactly as
/// <see cref="MxAccessSession"/> and <see cref="WnWrapAlarmConsumer"/>
/// require. The <c>OnDataChange</c> COM event also arrives on that STA.
/// Because every call is confined to the single owning STA, no lock is
/// taken around the COM object or the handle maps: a lock here would risk
/// deadlocking the STA against a re-entrant pump callback, and the STA
/// affinity already serializes access.
/// </para>
/// </remarks>
public sealed class LmxSubtagAlarmSource : ISubtagAlarmSource
{
private const string DefaultClientName = "OtOpcUa.ZB.MOM.WW.MxGateway.Worker.SubtagAlarms";
private readonly IMxAccessComObjectFactory? factory;
private readonly string clientName;
private readonly Dictionary<string, int> itemHandlesByAddress =
new Dictionary<string, int>(StringComparer.Ordinal);
private readonly Dictionary<int, string> addressesByItemHandle =
new Dictionary<int, string>();
private object? mxAccessComObject;
private IMxAccessServer? server;
private LMXProxyServerClass? comEventSource;
private int serverHandle;
private bool registered;
private bool disposed;
/// <summary>
/// Production constructor. The MXAccess COM object is created lazily on
/// the first <see cref="Advise"/> call (which must run on the worker's
/// STA) via the supplied <paramref name="factory"/>.
/// </summary>
/// <param name="factory">Factory that creates the MXAccess proxy-server COM object.</param>
/// <param name="clientName">
/// Optional MXAccess client/registration name used for
/// <see cref="IMxAccessServer.Register"/>; a worker-specific default is
/// used when null or whitespace. Mirrors the registration name
/// <see cref="MxAccessSession.Register"/> accepts.
/// </param>
/// <exception cref="ArgumentNullException"><paramref name="factory"/> is null.</exception>
public LmxSubtagAlarmSource(
IMxAccessComObjectFactory factory,
string? clientName = null)
{
this.factory = factory ?? throw new ArgumentNullException(nameof(factory));
this.clientName = string.IsNullOrWhiteSpace(clientName) ? DefaultClientName : clientName!;
}
/// <summary>
/// Test-only seam: constructs a source over a caller-supplied
/// <see cref="IMxAccessServer"/> with a pre-registered server handle,
/// bypassing the live COM factory. Exposed via
/// <c>InternalsVisibleTo("ZB.MOM.WW.MxGateway.Worker.Tests")</c> so tests can
/// exercise <see cref="Advise"/>/<see cref="Write"/> and
/// <see cref="HandleDataChange"/> without a live <c>LMXProxyServerClass</c>.
/// No COM event source is wired in this mode — tests drive
/// <see cref="HandleDataChange"/> directly to simulate an
/// <c>OnDataChange</c> callback.
/// </summary>
/// <param name="server">The server abstraction to drive.</param>
/// <param name="serverHandle">The already-registered server handle to use.</param>
internal LmxSubtagAlarmSource(
IMxAccessServer server,
int serverHandle)
{
this.factory = null;
this.clientName = DefaultClientName;
this.server = server ?? throw new ArgumentNullException(nameof(server));
this.serverHandle = serverHandle;
this.registered = true;
}
/// <inheritdoc />
public event EventHandler<SubtagValueChange>? ValueChanged;
/// <inheritdoc />
/// <remarks>
/// Idempotent per address: an address already advised is skipped. The
/// MXAccess COM object is created and the client registered on the first
/// call. Must be invoked on the worker's STA.
/// </remarks>
public void Advise(IReadOnlyCollection<string> itemAddresses)
{
if (itemAddresses is null)
{
throw new ArgumentNullException(nameof(itemAddresses));
}
ThrowIfDisposed();
EnsureRegistered();
IMxAccessServer mxServer = server!;
foreach (string? itemAddress in itemAddresses)
{
if (string.IsNullOrWhiteSpace(itemAddress))
{
continue;
}
if (itemHandlesByAddress.ContainsKey(itemAddress!))
{
continue;
}
int itemHandle = mxServer.AddItem(serverHandle, itemAddress!);
mxServer.Advise(serverHandle, itemHandle);
itemHandlesByAddress[itemAddress!] = itemHandle;
addressesByItemHandle[itemHandle] = itemAddress!;
}
}
/// <inheritdoc />
/// <remarks>
/// Adds the item if it has not been added yet — the write may target a
/// subtag (e.g. the ack-comment) that was never advised — then writes
/// with MXAccess user id 0 (unsecured). Must be invoked on the worker's
/// STA.
/// </remarks>
public void Write(string itemAddress, object? value)
{
if (itemAddress is null)
{
throw new ArgumentNullException(nameof(itemAddress));
}
ThrowIfDisposed();
EnsureRegistered();
IMxAccessServer mxServer = server!;
if (!itemHandlesByAddress.TryGetValue(itemAddress, out int itemHandle))
{
itemHandle = mxServer.AddItem(serverHandle, itemAddress);
itemHandlesByAddress[itemAddress] = itemHandle;
addressesByItemHandle[itemHandle] = itemAddress;
}
mxServer.Write(serverHandle, itemHandle, value, userId: 0);
}
/// <summary>
/// Normalizes an MXAccess <c>OnDataChange</c> callback into a
/// <see cref="SubtagValueChange"/> and raises <see cref="ValueChanged"/>
/// when the item handle maps to an advised address. Unknown handles are
/// ignored. Exposed <c>internal</c> as a unit-test seam so the
/// normalization can be exercised without a live <c>LMXProxyServerClass</c>,
/// mirroring <c>MxAccessBaseEventSink.OnDataChange</c>.
/// </summary>
/// <param name="phItemHandle">The MXAccess item handle that changed.</param>
/// <param name="pvItemValue">The new item value (left as a boxed COM variant; the state machine coerces it).</param>
/// <param name="pftItemTimeStamp">The item timestamp as delivered by MXAccess (a culture-formatted string).</param>
internal void HandleDataChange(
int phItemHandle,
object? pvItemValue,
object? pftItemTimeStamp)
{
if (!addressesByItemHandle.TryGetValue(phItemHandle, out string? itemAddress))
{
return;
}
// MXAccess delivers OnDataChange timestamps as a culture-formatted
// string (VT_BSTR), not a FILETIME or VT_DATE — see
// MxAccessEventMapper.TryParseSourceTimestamp, which is the single
// conversion helper the per-session pipeline already uses. Reuse it so
// the subtag path interprets the host-local string the same way; fall
// back to capture time when the string is absent or unparsable.
DateTime timestampUtc = ConvertTimestampUtc(pftItemTimeStamp);
EventHandler<SubtagValueChange>? handler = ValueChanged;
handler?.Invoke(this, new SubtagValueChange
{
ItemAddress = itemAddress,
Value = pvItemValue,
TimestampUtc = timestampUtc,
});
}
/// <inheritdoc />
public void Dispose()
{
if (disposed)
{
return;
}
disposed = true;
// Unsubscribe the COM event before tearing handles down so a late pump
// callback cannot re-enter HandleDataChange mid-teardown.
if (comEventSource is not null)
{
try { comEventSource.OnDataChange -= OnDataChange; } catch { /* swallow — best effort */ }
}
IMxAccessServer? mxServer = server;
if (mxServer is not null && registered)
{
foreach (KeyValuePair<int, string> entry in addressesByItemHandle)
{
try { mxServer.UnAdvise(serverHandle, entry.Key); } catch { /* swallow — best effort */ }
try { mxServer.RemoveItem(serverHandle, entry.Key); } catch { /* swallow — best effort */ }
}
try { mxServer.Unregister(serverHandle); } catch { /* swallow — best effort */ }
}
itemHandlesByAddress.Clear();
addressesByItemHandle.Clear();
object? comToRelease = mxAccessComObject;
mxAccessComObject = null;
comEventSource = null;
server = null;
registered = false;
if (comToRelease is not null && Marshal.IsComObject(comToRelease))
{
try { Marshal.FinalReleaseComObject(comToRelease); } catch { /* swallow — best effort */ }
}
}
private void EnsureRegistered()
{
if (registered)
{
return;
}
// factory is only null in the test-only constructor, which sets
// registered = true, so this path is never hit there.
object created = factory!.Create()
?? throw new InvalidOperationException("MXAccess COM factory returned null.");
try
{
mxAccessComObject = created;
// Wire OnDataChange the same way MxAccessBaseEventSink.Attach does:
// cast to the concrete LMXProxyServerClass RCW and subscribe. Test
// doubles cannot be cast to LMXProxyServerClass, so the COM event
// subscription is exercised only on the live path; unit tests drive
// HandleDataChange directly via the internal constructor seam.
if (created is LMXProxyServerClass comObject)
{
comEventSource = comObject;
comObject.OnDataChange += OnDataChange;
}
server = new MxAccessComServer(created);
serverHandle = server.Register(clientName);
registered = true;
}
catch
{
if (comEventSource is not null)
{
try { comEventSource.OnDataChange -= OnDataChange; } catch { /* swallow */ }
comEventSource = null;
}
if (Marshal.IsComObject(created))
{
try { Marshal.FinalReleaseComObject(created); } catch { /* swallow */ }
}
mxAccessComObject = null;
server = null;
throw;
}
}
/// <summary>
/// COM <c>OnDataChange</c> delegate target. Forwards to
/// <see cref="HandleDataChange"/>; the quality, status array, and server
/// handle carry no information the subtag fallback needs.
/// </summary>
private void OnDataChange(
int hLMXServerHandle,
int phItemHandle,
object pvItemValue,
int pwItemQuality,
object pftItemTimeStamp,
ref MXSTATUS_PROXY[] pVars)
{
HandleDataChange(phItemHandle, pvItemValue, pftItemTimeStamp);
}
private static DateTime ConvertTimestampUtc(object? pftItemTimeStamp)
{
if (pftItemTimeStamp is not null
&& MxAccessEventMapper.TryParseSourceTimestamp(pftItemTimeStamp.ToString(), out DateTime parsedUtc))
{
return parsedUtc;
}
return DateTime.UtcNow;
}
private void ThrowIfDisposed()
{
if (disposed)
{
throw new ObjectDisposedException(nameof(LmxSubtagAlarmSource));
}
}
}
@@ -86,6 +86,12 @@ public sealed class MxAccessAlarmEventSink : IMxAccessEventSink
/// <param name="operatorComment">The operator's comment, if any.</param>
/// <param name="category">The alarm category.</param>
/// <param name="description">The alarm description.</param>
/// <param name="degraded">
/// <see langword="true"/> when the transition was synthesized by the
/// subtag-provider fallback rather than the native alarmmgr path.
/// Defaults to <see langword="false"/> so existing alarmmgr callers
/// compile unchanged and stay on the parity (alarmmgr) path.
/// </param>
internal void EnqueueTransition(
string alarmFullReference,
string sourceObjectReference,
@@ -97,10 +103,16 @@ public sealed class MxAccessAlarmEventSink : IMxAccessEventSink
string operatorUser,
string operatorComment,
string category,
string description)
string description,
bool degraded = false)
{
try
{
// Degraded transitions come from the subtag fallback; the native
// alarmmgr (wnwrap) path stays degraded=false / ALARMMGR for parity.
AlarmProviderMode sourceProvider = degraded
? AlarmProviderMode.Subtag
: AlarmProviderMode.Alarmmgr;
MxEvent mxEvent = eventMapper.CreateOnAlarmTransition(
sessionId,
alarmFullReference,
@@ -114,7 +126,9 @@ public sealed class MxAccessAlarmEventSink : IMxAccessEventSink
operatorComment,
category,
description,
statuses: null);
statuses: null,
degraded: degraded,
sourceProvider: sourceProvider);
eventQueue.Enqueue(mxEvent);
}
catch (Exception exception)
@@ -598,7 +598,8 @@ public sealed class MxAccessCommandExecutor : IStaCommandExecutor
"SubscribeAlarms requires an alarm command handler; the worker was constructed without one.");
}
string subscription = command.Command.SubscribeAlarms.SubscriptionExpression ?? string.Empty;
SubscribeAlarmsCommand subscribeCommand = command.Command.SubscribeAlarms;
string subscription = subscribeCommand.SubscriptionExpression ?? string.Empty;
if (string.IsNullOrWhiteSpace(subscription))
{
return CreateInvalidRequestReply(command, "SubscribeAlarms.subscription_expression is required.");
@@ -606,7 +607,7 @@ public sealed class MxAccessCommandExecutor : IStaCommandExecutor
try
{
alarmCommandHandler.Subscribe(subscription, command.SessionId);
alarmCommandHandler.Subscribe(subscribeCommand, command.SessionId);
return CreateOkReply(command);
}
catch (Exception ex)
@@ -124,6 +124,15 @@ public sealed class MxAccessEventMapper
/// <param name="category">Alarm taxonomy bucket from the Galaxy template.</param>
/// <param name="description">Human-readable alarm description.</param>
/// <param name="statuses">Array of MxStatusProxy values from MXAccess.</param>
/// <param name="degraded">
/// <see langword="true"/> when this transition was synthesized by the
/// subtag-provider fallback rather than the native alarmmgr path.
/// Defaults to <see langword="false"/> to preserve alarmmgr parity.
/// </param>
/// <param name="sourceProvider">
/// The alarm provider that sourced this transition. Defaults to
/// <see cref="AlarmProviderMode.Alarmmgr"/> for the native path.
/// </param>
public MxEvent CreateOnAlarmTransition(
string sessionId,
string alarmFullReference,
@@ -137,7 +146,9 @@ public sealed class MxAccessEventMapper
string operatorComment,
string category,
string description,
Array? statuses)
Array? statuses,
bool degraded = false,
AlarmProviderMode sourceProvider = AlarmProviderMode.Alarmmgr)
{
MxEvent mxEvent = CreateBaseEvent(
MxEventFamily.OnAlarmTransition,
@@ -159,6 +170,8 @@ public sealed class MxAccessEventMapper
OperatorComment = operatorComment ?? string.Empty,
Category = category ?? string.Empty,
Description = description ?? string.Empty,
Degraded = degraded,
SourceProvider = sourceProvider,
};
if (originalRaiseTimestampUtc is { } orts)
{
@@ -169,6 +182,43 @@ public sealed class MxAccessEventMapper
return mxEvent;
}
/// <summary>
/// Creates an OnAlarmProviderModeChanged event from a
/// <see cref="FailoverAlarmConsumer"/> provider switch. The worker's
/// alarm path drives this when the failover composite switches between
/// the alarmmgr primary and the subtag standby, so connected gateway
/// clients can observe the degraded/recovered provider state.
/// </summary>
/// <param name="sessionId">Identifier of the session.</param>
/// <param name="mode">The provider mode now active after the switch.</param>
/// <param name="reason">Human-readable reason for the switch.</param>
/// <param name="hresult">The COM HRESULT that triggered a failover, or 0 for a clean failback.</param>
/// <param name="atUtc">The UTC instant the switch occurred.</param>
public MxEvent CreateOnAlarmProviderModeChanged(
string sessionId,
AlarmProviderMode mode,
string reason,
int hresult,
DateTime atUtc)
{
MxEvent mxEvent = CreateBaseEvent(
MxEventFamily.OnAlarmProviderModeChanged,
sessionId,
serverHandle: 0,
itemHandle: 0,
statuses: null);
mxEvent.OnAlarmProviderModeChanged = new OnAlarmProviderModeChangedEvent
{
Mode = mode,
Reason = reason ?? string.Empty,
Hresult = hresult,
At = Google.Protobuf.WellKnownTypes.Timestamp.FromDateTime(
DateTime.SpecifyKind(atUtc, DateTimeKind.Utc)),
};
return mxEvent;
}
/// <summary>Creates an OnBufferedDataChange event from MXAccess COM event arguments.</summary>
/// <param name="sessionId">Identifier of the session.</param>
/// <param name="serverHandle">Handle returned by the worker.</param>
@@ -23,7 +23,10 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
// then invokes the guard at the entry of every method that touches the
// wnwrap consumer, matching the STA-affinity invariant already enforced
// for the poll path via EnsureOnAlarmConsumerThread.
private readonly Func<MxAccessEventQueue, Action, IAlarmCommandHandler>? alarmCommandHandlerFactory;
// Worker-9: the third arg is the session's IMxAccessComObjectFactory, so
// the handler can build the subtag-fallback source's own proxy-server COM
// object on this STA when a subscribe selects the subtag / failover path.
private readonly Func<MxAccessEventQueue, Action, IMxAccessComObjectFactory, IAlarmCommandHandler>? alarmCommandHandlerFactory;
private StaCommandDispatcher? commandDispatcher;
private MxAccessSession? session;
private IAlarmCommandHandler? alarmCommandHandler;
@@ -51,7 +54,7 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
/// of alarm-side commands.
/// </summary>
/// <param name="alarmCommandHandlerFactory">Factory that constructs the alarm-command handler.</param>
internal MxAccessStaSession(Func<MxAccessEventQueue, Action, IAlarmCommandHandler>? alarmCommandHandlerFactory)
internal MxAccessStaSession(Func<MxAccessEventQueue, Action, IMxAccessComObjectFactory, IAlarmCommandHandler>? alarmCommandHandlerFactory)
: this(
new StaRuntime(),
new MxAccessComObjectFactory(),
@@ -103,7 +106,7 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
StaRuntime staRuntime,
IMxAccessComObjectFactory factory,
MxAccessEventQueue eventQueue,
Func<MxAccessEventQueue, Action, IAlarmCommandHandler>? alarmCommandHandlerFactory)
Func<MxAccessEventQueue, Action, IMxAccessComObjectFactory, IAlarmCommandHandler>? alarmCommandHandlerFactory)
: this(staRuntime, factory, new MxAccessBaseEventSink(eventQueue), eventQueue, alarmCommandHandlerFactory)
{
}
@@ -141,7 +144,7 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
IMxAccessComObjectFactory factory,
IMxAccessEventSink eventSink,
MxAccessEventQueue eventQueue,
Func<MxAccessEventQueue, Action, IAlarmCommandHandler>? alarmCommandHandlerFactory)
Func<MxAccessEventQueue, Action, IMxAccessComObjectFactory, IAlarmCommandHandler>? alarmCommandHandlerFactory)
{
this.staRuntime = staRuntime ?? throw new ArgumentNullException(nameof(staRuntime));
this.factory = factory ?? throw new ArgumentNullException(nameof(factory));
@@ -209,9 +212,16 @@ public sealed class MxAccessStaSession : IWorkerRuntimeSession
// on convention alone; a future refactor that let a
// command run off-STA would silently deadlock on
// cross-apartment marshaling against the wnwrap consumer.
// Worker-9: the factory also receives the session's
// IMxAccessComObjectFactory so the subtag-fallback source
// (LmxSubtagAlarmSource) can create its OWN proxy-server COM
// object on this STA, isolated from the item pipeline's
// MxAccessSession. The factory call runs on the STA, so the
// resulting source is bound to the correct apartment.
alarmCommandHandler = alarmCommandHandlerFactory(
eventQueue,
EnsureOnAlarmConsumerThread);
EnsureOnAlarmConsumerThread,
factory);
}
commandDispatcher = new StaCommandDispatcher(
staRuntime,
@@ -6,6 +6,8 @@ namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// Single alarm record as emitted by the wnwrapConsumer XML stream.
/// Field names match the captured XML schema (see
/// <c>docs/AlarmClientDiscovery.md</c> "Option A — captured" section).
/// Records are also produced by the subtag synthesis path when
/// <see cref="Degraded"/> is set.
/// </summary>
public sealed class MxAlarmSnapshotRecord
{
@@ -37,4 +39,12 @@ public sealed class MxAlarmSnapshotRecord
public string OperatorName { get; set; } = string.Empty;
/// <summary>Gets or sets the alarm comment.</summary>
public string AlarmComment { get; set; } = string.Empty;
/// <summary>
/// Gets or sets a value indicating whether this record was synthesized
/// by the subtag-provider fallback rather than emitted by the native
/// alarmmgr (wnwrap) path. Default <see langword="false"/> preserves
/// parity for the alarmmgr path; the subtag fallback sets it to
/// <see langword="true"/>.
/// </summary>
public bool Degraded { get; set; }
}
@@ -0,0 +1,317 @@
using System;
using System.Collections.Generic;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Subtag-fallback implementation of <see cref="IMxAccessAlarmConsumer"/>.
/// Where <see cref="WnWrapAlarmConsumer"/> polls the native alarmmgr
/// (wnwrap) COM stream, this consumer advises a configured set of MXAccess
/// alarm subtags via an <see cref="ISubtagAlarmSource"/> and synthesizes
/// alarm transitions through a <see cref="SubtagAlarmStateMachine"/>. Every
/// emitted record is flagged <see cref="MxAlarmSnapshotRecord.Degraded"/>
/// and assigned a deterministic <see cref="SyntheticAlarmGuid"/> derived
/// from the alarm's full reference, since the subtag path has no
/// alarmmgr-supplied GUID.
/// </summary>
/// <remarks>
/// <para>
/// Threading: like <see cref="WnWrapAlarmConsumer"/>, this consumer is
/// driven on the worker's STA thread. <see cref="Subscribe"/> binds the
/// source's <c>ValueChanged</c> event; the source raises that event on
/// the same STA in production, so <see cref="AlarmTransitionEmitted"/>
/// fires on the STA and subscribers must marshal off it themselves if
/// they need another thread.
/// </para>
/// <para>
/// The subtag path is event-driven, so <see cref="PollOnce"/> is a
/// no-op — value changes arrive through the source's advise stream
/// rather than an explicit poll. Acknowledgment in subtag mode writes
/// the comment to the target's writable ack-comment subtag rather than
/// calling a native <c>AlarmAckByGUID</c> / <c>AlarmAckByName</c>.
/// </para>
/// </remarks>
public sealed class SubtagAlarmConsumer : IMxAccessAlarmConsumer
{
private readonly ISubtagAlarmSource source;
private readonly SubtagAlarmStateMachine stateMachine;
private readonly Dictionary<string, AlarmSubtagTarget> targetsByReference;
private readonly Dictionary<Guid, string> referencesBySyntheticGuid;
private readonly EventHandler<SubtagValueChange> valueChangedHandler;
private bool subscribed;
private bool disposed;
/// <summary>Fires once per synthesized alarm-state transition.</summary>
public event EventHandler<MxAlarmTransitionEvent>? AlarmTransitionEmitted;
/// <summary>
/// Initializes the consumer over a subtag source and a watch list of
/// alarm targets.
/// </summary>
/// <param name="source">The subtag value source to advise and write through.</param>
/// <param name="watchList">The alarm subtag targets to observe.</param>
/// <exception cref="ArgumentNullException">
/// Thrown when <paramref name="source"/> or <paramref name="watchList"/>
/// is <see langword="null"/>.
/// </exception>
public SubtagAlarmConsumer(ISubtagAlarmSource source, IReadOnlyList<AlarmSubtagTarget> watchList)
{
this.source = source ?? throw new ArgumentNullException(nameof(source));
if (watchList is null)
{
throw new ArgumentNullException(nameof(watchList));
}
this.stateMachine = new SubtagAlarmStateMachine(watchList);
this.targetsByReference = new Dictionary<string, AlarmSubtagTarget>(StringComparer.OrdinalIgnoreCase);
this.referencesBySyntheticGuid = new Dictionary<Guid, string>();
foreach (AlarmSubtagTarget target in watchList)
{
string reference = target.AlarmFullReference ?? string.Empty;
this.targetsByReference[reference] = target;
this.referencesBySyntheticGuid[SyntheticAlarmGuid.ForReference(reference)] = reference;
}
this.valueChangedHandler = OnValueChanged;
}
/// <summary>
/// Advises every observable alarm subtag (active / acked / priority —
/// the ack-comment subtag is a write-only target and is not advised)
/// and begins listening for value changes. The <paramref name="subscription"/>
/// expression is ignored: the subtag set is fixed by the watch list.
/// </summary>
/// <param name="subscription">The subscription expression (unused in subtag mode).</param>
public void Subscribe(string subscription)
{
if (disposed)
{
throw new ObjectDisposedException(nameof(SubtagAlarmConsumer));
}
List<string> addresses = new List<string>();
foreach (AlarmSubtagTarget target in targetsByReference.Values)
{
AddIfNotEmpty(addresses, target.ActiveSubtag);
AddIfNotEmpty(addresses, target.AckedSubtag);
AddIfNotEmpty(addresses, target.PrioritySubtag);
}
source.Advise(addresses);
if (!subscribed)
{
source.ValueChanged += valueChangedHandler;
subscribed = true;
}
}
/// <summary>
/// Acknowledges an alarm by its synthetic GUID. Resolves the GUID back
/// to its alarm full reference and delegates to the by-name write path.
/// </summary>
/// <param name="alarmGuid">The synthetic alarm GUID.</param>
/// <param name="ackComment">The acknowledgment comment.</param>
/// <param name="ackOperatorName">The operator name (unused in subtag mode).</param>
/// <param name="ackOperatorNode">The operator node (unused in subtag mode).</param>
/// <param name="ackOperatorDomain">The operator domain (unused in subtag mode).</param>
/// <param name="ackOperatorFullName">The operator full name (unused in subtag mode).</param>
/// <returns>0 on success; non-zero when the GUID or ack-comment subtag is unknown.</returns>
public int AcknowledgeByGuid(
Guid alarmGuid,
string ackComment,
string ackOperatorName,
string ackOperatorNode,
string ackOperatorDomain,
string ackOperatorFullName)
{
if (disposed)
{
throw new ObjectDisposedException(nameof(SubtagAlarmConsumer));
}
if (!referencesBySyntheticGuid.TryGetValue(alarmGuid, out string reference) ||
!targetsByReference.TryGetValue(reference, out AlarmSubtagTarget target))
{
return -1;
}
return WriteAckComment(target, ackComment);
}
/// <summary>
/// Acknowledges an alarm by its (name, provider, group) tuple. In subtag
/// mode the comment is written to the target's writable ack-comment
/// subtag; the operator-identity arguments are not surfaced through the
/// subtag write.
/// </summary>
/// <param name="alarmName">The alarm name (object-rooted tag name as the dispatcher derives it).</param>
/// <param name="providerName">The provider name used to recompose the full reference for lookup.</param>
/// <param name="groupName">The group name used to recompose the full reference for lookup.</param>
/// <param name="ackComment">The acknowledgment comment.</param>
/// <param name="ackOperatorName">The operator name (unused in subtag mode).</param>
/// <param name="ackOperatorNode">The operator node (unused in subtag mode).</param>
/// <param name="ackOperatorDomain">The operator domain (unused in subtag mode).</param>
/// <param name="ackOperatorFullName">The operator full name (unused in subtag mode).</param>
/// <returns>0 on success; non-zero when no target matches or it lacks an ack-comment subtag.</returns>
public int AcknowledgeByName(
string alarmName,
string providerName,
string groupName,
string ackComment,
string ackOperatorName,
string ackOperatorNode,
string ackOperatorDomain,
string ackOperatorFullName)
{
if (disposed)
{
throw new ObjectDisposedException(nameof(SubtagAlarmConsumer));
}
AlarmSubtagTarget? target = ResolveTargetByName(alarmName, providerName, groupName);
if (target is null)
{
return -1;
}
return WriteAckComment(target, ackComment);
}
/// <summary>
/// Returns the state machine's currently-active alarm snapshot, with
/// each record stamped <see cref="MxAlarmSnapshotRecord.Degraded"/> and
/// assigned its synthetic GUID.
/// </summary>
/// <returns>The active alarm snapshot records.</returns>
public IReadOnlyList<MxAlarmSnapshotRecord> SnapshotActiveAlarms()
{
IReadOnlyList<MxAlarmSnapshotRecord> records = stateMachine.SnapshotActive();
foreach (MxAlarmSnapshotRecord record in records)
{
StampSynthetic(record);
}
return records;
}
/// <summary>
/// No-op: the subtag path is event-driven and owns no poll cadence.
/// </summary>
public void PollOnce()
{
// Subtag mode is event-driven; value changes arrive via the source's
// advise stream, so there is nothing to poll.
}
/// <inheritdoc />
public void Dispose()
{
if (disposed)
{
return;
}
disposed = true;
if (subscribed)
{
try
{
source.ValueChanged -= valueChangedHandler;
}
catch
{
// swallow — best-effort detach during dispose
}
subscribed = false;
}
try
{
source.Dispose();
}
catch
{
// swallow — best-effort source dispose
}
}
private void OnValueChanged(object? sender, SubtagValueChange change)
{
if (disposed || change is null)
{
return;
}
IReadOnlyList<MxAlarmTransitionEvent> transitions =
stateMachine.Apply(change.ItemAddress, change.Value, change.TimestampUtc);
if (transitions.Count == 0)
{
return;
}
EventHandler<MxAlarmTransitionEvent>? handler = AlarmTransitionEmitted;
foreach (MxAlarmTransitionEvent transition in transitions)
{
StampSynthetic(transition.Record);
handler?.Invoke(this, transition);
}
}
/// <summary>
/// Stamps the degraded flag and synthetic GUID onto a synthesized
/// record. The state machine sets only the provider / group / tag-name
/// parts, so the alarm reference is recomposed exactly as
/// <see cref="AlarmDispatcher"/> composes it — keying the synthetic GUID
/// off that recomposed reference keeps it stable across transitions and
/// snapshots for the same alarm.
/// </summary>
private static void StampSynthetic(MxAlarmSnapshotRecord record)
{
record.Degraded = true;
string reference = AlarmRecordTransitionMapper.ComposeFullReference(
record.ProviderName, record.Group, record.TagName);
record.AlarmGuid = SyntheticAlarmGuid.ForReference(reference);
}
private AlarmSubtagTarget? ResolveTargetByName(string? alarmName, string? providerName, string? groupName)
{
if (string.IsNullOrEmpty(alarmName))
{
return null;
}
// Recompose the full reference from the same (provider, group, name)
// tuple the dispatcher derived, then do an exact OrdinalIgnoreCase
// lookup in targetsByReference. This avoids the false-positive that a
// substring scan would produce when one alarm name is a prefix of
// another (e.g. "Level.Hi" matching "Level.HiHi").
string composed = AlarmRecordTransitionMapper.ComposeFullReference(providerName, groupName, alarmName);
targetsByReference.TryGetValue(composed, out AlarmSubtagTarget? target);
return target;
}
private int WriteAckComment(AlarmSubtagTarget target, string ackComment)
{
string ackCommentSubtag = target.AckCommentSubtag ?? string.Empty;
if (string.IsNullOrEmpty(ackCommentSubtag))
{
return -1;
}
source.Write(ackCommentSubtag, ackComment);
return 0;
}
private static void AddIfNotEmpty(List<string> addresses, string? address)
{
if (!string.IsNullOrEmpty(address))
{
addresses.Add(address!);
}
}
}
@@ -0,0 +1,381 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Synthesizes alarm transitions from raw subtag value changes when the
/// native alarmmgr (wnwrap) consumer is unavailable. Each configured
/// <see cref="AlarmSubtagTarget"/> contributes an <c>.active</c>,
/// <c>.acked</c>, optional <c>.priority</c>, and optional ack-comment
/// subtag; the machine tracks per-alarm state and emits
/// <see cref="MxAlarmTransitionEvent"/> records mirroring the wnwrap
/// consumer's UNACK_ALM / ACK_ALM / UNACK_RTN / ACK_RTN transitions.
/// This type performs no COM work and is fully unit-testable.
/// </summary>
/// <remarks>
/// This type has STA affinity: it is mutated and read on the worker STA
/// thread only and is not thread-safe. The per-target
/// <see cref="AlarmSubtagTarget.AckCommentSubtag"/> is a write-only ack
/// target and is deliberately not bound for observation. The synthesized
/// <see cref="MxAlarmSnapshotRecord.AlarmGuid"/> is left at its default
/// value: in subtag mode acks are correlated by name/comment rather than
/// by GUID, and a synthetic GUID is assigned by a later component.
/// </remarks>
public sealed class SubtagAlarmStateMachine
{
private static readonly IReadOnlyList<MxAlarmTransitionEvent> NoEvents = Array.Empty<MxAlarmTransitionEvent>();
private readonly Dictionary<string, AlarmState> _statesByReference = new(StringComparer.OrdinalIgnoreCase);
private readonly Dictionary<string, SubtagBinding> _bindingsByAddress = new(StringComparer.OrdinalIgnoreCase);
/// <summary>Initializes the machine from the configured alarm subtag targets.</summary>
/// <param name="targets">The alarm targets whose subtags will be observed.</param>
public SubtagAlarmStateMachine(IEnumerable<AlarmSubtagTarget> targets)
{
if (targets is null)
{
throw new ArgumentNullException(nameof(targets));
}
foreach (AlarmSubtagTarget target in targets)
{
var state = new AlarmState(target);
_statesByReference[target.AlarmFullReference] = state;
Bind(target.ActiveSubtag, state, SubtagRole.Active);
Bind(target.AckedSubtag, state, SubtagRole.Acked);
Bind(target.PrioritySubtag, state, SubtagRole.Priority);
}
}
/// <summary>
/// Applies a single subtag value change and returns any synthesized
/// transitions. Unknown addresses and no-op changes return an empty list.
/// </summary>
/// <param name="itemAddress">The subtag address whose value changed.</param>
/// <param name="value">The new value (bool, numeric, or string).</param>
/// <param name="timestampUtc">The UTC timestamp of the change.</param>
/// <returns>The synthesized alarm transitions, if any.</returns>
public IReadOnlyList<MxAlarmTransitionEvent> Apply(string itemAddress, object? value, DateTime timestampUtc)
{
if (itemAddress is null || !_bindingsByAddress.TryGetValue(itemAddress, out SubtagBinding binding))
{
return NoEvents;
}
AlarmState state = binding.State;
switch (binding.Role)
{
case SubtagRole.Active:
return ApplyActive(state, CoerceBool(value), timestampUtc);
case SubtagRole.Acked:
return ApplyAcked(state, CoerceBool(value), timestampUtc);
case SubtagRole.Priority:
state.Priority = CoerceInt(value, state.Priority);
return NoEvents;
default:
return NoEvents;
}
}
/// <summary>
/// Returns one snapshot record per currently-active alarm, with state
/// reflecting whether the alarm has been acknowledged.
/// </summary>
/// <returns>The active alarm snapshot records.</returns>
public IReadOnlyList<MxAlarmSnapshotRecord> SnapshotActive()
{
var records = new List<MxAlarmSnapshotRecord>();
foreach (AlarmState state in _statesByReference.Values)
{
if (!state.Active)
{
continue;
}
MxAlarmStateKind kind = state.Acked ? MxAlarmStateKind.AckAlm : MxAlarmStateKind.UnackAlm;
records.Add(state.BuildRecord(kind, state.FirstRaiseUtc));
}
return records;
}
private IReadOnlyList<MxAlarmTransitionEvent> ApplyActive(AlarmState state, bool newActive, DateTime timestampUtc)
{
if (newActive == state.Active)
{
return NoEvents;
}
if (newActive)
{
// false -> true: alarm raised, initially unacknowledged.
MxAlarmStateKind previous = state.DerivedState();
state.Active = true;
state.Acked = false;
state.AckedDuringEpisode = false;
state.FirstRaiseUtc = timestampUtc;
return Single(state, MxAlarmStateKind.UnackAlm, previous, timestampUtc);
}
else
{
// true -> false: alarm returned to normal. The return kind reflects
// whether the alarm was acknowledged at any point during the active
// episode, not just its instantaneous Acked flag (see remarks).
MxAlarmStateKind previous = state.DerivedState();
MxAlarmStateKind kind = state.AckedDuringEpisode ? MxAlarmStateKind.AckRtn : MxAlarmStateKind.UnackRtn;
state.Active = false;
state.Acked = false;
state.AckedDuringEpisode = false;
return Single(state, kind, previous, timestampUtc);
}
}
/// <remarks>
/// <see cref="AlarmState.AckedDuringEpisode"/> latches that the alarm was
/// acknowledged at some point while active. MXAccess does not guarantee
/// that the acked subtag update arrives before the active=false update on
/// clear, so an out-of-order un-ack (acked true->false) must not lose the
/// fact that the episode was acknowledged; the latch ensures the clear
/// still emits ACK_RTN rather than UNACK_RTN.
/// </remarks>
private IReadOnlyList<MxAlarmTransitionEvent> ApplyAcked(AlarmState state, bool newAcked, DateTime timestampUtc)
{
if (newAcked == state.Acked)
{
return NoEvents;
}
state.Acked = newAcked;
// Only a true ack of a currently-active alarm produces an ACK_ALM
// transition; the latch records that the episode was acked at some point.
if (newAcked && state.Active)
{
state.AckedDuringEpisode = true;
return Single(state, MxAlarmStateKind.AckAlm, MxAlarmStateKind.UnackAlm, timestampUtc);
}
// acked true->false while active: do NOT clear the latch (the episode was
// already acked); emit nothing.
return NoEvents;
}
private static IReadOnlyList<MxAlarmTransitionEvent> Single(
AlarmState state,
MxAlarmStateKind kind,
MxAlarmStateKind previous,
DateTime timestampUtc)
{
return new[]
{
new MxAlarmTransitionEvent
{
Record = state.BuildRecord(kind, timestampUtc),
PreviousState = previous,
},
};
}
private void Bind(string? address, AlarmState state, SubtagRole role)
{
if (!string.IsNullOrEmpty(address))
{
if (_bindingsByAddress.ContainsKey(address!))
{
throw new ArgumentException(
$"Duplicate subtag item address '{address}' is bound to more than one alarm target.",
nameof(address));
}
_bindingsByAddress[address!] = new SubtagBinding(state, role);
}
}
private static bool CoerceBool(object? value)
{
switch (value)
{
case null:
return false;
case bool b:
return b;
case string s:
if (bool.TryParse(s, out bool parsedBool))
{
return parsedBool;
}
if (double.TryParse(s, NumberStyles.Any, CultureInfo.InvariantCulture, out double parsedNum))
{
return parsedNum != 0d;
}
return false;
default:
try
{
return Convert.ToDouble(value, CultureInfo.InvariantCulture) != 0d;
}
catch (Exception ex) when (ex is FormatException or InvalidCastException or OverflowException)
{
return false;
}
}
}
private static int CoerceInt(object? value, int fallback)
{
switch (value)
{
case null:
return fallback;
case int i:
return i;
case string s when int.TryParse(s, NumberStyles.Any, CultureInfo.InvariantCulture, out int parsed):
return parsed;
case string:
return fallback;
default:
try
{
return Convert.ToInt32(value, CultureInfo.InvariantCulture);
}
catch (Exception ex) when (ex is FormatException or InvalidCastException or OverflowException)
{
return fallback;
}
}
}
private enum SubtagRole
{
Active,
Acked,
Priority,
}
private readonly struct SubtagBinding
{
public SubtagBinding(AlarmState state, SubtagRole role)
{
State = state;
Role = role;
}
public AlarmState State { get; }
public SubtagRole Role { get; }
}
private sealed class AlarmState
{
private readonly string _providerName;
private readonly string _group;
private readonly string _tagName;
public AlarmState(AlarmSubtagTarget target)
{
(_providerName, _group, _tagName) = DeriveNameParts(target);
}
public bool Active { get; set; }
public bool Acked { get; set; }
/// <summary>
/// Latches that the alarm was acknowledged at some point during the
/// current active episode, so a clear emits ACK_RTN even when the
/// un-ack subtag arrives before the active=false update.
/// </summary>
public bool AckedDuringEpisode { get; set; }
public int Priority { get; set; }
public DateTime FirstRaiseUtc { get; set; }
/// <summary>Derives the current alarm state from the tracked flags, before a transition is applied.</summary>
public MxAlarmStateKind DerivedState()
{
if (!Active)
{
return MxAlarmStateKind.Unspecified;
}
return Acked ? MxAlarmStateKind.AckAlm : MxAlarmStateKind.UnackAlm;
}
public MxAlarmSnapshotRecord BuildRecord(MxAlarmStateKind kind, DateTime timestampUtc)
{
return new MxAlarmSnapshotRecord
{
ProviderName = _providerName,
Group = _group,
TagName = _tagName,
Priority = Priority,
State = kind,
TransitionTimestampUtc = timestampUtc,
};
}
/// <summary>
/// Splits an alarm full reference such as <c>Galaxy!Area.Tank01.Level.HiHi</c>
/// into provider (<c>Galaxy</c>), group/area (<c>Area</c>), and the object-rooted
/// tag name (<c>Tank01.Level.HiHi</c>). The tag name is anchored on the target's
/// <see cref="AlarmSubtagTarget.SourceObjectReference"/> when present, so any
/// leading provider and area segments are stripped.
/// </summary>
private static (string ProviderName, string Group, string TagName) DeriveNameParts(AlarmSubtagTarget target)
{
string reference = target.AlarmFullReference ?? string.Empty;
string providerName = string.Empty;
string afterBang = reference;
int bang = reference.IndexOf('!');
if (bang >= 0)
{
providerName = reference.Substring(0, bang);
afterBang = reference.Substring(bang + 1);
}
string tagName = afterBang;
string source = target.SourceObjectReference ?? string.Empty;
string group = string.Empty;
if (source.Length > 0)
{
// Match the source on dot-delimited segment boundaries so a source
// such as "Tank" does not match mid-token inside "Tank01" and a
// source "Area" does not match inside "TankArea".
string[] segments = afterBang.Split('.');
int segmentIndex = -1;
for (int i = 0; i < segments.Length; i++)
{
if (string.Equals(segments[i], source, StringComparison.OrdinalIgnoreCase))
{
segmentIndex = i;
break;
}
}
if (segmentIndex > 0)
{
// Everything before the object is the area/group; the object
// and everything after it is the object-rooted tag name.
group = string.Join(".", segments, 0, segmentIndex);
tagName = string.Join(".", segments, segmentIndex, segments.Length - segmentIndex);
}
else
{
// segmentIndex == 0 (source is the first segment) or not found:
// no leading area to strip, so the tag name is the whole afterBang.
tagName = afterBang;
}
}
return (providerName, group, tagName);
}
}
}
@@ -0,0 +1,16 @@
using System;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>A change in one advised subtag value, normalized off the COM boundary.</summary>
public sealed class SubtagValueChange
{
/// <summary>Gets or sets the MXAccess item address (subtag reference) whose value changed.</summary>
public string ItemAddress { get; set; } = string.Empty;
/// <summary>Gets or sets the new value, as delivered by MXAccess (boxed COM variant or test value).</summary>
public object? Value { get; set; }
/// <summary>Gets or sets the UTC timestamp the change was observed.</summary>
public DateTime TimestampUtc { get; set; }
}
@@ -0,0 +1,42 @@
using System;
using System.Security.Cryptography;
using System.Text;
namespace ZB.MOM.WW.MxGateway.Worker.MxAccess;
/// <summary>
/// Derives a deterministic synthetic <see cref="Guid"/> from an alarm
/// reference for the subtag-provider fallback path, which has no native
/// MxAccess alarm GUID. Hashing the reference yields a stable identity so
/// repeated transitions for the same alarm reference correlate downstream
/// (acknowledge, snapshot, OPC UA mapping) without an alarmmgr-supplied GUID.
/// </summary>
internal static class SyntheticAlarmGuid
{
/// <summary>
/// Produces a stable <see cref="Guid"/> for the given alarm reference.
/// The same reference always maps to the same GUID; distinct references
/// map to distinct GUIDs with overwhelming probability.
/// </summary>
/// <param name="reference">
/// The fully-qualified alarm reference (for example
/// <c>"Galaxy!Area.Tag.HiHi"</c>). Treated as UTF-8 bytes.
/// </param>
/// <returns>A deterministic, non-empty GUID derived from the reference.</returns>
/// <exception cref="ArgumentNullException">
/// Thrown when <paramref name="reference"/> is <see langword="null"/>.
/// </exception>
public static Guid ForReference(string reference)
{
if (reference is null) throw new ArgumentNullException(nameof(reference));
byte[] bytes = Encoding.UTF8.GetBytes(reference);
// MD5 is used purely for a stable, non-cryptographic identity mapping
// (reference -> 16-byte GUID), never for security. Its 128-bit output
// fits a GUID exactly, which is why it is preferred here.
using MD5 md5 = MD5.Create();
byte[] hash = md5.ComputeHash(bytes);
return new Guid(hash);
}
}