Joseph Doherty
315 lines
20 KiB
Markdown
315 lines
20 KiB
Markdown
# MES-Reportable Alarm Flag — Design
|
||
|
||
**Date:** 2026-09-11
|
||
**Status:** IMPLEMENTED 2026-09-11 — merged to main @ 4cfa645b; rig live gate PASS 14/14 + Playwright override round-trip PASS (§8). QA/prod NOT deployed (remote exec on hold).
|
||
**Supersedes (partially):** the severity-band decision in
|
||
[`2026-06-30-mes-alarm-status-api.md`](2026-06-30-mes-alarm-status-api.md) §6.1 for **computed**
|
||
alarms only (native alarms keep the band — see §2.3).
|
||
**Component touchpoints:** Template Engine (#1), Configuration Database, Site Runtime (#3),
|
||
Script Analysis (#25), Central UI, CLI, Transport (#24), Inbound API (#14, deployed config only).
|
||
**Mockup:** https://claude.ai/code/artifact/02ffcbf9-b87f-4e12-852d-38e2910c6056 (artifact "MES Reportable Alarms").
|
||
**Implementation plan:** [`2026-09-11-mes-reportable-alarm-flag.md`](2026-09-11-mes-reportable-alarm-flag.md) (16 tasks).
|
||
|
||
---
|
||
|
||
## 1. Goal
|
||
|
||
The legacy WWSupport / APIServer MES alarm endpoints decided which alarms were MES-relevant by a
|
||
per-machine row in the BTDB `MachineAlarm` table (`Name`, `Severity`, `FlaggedForMES`). ScadaBridge
|
||
already owns the alarm definition on the template, so the flag belongs there too:
|
||
|
||
1. Add a **`IsMesReportable` boolean (checkbox, default `false`)** to template alarms, and a
|
||
matching **tri-state override** (`inherited` / `true` / `false`) on instance alarm overrides.
|
||
2. Carry the flag through flattening to the site so the script engine's alarm read
|
||
(`Alarms.CurrentAsync()`) exposes it and a script can **filter by it**.
|
||
3. Ship an **inbound API method that routes to the instance and performs the alarm check**,
|
||
returning the legacy `AlarmStatusResponse` shape with `IsFlaggedForMES` driven by the flag.
|
||
4. Prove all of it on the **local docker rig** (`docker/`) — seed template, instance, method, key,
|
||
trip an alarm, call the endpoint.
|
||
|
||
No BTDB dependency anywhere. Everything the check needs lives on the deployed configuration.
|
||
|
||
---
|
||
|
||
## 2. Decisions and scope
|
||
|
||
### 2.1 Where the flag lives (data model)
|
||
|
||
| Layer | Change | Semantics |
|
||
|---|---|---|
|
||
| `TemplateAlarm` (Commons entity) | `bool IsMesReportable` | Default `false`. Inherits down the template chain like `PriorityLevel`. |
|
||
| `InstanceAlarmOverride` (Commons entity) | `bool? IsMesReportableOverride` | `null` = inherit from template (the existing nullable-means-inherited convention, same as `PriorityLevelOverride`). |
|
||
| `ResolvedAlarm` (flattened DTO) | `bool IsMesReportable` | Effective value after override; rides `deployed_configurations.config_json` unchanged — **no site schema change**. |
|
||
| Config DB | migration `AddAlarmMesReportable` | `TemplateAlarms.IsMesReportable bit NOT NULL DEFAULT 0`; `InstanceAlarmOverrides.IsMesReportableOverride bit NULL`. |
|
||
|
||
Naming follows the entity idiom already in the file (`IsLocked`, `IsInherited`). The legacy JSON
|
||
name `IsFlaggedForMES` is preserved **only on the API response**, where MES clients depend on it.
|
||
|
||
**Locking:** the flag is an ordinary overridable field. A `Locked` alarm cannot be overridden at all
|
||
(existing rule, `LockEnforcer.ValidateAlarmOverride`); an unlocked alarm's flag can be flipped per
|
||
instance. `Name`/`TriggerType` stay the only fixed fields.
|
||
|
||
### 2.2 How the site exposes it (script engine)
|
||
|
||
The flag is stamped onto the alarm snapshot the script engine already reads:
|
||
|
||
- `InstanceActor` records `IsMesReportable` per alarm at Alarm Actor spawn (beside the existing
|
||
`_alarmPriorities` map) and **stamps it in `BuildAlarmStatesSnapshot()`** on every row
|
||
(`with { IsMesReportable = … }`), retained or fallback. One place, so it cannot drift from an emit
|
||
site. `AlarmActor` itself is untouched.
|
||
- `AlarmStateChanged` gains an additive init-only `bool IsMesReportable` (default `false`) — the
|
||
same additive convention as `AckTime`. **It is deliberately NOT added to the gRPC protos in this
|
||
slice**; central never sees it (see §2.4).
|
||
- `ScriptAlarm` gains a trailing `bool IsMesReportable` positional parameter. Native rows are
|
||
always `false`.
|
||
- `AlarmsAccessor` gains **`MesReportableAsync(ct)`** = `CurrentAsync` filtered to
|
||
`IsMesReportable`. `CurrentAsync()` is unchanged (an optional bool before the `CancellationToken`
|
||
would break positional callers). Scripts can equally `.Where(a => a.IsMesReportable)`.
|
||
- The accessor stub is mirrored on both hand-maintained compile surfaces — `ScriptCompileSurface`
|
||
(`CompileAlarmsAccessor`) and the Central UI `SandboxScriptHost` (`SandboxAlarmsAccessor`) — and
|
||
pinned by `CompileSurfaceParityTests`. `FlowShapePredictor` does not name accessor members, so
|
||
it needs no change.
|
||
|
||
### 2.3 Native alarms are OUT of scope (and why)
|
||
|
||
Native alarms (OPC UA A&C / MxGateway mirrors) have **no per-alarm configuration row** anywhere:
|
||
only the *source binding* (`TemplateNativeAlarmSource`) is configured, and the individual conditions
|
||
are discovered at runtime. There is nowhere to put a per-alarm checkbox.
|
||
|
||
Consequences, stated plainly:
|
||
|
||
- This feature covers **computed `TemplateAlarm`s**. The production CvdReactor endpoints, whose
|
||
alarms are all native sources, are **not changed** by it and keep the severity-band convention
|
||
(900–999) from the 2026-06-30 plan. `IsFlaggedForMES` for a native row therefore stays
|
||
`Severity in band`; for a computed row it becomes the flag.
|
||
- Follow-up options, recorded in the deferred-work register: (a) a source-level
|
||
`IsMesReportable` on `TemplateNativeAlarmSource` + its override ("every condition under this
|
||
binding is reportable"), or (b) a new per-condition config entity — the first of its kind. Neither
|
||
is designed here.
|
||
|
||
### 2.4 Central visibility is a follow-up
|
||
|
||
The Alarm Summary / Debug View rows are `AlarmStateChanged` values that arrive over gRPC. Showing a
|
||
"MES" badge there needs additive proto fields (`AlarmStateUpdate` field 25, `DebugAlarmStateDto`
|
||
field 20, manual `regen-proto.sh`) plus mapper changes. Not needed for the script check or the
|
||
inbound API, so it is **deferred** (register row) to keep this slice free of wire changes.
|
||
Design-time surfaces (template Alarms tab, Instance Configure overrides) DO show the flag.
|
||
|
||
### 2.5 The inbound API method (deployed config, rig)
|
||
|
||
`AlarmStatus` — the full-filter endpoint from the legacy spec, minus the BTDB machine lookup
|
||
(the rig has no BTDB; `MachineCode` is the instance `UniqueName`, which is exactly what
|
||
`Route.To(code)` resolves). Production adds its `SAPID → Code` lookup in front, as `MesMoveIn` does.
|
||
|
||
Request (`--parameters`, JSON Schema):
|
||
```json
|
||
{ "type":"object",
|
||
"properties":{
|
||
"MachineCode":{"type":"string"},
|
||
"AlarmFilter":{"type":"object","properties":{
|
||
"FlaggedOnly":{"type":"boolean"}, "IncludeAcked":{"type":"boolean"},
|
||
"MinSeverity":{"type":"integer"}, "MaxSeverity":{"type":"integer"},
|
||
"NameFilter":{"type":"string"}}}},
|
||
"required":["MachineCode"] }
|
||
```
|
||
Response (`--return-def`): `{ WasSuccessful, ErrorText, Alarms: AlarmInfo[] }` with
|
||
`AlarmInfo = { Name, HierarchicalName, Description, IsFlaggedForMES, Severity, StatusCode,
|
||
TriggeredDT, AckDT, AckComment }` — byte-compatible with the legacy contract.
|
||
|
||
Router body: `Route.To(MachineCode).Call("AlarmStatus", { MachineCode, ...AlarmFilter })`,
|
||
catching `"not found on instance"` → `Alarms status is not supported on machine '<code>'`, any
|
||
other exception → `WasSuccessful=false` with `Alarms` cleared (legacy all-or-nothing).
|
||
|
||
Site script `AlarmStatus` (Call, root of the demo template): `Alarms.CurrentAsync()` →
|
||
`Active && !IsConfiguredPlaceholder` → `FlaggedOnly ? IsMesReportable : true` → severity bounds →
|
||
name substring → `IncludeAcked` → project to `AlarmInfo` with `IsFlaggedForMES = a.IsMesReportable`.
|
||
Filter order matches the legacy spec.
|
||
|
||
A `SimpleAlarmStatus` variant (flagged-only, acked always included) is one extra `api-method create`
|
||
with the same site script and `FlaggedOnly=true` hard-wired; included in the seed as the second
|
||
endpoint so both legacy routes are demonstrated.
|
||
|
||
### 2.6 Rig demo (all local)
|
||
|
||
`docker/seed-mes-alarm-demo.sh` (idempotent by name lookup, same style as `seed-flow-log-demo.sh`):
|
||
|
||
| Object | Detail |
|
||
|---|---|
|
||
| Template `MesAlarmDemo` | static `Double` attribute `Temperature` = 20 (static ⇒ a value-change trips alarms synchronously, no device round-trip, no OPC UA String-only trap). |
|
||
| Alarm `HighTemp` | HiLo on `Temperature`, hi 80 / hihi 95, priority 500, **`--mes-reportable`**. |
|
||
| Alarm `TempSpike` | ValueMatch `Temperature == 100`, priority 300, not reportable. |
|
||
| Alarm `SensorFault` | ValueMatch `Temperature == -1`, priority 700, not reportable **on the template**. |
|
||
| Script `AlarmStatus` | Call, root; body per §2.5. |
|
||
| Instance `MesAlarmDemo01` | site 1; **alarm override `SensorFault` → `IsMesReportableOverride = true`** to prove the override path; deployed. |
|
||
| Inbound `AlarmStatus`, `SimpleAlarmStatus` | per §2.5; one API key `mes-alarm-demo` scoped to both. |
|
||
|
||
`docker/verify-mes-alarm-demo.sh`:
|
||
|
||
1. `flow-debug run-value-change --instance-id … --attribute Temperature --value 96` → HighTemp active.
|
||
2. `curl POST /api/AlarmStatus {"MachineCode":"MesAlarmDemo01"}` → 1 row, `IsFlaggedForMES=true`.
|
||
3. `--value 100` → TempSpike active; `AlarmStatus` with `FlaggedOnly=true` → still only
|
||
HighTemp-style rows (TempSpike excluded); `FlaggedOnly=false` → both, TempSpike `false`.
|
||
4. `--value -1` → SensorFault active; `SimpleAlarmStatus` → SensorFault present with
|
||
`IsFlaggedForMES=true` (override wins over the template's `false`).
|
||
5. `--value 20` → all clear; `AlarmStatus` → empty `Alarms`, `WasSuccessful=true`.
|
||
6. `AlarmStatus {"MachineCode":"DelmiaSim01"}` (no script) → `WasSuccessful=false`, "not
|
||
supported on machine".
|
||
7. Cross-check step 2 against `debug snapshot --id …` and the Flow Debugger replay.
|
||
|
||
---
|
||
|
||
## 3. Data flow
|
||
|
||
```
|
||
Central UI / CLI ──AddTemplateAlarm(IsMesReportable)──► ManagementActor ► TemplateService ► TemplateAlarms
|
||
Central UI / CLI ──SetInstanceAlarmOverride(IsMesReportableOverride?)──► InstanceService ► InstanceAlarmOverrides
|
||
│
|
||
FlatteningService.ResolveInheritedAlarms → ResolvedAlarm.IsMesReportable
|
||
FlatteningService.ApplyInstanceAlarmOverrides (ovr ?? existing)
|
||
│ deploy (config_json, unchanged schema)
|
||
▼
|
||
Site: InstanceActor spawns AlarmActors; _alarmMesReportable[name] = alarm.IsMesReportable
|
||
BuildAlarmStatesSnapshot() stamps IsMesReportable on every AlarmStateChanged row
|
||
GetAlarmSnapshotResponse ─► ScriptRuntimeContext.ToScriptAlarm ─► ScriptAlarm.IsMesReportable
|
||
│
|
||
site script AlarmStatus: Alarms.CurrentAsync() / MesReportableAsync() → AlarmInfo[]
|
||
▲
|
||
Central inbound method AlarmStatus ── Route.To(MachineCode).Call("AlarmStatus", filter) ──┘
|
||
◄── X-API-Key ── MES client (curl on the rig)
|
||
```
|
||
|
||
---
|
||
|
||
## 4. Alternatives considered
|
||
|
||
| Option | Verdict |
|
||
|---|---|
|
||
| **Keep the severity band (900–999) as the flag** (2026-06-30 decision) | Rejected for computed alarms: it couples MES relevance to priority, which operators tune for other reasons, and it is invisible in the template editor. Kept for native alarms only because there is no per-alarm row. |
|
||
| **Central lookup table** (a `MesReportableAlarms` table joined at request time, i.e. re-creating BTDB `MachineAlarm`) | Rejected: the check runs at the site (that is the whole point of `Route.To().Call`) and the central table would be a second source of truth beside the template. |
|
||
| **Stamp the flag in `AlarmActor` emits** instead of the snapshot | Rejected: three emit sites to keep in sync; the snapshot builder already owns the "fallback row for a quiet alarm" case, so stamping there covers fired and unfired alarms in one place. |
|
||
| **`CurrentAsync(bool reportableOnly = false, ct)`** | Rejected: an optional bool before the token breaks existing positional `CurrentAsync(ct)` callers on three mirrored surfaces. A separate `MesReportableAsync` is additive. |
|
||
| **Put `IsMesReportable` on the protos now** | Deferred (§2.4): no consumer in this slice; proto regen is manual and adds review surface. |
|
||
|
||
---
|
||
|
||
## 5. Error handling
|
||
|
||
- **Override on a locked alarm** → existing `LockEnforcer` error; UI hides locked alarms from the
|
||
override list already.
|
||
- **Override with no delta** — Instance Configure deletes an override when neither config nor
|
||
priority is set; the delta check must include the tri-state so a reportable-only override is
|
||
**not** silently deleted (known trap, `InstanceConfigure.razor` ~line 1628).
|
||
- **Old bundles / old artifacts**: the Transport DTO fields are trailing with defaults; a bundle
|
||
without them imports as `false`/`null`. A `FlattenedConfiguration` serialized before the change
|
||
deserializes with `IsMesReportable=false`.
|
||
- **Mixed version**: a NEW central deploying to an OLD site — the site ignores the unknown JSON
|
||
property; scripts there see `false` everywhere. Upgrade sites with the central, as always.
|
||
- **Inbound method**: unknown machine → `Instance '…' not found` text passed through as
|
||
`ErrorText`; missing script → the "not supported" mapping; everything else → `WasSuccessful=false`
|
||
with `Alarms` cleared. Never an HTTP 500 for a handled case.
|
||
|
||
---
|
||
|
||
## 6. Testing
|
||
|
||
| Layer | Tests |
|
||
|---|---|
|
||
| Commons | entity + message convention sweeps pass unchanged (plain `bool`, record init-prop); `ScriptAlarm` positional back-compat test. |
|
||
| Config DB | migration applies on SQL Server + SQLite test double; `SchemaConfigurationTests` persist round-trip of the two new columns. |
|
||
| Template Engine | `FlatteningServiceTests`: template `true` → resolved `true`; override `false` wins; override `null` inherits; inherited (derived-template) alarm carries base value; `LockEnforcerTests` unchanged. |
|
||
| Site Runtime | `InstanceActor` snapshot stamps the flag on fired AND unfired alarms; `AlarmsAccessorTests`: `MesReportableAsync` filters; `CompileSurfaceParityTests` sees the new member on all mirrors. |
|
||
| Script Analysis / Central UI | compile-surface + `SandboxScriptHost` compile a script using `MesReportableAsync()` and `a.IsMesReportable`. |
|
||
| Management | `ManagementActorTests` add/update/override carry the field; authz matrix unchanged (no new command names). |
|
||
| CLI | `EditingParityTests` rows for `template alarm add --mes-reportable` and `instance alarm-override set --mes-reportable false`; README updated. |
|
||
| Transport | `RoundTripEquivalenceTests` seeds non-default values (the reflection sweep fails otherwise); `TemplateChildEquality`/`ArtifactDiff` tests see the flag as a change. |
|
||
| Central UI | structural test on `TemplateAlarmDialog.razor` (`@bind="_isMesReportable"`); bUnit/Playwright override round-trip `AlarmOverride_MesReportable_RoundTrips` beside the priority one. |
|
||
| Rig | `docker/verify-mes-alarm-demo.sh` steps 1–7 (§2.6). |
|
||
|
||
---
|
||
|
||
## 7. Documents to update in the same change
|
||
|
||
`Component-TemplateEngine.md` (alarm fields, override granularity table), `Component-SiteRuntime.md`
|
||
(`Alarms` accessor + `MesReportableAsync`), `Component-ScriptAnalysis.md` (surface list),
|
||
`Component-InboundAPI.md` (routing example: AlarmStatus), `Component-CentralUI.md` (dialog + override
|
||
editor), `Component-ConfigurationDatabase.md` (migration), CLI `README.md`, `CLAUDE.md` native-alarm
|
||
bullet (flag vs band), `2026-06-30-mes-alarm-status-api.md` §6.1 (note: superseded for computed
|
||
alarms), deferred-work register (native source-level flag; central badge/proto).
|
||
|
||
---
|
||
|
||
## 8. Rig live gate results (2026-09-11)
|
||
|
||
Seeded by `docker/seed-mes-alarm-demo.sh`, gated by `docker/verify-mes-alarm-demo.sh` against the
|
||
`docker/` rig (8 nodes, rebuilt from this branch). Config DB migration applied on boot:
|
||
|
||
```
|
||
[14:19:43 INF] [Central/scadabridge-central-a] Applying migration '20260911135501_AddAlarmMesReportable'.
|
||
```
|
||
|
||
### 8.1 Seeded objects
|
||
|
||
| Object | Id | Notes |
|
||
|---|---|---|
|
||
| Template `MesAlarmDemo` | 5255 | `Temperature` Double **static**, value 20 |
|
||
| Alarm `HighTemp` | — | HiLo hi=80 hihi=95, priority 500, **`--mes-reportable`** |
|
||
| Alarm `TempSpike` | — | ValueMatch `100`, priority 300, not reportable |
|
||
| Alarm `SensorFault` | — | ValueMatch `-1`, priority 700, **not reportable on the template** |
|
||
| Site script `AlarmStatus` (Call) | 1045 | `Alarms.CurrentAsync()` / `Alarms.MesReportableAsync()` + the filter params |
|
||
| Instance `MesAlarmDemo01` (site-a) | 3847 | `instance alarm-override set --alarm SensorFault --mes-reportable true` |
|
||
| Inbound method `AlarmStatus` | 2224 | nested `AlarmFilter` object param, routes via `Route.To(MachineCode)` |
|
||
| Inbound method `SimpleAlarmStatus` | 2225 | flagged-only shorthand |
|
||
| API key `mes-alarm-demo` | — | scoped to both methods; token printed once |
|
||
|
||
Second seed run reported `exists` for template/attribute/all three alarms/instance, `updated` for
|
||
both scripts and both methods, and refused to mint a second API key — **idempotent**, same ids.
|
||
|
||
### 8.2 Gate: `PASS 14/14`
|
||
|
||
| Step | Stimulus | Result |
|
||
|---|---|---|
|
||
| A1 | `96` → `AlarmStatus` | `WasSuccessful=true`, exactly 1 row, `HighTemp` |
|
||
| A2 | same | `IsFlaggedForMES=true` (template flag), `Severity=500`, `StatusCode="Triggered.Acked"`, `AckDT=null` |
|
||
| A3 | same | `HierarchicalName="MesAlarmDemo01.HighTemp"` |
|
||
| H | same moment | `debug snapshot --id 3847` shows `HighTemp` Active (`condition.active=true`) |
|
||
| B1/B2 | `100` → `FlaggedOnly=false` | 2 rows: `HighTemp` **true**, `TempSpike` **false** (HiHi still active at 100, so both) |
|
||
| B3 | `100` → `FlaggedOnly=true` | 1 row, `HighTemp`; `TempSpike` excluded by `MesReportableAsync()` |
|
||
| C1/C2 | `-1` → `SimpleAlarmStatus` | 1 row, `SensorFault`, `IsFlaggedForMES=true`, `Severity=700` — **the instance override beat the template's `false`** |
|
||
| C3 | `-1` → `AlarmStatus` | `SensorFault` only; `HighTemp` absent (−1 trips no high band) |
|
||
| D1 | `-1` + `MinSeverity=600` | `SensorFault` only (700 ≥ 600; HighTemp 500 and TempSpike 300 filtered) |
|
||
| E1 | `20` | `WasSuccessful=true`, `Alarms=[]` |
|
||
| F1 | `MachineCode=DelmiaSim01` | `WasSuccessful=false`, `ErrorText` contains "not supported on machine" (`not found on instance` catch) |
|
||
| G1 | `MachineCode=NoSuchMachine` | `WasSuccessful=false`, `ErrorText` contains "not found" |
|
||
|
||
Browser round-trip: `dotnet test … --filter AlarmOverride_MesReportable_RoundTrips` →
|
||
**Passed: 1, Failed: 0** (12 s).
|
||
|
||
### 8.3 Facts the gate established (read before extending the demo)
|
||
|
||
- **`StatusCode` is `"Triggered.Acked"` for every computed alarm, and `AckDT` is always `null`.**
|
||
`AlarmConditionStateFactory.ForComputed` reports `Acknowledged: true` — computed alarms have no
|
||
ack-back, so there is nothing to be unacknowledged about. A filter such as the script's
|
||
`IncludeAcked=false` therefore drops *every* computed alarm; it is only meaningful over mirrored
|
||
native conditions. The plan's expected `"Triggered"` was wrong and the assertion was corrected.
|
||
- **`Severity` is the alarm's effective priority**, and for a HiLo alarm it is the *band* priority
|
||
(`LevelPriority`), falling back to the alarm priority when no per-band priority is configured —
|
||
which is why `HighTemp` reads 500 at HiHi and `MinSeverity=600` cleanly isolates `SensorFault`.
|
||
- **The `debug snapshot` / stream DTO reports `isMesReportable: false` for every alarm, including
|
||
`HighTemp`.** That is §2.2 working as specified — the flag is deliberately not on the gRPC protos
|
||
in this slice, so central never sees it and only the site-local script snapshot is stamped. Step H
|
||
therefore cross-checks alarm *activity*, not the flag. (Register item: central badge/proto.)
|
||
- **`ValueMatch` compares `value?.ToString()` as an ordinal string**, so `-1` and `100` matched the
|
||
Double attribute's rendered form on this rig. A ValueMatch alarm over a Double is a string
|
||
comparison, not a numeric one — `RangeViolation` is the numeric trigger.
|
||
- `--match-value=-1` must use the `=` form; a space-separated `-1` is parsed as another option.
|
||
|
||
### 8.4 Deviations from the plan
|
||
|
||
1. Step A's `StatusCode` expectation changed from `"Triggered"` to `"Triggered.Acked"` (§8.3).
|
||
2. Step H parses `alarmStates[].condition.active` (the snapshot has no textual state field; the
|
||
`state` enum is `Active=0`, `Normal=1`).
|
||
3. `SensorFault` is seeded with `--attribute Temperature` — a `ValueMatch` trigger needs the watched
|
||
attribute, which the plan's sketch omitted.
|
||
4. `site deploy-artifacts` is not called: the demo has no shared scripts or system-wide artifacts,
|
||
and `instance deploy` after the override is what puts the alarms and the script on site-a.
|