Galaxy native alarm ack reports success on an unconfirmed gateway OK; the ack never reached Galaxy (H6) #869

New Issue

2026-11-11T12:47:39-05:00

dohertj2 commented

2026-11-11 12:47:39 -05:00

Found by the #614 H6 live gate on 2026-11-11 against the DEV Galaxy on VD03 (one owner-approved ack of CVDAisle_1.TableAlarms_008.RuntimeAlert at 17:37:55Z).

What happened: OtOpcUa's Acknowledge returned Good and the condition went Acknowledged in OPC UA. The ack never reached Galaxy:

.Acked stayed False and .AckMsg stayed empty, checked at 17:39Z and 17:40Z.
The gateway alarm feed still showed the alarm unacknowledged.
Runtime.dbo.v_AlarmEventHistory2 has no ACK row up to 17:46Z.

Cause on our side: the gateway answers OK both when an ack is applied and when it is only sent but unconfirmed, and its docs tell clients to watch the alarm stream for the real transition. GatewayGalaxyAlarmAcknowledger.EvaluateReply treats that OK as success, never logs the gateway's diagnostic message, and never waits for Galaxy's own acknowledge. So the existing revert-on-failure path never fires, and ScadaBridge and other OPC UA clients believe the alarm was handled when it wasn't.

Fix: treat an unconfirmed OK as pending. Confirm through the alarm stream or the .Acked attribute within a bounded window. On timeout, revert the condition to Unacknowledged and return or record a Bad status (Uncertain at most). Log the gateway's diagnostic message.

Why the gateway-side ack didn't stick (dropped unconfirmed write, account permission, or gateway version) needs the VD03 gateway log around 17:37:55Z (see mxaccessgw#226).

Found by the #614 H6 live gate on 2026-11-11 against the DEV Galaxy on VD03 (one owner-approved ack of `CVDAisle_1.TableAlarms_008.RuntimeAlert` at 17:37:55Z). **What happened:** OtOpcUa's `Acknowledge` returned Good and the condition went Acknowledged in OPC UA. The ack never reached Galaxy: - `.Acked` stayed False and `.AckMsg` stayed empty, checked at 17:39Z and 17:40Z. - The gateway alarm feed still showed the alarm unacknowledged. - `Runtime.dbo.v_AlarmEventHistory2` has no ACK row up to 17:46Z. **Cause on our side:** the gateway answers OK both when an ack is applied and when it is only sent but unconfirmed, and its docs tell clients to watch the alarm stream for the real transition. `GatewayGalaxyAlarmAcknowledger.EvaluateReply` treats that OK as success, never logs the gateway's diagnostic message, and never waits for Galaxy's own acknowledge. So the existing revert-on-failure path never fires, and ScadaBridge and other OPC UA clients believe the alarm was handled when it wasn't. **Fix:** treat an unconfirmed OK as pending. Confirm through the alarm stream or the `.Acked` attribute within a bounded window. On timeout, revert the condition to Unacknowledged and return or record a Bad status (Uncertain at most). Log the gateway's diagnostic message. Why the gateway-side ack didn't stick (dropped unconfirmed write, account permission, or gateway version) needs the VD03 gateway log around 17:37:55Z (see mxaccessgw#226).

dohertj2 added the priority:p1 type:bug labels 2026-11-11 12:47:39 -05:00

dohertj2 referenced this issue

2026-11-11 12:47:40 -05:00

Galaxy: AVEVA/licence-gated proofs; fire-and-forget native ack #614

dohertj2 referenced this issue from a commit

2026-11-11 14:53:22 -05:00

fix(galaxy): confirm unconfirmed native alarm acks against Galaxy's acknowledged transition (#869)

dohertj2 referenced this issue from a commit

2026-11-11 14:53:22 -05:00

fix(runtime,galaxy): #869 review round 1 — confirmation off the actor scheduler, stop/async-fault tests

dohertj2 referenced this issue from a commit

2026-11-11 14:53:22 -05:00

fix(runtime): #869 review round 2 — a carry-through transition no longer settles a pending local ack

dohertj2 referenced this issue from a commit

2026-11-11 14:53:22 -05:00

fix(runtime): #869 review round 3 — key a native-ack revert to the ack that failed

dohertj2 referenced this issue from a commit

2026-11-11 14:53:22 -05:00

Merge #869 Galaxy native alarm ack: unconfirmed gateway OK waits for Galaxy's acknowledged transition, reverts on timeout

dohertj2 commented

2026-11-11 14:53:56 -05:00

Fixed in OtOpcUa. An unconfirmed gateway OK now waits (AlarmAck.ConfirmTimeoutMs, default 10 s) for Galaxy's acknowledged transition, otherwise the condition reverts to Unacknowledged with BadTimeout and the gateway diagnostic is logged. A projector bug that let any transition settle a pending ack is also fixed. Why the gateway didn't apply the VD03 ack is tracked in mxaccessgw#226; the live H6 re-run stays on #614. Pushed to master in e857135e..7a27ef1f; full suite green (20 projects, 0 failures, 0 warnings).

Fixed in OtOpcUa. An unconfirmed gateway OK now waits (AlarmAck.ConfirmTimeoutMs, default 10 s) for Galaxy's acknowledged transition, otherwise the condition reverts to Unacknowledged with BadTimeout and the gateway diagnostic is logged. A projector bug that let any transition settle a pending ack is also fixed. Why the gateway didn't apply the VD03 ack is tracked in mxaccessgw#226; the live H6 re-run stays on #614. Pushed to master in `e857135e..7a27ef1f`; full suite green (20 projects, 0 failures, 0 warnings).

dohertj2 closed this issue

2026-11-11 14:53:56 -05:00

dohertj2 referenced this issue

2026-11-11 14:53:57 -05:00

Galaxy: AVEVA/licence-gated proofs; fire-and-forget native ack #614

dohertj2 referenced this issue

2026-11-12 23:22:08 -05:00

Galaxy: AVEVA/licence-gated proofs; fire-and-forget native ack #614

dohertj2 referenced this issue

2026-11-12 23:34:14 -05:00

Galaxy native alarm authored by the AdminUI picker (attributeRef tag.Attr) never activates: feed reports Galaxy!Area.tag.Attr #895

dohertj2 referenced this issue from a commit