Galaxy picker-form alarm ack before the feed names it dials the bare tag.Attr (gateway refuses) #904

New Issue

2026-11-13T01:49:33-05:00

dohertj2 commented

2026-11-13 01:49:33 -05:00

Follow-up from the #895 review (d5288388).

Gap. For a Galaxy native alarm authored in the AdminUI picker form (attributeRef = <tag>.<Attr>), the driver learns the provider-qualified reference Galaxy!<Area>.<tag>.<Attr> from the alarm feed. Until this driver instance has seen a transition or snapshot entry for that alarm, AcknowledgeAsync dials the bare <tag>.<Attr>, and the gateway's AcknowledgeAlarm refuses it because the reference has no provider prefix.

When it can happen. The host restores condition state from LocalDb on bootstrap (seen in the G6 gate, docs/drivers/evidence/galaxy-g6-e2e-2026-11-12.md Step 6). So after a restart a condition can show Active, Unacknowledged before the feed's snapshot has reached the new driver, for example when the gateway is unreachable or slow to answer. An ack in that window is refused and the host reverts it. The failure is visible, not silent, and a retry after the snapshot arrives succeeds. The window is small. The provider-form attributeRef does not have this gap.

Options.

Driver side: look up the object's area in the Galaxy repository (GR browse is already wired for discovery) when no reference has been learned.
Gateway side (mxaccessgw, cross-repo): accept an area-less tag.Attr in AcknowledgeAlarm. The worker acknowledges by writing <alarm_name>.AckMsg, so the group is only needed to parse the reference.

Not urgent. Filed so the residual is tracked.

Follow-up from the #895 review (`d5288388`). **Gap.** For a Galaxy native alarm authored in the AdminUI picker form (`attributeRef = <tag>.<Attr>`), the driver learns the provider-qualified reference `Galaxy!<Area>.<tag>.<Attr>` from the alarm feed. Until this driver instance has seen a transition or snapshot entry for that alarm, `AcknowledgeAsync` dials the bare `<tag>.<Attr>`, and the gateway's `AcknowledgeAlarm` refuses it because the reference has no provider prefix. **When it can happen.** The host restores condition state from LocalDb on bootstrap (seen in the G6 gate, `docs/drivers/evidence/galaxy-g6-e2e-2026-11-12.md` Step 6). So after a restart a condition can show `Active, Unacknowledged` before the feed's snapshot has reached the new driver, for example when the gateway is unreachable or slow to answer. An ack in that window is refused and the host reverts it. The failure is visible, not silent, and a retry after the snapshot arrives succeeds. The window is small. The provider-form `attributeRef` does not have this gap. **Options.** - Driver side: look up the object's area in the Galaxy repository (GR browse is already wired for discovery) when no reference has been learned. - Gateway side (mxaccessgw, cross-repo): accept an area-less `tag.Attr` in `AcknowledgeAlarm`. The worker acknowledges by writing `<alarm_name>.AckMsg`, so the group is only needed to parse the reference. Not urgent. Filed so the residual is tracked.

dohertj2 added the type:followup priority:p3 labels 2026-11-13 01:49:33 -05:00

dohertj2 referenced this issue

2026-11-13 01:49:52 -05:00

Galaxy native alarm authored by the AdminUI picker (attributeRef tag.Attr) never activates: feed reports Galaxy!Area.tag.Attr #895

dohertj2 referenced this issue from a commit

2026-11-13 02:08:25 -05:00

fix(galaxy): picker-form alarm ack waits for the feed snapshot instead of dialling bare tag.Attr (#904)

dohertj2 commented

2026-11-13 02:08:44 -05:00

Fixed on the driver side in c9640f5c (pushed to master).

Fix

The alarm feed now has an InitialSnapshotCompleted signal. It completes when the gateway's snapshot_complete sentinel arrives, which is after every snapshot entry has been raised through ResolveAlarmRawPath.
AcknowledgeAsync now gets its reference from ResolveAlarmAckReferenceAsync. That method waits only when all three hold:
- the tag is picker-form (no !),
- nothing has been learned for it yet,
- the feed is running.
The wait lasts until the first snapshot arrives, up to 2 s, and also stops on the ack's token. The host's AlarmAcknowledge budget is 5 s. After the wait, the ack dials the full reference the snapshot names.
No reference is guessed. The area always comes from the feed, so a re-areaed object is acked under its current area. If the snapshot doesn't name the alarm, or doesn't arrive in time, the authored tag.Attr is dialled as before and the gateway refuses it visibly. A warning is logged and the host reverts the ack.

Why not the other options

GR area lookup: the GR area is deploy-time configuration, so it can be stale against the runtime alarm group. The driver also doesn't load the hierarchy for authored tags.
Gateway accepting tag.Attr: this needs a cross-repo deploy on every tier. The gateway would resolve the reference from the same monitor cache that the snapshot already delivers to the driver. The default wnwrap consumer ignores the group, but the subtag consumer needs the exact reference, so an area-free ack isn't uniform there anyway. No mxaccessgw change was made.

Tests

GalaxyDriverRawPathKeyingTests, 5 cases:
- an ack before the snapshot waits, then dials the snapshot's Galaxy!Area2…
- a snapshot without the alarm dials the authored ref
- a snapshot that never arrives gives a bounded wait
- cancellation dials nothing
- no wait for provider-form tags or a feed that isn't running
GatewayGalaxyAlarmFeedTests: the signal completes on the sentinel, after the entries.
Mutation-checked: with the ack call site reverted, 2 driver cases fail. With the sentinel TrySetResult removed, the feed case fails.
Galaxy.Tests 632 passed / 13 skipped. Release build: 0 warnings.

Live check (windev TEST Galaxy, gateway 10.100.0.48:5120)

Ran the new skip-gated GalaxyPickerAckBeforeSnapshotLiveTests with GALAXY_904_ALARM_REF=TestMachine_015.TestAlarm003. It raises the alarm with a supervisory write, waits until the feed names it Galaxy!TestArea.TestMachine_015.TestAlarm003, and then runs two checks:
- Negative control: a fresh driver that has learned nothing dials bare TestMachine_015.TestAlarm003. The gateway refuses it with InvalidRequest.
- The fix: a fresh driver acks before its feed snapshot completed and before anything was learned (both asserted). The ack dials Galaxy!TestArea.TestMachine_015.TestAlarm003 and is confirmed.
Passed 5 times.
With the call site reverted, the live check fails with exactly the #904 refusal: alarm_full_reference must be a canonical GUID or 'Provider!Group.Tag' format.
Cleanup:
- Each run cleared the alarm (write False returned Good).
- The temporary gateway key g904-ack-live was revoked.
- MxAccessGw was not restarted, and :5130/health/ready returned 200 throughout.
- Z28061 was not touched.

The live check was driver-level, not a full Host restart from LocalDb, but it exercises the same window: a new driver instance acking before its feed snapshot.

**Fixed on the driver side in `c9640f5c` (pushed to `master`).** **Fix** - The alarm feed now has an `InitialSnapshotCompleted` signal. It completes when the gateway's `snapshot_complete` sentinel arrives, which is after every snapshot entry has been raised through `ResolveAlarmRawPath`. - `AcknowledgeAsync` now gets its reference from `ResolveAlarmAckReferenceAsync`. That method waits only when all three hold: - the tag is picker-form (no `!`), - nothing has been learned for it yet, - the feed is running. - The wait lasts until the first snapshot arrives, up to 2 s, and also stops on the ack's token. The host's AlarmAcknowledge budget is 5 s. After the wait, the ack dials the full reference the snapshot names. - **No reference is guessed.** The area always comes from the feed, so a re-areaed object is acked under its current area. If the snapshot doesn't name the alarm, or doesn't arrive in time, the authored `tag.Attr` is dialled as before and the gateway refuses it visibly. A warning is logged and the host reverts the ack. **Why not the other options** - **GR area lookup:** the GR area is deploy-time configuration, so it can be stale against the runtime alarm group. The driver also doesn't load the hierarchy for authored tags. - **Gateway accepting `tag.Attr`:** this needs a cross-repo deploy on every tier. The gateway would resolve the reference from the same monitor cache that the snapshot already delivers to the driver. The default wnwrap consumer ignores the group, but the subtag consumer needs the exact reference, so an area-free ack isn't uniform there anyway. No mxaccessgw change was made. **Tests** - `GalaxyDriverRawPathKeyingTests`, 5 cases: - an ack before the snapshot waits, then dials the snapshot's `Galaxy!Area2…` - a snapshot without the alarm dials the authored ref - a snapshot that never arrives gives a bounded wait - cancellation dials nothing - no wait for provider-form tags or a feed that isn't running - `GatewayGalaxyAlarmFeedTests`: the signal completes on the sentinel, after the entries. - Mutation-checked: with the ack call site reverted, 2 driver cases fail. With the sentinel `TrySetResult` removed, the feed case fails. - Galaxy.Tests 632 passed / 13 skipped. Release build: 0 warnings. **Live check (windev TEST Galaxy, gateway `10.100.0.48:5120`)** - Ran the new skip-gated `GalaxyPickerAckBeforeSnapshotLiveTests` with `GALAXY_904_ALARM_REF=TestMachine_015.TestAlarm003`. It raises the alarm with a supervisory write, waits until the feed names it `Galaxy!TestArea.TestMachine_015.TestAlarm003`, and then runs two checks: - **Negative control:** a fresh driver that has learned nothing dials bare `TestMachine_015.TestAlarm003`. The gateway refuses it with `InvalidRequest`. - **The fix:** a fresh driver acks before its feed snapshot completed and before anything was learned (both asserted). The ack dials `Galaxy!TestArea.TestMachine_015.TestAlarm003` and is confirmed. - Passed 5 times. - With the call site reverted, the live check fails with exactly the #904 refusal: `alarm_full_reference must be a canonical GUID or 'Provider!Group.Tag' format`. - Cleanup: - Each run cleared the alarm (write False returned Good). - The temporary gateway key `g904-ack-live` was revoked. - MxAccessGw was not restarted, and `:5130/health/ready` returned 200 throughout. - Z28061 was not touched. The live check was driver-level, not a full Host restart from LocalDb, but it exercises the same window: a new driver instance acking before its feed snapshot.

dohertj2 closed this issue

2026-11-13 02:08:45 -05:00

dohertj2 referenced this issue from a commit

2026-11-13 02:57:08 -05:00

docs(deploy): vd03 + QA vq03/vq04 deployed to c9640f5c (#901 #902 #903 #834 #872 #904)

dohertj2 commented

2026-11-13 02:57:48 -05:00

Deployed to DEV vd03 + QA vq03/vq04 in c9640f5c on 2026-11-13. Not exercised live: neither tier has a Galaxy driver.

Record: docs/deploy/2026-11-13-vd03-qa-to-c9640f5c.md (c0ba450d).

Deployed to DEV vd03 + QA vq03/vq04 in `c9640f5c` on 2026-11-13. Not exercised live: neither tier has a Galaxy driver. Record: `docs/deploy/2026-11-13-vd03-qa-to-c9640f5c.md` (`c0ba450d`).

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dohertj2/lmxopcua#904