fix(galaxy): picker-form alarm ack waits for the feed snapshot instead of dialling bare tag.Attr (#904)
v2-ci / unit-tests (push) Has been cancelled

A picker-form native alarm (attributeRef tag.Attr) only learns the provider-qualified reference
Galaxy!<Area>.tag.Attr from the alarm feed. After a host restart the condition can be restored from
LocalDb before the feed's first snapshot reached the new driver, and an ack in that window dialled the
bare tag.Attr, which the gateway's AcknowledgeAlarm refuses.

The feed now exposes InitialSnapshotCompleted (set on snapshot_complete, after every snapshot entry has
been raised). AcknowledgeAsync resolves the reference through ResolveAlarmAckReferenceAsync: when a
picker-form tag has learned nothing yet and the feed is running, it waits for that snapshot (2 s, and the
caller's token) and dials the reference the snapshot names, so the area always comes from the feed and a
re-areaed object acks under its current area. No reference is guessed: if the snapshot does not name the
alarm, or does not arrive in time, the authored reference is dialled as before and refused visibly.

Chosen over the alternatives in the issue: a GR-browse area lookup can be stale against the runtime
area and is not loaded for authored tags, and a gateway-side tag.Attr resolution needs a cross-repo
deploy and would resolve from the same monitor cache the snapshot already carries.

Tests: 5 driver cases + 1 feed case (fail with the call site / sentinel reverted), and a skip-gated live
gate GalaxyPickerAckBeforeSnapshotLiveTests, run against windev's TEST Galaxy (TestMachine_015.TestAlarm003):
an ack issued before the snapshot dials Galaxy!TestArea.TestMachine_015.TestAlarm003 and is confirmed; with
the call site reverted it fails with the gateway's InvalidRequest refusal.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
2026-11-13 02:07:38 -05:00
parent f0c6cf9640
commit c9640f5ced
7 changed files with 413 additions and 2 deletions
+1 -1
View File
@@ -227,7 +227,7 @@ Every (re)open of `StreamAlarms` replays the gateway's active-alarm snapshot. `G
- **MXAccess status rows → OPC UA status (`StatusCodeMap.FromMxStatus`, #866/#872)**: an `Ok` row is graded by the item's OPC DA quality. Any other row's `detail` is an MxStatusDetail code (never a quality byte) and maps through the mxaccessgw vocabulary table (e.g. 1007 → `BadNotWritable`, 1008 → `BadUserAccessDenied`, 13/1005 → `BadTypeMismatch`, and the live-captured automation-object code **8001** — write rejected for type conversion → `BadTypeMismatch`). Unmapped details fall back to the category (`SecurityError` → `BadUserAccessDenied`, so 8018 is too; `CommunicationError` → `BadCommunicationError`; `ConfigurationError` → `BadConfigurationError`; else `Bad`), logged once per (detail, category). `detected_by` never decides the code. **`Pending` and `Warning` rows map to `Uncertain` (`0x40000000`) for any detail**: MXAccess sets success = -1 on Pending (accepted, in progress), so they are not failures. On a data change the value is published Uncertain. On a write reply the result is Uncertain, not Good, so the server reports the write as not committed and reverts the node; a commit MXAccess completes later arrives as a normal data change. Neither category has been seen on a data-change or write-reply row yet (mxaccessgw `docs/LiveCaptures-2026-11-13.md` §3), so the first one logs a warning. - **MXAccess status rows → OPC UA status (`StatusCodeMap.FromMxStatus`, #866/#872)**: an `Ok` row is graded by the item's OPC DA quality. Any other row's `detail` is an MxStatusDetail code (never a quality byte) and maps through the mxaccessgw vocabulary table (e.g. 1007 → `BadNotWritable`, 1008 → `BadUserAccessDenied`, 13/1005 → `BadTypeMismatch`, and the live-captured automation-object code **8001** — write rejected for type conversion → `BadTypeMismatch`). Unmapped details fall back to the category (`SecurityError` → `BadUserAccessDenied`, so 8018 is too; `CommunicationError` → `BadCommunicationError`; `ConfigurationError` → `BadConfigurationError`; else `Bad`), logged once per (detail, category). `detected_by` never decides the code. **`Pending` and `Warning` rows map to `Uncertain` (`0x40000000`) for any detail**: MXAccess sets success = -1 on Pending (accepted, in progress), so they are not failures. On a data change the value is published Uncertain. On a write reply the result is Uncertain, not Good, so the server reports the write as not committed and reverts the node; a commit MXAccess completes later arrives as a normal data change. Neither category has been seen on a data-change or write-reply row yet (mxaccessgw `docs/LiveCaptures-2026-11-13.md` §3), so the first one logs a warning.
- **Undecodable values** (`galaxy.events.undecodable`): a value kind this driver has no CLR representation for is published as **`BadTypeMismatch` with a null value**, and the first occurrence of each distinct kind is logged. It is *not* published as a null carrying the event's own quality: an MXAccess event can be Good-quality while the payload is one we cannot map, and a good null is indistinguishable from a genuine null reading — a client would consume a value it never received as a real one, with the failure visible only in a log. Non-zero here means the gateway is sending a wire type `MxValueDecoder` has no mapping for and those readings are being lost; the fix is a decoder mapping, not a quality change. - **Undecodable values** (`galaxy.events.undecodable`): a value kind this driver has no CLR representation for is published as **`BadTypeMismatch` with a null value**, and the first occurrence of each distinct kind is logged. It is *not* published as a null carrying the event's own quality: an MXAccess event can be Good-quality while the payload is one we cannot map, and a good null is indistinguishable from a genuine null reading — a client would consume a value it never received as a real one, with the failure visible only in a log. Non-zero here means the gateway is sending a wire type `MxValueDecoder` has no mapping for and those readings are being lost; the fix is a decoder mapping, not a quality change.
- **Initial values can overtake the subscribe reply (#859)** (`galaxy.events.early_parked` / `early_replayed` / `early_overflow`): mxaccessgw gives no reply-before-event ordering — the worker frames the SubscribeBulk reply on a thread-pool continuation while the STA is already pumping the OnDataChange that Advise queued, and the reply (unary Invoke) and the event (StreamEvents) are separate HTTP/2 streams anyway. An event for an unregistered handle is therefore only "stale after unsubscribe" when no subscribe is in flight. Every SubscribeBulk call site (`SubscribeAsync`, the subscribe-once `ReadAsync` — which waits only for the initial value and would otherwise read `BadTimeout` — the reconnect/replay-gap `ResubscribeAllAsync` (one window per subscription, not across the whole replay), `PerPlatformProbeWatcher`) opens `SubscriptionRegistry.BeginPendingSubscribe()` before the RPC and closes it right after registering, never later; inside the window unknown-handle events are parked, promoted by the `Register`/`Rebind` that binds the handle — map writes and promotion are one critical section, so no later event can resolve in between — and replayed on the pump's dispatch thread before any later event for that handle. Held events (parked + promoted-but-unreplayed) are bounded at 100 000 together. Unclaimed parked events are trimmed by **window sequence**, not only when every window has closed (concurrent reads keep some window open almost permanently): each window gets a monotonically increasing sequence when it opens, each parked event is stamped with the newest sequence, and once every window that opened at or before that stamp has closed the event is discarded — outside any window the stale drop is unchanged. **Sessions (#861):** item handles are per gateway session and the next session reuses the numbers. On reconnect `ReopenAsync` calls `SubscriptionRegistry.StartNewSession()` both before and after the gateway session swap (`ReplayAsync` does it only when no reopen ran), and the replay re-subscribes only subscriptions still holding the previous session's handles (one registered after the swap is already live). The pump is always restarted, even with nothing to replay. `StartNewSession()` discards everything held, **invalidates the reverse map** (subscriptions are kept; new-session events park until the replay's per-subscription `Rebind` promotes them, instead of resolving to the tag that owned the number before), and bumps a session number. Registration and rebind are refused for a stale session, so a `SubscribeAsync` whose reply straddled the reconnect is **re-issued on the new session** (up to 3 attempts), a subscribe-once read returns `BadCommunicationError`, and dead old-session handle numbers are never sent to `UnsubscribeBulk` (a read rebound by the replay releases the handles the registry holds, not its original ones); an old session's pump can no longer resolve through the map. Reads listen on the driver, not on a pump instance, so a read survives the reconnect's pump restart. `PerPlatformProbeWatcher` records the session with each probe handle: only current-session handles are published (fenced `TryUpsert`), unsubscribed or purged, and a seed reply that straddled a reconnect is re-issued. Purges are fenced by the session captured before the unsubscribe RPC. **Interim (I-3):** a completed `UnsubscribeBulk` purges anything parked for its handles, so a trailing event of a removed item is not promoted onto a reused handle number within one session. It cannot catch an event still in flight after the unsubscribe, and it also drops a genuine early value for a number reused that fast (a timeout rather than another tag's value). The complete fix needs a worker sequence on the SubscribeBulk reply so the driver can tell pre-subscribe events from post-subscribe ones — an optional cross-repo change in mxaccessgw. - **Initial values can overtake the subscribe reply (#859)** (`galaxy.events.early_parked` / `early_replayed` / `early_overflow`): mxaccessgw gives no reply-before-event ordering — the worker frames the SubscribeBulk reply on a thread-pool continuation while the STA is already pumping the OnDataChange that Advise queued, and the reply (unary Invoke) and the event (StreamEvents) are separate HTTP/2 streams anyway. An event for an unregistered handle is therefore only "stale after unsubscribe" when no subscribe is in flight. Every SubscribeBulk call site (`SubscribeAsync`, the subscribe-once `ReadAsync` — which waits only for the initial value and would otherwise read `BadTimeout` — the reconnect/replay-gap `ResubscribeAllAsync` (one window per subscription, not across the whole replay), `PerPlatformProbeWatcher`) opens `SubscriptionRegistry.BeginPendingSubscribe()` before the RPC and closes it right after registering, never later; inside the window unknown-handle events are parked, promoted by the `Register`/`Rebind` that binds the handle — map writes and promotion are one critical section, so no later event can resolve in between — and replayed on the pump's dispatch thread before any later event for that handle. Held events (parked + promoted-but-unreplayed) are bounded at 100 000 together. Unclaimed parked events are trimmed by **window sequence**, not only when every window has closed (concurrent reads keep some window open almost permanently): each window gets a monotonically increasing sequence when it opens, each parked event is stamped with the newest sequence, and once every window that opened at or before that stamp has closed the event is discarded — outside any window the stale drop is unchanged. **Sessions (#861):** item handles are per gateway session and the next session reuses the numbers. On reconnect `ReopenAsync` calls `SubscriptionRegistry.StartNewSession()` both before and after the gateway session swap (`ReplayAsync` does it only when no reopen ran), and the replay re-subscribes only subscriptions still holding the previous session's handles (one registered after the swap is already live). The pump is always restarted, even with nothing to replay. `StartNewSession()` discards everything held, **invalidates the reverse map** (subscriptions are kept; new-session events park until the replay's per-subscription `Rebind` promotes them, instead of resolving to the tag that owned the number before), and bumps a session number. Registration and rebind are refused for a stale session, so a `SubscribeAsync` whose reply straddled the reconnect is **re-issued on the new session** (up to 3 attempts), a subscribe-once read returns `BadCommunicationError`, and dead old-session handle numbers are never sent to `UnsubscribeBulk` (a read rebound by the replay releases the handles the registry holds, not its original ones); an old session's pump can no longer resolve through the map. Reads listen on the driver, not on a pump instance, so a read survives the reconnect's pump restart. `PerPlatformProbeWatcher` records the session with each probe handle: only current-session handles are published (fenced `TryUpsert`), unsubscribed or purged, and a seed reply that straddled a reconnect is re-issued. Purges are fenced by the session captured before the unsubscribe RPC. **Interim (I-3):** a completed `UnsubscribeBulk` purges anything parked for its handles, so a trailing event of a removed item is not promoted onto a reused handle number within one session. It cannot catch an event still in flight after the unsubscribe, and it also drops a genuine early value for a number reused that fast (a timeout rather than another tag's value). The complete fix needs a worker sequence on the SubscribeBulk reply so the driver can tell pre-subscribe events from post-subscribe ones — an optional cross-repo change in mxaccessgw.
- **Native-alarm reference forms (#895)**: the gateway alarm feed names an alarm `Galaxy!<Area>.<tag>.<Attr>`, with `SourceObjectReference = <tag>.<Attr>`. An alarm tag may carry either form as its `attributeRef`: the plain `<tag>.<Attr>` the AdminUI Galaxy picker authors, or the provider-qualified form. The driver matches a transition on the full reference, then the source reference, then the full reference with its `Provider!Area.` prefix stripped. If none of those match an authored `attributeRef`, the area-free names are matched against the provider-qualified ones with their area removed, so a hand-authored `Galaxy!<OldArea>.<tag>.<Attr>` still resolves after Galaxy moves the object to another area. A reference with an empty area (`Galaxy!<tag>.<Attr>`) also matches, and so does the gateway's subtag provider, which sends only `<tag>` as the source reference. The driver remembers the latest full reference per tag, including transitions that arrive before an alarm subscription is registered, so that an ack dials what `AcknowledgeAlarm` accepts, since the gateway refuses a non-provider reference. A picker-form tag that has had no transition since the driver started still dials the bare `<tag>.<Attr>`. When the same alarm is authored in both forms, the provider-form tag gets the transitions while the area matches, and a warning is logged. Before #895, a picker-authored alarm never went active (found in the G6 live gate, `docs/drivers/evidence/galaxy-g6-e2e-2026-11-12.md`). - **Native-alarm reference forms (#895)**: the gateway alarm feed names an alarm `Galaxy!<Area>.<tag>.<Attr>`, with `SourceObjectReference = <tag>.<Attr>`. An alarm tag may carry either form as its `attributeRef`: the plain `<tag>.<Attr>` the AdminUI Galaxy picker authors, or the provider-qualified form. The driver matches a transition on the full reference, then the source reference, then the full reference with its `Provider!Area.` prefix stripped. If none of those match an authored `attributeRef`, the area-free names are matched against the provider-qualified ones with their area removed, so a hand-authored `Galaxy!<OldArea>.<tag>.<Attr>` still resolves after Galaxy moves the object to another area. A reference with an empty area (`Galaxy!<tag>.<Attr>`) also matches, and so does the gateway's subtag provider, which sends only `<tag>` as the source reference. The driver remembers the latest full reference per tag, including transitions that arrive before an alarm subscription is registered, so that an ack dials what `AcknowledgeAlarm` accepts, since the gateway refuses a non-provider reference. If a picker-form tag has had no transition since the driver started (for example right after a host restart restored the condition from LocalDb), the ack waits for the alarm feed's first active-alarm snapshot, up to 2 s and within the ack's own timeout, and then dials the full reference the snapshot names, in the object's current area (#904). The area is never guessed. If the snapshot does not name the alarm, or does not arrive in time, the bare `<tag>.<Attr>` is dialled as before, the gateway refuses it, and the host reverts the ack. When the same alarm is authored in both forms, the provider-form tag gets the transitions while the area matches, and a warning is logged. Before #895, a picker-authored alarm never went active (found in the G6 live gate, `docs/drivers/evidence/galaxy-g6-e2e-2026-11-12.md`).
- **Duplicate `attributeRef` (#605 / 06-S-13)**: two authored RawPaths dialling the same Galaxy attribute are both dialled, but the reverse map is first-wins, so value fan-out and alarm `ConditionId` surface under the first RawPath only. The loser is logged at Warning and recorded in the skipped-tag ledger with the winner path in the reason. - **Duplicate `attributeRef` (#605 / 06-S-13)**: two authored RawPaths dialling the same Galaxy attribute are both dialled, but the reverse map is first-wins, so value fan-out and alarm `ConditionId` surface under the first RawPath only. The loser is logged at Warning and recorded in the skipped-tag ledger with the winner path in the reason.
- **Subscribe-once reads are bounded (#605 / 06-S-3)**: `ReadAsync` on the production path (SubscribeBulk, first event, Unsubscribe) carries an internal 30 s deadline in addition to the caller token; slots that never publish come back `BadTimeout`, so `CancellationToken.None` can no longer hang on a silent tag. - **Subscribe-once reads are bounded (#605 / 06-S-3)**: `ReadAsync` on the production path (SubscribeBulk, first event, Unsubscribe) carries an internal 30 s deadline in addition to the caller token; slots that never publish come back `BadTimeout`, so `CancellationToken.None` can no longer hang on a silent tag.
- **Supervisory advise is retried after a thrown failure (#605 / 06-S-12)**: the writer marks an item handle advised before the call; a thrown advise (not just a non-OK status) now unmarks it, fails the write closed and re-advises on the next write. - **Supervisory advise is retried after a thrown failure (#605 / 06-S-12)**: the writer marks an item handle advised before the call; a thrown advise (not just a non-OK status) now unmarks it, fails the write closed and re-advises on the next write.
@@ -552,6 +552,73 @@ public sealed class GalaxyDriver
internal string ToAlarmAckReference(string rawPath) => internal string ToAlarmAckReference(string rawPath) =>
_alarmFullRefByRawPath.TryGetValue(rawPath, out var full) ? full : ToAttributeRef(rawPath); _alarmFullRefByRawPath.TryGetValue(rawPath, out var full) ? full : ToAttributeRef(rawPath);
/// <summary>
/// Upper bound on how long an acknowledge of a picker-form alarm waits for the alarm feed's first snapshot
/// (lmxopcua#904). Kept well inside the host's AlarmAcknowledge capability timeout (5 s at the default tier),
/// which also covers the credential lookup and the gateway RPC.
/// </summary>
internal TimeSpan AckSnapshotWait { get; set; } = TimeSpan.FromSeconds(2);
/// <summary>Test-visible: the wired alarm feed's first-snapshot signal (lmxopcua#904 live gate), null without a feed.</summary>
internal Task? AlarmFeedInitialSnapshotForTests
{
get { lock (_alarmFeedLock) return _alarmFeed?.InitialSnapshotCompleted; }
}
/// <summary>
/// The reference an acknowledge of <paramref name="rawPath"/> dials (lmxopcua#904). A tag authored in the
/// picker form (<c>tag.Attr</c>, no <c>Provider!</c>) only acquires the provider-qualified reference the
/// gateway's <c>AcknowledgeAlarm</c> accepts from the alarm feed. Right after a host restart the condition
/// can be restored from LocalDb before the feed's first snapshot reached this driver instance, so when no
/// reference has been learned yet the ack waits — bounded by <see cref="AckSnapshotWait"/> and the caller's
/// token — for that snapshot, which names every active or unacknowledged alarm in its current area. Nothing
/// is guessed: the area comes only from the feed (so a re-areaed object acks under its current area), and
/// when the feed still has not named the alarm the authored reference is dialled as before and the gateway
/// refuses it visibly.
/// </summary>
/// <param name="rawPath">The condition's RawPath (its ConditionId).</param>
/// <param name="cancellationToken">The acknowledge's cancellation token.</param>
/// <returns>The gateway alarm reference to acknowledge.</returns>
internal async Task<string> ResolveAlarmAckReferenceAsync(string rawPath, CancellationToken cancellationToken)
{
var reference = ToAlarmAckReference(rawPath);
if (_alarmFullRefByRawPath.ContainsKey(rawPath)
|| !_attributeRefByRawPath.ContainsKey(rawPath)
|| reference.Contains('!'))
{
return reference;
}
Task snapshot;
lock (_alarmFeedLock)
{
// A feed that is not running delivers no snapshot — nothing to wait for.
if (_alarmFeed is null || !_alarmFeedWired) return reference;
snapshot = _alarmFeed.InitialSnapshotCompleted;
}
// Re-read: the snapshot may have landed since the first lookup.
if (snapshot.IsCompleted) return ToAlarmAckReference(rawPath);
try
{
await snapshot.WaitAsync(AckSnapshotWait, cancellationToken).ConfigureAwait(false);
}
catch (TimeoutException)
{
// Fall through: dial what we have; the gateway refuses it and the host reverts the ack.
}
var learned = ToAlarmAckReference(rawPath);
if (learned == reference)
{
_logger.LogWarning(
"Galaxy driver {DriverInstanceId}: acknowledge of {RawPath} dials the area-free {AlarmRef} — the alarm feed has {State} named this alarm with its provider-qualified reference, so the gateway is expected to refuse it.",
_driverInstanceId, rawPath, reference,
snapshot.IsCompleted ? "not" : "not yet (its first snapshot did not arrive in time)");
}
return learned;
}
/// <summary><c>Galaxy!Area.tag.Attr</c> → <c>tag.Attr</c>; null when the reference has no <c>!</c> provider /// <summary><c>Galaxy!Area.tag.Attr</c> → <c>tag.Attr</c>; null when the reference has no <c>!</c> provider
/// prefix or nothing follows the area segment.</summary> /// prefix or nothing follows the area segment.</summary>
private static string? StripProviderAndArea(string alarmFullReference) private static string? StripProviderAndArea(string alarmFullReference)
@@ -2740,7 +2807,8 @@ public sealed class GalaxyDriver
var conditionReference = !string.IsNullOrEmpty(ack.ConditionId) var conditionReference = !string.IsNullOrEmpty(ack.ConditionId)
? ack.ConditionId ? ack.ConditionId
: ack.SourceNodeId; : ack.SourceNodeId;
var alarmFullReference = ToAlarmAckReference(conditionReference); var alarmFullReference = await ResolveAlarmAckReferenceAsync(conditionReference, cancellationToken)
.ConfigureAwait(false);
// lmxopcua#749: the gateway acknowledges as a real Galaxy account (AuthenticateUser + // lmxopcua#749: the gateway acknowledges as a real Galaxy account (AuthenticateUser +
// AckMsg write), and the OPC UA session has no Galaxy password. Resolve the account mapped // AckMsg write), and the OPC UA session has no Galaxy password. Resolve the account mapped
@@ -65,6 +65,9 @@ internal sealed class GatewayGalaxyAlarmFeed : IGalaxyAlarmFeed
private readonly KeyValuePair<string, object?> _clientTag; private readonly KeyValuePair<string, object?> _clientTag;
private readonly CancellationTokenSource _cts = new(); private readonly CancellationTokenSource _cts = new();
// lmxopcua#904: completed by the first snapshot_complete sentinel — after every snapshot entry was raised.
private readonly TaskCompletionSource _initialSnapshot = new(TaskCreationOptions.RunContinuationsAsynchronously);
private Task? _loop; private Task? _loop;
private bool _disposed; private bool _disposed;
@@ -91,6 +94,9 @@ internal sealed class GatewayGalaxyAlarmFeed : IGalaxyAlarmFeed
_clientTag = new KeyValuePair<string, object?>("galaxy.client", clientName ?? "<unknown>"); _clientTag = new KeyValuePair<string, object?>("galaxy.client", clientName ?? "<unknown>");
} }
/// <inheritdoc />
public Task InitialSnapshotCompleted => _initialSnapshot.Task;
/// <inheritdoc /> /// <inheritdoc />
public void Start() public void Start()
{ {
@@ -159,6 +165,7 @@ internal sealed class GatewayGalaxyAlarmFeed : IGalaxyAlarmFeed
break; break;
case AlarmFeedMessage.PayloadOneofCase.SnapshotComplete: case AlarmFeedMessage.PayloadOneofCase.SnapshotComplete:
_logger.LogDebug("Galaxy alarm feed active-alarm snapshot complete."); _logger.LogDebug("Galaxy alarm feed active-alarm snapshot complete.");
_initialSnapshot.TrySetResult();
break; break;
default: default:
// Empty oneof — worker / gateway version skew. Count and drop. // Empty oneof — worker / gateway version skew. Count and drop.
@@ -26,4 +26,14 @@ internal interface IGalaxyAlarmFeed : IAsyncDisposable
/// calls are no-ops while the loop is running. /// calls are no-ops while the loop is running.
/// </summary> /// </summary>
void Start(); void Start();
/// <summary>
/// Completes once the feed has delivered its first full active-alarm snapshot (every snapshot entry has
/// been raised on <see cref="OnAlarmTransition"/>, then the gateway's <c>snapshot_complete</c> sentinel
/// arrived). Never faults; stays incomplete while the feed is not started or the gateway has not answered.
/// lmxopcua#904: an acknowledge of a picker-form alarm waits on it (bounded) so the provider-qualified
/// reference the snapshot names is known before the ack is dialled. The default is an already-completed
/// task, for feeds with no snapshot phase (the test fakes raise synchronously).
/// </summary>
Task InitialSnapshotCompleted => Task.CompletedTask;
} }
@@ -348,6 +348,140 @@ public sealed class GalaxyDriverRawPathKeyingTests
await TestWait.ForAsync(confirm, "the feed's acknowledged transition must confirm the picker-form ack"); await TestWait.ForAsync(confirm, "the feed's acknowledged transition must confirm the picker-form ack");
} }
// ===== lmxopcua#904: a picker-form ack before the feed has named the alarm =====
private static GalaxyDriver NewAckDriver(IGalaxyAlarmFeed feed, IGalaxyAlarmAcknowledger ack, GalaxyDriverOptions? opts = null)
=> new("g", opts ?? OptsWithRawTags(), hierarchySource: null, alarmAcknowledger: ack, alarmFeed: feed,
ackCredentialResolver: FixedAckCredentialResolver.Fallback());
/// <summary>
/// lmxopcua#904: after a host restart the condition is restored from LocalDb before the feed's first snapshot
/// reaches the new driver instance. An ack in that window must wait for the snapshot and dial the
/// provider-qualified reference it names — in the object's CURRENT area — not the bare <c>tag.Attr</c> the
/// gateway refuses.
/// </summary>
[Fact]
public async Task PickerForm_AckBeforeFeedSnapshot_WaitsForTheSnapshot_AndDialsItsFullReference()
{
const string fullRef = "Galaxy!Area2.Tank1.Level";
var feed = new SnapshotAlarmFeed();
var ack = new RecordingAcknowledger();
using var driver = NewAckDriver(feed, ack);
driver.AckSnapshotWait = TimeSpan.FromMinutes(2);
await driver.SubscribeAlarmsAsync(["Tank1"], CancellationToken.None);
var pending = driver.AcknowledgeAsync(
[new AlarmAcknowledgeRequest("Tank1", RawPath, "ack it")], CancellationToken.None);
ack.Calls.ShouldBeEmpty(); // nothing dialled while the feed has not named the alarm
feed.Emit(NewTransition(fullRef, AttributeRef) with
{
TransitionKind = GalaxyAlarmTransitionKind.Retrigger,
IsRestatement = true,
});
feed.CompleteSnapshot();
await TestWait.ForAsync(pending, "the ack must proceed once the feed snapshot arrives");
ack.Calls.ShouldHaveSingleItem().AlarmRef.ShouldBe(fullRef);
}
/// <summary>lmxopcua#904: a snapshot that does not name the alarm (not active in Galaxy) releases the ack at once;
/// nothing is guessed, the authored reference is dialled as before and the gateway decides.</summary>
[Fact]
public async Task PickerForm_SnapshotWithoutTheAlarm_DialsTheAuthoredReference()
{
var feed = new SnapshotAlarmFeed();
var ack = new RecordingAcknowledger();
using var driver = NewAckDriver(feed, ack);
driver.AckSnapshotWait = TimeSpan.FromMinutes(2);
await driver.SubscribeAlarmsAsync(["Tank1"], CancellationToken.None);
var pending = driver.AcknowledgeAsync(
[new AlarmAcknowledgeRequest("Tank1", RawPath, "ack it")], CancellationToken.None);
feed.Emit(NewTransition("Galaxy!Area1.Other.Attr", "Other.Attr"));
feed.CompleteSnapshot();
await TestWait.ForAsync(pending, "a completed snapshot must release the ack");
ack.Calls.ShouldHaveSingleItem().AlarmRef.ShouldBe(AttributeRef);
}
/// <summary>lmxopcua#904: the wait is bounded — a feed whose snapshot never comes (gateway unreachable) lets the ack
/// through after <see cref="GalaxyDriver.AckSnapshotWait"/> with the authored reference.</summary>
[Fact]
public async Task PickerForm_SnapshotNeverArrives_AckProceedsAfterTheBoundedWait()
{
var feed = new SnapshotAlarmFeed();
var ack = new RecordingAcknowledger();
using var driver = NewAckDriver(feed, ack);
driver.AckSnapshotWait = TimeSpan.FromMilliseconds(100);
await driver.SubscribeAlarmsAsync(["Tank1"], CancellationToken.None);
await driver.AcknowledgeAsync(
[new AlarmAcknowledgeRequest("Tank1", RawPath, "ack it")], CancellationToken.None);
ack.Calls.ShouldHaveSingleItem().AlarmRef.ShouldBe(AttributeRef);
}
/// <summary>lmxopcua#904: the caller's token cuts the wait, and nothing is dialled.</summary>
[Fact]
public async Task PickerForm_AckCancelledWhileWaitingForTheSnapshot_DialsNothing()
{
var feed = new SnapshotAlarmFeed();
var ack = new RecordingAcknowledger();
using var driver = NewAckDriver(feed, ack);
driver.AckSnapshotWait = TimeSpan.FromMinutes(2);
await driver.SubscribeAlarmsAsync(["Tank1"], CancellationToken.None);
using var cts = new CancellationTokenSource(TimeSpan.FromMilliseconds(100));
await Should.ThrowAsync<OperationCanceledException>(() => driver.AcknowledgeAsync(
[new AlarmAcknowledgeRequest("Tank1", RawPath, "ack it")], cts.Token));
ack.Calls.ShouldBeEmpty();
}
/// <summary>lmxopcua#904: no wait where the feed has nothing to add — a provider-form tag already carries the
/// reference, and a feed that is not running sends no snapshot.</summary>
[Fact]
public async Task NoSnapshotWait_ForProviderFormTags_OrWhenTheFeedIsNotRunning()
{
var feed = new SnapshotAlarmFeed();
var ack = new RecordingAcknowledger();
var providerOpts = OptsWithRawTags() with
{
RawTags = [new RawTagEntry(RawPath, "{\"attributeRef\":\"Galaxy!Area1.Tank1.Level\"}", false)],
};
using (var driver = NewAckDriver(feed, ack, providerOpts))
{
driver.AckSnapshotWait = TimeSpan.FromMinutes(2);
await driver.SubscribeAlarmsAsync(["Tank1"], CancellationToken.None);
var pending = driver.AcknowledgeAsync(
[new AlarmAcknowledgeRequest("Tank1", RawPath, "ack it")], CancellationToken.None);
await TestWait.ForAsync(pending, "a provider-form ack must not wait for the snapshot");
}
using (var driver = NewAckDriver(new SnapshotAlarmFeed(), ack))
{
driver.AckSnapshotWait = TimeSpan.FromMinutes(2);
var pending = driver.AcknowledgeAsync( // feed never started: no SubscribeAlarmsAsync
[new AlarmAcknowledgeRequest("Tank1", RawPath, "ack it")], CancellationToken.None);
await TestWait.ForAsync(pending, "an ack must not wait on a feed that is not running");
}
ack.Calls.Select(c => c.AlarmRef).ShouldBe(["Galaxy!Area1.Tank1.Level", AttributeRef]);
}
/// <summary>A feed with a snapshot phase the test completes by hand, like <see cref="GatewayGalaxyAlarmFeed"/>.</summary>
private sealed class SnapshotAlarmFeed : IGalaxyAlarmFeed
{
private readonly TaskCompletionSource _snapshot = new(TaskCreationOptions.RunContinuationsAsynchronously);
public event EventHandler<GalaxyAlarmTransition>? OnAlarmTransition;
public Task InitialSnapshotCompleted => _snapshot.Task;
public void Start() { }
public void Emit(GalaxyAlarmTransition transition) => OnAlarmTransition?.Invoke(this, transition);
public void CompleteSnapshot() => _snapshot.TrySetResult();
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
}
private sealed class UnconfirmedAcknowledger : IGalaxyAlarmAcknowledger private sealed class UnconfirmedAcknowledger : IGalaxyAlarmAcknowledger
{ {
public Task<GalaxyAckOutcome> AcknowledgeAsync( public Task<GalaxyAckOutcome> AcknowledgeAsync(
@@ -0,0 +1,153 @@
using Microsoft.Extensions.Logging;
using Shouldly;
using Xunit;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Config;
using ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Runtime;
using ZB.MOM.WW.Secrets.Abstractions;
namespace ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Tests.Runtime;
/// <summary>
/// lmxopcua#904 live gate: an acknowledge of a picker-form native alarm (<c>attributeRef = tag.Attr</c>) issued by a
/// freshly started driver BEFORE its alarm feed has delivered the first snapshot — the window a host restart opens
/// when it restores the condition from LocalDb. The driver must wait for the snapshot and dial the
/// provider-qualified reference it names, and Galaxy must record the ack.
/// <list type="number">
/// <item><description>Raise the alarm with a supervisory write through a separate driver instance.</description></item>
/// <item><description>Negative control: a fresh driver whose feed has named nothing dials the bare
/// <c>tag.Attr</c>, and the gateway refuses it (the pre-#904 behaviour).</description></item>
/// <item><description>A fresh driver with the production wait acks before its snapshot arrived: the ack
/// dials <c>Galaxy!&lt;Area&gt;.tag.Attr</c> and Galaxy's acknowledged transition confirms it.</description></item>
/// <item><description>Clear the alarm.</description></item>
/// </list>
/// <para>
/// Skip-gated: runs only when <c>MXGW_ENDPOINT</c>, <c>GALAXY_MXGW_API_KEY</c> and <c>GALAXY_904_ALARM_REF</c> (a
/// Boolean alarm attribute on a TEST Galaxy, e.g. <c>TestMachine_015.TestAlarm003</c>) are set. It raises, acks and
/// clears that alarm. The ack account is <c>GALAXY_904_ACK_USER</c> (default <c>Administrator</c>) with the
/// password in <c>GALAXY_904_ACK_PASSWORD</c> (default empty — the windev rig's identity).
/// </para>
/// </summary>
[Trait("Category", "Integration")]
public sealed class GalaxyPickerAckBeforeSnapshotLiveTests
{
private const string RawPath = "g904/dev/alarm";
[Fact]
public async Task Live_picker_form_ack_before_the_feed_snapshot_dials_the_provider_reference_and_reaches_galaxy()
{
var endpoint = Environment.GetEnvironmentVariable("MXGW_ENDPOINT");
var alarmRef = Environment.GetEnvironmentVariable("GALAXY_904_ALARM_REF");
if (string.IsNullOrWhiteSpace(endpoint) || string.IsNullOrWhiteSpace(alarmRef)
|| string.IsNullOrWhiteSpace(Environment.GetEnvironmentVariable("GALAXY_MXGW_API_KEY")))
Assert.Skip("Set MXGW_ENDPOINT + GALAXY_MXGW_API_KEY + GALAXY_904_ALARM_REF to run the #904 live gate.");
alarmRef.ShouldNotContain("!", customMessage: "the gate needs the picker form tag.Attr");
var ct = TestContext.Current.CancellationToken;
await using var raiser = NewDriver("g904-raise", endpoint!, alarmRef!, out _);
await raiser.InitializeAsync("{}", ct);
try
{
// 1. Raise, and wait until the gateway feed reports it active + unacked.
await WriteAsync(raiser, true, ct);
var raised = new TaskCompletionSource<string>(TaskCreationOptions.RunContinuationsAsynchronously);
raiser.OnAlarmEvent += (_, e) =>
{
if (e.ConditionId == RawPath && e.Kind is AlarmTransitionKind.Raise or AlarmTransitionKind.Retrigger)
raised.TrySetResult(raiser.ToAlarmAckReference(RawPath));
};
await raiser.SubscribeAlarmsAsync([RawPath], ct);
var fullRef = await TestWait.ForAsync(raised.Task, "the gateway feed must report the raised alarm");
fullRef.ShouldStartWith("Galaxy!");
fullRef.ShouldEndWith("." + alarmRef);
TestContext.Current.SendDiagnosticMessage($"#904 live: feed names the alarm {fullRef}");
// 2. Negative control — the pre-#904 state, deterministically: a fresh driver whose feed has named nothing
// (not started, so nothing to wait for) dials the bare tag.Attr, and the gateway refuses it.
await using (var control = NewDriver("g904-control", endpoint!, alarmRef!, out var controlLog))
{
await control.InitializeAsync("{}", ct);
control.ToAlarmAckReference(RawPath).ShouldBe(alarmRef);
Exception? refused = null;
try
{
await control.AcknowledgeAsync(
[new AlarmAcknowledgeRequest(RawPath, RawPath, "#904 control (must be refused)")], ct);
}
catch (Exception ex) { refused = ex; }
string text;
lock (controlLog.Entries) text = controlLog.AllText;
refused.ShouldNotBeNull($"the gateway must refuse the bare reference; log=\n{text}");
TestContext.Current.SendDiagnosticMessage($"#904 live control: {refused.GetType().Name}: {refused.Message}");
text.ShouldContain($"acknowledging {alarmRef} as Galaxy user", customMessage: text);
}
// 3. The fix — production wait: ack before the snapshot, dial the provider reference, Galaxy confirms.
await using (var restarted = NewDriver("g904-restarted", endpoint!, alarmRef!, out var log))
{
restarted.AckSnapshotWait.ShouldBe(TimeSpan.FromSeconds(2), "exercise the production wait");
await restarted.InitializeAsync("{}", ct);
await restarted.SubscribeAlarmsAsync([RawPath], ct);
var snapshot = restarted.AlarmFeedInitialSnapshotForTests!;
snapshot.IsCompleted.ShouldBeFalse("the ack must be issued before the feed snapshot arrived");
restarted.ToAlarmAckReference(RawPath).ShouldBe(alarmRef, "nothing learned yet — the ack must have to wait");
AlarmAcknowledgeRequest[] request = [new AlarmAcknowledgeRequest(RawPath, RawPath, "OtOpcUa #904 live gate")];
await restarted.AcknowledgeAsync(request, ct);
await restarted.ConfirmAcknowledgedAsync(request, ct);
snapshot.IsCompletedSuccessfully.ShouldBeTrue();
lock (log.Entries)
{
log.AllText.ShouldContain($"acknowledging {fullRef} as Galaxy user", customMessage: log.AllText);
log.AllText.ShouldNotContain("dials the area-free", customMessage: log.AllText);
}
TestContext.Current.SendDiagnosticMessage($"#904 live: ack before snapshot dialled {fullRef} and was confirmed");
}
}
finally
{
await WriteAsync(raiser, false, CancellationToken.None);
}
}
private static async Task WriteAsync(GalaxyDriver driver, bool value, CancellationToken ct)
{
var results = await driver.WriteAsync([new WriteRequest(RawPath, value)], ct);
results.ShouldHaveSingleItem().StatusCode.ShouldBe(StatusCodeMap.Good, $"write {value}");
}
private static GalaxyDriver NewDriver(string id, string endpoint, string alarmRef, out CapturingLogger log)
{
log = new CapturingLogger();
var opts = new GalaxyDriverOptions(
new GalaxyGatewayOptions(endpoint, "env:GALAXY_MXGW_API_KEY", UseTls: false),
new GalaxyMxAccessOptions("OtOpcUa904-" + id),
new GalaxyRepositoryOptions(WatchDeployEvents: false),
new GalaxyReconnectOptions())
{
RawTags = [new RawTagEntry(RawPath, $"{{\"attributeRef\":\"{alarmRef}\"}}", false)],
AlarmAck = new GalaxyAlarmAckOptions(
FallbackUser: Environment.GetEnvironmentVariable("GALAXY_904_ACK_USER") is { Length: > 0 } u ? u : "Administrator",
FallbackPasswordSecretRef: Environment.GetEnvironmentVariable("GALAXY_904_ACK_PASSWORD") is { Length: > 0 }
? "env:GALAXY_904_ACK_PASSWORD" : "dev:"),
};
return new GalaxyDriver(id, opts, new NullResolver(), new TypedLogger(log));
}
private sealed class NullResolver : ISecretResolver
{
public Task<string?> GetAsync(SecretName name, CancellationToken ct) => Task.FromResult<string?>(null);
}
private sealed class TypedLogger(CapturingLogger inner) : ILogger<GalaxyDriver>
{
public IDisposable? BeginScope<TState>(TState state) where TState : notnull => null;
public bool IsEnabled(LogLevel logLevel) => true;
public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception? exception,
Func<TState, Exception?, string> formatter)
{
lock (inner.Entries) inner.Log(logLevel, eventId, state, exception, formatter);
}
}
}
@@ -197,6 +197,45 @@ public sealed class GatewayGalaxyAlarmFeedTests
observed[0].TransitionKind.ShouldBe(GalaxyAlarmTransitionKind.Raise); observed[0].TransitionKind.ShouldBe(GalaxyAlarmTransitionKind.Raise);
} }
/// <summary>
/// lmxopcua#904: <see cref="GatewayGalaxyAlarmFeed.InitialSnapshotCompleted"/> completes on the
/// <c>snapshot_complete</c> sentinel, after every snapshot entry before it was raised — the driver's ack path
/// relies on that order to find the provider-qualified reference the snapshot named. It stays incomplete
/// until the sentinel arrives.
/// </summary>
[Fact]
public async Task InitialSnapshotCompleted_completes_on_the_sentinel_after_the_snapshot_entries()
{
var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
var observedAtCompletion = -1;
var observed = 0;
await using var feed = new GatewayGalaxyAlarmFeed(
(_, ct) => GatedStream(release.Task, ct), clientName: "SnapshotSignalTest");
feed.OnAlarmTransition += (_, _) => Interlocked.Increment(ref observed);
var completed = feed.InitialSnapshotCompleted.ContinueWith(
_ => observedAtCompletion = Volatile.Read(ref observed), TaskScheduler.Default);
feed.Start();
await TestWait.UntilAsync(() => Volatile.Read(ref observed) == 2, "both snapshot entries should dispatch");
feed.InitialSnapshotCompleted.IsCompleted.ShouldBeFalse(); // entries in, sentinel not yet
release.SetResult();
await TestWait.ForAsync(completed, "the snapshot_complete sentinel should complete the signal");
observedAtCompletion.ShouldBe(2);
static async IAsyncEnumerable<AlarmFeedMessage> GatedStream(
Task gate, [EnumeratorCancellation] CancellationToken ct = default)
{
var now = DateTime.UtcNow;
yield return SnapshotMessage("Galaxy!Area1.Tank01.Level.HiHi", AlarmConditionState.Active, 750, now);
yield return SnapshotMessage("Galaxy!Area1.Tank02.Level.HiHi", AlarmConditionState.Active, 750, now);
await gate.WaitAsync(ct);
yield return new AlarmFeedMessage { SnapshotComplete = true };
await Task.Delay(Timeout.Infinite, ct);
}
}
/// <summary> /// <summary>
/// Yields each message in order, then holds the stream open until the feed is /// Yields each message in order, then holds the stream open until the feed is
/// disposed — mirrors a live server-streaming RPC that does not complete on its /// disposed — mirrors a live server-streaming RPC that does not complete on its