fix(galaxy): picker-form alarm ack waits for the feed snapshot instead of dialling bare tag.Attr (#904)
v2-ci / unit-tests (push) Has been cancelled

A picker-form native alarm (attributeRef tag.Attr) only learns the provider-qualified reference
Galaxy!<Area>.tag.Attr from the alarm feed. After a host restart the condition can be restored from
LocalDb before the feed's first snapshot reached the new driver, and an ack in that window dialled the
bare tag.Attr, which the gateway's AcknowledgeAlarm refuses.

The feed now exposes InitialSnapshotCompleted (set on snapshot_complete, after every snapshot entry has
been raised). AcknowledgeAsync resolves the reference through ResolveAlarmAckReferenceAsync: when a
picker-form tag has learned nothing yet and the feed is running, it waits for that snapshot (2 s, and the
caller's token) and dials the reference the snapshot names, so the area always comes from the feed and a
re-areaed object acks under its current area. No reference is guessed: if the snapshot does not name the
alarm, or does not arrive in time, the authored reference is dialled as before and refused visibly.

Chosen over the alternatives in the issue: a GR-browse area lookup can be stale against the runtime
area and is not loaded for authored tags, and a gateway-side tag.Attr resolution needs a cross-repo
deploy and would resolve from the same monitor cache the snapshot already carries.

Tests: 5 driver cases + 1 feed case (fail with the call site / sentinel reverted), and a skip-gated live
gate GalaxyPickerAckBeforeSnapshotLiveTests, run against windev's TEST Galaxy (TestMachine_015.TestAlarm003):
an ack issued before the snapshot dials Galaxy!TestArea.TestMachine_015.TestAlarm003 and is confirmed; with
the call site reverted it fails with the gateway's InvalidRequest refusal.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
2026-11-13 02:07:38 -05:00
parent f0c6cf9640
commit c9640f5ced
7 changed files with 413 additions and 2 deletions
@@ -348,6 +348,140 @@ public sealed class GalaxyDriverRawPathKeyingTests
await TestWait.ForAsync(confirm, "the feed's acknowledged transition must confirm the picker-form ack");
}
// ===== lmxopcua#904: a picker-form ack before the feed has named the alarm =====
private static GalaxyDriver NewAckDriver(IGalaxyAlarmFeed feed, IGalaxyAlarmAcknowledger ack, GalaxyDriverOptions? opts = null)
=> new("g", opts ?? OptsWithRawTags(), hierarchySource: null, alarmAcknowledger: ack, alarmFeed: feed,
ackCredentialResolver: FixedAckCredentialResolver.Fallback());
/// <summary>
/// lmxopcua#904: after a host restart the condition is restored from LocalDb before the feed's first snapshot
/// reaches the new driver instance. An ack in that window must wait for the snapshot and dial the
/// provider-qualified reference it names — in the object's CURRENT area — not the bare <c>tag.Attr</c> the
/// gateway refuses.
/// </summary>
[Fact]
public async Task PickerForm_AckBeforeFeedSnapshot_WaitsForTheSnapshot_AndDialsItsFullReference()
{
const string fullRef = "Galaxy!Area2.Tank1.Level";
var feed = new SnapshotAlarmFeed();
var ack = new RecordingAcknowledger();
using var driver = NewAckDriver(feed, ack);
driver.AckSnapshotWait = TimeSpan.FromMinutes(2);
await driver.SubscribeAlarmsAsync(["Tank1"], CancellationToken.None);
var pending = driver.AcknowledgeAsync(
[new AlarmAcknowledgeRequest("Tank1", RawPath, "ack it")], CancellationToken.None);
ack.Calls.ShouldBeEmpty(); // nothing dialled while the feed has not named the alarm
feed.Emit(NewTransition(fullRef, AttributeRef) with
{
TransitionKind = GalaxyAlarmTransitionKind.Retrigger,
IsRestatement = true,
});
feed.CompleteSnapshot();
await TestWait.ForAsync(pending, "the ack must proceed once the feed snapshot arrives");
ack.Calls.ShouldHaveSingleItem().AlarmRef.ShouldBe(fullRef);
}
/// <summary>lmxopcua#904: a snapshot that does not name the alarm (not active in Galaxy) releases the ack at once;
/// nothing is guessed, the authored reference is dialled as before and the gateway decides.</summary>
[Fact]
public async Task PickerForm_SnapshotWithoutTheAlarm_DialsTheAuthoredReference()
{
var feed = new SnapshotAlarmFeed();
var ack = new RecordingAcknowledger();
using var driver = NewAckDriver(feed, ack);
driver.AckSnapshotWait = TimeSpan.FromMinutes(2);
await driver.SubscribeAlarmsAsync(["Tank1"], CancellationToken.None);
var pending = driver.AcknowledgeAsync(
[new AlarmAcknowledgeRequest("Tank1", RawPath, "ack it")], CancellationToken.None);
feed.Emit(NewTransition("Galaxy!Area1.Other.Attr", "Other.Attr"));
feed.CompleteSnapshot();
await TestWait.ForAsync(pending, "a completed snapshot must release the ack");
ack.Calls.ShouldHaveSingleItem().AlarmRef.ShouldBe(AttributeRef);
}
/// <summary>lmxopcua#904: the wait is bounded — a feed whose snapshot never comes (gateway unreachable) lets the ack
/// through after <see cref="GalaxyDriver.AckSnapshotWait"/> with the authored reference.</summary>
[Fact]
public async Task PickerForm_SnapshotNeverArrives_AckProceedsAfterTheBoundedWait()
{
var feed = new SnapshotAlarmFeed();
var ack = new RecordingAcknowledger();
using var driver = NewAckDriver(feed, ack);
driver.AckSnapshotWait = TimeSpan.FromMilliseconds(100);
await driver.SubscribeAlarmsAsync(["Tank1"], CancellationToken.None);
await driver.AcknowledgeAsync(
[new AlarmAcknowledgeRequest("Tank1", RawPath, "ack it")], CancellationToken.None);
ack.Calls.ShouldHaveSingleItem().AlarmRef.ShouldBe(AttributeRef);
}
/// <summary>lmxopcua#904: the caller's token cuts the wait, and nothing is dialled.</summary>
[Fact]
public async Task PickerForm_AckCancelledWhileWaitingForTheSnapshot_DialsNothing()
{
var feed = new SnapshotAlarmFeed();
var ack = new RecordingAcknowledger();
using var driver = NewAckDriver(feed, ack);
driver.AckSnapshotWait = TimeSpan.FromMinutes(2);
await driver.SubscribeAlarmsAsync(["Tank1"], CancellationToken.None);
using var cts = new CancellationTokenSource(TimeSpan.FromMilliseconds(100));
await Should.ThrowAsync<OperationCanceledException>(() => driver.AcknowledgeAsync(
[new AlarmAcknowledgeRequest("Tank1", RawPath, "ack it")], cts.Token));
ack.Calls.ShouldBeEmpty();
}
/// <summary>lmxopcua#904: no wait where the feed has nothing to add — a provider-form tag already carries the
/// reference, and a feed that is not running sends no snapshot.</summary>
[Fact]
public async Task NoSnapshotWait_ForProviderFormTags_OrWhenTheFeedIsNotRunning()
{
var feed = new SnapshotAlarmFeed();
var ack = new RecordingAcknowledger();
var providerOpts = OptsWithRawTags() with
{
RawTags = [new RawTagEntry(RawPath, "{\"attributeRef\":\"Galaxy!Area1.Tank1.Level\"}", false)],
};
using (var driver = NewAckDriver(feed, ack, providerOpts))
{
driver.AckSnapshotWait = TimeSpan.FromMinutes(2);
await driver.SubscribeAlarmsAsync(["Tank1"], CancellationToken.None);
var pending = driver.AcknowledgeAsync(
[new AlarmAcknowledgeRequest("Tank1", RawPath, "ack it")], CancellationToken.None);
await TestWait.ForAsync(pending, "a provider-form ack must not wait for the snapshot");
}
using (var driver = NewAckDriver(new SnapshotAlarmFeed(), ack))
{
driver.AckSnapshotWait = TimeSpan.FromMinutes(2);
var pending = driver.AcknowledgeAsync( // feed never started: no SubscribeAlarmsAsync
[new AlarmAcknowledgeRequest("Tank1", RawPath, "ack it")], CancellationToken.None);
await TestWait.ForAsync(pending, "an ack must not wait on a feed that is not running");
}
ack.Calls.Select(c => c.AlarmRef).ShouldBe(["Galaxy!Area1.Tank1.Level", AttributeRef]);
}
/// <summary>A feed with a snapshot phase the test completes by hand, like <see cref="GatewayGalaxyAlarmFeed"/>.</summary>
private sealed class SnapshotAlarmFeed : IGalaxyAlarmFeed
{
private readonly TaskCompletionSource _snapshot = new(TaskCreationOptions.RunContinuationsAsynchronously);
public event EventHandler<GalaxyAlarmTransition>? OnAlarmTransition;
public Task InitialSnapshotCompleted => _snapshot.Task;
public void Start() { }
public void Emit(GalaxyAlarmTransition transition) => OnAlarmTransition?.Invoke(this, transition);
public void CompleteSnapshot() => _snapshot.TrySetResult();
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
}
private sealed class UnconfirmedAcknowledger : IGalaxyAlarmAcknowledger
{
public Task<GalaxyAckOutcome> AcknowledgeAsync(
@@ -0,0 +1,153 @@
using Microsoft.Extensions.Logging;
using Shouldly;
using Xunit;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Config;
using ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Runtime;
using ZB.MOM.WW.Secrets.Abstractions;
namespace ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Tests.Runtime;
/// <summary>
/// lmxopcua#904 live gate: an acknowledge of a picker-form native alarm (<c>attributeRef = tag.Attr</c>) issued by a
/// freshly started driver BEFORE its alarm feed has delivered the first snapshot — the window a host restart opens
/// when it restores the condition from LocalDb. The driver must wait for the snapshot and dial the
/// provider-qualified reference it names, and Galaxy must record the ack.
/// <list type="number">
/// <item><description>Raise the alarm with a supervisory write through a separate driver instance.</description></item>
/// <item><description>Negative control: a fresh driver whose feed has named nothing dials the bare
/// <c>tag.Attr</c>, and the gateway refuses it (the pre-#904 behaviour).</description></item>
/// <item><description>A fresh driver with the production wait acks before its snapshot arrived: the ack
/// dials <c>Galaxy!&lt;Area&gt;.tag.Attr</c> and Galaxy's acknowledged transition confirms it.</description></item>
/// <item><description>Clear the alarm.</description></item>
/// </list>
/// <para>
/// Skip-gated: runs only when <c>MXGW_ENDPOINT</c>, <c>GALAXY_MXGW_API_KEY</c> and <c>GALAXY_904_ALARM_REF</c> (a
/// Boolean alarm attribute on a TEST Galaxy, e.g. <c>TestMachine_015.TestAlarm003</c>) are set. It raises, acks and
/// clears that alarm. The ack account is <c>GALAXY_904_ACK_USER</c> (default <c>Administrator</c>) with the
/// password in <c>GALAXY_904_ACK_PASSWORD</c> (default empty — the windev rig's identity).
/// </para>
/// </summary>
[Trait("Category", "Integration")]
public sealed class GalaxyPickerAckBeforeSnapshotLiveTests
{
private const string RawPath = "g904/dev/alarm";
[Fact]
public async Task Live_picker_form_ack_before_the_feed_snapshot_dials_the_provider_reference_and_reaches_galaxy()
{
var endpoint = Environment.GetEnvironmentVariable("MXGW_ENDPOINT");
var alarmRef = Environment.GetEnvironmentVariable("GALAXY_904_ALARM_REF");
if (string.IsNullOrWhiteSpace(endpoint) || string.IsNullOrWhiteSpace(alarmRef)
|| string.IsNullOrWhiteSpace(Environment.GetEnvironmentVariable("GALAXY_MXGW_API_KEY")))
Assert.Skip("Set MXGW_ENDPOINT + GALAXY_MXGW_API_KEY + GALAXY_904_ALARM_REF to run the #904 live gate.");
alarmRef.ShouldNotContain("!", customMessage: "the gate needs the picker form tag.Attr");
var ct = TestContext.Current.CancellationToken;
await using var raiser = NewDriver("g904-raise", endpoint!, alarmRef!, out _);
await raiser.InitializeAsync("{}", ct);
try
{
// 1. Raise, and wait until the gateway feed reports it active + unacked.
await WriteAsync(raiser, true, ct);
var raised = new TaskCompletionSource<string>(TaskCreationOptions.RunContinuationsAsynchronously);
raiser.OnAlarmEvent += (_, e) =>
{
if (e.ConditionId == RawPath && e.Kind is AlarmTransitionKind.Raise or AlarmTransitionKind.Retrigger)
raised.TrySetResult(raiser.ToAlarmAckReference(RawPath));
};
await raiser.SubscribeAlarmsAsync([RawPath], ct);
var fullRef = await TestWait.ForAsync(raised.Task, "the gateway feed must report the raised alarm");
fullRef.ShouldStartWith("Galaxy!");
fullRef.ShouldEndWith("." + alarmRef);
TestContext.Current.SendDiagnosticMessage($"#904 live: feed names the alarm {fullRef}");
// 2. Negative control — the pre-#904 state, deterministically: a fresh driver whose feed has named nothing
// (not started, so nothing to wait for) dials the bare tag.Attr, and the gateway refuses it.
await using (var control = NewDriver("g904-control", endpoint!, alarmRef!, out var controlLog))
{
await control.InitializeAsync("{}", ct);
control.ToAlarmAckReference(RawPath).ShouldBe(alarmRef);
Exception? refused = null;
try
{
await control.AcknowledgeAsync(
[new AlarmAcknowledgeRequest(RawPath, RawPath, "#904 control (must be refused)")], ct);
}
catch (Exception ex) { refused = ex; }
string text;
lock (controlLog.Entries) text = controlLog.AllText;
refused.ShouldNotBeNull($"the gateway must refuse the bare reference; log=\n{text}");
TestContext.Current.SendDiagnosticMessage($"#904 live control: {refused.GetType().Name}: {refused.Message}");
text.ShouldContain($"acknowledging {alarmRef} as Galaxy user", customMessage: text);
}
// 3. The fix — production wait: ack before the snapshot, dial the provider reference, Galaxy confirms.
await using (var restarted = NewDriver("g904-restarted", endpoint!, alarmRef!, out var log))
{
restarted.AckSnapshotWait.ShouldBe(TimeSpan.FromSeconds(2), "exercise the production wait");
await restarted.InitializeAsync("{}", ct);
await restarted.SubscribeAlarmsAsync([RawPath], ct);
var snapshot = restarted.AlarmFeedInitialSnapshotForTests!;
snapshot.IsCompleted.ShouldBeFalse("the ack must be issued before the feed snapshot arrived");
restarted.ToAlarmAckReference(RawPath).ShouldBe(alarmRef, "nothing learned yet — the ack must have to wait");
AlarmAcknowledgeRequest[] request = [new AlarmAcknowledgeRequest(RawPath, RawPath, "OtOpcUa #904 live gate")];
await restarted.AcknowledgeAsync(request, ct);
await restarted.ConfirmAcknowledgedAsync(request, ct);
snapshot.IsCompletedSuccessfully.ShouldBeTrue();
lock (log.Entries)
{
log.AllText.ShouldContain($"acknowledging {fullRef} as Galaxy user", customMessage: log.AllText);
log.AllText.ShouldNotContain("dials the area-free", customMessage: log.AllText);
}
TestContext.Current.SendDiagnosticMessage($"#904 live: ack before snapshot dialled {fullRef} and was confirmed");
}
}
finally
{
await WriteAsync(raiser, false, CancellationToken.None);
}
}
private static async Task WriteAsync(GalaxyDriver driver, bool value, CancellationToken ct)
{
var results = await driver.WriteAsync([new WriteRequest(RawPath, value)], ct);
results.ShouldHaveSingleItem().StatusCode.ShouldBe(StatusCodeMap.Good, $"write {value}");
}
private static GalaxyDriver NewDriver(string id, string endpoint, string alarmRef, out CapturingLogger log)
{
log = new CapturingLogger();
var opts = new GalaxyDriverOptions(
new GalaxyGatewayOptions(endpoint, "env:GALAXY_MXGW_API_KEY", UseTls: false),
new GalaxyMxAccessOptions("OtOpcUa904-" + id),
new GalaxyRepositoryOptions(WatchDeployEvents: false),
new GalaxyReconnectOptions())
{
RawTags = [new RawTagEntry(RawPath, $"{{\"attributeRef\":\"{alarmRef}\"}}", false)],
AlarmAck = new GalaxyAlarmAckOptions(
FallbackUser: Environment.GetEnvironmentVariable("GALAXY_904_ACK_USER") is { Length: > 0 } u ? u : "Administrator",
FallbackPasswordSecretRef: Environment.GetEnvironmentVariable("GALAXY_904_ACK_PASSWORD") is { Length: > 0 }
? "env:GALAXY_904_ACK_PASSWORD" : "dev:"),
};
return new GalaxyDriver(id, opts, new NullResolver(), new TypedLogger(log));
}
private sealed class NullResolver : ISecretResolver
{
public Task<string?> GetAsync(SecretName name, CancellationToken ct) => Task.FromResult<string?>(null);
}
private sealed class TypedLogger(CapturingLogger inner) : ILogger<GalaxyDriver>
{
public IDisposable? BeginScope<TState>(TState state) where TState : notnull => null;
public bool IsEnabled(LogLevel logLevel) => true;
public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception? exception,
Func<TState, Exception?, string> formatter)
{
lock (inner.Entries) inner.Log(logLevel, eventId, state, exception, formatter);
}
}
}
@@ -197,6 +197,45 @@ public sealed class GatewayGalaxyAlarmFeedTests
observed[0].TransitionKind.ShouldBe(GalaxyAlarmTransitionKind.Raise);
}
/// <summary>
/// lmxopcua#904: <see cref="GatewayGalaxyAlarmFeed.InitialSnapshotCompleted"/> completes on the
/// <c>snapshot_complete</c> sentinel, after every snapshot entry before it was raised — the driver's ack path
/// relies on that order to find the provider-qualified reference the snapshot named. It stays incomplete
/// until the sentinel arrives.
/// </summary>
[Fact]
public async Task InitialSnapshotCompleted_completes_on_the_sentinel_after_the_snapshot_entries()
{
var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
var observedAtCompletion = -1;
var observed = 0;
await using var feed = new GatewayGalaxyAlarmFeed(
(_, ct) => GatedStream(release.Task, ct), clientName: "SnapshotSignalTest");
feed.OnAlarmTransition += (_, _) => Interlocked.Increment(ref observed);
var completed = feed.InitialSnapshotCompleted.ContinueWith(
_ => observedAtCompletion = Volatile.Read(ref observed), TaskScheduler.Default);
feed.Start();
await TestWait.UntilAsync(() => Volatile.Read(ref observed) == 2, "both snapshot entries should dispatch");
feed.InitialSnapshotCompleted.IsCompleted.ShouldBeFalse(); // entries in, sentinel not yet
release.SetResult();
await TestWait.ForAsync(completed, "the snapshot_complete sentinel should complete the signal");
observedAtCompletion.ShouldBe(2);
static async IAsyncEnumerable<AlarmFeedMessage> GatedStream(
Task gate, [EnumeratorCancellation] CancellationToken ct = default)
{
var now = DateTime.UtcNow;
yield return SnapshotMessage("Galaxy!Area1.Tank01.Level.HiHi", AlarmConditionState.Active, 750, now);
yield return SnapshotMessage("Galaxy!Area1.Tank02.Level.HiHi", AlarmConditionState.Active, 750, now);
await gate.WaitAsync(ct);
yield return new AlarmFeedMessage { SnapshotComplete = true };
await Task.Delay(Timeout.Infinite, ct);
}
}
/// <summary>
/// Yields each message in order, then holds the stream open until the feed is
/// disposed — mirrors a live server-streaming RPC that does not complete on its