fix(galaxy): confirm unconfirmed native alarm acks against Galaxy's acknowledged transition (#869)

mxaccessgw answers AcknowledgeAlarm OK both when the AckMsg write was
applied and when it was only sent (no OnWriteComplete; the reply then
carries a diagnostic). EvaluateReply treated both as success, so the H6
live gate saw a condition go Acknowledged while Galaxy never applied it.

- EvaluateReply returns Applied (no diagnostic) or Unconfirmed (diagnostic)
  and logs the gateway diagnostic on every branch that carries one.
- GalaxyAckConfirmationTracker watches the alarm feed from before the RPC
  and, for an unconfirmed reply, waits up to AlarmAck.ConfirmTimeoutMs
  (default 10 s, 0 = old behaviour) for an Acknowledge/ActiveAcknowledged
  transition; on timeout it refuses (BadTimeout + diagnostic).
- New optional IAlarmAcknowledgeConfirmation: DriverInstanceActor awaits it
  after AcknowledgeAsync, off its mailbox and outside the resilience
  pipeline; a failure is the existing #760 AlarmAckFailed revert.
- AdminUI Galaxy form carries the new knob; docs updated.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
This commit is contained in:
Joseph Doherty
2026-11-11 13:03:04 -05:00
parent e857135e8b
commit 4e492e6550
19 changed files with 1046 additions and 34 deletions
+5
View File
@@ -46,6 +46,11 @@ disabled/unspawned/stubbed driver, is decided in the host itself (#765) — and
native ack, so it marks the whole `NativeAlarmAckPreflightView` unavailable and every native ack is refused
synchronously until `BecomeSteady` (#765). A test that routes an ack and expects it to STAND must start the owning
driver (`AckingAlarmDriver` in Runtime.Tests).
**A returned `AcknowledgeAsync` is not proof either (#869):** a driver implementing `IAlarmAcknowledgeConfirmation`
(Galaxy — the gateway's OK also covers "AckMsg write sent, unconfirmed") is asked to confirm AFTER it, off the
`DriverInstanceActor` mailbox and outside the resilience pipeline; a confirmation failure is the same
`AlarmAckFailed` revert. Never await that confirmation inside the ack handler (`ReceiveAsync` suspends the mailbox)
or inside the invoker (Tier-A 5 s timeout + breaker).
Behaviours that were wrong or missing and are now fixed — keep them: the SDK's `OnTimedUnshelve` is a
**no-op** (the engine's `AutoUnshelve` owns timed-shelve expiry, and `UnshelveTime` reports the real
+10 -3
View File
@@ -483,9 +483,16 @@ from the scripted `AlarmCommandRouter`:
before anything is applied — as is every native ack while the driver host
is Stale (configuration database unreachable at boot), when no ack can be
routed or reverted (#765). Galaxy value writes have a separate correlated
write-outcome path. An awaited
ack RPC does not prove the ack reached AVEVA event history: the H6 live
verification remains [operator-gated](plans/2026-06-15-stillpending-phase-3-opcua-standards-design.md).
write-outcome path. **An OK ack reply is not proof (#869):** the gateway
also answers OK when the `AckMsg` write was only *sent* (its diagnostic
says so). The driver then waits — off the actor mailbox, outside the
resilience pipeline, bounded by `AlarmAck.ConfirmTimeoutMs` (default 10 s)
— for Galaxy's own acknowledged transition on the alarm feed
(`IAlarmAcknowledgeConfirmation`); without one the ack is reverted exactly
as above and the `BadTimeout` verdict plus the gateway diagnostic are
logged. See [`drivers/Galaxy.md` §Ack confirmation](drivers/Galaxy.md#ack-confirmation-869).
The H6 live verification of a confirmed ack reaching AVEVA event history
remains [operator-gated](plans/2026-06-15-stillpending-phase-3-opcua-standards-design.md).
Only the **Acknowledge** is routed to the driver. **A native condition
supports Acknowledge and AddComment only** (Gitea #587 review): `Confirm`,
+35 -1
View File
@@ -85,7 +85,7 @@ Project root files:
- **`MxAccess`** — `ClientName` (must be unique per OtOpcUa instance — redundancy pairs enforce uniqueness at install time), `PublishingIntervalMs` (forwarded as `buffered_update_interval_ms` on subscribe), `WriteUserId` for ArchestrA secured-write, `EventPumpChannelCapacity` (default 50_000 — one second of headroom at 50k tags / 1Hz; tune via the `galaxy.events.dropped` metric).
- **`Repository`** — `DiscoverPageSize`, `WatchDeployEvents`.
- **`Reconnect`** — `InitialBackoffMs`, `MaxBackoffMs`, `ReplayOnSessionLost` (default `true`: after a gateway session is recreated, re-issue `SubscribeBulk` for every tracked subscription; `false` leaves them stranded — Bad + Degraded, see [Reconnect + Replay](#reconnect--replay)).
- **`AlarmAck`** (optional, lmxopcua#749) — `FallbackUser`, `FallbackDomain`, `FallbackPasswordSecretRef`. See [Alarm acknowledge credential](#alarm-acknowledge-credential).
- **`AlarmAck`** (optional, lmxopcua#749) — `FallbackUser`, `FallbackDomain`, `FallbackPasswordSecretRef`. See [Alarm acknowledge credential](#alarm-acknowledge-credential). `ConfirmTimeoutMs` (default `10000`, max `120000`, `0` = do not wait, negative = default; #869) — see [Ack confirmation](#ack-confirmation-869).
### Alarm acknowledge credential
@@ -132,6 +132,40 @@ Galaxy password, so `GalaxyAckCredentialResolver` picks the account per acknowle
`Retain=true`) and logs the reason. A driver transition projected after the ack wins over a late
failure.
### Ack confirmation (#869)
The gateway answers `AcknowledgeAlarm` **OK with `hresult` 0 in two cases**: the `AckMsg` write's
`OnWriteComplete` reported success (*applied* — empty `diagnostic_message`), or the write was sent and no
completion arrived inside the worker's window (*sent but unconfirmed* — `diagnostic_message` says so,
e.g. `AckMsg write to '…' sent; OnWriteComplete not observed within 5000 ms.`). The H6 live gate
(2026-11-11, VD03) caught the second case: OtOpcUa showed the condition Acknowledged while Galaxy's
`.Acked` stayed False and no ACK row was ever historized.
- `GatewayGalaxyAlarmAcknowledger.EvaluateReply` returns **Applied** for an OK reply with no diagnostic and
**Unconfirmed** for one with a diagnostic (any unexplained diagnostic counts — the safe direction), and
logs the gateway diagnostic on every branch that carries one.
- `GalaxyAckConfirmationTracker` registers a watch for the alarm **before** the RPC (the alarm feed can
deliver the confirming transition ahead of the unary reply) and, for an unconfirmed reply, waits up to
`AlarmAck.ConfirmTimeoutMs` for an `Acknowledge` / `ActiveAcknowledged` transition of that alarm on the
alarm feed (live or snapshot re-statement; the reference match is case-insensitive). Whoever acked it,
Galaxy now shows it acknowledged — that is the truth the condition must match.
- The wait is the optional `IAlarmAcknowledgeConfirmation` capability. `DriverInstanceActor` calls it after
`AcknowledgeAsync` returned, **off its mailbox** (a detached task) and **outside the resilience pipeline**
(a wait on an upstream event: it must not be cut by the Tier-A 5 s `AlarmAcknowledge` timeout nor count
toward the breaker). The OPC UA method thread and the node-manager `Lock` are never involved — the method
already returned when the fire-and-forget route started.
- **On timeout** the confirmation throws `DriverOperationRefusedException` (`… not confirm … (BadTimeout;
gateway: <diagnostic>)`), the driver logs a Warning with the diagnostic, and the actor reports
`AlarmAckFailed`: the host re-projects the condition **unacknowledged** — the same #760 revert path.
**What the OPC UA client sees:** the `Acknowledge` call has already returned `Good` (it cannot be changed
after the fact, exactly as for every other #760 revert); the failure reaches the client as a fresh
condition event with `AckedState = false`, and the `BadTimeout` verdict is recorded in the driver and host
Warning logs. A confirming transition that arrives after the revert simply re-acknowledges the condition.
A confirmation in flight when the driver actor stops is abandoned and reverted the same way.
- `ConfirmTimeoutMs = 0` restores the pre-#869 behaviour (an unconfirmed ack counts as acknowledged, with a
Warning) — an escape hatch only. Why the gateway-side ack did not stick on VD03 is tracked upstream
(mxaccessgw#226).
The comment written to `AckMsg` is suffixed with the OPC UA identity —
`"<comment> [OPC UA: <principal>, key <keyId>]"` or `"[OPC UA: anonymous]"` — so the client is not
lost when the Historian's Operator is a shared fallback account. The password is never logged: the
@@ -0,0 +1,33 @@
namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
/// <summary>
/// Optional companion to <see cref="IAlarmSource"/> for a driver whose upstream can ACCEPT an acknowledge
/// without CONFIRMING it was applied (lmxopcua#869). mxaccessgw answers its <c>AcknowledgeAlarm</c> RPC OK
/// both when the <c>AckMsg</c> write was applied and when it was only sent with no completion observed; the
/// truth arrives later, as the alarm's own acknowledged transition on the alarm stream.
/// </summary>
/// <remarks>
/// <para>
/// The runtime calls <see cref="ConfirmAcknowledgedAsync"/> after <see cref="IAlarmSource.AcknowledgeAsync"/>
/// returned normally, with the SAME request instances, OUTSIDE the resilience pipeline (it is a wait on an
/// upstream event, not I/O, so it must neither be cut by the capability timeout nor count toward the
/// breaker) and OFF the driver actor's mailbox (the wait is seconds long).
/// </para>
/// <para>
/// It completes normally when every request is confirmed, or never needed confirming (the upstream
/// reported it applied). It throws — normally a <see cref="DriverOperationRefusedException"/> naming why —
/// when an acknowledge is not confirmed within the driver's own bounded window; the runtime then reverts
/// the condition to unacknowledged exactly as it does for an <see cref="IAlarmSource.AcknowledgeAsync"/>
/// failure (lmxopcua#760). The window is the driver's to own and bound.
/// </para>
/// </remarks>
public interface IAlarmAcknowledgeConfirmation
{
/// <summary>Waits for the upstream to confirm the acknowledges <see cref="IAlarmSource.AcknowledgeAsync"/> sent.</summary>
/// <param name="acknowledgements">The request instances previously passed to <see cref="IAlarmSource.AcknowledgeAsync"/>.</param>
/// <param name="cancellationToken">Abandons the wait (the caller then treats the acknowledge as unconfirmed).</param>
/// <returns>A task that completes when every acknowledge is confirmed or needed no confirmation.</returns>
Task ConfirmAcknowledgedAsync(
IReadOnlyList<AlarmAcknowledgeRequest> acknowledgements,
CancellationToken cancellationToken);
}
@@ -79,6 +79,28 @@ public sealed record GalaxyAlarmAckOptions(
string? FallbackDomain = null,
string? FallbackPasswordSecretRef = null)
{
/// <summary>Default <see cref="ConfirmTimeoutMs"/>.</summary>
public const int DefaultConfirmTimeoutMs = 10_000;
/// <summary>Upper bound for <see cref="ConfirmTimeoutMs"/>.</summary>
public const int MaxConfirmTimeoutMs = 120_000;
/// <summary>
/// How long an acknowledge the gateway SENT but could not confirm may wait for Galaxy's own acknowledged
/// transition on the alarm feed before it is treated as failed and the OPC UA condition is reverted to
/// unacknowledged (lmxopcua#869). Default 10 000 ms; <c>0</c> disables confirmation (an unconfirmed ack
/// counts as success, with a Warning — the pre-#869 behaviour); a negative value means the default, and
/// anything above 120 000 is clamped to it.
/// </summary>
[Display(Name = "Ack confirm timeout (ms)", Description = "Wait for Galaxy to confirm an unconfirmed acknowledge. 0 = do not wait.", GroupName = "Alarms")]
[Range(0, MaxConfirmTimeoutMs)]
public int ConfirmTimeoutMs { get; init; } = DefaultConfirmTimeoutMs;
/// <summary>The confirmation window: negative ⇒ the default, else clamped to <see cref="MaxConfirmTimeoutMs"/>.</summary>
[System.Text.Json.Serialization.JsonIgnore]
public TimeSpan ConfirmTimeout => TimeSpan.FromMilliseconds(
ConfirmTimeoutMs < 0 ? DefaultConfirmTimeoutMs : Math.Min(ConfirmTimeoutMs, MaxConfirmTimeoutMs));
/// <summary>True when a fallback account is configured.</summary>
[System.Text.Json.Serialization.JsonIgnore]
public bool HasFallback => !string.IsNullOrWhiteSpace(FallbackUser);
@@ -38,7 +38,7 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Galaxy;
/// </para>
/// </remarks>
public sealed class GalaxyDriver
: IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IRediscoverable, IHostConnectivityProbe, IAlarmSource, IAlarmAcknowledgePreflight, ISkippedTagReporter, IDriverDiagnostics, IDisposable, IAsyncDisposable
: IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IRediscoverable, IHostConnectivityProbe, IAlarmSource, IAlarmAcknowledgePreflight, IAlarmAcknowledgeConfirmation, ISkippedTagReporter, IDriverDiagnostics, IDisposable, IAsyncDisposable
{
// ⚠️ lmxopcua#815 warn-only compatibility (deprecated): this RUNNING driver keeps resolving a pre-#815
// out-of-namespace credential reference (API key on InitializeAsync AND every gateway ReopenAsync, plus the
@@ -138,6 +138,15 @@ public sealed class GalaxyDriver
// lmxopcua#765: the cached key → has-Galaxy-identity set the synchronous ack pre-check reads (no I/O).
// Null = no directory wired, so a keyed session is never pre-refused (the pre-#765 behaviour).
private readonly IGalaxyAckIdentityDirectory? _ackIdentityDirectory;
// lmxopcua#869: confirms acks the gateway SENT but could not confirm, against Galaxy's own acknowledged
// transition on the alarm feed (bounded by AlarmAck.ConfirmTimeoutMs). Built in the ctor.
private readonly GalaxyAckConfirmationTracker _ackConfirmations;
/// <summary>The ack confirmation window in force (lmxopcua#869) — tests.</summary>
internal TimeSpan AckConfirmationWindow => _ackConfirmations.Window;
/// <summary>Acks still awaiting (or exempt from) confirmation (lmxopcua#869) — tests.</summary>
internal int PendingAckConfirmations => _ackConfirmations.PendingCount;
private IGalaxyAlarmFeed? _alarmFeed;
private readonly Lock _alarmHandlersLock = new();
private readonly Lock _alarmFeedLock = new();
@@ -377,6 +386,8 @@ public sealed class GalaxyDriver
_alarmFeed = alarmFeed;
_ackCredentialResolver = ackCredentialResolver
?? new GalaxyAckCredentialResolver(_driverInstanceId, _options.AlarmAck, _secretResolver, _logger, legacy: _legacyCredentials);
_ackConfirmations = new GalaxyAckConfirmationTracker(
_options.AlarmAck?.ConfirmTimeout ?? GalaxyAckConfirmationTracker.DefaultWindow, _logger, _driverInstanceId);
// Build the v3 RawPath ⇄ attributeRef tables from the authored raw tags. Empty when no tags
// are authored (live-browse path), leaving every translation an identity no-op.
@@ -2340,11 +2351,40 @@ public sealed class GalaxyDriver
_driverInstanceId, alarmFullReference, credential.DisplayName, credential.Source,
PrincipalOf(ack), ack.ApiKeyId ?? "none");
await _alarmAcknowledger.AcknowledgeAsync(
alarmFullReference, comment, credential, cancellationToken).ConfigureAwait(false);
// lmxopcua#869: the gateway's OK can mean "sent but unconfirmed". Watch for Galaxy's own acknowledged
// transition from BEFORE the RPC — the feed can deliver it ahead of the unary reply — and arm the
// watch when the reply is unconfirmed; ConfirmAcknowledgedAsync then awaits it, bounded.
var watch = _ackConfirmations.Begin(ack, alarmFullReference);
var armed = false;
try
{
var outcome = await _alarmAcknowledger.AcknowledgeAsync(
alarmFullReference, comment, credential, cancellationToken).ConfigureAwait(false);
if (!outcome.IsApplied)
{
_ackConfirmations.Arm(watch, outcome.Diagnostic);
armed = true;
}
}
finally
{
if (!armed) _ackConfirmations.Release(watch);
}
}
}
/// <inheritdoc />
/// <remarks>
/// lmxopcua#869: completes at once for an ack the gateway reported applied; for one it only SENT, waits up to
/// <c>AlarmAck.ConfirmTimeoutMs</c> (default 10 s) for Galaxy's acknowledged transition on the alarm feed and
/// throws <see cref="DriverOperationRefusedException"/> when it does not come — the host then reverts the
/// condition to unacknowledged. The feed must be running (it is whenever the host holds native conditions,
/// the only way an ack reaches this driver); without it nothing can confirm and the ack reverts.
/// </remarks>
public Task ConfirmAcknowledgedAsync(
IReadOnlyList<AlarmAcknowledgeRequest> acknowledgements, CancellationToken cancellationToken) =>
_ackConfirmations.ConfirmAsync(acknowledgements, cancellationToken);
/// <inheritdoc />
/// <remarks>
/// Only cases decidable without I/O, on a driver with no <c>AlarmAck.FallbackUser</c>: an anonymous session
@@ -2404,6 +2444,10 @@ public sealed class GalaxyDriver
/// </summary>
private void OnAlarmFeedTransition(object? sender, GalaxyAlarmTransition transition)
{
// lmxopcua#869: an acknowledged transition confirms any ack the gateway sent unconfirmed — before the
// subscription gate below, which only governs fan-out to OnAlarmEvent.
_ackConfirmations.Observe(transition);
GalaxyAlarmSubscriptionHandle? handle;
int subCount;
lock (_alarmHandlersLock)
@@ -114,7 +114,10 @@ public static class GalaxyDriverFactoryExtensions
AlarmAck = new GalaxyAlarmAckOptions(
FallbackUser: NullIfBlank(dto.AlarmAck?.FallbackUser),
FallbackDomain: NullIfBlank(dto.AlarmAck?.FallbackDomain),
FallbackPasswordSecretRef: NullIfBlank(dto.AlarmAck?.FallbackPasswordSecretRef)),
FallbackPasswordSecretRef: NullIfBlank(dto.AlarmAck?.FallbackPasswordSecretRef))
{
ConfirmTimeoutMs = dto.AlarmAck?.ConfirmTimeoutMs ?? GalaxyAlarmAckOptions.DefaultConfirmTimeoutMs,
},
};
return new GalaxyDriver(
@@ -160,6 +163,8 @@ public static class GalaxyDriverFactoryExtensions
public string? FallbackDomain { get; init; }
/// <summary>Gets or sets the fallback password reference (secret:/env:/file:/dev:).</summary>
public string? FallbackPasswordSecretRef { get; init; }
/// <summary>Gets or sets how long an unconfirmed ack waits for Galaxy to confirm it (lmxopcua#869); null = default.</summary>
public int? ConfirmTimeoutMs { get; init; }
}
/// <summary>Gateway configuration section.</summary>
@@ -0,0 +1,249 @@
using System.Diagnostics;
using Microsoft.Extensions.Logging;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
namespace ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Runtime;
/// <summary>
/// Confirms Galaxy acknowledges the gateway sent but could not confirm (lmxopcua#869) against Galaxy's own
/// acknowledged transition on the session-less alarm feed.
/// </summary>
/// <remarks>
/// <para>
/// Lifecycle of one acknowledge: <see cref="Begin"/> registers a watch BEFORE the gateway RPC is sent, so a
/// confirming transition that overtakes the RPC reply is not lost (the feed and the unary reply race). The
/// RPC outcome then either releases the watch (<see cref="Release"/>: applied, or the RPC failed) or arms it
/// (<see cref="Arm"/>: sent but unconfirmed — the confirmation window starts). <see cref="ConfirmAsync"/>
/// awaits the armed watches of the same request instances, outside the resilience pipeline and off the
/// driver actor's mailbox (see <see cref="IAlarmAcknowledgeConfirmation"/>), and throws a
/// <see cref="DriverOperationRefusedException"/> when the window passes without confirmation — the host
/// then reverts the condition to unacknowledged.
/// </para>
/// <para>
/// A confirmation is an <see cref="GalaxyAlarmTransitionKind.Acknowledge"/> or
/// <see cref="GalaxyAlarmTransitionKind.ActiveAcknowledged"/> transition (live or snapshot re-statement) for
/// the same alarm reference — whoever acknowledged it, Galaxy now shows it acknowledged, which is the
/// truth the OPC UA condition must match. Galaxy references are case-insensitive, so the match is too.
/// </para>
/// <para>
/// A watch whose <see cref="ConfirmAsync"/> never comes (an <see cref="IAlarmSource.AcknowledgeAsync"/>
/// caller that does not confirm) is pruned once it is well past its window, so the set stays bounded.
/// A zero window disables confirmation: an unconfirmed ack is counted as success with a Warning — the
/// pre-#869 behaviour, kept only as an operator escape hatch.
/// </para>
/// </remarks>
internal sealed class GalaxyAckConfirmationTracker
{
/// <summary>Default confirmation window — long enough for a gateway alarm-feed round trip, short enough
/// that an operator sees the revert while still looking at the alarm.</summary>
internal static readonly TimeSpan DefaultWindow = TimeSpan.FromSeconds(10);
/// <summary>How long past its window an armed watch nobody confirms is kept before it is pruned.</summary>
private static readonly TimeSpan AbandonedGrace = TimeSpan.FromMinutes(1);
private readonly TimeSpan _window;
private readonly ILogger _logger;
private readonly string _driverInstanceId;
private readonly Lock _lock = new();
private readonly List<Watch> _watches = [];
/// <summary>Initializes a new instance of the <see cref="GalaxyAckConfirmationTracker"/> class.</summary>
/// <param name="window">How long an unconfirmed ack may wait for Galaxy's transition; zero disables confirmation.</param>
/// <param name="logger">Logger for confirmation outcomes.</param>
/// <param name="driverInstanceId">The owning driver, for log context.</param>
public GalaxyAckConfirmationTracker(TimeSpan window, ILogger logger, string driverInstanceId)
{
if (window < TimeSpan.Zero) throw new ArgumentOutOfRangeException(nameof(window), window, "must not be negative");
_window = window;
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_driverInstanceId = driverInstanceId ?? string.Empty;
}
/// <summary>The confirmation window (zero = disabled).</summary>
public TimeSpan Window => _window;
/// <summary>Number of live watches (tests / diagnostics).</summary>
internal int PendingCount
{
get { lock (_lock) return _watches.Count; }
}
/// <summary>Registers a watch for <paramref name="alarmFullReference"/> before its acknowledge is sent.</summary>
/// <param name="request">The request instance the caller will later pass to <see cref="ConfirmAsync"/>.</param>
/// <param name="alarmFullReference">The Galaxy alarm reference the ack is sent for.</param>
/// <returns>The watch, to be <see cref="Release"/>d or <see cref="Arm"/>ed once the RPC answers.</returns>
public Watch Begin(AlarmAcknowledgeRequest request, string alarmFullReference)
{
ArgumentNullException.ThrowIfNull(request);
ArgumentException.ThrowIfNullOrEmpty(alarmFullReference);
var watch = new Watch(request, alarmFullReference);
lock (_lock)
{
PruneAbandoned();
_watches.Add(watch);
}
return watch;
}
/// <summary>Drops a watch whose acknowledge needs no confirmation (applied) or was never delivered (failed).</summary>
/// <param name="watch">The watch.</param>
public void Release(Watch watch)
{
lock (_lock) _watches.Remove(watch);
}
/// <summary>
/// Starts the confirmation window for an acknowledge the gateway sent but could not confirm. A watch the feed
/// already confirmed while the RPC was in flight is released at once; with confirmation disabled the ack is
/// counted as success (logged).
/// </summary>
/// <param name="watch">The watch.</param>
/// <param name="diagnostic">The gateway's diagnostic, carried into the timeout report.</param>
public void Arm(Watch watch, string diagnostic)
{
watch.Diagnostic = diagnostic ?? string.Empty;
if (_window == TimeSpan.Zero)
{
Release(watch);
_logger.LogWarning(
"Galaxy driver {DriverInstanceId}: acknowledge of {AlarmRef} was not confirmed by the gateway ({Diagnostic}) "
+ "and ack confirmation is disabled (AlarmAck.ConfirmTimeoutMs = 0); counting it as acknowledged.",
_driverInstanceId, watch.AlarmFullReference, watch.Diagnostic);
return;
}
if (watch.IsConfirmed)
{
Release(watch);
LogConfirmed(watch);
return;
}
lock (_lock) watch.ArmedTimestamp = Stopwatch.GetTimestamp();
}
/// <summary>
/// Feed hook: confirms every watch for the transition's alarm when the transition says Galaxy shows it
/// acknowledged. Runs on the feed's dispatch thread — never blocks.
/// </summary>
/// <param name="transition">The alarm transition.</param>
public void Observe(GalaxyAlarmTransition transition)
{
if (transition.TransitionKind is not (GalaxyAlarmTransitionKind.Acknowledge or GalaxyAlarmTransitionKind.ActiveAcknowledged))
return;
if (string.IsNullOrEmpty(transition.AlarmFullReference)) return;
lock (_lock)
{
foreach (var watch in _watches)
{
if (string.Equals(watch.AlarmFullReference, transition.AlarmFullReference, StringComparison.OrdinalIgnoreCase))
watch.Confirm();
}
}
}
/// <summary>
/// Awaits confirmation of every armed watch belonging to <paramref name="requests"/> (matched by instance).
/// Requests with no armed watch (applied, or never sent) need nothing. Throws
/// <see cref="DriverOperationRefusedException"/> when any is not confirmed inside the window.
/// </summary>
/// <param name="requests">The request instances the acknowledge was sent with.</param>
/// <param name="cancellationToken">Abandons the wait.</param>
/// <returns>A task that completes when every armed watch is confirmed.</returns>
public async Task ConfirmAsync(IReadOnlyList<AlarmAcknowledgeRequest> requests, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(requests);
List<Watch> mine;
lock (_lock)
{
mine = _watches.Where(w => w.ArmedTimestamp != 0 && requests.Any(r => ReferenceEquals(r, w.Request))).ToList();
}
if (mine.Count == 0) return;
var failures = new List<string>();
try
{
foreach (var watch in mine)
{
var remaining = _window - Stopwatch.GetElapsedTime(watch.ArmedTimestamp);
try
{
if (!watch.IsConfirmed)
{
if (remaining <= TimeSpan.Zero) throw new TimeoutException();
await watch.Confirmed.WaitAsync(remaining, cancellationToken).ConfigureAwait(false);
}
LogConfirmed(watch);
}
catch (TimeoutException)
{
_logger.LogWarning(
"Galaxy driver {DriverInstanceId}: acknowledge of {AlarmRef} was NOT confirmed — Galaxy reported no "
+ "acknowledged transition within {Window} after the gateway sent it unconfirmed (gateway diagnostic: "
+ "{Diagnostic}). Treating it as failed (BadTimeout); the OPC UA condition is reverted to unacknowledged.",
_driverInstanceId, watch.AlarmFullReference, _window, watch.Diagnostic);
failures.Add(
$"Galaxy did not confirm the acknowledge of '{watch.AlarmFullReference}' within "
+ $"{_window.TotalSeconds:0.#} s (BadTimeout; gateway: {watch.Diagnostic})");
}
}
}
finally
{
lock (_lock)
{
foreach (var watch in mine) _watches.Remove(watch);
}
}
if (failures.Count > 0)
throw new DriverOperationRefusedException(string.Join("; ", failures));
}
private void LogConfirmed(Watch watch) =>
_logger.LogInformation(
"Galaxy driver {DriverInstanceId}: acknowledge of {AlarmRef} confirmed by Galaxy's acknowledged transition "
+ "(the gateway had reported it sent but unconfirmed: {Diagnostic}).",
_driverInstanceId, watch.AlarmFullReference, watch.Diagnostic);
// Caller holds _lock.
private void PruneAbandoned()
{
var limit = _window + AbandonedGrace;
_watches.RemoveAll(w => w.ArmedTimestamp != 0 && Stopwatch.GetElapsedTime(w.ArmedTimestamp) > limit);
}
/// <summary>One acknowledge awaiting (or exempt from) confirmation.</summary>
internal sealed class Watch
{
private readonly TaskCompletionSource _confirmed = new(TaskCreationOptions.RunContinuationsAsynchronously);
internal Watch(AlarmAcknowledgeRequest request, string alarmFullReference)
{
Request = request;
AlarmFullReference = alarmFullReference;
}
/// <summary>The request instance this watch belongs to.</summary>
public AlarmAcknowledgeRequest Request { get; }
/// <summary>The Galaxy alarm reference.</summary>
public string AlarmFullReference { get; }
/// <summary>The gateway's diagnostic for the unconfirmed send.</summary>
public string Diagnostic { get; set; } = string.Empty;
/// <summary><see cref="Stopwatch"/> timestamp the window started at; 0 while not armed.</summary>
public long ArmedTimestamp { get; set; }
/// <summary>Completes when Galaxy reports the alarm acknowledged.</summary>
public Task Confirmed => _confirmed.Task;
/// <summary>True once Galaxy reported the alarm acknowledged.</summary>
public bool IsConfirmed => _confirmed.Task.IsCompleted;
/// <summary>Marks the watch confirmed (idempotent).</summary>
public void Confirm() => _confirmed.TrySetResult();
}
}
@@ -34,6 +34,18 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Runtime;
/// gateway answered — so it neither retries nor counts toward the resilience breaker (a mis-keyed
/// account must not open the breaker for every correctly-configured caller).
/// </para>
/// <para>
/// <b>An OK reply is not proof the ack was applied (lmxopcua#869).</b> The gateway answers OK with
/// <c>hresult</c> 0 both when the <c>AckMsg</c> write completed and when it was only SENT — its worker saw no
/// <c>OnWriteComplete</c> inside its window — and in the second case it sets <c>diagnostic_message</c>
/// ("AckMsg write to '…' sent; OnWriteComplete not observed within N ms."). An applied ack carries an empty
/// diagnostic. So an OK reply with a non-empty diagnostic is returned as
/// <see cref="GalaxyAckOutcome.Unconfirmed"/> and the driver waits for Galaxy's own acknowledged transition
/// (<see cref="GalaxyAckConfirmationTracker"/>). Any unexplained diagnostic on an OK reply is treated the
/// same way — the safe direction: at worst an applied ack waits for its (arriving) transition. The H6 live
/// gate (2026-11-11) caught an unconfirmed ack being reported Acknowledged while Galaxy never applied it.
/// The diagnostic is logged on every branch that carries one.
/// </para>
/// </remarks>
internal sealed class GatewayGalaxyAlarmAcknowledger : IGalaxyAlarmAcknowledger
{
@@ -53,7 +65,7 @@ internal sealed class GatewayGalaxyAlarmAcknowledger : IGalaxyAlarmAcknowledger
}
/// <inheritdoc />
public async Task AcknowledgeAsync(
public async Task<GalaxyAckOutcome> AcknowledgeAsync(
string alarmFullReference,
string comment,
GalaxyAckCredential credential,
@@ -77,7 +89,7 @@ internal sealed class GatewayGalaxyAlarmAcknowledger : IGalaxyAlarmAcknowledger
+ "details withheld because they echoed the operator credential.");
}
EvaluateReply(reply, alarmFullReference, credential, _logger);
return EvaluateReply(reply, alarmFullReference, credential, _logger);
}
/// <summary>Maps an acknowledge onto the gateway request. Pure — pinned by unit tests.</summary>
@@ -104,23 +116,32 @@ internal sealed class GatewayGalaxyAlarmAcknowledger : IGalaxyAlarmAcknowledger
/// <summary>
/// Applies the reply semantics: non-OK protocol status throws; a non-zero native code warns and throws a
/// <see cref="DriverOperationRefusedException"/> (lmxopcua#760).
/// Every message is scrubbed of the password.
/// <see cref="DriverOperationRefusedException"/> (lmxopcua#760); an OK reply is
/// <see cref="GalaxyAckOutcome.Applied"/> when it carries no diagnostic and
/// <see cref="GalaxyAckOutcome.Unconfirmed"/> when it does (lmxopcua#869). The gateway's diagnostic is
/// logged on every branch that carries one. Every message is scrubbed of the password.
/// </summary>
/// <param name="reply">The gateway reply.</param>
/// <param name="alarmFullReference">The alarm reference.</param>
/// <param name="credential">The credential the request carried.</param>
/// <param name="logger">The logger.</param>
internal static void EvaluateReply(
/// <returns>Whether the gateway confirmed the ack was applied.</returns>
internal static GalaxyAckOutcome EvaluateReply(
AcknowledgeAlarmReply reply, string alarmFullReference, GalaxyAckCredential credential, ILogger logger)
{
var diagnostic = Scrub(reply.DiagnosticMessage, credential.Password).Trim();
// Protocol status — the gateway failed before MXAccess saw the ack. This is a
// hard failure: the operator's request was not delivered at all.
if (reply.ProtocolStatus is { } proto && proto.Code != ProtocolStatusCode.Ok)
{
var protoMessage = Scrub(proto.Message, credential.Password);
logger.LogWarning(
"Galaxy AcknowledgeAlarm for {AlarmRef} as {GalaxyUser} failed at the gateway: {ProtocolCode} {ProtocolMessage} (gateway diagnostic: {Diagnostic})",
alarmFullReference, credential.DisplayName, proto.Code, protoMessage, DiagnosticText(diagnostic));
throw new InvalidOperationException(
$"Galaxy AcknowledgeAlarm for '{alarmFullReference}' as '{credential.DisplayName}' failed at the gateway: "
+ $"{proto.Code} {Scrub(proto.Message, credential.Password)}");
+ $"{proto.Code} {protoMessage}");
}
// hresult is the authoritative native ack return code (0 = success) — the AuthenticateUser HRESULT
@@ -129,14 +150,29 @@ internal sealed class GatewayGalaxyAlarmAcknowledger : IGalaxyAlarmAcknowledger
if (reply.HasHresult && reply.Hresult != 0)
{
logger.LogWarning(
"Galaxy AcknowledgeAlarm for {AlarmRef} as {GalaxyUser} returned native ack failure code {Hresult}.",
alarmFullReference, credential.DisplayName, reply.Hresult);
"Galaxy AcknowledgeAlarm for {AlarmRef} as {GalaxyUser} returned native ack failure code {Hresult} (gateway diagnostic: {Diagnostic}).",
alarmFullReference, credential.DisplayName, reply.Hresult, DiagnosticText(diagnostic));
throw new DriverOperationRefusedException(
$"MXAccess rejected the Galaxy acknowledge of '{alarmFullReference}' as '{credential.DisplayName}' "
+ $"(native code 0x{reply.Hresult:X8}); check the account's password and Galaxy acknowledge permission.");
}
// lmxopcua#869: OK + hresult 0 means "applied OR sent-but-unconfirmed"; only the unconfirmed case carries a
// diagnostic. Not a failure yet — the driver waits for Galaxy's own acknowledged transition.
if (diagnostic.Length > 0)
{
logger.LogInformation(
"Galaxy AcknowledgeAlarm for {AlarmRef} as {GalaxyUser} was sent but not confirmed by the gateway "
+ "(gateway diagnostic: {Diagnostic}); awaiting Galaxy's acknowledged transition.",
alarmFullReference, credential.DisplayName, diagnostic);
return GalaxyAckOutcome.Unconfirmed(diagnostic);
}
return GalaxyAckOutcome.Applied;
}
private static string DiagnosticText(string diagnostic) => diagnostic.Length > 0 ? diagnostic : "none";
/// <summary>Replaces every occurrence of <paramref name="secret"/> in <paramref name="text"/>.</summary>
/// <param name="text">Text that may echo the secret.</param>
/// <param name="secret">The secret; empty means nothing to scrub.</param>
@@ -22,10 +22,32 @@ internal interface IGalaxyAlarmAcknowledger
/// Carries a password — implementations must keep it out of logs and exception text.
/// </param>
/// <param name="cancellationToken">Cancels the gateway RPC.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
Task AcknowledgeAsync(
/// <returns>
/// Whether the gateway reported the ack APPLIED or only SENT without confirmation (lmxopcua#869). A refusal
/// or failure throws instead.
/// </returns>
Task<GalaxyAckOutcome> AcknowledgeAsync(
string alarmFullReference,
string comment,
GalaxyAckCredential credential,
CancellationToken cancellationToken);
}
/// <summary>
/// What the gateway said about an acknowledge it accepted (lmxopcua#869). mxaccessgw answers OK both when the
/// <c>AckMsg</c> write's <c>OnWriteComplete</c> reported success (<see cref="Applied"/>) and when the write was
/// sent but no completion arrived inside the worker's window (<see cref="Unconfirmed"/>) — its contract tells
/// clients to watch the alarm stream for the real acknowledged transition in that case.
/// </summary>
/// <param name="IsApplied">True when the gateway confirmed the ack write was applied.</param>
/// <param name="Diagnostic">The gateway's diagnostic message (password-scrubbed); empty when it sent none.</param>
internal sealed record GalaxyAckOutcome(bool IsApplied, string Diagnostic)
{
/// <summary>The gateway confirmed the ack write.</summary>
public static GalaxyAckOutcome Applied { get; } = new(true, string.Empty);
/// <summary>The gateway sent the ack write but could not confirm it.</summary>
/// <param name="diagnostic">The gateway's diagnostic message.</param>
/// <returns>The outcome.</returns>
public static GalaxyAckOutcome Unconfirmed(string diagnostic) => new(false, diagnostic ?? string.Empty);
}
@@ -109,6 +109,11 @@
placeholder="secret:otopcua/galaxy/ack-fallback-password" />
<div class="form-text">Forms: <code>secret:galaxy/NAME</code> or <code>secret:otopcua/galaxy/NAME</code> (production), <code>env:OTOPCUA_GALAXY_NAME</code>, <code>dev:VALUE</code>. A bare literal, <code>file:</code>, or a reference outside the Galaxy namespace is refused (#815).</div>
</div>
<div class="col-md-3">
<label class="form-label">Ack confirm timeout (ms)</label>
<InputNumber @bind-Value="_form.Galaxy.AckConfirmTimeoutMs" @bind-Value:after="EmitAsync" class="form-control form-control-sm" />
<div class="form-text">When the gateway sends an ack but cannot confirm it, wait this long for Galaxy's acknowledged transition before reverting the condition to unacknowledged (#869). Default 10000; 0 = do not wait.</div>
</div>
</div>
</div>
</section>
@@ -270,6 +275,7 @@
public string? AckFallbackUser { get; set; }
public string? AckFallbackDomain { get; set; }
public string? AckFallbackPasswordSecretRef { get; set; }
public int AckConfirmTimeoutMs { get; set; } = GalaxyAlarmAckOptions.DefaultConfirmTimeoutMs;
public static GalaxyFormModel FromRecord(GalaxyDriverOptions r)
{
@@ -300,6 +306,7 @@
AckFallbackUser = ack.FallbackUser,
AckFallbackDomain = ack.FallbackDomain,
AckFallbackPasswordSecretRef = ack.FallbackPasswordSecretRef,
AckConfirmTimeoutMs = ack.ConfirmTimeoutMs,
};
}
@@ -329,7 +336,10 @@
AlarmAck = new GalaxyAlarmAckOptions(
FallbackUser: string.IsNullOrWhiteSpace(AckFallbackUser) ? null : AckFallbackUser.Trim(),
FallbackDomain: string.IsNullOrWhiteSpace(AckFallbackDomain) ? null : AckFallbackDomain.Trim(),
FallbackPasswordSecretRef: string.IsNullOrWhiteSpace(AckFallbackPasswordSecretRef) ? null : AckFallbackPasswordSecretRef.Trim()),
FallbackPasswordSecretRef: string.IsNullOrWhiteSpace(AckFallbackPasswordSecretRef) ? null : AckFallbackPasswordSecretRef.Trim())
{
ConfirmTimeoutMs = AckConfirmTimeoutMs,
},
};
}
}
@@ -365,6 +365,10 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
/// shutdown rather than let it race the teardown on a restart.</summary>
private CancellationTokenSource? _initCts;
/// <summary>Cancelled by <see cref="PostStop"/> to abandon in-flight native-ack confirmations (lmxopcua#869);
/// never disposed — detached confirmation tasks may still hold its token.</summary>
private readonly CancellationTokenSource _ackConfirmCts = new();
/// <summary>
/// Timestamps of recent Faulted-state transitions; used to compute the 5-minute error count.
/// No lock needed — every read/write site runs inside an Akka message handler, which is
@@ -1367,6 +1371,13 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
cts.Token);
_log.Info("DriverInstance {Id}: acknowledged native condition {Cond} by {User}",
_driverInstanceId, msg.ConditionId, msg.OperatorUser);
// lmxopcua#869: an upstream may accept an ack without confirming it was applied (Galaxy: the gateway's
// "sent but unconfirmed" OK). The confirmation wait is seconds long, so it runs OFF this mailbox and
// OUTSIDE the resilience pipeline (a wait on an upstream event — it must neither be cut by the
// AlarmAcknowledge capability timeout nor count toward the breaker). Failure reverts like any other.
if (src is IAlarmAcknowledgeConfirmation confirmer)
_ = RunAckConfirmationAsync(confirmer, request, msg, replyTo, _ackConfirmCts.Token, _driverInstanceId, _log);
}
catch (DriverOperationRefusedException ex)
{
@@ -1385,6 +1396,48 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
}
}
/// <summary>
/// Backstop over a driver's own confirmation window (lmxopcua#869): the driver bounds the wait itself; this
/// only guarantees a driver that ignores its window cannot hold a confirmation forever.
/// </summary>
internal static readonly TimeSpan AckConfirmationCeiling = TimeSpan.FromMinutes(3);
/// <summary>
/// Awaits <see cref="IAlarmAcknowledgeConfirmation.ConfirmAcknowledgedAsync"/> off the actor context and, when
/// the upstream does not confirm, reports <see cref="AlarmAckFailed"/> so the host reverts the condition to
/// unacknowledged — the same report an <see cref="IAlarmSource.AcknowledgeAsync"/> failure makes. Reads no actor
/// state: everything it needs is captured by the caller on the actor thread.
/// </summary>
private static async Task RunAckConfirmationAsync(
IAlarmAcknowledgeConfirmation confirmer, IReadOnlyList<AlarmAcknowledgeRequest> request, RouteAlarmAck msg,
IActorRef replyTo, CancellationToken stopToken, string driverInstanceId, ILoggingAdapter log)
{
await Task.Yield();
try
{
await confirmer.ConfirmAcknowledgedAsync(request, stopToken).WaitAsync(AckConfirmationCeiling, stopToken);
}
catch (DriverOperationRefusedException ex)
{
// The driver's verdict (e.g. Galaxy: no acknowledged transition inside its window) — it names why.
log.Warning("DriverInstance {Id}: native-alarm acknowledge of {Cond} by {User} was not confirmed upstream — {Reason}",
driverInstanceId, msg.ConditionId, msg.OperatorUser, ex.Message);
ReportAckFailed(replyTo, msg, ex.Message);
}
catch (OperationCanceledException) when (stopToken.IsCancellationRequested)
{
log.Info("DriverInstance {Id}: confirmation of the native-alarm acknowledge of {Cond} abandoned — the driver stopped",
driverInstanceId, msg.ConditionId);
ReportAckFailed(replyTo, msg, "the driver stopped before the upstream confirmed the acknowledge");
}
catch (Exception ex)
{
log.Warning(ex, "DriverInstance {Id}: confirmation of the native-alarm acknowledge of {Cond} failed",
driverInstanceId, msg.ConditionId);
ReportAckFailed(replyTo, msg, $"the upstream acknowledge was not confirmed ({ex.GetType().Name})");
}
}
/// <summary>Tells <paramref name="replyTo"/> an <see cref="AlarmAckFailed"/> when the ack names a condition
/// NodeId (lmxopcua#760); a no-op for an ack routed without one.</summary>
private static void ReportAckFailed(IActorRef replyTo, RouteAlarmAck msg, string reason)
@@ -2004,6 +2057,10 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
try { _initCts?.Cancel(); }
catch (ObjectDisposedException) { /* the attempt already finished */ }
// In-flight native-ack confirmations (lmxopcua#869) are abandoned, and each reports its ack unconfirmed so
// the host reverts it — this incarnation can no longer observe the upstream's confirmation.
_ackConfirmCts.Cancel();
// BOUNDED by ONE shared budget across shutdown + dispose. PostStop runs on a dispatcher thread and this
// blocks it, so an unbounded wait on a driver whose shutdown hangs against a dead socket / COM proxy pinned
// that thread for the whole wait, and a few such stops starved the dispatcher. A shutdown that overruns is
@@ -317,10 +317,10 @@ public sealed class GalaxyDriverAlarmSourceTests
/// <param name="credential">The resolved Galaxy credential.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>A completed task.</returns>
public Task AcknowledgeAsync(string alarmFullReference, string comment, GalaxyAckCredential credential, CancellationToken cancellationToken)
public Task<GalaxyAckOutcome> AcknowledgeAsync(string alarmFullReference, string comment, GalaxyAckCredential credential, CancellationToken cancellationToken)
{
Calls.Add((alarmFullReference, comment, credential.User));
return Task.CompletedTask;
return Task.FromResult(GalaxyAckOutcome.Applied);
}
}
@@ -0,0 +1,381 @@
using Microsoft.Extensions.Logging;
using Shouldly;
using Xunit;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Config;
using ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Runtime;
namespace ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Tests.Runtime;
/// <summary>
/// lmxopcua#869 — mxaccessgw answers <c>AcknowledgeAlarm</c> OK both when the <c>AckMsg</c> write was APPLIED and
/// when it was only SENT (no <c>OnWriteComplete</c> inside the worker's window; the reply then carries a
/// diagnostic). The H6 live gate saw an unconfirmed ack reported Acknowledged while Galaxy never applied it.
/// The driver now treats an unconfirmed OK as pending and confirms it against Galaxy's own acknowledged
/// transition on the alarm feed, bounded by <c>AlarmAck.ConfirmTimeoutMs</c>.
/// </summary>
/// <remarks>
/// The "fake gateway" is a scripted <see cref="AcknowledgeAlarmReply"/> run through the production
/// <see cref="GatewayGalaxyAlarmAcknowledger.EvaluateReply"/> — the gRPC transport seam is internal to the
/// client package, so the reply mapping is the part under test.
/// </remarks>
public sealed class GalaxyAckConfirmationTests
{
private const string AlarmRef = "Tank01.Level.HiHi";
private const string UnconfirmedDiagnostic =
"AckMsg write to 'Tank01.Level.HiHi.AckMsg' sent; OnWriteComplete not observed within 5000 ms.";
private static readonly GalaxyAckCredential Alice =
new("alice", "CORP", "alice-pw", GalaxyAckCredentialSource.ApiKey);
// ------------------------------------------------------------------ reply mapping
[Fact]
public void EvaluateReply_ok_without_diagnostic_is_applied()
{
var logger = new CapturingLogger();
var outcome = GatewayGalaxyAlarmAcknowledger.EvaluateReply(AppliedReply(), AlarmRef, Alice, logger);
outcome.IsApplied.ShouldBeTrue();
logger.Entries.ShouldBeEmpty();
}
/// <summary>The H6 reply: OK + hresult 0 + a "sent; not observed" diagnostic is UNCONFIRMED, and the
/// diagnostic is logged. Before #869 the same reply returned silently as success.</summary>
[Fact]
public void EvaluateReply_ok_with_diagnostic_is_unconfirmed_and_logs_the_diagnostic()
{
var logger = new CapturingLogger();
var outcome = GatewayGalaxyAlarmAcknowledger.EvaluateReply(UnconfirmedReply(), AlarmRef, Alice, logger);
outcome.IsApplied.ShouldBeFalse();
outcome.Diagnostic.ShouldBe(UnconfirmedDiagnostic);
logger.AllText.ShouldContain(UnconfirmedDiagnostic);
}
[Fact]
public void EvaluateReply_native_refusal_logs_the_diagnostic_and_refuses()
{
var reply = new AcknowledgeAlarmReply
{
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
Hresult = 1008,
DiagnosticMessage = "AckMsg write to 'Tank01.Level.HiHi.AckMsg' rejected: category SecurityError, detail 1008.",
};
var logger = new CapturingLogger();
Should.Throw<DriverOperationRefusedException>(
() => GatewayGalaxyAlarmAcknowledger.EvaluateReply(reply, AlarmRef, Alice, logger));
logger.AllText.ShouldContain("rejected: category SecurityError");
}
[Fact]
public void EvaluateReply_scrubs_the_password_from_the_diagnostic()
{
var reply = new AcknowledgeAlarmReply
{
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
Hresult = 0,
DiagnosticMessage = "sent as alice/alice-pw; not observed",
};
var logger = new CapturingLogger();
var outcome = GatewayGalaxyAlarmAcknowledger.EvaluateReply(reply, AlarmRef, Alice, logger);
outcome.Diagnostic.ShouldNotContain("alice-pw");
logger.AllText.ShouldNotContain("alice-pw");
}
// ------------------------------------------------------------------ driver: confirmation
/// <summary>Applied OK → acknowledged: nothing pending, the confirmation completes at once.</summary>
[Fact]
public async Task Applied_ack_is_confirmed_at_once()
{
var feed = new FakeAlarmFeed();
var gateway = new ScriptedGateway(AppliedReply());
using var driver = NewDriver(feed, gateway, confirmTimeoutMs: 30_000);
await driver.SubscribeAlarmsAsync([AlarmRef], CancellationToken.None);
var request = Ack(AlarmRef);
await driver.AcknowledgeAsync(request, CancellationToken.None);
var confirm = driver.ConfirmAcknowledgedAsync(request, CancellationToken.None);
confirm.IsCompletedSuccessfully.ShouldBeTrue("an applied ack needs no confirmation");
gateway.Calls.ShouldBe(1);
}
/// <summary>
/// Unconfirmed OK + Galaxy's acknowledged transition on the feed → acknowledged. Positive control: the
/// confirmation is still pending after transitions that must NOT confirm (another alarm's ack, this
/// alarm's raise) — it only completes on the matching one.
/// </summary>
[Fact]
public async Task Unconfirmed_ack_is_confirmed_by_the_acknowledged_transition()
{
var feed = new FakeAlarmFeed();
var gateway = new ScriptedGateway(UnconfirmedReply());
var logger = new CapturingLogger();
using var driver = NewDriver(feed, gateway, confirmTimeoutMs: 120_000, logger);
await driver.SubscribeAlarmsAsync([AlarmRef], CancellationToken.None);
var request = Ack(AlarmRef);
await driver.AcknowledgeAsync(request, CancellationToken.None);
var confirm = driver.ConfirmAcknowledgedAsync(request, CancellationToken.None);
feed.Emit(Transition("Other01.Level.Hi", GalaxyAlarmTransitionKind.Acknowledge));
feed.Emit(Transition(AlarmRef, GalaxyAlarmTransitionKind.Raise));
feed.Emit(Transition(AlarmRef.ToUpperInvariant(), GalaxyAlarmTransitionKind.Acknowledge));
await TestWait.ForAsync(confirm, "the acknowledged transition must confirm the pending ack");
logger.AllText.ShouldContain(UnconfirmedDiagnostic);
}
/// <summary>
/// The feed and the unary reply race: Galaxy's acknowledged transition can arrive BEFORE the RPC answers
/// "unconfirmed". The watch is registered before the RPC, so that early transition still confirms — with a
/// 1 ms window, a lost transition would time out instead.
/// </summary>
[Fact]
public async Task Acknowledged_transition_that_overtakes_the_reply_still_confirms()
{
var feed = new FakeAlarmFeed();
var gateway = new ScriptedGateway(UnconfirmedReply())
{
DuringCall = () => feed.Emit(Transition(AlarmRef, GalaxyAlarmTransitionKind.Acknowledge)),
};
using var driver = NewDriver(feed, gateway, confirmTimeoutMs: 1);
await driver.SubscribeAlarmsAsync([AlarmRef], CancellationToken.None);
var request = Ack(AlarmRef);
await driver.AcknowledgeAsync(request, CancellationToken.None);
await driver.ConfirmAcknowledgedAsync(request, CancellationToken.None);
}
/// <summary>
/// Unconfirmed OK + no acknowledged transition → the confirmation fails with a refusal naming the timeout and
/// the gateway diagnostic (the runtime reverts the condition on it), and a Warning carrying the diagnostic is
/// logged. Transitions that must not confirm (another alarm's ack, this alarm's retrigger) are emitted during
/// the wait. Before #869 this ack counted as acknowledged.
/// </summary>
[Fact]
public async Task Unconfirmed_ack_without_confirmation_fails_with_the_diagnostic()
{
var feed = new FakeAlarmFeed();
var gateway = new ScriptedGateway(UnconfirmedReply());
var logger = new CapturingLogger();
using var driver = NewDriver(feed, gateway, confirmTimeoutMs: 300, logger);
await driver.SubscribeAlarmsAsync([AlarmRef], CancellationToken.None);
var request = Ack(AlarmRef);
await driver.AcknowledgeAsync(request, CancellationToken.None);
var confirm = driver.ConfirmAcknowledgedAsync(request, CancellationToken.None);
feed.Emit(Transition("Other01.Level.Hi", GalaxyAlarmTransitionKind.Acknowledge));
feed.Emit(Transition(AlarmRef, GalaxyAlarmTransitionKind.Retrigger));
var ex = await Should.ThrowAsync<DriverOperationRefusedException>(() => confirm);
ex.Message.ShouldContain(AlarmRef);
ex.Message.ShouldContain("BadTimeout");
ex.Message.ShouldContain(UnconfirmedDiagnostic);
logger.Entries.ShouldContain(e => e.Level == LogLevel.Warning && e.Text.Contains(UnconfirmedDiagnostic));
}
/// <summary>A refused ack (non-zero native code) throws from AcknowledgeAsync — the existing revert path —
/// and leaves nothing pending to confirm.</summary>
[Fact]
public async Task Refused_ack_throws_and_leaves_nothing_pending()
{
var feed = new FakeAlarmFeed();
var gateway = new ScriptedGateway(new AcknowledgeAlarmReply
{
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
Hresult = unchecked((int)0x80070005),
});
using var driver = NewDriver(feed, gateway, confirmTimeoutMs: 30_000);
await driver.SubscribeAlarmsAsync([AlarmRef], CancellationToken.None);
var request = Ack(AlarmRef);
await Should.ThrowAsync<DriverOperationRefusedException>(
() => driver.AcknowledgeAsync(request, CancellationToken.None));
driver.ConfirmAcknowledgedAsync(request, CancellationToken.None).IsCompletedSuccessfully.ShouldBeTrue();
driver.PendingAckConfirmations.ShouldBe(0);
}
/// <summary><c>ConfirmTimeoutMs = 0</c> is the opt-out: an unconfirmed ack counts as acknowledged, with a Warning.</summary>
[Fact]
public async Task Zero_timeout_counts_an_unconfirmed_ack_as_acknowledged_with_a_warning()
{
var feed = new FakeAlarmFeed();
var gateway = new ScriptedGateway(UnconfirmedReply());
var logger = new CapturingLogger();
using var driver = NewDriver(feed, gateway, confirmTimeoutMs: 0, logger);
var request = Ack(AlarmRef);
await driver.AcknowledgeAsync(request, CancellationToken.None);
driver.ConfirmAcknowledgedAsync(request, CancellationToken.None).IsCompletedSuccessfully.ShouldBeTrue();
logger.Entries.ShouldContain(e => e.Level == LogLevel.Warning && e.Text.Contains("confirmation is disabled"));
driver.PendingAckConfirmations.ShouldBe(0);
}
/// <summary>Each confirmation is for ITS request instance: confirming one leaves no watch behind, and a later
/// ack of the same alarm is confirmed independently.</summary>
[Fact]
public async Task Confirmations_release_their_watches()
{
var feed = new FakeAlarmFeed();
var gateway = new ScriptedGateway(UnconfirmedReply());
using var driver = NewDriver(feed, gateway, confirmTimeoutMs: 120_000);
await driver.SubscribeAlarmsAsync([AlarmRef], CancellationToken.None);
for (var i = 0; i < 2; i++)
{
var request = Ack(AlarmRef);
await driver.AcknowledgeAsync(request, CancellationToken.None);
driver.PendingAckConfirmations.ShouldBe(1);
var confirm = driver.ConfirmAcknowledgedAsync(request, CancellationToken.None);
feed.Emit(Transition(AlarmRef, GalaxyAlarmTransitionKind.ActiveAcknowledged));
await TestWait.ForAsync(confirm, $"ack #{i} must be confirmed");
driver.PendingAckConfirmations.ShouldBe(0);
}
}
[Fact]
public void Options_default_to_ten_seconds_and_map_negative_to_the_default()
{
new GalaxyAlarmAckOptions().ConfirmTimeout.ShouldBe(TimeSpan.FromSeconds(10));
new GalaxyAlarmAckOptions { ConfirmTimeoutMs = -5 }.ConfirmTimeout.ShouldBe(TimeSpan.FromSeconds(10));
new GalaxyAlarmAckOptions { ConfirmTimeoutMs = 0 }.ConfirmTimeout.ShouldBe(TimeSpan.Zero);
new GalaxyAlarmAckOptions { ConfirmTimeoutMs = 999_999 }.ConfirmTimeout.ShouldBe(TimeSpan.FromMinutes(2));
}
[Fact]
public void Factory_binds_ConfirmTimeoutMs_and_defaults_when_absent()
{
const string Json = """
{ "gateway": { "endpoint": "https://gw:5001", "apiKeySecretRef": "dev:k" },
"mxAccess": { "clientName": "C" }, "alarmAck": { "confirmTimeoutMs": 2500 } }
""";
using var withKnob = GalaxyDriverFactoryExtensions.CreateInstance("g", Json);
withKnob.AckConfirmationWindow.ShouldBe(TimeSpan.FromMilliseconds(2500));
using var without = GalaxyDriverFactoryExtensions.CreateInstance(
"g", """{ "gateway": { "endpoint": "https://gw:5001", "apiKeySecretRef": "dev:k" }, "mxAccess": { "clientName": "C" } }""");
without.AckConfirmationWindow.ShouldBe(TimeSpan.FromSeconds(10));
}
// ------------------------------------------------------------------ helpers
private static AcknowledgeAlarmReply AppliedReply() => new()
{
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
Hresult = 0,
};
private static AcknowledgeAlarmReply UnconfirmedReply() => new()
{
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
Hresult = 0,
DiagnosticMessage = UnconfirmedDiagnostic,
};
private static AlarmAcknowledgeRequest[] Ack(string conditionId) =>
[new AlarmAcknowledgeRequest(SourceNodeId: "Tank01", ConditionId: conditionId, Comment: "c")];
private static GalaxyDriver NewDriver(
IGalaxyAlarmFeed feed, IGalaxyAlarmAcknowledger acknowledger, int confirmTimeoutMs, CapturingLogger? logger = null)
{
var options = new GalaxyDriverOptions(
new GalaxyGatewayOptions("http://localhost:5000", "literal-api-key"),
new GalaxyMxAccessOptions("AckConfirmTest"),
new GalaxyRepositoryOptions(),
new GalaxyReconnectOptions())
{
AlarmAck = new GalaxyAlarmAckOptions { ConfirmTimeoutMs = confirmTimeoutMs },
};
return new GalaxyDriver(
driverInstanceId: "drv-ack",
options: options,
hierarchySource: null,
alarmAcknowledger: acknowledger,
alarmFeed: feed,
logger: logger is null ? null : new TypedLogger<GalaxyDriver>(logger),
ackCredentialResolver: FixedAckCredentialResolver.Fallback());
}
private static GalaxyAlarmTransition Transition(string alarmRef, GalaxyAlarmTransitionKind kind) => new(
AlarmFullReference: alarmRef,
SourceObjectReference: "Tank01",
AlarmTypeName: "AnalogLimitAlarm.HiHi",
TransitionKind: kind,
SeverityBucket: AlarmSeverity.High,
OpcUaSeverity: 700,
RawMxAccessSeverity: 500,
OriginalRaiseTimestampUtc: null,
TransitionTimestampUtc: DateTime.UtcNow,
OperatorUser: string.Empty,
OperatorComment: string.Empty,
Category: "Process",
Description: "Tank 01 high-high level");
/// <summary>
/// A fake gateway: every call answers with <paramref name="reply"/>, mapped by the production
/// <see cref="GatewayGalaxyAlarmAcknowledger.EvaluateReply"/>. <see cref="DuringCall"/> runs while the RPC is
/// "in flight" (before the reply is evaluated).
/// </summary>
private sealed class ScriptedGateway(AcknowledgeAlarmReply reply) : IGalaxyAlarmAcknowledger
{
private int _calls;
public Action? DuringCall { get; init; }
public int Calls => Volatile.Read(ref _calls);
public Task<GalaxyAckOutcome> AcknowledgeAsync(
string alarmFullReference, string comment, GalaxyAckCredential credential, CancellationToken cancellationToken)
{
Interlocked.Increment(ref _calls);
DuringCall?.Invoke();
try
{
return Task.FromResult(GatewayGalaxyAlarmAcknowledger.EvaluateReply(
reply, alarmFullReference, credential, Microsoft.Extensions.Logging.Abstractions.NullLogger.Instance));
}
catch (Exception ex)
{
return Task.FromException<GalaxyAckOutcome>(ex);
}
}
}
private sealed class FakeAlarmFeed : IGalaxyAlarmFeed
{
public event EventHandler<GalaxyAlarmTransition>? OnAlarmTransition;
public void Start() { }
public void Emit(GalaxyAlarmTransition transition) => OnAlarmTransition?.Invoke(this, transition);
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
}
private sealed class TypedLogger<T>(CapturingLogger inner) : ILogger<T>
{
private readonly Lock _gate = new();
public IDisposable? BeginScope<TState>(TState state) where TState : notnull => null;
public bool IsEnabled(LogLevel logLevel) => true;
public void Log<TState>(LogLevel l, EventId id, TState s, Exception? e, Func<TState, Exception?, string> f)
{
lock (_gate) inner.Log(l, id, s, e, f);
}
}
}
@@ -485,11 +485,11 @@ public sealed class GalaxyAckCredentialTests
{
public List<(string AlarmRef, string Comment, GalaxyAckCredential Credential)> Calls { get; } = [];
public Task AcknowledgeAsync(
public Task<GalaxyAckOutcome> AcknowledgeAsync(
string alarmFullReference, string comment, GalaxyAckCredential credential, CancellationToken cancellationToken)
{
Calls.Add((alarmFullReference, comment, credential));
return Task.CompletedTask;
return Task.FromResult(GalaxyAckOutcome.Applied);
}
}
@@ -242,10 +242,10 @@ public sealed class GalaxyDriverRawPathKeyingTests
{
public List<(string AlarmRef, string Comment, string Operator)> Calls { get; } = [];
public Task AcknowledgeAsync(string alarmFullReference, string comment, GalaxyAckCredential credential, CancellationToken cancellationToken)
public Task<GalaxyAckOutcome> AcknowledgeAsync(string alarmFullReference, string comment, GalaxyAckCredential credential, CancellationToken cancellationToken)
{
Calls.Add((alarmFullReference, comment, credential.User));
return Task.CompletedTask;
return Task.FromResult(GalaxyAckOutcome.Applied);
}
}
@@ -34,22 +34,22 @@ public sealed class GatewayGalaxyAlarmAcknowledgerTests
/// </summary>
private sealed class FakeAcknowledger : IGalaxyAlarmAcknowledger
{
private readonly Queue<Func<Task>> _outcomes = new();
private readonly Queue<Func<Task<GalaxyAckOutcome>>> _outcomes = new();
/// <summary>Calls recorded in the order received.</summary>
public List<(string AlarmRef, string Comment, string Operator)> Calls { get; } = [];
/// <summary>Enqueues an outcome for the next call (completes normally or throws).</summary>
public void Enqueue(Func<Task> outcome) => _outcomes.Enqueue(outcome);
public void Enqueue(Func<Task<GalaxyAckOutcome>> outcome) => _outcomes.Enqueue(outcome);
public Task AcknowledgeAsync(
public Task<GalaxyAckOutcome> AcknowledgeAsync(
string alarmFullReference,
string comment,
GalaxyAckCredential credential,
CancellationToken cancellationToken)
{
Calls.Add((alarmFullReference, comment, credential.User));
return _outcomes.Count > 0 ? _outcomes.Dequeue()() : Task.CompletedTask;
return _outcomes.Count > 0 ? _outcomes.Dequeue()() : Task.FromResult(GalaxyAckOutcome.Applied);
}
}
@@ -77,9 +77,9 @@ public sealed class GatewayGalaxyAlarmAcknowledgerTests
{
var fake = new FakeAcknowledger();
// First call: simulates a transient transport blip (e.g. RpcException from gRPC).
fake.Enqueue(() => Task.FromException(new IOException("synthetic transport fault")));
fake.Enqueue(() => Task.FromException<GalaxyAckOutcome>(new IOException("synthetic transport fault")));
// Second call: succeeds — proves no dead-state latch.
fake.Enqueue(() => Task.CompletedTask);
fake.Enqueue(() => Task.FromResult(GalaxyAckOutcome.Applied));
var driver = new GalaxyDriver("g", Opts(),
hierarchySource: null, alarmAcknowledger: fake,
@@ -109,8 +109,8 @@ public sealed class GatewayGalaxyAlarmAcknowledgerTests
public async Task Acknowledge_after_protocol_failure_still_issues_next_call()
{
var fake = new FakeAcknowledger();
fake.Enqueue(() => Task.FromException(new InvalidOperationException("gateway overload")));
fake.Enqueue(() => Task.CompletedTask);
fake.Enqueue(() => Task.FromException<GalaxyAckOutcome>(new InvalidOperationException("gateway overload")));
fake.Enqueue(() => Task.FromResult(GalaxyAckOutcome.Applied));
var driver = new GalaxyDriver("g", Opts(),
hierarchySource: null, alarmAcknowledger: fake,
@@ -259,4 +259,30 @@ public sealed class GalaxyDriverPageFormSerializationTests
json.ShouldNotContain("hasFallback");
JsonSerializer.Deserialize<GalaxyDriverOptions>(json, _pageOpts)!.AlarmAck.ShouldBe(original.AlarmAck);
}
/// <summary>
/// lmxopcua#869: the ack-confirmation window must survive FromRecord/ToRecord and JSON, or saving the page
/// would silently reset an operator's setting (e.g. the 0 = do-not-wait opt-out) to the default.
/// </summary>
[Fact]
public void FormModel_RoundTrips_AlarmAckConfirmTimeout()
{
var original = new GalaxyDriverOptions(
new GalaxyGatewayOptions("https://gw:5001", "env:K"),
new GalaxyMxAccessOptions("OtOpcUa"),
new GalaxyRepositoryOptions(),
new GalaxyReconnectOptions())
{
AlarmAck = new GalaxyAlarmAckOptions { ConfirmTimeoutMs = 0 },
};
var back = GalaxyDriverForm.GalaxyFormModel.FromRecord(original).ToRecord();
back.AlarmAck.ConfirmTimeoutMs.ShouldBe(0);
var json = JsonSerializer.Serialize(back, _pageOpts);
json.ShouldContain("\"confirmTimeoutMs\":0");
json.ShouldNotContain("\"confirmTimeout\":");
JsonSerializer.Deserialize<GalaxyDriverOptions>(json, _pageOpts)!.AlarmAck.ConfirmTimeoutMs.ShouldBe(0);
new GalaxyDriverForm.GalaxyFormModel().AckConfirmTimeoutMs.ShouldBe(GalaxyAlarmAckOptions.DefaultConfirmTimeoutMs);
}
}
@@ -217,6 +217,9 @@ public sealed class DriverHostActorNativeAlarmAckRoutingTests : RuntimeActorTest
RaiseAndExpectUnacked(actor, publish);
actor.Tell(new DriverHostActor.RouteNativeAlarmAck(AlarmRawPath, "cmt", "alice"));
AwaitAssert(() => recorder.Acks.Count.ShouldBe(1), duration: Timeout, cancellationToken: TestContext.Current.CancellationToken);
// lmxopcua#869: the driver is an IAlarmAcknowledgeConfirmation whose confirmation succeeds at once — it was
// asked, and a confirmed ack must not revert.
AwaitAssert(() => recorder.ConfirmCalls.ShouldBe(1), duration: Timeout, cancellationToken: TestContext.Current.CancellationToken);
actor.Tell(new DriverInstanceActor.AttributeAlarmPublished("drv-1", Evt(AlarmTransitionKind.Clear)));
var next = NextAlarmUpdate(publish);
@@ -224,6 +227,62 @@ public sealed class DriverHostActorNativeAlarmAckRoutingTests : RuntimeActorTest
next.State.Acknowledged.ShouldBeTrue("the successful ack must have survived — no revert");
}
/// <summary>
/// lmxopcua#869 — the driver ACCEPTED the ack (AcknowledgeAsync returned normally, e.g. the gateway's "sent but
/// unconfirmed" OK) but its upstream never confirmed it: the confirmation refusal reverts the condition to
/// unacknowledged exactly like an AcknowledgeAsync failure. Before #869 nothing awaited a confirmation, so the
/// condition stayed Acknowledged although Galaxy never applied the ack.
/// </summary>
[Fact]
public void An_accepted_but_unconfirmed_ack_reverts_the_condition_to_unacknowledged()
{
var db = NewInMemoryDbFactory();
var recorder = new RecordingAlarmDriverFactory("GalaxyMxGateway")
{
Confirmation = () => Task.FromException(new DriverOperationRefusedException(
"Galaxy did not confirm the acknowledge of 'Tank.Hi' within 10 s (BadTimeout; gateway: AckMsg write sent; OnWriteComplete not observed)")),
};
var deploymentId = SeedV3AlarmDeployment(db, RevA, Driver: "drv-1", Tag: "temp_hi");
var publish = this.CreateCleanOutcomePublishProbe();
var actor = SpawnHostAndApply(db, deploymentId, recorder, publish.Ref);
RaiseAndExpectUnacked(actor, publish);
actor.Tell(new DriverHostActor.RouteNativeAlarmAck(AlarmRawPath, "cmt", "alice"));
var reverted = NextAlarmUpdate(publish);
reverted.AlarmNodeId.ShouldBe(AlarmRawPath);
reverted.State.Active.ShouldBeTrue();
reverted.State.Acknowledged.ShouldBeFalse("an ack the upstream never confirmed must not stand");
recorder.Acks.Count.ShouldBe(1);
recorder.ConfirmCalls.ShouldBe(1);
}
/// <summary>
/// lmxopcua#869 — the confirmation wait runs OFF the driver actor's mailbox: while one ack's confirmation is
/// still pending (it never completes here), a second ack is still routed to the driver. A confirmation awaited
/// inside the ack handler would suspend the mailbox and the second ack would never arrive.
/// </summary>
[Fact]
public void A_pending_confirmation_does_not_block_the_driver_actor()
{
var db = NewInMemoryDbFactory();
var never = new TaskCompletionSource();
var recorder = new RecordingAlarmDriverFactory("GalaxyMxGateway")
{
Confirmation = () => never.Task,
};
var deploymentId = SeedV3AlarmDeployment(db, RevA, Driver: "drv-1", Tag: "temp_hi");
var actor = SpawnHostAndApply(db, deploymentId, recorder);
actor.Tell(new DriverHostActor.RouteNativeAlarmAck(AlarmRawPath, "first", "alice"));
AwaitAssert(() => recorder.ConfirmCalls.ShouldBe(1), duration: Timeout, cancellationToken: TestContext.Current.CancellationToken);
actor.Tell(new DriverHostActor.RouteNativeAlarmAck(AlarmRawPath, "second", "alice"));
AwaitAssert(() => recorder.Acks.ShouldContain(a => a.Comment == "second"), duration: Timeout,
cancellationToken: TestContext.Current.CancellationToken);
never.TrySetResult();
}
/// <summary>
/// lmxopcua#760 — the host publishes the condition → owning-driver pre-check snapshot the OPC UA node
/// manager uses to refuse an acknowledge synchronously: the driver's own verdict for the condition it
@@ -452,6 +511,12 @@ public sealed class DriverHostActorNativeAlarmAckRoutingTests : RuntimeActorTest
/// <summary>The driver's synchronous pre-check verdict (lmxopcua#760); null ⇒ always proceed.</summary>
public Func<string?, string?>? PreflightRefusal { init => _driver.PreflightRefusal = value; }
/// <summary>lmxopcua#869: what <see cref="IAlarmAcknowledgeConfirmation.ConfirmAcknowledgedAsync"/> returns; null ⇒ confirmed at once.</summary>
public Func<Task>? Confirmation { init => _driver.Confirmation = value; }
/// <summary>How many confirmations the spawned driver was asked for.</summary>
public int ConfirmCalls => _driver.ConfirmCalls;
/// <inheritdoc />
public IDriver? TryCreate(string driverType, string driverInstanceId, string driverConfigJson)
{
@@ -465,8 +530,24 @@ public sealed class DriverHostActorNativeAlarmAckRoutingTests : RuntimeActorTest
}
/// <summary>An <see cref="IDriver"/> + <see cref="IAlarmSource"/> that records every acknowledge.</summary>
private sealed class RecordingAlarmDriver : IDriver, IAlarmSource, IAlarmAcknowledgePreflight
private sealed class RecordingAlarmDriver : IDriver, IAlarmSource, IAlarmAcknowledgePreflight, IAlarmAcknowledgeConfirmation
{
/// <summary>lmxopcua#869: the confirmation outcome; null ⇒ confirmed at once.</summary>
public Func<Task>? Confirmation { get; set; }
private int _confirmCalls;
/// <summary>How many confirmations were requested.</summary>
public int ConfirmCalls => Volatile.Read(ref _confirmCalls);
/// <inheritdoc />
public Task ConfirmAcknowledgedAsync(
IReadOnlyList<AlarmAcknowledgeRequest> acknowledgements, CancellationToken cancellationToken)
{
Interlocked.Increment(ref _confirmCalls);
return Confirmation?.Invoke() ?? Task.CompletedTask;
}
/// <summary>When set, <see cref="AcknowledgeAsync"/> records the request and then throws this.</summary>
public Exception? AckFault { get; set; }