fix(alarms): surface a refused native alarm ack synchronously, revert one that fails later (lmxopcua#760)

The native-ack path is fire-and-forget after the Part 9 Acknowledge returns, so
a Galaxy ack refused for lack of a credential (#749), or rejected by MXAccess,
left the client with Good and the condition showing Acknowledged while Galaxy
was never acked. Both options from the issue, layered:

1. Synchronous refusal for what the driver can decide without I/O. New optional
   IAlarmAcknowledgePreflight (Core.Abstractions); GalaxyDriver refuses an
   anonymous session when the driver has no AlarmAck fallback account.
   DriverHostActor publishes a condition NodeId -> owning-driver snapshot into
   NativeAlarmAckPreflightView (lock-free, volatile); the node manager's new
   NativeAlarmAckPreflight seam, consulted after the authz and Primary gates,
   returns BadUserAccessDenied with the reason - nothing routed, applied or
   reported. A keyed session is not pre-checked (its identity lives in the
   Secrets store, which must not be read under the SDK lock).

2. Revert for everything else. RouteAlarmAck carries the condition NodeId;
   DriverInstanceActor replies AlarmAckFailed on a refusal, a backend failure,
   a driver without an alarm source, or an ack dropped while connecting /
   reconnecting. DriverHostActor re-projects the condition unacknowledged via
   NativeAlarmProjector.TryRevertAcknowledge, but only while the local ack is
   still pending - any driver transition projected since then wins. A disabled
   or unspawned driver is not reverted (unchanged).

GatewayGalaxyAlarmAcknowledger now throws DriverOperationRefusedException on a
non-zero native hresult (wrong password, no ack permission) instead of only
warning, so that case reverts too; as a refusal it neither retries nor trips
the breaker.

Tests (each fails with its fix mutated out): projector revert/supersede,
DriverHostActor refused + failed ack -> unacked AlarmStateUpdate (successful
ack control), preflight view snapshot, node-manager preflight unit test,
over-the-wire ConditionMethodEventTests (BadUserAccessDenied, no route, no
event; positive control), Galaxy preflight + hresult refusal.
Docs: Galaxy.md, AlarmTracking.md, ScriptedAlarms.md, .claude/rules/alarms.md.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 13:12:17 -05:00
parent 0217d36cf4
commit 8e8d4cdf13
21 changed files with 737 additions and 61 deletions
@@ -78,7 +78,20 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
/// <param name="OperatorUser">The authenticated principal performing the acknowledge.</param>
/// <param name="ApiKeyId">Public id of the API key the session authenticated with; null for anonymous. Not a
/// secret — the Galaxy driver uses it to find the key's Galaxy ack identity (lmxopcua#749).</param>
public sealed record RouteAlarmAck(string ConditionId, string? Comment, string OperatorUser, string? ApiKeyId = null);
/// <param name="ConditionNodeId">The OPC UA condition NodeId the client acknowledged. When set, an
/// acknowledge that does not reach the upstream alarm system (refused, failed, or dropped because the driver
/// is not connected) is reported back to the sender as an <see cref="AlarmAckFailed"/> so the host can revert
/// the locally-applied acknowledge (lmxopcua#760). Null ⇒ no report.</param>
public sealed record RouteAlarmAck(
string ConditionId, string? Comment, string OperatorUser, string? ApiKeyId = null, string? ConditionNodeId = null);
/// <summary>
/// Reply to a <see cref="RouteAlarmAck"/> carrying a <see cref="RouteAlarmAck.ConditionNodeId"/> whose
/// acknowledge never reached the upstream alarm system (lmxopcua#760). Success sends nothing.
/// </summary>
/// <param name="ConditionNodeId">The OPC UA condition NodeId the client acknowledged.</param>
/// <param name="Reason">Why the acknowledge did not go through; never contains a secret.</param>
public sealed record AlarmAckFailed(string ConditionNodeId, string Reason);
public sealed record Subscribe(IReadOnlyList<string> FullReferences, TimeSpan PublishingInterval);
/// <summary>
/// Sets the set of references this driver should keep subscribed for the lifetime of the
@@ -609,10 +622,13 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
Sender.Tell(new WriteAttributeResult(false, "driver not connected")));
// A write launched while Connected can finish after the transition out of it; its slot still frees.
Receive<WriteCompleted>(OnWriteCompleted);
// An ack arriving while still connecting can't reach the upstream alarm system; drop it (the ack is
// fire-and-forget — no reply to surface — and the OPC UA condition state already committed locally).
Receive<RouteAlarmAck>(_ =>
_log.Debug("DriverInstance {Id}: alarm ack arrived during connect — dropped (driver not connected)", _driverInstanceId));
// An ack arriving while still connecting can't reach the upstream alarm system; drop it and report it
// failed so the host reverts the locally-committed acknowledge (lmxopcua#760).
Receive<RouteAlarmAck>(msg =>
{
_log.Debug("DriverInstance {Id}: alarm ack arrived during connect — dropped (driver not connected)", _driverInstanceId);
ReportAckFailed(Sender, msg, "the driver is not connected");
});
Receive<InitializeSucceeded>(msg =>
{
if (msg.Generation != _initGeneration) return;
@@ -913,10 +929,13 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
Receive<WriteAttribute>(_ =>
Sender.Tell(new WriteAttributeResult(false, "driver not connected")));
Receive<WriteCompleted>(OnWriteCompleted);
// An ack arriving while reconnecting can't reach the upstream alarm system; drop it (fire-and-forget,
// no reply — the OPC UA condition state already committed locally on the Part 9 ack).
Receive<RouteAlarmAck>(_ =>
_log.Debug("DriverInstance {Id}: alarm ack arrived during reconnect — dropped (driver not connected)", _driverInstanceId));
// An ack arriving while reconnecting can't reach the upstream alarm system; drop it and report it
// failed so the host reverts the locally-committed acknowledge (lmxopcua#760).
Receive<RouteAlarmAck>(msg =>
{
_log.Debug("DriverInstance {Id}: alarm ack arrived during reconnect — dropped (driver not connected)", _driverInstanceId);
ReportAckFailed(Sender, msg, "the driver is not connected");
});
Receive<InitializeSucceeded>(msg =>
{
if (msg.Generation != _initGeneration) return;
@@ -1250,15 +1269,19 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
/// correlates on <see cref="AlarmAcknowledgeRequest.ConditionId"/> (= the authored alarm
/// full-reference); <see cref="AlarmAcknowledgeRequest.SourceNodeId"/> carries the same reference (the
/// driver's ack path keys on ConditionId). Bounded to 5s so a hung backend can't pin this actor.
/// Fire-and-forget — the OPC UA Part 9 ack already committed the local condition state and
/// <see cref="IAlarmSource.AcknowledgeAsync"/> returns no per-condition status — so there is no reply;
/// a failure is logged and dropped (the local condition stays Acknowledged regardless).
/// Success sends no reply (the OPC UA Part 9 ack already committed the local condition state). A refusal
/// or failure is logged and, when the message carries a condition NodeId, reported to the sender as an
/// <see cref="AlarmAckFailed"/> so the host reverts the condition to unacknowledged (lmxopcua#760) —
/// otherwise the node showed Acknowledged for an alarm the upstream system never acknowledged.
/// </summary>
private async Task HandleAcknowledgeAsync(RouteAlarmAck msg)
{
// Captured before any await: the failure report goes back to whoever routed the ack.
var replyTo = Sender;
if (_driver is not IAlarmSource src)
{
_log.Warning("DriverInstance {Id}: alarm ack dropped — driver does not implement IAlarmSource", _driverInstanceId);
ReportAckFailed(replyTo, msg, "the driver has no alarm source");
return;
}
@@ -1289,14 +1312,25 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
// configuration verdict, not a backend fault, so no stack trace; the message names the fix.
_log.Warning("DriverInstance {Id}: native-alarm acknowledge of {Cond} by {User} refused — {Reason}",
_driverInstanceId, msg.ConditionId, msg.OperatorUser, ex.Message);
ReportAckFailed(replyTo, msg, ex.Message);
}
catch (Exception ex)
{
_log.Warning(ex, "DriverInstance {Id}: native-alarm acknowledge of {Cond} failed",
_driverInstanceId, msg.ConditionId);
// The exception TYPE only: a backend message is not ours to vouch for (it could echo a credential).
ReportAckFailed(replyTo, msg, $"the upstream acknowledge failed ({ex.GetType().Name})");
}
}
/// <summary>Tells <paramref name="replyTo"/> an <see cref="AlarmAckFailed"/> when the ack names a condition
/// NodeId (lmxopcua#760); a no-op for an ack routed without one.</summary>
private static void ReportAckFailed(IActorRef replyTo, RouteAlarmAck msg, string reason)
{
if (string.IsNullOrEmpty(msg.ConditionNodeId) || replyTo.IsNobody()) return;
replyTo.Tell(new AlarmAckFailed(msg.ConditionNodeId, reason));
}
private async Task HandleSubscribeAsync(Subscribe msg)
{
// Capture Sender/Self BEFORE any await. The re-subscribe path below awaits