diff --git a/.claude/rules/alarms.md b/.claude/rules/alarms.md index 8c770d7b..e82bda20 100644 --- a/.claude/rules/alarms.md +++ b/.claude/rules/alarms.md @@ -31,6 +31,15 @@ Confirmed/Unshelved constants own those fields); a native ack re-asserts `Confir SDK's ack unconfirms any confirm-capable condition and ScadaBridge maps that field. Pinned over the wire by `ConditionMethodEventTests` (exactly-once after a sentinel, folder + `i=2253`). +**A native ack the driver refuses or fails is not left Acknowledged (#760).** Two layers: (1) a refusal the +driver can decide without I/O (`IAlarmAcknowledgePreflight` — Galaxy: anonymous session and no fallback ack +account) fails the Part 9 call `BadUserAccessDenied` via `NativeAlarmAckPreflight`, read lock-free from the +`NativeAlarmAckPreflightView` snapshot `DriverHostActor` publishes; never do I/O (e.g. a Secrets lookup) in a +pre-check — it runs under the SDK `Lock`. (2) anything else (keyed session without an identity, MXAccess +non-zero `hresult`, driver still connecting/reconnecting; a disabled/unspawned driver is NOT reverted) comes back as `DriverInstanceActor.AlarmAckFailed` and +`DriverHostActor` re-projects the condition unacknowledged — only while the ack is still pending in +`NativeAlarmProjector` (any driver transition since then wins). + Behaviours that were wrong or missing and are now fixed — keep them: the SDK's `OnTimedUnshelve` is a **no-op** (the engine's `AutoUnshelve` owns timed-shelve expiry, and `UnshelveTime` reports the real remaining time from `AlarmConditionSnapshot.UnshelveAtUtc`); a **commented** re-ack is stored as an diff --git a/docs/AlarmTracking.md b/docs/AlarmTracking.md index db1d3151..b7d540cb 100644 --- a/docs/AlarmTracking.md +++ b/docs/AlarmTracking.md @@ -457,12 +457,20 @@ from the scripted `AlarmCommandRouter`: authenticated `OperatorUser`. The driver forwards this to the Galaxy gateway → AVEVA via `GatewayGalaxyAlarmAcknowledger`. The Galaxy adapter awaits the correlated `AcknowledgeAlarmAsync` RPC: - gateway protocol failures throw, while a non-zero native return code is - warning-logged. **The actor route back to OPC UA is fire-and-forget** — - `DriverInstanceActor` logs and drops failures without replying to the - OPC UA client, whose local `AlarmConditionState` SDK update already - committed at step 1. This is a native-ack routing limitation; Galaxy - value writes have a separate correlated write-outcome path. An awaited + gateway protocol failures throw, and a non-zero native return code is + warning-logged and thrown as a refusal (#760). **A failed ack is reverted, + not surfaced on the call** — the OPC UA client's local + `AlarmConditionState` update already committed at step 1, so when the + driver refuses or fails the ack (or is still connecting / reconnecting), + `DriverInstanceActor` replies `AlarmAckFailed` and + `DriverHostActor` re-projects the condition **unacknowledged** (a fresh + condition event). Only an ack still pending in `NativeAlarmProjector` is + reverted: a driver transition projected since then is authoritative. A + refusal the driver can decide without I/O (Galaxy: an anonymous session + and no fallback ack account) is instead returned synchronously as + `BadUserAccessDenied` by the node manager's `NativeAlarmAckPreflight` + before anything is applied. Galaxy value writes have a separate correlated + write-outcome path. An awaited ack RPC does not prove the ack reached AVEVA event history: the H6 live verification remains [operator-gated](plans/2026-06-15-stillpending-phase-3-opcua-standards-design.md). @@ -619,8 +627,9 @@ reports nothing. commented re-ack from the AdminUI `/alerts` page. - **A native re-ack is still routed to the driver** (deliberate, #597). The route is fire-and-forget, so a node showing Acked does not prove the - upstream system acknowledged — an ack dropped during a reconnect leaves - exactly that state — and re-acking is the documented retry (§"Best-effort + upstream system acknowledged — an ack the driver reports as failed is + reverted (#760), but one lost without a report (e.g. a node stopped mid-ack) + is not — and re-acking is the documented retry (§"Best-effort during reconnect"). The cost is a duplicate ack upstream for an alarm that was already acknowledged there. - **Native conditions: Acknowledge and AddComment only** — see §"Native alarm diff --git a/docs/ScriptedAlarms.md b/docs/ScriptedAlarms.md index 79920c3e..9b20a7b9 100644 --- a/docs/ScriptedAlarms.md +++ b/docs/ScriptedAlarms.md @@ -325,8 +325,9 @@ routes through a separate `NativeAlarmAckRouter` seam (instead of the scripted FullName)` inverse map, Primary-gated) → the owning driver's `IAlarmSource.AcknowledgeAsync` → Galaxy gateway → AVEVA. The call carries the authenticated operator's display name via the `AlarmAcknowledgeRequest.OperatorUser` -field. The actor route back to OPC UA is fire-and-forget — a failed upstream ack -is logged rather than surfaced back to the OPC UA client. The Galaxy adapter +field. The actor route back to OPC UA is fire-and-forget; a failed or refused upstream +ack reverts the condition to unacknowledged (#760), and one the driver can refuse +without I/O fails the call synchronously with `BadUserAccessDenied`. The Galaxy adapter awaits its correlated RPC; this does not establish delivery to AVEVA event history. Galaxy value writes use a separate correlated write-outcome path. See [AlarmTracking.md §Native alarm acknowledge → AVEVA](AlarmTracking.md#native-alarm-acknowledge--aveva) diff --git a/docs/drivers/Galaxy.md b/docs/drivers/Galaxy.md index adc94d74..d3c90cc6 100644 --- a/docs/drivers/Galaxy.md +++ b/docs/drivers/Galaxy.md @@ -107,9 +107,17 @@ Galaxy password, so `GalaxyAckCredentialResolver` picks the account per acknowle `file:PATH` or `dev:VALUE`; a bare literal is refused so a password never sits in DriverConfig. 3. **Neither** → the driver throws `DriverOperationRefusedException`; **nothing is sent to the gateway**. The refusal is excluded from the resilience breaker (it says nothing about gateway - health). ⚠️ The native ack path is fire-and-forget after the Part 9 method returns, so the OPC UA - call itself still returns `Good` and the refusal surfaces as a `DriverInstance` warning naming the - fix — a synchronous status / revert is #760. + health). How the client learns of it (#760): + - **Synchronously** when the driver can tell without I/O — an *anonymous* session on a driver with + no fallback account: the Part 9 `Acknowledge` call itself fails `BadUserAccessDenied` with the + reason, and nothing is applied (`IAlarmAcknowledgePreflight`, published per condition by + `DriverHostActor` into `NativeAlarmAckPreflightView`). + - **By revert** otherwise — a keyed session whose key has no identity (that lives in the Secrets + store, which is not read under the SDK lock), or MXAccess rejecting the account (a non-zero + native `hresult`, e.g. a wrong password, now thrown as a refusal): the call returned `Good`, then + `DriverHostActor` re-projects the condition **unacknowledged** (a fresh condition event, + `Retain=true`) and logs the reason. A driver transition projected after the ack wins over a late + failure. The comment written to `AckMsg` is suffixed with the OPC UA identity — `" [OPC UA: , key ]"` or `"[OPC UA: anonymous]"` — so the client is not diff --git a/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/DriverOperationRefusedException.cs b/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/DriverOperationRefusedException.cs index 667d7c52..82078f5c 100644 --- a/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/DriverOperationRefusedException.cs +++ b/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/DriverOperationRefusedException.cs @@ -1,9 +1,10 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions; /// -/// Thrown by a driver that deliberately refuses an operation before contacting its backend — a -/// configuration / authorization verdict (e.g. no Galaxy ack credential is available for the -/// caller, lmxopcua#749), not a transport or device fault. +/// Thrown when an operation is refused as a configuration / authorization verdict rather than failing +/// on a transport or device fault: by the driver itself before contacting its backend (e.g. no Galaxy +/// ack credential is available for the caller, lmxopcua#749), or by a backend that answered and +/// rejected it (e.g. MXAccess rejecting the operator account on a Galaxy acknowledge, lmxopcua#760). /// /// /// The resilience pipeline neither retries it nor counts it toward the circuit breaker: a refusal diff --git a/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/IAlarmAcknowledgePreflight.cs b/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/IAlarmAcknowledgePreflight.cs new file mode 100644 index 00000000..d57310d2 --- /dev/null +++ b/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/IAlarmAcknowledgePreflight.cs @@ -0,0 +1,24 @@ +namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +/// +/// Optional companion to : lets the OPC UA Part 9 Acknowledge method +/// refuse, with a synchronous status, an acknowledge the driver already knows it would refuse +/// (lmxopcua#760). The native ack path is otherwise fire-and-forget after the method returns, so without +/// this the client sees Good for an acknowledge that never reached the upstream alarm system. +/// +/// +/// Called on an OPC UA server thread, under the SDK node-manager lock: an implementation must answer from +/// state it already holds — no I/O, no secret lookups, no blocking. It is a best-effort pre-check for +/// static misconfiguration only; an acknowledge it lets through can still fail later (that path reverts +/// the condition to unacknowledged instead). +/// +public interface IAlarmAcknowledgePreflight +{ + /// + /// Returns why an acknowledge from a session that authenticated with would + /// be refused, or null when it would proceed or the driver cannot tell without I/O. + /// + /// The session's API key id, or null for an anonymous session. + /// A refusal reason safe to show the OPC UA client (never a secret), or null. + string? GetAcknowledgeRefusal(string? apiKeyId); +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/GalaxyDriver.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/GalaxyDriver.cs index 56111541..bc377e6a 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/GalaxyDriver.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/GalaxyDriver.cs @@ -38,7 +38,7 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Galaxy; /// /// public sealed class GalaxyDriver - : IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IRediscoverable, IHostConnectivityProbe, IAlarmSource, ISkippedTagReporter, IDisposable, IAsyncDisposable + : IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IRediscoverable, IHostConnectivityProbe, IAlarmSource, IAlarmAcknowledgePreflight, ISkippedTagReporter, IDisposable, IAsyncDisposable { /// Authored tags this driver could not dial — a TagConfig with no usable /// attributeRef has no MXAccess address to bind. @@ -1966,6 +1966,21 @@ public sealed class GalaxyDriver } } + /// + /// + /// Only the case decidable without I/O (lmxopcua#760): an anonymous session (no API key, so no key-mapped + /// Galaxy identity can exist) on a driver with no AlarmAck.FallbackUser. A keyed session is let + /// through — whether the key has an identity lives in the Secrets store, which must not be read under the + /// SDK lock; still refuses it, and the host reverts the condition. + /// + public string? GetAcknowledgeRefusal(string? apiKeyId) + { + if (!string.IsNullOrWhiteSpace(apiKeyId) || (_options.AlarmAck?.HasFallback ?? false)) return null; + return $"the session is anonymous (no API key) and Galaxy driver '{_driverInstanceId}' has no " + + "AlarmAck.FallbackUser configured, so the acknowledge cannot be sent to Galaxy. Connect with an API key " + + "that has a Galaxy ack identity (AdminUI /api-keys) or configure the driver's fallback ack account."; + } + private static string PrincipalOf(AlarmAcknowledgeRequest ack) => string.IsNullOrWhiteSpace(ack.OperatorUser) ? "anonymous" : ack.OperatorUser; diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/Runtime/GatewayGalaxyAlarmAcknowledger.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/Runtime/GatewayGalaxyAlarmAcknowledger.cs index 242c90cc..211f11a6 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/Runtime/GatewayGalaxyAlarmAcknowledger.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/Runtime/GatewayGalaxyAlarmAcknowledger.cs @@ -1,6 +1,7 @@ using Microsoft.Extensions.Logging; using ZB.MOM.WW.MxGateway.Client; using ZB.MOM.WW.MxGateway.Contracts.Proto; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; namespace ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Runtime; @@ -26,8 +27,12 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Runtime; /// A non-OK means the gateway never reached MXAccess (transport / /// dispatch / validation failure) and is surfaced as a thrown exception. A non-zero native ack /// return code (hresult — now the AuthenticateUser HRESULT or the AckMsg write status) -/// means MXAccess rejected the ack; that is logged as a warning rather than thrown so a transient -/// MXAccess hiccup doesn't block the operator workflow — the operator can retry. +/// means MXAccess rejected the ack (a wrong password, an account without ack permission, a failed +/// AckMsg write). It is logged and thrown as a +/// (lmxopcua#760): the host then reverts the OPC UA condition to unacknowledged instead of leaving it +/// claiming an ack Galaxy never recorded, and the operator can retry. A refusal, not a fault — the +/// gateway answered — so it neither retries nor counts toward the resilience breaker (a mis-keyed +/// account must not open the breaker for every correctly-configured caller). /// /// internal sealed class GatewayGalaxyAlarmAcknowledger : IGalaxyAlarmAcknowledger @@ -98,7 +103,8 @@ internal sealed class GatewayGalaxyAlarmAcknowledger : IGalaxyAlarmAcknowledger } /// - /// Applies the reply semantics: non-OK protocol status throws; a non-zero native code warns. + /// Applies the reply semantics: non-OK protocol status throws; a non-zero native code warns and throws a + /// (lmxopcua#760). /// Every message is scrubbed of the password. /// /// The gateway reply. @@ -125,6 +131,9 @@ internal sealed class GatewayGalaxyAlarmAcknowledger : IGalaxyAlarmAcknowledger logger.LogWarning( "Galaxy AcknowledgeAlarm for {AlarmRef} as {GalaxyUser} returned native ack failure code {Hresult}.", alarmFullReference, credential.DisplayName, reply.Hresult); + throw new DriverOperationRefusedException( + $"MXAccess rejected the Galaxy acknowledge of '{alarmFullReference}' as '{credential.DisplayName}' " + + $"(native code 0x{reply.Hresult:X8}); check the account's password and Galaxy acknowledge permission."); } } diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs b/src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs index a14a576b..2e614594 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs @@ -49,6 +49,7 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl private readonly IDriverPrimaryRoleSource? _primaryRoleSource; private readonly HistorianUpdaterHolder _historianUpdater; private readonly IHistoryUpdateRoleView? _historyUpdateRoleView; + private readonly NativeAlarmAckPreflightView? _ackPreflightView; private readonly OpcUaEndpointState _endpointState; private readonly bool _rethrowStartFailure; @@ -113,6 +114,8 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl /// to serve this endpoint, so a failed SDK start RETHROWS there (the host's fatal handler exits non-zero and /// the supervisor restarts the process); a fused node keeps its AdminUI up and reports the failure through /// the health probe instead. Null (harnesses) behaves as fused. + /// The native-ack pre-check snapshot DriverHostActor publishes + /// (lmxopcua#760). Optional: registered on every driver-role node; null wires no pre-check. public OtOpcUaServerHostedService( IOptions options, DeferredAddressSpaceSink deferredSink, @@ -131,7 +134,8 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl HistorianUpdaterHolder? historianUpdater = null, IHistoryUpdateRoleView? historyUpdateRoleView = null, OpcUaEndpointState? endpointState = null, - EffectiveRoles? roles = null) + EffectiveRoles? roles = null, + NativeAlarmAckPreflightView? ackPreflightView = null) { ArgumentNullException.ThrowIfNull(aclOptions); ArgumentNullException.ThrowIfNull(time); @@ -156,6 +160,7 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl _primaryRoleSource = primaryRoleSource; _historianUpdater = historianUpdater ?? HistorianUpdaterHolder.None; _historyUpdateRoleView = historyUpdateRoleView; + _ackPreflightView = ackPreflightView; _endpointState = endpointState ?? new OpcUaEndpointState(); _rethrowStartFailure = roles is { HasDriver: true, HasAdmin: false }; _logger = loggerFactory.CreateLogger(); @@ -279,6 +284,14 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl } }); + // lmxopcua#760: refuse, with a synchronous BadUserAccessDenied, a native ack the owning driver already + // knows it would refuse (e.g. anonymous session + Galaxy driver without a fallback ack account). A + // volatile snapshot read — safe under the SDK Lock. + if (_ackPreflightView is { } ackPreflightView) + { + _server.SetNativeAlarmAckPreflight(ack => ackPreflightView.Check(ack.ConditionNodeId, ack.ApiKeyId)); + } + // A native ack is accepted only on the Primary. The verdict is the one DriverHostActor // publishes and gates the route above on, so a Secondary refuses (BadInvalidState) instead of applying an // ack the driver will never see and then reverting it. A volatile property read — safe under the SDK Lock. @@ -544,6 +557,7 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl // Clear the native-alarm-ack router so a late ack doesn't Tell a stopping DriverHostActor. _server?.SetNativeAlarmAckRouter(null); _server?.SetNativeAlarmAckPrimaryGate(null); + _server?.SetNativeAlarmAckPreflight(null); _server?.SetHistoryUpdatePrimaryGate(null); // Restore the Null historian so a late HistoryRead doesn't hit a disposed read client. _server?.SetHistorianDataSource(null); diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs b/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs index b8e36fb9..471eef2f 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs @@ -255,6 +255,15 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 /// public Func? NativeAlarmAckPrimaryGate { get; set; } + /// + /// Synchronous pre-check for an OPC UA Acknowledge of a NATIVE condition (lmxopcua#760): returns why the + /// owning driver would refuse it, or null to proceed. Consulted after the authorization and Primary gates; + /// a non-null answer refuses the method with BadUserAccessDenied carrying the reason, and nothing is + /// routed, applied or reported. Runs under the manager's Lock, so it must be a lock-free read of + /// state the host already holds. Null (the default) skips the check. + /// + public Func? NativeAlarmAckPreflight { get; set; } + /// /// Whether this node may serve an OPC UA HistoryUpdate right now: the redundancy data-plane /// verdict (PrimaryGatePolicy.ShouldServiceAsPrimary, as published to IDriverPrimaryRoleSource) — @@ -2123,7 +2132,8 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 /// The acknowledge comment text, or null when none was supplied. /// ServiceResult.Good when allowed (the SDK then applies state; the method path reports the event); /// BadUserAccessDenied when the gate vetoes, BadInvalidState when this node is not the redundancy - /// Primary () — in both cases no route, no state mutation. + /// Primary (), BadUserAccessDenied with the driver's reason when + /// refuses it — in every case no route, no state mutation. private ServiceResult HandleNativeAlarmAck(ISystemContext context, ConditionState condition, LocalizedText? comment) { // Resolve + gate the SAME way HandleAlarmCommand does so native and scripted acks share one authz @@ -2145,12 +2155,22 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 // Same condition-node-id extraction HandleAlarmCommand uses (condition.NodeId.Identifier?.ToString()): // for a native condition this is the folder-scoped NodeId string the DriverHostActor inverse map // (_alarmNodeIdByDriverRef value set) keys by, so the next task can resolve it back to (driver, ref). - // Non-blocking by contract (host wires a fire-and-forget dispatch); safe to call under Lock. - NativeAlarmAckRouter?.Invoke(new NativeAlarmAck( + var ack = new NativeAlarmAck( ConditionNodeId: condition.NodeId.Identifier?.ToString() ?? string.Empty, Comment: comment?.Text, OperatorUser: AlarmOperatorName(identity!), - ApiKeyId: identity!.ApiKeyId)); + ApiKeyId: identity!.ApiKeyId); + + // lmxopcua#760 — a driver that already knows it would refuse this acknowledge (e.g. an anonymous session + // on a Galaxy driver with no fallback ack account) refuses it HERE, synchronously, rather than the client + // getting Good for an ack the upstream alarm system never sees. Lock-free read; nothing routed or applied. + if (NativeAlarmAckPreflight?.Invoke(ack) is { } refusal) + { + return new ServiceResult(StatusCodes.BadUserAccessDenied, new LocalizedText(refusal)); + } + + // Non-blocking by contract (host wires a fire-and-forget dispatch); safe to call under Lock. + NativeAlarmAckRouter?.Invoke(ack); // Good ⇒ the SDK applies the node-state change; WireMethodTransitionReporting then reports the one // condition event (the SDK reports none of its own). diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaSdkServer.cs b/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaSdkServer.cs index 11efea7f..0a05f131 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaSdkServer.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaSdkServer.cs @@ -83,6 +83,20 @@ public sealed class OtOpcUaSdkServer : StandardServer return true; } + /// + /// Wire the synchronous native-ack pre-check (, + /// lmxopcua#760). Passing null clears it. No-op (returns false) when the node manager has not + /// been created yet (mirrors ). + /// + /// A lock-free read returning the refusal reason, or null to proceed. + /// true when set on a live node manager; false when no node manager exists yet. + public bool SetNativeAlarmAckPreflight(Func? preflight) + { + if (_otOpcUaNodeManager is null) return false; + _otOpcUaNodeManager.NativeAlarmAckPreflight = preflight; + return true; + } + /// /// Wire the redundancy verdict that decides whether this node serves an OPC UA HistoryUpdate /// (); on any other node every authorized node of a diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverHostActor.cs b/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverHostActor.cs index 48def4ed..962d134a 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverHostActor.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverHostActor.cs @@ -476,6 +476,9 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS /// Where the HistoryUpdate verdict is published. Null on nodes that wire no redundancy manager. private readonly IHistoryUpdateRoleView? _historyUpdateRoleView; + /// Where the synchronous native-ack pre-check snapshot is published (lmxopcua#760); null ⇒ none. + private readonly NativeAlarmAckPreflightView? _ackPreflightView; + /// Whether any redundancy snapshot has reached this host — the HistoryUpdate gate stays /// closed until one has (see ). private bool _redundancySnapshotReceived; @@ -617,6 +620,9 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS private sealed record ChildEntry(IActorRef Actor, DriverInstanceSpec Spec, bool Stubbed) { + /// The driver's synchronous native-ack pre-check, when it offers one (lmxopcua#760). + public IAlarmAcknowledgePreflight? AckPreflight { get; init; } + /// When this child was spawned — the diagnostics' last-change time until it publishes health. public DateTime SpawnedUtc { get; } = DateTime.UtcNow; @@ -749,7 +755,8 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS TimeSpan? applyOutcomeTimeout = null, bool captureWhilePrimaryUnreachable = true, IHistoryUpdateRoleView? historyUpdateRoleView = null, - ScriptingOptions? scripting = null) => + ScriptingOptions? scripting = null, + NativeAlarmAckPreflightView? ackPreflightView = null) => // WARNING: this forwarding list is POSITIONAL, and Props.Create compiles it into an // expression tree. Six IActorRef? parameters and several interface-typed ones mean a // mis-ordered argument is usually type-compatible and therefore compiles clean, then binds @@ -763,7 +770,7 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS alarmStateStore, telemetryHub, primaryRoleSource, historizationRoleView, sharesHistorizationScopeWithLocal, applyWedgeTimeout, reprovisionBackoff, commsRecoveryAuditDelay, sharesAlarmQueueWithLocal, applyOutcomeTimeout, captureWhilePrimaryUnreachable, historyUpdateRoleView, - scripting)); + scripting, ackPreflightView)); /// Initializes a new DriverHostActor with the specified dependencies. /// Database context factory for configuration database access. @@ -852,6 +859,8 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS /// redundancy manager. /// The node-wide Scripting run-guard options handed to the scripted-alarm /// engine. Null ⇒ the compiled defaults. + /// Where the condition → owning-driver native-ack pre-check snapshot is published + /// for the node manager's synchronous refusal (lmxopcua#760). Null ⇒ nothing published (no pre-check). public DriverHostActor( IDbContextFactory? dbFactory, CommonsNodeId localNode, @@ -885,8 +894,10 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS TimeSpan? applyOutcomeTimeout = null, bool captureWhilePrimaryUnreachable = true, IHistoryUpdateRoleView? historyUpdateRoleView = null, - ScriptingOptions? scripting = null) + ScriptingOptions? scripting = null, + NativeAlarmAckPreflightView? ackPreflightView = null) { + _ackPreflightView = ackPreflightView; _scripting = scripting; _captureWhilePrimaryUnreachable = captureWhilePrimaryUnreachable; _historyUpdateRoleView = historyUpdateRoleView; @@ -1545,6 +1556,7 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS Receive(HandleReconnectDriver); Receive(HandleRouteNodeWrite); Receive(HandleRouteNativeAlarmAck); + Receive(HandleAlarmAckFailed); Receive(OnRedundancyStateChanged); Receive(OnDriverMembershipChanged); Receive(OnReachabilityChanged); @@ -1609,6 +1621,7 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS Receive(HandleReconnectDriver); Receive(HandleRouteNodeWrite); Receive(HandleRouteNativeAlarmAck); + Receive(HandleAlarmAckFailed); Receive(OnRedundancyStateChanged); Receive(OnDriverMembershipChanged); Receive(OnReachabilityChanged); @@ -2260,9 +2273,9 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS /// /// Routes an inbound native-condition acknowledge (the host Tells this from the OPC UA node-manager /// side when a client Acknowledges a NATIVE Part 9 condition) to the owning driver child. Mirrors - /// 's order + gating, but the ack is fire-and-forget (no reply): - /// the OPC UA Part 9 acknowledge already committed the local condition state and the driver's - /// returns no per-condition status. + /// 's order + gating, but the ack is fire-and-forget (no reply to the + /// OPC UA client): the Part 9 acknowledge already committed the local condition state. A refused or failed + /// ack comes back as and reverts it (lmxopcua#760). /// /// PRIMARY gate FIRST — reuses the same redundancy signal as the /// inbound-write gate. Only the Primary pushes the ack to the shared upstream alarm system @@ -2316,13 +2329,68 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS { _log.Debug("DriverHost {Node}: driver {Driver} not running for alarm ack of {Cond} — dropped", _localNode, target.DriverInstanceId, msg.ConditionNodeId); + // Deliberately NOT reverted (lmxopcua#760 covers what a driver reports): a disabled or not-yet-spawned + // driver produces no transitions either, so the local ack stands, as before. return; } - // Fire-and-forget: the OPC UA Part 9 ack already committed the local condition state, and the - // driver's AcknowledgeAsync surfaces no per-condition status, so there is nothing to reply. The - // driver correlates on ConditionId (= the authored alarm FullName the inverse map keyed on). - entry.Actor.Tell(new DriverInstanceActor.RouteAlarmAck(target.RawPath, msg.Comment, msg.OperatorUser, msg.ApiKeyId)); + // The OPC UA Part 9 ack already committed the local condition state; success sends nothing back. A + // refusal or failure comes back as DriverInstanceActor.AlarmAckFailed (keyed by the condition NodeId + // carried here) and reverts the node (HandleAlarmAckFailed). The driver correlates on ConditionId + // (= the authored alarm FullName the inverse map keyed on). + entry.Actor.Tell(new DriverInstanceActor.RouteAlarmAck( + target.RawPath, msg.Comment, msg.OperatorUser, msg.ApiKeyId, ConditionNodeId: msg.ConditionNodeId)); + } + + /// + /// A native acknowledge the driver refused or failed to deliver upstream (lmxopcua#760): revert the + /// condition to unacknowledged so the address space stops claiming an ack the alarm system never saw. + /// Mirrors the optimistic-write self-correction — the client got Good on the method call, and the + /// revert reaches it as a fresh condition event (Acked=false, Retain=true). + /// + /// The failure report from the driver child. + private void HandleAlarmAckFailed(DriverInstanceActor.AlarmAckFailed msg) => + RevertNativeAlarmAck(msg.ConditionNodeId, msg.Reason); + + /// + /// Re-projects as unacknowledged when its local acknowledge is still + /// pending in the projector. A driver transition projected since the ack is authoritative, so a late + /// failure after it changes nothing. + /// + /// The bare condition NodeId (Raw realm). + /// Why the acknowledge did not reach the upstream alarm system. + private void RevertNativeAlarmAck(string conditionNodeId, string reason) + { + if (!_nativeAlarmProjector.TryRevertAcknowledge(conditionNodeId, out var snapshot) || snapshot is null) + { + _log.Info("DriverHost {Node}: native-alarm acknowledge of {Cond} did not reach the alarm system ({Reason}); " + + "the condition has since been re-stated by its driver, so it is left as projected", + _localNode, conditionNodeId, reason); + return; + } + + _log.Warning("DriverHost {Node}: native-alarm acknowledge of {Cond} did not reach the alarm system ({Reason}); " + + "reverting the condition to unacknowledged", _localNode, conditionNodeId, reason); + // Native conditions live in the Raw realm at their bare NodeId (the apply pass's single raw condition). + _opcUaPublishActor?.Tell(new ZB.MOM.WW.OtOpcUa.Runtime.OpcUa.OpcUaPublishActor.AlarmStateUpdate( + conditionNodeId, snapshot, DateTime.UtcNow, AddressSpaceRealm.Raw)); + } + + /// + /// Publishes the condition NodeId → owning-driver native-ack pre-check snapshot (lmxopcua#760). Called + /// whenever the condition map or the set of running children changes; a no-op without a view. + /// + private void PublishAckPreflight() + { + if (_ackPreflightView is null) return; + var entries = new List>(); + foreach (var (conditionNodeId, target) in _driverRefByAlarmNodeId) + { + if (_children.TryGetValue(target.DriverInstanceId, out var child) && child.AckPreflight is { } preflight) + entries.Add(new(conditionNodeId, preflight)); + } + + _ackPreflightView.Publish(entries); } /// Whether this node should service a Primary-only data-plane operation right now — the single @@ -2539,6 +2607,9 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS Receive(msg => _log.Debug("DriverHost {Node}: dropping native-alarm ack for {Node2} while Stale (config DB unreachable)", _localNode, msg.ConditionNodeId)); + // An ack routed before the host went Stale can still come back refused; the revert needs only the + // projector + publish actor, both of which survive Stale. + Receive(HandleAlarmAckFailed); // A child connectivity transition while the host is Stale has no live address space to annotate — drop // it (the post-recovery rebuild re-materialises conditions, and the child re-announces on its next entry). Receive(_ => { }); @@ -3654,6 +3725,7 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS // Acked=true — on any deploy, rebuild or not. The node manager carries the NODE's state across a rebuild, // so the two stay in step. Keyed by the bare condition NodeId, exactly as ForwardNativeAlarm projects. _nativeAlarmProjector.RetainOnly(_driverRefByAlarmNodeId.Keys.ToHashSet(StringComparer.Ordinal)); + PublishAckPreflight(); // Cache the per-driver payload as THE authority, in the same clear-and-repopulate pass as the routing // maps above so a rename/removal can never leave a stale entry behind. Keyed over the UNION of the two @@ -3812,7 +3884,11 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS child.Tell(new DriverInstanceActor.InitializeRequested(spec.DriverConfig)); } - _children[spec.DriverInstanceId] = new ChildEntry(child, spec, stub); + _children[spec.DriverInstanceId] = new ChildEntry(child, spec, stub) + { + AckPreflight = stub ? null : driver as IAlarmAcknowledgePreflight, + }; + PublishAckPreflight(); // EVERY spawn hands the child its desired set — deploy reconcile, operator Restart (03/S16), and the // supervision reprovision. Sending it HERE rather than at each call site is the point: a child that @@ -3870,6 +3946,7 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS RetireChild(entry.Actor); Context.Stop(entry.Actor); _children.Remove(driverInstanceId); + PublishAckPreflight(); EvictRetainedValues(driverInstanceId); // The instance left the plan (StopChild is only reached via plan.ToStop), so its cached // Health/Resilience snapshots must not replay to new telemetry subscribers. Release the shared diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverInstanceActor.cs b/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverInstanceActor.cs index 8fda0205..f5ed4670 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverInstanceActor.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverInstanceActor.cs @@ -78,7 +78,20 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers /// The authenticated principal performing the acknowledge. /// Public id of the API key the session authenticated with; null for anonymous. Not a /// secret — the Galaxy driver uses it to find the key's Galaxy ack identity (lmxopcua#749). - public sealed record RouteAlarmAck(string ConditionId, string? Comment, string OperatorUser, string? ApiKeyId = null); + /// The OPC UA condition NodeId the client acknowledged. When set, an + /// acknowledge that does not reach the upstream alarm system (refused, failed, or dropped because the driver + /// is not connected) is reported back to the sender as an so the host can revert + /// the locally-applied acknowledge (lmxopcua#760). Null ⇒ no report. + public sealed record RouteAlarmAck( + string ConditionId, string? Comment, string OperatorUser, string? ApiKeyId = null, string? ConditionNodeId = null); + + /// + /// Reply to a carrying a whose + /// acknowledge never reached the upstream alarm system (lmxopcua#760). Success sends nothing. + /// + /// The OPC UA condition NodeId the client acknowledged. + /// Why the acknowledge did not go through; never contains a secret. + public sealed record AlarmAckFailed(string ConditionNodeId, string Reason); public sealed record Subscribe(IReadOnlyList FullReferences, TimeSpan PublishingInterval); /// /// Sets the set of references this driver should keep subscribed for the lifetime of the @@ -609,10 +622,13 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers Sender.Tell(new WriteAttributeResult(false, "driver not connected"))); // A write launched while Connected can finish after the transition out of it; its slot still frees. Receive(OnWriteCompleted); - // An ack arriving while still connecting can't reach the upstream alarm system; drop it (the ack is - // fire-and-forget — no reply to surface — and the OPC UA condition state already committed locally). - Receive(_ => - _log.Debug("DriverInstance {Id}: alarm ack arrived during connect — dropped (driver not connected)", _driverInstanceId)); + // An ack arriving while still connecting can't reach the upstream alarm system; drop it and report it + // failed so the host reverts the locally-committed acknowledge (lmxopcua#760). + Receive(msg => + { + _log.Debug("DriverInstance {Id}: alarm ack arrived during connect — dropped (driver not connected)", _driverInstanceId); + ReportAckFailed(Sender, msg, "the driver is not connected"); + }); Receive(msg => { if (msg.Generation != _initGeneration) return; @@ -913,10 +929,13 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers Receive(_ => Sender.Tell(new WriteAttributeResult(false, "driver not connected"))); Receive(OnWriteCompleted); - // An ack arriving while reconnecting can't reach the upstream alarm system; drop it (fire-and-forget, - // no reply — the OPC UA condition state already committed locally on the Part 9 ack). - Receive(_ => - _log.Debug("DriverInstance {Id}: alarm ack arrived during reconnect — dropped (driver not connected)", _driverInstanceId)); + // An ack arriving while reconnecting can't reach the upstream alarm system; drop it and report it + // failed so the host reverts the locally-committed acknowledge (lmxopcua#760). + Receive(msg => + { + _log.Debug("DriverInstance {Id}: alarm ack arrived during reconnect — dropped (driver not connected)", _driverInstanceId); + ReportAckFailed(Sender, msg, "the driver is not connected"); + }); Receive(msg => { if (msg.Generation != _initGeneration) return; @@ -1250,15 +1269,19 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers /// correlates on (= the authored alarm /// full-reference); carries the same reference (the /// driver's ack path keys on ConditionId). Bounded to 5s so a hung backend can't pin this actor. - /// Fire-and-forget — the OPC UA Part 9 ack already committed the local condition state and - /// returns no per-condition status — so there is no reply; - /// a failure is logged and dropped (the local condition stays Acknowledged regardless). + /// Success sends no reply (the OPC UA Part 9 ack already committed the local condition state). A refusal + /// or failure is logged and, when the message carries a condition NodeId, reported to the sender as an + /// so the host reverts the condition to unacknowledged (lmxopcua#760) — + /// otherwise the node showed Acknowledged for an alarm the upstream system never acknowledged. /// private async Task HandleAcknowledgeAsync(RouteAlarmAck msg) { + // Captured before any await: the failure report goes back to whoever routed the ack. + var replyTo = Sender; if (_driver is not IAlarmSource src) { _log.Warning("DriverInstance {Id}: alarm ack dropped — driver does not implement IAlarmSource", _driverInstanceId); + ReportAckFailed(replyTo, msg, "the driver has no alarm source"); return; } @@ -1289,14 +1312,25 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers // configuration verdict, not a backend fault, so no stack trace; the message names the fix. _log.Warning("DriverInstance {Id}: native-alarm acknowledge of {Cond} by {User} refused — {Reason}", _driverInstanceId, msg.ConditionId, msg.OperatorUser, ex.Message); + ReportAckFailed(replyTo, msg, ex.Message); } catch (Exception ex) { _log.Warning(ex, "DriverInstance {Id}: native-alarm acknowledge of {Cond} failed", _driverInstanceId, msg.ConditionId); + // The exception TYPE only: a backend message is not ours to vouch for (it could echo a credential). + ReportAckFailed(replyTo, msg, $"the upstream acknowledge failed ({ex.GetType().Name})"); } } + /// Tells an when the ack names a condition + /// NodeId (lmxopcua#760); a no-op for an ack routed without one. + private static void ReportAckFailed(IActorRef replyTo, RouteAlarmAck msg, string reason) + { + if (string.IsNullOrEmpty(msg.ConditionNodeId) || replyTo.IsNobody()) return; + replyTo.Tell(new AlarmAckFailed(msg.ConditionNodeId, reason)); + } + private async Task HandleSubscribeAsync(Subscribe msg) { // Capture Sender/Self BEFORE any await. The re-subscribe path below awaits diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/NativeAlarmAckPreflightView.cs b/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/NativeAlarmAckPreflightView.cs new file mode 100644 index 00000000..be54db74 --- /dev/null +++ b/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/NativeAlarmAckPreflightView.cs @@ -0,0 +1,50 @@ +using System.Collections.Immutable; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Runtime.Drivers; + +/// +/// The lock-free snapshot the OPC UA node manager consults to refuse, synchronously, a native-condition +/// Acknowledge the owning driver would refuse anyway (lmxopcua#760). owns +/// the condition → driver map and the driver objects, and publishes a fresh immutable snapshot whenever +/// either changes; is a single volatile read, safe under the SDK node-manager lock. +/// +/// +/// Racy by design against a change published a moment later — it covers static misconfiguration only. An +/// ack it lets through can still be refused or fail at the driver; that path reverts the condition to +/// unacknowledged ('s NativeAlarmAckFailed handling). +/// +public sealed class NativeAlarmAckPreflightView +{ + private volatile ImmutableDictionary _byConditionNodeId = + ImmutableDictionary.Empty.WithComparers(StringComparer.Ordinal); + + /// + /// Returns why the driver owning would refuse an acknowledge from a + /// session authenticated with , or null to proceed (unknown condition, a + /// driver without a pre-check, or one that cannot tell without I/O). + /// + /// The bare condition NodeId identifier the node manager routes by. + /// The session's API key id, or null for anonymous. + /// The refusal reason, or null. + public string? Check(string conditionNodeId, string? apiKeyId) + { + if (string.IsNullOrEmpty(conditionNodeId)) return null; + if (!_byConditionNodeId.TryGetValue(conditionNodeId, out var preflight)) return null; + try + { + return preflight.GetAcknowledgeRefusal(apiKeyId); + } + catch + { + // A buggy pre-check must never break the Part 9 method path; let the ack proceed (the driver's own + // refusal + the host's revert still apply). + return null; + } + } + + /// Replaces the snapshot. Called by only. + /// Condition NodeId → the owning driver's pre-check. + internal void Publish(IEnumerable> byConditionNodeId) => + _byConditionNodeId = ImmutableDictionary.CreateRange(StringComparer.Ordinal, byConditionNodeId); +} diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/NativeAlarmProjector.cs b/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/NativeAlarmProjector.cs index 902aa764..e93bfa64 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/NativeAlarmProjector.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/NativeAlarmProjector.cs @@ -12,9 +12,16 @@ namespace ZB.MOM.WW.OtOpcUa.Runtime.Drivers; /// public sealed class NativeAlarmProjector { - private readonly Dictionary _prior = + private readonly Dictionary _prior = new(StringComparer.Ordinal); + /// + /// Conditions a client acknowledged locally () whose acknowledge has not yet + /// been confirmed or superseded by a driver transition — the only ones + /// may un-acknowledge (lmxopcua#760). + /// + private readonly HashSet _pendingLocalAck = new(StringComparer.Ordinal); + /// Project an alarm transition onto the full condition snapshot for . /// The materialised condition node's id (the projection's state key). /// The native alarm transition. @@ -71,8 +78,11 @@ public sealed class NativeAlarmProjector /// The full Part 9 condition snapshot to write to the node. public AlarmConditionSnapshot Project(string nodeId, AlarmEventArgs e, out IReadOnlyList historyKinds) { + // Any driver transition supersedes a pending local acknowledge: from here the projected state is the + // driver's word, so a late ack failure must not rewrite it. + _pendingLocalAck.Remove(nodeId); var prev = _prior.TryGetValue(nodeId, out var p) - ? p : (Active: false, Acked: true, Message: (string?)null, Severity: (ushort)0); + ? p : (Active: false, Acked: true, Message: (string?)null, Severity: (ushort)0, StatusCode: OpcUaStatus.Good); // A source with a finer scale than the four buckets supplies the exact Part 9 severity. var sev = e.OpcUaSeverity is { } exact ? (ushort)Math.Clamp(exact, 1, 1000) : MapSeverity(e.Severity); // Only a statement saying the SAME thing the condition already shows is a re-statement. @@ -143,14 +153,15 @@ public sealed class NativeAlarmProjector break; } - _prior[nodeId] = (active, acked, e.Message, sev); + var statusCode = e.QualityStatusCode ?? OpcUaStatus.Good; + _prior[nodeId] = (active, acked, e.Message, sev, statusCode); return new AlarmConditionSnapshot( Active: active, Acknowledged: acked, Confirmed: true, Enabled: true, Shelving: AlarmShelvingKind.Unshelved, Severity: sev, Message: e.Message, // A per-condition source quality (MTConnect Unavailable) when the driver reports one. Absent // (every earlier source) ⇒ Good, as before — driver-wide comms loss stays the host's // out-of-band WriteAlarmQuality annotation, not something the projector infers. - StatusCode: e.QualityStatusCode ?? OpcUaStatus.Good); + StatusCode: statusCode); } /// @@ -161,11 +172,39 @@ public sealed class NativeAlarmProjector /// The acknowledged condition node id. public void MarkAcknowledged(string nodeId) { - if (_prior.TryGetValue(nodeId, out var p)) _prior[nodeId] = p with { Acked = true }; + if (!_prior.TryGetValue(nodeId, out var p)) return; + // Only an ack that CHANGED the projected state is pending; re-acking an acked condition has nothing to undo. + if (!p.Acked) _pendingLocalAck.Add(nodeId); + _prior[nodeId] = p with { Acked = true }; + } + + /// + /// Undo a client acknowledgement the driver refused or failed to deliver upstream (lmxopcua#760), returning + /// the unacknowledged snapshot to re-project onto the node. Only an acknowledge still pending since + /// is reverted: once any driver transition for the condition has been + /// projected, that transition is authoritative and a late failure changes nothing. + /// + /// The condition node id whose acknowledge failed. + /// The unacknowledged snapshot to project; null when nothing was reverted. + /// True when the condition was reverted to unacknowledged. + public bool TryRevertAcknowledge(string nodeId, out AlarmConditionSnapshot? snapshot) + { + snapshot = null; + if (!_pendingLocalAck.Remove(nodeId) || !_prior.TryGetValue(nodeId, out var p)) return false; + _prior[nodeId] = p with { Acked = false }; + snapshot = new AlarmConditionSnapshot( + Active: p.Active, Acknowledged: false, Confirmed: true, Enabled: true, + Shelving: AlarmShelvingKind.Unshelved, Severity: p.Severity, Message: p.Message ?? string.Empty, + StatusCode: p.StatusCode); + return true; } /// Clears all tracked per-node state. - public void Clear() => _prior.Clear(); + public void Clear() + { + _prior.Clear(); + _pendingLocalAck.Clear(); + } /// /// Drop the tracked state of every condition NOT in , keeping @@ -180,7 +219,10 @@ public sealed class NativeAlarmProjector { ArgumentNullException.ThrowIfNull(deployedNodeIds); foreach (var nodeId in _prior.Keys.Where(k => !deployedNodeIds.Contains(k)).ToList()) + { _prior.Remove(nodeId); + _pendingLocalAck.Remove(nodeId); + } } private static ushort MapSeverity(AlarmSeverity s) => s switch diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/ServiceCollectionExtensions.cs b/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/ServiceCollectionExtensions.cs index 6e5c5c4e..498a0eb9 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/ServiceCollectionExtensions.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/ServiceCollectionExtensions.cs @@ -104,6 +104,10 @@ public static class ServiceCollectionExtensions // redundancy manager, wires no HistoryUpdate gate at all. services.TryAddSingleton(sp => sp.GetRequiredService()); + // The condition → owning-driver native-ack pre-check snapshot (lmxopcua#760): DriverHostActor publishes + // it, the OPC UA node manager reads it to refuse an acknowledge the driver would refuse anyway. + services.TryAddSingleton(); + // The node→central ack transport under MeshTransport:Mode = Grpc (mesh gRPC command plane). // Registered unconditionally but resolved only by the Grpc arm of // WithOtOpcUaRuntimeActors — the mode is not knowable here (this runs before configuration is @@ -654,7 +658,9 @@ public static class ServiceCollectionExtensions historyUpdateRoleView: resolver.GetService(), // The node-wide Scripting section (bound + clamped in Program.cs) for the scripted-alarm // engine. Unbound in test graphs ⇒ null ⇒ the compiled defaults. - scripting: resolver.GetService()), + scripting: resolver.GetService(), + // Where the native-ack pre-check snapshot is published (lmxopcua#760). + ackPreflightView: resolver.GetService()), DriverHostActorName); registry.Register(driverHost); diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Tests/Runtime/GalaxyAckCredentialTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Tests/Runtime/GalaxyAckCredentialTests.cs index 14610088..ec932f92 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Tests/Runtime/GalaxyAckCredentialTests.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Tests/Runtime/GalaxyAckCredentialTests.cs @@ -87,8 +87,13 @@ public sealed class GalaxyAckCredentialTests logger.AllText.ShouldNotContain(KeyPassword); } + /// + /// A non-zero native code (MXAccess rejected the ack — e.g. a wrong password) warns AND throws a + /// (lmxopcua#760) so the host reverts the OPC UA condition + /// instead of leaving it acknowledged; neither the log nor the exception carries the password. + /// [Fact] - public void EvaluateReply_native_failure_warns_without_password() + public void EvaluateReply_native_failure_warns_and_refuses_without_password() { var credential = new GalaxyAckCredential("alice", "CORP", KeyPassword, GalaxyAckCredentialSource.ApiKey); var reply = new AcknowledgeAlarmReply @@ -98,13 +103,34 @@ public sealed class GalaxyAckCredentialTests }; var logger = new CapturingLogger(); - GatewayGalaxyAlarmAcknowledger.EvaluateReply(reply, "A.B", credential, logger); + var ex = Should.Throw( + () => GatewayGalaxyAlarmAcknowledger.EvaluateReply(reply, "A.B", credential, logger)); + ex.Message.ShouldContain("0x80070005"); + ex.Message.ShouldContain("CORP\\alice"); + ex.ToString().ShouldNotContain(KeyPassword); logger.Entries.ShouldHaveSingleItem().Level.ShouldBe(LogLevel.Warning); logger.AllText.ShouldContain("CORP\\alice"); logger.AllText.ShouldNotContain(KeyPassword); } + /// A zero native code is success: nothing logged, nothing thrown. + [Fact] + public void EvaluateReply_success_is_silent() + { + var credential = new GalaxyAckCredential("alice", "CORP", KeyPassword, GalaxyAckCredentialSource.ApiKey); + var reply = new AcknowledgeAlarmReply + { + ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok }, + Hresult = 0, + }; + var logger = new CapturingLogger(); + + GatewayGalaxyAlarmAcknowledger.EvaluateReply(reply, "A.B", credential, logger); + + logger.Entries.ShouldBeEmpty(); + } + [Fact] public async Task Acknowledger_transport_failure_exception_text_has_no_password() { @@ -359,6 +385,28 @@ public sealed class GalaxyAckCredentialTests logger.Entries.ShouldContain(e => e.Level == LogLevel.Warning && e.Text.Contains("refused")); } + /// + /// lmxopcua#760 — the synchronous pre-check refuses exactly the case decidable without I/O: an anonymous + /// session on a driver with no fallback account. A keyed session is let through (its identity lives in the + /// Secrets store), and a fallback account makes every session acceptable. + /// + [Fact] + public void Preflight_refuses_only_an_anonymous_session_without_a_fallback() + { + using var noFallback = NewDriver(Opts(), new CredentialRecordingAcknowledger(), new DictSecretResolver(), new CapturingLogger()); + var refusal = noFallback.GetAcknowledgeRefusal(apiKeyId: null); + refusal.ShouldNotBeNull(); + refusal.ShouldContain("FallbackUser"); + refusal.ShouldContain("drv"); + noFallback.GetAcknowledgeRefusal(apiKeyId: " ").ShouldNotBeNull("a blank key id is no key"); + noFallback.GetAcknowledgeRefusal(KeyId).ShouldBeNull("a keyed session may have an identity in the Secrets store"); + + using var withFallback = NewDriver( + Opts() with { AlarmAck = WithFallback() }, new CredentialRecordingAcknowledger(), new DictSecretResolver(), new CapturingLogger()); + withFallback.GetAcknowledgeRefusal(apiKeyId: null).ShouldBeNull(); + withFallback.GetAcknowledgeRefusal(KeyId).ShouldBeNull(); + } + [Fact] public void Annotate_comment_forms() { diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/ConditionMethodEventTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/ConditionMethodEventTests.cs index af18bd20..9c025116 100644 --- a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/ConditionMethodEventTests.cs +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/ConditionMethodEventTests.cs @@ -493,6 +493,63 @@ public sealed class ConditionMethodEventTests finally { await rig.DisposeAsync(); } } + /// + /// lmxopcua#760 — an OPC UA Acknowledge of a NATIVE condition that the owning driver's synchronous pre-check + /// refuses (e.g. an anonymous session on a Galaxy driver with no fallback ack account) fails ON THE WIRE with + /// BadUserAccessDenied carrying the driver's reason: nothing is routed, applied or reported. Once the + /// pre-check has no objection the same call is accepted (the positive control). + /// + [Fact] + public async Task A_native_acknowledge_the_driver_would_refuse_is_refused_synchronously() + { + var ct = TestContext.Current.CancellationToken; + var rig = await StartAsync("CondNativeAckPreflight", encrypted: false, ct); + try + { + rig.Enforce(true, [Key()], ClusterGrant(KeyPrincipal, "acl-key", All)); + rig.MaterialiseNative(); + var current = PreState("Acknowledge", native: true) with { Message = "base" }; + WireSimulatedOwners(rig, native: true, () => current, s => current = s); + var routed = 0; + var driverAck = rig.Nm.NativeAlarmAckRouter!; + rig.Nm.NativeAlarmAckRouter = ack => { Interlocked.Increment(ref routed); driverAck(ack); }; + string? refusal = "no Galaxy ack account is configured for this session"; + rig.Nm.NativeAlarmAckPreflight = _ => refusal; + + using var session = await rig.ConnectAsync(anonymous: false, ct); + var events = new RecordedEvents(); + var subscription = await RecordedEvents.SubscribeAsync(session, events, ct); + var item = RecordedEvents.AddEventItem(subscription, ObjectIds.Server); + await subscription.ApplyChangesAsync(ct); + + rig.Project(native: true, current); + await RecordedEvents.WaitUntilAsync(() => events.Has(item, "base")); + var condition = ConditionNodeId(session, native: true); + + var refused = await Should.ThrowAsync( + CallAsync(session, condition, "Acknowledge", events.LatestEventId(item, "base"), ct)); + refused.StatusCode.ShouldBe(StatusCodes.BadUserAccessDenied); + routed.ShouldBe(0, "a refused ack reaches no driver"); + rig.Nm.TryGetAlarmCondition(RawAlarmPath, AddressSpaceRealm.Raw)!.AckedState.Id.Value + .ShouldBeFalse("a refused ack applies nothing"); + + // Positive control: with nothing to object to, the same call is accepted, routed and reported. + refusal = null; + await CallAsync(session, condition, "Acknowledge", events.LatestEventId(item, "base"), ct); + await rig.DrainProjectionsAsync(); + routed.ShouldBe(1); + + rig.Project(native: true, current = current with { Message = "sentinel" }); + await RecordedEvents.WaitUntilAsync(() => events.Has(item, "sentinel")); + var baseEvents = events.ConditionEvents(item, "base"); + baseEvents.Count.ShouldBe(2, "the base transition plus the accepted ack's event; the refusal reported nothing"); + baseEvents[1][RecordedEvents.AckedField].Value.ShouldBe(true); + + await subscription.DeleteAsync(true, ct); + } + finally { await rig.DisposeAsync(); } + } + /// /// Acknowledging an acknowledged condition WITHOUT a comment reports nothing, and does not flip a scripted /// condition's Confirmed out of step with its engine. With a comment it reports exactly one event carrying diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/AlarmCommandRouterTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/AlarmCommandRouterTests.cs index ff40ea1b..74f5d45c 100644 --- a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/AlarmCommandRouterTests.cs +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/AlarmCommandRouterTests.cs @@ -450,6 +450,47 @@ public sealed class AlarmCommandRouterTests : IDisposable await host.DisposeAsync(); } + /// + /// lmxopcua#760 — when the owning driver's synchronous pre-check refuses the acknowledge, OnAcknowledge + /// returns BadUserAccessDenied carrying the driver's reason, and nothing is routed (so the SDK applies + /// no state change and no event is reported). The pre-check sees the condition NodeId and the session. + /// + [Fact] + public async Task Native_OnAcknowledge_refused_by_preflight_returns_denied_with_reason_and_routes_nothing() + { + var (host, server) = await BootAsync(); + var nm = server.NodeManager!; + + var native = new List(); + var checkedAcks = new List(); + nm.NativeAlarmAckRouter = native.Add; + nm.NativeAlarmAckPreflight = ack => + { + checkedAcks.Add(ack); + return "no Galaxy ack account for an anonymous session"; + }; + + nm.EnsureFolder("eq-nak5", parentNodeId: null, displayName: "Equipment NAK5", realm: AddressSpaceRealm.Uns); + nm.MaterialiseAlarmCondition("alm-nak5", "eq-nak5", "HighTemp", "OffNormalAlarm", severity: 700, isNative: true, realm: AddressSpaceRealm.Uns); + var condition = nm.TryGetAlarmCondition("alm-nak5"); + condition.ShouldNotBeNull(); + + var ctx = SessionContext(server, "quinn", OpcUaDataPlaneRoles.AlarmAck); + var result = condition!.OnAcknowledge!(ctx, condition, EventIdBytes(), new LocalizedText("ack")); + + result.StatusCode.Code.ShouldBe(StatusCodes.BadUserAccessDenied); + result.LocalizedText.Text.ShouldBe("no Galaxy ack account for an anonymous session"); + native.ShouldBeEmpty(); + checkedAcks.ShouldHaveSingleItem().ConditionNodeId.ShouldBe("alm-nak5"); + + // Positive control: the same ack goes through once the pre-check has nothing to object to. + nm.NativeAlarmAckPreflight = _ => null; + condition.OnAcknowledge!(ctx, condition, EventIdBytes(), new LocalizedText("ack")).ShouldBe(ServiceResult.Good); + native.ShouldHaveSingleItem(); + + await host.DisposeAsync(); + } + /// H6c — a SCRIPTED condition's Acknowledge still routes through the scripted /// (Operation == "Acknowledge"); the native /// router is NOT invoked. diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/DriverHostActorNativeAlarmAckRoutingTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/DriverHostActorNativeAlarmAckRoutingTests.cs index 3be2afc9..23c7db40 100644 --- a/tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/DriverHostActorNativeAlarmAckRoutingTests.cs +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/DriverHostActorNativeAlarmAckRoutingTests.cs @@ -1,6 +1,7 @@ using System.Collections.Concurrent; using System.Text.Json; using Akka.Actor; +using Akka.TestKit; using Microsoft.EntityFrameworkCore; using Shouldly; using Xunit; @@ -8,12 +9,14 @@ using ZB.MOM.WW.OtOpcUa.Commons.Engines; using ZB.MOM.WW.OtOpcUa.Commons.Messages.Deploy; using ZB.MOM.WW.OtOpcUa.Commons.Messages.Fleet; using ZB.MOM.WW.OtOpcUa.Commons.Messages.Redundancy; +using ZB.MOM.WW.OtOpcUa.Commons.OpcUa; using ZB.MOM.WW.OtOpcUa.Commons.Types; using ZB.MOM.WW.OtOpcUa.Configuration; using ZB.MOM.WW.OtOpcUa.Configuration.Entities; using ZB.MOM.WW.OtOpcUa.Configuration.Enums; using ZB.MOM.WW.OtOpcUa.Core.Abstractions; using ZB.MOM.WW.OtOpcUa.Runtime.Drivers; +using ZB.MOM.WW.OtOpcUa.Runtime.OpcUa; using ZB.MOM.WW.OtOpcUa.Runtime.Tests.Harness; namespace ZB.MOM.WW.OtOpcUa.Runtime.Tests.Drivers; @@ -144,17 +147,140 @@ public sealed class DriverHostActorNativeAlarmAckRoutingTests : RuntimeActorTest recorder.Acks.Count.ShouldBe(1); } + /// + /// lmxopcua#760 — the driver REFUSES the acknowledge (no Galaxy ack credential): the host reverts the + /// condition the node manager optimistically acknowledged, re-projecting it Active + UNacknowledged with + /// its message and severity intact, so the address space stops claiming an ack Galaxy never received. + /// + [Fact] + public void A_refused_ack_reverts_the_condition_to_unacknowledged() + { + var db = NewInMemoryDbFactory(); + var recorder = new RecordingAlarmDriverFactory("GalaxyMxGateway") + { + AckFault = new DriverOperationRefusedException("no Galaxy ack credential"), + }; + var deploymentId = SeedV3AlarmDeployment(db, RevA, Driver: "drv-1", Tag: "temp_hi"); + var publish = this.CreateCleanOutcomePublishProbe(); + var actor = SpawnHostAndApply(db, deploymentId, recorder, publish.Ref); + + RaiseAndExpectUnacked(actor, publish); + + actor.Tell(new DriverHostActor.RouteNativeAlarmAck(AlarmRawPath, "cmt", "alice")); + + var reverted = NextAlarmUpdate(publish); + reverted.AlarmNodeId.ShouldBe(AlarmRawPath); + reverted.Realm.ShouldBe(AddressSpaceRealm.Raw); + reverted.State.Active.ShouldBeTrue(); + reverted.State.Acknowledged.ShouldBeFalse(); + reverted.State.Message.ShouldBe("temperature high"); + reverted.State.Severity.ShouldBe((ushort)700); + recorder.Acks.Count.ShouldBe(1, "the ack did reach the driver, which refused it"); + } + + /// + /// lmxopcua#760 — any backend failure (here: the gateway rejecting the ack) reverts the same way, not just + /// a deliberate refusal. + /// + [Fact] + public void A_failed_ack_reverts_the_condition_to_unacknowledged() + { + var db = NewInMemoryDbFactory(); + var recorder = new RecordingAlarmDriverFactory("GalaxyMxGateway") + { + AckFault = new InvalidOperationException("gateway hresult 0x80070005"), + }; + var deploymentId = SeedV3AlarmDeployment(db, RevA, Driver: "drv-1", Tag: "temp_hi"); + var publish = this.CreateCleanOutcomePublishProbe(); + var actor = SpawnHostAndApply(db, deploymentId, recorder, publish.Ref); + + RaiseAndExpectUnacked(actor, publish); + actor.Tell(new DriverHostActor.RouteNativeAlarmAck(AlarmRawPath, "cmt", "alice")); + + NextAlarmUpdate(publish).State.Acknowledged.ShouldBeFalse(); + } + + /// + /// lmxopcua#760 — a SUCCESSFUL ack reverts nothing. Positive control: the next driver Clear projects + /// inactive + ACKED, which it only can when the local ack stayed in the projector (a revert would have + /// left it unacked) — and it must be the very next condition update, so no revert slipped in between. + /// + [Fact] + public void A_successful_ack_is_not_reverted() + { + var db = NewInMemoryDbFactory(); + var recorder = new RecordingAlarmDriverFactory("GalaxyMxGateway"); + var deploymentId = SeedV3AlarmDeployment(db, RevA, Driver: "drv-1", Tag: "temp_hi"); + var publish = this.CreateCleanOutcomePublishProbe(); + var actor = SpawnHostAndApply(db, deploymentId, recorder, publish.Ref); + + RaiseAndExpectUnacked(actor, publish); + actor.Tell(new DriverHostActor.RouteNativeAlarmAck(AlarmRawPath, "cmt", "alice")); + AwaitAssert(() => recorder.Acks.Count.ShouldBe(1), duration: Timeout); + + actor.Tell(new DriverInstanceActor.AttributeAlarmPublished("drv-1", Evt(AlarmTransitionKind.Clear))); + var next = NextAlarmUpdate(publish); + next.State.Active.ShouldBeFalse(); + next.State.Acknowledged.ShouldBeTrue("the successful ack must have survived — no revert"); + } + + /// + /// lmxopcua#760 — the host publishes the condition → owning-driver pre-check snapshot the OPC UA node + /// manager uses to refuse an acknowledge synchronously: the driver's own verdict for the condition it + /// owns, nothing for an unknown condition. + /// + [Fact] + public void The_ack_preflight_view_answers_with_the_owning_drivers_verdict() + { + var db = NewInMemoryDbFactory(); + var recorder = new RecordingAlarmDriverFactory("GalaxyMxGateway") + { + PreflightRefusal = key => key is null ? "anonymous sessions cannot acknowledge" : null, + }; + var deploymentId = SeedV3AlarmDeployment(db, RevA, Driver: "drv-1", Tag: "temp_hi"); + var view = new NativeAlarmAckPreflightView(); + SpawnHostAndApply(db, deploymentId, recorder, ackPreflightView: view); + + AwaitAssert(() => view.Check(AlarmRawPath, apiKeyId: null).ShouldBe("anonymous sessions cannot acknowledge"), + duration: Timeout); + view.Check(AlarmRawPath, apiKeyId: "0a1b2c").ShouldBeNull(); + view.Check("Plant/gw/dev1/does-not-exist", apiKeyId: null).ShouldBeNull(); + } + + private static AlarmEventArgs Evt(AlarmTransitionKind kind) => new( + new StubAlarmHandleForTests(), SourceNodeId: "Temp", ConditionId: AlarmRawPath, AlarmType: "OffNormalAlarm", + Message: "temperature high", Severity: AlarmSeverity.High, SourceTimestampUtc: DateTime.UtcNow, Kind: kind); + + private void RaiseAndExpectUnacked(IActorRef actor, TestProbe publish) + { + actor.Tell(new DriverInstanceActor.AttributeAlarmPublished("drv-1", Evt(AlarmTransitionKind.Raise))); + var raised = NextAlarmUpdate(publish); + raised.State.Active.ShouldBeTrue(); + raised.State.Acknowledged.ShouldBeFalse(); + } + + private static OpcUaPublishActor.AlarmStateUpdate NextAlarmUpdate(TestProbe publish) => + publish.FishForMessage(_ => true, Timeout); + + private sealed class StubAlarmHandleForTests : IAlarmSubscriptionHandle + { + public string DiagnosticId => "test-alarm-sub"; + } + /// Spawns the host with the recording alarm-driver factory, dispatches the deployment, and waits /// for the Applied ACK so the apply (and thus the inverse-map build in PushDesiredSubscriptions) has /// completed before the test routes an ack. private IActorRef SpawnHostAndApply( - IDbContextFactory db, DeploymentId deploymentId, IDriverFactory factory) + IDbContextFactory db, DeploymentId deploymentId, IDriverFactory factory, + IActorRef? opcUaPublishActor = null, NativeAlarmAckPreflightView? ackPreflightView = null) { var coordinator = CreateTestProbe(); var actor = Sys.ActorOf(DriverHostActor.Props( db, TestNode, coordinator.Ref, driverFactory: factory, - localRoles: new HashSet { "driver" })); + localRoles: new HashSet { "driver" }, + opcUaPublishActor: opcUaPublishActor, + ackPreflightView: ackPreflightView)); actor.Tell(new DispatchDeployment(deploymentId, RevA, CorrelationId.NewId())); coordinator.ExpectMsg(Timeout).Outcome.ShouldBe(ApplyAckOutcome.Applied); @@ -248,6 +374,12 @@ public sealed class DriverHostActorNativeAlarmAckRoutingTests : RuntimeActorTest /// True once the driver child has reached Connected and opened its alarm subscription. public bool AlarmsSubscribed => _driver.AlarmsSubscribed; + /// When set, every acknowledge is recorded and then fails with this exception. + public Exception? AckFault { init => _driver.AckFault = value; } + + /// The driver's synchronous pre-check verdict (lmxopcua#760); null ⇒ always proceed. + public Func? PreflightRefusal { init => _driver.PreflightRefusal = value; } + /// public IDriver? TryCreate(string driverType, string driverInstanceId, string driverConfigJson) { @@ -261,8 +393,17 @@ public sealed class DriverHostActorNativeAlarmAckRoutingTests : RuntimeActorTest } /// An + that records every acknowledge. - private sealed class RecordingAlarmDriver : IDriver, IAlarmSource + private sealed class RecordingAlarmDriver : IDriver, IAlarmSource, IAlarmAcknowledgePreflight { + /// When set, records the request and then throws this. + public Exception? AckFault { get; set; } + + /// The verdict; null ⇒ always proceed. + public Func? PreflightRefusal { get; set; } + + /// + public string? GetAcknowledgeRefusal(string? apiKeyId) => PreflightRefusal?.Invoke(apiKeyId); + private readonly ConcurrentQueue _acks = new(); /// public string DriverInstanceId { get; private set; } = string.Empty; @@ -303,7 +444,7 @@ public sealed class DriverHostActorNativeAlarmAckRoutingTests : RuntimeActorTest IReadOnlyList acknowledgements, CancellationToken cancellationToken) { foreach (var a in acknowledgements) _acks.Enqueue(a); - return Task.CompletedTask; + return AckFault is { } fault ? Task.FromException(fault) : Task.CompletedTask; } /// diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/NativeAlarmProjectorTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/NativeAlarmProjectorTests.cs index 876ad68c..c3deefbd 100644 --- a/tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/NativeAlarmProjectorTests.cs +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/NativeAlarmProjectorTests.cs @@ -530,6 +530,62 @@ public class NativeAlarmProjectorTests snap.Active.ShouldBeTrue(); } + /// + /// lmxopcua#760 — an acknowledge the driver refused reverts the condition: the snapshot keeps the active + /// state, message, severity and source quality and is unacknowledged, and the next driver Clear projects + /// from the reverted (unacked) prior rather than the optimistic ack. + /// + [Fact] + public void TryRevertAcknowledge_reverts_a_pending_local_ack() + { + var sut = new NativeAlarmProjector(); + sut.Project("n1", Evt(AlarmTransitionKind.Raise, AlarmSeverity.Critical, "boom")); + sut.MarkAcknowledged("n1"); + + sut.TryRevertAcknowledge("n1", out var reverted).ShouldBeTrue(); + reverted.ShouldNotBeNull(); + reverted.Active.ShouldBeTrue(); + reverted.Acknowledged.ShouldBeFalse(); + reverted.Severity.ShouldBe((ushort)900); + reverted.Message.ShouldBe("boom"); + reverted.StatusCode.ShouldBe(OpcUaStatus.Good); + + sut.Project("n1", Evt(AlarmTransitionKind.Clear)).Acknowledged.ShouldBeFalse( + "the reverted ack must not come back on the next Clear"); + } + + /// lmxopcua#760 — a revert happens at most once per local ack, and never without one. + [Fact] + public void TryRevertAcknowledge_without_a_pending_ack_does_nothing() + { + var sut = new NativeAlarmProjector(); + sut.TryRevertAcknowledge("untracked", out var none).ShouldBeFalse(); + none.ShouldBeNull(); + + sut.Project("n1", Evt(AlarmTransitionKind.Raise)); + sut.TryRevertAcknowledge("n1", out _).ShouldBeFalse("nothing was acknowledged locally"); + + sut.MarkAcknowledged("n1"); + sut.TryRevertAcknowledge("n1", out _).ShouldBeTrue(); + sut.TryRevertAcknowledge("n1", out _).ShouldBeFalse("a second failure report must not revert twice"); + } + + /// + /// lmxopcua#760 — once a driver transition for the condition has been projected after the local ack, that + /// transition is authoritative: a late failure report does not rewrite it. + /// + [Fact] + public void A_driver_transition_after_the_ack_supersedes_a_late_failure() + { + var sut = new NativeAlarmProjector(); + sut.Project("n1", Evt(AlarmTransitionKind.Raise)); + sut.MarkAcknowledged("n1"); + sut.Project("n1", Evt(AlarmTransitionKind.Acknowledge)).Acknowledged.ShouldBeTrue(); + + sut.TryRevertAcknowledge("n1", out var snapshot).ShouldBeFalse(); + snapshot.ShouldBeNull(); + } + private static AlarmEventArgs Evt( AlarmTransitionKind kind, AlarmSeverity sev = AlarmSeverity.High,