fix(alarms): native ack refused off-Primary, commented re-ack reported, real UnshelveTime (#597)

- Native-condition Acknowledge on a non-Primary node returns BadInvalidState (nothing
  routed/applied/reported) via NativeAlarmAckPrimaryGate, fed from
  IDriverPrimaryRoleSource by OtOpcUaServerHostedService.
- A re-ack carrying a comment reports one comment event (Acked/Confirmed unchanged).
- TimedShelve applies its requested duration and the scripted projection carries
  UnshelveAtUtc, so ShelvingState/UnshelveTime counts down instead of ~24.8 days.
- Native re-ack still routed to the driver (deliberate: documented retry path), pinned.
- AdminUI /alerts client ack/shelve path test; docs/AlarmTracking.md decisions incl.
  accepted SDK EventsMonitored residual (item 6).
- Also #599/03/S20: RemoveEquipmentSubtree severs inverse Organizes edges.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
This commit is contained in:
Joseph Doherty
2026-10-08 07:12:29 -04:00
parent 0bd3ec00e4
commit bd63810645
10 changed files with 459 additions and 51 deletions
@@ -205,6 +205,26 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
/// </summary>
public Action<NativeAlarmAck>? NativeAlarmAckRouter { get; set; }
/// <summary>
/// Gitea #597 — whether this node may accept an OPC UA Acknowledge of a NATIVE (driver-fed) condition
/// right now: the redundancy data-plane verdict (<c>PrimaryGatePolicy.ShouldServiceAsPrimary</c>, as
/// published to <c>IDriverPrimaryRoleSource</c>) — the same verdict <c>DriverHostActor</c> gates the
/// upstream ack route and device writes on.
/// <para>
/// <c>false</c> ⇒ the Acknowledge is REFUSED with <c>BadInvalidState</c> before anything is routed or
/// applied. Without this the Secondary applied and reported the ack locally while
/// <c>DriverHostActor</c> dropped the route upstream, so the driver never acknowledged and its next
/// transition reverted the node — the client saw acked → unacked. Scripted conditions are not gated:
/// their engine runs on both nodes, so an ack against the Secondary is real.
/// </para>
/// <para>
/// Read under <c>Lock</c> on the SDK's Call thread, so it MUST be a cheap, non-blocking read (the
/// host passes a volatile-backed property). Null (the default) allows — the single-node posture, the
/// same default <c>UnmanagedDriverPrimaryRoleSource</c> answers with.
/// </para>
/// </summary>
public Func<bool>? NativeAlarmAckPrimaryGate { get; set; }
private volatile IOpcUaNodeWriteGateway _nodeWriteGateway = NullOpcUaNodeWriteGateway.Instance;
/// <summary>
@@ -849,13 +869,16 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
if (condition.ShelvingState is not null)
{
// SetShelvingState(shelved, oneShot, shelvingTime): map our 3-way kind onto the SDK's
// (shelved, oneShot) flag pair. Timed shelving's expiry is owned by the engine, not the
// SDK timer, so we pass shelvingTime=0 (no SDK-managed auto-unshelve).
// (shelved, oneShot) flag pair. Timed shelving's expiry is owned by the engine; the time
// passed here is the engine's remaining time, so the node's UnshelveTime counts down to the
// real expiry (Gitea #597 — a 0 here reads as MaxTimeShelved, ~24.8 days). The SDK timer it
// arms only routes an Unshelve back to the engine (OnTimedUnshelve), a no-op once the
// engine's own expiry has run.
condition.SetShelvingState(
SystemContext,
shelved: state.Shelving != AlarmShelvingKind.Unshelved,
oneShot: state.Shelving == AlarmShelvingKind.OneShot,
shelvingTime: 0);
shelvingTime: state.Shelving == AlarmShelvingKind.Timed ? RemainingShelveMs(state.UnshelveAtUtc) : 0);
}
condition.SetSeverity(SystemContext, MapSeverity(state.Severity));
condition.Message.Value = new LocalizedText(state.Message);
@@ -909,6 +932,22 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
}
}
/// <summary>
/// Gitea #597 — the <c>shelvingTime</c> (ms) to hand <c>SetShelvingState</c> for a timed shelve expiring at
/// <paramref name="unshelveAtUtc"/>. <c>0</c> when the expiry is unknown (the SDK then uses
/// <c>MaxTimeShelved</c>, the pre-#597 behaviour); otherwise the remaining time, clamped to at least 1 ms
/// (an already-passed expiry must not read as "unknown" — the engine's own expiry is about to land) and to
/// at most <c>int.MaxValue</c> (the SDK timer's range).
/// </summary>
/// <param name="unshelveAtUtc">The engine's expiry instant, or null.</param>
/// <returns>The shelving time in milliseconds.</returns>
internal static double RemainingShelveMs(DateTime? unshelveAtUtc)
{
if (unshelveAtUtc is not { } at) return 0;
var remaining = (at - DateTime.UtcNow).TotalMilliseconds;
return Math.Clamp(Math.Ceiling(remaining), 1, int.MaxValue);
}
/// <summary>
/// #477 Layer 2 — apply a source-data quality annotation to a materialised condition, OUT OF BAND
/// from any alarm transition. Used by the driver-connectivity path (comms lost → BadNoCommunication,
@@ -1545,9 +1584,10 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
/// </para>
/// <para>
/// <b>Shelving is applied here.</b> The SDK's shelve methods never change <c>ShelvingState</c>
/// themselves, so an accepted shelve call applies it — with the same <c>(shelved, oneShot, 0)</c>
/// mapping <see cref="WriteAlarmCondition"/> projects — before reporting. The engine's re-projection of
/// the shelve then finds no delta. Every other method's state the SDK has already applied.
/// themselves, so an accepted shelve call applies it — with the same <c>(shelved, oneShot)</c> mapping
/// <see cref="WriteAlarmCondition"/> projects, and a TimedShelve with its requested duration so
/// <c>UnshelveTime</c> is real (Gitea #597) — before reporting. The engine's re-projection of the shelve
/// then finds no delta. Every other method's state the SDK has already applied.
/// </para>
/// <para>
/// <b>Exactly once.</b> The whole call runs under <c>Lock</c>. The method handlers hand the command
@@ -1569,7 +1609,10 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
/// the SDK handler, and nothing is reported when it did not move. The SDK accepts acknowledging an
/// acknowledged condition and sets <c>Confirmed = false</c> again; a re-ack therefore also restores the
/// pre-call Confirmed and Retain, so a scripted node never drifts from its engine (which ignores a
/// re-ack) and the call is silent. AddComment always reports: adding a comment is itself the change.
/// re-ack) and the call is silent — unless it carried a comment, which the SDK writes and which is then
/// reported once like AddComment (Gitea #597). AddComment always reports: adding a comment is itself
/// the change. A NATIVE re-ack is still routed to the driver: the route is fire-and-forget, so a node
/// showing Acked does not prove the upstream system acknowledged, and re-acking is the documented retry.
/// </para>
/// <para>
/// A rejected call (bad <see cref="ServiceResult"/>, e.g. a denied gate, or a refused method on a native
@@ -1582,8 +1625,12 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
if (alarm.Acknowledge?.OnCall is { } acknowledge)
{
// A commented acknowledge always reports (Gitea #597): the SDK writes the comment even on a re-ack,
// and a new comment is a Part 9 change in its own right — exactly as AddComment always reports. An
// uncommented re-ack still changes nothing and stays silent.
alarm.Acknowledge.OnCall = (context, m, objectId, eventId, comment) =>
ReportAfterMethod(() => acknowledge(context, m, objectId, eventId, comment), apply: null, isAcknowledge: true);
ReportAfterMethod(() => acknowledge(context, m, objectId, eventId, comment), apply: null, isAcknowledge: true,
alwaysReport: !LocalizedText.IsNullOrEmpty(comment));
}
WrapComment(alarm.Confirm, alwaysReport: false);
@@ -1597,11 +1644,13 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
WrapGeneric(shelving.Unshelve, () => alarm.SetShelvingState(SystemContext, shelved: false, oneShot: false, shelvingTime: 0));
if (shelving.TimedShelve is { OnCall: { } timed } timedShelve)
{
// shelvingTime 0, as WriteAlarmCondition projects it: the scripted engine owns the expiry and
// re-projects Unshelved when it passes (the SDK's own timer only routes an Unshelve back to it).
// The requested duration, so UnshelveTime counts down to the real expiry (Gitea #597 — 0 reads as
// MaxTimeShelved, ~24.8 days). The scripted engine still owns the expiry and re-projects Unshelved
// when it passes (with its own remaining time meanwhile — WriteAlarmCondition); the SDK timer this
// arms only routes an Unshelve back to it (OnTimedUnshelve), a no-op once the engine has expired it.
timedShelve.OnCall = (context, method, objectId, shelvingTime) => ReportAfterMethod(
() => timed(context, method, objectId, shelvingTime),
() => alarm.SetShelvingState(SystemContext, shelved: true, oneShot: false, shelvingTime: 0));
() => alarm.SetShelvingState(SystemContext, shelved: true, oneShot: false, shelvingTime: shelvingTime));
}
}
@@ -1729,7 +1778,8 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
/// folder-scoped condition node id the driver-bound router resolves back to a driver ref.</param>
/// <param name="comment">The acknowledge comment text, or <c>null</c> when none was supplied.</param>
/// <returns><c>ServiceResult.Good</c> when allowed (the SDK then applies state; the method path reports the event);
/// <c>BadUserAccessDenied</c> when the gate vetoes (no route, no state mutation).</returns>
/// <c>BadUserAccessDenied</c> when the gate vetoes, <c>BadInvalidState</c> when this node is not the redundancy
/// Primary (<see cref="NativeAlarmAckPrimaryGate"/>, Gitea #597) — in both cases no route, no state mutation.</returns>
private ServiceResult HandleNativeAlarmAck(ISystemContext context, ConditionState condition, LocalizedText? comment)
{
// Resolve + gate the SAME way HandleAlarmCommand does so native and scripted acks share one authz
@@ -1738,6 +1788,16 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
var gate = EvaluateAlarmCommandGate(identity, condition, "Acknowledge");
if (gate is not null) return gate;
// Gitea #597 — PRIMARY GATE, after authorization (a caller who may not ack learns nothing about the
// node's role). Only the Primary routes a native ack upstream (DriverHostActor applies the same verdict
// to the route), so on any other node the ack is refused outright rather than applied locally and then
// reverted by the driver's next transition. Nothing is routed, applied or reported.
if (NativeAlarmAckPrimaryGate is { } primary && !primary())
{
return new ServiceResult(StatusCodes.BadInvalidState,
new LocalizedText("This server is not the redundancy Primary; acknowledge the alarm on the Primary (the server with the higher ServiceLevel)."));
}
// Same condition-node-id extraction HandleAlarmCommand uses (condition.NodeId.Identifier?.ToString()):
// for a native condition this is the folder-scoped NodeId string the DriverHostActor inverse map
// (_alarmNodeIdByDriverRef value set) keys by, so the next task can resolve it back to (driver, ref).
@@ -3046,6 +3106,10 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
{
if (_variables.TryRemove(id, out var v))
{
// Gitea #599 (archreview 03/S20): sever the inverse half of each cross-tree Organizes edge
// first, exactly as RemoveVariableNode does — a UNS reference variable in scope points at a
// RAW node that survives this UNS-scoped removal and would otherwise keep a dangling edge.
SeverOutgoingReferences(v);
v.Parent?.RemoveChild(v);
PredefinedNodes?.Remove(v.NodeId);
}