feat(alarms): bridge driver comms-loss into scripted-alarm input quality (#481)

Layer 4 of the condition-Quality chain (#477 -> #478). #478 gave a scripted condition the worst quality of its input tags, but only for inputs whose driver PUBLISHES a Bad data change. A poll driver whose device goes unreachable publishes nothing — it raises ConnectivityChanged(false) and goes silent — so the engine held its last Good value, WorstInputStatus stayed Good, and worse, AreInputsReady stayed true, so the predicate went on evaluating a stale value after the source had died. `DriverHostActor.FanCommsLossToDependencyMux` pushes a synthetic BadNoCommunication with a null value into the dependency mux for every value ref the disconnected driver owns. The issue asked for a driver -> all-tag-refs map to be built for this. It already exists: `_nodeIdByDriverRef`, keyed (DriverInstanceId, RawPath) and rebuilt from the composition on every apply. Alarm-bearing tags live in the separate `_alarmNodeIdByDriverRef` and are deliberately excluded — a native condition is not a mux value and takes the Part 9 quality path instead. Nothing is pushed on reconnect, deliberately: publishing Good would assert a value quality this actor has not observed. Recovery rides the driver's own republish, and the mechanism was verified rather than assumed — the Connected -> Reconnecting edge calls DetachSubscription, which nulls the subscription handle, so the reconnect's ResubscribeDesired builds a FRESH subscription whose PollGroupEngine initial push runs with forceRaise: true. Without that detach the poll engine would compare against its retained LastValues, see an unchanged value, raise nothing, and strand these refs Bad forever — the value-dedup trap that bit the v3 Batch 4 virtual-tag reassert. Closes #478 Finding 3, which the above depends on. WorstInputStatus skipped every input whose VALUE was null as "no data yet" — and Bad-with-no-value is exactly the shape a comms-lost driver produces, so the skip was hiding the case that matters most; without this half the fan-out above would be silently inert. The upstream source now stamps a never-pushed path BadWaitingForInitialData — the honest Part 4 code for that state — and the engine skips that specific code rather than every null value. Deleting the null-skip outright is the obvious fix and is wrong in the other direction: it makes every not-yet-published input drag its condition Bad at deploy, flooding the quality path with load-time annotations, which is what the skip existed to prevent. Both mistakes are pinned by a test pair that goes red under the opposite error, and both were falsified by reverting the production code: old null-skip restored -> A_pushed_Bad_carrying_no_value RED null-skip deleted outright -> An_input_that_was_never_pushed RED fan-out call removed -> the three DriverHostActor tests RED (1 of 7) The never-pushed test was vacuous on first writing and is worth recording: the quality bucket only emits on CHANGE, so the load-time evaluation is the one that flashes Bad, and subscribing to OnEvent after LoadAsync made the assertion unfalsifiable — it passed against the broken behaviour. It now subscribes first. `FakeUpstream.ReadTag` returned 0x80340000 (BadNotConnected) for an unset path, a different Bad code from production's placeholder. Harmless while any Bad behaved alike; load-bearing now, so it is aligned and pinned by a reflection guard asserting the engine's private constant agrees with what the production source stamps. Behaviour change to weigh, flagged in the issue and confirmed: virtual tags computed from a comms-lost input now go Bad too. That follows from using the shared mux and reads as correct, but it is broader than scripted alarms. Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-07-31 09:21:11 -04:00
parent b815446940
commit c81cef39f5
8 changed files with 360 additions and 26 deletions
@@ -1009,9 +1009,24 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers
    ///         <c>alerts</c> row is published here — driver comms health has its own status surface
    ///         (<see cref="IDriverHealthPublisher"/>); a row per condition would be alarm-fatigue.
    ///     </para>
+    ///     <para>
+    ///         <b>#481 (Layer 4) — the same transition also reaches SCRIPTED alarms, via the dependency
+    ///         mux.</b> #478 gave a scripted condition the worst quality of its input tags, but only for
+    ///         inputs whose driver actually <i>publishes</i> a Bad data change. A poll driver whose device
+    ///         goes unreachable publishes nothing at all — it emits this signal and goes silent — so the
+    ///         engine kept its last Good value, <c>WorstInputStatus</c> stayed Good, and worse,
+    ///         <c>AreInputsReady</c> stayed true, so the predicate went on evaluating a stale value after
+    ///         the source had died. <see cref="FanCommsLossToDependencyMux"/> closes that by pushing a
+    ///         synthetic Bad for every value ref the driver owns.
+    ///     </para>
    /// </summary>
    private void OnDriverConnectivityChanged(DriverInstanceActor.ConnectivityChanged msg)
    {
+        // #481: the mux fan-out is independent of the OPC UA sink — a node with scripted alarms but no
+        // publish actor (the mux-only test/dev path) must still see comms loss. Done before the early
+        // return for exactly that reason.
+        if (!msg.Connected) FanCommsLossToDependencyMux(msg.DriverInstanceId);
+
        if (_opcUaPublishActor is null) return;

        // #519: name the actual cause. This is a connectivity transition, so the Bad case is
@@ -1038,6 +1053,68 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers
                _localNode, msg.DriverInstanceId, msg.Connected ? "connected" : "disconnected", annotated, statusCode);
    }

+    /// <summary>
+    ///     #481 (Layer 4) — push a synthetic <see cref="OpcUaStatus.BadNoCommunication"/> into the dependency
+    ///     mux for every value ref the just-disconnected driver owns, so scripted alarms and virtual tags see
+    ///     the source die instead of holding its last Good value indefinitely.
+    /// </summary>
+    /// <remarks>
+    ///     <para>
+    ///         <b>The ref set already existed.</b> #481 asked for a driver → all-tag-refs map to be built;
+    ///         <see cref="_nodeIdByDriverRef"/> is exactly that, keyed <c>(DriverInstanceId, RawPath)</c> and
+    ///         rebuilt from the composition's <c>RawTags</c> ∪ <c>UnsReferenceVariables</c> on every apply.
+    ///         Alarm-bearing tags live in the separate <see cref="_alarmNodeIdByDriverRef"/> and are
+    ///         deliberately absent here — a native condition is not a mux value, and its comms quality is
+    ///         annotated by the caller through the Part 9 path instead.
+    ///     </para>
+    ///     <para>
+    ///         <b>Value <c>null</c>, not the last-known value.</b> The honest statement is "there is no
+    ///         current value", and the engine's <c>AreInputsReady</c> gate holds the predicate on any Bad
+    ///         regardless, so no script observes the null. It does mean the Bad must survive
+    ///         <c>WorstInputStatus</c>, which until #481 skipped every null-valued entry as cold start —
+    ///         hence the paired change stamping never-pushed paths <c>BadWaitingForInitialData</c> so the two
+    ///         are tellable apart. Without that half this fan-out would be silently inert.
+    ///     </para>
+    ///     <para>
+    ///         <b>Nothing is pushed on reconnect, deliberately.</b> Publishing Good would assert a value
+    ///         quality this actor has not observed. Recovery rides the driver's own republish:
+    ///         <c>DriverInstanceActor</c> calls <c>DetachSubscription</c> on the Connected → Reconnecting
+    ///         edge, which nulls the subscription handle, so the reconnect's <c>ResubscribeDesired</c> builds
+    ///         a <i>fresh</i> subscription whose <c>PollGroupEngine</c> initial push runs with
+    ///         <c>forceRaise: true</c> and re-publishes every tag with its real quality. That detach is what
+    ///         makes recovery work: without it the poll engine would compare against its retained
+    ///         <c>LastValues</c>, see an unchanged Good value, raise nothing, and leave these refs Bad
+    ///         forever — the value-dedup trap that bit the v3 Batch 4 virtual-tag reassert.
+    ///     </para>
+    ///     <para>
+    ///         Ungated by redundancy role, for the same reason as the native-condition annotation above: a
+    ///         Secondary must keep its scripted-alarm evaluation coherent for failover. The mux fan-out
+    ///         reaches only refs something is actually subscribed to, so a driver whose tags feed no script
+    ///         and no virtual tag costs one dictionary scan and no delivery.
+    ///     </para>
+    /// </remarks>
+    /// <param name="driverInstanceId">The driver child that just reported comms loss.</param>
+    private void FanCommsLossToDependencyMux(string driverInstanceId)
+    {
+        if (_dependencyMux is null) return;
+
+        var ts = DateTime.UtcNow;
+        var pushed = 0;
+        foreach (var (driverId, rawPath) in _nodeIdByDriverRef.Keys)
+        {
+            if (driverId != driverInstanceId) continue;
+
+            _dependencyMux.Tell(new DriverInstanceActor.AttributeValuePublished(
+                driverInstanceId, rawPath, Value: null, OpcUaStatus.BadNoCommunication, ts));
+            pushed++;
+        }
+
+        if (pushed > 0)
+            _log.Debug(
+                "DriverHost {Node}: driver {Driver} disconnected — pushed BadNoCommunication for {Count} mux ref(s)",
+                _localNode, driverInstanceId, pushed);
+    }
+
    private void ForwardNativeAlarm(DriverInstanceActor.AttributeAlarmPublished msg)
    {
        if (_opcUaPublishActor is null) return;
@@ -1,5 +1,6 @@
 using System.Collections.Concurrent;
 using System.Collections.Immutable;
+using ZB.MOM.WW.OtOpcUa.Commons.OpcUa;
 using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
 using ZB.MOM.WW.OtOpcUa.Core.ScriptedAlarms;

@@ -26,11 +27,20 @@ namespace ZB.MOM.WW.OtOpcUa.Runtime.ScriptedAlarms;
 /// </remarks>
 public sealed class DependencyMuxTagUpstreamSource : ITagUpstreamSource
 {
-    // OPC UA Part 4 StatusCode for an outright Bad value (severity 10, bit 31 set). The
-    // codebase has no shared Core.Abstractions constant — concrete producers (GalaxyDriver,
-    // the Wonderware historian client) inline this same 0x80000000u, and the engine's own
-    // AreInputsReady gate tests exactly this bit — so an "unknown path" snapshot uses it too.
-    private const uint StatusBad = 0x80000000u;
+    // The status an "unknown path" snapshot carries — a path nothing has ever pushed to.
+    //
+    // This is deliberately BadWaitingForInitialData rather than the bare Bad (0x80000000u) it used
+    // to be, and the distinction is load-bearing (#481, closing #478 Finding 3). Both codes are Bad,
+    // so the engine's AreInputsReady gate treats them identically and the cold-start hold is
+    // unchanged. What changed is that "no data has ever arrived" is now tellable apart from "a
+    // driver pushed Bad", which previously looked identical whenever the pushed Bad carried a null
+    // value — the shape a comms-lost driver produces. WorstInputStatus keys off exactly this code to
+    // decide which of the two it is looking at.
+    //
+    // The code is also the honest one: Part 4 defines BadWaitingForInitialData as precisely this
+    // state, and it is what a freshly materialised OPC UA variable already reports before its first
+    // publish.
+    private const uint StatusNeverPushed = OpcUaStatus.BadWaitingForInitialData;

    // Intentionally never pruned: cold-start semantics depend on retained last values so
    // ReadTag can answer synchronously for any path that has ever been pushed.
@@ -90,9 +100,9 @@ public sealed class DependencyMuxTagUpstreamSource : ITagUpstreamSource

        // No value has been pushed for this path yet — return a Bad-quality placeholder so
        // the engine's cold-start guard holds the prior condition rather than NRE-ing on a
-        // null value cast.
+        // null value cast. See StatusNeverPushed for why the specific code matters.
        var now = _clock();
-        return new DataValueSnapshot(Value: null, StatusCode: StatusBad, SourceTimestampUtc: null, ServerTimestampUtc: now);
+        return new DataValueSnapshot(Value: null, StatusCode: StatusNeverPushed, SourceTimestampUtc: null, ServerTimestampUtc: now);
    }

    // No-replay contract: a new subscriber does NOT receive a synthetic initial