feat(alarms): bridge driver comms-loss into scripted-alarm input quality (#481)

Layer 4 of the condition-Quality chain (#477 -> #478). #478 gave a scripted
condition the worst quality of its input tags, but only for inputs whose driver
PUBLISHES a Bad data change. A poll driver whose device goes unreachable
publishes nothing — it raises ConnectivityChanged(false) and goes silent — so
the engine held its last Good value, WorstInputStatus stayed Good, and worse,
AreInputsReady stayed true, so the predicate went on evaluating a stale value
after the source had died.

`DriverHostActor.FanCommsLossToDependencyMux` pushes a synthetic
BadNoCommunication with a null value into the dependency mux for every value ref
the disconnected driver owns.

The issue asked for a driver -> all-tag-refs map to be built for this. It
already exists: `_nodeIdByDriverRef`, keyed (DriverInstanceId, RawPath) and
rebuilt from the composition on every apply. Alarm-bearing tags live in the
separate `_alarmNodeIdByDriverRef` and are deliberately excluded — a native
condition is not a mux value and takes the Part 9 quality path instead.

Nothing is pushed on reconnect, deliberately: publishing Good would assert a
value quality this actor has not observed. Recovery rides the driver's own
republish, and the mechanism was verified rather than assumed — the
Connected -> Reconnecting edge calls DetachSubscription, which nulls the
subscription handle, so the reconnect's ResubscribeDesired builds a FRESH
subscription whose PollGroupEngine initial push runs with forceRaise: true.
Without that detach the poll engine would compare against its retained
LastValues, see an unchanged value, raise nothing, and strand these refs Bad
forever — the value-dedup trap that bit the v3 Batch 4 virtual-tag reassert.

Closes #478 Finding 3, which the above depends on. WorstInputStatus skipped
every input whose VALUE was null as "no data yet" — and Bad-with-no-value is
exactly the shape a comms-lost driver produces, so the skip was hiding the case
that matters most; without this half the fan-out above would be silently inert.
The upstream source now stamps a never-pushed path BadWaitingForInitialData —
the honest Part 4 code for that state — and the engine skips that specific code
rather than every null value.

Deleting the null-skip outright is the obvious fix and is wrong in the other
direction: it makes every not-yet-published input drag its condition Bad at
deploy, flooding the quality path with load-time annotations, which is what the
skip existed to prevent. Both mistakes are pinned by a test pair that goes red
under the opposite error, and both were falsified by reverting the production
code:

  old null-skip restored     -> A_pushed_Bad_carrying_no_value      RED
  null-skip deleted outright -> An_input_that_was_never_pushed      RED
  fan-out call removed       -> the three DriverHostActor tests     RED (1 of 7)

The never-pushed test was vacuous on first writing and is worth recording: the
quality bucket only emits on CHANGE, so the load-time evaluation is the one that
flashes Bad, and subscribing to OnEvent after LoadAsync made the assertion
unfalsifiable — it passed against the broken behaviour. It now subscribes first.

`FakeUpstream.ReadTag` returned 0x80340000 (BadNotConnected) for an unset path,
a different Bad code from production's placeholder. Harmless while any Bad
behaved alike; load-bearing now, so it is aligned and pinned by a reflection
guard asserting the engine's private constant agrees with what the production
source stamps.

Behaviour change to weigh, flagged in the issue and confirmed: virtual tags
computed from a comms-lost input now go Bad too. That follows from using the
shared mux and reads as correct, but it is broader than scripted alarms.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
This commit is contained in:
Joseph Doherty
2026-07-31 09:21:11 -04:00
parent b815446940
commit c81cef39f5
8 changed files with 360 additions and 26 deletions
+38 -10
View File
@@ -158,18 +158,46 @@ condition's state?") — mirroring the native OT semantic:
guard's job) — it contributes `Good`, so scripted conditions don't flash `Bad` at every deploy. The first
actually-`Bad` published value flips the bucket and annotates.
**Coverage boundary (#478 as shipped).** Scripted quality tracks input tags whose driver **publishes a
data change carrying a Bad/Uncertain `StatusCode`** (e.g. an OpcUaClient input forwarding a server's
per-item Bad). It does **not** yet cover a driver **comms loss**: a poll driver (Modbus/S7) whose device
goes unreachable emits only `ConnectivityChanged` and goes *silent* on the value feed (see
`DriverInstanceActor.Reconnecting`), so the scripted engine keeps the last-known Good value and the
condition stays `Good`. Bridging driver connectivity into scripted inputs — the symmetric of the native
`OnDriverConnectivityChanged` path above, plus resolving the null-value/cold-start asymmetry (a runtime
`Bad` with a null value is currently indistinguishable from cold start and contributes `Good`) — is tracked
as the Layer-4 follow-up (#481).
**Comms loss (Layer 4, #481).** #478 covered input tags whose driver **publishes a data change carrying a
Bad/Uncertain `StatusCode`** (e.g. an OpcUaClient input forwarding a server's per-item Bad). It did not
cover a driver **comms loss**: a poll driver (Modbus/S7) whose device goes unreachable emits only
`ConnectivityChanged` and goes *silent* on the value feed (see `DriverInstanceActor.Reconnecting`), so the
engine kept the last-known Good value, `WorstInputStatus` stayed Good — and worse, `AreInputsReady` stayed
true, so the predicate went on evaluating a stale value after the source had died.
`DriverHostActor.FanCommsLossToDependencyMux` closes it, symmetrically with the native
`OnDriverConnectivityChanged` path above: on `ConnectivityChanged(false)` it pushes a synthetic
`BadNoCommunication` with a **null value** into the dependency mux for every value ref the driver owns,
resolved from the existing `_nodeIdByDriverRef` map. Alarm-bearing tags are deliberately excluded — a
native condition is not a mux value, and its comms quality goes through the Part 9 annotation instead.
Two things worth knowing before changing this:
- **Nothing is pushed on reconnect, deliberately** — publishing Good would assert a value quality the host
has not observed. Recovery rides the driver's own republish: the `Connected → Reconnecting` edge calls
`DetachSubscription`, so the reconnect's `ResubscribeDesired` builds a *fresh* subscription whose
`PollGroupEngine` initial push runs with `forceRaise: true` and re-publishes every tag with its real
quality. That detach is load-bearing — without it the poll engine would compare against its retained
`LastValues`, see an unchanged value, raise nothing, and strand those refs Bad forever.
- **Virtual tags computed from a comms-lost input now go Bad too.** That follows from using the shared mux
and is the correct reading of "this input's source is unreachable", but it is a behaviour change beyond
scripted alarms.
**Cold start vs. a driver reporting Bad (#478 Finding 3, closed by #481).** `WorstInputStatus` used to skip
every input whose *value* was null as "no data yet". A comms-lost driver produces exactly that shape — Bad
with no value — so the skip was hiding the case that matters most. The upstream source now stamps a
never-pushed path `BadWaitingForInitialData` and nothing else does, so the engine skips *that specific
code* rather than every null. Deleting the null-skip outright would have been the obvious fix and is wrong
in the other direction: it makes every not-yet-published input drag its condition Bad at deploy, which is
what the skip existed to prevent. Both failure modes are pinned — see the engine-test pair below, each of
which goes red under the opposite mistake.
Guards: `ScriptedAlarmEngineTests` (transition carries `Uncertain`; `Bad` input with no transition emits
`QualityChanged(Bad)`; restore emits `QualityChanged(Good)`; unchanged bucket emits nothing),
`QualityChanged(Bad)`; restore emits `QualityChanged(Good)`; unchanged bucket emits nothing; plus the #481
pair — a pushed Bad carrying no value counts, an input never pushed does not),
`DriverHostActorLiveValueTests` (comms loss pushes Bad for every ref the driver owns, reconnect pushes
nothing, another driver's loss pushes nothing),
`DependencyMuxTagUpstreamSourceTests.The_engine_agrees_with_this_source_on_the_never_pushed_status_code`,
`ScriptedAlarmSourceTests.QualityChanged_emission_raises_no_alarm_event`,
`DependencyMuxActorTests.Publish_quality_is_forwarded_on_DependencyValueChanged`, and
`ScriptedAlarmHostActorTests` (`Bad_quality_dependency_publishes_AlarmQualityUpdate_and_no_alerts`,