docs: the stale-Good quality contract, stated once (R3-02)

This commit is contained in:
Joseph Doherty
2026-08-20 10:56:16 -04:00
parent e3a72dc385
commit 5dc8d34472
6 changed files with 123 additions and 4 deletions
+64 -2
View File
@@ -219,6 +219,37 @@ So a driver whose `InitializeAsync` disposes anything on its failure path must b
`MTConnectDriver` does today, because it is the only one that tears down inside a `catch`. The
shutdown paths deliberately keep the single trailing write.
### The quality contract
> **A value consumer never reports a quality better than its source's.** A node derived from a driver
> value — the raw node, every UNS mirror, a Calculation output, a virtual tag, a scripted condition's
> quality — carries, at worst, the worst status among its inputs: a Bad input makes the derived node Bad
> **with that input's sub-code and the last-known value**; an Uncertain input makes it Uncertain; a source
> that has stopped communicating is a Bad input (`BadNoCommunication`) for everything downstream of it,
> **including the external OPC UA data plane**. Recovery is a *publish*, never an assumption: the first
> post-recovery value must land even when it equals the value held before the outage, so every dedup on
> the path — driver caches, the node's own status, a consumer's last-published memory — must be keyed on
> **(value, status)**, not value alone.
This is one rule with one implementation of its comparison,
`ZB.MOM.WW.OtOpcUa.Core.Abstractions.InputQuality` (`WorstOf` / `IsUsable`), used by every derived-value
seam. It was written down because the same lie had been fixed four times in four places and re-opened in
a fifth (archreview round 3, theme "the stale-Good family"). The seams that implement it:
| Seam | What it does | Where |
|---|---|---|
| Driver comms loss → **value nodes** | one status-only batch stamps `BadNoCommunication` on the driver's raw NodeIds and every referencing UNS NodeId, **value retained** | `DriverHostActor.FanCommsLossToValueNodes` (below) |
| Driver comms loss → dependency mux | synthetic Bad per ref, so scripts and alarms see the source die | `DriverHostActor.FanCommsLossToDependencyMux` |
| Driver comms loss → Part 9 conditions | condition quality annotated out of band | `DriverHostActor.OnDriverConnectivityChanged` |
| MQTT / Sparkplug | connection loss stales observed refs; every (re)subscribe pushes the cached snapshot | [`docs/drivers/Mqtt.md`](drivers/Mqtt.md#quality-on-connection-loss-and-re-subscribe) |
| Calculation driver | holds on Bad, propagates the worst input status | [`docs/drivers/Calculation.md`](drivers/Calculation.md#input-quality) |
| Virtual tags | holds on Bad, inherits the worst input status, reassert carries it | [`docs/ScriptEditor.md`](ScriptEditor.md#input-quality-what-a-script-sees-and-when-it-runs-at-all) |
| Scripted alarms | worst-of-input quality annotated out of band; Bad inputs hold the condition | [`docs/ScriptedAlarms.md`](ScriptedAlarms.md) |
⚠️ **Consumer-visible:** value nodes of a comms-lost driver now go **Bad**, where they used to serve
their last value at Good. A consumer that treats a Bad value as "drop the tag" (ScadaBridge's data
connection layer, say) will see more Bads than before — that is the fix working, not a regression.
### The comms-loss watchdog (Gitea #527) — `GetHealth()` is now load-bearing
`GetHealth()` used to be **reporting-only**: `DriverInstanceActor.PublishHealthSnapshot` read it every 30 s
@@ -283,8 +314,8 @@ on it still gets its initialise attempt.
#### The fan-out that watchdog triggers: indexed per driver, and one batch
`DisconnectObserved` raises `ConnectivityChanged`, and `DriverHostActor` answers it by pushing
`BadNoCommunication` at every ref the flapping driver owns — the value fan to the dependency mux, and the
alarm fan to the publish actor's condition quality. Both used to answer "which refs are this driver's?"
`BadNoCommunication` at every ref the flapping driver owns — **three** fans: the value nodes (the
external OPC UA surface), the dependency mux, and the publish actor's condition quality. Both used to answer "which refs are this driver's?"
by **scanning the node's entire ref map and filtering** — the maps are keyed by the composite
`(DriverInstanceId, RawPath)`, so one driver's flap walked all ~50k keys on a large node to find its own
few thousand. Each now has a secondary index, `DriverInstanceId → [(RawPath, node set)]`
@@ -310,6 +341,37 @@ its last one — everything on the subscription path was already wrapped in `Att
`DependencyMuxActor` still registers the per-value receive, but in production it is dead code; it stays
for tests and direct senders.
#### The third fan: the value nodes (archreview 03/S15)
The two fans above reach *internal* consumers and *conditions*. Until archreview round 3 the driver's raw
value variables and their UNS mirrors — what an OPC UA client actually subscribes to — got nothing, so a
client saw the last value at **Good** quality for the whole outage. That was invisible on Modbus (which
degrades per tag within one poll interval on its own) and total on every driver whose failure shape is
"the reader stops producing": a coalesced-read throw, a dropped push session, or simply a driver whose
last act was a Good publish.
`FanCommsLossToValueNodes` closes it with **one status-only batch** — a new sink primitive,
`IOpcUaAddressSpaceSink.WriteStatuses` → `OtOpcUaNodeManager.WriteStatuses` — that changes each node's
StatusCode and Timestamp and **leaves its Value alone**. Status-only is not an optimisation: the host
holds no value cache (v3's single-source design deliberately removed the independent buffer), so only the
node itself can retain the last known value, and a Bad node still carrying its last reading is the OPC UA
Part 8 convention a SCADA consumer expects. Two details:
- It is **ungated by redundancy role**, like the condition annotation: a Secondary keeps its address space
honest for failover.
- An unmaterialised NodeId is **skipped, never created** — a status carries nothing to create a node from,
and the per-driver index can legitimately name a node a surgical remove has just dropped.
**Nothing is written on reconnect.** Publishing Good there would assert a quality the host has not
observed. Recovery is the driver's own re-publish: for a poll driver the watchdog's `DetachSubscription`
forces a fresh `PollGroupEngine` subscription whose initial poll runs `forceRaise: true`; for MQTT it is
the initial push added by 05/STAB-18. What the host does instead is **audit** it — `CommsRecoveryAudit`
fires `DefaultCommsRecoveryAuditDelay` (30 s) after the driver reconnects and reports any ref still
latched Bad, as a Warning naming the driver plus the `otopcua.driver.comms_latched_refs` counter (tagged
`driver`, `driver_type`). That counter going non-zero means a driver reconnected without re-publishing —
a push driver missing its initial push, or a value-only dedup swallowing an unchanged value. It is a
tripwire, not a repair: nothing downstream depends on it.
### The connect path feeds the circuit breaker (R2-10 T10 closure)
`IDriver.InitializeAsync` is routed through the capability invoker as