docs: perf roadmap remainder — outcomes + follow-up register

This commit is contained in:
Joseph Doherty
2026-08-16 05:08:53 -04:00
parent 451273c860
commit e75bb38734
5 changed files with 335 additions and 6 deletions
+76
View File
@@ -280,6 +280,36 @@ re-entered this path; the watchdog makes it automatic and unbounded, so a device
have leaked ~360 transports at the 10 s retry interval. The shutdown is best-effort: a driver that throws
on it still gets its initialise attempt.
#### The fan-out that watchdog triggers: indexed per driver, and one batch
`DisconnectObserved` raises `ConnectivityChanged`, and `DriverHostActor` answers it by pushing
`BadNoCommunication` at every ref the flapping driver owns — the value fan to the dependency mux, and the
alarm fan to the publish actor's condition quality. Both used to answer "which refs are this driver's?"
by **scanning the node's entire ref map and filtering** — the maps are keyed by the composite
`(DriverInstanceId, RawPath)`, so one driver's flap walked all ~50k keys on a large node to find its own
few thousand. Each now has a secondary index, `DriverInstanceId → [(RawPath, node set)]`
(`_valueRefsByDriver`, `_alarmRefsByDriver`), so the fan touches only that driver's refs.
The index cannot drift from its primary, because it is never maintained incrementally: both are cleared
and rebuilt whole inside `ApplyDesiredSubscriptions`, in the same pass and the same clear as the maps
they index. Two details are load-bearing if you touch that loop — an entry is added **only on set
creation** (so a repeated `(DriverInstanceId, RawPath)` contributes exactly one), and the node
`HashSet` is held **by reference, not copied**, so nodes registered against the same raw tag later in the
same pass (a UNS reference variable, say) are visible through the index with no second bookkeeping step.
⚠️ **The value fan is now a single message.** It sends one
`DriverInstanceActor.AttributeValueBatchPublished` carrying one item per RawPath, where it previously sent
one `AttributeValuePublished` per ref — 5 000 messages onto the mux that also carries the node's steady
value ingress, every time one driver flapped. Delivery is unchanged: the mux filters the batch per item
against the same registered-interest map the per-value receive consulted, so only the framing moved. The
**alarm** fan is indexed but deliberately not batched — one driver ref can back several condition NodeIds,
so it still emits one `AlarmQualityUpdate` per condition.
**Consequence worth knowing: `AttributeValuePublished` now has no production sender at all.** This fan was
its last one — everything on the subscription path was already wrapped in `AttributeValueBatchPublished`.
`DependencyMuxActor` still registers the per-value receive, but in production it is dead code; it stays
for tests and direct senders.
### The connect path feeds the circuit breaker (R2-10 T10 closure)
`IDriver.InitializeAsync` is routed through the capability invoker as
@@ -392,6 +422,52 @@ deployment prerequisites are in [Historian.md](Historian.md).
---
## PollGroupEngine — poll pacing and teardown
[`src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/PollGroupEngine.cs`](../src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/PollGroupEngine.cs)
The shared poll loop behind the seven polling drivers (Modbus, S7, AbCip, AbLegacy, FOCAS, Sql, TwinCAT).
Note it lives in `Core.Abstractions`, not under `src/Drivers/`.
**Pacing is fixed-rate, not fixed-delay.** The loop used to `await Task.Delay(interval)` *after* each
poll, so the real cadence was `interval + read time` and drifted with device latency — an 80 ms read on a
100 ms interval produced a 180 ms cycle, and a subscription's configured publishing interval was a floor
nobody actually got. Each poll is now timed (in a `finally`, so a *failed* poll is measured too) and the
next wait is `max(interval − elapsed, 0)`.
- **An overrun does not skip a cycle and does not burst to catch up.** When the remaining time is zero or
negative the loop does not call `Task.Delay` at all — it does a `Task.Yield()`, re-checks cancellation,
and starts the next poll immediately. The rate simply degrades to whatever the device sustains. The
`Yield` is load-bearing: without it a synchronously-completing reader turns a zero delay into a
thread-pool-starving tight loop that never observes cancellation.
- **The failure path is exempt.** When `backoffCap` is set *and* there have been consecutive failures, the
capped exponential `ConnectionBackoff.ComputeDelay` wins outright — a dead device must not be hammered
at fixed rate. (Previously the backoff formula was consulted whenever a cap was configured, including at
zero failures where it just returns the base interval.)
**`OverrunCount` is the signal that a device cannot keep up.** It counts poll cycles whose measured
duration exceeded that subscription's publishing interval, engine-wide across every subscription,
monotonic and never reset. A non-zero and *climbing* value means the loop is already running flat out.
- ⚠️ **It conflates a slow success with a slow failure** — a poll that overran because the device was slow
and one that overran because it timed out increment the same counter. Read it alongside the health
snapshot's error count, not on its own.
- **There is no meter and no log**, because `Core.Abstractions` deliberately carries no metrics
dependency. The surface is the `OverrunCount` property plus an optional `onOverrun` constructor
callback. **No driver wires either today** — a driver-level meter is a recorded follow-up, so treat the
counter as latent instrumentation rather than something you can read off a dashboard.
**Unsubscribe is awaited, not blocked on.** `Unsubscribe` drained the in-flight poll with a synchronous
`task.Wait(5s)` — and every driver's `UnsubscribeAsync` called it and returned `Task.CompletedTask`, so
that block landed *inside a `DriverInstanceActor` mailbox turn*, stalling the actor for up to five
seconds. `UnsubscribeAsync` now awaits `task.WaitAsync(DrainTimeout)` instead. The synchronous
`Unsubscribe` is retained for compatibility and has no production caller left; prefer the async form on
any path inside an actor turn or a request pipeline. Semantics are identical either way — the loop is
cancelled and awaited *before* the CTS is disposed, so no change callback fires after teardown and the
CTS is never disposed while a `Task.Delay` still holds its token.
---
## Commons — shared cross-cutting primitives
[`src/Core/ZB.MOM.WW.OtOpcUa.Commons/`](../src/Core/ZB.MOM.WW.OtOpcUa.Commons/)