dohertj2/lmxopcua
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'perf/roadmap-remainder' — 28-task perf remainder: historian batching, ACL cache re-shape, driver round-trips, alarm coalescing, AdminUI hot spots, off-mailbox apply

Browse Source
This commit is contained in:
Joseph Doherty
2026-08-17 03:18:36 -04:00
parent a8c93a0080 c8a7014388
commit e40a038527
88 changed files with 9851 additions and 720 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+20 -9
View File
@@ -281,10 +281,17 @@ non-`Healthy` state **and** its `LastSuccessfulRead` has been frozen for 90 s. T
all 12: nine poll drivers via `Degraded`, MQTT via live connection state, Galaxy via its reconnect
supervisor. `CalculationDriver` reports `Healthy` while running and so can never trip — correct, as it has
no transport. Re-check this if a new driver lands.
- ⚠️ **The reconnect retry MUST tear down before re-initialising.** `ModbusDriver` (`_transport`) and
`MqttDriver` (`_connection`) overwrite an `IAsyncDisposable` field inside `InitializeAsync` without
disposing the old one, so re-initialising without `ShutdownAsync` leaks a socket per attempt — ~360 for an
hour-long outage at the 10 s retry interval. Latent while only a manual Force Reconnect reached this path.
- **The reconnect retry MUST tear down before re-initialising — FIXED in both drivers, keep it that way.**
`ModbusDriver` (`_transport`) and `MqttDriver` (`_connection`) used to overwrite an `IAsyncDisposable`
field inside `InitializeAsync` without disposing the old one, so re-initialising without `ShutdownAsync`
leaked a socket per attempt — ~360 for an hour-long outage at the 10 s retry interval. Both now dispose
first: `ModbusDriver.InitializeCoreAsync` releases `_transport` **and** the `_probeCts`/`_reprobeCts`
probe loops before rebuilding them (`c3970a2e` + the lifecycle gate in `409a58b6`), deliberately *not*
via the shared `TeardownAsync()` because that would also drop `_poll` and every active subscription with
nothing to re-drive them; `MqttDriver.InitializeCoreAsync` calls its (idempotent) `TeardownAsync()` first,
which reaches `_connection` and the primary-host connection (`2b4b3fcb`). The path is no longer latent —
the comms-loss watchdog reaches it, not just a manual Force Reconnect — so a new driver, or a new
`IAsyncDisposable` field on these two, must repeat the dispose-before-overwrite.
- The threshold is hardcoded (`DefaultCommsLossTimeout`), matching `HealthPollInterval` /
`DefaultReconnectInterval`. Unlike the #523 drift detector this generates **no** traffic on a healthy
system — it is a local `GetHealth()` read, and only reaches the network once a driver has already stopped
@@ -748,15 +755,18 @@ Two things to know before relying on any of this:
*current* generation rather than the one the session activated under, and re-checks the
authenticating key — so revoking a key or a grant stops an open session immediately instead of at
the client's next reconnect. On a driver-only node the artifact is the only revocation channel.
- `UserAuthorizationState` is built **per call, not per session**, and its LDAP-shaped
- `UserAuthorizationState` is built **per decision walk, not per session** — and since the hot-path
change it is allocated only on a cache **miss**, so a cache hit never constructs one. Its LDAP-shaped
freshness/staleness predicates are switched off EXPLICITLY (`AuthCacheMaxStaleness = MaxValue`).
⚠️ Relying on "the stamp is always fresh" was not safe: `MembershipResolvedUtc` comes from
`DataPlaneAuthorizer`'s `TimeProvider` while `IsStale` is evaluated against
`TriePermissionEvaluator`'s, so **any divergence between the two clocks denies every request** with a
verdict indistinguishable from "no grant matched". Found by a test clock two hours ahead.
- **Stage 6's hot seams (read/browse/subscribe) go through a per-`(session, node, operation, realm)`
decision cache**, flushed when the generation advances. ⚠️ It caches the **grant walk only** — key
revocation and expiry are re-checked every call, because expiry moves with the clock rather than
decision cache**, cleared wholesale when the generation advances. It is **sharded per session**, so a
single client crawling a large tree past the per-session ceiling evicts only its own shard and not
every other session's decisions. ⚠️ It caches the **grant walk only** — key revocation and expiry are
re-checked **before** the cache lookup on every call, because expiry moves with the clock rather than
with a deployment.
- **Browse needs ancestor visibility, and it must NOT be a synthesized grant on the ancestors.** A
grant deep in the tree is reached by browsing the nodes above it, and the trie inherits downward
@@ -934,10 +944,11 @@ only the recorder + outbox knobs:
|---|---|---|
| `Enabled` | `false` | `true` (with `ServerHistorian` configured) wires + spawns the recorder |
| `OutboxPath` | `""` (required when enabled) | **Directory** holding the FasterLog segment + commit files. In production set an **absolute** path on durable storage |
| `CommitMode` | `PerEntry` | `PerEntry` = fsync before each append returns (no loss window); `Periodic` = batched commits every `CommitIntervalMs` |
| `CommitMode` | `PerEntry` | `PerEntry` = fsync before the append returns (no loss window); `Periodic` = batched commits every `CommitIntervalMs`. ⚠️ `PerEntry` now means **one fsync per ingest burst, not per value** — the recorder appends a burst through `AppendBatchAsync`, which enqueues every entry then issues a single group commit; the call still does not return until that commit lands |
| `CommitIntervalMs` | `100` | Periodic-mode commit cadence; required `> 0` only under `Periodic` |
| `DrainBatchSize` | `64` | Entries peeked + written per drain pass |
| `DrainBatchSize` | `1024` | Entries peeked + written per drain pass. **Raised from 64**: the peek is an in-memory scan and the pass is bounded-parallel (not sequential), so a wide batch no longer costs `tags × latency` — while 64 drew down ≤ 64 entries per `DrainIntervalSeconds`, letting the outbox fall permanently behind during any gateway hiccup |
| `DrainIntervalSeconds` | `2` | Steady drain cadence (and post-success reschedule) |
| `DrainConcurrency` | `8` | Historian tags one drain pass writes **concurrently** (a batch is grouped per tag; each group is its own gateway round-trip), so a sequential `tags × latency` pass becomes `⌈tags / concurrency⌉ × latency` with outstanding round-trips bounded. `1` = fully sequential. The ack is unchanged — all-or-nothing after every group reports, and the #538 quarantine still keys on "another group wrote this pass" |
| `Capacity` | `500000` | Max un-acked outbox entries before drop-oldest; `0` = unbounded. Non-zero **by default since #538** — the disk backstop for a historian refusing *every* write, the one case the per-tag quarantine deliberately cannot resolve |
| `MaxWriteAttempts` | `5` | #538: consecutive failed drain passes one tag may accumulate — counted **only across passes where other tags wrote successfully** — before its queued entries are abandoned so the FIFO head can advance |
| `MinBackoffSeconds` | `1` | Initial retry backoff after a failed drain pass |
docs/Configuration.md
+18
View File
@@ -383,6 +383,24 @@ key must carry the scopes `historian:read`, `historian:write`, `historian:tags:w
[`docs/Historian.md`](Historian.md) for the full key reference, the migration note (old Wonderware keys →
gateway keys), and the deployment prerequisites.
#### `ContinuousHistorization` (recorder + outbox knobs)
Only the knobs; the gateway connection comes from `ServerHistorian`. Full descriptions in
[`docs/Historian.md` §ContinuousHistorization configuration](Historian.md#continuoushistorization-configuration).
| Key | Default | Meaning |
|---|---|---|
| `ContinuousHistorization:Enabled` | `false` | `true` **and** a configured `ServerHistorian` wires the FasterLog outbox and spawns the recorder. |
| `ContinuousHistorization:OutboxPath` | `""` | **Directory** for the FasterLog segment + commit files. Required when enabled; use an **absolute** path on durable storage. |
| `ContinuousHistorization:CommitMode` | `PerEntry` | `PerEntry` fsyncs before the append returns; `Periodic` commits on a timer every `CommitIntervalMs`. **`PerEntry` is one fsync per ingest *burst*, not per value** — the recorder appends a burst through `AppendBatchAsync`, which enqueues every entry and issues a single group commit. Durability is unchanged: the call does not return until that commit lands. |
| `ContinuousHistorization:CommitIntervalMs` | `100` | Periodic-mode cadence (ms); must be `> 0` under `Periodic`. |
| `ContinuousHistorization:DrainBatchSize` | `1024` | Entries peeked + written per drain pass. **Raised from 64**: the peek is an in-memory scan (widening it costs allocation, not I/O) and the pass is now bounded-parallel rather than sequential, so a wide batch no longer costs `tags × latency`. 64 drew down ≤ 64 entries per `DrainIntervalSeconds`, so any gateway hiccup left the outbox permanently behind, growing toward `Capacity` and shedding oldest samples. |
| `ContinuousHistorization:DrainIntervalSeconds` | `2` | Steady drain cadence (and post-success reschedule). |
| `ContinuousHistorization:DrainConcurrency` | `8` | Historian tags one drain pass writes **concurrently** — a batch is grouped per tag and each group is its own gateway round-trip, so this turns a sequential `tags × latency` pass into `⌈tags / concurrency⌉ × latency` while bounding outstanding round-trips. `1` = fully sequential; `< 1` is clamped to 1 with a `Validate()` warning. The ack is unchanged (all-or-nothing after every group reports), as is the #538 quarantine rule. |
| `ContinuousHistorization:Capacity` | `500000` | Max un-acked entries before drop-oldest; `0` = unbounded. The #538 disk backstop for a historian refusing **every** write — the one case the per-tag quarantine cannot resolve. |
| `ContinuousHistorization:MaxWriteAttempts` | `5` | #538 poison-tag budget: consecutive failed passes one tag may accumulate — **counted only across passes where *other* tags wrote successfully** — before its entries are abandoned so the FIFO head can advance. |
| `ContinuousHistorization:MinBackoffSeconds` / `MaxBackoffSeconds` | `1` / `30` | Initial and capped exponential retry backoff after a failed drain pass. |
### `LocalDb` (pair-local config cache — driver role only)
Driver-role nodes keep a consolidated **`ZB.MOM.WW.LocalDb`** SQLite database that caches the deployed
docs/DriverLifecycle.md
+76
View File
@@ -280,6 +280,36 @@ re-entered this path; the watchdog makes it automatic and unbounded, so a device
have leaked ~360 transports at the 10 s retry interval. The shutdown is best-effort: a driver that throws
on it still gets its initialise attempt.
#### The fan-out that watchdog triggers: indexed per driver, and one batch
`DisconnectObserved` raises `ConnectivityChanged`, and `DriverHostActor` answers it by pushing
`BadNoCommunication` at every ref the flapping driver owns — the value fan to the dependency mux, and the
alarm fan to the publish actor's condition quality. Both used to answer "which refs are this driver's?"
by **scanning the node's entire ref map and filtering** — the maps are keyed by the composite
`(DriverInstanceId, RawPath)`, so one driver's flap walked all ~50k keys on a large node to find its own
few thousand. Each now has a secondary index, `DriverInstanceId → [(RawPath, node set)]`
(`_valueRefsByDriver`, `_alarmRefsByDriver`), so the fan touches only that driver's refs.
The index cannot drift from its primary, because it is never maintained incrementally: both are cleared
and rebuilt whole inside `ApplyDesiredSubscriptions`, in the same pass and the same clear as the maps
they index. Two details are load-bearing if you touch that loop — an entry is added **only on set
creation** (so a repeated `(DriverInstanceId, RawPath)` contributes exactly one), and the node
`HashSet` is held **by reference, not copied**, so nodes registered against the same raw tag later in the
same pass (a UNS reference variable, say) are visible through the index with no second bookkeeping step.
⚠️ **The value fan is now a single message.** It sends one
`DriverInstanceActor.AttributeValueBatchPublished` carrying one item per RawPath, where it previously sent
one `AttributeValuePublished` per ref — 5 000 messages onto the mux that also carries the node's steady
value ingress, every time one driver flapped. Delivery is unchanged: the mux filters the batch per item
against the same registered-interest map the per-value receive consulted, so only the framing moved. The
**alarm** fan is indexed but deliberately not batched — one driver ref can back several condition NodeIds,
so it still emits one `AlarmQualityUpdate` per condition.
**Consequence worth knowing: `AttributeValuePublished` now has no production sender at all.** This fan was
its last one — everything on the subscription path was already wrapped in `AttributeValueBatchPublished`.
`DependencyMuxActor` still registers the per-value receive, but in production it is dead code; it stays
for tests and direct senders.
### The connect path feeds the circuit breaker (R2-10 T10 closure)
`IDriver.InitializeAsync` is routed through the capability invoker as
@@ -392,6 +422,52 @@ deployment prerequisites are in [Historian.md](Historian.md).
---
## PollGroupEngine — poll pacing and teardown
[`src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/PollGroupEngine.cs`](../src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/PollGroupEngine.cs)
The shared poll loop behind the seven polling drivers (Modbus, S7, AbCip, AbLegacy, FOCAS, Sql, TwinCAT).
Note it lives in `Core.Abstractions`, not under `src/Drivers/`.
**Pacing is fixed-rate, not fixed-delay.** The loop used to `await Task.Delay(interval)` *after* each
poll, so the real cadence was `interval + read time` and drifted with device latency — an 80 ms read on a
100 ms interval produced a 180 ms cycle, and a subscription's configured publishing interval was a floor
nobody actually got. Each poll is now timed (in a `finally`, so a *failed* poll is measured too) and the
next wait is `max(interval − elapsed, 0)`.
- **An overrun does not skip a cycle and does not burst to catch up.** When the remaining time is zero or
negative the loop does not call `Task.Delay` at all — it does a `Task.Yield()`, re-checks cancellation,
and starts the next poll immediately. The rate simply degrades to whatever the device sustains. The
`Yield` is load-bearing: without it a synchronously-completing reader turns a zero delay into a
thread-pool-starving tight loop that never observes cancellation.
- **The failure path is exempt.** When `backoffCap` is set *and* there have been consecutive failures, the
capped exponential `ConnectionBackoff.ComputeDelay` wins outright — a dead device must not be hammered
at fixed rate. (Previously the backoff formula was consulted whenever a cap was configured, including at
zero failures where it just returns the base interval.)
**`OverrunCount` is the signal that a device cannot keep up.** It counts poll cycles whose measured
duration exceeded that subscription's publishing interval, engine-wide across every subscription,
monotonic and never reset. A non-zero and *climbing* value means the loop is already running flat out.
- ⚠️ **It conflates a slow success with a slow failure** — a poll that overran because the device was slow
and one that overran because it timed out increment the same counter. Read it alongside the health
snapshot's error count, not on its own.
- **There is no meter and no log**, because `Core.Abstractions` deliberately carries no metrics
dependency. The surface is the `OverrunCount` property plus an optional `onOverrun` constructor
callback. **No driver wires either today** — a driver-level meter is a recorded follow-up, so treat the
counter as latent instrumentation rather than something you can read off a dashboard.
**Unsubscribe is awaited, not blocked on.** `Unsubscribe` drained the in-flight poll with a synchronous
`task.Wait(5s)` — and every driver's `UnsubscribeAsync` called it and returned `Task.CompletedTask`, so
that block landed *inside a `DriverInstanceActor` mailbox turn*, stalling the actor for up to five
seconds. `UnsubscribeAsync` now awaits `task.WaitAsync(DrainTimeout)` instead. The synchronous
`Unsubscribe` is retained for compatibility and has no production caller left; prefer the async form on
any path inside an actor turn or a request pipeline. Semantics are identical either way — the loop is
cancelled and awaited *before* the CTS is disposed, so no change callback fires after teardown and the
CTS is never disposed while a `Task.Delay` still holds its token.
---
## Commons — shared cross-cutting primitives
[`src/Core/ZB.MOM.WW.OtOpcUa.Commons/`](../src/Core/ZB.MOM.WW.OtOpcUa.Commons/)
docs/Historian.md
+41
View File
@@ -142,6 +142,47 @@ source it from there.
---
## ContinuousHistorization configuration
`ContinuousHistorizationOptions` binds the `ContinuousHistorization` section. It carries **only** the
recorder + outbox knobs — the gateway connection (endpoint / key / TLS) is sourced from
`ServerHistorian` (above), never restated here.
| Key | Type | Default | Description |
|---|---|---|---|
| `Enabled` | bool | `false` | `true` **and** a configured `ServerHistorian` wires the FasterLog outbox + gateway value-writer and spawns the `ContinuousHistorizationRecorder`. `false` → no recorder; driver tag values are not historized. |
| `OutboxPath` | string | `""` | **Directory** holding the FasterLog segment + commit files. Required when `Enabled`. In production set an **absolute** path on durable storage — a relative path resolves against the host's working directory. |
| `CommitMode` | string | `PerEntry` | `PerEntry` fsyncs before the append returns (no loss window); `Periodic` batches commits onto a background timer every `CommitIntervalMs`. Parsed case-insensitively; an unrecognized value falls back to `PerEntry`. **See the batch-commit note below — `PerEntry` is one fsync per ingest *burst*, not per value.** |
| `CommitIntervalMs` | int | `100` | Periodic-mode commit cadence (ms); must be `> 0` under `Periodic`, ignored under `PerEntry`. |
| `DrainBatchSize` | int | `1024` | Outbox entries peeked + written per drain pass. **Raised from 64**: the peek is an in-memory scan of the outbox's live index (widening it costs allocation, not I/O), and a pass is now bounded-parallel over `DrainConcurrency` rather than sequential, so a wide batch no longer costs `tags × latency`. At 64 a drain drew down at most 64 entries every `DrainIntervalSeconds` — below the arrival rate of even a modest historized tag set — so any gateway hiccup left the outbox permanently behind, growing toward `Capacity` and shedding oldest samples instead of catching up. |
| `DrainIntervalSeconds` | double | `2` | Steady drain cadence (and the post-success reschedule). |
| `DrainConcurrency` | int | `8` | Historian tags a single drain pass writes **concurrently**. One peeked batch is grouped per historian tag and each group is an independent gateway round-trip, so a sequential pass cost `tags × latency`; this window makes it `⌈tags / concurrency⌉ × latency` while bounding how many round-trips can be outstanding against the gateway at once. `1` restores the fully sequential drain; a value `< 1` is clamped to 1 (with a `Validate()` warning). **The ack is unchanged** — still all-or-nothing after every group has reported, and the #538 per-tag quarantine still keys on "another group wrote this pass". |
| `Capacity` | int | `500000` | Max un-acked outbox entries before the drop-oldest policy sheds the front of the queue; `0` = unbounded. Roughly 50 MB of FasterLog. The #538 **disk backstop** for the one case the per-tag quarantine deliberately cannot resolve — a historian unreachable or refusing *every* write, where nothing proves any particular tag is at fault so the recorder correctly retains. |
| `MaxWriteAttempts` | int | `5` | #538 poison-tag budget: consecutive failed drain passes one historian tag may accumulate before its queued entries are abandoned and the FIFO head is allowed to advance. **Counted only across passes where *other* tags wrote successfully**, so an ordinary gateway outage never advances it. Clamped to at least 1; each abandonment logs a Warning naming the tag. |
| `MinBackoffSeconds` | double | `1` | Initial retry backoff after a failed drain pass. |
| `MaxBackoffSeconds` | double | `30` | Cap on the exponential retry backoff. |
> **`PerEntry` now means one fsync per ingest *burst*, not per value.** The recorder appends a whole
> value burst through `IHistorizationOutbox.AppendBatchAsync`, which enqueues every entry and then
> issues a **single** group commit — so a burst of N values costs one fsync rather than N. The
> durability contract is unchanged: the call does not return until that commit completes, so nothing
> the caller has been told is buffered can be lost to an unclean restart. (The documented residual is
> unchanged too: a throw *before* the commit leaves the enqueued prefix tracked-but-uncommitted —
> served correctly for the life of the process, absent after an unclean restart — exactly the
> enqueue-then-commit window the single-entry path always had, widened from one entry to the batch.)
> **Cross-repo follow-up — a multi-tag batch `WriteLiveValues` on the gateway contract.** The drain's
> unit of work is one gateway round-trip **per historian tag**, because
> `ZB.MOM.WW.HistorianGateway.Client` **0.3.0** exposes no RPC that writes values for more than one tag
> in a call. Bounded parallelism (`DrainConcurrency`) is therefore the ceiling: it overlaps the
> round-trips but cannot remove them, so a wide tag set still pays `⌈tags / concurrency⌉` network
> hops per drain pass. Collapsing that to a single hop needs a **batch/multi-tag `WriteLiveValues`** on
> the `historian_gateway.v1` contract in the `~/Desktop/HistorianGateway` sister repo. Until that
> lands, raising `DrainBatchSize` widens each hop's payload but not the hop count — which is why the
> concurrency window, not the batch width, is what governs drain throughput today.
---
## A poison tag no longer pins the outbox (#538)
`DrainOnceAsync` peeks a batch, writes **every** per-tag group in it, and then acks **all-or-nothing** —
docs/ScriptedAlarms.md
+48 -2
View File
@@ -57,12 +57,58 @@ Alarm predicates reuse the same Roslyn sandbox as virtual tags — `ScriptEvalua
Evaluation cadence:
- On every upstream tag change that any alarm's input set references (`OnUpstreamChange` → `ReevaluateAsync`). The engine maintains an inverse index `tag path → alarm ids` (`_alarmsReferencing`); only affected alarms re-run.
- On a 5-second shelving-check timer (`_shelvingTimer`) for timed-shelve expiry.
- On every upstream tag change that any alarm's input set references (`OnUpstreamChange` → the dirty set → `DrainDirtyAsync` → `ReevaluateAsync`). The engine maintains an inverse index `tag path → alarm ids` (`_alarmsReferencing`); only affected alarms re-run.
- On a 5-second shelving-check timer (`_shelvingTimer`) for timed-shelve expiry. This is **not** routed through the dirty set — it remains an independent producer contending for the same evaluation gate.
- At `LoadAsync` for every alarm, to re-derive `ActiveState` per plan decision #14 (startup recovery).
If a predicate throws or times out, the engine logs the failure and leaves the prior `ActiveState` intact — it does not synthesise a clear. Operators investigating a broken predicate should never see a phantom clear preceding the error.
### Dirty-set coalescing
Every upstream push used to spawn its **own** fire-and-forget re-evaluation, each allocating a fresh
array of affected alarm ids and each queueing on the single non-reentrant evaluation gate. N pushes
arriving during one slow evaluation meant **N serialized re-evaluations**, every one of them re-reading
the same latest values and every one paying the per-predicate timeout machinery again.
A push now unions the affected alarm ids into a `HashSet<string> _dirtyAlarmIds` under a short leaf lock
and, if no drain is already scheduled, starts one. The drain loops: take the set, swap in an empty one,
release the lock, evaluate that batch, repeat until the set is empty. Exactly one drainer runs at a time.
- **There is no timer and no wall-clock window.** A push landing on an idle engine is evaluated
immediately; the coalescing window is exactly "whatever queued while the gate was busy".
- **"Latest values win" is structural, not a heuristic.** No per-push value snapshot is carried into the
drain. Every push writes the newest value into `_valueCache` synchronously, and the surviving
evaluation refills its scratch read-cache from `_valueCache` *at drain time, inside the gate* — so it
reads whatever the newest push wrote. The collapse is loss-free by construction.
- **Transition semantics are unchanged**, because evaluation was already level-triggered: the old code
never read the pushed value either, it always re-read the cache. All N of those serialized passes
already saw the same latest values, so the dirty set removes redundant work, not information.
- **An alarm is evaluated at most once per pass** even when ten of its input tags changed — the fan-in
across inputs dedups, not just repeat pushes of one tag.
- ⚠️ **What can still be collapsed away** (and always could be) is an intermediate state the engine would
have transited had it evaluated between pushes. With a slow predicate or a blocked store, a value that
crosses a threshold and crosses back before the drain reaches it produces **no transition and no
event**; the alarm settles on the latest value. The #478 quality-bucket tracking sees only the
coalesced-latest bucket for the same reason.
- **Batch order is unordered** — the batch comes out of a `HashSet`, so alarms within a pass are no longer
evaluated in push order. Emissions still fire strictly after the gate is released.
#### Why the store write was NOT moved behind a write-behind queue
The review that prompted this work suggested a write-behind queue for `IAlarmStateStore.SaveAsync` —
deferring the Part 9 condition-state persist (`LocalDbAlarmConditionStateStore`, the replicated
`alarm_condition_state` table) out of the evaluation gate. It was **deliberately rejected**, and the
rationale is recorded in the engine's class doc so it is not re-proposed:
1. **It trades away a real crash-consistency property.** The store write stays *inside* the gate and
*before* the in-memory update, so a store failure leaves the in-memory condition and the persisted one
both at the prior state rather than diverging. That ordering is pinned by
`Store_save_failure_leaves_in_memory_state_unchanged`.
2. **The throughput it would buy is no longer needed.** After coalescing, a `SaveAsync` happens only on a
genuine Part 9 transition, which is rare relative to the push rate — so the gate-held store call is not
on the hot path any more. Write-behind would trade a correctness property for latency the dirty set
already removed.
## Part 9 state machine
`Part9StateMachine` (`Part9StateMachine.cs`) is a pure `static` function set. Every transition takes the current `AlarmConditionState` plus the event, returns a new record and an `EmissionKind`. No I/O, no mutation, trivially unit-testable. Transitions map to OPC UA Part 9:
docs/plans/2026-08-15-perf-roadmap-remainder.md
+1425
View File
File diff suppressed because it is too large Load Diff
docs/plans/2026-08-15-perf-roadmap-remainder.md.tasks.json
+35
View File
@@ -0,0 +1,35 @@
{
"planPath": "docs/plans/2026-08-15-perf-roadmap-remainder.md",
"tasks": [
{"id": 1, "subject": "Task 1: Outbox batch append with group commit", "status": "in_progress", "note": "6ff64ca0 + fix 73205865 (incremental bookkeeping, second-reader durability pin); re-review in flight"},
{"id": 2, "subject": "Task 2: Batched mux→subscriber delivery, single-eval virtual tags", "status": "completed", "note": "content inside 140bf0c1 (shared-index sweep; message belongs to Task 22 — needs an outcomes note); spec + code approved; code review's two Importants (yielding-fake batch test, mid-batch exception policy) folded into Task 4's brief"},
{"id": 3, "subject": "Task 3: Parallel drain with bounded concurrency", "status": "completed", "note": "a76b4e07; spec + code APPROVED; follow-up (inherited, pre-existing): bare catch around WriteLiveValuesAsync swallows OperationCanceledException at shutdown → spurious _tagWriteFailures increments when succeeded>0; recommend rethrowing genuine ct cancellations"},
{"id": 4, "subject": "Task 4: Recorder batched ingest — one fsync per burst", "status": "completed", "blockedBy": [1, 2, 3], "note": "384568b4 + follow-up 451273c8 (append-failure branch pinned via 3 independent mutations, loss-accounting docs corrected, recorder 64→1024 literals aligned); all reviews APPROVED; register: #491 falsification nondeterministic under load, IngestFaults not surfaced in docs/gauges, latch re-arms on any non-throwing item"},
{"id": 5, "subject": "Task 5: Historian defaults, config drift, docs", "status": "completed", "blockedBy": [3, 4], "note": "0298a0b9 (new Historian.md key table, cross-repo batch-RPC follow-up recorded); code APPROVED; recorder 64→1024 literals aligned in 451273c8; minor: pinning-test doc comment overclaims appsettings-drift coverage"},
{"id": 6, "subject": "Task 6: Per-session decision-cache shards", "status": "completed", "note": "debd44c2; follow-ups: _globalCount drift under races, concurrency stress test, bound-ratio comment"},
{"id": 7, "subject": "Task 7: Authorizer hot path — lookup first, no closures, lazy deny reasons", "status": "in_progress", "blockedBy": [6]},
{"id": 8, "subject": "Task 8: Browse and TranslateBrowsePath compaction", "status": "completed", "note": "541a223b + follow-up test aad7583b (survivor-order pin, falsified red, review APPROVED); minor: FanOutTargetIdList fake couples to SDK single-Add contract — comment if pattern reused"},
{"id": 9, "subject": "Task 9: RevalidateMonitoredItems dedup", "status": "in_progress", "blockedBy": [8], "note": "fbe8aaad; realm-qualified 3-tuple key (safer than spec); spec+code reviews in flight; watch: its clock-count observable couples to DataPlaneAuthorizer reading the clock once — Task 7 edits that file"},
{"id": 10, "subject": "Task 10: Modbus InitializeAsync transport leak", "status": "completed", "note": "c3970a2e + 409a58b6"},
{"id": 11, "subject": "Task 11: PollGroupEngine fixed-rate pacing, overrun counter, async unsubscribe", "status": "completed", "blockedBy": [10, 12, 13, 14], "note": "9cf400f1 + tolerance fix 485f4ea1 (fast-reader test: 200ms interval, <=1 blip); spec ✅ + code APPROVED; follow-ups: NoteOverrun conflates slow-success/slow-failure (doc caveat), backoffCap param doc note for future callers"},
{"id": 12, "subject": "Task 12: S7 narrow-scalar span codec", "status": "completed", "note": "8d91833f; SByte spec mention was a plan error"},
{"id": 13, "subject": "Task 13: S7 span planner + grouped ReadAsync", "status": "in_progress", "blockedBy": [12], "note": "39f67a0f; spec passed (demotion rule + factory-binding 6th file accepted); code review in flight; follow-up: live gate vs s7_1500 fixture before trusting 16/220 defaults"},
{"id": 14, "subject": "Task 14: AbCip bounded-parallel reads", "status": "completed", "note": "d6a4fbd6 + fix d5f573b0 (per-device ConnectGate single-flight); re-review APPROVED; minor follow-ups: stale EvictRuntime-race comment ~L1161, test timing margin on slow CI"},
{"id": 15, "subject": "Task 15: Dirty-set coalescing in ScriptedAlarmEngine", "status": "completed", "note": "5df793e7; follow-ups: TrackBackgroundTask registration-ordering gap (pre-existing), stale line-334 comment, non-volatile _disposed"},
{"id": 16, "subject": "Task 16: Deployments list projection", "status": "completed", "note": "dbc9a132"},
{"id": 17, "subject": "Task 17: Drift watermark probe", "status": "completed", "blockedBy": [16], "note": "684e59d8; spec ✅ + code APPROVED; follow-ups: GroupBy(_=>1) aggregate untested against real SQL Server (InMemory only — perf-claim unverified), compose seam carries no ct (pre-existing, worth a comment), single-entry cache thrash during Sealed transition worth a doc line"},
{"id": 18, "subject": "Task 18: ScriptTagCatalog TTL snapshot cache", "status": "completed", "note": "3795899d"},
{"id": 19, "subject": "Task 19: EffectiveNameGuard batch check + bulk add-references", "status": "completed", "note": "fa36d64e; minor error-precedence shift for mixed-failure batches recorded for outcomes"},
{"id": 20, "subject": "Task 20: Raw tree tag-name projection", "status": "completed", "note": "ba977dbe"},
{"id": 21, "subject": "Task 21: Rename script-scan filters in SQL", "status": "completed", "blockedBy": [20], "note": "fe8d132c + fix ca87329a (Distinct + 200-term chunking, ScriptId-keyed cross-batch dedup); both APPROVED; follow-ups: CHARINDEX/collation widening needs SQL-Server-backed test; worst-case 250 sequential round-trips undocumented (one-line comment)"},
{"id": 22, "subject": "Task 22: Pooled context factory + RoleGrants off the circuit-scoped context", "status": "completed", "note": "140bf0c1 (also carries Task 2's files — record in outcomes); spec + code approved"},
{"id": 23, "subject": "Task 23: Fleet/Hosts poll overlap guards", "status": "completed", "note": "4b6a93bb + 63081d52 (Fleet async dispose)"},
{"id": 24, "subject": "Task 24: MQTT unchanged-value suppression", "status": "completed", "note": "2c708f55 + 50035b36 (Sparkplug-bypass note, status-code-half test with falsification)"},
{"id": 25, "subject": "Task 25: DriverHostActor per-driver index + batched comms-loss fan", "status": "completed", "blockedBy": [2], "note": "b6275f43; spec ✅ + code APPROVED (minor: cross-sender RegisterInterest ordering in test not structurally guaranteed; unreachable empty-bucket guard); flags for T28 docs: stale DependencyMuxActor.cs lines 8/19 comment (per-value receive now test-only); NativeAlarmAckRoutingTests local 5s timeout flake (#500 class)"},
{"id": 26, "subject": "Task 26: Direct-apply artifact read off the mailbox", "status": "completed", "blockedBy": [25], "note": "5b92eb0f; spec ✅ + code APPROVED (BusLogging comment dispute resolved in implementer's favor); follow-ups: Direct-mode SQL artifact read carries no timeout/ct (Important — confirm connection-string command timeout), 3 stale ApplyAndAck prose comments in sibling test files (fold into T28), Bootstrap orphan replay exits PreStart in Applying"},
{"id": 27, "subject": "Task 27: MTConnect sample chunks parse from bytes", "status": "completed", "note": "34f836e0"},
{"id": 28, "subject": "Task 28: Full sweep + plan close-out docs", "status": "completed", "blockedBy": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27], "note": "e75bb387 (outcomes + register) + dbc92232 (comment fixes, incl. 5th stale comment at ServiceCollectionExtensions.cs:405); FULL SWEEP: 10310 passed / 0 failed / 19 skipped, build 0 errors, DB suites ran against 10.100.0.35; no marker commit for Task 2 (recorded in Outcomes); stale empty Wonderware test dirs noted in plan doc"}
],
"lastUpdated": "2026-08-16T04:30:00Z",
"closeOut": "All 28 tasks completed and reviewed. Final integration review: READY, no Critical. Post-review fixes: 445bb855 (drain-latch handover, page poll-timer teardown), b9e9df40 (comment/doc honesty sweep). Full sweep: 10310 passed / 0 failed / 19 skipped. Release condition (not merge condition): docker-dev live gate + s7_1500 fixture run before plant deployment — see the plan's follow-up register."
}
docs/security.md
+64 -1
View File
@@ -438,13 +438,76 @@ Behaviours worth knowing before enabling it:
Read, Browse and subscription creation ask the same question many times per request — one Browse of a
device folder asks once per child. Those seams go through a per-`(session, node, operation, realm)`
decision cache, flushed wholesale whenever the authority generation advances.
decision cache (`SessionAclDecisionCache`), cleared wholesale whenever the authority generation
advances.
⚠️ **The cache holds the grant walk, not the decision.** Grants, principals and the trie are all fixed
for the life of a generation, so memoizing the walk is sound. A key's `ExpiresUtc` passes with the
*clock*, not with a deployment — so revocation and expiry are re-checked on every call, outside the
cache. Caching them would have let an expired key keep working until the next deploy.
**The cache is sharded per calling KEY, because a flat one let any client evict every other client's
decisions.** It used to be a single dictionary with the caller identity folded *into* the composite key
and one process-wide bound of `MaxEntries = 100_000`, whose only eviction was a wholesale `Clear()` at
the cap. So one machine consumer crawling a large address space past 100k entries wiped the memoized
walks of every other caller on the server, and each store paid a `ConcurrentDictionary.Count` — an
all-locks snapshot — to test the cap. It is now a dictionary of shards keyed by
**`ApiKeyId ?? Principal`**, each holding its own `(operation, realm, nodeIdentifier)` entries, with two
bounds:
| Bound | Value | What it clears |
|---|---|---|
| `MaxEntriesPerSession` | 20 000 | that one **key's** shard |
| `MaxEntries` | 100 000 | every shard (the shard *objects* are dropped, not emptied) |
⚠️ **The shard key is the API key, not the session — the constant's name predates it.** Every session
authenticating with the same `otua_<keyId>_…` credential shares one shard and one 20 000-entry budget,
so a crawl by one of them *can* clear the memoized decisions of its sibling sessions. What the sharding
guarantees is isolation **across distinct keys**: one key's crawl can never evict another key's entries.
(An Anonymous session shards under the reserved `anonymous` principal, so all of them likewise share
one shard.)
Both bounds are compile-time constants, not appsettings keys. Five keys simultaneously at their
per-key ceiling reach the global cap; **the number of shards is not itself bounded** — the bound is
entries.
Things to know before changing it:
- **There is no session-close eviction hook**, deliberately. The shard of a key with no live sessions
left is shed the next time a wholesale clear happens — a generation bump, or the global cap tripping. Dropping the shard
objects rather than emptying them is what makes that sweep work.
- **The counters are approximate under concurrency, by design.** The cap test is a check-then-act over
`Volatile.Read` of `Interlocked` counters rather than a lock, so concurrent stores can overshoot a cap
by roughly one shard's worth. Every clear resets the counters, so drift cannot ratchet. Inserts use
`TryAdd` (not an indexer assignment) so that only a genuine insert moves a counter — otherwise two
threads racing the same key would inflate them and permanently pull the ceilings in.
- **Only the CAS winner clears on a generation bump.** `InvalidateIfStale` compare-exchanges the
generation, so a post-deploy burst of requests repopulates the cache once instead of each thread
clearing it out from under the others.
**The hot path looks the entry up before it builds anything.** The gate used to construct the node scope
— splitting the `s=` identifier and allocating a `NodeScope` — and then allocate a five-capture closure
for the walk, on *every* call including cache hits. Now the identifier is extracted once, the cache is
consulted immediately, and a hit returns having built no scope, no closure and no
`UserAuthorizationState`; the scope and the walk happen only on a miss. `GetOrAdd` was split into an
allocation-free `TryGet` plus a `Set` to make that possible, and `TryGet` deliberately does not
create a shard, so a hit-only session never mints one and a just-cleared shard is not resurrected until
something is actually stored.
⚠️ **Revocation is checked ahead of the cache, not behind it** — the key-usability lookup runs *above*
the `TryGet`, so a cache hit cannot serve an allow for a key that has since been disabled or expired.
Reversing those two lines is the subtle way to reintroduce the bug the paragraph above warns about: the
generation never moves when a key merely expires, so nothing would ever drop the stale allow. Two tests
pin the ordering (`Key_revocation_is_rechecked_on_a_cache_hit` and
`An_expired_key_is_denied_even_though_its_allow_is_cached`).
**Deny reasons are composed lazily.** A denial on the cached path logs at `Debug`, and the interpolated
reason strings (which generation, which key, which operation) used to be built at every call site before
the log level was consulted — so a Browse denied across a thousand children formatted a thousand strings
with `Debug` off. The interpolating denials now guard on `_log.IsEnabled(level)` first, and the
`BadUserAccessDenied` `ServiceResult` is a shared static instance rather than one allocation per denied
node.
#### Everything denies when something is missing
No identity, no principal on the identity, **no cluster identity on the node**, a NodeId with no string
src/Core/ZB.MOM.WW.OtOpcUa.Configuration/ServiceCollectionExtensions.cs
+19 -5
View File
@@ -22,15 +22,29 @@ public static class ServiceCollectionExtensions
?? throw new InvalidOperationException(
$"Connection string '{ConnectionStringName}' is required. Add it to appsettings.json or the OTOPCUA_CONFIG_CONNECTION env var.");
services.AddDbContextFactory<OtOpcUaConfigDbContext>(opt => opt.UseSqlServer(connectionString));
// POOLED, deliberately: every consumer in this repo takes the IDbContextFactory<> and creates
// a short-lived context per call/query, so context construction (and its model/service-provider
// lookup) is on the hot path of every AdminUI render. Pooling makes each CreateDbContext a rent
// from a reset instance instead of a fresh build. OtOpcUaConfigDbContext qualifies: one public
// ctor taking DbContextOptions<>, no injected scoped services, no instance state beyond EF's own.
services.AddPooledDbContextFactory<OtOpcUaConfigDbContext>(opt => opt.UseSqlServer(connectionString));
// AddDbContextFactory registers only the IDbContextFactory<> — it does NOT also register
// a scoped OtOpcUaConfigDbContext. Config services that take the context directly (e.g.
// LdapGroupRoleMappingService) need a scoped instance, so bridge one off the factory.
// AddPooledDbContextFactory registers only the IDbContextFactory<> — it does NOT also register
// a scoped OtOpcUaConfigDbContext. This bridge exists for ONE consumer that cannot be moved to
// the factory because it is not ours: ASP.NET Core Data Protection's EF Core key repository
// (Security's AddDataProtection().PersistKeysToDbContext<OtOpcUaConfigDbContext>()), which
// resolves TContext with CreateScope() + GetRequiredService<TContext>() and has no factory
// support. Deleting this bridge breaks cookie-key persistence — i.e. AdminUI login — at runtime,
// not at build time. Nothing else should take OtOpcUaConfigDbContext from DI directly; take the
// factory and own the context's lifetime per call, as LdapGroupRoleMappingService does. Disposing
// a pooled context returns it to the pool, so the scope end is a release, not a teardown.
services.AddScoped(sp => sp.GetRequiredService<IDbContextFactory<OtOpcUaConfigDbContext>>().CreateDbContext());
// Config-DB services consumed by both the AdminUI (RoleGrants page) and the auth/login
// host (AuthEndpoints.LoginAsync). Scoped to match the request/render scope of both callers.
// host (AuthEndpoints.LoginAsync). Scoped is a registration-shape choice, not a data-access
// one: the driver-role block TryAdds NullLdapGroupRoleMappingService at the same lifetime, and
// the service itself is stateless — it creates and disposes a context per call off the pooled
// factory rather than pinning one for the scope (a Blazor circuit is a single long scope).
services.AddScoped<ILdapGroupRoleMappingService, LdapGroupRoleMappingService>();
return services;
src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Services/LdapGroupRoleMappingService.cs
+19 -2
View File
@@ -8,7 +8,18 @@ namespace ZB.MOM.WW.OtOpcUa.Configuration.Services;
/// "exactly one of (ClusterId, IsSystemWide)" invariant at the write surface so a
/// malformed row can't land in the DB.
/// </summary>
public sealed class LdapGroupRoleMappingService(OtOpcUaConfigDbContext db) : ILdapGroupRoleMappingService
/// <remarks>
/// Takes the <see cref="IDbContextFactory{TContext}"/> and creates + disposes one context PER CALL
/// rather than holding an injected context for its lifetime. Its consumers are the RoleGrants Blazor
/// page (InteractiveServer, so its DI scope is the whole circuit) and <c>AuthEndpoints.LoginAsync</c>;
/// an injected context would pin one context — and one pooled connection — for the life of a circuit,
/// and would fault outright ("a second operation was started on this context instance") the moment two
/// of that page's renders overlapped. The factory is pooled (see
/// <c>ServiceCollectionExtensions.AddOtOpcUaConfigDb</c>), so the per-call create is a pool rent, not
/// a fresh context build.
/// </remarks>
public sealed class LdapGroupRoleMappingService(IDbContextFactory<OtOpcUaConfigDbContext> dbFactory)
: ILdapGroupRoleMappingService
{
/// <inheritdoc />
public async Task<IReadOnlyList<LdapGroupRoleMapping>> GetByGroupsAsync(
@@ -18,6 +29,7 @@ public sealed class LdapGroupRoleMappingService(OtOpcUaConfigDbContext db) : ILd
var groupSet = ldapGroups.ToList();
if (groupSet.Count == 0) return [];
await using var db = await dbFactory.CreateDbContextAsync(cancellationToken).ConfigureAwait(false);
return await db.LdapGroupRoleMappings
.AsNoTracking()
.Where(m => groupSet.Contains(m.LdapGroup))
@@ -27,12 +39,15 @@ public sealed class LdapGroupRoleMappingService(OtOpcUaConfigDbContext db) : ILd
/// <inheritdoc />
public async Task<IReadOnlyList<LdapGroupRoleMapping>> ListAllAsync(CancellationToken cancellationToken)
=> await db.LdapGroupRoleMappings
{
await using var db = await dbFactory.CreateDbContextAsync(cancellationToken).ConfigureAwait(false);
return await db.LdapGroupRoleMappings
.AsNoTracking()
.OrderBy(m => m.LdapGroup)
.ThenBy(m => m.ClusterId)
.ToListAsync(cancellationToken)
.ConfigureAwait(false);
}
/// <inheritdoc />
public async Task<LdapGroupRoleMapping> CreateAsync(LdapGroupRoleMapping row, CancellationToken cancellationToken)
@@ -43,6 +58,7 @@ public sealed class LdapGroupRoleMappingService(OtOpcUaConfigDbContext db) : ILd
if (row.Id == Guid.Empty) row.Id = Guid.NewGuid();
if (row.CreatedAtUtc == default) row.CreatedAtUtc = DateTime.UtcNow;
await using var db = await dbFactory.CreateDbContextAsync(cancellationToken).ConfigureAwait(false);
db.LdapGroupRoleMappings.Add(row);
await db.SaveChangesAsync(cancellationToken).ConfigureAwait(false);
return row;
@@ -51,6 +67,7 @@ public sealed class LdapGroupRoleMappingService(OtOpcUaConfigDbContext db) : ILd
/// <inheritdoc />
public async Task DeleteAsync(Guid id, CancellationToken cancellationToken)
{
await using var db = await dbFactory.CreateDbContextAsync(cancellationToken).ConfigureAwait(false);
var existing = await db.LdapGroupRoleMappings.FindAsync([id], cancellationToken).ConfigureAwait(false);
if (existing is null) return;
db.LdapGroupRoleMappings.Remove(existing);
src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorizationOutbox.cs
+40 -1
View File
@@ -13,12 +13,51 @@ public interface IHistorizationOutbox : IDisposable
/// <summary>Lifetime count of entries dropped because an append would have exceeded capacity.</summary>
long DroppedCount { get; }
/// <summary>Appends <paramref name="entry"/> to the tail of the durable buffer.</summary>
/// <summary>
/// Appends <paramref name="entry"/> to the tail of the durable buffer.
/// <para>
/// <b>Retained for tests and direct callers; production appends only through
/// <see cref="AppendBatchAsync"/>.</b> The recorder coalesces a driver burst into one group
/// commit, so this single-entry form has no production call site — it is kept as the
/// one-entry expression of the same contract (and is what <see cref="AppendBatchAsync"/>'s
/// durability window is described against), not as a deprecated path.
/// </para>
/// </summary>
/// <param name="entry">The value record to buffer.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
ValueTask AppendAsync(HistorizationOutboxEntry entry, CancellationToken ct);
/// <summary>
/// Appends a burst of entries with ONE durability commit for the whole batch (group
/// commit). Under PerEntry commit mode this is the per-batch fsync that replaces N
/// per-entry fsyncs; under Periodic it is indistinguishable from N AppendAsync calls.
/// An empty list is a no-op.
/// </summary>
/// <remarks>
/// <para>
/// <strong>Failure semantics — prefix, never orphan.</strong> The call is not atomic. If it
/// throws part-way through (cancellation, an I/O fault), the entries appended before the
/// failure remain in the buffer and are fully tracked: they are visible to
/// <see cref="PeekBatchAsync"/>, ackable via <see cref="RemoveAsync"/>, counted by
/// <see cref="CountAsync"/>, and subject to the capacity bound — exactly as if the caller
/// had passed only that prefix. Nothing is left buffered-but-unackable, which would be
/// redelivered on every drain pass and block the buffer's head from advancing. Entries
/// after the failure point are not appended at all; the caller may retry them.
/// </para>
/// <para>
/// <strong>Accepted residual.</strong> The batch's single durability commit runs after the
/// last append, so a throw before it leaves that prefix tracked but not yet committed —
/// served correctly for the life of the process, but absent after an unclean restart. This
/// is the same enqueue-then-commit window <see cref="AppendAsync"/> has always had, widened
/// from one entry to the batch.
/// </para>
/// </remarks>
/// <param name="entries">The value records to buffer, in append (FIFO) order.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
ValueTask AppendBatchAsync(IReadOnlyList<HistorizationOutboxEntry> entries, CancellationToken ct);
/// <summary>
/// Returns up to <paramref name="max"/> oldest un-acked entries in FIFO order without removing
/// them. Removal happens via <see cref="RemoveAsync"/> once each entry is durably written.
src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorizedTagSubscriptionSink.cs
+5 -3
View File
@@ -5,9 +5,11 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
/// because a <c>HistorianTagname</c> override makes them diverge:
/// <list type="bullet">
/// <item>
/// <see cref="MuxRef"/> — the driver-published reference the per-node dependency mux fans its
/// <c>DependencyValueChanged</c> by (the tag's driver-side <c>FullName</c>). The recorder
/// registers mux interest and matches incoming values by THIS.
/// <see cref="MuxRef"/> — the driver-published reference the per-node dependency mux fans values
/// by (the tag's driver-side <c>FullName</c>); in production those arrive as a
/// <c>DependencyValueBatch</c> per driver burst, with the per-value <c>DependencyValueChanged</c>
/// retained for direct senders and tests. The recorder registers mux interest and matches
/// incoming values by THIS.
/// </item>
/// <item>
/// <see cref="HistorianName"/> — the resolved historian tag name the value is written under (a
src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/PollGroupEngine.cs
+147 -20
View File
@@ -1,5 +1,6 @@
using System.Collections;
using System.Collections.Concurrent;
using System.Diagnostics;
namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
@@ -33,6 +34,21 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
/// behaves byte-identically to before: no batch is collected and no per-cycle allocation is
/// made.</para>
///
/// <para>Pacing is <b>fixed-rate, not fixed-delay</b>: the post-poll wait is
/// <c>max(interval - elapsed, 0)</c>, so a read that takes 80 ms on a 100 ms interval is
/// followed by a 20 ms wait rather than a full 100 ms one, and the observed cadence stays at
/// the configured interval instead of drifting out to (read + interval). When a poll takes
/// LONGER than the interval the next one starts immediately, the overrun is counted in
/// <see cref="OverrunCount"/>, and the optional <c>onOverrun</c> callback is invoked with the
/// measured duration. The failure/backoff branch is deliberately exempt: once
/// <c>consecutiveFailures &gt; 0</c> with a <c>backoffCap</c> configured, the capped-exponential
/// schedule wins, because a failing device must not be hammered at fixed rate.</para>
///
/// <para><b>No <c>Meter</c> lives here.</b> <c>Core.Abstractions</c> deliberately references
/// nothing, so <see cref="OverrunCount"/> plus the <c>onOverrun</c> callback ARE the
/// instrumentation surface — a driver that wants a metric wires its own meter to the callback.
/// (Driver-level meters are a recorded follow-up, not part of this type.)</para>
///
/// <para>Programmer errors and obviously-fatal exceptions (<see cref="OutOfMemoryException"/>,
/// <see cref="ThreadAbortException"/>, <see cref="StackOverflowException"/>,
/// <see cref="AccessViolationException"/>) are NOT caught — they propagate and tear the poll
@@ -44,10 +60,12 @@ public sealed class PollGroupEngine : IAsyncDisposable
private readonly Action<ISubscriptionHandle, string, DataValueSnapshot> _onChange;
private readonly Action<Exception>? _onError;
private readonly Action<ISubscriptionHandle, IReadOnlyList<DataChangeItem>>? _onBatch;
private readonly Action<TimeSpan>? _onOverrun;
private readonly TimeSpan _minInterval;
private readonly TimeSpan? _backoffCap;
private readonly ConcurrentDictionary<long, SubscriptionState> _subscriptions = new();
private long _nextId;
private long _overrunCount;
/// <summary>Default floor for publishing intervals — matches the Modbus 100 ms cap.</summary>
public static readonly TimeSpan DefaultMinInterval = TimeSpan.FromMilliseconds(100);
@@ -73,13 +91,19 @@ public sealed class PollGroupEngine : IAsyncDisposable
/// order, after all of the cycle's per-tag <paramref name="onChange"/> callbacks. The
/// driver forwards it to its own <see cref="IBatchSubscribable.OnDataChangeBatch"/> event.
/// <c>null</c> (default) collects nothing and allocates nothing per cycle.</param>
/// <param name="onOverrun">Optional overrun sink — invoked with the measured poll duration
/// every time a poll takes longer than its subscription's interval (the same events counted
/// by <see cref="OverrunCount"/>). Since this assembly carries no metrics dependency, this
/// callback is how a driver surfaces the condition to its own meter/log. Defensive: a
/// handler that throws is absorbed, exactly like <c>onError</c>.</param>
public PollGroupEngine(
Func<IReadOnlyList<string>, CancellationToken, Task<IReadOnlyList<DataValueSnapshot>>> reader,
Action<ISubscriptionHandle, string, DataValueSnapshot> onChange,
TimeSpan? minInterval = null,
Action<Exception>? onError = null,
TimeSpan? backoffCap = null,
Action<ISubscriptionHandle, IReadOnlyList<DataChangeItem>>? onBatch = null)
Action<ISubscriptionHandle, IReadOnlyList<DataChangeItem>>? onBatch = null,
Action<TimeSpan>? onOverrun = null)
{
ArgumentNullException.ThrowIfNull(reader);
ArgumentNullException.ThrowIfNull(onChange);
@@ -89,8 +113,16 @@ public sealed class PollGroupEngine : IAsyncDisposable
_minInterval = minInterval ?? DefaultMinInterval;
_backoffCap = backoffCap;
_onBatch = onBatch;
_onOverrun = onOverrun;
}
/// <summary>
/// Total number of poll cycles, across every subscription of this engine, whose read took
/// longer than the subscription's publishing interval. A non-zero and climbing value means
/// the device cannot sustain the configured rate — the loop is already running flat out.
/// </summary>
public long OverrunCount => Interlocked.Read(ref _overrunCount);
/// <summary>Register a new polled subscription and start its background loop.</summary>
/// <param name="fullReferences">The list of tag references to poll.</param>
/// <param name="publishingInterval">The desired polling interval; will be clamped to the configured minimum.</param>
@@ -108,43 +140,94 @@ public sealed class PollGroupEngine : IAsyncDisposable
return handle;
}
/// <summary>Cancel the background loop for a handle returned by <see cref="Subscribe"/>.</summary>
/// <summary>
/// Cancel the background loop for a handle returned by <see cref="Subscribe"/>, BLOCKING the
/// calling thread until the in-flight poll drains. Retained for compatibility; prefer
/// <see cref="UnsubscribeAsync"/> on any path that runs inside an actor mailbox turn or a
/// request pipeline, where blocking a thread for up to 5 s is never acceptable.
/// </summary>
/// <returns><c>true</c> when the handle was known to the engine and has been torn down.</returns>
/// <param name="handle">The subscription handle to cancel.</param>
public bool Unsubscribe(ISubscriptionHandle handle)
{
if (handle is PollSubscriptionHandle h && _subscriptions.TryRemove(h.Id, out var state))
if (!TryTake(handle, out var state)) return false;
StopState(state);
return true;
}
/// <summary>
/// Non-blocking counterpart of <see cref="Unsubscribe"/>: identical teardown semantics (the
/// loop is cancelled and awaited before the CTS is disposed, so no callback can fire after
/// this completes) but it awaits the loop task instead of blocking a thread on it.
/// </summary>
/// <param name="handle">The subscription handle to cancel.</param>
/// <returns><c>true</c> when the handle was known to the engine and has been torn down.</returns>
public async ValueTask<bool> UnsubscribeAsync(ISubscriptionHandle handle)
{
if (!TryTake(handle, out var state)) return false;
await StopStateAsync(state).ConfigureAwait(false);
return true;
}
/// <summary>
/// Shared front half of both teardown paths: recognise the handle and remove it from the
/// registry exactly once, so a double unsubscribe (or a race between the sync and async
/// overloads) tears one subscription down once and reports <c>false</c> for the loser.
/// </summary>
private bool TryTake(ISubscriptionHandle handle, out SubscriptionState state)
{
if (handle is PollSubscriptionHandle h && _subscriptions.TryRemove(h.Id, out var found))
{
StopState(state);
state = found;
return true;
}
state = null!;
return false;
}
// Cancel + drain + dispose, shared by both teardown paths. The loop task is awaited (with a
// generous timeout) before the CTS is disposed so:
// (a) no _onChange callback fires after the caller considers the subscription torn down, and
// (b) the CTS is not disposed while Task.Delay is still holding a reference to its token,
// which can turn OperationCanceledException into ObjectDisposedException.
private static void StopState(SubscriptionState state)
{
try { state.Cts.Cancel(); } catch { }
// Await the loop task (with a generous timeout) before disposing the CTS so:
// (a) no _onChange callback fires after the caller considers the engine torn down, and
// (b) the CTS is not disposed while Task.Delay is still holding a reference to its token,
// which can turn OperationCanceledException into ObjectDisposedException.
var task = state.LoopTask;
if (task is not null)
{
try { task.Wait(TimeSpan.FromSeconds(5)); } catch { }
try { task.Wait(DrainTimeout); } catch { }
}
state.Cts.Dispose();
}
private static async ValueTask StopStateAsync(SubscriptionState state)
{
try { state.Cts.Cancel(); } catch { }
var task = state.LoopTask;
if (task is not null)
{
try { await task.WaitAsync(DrainTimeout).ConfigureAwait(false); } catch { }
}
state.Cts.Dispose();
}
/// <summary>Ceiling on how long a teardown waits for an in-flight poll to unwind.</summary>
private static readonly TimeSpan DrainTimeout = TimeSpan.FromSeconds(5);
/// <summary>Snapshot of active subscription count — exposed for driver diagnostics.</summary>
public int ActiveSubscriptionCount => _subscriptions.Count;
private async Task PollLoopAsync(SubscriptionState state, CancellationToken ct)
{
var consecutiveFailures = 0;
// Duration of the poll just performed — the fixed-rate pacing subtracts it from the
// interval so the cadence is the interval, not (interval + read time).
TimeSpan lastPollDuration;
// Initial-data push: every subscribed tag fires once at subscribe time regardless of
// whether it has changed, satisfying OPC UA Part 4 initial-value semantics.
var started = Stopwatch.GetTimestamp();
try
{
await PollOnceAsync(state, forceRaise: true, ct).ConfigureAwait(false);
@@ -161,21 +244,49 @@ public sealed class PollGroupEngine : IAsyncDisposable
consecutiveFailures++;
ReportError(ex);
}
finally { lastPollDuration = Stopwatch.GetElapsedTime(started); }
NoteOverrun(state.Interval, lastPollDuration);
while (!ct.IsCancellationRequested)
{
// Fixed cadence by default; when a backoff cap is configured, sustained failures
// stretch the delay (capped exponential) and a success snaps it back to Interval.
var delay = _backoffCap is null
? state.Interval
: ConnectionBackoff.ComputeDelay(state.Interval, consecutiveFailures, _backoffCap.Value);
try { await Task.Delay(delay, ct).ConfigureAwait(false); }
catch (OperationCanceledException) { return; }
// Defensive: the CTS may be disposed by Unsubscribe/DisposeAsync between the
// cancellation check above and the Task.Delay touching the token. Treat that race
// as a normal cancellation rather than a fatal exception.
catch (ObjectDisposedException) { return; }
// Fixed RATE, not fixed delay: the wait is what remains of the interval after the poll
// that just ran, so a slow-but-succeeding device keeps the configured cadence and a
// poll that overran starts the next one immediately.
//
// The failure branch is deliberately exempt: once a backoff cap is configured AND the
// device is failing, the capped-exponential schedule wins outright — a dead device must
// not be hammered at fixed rate. With no failures ComputeDelay returns the base interval
// anyway, so it is only consulted when it can actually change the answer.
TimeSpan delay;
if (_backoffCap is not null && consecutiveFailures > 0)
{
delay = ConnectionBackoff.ComputeDelay(state.Interval, consecutiveFailures, _backoffCap.Value);
}
else
{
var remaining = state.Interval - lastPollDuration;
delay = remaining > TimeSpan.Zero ? remaining : TimeSpan.Zero;
}
if (delay > TimeSpan.Zero)
{
try { await Task.Delay(delay, ct).ConfigureAwait(false); }
catch (OperationCanceledException) { return; }
// Defensive: the CTS may be disposed by Unsubscribe/DisposeAsync between the
// cancellation check above and the Task.Delay touching the token. Treat that race
// as a normal cancellation rather than a fatal exception.
catch (ObjectDisposedException) { return; }
}
else
{
// Overrun: no wait is owed. Still yield, so a reader that completes synchronously
// can never turn a zero delay into a thread-pool-starving tight loop, and so the
// cancellation check below is reached.
await Task.Yield();
if (ct.IsCancellationRequested) return;
}
started = Stopwatch.GetTimestamp();
try
{
await PollOnceAsync(state, forceRaise: false, ct).ConfigureAwait(false);
@@ -191,9 +302,25 @@ public sealed class PollGroupEngine : IAsyncDisposable
consecutiveFailures++;
ReportError(ex);
}
finally { lastPollDuration = Stopwatch.GetElapsedTime(started); }
NoteOverrun(state.Interval, lastPollDuration);
}
}
/// <summary>
/// Record a poll that took longer than its interval: bump <see cref="OverrunCount"/> and
/// hand the measured duration to the optional <c>onOverrun</c> sink. Defensive against a
/// sink that throws, for the same reason <see cref="ReportError"/> is.
/// </summary>
private void NoteOverrun(TimeSpan interval, TimeSpan elapsed)
{
if (elapsed <= interval) return;
Interlocked.Increment(ref _overrunCount);
if (_onOverrun is null) return;
try { _onOverrun(elapsed); }
catch { /* never let a buggy overrun sink stop the poll loop */ }
}
/// <summary>
/// Programmer-error / process-fatal exception classification: anything that cannot be
/// safely "swallowed and retry on the next tick" must escape the poll loop instead.
src/Core/ZB.MOM.WW.OtOpcUa.Core.ScriptedAlarms/ScriptedAlarmEngine.cs
+165 -9
View File
@@ -29,6 +29,29 @@ namespace ZB.MOM.WW.OtOpcUa.Core.ScriptedAlarms;
/// Operators investigating a broken predicate shouldn't see a phantom
/// clear-event preceding the failure.
/// </para>
/// <para>
/// <b>Upstream pushes coalesce through a dirty set.</b> A push does not spawn its own
/// re-evaluation: it marks the referencing alarm ids dirty and, if no drainer is already
/// running, starts one. <see cref="DrainDirtyAsync"/> then swaps the set out and evaluates
/// the batch, looping until the set is empty. N pushes arriving during one slow evaluation
/// therefore collapse into ONE further pass instead of N serialized ones that would each
/// re-read the same latest values. The collapse is loss-free by construction:
/// <c>RefillReadCache</c> always reads <see cref="_valueCache"/>, which every push has
/// already updated, so the surviving pass evaluates the latest values — latest-wins, never
/// a lost update. There is no timer and no wall-clock batching window; a push that lands on
/// an idle engine is evaluated immediately, and the coalescing window is exactly "whatever
/// queued while <see cref="_evalGate"/> was busy".
/// </para>
/// <para>
/// <b><see cref="IAlarmStateStore.SaveAsync"/> deliberately stays INSIDE the gate, before
/// the in-memory update.</b> The persist-before-update ordering is a real crash-consistency
/// property — a store failure must leave the in-memory condition and the persisted one both
/// at the prior state rather than diverging — and it is pinned by
/// <c>Store_save_failure_leaves_in_memory_state_unchanged</c>. Post-coalescing a save only
/// happens on a genuine Part 9 transition, which is rare relative to the push rate, so the
/// gate-held store call is no longer on the hot path and a write-behind queue would trade a
/// correctness property for throughput the engine does not need.
/// </para>
/// </remarks>
public sealed class ScriptedAlarmEngine : IDisposable
{
@@ -149,7 +172,7 @@ public sealed class ScriptedAlarmEngine : IDisposable
private bool _disposed;
// Tracks fire-and-forget background work launched by OnUpstreamChange
// (ReevaluateAsync) and RunShelvingCheck (ShelvingCheckAsync). Dispose drains
// (DrainDirtyAsync) and RunShelvingCheck (ShelvingCheckAsync). Dispose drains
// these so a re-evaluation in flight when shutdown begins finishes its
// SaveAsync before the engine returns control to the caller. The HashSet is
// accessed under its own lock — never under _evalGate — so registration /
@@ -157,6 +180,15 @@ public sealed class ScriptedAlarmEngine : IDisposable
private readonly HashSet<Task> _inFlight = [];
private readonly object _inFlightLock = new();
// Coalescing state for the upstream-push path. _dirtyAlarmIds accumulates the
// alarms a push made stale; _drainScheduled says a DrainDirtyAsync loop is
// already running and will pick them up, so a push storm starts exactly one
// drainer. Both are guarded by _dirtyLock — a leaf lock, never taken while
// holding _evalGate or _inFlightLock, and never held across an await.
private readonly HashSet<string> _dirtyAlarmIds = new(StringComparer.Ordinal);
private readonly object _dirtyLock = new();
private bool _drainScheduled; // guarded by _dirtyLock
/// <summary>
/// Initializes a new ScriptedAlarmEngine with the provided dependencies.
/// </summary>
@@ -497,10 +529,12 @@ public sealed class ScriptedAlarmEngine : IDisposable
}
/// <summary>
/// Upstream-change callback. Updates the value cache + enqueues predicate
/// re-evaluation for every alarm referencing the changed path. Fire-and-forget
/// so driver-side dispatch isn't blocked; the background task is tracked so
/// <see cref="Dispose"/> can drain it.
/// Upstream-change callback. Updates the value cache + marks every alarm
/// referencing the changed path dirty, starting the drainer if one isn't
/// already running. Never blocks driver-side dispatch, and never spawns work
/// per push: a push landing while a drain is in flight only grows the dirty
/// set, so a storm collapses into one further evaluation pass over the latest
/// values. The drainer is tracked so <see cref="Dispose"/> can drain it.
/// </summary>
/// <param name="path">The upstream tag path that changed.</param>
/// <param name="value">The new data value snapshot.</param>
@@ -508,9 +542,129 @@ public sealed class ScriptedAlarmEngine : IDisposable
{
_valueCache[path] = value;
if (_disposed) return; // don't queue new work against a disposing engine
if (_alarmsReferencing.TryGetValue(path, out var alarmIds))
if (!_alarmsReferencing.TryGetValue(path, out var alarmIds)) return;
bool startDrain;
lock (_dirtyLock)
{
TrackBackgroundTask(ReevaluateAsync(alarmIds.ToArray(), CancellationToken.None));
// Iterate the referencing set directly — the pre-coalescing path
// allocated a fresh string[] on every single push.
foreach (var id in alarmIds) _dirtyAlarmIds.Add(id);
startDrain = !_drainScheduled;
if (startDrain) _drainScheduled = true;
}
// Started outside the lock: DrainDirtyAsync runs synchronously up to its
// first real await, and nothing it does should happen under _dirtyLock.
if (startDrain) TrackBackgroundTask(DrainDirtyAsync(CancellationToken.None));
}
/// <summary>
/// The single drainer for the upstream-push dirty set. Swaps the accumulated
/// alarm ids out under <c>_dirtyLock</c> and re-evaluates that batch, looping
/// until the set is empty — so every push that landed while the previous pass
/// held <c>_evalGate</c> is served by the next pass rather than by a queue of
/// redundant per-push evaluations. Exactly one instance runs at a time
/// (<c>_drainScheduled</c>), and it clears that flag under the same lock it
/// reads the set under, so the last drainer out and the next push in can never
/// both decide "someone else will handle it".
/// </summary>
/// <remarks>
/// <b>INVARIANT: <c>_drainScheduled</c> is released on EVERY exit path, the
/// exceptional one included.</b> The flag is the only thing that lets
/// <see cref="OnUpstreamChange"/> start a drainer, so an escape that leaves it
/// stuck <c>true</c> permanently and silently stops evaluation for every scripted
/// alarm in the process: pushes keep filling <c>_valueCache</c> and the dirty set,
/// no drainer is ever started again, no predicate ever runs again, and nothing is
/// logged. <see cref="ReevaluateAsync"/> catches everything today, so the throw
/// surface is currently empty — the <c>finally</c> below is what keeps the
/// invariant true when it stops being (an audit-mode Serilog sink rethrowing out
/// of that catch block's own <c>Error()</c> call is enough).
/// </remarks>
private async Task DrainDirtyAsync(CancellationToken ct)
{
// Set inside _dirtyLock, together with the flag clear, so the finally can tell
// "I already released" from "I am unwinding" without ever double-releasing —
// a second clear could stomp a _drainScheduled = true a racing push had just
// set for the drainer it started, leaving two drainers running.
var released = false;
try
{
while (true)
{
string[] batch;
lock (_dirtyLock)
{
// Bail promptly on a disposing engine, but ALWAYS release
// _drainScheduled on the way out — leaving it stuck true would make a
// hypothetical post-dispose push believe a drainer is coming. (Such
// pushes are already dropped in OnUpstreamChange; this keeps the flag
// honest regardless.)
if (_disposed || _dirtyAlarmIds.Count == 0)
{
_dirtyAlarmIds.Clear();
_drainScheduled = false;
released = true;
return;
}
batch = [.. _dirtyAlarmIds];
_dirtyAlarmIds.Clear();
}
// Reused as-is: the gate / evaluate / persist / emit-outside-the-gate shape
// is already correct for a batch. Only the spawn-per-push is gone.
await ReevaluateAsync(batch, ct).ConfigureAwait(false);
}
}
finally
{
// Exceptional exit only (see `released`). The batch the faulted pass was
// carrying dies with it — ReevaluateAsync's own catch is what normally makes
// that a logged non-event — but ids that queued BEHIND it are still in the
// set and would otherwise sit there until some later push happened to arrive,
// so hand the latch straight to a replacement drainer instead of releasing it.
//
// Exactly-one-drainer survives the handover because _drainScheduled never
// transiently CLEARS during one: deciding to hand over and clearing the flag
// are the same guarded read-modify-write, and the clear happens only in the
// branch that starts no replacement. The spawn itself is deliberately OUTSIDE
// the lock (nothing it does belongs under _dirtyLock) and is safe there
// precisely because the flag is still set — a push landing in that window only
// grows the set, which the replacement then drains.
//
// Two bounds, both honest:
// * CPU — every pass swaps the set out and clears it before evaluating, so a
// replacement finds work only if fresh pushes arrived. A permanently
// throwing evaluation therefore costs one pass per push, the same rate as
// a healthy one; it cannot spin against an idle upstream.
// * Stack — the replacement is QUEUED (Task.Run), not called. A replacement
// that is called runs INLINE for as long as it does not truly suspend, and
// its own escape then unwinds back into this same finally block, nesting a
// frame per throw with no ceiling — a StackOverflowException no catch can
// recover from. Whether it suspends first is not a property this method
// controls: _evalGate is already free when the escape is thrown (released
// in ReevaluateAsync's inner finally, before the outer catch's logging
// call), so a failure raised BEFORE the first awaited work — the clock
// read, anything synchronous ahead of the predicate — unwinds with no
// suspension at all. A failure raised at or after the predicate does not,
// because TimedScriptEvaluator hops to the pool on every evaluation;
// measured, that path holds depth flat even when called directly. Queuing
// makes the fresh stack unconditional instead of dependent on which throw
// shape happens to occur, at the price of one pool dispatch per faulted
// pass. Pinned by
// Sustained_synchronous_drain_failures_hand_over_without_deepening_the_stack.
if (!released)
{
bool restart;
lock (_dirtyLock)
{
restart = !_disposed && _dirtyAlarmIds.Count > 0;
if (!restart) _drainScheduled = false;
}
if (restart) TrackBackgroundTask(Task.Run(() => DrainDirtyAsync(ct)));
}
}
}
@@ -925,9 +1079,11 @@ public sealed class ScriptedAlarmEngine : IDisposable
_shelvingTimer?.Dispose();
UnsubscribeFromUpstream();
// Drain any fire-and-forget background work (ReevaluateAsync from
// Drain any fire-and-forget background work (DrainDirtyAsync from
// OnUpstreamChange + ShelvingCheckAsync from the 5s timer) that started
// before _disposed = true was visible. Without this, a SaveAsync in
// before _disposed = true was visible. The drain loop observes _disposed
// at the top of every iteration and exits, so it cannot spin here. Without
// this, a SaveAsync in
// flight can outlive the engine and write to a (possibly disposed) store
// after Dispose() has returned. The tasks re-check _disposed after
// acquiring the gate and bail out, but the await still has to complete.
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.AbCip.Contracts/AbCipDriverOptions.cs
+12
View File
@@ -82,6 +82,18 @@ public sealed class AbCipDriverOptions
/// </summary>
public bool EnableDeclarationOnlyUdtGrouping { get; init; }
/// <summary>
/// Upper bound on how many planned read work items (whole-UDT groups + per-tag fallbacks)
/// one <c>ReadAsync</c> batch may have in flight against the controller at once. Bounds
/// concurrency ACROSS tags only — the per-tag runtime lock (05/STAB-4) still serialises
/// Read → GetStatus → Decode against any one cached libplctag handle, so the same tag
/// requested twice in a batch never overlaps itself. Raising it lets libplctag overlap
/// device round trips (the whole point: a strictly sequential batch pays one full round
/// trip per tag); lowering it to <c>1</c> restores strictly sequential reads. Values below
/// <c>1</c> are clamped to <c>1</c> at the use site.
/// </summary>
public int MaxConcurrentReads { get; init; } = 8;
/// <summary>
/// Timeout for the AdminUI Test Connect probe, in seconds. The AdminUI clamps to a
/// 60s server-side maximum; this default is what the form pre-fills for new instances.
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.AbCip/AbCipDriver.cs
+230 -52
View File
@@ -448,10 +448,11 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
Task.FromResult(_poll.Subscribe(fullReferences, publishingInterval));
/// <inheritdoc />
public Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken)
public async Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken)
{
_poll.Unsubscribe(handle);
return Task.CompletedTask;
// Async teardown: the engine drains the in-flight poll without blocking the caller's
// thread — this runs inside a DriverInstanceActor mailbox turn.
await _poll.UnsubscribeAsync(handle).ConfigureAwait(false);
}
// ---- IAlarmSource (ALMD projection) ----
@@ -495,6 +496,11 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
while (!ct.IsCancellationRequested)
{
var success = false;
// The probe's backoff verdict for this tick: true = record success, false = record
// failure, null = no signal (a read that returned a non-zero status without throwing
// never recorded either, and still doesn't). Recorded AFTER the try/catch so the
// per-device connect gate is taken outside a catch block — see below.
bool? backoffVerdict = null;
try
{
probeRuntime ??= _tagFactory.Create(probeParams);
@@ -508,7 +514,7 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
await probeRuntime.ReadAsync(ct).ConfigureAwait(false);
success = probeRuntime.GetStatus() == 0;
if (success)
state.Backoff.RecordSuccess(); // 05/STAB-8: device reachable — reset the data-path window
backoffVerdict = true; // 05/STAB-8: device reachable — reset the data-path window
}
catch (OperationCanceledException) when (ct.IsCancellationRequested)
{
@@ -525,7 +531,34 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
try { probeRuntime?.Dispose(); } catch { }
probeRuntime = null;
state.ProbeInitialized = false;
state.Backoff.RecordFailure(DateTime.UtcNow); // 05/STAB-8: keep the data-path window open
backoffVerdict = false; // 05/STAB-8: keep the data-path window open
}
// ConnectionBackoff is caller-serialized, and the data path now records under the
// per-device connect gate — so the probe records under it too, or its reset/increment
// could interleave with a concurrent cold-tag connect record and lose one. The gate is
// held by the data path only across ONE connect attempt, so this wait is bounded by a
// Forward-Open timeout; the data path never waits on anything the probe holds, so no
// deadlock is possible. Cancellation exits the loop as it does anywhere else.
if (backoffVerdict is { } verdict)
{
try
{
await state.ConnectGate.WaitAsync(ct).ConfigureAwait(false);
}
catch (OperationCanceledException)
{
break;
}
try
{
if (verdict) state.Backoff.RecordSuccess();
else state.Backoff.RecordFailure(DateTime.UtcNow);
}
finally
{
state.ConnectGate.Release();
}
}
TransitionDeviceState(state, success ? HostState.Running : HostState.Stopped);
@@ -590,22 +623,128 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
var plan = AbCipUdtReadPlanner.Build(
fullReferences, _tagsByRawPath, _options.EnableDeclarationOnlyUdtGrouping);
foreach (var group in plan.Groups)
await ReadGroupAsync(group, results, now, cancellationToken).ConfigureAwait(false);
var groups = plan.Groups;
var fallbacks = plan.Fallbacks;
var itemCount = groups.Count + fallbacks.Count;
if (itemCount == 0) return results;
foreach (var fb in plan.Fallbacks)
await ReadSingleAsync(fb, fullReferences[fb.OriginalIndex], results, now, cancellationToken).ConfigureAwait(false);
// Bounded-parallel window ACROSS work items (groups ∪ fallbacks). Each item still takes its
// OWN per-tag runtime lock inside ReadGroupAsync / ReadSingleAsync, so one libplctag handle
// is never touched concurrently (05/STAB-4) — the window only lets DIFFERENT tags' device
// round trips overlap, which a strictly sequential await chain made impossible. Results are
// index-addressed, so completion order is irrelevant. Per-item health verdicts are COLLECTED
// and written once after the batch: an inline write to the shared _health field would race.
var maxConcurrent = Math.Max(1, _options.MaxConcurrentReads);
var outcomes = new ReadOutcome[itemCount];
using var window = new SemaphoreSlim(maxConcurrent, maxConcurrent);
var tasks = new Task[itemCount];
for (var i = 0; i < itemCount; i++)
tasks[i] = RunItemAsync(i);
// Task.WhenAll completes only once EVERY item has finished (it does not short-circuit on the
// first fault), so disposing the window on the way out can never pull the semaphore out from
// under an item still waiting on it.
await Task.WhenAll(tasks).ConfigureAwait(false);
ApplyBatchHealth(outcomes, now);
return results;
async Task RunItemAsync(int slot)
{
// Acquire OUTSIDE the try: a wait that throws never acquired, so it must not release.
await window.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
if (slot < groups.Count)
{
outcomes[slot] = await ReadGroupAsync(groups[slot], results, now, cancellationToken)
.ConfigureAwait(false);
}
else
{
var fb = fallbacks[slot - groups.Count];
outcomes[slot] = await ReadSingleAsync(
fb, fullReferences[fb.OriginalIndex], results, now, cancellationToken)
.ConfigureAwait(false);
}
}
finally
{
window.Release();
}
}
}
private async Task ReadSingleAsync(
/// <summary>
/// One read work item's health verdict. Returned by <see cref="ReadSingleAsync"/> /
/// <see cref="ReadGroupAsync"/> instead of written to the shared health field inline, because
/// under the bounded-parallel read window those writes would race (last-writer-wins across
/// threads could report Healthy for a batch that mostly failed, or vice versa).
/// A <c>null</c> <see cref="State"/> means "no health signal" — a config-level miss such as an
/// unknown reference or unknown device, which never touched the wire and never touched health.
/// </summary>
private readonly record struct ReadOutcome(DriverState? State, string? Error)
{
/// <summary>The item produced no health signal at all.</summary>
public static ReadOutcome None => new(null, null);
/// <summary>The item read successfully.</summary>
public static ReadOutcome Success => new(DriverState.Healthy, null);
/// <summary>The item failed on the wire.</summary>
/// <param name="message">The failure detail surfaced as <c>LastError</c>.</param>
public static ReadOutcome Degraded(string message) => new(DriverState.Degraded, message);
}
private static int HealthRank(DriverState state) => state switch
{
// No ReadOutcome factory produces Faulted today (a read item is Healthy, Degraded, or no
// signal at all); the tier is future-proofing for an outcome source that reports it, so the
// worst-wins order stays correct without a second edit here.
DriverState.Faulted => 2,
DriverState.Degraded => 1,
_ => 0,
};
/// <summary>
/// Write the batch's health ONCE, after every work item has finished: worst outcome wins
/// (Faulted &gt; Degraded &gt; Healthy) and <c>LastSuccessfulRead</c> advances if ANY item
/// succeeded — the contract the comms-loss watchdog reads (a success advances it; a batch
/// where everything failed freezes it, which is what trips the watchdog).
/// </summary>
private void ApplyBatchHealth(ReadOnlySpan<ReadOutcome> outcomes, DateTime now)
{
var anySuccess = false;
DriverState? worst = null;
string? worstError = null;
foreach (var outcome in outcomes)
{
if (outcome.State is not { } state) continue;
if (state == DriverState.Healthy) anySuccess = true;
if (worst is null || HealthRank(state) > HealthRank(worst.Value))
{
worst = state;
worstError = outcome.Error;
}
}
// Nothing reached the wire (every reference was unknown / unroutable) — leave health alone,
// exactly as the per-item writes did before.
if (worst is not { } worstState) return;
_health = new DriverHealth(
worstState,
anySuccess ? now : _health.LastSuccessfulRead,
worstState == DriverState.Healthy ? null : worstError);
}
private async Task<ReadOutcome> ReadSingleAsync(
AbCipUdtReadFallback fb, string reference, DataValueSnapshot[] results, DateTime now, CancellationToken ct)
{
if (!_resolver.TryResolve(reference, out var def))
{
results[fb.OriginalIndex] = new DataValueSnapshot(null, AbCipStatusMapper.BadNodeIdUnknown, null, now);
return;
return ReadOutcome.None;
}
// A Structure tag whose Members are declared is a container —
// its bare name is readable via the whole-UDT grouping path (ReadGroupAsync), not the
@@ -614,12 +753,12 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
if (def.DataType == AbCipDataType.Structure && def.Members is { Count: > 0 })
{
results[fb.OriginalIndex] = new DataValueSnapshot(null, AbCipStatusMapper.BadNotSupported, null, now);
return;
return ReadOutcome.None;
}
if (!_devicesByRoutingKey.TryGetValue(def.DeviceHostAddress, out var device))
{
results[fb.OriginalIndex] = new DataValueSnapshot(null, AbCipStatusMapper.BadNodeIdUnknown, null, now);
return;
return ReadOutcome.None;
}
try
@@ -644,13 +783,11 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
EvictRuntime(device, def.Name);
results[fb.OriginalIndex] = new DataValueSnapshot(null,
AbCipStatusMapper.MapLibplctagStatus(status), null, now);
_health = new DriverHealth(DriverState.Degraded, _health.LastSuccessfulRead,
$"libplctag status {status} reading {reference}");
_logger.LogWarning(
"AbCip read returned non-zero libplctag status {LibplctagStatus} for tag {Tag} on device {Device}; " +
"evicting cached runtime so next call re-creates it",
status, reference, def.DeviceHostAddress);
return;
return ReadOutcome.Degraded($"libplctag status {status} reading {reference}");
}
var tagPath = AbCipTagPath.TryParse(def.TagPath);
@@ -662,7 +799,7 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
? runtime.DecodeArray(def.DataType, Math.Max(1, def.ElementCount))
: runtime.DecodeValue(def.DataType, bitIndex);
results[fb.OriginalIndex] = new DataValueSnapshot(value, AbCipStatusMapper.Good, now, now);
_health = new DriverHealth(DriverState.Healthy, now, null);
return ReadOutcome.Success;
}
finally
{
@@ -679,10 +816,10 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
EvictRuntime(device, def.Name);
results[fb.OriginalIndex] = new DataValueSnapshot(null,
AbCipStatusMapper.BadCommunicationError, null, now);
_health = new DriverHealth(DriverState.Degraded, _health.LastSuccessfulRead, ex.Message);
_logger.LogWarning(ex,
"AbCip read transport exception for tag {Tag} on device {Device}",
reference, def.DeviceHostAddress);
return ReadOutcome.Degraded(ex.Message);
}
}
@@ -693,7 +830,7 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
/// the mapped fault across every grouped member only — sibling groups + the
/// per-tag fallback list are unaffected.
/// </summary>
private async Task ReadGroupAsync(
private async Task<ReadOutcome> ReadGroupAsync(
AbCipUdtReadGroup group, DataValueSnapshot[] results, DateTime now, CancellationToken ct)
{
var parent = group.ParentDefinition;
@@ -701,7 +838,7 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
if (!_devicesByRoutingKey.TryGetValue(parent.DeviceHostAddress, out var device))
{
StampGroupStatus(group, results, now, AbCipStatusMapper.BadNodeIdUnknown);
return;
return ReadOutcome.None;
}
try
@@ -722,13 +859,11 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
EvictRuntime(device, parent.Name);
var mapped = AbCipStatusMapper.MapLibplctagStatus(status);
StampGroupStatus(group, results, now, mapped);
_health = new DriverHealth(DriverState.Degraded, _health.LastSuccessfulRead,
$"libplctag status {status} reading UDT {group.ParentName}");
_logger.LogWarning(
"AbCip whole-UDT read returned non-zero libplctag status {LibplctagStatus} for parent {Parent} " +
"on device {Device}; {MemberCount} member values stamped with mapped status",
status, group.ParentName, parent.DeviceHostAddress, group.Members.Count);
return;
return ReadOutcome.Degraded($"libplctag status {status} reading UDT {group.ParentName}");
}
foreach (var member in group.Members)
@@ -736,7 +871,7 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
var value = runtime.DecodeValueAt(member.Definition.DataType, member.Offset, bitIndex: null);
results[member.OriginalIndex] = new DataValueSnapshot(value, AbCipStatusMapper.Good, now, now);
}
_health = new DriverHealth(DriverState.Healthy, now, null);
return ReadOutcome.Success;
}
finally
{
@@ -751,10 +886,10 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
{
EvictRuntime(device, parent.Name);
StampGroupStatus(group, results, now, AbCipStatusMapper.BadCommunicationError);
_health = new DriverHealth(DriverState.Degraded, _health.LastSuccessfulRead, ex.Message);
_logger.LogWarning(ex,
"AbCip whole-UDT read transport exception for parent {Parent} on device {Device}",
group.ParentName, parent.DeviceHostAddress);
return ReadOutcome.Degraded(ex.Message);
}
}
@@ -976,6 +1111,13 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
/// + initialises the libplctag Tag; subsequent calls reuse the cached handle for the
/// lifetime of the device.
/// </summary>
/// <remarks>
/// The COLD path (no cached handle) runs single-flight per device under
/// <see cref="DeviceState.ConnectGate"/> so the backoff check, the Forward-Open, and the
/// failure/success record form one atomic sequence — see that property for why. The HOT path
/// (cached handle) never touches the gate, so the bounded-parallel read window is undiminished
/// in steady state.
/// </remarks>
private async Task<IAbCipTagRuntime> EnsureTagRuntimeAsync(
DeviceState device, AbCipTagDefinition def, CancellationToken ct)
{
@@ -985,37 +1127,50 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
?? throw new InvalidOperationException(
$"AbCip tag '{def.Name}' has malformed TagPath '{def.TagPath}'.");
// 05/STAB-8: inside an open backoff window a data-path caller fails fast — no runtime
// create, no Forward-Open. The malformed-path config error above still surfaces (a permanent
// fault must never be masked by the window); only the expensive create+init is throttled. The
// probe loop bypasses this (its own runtime) and drives the failure/success recording.
if (!device.Backoff.ShouldAttempt(DateTime.UtcNow))
throw new ConnectionThrottledException(
$"AbCip runtime create for '{def.Name}' on {device.Options.HostAddress} throttled — backoff window open after a recent Forward-Open failure.");
// Review I-1 — an array tag (the EXPLICIT IsArray flag, incl. a 1-element array) sets
// libplctag's elem_count so the read pulls every element in one CIP transaction; the read
// path then boxes them into a typed CLR array. Scalar tags pass count 1 + IsArray false.
var runtime = _tagFactory.Create(
device.BuildCreateParams(
parsed.ToLibplctagName(), _options.Timeout, def.ElementCount, IsArrayTag(def)));
await device.ConnectGate.WaitAsync(ct).ConfigureAwait(false);
try
{
await runtime.InitializeAsync(ct).ConfigureAwait(false);
}
catch
{
// Re-check under the gate: a sibling cold tag may have connected (or a concurrent write
// may have materialised this very handle) while this caller waited.
if (device.Runtimes.TryGetValue(def.Name, out var raced)) return raced;
// 05/STAB-8: inside an open backoff window a data-path caller fails fast — no runtime
// create, no Forward-Open. The malformed-path config error above still surfaces (a permanent
// fault must never be masked by the window); only the expensive create+init is throttled. The
// probe loop uses its own runtime, and records under this same gate.
if (!device.Backoff.ShouldAttempt(DateTime.UtcNow))
throw new ConnectionThrottledException(
$"AbCip runtime create for '{def.Name}' on {device.Options.HostAddress} throttled — backoff window open after a recent Forward-Open failure.");
// Review I-1 — an array tag (the EXPLICIT IsArray flag, incl. a 1-element array) sets
// libplctag's elem_count so the read pulls every element in one CIP transaction; the read
// path then boxes them into a typed CLR array. Scalar tags pass count 1 + IsArray false.
var runtime = _tagFactory.Create(
device.BuildCreateParams(
parsed.ToLibplctagName(), _options.Timeout, def.ElementCount, IsArrayTag(def)));
try
{
await runtime.InitializeAsync(ct).ConfigureAwait(false);
}
catch
{
runtime.Dispose();
device.Backoff.RecordFailure(DateTime.UtcNow); // 05/STAB-8: open the backoff window
throw;
}
device.Backoff.RecordSuccess(); // 05/STAB-8: Forward-Open succeeded — reset the window
// The gate makes a cold create single-flight, but a concurrent EvictRuntime (a failed read
// on this handle) can still land between the re-check and the add; only the first TryAdd
// wins. Dispose the loser so it doesn't leak a native tag handle.
if (device.Runtimes.TryAdd(def.Name, runtime))
return runtime;
runtime.Dispose();
device.Backoff.RecordFailure(DateTime.UtcNow); // 05/STAB-8: open the backoff window
throw;
return device.Runtimes[def.Name];
}
finally
{
device.ConnectGate.Release();
}
device.Backoff.RecordSuccess(); // 05/STAB-8: Forward-Open succeeded — reset the window
// Two concurrent callers can both miss the cache + both initialize a runtime; only the
// first TryAdd wins. Dispose the loser so it doesn't leak a native tag handle.
if (device.Runtimes.TryAdd(def.Name, runtime))
return runtime;
runtime.Dispose();
return device.Runtimes[def.Name];
}
/// <summary>
@@ -1350,6 +1505,29 @@ public sealed class AbCipDriver : IDriver, IReadable, IWritable, ITagDiscovery,
/// (its own runtime) and drives failure/success so a recovered device resets the window.
/// </summary>
public ConnectionBackoff Backoff { get; } = new(TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(30));
/// <summary>
/// Per-device connect gate. <see cref="ConnectionBackoff"/> is documented as
/// caller-serialized — it mutates plain fields — so EVERY touch of <see cref="Backoff"/>
/// plus the connect attempt it guards runs under this semaphore. Task 14's bounded-parallel
/// read window otherwise let up to <c>MaxConcurrentReads</c> work items run the
/// ShouldAttempt → create+Initialize → RecordFailure/RecordSuccess sequence at once for
/// DIFFERENT cold tags on the SAME device: every one passed <c>ShouldAttempt</c> before any
/// failure was recorded (N concurrent Forward-Opens at a down device) and the unsynchronized
/// <c>_consecutiveFailures++</c> lost increments, weakening the 05/STAB-8 window into a
/// reconnect storm. Cold-tag creation is therefore single-flight per device; an
/// already-created runtime bypasses the gate entirely, so the steady-state read path keeps
/// the full window.
/// <para>
/// Deliberately NOT disposed in <see cref="DisposeHandles"/>, unlike the per-tag gates:
/// the probe loop also waits on it, and <see cref="AbCipDriver.ShutdownAsync"/>'s wait
/// for that loop is bounded (10 s), so disposal could race a still-unwinding probe into
/// an <see cref="ObjectDisposedException"/>. A <see cref="SemaphoreSlim"/> whose
/// <c>AvailableWaitHandle</c> was never touched holds no unmanaged resource, and the
/// whole <see cref="DeviceState"/> is dropped on reinitialize.
/// </para>
/// </summary>
public SemaphoreSlim ConnectGate { get; } = new(1, 1);
/// <summary>Gets or sets the current host state of this device.</summary>
public HostState HostState { get; set; } = HostState.Unknown;
/// <summary>Gets or sets the UTC timestamp when the host state was last changed.</summary>
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.AbCip/AbCipDriverFactoryExtensions.cs
+8
View File
@@ -85,6 +85,7 @@ public static class AbCipDriverFactoryExtensions
EnableAlarmProjection = dto.EnableAlarmProjection ?? false,
AlarmPollInterval = TimeSpan.FromMilliseconds(dto.AlarmPollIntervalMs ?? 1_000),
EnableDeclarationOnlyUdtGrouping = dto.EnableDeclarationOnlyUdtGrouping ?? false,
MaxConcurrentReads = dto.MaxConcurrentReads ?? 8,
};
}
@@ -147,6 +148,13 @@ public static class AbCipDriverFactoryExtensions
/// </summary>
public bool? EnableDeclarationOnlyUdtGrouping { get; init; }
/// <summary>
/// Gets or sets the maximum number of read work items one ReadAsync batch may have in
/// flight against the controller at once. Bounds concurrency across tags only; the per-tag
/// runtime lock still serialises operations on any one handle. Defaults to 8; 1 is sequential.
/// </summary>
public int? MaxConcurrentReads { get; init; }
/// <summary>
/// Gets or sets the alarm poll interval in milliseconds.
/// </summary>
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.AbLegacy/AbLegacyDriver.cs
+4 -3
View File
@@ -489,10 +489,11 @@ public sealed class AbLegacyDriver : IDriver, IReadable, IWritable, ITagDiscover
Task.FromResult(_poll.Subscribe(fullReferences, publishingInterval));
/// <inheritdoc />
public Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken)
public async Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken)
{
_poll.Unsubscribe(handle);
return Task.CompletedTask;
// Async teardown: the engine drains the in-flight poll without blocking the caller's
// thread — this runs inside a DriverInstanceActor mailbox turn.
await _poll.UnsubscribeAsync(handle).ConfigureAwait(false);
}
// ---- IHostConnectivityProbe ----
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.FOCAS/FocasDriver.cs
+4 -3
View File
@@ -838,10 +838,11 @@ public sealed class FocasDriver : IDriver, IReadable, IWritable, ITagDiscovery,
Task.FromResult(_poll.Subscribe(fullReferences, publishingInterval));
/// <inheritdoc />
public Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken)
public async Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken)
{
_poll.Unsubscribe(handle);
return Task.CompletedTask;
// Async teardown: the engine drains the in-flight poll without blocking the caller's
// thread — this runs inside a DriverInstanceActor mailbox turn.
await _poll.UnsubscribeAsync(handle).ConfigureAwait(false);
}
// ---- IHostConnectivityProbe ----
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/FasterLogHistorizationOutbox.cs
+54 -9
View File
@@ -99,23 +99,68 @@ public sealed class FasterLogHistorizationOutbox : IHistorizationOutbox
{
ArgumentNullException.ThrowIfNull(entry);
byte[] payload = HistorizationOutboxEntrySerializer.Serialize(entry);
long startAddress = await _log.EnqueueAsync(payload, ct).ConfigureAwait(false);
// A one-entry batch is byte-for-byte the single-append path (one enqueue, one indexing step,
// one PerEntry commit), so there is no second copy of the drop-oldest policy.
await AppendBatchAsync([entry], ct).ConfigureAwait(false);
}
/// <inheritdoc />
public async ValueTask AppendBatchAsync(IReadOnlyList<HistorizationOutboxEntry> entries, CancellationToken ct)
{
ArgumentNullException.ThrowIfNull(entries);
if (entries.Count == 0)
return;
// Validate before ANY enqueue: a null halfway through would otherwise leave the batch's
// earlier entries in the log with no in-memory index entry (invisible but not reclaimable).
for (int i = 0; i < entries.Count; i++)
ArgumentNullException.ThrowIfNull(entries[i]);
// INCREMENTAL bookkeeping: each entry is indexed under _state immediately after ITS OWN
// enqueue returns — never accumulated and applied after the loop. That is what makes a
// mid-loop failure (cancellation, I/O fault) safe: entries 0..i-1 are then both physically in
// the log AND in _live/_index, so PeekBatchAsync (which scans the raw log) and RemoveAsync
// (which looks the id up in _index) agree about them, and the head can advance past them.
// Deferring the indexing until after the loop stranded that prefix physically-present but
// untracked — peekable, un-removable (RemoveAsync no-ops on an unknown id), hence redelivered
// to the historian on every drain pass and blocking truncation until a restart's
// RecoverState() re-indexed it — and it also escaped the capacity bound.
for (int i = 0; i < entries.Count; i++)
{
HistorizationOutboxEntry entry = entries[i];
byte[] payload = HistorizationOutboxEntrySerializer.Serialize(entry);
long start = await _log.EnqueueAsync(payload, ct).ConfigureAwait(false);
lock (_state)
{
_index[entry.Id] = _live.AddLast(new LiveEntry(entry.Id, start));
}
}
if (_commitMode == HistorizationCommitMode.PerEntry)
{
// PerEntry: durable before returning. Periodic skips this — the background timer (and
// Dispose) commit on their cadence (accepted throughput/latency trade-off).
// Group commit: ONE fsync makes the whole batch durable before returning, replacing the
// N per-entry fsyncs a loop of AppendAsync would have cost. Periodic skips this — the
// background timer (and Dispose) commit on their cadence (accepted trade-off).
//
// Residual (documented on IHistorizationOutbox.AppendBatchAsync): a throw before this
// commit leaves the enqueued prefix tracked but not yet committed, exactly the exposure
// the single-entry path has always had between its enqueue and its commit. Such entries
// are served from memory until a restart, where RecoverState() rebuilds from the
// committed log and simply does not see them.
await _log.CommitAsync(ct).ConfigureAwait(false);
}
long? truncateTo = null;
lock (_state)
{
LinkedListNode<LiveEntry> node = _live.AddLast(new LiveEntry(entry.Id, startAddress));
_index[entry.Id] = node;
// Drop-oldest on overflow. The new head is the start address of whatever entry survives
// at the front (or the tail if the log emptied); truncate to the furthest such address.
// Drop-oldest on overflow, run ONCE for the whole batch (an over-capacity batch sheds
// every surplus entry here). The new head is the start address of whatever entry
// survives at the front (or the tail if the log emptied); truncate to the furthest such
// address, so the batch needs at most one truncate + one extra commit.
//
// A batch that throws mid-loop skips this pass, so _live can sit transiently above
// _capacity — the same convergent overflow RecoverState() documents, shed by the next
// append's while-loop.
while (_capacity > 0 && _live.Count > _capacity)
{
LinkedListNode<LiveEntry> oldest = _live.First!;
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectAgentClient.cs
+31 -7
View File
@@ -231,7 +231,7 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable
var delivered = 0L;
while (true)
{
var part = await reader.ReadNextPartAsync(live).ConfigureAwait(false);
var part = await reader.ReadNextPartBytesAsync(live).ConfigureAwait(false);
if (part is null)
{
// The caller cancelling is the ONE contracted way out; anything else is reported.
@@ -244,7 +244,7 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable
// Agent's real keep-alive is a well-formed observation-free MTConnectStreams document,
// and that one IS yielded — it advances the sequence, so swallowing it would strand the
// caller's `from` behind the Agent's buffer.
if (string.IsNullOrWhiteSpace(part))
if (IsFramingOnly(part))
{
continue;
}
@@ -562,6 +562,24 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable
return parsed.GetLeftPart(UriPartial.Path).TrimEnd('/');
}
/// <summary>
/// Whether a multipart part carries nothing but transport filler — the byte-level equivalent
/// of the <c>IsNullOrWhiteSpace</c> check the string-shaped framer used to run, and the same
/// ASCII set the streams parser trims off a chunk before parsing it.
/// </summary>
private static bool IsFramingOnly(ReadOnlySpan<byte> part)
{
foreach (var value in part)
{
if (value is not ((byte)' ' or (byte)'\t' or (byte)'\r' or (byte)'\n'))
{
return false;
}
}
return true;
}
/// <summary>
/// Frames a <c>multipart/x-mixed-replace</c> body into individual part payloads, reading
/// incrementally and never buffering more than the part currently in flight.
@@ -610,10 +628,16 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable
public MTConnectStreamEndReason EndReason { get; private set; } = MTConnectStreamEndReason.ConnectionClosed;
/// <summary>
/// Reads the next part's payload, or <c>null</c> once the closing delimiter is seen or
/// the Agent closes the connection — see <see cref="EndReason"/> for which.
/// Reads the next part's payload as its raw bytes, or <c>null</c> once the closing
/// delimiter is seen or the Agent closes the connection — see <see cref="EndReason"/>
/// for which.
/// </summary>
public async Task<string?> ReadNextPartAsync(CancellationToken ct)
/// <remarks>
/// Deliberately <b>not</b> decoded to a string: the one consumer is the streams parser,
/// which reads UTF-8 off a stream itself. Decoding here would materialise every chunk of
/// a never-ending stream twice for a document that is parsed once and dropped.
/// </remarks>
public async Task<byte[]?> ReadNextPartBytesAsync(CancellationToken ct)
{
if (_finished)
{
@@ -652,7 +676,7 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable
body = await ReadToNextDelimiterAsync(ct).ConfigureAwait(false);
}
return body is null ? null : Encoding.UTF8.GetString(body);
return body;
}
private void WarnOnceAboutMissingContentLength()
@@ -669,7 +693,7 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable
uri);
}
private string? Finish(MTConnectStreamEndReason reason)
private byte[]? Finish(MTConnectStreamEndReason reason)
{
_finished = true;
EndReason = reason;
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectStreamsParser.cs
+77
View File
@@ -1,4 +1,5 @@
using System.Globalization;
using System.Runtime.InteropServices;
using System.Xml;
using System.Xml.Linq;
@@ -121,6 +122,33 @@ internal static class MTConnectStreamsParser
return Build(document);
}
/// <summary>
/// Parses one <c>/sample</c> chunk straight from the bytes the multipart framer produced.
/// </summary>
/// <remarks>
/// The <c>/sample</c> hot path used to materialise every chunk twice — once as a decoded
/// string, once again as the trimmed copy handed to the parser — for a document that is read
/// once and thrown away. Parsing from the bytes drops both copies; <see cref="XDocument"/>
/// does its own UTF-8 decoding off the stream. The framing tolerance the string overload's
/// <c>Trim</c> provided is preserved by <see cref="TrimFraming"/>, which slices rather than
/// copies.
/// </remarks>
/// <param name="xml">The chunk's raw bytes; a UTF-8 BOM and surrounding framing whitespace are tolerated.</param>
/// <exception cref="InvalidDataException">As for <see cref="Parse(string)"/>.</exception>
public static MTConnectStreamsResult Parse(ReadOnlyMemory<byte> xml)
{
var trimmed = TrimFraming(xml);
if (trimmed.IsEmpty)
{
throw new InvalidDataException(
"MTConnect streams response was empty; expected an MTConnectStreams XML document.");
}
using var stream = AsStream(trimmed);
return Parse(stream);
}
/// <summary>Parses a <c>/current</c> response read from a stream.</summary>
/// <param name="stream">A stream positioned at the start of the XML document.</param>
/// <exception cref="InvalidDataException">As for <see cref="Parse(string)"/>.</exception>
@@ -141,6 +169,55 @@ internal static class MTConnectStreamsParser
return Build(document);
}
/// <summary>
/// Slices off what the string overload's <c>Trim('', ' ', '\t', '\r', '\n')</c> removed:
/// a UTF-8 byte-order mark and the multipart framing's surrounding line breaks, either of
/// which <see cref="XmlReader"/> rejects ahead of the XML declaration.
/// </summary>
private static ReadOnlyMemory<byte> TrimFraming(ReadOnlyMemory<byte> xml)
{
var span = xml.Span;
var start = 0;
var end = span.Length;
while (start < end)
{
if (end - start >= 3 && span[start] == 0xEF && span[start + 1] == 0xBB && span[start + 2] == 0xBF)
{
start += 3;
continue;
}
if (!IsFramingWhitespace(span[start]))
{
break;
}
start++;
}
while (end > start && IsFramingWhitespace(span[end - 1]))
{
end--;
}
return xml[start..end];
}
private static bool IsFramingWhitespace(byte value) =>
value is (byte)' ' or (byte)'\t' or (byte)'\r' or (byte)'\n';
/// <summary>
/// Views the (already trimmed) bytes as a stream without copying them where the memory is
/// array-backed — which it always is on the <c>/sample</c> path, whose framer hands over the
/// part's own <c>byte[]</c>.
/// </summary>
private static MemoryStream AsStream(ReadOnlyMemory<byte> xml) =>
MemoryMarshal.TryGetArray(xml, out ArraySegment<byte> segment) && segment.Array is not null
? new MemoryStream(segment.Array, segment.Offset, segment.Count, writable: false)
: new MemoryStream(xml.ToArray(), writable: false);
private static MTConnectStreamsResult Build(XDocument document)
{
var root = document.Root
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/ModbusDriver.cs
+118 -6
View File
@@ -87,6 +87,25 @@ public sealed class ModbusDriver
private CancellationTokenSource? _probeCts;
private CancellationTokenSource? _reprobeCts;
/// <summary>
/// Guards <see cref="InitializeAsync"/> / <see cref="ReinitializeAsync"/> /
/// <see cref="ShutdownAsync"/> / <see cref="DisposeAsync"/> against each other. The three
/// fields above are published by one lifecycle operation and released by the next, so two
/// concurrent operations would dispose each other's freshly-built transport or null
/// <c>_transport</c> out from under an in-flight connect. Not hypothetical:
/// <c>DriverInstanceActor</c> dispatches initialize on a fire-and-forget <c>Task.Run</c> and
/// its <c>AdoptConfigDuringInit</c> can fire twice back to back. Same shape as
/// <c>MqttDriver._lifecycleGate</c>.
/// </summary>
private readonly SemaphoreSlim _lifecycleGate = new(1, 1);
/// <summary>
/// Bounded wait the teardown paths take on <see cref="_lifecycleGate"/>. A wedged initialize
/// (a connect that never returns) must not turn shutdown or dispose into a hang — past the
/// timeout they tear down anyway, which is the lesser evil of the two.
/// </summary>
private static readonly TimeSpan LifecycleGateTimeout = TimeSpan.FromSeconds(10);
private DriverHealth _health = new(DriverState.Unknown, null, null);
/// <summary>Occurs when a subscribed tag value changes.</summary>
@@ -251,10 +270,49 @@ public sealed class ModbusDriver
/// <inheritdoc />
public async Task InitializeAsync(string driverConfigJson, CancellationToken cancellationToken)
{
await _lifecycleGate.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
await InitializeCoreAsync(driverConfigJson, cancellationToken).ConfigureAwait(false);
}
finally
{
_lifecycleGate.Release();
}
}
/// <summary>The gate-held body of <see cref="InitializeAsync"/>.</summary>
private async Task InitializeCoreAsync(string driverConfigJson, CancellationToken cancellationToken)
{
WriteHealth(new DriverHealth(DriverState.Initializing, null, null));
try
{
// Release whatever a PREVIOUS initialize published before building the replacements. All
// three fields assigned below (_transport, _probeCts, _reprobeCts) were simply overwritten,
// so a second initialize on a live driver stranded a connected transport plus two running
// background loops that no later dispose could reach. Reachable without any teardown of
// ours: DriverInstanceActor re-runs InitializeAsync with no ShutdownAsync when an ApplyDelta
// lands while the driver is in Connecting/Reconnecting (AdoptConfigDuringInit).
// Deliberately NOT the shared TeardownAsync(): that also disposes _poll, which tears down
// every ACTIVE SUBSCRIPTION — and nothing re-subscribes them afterwards, because a
// mid-life re-initialize is not a shutdown the OPC UA layer knows to re-drive. A caller
// that genuinely wants the subscriptions gone calls ShutdownAsync. A first initialize
// sees three nulls, so this block is a no-op on the initial call.
try { _probeCts?.Cancel(); } catch { }
_probeCts?.Dispose();
_probeCts = null;
try { _reprobeCts?.Cancel(); } catch { }
_reprobeCts?.Dispose();
_reprobeCts = null;
if (_transport is not null)
{
await _transport.DisposeAsync().ConfigureAwait(false);
_transport = null;
}
// #516: re-parse the supplied DriverConfig so a config change delivered through the IDriver
// contract is honoured. Without this the driver keeps serving the options it was CONSTRUCTED
// with, so an operator's edit is discarded while the deployment still seals green. An empty /
@@ -309,12 +367,48 @@ public sealed class ModbusDriver
/// <inheritdoc />
public async Task ReinitializeAsync(string driverConfigJson, CancellationToken cancellationToken)
{
await ShutdownAsync(cancellationToken);
await InitializeAsync(driverConfigJson, cancellationToken);
// Holds the gate across BOTH halves and calls the ungated cores: taking it once here is what
// makes the shutdown-then-initialize pair atomic. Routing through the public (gated) methods
// instead would both self-deadlock and leave a window between them for a concurrent
// initialize to slip into.
await _lifecycleGate.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
await ShutdownCoreAsync().ConfigureAwait(false);
await InitializeCoreAsync(driverConfigJson, cancellationToken).ConfigureAwait(false);
}
finally
{
_lifecycleGate.Release();
}
}
/// <inheritdoc />
public async Task ShutdownAsync(CancellationToken cancellationToken)
{
if (await _lifecycleGate.WaitAsync(LifecycleGateTimeout, cancellationToken).ConfigureAwait(false))
{
try
{
await ShutdownCoreAsync().ConfigureAwait(false);
}
finally
{
_lifecycleGate.Release();
}
}
else
{
_logger.LogWarning(
"Modbus lifecycle gate still held at shutdown; tearing the transport down anyway. Driver={DriverInstanceId}",
_driverInstanceId);
await ShutdownCoreAsync().ConfigureAwait(false);
}
}
/// <summary>The gate-held body of <see cref="ShutdownAsync"/>, shared with
/// <see cref="ReinitializeAsync"/>.</summary>
private async Task ShutdownCoreAsync()
{
var lastRead = ReadHealth().LastSuccessfulRead;
await TeardownAsync().ConfigureAwait(false);
@@ -1247,13 +1341,14 @@ public sealed class ModbusDriver
Task.FromResult(_poll.Subscribe(fullReferences, publishingInterval));
/// <inheritdoc />
public Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken)
public async Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken)
{
_poll.Unsubscribe(handle);
// Async teardown: the engine drains the in-flight poll without blocking the caller's
// thread — this runs inside a DriverInstanceActor mailbox turn.
await _poll.UnsubscribeAsync(handle).ConfigureAwait(false);
// Drop any partially-collected cycle buffer so a torn-down subscription leaks nothing.
// Unsubscribe awaits the loop task, so no onChange can re-add after this point.
// The teardown above awaits the loop task, so no onChange can re-add after this point.
_cycleBatchItems.TryRemove(handle, out _);
return Task.CompletedTask;
}
// ---- IHostConnectivityProbe ----
@@ -1669,6 +1764,23 @@ public sealed class ModbusDriver
/// <returns>A task that represents the asynchronous operation.</returns>
public async ValueTask DisposeAsync()
{
if (await _lifecycleGate.WaitAsync(LifecycleGateTimeout).ConfigureAwait(false))
{
try
{
await TeardownAsync().ConfigureAwait(false);
}
finally
{
_lifecycleGate.Release();
}
return;
}
_logger.LogWarning(
"Modbus lifecycle gate still held at dispose; tearing the transport down anyway. Driver={DriverInstanceId}",
_driverInstanceId);
await TeardownAsync().ConfigureAwait(false);
}
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttSubscriptionManager.cs
+37 -1
View File
@@ -1,3 +1,4 @@
using System.Collections;
using System.Collections.Concurrent;
using System.Collections.Frozen;
using System.Globalization;
@@ -748,8 +749,43 @@ public sealed class MqttSubscriptionManager
snapshot.StatusCode);
}
// Unchanged-value suppression. Every poll driver gets this dedupe for free at the edge from
// PollGroupEngine; MQTT is push-based, so a broker that timer-republishes an identical payload
// would otherwise re-trigger the ENTIRE downstream fan-out (actor hop → dependency mux → OPC UA
// sink write) per message. The cache is still freshened — a read must serve the current
// timestamp — and only the notification is skipped. "Cached but not raised" is already a valid
// state on this path (see Raise: an authored-but-unsubscribed tag does exactly that).
//
// Sparkplug bypasses this entirely. SparkplugIngestor.cs updates Values and raises directly
// (its own Values.Update()/Raise() calls, not through this Deliver method), so Sparkplug B
// ingestion gets no unchanged-value suppression — deliberately out of scope here.
var prior = Values.Read(def.Name);
var unchanged = prior.StatusCode == snapshot.StatusCode && !ValuesAreDifferent(prior.Value, snapshot.Value);
Values.Update(def.Name, snapshot);
Raise(def.Name, snapshot);
if (!unchanged) Raise(def.Name, snapshot);
}
/// <summary>
/// Returns <c>true</c> when <paramref name="previous"/> and <paramref name="current"/> represent
/// different values. Mirrors <c>PollGroupEngine.ValuesAreDifferent</c> — deliberately duplicated
/// rather than shared, because that one is private to the poll-loop engine this push-based driver
/// does not run through.
/// </summary>
/// <remarks>
/// Arrays are compared structurally: a JSON array payload decodes to a fresh array instance on
/// every message, so reference equality would report every republish as a change and suppress
/// nothing at all for exactly the tags whose payloads are largest.
/// </remarks>
/// <param name="previous">The last cached value for the reference.</param>
/// <param name="current">The value just extracted from the inbound message.</param>
/// <returns><c>true</c> when the two differ.</returns>
private static bool ValuesAreDifferent(object? previous, object? current)
{
if (previous is Array prevArr && current is Array currArr)
return !StructuralComparisons.StructuralEqualityComparer.Equals(prevArr, currArr);
return !Equals(previous, current);
}
/// <summary>
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7.Contracts/S7DriverOptions.cs
+50
View File
@@ -79,6 +79,56 @@ public sealed class S7DriverOptions
[Display(Name = "Probe timeout (seconds)", Description = "Connection test timeout. Default 5s.", GroupName = "Diagnostics")]
[Range(1, 60)]
public int ProbeTimeoutSeconds { get; init; } = 5;
/// <summary>
/// Largest hole, in bytes, the read planner will read THROUGH when merging two neighbouring
/// tags into one block request. It is a budget for deliberately wasted bytes: raising it
/// merges more aggressively (fewer wire requests, more bytes nobody asked for), lowering it
/// to <c>0</c> merges only tags that are strictly adjacent or overlapping.
/// </summary>
/// <remarks>
/// <para>
/// <b>Any value <c>&lt;= -1</c> disables span grouping entirely</b> — every tag is read
/// on its own, exactly as the driver behaved before grouping existed. That is the
/// escape hatch for a CPU that dislikes wide block reads, and the control arm the
/// parity tests read through.
/// </para>
/// <para>
/// Grouping cannot make a read fail where the per-tag reads would have succeeded: a
/// span never extends past its LAST member's end, so every byte it requests lies inside
/// a range some tag was already going to read. Timer and Counter tags are never grouped
/// — their offset is a device number, not a byte address.
/// </para>
/// </remarks>
[Display(Name = "Span read max gap (bytes)",
Description = "Largest hole read through when merging neighbouring tags into one block request. -1 disables grouping.",
GroupName = "Performance")]
public int SpanReadMaxGapBytes { get; init; } = DefaultSpanReadMaxGapBytes;
/// <summary>Default for <see cref="SpanReadMaxGapBytes"/>, shared with the config DTO so the two can't drift.</summary>
public const int DefaultSpanReadMaxGapBytes = 16;
/// <summary>
/// Largest single planned span, in bytes. A run of tags that would exceed it is broken into
/// several spans; a single tag wider than the cap still gets its own span, because one tag
/// is never split across requests.
/// </summary>
/// <remarks>
/// This is <b>not</b> a correctness bound: S7.Net transparently chunks a <c>ReadBytes</c>
/// call larger than the negotiated PDU into several wire requests, so an oversized span
/// reads correctly either way. The cap exists to bound wasted-gap amplification — without
/// it, a long sparse run of tags each separated by an under-budget hole would coalesce into
/// one enormous request that is mostly padding. The 220-byte default sits just under the
/// usable payload of the common 240-byte S7-300/400 PDU, so a typical span is one request
/// on the wire.
/// </remarks>
[Display(Name = "Span read max span (bytes)",
Description = "Largest single planned block read. Bounds wasted-gap amplification, not correctness.",
GroupName = "Performance")]
public int SpanReadMaxSpanBytes { get; init; } = DefaultSpanReadMaxSpanBytes;
/// <summary>Default for <see cref="SpanReadMaxSpanBytes"/>, shared with the config DTO so the two can't drift.</summary>
public const int DefaultSpanReadMaxSpanBytes = 220;
}
public sealed class S7ProbeOptions
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7/S7Driver.cs
+334 -40
View File
@@ -500,6 +500,25 @@ public sealed class S7Driver
// ---- IReadable ----
/// <inheritdoc />
/// <remarks>
/// <para>
/// The batch is served in two passes under ONE gate acquisition. First
/// <see cref="S7ReadSpanPlanner"/> groups the resolved tags into contiguous
/// <c>(area, DB)</c> byte spans and each span is read with a single
/// <c>ReadBytesAsync</c>, its members decoded by slicing the returned block — this is
/// what turns one wire request per tag into one per contiguous block. Whatever the
/// planner declines (Timer/Counter, whose offsets are device numbers) plus any span not
/// worth issuing (see <see cref="PrefersPerTagRead"/>) then goes through the unchanged
/// per-tag <see cref="ReadOneAsync"/> path.
/// </para>
/// <para>
/// Setting <c>SpanReadMaxGapBytes</c> below zero skips the first pass entirely, which
/// restores the pre-grouping behaviour exactly. Either way the result array is written
/// by original index, so the caller sees its own ordering regardless of how the batch
/// was partitioned, and both passes share <see cref="RecordReadFailure"/> so a fault
/// stamps the same status whichever path observed it.
/// </para>
/// </remarks>
public async Task<IReadOnlyList<DataValueSnapshot>> ReadAsync(
IReadOnlyList<string> fullReferences, CancellationToken cancellationToken)
{
@@ -534,17 +553,69 @@ public sealed class S7Driver
return results;
}
// Resolve every reference first. An unresolvable one is a miss, stamped here and never
// offered to the planner — grouping must not be able to change what "unknown tag" means.
var resolved = new S7TagDefinition?[fullReferences.Count];
for (var i = 0; i < fullReferences.Count; i++)
{
var name = fullReferences[i];
if (!_resolver.TryResolve(name, out var tag))
{
if (_resolver.TryResolve(fullReferences[i], out var tag))
resolved[i] = tag;
else
results[i] = new DataValueSnapshot(null, StatusBadNodeIdUnknown, null, now);
continue;
}
// Indexes the per-tag path must serve: everything, when grouping is off; otherwise the
// planner's leftovers (Timer/Counter) plus any tag whose address won't parse.
var perTag = new List<int>();
if (_options.SpanReadMaxGapBytes >= 0)
{
List<(int OriginalIndex, S7TagDefinition Tag, S7ParsedAddress Addr)>? plannable = null;
for (var i = 0; i < fullReferences.Count; i++)
{
if (resolved[i] is not { } tag) continue;
if (!TryGetParsedAddress(tag, out var addr))
{
// Defensive: every tag pre-parses at init. A tag that somehow didn't goes
// to the per-tag path, where ReadOneAsync's Parse throws the FormatException
// the catch ladder has always mapped to BadCommunicationError.
perTag.Add(i);
continue;
}
(plannable ??= []).Add((i, tag, addr));
}
if (plannable is not null)
{
var plan = S7ReadSpanPlanner.Build(
plannable, _options.SpanReadMaxGapBytes, _options.SpanReadMaxSpanBytes);
foreach (var span in plan.Spans)
{
// A span that saves no round-trip is not worth changing the wire shape for.
if (PrefersPerTagRead(span))
{
perTag.Add(span.Members[0].OriginalIndex);
continue;
}
await ReadSpanAsync(plc, span, results, now, cancellationToken).ConfigureAwait(false);
}
perTag.AddRange(plan.LeftoverIndexes);
}
}
else
{
for (var i = 0; i < fullReferences.Count; i++)
if (resolved[i] is not null) perTag.Add(i);
}
// Sorted so a batch is served in reference order regardless of how it was partitioned —
// the per-tag tail is otherwise ordered by the planner's grouping, not by the caller's.
perTag.Sort();
foreach (var i in perTag)
{
try
{
var value = await ReadOneAsync(plc, tag, cancellationToken).ConfigureAwait(false);
var value = await ReadOneAsync(plc, resolved[i]!, cancellationToken).ConfigureAwait(false);
results[i] = new DataValueSnapshot(value, 0u, now, now);
_health = new DriverHealth(DriverState.Healthy, now, null);
}
@@ -555,42 +626,14 @@ public sealed class S7Driver
// dead (we hold _gate) and propagate: the caller cancelled, nobody consumes the
// batch. Marking is unconditional on OCE (not routed through the type classifier):
// whether bytes moved is unknowable, and a false positive costs one cheap reopen
// versus permanent desync for a false negative.
// versus permanent desync for a false negative. Kept AT the call site rather
// than inside RecordReadFailure so the rethrow preserves the original stack.
_plcDead = true;
throw;
}
catch (NotSupportedException)
{
results[i] = new DataValueSnapshot(null, StatusBadNotSupported, null, now);
}
catch (PlcException pex) when (IsAccessDenied(pex))
{
// PUT/GET-disabled (S7-1200/1500) / access-protection — a permanent
// configuration fault, NOT a transient one. Blind retry is wasted effort,
// so map it to BadNotSupported and flag the driver as a config alert
// (Faulted) rather than Degraded — per driver-specs.md §5.
results[i] = new DataValueSnapshot(null, StatusBadNotSupported, null, now);
_health = new DriverHealth(DriverState.Faulted, _health.LastSuccessfulRead,
"S7 access denied — enable PUT/GET communication in TIA Portal " +
$"(Protection & Security) for this CPU. PLC reported: {pex.Message}");
}
catch (PlcException pex)
{
// A genuine device-layer fault (CPU error, hardware fault) — transient
// enough to keep retrying; report BadDeviceFailure and degrade health. A
// connection-fatal PlcException (socket drop / ConnectionError) additionally
// marks the handle dead so the next read reopens it.
MarkConnectionDeadIfFatal(pex);
results[i] = new DataValueSnapshot(null, StatusBadDeviceFailure, null, now);
_health = new DriverHealth(DriverState.Degraded, _health.LastSuccessfulRead, pex.Message);
}
catch (Exception ex)
{
// Socket / timeout / transport fault — mark the handle dead when it's a
// connection loss so EnsureConnectedAsync reopens on the next call.
MarkConnectionDeadIfFatal(ex);
results[i] = new DataValueSnapshot(null, StatusBadCommunicationError, null, now);
_health = new DriverHealth(DriverState.Degraded, _health.LastSuccessfulRead, ex.Message);
results[i] = new DataValueSnapshot(null, RecordReadFailure(ex), null, now);
}
}
}
@@ -598,6 +641,182 @@ public sealed class S7Driver
return results;
}
/// <summary>
/// True when a planned span is better served by the per-tag path: it holds exactly ONE
/// narrow scalar, so issuing it as a block read saves no round-trip and buys nothing.
/// </summary>
/// <remarks>
/// The point of grouping is to replace N wire requests with one. A lone narrow tag is one
/// wire request either way — but a block read reaches the CPU differently from S7.Net's
/// per-address string read, so converting it would change the observable wire shape of every
/// single-tag read on the server for zero benefit. A lone ARRAY or wide/buffer scalar is NOT
/// demoted: those already read a block on the per-tag path, so serving them as a
/// single-member span is byte-for-byte the request they always issued.
/// </remarks>
/// <param name="span">The planned span to judge.</param>
/// <returns><c>true</c> when the span's single member should go through <see cref="ReadOneAsync"/>.</returns>
private static bool PrefersPerTagRead(S7ReadSpanPlanner.Span span)
=> span.Members.Count == 1
&& span.Members[0].Tag.ArrayCount is not >= 1
&& !IsBufferType(span.Members[0].Tag, span.Members[0].Addr);
/// <summary>
/// Reads one planned span — a SINGLE <c>ReadBytesAsync</c> serving every member — and
/// decodes each member from its slice of the returned block.
/// </summary>
/// <remarks>
/// A span read is one wire request standing in for many per-tag requests, so a transport
/// fault that used to spoil one tag now spoils all of them: every member is stamped with
/// the SAME status <see cref="RecordReadFailure"/> would have produced for that exception on
/// the per-tag path, and health degrades identically. Decoding, by contrast, is per member
/// inside its own guard — a single mis-authored tag (declared type vs. address size) must
/// not spoil the neighbours that shared its block.
/// </remarks>
/// <param name="plc">The live connection (caller holds <see cref="_gate"/>).</param>
/// <param name="span">The planned block read.</param>
/// <param name="results">The batch's result array, written in place at each member's original index.</param>
/// <param name="now">The batch timestamp, shared by every tag in the batch.</param>
/// <param name="ct">Cancellation token for the read.</param>
/// <returns>A task that completes once every member of the span has been stamped.</returns>
private async Task ReadSpanAsync(
IS7Plc plc, S7ReadSpanPlanner.Span span, DataValueSnapshot[] results, DateTime now, CancellationToken ct)
{
byte[] block;
try
{
block = await plc.ReadBytesAsync(
ToS7NetArea(span.Area), span.DbNumber, span.StartByte, span.LengthBytes, ct)
.ConfigureAwait(false)
?? throw new System.IO.InvalidDataException(
$"S7.Net returned null block for {span.Area} db{span.DbNumber} " +
$"@{span.StartByte}+{span.LengthBytes}");
}
catch (OperationCanceledException)
{
// Same reasoning as the per-tag path: a cancelled block read leaves the single gated
// stream at an untrustworthy position, so the handle must be reopened (STAB-15a).
_plcDead = true;
throw;
}
catch (Exception ex)
{
var status = RecordReadFailure(ex);
foreach (var member in span.Members)
results[member.OriginalIndex] = new DataValueSnapshot(null, status, null, now);
return;
}
foreach (var member in span.Members)
{
try
{
var value = DecodeMember(member, block);
results[member.OriginalIndex] = new DataValueSnapshot(value, 0u, now, now);
_health = new DriverHealth(DriverState.Healthy, now, null);
}
catch (Exception ex)
{
// Decoding is pure and synchronous — no cancellation can surface here, so there is
// deliberately no OperationCanceledException arm to shadow the block-read one above.
results[member.OriginalIndex] = new DataValueSnapshot(null, RecordReadFailure(ex), null, now);
}
}
}
/// <summary>
/// Decodes one span member from its slice of the block, routing to exactly the decoder
/// <see cref="ReadOneAsync"/> would have chosen for that tag, in the same order.
/// </summary>
/// <param name="member">The member to decode, carrying its slice position and width.</param>
/// <param name="block">The block the span read.</param>
/// <returns>The decoded value, boxed identically to the per-tag path's.</returns>
private static object DecodeMember(S7ReadSpanPlanner.Member member, byte[] block)
{
// The two block decoders take byte[] (they hand the buffer to S7.Net's own
// S7String/DateTime/Timer/Counter FromByteArray helpers, which are byte[]-only), so a
// member that is not the whole block is copied out. Narrow scalars need no copy at all —
// DecodeNarrowScalar reads straight from a ReadOnlySpan positioned at the tag's own byte.
if (member.Tag.ArrayCount is >= 1)
return DecodeArrayBlock(member.Tag, member.Addr, SliceForMember(member, block));
if (IsBufferType(member.Tag, member.Addr))
return DecodeScalarBlock(member.Tag, member.Addr, SliceForMember(member, block));
return DecodeNarrowScalar(member.Tag, member.Addr, block.AsSpan(member.RelativeOffset, member.WidthBytes));
}
/// <summary>
/// The member's own bytes as a <c>byte[]</c> for the block decoders. Returns the block
/// itself — no copy — when the member IS the whole block, which is the single-member span
/// every array / wide scalar produces today.
/// </summary>
/// <param name="member">The member whose bytes are wanted.</param>
/// <param name="block">The block the span read.</param>
/// <returns>The member's bytes, starting at its own first byte.</returns>
private static byte[] SliceForMember(S7ReadSpanPlanner.Member member, byte[] block)
=> member.RelativeOffset == 0 && member.WidthBytes == block.Length
? block
: block[member.RelativeOffset..(member.RelativeOffset + member.WidthBytes)];
/// <summary>
/// The tag's parsed address: the table built at init, with an on-demand parse as the
/// defensive fallback for a tag that isn't in it. Mirrors <see cref="ReadOneAsync"/>'s
/// lookup, but never throws — an unparseable address returns <c>false</c> so the caller can
/// route the tag to the per-tag path, where the <see cref="FormatException"/> surfaces
/// through the ladder exactly as it always has.
/// </summary>
/// <param name="tag">The tag whose address is wanted.</param>
/// <param name="addr">The parsed address, valid only when this returns <c>true</c>.</param>
/// <returns><c>true</c> when the address is known or parses.</returns>
private bool TryGetParsedAddress(S7TagDefinition tag, out S7ParsedAddress addr)
=> _parsedByRawPath.TryGetValue(tag.Name, out addr) || S7AddressParser.TryParse(tag.Address, out addr);
/// <summary>
/// The read-failure classification ladder, shared by the per-tag path and the span path so
/// a failed span stamps every member with exactly the status the same exception would have
/// produced on a single tag. Applies the health / dead-handle side effects and returns the
/// status code the caller must stamp.
/// </summary>
/// <remarks>
/// <see cref="OperationCanceledException"/> is deliberately NOT handled here: it is control
/// flow rather than a per-tag status (it marks the handle dead and propagates), and
/// rethrowing it from a helper would rewrite the stack. Both call sites catch it first.
/// </remarks>
/// <param name="ex">The exception observed on a read.</param>
/// <returns>The OPC UA status code every affected tag must be stamped with.</returns>
private uint RecordReadFailure(Exception ex)
{
switch (ex)
{
case NotSupportedException:
return StatusBadNotSupported;
case PlcException pex when IsAccessDenied(pex):
// PUT/GET-disabled (S7-1200/1500) / access-protection — a permanent configuration
// fault, NOT a transient one. Blind retry is wasted effort, so map it to
// BadNotSupported and flag the driver as a config alert (Faulted) rather than
// Degraded — per driver-specs.md §5.
_health = new DriverHealth(DriverState.Faulted, _health.LastSuccessfulRead,
"S7 access denied — enable PUT/GET communication in TIA Portal " +
$"(Protection & Security) for this CPU. PLC reported: {pex.Message}");
return StatusBadNotSupported;
case PlcException pex:
// A genuine device-layer fault (CPU error, hardware fault) — transient enough to
// keep retrying; report BadDeviceFailure and degrade health. A connection-fatal
// PlcException (socket drop / ConnectionError) additionally marks the handle dead
// so the next read reopens it.
MarkConnectionDeadIfFatal(pex);
_health = new DriverHealth(DriverState.Degraded, _health.LastSuccessfulRead, pex.Message);
return StatusBadDeviceFailure;
default:
// Socket / timeout / transport fault — mark the handle dead when it's a connection
// loss so EnsureConnectedAsync reopens on the next call.
MarkConnectionDeadIfFatal(ex);
_health = new DriverHealth(DriverState.Degraded, _health.LastSuccessfulRead, ex.Message);
return StatusBadCommunicationError;
}
}
private async Task<object> ReadOneAsync(IS7Plc plc, S7TagDefinition tag, CancellationToken ct)
{
// Every tag pre-parses its address at init (_parsedByRawPath); the on-demand
@@ -681,7 +900,7 @@ public sealed class S7Driver
/// <param name="tag">Tag definition carrying the <see cref="S7DataType"/>.</param>
/// <param name="addr">Parsed address — its <see cref="S7Area"/> routes Timer/Counter into the seam regardless of <see cref="S7DataType"/>.</param>
/// <returns><c>true</c> if the tag routes through the byte-buffer codec.</returns>
private static bool IsBufferType(S7TagDefinition tag, S7ParsedAddress addr) =>
internal static bool IsBufferType(S7TagDefinition tag, S7ParsedAddress addr) =>
addr.Area is S7Area.Timer or S7Area.Counter
|| tag.DataType is S7DataType.Int64 or S7DataType.UInt64 or S7DataType.Float64
or S7DataType.String or S7DataType.DateTime;
@@ -986,6 +1205,80 @@ public sealed class S7Driver
((uint)block[offset] << 24) | ((uint)block[offset + 1] << 16)
| ((uint)block[offset + 2] << 8) | block[offset + 3];
/// <summary>
/// Byte width one narrow scalar (Bit/Byte/Word/DWord) occupies inside a contiguous block —
/// the stride a span planner uses to size and slice a group read. Delegates to
/// <see cref="ElementByteSize"/> so the scalar and array paths can never disagree about a
/// size's width (a Bit costs a whole byte: S7 block reads are byte-granular).
/// </summary>
/// <param name="addr">Parsed address carrying the access <see cref="S7Size"/>.</param>
/// <returns>Byte width: Bit/Byte = 1, Word = 2, DWord = 4.</returns>
internal static int NarrowScalarWidth(S7ParsedAddress addr) => ElementByteSize(addr.Size);
/// <summary>
/// Pure decode step — turns the tag's own slice of a contiguous big-endian block into the
/// SEMANTIC CLR value declared by its <see cref="S7DataType"/>, boxed as
/// <see cref="object"/>. No network I/O and no driver state: this is the span-read twin of
/// <see cref="ReinterpretRawValue"/> (which reinterprets the boxed value S7.Net's
/// per-tag string <c>ReadAsync</c> returns), and it deliberately mirrors that method's
/// sign/width/conversion semantics arm for arm so a tag reads identically whether it was
/// served by one wire request of its own or by one block read shared with its neighbours.
/// </summary>
/// <remarks>
/// <paramref name="slice"/> is positioned at the tag's OWN first byte —
/// <c>slice[0]</c> is the tag's first byte, so <see cref="S7ParsedAddress.ByteOffset"/> is
/// the caller's business and is never re-applied here. Only the leading
/// <see cref="NarrowScalarWidth"/> bytes are consumed; a longer slice (the rest of the
/// block) is fine. Wide/structured types (Int64/UInt64/Float64/String/DateTime) and the
/// Timer/Counter areas are NOT narrow — they route through
/// <see cref="DecodeScalarBlock"/> — and fall into the mismatch arm here so a mis-routed
/// tag fails loudly instead of decoding garbage from a truncated slice.
/// </remarks>
/// <param name="tag">Tag definition carrying the narrow <see cref="S7DataType"/>.</param>
/// <param name="addr">Parsed address carrying the access <see cref="S7Size"/> and (for Bool) the bit position.</param>
/// <param name="slice">The tag's slice of the block, big-endian, at least <see cref="NarrowScalarWidth"/> bytes long.</param>
/// <returns>The decoded value boxed as <see cref="object"/>, with the same box type <see cref="ReinterpretRawValue"/> produces.</returns>
internal static object DecodeNarrowScalar(S7TagDefinition tag, S7ParsedAddress addr, ReadOnlySpan<byte> slice)
{
// Each arm boxes explicitly: a bare switch expression would unify bool/byte/ushort/short/
// uint/int/float to a common type and box THAT, mis-typing every narrow tag (the same trap
// DecodeScalarBlock documents). The box type is part of the contract — the publish pipeline
// and the OPC UA variable's DataType both key off it.
switch (tag.DataType, addr.Size)
{
// DBX / M{n}.{b}: S7.Net's string read returns the already-extracted bool. From a block
// we extract it ourselves at the parsed bit position (0-7, guaranteed by the parser).
case (S7DataType.Bool, S7Size.Bit):
return (object)((slice[0] & (1 << addr.BitOffset)) != 0);
// DBB: the whole byte, unsigned (S7.Net boxes a byte).
case (S7DataType.Byte, S7Size.Byte):
return (object)slice[0];
// DBW: one big-endian word. UInt16 is the raw word; Int16 is the SAME bits reinterpreted
// unchecked (0x8000 => -32768), exactly as ReinterpretRawValue casts S7.Net's ushort.
case (S7DataType.UInt16, S7Size.Word):
return (object)System.Buffers.Binary.BinaryPrimitives.ReadUInt16BigEndian(slice);
case (S7DataType.Int16, S7Size.Word):
return (object)unchecked((short)System.Buffers.Binary.BinaryPrimitives.ReadUInt16BigEndian(slice));
// DBD: one big-endian dword. UInt32 raw; Int32 the same bits unchecked; Float32 the same
// bits reinterpreted as IEEE-754 single (never a numeric conversion).
case (S7DataType.UInt32, S7Size.DWord):
return (object)System.Buffers.Binary.BinaryPrimitives.ReadUInt32BigEndian(slice);
case (S7DataType.Int32, S7Size.DWord):
return (object)unchecked((int)System.Buffers.Binary.BinaryPrimitives.ReadUInt32BigEndian(slice));
case (S7DataType.Float32, S7Size.DWord):
return (object)BitConverter.UInt32BitsToSingle(
System.Buffers.Binary.BinaryPrimitives.ReadUInt32BigEndian(slice));
default:
throw new System.IO.InvalidDataException(
$"S7 narrow-span Read type-mismatch: tag '{tag.Name}' declared {tag.DataType} but address " +
$"'{tag.Address}' parsed as Size={addr.Size}");
}
}
/// <summary>
/// Pure reinterpret step — converts the boxed value that S7.Net returns (always an
/// unsigned type: <c>bool</c>, <c>byte</c>, <c>ushort</c>, <c>uint</c>) into the
@@ -1439,10 +1732,11 @@ public sealed class S7Driver
=> Task.FromResult(_poll.Subscribe(fullReferences, publishingInterval));
/// <inheritdoc />
public Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken)
public async Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken)
{
_poll.Unsubscribe(handle);
return Task.CompletedTask;
// Async teardown: the engine drains the in-flight poll without blocking the caller's
// thread — this runs inside a DriverInstanceActor mailbox turn.
await _poll.UnsubscribeAsync(handle).ConfigureAwait(false);
}
/// <summary>
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7/S7DriverFactoryExtensions.cs
+10
View File
@@ -69,6 +69,11 @@ public static class S7DriverFactoryExtensions
// tags. The driver maps each RawTagEntry's TagConfig blob through S7TagDefinitionFactory
// at Initialize; the legacy pre-declared DTO tag path (Name/Address → BuildTag) is retired.
RawTags = dto.RawTags is { Count: > 0 } ? [.. dto.RawTags] : [],
// Block-read grouping knobs. Mapped explicitly like every other key on this DTO — the
// options object is never deserialised directly — so omitting them here would leave the
// documented "-1 disables grouping" escape hatch unreachable from a deployed config.
SpanReadMaxGapBytes = dto.SpanReadMaxGapBytes ?? S7DriverOptions.DefaultSpanReadMaxGapBytes,
SpanReadMaxSpanBytes = dto.SpanReadMaxSpanBytes ?? S7DriverOptions.DefaultSpanReadMaxSpanBytes,
Probe = new S7ProbeOptions
{
Enabled = dto.Probe?.Enabled ?? true,
@@ -122,6 +127,11 @@ public static class S7DriverFactoryExtensions
public List<RawTagEntry>? RawTags { get; init; }
/// <summary>Gets the probe configuration.</summary>
public S7ProbeDto? Probe { get; init; }
/// <summary>Gets the largest hole the read planner reads through when merging tags into one
/// block request; any value below zero disables block-read grouping. Null keeps the default.</summary>
public int? SpanReadMaxGapBytes { get; init; }
/// <summary>Gets the largest single planned block read in bytes. Null keeps the default.</summary>
public int? SpanReadMaxSpanBytes { get; init; }
}
/// <summary>Data transfer object for S7 probe configuration.</summary>
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7/S7ReadSpanPlanner.cs
+198
View File
@@ -0,0 +1,198 @@
namespace ZB.MOM.WW.OtOpcUa.Driver.S7;
/// <summary>
/// Pure planner that turns one read batch into contiguous <c>(area, DB)</c> byte spans, so the
/// driver can serve many tags with one <c>ReadBytesAsync</c> instead of one wire request per
/// tag. No I/O, no driver state — the grouping rules are decided here and executed by
/// <see cref="S7Driver.ReadAsync"/>.
/// </summary>
/// <remarks>
/// <para>
/// <b>The safety invariant is that a span never extends past its last member's end.</b> A
/// span may deliberately read bytes nobody asked for BETWEEN its members (that is what the
/// gap budget buys), but every byte it requests lies inside a range some member was going
/// to read anyway. A grouped read therefore cannot fault (area overrun, DB overrun) where
/// the individual per-tag reads would have succeeded.
/// </para>
/// <para>
/// <b>Timer and Counter tags are never grouped.</b> Their
/// <see cref="S7ParsedAddress.ByteOffset"/> holds the timer/counter DEVICE NUMBER rather
/// than a byte address, and S7.Net addresses those areas in per-device units — merging one
/// with a byte-addressed neighbour, or even with another Timer, would read from a
/// completely unrelated place. They come back as leftovers for the per-tag path.
/// </para>
/// </remarks>
internal static class S7ReadSpanPlanner
{
/// <summary>One tag inside a planned span, positioned relative to the span's start byte.</summary>
/// <param name="OriginalIndex">The tag's index in the caller's read batch — where its result must be written.</param>
/// <param name="Tag">The resolved tag definition (carries the data type, array count and string length).</param>
/// <param name="Addr">The tag's parsed address (carries the area, DB, access size and bit position).</param>
/// <param name="WidthBytes">Bytes this tag occupies — the exact number its decoder consumes.</param>
/// <param name="RelativeOffset">The tag's first byte expressed against <see cref="Span.StartByte"/>; the slice offset into the block.</param>
internal sealed record Member(
int OriginalIndex,
S7TagDefinition Tag,
S7ParsedAddress Addr,
int WidthBytes,
int RelativeOffset);
/// <summary>One planned block read: a contiguous byte range in one area, serving several tags.</summary>
/// <param name="Area">The S7 area the block is read from.</param>
/// <param name="DbNumber">Data-block number (0 for non-DB areas).</param>
/// <param name="StartByte">First byte of the block — the lowest member's byte offset.</param>
/// <param name="LengthBytes">Block length — always exactly the furthest member's end minus <paramref name="StartByte"/>.</param>
/// <param name="Members">The tags this block serves, in ascending byte order.</param>
internal sealed record Span(
S7Area Area,
int DbNumber,
int StartByte,
int LengthBytes,
IReadOnlyList<Member> Members);
/// <summary>The planned grouping of one read batch.</summary>
/// <param name="Spans">Block reads to issue, one wire request each.</param>
/// <param name="LeftoverIndexes">Batch indexes the planner declined — the caller reads these one at a time.</param>
internal sealed record Plan(IReadOnlyList<Span> Spans, IReadOnlyList<int> LeftoverIndexes);
/// <summary>
/// Groups the resolved tags of one read batch into block-read spans.
/// </summary>
/// <param name="resolved">
/// The batch's resolved tags, each paired with its parsed address and its index in the
/// caller's reference list. Unresolvable references must be handled by the caller and left
/// out of this list.
/// </param>
/// <param name="maxGapBytes">
/// Largest hole to read through when merging (inclusive). <c>0</c> merges only strictly
/// adjacent or overlapping tags; any value below zero disables grouping and returns every
/// index as a leftover.
/// </param>
/// <param name="maxSpanBytes">
/// Largest planned span. A run that would exceed it is broken; a single tag wider than it
/// still forms its own span, because a tag is never split across requests.
/// </param>
/// <returns>The spans to read plus the indexes the caller must read individually.</returns>
internal static Plan Build(
IReadOnlyList<(int OriginalIndex, S7TagDefinition Tag, S7ParsedAddress Addr)> resolved,
int maxGapBytes,
int maxSpanBytes)
{
ArgumentNullException.ThrowIfNull(resolved);
var leftovers = new List<int>();
// The off switch. Checked here (as well as at the call site) so the disabled behaviour is
// a property of the planner and can be proven without standing up a driver.
if (maxGapBytes < 0)
{
foreach (var (index, _, _) in resolved) leftovers.Add(index);
return new Plan([], leftovers);
}
List<(int Index, S7TagDefinition Tag, S7ParsedAddress Addr, int Width)>? eligible = null;
foreach (var (index, tag, addr) in resolved)
{
// Timer/Counter offsets are device numbers, not byte addresses — see the type remarks.
if (addr.Area is S7Area.Timer or S7Area.Counter)
{
leftovers.Add(index);
continue;
}
(eligible ??= []).Add((index, tag, addr, MemberWidthBytes(tag, addr)));
}
if (eligible is null) return new Plan([], leftovers);
var spans = new List<Span>();
// Group by (area, DB): two areas — or two data blocks — are separate address spaces, so a
// byte range in one says nothing about the same range in the other.
foreach (var group in eligible.GroupBy(e => (e.Addr.Area, e.Addr.DbNumber)))
{
OpenSpan? open = null;
// Ascending byte order. OrderBy is stable, so tags sharing a byte (two Bools at
// different bit positions) keep their batch order inside the span.
foreach (var (index, tag, addr, width) in group.OrderBy(e => e.Addr.ByteOffset))
{
var start = addr.ByteOffset;
var end = start + width; // exclusive
if (open is not null)
{
// A negative gap means the tags overlap (a word inside an already-open dword,
// or a second Bool in the same byte) — that is a gap of zero or better, so it
// always passes a non-negative budget without any clamping.
var gap = start - open.EndByte;
var mergedEnd = Math.Max(open.EndByte, end);
if (gap <= maxGapBytes && mergedEnd - open.StartByte <= maxSpanBytes)
{
open.Members.Add(new Member(index, tag, addr, width, start - open.StartByte));
// Max, not `end`: an overlapping member must never pull the span's end back.
open.EndByte = mergedEnd;
continue;
}
spans.Add(Seal(group.Key.Area, group.Key.DbNumber, open));
}
// Start a fresh span. Deliberately unconditional — a tag wider than maxSpanBytes
// gets its own oversized span rather than being split, and because the merge test
// above measures the RESULTING length, that oversized span can never absorb a
// neighbour afterwards.
open = new OpenSpan(start, end);
open.Members.Add(new Member(index, tag, addr, width, 0));
}
if (open is not null) spans.Add(Seal(group.Key.Area, group.Key.DbNumber, open));
}
return new Plan(spans, leftovers);
}
/// <summary>
/// Bytes one tag occupies inside a contiguous block. The three arms are checked in exactly
/// the order <see cref="S7Driver.ReadAsync"/> routes a tag to its decoder, so the width
/// planned here is always the width the decoder that runs will consume.
/// </summary>
/// <param name="tag">The resolved tag definition.</param>
/// <param name="addr">The tag's parsed address.</param>
/// <returns>The tag's byte width.</returns>
internal static int MemberWidthBytes(S7TagDefinition tag, S7ParsedAddress addr)
{
if (tag.ArrayCount is >= 1)
return tag.ArrayCount.Value * S7Driver.ElementByteSize(addr.Size);
if (S7Driver.IsBufferType(tag, addr))
return S7Driver.ScalarByteWidth(tag, addr);
return S7Driver.NarrowScalarWidth(addr);
}
/// <summary>Freezes an in-progress span into its immutable <see cref="Span"/> form.</summary>
/// <param name="area">The area the span reads from.</param>
/// <param name="dbNumber">The data-block number.</param>
/// <param name="open">The in-progress span.</param>
/// <returns>The sealed span; its length is exactly the furthest member's end minus the start.</returns>
private static Span Seal(S7Area area, int dbNumber, OpenSpan open)
=> new(area, dbNumber, open.StartByte, open.EndByte - open.StartByte, open.Members);
/// <summary>
/// Mutable accumulator for a span still being extended. <see cref="StartByte"/> is fixed at
/// creation (members arrive in ascending order, so nothing can ever start earlier);
/// <see cref="EndByte"/> is exclusive and only ever grows.
/// </summary>
/// <param name="startByte">First byte of the span.</param>
/// <param name="endByte">Exclusive end byte of the span so far.</param>
private sealed class OpenSpan(int startByte, int endByte)
{
/// <summary>First byte of the span — fixed for the span's lifetime.</summary>
public int StartByte { get; } = startByte;
/// <summary>Exclusive end byte of the furthest member seen so far.</summary>
public int EndByte { get; set; } = endByte;
/// <summary>Members accumulated so far, in ascending byte order.</summary>
public List<Member> Members { get; } = [];
}
}
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql/SqlDriver.cs
+4 -3
View File
@@ -570,10 +570,11 @@ public sealed class SqlDriver
}
/// <inheritdoc />
public Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken)
public async Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken)
{
_poll.Unsubscribe(handle);
return Task.CompletedTask;
// Async teardown: the engine drains the in-flight poll without blocking the caller's
// thread — this runs inside a DriverInstanceActor mailbox turn.
await _poll.UnsubscribeAsync(handle).ConfigureAwait(false);
}
// ---- IHostConnectivityProbe ----
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT/TwinCATDriver.cs
+5 -4
View File
@@ -576,17 +576,18 @@ public sealed class TwinCATDriver : IDriver, IReadable, IWritable, ITagDiscovery
}
/// <inheritdoc />
public Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken)
public async Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken)
{
if (handle is NativeSubscriptionHandle native && _nativeSubs.TryRemove(native.Id, out var sub))
{
// NativeRegistration.Dispose removes itself from its device registry (so a later replay
// won't touch it) and disposes its live handle.
foreach (var r in sub.Registrations) { try { r.Dispose(); } catch { } }
return Task.CompletedTask;
return;
}
_poll.Unsubscribe(handle);
return Task.CompletedTask;
// Async teardown: the engine drains the in-flight poll without blocking the caller's
// thread — this runs inside a DriverInstanceActor mailbox turn.
await _poll.UnsubscribeAsync(handle).ConfigureAwait(false);
}
private sealed record NativeSubscriptionHandle(long Id) : ISubscriptionHandle
src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Pages/Deployments.razor
+26 -11
View File
@@ -6,13 +6,13 @@
@using ZB.MOM.WW.OtOpcUa.Configuration
@using ZB.MOM.WW.OtOpcUa.Configuration.Entities
@using ZB.MOM.WW.OtOpcUa.Configuration.Enums
@using ZB.MOM.WW.OtOpcUa.ControlPlane.AdminOperations
@attribute [Authorize(Policy = AdminUiPolicies.ConfigEditor)]
@inject IDbContextFactory<OtOpcUaConfigDbContext> DbFactory
@inject IAdminOperationsClient AdminOps
@inject AuthenticationStateProvider AuthState
@inject ZB.MOM.WW.OtOpcUa.AdminUI.Deployments.DeploymentDriftProbe DriftProbe
@rendermode InteractiveServer
<PageTitle>Deployments</PageTitle>
@@ -66,7 +66,18 @@
</table>
@code {
private IReadOnlyList<Deployment> _deployments = Array.Empty<Deployment>();
// Blob-free projection: the Deployment entity carries a varbinary(max) ArtifactBlob (a whole-fleet
// gzip config snapshot), which materialising the full entity would drag across the wire 50 times
// over just to render six scalar columns.
private sealed record DeploymentRow(
Guid DeploymentId,
string RevisionHash,
DeploymentStatus Status,
string CreatedBy,
DateTime CreatedAtUtc,
DateTime? SealedAtUtc);
private IReadOnlyList<DeploymentRow> _deployments = Array.Empty<DeploymentRow>();
private bool _busy;
private bool _lastSuccess;
private string? _lastMessage;
@@ -83,19 +94,23 @@
_deployments = await db.Deployments
.AsNoTracking()
.OrderByDescending(d => d.CreatedAtUtc)
.Select(d => new DeploymentRow(
d.DeploymentId,
d.RevisionHash,
d.Status,
d.CreatedBy,
d.CreatedAtUtc,
d.SealedAtUtc))
.Take(50)
.ToListAsync();
// Drift: if no sealed deployment yet, no drift to report. Otherwise compare the latest
// sealed revision hash to a fresh snapshot of the live-edit state.
// Drift: if no sealed deployment yet, no drift to report. Otherwise the probe compares the
// latest sealed revision hash to the live-edit state — recomposing only when the ConfigEdit
// audit watermark has moved, so a re-render of this page is not a full fleet snapshot.
var latestSealed = _deployments.FirstOrDefault(d => d.Status == DeploymentStatus.Sealed);
if (latestSealed is null)
{
_drift = null;
return;
}
var current = await ConfigComposer.SnapshotAndFlattenAsync(db);
_drift = !string.Equals(current.RevisionHash, latestSealed.RevisionHash, StringComparison.Ordinal);
_drift = latestSealed is null
? null
: await DriftProbe.GetDriftAsync(latestSealed.RevisionHash, CancellationToken.None);
}
private async Task StartDeploymentAsync()
src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Pages/Fleet.razor
+46 -7
View File
@@ -12,7 +12,7 @@
@using ZB.MOM.WW.OtOpcUa.Configuration.Enums
@inject IClusterRoleInfo Cluster
@inject IDbContextFactory<OtOpcUaConfigDbContext> DbFactory
@implements IDisposable
@implements IAsyncDisposable
<div class="d-flex justify-content-between align-items-center mb-3">
<h4 class="mb-0">Fleet status</h4>
@@ -106,6 +106,18 @@ else
private DateTime? _lastRefreshUtc;
private Timer? _timer;
// Overlap guard: a tick whose DB query outruns the refresh interval must not stack behind the
// next one. A manual Refresh landing on top of an in-flight tick is a no-op by the same guard.
private int _tickRunning;
// Page-lifetime cancellation: cancelled in DisposeAsync() so a tick in flight when the circuit
// goes away aborts its query instead of running it to completion against a component that's gone.
private readonly CancellationTokenSource _cts = new();
// Set first thing in DisposeAsync. A tick that resumes after teardown must not call
// StateHasChanged — the renderer is gone and it throws, inside a task nobody awaits.
private volatile bool _disposed;
protected override async Task OnInitializedAsync()
{
await LoadAsync();
@@ -118,17 +130,18 @@ else
private async Task LoadAsync()
{
_refreshing = true;
StateHasChanged();
if (Interlocked.Exchange(ref _tickRunning, 1) == 1) return;
try
{
await using var db = await DbFactory.CreateDbContextAsync();
_refreshing = true;
StateHasChanged();
await using var db = await DbFactory.CreateDbContextAsync(_cts.Token);
// Project the most-recent NodeDeploymentState per node — that's the row the
// DriverHostActor most recently touched, regardless of which deployment it was for.
var states = await db.NodeDeploymentStates.AsNoTracking()
.GroupBy(s => s.NodeId)
.Select(g => g.OrderByDescending(s => s.StartedAtUtc).First())
.ToListAsync();
.ToListAsync(_cts.Token);
var byNode = states.ToDictionary(s => s.NodeId);
// Union with current Akka driver members so a freshly-joined node that has no
@@ -151,10 +164,20 @@ else
}).ToList();
_lastRefreshUtc = DateTime.UtcNow;
}
catch (Exception ex) when (ex is OperationCanceledException or ObjectDisposedException)
{
// Circuit disposed while a tick was in flight — a cancelled tick is a clean exit, not a
// fault (the discarded timer task would otherwise swallow it silently).
}
finally
{
// Release first: a StateHasChanged on a torn-down circuit must never leave the guard set.
Volatile.Write(ref _tickRunning, 0);
_refreshing = false;
StateHasChanged();
// This one sits OUTSIDE the try, so it has no catch to fall into — and a render
// request against a disposed renderer throws. Skip it once the page is gone; there
// is nothing left to render to.
if (!_disposed) StateHasChanged();
}
}
@@ -168,7 +191,23 @@ else
private static string StatusLabel(NodeDeploymentStatus? status) => status?.ToString() ?? "waiting";
public void Dispose() => _timer?.Dispose();
public async ValueTask DisposeAsync()
{
_disposed = true;
// Cancel before draining so a tick already in flight aborts its queries rather than running
// them to completion against a component that's gone.
_cts.Cancel();
// Drain the timer so an in-flight callback can't touch a component that's already gone
// (System.Threading.Timer's async dispose awaits any in-flight callback — .NET 6+).
if (_timer is not null) await _timer.DisposeAsync();
// ...but the callback body is `_ = InvokeAsync(LoadAsync)`, so that await covered only the
// SCHEDULING of the tick, never the tick itself: a query can still be running against
// _cts.Token right now. Disposing the source out from under it turns the clean
// cancellation the line above arranged into an ObjectDisposedException mid-query, so skip
// the dispose while a tick holds it. An undisposed CancellationTokenSource with no timer
// and no materialised WaitHandle costs nothing and is collected with the component.
if (Volatile.Read(ref _tickRunning) == 0) _cts.Dispose();
}
private sealed record NodeRow(
string NodeId,
src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Pages/Hosts.razor
+73 -22
View File
@@ -329,46 +329,75 @@ else
// HostsDriverRow instance and would otherwise close a dialog an operator just opened.
private string? _controlsDriverId;
// Overlap guard: a tick whose config load outruns the 5 s interval must not stack behind the
// next one. A manual Refresh landing on top of an in-flight tick is a no-op by the same guard.
private int _tickRunning;
// Page-lifetime cancellation: cancelled in DisposeAsync() so a tick in flight when the circuit
// goes away aborts its queries instead of running them against a component that's gone.
private readonly CancellationTokenSource _cts = new();
// Set first thing in DisposeAsync. A tick that resumes after teardown must not call
// StateHasChanged — the renderer is gone and it throws, inside a task nobody awaits.
private volatile bool _disposed;
protected override async Task OnInitializedAsync()
{
Refresh();
await LoadConfigAsync();
RebuildDriverGroups();
DriverStore.SnapshotChanged += OnSnapshotChanged;
_timer = new Timer(_ => _ = InvokeAsync(async () =>
{
try
{
Refresh();
await LoadConfigAsync();
RebuildDriverGroups();
StateHasChanged();
}
catch (Exception ex) when (ex is ObjectDisposedException or OperationCanceledException)
{
// Circuit disposed while a tick was in flight — ignore (the discarded task would
// otherwise swallow this silently). Mirrors DriverStatusPanel's drain-on-dispose.
}
}), null,
_timer = new Timer(_ => _ = InvokeAsync(TickAsync), null,
TimeSpan.FromSeconds(RefreshIntervalSeconds),
TimeSpan.FromSeconds(RefreshIntervalSeconds));
}
private async Task RefreshAsync()
private async Task TickAsync()
{
_refreshing = true;
StateHasChanged();
if (Interlocked.Exchange(ref _tickRunning, 1) == 1) return;
try
{
Refresh();
await LoadConfigAsync();
RebuildDriverGroups();
StateHasChanged();
}
catch (Exception ex) when (ex is ObjectDisposedException or OperationCanceledException)
{
// Circuit disposed while a tick was in flight — ignore (the discarded task would
// otherwise swallow this silently). Mirrors DriverStatusPanel's drain-on-dispose.
}
finally
{
Volatile.Write(ref _tickRunning, 0);
}
}
private async Task RefreshAsync()
{
if (Interlocked.Exchange(ref _tickRunning, 1) == 1) return;
try
{
_refreshing = true;
StateHasChanged();
await Task.Yield();
Refresh();
await LoadConfigAsync();
RebuildDriverGroups();
}
catch (Exception ex) when (ex is ObjectDisposedException or OperationCanceledException)
{
// Same clean-exit tolerance as the timer path.
}
finally
{
// Release first: a StateHasChanged on a torn-down circuit must never leave the guard set.
Volatile.Write(ref _tickRunning, 0);
_refreshing = false;
StateHasChanged();
// This one sits OUTSIDE the try, so it has no catch to fall into — and a render
// request against a disposed renderer throws. Skip it once the page is gone; there
// is nothing left to render to.
if (!_disposed) StateHasChanged();
}
}
@@ -379,13 +408,24 @@ else
{
try
{
await using var db = await DbFactory.CreateDbContextAsync();
await using var db = await DbFactory.CreateDbContextAsync(_cts.Token);
_nodes = await db.ClusterNodes.AsNoTracking()
.Select(n => new HostsNodeInfo(n.ClusterId, n.NodeId, n.Host, n.OpcUaPort))
.ToListAsync();
.ToListAsync(_cts.Token);
_instances = await db.DriverInstances.AsNoTracking()
.Select(d => new HostsDriverInstanceInfo(d.DriverInstanceId, d.ClusterId, d.Name, d.DriverType, d.Enabled))
.ToListAsync();
.ToListAsync(_cts.Token);
}
catch (Exception ex) when (ex is OperationCanceledException or ObjectDisposedException)
{
// The circuit went away mid-load. Rethrow so the caller treats it as a clean exit — it
// must NOT fall into the swallow-and-clear below, which would blank the cached config.
// ObjectDisposedException belongs here for the same reason: teardown disposes _cts (and
// with it the DbContext's token registration) while a query may still be running, and
// that shape reaches this method as ODE rather than OCE. Sending it to the generic catch
// blanked _nodes/_instances on the way out — the exact swallow-and-clear this rethrow
// exists to prevent, and permanent, because a torn-down page never ticks again.
throw;
}
catch (Exception ex)
{
@@ -485,11 +525,22 @@ else
public async ValueTask DisposeAsync()
{
_disposed = true;
// Unsubscribe first so the singleton store can't invoke a handler on a disposed component.
DriverStore.SnapshotChanged -= OnSnapshotChanged;
// Cancel before draining so a tick already in flight aborts its queries rather than running
// them to completion against a component that's gone.
_cts.Cancel();
// Drain the timer so an in-flight callback can't touch a component that's already gone
// (System.Threading.Timer's async dispose awaits any in-flight callback — .NET 6+).
if (_timer is not null) await _timer.DisposeAsync();
// ...but the callback body is `_ = InvokeAsync(TickAsync)`, so that await covered only the
// SCHEDULING of the tick, never the tick itself: LoadConfigAsync can still be running
// against _cts.Token right now. Disposing the source out from under it turns the clean
// cancellation the line above arranged into an ObjectDisposedException mid-query, so skip
// the dispose while a tick holds it. An undisposed CancellationTokenSource with no timer
// and no materialised WaitHandle costs nothing and is collected with the component.
if (Volatile.Read(ref _tickRunning) == 0) _cts.Dispose();
}
private sealed record MemberRow(
src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Deployments/DeploymentDriftProbe.cs
+145
View File
@@ -0,0 +1,145 @@
using Microsoft.EntityFrameworkCore;
using ZB.MOM.WW.OtOpcUa.Configuration;
using ZB.MOM.WW.OtOpcUa.ControlPlane.AdminOperations;
namespace ZB.MOM.WW.OtOpcUa.AdminUI.Deployments;
/// <summary>
/// Answers the /deployments page's "Configuration drift" chip without recomposing the whole
/// configuration on every page load.
/// </summary>
/// <remarks>
/// <para>
/// <b>Why.</b> The chip's question — "does the live-edit state still hash to the latest sealed
/// revision?" — was answered by calling <c>ConfigComposer.SnapshotAndFlattenAsync</c> on every
/// <c>ReloadAsync</c>: eighteen unfiltered table reads, a deterministic JSON serialisation of the
/// entire fleet configuration, a gzip pass and a SHA-256, all to render one badge. Opening
/// /deployments, deploying, or simply re-rendering the page paid for it again.
/// </para>
/// <para>
/// <b>The watermark.</b> <c>ConfigEdit</c> is the append-only audit row that every mutating
/// live-edit operation writes (see the entity's remarks — <c>AdminOperationsActor</c> appends one
/// per mutation), so <c>(MAX(EditedAtUtc), COUNT(*))</c> moves whenever the configuration could
/// have changed. That pair is one indexed aggregate (<c>IX_ConfigEdit_EditedAt</c>) against a
/// full recomposition. When neither the watermark nor the sealed revision hash has moved since
/// the last verdict, the cached verdict is returned unchanged.
/// </para>
/// <para>
/// <b>Accepted staleness.</b> An edit path that mutated a configuration table WITHOUT writing a
/// <c>ConfigEdit</c> row would not invalidate this cache, and the chip could then read "In sync"
/// over a genuinely drifted configuration until the next audited edit. That is acceptable
/// precisely because the chip is advisory: the authoritative comparison is
/// <c>AdminOperationsActor</c>'s own recomposition at deploy time, which always reads live and
/// which reports <c>NoChanges</c> for a deploy that would be a no-op. Today every
/// <c>AdminOperationsActor</c> mutation writes the audit row, so the gap is theoretical — but a
/// future write path that skips the audit must not be relied on to move this chip. The
/// <c>COUNT(*)</c> half is what keeps a same-timestamp edit (two mutations inside one tick, or
/// an edit deleted and replaced) from hiding behind an unchanged maximum.
/// </para>
/// <para>
/// <b>Concurrency.</b> The cheap watermark read runs outside the gate, so concurrent circuits do
/// not serialise on it. Only the recompose path takes the <see cref="SemaphoreSlim"/>, and it
/// double-checks the cache after acquiring it — so N circuits that miss at the same instant
/// share ONE recomposition rather than launching N. A throw leaves the previous verdict in
/// place and propagates to its caller alone.
/// </para>
/// </remarks>
/// <param name="dbFactory">Factory for the configuration database context.</param>
/// <param name="composeRevisionHash">
/// The compose seam — returns the revision hash of a fresh snapshot of the supplied context.
/// Defaults to the real <c>ConfigComposer.SnapshotAndFlattenAsync(...).RevisionHash</c>; tests
/// inject a counting fake to prove the cache actually elides recompositions.
/// </param>
public sealed class DeploymentDriftProbe(
IDbContextFactory<OtOpcUaConfigDbContext> dbFactory,
Func<OtOpcUaConfigDbContext, Task<string>>? composeRevisionHash = null)
{
private readonly Func<OtOpcUaConfigDbContext, Task<string>> _composeRevisionHash =
composeRevisionHash ?? ComposeRevisionHashAsync;
private readonly SemaphoreSlim _recomposeGate = new(1, 1);
/// <summary>The last computed verdict, or <see langword="null"/> before the first one. Volatile: the
/// fast path reads it off the gate, so the publishing write must be visible to every circuit.</summary>
private volatile Verdict? _verdict;
/// <summary>
/// Reports whether the live-edit configuration has drifted from the supplied sealed revision.
/// </summary>
/// <param name="latestSealedRevisionHash">
/// The revision hash of the most recent sealed deployment. Empty means there is nothing sealed to
/// compare against.
/// </param>
/// <param name="ct">Cancellation token.</param>
/// <returns>
/// <see langword="true"/> when the current configuration hashes to something other than
/// <paramref name="latestSealedRevisionHash"/>, <see langword="false"/> when it matches, and
/// <see langword="null"/> when no verdict applies (no sealed revision supplied).
/// </returns>
public async Task<bool?> GetDriftAsync(string latestSealedRevisionHash, CancellationToken ct = default)
{
if (string.IsNullOrEmpty(latestSealedRevisionHash)) return null;
await using var db = await dbFactory.CreateDbContextAsync(ct);
var watermark = await ReadWatermarkAsync(db, ct);
// Fast path: no gate, because a hit reads one immutable record reference.
if (IsCurrent(_verdict, watermark, latestSealedRevisionHash, out var cached)) return cached;
await _recomposeGate.WaitAsync(ct);
try
{
// Double-check: a caller ahead of us in the queue may have just recomposed the same state.
if (IsCurrent(_verdict, watermark, latestSealedRevisionHash, out cached)) return cached;
var current = await _composeRevisionHash(db);
var drift = !string.Equals(current, latestSealedRevisionHash, StringComparison.Ordinal);
_verdict = new Verdict(watermark, latestSealedRevisionHash, drift);
return drift;
}
finally
{
_recomposeGate.Release();
}
}
private static bool IsCurrent(
Verdict? verdict, Watermark watermark, string sealedHash, out bool drift)
{
if (verdict is not null
&& verdict.Watermark == watermark
&& string.Equals(verdict.SealedRevisionHash, sealedHash, StringComparison.Ordinal))
{
drift = verdict.Drift;
return true;
}
drift = false;
return false;
}
/// <summary>Reads the audit watermark as a single aggregate row (no rows at all ⇒ the empty watermark).</summary>
private static async Task<Watermark> ReadWatermarkAsync(OtOpcUaConfigDbContext db, CancellationToken ct)
{
// GroupBy(_ => 1) collapses the whole table to one aggregate row, so this is ONE round-trip for
// both scalars rather than two. An empty table produces no group at all, so FirstOrDefaultAsync
// lands on null and we return the default (never-edited) watermark; the Max is cast to nullable
// so the projection stays valid for the no-rows shape either way.
var row = await db.ConfigEdits
.AsNoTracking()
.GroupBy(_ => 1)
.Select(g => new { LastEditUtc = g.Max(e => (DateTime?)e.EditedAtUtc), EditCount = g.Count() })
.FirstOrDefaultAsync(ct);
return row is null ? default : new Watermark(row.LastEditUtc, row.EditCount);
}
private static async Task<string> ComposeRevisionHashAsync(OtOpcUaConfigDbContext db) =>
(await ConfigComposer.SnapshotAndFlattenAsync(db)).RevisionHash;
/// <summary>The ConfigEdit audit watermark: newest edit timestamp plus the total edit count.</summary>
private readonly record struct Watermark(DateTime? LastEditUtc, int EditCount);
/// <summary>A drift answer, valid only for the watermark and sealed revision it was computed against.</summary>
private sealed record Verdict(Watermark Watermark, string SealedRevisionHash, bool Drift);
}
src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/EndpointRouteBuilderExtensions.cs
+15
View File
@@ -3,11 +3,14 @@ using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Components;
using Microsoft.AspNetCore.Components.Web;
using Microsoft.AspNetCore.Routing;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using Microsoft.Extensions.Logging;
using ZB.MOM.WW.OtOpcUa.AdminUI.Hubs;
using ZB.MOM.WW.OtOpcUa.AdminUI.Uns;
using ZB.MOM.WW.OtOpcUa.Commons.Browsing;
using ZB.MOM.WW.OtOpcUa.Configuration;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Browser;
using ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Browser;
@@ -96,6 +99,18 @@ public static class EndpointRouteBuilderExtensions
// Roslyn-backed Monaco script-editor analysis (diagnostics/completions/hover/...).
services.AddScoped<ScriptAnalysis.ScriptAnalysisService>();
services.AddScoped<ScriptAnalysis.IScriptTagCatalog, ScriptAnalysis.ScriptTagCatalog>();
// The catalog stays Scoped (per circuit); its six-table fleet-wide projection is shared across
// circuits behind a 5 s TTL snapshot — Monaco fires a completion request per keystroke.
services.AddSingleton<ScriptAnalysis.ScriptTagCatalogSnapshotCache>();
// TryAdd so a host that already bound a clock (test harness, future fused-host registration) wins.
services.TryAddSingleton<TimeProvider>(TimeProvider.System);
// /deployments drift chip. A singleton so its ConfigEdit-watermark cache is shared across
// circuits — otherwise every page load would recompose the whole fleet configuration.
// Constructed explicitly rather than by ctor scan: the compose seam is an optional parameter
// that only tests supply, and production must always get the real ConfigComposer default.
services.AddSingleton(sp => new Deployments.DeploymentDriftProbe(
sp.GetRequiredService<IDbContextFactory<OtOpcUaConfigDbContext>>()));
// Certificate-store actions (trust/untrust/delete) for the /certificates page.
services.AddSingleton<Certificates.CertificateStoreManager>();
src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/ScriptAnalysis/IScriptTagCatalog.cs
+18 -4
View File
@@ -79,11 +79,25 @@ public sealed record ScriptTagInfo(string Path, string Kind, string DataType, st
/// same RawPath collapse to one suggestion, which is correct — the string is what resolves.
/// </para>
/// <para>
/// Each call creates and disposes its own context via the pooled factory — the same pattern
/// Each call creates and disposes its own context via the factory — the same pattern
/// <c>UnsTreeService</c> uses — so the service is safe to register Scoped per Blazor circuit.
/// The factory really is pooled: <c>ServiceCollectionExtensions.AddOtOpcUaConfigDb</c> registers it
/// with <c>AddPooledDbContextFactory</c>, so a per-call create is a rent from the pool.
/// </para>
/// <para>
/// <b>The six-table projection is cached for 5 s</b> behind the singleton
/// <see cref="ScriptTagCatalogSnapshotCache"/>, shared across circuits, because Monaco fires a
/// completion request per keystroke inside a string literal. Completions and hover may therefore
/// lag a just-saved tag by up to five seconds — see that type's remarks for why the bound is safe.
/// <see cref="GetEquipmentReferenceNamesAsync"/> is deliberately NOT cached: it is two
/// equipment-scoped queries, not a fleet-wide scan.
/// </para>
/// </remarks>
public sealed class ScriptTagCatalog(IDbContextFactory<OtOpcUaConfigDbContext> dbFactory) : IScriptTagCatalog
/// <param name="dbFactory">Pooled ConfigDb context factory.</param>
/// <param name="cache">Process-wide 5 s snapshot of the fleet-wide projection.</param>
public sealed class ScriptTagCatalog(
IDbContextFactory<OtOpcUaConfigDbContext> dbFactory,
ScriptTagCatalogSnapshotCache cache) : IScriptTagCatalog
{
/// <summary>Upper bound on returned suggestions — keeps the completion list responsive on large fleets.</summary>
private const int MaxResults = 200;
@@ -91,7 +105,7 @@ public sealed class ScriptTagCatalog(IDbContextFactory<OtOpcUaConfigDbContext> d
/// <inheritdoc />
public async Task<IReadOnlyList<string>> GetPathsAsync(string? filter, CancellationToken ct)
{
var entries = await BuildEntriesAsync(ct);
var entries = await cache.GetOrBuildAsync(BuildEntriesAsync, ct);
var paths = new HashSet<string>(entries.Select(e => e.Path), StringComparer.OrdinalIgnoreCase);
@@ -110,7 +124,7 @@ public sealed class ScriptTagCatalog(IDbContextFactory<OtOpcUaConfigDbContext> d
/// <inheritdoc />
public async Task<ScriptTagInfo?> GetTagInfoAsync(string path, CancellationToken ct)
{
var entries = await BuildEntriesAsync(ct);
var entries = await cache.GetOrBuildAsync(BuildEntriesAsync, ct);
// Exact, case-SENSITIVE (Ordinal) match: the live runtime resolves a ctx.GetTag("X") literal
// via DependencyMuxActor keyed with StringComparer.Ordinal, so a case-mismatched path would
src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/ScriptAnalysis/ScriptTagCatalogSnapshotCache.cs
+86
View File
@@ -0,0 +1,86 @@
namespace ZB.MOM.WW.OtOpcUa.AdminUI.ScriptAnalysis;
/// <summary>
/// A short-TTL, process-wide snapshot of the <see cref="ScriptTagCatalog"/>'s projection, shared by
/// every Blazor circuit and every script-analysis endpoint call.
/// </summary>
/// <remarks>
/// <para>
/// <b>Why.</b> Building the projection is SIX unfiltered table scans — <c>RawFolders</c>,
/// <c>DriverInstances</c>, <c>Devices</c>, <c>TagGroups</c>, <c>Tags</c>, <c>VirtualTags</c> — plus
/// a <c>RawPathResolver</c> pass over every tag. Monaco fires a completion request on <i>every
/// keystroke</i> inside a <c>ctx.GetTag("…")</c> string literal, and hover fires again on top of
/// that, so on a large fleet the catalog was re-scanning the whole raw topology dozens of times per
/// second per editing author. The catalog itself stays Scoped (per circuit); only this snapshot is
/// a singleton.
/// </para>
/// <para>
/// <b>The staleness trade-off is deliberate and bounded at <see cref="TtlSeconds"/> seconds.</b>
/// A tag saved in the AdminUI may be missing from completion — and hover over it may report "not a
/// known configured tag path" — for up to five seconds after the save. That is acceptable because
/// the catalog feeds <i>editor assistance</i> only: the authoritative checks (the deploy gate's
/// <c>DraftValidator</c> and the runtime's dependency mux) always read the live database, so a
/// momentarily-stale suggestion list can never let a bad script publish, nor block a good one for
/// longer than one TTL window. Do NOT reuse this cache for anything that decides whether a change
/// is <i>valid</i>.
/// </para>
/// <para>
/// <b>Concurrency.</b> The fresh path is a single volatile read with no locking. A stale path takes
/// a <see cref="SemaphoreSlim"/> and double-checks, so N circuits missing at the same instant share
/// ONE rebuild rather than launching N six-table scans at each other. A build that throws releases
/// the gate, propagates to its caller, and leaves any previous snapshot in place — a transient
/// database blip degrades to "one caller saw an error", never to "the cache now serves nothing".
/// </para>
/// </remarks>
/// <param name="timeProvider">Clock the TTL is measured against (<see cref="TimeProvider.System"/> in production).</param>
public sealed class ScriptTagCatalogSnapshotCache(TimeProvider timeProvider)
{
/// <summary>How long a built snapshot is served before the next caller rebuilds it.</summary>
private const int TtlSeconds = 5;
private static readonly TimeSpan Ttl = TimeSpan.FromSeconds(TtlSeconds);
private readonly SemaphoreSlim _rebuildGate = new(1, 1);
/// <summary>The current snapshot, or <see langword="null"/> before the first successful build.</summary>
private volatile Snapshot? _snapshot;
/// <summary>
/// Returns the cached projection when it is younger than the TTL, otherwise rebuilds it via
/// <paramref name="build"/> under a gate that collapses concurrent misses into one rebuild.
/// </summary>
/// <param name="build">Builds a fresh projection; invoked at most once per TTL window.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>The catalog entries — a fresh snapshot, or one just built.</returns>
public async Task<IReadOnlyList<ScriptTagInfo>> GetOrBuildAsync(
Func<CancellationToken, Task<IReadOnlyList<ScriptTagInfo>>> build,
CancellationToken ct)
{
var current = _snapshot;
if (current is not null && !IsStale(current)) return current.Entries;
await _rebuildGate.WaitAsync(ct);
try
{
// Double-check: while we queued, the caller ahead of us may have rebuilt already.
current = _snapshot;
if (current is not null && !IsStale(current)) return current.Entries;
// A throw here leaves _snapshot untouched (the previous one, if any, keeps serving) and
// propagates to this caller only; the finally still releases the gate.
var entries = await build(ct);
// Stamped AFTER the build so a slow scan does not hand back an already-expired snapshot.
_snapshot = new Snapshot(entries, timeProvider.GetUtcNow());
return entries;
}
finally
{
_rebuildGate.Release();
}
}
private bool IsStale(Snapshot snapshot) => timeProvider.GetUtcNow() - snapshot.TakenAt >= Ttl;
private sealed record Snapshot(IReadOnlyList<ScriptTagInfo> Entries, DateTimeOffset TakenAt);
}
src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/EffectiveNameGuard.cs
+89 -25
View File
@@ -12,8 +12,11 @@ namespace ZB.MOM.WW.OtOpcUa.AdminUI.Uns;
/// <c>DraftValidator</c> <c>UnsEffectiveNameCollision</c> rule.
/// </summary>
/// <remarks>
/// Each call creates and disposes its own pooled context — the same per-call pattern
/// <see cref="UnsTreeService"/> uses — so the guard is safe to register <c>Scoped</c>.
/// Each call creates and disposes its own context — the same per-call pattern
/// <see cref="UnsTreeService"/> uses — so the guard is safe to register <c>Scoped</c>. The factory
/// really is pooled: <c>ServiceCollectionExtensions.AddOtOpcUaConfigDb</c> registers it with
/// <c>AddPooledDbContextFactory</c>, so the per-call create rents from the pool and the dispose
/// returns the context to it.
/// </remarks>
public sealed class EffectiveNameGuard(IDbContextFactory<OtOpcUaConfigDbContext> dbFactory) : IEffectiveNameGuard
{
@@ -27,6 +30,48 @@ public sealed class EffectiveNameGuard(IDbContextFactory<OtOpcUaConfigDbContext>
{
await using var db = await dbFactory.CreateDbContextAsync(ct);
var nameSpace = await LoadNameSpaceAsync(db, equipmentId, ct);
return FirstCollision(nameSpace, proposedEffectiveName, kind, excludeSourceId, equipmentId);
}
/// <inheritdoc />
/// <remarks>
/// The whole point of the batch entry point: ONE context and ONE load of the equipment's name
/// space, then every proposed name tested in memory against it — the per-name rules and message
/// text come from the same <see cref="FirstCollision"/> helper <see cref="CheckAsync"/> uses, so
/// the two can never drift. A batch is a create of N new rows, so there is no self-row to exclude.
/// </remarks>
public async Task<string?> CheckManyAsync(
string equipmentId,
IReadOnlyList<string> proposedEffectiveNames,
EffectiveNameSourceKind kind,
CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(proposedEffectiveNames);
if (proposedEffectiveNames.Count == 0) return null;
await using var db = await dbFactory.CreateDbContextAsync(ct);
var nameSpace = await LoadNameSpaceAsync(db, equipmentId, ct);
foreach (var proposed in proposedEffectiveNames)
{
var collision = FirstCollision(nameSpace, proposed, kind, excludeSourceId: null, equipmentId);
if (collision is not null) return collision;
}
return null;
}
/// <summary>
/// Loads the equipment's current effective-name space as a flat, ordered row list: references
/// first (override else the backing raw tag's <c>Name</c>), then VirtualTags, then ScriptedAlarms
/// — the order <see cref="CheckAsync"/> has always reported collisions in.
/// </summary>
private static async Task<List<NameRow>> LoadNameSpaceAsync(
OtOpcUaConfigDbContext db, string equipmentId, CancellationToken ct)
{
// References for this equipment. The effective name is the display-name override else the
// backing raw tag's current Name, so the two sets (references + tags) are loaded and joined
// in memory — the per-equipment set is small and this mirrors the deploy rule exactly
@@ -46,47 +91,66 @@ public sealed class EffectiveNameGuard(IDbContextFactory<OtOpcUaConfigDbContext>
.GroupBy(t => t.TagId, StringComparer.Ordinal)
.ToDictionary(g => g.Key, g => g.First().Name, StringComparer.Ordinal);
foreach (var r in references)
{
if (IsSelf(EffectiveNameSourceKind.Reference, r.UnsTagReferenceId, kind, excludeSourceId))
continue;
var effective = r.DisplayNameOverride
?? (tagNameById.TryGetValue(r.TagId, out var n) ? n : null);
if (Collides(effective, proposedEffectiveName))
return Message(proposedEffectiveName, "reference", r.UnsTagReferenceId, equipmentId);
}
var virtualTags = await db.VirtualTags
.AsNoTracking()
.Where(v => v.EquipmentId == equipmentId)
.Select(v => new { v.VirtualTagId, v.Name })
.ToListAsync(ct);
foreach (var v in virtualTags)
{
if (IsSelf(EffectiveNameSourceKind.VirtualTag, v.VirtualTagId, kind, excludeSourceId))
continue;
if (Collides(v.Name, proposedEffectiveName))
return Message(proposedEffectiveName, "VirtualTag", v.VirtualTagId, equipmentId);
}
var scriptedAlarms = await db.ScriptedAlarms
.AsNoTracking()
.Where(a => a.EquipmentId == equipmentId)
.Select(a => new { a.ScriptedAlarmId, a.Name })
.ToListAsync(ct);
foreach (var a in scriptedAlarms)
var rows = new List<NameRow>(references.Count + virtualTags.Count + scriptedAlarms.Count);
foreach (var r in references)
{
if (IsSelf(EffectiveNameSourceKind.ScriptedAlarm, a.ScriptedAlarmId, kind, excludeSourceId))
continue;
if (Collides(a.Name, proposedEffectiveName))
return Message(proposedEffectiveName, "ScriptedAlarm", a.ScriptedAlarmId, equipmentId);
var effective = r.DisplayNameOverride
?? (tagNameById.TryGetValue(r.TagId, out var n) ? n : null);
rows.Add(new NameRow(EffectiveNameSourceKind.Reference, r.UnsTagReferenceId, effective, "reference"));
}
foreach (var v in virtualTags)
rows.Add(new NameRow(EffectiveNameSourceKind.VirtualTag, v.VirtualTagId, v.Name, "VirtualTag"));
foreach (var a in scriptedAlarms)
rows.Add(new NameRow(EffectiveNameSourceKind.ScriptedAlarm, a.ScriptedAlarmId, a.Name, "ScriptedAlarm"));
return rows;
}
/// <summary>
/// The single in-memory collision rule shared by <see cref="CheckAsync"/> and
/// <see cref="CheckManyAsync"/> — same self-row exclusion, same ordinal comparison, same message.
/// </summary>
private static string? FirstCollision(
List<NameRow> nameSpace,
string proposedEffectiveName,
EffectiveNameSourceKind kind,
string? excludeSourceId,
string equipmentId)
{
foreach (var row in nameSpace)
{
if (IsSelf(row.Kind, row.SourceId, kind, excludeSourceId)) continue;
if (Collides(row.EffectiveName, proposedEffectiveName))
return Message(proposedEffectiveName, row.SourceLabel, row.SourceId, equipmentId);
}
return null;
}
/// <summary>One existing effective-name-space row: what it is, its logical id, its effective name
/// (null when a reference's backing tag is missing — such a row contributes no name), and the label
/// the collision message uses.</summary>
private readonly record struct NameRow(
EffectiveNameSourceKind Kind,
string SourceId,
string? EffectiveName,
string SourceLabel);
/// <summary>True when a row is the self-row being edited — same kind AND same logical id — so it
/// is excluded from the collision set (a rename / override-change of a row cannot collide with itself).</summary>
private static bool IsSelf(
src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/IEffectiveNameGuard.cs
+34
View File
@@ -58,4 +58,38 @@ public interface IEffectiveNameGuard
EffectiveNameSourceKind kind,
string? excludeSourceId,
CancellationToken ct = default);
/// <summary>
/// Checks many proposed names against ONE load of the equipment's name space. Returns the first
/// collision message (in input order), or <see langword="null"/> when all are free.
/// </summary>
/// <remarks>
/// The bulk-authoring entry point (a select-all "add references" over N raw tags), where the
/// per-name <see cref="CheckAsync"/> would cost N contexts × 4 queries. There is no
/// <c>excludeSourceId</c>: a batch is a CREATE of N new rows, so no self-row exists to exclude.
/// The default implementation simply loops <see cref="CheckAsync"/> — correct but not fast — so
/// a stub / no-op guard needs no change; <c>EffectiveNameGuard</c> overrides it with the
/// single-load version.
/// </remarks>
/// <param name="equipmentId">The owning equipment's logical id.</param>
/// <param name="proposedEffectiveNames">The effective names to be authored, in the order the caller wants them reported.</param>
/// <param name="kind">Which surface the proposed names are for (words the error; identifies the self-row kind).</param>
/// <param name="ct">Cancellation token.</param>
/// <returns><see langword="null"/> when every name is free; otherwise the first name's readable error.</returns>
async Task<string?> CheckManyAsync(
string equipmentId,
IReadOnlyList<string> proposedEffectiveNames,
EffectiveNameSourceKind kind,
CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(proposedEffectiveNames);
foreach (var proposed in proposedEffectiveNames)
{
var collision = await CheckAsync(equipmentId, proposed, kind, excludeSourceId: null, ct);
if (collision is not null) return collision;
}
return null;
}
}
src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/RawTreeService.cs
+91 -14
View File
@@ -1,3 +1,4 @@
using System.Linq.Expressions;
using System.Text.Json;
using Microsoft.EntityFrameworkCore;
using ZB.MOM.WW.OtOpcUa.AdminUI.Uns.TagEditors;
@@ -173,6 +174,7 @@ public sealed class RawTreeService(IDbContextFactory<OtOpcUaConfigDbContext> dbF
var tags = await db.Tags.AsNoTracking()
.Where(t => t.DeviceId == deviceId && t.TagGroupId == null)
.OrderBy(t => t.Name)
.Select(t => new TagRow(t.TagId, t.Name, t.RowVersion))
.ToListAsync(ct);
var nodes = new List<RawNode>(groups.Count + tags.Count);
@@ -194,6 +196,7 @@ public sealed class RawTreeService(IDbContextFactory<OtOpcUaConfigDbContext> dbF
var tags = await db.Tags.AsNoTracking()
.Where(t => t.TagGroupId == groupId)
.OrderBy(t => t.Name)
.Select(t => new TagRow(t.TagId, t.Name, t.RowVersion))
.ToListAsync(ct);
var nodes = new List<RawNode>(subGroups.Count + tags.Count);
@@ -299,7 +302,13 @@ public sealed class RawTreeService(IDbContextFactory<OtOpcUaConfigDbContext> dbF
}).ToList();
}
private static RawNode BuildTagNode(RawNode parent, Tag t) => new()
/// <summary>
/// The only columns a tree row needs from a <see cref="Tag" />. Projecting to this keeps the
/// <c>TagConfig</c> nvarchar(max) JSON blob out of the tree's SELECT — it is never rendered.
/// </summary>
private sealed record TagRow(string TagId, string Name, byte[] RowVersion);
private static RawNode BuildTagNode(RawNode parent, TagRow t) => new()
{
Kind = RawNodeKind.Tag,
Key = $"tag:{t.TagId}",
@@ -1407,10 +1416,23 @@ public sealed class RawTreeService(IDbContextFactory<OtOpcUaConfigDbContext> dbF
}
/// <summary>
/// Scans every <see cref="Script"/> body for the OLD RawPath of any affected tag as a plain ordinal
/// substring (no AST — false positives tolerated by design; <c>{{equip}}</c>-relative refs are NOT
/// scanned, they resolve through references and the deploy gate catches breakage). Returns a warning
/// naming the matched scripts, or <see langword="null"/> when none match.
/// Max <c>OR</c> terms in one script-scan predicate — bounds both EF's recursive expression-visitor
/// depth over the left-nested <c>OrElse</c> chain and SQL Server's boolean-expression complexity limit.
/// </summary>
private const int ScriptScanBatchSize = 200;
/// <summary>
/// Scans every <see cref="Script"/> body for the OLD RawPath of any affected tag as a plain substring
/// (no AST — false positives tolerated by design; <c>{{equip}}</c>-relative refs are NOT scanned, they
/// resolve through references and the deploy gate catches breakage). Returns a warning naming the
/// matched scripts, or <see langword="null"/> when none match.
/// <para>
/// The substring match runs in the DATABASE (see <see cref="AnyPathContained"/>), so unmatched script
/// bodies — the overwhelming majority, and the largest text column in the config DB — never cross the
/// wire; only the matched scripts' names are projected. The paths are deduplicated and scanned in
/// batches of <see cref="ScriptScanBatchSize"/>, so the disjunction stays bounded whatever the size of
/// the renamed subtree.
/// </para>
/// </summary>
private static async Task<string?> BuildScriptReferenceWarningAsync(
OtOpcUaConfigDbContext db, IReadOnlyList<AffectedTag> tags, CancellationToken ct)
@@ -1418,23 +1440,78 @@ public sealed class RawTreeService(IDbContextFactory<OtOpcUaConfigDbContext> dbF
var oldPaths = await ComputeOldRawPathsAsync(db, tags, ct);
if (oldPaths.Count == 0) return null;
var scripts = await db.Scripts.AsNoTracking()
.Select(s => new { s.ScriptId, s.Name, s.SourceCode })
.ToListAsync(ct);
// One path per affected TAG, so an ancestor rename yields thousands of them, many identical (a
// tag name repeated under sibling devices). Deduplicating first shortens the disjunction without
// narrowing the match — the SQL comparison runs under the column collation either way.
var distinctPaths = oldPaths.Distinct(StringComparer.Ordinal).ToArray();
var matched = scripts
.Where(s => oldPaths.Any(p => s.SourceCode.Contains(p, StringComparison.Ordinal)))
.OrderBy(s => s.Name, StringComparer.Ordinal)
.Select(s => s.Name)
.ToList();
// ScriptId, not Name: Name carries no unique index, and collapsing two same-named scripts into one
// would change the count this warning reports.
var matched = new Dictionary<string, string>(StringComparer.Ordinal);
for (var offset = 0; offset < distinctPaths.Length; offset += ScriptScanBatchSize)
{
var batch = distinctPaths[offset..Math.Min(offset + ScriptScanBatchSize, distinctPaths.Length)];
var rows = await db.Scripts.AsNoTracking()
.Where(AnyPathContained(batch))
.Select(s => new { s.ScriptId, s.Name })
.ToListAsync(ct);
foreach (var row in rows) matched[row.ScriptId] = row.Name;
}
if (matched.Count == 0) return null;
var names = string.Join(", ", matched.Select(n => $"'{n}'"));
// Ordered in memory: SQL's ORDER BY would sort under the column collation, not ordinally.
var matchedNames = matched.Values.ToList();
matchedNames.Sort(StringComparer.Ordinal);
var names = string.Join(", ", matchedNames.Select(n => $"'{n}'"));
return matched.Count == 1
? $"1 script ({names}) references a tag beneath this node by its raw path; the reference will break unless the script is updated."
: $"{matched.Count} scripts ({names}) reference tags beneath this node by their raw paths; the references will break unless the scripts are updated.";
}
/// <summary>
/// Builds <c>s =&gt; s.SourceCode.Contains(p0) || s.SourceCode.Contains(p1) || …</c> over
/// <paramref name="paths"/> — one disjunction sharing a single parameter expression, so it composes
/// into an EF <c>Where</c> and the scan runs server-side (SQL Server: <c>CHARINDEX</c>; the EF InMemory
/// provider: the same in-memory <see cref="string.Contains(string)"/> the caller used to run itself).
/// Callers must guarantee <paramref name="paths"/> is non-empty (a disjunction of nothing has no
/// meaningful identity here — the caller's degenerate guard returns before this point).
/// </summary>
/// <remarks>
/// ⚠️ Match semantics widen slightly on SQL Server: <c>CHARINDEX</c> runs under the column's collation,
/// which is case-insensitive on a default install, where the previous in-memory scan was
/// <see cref="StringComparison.Ordinal"/>. Accepted deliberately — RawPaths are operator-authored
/// identifiers, this warning is advisory, and a false positive costs an operator one extra glance
/// while a false negative (the risk the old scan actually carried, since a script may name a path in
/// any casing) silently ships a broken rename.
/// </remarks>
private static Expression<Func<Script, bool>> AnyPathContained(IReadOnlyList<string> paths)
{
var parameter = Expression.Parameter(typeof(Script), "s");
var sourceCode = Expression.Property(parameter, nameof(Script.SourceCode));
var contains = typeof(string).GetMethod(nameof(string.Contains), [typeof(string)])!;
// Indexing an array (rather than Expression.Constant(path) directly) keeps the expression tree
// cache-friendly: every rename of the same path-count produces the same tree shape.
// ⚠️ It does NOT guarantee parameterisation. The array here is a constant with no closure over it,
// so EF may well inline each path into the SQL as an escaped literal — which is safe (EF escapes
// the Contains argument either way, so there is no injection surface) but means a rename can emit up
// to ScriptScanBatchSize (200) literals per batch and churn the plan cache. Whether the provider
// actually parameterises this shape is unverified: proving it needs a SQL-Server-backed test, not
// EF InMemory. Recorded in the follow-up register of
// docs/plans/2026-08-15-perf-roadmap-remainder.md.
var values = paths.ToArray();
Expression? body = null;
for (var i = 0; i < values.Length; i++)
{
var value = Expression.ArrayIndex(Expression.Constant(values), Expression.Constant(i));
var call = Expression.Call(sourceCode, contains, value);
body = body is null ? call : Expression.OrElse(body, call);
}
return Expression.Lambda<Func<Script, bool>>(body!, parameter);
}
/// <summary>
/// Recomputes the OLD RawPath of each affected tag from the current (pre-save) in-DB raw topology,
/// dropping any tag whose ancestry chain is broken (a null RawPath). Loads only the topology the affected
src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/UnsTreeService.cs
+12 -2
View File
@@ -209,6 +209,11 @@ public sealed class UnsTreeService(
// Track effective names proposed within THIS batch so two raw tags sharing a Name are caught (the
// guard sees only persisted rows; a same-batch collision would otherwise slip through).
var batchNames = new HashSet<string>(StringComparer.Ordinal);
// The proposed effective names, in selection order, for the ONE post-loop guard round-trip. The
// per-name guard call used to sit inside this loop, costing a fresh context + 4 queries per
// selected tag (a select-all of 500 tags ≈ 2,000 queries); the batch check loads the equipment's
// name space once and reports the first collision in this same order.
var proposedNames = new List<string>(distinctTagIds.Count);
var pending = new List<UnsTagReference>();
foreach (var tagId in distinctTagIds)
@@ -228,8 +233,7 @@ public sealed class UnsTreeService(
return new UnsMutationResult(false,
$"Two selected raw tags share the effective name '{tag.Name}'; set a display-name override so they stay unique within the equipment.");
var collision = await _guard.CheckAsync(equipmentId, tag.Name, EffectiveNameSourceKind.Reference, null, ct);
if (collision is not null) return new UnsMutationResult(false, collision);
proposedNames.Add(tag.Name);
pending.Add(new UnsTagReference
{
@@ -241,6 +245,12 @@ public sealed class UnsTreeService(
});
}
// One guard round-trip for the whole batch (one context + one load of the equipment's existing
// effective-name space), regardless of how many tags were selected.
var collision = await _guard.CheckManyAsync(
equipmentId, proposedNames, EffectiveNameSourceKind.Reference, ct);
if (collision is not null) return new UnsMutationResult(false, collision);
db.UnsTagReferences.AddRange(pending);
try
{
src/Server/ZB.MOM.WW.OtOpcUa.Host/appsettings.json
+9 -2
View File
@@ -43,10 +43,17 @@
"OutboxPath": "",
"_OutboxPathComment": "Directory holding the FasterLog segment + commit files. Required when Enabled=true. In production set an ABSOLUTE path on durable storage.",
"CommitMode": "PerEntry",
"_CommitModeComment": "PerEntry means one fsync per ingest BURST, not per value: the recorder appends a whole burst through AppendBatchAsync, which enqueues every entry and issues a single group commit. Periodic batches commits onto a timer every CommitIntervalMs (bounded loss window).",
"CommitIntervalMs": 100,
"DrainBatchSize": 64,
"DrainBatchSize": 1024,
"_DrainBatchSizeComment": "Entries peeked + written per drain pass. Was 64 when a pass wrote its per-tag groups sequentially; the peek is an in-memory scan and the pass is now bounded-parallel over DrainConcurrency, so a wide batch is cheap — and 64 let the outbox fall permanently behind during any gateway hiccup.",
"DrainIntervalSeconds": 2,
"Capacity": 0,
"DrainConcurrency": 8,
"_DrainConcurrencyComment": "Historian tags a single drain pass writes concurrently (one gateway round-trip per tag group). 1 restores the fully sequential drain. The ack is unchanged: still all-or-nothing after every group reports.",
"Capacity": 500000,
"_CapacityComment": "Max un-acked outbox entries before drop-oldest; 0 = unbounded. The Gitea #538 disk backstop for a historian refusing EVERY write — the one case the per-tag quarantine cannot resolve.",
"MaxWriteAttempts": 5,
"_MaxWriteAttemptsComment": "Gitea #538: consecutive failed drain passes one tag may accumulate — counted only across passes where OTHER tags wrote successfully — before its queued entries are abandoned so the FIFO head can advance.",
"MinBackoffSeconds": 1,
"MaxBackoffSeconds": 30
},
src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs
+127 -14
View File
@@ -316,6 +316,14 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
/// <summary>
/// Re-authorizes live subscriptions when a deployment installs a new authorization generation.
/// </summary>
/// <remarks>
/// ⚠️ <b>This is the only trigger, and the pass's memo depends on that.</b>
/// <see cref="IsSubscriptionStillPermitted"/> memoizes key revocation and expiry for the span of one
/// pass — safe only while revalidation is deploy-triggered, because a pass is then a momentary
/// event whose verdicts cannot go stale within it. A future periodic trigger must give each tick a
/// fresh memo (it already does — the dictionary is pass-local) and must not be tempted to reuse one
/// across ticks, or an expiry landing between ticks would be served the previous tick's allow.
/// </remarks>
/// <param name="snapshot">The newly published snapshot; unused — the current state is re-read per item.</param>
private void OnAuthorityChanged(DataPlaneAuthoritySnapshot snapshot) => RevalidateMonitoredItems();
@@ -2943,22 +2951,36 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
base.Browse(context, ref continuationPoint, references);
var returned = references.Count - before;
// Forward two-pointer compaction over the appended range, NOT a per-denial RemoveAt. Removing
// from the middle shifts every later element down one slot, so denying a large prefix of a wide
// browse costs O(kept × removed) element moves; copying each survivor down to a write cursor and
// trimming the tail afterwards is one pass. The tail trim removes from the END, where RemoveAt
// moves nothing.
var filtered = 0;
for (var ii = references.Count - 1; ii >= before; ii--)
var write = before;
for (var ii = before; ii < references.Count; ii++)
{
// ExpandedNodeId → NodeId only makes sense for a reference into our own server; a reference
// to another server's node is left alone, as this manager has no grants that could speak to it.
var target = references[ii].NodeId;
if (target is null || target.IsAbsolute) continue;
var keep = target is null
|| target.IsAbsolute
|| ExpandedNodeId.ToNodeId(target, Server.NamespaceUris) is not { } targetId
|| !OwnsNamespace(targetId)
|| Permits(identity, OpcUaOperation.Browse, targetId);
var targetId = ExpandedNodeId.ToNodeId(target, Server.NamespaceUris);
if (targetId is null || !OwnsNamespace(targetId)) continue;
if (Permits(identity, OpcUaOperation.Browse, targetId)) continue;
if (!keep)
{
filtered++;
continue;
}
references.RemoveAt(ii);
filtered++;
if (write != ii) references[write] = references[ii];
write++;
}
for (var ii = references.Count - 1; ii >= write; ii--) references.RemoveAt(ii);
LogAclFiltering("Browse", identity, filtered, returned);
}
@@ -2995,14 +3017,24 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
var before = targetIds.Count;
base.TranslateBrowsePath(context, sourceHandle, relativePath, targetIds, unresolvedTargetIds);
for (var ii = targetIds.Count - 1; ii >= before; ii--)
// Forward two-pointer compaction over the appended range — same reasoning as Browse above: a
// mid-list RemoveAt per denial is O(kept × removed) element moves, one compacting pass plus a
// tail trim is O(n).
var write = before;
for (var ii = before; ii < targetIds.Count; ii++)
{
var targetId = ExpandedNodeId.ToNodeId(targetIds[ii], Server.NamespaceUris);
if (targetId is null || !OwnsNamespace(targetId)) continue;
if (Permits(identity, OpcUaOperation.Browse, targetId)) continue;
var keep = targetId is null
|| !OwnsNamespace(targetId)
|| Permits(identity, OpcUaOperation.Browse, targetId);
targetIds.RemoveAt(ii);
if (!keep) continue;
if (write != ii) targetIds[write] = targetIds[ii];
write++;
}
for (var ii = targetIds.Count - 1; ii >= write; ii--) targetIds.RemoveAt(ii);
}
/// <inheritdoc />
@@ -3103,6 +3135,18 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
ServerSystemContext context, NodeHandle handle, ISampledDataChangeMonitoredItem monitoredItem)
{
base.OnMonitoredItemCreated(context, handle, monitoredItem);
TrackLiveMonitoredItem(handle, monitoredItem);
}
/// <summary>Starts tracking <paramref name="monitoredItem"/> for revalidation, if we own its node.</summary>
/// <param name="handle">The handle the item was created against.</param>
/// <param name="monitoredItem">The item.</param>
/// <remarks>
/// Split out of <see cref="OnMonitoredItemCreated"/> so tests can register an item without a client
/// session; the SDK reaches it only through that override.
/// </remarks>
internal void TrackLiveMonitoredItem(NodeHandle handle, ISampledDataChangeMonitoredItem monitoredItem)
{
if (OwnsNamespace(handle.NodeId)) _liveItems[monitoredItem] = new LiveItem(handle);
}
@@ -3169,6 +3213,20 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
/// subscription, and inventing one would leave the client's own bookkeeping describing items the
/// server had silently discarded.
/// </para>
/// <para>
/// <b>One verdict per (session, node), not per item.</b> A session commonly holds several items
/// on the same node — one per attribute, or one per downstream consumer sharing a session — and
/// they all ask the identical question. The pass memoizes the verdict locally so N items on a
/// node cost one <see cref="Security.DataPlaneAuthorizer.IsAllowedCached"/> call rather than N.
/// The authorizer's own decision cache does not remove this work: it memoizes the grant walk,
/// while the per-call key revocation/expiry check in front of it is deliberately outside that
/// cache and runs every time.
/// </para>
/// <para>
/// The per-item <c>lock (Lock)</c> takes are deliberately left alone. Batching them under one
/// hold was considered and rejected: it would stretch the SDK lock across the
/// <c>QueueValue</c> calls between them, and the attach/detach is what actually needs the lock.
/// </para>
/// </remarks>
public void RevalidateMonitoredItems()
{
@@ -3178,6 +3236,11 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
var stoodDown = 0;
var restored = 0;
// Pass-local, so single-threaded by construction. A concurrent deployment can advance the
// generation mid-pass and leave later items decided on the earlier one — exactly as the
// un-memoized walk already could, and the change that advanced it runs its own pass.
var verdicts = new Dictionary<(string Session, AddressSpaceRealm Realm, string Node), bool>();
foreach (var (item, tracked) in _liveItems)
{
// The item carries the identity of the session that created it, which is what must be
@@ -3185,9 +3248,8 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
if (item is not IMonitoredItem live) continue;
var nodeId = tracked.Handle.NodeId;
var permitted = _authorizer!.IsAllowedCached(
live.EffectiveIdentity as RoleCarryingUserIdentity,
OpcUaOperation.CreateMonitoredItems, nodeId, RealmOf(nodeId));
var permitted = IsSubscriptionStillPermitted(
live.EffectiveIdentity as RoleCarryingUserIdentity, nodeId, verdicts);
if (permitted)
{
@@ -3220,6 +3282,57 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
}
}
/// <summary>
/// The per-item verdict for <see cref="RevalidateMonitoredItems"/>, memoized across the pass.
/// </summary>
/// <param name="identity">The identity of the session that created the item.</param>
/// <param name="nodeId">The node the item monitors.</param>
/// <param name="verdicts">The pass-local memo, keyed by session, realm and node identifier.</param>
/// <returns><see langword="true"/> when the item may keep receiving values.</returns>
/// <remarks>
/// <para>
/// The session key is <c>ApiKeyId ?? Principal</c> — the same derivation
/// <see cref="Security.DataPlaneAuthorizer"/> uses for its own decision cache, so this memo is
/// never coarser than the cache it sits in front of. The realm is part of the key for the same
/// reason it is part of the authorizer's: the raw and UNS namespaces have independent string
/// identifier spaces, so an identifier alone does not name a node.
/// </para>
/// <para>
/// Anything the key cannot be built from — no role-carrying identity, no principal, a non-string
/// node identifier — falls through to the un-memoized call and keeps its existing verdict. Those
/// are the cases the authorizer denies anyway; memoizing them would only make a deny cheaper.
/// </para>
/// <para>
/// This memo does cover the key expiry check that the authorizer's own cache excludes, but only
/// for the duration of one pass — and a subscription is re-checked at a deployment, never on a
/// timer, so an expiry that lands inside a single pass was already going to be caught by the
/// next deployment rather than by this one.
/// </para>
/// </remarks>
private bool IsSubscriptionStillPermitted(
RoleCarryingUserIdentity? identity,
NodeId nodeId,
Dictionary<(string Session, AddressSpaceRealm Realm, string Node), bool> verdicts)
{
var realm = RealmOf(nodeId);
if (identity is null
|| (identity.ApiKeyId ?? identity.Principal) is not { Length: > 0 } sessionKey
|| nodeId.Identifier is not string identifier)
{
return _authorizer!.IsAllowedCached(
identity, OpcUaOperation.CreateMonitoredItems, nodeId, realm);
}
var key = (sessionKey, realm, identifier);
if (verdicts.TryGetValue(key, out var memoized)) return memoized;
var permitted = _authorizer!.IsAllowedCached(
identity, OpcUaOperation.CreateMonitoredItems, nodeId, realm);
verdicts[key] = permitted;
return permitted;
}
/// <summary>Emits the one operator-facing summary line per gated request.</summary>
/// <param name="operation">The service being filtered.</param>
/// <param name="identity">The calling session's identity.</param>
src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Security/DataPlaneAuthorizer.cs
+158 -45
View File
@@ -54,6 +54,25 @@ namespace ZB.MOM.WW.OtOpcUa.OpcUaServer.Security;
/// </remarks>
public sealed class DataPlaneAuthorizer
{
/// <summary>
/// The single veto every denial returns. Logged, never explained to the client: the client gets an
/// undifferentiated <c>BadUserAccessDenied</c> so the server cannot be used to probe which nodes
/// exist or which grants a principal holds, and the operator debugging a denial reads the reason in
/// the log. Same posture as the authenticator's <c>DenyMessage</c>.
/// </summary>
/// <remarks>
/// <b>Shared rather than per-call, because <see cref="ServiceResult"/> is immutable to us.</b>
/// Verified by reflection against the pinned Opc.Ua.Core 1.5.378.106: every one of its properties
/// (<c>StatusCode</c>, <c>SymbolicId</c>, <c>NamespaceUri</c>, <c>LocalizedText</c>,
/// <c>AdditionalInfo</c>, <c>InnerResult</c>) has a <b>private</b> setter, and its only non-getter
/// method — <c>Append(StringBuilder)</c> — formats rather than mutates, so no code outside the type
/// can change an instance. The SDK itself publishes <c>ServiceResult.Good</c> and
/// <c>ServiceResult.Bad</c> as shared <c>static readonly</c> singletons, which is its own
/// endorsement of the pattern. Every call site here treats the result as an opaque value — it is
/// stored into an SDK errors array or returned — and none compares by reference.
/// </remarks>
private static readonly ServiceResult DeniedResult = new(StatusCodes.BadUserAccessDenied);
private readonly IDataPlaneAuthoritySource _authority;
private readonly IPermissionEvaluator _evaluator;
private readonly ILogger _log;
@@ -173,6 +192,23 @@ public sealed class DataPlaneAuthorizer
AddressSpaceRealm realm)
=> Evaluate(identity, operation, nodeId, realm, cached: false);
/// <summary>
/// The decision itself. Ordered so the cached seams reach their answer having allocated nothing:
/// every check that must run per call comes first, the memoized verdict is consulted next, and the
/// scope build and grant walk happen only on a miss.
/// </summary>
/// <remarks>
/// The ordering is a correctness constraint before it is a performance one — see the key-revocation
/// comment inline, and <c>Key_revocation_is_rechecked_on_a_cache_hit</c> /
/// <c>NodeManagerStage6EnforcementTests.An_expired_key_is_denied_even_though_its_allow_is_cached</c>,
/// which fail if the cache lookup is moved above it.
/// </remarks>
/// <param name="identity">The role-carrying identity off the SDK context.</param>
/// <param name="operation">The operation being attempted.</param>
/// <param name="nodeId">The target node.</param>
/// <param name="realm">The namespace the node was resolved in.</param>
/// <param name="cached">Whether the grant walk may be memoized (the request-rate seams).</param>
/// <returns><c>null</c> to proceed; otherwise the veto to return to the SDK.</returns>
private ServiceResult? Evaluate(
RoleCarryingUserIdentity? identity,
OpcUaOperation operation,
@@ -203,33 +239,43 @@ public sealed class DataPlaneAuthorizer
// since then may have disabled or expired it — and on a driver-only node the artifact is the ONLY
// revocation channel. Without this a revoked key keeps its open session's authority indefinitely,
// which on a machine-to-machine data plane means until the client chooses to reconnect.
// ⚠️ Deliberately OUTSIDE the decision cache, and it must stay there. Grants and principals are
// fixed for a generation, so memoizing the grant walk is sound; a key's expiry passes with the
// CLOCK, so a cached allow would keep an expired key working until the next deploy.
// ⚠️ Deliberately OUTSIDE the decision cache, and it must stay ABOVE the lookup below. Grants and
// principals are fixed for a generation, so memoizing the grant walk is sound; a key's expiry
// passes with the CLOCK, so a cached allow consulted first would keep an expired key working until
// the next deploy — the generation never moves, so nothing would drop the entry.
var now = _time.GetUtcNow().UtcDateTime;
var sessionKey = identity.ApiKeyId ?? identity.Principal;
if (identity.ApiKeyId is { Length: > 0 } keyId && snapshot.FindUsableKey(keyId, now) is null)
{
return Deny(operation, nodeId, identity.Principal,
$"key {keyId} is no longer enabled or has expired in generation {snapshot.GenerationId}", cached);
}
return DenyRevokedKey(operation, nodeId, identity.Principal, keyId, snapshot.GenerationId, cached);
if (!TryBuildScope(nodeId, realm, snapshot.ClusterId, out var scope))
// The s= identifier, read once: it is both the cache key and the raw material the scope is split
// from, and pulling it up here absorbs what used to be TryBuildScope's own guard. A NodeId that
// carries no string addresses no ACL scope at all, so it can never be answered from the cache.
if (nodeId?.Identifier is not string identifier || identifier.Length == 0)
{
return Deny(operation, nodeId, identity.Principal,
"the node id carries no string identifier, so it addresses no ACL scope", cached);
}
var allowed = cached
? Decisions.GetOrAdd(
snapshot.GenerationId, sessionKey, operation, realm, (string)nodeId.Identifier,
() => Walk(snapshot, identity, operation, scope, now))
: Walk(snapshot, identity, operation, scope, now);
var sessionKey = identity.ApiKeyId ?? identity.Principal;
if (allowed) return null;
if (cached &&
Decisions.TryGet(snapshot.GenerationId, sessionKey, operation, realm, identifier, out var memoized))
{
// The whole point of the reorder: a hit returns HERE, having built no scope, no closure and no
// UserAuthorizationState. One Browse of a device folder asks this once per child.
return memoized
? null
: DenyNoGrant(operation, nodeId, identity.Principal, snapshot.GenerationId, cached);
}
return Deny(operation, nodeId, identity.Principal,
$"no grant in generation {snapshot.GenerationId} confers {operation} here", cached);
var scope = BuildScope(identifier, realm, snapshot.ClusterId);
var allowed = Walk(snapshot, identity, operation, scope, now);
if (cached) Decisions.Set(snapshot.GenerationId, sessionKey, operation, realm, identifier, allowed);
return allowed
? null
: DenyNoGrant(operation, nodeId, identity.Principal, snapshot.GenerationId, cached);
}
/// <summary>The grant walk itself — the only part of a decision that is safe to memoize.</summary>
@@ -274,7 +320,7 @@ public sealed class DataPlaneAuthorizer
}
/// <summary>
/// Projects an SDK <see cref="NodeId"/> onto the scope the permission trie is keyed by.
/// Projects a node's <c>s=</c> identifier onto the scope the permission trie is keyed by.
/// </summary>
/// <remarks>
/// The <c>s=</c> identifier IS the path in both realms — a RawPath
@@ -286,41 +332,108 @@ public sealed class DataPlaneAuthorizer
/// Segments are compared ordinally, matching <c>RawPaths.Comparer</c> and OPC UA NodeId
/// identity — <c>Speed</c> and <c>speed</c> are different nodes and must be different scopes.
/// </para>
/// <para>
/// Takes the identifier rather than the <see cref="NodeId"/>, and cannot fail: the "not a
/// string" guard this used to carry moved up into <c>Evaluate</c>, where it has to run before
/// the cache lookup anyway — the identifier is the cache key.
/// </para>
/// </remarks>
/// <param name="nodeId">The target node.</param>
/// <param name="identifier">The node's non-empty <c>s=</c> identifier.</param>
/// <param name="realm">The namespace the node was resolved in.</param>
/// <param name="clusterId">The node's cluster.</param>
/// <param name="scope">The resolved scope.</param>
/// <returns><see langword="true"/> when a scope could be built.</returns>
private static bool TryBuildScope(
NodeId nodeId, AddressSpaceRealm realm, string clusterId, out NodeScope scope)
/// <returns>The scope to evaluate against.</returns>
private static NodeScope BuildScope(string identifier, AddressSpaceRealm realm, string clusterId) => new()
{
scope = null!;
if (nodeId?.Identifier is not string identifier || identifier.Length == 0) return false;
scope = new NodeScope
{
ClusterId = clusterId,
Kind = realm == AddressSpaceRealm.Raw ? NodeHierarchyKind.Raw : NodeHierarchyKind.Uns,
Segments = identifier.Split('/', StringSplitOptions.RemoveEmptyEntries),
};
return true;
}
ClusterId = clusterId,
Kind = realm == AddressSpaceRealm.Raw ? NodeHierarchyKind.Raw : NodeHierarchyKind.Uns,
Segments = identifier.Split('/', StringSplitOptions.RemoveEmptyEntries),
};
/// <summary>Denies with a constant reason — nothing to build, so nothing to defer.</summary>
/// <param name="operation">The operation being attempted.</param>
/// <param name="nodeId">The target node.</param>
/// <param name="principal">The calling principal, when one resolved.</param>
/// <param name="reason">The operator-facing explanation.</param>
/// <param name="cached">Whether this came from a request-rate seam (which logs at Debug).</param>
/// <returns>The veto to return to the SDK.</returns>
private ServiceResult Deny(
OpcUaOperation operation, NodeId? nodeId, string? principal, string reason, bool cached = false)
{
// Logged, never returned. The client gets an undifferentiated BadUserAccessDenied so the server
// cannot be used to probe which nodes exist or which grants a principal holds; the operator
// debugging a denial reads the reason here. Same posture as the authenticator's DenyMessage.
//
// The hot seams log at Debug because a single Browse of an ungranted folder would otherwise emit
// one Information line per child. Their operator-facing signal is the per-request summary the node
// manager logs, which says how many references were filtered and for whom.
_log.Log(
cached ? LogLevel.Debug : LogLevel.Information,
var level = LevelFor(cached);
if (_log.IsEnabled(level)) LogDenial(level, operation, nodeId, principal, reason);
return DeniedResult;
}
/// <summary>
/// Denies because the calling key is disabled or expired, composing the reason only if it will be
/// logged.
/// </summary>
/// <param name="operation">The operation being attempted.</param>
/// <param name="nodeId">The target node.</param>
/// <param name="principal">The calling principal.</param>
/// <param name="keyId">The key that is no longer usable.</param>
/// <param name="generation">The authorization generation the check ran under.</param>
/// <param name="cached">Whether this came from a request-rate seam (which logs at Debug).</param>
/// <returns>The veto to return to the SDK.</returns>
private ServiceResult DenyRevokedKey(
OpcUaOperation operation, NodeId? nodeId, string? principal, string keyId, long generation, bool cached)
{
var level = LevelFor(cached);
if (_log.IsEnabled(level))
{
LogDenial(level, operation, nodeId, principal,
$"key {keyId} is no longer enabled or has expired in generation {generation}");
}
return DeniedResult;
}
/// <summary>
/// Denies because no grant conferred the operation — the hot denial, hit once per child of an
/// ungranted folder, so the reason string is built only when Debug is actually on.
/// </summary>
/// <param name="operation">The operation being attempted.</param>
/// <param name="nodeId">The target node.</param>
/// <param name="principal">The calling principal.</param>
/// <param name="generation">The authorization generation the walk ran under.</param>
/// <param name="cached">Whether this came from a request-rate seam (which logs at Debug).</param>
/// <returns>The veto to return to the SDK.</returns>
private ServiceResult DenyNoGrant(
OpcUaOperation operation, NodeId? nodeId, string? principal, long generation, bool cached)
{
var level = LevelFor(cached);
if (_log.IsEnabled(level))
{
LogDenial(level, operation, nodeId, principal,
$"no grant in generation {generation} confers {operation} here");
}
return DeniedResult;
}
/// <summary>
/// The hot seams log at Debug because a single Browse of an ungranted folder would otherwise emit
/// one Information line per child. Their operator-facing signal is the per-request summary
/// <see cref="LogFiltered"/> emits, which says how many references were filtered and for whom.
/// </summary>
/// <param name="cached">Whether the denial came from a request-rate seam.</param>
/// <returns>The level to log the denial at.</returns>
private static LogLevel LevelFor(bool cached) => cached ? LogLevel.Debug : LogLevel.Information;
/// <summary>
/// Emits the denial detail. Callers guard with <see cref="ILogger.IsEnabled"/> first — not for the
/// message template, which costs nothing, but because every caller's <paramref name="reason"/> is a
/// composed string and two of them interpolate on the request-rate path.
/// </summary>
/// <param name="level">The level chosen by <see cref="LevelFor"/>.</param>
/// <param name="operation">The operation being attempted.</param>
/// <param name="nodeId">The target node.</param>
/// <param name="principal">The calling principal, when one resolved.</param>
/// <param name="reason">The operator-facing explanation.</param>
private void LogDenial(
LogLevel level, OpcUaOperation operation, NodeId? nodeId, string? principal, string reason)
=> _log.Log(
level,
"AclGate: denied {Operation} on {NodeId} for principal {Principal} — {Reason}",
operation, nodeId, principal ?? "(none)", reason);
return new ServiceResult(StatusCodes.BadUserAccessDenied);
}
}
src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Security/SessionAclDecisionCache.cs
+180 -27
View File
@@ -31,34 +31,175 @@ namespace ZB.MOM.WW.OtOpcUa.OpcUaServer.Security;
/// nodes a changed grant covers, which is the trie walk run backwards; dropping everything on a
/// deploy costs one repopulation and cannot be wrong. Deployments are rare relative to reads.
/// </para>
/// <para>
/// <b>Storage is sharded per calling KEY, under a two-level bound.</b> The shard key is
/// <c>ApiKeyId ?? Principal</c>, so <i>sessions sharing one API key share one shard and one
/// <see cref="MaxEntriesPerSession"/> cap</i> — the isolation is across distinct keys, not across
/// sessions. Overflowing a shard clears <i>that shard only</i>, so a crawl by one key cannot evict
/// another key's entries. A second, global <see cref="MaxEntries"/> bound clears every shard, and
/// exists purely so many keys each staying under their own cap cannot add up to unbounded growth.
/// The flat cache this replaced had only the global bound, which made one machine consumer
/// crawling a large tree an eviction event for every other caller on the server — the callers
/// doing ordinary work paid for the crawl. Both caps are enforced against
/// <see cref="Interlocked"/> counters, never
/// <see cref="ConcurrentDictionary{TKey,TValue}.Count"/>, which locks the whole table.
/// </para>
/// </remarks>
public sealed class SessionAclDecisionCache
{
/// <summary>
/// Entry ceiling, past which the cache clears rather than growing. Large enough that a realistic
/// address space browsed by a handful of machine consumers never reaches it, small enough that a
/// pathological client cannot use it to exhaust memory.
/// Process-wide entry ceiling, past which <b>every</b> shard clears. The outer of the two bounds,
/// and a rare fallback: it exists so a large fleet of distinct keys each staying well under
/// <see cref="MaxEntriesPerSession"/> still cannot grow the cache without limit.
/// </summary>
/// <remarks>
/// Clearing beats evicting here: an LRU needs per-entry bookkeeping on the hot path to save a trie
/// walk that is itself cheap, so the eviction policy would cost more than the miss it prevents. The
/// failure mode of clearing is a burst of recomputation, which is exactly the steady state of not
/// having a cache at all.
/// Enforced against an <see cref="Interlocked"/> counter rather than
/// <see cref="ConcurrentDictionary{TKey,TValue}.Count"/>, which takes every lock in the table — a
/// price the store path would pay on every miss to answer a question that only matters at the
/// ceiling. The counter is approximate under concurrency by design, and is reset by every clear so
/// drift cannot ratchet permanently.
/// </remarks>
public const int MaxEntries = 100_000;
private readonly ConcurrentDictionary<DecisionKey, bool> _entries = new();
/// <summary>
/// Per-<b>key</b> entry ceiling (the shard key is <c>ApiKeyId ?? Principal</c>, so every session
/// using the same API key shares this one budget), past which that shard alone clears. The inner
/// and normally the only bound reached: it is what keeps one machine consumer crawling a 50k-node
/// tree from evicting the decisions taken for every OTHER key on the server. The name predates the
/// shard key and is kept for source compatibility.
/// </summary>
/// <remarks>
/// Clearing beats evicting <i>within a shard</i>: an LRU needs per-entry bookkeeping on the hot path
/// to save a trie walk that is itself cheap, so the eviction policy would cost more than the miss it
/// prevents. The failure mode of clearing is a burst of recomputation, which is exactly the steady
/// state of not having a cache at all — and sharding confines that burst to the key that caused it,
/// which is the part the old flat cache got wrong.
/// </remarks>
public const int MaxEntriesPerSession = 20_000;
private readonly ConcurrentDictionary<string, Shard> _shards = new(StringComparer.Ordinal);
private long _generation = -1;
private int _globalCount;
/// <summary>The authorization generation the cached decisions were taken under.</summary>
public long Generation => Interlocked.Read(ref _generation);
/// <summary>Number of decisions currently memoized. Diagnostics and tests.</summary>
public int Count => _entries.Count;
/// <summary>
/// Exact number of decisions currently memoized, across every shard. Diagnostics and tests only —
/// it enumerates the shards and takes each one's locks, so it must stay off the store path (which
/// uses the approximate counters above instead).
/// </summary>
public int Count
{
get
{
var total = 0;
foreach (var shard in _shards.Values) total += shard.Entries.Count;
return total;
}
}
/// <summary>
/// Looks a decision up without allocating anything — the hot half, and the reason the store half is
/// a separate method rather than a <see cref="Func{TResult}"/> parameter.
/// </summary>
/// <remarks>
/// <para>
/// <b>Why split from <see cref="GetOrAdd"/>.</b> A closure-taking API forces its caller to
/// build the closure — and everything the closure captures — <i>before</i> it knows whether the
/// answer is already memoized, so the allocation the cache exists to avoid is paid on the hit
/// path too. Read and Browse ask this question once per node, which is precisely where paying
/// for a walk that never runs is worth removing. <see cref="GetOrAdd"/> remains, delegating
/// here, but has no production caller — it is exercised by the cache's own tests.
/// </para>
/// <para>
/// The generation is checked (and a stale cache dropped) here rather than in
/// <see cref="Set"/> alone, so a lookup landing first after a deploy cannot read the previous
/// generation's verdicts.
/// </para>
/// </remarks>
/// <param name="generation">The authorization generation the decision is being taken under.</param>
/// <param name="sessionKey">Stable identity of the calling session — its key id, else its principal.</param>
/// <param name="operation">The operation being attempted.</param>
/// <param name="realm">The realm the node was resolved in.</param>
/// <param name="nodeIdentifier">The node's <c>s=</c> identifier.</param>
/// <param name="decision">The memoized verdict, when one exists.</param>
/// <returns><see langword="true"/> when a decision was memoized for this key.</returns>
public bool TryGet(
long generation,
string sessionKey,
OpcUaOperation operation,
AddressSpaceRealm realm,
string nodeIdentifier,
out bool decision)
{
InvalidateIfStale(generation);
decision = false;
// TryGetValue rather than GetOrAdd: a session that only ever hits must not mint a shard, and a
// session whose shard was just cleared must not resurrect one until it actually stores something.
return _shards.TryGetValue(sessionKey, out var shard)
&& shard.Entries.TryGetValue(new ShardKey(operation, realm, nodeIdentifier), out decision);
}
/// <summary>
/// Memoizes a decision, subject to the two entry bounds and the overtaken-generation drop rule.
/// </summary>
/// <param name="generation">The generation the decision was taken under — <b>not</b> re-read from the
/// caller, so a deploy landing mid-walk is detected here.</param>
/// <param name="sessionKey">Stable identity of the calling session — its key id, else its principal.</param>
/// <param name="operation">The operation that was attempted.</param>
/// <param name="realm">The realm the node was resolved in.</param>
/// <param name="nodeIdentifier">The node's <c>s=</c> identifier.</param>
/// <param name="decision">The verdict the grant walk produced.</param>
public void Set(
long generation,
string sessionKey,
OpcUaOperation operation,
AddressSpaceRealm realm,
string nodeIdentifier,
bool decision)
{
// Re-read the generation AFTER the walk. A deploy landing mid-walk means the verdict may have been
// taken against either generation, and storing it under the new one would pin a decision the new
// trie never made. Dropping it costs one recomputation on the next call.
if (Generation != generation) return;
var shard = _shards.GetOrAdd(sessionKey, static _ => new Shard());
if (Volatile.Read(ref _globalCount) >= MaxEntries)
{
// The outer bound: no single session is over its own ceiling, so there is nothing narrower to
// drop. Re-acquire the shard — ClearAll drops the shard objects themselves.
ClearAll();
shard = _shards.GetOrAdd(sessionKey, static _ => new Shard());
}
else if (Volatile.Read(ref shard.ApproxCount) >= MaxEntriesPerSession)
{
shard.Entries.Clear();
Interlocked.Add(ref _globalCount, -Interlocked.Exchange(ref shard.ApproxCount, 0));
}
// TryAdd, not an indexer assignment: only a genuine insert may move the counters, or a racing pair
// of walks over the same key would inflate them and pull the ceilings in for good.
if (shard.Entries.TryAdd(new ShardKey(operation, realm, nodeIdentifier), decision))
{
Interlocked.Increment(ref shard.ApproxCount);
Interlocked.Increment(ref _globalCount);
}
}
/// <summary>
/// Returns the memoized decision for this key, computing and storing it when absent or when the
/// generation has moved on.
/// generation has moved on. A convenience over <see cref="TryGet"/> + <see cref="Set"/>.
/// <para>
/// <b>Retained for tests; production has no caller.</b> <see cref="DataPlaneAuthorizer"/> uses
/// the split <see cref="TryGet"/>/<see cref="Set"/> pair precisely so a hit pays for no closure,
/// and no other caller has appeared since. Kept because it is the readable single-expression
/// form of the same protocol and is how the cache's tests drive it — not because a production
/// caller is expected back.
/// </para>
/// </summary>
/// <param name="generation">The authorization generation the decision is being taken under.</param>
/// <param name="sessionKey">Stable identity of the calling session — its key id, else its principal.</param>
@@ -77,25 +218,15 @@ public sealed class SessionAclDecisionCache
{
ArgumentNullException.ThrowIfNull(compute);
InvalidateIfStale(generation);
var key = new DecisionKey(sessionKey, operation, realm, nodeIdentifier);
if (_entries.TryGetValue(key, out var cached)) return cached;
if (TryGet(generation, sessionKey, operation, realm, nodeIdentifier, out var cached)) return cached;
var decided = compute();
// Re-read the generation AFTER computing. A deploy landing mid-walk means the verdict may have been
// taken against either generation, and storing it under the new one would pin a decision the new
// trie never made. Dropping it costs one recomputation on the next call.
if (Generation != generation) return decided;
if (_entries.Count >= MaxEntries) _entries.Clear();
_entries[key] = decided;
Set(generation, sessionKey, operation, realm, nodeIdentifier, decided);
return decided;
}
/// <summary>Drops every memoized decision. Called when the authorization generation advances.</summary>
public void Clear() => _entries.Clear();
public void Clear() => ClearAll();
private void InvalidateIfStale(long generation)
{
@@ -104,9 +235,31 @@ public sealed class SessionAclDecisionCache
// Only the thread that actually moves the stamp clears, so a burst of concurrent requests arriving
// just after a deploy repopulates once rather than each clearing the others' fresh entries.
if (Interlocked.CompareExchange(ref _generation, generation, seen) == seen) _entries.Clear();
if (Interlocked.CompareExchange(ref _generation, generation, seen) == seen) ClearAll();
}
private readonly record struct DecisionKey(
string SessionKey, OpcUaOperation Operation, AddressSpaceRealm Realm, string NodeIdentifier);
/// <summary>
/// Drops every shard and resets the global counter. Dropping the shard objects rather than emptying
/// them also sheds the shards of sessions that have since closed. A walk racing this holds a shard
/// that is no longer in the table; its entry is simply lost, which is the same outcome the clear
/// itself produces.
/// </summary>
private void ClearAll()
{
_shards.Clear();
Interlocked.Exchange(ref _globalCount, 0);
}
/// <summary>One session's decisions, plus the approximate entry count its own ceiling is enforced on.</summary>
private sealed class Shard
{
public readonly ConcurrentDictionary<ShardKey, bool> Entries = new();
/// <summary>Approximate by design — read and written with <see cref="Interlocked"/>, never derived
/// from <see cref="ConcurrentDictionary{TKey,TValue}.Count"/> on the store path.</summary>
public int ApproxCount;
}
private readonly record struct ShardKey(
OpcUaOperation Operation, AddressSpaceRealm Realm, string NodeIdentifier);
}
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverHostActor.cs
+267 -188
View File
@@ -50,9 +50,9 @@ namespace ZB.MOM.WW.OtOpcUa.Runtime.Drivers;
/// maintains the <c>FullName→NodeId</c> live-value routing map, and forwards results to the
/// OPC UA publish actor via <c>ForwardBatchToMux</c> — one whole driver production burst per message,
/// and the ONLY value ingress from driver children (<see cref="DriverInstanceActor"/> wraps even a
/// single data change in an <c>AttributeValueBatchPublished</c>). The per-value
/// <c>AttributeValuePublished</c> survives only as the message <see cref="FanCommsLossToDependencyMux"/>
/// fabricates straight to the dependency mux; the host itself no longer receives it.
/// single data change in an <c>AttributeValueBatchPublished</c>). <see cref="FanCommsLossToDependencyMux"/>
/// fabricates a batch of the same shape straight to the dependency mux, so the host neither receives nor
/// sends the per-value <c>AttributeValuePublished</c> any more.
/// </summary>
public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedStash
{
@@ -284,6 +284,34 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
/// value-subscription set.</summary>
private readonly Dictionary<(string DriverInstanceId, string RawPath), HashSet<NodeRealmRef>> _alarmNodeIdByDriverRef = new();
/// <summary>
/// Secondary index over <see cref="_nodeIdByDriverRef"/>, grouping its entries by
/// <c>DriverInstanceId</c>: <c>DriverInstanceId → [(RawPath, the SAME node set)]</c>. Rebuilt whole in
/// the same <see cref="ApplyDesiredSubscriptions"/> pass that rebuilds the primary — never maintained
/// incrementally, so the two cannot drift.
/// <para>
/// It exists because the connectivity handlers are the only readers that need <i>one driver's</i>
/// entries, and the primary is keyed by the composite <c>(DriverInstanceId, RawPath)</c> — so
/// answering "which refs does this driver own?" meant scanning every entry, i.e. the whole node's
/// address space, for a single driver's disconnect. On a 50k-tag node a driver flap walked 50k
/// keys to find its own few thousand.
/// </para>
/// <para>
/// The node set is held BY REFERENCE, not copied: the entry is captured when the primary's set is
/// created, so any node later added to that set (a UNS reference registered in the same pass) is
/// visible here without a second bookkeeping step.
/// </para>
/// </summary>
private readonly Dictionary<string, List<(string RawPath, HashSet<NodeRealmRef> Nodes)>> _valueRefsByDriver =
new(StringComparer.Ordinal);
/// <summary>The alarm-side twin of <see cref="_valueRefsByDriver"/>: a secondary index over
/// <see cref="_alarmNodeIdByDriverRef"/> grouping its entries by <c>DriverInstanceId</c>, so the
/// connectivity handler annotates one driver's conditions without scanning every condition on the node.
/// Rebuilt whole in the same pass, and likewise holding each condition-node set by reference.</summary>
private readonly Dictionary<string, List<(string RawPath, HashSet<NodeRealmRef> Nodes)>> _alarmRefsByDriver =
new(StringComparer.Ordinal);
/// <summary>
/// Inverse of <see cref="_alarmNodeIdByDriverRef"/>: <c>folder-scoped condition NodeId →
/// (DriverInstanceId, FullName = alarm ConditionId/AlarmFullReference)</c>. Built in the SAME apply
@@ -825,7 +853,7 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
case NodeDeploymentStatus.Applying:
_log.Warning("DriverHost {Node}: found orphan Applying row for deployment {Id}; replaying",
_localNode, latest.DeploymentId);
if (revision is not null) ApplyAndAck(new DeploymentId(latest.DeploymentId), revision.Value, CorrelationId.NewId());
if (revision is not null) BeginDirectApply(new DeploymentId(latest.DeploymentId), revision.Value, CorrelationId.NewId());
else BecomeSteady();
break;
case NodeDeploymentStatus.Failed:
@@ -1087,9 +1115,10 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
_localNode, msg.DeploymentId, _applyingDeploymentId);
Stash.Stash();
});
// The FetchAndCache fetch pipes its result back here (BeginFetchAndCacheApply ran in Steady,
// then Become(Applying)); completing the apply transitions back to Steady.
Receive<FetchedForApply>(HandleFetchedForApply);
// Both apply paths pipe their artifact bytes back here — the FetchAndCache gRPC fetch and the
// Direct ConfigDb read (BeginFetchAndCacheApply / BeginDirectApply ran in Steady, then
// Become(Applying)); completing the apply transitions back to Steady.
Receive<ArtifactForApply>(HandleArtifactForApply);
// The cache write is piped back too, and it outlives the apply that started it by design — so
// its completion can land in EITHER state and both must handle it or it dead-letters.
Receive<ArtifactCached>(HandleArtifactCached);
@@ -1231,16 +1260,20 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
var statusCode = msg.Connected ? OpcUaStatus.Good : OpcUaStatus.BadNoCommunication;
var ts = DateTime.UtcNow;
var annotated = 0;
// Fan out to every condition this driver owns. _alarmNodeIdByDriverRef is keyed by
// (DriverInstanceId, RawPath); one driver ref can back several condition NodeIds (identical machines).
foreach (var ((driverId, _), nodeIds) in _alarmNodeIdByDriverRef)
// Fan out to every condition this driver owns, resolved through the per-driver index rather than by
// scanning every condition on the node: _alarmNodeIdByDriverRef is keyed by the composite
// (DriverInstanceId, RawPath), so a filtered walk of it costs the whole address space per flap. One
// driver ref can still back several condition NodeIds (identical machines) — hence the inner loop.
if (_alarmRefsByDriver.TryGetValue(msg.DriverInstanceId, out var alarmEntries))
{
if (driverId != msg.DriverInstanceId) continue;
foreach (var n in nodeIds)
foreach (var (_, nodeIds) in alarmEntries)
{
_opcUaPublishActor.Tell(new ZB.MOM.WW.OtOpcUa.Runtime.OpcUa.OpcUaPublishActor.AlarmQualityUpdate(
n.NodeId, statusCode, ts, n.Realm));
annotated++;
foreach (var n in nodeIds)
{
_opcUaPublishActor.Tell(new ZB.MOM.WW.OtOpcUa.Runtime.OpcUa.OpcUaPublishActor.AlarmQualityUpdate(
n.NodeId, statusCode, ts, n.Realm));
annotated++;
}
}
}
@@ -1258,7 +1291,9 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
/// <para>
/// <b>The ref set already existed.</b> #481 asked for a driver → all-tag-refs map to be built;
/// <see cref="_nodeIdByDriverRef"/> is exactly that, keyed <c>(DriverInstanceId, RawPath)</c> and
/// rebuilt from the composition's <c>RawTags</c> ∪ <c>UnsReferenceVariables</c> on every apply.
/// rebuilt from the composition's <c>RawTags</c> ∪ <c>UnsReferenceVariables</c> on every apply —
/// read here through the per-driver <see cref="_valueRefsByDriver"/> index, since that composite
/// key cannot answer "this driver's refs" without walking the node's whole address space.
/// Alarm-bearing tags live in the separate <see cref="_alarmNodeIdByDriverRef"/> and are
/// deliberately absent here — a native condition is not a mux value, and its comms quality is
/// annotated by the caller through the Part 9 path instead.
@@ -1286,29 +1321,41 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
/// Ungated by redundancy role, for the same reason as the native-condition annotation above: a
/// Secondary must keep its scripted-alarm evaluation coherent for failover. The mux fan-out
/// reaches only refs something is actually subscribed to, so a driver whose tags feed no script
/// and no virtual tag costs one dictionary scan and no delivery.
/// and no virtual tag costs one dictionary lookup, one envelope, and no delivery.
/// </para>
/// <para>
/// <b>ONE batch, not one message per ref.</b> The mux's batch receive filters per item against the
/// same registered-interest map its per-value receive consulted, so what each subscriber is handed
/// is unchanged — but a driver owning 5k tags now costs one Tell on its disconnect edge instead of
/// 5k, on the actor that also carries the node's steady value ingress.
/// </para>
/// </remarks>
/// <param name="driverInstanceId">The driver child that just reported comms loss.</param>
private void FanCommsLossToDependencyMux(string driverInstanceId)
{
if (_dependencyMux is null) return;
// Indexed, not a filtered scan of the composite-keyed primary: this used to walk every value ref on
// the node to find one driver's.
if (!_valueRefsByDriver.TryGetValue(driverInstanceId, out var entries) || entries.Count == 0) return;
var ts = DateTime.UtcNow;
var pushed = 0;
foreach (var (driverId, rawPath) in _nodeIdByDriverRef.Keys)
var pushed = entries.Count;
// ONE envelope for the whole driver, not one Tell per ref: a driver owning 5k tags cost 5k messages
// on a single connectivity edge. The mux's batch path filters the items per registered interest
// exactly as it does a steady production burst, so the delivery is unchanged — only the framing is.
var items = new DriverInstanceActor.AttributeValueItem[pushed];
for (var i = 0; i < pushed; i++)
{
if (driverId != driverInstanceId) continue;
_dependencyMux.Tell(new DriverInstanceActor.AttributeValuePublished(
driverInstanceId, rawPath, Value: null, OpcUaStatus.BadNoCommunication, ts));
pushed++;
items[i] = new DriverInstanceActor.AttributeValueItem(
entries[i].RawPath, Value: null, OpcUaStatus.BadNoCommunication, ts);
}
if (pushed > 0)
_log.Debug(
"DriverHost {Node}: driver {Driver} disconnected — pushed BadNoCommunication for {Count} mux ref(s)",
_localNode, driverInstanceId, pushed);
_dependencyMux.Tell(new DriverInstanceActor.AttributeValueBatchPublished(driverInstanceId, items));
// Unconditional: the empty case returned above, so reaching here means at least one ref was pushed.
_log.Debug(
"DriverHost {Node}: driver {Driver} disconnected — pushed BadNoCommunication for {Count} mux ref(s) in one batch",
_localNode, driverInstanceId, pushed);
}
private void ForwardNativeAlarm(DriverInstanceActor.AttributeAlarmPublished msg)
@@ -1657,7 +1704,7 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
_log.Warning("DriverHost {Node}: ignoring DispatchDeployment while Stale (DB unreachable)", _localNode);
});
// A cache store started before this node fell Stale still pipes back here (see
// CacheAppliedArtifact: Bootstrap's orphan-replay ApplyAndAck and RestoreApplied both start
// CacheAppliedArtifact: Bootstrap's orphan-replay BeginDirectApply and RestoreApplied both start
// stores, and either can be followed by the ConfigDb failure that lands us in Stale). Without
// these the pipe-back dead-letters and a failed cache write goes unreported in exactly the
// situation — central unreachable — where the local cache is the thing keeping the node alive.
@@ -1729,23 +1776,22 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
return;
}
ApplyAndAck(msg.DeploymentId, msg.RevisionHash, msg.CorrelationId);
BeginDirectApply(msg.DeploymentId, msg.RevisionHash, msg.CorrelationId);
}
/// <summary>
/// Per-cluster mesh Phase 3 (<c>FetchAndCache</c>). Kicks off a gRPC artifact fetch and pipes
/// the result back to <see cref="Self"/> as a <see cref="FetchedForApply"/> — the fetch is I/O
/// the result back to <see cref="Self"/> as an <see cref="ArtifactForApply"/> — the fetch is I/O
/// and MUST NOT block the mailbox for the fetch deadline, so it never runs as an awaited call
/// inside a receive. The apply itself completes synchronously on the actor thread in
/// <see cref="HandleFetchedForApply"/>, mirroring <see cref="ApplyAndAck"/>.
/// <see cref="HandleArtifactForApply"/>, the continuation both modes share.
/// </summary>
/// <remarks>
/// This is the ONLY path that arms the <see cref="ApplyWedgeTimerKey"/> watchdog, deliberately.
/// <see cref="ApplyAndAck"/> enters and leaves <see cref="Applying"/> inside a single mailbox
/// turn, so it cannot be "stuck in Applying" in any sense a watchdog could report: if it hung,
/// it would be hanging the actor thread, and a watchdog delivered as a mailbox message could not
/// be dequeued to say so. The fetch is the only unbounded wait during which the host is still
/// alive and dequeuing — which is exactly what makes a silent wedge possible there.
/// Both begin-paths arm the <see cref="ApplyWedgeTimerKey"/> watchdog, because both leave the
/// host alive and dequeuing while it waits for bytes it does not have yet — which is exactly
/// what makes a silent wedge possible. (Before the Direct read moved off the mailbox this was
/// the only such path: an inline read could not be "stuck in Applying" in any sense a watchdog
/// could report, since a mailbox message cannot be dequeued by an actor thread that is hung.)
/// </remarks>
private void BeginFetchAndCacheApply(DeploymentId deploymentId, RevisionHash revision, CorrelationId correlation)
{
@@ -1753,7 +1799,7 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
Become(Applying);
Timers.StartSingleTimer(ApplyWedgeTimerKey, new ApplyWedged(deploymentId), _applyWedgeTimeout);
// Persist Applying row (idempotent on PK) — same crash-boundary marker as ApplyAndAck.
// Persist Applying row (idempotent on PK) — same crash-boundary marker as the Direct apply.
UpsertNodeDeploymentState(deploymentId, NodeDeploymentStatus.Applying, failureReason: null);
if (_artifactFetcher is null)
@@ -1777,21 +1823,70 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
_artifactFetcher.FetchAsync(deploymentId.ToString(), revision.Value, CancellationToken.None)
.PipeTo(
Self,
success: bytes => new FetchedForApply(deploymentId, revision, correlation, bytes),
failure: _ => new FetchedForApply(deploymentId, revision, correlation, Bytes: null));
success: bytes => new ArtifactForApply(
deploymentId, revision, correlation, bytes, ArtifactSource.CentralFetch),
failure: _ => new ArtifactForApply(
deploymentId, revision, correlation, Bytes: null, ArtifactSource.CentralFetch));
}
/// <summary>
/// Completes a <c>FetchAndCache</c> apply once the fetched bytes are in hand. Null bytes are an
/// apply FAILURE (keep last-known-good, do not advance the revision — #485); non-null bytes are
/// applied from hand (no ConfigDb read) and cached.
/// Perf roadmap Task 26 (<c>Direct</c>). Kicks off the central-SQL artifact read on a pool
/// thread and pipes the bytes back to <see cref="Self"/> as an <see cref="ArtifactForApply"/>,
/// the same shape <see cref="BeginFetchAndCacheApply"/> has always had for its gRPC fetch. The
/// apply itself completes on the actor thread in <see cref="HandleArtifactForApply"/>.
/// </summary>
private void HandleFetchedForApply(FetchedForApply msg)
/// <remarks>
/// <para>
/// The read used to run INLINE in the receive. <c>Deployment.ArtifactBlob</c> is a
/// potentially multi-MB gzip blob in central SQL, so for the whole of that round trip the
/// host dequeued nothing: no driver value batch, no diagnostics request, no driver command,
/// no next dispatch. Only the byte SOURCE differs between the two modes, and only the
/// source ever should — everything downstream of "bytes in hand" is one continuation.
/// </para>
/// <para>
/// The bytes come back as a MESSAGE, never as an escaping fault (#491): the task closes over
/// <see cref="_dbFactory"/>, <see cref="_localNode"/> and <see cref="_log"/> and touches
/// neither <see cref="Self"/> nor <c>Context</c>, <see cref="LoadArtifactBlob"/> already
/// swallows its own DB failures into a null return, and the failure mapper turns anything it
/// cannot (an OOM on an oversized blob, a factory that throws) into the same null-bytes
/// apply failure. Supervision never sees a throw from here.
/// </para>
/// </remarks>
private void BeginDirectApply(DeploymentId deploymentId, RevisionHash revision, CorrelationId correlation)
{
_applyingDeploymentId = deploymentId;
Become(Applying);
Timers.StartSingleTimer(ApplyWedgeTimerKey, new ApplyWedged(deploymentId), _applyWedgeTimeout);
// Persist Applying row (idempotent on PK) — the crash-boundary marker, and deliberately still
// on the actor thread: one bounded row write, and the apply must be recorded as in flight
// before the read that could outlive this process starts. See UpsertNodeDeploymentState.
UpsertNodeDeploymentState(deploymentId, NodeDeploymentStatus.Applying, failureReason: null);
_log.Debug("DriverHost {Node}: reading artifact for {Id} (rev {Rev}) from the ConfigDb, off the mailbox",
_localNode, deploymentId, revision);
Task.Run(() => LoadArtifactBlob(deploymentId))
.PipeTo(
Self,
success: bytes => new ArtifactForApply(
deploymentId, revision, correlation, bytes, ArtifactSource.ConfigDb),
failure: _ => new ArtifactForApply(
deploymentId, revision, correlation, Bytes: null, ArtifactSource.ConfigDb));
}
/// <summary>
/// Completes an apply once the artifact bytes are in hand — the ONE continuation both modes
/// share, whether the bytes were read from the ConfigDb or fetched from central. Null bytes are
/// an apply FAILURE (keep last-known-good, do not advance the revision — #485/#486); non-null
/// bytes are applied from hand and cached.
/// </summary>
private void HandleArtifactForApply(ArtifactForApply msg)
{
// Ignore a result for anything but the in-flight apply (a stale/duplicate pipe-back).
if (_applyingDeploymentId is null || msg.DeploymentId != _applyingDeploymentId.Value)
{
_log.Debug("DriverHost {Node}: dropping stale fetch result for {Id} (in-flight={Cur})",
_log.Debug("DriverHost {Node}: dropping stale artifact result for {Id} (in-flight={Cur})",
_localNode, msg.DeploymentId, _applyingDeploymentId);
return;
}
@@ -1807,21 +1902,27 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
ParsedDeploymentArtifact? parsed = null;
try
{
// #485: null bytes = "no answer" (every endpoint unreachable / NotFound / SHA mismatch).
// Fail the apply and stay on the current revision so a re-dispatch actually retries; keep
// serving the last-known-good address space, drivers and subscriptions throughout.
// #485/#486: null bytes = "no answer" — the ConfigDb was unreachable or the row is
// missing/empty (Direct), or every endpoint was unreachable / NotFound / SHA mismatch
// (FetchAndCache). Either way NOTHING was applied, so fail the apply and stay on the
// current revision: HandleDispatchFromSteady short-circuits on a revision match, so
// claiming Applied here would make the node unhealable by re-dispatching that revision.
// The last-known-good address space, drivers and subscriptions keep serving throughout.
var (appliedBlob, parsedArtifact) = ReconcileDriversFromParsed(msg.DeploymentId, msg.Bytes);
parsed = parsedArtifact;
if (appliedBlob is null || parsed is null)
{
const string reason =
"the deployment artifact could not be fetched from central (all endpoints unreachable, NotFound, or SHA-256 mismatch); nothing was applied";
var reason = msg.Source == ArtifactSource.ConfigDb
? "the deployment artifact could not be read (ConfigDb unreachable, or the row is missing/empty); nothing was applied"
: "the deployment artifact could not be fetched from central (all endpoints unreachable, NotFound, or SHA-256 mismatch); nothing was applied";
UpsertNodeDeploymentState(msg.DeploymentId, NodeDeploymentStatus.Failed, reason);
SendAck(msg.DeploymentId, ApplyAckOutcome.Failed, reason, msg.Correlation);
OtOpcUaTelemetry.DeploymentApplied.Add(1, new KeyValuePair<string, object?>("outcome", "reject"));
span?.SetStatus(ActivityStatusCode.Error, reason);
_log.Error(
"DriverHost {Node}: FetchAndCache apply of {Id} FAILED — {Reason}. Staying on revision {Rev} and continuing to serve it; re-dispatch once central is reachable",
msg.Source == ArtifactSource.ConfigDb
? "DriverHost {Node}: apply of {Id} FAILED — {Reason}. Staying on revision {Rev} and continuing to serve it; re-dispatch once the ConfigDb is readable"
: "DriverHost {Node}: FetchAndCache apply of {Id} FAILED — {Reason}. Staying on revision {Rev} and continuing to serve it; re-dispatch once central is reachable",
_localNode, msg.DeploymentId, reason, _currentRevision);
return;
}
@@ -1829,19 +1930,34 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
_currentRevision = msg.Revision;
UpsertNodeDeploymentState(msg.DeploymentId, NodeDeploymentStatus.Applied, failureReason: null);
SendAck(msg.DeploymentId, ApplyAckOutcome.Applied, failureReason: null, msg.Correlation);
// Rebuild the address space + push subscriptions FROM the fetched bytes so OpcUaPublishActor
// never reads central SQL (the whole point of FetchAndCache).
// The message carries the raw BYTES, not `parsed`: the publish actor is a different actor with
// its own mailbox, and ParsedDeploymentArtifact is a disposable, non-thread-safe object owned
// by this handler. It opens its own (single) parse on the other side of the mailbox.
// Rebuild the address space + push subscriptions FROM the bytes this apply already has, so
// nothing downstream re-reads the artifact: under FetchAndCache OpcUaPublishActor has no
// central SQL to read at all, and under Direct a bytes-less RebuildAddressSpace would make
// it re-read the same immutable row while the SubscribeBulk pass read it a THIRD time.
// Consequence: OpcUaPublishActor.LoadArtifact / LoadLatestArtifact are on no apply path;
// they remain only for a legacy/dev caller that Tells RebuildAddressSpace without bytes.
//
// What rides along is the raw BYTES, deliberately — not `parsed`. The publish actor is a
// different actor with its own mailbox, and ParsedDeploymentArtifact is a disposable,
// non-thread-safe object owned by this handler (disposed in the finally below, likely before
// the other actor even dequeues). It opens its own single parse on the far side.
_opcUaPublishActor?.Tell(new ZB.MOM.WW.OtOpcUa.Runtime.OpcUa.OpcUaPublishActor.RebuildAddressSpace(
msg.Correlation, msg.DeploymentId, appliedBlob));
// SubscribeBulk pass: hand each driver its desired tag references so live values flow into
// the just-rebuilt address space instead of staying BadWaitingForInitialData.
PushDesiredSubscriptionsFromParsed(msg.DeploymentId, parsed);
// Cache the fetched bytes as this node's last-known-good (idempotent store; skips empty).
// Cache the applied bytes as this node's last-known-good (idempotent store; skips empty).
CacheAppliedArtifact(msg.DeploymentId, msg.Revision, appliedBlob, parsed);
// Reaching here means the artifact came from central, so the node is no longer serving a
// cache-sourced configuration. Note this only clears on a real apply: a dispatch of the
// revision already booted from cache short-circuits in HandleDispatchFromSteady without
// touching the ConfigDb, and the flag correctly stays set.
_isRunningFromCache = false;
OtOpcUaTelemetry.DeploymentApplied.Add(1, new KeyValuePair<string, object?>("outcome", "ack"));
_log.Info("DriverHost {Node}: applied fetched deployment {Id} (rev {Rev}, children={Count})",
_log.Info(
msg.Source == ArtifactSource.ConfigDb
? "DriverHost {Node}: applied deployment {Id} (rev {Rev}, children={Count})"
: "DriverHost {Node}: applied fetched deployment {Id} (rev {Rev}, children={Count})",
_localNode, msg.DeploymentId, msg.Revision, _children.Count);
}
catch (Exception ex)
@@ -1850,7 +1966,12 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
SendAck(msg.DeploymentId, ApplyAckOutcome.Failed, ex.Message, msg.Correlation);
OtOpcUaTelemetry.DeploymentApplied.Add(1, new KeyValuePair<string, object?>("outcome", "reject"));
span?.SetStatus(ActivityStatusCode.Error, ex.Message);
_log.Error(ex, "DriverHost {Node}: FetchAndCache apply of {Id} failed", _localNode, msg.DeploymentId);
_log.Error(
ex,
msg.Source == ArtifactSource.ConfigDb
? "DriverHost {Node}: apply of {Id} failed"
: "DriverHost {Node}: FetchAndCache apply of {Id} failed",
_localNode, msg.DeploymentId);
}
finally
{
@@ -1889,110 +2010,30 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
_log.Warning(
"DriverHost {Node}: deployment {Id} appears WEDGED — still applying after {Timeout} " +
"(the artifact fetch has neither answered nor failed); {Deferred} further dispatch(es) are " +
"deferred behind it. The node keeps serving revision {Rev}.",
"(the artifact read/fetch has neither answered nor failed); {Deferred} further dispatch(es) " +
"are deferred behind it. The node keeps serving revision {Rev}.",
_localNode, msg.DeploymentId, _applyWedgeTimeout, _deferredDispatches.Count, _currentRevision);
}
/// <summary>Pipe-back carrier for a <c>FetchAndCache</c> fetch result (null bytes = no answer).</summary>
private sealed record FetchedForApply(
DeploymentId DeploymentId, RevisionHash Revision, CorrelationId Correlation, byte[]? Bytes);
private void ApplyAndAck(DeploymentId deploymentId, RevisionHash revision, CorrelationId correlation)
/// <summary>Where an apply's artifact bytes came from — the ONLY difference between the two apply
/// paths, and therefore the only thing <see cref="HandleArtifactForApply"/> branches on.</summary>
private enum ArtifactSource
{
_applyingDeploymentId = deploymentId;
Become(Applying);
/// <summary><c>Direct</c> mode: read from <c>Deployment.ArtifactBlob</c> in central SQL.</summary>
ConfigDb,
using var span = OtOpcUaTelemetry.StartDeployApplySpan(deploymentId.ToString());
span?.SetTag("otopcua.node_id", _localNode.ToString());
span?.SetTag("otopcua.revision", revision.ToString());
span?.SetTag("otopcua.correlation_id", correlation.ToString());
var sw = Stopwatch.StartNew();
// Persist Applying row (idempotent on PK).
UpsertNodeDeploymentState(deploymentId, NodeDeploymentStatus.Applying, failureReason: null);
// Owned by this handler, disposed in the finally: ONE JsonDocument for the whole apply. Declared
// out here (rather than a `using` inside the try) so the catch/finally still see it.
ParsedDeploymentArtifact? parsed = null;
try
{
var (appliedBlob, parsedArtifact) = ReconcileDriversForApply(deploymentId);
parsed = parsedArtifact;
// Issue #486 — a null blob means the artifact could not be read (unreachable ConfigDb, or the
// empty bytes a missing row yields), so NOTHING was applied. Reporting Applied here used to
// advance _currentRevision too, and HandleDispatchFromSteady short-circuits on a revision
// match — so the node claimed a configuration it never applied AND could never be healed by
// re-dispatching that revision. Failing the apply keeps the revision where it was, which is
// what makes the retry land instead of being waved through. The node keeps serving its
// last-known-good address space, drivers and subscriptions throughout (issue #485) — this is
// about telling the truth, not about tearing anything down.
if (appliedBlob is null || parsed is null)
{
const string reason =
"the deployment artifact could not be read (ConfigDb unreachable, or the row is missing/empty); nothing was applied";
UpsertNodeDeploymentState(deploymentId, NodeDeploymentStatus.Failed, reason);
SendAck(deploymentId, ApplyAckOutcome.Failed, reason, correlation);
OtOpcUaTelemetry.DeploymentApplied.Add(1, new KeyValuePair<string, object?>("outcome", "reject"));
span?.SetStatus(ActivityStatusCode.Error, reason);
_log.Error(
"DriverHost {Node}: apply of {Id} FAILED — {Reason}. Staying on revision {Rev} and continuing to serve it; re-dispatch once the ConfigDb is readable",
_localNode, deploymentId, reason, _currentRevision);
return;
}
_currentRevision = revision;
UpsertNodeDeploymentState(deploymentId, NodeDeploymentStatus.Applied, failureReason: null);
SendAck(deploymentId, ApplyAckOutcome.Applied, failureReason: null, correlation);
// Trigger the OPC UA address-space rebuild so the local SDK reflects the new
// composition. The publish actor handles the load-compose-diff-apply pipeline; we
// just forward the same correlation id so the audit trail joins up.
//
// The already-read bytes ride along. ReconcileDrivers just read this exact blob out of central
// SQL and handed it back (it is non-null here — the null case returned above), so a bytes-less
// RebuildAddressSpace would make the publish actor re-read the same immutable row, and the
// SubscribeBulk pass would read it a THIRD time: three round-trips per apply for one blob that
// cannot have changed between them. Threading it is also the shape the FetchAndCache apply has
// always had (HandleFetchedForApply), for the harder reason that it has no ConfigDb to re-read.
// Consequence: OpcUaPublishActor.LoadArtifact / LoadLatestArtifact are no longer on any apply
// path — they remain only for a legacy/dev caller that Tells RebuildAddressSpace without bytes.
//
// What rides along is the BYTES, deliberately — not this apply's `parsed`. The publish actor is
// a different actor with its own mailbox, and ParsedDeploymentArtifact is a disposable,
// non-thread-safe object owned by this handler (disposed in the finally below, likely before the
// other actor even dequeues). It opens its own single parse on the far side.
_opcUaPublishActor?.Tell(new ZB.MOM.WW.OtOpcUa.Runtime.OpcUa.OpcUaPublishActor.RebuildAddressSpace(correlation, deploymentId, appliedBlob));
// SubscribeBulk pass: hand each driver its desired tag references so live values flow into
// the just-rebuilt address space instead of staying BadWaitingForInitialData.
PushDesiredSubscriptionsFromParsed(deploymentId, parsed);
CacheAppliedArtifact(deploymentId, revision, appliedBlob, parsed);
// Reaching here means the artifact came from central, so the node is no longer serving a
// cache-sourced configuration. Note this only clears on a real apply: a dispatch of the
// revision already booted from cache short-circuits in HandleDispatchFromSteady without
// touching the ConfigDb, and the flag correctly stays set.
_isRunningFromCache = false;
OtOpcUaTelemetry.DeploymentApplied.Add(1, new KeyValuePair<string, object?>("outcome", "ack"));
_log.Info("DriverHost {Node}: applied deployment {Id} (rev {Rev}, children={Count})",
_localNode, deploymentId, revision, _children.Count);
}
catch (Exception ex)
{
UpsertNodeDeploymentState(deploymentId, NodeDeploymentStatus.Failed, ex.Message);
SendAck(deploymentId, ApplyAckOutcome.Failed, ex.Message, correlation);
OtOpcUaTelemetry.DeploymentApplied.Add(1, new KeyValuePair<string, object?>("outcome", "reject"));
span?.SetStatus(ActivityStatusCode.Error, ex.Message);
_log.Error(ex, "DriverHost {Node}: apply of {Id} failed", _localNode, deploymentId);
}
finally
{
parsed?.Dispose();
OtOpcUaTelemetry.DeploymentApplyDurationSec.Record(sw.Elapsed.TotalSeconds);
_applyingDeploymentId = null;
BecomeSteady();
}
/// <summary><c>FetchAndCache</c> mode: fetched from central over gRPC.</summary>
CentralFetch,
}
/// <summary>Pipe-back carrier for an apply's artifact bytes (null bytes = no answer).</summary>
private sealed record ArtifactForApply(
DeploymentId DeploymentId,
RevisionHash Revision,
CorrelationId Correlation,
byte[]? Bytes,
ArtifactSource Source);
/// <summary>
/// Read the deployment artifact + reconcile the set of running <see cref="DriverInstanceActor"/>
/// children. Spawn missing, ApplyDelta on config change, stop removed/disabled drivers.
@@ -2007,7 +2048,7 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
/// <remarks>
/// <para>
/// The <b>bootstrap-restore</b> entry point (<see cref="RestoreApplied"/>). The apply paths use
/// <see cref="ReconcileDriversForApply"/>, which keeps its parse for the rest of the apply.
/// <see cref="ReconcileDriversFromParsed"/>, which keeps its parse for the rest of the apply.
/// </para>
/// <para>
/// Returning the blob rather than swallowing it is load-bearing for the artifact cache. This
@@ -2026,9 +2067,16 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
/// <summary>
/// The ConfigDb read half of <see cref="ReconcileDrivers"/>, shared with the Direct apply path
/// (<see cref="ReconcileDriversForApply"/>) so both read the artifact exactly once and log the
/// (<see cref="BeginDirectApply"/>) so both read the artifact exactly once and log the
/// same way when they cannot.
/// </summary>
/// <remarks>
/// Called on the actor thread by the PreStart restore path and on a POOL thread by
/// <see cref="BeginDirectApply"/>, so it must stay free of <see cref="Self"/> / <c>Context</c>
/// (#491). It touches only <see cref="_dbFactory"/>, <see cref="_localNode"/> and
/// <see cref="_log"/>, all of which are safe off the actor thread — the logger publishes to the
/// EventStream and captures nothing from the cell.
/// </remarks>
/// <returns>The artifact bytes (possibly empty, when the row is missing), or <see langword="null"/>
/// when there is no ConfigDb to read or the read failed.</returns>
private byte[]? LoadArtifactBlob(DeploymentId deploymentId)
@@ -2068,28 +2116,18 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
private readonly record struct ReconciledArtifact(byte[]? Blob, ParsedDeploymentArtifact? Parsed);
/// <summary>
/// The Direct apply path's reconcile: read the artifact from the ConfigDb, parse it ONCE, and
/// reconcile the driver children from that parse. The caller owns (and disposes) the returned
/// <see cref="ParsedDeploymentArtifact"/> and threads it through the rest of the apply, so a
/// Direct apply builds exactly one <c>JsonDocument</c> in this actor.
/// </summary>
/// <exception cref="System.Text.Json.JsonException">The artifact bytes are not valid JSON. Unlike the
/// lenient blob wrappers (malformed ⇒ no specs ⇒ every running child planned for StopChild) this
/// surfaces, and <see cref="ApplyAndAck"/>'s catch turns it into a Failed ACK with the children left
/// running — the honest reading of bytes that cannot be understood.</exception>
private ReconciledArtifact ReconcileDriversForApply(DeploymentId deploymentId)
{
var blob = LoadArtifactBlob(deploymentId);
return blob is null ? default : ReconcileDriversFromParsed(deploymentId, blob);
}
/// <summary>
/// The parse-once reconcile shared by both apply paths (<see cref="ApplyAndAck"/> via
/// <see cref="ReconcileDriversForApply"/>, and <see cref="HandleFetchedForApply"/> with the
/// fetched bytes already in hand).
/// The parse-once reconcile every apply runs, with the artifact bytes already in hand: parse
/// ONCE and reconcile the driver children from that parse. The caller
/// (<see cref="HandleArtifactForApply"/>) owns and disposes the returned
/// <see cref="ParsedDeploymentArtifact"/> and threads it through the rest of the apply, so an
/// apply builds exactly one <c>JsonDocument</c> in this actor.
/// </summary>
/// <returns>The reconciled blob + its parsed artifact, or a default (both null) when the blob was
/// empty (#485).</returns>
/// empty or absent (#485).</returns>
/// <exception cref="System.Text.Json.JsonException">The artifact bytes are not valid JSON. Unlike the
/// lenient blob wrappers (malformed ⇒ no specs ⇒ every running child planned for StopChild) this
/// surfaces, and <see cref="HandleArtifactForApply"/>'s catch turns it into a Failed ACK with the
/// children left running — the honest reading of bytes that cannot be understood.</exception>
private ReconciledArtifact ReconcileDriversFromParsed(DeploymentId deploymentId, byte[]? blob)
{
blob ??= Array.Empty<byte>();
@@ -2181,7 +2219,7 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
/// <para>
/// <b>Never throws.</b> By the time this runs the deployment is already recorded Applied
/// in central SQL and an Applied ACK has been sent to the coordinator. An exception
/// escaping here would unwind into <see cref="ApplyAndAck"/>'s catch and send a second,
/// escaping here would unwind into <see cref="HandleArtifactForApply"/>'s catch and send a second,
/// contradictory Failed ACK for a deployment the fleet already believes is live.
/// </para>
/// <para>
@@ -2204,7 +2242,7 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
/// <b>Stale is included on purpose.</b> This used to claim Stale could not be reached after a
/// store had started, because it is only ever entered from <see cref="Bootstrap"/>. That is
/// not a property of the Stale entry — <see cref="Bootstrap"/> starts stores of its own, via
/// the orphan-replay <see cref="ApplyAndAck"/> and via <see cref="RestoreApplied"/>, and both
/// the orphan-replay <see cref="BeginDirectApply"/> and via <see cref="RestoreApplied"/>, and both
/// run inside the very try block whose catch Becomes Stale. Today no statement between those
/// calls and the catch happens to throw, so the old claim held by accident of ordering rather
/// than by design; a restart can orphan an in-flight pipe-back the same way. Registering the
@@ -2322,6 +2360,12 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
/// drivers, rebuilds the address space from the applied artifact, and re-pushes SubscribeBulk.
/// No re-ack: the deployment is already Applied.
/// </summary>
/// <remarks>
/// Its two ConfigDb reads (<see cref="ReconcileDrivers"/> and
/// <see cref="PushDesiredSubscriptions"/>) stay SYNCHRONOUS deliberately: this runs in
/// <see cref="PreStart"/>, before the host has processed a single message, so there is no live
/// traffic behind them to block — unlike the apply path, which moved its read off the mailbox.
/// </remarks>
private void RestoreApplied(DeploymentId deploymentId, RevisionHash? revision)
{
var correlation = CorrelationId.NewId();
@@ -2332,7 +2376,7 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
PushDesiredSubscriptions(deploymentId);
// Populate the node-local cache from the artifact this restored node is now serving. The
// cache invariant is "holds what the node currently serves"; without this only a FRESH
// apply (ApplyAndAck) ever wrote it, so a node whose cache was lost — a wiped/fresh volume,
// apply (BeginDirectApply) ever wrote it, so a node whose cache was lost — a wiped/fresh volume,
// a disk failure — would recover its served state here yet stay cache-less, unable to
// boot-from-cache on the NEXT central outage until some future new deploy happened to land.
// Replication does not heal that gap either: a peer's already-acked rows are pruned from its
@@ -2357,10 +2401,10 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
/// (which clears any stale subscription from a previous deployment).
/// </summary>
/// <remarks>
/// The <b>bootstrap-restore</b> entry point only. Neither apply path reaches it any more: both
/// <see cref="ApplyAndAck"/> and <see cref="HandleFetchedForApply"/> already hold the apply's single
/// parse and call <see cref="PushDesiredSubscriptionsFromParsed"/> directly, so this ConfigDb read
/// exists for <see cref="RestoreApplied"/>, which restores from a deployment id recovered out of
/// The <b>bootstrap-restore</b> entry point only. Neither apply path reaches it any more:
/// <see cref="HandleArtifactForApply"/> already holds the apply's single parse and calls
/// <see cref="PushDesiredSubscriptionsFromParsed"/> directly, so this ConfigDb read exists for
/// <see cref="RestoreApplied"/>, which restores from a deployment id recovered out of
/// NodeDeploymentState and may hold no blob of its own.
/// </remarks>
private void PushDesiredSubscriptions(DeploymentId deploymentId)
@@ -2377,6 +2421,8 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
byte[] blob;
try
{
// Synchronous by design: the only caller is the PreStart restore path above, where nothing
// is queued behind this read (the apply path reads off the mailbox instead).
using var db = _dbFactory.CreateDbContext();
blob = db.Deployments.AsNoTracking()
.Where(d => d.DeploymentId == deploymentId.Value)
@@ -2455,6 +2501,18 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
ApplyDesiredSubscriptions(composition);
}
/// <summary>Gets (creating on first use) the per-driver bucket of a secondary index.</summary>
/// <param name="index">Either <see cref="_valueRefsByDriver"/> or <see cref="_alarmRefsByDriver"/>.</param>
/// <param name="driverInstanceId">The driver whose bucket is wanted.</param>
/// <returns>The mutable bucket for that driver.</returns>
private static List<(string RawPath, HashSet<NodeRealmRef> Nodes)> IndexFor(
Dictionary<string, List<(string RawPath, HashSet<NodeRealmRef> Nodes)>> index, string driverInstanceId)
{
if (!index.TryGetValue(driverInstanceId, out var bucket))
index[driverInstanceId] = bucket = new List<(string, HashSet<NodeRealmRef>)>();
return bucket;
}
/// <summary>
/// Hand each running driver child its desired value + alarm subscription set for
/// <paramref name="composition"/>, rebuild the live-value/write routing maps, and forward the
@@ -2509,6 +2567,10 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
_driverRefByAlarmNodeId.Clear();
_alarmMetaByNodeId.Clear();
_nativeAlarmProjector.Clear();
// The per-driver secondary indexes are rebuilt WHOLE alongside their primaries below — same pass,
// same clear — so no incremental maintenance exists to get out of step with them.
_valueRefsByDriver.Clear();
_alarmRefsByDriver.Clear();
foreach (var t in composition.RawTags)
{
var key = (t.DriverInstanceId, t.NodeId); // (DriverInstanceId, RawPath) — RawPath IS the wire-ref
@@ -2517,7 +2579,13 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
// Native alarm → a single Part 9 condition at the RawPath (Raw realm). WP4 adds the referencing
// equipment notifier NodeIds; WP3 wires the single raw condition.
if (!_alarmNodeIdByDriverRef.TryGetValue(key, out var aset))
{
_alarmNodeIdByDriverRef[key] = aset = new HashSet<NodeRealmRef>();
// Indexed on CREATION only, so a repeated key contributes exactly one entry — and the set
// is captured by reference, so later additions to it are seen through the index.
IndexFor(_alarmRefsByDriver, t.DriverInstanceId).Add((t.NodeId, aset));
}
aset.Add(new NodeRealmRef(t.NodeId, AddressSpaceRealm.Raw));
// Inverse 1:1 map for the inbound native-condition ack path (keyed by BARE condition NodeId).
_driverRefByAlarmNodeId[t.NodeId] = key;
@@ -2532,7 +2600,11 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
// Value tag: register the RAW NodeId (Raw realm) AND every referencing UNS NodeId (Uns realm)
// against the SAME driver ref — the single value source fans to all of them.
if (!_nodeIdByDriverRef.TryGetValue(key, out var set))
{
_nodeIdByDriverRef[key] = set = new HashSet<NodeRealmRef>();
IndexFor(_valueRefsByDriver, t.DriverInstanceId).Add((t.NodeId, set));
}
set.Add(new NodeRealmRef(t.NodeId, AddressSpaceRealm.Raw));
_driverRefByNodeId[(AddressSpaceRealm.Raw, t.NodeId)] = key;
@@ -2575,7 +2647,7 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
// Hand the Equipment-namespace VirtualTags to the host so it spawns/reconciles a
// VirtualTagActor per plan and streams their evaluated values back onto the just-rebuilt
// address space. Runs on BOTH the fresh-apply path (ApplyAndAck) and the bootstrap-restore
// address space. Runs on BOTH the fresh-apply path (HandleArtifactForApply) and the bootstrap-restore
// path (RestoreApplied) because both call this method, so one send covers both.
// NOTE: the Stale-recovery path (TryRecoverFromStale) does NOT call PushDesiredSubscriptions,
// so — like drivers — VirtualTags remain empty after a Stale recovery until the next
@@ -2600,7 +2672,7 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
_localNode, composition.EquipmentScriptedAlarms.Count);
}
// Cache the applied composition LAST. Set here (not in ApplyAndAck) so both the fresh-apply and
// Cache the applied composition LAST. Set here (not in HandleArtifactForApply) so both the fresh-apply and
// bootstrap-restore paths — which both route through this method — leave a current composition.
_lastComposition = composition;
@@ -2855,6 +2927,9 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
try
{
// Synchronous, like the Applying/Applied row writes below: one bounded row read on a timer
// tick, not a multi-MB blob. Recorded follow-up — if the Stale-recovery tick is ever shown
// to stall a busy host, it moves off the mailbox the same way BeginDirectApply's read did.
using var db = _dbFactory.CreateDbContext();
var latestSealed = db.Deployments
.AsNoTracking()
@@ -2901,6 +2976,10 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
try
{
// Synchronous by design, and a recorded follow-up rather than an oversight: this is a
// bounded per-apply write of ONE row (at most three per deployment), not the multi-MB
// artifact read that Task 26 moved off the mailbox. It is also the apply's crash-boundary
// marker, which is easiest to reason about while it is ordered on the actor thread.
using var db = _dbFactory.CreateDbContext();
var existing = db.NodeDeploymentStates.FirstOrDefault(
x => x.NodeId == _localNode.Value && x.DeploymentId == deploymentId.Value);
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverInstanceActor.cs
+4 -4
View File
@@ -101,10 +101,10 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
/// state — a no-op when already Reconnecting or Connecting.
/// </summary>
public sealed record ForceReconnect;
/// <summary>One driver-published value. NOT sent to the parent by the subscription path any more — the
/// actor batches every data change into an <see cref="AttributeValueBatchPublished"/>. Still live as the
/// per-value message the parent fans onward (the dependency mux consumes it, and
/// <c>DriverHostActor.FanCommsLossToDependencyMux</c> fabricates it).
/// <summary>One driver-published value. <b>No production sender remains</b> — the subscription path
/// batches every data change into an <see cref="AttributeValueBatchPublished"/>, and the comms-loss fan
/// (<c>DriverHostActor.FanCommsLossToDependencyMux</c>), which was the last sender to fabricate this
/// message, now fabricates a batch too. The dependency mux still accepts it for tests and direct senders.
/// <para><paramref name="StatusCode"/> is the driver's full OPC UA status code including its sub-code
/// (#519) — not a severity bucket. Classify it with <see cref="OpcUaStatus.IsGood"/> and friends rather
/// than comparing it to <see cref="OpcUaStatus.Good"/>.</para></summary>
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationOptions.cs
+43 -3
View File
@@ -43,6 +43,15 @@ public sealed class ContinuousHistorizationOptions
/// every <see cref="CommitIntervalMs"/> ms (higher throughput, a bounded worst-case loss window).
/// Parsed case-insensitively against <c>HistorizationCommitMode</c>; an unrecognized value falls
/// back to <c>PerEntry</c>.
/// <para>
/// <b><c>PerEntry</c> now means one fsync per ingest <em>burst</em>, not per value.</b> The
/// recorder appends a whole value burst through
/// <c>IHistorizationOutbox.AppendBatchAsync</c>, which enqueues every entry and then issues a
/// <em>single</em> group commit — so a burst of N values costs one fsync instead of N. The
/// durability contract is unchanged: <c>AppendBatchAsync</c> does not return until that
/// commit completes, so nothing the caller has been told is buffered can be lost to an
/// unclean restart.
/// </para>
/// </summary>
public string CommitMode { get; init; } = "PerEntry";
@@ -50,13 +59,42 @@ public sealed class ContinuousHistorizationOptions
/// <see cref="CommitMode"/> is <c>Periodic</c>. Ignored under <c>PerEntry</c>.</summary>
public int CommitIntervalMs { get; init; } = 100;
/// <summary>Maximum outbox entries peeked + written per drain pass; clamped to a positive value by
/// the recorder (a non-positive value falls back to 64).</summary>
public int DrainBatchSize { get; init; } = 64;
/// <summary>
/// Maximum outbox entries peeked + written per drain pass; clamped to a positive value by the
/// recorder (a non-positive value falls back to the default).
/// <para>
/// Defaults to 1024. It was 64 when a pass wrote its per-tag groups <em>sequentially</em>, so
/// a wide batch cost <c>tags × latency</c> on the drain thread. Both halves of that reasoning
/// are gone: the peek is an in-memory scan of the outbox's live index (widening it costs
/// allocation, not I/O), and the pass is now bounded-parallel over
/// <see cref="DrainConcurrency"/>, so its cost is <c>⌈tags / concurrency⌉ × latency</c>
/// regardless of batch width. At 64, a drain draws down at most 64 entries every
/// <see cref="DrainIntervalSeconds"/> — below the arrival rate of even a modest historized
/// tag set — so any gateway hiccup left the outbox permanently behind, growing toward
/// <see cref="Capacity"/> and shedding oldest samples, rather than catching up once the
/// gateway recovered.
/// </para>
/// </summary>
public int DrainBatchSize { get; init; } = 1024;
/// <summary>Steady drain cadence in seconds (also the post-success reschedule). Defaults to 2.</summary>
public double DrainIntervalSeconds { get; init; } = 2;
/// <summary>
/// Maximum historian tags a single drain pass writes concurrently. Defaults to 8; clamped to at
/// least 1 by the recorder (1 restores the fully sequential drain).
/// <para>
/// One peeked batch is grouped per historian tag and each group is an independent gateway
/// round-trip, so a sequential pass costs <c>tags × latency</c> — with a wide batch that is the
/// dominant term in how fast the outbox can drain. The window bounds how many of those
/// round-trips are outstanding at once, so a large tag count cannot flood the gateway. Nothing
/// about the ack changes: the pass is still acked all-or-nothing after every group has
/// reported, and the Gitea #538 per-tag quarantine still keys on "another group wrote this
/// pass".
/// </para>
/// </summary>
public int DrainConcurrency { get; init; } = 8;
/// <summary>
/// Maximum un-acked outbox entries before the drop-oldest capacity policy kicks in; <c>0</c>
/// means unbounded.
@@ -102,6 +140,8 @@ public sealed class ContinuousHistorizationOptions
warnings.Add("ContinuousHistorization:OutboxPath is empty while historization is enabled — the durable outbox has no directory to persist to; host startup will fail when the outbox is constructed. Set an absolute path on durable storage.");
if (string.Equals(CommitMode, "Periodic", StringComparison.OrdinalIgnoreCase) && CommitIntervalMs <= 0)
warnings.Add($"ContinuousHistorization:CommitIntervalMs is {CommitIntervalMs} — must be > 0 in Periodic commit mode; the periodic-commit loop cannot run.");
if (DrainConcurrency < 1)
warnings.Add($"ContinuousHistorization:DrainConcurrency is {DrainConcurrency} — must be >= 1; the recorder clamps it to 1 (a fully sequential drain).");
return warnings;
}
}
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationRecorder.cs
+291 -91
View File
@@ -8,16 +8,20 @@ namespace ZB.MOM.WW.OtOpcUa.Runtime.Historian;
/// <summary>
/// Continuous-historization engine for non-Galaxy (driver) tags. Registers interest with the
/// per-node <see cref="DependencyMuxActor"/> for the configured historized tag refs, then taps the
/// <see cref="VirtualTagActor.DependencyValueChanged"/> values the mux fans for those refs:
/// values the mux fans for those refs — in production a
/// <see cref="VirtualTagActor.DependencyValueBatch"/> per driver burst, with the per-value
/// <see cref="VirtualTagActor.DependencyValueChanged"/> receive retained for direct senders and tests:
/// value → coerce to numeric → append to the durable <see cref="IHistorizationOutbox"/> (the crash
/// boundary) → a self-paced drain writes the outbox through <see cref="IHistorianValueWriter"/>'s
/// live-value path and acks (truncates) on success.
/// </summary>
/// <remarks>
/// <para>
/// <b>Message path (grounded against the real actors).</b> The mux fans
/// <see cref="VirtualTagActor.DependencyValueChanged"/> — which carries only
/// <c>(TagId, Value, TimestampUtc)</c> and <em>no quality</em> — to every registered subscriber.
/// <b>Message path (grounded against the real actors).</b> The mux fans each value — carrying
/// only <c>(TagId, Value, TimestampUtc)</c> and <em>no quality</em> — to every registered
/// subscriber, batched as a <see cref="VirtualTagActor.DependencyValueBatch"/> on the production
/// path and as a bare <see cref="VirtualTagActor.DependencyValueChanged"/> from a direct sender
/// or a test.
/// The recorder reuses that existing fan-out (no change to <c>DriverHostActor</c>), so values
/// arrive Good-quality by the same convention the scripted-alarm host uses (the mux only
/// forwards driver-published values); historized samples are recorded with
@@ -28,8 +32,10 @@ namespace ZB.MOM.WW.OtOpcUa.Runtime.Historian;
/// non-coercible value (string/null/reference) is dropped and metered rather than appended.
/// </para>
/// <para>
/// <b>Non-blocking drain.</b> Appends run on the actor thread (an awaited
/// <see cref="IHistorizationOutbox.AppendAsync"/> so appends stay serialized — the durable
/// <b>Batched ingest, non-blocking drain.</b> A mux burst arrives as one
/// <see cref="VirtualTagActor.DependencyValueBatch"/> and is committed with ONE
/// <see cref="IHistorizationOutbox.AppendBatchAsync"/> group commit rather than N per-entry
/// fsyncs. Appends run on the actor thread (awaited, so appends stay serialized — the durable
/// boundary completes before the next message), but the drain runs off the actor thread and
/// pipes its outcome back to <c>Self</c> (<see cref="DrainResult"/>) so the mailbox is never
/// blocked on the gateway write. A drain never throws into the actor: the drain task catches
@@ -90,16 +96,26 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
/// <summary>A point-in-time snapshot of the recorder's counters.</summary>
/// <param name="QueuedDepth">Un-acked entries currently held in the durable outbox.</param>
/// <param name="TotalRecorded">Lifetime count of values appended to the outbox.</param>
/// <param name="TotalRecorded">Lifetime count of values durably appended to the outbox. Credited per
/// BURST, on a successful group commit — so it UNDER-counts when a batch append fails after the outbox
/// durably committed a partial prefix: the outbox does not report how far it got, so the recorder cannot
/// credit that prefix. Those entries still drain; they are simply never counted here.</param>
/// <param name="DroppedNonNumeric">Lifetime count of values dropped for not being numeric-coercible.</param>
/// <param name="OutboxDropped">Lifetime count of entries the outbox dropped on capacity overflow.</param>
/// <param name="OutboxAppendFailures">Lifetime count of durable-boundary append failures (the value
/// was dropped, not recorded, and the actor stayed alive rather than restart-looping).</param>
/// <param name="OutboxAppendFailures">Lifetime count of FAILED BATCH APPENDS at the durable boundary —
/// ONE per failed burst whatever its size, never one per value (the granularity changed with the batched
/// ingest; the per-value era counted one per value). The actor stays alive rather than restart-looping.
/// It is NOT a count of values lost: the outbox contract is "prefix, never orphan", so a failed burst
/// may still have left entries durably queued that drain normally — see <see cref="IngestAsync"/>.</param>
/// <param name="LastDrainSucceeded">Whether the most recent drain pass acked cleanly.</param>
/// <param name="SuppressedNotPrimary">Lifetime count of values not captured because this node is
/// the Secondary of a pair whose Primary records the same refs (Gitea #537).</param>
/// <param name="AbandonedEntries">Lifetime count of entries discarded because their historian tag
/// kept failing while other tags were writing successfully (Gitea #538).</param>
/// <param name="IngestFaults">Lifetime count of values dropped because the per-value gate/coercion step
/// threw — distinct from <paramref name="DroppedNonNumeric"/>, which counts values the historian's
/// numeric-only write path cannot carry. Non-zero here means an ingest dependency is faulting (most
/// plausibly the supplied capture gate), not that the data was unsuitable.</param>
public sealed record RecorderStatus(
int QueuedDepth,
long TotalRecorded,
@@ -108,7 +124,8 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
long OutboxAppendFailures,
bool LastDrainSucceeded,
long SuppressedNotPrimary = 0,
long AbandonedEntries = 0);
long AbandonedEntries = 0,
long IngestFaults = 0);
private readonly IActorRef _dependencyMux;
private readonly IHistorianValueWriter _writer;
@@ -135,6 +152,11 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
/// writing successfully — before its queued entries are abandoned so the FIFO head can advance.</summary>
private readonly int _maxWriteAttempts;
/// <summary>Max per-tag groups one drain pass writes concurrently. The groups in a pass are
/// independent gateway round-trips, so a sequential pass costs <c>tags × latency</c>; this bounds how
/// many are outstanding at once so a wide batch cannot flood the gateway.</summary>
private readonly int _drainConcurrency;
/// <summary>Consecutive failed write passes per historian tag, counted only across passes that
/// proved the backend reachable. Reset on the tag's first success. Mutated exclusively inside
/// <see cref="DrainOnceAsync"/>, which <see cref="_draining"/> keeps single-flight.</summary>
@@ -151,8 +173,14 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
private long _outboxAppendFailures;
private long _suppressedNotPrimary;
private long _abandonedEntries;
private long _ingestFaults;
private bool _lastDrainSucceeded = true;
/// <summary>Whether the current run of per-item ingest faults has already been reported at Warning, so
/// a per-value fault mode (a throwing capture gate) logs once rather than once per sample. Cleared by
/// the first item that ingests cleanly — see <see cref="IngestAsync"/>.</summary>
private bool _ingestFaultLogged;
/// <summary>Last observed state of <see cref="_captureGate"/>, so a change is logged once rather
/// than on every value. Seeded open: a gate that starts closed logs its first suppression.</summary>
private bool _captureGateOpen = true;
@@ -174,21 +202,24 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
/// and for tests.</param>
/// <param name="maxWriteAttempts">Gitea #538: consecutive failed write passes one tag may
/// accumulate, while other tags are succeeding, before its entries are abandoned.</param>
/// <param name="drainConcurrency">Max historian tags one drain pass writes concurrently (must be
/// positive; 1 is the fully sequential drain).</param>
/// <returns>Props for the actor.</returns>
public static Props Props(
IActorRef dependencyMux,
IHistorianValueWriter writer,
IHistorizationOutbox outbox,
IReadOnlyList<string> historizedRefs,
int drainBatchSize = 64,
int drainBatchSize = 1024,
TimeSpan? drainInterval = null,
TimeSpan? minBackoff = null,
TimeSpan? maxBackoff = null,
Func<bool>? captureGate = null,
int maxWriteAttempts = 5) =>
int maxWriteAttempts = 5,
int drainConcurrency = 8) =>
Akka.Actor.Props.Create(() => new ContinuousHistorizationRecorder(
dependencyMux, writer, outbox, historizedRefs, drainBatchSize, drainInterval, minBackoff, maxBackoff,
captureGate, maxWriteAttempts));
captureGate, maxWriteAttempts, drainConcurrency));
/// <summary>Initializes a new instance of the <see cref="ContinuousHistorizationRecorder"/> class.</summary>
/// <param name="dependencyMux">The per-node dependency mux to register historized-ref interest with.</param>
@@ -204,17 +235,20 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
/// <param name="maxWriteAttempts">Gitea #538: consecutive failed write passes one tag may
/// accumulate, while other tags are succeeding, before its entries are abandoned. Clamped to at
/// least 1.</param>
/// <param name="drainConcurrency">Max historian tags one drain pass writes concurrently. Clamped to
/// at least 1; 1 is the fully sequential drain.</param>
public ContinuousHistorizationRecorder(
IActorRef dependencyMux,
IHistorianValueWriter writer,
IHistorizationOutbox outbox,
IReadOnlyList<string> historizedRefs,
int drainBatchSize = 64,
int drainBatchSize = 1024,
TimeSpan? drainInterval = null,
TimeSpan? minBackoff = null,
TimeSpan? maxBackoff = null,
Func<bool>? captureGate = null,
int maxWriteAttempts = 5)
int maxWriteAttempts = 5,
int drainConcurrency = 8)
{
_dependencyMux = dependencyMux ?? throw new ArgumentNullException(nameof(dependencyMux));
_writer = writer ?? throw new ArgumentNullException(nameof(writer));
@@ -228,7 +262,10 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
AddRef(r, r);
}
_drainBatchSize = drainBatchSize > 0 ? drainBatchSize : 64;
// Both this fallback and the ctor/Props defaults above must track ContinuousHistorizationOptions
// .DrainBatchSize (1024): production always passes the option through, so a divergent literal here
// only ever shows up as a quietly different batch size in tests and in degenerate-input callers.
_drainBatchSize = drainBatchSize > 0 ? drainBatchSize : 1024;
_drainInterval = drainInterval is { } di && di > TimeSpan.Zero ? di : TimeSpan.FromSeconds(2);
_minBackoff = minBackoff is { } mb && mb > TimeSpan.Zero ? mb : TimeSpan.FromSeconds(1);
_maxBackoff = maxBackoff is { } xb && xb > TimeSpan.Zero ? xb : TimeSpan.FromSeconds(30);
@@ -238,8 +275,10 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
// gate), and a test that does not care about the role.
_captureGate = captureGate ?? (static () => true);
_maxWriteAttempts = maxWriteAttempts > 0 ? maxWriteAttempts : 1;
_drainConcurrency = drainConcurrency > 0 ? drainConcurrency : 1;
ReceiveAsync<VirtualTagActor.DependencyValueChanged>(OnValueChangedAsync);
ReceiveAsync<VirtualTagActor.DependencyValueBatch>(OnValueBatchAsync);
Receive<UpdateHistorizedRefs>(OnUpdateHistorizedRefs);
Receive<DrainTick>(_ => OnDrainTick());
Receive<DrainResult>(OnDrainResult);
@@ -259,7 +298,7 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
// Register interest by mux ref (the key the mux fans DependencyValueChanged by) so it fans those
// tags' values to us. Historian-name overrides are tracked separately for the write side.
_dependencyMux.Tell(new DependencyMuxActor.RegisterInterest(_refMap.Keys.ToList(), Self));
// Seed the steady drain cadence; appends also nudge a prompt drain (see OnValueChangedAsync).
// Seed the steady drain cadence; appends also nudge a prompt drain (see IngestAsync).
Timers.StartSingleTimer(DrainTimerKey, DrainTick.Instance, _drainInterval);
base.PreStart();
}
@@ -275,7 +314,177 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
base.PostStop();
}
private async Task OnValueChangedAsync(VirtualTagActor.DependencyValueChanged msg)
/// <summary>
/// The mux's batched delivery: one message carrying this recorder's whole matched slice of a driver
/// burst. The whole burst is ingested as ONE durable append (see <see cref="IngestAsync"/>), so a
/// 50-value scan costs one group commit instead of 50 per-entry fsyncs.
/// </summary>
/// <param name="msg">The batch of matched dependency values.</param>
/// <returns>A task that completes when the burst has been appended (or dropped).</returns>
private Task OnValueBatchAsync(VirtualTagActor.DependencyValueBatch msg) => IngestAsync(msg.Items);
/// <summary>
/// A single mux-fanned value. Delegates to the same core as the batch path as a one-item burst, so
/// a direct sender (and every test that Tells one value) behaves identically to a batch of one.
/// </summary>
/// <param name="msg">The matched dependency value.</param>
/// <returns>A task that completes when the value has been appended (or dropped).</returns>
private Task OnValueChangedAsync(VirtualTagActor.DependencyValueChanged msg) => IngestAsync([msg]);
/// <summary>
/// Ingest one burst of mux-fanned values: run the per-value gates over every item, then append the
/// survivors with a SINGLE <see cref="IHistorizationOutbox.AppendBatchAsync"/> group commit.
/// </summary>
/// <remarks>
/// <para>
/// <b>The gates stay per value.</b> The ref filter, the Gitea #537 capture gate and the numeric
/// coercion are evaluated for each item in turn — the capture gate in particular is a live read
/// per value, never hoisted to once per batch, so a promotion resumes capture on the very next
/// sample (pinned by <c>The_capture_gate_is_re_read_per_value_so_promotion_resumes_capture</c>).
/// Only the durable append is batched.
/// </para>
/// <para>
/// <b>One bad item cannot drop the tail.</b> The per-item gate/coercion step is wrapped: an
/// unexpected throw (most plausibly a faulting <c>captureGate</c> delegate) is metered as an
/// <c>IngestFaults</c>, logged by category, and the loop continues with the next item. Left
/// unguarded it would escape the handler into supervision, discarding every item after it in the
/// burst AND restarting the actor — which in production discards the historized-ref set (see the
/// #491 note below).
/// </para>
/// <para>
/// <b>Append failure is batch-wide, and it is NOT a whole-burst loss.</b> A failed group commit
/// is metered ONCE (<c>OutboxAppendFailures</c> — one per failed burst whatever its size, never
/// one per value) and logged by category, keeping the same never-restart-loop policy the
/// per-value path had. What is actually lost is only what the outbox never committed: its
/// contract is "prefix, never orphan", so anything it durably appended before the failure stays
/// fully tracked and ackable and the trailing drain nudge flushes it normally — nothing is left
/// buffered-but-unackable. That prefix does NOT credit <c>TotalRecorded</c>, because the outbox
/// does not report how far it got; the counter under-counts rather than over-claims.
/// </para>
/// </remarks>
/// <param name="items">The burst's matched dependency values, in fan-out order.</param>
/// <returns>A task that completes when the burst has been appended (or dropped).</returns>
private async Task IngestAsync(IReadOnlyList<VirtualTagActor.DependencyValueChanged> items)
{
// Capture Self BEFORE the await below. Self reads ActorBase.Context, and the append is awaited with
// ConfigureAwait(false) — which drops Akka's ActorTaskScheduler synchronization context, so a
// continuation that actually resumes off the actor thread finds NO ActorContext and throws
// NotSupportedException. This is the same idiom the GetStatus handler applies to Sender.
//
// It only bites when the append genuinely YIELDS: the production FasterLog outbox fsyncs per group
// commit (PerEntry commit mode), so it does; a fake outbox returning ValueTask.CompletedTask
// completes synchronously, the continuation runs inline with the context still installed, and the
// bug is invisible. That is exactly how it reached production — see the yielding fake in
// ContinuousHistorizationRecorderTests. Live-caught on the docker-dev rig (Gitea #491), where the
// throw escaped the handler into supervision: the actor restarted, and because production seeds an
// EMPTY ref set (the deployed set arrives later via UpdateHistorizedRefs), the restart silently
// dropped every historized ref — so historization stopped dead after the FIRST value.
IActorRef self = Self;
List<HistorizationOutboxEntry>? entries = null;
for (int i = 0; i < items.Count; i++)
{
try
{
CollectEntries(items[i], ref entries);
}
catch (Exception ex)
{
// Defensive: keep one malformed item from dropping the rest of the burst (and from
// restarting the actor). Category only — no value content is logged.
//
// Metered under its own counter rather than folded into DroppedNonNumeric: the realistic
// source is a throwing _captureGate (a supplied delegate reading the redundancy role view),
// not a value the historian cannot carry, and conflating the two would make the
// non-numeric metric lie about a redundancy fault.
//
// Logged once per run of faults (the same latch idiom the capture gate uses above): the
// plausible fault modes are per-VALUE, so an unlatched Warning would emit one line per
// sample — at scan rates that is the log flood, not the diagnosis.
_ingestFaults++;
if (!_ingestFaultLogged)
{
_ingestFaultLogged = true;
_log.Warning(
"ContinuousHistorization: value ingest failed ({Exception}); value dropped. Further "
+ "ingest faults are logged at Debug until one succeeds.", ex.GetType().Name);
}
else
{
_log.Debug("ContinuousHistorization: value ingest failed ({Exception}); value dropped.",
ex.GetType().Name);
}
continue;
}
// A clean item re-arms the fault log, so a NEW run of faults is reported at Warning again.
_ingestFaultLogged = false;
}
if (entries is null)
{
// Everything was filtered, suppressed or dropped — nothing to commit, and no drain to nudge
// (this burst added nothing; the steady timer still covers any earlier residue).
return;
}
// Durable boundary: ONE group commit for the whole burst, awaited so appends stay serialized,
// BEFORE the values are considered captured. The outbox drops the oldest entry on capacity
// overflow and tracks DroppedCount.
try
{
await _outbox.AppendBatchAsync(entries, _lifetime.Token).ConfigureAwait(false);
_totalRecorded += entries.Count;
}
catch (OperationCanceledException) when (_lifetime.IsCancellationRequested)
{
// Normal shutdown raced the append — not a fault. Drop quietly, and do not nudge a drain on
// an actor that is stopping.
return;
}
catch (Exception ex)
{
// A durable-boundary failure (e.g. an fsync hitting disk-full / I/O error) must NEVER propagate
// out of the handler — that would trip Akka supervision into a restart, and under a persistent
// disk fault the actor would restart-loop (re-register → next burst → append fails → restart →
// …). Mirror the drain path's catch-all: meter the failure (category only, no value content)
// and carry on.
//
// The meter is ONE per failed burst, whatever the burst's size — a deliberate granularity change
// from the per-value era, where it was one per value. And it is NOT a count of values lost: the
// outbox commits a PREFIX or nothing (never an orphan), so an unknown number of this burst's
// entries may already be durable. Hence the message says "failed to append", not "dropped" —
// claiming total loss here would be wrong about the prefix the drain nudge below is about to
// flush. TotalRecorded is likewise not credited for that prefix: the outbox does not report how
// far it got, so the counter under-counts rather than over-claims.
_outboxAppendFailures++;
_log.Warning(
"ContinuousHistorization: outbox batch append failed ({Exception}); {Count} value(s) in the "
+ "burst failed to append. Any prefix the outbox committed before the failure stays queued "
+ "and drains normally; the remainder are lost.",
ex.GetType().Name, entries.Count);
}
// Nudge a prompt drain attempt; the DrainTick handler de-dups (already draining) and honours
// any active backoff window, so this never bypasses a failure cooldown. Sent even after a failed
// append, because the outbox contract leaves any durably-appended prefix tracked and ackable —
// it should drain now rather than wait for the steady timer. Uses the pre-await capture, never
// `Self` — see the note above.
self.Tell(DrainTick.Instance);
}
/// <summary>
/// Runs the per-value gates for one mux-fanned value and, if it survives them, adds one outbox
/// entry per historian name to <paramref name="entries"/> (lazily created on first survivor).
/// Purely synchronous — it runs on the actor thread with the mailbox suspended, so it may read and
/// mutate actor state freely.
/// </summary>
/// <param name="msg">The matched dependency value.</param>
/// <param name="entries">The burst's accumulating entry list; null until the first survivor.</param>
private void CollectEntries(
VirtualTagActor.DependencyValueChanged msg,
ref List<HistorizationOutboxEntry>? entries)
{
// The mux fans values keyed by the driver ref (msg.TagId == the mux ref). Only historize refs we
// registered interest for, and resolve the HISTORIAN NAME(s) to write under — which differ from the
@@ -291,9 +500,10 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
// fan-out, so an ungated recorder writes every sample TWICE (measured on the docker-dev central
// pair: the gateway's write counter advanced twice per value, and halved when one node stopped).
// The gate closes only on a node whose pair partner is the Primary and is recording the same
// refs; it is read PER VALUE rather than latched, so promotion resumes capture on the very next
// sample with no restart. Note what is NOT gated: the drain. Entries already captured are this
// node's to deliver, and a demoted node must still empty its own outbox.
// refs; it is read PER VALUE rather than latched (and never hoisted out of this per-item step),
// so promotion resumes capture on the very next sample with no restart. Note what is NOT gated:
// the drain. Entries already captured are this node's to deliver, and a demoted node must still
// empty its own outbox.
if (!_captureGate())
{
_suppressedNotPrimary++;
@@ -325,67 +535,23 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
var timestampUtc = DateTime.SpecifyKind(msg.TimestampUtc, DateTimeKind.Utc);
// Capture Self BEFORE the first await. Self reads ActorBase.Context, and the awaits below use
// ConfigureAwait(false) — which drops Akka's ActorTaskScheduler synchronization context, so any
// continuation that actually resumes off the actor thread finds NO ActorContext and throws
// NotSupportedException. This is the same idiom the GetStatus handler already applies to Sender.
//
// It only bites when the append genuinely YIELDS: the production FasterLog outbox fsyncs per entry
// (PerEntry commit mode), so it does; a fake outbox returning ValueTask.CompletedTask completes
// synchronously, the continuation runs inline with the context still installed, and the bug is
// invisible. That is exactly how it reached production — see the yielding fake in
// ContinuousHistorizationRecorderTests. Live-caught on the docker-dev rig (Gitea #491), where the
// throw escaped the handler into supervision: the actor restarted, and because production seeds an
// EMPTY ref set (the deployed set arrives later via UpdateHistorizedRefs), the restart silently
// dropped every historized ref — so historization stopped dead after the FIRST value.
IActorRef self = Self;
entries ??= new List<HistorizationOutboxEntry>();
// One outbox entry PER historian name: a single mux fan-out must reach every aliased historized tag
// (each its own historian name), not just one — collapsing to one name here would silently drop the
// rest. Snapshot the names first: although the mailbox is suspended across the awaits below (so the
// set cannot be mutated concurrently), iterating a snapshot is robust and clearer.
foreach (string historianName in historianNames.ToList())
// rest. No snapshot copy is needed: this method is synchronous, so the set cannot be mutated while
// it is iterated.
foreach (string historianName in historianNames)
{
// Record under the historian name (override-or-FullName), NOT the mux ref — the outbox entry +
// the drain's WriteLiveValues target the historian tag the EnsureTags hook provisioned.
var entry = new HistorizationOutboxEntry(
entries.Add(new HistorizationOutboxEntry(
Guid.NewGuid(),
historianName,
numeric,
GoodQuality,
timestampUtc);
// Durable boundary: append (awaited so appends stay serialized) BEFORE the value is considered
// captured. The outbox drops the oldest entry on capacity overflow and tracks DroppedCount.
try
{
await _outbox.AppendAsync(entry, _lifetime.Token).ConfigureAwait(false);
}
catch (OperationCanceledException) when (_lifetime.IsCancellationRequested)
{
// Normal shutdown raced the append — not a fault. Drop quietly (abandon the rest too).
return;
}
catch (Exception ex)
{
// A durable-boundary failure (e.g. a PerEntry fsync hitting disk-full / I/O error) must
// NEVER propagate out of the handler — that would trip Akka supervision into a restart, and
// under a persistent disk fault the actor would restart-loop (re-register → next value →
// append fails → restart → …). Mirror the drain path's catch-all: meter the failure
// (category only, no value content), drop THIS name's value, and continue with the rest.
_outboxAppendFailures++;
_log.Warning("ContinuousHistorization: outbox append failed ({Exception}); value dropped.",
ex.GetType().Name);
continue;
}
_totalRecorded++;
timestampUtc));
}
// Nudge a prompt drain attempt; the DrainTick handler de-dups (already draining) and honours
// any active backoff window, so this never bypasses a failure cooldown. Uses the pre-await
// capture, never `Self` — see the note above.
self.Tell(DrainTick.Instance);
}
/// <summary>
@@ -512,35 +678,68 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
// id, so partial per-tag acking could drop un-written older entries; the ack stays
// all-or-nothing — only when nothing is left blocking do we truncate the whole peeked
// prefix (one commit).
var succeeded = 0;
var failedGroups = new List<(string Tag, int Count)>();
foreach (IGrouping<string, HistorizationOutboxEntry> group in
batch.GroupBy(e => e.Tag, StringComparer.Ordinal))
{
var values = group
.Select(e => new HistorizationValue(e.TimestampUtc, e.NumericValue, e.Quality))
.ToList();
//
// The groups are INDEPENDENT gateway round-trips against distinct historian tags, so they run
// concurrently under a bounded window rather than one after another: a sequential pass costs
// `tags × latency`, which is the dominant term in how fast a wide batch can drain. The window
// (_drainConcurrency) is what stops a large tag count from flooding the gateway; a window of 1
// is exactly the old sequential loop.
var groups = batch
.GroupBy(e => e.Tag, StringComparer.Ordinal)
.Select(g => (
Key: g.Key,
Values: (IReadOnlyList<HistorizationValue>)g
.Select(e => new HistorizationValue(e.TimestampUtc, e.NumericValue, e.Quality))
.ToList()))
.ToList();
bool ok;
using var window = new SemaphoreSlim(_drainConcurrency);
var tasks = groups.Select(async g =>
{
await window.WaitAsync(ct).ConfigureAwait(false);
try
{
ok = await _writer.WriteLiveValuesAsync(group.Key, values, ct).ConfigureAwait(false);
}
catch
{
// The writer is non-throwing by contract; guard defensively so one tag's fault
// cannot fail the whole drain task.
ok = false;
}
bool ok;
try
{
ok = await _writer.WriteLiveValuesAsync(g.Key, g.Values, ct).ConfigureAwait(false);
}
catch
{
// The writer is non-throwing by contract; guard defensively so one tag's fault
// cannot fail the whole drain task (and, now, cannot fault the WhenAll either).
ok = false;
}
return (Tag: g.Key, Count: g.Values.Count, Ok: ok);
}
finally
{
window.Release();
}
}).ToList();
// Task.WhenAll waits for EVERY task to complete before it faults, so no lambda can still be
// touching `window` when the using-scope disposes it — including on the cancellation path,
// where the not-yet-entered tasks fault out of WaitAsync without reaching the finally.
(string Tag, int Count, bool Ok)[] outcomes = await Task.WhenAll(tasks).ConfigureAwait(false);
// Fold the outcomes on the single logical drain flow — NEVER inside the parallel lambdas.
// _tagWriteFailures is actor state (kept single-flight by _draining); mutating it from N
// concurrent tasks would corrupt the Gitea #538 counters. Nothing below observes completion
// ORDER, so the fold is deterministic whatever order the writes finished in.
var succeeded = 0;
var failedGroups = new List<(string Tag, int Count)>();
foreach ((string tag, int count, bool ok) in outcomes)
{
if (ok)
{
succeeded++;
_tagWriteFailures.Remove(group.Key);
_tagWriteFailures.Remove(tag);
}
else
{
failedGroups.Add((group.Key, values.Count));
failedGroups.Add((tag, count));
}
}
@@ -661,7 +860,8 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
_outboxAppendFailures,
_lastDrainSucceeded,
_suppressedNotPrimary,
_abandonedEntries);
_abandonedEntries,
_ingestFaults);
}
private static TimeSpan Min(TimeSpan a, TimeSpan b) => a < b ? a : b;
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/ScriptedAlarms/ScriptedAlarmHostActor.cs
+38 -6
View File
@@ -29,9 +29,11 @@ namespace ZB.MOM.WW.OtOpcUa.Runtime.ScriptedAlarms;
/// </para>
///
/// <para>
/// <b>Data flow.</b> <see cref="DependencyMuxActor"/> delivers a
/// <see cref="VirtualTagActor.DependencyValueChanged"/> for every tag the loaded alarms
/// depend on; the host pushes each into the upstream. The engine self-evaluates: its own
/// <b>Data flow.</b> <see cref="DependencyMuxActor"/> delivers the values of every tag the
/// loaded alarms depend on — in production as a batched
/// <see cref="VirtualTagActor.DependencyValueBatch"/> per driver burst, with the per-value
/// <see cref="VirtualTagActor.DependencyValueChanged"/> receive retained for direct senders and
/// tests; the host pushes each into the upstream. The engine self-evaluates: its own
/// <see cref="ITagUpstreamSource.SubscribeTag"/> observer re-runs the predicates and raises
/// <see cref="ScriptedAlarmEngine.OnEvent"/>. The host does NOT call evaluate — it only
/// feeds values in and fans emissions out.
@@ -134,9 +136,10 @@ public sealed class ScriptedAlarmHostActor : ReceiveActor
/// <summary>Factory method to create Props for a <see cref="ScriptedAlarmHostActor"/>.</summary>
/// <param name="publishActor">The OPC UA publish actor that consumes
/// <see cref="OpcUaPublishActor.AlarmStateUpdate"/> bridged from engine emissions.</param>
/// <param name="mux">Optional dependency multiplexer the host registers interest with so it
/// receives a <see cref="VirtualTagActor.DependencyValueChanged"/> per dependency tag. Null on the
/// dev/Mac path (no live values).</param>
/// <param name="mux">Optional dependency multiplexer the host registers interest with so it receives
/// its dependency tags' values — a <see cref="VirtualTagActor.DependencyValueBatch"/> per burst in
/// production, or a per-value <see cref="VirtualTagActor.DependencyValueChanged"/> from a direct
/// sender or a test. Null on the dev/Mac path (no live values).</param>
/// <param name="upstream">The mux-fed upstream the engine reads + subscribes from. MUST be the
/// same instance the <paramref name="engine"/> was constructed around.</param>
/// <param name="engine">The scripted-alarm engine this host owns + disposes.</param>
@@ -201,6 +204,10 @@ public sealed class ScriptedAlarmHostActor : ReceiveActor
Receive<ApplyScriptedAlarms>(OnApply);
Receive<AlarmsLoaded>(OnAlarmsLoaded);
Receive<VirtualTagActor.DependencyValueChanged>(OnDependencyChanged);
// The mux's batched delivery: one message carrying this host's whole matched slice of a driver
// burst. The engine upstream is fed per value, so push each item in order — the saving is the
// mailbox hop, not the push.
Receive<VirtualTagActor.DependencyValueBatch>(OnDependencyBatch);
Receive<EngineEmission>(OnEngineEmission);
// Cluster redundancy snapshots (published on the `redundancy-state` topic, subscribed in PreStart)
// cache this node's role so OnEngineEmission can gate the cluster-wide alerts publish to the Primary.
@@ -395,6 +402,31 @@ public sealed class ScriptedAlarmHostActor : ReceiveActor
new DataValueSnapshot(msg.Value, msg.StatusCode, msg.TimestampUtc, msg.TimestampUtc));
}
/// <summary>One batched mux delivery — every item already matched this host's registered interest.
/// Loops the per-value handler, preserving in-burst order.</summary>
/// <remarks>
/// <b>Mid-batch exception policy: unguarded, deliberately.</b> An exception escaping
/// <see cref="OnDependencyChanged"/> drops the remaining items of this burst and restarts the actor.
/// That IS wider than the per-message path — Akka restarts do not flush the mailbox, only the
/// failing message is dropped, so as N separate messages the same fault would cost one value rather
/// than the burst's tail. Accepted: the engine tolerates a missed intermediate value (the next
/// delivery for that tag re-converges its state). Catching per item is deliberately NOT done: the
/// handler is a single
/// <c>IUpstreamValueSource.Push</c> into an in-memory cache, so a throw means the upstream itself is
/// broken and swallowing it would feed the engine a silently truncated burst. Contrast
/// <c>ContinuousHistorizationRecorder.IngestAsync</c>, which DOES catch per item because its
/// per-item step runs value coercion.
/// </remarks>
/// <param name="msg">The batch of matched dependency values.</param>
private void OnDependencyBatch(VirtualTagActor.DependencyValueBatch msg)
{
var items = msg.Items;
for (var i = 0; i < items.Count; i++)
{
OnDependencyChanged(items[i]);
}
}
private void OnEngineEmission(EngineEmission msg)
{
var e = msg.Event;
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/ServiceCollectionExtensions.cs
+5 -2
View File
@@ -402,7 +402,7 @@ public static class ServiceCollectionExtensions
}
// Dependency mux must be spawned before DriverHostActor so the host can forward
// AttributeValuePublished into it from the very first driver spawn.
// AttributeValueBatchPublished into it from the very first driver spawn.
var mux = system.ActorOf(DependencyMuxActor.Props(), DependencyMuxActorName);
registry.Register<DependencyMuxActorKey>(mux);
@@ -477,7 +477,10 @@ public static class ServiceCollectionExtensions
captureGate: resolver.GetService<IHistorizationRoleView>() is { } histRoleView
? () => histRoleView.ShouldRecordHistorizedValues
: null,
maxWriteAttempts: continuousOptions.MaxWriteAttempts),
maxWriteAttempts: continuousOptions.MaxWriteAttempts,
// Bounded-parallel per-tag drain: the groups in one pass are independent
// gateway round-trips, so a sequential pass costs `tags × latency`.
drainConcurrency: continuousOptions.DrainConcurrency),
ContinuousHistorizationRecorderActorName);
registry.Register<ContinuousHistorizationRecorderKey>(continuousRecorder);
}
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/VirtualTags/DependencyConsumerMuxAdapter.cs
+30 -3
View File
@@ -8,9 +8,11 @@ namespace ZB.MOM.WW.OtOpcUa.Runtime.VirtualTags;
/// Bridges a driver that implements <see cref="IDependencyConsumer"/> onto the per-node
/// <see cref="DependencyMuxActor"/>. Spawned as a child of <c>DriverHostActor</c> for each such
/// driver: on start (and on every apply via <see cref="ReRegister"/>) it registers the driver's
/// declared <see cref="IDependencyConsumer.DependencyRefs"/> with the mux, and forwards each
/// <see cref="VirtualTagActor.DependencyValueChanged"/> the mux fans out into
/// <see cref="IDependencyConsumer.OnDependencyValue"/>.
/// declared <see cref="IDependencyConsumer.DependencyRefs"/> with the mux, and forwards each value
/// the mux fans out into <see cref="IDependencyConsumer.OnDependencyValue"/>. The production
/// delivery is the batched <see cref="VirtualTagActor.DependencyValueBatch"/> — one message per
/// driver burst; the per-value <see cref="VirtualTagActor.DependencyValueChanged"/> receive remains
/// for direct senders and tests, and the batch handler simply loops it.
///
/// <para>
/// This is the exact seam a <c>VirtualTagActor</c> uses to receive its inputs, generalised so a
@@ -45,6 +47,7 @@ public sealed class DependencyConsumerMuxAdapter : ReceiveActor
_mux = mux ?? throw new ArgumentNullException(nameof(mux));
Receive<VirtualTagActor.DependencyValueChanged>(OnDependencyValueChanged);
Receive<VirtualTagActor.DependencyValueBatch>(OnDependencyValueBatch);
Receive<ReRegister>(_ => RegisterInterest());
}
@@ -68,4 +71,28 @@ public sealed class DependencyConsumerMuxAdapter : ReceiveActor
// Quality is stripped at the mux boundary (value + timestamp only), so forward Good (0).
_consumer.OnDependencyValue(msg.TagId, msg.Value, statusCode: 0u, msg.TimestampUtc);
}
/// <summary>The mux's batched delivery: one message carrying this adapter's whole matched slice of a
/// driver burst. The consumer contract is per value, so dispatch each item in order — the saving is the
/// mailbox hop, not the consumer call.</summary>
/// <remarks>
/// <b>Mid-batch exception policy: unguarded, matching the single-value path.</b> An exception
/// escaping <see cref="IDependencyConsumer.OnDependencyValue"/> drops the remaining items of this
/// burst and restarts the adapter. This DOES widen the per-fault loss: Akka restarts do not flush
/// the mailbox (only the failing message is dropped), so as N separate messages the same fault
/// would have cost one value, while batched it costs the rest of the burst's tail. Accepted,
/// because an <see cref="IDependencyConsumer"/> tolerates missed intermediate values — the next
/// delivery re-converges its state — and catching per item HERE only would make the batch path
/// quietly more forgiving than the single-value one for the same consumer fault. A consumer that
/// needs per-value fault tolerance owns it inside its own <c>OnDependencyValue</c>.
/// </remarks>
/// <param name="msg">The batch of matched dependency values.</param>
private void OnDependencyValueBatch(VirtualTagActor.DependencyValueBatch msg)
{
var items = msg.Items;
for (var i = 0; i < items.Count; i++)
{
OnDependencyValueChanged(items[i]);
}
}
}
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/VirtualTags/DependencyMuxActor.cs
+46 -14
View File
@@ -5,21 +5,25 @@ using ZB.MOM.WW.OtOpcUa.Runtime.Drivers;
namespace ZB.MOM.WW.OtOpcUa.Runtime.VirtualTags;
/// <summary>
/// Per-node fan-out router from <see cref="DriverInstanceActor.AttributeValuePublished"/>
/// Per-node fan-out router from <see cref="DriverInstanceActor.AttributeValueBatchPublished"/>
/// to interested <see cref="VirtualTagActor"/> instances. VirtualTagActor sends
/// <see cref="RegisterInterest"/> on start-up listing the tag refs it depends on; the mux
/// keeps a map of <c>tagRef → Set&lt;IActorRef&gt;</c> and on every AttributeValuePublished
/// forwards a <see cref="VirtualTagActor.DependencyValueChanged"/> to each interested
/// subscriber.
/// keeps a map of <c>tagRef → Set&lt;IActorRef&gt;</c> and on every ingress forwards each
/// interested subscriber ONE <see cref="VirtualTagActor.DependencyValueBatch"/> carrying that
/// subscriber's matched slice of the burst.
///
/// DriverHostActor forwards driver-child value publishes to this mux (one mux per driver-role
/// node). The ONLY driver ingress is the batched
/// <see cref="DriverInstanceActor.AttributeValueBatchPublished"/> — one envelope per driver
/// production burst, which the mux filters per item. The per-value
/// <see cref="DriverInstanceActor.AttributeValuePublished"/> receive stays for the comms-loss fan
/// ALONE: <c>DriverHostActor.FanCommsLossToDependencyMux</c> fabricates that message and Tells this
/// mux directly, never through the host's own receive. Both
/// produce the same <see cref="VirtualTagActor.DependencyValueChanged"/> per interested subscriber.
/// <see cref="DriverInstanceActor.AttributeValuePublished"/> receive has <b>no production sender left
/// at all</b>: the comms-loss fan (<c>DriverHostActor.FanCommsLossToDependencyMux</c>) was its last
/// one and now fabricates an <c>AttributeValueBatchPublished</c> instead. The receive is kept for
/// tests and direct senders — treat it as dead in production.
///
/// <b>The mux emits ONLY batches</b>, so a subscriber's
/// batch receive is its complete hot path. Every subscriber keeps a per-value
/// <see cref="VirtualTagActor.DependencyValueChanged"/> receive as well, for direct senders and tests.
/// The mux is deliberately not a DPS subscriber — virtual-tag evaluation is local to each node and
/// would over-emit if it spanned the cluster.
/// </summary>
@@ -110,14 +114,20 @@ public sealed class DependencyMuxActor : ReceiveActor
var dep = new VirtualTagActor.DependencyValueChanged(msg.FullReference, msg.Value, msg.TimestampUtc, msg.StatusCode);
foreach (var sub in subscribers)
{
sub.Tell(dep);
// A single-item batch, so the mux emits exactly ONE message shape and every subscriber needs
// only the batch receive on its hot path. One list per subscriber: batch ownership transfers
// to the receiver, so a shared list must never cross mailboxes. Nothing in production reaches
// this path (see the class doc) — the extra allocation is noise on a test-only seam.
sub.Tell(new VirtualTagActor.DependencyValueBatch(new[] { dep }));
}
}
/// <summary>Filters one batched driver production burst per item — the batched analogue of
/// <see cref="OnAttributeValuePublished"/>. Items nobody registered for are dropped (the common case:
/// only a fraction of an address space feeds virtual-tag expressions), so a burst of N values costs one
/// message plus one DependencyValueChanged per genuinely interested (item, subscriber) pair.
/// <summary>Filters one batched driver production burst per item and delivers each subscriber's matched
/// slice as ONE <see cref="VirtualTagActor.DependencyValueBatch"/>. Items nobody registered for are
/// dropped (the common case: only a fraction of an address space feeds virtual-tag expressions), so a
/// burst of N values costs one inbound message plus one outbound message per genuinely interested
/// SUBSCRIBER — not per (item, subscriber) pair as it did when this fanned out per value. That is what
/// lets <see cref="VirtualTagActor"/> evaluate its script once per burst instead of once per input.
/// <para>The registered set is NOT just virtual-tag inputs: <c>ContinuousHistorizationRecorder</c>
/// registers interest in every historized ref, so on a historizing node <see cref="_byRef"/> can be
/// large and a sizeable fraction of each burst resolves.</para></summary>
@@ -126,6 +136,10 @@ public sealed class DependencyMuxActor : ReceiveActor
// Nobody has registered anything — skip the whole burst rather than probing an empty map per item.
if (_byRef.Count == 0) return;
// Allocated lazily on the first matched item: a burst in which nothing is registered for — still
// the common case on a node with a handful of virtual tags — allocates nothing at all.
Dictionary<IActorRef, List<VirtualTagActor.DependencyValueChanged>>? perSubscriber = null;
// Indexed for-loop, not foreach: this is the hot value ingress and an IReadOnlyList<T> enumerator
// would allocate once per burst.
var count = msg.Items.Count;
@@ -134,7 +148,25 @@ public sealed class DependencyMuxActor : ReceiveActor
var item = msg.Items[i];
if (!_byRef.TryGetValue(item.FullReference, out var subscribers) || subscribers.Count == 0) continue;
var dep = new VirtualTagActor.DependencyValueChanged(item.FullReference, item.Value, item.TimestampUtc, item.StatusCode);
foreach (var sub in subscribers) sub.Tell(dep);
perSubscriber ??= new Dictionary<IActorRef, List<VirtualTagActor.DependencyValueChanged>>();
foreach (var sub in subscribers)
{
if (!perSubscriber.TryGetValue(sub, out var list))
{
list = new List<VirtualTagActor.DependencyValueChanged>();
perSubscriber[sub] = list;
}
list.Add(dep);
}
}
if (perSubscriber is null) return;
// One message per subscriber, carrying that subscriber's own list. The lists are built here and
// never touched again, so handing ownership to the receiver is safe.
foreach (var (sub, list) in perSubscriber)
{
sub.Tell(new VirtualTagActor.DependencyValueBatch(list));
}
}
}
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/VirtualTags/VirtualTagActor.cs
+45 -3
View File
@@ -37,6 +37,13 @@ public sealed class VirtualTagActor : ReceiveActor
public sealed record DependencyValueChanged(
string TagId, object? Value, DateTime TimestampUtc, uint StatusCode = OpcUaStatus.Good);
/// <summary>
/// One subscriber's slice of one upstream burst: every item in this batch matched
/// this subscriber's registered interest set. Never empty. List ownership transfers
/// to the receiver (the mux allocates a fresh list per subscriber per burst).
/// </summary>
public sealed record DependencyValueBatch(IReadOnlyList<DependencyValueChanged> Items);
/// <summary>Re-assert the last emitted value/quality to the parent bridge, bypassing the value dedup.
/// Sent by <see cref="VirtualTagHostActor"/> on every apply (deploy) to a surviving child: a deploy
/// re-materialises the child's published UNS node to <c>BadWaitingForInitialData</c>, but an
@@ -132,6 +139,7 @@ public sealed class VirtualTagActor : ReceiveActor
_telemetryHub = telemetryHub;
Receive<DependencyValueChanged>(OnDependencyChanged);
Receive<DependencyValueBatch>(OnDependencyBatch);
Receive<ReassertValue>(_ => OnReassertValue());
}
@@ -179,7 +187,41 @@ public sealed class VirtualTagActor : ReceiveActor
private void OnDependencyChanged(DependencyValueChanged msg)
{
_dependencies[msg.TagId] = msg.Value;
Evaluate(msg.TimestampUtc);
}
/// <summary>Applies EVERY item in the batch to the dependency map first, then evaluates the script
/// ONCE. This is the whole point of the batched mux delivery: a burst that moves K of this tag's inputs
/// used to arrive as K messages and cost K script evaluations, K-1 of them against a half-updated input
/// set whose intermediate results nobody wants (and which the value dedup could not suppress, since each
/// intermediate value usually differs). Deliberately NOT a loop over
/// <see cref="OnDependencyChanged"/>.
/// <para>The single evaluation is stamped with the LAST item's timestamp — the newest sample in the
/// burst, and the same stamp the final evaluation of the old per-item fan-out carried.</para></summary>
private void OnDependencyBatch(DependencyValueBatch msg)
{
var items = msg.Items;
var count = items.Count;
// The mux never sends an empty batch; guard anyway rather than evaluate on no new input.
if (count == 0) return;
// Indexed for-loop, not foreach: this is the hot value ingress and an IReadOnlyList<T> enumerator
// would allocate once per burst.
for (var i = 0; i < count; i++)
{
var item = items[i];
_dependencies[item.TagId] = item.Value;
}
Evaluate(items[count - 1].TimestampUtc);
}
/// <summary>Recomputes the expression against the current dependency map and emits/dedups the result.
/// Shared by the single-value and batched dependency handlers — the only difference between them is how
/// many inputs were applied before this runs.</summary>
/// <param name="timestampUtc">The triggering dependency change's timestamp, stamped on the result.</param>
private void Evaluate(DateTime timestampUtc)
{
VirtualTagEvalResult result;
try
{
@@ -188,13 +230,13 @@ public sealed class VirtualTagActor : ReceiveActor
catch (Exception ex)
{
_log.Warning(ex, "VirtualTag {Id}: evaluator threw", _virtualTagId);
OnEvaluationFailed($"evaluator threw: {ex.Message}", "Error", msg.TimestampUtc);
OnEvaluationFailed($"evaluator threw: {ex.Message}", "Error", timestampUtc);
return;
}
if (!result.Success)
{
OnEvaluationFailed(result.Reason ?? "evaluator failure", "Warning", msg.TimestampUtc);
OnEvaluationFailed(result.Reason ?? "evaluator failure", "Warning", timestampUtc);
return;
}
@@ -219,7 +261,7 @@ public sealed class VirtualTagActor : ReceiveActor
_lastValue = result.Value;
_lastPublishedBad = false;
OtOpcUaTelemetry.VirtualTagEval.Add(1, new KeyValuePair<string, object?>("outcome", "ok"));
var evalResult = new EvaluationResult(_virtualTagId, result.Value, msg.TimestampUtc, CorrelationId.NewId());
var evalResult = new EvaluationResult(_virtualTagId, result.Value, timestampUtc, CorrelationId.NewId());
Context.Parent.Tell(evalResult);
}
tests/Core/ZB.MOM.WW.OtOpcUa.Configuration.Tests/LdapGroupRoleMappingServiceTests.cs
+47 -14
View File
@@ -1,4 +1,5 @@
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Storage;
using Shouldly;
using Xunit;
using ZB.MOM.WW.OtOpcUa.Configuration.Entities;
@@ -8,21 +9,9 @@ using ZB.MOM.WW.OtOpcUa.Configuration.Services;
namespace ZB.MOM.WW.OtOpcUa.Configuration.Tests;
[Trait("Category", "Unit")]
public sealed class LdapGroupRoleMappingServiceTests : IDisposable
public sealed class LdapGroupRoleMappingServiceTests
{
private readonly OtOpcUaConfigDbContext _db;
/// <summary>Initializes a new instance of the LdapGroupRoleMappingServiceTests class.</summary>
public LdapGroupRoleMappingServiceTests()
{
var options = new DbContextOptionsBuilder<OtOpcUaConfigDbContext>()
.UseInMemoryDatabase($"ldap-grm-{Guid.NewGuid():N}")
.Options;
_db = new OtOpcUaConfigDbContext(options);
}
/// <summary>Disposes the database context.</summary>
public void Dispose() => _db.Dispose();
private readonly RecordingContextFactory _db = new();
private LdapGroupRoleMapping Make(string group, AdminRole role, string? clusterId = null, bool? isSystemWide = null) =>
new()
@@ -165,4 +154,48 @@ public sealed class LdapGroupRoleMappingServiceTests : IDisposable
var byGroup = await svc.GetByGroupsAsync(["cn=sysadmins,dc=x"], CancellationToken.None);
byGroup.ShouldContain(r => r.Id == saved.Id);
}
/// <summary>
/// The service must take its context from the factory PER CALL and dispose it before returning —
/// it must never pin one context (and therefore one pooled connection) for its own lifetime. That
/// is what makes it safe on the RoleGrants Blazor page, where the DI scope is the circuit and two
/// renders can overlap: a pinned context would throw
/// "A second operation was started on this context instance" the first time they did.
/// </summary>
[Fact]
public async Task Every_call_creates_and_disposes_its_own_context()
{
var svc = new LdapGroupRoleMappingService(_db);
await svc.CreateAsync(Make("cn=fleet,dc=x", AdminRole.Administrator), CancellationToken.None);
await svc.ListAllAsync(CancellationToken.None);
await svc.GetByGroupsAsync(["cn=fleet,dc=x"], CancellationToken.None);
// One context per call, and no leftovers: a service holding its own context would show 0.
_db.Created.Count.ShouldBe(3);
foreach (var ctx in _db.Created)
Should.Throw<ObjectDisposedException>(() => ctx.LdapGroupRoleMappings.ToList());
}
/// <summary>
/// Hands out a fresh <see cref="OtOpcUaConfigDbContext"/> per call over one shared in-memory store
/// (an explicit <see cref="InMemoryDatabaseRoot"/>, so every context genuinely sees the same rows),
/// and records what it created so a test can assert the per-call create/dispose discipline.
/// </summary>
private sealed class RecordingContextFactory : IDbContextFactory<OtOpcUaConfigDbContext>
{
private readonly DbContextOptions<OtOpcUaConfigDbContext> _options =
new DbContextOptionsBuilder<OtOpcUaConfigDbContext>()
.UseInMemoryDatabase($"ldap-grm-{Guid.NewGuid():N}", new InMemoryDatabaseRoot())
.Options;
public List<OtOpcUaConfigDbContext> Created { get; } = [];
public OtOpcUaConfigDbContext CreateDbContext()
{
var ctx = new OtOpcUaConfigDbContext(_options);
Created.Add(ctx);
return ctx;
}
}
}
tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/PollGroupEngineTests.cs
+102
View File
@@ -598,6 +598,108 @@ public sealed class PollGroupEngineTests
sw.Elapsed.ShouldBeLessThan(TimeSpan.FromSeconds(2));
}
// ---- Fixed-rate pacing + overrun counter + async unsubscribe (Task 11) ----
/// <summary>
/// A reader slower than the publishing interval must NOT push the cycle out to
/// (read + interval): the loop paces at a fixed RATE, so the post-poll delay is
/// <c>max(interval - elapsed, 0)</c> and the next poll starts immediately. The overrun is
/// counted and reported through the <c>onOverrun</c> callback.
/// </summary>
/// <remarks>
/// Numbers: interval 150 ms, reader 250 ms. Old (delay-after-completion) cycle = 400 ms →
/// ≈5 polls started in 2 s. Fixed-rate cycle = 250 ms → ≈8. The ≥7 assertion sits clear of
/// both the old shape's ceiling and CI jitter.
/// </remarks>
[Fact]
public async Task A_slow_reader_overrun_is_counted_and_the_next_poll_starts_promptly()
{
var readCount = 0;
async Task<IReadOnlyList<DataValueSnapshot>> Reader(IReadOnlyList<string> refs, CancellationToken ct)
{
Interlocked.Increment(ref readCount);
await Task.Delay(250, ct);
var now = DateTime.UtcNow;
return refs.Select(_ => new DataValueSnapshot(1, 0u, now, now)).ToList();
}
var overruns = new ConcurrentQueue<TimeSpan>();
await using var engine = new PollGroupEngine(
Reader,
(_, _, _) => { },
minInterval: TimeSpan.FromMilliseconds(50),
onOverrun: elapsed => overruns.Enqueue(elapsed));
var handle = engine.Subscribe(["X"], TimeSpan.FromMilliseconds(150));
await Task.Delay(2000, TestContext.Current.CancellationToken);
var observedReads = Volatile.Read(ref readCount);
var observedOverruns = engine.OverrunCount;
await engine.UnsubscribeAsync(handle);
observedReads.ShouldBeGreaterThanOrEqualTo(7);
observedOverruns.ShouldBeGreaterThanOrEqualTo(4);
overruns.Count.ShouldBeGreaterThanOrEqualTo(4);
overruns.ShouldAllBe(e => e > TimeSpan.FromMilliseconds(150));
}
/// <summary>
/// Falsification control for the overrun counter: a reader that completes well inside the
/// interval must never register an overrun (an unconditional increment would fail here).
/// </summary>
[Fact]
public async Task A_fast_reader_never_overruns()
{
var src = new FakeSource();
src.Values["X"] = 1;
var overruns = new ConcurrentQueue<TimeSpan>();
await using var engine = new PollGroupEngine(
src.ReadAsync,
(_, _, _) => { },
minInterval: TimeSpan.FromMilliseconds(50),
onOverrun: e => overruns.Enqueue(e));
var handle = engine.Subscribe(["X"], TimeSpan.FromMilliseconds(200));
await WaitForAsync(() => Volatile.Read(ref src.ReadCount) >= 4, TimeSpan.FromSeconds(3));
await engine.UnsubscribeAsync(handle);
Volatile.Read(ref src.ReadCount).ShouldBeGreaterThanOrEqualTo(4);
// Tolerate at most one blip: a loaded CI runner can legitimately delay a single cycle past
// the interval, but a real pacing regression produces overruns on most cycles, far above 1.
engine.OverrunCount.ShouldBeLessThanOrEqualTo(1);
overruns.Count.ShouldBeLessThanOrEqualTo(1);
}
/// <summary>
/// The async teardown must be behaviourally identical to the sync one: the handle is known,
/// the loop stops, and no further callback fires. Mirrors
/// <see cref="Unsubscribe_halts_the_loop"/> through <c>UnsubscribeAsync</c>.
/// </summary>
[Fact]
public async Task UnsubscribeAsync_stops_callbacks_like_the_sync_path()
{
var src = new FakeSource();
src.Values["X"] = 1;
var events = new ConcurrentQueue<(ISubscriptionHandle, string, DataValueSnapshot)>();
await using var engine = new PollGroupEngine(src.ReadAsync,
(h, r, s) => events.Enqueue((h, r, s)));
var handle = engine.Subscribe(["X"], TimeSpan.FromMilliseconds(100));
await WaitForAsync(() => events.Count >= 1, TimeSpan.FromSeconds(1));
(await engine.UnsubscribeAsync(handle)).ShouldBeTrue();
var afterUnsub = events.Count;
engine.ActiveSubscriptionCount.ShouldBe(0);
src.Values["X"] = 999;
await Task.Delay(400, TestContext.Current.CancellationToken);
events.Count.ShouldBe(afterUnsub);
// An unknown handle is false through the async path too.
(await engine.UnsubscribeAsync(new DummyHandle())).ShouldBeFalse();
(await engine.UnsubscribeAsync(handle)).ShouldBeFalse();
}
private sealed record DummyHandle : ISubscriptionHandle
{
/// <summary>Gets a diagnostic identifier for this handle.</summary>
tests/Core/ZB.MOM.WW.OtOpcUa.Core.ScriptedAlarms.Tests/ScriptedAlarmEngineTests.cs
+405
View File
@@ -1434,4 +1434,409 @@ public sealed class ScriptedAlarmEngineTests
.GetValue(evaluator)!;
return new WeakReference(del.Method.Module.Assembly);
}
// -------------------------------------------------------------------------
// Dirty-set coalescing (perf roadmap task 15).
//
// Pre-fix, every upstream push spawned its own fire-and-forget
// ReevaluateAsync, and each one queued on the single _evalGate. N pushes
// arriving during one slow evaluation therefore produced N serialized
// re-evaluations of the same alarms, all of them reading the SAME (latest)
// cached values — N-1 of them redundant by construction.
//
// Post-fix a push only marks its alarms dirty; a single drainer swaps the
// set and evaluates it once. The evaluation-count probe below is a
// ctx.Logger line inside the predicate counted by a Serilog sink: the store
// save-count alone cannot tell the two shapes apart, because the redundant
// passes produce no transition and therefore no save.
// -------------------------------------------------------------------------
/// <summary>Verifies that a storm of pushes arriving while an evaluation is in flight collapses
/// into a single further drain pass whose evaluation sees the LATEST pushed value, with a
/// falsification control proving the collapsed count comes from coalescing and not from dedup.</summary>
[Fact]
public async Task A_storm_of_pushes_coalesces_to_one_drain_pass()
{
var up = new FakeUpstream();
up.Set("Temp", 50);
var probe = new CountingSink("EVALPROBE");
var logger = new LoggerConfiguration().WriteTo.Sink(probe).CreateLogger();
var store = new CountingAlarmStateStore();
using var eng = new ScriptedAlarmEngine(up, store, new ScriptLoggerFactory(logger), logger);
await eng.LoadAsync([
Alarm("A", """ctx.Logger.Information("EVALPROBE"); return (int)ctx.GetTag("Temp").Value > 100;"""),
], TestContext.Current.CancellationToken);
// LoadAsync itself evaluates once (startup recovery) — start counting after it.
probe.Reset();
store.ResetCount();
// Block the save the first push's evaluation performs. That evaluation then
// sits INSIDE _evalGate for as long as the test wants, which is the window
// every subsequent push has to coalesce into.
var saveGate = new TaskCompletionSource();
store.BlockNextSave = saveGate;
up.Push("Temp", 150); // evaluation #1 → Activated → blocks in SaveAsync
await WaitForAsync(() => store.SaveInProgress, timeoutMs: 3000);
// The storm: 10 further pushes while the gate is held. The LAST one drops
// back below the threshold, so "the drain evaluated the latest values" is
// observable as the alarm ending Inactive rather than Active.
for (var i = 1; i <= 9; i++) up.Push("Temp", 150 + i);
up.Push("Temp", 50);
saveGate.SetResult();
await WaitForAsync(() => eng.GetState("A")!.Active == AlarmActiveState.Inactive, timeoutMs: 3000);
probe.Count.ShouldBeLessThanOrEqualTo(2,
"11 pushes over one blocked evaluation must produce the in-flight evaluation plus at " +
"most ONE coalesced drain pass — pre-fix this was 11 serialized re-evaluations.");
store.SaveCount.ShouldBe(2,
"only genuine transitions persist: Activated (the blocked save) + Cleared (the drain).");
eng.GetState("A")!.Active.ShouldBe(AlarmActiveState.Inactive,
"the coalesced drain must evaluate the LATEST cached values — latest-wins, never lost-update.");
// --- Falsification control -------------------------------------------
// Spaced pushes (each drain completes before the next push lands) must
// still produce one evaluation EACH. If this collapsed too, the counts
// above would be proving dedup/short-circuiting rather than coalescing.
probe.Reset();
store.ResetCount();
foreach (var v in new[] { 150, 50, 150, 50 })
{
up.Push("Temp", v);
var want = v > 100 ? AlarmActiveState.Active : AlarmActiveState.Inactive;
await WaitForAsync(() => eng.GetState("A")!.Active == want, timeoutMs: 3000);
}
probe.Count.ShouldBe(4,
"four pushes that each land on an idle engine must each be evaluated — the collapsed " +
"count in the storm above comes from coalescing, not from the engine skipping work.");
store.SaveCount.ShouldBe(4);
}
/// <summary>Verifies that a push landing while a drain pass holds the evaluation gate is not
/// dropped — the drainer loops and evaluates it in a following pass.</summary>
[Fact]
public async Task A_push_during_a_drain_is_not_lost()
{
var up = new FakeUpstream();
up.Set("Temp", 50);
var probe = new CountingSink("EVALPROBE");
var logger = new LoggerConfiguration().WriteTo.Sink(probe).CreateLogger();
var store = new CountingAlarmStateStore();
using var eng = new ScriptedAlarmEngine(up, store, new ScriptLoggerFactory(logger), logger);
await eng.LoadAsync([
Alarm("A", """ctx.Logger.Information("EVALPROBE"); return (int)ctx.GetTag("Temp").Value > 100;"""),
], TestContext.Current.CancellationToken);
probe.Reset();
var saveGate = new TaskCompletionSource();
store.BlockNextSave = saveGate;
up.Push("Temp", 150); // drain pass #1 → Activated → blocks in SaveAsync
await WaitForAsync(() => store.SaveInProgress, timeoutMs: 3000);
// Lands while the drainer is inside _evalGate: it must be recorded in the
// dirty set and picked up by the drainer's next loop iteration, NOT dropped
// because "a drain is already scheduled".
up.Push("Temp", 40);
saveGate.SetResult();
await WaitForAsync(() => eng.GetState("A")!.Active == AlarmActiveState.Inactive, timeoutMs: 3000);
probe.Count.ShouldBe(2, "the in-flight pass plus exactly one follow-up pass for the late push");
}
/// <summary>
/// Verifies the drain latch (<c>_drainScheduled</c>) is released even when an exception
/// escapes the drain loop, so a later push still starts a drainer. A latch left stuck true
/// stops evaluation for EVERY scripted alarm, permanently and silently.
/// </summary>
/// <remarks>
/// The escape is produced without touching the engine: the store's SaveAsync throws (which
/// <c>ReevaluateAsync</c> catches, as designed), and the engine logger is configured with a
/// Serilog <b>audit</b> sink that throws on that catch block's own "reevaluate failed"
/// Error() call. Audit sinks propagate by contract, so the throw leaves <c>ReevaluateAsync</c>
/// through its catch and unwinds <c>DrainDirtyAsync</c> — the only escape path the current
/// code actually has, and a real one for any deployment auditing its logs.
/// </remarks>
[Fact]
public async Task A_throw_escaping_the_drain_loop_still_releases_the_drain_latch()
{
var up = new FakeUpstream();
up.Set("Temp", 50);
var probe = new CountingSink("EVALPROBE");
var logger = new LoggerConfiguration()
.WriteTo.Sink(probe)
.AuditTo.Sink(new ThrowingSink("reevaluate failed"))
.CreateLogger();
var store = new CountingAlarmStateStore();
using var eng = new ScriptedAlarmEngine(up, store, new ScriptLoggerFactory(logger), logger);
await eng.LoadAsync([
Alarm("A", """ctx.Logger.Information("EVALPROBE"); return (int)ctx.GetTag("Temp").Value > 100;"""),
], TestContext.Current.CancellationToken);
// LoadAsync itself evaluates + persists once (startup recovery) — count from zero.
probe.Reset();
store.ResetCount();
// Push #1: transition → SaveAsync throws → the catch's Error() hits the audit
// sink and throws → the exception unwinds out of the drain loop. Waiting on the
// save ATTEMPT (counted before the throw) rather than on the probe keeps the
// predicate-evaluation count below assertable.
store.ThrowOnNextSave = true;
up.Push("Temp", 150);
await WaitForAsync(() => store.SaveCount == 1, timeoutMs: 3000);
probe.Count.ShouldBe(1, "the faulted pass evaluated the predicate exactly once");
eng.GetState("A")!.Active.ShouldBe(AlarmActiveState.Inactive,
"persist-before-update: a failed save must leave the in-memory condition at the prior state");
// The pin. This push either starts a fresh drainer (latch already released) or is
// handed to the replacement drainer the unwinding pass starts (latch still held,
// dirty set non-empty) — the two orderings the race between "SaveAsync was called"
// and "the drain finished unwinding" produces, and both must evaluate it. With the
// latch stuck true, OnUpstreamChange only grows the dirty set, no predicate ever
// runs again, and the alarm silently never activates.
up.Push("Temp", 160);
await WaitForAsync(() => eng.GetState("A")!.Active == AlarmActiveState.Active, timeoutMs: 3000);
probe.Count.ShouldBe(2, "the push after the faulted drain must itself be evaluated");
}
/// <summary>
/// Verifies a long chain of drain-latch handovers under sustained <b>synchronous</b> failure
/// neither stalls the engine nor deepens the native stack: each replacement drainer must
/// start on a FRESH stack rather than nesting inside the finally block that spawned it.
/// </summary>
/// <remarks>
/// <para><b>The escape has to be synchronous, and picking the wrong one makes this test
/// vacuous.</b> A drain pass whose failure comes from the STORE cannot nest: the predicate
/// runs first, and <c>TimedScriptEvaluator.RunAsync</c> hops to the pool
/// (<c>Task.Run(...).WaitAsync(timeout)</c>) on every evaluation, so the drainer has already
/// suspended and returned to its caller long before the throw. Measured: depth is a flat 36
/// across 200 store-throw handovers whether the replacement is queued or called. The escape
/// used here is therefore the injected <b>clock</b>, which <c>ReevaluateAsync</c> calls at
/// the top of each alarm's iteration — under a free <c>_evalGate</c>, before any awaited
/// work — so the throw, the catch, the audit sink's rethrow and the whole unwind happen
/// without a single suspension.</para>
/// <para>The chain is deterministic rather than raced: the clock re-dirties the alarm from
/// inside the failing call (a re-entrant upstream push, which only grows the dirty set
/// because the latch is already held), so every unwinding pass finds work and hands the
/// latch to a replacement — exactly <c>Handovers</c> times, with no dependence on thread
/// timing. Depth is sampled per call: flat when the handover goes through <c>Task.Run</c>,
/// climbing monotonically when the replacement is called inline.</para>
/// </remarks>
[Fact]
public async Task Sustained_synchronous_drain_failures_hand_over_without_deepening_the_stack()
{
const int Handovers = 200;
var up = new FakeUpstream();
up.Set("Temp", 50);
var logger = new LoggerConfiguration()
.AuditTo.Sink(new ThrowingSink("reevaluate failed"))
.CreateLogger();
var clock = new RedirtyingThrowingClock(up, "Temp", Handovers);
using var eng = new ScriptedAlarmEngine(
up, new InMemoryAlarmStateStore(), new ScriptLoggerFactory(logger), logger, clock.Now);
await eng.LoadAsync([
Alarm("A", """return (int)ctx.GetTag("Temp").Value > 100;"""),
], TestContext.Current.CancellationToken);
// Arm AFTER load: LoadAsync's startup-recovery pass reads the clock too, and it must not
// be the pass that throws.
clock.Arm();
// One push starts the chain; every failing read re-dirties and throws, so the chain
// self-sustains for exactly Handovers handovers and then stops.
up.Push("Temp", 150);
await WaitForAsync(() => clock.CallCount >= Handovers + 1, timeoutMs: 60000);
clock.CallCount.ShouldBe(Handovers + 1,
"every handover must produce exactly one further evaluation pass — no stall, no duplication");
(clock.MaxDepth - clock.FirstDepth).ShouldBeLessThan(200,
$"a queued handover starts on a fresh stack, so depth must stay flat across {Handovers} " +
$"handovers (first={clock.FirstDepth}, max={clock.MaxDepth})");
// Liveness: the latch was released at the end of the chain, so the engine still works.
clock.Disarm();
up.Push("Temp", 170);
await WaitForAsync(() => eng.GetState("A")!.Active == AlarmActiveState.Active, timeoutMs: 5000);
}
/// <summary>
/// A clock that, once armed, re-dirties an alarm by pushing to the upstream and then throws —
/// driving a deterministic chain of drain-latch handovers whose every step is synchronous —
/// while sampling the native stack depth of each call.
/// </summary>
private sealed class RedirtyingThrowingClock
{
private readonly FakeUpstream _up;
private readonly string _path;
private readonly int _redirtyLimit;
private readonly object _gate = new();
private volatile bool _armed;
private int _calls;
private int _firstDepth;
private int _maxDepth;
/// <summary>Initializes the clock with the upstream to re-dirty and the handover count to drive.</summary>
/// <param name="up">Upstream fake the failing call re-pushes to.</param>
/// <param name="path">Tag path to re-push, which must be one the alarm reads.</param>
/// <param name="redirtyLimit">Number of calls that re-dirty before the chain is allowed to end.</param>
public RedirtyingThrowingClock(FakeUpstream up, string path, int redirtyLimit)
{ _up = up; _path = path; _redirtyLimit = redirtyLimit; }
/// <summary>Gets the number of clock reads taken while armed.</summary>
public int CallCount => Volatile.Read(ref _calls);
/// <summary>Gets the stack depth sampled on the first armed read.</summary>
public int FirstDepth => Volatile.Read(ref _firstDepth);
/// <summary>Gets the deepest stack sampled across all armed reads.</summary>
public int MaxDepth => Volatile.Read(ref _maxDepth);
/// <summary>Starts failing (and re-dirtying) on every read.</summary>
public void Arm() => _armed = true;
/// <summary>Stops failing, so the clock behaves normally again.</summary>
public void Disarm() => _armed = false;
/// <summary>Samples the stack depth, re-dirties the alarm, and throws — or returns the time when disarmed.</summary>
public DateTime Now()
{
if (!_armed) return DateTime.UtcNow;
var n = Interlocked.Increment(ref _calls);
var depth = new System.Diagnostics.StackTrace(false).FrameCount;
lock (_gate)
{
if (n == 1) _firstDepth = depth;
if (depth > _maxDepth) _maxDepth = depth;
}
// Re-dirty from inside the failing pass: the latch is held by the drainer that
// called us, so this only grows the dirty set — which is exactly the condition
// that makes the unwinding drainer hand the latch over instead of releasing it.
if (n <= _redirtyLimit) _up.Push(_path, 150);
throw new InvalidOperationException("clock unavailable (test)");
}
}
/// <summary>
/// A Serilog sink that throws when a marker event is emitted. Registered through
/// <c>AuditTo</c> so the failure propagates to the caller instead of being swallowed by
/// Serilog's normal best-effort emit — the mechanism the drain-latch test uses to make an
/// exception escape a method that otherwise catches everything.
/// </summary>
private sealed class ThrowingSink : Serilog.Core.ILogEventSink
{
private readonly string _marker;
/// <summary>Initializes the sink with the marker substring that triggers the throw.</summary>
/// <param name="marker">Substring the message template must contain to throw.</param>
public ThrowingSink(string marker) => _marker = marker;
/// <summary>Throws if the event's message template carries the marker; otherwise ignores it.</summary>
/// <param name="logEvent">The Serilog event being emitted.</param>
public void Emit(Serilog.Events.LogEvent logEvent)
{
if (logEvent.MessageTemplate.Text.Contains(_marker, StringComparison.Ordinal))
throw new InvalidOperationException($"audit sink refused '{_marker}' (test)");
}
}
/// <summary>
/// A Serilog sink that counts the events whose message template contains a marker string.
/// Used to count predicate EVALUATIONS (the predicate emits the marker through
/// <c>ctx.Logger</c>) — the store's save count cannot serve as that probe, because a
/// redundant re-evaluation produces no transition and therefore no save.
/// </summary>
private sealed class CountingSink : Serilog.Core.ILogEventSink
{
private readonly string _marker;
private int _count;
/// <summary>Initializes the sink with the marker substring it counts.</summary>
/// <param name="marker">Substring the message template must contain to be counted.</param>
public CountingSink(string marker) => _marker = marker;
/// <summary>Gets the number of matching events emitted since the last <see cref="Reset"/>.</summary>
public int Count => Volatile.Read(ref _count);
/// <summary>Resets the counter to zero.</summary>
public void Reset() => Interlocked.Exchange(ref _count, 0);
/// <summary>Counts the event if its message template carries the marker.</summary>
/// <param name="logEvent">The Serilog event being emitted.</param>
public void Emit(Serilog.Events.LogEvent logEvent)
{
if (logEvent.MessageTemplate.Text.Contains(_marker, StringComparison.Ordinal))
Interlocked.Increment(ref _count);
}
}
/// <summary>
/// Store decorator that counts SaveAsync calls and can hold the next save open until the
/// test releases it — the two capabilities the coalescing tests need together (block one
/// evaluation inside <c>_evalGate</c>, then count what the released drain actually did).
/// </summary>
private sealed class CountingAlarmStateStore : IAlarmStateStore
{
private readonly InMemoryAlarmStateStore _inner = new();
private int _saveCount;
/// <summary>Gets the number of SaveAsync calls since the last <see cref="ResetCount"/>.</summary>
public int SaveCount => Volatile.Read(ref _saveCount);
/// <summary>Resets the save counter to zero.</summary>
public void ResetCount() => Interlocked.Exchange(ref _saveCount, 0);
/// <summary>Gets or sets a TaskCompletionSource that, when set, blocks the next SaveAsync until signalled.</summary>
public TaskCompletionSource? BlockNextSave { get; set; }
/// <summary>Gets a value indicating whether a SaveAsync call is currently blocked waiting on BlockNextSave.</summary>
public bool SaveInProgress { get; private set; }
/// <summary>
/// Gets or sets a value indicating whether the next SaveAsync throws instead of persisting.
/// Self-clearing, so one failed save can be followed by a healthy one.
/// </summary>
public bool ThrowOnNextSave { get; set; }
/// <summary>Loads an alarm condition state by ID from the inner store.</summary>
/// <param name="alarmId">The ID of the alarm condition state to load.</param>
/// <param name="ct">A cancellation token.</param>
public Task<AlarmConditionState?> LoadAsync(string alarmId, CancellationToken ct)
=> _inner.LoadAsync(alarmId, ct);
/// <summary>Loads all alarm condition states from the inner store.</summary>
/// <param name="ct">A cancellation token.</param>
public Task<IReadOnlyList<AlarmConditionState>> LoadAllAsync(CancellationToken ct)
=> _inner.LoadAllAsync(ct);
/// <summary>Counts the save, optionally throwing or blocking on the BlockNextSave gate, then delegates.</summary>
/// <param name="state">The alarm condition state to save.</param>
/// <param name="ct">A cancellation token.</param>
public async Task SaveAsync(AlarmConditionState state, CancellationToken ct)
{
Interlocked.Increment(ref _saveCount);
if (ThrowOnNextSave)
{
ThrowOnNextSave = false;
throw new InvalidOperationException("store unavailable (test)");
}
var gate = BlockNextSave;
if (gate is not null)
{
BlockNextSave = null;
SaveInProgress = true;
try { await gate.Task.WaitAsync(ct).ConfigureAwait(false); }
finally { SaveInProgress = false; }
}
await _inner.SaveAsync(state, ct).ConfigureAwait(false);
}
/// <summary>Removes an alarm condition state by ID from the inner store.</summary>
/// <param name="alarmId">The ID of the alarm condition state to remove.</param>
/// <param name="ct">A cancellation token.</param>
public Task RemoveAsync(string alarmId, CancellationToken ct)
=> _inner.RemoveAsync(alarmId, ct);
}
}
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.AbCip.Tests/AbCipParallelReadTests.cs
+298
View File
@@ -0,0 +1,298 @@
using Shouldly;
using Xunit;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using ZB.MOM.WW.OtOpcUa.Driver.AbCip;
namespace ZB.MOM.WW.OtOpcUa.Driver.AbCip.Tests;
/// <summary>
/// Task 14 — <c>ReadAsync</c> runs its planned groups + per-tag fallbacks inside a bounded
/// concurrency window (<see cref="AbCipDriverOptions.MaxConcurrentReads"/>) instead of strictly
/// one device round-trip at a time, so libplctag can overlap requests. The window is ACROSS
/// tags only: the per-tag runtime lock (05/STAB-4) still serialises Read → GetStatus → Decode
/// against any one cached handle. Health is aggregated once after the batch (worst outcome
/// wins, <c>LastSuccessfulRead</c> advances if any tag succeeded) because per-item writes to
/// the shared health field would race under the window.
/// </summary>
[Trait("Category", "Unit")]
public sealed class AbCipParallelReadTests
{
private const string DeviceHost = "ab://10.0.0.5/1,0";
/// <summary>
/// Holds every in-flight fake read until either <paramref name="target"/> reads overlap
/// (proving the window opened) or a short grace period elapses (so a window of one, which
/// can never reach the target, still completes instead of deadlocking).
/// </summary>
private sealed class ReadGate(int target)
{
private readonly TaskCompletionSource _release = new(TaskCreationOptions.RunContinuationsAsynchronously);
private int _inFlight;
/// <summary>Gets the highest number of fake reads observed in flight simultaneously.</summary>
public int MaxConcurrent { get; private set; }
/// <summary>Records one in-flight read and holds it open until the gate releases.</summary>
/// <param name="cancellationToken">The read's cancellation token.</param>
public async Task HoldAsync(CancellationToken cancellationToken)
{
var n = Interlocked.Increment(ref _inFlight);
lock (this)
{
if (n > MaxConcurrent) MaxConcurrent = n;
}
try
{
if (n >= target) _release.TrySetResult();
try
{
await _release.Task.WaitAsync(TimeSpan.FromMilliseconds(150), cancellationToken)
.ConfigureAwait(false);
}
catch (TimeoutException)
{
// The window never reached the target — the sequential control case.
}
}
finally
{
Interlocked.Decrement(ref _inFlight);
}
}
}
/// <summary>A fake whose read blocks on the shared gate and records its OWN overlap.</summary>
private sealed class GatedFake(AbCipTagCreateParams p, ReadGate gate) : FakeAbCipTag(p)
{
private int _own;
/// <summary>Gets the highest number of concurrent reads seen on THIS handle.</summary>
public int MaxOwnConcurrent { get; private set; }
/// <inheritdoc />
public override async Task ReadAsync(CancellationToken cancellationToken)
{
var mine = Interlocked.Increment(ref _own);
lock (this)
{
if (mine > MaxOwnConcurrent) MaxOwnConcurrent = mine;
}
try
{
await gate.HoldAsync(cancellationToken).ConfigureAwait(false);
await base.ReadAsync(cancellationToken).ConfigureAwait(false);
}
finally
{
Interlocked.Decrement(ref _own);
}
}
}
private static AbCipDriver NewDriver(
IAbCipTagFactory factory, int maxConcurrentReads, params AbCipTagDefinition[] tags)
{
var opts = new AbCipDriverOptions
{
Devices = [new AbCipDeviceOptions(DeviceHost)],
RawTags = AbCipRawTags.From(tags),
Probe = new AbCipProbeOptions { Enabled = false },
MaxConcurrentReads = maxConcurrentReads,
};
return new AbCipDriver(opts, "drv-1", factory);
}
private static AbCipTagDefinition Tag(string name) =>
new(name, DeviceHost, $"Motor1.{name}", AbCipDataType.DInt);
/// <summary>Four distinct tags under a window of eight overlap on the wire.</summary>
[Fact]
public async Task Reads_overlap_up_to_the_window()
{
var gate = new ReadGate(target: 2);
var factory = new FakeAbCipTagFactory { Customise = p => new GatedFake(p, gate) { Value = 1 } };
var drv = NewDriver(factory, maxConcurrentReads: 8,
Tag("A"), Tag("B"), Tag("C"), Tag("D"));
await drv.InitializeAsync("{}", CancellationToken.None);
var snapshots = await drv.ReadAsync(["A", "B", "C", "D"], CancellationToken.None);
gate.MaxConcurrent.ShouldBeGreaterThanOrEqualTo(2,
"a window of 8 must let distinct tags overlap on the device");
snapshots.Count.ShouldBe(4);
snapshots.ShouldAllBe(s => s.StatusCode == AbCipStatusMapper.Good);
}
/// <summary>
/// Falsification control for <see cref="Reads_overlap_up_to_the_window"/> — the same
/// fixture with a window of one must observe no overlap at all, i.e. today's behaviour.
/// </summary>
[Fact]
public async Task A_window_of_one_is_sequential()
{
var gate = new ReadGate(target: 2);
var factory = new FakeAbCipTagFactory { Customise = p => new GatedFake(p, gate) { Value = 1 } };
var drv = NewDriver(factory, maxConcurrentReads: 1,
Tag("A"), Tag("B"), Tag("C"), Tag("D"));
await drv.InitializeAsync("{}", CancellationToken.None);
var snapshots = await drv.ReadAsync(["A", "B", "C", "D"], CancellationToken.None);
gate.MaxConcurrent.ShouldBe(1, "MaxConcurrentReads = 1 must be observably sequential");
snapshots.Count.ShouldBe(4);
snapshots.ShouldAllBe(s => s.StatusCode == AbCipStatusMapper.Good);
}
/// <summary>
/// The window parallelises ACROSS tags, never within one — the same reference twice in a
/// batch still serialises on the per-tag runtime lock (05/STAB-4).
/// </summary>
[Fact]
public async Task Same_tag_requested_twice_is_still_serialised()
{
var gate = new ReadGate(target: 2);
GatedFake? shared = null;
var factory = new FakeAbCipTagFactory
{
Customise = p => shared = new GatedFake(p, gate) { Value = 5 },
};
var drv = NewDriver(factory, maxConcurrentReads: 8, Tag("A"));
await drv.InitializeAsync("{}", CancellationToken.None);
var snapshots = await drv.ReadAsync(["A", "A"], CancellationToken.None);
shared.ShouldNotBeNull();
shared!.MaxOwnConcurrent.ShouldBe(1,
"operations on one shared libplctag Tag handle must never overlap");
snapshots.Count.ShouldBe(2);
snapshots.ShouldAllBe(s => s.StatusCode == AbCipStatusMapper.Good);
}
/// <summary>
/// Counts + overlaps fake <c>InitializeAsync</c> (the Forward-Open stand-in) calls. Holds
/// every in-flight connect until either <paramref name="target"/> overlap (proving the
/// connects were NOT single-flight) or a short grace period elapses, so a correctly
/// serialised run still completes instead of deadlocking.
/// </summary>
private sealed class ConnectProbe(int target)
{
private readonly TaskCompletionSource _release = new(TaskCreationOptions.RunContinuationsAsynchronously);
private int _inFlight;
private int _attempts;
/// <summary>Gets the highest number of connect attempts observed in flight simultaneously.</summary>
public int MaxConcurrent { get; private set; }
/// <summary>Gets the total number of connect attempts that reached the wire.</summary>
public int Attempts => Volatile.Read(ref _attempts);
/// <summary>Records one in-flight connect and holds it open until the probe releases.</summary>
/// <param name="cancellationToken">The connect's cancellation token.</param>
public async Task HoldAsync(CancellationToken cancellationToken)
{
Interlocked.Increment(ref _attempts);
var n = Interlocked.Increment(ref _inFlight);
lock (this)
{
if (n > MaxConcurrent) MaxConcurrent = n;
}
try
{
if (n >= target) _release.TrySetResult();
try
{
await _release.Task.WaitAsync(TimeSpan.FromMilliseconds(150), cancellationToken)
.ConfigureAwait(false);
}
catch (TimeoutException)
{
// No second connect overlapped — the single-flight case.
}
}
finally
{
Interlocked.Decrement(ref _inFlight);
}
}
}
/// <summary>A fake standing in for a device that is down — every connect attempt fails.</summary>
private sealed class DownDeviceFake(AbCipTagCreateParams p, ConnectProbe probe) : FakeAbCipTag(p)
{
/// <inheritdoc />
public override async Task InitializeAsync(CancellationToken cancellationToken)
{
await probe.HoldAsync(cancellationToken).ConfigureAwait(false);
throw new InvalidOperationException("fake Forward-Open failure");
}
}
/// <summary>
/// Review fix — cold (never-connected) tags on ONE down device must connect single-flight
/// under the parallel read window. The window let up to <c>MaxConcurrentReads</c> work items
/// run the unsynchronised <c>ShouldAttempt</c> → connect → <c>RecordFailure</c> sequence on
/// the shared per-device <c>ConnectionBackoff</c> concurrently: every cold tag passed
/// <c>ShouldAttempt</c> before any failure was recorded (N simultaneous Forward-Opens — a
/// thundering herd at the device), and the unsynchronised <c>_consecutiveFailures++</c> lost
/// increments, weakening the 05/STAB-8 backoff into a reconnect storm.
/// </summary>
[Fact]
public async Task Cold_tags_on_one_down_device_connect_single_flight()
{
var probe = new ConnectProbe(target: 2);
var factory = new FakeAbCipTagFactory { Customise = p => new DownDeviceFake(p, probe) };
var drv = NewDriver(factory, maxConcurrentReads: 8,
Tag("A"), Tag("B"), Tag("C"), Tag("D"), Tag("E"), Tag("F"));
await drv.InitializeAsync("{}", CancellationToken.None);
var snapshots = await drv.ReadAsync(["A", "B", "C", "D", "E", "F"], CancellationToken.None);
probe.MaxConcurrent.ShouldBe(1,
"cold connects against ONE device must be single-flight — the backoff check + connect + " +
"record sequence is not internally synchronized");
probe.Attempts.ShouldBe(1,
"the one recorded failure must be visible to every sibling still inside the read window, " +
"so only ONE Forward-Open reaches a down device per batch");
snapshots.Count.ShouldBe(6);
snapshots.ShouldAllBe(s => s.StatusCode == AbCipStatusMapper.BadCommunicationError);
// The failure increment landed and is durable: the 05/STAB-8 window opened by that single
// recorded failure (base 1 s) is still open, so a second batch makes NO further attempt.
var again = await drv.ReadAsync(["A", "B", "C", "D", "E", "F"], CancellationToken.None);
probe.Attempts.ShouldBe(1,
"no lost increment — the recorded failure must still hold the backoff window shut");
again.ShouldAllBe(s => s.StatusCode == AbCipStatusMapper.BadCommunicationError);
}
/// <summary>
/// Health is written once after the batch: the worst per-item outcome wins, and
/// <c>LastSuccessfulRead</c> still advances because at least one tag read succeeded — the
/// contract the comms-loss watchdog reads.
/// </summary>
[Fact]
public async Task Health_after_a_mixed_batch_is_worst_outcome_with_advanced_LastSuccessfulRead()
{
var factory = new FakeAbCipTagFactory
{
Customise = p => p.TagName.Contains("Bad", StringComparison.Ordinal)
? new FakeAbCipTag(p) { ThrowOnRead = true }
: new FakeAbCipTag(p) { Value = 3 },
};
var drv = NewDriver(factory, maxConcurrentReads: 8, Tag("Good"), Tag("Bad"));
await drv.InitializeAsync("{}", CancellationToken.None);
var beforeRead = drv.GetHealth().LastSuccessfulRead;
beforeRead.ShouldNotBeNull();
await Task.Delay(30, TestContext.Current.CancellationToken);
var snapshots = await drv.ReadAsync(["Good", "Bad"], CancellationToken.None);
snapshots[0].StatusCode.ShouldBe(AbCipStatusMapper.Good);
snapshots[1].StatusCode.ShouldBe(AbCipStatusMapper.BadCommunicationError);
var health = drv.GetHealth();
health.State.ShouldBe(DriverState.Degraded, "the worst per-item outcome wins");
health.LastSuccessfulRead.ShouldNotBeNull();
health.LastSuccessfulRead!.Value.ShouldBeGreaterThan(beforeRead!.Value,
"a successful tag in the batch must still advance LastSuccessfulRead");
}
}
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.AbCip.Tests/FakeAbCipTag.cs
+9 -2
View File
@@ -129,8 +129,15 @@ internal sealed class ArrayFakeAbCipTag : FakeAbCipTag
/// <summary>Test factory that produces <see cref="FakeAbCipTag"/>s and indexes them for assertion.</summary>
internal sealed class FakeAbCipTagFactory : IAbCipTagFactory
{
/// <summary>Gets a dictionary of created tags indexed by tag name for assertion.</summary>
public Dictionary<string, FakeAbCipTag> Tags { get; } = new(StringComparer.OrdinalIgnoreCase);
/// <summary>
/// Gets a dictionary of created tags indexed by tag name for assertion. Concurrent because
/// <see cref="AbCipDriver.ReadAsync"/> creates cold runtimes from inside its bounded-parallel
/// read window — two work items for different tags can call <see cref="Create"/> at the same
/// time, which a plain <see cref="Dictionary{TKey,TValue}"/> would answer with a corrupt
/// bucket chain or a lost entry rather than an honest failure.
/// </summary>
public System.Collections.Concurrent.ConcurrentDictionary<string, FakeAbCipTag> Tags { get; } =
new(StringComparer.OrdinalIgnoreCase);
/// <summary>Gets or sets an optional customization function to override the tag creation.</summary>
public Func<AbCipTagCreateParams, FakeAbCipTag>? Customise { get; set; }
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests/Recorder/FasterLogHistorizationOutboxTests.cs
+105
View File
@@ -1,4 +1,5 @@
using System.Linq;
using FASTER.core;
using Xunit;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Recorder;
@@ -140,6 +141,110 @@ public sealed class FasterLogHistorizationOutboxTests : IDisposable
Assert.Equal(new[] { a.Id, b.Id }, batch.Select(e => e.Id));
}
[Fact]
public async Task AppendBatch_then_peek_returns_fifo_in_batch_order()
{
// Group commit: one fsync for the whole batch, and FIFO order is preserved across a restart.
// The reopen leg here proves ORDER and COUNT survive recovery — it deliberately does NOT
// prove the batch committed itself, because Dispose() unconditionally runs
// _log.Commit(spinWait: true) and would rescue a batch that never committed. That claim is
// pinned separately by AppendBatch_group_commit_is_durable_before_dispose.
var dir = NewTempDir();
var a = E("A", 1);
var b = E("B", 2);
var c = E("C", 3);
{
using var o = new FasterLogHistorizationOutbox(dir, HistorizationCommitMode.PerEntry);
await o.AppendBatchAsync([a, b, c], TestContext.Current.CancellationToken);
Assert.Equal(3, await o.CountAsync(TestContext.Current.CancellationToken));
var batch = await o.PeekBatchAsync(10, TestContext.Current.CancellationToken);
Assert.Equal(new[] { a.Id, b.Id, c.Id }, batch.Select(e => e.Id));
}
using var reopened = new FasterLogHistorizationOutbox(dir, HistorizationCommitMode.PerEntry);
Assert.Equal(3, await reopened.CountAsync(TestContext.Current.CancellationToken));
var survived = await reopened.PeekBatchAsync(10, TestContext.Current.CancellationToken);
Assert.Equal(new[] { a.Id, b.Id, c.Id }, survived.Select(e => e.Id));
}
[Fact]
public async Task AppendBatch_group_commit_is_durable_before_dispose()
{
// THE group-commit pin. Every other durability test disposes the writer before reopening, and
// Dispose() unconditionally commits — so an AppendBatchAsync that never called CommitAsync
// would still pass them. This one never disposes the writer before observing durability:
// it opens a SECOND, read-only FasterLog over the same directory while the writing outbox is
// still alive, and reads only up to CommittedUntilAddress. Anything visible there was made
// durable by AppendBatchAsync's own group commit and nothing else. (PerEntry mode, so no
// periodic-commit loop exists to commit behind the batch's back either.)
var dir = NewTempDir();
var a = E("A", 1);
var b = E("B", 2);
var c = E("C", 3);
var writer = new FasterLogHistorizationOutbox(dir, HistorizationCommitMode.PerEntry);
try
{
await writer.AppendBatchAsync([a, b, c], TestContext.Current.CancellationToken);
Assert.Equal(new[] { a.Id, b.Id, c.Id }, ReadCommitted(dir));
}
finally
{
writer.Dispose();
}
}
// Opens an independent read-only FasterLog over the outbox's directory and returns the ids of
// every record the writer has COMMITTED (BeginAddress .. CommittedUntilAddress). Uncommitted
// enqueues sit past CommittedUntilAddress and are therefore invisible here — which is exactly
// what makes this a durability observation rather than a memory one.
private static Guid[] ReadCommitted(string dir)
{
using var device = new ManagedLocalStorageDevice(Path.Combine(dir, "hlog.log"));
using var reader = new FasterLog(new FasterLogSettings { LogDevice = device, ReadOnlyMode = true });
var ids = new List<Guid>();
using FasterLogScanIterator iter = reader.Scan(reader.BeginAddress, reader.CommittedUntilAddress, recover: false);
while (iter.GetNext(out byte[] bytes, out int len, out _, out _))
ids.Add(HistorizationOutboxEntrySerializer.Deserialize(bytes.AsSpan(0, len)).Id);
return ids.ToArray();
}
[Fact]
public async Task AppendBatch_at_capacity_drops_oldest_across_the_whole_batch()
{
// The drop-oldest loop runs ONCE for the batch, so an over-capacity batch must shed every
// surplus entry (not just one) and count each drop.
var dir = NewTempDir();
using var o = new FasterLogHistorizationOutbox(dir, HistorizationCommitMode.PerEntry, capacity: 2);
var a = E("A", 1);
var b = E("B", 2);
var c = E("C", 3);
var d = E("D", 4);
await o.AppendBatchAsync([a, b, c, d], TestContext.Current.CancellationToken);
Assert.Equal(2, await o.CountAsync(TestContext.Current.CancellationToken));
Assert.Equal(2, o.DroppedCount);
var tags = (await o.PeekBatchAsync(10, TestContext.Current.CancellationToken)).Select(e => e.Tag).ToArray();
Assert.Equal(new[] { "C", "D" }, tags);
}
[Fact]
public async Task AppendBatch_empty_is_noop()
{
var dir = NewTempDir();
using var o = new FasterLogHistorizationOutbox(dir, HistorizationCommitMode.PerEntry);
await o.AppendAsync(E("A", 1), TestContext.Current.CancellationToken);
await o.AppendBatchAsync([], TestContext.Current.CancellationToken);
Assert.Equal(1, await o.CountAsync(TestContext.Current.CancellationToken));
Assert.Equal(0, o.DroppedCount);
}
[Fact]
public async Task Remove_unknown_id_is_noop()
{
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectStreamsParseTests.cs
+107
View File
@@ -294,6 +294,113 @@ public sealed class MTConnectStreamsParseTests
fromStream.Observations.ShouldBe(fromString.Observations);
}
// ---------------------------------------------------- the byte overload the /sample pump feeds on
[Theory]
[InlineData(CurrentFixture)]
[InlineData(SampleFixture)]
[InlineData(GapFixture)]
[InlineData(UnavailableFixture)]
public void Parse_from_bytes_matches_the_string_overload_on_every_fixture(string fixture)
{
// The /sample pump hands the parser the chunk's raw bytes; a chunk parsed from bytes must be
// indistinguishable from the same chunk round-tripped through a string.
var fromBytes = MTConnectStreamsParser.Parse(File.ReadAllBytes(fixture));
var fromString = MTConnectStreamsParser.Parse(File.ReadAllText(fixture));
fromBytes.InstanceId.ShouldBe(fromString.InstanceId);
fromBytes.FirstSequence.ShouldBe(fromString.FirstSequence);
fromBytes.NextSequence.ShouldBe(fromString.NextSequence);
fromBytes.Observations.ShouldBe(fromString.Observations);
}
[Theory]
[InlineData("\r\n")]
[InlineData("\n")]
[InlineData(" \t")]
[InlineData("\r\n\r\n")]
public void Parse_from_bytes_tolerates_framing_whitespace_AHEAD_of_the_xml_declaration(string residue)
{
// THE load-bearing leg of the byte path's trim, and the one that is not falsifiable by
// accident: every fixture opens with an <?xml ...?> declaration, and XmlReader rejects even
// one space ahead of one ("no whitespace characters are allowed to appear before it"). The
// string overload trimmed it; a byte path that fed the frame straight to XDocument.Load
// would turn an Agent that writes a blank line after the part headers into a hard parse
// failure on every chunk. (Trailing whitespace and a leading BOM, by contrast, XmlReader
// handles natively — the cases below pin that rather than the trim.)
var bytes = Encoding.UTF8.GetBytes(residue).Concat(File.ReadAllBytes(SampleFixture)).ToArray();
var withResidue = MTConnectStreamsParser.Parse(bytes);
withResidue.NextSequence.ShouldBe(113);
withResidue.Observations.ShouldBe(MTConnectStreamsParser.Parse(File.ReadAllText(SampleFixture)).Observations);
}
[Theory]
[InlineData(CurrentFixture)]
[InlineData(SampleFixture)]
public void Parse_from_bytes_tolerates_a_utf8_byte_order_mark_ahead_of_the_declaration(string fixture)
{
// A chunk lifted out of a multipart frame can carry a BOM; the string path trimmed ''
// for exactly this reason, so the byte path must handle the EF BB BF that decodes to it.
var bytes = new byte[] { 0xEF, 0xBB, 0xBF }.Concat(File.ReadAllBytes(fixture)).ToArray();
var withBom = MTConnectStreamsParser.Parse(bytes);
withBom.Observations.ShouldBe(MTConnectStreamsParser.Parse(File.ReadAllText(fixture)).Observations);
}
[Theory]
[InlineData("\r\n")]
[InlineData("\r\n\r\n")]
[InlineData("\n")]
[InlineData(" \t\r\n")]
public void Parse_from_bytes_tolerates_the_multipart_frames_trailing_line_break(string residue)
{
// The other half of what the string path's Trim covered: the framing's trailing line break.
var bytes = File.ReadAllBytes(SampleFixture).Concat(Encoding.UTF8.GetBytes(residue)).ToArray();
var withResidue = MTConnectStreamsParser.Parse(bytes);
withResidue.NextSequence.ShouldBe(113);
withResidue.Observations.ShouldBe(MTConnectStreamsParser.Parse(File.ReadAllText(SampleFixture)).Observations);
}
[Fact]
public void Parse_from_bytes_tolerates_a_bom_behind_leading_whitespace()
{
// Whitespace THEN a BOM defeats XmlReader's encoding sniffing outright ("Data at the root
// level is invalid"), so the trim has to interleave the two rather than strip a BOM only at
// offset 0 — which is exactly what the string overload's Trim('', ' ', …) set did.
var bytes = " \r\n"u8.ToArray()
.Concat(new byte[] { 0xEF, 0xBB, 0xBF })
.Concat(File.ReadAllBytes(CurrentFixture))
.Concat("\r\n"u8.ToArray())
.ToArray();
MTConnectStreamsParser.Parse(bytes).Observations
.ShouldBe(MTConnectStreamsParser.Parse(File.ReadAllText(CurrentFixture)).Observations);
}
[Theory]
[InlineData(new byte[0])]
[InlineData(new byte[] { (byte)' ', (byte)'\r', (byte)'\n' })]
[InlineData(new byte[] { 0xEF, 0xBB, 0xBF })]
public void Parse_from_bytes_throws_on_an_empty_payload(byte[] bytes)
{
// Same failure family as the string path: never a silently empty result.
var ex = Should.Throw<InvalidDataException>(() => MTConnectStreamsParser.Parse(bytes));
ex.Message.ShouldContain("empty");
}
[Fact]
public void Parse_from_bytes_throws_on_malformed_xml()
{
Should.Throw<InvalidDataException>(
() => MTConnectStreamsParser.Parse("not xml at all"u8.ToArray()));
}
// ------------------------------------------------------------- fails loudly, never silently empty
[Theory]
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.Tests/ModbusLifecycleHygieneTests.cs
+236
View File
@@ -218,6 +218,242 @@ public sealed class ModbusLifecycleHygieneTests
"DisposeAsync must dispose the PollGroupEngine so its background Task stops, not just the transport");
}
// -------------------- Re-initialize transport hygiene --------------------
/// <summary>
/// A second <c>InitializeAsync</c> on a live driver — the shape
/// <c>DriverInstanceActor.AdoptConfigDuringInit</c> produces when an ApplyDelta lands while the
/// driver is Connecting/Reconnecting — must dispose the transport the previous initialize
/// published before publishing a new one. Pre-fix <c>_transport</c> was simply overwritten,
/// stranding a connected transport no later dispose could reach.
/// </summary>
[Fact]
public async Task A_second_InitializeAsync_disposes_the_first_transport()
{
var first = new FakeTransport();
var second = new FakeTransport();
var handed = new List<FakeTransport>();
var opts = new ModbusDriverOptions
{
Host = "fake",
Probe = new ModbusProbeOptions { Enabled = false },
RawTags = ModbusRawTags.Entries([new ModbusTagDefinition("A", ModbusRegion.HoldingRegisters, 0, ModbusDataType.Int16)]),
};
var drv = new ModbusDriver(opts, "modbus-1", _ =>
{
var next = handed.Count == 0 ? first : second;
handed.Add(next);
return next;
});
await drv.InitializeAsync("{}", CancellationToken.None);
// No ShutdownAsync in between — exactly what AdoptConfigDuringInit does.
await drv.InitializeAsync("{}", CancellationToken.None);
handed.Count.ShouldBe(2, "each initialize must build its own transport");
first.DisposeCount.ShouldBe(1, "the first initialize's transport must be disposed by the second initialize");
second.DisposeCount.ShouldBe(0, "the live transport must not be disposed");
// …and the driver is still functional, serving reads through the SECOND transport.
drv.GetHealth().State.ShouldNotBe(DriverState.Faulted);
second.HoldingRegisters[0] = 7;
var r = await drv.ReadAsync(["A"], CancellationToken.None);
r[0].StatusCode.ShouldBe(0u);
Convert.ToInt32(r[0].Value).ShouldBe(7);
second.SendCount.ShouldBeGreaterThan(0, "reads must route through the transport the second initialize published");
}
/// <summary>
/// <c>_probeCts</c> is re-assigned by every initialize, so a second initialize must cancel the
/// previous one. <c>ProbeLoopAsync</c> CAPTURES its transport reference at loop entry, so a
/// leaked probe loop keeps hammering the transport the first initialize published — which is
/// exactly what this measures. Re-probe is off here so every send is attributable to the probe.
/// </summary>
[Fact]
public async Task A_second_InitializeAsync_stops_the_first_probe_loop()
{
var first = new FakeTransport();
var second = new FakeTransport();
var handed = 0;
var opts = new ModbusDriverOptions
{
Host = "fake",
Probe = new ModbusProbeOptions
{
Enabled = true,
Interval = TimeSpan.FromMilliseconds(25),
Timeout = TimeSpan.FromSeconds(1),
},
AutoProhibitReprobeInterval = null, // re-probe off: isolate the probe loop
RawTags = ModbusRawTags.Entries([new ModbusTagDefinition("A", ModbusRegion.HoldingRegisters, 0, ModbusDataType.Int16)]),
};
var drv = new ModbusDriver(opts, "modbus-1", _ => Interlocked.Increment(ref handed) == 1 ? first : second);
await drv.InitializeAsync("{}", CancellationToken.None);
await Task.Delay(200);
Interlocked.CompareExchange(ref first.SendCount, 0, 0).ShouldBeGreaterThan(0,
"precondition: the first initialize's probe loop must actually be sending");
await drv.InitializeAsync("{}", CancellationToken.None);
// Tolerate one in-flight send straddling the cancel; a leaked loop produces ~16 in 400ms.
var atReinit = Interlocked.CompareExchange(ref first.SendCount, 0, 0);
await Task.Delay(400);
var atRest = Interlocked.CompareExchange(ref first.SendCount, 0, 0);
(atRest - atReinit).ShouldBeLessThanOrEqualTo(1,
"the previous initialize's probe loop must be cancelled by the next initialize");
await drv.DisposeAsync();
}
/// <summary>
/// Transport for the re-probe test: counts every send and fails one address permanently, so a
/// seeded prohibition on that address is never cleared and the re-probe loop keeps re-probing
/// it tick after tick. A transport that answered normally would let the very first re-probe
/// CLEAR the prohibition, emptying the candidate set and making the test vacuous.
/// </summary>
private sealed class ReprobeCountingTransport : IModbusTransport
{
/// <summary>Register address whose reads always fail.</summary>
public ushort FailAddress { get; init; }
/// <summary>Total sends this transport has been asked to perform.</summary>
public int SendCount;
/// <summary>Establishes a connection asynchronously.</summary>
/// <param name="ct">Cancellation token for the operation.</param>
public Task ConnectAsync(CancellationToken ct) => Task.CompletedTask;
/// <summary>Counts the send, failing reads aimed at <see cref="FailAddress"/>.</summary>
/// <param name="unitId">The Modbus unit identifier.</param>
/// <param name="pdu">The Protocol Data Unit to send.</param>
/// <param name="ct">Cancellation token for the operation.</param>
public Task<byte[]> SendAsync(byte unitId, byte[] pdu, CancellationToken ct)
{
Interlocked.Increment(ref SendCount);
var addr = (ushort)((pdu[1] << 8) | pdu[2]);
if (addr == FailAddress)
return Task.FromException<byte[]>(new IOException($"address {addr} is unreadable"));
return Task.FromResult(new byte[] { pdu[0], 2, 0, 0 });
}
/// <summary>Disposes the transport asynchronously.</summary>
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
}
/// <summary>Reads a private instance field off the driver.</summary>
private static object GetPrivateField(ModbusDriver drv, string name) =>
typeof(ModbusDriver).GetField(name, BindingFlags.NonPublic | BindingFlags.Instance)!.GetValue(drv)!;
/// <summary>
/// <c>_reprobeCts</c> is re-assigned by every initialize, so a second initialize must cancel the
/// previous one. Two things make this measurable where the probe test's approach would not:
/// the re-probe loop only sends when the prohibition set is NON-EMPTY (so one is seeded, on an
/// address the transport fails permanently), and it re-reads the <c>_transport</c> FIELD every
/// pass rather than capturing it — so a leaked loop hits the transport the SECOND initialize
/// published. The second initialize is therefore given re-probe-less options: any send at all on
/// the second transport is the leaked loop from the first.
/// </summary>
[Fact]
public async Task A_second_InitializeAsync_stops_the_first_reprobe_loop()
{
const ushort prohibited = 100;
var first = new ReprobeCountingTransport { FailAddress = prohibited };
var second = new ReprobeCountingTransport { FailAddress = prohibited };
var handed = 0;
var opts = new ModbusDriverOptions
{
Host = "fake",
Probe = new ModbusProbeOptions { Enabled = false }, // probe off: isolate the re-probe loop
AutoProhibitReprobeInterval = TimeSpan.FromMilliseconds(25),
RawTags = ModbusRawTags.Entries([new ModbusTagDefinition("A", ModbusRegion.HoldingRegisters, 0, ModbusDataType.Int16)]),
};
var drv = new ModbusDriver(opts, "modbus-1", _ => Interlocked.Increment(ref handed) == 1 ? first : second);
// Seed a single-register prohibition BEFORE the first initialize, so no loop is running while
// the (unsynchronised) dictionary is mutated. Start == End keeps it off the bisection path, so
// each tick is exactly one StraightReprobeAsync send.
var prohibitions = (System.Collections.IDictionary)GetPrivateField(drv, "_autoProhibited");
var stateType = typeof(ModbusDriver).GetNestedType("ProhibitionState", BindingFlags.NonPublic)!;
prohibitions.Add(
((byte)1, ModbusRegion.HoldingRegisters, prohibited, prohibited),
Activator.CreateInstance(stateType, nonPublic: true)!);
await drv.InitializeAsync("{}", CancellationToken.None);
await Task.Delay(200);
Interlocked.CompareExchange(ref first.SendCount, 0, 0).ShouldBeGreaterThan(0,
"precondition: the seeded prohibition must make the re-probe loop actually send");
// The second initialize must NOT start a re-probe loop of its own, or a leaked first loop and a
// healthy second one would be indistinguishable by send count. "{}" keeps these options.
typeof(ModbusDriver).GetField("_options", BindingFlags.NonPublic | BindingFlags.Instance)!
.SetValue(drv, new ModbusDriverOptions
{
Host = "fake",
Probe = new ModbusProbeOptions { Enabled = false },
AutoProhibitReprobeInterval = null,
RawTags = opts.RawTags,
});
await drv.InitializeAsync("{}", CancellationToken.None);
// Non-vacuity guard: the candidate set the leaked loop would iterate must still be non-empty,
// so a zero send count is attributable to the cancel and not to an emptied prohibition set.
prohibitions.Count.ShouldBe(1, "a re-initialize must not clear the prohibition set");
var atReinit = Interlocked.CompareExchange(ref second.SendCount, 0, 0);
await Task.Delay(400);
var atRest = Interlocked.CompareExchange(ref second.SendCount, 0, 0);
(atRest - atReinit).ShouldBeLessThanOrEqualTo(1,
"the previous initialize's re-probe loop must be cancelled by the next initialize");
await drv.DisposeAsync();
}
/// <summary>
/// Two initializes racing on the same driver — the shape <c>DriverInstanceActor</c> produces by
/// dispatching initialize on a fire-and-forget <c>Task.Run</c> with <c>AdoptConfigDuringInit</c>
/// firing twice — must serialize on the lifecycle gate. Ungated, the teardown at the top of one
/// initialize disposes the transport the other just published (or nulls <c>_transport</c>
/// mid-connect), leaving the driver holding a disposed transport.
/// </summary>
[Fact]
public async Task Concurrent_InitializeAsync_calls_serialize_and_leave_exactly_one_live_transport()
{
var handed = new ConcurrentQueue<FakeTransport>();
var opts = new ModbusDriverOptions
{
Host = "fake",
Probe = new ModbusProbeOptions { Enabled = false },
RawTags = ModbusRawTags.Entries([new ModbusTagDefinition("A", ModbusRegion.HoldingRegisters, 0, ModbusDataType.Int16)]),
};
var drv = new ModbusDriver(opts, "modbus-1", _ =>
{
var t = new FakeTransport();
handed.Enqueue(t);
return t;
});
await Task.WhenAll(
Task.Run(() => drv.InitializeAsync("{}", CancellationToken.None)),
Task.Run(() => drv.InitializeAsync("{}", CancellationToken.None)));
var built = handed.ToArray();
built.Length.ShouldBe(2);
// Serialized: the earlier initialize's transport is disposed by the later one, and exactly one
// survives — never two live transports, and never a disposed transport left installed.
built.Count(t => t.DisposeCount == 0).ShouldBe(1, "exactly one transport must be left live");
built.Count(t => t.DisposeCount == 1).ShouldBe(1, "the superseded transport must be disposed once");
// The survivor is the installed one and the driver still works through it.
var live = built.Single(t => t.DisposeCount == 0);
live.HoldingRegisters[0] = 9;
var r = await drv.ReadAsync(["A"], CancellationToken.None);
r[0].StatusCode.ShouldBe(0u);
Convert.ToInt32(r[0].Value).ShouldBe(9);
await drv.DisposeAsync();
}
// -------------------- Finding -005 / -012 (3) --------------------
/// <summary>
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Tests/MqttSubscriptionManagerTests.cs
+133
View File
@@ -964,6 +964,139 @@ public sealed class MqttSubscriptionManagerTests
transport.Calls.Count.ShouldBe(1);
}
// ---------------------------------------------------------------------------------
// Unchanged-value suppression — MQTT is push-based, so the edge dedupe every poll driver
// gets from PollGroupEngine has to live here instead.
// ---------------------------------------------------------------------------------
/// <summary>
/// A broker that timer-republishes an identical payload must not re-trigger the whole
/// downstream fan-out (actor hop → mux → OPC UA sink write). The value is still <i>cached</i>
/// — the republish freshens the snapshot so <c>IReadable.ReadAsync</c> stays current — but no
/// <c>OnDataChange</c> is raised. "Cached but not raised" is an established state here; see
/// <see cref="HandleMessage_AuthoredButUnsubscribedTag_CachesValue_RaisesNothing"/>.
/// </summary>
[Fact]
public async Task An_identical_republish_raises_nothing()
{
var mgr = NewManager();
mgr.Register([Tag(OvenTempPath, $$"""{"topic":"{{OvenTopic}}","payloadFormat":"Json","jsonPath":"$.value","dataType":"Float64"}""")]);
await mgr.SubscribeAsync([OvenTempPath], TimeSpan.FromSeconds(1), TestContext.Current.CancellationToken);
var raised = new List<DataValueSnapshot>();
mgr.OnDataChange += (_, e) => raised.Add(e.Snapshot);
mgr.HandleMessage(OvenTopic, """{"value":21.5}"""u8.ToArray(), retained: false);
mgr.HandleMessage(OvenTopic, """{"value":21.5}"""u8.ToArray(), retained: false);
raised.Count.ShouldBe(1);
raised[0].Value.ShouldBe(21.5);
// The suppressed republish still WROTE the cache: the cached snapshot is the second message's
// instance, not the one that was raised. Asserting on the timestamp alone would be flaky —
// two dispatcher-thread calls can share a clock tick — while the instance identity is exact.
var cached = mgr.Values.Read(OvenTempPath);
cached.Value.ShouldBe(21.5);
cached.StatusCode.ShouldBe(Good);
cached.ShouldNotBeSameAs(raised[0]);
cached.ServerTimestampUtc.ShouldBeGreaterThanOrEqualTo(raised[0].ServerTimestampUtc);
}
/// <summary>
/// The falsification control for <see cref="An_identical_republish_raises_nothing"/>:
/// suppression must be <i>value</i>-based, not once-only. An implementation that simply
/// raised the first message per reference and went quiet passes that test and takes the
/// driver permanently dark.
/// </summary>
[Fact]
public async Task A_changed_value_after_a_suppressed_republish_raises()
{
var mgr = NewManager();
mgr.Register([Tag(OvenTempPath, $$"""{"topic":"{{OvenTopic}}","payloadFormat":"Json","jsonPath":"$.value","dataType":"Float64"}""")]);
await mgr.SubscribeAsync([OvenTempPath], TimeSpan.FromSeconds(1), TestContext.Current.CancellationToken);
var raised = new List<object?>();
mgr.OnDataChange += (_, e) => raised.Add(e.Snapshot.Value);
mgr.HandleMessage(OvenTopic, """{"value":21.5}"""u8.ToArray(), retained: false);
mgr.HandleMessage(OvenTopic, """{"value":21.5}"""u8.ToArray(), retained: false); // suppressed
mgr.HandleMessage(OvenTopic, """{"value":22.5}"""u8.ToArray(), retained: false);
mgr.HandleMessage(OvenTopic, """{"value":21.5}"""u8.ToArray(), retained: false); // back again — a real change
raised.ShouldBe([21.5, 22.5, 21.5]);
}
/// <summary>
/// Suppression compares status as well as value, so a quality transition on an otherwise
/// unchanged reference still publishes. The case that matters is
/// <c>DegradeTopic</c>: a Good value followed by a refused subscription must reach the
/// subscriber as Bad, or the client keeps a stale Good forever.
/// </summary>
/// <remarks>
/// This pins the <c>DegradeTopic</c> path specifically — <c>DegradeTopic</c>'s own
/// <c>Raise</c> call was already unconditional before the suppression change and is untouched
/// by it. It does NOT exercise the <c>prior.StatusCode == snapshot.StatusCode</c> comparison
/// added inside <see cref="MqttSubscriptionManager.Deliver"/>; see
/// <see cref="Different_malformed_payloads_with_the_same_null_value_both_raise"/> for that.
/// </remarks>
[Fact]
public async Task A_status_transition_still_raises()
{
var transport = new FakeTransport();
var mgr = NewManager(transport);
mgr.Register(
[
Tag(OvenTempPath, $$"""{"topic":"{{OvenTopic}}","payloadFormat":"Scalar","dataType":"Float64"}"""),
Tag(OvenPressPath, """{"topic":"f/oven/press","payloadFormat":"Scalar","dataType":"Float64"}"""),
]);
await mgr.SubscribeAsync([OvenTempPath, OvenPressPath], TimeSpan.FromSeconds(1), TestContext.Current.CancellationToken);
mgr.HandleMessage(OvenTopic, "21.5"u8.ToArray(), retained: false);
var raised = new List<(string Reference, uint Status)>();
mgr.OnDataChange += (_, e) => raised.Add((e.FullReference, e.Snapshot.StatusCode));
// Only the oven-temp topic is refused, so the re-subscribe is a PARTIAL rejection and does
// not throw — one topic is still granted.
transport.Reject = t => t == OvenTopic;
await mgr.OnReconnectedAsync(TestContext.Current.CancellationToken);
raised.ShouldContain((OvenTempPath, BadCommunicationError));
mgr.Values.Read(OvenTempPath).StatusCode.ShouldBe(BadCommunicationError);
}
/// <summary>
/// Pins the status-code half of <c>Deliver</c>'s suppression comparison
/// (<c>prior.StatusCode == snapshot.StatusCode</c>) through the LIVE
/// <see cref="MqttSubscriptionManager.HandleMessage"/> path, which
/// <see cref="A_status_transition_still_raises"/> does not reach (that test goes through the
/// unconditional <c>DegradeTopic</c> path instead). Two different malformed payloads on the
/// SAME topic each decode to a <c>null</c> value but a DIFFERENT Bad status code — a decode
/// failure (<c>BadDecodingError</c>) followed by a value that decodes but fails type coercion
/// (<c>BadTypeMismatch</c>). A value-only comparison (ignoring status) would see
/// <c>null == null</c> on both transitions and wrongly suppress the second.
/// </summary>
[Fact]
public async Task Different_malformed_payloads_with_the_same_null_value_both_raise()
{
var mgr = NewManager();
mgr.Register([Tag(OvenTempPath, $$"""{"topic":"{{OvenTopic}}","payloadFormat":"Json","jsonPath":"$.value","dataType":"Float64"}""")]);
await mgr.SubscribeAsync([OvenTempPath], TimeSpan.FromSeconds(1), TestContext.Current.CancellationToken);
var raised = new List<DataValueSnapshot>();
mgr.OnDataChange += (_, e) => raised.Add(e.Snapshot);
// Decode failure: not valid JSON at all.
mgr.HandleMessage(OvenTopic, "{not json"u8.ToArray(), retained: false);
// Decodes fine, but the selected value does not fit the declared Float64 type.
mgr.HandleMessage(OvenTopic, """{"value":"not-a-number"}"""u8.ToArray(), retained: false);
raised.Count.ShouldBe(2);
raised[0].StatusCode.ShouldBe(BadDecodingError);
raised[0].Value.ShouldBeNull();
raised[1].StatusCode.ShouldBe(BadTypeMismatch);
raised[1].Value.ShouldBeNull();
}
/// <summary>Records every subscribe call so dedupe / re-subscribe can be asserted without a broker.</summary>
private sealed class FakeTransport : IMqttSubscribeTransport
{
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7.Tests/S7ReadSpanPlannerTests.cs
+301
View File
@@ -0,0 +1,301 @@
using Shouldly;
using Xunit;
namespace ZB.MOM.WW.OtOpcUa.Driver.S7.Tests;
/// <summary>
/// Unit tests for <see cref="S7ReadSpanPlanner"/> — the pure grouping step that turns a batch
/// of resolved S7 tags into contiguous <c>(area, DB)</c> byte spans, one wire request each.
/// The planner does no I/O, so every rule it enforces (gap ceiling, span cap, area/DB
/// separation, Timer/Counter exclusion, never-overshoot) is provable here without a PLC.
/// </summary>
/// <remarks>
/// The single most important invariant in this file is
/// <see cref="A_span_never_extends_past_its_last_members_end"/>: a span is allowed to read
/// bytes NOBODY asked for in the middle (that is the whole point of a gap budget), but it must
/// never read past the end of its last member. Reading past the end is what could make a
/// grouped read fail (area/DB overrun) where the individual per-tag reads would have
/// succeeded — the one way this optimization could lose data the old path served.
/// </remarks>
[Trait("Category", "Unit")]
public sealed class S7ReadSpanPlannerTests
{
// ── Helpers ──────────────────────────────────────────────────────────────────────────
/// <summary>Builds one planner input row: a resolved tag paired with its parsed address.</summary>
private static (int OriginalIndex, S7TagDefinition Tag, S7ParsedAddress Addr) T(
int index, string address, S7DataType dataType, int? arrayCount = null, int stringLength = 254)
=> (index,
new S7TagDefinition($"Tag{index}", address, dataType,
Writable: false, StringLength: stringLength, ArrayCount: arrayCount),
S7AddressParser.Parse(address));
/// <summary>Plans with the shipped defaults (gap 16, cap 220).</summary>
private static S7ReadSpanPlanner.Plan Plan(
params (int OriginalIndex, S7TagDefinition Tag, S7ParsedAddress Addr)[] tags)
=> S7ReadSpanPlanner.Build(tags, maxGapBytes: 16, maxSpanBytes: 220);
// ── Merging ──────────────────────────────────────────────────────────────────────────
/// <summary>Three back-to-back words collapse into one 6-byte span with correct member offsets.</summary>
[Fact]
public void Contiguous_tags_merge_into_a_single_span()
{
var plan = Plan(
T(0, "DB1.DBW0", S7DataType.UInt16),
T(1, "DB1.DBW2", S7DataType.Int16),
T(2, "DB1.DBW4", S7DataType.UInt16));
plan.LeftoverIndexes.ShouldBeEmpty();
var span = plan.Spans.ShouldHaveSingleItem();
span.Area.ShouldBe(S7Area.DataBlock);
span.DbNumber.ShouldBe(1);
span.StartByte.ShouldBe(0);
span.LengthBytes.ShouldBe(6);
span.Members.Select(m => m.RelativeOffset).ShouldBe([0, 2, 4]);
span.Members.Select(m => m.WidthBytes).ShouldBe([2, 2, 2]);
}
/// <summary>A hole wider than the gap budget breaks the run into two independent spans.</summary>
[Fact]
public void A_gap_wider_than_the_budget_splits_the_span()
{
// 0..2 then 100..102 — a 98-byte hole, far past the 16-byte budget.
var plan = Plan(
T(0, "DB1.DBW0", S7DataType.UInt16),
T(1, "DB1.DBW100", S7DataType.UInt16));
plan.Spans.Count.ShouldBe(2);
plan.Spans[0].StartByte.ShouldBe(0);
plan.Spans[0].LengthBytes.ShouldBe(2);
plan.Spans[1].StartByte.ShouldBe(100);
plan.Spans[1].LengthBytes.ShouldBe(2);
}
/// <summary>A hole exactly equal to the budget still merges — the bound is inclusive.</summary>
[Fact]
public void A_gap_exactly_equal_to_the_budget_still_merges()
{
// First tag ends at byte 2; the second starts at 18 => a gap of exactly 16.
var plan = Plan(
T(0, "DB1.DBW0", S7DataType.UInt16),
T(1, "DB1.DBW18", S7DataType.UInt16));
var span = plan.Spans.ShouldHaveSingleItem();
span.StartByte.ShouldBe(0);
span.LengthBytes.ShouldBe(20);
}
/// <summary>A zero gap budget merges strictly-adjacent tags but nothing with a hole in it.</summary>
[Fact]
public void A_zero_gap_budget_merges_only_strictly_adjacent_tags()
{
var plan = S7ReadSpanPlanner.Build(
[T(0, "DB1.DBW0", S7DataType.UInt16),
T(1, "DB1.DBW2", S7DataType.UInt16),
T(2, "DB1.DBW6", S7DataType.UInt16)],
maxGapBytes: 0, maxSpanBytes: 220);
plan.Spans.Count.ShouldBe(2);
plan.Spans[0].StartByte.ShouldBe(0);
plan.Spans[0].LengthBytes.ShouldBe(4);
plan.Spans[1].StartByte.ShouldBe(6);
}
/// <summary>The span cap breaks a contiguous run that would otherwise merge end-to-end.</summary>
[Fact]
public void The_span_cap_splits_a_run_that_would_otherwise_merge()
{
// Eight adjacent words = 16 bytes contiguous; a 6-byte cap admits 3 words per span.
var tags = Enumerable.Range(0, 8)
.Select(i => T(i, $"DB1.DBW{i * 2}", S7DataType.UInt16))
.ToArray();
var plan = S7ReadSpanPlanner.Build(tags, maxGapBytes: 16, maxSpanBytes: 6);
plan.Spans.Count.ShouldBe(3);
plan.Spans.Select(s => s.Members.Count).ShouldBe([3, 3, 2]);
plan.Spans.ShouldAllBe(s => s.LengthBytes <= 6);
// Every tag is still served — the cap splits, it never drops.
plan.Spans.SelectMany(s => s.Members).Select(m => m.OriginalIndex)
.OrderBy(i => i).ShouldBe(Enumerable.Range(0, 8));
}
/// <summary>One tag wider than the cap still forms its own span — a tag is never split.</summary>
[Fact]
public void A_single_tag_wider_than_the_cap_forms_its_own_span()
{
// A 200-char S7 STRING occupies 202 bytes; the cap is 8.
var plan = S7ReadSpanPlanner.Build(
[T(0, "DB1.DBB0", S7DataType.String, stringLength: 200)],
maxGapBytes: 16, maxSpanBytes: 8);
var span = plan.Spans.ShouldHaveSingleItem();
span.LengthBytes.ShouldBe(202);
span.Members.ShouldHaveSingleItem().WidthBytes.ShouldBe(202);
plan.LeftoverIndexes.ShouldBeEmpty();
}
/// <summary>An oversized span does not swallow the neighbour that follows it.</summary>
[Fact]
public void An_oversized_span_does_not_absorb_the_next_tag()
{
var plan = S7ReadSpanPlanner.Build(
[T(0, "DB1.DBB0", S7DataType.String, stringLength: 200),
T(1, "DB1.DBW202", S7DataType.UInt16)],
maxGapBytes: 16, maxSpanBytes: 8);
plan.Spans.Count.ShouldBe(2);
plan.Spans[0].LengthBytes.ShouldBe(202);
plan.Spans[1].StartByte.ShouldBe(202);
}
// ── Separation ───────────────────────────────────────────────────────────────────────
/// <summary>Two DBs at the same byte offsets are different address spaces and never merge.</summary>
[Fact]
public void Distinct_data_blocks_never_merge()
{
var plan = Plan(
T(0, "DB1.DBW0", S7DataType.UInt16),
T(1, "DB2.DBW2", S7DataType.UInt16));
plan.Spans.Count.ShouldBe(2);
plan.Spans.Select(s => s.DbNumber).OrderBy(n => n).ShouldBe([1, 2]);
}
/// <summary>Different areas (M vs I vs Q) are different address spaces and never merge.</summary>
[Fact]
public void Distinct_areas_never_merge()
{
var plan = Plan(
T(0, "MW0", S7DataType.UInt16),
T(1, "IW2", S7DataType.UInt16),
T(2, "QW4", S7DataType.UInt16));
plan.Spans.Count.ShouldBe(3);
plan.Spans.Select(s => s.Area).ShouldBe([S7Area.Memory, S7Area.Input, S7Area.Output], ignoreOrder: true);
}
/// <summary>
/// Timer and Counter tags are excluded from every span. Their <c>ByteOffset</c> holds the
/// DEVICE NUMBER, not a byte address, so grouping them with byte-addressed tags — or with
/// each other — would read from a completely unrelated place on the wire.
/// </summary>
[Fact]
public void Timer_and_Counter_tags_are_excluded_to_leftovers()
{
var plan = Plan(
T(0, "DB1.DBW0", S7DataType.UInt16),
T(1, "T1", S7DataType.Float64),
T(2, "C1", S7DataType.Int32),
T(3, "DB1.DBW2", S7DataType.UInt16));
plan.LeftoverIndexes.OrderBy(i => i).ShouldBe([1, 2]);
var span = plan.Spans.ShouldHaveSingleItem();
span.Members.Select(m => m.OriginalIndex).ShouldBe([0, 3]);
}
// ── Overlap ──────────────────────────────────────────────────────────────────────────
/// <summary>
/// Two Bools at different bit positions in the SAME byte cost one byte between them —
/// overlap is a gap of zero, and the span covers exactly that one byte.
/// </summary>
[Fact]
public void Two_bools_in_the_same_byte_merge_into_one_single_byte_span()
{
var plan = Plan(
T(0, "DB1.DBX0.0", S7DataType.Bool),
T(1, "DB1.DBX0.7", S7DataType.Bool));
var span = plan.Spans.ShouldHaveSingleItem();
span.StartByte.ShouldBe(0);
span.LengthBytes.ShouldBe(1);
span.Members.Count.ShouldBe(2);
span.Members.ShouldAllBe(m => m.RelativeOffset == 0);
span.Members.Select(m => m.Addr.BitOffset).ShouldBe([0, 7]);
}
/// <summary>A word overlapping an already-open byte does not shrink the span's end.</summary>
[Fact]
public void An_overlapping_tag_extends_the_span_to_the_furthest_end()
{
// DBD0 covers 0..4; DBW2 sits INSIDE it. The span must stay 4 bytes, not shrink to 2+2.
var plan = Plan(
T(0, "DB1.DBD0", S7DataType.UInt32),
T(1, "DB1.DBW2", S7DataType.UInt16));
var span = plan.Spans.ShouldHaveSingleItem();
span.StartByte.ShouldBe(0);
span.LengthBytes.ShouldBe(4);
span.Members.Select(m => m.RelativeOffset).ShouldBe([0, 2]);
}
// ── The never-overshoot invariant ────────────────────────────────────────────────────
/// <summary>
/// Across a deliberately messy mixed batch, every span starts at its first member's byte
/// and ends at its furthest member's end — never one byte further. A span that overshot
/// could fault (area / DB overrun) on a read the per-tag path would have served.
/// </summary>
[Fact]
public void A_span_never_extends_past_its_last_members_end()
{
var plan = Plan(
T(0, "DB1.DBX0.3", S7DataType.Bool),
T(1, "DB1.DBB1", S7DataType.Byte),
T(2, "DB1.DBW2", S7DataType.UInt16),
T(3, "DB1.DBD8", S7DataType.Float32),
T(4, "DB1.DBB24", S7DataType.Int64),
T(5, "DB1.DBB36", S7DataType.String, stringLength: 10),
T(6, "DB1.DBW60", S7DataType.UInt16, arrayCount: 4),
T(7, "DB1.DBW200", S7DataType.UInt16),
T(8, "MW0", S7DataType.UInt16));
plan.Spans.ShouldNotBeEmpty();
foreach (var span in plan.Spans)
{
span.Members.ShouldNotBeEmpty();
span.StartByte.ShouldBe(span.Members.Min(m => m.Addr.ByteOffset));
span.LengthBytes.ShouldBe(span.Members.Max(m => m.RelativeOffset + m.WidthBytes));
// Every member must be wholly inside the block the span will read.
span.Members.ShouldAllBe(m => m.RelativeOffset >= 0
&& m.RelativeOffset + m.WidthBytes <= span.LengthBytes);
// RelativeOffset is the member's own byte, expressed against the span start.
span.Members.ShouldAllBe(m => m.RelativeOffset == m.Addr.ByteOffset - span.StartByte);
}
// Nothing is lost or duplicated between spans and leftovers.
plan.Spans.SelectMany(s => s.Members).Select(m => m.OriginalIndex)
.Concat(plan.LeftoverIndexes)
.OrderBy(i => i)
.ShouldBe(Enumerable.Range(0, 9));
}
// ── The off switch ───────────────────────────────────────────────────────────────────
/// <summary>A negative gap budget disables grouping entirely — every tag becomes a leftover.</summary>
[Fact]
public void A_negative_max_gap_disables_grouping_entirely()
{
var plan = S7ReadSpanPlanner.Build(
[T(0, "DB1.DBW0", S7DataType.UInt16),
T(1, "DB1.DBW2", S7DataType.UInt16),
T(2, "T1", S7DataType.Float64)],
maxGapBytes: -1, maxSpanBytes: 220);
plan.Spans.ShouldBeEmpty();
plan.LeftoverIndexes.OrderBy(i => i).ShouldBe([0, 1, 2]);
}
/// <summary>An empty batch plans nothing rather than throwing.</summary>
[Fact]
public void An_empty_batch_produces_an_empty_plan()
{
var plan = Plan();
plan.Spans.ShouldBeEmpty();
plan.LeftoverIndexes.ShouldBeEmpty();
}
}
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7.Tests/S7SpanCodecTests.cs
+220
View File
@@ -0,0 +1,220 @@
using Shouldly;
using Xunit;
namespace ZB.MOM.WW.OtOpcUa.Driver.S7.Tests;
/// <summary>
/// Unit tests for the pure narrow-scalar SPAN codec —
/// <see cref="S7Driver.DecodeNarrowScalar"/> and <see cref="S7Driver.NarrowScalarWidth"/>.
/// These decode a Bit/Byte/Word/DWord scalar out of a contiguous big-endian byte span (the
/// block a future span-planner will read once per group with a single
/// <c>Plc.ReadBytesAsync</c>) instead of issuing one string <c>Plc.ReadAsync</c> per tag.
/// The requirement is <b>byte-for-byte parity with the string-read path</b>, so every arm is
/// also asserted against <see cref="S7Driver.ReinterpretRawValue"/> fed the exact boxed value
/// S7.Net's string read would have returned for the same bytes (bool / byte / ushort / uint).
/// Wide types (Int64/UInt64/Float64/String/DateTime) and the Timer/Counter areas are NOT part
/// of the narrow path — they route through <see cref="S7Driver.DecodeScalarBlock"/> — so they
/// are only asserted to fail loudly here.
/// </summary>
[Trait("Category", "Unit")]
public sealed class S7SpanCodecTests
{
// ── Helpers ──────────────────────────────────────────────────────────────────────────
// The address string is only used in error messages; the decode routes on DataType + Size.
private static S7TagDefinition Tag(S7DataType dt) => new("SpanTag", "DB1.DBW0", dt);
private static S7ParsedAddress Addr(S7Size size, int bitOffset = 0) =>
new(S7Area.DataBlock, DbNumber: 1, size, ByteOffset: 0, BitOffset: bitOffset);
// ── NarrowScalarWidth ────────────────────────────────────────────────────────────────
/// <summary>Width per access size — Bit/Byte = 1, Word = 2, DWord = 4 (matches ElementByteSize).</summary>
[Theory]
[InlineData(S7Size.Bit, 1)]
[InlineData(S7Size.Byte, 1)]
[InlineData(S7Size.Word, 2)]
[InlineData(S7Size.DWord, 4)]
public void NarrowScalarWidth_matches_the_access_size(S7Size size, int expected)
=> S7Driver.NarrowScalarWidth(Addr(size)).ShouldBe(expected);
/// <summary>The span width agrees with the array-element width for every narrow size.</summary>
[Theory]
[InlineData(S7Size.Bit)]
[InlineData(S7Size.Byte)]
[InlineData(S7Size.Word)]
[InlineData(S7Size.DWord)]
public void NarrowScalarWidth_agrees_with_ElementByteSize(S7Size size)
=> S7Driver.NarrowScalarWidth(Addr(size)).ShouldBe(S7Driver.ElementByteSize(size));
// ── DecodeNarrowScalar: the type × bytes matrix ──────────────────────────────────────
/// <summary>
/// Known big-endian byte patterns decode to the exact CLR value AND the exact CLR box type.
/// The runtime type is asserted SEPARATELY and deliberately: Shouldly's <c>ShouldBe</c>
/// coerces across numeric types, so an <c>Int16</c> surfaced as a boxed <c>int</c> passes
/// the value assertion alone (verified by falsifying the Int16 arm) — and the box type is
/// the contract, since the publish pipeline and the OPC UA variable's DataType key off it.
/// Covers both sign edges of every signed width.
/// </summary>
[Theory]
// Bool — bit extracted at BitOffset, set and clear, at both ends of the byte.
[InlineData(new byte[] { 0x01 }, S7DataType.Bool, S7Size.Bit, 0, true)]
[InlineData(new byte[] { 0xFE }, S7DataType.Bool, S7Size.Bit, 0, false)]
[InlineData(new byte[] { 0x80 }, S7DataType.Bool, S7Size.Bit, 7, true)]
[InlineData(new byte[] { 0x7F }, S7DataType.Bool, S7Size.Bit, 7, false)]
[InlineData(new byte[] { 0x08 }, S7DataType.Bool, S7Size.Bit, 3, true)]
[InlineData(new byte[] { 0xF7 }, S7DataType.Bool, S7Size.Bit, 3, false)]
[InlineData(new byte[] { 0xFF }, S7DataType.Bool, S7Size.Bit, 5, true)]
[InlineData(new byte[] { 0x00 }, S7DataType.Bool, S7Size.Bit, 0, false)]
// Byte — the whole first byte, unsigned.
[InlineData(new byte[] { 0x00 }, S7DataType.Byte, S7Size.Byte, 0, (byte)0)]
[InlineData(new byte[] { 0xAB }, S7DataType.Byte, S7Size.Byte, 0, (byte)0xAB)]
[InlineData(new byte[] { 0xFF }, S7DataType.Byte, S7Size.Byte, 0, (byte)0xFF)]
// UInt16 / Int16 — the SAME big-endian word, two semantic types (the sign edge).
[InlineData(new byte[] { 0x80, 0x00 }, S7DataType.UInt16, S7Size.Word, 0, (ushort)32768)]
[InlineData(new byte[] { 0x80, 0x00 }, S7DataType.Int16, S7Size.Word, 0, (short)-32768)]
[InlineData(new byte[] { 0x7F, 0xFF }, S7DataType.UInt16, S7Size.Word, 0, (ushort)32767)]
[InlineData(new byte[] { 0x7F, 0xFF }, S7DataType.Int16, S7Size.Word, 0, (short)32767)]
[InlineData(new byte[] { 0xFF, 0xFF }, S7DataType.UInt16, S7Size.Word, 0, (ushort)65535)]
[InlineData(new byte[] { 0xFF, 0xFF }, S7DataType.Int16, S7Size.Word, 0, (short)-1)]
[InlineData(new byte[] { 0x01, 0x02 }, S7DataType.UInt16, S7Size.Word, 0, (ushort)0x0102)]
[InlineData(new byte[] { 0x01, 0x02 }, S7DataType.Int16, S7Size.Word, 0, (short)0x0102)]
// UInt32 / Int32 — the SAME big-endian dword, two semantic types (the sign edge).
[InlineData(new byte[] { 0x80, 0x00, 0x00, 0x00 }, S7DataType.UInt32, S7Size.DWord, 0, 2147483648u)]
[InlineData(new byte[] { 0x80, 0x00, 0x00, 0x00 }, S7DataType.Int32, S7Size.DWord, 0, int.MinValue)]
[InlineData(new byte[] { 0x7F, 0xFF, 0xFF, 0xFF }, S7DataType.UInt32, S7Size.DWord, 0, 2147483647u)]
[InlineData(new byte[] { 0x7F, 0xFF, 0xFF, 0xFF }, S7DataType.Int32, S7Size.DWord, 0, int.MaxValue)]
[InlineData(new byte[] { 0xFF, 0xFF, 0xFF, 0xFF }, S7DataType.UInt32, S7Size.DWord, 0, 4294967295u)]
[InlineData(new byte[] { 0xFF, 0xFF, 0xFF, 0xFF }, S7DataType.Int32, S7Size.DWord, 0, -1)]
[InlineData(new byte[] { 0x00, 0x00, 0x01, 0x02 }, S7DataType.UInt32, S7Size.DWord, 0, 258u)]
[InlineData(new byte[] { 0x00, 0x00, 0x01, 0x02 }, S7DataType.Int32, S7Size.DWord, 0, 258)]
// Float32 — IEEE-754 single from the big-endian dword.
[InlineData(new byte[] { 0x42, 0x28, 0x00, 0x00 }, S7DataType.Float32, S7Size.DWord, 0, 42.0f)]
[InlineData(new byte[] { 0xC2, 0x28, 0x00, 0x00 }, S7DataType.Float32, S7Size.DWord, 0, -42.0f)]
[InlineData(new byte[] { 0x00, 0x00, 0x00, 0x00 }, S7DataType.Float32, S7Size.DWord, 0, 0.0f)]
[InlineData(new byte[] { 0x3F, 0x80, 0x00, 0x00 }, S7DataType.Float32, S7Size.DWord, 0, 1.0f)]
public void DecodeNarrowScalar_decodes_known_big_endian_patterns(
byte[] bytes, S7DataType dt, S7Size size, int bitOffset, object expected)
{
var actual = S7Driver.DecodeNarrowScalar(Tag(dt), Addr(size, bitOffset), bytes);
actual.ShouldBe(expected);
actual.GetType().ShouldBe(expected.GetType());
}
/// <summary>
/// PARITY: the span decode returns exactly what the string-read path
/// (<see cref="S7Driver.ReinterpretRawValue"/>) returns for the same bytes, fed the boxed
/// unsigned value S7.Net's <c>ReadAsync</c> would have produced (DBX=bool, DBB=byte,
/// DBW=ushort, DBD=uint). This is the requirement the span planner rests on.
/// </summary>
[Theory]
[InlineData(new byte[] { 0x80 }, S7DataType.Bool, S7Size.Bit, 7)]
[InlineData(new byte[] { 0x7F }, S7DataType.Bool, S7Size.Bit, 7)]
[InlineData(new byte[] { 0x01 }, S7DataType.Bool, S7Size.Bit, 0)]
[InlineData(new byte[] { 0xAB }, S7DataType.Byte, S7Size.Byte, 0)]
[InlineData(new byte[] { 0x80, 0x00 }, S7DataType.UInt16, S7Size.Word, 0)]
[InlineData(new byte[] { 0x80, 0x00 }, S7DataType.Int16, S7Size.Word, 0)]
[InlineData(new byte[] { 0xFF, 0xFF }, S7DataType.Int16, S7Size.Word, 0)]
[InlineData(new byte[] { 0x80, 0x00, 0x00, 0x00 }, S7DataType.UInt32, S7Size.DWord, 0)]
[InlineData(new byte[] { 0x80, 0x00, 0x00, 0x00 }, S7DataType.Int32, S7Size.DWord, 0)]
[InlineData(new byte[] { 0x42, 0x28, 0x00, 0x00 }, S7DataType.Float32, S7Size.DWord, 0)]
[InlineData(new byte[] { 0xC2, 0x28, 0x00, 0x00 }, S7DataType.Float32, S7Size.DWord, 0)]
public void DecodeNarrowScalar_matches_the_string_read_path(
byte[] bytes, S7DataType dt, S7Size size, int bitOffset)
{
var tag = Tag(dt);
var addr = Addr(size, bitOffset);
// What S7.Net's string ReadAsync would have boxed for these exact wire bytes.
object raw = size switch
{
S7Size.Bit => (bytes[0] & (1 << bitOffset)) != 0,
S7Size.Byte => bytes[0],
S7Size.Word => (ushort)((bytes[0] << 8) | bytes[1]),
_ => ((uint)bytes[0] << 24) | ((uint)bytes[1] << 16) | ((uint)bytes[2] << 8) | bytes[3],
};
var fromSpan = S7Driver.DecodeNarrowScalar(tag, addr, bytes);
var fromString = S7Driver.ReinterpretRawValue(tag, addr, raw);
fromSpan.ShouldBe(fromString);
// Value parity is not enough — ShouldBe coerces numerics, so the box type is asserted too
// (an Int16 tag published as a boxed int would change its OPC UA DataType).
fromSpan.GetType().ShouldBe(fromString.GetType());
}
// ── Span positioning ─────────────────────────────────────────────────────────────────
/// <summary>
/// The slice is positioned at the tag's own first byte and only the leading
/// <see cref="S7Driver.NarrowScalarWidth"/> bytes are consumed — trailing bytes belonging
/// to the NEXT tag in the block must not leak into the value.
/// </summary>
[Fact]
public void DecodeNarrowScalar_reads_only_the_leading_width_of_a_longer_slice()
{
// A 6-byte "block": [word 0x0102][dword 0x7FFFFFFF trailing garbage after it].
byte[] block = [0x01, 0x02, 0x7F, 0xFF, 0xFF, 0xFF];
S7Driver.DecodeNarrowScalar(Tag(S7DataType.Int16), Addr(S7Size.Word), block)
.ShouldBe((short)0x0102);
S7Driver.DecodeNarrowScalar(Tag(S7DataType.Byte), Addr(S7Size.Byte), block)
.ShouldBe((byte)0x01);
S7Driver.DecodeNarrowScalar(Tag(S7DataType.Bool), Addr(S7Size.Bit, 0), block)
.ShouldBe(true);
}
/// <summary>
/// A tag mid-block decodes from its own offset slice — the caller positions the span, the
/// codec never re-applies <c>addr.ByteOffset</c>.
/// </summary>
[Fact]
public void DecodeNarrowScalar_decodes_from_a_mid_block_slice()
{
byte[] block = [0xDE, 0xAD, 0x42, 0x28, 0x00, 0x00, 0xBE, 0xEF];
// The Float32 lives at block byte 2 — the address's own ByteOffset is irrelevant here.
var addr = new S7ParsedAddress(S7Area.DataBlock, DbNumber: 1, S7Size.DWord, ByteOffset: 2, BitOffset: 0);
S7Driver.DecodeNarrowScalar(Tag(S7DataType.Float32), addr, block.AsSpan(2))
.ShouldBeOfType<float>().ShouldBe(42.0f);
}
// ── Mismatches fail loudly ───────────────────────────────────────────────────────────
/// <summary>
/// A declared type that does not match the parsed access size is a config error, not a
/// silent mis-decode (mirrors <see cref="S7Driver.ReinterpretRawValue"/>'s default arm).
/// </summary>
[Theory]
[InlineData(S7DataType.Int16, S7Size.DWord)]
[InlineData(S7DataType.Float32, S7Size.Word)]
[InlineData(S7DataType.Bool, S7Size.Byte)]
[InlineData(S7DataType.Byte, S7Size.Word)]
public void DecodeNarrowScalar_rejects_a_type_size_mismatch(S7DataType dt, S7Size size)
{
byte[] bytes = [0x00, 0x00, 0x00, 0x00];
Should.Throw<System.IO.InvalidDataException>(
() => S7Driver.DecodeNarrowScalar(Tag(dt), Addr(size), bytes));
}
/// <summary>
/// Wide types never belong on the narrow span path (they route through
/// <see cref="S7Driver.DecodeScalarBlock"/>); a mis-routed one must fail here rather than
/// decode garbage from a truncated slice.
/// </summary>
[Theory]
[InlineData(S7DataType.Int64)]
[InlineData(S7DataType.UInt64)]
[InlineData(S7DataType.Float64)]
[InlineData(S7DataType.String)]
[InlineData(S7DataType.DateTime)]
public void DecodeNarrowScalar_rejects_wide_types(S7DataType dt)
{
byte[] bytes = [0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00];
Should.Throw<System.IO.InvalidDataException>(
() => S7Driver.DecodeNarrowScalar(Tag(dt), Addr(S7Size.DWord), bytes));
}
}
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7.Tests/S7SpanReadParityTests.cs
+373
View File
@@ -0,0 +1,373 @@
using S7.Net;
using Shouldly;
using Xunit;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using S7NetDataType = global::S7.Net.DataType;
namespace ZB.MOM.WW.OtOpcUa.Driver.S7.Tests;
/// <summary>
/// Falsification gate for span-grouped S7 reads. Every tag must decode to the SAME value, the
/// same CLR box type and the same status code whether it was served by a wire request of its
/// own (the pre-existing per-tag path, reachable by setting
/// <c>SpanReadMaxGapBytes = -1</c>) or by a block read shared with its neighbours.
/// </summary>
/// <remarks>
/// <para>
/// The two drivers read the same in-memory PLC image through the same fake, so any
/// difference between the two result sets is a defect in the grouping path — there is no
/// other variable. Grouping is proven to have actually engaged by
/// <see cref="Grouping_issues_far_fewer_wire_requests_than_there_are_tags"/>: without that
/// assertion this whole file would pass vacuously if the planner ever produced only
/// leftovers, because "leftover" IS the per-tag path.
/// </para>
/// <para>
/// The fake serves <c>ReadBytesAsync</c> (block reads — the span path, plus the wide /
/// array / Timer-Counter tags that always read blocks) and <c>ReadAsync(string)</c> (the
/// narrow per-tag path) from ONE byte image per <c>(area, DB)</c>, decoding the latter
/// exactly as S7.Net does: <c>DBX</c> boxes a <c>bool</c>, <c>DBB</c> a <c>byte</c>,
/// <c>DBW</c> a <c>ushort</c>, <c>DBD</c> a <c>uint</c>.
/// </para>
/// </remarks>
[Trait("Category", "Unit")]
public sealed class S7SpanReadParityTests
{
/// <summary>OPC UA <c>BadDeviceFailure</c> — what a device-layer read fault stamps.</summary>
private const uint StatusBadDeviceFailure = 0x808B0000u;
/// <summary>OPC UA <c>BadNodeIdUnknown</c> — what an unresolvable reference stamps.</summary>
private const uint StatusBadNodeIdUnknown = 0x80340000u;
// ── The mixed tag set ────────────────────────────────────────────────────────────────
/// <summary>
/// A deliberately awkward batch: two Bools sharing one byte (bit 0 and bit 7), a Byte, a
/// word pair, three dwords behind a small hole, a wide Int64, an S7 STRING, a UInt16 array,
/// a far-away word past the gap budget, a second AREA, and a Counter (whose offset is a
/// device number, so it can never join a byte span).
/// </summary>
private static IReadOnlyList<RawTagEntry> MixedTags() => S7RawTags.Entries(
new S7TagDefinition("Bit0", "DB1.DBX0.0", S7DataType.Bool, Writable: false),
new S7TagDefinition("Bit7", "DB1.DBX0.7", S7DataType.Bool, Writable: false),
new S7TagDefinition("Byte1", "DB1.DBB1", S7DataType.Byte, Writable: false),
new S7TagDefinition("U16", "DB1.DBW2", S7DataType.UInt16, Writable: false),
new S7TagDefinition("I16", "DB1.DBW4", S7DataType.Int16, Writable: false),
new S7TagDefinition("U32", "DB1.DBD8", S7DataType.UInt32, Writable: false),
new S7TagDefinition("I32", "DB1.DBD12", S7DataType.Int32, Writable: false),
new S7TagDefinition("F32", "DB1.DBD16", S7DataType.Float32, Writable: false),
new S7TagDefinition("I64", "DB1.DBB24", S7DataType.Int64, Writable: false),
new S7TagDefinition("Str", "DB1.DBB36", S7DataType.String, Writable: false, StringLength: 10),
new S7TagDefinition("Arr", "DB1.DBW60", S7DataType.UInt16, Writable: false, ArrayCount: 4),
new S7TagDefinition("Far", "DB1.DBW200", S7DataType.UInt16, Writable: false),
new S7TagDefinition("Mem", "MW0", S7DataType.UInt16, Writable: false),
new S7TagDefinition("Cnt", "C1", S7DataType.Int32, Writable: false));
/// <summary>Read references, in the order the batch presents them.</summary>
private static readonly string[] MixedRefs =
[
"Bit0", "Bit7", "Byte1", "U16", "I16", "U32", "I32", "F32",
"Str", "I64", "Arr", "Far", "Mem", "Cnt",
];
/// <summary>Only the narrow, byte-addressed DB1 tags — every one of them lands in one span.</summary>
private static IReadOnlyList<RawTagEntry> NarrowTags() => S7RawTags.Entries(
new S7TagDefinition("A", "DB1.DBW0", S7DataType.UInt16, Writable: false),
new S7TagDefinition("B", "DB1.DBW2", S7DataType.UInt16, Writable: false),
new S7TagDefinition("C", "DB1.DBW4", S7DataType.UInt16, Writable: false));
// ── Parity ───────────────────────────────────────────────────────────────────────────
/// <summary>
/// THE gate: span-grouped reads and per-tag reads produce identical snapshots — same
/// value, same box type, same status — for every tag in the mixed batch.
/// </summary>
[Fact]
public async Task Span_grouped_reads_match_the_per_tag_path_snapshot_for_snapshot()
{
var images = Images();
var ct = TestContext.Current.CancellationToken;
var grouping = new ImageS7PlcFactory(images);
var perTag = new ImageS7PlcFactory(images);
using var groupedDriver = new S7Driver(Options(MixedTags(), maxGap: 16), "s7-grouped", grouping);
using var perTagDriver = new S7Driver(Options(MixedTags(), maxGap: -1), "s7-per-tag", perTag);
await groupedDriver.InitializeAsync("{}", ct);
await perTagDriver.InitializeAsync("{}", ct);
var grouped = await groupedDriver.ReadAsync(MixedRefs, ct);
var individual = await perTagDriver.ReadAsync(MixedRefs, ct);
grouped.Count.ShouldBe(MixedRefs.Length);
for (var i = 0; i < MixedRefs.Length; i++)
{
var name = MixedRefs[i];
grouped[i].StatusCode.ShouldBe(individual[i].StatusCode, $"status differs for '{name}'");
grouped[i].StatusCode.ShouldBe(0u, $"'{name}' should have read Good");
grouped[i].Value.ShouldNotBeNull($"'{name}' returned no value");
grouped[i].Value!.GetType().ShouldBe(individual[i].Value!.GetType(), $"box type differs for '{name}'");
grouped[i].Value.ShouldBe(individual[i].Value, $"value differs for '{name}'");
}
// The per-tag control really did take the per-tag path (no grouping happened there): its
// only block reads are the four tags that ALWAYS read blocks (Int64, String, array, Counter).
perTag.ReadBytesCalls.ShouldBe(4);
perTag.StringReadCalls.ShouldBe(10);
}
/// <summary>
/// Proves grouping actually engaged. Without this the parity test above would pass
/// vacuously if the planner degenerated to all-leftovers, since a leftover IS a per-tag
/// read.
/// </summary>
/// <remarks>
/// 14 tags collapse to 4 wire requests: ONE block read covering the eleven DB1 tags from
/// byte 0 to byte 68, the Counter's own unavoidable block read, and two per-address string
/// reads — the far DB1 word (past the gap budget) and the lone Memory word, each of which
/// the driver deliberately leaves on the per-tag path because a single-member narrow span
/// saves no round-trip.
/// </remarks>
[Fact]
public async Task Grouping_issues_far_fewer_wire_requests_than_there_are_tags()
{
var ct = TestContext.Current.CancellationToken;
var factory = new ImageS7PlcFactory(Images());
using var driver = new S7Driver(Options(MixedTags(), maxGap: 16), "s7-wire-count", factory);
await driver.InitializeAsync("{}", ct);
_ = await driver.ReadAsync(MixedRefs, ct);
(factory.ReadBytesCalls + factory.StringReadCalls).ShouldBeLessThan(MixedRefs.Length);
factory.ReadBytesCalls.ShouldBeLessThan(MixedRefs.Length);
factory.ReadBytesCalls.ShouldBe(2);
// The nine narrow tags inside the big span issue NO string reads — only the two tags that
// ended up alone in a span of their own do.
factory.StringReadCalls.ShouldBe(2);
}
// ── Failure parity ───────────────────────────────────────────────────────────────────
/// <summary>
/// A failed span stamps EVERY one of its members with exactly the status the per-tag path
/// would have produced for that exception, and degrades health identically. A span read is
/// one wire request serving many tags, so a fault that used to affect one tag now affects
/// all of them — they must all be told the same thing.
/// </summary>
[Fact]
public async Task A_span_read_failure_degrades_every_member_like_the_per_tag_path()
{
var ct = TestContext.Current.CancellationToken;
// ErrorCode.ReadData with a message that is NOT access-denied ("not allowed") — a genuine
// device-layer fault, which the driver maps to BadDeviceFailure + Degraded.
var fault = new PlcException(ErrorCode.ReadData, "S7 CPU reported a device fault");
var groupingFactory = new ImageS7PlcFactory(Images()) { ReadFault = fault };
var perTagFactory = new ImageS7PlcFactory(Images()) { ReadFault = fault };
using var groupedDriver = new S7Driver(Options(NarrowTags(), maxGap: 16), "s7-grouped-fail", groupingFactory);
using var perTagDriver = new S7Driver(Options(NarrowTags(), maxGap: -1), "s7-per-tag-fail", perTagFactory);
await groupedDriver.InitializeAsync("{}", ct);
await perTagDriver.InitializeAsync("{}", ct);
string[] refs = ["A", "B", "C"];
var grouped = await groupedDriver.ReadAsync(refs, ct);
var individual = await perTagDriver.ReadAsync(refs, ct);
// Every member of the failed span — not just the first — is stamped Bad.
grouped.Count.ShouldBe(3);
grouped.ShouldAllBe(s => s.StatusCode == StatusBadDeviceFailure);
grouped.ShouldAllBe(s => s.Value == null);
grouped.Select(s => s.StatusCode).ShouldBe(individual.Select(s => s.StatusCode));
groupedDriver.GetHealth().State.ShouldBe(DriverState.Degraded);
groupedDriver.GetHealth().State.ShouldBe(perTagDriver.GetHealth().State);
// One failed block read, not three — the span path fails once for the whole group.
groupingFactory.ReadBytesCalls.ShouldBe(1);
}
/// <summary>
/// Unresolved references still stamp <c>BadNodeIdUnknown</c> and never enter a span, and
/// the resolvable tags around them still read Good.
/// </summary>
[Fact]
public async Task An_unresolved_reference_is_still_BadNodeIdUnknown_under_grouping()
{
var ct = TestContext.Current.CancellationToken;
var factory = new ImageS7PlcFactory(Images());
using var driver = new S7Driver(Options(NarrowTags(), maxGap: 16), "s7-unknown", factory);
await driver.InitializeAsync("{}", ct);
var results = await driver.ReadAsync(["A", "NoSuchTag", "C"], ct);
results[0].StatusCode.ShouldBe(0u);
results[1].StatusCode.ShouldBe(StatusBadNodeIdUnknown);
results[1].Value.ShouldBeNull();
results[2].StatusCode.ShouldBe(0u);
}
// ── Configurability ──────────────────────────────────────────────────────────────────
/// <summary>
/// The grouping knobs must be reachable from a deployed driver config. <c>ParseOptions</c>
/// maps an explicit DTO rather than deserialising <see cref="S7DriverOptions"/>, so a knob
/// that is not mapped there is silently pinned to its default forever — which would leave
/// the documented "-1 disables grouping" escape hatch unreachable in production.
/// </summary>
[Fact]
public void The_span_read_knobs_bind_from_driver_config_json()
{
var configured = S7DriverFactoryExtensions.ParseOptions(
"s7-cfg",
"""{"host":"10.0.0.1","spanReadMaxGapBytes":-1,"spanReadMaxSpanBytes":64}""");
configured.SpanReadMaxGapBytes.ShouldBe(-1);
configured.SpanReadMaxSpanBytes.ShouldBe(64);
// Omitted keys keep the shipped defaults rather than collapsing to zero.
var defaulted = S7DriverFactoryExtensions.ParseOptions("s7-cfg", """{"host":"10.0.0.1"}""");
defaulted.SpanReadMaxGapBytes.ShouldBe(S7DriverOptions.DefaultSpanReadMaxGapBytes);
defaulted.SpanReadMaxSpanBytes.ShouldBe(S7DriverOptions.DefaultSpanReadMaxSpanBytes);
}
// ── Fixture ──────────────────────────────────────────────────────────────────────────
/// <summary>Driver options over the given tags with the span-grouping knobs under test.</summary>
private static S7DriverOptions Options(IReadOnlyList<RawTagEntry> tags, int maxGap) => new()
{
Host = "192.0.2.1",
Timeout = TimeSpan.FromMilliseconds(250),
Probe = new S7ProbeOptions { Enabled = false },
SpanReadMaxGapBytes = maxGap,
SpanReadMaxSpanBytes = 220,
RawTags = tags,
};
/// <summary>
/// The in-memory PLC images the fake serves, one byte array per <c>(area, DB)</c>. Filled
/// with a deterministic non-repeating pattern so no tag reads a vacuous all-zero value, then
/// seeded with a well-formed S7 STRING field at DB1 byte 36 (<c>[maxLen][curLen][chars…]</c>,
/// which <c>S7String.FromByteArray</c> validates).
/// </summary>
private static Dictionary<(S7NetDataType Area, int Db), byte[]> Images()
{
var db1 = new byte[256];
for (var i = 0; i < db1.Length; i++) db1[i] = (byte)((i * 37 + 11) & 0xFF);
db1[36] = 10; // declared max length
db1[37] = 5; // current length
var chars = "HELLO"u8.ToArray();
chars.CopyTo(db1, 38);
for (var i = 38 + chars.Length; i < 36 + 12; i++) db1[i] = 0x20; // reserved padding
var mem = new byte[64];
for (var i = 0; i < mem.Length; i++) mem[i] = (byte)((i * 53 + 7) & 0xFF);
var counters = new byte[64];
for (var i = 0; i < counters.Length; i++) counters[i] = (byte)((i * 29 + 3) & 0xFF);
return new Dictionary<(S7NetDataType, int), byte[]>
{
[(S7NetDataType.DataBlock, 1)] = db1,
[(S7NetDataType.Memory, 0)] = mem,
[(S7NetDataType.Counter, 0)] = counters,
};
}
/// <summary>
/// Fake <see cref="IS7PlcFactory"/> serving both read shapes from one shared byte image per
/// <c>(area, DB)</c>, and counting each shape so a test can prove which path ran. Counters
/// live on the FACTORY (not the connection) so a reconnect cannot reset them.
/// </summary>
private sealed class ImageS7PlcFactory(Dictionary<(S7NetDataType Area, int Db), byte[]> images) : IS7PlcFactory
{
/// <summary>The shared per-area byte images both read shapes decode from.</summary>
public Dictionary<(S7NetDataType Area, int Db), byte[]> Images { get; } = images;
/// <summary>Number of block reads issued — one per planned span, plus the always-block tags.</summary>
public int ReadBytesCalls;
/// <summary>Number of per-address string reads issued — the un-grouped narrow path.</summary>
public int StringReadCalls;
/// <summary>When set, every read throws this instead of serving the image.</summary>
public Exception? ReadFault { get; init; }
/// <inheritdoc />
public IS7Plc Create(S7CpuType cpuType, string host, int port, short rack, short slot, TimeSpan timeout)
=> new ImageS7Plc(this);
private sealed class ImageS7Plc(ImageS7PlcFactory owner) : IS7Plc
{
/// <inheritdoc />
public bool IsConnected { get; private set; }
/// <inheritdoc />
public Task OpenAsync(CancellationToken ct) { IsConnected = true; return Task.CompletedTask; }
/// <inheritdoc />
public void Close() => IsConnected = false;
/// <inheritdoc />
public Task<byte[]> ReadBytesAsync(S7NetDataType area, int db, int startByteAdr, int count, CancellationToken ct)
{
// Counted BEFORE the injected fault so a test can assert how many block reads were
// ATTEMPTED — a failure test that counted only successes would read zero.
Interlocked.Increment(ref owner.ReadBytesCalls);
if (owner.ReadFault is { } fault) throw fault;
var image = owner.Images[(area, db)];
if (startByteAdr + count > image.Length)
{
throw new PlcException(ErrorCode.ReadData,
$"fake image overrun: {area} db{db} @{startByteAdr}+{count} exceeds {image.Length}");
}
return Task.FromResult(image[startByteAdr..(startByteAdr + count)]);
}
/// <inheritdoc />
public Task<object?> ReadAsync(string address, CancellationToken ct)
{
Interlocked.Increment(ref owner.StringReadCalls);
if (owner.ReadFault is { } fault) throw fault;
var addr = S7AddressParser.Parse(address);
var image = owner.Images[(ToS7NetArea(addr.Area), addr.DbNumber)];
var at = addr.ByteOffset;
// Box exactly as S7.Net's string read does — the size suffix picks the type.
return Task.FromResult<object?>(addr.Size switch
{
S7Size.Bit => (object)((image[at] & (1 << addr.BitOffset)) != 0),
S7Size.Byte => image[at],
S7Size.Word => (ushort)((image[at] << 8) | image[at + 1]),
S7Size.DWord => ((uint)image[at] << 24) | ((uint)image[at + 1] << 16)
| ((uint)image[at + 2] << 8) | image[at + 3],
_ => throw new NotSupportedException($"fake cannot serve size {addr.Size}"),
});
}
/// <summary>Mirrors the driver's own area mapping so both read shapes hit the same image.</summary>
private static S7NetDataType ToS7NetArea(S7Area area) => area switch
{
S7Area.DataBlock => S7NetDataType.DataBlock,
S7Area.Memory => S7NetDataType.Memory,
S7Area.Input => S7NetDataType.Input,
S7Area.Output => S7NetDataType.Output,
S7Area.Timer => S7NetDataType.Timer,
S7Area.Counter => S7NetDataType.Counter,
_ => throw new NotSupportedException($"fake cannot map area {area}"),
};
/// <inheritdoc />
public Task WriteAsync(string address, object value, CancellationToken ct) => Task.CompletedTask;
/// <inheritdoc />
public Task WriteBytesAsync(S7NetDataType area, int db, int startByteAdr, byte[] value, CancellationToken ct)
=> Task.CompletedTask;
/// <inheritdoc />
public Task ReadStatusAsync(CancellationToken ct) => Task.CompletedTask;
/// <inheritdoc />
public void Dispose() => IsConnected = false;
}
}
}
tests/Server/ZB.MOM.WW.OtOpcUa.AdminUI.Tests/Authorization/PageAuthorizationGuardTests.cs
+3 -1
View File
@@ -40,7 +40,9 @@ public class PageAuthorizationGuardTests
[typeof(NodeEdit)] = AdminUiPolicies.ConfigEditor,
[typeof(NamespaceEdit)] = AdminUiPolicies.ConfigEditor,
[typeof(AclEdit)] = AdminUiPolicies.ConfigEditor,
[typeof(Deployments)] = AdminUiPolicies.ConfigEditor,
// Fully qualified for the same reason as Certificates/Hosts below: the AdminUI.Deployments
// namespace (DeploymentDriftProbe) shadows the page's simple name from here.
[typeof(global::ZB.MOM.WW.OtOpcUa.AdminUI.Components.Pages.Deployments)] = AdminUiPolicies.ConfigEditor,
[typeof(Scripts)] = AdminUiPolicies.ConfigEditor,
[typeof(ScriptEdit)] = AdminUiPolicies.ConfigEditor,
tests/Server/ZB.MOM.WW.OtOpcUa.AdminUI.Tests/Deployments/DeploymentDriftProbeTests.cs
+83
View File
@@ -0,0 +1,83 @@
using Shouldly;
using Xunit;
using ZB.MOM.WW.OtOpcUa.AdminUI.Deployments;
using ZB.MOM.WW.OtOpcUa.AdminUI.Tests.Uns;
using ZB.MOM.WW.OtOpcUa.Configuration;
using ZB.MOM.WW.OtOpcUa.Configuration.Entities;
namespace ZB.MOM.WW.OtOpcUa.AdminUI.Tests.Deployments;
/// <summary>
/// Pins the <see cref="DeploymentDriftProbe"/> watermark contract: the expensive
/// <c>ConfigComposer</c> recomposition runs only when the <c>ConfigEdit</c> audit watermark (or the
/// sealed revision hash) has moved, and it definitely still runs when it has.
/// </summary>
public sealed class DeploymentDriftProbeTests
{
private const string SealedHash = "sealed-hash-aaaa";
[Fact]
public async Task An_unchanged_watermark_reuses_the_cached_verdict()
{
var name = $"drift-{Guid.NewGuid():N}";
var factory = UnsTreeTestDb.Factory(name);
var compose = new CountingCompose(SealedHash);
var probe = new DeploymentDriftProbe(factory, compose.ComposeAsync);
var first = await probe.GetDriftAsync(SealedHash, CancellationToken.None);
var second = await probe.GetDriftAsync(SealedHash, CancellationToken.None);
// Nothing mutated between the calls, so the second call must be served from the cache.
compose.Count.ShouldBe(1);
first.ShouldBe(false);
second.ShouldBe(false);
}
[Fact]
public async Task A_new_config_edit_recomposes()
{
var name = $"drift-{Guid.NewGuid():N}";
var factory = UnsTreeTestDb.Factory(name);
var compose = new CountingCompose(SealedHash);
var probe = new DeploymentDriftProbe(factory, compose.ComposeAsync);
var before = await probe.GetDriftAsync(SealedHash, CancellationToken.None);
before.ShouldBe(false);
// An edit lands: the audit row moves the watermark, and the (faked) composer now reports a
// different revision — the probe must recompose and flip the verdict.
await using (var db = UnsTreeTestDb.CreateNamed(name))
{
db.ConfigEdits.Add(new ConfigEdit
{
EntityType = "Equipment",
EntityId = Guid.NewGuid(),
FieldsJson = "{}",
EditedBy = "test",
SourceNode = "node-1",
EditedAtUtc = DateTime.UtcNow,
});
await db.SaveChangesAsync();
}
compose.Result = "a-different-hash";
var after = await probe.GetDriftAsync(SealedHash, CancellationToken.None);
compose.Count.ShouldBe(2);
after.ShouldBe(true);
}
/// <summary>Stand-in for the <c>ConfigComposer</c> seam that counts how often it was invoked.</summary>
private sealed class CountingCompose(string result)
{
public string Result { get; set; } = result;
public int Count { get; private set; }
public Task<string> ComposeAsync(OtOpcUaConfigDbContext db)
{
Count++;
return Task.FromResult(Result);
}
}
}
tests/Server/ZB.MOM.WW.OtOpcUa.AdminUI.Tests/PollTimerTeardownOrderingTests.cs
+117
View File
@@ -0,0 +1,117 @@
using Shouldly;
using Xunit;
namespace ZB.MOM.WW.OtOpcUa.AdminUI.Tests;
/// <summary>
/// Pins the teardown ordering of the two self-polling pages (<c>/fleet</c>, <c>/hosts</c>).
/// Both schedule their refresh as <c>_ = InvokeAsync(...)</c> from a <see cref="System.Threading.Timer"/>
/// callback, so <c>await _timer.DisposeAsync()</c> awaits only the SCHEDULING of a tick, never the
/// tick itself: a query can still be running against <c>_cts.Token</c> after that await returns.
/// Three properties follow, and none of them is visible in a passing render test.
/// </summary>
/// <remarks>
/// Source scans rather than rendering assertions because the repo has no bUnit (same technique as
/// <c>HostsPanelWiringTests</c>). A scan proves the shape of the teardown, not that a real circuit
/// tears down cleanly — the fix itself is inspection-verified against the disposal race.
/// </remarks>
[Trait("Category", "Unit")]
public sealed class PollTimerTeardownOrderingTests
{
private static string ReadPageSource(string fileName)
{
var dir = new DirectoryInfo(AppContext.BaseDirectory);
while (dir is not null && !File.Exists(Path.Combine(dir.FullName, "ZB.MOM.WW.OtOpcUa.slnx")))
dir = dir.Parent;
dir.ShouldNotBeNull("could not locate the repository root (no ZB.MOM.WW.OtOpcUa.slnx above the test binary)");
return File.ReadAllText(Path.Combine(
dir!.FullName, "src", "Server", "ZB.MOM.WW.OtOpcUa.AdminUI",
"Components", "Pages", fileName));
}
/// <summary>
/// The CTS must be cancelled BEFORE the timer is drained. Reversed, a tick that is in flight
/// during teardown runs its queries to completion against a component that is already gone
/// instead of aborting on a cancelled token.
/// </summary>
[Theory]
[InlineData("Fleet.razor")]
[InlineData("Hosts.razor")]
public void The_page_cancels_its_token_source_before_draining_the_poll_timer(string page)
{
var source = ReadPageSource(page);
var cancelIndex = source.IndexOf("_cts.Cancel();", StringComparison.Ordinal);
var drainIndex = source.IndexOf("_timer.DisposeAsync();", StringComparison.Ordinal);
cancelIndex.ShouldBeGreaterThan(-1, $"{page} no longer cancels its page-lifetime CancellationTokenSource");
drainIndex.ShouldBeGreaterThan(-1, $"{page} no longer drains its poll timer on dispose");
cancelIndex.ShouldBeLessThan(drainIndex,
$"{page} must cancel before draining — a tick in flight has to observe cancellation, not " +
"run its queries out against a torn-down component");
}
/// <summary>
/// Disposing the CTS is gated on the overlap guard. Ungated it can be disposed out from under a
/// still-running tick, which converts the clean <c>OperationCanceledException</c> the cancel
/// above arranged into an <c>ObjectDisposedException</c> thrown mid-query.
/// </summary>
[Theory]
[InlineData("Fleet.razor")]
[InlineData("Hosts.razor")]
public void The_page_only_disposes_its_token_source_when_no_tick_holds_it(string page)
{
ReadPageSource(page).ShouldContain(
"if (Volatile.Read(ref _tickRunning) == 0) _cts.Dispose();",
Case.Sensitive,
customMessage:
$"{page} disposes its CancellationTokenSource without checking the overlap guard — " +
"the timer callback discards its InvokeAsync task, so DisposeAsync awaited only the " +
"scheduling of the tick and a query may still hold the token");
}
/// <summary>
/// The <c>StateHasChanged</c> in the refresh <c>finally</c> sits outside the try, so it has no
/// catch to fall into; rendering into a disposed renderer throws inside a task nobody awaits.
/// </summary>
[Theory]
[InlineData("Fleet.razor")]
[InlineData("Hosts.razor")]
public void The_page_skips_the_teardown_render_once_disposed(string page)
{
ReadPageSource(page).ShouldContain(
"if (!_disposed) StateHasChanged();",
Case.Sensitive,
customMessage:
$"{page}'s finally-block StateHasChanged is unguarded — it is outside the try, so a " +
"render against a torn-down circuit escapes into a discarded task");
}
/// <summary>
/// THE regression pin for the /hosts swallow-and-clear: an <c>ObjectDisposedException</c> raised
/// by teardown must rethrow as a clean exit, never reach the generic catch that blanks
/// <c>_nodes</c>/<c>_instances</c>. That blanking is permanent — a torn-down page never ticks
/// again to refill them — and its own comment forbids it.
/// </summary>
[Fact]
public void The_hosts_config_load_rethrows_a_disposal_race_instead_of_blanking_its_cache()
{
var source = ReadPageSource("Hosts.razor");
var rethrowIndex = source.IndexOf(
"catch (Exception ex) when (ex is OperationCanceledException or ObjectDisposedException)",
StringComparison.Ordinal);
// Anchored on the warning line, not on `_nodes = new();` — that assignment also appears as a
// field initializer far earlier in the file, which would make the ordering check vacuous.
var blankIndex = source.IndexOf(
"/hosts: failed to load driver-instance config", StringComparison.Ordinal);
rethrowIndex.ShouldBeGreaterThan(-1,
"/hosts LoadConfigAsync no longer rethrows the teardown exceptions — an ObjectDisposedException " +
"from disposing _cts under a live tick falls into the generic catch and blanks the cached config");
blankIndex.ShouldBeGreaterThan(-1, "/hosts LoadConfigAsync no longer has the swallow-and-clear fallback");
rethrowIndex.ShouldBeLessThan(blankIndex,
"the rethrow filter must precede the blanking catch, or it never runs");
}
}
tests/Server/ZB.MOM.WW.OtOpcUa.AdminUI.Tests/ScriptAnalysis/ScriptTagCatalogTests.cs
+149 -4
View File
@@ -35,18 +35,62 @@ public sealed class ScriptTagCatalogTests
private sealed class TestFactory(DbContextOptions<OtOpcUaConfigDbContext> opts)
: IDbContextFactory<OtOpcUaConfigDbContext>
{
public OtOpcUaConfigDbContext CreateDbContext() => new(opts);
/// <summary>Number of contexts handed out — one per DB scan, the thing the TTL cache elides.</summary>
private int _created;
public Task<OtOpcUaConfigDbContext> CreateDbContextAsync(CancellationToken ct = default) =>
Task.FromResult(new OtOpcUaConfigDbContext(opts));
/// <summary>Signalled the first time a context is requested (so a test can order two callers).</summary>
public TaskCompletionSource FirstRequest { get; } = new(TaskCreationOptions.RunContinuationsAsynchronously);
/// <summary>When set, every async creation awaits this before returning — lets a test pin a build open.</summary>
public TaskCompletionSource? Gate { get; set; }
public int Created => Volatile.Read(ref _created);
public OtOpcUaConfigDbContext CreateDbContext()
{
Interlocked.Increment(ref _created);
return new OtOpcUaConfigDbContext(opts);
}
public async Task<OtOpcUaConfigDbContext> CreateDbContextAsync(CancellationToken ct = default)
{
Interlocked.Increment(ref _created);
FirstRequest.TrySetResult();
if (Gate is not null) await Gate.Task.WaitAsync(ct);
return new OtOpcUaConfigDbContext(opts);
}
}
/// <summary>A hand-driven clock — the test project has no <c>FakeTimeProvider</c> package reference.</summary>
private sealed class ManualTimeProvider(DateTimeOffset start) : TimeProvider
{
private long _ticks = start.UtcTicks;
public override DateTimeOffset GetUtcNow() => new(Volatile.Read(ref _ticks), TimeSpan.Zero);
public void Advance(TimeSpan by) => Volatile.Write(ref _ticks, Volatile.Read(ref _ticks) + by.Ticks);
}
private static (ScriptTagCatalog Catalog, DbContextOptions<OtOpcUaConfigDbContext> Opts) Fresh()
{
var (catalog, opts, _, _) = FreshInstrumented();
return (catalog, opts);
}
/// <summary>
/// Same catalog as <see cref="Fresh"/>, but exposing the scan-counting factory and the hand-driven
/// clock the snapshot-cache tests drive. Each call gets its OWN
/// <see cref="ScriptTagCatalogSnapshotCache"/> so the process-wide singleton never leaks across tests.
/// </summary>
private static (ScriptTagCatalog Catalog, DbContextOptions<OtOpcUaConfigDbContext> Opts, TestFactory Factory, ManualTimeProvider Time)
FreshInstrumented()
{
var opts = new DbContextOptionsBuilder<OtOpcUaConfigDbContext>()
.UseInMemoryDatabase($"scripttagcatalog-{Guid.NewGuid():N}")
.Options;
return (new ScriptTagCatalog(new TestFactory(opts)), opts);
var factory = new TestFactory(opts);
var time = new ManualTimeProvider(new DateTimeOffset(2026, 8, 15, 12, 0, 0, TimeSpan.Zero));
return (new ScriptTagCatalog(factory, new ScriptTagCatalogSnapshotCache(time)), opts, factory, time);
}
/// <summary>
@@ -335,6 +379,107 @@ public sealed class ScriptTagCatalogTests
(await catalog.GetTagInfoAsync("Plant/Modbus/dev1/Speed", default)).ShouldNotBeNull();
}
// -----------------------------------------------------------------------
// ScriptTagCatalogSnapshotCache — the 5 s TTL in front of the six-table topology scan.
// -----------------------------------------------------------------------
// Monaco fires a completion request per keystroke inside a string literal, and each one used to
// re-scan RawFolders + DriverInstances + Devices + TagGroups + Tags + VirtualTags unfiltered.
// -----------------------------------------------------------------------
/// <summary>Two calls inside the TTL window scan the database exactly once — the whole point.</summary>
[Fact]
public async Task Two_calls_within_ttl_scan_the_db_once()
{
var (catalog, opts, factory, _) = FreshInstrumented();
Seed(opts);
var first = await catalog.GetPathsAsync(null, default);
var second = await catalog.GetPathsAsync(null, default);
factory.Created.ShouldBe(1);
second.ShouldBe(first);
second.ShouldContain("Plant/Modbus/dev1/Speed");
}
/// <summary>Past the TTL the snapshot is rebuilt, so a tag saved in the meantime becomes completable —
/// the staleness bound is 5 s, not forever.</summary>
[Fact]
public async Task An_expired_ttl_rebuilds_and_sees_new_tags()
{
var (catalog, opts, factory, time) = FreshInstrumented();
Seed(opts);
(await catalog.GetPathsAsync(null, default)).ShouldNotContain("Plant/Modbus/dev1/Pressure");
factory.Created.ShouldBe(1);
using (var db = new OtOpcUaConfigDbContext(opts))
{
db.Tags.Add(new Tag
{
TagId = "TAG-NEW", DeviceId = "DEV-1", Name = "Pressure",
DataType = "Float", AccessLevel = TagAccessLevel.Read, TagConfig = "{}",
});
db.SaveChanges();
}
time.Advance(TimeSpan.FromSeconds(6));
var after = await catalog.GetPathsAsync(null, default);
factory.Created.ShouldBe(2);
after.ShouldContain("Plant/Modbus/dev1/Pressure");
}
/// <summary>Two circuits missing at the same instant share ONE rebuild — without the gate the second
/// caller would start its own six-table scan while the first is still in flight.</summary>
[Fact]
public async Task Concurrent_misses_share_one_rebuild()
{
var (catalog, opts, factory, _) = FreshInstrumented();
Seed(opts);
var gate = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
factory.Gate = gate;
var first = Task.Run(() => catalog.GetPathsAsync(null, default));
await factory.FirstRequest.Task.WaitAsync(TimeSpan.FromSeconds(10));
var second = Task.Run(() => catalog.GetPathsAsync(null, default));
// Give the second caller ample time to reach the cache. It must be parked on the rebuild gate,
// NOT scanning: a second scan here is exactly the duplicated work this test falsifies.
for (var i = 0; i < 50 && factory.Created < 2; i++) await Task.Delay(10);
factory.Created.ShouldBe(1);
gate.SetResult();
var a = await first.WaitAsync(TimeSpan.FromSeconds(10));
var b = await second.WaitAsync(TimeSpan.FromSeconds(10));
factory.Created.ShouldBe(1);
a.ShouldContain("Plant/Modbus/dev1/Speed");
b.ShouldBe(a);
}
/// <summary>A failed rebuild propagates and leaves the previous snapshot intact — a transient DB blip
/// must not poison the cache into serving nothing.</summary>
[Fact]
public async Task A_failed_rebuild_does_not_poison_the_cache()
{
var time = new ManualTimeProvider(new DateTimeOffset(2026, 8, 15, 12, 0, 0, TimeSpan.Zero));
var cache = new ScriptTagCatalogSnapshotCache(time);
IReadOnlyList<ScriptTagInfo> good = [new ScriptTagInfo("A/B", "Tag", "Float", "DRV-1")];
(await cache.GetOrBuildAsync(_ => Task.FromResult(good), default)).ShouldBe(good);
time.Advance(TimeSpan.FromSeconds(6));
await Should.ThrowAsync<InvalidOperationException>(
cache.GetOrBuildAsync(_ => Task.FromException<IReadOnlyList<ScriptTagInfo>>(new InvalidOperationException("boom")), default));
// The gate was released and the old snapshot survived, so the next build still runs and succeeds.
(await cache.GetOrBuildAsync(_ => Task.FromResult(good), default)).ShouldBe(good);
}
// -----------------------------------------------------------------------
// GetEquipmentReferenceNamesAsync — DB-backed tests (v3 reference-relative {{equip}}/<RefName>)
// -----------------------------------------------------------------------
tests/Server/ZB.MOM.WW.OtOpcUa.AdminUI.Tests/Uns/EffectiveNameGuardTests.cs
+53
View File
@@ -190,6 +190,59 @@ public sealed class EffectiveNameGuardTests
result.ShouldBeNull();
}
/// <summary>
/// The batch check is a pure round-trip saving: for every proposed name it must return exactly
/// what <see cref="EffectiveNameGuard.CheckAsync"/> returns for the FIRST colliding name in input
/// order — including the source-specific message text — across all three source kinds.
/// </summary>
[Theory]
// First collision is the no-override reference ("speed"), even though an alarm name follows it.
[InlineData(new[] { "free-a", "speed", "highpress", "temperature" }, "speed")]
// First collision is the OVERRIDE-derived reference effective name.
[InlineData(new[] { "free-a", "temperature", "speed" }, "temperature")]
// First collision is the VirtualTag.
[InlineData(new[] { "free-a", "computed", "speed" }, "computed")]
// First collision is the ScriptedAlarm.
[InlineData(new[] { "free-a", "highpress", "speed" }, "highpress")]
public async Task CheckMany_matches_CheckAsync_verdicts(string[] names, string expectedCollidingName)
{
var factory = SeedFactory();
var guard = Guard(factory);
var expected = await guard.CheckAsync(
EquipmentId, expectedCollidingName, EffectiveNameSourceKind.Reference, excludeSourceId: null);
expected.ShouldNotBeNull();
var actual = await guard.CheckManyAsync(EquipmentId, names, EffectiveNameSourceKind.Reference);
actual.ShouldBe(expected);
}
/// <summary>A batch where every proposed name is free reports no collision.</summary>
[Fact]
public async Task CheckMany_all_free_returns_null()
{
var guard = Guard(SeedFactory());
var result = await guard.CheckManyAsync(
EquipmentId,
["free-a", "free-b", "Speed" /* ordinal: capital S does not collide */, "free-c"],
EffectiveNameSourceKind.Reference);
result.ShouldBeNull();
}
/// <summary>An empty batch is trivially free (and must not throw).</summary>
[Fact]
public async Task CheckMany_empty_batch_returns_null()
{
var guard = Guard(SeedFactory());
var result = await guard.CheckManyAsync(EquipmentId, [], EffectiveNameSourceKind.Reference);
result.ShouldBeNull();
}
private sealed class NamedFactory(string name) : IDbContextFactory<OtOpcUaConfigDbContext>
{
public OtOpcUaConfigDbContext CreateDbContext() =>
tests/Server/ZB.MOM.WW.OtOpcUa.AdminUI.Tests/Uns/RawTreeServiceRenameWarningTests.cs
+80 -1
View File
@@ -43,11 +43,12 @@ public sealed class RawTreeServiceRenameWarningTests
private const string EquipmentName = "packer-1";
private const string ScriptName = "AreaCalc";
private static (RawTreeService Service, string DbName) Seeded()
private static (RawTreeService Service, string DbName) Seeded(Action<OtOpcUaConfigDbContext>? extra = null)
{
var name = $"rawwarn-{Guid.NewGuid():N}";
using var db = CreateNamed(name);
Seed(db);
extra?.Invoke(db);
return (new RawTreeService(new NamedFactory(name)), name);
}
@@ -124,6 +125,84 @@ public sealed class RawTreeServiceRenameWarningTests
result.Warnings.ShouldContain(w => w.Contains("script") && w.Contains(ScriptName));
}
[Fact]
public async Task Duplicate_old_paths_are_deduplicated_before_the_scan()
{
// Two tags share a name under the same device, so both resolve to the SAME old RawPath — the scan
// must not carry the duplicate into its predicate, and the warning must still name the script once.
var (service, dbName) = Seeded(db =>
{
db.Tags.Add(new Tag
{
TagId = "TAG-flow-dup",
DeviceId = DevAllId,
TagGroupId = null,
Name = "flow",
DataType = "Float",
AccessLevel = TagAccessLevel.Read,
TagConfig = "{}",
});
db.SaveChanges();
});
var rv = DeviceRowVersion(dbName, DevAllId);
var result = await service.RenameDeviceAsync(DevAllId, "Dev-renamed", rv);
result.Ok.ShouldBeTrue();
var scriptWarnings = result.Warnings.Where(w => w.Contains("script")).ToList();
scriptWarnings.Count.ShouldBe(1);
scriptWarnings[0].ShouldBe(
$"1 script ('{ScriptName}') references a tag beneath this node by its raw path; " +
"the reference will break unless the script is updated.");
}
[Fact]
public async Task A_rename_with_more_paths_than_one_batch_still_finds_matches_in_every_batch()
{
// 401 affected tags → three scan batches at the 200-term cap. One script names a path in the first
// batch, one in the middle, one in the last: a loop that drops any batch loses a name.
const int bulkTags = 401;
var (service, dbName) = Seeded(db =>
{
db.Devices.Add(new Device { DeviceId = "DEV-bulk", DriverInstanceId = DriverId, Name = "Dev-bulk", DeviceConfig = "{}" });
for (var i = 0; i < bulkTags; i++)
{
db.Tags.Add(new Tag
{
TagId = $"TAG-bulk-{i:D3}",
DeviceId = "DEV-bulk",
TagGroupId = null,
Name = $"bulk-{i:D3}",
DataType = "Float",
AccessLevel = TagAccessLevel.Read,
TagConfig = "{}",
});
}
foreach (var (scriptName, index) in new[] { ("S-first", 0), ("S-middle", 200), ("S-last", bulkTags - 1) })
{
db.Scripts.Add(new Script
{
ScriptId = $"SC-{scriptName}",
Name = scriptName,
SourceCode = $"var v = ctx.GetTag(\"Plant/modbus-1/Dev-bulk/bulk-{index:D3}\"); return v;",
SourceHash = $"hash-{scriptName}",
});
}
db.SaveChanges();
});
var rv = DeviceRowVersion(dbName, "DEV-bulk");
var result = await service.RenameDeviceAsync("DEV-bulk", "Dev-bulk2", rv);
result.Ok.ShouldBeTrue();
// Ordinal sort: 'S-first' < 'S-last' < 'S-middle'.
result.Warnings.ShouldContain(
"3 scripts ('S-first', 'S-last', 'S-middle') reference tags beneath this node by their raw " +
"paths; the references will break unless the scripts are updated.");
}
// ------------------------------------------------------------------------------------------ Seed
private static void Seed(OtOpcUaConfigDbContext db)
tests/Server/ZB.MOM.WW.OtOpcUa.AdminUI.Tests/Uns/RawTreeServiceTests.cs
+35
View File
@@ -102,6 +102,41 @@ public sealed class RawTreeServiceTests
(await service.LoadChildrenAsync(tags[0])).ShouldBeEmpty(); // leaf
}
/// <summary>
/// Tag rows are read through a narrow projection that leaves the <c>TagConfig</c> JSON blob in the
/// database, so this pins that the three fields a tree row actually renders — identity, name and the
/// concurrency token the mutation handles round-trip — still arrive intact, from both tag loaders
/// (a tag under a group, and a tag hanging directly off a device).
/// </summary>
[Fact]
public async Task LoadChildren_yields_tag_rows_carrying_id_name_and_row_version()
{
var (service, dbName) = Seeded();
var cluster = (await service.LoadRootsAsync()).Single().Children.Single();
// Group-child loader: two tags under "Motors".
var folder = (await service.LoadChildrenAsync(cluster)).Single(n => n.Kind == RawNodeKind.Folder);
var driver = (await service.LoadChildrenAsync(folder)).ShouldHaveSingleItem();
var device = (await service.LoadChildrenAsync(driver)).ShouldHaveSingleItem();
var group = (await service.LoadChildrenAsync(device)).ShouldHaveSingleItem();
var plain = (await service.LoadChildrenAsync(group)).Single(t => t.EntityId == RawTreeTestDb.PlainTagId);
plain.DisplayName.ShouldBe("state");
plain.RowVersion.ShouldBe(
RowVersionOf(dbName, db => db.Tags.Single(t => t.TagId == RawTreeTestDb.PlainTagId).RowVersion));
// Device-child loader: the cluster-root device's own tag.
var rootDriver = (await service.LoadChildrenAsync(cluster)).Single(n => n.Kind == RawNodeKind.Driver);
var rootDevice = (await service.LoadChildrenAsync(rootDriver)).ShouldHaveSingleItem();
var referenced = (await service.LoadChildrenAsync(rootDevice))
.Single(t => t.EntityId == RawTreeTestDb.ReferencedTagId);
referenced.Kind.ShouldBe(RawNodeKind.Tag);
referenced.DisplayName.ShouldBe("temperature");
referenced.RowVersion.ShouldBe(
RowVersionOf(dbName, db => db.Tags.Single(t => t.TagId == RawTreeTestDb.ReferencedTagId).RowVersion));
}
// -------------------------------------------------------------------------------- Create/happy
[Fact]
tests/Server/ZB.MOM.WW.OtOpcUa.AdminUI.Tests/Uns/UnsTreeServiceReferenceTests.cs
+63 -1
View File
@@ -23,13 +23,24 @@ public sealed class UnsTreeServiceReferenceTests
{
private const string EquipmentId = "EQ-1";
/// <summary>A configurable <see cref="IEffectiveNameGuard"/>: returns <see cref="Result"/> from every check.</summary>
/// <summary>
/// A configurable <see cref="IEffectiveNameGuard"/>: returns <see cref="Result"/> from every check.
/// <see cref="CheckManyAsync"/> is implemented explicitly (rather than inheriting the interface's
/// looping default) so <see cref="CheckManyCalls"/> counts BATCH round-trips — that count is what
/// pins "one guard round-trip per bulk add, regardless of how many tags were selected".
/// </summary>
private sealed class FakeGuard : IEffectiveNameGuard
{
public string? Result { get; set; }
public string? LastProposedName { get; private set; }
/// <summary>How many times the BATCH entry point was called.</summary>
public int CheckManyCalls { get; private set; }
/// <summary>The names passed to the last <see cref="CheckManyAsync"/> call, in input order.</summary>
public IReadOnlyList<string> LastProposedNames { get; private set; } = [];
public Task<string?> CheckAsync(
string equipmentId, string proposedEffectiveName, EffectiveNameSourceKind kind,
string? excludeSourceId, CancellationToken ct = default)
@@ -37,6 +48,16 @@ public sealed class UnsTreeServiceReferenceTests
LastProposedName = proposedEffectiveName;
return Task.FromResult(Result);
}
public Task<string?> CheckManyAsync(
string equipmentId, IReadOnlyList<string> proposedEffectiveNames,
EffectiveNameSourceKind kind, CancellationToken ct = default)
{
CheckManyCalls++;
LastProposedNames = [.. proposedEffectiveNames];
LastProposedName = proposedEffectiveNames.Count > 0 ? proposedEffectiveNames[^1] : null;
return Task.FromResult(proposedEffectiveNames.Count > 0 ? Result : null);
}
}
private static OtOpcUaConfigDbContext Db(string name) =>
@@ -178,6 +199,47 @@ public sealed class UnsTreeServiceReferenceTests
db.UnsTagReferences.Any(r => r.EquipmentId == EquipmentId).ShouldBeFalse();
}
/// <summary>
/// A multi-tag add asks the guard ONCE — one batch call carrying every proposed effective name in
/// selection order — instead of one round-trip per selected tag.
/// </summary>
[Fact]
public async Task AddReferences_asks_the_guard_once_for_the_whole_batch()
{
var name = $"ref-batch-guard-{Guid.NewGuid():N}";
Seed(name);
var guard = new FakeGuard();
var service = new UnsTreeService(Factory(name), guard);
var result = await service.AddReferencesAsync(EquipmentId, ["TAG-speed", "TAG-running", "TAG-grouped"]);
result.Ok.ShouldBeTrue();
guard.CheckManyCalls.ShouldBe(1);
guard.LastProposedNames.ShouldBe(new[] { "speed", "running", "nested" });
using var db = Db(name);
db.UnsTagReferences.Count(r => r.EquipmentId == EquipmentId).ShouldBe(3);
}
/// <summary>A collision reported by the batch guard fails the whole add; nothing is persisted.</summary>
[Fact]
public async Task AddReferences_batch_guard_collision_rejected()
{
var name = $"ref-batch-guard-fail-{Guid.NewGuid():N}";
Seed(name);
var guard = new FakeGuard { Result = "'running' already exists on equipment 'machine-1'." };
var service = new UnsTreeService(Factory(name), guard);
var result = await service.AddReferencesAsync(EquipmentId, ["TAG-speed", "TAG-running"]);
result.Ok.ShouldBeFalse();
result.Error.ShouldBe("'running' already exists on equipment 'machine-1'.");
guard.CheckManyCalls.ShouldBe(1);
using var db = Db(name);
db.UnsTagReferences.Any(r => r.EquipmentId == EquipmentId).ShouldBeFalse();
}
// ----- SetReferenceOverride -----
/// <summary>Setting an override changes the effective name; clearing it falls back to the raw name.</summary>
tests/Server/ZB.MOM.WW.OtOpcUa.AdminUI.Tests/Uns/UnsTreeTestDb.cs
+4 -1
View File
@@ -139,7 +139,10 @@ internal static class UnsTreeTestDb
/// <summary>
/// Minimal <see cref="IDbContextFactory{TContext}"/> that hands back contexts sharing a
/// single InMemory database name — the test-side stand-in for the runtime pooled factory.
/// single InMemory database name — the test-side stand-in for the runtime factory, which IS
/// pooled (<c>AddPooledDbContextFactory</c> in <c>ServiceCollectionExtensions.AddOtOpcUaConfigDb</c>).
/// This stand-in builds a fresh context per call rather than renting one: pooling is a
/// construction-cost optimisation, and nothing under test can observe the difference.
/// </summary>
private sealed class NamedFactory(string name) : IDbContextFactory<OtOpcUaConfigDbContext>
{
tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/DataPlaneAuthorizerTests.cs
+85
View File
@@ -320,6 +320,66 @@ public sealed class DataPlaneAuthorizerTests
Veto(gate.Authorize(Identity(), OpcUaOperation.WriteOperate, node, AddressSpaceRealm.Raw));
}
// ───────────────────────────── the cached hot path ─────────────────────────────
/// <summary>
/// A second identical question is answered from the decision cache without re-walking the grants —
/// which on the reordered hot path also means without rebuilding the <see cref="NodeScope"/> the
/// walk would need, because the lookup happens before the scope is built at all.
/// </summary>
/// <remarks>
/// The evaluator count is the observable half: a hit that still built a scope would have nothing to
/// do with it, so the only way to see the short-circuit from outside is that the walk never ran. The
/// cache size pins that the second call was a hit rather than a second miss that happened to
/// overwrite the first entry.
/// </remarks>
[Fact]
public void A_cache_hit_does_not_rebuild_the_scope()
{
var gate = CountingGate(ClusterGrant(NodePermissions.Read), out var evaluator);
var node = RawNode("Plant/Modbus/dev1/Speed");
gate.AuthorizeCached(Identity(), OpcUaOperation.Read, node, AddressSpaceRealm.Raw).ShouldBeNull();
gate.AuthorizeCached(Identity(), OpcUaOperation.Read, node, AddressSpaceRealm.Raw).ShouldBeNull();
evaluator.Calls.ShouldBe(1);
gate.Decisions.Count.ShouldBe(1);
}
/// <summary>
/// <b>Pins the ordering the hot path must keep:</b> the key's usability is re-checked BEFORE the
/// decision cache is consulted, so a cached allow cannot outlive the key that earned it.
/// </summary>
/// <remarks>
/// Expiry rather than a disabling redeploy is the falsifiable form of this. A redeploy advances the
/// generation, which drops the cache wholesale — so a gate that consulted the cache first would
/// still deny, and the test would pass against the broken ordering. Expiry moves with the CLOCK and
/// leaves the generation (and therefore the memoized allow) exactly where it was, so the denial can
/// only have come from the key check running ahead of the lookup.
/// </remarks>
[Fact]
public void Key_revocation_is_rechecked_on_a_cache_hit()
{
var clock = new FakeTimeProvider(new DateTime(2026, 8, 1, 12, 0, 0, DateTimeKind.Utc));
var state = new DataPlaneAuthorityState();
var key = Key(enabled: true);
key.ExpiresUtc = new DateTime(2026, 8, 1, 13, 0, 0, DateTimeKind.Utc);
state.Apply(Cluster, "rev-1", [key], [ClusterGrant(NodePermissions.Read)], Scopes());
var gate = new DataPlaneAuthorizer(
state, new TriePermissionEvaluator(state.Tries), enabled: true, NullLogger.Instance, clock);
var node = RawNode("Plant/Modbus/dev1/Speed");
gate.AuthorizeCached(Identity(), OpcUaOperation.Read, node, AddressSpaceRealm.Raw).ShouldBeNull();
gate.Decisions.Count.ShouldBe(1, "the allow was memoized, so the next call takes the hit path");
clock.Advance(TimeSpan.FromHours(2));
Veto(gate.AuthorizeCached(Identity(), OpcUaOperation.Read, node, AddressSpaceRealm.Raw));
gate.Decisions.Generation.ShouldBe(state.Current.GenerationId);
gate.Decisions.Count.ShouldBe(1, "the memoized allow is untouched — the denial came from the key check");
}
/// <summary>The flag is carried faithfully; the seams read it to decide whether to consult the gate at all.</summary>
[Theory]
[InlineData(true)]
@@ -349,6 +409,18 @@ public sealed class DataPlaneAuthorizerTests
state, new TriePermissionEvaluator(state.Tries), enabled: true, NullLogger.Instance);
}
/// <summary>
/// The same gate <see cref="Gate"/> builds, with the REAL evaluator wrapped in a call counter — the
/// chain under test stays end-to-end, and only the number of walks becomes observable.
/// </summary>
private static DataPlaneAuthorizer CountingGate(NodeAcl grant, out CountingEvaluator evaluator)
{
var state = new DataPlaneAuthorityState();
state.Apply(Cluster, "rev-1", [Key(enabled: true)], [grant], Scopes());
evaluator = new CountingEvaluator(new TriePermissionEvaluator(state.Tries));
return new DataPlaneAuthorizer(state, evaluator, enabled: true, NullLogger.Instance);
}
private static RoleCarryingUserIdentity Identity() =>
new(new UserNameIdentityToken { UserName = "apikey" }, [], Principal, KeyId);
@@ -381,6 +453,19 @@ public sealed class DataPlaneAuthorizerTests
return map;
}
/// <summary>Counts grant walks without changing any of their answers.</summary>
private sealed class CountingEvaluator(IPermissionEvaluator inner) : IPermissionEvaluator
{
public int Calls { get; private set; }
public AuthorizationDecision Authorize(
UserAuthorizationState session, OpcUaOperation operation, NodeScope scope)
{
Calls++;
return inner.Authorize(session, operation, scope);
}
}
/// <summary>Minimal controllable clock — the suite needs only "now" and "advance".</summary>
private sealed class FakeTimeProvider(DateTime utcNow) : TimeProvider
{
tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerStage6EnforcementTests.cs
+535
View File
@@ -1,3 +1,4 @@
using System.Collections;
using Microsoft.Extensions.Logging.Abstractions;
using Opc.Ua;
using Opc.Ua.Server;
@@ -268,8 +269,276 @@ public sealed class NodeManagerStage6EnforcementTests : IDisposable
finally { await host.DisposeAsync(); }
}
/// <summary>
/// Browse filtering removes the denied references from the page it just appended <b>in place</b>,
/// so the survivors must come back exactly once each, in the order the base browser produced them,
/// and an earlier BrowseNext page already sitting in the list must be left untouched.
/// </summary>
/// <remarks>
/// <para>
/// The grant pattern is deliberately non-contiguous and denies the <b>first</b> child: the
/// filtering walk compacts survivors down over the gaps, so a denied prefix is what actually
/// exercises the copy-down path. A pattern granting a contiguous head would pass against a
/// compaction that never moves anything.
/// </para>
/// <para>
/// The expectation is taken from a baseline browse under a blanket grant rather than
/// hand-written, so the assertion is about what filtering <em>did</em> to the base browser's
/// page — not about the SDK's own reference ordering, which is not this manager's contract.
/// </para>
/// <para>
/// The pre-seeded sentinel reference is the range-boundary check: it belongs to a previous
/// page, sits below the filtering range, and names a node the session is <b>not</b> granted —
/// so an off-by-one that walks from index 0 rather than from the appended range removes it.
/// </para>
/// </remarks>
[Fact]
public async Task Browse_filtering_preserves_survivor_order()
{
var (host, server) = await BootAsync();
try
{
var nm = server.NodeManager!;
const string device = "Plant/Modbus/dev1";
nm.EnsureFolder(device, parentNodeId: null, displayName: "dev1", AddressSpaceRealm.Raw);
var children = new List<NodeId>();
for (var ii = 1; ii <= 6; ii++)
{
var rawPath = $"{device}/T{ii}";
nm.EnsureVariable(rawPath, parentFolderNodeId: device, displayName: rawPath, dataType: "Float",
writable: false, historianTagname: null, realm: AddressSpaceRealm.Raw);
children.Add(nm.TryGetVariable(rawPath, AddressSpaceRealm.Raw)!.NodeId);
}
var deviceId = new NodeId(device, children[0].NamespaceIndex);
// Baseline: everything the base browser returns for this folder, unfiltered.
nm.Authorizer = Authorizer(enabled: true, GrantOn(NodePermissions.Browse));
var baseline = Browse(nm, Identity(), deviceId);
baseline.Count.ShouldBeGreaterThanOrEqualTo(6);
// Now grant Browse on the 1st, 3rd and 5th child only.
var granted = new[] { children[0], children[2], children[4] };
nm.Authorizer = TagBrowseAuthorizer($"{device}/T1", $"{device}/T3", $"{device}/T5");
var sentinel = new ReferenceDescription
{
NodeId = children[1], // an explicitly DENIED node, from an earlier page
DisplayName = "previous-page",
};
var references = Browse(nm, Identity(), deviceId, sentinel);
var denied = new[] { children[1], children[3], children[5] };
var expected = new[] { (ExpandedNodeId)children[1] }
.Concat(baseline
.Select(r => r.NodeId)
.Where(id => !denied.Any(d => (ExpandedNodeId)d == id)))
.ToList();
// Non-vacuity: three references really were dropped (and the sentinel really was added).
expected.Count.ShouldBe(baseline.Count - 2);
references.Select(r => r.NodeId).ShouldBe(expected);
// Belt and braces: each survivor appears exactly once, and every granted child survived.
foreach (var id in granted)
references.Count(r => r.NodeId == (ExpandedNodeId)id).ShouldBe(1, $"{id} survives once");
}
finally { await host.DisposeAsync(); }
}
/// <summary>
/// <see cref="OtOpcUaNodeManager.TranslateBrowsePath"/> compacts its <c>targetIds</c> with the exact
/// same forward two-pointer algorithm as <see cref="Browse_filtering_preserves_survivor_order"/>
/// above, over the appended range <c>[before, targetIds.Count)</c> — but PR 7's review (commit
/// <c>541a223b</c>) flagged that half of the rewrite had zero coverage anywhere. This pins it the
/// same way: a non-contiguous denial pattern that includes the FIRST appended target (forcing the
/// copy-down path from the very start of the range, not partway through it), plus a sentinel
/// entry sitting BELOW the appended range that must survive an off-by-one walking from index 0.
/// </summary>
/// <remarks>
/// <para>
/// The SDK's own <c>TranslateBrowsePath</c> resolves one <see cref="RelativePathElement"/>
/// against however many of the source's children share its <c>TargetName</c> — ordinarily
/// exactly one, since <see cref="OtOpcUaNodeManager.EnsureVariable"/> derives every BrowseName
/// from its (unique) RawPath, so getting six REAL appended targets out of one call would need
/// six duplicate BrowseNames under one folder, which the public address-space surface cannot
/// produce. Instead this drives the production compaction against a
/// <see cref="FanOutTargetIdList"/>: it lets the SDK's one real match through to its single
/// <c>Add</c> call, then substitutes the six ids the test actually wants filtered. What is
/// under test — the compaction that runs AFTER <c>base.TranslateBrowsePath</c> returns — sees
/// exactly the same <c>IList&lt;ExpandedNodeId&gt;</c> either way; only how it got populated
/// differs.
/// </para>
/// <para>
/// The sentinel is a DENIED node (T3) pre-seeded below the appended range, same trick as the
/// Browse test's sentinel: an off-by-one that walks from index 0 rather than from <c>before</c>
/// would re-examine and remove it.
/// </para>
/// </remarks>
[Fact]
public async Task TranslateBrowsePath_filtering_preserves_survivor_order()
{
var (host, server) = await BootAsync();
try
{
var nm = server.NodeManager!;
const string device = "Plant/Modbus/dev2";
nm.EnsureFolder(device, parentNodeId: null, displayName: "dev2", AddressSpaceRealm.Raw);
var children = new List<NodeId>();
for (var ii = 1; ii <= 6; ii++)
{
var rawPath = $"{device}/T{ii}";
nm.EnsureVariable(rawPath, parentFolderNodeId: device, displayName: rawPath, dataType: "Float",
writable: false, historianTagname: null, realm: AddressSpaceRealm.Raw);
children.Add(nm.TryGetVariable(rawPath, AddressSpaceRealm.Raw)!.NodeId);
}
var deviceId = new NodeId(device, children[0].NamespaceIndex);
// Grant Browse on the 2nd, 4th and 6th child only — the FIRST target (T1) is denied, and the
// denials (T1, T3, T5) are non-contiguous with the survivors (T2, T4, T6).
nm.Authorizer = TagBrowseAuthorizer($"{device}/T2", $"{device}/T4", $"{device}/T6");
var context = new OperationContext(
new RequestHeader(), secureChannelContext: null, RequestType.Browse, Identity());
// A relative path that resolves to exactly one REAL match (T1) under the SDK's own path
// resolution — the trigger for the single Add() call FanOutTargetIdList intercepts below.
var relativePath = new RelativePathElement
{
ReferenceTypeId = ReferenceTypeIds.Organizes,
IsInverse = false,
IncludeSubtypes = true,
TargetName = new QualifiedName($"{device}/T1", children[0].NamespaceIndex),
};
// Sentinel: a DENIED node (T3), seeded BELOW the appended range, standing in for a target an
// earlier call already resolved. It must come back untouched.
var sentinel = (ExpandedNodeId)children[2];
var targetIds = new FanOutTargetIdList([sentinel], children.Select(c => (ExpandedNodeId)c).ToList());
var unresolvedTargetIds = new List<NodeId>();
nm.TranslateBrowsePath(
context, nm.GetManagerHandle(deviceId), relativePath, targetIds, unresolvedTargetIds);
var granted = new[] { children[1], children[3], children[5] };
var expected = new List<ExpandedNodeId> { sentinel }
.Concat(granted.Select(c => (ExpandedNodeId)c))
.ToList();
// Non-vacuity: three of the six appended targets really were dropped.
expected.Count.ShouldBe(4);
targetIds.ToList().ShouldBe(expected);
// Belt and braces: each survivor appears exactly once, in original order, and the sentinel is
// intact.
foreach (var id in granted)
targetIds.Count(t => t == (ExpandedNodeId)id).ShouldBe(1, $"{id} survives once");
targetIds[0].ShouldBe(sentinel);
}
finally { await host.DisposeAsync(); }
}
/// <summary>
/// Revalidation asks the authorizer once per distinct (session, node), not once per monitored item.
/// A client that subscribes to the same tag from several items — one per attribute, or one per
/// downstream consumer sharing a session — is the ordinary case, and every one of those items used
/// to pay for its own grant walk on every deployment.
/// </summary>
/// <remarks>
/// <para>
/// The observable is the authorizer's <see cref="TimeProvider"/>. <c>Evaluate</c> reads the clock
/// exactly once per call, for the key revocation/expiry check that is deliberately outside the
/// decision cache — so counting clock reads counts <c>IsAllowedCached</c> entries, which is the
/// work this dedup removes. Counting the permission evaluator instead would be vacuous: the
/// authorizer's own decision cache already memoizes the grant walk, so it would report one
/// either way.
/// </para>
/// <para>
/// <see cref="OtOpcUaNodeManager.RevalidateMonitoredItems"/> is called directly rather than via
/// an authority flip so the count covers exactly one pass — a flip would fire the pass through
/// <c>OnAuthorityChanged</c> as well and the assertion would be about two.
/// </para>
/// </remarks>
[Fact]
public async Task Revalidation_evaluates_one_verdict_per_session_and_node()
{
var (host, server) = await BootAsync();
try
{
var nm = server.NodeManager!;
var clock = new CountingTimeProvider(new DateTime(2026, 8, 15, 12, 0, 0, DateTimeKind.Utc));
nm.Authorizer = SubscribeAuthorizer(clock);
var nodeId = Variable(nm, "Plant/Modbus/dev1/Speed");
var identity = Identity();
var items = Enumerable.Range(0, 3)
.Select(_ => TrackItem(nm, nodeId, identity))
.ToList();
clock.Reset();
nm.RevalidateMonitoredItems();
clock.Calls.ShouldBe(1, "three items on one node share one (session, node) verdict");
// Non-vacuity: the grant really permitted, so the memoized verdict is an allow rather than the
// deny every unresolvable case produces.
items.ShouldAllBe(i => i.Denials == 0);
}
finally { await host.DisposeAsync(); }
}
/// <summary>
/// The falsification control for <see cref="Revalidation_evaluates_one_verdict_per_session_and_node"/>:
/// three items on three DISTINCT nodes still cost three evaluations, so the dedup is keyed on the
/// node and not collapsing every item in the pass to a single verdict.
/// </summary>
[Fact]
public async Task Revalidation_still_evaluates_each_distinct_node()
{
var (host, server) = await BootAsync();
try
{
var nm = server.NodeManager!;
var clock = new CountingTimeProvider(new DateTime(2026, 8, 15, 12, 0, 0, DateTimeKind.Utc));
nm.Authorizer = SubscribeAuthorizer(clock);
var identity = Identity();
foreach (var tag in new[] { "Speed", "Torque", "Pressure" })
TrackItem(nm, Variable(nm, $"Plant/Modbus/dev1/{tag}"), identity);
clock.Reset();
nm.RevalidateMonitoredItems();
clock.Calls.ShouldBe(3);
}
finally { await host.DisposeAsync(); }
}
// ───────────────────────────── helpers ─────────────────────────────
/// <summary>A cluster-wide Subscribe grant — what <c>CreateMonitoredItems</c> resolves to.</summary>
private static DataPlaneAuthorizer SubscribeAuthorizer(TimeProvider clock)
{
var state = new DataPlaneAuthorityState();
state.Apply(Cluster, "rev-1", [], [GrantOn(NodePermissions.Subscribe)], Scopes());
return new DataPlaneAuthorizer(
state, new TriePermissionEvaluator(state.Tries), enabled: true, NullLogger.Instance, clock);
}
/// <summary>Registers a stub data-change item against <paramref name="nodeId"/> as if the SDK had created it.</summary>
private static StubSampledItem TrackItem(OtOpcUaNodeManager nm, NodeId nodeId, IUserIdentity identity)
{
var handle = (NodeHandle)nm.GetManagerHandle(nodeId);
var item = new StubSampledItem(handle, identity);
nm.TrackLiveMonitoredItem(handle, item);
return item;
}
private static NodeId Variable(OtOpcUaNodeManager nm, string rawPath)
{
nm.EnsureVariable(rawPath, parentFolderNodeId: null, displayName: rawPath, dataType: "Float",
@@ -297,6 +566,64 @@ public sealed class NodeManagerStage6EnforcementTests : IDisposable
return (values, errors);
}
/// <summary>
/// Drives the manager's <c>Browse</c> override directly, optionally with references already in the
/// list to stand in for an earlier BrowseNext page.
/// </summary>
private static List<ReferenceDescription> Browse(
OtOpcUaNodeManager nm,
IUserIdentity? identity,
NodeId source,
params ReferenceDescription[] previousPage)
{
var context = new OperationContext(
new RequestHeader(), secureChannelContext: null, RequestType.Browse, identity);
var continuationPoint = new ContinuationPoint
{
NodeToBrowse = nm.GetManagerHandle(source),
BrowseDirection = BrowseDirection.Forward,
ReferenceTypeId = null,
IncludeSubtypes = true,
NodeClassMask = 0,
ResultMask = BrowseResultMask.All,
MaxResultsToReturn = 0,
};
var references = new List<ReferenceDescription>(previousPage);
nm.Browse(context, ref continuationPoint, references);
continuationPoint?.Dispose();
return references;
}
/// <summary>A Browse-only grant per named raw tag path, each in its own <c>Tag</c>-kind scope.</summary>
private static DataPlaneAuthorizer TagBrowseAuthorizer(params string[] rawPaths)
{
var grants = rawPaths
.Select((_, ii) => new NodeAcl
{
NodeAclId = $"acl-tag-{ii}",
ClusterId = Cluster,
Principal = Principal,
ScopeKind = NodeAclScopeKind.Tag,
ScopeId = $"tag-{ii}",
PermissionFlags = NodePermissions.Browse,
})
.ToList();
var scopePaths = rawPaths
.Select((path, ii) => (Ref: new AclScopeRef(NodeAclScopeKind.Tag, $"tag-{ii}"), Path: path))
.ToDictionary(
e => e.Ref,
e => (IReadOnlyList<NodeAclPath>)
[new NodeAclPath([NodeAclRealms.Raw, .. e.Path.Split('/')])]);
var state = new DataPlaneAuthorityState();
state.Apply(Cluster, "rev-1", [], grants, scopePaths);
return new DataPlaneAuthorizer(
state, new TriePermissionEvaluator(state.Tries), enabled: true, NullLogger.Instance);
}
private static DataPlaneAuthorizer Authorizer(
bool enabled,
NodeAcl grant,
@@ -372,6 +699,214 @@ public sealed class NodeManagerStage6EnforcementTests : IDisposable
public void Advance(TimeSpan by) => _now = _now.Add(by);
}
/// <summary>A fixed clock that counts how many times it was read.</summary>
/// <remarks>
/// The authorizer reads its clock exactly once per <c>Evaluate</c>, so this counts authorization
/// evaluations without needing a seam on the authorizer itself.
/// </remarks>
private sealed class CountingTimeProvider(DateTime utcNow) : TimeProvider
{
private readonly DateTimeOffset _now = new(utcNow, TimeSpan.Zero);
public int Calls { get; private set; }
public override DateTimeOffset GetUtcNow()
{
Calls++;
return _now;
}
public void Reset() => Calls = 0;
}
/// <summary>
/// The minimum of a sampled data-change monitored item that revalidation touches: the identity that
/// created it, and the denial queue it is told through.
/// </summary>
/// <remarks>
/// Everything else throws rather than returning a plausible default, so a revalidation that started
/// depending on more of the item surfaces here instead of being silently absorbed.
/// </remarks>
private sealed class StubSampledItem(NodeHandle handle, IUserIdentity identity)
: ISampledDataChangeMonitoredItem
{
/// <summary>How many <c>BadUserAccessDenied</c> values revalidation queued on this item.</summary>
public int Denials { get; private set; }
public IUserIdentity EffectiveIdentity => identity;
public object ManagerHandle => handle;
public NodeId NodeId => handle.NodeId;
public void QueueValue(DataValue value, ServiceResult error, bool ignoreFilters)
{
if (error?.StatusCode.Code == StatusCodes.BadUserAccessDenied) Denials++;
}
public void QueueValue(DataValue value, ServiceResult error) => QueueValue(value, error, false);
public void Dispose() { }
// ── everything below is untouched by revalidation ──
public INodeManager NodeManager => throw new NotSupportedException();
public ISession Session => throw new NotSupportedException();
public uint Id => throw new NotSupportedException();
public uint SubscriptionId => throw new NotSupportedException();
public bool IsDurable => throw new NotSupportedException();
public uint ClientHandle => throw new NotSupportedException();
public ISubscription SubscriptionCallback
{
get => throw new NotSupportedException();
set => throw new NotSupportedException();
}
public int MonitoredItemType => throw new NotSupportedException();
public bool IsReadyToPublish => throw new NotSupportedException();
public bool IsReadyToTrigger
{
get => throw new NotSupportedException();
set => throw new NotSupportedException();
}
public bool IsResendData => throw new NotSupportedException();
public MonitoringMode MonitoringMode => throw new NotSupportedException();
public double SamplingInterval => throw new NotSupportedException();
public DataChangeFilter DataChangeFilter => throw new NotSupportedException();
public uint AttributeId => throw new NotSupportedException();
public NumericRange IndexRange => throw new NotSupportedException();
public QualifiedName DataEncoding => throw new NotSupportedException();
public bool AlwaysReportUpdates
{
get => throw new NotSupportedException();
set => throw new NotSupportedException();
}
public DiagnosticsMasks DiagnosticsMasks => throw new NotSupportedException();
public uint QueueSize => throw new NotSupportedException();
public double MinimumSamplingInterval => throw new NotSupportedException();
public int TimeToNextSample => throw new NotSupportedException();
public void SetupResendDataTrigger() => throw new NotSupportedException();
public ServiceResult GetCreateResult(out MonitoredItemCreateResult result) =>
throw new NotSupportedException();
public ServiceResult GetModifyResult(out MonitoredItemModifyResult result) =>
throw new NotSupportedException();
public IStoredMonitoredItem ToStorableMonitoredItem() => throw new NotSupportedException();
public MonitoringMode SetMonitoringMode(MonitoringMode monitoringMode) =>
throw new NotSupportedException();
public bool Publish(
OperationContext context,
Queue<MonitoredItemNotification> notifications,
Queue<DiagnosticInfo> diagnostics,
uint maxNotificationsPerPublish,
Microsoft.Extensions.Logging.ILogger logger) => throw new NotSupportedException();
public void SetSemanticsChanged() => throw new NotSupportedException();
public void SetStructureChanged() => throw new NotSupportedException();
public bool SamplingIntervalExpired() => throw new NotSupportedException();
public ReadValueId GetReadValueId() => throw new NotSupportedException();
public ServiceResult ModifyAttributes(
DiagnosticsMasks diagnosticsMasks,
TimestampsToReturn timestampsToReturn,
uint clientHandle,
MonitoringFilter originalFilter,
MonitoringFilter filterToUse,
Opc.Ua.Range range,
double samplingInterval,
uint queueSize,
bool discardOldest) => throw new NotSupportedException();
public void SetSamplingInterval(double samplingInterval) => throw new NotSupportedException();
}
/// <summary>
/// Target-id sink for <see cref="TranslateBrowsePath_filtering_preserves_survivor_order"/>: seeds a
/// pre-existing (pre-<c>before</c>) range untouched, then substitutes a FIXED batch of ids for
/// whatever single real match the SDK's own path resolution adds via its one <c>Add</c> call —
/// that is what turns one real append into the multi-item appended range the compaction under test
/// needs to run over, without forging duplicate BrowseNames anywhere in the address space.
/// </summary>
private sealed class FanOutTargetIdList : IList<ExpandedNodeId>
{
private readonly List<ExpandedNodeId> _inner;
private readonly IReadOnlyList<ExpandedNodeId> _fanOut;
private bool _fanned;
public FanOutTargetIdList(IEnumerable<ExpandedNodeId> seed, IReadOnlyList<ExpandedNodeId> fanOut)
{
_inner = new List<ExpandedNodeId>(seed);
_fanOut = fanOut;
}
public void Add(ExpandedNodeId item)
{
if (_fanned)
{
_inner.Add(item);
return;
}
_fanned = true;
_inner.AddRange(_fanOut);
}
public int Count => _inner.Count;
public bool IsReadOnly => false;
public ExpandedNodeId this[int index]
{
get => _inner[index];
set => _inner[index] = value;
}
public void Clear() => _inner.Clear();
public bool Contains(ExpandedNodeId item) => _inner.Contains(item);
public void CopyTo(ExpandedNodeId[] array, int arrayIndex) => _inner.CopyTo(array, arrayIndex);
public IEnumerator<ExpandedNodeId> GetEnumerator() => _inner.GetEnumerator();
IEnumerator IEnumerable.GetEnumerator() => _inner.GetEnumerator();
public int IndexOf(ExpandedNodeId item) => _inner.IndexOf(item);
public void Insert(int index, ExpandedNodeId item) => _inner.Insert(index, item);
public bool Remove(ExpandedNodeId item) => _inner.Remove(item);
public void RemoveAt(int index) => _inner.RemoveAt(index);
}
/// <summary>Cleans up the PKI root directory.</summary>
public void Dispose()
{
tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/SessionAclDecisionCacheTests.cs
+43 -5
View File
@@ -111,20 +111,58 @@ public sealed class SessionAclDecisionCacheTests
}
/// <summary>
/// The cache is bounded. A client walking a huge address space must not be able to grow it without
/// limit; clearing at the ceiling degrades to recomputation, which is the no-cache steady state.
/// The global bound still holds when no single session is anywhere near its own ceiling. Spreading
/// the entries over many sessions is the case the per-session cap cannot catch, so the global
/// counter is the only thing standing between a large fleet and unbounded growth.
/// </summary>
[Fact]
public void The_cache_is_bounded()
public void The_global_bound_still_holds()
{
var cache = new SessionAclDecisionCache();
for (var i = 0; i <= SessionAclDecisionCache.MaxEntries + 10; i++)
Ask(cache, generation: 1, node: $"Plant/dev1/Tag{i}", compute: () => true);
// Well under MaxEntriesPerSession, so every clear observed here is the global one.
const int perSession = 5_000;
var total = SessionAclDecisionCache.MaxEntries + perSession;
for (var i = 0; i < total; i++)
Ask(
cache, generation: 1, session: $"key-{i / perSession}", node: $"Plant/dev1/Tag{i}",
compute: () => true);
// The global counter is approximate under concurrency, so the contract allows one shard's worth of
// slack; single-threaded it is exact, which is the stronger fact pinned here.
cache.Count.ShouldBeLessThanOrEqualTo(SessionAclDecisionCache.MaxEntries);
}
/// <summary>
/// One session filling its own shard evicts only itself. This is the whole point of sharding: a
/// machine consumer crawling a 50k-node tree used to blow away every other session's decisions on
/// its way past the ceiling, so the sessions doing ordinary work paid for the crawl.
/// </summary>
[Fact]
public void A_full_session_shard_clears_only_that_session()
{
var cache = new SessionAclDecisionCache();
const string crawler = "crawler-key";
const string neighbour = "neighbour-key";
var neighbourCalls = 0;
Ask(cache, generation: 1, session: neighbour, compute: () => { neighbourCalls++; return true; })
.ShouldBeTrue();
neighbourCalls.ShouldBe(1);
for (var i = 0; i <= SessionAclDecisionCache.MaxEntriesPerSession + 10; i++)
Ask(cache, generation: 1, session: crawler, node: $"Plant/dev1/Tag{i}", compute: () => true);
// The crawler's own shard was cleared and restarted rather than growing without limit.
cache.Count.ShouldBeLessThanOrEqualTo(SessionAclDecisionCache.MaxEntriesPerSession);
// The neighbour's single decision survived the crawl — its walk is not re-run.
Ask(cache, generation: 1, session: neighbour, compute: () => { neighbourCalls++; return true; })
.ShouldBeTrue();
neighbourCalls.ShouldBe(1);
}
/// <summary>
/// A decision taken while a deployment lands mid-walk is returned but NOT stored, so the new
/// generation never inherits a verdict its own trie did not make.
tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/DriverHostActorArtifactCacheTests.cs
+1 -1
View File
@@ -143,7 +143,7 @@ public sealed class DriverHostActorArtifactCacheTests : RuntimeActorTestBase
public void AThrowingCache_DoesNotFailTheApply()
{
// By the time the cache runs, an Applied ACK has already gone to the coordinator. If a cache
// failure escaped, ApplyAndAck's catch would send a SECOND, contradictory Failed ACK for a
// failure escaped, HandleArtifactForApply's catch would send a SECOND, contradictory Failed ACK for a
// deployment the fleet already believes is live.
var db = NewInMemoryDbFactory();
var (deploymentId, _) = SeedDeployment(db, RevA, clusterId: null);
tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/DriverHostActorAsyncCacheWriteTests.cs
+1 -1
View File
@@ -23,7 +23,7 @@ namespace ZB.MOM.WW.OtOpcUa.Runtime.Tests.Drivers;
/// </summary>
/// <remarks>
/// <para>
/// The ACK was never the thing at risk: <c>ApplyAndAck</c> sends it BEFORE it caches, so a
/// The ACK was never the thing at risk: <c>HandleArtifactForApply</c> sends it BEFORE it caches, so a
/// blocking store still acked promptly. What it blocked was <b>everything after</b> — the host
/// could not dequeue a driver value burst, a diagnostics request or the next dispatch until
/// SQLite was done. So the load-bearing assertion here is that the actor answers an unrelated
tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/DriverHostActorCommsLossTests.cs
+130
View File
@@ -0,0 +1,130 @@
using Akka.Actor;
using Akka.TestKit;
using Microsoft.EntityFrameworkCore;
using Shouldly;
using Xunit;
using ZB.MOM.WW.OtOpcUa.Commons.Engines;
using ZB.MOM.WW.OtOpcUa.Commons.Messages.Deploy;
using ZB.MOM.WW.OtOpcUa.Commons.OpcUa;
using ZB.MOM.WW.OtOpcUa.Commons.Types;
using ZB.MOM.WW.OtOpcUa.Configuration;
using ZB.MOM.WW.OtOpcUa.Runtime.Drivers;
using ZB.MOM.WW.OtOpcUa.Runtime.VirtualTags;
using ZB.MOM.WW.OtOpcUa.Runtime.Tests.Harness;
using static ZB.MOM.WW.OtOpcUa.Runtime.Tests.Harness.V3ArtifactSeeder;
namespace ZB.MOM.WW.OtOpcUa.Runtime.Tests.Drivers;
/// <summary>
/// The comms-loss fan (<c>DriverHostActor.FanCommsLossToDependencyMux</c>) end-to-end through a REAL
/// <see cref="DependencyMuxActor"/>, rather than the mux probe the other host suites use.
/// <para>
/// Two properties are pinned here that a probe cannot show. First, the fan is ONE
/// <see cref="DriverInstanceActor.AttributeValueBatchPublished"/> for the whole driver rather than one
/// Tell per ref — a driver owning 5k tags used to cost 5k messages on a single connectivity edge — and
/// the mux's batch path still delivers every item to an interested subscriber, so batching did not
/// quietly drop the delivery it exists to make.
/// </para>
/// <para>
/// Second, the driver scoping: the fan resolves its refs through a per-driver index built alongside
/// <c>_nodeIdByDriverRef</c> instead of scanning every entry in it, and an index is exactly the thing
/// that can key a lookup wrong. The two-driver case is therefore the falsification control — the
/// untouched driver's ref must be absent from the batch, which a scan-and-filter implementation and a
/// correctly-keyed index both satisfy but a mis-keyed one does not.
/// </para>
/// </summary>
public sealed class DriverHostActorCommsLossTests : RuntimeActorTestBase
{
private static readonly NodeId TestNode = NodeId.Parse("driver-commsloss-test");
private static readonly RevisionHash RevA = RevisionHash.Parse(new string('c', 64));
/// <summary>Spawns a host whose dependency mux is a REAL <see cref="DependencyMuxActor"/>, applies the
/// seeded deployment, and registers <paramref name="subscriber"/>'s interest in <paramref name="refs"/>.
/// </summary>
/// <param name="db">The factory whose deployment was seeded by <see cref="SeedV3Deployment"/>.</param>
/// <param name="deploymentId">The deployment to dispatch.</param>
/// <param name="subscriber">The probe standing in for a VirtualTagActor / scripted-alarm subscriber.</param>
/// <param name="refs">The RawPaths that subscriber registers interest in.</param>
/// <returns>The applied host actor.</returns>
private IActorRef SpawnHostWithRealMux(
IDbContextFactory<OtOpcUaConfigDbContext> db, DeploymentId deploymentId, TestProbe subscriber, params string[] refs)
{
var coordinator = CreateTestProbe();
var mux = Sys.ActorOf(DependencyMuxActor.Props());
mux.Tell(new DependencyMuxActor.RegisterInterest(refs, subscriber.Ref));
var actor = Sys.ActorOf(DriverHostActor.Props(
db, TestNode, coordinator.Ref,
localRoles: new HashSet<string> { "driver" },
dependencyMux: mux,
virtualTagEvaluator: NullVirtualTagEvaluator.Instance,
virtualTagHostOverride: CreateTestProbe().Ref));
actor.Tell(new DispatchDeployment(deploymentId, RevA, CorrelationId.NewId()));
coordinator.ExpectMsg<ApplyAck>(PresenceBudget).Outcome.ShouldBe(ApplyAckOutcome.Applied);
return actor;
}
/// <summary>A driver's comms loss reaches the mux as ONE batch carrying a Bad item per ref it owns, and
/// the mux delivers that whole slice to an interested subscriber in ONE
/// <see cref="VirtualTagActor.DependencyValueBatch"/>.</summary>
[Fact]
public void Comms_loss_reaches_the_mux_as_one_batch_of_bad_items()
{
var db = NewInMemoryDbFactory();
var deploymentId = SeedV3Deployment(db, RevA,
rawTags: new[]
{
(Driver: "drv-1", DriverName: "Modbus", Device: "dev1", Tag: "speed", DataType: "Double", Writable: false),
(Driver: "drv-1", DriverName: "Modbus", Device: "dev1", Tag: "temp", DataType: "Double", Writable: false),
},
unsRefs: Array.Empty<(string, string, string, string, string?)>());
var subscriber = CreateTestProbe();
var actor = SpawnHostWithRealMux(db, deploymentId, subscriber,
"Plant/Modbus/dev1/speed", "Plant/Modbus/dev1/temp");
actor.Tell(new DriverInstanceActor.ConnectivityChanged("drv-1", Connected: false));
var batch = subscriber.ExpectMsg<VirtualTagActor.DependencyValueBatch>(PresenceBudget);
batch.Items.Select(i => i.TagId).ShouldBe(
new[] { "Plant/Modbus/dev1/speed", "Plant/Modbus/dev1/temp" }, ignoreOrder: true);
foreach (var item in batch.Items)
{
item.StatusCode.ShouldBe(OpcUaStatus.BadNoCommunication);
item.Value.ShouldBeNull();
}
// ONE batch, not one per ref: the whole point of the change. A per-ref fan would land a second
// single-item DependencyValueBatch here.
subscriber.ExpectNoMsg(TimeSpan.FromMilliseconds(300));
}
/// <summary>Falsification control for the per-driver index: with TWO drivers configured, one driver's
/// comms loss carries ONLY its own refs — the other driver's tag is untouched.</summary>
[Fact]
public void Comms_loss_batch_carries_only_the_dead_drivers_refs()
{
var db = NewInMemoryDbFactory();
var deploymentId = SeedV3Deployment(db, RevA,
rawTags: new[]
{
(Driver: "drv-1", DriverName: "Modbus", Device: "dev1", Tag: "speed", DataType: "Double", Writable: false),
(Driver: "drv-1", DriverName: "Modbus", Device: "dev1", Tag: "temp", DataType: "Double", Writable: false),
(Driver: "drv-2", DriverName: "Modbus2", Device: "dev2", Tag: "flow", DataType: "Double", Writable: false),
},
unsRefs: Array.Empty<(string, string, string, string, string?)>());
var subscriber = CreateTestProbe();
var actor = SpawnHostWithRealMux(db, deploymentId, subscriber,
"Plant/Modbus/dev1/speed", "Plant/Modbus/dev1/temp", "Plant/Modbus2/dev2/flow");
actor.Tell(new DriverInstanceActor.ConnectivityChanged("drv-1", Connected: false));
var batch = subscriber.ExpectMsg<VirtualTagActor.DependencyValueBatch>(PresenceBudget);
batch.Items.Select(i => i.TagId).ShouldBe(
new[] { "Plant/Modbus/dev1/speed", "Plant/Modbus/dev1/temp" }, ignoreOrder: true);
subscriber.ExpectNoMsg(TimeSpan.FromMilliseconds(300));
}
}
tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/DriverHostActorDirectApplyOffMailboxTests.cs
+152
View File
@@ -0,0 +1,152 @@
using System.Text.Json;
using Akka.Actor;
using Microsoft.EntityFrameworkCore;
using Shouldly;
using Xunit;
using ZB.MOM.WW.OtOpcUa.Commons.Interfaces;
using ZB.MOM.WW.OtOpcUa.Commons.Messages.Deploy;
using ZB.MOM.WW.OtOpcUa.Commons.Messages.Fleet;
using ZB.MOM.WW.OtOpcUa.Commons.Types;
using ZB.MOM.WW.OtOpcUa.Configuration;
using ZB.MOM.WW.OtOpcUa.Configuration.Entities;
using ZB.MOM.WW.OtOpcUa.Configuration.Enums;
using ZB.MOM.WW.OtOpcUa.Runtime.Drivers;
using ZB.MOM.WW.OtOpcUa.Runtime.Tests.Harness;
namespace ZB.MOM.WW.OtOpcUa.Runtime.Tests.Drivers;
/// <summary>
/// Perf roadmap Task 26 — the <c>Direct</c>-mode artifact read is I/O (an EF read of a potentially
/// multi-MB gzip blob out of central SQL) and must not run on the actor thread.
/// </summary>
/// <remarks>
/// <para>
/// The <c>FetchAndCache</c> path has always piped its artifact fetch back to <c>Self</c> for
/// exactly this reason. <c>Direct</c> read the blob INLINE inside the receive, so every value
/// batch, diagnostics request and driver command queued behind central SQL for the whole read —
/// on a WAN link or a loaded server, for as long as that read takes.
/// </para>
/// <para>
/// The load-bearing assertion is therefore that the host answers an <b>unrelated</b> message
/// while the read is still in flight, not that the ACK eventually arrives: a blocking read still
/// acks, just late and behind everything else.
/// </para>
/// </remarks>
public sealed class DriverHostActorDirectApplyOffMailboxTests : RuntimeActorTestBase
{
private static readonly NodeId TestNode = NodeId.Parse("driver-test");
private static readonly RevisionHash RevA = RevisionHash.Parse(new string('a', 64));
[Fact]
public void A_slow_direct_artifact_read_does_not_block_the_host_mailbox()
{
var inner = NewInMemoryDbFactory();
var db = new GatedConfigDbFactory(inner);
var deploymentId = SeedDeployment(inner, RevA);
var coordinator = CreateTestProbe();
var actor = Sys.ActorOf(DriverHostActor.Props(
db, TestNode, coordinator.Ref,
localRoles: new HashSet<string> { "driver" }));
try
{
// Barrier: an answered GetDiagnostics proves PreStart (and its bootstrap ConfigDb read) is
// done, so the allowance below counts only the apply's own contexts.
actor.Tell(new GetDiagnostics(CorrelationId.NewId()), coordinator.Ref);
coordinator.ExpectMsg<NodeDiagnosticsSnapshot>(TimeSpan.FromSeconds(5));
// Let ONE more context through — the apply's Applying-row upsert, which is a deliberate
// on-thread write (bounded, one row) — and block the next, which is the artifact read.
// Both misconfigurations of this number fail LOUDLY rather than passing vacuously: too
// small blocks the upsert and the assertion below times out even on a fixed build; too
// large never engages the gate and the presence assertion on Blocked fails.
db.CloseAfter(allowThrough: 1);
actor.Tell(new DispatchDeployment(deploymentId, RevA, CorrelationId.NewId()));
// The read was entered and has NOT completed — the gate is still closed.
AwaitAssert(() => db.Blocked.ShouldBe(1), duration: PresenceBudget);
db.Unblocked.ShouldBe(0);
// THE assertion: with the artifact read still in flight the host is still processing its
// mailbox. Read inline on the actor thread, this times out.
actor.Tell(new GetDiagnostics(CorrelationId.NewId()), coordinator.Ref);
coordinator.ExpectMsg<NodeDiagnosticsSnapshot>(TimeSpan.FromSeconds(3));
// Releasing the read completes the apply exactly as before — same Applied ACK, same revision.
db.Open();
coordinator.ExpectMsg<ApplyAck>(TimeSpan.FromSeconds(10)).Outcome.ShouldBe(ApplyAckOutcome.Applied);
actor.Tell(new GetDiagnostics(CorrelationId.NewId()), coordinator.Ref);
coordinator.ExpectMsg<NodeDiagnosticsSnapshot>(TimeSpan.FromSeconds(5)).CurrentRevision.ShouldBe(RevA);
}
finally
{
// Never leave the gate closed: a parked read would hang ActorSystem shutdown.
db.Open();
}
}
/// <summary>Seeds a sealed, parseable deployment and returns its id.</summary>
private static DeploymentId SeedDeployment(IDbContextFactory<OtOpcUaConfigDbContext> db, RevisionHash rev)
{
var artifact = JsonSerializer.SerializeToUtf8Bytes(new { DriverInstances = Array.Empty<object>() });
var id = DeploymentId.NewId();
using var ctx = db.CreateDbContext();
ctx.Deployments.Add(new Deployment
{
DeploymentId = id.Value,
RevisionHash = rev.Value,
Status = DeploymentStatus.Sealed,
CreatedBy = "test",
SealedAtUtc = DateTime.UtcNow,
ArtifactBlob = artifact,
});
ctx.SaveChanges();
return id;
}
/// <summary>
/// A ConfigDb factory that parks the caller on a gate — the fake shape of a slow central-SQL
/// read. It blocks the calling THREAD (not a task), because that is what an EF read does and it
/// is the only shape that can prove which thread the read runs on.
/// </summary>
private sealed class GatedConfigDbFactory(IDbContextFactory<OtOpcUaConfigDbContext> inner)
: IDbContextFactory<OtOpcUaConfigDbContext>
{
private readonly ManualResetEventSlim _gate = new(initialState: true);
private int _allowance = int.MaxValue;
private int _blocked;
private int _unblocked;
/// <summary>How many callers have reached the closed gate.</summary>
public int Blocked => Volatile.Read(ref _blocked);
/// <summary>How many of those have been let through again.</summary>
public int Unblocked => Volatile.Read(ref _unblocked);
/// <summary>Closes the gate, after letting <paramref name="allowThrough"/> more contexts past.</summary>
public void CloseAfter(int allowThrough)
{
Volatile.Write(ref _allowance, allowThrough);
_gate.Reset();
}
/// <summary>Opens the gate for good — also the test-teardown safety net.</summary>
public void Open() => _gate.Set();
public OtOpcUaConfigDbContext CreateDbContext()
{
if (Interlocked.Decrement(ref _allowance) < 0 && !_gate.IsSet)
{
Interlocked.Increment(ref _blocked);
// Bounded so a mis-wired test fails rather than hanging the whole assembly.
_gate.Wait(TimeSpan.FromSeconds(30));
Interlocked.Increment(ref _unblocked);
}
return inner.CreateDbContext();
}
}
}
tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/DriverHostActorLiveValueTests.cs
+8 -8
View File
@@ -31,7 +31,8 @@ namespace ZB.MOM.WW.OtOpcUa.Runtime.Tests.Drivers;
/// Drives a real apply through <see cref="V3ArtifactSeeder"/>: the seeded artifact carries the v3
/// raw-tag chain (RawFolder → DriverInstance(RawFolderId) → Device → Tag) + an optional
/// UnsTagReference projecting each raw tag into an equipment. <c>DispatchDeployment</c> triggers the
/// <c>ApplyAndAck → PushDesiredSubscriptions</c> pass that builds the map; the sink and mux are
/// <c>BeginDirectApply → HandleArtifactForApply → PushDesiredSubscriptionsFromParsed</c> pass that
/// builds the map; the sink and mux are
/// injected as <see cref="Akka.TestKit.TestProbe"/>s.
/// </para>
/// </summary>
@@ -160,7 +161,8 @@ public sealed class DriverHostActorLiveValueTests : RuntimeActorTestBase
// went on evaluating it. These pin the synthetic Bad that closes that hole.
/// <summary>Comms loss pushes a Bad for every value ref the driver owns — not one, and not the
/// alarm-bearing ones (those take the Part 9 quality path instead).</summary>
/// alarm-bearing ones (those take the Part 9 quality path instead) — as ONE batch envelope rather than
/// one Tell per ref.</summary>
[Fact]
public void Driver_comms_loss_pushes_BadNoCommunication_into_the_mux_for_every_ref_it_owns()
{
@@ -177,11 +179,10 @@ public sealed class DriverHostActorLiveValueTests : RuntimeActorTestBase
actor.Tell(new DriverInstanceActor.ConnectivityChanged("drv-1", Connected: false));
var pushes = new[]
{
mux.ExpectMsg<DriverInstanceActor.AttributeValuePublished>(PresenceBudget),
mux.ExpectMsg<DriverInstanceActor.AttributeValuePublished>(PresenceBudget),
};
// ONE envelope for the whole driver: a driver owning 5k tags cost 5k Tells before this was batched.
var fan = mux.ExpectMsg<DriverInstanceActor.AttributeValueBatchPublished>(PresenceBudget);
fan.DriverInstanceId.ShouldBe("drv-1");
var pushes = fan.Items;
pushes.Select(p => p.FullReference).ShouldBe(
new[] { "Plant/Modbus/dev1/speed", "Plant/Modbus/dev1/temp" }, ignoreOrder: true);
@@ -194,7 +195,6 @@ public sealed class DriverHostActorLiveValueTests : RuntimeActorTestBase
// Null rather than the last-known value: the honest statement is "no current value". The
// engine's AreInputsReady gate holds the predicate on any Bad, so no script sees the null.
p.Value.ShouldBeNull();
p.DriverInstanceId.ShouldBe("drv-1");
}
mux.ExpectNoMsg(TimeSpan.FromMilliseconds(300));
tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Historian/ContinuousHistorizationOptionsTests.cs
+20
View File
@@ -36,6 +36,26 @@ public sealed class ContinuousHistorizationOptionsTests
}.Validate(),
m => m.Contains("CommitIntervalMs"));
/// <summary>
/// Pins the shipped defaults, which are a tuned set rather than arbitrary numbers: the drain is
/// bounded-parallel over an in-memory peek (so a wide <c>DrainBatchSize</c> is cheap and a narrow
/// one lets the outbox fill during a gateway hiccup), <c>Capacity</c> is the #538 disk backstop,
/// and <c>MaxWriteAttempts</c> is the #538 poison-tag budget. The shipped
/// <c>appsettings.json</c> restates these values, so a silent drift between the two is exactly
/// what this test exists to catch.
/// </summary>
[Fact]
public void Defaults_are_the_documented_batch_era_values()
{
var options = new ContinuousHistorizationOptions();
Assert.Equal("PerEntry", options.CommitMode);
Assert.Equal(1024, options.DrainBatchSize);
Assert.Equal(8, options.DrainConcurrency);
Assert.Equal(500_000, options.Capacity);
Assert.Equal(5, options.MaxWriteAttempts);
}
/// <summary>A fully-configured enabled recorder produces no warnings.</summary>
[Fact]
public void Valid_config_is_clean()
tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Historian/ContinuousHistorizationRecorderTests.cs
+657
View File
@@ -607,6 +607,542 @@ public sealed class ContinuousHistorizationRecorderTests : TestKit
}, TimeSpan.FromSeconds(10));
}
// ---------------- Bounded-parallel drain (DrainConcurrency) ----------------
/// <summary>
/// A drain pass writes distinct historian tags CONCURRENTLY, bounded by <c>DrainConcurrency</c>.
/// </summary>
/// <remarks>
/// The tags in one peeked batch are independent gateway round-trips, so a sequential foreach makes
/// a pass cost <c>tags × latency</c>. The load-bearing observation is a POSITIVE one — two writes
/// genuinely in flight at the same instant — which is why the writer double blocks every call on a
/// TCS and records the peak in-flight count. Measuring wall-clock time instead would be a flake
/// generator and would not distinguish "parallel" from "fast".
/// <para>
/// The values arrive as ONE <see cref="VirtualTagActor.DependencyValueBatch"/> deliberately:
/// the per-value handler nudges a DrainTick after each append, so three separate messages could
/// start the drain on a batch of one and the pass would hold a single group — nothing to
/// parallelise. A batch message appends all three before any DrainTick is dequeued.
/// </para>
/// </remarks>
[Fact]
public async Task Drain_writes_distinct_tags_concurrently_within_the_window()
{
var mux = CreateTestProbe();
var writer = new GatingValueWriter();
var outbox = new InMemoryOutbox();
IActorRef rec = Sys.ActorOf(ContinuousHistorizationRecorder.Props(
mux.Ref, writer, outbox, historizedRefs: new[] { "A", "B", "C" },
drainInterval: TimeSpan.FromMilliseconds(50),
drainConcurrency: 8));
rec.Tell(new VirtualTagActor.DependencyValueBatch(new[]
{
new VirtualTagActor.DependencyValueChanged("A", 1.0, DateTime.UtcNow),
new VirtualTagActor.DependencyValueChanged("B", 2.0, DateTime.UtcNow),
new VirtualTagActor.DependencyValueChanged("C", 3.0, DateTime.UtcNow),
}));
await AwaitAssertAsync(
() =>
{
Assert.True(
writer.MaxObservedConcurrency >= 2,
$"expected overlapping writes; peak in-flight was {writer.MaxObservedConcurrency}");
return Task.CompletedTask;
},
TimeSpan.FromSeconds(10));
writer.Release();
// …and the pass still acks all-or-nothing: the whole peeked prefix leaves the outbox.
await AwaitAssertAsync(async () =>
{
var status = await rec.Ask<ContinuousHistorizationRecorder.RecorderStatus>(
ContinuousHistorizationRecorder.GetStatus.Instance, TimeSpan.FromSeconds(3));
Assert.Equal(0, status.QueuedDepth);
Assert.Equal(0, status.AbandonedEntries);
}, TimeSpan.FromSeconds(10));
}
/// <summary>
/// Falsification control: <c>DrainConcurrency = 1</c> is exactly the old sequential drain.
/// </summary>
/// <remarks>
/// Same fixture as the test above, so the only difference is the window size. Without this an
/// implementation that ignored the option entirely (always unbounded) would pass the concurrency
/// test and nothing would pin the bound.
/// </remarks>
[Fact]
public async Task Drain_concurrency_of_one_is_sequential()
{
var mux = CreateTestProbe();
var writer = new GatingValueWriter();
var outbox = new InMemoryOutbox();
IActorRef rec = Sys.ActorOf(ContinuousHistorizationRecorder.Props(
mux.Ref, writer, outbox, historizedRefs: new[] { "A", "B", "C" },
drainInterval: TimeSpan.FromMilliseconds(50),
drainConcurrency: 1));
rec.Tell(new VirtualTagActor.DependencyValueBatch(new[]
{
new VirtualTagActor.DependencyValueChanged("A", 1.0, DateTime.UtcNow),
new VirtualTagActor.DependencyValueChanged("B", 2.0, DateTime.UtcNow),
new VirtualTagActor.DependencyValueChanged("C", 3.0, DateTime.UtcNow),
}));
// The first write is in flight and blocked…
await AwaitAssertAsync(
() =>
{
Assert.True(writer.CallCount >= 1, "the drain must have started a write");
return Task.CompletedTask;
},
TimeSpan.FromSeconds(10));
// …and stays alone: with a window of 8 the other two groups would have entered by now.
await Task.Delay(TimeSpan.FromMilliseconds(300));
Assert.Equal(1, writer.MaxObservedConcurrency);
Assert.Equal(1, writer.CallCount);
writer.Release();
await AwaitAssertAsync(async () =>
{
var status = await rec.Ask<ContinuousHistorizationRecorder.RecorderStatus>(
ContinuousHistorizationRecorder.GetStatus.Instance, TimeSpan.FromSeconds(3));
Assert.Equal(0, status.QueuedDepth);
}, TimeSpan.FromSeconds(10));
Assert.Equal(1, writer.MaxObservedConcurrency);
}
// ---------------- Batched ingest: ONE group commit per value burst ----------------
/// <summary>
/// A whole driver burst delivered as one <see cref="VirtualTagActor.DependencyValueBatch"/> costs
/// exactly ONE durable append, and every value in it still reaches the writer.
/// </summary>
/// <remarks>
/// The load-bearing assertion is <c>AppendBatchCallCount == 1</c> paired with
/// <c>AppendCallCount == 0</c>. "All five values arrived" was already true of the loop-over-the-
/// per-value-handler batch path this replaces, so on its own it does not distinguish the two
/// implementations at all; the call counts are what state the actual change — five per-entry
/// fsyncs (production <c>FasterLogHistorizationOutbox</c> under <c>PerEntry</c> commit mode)
/// collapsed into one group commit.
/// </remarks>
[Fact]
public async Task A_burst_appends_once_and_drains_all_values()
{
var mux = CreateTestProbe();
var writer = new FakeValueWriter { Succeed = true };
var outbox = new InMemoryOutbox();
var refs = new[] { "T1", "T2", "T3", "T4", "T5" };
IActorRef rec = Sys.ActorOf(ContinuousHistorizationRecorder.Props(
mux.Ref, writer, outbox, historizedRefs: refs,
drainInterval: TimeSpan.FromMilliseconds(50)));
mux.ExpectMsg<DependencyMuxActor.RegisterInterest>();
var stamp = new DateTime(2026, 8, 15, 9, 0, 0, DateTimeKind.Utc);
rec.Tell(new VirtualTagActor.DependencyValueBatch(
refs.Select((r, i) => new VirtualTagActor.DependencyValueChanged(r, (double)(i + 1), stamp))
.ToArray()));
await AwaitAssertAsync(
() =>
{
var written = writer.Snapshot();
Assert.Equal(5, written.Count);
Assert.Equal(refs.OrderBy(x => x, StringComparer.Ordinal),
written.Select(w => w.Tag).OrderBy(x => x, StringComparer.Ordinal));
return Task.CompletedTask;
},
TimeSpan.FromSeconds(10));
Assert.Equal(1, outbox.AppendBatchCallCount);
Assert.Equal(0, outbox.AppendCallCount);
// …and the pass acked: the whole prefix left the outbox.
await AwaitAssertAsync(async () =>
{
var status = await rec.Ask<ContinuousHistorizationRecorder.RecorderStatus>(
ContinuousHistorizationRecorder.GetStatus.Instance, TimeSpan.FromSeconds(3));
Assert.Equal(0, status.QueuedDepth);
Assert.Equal(5, status.TotalRecorded);
}, TimeSpan.FromSeconds(10));
}
/// <summary>
/// The batched append must also survive genuinely yielding — the Gitea #491 bug class, on the
/// path that now owns the only await in the ingest handler.
/// </summary>
/// <remarks>
/// Mirrors <see cref="A_yielding_append_neither_faults_the_actor_nor_loses_the_value"/>, and for
/// the same reason: the ingest handler awaits
/// <see cref="IHistorizationOutbox.AppendBatchAsync"/> with <c>ConfigureAwait(false)</c>, so a
/// <c>Self</c> read after that await finds no ActorContext, throws
/// <see cref="NotSupportedException"/> into supervision, and RESTARTS the actor — which in
/// production discards every historized ref (the deployed set arrives later via
/// <see cref="ContinuousHistorizationRecorder.UpdateHistorizedRefs"/>).
/// <para>
/// The outbox double must complete on a POOL thread (<c>Task.Delay(1).ConfigureAwait(false)</c>),
/// which is what a real group-commit fsync does. A synchronously-completing fake — or a bare
/// <c>Task.Yield()</c>, which posts back to the CAPTURED ActorTaskScheduler context — leaves the
/// context installed and the defect invisible.
/// </para>
/// </remarks>
[Fact]
public async Task A_yielding_batch_append_neither_faults_the_actor_nor_loses_values()
{
var mux = CreateTestProbe();
var writer = new FakeValueWriter { Succeed = true };
var outbox = new InMemoryOutbox(yieldOnAppend: true);
IActorRef rec = Sys.ActorOf(ContinuousHistorizationRecorder.Props(
mux.Ref, writer, outbox, historizedRefs: new[] { "A", "B", "C" },
drainInterval: TimeSpan.FromMilliseconds(200)));
mux.ExpectMsg<DependencyMuxActor.RegisterInterest>();
var stamp = new DateTime(2026, 8, 15, 10, 0, 0, DateTimeKind.Utc);
rec.Tell(new VirtualTagActor.DependencyValueBatch(new[]
{
new VirtualTagActor.DependencyValueChanged("A", 1.0, stamp),
new VirtualTagActor.DependencyValueChanged("B", 2.0, stamp),
new VirtualTagActor.DependencyValueChanged("C", 3.0, stamp),
}));
// Every value in the burst reaches the writer…
await AwaitAssertAsync(
() =>
{
Assert.Equal(3, writer.Snapshot().Count);
return Task.CompletedTask;
},
TimeSpan.FromSeconds(10));
// …and — the load-bearing half — the actor is still the SAME incarnation, holding the ref set it
// was seeded with. A restart would reset TotalRecorded and re-register interest from PreStart.
var status = await rec.Ask<ContinuousHistorizationRecorder.RecorderStatus>(
ContinuousHistorizationRecorder.GetStatus.Instance, TimeSpan.FromSeconds(3));
Assert.Equal(3, status.TotalRecorded);
mux.ExpectNoMsg(TimeSpan.FromMilliseconds(300)); // no PreStart re-register == no restart
}
/// <summary>
/// A closed capture gate suppresses every item of a batch, and appends NOTHING (Gitea #537 on the
/// batched path).
/// </summary>
/// <remarks>
/// Extends <see cref="A_closed_capture_gate_suppresses_the_value"/> to the batch handler: the
/// rewrite runs the gate per item while collecting entries, so a batched path that hoisted the
/// gate read (or skipped it entirely) would silently double-record on the Secondary of every
/// redundant pair. <c>SuppressedNotPrimary == 3</c> is the positive observation — "the outbox is
/// empty" is equally true of a recorder that never ran.
/// </remarks>
[Fact]
public async Task The_capture_gate_suppresses_items_within_a_batch()
{
var mux = CreateTestProbe();
var writer = new FakeValueWriter();
var outbox = new InMemoryOutbox();
IActorRef rec = Sys.ActorOf(ContinuousHistorizationRecorder.Props(
mux.Ref, writer, outbox, historizedRefs: new[] { "A", "B", "C" },
captureGate: () => false));
var stamp = new DateTime(2026, 8, 15, 11, 0, 0, DateTimeKind.Utc);
rec.Tell(new VirtualTagActor.DependencyValueBatch(new[]
{
new VirtualTagActor.DependencyValueChanged("A", 1.0, stamp),
new VirtualTagActor.DependencyValueChanged("B", 2.0, stamp),
new VirtualTagActor.DependencyValueChanged("C", 3.0, stamp),
}));
await AwaitAssertAsync(async () =>
{
var status = await rec.Ask<ContinuousHistorizationRecorder.RecorderStatus>(
ContinuousHistorizationRecorder.GetStatus.Instance, TimeSpan.FromSeconds(3));
Assert.Equal(3, status.SuppressedNotPrimary);
Assert.Equal(0, status.TotalRecorded);
Assert.Equal(0, status.QueuedDepth);
});
Assert.Empty(writer.Snapshot());
Assert.Equal(0, outbox.AppendBatchCallCount);
Assert.Equal(0, outbox.AppendCallCount);
}
/// <summary>
/// One item whose gate step THROWS is skipped; the rest of the burst still records, and the actor
/// does not restart.
/// </summary>
/// <remarks>
/// Batching turns a per-item throw into a tail-dropper: unguarded, it escapes the handler with the
/// whole burst still uncommitted (the group commit happens after the loop), so every value —
/// including the ones already collected — is lost, and supervision restarts the actor, which in
/// production discards the historized-ref set. The realistic source is a faulting
/// <c>captureGate</c>, which is what this drives: the gate throws on exactly the SECOND of three
/// items.
/// <para>
/// The load-bearing assertion is that BOTH surviving values reached the writer — the one BEFORE
/// the fault and the one AFTER it. The trailing one alone would not distinguish the guard from
/// a restart that happened to re-drain, and the leading one alone is not evidence the loop
/// continued.
/// </para>
/// </remarks>
[Fact]
public async Task A_faulting_item_does_not_drop_the_rest_of_the_burst()
{
var mux = CreateTestProbe();
var writer = new FakeValueWriter { Succeed = true };
var outbox = new InMemoryOutbox();
var gateCalls = 0;
IActorRef rec = Sys.ActorOf(ContinuousHistorizationRecorder.Props(
mux.Ref, writer, outbox, historizedRefs: new[] { "A", "B", "C" },
drainInterval: TimeSpan.FromMilliseconds(50),
captureGate: () => Interlocked.Increment(ref gateCalls) == 2
? throw new InvalidOperationException("role view unavailable")
: true));
mux.ExpectMsg<DependencyMuxActor.RegisterInterest>();
var stamp = new DateTime(2026, 8, 15, 12, 0, 0, DateTimeKind.Utc);
rec.Tell(new VirtualTagActor.DependencyValueBatch(new[]
{
new VirtualTagActor.DependencyValueChanged("A", 1.0, stamp),
new VirtualTagActor.DependencyValueChanged("B", 2.0, stamp),
new VirtualTagActor.DependencyValueChanged("C", 3.0, stamp),
}));
await AwaitAssertAsync(
() =>
{
var written = writer.Snapshot();
Assert.Contains(written, w => w.Tag == "A");
Assert.Contains(written, w => w.Tag == "C");
Assert.DoesNotContain(written, w => w.Tag == "B");
return Task.CompletedTask;
},
TimeSpan.FromSeconds(10));
var status = await rec.Ask<ContinuousHistorizationRecorder.RecorderStatus>(
ContinuousHistorizationRecorder.GetStatus.Instance, TimeSpan.FromSeconds(3));
Assert.Equal(1, status.IngestFaults);
Assert.Equal(2, status.TotalRecorded);
// The fault is NOT a "value the historian cannot carry" — that counter must stay clean, or an
// ingest-dependency outage is indistinguishable from bad data on the /hosts surface.
Assert.Equal(0, status.DroppedNonNumeric);
mux.ExpectNoMsg(TimeSpan.FromMilliseconds(300)); // no PreStart re-register == no restart
}
/// <summary>
/// A failed group commit is metered ONCE for the whole burst, credits nothing to
/// <c>TotalRecorded</c>, leaves the actor alive, and STILL nudges the drain — which flushes whatever
/// prefix the outbox durably committed before it failed.
/// </summary>
/// <remarks>
/// Pins the batched ingest's append-failure catch branch, which had no coverage at all: every other
/// test here uses an outbox that always succeeds, so a catch that miscounted, swallowed the drain
/// nudge, or rethrew into supervision would go unnoticed.
/// <para>
/// Three separate claims, each of which is a distinct way the branch can be wrong:
/// </para>
/// <para>
/// (1) <c>OutboxAppendFailures == 1</c> for a THREE-item burst. The counter is per failed batch,
/// not per value — a deliberate granularity change from the per-value era — so a catch that
/// credited <c>entries.Count</c> would report 3 and make an operator read one bad fsync as three
/// separate faults.
/// </para>
/// <para>
/// (2) <c>TotalRecorded == 0</c> even though entry "A" IS durable. The outbox does not report how
/// far it got, so the recorder deliberately under-counts rather than over-claims; this asserts
/// the documented direction of the inaccuracy.
/// </para>
/// <para>
/// (3) The durable prefix reaches the writer. This is the load-bearing half — it is the positive
/// observation that the post-failure <c>DrainTick</c> was sent, and it is why the log message
/// must not claim the burst was "dropped". <c>drainInterval</c> is set to 30 s so the steady
/// cadence cannot fire inside the test: the only thing that can produce a drain here is the
/// nudge. Deleting <c>self.Tell(DrainTick.Instance)</c> from the failure path (or returning from
/// the catch) leaves "A" queued forever and turns this red.
/// </para>
/// </remarks>
[Fact]
public async Task A_failed_batch_append_counts_one_failure_and_still_ticks_the_drain()
{
var mux = CreateTestProbe();
var writer = new FakeValueWriter { Succeed = true };
var outbox = new InMemoryOutbox
{
// The partial-prefix shape: "A" commits durably, then the commit fails — so "B" and "C" are
// genuinely lost while "A" is merely un-drained.
AppendBatchFault = new IOException("simulated fsync failure (disk full)"),
AppendBatchFaultAfter = 1,
};
IActorRef rec = Sys.ActorOf(ContinuousHistorizationRecorder.Props(
mux.Ref, writer, outbox, historizedRefs: new[] { "A", "B", "C" },
drainInterval: TimeSpan.FromSeconds(30)));
mux.ExpectMsg<DependencyMuxActor.RegisterInterest>();
var stamp = new DateTime(2026, 8, 15, 13, 0, 0, DateTimeKind.Utc);
rec.Tell(new VirtualTagActor.DependencyValueBatch(new[]
{
new VirtualTagActor.DependencyValueChanged("A", 1.0, stamp),
new VirtualTagActor.DependencyValueChanged("B", 2.0, stamp),
new VirtualTagActor.DependencyValueChanged("C", 3.0, stamp),
}));
// Answering GetStatus at all is itself the "actor is still alive" observation — an append failure
// that reached supervision would have restarted (or stopped) it.
await AwaitAssertAsync(
async () =>
{
var status = await rec.Ask<ContinuousHistorizationRecorder.RecorderStatus>(
ContinuousHistorizationRecorder.GetStatus.Instance, TimeSpan.FromSeconds(3));
Assert.Equal(1, status.OutboxAppendFailures);
Assert.Equal(0, status.TotalRecorded);
},
TimeSpan.FromSeconds(10));
await AwaitAssertAsync(
() =>
{
Assert.True(outbox.PeekBatchCallCount >= 1, "the failure must still have nudged a drain");
Assert.Contains(writer.Snapshot(), w => w.Tag == "A" && w.Value == 1.0);
return Task.CompletedTask;
},
TimeSpan.FromSeconds(10));
// What never committed stays gone — the prefix drained, the remainder did not resurrect.
Assert.DoesNotContain(writer.Snapshot(), w => w.Tag == "B");
Assert.DoesNotContain(writer.Snapshot(), w => w.Tag == "C");
mux.ExpectNoMsg(TimeSpan.FromMilliseconds(300)); // no PreStart re-register == no restart
}
/// <summary>
/// An <see cref="OperationCanceledException"/> that is NOT shutdown is metered as an append failure,
/// not swallowed.
/// </summary>
/// <remarks>
/// The catch that quietly discards a cancelled append is filtered on
/// <c>when (_lifetime.IsCancellationRequested)</c> — i.e. only during the actor's own shutdown.
/// Drop the filter and every cancelled append becomes invisible: no metric, and no drain nudge for
/// the prefix. This drives the observable side of that filter (the lifetime is open here, so the
/// general catch must take it).
/// <para>
/// The shutdown side itself is deliberately not tested: <c>_lifetime</c> is cancelled in
/// <c>PostStop</c>, so reaching that branch means racing an append against a stopping actor —
/// and once it has stopped there is no <c>GetStatus</c> to read the counter from. There is no
/// cheap seam for it; the filter's correctness is pinned from this side instead.
/// </para>
/// </remarks>
[Fact]
public async Task A_cancellation_that_is_not_shutdown_is_metered_as_an_append_failure()
{
var mux = CreateTestProbe();
var writer = new FakeValueWriter { Succeed = true };
var outbox = new InMemoryOutbox
{
AppendBatchFault = new OperationCanceledException("not a shutdown"),
AppendBatchFaultAfter = 0,
};
IActorRef rec = Sys.ActorOf(ContinuousHistorizationRecorder.Props(
mux.Ref, writer, outbox, historizedRefs: new[] { "A" },
drainInterval: TimeSpan.FromSeconds(30)));
mux.ExpectMsg<DependencyMuxActor.RegisterInterest>();
rec.Tell(new VirtualTagActor.DependencyValueChanged(
"A", 1.0, new DateTime(2026, 8, 15, 14, 0, 0, DateTimeKind.Utc)));
await AwaitAssertAsync(
async () =>
{
var status = await rec.Ask<ContinuousHistorizationRecorder.RecorderStatus>(
ContinuousHistorizationRecorder.GetStatus.Instance, TimeSpan.FromSeconds(3));
Assert.Equal(1, status.OutboxAppendFailures);
Assert.Equal(0, status.TotalRecorded);
},
TimeSpan.FromSeconds(10));
mux.ExpectNoMsg(TimeSpan.FromMilliseconds(300)); // no PreStart re-register == no restart
}
/// <summary>
/// <see cref="IHistorianValueWriter"/> double that BLOCKS every call until <see cref="Release"/>,
/// recording the peak number of calls in flight simultaneously. That peak is what distinguishes a
/// bounded-parallel drain from a sequential one; a writer that returns synchronously (like
/// <see cref="FakeValueWriter"/>) can never observe overlap however parallel the caller is.
/// </summary>
private sealed class GatingValueWriter : IHistorianValueWriter
{
private readonly Lock _gate = new();
private readonly TaskCompletionSource _release = new(TaskCreationOptions.RunContinuationsAsynchronously);
private int _inFlight;
private int _maxObserved;
private int _calls;
/// <summary>Peak number of <see cref="WriteLiveValuesAsync"/> calls observed in flight at once.</summary>
public int MaxObservedConcurrency
{
get
{
lock (_gate)
{
return _maxObserved;
}
}
}
/// <summary>Lifetime count of <see cref="WriteLiveValuesAsync"/> invocations.</summary>
public int CallCount
{
get
{
lock (_gate)
{
return _calls;
}
}
}
/// <summary>Unblocks every pending and future call.</summary>
public void Release() => _release.TrySetResult();
public async Task<bool> WriteLiveValuesAsync(
string tag, IReadOnlyList<HistorizationValue> values, CancellationToken ct)
{
lock (_gate)
{
_calls++;
_inFlight++;
if (_inFlight > _maxObserved)
{
_maxObserved = _inFlight;
}
}
try
{
// A bounded wait rather than an unbounded one: a regression that never releases fails the
// assertion rather than hanging the whole test run.
await _release.Task.WaitAsync(TimeSpan.FromSeconds(30), ct).ConfigureAwait(false);
}
finally
{
lock (_gate)
{
_inFlight--;
}
}
return true;
}
}
/// <summary>In-memory <see cref="IHistorianValueWriter"/> double: records every value written and
/// returns <see cref="Succeed"/> as the ack. Thread-safe — the recorder drains off the actor thread.</summary>
private sealed class FakeValueWriter : IHistorianValueWriter
@@ -691,8 +1227,38 @@ public sealed class ContinuousHistorizationRecorderTests : TestKit
private readonly List<HistorizationOutboxEntry> _entries = new();
private readonly int _capacity;
private long _dropped;
private int _appendCalls;
private int _appendBatchCalls;
private readonly bool _yieldOnAppend;
private int _peekCalls;
/// <summary>When non-null, <see cref="AppendBatchAsync"/> durably appends the first
/// <see cref="AppendBatchFaultAfter"/> entries and then throws this — the production outbox's
/// "prefix, never orphan" contract on its failure path (an fsync hitting disk-full mid-commit).
/// The throw happens AFTER a pool-thread yield, as a real group-commit fsync does, so the
/// recorder's <c>ConfigureAwait(false)</c> continuation runs its catch block off the actor
/// thread — the Gitea #491 shape, on the failure path.</summary>
public Exception? AppendBatchFault { get; init; }
/// <summary>How many of the batch's entries are durably committed before
/// <see cref="AppendBatchFault"/> is thrown. 0 = nothing committed; a positive value models the
/// PARTIAL-PREFIX failure, where some entries are durable and will drain even though the append
/// as a whole reported failure.</summary>
public int AppendBatchFaultAfter { get; init; }
/// <summary>Lifetime count of <see cref="PeekBatchAsync"/> calls — one per drain pass, so a
/// non-zero value is positive evidence that a drain actually RAN.</summary>
public int PeekBatchCallCount
{
get
{
lock (_gate)
{
return _peekCalls;
}
}
}
/// <param name="capacity">Drop-oldest capacity; 0 means unbounded.</param>
/// <param name="yieldOnAppend">
@@ -722,8 +1288,39 @@ public sealed class ContinuousHistorizationRecorderTests : TestKit
}
}
/// <summary>Lifetime count of single-entry <see cref="AppendAsync"/> calls. Distinguishes the
/// batched ingest path from the per-value loop it replaced — both record the same values, so only
/// the call shape tells them apart.</summary>
public int AppendCallCount
{
get
{
lock (_gate)
{
return _appendCalls;
}
}
}
/// <summary>Lifetime count of <see cref="AppendBatchAsync"/> calls — one per group commit.</summary>
public int AppendBatchCallCount
{
get
{
lock (_gate)
{
return _appendBatchCalls;
}
}
}
public ValueTask AppendAsync(HistorizationOutboxEntry entry, CancellationToken ct)
{
lock (_gate)
{
_appendCalls++;
}
if (_yieldOnAppend)
{
return AppendYieldingAsync(entry);
@@ -733,6 +1330,65 @@ public sealed class ContinuousHistorizationRecorderTests : TestKit
return ValueTask.CompletedTask;
}
public ValueTask AppendBatchAsync(IReadOnlyList<HistorizationOutboxEntry> entries, CancellationToken ct)
{
lock (_gate)
{
_appendBatchCalls++;
}
if (AppendBatchFault is not null)
{
return AppendBatchFaultingAsync(entries);
}
if (entries.Count == 0)
{
return ValueTask.CompletedTask;
}
if (_yieldOnAppend)
{
return AppendBatchYieldingAsync(entries);
}
foreach (HistorizationOutboxEntry entry in entries)
{
AppendCore(entry);
}
return ValueTask.CompletedTask;
}
private async ValueTask AppendBatchFaultingAsync(IReadOnlyList<HistorizationOutboxEntry> entries)
{
// Complete on a POOL thread before faulting (the same requirement AppendYieldingAsync documents):
// a real group commit fsyncs, so the recorder's ConfigureAwait(false) continuation — here, its
// CATCH block — resumes with no ActorContext installed. A synchronous throw would run the catch
// inline on the actor thread and would not exercise that.
await Task.Delay(1).ConfigureAwait(false);
// "Prefix, never orphan": whatever was committed before the failure is durably tracked and
// ackable, which is exactly what makes the post-failure drain nudge meaningful.
for (var i = 0; i < AppendBatchFaultAfter && i < entries.Count; i++)
{
AppendCore(entries[i]);
}
throw AppendBatchFault!;
}
private async ValueTask AppendBatchYieldingAsync(IReadOnlyList<HistorizationOutboxEntry> entries)
{
// Same pool-thread-completion requirement as AppendYieldingAsync below — a group commit
// fsyncs once for the batch, so the yield happens once, before the entries are recorded.
await Task.Delay(1).ConfigureAwait(false);
foreach (HistorizationOutboxEntry entry in entries)
{
AppendCore(entry);
}
}
private async ValueTask AppendYieldingAsync(HistorizationOutboxEntry entry)
{
// Must complete on a POOL thread, carrying no synchronization context — that is what a real
@@ -764,6 +1420,7 @@ public sealed class ContinuousHistorizationRecorderTests : TestKit
{
lock (_gate)
{
_peekCalls++;
IReadOnlyList<HistorizationOutboxEntry> batch = _entries.Take(max).ToArray();
return ValueTask.FromResult(batch);
}
tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/ServiceCollectionExtensionsTests.cs
+3
View File
@@ -245,6 +245,9 @@ public sealed class ServiceCollectionExtensionsTests
public long DroppedCount => 0;
/// <summary>No-op append.</summary>
public ValueTask AppendAsync(HistorizationOutboxEntry entry, CancellationToken ct) => ValueTask.CompletedTask;
/// <summary>No-op batch append.</summary>
public ValueTask AppendBatchAsync(IReadOnlyList<HistorizationOutboxEntry> entries, CancellationToken ct)
=> ValueTask.CompletedTask;
/// <summary>Always returns an empty batch.</summary>
public ValueTask<IReadOnlyList<HistorizationOutboxEntry>> PeekBatchAsync(int max, CancellationToken ct)
=> ValueTask.FromResult<IReadOnlyList<HistorizationOutboxEntry>>(Array.Empty<HistorizationOutboxEntry>());
tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/VirtualTags/DependencyMuxActorTests.cs
+46 -14
View File
@@ -25,20 +25,21 @@ public sealed class DependencyMuxActorTests : RuntimeActorTestBase
mux.Tell(new DriverInstanceActor.AttributeValuePublished("driver-1", "tag-3", 30, OpcUaStatus.Good, DateTime.UtcNow));
mux.Tell(new DriverInstanceActor.AttributeValuePublished("driver-1", "tag-2", 20, OpcUaStatus.Good, DateTime.UtcNow));
// The mux emits ONLY batches — the per-value comms-loss ingress fans out as a single-item batch.
// subA hears tag-1 + tag-2.
var aMsgs = new[]
{
subA.ExpectMsg<VirtualTagActor.DependencyValueChanged>(),
subA.ExpectMsg<VirtualTagActor.DependencyValueChanged>(),
}.OrderBy(m => m.TagId).ToList();
subA.ExpectMsg<VirtualTagActor.DependencyValueBatch>(),
subA.ExpectMsg<VirtualTagActor.DependencyValueBatch>(),
}.Select(b => b.Items.ShouldHaveSingleItem()).OrderBy(m => m.TagId).ToList();
aMsgs.Select(m => m.TagId).ShouldBe(new[] { "tag-1", "tag-2" });
// subB hears tag-3 + tag-2.
var bMsgs = new[]
{
subB.ExpectMsg<VirtualTagActor.DependencyValueChanged>(),
subB.ExpectMsg<VirtualTagActor.DependencyValueChanged>(),
}.OrderBy(m => m.TagId).ToList();
subB.ExpectMsg<VirtualTagActor.DependencyValueBatch>(),
subB.ExpectMsg<VirtualTagActor.DependencyValueBatch>(),
}.Select(b => b.Items.ShouldHaveSingleItem()).OrderBy(m => m.TagId).ToList();
bMsgs.Select(m => m.TagId).ShouldBe(new[] { "tag-2", "tag-3" });
subA.ExpectNoMsg(TimeSpan.FromMilliseconds(100));
@@ -67,7 +68,7 @@ public sealed class DependencyMuxActorTests : RuntimeActorTestBase
mux.Tell(new DependencyMuxActor.RegisterInterest(new[] { "tag-1" }, sub.Ref));
mux.Tell(new DriverInstanceActor.AttributeValuePublished("driver-1", "tag-1", 10, OpcUaStatus.Good, DateTime.UtcNow));
sub.ExpectMsg<VirtualTagActor.DependencyValueChanged>();
sub.ExpectMsg<VirtualTagActor.DependencyValueBatch>();
mux.Tell(new DependencyMuxActor.UnregisterInterest(sub.Ref));
mux.Tell(new DriverInstanceActor.AttributeValuePublished("driver-1", "tag-1", 20, OpcUaStatus.Good, DateTime.UtcNow));
@@ -88,7 +89,7 @@ public sealed class DependencyMuxActorTests : RuntimeActorTestBase
sub.ExpectNoMsg(TimeSpan.FromMilliseconds(200));
mux.Tell(new DriverInstanceActor.AttributeValuePublished("driver-1", "tag-2", 20, OpcUaStatus.Good, DateTime.UtcNow));
sub.ExpectMsg<VirtualTagActor.DependencyValueChanged>().TagId.ShouldBe("tag-2");
sub.ExpectMsg<VirtualTagActor.DependencyValueBatch>().Items.ShouldHaveSingleItem().TagId.ShouldBe("tag-2");
}
/// <summary>Verifies that a virtual tag actor registers dependencies with the mux during PreStart and that evaluation fires end-to-end.</summary>
@@ -148,12 +149,13 @@ public sealed class DependencyMuxActorTests : RuntimeActorTestBase
new DriverInstanceActor.AttributeValueItem("ref-1", 42, OpcUaStatus.Good, DateTime.UtcNow),
}));
subscriber.ExpectMsg<VirtualTagActor.DependencyValueChanged>().TagId.ShouldBe("ref-1");
subscriber.ExpectMsg<VirtualTagActor.DependencyValueBatch>()
.Items.ShouldHaveSingleItem().TagId.ShouldBe("ref-1");
}
/// <summary>A batched driver burst (the primary ingress from DriverHostActor since the value-pipeline
/// batching work) is filtered per item: a subscriber hears exactly one DependencyValueChanged for the
/// ref it registered, carrying that item's own value/timestamp/quality, and nothing for the rest of the
/// batching work) is filtered per item: a subscriber hears one batch carrying exactly the item for the
/// ref it registered, with that item's own value/timestamp/quality, and nothing for the rest of the
/// burst.</summary>
[Fact]
public void AttributeValueBatchPublished_is_filtered_per_item_to_interested_subscribers()
@@ -170,14 +172,43 @@ public sealed class DependencyMuxActorTests : RuntimeActorTestBase
new DriverInstanceActor.AttributeValueItem("tag-x", 10, OpcUaStatus.BadNoCommunication, tsX),
}));
var dep = sub.ExpectMsg<VirtualTagActor.DependencyValueChanged>(PresenceBudget);
// Nothing for tag-y — no subscriber registered for it, so the batch carries tag-x alone.
var dep = sub.ExpectMsg<VirtualTagActor.DependencyValueBatch>(PresenceBudget).Items.ShouldHaveSingleItem();
dep.TagId.ShouldBe("tag-x");
dep.Value.ShouldBe(10);
dep.TimestampUtc.ShouldBe(tsX);
// #519: the item's own status sub-code rides through, not a Good/Bad bucket.
dep.StatusCode.ShouldBe(OpcUaStatus.BadNoCommunication);
// Nothing for tag-y — no subscriber registered for it.
sub.ExpectNoMsg(TimeSpan.FromMilliseconds(200));
}
/// <summary>One driver burst costs each subscriber exactly ONE mailbox message, not one per matched
/// item: the mux accumulates each subscriber's matched slice and Tells it as a single
/// <c>DependencyValueBatch</c>. The unmatched item never reaches the subscriber, and no second message
/// follows the batch.</summary>
[Fact]
public void A_burst_reaches_each_subscriber_as_one_batch_message()
{
var mux = Sys.ActorOf(DependencyMuxActor.Props());
var sub = CreateTestProbe();
mux.Tell(new DependencyMuxActor.RegisterInterest(new[] { "a", "b" }, sub.Ref));
var tsA = new DateTime(2026, 8, 15, 10, 0, 0, DateTimeKind.Utc);
var tsB = new DateTime(2026, 8, 15, 10, 0, 1, DateTimeKind.Utc);
mux.Tell(new DriverInstanceActor.AttributeValueBatchPublished("driver-1", new[]
{
new DriverInstanceActor.AttributeValueItem("a", 1, OpcUaStatus.Good, tsA),
new DriverInstanceActor.AttributeValueItem("b", 2, OpcUaStatus.Good, tsB),
new DriverInstanceActor.AttributeValueItem("c", 3, OpcUaStatus.Good, tsB),
}));
var batch = sub.ExpectMsg<VirtualTagActor.DependencyValueBatch>(PresenceBudget);
batch.Items.Select(i => i.TagId).ShouldBe(new[] { "a", "b" });
batch.Items.Select(i => i.Value).ShouldBe(new object?[] { 1, 2 });
batch.Items.Select(i => i.TimestampUtc).ShouldBe(new[] { tsA, tsB });
// The unmatched "c" produced nothing, and the two matched items did NOT arrive as two messages.
sub.ExpectNoMsg(TimeSpan.FromMilliseconds(200));
}
@@ -193,7 +224,8 @@ public sealed class DependencyMuxActorTests : RuntimeActorTestBase
mux.Tell(new DriverInstanceActor.AttributeValuePublished(
"driver-1", "tag-1", 10, OpcUaStatus.Bad, DateTime.UtcNow));
sub.ExpectMsg<VirtualTagActor.DependencyValueChanged>().StatusCode.ShouldBe(OpcUaStatus.Bad);
sub.ExpectMsg<VirtualTagActor.DependencyValueBatch>()
.Items.ShouldHaveSingleItem().StatusCode.ShouldBe(OpcUaStatus.Bad);
}
private sealed class EchoSumEvaluator : ZB.MOM.WW.OtOpcUa.Commons.Engines.IVirtualTagEvaluator
tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/VirtualTags/VirtualTagActorTests.cs
+71
View File
@@ -205,6 +205,77 @@ public sealed class VirtualTagActorTests : RuntimeActorTestBase
parent.ExpectMsg<VirtualTagActor.EvaluationResult>().StatusCode.ShouldBe(OpcUaStatus.BadInternalError);
}
/// <summary>A batch carrying K changed inputs evaluates the script ONCE, with every input already
/// applied — not once per input. The emitted result is computed from all three values (7 = 1+2+4), so
/// the collapse cannot be an evaluation that simply missed the later items.</summary>
[Fact]
public void A_batch_touching_K_inputs_evaluates_once()
{
var parent = CreateTestProbe();
var evaluator = new CountingSumEvaluator();
var actor = parent.ChildActorOf(VirtualTagActor.Props("vt-1", "a+b+c", evaluator: evaluator));
var ts = new DateTime(2026, 8, 15, 10, 0, 0, DateTimeKind.Utc);
actor.Tell(new VirtualTagActor.DependencyValueBatch(new[]
{
new VirtualTagActor.DependencyValueChanged("a", 1, ts.AddSeconds(-2)),
new VirtualTagActor.DependencyValueChanged("b", 2, ts.AddSeconds(-1)),
new VirtualTagActor.DependencyValueChanged("c", 4, ts),
}));
var result = parent.ExpectMsg<VirtualTagActor.EvaluationResult>();
result.Value.ShouldBe(7);
// The single evaluation is stamped with the newest item in the burst.
result.TimestampUtc.ShouldBe(ts);
parent.ExpectNoMsg(TimeSpan.FromMilliseconds(200));
evaluator.Calls.ShouldBe(1);
}
/// <summary>Falsification control for <see cref="A_batch_touching_K_inputs_evaluates_once"/>: the SAME
/// three values delivered as three separate <c>DependencyValueChanged</c> messages evaluate three times
/// and emit three results (1 → 3 → 7, all distinct so the value dedup cannot collapse them). Without
/// this, a batch test that counted 1 would also pass against a build where dedup — not the batch
/// apply-then-evaluate split — did the collapsing.</summary>
[Fact]
public void The_same_values_sent_singly_evaluate_once_per_message()
{
var parent = CreateTestProbe();
var evaluator = new CountingSumEvaluator();
var actor = parent.ChildActorOf(VirtualTagActor.Props("vt-1", "a+b+c", evaluator: evaluator));
var ts = new DateTime(2026, 8, 15, 10, 0, 0, DateTimeKind.Utc);
actor.Tell(new VirtualTagActor.DependencyValueChanged("a", 1, ts.AddSeconds(-2)));
actor.Tell(new VirtualTagActor.DependencyValueChanged("b", 2, ts.AddSeconds(-1)));
actor.Tell(new VirtualTagActor.DependencyValueChanged("c", 4, ts));
parent.ExpectMsg<VirtualTagActor.EvaluationResult>().Value.ShouldBe(1);
parent.ExpectMsg<VirtualTagActor.EvaluationResult>().Value.ShouldBe(3);
parent.ExpectMsg<VirtualTagActor.EvaluationResult>().Value.ShouldBe(7);
evaluator.Calls.ShouldBe(3);
}
/// <summary>Test evaluator that sums integer dependencies and counts how many times it was called.</summary>
private sealed class CountingSumEvaluator : IVirtualTagEvaluator
{
private int _calls;
/// <summary>Gets the number of <see cref="Evaluate"/> calls observed so far.</summary>
public int Calls => Volatile.Read(ref _calls);
/// <summary>Counts the call and evaluates the expression by summing integer dependencies.</summary>
/// <param name="id">The tag identifier.</param>
/// <param name="expr">The expression string.</param>
/// <param name="deps">The dependency values.</param>
/// <returns>The sum of integer values in the dependencies.</returns>
public VirtualTagEvalResult Evaluate(string id, string expr, IReadOnlyDictionary<string, object?> deps)
{
Interlocked.Increment(ref _calls);
return VirtualTagEvalResult.Ok(deps.Values.OfType<int>().Sum());
}
}
/// <summary>Test evaluator returning a fixed sequence of results, one per Evaluate call; the last
/// result repeats once the queue is drained.</summary>
private sealed class QueuedEvaluator : IVirtualTagEvaluator