docs: record value-pipeline batching (plan 2026-08-15)
- docs/v2/Galaxy.Performance.md gains a "Value-path batching (2026-08-15)" section: the batch seams (IBatchSubscribable / PollGroupEngine onBatch -> AttributeValueBatchPublished -> AttributeValueBatchUpdate -> sink WriteValues with 512-chunked single-lock batches in the node manager), the invariants (#519 status codes verbatim, per-ITEM metering, Modbus deadband honored via the driver-filtered buffer, the ValueWriteBatchException partial-failure contract), the RealmKey re-key, and the pinned dispatcher's throughput 1 -> 100. - The plan doc gains a "Follow-ups (recorded, not implemented)" section: the PollGroupEngine unconditional-collection trap (a future per-tag filter in S7/AbCip/FOCAS would silently reintroduce the deadband-bypass bug), a two-level driverId -> (rawPath -> refs) routing map, null/empty guard consistency on the internal batch messages, a possible DriverHostActorLiveValueTests merge, and the pre-existing aborted-cycle tail-loss on the three unbuffered drivers. - tasks.json: all 13 tasks completed. CLAUDE.md needed no edit: its only statement in this area ("One driver publish for a RawPath fans (in DriverHostActor) to the raw NodeId AND every referencing UNS NodeId") is still true — batching changed the message granularity, not the fan-out semantics. Verification: dotnet build ZB.MOM.WW.OtOpcUa.slnx clean (0 errors, no new warning classes, zero warnings in any production file). 9 unit projects green, 3470 passed / 5 skipped / 0 failed: Core.Abstractions 305, Core 351+1s, Commons 348, Runtime 575+3s, OpcUaServer 424+1s, Modbus 399, S7 302, AbCip 410, FOCAS 356. No DB-backed suite was run — tests/Server/ZB.MOM.WW.OtOpcUa.Server.Tests does not exist, and nothing this branch touches is covered by the SQL-backed projects.
This commit is contained in:
@@ -181,3 +181,56 @@ Safety: reuse is serialised under `_evalGate`, so two threads can never observe
|
||||
- `Reevaluation_reuses_the_same_read_cache_dictionary` — asserts dictionary instance identity across two evaluations.
|
||||
- `Reevaluation_reuses_the_same_predicate_context` — same, for the context.
|
||||
- `LoadAsync_drops_the_prior_generations_scratch` — asserts a publish resets the scratch.
|
||||
|
||||
## Value-path batching (2026-08-15)
|
||||
|
||||
The hot value path is batched end to end: one message per driver production burst (a poll cycle, an
|
||||
inbound message, a stream chunk) instead of one per changed tag at every hop, one SDK-lock acquisition
|
||||
per batch instead of one per value, and no per-write key allocation in the node manager. Nothing about
|
||||
*what* is published changed — only how many messages and lock takes it costs. Plan +
|
||||
task-by-task rationale: [`docs/plans/2026-08-15-value-pipeline-batching.md`](../plans/2026-08-15-value-pipeline-batching.md).
|
||||
|
||||
**The seams, in order:**
|
||||
|
||||
- **`IBatchSubscribable` / `PollGroupEngine`'s `onBatch`** — an optional driver capability raising
|
||||
`OnDataChangeBatch` once per burst. The four engine-based poll drivers (Modbus, S7, AbCip, FOCAS) get
|
||||
it from the engine's batch callback. The per-tag `ISubscribable.OnDataChange` is unchanged and still
|
||||
raised — Galaxy cold reads and the driver CLIs subscribe it. A consumer subscribes to **exactly one**
|
||||
of the two; subscribing both double-delivers.
|
||||
- **`DriverInstanceActor` → `AttributeValueBatchPublished`** — one per burst, preferring the batch event
|
||||
when the driver implements `IBatchSubscribable` and wrapping a non-batch driver's per-tag events as
|
||||
single-item batches, so downstream sees one shape. The detach path tears the batch handler down too
|
||||
(load-bearing: the `IDriver` outlives the actor across host respawns).
|
||||
- **`DriverHostActor` → `AttributeValueBatchUpdate`** — resolves the whole batch's `NodeRealmRef`s
|
||||
(raw NodeId + every referencing UNS NodeId) and sends **one** message to `OpcUaPublishActor`, plus one
|
||||
batch Tell to the dependency mux. The mux keeps its per-item receive — the comms-loss fan-out
|
||||
fabricates per-item messages.
|
||||
- **`OtOpcUaNodeManager.WriteValues`** — the sink's batch override takes the manager-wide `Lock` once per
|
||||
chunk of **512** items rather than once per value, with each item isolated so one bad node cannot lose
|
||||
the rest of the batch.
|
||||
|
||||
**Invariants the batching preserves — do not break them:**
|
||||
|
||||
- **#519 status codes ride verbatim.** The full `uint` status code, sub-code included, reaches the SDK
|
||||
unchanged at every hop; batching moved the values, it did not re-derive them. Timestamps likewise come
|
||||
from the driver — nothing on this path stamps `DateTime.UtcNow`.
|
||||
- **Metering stays per ITEM, not per batch** (`OpcUaPublishActor`'s write tally + the
|
||||
`OtOpcUaTelemetry.OpcUaSinkWrite` meter). Counting per message would have silently dropped the
|
||||
dashboards by up to 100×.
|
||||
- **Deadband is honored on the batch event.** ⚠️ `PollGroupEngine` collects its batch list
|
||||
**unconditionally** — it has no knowledge of a driver's publish filter — so `ModbusDriver` ignores the
|
||||
engine's list and raises from its own buffer of what actually passed `ShouldPublish`, using `onBatch`
|
||||
purely as the end-of-cycle signal. Forwarding the engine's list instead would carry changes the per-tag
|
||||
event suppressed, and a consumer preferring the batch event would see deadband silently stop working.
|
||||
- **Partial failure is typed, not swallowed.** A sink that applies some of a batch and fails the rest
|
||||
reports `ValueWriteBatchException` (first failed NodeId + failed/succeeded counts); the publish actor's
|
||||
typed catch advances the tally by the **succeeded** count — not zero (the general catch's behaviour) and
|
||||
not the whole batch.
|
||||
|
||||
**Two supporting changes:** the node manager's variable / folder / alarm / historized-tagname maps are
|
||||
re-keyed by a `RealmKey(Realm, Id)` struct, so a write no longer allocates `new NodeId(id, ns).ToString()`
|
||||
per value (the realm still travels explicitly — it is never parsed back out of an id string). And the
|
||||
pinned `opcua-synchronized-dispatcher`'s `throughput` rose **1 → 100**: the actor keeps its own dedicated
|
||||
thread (that is what the `PinnedDispatcher` guarantees, not `throughput`), so this only removes a mailbox
|
||||
re-schedule round-trip per message on the hottest actor in the process, with zero change to concurrency.
|
||||
Keep it in sync with the inline HOCON copy in `ServiceCollectionExtensions.WithOtOpcUaRuntimeActors`.
|
||||
|
||||
Reference in New Issue
Block a user