# Value-Pipeline Batching Implementation Plan > **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task. **Goal:** Batch the hot value path end to end — one message per poll cycle instead of one per tag at every hop (driver → DriverInstanceActor → DriverHostActor → OpcUaPublishActor → node manager), one SDK-lock acquisition per batch instead of per value, and zero per-write key allocations in the node manager. **Architecture:** A new optional `IBatchSubscribable` capability lets a driver raise one `OnDataChangeBatch` per production burst (poll cycle); `PollGroupEngine` gains a batch callback so the four engine-based poll drivers get it for free. `DriverInstanceActor` forwards one `AttributeValueBatchPublished` per burst (wrapping per-tag events from non-batch drivers as single-item batches); `DriverHostActor` resolves the whole batch's NodeRealmRefs and sends ONE `AttributeValueBatchUpdate` to `OpcUaPublishActor`, plus ONE batch Tell to the dependency mux. The sink gains `WriteValues(IReadOnlyList)` (default-interface-method fallback loops `WriteValue`, so fakes/Null sink are untouched); `OtOpcUaNodeManager` overrides it taking `Lock` once per batch, and its node maps are re-keyed by a `(realm, id)` struct so the per-write `new NodeId(...).ToString()` allocations disappear. The pinned dispatcher's `throughput` rises from 1 to 100 (same single thread, fewer mailbox re-schedules). **Tech Stack:** C# / .NET 10, Akka.NET (classic actors), OPC Foundation UA .NET Standard, xUnit + Shouldly, Akka.TestKit. **Branch:** `perf/value-pipeline-batching` (off `master`). --- ## Context for implementers (read before any task) Verified facts about the current code — do not re-derive, and do not violate the invariants: - **The per-tag path today:** `PollGroupEngine.PollOnceAsync` (`src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/PollGroupEngine.cs:202-226`) diffs per tag and invokes `_onChange` per changed tag. Drivers forward that to `ISubscribable.OnDataChange` (per-tag event, `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/ISubscribable.cs:36`). `DriverInstanceActor.HandleSubscribeAsync` attaches `_dataChangeHandler` at `DriverInstanceActor.cs:880` (one `DataChangeForward` Tell per tag), `OnDataChangeForward` (`:1138-1143`) Tells parent one `AttributeValuePublished` per tag. `DriverHostActor.ForwardToMux` (`DriverHostActor.cs:976-1006`) Tells the mux per value and Tells `OpcUaPublishActor` one `AttributeValueUpdate` **per NodeRealmRef** (UNS fan-out at `:997-999`). `OpcUaPublishActor.HandleAttributeUpdate` (`OpcUaPublishActor.cs:319-331`) calls `_sink.WriteValue` per message. `OtOpcUaNodeManager.WriteValue` (`OtOpcUaNodeManager.cs:508-533`) takes the manager-wide `lock (Lock)` per value and allocates via `MapKey(realm, id)` = `new NodeId(id, ns).ToString()` (`:133`). - **Invariants to preserve:** 1. **#519 status codes ride verbatim** — the full `uint` status code, sub-code included, must reach the SDK unchanged at every hop. 2. **Timestamps are carried from the driver** — never stamp `DateTime.UtcNow` on the value path. 3. **PollGroupEngine's no-overlap guarantee** (delay-after-completion, one loop task per subscription) must not change in this plan. 4. **Per-tag `OnDataChange` keeps working** — Galaxy's cold-read path and driver CLIs subscribe it. A batch-capable driver raises BOTH events (batch once per cycle + per-tag per item); a consumer subscribes to exactly ONE of the two. `DriverInstanceActor` prefers batch when the driver implements `IBatchSubscribable`. 5. **`DriverInstanceActor.DetachSubscription`** (`:934-943`) must detach the batch handler too — the detach is load-bearing (the `IDriver` outlives the actor across respawns; see CLAUDE.md §DeployWatcher). 6. **The mux's per-item receive stays** — `DriverHostActor.FanCommsLossToDependencyMux` (`:1135-1142`) fabricates per-item `AttributeValuePublished` messages; `DependencyMuxActor` keeps that receive and gains a batch receive alongside. 7. **`OpcUaPublishActor` keeps its per-item `AttributeValueUpdate` receive** — `VirtualTagHostActor` and the alarm paths still send it. The batch message is additive. 8. **The pinned dispatcher stays pinned** — the OPC UA SDK requires a single thread. Only `throughput` changes. 9. **`_writes` increments** (`OpcUaPublishActor.cs:324`) and the `OtOpcUaTelemetry.OpcUaSinkWrite` meter must count per-VALUE (per item), not per batch, so dashboards don't drop by 100×. - **Message-record homes:** `AttributeValuePublished` is declared in `DriverInstanceActor` (`:109`); `AttributeValueUpdate` in `OpcUaPublishActor` (`:52`); `NodeRealmRef` is a private struct of `DriverHostActor` (`:183`). - **Sink implementors that must keep compiling:** `NullOpcUaAddressSpaceSink` (same file as the interface), `DeferredAddressSpaceSink` (`src/Core/ZB.MOM.WW.OtOpcUa.Commons/OpcUa/DeferredAddressSpaceSink.cs`), `SdkAddressSpaceSink` (`src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/SdkAddressSpaceSink.cs`), plus test fakes. Using a **default interface method** for `WriteValues` means only the ones that want batching override it. - **Commands:** build `dotnet build ZB.MOM.WW.OtOpcUa.slnx`; single project `dotnet test tests//`; single test `dotnet test --filter "FullyQualifiedName~."`. DB-backed suites need the central SQL Server — on this Mac, prefer running the specific unit projects named per task, not the whole solution, until the final task. - **Commit style:** follow recent history (`feat(...)`, `fix(...)`, `chore(...)` with a scope). One commit per task. --- ### Task 1: Batch contract types in Core.Abstractions **Classification:** small **Estimated implement time:** ~3 min **Parallelizable with:** Task 10 **Files:** - Modify: `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/ISubscribable.cs` - Test: `tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests/DataChangeBatchContractTests.cs` (create) **Step 1: Write the failing test** ```csharp using Shouldly; using ZB.MOM.WW.OtOpcUa.Core.Abstractions; using Xunit; namespace ZB.MOM.WW.OtOpcUa.Core.Tests; public class DataChangeBatchContractTests { private sealed record FakeHandle(string DiagnosticId) : ISubscriptionHandle; [Fact] public void Batch_args_carry_handle_and_items() { var handle = new FakeHandle("h1"); var snap = new DataValueSnapshot(42, 0u, DateTime.UtcNow, DateTime.UtcNow); var args = new DataChangeBatchEventArgs(handle, new[] { new DataChangeItem("plc/tag1", snap) }); args.SubscriptionHandle.ShouldBe(handle); args.Items.Count.ShouldBe(1); args.Items[0].FullReference.ShouldBe("plc/tag1"); args.Items[0].Snapshot.ShouldBe(snap); } } ``` Note: check `DataValueSnapshot`'s actual constructor shape in `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/` first and adjust the test's construction to it (positional record with Value/StatusCode/timestamps). Do not change `DataValueSnapshot`. **Step 2: Run test to verify it fails** Run: `dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests --filter "FullyQualifiedName~DataChangeBatchContractTests"` Expected: FAIL — `DataChangeBatchEventArgs` / `DataChangeItem` not defined. **Step 3: Implement** Append to `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/ISubscribable.cs`: ```csharp /// One changed attribute within a . /// Driver-side full reference of the changed attribute. /// New value + quality + timestamps. public sealed record DataChangeItem(string FullReference, DataValueSnapshot Snapshot); /// Event payload for — every attribute /// that changed within ONE production burst (a poll cycle, an inbound message, a stream chunk). /// The handle returned by the original call. /// The changed attributes, one entry per changed tag. Never empty. public sealed record DataChangeBatchEventArgs( ISubscriptionHandle SubscriptionHandle, IReadOnlyList Items); /// /// Optional driver capability: data changes delivered one event per production burst instead of /// one event per tag. A driver implementing this raises once per /// burst AND continues raising the per-tag for each item /// (legacy consumers: Galaxy cold reads, driver CLIs). A consumer subscribes to EXACTLY ONE of the /// two events — subscribing both double-delivers. DriverInstanceActor prefers the batch event. /// public interface IBatchSubscribable : ISubscribable { /// Batched data-change notification — one raise per production burst. event EventHandler? OnDataChangeBatch; } ``` **Step 4: Run test to verify it passes** Run: same filter. Expected: PASS. **Step 5: Commit** ```bash git add src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/ISubscribable.cs tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests/DataChangeBatchContractTests.cs git commit -m "feat(core): add IBatchSubscribable batch data-change contract" ``` --- ### Task 2: PollGroupEngine batch callback **Classification:** standard **Estimated implement time:** ~5 min **Parallelizable with:** none (Task 1 first) **Files:** - Modify: `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/PollGroupEngine.cs` - Test: `tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests/PollGroupEngineBatchTests.cs` (create) **Step 1: Write the failing tests** (find the existing PollGroupEngine test file first — if one exists in `tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests`, put these in a new class beside it and copy its reader-fake idiom): Test cases (write all four): 1. `Batch_fires_once_per_cycle_with_only_changed_items` — reader returns 3 refs; first poll (initial push) batch has all 3; then reader changes only 1 value; next batch has exactly that 1 item; per-tag `onChange` also fired for the same items (both callbacks see identical content). 2. `No_batch_when_nothing_changed` — second poll with identical snapshots ⇒ no batch invocation (and no per-tag invocation). 3. `Batch_is_not_invoked_when_callback_absent` — construct without `onBatch`; engine behaves exactly as today (per-tag only). Assert per-tag count. 4. `Batch_items_are_in_reader_order`. Use a short interval (e.g. 30 ms) + `TaskCompletionSource`/polling-with-timeout to await callbacks, mirroring the existing engine tests' style. Never `Thread.Sleep` long constants. **Step 2: Run to verify failure** Run: `dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests --filter "FullyQualifiedName~PollGroupEngineBatchTests"` Expected: FAIL — constructor has no `onBatch` parameter. **Step 3: Implement** In `PollGroupEngine`: - Add field `private readonly Action>? _onBatch;` - Add optional ctor param `Action>? onBatch = null` (AFTER the existing optional params, so every existing call site compiles unchanged) and assign it. - In `PollOnceAsync` (currently `:202-226`): collect changed items lazily and raise the batch after the loop: ```csharp List? changed = null; for (var i = 0; i < state.TagReferences.Count; i++) { var tagRef = state.TagReferences[i]; var current = snapshots[i]; var lastSeen = state.LastValues.TryGetValue(tagRef, out var prev) ? prev : default; if (forceRaise || ValuesAreDifferent(lastSeen?.Value, current.Value) || lastSeen?.StatusCode != current.StatusCode) { state.LastValues[tagRef] = current; _onChange(state.Handle, tagRef, current); if (_onBatch is not null) (changed ??= new List()).Add(new DataChangeItem(tagRef, current)); } } if (changed is { Count: > 0 }) _onBatch!(state.Handle, changed); ``` Keep the surrounding contract-violation check and everything else byte-identical. Update the class remarks to document the batch callback (one raise per cycle, only when non-empty, after all per-tag callbacks of that cycle). **Step 4: Run tests** Run: `dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests` (whole project — the existing engine tests must stay green). Expected: PASS. **Step 5: Commit** ```bash git add -A && git commit -m "feat(core): PollGroupEngine optional per-cycle batch callback" ``` --- ### Task 3a: Modbus + S7 raise the batch event **Classification:** standard **Estimated implement time:** ~5 min **Parallelizable with:** Task 3b, Task 5 **Files:** - Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/ModbusDriver.cs` - Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7/S7Driver.cs` - Test: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.Tests/ModbusBatchSubscriptionTests.cs` (create; mirror the fixture idiom of `ModbusSubscriptionTests.cs`) **Step 1: Write the failing test** — subscribe via the driver against the existing in-test transport fake (copy `ModbusSubscriptionTests`' setup); assert: 1. `ModbusDriver` is `IBatchSubscribable`. 2. After one poll cycle with N changed registers, `OnDataChangeBatch` fired once with N items AND `OnDataChange` fired N times (both events, identical content — contract from Task 1). **Step 2:** Run: `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.Tests --filter "FullyQualifiedName~ModbusBatchSubscriptionTests"` — FAIL (interface not implemented). **Step 3: Implement** — in each driver: - Change the class declaration to implement `IBatchSubscribable` (it already implements `ISubscribable`). - Add `public event EventHandler? OnDataChangeBatch;` - Find the `new PollGroupEngine(...)` construction; pass `onBatch: (handle, items) => OnDataChangeBatch?.Invoke(this, new DataChangeBatchEventArgs(handle, items))`. Per-tag raising is already wired via the engine's `onChange` — leave it. **Step 4:** Run the Modbus + S7 test projects: `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.Tests && dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7.Tests` — PASS (all existing tests too). **Step 5:** `git add -A && git commit -m "feat(drivers): Modbus + S7 implement IBatchSubscribable via PollGroupEngine batch callback"` **Deviation (landed):** Modbus does **not** forward the engine's `items` argument — it ignores it and raises the batch from its own per-cycle buffer of the items that passed `ShouldPublish`. The engine collects its batch list unconditionally and has no knowledge of the driver's deadband filter, so a straight pass-through would carry changes the per-tag event suppressed, and a consumer that preferred the batch event would see deadband silently stop working (review finding). `onBatch` is therefore used only as the end-of-cycle signal, draining the buffer; a cycle whose every change was suppressed raises nothing. Because that buffer is the only piece of cross-cycle state any of the four poll drivers holds, the driver's `onChange` bridge also wraps `OnDataChange?.Invoke(...)` in a swallowing try/catch (mirroring `PollGroupEngine.ReportError`): the engine invokes the bridge unguarded, so a throwing external subscriber would unwind the cycle before `onBatch` ran and the undrained items would merge into a later cycle's batch. The buffer append stays outside the catch — the item was published as far as the batch contract is concerned; only the per-tag subscriber faulted. S7/AbCip/FOCAS forward the engine's list as this task describes and hold no such state. --- ### Task 3b: AbCip + FOCAS raise the batch event **Classification:** standard **Estimated implement time:** ~4 min **Parallelizable with:** Task 3a, Task 5 **Files:** - Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.AbCip/AbCipDriver.cs` - Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.FOCAS/FocasDriver.cs` Same mechanical change as Task 3a (interface + event + `onBatch` at the `new PollGroupEngine(...)` site). If either driver constructs the engine differently (check first), adapt but keep the identical contract. No new per-driver test required (the engine behavior is pinned by Task 2; the wiring pattern is pinned by Task 3a's test) — but the projects' existing suites must stay green. Run: `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.AbCip.Tests && dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Tests` Commit: `feat(drivers): AbCip + FOCAS implement IBatchSubscribable` --- ### Task 4: DriverInstanceActor batch forwarding **Classification:** high-risk (actor model, subscription lifecycle) **Estimated implement time:** ~5 min **Parallelizable with:** Task 5 **Files:** - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverInstanceActor.cs` - Test: `tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/DriverInstanceActorBatchForwardTests.cs` (create; copy the TestKit + stub-driver idiom from the existing `DriverInstanceActor*Tests` in that directory) **Step 1: Write the failing tests** 1. `Batch_driver_forwards_one_batch_message` — stub driver implementing `IBatchSubscribable`; raise `OnDataChangeBatch` with 3 items after subscribe; parent probe receives exactly ONE `AttributeValueBatchPublished` with 3 items (driver id + per-item FullReference/Value/StatusCode/timestamp mapped as `OnDataChangeForward` maps them today: `ts = Snapshot.SourceTimestampUtc ?? Snapshot.ServerTimestampUtc`). 2. `Batch_driver_per_tag_event_not_double_forwarded` — the stub raises BOTH events (per contract); parent receives only the batch message, no legacy `AttributeValuePublished`. 3. `Per_tag_driver_forwards_single_item_batches` — stub implementing only `ISubscribable`; one `OnDataChange` ⇒ parent receives one `AttributeValueBatchPublished` with 1 item. 4. `Detach_removes_batch_handler` — after Unsubscribe (or Connected→Reconnecting, whichever the existing detach tests exercise), raising the batch event produces no message. **Step 2:** Run: `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests --filter "FullyQualifiedName~DriverInstanceActorBatchForwardTests"` — FAIL. **Step 3: Implement** - Add records next to `AttributeValuePublished` (`:109`): ```csharp /// One value within an . public sealed record AttributeValueItem(string FullReference, object? Value, uint StatusCode, DateTime TimestampUtc); /// Batched form of — every changed attribute of ONE /// driver production burst in one message. The parent fans the whole batch to the mux and the OPC UA /// publish path in one hop each. Status codes ride verbatim (#519). public sealed record AttributeValueBatchPublished(string DriverInstanceId, IReadOnlyList Items); ``` - Private message `private sealed record DataChangeBatchForward(IReadOnlyList Items);` + field `private EventHandler? _batchChangeHandler;` - In `HandleSubscribeAsync` (`:878-882`): when `subscribable is IBatchSubscribable batchable`, attach ONLY the batch handler (`_batchChangeHandler = (_, args) => self.Tell(new DataChangeBatchForward(args.Items)); batchable.OnDataChangeBatch += _batchChangeHandler;`) and do NOT attach `_dataChangeHandler`; else attach `_dataChangeHandler` exactly as today. - `Receive(OnDataChangeBatchForward);` beside the existing receive (`:480`); handler builds the items list once and Tells parent one `AttributeValueBatchPublished`. - `OnDataChangeForward` (`:1138`) becomes a single-item batch: `Context.Parent.Tell(new AttributeValueBatchPublished(_driverInstanceId, new[] { new AttributeValueItem(msg.FullReference, msg.Snapshot.Value, msg.Snapshot.StatusCode, ts) }));` — keep the `AttributeValuePublished` record declared (the mux comms-loss path and existing tests use the type), but the actor no longer sends it. - `DetachSubscription` (`:934-943`): also detach `_batchChangeHandler` from `IBatchSubscribable` and null it. **IMPORTANT — downstream compile/behavior:** `DriverHostActor` currently receives `AttributeValuePublished` (`DriverHostActor.cs:960`). Until Task 8 lands, add a minimal bridge in THIS task so the system stays green: in `DriverHostActor`, add `Receive(batch => { foreach (var i in batch.Items) ForwardToMux(new DriverInstanceActor.AttributeValuePublished(batch.DriverInstanceId, i.FullReference, i.Value, i.StatusCode, i.TimestampUtc)); });` — Task 8 replaces this bridge with the real batched fan-out. Existing `DriverHostActor` tests that send `AttributeValuePublished` directly stay green (the receive stays). **Step 4:** Run: `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests` — PASS (existing DriverInstanceActor forward tests may need updating IF they assert the parent receives `AttributeValuePublished` from a live driver event — update those to expect the single-item batch; do NOT weaken assertions on content). **Step 5:** `git add -A && git commit -m "feat(runtime): DriverInstanceActor forwards batched data changes"` --- ### Task 5: `ValueWrite` + sink `WriteValues` default method **Classification:** small **Estimated implement time:** ~4 min **Parallelizable with:** Task 3a, 3b, 4 **Files:** - Modify: `src/Core/ZB.MOM.WW.OtOpcUa.Commons/OpcUa/IOpcUaAddressSpaceSink.cs` - Modify: `src/Core/ZB.MOM.WW.OtOpcUa.Commons/OpcUa/DeferredAddressSpaceSink.cs` - Test: `tests/Core/ZB.MOM.WW.OtOpcUa.Commons.Tests/AddressSpaceSinkBatchTests.cs` (create) **Step 1: Failing tests** 1. `Default_WriteValues_loops_WriteValue` — a minimal fake implementing only the existing members; call `WriteValues` with 3 items; the fake's recorded `WriteValue` calls match in order/content. 2. `Deferred_sink_forwards_WriteValues_to_inner_once` — a fake inner that overrides `WriteValues` (records one batch call); `DeferredAddressSpaceSink.WriteValues` reaches the inner's override (NOT 3 `WriteValue` calls). **Step 3: Implement** In `IOpcUaAddressSpaceSink.cs`: ```csharp /// One value write within a batch. /// Same semantics per field as (#519: status verbatim). public readonly record struct ValueWrite( string NodeId, object? Value, uint StatusCode, DateTime SourceTimestampUtc, AddressSpaceRealm Realm); ``` Add to the interface (after `WriteValue`): ```csharp /// Write a batch of Variable values. Semantically identical to calling /// per item; an SDK-backed sink overrides this to take its internal /// lock ONCE per batch instead of once per value (the hot-path win). The default implementation /// loops so existing sinks and fakes behave unchanged. void WriteValues(IReadOnlyList values) { ArgumentNullException.ThrowIfNull(values); for (var i = 0; i < values.Count; i++) { var v = values[i]; WriteValue(v.NodeId, v.Value, v.StatusCode, v.SourceTimestampUtc, v.Realm); } } ``` In `DeferredAddressSpaceSink`, add the forwarding override: `public void WriteValues(IReadOnlyList values) => _inner.WriteValues(values);` **Step 4:** `dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Commons.Tests` — PASS. **Step 5:** `git add -A && git commit -m "feat(commons): batched WriteValues on the address-space sink"` --- ### Task 6: OtOpcUaNodeManager batched write under one lock **Classification:** high-risk (SDK locking) **Estimated implement time:** ~4 min **Parallelizable with:** Task 7 **Files:** - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs` (WriteValue region, `:508-533`) - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/SdkAddressSpaceSink.cs` (forward `WriteValues` to the manager) - Test: `tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerBatchWriteTests.cs` (create; copy the manager-construction idiom from an existing OpcUaServer.Tests file that instantiates `OtOpcUaNodeManager`) **Step 1: Failing tests** 1. `WriteValues_lands_every_value` — materialise 3 variables (or rely on create-on-first-write), `WriteValues` a 3-item batch, read each variable's Value/StatusCode/Timestamp back via the manager's public surface. 2. `WriteValues_creates_missing_variables` — same lazy-create semantics as `WriteValue`. **Step 3: Implement** - Extract the body of `WriteValue`'s `lock (Lock)` block into `private void WriteValueLocked(string nodeId, ushort ns, string key, object? value, uint statusCode, DateTime ts)` (verbatim move). - `WriteValue` computes `ns`/`key` then `lock (Lock) { WriteValueLocked(...); }` — behavior identical. - Add: ```csharp /// Batched WriteValue: one Lock acquisition for the whole batch. Same per-item semantics. public void WriteValues(IReadOnlyList values) { ArgumentNullException.ThrowIfNull(values); if (values.Count == 0) return; EnsureAddressSpaceCreated(); lock (Lock) { for (var i = 0; i < values.Count; i++) { var v = values[i]; WriteValueLocked(v.NodeId, NamespaceIndexForRealm(v.Realm), MapKey(v.Realm, v.NodeId), v.Value, v.StatusCode, v.SourceTimestampUtc); } } } ``` - In `SdkAddressSpaceSink`, override `WriteValues` forwarding to the manager's method (it currently forwards `WriteValue` 1:1 — mirror that). **Step 4:** `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests` — PASS. **Step 5:** `git add -A && git commit -m "feat(opcuaserver): node manager batched WriteValues under one lock"` --- ### Task 7: OpcUaPublishActor batch message **Classification:** standard **Estimated implement time:** ~4 min **Parallelizable with:** Task 6 **Files:** - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/OpcUa/OpcUaPublishActor.cs` - Test: `tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/OpcUa/OpcUaPublishActorBatchTests.cs` (create; copy the actor-construction idiom from `OpcUaPublishActorApplyFailureTests.cs`) **Step 1: Failing tests** 1. `Batch_message_reaches_sink_as_one_WriteValues_call` — fake sink records `WriteValues` batches; send `AttributeValueBatchUpdate` with 3 items; sink saw one batch of 3. 2. `Batch_increments_write_counter_per_item` — whatever surface exposes `_writes` (check for an existing status/metrics accessor in the actor's tests; if none, assert via the telemetry meter or skip the counter assertion and note it) — at minimum assert behavior parity: 3 items produce the same observable effect as 3 single `AttributeValueUpdate` messages. **Step 3: Implement** - Add next to `AttributeValueUpdate` (`:52`): ```csharp /// Batched form of : every fan-out write of one driver /// production burst in one message — the sink takes its lock once for the whole batch. public sealed record AttributeValueBatchUpdate(IReadOnlyList Items); ``` (`ValueWrite` comes from `ZB.MOM.WW.OtOpcUa.Commons.OpcUa` — already imported.) - Register `Receive(HandleAttributeBatchUpdate);` beside `:288`. - Handler mirrors `HandleAttributeUpdate` (`:319-331`): try/catch WARN on throw, `_sink.WriteValues(msg.Items)`, then `_writes += msg.Items.Count` via `Interlocked.Add(ref _writes, msg.Items.Count)` and `OtOpcUaTelemetry.OpcUaSinkWrite.Add(msg.Items.Count, new KeyValuePair("kind", "value"))` — per-item counting (invariant 9). **Step 4:** `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests --filter "FullyQualifiedName~OpcUaPublishActorBatch"` then the whole project — PASS. **Step 5:** `git add -A && git commit -m "feat(runtime): OpcUaPublishActor batched value updates"` --- ### Task 8: DriverHostActor batched fan-out (replaces the Task 4 bridge) **Classification:** high-risk (the hottest routing seam) **Estimated implement time:** ~5 min **Parallelizable with:** none (needs 4 + 7; Task 9 lands with it) **Files:** - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverHostActor.cs` - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/VirtualTags/DependencyMuxActor.cs` (Task 9 content — same commit is fine if splitting is awkward, but prefer two commits) - Test: `tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/DriverHostActorBatchFanOutTests.cs` (create; copy the host-actor fixture idiom from `DriverHostActorNativeAlarmTests.cs`) **Step 1: Failing tests** 1. `Batch_fans_out_to_all_node_realm_refs_in_one_publish_message` — arrange the host with an applied composition where tag A maps to 1 raw + 1 UNS NodeId and tag B maps to 1 raw; send `AttributeValueBatchPublished` with A+B; the publish-actor probe receives exactly ONE `AttributeValueBatchUpdate` containing 3 `ValueWrite`s with correct NodeId/Realm/value/status/timestamp. 2. `Unmapped_items_are_dropped_not_faulted` — batch with one unmapped ref: other items still land; no crash. 3. `Batch_reaches_mux_once` — mux probe receives ONE batch message carrying all items (not N). **Step 3: Implement** Replace the Task 4 bridge with a real handler: ```csharp private void ForwardBatchToMux(DriverInstanceActor.AttributeValueBatchPublished msg) { _dependencyMux?.Tell(msg); // one envelope per burst; the mux filters per item (Task 9) if (_opcUaPublishActor is null) return; List? writes = null; foreach (var item in msg.Items) { if (_nodeIdByDriverRef.TryGetValue((msg.DriverInstanceId, item.FullReference), out var nodeIds)) { writes ??= new List(msg.Items.Count * 2); foreach (var n in nodeIds) writes.Add(new ValueWrite(n.NodeId, item.Value, item.StatusCode, item.TimestampUtc, n.Realm)); } else if (_log.IsDebugEnabled) { _log.Debug("DriverHost {Node}: no bound NodeId for ({Driver},{Ref}) — value dropped", _localNode, msg.DriverInstanceId, item.FullReference); } } if (writes is not null) _opcUaPublishActor.Tell(new ZB.MOM.WW.OtOpcUa.Runtime.OpcUa.OpcUaPublishActor.AttributeValueBatchUpdate(writes)); } ``` Keep `ForwardToMux` and its `Receive` registration untouched (comms-loss fan + legacy tests). Register `Receive(ForwardBatchToMux);` in place of the Task 4 bridge. **Step 4:** `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests` — PASS. **Step 5:** `git add -A && git commit -m "feat(runtime): DriverHostActor batched value fan-out"` --- ### Task 9: DependencyMuxActor batch receive **Classification:** small **Estimated implement time:** ~3 min **Parallelizable with:** Task 8 (same PR-let; do after 8's records exist) **Files:** - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/VirtualTags/DependencyMuxActor.cs` - Test: `tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/` — add a batch case beside the existing mux tests (find them via `CalculationDependencyFlowTests.cs` / grep `DependencyMuxActor` in that project) **Step 1: Failing test** — register a subscriber for ref X; send a batch containing X and an uninteresting Y; subscriber receives exactly one `DependencyValueChanged` for X, nothing for Y. **Step 3: Implement** — add: ```csharp Receive(OnAttributeValueBatchPublished); ``` ```csharp private void OnAttributeValueBatchPublished(DriverInstanceActor.AttributeValueBatchPublished msg) { foreach (var item in msg.Items) { if (!_byRef.TryGetValue(item.FullReference, out var subscribers) || subscribers.Count == 0) continue; var dep = new VirtualTagActor.DependencyValueChanged(item.FullReference, item.Value, item.TimestampUtc, item.StatusCode); foreach (var sub in subscribers) sub.Tell(dep); } } ``` Keep the per-item receive (comms-loss path). **Step 4/5:** project tests green; `git commit -m "feat(runtime): dependency mux accepts batched publishes"` --- ### Task 10: Raise pinned-dispatcher throughput **Classification:** trivial **Estimated implement time:** ~2 min **Parallelizable with:** Task 1 **Files:** - Modify: `src/Core/ZB.MOM.WW.OtOpcUa.Cluster/Resources/akka.conf:112-116` - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/ServiceCollectionExtensions.cs:301-312` (the inline HOCON copy) Change `throughput = 1` → `throughput = 100` in BOTH places (they must match — the inline HOCON exists for consumers without akka.conf; see the comment at `ServiceCollectionExtensions.cs:301-303`). Update the comments: pinning alone guarantees the SDK's single-thread requirement; throughput only sets how many queued messages drain per scheduling run, so 100 removes a mailbox re-schedule round-trip per message with zero concurrency change. Run: `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests --filter "FullyQualifiedName~ServiceCollectionExtensionsTests"` Commit: `perf(runtime): raise opcua pinned dispatcher throughput 1 -> 100` --- ### Task 11: Re-key node-manager maps (kill per-write key allocations) **Classification:** high-risk (38 call sites, mechanical but wide) **Estimated implement time:** ~5 min (compiler-driven) **Parallelizable with:** none (after Task 6 — same file) **Files:** - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs` **Step 1: Baseline** — run `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests` green BEFORE starting (this task relies on the existing suite as its net; it should need no new tests, but the whole suite must pass unchanged after). **Step 2: Implement** - Add near `MapKey` (`:126-139`): ```csharp /// Zero-allocation realm-qualified map key. Replaces the formatted "ns=N;s=id" string key — /// the (realm, id) pair is the same identity without the per-write NodeId + string allocations. private readonly record struct RealmKey(AddressSpaceRealm Realm, string Id); ``` - Change `MapKey(AddressSpaceRealm realm, string id)` to return `new RealmKey(realm, id)`; change `MapKey(NodeId nodeId)` to `new RealmKey(RealmOf(nodeId.NamespaceIndex), (string)nodeId.Identifier)` — `RealmOf` already exists (~`:123`); confirm its exact name/signature and that every `MapKey(NodeId)` call site only ever passes nodes from the two owned namespaces (they do — SDK-facing seams on owned handles). If `nodeId.Identifier` can be non-string on any call path (numeric-id nodes), guard with `as string ?? nodeId.Identifier?.ToString() ?? string.Empty`. - Change every map/set keyed by the old string key to `RealmKey`: `_variables`, `_folders`, `_alarmConditions`, `_historizedTagnames`, `_eventNotifierSources`, `_nativeAlarmNodeIds`, and the notifier-wiring bookkeeping — let the compiler find all of them (38 `MapKey` call sites). - Prefix scans (`RemoveEquipmentSubtree` ~`:2539-2596`, `InScope` usages): match on `key.Realm == realm && key.Id.StartsWith(prefix, ...)` — semantics identical (the old string key embedded the realm as `ns=N;`). - `ResolveNodeState(string key)` (~`:2240-2261`): change its parameter to `RealmKey`. - Do NOT change public API surface or any behavior — this is a pure key-representation change. **Step 3:** `dotnet build ZB.MOM.WW.OtOpcUa.slnx` then `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests` and `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests` — ALL green, zero test edits expected (tests drive the public surface). If a test pokes the key format directly, stop and reassess rather than bending the test. **Step 4:** `git add -A && git commit -m "perf(opcuaserver): struct realm keys replace formatted NodeId string keys"` --- ### Task 12: Full verification + docs **Classification:** standard **Estimated implement time:** ~5 min **Parallelizable with:** none (last) **Files:** - Modify: `CLAUDE.md` (only if a statement it makes is now false — check the pinned-dispatcher/`throughput` mentions; keep edits surgical) - Modify: `docs/v2/Galaxy.Performance.md` — add a short "value-path batching" note (one paragraph: the batch seams, the invariants, the dispatcher change) Steps: 1. `dotnet build ZB.MOM.WW.OtOpcUa.slnx` — zero warnings introduced. 2. `dotnet test ZB.MOM.WW.OtOpcUa.slnx` — run what the environment can (DB-backed suites need `10.100.0.35`; if unreachable, run every non-DB project under tests/Core, tests/Drivers (unit), tests/Server/Runtime.Tests, tests/Server/OpcUaServer.Tests and record which suites were skipped and why in the commit message). 3. Docs + commit: `docs: record value-pipeline batching (plan 2026-08-15)`. --- ## Explicit non-goals (do NOT let scope creep into these) - **No conflation / bounded mailboxes** — batching per burst is this plan's backpressure story; keyed conflation is a follow-up. - **No fan-out-map push-down to DriverInstanceActor** — the host stays the router. - **No MQTT / MTConnect / Sparkplug / Galaxy / OpcUaClient batch implementations** — they flow through the single-item-batch path unchanged. - **No change to alarm paths** (`AttributeAlarmPublished`, `AlarmStateUpdate`) — value path only. - **No PollGroupEngine cadence change** (fixed-rate scheduling is a separate item). --- ## Follow-ups (recorded, not implemented) Raised during Tasks 1–12 review and deliberately left out of this plan's scope. None is a defect in the delivered work; each is a place where the next change in the area could go wrong quietly. 1. **`PollGroupEngine` collects batch items unconditionally — the batch event is only as trustworthy as the driver's `onChange`.** The engine has no knowledge of a driver's publish filter, so its items list is pre-filter by construction. That is safe *today* only because S7, AbCip and FOCAS forward every `onChange` unconditionally, and Modbus — the one driver with a deadband — ignores the engine's list and raises from its own filtered buffer. The trap: a future per-tag filter added to S7/AbCip/FOCAS's `onChange` would silently reintroduce the exact deadband-bypass class of bug fixed for Modbus in `cd4ce305`, with a green test suite (the two events would simply disagree, and only a consumer preferring the batch would notice). Consider changing the engine's `onChange` contract to `Func<..., bool>` so the engine collects an item only when the driver reports it published — making the filter structurally impossible to skip rather than a convention each driver must remember. 2. **Two-level `driverId → (rawPath → refs)` routing map in `DriverHostActor`.** The fan-out map is keyed by a composite of driver id + raw path, so resolving a batch re-hashes the driver id once per item even though a batch is by definition single-driver. Nesting the map would hash the driver id once per batch and the raw path once per item. Measure first — batches are small enough that this may be noise relative to the message-count win already banked. 3. **Null/empty guard consistency on the internal batch messages.** The guards on `AttributeValueBatchPublished` / `AttributeValueBatchUpdate` grew per task and are not uniform — some seams drop an empty batch early, others tolerate it downstream. These are internal messages with trusted senders so nothing is unsafe today; worth one pass to make the contract identical at every hop and stated once, rather than re-derived per reader. 4. **Possible merge of `DriverHostActorLiveValueTests` into the batch suite.** After Task 8 both files drive the same fan-out through the same message; the split is now historical rather than meaningful. Merging would remove a genuine ambiguity about where a new fan-out test belongs — but check first that the older suite's per-item comms-loss cases really are covered, since those enter via a different seam. 5. **Aborted-cycle tail-loss when a per-tag handler throws, for the three unbuffered drivers.** If a consumer's `onChange` throws mid-cycle in S7/AbCip/FOCAS, the cycle aborts and the items already collected for that cycle never reach a batch raise. **Pre-existing and narrower than the Modbus case** (Modbus's own buffer is per-handle and is dropped on unsubscribe/dispose, so it cannot leak), and it predates this plan — the per-tag path lost the same tail before batching existed. Recorded so it is not mistaken for a regression introduced here.