- docs/v2/Galaxy.Performance.md gains a "Value-path batching (2026-08-15)" section: the batch seams (IBatchSubscribable / PollGroupEngine onBatch -> AttributeValueBatchPublished -> AttributeValueBatchUpdate -> sink WriteValues with 512-chunked single-lock batches in the node manager), the invariants (#519 status codes verbatim, per-ITEM metering, Modbus deadband honored via the driver-filtered buffer, the ValueWriteBatchException partial-failure contract), the RealmKey re-key, and the pinned dispatcher's throughput 1 -> 100. - The plan doc gains a "Follow-ups (recorded, not implemented)" section: the PollGroupEngine unconditional-collection trap (a future per-tag filter in S7/AbCip/FOCAS would silently reintroduce the deadband-bypass bug), a two-level driverId -> (rawPath -> refs) routing map, null/empty guard consistency on the internal batch messages, a possible DriverHostActorLiveValueTests merge, and the pre-existing aborted-cycle tail-loss on the three unbuffered drivers. - tasks.json: all 13 tasks completed. CLAUDE.md needed no edit: its only statement in this area ("One driver publish for a RawPath fans (in DriverHostActor) to the raw NodeId AND every referencing UNS NodeId") is still true — batching changed the message granularity, not the fan-out semantics. Verification: dotnet build ZB.MOM.WW.OtOpcUa.slnx clean (0 errors, no new warning classes, zero warnings in any production file). 9 unit projects green, 3470 passed / 5 skipped / 0 failed: Core.Abstractions 305, Core 351+1s, Commons 348, Runtime 575+3s, OpcUaServer 424+1s, Modbus 399, S7 302, AbCip 410, FOCAS 356. No DB-backed suite was run — tests/Server/ZB.MOM.WW.OtOpcUa.Server.Tests does not exist, and nothing this branch touches is covered by the SQL-backed projects.
40 KiB
Value-Pipeline Batching Implementation Plan
For Claude: REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task.
Goal: Batch the hot value path end to end — one message per poll cycle instead of one per tag at every hop (driver → DriverInstanceActor → DriverHostActor → OpcUaPublishActor → node manager), one SDK-lock acquisition per batch instead of per value, and zero per-write key allocations in the node manager.
Architecture: A new optional IBatchSubscribable capability lets a driver raise one OnDataChangeBatch per production burst (poll cycle); PollGroupEngine gains a batch callback so the four engine-based poll drivers get it for free. DriverInstanceActor forwards one AttributeValueBatchPublished per burst (wrapping per-tag events from non-batch drivers as single-item batches); DriverHostActor resolves the whole batch's NodeRealmRefs and sends ONE AttributeValueBatchUpdate to OpcUaPublishActor, plus ONE batch Tell to the dependency mux. The sink gains WriteValues(IReadOnlyList<ValueWrite>) (default-interface-method fallback loops WriteValue, so fakes/Null sink are untouched); OtOpcUaNodeManager overrides it taking Lock once per batch, and its node maps are re-keyed by a (realm, id) struct so the per-write new NodeId(...).ToString() allocations disappear. The pinned dispatcher's throughput rises from 1 to 100 (same single thread, fewer mailbox re-schedules).
Tech Stack: C# / .NET 10, Akka.NET (classic actors), OPC Foundation UA .NET Standard, xUnit + Shouldly, Akka.TestKit.
Branch: perf/value-pipeline-batching (off master).
Context for implementers (read before any task)
Verified facts about the current code — do not re-derive, and do not violate the invariants:
- The per-tag path today:
PollGroupEngine.PollOnceAsync(src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/PollGroupEngine.cs:202-226) diffs per tag and invokes_onChangeper changed tag. Drivers forward that toISubscribable.OnDataChange(per-tag event,src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/ISubscribable.cs:36).DriverInstanceActor.HandleSubscribeAsyncattaches_dataChangeHandleratDriverInstanceActor.cs:880(oneDataChangeForwardTell per tag),OnDataChangeForward(:1138-1143) Tells parent oneAttributeValuePublishedper tag.DriverHostActor.ForwardToMux(DriverHostActor.cs:976-1006) Tells the mux per value and TellsOpcUaPublishActoroneAttributeValueUpdateper NodeRealmRef (UNS fan-out at:997-999).OpcUaPublishActor.HandleAttributeUpdate(OpcUaPublishActor.cs:319-331) calls_sink.WriteValueper message.OtOpcUaNodeManager.WriteValue(OtOpcUaNodeManager.cs:508-533) takes the manager-widelock (Lock)per value and allocates viaMapKey(realm, id)=new NodeId(id, ns).ToString()(:133). - Invariants to preserve:
- #519 status codes ride verbatim — the full
uintstatus code, sub-code included, must reach the SDK unchanged at every hop. - Timestamps are carried from the driver — never stamp
DateTime.UtcNowon the value path. - PollGroupEngine's no-overlap guarantee (delay-after-completion, one loop task per subscription) must not change in this plan.
- Per-tag
OnDataChangekeeps working — Galaxy's cold-read path and driver CLIs subscribe it. A batch-capable driver raises BOTH events (batch once per cycle + per-tag per item); a consumer subscribes to exactly ONE of the two.DriverInstanceActorprefers batch when the driver implementsIBatchSubscribable. DriverInstanceActor.DetachSubscription(:934-943) must detach the batch handler too — the detach is load-bearing (theIDriveroutlives the actor across respawns; see CLAUDE.md §DeployWatcher).- The mux's per-item receive stays —
DriverHostActor.FanCommsLossToDependencyMux(:1135-1142) fabricates per-itemAttributeValuePublishedmessages;DependencyMuxActorkeeps that receive and gains a batch receive alongside. OpcUaPublishActorkeeps its per-itemAttributeValueUpdatereceive —VirtualTagHostActorand the alarm paths still send it. The batch message is additive.- The pinned dispatcher stays pinned — the OPC UA SDK requires a single thread. Only
throughputchanges. _writesincrements (OpcUaPublishActor.cs:324) and theOtOpcUaTelemetry.OpcUaSinkWritemeter must count per-VALUE (per item), not per batch, so dashboards don't drop by 100×.
- #519 status codes ride verbatim — the full
- Message-record homes:
AttributeValuePublishedis declared inDriverInstanceActor(:109);AttributeValueUpdateinOpcUaPublishActor(:52);NodeRealmRefis a private struct ofDriverHostActor(:183). - Sink implementors that must keep compiling:
NullOpcUaAddressSpaceSink(same file as the interface),DeferredAddressSpaceSink(src/Core/ZB.MOM.WW.OtOpcUa.Commons/OpcUa/DeferredAddressSpaceSink.cs),SdkAddressSpaceSink(src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/SdkAddressSpaceSink.cs), plus test fakes. Using a default interface method forWriteValuesmeans only the ones that want batching override it. - Commands: build
dotnet build ZB.MOM.WW.OtOpcUa.slnx; single projectdotnet test tests/<module>/<project>; single testdotnet test --filter "FullyQualifiedName~<Class>.<Method>". DB-backed suites need the central SQL Server — on this Mac, prefer running the specific unit projects named per task, not the whole solution, until the final task. - Commit style: follow recent history (
feat(...),fix(...),chore(...)with a scope). One commit per task.
Task 1: Batch contract types in Core.Abstractions
Classification: small Estimated implement time: ~3 min Parallelizable with: Task 10
Files:
- Modify:
src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/ISubscribable.cs - Test:
tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests/DataChangeBatchContractTests.cs(create)
Step 1: Write the failing test
using Shouldly;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using Xunit;
namespace ZB.MOM.WW.OtOpcUa.Core.Tests;
public class DataChangeBatchContractTests
{
private sealed record FakeHandle(string DiagnosticId) : ISubscriptionHandle;
[Fact]
public void Batch_args_carry_handle_and_items()
{
var handle = new FakeHandle("h1");
var snap = new DataValueSnapshot(42, 0u, DateTime.UtcNow, DateTime.UtcNow);
var args = new DataChangeBatchEventArgs(handle, new[] { new DataChangeItem("plc/tag1", snap) });
args.SubscriptionHandle.ShouldBe(handle);
args.Items.Count.ShouldBe(1);
args.Items[0].FullReference.ShouldBe("plc/tag1");
args.Items[0].Snapshot.ShouldBe(snap);
}
}
Note: check DataValueSnapshot's actual constructor shape in src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/ first and adjust the test's construction to it (positional record with Value/StatusCode/timestamps). Do not change DataValueSnapshot.
Step 2: Run test to verify it fails
Run: dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests --filter "FullyQualifiedName~DataChangeBatchContractTests"
Expected: FAIL — DataChangeBatchEventArgs / DataChangeItem not defined.
Step 3: Implement
Append to src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/ISubscribable.cs:
/// <summary>One changed attribute within a <see cref="DataChangeBatchEventArgs"/>.</summary>
/// <param name="FullReference">Driver-side full reference of the changed attribute.</param>
/// <param name="Snapshot">New value + quality + timestamps.</param>
public sealed record DataChangeItem(string FullReference, DataValueSnapshot Snapshot);
/// <summary>Event payload for <see cref="IBatchSubscribable.OnDataChangeBatch"/> — every attribute
/// that changed within ONE production burst (a poll cycle, an inbound message, a stream chunk).</summary>
/// <param name="SubscriptionHandle">The handle returned by the original <see cref="ISubscribable.SubscribeAsync"/> call.</param>
/// <param name="Items">The changed attributes, one entry per changed tag. Never empty.</param>
public sealed record DataChangeBatchEventArgs(
ISubscriptionHandle SubscriptionHandle,
IReadOnlyList<DataChangeItem> Items);
/// <summary>
/// Optional driver capability: data changes delivered one event per production burst instead of
/// one event per tag. A driver implementing this raises <see cref="OnDataChangeBatch"/> once per
/// burst AND continues raising the per-tag <see cref="ISubscribable.OnDataChange"/> for each item
/// (legacy consumers: Galaxy cold reads, driver CLIs). A consumer subscribes to EXACTLY ONE of the
/// two events — subscribing both double-delivers. DriverInstanceActor prefers the batch event.
/// </summary>
public interface IBatchSubscribable : ISubscribable
{
/// <summary>Batched data-change notification — one raise per production burst.</summary>
event EventHandler<DataChangeBatchEventArgs>? OnDataChangeBatch;
}
Step 4: Run test to verify it passes
Run: same filter. Expected: PASS.
Step 5: Commit
git add src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/ISubscribable.cs tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests/DataChangeBatchContractTests.cs
git commit -m "feat(core): add IBatchSubscribable batch data-change contract"
Task 2: PollGroupEngine batch callback
Classification: standard Estimated implement time: ~5 min Parallelizable with: none (Task 1 first)
Files:
- Modify:
src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/PollGroupEngine.cs - Test:
tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests/PollGroupEngineBatchTests.cs(create)
Step 1: Write the failing tests (find the existing PollGroupEngine test file first — if one exists in tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests, put these in a new class beside it and copy its reader-fake idiom):
Test cases (write all four):
Batch_fires_once_per_cycle_with_only_changed_items— reader returns 3 refs; first poll (initial push) batch has all 3; then reader changes only 1 value; next batch has exactly that 1 item; per-tagonChangealso fired for the same items (both callbacks see identical content).No_batch_when_nothing_changed— second poll with identical snapshots ⇒ no batch invocation (and no per-tag invocation).Batch_is_not_invoked_when_callback_absent— construct withoutonBatch; engine behaves exactly as today (per-tag only). Assert per-tag count.Batch_items_are_in_reader_order.
Use a short interval (e.g. 30 ms) + TaskCompletionSource/polling-with-timeout to await callbacks, mirroring the existing engine tests' style. Never Thread.Sleep long constants.
Step 2: Run to verify failure
Run: dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests --filter "FullyQualifiedName~PollGroupEngineBatchTests"
Expected: FAIL — constructor has no onBatch parameter.
Step 3: Implement
In PollGroupEngine:
- Add field
private readonly Action<ISubscriptionHandle, IReadOnlyList<DataChangeItem>>? _onBatch; - Add optional ctor param
Action<ISubscriptionHandle, IReadOnlyList<DataChangeItem>>? onBatch = null(AFTER the existing optional params, so every existing call site compiles unchanged) and assign it. - In
PollOnceAsync(currently:202-226): collect changed items lazily and raise the batch after the loop:
List<DataChangeItem>? changed = null;
for (var i = 0; i < state.TagReferences.Count; i++)
{
var tagRef = state.TagReferences[i];
var current = snapshots[i];
var lastSeen = state.LastValues.TryGetValue(tagRef, out var prev) ? prev : default;
if (forceRaise || ValuesAreDifferent(lastSeen?.Value, current.Value) || lastSeen?.StatusCode != current.StatusCode)
{
state.LastValues[tagRef] = current;
_onChange(state.Handle, tagRef, current);
if (_onBatch is not null) (changed ??= new List<DataChangeItem>()).Add(new DataChangeItem(tagRef, current));
}
}
if (changed is { Count: > 0 }) _onBatch!(state.Handle, changed);
Keep the surrounding contract-violation check and everything else byte-identical. Update the class remarks to document the batch callback (one raise per cycle, only when non-empty, after all per-tag callbacks of that cycle).
Step 4: Run tests
Run: dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests (whole project — the existing engine tests must stay green).
Expected: PASS.
Step 5: Commit
git add -A && git commit -m "feat(core): PollGroupEngine optional per-cycle batch callback"
Task 3a: Modbus + S7 raise the batch event
Classification: standard Estimated implement time: ~5 min Parallelizable with: Task 3b, Task 5
Files:
- Modify:
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/ModbusDriver.cs - Modify:
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7/S7Driver.cs - Test:
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.Tests/ModbusBatchSubscriptionTests.cs(create; mirror the fixture idiom ofModbusSubscriptionTests.cs)
Step 1: Write the failing test — subscribe via the driver against the existing in-test transport fake (copy ModbusSubscriptionTests' setup); assert:
ModbusDriverisIBatchSubscribable.- After one poll cycle with N changed registers,
OnDataChangeBatchfired once with N items ANDOnDataChangefired N times (both events, identical content — contract from Task 1).
Step 2: Run: dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.Tests --filter "FullyQualifiedName~ModbusBatchSubscriptionTests" — FAIL (interface not implemented).
Step 3: Implement — in each driver:
- Change the class declaration to implement
IBatchSubscribable(it already implementsISubscribable). - Add
public event EventHandler<DataChangeBatchEventArgs>? OnDataChangeBatch; - Find the
new PollGroupEngine(...)construction; passonBatch: (handle, items) => OnDataChangeBatch?.Invoke(this, new DataChangeBatchEventArgs(handle, items)).
Per-tag raising is already wired via the engine's onChange — leave it.
Step 4: Run the Modbus + S7 test projects:
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.Tests && dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7.Tests — PASS (all existing tests too).
Step 5: git add -A && git commit -m "feat(drivers): Modbus + S7 implement IBatchSubscribable via PollGroupEngine batch callback"
Deviation (landed): Modbus does not forward the engine's items argument — it ignores it and
raises the batch from its own per-cycle buffer of the items that passed ShouldPublish. The engine
collects its batch list unconditionally and has no knowledge of the driver's deadband filter, so a
straight pass-through would carry changes the per-tag event suppressed, and a consumer that preferred
the batch event would see deadband silently stop working (review finding). onBatch is therefore used
only as the end-of-cycle signal, draining the buffer; a cycle whose every change was suppressed raises
nothing. Because that buffer is the only piece of cross-cycle state any of the four poll drivers holds,
the driver's onChange bridge also wraps OnDataChange?.Invoke(...) in a swallowing try/catch
(mirroring PollGroupEngine.ReportError): the engine invokes the bridge unguarded, so a throwing
external subscriber would unwind the cycle before onBatch ran and the undrained items would merge
into a later cycle's batch. The buffer append stays outside the catch — the item was published as far
as the batch contract is concerned; only the per-tag subscriber faulted. S7/AbCip/FOCAS forward the
engine's list as this task describes and hold no such state.
Task 3b: AbCip + FOCAS raise the batch event
Classification: standard Estimated implement time: ~4 min Parallelizable with: Task 3a, Task 5
Files:
- Modify:
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.AbCip/AbCipDriver.cs - Modify:
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.FOCAS/FocasDriver.cs
Same mechanical change as Task 3a (interface + event + onBatch at the new PollGroupEngine(...) site). If either driver constructs the engine differently (check first), adapt but keep the identical contract. No new per-driver test required (the engine behavior is pinned by Task 2; the wiring pattern is pinned by Task 3a's test) — but the projects' existing suites must stay green.
Run: dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.AbCip.Tests && dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Tests
Commit: feat(drivers): AbCip + FOCAS implement IBatchSubscribable
Task 4: DriverInstanceActor batch forwarding
Classification: high-risk (actor model, subscription lifecycle) Estimated implement time: ~5 min Parallelizable with: Task 5
Files:
- Modify:
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverInstanceActor.cs - Test:
tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/DriverInstanceActorBatchForwardTests.cs(create; copy the TestKit + stub-driver idiom from the existingDriverInstanceActor*Testsin that directory)
Step 1: Write the failing tests
Batch_driver_forwards_one_batch_message— stub driver implementingIBatchSubscribable; raiseOnDataChangeBatchwith 3 items after subscribe; parent probe receives exactly ONEAttributeValueBatchPublishedwith 3 items (driver id + per-item FullReference/Value/StatusCode/timestamp mapped asOnDataChangeForwardmaps them today:ts = Snapshot.SourceTimestampUtc ?? Snapshot.ServerTimestampUtc).Batch_driver_per_tag_event_not_double_forwarded— the stub raises BOTH events (per contract); parent receives only the batch message, no legacyAttributeValuePublished.Per_tag_driver_forwards_single_item_batches— stub implementing onlyISubscribable; oneOnDataChange⇒ parent receives oneAttributeValueBatchPublishedwith 1 item.Detach_removes_batch_handler— after Unsubscribe (or Connected→Reconnecting, whichever the existing detach tests exercise), raising the batch event produces no message.
Step 2: Run: dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests --filter "FullyQualifiedName~DriverInstanceActorBatchForwardTests" — FAIL.
Step 3: Implement
- Add records next to
AttributeValuePublished(:109):
/// <summary>One value within an <see cref="AttributeValueBatchPublished"/>.</summary>
public sealed record AttributeValueItem(string FullReference, object? Value, uint StatusCode, DateTime TimestampUtc);
/// <summary>Batched form of <see cref="AttributeValuePublished"/> — every changed attribute of ONE
/// driver production burst in one message. The parent fans the whole batch to the mux and the OPC UA
/// publish path in one hop each. Status codes ride verbatim (#519).</summary>
public sealed record AttributeValueBatchPublished(string DriverInstanceId, IReadOnlyList<AttributeValueItem> Items);
- Private message
private sealed record DataChangeBatchForward(IReadOnlyList<DataChangeItem> Items);+ fieldprivate EventHandler<DataChangeBatchEventArgs>? _batchChangeHandler; - In
HandleSubscribeAsync(:878-882): whensubscribable is IBatchSubscribable batchable, attach ONLY the batch handler (_batchChangeHandler = (_, args) => self.Tell(new DataChangeBatchForward(args.Items)); batchable.OnDataChangeBatch += _batchChangeHandler;) and do NOT attach_dataChangeHandler; else attach_dataChangeHandlerexactly as today. Receive<DataChangeBatchForward>(OnDataChangeBatchForward);beside the existing receive (:480); handler builds the items list once and Tells parent oneAttributeValueBatchPublished.OnDataChangeForward(:1138) becomes a single-item batch:Context.Parent.Tell(new AttributeValueBatchPublished(_driverInstanceId, new[] { new AttributeValueItem(msg.FullReference, msg.Snapshot.Value, msg.Snapshot.StatusCode, ts) }));— keep theAttributeValuePublishedrecord declared (the mux comms-loss path and existing tests use the type), but the actor no longer sends it.DetachSubscription(:934-943): also detach_batchChangeHandlerfromIBatchSubscribableand null it.
IMPORTANT — downstream compile/behavior: DriverHostActor currently receives AttributeValuePublished (DriverHostActor.cs:960). Until Task 8 lands, add a minimal bridge in THIS task so the system stays green: in DriverHostActor, add Receive<DriverInstanceActor.AttributeValueBatchPublished>(batch => { foreach (var i in batch.Items) ForwardToMux(new DriverInstanceActor.AttributeValuePublished(batch.DriverInstanceId, i.FullReference, i.Value, i.StatusCode, i.TimestampUtc)); }); — Task 8 replaces this bridge with the real batched fan-out. Existing DriverHostActor tests that send AttributeValuePublished directly stay green (the receive stays).
Step 4: Run: dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests — PASS (existing DriverInstanceActor forward tests may need updating IF they assert the parent receives AttributeValuePublished from a live driver event — update those to expect the single-item batch; do NOT weaken assertions on content).
Step 5: git add -A && git commit -m "feat(runtime): DriverInstanceActor forwards batched data changes"
Task 5: ValueWrite + sink WriteValues default method
Classification: small Estimated implement time: ~4 min Parallelizable with: Task 3a, 3b, 4
Files:
- Modify:
src/Core/ZB.MOM.WW.OtOpcUa.Commons/OpcUa/IOpcUaAddressSpaceSink.cs - Modify:
src/Core/ZB.MOM.WW.OtOpcUa.Commons/OpcUa/DeferredAddressSpaceSink.cs - Test:
tests/Core/ZB.MOM.WW.OtOpcUa.Commons.Tests/AddressSpaceSinkBatchTests.cs(create)
Step 1: Failing tests
Default_WriteValues_loops_WriteValue— a minimal fake implementing only the existing members; callWriteValueswith 3 items; the fake's recordedWriteValuecalls match in order/content.Deferred_sink_forwards_WriteValues_to_inner_once— a fake inner that overridesWriteValues(records one batch call);DeferredAddressSpaceSink.WriteValuesreaches the inner's override (NOT 3WriteValuecalls).
Step 3: Implement
In IOpcUaAddressSpaceSink.cs:
/// <summary>One value write within a <see cref="IOpcUaAddressSpaceSink.WriteValues"/> batch.
/// Same semantics per field as <see cref="IOpcUaAddressSpaceSink.WriteValue"/> (#519: status verbatim).</summary>
public readonly record struct ValueWrite(
string NodeId, object? Value, uint StatusCode, DateTime SourceTimestampUtc, AddressSpaceRealm Realm);
Add to the interface (after WriteValue):
/// <summary>Write a batch of Variable values. Semantically identical to calling
/// <see cref="WriteValue"/> per item; an SDK-backed sink overrides this to take its internal
/// lock ONCE per batch instead of once per value (the hot-path win). The default implementation
/// loops <see cref="WriteValue"/> so existing sinks and fakes behave unchanged.</summary>
void WriteValues(IReadOnlyList<ValueWrite> values)
{
ArgumentNullException.ThrowIfNull(values);
for (var i = 0; i < values.Count; i++)
{
var v = values[i];
WriteValue(v.NodeId, v.Value, v.StatusCode, v.SourceTimestampUtc, v.Realm);
}
}
In DeferredAddressSpaceSink, add the forwarding override: public void WriteValues(IReadOnlyList<ValueWrite> values) => _inner.WriteValues(values);
Step 4: dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Commons.Tests — PASS.
Step 5: git add -A && git commit -m "feat(commons): batched WriteValues on the address-space sink"
Task 6: OtOpcUaNodeManager batched write under one lock
Classification: high-risk (SDK locking) Estimated implement time: ~4 min Parallelizable with: Task 7
Files:
- Modify:
src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs(WriteValue region,:508-533) - Modify:
src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/SdkAddressSpaceSink.cs(forwardWriteValuesto the manager) - Test:
tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerBatchWriteTests.cs(create; copy the manager-construction idiom from an existing OpcUaServer.Tests file that instantiatesOtOpcUaNodeManager)
Step 1: Failing tests
WriteValues_lands_every_value— materialise 3 variables (or rely on create-on-first-write),WriteValuesa 3-item batch, read each variable's Value/StatusCode/Timestamp back via the manager's public surface.WriteValues_creates_missing_variables— same lazy-create semantics asWriteValue.
Step 3: Implement
- Extract the body of
WriteValue'slock (Lock)block intoprivate void WriteValueLocked(string nodeId, ushort ns, string key, object? value, uint statusCode, DateTime ts)(verbatim move). WriteValuecomputesns/keythenlock (Lock) { WriteValueLocked(...); }— behavior identical.- Add:
/// <summary>Batched WriteValue: one Lock acquisition for the whole batch. Same per-item semantics.</summary>
public void WriteValues(IReadOnlyList<ValueWrite> values)
{
ArgumentNullException.ThrowIfNull(values);
if (values.Count == 0) return;
EnsureAddressSpaceCreated();
lock (Lock)
{
for (var i = 0; i < values.Count; i++)
{
var v = values[i];
WriteValueLocked(v.NodeId, NamespaceIndexForRealm(v.Realm), MapKey(v.Realm, v.NodeId), v.Value, v.StatusCode, v.SourceTimestampUtc);
}
}
}
- In
SdkAddressSpaceSink, overrideWriteValuesforwarding to the manager's method (it currently forwardsWriteValue1:1 — mirror that).
Step 4: dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests — PASS.
Step 5: git add -A && git commit -m "feat(opcuaserver): node manager batched WriteValues under one lock"
Task 7: OpcUaPublishActor batch message
Classification: standard Estimated implement time: ~4 min Parallelizable with: Task 6
Files:
- Modify:
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/OpcUa/OpcUaPublishActor.cs - Test:
tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/OpcUa/OpcUaPublishActorBatchTests.cs(create; copy the actor-construction idiom fromOpcUaPublishActorApplyFailureTests.cs)
Step 1: Failing tests
Batch_message_reaches_sink_as_one_WriteValues_call— fake sink recordsWriteValuesbatches; sendAttributeValueBatchUpdatewith 3 items; sink saw one batch of 3.Batch_increments_write_counter_per_item— whatever surface exposes_writes(check for an existing status/metrics accessor in the actor's tests; if none, assert via the telemetry meter or skip the counter assertion and note it) — at minimum assert behavior parity: 3 items produce the same observable effect as 3 singleAttributeValueUpdatemessages.
Step 3: Implement
- Add next to
AttributeValueUpdate(:52):
/// <summary>Batched form of <see cref="AttributeValueUpdate"/>: every fan-out write of one driver
/// production burst in one message — the sink takes its lock once for the whole batch.</summary>
public sealed record AttributeValueBatchUpdate(IReadOnlyList<ValueWrite> Items);
(ValueWrite comes from ZB.MOM.WW.OtOpcUa.Commons.OpcUa — already imported.)
- Register
Receive<AttributeValueBatchUpdate>(HandleAttributeBatchUpdate);beside:288. - Handler mirrors
HandleAttributeUpdate(:319-331): try/catch WARN on throw,_sink.WriteValues(msg.Items), then_writes += msg.Items.CountviaInterlocked.Add(ref _writes, msg.Items.Count)andOtOpcUaTelemetry.OpcUaSinkWrite.Add(msg.Items.Count, new KeyValuePair<string, object?>("kind", "value"))— per-item counting (invariant 9).
Step 4: dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests --filter "FullyQualifiedName~OpcUaPublishActorBatch" then the whole project — PASS.
Step 5: git add -A && git commit -m "feat(runtime): OpcUaPublishActor batched value updates"
Task 8: DriverHostActor batched fan-out (replaces the Task 4 bridge)
Classification: high-risk (the hottest routing seam) Estimated implement time: ~5 min Parallelizable with: none (needs 4 + 7; Task 9 lands with it)
Files:
- Modify:
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverHostActor.cs - Modify:
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/VirtualTags/DependencyMuxActor.cs(Task 9 content — same commit is fine if splitting is awkward, but prefer two commits) - Test:
tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/DriverHostActorBatchFanOutTests.cs(create; copy the host-actor fixture idiom fromDriverHostActorNativeAlarmTests.cs)
Step 1: Failing tests
Batch_fans_out_to_all_node_realm_refs_in_one_publish_message— arrange the host with an applied composition where tag A maps to 1 raw + 1 UNS NodeId and tag B maps to 1 raw; sendAttributeValueBatchPublishedwith A+B; the publish-actor probe receives exactly ONEAttributeValueBatchUpdatecontaining 3ValueWrites with correct NodeId/Realm/value/status/timestamp.Unmapped_items_are_dropped_not_faulted— batch with one unmapped ref: other items still land; no crash.Batch_reaches_mux_once— mux probe receives ONE batch message carrying all items (not N).
Step 3: Implement
Replace the Task 4 bridge with a real handler:
private void ForwardBatchToMux(DriverInstanceActor.AttributeValueBatchPublished msg)
{
_dependencyMux?.Tell(msg); // one envelope per burst; the mux filters per item (Task 9)
if (_opcUaPublishActor is null) return;
List<ValueWrite>? writes = null;
foreach (var item in msg.Items)
{
if (_nodeIdByDriverRef.TryGetValue((msg.DriverInstanceId, item.FullReference), out var nodeIds))
{
writes ??= new List<ValueWrite>(msg.Items.Count * 2);
foreach (var n in nodeIds)
writes.Add(new ValueWrite(n.NodeId, item.Value, item.StatusCode, item.TimestampUtc, n.Realm));
}
else if (_log.IsDebugEnabled)
{
_log.Debug("DriverHost {Node}: no bound NodeId for ({Driver},{Ref}) — value dropped",
_localNode, msg.DriverInstanceId, item.FullReference);
}
}
if (writes is not null)
_opcUaPublishActor.Tell(new ZB.MOM.WW.OtOpcUa.Runtime.OpcUa.OpcUaPublishActor.AttributeValueBatchUpdate(writes));
}
Keep ForwardToMux and its Receive<AttributeValuePublished> registration untouched (comms-loss fan + legacy tests). Register Receive<DriverInstanceActor.AttributeValueBatchPublished>(ForwardBatchToMux); in place of the Task 4 bridge.
Step 4: dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests — PASS.
Step 5: git add -A && git commit -m "feat(runtime): DriverHostActor batched value fan-out"
Task 9: DependencyMuxActor batch receive
Classification: small Estimated implement time: ~3 min Parallelizable with: Task 8 (same PR-let; do after 8's records exist)
Files:
- Modify:
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/VirtualTags/DependencyMuxActor.cs - Test:
tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/— add a batch case beside the existing mux tests (find them viaCalculationDependencyFlowTests.cs/ grepDependencyMuxActorin that project)
Step 1: Failing test — register a subscriber for ref X; send a batch containing X and an uninteresting Y; subscriber receives exactly one DependencyValueChanged for X, nothing for Y.
Step 3: Implement — add:
Receive<DriverInstanceActor.AttributeValueBatchPublished>(OnAttributeValueBatchPublished);
private void OnAttributeValueBatchPublished(DriverInstanceActor.AttributeValueBatchPublished msg)
{
foreach (var item in msg.Items)
{
if (!_byRef.TryGetValue(item.FullReference, out var subscribers) || subscribers.Count == 0) continue;
var dep = new VirtualTagActor.DependencyValueChanged(item.FullReference, item.Value, item.TimestampUtc, item.StatusCode);
foreach (var sub in subscribers) sub.Tell(dep);
}
}
Keep the per-item receive (comms-loss path).
Step 4/5: project tests green; git commit -m "feat(runtime): dependency mux accepts batched publishes"
Task 10: Raise pinned-dispatcher throughput
Classification: trivial Estimated implement time: ~2 min Parallelizable with: Task 1
Files:
- Modify:
src/Core/ZB.MOM.WW.OtOpcUa.Cluster/Resources/akka.conf:112-116 - Modify:
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/ServiceCollectionExtensions.cs:301-312(the inline HOCON copy)
Change throughput = 1 → throughput = 100 in BOTH places (they must match — the inline HOCON exists for consumers without akka.conf; see the comment at ServiceCollectionExtensions.cs:301-303). Update the comments: pinning alone guarantees the SDK's single-thread requirement; throughput only sets how many queued messages drain per scheduling run, so 100 removes a mailbox re-schedule round-trip per message with zero concurrency change.
Run: dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests --filter "FullyQualifiedName~ServiceCollectionExtensionsTests"
Commit: perf(runtime): raise opcua pinned dispatcher throughput 1 -> 100
Task 11: Re-key node-manager maps (kill per-write key allocations)
Classification: high-risk (38 call sites, mechanical but wide) Estimated implement time: ~5 min (compiler-driven) Parallelizable with: none (after Task 6 — same file)
Files:
- Modify:
src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs
Step 1: Baseline — run dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests green BEFORE starting (this task relies on the existing suite as its net; it should need no new tests, but the whole suite must pass unchanged after).
Step 2: Implement
- Add near
MapKey(:126-139):
/// <summary>Zero-allocation realm-qualified map key. Replaces the formatted "ns=N;s=id" string key —
/// the (realm, id) pair is the same identity without the per-write NodeId + string allocations.</summary>
private readonly record struct RealmKey(AddressSpaceRealm Realm, string Id);
- Change
MapKey(AddressSpaceRealm realm, string id)to returnnew RealmKey(realm, id); changeMapKey(NodeId nodeId)tonew RealmKey(RealmOf(nodeId.NamespaceIndex), (string)nodeId.Identifier)—RealmOfalready exists (~:123); confirm its exact name/signature and that everyMapKey(NodeId)call site only ever passes nodes from the two owned namespaces (they do — SDK-facing seams on owned handles). IfnodeId.Identifiercan be non-string on any call path (numeric-id nodes), guard withas string ?? nodeId.Identifier?.ToString() ?? string.Empty. - Change every map/set keyed by the old string key to
RealmKey:_variables,_folders,_alarmConditions,_historizedTagnames,_eventNotifierSources,_nativeAlarmNodeIds, and the notifier-wiring bookkeeping — let the compiler find all of them (38MapKeycall sites). - Prefix scans (
RemoveEquipmentSubtree~:2539-2596,InScopeusages): match onkey.Realm == realm && key.Id.StartsWith(prefix, ...)— semantics identical (the old string key embedded the realm asns=N;). ResolveNodeState(string key)(~:2240-2261): change its parameter toRealmKey.- Do NOT change public API surface or any behavior — this is a pure key-representation change.
Step 3: dotnet build ZB.MOM.WW.OtOpcUa.slnx then dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests and dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests — ALL green, zero test edits expected (tests drive the public surface). If a test pokes the key format directly, stop and reassess rather than bending the test.
Step 4: git add -A && git commit -m "perf(opcuaserver): struct realm keys replace formatted NodeId string keys"
Task 12: Full verification + docs
Classification: standard Estimated implement time: ~5 min Parallelizable with: none (last)
Files:
- Modify:
CLAUDE.md(only if a statement it makes is now false — check the pinned-dispatcher/throughputmentions; keep edits surgical) - Modify:
docs/v2/Galaxy.Performance.md— add a short "value-path batching" note (one paragraph: the batch seams, the invariants, the dispatcher change)
Steps:
dotnet build ZB.MOM.WW.OtOpcUa.slnx— zero warnings introduced.dotnet test ZB.MOM.WW.OtOpcUa.slnx— run what the environment can (DB-backed suites need10.100.0.35; if unreachable, run every non-DB project under tests/Core, tests/Drivers (unit), tests/Server/Runtime.Tests, tests/Server/OpcUaServer.Tests and record which suites were skipped and why in the commit message).- Docs + commit:
docs: record value-pipeline batching (plan 2026-08-15).
Explicit non-goals (do NOT let scope creep into these)
- No conflation / bounded mailboxes — batching per burst is this plan's backpressure story; keyed conflation is a follow-up.
- No fan-out-map push-down to DriverInstanceActor — the host stays the router.
- No MQTT / MTConnect / Sparkplug / Galaxy / OpcUaClient batch implementations — they flow through the single-item-batch path unchanged.
- No change to alarm paths (
AttributeAlarmPublished,AlarmStateUpdate) — value path only. - No PollGroupEngine cadence change (fixed-rate scheduling is a separate item).
Follow-ups (recorded, not implemented)
Raised during Tasks 1–12 review and deliberately left out of this plan's scope. None is a defect in the delivered work; each is a place where the next change in the area could go wrong quietly.
PollGroupEnginecollects batch items unconditionally — the batch event is only as trustworthy as the driver'sonChange. The engine has no knowledge of a driver's publish filter, so its items list is pre-filter by construction. That is safe today only because S7, AbCip and FOCAS forward everyonChangeunconditionally, and Modbus — the one driver with a deadband — ignores the engine's list and raises from its own filtered buffer. The trap: a future per-tag filter added to S7/AbCip/FOCAS'sonChangewould silently reintroduce the exact deadband-bypass class of bug fixed for Modbus incd4ce305, with a green test suite (the two events would simply disagree, and only a consumer preferring the batch would notice). Consider changing the engine'sonChangecontract toFunc<..., bool>so the engine collects an item only when the driver reports it published — making the filter structurally impossible to skip rather than a convention each driver must remember.- Two-level
driverId → (rawPath → refs)routing map inDriverHostActor. The fan-out map is keyed by a composite of driver id + raw path, so resolving a batch re-hashes the driver id once per item even though a batch is by definition single-driver. Nesting the map would hash the driver id once per batch and the raw path once per item. Measure first — batches are small enough that this may be noise relative to the message-count win already banked. - Null/empty guard consistency on the internal batch messages. The guards on
AttributeValueBatchPublished/AttributeValueBatchUpdategrew per task and are not uniform — some seams drop an empty batch early, others tolerate it downstream. These are internal messages with trusted senders so nothing is unsafe today; worth one pass to make the contract identical at every hop and stated once, rather than re-derived per reader. - Possible merge of
DriverHostActorLiveValueTestsinto the batch suite. After Task 8 both files drive the same fan-out through the same message; the split is now historical rather than meaningful. Merging would remove a genuine ambiguity about where a new fan-out test belongs — but check first that the older suite's per-item comms-loss cases really are covered, since those enter via a different seam. - Aborted-cycle tail-loss when a per-tag handler throws, for the three unbuffered drivers. If a
consumer's
onChangethrows mid-cycle in S7/AbCip/FOCAS, the cycle aborts and the items already collected for that cycle never reach a batch raise. Pre-existing and narrower than the Modbus case (Modbus's own buffer is per-handle and is dropped on unsubscribe/dispose, so it cannot leak), and it predates this plan — the per-tag path lost the same tail before batching existed. Recorded so it is not mistaken for a regression introduced here.