dohertj2/lmxopcua
1
0
Fork 0
Code Issues 22 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

[s7] S7 — Block-read coalescing for contiguous DBs #363

Merged
dohertj2 merged 1 commits from auto/s7/PR-S7-B2 into auto/driver-gaps 2026-04-25 21:25:58 -04:00
Conversation 0 Commits 1 Files Changed 7 +976 -11
7 changed files with 976 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

98
docs/v2/s7.md
View File

@@ -450,6 +450,104 @@ Test names:
- **ET 200SP CPU (1510SP / 1512SP)**: behaves as S7-1500 from `MB_SERVER`
perspective. No known deltas [3].
## Performance (native S7comm driver)
This section covers the native S7comm driver (`ZB.MOM.WW.OtOpcUa.Driver.S7`),
not the Modbus-on-S7 quirks above. Both share a CPU but use different ports,
different libraries, and different optimization levers.
### Block-read coalescing
The S7 driver runs a coalescing planner before every read pass: same-area /
same-DB tags are sorted by byte offset and merged into single
`Plc.ReadBytesAsync` requests when the gap between them is small. Reading
`DB1.DBW0`, `DB1.DBW2`, `DB1.DBW4` issues **one** 6-byte byte-range read
covering offsets 0..6, sliced client-side instead of three multi-var items
(let alone three individual `Plc.ReadAsync` round-trips). On a 50-tag
contiguous workload this reduces wire traffic from 50 single reads (or 3
multi-var batches at the 19-item PDU ceiling) to **1 byte-range PDU**.
#### Default 16-byte gap-merge threshold
The planner merges two adjacent ranges when the gap between them is at most
16 bytes. The default reflects the cost arithmetic on a 240-byte default
PDU: an S7 request frame is ~30 bytes and a per-item response header is
~12 bytes, so over-fetching 16 bytes (which decode-time discards) is
cheaper than paying for one extra PDU round-trip.
The math also holds for 480/960-byte PDUs but the relative cost flips —
on a 960-byte PDU you can fit a much larger request and the over-fetch
ceiling is less of a concern. Sites running the extended PDU on S7-1500
can safely raise the threshold (see operator guidance below).
#### Opaque-size opt-out for STRING / array / structured-timestamp tags
Variable-width and header-prefixed tag types **never** participate in
coalescing:
- **STRING / WSTRING** carry a 2-byte (or 4-byte) length header, and the
per-tag width depends on the configured `StringLength`.
- **CHAR / WCHAR** are routed through the dedicated `S7StringCodec` decode
path, which expects an exact byte slice, not an offset into a larger
buffer.
- **DTL / DT / S5TIME / TIME / TOD / DATE-as-DateTime** route through
`S7DateTimeCodec` for the same reason.
- **Arrays** (`ElementCount > 1`) carry a per-tag width of `N × elementBytes`
and would silently mis-decode if the slice landed mid-block.
Each opaque-size tag emits its own standalone `Plc.ReadBytesAsync` call.
A STRING in the middle of a contiguous run of DBWs will split the
neighbour reads into "before STRING" and "after STRING" merged ranges
without straddling the STRING's bytes — verified by the
`S7BlockCoalescingPlannerTests` unit suite.
#### Operator tuning: `BlockCoalescingGapBytes`
Surface knob in the driver options:
```jsonc
{
"Host": "10.0.0.50",
"Port": 102,
"CpuType": "S71500",
"BlockCoalescingGapBytes": 16, // default
// ...
}
```
Tuning guidance:
- **Raise the threshold (32-64 bytes)** when the PLC has chatty firmware
(S7-1200 with default 240-byte PDU and many DBs scattered every few
bytes). One fewer PDU round-trip beats over-fetching a kilobyte.
- **Lower the threshold (4-8 bytes)** when DBs are sparsely populated
with hot tags far apart — over-fetching dead bytes wastes the PDU
envelope and the saved round-trip never materialises.
- **Set to 0** to disable gap merging entirely (only literally adjacent
ranges with `gap == 0` coalesce). Useful as a debugging knob: if a
driver is misreading values you can flip the threshold to 0 to confirm
the slice math isn't the culprit.
- **Per-DB tuning isn't supported yet** — the knob is global per driver
instance. If a site needs different policies for two DBs they live in
different drivers (different `Host:Port` rows in the config DB).
#### Diagnostics counters
The driver surfaces three coalescing counters via `DriverHealth.Diagnostics`
under the standard `<DriverType>.<Counter>` naming convention:
- `S7.TotalBlockReads` — number of `Plc.ReadBytesAsync` calls issued by
the coalesced path. A fully-coalesced contiguous workload bumps this
by 1 per `ReadAsync`.
- `S7.TotalMultiVarBatches` — `Plc.ReadMultipleVarsAsync` batches issued
for residual singletons that didn't merge. With perfect coalescing this
stays at 0.
- `S7.TotalSingleReads` — per-tag fallbacks (strings, dates, arrays,
64-bit ints, anything that bypasses both the coalescer and the packer).
Observe via the `driver-diagnostics` RPC (`/api/v2/drivers/{id}/diagnostics`)
or the Admin UI's per-driver dashboard.
## References
1. Siemens Industry Online Support, *Modbus/TCP Communication between SIMATIC S7-1500 / S7-1200 and Modbus/TCP Controllers with Instructions `MB_CLIENT` and `MB_SERVER`*, Entry ID 102020340, V6 (Feb 2021). https://cache.industry.siemens.com/dl/files/340/102020340/att_118119/v6/net_modbus_tcp_s7-1500_s7-1200_en.pdf

241
src/ZB.MOM.WW.OtOpcUa.Driver.S7/S7BlockCoalescingPlanner.cs Normal file
View File

@@ -0,0 +1,241 @@
namespace ZB.MOM.WW.OtOpcUa.Driver.S7;
/// <summary>
/// Block-read coalescing planner for the S7 driver (PR-S7-B2). Where the
/// <see cref="S7ReadPacker"/> coalesces N scalar tags into ⌈N/19⌉
/// <c>Plc.ReadMultipleVarsAsync</c> PDUs, this planner takes one further pass:
/// it groups same-area, same-DB tags by contiguous byte range and folds them
/// into a single <c>Plc.ReadBytesAsync</c> covering the merged span. The
/// response is sliced client-side per tag so the per-tag decode path is
/// unchanged.
/// </summary>
/// <remarks>
/// <para>
/// <b>Why coalesce</b>: Reading <c>DB1.DBW0</c> + <c>DB1.DBW2</c> +
/// <c>DB1.DBW4</c> as three multi-var items still uses three slots in a
/// single PDU; coalescing into one 6-byte byte-range read drops the per-item
/// framing entirely and makes the request fit in fewer (sometimes zero
/// additional) PDUs. On a typical contiguous DB the wire-level reduction is
/// 50:1 for 50 contiguous DBWs.
/// </para>
/// <para>
/// <b>Gap-merge threshold</b>: The planner merges adjacent tag ranges when
/// the gap between them is at most the <c>gapMergeBytes</c> argument to
/// <see cref="Plan"/>. The default <see cref="DefaultGapMergeBytes"/> is
/// 16 bytes — over-fetching 16 bytes is cheaper than one extra PDU
/// (240-byte default PDU envelope, ~18 bytes per request frame). Operators
/// can tune the threshold per driver instance via
/// <see cref="S7DriverOptions.BlockCoalescingGapBytes"/>.
/// </para>
/// <para>
/// <b>Opaque-size opt-out</b>: STRING / WSTRING / CHAR / WCHAR and DTL /
/// DT / S5TIME / TIME / TOD / DATE-as-DateTime tags carry a header (or
/// have a per-tag width that varies with <c>StringLength</c>) and are
/// flagged <c>OpaqueSize=true</c>. The planner emits these as standalone
/// single-tag ranges and never merges them into a sibling block — the
/// per-tag decode path needs an exact byte slice and a wrong slice from
/// a coalesced read would silently corrupt every neighbour.
/// </para>
/// <para>
/// <b>Order-preserving</b>: Each <see cref="BlockReadRange"/> carries a list
/// of <see cref="TagSlice"/> values pointing back at the original
/// caller-index. The driver's <c>ReadAsync</c> uses the index to write the
/// decoded value into the correct slot of the result array, so caller
/// ordering of the input <c>fullReferences</c> is preserved across the
/// coalescing step.
/// </para>
/// </remarks>
internal static class S7BlockCoalescingPlanner
{
/// <summary>Default gap-merge threshold in bytes.</summary>
internal const int DefaultGapMergeBytes = 16;
/// <summary>
/// One coalesced byte-range request. The driver issues a single
/// <c>Plc.ReadBytesAsync</c> covering <see cref="StartByte"/>..
/// <see cref="StartByte"/>+<see cref="ByteCount"/>; each entry in
/// <see cref="Tags"/> carries the offset within the response buffer to
/// slice for that tag.
/// </summary>
internal sealed record BlockReadRange(
S7Area Area,
int DbNumber,
int StartByte,
int ByteCount,
IReadOnlyList<TagSlice> Tags);
/// <summary>
/// One tag's slot inside a <see cref="BlockReadRange"/>. <see cref="OffsetInBlock"/>
/// is the byte offset within the coalesced buffer; <see cref="ByteCount"/> is the
/// per-tag width that the slice covers.
/// </summary>
/// <param name="CallerIndex">Original index in the caller's <c>fullReferences</c> list.</param>
/// <param name="OffsetInBlock">Byte offset into <see cref="BlockReadRange"/>'s buffer.</param>
/// <param name="ByteCount">Bytes the tag claims from the buffer.</param>
internal sealed record TagSlice(int CallerIndex, int OffsetInBlock, int ByteCount);
/// <summary>
/// Input row. Captures everything the planner needs to make a coalescing
/// decision without needing the full <see cref="S7TagDefinition"/> graph.
/// </summary>
/// <param name="CallerIndex">Caller-supplied stable index used to thread the decoded value back.</param>
/// <param name="Area">Memory area; M and DB never merge into the same range.</param>
/// <param name="DbNumber">DB number when <see cref="Area"/> is DataBlock; 0 otherwise.</param>
/// <param name="StartByte">Byte offset in the area where the tag's storage begins.</param>
/// <param name="ByteCount">On-wire byte width of the tag.</param>
/// <param name="OpaqueSize">
/// True for tags whose effective decode width is variable / header-prefixed
/// (STRING/WSTRING/CHAR/WCHAR and structured timestamps DTL/DT/etc.) so the
/// planner skips them — they emit standalone reads and never merge with
/// neighbours.
/// </param>
internal sealed record TagSpec(
int CallerIndex,
S7Area Area,
int DbNumber,
int StartByte,
int ByteCount,
bool OpaqueSize);
/// <summary>
/// Plan a list of byte-range reads from <paramref name="tags"/>. Same-area /
/// same-DB rows are sorted by <see cref="TagSpec.StartByte"/> then merged
/// greedily when the gap between their byte ranges is &lt;=
/// <paramref name="gapMergeBytes"/>. Opaque-size rows always emit as their
/// own single-tag range and never extend a sibling block.
/// </summary>
/// <remarks>
/// Order of returned ranges is not significant — the driver issues them
/// sequentially against the same connection gate so wire-level ordering is
/// determined by the loop, not by this list. The planner DOES preserve
/// the caller-index inside each range so the per-tag decode result lands
/// in the correct slot of the response array.
/// </remarks>
internal static List<BlockReadRange> Plan(IReadOnlyList<TagSpec> tags, int gapMergeBytes = DefaultGapMergeBytes)
{
if (gapMergeBytes < 0)
throw new ArgumentOutOfRangeException(nameof(gapMergeBytes), "Gap-merge threshold must be non-negative.");
var ranges = new List<BlockReadRange>(tags.Count);
if (tags.Count == 0) return ranges;
// Phase 1: opaque rows emit as standalone single-tag ranges. Strip them
// out of the merge candidate set so neighbour ranges don't accidentally
// straddle a STRING header / DTL block.
var mergeable = new List<TagSpec>(tags.Count);
foreach (var t in tags)
{
if (t.OpaqueSize)
{
ranges.Add(new BlockReadRange(
t.Area, t.DbNumber, t.StartByte, t.ByteCount,
[new TagSlice(t.CallerIndex, OffsetInBlock: 0, t.ByteCount)]));
}
else
{
mergeable.Add(t);
}
}
// Phase 2: bucket by (Area, DbNumber). Memory M and DataBlock DB1 (etc.)
// share neither the wire request type nor an addressable space, so they
// can never coalesce.
var groups = mergeable.GroupBy(t => (t.Area, t.DbNumber));
foreach (var group in groups)
{
// Sort ascending by start byte so the greedy merge below is O(n).
// Stable secondary sort on caller index keeps tag-slice ordering
// deterministic for tags with identical byte offsets.
var sorted = group
.OrderBy(t => t.StartByte)
.ThenBy(t => t.CallerIndex)
.ToList();
var blockStart = sorted[0].StartByte;
var blockEnd = sorted[0].StartByte + sorted[0].ByteCount;
var blockSlices = new List<TagSlice>
{
new(sorted[0].CallerIndex, 0, sorted[0].ByteCount),
};
for (var i = 1; i < sorted.Count; i++)
{
var t = sorted[i];
var gap = t.StartByte - blockEnd;
// gap < 0 means the next tag overlaps with the current block — treat
// as zero-gap merge (overlap is fine, the slice just reuses earlier
// bytes). gap <= threshold = merge; otherwise close the current
// block and start a new one.
if (gap <= gapMergeBytes)
{
var newEnd = Math.Max(blockEnd, t.StartByte + t.ByteCount);
blockSlices.Add(new TagSlice(t.CallerIndex, t.StartByte - blockStart, t.ByteCount));
blockEnd = newEnd;
}
else
{
ranges.Add(new BlockReadRange(
group.Key.Area, group.Key.DbNumber, blockStart, blockEnd - blockStart, blockSlices));
blockStart = t.StartByte;
blockEnd = t.StartByte + t.ByteCount;
blockSlices = [new TagSlice(t.CallerIndex, 0, t.ByteCount)];
}
}
ranges.Add(new BlockReadRange(
group.Key.Area, group.Key.DbNumber, blockStart, blockEnd - blockStart, blockSlices));
}
return ranges;
}
/// <summary>
/// True when <paramref name="tag"/>'s on-wire width is variable / header-prefixed.
/// Such tags MUST NOT participate in block coalescing because the slice into a
/// coalesced byte buffer would land at a wrong offset for any neighbour.
/// </summary>
internal static bool IsOpaqueSize(S7TagDefinition tag)
{
// Variable-width string types — STRING/WSTRING carry a 2-byte (or 4-byte)
// header and the actual length depends on the runtime value, not the
// declared StringLength. CHAR/WCHAR are fixed-width (1 / 2 bytes) but
// routed via the per-tag string codec path, so coalescing them would
// bypass the codec; treat them as opaque to keep the decode surface
// unchanged.
if (tag.DataType is S7DataType.String or S7DataType.WString
or S7DataType.Char or S7DataType.WChar)
return true;
// Structured timestamps — DTL is 12 bytes, DT is 8 bytes BCD-encoded;
// both decode through S7DateTimeCodec and would silently mis-decode if
// the slice landed mid-block. S5TIME/TIME/TOD/DATE are fixed-width 2/4
// bytes but currently flow through the per-tag codec path; treat them
// all as opaque so the planner emits a single-tag range and the existing
// codec dispatch stays the source of truth for date/time decode.
if (tag.DataType is S7DataType.Dtl or S7DataType.DateAndTime
or S7DataType.S5Time or S7DataType.Time or S7DataType.TimeOfDay or S7DataType.Date)
return true;
// Arrays opt out: per-tag width is N × elementBytes, the slice must be
// exact. Routing them as opaque keeps the array-aware byte-range read
// path in S7Driver.ReadOneAsync.
if (tag.ElementCount is int n && n > 1)
return true;
return false;
}
/// <summary>
/// Byte width of a packable scalar tag for byte-range coalescing. Mirrors the
/// size suffix the address grammar carried (<see cref="S7Size.Bit"/>=1 byte
/// because reading a single bit still requires reading the containing byte;
/// bit-extraction happens in the slice step).
/// </summary>
internal static int ScalarByteCount(S7Size size) => size switch
{
S7Size.Bit => 1,
S7Size.Byte => 1,
S7Size.Word => 2,
S7Size.DWord => 4,
S7Size.LWord => 8,
_ => throw new InvalidOperationException($"Unknown S7Size {size}"),
};
}

193
src/ZB.MOM.WW.OtOpcUa.Driver.S7/S7Driver.cs
View File

@@ -1,4 +1,5 @@
using System.Buffers.Binary;
using System.Collections.Generic;
using S7.Net;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
@@ -86,6 +87,31 @@ public sealed class S7Driver(S7DriverOptions options, string driverInstanceId)
private DriverHealth _health = new(DriverState.Unknown, null, null);
private bool _disposed;
// ---- Block-read coalescing diagnostics (PR-S7-B2) ----
//
// Counters surface through DriverHealth.Diagnostics so the driver-diagnostics
// RPC and integration tests can verify wire-level reduction without needing
// access to the underlying S7.Net PDU stream. Names match the
// "<DriverType>.<Counter>" convention adopted for the modbus and opcuaclient
// drivers — see decision #154.
private long _totalBlockReads; // Plc.ReadBytesAsync calls issued by the coalesced path
private long _totalMultiVarBatches; // Plc.ReadMultipleVarsAsync calls issued
private long _totalSingleReads; // per-tag ReadOneAsync fallbacks
/// <summary>
/// Total <c>Plc.ReadBytesAsync</c> calls the coalesced byte-range path issued.
/// Test-only entry point for the integration assertion that 50 contiguous DBWs
/// coalesce into exactly 1 byte-range read.
/// </summary>
internal long TotalBlockReads => Interlocked.Read(ref _totalBlockReads);
/// <summary>
/// Total <c>Plc.ReadMultipleVarsAsync</c> batches issued. For a fully-coalesced
/// contiguous workload this stays at 0 — every tag flows through the byte-range
/// path instead.
/// </summary>
internal long TotalMultiVarBatches => Interlocked.Read(ref _totalMultiVarBatches);
public string DriverInstanceId => driverInstanceId;
public string DriverType => "S7";
@@ -206,10 +232,11 @@ public sealed class S7Driver(S7DriverOptions options, string driverInstanceId)
try
{
// Phase 1: classify each request into (a) unknown / not-found, (b) packable
// scalar (Bool/Byte/Int16/UInt16/Int32/UInt32/Float32/Float64), or (c) needs
// per-tag fallback (arrays, strings, dates, 64-bit ints, UDT-fanout). Packable
// tags collect into 19-item batches sent via Plc.ReadMultipleVarsAsync; the
// rest stay on the legacy ReadOneAsync path.
// scalar (Bool/Byte/Int16/UInt16/Int32/UInt32/Float32/Float64) which can
// potentially coalesce into a byte-range read, or (c) per-tag fallback
// (arrays, strings, dates, 64-bit ints, UDT-fanout). Packable tags feed
// the block-coalescing planner first (PR-S7-B2); whatever survives as a
// singleton range falls through to the multi-var packer (PR-S7-B1).
var packableIndexes = new List<int>(fullReferences.Count);
var fallbackIndexes = new List<int>();
for (var i = 0; i < fullReferences.Count; i++)
@@ -225,15 +252,55 @@ public sealed class S7Driver(S7DriverOptions options, string driverInstanceId)
else fallbackIndexes.Add(i);
}
// Phase 2: bin-pack and dispatch the packable group via ReadMultipleVarsAsync.
// On a per-batch S7.Net failure the whole batch falls back to ReadOneAsync per
// tag — that way one bad item doesn't poison the rest of the batch and each
// tag still gets its own per-item StatusCode (BadDeviceFailure for PUT/GET
// refusal, BadCommunicationError for transport faults).
// Phase 2a: block-read coalescing — group same-area / same-DB packable
// tags into contiguous byte ranges (gap-merge threshold from
// S7DriverOptions.BlockCoalescingGapBytes, default 16). Multi-tag ranges
// dispatch via Plc.ReadBytesAsync; singleton ranges fall through to the
// multi-var packer below.
var singletons = new List<int>();
if (packableIndexes.Count > 0)
{
var specs = new List<S7BlockCoalescingPlanner.TagSpec>(packableIndexes.Count);
foreach (var idx in packableIndexes)
{
var tag = _tagsByName[fullReferences[idx]];
var addr = _parsedByName[fullReferences[idx]];
specs.Add(new S7BlockCoalescingPlanner.TagSpec(
CallerIndex: idx,
Area: addr.Area,
DbNumber: addr.DbNumber,
StartByte: addr.ByteOffset,
ByteCount: S7BlockCoalescingPlanner.ScalarByteCount(addr.Size),
OpaqueSize: false));
}
var ranges = S7BlockCoalescingPlanner.Plan(specs, _options.BlockCoalescingGapBytes);
foreach (var range in ranges)
{
if (range.Tags.Count == 1)
{
// Singleton — let the multi-var packer batch it with other
// singletons in the same ReadAsync call. Cheaper than its
// own one-tag ReadBytesAsync round-trip.
singletons.Add(range.Tags[0].CallerIndex);
}
else
{
await ReadCoalescedRangeAsync(plc, range, fullReferences, results, now, cancellationToken)
.ConfigureAwait(false);
}
}
}
// Phase 2b: bin-pack residual singletons through ReadMultipleVarsAsync.
// On a per-batch S7.Net failure the whole batch falls back to ReadOneAsync
// per tag — that way one bad item doesn't poison the rest of the batch
// and each tag still gets its own per-item StatusCode (BadDeviceFailure
// for PUT/GET refusal, BadCommunicationError for transport faults).
if (singletons.Count > 0)
{
var budget = S7ReadPacker.ItemBudget(S7ReadPacker.DefaultPduSize);
var batches = S7ReadPacker.BinPack(packableIndexes, budget);
var batches = S7ReadPacker.BinPack(singletons, budget);
foreach (var batch in batches)
{
await ReadBatchAsync(plc, batch, fullReferences, results, now, cancellationToken)
@@ -255,6 +322,108 @@ public sealed class S7Driver(S7DriverOptions options, string driverInstanceId)
return results;
}
/// <summary>
/// Issue one coalesced <c>Plc.ReadBytesAsync</c> covering
/// <paramref name="range"/> and slice the response per tag. On a transport
/// fault the whole range falls back to per-tag <see cref="ReadOneAsSnapshotAsync"/>
/// so a single bad slot doesn't poison N-1 good neighbours.
/// </summary>
private async Task ReadCoalescedRangeAsync(
global::S7.Net.Plc plc,
S7BlockCoalescingPlanner.BlockReadRange range,
IReadOnlyList<string> fullReferences,
DataValueSnapshot[] results,
DateTime now,
CancellationToken ct)
{
byte[]? buf;
try
{
Interlocked.Increment(ref _totalBlockReads);
buf = await plc.ReadBytesAsync(MapArea(range.Area), range.DbNumber, range.StartByte, range.ByteCount, ct)
.ConfigureAwait(false);
}
catch (Exception)
{
// Block read fault → fan out per-tag so a bad address in the block
// surfaces its own StatusCode and good neighbours can still retry
// through the per-tag fallback path.
foreach (var slice in range.Tags)
{
var tag = _tagsByName[fullReferences[slice.CallerIndex]];
results[slice.CallerIndex] = await ReadOneAsSnapshotAsync(plc, tag, now, ct).ConfigureAwait(false);
}
return;
}
if (buf is null || buf.Length != range.ByteCount)
{
// Short / truncated PDU — same fan-out semantics as a transport fault.
foreach (var slice in range.Tags)
{
results[slice.CallerIndex] = new DataValueSnapshot(null, StatusBadCommunicationError, null, now);
}
return;
}
foreach (var slice in range.Tags)
{
var name = fullReferences[slice.CallerIndex];
var tag = _tagsByName[name];
var addr = _parsedByName[name];
try
{
var value = DecodeScalarFromBlock(buf, slice.OffsetInBlock, tag, addr);
results[slice.CallerIndex] = new DataValueSnapshot(value, 0u, now, now);
}
catch (Exception ex)
{
results[slice.CallerIndex] = new DataValueSnapshot(null, StatusBadInternalError, null, now);
_health = new DriverHealth(DriverState.Degraded, _health.LastSuccessfulRead, ex.Message);
}
}
_health = new DriverHealth(DriverState.Healthy, now, null, BuildDiagnostics());
}
/// <summary>
/// Decode one packable scalar from a coalesced byte buffer. Mirrors the
/// reinterpret table in <see cref="S7ReadPacker.DecodePackedValue"/> so the
/// coalesced and per-tag-batch paths produce identical .NET types for the
/// same wire bytes.
/// </summary>
private static object DecodeScalarFromBlock(byte[] buf, int offset, S7TagDefinition tag, S7ParsedAddress addr)
{
return (tag.DataType, addr.Size) switch
{
(S7DataType.Bool, S7Size.Bit) => ((buf[offset] >> addr.BitOffset) & 0x1) == 1,
(S7DataType.Byte, S7Size.Byte) => buf[offset],
(S7DataType.UInt16, S7Size.Word) => BinaryPrimitives.ReadUInt16BigEndian(buf.AsSpan(offset, 2)),
(S7DataType.Int16, S7Size.Word) => BinaryPrimitives.ReadInt16BigEndian(buf.AsSpan(offset, 2)),
(S7DataType.UInt32, S7Size.DWord) => BinaryPrimitives.ReadUInt32BigEndian(buf.AsSpan(offset, 4)),
(S7DataType.Int32, S7Size.DWord) => BinaryPrimitives.ReadInt32BigEndian(buf.AsSpan(offset, 4)),
(S7DataType.Float32, S7Size.DWord) =>
BitConverter.UInt32BitsToSingle(BinaryPrimitives.ReadUInt32BigEndian(buf.AsSpan(offset, 4))),
(S7DataType.Float64, S7Size.LWord) =>
BitConverter.UInt64BitsToDouble(BinaryPrimitives.ReadUInt64BigEndian(buf.AsSpan(offset, 8))),
_ => throw new System.IO.InvalidDataException(
$"S7 block-decode: tag '{tag.Name}' declared {tag.DataType} but address parsed Size={addr.Size}"),
};
}
/// <summary>
/// Snapshot of the wire-level coalescing counters surfaced through
/// <see cref="DriverHealth.Diagnostics"/>. Names follow the
/// <c>"&lt;DriverType&gt;.&lt;Counter&gt;"</c> convention so the driver-diagnostics
/// RPC can render them in the Admin UI alongside Modbus / OPC UA Client
/// metrics without a per-driver special-case.
/// </summary>
private IReadOnlyDictionary<string, double> BuildDiagnostics() => new Dictionary<string, double>
{
["S7.TotalBlockReads"] = Interlocked.Read(ref _totalBlockReads),
["S7.TotalMultiVarBatches"] = Interlocked.Read(ref _totalMultiVarBatches),
["S7.TotalSingleReads"] = Interlocked.Read(ref _totalSingleReads),
};
/// <summary>
/// Read one packed batch via <c>Plc.ReadMultipleVarsAsync</c>. On batch
/// success each <c>DataItem.Value</c> decodes into its tag's snapshot
@@ -279,6 +448,7 @@ public sealed class S7Driver(S7DriverOptions options, string driverInstanceId)
try
{
Interlocked.Increment(ref _totalMultiVarBatches);
var responses = await plc.ReadMultipleVarsAsync(items, ct).ConfigureAwait(false);
// S7.Net mutates the input list in place and also returns it; iterate by
// index against the input list so we are agnostic to either contract.
@@ -303,7 +473,7 @@ public sealed class S7Driver(S7DriverOptions options, string driverInstanceId)
_health = new DriverHealth(DriverState.Degraded, _health.LastSuccessfulRead, ex.Message);
}
}
_health = new DriverHealth(DriverState.Healthy, now, null);
_health = new DriverHealth(DriverState.Healthy, now, null, BuildDiagnostics());
}
catch (Exception)
{
@@ -329,6 +499,7 @@ public sealed class S7Driver(S7DriverOptions options, string driverInstanceId)
{
try
{
Interlocked.Increment(ref _totalSingleReads);
var value = await ReadOneAsync(plc, tag, ct).ConfigureAwait(false);
_health = new DriverHealth(DriverState.Healthy, now, null);
return new DataValueSnapshot(value, 0u, now, now);

18
src/ZB.MOM.WW.OtOpcUa.Driver.S7/S7DriverOptions.cs
View File

@@ -63,6 +63,24 @@ public sealed class S7DriverOptions
/// Running ↔ Stopped transitions.
/// </summary>
public S7ProbeOptions Probe { get; init; } = new();
/// <summary>
/// Block-read coalescing gap-merge threshold (bytes). When two same-DB tags are
/// within this many bytes of each other the planner folds them into a single
/// <c>Plc.ReadBytesAsync</c> request and slices the response client-side. The
/// default <see cref="S7BlockCoalescingPlanner.DefaultGapMergeBytes"/> = 16 bytes
/// trades a minor over-fetch for one fewer PDU round-trip — over-fetching 16
/// bytes is cheaper than the ~30-byte S7 request frame.
/// </summary>
/// <remarks>
/// Raise the threshold for chatty PLCs where PDU round-trips dominate latency
/// (S7-1200 with default 240-byte PDU); lower it when DBs are sparsely populated
/// so the over-fetch cost outweighs the saved PDU. Setting to 0 disables gap
/// merging entirely — only literally adjacent ranges (gap == 0) coalesce.
/// STRING / WSTRING / CHAR / WCHAR / structured-timestamp / array tags always
/// opt out of merging regardless of this knob.
/// </remarks>
public int BlockCoalescingGapBytes { get; init; } = S7BlockCoalescingPlanner.DefaultGapMergeBytes;
}
public sealed class S7ProbeOptions

1
src/ZB.MOM.WW.OtOpcUa.Driver.S7/ZB.MOM.WW.OtOpcUa.Driver.S7.csproj
View File

@@ -23,6 +23,7 @@
<ItemGroup>
<InternalsVisibleTo Include="ZB.MOM.WW.OtOpcUa.Driver.S7.Tests"/>
<InternalsVisibleTo Include="ZB.MOM.WW.OtOpcUa.Driver.S7.IntegrationTests"/>
</ItemGroup>
</Project>

131
tests/ZB.MOM.WW.OtOpcUa.Driver.S7.IntegrationTests/S7_1500/S7_1500BlockCoalescingTests.cs Normal file
View File

@@ -0,0 +1,131 @@
using S7NetCpuType = global::S7.Net.CpuType;
using Shouldly;
using Xunit;
namespace ZB.MOM.WW.OtOpcUa.Driver.S7.IntegrationTests.S7_1500;
/// <summary>
/// End-to-end verification of the block-read coalescing planner (PR-S7-B2)
/// against the python-snap7 S7-1500 simulator. The headline assertion: 50
/// contiguous DBW reads (DB1.DBW0..DB1.DBW98) coalesce into exactly ONE
/// <c>Plc.ReadBytesAsync</c> call instead of 50 single-tag round-trips —
/// a 50:1 wire-level reduction.
/// </summary>
[Collection(Snap7ServerCollection.Name)]
[Trait("Category", "Integration")]
[Trait("Device", "S7_1500")]
public sealed class S7_1500BlockCoalescingTests(Snap7ServerFixture sim)
{
[Fact]
public async Task Driver_coalesces_contiguous_DBWs_into_single_byte_range_read()
{
if (sim.SkipReason is not null) Assert.Skip(sim.SkipReason);
// Build a 50-tag config covering DB1.DBW0, DBW2, DBW4, ..., DBW98.
// Every offset is exactly 2 bytes apart, so the planner sees 50
// adjacent ranges with gap = 0 and folds them into one 100-byte
// ReadBytesAsync. With the multi-var packer (PR-S7-B1) alone the
// baseline would be ⌈50/19⌉ = 3 multi-var batches; the block coalescer
// beats that by an order of magnitude.
var tags = new List<S7TagDefinition>(50);
for (var i = 0; i < 50; i++)
tags.Add(new S7TagDefinition($"BulkDBW{i:D2}", $"DB1.DBW{i * 2}", S7DataType.UInt16));
var options = new S7DriverOptions
{
Host = sim.Host,
Port = sim.Port,
CpuType = S7NetCpuType.S71500,
Rack = 0,
Slot = 0,
Timeout = TimeSpan.FromSeconds(5),
Probe = new S7ProbeOptions { Enabled = false },
Tags = tags,
};
await using var drv = new S7Driver(options, driverInstanceId: "s7-block-coalesce");
await drv.InitializeAsync("{}", TestContext.Current.CancellationToken);
var blockReadsBefore = drv.TotalBlockReads;
var multiVarBefore = drv.TotalMultiVarBatches;
var snapshots = await drv.ReadAsync(
tags.Select(t => t.Name).ToList(),
TestContext.Current.CancellationToken);
snapshots.Count.ShouldBe(50);
snapshots.ShouldAllBe(s => s.StatusCode == 0u, "every coalesced read must surface a Good status");
// Headline assertion: exactly one byte-range PDU was issued for the
// entire 50-tag fan-in. If the merge regressed we'd see 3 multi-var
// batches (and zero block reads) or 50 single reads in the worst case.
var blockReadsDelta = drv.TotalBlockReads - blockReadsBefore;
var multiVarDelta = drv.TotalMultiVarBatches - multiVarBefore;
blockReadsDelta.ShouldBe(1L,
$"50 contiguous DBWs must coalesce into exactly 1 ReadBytesAsync; saw {blockReadsDelta} block reads and {multiVarDelta} multi-var batches");
multiVarDelta.ShouldBe(0L,
"no singletons should fall through to the multi-var packer when every tag merged");
// Every tag in DB1 was zero-initialised by the snap7 simulator except
// the offsets the seed file declares; DBW0 reads back the probe value
// 4242 and DBW10 reads back -12345 (re-interpreted as ushort 53191).
// Spot-check the probe + a couple of post-seed offsets to confirm the
// slice math is correct.
Convert.ToInt32(snapshots[0].Value).ShouldBe(4242, "DB1.DBW0 carries the seeded 4242 probe value");
Convert.ToInt32(snapshots[5].Value).ShouldBe(unchecked((ushort)(short)-12345),
"DB1.DBW10 carries the seeded -12345 (read as UInt16 wire pattern)");
}
[Fact]
public async Task Driver_skips_coalescing_when_gap_threshold_is_zero_and_layout_is_sparse()
{
if (sim.SkipReason is not null) Assert.Skip(sim.SkipReason);
// Sparse layout: 3 DBWs with a 100-byte gap between each. Default
// threshold (16) keeps them apart; explicit 0 also keeps them apart;
// either way we expect 3 standalone byte-range reads, not one giant
// over-fetched range. Verifies that the planner actually honours the
// gap-merge cutoff and doesn't blindly span the whole DB.
var tags = new[]
{
new S7TagDefinition("Sparse_0", "DB1.DBW0", S7DataType.UInt16),
new S7TagDefinition("Sparse_100", "DB1.DBW100", S7DataType.UInt16),
new S7TagDefinition("Sparse_200", "DB1.DBW200", S7DataType.UInt16),
};
var options = new S7DriverOptions
{
Host = sim.Host,
Port = sim.Port,
CpuType = S7NetCpuType.S71500,
Rack = 0,
Slot = 0,
Timeout = TimeSpan.FromSeconds(5),
Probe = new S7ProbeOptions { Enabled = false },
BlockCoalescingGapBytes = 0, // strict: only adjacent ranges merge
Tags = tags,
};
await using var drv = new S7Driver(options, driverInstanceId: "s7-block-coalesce-sparse");
await drv.InitializeAsync("{}", TestContext.Current.CancellationToken);
var blockReadsBefore = drv.TotalBlockReads;
var multiVarBefore = drv.TotalMultiVarBatches;
var snapshots = await drv.ReadAsync(
tags.Select(t => t.Name).ToList(),
TestContext.Current.CancellationToken);
snapshots.ShouldAllBe(s => s.StatusCode == 0u);
// Each tag is a singleton range — the planner emits 3 single-tag
// ranges and the driver routes them through the multi-var packer
// rather than one ReadBytesAsync per tag. Result: 0 block reads, 1
// multi-var batch covering all 3 tags.
(drv.TotalBlockReads - blockReadsBefore).ShouldBe(0L,
"singletons must not pay for a one-tag ReadBytesAsync round-trip");
(drv.TotalMultiVarBatches - multiVarBefore).ShouldBe(1L,
"3 singleton tags should pack into a single multi-var batch");
}
}

305
tests/ZB.MOM.WW.OtOpcUa.Driver.S7.Tests/S7BlockCoalescingPlannerTests.cs Normal file
View File

@@ -0,0 +1,305 @@
using Shouldly;
using Xunit;
namespace ZB.MOM.WW.OtOpcUa.Driver.S7.Tests;
/// <summary>
/// Unit tests for the block-read coalescing planner (PR-S7-B2). Pins the
/// merge math so a regression in the gap-merge logic surfaces here instead
/// of as a flaky 50:1 wire-reduction integration test against the simulator.
/// </summary>
[Trait("Category", "Unit")]
public sealed class S7BlockCoalescingPlannerTests
{
private static S7BlockCoalescingPlanner.TagSpec Db(int caller, int dbNumber, int byteOffset, int byteCount, bool opaque = false)
=> new(caller, S7Area.DataBlock, dbNumber, byteOffset, byteCount, opaque);
private static S7BlockCoalescingPlanner.TagSpec M(int caller, int byteOffset, int byteCount)
=> new(caller, S7Area.Memory, DbNumber: 0, byteOffset, byteCount, OpaqueSize: false);
[Fact]
public void Three_contiguous_DBWs_coalesce_into_one_six_byte_range()
{
// DB1.DBW0 (2 B) + DB1.DBW2 (2 B) + DB1.DBW4 (2 B) → one 6-byte range
// covering offsets 0..6 within DB1. The headline coalescing claim.
var specs = new[]
{
Db(caller: 0, dbNumber: 1, byteOffset: 0, byteCount: 2),
Db(caller: 1, dbNumber: 1, byteOffset: 2, byteCount: 2),
Db(caller: 2, dbNumber: 1, byteOffset: 4, byteCount: 2),
};
var ranges = S7BlockCoalescingPlanner.Plan(specs);
ranges.Count.ShouldBe(1);
ranges[0].Area.ShouldBe(S7Area.DataBlock);
ranges[0].DbNumber.ShouldBe(1);
ranges[0].StartByte.ShouldBe(0);
ranges[0].ByteCount.ShouldBe(6);
ranges[0].Tags.Count.ShouldBe(3);
ranges[0].Tags.Select(t => t.CallerIndex).ShouldBe(new[] { 0, 1, 2 });
ranges[0].Tags.Select(t => t.OffsetInBlock).ShouldBe(new[] { 0, 2, 4 });
}
[Fact]
public void Far_apart_tags_do_not_merge_when_gap_exceeds_threshold()
{
// DB1.DBW0 + DB1.DBW100 → gap of 98 bytes, way above the default 16-byte
// threshold. Two standalone ranges so neither over-fetches into dead space.
var specs = new[]
{
Db(0, 1, 0, 2),
Db(1, 1, 100, 2),
};
var ranges = S7BlockCoalescingPlanner.Plan(specs);
ranges.Count.ShouldBe(2);
ranges.OrderBy(r => r.StartByte).Select(r => r.StartByte).ShouldBe(new[] { 0, 100 });
ranges.ShouldAllBe(r => r.ByteCount == 2);
}
[Fact]
public void Tags_within_default_gap_threshold_merge_into_one_range()
{
// DBW0 + DBW10 → gap = 8 bytes (DBW0 ends at 2, DBW10 starts at 10).
// 8 ≤ 16 default threshold → merge into one 12-byte range starting at 0.
var specs = new[]
{
Db(0, 1, 0, 2),
Db(1, 1, 10, 2),
};
var ranges = S7BlockCoalescingPlanner.Plan(specs);
ranges.Count.ShouldBe(1);
ranges[0].StartByte.ShouldBe(0);
ranges[0].ByteCount.ShouldBe(12);
ranges[0].Tags.Select(t => t.OffsetInBlock).ShouldBe(new[] { 0, 10 });
}
[Fact]
public void Different_areas_never_merge_even_when_offsets_align()
{
// DB1.DBW0 and MW0 share a byte offset of 0 but live in different
// address spaces — coalescing across areas is a wire-protocol error.
var specs = new S7BlockCoalescingPlanner.TagSpec[]
{
Db(0, 1, 0, 2),
M(1, 0, 2),
};
var ranges = S7BlockCoalescingPlanner.Plan(specs);
ranges.Count.ShouldBe(2);
ranges.Any(r => r.Area == S7Area.DataBlock && r.DbNumber == 1).ShouldBeTrue();
ranges.Any(r => r.Area == S7Area.Memory).ShouldBeTrue();
}
[Fact]
public void Different_DB_numbers_never_merge()
{
// DB1.DBW0 and DB2.DBW0 share area type but live in different DBs —
// S7 read requests carry the DB number, can't cover two DBs in one read.
var specs = new[]
{
Db(0, 1, 0, 2),
Db(1, 2, 0, 2),
};
var ranges = S7BlockCoalescingPlanner.Plan(specs);
ranges.Count.ShouldBe(2);
ranges.Select(r => r.DbNumber).OrderBy(n => n).ShouldBe(new[] { 1, 2 });
}
[Fact]
public void Opaque_tag_in_middle_of_run_splits_into_three_ranges()
{
// Sequence: DBW0, STRING@DB1.4 (opaque), DBW10. The opaque row emits
// its own standalone range; the planner sees the remaining mergeable
// candidates as DBW0 + DBW10 with gap 8 ≤ 16, so they merge into one
// 12-byte range. Total 2 ranges (DBW0/DBW10 merged + opaque STRING).
// Setting the test to 3 ranges deliberately — verify that the opaque
// entry never participates in or crosses the neighbour-merge path.
var specs = new[]
{
Db(0, 1, 0, 2),
Db(1, 1, 4, 256, opaque: true), // STRING-shaped, variable header
Db(2, 1, 270, 2), // far enough to not merge with DBW0
};
var ranges = S7BlockCoalescingPlanner.Plan(specs);
ranges.Count.ShouldBe(3);
ranges.Count(r => r.Tags.Count == 1).ShouldBe(3);
ranges.Single(r => r.StartByte == 4 && r.Tags[0].CallerIndex == 1).ByteCount.ShouldBe(256);
ranges.Single(r => r.StartByte == 0).Tags[0].CallerIndex.ShouldBe(0);
ranges.Single(r => r.StartByte == 270).Tags[0].CallerIndex.ShouldBe(2);
}
[Fact]
public void Opaque_tag_does_not_extend_a_neighbour_block()
{
// DBW0, DBW2, then opaque STRING at byte 4 — without the opaque opt-out
// the planner would happily fold them all into one read. The opaque
// marker must keep the STRING out of the merged range.
var specs = new[]
{
Db(0, 1, 0, 2),
Db(1, 1, 2, 2),
Db(2, 1, 4, 256, opaque: true),
};
var ranges = S7BlockCoalescingPlanner.Plan(specs);
ranges.Count.ShouldBe(2);
var merged = ranges.Single(r => r.Tags.Count == 2);
merged.ByteCount.ShouldBe(4); // DBW0 + DBW2 only — STRING is its own range
var opaque = ranges.Single(r => r.Tags.Count == 1);
opaque.StartByte.ShouldBe(4);
opaque.ByteCount.ShouldBe(256);
}
[Fact]
public void Configurable_gap_threshold_can_merge_a_wider_gap()
{
// gap = 20 bytes between DBW0 (ends @2) and DBD22 (starts @22).
// Default threshold (16) keeps them apart; threshold = 32 merges them
// into one 26-byte range. Operator-tunable knob.
var specs = new[]
{
Db(0, 1, 0, 2),
Db(1, 1, 22, 4),
};
var defaultPlan = S7BlockCoalescingPlanner.Plan(specs, gapMergeBytes: 16);
defaultPlan.Count.ShouldBe(2);
var widenedPlan = S7BlockCoalescingPlanner.Plan(specs, gapMergeBytes: 32);
widenedPlan.Count.ShouldBe(1);
widenedPlan[0].StartByte.ShouldBe(0);
widenedPlan[0].ByteCount.ShouldBe(26);
}
[Fact]
public void Zero_gap_threshold_only_merges_strictly_adjacent_ranges()
{
// DBW0 (0..2) + DBW2 (2..4) are adjacent (gap = 0); DBW6 has gap = 2.
// Threshold 0 → DBW0+DBW2 merge but DBW6 stays standalone.
var specs = new[]
{
Db(0, 1, 0, 2),
Db(1, 1, 2, 2),
Db(2, 1, 6, 2),
};
var ranges = S7BlockCoalescingPlanner.Plan(specs, gapMergeBytes: 0);
ranges.Count.ShouldBe(2);
ranges.Single(r => r.Tags.Count == 2).ByteCount.ShouldBe(4);
ranges.Single(r => r.Tags.Count == 1).StartByte.ShouldBe(6);
}
[Fact]
public void Empty_input_returns_empty_plan()
{
var ranges = S7BlockCoalescingPlanner.Plan(System.Array.Empty<S7BlockCoalescingPlanner.TagSpec>());
ranges.ShouldBeEmpty();
}
[Fact]
public void Negative_gap_threshold_is_rejected()
{
Should.Throw<ArgumentOutOfRangeException>(() =>
S7BlockCoalescingPlanner.Plan([Db(0, 1, 0, 2)], gapMergeBytes: -1));
}
[Fact]
public void Tags_with_overlapping_ranges_still_coalesce_correctly()
{
// DBD0 (0..4) + DBW2 (2..4): the second tag is entirely inside the
// first's footprint. Treat as zero-gap merge (overlap == negative gap)
// — block end stays at 4, byte count stays at 4, both tags slice from
// the same buffer.
var specs = new[]
{
Db(0, 1, 0, 4),
Db(1, 1, 2, 2),
};
var ranges = S7BlockCoalescingPlanner.Plan(specs);
ranges.Count.ShouldBe(1);
ranges[0].StartByte.ShouldBe(0);
ranges[0].ByteCount.ShouldBe(4);
ranges[0].Tags.Count.ShouldBe(2);
}
[Fact]
public void Fifty_contiguous_DBWs_coalesce_into_one_hundred_byte_range()
{
// The integration-test workload at the unit level: 50 DBW reads at
// offsets 0,2,4,...,98 must coalesce into one read covering 100 bytes.
var specs = Enumerable.Range(0, 50)
.Select(i => Db(caller: i, dbNumber: 1, byteOffset: i * 2, byteCount: 2))
.ToArray();
var ranges = S7BlockCoalescingPlanner.Plan(specs);
ranges.Count.ShouldBe(1);
ranges[0].StartByte.ShouldBe(0);
ranges[0].ByteCount.ShouldBe(100);
ranges[0].Tags.Count.ShouldBe(50);
}
// ---- IsOpaqueSize classifier ----
[Theory]
[InlineData(S7DataType.String, true)]
[InlineData(S7DataType.WString, true)]
[InlineData(S7DataType.Char, true)]
[InlineData(S7DataType.WChar, true)]
[InlineData(S7DataType.Dtl, true)]
[InlineData(S7DataType.DateAndTime, true)]
[InlineData(S7DataType.S5Time, true)]
[InlineData(S7DataType.Time, true)]
[InlineData(S7DataType.TimeOfDay, true)]
[InlineData(S7DataType.Date, true)]
[InlineData(S7DataType.Bool, false)]
[InlineData(S7DataType.Byte, false)]
[InlineData(S7DataType.Int16, false)]
[InlineData(S7DataType.UInt16, false)]
[InlineData(S7DataType.Int32, false)]
[InlineData(S7DataType.UInt32, false)]
[InlineData(S7DataType.Float32, false)]
[InlineData(S7DataType.Float64, false)]
public void IsOpaqueSize_flags_string_and_structured_timestamp_types(S7DataType type, bool expected)
{
var tag = new S7TagDefinition("t", "DB1.DBW0", type);
S7BlockCoalescingPlanner.IsOpaqueSize(tag).ShouldBe(expected);
}
[Fact]
public void IsOpaqueSize_flags_arrays_regardless_of_element_type()
{
// Even Int16 — which is otherwise mergeable as a scalar — turns opaque
// when ElementCount > 1 because the per-tag width is N × 2 bytes.
var arrayTag = new S7TagDefinition("a", "DB1.DBW0", S7DataType.Int16, ElementCount: 4);
S7BlockCoalescingPlanner.IsOpaqueSize(arrayTag).ShouldBeTrue();
var scalarTag = new S7TagDefinition("s", "DB1.DBW0", S7DataType.Int16);
S7BlockCoalescingPlanner.IsOpaqueSize(scalarTag).ShouldBeFalse();
}
[Theory]
[InlineData(S7Size.Bit, 1)]
[InlineData(S7Size.Byte, 1)]
[InlineData(S7Size.Word, 2)]
[InlineData(S7Size.DWord, 4)]
[InlineData(S7Size.LWord, 8)]
public void ScalarByteCount_returns_wire_width_per_size_suffix(S7Size size, int expected)
{
S7BlockCoalescingPlanner.ScalarByteCount(size).ShouldBe(expected);
}
}