Files
lmxopcua/docs/plans/2026-07-15-modbus-rtu-driver-design.md
T
Joseph Doherty fd0bec4ffe
v2-ci / build (push) Successful in 3m30s
v2-ci / unit-tests (push) Failing after 8m53s
docs(driver-expansion): resolve all open design concerns from the review pass
Every concern the 7-agent review parked is now decided and integrated:

- universal browser: in-flight capture coalescing keyed (driverType,
  config-hash) + global cap of 4 concurrent captures; CanBrowse catches a
  throwing TryCreate and defensively shuts down the throwaway instance;
  cleanup ShutdownAsync bounded at 10s (R2-01); picker Refresh = close +
  re-capture. Coalesced sessions share an immutable tree, so DisposeAsync
  drops the session's reference, not the tree.
- mtconnect: UNAVAILABLE pinned to BadNoCommunication (fleet's
  BadCommunicationError stays reserved for the driver's own transport
  failures); TIME_SERIES sampleCount flows into ArrayDim; Subscribe timeout
  bounds only the stream-start handshake; library version/TFM folded into
  the license checklist.
- mqtt: browse rebirth is an explicit DriverOperator-gated "Request rebirth"
  button (RequestRebirthAsync(scope)) - never fired by open/root/expand;
  hand-rolled reconnect loop committed for P1 (MQTTnet v5 dropped
  ManagedMqttClient); Wave-2-start library re-verification checkbox;
  implement from the Sparkplug v3.0 spec text.
- bacnet: P1 opens with a first-spike checklist (package TFM, BBMD/
  segmented-RPM/COV API smoke, unicast-I-Am fixture behavior); AdminUI-node
  browse registers a second foreign device - BBMD FD-table sizing note.
- sql-poll: split-node browse needs env parity for Sql__ConnectionStrings__
  refs on admin nodes (actionable error + session-only pasted literal);
  one-row-per-key query contract (last-wins + rate-limited warning);
  absent key -> BadNoData; operationTimeout > commandTimeout authoring rule;
  Browser->Driver.Sql SqlClient transitive on AdminUI accepted on record.
- omron: FINS framer gated on W227 + live golden vectors; CIP string layout
  is the first P1 live-gate item; AbCip harness static-init anti-pattern
  note; writable-defaults-true operator warning.
- modbus-rtu: first P1 step confirms pymodbus exposes framer=rtu on a TCP
  server (custom-script fallback otherwise).
- program doc: new cross-cutting rule - driver ctors must be connection-free
  (the universal browser's CanBrowse throwaway instances depend on it).
2026-07-15 16:54:09 -04:00

342 lines
19 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Modbus RTU (RTU-over-TCP) — implementation design
**Status:** Design / build-ready. Not implemented.
**Date:** 2026-07-15
**Scope:** Add the Modbus **RTU-over-TCP** transport mode to the existing `ModbusDriver`.
**Descoped (user, 2026-07-15):** the direct-serial transport (`System.IO.Ports` /
`ModbusRtuTransport`) is **not being built**. RTU buses are reached exclusively via a
serial→Ethernet gateway (§3 explains why that's the right topology for this containerised
server anyway). §2b and the serial-specific config/fixture material are retained below,
clearly marked, as a design record should bare-metal serial ever be needed.
**Research input:** [`docs/research/drivers/modbus-rtu.md`](../research/drivers/modbus-rtu.md).
**Related:** [`docs/drivers/Modbus.md`](../drivers/Modbus.md),
[`docs/v2/modbus-addressing.md`](../v2/modbus-addressing.md),
[`docs/plans/2026-07-15-universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md).
---
## 1. Motivation + extend-vs-new verdict — **EXTEND**
Add Modbus RTU as **a new `IModbusTransport` implementation behind the existing
transport seam + driver-level config plumbing to select it.** Do **not** create a
sibling driver. The Modbus application protocol above the wire (register model, function
codes FC0106/15/16, exception-PDU convention, data-type codecs, byte order, arrays,
strings, BCD, bit-in-register, read planner + coalescing, auto-prohibit, deadband,
write path, connectivity probe, OPC-UA materialisation, HistoryRead) is **byte-for-byte
identical** across TCP and RTU — only the wire framing and the physical link differ. A
second driver would clone that entire surface for zero benefit.
### The exact seam this plugs into
The driver already splits the *PDU* (function code + data) from the *transport*
(framing + link I/O) behind one interface:
`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/IModbusTransport.cs`
```csharp
public interface IModbusTransport : IAsyncDisposable
{
Task ConnectAsync(CancellationToken ct);
Task<byte[]> SendAsync(byte unitId, byte[] pdu, CancellationToken ct); // pdu = [fc, ...data], returns response PDU
}
```
- **`ModbusDriver`** builds every request as a raw PDU `byte[]` and calls
`transport.SendAsync(ResolveUnitId(tag), pdu, ct)` (e.g. `ModbusDriver.cs` lines
~1014/1035/1052/1069/1142/1154). It never touches MBAP, sockets, or CRC.
- **`ModbusTcpTransport`** is the *only* place MBAP framing, the transaction-id counter
(`_nextTx`), and `TcpClient`/`NetworkStream` I/O live — specifically `SendOnceAsync`
(`ModbusTcpTransport.cs` lines 231296) wraps the PDU as
`[TxId(2)][Proto=0(2)][Length(2)][UnitId(1)] + PDU`, single-flights via `SemaphoreSlim
_gate`, and owns reconnect/keepalive/idle-disconnect.
- **The transport is injected**: `ModbusDriver`'s ctor takes
`Func<ModbusDriverOptions, IModbusTransport>? transportFactory`
(`ModbusDriver.cs` lines 98114); the default closure builds `ModbusTcpTransport`.
Tests already substitute in-memory fakes through this hook.
So the work is **purely additive**: one new transport class + a CRC-16 helper +
config/factory selection + one AdminUI field. **Zero** changes to codecs, planner,
coalescing, health, materialisation, HistoryRead, or the address parser.
### New/changed files at a glance
| File | Change |
|---|---|
| `…/Driver.Modbus/ModbusCrc.cs` | **new** — CRC-16 (poly `0xA001`) helper |
| `…/Driver.Modbus/ModbusRtuFraming.cs` | **new** — shared ADU frame/deframe + FC-aware response sizing |
| ~~`…/Driver.Modbus/ModbusRtuTransport.cs`~~ | ~~`System.IO.Ports.SerialPort` transport~~**descoped** (§2b kept as design record) |
| `…/Driver.Modbus/ModbusRtuOverTcpTransport.cs` | **new** — RTU framing over a socket |
| `…/Driver.Modbus/ModbusSocketLifecycle.cs` | **new** (refactor) — socket connect/reconnect/keepalive/idle extracted from `ModbusTcpTransport`, shared by TCP + RtuOverTcp |
| `…/Driver.Modbus/ModbusTransportFactory.cs` | **new**`Create(ModbusDriverOptions)` switch on `Transport`; used by the driver default closure **and** the probe |
| `…/Driver.Modbus.Contracts/ModbusDriverOptions.cs` | add the `Transport` discriminator enum |
| `…/Driver.Modbus/ModbusDriverFactoryExtensions.cs` | DTO fields + wire the default closure to `ModbusTransportFactory.Create` |
| `…/Driver.Modbus/ModbusDriverProbe.cs` | build the transport via `ModbusTransportFactory` (currently hardcodes `new ModbusTcpTransport`, line 77) |
| `…/Driver.Modbus/ModbusDriver.cs` | `BuildSlaveHostName` endpoint string reflects the transport (COM/gateway vs host:port) |
| `…/AdminUI/…/Drivers/ModbusDriverPage.razor` | `Transport` selector (Tcp / RtuOverTcp) |
**No new package dependency**`System.IO.Ports` was only needed by the descoped
direct-serial transport.
---
## 2. Transport implementations
The transport produces the **RTU ADU**: `[slaveAddress(1)][PDU][CRC-lo][CRC-hi]`
no MBAP header, no transaction id. CRC-16 (poly `0xA001` reflected, appended
**low byte first**) replaces TCP's transport-level integrity. The RTU framing/deframing
lives in its own `ModbusRtuFraming` class (framing is byte-stream-agnostic, so a future
direct-serial transport would share it unchanged).
### 2a. `ModbusRtuFraming` (shared)
- `byte[] BuildAdu(byte unitId, ReadOnlySpan<byte> pdu)``[unit][pdu][crcLo][crcHi]`.
- `int ExpectedResponseLength(byte functionCode)` / a streaming reader that determines
response length **by parsing the function code** (RTU frames are length-less — see
§7). Read `addr(1)` + `fc(1)` first, then:
- **Exception** (`fc & 0x80`): read `excCode(1) + CRC(2)` → 5-byte ADU total; throw
`ModbusException(fc & 0x7F, excCode, …)` after CRC-validating.
- **Read responses** FC01/02/03/04: read `byteCount(1)`, then `byteCount + CRC(2)`.
- **Write echoes** FC05/06/15/16: fixed `addr + qty/value(4) + CRC(2)` → read 6 more.
- After the full ADU is in hand: validate the trailing CRC-16 (mismatch →
`ModbusTransportDesyncException(DesyncReason.…)` so it maps onto the existing
`BadCommunicationError` handling), strip the address byte + CRC, and return the bare
response PDU `[fc, ...data]` — exactly what `SendAsync`'s callers already decode.
### 2b. ~~`ModbusRtuTransport : IModbusTransport` (direct serial)~~ — **DESCOPED** (design record only)
> Not being built (user, 2026-07-15). Kept verbatim below so a future bare-metal-serial
> need doesn't re-derive it — the R2-01 `BaseStream`-ignores-`ReadTimeout` finding in
> particular.
- Wraps a `System.IO.Ports.SerialPort`. `ConnectAsync` opens the port with the
configured `PortName`/`BaudRate`/`DataBits`/`Parity`/`StopBits`; sets
`ReadTimeout`/`WriteTimeout` from `ModbusDriverOptions.Timeout`.
- **Per-op deadline (R2-01): `SerialPort.ReadTimeout` only governs the synchronous read
API — async reads over `SerialPort.BaseStream` ignore `ReadTimeout`/`WriteTimeout`
entirely** (the exact wall-clock gap class R2-01 found in the S7 driver, where async
socket reads ignored `ReadTimeout` and a frozen peer wedged the poll). Every
transaction MUST therefore run under a linked-CTS
`CancellationTokenSource.CancelAfter(Options.Timeout)` deadline, mirroring
`ModbusTcpTransport.SendOnceAsync` — timeout ⇒ close the port + surface
`ModbusTransportDesyncException(DesyncReason.Timeout)`, distinct from caller
cancellation (no teardown).
- `SendAsync`: single-flight via the same `SemaphoreSlim _gate` pattern (**mandatory** on
RTU — no TxId means at most one transaction on the bus at a time). Enforce ≥3.5-char
inter-frame idle before transmit (computed from baud/word-length, or the
`InterFrameDelayMs` override), write the ADU, then read the response using
`ModbusRtuFraming`'s FC-aware sizing, backstopped by the linked-CTS per-op deadline.
- Failure model is **simpler than TCP** — a COM port doesn't "drop" the way a NAT'd
socket does. On an I/O error or CRC desync, close + reopen the port once and retry
(reuse the single-retry shape from `ModbusTcpTransport.SendAsync`); no keepalive, no
idle-disconnect, no geometric socket backoff.
### 2c. `ModbusRtuOverTcpTransport : IModbusTransport` (RTU tunnelled over a socket)
- **Identical RTU framing** (`ModbusRtuFraming`) but the byte stream rides a
`TcpClient`/`NetworkStream` to a serial→Ethernet gateway instead of a COM port.
- **Reuses the hardened socket lifecycle** — extract the connect (IPv4-preference DNS),
`SO_KEEPALIVE`, idle-disconnect, and reconnect-with-backoff machinery from
`ModbusTcpTransport` into `ModbusSocketLifecycle` and have **both** TCP variants
compose it. The *only* delta from `ModbusTcpTransport` is: CRC framing instead of MBAP,
and no transaction id. Low-risk to build and test (see §9).
**Modbus ASCII is out of scope** — a third, rarely-used framing (`:`-delimited hex,
LRC instead of CRC). It would be another `IModbusTransport` behind the same seam if ever
needed; not planned.
---
## 3. Why RTU-over-TCP only (the serial reality that drove the descope)
OtOpcUa deploys as a containerised Linux server (docker-dev rig; docker host
`10.100.0.35`) that is generally **not** attached to an RS-485 bus. The idiomatic
topology is a **serial→Ethernet gateway** (Moxa NPort, Digi One, Lantronix, USR-TCP232)
on the RS-485 multidrop, exposed over TCP. The server talks **RTU-over-TCP** to it — a
plain socket, **zero host-device mapping, no `System.IO.Ports` dependency, no udev
fragility**, and it reuses the already-hardened socket lifecycle.
Direct serial was descoped because every one of its costs lands on the deployment side:
`SerialPort` is macOS-weak (this repo's dev machine — untestable locally), containers
need explicit `--device=` mapping, USB-serial adapters re-enumerate across replug
(needing `/dev/serial/by-id/...` pinning), and Windows-container COM passthrough is
unreliable. A gateway sidesteps all of it for the price of commodity hardware.
---
## 4. Config JSON shape
**Per-tag `TagConfig` is unchanged.** `ModbusTagDefinition` / `ModbusTagDto` already carry
everything RTU needs — including the per-tag **`UnitId`** override that makes an RS-485
multi-drop bus "just work" (the read planner already refuses to coalesce across UnitIds;
`ResolveUnitId` + `BuildSlaveHostName` already key per-slave resilience by unit). The
`ModbusTagConfigEditor` in `TagConfigEditorMap` needs **no change**. Additions are
**driver-level only**.
One new field on `ModbusDriverOptions` (and a matching optional field on
`ModbusDriverConfigDto`):
| Field | Type | Applies to | Notes |
|---|---|---|---|
| `Transport` | `ModbusTransportMode` enum (`Tcp`\|`RtuOverTcp`) | all | **Default `Tcp`** (back-compat: existing configs omit it). An `Rtu` member is reserved for a future direct-serial leg — do not number-squat it |
| `Host` / `Port` | reused | Tcp, **RtuOverTcp** | the gateway's socket for RtuOverTcp |
`Host`/`Port`/`UnitId`/`TimeoutMs`/`MaxRegistersPerRead`/keepalive/reconnect all stay and
apply identically to both modes. The descoped direct-serial leg would have added
`SerialPort`/`BaudRate`/`DataBits`/`Parity`/`StopBits`/`InterFrameDelayMs` (with local
enums in Contracts, NOT the `System.IO.Ports` BCL enums, to keep Contracts
backend-dep-free) — none of that ships now. Per-tag multi-drop still works today via the
existing per-tag `unitId` override (the RS-485 drops sit behind the gateway; the planner
already refuses to coalesce across UnitIds).
### Example — RTU-over-TCP to a serial→Ethernet gateway
```json
{
"transport": "RtuOverTcp",
"host": "10.20.0.50",
"port": 4001,
"unitId": 1,
"timeoutMs": 1500,
"tags": [
{ "name": "Temp", "addressString": "30001:I" },
{ "name": "Alarm", "region": "DiscreteInputs", "address": 5, "dataType": "Bool" }
]
}
```
Same `host`/`port` shape as Modbus TCP, but the wire frames are raw RTU (address + CRC,
no MBAP). The `Transport` discriminator **must match the gateway's configured mode**:
`"Tcp"` for a gateway doing Modbus/TCP translation, `"RtuOverTcp"` for
transparent/RTU-passthrough. The two wire formats are mutually unparseable.
---
## 5. The `JsonStringEnumConverter` trap — **must handle**
Per the project-wide enum-serialization bug (memory: *Driver enum-serialization bug*),
driver pages serialize enums but factory DTOs are string-typed. The new `Transport` enum
**must round-trip as a string**, or an AdminUI-authored RTU config faults the driver at
deploy.
Good news — the plumbing is **already correct on both ends** and just needs the new field:
- `ModbusDriverPage.razor`'s serializer (`_jsonOpts`, line 328334) already has
`Converters = { new JsonStringEnumConverter() }` + camelCase. It serializes a
`ModbusDriverOptions` directly, so a `ModbusTransportMode` on `ModbusDriverOptions`
emits as `"RtuOverTcp"` automatically.
- `ModbusDriverFactoryExtensions` DTO fields are typed `string?` and parsed via the
existing `ParseEnum<T>` helper (case-insensitive) — mirror how `Family` /
`MelsecSubFamily` are already handled. **Do not** type the DTO field as the enum.
- `ModbusDriverProbe._opts` (line 1924) already carries `JsonStringEnumConverter`.
Add a driver-page/factory round-trip unit test asserting `"transport":"RtuOverTcp"`
(string, not a number) — the same guard that caught S7/Modbus previously.
---
## 6. Capability mapping — identical to TCP, four deltas
Same register model, function codes, data types, read/write semantics, coalescing,
deadband, `WriteOnChangeOnly`, connectivity probe (FC03@0). **Read + write both fully
supported.** The only deltas are below the seam:
| Delta | TCP (today) | RtuOverTcp (added) |
|---|---|---|
| Physical link | `TcpClient` | `TcpClient` (to the serial→Ethernet gateway) |
| Framing | 7-byte MBAP + TxId | `[addr][PDU][CRC-16]`, no TxId |
| Integrity | TCP guarantees | app-level CRC-16 (`0xA001`) |
| Unit/slave id | often 1 | **central** — one bus, many drops by unit id (already supported per-tag) |
| Response length | MBAP `Length` field | **no length field** → FC-aware sizing (§7) |
| Timing | none | single-flight mandatory (no TxId); bus-side T3.5 idle gating is the gateway's job |
`ResolveHost` / `BuildSlaveHostName` (`ModbusDriver.cs` line 162,
`"{host}:{port}/unit{n}"`) already produces the right per-slave resilience key for
RtuOverTcp (`gatewayHost:port/unit{n}`) — no change needed.
### Browseability — **NO** (reconcile w/ universal browser)
Modbus RTU is **not browsable** — a flat, untyped register space with no discovery
protocol, identical to Modbus TCP. `SupportsOnlineDiscovery=false`; the driver keeps
`RediscoverPolicy = Once` and materialises the authored tag list into a flat folder. No
browser (universal or bespoke). The AdminUI `ModbusAddressPickerBody`/`ModbusAddressBuilder`
is an address *builder* (grammar helper), not a live browser, and stays valid for RTU.
---
## 7. Resilience / timeout
- **Per-op deadline.** The linked-CTS `CancelAfter(Options.Timeout)` per-op deadline
(exactly as `ModbusTcpTransport.SendOnceAsync` does today) is the hard backstop; a
frozen gateway must never wedge a poll (R2-01). FC-aware sizing is the length signal —
there is no MBAP length field to trust. Bus-side T1.5/T3.5 inter-frame timing is the
serial→Ethernet gateway's responsibility, not this transport's.
- **The one genuinely new correctness risk: RTU response sizing without a length field.**
Get the FC-aware calculation + exception-PDU short-frame detection right (§2a), or the
read hangs to timeout / mis-frames. Cover exhaustively with fake-stream unit tests
(below). A CRC mismatch or truncated frame maps to
`ModbusTransportDesyncException` → the existing single reconnect-retry + status mapping.
- **RtuOverTcp reuses the socket lifecycle** — keepalive, idle-disconnect, reconnect
backoff, IPv4-preference connect — unchanged from `ModbusTcpTransport` via the extracted
`ModbusSocketLifecycle`.
- **Single-flight is mandatory on RTU** (no TxId to correlate an interleaved response) —
the existing `_gate` semaphore already provides it; keep it in both new transports.
---
## 8. Test fixtures
1. **RTU-over-TCP against pymodbus (highest ROI, no serial hardware).** Add an
`rtu_over_tcp` profile to the existing fixture
`tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.IntegrationTests/Docker/docker-compose.yml`
(alongside `standard`/`dl205`/`mitsubishi`/`s7_1500`/`exception_injection`), running
`pymodbus.simulator` in **RTU-framed-TCP** mode. The existing profiles all share host
port `:5020` and are mutually exclusive by design; bind `rtu_over_tcp` to a fresh port
so it can co-run with `standard` (and sidestep the known shared-port stale-container
trap). Add the `project=lmxopcua` label per the program-doc fixture convention (the
checked-in compose services carry no `labels:` entries today). Drive it via
`lmxopcua-fix up modbus rtu_over_tcp` + `lmxopcua-fix sync modbus`. This exercises the
real CRC + FC-aware framing end-to-end with **no serial anything** — the only
genuinely new logic. **First P1 step: confirm the pymodbus simulator entrypoint this
fixture uses actually exposes the RTU framer on a TCP server** (pymodbus 3.x supports
`framer=rtu` on TCP servers generally, but the simulator CLI/json config may not
surface the knob); if it doesn't, fall back to a small custom server script alongside
the existing `exception_injector.py` pattern.
2. **`diagslave`** (TCP RTU mode) for manual/soak — optional, not unit CI.
3. ~~Virtual serial pair (socat pty ↔ pymodbus RTU serial slave; com0com on Windows)~~
**descoped with the direct-serial transport**; would return with it.
**Unit-level (no PLC):**
- `ModbusCrc` table-driven test against known Modbus CRC vectors.
- `ModbusRtuFraming` / `ModbusRtuOverTcpTransport` against an **in-memory duplex stream
fake** (fake one level below the existing `IModbusTransport` fakes — the byte stream)
asserting ADU build, CRC append/validate, FC-aware response parse for each FC group,
and exception-PDU short-frame handling.
- Driver-page/factory round-trip test for the `JsonStringEnumConverter` guard (§5).
**Recommended CI shape:** unit (CRC + framing + config round-trip) + the `rtu_over_tcp`
pymodbus profile for integration. No live serial gate — nothing serial ships.
---
## 9. Phasing + effort
**Effort: LOW — likely the lowest-effort item on the driver roadmap.** The seam is
purpose-built; net-new code is small and localised.
- **P0 — shared plumbing:** `ModbusCrc` (~30 lines) + `ModbusRtuFraming` (FC-aware sizing)
+ extract `ModbusSocketLifecycle` from `ModbusTcpTransport` (mechanical refactor;
`ModbusTcpTransport` keeps behaviour) + `ModbusTransportFactory.Create` + the `Transport`
field on `ModbusDriverOptions`/DTO + factory closure + probe wiring. Unit tests for CRC +
framing + config round-trip.
- **P1 — `ModbusRtuOverTcpTransport`** (reuses the socket lifecycle; delta = CRC framing +
FC-aware length, no TxId) + `rtu_over_tcp` pymodbus docker profile + the AdminUI
`Transport` selector. No host-device dependency, testable on the existing pymodbus
harness, reuses hardened socket code.
- ~~**P2 — `ModbusRtuTransport`** (direct serial)~~ — **DESCOPED** (user, 2026-07-15).
§2b holds the design record if it's ever revived.
**Top risk:** the length-less-frame response-sizing correctness (FC-aware sizing +
exception-PDU short frames). Mitigated by exhaustive fake-stream framing tests and the
pymodbus RTU-over-TCP integration profile. All the serial-specific risks (T1.5/T3.5
gating, device enumeration, container device mapping) left with the descope.