Files
lmxopcua/docs/plans/2026-07-15-modbus-rtu-driver-design.md
T
Joseph Doherty fd0bec4ffe
v2-ci / build (push) Successful in 3m30s
v2-ci / unit-tests (push) Failing after 8m53s
docs(driver-expansion): resolve all open design concerns from the review pass
Every concern the 7-agent review parked is now decided and integrated:

- universal browser: in-flight capture coalescing keyed (driverType,
  config-hash) + global cap of 4 concurrent captures; CanBrowse catches a
  throwing TryCreate and defensively shuts down the throwaway instance;
  cleanup ShutdownAsync bounded at 10s (R2-01); picker Refresh = close +
  re-capture. Coalesced sessions share an immutable tree, so DisposeAsync
  drops the session's reference, not the tree.
- mtconnect: UNAVAILABLE pinned to BadNoCommunication (fleet's
  BadCommunicationError stays reserved for the driver's own transport
  failures); TIME_SERIES sampleCount flows into ArrayDim; Subscribe timeout
  bounds only the stream-start handshake; library version/TFM folded into
  the license checklist.
- mqtt: browse rebirth is an explicit DriverOperator-gated "Request rebirth"
  button (RequestRebirthAsync(scope)) - never fired by open/root/expand;
  hand-rolled reconnect loop committed for P1 (MQTTnet v5 dropped
  ManagedMqttClient); Wave-2-start library re-verification checkbox;
  implement from the Sparkplug v3.0 spec text.
- bacnet: P1 opens with a first-spike checklist (package TFM, BBMD/
  segmented-RPM/COV API smoke, unicast-I-Am fixture behavior); AdminUI-node
  browse registers a second foreign device - BBMD FD-table sizing note.
- sql-poll: split-node browse needs env parity for Sql__ConnectionStrings__
  refs on admin nodes (actionable error + session-only pasted literal);
  one-row-per-key query contract (last-wins + rate-limited warning);
  absent key -> BadNoData; operationTimeout > commandTimeout authoring rule;
  Browser->Driver.Sql SqlClient transitive on AdminUI accepted on record.
- omron: FINS framer gated on W227 + live golden vectors; CIP string layout
  is the first P1 live-gate item; AbCip harness static-init anti-pattern
  note; writable-defaults-true operator warning.
- modbus-rtu: first P1 step confirms pymodbus exposes framer=rtu on a TCP
  server (custom-script fallback otherwise).
- program doc: new cross-cutting rule - driver ctors must be connection-free
  (the universal browser's CanBrowse throwaway instances depend on it).
2026-07-15 16:54:09 -04:00

19 KiB
Raw Blame History

Modbus RTU (RTU-over-TCP) — implementation design

Status: Design / build-ready. Not implemented. Date: 2026-07-15 Scope: Add the Modbus RTU-over-TCP transport mode to the existing ModbusDriver. Descoped (user, 2026-07-15): the direct-serial transport (System.IO.Ports / ModbusRtuTransport) is not being built. RTU buses are reached exclusively via a serial→Ethernet gateway (§3 explains why that's the right topology for this containerised server anyway). §2b and the serial-specific config/fixture material are retained below, clearly marked, as a design record should bare-metal serial ever be needed. Research input: docs/research/drivers/modbus-rtu.md. Related: docs/drivers/Modbus.md, docs/v2/modbus-addressing.md, docs/plans/2026-07-15-universal-discovery-browser-design.md.


1. Motivation + extend-vs-new verdict — EXTEND

Add Modbus RTU as a new IModbusTransport implementation behind the existing transport seam + driver-level config plumbing to select it. Do not create a sibling driver. The Modbus application protocol above the wire (register model, function codes FC0106/15/16, exception-PDU convention, data-type codecs, byte order, arrays, strings, BCD, bit-in-register, read planner + coalescing, auto-prohibit, deadband, write path, connectivity probe, OPC-UA materialisation, HistoryRead) is byte-for-byte identical across TCP and RTU — only the wire framing and the physical link differ. A second driver would clone that entire surface for zero benefit.

The exact seam this plugs into

The driver already splits the PDU (function code + data) from the transport (framing + link I/O) behind one interface:

src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/IModbusTransport.cs

public interface IModbusTransport : IAsyncDisposable
{
    Task ConnectAsync(CancellationToken ct);
    Task<byte[]> SendAsync(byte unitId, byte[] pdu, CancellationToken ct); // pdu = [fc, ...data], returns response PDU
}
  • ModbusDriver builds every request as a raw PDU byte[] and calls transport.SendAsync(ResolveUnitId(tag), pdu, ct) (e.g. ModbusDriver.cs lines ~1014/1035/1052/1069/1142/1154). It never touches MBAP, sockets, or CRC.
  • ModbusTcpTransport is the only place MBAP framing, the transaction-id counter (_nextTx), and TcpClient/NetworkStream I/O live — specifically SendOnceAsync (ModbusTcpTransport.cs lines 231296) wraps the PDU as [TxId(2)][Proto=0(2)][Length(2)][UnitId(1)] + PDU, single-flights via SemaphoreSlim _gate, and owns reconnect/keepalive/idle-disconnect.
  • The transport is injected: ModbusDriver's ctor takes Func<ModbusDriverOptions, IModbusTransport>? transportFactory (ModbusDriver.cs lines 98114); the default closure builds ModbusTcpTransport. Tests already substitute in-memory fakes through this hook.

So the work is purely additive: one new transport class + a CRC-16 helper + config/factory selection + one AdminUI field. Zero changes to codecs, planner, coalescing, health, materialisation, HistoryRead, or the address parser.

New/changed files at a glance

File Change
…/Driver.Modbus/ModbusCrc.cs new — CRC-16 (poly 0xA001) helper
…/Driver.Modbus/ModbusRtuFraming.cs new — shared ADU frame/deframe + FC-aware response sizing
…/Driver.Modbus/ModbusRtuTransport.cs System.IO.Ports.SerialPort transportdescoped (§2b kept as design record)
…/Driver.Modbus/ModbusRtuOverTcpTransport.cs new — RTU framing over a socket
…/Driver.Modbus/ModbusSocketLifecycle.cs new (refactor) — socket connect/reconnect/keepalive/idle extracted from ModbusTcpTransport, shared by TCP + RtuOverTcp
…/Driver.Modbus/ModbusTransportFactory.cs newCreate(ModbusDriverOptions) switch on Transport; used by the driver default closure and the probe
…/Driver.Modbus.Contracts/ModbusDriverOptions.cs add the Transport discriminator enum
…/Driver.Modbus/ModbusDriverFactoryExtensions.cs DTO fields + wire the default closure to ModbusTransportFactory.Create
…/Driver.Modbus/ModbusDriverProbe.cs build the transport via ModbusTransportFactory (currently hardcodes new ModbusTcpTransport, line 77)
…/Driver.Modbus/ModbusDriver.cs BuildSlaveHostName endpoint string reflects the transport (COM/gateway vs host:port)
…/AdminUI/…/Drivers/ModbusDriverPage.razor Transport selector (Tcp / RtuOverTcp)

No new package dependencySystem.IO.Ports was only needed by the descoped direct-serial transport.


2. Transport implementations

The transport produces the RTU ADU: [slaveAddress(1)][PDU][CRC-lo][CRC-hi] — no MBAP header, no transaction id. CRC-16 (poly 0xA001 reflected, appended low byte first) replaces TCP's transport-level integrity. The RTU framing/deframing lives in its own ModbusRtuFraming class (framing is byte-stream-agnostic, so a future direct-serial transport would share it unchanged).

2a. ModbusRtuFraming (shared)

  • byte[] BuildAdu(byte unitId, ReadOnlySpan<byte> pdu)[unit][pdu][crcLo][crcHi].
  • int ExpectedResponseLength(byte functionCode) / a streaming reader that determines response length by parsing the function code (RTU frames are length-less — see §7). Read addr(1) + fc(1) first, then:
    • Exception (fc & 0x80): read excCode(1) + CRC(2) → 5-byte ADU total; throw ModbusException(fc & 0x7F, excCode, …) after CRC-validating.
    • Read responses FC01/02/03/04: read byteCount(1), then byteCount + CRC(2).
    • Write echoes FC05/06/15/16: fixed addr + qty/value(4) + CRC(2) → read 6 more.
  • After the full ADU is in hand: validate the trailing CRC-16 (mismatch → ModbusTransportDesyncException(DesyncReason.…) so it maps onto the existing BadCommunicationError handling), strip the address byte + CRC, and return the bare response PDU [fc, ...data] — exactly what SendAsync's callers already decode.

2b. ModbusRtuTransport : IModbusTransport (direct serial)DESCOPED (design record only)

Not being built (user, 2026-07-15). Kept verbatim below so a future bare-metal-serial need doesn't re-derive it — the R2-01 BaseStream-ignores-ReadTimeout finding in particular.

  • Wraps a System.IO.Ports.SerialPort. ConnectAsync opens the port with the configured PortName/BaudRate/DataBits/Parity/StopBits; sets ReadTimeout/WriteTimeout from ModbusDriverOptions.Timeout.
  • Per-op deadline (R2-01): SerialPort.ReadTimeout only governs the synchronous read API — async reads over SerialPort.BaseStream ignore ReadTimeout/WriteTimeout entirely (the exact wall-clock gap class R2-01 found in the S7 driver, where async socket reads ignored ReadTimeout and a frozen peer wedged the poll). Every transaction MUST therefore run under a linked-CTS CancellationTokenSource.CancelAfter(Options.Timeout) deadline, mirroring ModbusTcpTransport.SendOnceAsync — timeout ⇒ close the port + surface ModbusTransportDesyncException(DesyncReason.Timeout), distinct from caller cancellation (no teardown).
  • SendAsync: single-flight via the same SemaphoreSlim _gate pattern (mandatory on RTU — no TxId means at most one transaction on the bus at a time). Enforce ≥3.5-char inter-frame idle before transmit (computed from baud/word-length, or the InterFrameDelayMs override), write the ADU, then read the response using ModbusRtuFraming's FC-aware sizing, backstopped by the linked-CTS per-op deadline.
  • Failure model is simpler than TCP — a COM port doesn't "drop" the way a NAT'd socket does. On an I/O error or CRC desync, close + reopen the port once and retry (reuse the single-retry shape from ModbusTcpTransport.SendAsync); no keepalive, no idle-disconnect, no geometric socket backoff.

2c. ModbusRtuOverTcpTransport : IModbusTransport (RTU tunnelled over a socket)

  • Identical RTU framing (ModbusRtuFraming) but the byte stream rides a TcpClient/NetworkStream to a serial→Ethernet gateway instead of a COM port.
  • Reuses the hardened socket lifecycle — extract the connect (IPv4-preference DNS), SO_KEEPALIVE, idle-disconnect, and reconnect-with-backoff machinery from ModbusTcpTransport into ModbusSocketLifecycle and have both TCP variants compose it. The only delta from ModbusTcpTransport is: CRC framing instead of MBAP, and no transaction id. Low-risk to build and test (see §9).

Modbus ASCII is out of scope — a third, rarely-used framing (:-delimited hex, LRC instead of CRC). It would be another IModbusTransport behind the same seam if ever needed; not planned.


3. Why RTU-over-TCP only (the serial reality that drove the descope)

OtOpcUa deploys as a containerised Linux server (docker-dev rig; docker host 10.100.0.35) that is generally not attached to an RS-485 bus. The idiomatic topology is a serial→Ethernet gateway (Moxa NPort, Digi One, Lantronix, USR-TCP232) on the RS-485 multidrop, exposed over TCP. The server talks RTU-over-TCP to it — a plain socket, zero host-device mapping, no System.IO.Ports dependency, no udev fragility, and it reuses the already-hardened socket lifecycle.

Direct serial was descoped because every one of its costs lands on the deployment side: SerialPort is macOS-weak (this repo's dev machine — untestable locally), containers need explicit --device= mapping, USB-serial adapters re-enumerate across replug (needing /dev/serial/by-id/... pinning), and Windows-container COM passthrough is unreliable. A gateway sidesteps all of it for the price of commodity hardware.


4. Config JSON shape

Per-tag TagConfig is unchanged. ModbusTagDefinition / ModbusTagDto already carry everything RTU needs — including the per-tag UnitId override that makes an RS-485 multi-drop bus "just work" (the read planner already refuses to coalesce across UnitIds; ResolveUnitId + BuildSlaveHostName already key per-slave resilience by unit). The ModbusTagConfigEditor in TagConfigEditorMap needs no change. Additions are driver-level only.

One new field on ModbusDriverOptions (and a matching optional field on ModbusDriverConfigDto):

Field Type Applies to Notes
Transport ModbusTransportMode enum (Tcp|RtuOverTcp) all Default Tcp (back-compat: existing configs omit it). An Rtu member is reserved for a future direct-serial leg — do not number-squat it
Host / Port reused Tcp, RtuOverTcp the gateway's socket for RtuOverTcp

Host/Port/UnitId/TimeoutMs/MaxRegistersPerRead/keepalive/reconnect all stay and apply identically to both modes. The descoped direct-serial leg would have added SerialPort/BaudRate/DataBits/Parity/StopBits/InterFrameDelayMs (with local enums in Contracts, NOT the System.IO.Ports BCL enums, to keep Contracts backend-dep-free) — none of that ships now. Per-tag multi-drop still works today via the existing per-tag unitId override (the RS-485 drops sit behind the gateway; the planner already refuses to coalesce across UnitIds).

Example — RTU-over-TCP to a serial→Ethernet gateway

{
  "transport": "RtuOverTcp",
  "host": "10.20.0.50",
  "port": 4001,
  "unitId": 1,
  "timeoutMs": 1500,
  "tags": [
    { "name": "Temp",  "addressString": "30001:I" },
    { "name": "Alarm", "region": "DiscreteInputs", "address": 5, "dataType": "Bool" }
  ]
}

Same host/port shape as Modbus TCP, but the wire frames are raw RTU (address + CRC, no MBAP). The Transport discriminator must match the gateway's configured mode: "Tcp" for a gateway doing Modbus/TCP translation, "RtuOverTcp" for transparent/RTU-passthrough. The two wire formats are mutually unparseable.


5. The JsonStringEnumConverter trap — must handle

Per the project-wide enum-serialization bug (memory: Driver enum-serialization bug), driver pages serialize enums but factory DTOs are string-typed. The new Transport enum must round-trip as a string, or an AdminUI-authored RTU config faults the driver at deploy.

Good news — the plumbing is already correct on both ends and just needs the new field:

  • ModbusDriverPage.razor's serializer (_jsonOpts, line 328334) already has Converters = { new JsonStringEnumConverter() } + camelCase. It serializes a ModbusDriverOptions directly, so a ModbusTransportMode on ModbusDriverOptions emits as "RtuOverTcp" automatically.
  • ModbusDriverFactoryExtensions DTO fields are typed string? and parsed via the existing ParseEnum<T> helper (case-insensitive) — mirror how Family / MelsecSubFamily are already handled. Do not type the DTO field as the enum.
  • ModbusDriverProbe._opts (line 1924) already carries JsonStringEnumConverter.

Add a driver-page/factory round-trip unit test asserting "transport":"RtuOverTcp" (string, not a number) — the same guard that caught S7/Modbus previously.


6. Capability mapping — identical to TCP, four deltas

Same register model, function codes, data types, read/write semantics, coalescing, deadband, WriteOnChangeOnly, connectivity probe (FC03@0). Read + write both fully supported. The only deltas are below the seam:

Delta TCP (today) RtuOverTcp (added)
Physical link TcpClient TcpClient (to the serial→Ethernet gateway)
Framing 7-byte MBAP + TxId [addr][PDU][CRC-16], no TxId
Integrity TCP guarantees app-level CRC-16 (0xA001)
Unit/slave id often 1 central — one bus, many drops by unit id (already supported per-tag)
Response length MBAP Length field no length field → FC-aware sizing (§7)
Timing none single-flight mandatory (no TxId); bus-side T3.5 idle gating is the gateway's job

ResolveHost / BuildSlaveHostName (ModbusDriver.cs line 162, "{host}:{port}/unit{n}") already produces the right per-slave resilience key for RtuOverTcp (gatewayHost:port/unit{n}) — no change needed.

Browseability — NO (reconcile w/ universal browser)

Modbus RTU is not browsable — a flat, untyped register space with no discovery protocol, identical to Modbus TCP. SupportsOnlineDiscovery=false; the driver keeps RediscoverPolicy = Once and materialises the authored tag list into a flat folder. No browser (universal or bespoke). The AdminUI ModbusAddressPickerBody/ModbusAddressBuilder is an address builder (grammar helper), not a live browser, and stays valid for RTU.


7. Resilience / timeout

  • Per-op deadline. The linked-CTS CancelAfter(Options.Timeout) per-op deadline (exactly as ModbusTcpTransport.SendOnceAsync does today) is the hard backstop; a frozen gateway must never wedge a poll (R2-01). FC-aware sizing is the length signal — there is no MBAP length field to trust. Bus-side T1.5/T3.5 inter-frame timing is the serial→Ethernet gateway's responsibility, not this transport's.
  • The one genuinely new correctness risk: RTU response sizing without a length field. Get the FC-aware calculation + exception-PDU short-frame detection right (§2a), or the read hangs to timeout / mis-frames. Cover exhaustively with fake-stream unit tests (below). A CRC mismatch or truncated frame maps to ModbusTransportDesyncException → the existing single reconnect-retry + status mapping.
  • RtuOverTcp reuses the socket lifecycle — keepalive, idle-disconnect, reconnect backoff, IPv4-preference connect — unchanged from ModbusTcpTransport via the extracted ModbusSocketLifecycle.
  • Single-flight is mandatory on RTU (no TxId to correlate an interleaved response) — the existing _gate semaphore already provides it; keep it in both new transports.

8. Test fixtures

  1. RTU-over-TCP against pymodbus (highest ROI, no serial hardware). Add an rtu_over_tcp profile to the existing fixture tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.IntegrationTests/Docker/docker-compose.yml (alongside standard/dl205/mitsubishi/s7_1500/exception_injection), running pymodbus.simulator in RTU-framed-TCP mode. The existing profiles all share host port :5020 and are mutually exclusive by design; bind rtu_over_tcp to a fresh port so it can co-run with standard (and sidestep the known shared-port stale-container trap). Add the project=lmxopcua label per the program-doc fixture convention (the checked-in compose services carry no labels: entries today). Drive it via lmxopcua-fix up modbus rtu_over_tcp + lmxopcua-fix sync modbus. This exercises the real CRC + FC-aware framing end-to-end with no serial anything — the only genuinely new logic. First P1 step: confirm the pymodbus simulator entrypoint this fixture uses actually exposes the RTU framer on a TCP server (pymodbus 3.x supports framer=rtu on TCP servers generally, but the simulator CLI/json config may not surface the knob); if it doesn't, fall back to a small custom server script alongside the existing exception_injector.py pattern.
  2. diagslave (TCP RTU mode) for manual/soak — optional, not unit CI.
  3. Virtual serial pair (socat pty ↔ pymodbus RTU serial slave; com0com on Windows)descoped with the direct-serial transport; would return with it.

Unit-level (no PLC):

  • ModbusCrc table-driven test against known Modbus CRC vectors.
  • ModbusRtuFraming / ModbusRtuOverTcpTransport against an in-memory duplex stream fake (fake one level below the existing IModbusTransport fakes — the byte stream) asserting ADU build, CRC append/validate, FC-aware response parse for each FC group, and exception-PDU short-frame handling.
  • Driver-page/factory round-trip test for the JsonStringEnumConverter guard (§5).

Recommended CI shape: unit (CRC + framing + config round-trip) + the rtu_over_tcp pymodbus profile for integration. No live serial gate — nothing serial ships.


9. Phasing + effort

Effort: LOW — likely the lowest-effort item on the driver roadmap. The seam is purpose-built; net-new code is small and localised.

  • P0 — shared plumbing: ModbusCrc (~30 lines) + ModbusRtuFraming (FC-aware sizing)
    • extract ModbusSocketLifecycle from ModbusTcpTransport (mechanical refactor; ModbusTcpTransport keeps behaviour) + ModbusTransportFactory.Create + the Transport field on ModbusDriverOptions/DTO + factory closure + probe wiring. Unit tests for CRC + framing + config round-trip.
  • P1 — ModbusRtuOverTcpTransport (reuses the socket lifecycle; delta = CRC framing + FC-aware length, no TxId) + rtu_over_tcp pymodbus docker profile + the AdminUI Transport selector. No host-device dependency, testable on the existing pymodbus harness, reuses hardened socket code.
  • P2 — ModbusRtuTransport (direct serial)DESCOPED (user, 2026-07-15). §2b holds the design record if it's ever revived.

Top risk: the length-less-frame response-sizing correctness (FC-aware sizing + exception-PDU short frames). Mitigated by exhaustive fake-stream framing tests and the pymodbus RTU-over-TCP integration profile. All the serial-specific risks (T1.5/T3.5 gating, device enumeration, container device mapping) left with the descope.