lmxopcua/docs/v2/modbus-test-plan.md

Modbus driver — test plan + device-quirk catalog

The Modbus TCP driver unit tests (PRs 21–24) cover the protocol surface against an in-memory fake transport. They validate the codec, state machine, and function-code routing against a textbook Modbus server. That's necessary but not sufficient: real PLC populations disagree with the spec in small, device-specific ways, and a driver that passes textbook tests can still misbehave against actual equipment.

This doc is the harness-and-quirks playbook. It's what gets wired up in the tests/Driver.Modbus.IntegrationTests project when we ship that (PR 26 candidate).

Harness

Chosen simulator: ModbusPal (Java, scriptable). Rationale:

• Scriptable enough to mimic device-specific behaviors (non-standard register layouts, custom exception codes, intentional response delays).
• Runs locally, no CI dependency. Tests skip when localhost:502 (or the configured simulator endpoint) isn't reachable.
• Free + long-maintained — physical PLC bench is unavailable in most dev environments, and renting cloud PLCs isn't worth the per-test cost.

Setup pattern (not yet codified in a script — will land alongside the integration test project):

1. Install ModbusPal, load the per-device .xmpp profile from tests/Driver.Modbus.IntegrationTests/ModbusPal/ (TBD directory).
2. Start the simulator listening on localhost:502 (or override via MODBUS_SIM_ENDPOINT env var).
3. dotnet test the integration project — tests auto-skip when the endpoint is unreachable, so forgetting to start the simulator doesn't wedge CI.

Per-device quirk catalog

AutomationDirect DL205

First known target device. Quirks to document and cover with named tests (to be filled in when user validates each behavior in ModbusPal with a DL205 profile):

• Word order for 32-bit values: pending — confirm whether DL205 uses ABCD (Modbus TCP standard) or CDAB (Siemens-style word-swap) for Int32/UInt32/Float32. Test name: DL205_Float32_word_order_is_CDAB (or ABCD, whichever proves out).
• Register-zero access: pending — some DL205 configurations reject FC03 at register 0 with exception code 02 (illegal data address). If confirmed, the integration test suite verifies ModbusProbeOptions.ProbeAddress default of 0 triggers the rejection and operators must override; test name: DL205_FC03_at_register_0_returns_IllegalDataAddress.
• Coil addressing base: pending — DL205 documentation sometimes uses 1-based coil addresses; verify the driver's zero-based addressing matches the physical PLC without an off-by-one adjustment.
• Maximum registers per FC03: pending — Modbus spec caps at 125; some DL205 models enforce a lower limit (e.g., 64). Test name: DL205_FC03_beyond_max_registers_returns_IllegalDataValue.
• Response framing under sustained load: pending — the driver's single-flight semaphore assumes the server pairs requests/responses by transaction id; at least one DL205 firmware revision is reported to drop the TxId under load. If reproduced in ModbusPal we add a retry + log-and-continue path to ModbusTcpTransport.
• Exception code on coil write to a protected bit: pending — some DL205 setups protect internal coils; the driver should surface the PLC's exception PDU as BadNotWritable rather than BadInternalError.

User action item: as each quirk is validated in ModbusPal, replace the pending marker with the confirmed behavior and file a named test in the integration suite.

Future devices

One section per device class, same shape as DL205. Quirks that apply across multiple devices (e.g., "all AB PLCs use CDAB") can be noted in the cross-device patterns section below once we have enough data points.

Cross-device patterns

Once multiple device catalogs accumulate, quirks that recur across two or more vendors get promoted into driver defaults or opt-in options:

• (empty — filled in as catalogs grow)

Test conventions

• One named test per quirk. DL205_word_order_is_CDAB_for_Float32 is easier to diagnose on failure than a generic Float32_roundtrip. The DL205_ prefix makes filtering by device class trivial (--filter "DisplayName~DL205").
• Skip with a clear SkipReason. Follow the pattern from GalaxyRepositoryLiveSmokeTests: check reachability in the fixture, capture a SkipReason string, and have each test call Assert.Skip(SkipReason) when it's set. Don't throw — skipped tests read cleanly in CI logs.
• Use the real ModbusTcpTransport. Integration tests exercise the wire protocol end-to-end. The in-memory FakeTransport from the unit test suite is deliberately not used here — its value is speed + determinism, which doesn't help reproduce device-specific issues.
• Don't depend on ModbusPal state between tests. Each test resets the simulator's register bank or uses a unique address range. Avoid relying on "previous test left value at register 10" setups that flake when tests run in parallel or re-order.

Next concrete PRs

• PR 26 — Integration test project + DL205 profile scaffold: creates tests/Driver.Modbus.IntegrationTests, imports the ModbusPal profile (or generates it from JSON), adds the fixture with skip-when-unreachable, plus one smoke test that reads a register. No DL205-specific assertions yet — that waits for the user to validate each quirk.
• PR 27+: one PR per confirmed DL205 quirk, landing the named test + any driver-side adjustment (e.g., retry on dropped TxId) needed to pass it.