Compare commits
2 Commits
phase-3-pr
...
phase-3-pr
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
77d09bf64e | ||
| 163c821e74 |
103
docs/v2/modbus-test-plan.md
Normal file
103
docs/v2/modbus-test-plan.md
Normal file
@@ -0,0 +1,103 @@
|
||||
# Modbus driver — test plan + device-quirk catalog
|
||||
|
||||
The Modbus TCP driver unit tests (PRs 21–24) cover the protocol surface against an
|
||||
in-memory fake transport. They validate the codec, state machine, and function-code
|
||||
routing against a textbook Modbus server. That's necessary but not sufficient: real PLC
|
||||
populations disagree with the spec in small, device-specific ways, and a driver that
|
||||
passes textbook tests can still misbehave against actual equipment.
|
||||
|
||||
This doc is the harness-and-quirks playbook. It's what gets wired up in the
|
||||
`tests/Driver.Modbus.IntegrationTests` project when we ship that (PR 26 candidate).
|
||||
|
||||
## Harness
|
||||
|
||||
**Chosen simulator: ModbusPal** (Java, scriptable). Rationale:
|
||||
- Scriptable enough to mimic device-specific behaviors (non-standard register
|
||||
layouts, custom exception codes, intentional response delays).
|
||||
- Runs locally, no CI dependency. Tests skip when `localhost:502` (or the configured
|
||||
simulator endpoint) isn't reachable.
|
||||
- Free + long-maintained — physical PLC bench is unavailable in most dev
|
||||
environments, and renting cloud PLCs isn't worth the per-test cost.
|
||||
|
||||
**Setup pattern** (not yet codified in a script — will land alongside the integration
|
||||
test project):
|
||||
1. Install ModbusPal, load the per-device `.xmpp` profile from
|
||||
`tests/Driver.Modbus.IntegrationTests/ModbusPal/` (TBD directory).
|
||||
2. Start the simulator listening on `localhost:502` (or override via
|
||||
`MODBUS_SIM_ENDPOINT` env var).
|
||||
3. `dotnet test` the integration project — tests auto-skip when the endpoint is
|
||||
unreachable, so forgetting to start the simulator doesn't wedge CI.
|
||||
|
||||
## Per-device quirk catalog
|
||||
|
||||
### AutomationDirect DL205
|
||||
|
||||
First known target device. Quirks to document and cover with named tests (to be
|
||||
filled in when user validates each behavior in ModbusPal with a DL205 profile):
|
||||
|
||||
- **Word order for 32-bit values**: _pending_ — confirm whether DL205 uses ABCD
|
||||
(Modbus TCP standard) or CDAB (Siemens-style word-swap) for Int32/UInt32/Float32.
|
||||
Test name: `DL205_Float32_word_order_is_CDAB` (or `ABCD`, whichever proves out).
|
||||
- **Register-zero access**: _pending_ — some DL205 configurations reject FC03 at
|
||||
register 0 with exception code 02 (illegal data address). If confirmed, the
|
||||
integration test suite verifies `ModbusProbeOptions.ProbeAddress` default of 0
|
||||
triggers the rejection and operators must override; test name:
|
||||
`DL205_FC03_at_register_0_returns_IllegalDataAddress`.
|
||||
- **Coil addressing base**: _pending_ — DL205 documentation sometimes uses 1-based
|
||||
coil addresses; verify the driver's zero-based addressing matches the physical
|
||||
PLC without an off-by-one adjustment.
|
||||
- **Maximum registers per FC03**: _pending_ — Modbus spec caps at 125; some DL205
|
||||
models enforce a lower limit (e.g., 64). Test name:
|
||||
`DL205_FC03_beyond_max_registers_returns_IllegalDataValue`.
|
||||
- **Response framing under sustained load**: _pending_ — the driver's
|
||||
single-flight semaphore assumes the server pairs requests/responses by
|
||||
transaction id; at least one DL205 firmware revision is reported to drop the
|
||||
TxId under load. If reproduced in ModbusPal we add a retry + log-and-continue
|
||||
path to `ModbusTcpTransport`.
|
||||
- **Exception code on coil write to a protected bit**: _pending_ — some DL205
|
||||
setups protect internal coils; the driver should surface the PLC's exception
|
||||
PDU as `BadNotWritable` rather than `BadInternalError`.
|
||||
|
||||
_User action item_: as each quirk is validated in ModbusPal, replace the _pending_
|
||||
marker with the confirmed behavior and file a named test in the integration suite.
|
||||
|
||||
### Future devices
|
||||
|
||||
One section per device class, same shape as DL205. Quirks that apply across
|
||||
multiple devices (e.g., "all AB PLCs use CDAB") can be noted in the cross-device
|
||||
patterns section below once we have enough data points.
|
||||
|
||||
## Cross-device patterns
|
||||
|
||||
Once multiple device catalogs accumulate, quirks that recur across two or more
|
||||
vendors get promoted into driver defaults or opt-in options:
|
||||
|
||||
- _(empty — filled in as catalogs grow)_
|
||||
|
||||
## Test conventions
|
||||
|
||||
- **One named test per quirk.** `DL205_word_order_is_CDAB_for_Float32` is easier to
|
||||
diagnose on failure than a generic `Float32_roundtrip`. The `DL205_` prefix makes
|
||||
filtering by device class trivial (`--filter "DisplayName~DL205"`).
|
||||
- **Skip with a clear SkipReason.** Follow the pattern from
|
||||
`GalaxyRepositoryLiveSmokeTests`: check reachability in the fixture, capture
|
||||
a `SkipReason` string, and have each test call `Assert.Skip(SkipReason)` when
|
||||
it's set. Don't throw — skipped tests read cleanly in CI logs.
|
||||
- **Use the real `ModbusTcpTransport`.** Integration tests exercise the wire
|
||||
protocol end-to-end. The in-memory `FakeTransport` from the unit test suite is
|
||||
deliberately not used here — its value is speed + determinism, which doesn't
|
||||
help reproduce device-specific issues.
|
||||
- **Don't depend on ModbusPal state between tests.** Each test resets the
|
||||
simulator's register bank or uses a unique address range. Avoid relying on
|
||||
"previous test left value at register 10" setups that flake when tests run in
|
||||
parallel or re-order.
|
||||
|
||||
## Next concrete PRs
|
||||
|
||||
- **PR 26 — Integration test project + DL205 profile scaffold**: creates
|
||||
`tests/Driver.Modbus.IntegrationTests`, imports the ModbusPal profile (or
|
||||
generates it from JSON), adds the fixture with skip-when-unreachable, plus
|
||||
one smoke test that reads a register. No DL205-specific assertions yet — that
|
||||
waits for the user to validate each quirk.
|
||||
- **PR 27+**: one PR per confirmed DL205 quirk, landing the named test + any
|
||||
driver-side adjustment (e.g., retry on dropped TxId) needed to pass it.
|
||||
Reference in New Issue
Block a user