Compare commits

...

2 Commits

Author SHA1 Message Date
Joseph Doherty
77d09bf64e Phase 3 PR 25 — modbus-test-plan.md: integration-test playbook with per-device quirk catalog. ModbusPal is the chosen simulator; AutomationDirect DL205 is the first target device class with 6 pending quirks to document and cover with named tests (word order for 32-bit values, register-zero access policy, coil addressing base, maximum registers per FC03, response framing under sustained load, exception code on protected-bit coil write). Each quirk placeholder has a proposed test name so the user's validation work translates directly into integration tests. Test conventions section codifies the named-per-quirk pattern, skip-when-unreachable guard, real ModbusTcpTransport usage, and inter-test isolation. Sets up the harness-and-catalog structure future device families (Allen-Bradley Micrologix, Siemens S7-1200 Modbus gateway, Schneider M340, whatever the user hits) will slot into — same per-device catalog shape, cross-device patterns section for recurring quirks that can get promoted into driver defaults. Next concrete PRs proposed: PR 26 for the integration test project scaffold + DL205 profile + fixture with skip-guard + one smoke test, PR 27+ for the individual confirmed quirks one-per-PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 12:45:21 -04:00
163c821e74 Merge pull request 'Phase 3 PR 24 — Modbus PLC data type extensions' (#23) from phase-3-pr24-modbus-types into v2 2026-04-18 12:32:55 -04:00

103
docs/v2/modbus-test-plan.md Normal file
View File

@@ -0,0 +1,103 @@
# Modbus driver — test plan + device-quirk catalog
The Modbus TCP driver unit tests (PRs 2124) cover the protocol surface against an
in-memory fake transport. They validate the codec, state machine, and function-code
routing against a textbook Modbus server. That's necessary but not sufficient: real PLC
populations disagree with the spec in small, device-specific ways, and a driver that
passes textbook tests can still misbehave against actual equipment.
This doc is the harness-and-quirks playbook. It's what gets wired up in the
`tests/Driver.Modbus.IntegrationTests` project when we ship that (PR 26 candidate).
## Harness
**Chosen simulator: ModbusPal** (Java, scriptable). Rationale:
- Scriptable enough to mimic device-specific behaviors (non-standard register
layouts, custom exception codes, intentional response delays).
- Runs locally, no CI dependency. Tests skip when `localhost:502` (or the configured
simulator endpoint) isn't reachable.
- Free + long-maintained — physical PLC bench is unavailable in most dev
environments, and renting cloud PLCs isn't worth the per-test cost.
**Setup pattern** (not yet codified in a script — will land alongside the integration
test project):
1. Install ModbusPal, load the per-device `.xmpp` profile from
`tests/Driver.Modbus.IntegrationTests/ModbusPal/` (TBD directory).
2. Start the simulator listening on `localhost:502` (or override via
`MODBUS_SIM_ENDPOINT` env var).
3. `dotnet test` the integration project — tests auto-skip when the endpoint is
unreachable, so forgetting to start the simulator doesn't wedge CI.
## Per-device quirk catalog
### AutomationDirect DL205
First known target device. Quirks to document and cover with named tests (to be
filled in when user validates each behavior in ModbusPal with a DL205 profile):
- **Word order for 32-bit values**: _pending_ — confirm whether DL205 uses ABCD
(Modbus TCP standard) or CDAB (Siemens-style word-swap) for Int32/UInt32/Float32.
Test name: `DL205_Float32_word_order_is_CDAB` (or `ABCD`, whichever proves out).
- **Register-zero access**: _pending_ — some DL205 configurations reject FC03 at
register 0 with exception code 02 (illegal data address). If confirmed, the
integration test suite verifies `ModbusProbeOptions.ProbeAddress` default of 0
triggers the rejection and operators must override; test name:
`DL205_FC03_at_register_0_returns_IllegalDataAddress`.
- **Coil addressing base**: _pending_ — DL205 documentation sometimes uses 1-based
coil addresses; verify the driver's zero-based addressing matches the physical
PLC without an off-by-one adjustment.
- **Maximum registers per FC03**: _pending_ — Modbus spec caps at 125; some DL205
models enforce a lower limit (e.g., 64). Test name:
`DL205_FC03_beyond_max_registers_returns_IllegalDataValue`.
- **Response framing under sustained load**: _pending_ — the driver's
single-flight semaphore assumes the server pairs requests/responses by
transaction id; at least one DL205 firmware revision is reported to drop the
TxId under load. If reproduced in ModbusPal we add a retry + log-and-continue
path to `ModbusTcpTransport`.
- **Exception code on coil write to a protected bit**: _pending_ — some DL205
setups protect internal coils; the driver should surface the PLC's exception
PDU as `BadNotWritable` rather than `BadInternalError`.
_User action item_: as each quirk is validated in ModbusPal, replace the _pending_
marker with the confirmed behavior and file a named test in the integration suite.
### Future devices
One section per device class, same shape as DL205. Quirks that apply across
multiple devices (e.g., "all AB PLCs use CDAB") can be noted in the cross-device
patterns section below once we have enough data points.
## Cross-device patterns
Once multiple device catalogs accumulate, quirks that recur across two or more
vendors get promoted into driver defaults or opt-in options:
- _(empty — filled in as catalogs grow)_
## Test conventions
- **One named test per quirk.** `DL205_word_order_is_CDAB_for_Float32` is easier to
diagnose on failure than a generic `Float32_roundtrip`. The `DL205_` prefix makes
filtering by device class trivial (`--filter "DisplayName~DL205"`).
- **Skip with a clear SkipReason.** Follow the pattern from
`GalaxyRepositoryLiveSmokeTests`: check reachability in the fixture, capture
a `SkipReason` string, and have each test call `Assert.Skip(SkipReason)` when
it's set. Don't throw — skipped tests read cleanly in CI logs.
- **Use the real `ModbusTcpTransport`.** Integration tests exercise the wire
protocol end-to-end. The in-memory `FakeTransport` from the unit test suite is
deliberately not used here — its value is speed + determinism, which doesn't
help reproduce device-specific issues.
- **Don't depend on ModbusPal state between tests.** Each test resets the
simulator's register bank or uses a unique address range. Avoid relying on
"previous test left value at register 10" setups that flake when tests run in
parallel or re-order.
## Next concrete PRs
- **PR 26 — Integration test project + DL205 profile scaffold**: creates
`tests/Driver.Modbus.IntegrationTests`, imports the ModbusPal profile (or
generates it from JSON), adds the fixture with skip-when-unreachable, plus
one smoke test that reads a register. No DL205-specific assertions yet — that
waits for the user to validate each quirk.
- **PR 27+**: one PR per confirmed DL205 quirk, landing the named test + any
driver-side adjustment (e.g., retry on dropped TxId) needed to pass it.