Root cause: pymodbus 3.13's `pymodbus.simulator` console script hard-codes
asyncio.run(run_main(), debug=True). Debug mode captures a traceback per
scheduled callback and logs callbacks slower than 100 ms (the "took 0.146
seconds" lines in #909). Side by side on the shared docker host (closed-loop
FC03 clients, 2 s deadline): 100 clients got ~120-170 req/s, p99 1.6-1.9 s
with client timeouts in debug mode vs ~2400 req/s, p99 ~135 ms without it.
The shared sim also carries the docker-dev rig's 49 Modbus drivers on each of
central-1/central-2 (98 sockets, ~220 req/s at idle), which is above the
debug-mode capacity, so a test run on top was answered late ("Cancel send,
because not connected!") and timed out.
- run_simulator.py: same CLI as pymodbus.simulator, debug=False, plus a watchdog
thread that exits after 3 failed FC03 probes so restart: unless-stopped
replaces the container (docker has no restart-on-unhealthy).
- fc3_probe.py: stdlib FC03 read with a deadline (MBAP or RTU framing); the
compose healthcheck.
- docker-compose.yml: shared anchor for every pymodbus service: new entrypoint,
`project: lmxopcua` label, restart: unless-stopped, healthcheck, bounded
json-file logs.
- README: why, the numbers, and how to diagnose load.
Joseph Doherty
fc08892eb3