Task #253 follow-up — fix test-all.ps1 StrictMode crash on missing JSON keys #215

Merged
dohertj2 merged 1 commits from task-253d-e2e-debug-harness into v2 2026-04-21 10:57:36 -04:00
Owner

Summary

Running the e2e suite end-to-end surfaced one more harness bug: test-all.ps1 crashed with "The property 'X' cannot be found on this object" whenever the sidecar JSON omitted an optional key.

Root cause

_common.ps1 sets Set-StrictMode -Version 3.0. Under strict mode, missing-property access on a PSCustomObject (what ConvertFrom-Json returns by default) throws. Every $config.<driver>.<optional> ?? $default + if ($config.<missing-section>) was unsafe.

Fix

  • ConvertFrom-Json -AsHashtable — hashtables tolerate .ContainsKey() / indexer lookup under strict mode.
  • Get-Or helper: Get-Or $table "key" $default returns the value if present, else the default.
  • Rewrite every per-driver section (Modbus / AB CIP / AB Legacy / S7 / FOCAS / TwinCAT / Phase 7) to use the helper.

Verified end-to-end

With docker compose up for pymodbus / ab_server / python-snap7 fixtures, plus the three driver sections populated in a local e2e-config.json:

==================== FINAL MATRIX ====================
  modbus     FAIL  # stages 1+2 PASS, 3-5 FAIL (blocked on #209)
  abcip      FAIL  # same
  ablegacy   SKIP (no config entry)
  s7         FAIL  # same
  focas      SKIP (no config entry)
  twincat    SKIP (no config entry)
  phase7     SKIP (no config entry)

Also ran each per-driver script directly:

  • test-modbus.ps1 / test-abcip.ps1 / test-s7.ps1 — 2/5 PASS (probe + driver loopback), 3/5 FAIL at the bridge as expected (server-wiring blocker #209).
  • test-focas.ps1 / test-twincat.ps1 / test-ablegacy.ps1 — SKIP with clear gate messages when the respective *_TRUST_WIRE env var isn't set.

Test plan

  • Harness runs to completion against live fixtures
  • Final matrix renders PASS / FAIL / SKIP correctly
  • SKIP gates fire for hardware-only drivers
  • All-green run — blocked on #209 (server factory wiring)
## Summary Running the e2e suite end-to-end surfaced one more harness bug: `test-all.ps1` crashed with "The property 'X' cannot be found on this object" whenever the sidecar JSON omitted an optional key. ## Root cause `_common.ps1` sets `Set-StrictMode -Version 3.0`. Under strict mode, missing-property access on a `PSCustomObject` (what `ConvertFrom-Json` returns by default) throws. Every `$config.<driver>.<optional> ?? $default` + `if ($config.<missing-section>)` was unsafe. ## Fix - `ConvertFrom-Json -AsHashtable` — hashtables tolerate `.ContainsKey()` / indexer lookup under strict mode. - `Get-Or` helper: `Get-Or $table "key" $default` returns the value if present, else the default. - Rewrite every per-driver section (Modbus / AB CIP / AB Legacy / S7 / FOCAS / TwinCAT / Phase 7) to use the helper. ## Verified end-to-end With `docker compose up` for pymodbus / ab_server / python-snap7 fixtures, plus the three driver sections populated in a local `e2e-config.json`: ``` ==================== FINAL MATRIX ==================== modbus FAIL # stages 1+2 PASS, 3-5 FAIL (blocked on #209) abcip FAIL # same ablegacy SKIP (no config entry) s7 FAIL # same focas SKIP (no config entry) twincat SKIP (no config entry) phase7 SKIP (no config entry) ``` Also ran each per-driver script directly: - `test-modbus.ps1` / `test-abcip.ps1` / `test-s7.ps1` — 2/5 PASS (probe + driver loopback), 3/5 FAIL at the bridge as expected (server-wiring blocker #209). - `test-focas.ps1` / `test-twincat.ps1` / `test-ablegacy.ps1` — SKIP with clear gate messages when the respective `*_TRUST_WIRE` env var isn't set. ## Test plan - [x] Harness runs to completion against live fixtures - [x] Final matrix renders PASS / FAIL / SKIP correctly - [x] SKIP gates fire for hardware-only drivers - [ ] All-green run — blocked on #209 (server factory wiring)
dohertj2 added 1 commit 2026-04-21 10:57:32 -04:00
Running `test-all.ps1` end-to-end with a partial sidecar (only modbus/
abcip/s7 populated, no focas/twincat/phase7) crashed:

    [FAIL] modbus runner crashed: The property 'opcUaUrl' cannot be
    found on this object. Verify that the property exists.

Root cause: `_common.ps1` sets `Set-StrictMode -Version 3.0`, which
turns missing-property access on PSCustomObject into a throw. Every
`$config.<driver>.<optional-field> ?? $default` and `if
($config.<missing-section>)` check is therefore unsafe against a
normal JSON where optional fields are omitted.

Fix: switch to `ConvertFrom-Json -AsHashtable` and add a `Get-Or`
helper. Hashtables tolerate `.ContainsKey()` / indexer access even
under StrictMode, so the per-driver sections now read:

    $modbus = Get-Or $config "modbus"
    if ($modbus) {
        ... -OpcUaUrl (Get-Or $modbus "opcUaUrl" $OpcUaUrl) ...
    }

Verified end-to-end with live docker-compose fixtures:
 - Modbus / AB CIP / S7 each run to completion, report 2/5 PASS (the
   driver-only stages) and FAIL the 3 server-bridge stages (expected —
   server-side factory wiring is blocked on #209).
 - The FINAL MATRIX header renders cleanly with SKIP rows for the
   drivers not present in the sidecar + FAIL rows for the present ones.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
dohertj2 merged commit 16d9592a8a into v2 2026-04-21 10:57:36 -04:00
dohertj2 deleted branch task-253d-e2e-debug-harness 2026-04-21 10:57:36 -04:00
dohertj2 referenced this issue from a commit 2026-04-30 08:21:26 -04:00
OpcUaClient integration fixture — opc-plc in Docker closes the wire-level gap (#215). Closes task #215. The OpcUaClient driver had the richest capability matrix in the fleet (reads/writes/subscribe/alarms/history across 11 unit-test classes) + zero wire-level coverage; every test mocked the Session surface. opc-plc is Microsoft Industrial IoT's OPC UA PLC simulator — already containerized, already on MCR, pinned to 2.14.10 here. Wins vs the loopback-against-our-own-server option we'd originally scoped: (a) independent cert chain + user-token handling catches interop bugs loopback can't because both endpoints would share our own cert store; (b) pinned image tag fixes the test surface in a way our evolving server wouldn't; (c) the --alm flag opens the door to real IAlarmSource coverage later without building a custom FakeAlarmDriver. Loss vs loopback: both use the OPCFoundation.NetStandard stack internally so bugs common to that stack don't surface — addressed by a follow-up to add open62541/open62541 as a second independent-stack image (tracked). Docker is the fixture launcher — no PowerShell/Python wrapper like Modbus/pymodbus or S7/python-snap7 because opc-plc ships containerized. Docker/docker-compose.yml pins 2.14.10 + maps port 50000 + command flags --pn=50000 --ut --aa --alm; the healthcheck TCP-probes 50000 so `docker ps` surfaces ready state. Fixture OpcPlcFixture follows the same shape as Snap7ServerFixture + ModbusSimulatorFixture: collection-scoped, parses OPCUA_SIM_ENDPOINT (default opc.tcp://localhost:50000) into host + port, 2-second TCP probe at init, `SkipReason` records the failure for Assert.Skip. Forced IPv4 on the probe socket for the same reason those two fixtures do — .NET's dual-stack "localhost" resolves IPv6 ::1 first + hangs the full connect timeout when the target binds 0.0.0.0 (IPv4). OpcPlcProfile holds well-known node identifiers opc-plc exposes (`ns=3;s=StepUp`, `FastUInt1`, `RandomSignedInt32`, `AlternatingBoolean`) + builds `OpcUaClientDriverOptions` with SecurityPolicy.None + AutoAcceptCertificates=true since opc-plc regenerates its server cert on every container spin-up + there's no meaningful chain to validate against in CI. Three smoke tests covering what the unit suite couldn't reach: (1) Client_connects_and_reads_StepUp_node_through_real_OPC_UA_stack — full Secure Channel + Session + Read on `ns=3;s=StepUp` (counter that ticks every 1 s); (2) Client_reads_batch_of_varied_types_from_live_simulator — batch Read of UInt32 / Int32 / Boolean to prove typed Variant decoding, with an explicit ShouldBeOfType<bool> assertion on AlternatingBoolean to catch the common "variant gets stringified" regression; (3) Client_subscribe_receives_StepUp_data_changes_from_live_server — real MonitoredItem subscription on FastUInt1 (100 ms cadence) with a SemaphoreSlim gate + 3 s deadline on the first OnDataChange fire, tolerating container warm-up. Driver ran end-to-end against a live 2.14.10 container: all 3 pass; unit suite 78/78 unchanged. Container lifecycle verified (compose up → tests → compose down) clean, no leaked state. Docker/README.md documents install (Docker Desktop already on the dev box per Phase 1 decision #134), run (compose up / compose up -d / compose down), endpoint override (OPCUA_SIM_ENDPOINT), what opc-plc advertises with the current command flags, what's tunable via compose-file tweaks (--daa for username auth tests; --fn/--fr/--ft for subscription-stress nodes), known limitation that opc-plc shares the OPCFoundation stack with our driver. OpcUaClient-Test-Fixture.md updated — TL;DR flipped from "there is no integration fixture" to the new reality; "What it actually covers" gains an Integration section listing the three smoke tests. Follow-up the doc flags: add open62541/open62541 as a second image for fully-independent-stack interop coverage; once #219 (server-side IAlarmSource/IHistoryProvider integration tests) lands, re-run the client-side suite against opc-plc's --alm nodes to close the alarm gap from the client side too.
dohertj2 referenced this issue from a commit 2026-04-30 08:21:26 -04:00
Dockerize Modbus + AB CIP + S7 test fixtures for reproducibility. Every driver integration simulator now has a pinned Docker image alongside the existing native launcher — Docker is the primary path, native fallbacks kept for contributors who prefer them. Matches the already-Dockerized OpcUaClient/opc-plc pattern from #215 so every fixture in the fleet presents the same compose-up/test/compose-down loop. Reproducibility gain: what used to require a local pip/Python install (Modbus pymodbus, S7 python-snap7) or a per-OS C build from source (AB CIP ab_server from libplctag) now collapses to a Dockerfile + `docker compose up`. Modbus — new tests/ZB.MOM.WW.OtOpcUa.Driver.Modbus.IntegrationTests/Docker/ with Dockerfile (`python:3.12-slim-bookworm` + `pymodbus[simulator]==3.13.0`) + docker-compose.yml with four compose profiles (`standard` / `dl205` / `mitsubishi` / `s7_1500`) backed by the existing profile JSONs copied under `Docker/profiles/` as canonical; native fallback in `Pymodbus/` retained with the same JSON set (symlink-equivalent — manual re-sync when profiles change, noted in both READMEs). Port 5020 unchanged so `MODBUS_SIM_ENDPOINT` + `ModbusSimulatorFixture` work without code change. Dropped the `--no_http` CLI arg the old serve.ps1 + compose draft passed — pymodbus 3.13 doesn't recognize it; the simulator's http ui just binds inside the container where nothing maps it out and costs nothing. S7 — new tests/ZB.MOM.WW.OtOpcUa.Driver.S7.IntegrationTests/Docker/ with Dockerfile (`python:3.12-slim-bookworm` + `python-snap7>=2.0`) + docker-compose.yml with one `s7_1500` compose profile; copies the existing `server.py` shim + `s7_1500.json` seed profile; runs `python -u server.py ... --port 1102`. Native fallback in `PythonSnap7/` retained. Port 1102 unchanged. AB CIP — hardest because ab_server is a source-only C tool in libplctag's `src/tools/ab_server/`. New tests/ZB.MOM.WW.OtOpcUa.Driver.AbCip.IntegrationTests/Docker/ Dockerfile is multi-stage: build stage (`debian:bookworm-slim` + build-essential + cmake) clones libplctag at a pinned tag + `cmake --build build --target ab_server`; runtime stage (`debian:bookworm-slim`) copies just the binary from `/src/build/bin_dist/ab_server`. docker-compose.yml ships four compose profiles (`controllogix` / `compactlogix` / `micro800` / `guardlogix`) with per-family `ab_server` CLI args matching `AbServerProfile.cs`. AbServerFixture updated: tries TCP probe on `127.0.0.1:44818` first (Docker path) + spawns the native binary only as fallback when no listener is there. `AB_SERVER_ENDPOINT` env var supported for pointing at a real PLC. AbServerFact/Theory attributes updated to `IsServerAvailable()` which accepts any of: live listener on 44818, AB_SERVER_ENDPOINT set, or binary on PATH. Required two CLI-compat fixes to ab_server's argument expectations that the existing native profile never caught because it was never actually run at CI: `--plc` is case-sensitive (`ControlLogix` not `controllogix`), CIP tags need `[size]` bracket notation (`DINT[1]` not bare `DINT`), ControlLogix also requires `--path=1,0`. Compose files carry the corrected flags; the existing native-path `AbServerProfile.cs` was never invoked in practice so we don't rewrite it here. `Micro800` now uses the `--plc=Micro800` mode rather than falling back to ControlLogix emulation — ab_server does have the dedicated mode, the old Notes saying otherwise were wrong. Updated docs: three fixture coverage docs (Modbus-Test-Fixture.md, S7-Test-Fixture.md, AbServer-Test-Fixture.md) flip their "What the fixture is" section from native-only to Docker-primary-with-native-fallback; dev-environment.md §Resource Inventory replaces the old ambiguous "Docker Desktop + ab_server native" mix with four per-driver rows (each listing the image, compose file, compose profiles, port, credentials) + a new `Docker fixtures — quick reference` subsection giving the one-line `docker compose -f <…> --profile <…> up` for each driver + the env-var override names + the native fallback install recipes. drivers/README.md coverage map table updated — Modbus/AB CIP/S7 entries now read "Dockerized …" consistent with OpcUaClient's line. Verified end-to-end against live containers: Modbus DL205 smoke 1/1, S7 3/3, AB CIP ControlLogix 4/4 (all family theory rows). Container lifecycle clean (up/test/down, no leaked state). Every fixture keeps its skip-when-absent probe + env-var endpoint override so `dotnet test` on a fresh clone without Docker running still gets a green run.
Sign in to join this conversation.