lmxopcua/scripts/e2e/README.md

E2E CLI test scripts

End-to-end black-box tests that drive each protocol through its driver CLI and verify the resulting OPC UA address-space state through otopcua-cli. They answer one question per driver:

If I poke the real PLC through the driver, does the running OtOpcUa server see the change?

This is the acceptance gate v1 was missing — the driver-level integration tests (tests/.../IntegrationTests/) confirm the driver sees the PLC, and the OPC UA Client.CLI.Tests confirm the client sees the server — but nothing glued them end-to-end. These scripts close that loop.

Five-stage test per driver

Every per-driver script runs the same five tests. The goal is to prove both directions across the bridge plus subscription delivery — forward-only coverage would miss writable-flag drops, IWritable dispatch bugs, and broken data-change notification paths where a fresh read still returns the right value.

1. probe — driver CLI opens a session + reads a sentinel. Confirms the simulator / PLC is reachable and speaking the protocol.
2. Driver loopback — write a random value via the driver CLI, read it back via the same CLI. Confirms the driver round-trips without involving the OPC UA server. A failure here is a driver bug, not a server-bridge bug.
3. Forward bridge (driver → server → client) — write a different random value via the driver CLI, wait --ServerPollDelaySec (default 3s), read the OPC UA NodeId the server publishes that tag at via otopcua-cli read. Confirms reads propagate from PLC to OPC UA client.
4. Reverse bridge (client → server → driver) — write a fresh random value via otopcua-cli write against the same NodeId, wait --DriverPollDelaySec (default 3s), read the PLC-side via the driver CLI. Confirms writes propagate the other way — catches writable-flag drops, ACL misconfiguration, and IWritable dispatch bugs the forward test can't see.
5. Subscribe-sees-change — start otopcua-cli subscribe --duration N in the background, give it --SettleSec (default 2s) to attach, write a random value via the driver CLI, wait for the subscription window to close, and assert the captured output mentions the new value. Confirms the server's monitored-item + data-change path actually fires — not just that a fresh read returns the new value.

The OtOpcUa server must already be running with a config that (a) binds a driver instance to the same PLC the script points at, and (b) publishes the address the script writes under a NodeId the script knows. Those NodeIds live in e2e-config.json (see below). The published tag must be writable — stages 4 + 5 will fail against a read-only tag.

Status

Stages 1 + 2 (driver-side probe + loopback) are verified end-to-end against the pymodbus / ab_server / python-snap7 fixtures. Stages 3-5 (anything crossing the OtOpcUa server) are blocked on server-side driver factory wiring:

• src/ZB.MOM.WW.OtOpcUa.Server/Program.cs only registers Galaxy + FOCAS factories. DriverInstanceBootstrapper skips any DriverType without a registered factory — so Modbus / AB CIP / AB Legacy / S7 / TwinCAT rows in the Config DB are silently no-op'd even when the seed is perfect.
• No Config DB seed script exists for non-Galaxy drivers; Admin UI is currently the only path to author one.

Tracking: #209 (umbrella) → #210 (Modbus), #211 (AB CIP), #212 (S7), #213 (AB Legacy, also hardware-gated — #222). Each child issue lists the factory class to write + the seed SQL shape + the verification command.

Until those ship, stages 3-5 will fail with "read failed" (nothing published at that NodeId) and [FAIL] the suite even on a running server.

Prereqs

1. OtOpcUa server running on opc.tcp://localhost:4840 (or pass -OpcUaUrl to override). The server's Config DB must define a driver instance per protocol you want to test, bound to the matching simulator endpoint.
2. Per-driver simulators running. See docs/v2/test-data-sources.md for the simulator matrix — pymodbus / ab_server / python-snap7 / opc-plc cover Modbus / AB / S7 / OPC UA Client. FOCAS and TwinCAT have no public simulator; they are gated with env-var skip flags below.
3. PowerShell 7+. The runner uses null-coalescing + Set-StrictMode; the Windows-PowerShell-5.1 shell will not parse test-all.ps1.
4. .NET 10 SDK. Each script either runs dotnet run --project src/ZB.MOM.WW.OtOpcUa.Driver.<Name>.Cli directly, or if $env:OTOPCUA_CLI_BIN points at a publish folder, runs the pre-built otopcua-*.exe from there (faster for repeat loops).

Running

One protocol at a time

./scripts/e2e/test-modbus.ps1 `
  -ModbusHost    127.0.0.1:5502 `
  -BridgeNodeId "ns=2;s=Modbus/HR100"

Every per-protocol script takes the driver endpoint, the address to write, and the OPC UA NodeId the server exposes it at.

Full matrix

./scripts/e2e/test-all.ps1 `
  -ConfigFile ./scripts/e2e/e2e-config.json

The runner reads the sidecar JSON, invokes each driver's script with the parameters from that section, and prints a FINAL MATRIX showing PASS / FAIL / SKIP per driver. Any driver absent from the sidecar is SKIP-ed rather than failing hard — useful on dev boxes that only have one simulator up.

Sidecar format

Copy e2e-config.sample.json → e2e-config.json and fill in the NodeIds from your server's Config DB. The file is .gitignore-d (each dev's NodeIds are specific to their local seed). Omit a driver section to skip it.

Expected pass/fail matrix (default config)

Driver	Gate	Default state on a clean dev box
Modbus	—	PASS (pymodbus fixture)
AB CIP	—	PASS (ab_server fixture)
AB Legacy	—	PASS (ab_server SLC500/MicroLogix/PLC-5 profiles; /1,0 cip-path required for the Docker fixture)
Galaxy	—	PASS (requires OtOpcUaGalaxyHost + a live Galaxy; 7 stages including alarms + history)
S7	—	PASS (python-snap7 fixture)
FOCAS	FOCAS_TRUST_WIRE=1	SKIP (no public simulator — task #222 lab rig)
TwinCAT	TWINCAT_TRUST_WIRE=1	SKIP (needs XAR or standalone Router — task #221)
Phase 7	—	PASS if the Modbus instance seeds a VT_DoubledHR100 virtual tag + AlarmHigh scripted alarm

Set the *_TRUST_WIRE env vars to 1 when you've pointed the script at real hardware or a properly-configured simulator.

Output

Each step prints one of:

• [PASS] ... — step succeeded
• [FAIL] ... — step failed, stdout of the failing CLI is echoed below for diagnosis
• [SKIP] ... — step short-circuited (env-var gate)
• [INFO] ... — progress note (e.g., "waiting 3s for server-side poll")

The runner ends with a coloured summary per driver:

==================== FINAL MATRIX ====================
  modbus     PASS
  abcip      PASS
  ablegacy   SKIP (no config entry)
  s7         PASS
  focas      SKIP (no config entry)
  twincat    SKIP (no config entry)
  phase7     PASS
All present suites passed.

Non-zero exit if any present suite failed. SKIPs do not fail the run.

Why this is separate from dotnet test

dotnet test covers driver-layer + server-layer correctness in isolation — mocks + in-process test hosts. These e2e scripts cover the integration seam that unit tests can't cover by design: a live OPC UA server process, a live simulator, and the wire between them. Run them before a v2 release-readiness sign-off, after a driver-layer change that could plausibly affect the NodeManager contract, and before any "it works on my box" handoff to QA.