lmxopcua/scripts/e2e/README.md

E2E CLI test scripts

End-to-end black-box tests that drive each protocol through its driver CLI and verify the resulting OPC UA address-space state through otopcua-cli. They answer one question per driver:

If I poke the real PLC through the driver, does the running OtOpcUa server see the change?

This is the acceptance gate v1 was missing — the driver-level integration tests (tests/.../IntegrationTests/) confirm the driver sees the PLC, and the OPC UA Client.CLI.Tests confirm the client sees the server — but nothing glued them end-to-end. These scripts close that loop.

Five-stage test per driver

Every per-driver script runs the same five tests. The goal is to prove both directions across the bridge plus subscription delivery — forward-only coverage would miss writable-flag drops, IWritable dispatch bugs, and broken data-change notification paths where a fresh read still returns the right value.

1. probe — driver CLI opens a session + reads a sentinel. Confirms the simulator / PLC is reachable and speaking the protocol.
2. Driver loopback — write a random value via the driver CLI, read it back via the same CLI. Confirms the driver round-trips without involving the OPC UA server. A failure here is a driver bug, not a server-bridge bug.
3. Forward bridge (driver → server → client) — write a different random value via the driver CLI, wait --ServerPollDelaySec (default 3s), read the OPC UA NodeId the server publishes that tag at via otopcua-cli read. Confirms reads propagate from PLC to OPC UA client.
4. Reverse bridge (client → server → driver) — write a fresh random value via otopcua-cli write against the same NodeId, wait --DriverPollDelaySec (default 3s), read the PLC-side via the driver CLI. Confirms writes propagate the other way — catches writable-flag drops, ACL misconfiguration, and IWritable dispatch bugs the forward test can't see.
5. Subscribe-sees-change — start otopcua-cli subscribe --duration N in the background, give it --SettleSec (default 2s) to attach, write a random value via the driver CLI, wait for the subscription window to close, and assert the captured output mentions the new value. Confirms the server's monitored-item + data-change path actually fires — not just that a fresh read returns the new value.

The OtOpcUa server must already be running with a config that (a) binds a driver instance to the same PLC the script points at, and (b) publishes the address the script writes under a NodeId the script knows. Those NodeIds live in e2e-config.json (see below). The published tag must be writable — stages 4 + 5 will fail against a read-only tag.

Status

All seven driver factories are registered in src/ZB.MOM.WW.OtOpcUa.Server/Program.cs — Galaxy, FOCAS, Modbus, AB CIP, AB Legacy, S7, TwinCAT. DriverInstanceBootstrapper can materialise any DriverType row from the central Config DB into a live driver. The factory-wiring block that originally gated stages 3-5 is closed.

Live-boot verification:

• Galaxy — 7/7 stages (read / write / subscribe / alarms / history) against a real Galaxy via the in-process GalaxyDriver → mxaccessgw (gRPC). PR 7.2 retired the legacy OtOpcUaGalaxyHost out-of-process driver path.
• AB CIP, S7 — 5/5 stages each under task #220 against the ab_server + python-snap7 fixtures.
• AB Legacy — 5/5 stages under task #222 against ab_server SLC500 / MicroLogix / PLC-5 profiles (requires the cip-path /1,0 workaround for the Docker fixture).
• Modbus — 5/5 stages against the pymodbus + dl205 profile, including HR[200] scratch register + per-protocol bidirectional + subscribe-sees-change stages.
• TwinCAT — factory registered; driver features validated against the TCBSD VM virtual-PLC fixture (FreeBSD + TwinCAT/BSD runtime on ESXi — bypasses the Hyper-V/RTIME conflict that blocks XAR on the dev box). TWINCAT_TRUST_WIRE=1 is still required to run the script — false-pass-prevention belt, not an "unverified" flag.
• FOCAS — factory registered; gated by FOCAS_TRUST_WIRE=1 pending the lab-rig CNC (task #222 follow-up).
• OpcUaClient (gateway) — eight-stage script (test-opcuaclient.ps1) covers probe / remote read / forward bridge / subscribe / reverse bridge / browse mirror / alarm / history against the opc-plc Docker fixture at opc.tcp://localhost:50000. Reverse-bridge / alarm / history stages are opt-in per the parameter docs (opc-plc's default image has no writable nodes and does not historize).

Remaining work is per-protocol seed authoring: each dev fills in the NodeIds their server publishes under e2e-config.json (sidecar is .gitignore-d; see e2e-config.sample.json for the shape). Admin UI remains the supported path for authoring the matching driver instance rows in the Config DB.

Tracking: umbrella #209 is closed; remaining TwinCAT / FOCAS work tracks under their hardware-fixture tasks (#221 / #222).

Prereqs

1. OtOpcUa server running on opc.tcp://localhost:4840 (or pass -OpcUaUrl to override). The server's Config DB must define a driver instance per protocol you want to test, bound to the matching simulator endpoint.
2. Per-driver simulators running. See docs/v2/test-data-sources.md for the simulator matrix — pymodbus / ab_server / python-snap7 / opc-plc cover Modbus / AB / S7 / OPC UA Client. FOCAS and TwinCAT have no public simulator; they are gated with env-var skip flags below. For OpcUaClient, docker compose -f tests/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.IntegrationTests/Docker/ docker-compose.yml up -d brings up opc-plc on port 50000.
3. PowerShell 7+. The runner uses null-coalescing + Set-StrictMode; the Windows-PowerShell-5.1 shell will not parse test-all.ps1.
4. .NET 10 SDK. Each script either runs dotnet run --project src/ZB.MOM.WW.OtOpcUa.Driver.<Name>.Cli directly, or if $env:OTOPCUA_CLI_BIN points at a publish folder, runs the pre-built otopcua-*.exe from there (faster for repeat loops).

Running

One protocol at a time

./scripts/e2e/test-modbus.ps1 `
  -ModbusHost    127.0.0.1:5502 `
  -BridgeNodeId "ns=2;s=Modbus/HR100"

Every per-protocol script takes the driver endpoint, the address to write, and the OPC UA NodeId the server exposes it at.

Full matrix

./scripts/e2e/test-all.ps1 `
  -ConfigFile ./scripts/e2e/e2e-config.json

The runner reads the sidecar JSON, invokes each driver's script with the parameters from that section, and prints a FINAL MATRIX showing PASS / FAIL / SKIP per driver. Any driver absent from the sidecar is SKIP-ed rather than failing hard — useful on dev boxes that only have one simulator up.

Sidecar format

Copy e2e-config.sample.json → e2e-config.json and fill in the NodeIds from your server's Config DB. The file is .gitignore-d (each dev's NodeIds are specific to their local seed). Omit a driver section to skip it.

Expected pass/fail matrix (default config)

Driver	Gate	Default state on a clean dev box
Modbus	—	PASS (pymodbus fixture)
AB CIP	—	PASS (ab_server fixture)
AB Legacy	—	PASS (ab_server SLC500/MicroLogix/PLC-5 profiles; /1,0 cip-path required for the Docker fixture)
Galaxy	—	PASS (requires mxaccessgw running + a live Galaxy; 7 stages including alarms + history; PR 7.2 retired the legacy OtOpcUaGalaxyHost path)
S7	—	PASS (python-snap7 fixture)
FOCAS	FOCAS_TRUST_WIRE=1	SKIP (no public simulator — task #222 lab rig)
TwinCAT	TWINCAT_TRUST_WIRE=1	SKIP by default; features validated against the TCBSD VM fixture — set the env var to run
OpcUaClient	—	PASS stages 1-4 + browse (opc-plc Docker fixture); stages 5/7/8 are opt-in (require writable / alarm / historizing upstream)
Phase 7	—	PASS if the Modbus instance seeds a VT_DoubledHR100 virtual tag + AlarmHigh scripted alarm

Set the *_TRUST_WIRE env vars to 1 when you've pointed the script at real hardware or a properly-configured simulator.

Output

Each step prints one of:

• [PASS] ... — step succeeded
• [FAIL] ... — step failed, stdout of the failing CLI is echoed below for diagnosis
• [SKIP] ... — step short-circuited (env-var gate)
• [INFO] ... — progress note (e.g., "waiting 3s for server-side poll")

The runner ends with a coloured summary per driver:

==================== FINAL MATRIX ====================
  modbus     PASS
  abcip      PASS
  ablegacy   SKIP (no config entry)
  s7         PASS
  focas      SKIP (no config entry)
  twincat    SKIP (no config entry)
  phase7     PASS
All present suites passed.

Non-zero exit if any present suite failed. SKIPs do not fail the run.

Why this is separate from dotnet test

dotnet test covers driver-layer + server-layer correctness in isolation — mocks + in-process test hosts. These e2e scripts cover the integration seam that unit tests can't cover by design: a live OPC UA server process, a live simulator, and the wire between them. Run them before a v2 release-readiness sign-off, after a driver-layer change that could plausibly affect the NodeManager contract, and before any "it works on my box" handoff to QA.