lmxopcua/scripts/e2e/README.md

# E2E CLI test scripts

End-to-end black-box tests that drive each protocol through its driver CLI
and verify the resulting OPC UA address-space state through
`otopcua-cli`. They answer one question per driver:

> **If I poke the real PLC through the driver, does the running OtOpcUa
> server see the change?**

This is the acceptance gate v1 was missing — the driver-level integration
tests (`tests/.../IntegrationTests/`) confirm the driver sees the PLC, and
the OPC UA `Client.CLI.Tests` confirm the client sees the server — but
nothing glued them end-to-end. These scripts close that loop.

## Three-stage test per driver

Every per-driver script runs the same three tests:

1. **`probe`** — driver CLI opens a session + reads a sentinel. Confirms
   the simulator / PLC is reachable and speaking the protocol.
2. **Driver loopback** — write a random value via the driver CLI, read it
   back via the same CLI. Confirms the driver round-trips without
   involving the OPC UA server. A failure here is a driver bug, not a
   server-bridge bug.
3. **Server bridge** — write a different random value via the driver
   CLI, wait `--ServerPollDelaySec` (default 3s), read the OPC UA NodeId
   the server publishes that tag at via `otopcua-cli read`. Confirms the
   full path: driver CLI → PLC → OtOpcUa server → OPC UA client.

The OtOpcUa server must already be running with a config that
(a) binds a driver instance to the same PLC the script points at, and
(b) publishes the address the script writes under a NodeId the script
knows. Those NodeIds live in `e2e-config.json` (see below).

## Prereqs

1. **OtOpcUa server** running on `opc.tcp://localhost:4840` (or pass
   `-OpcUaUrl` to override). The server's Config DB must define a
   driver instance per protocol you want to test, bound to the matching
   simulator endpoint.
2. **Per-driver simulators** running. See `docs/v2/test-data-sources.md`
   for the simulator matrix — pymodbus / ab_server / python-snap7 /
   opc-plc cover Modbus / AB / S7 / OPC UA Client. FOCAS and TwinCAT
   have no public simulator; they are gated with env-var skip flags
   below.
3. **PowerShell 7+**. The runner uses null-coalescing + `Set-StrictMode`;
   the Windows-PowerShell-5.1 shell will not parse `test-all.ps1`.
4. **.NET 10 SDK**. Each script either runs `dotnet run --project
   src/ZB.MOM.WW.OtOpcUa.Driver.<Name>.Cli` directly, or if
   `$env:OTOPCUA_CLI_BIN` points at a publish folder, runs the pre-built
   `otopcua-*.exe` from there (faster for repeat loops).

## Running

### One protocol at a time

```powershell
./scripts/e2e/test-modbus.ps1 `
  -ModbusHost    127.0.0.1:5502 `
  -BridgeNodeId "ns=2;s=Modbus/HR100"
```

Every per-protocol script takes the driver endpoint, the address to
write, and the OPC UA NodeId the server exposes it at.

### Full matrix

```powershell
./scripts/e2e/test-all.ps1 `
  -ConfigFile ./scripts/e2e/e2e-config.json
```

The runner reads the sidecar JSON, invokes each driver's script with the
parameters from that section, and prints a `FINAL MATRIX` showing
PASS / FAIL / SKIP per driver. Any driver absent from the sidecar is
SKIP-ed rather than failing hard — useful on dev boxes that only have
one simulator up.

### Sidecar format

Copy `e2e-config.sample.json` → `e2e-config.json` and fill in the
NodeIds from **your** server's Config DB. The file is `.gitignore`-d
(each dev's NodeIds are specific to their local seed). Omit a driver
section to skip it.

## Expected pass/fail matrix (default config)

| Driver | Gate | Default state on a clean dev box |
|---|---|---|
| Modbus | — | **PASS** (pymodbus fixture) |
| AB CIP | — | **PASS** (ab_server fixture) |
| AB Legacy | `AB_LEGACY_TRUST_WIRE=1` | **SKIP** (ab_server PCCC path upstream-broken — task #222) |
| S7 | — | **PASS** (python-snap7 fixture) |
| FOCAS | `FOCAS_TRUST_WIRE=1` | **SKIP** (no public simulator — task #222 lab rig) |
| TwinCAT | `TWINCAT_TRUST_WIRE=1` | **SKIP** (needs XAR or standalone Router — task #221) |
| Phase 7 | — | **PASS** if the Modbus instance seeds a `VT_DoubledHR100` virtual tag + `AlarmHigh` scripted alarm |

Set the `*_TRUST_WIRE` env vars to `1` when you've pointed the script at
real hardware or a properly-configured simulator.

## Output

Each step prints one of:

- `[PASS] ...` — step succeeded
- `[FAIL] ...` — step failed, stdout of the failing CLI is echoed below
  for diagnosis
- `[SKIP] ...` — step short-circuited (env-var gate)
- `[INFO] ...` — progress note (e.g., "waiting 3s for server-side poll")

The runner ends with a coloured summary per driver:

```
==================== FINAL MATRIX ====================
  modbus     PASS
  abcip      PASS
  ablegacy   SKIP (no config entry)
  s7         PASS
  focas      SKIP (no config entry)
  twincat    SKIP (no config entry)
  phase7     PASS
All present suites passed.
```

Non-zero exit if any present suite failed. SKIPs do not fail the run.

## Why this is separate from `dotnet test`

`dotnet test` covers driver-layer + server-layer correctness in
isolation — mocks + in-process test hosts. These e2e scripts cover the
integration seam that unit tests *can't* cover by design: a live OPC UA
server process, a live simulator, and the wire between them. Run them
before a v2 release-readiness sign-off, after a driver-layer change
that could plausibly affect the NodeManager contract, and before any
"it works on my box" handoff to QA.