lmxopcua/scripts/e2e/README.md

# E2E CLI test scripts

End-to-end black-box tests that drive each protocol through its driver CLI
and verify the resulting OPC UA address-space state through
`otopcua-cli`. They answer one question per driver:

> **If I poke the real PLC through the driver, does the running OtOpcUa
> server see the change?**

This is the acceptance gate v1 was missing — the driver-level integration
tests (`tests/.../IntegrationTests/`) confirm the driver sees the PLC, and
the OPC UA `Client.CLI.Tests` confirm the client sees the server — but
nothing glued them end-to-end. These scripts close that loop.

## Five-stage test per driver

Every per-driver script runs the same five tests. The goal is to prove
**both directions** across the bridge plus subscription delivery —
forward-only coverage would miss writable-flag drops, `IWritable`
dispatch bugs, and broken data-change notification paths where a fresh
read still returns the right value.

1. **`probe`** — driver CLI opens a session + reads a sentinel. Confirms
   the simulator / PLC is reachable and speaking the protocol.
2. **Driver loopback** — write a random value via the driver CLI, read
   it back via the same CLI. Confirms the driver round-trips without
   involving the OPC UA server. A failure here is a driver bug, not a
   server-bridge bug.
3. **Forward bridge (driver → server → client)** — write a different
   random value via the driver CLI, wait `--ServerPollDelaySec` (default
   3s), read the OPC UA NodeId the server publishes that tag at via
   `otopcua-cli read`. Confirms reads propagate from PLC to OPC UA
   client.
4. **Reverse bridge (client → server → driver)** — write a fresh random
   value via `otopcua-cli write` against the same NodeId, wait
   `--DriverPollDelaySec` (default 3s), read the PLC-side via the
   driver CLI. Confirms writes propagate the other way — catches
   writable-flag drops, ACL misconfiguration, and `IWritable` dispatch
   bugs the forward test can't see.
5. **Subscribe-sees-change** — start `otopcua-cli subscribe --duration N`
   in the background, give it `--SettleSec` (default 2s) to attach,
   write a random value via the driver CLI, wait for the subscription
   window to close, and assert the captured output mentions the new
   value. Confirms the server's monitored-item + data-change path
   actually fires — not just that a fresh read returns the new value.

The OtOpcUa server must already be running with a config that
(a) binds a driver instance to the same PLC the script points at, and
(b) publishes the address the script writes under a NodeId the script
knows. Those NodeIds live in `e2e-config.json` (see below). The
published tag must be **writable** — stages 4 + 5 will fail against a
read-only tag.

## Status

All seven driver factories are registered in
`src/ZB.MOM.WW.OtOpcUa.Server/Program.cs` — Galaxy, FOCAS, Modbus,
AB CIP, AB Legacy, S7, TwinCAT. `DriverInstanceBootstrapper` can
materialise any `DriverType` row from the central Config DB into a
live driver. The factory-wiring block that originally gated stages
3-5 is closed.

Live-boot verification:

- **Galaxy** — 7/7 stages (read / write / subscribe / alarms / history)
  against a real Galaxy via the in-process `GalaxyDriver` →
  `mxaccessgw` (gRPC). PR 7.2 retired the legacy `OtOpcUaGalaxyHost`
  out-of-process driver path.
- **AB CIP, S7** — 5/5 stages each under task #220 against the
  `ab_server` + `python-snap7` fixtures.
- **AB Legacy** — 5/5 stages under task #222 against `ab_server` SLC500
  / MicroLogix / PLC-5 profiles (requires the `cip-path /1,0` workaround
  for the Docker fixture).
- **Modbus** — 5/5 stages against the `pymodbus` + dl205 profile,
  including HR[200] scratch register + per-protocol bidirectional +
  subscribe-sees-change stages.
- **TwinCAT** — factory registered; driver features validated against the
  TCBSD VM virtual-PLC fixture (FreeBSD + TwinCAT/BSD runtime on ESXi —
  bypasses the Hyper-V/RTIME conflict that blocks XAR on the dev box).
  `TWINCAT_TRUST_WIRE=1` is still required to run the script —
  false-pass-prevention belt, not an "unverified" flag.
- **FOCAS** — factory registered; gated by `FOCAS_TRUST_WIRE=1` pending
  the lab-rig CNC (task #222 follow-up).
- **OpcUaClient (gateway)** — eight-stage script (`test-opcuaclient.ps1`)
  covers probe / remote read / forward bridge / subscribe / reverse
  bridge / browse mirror / alarm / history against the opc-plc Docker
  fixture at `opc.tcp://localhost:50000`. Reverse-bridge / alarm /
  history stages are opt-in per the parameter docs (opc-plc's default
  image has no writable nodes and does not historize).

Remaining work is **per-protocol seed authoring**: each dev fills in
the NodeIds their server publishes under `e2e-config.json` (sidecar
is `.gitignore`-d; see `e2e-config.sample.json` for the shape). Admin
UI remains the supported path for authoring the matching driver
instance rows in the Config DB.

Tracking: umbrella #209 is closed; remaining TwinCAT / FOCAS work
tracks under their hardware-fixture tasks (#221 / #222).

## Prereqs

1. **OtOpcUa server** running on `opc.tcp://localhost:4840` (or pass
   `-OpcUaUrl` to override). The server's Config DB must define a
   driver instance per protocol you want to test, bound to the matching
   simulator endpoint.
2. **Per-driver simulators** running. See `docs/v2/test-data-sources.md`
   for the simulator matrix — pymodbus / ab_server / python-snap7 /
   opc-plc cover Modbus / AB / S7 / OPC UA Client. FOCAS and TwinCAT
   have no public simulator; they are gated with env-var skip flags
   below. For OpcUaClient, `docker compose -f
   tests/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.IntegrationTests/Docker/
   docker-compose.yml up -d` brings up `opc-plc` on port 50000.
3. **PowerShell 7+**. The runner uses null-coalescing + `Set-StrictMode`;
   the Windows-PowerShell-5.1 shell will not parse `test-all.ps1`.
4. **.NET 10 SDK**. Each script either runs `dotnet run --project
   src/ZB.MOM.WW.OtOpcUa.Driver.<Name>.Cli` directly, or if
   `$env:OTOPCUA_CLI_BIN` points at a publish folder, runs the pre-built
   `otopcua-*.exe` from there (faster for repeat loops).

## Running

### One protocol at a time

```powershell
./scripts/e2e/test-modbus.ps1 `
  -ModbusHost    127.0.0.1:5502 `
  -BridgeNodeId "ns=2;s=Modbus/HR100"
```

Every per-protocol script takes the driver endpoint, the address to
write, and the OPC UA NodeId the server exposes it at.

### Full matrix

```powershell
./scripts/e2e/test-all.ps1 `
  -ConfigFile ./scripts/e2e/e2e-config.json
```

The runner reads the sidecar JSON, invokes each driver's script with the
parameters from that section, and prints a `FINAL MATRIX` showing
PASS / FAIL / SKIP per driver. Any driver absent from the sidecar is
SKIP-ed rather than failing hard — useful on dev boxes that only have
one simulator up.

### Sidecar format

Copy `e2e-config.sample.json` → `e2e-config.json` and fill in the
NodeIds from **your** server's Config DB. The file is `.gitignore`-d
(each dev's NodeIds are specific to their local seed). Omit a driver
section to skip it.

## Expected pass/fail matrix (default config)

| Driver | Gate | Default state on a clean dev box |
|---|---|---|
| Modbus | — | **PASS** (pymodbus fixture) |
| AB CIP | — | **PASS** (ab_server fixture) |
| AB Legacy | — | **PASS** (ab_server SLC500/MicroLogix/PLC-5 profiles; `/1,0` cip-path required for the Docker fixture) |
| Galaxy | — | **PASS** (requires mxaccessgw running + a live Galaxy; 7 stages including alarms + history; PR 7.2 retired the legacy OtOpcUaGalaxyHost path) |
| S7 | — | **PASS** (python-snap7 fixture) |
| FOCAS | `FOCAS_TRUST_WIRE=1` | **SKIP** (no public simulator — task #222 lab rig) |
| TwinCAT | `TWINCAT_TRUST_WIRE=1` | **SKIP** by default; features **validated** against the TCBSD VM fixture — set the env var to run |
| OpcUaClient | — | **PASS** stages 1-4 + browse (opc-plc Docker fixture); stages 5/7/8 are opt-in (require writable / alarm / historizing upstream) |
| Phase 7 | — | **PASS** if the Modbus instance seeds a `VT_DoubledHR100` virtual tag + `AlarmHigh` scripted alarm |

Set the `*_TRUST_WIRE` env vars to `1` when you've pointed the script at
real hardware or a properly-configured simulator.

## Output

Each step prints one of:

- `[PASS] ...` — step succeeded
- `[FAIL] ...` — step failed, stdout of the failing CLI is echoed below
  for diagnosis
- `[SKIP] ...` — step short-circuited (env-var gate)
- `[INFO] ...` — progress note (e.g., "waiting 3s for server-side poll")

The runner ends with a coloured summary per driver:

```
==================== FINAL MATRIX ====================
  modbus     PASS
  abcip      PASS
  ablegacy   SKIP (no config entry)
  s7         PASS
  focas      SKIP (no config entry)
  twincat    SKIP (no config entry)
  phase7     PASS
All present suites passed.
```

Non-zero exit if any present suite failed. SKIPs do not fail the run.

## Why this is separate from `dotnet test`

`dotnet test` covers driver-layer + server-layer correctness in
isolation — mocks + in-process test hosts. These e2e scripts cover the
integration seam that unit tests *can't* cover by design: a live OPC UA
server process, a live simulator, and the wire between them. Run them
before a v2 release-readiness sign-off, after a driver-layer change
that could plausibly affect the NodeManager contract, and before any
"it works on my box" handoff to QA.