a9b585ac5b
The original three-stage design (probe / driver-loopback / forward- bridge) only proved driver-write → server-read. It missed: - OPC UA write → server → driver → PLC (the reverse direction) - server-side data-change notifications actually firing (a stale subscription can still let a read-after-the-fact return the new value and look fine) Extend _common.ps1 with two helpers: - Test-OpcUaWriteBridge: otopcua-cli write the NodeId -> wait 3s -> driver CLI read the PLC side, assert equality. - Test-SubscribeSeesChange: Start-Process otopcua-cli subscribe in the background with --duration N, settle 2s, driver-side write, wait for the subscription window to close, assert captured stdout contains the new value. Wire both into test-modbus / test-abcip / test-ablegacy / test-s7 / test-focas / test-twincat after the existing forward-bridge stage. Update README to describe the five-stage design + note that the published NodeId must be writable for stages 4 + 5. Also prepend UTF-8 BOM to every script in scripts/e2e so Windows PowerShell 5.1 parsers agree on em-dash byte sequences the way PowerShell 7 already does. The scripts still #Requires -Version 7.0 — the BOM is purely defensive for IDE / CI step parsers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
155 lines
6.5 KiB
Markdown
155 lines
6.5 KiB
Markdown
# E2E CLI test scripts
|
|
|
|
End-to-end black-box tests that drive each protocol through its driver CLI
|
|
and verify the resulting OPC UA address-space state through
|
|
`otopcua-cli`. They answer one question per driver:
|
|
|
|
> **If I poke the real PLC through the driver, does the running OtOpcUa
|
|
> server see the change?**
|
|
|
|
This is the acceptance gate v1 was missing — the driver-level integration
|
|
tests (`tests/.../IntegrationTests/`) confirm the driver sees the PLC, and
|
|
the OPC UA `Client.CLI.Tests` confirm the client sees the server — but
|
|
nothing glued them end-to-end. These scripts close that loop.
|
|
|
|
## Five-stage test per driver
|
|
|
|
Every per-driver script runs the same five tests. The goal is to prove
|
|
**both directions** across the bridge plus subscription delivery —
|
|
forward-only coverage would miss writable-flag drops, `IWritable`
|
|
dispatch bugs, and broken data-change notification paths where a fresh
|
|
read still returns the right value.
|
|
|
|
1. **`probe`** — driver CLI opens a session + reads a sentinel. Confirms
|
|
the simulator / PLC is reachable and speaking the protocol.
|
|
2. **Driver loopback** — write a random value via the driver CLI, read
|
|
it back via the same CLI. Confirms the driver round-trips without
|
|
involving the OPC UA server. A failure here is a driver bug, not a
|
|
server-bridge bug.
|
|
3. **Forward bridge (driver → server → client)** — write a different
|
|
random value via the driver CLI, wait `--ServerPollDelaySec` (default
|
|
3s), read the OPC UA NodeId the server publishes that tag at via
|
|
`otopcua-cli read`. Confirms reads propagate from PLC to OPC UA
|
|
client.
|
|
4. **Reverse bridge (client → server → driver)** — write a fresh random
|
|
value via `otopcua-cli write` against the same NodeId, wait
|
|
`--DriverPollDelaySec` (default 3s), read the PLC-side via the
|
|
driver CLI. Confirms writes propagate the other way — catches
|
|
writable-flag drops, ACL misconfiguration, and `IWritable` dispatch
|
|
bugs the forward test can't see.
|
|
5. **Subscribe-sees-change** — start `otopcua-cli subscribe --duration N`
|
|
in the background, give it `--SettleSec` (default 2s) to attach,
|
|
write a random value via the driver CLI, wait for the subscription
|
|
window to close, and assert the captured output mentions the new
|
|
value. Confirms the server's monitored-item + data-change path
|
|
actually fires — not just that a fresh read returns the new value.
|
|
|
|
The OtOpcUa server must already be running with a config that
|
|
(a) binds a driver instance to the same PLC the script points at, and
|
|
(b) publishes the address the script writes under a NodeId the script
|
|
knows. Those NodeIds live in `e2e-config.json` (see below). The
|
|
published tag must be **writable** — stages 4 + 5 will fail against a
|
|
read-only tag.
|
|
|
|
## Prereqs
|
|
|
|
1. **OtOpcUa server** running on `opc.tcp://localhost:4840` (or pass
|
|
`-OpcUaUrl` to override). The server's Config DB must define a
|
|
driver instance per protocol you want to test, bound to the matching
|
|
simulator endpoint.
|
|
2. **Per-driver simulators** running. See `docs/v2/test-data-sources.md`
|
|
for the simulator matrix — pymodbus / ab_server / python-snap7 /
|
|
opc-plc cover Modbus / AB / S7 / OPC UA Client. FOCAS and TwinCAT
|
|
have no public simulator; they are gated with env-var skip flags
|
|
below.
|
|
3. **PowerShell 7+**. The runner uses null-coalescing + `Set-StrictMode`;
|
|
the Windows-PowerShell-5.1 shell will not parse `test-all.ps1`.
|
|
4. **.NET 10 SDK**. Each script either runs `dotnet run --project
|
|
src/ZB.MOM.WW.OtOpcUa.Driver.<Name>.Cli` directly, or if
|
|
`$env:OTOPCUA_CLI_BIN` points at a publish folder, runs the pre-built
|
|
`otopcua-*.exe` from there (faster for repeat loops).
|
|
|
|
## Running
|
|
|
|
### One protocol at a time
|
|
|
|
```powershell
|
|
./scripts/e2e/test-modbus.ps1 `
|
|
-ModbusHost 127.0.0.1:5502 `
|
|
-BridgeNodeId "ns=2;s=Modbus/HR100"
|
|
```
|
|
|
|
Every per-protocol script takes the driver endpoint, the address to
|
|
write, and the OPC UA NodeId the server exposes it at.
|
|
|
|
### Full matrix
|
|
|
|
```powershell
|
|
./scripts/e2e/test-all.ps1 `
|
|
-ConfigFile ./scripts/e2e/e2e-config.json
|
|
```
|
|
|
|
The runner reads the sidecar JSON, invokes each driver's script with the
|
|
parameters from that section, and prints a `FINAL MATRIX` showing
|
|
PASS / FAIL / SKIP per driver. Any driver absent from the sidecar is
|
|
SKIP-ed rather than failing hard — useful on dev boxes that only have
|
|
one simulator up.
|
|
|
|
### Sidecar format
|
|
|
|
Copy `e2e-config.sample.json` → `e2e-config.json` and fill in the
|
|
NodeIds from **your** server's Config DB. The file is `.gitignore`-d
|
|
(each dev's NodeIds are specific to their local seed). Omit a driver
|
|
section to skip it.
|
|
|
|
## Expected pass/fail matrix (default config)
|
|
|
|
| Driver | Gate | Default state on a clean dev box |
|
|
|---|---|---|
|
|
| Modbus | — | **PASS** (pymodbus fixture) |
|
|
| AB CIP | — | **PASS** (ab_server fixture) |
|
|
| AB Legacy | `AB_LEGACY_TRUST_WIRE=1` | **SKIP** (ab_server PCCC path upstream-broken — task #222) |
|
|
| S7 | — | **PASS** (python-snap7 fixture) |
|
|
| FOCAS | `FOCAS_TRUST_WIRE=1` | **SKIP** (no public simulator — task #222 lab rig) |
|
|
| TwinCAT | `TWINCAT_TRUST_WIRE=1` | **SKIP** (needs XAR or standalone Router — task #221) |
|
|
| Phase 7 | — | **PASS** if the Modbus instance seeds a `VT_DoubledHR100` virtual tag + `AlarmHigh` scripted alarm |
|
|
|
|
Set the `*_TRUST_WIRE` env vars to `1` when you've pointed the script at
|
|
real hardware or a properly-configured simulator.
|
|
|
|
## Output
|
|
|
|
Each step prints one of:
|
|
|
|
- `[PASS] ...` — step succeeded
|
|
- `[FAIL] ...` — step failed, stdout of the failing CLI is echoed below
|
|
for diagnosis
|
|
- `[SKIP] ...` — step short-circuited (env-var gate)
|
|
- `[INFO] ...` — progress note (e.g., "waiting 3s for server-side poll")
|
|
|
|
The runner ends with a coloured summary per driver:
|
|
|
|
```
|
|
==================== FINAL MATRIX ====================
|
|
modbus PASS
|
|
abcip PASS
|
|
ablegacy SKIP (no config entry)
|
|
s7 PASS
|
|
focas SKIP (no config entry)
|
|
twincat SKIP (no config entry)
|
|
phase7 PASS
|
|
All present suites passed.
|
|
```
|
|
|
|
Non-zero exit if any present suite failed. SKIPs do not fail the run.
|
|
|
|
## Why this is separate from `dotnet test`
|
|
|
|
`dotnet test` covers driver-layer + server-layer correctness in
|
|
isolation — mocks + in-process test hosts. These e2e scripts cover the
|
|
integration seam that unit tests *can't* cover by design: a live OPC UA
|
|
server process, a live simulator, and the wire between them. Run them
|
|
before a v2 release-readiness sign-off, after a driver-layer change
|
|
that could plausibly affect the NodeManager contract, and before any
|
|
"it works on my box" handoff to QA.
|