95c7e0b490
Replaced the "ab_server PCCC upstream-broken" skip gate with the actual
root cause: libplctag's ab_server rejects empty CIP routing paths at the
unconnected-send layer before the PCCC dispatcher runs. Real SLC/
MicroLogix/PLC-5 hardware accepts empty paths (no backplane); ab_server
does not. With `/1,0` in place, N (Int16), F (Float32), and L (Int32)
file reads + writes round-trip cleanly across all three compose profiles.
## Fixture changes
- `AbLegacyServerFixture.cs`:
- Drop `AB_LEGACY_TRUST_WIRE` env var + the reachable-but-untrusted
skip branch. Fixture now only skips on TCP unreachability.
- Add `AB_LEGACY_CIP_PATH` env var (default `1,0`) + expose `CipPath`
property. Set `AB_LEGACY_CIP_PATH=` (empty) against real hardware.
- Shorter skip messages on the `[AbLegacyFact]` / `[AbLegacyTheory]`
attributes — one reason: endpoint not reachable.
- `AbLegacyReadSmokeTests.cs`:
- Device URI built from `sim.CipPath` instead of hardcoded empty path.
- New `AB_LEGACY_COMPOSE_PROFILE` env var filters the parametric
theory to the running container's family. Only one container binds
`:44818` at a time, so cross-family params would otherwise fail.
- `Slc500_write_then_read_round_trip` skips cleanly when the running
profile isn't `slc500`.
## E2E + seed + docs
- `scripts/e2e/test-ablegacy.ps1` — drop the `AB_LEGACY_TRUST_WIRE`
skip gate; synopsis calls out the `/1,0` vs empty cip-path split
between the Docker fixture and real hardware.
- `scripts/e2e/e2e-config.sample.json` — sample gateway flipped from
the hardware placeholder (`192.168.1.10`) to the Docker fixture
(`127.0.0.1/1,0`); comment rewritten.
- `scripts/e2e/README.md` — AB Legacy expected-matrix row goes from
SKIP to PASS.
- `scripts/smoke/seed-ablegacy-smoke.sql` — default HostAddress points
at the Docker fixture + header / footer text reflect the new state.
- `tests/.../Docker/README.md` — "Known limitations" section rewritten
to describe the cip-path gate (not a dispatcher gap); env-var table
picks up `AB_LEGACY_CIP_PATH` + `AB_LEGACY_COMPOSE_PROFILE`.
- `docs/drivers/AbLegacy-Test-Fixture.md` + `docs/drivers/README.md`
+ `docs/DriverClis.md` — flip status from blocked to functional;
residual bit-file-write gap (B3:0/5 → 0x803D0000) documented.
## Residual gap
Bit-file writes (`B3:0/5` style) surface `0x803D0000` against
`ab_server --plc=SLC500`; bit reads work. Non-blocking for smoke
coverage — N/F/L round-trip is enough. Real hardware / RSEmulate 500
for bit-write fidelity. Documented in `Docker/README.md` §"Known
limitations" + the `AbLegacy-Test-Fixture.md` follow-ups list.
## Verified
- Full-solution build: 0 errors, 334 pre-existing warnings.
- Integration suite passes per-profile with
`AB_LEGACY_COMPOSE_PROFILE=<slc500|micrologix|plc5>` + matching
compose container up.
- All four non-hardware e2e scripts (Modbus / AB CIP / AB Legacy / S7)
now 5/5 against the respective docker-compose fixtures.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
179 lines
7.5 KiB
Markdown
179 lines
7.5 KiB
Markdown
# E2E CLI test scripts
|
|
|
|
End-to-end black-box tests that drive each protocol through its driver CLI
|
|
and verify the resulting OPC UA address-space state through
|
|
`otopcua-cli`. They answer one question per driver:
|
|
|
|
> **If I poke the real PLC through the driver, does the running OtOpcUa
|
|
> server see the change?**
|
|
|
|
This is the acceptance gate v1 was missing — the driver-level integration
|
|
tests (`tests/.../IntegrationTests/`) confirm the driver sees the PLC, and
|
|
the OPC UA `Client.CLI.Tests` confirm the client sees the server — but
|
|
nothing glued them end-to-end. These scripts close that loop.
|
|
|
|
## Five-stage test per driver
|
|
|
|
Every per-driver script runs the same five tests. The goal is to prove
|
|
**both directions** across the bridge plus subscription delivery —
|
|
forward-only coverage would miss writable-flag drops, `IWritable`
|
|
dispatch bugs, and broken data-change notification paths where a fresh
|
|
read still returns the right value.
|
|
|
|
1. **`probe`** — driver CLI opens a session + reads a sentinel. Confirms
|
|
the simulator / PLC is reachable and speaking the protocol.
|
|
2. **Driver loopback** — write a random value via the driver CLI, read
|
|
it back via the same CLI. Confirms the driver round-trips without
|
|
involving the OPC UA server. A failure here is a driver bug, not a
|
|
server-bridge bug.
|
|
3. **Forward bridge (driver → server → client)** — write a different
|
|
random value via the driver CLI, wait `--ServerPollDelaySec` (default
|
|
3s), read the OPC UA NodeId the server publishes that tag at via
|
|
`otopcua-cli read`. Confirms reads propagate from PLC to OPC UA
|
|
client.
|
|
4. **Reverse bridge (client → server → driver)** — write a fresh random
|
|
value via `otopcua-cli write` against the same NodeId, wait
|
|
`--DriverPollDelaySec` (default 3s), read the PLC-side via the
|
|
driver CLI. Confirms writes propagate the other way — catches
|
|
writable-flag drops, ACL misconfiguration, and `IWritable` dispatch
|
|
bugs the forward test can't see.
|
|
5. **Subscribe-sees-change** — start `otopcua-cli subscribe --duration N`
|
|
in the background, give it `--SettleSec` (default 2s) to attach,
|
|
write a random value via the driver CLI, wait for the subscription
|
|
window to close, and assert the captured output mentions the new
|
|
value. Confirms the server's monitored-item + data-change path
|
|
actually fires — not just that a fresh read returns the new value.
|
|
|
|
The OtOpcUa server must already be running with a config that
|
|
(a) binds a driver instance to the same PLC the script points at, and
|
|
(b) publishes the address the script writes under a NodeId the script
|
|
knows. Those NodeIds live in `e2e-config.json` (see below). The
|
|
published tag must be **writable** — stages 4 + 5 will fail against a
|
|
read-only tag.
|
|
|
|
## Status
|
|
|
|
Stages 1 + 2 (driver-side probe + loopback) are verified end-to-end
|
|
against the pymodbus / ab_server / python-snap7 fixtures. Stages 3-5
|
|
(anything crossing the OtOpcUa server) are **blocked** on server-side
|
|
driver factory wiring:
|
|
|
|
- `src/ZB.MOM.WW.OtOpcUa.Server/Program.cs` only registers Galaxy +
|
|
FOCAS factories. `DriverInstanceBootstrapper` skips any `DriverType`
|
|
without a registered factory — so Modbus / AB CIP / AB Legacy / S7 /
|
|
TwinCAT rows in the Config DB are silently no-op'd even when the seed
|
|
is perfect.
|
|
- No Config DB seed script exists for non-Galaxy drivers; Admin UI is
|
|
currently the only path to author one.
|
|
|
|
Tracking: **#209** (umbrella) → #210 (Modbus), #211 (AB CIP), #212 (S7),
|
|
#213 (AB Legacy, also hardware-gated — #222). Each child issue lists
|
|
the factory class to write + the seed SQL shape + the verification
|
|
command.
|
|
|
|
Until those ship, stages 3-5 will fail with "read failed" (nothing
|
|
published at that NodeId) and `[FAIL]` the suite even on a running
|
|
server.
|
|
|
|
## Prereqs
|
|
|
|
1. **OtOpcUa server** running on `opc.tcp://localhost:4840` (or pass
|
|
`-OpcUaUrl` to override). The server's Config DB must define a
|
|
driver instance per protocol you want to test, bound to the matching
|
|
simulator endpoint.
|
|
2. **Per-driver simulators** running. See `docs/v2/test-data-sources.md`
|
|
for the simulator matrix — pymodbus / ab_server / python-snap7 /
|
|
opc-plc cover Modbus / AB / S7 / OPC UA Client. FOCAS and TwinCAT
|
|
have no public simulator; they are gated with env-var skip flags
|
|
below.
|
|
3. **PowerShell 7+**. The runner uses null-coalescing + `Set-StrictMode`;
|
|
the Windows-PowerShell-5.1 shell will not parse `test-all.ps1`.
|
|
4. **.NET 10 SDK**. Each script either runs `dotnet run --project
|
|
src/ZB.MOM.WW.OtOpcUa.Driver.<Name>.Cli` directly, or if
|
|
`$env:OTOPCUA_CLI_BIN` points at a publish folder, runs the pre-built
|
|
`otopcua-*.exe` from there (faster for repeat loops).
|
|
|
|
## Running
|
|
|
|
### One protocol at a time
|
|
|
|
```powershell
|
|
./scripts/e2e/test-modbus.ps1 `
|
|
-ModbusHost 127.0.0.1:5502 `
|
|
-BridgeNodeId "ns=2;s=Modbus/HR100"
|
|
```
|
|
|
|
Every per-protocol script takes the driver endpoint, the address to
|
|
write, and the OPC UA NodeId the server exposes it at.
|
|
|
|
### Full matrix
|
|
|
|
```powershell
|
|
./scripts/e2e/test-all.ps1 `
|
|
-ConfigFile ./scripts/e2e/e2e-config.json
|
|
```
|
|
|
|
The runner reads the sidecar JSON, invokes each driver's script with the
|
|
parameters from that section, and prints a `FINAL MATRIX` showing
|
|
PASS / FAIL / SKIP per driver. Any driver absent from the sidecar is
|
|
SKIP-ed rather than failing hard — useful on dev boxes that only have
|
|
one simulator up.
|
|
|
|
### Sidecar format
|
|
|
|
Copy `e2e-config.sample.json` → `e2e-config.json` and fill in the
|
|
NodeIds from **your** server's Config DB. The file is `.gitignore`-d
|
|
(each dev's NodeIds are specific to their local seed). Omit a driver
|
|
section to skip it.
|
|
|
|
## Expected pass/fail matrix (default config)
|
|
|
|
| Driver | Gate | Default state on a clean dev box |
|
|
|---|---|---|
|
|
| Modbus | — | **PASS** (pymodbus fixture) |
|
|
| AB CIP | — | **PASS** (ab_server fixture) |
|
|
| AB Legacy | — | **PASS** (ab_server SLC500/MicroLogix/PLC-5 profiles; `/1,0` cip-path required for the Docker fixture) |
|
|
| S7 | — | **PASS** (python-snap7 fixture) |
|
|
| FOCAS | `FOCAS_TRUST_WIRE=1` | **SKIP** (no public simulator — task #222 lab rig) |
|
|
| TwinCAT | `TWINCAT_TRUST_WIRE=1` | **SKIP** (needs XAR or standalone Router — task #221) |
|
|
| Phase 7 | — | **PASS** if the Modbus instance seeds a `VT_DoubledHR100` virtual tag + `AlarmHigh` scripted alarm |
|
|
|
|
Set the `*_TRUST_WIRE` env vars to `1` when you've pointed the script at
|
|
real hardware or a properly-configured simulator.
|
|
|
|
## Output
|
|
|
|
Each step prints one of:
|
|
|
|
- `[PASS] ...` — step succeeded
|
|
- `[FAIL] ...` — step failed, stdout of the failing CLI is echoed below
|
|
for diagnosis
|
|
- `[SKIP] ...` — step short-circuited (env-var gate)
|
|
- `[INFO] ...` — progress note (e.g., "waiting 3s for server-side poll")
|
|
|
|
The runner ends with a coloured summary per driver:
|
|
|
|
```
|
|
==================== FINAL MATRIX ====================
|
|
modbus PASS
|
|
abcip PASS
|
|
ablegacy SKIP (no config entry)
|
|
s7 PASS
|
|
focas SKIP (no config entry)
|
|
twincat SKIP (no config entry)
|
|
phase7 PASS
|
|
All present suites passed.
|
|
```
|
|
|
|
Non-zero exit if any present suite failed. SKIPs do not fail the run.
|
|
|
|
## Why this is separate from `dotnet test`
|
|
|
|
`dotnet test` covers driver-layer + server-layer correctness in
|
|
isolation — mocks + in-process test hosts. These e2e scripts cover the
|
|
integration seam that unit tests *can't* cover by design: a live OPC UA
|
|
server process, a live simulator, and the wire between them. Run them
|
|
before a v2 release-readiness sign-off, after a driver-layer change
|
|
that could plausibly affect the NodeManager contract, and before any
|
|
"it works on my box" handoff to QA.
|