5fc596a9a1
Seven-stage e2e script covering every Galaxy-specific capability surface:
IReadable + IWritable + ISubscribable + IAlarmSource + IHistoryProvider.
Unlike the other drivers there is no per-protocol CLI — Galaxy's proxy
lives in-process with the server + talks to OtOpcUaGalaxyHost over a
named pipe (MXAccess COM is 32-bit-only), so every stage runs through
`otopcua-cli` against the published OPC UA address space.
## Stages
1. Probe — otopcua-cli read on the source NodeId
2. Source read — capture value for downstream comparison
3. Virtual-tag bridge — Phase 7 VirtualTag (source × 2) through
CachedTagUpstreamSource
4. Subscribe-sees-change — data-change events propagate
5. Reverse bridge — opc-ua write → Galaxy; soft-passes if the
attribute's Galaxy-side ACL forbids writes
(`BadUserAccessDenied` / `BadNotWritable`)
6. Alarm fires — scripted-alarm Condition fires with Active
state when source crosses threshold
7. History read — historyread returns samples from the Aveva
Historian → IHistoryProvider path
## Two new helpers in _common.ps1
- `Test-AlarmFiresOnThreshold` — start `otopcua-cli alarms --refresh`
in the background on a Condition NodeId, drive the source change,
assert captured stdout contains `ALARM` + `Active`. Uses the same
Start-Process + temp-file pattern as `Test-SubscribeSeesChange` since
the alarms command runs until Ctrl+C (no built-in --duration).
- `Test-HistoryHasSamples` — call `otopcua-cli historyread` over a
configurable lookback window, parse `N values returned.` marker, fail
if below MinSamples. Works for driver-sourced, virtual, or scripted-
alarm historized nodes.
## Wiring
- `test-all.ps1` picks up the optional `galaxy` sidecar section and
runs the script with the configured NodeIds + wait windows.
- `e2e-config.sample.json` adds a `galaxy` section seeded with the
Phase 7 defaults (`p7-smoke-tag-source` / `-vt-derived` /
`-al-overtemp`) — matches `scripts/smoke/seed-phase-7-smoke.sql`.
- `scripts/e2e/README.md` expected-matrix gains a Galaxy row.
## Prereqs
- OtOpcUaGalaxyHost running (NSSM-wrapped) with the Galaxy + MXAccess
runtime available
- `seed-phase-7-smoke.sql` applied with a live Galaxy attribute
substituted into `dbo.Tag.TagConfig`
- OtOpcUa server running against the `p7-smoke` cluster
- Non-elevated shell (Galaxy.Host pipe ACL denies Admins)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
180 lines
7.6 KiB
Markdown
180 lines
7.6 KiB
Markdown
# E2E CLI test scripts
|
|
|
|
End-to-end black-box tests that drive each protocol through its driver CLI
|
|
and verify the resulting OPC UA address-space state through
|
|
`otopcua-cli`. They answer one question per driver:
|
|
|
|
> **If I poke the real PLC through the driver, does the running OtOpcUa
|
|
> server see the change?**
|
|
|
|
This is the acceptance gate v1 was missing — the driver-level integration
|
|
tests (`tests/.../IntegrationTests/`) confirm the driver sees the PLC, and
|
|
the OPC UA `Client.CLI.Tests` confirm the client sees the server — but
|
|
nothing glued them end-to-end. These scripts close that loop.
|
|
|
|
## Five-stage test per driver
|
|
|
|
Every per-driver script runs the same five tests. The goal is to prove
|
|
**both directions** across the bridge plus subscription delivery —
|
|
forward-only coverage would miss writable-flag drops, `IWritable`
|
|
dispatch bugs, and broken data-change notification paths where a fresh
|
|
read still returns the right value.
|
|
|
|
1. **`probe`** — driver CLI opens a session + reads a sentinel. Confirms
|
|
the simulator / PLC is reachable and speaking the protocol.
|
|
2. **Driver loopback** — write a random value via the driver CLI, read
|
|
it back via the same CLI. Confirms the driver round-trips without
|
|
involving the OPC UA server. A failure here is a driver bug, not a
|
|
server-bridge bug.
|
|
3. **Forward bridge (driver → server → client)** — write a different
|
|
random value via the driver CLI, wait `--ServerPollDelaySec` (default
|
|
3s), read the OPC UA NodeId the server publishes that tag at via
|
|
`otopcua-cli read`. Confirms reads propagate from PLC to OPC UA
|
|
client.
|
|
4. **Reverse bridge (client → server → driver)** — write a fresh random
|
|
value via `otopcua-cli write` against the same NodeId, wait
|
|
`--DriverPollDelaySec` (default 3s), read the PLC-side via the
|
|
driver CLI. Confirms writes propagate the other way — catches
|
|
writable-flag drops, ACL misconfiguration, and `IWritable` dispatch
|
|
bugs the forward test can't see.
|
|
5. **Subscribe-sees-change** — start `otopcua-cli subscribe --duration N`
|
|
in the background, give it `--SettleSec` (default 2s) to attach,
|
|
write a random value via the driver CLI, wait for the subscription
|
|
window to close, and assert the captured output mentions the new
|
|
value. Confirms the server's monitored-item + data-change path
|
|
actually fires — not just that a fresh read returns the new value.
|
|
|
|
The OtOpcUa server must already be running with a config that
|
|
(a) binds a driver instance to the same PLC the script points at, and
|
|
(b) publishes the address the script writes under a NodeId the script
|
|
knows. Those NodeIds live in `e2e-config.json` (see below). The
|
|
published tag must be **writable** — stages 4 + 5 will fail against a
|
|
read-only tag.
|
|
|
|
## Status
|
|
|
|
Stages 1 + 2 (driver-side probe + loopback) are verified end-to-end
|
|
against the pymodbus / ab_server / python-snap7 fixtures. Stages 3-5
|
|
(anything crossing the OtOpcUa server) are **blocked** on server-side
|
|
driver factory wiring:
|
|
|
|
- `src/ZB.MOM.WW.OtOpcUa.Server/Program.cs` only registers Galaxy +
|
|
FOCAS factories. `DriverInstanceBootstrapper` skips any `DriverType`
|
|
without a registered factory — so Modbus / AB CIP / AB Legacy / S7 /
|
|
TwinCAT rows in the Config DB are silently no-op'd even when the seed
|
|
is perfect.
|
|
- No Config DB seed script exists for non-Galaxy drivers; Admin UI is
|
|
currently the only path to author one.
|
|
|
|
Tracking: **#209** (umbrella) → #210 (Modbus), #211 (AB CIP), #212 (S7),
|
|
#213 (AB Legacy, also hardware-gated — #222). Each child issue lists
|
|
the factory class to write + the seed SQL shape + the verification
|
|
command.
|
|
|
|
Until those ship, stages 3-5 will fail with "read failed" (nothing
|
|
published at that NodeId) and `[FAIL]` the suite even on a running
|
|
server.
|
|
|
|
## Prereqs
|
|
|
|
1. **OtOpcUa server** running on `opc.tcp://localhost:4840` (or pass
|
|
`-OpcUaUrl` to override). The server's Config DB must define a
|
|
driver instance per protocol you want to test, bound to the matching
|
|
simulator endpoint.
|
|
2. **Per-driver simulators** running. See `docs/v2/test-data-sources.md`
|
|
for the simulator matrix — pymodbus / ab_server / python-snap7 /
|
|
opc-plc cover Modbus / AB / S7 / OPC UA Client. FOCAS and TwinCAT
|
|
have no public simulator; they are gated with env-var skip flags
|
|
below.
|
|
3. **PowerShell 7+**. The runner uses null-coalescing + `Set-StrictMode`;
|
|
the Windows-PowerShell-5.1 shell will not parse `test-all.ps1`.
|
|
4. **.NET 10 SDK**. Each script either runs `dotnet run --project
|
|
src/ZB.MOM.WW.OtOpcUa.Driver.<Name>.Cli` directly, or if
|
|
`$env:OTOPCUA_CLI_BIN` points at a publish folder, runs the pre-built
|
|
`otopcua-*.exe` from there (faster for repeat loops).
|
|
|
|
## Running
|
|
|
|
### One protocol at a time
|
|
|
|
```powershell
|
|
./scripts/e2e/test-modbus.ps1 `
|
|
-ModbusHost 127.0.0.1:5502 `
|
|
-BridgeNodeId "ns=2;s=Modbus/HR100"
|
|
```
|
|
|
|
Every per-protocol script takes the driver endpoint, the address to
|
|
write, and the OPC UA NodeId the server exposes it at.
|
|
|
|
### Full matrix
|
|
|
|
```powershell
|
|
./scripts/e2e/test-all.ps1 `
|
|
-ConfigFile ./scripts/e2e/e2e-config.json
|
|
```
|
|
|
|
The runner reads the sidecar JSON, invokes each driver's script with the
|
|
parameters from that section, and prints a `FINAL MATRIX` showing
|
|
PASS / FAIL / SKIP per driver. Any driver absent from the sidecar is
|
|
SKIP-ed rather than failing hard — useful on dev boxes that only have
|
|
one simulator up.
|
|
|
|
### Sidecar format
|
|
|
|
Copy `e2e-config.sample.json` → `e2e-config.json` and fill in the
|
|
NodeIds from **your** server's Config DB. The file is `.gitignore`-d
|
|
(each dev's NodeIds are specific to their local seed). Omit a driver
|
|
section to skip it.
|
|
|
|
## Expected pass/fail matrix (default config)
|
|
|
|
| Driver | Gate | Default state on a clean dev box |
|
|
|---|---|---|
|
|
| Modbus | — | **PASS** (pymodbus fixture) |
|
|
| AB CIP | — | **PASS** (ab_server fixture) |
|
|
| AB Legacy | — | **PASS** (ab_server SLC500/MicroLogix/PLC-5 profiles; `/1,0` cip-path required for the Docker fixture) |
|
|
| Galaxy | — | **PASS** (requires OtOpcUaGalaxyHost + a live Galaxy; 7 stages including alarms + history) |
|
|
| S7 | — | **PASS** (python-snap7 fixture) |
|
|
| FOCAS | `FOCAS_TRUST_WIRE=1` | **SKIP** (no public simulator — task #222 lab rig) |
|
|
| TwinCAT | `TWINCAT_TRUST_WIRE=1` | **SKIP** (needs XAR or standalone Router — task #221) |
|
|
| Phase 7 | — | **PASS** if the Modbus instance seeds a `VT_DoubledHR100` virtual tag + `AlarmHigh` scripted alarm |
|
|
|
|
Set the `*_TRUST_WIRE` env vars to `1` when you've pointed the script at
|
|
real hardware or a properly-configured simulator.
|
|
|
|
## Output
|
|
|
|
Each step prints one of:
|
|
|
|
- `[PASS] ...` — step succeeded
|
|
- `[FAIL] ...` — step failed, stdout of the failing CLI is echoed below
|
|
for diagnosis
|
|
- `[SKIP] ...` — step short-circuited (env-var gate)
|
|
- `[INFO] ...` — progress note (e.g., "waiting 3s for server-side poll")
|
|
|
|
The runner ends with a coloured summary per driver:
|
|
|
|
```
|
|
==================== FINAL MATRIX ====================
|
|
modbus PASS
|
|
abcip PASS
|
|
ablegacy SKIP (no config entry)
|
|
s7 PASS
|
|
focas SKIP (no config entry)
|
|
twincat SKIP (no config entry)
|
|
phase7 PASS
|
|
All present suites passed.
|
|
```
|
|
|
|
Non-zero exit if any present suite failed. SKIPs do not fail the run.
|
|
|
|
## Why this is separate from `dotnet test`
|
|
|
|
`dotnet test` covers driver-layer + server-layer correctness in
|
|
isolation — mocks + in-process test hosts. These e2e scripts cover the
|
|
integration seam that unit tests *can't* cover by design: a live OPC UA
|
|
server process, a live simulator, and the wire between them. Run them
|
|
before a v2 release-readiness sign-off, after a driver-layer change
|
|
that could plausibly affect the NodeManager contract, and before any
|
|
"it works on my box" handoff to QA.
|