From a9b585ac5b559c94c2d0b8a08660795dd3513b6a Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Tue, 21 Apr 2026 10:08:52 -0400 Subject: [PATCH] =?UTF-8?q?Task=20#253=20follow-up=20=E2=80=94=20bidirecti?= =?UTF-8?q?onal=20+=20subscribe-sees-change=20e2e=20stages?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The original three-stage design (probe / driver-loopback / forward- bridge) only proved driver-write → server-read. It missed: - OPC UA write → server → driver → PLC (the reverse direction) - server-side data-change notifications actually firing (a stale subscription can still let a read-after-the-fact return the new value and look fine) Extend _common.ps1 with two helpers: - Test-OpcUaWriteBridge: otopcua-cli write the NodeId -> wait 3s -> driver CLI read the PLC side, assert equality. - Test-SubscribeSeesChange: Start-Process otopcua-cli subscribe in the background with --duration N, settle 2s, driver-side write, wait for the subscription window to close, assert captured stdout contains the new value. Wire both into test-modbus / test-abcip / test-ablegacy / test-s7 / test-focas / test-twincat after the existing forward-bridge stage. Update README to describe the five-stage design + note that the published NodeId must be writable for stages 4 + 5. Also prepend UTF-8 BOM to every script in scripts/e2e so Windows PowerShell 5.1 parsers agree on em-dash byte sequences the way PowerShell 7 already does. The scripts still #Requires -Version 7.0 — the BOM is purely defensive for IDE / CI step parsers. Co-Authored-By: Claude Opus 4.7 (1M context) --- scripts/e2e/README.md | 37 ++++++-- scripts/e2e/_common.ps1 | 110 +++++++++++++++++++++++- scripts/e2e/test-abcip.ps1 | 23 ++++- scripts/e2e/test-ablegacy.ps1 | 23 ++++- scripts/e2e/test-all.ps1 | 2 +- scripts/e2e/test-focas.ps1 | 20 ++++- scripts/e2e/test-modbus.ps1 | 26 +++++- scripts/e2e/test-phase7-virtualtags.ps1 | 2 +- scripts/e2e/test-s7.ps1 | 26 +++++- scripts/e2e/test-twincat.ps1 | 20 ++++- 10 files changed, 264 insertions(+), 25 deletions(-) diff --git a/scripts/e2e/README.md b/scripts/e2e/README.md index 17bd72c..aaf9e01 100644 --- a/scripts/e2e/README.md +++ b/scripts/e2e/README.md @@ -12,25 +12,44 @@ tests (`tests/.../IntegrationTests/`) confirm the driver sees the PLC, and the OPC UA `Client.CLI.Tests` confirm the client sees the server — but nothing glued them end-to-end. These scripts close that loop. -## Three-stage test per driver +## Five-stage test per driver -Every per-driver script runs the same three tests: +Every per-driver script runs the same five tests. The goal is to prove +**both directions** across the bridge plus subscription delivery — +forward-only coverage would miss writable-flag drops, `IWritable` +dispatch bugs, and broken data-change notification paths where a fresh +read still returns the right value. 1. **`probe`** — driver CLI opens a session + reads a sentinel. Confirms the simulator / PLC is reachable and speaking the protocol. -2. **Driver loopback** — write a random value via the driver CLI, read it - back via the same CLI. Confirms the driver round-trips without +2. **Driver loopback** — write a random value via the driver CLI, read + it back via the same CLI. Confirms the driver round-trips without involving the OPC UA server. A failure here is a driver bug, not a server-bridge bug. -3. **Server bridge** — write a different random value via the driver - CLI, wait `--ServerPollDelaySec` (default 3s), read the OPC UA NodeId - the server publishes that tag at via `otopcua-cli read`. Confirms the - full path: driver CLI → PLC → OtOpcUa server → OPC UA client. +3. **Forward bridge (driver → server → client)** — write a different + random value via the driver CLI, wait `--ServerPollDelaySec` (default + 3s), read the OPC UA NodeId the server publishes that tag at via + `otopcua-cli read`. Confirms reads propagate from PLC to OPC UA + client. +4. **Reverse bridge (client → server → driver)** — write a fresh random + value via `otopcua-cli write` against the same NodeId, wait + `--DriverPollDelaySec` (default 3s), read the PLC-side via the + driver CLI. Confirms writes propagate the other way — catches + writable-flag drops, ACL misconfiguration, and `IWritable` dispatch + bugs the forward test can't see. +5. **Subscribe-sees-change** — start `otopcua-cli subscribe --duration N` + in the background, give it `--SettleSec` (default 2s) to attach, + write a random value via the driver CLI, wait for the subscription + window to close, and assert the captured output mentions the new + value. Confirms the server's monitored-item + data-change path + actually fires — not just that a fresh read returns the new value. The OtOpcUa server must already be running with a config that (a) binds a driver instance to the same PLC the script points at, and (b) publishes the address the script writes under a NodeId the script -knows. Those NodeIds live in `e2e-config.json` (see below). +knows. Those NodeIds live in `e2e-config.json` (see below). The +published tag must be **writable** — stages 4 + 5 will fail against a +read-only tag. ## Prereqs diff --git a/scripts/e2e/_common.ps1 b/scripts/e2e/_common.ps1 index 1a6b17f..595880c 100644 --- a/scripts/e2e/_common.ps1 +++ b/scripts/e2e/_common.ps1 @@ -1,4 +1,4 @@ -# Shared PowerShell helpers for the OtOpcUa end-to-end CLI test scripts. +# Shared PowerShell helpers for the OtOpcUa end-to-end CLI test scripts. # # Every per-protocol script dot-sources this file and calls the Test-* functions # below. Keeps the per-script code down to ~50 lines of parameterisation + @@ -202,6 +202,114 @@ function Test-ServerBridge { return @{ Passed = $false; Reason = "bridge value mismatch" } } +# Test 4 — reverse bridge. Write via the OPC UA client CLI, then read the PLC +# side via the driver CLI. Confirms the write path: OPC UA client → server → +# driver → PLC. This is the direction Test-ServerBridge does NOT cover — a +# clean Test-ServerBridge only proves reads flow server-ward. +function Test-OpcUaWriteBridge { + param( + [Parameter(Mandatory)] $OpcUaCli, + [Parameter(Mandatory)] [string]$OpcUaUrl, + [Parameter(Mandatory)] [string]$OpcUaNodeId, + [Parameter(Mandatory)] $DriverCli, + [Parameter(Mandatory)] [string[]]$DriverReadArgs, + [Parameter(Mandatory)] [string]$ExpectedValue, + [int]$DriverPollDelaySec = 3 + ) + Write-Header "OPC UA write bridge" + + $w = Invoke-Cli -Cli $OpcUaCli -Args @( + "write", "-u", $OpcUaUrl, "-n", $OpcUaNodeId, "-v", $ExpectedValue) + if ($w.ExitCode -ne 0 -or $w.Output -notmatch "Write successful") { + Write-Fail "OPC UA client write failed (exit=$($w.ExitCode))" + Write-Host $w.Output + return @{ Passed = $false; Reason = "opc-ua write failed" } + } + Write-Info "opc-ua write ok, waiting ${DriverPollDelaySec}s for driver-side apply" + Start-Sleep -Seconds $DriverPollDelaySec + + $r = Invoke-Cli -Cli $DriverCli -Args $DriverReadArgs + if ($r.ExitCode -ne 0) { + Write-Fail "driver-side read failed (exit=$($r.ExitCode))" + Write-Host $r.Output + return @{ Passed = $false; Reason = "driver read failed" } + } + + if ($r.Output -match "Value:\s+$([Regex]::Escape($ExpectedValue))\b") { + Write-Pass "PLC-side value equals $ExpectedValue" + return @{ Passed = $true } + } + Write-Fail "PLC-side value mismatch — expected $ExpectedValue" + Write-Host $r.Output + return @{ Passed = $false; Reason = "reverse-bridge value mismatch" } +} + +# Test 5 — subscribe-sees-change. Start `otopcua-cli subscribe --duration N` +# in the background, give it ~2s to attach, then write a known value via the +# driver CLI. After the subscription window closes, assert its captured +# output mentions the new value. Confirms the OPC UA server is actually +# pushing data-change notifications for driver-originated changes — not just +# that a fresh read returns the new value. +function Test-SubscribeSeesChange { + param( + [Parameter(Mandatory)] $OpcUaCli, + [Parameter(Mandatory)] [string]$OpcUaUrl, + [Parameter(Mandatory)] [string]$OpcUaNodeId, + [Parameter(Mandatory)] $DriverCli, + [Parameter(Mandatory)] [string[]]$DriverWriteArgs, + [Parameter(Mandatory)] [string]$ExpectedValue, + [int]$DurationSec = 8, + [int]$SettleSec = 2 + ) + Write-Header "Subscribe sees change" + + # `Start-Job` would spin up a fresh PowerShell runtime and cost 2s+. Use + # Start-Process + a temp file instead — it's the same shape Invoke-Cli + # uses but non-blocking. + $stdout = New-TemporaryFile + $stderr = New-TemporaryFile + $allArgs = @($OpcUaCli.PrefixArgs) + @( + "subscribe", "-u", $OpcUaUrl, "-n", $OpcUaNodeId, + "-i", "200", "--duration", "$DurationSec") + $proc = Start-Process -FilePath $OpcUaCli.File ` + -ArgumentList $allArgs ` + -NoNewWindow -PassThru ` + -RedirectStandardOutput $stdout.FullName ` + -RedirectStandardError $stderr.FullName + Write-Info "subscription started (pid $($proc.Id)), waiting ${SettleSec}s to settle" + Start-Sleep -Seconds $SettleSec + + $w = Invoke-Cli -Cli $DriverCli -Args $DriverWriteArgs + if ($w.ExitCode -ne 0) { + Stop-Process -Id $proc.Id -Force -ErrorAction SilentlyContinue + Remove-Item $stdout.FullName, $stderr.FullName -ErrorAction SilentlyContinue + Write-Fail "driver write during subscribe failed (exit=$($w.ExitCode))" + Write-Host $w.Output + return @{ Passed = $false; Reason = "driver write failed" } + } + Write-Info "driver write ok, waiting for subscription window to close" + + # Wait for the subscribe process to exit its --duration timer. Grace + # margin on top of the duration in case the first data-change races the + # final flush. + $proc.WaitForExit(($DurationSec + 5) * 1000) | Out-Null + if (-not $proc.HasExited) { Stop-Process -Id $proc.Id -Force } + + $out = (Get-Content $stdout.FullName -Raw) + (Get-Content $stderr.FullName -Raw) + Remove-Item $stdout.FullName, $stderr.FullName -ErrorAction SilentlyContinue + + # The subscribe command prints `[timestamp] displayName = value (status)` + # per data-change event. We only care that one of those events carried + # the new value. + if ($out -match "=\s*$([Regex]::Escape($ExpectedValue))\b") { + Write-Pass "subscribe saw $ExpectedValue" + return @{ Passed = $true } + } + Write-Fail "subscribe did not observe $ExpectedValue in ${DurationSec}s" + Write-Host $out + return @{ Passed = $false; Reason = "change not observed on subscription" } +} + # --------------------------------------------------------------------------- # Summary helper — caller passes an array of test results. # --------------------------------------------------------------------------- diff --git a/scripts/e2e/test-abcip.ps1 b/scripts/e2e/test-abcip.ps1 index 83a5c35..7d33b45 100644 --- a/scripts/e2e/test-abcip.ps1 +++ b/scripts/e2e/test-abcip.ps1 @@ -1,4 +1,4 @@ -#Requires -Version 7.0 +#Requires -Version 7.0 <# .SYNOPSIS End-to-end CLI test for the AB CIP driver (ControlLogix / CompactLogix / @@ -6,7 +6,8 @@ .DESCRIPTION Mirrors test-modbus.ps1 but against libplctag's ab_server (or a real Logix - controller). Three assertions: probe / driver-loopback / server-bridge. + controller). Five assertions: probe / driver-loopback / forward-bridge / + reverse-bridge / subscribe-sees-change. Prereqs: - ab_server container up (tests/.../AbCip.IntegrationTests/Docker/docker-compose.yml, @@ -81,5 +82,23 @@ $results += Test-ServerBridge ` -OpcUaNodeId $BridgeNodeId ` -ExpectedValue "$bridgeValue" +$reverseValue = Get-Random -Minimum 20000 -Maximum 29999 +$results += Test-OpcUaWriteBridge ` + -OpcUaCli $opcUaCli ` + -OpcUaUrl $OpcUaUrl ` + -OpcUaNodeId $BridgeNodeId ` + -DriverCli $abcipCli ` + -DriverReadArgs (@("read") + $commonAbCip + @("-t", $TagPath, "--type", "DInt")) ` + -ExpectedValue "$reverseValue" + +$subValue = Get-Random -Minimum 30000 -Maximum 39999 +$results += Test-SubscribeSeesChange ` + -OpcUaCli $opcUaCli ` + -OpcUaUrl $OpcUaUrl ` + -OpcUaNodeId $BridgeNodeId ` + -DriverCli $abcipCli ` + -DriverWriteArgs (@("write") + $commonAbCip + @("-t", $TagPath, "--type", "DInt", "-v", $subValue)) ` + -ExpectedValue "$subValue" + Write-Summary -Title "AB CIP e2e" -Results $results if ($results | Where-Object { -not $_.Passed }) { exit 1 } diff --git a/scripts/e2e/test-ablegacy.ps1 b/scripts/e2e/test-ablegacy.ps1 index 2a8aa68..2ede8b6 100644 --- a/scripts/e2e/test-ablegacy.ps1 +++ b/scripts/e2e/test-ablegacy.ps1 @@ -1,4 +1,4 @@ -#Requires -Version 7.0 +#Requires -Version 7.0 <# .SYNOPSIS End-to-end CLI test for the AB Legacy (PCCC) driver. @@ -9,7 +9,8 @@ golden-box. Against the Docker ab_server the tests deliberately skip — same gate as tests/.../AbLegacy.IntegrationTests (AB_LEGACY_TRUST_WIRE=1). - Three assertions: probe / driver-loopback / server-bridge. + Five assertions: probe / driver-loopback / forward-bridge / reverse-bridge / + subscribe-sees-change. .PARAMETER Gateway ab://host[:port]/cip-path. Default ab://127.0.0.1/1,0. @@ -76,5 +77,23 @@ $results += Test-ServerBridge ` -OpcUaNodeId $BridgeNodeId ` -ExpectedValue "$bridgeValue" +$reverseValue = Get-Random -Minimum 20000 -Maximum 29999 +$results += Test-OpcUaWriteBridge ` + -OpcUaCli $opcUaCli ` + -OpcUaUrl $OpcUaUrl ` + -OpcUaNodeId $BridgeNodeId ` + -DriverCli $abLegacyCli ` + -DriverReadArgs (@("read") + $commonAbLegacy + @("-a", $Address, "-t", "Int")) ` + -ExpectedValue "$reverseValue" + +$subValue = Get-Random -Minimum 30000 -Maximum 32766 +$results += Test-SubscribeSeesChange ` + -OpcUaCli $opcUaCli ` + -OpcUaUrl $OpcUaUrl ` + -OpcUaNodeId $BridgeNodeId ` + -DriverCli $abLegacyCli ` + -DriverWriteArgs (@("write") + $commonAbLegacy + @("-a", $Address, "-t", "Int", "-v", $subValue)) ` + -ExpectedValue "$subValue" + Write-Summary -Title "AB Legacy e2e" -Results $results if ($results | Where-Object { -not $_.Passed }) { exit 1 } diff --git a/scripts/e2e/test-all.ps1 b/scripts/e2e/test-all.ps1 index a57c2f9..589ee80 100644 --- a/scripts/e2e/test-all.ps1 +++ b/scripts/e2e/test-all.ps1 @@ -1,4 +1,4 @@ -#Requires -Version 7.0 +#Requires -Version 7.0 <# .SYNOPSIS Runs every scripts/e2e/test-*.ps1 and tallies PASS / FAIL / SKIP. diff --git a/scripts/e2e/test-focas.ps1 b/scripts/e2e/test-focas.ps1 index b5e61ae..ef6e9a5 100644 --- a/scripts/e2e/test-focas.ps1 +++ b/scripts/e2e/test-focas.ps1 @@ -1,4 +1,4 @@ -#Requires -Version 7.0 +#Requires -Version 7.0 <# .SYNOPSIS End-to-end CLI test for the FOCAS (Fanuc CNC) driver. @@ -74,5 +74,23 @@ $results += Test-ServerBridge ` -OpcUaNodeId $BridgeNodeId ` -ExpectedValue "$bridgeValue" +$reverseValue = Get-Random -Minimum 20000 -Maximum 29999 +$results += Test-OpcUaWriteBridge ` + -OpcUaCli $opcUaCli ` + -OpcUaUrl $OpcUaUrl ` + -OpcUaNodeId $BridgeNodeId ` + -DriverCli $focasCli ` + -DriverReadArgs (@("read") + $commonFocas + @("-a", $Address, "-t", "Int16")) ` + -ExpectedValue "$reverseValue" + +$subValue = Get-Random -Minimum 30000 -Maximum 32766 +$results += Test-SubscribeSeesChange ` + -OpcUaCli $opcUaCli ` + -OpcUaUrl $OpcUaUrl ` + -OpcUaNodeId $BridgeNodeId ` + -DriverCli $focasCli ` + -DriverWriteArgs (@("write") + $commonFocas + @("-a", $Address, "-t", "Int16", "-v", $subValue)) ` + -ExpectedValue "$subValue" + Write-Summary -Title "FOCAS e2e" -Results $results if ($results | Where-Object { -not $_.Passed }) { exit 1 } diff --git a/scripts/e2e/test-modbus.ps1 b/scripts/e2e/test-modbus.ps1 index 9d57b95..7e68b29 100644 --- a/scripts/e2e/test-modbus.ps1 +++ b/scripts/e2e/test-modbus.ps1 @@ -1,13 +1,15 @@ -#Requires -Version 7.0 +#Requires -Version 7.0 <# .SYNOPSIS End-to-end CLI test for the Modbus-TCP driver bridged through the OtOpcUa server. .DESCRIPTION - Three assertions: + Five assertions: 1. `otopcua-modbus-cli probe` hits the simulator 2. Driver-loopback write + read-back via modbus-cli - 3. Bridge: modbus-cli writes HR[100], OPC UA client reads the bridged NodeId + 3. Forward bridge: modbus-cli writes HR[100], OPC UA client reads the bridged NodeId + 4. Reverse bridge: OPC UA client writes the NodeId, modbus-cli reads HR[100] + 5. Subscribe-sees-change: OPC UA subscription observes a modbus-cli write Requires a running Modbus simulator on localhost:5502 (the pymodbus fixture default per docs/drivers/Modbus-Test-Fixture.md) and a running OtOpcUa server @@ -70,5 +72,23 @@ $results += Test-ServerBridge ` -OpcUaNodeId $BridgeNodeId ` -ExpectedValue "$bridgeValue" +$reverseValue = Get-Random -Minimum 20000 -Maximum 29999 +$results += Test-OpcUaWriteBridge ` + -OpcUaCli $opcUaCli ` + -OpcUaUrl $OpcUaUrl ` + -OpcUaNodeId $BridgeNodeId ` + -DriverCli $modbusCli ` + -DriverReadArgs (@("read") + $commonModbus + @("-r", "HoldingRegisters", "-a", "100", "-t", "UInt16")) ` + -ExpectedValue "$reverseValue" + +$subValue = Get-Random -Minimum 30000 -Maximum 39999 +$results += Test-SubscribeSeesChange ` + -OpcUaCli $opcUaCli ` + -OpcUaUrl $OpcUaUrl ` + -OpcUaNodeId $BridgeNodeId ` + -DriverCli $modbusCli ` + -DriverWriteArgs (@("write") + $commonModbus + @("-r", "HoldingRegisters", "-a", "100", "-t", "UInt16", "-v", $subValue)) ` + -ExpectedValue "$subValue" + Write-Summary -Title "Modbus e2e" -Results $results if ($results | Where-Object { -not $_.Passed }) { exit 1 } diff --git a/scripts/e2e/test-phase7-virtualtags.ps1 b/scripts/e2e/test-phase7-virtualtags.ps1 index e0b490d..6eb1e48 100644 --- a/scripts/e2e/test-phase7-virtualtags.ps1 +++ b/scripts/e2e/test-phase7-virtualtags.ps1 @@ -1,4 +1,4 @@ -#Requires -Version 7.0 +#Requires -Version 7.0 <# .SYNOPSIS End-to-end test for Phase 7 virtual tags + scripted alarms, driven via the diff --git a/scripts/e2e/test-s7.ps1 b/scripts/e2e/test-s7.ps1 index ef7a36c..2060d8c 100644 --- a/scripts/e2e/test-s7.ps1 +++ b/scripts/e2e/test-s7.ps1 @@ -1,12 +1,12 @@ -#Requires -Version 7.0 +#Requires -Version 7.0 <# .SYNOPSIS End-to-end CLI test for the Siemens S7 driver bridged through the OtOpcUa server. .DESCRIPTION - Probe + driver-loopback + server-bridge against a Siemens S7-300/400/1200/1500 - or compatible soft-PLC. python-snap7 simulator (task #216) or real hardware - both work. + Five assertions (probe / driver-loopback / forward-bridge / reverse-bridge / + subscribe-sees-change) against a Siemens S7-300/400/1200/1500 or compatible + soft-PLC. python-snap7 simulator (task #216) or real hardware both work. Prereqs: - S7 simulator / PLC on $S7Host:$S7Port @@ -78,5 +78,23 @@ $results += Test-ServerBridge ` -OpcUaNodeId $BridgeNodeId ` -ExpectedValue "$bridgeValue" +$reverseValue = Get-Random -Minimum 20000 -Maximum 29999 +$results += Test-OpcUaWriteBridge ` + -OpcUaCli $opcUaCli ` + -OpcUaUrl $OpcUaUrl ` + -OpcUaNodeId $BridgeNodeId ` + -DriverCli $s7Cli ` + -DriverReadArgs (@("read") + $commonS7 + @("-a", $Address, "-t", "Int16")) ` + -ExpectedValue "$reverseValue" + +$subValue = Get-Random -Minimum 30000 -Maximum 32766 +$results += Test-SubscribeSeesChange ` + -OpcUaCli $opcUaCli ` + -OpcUaUrl $OpcUaUrl ` + -OpcUaNodeId $BridgeNodeId ` + -DriverCli $s7Cli ` + -DriverWriteArgs (@("write") + $commonS7 + @("-a", $Address, "-t", "Int16", "-v", $subValue)) ` + -ExpectedValue "$subValue" + Write-Summary -Title "S7 e2e" -Results $results if ($results | Where-Object { -not $_.Passed }) { exit 1 } diff --git a/scripts/e2e/test-twincat.ps1 b/scripts/e2e/test-twincat.ps1 index 770719c..68d7ab8 100644 --- a/scripts/e2e/test-twincat.ps1 +++ b/scripts/e2e/test-twincat.ps1 @@ -1,4 +1,4 @@ -#Requires -Version 7.0 +#Requires -Version 7.0 <# .SYNOPSIS End-to-end CLI test for the TwinCAT (Beckhoff ADS) driver. @@ -77,5 +77,23 @@ $results += Test-ServerBridge ` -OpcUaNodeId $BridgeNodeId ` -ExpectedValue "$bridgeValue" +$reverseValue = Get-Random -Minimum 20000 -Maximum 29999 +$results += Test-OpcUaWriteBridge ` + -OpcUaCli $opcUaCli ` + -OpcUaUrl $OpcUaUrl ` + -OpcUaNodeId $BridgeNodeId ` + -DriverCli $twinCatCli ` + -DriverReadArgs (@("read") + $commonTc + @("-s", $SymbolPath, "-t", "DInt")) ` + -ExpectedValue "$reverseValue" + +$subValue = Get-Random -Minimum 30000 -Maximum 39999 +$results += Test-SubscribeSeesChange ` + -OpcUaCli $opcUaCli ` + -OpcUaUrl $OpcUaUrl ` + -OpcUaNodeId $BridgeNodeId ` + -DriverCli $twinCatCli ` + -DriverWriteArgs (@("write") + $commonTc + @("-s", $SymbolPath, "-t", "DInt", "-v", $subValue)) ` + -ExpectedValue "$subValue" + Write-Summary -Title "TwinCAT e2e" -Results $results if ($results | Where-Object { -not $_.Passed }) { exit 1 }