fix(install): swap waits for the host process to exit before moving the folder (#908)
v2-ci / unit-tests (push) Has been cancelled

On QA (76e73e7d deploy) Invoke-OtOpcUaInstallSwap.ps1's first Move-Item of
app\ failed "file in use" right after the service reported Stopped: SCM
reports Stopped while the host is still exiting (e.g. a log file under
InstallRoot still open).

- After the stop, wait (bounded, -ProcessExitTimeoutSeconds, default 60)
  until the service's host process (PID + start time captured before the
  stop) and every process imaged under InstallRoot have exited; otherwise
  throw naming them, with nothing moved.
- The renames use [IO.Directory]::Move instead of Move-Item. On windev,
  PowerShell 7's Move-Item falls back to an item-by-item move when the
  rename fails because a file under the folder is open, and leaves a partial
  PreviousInstall tree behind (a retry then nests into it).
- The first rename is retried for at most -MoveRetrySeconds (default 10)
  while it fails because something under InstallRoot is open (a backstop
  for a holder whose image is outside InstallRoot).
- -Verify/-Reapply/-Scope semantics unchanged.

Tests: dry-run harness gains fake-process-list tests for the wait and the
retry plus AST order checks (145 passed); the Windows harness gains three
real cases (holder past the budget, holder released 2 s into the wait,
holder outside InstallRoot) - 77/0 on pwsh 7 and Windows PowerShell 5.1 on
windev; with the wait removed the first two fail.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
2026-11-13 08:46:26 -05:00
parent f416b88324
commit a157b27628
6 changed files with 332 additions and 13 deletions
@@ -257,15 +257,89 @@ function First-Offset([string]$Name, [string]$Contains = '') {
$hit = @($cmds | Where-Object { $_.GetCommandName() -eq $Name -and $_.Extent.Text -like "*$Contains*" } | Sort-Object { $_.Extent.StartOffset })
if ($hit.Count) { $hit[0].Extent.StartOffset } else { -1 }
}
$swapText = $swapAst.Extent.Text
function Text-Offset([string]$Text) { $swapText.IndexOf($Text, [StringComparison]::Ordinal) }
$iso = @($cmds | Where-Object { $_.GetCommandName() -eq 'Invoke-Isolation' })
Assert ($iso.Count -eq 4 -and @($iso | Where-Object { $_.CommandElements.Count -ne 3 }).Count -eq 0) "every Invoke-Isolation call names a scope ($(@($iso | ForEach-Object { $_.Extent.Text }) -join ' | '))"
$order = @(
(First-Offset 'Invoke-Isolation' "'Reapply' 'Outside'"), (First-Offset 'Invoke-Isolation' "'Verify' 'Outside'"), (First-Offset 'Stop-Service'),
(First-Offset 'Move-Item' '$Candidate'), (First-Offset 'Invoke-Isolation' "'Reapply' 'InstallRoot'"),
(Text-Offset '[System.IO.Directory]::Move($Candidate, $InstallRoot)'), (First-Offset 'Invoke-Isolation' "'Reapply' 'InstallRoot'"),
(First-Offset 'Invoke-Isolation' "'Verify' 'InstallRoot'"), (First-Offset 'Start-Service'))
Assert (@($order | Where-Object { $_ -lt 0 }).Count -eq 0 -and (($order -join ',') -eq (@($order | Sort-Object) -join ','))) `
"Reapply/Verify Outside -> stop -> swap -> Reapply/Verify InstallRoot -> start ($($order -join ','))"
$swapExit = First-Offset 'Wait-OtOpcUaInstallReleased'
$swapRetry = First-Offset 'Invoke-OtOpcUaMoveWithRetry' '::Move($InstallRoot, $PreviousInstall)'
$swapPid = First-Offset 'Get-CimInstance' 'Win32_Service'
Assert ($swapPid -ge 0 -and $swapPid -lt (First-Offset 'Stop-Service') -and (First-Offset 'Stop-Service') -lt $swapExit -and $swapExit -lt $swapRetry -and
$swapRetry -lt (Text-Offset '[System.IO.Directory]::Move($Candidate')) `
"lmxopcua#908: host PID captured -> stop -> wait for the processes to exit -> first move (with retry) -> candidate move ($swapPid,$(First-Offset 'Stop-Service'),$swapExit,$swapRetry,$(Text-Offset '[System.IO.Directory]::Move($Candidate'))"
function Test-InsideCommand($Ast, [string]$Name) {
for ($p = $Ast.Parent; $p; $p = $p.Parent) { if ($p -is [System.Management.Automation.Language.CommandAst] -and $p.GetCommandName() -eq $Name) { return $true } }
$false
}
$firstMoves = @($swapAst.FindAll({ param($n) $n -is [System.Management.Automation.Language.InvokeMemberExpressionAst] -and
$n.Extent.Text -like '*::Move($InstallRoot, $PreviousInstall)' }, $true))
Assert ($firstMoves.Count -eq 1 -and @($firstMoves | Where-Object { -not (Test-InsideCommand $_ 'Invoke-OtOpcUaMoveWithRetry') }).Count -eq 0) `
"the first move (InstallRoot -> PreviousInstall) runs only inside Invoke-OtOpcUaMoveWithRetry ($($firstMoves.Count))"
$moveItems = @($cmds | Where-Object { $_.GetCommandName() -eq 'Move-Item' })
Assert ($moveItems.Count -eq 0) "no Move-Item: PowerShell 7's falls back to a partial item-by-item move when the rename fails (lmxopcua#908) ($(@($moveItems | ForEach-Object { $_.Extent.Text }) -join ' | '))"
Write-Host 'Host process exit wait (lmxopcua#908; fake process lists)'
$qaApp = 'E:\ApiInstall\OtOpcUa\app'
$t0 = [datetime]::new(2026, 11, 13, 12, 0, 0, [DateTimeKind]::Utc)
$hostP = [pscustomobject]@{ Id = 4100; Name = 'OtOpcUa.Host.exe'; Path = "$qaApp\OtOpcUa.Host.exe"; StartTime = $t0 }
$workerP = [pscustomobject]@{ Id = 4200; Name = 'ZB.MOM.WW.OtOpcUa.ScriptWorker.exe'; Path = "$qaApp\script-worker\ZB.MOM.WW.OtOpcUa.ScriptWorker.exe"; StartTime = $t0 }
$hostNoPath = [pscustomobject]@{ Id = 4100; Name = 'OtOpcUa.Host.exe'; Path = ''; StartTime = $t0 }
$others = @(
[pscustomobject]@{ Id = 10; Name = 'svchost.exe'; Path = 'C:\Windows\System32\svchost.exe'; StartTime = $t0 },
[pscustomobject]@{ Id = 11; Name = 'OtOpcUa.Host.exe'; Path = "$qaApp.new-76e73e7d\OtOpcUa.Host.exe"; StartTime = $t0 }, # the candidate: a prefix sibling
[pscustomobject]@{ Id = 12; Name = 'pwsh.exe'; Path = $null; StartTime = $null })
$h = @(Find-OtOpcUaInstallHolder -Snapshot (@($hostP, $workerP) + $others) -InstallRoot "$qaApp\" -ProcessId 4100 -ProcessStartTime $t0)
Assert ((@($h | ForEach-Object { $_.Id }) -join ',') -eq '4100,4200') "host + worker under app\ hold it; the candidate folder and others do not ($(@($h | ForEach-Object { $_.Id }) -join ','))"
$h = @(Find-OtOpcUaInstallHolder -Snapshot (@($hostNoPath) + $others) -InstallRoot $qaApp -ProcessId 4100 -ProcessStartTime $t0)
Assert ((@($h | ForEach-Object { $_.Id }) -join ',') -eq '4100') "the host counts by PID even when its image path is unreadable ($(@($h | ForEach-Object { $_.Id }) -join ','))"
$reused = [pscustomobject]@{ Id = 4100; Name = 'notepad.exe'; Path = 'C:\Windows\notepad.exe'; StartTime = $t0.AddSeconds(30) }
$h = @(Find-OtOpcUaInstallHolder -Snapshot (@($reused) + $others) -InstallRoot $qaApp -ProcessId 4100 -ProcessStartTime $t0)
Assert ($h.Count -eq 0) 'a different process that reused the PID does not count'
$h = @(Find-OtOpcUaInstallHolder -Snapshot $others -InstallRoot $qaApp)
Assert ($h.Count -eq 0) 'nothing under app\ and no PID: released'
# The QA case: SCM said Stopped, the host + worker are gone ~1 s later. The wait must see them, then return.
$script:calls = 0
$fake = { $script:calls++; if ($script:calls -le 3) { @($hostP, $workerP) + $others } else { $others } }
$waited = Wait-OtOpcUaInstallReleased -InstallRoot $qaApp -ProcessId 4100 -ProcessStartTime $t0 -TimeoutSeconds 10 -PollMilliseconds 20 -GetSnapshot $fake
Assert ($script:calls -eq 4) "waits while the host and worker are still exiting, returns once they are gone (snapshots taken: $($script:calls), $([math]::Round($waited, 2)) s)"
$script:calls = 0
$null = Wait-OtOpcUaInstallReleased -InstallRoot $qaApp -ProcessId 4100 -TimeoutSeconds 10 -PollMilliseconds 20 -GetSnapshot { $script:calls++; $others }
Assert ($script:calls -eq 1) "already released: one snapshot, no wait ($($script:calls))"
$sw908 = [System.Diagnostics.Stopwatch]::StartNew()
Assert-Throws { Wait-OtOpcUaInstallReleased -InstallRoot $qaApp -ProcessId 4100 -TimeoutSeconds 1 -PollMilliseconds 50 -GetSnapshot { @($workerP) + $others } } `
'*still in use 1 s after*ZB.MOM.WW.OtOpcUa.ScriptWorker.exe (PID 4200*Nothing was moved*' 'a process that never exits fails clearly, naming it'
Assert ($sw908.Elapsed.TotalSeconds -lt 5) "the wait is bounded ($([math]::Round($sw908.Elapsed.TotalSeconds, 1)) s for a 1 s budget)"
Write-Host 'First-move retry on a sharing violation (lmxopcua#908)'
$inUse = [System.IO.IOException]::new('The process cannot access the file because it is being used by another process.', -2147024864) # 0x80070020
Assert (Test-OtOpcUaSharingViolation -Exception $inUse) 'ERROR_SHARING_VIOLATION is recognised'
Assert (Test-OtOpcUaSharingViolation -Exception ([System.Management.Automation.RuntimeException]::new('wrapped', $inUse))) 'also when wrapped'
Assert (-not (Test-OtOpcUaSharingViolation -Exception ([System.IO.IOException]::new('Cannot create a file when that file already exists.', -2147024713)))) 'ERROR_ALREADY_EXISTS is not'
Assert (Test-OtOpcUaSharingViolation -Exception ([System.Management.Automation.MethodInvocationException]::new('Exception calling "Move" with "2" argument(s)',
[System.IO.IOException]::new("Access to the path 'E:\ApiInstall\OtOpcUa\app' is denied.", -2147024891)))) `
'the [IO.Directory]::Move shape (MethodInvocationException over IOException 0x80070005, both hosts, windev)'
Assert (Test-OtOpcUaSharingViolation -Exception ([System.IO.IOException]::new("Access to the path 'E:\ApiInstall\OtOpcUa\app' is denied.", -2146232800))) `
'the Windows PowerShell 5.1 shape of the same failure (IOException COR_E_IO "Access to the path ... is denied", windev)'
Assert (-not (Test-OtOpcUaSharingViolation -Exception ([System.UnauthorizedAccessException]::new("Access to the path 'x' is denied.")))) 'an UnauthorizedAccessException is not'
$script:calls = 0
$n = try { Invoke-OtOpcUaMoveWithRetry -RetrySeconds 5 -DelayMilliseconds 10 -Move { $script:calls++; if ($script:calls -le 2) { throw $inUse } } 3>$null } catch { "threw: $($_.Exception.Message)" }
Assert ($n -eq 3 -and $script:calls -eq 3) "in use twice, then moved on attempt 3 ($n)"
$script:calls = 0
Assert-Throws { Invoke-OtOpcUaMoveWithRetry -RetrySeconds 5 -DelayMilliseconds 10 -Move { $script:calls++; throw [System.UnauthorizedAccessException]::new('Access to the path is denied.') } 3>$null } `
'*denied*' 'any other error is not retried'
Assert ($script:calls -eq 1) " (one attempt: $($script:calls))"
$script:calls = 0
Assert-Throws { Invoke-OtOpcUaMoveWithRetry -RetrySeconds 0 -DelayMilliseconds 10 -Move { $script:calls++; throw $inUse } 3>$null } '*being used by another process*' '-MoveRetrySeconds 0: no retry'
Assert ($script:calls -eq 1) " (one attempt: $($script:calls))"
if ($IsWindows) {
Write-Host 'SDDL parses on Windows'
$ds = [System.Security.AccessControl.DirectorySecurity]::new()