fix(install): swap waits for the host process to exit before moving the folder (#908)
v2-ci / unit-tests (push) Has been cancelled

On QA (76e73e7d deploy) Invoke-OtOpcUaInstallSwap.ps1's first Move-Item of
app\ failed "file in use" right after the service reported Stopped: SCM
reports Stopped while the host is still exiting (e.g. a log file under
InstallRoot still open).

- After the stop, wait (bounded, -ProcessExitTimeoutSeconds, default 60)
  until the service's host process (PID + start time captured before the
  stop) and every process imaged under InstallRoot have exited; otherwise
  throw naming them, with nothing moved.
- The renames use [IO.Directory]::Move instead of Move-Item. On windev,
  PowerShell 7's Move-Item falls back to an item-by-item move when the
  rename fails because a file under the folder is open, and leaves a partial
  PreviousInstall tree behind (a retry then nests into it).
- The first rename is retried for at most -MoveRetrySeconds (default 10)
  while it fails because something under InstallRoot is open (a backstop
  for a holder whose image is outside InstallRoot).
- -Verify/-Reapply/-Scope semantics unchanged.

Tests: dry-run harness gains fake-process-list tests for the wait and the
retry plus AST order checks (145 passed); the Windows harness gains three
real cases (holder past the budget, holder released 2 s into the wait,
holder outside InstallRoot) - 77/0 on pwsh 7 and Windows PowerShell 5.1 on
windev; with the wait removed the first two fail.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
2026-11-13 08:46:26 -05:00
parent f416b88324
commit a157b27628
6 changed files with 332 additions and 13 deletions
+39 -7
View File
@@ -16,8 +16,14 @@
PE cache, denies outside InstallRoot, firewall rule, host env) and the coverage scan of the denied folders
outside InstallRoot.
If either fails, the script stops here: nothing is stopped or moved.
2. Stop the service (unless -NoServiceControl). The stopped window starts here.
3. Move InstallRoot -> PreviousInstall, then Candidate -> InstallRoot.
2. Stop the service (unless -NoServiceControl). The stopped window starts here. SCM can report Stopped while the
host process is still exiting (lmxopcua#908), so the script then waits, at most -ProcessExitTimeoutSeconds, until
the service's host process (PID captured before the stop) and every process whose image is under InstallRoot
(OtOpcUa.Host, the ScriptWorker) have exited. If one is still there, it throws naming them: nothing has moved.
3. Move InstallRoot -> PreviousInstall, then Candidate -> InstallRoot, each one rename ([IO.Directory]::Move; not
Move-Item, which in PowerShell 7 falls back to a partial item-by-item move when the rename fails). The first
move is retried for at most -MoveRetrySeconds while it fails because something under InstallRoot is open
(nothing has moved yet; a backstop for a handle the process wait cannot see).
4. Copy each -Carry item (relative to InstallRoot; wildcards allowed) from PreviousInstall into the new InstallRoot.
Folders are copied with robocopy /E /COPY:DAT (data, attributes, timestamps; not ACLs: step 5 sets those).
5. Isolation, when recorded: -Reapply -Scope InstallRoot, then -Verify -Scope InstallRoot: only the entries a swap
@@ -66,6 +72,13 @@
.PARAMETER SkipIsolation
Do not re-apply/verify the isolation even when it is installed. WARNs. Only for a box where it is being removed.
.PARAMETER ProcessExitTimeoutSeconds
After the stop, how long to wait for the host process and any process running from InstallRoot to exit before
failing with nothing moved (default 60; lmxopcua#908).
.PARAMETER MoveRetrySeconds
How long to keep retrying the first folder move while it fails with a sharing violation (default 10; 0 = no retry).
.PARAMETER WorkerServiceName
Passed to the isolation installer (default OtOpcUaScriptWorker).
@@ -96,7 +109,9 @@ param(
[switch]$NoStart,
[switch]$SkipIsolation,
[ValidatePattern('^[A-Za-z0-9_.\-]{1,80}$')] [string]$WorkerServiceName = 'OtOpcUaScriptWorker',
[int]$StopTimeoutSeconds = 120
[int]$StopTimeoutSeconds = 120,
[ValidateRange(0, 3600)] [int]$ProcessExitTimeoutSeconds = 60,
[ValidateRange(0, 600)] [int]$MoveRetrySeconds = 10
)
Set-StrictMode -Version Latest
@@ -131,7 +146,7 @@ if (-not (Test-Path -LiteralPath (Join-Path $Candidate 'script-worker') -PathTyp
$candidateWorkerExe = Join-Path $Candidate 'script-worker\ZB.MOM.WW.OtOpcUa.ScriptWorker.exe'
if (Test-Path -LiteralPath $PreviousInstall) { throw "PreviousInstall $PreviousInstall already exists; pick a fresh name" }
if ([IO.Path]::GetPathRoot($InstallRoot) -ne [IO.Path]::GetPathRoot($PreviousInstall) -or [IO.Path]::GetPathRoot($InstallRoot) -ne [IO.Path]::GetPathRoot($Candidate)) {
throw 'InstallRoot, Candidate and PreviousInstall must be on the same volume (Move-Item renames)'
throw 'InstallRoot, Candidate and PreviousInstall must be on the same volume (the swap renames folders)'
}
$isolation = $null
if (-not $SkipIsolation) {
@@ -169,26 +184,43 @@ if ($isolation) {
}
# ---- 2. Stop ---------------------------------------------------------------------------------------------------------
$hostPid = 0; $hostStart = $null
$window.Start()
if (-not $NoServiceControl) {
Step "stop $ServiceName"
$svc = Get-Service -Name $ServiceName
if ($svc.Status -ne 'Stopped') {
# lmxopcua#908: remember the host process (PID + start time, so a reused PID does not count) before stopping it.
$hostPid = [int](Get-CimInstance -ClassName Win32_Service -Filter "Name='$ServiceName'").ProcessId
if ($hostPid -gt 0) {
$hostProc = Get-Process -Id $hostPid -ErrorAction SilentlyContinue
if ($hostProc) { try { $hostStart = $hostProc.StartTime.ToUniversalTime() } catch { } }
}
Stop-Service -Name $ServiceName -Force
$svc.WaitForStatus('Stopped', [TimeSpan]::FromSeconds($StopTimeoutSeconds))
}
Mark 'stop' $window
}
# lmxopcua#908: SCM reports Stopped before the host process is gone; the folder move fails "in use" until it is.
$sw = [System.Diagnostics.Stopwatch]::StartNew()
Step "wait for the host process (PID $(if ($hostPid) { $hostPid } else { 'n/a' })) and any process under $InstallRoot to exit (at most $ProcessExitTimeoutSeconds s)"
$waited = Wait-OtOpcUaInstallReleased -InstallRoot $InstallRoot -ProcessId $hostPid -ProcessStartTime $hostStart -TimeoutSeconds $ProcessExitTimeoutSeconds
Write-Host (' released after {0:N1} s' -f $waited)
Mark 'exit' $sw
# ---- 3. Swap ---------------------------------------------------------------------------------------------------------
$sw = [System.Diagnostics.Stopwatch]::StartNew()
Step "move $InstallRoot -> $PreviousInstall"
Move-Item -LiteralPath $InstallRoot -Destination $PreviousInstall
# lmxopcua#908: [IO.Directory]::Move, not Move-Item. It is one rename (MoveFileEx, same volume: checked in preflight) that
# fails as a whole. PowerShell 7's Move-Item, when the rename fails because a file under the folder is open, falls back
# to moving item by item and leaves a partial PreviousInstall tree behind (windev), so a retry would then nest into it.
$attempts = Invoke-OtOpcUaMoveWithRetry -RetrySeconds $MoveRetrySeconds -Move { [System.IO.Directory]::Move($InstallRoot, $PreviousInstall) }
if ($attempts -gt 1) { Write-Host " moved on attempt $attempts" }
Step "move $Candidate -> $InstallRoot"
try { Move-Item -LiteralPath $Candidate -Destination $InstallRoot }
try { [System.IO.Directory]::Move($Candidate, $InstallRoot) }
catch {
Write-Warning "candidate move failed; putting the previous install back: $($_.Exception.Message)"
Move-Item -LiteralPath $PreviousInstall -Destination $InstallRoot
[System.IO.Directory]::Move($PreviousInstall, $InstallRoot)
throw
}