fix(install): cut the swap's stopped window to the swapped-in folder (#905)
v2-ci / unit-tests (push) Has been cancelled
v2-ci / unit-tests (push) Has been cancelled
Root cause: the protected-DACL coverage scan, not the ACL writes. Both -Reapply and -Verify walked every denied folder (all of E:\ApiInstall on vd03, nested roots again) with Get-ChildItem -Recurse + Get-Acl per item + a pipeline per item for -Skip, about 3.4 ms an item on PS 5.1, all while the service was stopped. An unchanged deny is read, not rewritten; the re-applied install-folder deny propagates over ~1,400 files in 0.5 s. - Find-OtOpcUaDenyCoverageGap: .NET enumeration, -Skip folders pruned, DACL-only GetAccessControl(Access) (about 0.15 ms an item); reparse points not followed. - Get-OtOpcUaCoverageWalkRoots: each denied folder walked once (outermost only). - Install-ScriptWorkerIsolation.ps1 -Scope All|InstallRoot|Outside (with -Reapply, -Verify, -PlanOnly). InstallRoot = the folder ACEs a swap drops; Outside = the rest, including the firewall rule (its query costs 3-5 s; its exe path does not change). The two partition the plan. Prints the scan time. - Invoke-OtOpcUaInstallSwap.ps1: -Reapply/-Verify -Scope Outside before the stop (a failure stops before anything moves), -Scope InstallRoot inside the window; preflight checks the candidate carries the worker exe; prints the stopped window and its parts. -NoStart, -NoCarry and the pre-#901 refusal unchanged. windev, vd03-like tree (17,851 items): stopped window 153.1 s -> 6.0 s (PS 5.1), 101.6 s -> 6.2 s (PS 7); the Outside part takes ~23 s with the old build serving. Dry-run harness 101 -> 123 passed; Windows test 72 passed on PS 5.1 and PS 7. Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
@@ -11,13 +11,27 @@
|
||||
0. Preflight (nothing changed yet): the candidate exists and carries script-worker\; PreviousInstall does not exist
|
||||
and is on the same volume; when isolation is installed, its settings are recorded (an install by an installer
|
||||
older than #901 recorded only DenyPaths; then run the full-argument install once first, or pass -SkipIsolation).
|
||||
1. Stop the service (unless -NoServiceControl).
|
||||
2. Move InstallRoot -> PreviousInstall, then Candidate -> InstallRoot.
|
||||
3. Copy each -Carry item (relative to InstallRoot; wildcards allowed) from PreviousInstall into the new InstallRoot.
|
||||
Folders are copied with robocopy /E /COPY:DAT (data, attributes, timestamps; not ACLs: step 4 sets those).
|
||||
4. Isolation, when recorded: Install-ScriptWorkerIsolation.ps1 -Reapply, then -Verify. If -Verify fails, the script
|
||||
stops here with the service STOPPED (fail closed): fix it, re-run -Reapply/-Verify, then start the service.
|
||||
5. Start the service (unless -NoStart, e.g. a Primary/Secondary pair window where both nodes start together).
|
||||
1. Isolation, when recorded, with the old build still serving (lmxopcua#905): Install-ScriptWorkerIsolation.ps1
|
||||
-Reapply -Scope Outside, then -Verify -Scope Outside: every step the swap cannot undo (anchor service, records,
|
||||
PE cache, denies outside InstallRoot, firewall rule, host env) and the coverage scan of the denied folders
|
||||
outside InstallRoot.
|
||||
If either fails, the script stops here: nothing is stopped or moved.
|
||||
2. Stop the service (unless -NoServiceControl). The stopped window starts here.
|
||||
3. Move InstallRoot -> PreviousInstall, then Candidate -> InstallRoot.
|
||||
4. Copy each -Carry item (relative to InstallRoot; wildcards allowed) from PreviousInstall into the new InstallRoot.
|
||||
Folders are copied with robocopy /E /COPY:DAT (data, attributes, timestamps; not ACLs: step 5 sets those).
|
||||
5. Isolation, when recorded: -Reapply -Scope InstallRoot, then -Verify -Scope InstallRoot: only the entries a swap
|
||||
drops (the ACEs on InstallRoot, script-worker\ and a denied folder inside InstallRoot such as pki\) and the
|
||||
coverage scan of InstallRoot. Steps 1 and 5 together check every install step (the firewall rule names the
|
||||
worker exe by its unchanged path; preflight checks the candidate carries that exe), so the new build is never
|
||||
started unverified. If -Verify fails, the script stops here with the service STOPPED
|
||||
(fail closed): fix it, run -Reapply/-Verify (no -Scope), then start the service.
|
||||
6. Start the service (unless -NoStart, e.g. a Primary/Secondary pair window where both nodes start together).
|
||||
Prints the stopped window (stop issued -> start issued) and the time of each step in it.
|
||||
|
||||
lmxopcua#905: the coverage scan used to run over every denied folder in both -Reapply and -Verify while the service
|
||||
was stopped, with Get-Acl per item; on vd03 (E:\ApiInstall denied) that was most of a 101 s outage. It is now about
|
||||
20x faster per item and, inside the stopped window, walks only the new InstallRoot.
|
||||
|
||||
Not covered (layout-specific; keep them in the deploy's own scripts): backups of data\ / the registry / the ConfigDb,
|
||||
restoring data\ on a rollback, migrations, health checks.
|
||||
@@ -91,12 +105,16 @@ Import-Module (Join-Path $PSScriptRoot 'ScriptWorkerIsolation.psm1') -Force
|
||||
$installer = Join-Path $PSScriptRoot 'Install-ScriptWorkerIsolation.ps1'
|
||||
|
||||
function Step([string]$Message) { Write-Host "==> $Message" }
|
||||
function Invoke-Isolation([string]$Mode) {
|
||||
function Invoke-Isolation([string]$Mode, [string]$Scope) {
|
||||
# Same PowerShell host as this script; the installer's exit code is the gate.
|
||||
$exe = if ($PSVersionTable.PSEdition -eq 'Core') { (Get-Process -Id $PID).Path } else { 'powershell.exe' }
|
||||
& $exe -NoProfile -ExecutionPolicy Bypass -File $installer "-$Mode" -WorkerServiceName $WorkerServiceName | ForEach-Object { Write-Host " $_" }
|
||||
& $exe -NoProfile -ExecutionPolicy Bypass -File $installer "-$Mode" -Scope $Scope -WorkerServiceName $WorkerServiceName | ForEach-Object { Write-Host " $_" }
|
||||
$LASTEXITCODE
|
||||
}
|
||||
# lmxopcua#905: the stopped window and what it is spent on.
|
||||
$window = [System.Diagnostics.Stopwatch]::new()
|
||||
$phases = [System.Collections.Generic.List[string]]::new()
|
||||
function Mark([string]$Name, [System.Diagnostics.Stopwatch]$Sw) { $phases.Add(('{0} {1:N1} s' -f $Name, $Sw.Elapsed.TotalSeconds)) }
|
||||
|
||||
$InstallRoot = $InstallRoot.TrimEnd('\')
|
||||
$Candidate = $Candidate.TrimEnd('\')
|
||||
@@ -109,6 +127,8 @@ $principal = [Security.Principal.WindowsPrincipal][Security.Principal.WindowsIde
|
||||
if (-not $principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)) { throw 'run elevated' }
|
||||
if (-not (Test-Path -LiteralPath $InstallRoot -PathType Container)) { throw "InstallRoot $InstallRoot not found" }
|
||||
if (-not (Test-Path -LiteralPath (Join-Path $Candidate 'script-worker') -PathType Container)) { throw "candidate $Candidate has no script-worker\ (not a Host publish?)" }
|
||||
# lmxopcua#905: the firewall rule (verified before the stop) blocks this exe by path; the swapped-in folder must carry it.
|
||||
$candidateWorkerExe = Join-Path $Candidate 'script-worker\ZB.MOM.WW.OtOpcUa.ScriptWorker.exe'
|
||||
if (Test-Path -LiteralPath $PreviousInstall) { throw "PreviousInstall $PreviousInstall already exists; pick a fresh name" }
|
||||
if ([IO.Path]::GetPathRoot($InstallRoot) -ne [IO.Path]::GetPathRoot($PreviousInstall) -or [IO.Path]::GetPathRoot($InstallRoot) -ne [IO.Path]::GetPathRoot($Candidate)) {
|
||||
throw 'InstallRoot, Candidate and PreviousInstall must be on the same volume (Move-Item renames)'
|
||||
@@ -126,14 +146,30 @@ if (-not $SkipIsolation) {
|
||||
if ($isolation -and $isolation.InstallRoot.TrimEnd('\') -ine $InstallRoot) {
|
||||
throw "the isolation record is for InstallRoot '$($isolation.InstallRoot)', not '$InstallRoot'; fix -InstallRoot or re-run the isolation install"
|
||||
}
|
||||
if ($isolation) { Write-Host "Isolation installed (record: InstallRoot $($isolation.InstallRoot)); it is re-applied and verified after the swap." }
|
||||
if ($isolation -and -not (Test-Path -LiteralPath $candidateWorkerExe -PathType Leaf)) {
|
||||
throw "candidate has no $candidateWorkerExe; the worker firewall rule blocks that path (a framework-dependent publish is not covered by it)"
|
||||
}
|
||||
if ($isolation) { Write-Host "Isolation installed (record: InstallRoot $($isolation.InstallRoot)); re-applied and verified outside InstallRoot before the stop, inside it after the swap." }
|
||||
else { Write-Host 'Isolation not installed on this node; nothing to re-apply.' }
|
||||
}
|
||||
else { Write-Warning '-SkipIsolation: the worker-isolation ACLs are NOT re-applied; a swapped folder has lost them (lmxopcua#901).' }
|
||||
|
||||
if (-not $PSCmdlet.ShouldProcess($InstallRoot, "swap in $Candidate (previous install -> $PreviousInstall)")) { return }
|
||||
|
||||
# ---- 1. Stop ---------------------------------------------------------------------------------------------------------
|
||||
# ---- 1. Isolation outside InstallRoot, old build still serving (lmxopcua#905) ---------------------------------------
|
||||
if ($isolation) {
|
||||
$sw = [System.Diagnostics.Stopwatch]::StartNew()
|
||||
Step 'worker isolation outside the install folder: -Reapply -Scope Outside (service still running)'
|
||||
$code = Invoke-Isolation 'Reapply' 'Outside'
|
||||
if ($code -ne 0) { throw "Install-ScriptWorkerIsolation.ps1 -Reapply -Scope Outside failed ($code); nothing stopped or swapped" }
|
||||
Step 'worker isolation outside the install folder: -Verify -Scope Outside (pre-stop gate)'
|
||||
$code = Invoke-Isolation 'Verify' 'Outside'
|
||||
if ($code -ne 0) { throw "Install-ScriptWorkerIsolation.ps1 -Verify -Scope Outside failed ($code); nothing stopped or swapped. Fix the [MISSING] steps (-Reapply), then re-run the swap." }
|
||||
Write-Host (' (pre-stop isolation {0:N1} s, service still running)' -f $sw.Elapsed.TotalSeconds)
|
||||
}
|
||||
|
||||
# ---- 2. Stop ---------------------------------------------------------------------------------------------------------
|
||||
$window.Start()
|
||||
if (-not $NoServiceControl) {
|
||||
Step "stop $ServiceName"
|
||||
$svc = Get-Service -Name $ServiceName
|
||||
@@ -141,9 +177,11 @@ if (-not $NoServiceControl) {
|
||||
Stop-Service -Name $ServiceName -Force
|
||||
$svc.WaitForStatus('Stopped', [TimeSpan]::FromSeconds($StopTimeoutSeconds))
|
||||
}
|
||||
Mark 'stop' $window
|
||||
}
|
||||
|
||||
# ---- 2. Swap ---------------------------------------------------------------------------------------------------------
|
||||
# ---- 3. Swap ---------------------------------------------------------------------------------------------------------
|
||||
$sw = [System.Diagnostics.Stopwatch]::StartNew()
|
||||
Step "move $InstallRoot -> $PreviousInstall"
|
||||
Move-Item -LiteralPath $InstallRoot -Destination $PreviousInstall
|
||||
Step "move $Candidate -> $InstallRoot"
|
||||
@@ -154,7 +192,10 @@ catch {
|
||||
throw
|
||||
}
|
||||
|
||||
# ---- 3. Carry --------------------------------------------------------------------------------------------------------
|
||||
Mark 'move' $sw
|
||||
|
||||
# ---- 4. Carry --------------------------------------------------------------------------------------------------------
|
||||
$sw = [System.Diagnostics.Stopwatch]::StartNew()
|
||||
foreach ($item in $Carry) {
|
||||
# Get-Item, not Get-ChildItem: a folder name must give the folder itself, not its children (wildcards still expand).
|
||||
$sources = @(Get-Item -Path (Join-Path $PreviousInstall $item) -Force -ErrorAction SilentlyContinue)
|
||||
@@ -170,17 +211,24 @@ foreach ($item in $Carry) {
|
||||
}
|
||||
}
|
||||
|
||||
# ---- 4. Isolation re-apply + verify (fail closed) --------------------------------------------------------------------
|
||||
Mark 'carry' $sw
|
||||
|
||||
# ---- 5. Isolation inside InstallRoot: re-apply + verify (fail closed) ------------------------------------------------
|
||||
if ($isolation) {
|
||||
Step 'worker isolation: -Reapply'
|
||||
$code = Invoke-Isolation 'Reapply'
|
||||
if ($code -ne 0) { throw "Install-ScriptWorkerIsolation.ps1 -Reapply failed ($code); $ServiceName left stopped" }
|
||||
Step 'worker isolation: -Verify (post-swap gate)'
|
||||
$code = Invoke-Isolation 'Verify'
|
||||
if ($code -ne 0) { throw "Install-ScriptWorkerIsolation.ps1 -Verify failed ($code); $ServiceName left stopped. Fix the [MISSING] steps, re-run -Reapply/-Verify, then start it." }
|
||||
$sw = [System.Diagnostics.Stopwatch]::StartNew()
|
||||
Step 'worker isolation in the swapped-in folder: -Reapply -Scope InstallRoot'
|
||||
$code = Invoke-Isolation 'Reapply' 'InstallRoot'
|
||||
if ($code -ne 0) { throw "Install-ScriptWorkerIsolation.ps1 -Reapply -Scope InstallRoot failed ($code); $ServiceName left stopped" }
|
||||
Step 'worker isolation in the swapped-in folder: -Verify -Scope InstallRoot (post-swap gate)'
|
||||
$code = Invoke-Isolation 'Verify' 'InstallRoot'
|
||||
if ($code -ne 0) { throw "Install-ScriptWorkerIsolation.ps1 -Verify -Scope InstallRoot failed ($code); $ServiceName left stopped. Fix the [MISSING] steps, re-run -Reapply/-Verify, then start it." }
|
||||
Mark 'isolation' $sw
|
||||
}
|
||||
|
||||
# ---- 5. Start --------------------------------------------------------------------------------------------------------
|
||||
# ---- 6. Start --------------------------------------------------------------------------------------------------------
|
||||
$window.Stop()
|
||||
$windowName = if ($NoServiceControl) { 'swap window (service control is the caller''s)' } elseif ($NoStart) { 'stop -> swapped and verified (left stopped)' } else { 'stopped window (stop issued -> start issued)' }
|
||||
Write-Host ('{0}: {1:N1} s [{2}]' -f $windowName, $window.Elapsed.TotalSeconds, ($phases -join ', '))
|
||||
if (-not $NoServiceControl -and -not $NoStart) {
|
||||
Step "start $ServiceName"
|
||||
Start-Service -Name $ServiceName
|
||||
|
||||
Reference in New Issue
Block a user