fix(install): cut the swap's stopped window to the swapped-in folder (#905)
v2-ci / unit-tests (push) Has been cancelled

Root cause: the protected-DACL coverage scan, not the ACL writes. Both -Reapply and
-Verify walked every denied folder (all of E:\ApiInstall on vd03, nested roots again)
with Get-ChildItem -Recurse + Get-Acl per item + a pipeline per item for -Skip, about
3.4 ms an item on PS 5.1, all while the service was stopped. An unchanged deny is read,
not rewritten; the re-applied install-folder deny propagates over ~1,400 files in 0.5 s.

- Find-OtOpcUaDenyCoverageGap: .NET enumeration, -Skip folders pruned, DACL-only
  GetAccessControl(Access) (about 0.15 ms an item); reparse points not followed.
- Get-OtOpcUaCoverageWalkRoots: each denied folder walked once (outermost only).
- Install-ScriptWorkerIsolation.ps1 -Scope All|InstallRoot|Outside (with -Reapply,
  -Verify, -PlanOnly). InstallRoot = the folder ACEs a swap drops; Outside = the rest,
  including the firewall rule (its query costs 3-5 s; its exe path does not change).
  The two partition the plan. Prints the scan time.
- Invoke-OtOpcUaInstallSwap.ps1: -Reapply/-Verify -Scope Outside before the stop (a
  failure stops before anything moves), -Scope InstallRoot inside the window; preflight
  checks the candidate carries the worker exe; prints the stopped window and its parts.
  -NoStart, -NoCarry and the pre-#901 refusal unchanged.

windev, vd03-like tree (17,851 items): stopped window 153.1 s -> 6.0 s (PS 5.1),
101.6 s -> 6.2 s (PS 7); the Outside part takes ~23 s with the old build serving.
Dry-run harness 101 -> 123 passed; Windows test 72 passed on PS 5.1 and PS 7.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
2026-11-13 03:51:41 -05:00
parent f817c77e02
commit 0c19f26c81
8 changed files with 475 additions and 58 deletions
+26 -5
View File
@@ -138,25 +138,46 @@ Two argument-free modes use the record:
|---|---| |---|---|
| `Install-ScriptWorkerIsolation.ps1 -Reapply` | Re-applies with the recorded settings; one `[changed]`/`[unchanged]` line per step and a `summary:` line. After a swap: the three folder ACEs `changed`, the rest `unchanged`; a second run all `unchanged`. | | `Install-ScriptWorkerIsolation.ps1 -Reapply` | Re-applies with the recorded settings; one `[changed]`/`[unchanged]` line per step and a `summary:` line. After a swap: the three folder ACEs `changed`, the rest `unchanged`; a second run all `unchanged`. |
| `Install-ScriptWorkerIsolation.ps1 -Verify` | Check only. `[ok]`/`[MISSING]` per step (anchor service, record + key ACL, PE cache ACL, `pe\`, `script-worker\`, install-folder deny, every `-DenyPaths` deny, firewall rule and the worker exe it names, the host env entries when recorded). **Exit 1** with `VERIFY FAILED: …` when anything is missing, exit 0 with `VERIFY OK`. Protected-DACL coverage gaps are WARNed, not failed. | | `Install-ScriptWorkerIsolation.ps1 -Verify` | Check only. `[ok]`/`[MISSING]` per step (anchor service, record + key ACL, PE cache ACL, `pe\`, `script-worker\`, install-folder deny, every `-DenyPaths` deny, firewall rule and the worker exe it names, the host env entries when recorded). **Exit 1** with `VERIFY FAILED: …` when anything is missing, exit 0 with `VERIFY OK`. Protected-DACL coverage gaps are WARNed, not failed. |
| `… -Reapply -Scope InstallRoot` / `-Verify -Scope InstallRoot` | (lmxopcua#905) Only what a swap can undo: the ACEs on the install folder, `script-worker\` and a denied folder inside it (vd03's `pki\`). The coverage scan walks the install folder only. |
| `… -Reapply -Scope Outside` / `-Verify -Scope Outside` | Every other step (anchor service, records, PE cache, denies outside the install folder, firewall rule, host env). The firewall rule names the worker exe by a path a swap does not change, and its query costs 3–5 s; the swap checks in its preflight that the candidate carries the exe. The scan walks the denied folders and skips the install folder. `InstallRoot` + `Outside` = the full plan. |
Both refuse to run without a full record. A node isolated by an installer older than #901 recorded only `DenyPaths`: Both refuse to run without a full record. A node isolated by an installer older than #901 recorded only `DenyPaths`:
run the install once with its full arguments (the per-box deploy record lists them) to write the record. run the install once with its full arguments (the per-box deploy record lists them) to write the record.
`scripts/install/Invoke-OtOpcUaInstallSwap.ps1` is the swap with the gate built in: preflight (candidate carries `scripts/install/Invoke-OtOpcUaInstallSwap.ps1` is the swap with the gate built in: preflight (candidate carries
`script-worker\`, `PreviousInstall` is fresh and on the same volume, isolation record present when isolation is `script-worker\`, `PreviousInstall` is fresh and on the same volume, isolation record present when isolation is
installed, record `InstallRoot` equals `-InstallRoot`) → stop → move live → previous, candidate → live → copy `-Carry` installed, record `InstallRoot` equals `-InstallRoot`) → `-Reapply -Scope Outside` + `-Verify -Scope Outside` **while
items back (default `appsettings*.json`, `pki`, `logs`; QA: `'appsettings*.json','logs'`; rollback: `-NoCarry`) → the old build still serves** (a failure stops here: nothing stopped or moved) → stop → move live → previous, candidate →
`-Reapply` → `-Verify` → start (`-NoStart` for a pair window). A failed `-Verify` leaves the service **stopped**. Backups, live → copy `-Carry` items back (default `appsettings*.json`, `pki`, `logs`; QA: `'appsettings*.json','logs'`; rollback:
`-NoCarry`) → `-Reapply -Scope InstallRoot` → `-Verify -Scope InstallRoot` → start (`-NoStart` for a pair window). The
two scoped gates together check every install step, so the new build never starts unverified; a failed post-swap
`-Verify` leaves the service **stopped**. The script prints the stopped window and its parts
(`stopped window (stop issued -> start issued): 6.0 s [stop …, move 0.0 s, carry 0.1 s, isolation 5.8 s]`). Backups,
`data\` restore, migrations and health checks stay in the deploy's own steps. Every deploy pack that swaps the folder runs `data\` restore, migrations and health checks stay in the deploy's own steps. Every deploy pack that swaps the folder runs
it (or `-Reapply` + `-Verify` right after its own swap), and `-Verify` is a post-deploy gate. it (or `-Reapply` + `-Verify` right after its own swap), and `-Verify` is a post-deploy gate.
`Test-ScriptWorkerIsolation.Windows.ps1` exercises the whole sequence (swap → `-Verify` fails → `-Reapply` → `-Verify` `Test-ScriptWorkerIsolation.Windows.ps1` exercises the whole sequence (swap → `-Verify` fails → `-Reapply` → `-Verify`
passes → `-Reapply` unchanged → the swap script) on a scratch tree. passes → `-Reapply` unchanged → the swap script → the scoped modes, the pre-stop gate, `-NoStart`, a `-NoCarry` rollback)
on a scratch tree; `-TimingFiles N` times the swap on a vd03-like tree (`-ScriptsDir` for a before/after).
**Expected stopped window (lmxopcua#905).** Before #905 the swap ran the full `-Reapply` and `-Verify` while the service
was stopped, and each of them walked every denied folder for the protected-DACL coverage scan with `Get-Acl` per item
(about 3.4 ms an item on Windows PowerShell 5.1). vd03 denies all of `E:\ApiInstall`, so that was most of its 101 s outage
on 2026-11-13 (QA: 50–60 s per node). The scan now reads only the DACL through .NET (about 20× faster per item), walks
each denied folder once (a folder nested in another denied folder is no longer walked again), and inside the stopped
window walks only the swapped-in install folder. Setting the ACEs themselves was never the cost: an unchanged deny is
read, not rewritten, and the re-applied install-folder deny propagates over about 1,400 files in about 0.5 s. The firewall
query (3–5 s a run) moved to the Outside scope. Measured on windev with a vd03-like tree (17,851 items under the denied
folder, 48 protected files): the swap's stopped window went from **153.1 s to 6.0 s** on Windows PowerShell 5.1 and from
**101.6 s to 6.2 s** on PowerShell 7 (`Test-ScriptWorkerIsolation.Windows.ps1 -TimingFiles 17000`). Of the 6 s, about
1.5 s per installer run is process start + script parse + module import. The Outside part (full scan, firewall,
anchor-service query) takes about 23 s **before** the stop, with the old build serving. Add the stop, the deploy's own
steps inside the window (a `data\` backup) and the host start-up to `/health/ready`.
## Install / Uninstall ## Install / Uninstall
- `scripts/install/Install-Services.ps1 -Roles admin,driver` — installs `OtOpcUaHost`. - `scripts/install/Install-Services.ps1 -Roles admin,driver` — installs `OtOpcUaHost`.
- `scripts/install/Install-ScriptWorkerIsolation.ps1` — the ScriptWorker identity, ACLs and firewall rule (above); only for `Scripting:Runner` ≠ `InProcess`. - `scripts/install/Install-ScriptWorkerIsolation.ps1` — the ScriptWorker identity, ACLs and firewall rule (above); only for `Scripting:Runner` ≠ `InProcess`.
- `scripts/install/Invoke-OtOpcUaInstallSwap.ps1` — upgrade/rollback folder swap with the isolation `-Reapply` + `-Verify` gate built in (lmxopcua#901, above). - `scripts/install/Invoke-OtOpcUaInstallSwap.ps1` — upgrade/rollback folder swap with the isolation `-Reapply` + `-Verify` gate built in (lmxopcua#901), scoped so only the swapped-in folder is handled while the service is stopped (lmxopcua#905, above).
- `scripts/install/Uninstall-Services.ps1` — stops + removes the host service. (The historian backend is the external HistorianGateway — not installed/removed by these scripts.) - `scripts/install/Uninstall-Services.ps1` — stops + removes the host service. (The historian backend is the external HistorianGateway — not installed/removed by these scripts.)
## Logging ## Logging
@@ -331,3 +331,13 @@ rollback.
| vd03 | `fea7de0a` about **10 s** (stop 2 s); `6c4d2990` about **15 s** (stop 6 s), stop → `/health/ready` | **about 20–40 s.** The data robocopy and the in-script `-Reapply`/`-Verify` now run while the service is stopped. The step-2 record write happens with the service up. | | vd03 | `fea7de0a` about **10 s** (stop 2 s); `6c4d2990` about **15 s** (stop 6 s), stop → `/health/ready` | **about 20–40 s.** The data robocopy and the in-script `-Reapply`/`-Verify` now run while the service is stopped. The step-2 record write happens with the service up. |
| QA pair | `fea7de0a` window **about 2 min** pair outage (16:48:13 → 16:50:14), of which about 70 s was the isolation install + uninstall drill. The first node took 39 s to ready because it waits for its peer; the second took 8 s. | **about 1–1.5 min** pair outage: stops about 1–2 s each, backup + swap + `-Reapply`/`-Verify` per node, then about 40 s + 8 s to ready. One-node restarts (the OTel record) took up to 32 s to stop a Secondary with its peer up. Stopping both together avoids that graceful-leave wait. | | QA pair | `fea7de0a` window **about 2 min** pair outage (16:48:13 → 16:50:14), of which about 70 s was the isolation install + uninstall drill. The first node took 39 s to ready because it waits for its peer; the second took 8 s. | **about 1–1.5 min** pair outage: stops about 1–2 s each, backup + swap + `-Reapply`/`-Verify` per node, then about 40 s + 8 s to ready. One-node restarts (the OTel record) took up to 32 s to stop a Secondary with its peer up. Stopping both together avoids that graceful-leave wait. |
| QA failover gate | 30 s graceful leave, then the new Primary at 250 on its first sample | same | | QA failover gate | 30 s graceful leave, then the new Primary at 250 on its first sample | same |
**After lmxopcua#905 (for the next pack that uses `Invoke-OtOpcUaInstallSwap.ps1`).** The 101 s (vd03) and 50–60 s per
QA node were the protected-DACL coverage scan (`Get-Acl` per item over every denied folder, in both `-Reapply` and
`-Verify`, while stopped), not the ACL writes. The swap now runs `-Reapply`/`-Verify -Scope Outside` (the full scan
included) **before** the stop and only `-Scope InstallRoot` inside the window. windev, a vd03-like tree of 17,851 items
under the denied folder: stopped window **153.1 s → 6.0 s** (Windows PowerShell 5.1) and **101.6 s → 6.2 s** (PowerShell 7);
the pre-stop part takes about 23 s with the old build still serving. Estimate for the next vd03 swap: stop (2–6 s) +
the `data\` backup + about 6 s + start-up to `/health/ready` (about 8–10 s), i.e. **about 20–30 s**. QA: about 10 s of
swap per node inside the pair outage instead of 50–60 s, and the Secondary-first order (deviation 3) is no longer needed
to hide it. The script prints `stopped window (…): N s [stop, move, carry, isolation]`; record that line.
@@ -130,6 +130,8 @@ old Primary first.
1. vd03 downtime about 101 s against an estimate of 20–40 s, because the coverage walk runs inside `-Reapply`/`-Verify` 1. vd03 downtime about 101 s against an estimate of 20–40 s, because the coverage walk runs inside `-Reapply`/`-Verify`
while the service is stopped. QA swaps took about 50–60 s per node for the same reason. while the service is stopped. QA swaps took about 50–60 s per node for the same reason.
→ lmxopcua#905: fixed (scan about 20× faster per item, walked once, and only the swapped-in folder inside the
window; windev 17.9k items: stopped window 153 s → 6 s on PS 5.1, 102 s → 6 s on PS 7).
2. #903: the edit-and-revert cannot return the chip to "In sync", because `RowVersion` is part of the revision hash (see 2. #903: the edit-and-revert cannot return the chip to "In sync", because `RowVersion` is part of the revision hash (see
above). Draft content is restored; the chip shows drift until the next deploy. above). Draft content is restored; the chip shows drift until the next deploy.
3. QA order: the Secondary was stopped and swapped while the Primary still served, then the Primary. Both started 3. QA order: the Secondary was stopped and swapped while the Primary still served, then the Primary. Both started
@@ -50,12 +50,15 @@
.\Install-ScriptWorkerIsolation.ps1 -Reapply # the recorded settings; prints changed/unchanged per step .\Install-ScriptWorkerIsolation.ps1 -Reapply # the recorded settings; prints changed/unchanged per step
.\Install-ScriptWorkerIsolation.ps1 -Verify # check only; exit 1 and a [MISSING] line per gap .\Install-ScriptWorkerIsolation.ps1 -Verify # check only; exit 1 and a [MISSING] line per gap
Invoke-OtOpcUaInstallSwap.ps1 does both after its swap and does not start the service when -Verify fails. The host Invoke-OtOpcUaInstallSwap.ps1 does both around its swap and does not start the service when -Verify fails: -Scope
must be restarted for -ConfigureHostService to take effect. Outside while the old build still serves, -Scope InstallRoot after the swap (lmxopcua#905). The host must be restarted
for -ConfigureHostService to take effect.
After an apply it walks every denied folder and WARNs about any file or folder whose DACL is protected (does not After an apply it walks every denied folder and WARNs about any file or folder whose DACL is protected (does not
inherit) and has no deny of its own; the inherited deny does not reach those. Without -DenyPaths it WARNs when the inherit) and has no deny of its own; the inherited deny does not reach those. Without -DenyPaths it WARNs when the
folder around InstallRoot has a pki, data, logs, backup, config or secrets sibling. folder around InstallRoot has a pki, data, logs, backup, config or secrets sibling. The walk covers each denied
folder once (a folder inside another denied folder is walked with it) and prints its time; about 0.15 ms an item
(lmxopcua#905: before, Get-Acl per item made it about 3.4 ms, 59 s for 17.5k items).
.PARAMETER InstallRoot .PARAMETER InstallRoot
The host install folder (holds OtOpcUa.Host.exe and script-worker\). The host install folder (holds OtOpcUa.Host.exe and script-worker\).
@@ -94,6 +97,18 @@
when anything is missing (0 when all is in place). Uses the recorded settings, or -InstallRoot/-HostServiceAccount/... when anything is missing (0 when all is in place). Uses the recorded settings, or -InstallRoot/-HostServiceAccount/...
when given. Protected-DACL coverage gaps are WARNed, as after an apply, and do not fail the check. when given. Protected-DACL coverage gaps are WARNed, as after an apply, and do not fail the check.
.PARAMETER Scope
lmxopcua#905, with -Reapply, -Verify or -PlanOnly. Which install steps run:
All every step (default).
InstallRoot only what a folder swap can undo: the folder ACEs on InstallRoot and below (WorkerFolderAcl,
InstallRootDeny, a -DenyPaths folder inside InstallRoot such as vd03's pki\). The coverage scan walks
InstallRoot only.
Outside every other step: the anchor service, the records, the PE cache, the denies outside InstallRoot, the
firewall rule (its path to the worker exe does not change with a swap; the query costs seconds), the
host service env. The coverage scan walks the denied folders and skips InstallRoot.
InstallRoot and Outside together are exactly All. Invoke-OtOpcUaInstallSwap.ps1 runs Outside before it stops the
service and InstallRoot inside the stopped window.
.PARAMETER PlanOnly .PARAMETER PlanOnly
Print the plan and return it as objects without touching (or even reading) the machine. Runs on any OS; used by the Print the plan and return it as objects without touching (or even reading) the machine. Runs on any OS; used by the
dry-run harness scripts/install/tests/Test-ScriptWorkerIsolation.ps1. -HostServiceAccount must then be a SID. dry-run harness scripts/install/tests/Test-ScriptWorkerIsolation.ps1. -HostServiceAccount must then be a SID.
@@ -131,7 +146,9 @@ param(
[switch]$RemovePeCache, [switch]$RemovePeCache,
[switch]$Reapply, [switch]$Reapply,
[switch]$Verify, [switch]$Verify,
[switch]$PlanOnly [switch]$PlanOnly,
# lmxopcua#905: with -Reapply / -Verify / -PlanOnly only. Which steps run (see .PARAMETER Scope).
[ValidateSet('All', 'InstallRoot', 'Outside')] [string]$Scope = 'All'
) )
Set-StrictMode -Version Latest Set-StrictMode -Version Latest
@@ -140,6 +157,7 @@ Import-Module (Join-Path $PSScriptRoot 'ScriptWorkerIsolation.psm1') -Force
# lmxopcua#901: -Reapply / -Verify / -Uninstall without arguments run from what the last install recorded. # lmxopcua#901: -Reapply / -Verify / -Uninstall without arguments run from what the last install recorded.
if (@($Reapply, $Verify, $Uninstall, $PlanOnly | Where-Object { $_ }).Count -gt 1) { throw '-Reapply, -Verify, -Uninstall and -PlanOnly are exclusive' } if (@($Reapply, $Verify, $Uninstall, $PlanOnly | Where-Object { $_ }).Count -gt 1) { throw '-Reapply, -Verify, -Uninstall and -PlanOnly are exclusive' }
if ($Scope -ne 'All' -and -not ($Reapply -or $Verify -or $PlanOnly)) { throw '-Scope works with -Reapply, -Verify or -PlanOnly only (an install or -Uninstall always covers every step)' }
if ($Reapply) { if ($Reapply) {
$given = @('InstallRoot', 'HostServiceAccount', 'PeCacheDir', 'HostServiceName', 'DenyPaths', 'ConfigureHostService', 'RemovePeCache' | $given = @('InstallRoot', 'HostServiceAccount', 'PeCacheDir', 'HostServiceName', 'DenyPaths', 'ConfigureHostService', 'RemovePeCache' |
Where-Object { $PSBoundParameters.ContainsKey($_) }) Where-Object { $PSBoundParameters.ContainsKey($_) })
@@ -188,11 +206,25 @@ $plan = Get-ScriptWorkerIsolationPlan -InstallRoot $InstallRoot -HostSid $hostSi
-WorkerServiceName $WorkerServiceName -HostServiceName $HostServiceName -DenyPaths $DenyPaths -RecordedDenyPaths $recorded ` -WorkerServiceName $WorkerServiceName -HostServiceName $HostServiceName -DenyPaths $DenyPaths -RecordedDenyPaths $recorded `
-ConfigureHostService:$ConfigureHostService -Uninstall:$Uninstall -RemovePeCache:$RemovePeCache -ConfigureHostService:$ConfigureHostService -Uninstall:$Uninstall -RemovePeCache:$RemovePeCache
$denied = @($plan | Where-Object { $_.Id -eq 'DenyPathRecord' -and $_.Data.ContainsKey('Paths') } | ForEach-Object { $_.Data.Paths } | Where-Object { $_ }) $denied = @($plan | Where-Object { $_.Id -eq 'DenyPathRecord' -and $_.Data.ContainsKey('Paths') } | ForEach-Object { $_.Data.Paths } | Where-Object { $_ })
# lmxopcua#905: a swap runs -Scope Outside while the old build still serves and -Scope InstallRoot after the swap, so only
# the steps the swap can undo run while the service is stopped. The two scopes partition the plan.
$plan = @(Select-ScriptWorkerIsolationPlanScope -Plan $plan -InstallRoot $InstallRoot -Scope $Scope)
$deniedInScope = @($denied | Where-Object { $Scope -eq 'All' -or ((Test-OtOpcUaPathWithin -Path $_ -Ancestor $InstallRoot) -eq ($Scope -eq 'InstallRoot')) })
function Invoke-CoverageScan {
# The protected-DACL coverage scan (lmxopcua#890), over the outermost folders of this scope only (lmxopcua#905).
$walk = Get-OtOpcUaCoverageWalkRoots -InstallRoot $InstallRoot -DeniedPaths $denied -Scope $Scope
$skip = @((Join-Path $InstallRoot.TrimEnd('\') 'script-worker'), $PeCacheDir.TrimEnd('\')) + @($walk.Skip)
$sw = [System.Diagnostics.Stopwatch]::StartNew()
$found = @(foreach ($p in $walk.Roots) { Find-OtOpcUaDenyCoverageGap -Path $p -WorkerSid $workerSid -Skip $skip })
$sw.Stop()
[pscustomobject]@{ Gaps = @($found | Sort-Object -Unique); Roots = @($walk.Roots); Seconds = $sw.Elapsed.TotalSeconds }
}
$workerSid = Get-OtOpcUaServiceSid -Name $WorkerServiceName $workerSid = Get-OtOpcUaServiceSid -Name $WorkerServiceName
Write-Host ("ScriptWorker isolation {0}: worker identity NT SERVICE\{1} ({2}), host account {3} ({4}){5}" -f ` Write-Host ("ScriptWorker isolation {0}: worker identity NT SERVICE\{1} ({2}), host account {3} ({4}){5}{6}" -f `
$(if ($Uninstall) { 'ROLLBACK' } elseif ($Verify) { 'VERIFY' } elseif ($Reapply) { 'REAPPLY' } else { 'install' }), $WorkerServiceName, $(if ($Uninstall) { 'ROLLBACK' } elseif ($Verify) { 'VERIFY' } elseif ($Reapply) { 'REAPPLY' } else { 'install' }), $WorkerServiceName,
$workerSid, $HostServiceAccount, $hostSid, $(if ($fromRecord) { ', settings from the record' } else { '' })) $workerSid, $HostServiceAccount, $hostSid, $(if ($fromRecord) { ', settings from the record' } else { '' }),
$(if ($Scope -ne 'All') { ", scope $Scope ($($plan.Count) step(s))" } else { '' }))
if ($fromRecord) { if ($fromRecord) {
Write-Host (" InstallRoot {0}; PeCacheDir {1}; DenyPaths {2}; ConfigureHostService {3}" -f $InstallRoot, $PeCacheDir, Write-Host (" InstallRoot {0}; PeCacheDir {1}; DenyPaths {2}; ConfigureHostService {3}" -f $InstallRoot, $PeCacheDir,
$(if (@($DenyPaths).Count) { $DenyPaths -join ', ' } else { 'none' }), [bool]$ConfigureHostService) $(if (@($DenyPaths).Count) { $DenyPaths -join ', ' } else { 'none' }), [bool]$ConfigureHostService)
@@ -212,17 +244,19 @@ if ($Verify) {
Write-Host (" [{0,-7}] {1,-22} {2} - {3}" -f $(if ($r.Ok) { 'ok' } else { 'MISSING' }), $step.Id, $step.Target, $r.Detail) Write-Host (" [{0,-7}] {1,-22} {2} - {3}" -f $(if ($r.Ok) { 'ok' } else { 'MISSING' }), $step.Id, $step.Target, $r.Detail)
[pscustomobject]@{ Id = $step.Id; Target = $step.Target; Ok = [bool]$r.Ok; Detail = $r.Detail } [pscustomobject]@{ Id = $step.Id; Target = $step.Target; Ok = [bool]$r.Ok; Detail = $r.Detail }
} }
$workerFolder = Join-Path $InstallRoot.TrimEnd('\') 'script-worker' $scan = Invoke-CoverageScan
$gaps = @(foreach ($p in @($InstallRoot.TrimEnd('\')) + $denied) { Find-OtOpcUaDenyCoverageGap -Path $p -WorkerSid $workerSid -Skip @($workerFolder, $PeCacheDir.TrimEnd('\')) }) foreach ($g in $scan.Gaps) { Write-Warning "$g does not inherit the worker deny (protected DACL, no deny of its own); check it grants nothing to Everyone, BUILTIN\Users or RESTRICTED." }
foreach ($g in @($gaps | Sort-Object -Unique)) { Write-Warning "$g does not inherit the worker deny (protected DACL, no deny of its own); check it grants nothing to Everyone, BUILTIN\Users or RESTRICTED." } Write-Host (" coverage scan: {0} folder(s) walked ({1}), {2} protected item(s) the deny does not reach, {3:N1} s" -f `
$scan.Roots.Count, $(if ($scan.Roots.Count) { $scan.Roots -join '; ' } else { 'none' }), $scan.Gaps.Count, $scan.Seconds)
$scopeNote = if ($Scope -ne 'All') { " (scope $Scope)" } else { '' }
$missing = @($checks | Where-Object { -not $_.Ok }) $missing = @($checks | Where-Object { -not $_.Ok })
Write-Host '' Write-Host ''
if ($missing.Count) { if ($missing.Count) {
Write-Host ("VERIFY FAILED: {0} of {1} step(s) not in place: {2}. Run -Reapply (as administrator) and verify again." -f ` Write-Host ("VERIFY FAILED{3}: {0} of {1} step(s) not in place: {2}. Run -Reapply (as administrator) and verify again." -f `
$missing.Count, @($checks).Count, (($missing | ForEach-Object { "$($_.Id) $($_.Target)" }) -join '; ')) $missing.Count, @($checks).Count, (($missing | ForEach-Object { "$($_.Id) $($_.Target)" }) -join '; '), $scopeNote)
exit 1 exit 1
} }
Write-Host ("VERIFY OK: {0}/{0} step(s) in place." -f @($checks).Count) Write-Host ("VERIFY OK{1}: {0}/{0} step(s) in place." -f @($checks).Count, $scopeNote)
exit 0 exit 0
} }
if (-not $Uninstall -and -not (Test-Path -LiteralPath (Join-Path $InstallRoot 'script-worker'))) { if (-not $Uninstall -and -not (Test-Path -LiteralPath (Join-Path $InstallRoot 'script-worker'))) {
@@ -230,7 +264,7 @@ if (-not $Uninstall -and -not (Test-Path -LiteralPath (Join-Path $InstallRoot 's
} }
if (-not $Uninstall) { if (-not $Uninstall) {
# Checked up front so a typo stops the run before any step changes the machine. # Checked up front so a typo stops the run before any step changes the machine.
$missing = @($denied | Where-Object { -not (Test-Path -LiteralPath $_ -PathType Container) }) $missing = @($deniedInScope | Where-Object { -not (Test-Path -LiteralPath $_ -PathType Container) })
if ($missing.Count) { throw "DenyPaths folder(s) not found: $($missing -join '; ')" } if ($missing.Count) { throw "DenyPaths folder(s) not found: $($missing -join '; ')" }
if ($denied.Count -eq 0) { if ($denied.Count -eq 0) {
$parent = Split-Path -Parent $InstallRoot.TrimEnd('\') $parent = Split-Path -Parent $InstallRoot.TrimEnd('\')
@@ -262,13 +296,13 @@ if (-not $WhatIfPreference) {
if (-not $WhatIfPreference -and -not $Uninstall) { if (-not $WhatIfPreference -and -not $Uninstall) {
# The inherited deny stops at a protected DACL; list every such item that has no deny of its own (lmxopcua#890). # The inherited deny stops at a protected DACL; list every such item that has no deny of its own (lmxopcua#890).
$workerFolder = Join-Path $InstallRoot.TrimEnd('\') 'script-worker' $scan = Invoke-CoverageScan
$gaps = @(foreach ($p in @($InstallRoot.TrimEnd('\')) + $denied) { Find-OtOpcUaDenyCoverageGap -Path $p -WorkerSid $workerSid -Skip @($workerFolder, $PeCacheDir.TrimEnd('\')) }) $gaps = $scan.Gaps
$gaps = @($gaps | Sort-Object -Unique)
foreach ($g in $gaps) { foreach ($g in $gaps) {
Write-Warning "$g does not inherit the worker deny (protected DACL, no deny of its own). Check its ACL grants nothing to Everyone, BUILTIN\Users or RESTRICTED, or add it to -DenyPaths." Write-Warning "$g does not inherit the worker deny (protected DACL, no deny of its own). Check its ACL grants nothing to Everyone, BUILTIN\Users or RESTRICTED, or add it to -DenyPaths."
} }
Write-Host (" coverage: {0} denied folder(s); {1} protected item(s) the deny does not reach" -f (1 + $denied.Count), $gaps.Count) Write-Host (" coverage: {0} denied folder(s); {1} folder(s) walked; {2} protected item(s) the deny does not reach; {3:N1} s" -f `
(1 + $denied.Count), $scan.Roots.Count, $gaps.Count, $scan.Seconds)
} }
if (-not $WhatIfPreference) { if (-not $WhatIfPreference) {
+69 -21
View File
@@ -11,13 +11,27 @@
0. Preflight (nothing changed yet): the candidate exists and carries script-worker\; PreviousInstall does not exist 0. Preflight (nothing changed yet): the candidate exists and carries script-worker\; PreviousInstall does not exist
and is on the same volume; when isolation is installed, its settings are recorded (an install by an installer and is on the same volume; when isolation is installed, its settings are recorded (an install by an installer
older than #901 recorded only DenyPaths; then run the full-argument install once first, or pass -SkipIsolation). older than #901 recorded only DenyPaths; then run the full-argument install once first, or pass -SkipIsolation).
1. Stop the service (unless -NoServiceControl). 1. Isolation, when recorded, with the old build still serving (lmxopcua#905): Install-ScriptWorkerIsolation.ps1
2. Move InstallRoot -> PreviousInstall, then Candidate -> InstallRoot. -Reapply -Scope Outside, then -Verify -Scope Outside: every step the swap cannot undo (anchor service, records,
3. Copy each -Carry item (relative to InstallRoot; wildcards allowed) from PreviousInstall into the new InstallRoot. PE cache, denies outside InstallRoot, firewall rule, host env) and the coverage scan of the denied folders
Folders are copied with robocopy /E /COPY:DAT (data, attributes, timestamps; not ACLs: step 4 sets those). outside InstallRoot.
4. Isolation, when recorded: Install-ScriptWorkerIsolation.ps1 -Reapply, then -Verify. If -Verify fails, the script If either fails, the script stops here: nothing is stopped or moved.
stops here with the service STOPPED (fail closed): fix it, re-run -Reapply/-Verify, then start the service. 2. Stop the service (unless -NoServiceControl). The stopped window starts here.
5. Start the service (unless -NoStart, e.g. a Primary/Secondary pair window where both nodes start together). 3. Move InstallRoot -> PreviousInstall, then Candidate -> InstallRoot.
4. Copy each -Carry item (relative to InstallRoot; wildcards allowed) from PreviousInstall into the new InstallRoot.
Folders are copied with robocopy /E /COPY:DAT (data, attributes, timestamps; not ACLs: step 5 sets those).
5. Isolation, when recorded: -Reapply -Scope InstallRoot, then -Verify -Scope InstallRoot: only the entries a swap
drops (the ACEs on InstallRoot, script-worker\ and a denied folder inside InstallRoot such as pki\) and the
coverage scan of InstallRoot. Steps 1 and 5 together check every install step (the firewall rule names the
worker exe by its unchanged path; preflight checks the candidate carries that exe), so the new build is never
started unverified. If -Verify fails, the script stops here with the service STOPPED
(fail closed): fix it, run -Reapply/-Verify (no -Scope), then start the service.
6. Start the service (unless -NoStart, e.g. a Primary/Secondary pair window where both nodes start together).
Prints the stopped window (stop issued -> start issued) and the time of each step in it.
lmxopcua#905: the coverage scan used to run over every denied folder in both -Reapply and -Verify while the service
was stopped, with Get-Acl per item; on vd03 (E:\ApiInstall denied) that was most of a 101 s outage. It is now about
20x faster per item and, inside the stopped window, walks only the new InstallRoot.
Not covered (layout-specific; keep them in the deploy's own scripts): backups of data\ / the registry / the ConfigDb, Not covered (layout-specific; keep them in the deploy's own scripts): backups of data\ / the registry / the ConfigDb,
restoring data\ on a rollback, migrations, health checks. restoring data\ on a rollback, migrations, health checks.
@@ -91,12 +105,16 @@ Import-Module (Join-Path $PSScriptRoot 'ScriptWorkerIsolation.psm1') -Force
$installer = Join-Path $PSScriptRoot 'Install-ScriptWorkerIsolation.ps1' $installer = Join-Path $PSScriptRoot 'Install-ScriptWorkerIsolation.ps1'
function Step([string]$Message) { Write-Host "==> $Message" } function Step([string]$Message) { Write-Host "==> $Message" }
function Invoke-Isolation([string]$Mode) { function Invoke-Isolation([string]$Mode, [string]$Scope) {
# Same PowerShell host as this script; the installer's exit code is the gate. # Same PowerShell host as this script; the installer's exit code is the gate.
$exe = if ($PSVersionTable.PSEdition -eq 'Core') { (Get-Process -Id $PID).Path } else { 'powershell.exe' } $exe = if ($PSVersionTable.PSEdition -eq 'Core') { (Get-Process -Id $PID).Path } else { 'powershell.exe' }
& $exe -NoProfile -ExecutionPolicy Bypass -File $installer "-$Mode" -WorkerServiceName $WorkerServiceName | ForEach-Object { Write-Host " $_" } & $exe -NoProfile -ExecutionPolicy Bypass -File $installer "-$Mode" -Scope $Scope -WorkerServiceName $WorkerServiceName | ForEach-Object { Write-Host " $_" }
$LASTEXITCODE $LASTEXITCODE
} }
# lmxopcua#905: the stopped window and what it is spent on.
$window = [System.Diagnostics.Stopwatch]::new()
$phases = [System.Collections.Generic.List[string]]::new()
function Mark([string]$Name, [System.Diagnostics.Stopwatch]$Sw) { $phases.Add(('{0} {1:N1} s' -f $Name, $Sw.Elapsed.TotalSeconds)) }
$InstallRoot = $InstallRoot.TrimEnd('\') $InstallRoot = $InstallRoot.TrimEnd('\')
$Candidate = $Candidate.TrimEnd('\') $Candidate = $Candidate.TrimEnd('\')
@@ -109,6 +127,8 @@ $principal = [Security.Principal.WindowsPrincipal][Security.Principal.WindowsIde
if (-not $principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)) { throw 'run elevated' } if (-not $principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)) { throw 'run elevated' }
if (-not (Test-Path -LiteralPath $InstallRoot -PathType Container)) { throw "InstallRoot $InstallRoot not found" } if (-not (Test-Path -LiteralPath $InstallRoot -PathType Container)) { throw "InstallRoot $InstallRoot not found" }
if (-not (Test-Path -LiteralPath (Join-Path $Candidate 'script-worker') -PathType Container)) { throw "candidate $Candidate has no script-worker\ (not a Host publish?)" } if (-not (Test-Path -LiteralPath (Join-Path $Candidate 'script-worker') -PathType Container)) { throw "candidate $Candidate has no script-worker\ (not a Host publish?)" }
# lmxopcua#905: the firewall rule (verified before the stop) blocks this exe by path; the swapped-in folder must carry it.
$candidateWorkerExe = Join-Path $Candidate 'script-worker\ZB.MOM.WW.OtOpcUa.ScriptWorker.exe'
if (Test-Path -LiteralPath $PreviousInstall) { throw "PreviousInstall $PreviousInstall already exists; pick a fresh name" } if (Test-Path -LiteralPath $PreviousInstall) { throw "PreviousInstall $PreviousInstall already exists; pick a fresh name" }
if ([IO.Path]::GetPathRoot($InstallRoot) -ne [IO.Path]::GetPathRoot($PreviousInstall) -or [IO.Path]::GetPathRoot($InstallRoot) -ne [IO.Path]::GetPathRoot($Candidate)) { if ([IO.Path]::GetPathRoot($InstallRoot) -ne [IO.Path]::GetPathRoot($PreviousInstall) -or [IO.Path]::GetPathRoot($InstallRoot) -ne [IO.Path]::GetPathRoot($Candidate)) {
throw 'InstallRoot, Candidate and PreviousInstall must be on the same volume (Move-Item renames)' throw 'InstallRoot, Candidate and PreviousInstall must be on the same volume (Move-Item renames)'
@@ -126,14 +146,30 @@ if (-not $SkipIsolation) {
if ($isolation -and $isolation.InstallRoot.TrimEnd('\') -ine $InstallRoot) { if ($isolation -and $isolation.InstallRoot.TrimEnd('\') -ine $InstallRoot) {
throw "the isolation record is for InstallRoot '$($isolation.InstallRoot)', not '$InstallRoot'; fix -InstallRoot or re-run the isolation install" throw "the isolation record is for InstallRoot '$($isolation.InstallRoot)', not '$InstallRoot'; fix -InstallRoot or re-run the isolation install"
} }
if ($isolation) { Write-Host "Isolation installed (record: InstallRoot $($isolation.InstallRoot)); it is re-applied and verified after the swap." } if ($isolation -and -not (Test-Path -LiteralPath $candidateWorkerExe -PathType Leaf)) {
throw "candidate has no $candidateWorkerExe; the worker firewall rule blocks that path (a framework-dependent publish is not covered by it)"
}
if ($isolation) { Write-Host "Isolation installed (record: InstallRoot $($isolation.InstallRoot)); re-applied and verified outside InstallRoot before the stop, inside it after the swap." }
else { Write-Host 'Isolation not installed on this node; nothing to re-apply.' } else { Write-Host 'Isolation not installed on this node; nothing to re-apply.' }
} }
else { Write-Warning '-SkipIsolation: the worker-isolation ACLs are NOT re-applied; a swapped folder has lost them (lmxopcua#901).' } else { Write-Warning '-SkipIsolation: the worker-isolation ACLs are NOT re-applied; a swapped folder has lost them (lmxopcua#901).' }
if (-not $PSCmdlet.ShouldProcess($InstallRoot, "swap in $Candidate (previous install -> $PreviousInstall)")) { return } if (-not $PSCmdlet.ShouldProcess($InstallRoot, "swap in $Candidate (previous install -> $PreviousInstall)")) { return }
# ---- 1. Stop --------------------------------------------------------------------------------------------------------- # ---- 1. Isolation outside InstallRoot, old build still serving (lmxopcua#905) ---------------------------------------
if ($isolation) {
$sw = [System.Diagnostics.Stopwatch]::StartNew()
Step 'worker isolation outside the install folder: -Reapply -Scope Outside (service still running)'
$code = Invoke-Isolation 'Reapply' 'Outside'
if ($code -ne 0) { throw "Install-ScriptWorkerIsolation.ps1 -Reapply -Scope Outside failed ($code); nothing stopped or swapped" }
Step 'worker isolation outside the install folder: -Verify -Scope Outside (pre-stop gate)'
$code = Invoke-Isolation 'Verify' 'Outside'
if ($code -ne 0) { throw "Install-ScriptWorkerIsolation.ps1 -Verify -Scope Outside failed ($code); nothing stopped or swapped. Fix the [MISSING] steps (-Reapply), then re-run the swap." }
Write-Host (' (pre-stop isolation {0:N1} s, service still running)' -f $sw.Elapsed.TotalSeconds)
}
# ---- 2. Stop ---------------------------------------------------------------------------------------------------------
$window.Start()
if (-not $NoServiceControl) { if (-not $NoServiceControl) {
Step "stop $ServiceName" Step "stop $ServiceName"
$svc = Get-Service -Name $ServiceName $svc = Get-Service -Name $ServiceName
@@ -141,9 +177,11 @@ if (-not $NoServiceControl) {
Stop-Service -Name $ServiceName -Force Stop-Service -Name $ServiceName -Force
$svc.WaitForStatus('Stopped', [TimeSpan]::FromSeconds($StopTimeoutSeconds)) $svc.WaitForStatus('Stopped', [TimeSpan]::FromSeconds($StopTimeoutSeconds))
} }
Mark 'stop' $window
} }
# ---- 2. Swap --------------------------------------------------------------------------------------------------------- # ---- 3. Swap ---------------------------------------------------------------------------------------------------------
$sw = [System.Diagnostics.Stopwatch]::StartNew()
Step "move $InstallRoot -> $PreviousInstall" Step "move $InstallRoot -> $PreviousInstall"
Move-Item -LiteralPath $InstallRoot -Destination $PreviousInstall Move-Item -LiteralPath $InstallRoot -Destination $PreviousInstall
Step "move $Candidate -> $InstallRoot" Step "move $Candidate -> $InstallRoot"
@@ -154,7 +192,10 @@ catch {
throw throw
} }
# ---- 3. Carry -------------------------------------------------------------------------------------------------------- Mark 'move' $sw
# ---- 4. Carry --------------------------------------------------------------------------------------------------------
$sw = [System.Diagnostics.Stopwatch]::StartNew()
foreach ($item in $Carry) { foreach ($item in $Carry) {
# Get-Item, not Get-ChildItem: a folder name must give the folder itself, not its children (wildcards still expand). # Get-Item, not Get-ChildItem: a folder name must give the folder itself, not its children (wildcards still expand).
$sources = @(Get-Item -Path (Join-Path $PreviousInstall $item) -Force -ErrorAction SilentlyContinue) $sources = @(Get-Item -Path (Join-Path $PreviousInstall $item) -Force -ErrorAction SilentlyContinue)
@@ -170,17 +211,24 @@ foreach ($item in $Carry) {
} }
} }
# ---- 4. Isolation re-apply + verify (fail closed) -------------------------------------------------------------------- Mark 'carry' $sw
# ---- 5. Isolation inside InstallRoot: re-apply + verify (fail closed) ------------------------------------------------
if ($isolation) { if ($isolation) {
Step 'worker isolation: -Reapply' $sw = [System.Diagnostics.Stopwatch]::StartNew()
$code = Invoke-Isolation 'Reapply' Step 'worker isolation in the swapped-in folder: -Reapply -Scope InstallRoot'
if ($code -ne 0) { throw "Install-ScriptWorkerIsolation.ps1 -Reapply failed ($code); $ServiceName left stopped" } $code = Invoke-Isolation 'Reapply' 'InstallRoot'
Step 'worker isolation: -Verify (post-swap gate)' if ($code -ne 0) { throw "Install-ScriptWorkerIsolation.ps1 -Reapply -Scope InstallRoot failed ($code); $ServiceName left stopped" }
$code = Invoke-Isolation 'Verify' Step 'worker isolation in the swapped-in folder: -Verify -Scope InstallRoot (post-swap gate)'
if ($code -ne 0) { throw "Install-ScriptWorkerIsolation.ps1 -Verify failed ($code); $ServiceName left stopped. Fix the [MISSING] steps, re-run -Reapply/-Verify, then start it." } $code = Invoke-Isolation 'Verify' 'InstallRoot'
if ($code -ne 0) { throw "Install-ScriptWorkerIsolation.ps1 -Verify -Scope InstallRoot failed ($code); $ServiceName left stopped. Fix the [MISSING] steps, re-run -Reapply/-Verify, then start it." }
Mark 'isolation' $sw
} }
# ---- 5. Start -------------------------------------------------------------------------------------------------------- # ---- 6. Start --------------------------------------------------------------------------------------------------------
$window.Stop()
$windowName = if ($NoServiceControl) { 'swap window (service control is the caller''s)' } elseif ($NoStart) { 'stop -> swapped and verified (left stopped)' } else { 'stopped window (stop issued -> start issued)' }
Write-Host ('{0}: {1:N1} s [{2}]' -f $windowName, $window.Elapsed.TotalSeconds, ($phases -join ', '))
if (-not $NoServiceControl -and -not $NoStart) { if (-not $NoServiceControl -and -not $NoStart) {
Step "start $ServiceName" Step "start $ServiceName"
Start-Service -Name $ServiceName Start-Service -Name $ServiceName
+97 -7
View File
@@ -588,22 +588,112 @@ function Find-OtOpcUaDenyCoverageGap {
After the denies are applied (lmxopcua#890): the files and folders under $Path that the inherited deny does NOT After the denies are applied (lmxopcua#890): the files and folders under $Path that the inherited deny does NOT
reach because their DACL is protected (inheritance switched off, e.g. a store that sets its own ACL on a private reach because their DACL is protected (inheritance switched off, e.g. a store that sets its own ACL on a private
key) and that carry no explicit deny of their own. Each one is reported as a WARN; the operator decides. key) and that carry no explicit deny of their own. Each one is reported as a WARN; the operator decides.
$Skip (script-worker\) is not walked. $Skip folders (script-worker\, the PE cache, ...) are not walked.
.NOTES
lmxopcua#905: this walk was the whole cost of a swap's stopped window (vd03 denies all of E:\ApiInstall). It used
Get-ChildItem -Recurse + Get-Acl per item + a pipeline per item for $Skip: about 3.4 ms an item on Windows
PowerShell 5.1 (59 s for 17.5k items on windev). It now enumerates with .NET, prunes $Skip folders instead of
filtering every descendant, and reads only the DACL (GetAccessControl(Access): no owner/group lookup, no PS
wrapping), about 0.15 ms an item. Reparse-point folders (junctions, symlinks) are not followed: the target keeps
its own DACL, and following them can loop.
#> #>
param([Parameter(Mandatory)] [string]$Path, [Parameter(Mandatory)] [string]$WorkerSid, [string[]]$Skip = @()) param([Parameter(Mandatory)] [string]$Path, [Parameter(Mandatory)] [string]$WorkerSid, [string[]]$Skip = @())
if (-not (Test-Path -LiteralPath $Path)) { return } $start = $Path.TrimEnd('\')
if (-not [System.IO.Directory]::Exists($start)) { return }
$skipList = @($Skip | Where-Object { $_ } | ForEach-Object { $_.TrimEnd('\') })
foreach ($s in $skipList) { if (Test-OtOpcUaPathWithin -Path $start -Ancestor $s) { return } }
$identity = [System.Security.Principal.SecurityIdentifier]::new($WorkerSid) $identity = [System.Security.Principal.SecurityIdentifier]::new($WorkerSid)
foreach ($item in @(Get-ChildItem -LiteralPath $Path -Recurse -Force -ErrorAction SilentlyContinue)) { $core = $PSVersionTable.PSEdition -eq 'Core' # #800: FileSystemInfo.GetAccessControl does not exist on .NET (PS 7)
if (@($Skip | Where-Object { $_ -and (Test-OtOpcUaPathWithin -Path $item.FullName -Ancestor $_) }).Count) { continue } $sections = [System.Security.AccessControl.AccessControlSections]::Access
try { $acl = Get-Acl -LiteralPath $item.FullName } catch { continue } $dirAttr = [System.IO.FileAttributes]::Directory
$reparse = [System.IO.FileAttributes]::ReparsePoint
$stack = [System.Collections.Generic.Stack[System.IO.DirectoryInfo]]::new()
$stack.Push([System.IO.DirectoryInfo]::new($start))
while ($stack.Count -gt 0) {
$dir = $stack.Pop()
$entries = $null
try { $entries = @($dir.EnumerateFileSystemInfos()) } catch { $entries = @() } # unreadable folder: skip, as before
foreach ($item in $entries) {
if (($item.Attributes -band $dirAttr) -ne 0) {
$skipped = $false
foreach ($s in $skipList) { if ($item.FullName -ieq $s) { $skipped = $true; break } }
if ($skipped) { continue }
if (($item.Attributes -band $reparse) -eq 0) { $stack.Push($item) }
}
$acl = $null
try {
$acl = if ($core) { [System.IO.FileSystemAclExtensions]::GetAccessControl($item, $sections) }
else { $item.GetAccessControl($sections) }
}
catch { continue }
if (-not $acl.AreAccessRulesProtected) { continue } if (-not $acl.AreAccessRulesProtected) { continue }
$explicitDeny = @($acl.GetAccessRules($true, $false, [System.Security.Principal.SecurityIdentifier]) | $explicitDeny = @($acl.GetAccessRules($true, $false, [System.Security.Principal.SecurityIdentifier]) |
Where-Object { $_.IdentityReference -eq $identity -and $_.AccessControlType -eq 'Deny' }) Where-Object { $_.IdentityReference -eq $identity -and $_.AccessControlType -eq 'Deny' })
if ($explicitDeny.Count -eq 0) { $item.FullName } if ($explicitDeny.Count -eq 0) { $item.FullName }
} }
} }
}
function Test-OtOpcUaStepInInstallRoot {
# lmxopcua#905: true when a plan step's state lives in the install folder, so a folder swap can lose it: a folder ACE
# on InstallRoot or below (WorkerFolderAcl, InstallRootDeny, a -DenyPaths folder inside it such as vd03's pki\).
# Everything else (the anchor service, the records, the PE cache, folders outside, the host env, the firewall rule) is
# untouched by a swap. The firewall rule names the worker exe by path, and that path does not change; its check is
# Outside because Get-NetFirewallRule + its application filter cost about 3-5 s a run (windev), which the stopped
# window should not pay. Invoke-OtOpcUaInstallSwap.ps1 checks that the candidate carries the worker exe before the stop.
# Pure.
param([Parameter(Mandatory)] $Step, [Parameter(Mandatory)] [string]$InstallRoot)
$d = $Step.Data
if ($d -is [System.Collections.IDictionary] -and $d.Contains('Path') -and $d['Path']) {
return [bool](Test-OtOpcUaPathWithin -Path ([string]$d['Path']) -Ancestor $InstallRoot)
}
$false
}
function Select-ScriptWorkerIsolationPlanScope {
<#
.SYNOPSIS
lmxopcua#905: the part of a plan a -Scope covers. 'InstallRoot' = the steps a folder swap can undo (see
Test-OtOpcUaStepInInstallRoot); 'Outside' = all the others; 'All' = the plan unchanged. InstallRoot and Outside
partition the plan, so an Outside run before the service stops plus an InstallRoot run after the swap cover every
step. Pure.
#>
param([Parameter(Mandatory)] [AllowEmptyCollection()] [object[]]$Plan, [Parameter(Mandatory)] [string]$InstallRoot,
[ValidateSet('All', 'InstallRoot', 'Outside')] [string]$Scope = 'All')
if ($Scope -eq 'All') { return $Plan }
$wantIn = $Scope -eq 'InstallRoot'
foreach ($step in $Plan) { if ((Test-OtOpcUaStepInInstallRoot -Step $step -InstallRoot $InstallRoot) -eq $wantIn) { $step } }
}
function Get-OtOpcUaCoverageWalkRoots {
<#
.SYNOPSIS
lmxopcua#905: which folders the protected-DACL coverage scan walks for a -Scope, and what it must skip besides
script-worker\ and the PE cache. Only the outermost folders are walked: a denied folder inside another one
(InstallRoot or pki\ inside E:\ApiInstall on vd03) was walked a second (and third) time before. 'InstallRoot'
walks InstallRoot alone; 'Outside' walks the denied folders not inside InstallRoot and skips InstallRoot; 'All'
walks both. Pure. Returns [pscustomobject]@{ Roots; Skip }.
#>
param([Parameter(Mandatory)] [string]$InstallRoot, [string[]]$DeniedPaths = @(),
[ValidateSet('All', 'InstallRoot', 'Outside')] [string]$Scope = 'All')
$root = $InstallRoot.TrimEnd('\')
$denied = @($DeniedPaths | Where-Object { $_ } | ForEach-Object { $_.TrimEnd('\') })
$candidates = switch ($Scope) {
'InstallRoot' { @($root) }
'Outside' { @($denied | Where-Object { -not (Test-OtOpcUaPathWithin -Path $_ -Ancestor $root) }) }
default { @(@($root) + $denied) }
}
$outer = [System.Collections.Generic.List[string]]::new()
foreach ($c in $candidates) {
$nested = @($candidates | Where-Object { $_ -ine $c -and (Test-OtOpcUaPathWithin -Path $c -Ancestor $_) }).Count -gt 0
if (-not $nested -and -not ($outer | Where-Object { $_ -ieq $c })) { $outer.Add($c) }
}
[pscustomobject]@{ Roots = $outer.ToArray(); Skip = $(if ($Scope -eq 'Outside') { @($root) } else { @() }) }
}
Export-ModuleMember -Function Get-OtOpcUaServiceSid, Get-OtOpcUaPeCacheSddl, Get-OtOpcUaPeEntriesAce, Export-ModuleMember -Function Get-OtOpcUaServiceSid, Get-OtOpcUaPeCacheSddl, Get-OtOpcUaPeEntriesAce,
Get-OtOpcUaScriptWorkerFolderAce, Get-OtOpcUaInstallRootDenyAce, Resolve-OtOpcUaDenyPaths, Get-ScriptWorkerIsolationPlan, Get-OtOpcUaScriptWorkerFolderAce, Get-OtOpcUaInstallRootDenyAce, Resolve-OtOpcUaDenyPaths, Get-ScriptWorkerIsolationPlan,
Invoke-ScriptWorkerIsolationStep, Get-OtOpcUaRecordedDenyPaths, Find-OtOpcUaDenyCoverageGap, Get-OtOpcUaIsolationSettings, Invoke-ScriptWorkerIsolationStep, Get-OtOpcUaRecordedDenyPaths, Find-OtOpcUaDenyCoverageGap, Get-OtOpcUaIsolationSettings, Test-OtOpcUaPathWithin,
Test-ScriptWorkerIsolationStep, Test-OtOpcUaRecordKeyAcl Test-ScriptWorkerIsolationStep, Test-OtOpcUaRecordKeyAcl, Test-OtOpcUaStepInInstallRoot, Select-ScriptWorkerIsolationPlanScope,
Get-OtOpcUaCoverageWalkRoots
@@ -2,7 +2,8 @@
.SYNOPSIS .SYNOPSIS
Windows-executed test of Install-ScriptWorkerIsolation.ps1 (lmxopcua#883, #890, #901): apply -> re-apply -> verify -> shorter Windows-executed test of Install-ScriptWorkerIsolation.ps1 (lmxopcua#883, #890, #901): apply -> re-apply -> verify -> shorter
-DenyPaths -> -Verify -> folder swap -> -Verify fails -> -Reapply -> -Verify passes -> -Reapply unchanged -> -DenyPaths -> -Verify -> folder swap -> -Verify fails -> -Reapply -> -Verify passes -> -Reapply unchanged ->
Invoke-OtOpcUaInstallSwap.ps1 (re-apply + verify built in; refuses a pre-#901 record) -> uninstall (from the record, Invoke-OtOpcUaInstallSwap.ps1 (re-apply + verify built in; refuses a pre-#901 record) -> -Scope InstallRoot/Outside
after a swap, the swap's pre-stop gate, -NoStart, a -NoCarry rollback (lmxopcua#905) -> uninstall (from the record,
without -DenyPaths) -> uninstall. ACL level only; the restricted-token proof that without -DenyPaths) -> uninstall. ACL level only; the restricted-token proof that
the worker cannot read the denied folders is InstallerLayoutIsolationTests (ScriptWorker.Tests, OTOPCUA_ISOLATION_INSTALLER_E2E=1). the worker cannot read the denied folders is InstallerLayoutIsolationTests (ScriptWorker.Tests, OTOPCUA_ISOLATION_INSTALLER_E2E=1).
The dry-run harness (Test-ScriptWorkerIsolation.ps1) only asserts plan data; it cannot see host-specific failures such as The dry-run harness (Test-ScriptWorkerIsolation.ps1) only asserts plan data; it cannot see host-specific failures such as
@@ -12,6 +13,13 @@
pwsh -NoProfile -File scripts\install\tests\Test-ScriptWorkerIsolation.Windows.ps1 pwsh -NoProfile -File scripts\install\tests\Test-ScriptWorkerIsolation.Windows.ps1
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\install\tests\Test-ScriptWorkerIsolation.Windows.ps1 powershell -NoProfile -ExecutionPolicy Bypass -File scripts\install\tests\Test-ScriptWorkerIsolation.Windows.ps1
lmxopcua#905 timing: -TimingFiles N also builds a vd03-like tree (a denied ApiInstall folder holding the install, a
candidate and N more files in other app folders, 48 protected-DACL files), installs the isolation on it and times
Invoke-OtOpcUaInstallSwap.ps1. -TimingOnly skips the functional part. -ScriptsDir runs the timing against another
copy of scripts\install (e.g. the scripts before #905) for a before/after:
pwsh -NoProfile -File ...\Test-ScriptWorkerIsolation.Windows.ps1 -TimingOnly -TimingFiles 17000 -ScriptsDir C:\old\install
.DESCRIPTION .DESCRIPTION
Needs an elevated Windows session. It works in a scratch folder under $env:TEMP with a scratch worker service name and a Needs an elevated Windows session. It works in a scratch folder under $env:TEMP with a scratch worker service name and a
scratch stand-in host service (a disabled service that is never started), and uses -HostServiceAccount 'NT AUTHORITY\SYSTEM', scratch stand-in host service (a disabled service that is never started), and uses -HostServiceAccount 'NT AUTHORITY\SYSTEM',
@@ -22,7 +30,7 @@
#> #>
[CmdletBinding()] [CmdletBinding()]
param([string]$WorkerServiceName = 'OtOpcUaIsoTestWorker', [string]$HostServiceName = 'OtOpcUaIsoTestHost', [switch]$LocalHostAccount, param([string]$WorkerServiceName = 'OtOpcUaIsoTestWorker', [string]$HostServiceName = 'OtOpcUaIsoTestHost', [switch]$LocalHostAccount,
[string]$LocalHostAccountName = 'otopcuaIsoTestAcct') [string]$LocalHostAccountName = 'otopcuaIsoTestAcct', [int]$TimingFiles = 0, [switch]$TimingOnly, [string]$ScriptsDir)
Set-StrictMode -Version Latest Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop' $ErrorActionPreference = 'Stop'
@@ -32,6 +40,8 @@ $principal = [Security.Principal.WindowsPrincipal][Security.Principal.WindowsIde
if (-not $principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)) { Write-Host 'run elevated'; exit 2 } if (-not $principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)) { Write-Host 'run elevated'; exit 2 }
$installer = Join-Path (Split-Path -Parent $PSScriptRoot) 'Install-ScriptWorkerIsolation.ps1' $installer = Join-Path (Split-Path -Parent $PSScriptRoot) 'Install-ScriptWorkerIsolation.ps1'
if (-not $ScriptsDir) { $ScriptsDir = Split-Path -Parent $PSScriptRoot }
$exe = if ($PSVersionTable.PSEdition -eq 'Core') { (Get-Process -Id $PID).Path } else { 'powershell.exe' }
Import-Module (Join-Path (Split-Path -Parent $PSScriptRoot) 'ScriptWorkerIsolation.psm1') -Force Import-Module (Join-Path (Split-Path -Parent $PSScriptRoot) 'ScriptWorkerIsolation.psm1') -Force
$ruleName = 'OtOpcUa-ScriptWorker-BlockOutbound' $ruleName = 'OtOpcUa-ScriptWorker-BlockOutbound'
if (Get-NetFirewallRule -Name $ruleName -ErrorAction SilentlyContinue) { Write-Host "refusing: firewall rule $ruleName already exists (real isolation installed on this node)"; exit 2 } if (Get-NetFirewallRule -Name $ruleName -ErrorAction SilentlyContinue) { Write-Host "refusing: firewall rule $ruleName already exists (real isolation installed on this node)"; exit 2 }
@@ -54,12 +64,12 @@ function Invoke-Installer([string[]]$Extra) {
$steps = @([regex]::Matches($out, '\[(?<o>[a-z]+)\s*\]\s+(?<id>[A-Za-z]+)\s') | ForEach-Object { [pscustomobject]@{ Outcome = $_.Groups['o'].Value; Id = $_.Groups['id'].Value } }) $steps = @([regex]::Matches($out, '\[(?<o>[a-z]+)\s*\]\s+(?<id>[A-Za-z]+)\s') | ForEach-Object { [pscustomobject]@{ Outcome = $_.Groups['o'].Value; Id = $_.Groups['id'].Value } })
[pscustomobject]@{ Exit = $LASTEXITCODE; Output = $out; Steps = $steps } [pscustomobject]@{ Exit = $LASTEXITCODE; Output = $out; Steps = $steps }
} }
function Invoke-Mode([string]$Mode) { function Invoke-Mode([string]$Mode, [string[]]$Extra = @()) {
# lmxopcua#901: -Reapply / -Verify with NO settings argument, only the scratch worker name (the record selector). # lmxopcua#901: -Reapply / -Verify with NO settings argument, only the scratch worker name (the record selector).
$exe = if ($PSVersionTable.PSEdition -eq 'Core') { (Get-Process -Id $PID).Path } else { 'powershell.exe' } $exe = if ($PSVersionTable.PSEdition -eq 'Core') { (Get-Process -Id $PID).Path } else { 'powershell.exe' }
# Continue: an expected failure writes to stderr, which Stop would turn into a terminating error here. Whitespace is # Continue: an expected failure writes to stderr, which Stop would turn into a terminating error here. Whitespace is
# collapsed (and pwsh's '|' margins dropped) because both hosts wrap the error text at the console width. # collapsed (and pwsh's '|' margins dropped) because both hosts wrap the error text at the console width.
$out = & { $ErrorActionPreference = 'Continue'; & $exe -NoProfile -ExecutionPolicy Bypass -File $installer "-$Mode" -WorkerServiceName $WorkerServiceName 2>&1 | Out-String } $out = & { $ErrorActionPreference = 'Continue'; & $exe -NoProfile -ExecutionPolicy Bypass -File $installer "-$Mode" -WorkerServiceName $WorkerServiceName @Extra 2>&1 | Out-String }
$steps = @([regex]::Matches($out, '\[(?<o>[a-z]+)\s*\]\s+(?<id>[A-Za-z]+)\s+(?<t>\S+)') | ForEach-Object { [pscustomobject]@{ Outcome = $_.Groups['o'].Value; Id = $_.Groups['id'].Value; Target = $_.Groups['t'].Value } }) $steps = @([regex]::Matches($out, '\[(?<o>[a-z]+)\s*\]\s+(?<id>[A-Za-z]+)\s+(?<t>\S+)') | ForEach-Object { [pscustomobject]@{ Outcome = $_.Groups['o'].Value; Id = $_.Groups['id'].Value; Target = $_.Groups['t'].Value } })
$code = $LASTEXITCODE $code = $LASTEXITCODE
$missing = @([regex]::Matches($out, '\[MISSING\]\s+(?<id>[A-Za-z]+)\s+(?<t>\S+)') | ForEach-Object { "$($_.Groups['id'].Value) $($_.Groups['t'].Value)" }) $missing = @([regex]::Matches($out, '\[MISSING\]\s+(?<id>[A-Za-z]+)\s+(?<t>\S+)') | ForEach-Object { "$($_.Groups['id'].Value) $($_.Groups['t'].Value)" })
@@ -100,6 +110,7 @@ try {
Set-ItemProperty -Path $hostKey -Name Environment -Type MultiString -Value ([string[]]@('Keep__Me=1')) Set-ItemProperty -Path $hostKey -Name Environment -Type MultiString -Value ([string[]]@('Keep__Me=1'))
Write-Host "Host: $($PSVersionTable.PSEdition) $($PSVersionTable.PSVersion); scratch $scratch; host account $script:hostAccount" Write-Host "Host: $($PSVersionTable.PSEdition) $($PSVersionTable.PSVersion); scratch $scratch; host account $script:hostAccount"
if (-not $TimingOnly) {
Write-Host 'Dry run' Write-Host 'Dry run'
$dry = Invoke-Installer @('-WhatIf', '-DenyPaths', ($deny -join ',')) $dry = Invoke-Installer @('-WhatIf', '-DenyPaths', ($deny -join ','))
Assert ($dry.Exit -eq 0) "-WhatIf exits 0 ($($dry.Output))" Assert ($dry.Exit -eq 0) "-WhatIf exits 0 ($($dry.Output))"
@@ -223,6 +234,95 @@ try {
Assert ($r4.Exit -ne 0 -and $r4.Flat -match 'full arguments') "-Reapply without a full record refuses and says how to fix it ($($r4.Exit))" Assert ($r4.Exit -ne 0 -and $r4.Flat -match 'full arguments') "-Reapply without a full record refuses and says how to fix it ($($r4.Exit))"
Set-ItemProperty -Path $recordKey -Name InstallRoot -Value $saved Set-ItemProperty -Path $recordKey -Name InstallRoot -Value $saved
Write-Host 'Scoped -Reapply / -Verify after a raw swap (lmxopcua#905)'
$cand4 = Join-Path $scratch 'install.new4'
New-Item -ItemType Directory -Path (Join-Path $cand4 'script-worker') -Force | Out-Null
Set-Content -LiteralPath (Join-Path $cand4 'script-worker\ZB.MOM.WW.OtOpcUa.ScriptWorker.exe') -Value 'stub v4'
$prev4 = Join-Path $scratch 'install.bak.4'
Move-Item -LiteralPath $root -Destination $prev4
Move-Item -LiteralPath $cand4 -Destination $root
& robocopy (Join-Path $prev4 'pki') (Join-Path $root 'pki') /E /COPY:DAT /R:0 /W:0 /NFL /NDL /NJH /NJS /NP | Out-Null
$vo = Invoke-Mode 'Verify' @('-Scope', 'Outside'); Show 'Verify -Scope Outside (after swap)' $vo
Assert ($vo.Exit -eq 0 -and $vo.Output -match 'VERIFY OK \(scope Outside\): 9/9') "a swap leaves the 9 Outside steps intact ($($vo.Output))"
$vi = Invoke-Mode 'Verify' @('-Scope', 'InstallRoot'); Show 'Verify -Scope InstallRoot (after swap)' $vi
Assert ($vi.Exit -eq 1 -and (@($vi.Missing | Sort-Object) -join '|') -eq (@($expectMissing | Sort-Object) -join '|') -and ($vi.Steps.Count + $vi.Missing.Count) -eq 3) `
"the InstallRoot scope checks 3 steps and finds exactly the three dropped ACEs ($($vi.Missing -join ' | '))"
$ri = Invoke-Mode 'Reapply' @('-Scope', 'InstallRoot'); Show 'Reapply -Scope InstallRoot' $ri
$riChanged = @($ri.Steps | Where-Object Outcome -eq 'changed' | ForEach-Object { "$($_.Id) $($_.Target)" })
Assert ($ri.Exit -eq 0 -and $ri.Steps.Count -eq 3 -and (@($riChanged | Sort-Object) -join '|') -eq (@($expectMissing | Sort-Object) -join '|')) `
"scoped reapply runs 3 steps and changes exactly the three ($($riChanged -join ' | '))"
$covLine = [regex]::Match($ri.Flat, 'coverage: \d+ denied folder.s.; \d+ folder.s. walked').Value
Assert ($ri.Flat -match 'coverage: 4 denied folder\(s\); 1 folder\(s\) walked') "scoped reapply walks InstallRoot only ($covLine)"
$vi2 = Invoke-Mode 'Verify' @('-Scope', 'InstallRoot'); $vAll = Invoke-Mode 'Verify'; Show 'Verify (full, after scoped Reapply)' $vAll
Assert ($vi2.Exit -eq 0 -and $vAll.Exit -eq 0 -and $vAll.Output -match 'VERIFY OK: 12/12') 'after the scoped reapply both the scoped and the full verify pass'
$scanLine = [regex]::Match($vAll.Flat, 'coverage scan: \d+ folder.s. walked').Value
Assert ($vAll.Flat -match 'coverage scan: 1 folder\(s\) walked') "the full verify walks the outermost denied folder once ($scanLine)"
Write-Host 'Negative control: each scope sees only its own entries'
& icacls $cache /grant '*S-1-5-32-545:(OI)(CI)(RX)' /Q | Out-Null # an Outside entry broken
$vi3 = Invoke-Mode 'Verify' @('-Scope', 'InstallRoot'); $vo3 = Invoke-Mode 'Verify' @('-Scope', 'Outside'); Show 'Verify -Scope Outside (cache broken)' $vo3
Assert ($vi3.Exit -eq 0 -and $vo3.Exit -eq 1 -and ($vo3.Missing -join '|') -eq "PeCacheDir $cache") "the cache ACL is Outside's to catch, not InstallRoot's ($($vo3.Missing -join ' | '))"
$ro3 = Invoke-Mode 'Reapply' @('-Scope', 'Outside')
Assert ($ro3.Exit -eq 0 -and (@($ro3.Steps | Where-Object Outcome -eq 'changed' | ForEach-Object Id) -join ',') -eq 'PeCacheDir' -and $ro3.Steps.Count -eq 9) 'Reapply -Scope Outside repairs it (9 steps, 1 changed)'
Write-Host 'Coverage scan: protected items found once, script-worker\ skipped, scope-limited'
$prot = Join-Path $root 'pki\own.key'; $protSkip = Join-Path $root 'script-worker\private.key'
foreach ($f in $prot, $protSkip) { Set-Content -LiteralPath $f -Value 'k'; & icacls $f /inheritance:r /grant:r '*S-1-5-18:(F)' '*S-1-5-32-544:(F)' /Q | Out-Null }
$squash = { param($t) $t -replace '[\s|]+', '' }
$vc = Invoke-Mode 'Verify' @('-Scope', 'InstallRoot'); $vco = Invoke-Mode 'Verify' @('-Scope', 'Outside'); $vca = Invoke-Mode 'Verify'
$hit = (& $squash $prot) + 'doesnotinherit'
Assert ($vc.Exit -eq 0 -and (& $squash $vc.Output).Contains($hit)) 'InstallRoot scope warns the protected pki\ file (a warning, not a failure)'
Assert (-not (& $squash $vc.Output).Contains((& $squash $protSkip))) 'script-worker\ is not walked'
Assert (-not (& $squash $vco.Output).Contains($hit)) 'the Outside scope skips InstallRoot'
Assert (([regex]::Matches((& $squash $vca.Output), [regex]::Escape($hit))).Count -eq 1) 'the full verify reports it exactly once (pki\ is inside two denied folders)'
foreach ($f in $prot, $protSkip) { & icacls $f /reset /Q | Out-Null; Remove-Item -LiteralPath $f -Force }
Write-Host 'Invoke-OtOpcUaInstallSwap.ps1 refuses a candidate without the worker exe the firewall rule names (lmxopcua#905)'
$cand6 = Join-Path $scratch 'install.new6'
New-Item -ItemType Directory -Path (Join-Path $cand6 'script-worker') -Force | Out-Null
$sw6 = & { $ErrorActionPreference = 'Continue'; & $exe -NoProfile -ExecutionPolicy Bypass -File $swapScript -InstallRoot $root -Candidate $cand6 `
-PreviousInstall (Join-Path $scratch 'install.bak.6') -NoServiceControl -WorkerServiceName $WorkerServiceName 2>&1 | Out-String }
$sw6Exit = $LASTEXITCODE
Assert ($sw6Exit -ne 0 -and ($sw6 -replace '[\s|]+', '') -match 'candidatehasno' -and (Test-Path $cand6) -and -not (Test-Path (Join-Path $scratch 'install.bak.6')) -and
$sw6 -notmatch 'Scope Outside') "refused in preflight, before the pre-stop isolation; nothing moved ($sw6)"
Remove-Item -LiteralPath $cand6 -Recurse -Force
Write-Host 'Invoke-OtOpcUaInstallSwap.ps1: the pre-stop gate stops before anything moves'
$dataDir = Join-Path $scratch 'data'
& icacls $dataDir /reset /Q | Out-Null; Remove-Item -LiteralPath $dataDir -Force # an Outside deny folder gone: -Reapply -Scope Outside refuses
$cand5 = Join-Path $scratch 'install.new5'
New-Item -ItemType Directory -Path (Join-Path $cand5 'script-worker') -Force | Out-Null
Set-Content -LiteralPath (Join-Path $cand5 'script-worker\ZB.MOM.WW.OtOpcUa.ScriptWorker.exe') -Value 'stub v5'
$prev5 = Join-Path $scratch 'install.bak.5'
$sw3 = & { $ErrorActionPreference = 'Continue'; & $exe -NoProfile -ExecutionPolicy Bypass -File $swapScript -InstallRoot $root -Candidate $cand5 `
-PreviousInstall $prev5 -Carry 'appsettings*.json,pki' -ServiceName $HostServiceName -WorkerServiceName $WorkerServiceName 2>&1 | Out-String }
$sw3Exit = $LASTEXITCODE
Write-Host (" {0,-34} exit {1}; {2}" -f 'swap with an Outside gap', $sw3Exit, $(if ((& $squash $sw3) -match 'ScopeOutsidefailed') { 'refused before the stop' } else { 'NOT refused' }))
Assert ($sw3Exit -ne 0 -and (& $squash $sw3) -match 'ScopeOutsidefailed.*nothingstoppedorswapped' -and (Test-Path $cand5) -and -not (Test-Path $prev5) -and
(Get-Content -LiteralPath $workerExe) -eq 'stub v4' -and $sw3 -notmatch '==> stop') "refused before the stop, nothing moved ($sw3)"
New-Item -ItemType Directory -Path $dataDir -Force | Out-Null
Write-Host 'Invoke-OtOpcUaInstallSwap.ps1 -NoStart with service control (the stand-in host service is disabled: a start would fail)'
$sw4 = & { $ErrorActionPreference = 'Continue'; & $exe -NoProfile -ExecutionPolicy Bypass -File $swapScript -InstallRoot $root -Candidate $cand5 `
-PreviousInstall $prev5 -Carry 'appsettings*.json,pki' -ServiceName $HostServiceName -NoStart -WorkerServiceName $WorkerServiceName 2>&1 | Out-String }
$sw4Exit = $LASTEXITCODE
$w4 = [regex]::Match($sw4, 'stop -> swapped and verified \(left stopped\): [^\r\n]*').Value
Write-Host (" {0,-34} exit {1}; {2}" -f 'swap -NoStart', $sw4Exit, $w4)
Assert ($sw4Exit -eq 0 -and $sw4 -match 'Swap complete' -and $w4 -and $sw4 -notmatch '==> start' -and (Get-Service $HostServiceName).Status -eq 'Stopped') "-NoStart: swapped, verified, not started ($sw4)"
Assert ($sw4 -match 'pre-stop isolation' -and $sw4 -match 'VERIFY OK \(scope Outside\)' -and $sw4 -match 'VERIFY OK \(scope InstallRoot\)') 'both scoped gates ran (Outside before the stop, InstallRoot after the swap)'
Assert ($sw4.IndexOf('VERIFY OK (scope Outside)') -lt $sw4.IndexOf('==> stop') -and $sw4.IndexOf('==> stop') -lt $sw4.IndexOf('-Reapply -Scope InstallRoot')) 'Outside gate before the stop, InstallRoot after it'
Assert ((Get-Content -LiteralPath $workerExe) -eq 'stub v5' -and @(Get-Explicit $root $workerSid).Count -eq 1 -and @(Get-Explicit (Join-Path $root 'pki') $workerSid).Count -eq 1) 'v5 live, its ACEs in place'
Write-Host 'Invoke-OtOpcUaInstallSwap.ps1 -NoCarry: the rollback swaps the previous install back'
$failed5 = Join-Path $scratch 'install.failed.5'
$sw5 = & { $ErrorActionPreference = 'Continue'; & $exe -NoProfile -ExecutionPolicy Bypass -File $swapScript -InstallRoot $root -Candidate $prev5 `
-PreviousInstall $failed5 -NoCarry -NoServiceControl -WorkerServiceName $WorkerServiceName 2>&1 | Out-String }
$sw5Exit = $LASTEXITCODE
Write-Host (" {0,-34} exit {1}; {2}" -f 'swap -NoCarry (rollback)', $sw5Exit, [regex]::Match($sw5, 'swap window[^\r\n]*').Value)
Assert ($sw5Exit -eq 0 -and $sw5 -notmatch 'carried' -and (Get-Content -LiteralPath $workerExe) -eq 'stub v4' -and $sw5 -match 'VERIFY OK \(scope InstallRoot\)') "rollback: v4 back, nothing carried, gate passed ($sw5)"
$vEnd = Invoke-Mode 'Verify'
Assert ($vEnd.Exit -eq 0) 'full verify passes after the rollback'
Write-Host 'Uninstall WITHOUT -DenyPaths (the record drives it), twice' Write-Host 'Uninstall WITHOUT -DenyPaths (the record drives it), twice'
$u1 = Invoke-Installer @('-Uninstall') $u1 = Invoke-Installer @('-Uninstall')
Assert ($u1.Exit -eq 0) "uninstall exits 0 ($($u1.Output))" Assert ($u1.Exit -eq 0) "uninstall exits 0 ($($u1.Output))"
@@ -240,6 +340,53 @@ try {
Assert ($u2.Exit -eq 0) "second uninstall exits 0 ($($u2.Output))" Assert ($u2.Exit -eq 0) "second uninstall exits 0 ($($u2.Output))"
Assert ($u2.Steps.Count -eq 7 -and @($u2.Steps | Where-Object Outcome -ne 'unchanged').Count -eq 0) "second uninstall: 7 steps, all unchanged ($(($u2.Steps | ForEach-Object { "$($_.Id)=$($_.Outcome)" }) -join ','))" Assert ($u2.Steps.Count -eq 7 -and @($u2.Steps | Where-Object Outcome -ne 'unchanged').Count -eq 0) "second uninstall: 7 steps, all unchanged ($(($u2.Steps | ForEach-Object { "$($_.Id)=$($_.Outcome)" }) -join ','))"
} }
if ($TimingFiles -gt 0) {
# lmxopcua#905: a vd03-like layout. ApiInstall is denied as a whole and holds the install folder, the candidate and
# other app folders; pki\ is inside the install; 48 protected-DACL files (vd03 reported 48).
Write-Host "Swap timing (lmxopcua#905): scripts from $ScriptsDir"
$tInstaller = Join-Path $ScriptsDir 'Install-ScriptWorkerIsolation.ps1'
$tSwap = Join-Path $ScriptsDir 'Invoke-OtOpcUaInstallSwap.ps1'
$api = Join-Path $scratch 'ApiInstall'; $tRoot = Join-Path $api 'OtOpcUa'; $tCand = Join-Path $api 'OtOpcUa.new'
$tData = Join-Path $api 'OtOpcUa-data'; $tKek = Join-Path $scratch 'kek'; $tCache = Join-Path $scratch 'pe-cache-t'
function New-DummyApp([string]$Dir, [int]$N, [string]$Stub) {
for ($d = 0; $d -lt 60; $d++) { [IO.Directory]::CreateDirectory("$Dir\sub$d") | Out-Null }
for ($i = 0; $i -lt $N; $i++) { [IO.File]::WriteAllText("$Dir\sub$($i % 60)\f$i.dll", 'x') }
if ($Stub) { [IO.Directory]::CreateDirectory("$Dir\script-worker") | Out-Null; [IO.File]::WriteAllText("$Dir\script-worker\ZB.MOM.WW.OtOpcUa.ScriptWorker.exe", $Stub) }
}
New-DummyApp $tRoot 1400 'live'
foreach ($d in "$tRoot\pki\own\private", "$tRoot\logs", $tData, $tKek) { [IO.Directory]::CreateDirectory($d) | Out-Null }
[IO.File]::WriteAllText("$tRoot\appsettings.json", '{}')
New-DummyApp $tCand 1400 'candidate'
$left = [Math]::Max(0, $TimingFiles - 2800); $n = 0
while ($left -gt 0) { $n++; $take = [Math]::Min(1400, $left); New-DummyApp (Join-Path $api "app$n") $take ''; $left -= $take }
$protDir = if ($n) { Join-Path $api 'app1\sub0' } else { $tData }
for ($p = 0; $p -lt 48; $p++) { $f = "$protDir\prot$p.key"; [IO.File]::WriteAllText($f, 'k'); & icacls $f /inheritance:r /grant:r '*S-1-5-18:(F)' '*S-1-5-32-544:(F)' /Q | Out-Null }
$items = @([IO.DirectoryInfo]::new($api).EnumerateFileSystemInfos('*', [IO.SearchOption]::AllDirectories)).Count
$tDeny = "$api,$tRoot\pki,$tData,$tKek"
$ti = [Diagnostics.Stopwatch]::StartNew()
$tiOut = & { $ErrorActionPreference = 'Continue'; & $exe -NoProfile -ExecutionPolicy Bypass -File $tInstaller -InstallRoot $tRoot -HostServiceAccount $script:hostAccount `
-PeCacheDir $tCache -WorkerServiceName $WorkerServiceName -HostServiceName $HostServiceName -ConfigureHostService -DenyPaths $tDeny 2>&1 | Out-String }
$tiExit = $LASTEXITCODE; $ti.Stop()
Assert ($tiExit -eq 0) "timing: isolation install exits 0 ($tiOut)"
$ts = [Diagnostics.Stopwatch]::StartNew()
$tsOut = & { $ErrorActionPreference = 'Continue'; & $exe -NoProfile -ExecutionPolicy Bypass -File $tSwap -InstallRoot $tRoot -Candidate $tCand `
-PreviousInstall (Join-Path $api 'OtOpcUa.bak.t') -NoServiceControl -WorkerServiceName $WorkerServiceName 2>&1 | Out-String }
$tsExit = $LASTEXITCODE; $ts.Stop()
Assert ($tsExit -eq 0 -and $tsOut -match 'Swap complete' -and [IO.File]::ReadAllText("$tRoot\script-worker\ZB.MOM.WW.OtOpcUa.ScriptWorker.exe") -eq 'candidate') "timing: swap exits 0 ($tsOut)"
$wm = [regex]::Match($tsOut, 'swap window[^:]*: (?<s>[0-9.,]+) s(?<rest>[^\r\n]*)')
# Before #905 the whole swap script ran inside the stopped window (nothing ran before the stop).
$window = if ($wm.Success) { [double]::Parse($wm.Groups['s'].Value, [Globalization.CultureInfo]::CurrentCulture) } else { $ts.Elapsed.TotalSeconds }
$pre = [regex]::Match($tsOut, 'pre-stop isolation (?<s>[0-9.,]+) s').Groups['s'].Value
Write-Host (" TIMING {0} {1}: {2} items under the denied folder; install {3:N1} s; swap script {4:N1} s; pre-stop {5}; STOPPED WINDOW {6:N1} s {7}" -f `
$PSVersionTable.PSEdition, $PSVersionTable.PSVersion, $items, $ti.Elapsed.TotalSeconds, $ts.Elapsed.TotalSeconds,
$(if ($pre) { "$pre s" } else { 'none (all in the window)' }), $window, $wm.Groups['rest'].Value)
$tv = & { $ErrorActionPreference = 'Continue'; & $exe -NoProfile -ExecutionPolicy Bypass -File $tInstaller -Verify -WorkerServiceName $WorkerServiceName 2>&1 | Out-String }
Assert ($LASTEXITCODE -eq 0) 'timing: full verify passes after the swap'
$tu = & { $ErrorActionPreference = 'Continue'; & $exe -NoProfile -ExecutionPolicy Bypass -File $tInstaller -Uninstall -WorkerServiceName $WorkerServiceName 2>&1 | Out-String }
Assert ($LASTEXITCODE -eq 0 -and -not (Get-NetFirewallRule -Name $ruleName -ErrorAction SilentlyContinue)) "timing: uninstall from the record ($tu)"
}
}
catch { $script:failures++; Write-Host " FAIL: unexpected error: $($_ | Out-String)" -ForegroundColor Red } catch { $script:failures++; Write-Host " FAIL: unexpected error: $($_ | Out-String)" -ForegroundColor Red }
finally { finally {
# Best-effort cleanup that does not depend on the code under test. # Best-effort cleanup that does not depend on the code under test.
@@ -201,6 +201,71 @@ Assert-Throws { & $entry -Reapply -Verify 6>$null } '*exclusive*' '-Reapply and
Assert-Throws { & $entry -Verify -Uninstall 6>$null } '*exclusive*' '-Verify and -Uninstall are exclusive' Assert-Throws { & $entry -Verify -Uninstall 6>$null } '*exclusive*' '-Verify and -Uninstall are exclusive'
Assert-Throws { & $entry -InstallRoot $root 6>$null } '*required*' 'a plain install still needs -HostServiceAccount' Assert-Throws { & $entry -InstallRoot $root 6>$null } '*required*' 'a plain install still needs -HostServiceAccount'
Write-Host 'Swap scopes: Outside before the stop, InstallRoot in the stopped window (lmxopcua#905)'
# vd03: the install folder itself is the root, pki\ inside it, the whole E:\ApiInstall denied, data beside it.
$vdRoot = 'E:\ApiInstall\OtOpcUa'
$vdDeny = @('E:\ApiInstall', "$vdRoot\pki", 'E:\ApiInstall\OtOpcUa-data', 'C:\ProgramData\ZB\kek', 'E:\psrem-stage')
$vd = @(Get-ScriptWorkerIsolationPlan -InstallRoot $vdRoot -HostSid 'S-1-5-18' -PeCacheDir $cache -DenyPaths $vdDeny)
$vdIn = @(Select-ScriptWorkerIsolationPlanScope -Plan $vd -InstallRoot $vdRoot -Scope InstallRoot)
$vdOut = @(Select-ScriptWorkerIsolationPlanScope -Plan $vd -InstallRoot $vdRoot -Scope Outside)
$vdAll = @(Select-ScriptWorkerIsolationPlanScope -Plan $vd -InstallRoot $vdRoot -Scope All)
Assert ((@($vdIn | ForEach-Object { "$($_.Id) $($_.Target)" }) -join '|') -eq
"WorkerFolderAcl $vdRoot\script-worker|InstallRootDeny $vdRoot|SensitiveDirDeny $vdRoot\pki") `
"vd03 InstallRoot scope = exactly the three ACEs a swap drops, in plan order ($(@($vdIn | ForEach-Object { "$($_.Id) $($_.Target)" }) -join '|'))"
Assert ((@($vdOut | ForEach-Object { "$($_.Id) $($_.Target)" }) -join '|') -eq
("AnchorService OtOpcUaScriptWorker|DenyPathRecord $($rec.Key)|SettingsRecord $($rec.Key)|PeCacheDir $cache|PeEntriesDir $cache\pe|" +
"SensitiveDirDeny E:\ApiInstall|SensitiveDirDeny E:\ApiInstall\OtOpcUa-data|SensitiveDirDeny C:\ProgramData\ZB\kek|SensitiveDirDeny E:\psrem-stage|" +
"FirewallBlockOutbound OtOpcUa-ScriptWorker-BlockOutbound")) `
"vd03 Outside scope = everything else ($(@($vdOut | ForEach-Object { "$($_.Id) $($_.Target)" }) -join '|'))"
Assert (($vdIn.Count + $vdOut.Count) -eq $vd.Count -and @($vdIn | Where-Object { $vdOut -contains $_ }).Count -eq 0) 'the two scopes partition the plan (no step lost, none twice)'
Assert ($vdAll.Count -eq $vd.Count) '-Scope All is the whole plan'
Assert (@($vdOut | Where-Object { $_.Target -eq 'E:\ApiInstall\OtOpcUa-data' }).Count -eq 1) 'a prefix-sharing sibling (OtOpcUa-data) is Outside (negative control for the prefix match)'
$qaIn = @(Select-ScriptWorkerIsolationPlanScope -Plan $qa -InstallRoot $qaApp -Scope InstallRoot)
Assert ((@($qaIn.Id) -join ',') -eq 'WorkerFolderAcl,InstallRootDeny') "QA (pki\ beside app\): the InstallRoot scope has no SensitiveDirDeny ($(@($qaIn.Id) -join ','))"
Assert (@(Select-ScriptWorkerIsolationPlanScope -Plan $qa -InstallRoot $qaApp -Scope Outside | Where-Object Id -eq 'HostServiceEnv').Count -eq 1) 'the host env is set before the stop (it takes effect on the next start, the new build)'
$staleIn = @(Get-ScriptWorkerIsolationPlan -InstallRoot $vdRoot -HostSid 'S-1-5-18' -PeCacheDir $cache -DenyPaths @('E:\ApiInstall') -RecordedDenyPaths @('E:\ApiInstall', "$vdRoot\pki"))
$staleInScoped = @(Select-ScriptWorkerIsolationPlanScope -Plan $staleIn -InstallRoot $vdRoot -Scope InstallRoot | Where-Object { $_.Data.ContainsKey('Remove') })
Assert ((Join-Targets $staleInScoped) -eq "$vdRoot\pki") 'a stale deny inside InstallRoot is removed in the InstallRoot scope'
Assert (-not (Test-OtOpcUaStepInInstallRoot -Step ($vd | Where-Object Id -eq 'FirewallBlockOutbound') -InstallRoot $vdRoot)) 'the firewall rule is Outside (its exe path does not change; its query costs seconds)'
Assert (Test-OtOpcUaStepInInstallRoot -Step ($vd | Where-Object Id -eq 'WorkerFolderAcl') -InstallRoot "$vdRoot\") 'a trailing backslash on InstallRoot still matches (positive control)'
Write-Host 'Coverage scan roots: each denied folder walked once (lmxopcua#905)'
$cw = Get-OtOpcUaCoverageWalkRoots -InstallRoot $vdRoot -DeniedPaths $vdDeny -Scope All
Assert (($cw.Roots -join '|') -eq 'E:\ApiInstall|C:\ProgramData\ZB\kek|E:\psrem-stage' -and @($cw.Skip).Count -eq 0) "vd03 All: nested InstallRoot, pki\ and OtOpcUa-data are walked inside E:\ApiInstall, once ($($cw.Roots -join '|'))"
$cwIn = Get-OtOpcUaCoverageWalkRoots -InstallRoot $vdRoot -DeniedPaths $vdDeny -Scope InstallRoot
Assert (($cwIn.Roots -join '|') -eq $vdRoot -and @($cwIn.Skip).Count -eq 0) "vd03 InstallRoot: only the swapped-in folder ($($cwIn.Roots -join '|'))"
$cwOut = Get-OtOpcUaCoverageWalkRoots -InstallRoot $vdRoot -DeniedPaths $vdDeny -Scope Outside
Assert (($cwOut.Roots -join '|') -eq 'E:\ApiInstall|C:\ProgramData\ZB\kek|E:\psrem-stage' -and (@($cwOut.Skip) -join '|') -eq $vdRoot) "vd03 Outside: the denied folders, InstallRoot skipped ($($cwOut.Roots -join '|'); skip $(@($cwOut.Skip) -join '|'))"
$cwQa = Get-OtOpcUaCoverageWalkRoots -InstallRoot $qaApp -DeniedPaths $qaDeny -Scope All
Assert (($cwQa.Roots -join '|') -eq $qaRoot) "QA All: app\, pki\, data\ all inside E:\ApiInstall\OtOpcUa ($($cwQa.Roots -join '|'))"
$cwCase = Get-OtOpcUaCoverageWalkRoots -InstallRoot 'C:\x\app' -DeniedPaths @('D:\Keys\', 'd:\keys', 'D:\KeysOld') -Scope Outside
Assert (($cwCase.Roots -join '|') -eq 'D:\Keys|D:\KeysOld') "case and trailing '\' collapse; a prefix-sharing sibling is its own root ($($cwCase.Roots -join '|'))"
Write-Host 'Entry script -Scope (lmxopcua#905)'
$inEntry = @(& $entry -InstallRoot $vdRoot -HostServiceAccount 'S-1-5-18' -PeCacheDir $cache -DenyPaths ($vdDeny -join ',') -PlanOnly -Scope InstallRoot 6>$null)
Assert ((@($inEntry.Id) -join ',') -eq 'WorkerFolderAcl,InstallRootDeny,SensitiveDirDeny') "-PlanOnly -Scope InstallRoot ($(@($inEntry.Id) -join ','))"
$outEntry = @(& $entry -InstallRoot $vdRoot -HostServiceAccount 'S-1-5-18' -PeCacheDir $cache -DenyPaths ($vdDeny -join ',') -PlanOnly -Scope Outside 6>$null)
Assert ($outEntry.Count -eq 10 -and -not ($outEntry.Id -contains 'WorkerFolderAcl')) "-PlanOnly -Scope Outside ($(@($outEntry.Id) -join ','))"
Assert-Throws { & $entry -InstallRoot $root -HostServiceAccount $hostSid -Scope InstallRoot 6>$null } '*-Scope works with*' 'a plain install refuses -Scope'
Assert-Throws { & $entry -Uninstall -InstallRoot $root -HostServiceAccount $hostSid -Scope Outside 6>$null } '*-Scope works with*' '-Uninstall refuses -Scope'
Assert-Throws { & $entry -Verify -Scope Elsewhere 6>$null } '*Elsewhere*' 'an unknown scope is refused'
Write-Host 'Swap script order (lmxopcua#905; parsed, not run)'
$swapAst = [System.Management.Automation.Language.Parser]::ParseFile((Join-Path $installDir 'Invoke-OtOpcUaInstallSwap.ps1'), [ref]$null, [ref]$null)
$cmds = @($swapAst.FindAll({ param($n) $n -is [System.Management.Automation.Language.CommandAst] }, $true) | Where-Object { $_.GetCommandName() })
function First-Offset([string]$Name, [string]$Contains = '') {
$hit = @($cmds | Where-Object { $_.GetCommandName() -eq $Name -and $_.Extent.Text -like "*$Contains*" } | Sort-Object { $_.Extent.StartOffset })
if ($hit.Count) { $hit[0].Extent.StartOffset } else { -1 }
}
$iso = @($cmds | Where-Object { $_.GetCommandName() -eq 'Invoke-Isolation' })
Assert ($iso.Count -eq 4 -and @($iso | Where-Object { $_.CommandElements.Count -ne 3 }).Count -eq 0) "every Invoke-Isolation call names a scope ($(@($iso | ForEach-Object { $_.Extent.Text }) -join ' | '))"
$order = @(
(First-Offset 'Invoke-Isolation' "'Reapply' 'Outside'"), (First-Offset 'Invoke-Isolation' "'Verify' 'Outside'"), (First-Offset 'Stop-Service'),
(First-Offset 'Move-Item' '$Candidate'), (First-Offset 'Invoke-Isolation' "'Reapply' 'InstallRoot'"),
(First-Offset 'Invoke-Isolation' "'Verify' 'InstallRoot'"), (First-Offset 'Start-Service'))
Assert (@($order | Where-Object { $_ -lt 0 }).Count -eq 0 -and (($order -join ',') -eq (@($order | Sort-Object) -join ','))) `
"Reapply/Verify Outside -> stop -> swap -> Reapply/Verify InstallRoot -> start ($($order -join ','))"
if ($IsWindows) { if ($IsWindows) {
Write-Host 'SDDL parses on Windows' Write-Host 'SDDL parses on Windows'
$ds = [System.Security.AccessControl.DirectorySecurity]::new() $ds = [System.Security.AccessControl.DirectorySecurity]::new()