fix(scripting): ScriptWorker isolation denies the worker every sensitive sibling, not just app\ (#890)
v2-ci / unit-tests (push) Waiting to run

The installer denied NT SERVICE\OtOpcUaScriptWorker only on -InstallRoot. On the wonder layout the node's keys and
secrets sit beside it (E:\ApiInstall\OtOpcUa\pki with the application certificate's private key, data\ with
secrets.db and LocalDb), inheriting BUILTIN\Users:RX. Users is a restricting SID of the worker token, so a worker
could read them.

- Install-ScriptWorkerIsolation.ps1 -DenyPaths <dir>,...: each folder gets the same inherited deny as InstallRoot
  (new SensitiveDirDeny steps). Refuses drive/share roots, script-worker\ and the PE cache; folders must exist;
  a comma/semicolon string works for powershell -File.
- The deny is read AND write (0xD0157: ReadData, WriteData, AppendData, WriteEA, DeleteChild, WriteAttributes,
  Delete, WRITE_DAC, WRITE_OWNER). Traverse/Execute, ReadAttributes, ReadControl and Synchronize stay so the
  loader still reaches script-worker\. The write half stops a worker planting a cert in pki\trusted\certs where
  a data drive lets Users create files in subfolders.
- The list is recorded under the anchor service's key (Parameters\DenyPaths, step DenyPathRecord): a re-run
  removes the deny from folders dropped from the list, -Uninstall removes every recorded one without repeating
  -DenyPaths. -WhatIf/-PlanOnly print every step.
- After an apply, WARN for each protected-DACL item under a denied folder that the inherited deny cannot reach;
  without -DenyPaths, WARN for pki/data/logs/backup/config/secrets siblings of InstallRoot.

Tests:
- Dry-run harness 46 -> 86 assertions (QA layout plan, mask bits, refusals, stale removal, record-driven
  rollback, host account never denied). Mutations (no sibling deny, deny->allow, mask back to ReadData, no
  stale removal, rollback ignoring the record, script-worker not refused) each turn it red.
- Test-ScriptWorkerIsolation.Windows.ps1 (Windows PowerShell 5.1, windev): 34/34, incl. -DenyPaths apply,
  11/11 unchanged re-apply, shorter-list re-apply, uninstall from the record, second uninstall unchanged.
- New InstallerLayoutIsolationTests (OTOPCUA_ISOLATION_INSTALLER_E2E=1, elevated Windows): a scratch wonder
  layout under the real restricted worker token. Before install and after uninstall the token reads pki/data/
  logs/backup/appsettings and plants a trust-list file; installed it can do neither, still reads script-worker\,
  a real worker launched from the layout runs a script, and the host still reads everything. Mutating the
  -DenyPaths denies to a no-op mask turns it red on pki\own\private\otopcua.pfx (reproduces #890).

QA vq03/vq04 are NOT re-applied; the command is in docs/deploy/qa-upgrade-b8b4c18f-to-fea7de0a.md.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
2026-11-12 17:31:28 -05:00
parent c80d187c18
commit f75ee7d089
8 changed files with 597 additions and 34 deletions
+6 -2
View File
@@ -105,7 +105,7 @@ dotnet muxer. See `docs/VirtualTags.md` §"Out-of-process runner".
### Worker isolation on Windows (`Install-ScriptWorkerIsolation.ps1`, design §14 #5/#6)
`scripts/install/Install-ScriptWorkerIsolation.ps1 -InstallRoot <dir> -HostServiceAccount <account> [-ConfigureHostService]`
`scripts/install/Install-ScriptWorkerIsolation.ps1 -InstallRoot <dir> -HostServiceAccount <account> [-DenyPaths <dir>,…] [-ConfigureHostService]`
(idempotent; `-WhatIf` prints every change; `-Uninstall` rolls back; `-PlanOnly` prints the plan on any OS) sets up:
| Step | Change |
@@ -114,10 +114,14 @@ dotnet muxer. See `docs/VirtualTags.md` §"Out-of-process runner".
| `Scripting:Worker:Identity` | With `NT SERVICE\OtOpcUaScriptWorker` set (by `-ConfigureHostService`, or in appsettings), the host starts every worker under a **restricted token** derived from its own: every privilege but `SeChangeNotifyPrivilege` removed, and restricting SIDs = the identity + `Everyone` + `BUILTIN\Users` + `RESTRICTED`. Every access the worker makes must pass for the host account **and** for that set. The token's default DACL is host + SYSTEM + identity; the worker is created suspended and put in its Job Object before it runs; `TEMP`/`TMP` point at its per-launch working directory (ACL: host + SYSTEM full, identity Modify); `DOTNET_EnableDiagnostics=0`. The host also adds the identity to its own window station and desktop DACL (enumerate/read, and create-window on the desktop), once per process. USER32 connects to them in its DllMain and the runtime imports USER32, so without the grant `coreclr.dll` fails to load (`ERROR_DLL_INIT_FAILED`, measured on windev). The grant lives on the session's kernel objects and ends with the session; nothing persistent changes. The logon SID, which those objects normally grant, is deliberately not a restricting SID: the host's process DACL gives it `PROCESS_VM_READ`. Windows cannot start an ad-hoc child process *as* a virtual account (only the SCM can, for a service), and a separate local user would need a stored password plus `SeAssignPrimaryTokenPrivilege` on the host account; a restricted copy of the host's own token needs neither. Startup refuses the key off Windows or when malformed. |
| Pipe ACL | On Windows the worker pipe always carries an explicit, protected DACL: the host account and (with an identity) the worker identity, read/write, nobody else. No SYSTEM, no Administrators, no Everyone. The owner is the host token's owner, so the worker's `CurrentUserOnly` check of the server end still holds. |
| `PeCacheDir` | Created with a protected ACL: SYSTEM, Administrators and the host account full control, no Users. `pe\` grants the identity Modify (workers compile and write entries). The HMAC key and `workers\` markers are host-only. |
| Install folder | The identity is **denied** ReadData on `<InstallRoot>` (inherited), so the worker cannot read `appsettings*.json`, logs or host binaries even where `Users` can. `script-worker\` grants it read + execute. |
| Install folder | The identity is **denied** read and write on `<InstallRoot>` (inherited: ReadData/List, WriteData/AddFile, AppendData/AddSubdirectory, write attributes/EA, Delete, DeleteChild, ChangePermissions, TakeOwnership; Traverse/Execute, ReadAttributes and ReadPermissions stay so the loader reaches `script-worker\`). The worker cannot read `appsettings*.json`, logs or host binaries even where `Users` can. `script-worker\` grants it read + execute. |
| Sensitive folders (`-DenyPaths`, lmxopcua#890) | The same inherited deny on every listed folder. **Name every folder outside `<InstallRoot>` that holds keys, secrets, stores, logs or backups**: the worker's restricting SIDs include `Everyone` and `BUILTIN\Users`, so anything Users can read is otherwise readable to it (and a data drive usually lets Users create files in subfolders, e.g. a certificate in `pki\trusted\certs`). Name the deployment folder around `<InstallRoot>` (one ACE covers `pki\`, `data\`, `logs\`, `backup\`, old `app.bak.*`) **and** `pki\` and `data\` themselves, so the deny holds even on a folder whose DACL does not inherit. Drive and share roots, `script-worker\` and the PE cache are refused; each folder must exist. The list is recorded under `HKLM\…\Services\OtOpcUaScriptWorker\Parameters` (`DenyPaths`): a re-run removes the deny from folders dropped from the list, and `-Uninstall` removes every recorded one without repeating `-DenyPaths`. After an apply the installer WARNs about any item under a denied folder whose DACL is protected and has no deny of its own (the inherited deny does not reach it); without `-DenyPaths` it WARNs about `pki`/`data`/`logs`/`backup`/`config`/`secrets` siblings of `<InstallRoot>`. Wonder layout: `-InstallRoot E:\ApiInstall\OtOpcUa\app -DenyPaths E:\ApiInstall\OtOpcUa,E:\ApiInstall\OtOpcUa\pki,E:\ApiInstall\OtOpcUa\data`. |
| Firewall | Outbound **block** rule `OtOpcUa-ScriptWorker-BlockOutbound` on `script-worker\ZB.MOM.WW.OtOpcUa.ScriptWorker.exe`, every profile. It covers the self-contained apphost the Windows installs run. A framework-dependent worker runs under `dotnet.exe` and is not covered, so there the analyzer's `System.Net` ban is the only network control. |
The plan is pure data asserted by `scripts/install/tests/Test-ScriptWorkerIsolation.ps1` (`pwsh -File …`, any OS).
`Test-ScriptWorkerIsolation.Windows.ps1` applies it for real in a scratch folder (elevated), and
`InstallerLayoutIsolationTests` (ScriptWorker.Tests, `OTOPCUA_ISOLATION_INSTALLER_E2E=1`, elevated) proves under the real
restricted token that a worker cannot read `pki\`/`data\`/`logs\`/`backup\` of a scratch wonder layout and still runs.
Re-run the installer after a deploy that replaces the install folder: a rename/swap drops the folder ACEs, but files
copied into the existing folder inherit them. Rollback: `-Uninstall -ConfigureHostService`, then restart the host.
+32 -1
View File
@@ -152,6 +152,36 @@ siblings `E:\ApiInstall\OtOpcUa\pki` (including `own\private\*.pfx`) and `data\`
`BUILTIN\Users:ReadAndExecute`. Users is one of the worker token's restricting SIDs, so those folders are not covered
by the deny. File-level ACLs were not inspected. Follow-up: #890.
**#890 fix, PENDING re-apply on vq03 and vq04.** Not run on QA yet. This is a deploy step for a later, user-requested window.
The installer now takes `-DenyPaths` and records them. Once the scripts at that commit are staged on each node, run the
following in an elevated Windows PowerShell 5.1 session. The services can stay up: only folder ACLs change for the worker
identity, and no worker runs on QA today.
```powershell
cd <staged scripts\install>
$deny = 'E:\ApiInstall\OtOpcUa','E:\ApiInstall\OtOpcUa\pki','E:\ApiInstall\OtOpcUa\data'
.\Install-ScriptWorkerIsolation.ps1 -InstallRoot E:\ApiInstall\OtOpcUa\app -HostServiceAccount LocalSystem -ConfigureHostService -DenyPaths $deny -WhatIf
.\Install-ScriptWorkerIsolation.ps1 -InstallRoot E:\ApiInstall\OtOpcUa\app -HostServiceAccount LocalSystem -ConfigureHostService -DenyPaths $deny
.\Install-ScriptWorkerIsolation.ps1 -InstallRoot E:\ApiInstall\OtOpcUa\app -HostServiceAccount LocalSystem -ConfigureHostService -DenyPaths $deny
```
Before running it, check `Secrets__SqlitePath`, its key file and `LocalDb__Path` in the service environment. Add any folder
that sits outside `E:\ApiInstall\OtOpcUa` to `$deny`.
Expected results:
- **First apply: 11 steps, 5 `changed`.** They are `DenyPathRecord`, `InstallRootDeny` (mask `0x1` → `0xD0157`, read + write)
and the three `SensitiveDirDeny`. The other 6 steps are `unchanged`.
- **Second apply: 11/11 `unchanged`.**
- **Check:**
- `icacls E:\ApiInstall\OtOpcUa`, `…\pki` and `…\data` and `…\app` each show
`NT SERVICE\OtOpcUaScriptWorker:(OI)(CI)(DENY)(W,D,WDAC,WO,RD,DC)`, which is what windev's PowerShell 5.1 renders for
`0xD0157`.
- `reg query HKLM\SYSTEM\CurrentControlSet\Services\OtOpcUaScriptWorker\Parameters /v DenyPaths` lists the three folders.
- The `coverage:` line reports the number of protected items the deny does not reach. Inspect each one WARNed (for
example a `pki\own\private` file the SDK wrote with its own DACL).
Rollback stays `… -ConfigureHostService -Uninstall`. It now also removes the deny from the recorded folders.
## Configuration set (service registry environment; no file changes, no secrets added)
| Node | Added |
@@ -165,7 +195,8 @@ by the deny. File-level ACLs were not inspected. Follow-up: #890.
- **Runner only:** set `Scripting__Runner=InProcess` (or remove the four `Scripting__Runner*` entries) and restart the
node. To remove the isolation, run `Install-ScriptWorkerIsolation.ps1 -InstallRoot E:\ApiInstall\OtOpcUa\app
-HostServiceAccount LocalSystem -ConfigureHostService -Uninstall`.
-HostServiceAccount LocalSystem -ConfigureHostService -Uninstall`. After the #890 re-apply, this also removes the recorded
`-DenyPaths` denies, so there is no need to repeat them.
- **QA binaries:** run `E:\psrem-stage\otopcua-fea7de0a\rollback.ps1` on **both** nodes (stop both first), then start both.
The script does three things:
1. `app` → `app.failed-fea7de0a`, and `app.bak.20261112-pre-fea7de0a` → `app`;
@@ -1301,6 +1301,23 @@ Owner decisions §14 #5 and #6, as built. Operator reference: `docs/ServiceHosti
No account, firewall rule or persistent ACL was created there.
- **Not verified live:** the installer on a real node. That is the QA deploy gate on lmxopcua#780's follow-up (see
that issue for the checklist).
- **lmxopcua#890 (2026-11-12): sensitive siblings.** The QA deploy found the deny covered `InstallRoot` (`app\`) only, while the
wonder layout keeps `pki\` (application certificate private key) and `data\` (`secrets.db`, LocalDb) **beside** it, inheriting
`BUILTIN\Users:RX` from `E:\`. Users is a restricting SID, so a worker could read them. The installer now:
- takes `-DenyPaths` (the deployment folder around `InstallRoot` plus each key/secret folder) and gives each the same
inherited deny as `InstallRoot`;
- widens the deny from ReadData to read **and write** (`0xD0157`: ReadData, WriteData, AppendData, WriteEA, DeleteChild,
WriteAttributes, Delete, WRITE_DAC, WRITE_OWNER). Traverse/Execute, ReadAttributes, ReadControl and Synchronize stay
allowed, so the loader still reaches `script-worker\`. The write half closes a planted trust-list certificate, since
a data drive lets Users create files in subfolders;
- records the list under the anchor service's key, so a re-run drops stale denies and `-Uninstall` removes exactly the
recorded ones;
- WARNs about protected-DACL items under a denied folder that the inherited deny cannot reach.
Proof: `InstallerLayoutIsolationTests` on windev, a scratch wonder layout under a real restricted token. Before the install
and after the uninstall, the token reads every file and plants a trust-list file. Installed, it cannot do either, still reads
`script-worker\`, and a real worker launched from the layout runs a script. With the `-DenyPaths` denies mutated to a no-op
mask, the test goes red on `pki\own\private\*.pfx`, which reproduces #890.
### 12.6 lmxopcua#789 host CPU on Windows: blocking pipe I/O off the completion port (2026-11-11)