feat(scripting): ScriptWorker isolation on Windows: restricted worker identity, pipe ACL, firewall + PeCacheDir installer (lmxopcua#780)
v2-ci / unit-tests (push) Has been cancelled
v2-ci / unit-tests (push) Has been cancelled
Owner decisions §14 #5/#6 as built (design §12.5): - Scripting:Worker:Identity (default unset, Windows only, fail-closed validation): each worker starts under a restricted token derived from the host's. Every privilege but SeChangeNotify is removed; the restricting SIDs are NT SERVICE\OtOpcUaScriptWorker's SID + Everyone + Users + RESTRICTED; the token and default DACLs are host + SYSTEM + identity + RESTRICTED. The worker is created suspended, put in its Job Object before it runs, inherits only its stdio pipes, gets an ACL'd work dir as TEMP, and runs with DOTNET_EnableDiagnostics=0. The identity is granted on the host's window station and desktop, because USER32 init made coreclr.dll fail with ERROR_DLL_INIT_FAILED on windev. The logon SID is not used because the host's process DACL gives it PROCESS_VM_READ. - The worker pipe on Windows always has an explicit protected DACL: host account + identity only. There is no owner ACE, so an elevated host's Administrators gets no access. - The worker treats access denied on its parent-pid poll as alive; the Job Object covers that case. - scripts/install/Install-ScriptWorkerIsolation.ps1 + ScriptWorkerIsolation.psm1 are idempotent and support -WhatIf, -Uninstall and -PlanOnly. They create the disabled anchor service, the PeCacheDir ACL (protected, no Users, key and markers host-only, pe\ Modify for the identity), read+execute on script-worker\, a ReadData deny on the install root, the outbound firewall block on the worker exe, and optionally the host-service env. - Tests: the dry-run harness scripts/install/tests/Test-ScriptWorkerIsolation.ps1 (46 assertions, mutation-checked); ScriptWorkerIdentityTests; WorkerIsolationTests, which cover the pipe ACL, token access and a real restricted worker running a script, all green on windev. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
+20
-3
@@ -101,13 +101,30 @@ or one inside the content root) on a directory only the service account can read
|
||||
is at worst Degraded. A self-contained host publish (`dotnet publish … -r win-x64 --self-contained`, the Windows installs, which have no
|
||||
.NET runtime) ships a **self-contained** worker in `script-worker/` (~97 MB: it carries its own runtime) and the host starts its
|
||||
`ZB.MOM.WW.OtOpcUa.ScriptWorker.exe` apphost directly; a framework-dependent host (docker-dev) runs the worker's dll through the
|
||||
dotnet muxer. The worker currently runs under the host's account; the dedicated low-privilege account,
|
||||
pipe ACL and firewall outbound rule (design §14 #5/#6) are installer work still open on lmxopcua#780. See
|
||||
`docs/VirtualTags.md` §"Out-of-process runner".
|
||||
dotnet muxer. See `docs/VirtualTags.md` §"Out-of-process runner".
|
||||
|
||||
### Worker isolation on Windows (`Install-ScriptWorkerIsolation.ps1`, design §14 #5/#6)
|
||||
|
||||
`scripts/install/Install-ScriptWorkerIsolation.ps1 -InstallRoot <dir> -HostServiceAccount <account> [-ConfigureHostService]`
|
||||
(idempotent; `-WhatIf` prints every change; `-Uninstall` rolls back; `-PlanOnly` prints the plan on any OS) sets up:
|
||||
|
||||
| Step | Change |
|
||||
|---|---|
|
||||
| Worker identity | A **disabled** anchor service `OtOpcUaScriptWorker` running as `NT SERVICE\OtOpcUaScriptWorker`, so the virtual account exists. It is never started. |
|
||||
| `Scripting:Worker:Identity` | With `NT SERVICE\OtOpcUaScriptWorker` set (by `-ConfigureHostService`, or in appsettings), the host starts every worker under a **restricted token** derived from its own: every privilege but `SeChangeNotifyPrivilege` removed, and restricting SIDs = the identity + `Everyone` + `BUILTIN\Users` + `RESTRICTED`. Every access the worker makes must pass for the host account **and** for that set. The token's default DACL is host + SYSTEM + identity; the worker is created suspended and put in its Job Object before it runs; `TEMP`/`TMP` point at its per-launch working directory (ACL: host + SYSTEM full, identity Modify); `DOTNET_EnableDiagnostics=0`. The host also adds the identity to its own window station and desktop DACL (enumerate/read, and create-window on the desktop), once per process. USER32 connects to them in its DllMain and the runtime imports USER32, so without the grant `coreclr.dll` fails to load (`ERROR_DLL_INIT_FAILED`, measured on windev). The grant lives on the session's kernel objects and ends with the session; nothing persistent changes. The logon SID, which those objects normally grant, is deliberately not a restricting SID: the host's process DACL gives it `PROCESS_VM_READ`. Windows cannot start an ad-hoc child process *as* a virtual account (only the SCM can, for a service), and a separate local user would need a stored password plus `SeAssignPrimaryTokenPrivilege` on the host account; a restricted copy of the host's own token needs neither. Startup refuses the key off Windows or when malformed. |
|
||||
| Pipe ACL | On Windows the worker pipe always carries an explicit, protected DACL: the host account and (with an identity) the worker identity, read/write, nobody else. No SYSTEM, no Administrators, no Everyone. The owner is the host token's owner, so the worker's `CurrentUserOnly` check of the server end still holds. |
|
||||
| `PeCacheDir` | Created with a protected ACL: SYSTEM, Administrators and the host account full control, no Users. `pe\` grants the identity Modify (workers compile and write entries). The HMAC key and `workers\` markers are host-only. |
|
||||
| Install folder | The identity is **denied** ReadData on `<InstallRoot>` (inherited), so the worker cannot read `appsettings*.json`, logs or host binaries even where `Users` can. `script-worker\` grants it read + execute. |
|
||||
| Firewall | Outbound **block** rule `OtOpcUa-ScriptWorker-BlockOutbound` on `script-worker\ZB.MOM.WW.OtOpcUa.ScriptWorker.exe`, every profile. It covers the self-contained apphost the Windows installs run. A framework-dependent worker runs under `dotnet.exe` and is not covered, so there the analyzer's `System.Net` ban is the only network control. |
|
||||
|
||||
The plan is pure data asserted by `scripts/install/tests/Test-ScriptWorkerIsolation.ps1` (`pwsh -File …`, any OS).
|
||||
Re-run the installer after a deploy that replaces the install folder: a rename/swap drops the folder ACEs, but files
|
||||
copied into the existing folder inherit them. Rollback: `-Uninstall -ConfigureHostService`, then restart the host.
|
||||
|
||||
## Install / Uninstall
|
||||
|
||||
- `scripts/install/Install-Services.ps1 -Roles admin,driver` — installs `OtOpcUaHost`.
|
||||
- `scripts/install/Install-ScriptWorkerIsolation.ps1` — the ScriptWorker identity, ACLs and firewall rule (above); only for `Scripting:Runner` ≠ `InProcess`.
|
||||
- `scripts/install/Uninstall-Services.ps1` — stops + removes the host service. (The historian backend is the external HistorianGateway — not installed/removed by these scripts.)
|
||||
|
||||
## Logging
|
||||
|
||||
+7
-3
@@ -71,7 +71,7 @@ processes:
|
||||
The runner **fails closed**: no worker means Bad, never an in-process run.
|
||||
- **Containment.** The worker gets a cleared, allow-listed environment (no `ServerHistorian__ApiKey`, no
|
||||
connection strings), an empty temp working directory, `DOTNET_GCHeapHardLimit = Scripting:Worker:HeapLimitBytes`
|
||||
(512 MiB), a Windows Job Object (kill-on-close, one process, a commit backstop), a random current-user-only pipe
|
||||
(512 MiB), a Windows Job Object (kill-on-close, one process, a commit backstop), a random pipe (current-user-only on Linux; an explicit host + worker-identity ACL on Windows)
|
||||
with a launch nonce in its environment, and no Akka / gRPC assemblies in its folder. It exits on pipe end of stream
|
||||
or when its parent pid disappears. Results cross as protobuf `Variant`s carrying the exact CLR type, so
|
||||
`(short)ctx.GetTag(x).Value` behaves as in process; an input or result type with no tag (a `List<T>`, a `Guid`)
|
||||
@@ -121,8 +121,12 @@ processes:
|
||||
runtime's reference assemblies on disk), and the launcher starts its apphost `ZB.MOM.WW.OtOpcUa.ScriptWorker.exe`
|
||||
directly: `OutOfProcessScriptRunner.ResolveWorkerEntry` picks the apphost only when the worker's `runtimeconfig.json` lists
|
||||
`includedFrameworks`. Gated on windev against a real `-r win-x64` publish (`OTOPCUA_SCRIPTWORKER_TEST_PATH` points the
|
||||
worker suite at the published apphost). Not yet done (lmxopcua#780): the low-privilege worker account, pipe ACL, firewall
|
||||
rule and PeCacheDir ACL (installer).
|
||||
worker suite at the published apphost).
|
||||
- **Windows isolation** (`Scripting:Worker:Identity`, `scripts/install/Install-ScriptWorkerIsolation.ps1`): the worker runs
|
||||
under a restricted, privilege-free token for the virtual account `NT SERVICE\OtOpcUaScriptWorker`. The pipe ACL admits
|
||||
the host and that identity only. The install folder is denied to it, `PeCacheDir` is host-only apart from `pe\`, and an
|
||||
outbound firewall block covers the worker exe. See `docs/ServiceHosting.md` §"Worker isolation on Windows". The live
|
||||
install on QA is a deploy gate (lmxopcua#780).
|
||||
- **docker-dev.** `docker-dev/docker-compose.script-runner.yml` turns `OutOfProcess` on for the central pair's
|
||||
VirtualTag and Calculation seams. The 2026-11-10 live drill (runaway kill and quarantine, `kill -9` recovery, open circuit
|
||||
with readiness still 200) is recorded in the design doc's rollout section.
|
||||
|
||||
@@ -647,6 +647,7 @@ per-shard state, restarts in the last hour and the quarantine list. Quarantines
|
||||
| `Scripting:Worker:QuarantineAfterKills` | `2` | §5.3 |
|
||||
| `Scripting:Worker:UnavailableGrace` | `5s` | §9 |
|
||||
| `Scripting:Worker:PeCacheDir` | none (required when not `InProcess`) | §7; absolute, outside the content root, fail-closed |
|
||||
| `Scripting:Worker:Identity` | none (host's token) | Windows only: `NT SERVICE\OtOpcUaScriptWorker` → restricted worker token + pipe ACL (§14 #5, record §12.5) |
|
||||
|
||||
**Phases** (each one is its own issue and PR):
|
||||
|
||||
@@ -1196,6 +1197,56 @@ one running the in-flight predicate, which is what the step needed.)
|
||||
the state the rig was already in (most likely from the earlier drill's store loss), not something this drill did.
|
||||
- The drill image `otopcua-host:w34-shadow` was removed afterwards. The site nodes, sql and traefik were not touched.
|
||||
|
||||
### 12.5 Installer record: worker identity, pipe ACL, firewall, PeCacheDir ACL (2026-11-10)
|
||||
|
||||
Owner decisions §14 #5 and #6, as built. Operator reference: `docs/ServiceHosting.md` §"Worker isolation on Windows".
|
||||
|
||||
- **Identity, deviating in mechanism from "run as a virtual account".** Windows starts a process as `NT SERVICE\<name>` only
|
||||
when the SCM starts that service. A host cannot launch K + 1 children that way, each with its own nonce, pipe and Job
|
||||
Object. A separate local user would need a stored password plus `SeAssignPrimaryTokenPrivilege` on the host account.
|
||||
`Scripting:Worker:Identity` (default unset; Windows only; startup refuses it elsewhere or when malformed) therefore starts
|
||||
each worker under a **restricted token** derived from the host's (`RestrictedWorkerLauncher`):
|
||||
- `DISABLE_MAX_PRIVILEGE`, so only `SeChangeNotifyPrivilege` is left;
|
||||
- restricting SIDs = the virtual account's SID (computed from the name, `S-1-5-80-…`) + Everyone + Users + RESTRICTED;
|
||||
- token and default DACL = host + SYSTEM + identity + RESTRICTED;
|
||||
- created suspended and put in its Job Object before it runs, inheriting exactly its two stdio pipes;
|
||||
- `TEMP` set to its ACL'd work directory, and `DOTNET_EnableDiagnostics=0`.
|
||||
|
||||
The installer registers a disabled anchor service, so the account exists and resolves by name.
|
||||
- **Found on windev.** The runtime would not start under the restricted token (`coreclr.dll` → `ERROR_DLL_INIT_FAILED`). A
|
||||
bisection over the token's groups showed that only the user or Administrators SID fixed it, not the logon SID. The cause
|
||||
was USER32's window station and desktop connection, so the host now grants the identity on its own window station and
|
||||
desktop. The logon SID stays out of the restricting set because the host's process DACL gives it `PROCESS_VM_READ`. The
|
||||
token's own DACL also needed rewriting (the handle needs `WRITE_DAC`).
|
||||
- **Pipe ACL.** On Windows the pipe always has an explicit, protected DACL: the host account and the identity, read/write,
|
||||
nothing else. .NET's `CurrentUserOnly` grants the token *owner*, which is Administrators on an elevated host; this ACL
|
||||
does not. The owner is still the token owner, so the worker's `CurrentUserOnly` client check is unchanged.
|
||||
- **Installer** `scripts/install/Install-ScriptWorkerIsolation.ps1` + `ScriptWorkerIsolation.psm1`. It is idempotent, with
|
||||
`-WhatIf`, `-Uninstall` and `-PlanOnly`, and it sets up:
|
||||
- the anchor service;
|
||||
- `PeCacheDir`: protected, SYSTEM / Administrators / host full control, no Users, and the key and markers host-only;
|
||||
- `pe\`: the identity gets Modify;
|
||||
- `script-worker\`: the identity gets read+execute;
|
||||
- **deny** ReadData for the identity on the install folder (appsettings, logs, host binaries);
|
||||
- outbound **block** firewall rule `OtOpcUa-ScriptWorker-BlockOutbound` on the worker apphost;
|
||||
- optional host-service env (`-ConfigureHostService`).
|
||||
|
||||
The runner flip itself is not part of it.
|
||||
- **Tests.**
|
||||
- `scripts/install/tests/Test-ScriptWorkerIsolation.ps1` is the dry-run harness. It runs on any OS and asserts ids,
|
||||
order, targets, SDDL and the rollback (46 assertions). A mutation check (deny → allow, protected → inherited) turns
|
||||
4 red.
|
||||
- `ScriptWorkerIdentityTests` (Core.Scripting) pin the SID algorithm against `NT SERVICE\TrustedInstaller`. The
|
||||
PowerShell harness pins the same default SID.
|
||||
- `WorkerIsolationTests` (ScriptWorker) cover the pipe ACL (built and live), the token, and a real worker under the
|
||||
identity running a script with a restricted token. A plain worker's token is unrestricted (positive control).
|
||||
- The token case is restricted, privilege-free, and impersonated: it reads System32 and writes its work dir, but
|
||||
cannot read or write the host's temp.
|
||||
- They ran green on windev without changing machine state, apart from the session's window station / desktop ACE.
|
||||
No account, firewall rule or persistent ACL was created there.
|
||||
- **Not verified live:** the installer on a real node. That is the QA deploy gate on lmxopcua#780's follow-up (see
|
||||
that issue for the checklist).
|
||||
|
||||
---
|
||||
|
||||
## 13. Test plan
|
||||
|
||||
Reference in New Issue
Block a user