Re-runs only the ScriptWorker step on windev (master da948e5f, SDK 2.0.1, 16 vCPUs): install as the OtOpcUaHost service,
repo Install-ScriptWorkerIsolation.ps1 applied/re-applied/uninstalled twice under pwsh 7 and Windows PowerShell 5.1,
restricted worker, pipe refuses the admin, runner Healthy, strict-peer/API-key matrix on, then cleanup. The harness drops
-PatchedIsolation, gains -PrepareFrom candidate and -ApplyHost, and suffixes the isolation evidence with -Label.
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
14 KiB
lmxopcua#800: Windows service gate on windev (2026-11-12)
Follow-up (2026-11-12, later): lmxopcua#883 is fixed and step 3 was re-run with the repository installer on both PowerShell hosts: see
../2026-11-12-windev-isolation-rerun/RESULTS.md. The step 3 text below is the record of the first run.
Result: steps 1, 2, 4 and 5 passed; step 3 passed only partly. The SDK 2 candidate upgraded in place under the real
OtOpcUaHost Windows service:
- it reused the existing application certificate byte-for-byte, with its file ACLs unchanged;
- it passed the strict-peer and API-key checks on three starts;
- it rolled back to the known-good build with the same identity.
The ScriptWorker runtime isolation works under a real service session, but the repository installer fails on Windows (lmxopcua#883). The runtime part of step 3 was proven with the #883 patch, not with the shipped script. Three pre-existing findings were filed: #879, #880 (a rollback and QA-compat risk) and #883.
Environment
- Node: windev (
DESKTOP-6JL3KKO, 10.100.0.48). Elevated SSH session, PowerShell 7.6.10; Windows PowerShell 5.1 was also used for the installer. - Host change during the run: the coordinator power-cycled the node between steps 1 and 2 to reconfigure it (36 → 16 vCPUs,
latency sensitivity High). The installed service auto-started across that reboot (row
b2reboot). Timings before and after the reboot are not like-for-like. - Clock: the Windows Time service is stopped. Before the reboot the clock was about 9 minutes slow, after it about 14 s. All probes ran on the same box, so #873 skew does not apply.
- Left untouched: the existing MxAccessGw (
:5120/:5130) and OtOpcUaWonderwareHistorian services were never touched. - Ports: the test service used only task ports: HTTP
127.0.0.1:18800, OPC UA14841, Akka18801.
Artifacts
Every artifact is a self-contained win-x64 publish, built on macOS (.NET SDK 10.0.204). Each manifest lists sha256 path and
was matched file-for-file on windev before use.
| Artifact | Source | Files | Manifest SHA-256 | Main binary SHA-256 |
|---|---|---|---|---|
| Known-good Host | master efb9197a0f48802450474413c58b9d0c461b836d (last SDK 1.5.378.182 master commit) |
1376 | 28bdba9e…49d18d |
OtOpcUa.Host.exe f9a193c6c7d224006094e26ce4cd1616961bcb9048ff38a468786618a26e3a91 |
| Candidate Host | master 438484fdca4df12be6fbcae71cfbbeabb9386004 (SDK 2.0.1; Opc.Ua.Core file version 2.0.1.2) |
1396 | e7b6683e…090c5e34 |
OtOpcUa.Host.exe 6e9efdde592eb342719e9c0d47783e306bc9dd697ae80fc32d45dd1538a52000 |
| Candidate CLI | 438484fd |
241 | dee9a60e…2fdd1504 |
e2c88c071733e5807a260ee33d5064e1ec13178d2562aab8d7faff763b6974a0 |
| Known-good CLI | efb9197a |
210 | f7715725…0c217c66 |
257b5b13e980072e46f9405aeddbbddb440a17ad9d26e122ede3fddb15cb27cf |
| Probe (SDK 2.0.1) | this directory at 438484fd |
1132 | abe1d8b6…ba070 |
Probe.dll 64597faadab6511b174c4c9aa1f5fd06054ffccbb5b77b841c318838e6cbb9fa |
| Seeder | this directory, built in the efb9197a tree |
413 | ec93daf8…7106561 |
Seeder.dll 18690a15628f028df95bd4a578049a3997f31e41c22a5a06ee04e50739920de5 |
The full manifests are manifest-*.txt in this directory. The candidate script-worker\ZB.MOM.WW.OtOpcUa.ScriptWorker.exe is
9dc0558bb62231487dafcd1b792bbe280b5bafc8849cb0347b14fb50483cfc04, one of the 275 files in script-worker\.
Install
- Install command:
scripts/install/Install-Services.ps1 -InstallRoot C:\OtOpcUa800\OtOpcUa -ServiceAccount 'NT SERVICE\OtOpcUaHost' -Roles driver, from the known-good tree. It is the repository-supported method; vd03 runs the sameOtOpcUaHostsc.exe service. - Node configuration: set in the service
Environmentvalue, 26 entries, digestfd534f8d…. The digest was the same at every known-good and candidate start exceptc3, where the isolation run adds 3 entries.- single-node driver:
FetchAndCache, with an unreachable central; - secrets SQLite outside the content root;
- pepper;
- LDAP and ACL off;
AllowAnonymous=false, auto-accept off;EnabledSecurityProfiles=[SignAndEncrypt, Sign].
- single-node driver:
- ACLs:
C:\OtOpcUa800was protected with SYSTEM and Administrators only. The service account got RX on the install folder and Modify onpki\,data\andlogs\. - First start (
b0): the service created its PKI (owner = the service SID; the private key inherits service-M, SYSTEM-F, Administrators-F), its LocalDb and its logs. It reached Healthy with no deployment. - Seed: the service was stopped, and the
efb9197a-built seeder wrote one deployment throughIDeploymentArtifactCache. The deployment holds a raw folder, a UNS equipment with a timer virtual tag (return 800;), a 65-character key, a 54-character key, a disabled key and read grants. - Trust: the probe and CLI certificates were placed in
pki\trusted\certs.
Start/stop record
Every start was through Start-Service and every stop through Stop-Service, except b2reboot. Health means /health/ready,
/healthz and /health/active, all 200 and Healthy. ServiceLevel was read over OPC UA.
| Start | Build | How | Ready after | ServiceLevel | App cert thumbprint (SHA-1) | Stop | Exit | Secrets in logs |
|---|---|---|---|---|---|---|---|---|
| b0 | known-good | Start-Service, empty LocalDb | 13.4 s | — | CEF9D91F8CBC0EE78B317D3AA51EB40CD9C41ACA (created) |
1.6 s | 0 | 0 |
| b1 | known-good | Start-Service | 12.4 s | 250 | same | 1.6 s | 0 | 0 |
| b2reboot | known-good | SCM auto-start after the host reboot | — | — | same | 1.6 s | 0 | 0 |
| b3, b4 | known-good | Start-Service (b4 after re-seeding the negative keys) | 8.4 s, 8.3 s | 250 | same | 1.6 s | 0 | 0 |
| c1 | candidate | Start-Service after the in-place upgrade | 15.3 s | 255 | same | 1.8 s | 0 | 0 |
| c2 | candidate | Start-Service | 15.1 s | 255 | same | 1.8 s | 0 | 0 |
| c3 | candidate | Start-Service, Scripting:Runner=OutOfProcess with a worker identity |
15.1 s | 255 | same | 1.8 s | 0 | 0 |
| r1 | known-good (rollback) | Start-Service | 8.5 s | 250 | same | 1.6 s | 0 | 0 |
Certificate: the DER SHA-256 was F38E6B2E…3F845 at every start.
PKI tree: compare-b2reboot-…-r1.json digests every PKI path with its SHA-256 and SDDL. The digest was identical
(c8517c1d0298…, 13 entries) at b2reboot, b3, b4, c1, c2 and r1. At c3 the only difference was the inherited
(D;…;CC;;;<worker SID>) deny ACE from the isolation installer; file hashes were unchanged, and the ACE was gone again at r1.
b1 had 10 entries; the 3 later ones are pki\rejected\…, the untrusted test client that the SDK filed as rejected.
Other observations:
- The LocalDb file ACLs were the same across all starts.
- Each stop left no leftover host process and no leftover workers.
- The scans covered the rolling logs, the Serilog selflog and the Application event log, with redacted copies kept. They found 0 occurrences of any task secret: the API keys and their secret parts, the pepper, the fetch key and the KEK.
- ServiceLevel is 250 on the known-good build and 255 on the candidate, which matches vd03's 255 on
5fb4b4db. - Readiness took about 15 s on the candidate against about 8.4 s on the known-good build after the reboot. This is an observation, not a gate criterion.
Protocol checks (same case set at every checked start; checks-*.json)
Every check used the SDK 2.0.1 probe with peer auto-accept off, Basic256Sha256. "Gating" cases must all pass.
| Case | Known-good (b4, r1) | Candidate (c1, c2, c3) |
|---|---|---|
| SignAndEncrypt + Sign, 54-char valid key | admitted; State 0 Good, ServiceLevel 250, VT 800 Good | admitted; ServiceLevel 255, VT 800 Good |
| SignAndEncrypt + Sign, 65-char valid key (production form) | BadIdentityTokenInvalid (#880, expected denial) |
admitted, Good |
| wrong secret (Sign + SignAndEncrypt), unknown id, disabled key, 54-char | BadIdentityTokenRejected |
BadIdentityTokenRejected |
| wrong secret / unknown id, 65-char | BadIdentityTokenInvalid |
BadIdentityTokenRejected |
| anonymous | denied (BadIdentityTokenInvalid; not advertised) |
denied |
| untrusted client, Sign + SignAndEncrypt | BadSecurityChecksFailed |
BadSecurityChecksFailed |
| untrusted server (client trust list emptied) | BadCertificateUntrusted |
BadCertificateUntrusted |
actual CLI read ServerStatus, SignAndEncrypt + API key |
SDK 1.5 CLI 65-char Good; SDK 2 CLI 54-char Good | SDK 1.5 CLI 65-char Good; SDK 2 CLI 54- and 65-char Good |
| finding: SDK 2 CLI with the 65-char key | BadIdentityTokenInvalid (#880) |
Good |
| finding: advertised endpoints | None/None + Sign + SignAndEncrypt (#879) | same |
| finding: session over the None endpoint, valid key | admitted (#879) | admitted |
Step 3: ScriptWorker (isolation-*.json)
The out-of-process runner is opt-in, and no deployed node configures it; vd03 runs InProcess. On windev:
- Package. The candidate's
script-worker\holds the self-contained worker apphost and is 275 files. - The repository installer fails (#883). This is the step #794 requires.
- pwsh 7:
-WhatIflists 7 steps and changes nothing. The real run creates the anchor service and the PE cache, then throwsDirectoryInfo … GetAccessControl. - Windows PowerShell 5.1: it throws
property 'Count' cannot be found(StrictMode) at PeEntriesDir. -Uninstallfails the same way, so the partial state could not be rolled back with the tool.
- pwsh 7:
- With the #883 patch (
isolation-installer-883.diff):- Install and re-apply. Apply ran under pwsh, then re-apply under 5.1 left all 7 steps
unchanged. - Anchor service.
OtOpcUaScriptWorkeris DISABLED, asNT SERVICE\OtOpcUaScriptWorker, SIDS-1-5-80-1846135827-…-3023753135(matches #794). - Folder ACLs. The install folder has
(OI)(CI)(DENY)(S,RD)for the worker andscript-worker\has(RX). - PE cache. It is protected with SYSTEM, Administrators and the host account at F;
pe\adds the worker at M. - Firewall.
OtOpcUa-ScriptWorker-BlockOutboundis Outbound/Block/Any on the worker exe. - Service environment. It gained exactly
Scripting__Worker__IdentityandScripting__Worker__PeCacheDir, plus the gate'sScripting__Runner=OutOfProcess; the other entries were kept.
- Install and re-apply. Apply ran under pwsh, then re-apply under 5.1 left all 7 steps
- Runtime under the real service (c3).
- Worker process.
ScriptWorker[0] startedas the child of the host, run as a restricted copy ofNT SERVICE\OtOpcUaHost(IsTokenRestricted=true, 1 privilege). - Window-station grant. It works under a real service session: coreclr loaded, which windev over SSH could not prove for #794.
- Pipe ACL. The
otopcua-sw-*pipe refuses the elevated administrator (UnauthorizedAccessException). - Health.
script-runnerreports Healthy "Out-of-process script runner ready", with the PE cache on and 1pe\entry. - Virtual tag. It evaluated out of process to 800 Good.
- Stop. No worker survived the stop.
- Not exercised: dropped-packet firewall logging.
- Worker process.
- Uninstall.
-Uninstallunder 5.1 and then again under pwsh left every stepunchanged. The anchor and the firewall rule were gone, the environment digest was back tofd534f8d…, and the install-folder ACL was back to its original state.
Upgrade and rollback mechanics (upgrade.json, rollback.json)
Upgrade, with the service stopped:
reg exportof the service key, and arobocopy /E /COPYALL /DCOPY:DATbackup (1386 files).- The candidate was staged in a separate folder and given the install-folder DACL.
- The box's
appsettings*.jsonwere carried over; they are byte-identical to the candidate publisher's. pki\,data\andlogs\were carried over with/COPYALL.- The folders were swapped with
Move-Item. The servicePathNamewas unchanged.
Rollback, with the service stopped:
- The candidate was moved to quarantine and the known-good folder was moved back.
- The current
pki\anddata\were carried back with/COPYALL. No data rollback: the candidate-written LocalDb opened cleanly in the known-good build, as on vd03.
Cleanup
These were removed:
- the service, with
sc.exe delete; - the
OtOpcUaScriptWorkeranchor and its firewall rule; C:\OtOpcUa800;- the gate scripts in the profile.
Do not use Uninstall-Services.ps1 on windev: it also stops and deletes OtOpcUaWonderwareHistorian.
After cleanup:
%LOCALAPPDATA%\OtOpcUaClientwas restored from its backup (itssettings.jsonandlogs\are from 2026-05-24).- The firewall rule count is back to 876.
- MxAccessGw is Running with
:5130/health/ready200, and OtOpcUaWonderwareHistorian is Running.
Residual: the virtual-account profile C:\Windows\ServiceProfiles\OtOpcUaHost, with its ProfileList entry, stays loaded
until the next reboot. Remove it after that with
Get-CimInstance Win32_UserProfile | ? SID -eq 'S-1-5-80-3928861625-3486739760-3962757976-1980707066-3667558059' | Remove-CimInstance.
gate-run.log lists every phase invocation and its exit code. The first Prepare line reported exit 0 even though the wrapper
had failed (the remote $PSStyle quoting). That attempt stopped before creating the service, and the phase was re-run.
Real-world cross-reference
wonder-app-vd03 (DEV) has run master 5fb4b4db on SDK 2.0.1 since 2026-11-12. It reused the app cert A873DF51… and reports
ServiceLevel 255 with Good reads; see #782 and docs/deploy/vd03-upgrade-b8b4c18f-to-master.md. That record is consistent
with this run but is not a controlled install/rollback.