diff --git a/docs/plans/2026-11-09-out-of-process-script-runner-design.md b/docs/plans/2026-11-09-out-of-process-script-runner-design.md index b48d3db87..f9bd8e713 100644 --- a/docs/plans/2026-11-09-out-of-process-script-runner-design.md +++ b/docs/plans/2026-11-09-out-of-process-script-runner-design.md @@ -884,6 +884,58 @@ Both misses → lmxopcua#789 (the §4.5 batching and per-eval allocation work). The host load timeout still counts the time a load waits behind other compiles → lmxopcua#790. +**windev re-measure, 2026-11-11 (lmxopcua#789, after the §12.2–§12.3 changes; budget revision applied).** +- **windev is a VM:** VMware20,1 on an Intel Xeon E5-2697 v4 @ 2.30 GHz, 36 logical CPUs, 32 GB RAM, Windows, + .NET 10.0.8. The box was otherwise idle (about 4 % total CPU before the run). The issue's "re-measure on a non-VM + box" request is therefore still open; these are VM figures. +- Run from `C:\build\otopcua-842` (a `git archive` of the branch head) with `-c Release`, `OTOPCUA_SCRIPT_PERF=1`, + 10 s phases, filtered to `Throughput_and_latency_against_the_accepted_gate` and `Restart_rss_and_kill_latency`. + The restart test ran twice; both figures are given. +- Framework-dependent worker built beside the tests (`dotnet exec`), as in the earlier windev rows. + +| Measure (K = 2 unless stated) | windev 2026-11-11 | §8.2 budget | Verdict | +|---|---:|---|---| +| Added latency p50, idle sequential | 581 µs (K=1 490, K=4 537; InProcess 134) | ≤ 150 µs | MISS | +| p99 at paced 10k/s | 3,220 µs (K=1 5,380, K=4 1,358) | ≤ 2 ms | MISS at K=2 (was 1,676 µs in §12.2) | +| Host CPU at 10k/s, vs InProcess | +131 % (raw 141 %, InProcess 10 %) | ≤ 25 % | MISS (not tuned in this task) | +| Closed loop ×64 | 54,152 evals/s, 0 not ok | gate ≥ 5,000 | PASS | +| Kill latency after T+G (5 runs × 2) | 16 / 11 / 10 / 11 / 12 and 16 / 12 / 8 / 11 / 12 ms | ≤ 100 ms | PASS | +| Kill both shards → every script's next correct result (1,000 cached) | 2,135 ms and 2,117 ms | ≤ 2,000 ms | MISS by about 6 % | +| Warm restart, 1,000 cached, fresh runner | 1,847 ms and 1,829 ms | ≤ 2,000 ms | PASS | +| Worker RSS after first eval, compiled (max of 2 shards) | 160 MB (both runs) | ≤ 200 MB + scripts | PASS | +| Worker RSS after first eval, warm PE cache, no compile | 36 MB (both runs) | ≤ 200 MB + scripts | PASS | +| Worker RSS, 500 scripts compiled per shard | 236 / 262 MB | — (200 MB + scripts) | — | +| Worker RSS, 500 scripts loaded from the cache per shard | 93 / 96 MB | — | — | +| 1,000 scripts, empty cache, all first results | 12,380 / 12,800 ms | — | — | + +Notes: +- Under the revised budget, RSS passes. Kill latency stays fixed since §12.2: the earlier 210 ms was compile time. +- Kill-both-shards is about 120 ms over the line. It includes the 250 ms first restart backoff and the warm-cache + load of 1,000 PEs. The warm-cache restart alone takes about 1.83 s on this VM, so the kill path adds only about + 300 ms on top. +- Idle p50 and host CPU are the overlapped-I/O path that §12.2 left open; CPU tuning is deliberately not part of + this re-measure. The K=2 p99 at 10k/s reads worse than §12.2's 1,676 µs. K=4 is 1,358 µs in the same run, so this + is likely VM scheduling noise on a single run, but it was not re-run to confirm. + +**Windows `ScriptWorker.Tests` run, 2026-11-11 (lmxopcua#842).** `--filter Category=ScriptWorker`, Debug, on the +same VM. This is the first Windows run of the #827 soft-clock tests, which freeze the worker with `NtSuspendProcess` +/ `NtResumeProcess`. +- From `C:\build\otopcua-842`: **107 passed, 0 failed, 2 skipped (109)**, 13 m 37 s, 0 build warnings. + - All 13 `WorkerSoftClockTests` cases passed, including the 5 that suspend a real worker process: stalled > T, + stalled past T+G, late-starting runaway, Shadow under a stall (two freezes), and a notice that arrives after the + transport deadline. No code change was needed. + - Skipped by design: `A_published_self_contained_worker_runs_as_its_own_process` (needs + `OTOPCUA_SCRIPTWORKER_TEST_PATH`, §12.1) and the off-Windows identity refusal. +- From `C:\Users\dohertj2\otopcua-842`: 106 / **1 failed** / 2. The failure is + `WorkerIsolationTests.A_worker_under_the_identity_runs_scripts_with_a_restricted_token_and_a_plain_one_does_not`: + "The application to execute does not exist: …\script-worker\ZB.MOM.WW.OtOpcUa.ScriptWorker.dll", exit + `0x80008081`. + - **Cause: the test environment, not the code.** The restricted worker token cannot read the worker dll inside a + user profile, whose ACL is SYSTEM, Administrators and the owner only. That is the access §6 intends. + - In production the installer grants the identity read+execute on `script-worker\` (§12.5). + - Run the Windows suite from a directory the `BUILTIN\Users` group can read, such as `C:\build`. The §12.5 green + run used that. + ### 12.2 lmxopcua#789 / #790 / #788 record (2026-11-10) **What cost the host CPU (#789).** With `DOTNET_ThreadPool_UnfairSemaphoreSpinLimit=0` a scratch probe showed the