From 464b699e18ce51a45da3035bf97672d0ad00ece1 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Wed, 11 Nov 2026 22:28:58 -0500 Subject: [PATCH] =?UTF-8?q?docs(scripting):=20#789=20=C2=A712.7=20?= =?UTF-8?q?=E2=80=94=20dropping=20the=20result-dispatcher=20hop,=20measure?= =?UTF-8?q?d=20and=20not=20kept?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Measured on windev (3 alternated sessions) and macOS: a pool resume in place of the dispatcher, and an inline-on-reader upper bound, with the harness as committed and with its paced continuation made synchronous. - pool: 10k/s host CPU unchanged, idle p50 and idle CPU worse (macOS idle CPU x3 from pool spin); only the closed loop gains. Not kept. - inline (unsafe for every seam; the ceiling for a VirtualTag-only opt-in): ~8 points of a core at 10k/s raw and ~50 us idle p50 on windev with a cheap continuation, nothing with the harness as committed. Both budgets still missed by a wide margin, so the opt-in was not built. No code change. Records the remaining attribution (harness pool hop ~15 points raw; reader wake + syscalls; worker side untouched). Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk --- ...-09-out-of-process-script-runner-design.md | 102 +++++++++++++++++- 1 file changed, 101 insertions(+), 1 deletion(-) diff --git a/docs/plans/2026-11-09-out-of-process-script-runner-design.md b/docs/plans/2026-11-09-out-of-process-script-runner-design.md index 4e6f19209..3b2daa901 100644 --- a/docs/plans/2026-11-09-out-of-process-script-runner-design.md +++ b/docs/plans/2026-11-09-out-of-process-script-runner-design.md @@ -1383,7 +1383,107 @@ and the restricted-token worker. On macOS: 104 passed, 5 skipped (109). Unit cla **Still missed:** Windows host CPU vs InProcess (+53 to +78 against ≤ 25) and Windows idle p50 (363–386 µs against ≤ 150 µs; InProcess alone is about 105 µs on this VM). Next candidates: drop the dispatcher wake when the caller's continuation is known to be cheap (needs a seam-level guarantee, the reason it exists is a slow consumer), or measure -on non-VM Windows hardware before tuning further. +on non-VM Windows hardware before tuning further. (The first candidate was measured and dropped: §12.7.) + +### 12.7 lmxopcua#789: dropping the result-dispatcher hop, measured and not kept (2026-11-11) + +**Question.** A worker result wakes three things in turn: the host reader, then the `ResultDispatcher` thread, then the +caller's continuation. Can the dispatcher wake be removed without the caller's code running on the reader? + +**Where callers resume today.** +- Every production consumer resumes synchronously on whichever thread completes `PendingEval.Outcome`: + - `AwaitSubmittedAsync`, the Calculation evaluator, the VirtualTag evaluator, the alarm engine and + `ShadowScriptRunner` all use `ConfigureAwait(false)`. `AwaitSubmittedAsync` goes through `Outcome.WaitAsync(ct)` when + the token can be cancelled, which also resumes inline in practice. + - `ScriptBatchCoalescer.Link` and `VirtualTagActor` use `ContinueWith(ExecuteSynchronously)`. + - The coalescer's completion source has no `RunContinuationsAsynchronously`. +- So that thread runs the runner's mapping (`OutOfProcessScriptRunner.Complete`) and then the seam's own code. The Calculation + pump publishes inline after its await. The alarm engine runs its state machine and persists. +- The runner's mapping is not safe on the reader: + - A soft-timeout answer calls `connection.Cancel`, a pipe write that blocks on a full pipe. + - `OutOfProcessScriptRunner.Complete` can quarantine a key and log, and can log the return from isolation. +- The dispatcher carries only results and the worker's soft-deadline notice (`SoftExpired`). Host-clock soft timeouts + and transport timeouts already complete `Outcome` on the timer thread (`PendingEval.OnTimer`), so their `Cancel` + write never touches the reader. +- **One seam would tolerate running on the reader: VirtualTag.** After the coalescer's `Settle` and the evaluator's + `ToVirtualTagEvalResult`, `VirtualTagActor` only `Tell`s itself; `ApplyEvaluation` runs later on the Akka dispatcher. + A per-submission "resume on the reader" flag set only by that seam could complete real results inline. Soft-deadline + notices and worker deaths would still go through the dispatcher, and the runner's rare quarantine and isolation log + lines would move to the pool. That cheapness is a property of code in three assemblies (`Core.Scripting`, `Host`, + `Runtime`), and nothing would enforce it. Its value was measured before building it (the "inline" rows below). + +**Variants**, selected by a temporary environment switch in `WorkerConnection`. None was committed. +- **pool**: no dispatcher. `PendingEval` completes `Outcome` with `RunContinuationsAsynchronously`, so the mapping and + the caller resume on the thread pool. This is the pre-§12.2 path, now with blocking pipe I/O. It is safe and costs the + same number of wakes as the dispatcher, with a pool thread in place of the dedicated one. +- **inline**: no dispatcher. `Outcome` completes on the reader, so the mapping and the caller's continuation run there. + It is **not shippable for every seam**: it is the mutation that §12.2's test caught. It was measured as the most that + removing the hop can buy, which is also the most the VirtualTag opt-in above could buy. +- The harness's paced consumer is `ContinueWith(…, TaskScheduler.Default)` with no `ExecuteSynchronously`. That is a pool + hop of its own per result, and it hides the dispatcher in the 10k/s figure: it is one of the hops §12.6 counts in the + ~60 µs. So the last session repeated inline against the dispatcher with that continuation made `ExecuteSynchronously` + (temporary switch `OTOPCUA_SCRIPT_PERF_SYNC=1`, not committed). That is the shape of the VirtualTag seam (a cheap + continuation, no pool hop). + +**windev** (same VM as §12.6, idle, Release, `Throughput_and_latency_against_the_accepted_gate`, 10 s phases, +`master` = `31be927f`, K = 2, three sessions; within a session the runs alternated): + +| K = 2 | dispatcher | pool | inline | +|---|---:|---:|---:| +| *Session 1, harness as committed*: 10k/s vs InProcess | +79 / +67 / +55 | +59 / +64 / +72 | — | +| … idle p50 | 366 / 328 / 366 µs | 386 / 405 / 377 µs | — | +| … idle host CPU | 40 / 36 / 33 % | 50 / 49 / 53 % | — | +| … closed loop ×64 | 51k / 58k / 59k | 88k / 82k / 86k | — | +| *Session 2, harness as committed*: 10k/s raw (vs InProcess) | 58 / 88 / 65 % (+52 / +81 / +59) | — | 71 / 85 / 66 % (+52 / +70 / +56) | +| … idle p50 | 376 / 390 / 318 µs | — | 316 / 319 / 296 µs | +| … idle host CPU | 31 / 38 / 32 % | — | 32 / 29 / 25 % | +| *Session 3, paced continuation synchronous*: 10k/s raw (vs InProcess) | 57 / 51 / 60 % (+46 / +22 / +32) | — | 50 / 49 / 46 % (+41 / +37 / +35) | +| … idle p50 | 388 / 392 / 366 µs | — | 321 / 334 / 344 µs | +| … p99 at 10k/s | 803 / 755 / 1,180 µs | — | 668 / 716 / 732 µs | + +In session 3 the InProcess baseline at 10k/s read 11 / 29 / 27 % in the dispatcher runs against 9 / 12 / 11 % in the +inline runs. That makes the "vs InProcess" column noisy, so the raw host CPU is the figure to compare. + +**macOS** (2 runs per variant, alternated; load average 3–19 from other sessions on the box): +- Harness as committed: + - dispatcher: idle p50 25–28 µs, idle host CPU 51–63 %, 10k/s vs InProcess −5 to +19. + - pool: idle p50 35 / 25 µs, **idle host CPU 168 / 201 %** (thread-pool spin, as in §12.2), 10k/s −4 / +17. + - inline: idle p50 20 / 25 µs, idle CPU 53 / 44 %, 10k/s +11 / +13. +- Paced continuation synchronous: + - dispatcher: 10k/s raw 61 / 59 %, idle CPU 55 / 59 %. + - inline: 10k/s raw 53 / 52 %, idle CPU 46 / 46 %. + +**Reading.** +- **pool is not kept.** + - Its 10k/s figure sits inside the dispatcher's spread. + - It is worse where the dispatcher was meant to help. On windev, in session 1, idle p50 is +20 µs (median) or +36 µs + (mean), and idle CPU is about +15 points. On macOS, idle CPU is three times higher. + - Its one gain is the closed loop: ~85k against ~56k evals/s on windev. The dispatcher's ~56k is already 11 times + the accepted gate. +- **The most any dispatcher removal can buy is small.** + - Against the harness as committed, inline does not move 10k/s host CPU. Its spread overlaps the dispatcher's. + - With a cheap synchronous continuation (session 3), inline saves about 8 points of one core at 10k/s raw on windev + (mean 48 against 56 %; the ranges are 46–50 and 51–60) and about 7 on macOS. + - Idle p50 drops by about 50 µs on windev, consistently but with three runs per variant (sessions 2–3: 296–344 against + 318–392 µs). On macOS it drops by a few µs. + - Even at that bound Windows misses both budgets by a wide margin: about +35 to +41 against ≤ 25 at 10k/s, and an + idle p50 above 300 µs against ≤ 150 µs (InProcess alone is about 105 µs on this VM). +- **The VirtualTag opt-in was therefore not built.** It would deliver at most that bound, and only on the VirtualTag + seam. In exchange, the reader would come to rely on an unenforced "cheap continuation" property spread across three + assemblies. A later edit after any of those awaits would run on the reader unnoticed, which is exactly the + heartbeat-starvation failure §12.2 and §12.6 had to design out. If the owner wants the ~8 points regardless, the shape + is the one above, plus a test that makes the VirtualTag continuation block and expects the heartbeat to survive. +- **Kept:** nothing. The code is unchanged, and the dispatcher stays as §12.2 describes it. No tests were re-run, since + no code changed. The experiment builds were only measured. + +**Still missed (unchanged from §12.6):** +- Windows host CPU at 10k/s: +52 to +81 against ≤ 25 with the harness as committed. The harness's own pool hop accounts + for about 15 points of raw host CPU (session 3's dispatcher runs, mean 56 %, against sessions 1–2, mean 72 %). +- Windows idle p50: about 320–390 µs against ≤ 150 µs. + +What is left after this slice: the reader's own wake per frame and the syscalls behind it (about 60 µs per evaluation on +this VM against about 13 µs on macOS, §12.6), and the worker side of the round trip (its reader, the dedicated +evaluation threads and its writer), which this slice did not touch. A non-VM Windows box is still unmeasured. ---