From d4a2aa6142d5a54c745a39bacfa7c5c5b12dc84f Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Wed, 11 Nov 2026 20:10:58 -0500 Subject: [PATCH] =?UTF-8?q?docs(scripting):=20#789=20design=20=C2=A712.6?= =?UTF-8?q?=20record=20=E2=80=94=20Windows=20blocking=20pipe=20I/O,=20wind?= =?UTF-8?q?ev=20before/after?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk --- ...-09-out-of-process-script-runner-design.md | 58 +++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/docs/plans/2026-11-09-out-of-process-script-runner-design.md b/docs/plans/2026-11-09-out-of-process-script-runner-design.md index f9bd8e713..beeadb7e3 100644 --- a/docs/plans/2026-11-09-out-of-process-script-runner-design.md +++ b/docs/plans/2026-11-09-out-of-process-script-runner-design.md @@ -965,6 +965,7 @@ on Unix by the socket engine. Changes: - A full pipe blocks only the one thread draining the writer; every other producer just queues. A peer that stops reading is killed by the heartbeat check within 3 s, which fails that write. - Windows keeps overlapped async I/O, because a synchronous full-duplex handle serialises reads against writes. + (Superseded in §12.6: Windows now blocks too, through `WindowsBlockingPipeStream`.) - **Kill via `TerminateJobObject`** on Windows. The job admits one process, so no system-wide snapshot is needed. - Measured and **not** kept for memory, because none moved the compiled-worker RSS: - ReadyToRun worker publish: 195 MB; @@ -1299,6 +1300,63 @@ Owner decisions §14 #5 and #6, as built. Operator reference: `docs/ServiceHosti - **Not verified live:** the installer on a real node. That is the QA deploy gate on lmxopcua#780's follow-up (see that issue for the checklist). +### 12.6 lmxopcua#789 host CPU on Windows: blocking pipe I/O off the completion port (2026-11-11) + +**Already in place before this slice** (checked against the issue's candidates): one timer per evaluation for T and +T + G (§12.2); the source SHA-256, its hex form and the negative-cache key are computed once per key's source +(`KeyState.Sha`), not per `RunAsync`; batch submit (§12.3) is used by the Calculation pump, the VirtualTag coalescer and +the alarm engine on the out-of-process path. The §8 paced harness phases call `RunAsync` one at a time on purpose (one +trigger per evaluation); `Batch_throughput_and_alarm_pass_latency` measures the batched path. + +**What changed.** +- **`WindowsBlockingPipeStream`** (Protocol). A .NET pipe opened `Asynchronous` completes even its synchronous + `Read`/`Write` through the thread pool's completion port, so every frame on Windows cost a completion packet plus a + pool work item. The stream issues its own overlapped `ReadFile`/`WriteFile` on the pipe handle with the + `OVERLAPPED.hEvent` low-order bit set, which tells the kernel not to queue a completion packet, and waits on that + event on the calling thread. Reads and writes stay concurrent (a handle opened for synchronous I/O would serialise + them). Dispose cancels a waiting read or write (`CancelIoEx`) and closes the pipe; a peer that goes away ends a read as + end of stream and fails a write with `IOException`, as `PipeStream` does. +- **`ScriptWorkerFrameCodec.BlockingStream(pipe)`** picks that stream on Windows and the pipe itself on Unix. Both ends + now use the dedicated blocking reader thread and the synchronous combining writer on every OS. `UseBlockingIo` and the + two async read loops are gone; `HandshakeAsync` stays for its tests. +- **1 MiB pipe buffer quota each way** (`WorkerPipeSecurity.BufferBytes`). With a quota of 0 an overlapped write stays + pending until the peer has read the bytes, so the blocking writer waited out a peer wake-up per frame. The first cut + without it halved the windev closed loop (54k → 26–27k evals/s); with it the closed loop is back. +- Tests: `BlockingPipeStreamTests` (duplex with reads blocked on both ends, a 1 MB frame, peer gone, self-dispose wakes + the reader). + +**Measured on windev** (same VM as §12.1, idle, Release, `Throughput_and_latency_against_the_accepted_gate`, 10 s +phases, base = `f109e5f2`, runs alternated base / new): + +| K = 2 | base (3 runs) | new (2 runs) | §8.2 budget | +|---|---:|---:|---| +| Host CPU at paced 10k/s, raw | 116 / 118 / 120 % | 61 / 84 % | — | +| … vs InProcess (InProcess 6–9 %) | +108 / +112 / +112 | **+53 / +78** | ≤ 25 ✗ | +| Host CPU at paced 5k/s, vs InProcess | +35 / +37 / +38 | −26 / −38 | — | +| Idle p50 (sequential) | 491 / 499 / 560 µs | 363 / 386 µs | ≤ 150 µs ✗ | +| Idle host CPU (sequential) | 57 / 61 / 66 % | 27 / 37 % | — | +| p99 at 10k/s | 2,108 / 2,264 / 2,373 µs | 878 / 944 µs | ≤ 2 ms ✓ (was ✗) | +| Closed loop ×64 | 51k / 54k / 55k evals/s | 54k / 55k evals/s | gate ≥ 5k ✓ | + +- The first cut (no buffer quota), two runs: 10k/s raw 118 / 127 %, idle p50 354 / 362 µs, closed loop 26–27k/s. +- **Attribution of what is left.** With `DOTNET_ThreadPool_UnfairSemaphoreSpinLimit=0` the new build reads 71 % raw at + 10k/s (+65) and the base 93 % (+85). So pool spinning is no longer what the remainder is made of: it is about 60 µs of + host CPU per evaluation in syscalls and thread wake-ups on this VM (reader wake, dispatcher wake, and the harness's + own `ContinueWith` pool hop per result), against about 13 µs on macOS. A non-VM Windows box is still unmeasured. + +**macOS** (Unix path unchanged; 2 runs each, alternated, load average 3–14): K = 2 at 10k/s +9 / +12 vs base +16 / +5 +(InProcess 46–57 %); idle p50 26 µs both; closed loop 365–374k vs 364–379k evals/s. No change, as expected. + +**Tests.** ScriptWorker `Category=ScriptWorker` on windev: 107 passed, 2 skipped by design (109), including all +`WorkerSoftClockTests` suspend cases, the deferred-start / runaway-in-batch deadline tests, the kill and quarantine tests +and the restricted-token worker. On macOS: 104 passed, 5 skipped (109). Unit classes `BlockingPipeStreamTests`, +`ProtocolTests` and `HandshakeTests`: 67 / 67 on both. + +**Still missed:** Windows host CPU vs InProcess (+53 to +78 against ≤ 25) and Windows idle p50 (363–386 µs against +≤ 150 µs; InProcess alone is about 105 µs on this VM). Next candidates: drop the dispatcher wake when the caller's +continuation is known to be cheap (needs a seam-level guarantee, the reason it exists is a slow consumer), or measure +on non-VM Windows hardware before tuning further. + --- ## 13. Test plan