feat(scripting): out-of-process pre-emptible script runner behind Scripting:Runner (lmxopcua#780 P1)
v2-ci / unit-tests (push) Waiting to run

Opt-in (Scripting:Runner=OutOfProcess; InProcess stays the default and its code path is unchanged). Implements
the P1 slice of docs/plans/2026-11-09-out-of-process-script-runner-design.md with the owner's §14 decisions:

- Options: Scripting:Runner (InProcess | Shadow | OutOfProcess), Scripting:Runner:Seams, Scripting:Worker:*
  (Shards 2, HardGrace max(1 s,T), HeapLimitBytes 512 MiB, QuarantineAfterKills 2, UnavailableGrace 5 s,
  IsolationProbationRuns 20, PeCacheDir validated absolute + outside the content root (fail closed),
  ExecutablePath). Shadow is not implemented yet: runs InProcess with a startup Warning.
- Core.Scripting: IScriptRunner seam + InProcessScriptRunner (today's guard, the parity baseline).
- Core.Scripting.Protocol: protobuf envelopes, 4-byte LE framing (zero/oversize/truncated/garbage refused),
  Variant carrying the exact CLR type; unsupported types are explicit failures.
- ScriptWorker app: compiles in the worker, P-thread scheduler, host-driven soft cancel, fail-fast
  watchdog, heartbeats, ctx.Logger forwarding, exits on pipe EOF / parent gone, forbidden-assembly
  self-check; Akka and Grpc.Core.Api excluded from its folder. Shipped in script-worker/ (Build + Publish).
- Core.Scripting.OutOfProcess: K shards + isolation worker, soft T / hard T+G kill, attribution
  (deadline kill = one key; crash = suspects -> isolation; isolation crash = strike), kill-strike
  quarantine, backoff 250 ms -> 30 s, fail closed, cleared allow-listed env, nonce + pid handshake on a
  current-user pipe, Windows Job Object.
- VirtualTag + Calculation seams use the runner when selected; a runner-side failure (worker restarted /
  runner unavailable) publishes BadResourceUnavailable (0x80040000) — ScadaBridge#112. Script faults keep
  BadInternalError. The scripted-alarm seam stays in process (alarms last).

Tests: new ZB.MOM.WW.OtOpcUa.ScriptWorker.Tests (Category=ScriptWorker, real processes; green on macOS and
on windev/Windows), seam tests in Host/Runtime/Calculation suites; mutation-checked.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 23:31:51 -05:00
parent f5c30ed0ef
commit 69a6e0b4c8
49 changed files with 5107 additions and 25 deletions
@@ -665,6 +665,18 @@ per-shard state, restarts in the last hour and the quarantine list. Quarantines
- Default stays `InProcess`. Enable on the docker-dev rig only.
- Gate: §13 runaway / containment tests green on Linux (docker-dev) **and** Windows (windev box, which has
the .NET 10 toolchain).
- **Partly landed in lmxopcua#780 (P1 slice):** `Scripting:Runner` / `Scripting:Runner:Seams` / `Scripting:Worker:*`
(bound, clamped, `PeCacheDir` path rule fail-closed), `IScriptRunner` + `InProcessScriptRunner`, the protobuf frame
protocol (`Core.Scripting.Protocol`), the `ScriptWorker` app (compile in the worker, P-thread scheduler, watchdog,
heartbeats, log forwarding, pipe-EOF / parent-pid exit, forbidden-assembly self-check), `OutOfProcessScriptRunner`
(K shards + isolation worker, soft/hard deadlines, attribution, kill-strike quarantine, backoff, fail closed,
Job Object on Windows), the VirtualTag and Calculation seams over it, and `BadResourceUnavailable` for
runner-unavailable / worker-restarted (ScadaBridge#112). The §13 containment cases run on macOS and Windows
(windev). Deviations: supervision is a plain object owned by DI, not `ScriptRunnerActor`/`ScriptWorkerActor`;
the per-seam batch submit (P0) is still one-at-a-time. Remaining: the HMAC'd PE cache, a self-contained worker
publish + the low-privilege account / pipe ACL / firewall rule (installer), the runner-state gauge and readiness
component, victim re-evaluation on shard return, isolation-worker idle stop, the orphan sweep by nonce file, the
scripted-alarm seam, the docker-dev enablement and the §8 perf harness.
3. **P2, shadow mode on docker-dev and QA.**
- `Shadow` runs the **in-process** evaluation as the published result, and *also* submits to the worker.
- It compares value, status class and failure kind per evaluation, and counts