feat(scripting): out-of-process pre-emptible script runner behind Scripting:Runner (lmxopcua#780 P1)
v2-ci / unit-tests (push) Waiting to run

Opt-in (Scripting:Runner=OutOfProcess; InProcess stays the default and its code path is unchanged). Implements
the P1 slice of docs/plans/2026-11-09-out-of-process-script-runner-design.md with the owner's §14 decisions:

- Options: Scripting:Runner (InProcess | Shadow | OutOfProcess), Scripting:Runner:Seams, Scripting:Worker:*
  (Shards 2, HardGrace max(1 s,T), HeapLimitBytes 512 MiB, QuarantineAfterKills 2, UnavailableGrace 5 s,
  IsolationProbationRuns 20, PeCacheDir validated absolute + outside the content root (fail closed),
  ExecutablePath). Shadow is not implemented yet: runs InProcess with a startup Warning.
- Core.Scripting: IScriptRunner seam + InProcessScriptRunner (today's guard, the parity baseline).
- Core.Scripting.Protocol: protobuf envelopes, 4-byte LE framing (zero/oversize/truncated/garbage refused),
  Variant carrying the exact CLR type; unsupported types are explicit failures.
- ScriptWorker app: compiles in the worker, P-thread scheduler, host-driven soft cancel, fail-fast
  watchdog, heartbeats, ctx.Logger forwarding, exits on pipe EOF / parent gone, forbidden-assembly
  self-check; Akka and Grpc.Core.Api excluded from its folder. Shipped in script-worker/ (Build + Publish).
- Core.Scripting.OutOfProcess: K shards + isolation worker, soft T / hard T+G kill, attribution
  (deadline kill = one key; crash = suspects -> isolation; isolation crash = strike), kill-strike
  quarantine, backoff 250 ms -> 30 s, fail closed, cleared allow-listed env, nonce + pid handshake on a
  current-user pipe, Windows Job Object.
- VirtualTag + Calculation seams use the runner when selected; a runner-side failure (worker restarted /
  runner unavailable) publishes BadResourceUnavailable (0x80040000) — ScadaBridge#112. Script faults keep
  BadInternalError. The scripted-alarm seam stays in process (alarms last).

Tests: new ZB.MOM.WW.OtOpcUa.ScriptWorker.Tests (Category=ScriptWorker, real processes; green on macOS and
on windev/Windows), seam tests in Host/Runtime/Calculation suites; mutation-checked.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 23:31:51 -05:00
parent f5c30ed0ef
commit 69a6e0b4c8
49 changed files with 5107 additions and 25 deletions
@@ -0,0 +1,116 @@
using System.Buffers;
using System.Buffers.Binary;
using Google.Protobuf;
using ZB.MOM.WW.OtOpcUa.Core.Scripting.Protocol.V1;
namespace ZB.MOM.WW.OtOpcUa.Core.Scripting.Protocol;
/// <summary>
/// Length-prefixed protobuf framing for the ScriptWorker pipe (lmxopcua#780, design §4.2): a 4-byte little-endian
/// length, then one serialized <see cref="Envelope"/>. Copies mxaccessgw's <c>WorkerFrameProtocol</c> discipline:
/// zero-length, oversize, truncated and unparseable frames are all a <see cref="ScriptWorkerProtocolException"/>,
/// after which the stream is unusable and the peer is dropped.
/// </summary>
public static class ScriptWorkerFrameCodec
{
/// <summary>The protocol version both ends put on every envelope.</summary>
public const uint ProtocolVersion = 1;
/// <summary>Default frame cap: 4 MiB.</summary>
public const int DefaultMaxFrameBytes = 4 * 1024 * 1024;
/// <summary>Writes one frame. Not thread-safe: callers serialise writes.</summary>
/// <param name="stream">The pipe.</param>
/// <param name="envelope">The envelope.</param>
/// <param name="maxFrameBytes">The negotiated cap; a larger envelope is refused before anything is written.</param>
/// <param name="ct">Cancellation.</param>
/// <returns>A task that completes when the frame is written (not flushed).</returns>
public static async ValueTask WriteAsync(Stream stream, Envelope envelope, int maxFrameBytes, CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(stream);
ArgumentNullException.ThrowIfNull(envelope);
var size = envelope.CalculateSize();
if (size == 0) throw new ScriptWorkerProtocolException("refusing to write an empty frame");
if (size > maxFrameBytes)
throw new ScriptWorkerProtocolException($"frame of {size} bytes exceeds the {maxFrameBytes}-byte cap");
var buffer = ArrayPool<byte>.Shared.Rent(size + 4);
try
{
BinaryPrimitives.WriteInt32LittleEndian(buffer, size);
envelope.WriteTo(new Span<byte>(buffer, 4, size));
await stream.WriteAsync(buffer.AsMemory(0, size + 4), ct).ConfigureAwait(false);
}
finally
{
ArrayPool<byte>.Shared.Return(buffer);
}
}
/// <summary>Reads one frame.</summary>
/// <param name="stream">The pipe.</param>
/// <param name="maxFrameBytes">The negotiated cap.</param>
/// <param name="ct">Cancellation.</param>
/// <returns>The envelope, or null on a clean end of stream at a frame boundary.</returns>
/// <exception cref="ScriptWorkerProtocolException">A zero-length, oversize, truncated or unparseable frame.</exception>
public static async ValueTask<Envelope?> ReadAsync(Stream stream, int maxFrameBytes, CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(stream);
var header = new byte[4];
var got = await ReadFullyAsync(stream, header, ct).ConfigureAwait(false);
if (got == 0) return null;
if (got < 4) throw new ScriptWorkerProtocolException("truncated frame header");
var size = BinaryPrimitives.ReadInt32LittleEndian(header);
if (size == 0) throw new ScriptWorkerProtocolException("zero-length frame");
if (size < 0 || size > maxFrameBytes)
throw new ScriptWorkerProtocolException($"frame length {size} is outside 1..{maxFrameBytes}");
var body = ArrayPool<byte>.Shared.Rent(size);
try
{
var read = await ReadFullyAsync(stream, body.AsMemory(0, size), ct).ConfigureAwait(false);
if (read < size) throw new ScriptWorkerProtocolException($"truncated frame: {read} of {size} bytes");
Envelope envelope;
try
{
envelope = Envelope.Parser.ParseFrom(body, 0, size);
}
catch (InvalidProtocolBufferException ex)
{
throw new ScriptWorkerProtocolException($"unparseable frame: {ex.Message}", ex);
}
if (envelope.ProtocolVersion != ProtocolVersion)
throw new ScriptWorkerProtocolException(
$"protocol version {envelope.ProtocolVersion} is not the supported {ProtocolVersion}");
if (envelope.BodyCase == Envelope.BodyOneofCase.None)
throw new ScriptWorkerProtocolException("frame carries no body");
return envelope;
}
finally
{
ArrayPool<byte>.Shared.Return(body);
}
}
private static async ValueTask<int> ReadFullyAsync(Stream stream, Memory<byte> buffer, CancellationToken ct)
{
var total = 0;
while (total < buffer.Length)
{
var n = await stream.ReadAsync(buffer[total..], ct).ConfigureAwait(false);
if (n == 0) break;
total += n;
}
return total;
}
}
/// <summary>A framing or schema violation on the ScriptWorker pipe; the connection is dropped.</summary>
public sealed class ScriptWorkerProtocolException : Exception
{
/// <summary>Initializes the exception.</summary>
/// <param name="message">What was wrong.</param>
/// <param name="inner">The underlying parser error, if any.</param>
public ScriptWorkerProtocolException(string message, Exception? inner = null) : base(message, inner) { }
}