feat(scripting): batch submit, exact per-item deadlines, scripted-alarm seam on the out-of-process runner (lmxopcua#780)
v2-ci / unit-tests (push) Waiting to run

- IScriptRunner.RunBatch: one EvalBatch frame per worker for every evaluation ready to run; per-item, streamed results.
  The default (InProcessScriptRunner) is RunAsync per item. Used by the Calculation pump (dequeue all, split at a
  repeated tag, publish in dequeue order), the VirtualTag seam (ScriptBatchCoalescer, no timer) and the scripted-alarm
  engine (whole dirty set). In process the pump and the engine stay one at a time, unchanged.
- Exact per-item deadlines: the worker hands work to an idle evaluation thread or defers it (EvalResults
  deferred_eval_ids); the host stops that run's clock and restarts T then G when it starts (started_eval_ids). A run
  queued behind a runaway is neither timed out nor struck and is a victim, not the culprit, at the kill.
- Scripted-alarm seam behind Scripting:Runner:Seams; unset Seams now means VirtualTag + Calculation, so alarms run out of
  process only when listed. Load still compiles each predicate (to PE bytes only) so a broken one fails the load; passes
  batch and apply in the serial order under the gate. Script faults freeze the condition BadInternalError; runner faults
  freeze it BadResourceUnavailable and re-evaluate on recovery; a dead worker never clears an alarm. IScriptRunnerProvider
  carries the runner to DriverHostActor.
- Perf harness: batch throughput (RunBatch vs RunAsync x n) and alarm-pass latency (in process / batched / serial);
  design §12.3 records macOS + windev. On windev the serial out-of-process alarm pass misses 250 ms from ~500 alarms;
  batched p99 <= 53 ms at 1,000.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-10 06:37:36 -05:00
parent 11a5c1d4e2
commit 0796928504
33 changed files with 2168 additions and 117 deletions
@@ -0,0 +1,106 @@
namespace ZB.MOM.WW.OtOpcUa.Core.Scripting;
/// <summary>
/// Opportunistic batching for a seam whose evaluations are submitted independently — one virtual-tag actor each —
/// but often become ready together (one upstream burst fans out to many tags): evaluations submitted while another
/// submission is in progress are handed to <see cref="IScriptRunner.RunBatch"/> together, in arrival order, by the
/// submitter already holding the flush (lmxopcua#780, design §4.5).
/// </summary>
/// <remarks>
/// <para>There is no timer: an evaluation on an idle seam is submitted at once by its own caller, so batching never
/// adds latency. Per-item semantics are <see cref="IScriptRunner.RunBatch"/>'s — each item keeps its own budget, slot
/// and blame.</para>
/// <para>Each caller's task completes synchronously on the thread that completes its run (the runner's result
/// dispatcher), exactly as awaiting <see cref="IScriptRunner.RunAsync"/> directly does, so there is no extra thread
/// hop.</para>
/// <para>A key submitted twice concurrently is refused for the second exactly as two concurrent
/// <see cref="IScriptRunner.RunAsync"/> calls would be; the virtual-tag actor never has two evaluations in flight.</para>
/// </remarks>
public sealed class ScriptBatchCoalescer
{
private readonly IScriptRunner _runner;
private readonly object _lock = new();
private List<(ScriptInvocation Invocation, TaskCompletionSource<ScriptRunOutcome> Done)> _queue = [];
private bool _flushing;
/// <summary>Initializes the coalescer.</summary>
/// <param name="runner">The runner every batch goes to.</param>
public ScriptBatchCoalescer(IScriptRunner runner)
{
_runner = runner ?? throw new ArgumentNullException(nameof(runner));
}
/// <summary>Test hook: runs once per <see cref="IScriptRunner.RunBatch"/> call, with the batch size.</summary>
internal Action<int>? BatchSubmittedForTest { get; set; }
/// <summary>Submits one evaluation, together with any submitted concurrently.</summary>
/// <param name="invocation">What to run.</param>
/// <returns>The outcome; never faults.</returns>
public Task<ScriptRunOutcome> RunAsync(ScriptInvocation invocation)
{
ArgumentNullException.ThrowIfNull(invocation);
// Synchronous continuations: the caller resumes where its run completed, as with a direct await of RunAsync.
var done = new TaskCompletionSource<ScriptRunOutcome>(TaskCreationOptions.None);
lock (_lock)
{
_queue.Add((invocation, done));
if (_flushing) return done.Task;
_flushing = true;
}
Flush();
return done.Task;
}
private void Flush()
{
while (true)
{
List<(ScriptInvocation Invocation, TaskCompletionSource<ScriptRunOutcome> Done)> batch;
lock (_lock)
{
if (_queue.Count == 0)
{
_flushing = false;
return;
}
batch = _queue;
_queue = [];
}
IReadOnlyList<Task<ScriptRunOutcome>> outcomes;
try
{
var invocations = new ScriptInvocation[batch.Count];
for (var i = 0; i < invocations.Length; i++) invocations[i] = batch[i].Invocation;
BatchSubmittedForTest?.Invoke(invocations.Length);
outcomes = _runner.RunBatch(invocations);
}
catch (Exception ex)
{
// A defect in the runner must not strand the callers (nor the flush flag).
foreach (var (_, done) in batch) done.TrySetResult(ScriptRunOutcome.Failed($"script runner threw: {ex.Message}"));
continue;
}
for (var i = 0; i < batch.Count; i++) Link(outcomes[i], batch[i].Done);
}
}
private static void Link(Task<ScriptRunOutcome> outcome, TaskCompletionSource<ScriptRunOutcome> done)
{
if (outcome.IsCompleted)
{
Settle(outcome, done);
return;
}
outcome.ContinueWith(static (t, s) => Settle(t, (TaskCompletionSource<ScriptRunOutcome>)s!), done,
CancellationToken.None, TaskContinuationOptions.ExecuteSynchronously, TaskScheduler.Default);
}
private static void Settle(Task<ScriptRunOutcome> outcome, TaskCompletionSource<ScriptRunOutcome> done)
{
if (outcome.IsCompletedSuccessfully) done.TrySetResult(outcome.Result);
else done.TrySetResult(ScriptRunOutcome.Failed(
$"script runner threw: {outcome.Exception?.GetBaseException().Message ?? "cancelled"}"));
}
}