feat(scripting): out-of-process pre-emptible script runner behind Scripting:Runner (lmxopcua#780 P1)
v2-ci / unit-tests (push) Waiting to run
v2-ci / unit-tests (push) Waiting to run
Opt-in (Scripting:Runner=OutOfProcess; InProcess stays the default and its code path is unchanged). Implements the P1 slice of docs/plans/2026-11-09-out-of-process-script-runner-design.md with the owner's §14 decisions: - Options: Scripting:Runner (InProcess | Shadow | OutOfProcess), Scripting:Runner:Seams, Scripting:Worker:* (Shards 2, HardGrace max(1 s,T), HeapLimitBytes 512 MiB, QuarantineAfterKills 2, UnavailableGrace 5 s, IsolationProbationRuns 20, PeCacheDir validated absolute + outside the content root (fail closed), ExecutablePath). Shadow is not implemented yet: runs InProcess with a startup Warning. - Core.Scripting: IScriptRunner seam + InProcessScriptRunner (today's guard, the parity baseline). - Core.Scripting.Protocol: protobuf envelopes, 4-byte LE framing (zero/oversize/truncated/garbage refused), Variant carrying the exact CLR type; unsupported types are explicit failures. - ScriptWorker app: compiles in the worker, P-thread scheduler, host-driven soft cancel, fail-fast watchdog, heartbeats, ctx.Logger forwarding, exits on pipe EOF / parent gone, forbidden-assembly self-check; Akka and Grpc.Core.Api excluded from its folder. Shipped in script-worker/ (Build + Publish). - Core.Scripting.OutOfProcess: K shards + isolation worker, soft T / hard T+G kill, attribution (deadline kill = one key; crash = suspects -> isolation; isolation crash = strike), kill-strike quarantine, backoff 250 ms -> 30 s, fail closed, cleared allow-listed env, nonce + pid handshake on a current-user pipe, Windows Job Object. - VirtualTag + Calculation seams use the runner when selected; a runner-side failure (worker restarted / runner unavailable) publishes BadResourceUnavailable (0x80040000) — ScadaBridge#112. Script faults keep BadInternalError. The scripted-alarm seam stays in process (alarms last). Tests: new ZB.MOM.WW.OtOpcUa.ScriptWorker.Tests (Category=ScriptWorker, real processes; green on macOS and on windev/Windows), seam tests in Host/Runtime/Calculation suites; mutation-checked. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
@@ -0,0 +1,36 @@
|
||||
using System.IO.Pipes;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Scripting.Protocol;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.ScriptWorker;
|
||||
|
||||
/// <summary>Entry point: connect to the host's pipe named in the environment and serve until told to stop.</summary>
|
||||
internal static class Program
|
||||
{
|
||||
private static async Task<int> Main()
|
||||
{
|
||||
var pipeName = Environment.GetEnvironmentVariable(ScriptWorkerEnvironment.PipeName);
|
||||
var nonce = Environment.GetEnvironmentVariable(ScriptWorkerEnvironment.Nonce);
|
||||
if (string.IsNullOrEmpty(pipeName) || string.IsNullOrEmpty(nonce))
|
||||
{
|
||||
await Console.Error.WriteLineAsync("ScriptWorker: launched without a pipe name or nonce; exiting.");
|
||||
return ScriptWorkerEnvironment.ExitProtocolFault;
|
||||
}
|
||||
_ = int.TryParse(Environment.GetEnvironmentVariable(ScriptWorkerEnvironment.ParentPid), out var parentPid);
|
||||
|
||||
// CurrentUserOnly: the client verifies the server end is owned by this account as well.
|
||||
var pipe = new NamedPipeClientStream(".", pipeName, PipeDirection.InOut,
|
||||
PipeOptions.Asynchronous | PipeOptions.CurrentUserOnly);
|
||||
try
|
||||
{
|
||||
await pipe.ConnectAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
await Console.Error.WriteLineAsync($"ScriptWorker: could not connect to the host pipe: {ex.Message}");
|
||||
return ScriptWorkerEnvironment.ExitProtocolFault;
|
||||
}
|
||||
|
||||
var host = new WorkerHost(pipe, nonce, parentPid);
|
||||
return await host.RunAsync();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,576 @@
|
||||
using System.Collections;
|
||||
using System.Collections.Concurrent;
|
||||
using System.Diagnostics;
|
||||
using System.IO.Pipes;
|
||||
using System.Threading.Channels;
|
||||
using Serilog;
|
||||
using Serilog.Core;
|
||||
using Serilog.Events;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Scripting;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Scripting.Protocol;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Scripting.Protocol.V1;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.ScriptedAlarms;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.VirtualTags;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.ScriptWorker;
|
||||
|
||||
/// <summary>
|
||||
/// One worker's whole life (lmxopcua#780, design §3.2): handshake, compile-on-load, a bounded evaluation scheduler
|
||||
/// on dedicated threads, a hard-deadline watchdog, heartbeats, and log forwarding. The host is authoritative for
|
||||
/// deadlines (it kills this process); the watchdog is the belt to that brace and fails fast slightly later, so the
|
||||
/// exit names the key that overran.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Anything that means "the host is gone" — pipe end of stream, a protocol fault, the parent pid disappearing —
|
||||
/// ends the process. The worker never outlives its host (design §3.3).
|
||||
/// </remarks>
|
||||
internal sealed class WorkerHost
|
||||
{
|
||||
/// <summary>The watchdog fails fast this long after the host's own hard deadline, so the host normally wins.</summary>
|
||||
private static readonly TimeSpan WatchdogMargin = TimeSpan.FromMilliseconds(500);
|
||||
|
||||
private readonly NamedPipeClientStream _pipe;
|
||||
private readonly string _nonce;
|
||||
private readonly int _parentPid;
|
||||
private readonly Channel<Envelope> _outbound = Channel.CreateUnbounded<Envelope>(
|
||||
new UnboundedChannelOptions { SingleReader = true });
|
||||
private readonly ConcurrentDictionary<string, LoadedScript> _scripts = new(StringComparer.Ordinal);
|
||||
private readonly ConcurrentDictionary<long, RunningEval> _running = new();
|
||||
private readonly BlockingCollection<EvalRequest> _queue = new(new ConcurrentQueue<EvalRequest>());
|
||||
private readonly ForwardingLogSink _logSink;
|
||||
private readonly ILogger _scriptLogger;
|
||||
private int _maxFrameBytes = ScriptWorkerFrameCodec.DefaultMaxFrameBytes;
|
||||
private int _queueCapacity = 4096;
|
||||
private TimeSpan _hardGrace = TimeSpan.FromSeconds(1);
|
||||
private TimeSpan _heartbeatInterval = TimeSpan.FromSeconds(1);
|
||||
private long _seq;
|
||||
|
||||
public WorkerHost(NamedPipeClientStream pipe, string nonce, int parentPid)
|
||||
{
|
||||
_pipe = pipe;
|
||||
_nonce = nonce;
|
||||
_parentPid = parentPid;
|
||||
_logSink = new ForwardingLogSink();
|
||||
_scriptLogger = new LoggerConfiguration().MinimumLevel.Verbose().WriteTo.Sink(_logSink).CreateLogger();
|
||||
}
|
||||
|
||||
public async Task<int> RunAsync()
|
||||
{
|
||||
try
|
||||
{
|
||||
var hello = await ScriptWorkerFrameCodec.ReadAsync(_pipe, ScriptWorkerFrameCodec.DefaultMaxFrameBytes);
|
||||
if (hello is null) return ScriptWorkerEnvironment.ExitPipeClosed;
|
||||
if (hello.BodyCase != Envelope.BodyOneofCase.Hello)
|
||||
throw new ScriptWorkerProtocolException($"expected Hello, got {hello.BodyCase}");
|
||||
Configure(hello.Hello);
|
||||
|
||||
// The nonce is echoed, not trusted: the host compares it with the one it put in OUR environment, which
|
||||
// proves the process on the other end of its pipe is the one it launched.
|
||||
await ScriptWorkerFrameCodec.WriteAsync(_pipe, Wrap(new Envelope
|
||||
{
|
||||
Ready = new Ready
|
||||
{
|
||||
Nonce = _nonce,
|
||||
Pid = Environment.ProcessId,
|
||||
RuntimeVersion = Environment.Version.ToString(),
|
||||
},
|
||||
}), _maxFrameBytes);
|
||||
|
||||
_logSink.Capacity = (int)Math.Max(1, hello.Hello.MaxLogLines == 0 ? 1000 : hello.Hello.MaxLogLines);
|
||||
StartBackground(Math.Max(1, (int)hello.Hello.EvalThreads));
|
||||
var writer = Task.Run(WriteLoopAsync);
|
||||
var exit = await ReadLoopAsync();
|
||||
_outbound.Writer.TryComplete();
|
||||
try { await writer.WaitAsync(TimeSpan.FromSeconds(1)); } catch { /* exiting regardless */ }
|
||||
return exit;
|
||||
}
|
||||
catch (ScriptWorkerProtocolException ex)
|
||||
{
|
||||
await Console.Error.WriteLineAsync($"ScriptWorker: protocol fault: {ex.Message}");
|
||||
return ScriptWorkerEnvironment.ExitProtocolFault;
|
||||
}
|
||||
catch (IOException)
|
||||
{
|
||||
return ScriptWorkerEnvironment.ExitPipeClosed;
|
||||
}
|
||||
}
|
||||
|
||||
private void Configure(Hello hello)
|
||||
{
|
||||
if (hello.MaxFrameBytes > 0) _maxFrameBytes = (int)Math.Min(hello.MaxFrameBytes, 64u * 1024 * 1024);
|
||||
if (hello.QueueCapacity > 0) _queueCapacity = (int)hello.QueueCapacity;
|
||||
if (hello.HardGraceMs > 0) _hardGrace = TimeSpan.FromMilliseconds(hello.HardGraceMs);
|
||||
if (hello.HeartbeatIntervalMs > 0) _heartbeatInterval = TimeSpan.FromMilliseconds(hello.HeartbeatIntervalMs);
|
||||
}
|
||||
|
||||
private void StartBackground(int evalThreads)
|
||||
{
|
||||
for (var i = 0; i < evalThreads; i++)
|
||||
{
|
||||
new Thread(EvalThreadLoop) { IsBackground = true, Name = $"ScriptWorker.Eval{i}" }.Start();
|
||||
}
|
||||
new Thread(WatchdogLoop) { IsBackground = true, Name = "ScriptWorker.Watchdog" }.Start();
|
||||
_ = Task.Run(HeartbeatLoopAsync);
|
||||
_ = Task.Run(LogFlushLoopAsync);
|
||||
if (_parentPid > 0) _ = Task.Run(ParentWatchLoopAsync);
|
||||
}
|
||||
|
||||
private async Task<int> ReadLoopAsync()
|
||||
{
|
||||
while (true)
|
||||
{
|
||||
Envelope? frame;
|
||||
try
|
||||
{
|
||||
frame = await ScriptWorkerFrameCodec.ReadAsync(_pipe, _maxFrameBytes);
|
||||
}
|
||||
catch (ScriptWorkerProtocolException ex)
|
||||
{
|
||||
await Console.Error.WriteLineAsync($"ScriptWorker: protocol fault: {ex.Message}");
|
||||
return ScriptWorkerEnvironment.ExitProtocolFault;
|
||||
}
|
||||
catch (IOException)
|
||||
{
|
||||
return ScriptWorkerEnvironment.ExitPipeClosed;
|
||||
}
|
||||
if (frame is null) return ScriptWorkerEnvironment.ExitPipeClosed; // host gone: fatal by design
|
||||
|
||||
switch (frame.BodyCase)
|
||||
{
|
||||
case Envelope.BodyOneofCase.Load:
|
||||
{
|
||||
// Compile off the reader, so evaluations for already-loaded keys keep flowing during a load.
|
||||
var load = frame.Load;
|
||||
_ = Task.Run(() => Load(load));
|
||||
break;
|
||||
}
|
||||
case Envelope.BodyOneofCase.Unload:
|
||||
foreach (var key in frame.Unload.Keys)
|
||||
if (_scripts.TryRemove(key, out var gone)) gone.Dispose();
|
||||
break;
|
||||
case Envelope.BodyOneofCase.Eval:
|
||||
foreach (var item in frame.Eval.Items) Admit(item);
|
||||
break;
|
||||
case Envelope.BodyOneofCase.Cancel:
|
||||
if (_running.TryGetValue((long)frame.Cancel.EvalId, out var toCancel))
|
||||
{
|
||||
try { toCancel.Cancellation.Cancel(); } catch (ObjectDisposedException) { /* just finished */ }
|
||||
}
|
||||
break;
|
||||
case Envelope.BodyOneofCase.DiagRequest:
|
||||
Send(new Envelope { DiagReply = Diagnostics() });
|
||||
break;
|
||||
case Envelope.BodyOneofCase.Shutdown:
|
||||
return 0;
|
||||
case Envelope.BodyOneofCase.Fault:
|
||||
await Console.Error.WriteLineAsync($"ScriptWorker: host fault: {frame.Fault.Message}");
|
||||
return ScriptWorkerEnvironment.ExitProtocolFault;
|
||||
default:
|
||||
await Console.Error.WriteLineAsync($"ScriptWorker: unexpected frame {frame.BodyCase}");
|
||||
return ScriptWorkerEnvironment.ExitProtocolFault;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void Load(LoadScripts load)
|
||||
{
|
||||
var result = new LoadResult { LoadId = load.LoadId };
|
||||
foreach (var script in load.Scripts)
|
||||
{
|
||||
var outcome = new LoadOutcome { Key = script.Key, Ok = true };
|
||||
try
|
||||
{
|
||||
var loaded = LoadedScript.Compile(script);
|
||||
if (_scripts.TryRemove(script.Key, out var previous)) previous.Dispose();
|
||||
_scripts[script.Key] = loaded;
|
||||
}
|
||||
catch (CompilationErrorException ex)
|
||||
{
|
||||
outcome.Ok = false;
|
||||
outcome.Error = new ScriptError { Kind = ScriptError.Types.Kind.Compile, Message = ex.Message };
|
||||
}
|
||||
catch (ScriptSandboxViolationException ex)
|
||||
{
|
||||
outcome.Ok = false;
|
||||
outcome.Error = new ScriptError { Kind = ScriptError.Types.Kind.Sandbox, Message = ex.Message };
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
outcome.Ok = false;
|
||||
outcome.Error = new ScriptError { Kind = ScriptError.Types.Kind.Threw, Message = ex.Message };
|
||||
}
|
||||
result.Outcomes.Add(outcome);
|
||||
}
|
||||
if (ForbiddenAssemblyLoaded() is { } forbidden) DieForbidden(forbidden);
|
||||
Send(new Envelope { LoadResult = result });
|
||||
}
|
||||
|
||||
private void Admit(EvalRequest request)
|
||||
{
|
||||
if (_queue.Count >= _queueCapacity)
|
||||
{
|
||||
SendResult(new EvalResult
|
||||
{
|
||||
EvalId = request.EvalId,
|
||||
Error = new ScriptError { Kind = ScriptError.Types.Kind.Busy, Message = "script worker busy" },
|
||||
});
|
||||
return;
|
||||
}
|
||||
_queue.Add(request);
|
||||
}
|
||||
|
||||
private void EvalThreadLoop()
|
||||
{
|
||||
foreach (var request in _queue.GetConsumingEnumerable())
|
||||
{
|
||||
EvalResult result;
|
||||
try
|
||||
{
|
||||
result = Evaluate(request);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
result = new EvalResult
|
||||
{
|
||||
EvalId = request.EvalId,
|
||||
Error = new ScriptError { Kind = ScriptError.Types.Kind.Threw, Message = ex.Message },
|
||||
};
|
||||
}
|
||||
SendResult(result);
|
||||
}
|
||||
}
|
||||
|
||||
private EvalResult Evaluate(EvalRequest request)
|
||||
{
|
||||
var result = new EvalResult { EvalId = request.EvalId };
|
||||
if (!_scripts.TryGetValue(request.Key, out var script))
|
||||
{
|
||||
result.Error = new ScriptError { Kind = ScriptError.Types.Kind.NotLoaded, Message = $"script '{request.Key}' is not loaded" };
|
||||
return result;
|
||||
}
|
||||
|
||||
var inputs = new Dictionary<string, DataValueSnapshot>(request.Inputs.Count, StringComparer.Ordinal);
|
||||
foreach (var input in request.Inputs)
|
||||
{
|
||||
string path;
|
||||
if (!string.IsNullOrEmpty(input.Path)) path = input.Path;
|
||||
else if (input.Slot < script.InputPaths.Length) path = script.InputPaths[input.Slot];
|
||||
else throw new ScriptWorkerProtocolException($"input slot {input.Slot} is not registered for '{request.Key}'");
|
||||
|
||||
if (input.Unsupported)
|
||||
{
|
||||
result.Error = new ScriptError
|
||||
{
|
||||
Kind = ScriptError.Types.Kind.UnsupportedType,
|
||||
Message = $"input '{path}' has a value of unsupported type {input.UnsupportedType}",
|
||||
};
|
||||
return result;
|
||||
}
|
||||
inputs[path] = new DataValueSnapshot(
|
||||
VariantCodec.Decode(input.Value),
|
||||
input.StatusCode,
|
||||
input.SourceTsTicks == 0 ? null : new DateTime(input.SourceTsTicks, DateTimeKind.Utc),
|
||||
new DateTime(input.ServerTsTicks, DateTimeKind.Utc));
|
||||
}
|
||||
|
||||
// ctx.Now follows the HOST clock (design §4.3) but keeps ticking: a fixed instant would never let a script
|
||||
// that waits on ctx.Now finish.
|
||||
var offset = request.NowUtcTicks == 0
|
||||
? TimeSpan.Zero
|
||||
: new DateTime(request.NowUtcTicks, DateTimeKind.Utc) - DateTime.UtcNow;
|
||||
Func<DateTime> clock = () => DateTime.UtcNow + offset;
|
||||
var logger = _scriptLogger.ForContext(ForwardingLogSink.KeyProperty, request.Key);
|
||||
var soft = TimeSpan.FromMilliseconds(Math.Max(1, request.SoftTimeoutMs));
|
||||
|
||||
// Cancelled by the host's Cancel frame at the soft deadline, never by our own clock (see the proto).
|
||||
using var cts = new CancellationTokenSource();
|
||||
var grace = request.HardGraceMs > 0 ? TimeSpan.FromMilliseconds(request.HardGraceMs) : _hardGrace;
|
||||
var running = new RunningEval(request.Key, Stopwatch.GetTimestamp(), soft + grace + WatchdogMargin, cts);
|
||||
_running[(long)request.EvalId] = running;
|
||||
var before = GC.GetAllocatedBytesForCurrentThread();
|
||||
var sw = Stopwatch.StartNew();
|
||||
try
|
||||
{
|
||||
object? value = script.Run(inputs, logger, clock, cts.Token, out var isPredicate);
|
||||
if (isPredicate)
|
||||
{
|
||||
result.Predicate = (bool)value!;
|
||||
}
|
||||
else if (VariantCodec.TryEncode(value, out var variant, out var unsupported))
|
||||
{
|
||||
result.Value = variant;
|
||||
}
|
||||
else
|
||||
{
|
||||
result.Error = new ScriptError
|
||||
{
|
||||
Kind = ScriptError.Types.Kind.UnsupportedType,
|
||||
Message = $"script returned a value of unsupported type {unsupported}",
|
||||
};
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
result.Error = new ScriptError { Kind = ScriptError.Types.Kind.Threw, Message = ex.Message };
|
||||
}
|
||||
finally
|
||||
{
|
||||
_running.TryRemove((long)request.EvalId, out _);
|
||||
}
|
||||
result.AllocatedBytes = (ulong)Math.Max(0, GC.GetAllocatedBytesForCurrentThread() - before);
|
||||
result.RunMicros = (uint)Math.Min(uint.MaxValue, sw.Elapsed.TotalMicroseconds);
|
||||
return result;
|
||||
}
|
||||
|
||||
private void WatchdogLoop()
|
||||
{
|
||||
while (true)
|
||||
{
|
||||
Thread.Sleep(50);
|
||||
foreach (var (_, run) in _running)
|
||||
{
|
||||
if (Stopwatch.GetElapsedTime(run.StartedTimestamp) > run.FailFastAfter)
|
||||
{
|
||||
// The host should have killed us already; this names the culprit in the exit/dump.
|
||||
Environment.FailFast($"ScriptWorker: script '{run.Key}' exceeded its hard deadline ({run.FailFastAfter.TotalMilliseconds:F0} ms)");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async Task HeartbeatLoopAsync()
|
||||
{
|
||||
using var timer = new PeriodicTimer(_heartbeatInterval);
|
||||
while (await timer.WaitForNextTickAsync())
|
||||
{
|
||||
Send(new Envelope
|
||||
{
|
||||
Heartbeat = new Heartbeat
|
||||
{
|
||||
Inflight = (uint)_running.Count,
|
||||
Queued = (uint)_queue.Count,
|
||||
HeapBytes = GC.GetTotalMemory(false),
|
||||
Gen2Count = (uint)GC.CollectionCount(2),
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
private async Task LogFlushLoopAsync()
|
||||
{
|
||||
using var timer = new PeriodicTimer(TimeSpan.FromMilliseconds(100));
|
||||
while (await timer.WaitForNextTickAsync())
|
||||
{
|
||||
if (_logSink.Drain() is { } batch) Send(new Envelope { Logs = batch });
|
||||
}
|
||||
}
|
||||
|
||||
private async Task ParentWatchLoopAsync()
|
||||
{
|
||||
using var timer = new PeriodicTimer(TimeSpan.FromMilliseconds(500));
|
||||
while (await timer.WaitForNextTickAsync())
|
||||
{
|
||||
if (!ParentAlive())
|
||||
{
|
||||
await Console.Error.WriteLineAsync("ScriptWorker: parent process is gone; exiting.");
|
||||
Environment.Exit(ScriptWorkerEnvironment.ExitParentGone);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private bool ParentAlive()
|
||||
{
|
||||
try
|
||||
{
|
||||
using var parent = Process.GetProcessById(_parentPid);
|
||||
return !parent.HasExited;
|
||||
}
|
||||
catch (ArgumentException)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
catch (InvalidOperationException)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
private async Task WriteLoopAsync()
|
||||
{
|
||||
var reader = _outbound.Reader;
|
||||
var pendingResults = new EvalResults();
|
||||
try
|
||||
{
|
||||
while (await reader.WaitToReadAsync())
|
||||
{
|
||||
// Opportunistic coalescing (design §4.5): drain whatever is queued right now, merging every completed
|
||||
// evaluation into one EvalResults frame; no timer, so an idle pipe adds no latency.
|
||||
while (reader.TryRead(out var envelope))
|
||||
{
|
||||
if (envelope.BodyCase == Envelope.BodyOneofCase.Results)
|
||||
{
|
||||
pendingResults.Items.AddRange(envelope.Results.Items);
|
||||
continue;
|
||||
}
|
||||
await ScriptWorkerFrameCodec.WriteAsync(_pipe, envelope, _maxFrameBytes);
|
||||
}
|
||||
if (pendingResults.Items.Count > 0)
|
||||
{
|
||||
await ScriptWorkerFrameCodec.WriteAsync(_pipe, Wrap(new Envelope { Results = pendingResults }), _maxFrameBytes);
|
||||
pendingResults = new EvalResults();
|
||||
}
|
||||
await _pipe.FlushAsync();
|
||||
}
|
||||
}
|
||||
catch (IOException)
|
||||
{
|
||||
Environment.Exit(ScriptWorkerEnvironment.ExitPipeClosed);
|
||||
}
|
||||
catch (ObjectDisposedException)
|
||||
{
|
||||
Environment.Exit(ScriptWorkerEnvironment.ExitPipeClosed);
|
||||
}
|
||||
}
|
||||
|
||||
private void SendResult(EvalResult result)
|
||||
{
|
||||
var results = new EvalResults();
|
||||
results.Items.Add(result);
|
||||
Send(new Envelope { Results = results });
|
||||
}
|
||||
|
||||
private void Send(Envelope envelope) => _outbound.Writer.TryWrite(Wrap(envelope));
|
||||
|
||||
private Envelope Wrap(Envelope envelope)
|
||||
{
|
||||
envelope.ProtocolVersion = ScriptWorkerFrameCodec.ProtocolVersion;
|
||||
envelope.Seq = (ulong)Interlocked.Increment(ref _seq);
|
||||
return envelope;
|
||||
}
|
||||
|
||||
private static DiagnosticsReply Diagnostics()
|
||||
{
|
||||
var reply = new DiagnosticsReply
|
||||
{
|
||||
Pid = Environment.ProcessId,
|
||||
WorkingDirectory = Environment.CurrentDirectory,
|
||||
};
|
||||
reply.LoadedAssemblies.AddRange(AppDomain.CurrentDomain.GetAssemblies()
|
||||
.Select(a => a.GetName().Name ?? string.Empty)
|
||||
.Where(n => n.Length > 0));
|
||||
foreach (DictionaryEntry entry in Environment.GetEnvironmentVariables())
|
||||
reply.EnvironmentNames.Add((string)entry.Key);
|
||||
return reply;
|
||||
}
|
||||
|
||||
private static string? ForbiddenAssemblyLoaded()
|
||||
{
|
||||
foreach (var assembly in AppDomain.CurrentDomain.GetAssemblies())
|
||||
{
|
||||
var name = assembly.GetName().Name ?? string.Empty;
|
||||
foreach (var prefix in ScriptWorkerEnvironment.ForbiddenAssemblyPrefixes)
|
||||
if (name.StartsWith(prefix, StringComparison.Ordinal)) return name;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private static void DieForbidden(string assembly)
|
||||
{
|
||||
Console.Error.WriteLine($"ScriptWorker: forbidden assembly '{assembly}' is loaded; exiting.");
|
||||
Environment.Exit(ScriptWorkerEnvironment.ExitForbiddenAssembly);
|
||||
}
|
||||
|
||||
private sealed record RunningEval(string Key, long StartedTimestamp, TimeSpan FailFastAfter, CancellationTokenSource Cancellation);
|
||||
|
||||
/// <summary>One loaded script: its compiled evaluator and its input slot order.</summary>
|
||||
private sealed class LoadedScript : IDisposable
|
||||
{
|
||||
private readonly ScriptEvaluator<VirtualTagContext, object?>? _value;
|
||||
private readonly ScriptEvaluator<AlarmPredicateContext, bool>? _predicate;
|
||||
|
||||
private LoadedScript(string[] inputPaths, ScriptEvaluator<VirtualTagContext, object?>? value,
|
||||
ScriptEvaluator<AlarmPredicateContext, bool>? predicate)
|
||||
{
|
||||
InputPaths = inputPaths;
|
||||
_value = value;
|
||||
_predicate = predicate;
|
||||
}
|
||||
|
||||
public string[] InputPaths { get; }
|
||||
|
||||
public static LoadedScript Compile(ScriptSource source) => source.Kind switch
|
||||
{
|
||||
ContextKind.VirtualTag or ContextKind.Calculation => new LoadedScript(
|
||||
[.. source.InputPaths], ScriptEvaluator<VirtualTagContext, object?>.Compile(source.Source), null),
|
||||
ContextKind.AlarmPredicate => new LoadedScript(
|
||||
[.. source.InputPaths], null, ScriptEvaluator<AlarmPredicateContext, bool>.Compile(source.Source)),
|
||||
_ => throw new ScriptWorkerProtocolException($"unknown script kind {source.Kind}"),
|
||||
};
|
||||
|
||||
public object? Run(IReadOnlyDictionary<string, DataValueSnapshot> inputs, ILogger logger, Func<DateTime> clock,
|
||||
CancellationToken ct, out bool isPredicate)
|
||||
{
|
||||
if (_predicate is not null)
|
||||
{
|
||||
isPredicate = true;
|
||||
return _predicate.RunAsync(new AlarmPredicateContext(inputs, logger, clock), ct).GetAwaiter().GetResult();
|
||||
}
|
||||
isPredicate = false;
|
||||
// Single-tag mode, as in the host: a cross-tag write is dropped.
|
||||
return _value!.RunAsync(new VirtualTagContext(inputs, static (_, _) => { }, logger, clock), ct)
|
||||
.GetAwaiter().GetResult();
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_value?.Dispose();
|
||||
_predicate?.Dispose();
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Buffers <c>ctx.Logger</c> events for the host, bounded, counting what it had to drop.</summary>
|
||||
private sealed class ForwardingLogSink : ILogEventSink
|
||||
{
|
||||
public const string KeyProperty = "ScriptWorkerKey";
|
||||
private readonly ConcurrentQueue<LogLine> _lines = new();
|
||||
private int _count;
|
||||
private int _dropped;
|
||||
|
||||
public int Capacity { get; set; } = 1000;
|
||||
|
||||
public void Emit(LogEvent logEvent)
|
||||
{
|
||||
if (Interlocked.Increment(ref _count) > Capacity)
|
||||
{
|
||||
Interlocked.Decrement(ref _count);
|
||||
Interlocked.Increment(ref _dropped);
|
||||
return;
|
||||
}
|
||||
var key = logEvent.Properties.TryGetValue(KeyProperty, out var k) && k is ScalarValue { Value: string s } ? s : string.Empty;
|
||||
_lines.Enqueue(new LogLine
|
||||
{
|
||||
Key = key,
|
||||
Level = (uint)logEvent.Level,
|
||||
Rendered = Truncate(logEvent.RenderMessage(), 8 * 1024),
|
||||
Exception = Truncate(logEvent.Exception?.ToString() ?? string.Empty, 16 * 1024),
|
||||
TsTicks = logEvent.Timestamp.UtcTicks,
|
||||
});
|
||||
}
|
||||
|
||||
public LogBatch? Drain()
|
||||
{
|
||||
var dropped = Interlocked.Exchange(ref _dropped, 0);
|
||||
if (_lines.IsEmpty && dropped == 0) return null;
|
||||
var batch = new LogBatch { Dropped = (uint)dropped };
|
||||
while (_lines.TryDequeue(out var line))
|
||||
{
|
||||
Interlocked.Decrement(ref _count);
|
||||
batch.Lines.Add(line);
|
||||
}
|
||||
return batch;
|
||||
}
|
||||
|
||||
private static string Truncate(string value, int max) => value.Length <= max ? value : value[..max];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<!-- lmxopcua#780 — the out-of-process, pre-emptible script runner's worker (design §3, §6). A plain console app the
|
||||
host launches per shard over a named pipe. It references only the compile/run core and the frame protocol, so
|
||||
the host's cluster, database, OPC UA and secrets stacks are not in its assembly set; it checks itself against
|
||||
ScriptWorkerEnvironment.ForbiddenAssemblyPrefixes at runtime as well. -->
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<RootNamespace>ZB.MOM.WW.OtOpcUa.ScriptWorker</RootNamespace>
|
||||
<AssemblyName>ZB.MOM.WW.OtOpcUa.ScriptWorker</AssemblyName>
|
||||
<InvariantGlobalization>true</InvariantGlobalization>
|
||||
<!-- Workstation, non-concurrent GC: the heap hard limit (DOTNET_GCHeapHardLimit) is the worker's memory bound. -->
|
||||
<ServerGarbageCollection>false</ServerGarbageCollection>
|
||||
<ConcurrentGarbageCollection>false</ConcurrentGarbageCollection>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\Core\ZB.MOM.WW.OtOpcUa.Core.Scripting\ZB.MOM.WW.OtOpcUa.Core.Scripting.csproj"/>
|
||||
<ProjectReference Include="..\..\Core\ZB.MOM.WW.OtOpcUa.Core.Scripting.Protocol\ZB.MOM.WW.OtOpcUa.Core.Scripting.Protocol.csproj"/>
|
||||
</ItemGroup>
|
||||
|
||||
<!-- Core.Scripting reaches Akka and Grpc.Core.Api transitively through Commons (telemetry meter, log messages),
|
||||
none of which the worker executes. Excluding their assets keeps them out of the worker's deps.json and output
|
||||
folder, so they are not loadable at all, not merely unloaded (design §6 "assembly allow-list by
|
||||
construction"). A code path that did touch them would fail with FileNotFoundException, which the
|
||||
containment tests would catch. -->
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Akka" ExcludeAssets="all" PrivateAssets="all"/>
|
||||
<PackageReference Include="Grpc.Core.Api" ExcludeAssets="all" PrivateAssets="all"/>
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -0,0 +1,43 @@
|
||||
<Project>
|
||||
<!--
|
||||
lmxopcua#780 — places the ScriptWorker next to a host that launches it: the worker's build output is copied into
|
||||
$(OutDir)script-worker/ after Build, and into $(PublishDir)script-worker/ after Publish. Imported by the Host and by
|
||||
the worker test project. The worker is its own app (own deps.json / runtimeconfig.json), so it lives in its own folder
|
||||
rather than beside the host's assemblies.
|
||||
|
||||
Publish note: the copy is the worker's framework-dependent BUILD output, run through the dotnet muxer. That suits the
|
||||
docker-dev image (aspnet:10.0 runtime). A self-contained host publish (the Windows deploys) needs a self-contained
|
||||
worker publish too; that is part of the installer work tracked on lmxopcua#780.
|
||||
-->
|
||||
<PropertyGroup>
|
||||
<_OtScriptWorkerProject>$(MSBuildThisFileDirectory)../ZB.MOM.WW.OtOpcUa.ScriptWorker.csproj</_OtScriptWorkerProject>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="$(_OtScriptWorkerProject)" ReferenceOutputAssembly="false" Private="false" />
|
||||
</ItemGroup>
|
||||
|
||||
<Target Name="_OtResolveScriptWorkerOutput">
|
||||
<MSBuild Projects="$(_OtScriptWorkerProject)" Targets="GetTargetPath" Properties="Configuration=$(Configuration)">
|
||||
<Output TaskParameter="TargetOutputs" ItemName="_OtScriptWorkerTargetPath" />
|
||||
</MSBuild>
|
||||
<PropertyGroup>
|
||||
<_OtScriptWorkerOutDir>@(_OtScriptWorkerTargetPath->'%(RootDir)%(Directory)')</_OtScriptWorkerOutDir>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<_OtScriptWorkerFiles Include="$(_OtScriptWorkerOutDir)**/*" />
|
||||
</ItemGroup>
|
||||
</Target>
|
||||
|
||||
<Target Name="_OtCopyScriptWorkerAfterBuild" AfterTargets="Build" DependsOnTargets="_OtResolveScriptWorkerOutput">
|
||||
<Copy SourceFiles="@(_OtScriptWorkerFiles)"
|
||||
DestinationFiles="@(_OtScriptWorkerFiles->'$(OutDir)script-worker/%(RecursiveDir)%(Filename)%(Extension)')"
|
||||
SkipUnchangedFiles="true" />
|
||||
</Target>
|
||||
|
||||
<Target Name="_OtCopyScriptWorkerAfterPublish" AfterTargets="Publish" DependsOnTargets="_OtResolveScriptWorkerOutput">
|
||||
<Copy SourceFiles="@(_OtScriptWorkerFiles)"
|
||||
DestinationFiles="@(_OtScriptWorkerFiles->'$(PublishDir)script-worker/%(RecursiveDir)%(Filename)%(Extension)')"
|
||||
SkipUnchangedFiles="true" />
|
||||
</Target>
|
||||
</Project>
|
||||
Reference in New Issue
Block a user