dohertj2/lmxopcua
1
0
Fork 0
Code Issues 5 Pull Requests Actions Packages Projects Releases Wiki Activity

test(script-worker): prove Shadow non-blocking causally, not by stopwatch (#798)

Browse Source 
ShadowModeTests' runaway case asserted "slowest < 250 ms" around evaluations
whose timing included cold Roslyn compile/JIT of new script text, so it failed
on a healthy published path (540-880 ms measured, both SDK baselines).

The test now puts a test-owned barrier runner (GatedShadow) between the real
ShadowScriptRunner and the real worker pool. The runaway's first shadow run is
held before it reaches the pool, then (after it runs away in the worker, the
pool answers T and the worker is KilledDeadline with the runaway as culprit)
its outcome is held before comparison. Published evaluations made while a
barrier is closed must return the in-process values; a Mirror that waited for
its shadow hangs into a 30 s liveness failure (mutation-checked). Output
correctness, pool kill/quarantine, published-side never quarantined, the single
runaway mismatch (sibling is a victim, not compared) and key_busy drops are kept.

The wall-clock number moves to an opt-in Category=Perf case
(Shadow_published_path_latency_while_the_shadow_runs_away, OTOPCUA_SCRIPT_PERF=1)
on warmed evaluators, reporting hardware and load average.

Sibling: ContainmentTests' hard-deadline case timed the runaway's first
(cold-compile) run; it now compiles the runaway's source first with an input
that does not spin.
This commit is contained in:
Joseph Doherty
2026-11-10 21:15:12 -05:00
parent b36e688f0a
commit 304d1f1e6c
3 changed files with 214 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/Server/ZB.MOM.WW.OtOpcUa.ScriptWorker.Tests/ContainmentTests.cs
+7 -3
View File
@@ -61,10 +61,14 @@ public sealed class ContainmentTests
await using var runner = Scripts.Runner(Scripts.Options(hardGrace: grace, quarantineAfterKills: 2));
(await runner.RunAsync(Scripts.Vt("sibling", Healthy), Ct)).Value.ShouldBe(7);
// Compile the runaway's source first (it only spins when told to), so the stopwatch below times the soft answer
// and the kill, not the worker's first compile of new script text (lmxopcua#798's flaw, in its runner form).
const string runaway = "if ((int)ctx.GetTag(\"spin\").Value == 1) { while (true) { } } return 0;";
(await runner.RunAsync(Scripts.Vt("runaway", runaway, softT, Scripts.Input("spin", 0)), Ct)).Value.ShouldBe(0);
var worker = runner.Shards[0].Current.ShouldNotBeNull();
var sw = Stopwatch.StartNew();
var outcome = await runner.RunAsync(Scripts.Vt("runaway", Runaway, softT), Ct);
var outcome = await runner.RunAsync(Scripts.Vt("runaway", runaway, softT, Scripts.Input("spin", 1)), Ct);
var softElapsed = sw.Elapsed;
outcome.Status.ShouldBe(ScriptRunStatus.Failed);
outcome.Reason!.ShouldStartWith("script timed out after ");
@@ -90,12 +94,12 @@ public sealed class ContainmentTests
// A second kill quarantines (QuarantineAfterKills = 2); the third trigger is refused without running.
var second = runner.Shards[0].Current!;
(await runner.RunAsync(Scripts.Vt("runaway", Runaway, softT), Ct)).Status.ShouldBe(ScriptRunStatus.Failed);
(await runner.RunAsync(Scripts.Vt("runaway", runaway, softT, Scripts.Input("spin", 1)), Ct)).Status.ShouldBe(ScriptRunStatus.Failed);
(await second.Exited.WaitAsync(TimeSpan.FromSeconds(10), Ct)).Reason.ShouldBe(WorkerExitReason.KilledDeadline);
// The strike is recorded by the shard's exit handler, which runs asynchronously to the Exited completion.
await Scripts.Eventually(() => runner.QuarantinedKeys(ScriptContextKind.VirtualTag).Count == 1,
TimeSpan.FromSeconds(5), "the second kill strike");
var refused = await runner.RunAsync(Scripts.Vt("runaway", Runaway, softT), Ct);
var refused = await runner.RunAsync(Scripts.Vt("runaway", runaway, softT, Scripts.Input("spin", 1)), Ct);
refused.Reason!.ShouldStartWith(ScriptRunGuard.QuarantinedReasonPrefix);
refused.Reason.ShouldContain("hard deadline");
runner.QuarantinedKeys(ScriptContextKind.VirtualTag).ShouldBe(["runaway"]);