ScriptWorker shadow isolation timing test includes cold compile and fails the 250ms bound on baseline #798

Open
opened 2026-11-10 12:50:19 -05:00 by dohertj2 · 0 comments
Owner

Observed in the full #782 solution run and independently reproduced with the unchanged test on baseline 83ed880e (SDK 1.5.378.182) and candidate SDK 2.0.1. This does not establish a new OPC UA regression.

ShadowModeTests.A_runaway_that_runs_away_only_in_the_worker_is_killed_and_quarantined_there_and_never_reaches_a_published_value fails its slowest <250ms assertion. Original baseline focused run measured 880.1332ms; candidate full solution measured 595.9162ms, and its focused run also fails. Functional return-value assertions and worker quarantine checks precede the timing failure. Source is unchanged between versions.

The stopwatch includes the initial in-process evaluations of newly supplied script text as well as shadow scheduling. The test can therefore fail from cold compilation/JIT or shared host scheduling even when the published path does not wait for shadow execution. Runtime was .NET10.0.14 / SDK10.0.403 on macOS arm64; the full solution used ci.runsettings and AKKA_TEST_TIMEFACTOR=3.

Repair the causal nonblocking proof without merely raising the threshold: separate a warmed in-process control from shadow submission, or use a deterministic synchronization barrier proving published completion while the shadow remains blocked/quarantined. Retain output correctness, worker kill/quarantine and sibling victim semantics. Keep any representative timing benchmark as an explicit performance gate with recorded hardware/load.

No threshold, skip, filter or expected result was changed during #782. The original failing TRXs are retained: baseline artifacts/opcua2/83ed880e/script-shadow-probe1; candidate artifacts/opcua2/da50123a/solution-tests1 and artifacts/opcua2/7b353d7b/core-parity-and-shadow1/shadow. #789 remains the separate deployed Windows script-worker performance follow-up.

Observed in the full #782 solution run and independently reproduced with the unchanged test on baseline 83ed880e (SDK 1.5.378.182) and candidate SDK 2.0.1. This does not establish a new OPC UA regression. ShadowModeTests.A_runaway_that_runs_away_only_in_the_worker_is_killed_and_quarantined_there_and_never_reaches_a_published_value fails its slowest <250ms assertion. Original baseline focused run measured 880.1332ms; candidate full solution measured 595.9162ms, and its focused run also fails. Functional return-value assertions and worker quarantine checks precede the timing failure. Source is unchanged between versions. The stopwatch includes the initial in-process evaluations of newly supplied script text as well as shadow scheduling. The test can therefore fail from cold compilation/JIT or shared host scheduling even when the published path does not wait for shadow execution. Runtime was .NET10.0.14 / SDK10.0.403 on macOS arm64; the full solution used ci.runsettings and AKKA_TEST_TIMEFACTOR=3. Repair the causal nonblocking proof without merely raising the threshold: separate a warmed in-process control from shadow submission, or use a deterministic synchronization barrier proving published completion while the shadow remains blocked/quarantined. Retain output correctness, worker kill/quarantine and sibling victim semantics. Keep any representative timing benchmark as an explicit performance gate with recorded hardware/load. No threshold, skip, filter or expected result was changed during #782. The original failing TRXs are retained: baseline artifacts/opcua2/83ed880e/script-shadow-probe1; candidate artifacts/opcua2/da50123a/solution-tests1 and artifacts/opcua2/7b353d7b/core-parity-and-shadow1/shadow. #789 remains the separate deployed Windows script-worker performance follow-up.
dohertj2 added the priority:p2 label 2026-11-10 12:50:19 -05:00
dohertj2 added the type:test label 2026-11-10 13:29:59 -05:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dohertj2/lmxopcua#798