fix(vtags): real script timeout + ALC-safe compile cache (arch-review 02/U2,U3)

U2 (Critical): RoslynVirtualTagEvaluator ran scripts with a CancellationToken
that a synchronous Roslyn script can never observe, so one CPU-bound/infinite-
loop virtual-tag script hung the owning VirtualTagActor forever (no timeout, no
recovery). Route evaluation through the existing TimedScriptEvaluator (Task.Run
+ WaitAsync) so the wall-clock budget is real; a runaway script now returns Bad
within the timeout instead of wedging the actor.

U3 (High): the live path cached evaluators in a raw ConcurrentDictionary never
cleared on republish, leaking a collectible AssemblyLoadContext per edited
script forever. Swap in the purpose-built CompiledScriptCache (Lazy single-
compile, dispose-on-Clear) and add an apply-boundary clear: a new narrow
IScriptCacheOwner seam the VirtualTagHostActor calls on each ApplyVirtualTags
generation (mirrors ScriptedAlarmEngine's per-generation clear).

Tests: infinite-loop-fails-within-timeout (bounded by WaitAsync so a regression
fails instead of hanging), happy-path-after-wrapping, ClearCompiledScripts-
empties-cache, and a host-actor wiring guard asserting each apply generation
calls ClearCompiledScripts (theme #1 production-caller assertion). Host.Integration
VT suite 18/18, Runtime VT suite 12/12.
This commit is contained in:
Joseph Doherty
2026-07-08 16:21:47 -04:00
parent 9cad9ed0fc
commit 7fd44f0fa7
5 changed files with 149 additions and 17 deletions
@@ -251,7 +251,7 @@ public sealed class RoslynVirtualTagEvaluatorTests
}
/// <summary>Decisive proof the fast-path skips Roslyn: the compiled-script cache (which every
/// Roslyn evaluation populates via <c>GetOrAdd</c>) stays EMPTY after a mirror evaluation, then
/// Roslyn evaluation populates via <c>GetOrCompile</c>) stays EMPTY after a mirror evaluation, then
/// grows to one entry once a genuine (non-mirror) expression forces compilation.</summary>
[Fact]
public void Passthrough_does_not_populate_the_compiled_script_cache()
@@ -299,13 +299,74 @@ public sealed class RoslynVirtualTagEvaluatorTests
entry.VirtualTagId.ShouldBe("vt-log");
}
/// <summary>Reads the count of the private compiled-script cache via reflection.</summary>
// ── U2 — production timeout must be REAL: a CPU-bound/infinite-loop script must fail within the
// wall-clock budget, never hang the owning actor. The pre-fix CTS-only path could not interrupt
// a synchronous Roslyn script, so this class of script wedged the VirtualTagActor forever. ──
/// <summary>An infinite-loop script returns Failure ("timed out") within the timeout budget instead
/// of hanging. The whole call is bounded by <c>WaitAsync</c> so a regression FAILS the test (with a
/// TimeoutException) rather than wedging the suite.</summary>
[Fact]
public async Task Evaluate_infinite_loop_script_fails_within_timeout_and_does_not_hang()
{
using var sut = new RoslynVirtualTagEvaluator(
NullLogger<RoslynVirtualTagEvaluator>.Instance, NoOpScriptRoot(), TimeSpan.FromMilliseconds(200));
var evalTask = Task.Run(
() => sut.Evaluate("vt-loop", "while(true){}return 0;", new Dictionary<string, object?>()),
TestContext.Current.CancellationToken);
var result = await evalTask.WaitAsync(TimeSpan.FromSeconds(5), TestContext.Current.CancellationToken);
result.Success.ShouldBeFalse();
result.Reason.ShouldNotBeNull();
result.Reason.ShouldContain("timed out");
}
/// <summary>Regression: routing through <see cref="TimedScriptEvaluator{TContext, TResult}"/> did not
/// break the happy path — a fast compiled script still returns its value well within the budget.</summary>
[Fact]
public void Evaluate_fast_script_still_returns_value_after_timeout_wrapping()
{
using var sut = new RoslynVirtualTagEvaluator(
NullLogger<RoslynVirtualTagEvaluator>.Instance, NoOpScriptRoot(), TimeSpan.FromMilliseconds(200));
var result = sut.Evaluate("vt-fast", "return (int)ctx.GetTag(\"a\").Value + 1;",
new Dictionary<string, object?> { ["a"] = 41 });
result.Success.ShouldBeTrue(result.Reason);
result.Value.ShouldBe(42);
}
// ── U3 — the apply-boundary clear (IScriptCacheOwner) drops compiled entries so their collectible
// AssemblyLoadContexts are reclaimed. (The ALC-unload-on-Clear proof lives in CompiledScriptCacheTests;
// here we assert the adapter's ClearCompiledScripts actually empties its cache.) ──
/// <summary>Verifies <see cref="RoslynVirtualTagEvaluator.ClearCompiledScripts"/> drops every cached
/// compile — the hook the host actor calls per deploy generation to release stale ALCs.</summary>
[Fact]
public void ClearCompiledScripts_empties_the_compile_cache()
{
using var sut = new RoslynVirtualTagEvaluator(NullLogger<RoslynVirtualTagEvaluator>.Instance, NoOpScriptRoot());
sut.Evaluate("vt-a", "return (int)ctx.GetTag(\"a\").Value + 1;", new Dictionary<string, object?> { ["a"] = 1 });
CompiledCacheCount(sut).ShouldBe(1);
sut.ClearCompiledScripts();
CompiledCacheCount(sut).ShouldBe(0);
}
/// <summary>Reads the count of the private compiled-script cache via reflection. The cache is a
/// <c>CompiledScriptCache&lt;,&gt;</c> which exposes a <c>Count</c> property (not <c>ICollection</c>).</summary>
private static int CompiledCacheCount(RoslynVirtualTagEvaluator sut)
{
var field = typeof(RoslynVirtualTagEvaluator)
.GetField("_cache", BindingFlags.Instance | BindingFlags.NonPublic);
field.ShouldNotBeNull();
var cache = (ICollection)field.GetValue(sut)!;
return cache.Count;
var cache = field.GetValue(sut)!;
var countProp = cache.GetType().GetProperty("Count");
countProp.ShouldNotBeNull();
return (int)countProp.GetValue(cache)!;
}
}
@@ -345,6 +345,26 @@ public sealed class VirtualTagHostActorTests : RuntimeActorTestBase
mux.LastSender.ShouldNotBe(firstChild);
}
/// <summary>Wiring guard (arch-review 02/U3, theme #1): every ApplyVirtualTags calls
/// <see cref="IScriptCacheOwner.ClearCompiledScripts"/> on the evaluator at the apply boundary.
/// This is the load-bearing wire — the production evaluator roots one collectible ALC per compiled
/// script, and without this per-generation clear they accrete across deploys. A pure unit test of
/// the evaluator can't catch a missing call here; this asserts the host actor actually makes it.</summary>
[Fact]
public void ApplyVirtualTags_clears_the_evaluator_compile_cache_each_generation()
{
var publish = CreateTestProbe();
var evaluator = new CacheOwningStubEvaluator();
var host = Sys.ActorOf(VirtualTagHostActor.Props(publish.Ref, mux: null, evaluator));
host.Tell(new VirtualTagHostActor.ApplyVirtualTags(new[] { Plan("vt-1", "eq-1", "speed-rpm") }));
AwaitAssert(() => evaluator.ClearCount.ShouldBe(1));
// A second apply generation clears again — proves it's per-generation, not one-shot.
host.Tell(new VirtualTagHostActor.ApplyVirtualTags(new[] { Plan("vt-1", "eq-1", "speed-rpm") }));
AwaitAssert(() => evaluator.ClearCount.ShouldBe(2));
}
/// <summary>Capturing <see cref="IHistoryWriter"/>: records every Record call so H5c tests can
/// assert the host historizes (or does not) and with what path + snapshot.</summary>
private sealed class CapturingHistoryWriter : IHistoryWriter
@@ -371,4 +391,22 @@ public sealed class VirtualTagHostActorTests : RuntimeActorTestBase
public VirtualTagEvalResult Evaluate(string id, string expr, IReadOnlyDictionary<string, object?> deps)
=> VirtualTagEvalResult.NoChange;
}
/// <summary>Spy evaluator that also owns a script cache: records how many times the host actor
/// calls <see cref="IScriptCacheOwner.ClearCompiledScripts"/> so the apply-boundary wiring can be
/// asserted. Counter is interlocked/volatile — written on the actor thread, read on the test thread.</summary>
private sealed class CacheOwningStubEvaluator : IVirtualTagEvaluator, IScriptCacheOwner
{
private int _clearCount;
/// <summary>Number of ClearCompiledScripts calls observed.</summary>
public int ClearCount => Volatile.Read(ref _clearCount);
/// <inheritdoc/>
public VirtualTagEvalResult Evaluate(string id, string expr, IReadOnlyDictionary<string, object?> deps)
=> VirtualTagEvalResult.NoChange;
/// <inheritdoc/>
public void ClearCompiledScripts() => Interlocked.Increment(ref _clearCount);
}
}