fix(scripting): unload compiled-script assemblies via collectible ALC

Core.Scripting-008 resolution: replace the legacy CSharpScript.CreateDelegate
path with hand-rolled CSharpCompilation + Emit + collectible AssemblyLoadContext,
so per-publish compile accretion no longer requires a server restart to reclaim.

Why this was needed:
  Roslyn's CSharpScript path emits dynamically-compiled script assemblies into
  the default AssemblyLoadContext, which is non-collectible. Across config-
  publish generations each Clear() drops dictionary entries but the emitted
  assemblies stay loaded for process lifetime, so memory grows steadily on
  long-running servers with frequent publishes. The accepted-limitation note
  in docs/VirtualTags.md recommended scheduled restarts as the workaround;
  operator feedback was that restarts are difficult, so the underlying
  limitation was the right thing to fix.

Implementation:
  - New ScriptAssemblyLoadContext(name, isCollectible: true) hosts one emitted
    script assembly per evaluator.
  - ScriptEvaluator.Compile synthesises a wrapper class around the user source
    (CompiledScript.Run(globals) — explicit return required per ordinary C#
    semantics, which every existing script already uses), builds a
    CSharpCompilation against the sandbox references, runs the
    ForbiddenTypeAnalyzer over the semantic model unchanged, emits to an
    in-memory PE stream, loads via ScriptAssemblyLoadContext.LoadFromStream,
    and binds a strongly-typed Func<ScriptGlobals<TContext>, TResult> delegate
    via reflection.
  - ScriptEvaluator now implements IDisposable — Dispose calls
    AssemblyLoadContext.Unload(), which makes the emitted assembly eligible
    for GC at the next collection cycle.
  - CompiledScriptCache.Clear() disposes every materialised evaluator before
    dropping its dictionary entry; CompiledScriptCache itself is now
    IDisposable for graceful server shutdown.
  - ScriptSandbox.Build returns a new SandboxConfig (References + Imports)
    instead of a Roslyn ScriptOptions; references now span BCL via the
    TRUSTED_PLATFORM_ASSEMBLIES set filtered to System.* + netstandard +
    Microsoft.Win32.Registry, so forbidden BCL types resolve at compile and
    ForbiddenTypeAnalyzer is the sole security gate (consistent with the
    Core.Scripting-001 / -002 model — references-list-only restriction is
    porous against type forwarding, so the analyzer must be the real gate).

Verification:
  - All 104 Core.Scripting tests pass (was 101 — three new regression tests
    locking the unload contract).
  - All 56 VirtualTags tests pass (unchanged).
  - All 63 ScriptedAlarms tests pass (unchanged).
  - New CompiledScriptCacheTests:
    - Dispose_unloads_compiled_script_assembly_load_context — proves single-
      evaluator ALC unload via WeakReference + bounded GC.Collect() loop.
    - Clear_disposes_every_materialised_evaluator — proves publish-replace
      releases every prior generation's ALC.
    - GetOrCompile_after_Dispose_throws_ObjectDisposedException — locks the
      post-dispose contract.

Docs:
  - docs/VirtualTags.md "Compile cache" section rewritten: the accepted-
    limitation note replaced with the unload contract + the new authoring
    convention (explicit return).
  - docs/ScriptedAlarms.md cross-reference updated to drop the obsolete
    restart guidance.
  - code-reviews/Core.Scripting/findings.md Core.Scripting-008 flipped
    Won't Fix → Resolved with the implementation summary.
  - code-reviews/README.md regenerated.

Pre-existing breakage note: Driver.Galaxy fails the solution-wide build on
master because its ProjectReference to the sibling mxaccessgw repo's
MxGateway.Client targets a path that the sibling repo no longer has after a
recent restructuring. This is unrelated to Core.Scripting-008 and was
verified to exist on master before this branch was cut.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Joseph Doherty
2026-05-23 15:55:04 -04:00
parent 5a9c4591b9
commit 7b6ab2ec6f
8 changed files with 553 additions and 69 deletions

View File

@@ -221,4 +221,107 @@ public sealed class CompiledScriptCacheTests
Should.Throw<Exception>(() => cache.GetOrCompile("""return unknownIdentifier + 1;"""));
cache.Count.ShouldBe(0, "faulted Lazy must still be evicted after compile failure");
}
// --- Core.Scripting-008: collectible AssemblyLoadContext unload ---
[Fact]
public void Dispose_unloads_compiled_script_assembly_load_context()
{
// The whole point of switching the emit path off CSharpScript.CreateDelegate
// and onto a collectible ALC: after Dispose, the runtime can reclaim the
// emitted assembly. We assert this via a WeakReference to the compiled
// assembly itself — if the ALC unloads correctly the reference is dead after
// a forced GC; if the assembly stayed rooted (the pre-fix behaviour) the
// reference survives. The exact reclaim is GC-timing-sensitive, so we loop a
// bounded number of times to absorb GC scheduling noise.
var weak = CompileAndCaptureWeakAssembly();
// Help the runtime — ALC.Unload is *eligible-for-collection*, not synchronous.
for (int i = 0; i < 10 && weak.IsAlive; i++)
{
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
}
weak.IsAlive.ShouldBeFalse(
"the collectible ALC must release the emitted script assembly after Dispose " +
"(Core.Scripting-008). If this fails, either the cache held a reference past " +
"Dispose, or a delegate/closure rooted the assembly in the default ALC.");
}
[System.Runtime.CompilerServices.MethodImpl(
System.Runtime.CompilerServices.MethodImplOptions.NoInlining)]
private static WeakReference CompileAndCaptureWeakAssembly()
{
// Isolation method so the JIT cannot keep the local references rooted past
// its return — without [NoInlining] the GC may decide the locals are still
// live and the WeakReference test becomes flaky.
var evaluator = ScriptEvaluator<FakeScriptContext, int>.Compile("""return 42;""");
var weak = new WeakReference(evaluator.GetType().Assembly is var asm &&
asm is not null ? GetEmittedAssembly(evaluator) : null);
evaluator.Dispose();
return weak;
}
[System.Runtime.CompilerServices.MethodImpl(
System.Runtime.CompilerServices.MethodImplOptions.NoInlining)]
private static object GetEmittedAssembly(ScriptEvaluator<FakeScriptContext, int> evaluator)
{
// The evaluator's delegate Method lives on the synthesized wrapper class in
// the emitted assembly. The delegate field is private; we reach it via a
// reflection probe rather than expose internals — keeps the public surface
// unchanged.
var funcField = typeof(ScriptEvaluator<FakeScriptContext, int>).GetField(
"_func", System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.NonPublic)!;
var del = (Delegate)funcField.GetValue(evaluator)!;
return del.Method.Module.Assembly;
}
[Fact]
public void Clear_disposes_every_materialised_evaluator()
{
// Locks the contract that Clear() is publish-safe: after a config-generation
// publish drops the cache, every prior script's ALC should unload so the
// process memory plateaus rather than growing across publishes.
var weaks = CompileFiveAndCaptureWeakAssemblies();
for (int i = 0; i < 10 && weaks.Any(w => w.IsAlive); i++)
{
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
}
weaks.ShouldAllBe(w => !w.IsAlive,
"after Clear() every compiled-script ALC must be unloadable " +
"(Core.Scripting-008). If this fails the publish-replace pattern leaks " +
"emitted assemblies, which is exactly the v3 concern this rewrite fixes.");
}
[System.Runtime.CompilerServices.MethodImpl(
System.Runtime.CompilerServices.MethodImplOptions.NoInlining)]
private static List<WeakReference> CompileFiveAndCaptureWeakAssemblies()
{
var cache = new CompiledScriptCache<FakeScriptContext, int>();
var weaks = new List<WeakReference>();
for (int i = 0; i < 5; i++)
{
// Distinct source per iteration so each compiles to its own assembly.
var e = cache.GetOrCompile($"""return {i};""");
var funcField = typeof(ScriptEvaluator<FakeScriptContext, int>).GetField(
"_func", System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.NonPublic)!;
var del = (Delegate)funcField.GetValue(e)!;
weaks.Add(new WeakReference(del.Method.Module.Assembly));
}
cache.Count.ShouldBe(5);
cache.Clear();
cache.Count.ShouldBe(0);
return weaks;
}
[Fact]
public void GetOrCompile_after_Dispose_throws_ObjectDisposedException()
{
var cache = new CompiledScriptCache<FakeScriptContext, int>();
cache.GetOrCompile("""return 1;""");
cache.Dispose();
Should.Throw<ObjectDisposedException>(() => cache.GetOrCompile("""return 2;"""));
}
}