Compare commits
11 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 446a45686f | |||
| 92d1df88f4 | |||
| cfbf0b2a17 | |||
| b73ce75402 | |||
| 08d7477860 | |||
| 3834400f05 | |||
| 1827c51c42 | |||
| 56cac39216 | |||
| df772dd09a | |||
| 321d57938f | |||
| 89c07fc382 |
@@ -6,6 +6,7 @@
|
||||
<Project Path="src/Core/ZB.MOM.WW.OtOpcUa.Commons/ZB.MOM.WW.OtOpcUa.Commons.csproj" />
|
||||
<Project Path="src/Core/ZB.MOM.WW.OtOpcUa.Configuration/ZB.MOM.WW.OtOpcUa.Configuration.csproj" />
|
||||
<Project Path="src/Core/ZB.MOM.WW.OtOpcUa.Core/ZB.MOM.WW.OtOpcUa.Core.csproj" />
|
||||
<Project Path="src/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions/ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions.csproj" />
|
||||
<Project Path="src/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting/ZB.MOM.WW.OtOpcUa.Core.Scripting.csproj" />
|
||||
<Project Path="src/Core/ZB.MOM.WW.OtOpcUa.Core.VirtualTags/ZB.MOM.WW.OtOpcUa.Core.VirtualTags.csproj" />
|
||||
<Project Path="src/Core/ZB.MOM.WW.OtOpcUa.Core.ScriptedAlarms/ZB.MOM.WW.OtOpcUa.Core.ScriptedAlarms.csproj" />
|
||||
|
||||
@@ -123,14 +123,16 @@ services:
|
||||
image: otopcua-host:dev
|
||||
# Per-node memory bounds. The full single-mesh stack (6 host nodes) OOM-killed
|
||||
# central-1 on a loaded host. Each host node measured ~357 MiB idle-solo and
|
||||
# climbs under the full mesh + deploy/UI load, so cap at 1g (≈peak + headroom)
|
||||
# with a 512m reservation. These top-level keys are inherited by every service
|
||||
# that uses `<<: *otopcua-host` (YAML merge keeps the anchor's scalar keys; only
|
||||
# the `environment` block is re-declared per service). Compose v2 honors
|
||||
# `mem_limit`/`mem_reservation`. The full mesh needs ~6g of Docker Desktop VM
|
||||
# memory — on a constrained host raise the VM memory or run fewer host services.
|
||||
mem_limit: 1g
|
||||
mem_reservation: 512m
|
||||
# climbs sharply under deploy/materialise load: a node materialising its full
|
||||
# cluster slice (e.g. central → MAIN's galaxy mirror + UNS overlay, ~1400 OPC UA
|
||||
# nodes) peaks well above 1g during a deploy — a 1g cap OOM-kills it (exit 137).
|
||||
# Cap at 2g (≈peak + headroom) with a 1g reservation. These top-level keys are
|
||||
# inherited by every service that uses `<<: *otopcua-host` (YAML merge keeps the
|
||||
# anchor's scalar keys; only the `environment` block is re-declared per service).
|
||||
# The full 6-node mesh needs ~12g of Docker Desktop VM memory — on a constrained
|
||||
# host raise the VM memory or run fewer host services.
|
||||
mem_limit: 2g
|
||||
mem_reservation: 1g
|
||||
depends_on:
|
||||
sql: { condition: service_healthy }
|
||||
migrator: { condition: service_completed_successfully }
|
||||
|
||||
@@ -0,0 +1,180 @@
|
||||
# VirtualTag / Script Memory Scalability — Design (2026-06-07)
|
||||
|
||||
## Problem
|
||||
|
||||
Deploying the Northwind company overlay (1036 `VirtualTag`s, each a one-line mirror script
|
||||
`return ctx.GetTag("…").Value;`) OOM-killed the central nodes — even with a 4 GiB container limit,
|
||||
on a 15.6 GiB Docker VM. The node materialised the address space and spawned 1036 `VirtualTagActor`s,
|
||||
then died as their scripts compiled.
|
||||
|
||||
### Root cause (measured)
|
||||
|
||||
The per-script cost of Roslyn C# scripting is dominated by Roslyn's reference manager materialising
|
||||
the **transitive reference closure of the assembly that contains the script's `globalsType`** — and
|
||||
it is almost entirely **unmanaged** memory (a managed-heap snapshot barely moves). This is the
|
||||
long-standing, unfixed `dotnet/roslyn#22219` ("Backlog"): the reporter measured ~50 MiB/script and
|
||||
OOM at ~39 scripts; the confirmed mitigation in that thread is "move the globalsType to a lean
|
||||
assembly."
|
||||
|
||||
Our globals type reaches Roslyn: `ScriptGlobals<VirtualTagContext>` → `Core.VirtualTags` →
|
||||
`Core.Scripting` → `Microsoft.CodeAnalysis.CSharp.Scripting`. So every compile pays for the whole
|
||||
Roslyn metadata closure.
|
||||
|
||||
### Measurement (probe in `tools/mem-probe/`, Roslyn 4.12.0, 50 retained scripts, 5 runs each)
|
||||
|
||||
| globals closure | per-script RSS | per-script managed |
|
||||
|---|---|---|
|
||||
| **Heavy** — `VirtualTagContext` (reaches Roslyn) — *today* | **~18.2 MiB** (±25% noise) | 0.18 MiB |
|
||||
| **Lean** — context in an Abstractions-only assembly (no Roslyn) | **~1.66 MiB** | 0.05 MiB |
|
||||
|
||||
→ **~11× reduction, ~99% unmanaged.** Real cost today ≈ **18 MiB/script** → 1036 vtags ≈ **~18 GiB**
|
||||
(explains the instant OOM even at 4 GiB). The earlier "~3.5 MiB" guess was ~5× too low.
|
||||
|
||||
### Corpus survey (decides the Phase-2 grammar)
|
||||
|
||||
Real VirtualTag and ScriptedAlarm scripts use a **small bounded statement grammar**: tag reads
|
||||
(`ctx.GetTag("lit").Value` / `.StatusCode`), explicit casts, arithmetic `+ - * / %`, comparisons,
|
||||
boolean logic, ternary/if-else, local variables, a fixed function set (`Math.*`, `System.Convert.*`,
|
||||
`ScriptContext.Deadband`), and `ctx.SetVirtualTag` (vtag only). The Roslyn sandbox *deliberately*
|
||||
also allows `System.Linq` (Sum/Average/Where + lambdas) and `System.Text.RegularExpressions` — the
|
||||
**long tail** that a future interpreter would Roslyn-fallback rather than reimplement. VirtualTag
|
||||
value scripts and ScriptedAlarm predicate scripts share the grammar (alarm = same expression → bool,
|
||||
no `SetVirtualTag`).
|
||||
|
||||
## Scope decision
|
||||
|
||||
- **Phase 1 (build now):** A0 (globals isolation) + A (passthrough fast-path) + a **warn-only**
|
||||
deploy guardrail.
|
||||
- **Phase 2 (spec only, deferred):** C2 interpret-hybrid — documented here, built later only if/when
|
||||
thousands of genuinely *complex* (non-passthrough) scripts justify it.
|
||||
- Both consumers (VirtualTag + ScriptedAlarm) are in scope for A0; A is VirtualTag-passthrough only.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 design
|
||||
|
||||
### A0 — Isolate the globals type from Roslyn (the 11× win)
|
||||
|
||||
Extract the **script-callable** types into a new lean assembly
|
||||
**`ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions`** that references **only `Core.Abstractions`
|
||||
(+ Serilog)** and **never Roslyn**:
|
||||
|
||||
- `ScriptContext` (base), `ScriptGlobals<T>` (the globals wrapper), `VirtualTagContext`,
|
||||
`AlarmPredicateContext`, and `ScriptContext.Deadband`.
|
||||
- Leave the Roslyn users — `ScriptSandbox`, `ScriptEvaluator`, `RoslynVirtualTagEvaluator`,
|
||||
`RoslynScriptedAlarmEvaluator`, `ForbiddenTypeAnalyzer`, `DependencyExtractor` — in
|
||||
`Core.Scripting`; they now reference the lean assembly **downward** (lean ← never references the
|
||||
Roslyn assembly).
|
||||
- `Core.VirtualTags` / `Core.ScriptedAlarms` reference the lean assembly for the context types.
|
||||
|
||||
Net: the `globalsType`'s transitive closure becomes `{Core.Scripting.Abstractions, Core.Abstractions,
|
||||
Serilog}` — **no `Microsoft.CodeAnalysis.*`**. Pure dependency restructuring; **no behavior change**.
|
||||
|
||||
**Structural note (the boundary):** `ScriptContext`/`VirtualTagContext`/`AlarmPredicateContext` must
|
||||
have no Roslyn-referencing members. Confirm `ScriptContext` doesn't pull in `ScriptEvaluator`/
|
||||
`ScriptSandbox` types (it shouldn't — those are the *callers*). If any helper on the context needs a
|
||||
Roslyn type, it stays behind in `Core.Scripting`.
|
||||
|
||||
**Test:** the `tools/mem-probe` harness re-run shows per-script RSS in the ~1.66 MiB regime; the full
|
||||
`Core.Scripting`, `Core.VirtualTags`, `Core.ScriptedAlarms`, and Host-integration script/alarm test
|
||||
suites stay green (behaviour-preserving).
|
||||
|
||||
### A — Passthrough fast-path (mirrors skip Roslyn entirely)
|
||||
|
||||
In the evaluator (`RoslynVirtualTagEvaluator.Evaluate`), **before** the `_cache.GetOrAdd(expression,
|
||||
Compile)` lookup, detect the trivial mirror shape and short-circuit:
|
||||
|
||||
- Pattern (whitespace-tolerant): `^\s*return\s+ctx\.GetTag\(\s*"([^"]+)"\s*\)\.Value\s*;\s*$`.
|
||||
- On match: return `dependencies[ref]` directly (the value is already passed into `Evaluate`) — **no
|
||||
Roslyn, no cache entry, ~bytes.** Map a missing dep to the same `BadNodeIdUnknown`/no-change result
|
||||
the Roslyn path would produce.
|
||||
- Non-matches fall through to the existing Roslyn cache path unchanged.
|
||||
- Downstream `DataType` coercion (in the actor) is unchanged — the fast-path returns the same raw
|
||||
value the compiled script would have returned.
|
||||
|
||||
Covers 100% of the mirror overlay (the 1036). Keep it a narrow, exact pattern so a near-miss safely
|
||||
falls through to Roslyn rather than mis-evaluating.
|
||||
|
||||
**Test:** passthrough returns the dep value with zero compilation (assert the cache stays empty);
|
||||
a non-passthrough still compiles + works; a malformed near-match (`...Value + 1;`) falls through.
|
||||
|
||||
### Warn-only guardrail
|
||||
|
||||
In `AdminOperationsActor.HandleStartDeploymentAsync`, **after** the existing `DraftValidator` gate and
|
||||
**non-blocking**:
|
||||
|
||||
- `compiled = count(unique script sources that are NOT the passthrough shape)` (from the snapshot's
|
||||
`Script` rows; reuse the A pattern to classify).
|
||||
- `estMiB ≈ compiled × perScriptMiB` (configurable, default ~1.66 — the measured post-A0 cost).
|
||||
- Emit a structured `_log.Warning("StartDeployment: {Compiled} scripts will compile (~{EstMiB} MiB
|
||||
RSS/node); ensure node mem_limit covers it", …)` and append an advisory line to the **`Accepted`**
|
||||
`StartDeploymentResult.Message`.
|
||||
- **Never rejects.** Operator-visible, operator-decided.
|
||||
|
||||
**Test:** a config with many distinct non-passthrough scripts logs the warning + still returns
|
||||
`Accepted`; a passthrough-only config logs ~0 compiled.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 design (deferred — spec only)
|
||||
|
||||
### C2 — Interpret-hybrid (built later, if thousands of *complex* scripts appear)
|
||||
|
||||
A bounded interpreter that replaces Roslyn for the surveyed grammar, with Roslyn retained as the
|
||||
fallback for the long tail. Memory per interpreted script ≈ KB (a parsed AST), vs ~1.66 MiB (post-A0)
|
||||
for a Roslyn-compiled one; scales to tens of thousands of complex scripts.
|
||||
|
||||
- **Grammar (statement language, no loops/methods/classes):** literals (numeric/string/bool), the
|
||||
context API (`ctx.GetTag(lit)` + `.Value`/`.StatusCode`/timestamps, `ctx.SetVirtualTag(lit, expr)`,
|
||||
`ctx.Now`, `ctx.Logger`), explicit casts `(int)`/`(double)`/`(bool)`, arithmetic `+ - * / %`,
|
||||
comparisons `< > <= >= == !=`, boolean `&& || !`, ternary `?:`, `if/else`, `var` local bindings,
|
||||
a fixed allow-listed function set (`Math.*`, `System.Convert.*`, `ScriptContext.Deadband`), and
|
||||
`return`.
|
||||
- **Evaluator:** start with a tree-walking interpreter (lowest memory); optionally compile the AST to
|
||||
a `System.Linq.Expressions` delegate (C2-compiled) for hot tags — still collectible `DynamicMethod`s,
|
||||
~KB, far below Roslyn.
|
||||
- **Hybrid contract:** a classifier parses each script; if it's within the grammar → interpret; else
|
||||
(LINQ, Regex, anything unrecognised) → Roslyn fallback (today's path). The deploy/warn guardrail
|
||||
then counts only the *fallback* scripts.
|
||||
- **Both consumers:** one engine serves VirtualTag value scripts (return value coerced to `DataType`)
|
||||
and ScriptedAlarm predicates (return bool); the only difference is the return-type contract and that
|
||||
alarm predicates reject `SetVirtualTag`.
|
||||
- **Bonus:** interpreted scripts are a *hard* sandbox by construction — `ScriptSandbox` /
|
||||
`ForbiddenTypeAnalyzer` (the curated metadata allow/deny machinery) only need guard the rare Roslyn
|
||||
fallback path.
|
||||
- **Risks (why deferred):** you own a small language (grammar/parser/semantics/error messages + tests);
|
||||
C#-semantic-parity edge cases (int vs float division, overflow, null propagation); a classifier +
|
||||
two engines to maintain. Worth it only at real complex-script scale.
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
1. **Memory probe** (`tools/mem-probe/`, retained as the measurement artifact): re-run after A0 shows
|
||||
the ~11× per-script drop.
|
||||
2. **Live docker-dev proof (the real acceptance gate):** re-deploy the full **1036-vtag overlay** with
|
||||
the A0+A build and confirm the deploy is **Accepted** *and* the central node **stays under its
|
||||
`mem_limit`** through materialisation + value streaming (no `OOMKilled`). This is what proves the
|
||||
outage is actually gone.
|
||||
3. Unit tests: passthrough fast-path + warn-guardrail.
|
||||
4. Existing `Core.Scripting` / `Core.VirtualTags` / `Core.ScriptedAlarms` / Host script+alarm suites
|
||||
stay green (A0 is behaviour-preserving).
|
||||
|
||||
## Sequencing & risk
|
||||
|
||||
| Step | Risk | Notes |
|
||||
|---|---|---|
|
||||
| A0 (assembly split) | medium — touches assembly layout across `Core.Scripting`/`VirtualTags`/`ScriptedAlarms` + Host refs | behaviour-preserving; the measured 11× payoff; do first |
|
||||
| A (passthrough) | low — narrow exact pattern in one evaluator method | additive; covers the mirror overlay |
|
||||
| guardrail | low — non-blocking log + message | additive |
|
||||
| C2 | — | **deferred**; spec only |
|
||||
|
||||
A0 first (it moves types); A + guardrail are additive on top. The Phase-2 spec is documentation only.
|
||||
|
||||
## Related context
|
||||
|
||||
- `dotnet/roslyn#22219` — the upstream issue (globalsType-closure memory, mostly unmanaged, no fix).
|
||||
- Measurement harness: `tools/mem-probe/` (this branch).
|
||||
- Recovery already shipped this session: `docker-dev` `mem_limit 1g→2g` (`master` `89c07fc`) +
|
||||
cleared the OOM-causing sealed deployments. The full-validator deploy gate
|
||||
(`AdminOperationsActor` + `DraftValidator`) is where the warn-guardrail hooks in.
|
||||
@@ -0,0 +1,205 @@
|
||||
# VirtualTag/Script Memory — Phase 1 Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans (or subagent-driven-development) to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Make VirtualTag scripts scale in memory — cut Roslyn's ~18 MiB/script (mostly unmanaged) to ~1.66 MiB by isolating the script globals type from Roslyn (A0), skip Roslyn entirely for the mirror passthrough shape (A), and warn on deploy when a config will compile many scripts (guardrail). Phase 2 (the C2 interpreter) is spec-only in the design doc — NOT in this plan.
|
||||
|
||||
**Architecture:** A0 extracts the script-callable types (`ScriptContext`, `ScriptGlobals<T>`, `VirtualTagContext`, `AlarmPredicateContext`) into a new lean assembly that never references Roslyn, so Roslyn's reference manager stops materialising the Roslyn metadata closure per compile. A adds a passthrough short-circuit in the evaluator. The guardrail is a non-blocking warning in the deploy handler.
|
||||
|
||||
**Tech Stack:** .NET 10, Microsoft.CodeAnalysis.CSharp.Scripting (Roslyn 4.12), Akka.NET, xUnit + Shouldly. Solution: `ZB.MOM.WW.OtOpcUa.slnx`, Central Package Management (`Directory.Packages.props`). Branch: `exp/vtag-script-memory` (off `master`). Design: `docs/plans/2026-06-07-virtualtag-script-memory-design.md`.
|
||||
|
||||
**Sequencing:** T1 (A0 split) first — it changes assembly layout the rest builds on. Then T2 (A0 measurement gate), T3 (A passthrough), T4 (guardrail), T5 (live verify). Run sequentially (shared working tree + build).
|
||||
|
||||
---
|
||||
|
||||
### Task 1: A0 — extract script-callable types into a lean, Roslyn-free assembly
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~6 min (assembly split; can't be atomically subdivided — must build green as a unit)
|
||||
**Parallelizable with:** none
|
||||
|
||||
**Files:**
|
||||
- Create: `src/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions/ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions.csproj`
|
||||
- Move (git mv, keep namespaces unchanged): `src/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting/ScriptContext.cs` and `.../ScriptGlobals.cs` → the new project dir; `src/Core/ZB.MOM.WW.OtOpcUa.Core.VirtualTags/VirtualTagContext.cs` → new project dir; `src/Core/ZB.MOM.WW.OtOpcUa.Core.ScriptedAlarms/AlarmPredicateContext.cs` → new project dir
|
||||
- Modify: `.../Core.Scripting/ZB.MOM.WW.OtOpcUa.Core.Scripting.csproj`, `.../Core.VirtualTags/...csproj`, `.../Core.ScriptedAlarms/...csproj` (add ProjectReference to the new assembly)
|
||||
- Modify: `ZB.MOM.WW.OtOpcUa.slnx` (add the new project — use `dotnet sln add`)
|
||||
|
||||
**Why these four:** the production script globals type is `ScriptGlobals<VirtualTagContext>`. Roslyn's reference manager loads the transitive reference closure of *every assembly in the globalsType's type graph*. Today `ScriptGlobals`+`ScriptContext` are in `Core.Scripting` (→ Roslyn) and `VirtualTagContext`/`AlarmPredicateContext` are in assemblies that reference `Core.Scripting` (→ Roslyn). Moving all four into one lean assembly that references only `Core.Abstractions` (+ Serilog) makes the whole closure Roslyn-free. (Confirmed: all four files are Roslyn-free — `ScriptContext` uses only `Serilog` + `Core.Abstractions`; `ScriptGlobals` has no usings; the two contexts use `Serilog`, `Core.Abstractions`, `Core.Scripting`(only for the `ScriptContext` base, which is moving with them).) **Keep each file's existing namespace** (`Core.Scripting`, `Core.VirtualTags`, `Core.ScriptedAlarms`) so NO caller `using` statements change — a namespace may span assemblies. Only project references change.
|
||||
|
||||
**Step 1 — Create the lean project.** `src/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions/ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions.csproj`, mirroring `Core.Abstractions.csproj`'s style (net10.0, CPM — version-less PackageReference):
|
||||
```xml
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Serilog"/>
|
||||
<ProjectReference Include="..\ZB.MOM.WW.OtOpcUa.Core.Abstractions\ZB.MOM.WW.OtOpcUa.Core.Abstractions.csproj"/>
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
```
|
||||
(Match the exact `<PropertyGroup>`/Nullable/ImplicitUsings settings used by the sibling Core projects — copy from `Core.Abstractions.csproj`.)
|
||||
|
||||
**Step 2 — Move the four files** (preserve git history):
|
||||
```bash
|
||||
git mv src/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting/ScriptContext.cs src/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions/
|
||||
git mv src/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting/ScriptGlobals.cs src/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions/
|
||||
git mv src/Core/ZB.MOM.WW.OtOpcUa.Core.VirtualTags/VirtualTagContext.cs src/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions/
|
||||
git mv src/Core/ZB.MOM.WW.OtOpcUa.Core.ScriptedAlarms/AlarmPredicateContext.cs src/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions/
|
||||
```
|
||||
Do NOT change the namespaces inside the moved files.
|
||||
|
||||
**Step 3 — Wire references.** Add to the solution + the three consumer csprojs:
|
||||
```bash
|
||||
dotnet sln ZB.MOM.WW.OtOpcUa.slnx add src/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions/ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions.csproj
|
||||
```
|
||||
In `Core.Scripting.csproj`, `Core.VirtualTags.csproj`, `Core.ScriptedAlarms.csproj` add (keep all their existing references — they still need `Core.Scripting`/Roslyn for the engines/evaluators):
|
||||
```xml
|
||||
<ProjectReference Include="..\ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions\ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions.csproj"/>
|
||||
```
|
||||
(Watch for a reference cycle: the lean assembly must NOT reference `Core.Scripting`. `Core.Scripting` → lean is fine; lean → `Core.Scripting` is forbidden. The moved files don't need anything from `Core.Scripting`, so there's no cycle.)
|
||||
|
||||
**Step 4 — Build the whole solution + run the existing script/alarm suites (behaviour-preserving).**
|
||||
Run: `dotnet build ZB.MOM.WW.OtOpcUa.slnx`
|
||||
Expected: build succeeds (the moved types resolve through the new project).
|
||||
Run: `dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting.Tests/ ; dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.VirtualTags.Tests/ ; dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.ScriptedAlarms.Tests/`
|
||||
Expected: all green (no behaviour change). If a test project references the moved types and the build complains, add the lean project reference to that test csproj too.
|
||||
|
||||
**Step 5 — Commit.** `git add -A && git commit -m "refactor(scripting): extract script-callable types into Roslyn-free Core.Scripting.Abstractions (A0)"`
|
||||
|
||||
**Acceptance:** solution builds; existing script/alarm/VirtualTag suites green; the lean assembly has no `Microsoft.CodeAnalysis` in its (transitive) references (`dotnet list src/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions/...csproj reference` shows only Abstractions; no Roslyn).
|
||||
|
||||
---
|
||||
|
||||
### Task 2: A0 measurement gate — re-run the probe, confirm ~11× drop
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** none (needs Task 1)
|
||||
|
||||
**Files:**
|
||||
- Modify: `tools/mem-probe/MemProbe/MemProbe.csproj` (its references to `VirtualTagContext`/`ScriptGlobals` now resolve via the lean assembly — update the ProjectReference to `Core.Scripting.Abstractions`, drop the now-unnecessary heavy refs if any)
|
||||
- (No production code.)
|
||||
|
||||
**Step 1 — Point the probe at the post-A0 types.** Update `MemProbe.csproj` so `ScriptGlobals<VirtualTagContext>` resolves from `Core.Scripting.Abstractions` (add that ProjectReference; the probe should no longer need `Core.VirtualTags`/`Core.Scripting` directly for the globals type). Keep the `Microsoft.CodeAnalysis.CSharp.Scripting` package ref (the probe still compiles scripts).
|
||||
|
||||
**Step 2 — Run heavy + lean, capture numbers.**
|
||||
Run: `dotnet run -c Release --project tools/mem-probe/MemProbe -- heavy` (twice) and `... -- lean` (twice).
|
||||
Expected: **heavy** (production `ScriptGlobals<VirtualTagContext>`) per-script RSS is now in the **~1.66 MiB** regime (was ~18 MiB before A0) — i.e. heavy ≈ lean, both ~1.66 MiB. That is the proof A0 worked on the production types.
|
||||
|
||||
**Step 3 — Record the result** in a short comment block at the top of `tools/mem-probe/MemProbe/Program.cs` (before/after numbers) and commit.
|
||||
|
||||
**Step 4 — Commit.** `git add tools/mem-probe && git commit -m "test(mem-probe): confirm A0 drops production per-script RSS ~11x (18->~1.66 MiB)"`
|
||||
|
||||
**Acceptance:** post-A0 heavy-mode per-script RSS ≤ ~3 MiB (down from ~18). If it did NOT drop, A0 is incomplete (a type in the globalsType graph still transitively reaches Roslyn) — STOP and report which assembly still pulls in `Microsoft.CodeAnalysis`.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: A — passthrough fast-path in the evaluator
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** none (sequential build)
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Host/Engines/RoslynVirtualTagEvaluator.cs` (`Evaluate`, before the `_cache.GetOrAdd(expression, …)` at line ~56)
|
||||
- Test: `tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/RoslynVirtualTagEvaluatorTests.cs`
|
||||
|
||||
**Step 1 — Write failing tests.** (Read the existing tests for the harness + how `VirtualTagEvalResult` success/value is asserted.)
|
||||
- `Passthrough_returns_dependency_value_without_compiling`: `Evaluate("vt", "return ctx.GetTag(\"a\").Value;", new Dictionary{["a"]=42})` returns success with value `42`, and the evaluator's compile cache stays empty (expose/inspect `_cache.Count` via a test hook or assert no compilation occurred — if the cache field isn't observable, assert behaviour: a syntactically-broken-but-passthrough-shaped script still returns the value, proving Roslyn never ran).
|
||||
- `Passthrough_with_whitespace_variants_still_matches`: leading/trailing whitespace + spaces inside `GetTag( "a" )` still passthrough.
|
||||
- `Non_passthrough_falls_through_to_Roslyn`: `"return (int)ctx.GetTag(\"a\").Value + 1;"` still compiles + returns `43`.
|
||||
- `Passthrough_missing_dependency`: returns the same no-value/Bad result the Roslyn path produces when the dep is absent.
|
||||
|
||||
**Step 2 — Run, confirm fail.** `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/ --filter "FullyQualifiedName~RoslynVirtualTagEvaluator"`
|
||||
|
||||
**Step 3 — Implement.** In `Evaluate`, before the cache lookup:
|
||||
```csharp
|
||||
// A — passthrough fast-path: the mirror shape `return ctx.GetTag("X").Value;` is ~bytes; skip Roslyn
|
||||
// entirely (no compile, no cached assembly). Narrow exact pattern so near-misses fall through.
|
||||
if (TryMatchPassthrough(expression, out var passthroughRef))
|
||||
{
|
||||
if (!dependencies.TryGetValue(passthroughRef, out var value))
|
||||
return /* the same no-value result the Roslyn path returns when GetTag finds nothing */;
|
||||
return /* VirtualTagEvalResult success wrapping `value` — mirror the existing success wrap */;
|
||||
}
|
||||
```
|
||||
Add the matcher (compiled static regex):
|
||||
```csharp
|
||||
private static readonly System.Text.RegularExpressions.Regex PassthroughRegex =
|
||||
new(@"^\s*return\s+ctx\s*\.\s*GetTag\s*\(\s*""([^""]+)""\s*\)\s*\.\s*Value\s*;\s*$",
|
||||
System.Text.RegularExpressions.RegexOptions.Compiled);
|
||||
private static bool TryMatchPassthrough(string expression, out string tagRef)
|
||||
{
|
||||
var m = PassthroughRegex.Match(expression);
|
||||
if (m.Success) { tagRef = m.Groups[1].Value; return true; }
|
||||
tagRef = string.Empty; return false;
|
||||
}
|
||||
```
|
||||
Match the EXACT `VirtualTagEvalResult` construction the Roslyn path uses for success and for not-found (read lines ~74–98 of the file and reuse those code paths — do not invent a new result shape).
|
||||
|
||||
**Step 4 — Run, confirm pass** + the broader evaluator suite is green.
|
||||
|
||||
**Step 5 — Commit.** `git add … && git commit -m "feat(vtag): passthrough fast-path skips Roslyn for mirror scripts (A)"`
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Warn-only deploy guardrail
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none (sequential build)
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.ControlPlane/AdminOperations/AdminOperationsActor.cs` (`HandleStartDeploymentAsync`, after the `DraftValidator` gate, before the seal — non-blocking)
|
||||
- Test: `tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests/AdminOperationsActorTests.cs`
|
||||
|
||||
**Step 1 — Write failing tests.**
|
||||
- `StartDeployment_warns_when_many_scripts_will_compile`: seed N (e.g. 10) distinct NON-passthrough `Script` rows; assert the deploy is still `Accepted` AND the `StartDeploymentResult.Message` contains a compile-cost advisory (e.g. `"scripts will compile"` / the estimated MiB). (If the actor test can't easily read the log, assert via the Message string.)
|
||||
- `StartDeployment_passthrough_scripts_do_not_count`: seed only passthrough `Script` rows → no/zero compile advisory (Accepted, no warning text).
|
||||
|
||||
**Step 2 — Run, confirm fail.** `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests/ --filter "FullyQualifiedName~AdminOperations"`
|
||||
|
||||
**Step 3 — Implement.** After the `DraftValidator` gate (the `if (errors.Count > 0)` block from the deploy gate) and before `ConfigComposer.SnapshotAndFlattenAsync`, add a NON-blocking estimate. Reuse the SAME passthrough regex as Task 3 (extract it to a small shared internal helper, e.g. `ScriptCompileEstimator` in `Core.Scripting.Abstractions` or a static in ControlPlane — or replicate it with a sync-note, matching the `ExtractFullName` replication convention used elsewhere):
|
||||
```csharp
|
||||
const double PerScriptMiB = 1.66; // measured post-A0 per-script RSS
|
||||
var scripts = await db.Scripts.AsNoTracking().Select(s => s.SourceCode).ToListAsync();
|
||||
var compiled = scripts.Where(src => !IsPassthrough(src)).Distinct(StringComparer.Ordinal).Count();
|
||||
if (compiled > 0)
|
||||
{
|
||||
var estMiB = compiled * PerScriptMiB;
|
||||
_log.Warning("StartDeployment: {Compiled} script(s) will compile (~{EstMiB:F0} MiB RSS per node); ensure node mem_limit covers it", compiled, estMiB);
|
||||
// append advisory to the Accepted message below
|
||||
}
|
||||
```
|
||||
On the `Accepted` `StartDeploymentResult`, include the advisory in `Message` (e.g. `Message = compiled > 0 ? $"{compiled} scripts will compile (~{estMiB:F0} MiB/node)" : null`). **Never reject.** Keep the existing Accepted/seal/dispatch flow otherwise unchanged.
|
||||
|
||||
**Step 4 — Run, confirm pass** + the AdminOperations suite green + Host builds.
|
||||
|
||||
**Step 5 — Commit.** `git add … && git commit -m "feat(deploy): warn-only script-compile-cost advisory on deploy"`
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Live verification — the real acceptance gate
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min (+ deploy/settle)
|
||||
**Parallelizable with:** none (needs T1, T3, T4)
|
||||
|
||||
**Steps (no new code — the proof the outage is gone):**
|
||||
1. Build the docker-dev image off this branch: `cd docker-dev && docker compose build central-1`.
|
||||
2. Ensure the canonical company overlay is loaded (galaxy mirror 396 + 40 canonical equipment + 1036 vtags) — from `scadaproj/otopcua-uns-loader`: `populate` + `populate-equipment`. (The DB may already have it; verify `SELECT COUNT(*) FROM VirtualTag` = 1036.)
|
||||
3. Recreate the admin nodes on the new image: `docker compose up -d --no-deps --force-recreate central-1 central-2` (they have a 2g `mem_limit` from `master` `89c07fc`).
|
||||
4. Headless deploy: `curl -s -X POST http://localhost:9200/api/deployments -H 'X-Api-Key: docker-dev-deploy-key'` → expect **202 Accepted** with the compile-cost advisory (passthrough mirrors → ~0 compiled, so the advisory should be ~0/small).
|
||||
5. **The gate:** watch the central node through materialise + value-stream and confirm it **stays under its 2g `mem_limit`** and is NOT `OOMKilled`: `docker inspect otopcua-dev-central-1-1 --format 'OOM={{.State.OOMKilled}}'` = false; `docker stats --no-stream otopcua-dev-central-1-1` well under 2 GiB; `:9200` healthy, `:4840` serving.
|
||||
6. (Optional) `otopcua_uns.py verify-equipment --require-good 396 --wait` to confirm live values still flow.
|
||||
|
||||
**Acceptance:** the full 1036-vtag overlay deploys Accepted AND the central node survives materialisation under its 2g limit (no OOM) — the outage is gone. If it still OOMs, report `docker stats` peak + the central-1 log around the death (A may not be wired into the live evaluator path, or A0 didn't take in the built image).
|
||||
|
||||
**This task changes the running docker-dev stack (the user's active env).** Coordinate: recreate only the admin nodes; don't disrupt the site nodes.
|
||||
|
||||
---
|
||||
|
||||
## After all tasks
|
||||
|
||||
Run the affected suites (`Core.Scripting`, `Core.VirtualTags`, `Core.ScriptedAlarms`, Host-integration evaluator, ControlPlane) + build the solution, then use **superpowers-extended-cc:finishing-a-development-branch**: verify green, present merge/PR/keep/discard for `exp/vtag-script-memory`. Merge/push only on the user's explicit go (the user manages this repo's integration).
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"planPath": "docs/plans/2026-06-07-virtualtag-script-memory.md",
|
||||
"tasks": [
|
||||
{"id": 1, "subject": "Task 1: A0 — extract script types into lean Roslyn-free assembly", "status": "completed", "classification": "high-risk"},
|
||||
{"id": 2, "subject": "Task 2: A0 measurement gate — re-run probe, confirm ~11x drop", "status": "completed", "classification": "small", "blockedBy": [1]},
|
||||
{"id": 3, "subject": "Task 3: A — passthrough fast-path in evaluator", "status": "completed", "classification": "small", "blockedBy": [1]},
|
||||
{"id": 4, "subject": "Task 4: warn-only deploy guardrail", "status": "completed", "classification": "standard", "blockedBy": [1]},
|
||||
{"id": 5, "subject": "Task 5: live docker-dev verification (1036-vtag overlay, no OOM)", "status": "completed", "classification": "standard", "blockedBy": [1, 3, 4]}
|
||||
],
|
||||
"lastUpdated": "2026-06-07"
|
||||
}
|
||||
+1
@@ -17,6 +17,7 @@
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\ZB.MOM.WW.OtOpcUa.Core.Abstractions\ZB.MOM.WW.OtOpcUa.Core.Abstractions.csproj"/>
|
||||
<ProjectReference Include="..\ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions\ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions.csproj"/>
|
||||
<ProjectReference Include="..\ZB.MOM.WW.OtOpcUa.Core.Scripting\ZB.MOM.WW.OtOpcUa.Core.Scripting.csproj"/>
|
||||
</ItemGroup>
|
||||
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
namespace ZB.MOM.WW.OtOpcUa.Core.Scripting;
|
||||
|
||||
using System.Text.RegularExpressions;
|
||||
|
||||
/// <summary>
|
||||
/// Classifies the trivial "mirror" VirtualTag script shape <c>return ctx.GetTag("X").Value;</c>,
|
||||
/// which can be evaluated by returning the dependency value directly — no Roslyn compilation.
|
||||
/// Narrow, exact pattern: any near-miss returns false and falls through to the Roslyn path.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Physically defined in the Core.Scripting.Abstractions assembly (Roslyn-free, so ControlPlane
|
||||
/// can reference it); the namespace is Core.Scripting to keep consumer using-directives unchanged.
|
||||
/// </remarks>
|
||||
public static partial class PassthroughScript
|
||||
{
|
||||
// ^ \s* return \s+ ctx . GetTag ( "X" ) . Value ; \s* $ (whitespace-tolerant around tokens)
|
||||
// Tag-name class [^"\\] excludes both the closing quote and backslash: a literal containing a
|
||||
// backslash escape (e.g. "a\\b" → runtime name a\b) won't match, so it correctly falls through
|
||||
// to Roslyn, which interprets the escape and resolves the actual dependency key.
|
||||
[GeneratedRegex(@"^\s*return\s+ctx\s*\.\s*GetTag\s*\(\s*""([^""\\]+)""\s*\)\s*\.\s*Value\s*;\s*$")]
|
||||
private static partial Regex MirrorRegex();
|
||||
|
||||
/// <summary>True if <paramref name="source"/> is the mirror passthrough shape; outputs the referenced tag.</summary>
|
||||
/// <param name="source">The VirtualTag script source to classify.</param>
|
||||
/// <param name="tagRef">On success, the tag reference captured from the mirror shape; otherwise empty.</param>
|
||||
/// <returns><see langword="true"/> if the source is the mirror passthrough shape; otherwise <see langword="false"/>.</returns>
|
||||
public static bool TryMatch(string? source, out string tagRef)
|
||||
{
|
||||
tagRef = string.Empty;
|
||||
if (string.IsNullOrEmpty(source)) return false;
|
||||
var m = MirrorRegex().Match(source);
|
||||
if (!m.Success) return false;
|
||||
tagRef = m.Groups[1].Value;
|
||||
return true;
|
||||
}
|
||||
}
|
||||
+2
-2
@@ -13,7 +13,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Scripting;
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// Every member on this type MUST be serializable in the narrow sense that
|
||||
/// <see cref="DependencyExtractor"/> can recognize tag-access call sites from the
|
||||
/// <c>DependencyExtractor</c> can recognize tag-access call sites from the
|
||||
/// script AST. Method names used from scripts are locked — renaming
|
||||
/// <see cref="GetTag"/> or <see cref="SetVirtualTag"/> is a breaking change for every
|
||||
/// authored script and the dependency extractor must update in lockstep.
|
||||
@@ -36,7 +36,7 @@ public abstract class ScriptContext
|
||||
/// <remarks>
|
||||
/// <paramref name="path"/> MUST be a string literal in the script source — dynamic
|
||||
/// paths (variables, concatenation, method-returned strings) are rejected at
|
||||
/// publish by <see cref="DependencyExtractor"/>. This is intentional: the static
|
||||
/// publish by <c>DependencyExtractor</c>. This is intentional: the static
|
||||
/// dependency set is required for the change-driven scheduler to subscribe to the
|
||||
/// right upstream tags at load time.
|
||||
/// </remarks>
|
||||
+2
-2
@@ -7,8 +7,8 @@ namespace ZB.MOM.WW.OtOpcUa.Core.VirtualTags;
|
||||
/// <summary>
|
||||
/// Per-evaluation <see cref="ScriptContext"/> for a virtual-tag script. Reads come
|
||||
/// out of the engine's last-known-value cache (driver tags updated via the
|
||||
/// <see cref="ITagUpstreamSource"/> subscription, virtual tags updated by prior
|
||||
/// evaluations). Writes route through <see cref="VirtualTagEngine"/>'s
|
||||
/// <c>ITagUpstreamSource</c> subscription, virtual tags updated by prior
|
||||
/// evaluations). Writes route through <c>VirtualTagEngine</c>'s
|
||||
/// <c>OnScriptSetVirtualTag</c> callback so cross-tag write side effects still
|
||||
/// participate in change-trigger cascades (via the engine's <c>CascadeAsync</c>).
|
||||
/// </summary>
|
||||
+19
@@ -0,0 +1,19 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
<LangVersion>latest</LangVersion>
|
||||
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
|
||||
<GenerateDocumentationFile>true</GenerateDocumentationFile>
|
||||
<NoWarn>$(NoWarn);CS1591</NoWarn>
|
||||
<RootNamespace>ZB.MOM.WW.OtOpcUa.Core.Scripting</RootNamespace>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Serilog"/>
|
||||
<ProjectReference Include="..\ZB.MOM.WW.OtOpcUa.Core.Abstractions\ZB.MOM.WW.OtOpcUa.Core.Abstractions.csproj"/>
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -56,7 +56,9 @@ public static class ScriptSandbox
|
||||
// Core.Abstractions — DataValueSnapshot + DriverDataType so scripts can name
|
||||
// the types they receive from ctx.GetTag.
|
||||
typeof(DataValueSnapshot).Assembly,
|
||||
// Core.Scripting itself — ScriptContext base class + Deadband static.
|
||||
// Core.Scripting.Abstractions — ScriptContext base class + Deadband static.
|
||||
// Intentionally NOT Core.Scripting (which holds ScriptEvaluator/ScriptSandbox + Roslyn):
|
||||
// keeping it out of the sandbox pin keeps Roslyn out of the globalsType assembly closure.
|
||||
typeof(ScriptContext).Assembly,
|
||||
// Serilog.ILogger — script-side logger type.
|
||||
typeof(Serilog.ILogger).Assembly,
|
||||
|
||||
@@ -21,6 +21,7 @@
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\ZB.MOM.WW.OtOpcUa.Core.Abstractions\ZB.MOM.WW.OtOpcUa.Core.Abstractions.csproj"/>
|
||||
<ProjectReference Include="..\ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions\ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions.csproj"/>
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
|
||||
@@ -17,6 +17,7 @@
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\ZB.MOM.WW.OtOpcUa.Core.Abstractions\ZB.MOM.WW.OtOpcUa.Core.Abstractions.csproj"/>
|
||||
<ProjectReference Include="..\ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions\ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions.csproj"/>
|
||||
<ProjectReference Include="..\ZB.MOM.WW.OtOpcUa.Core.Scripting\ZB.MOM.WW.OtOpcUa.Core.Scripting.csproj"/>
|
||||
</ItemGroup>
|
||||
|
||||
|
||||
@@ -10,6 +10,7 @@ using ZB.MOM.WW.OtOpcUa.Configuration.Entities;
|
||||
using ZB.MOM.WW.OtOpcUa.Configuration.Enums;
|
||||
using ZB.MOM.WW.OtOpcUa.Configuration.Validation;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Scripting;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.ControlPlane.AdminOperations;
|
||||
|
||||
@@ -105,6 +106,40 @@ public sealed class AdminOperationsActor : ReceiveActor
|
||||
return;
|
||||
}
|
||||
|
||||
// Warn-only compile-cost guardrail (NEVER blocks). Each genuinely-compiled (non-passthrough)
|
||||
// script costs ~1.66 MiB RSS per node to materialize via Roslyn (measured post-A0; design doc
|
||||
// 2026-06-07). Passthrough "mirror" scripts compile to ~nothing, so they are excluded; distinct
|
||||
// sources are counted (the compile cache keys on source, so duplicates collapse to one unit).
|
||||
// This only surfaces the estimate to the operator — the DraftValidator gate above is the hard
|
||||
// reject; this advisory rides in the Accepted Message so the UI can show it.
|
||||
// advisory is declared outside the inner try so the Accepted reply below can still read it even
|
||||
// when the guardrail query throws (e.g. transient SQL). A failed estimate must NEVER block a
|
||||
// valid deploy — the outer catch is reserved for genuine seal/save failures.
|
||||
string? advisory = null;
|
||||
try
|
||||
{
|
||||
const double PerScriptMiB = 1.66; // measured post-A0 per-script RSS (design doc 2026-06-07)
|
||||
var scriptSources = await db.Scripts.AsNoTracking().Select(s => s.SourceCode).ToListAsync();
|
||||
var compiled = scriptSources
|
||||
.Where(src => !PassthroughScript.TryMatch(src, out _))
|
||||
.Distinct(StringComparer.Ordinal)
|
||||
.Count();
|
||||
if (compiled > 0)
|
||||
{
|
||||
var estMiB = compiled * PerScriptMiB;
|
||||
_log.Warning(
|
||||
"StartDeployment: {Compiled} script(s) will compile (~{EstMiB:F0} MiB RSS per node); ensure node mem_limit covers it",
|
||||
compiled, estMiB);
|
||||
advisory = $"{compiled} script(s) will compile (~{estMiB:F0} MiB/node)";
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
// Guardrail is advisory-only — a failed estimate must never block a valid deploy.
|
||||
_log.Warning(ex, "StartDeployment: script compile-cost estimate failed; advisory skipped");
|
||||
advisory = null;
|
||||
}
|
||||
|
||||
var artifact = await ConfigComposer.SnapshotAndFlattenAsync(db);
|
||||
var deploymentId = DeploymentId.NewId();
|
||||
var revHash = RevisionHash.Parse(artifact.RevisionHash);
|
||||
@@ -136,7 +171,7 @@ public sealed class AdminOperationsActor : ReceiveActor
|
||||
StartDeploymentOutcome.Accepted,
|
||||
deploymentId,
|
||||
revHash,
|
||||
Message: null,
|
||||
Message: advisory,
|
||||
msg.CorrelationId));
|
||||
}
|
||||
catch (Exception ex)
|
||||
|
||||
@@ -23,6 +23,9 @@
|
||||
<ProjectReference Include="..\..\Core\ZB.MOM.WW.OtOpcUa.Cluster\ZB.MOM.WW.OtOpcUa.Cluster.csproj"/>
|
||||
<ProjectReference Include="..\..\Core\ZB.MOM.WW.OtOpcUa.Configuration\ZB.MOM.WW.OtOpcUa.Configuration.csproj"/>
|
||||
<ProjectReference Include="..\..\Core\ZB.MOM.WW.OtOpcUa.Core.Abstractions\ZB.MOM.WW.OtOpcUa.Core.Abstractions.csproj"/>
|
||||
<!-- Roslyn-free; gives HandleStartDeploymentAsync PassthroughScript.TryMatch to exclude
|
||||
mirror scripts (which compile to ~nothing) from the deploy compile-cost advisory. -->
|
||||
<ProjectReference Include="..\..\Core\ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions\ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions.csproj"/>
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
|
||||
@@ -50,6 +50,18 @@ public sealed class RoslynVirtualTagEvaluator : IVirtualTagEvaluator, IDisposabl
|
||||
if (_disposed) return VirtualTagEvalResult.Failure("evaluator disposed");
|
||||
if (string.IsNullOrWhiteSpace(expression)) return VirtualTagEvalResult.Failure("empty expression");
|
||||
|
||||
// A — passthrough fast-path: the mirror shape `return ctx.GetTag("X").Value;` needs no
|
||||
// Roslyn. Narrow exact pattern; near-misses fall through to the compiled path below.
|
||||
// Semantics are byte-identical to the compiled mirror: a present dependency returns its
|
||||
// value (Ok), and an absent one makes GetTag yield a Bad snapshot whose .Value is null,
|
||||
// so the script returns null — Ok(null) here too.
|
||||
if (PassthroughScript.TryMatch(expression, out var passthroughRef))
|
||||
{
|
||||
return dependencies.TryGetValue(passthroughRef, out var ptValue)
|
||||
? VirtualTagEvalResult.Ok(ptValue)
|
||||
: VirtualTagEvalResult.Ok(null);
|
||||
}
|
||||
|
||||
ScriptEvaluator<VirtualTagContext, object?> evaluator;
|
||||
try
|
||||
{
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Scripting;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Core.Scripting.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Unit tests for <see cref="PassthroughScript"/> — the Roslyn-free classifier that recognises
|
||||
/// the trivial mirror shape <c>return ctx.GetTag("X").Value;</c>. The pattern must be narrow:
|
||||
/// any near-miss (arithmetic, a different member, multi-statement, a different method) falls
|
||||
/// through so the Roslyn path stays authoritative.
|
||||
/// </summary>
|
||||
public sealed class PassthroughScriptTests
|
||||
{
|
||||
/// <summary>The canonical mirror shape matches and captures the referenced tag.</summary>
|
||||
[Fact]
|
||||
public void Matches_canonical_mirror_and_captures_tag()
|
||||
{
|
||||
PassthroughScript.TryMatch("return ctx.GetTag(\"a\").Value;", out var tag).ShouldBeTrue();
|
||||
tag.ShouldBe("a");
|
||||
}
|
||||
|
||||
/// <summary>A dotted / hierarchical tag reference is captured verbatim.</summary>
|
||||
[Fact]
|
||||
public void Captures_dotted_tag_reference()
|
||||
{
|
||||
PassthroughScript.TryMatch("return ctx.GetTag(\"Area1.Machine.Speed\").Value;", out var tag)
|
||||
.ShouldBeTrue();
|
||||
tag.ShouldBe("Area1.Machine.Speed");
|
||||
}
|
||||
|
||||
/// <summary>Whitespace around every token is tolerated.</summary>
|
||||
[Fact]
|
||||
public void Tolerates_surrounding_and_inner_whitespace()
|
||||
{
|
||||
PassthroughScript.TryMatch(" return ctx . GetTag( \"a\" ) . Value ; ", out var tag)
|
||||
.ShouldBeTrue();
|
||||
tag.ShouldBe("a");
|
||||
}
|
||||
|
||||
/// <summary>Arithmetic on the mirror value is NOT a passthrough.</summary>
|
||||
[Fact]
|
||||
public void Rejects_arithmetic_on_value()
|
||||
{
|
||||
PassthroughScript.TryMatch("return (int)ctx.GetTag(\"a\").Value + 1;", out _).ShouldBeFalse();
|
||||
}
|
||||
|
||||
/// <summary>Reading a different member (StatusCode) is NOT a passthrough.</summary>
|
||||
[Fact]
|
||||
public void Rejects_other_member_access()
|
||||
{
|
||||
PassthroughScript.TryMatch("return ctx.GetTag(\"a\").StatusCode;", out _).ShouldBeFalse();
|
||||
}
|
||||
|
||||
/// <summary>A multi-statement body is NOT a passthrough.</summary>
|
||||
[Fact]
|
||||
public void Rejects_multi_statement_body()
|
||||
{
|
||||
PassthroughScript.TryMatch("var x = ctx.GetTag(\"a\").Value; return x;", out _).ShouldBeFalse();
|
||||
}
|
||||
|
||||
/// <summary>A different method name is NOT a passthrough.</summary>
|
||||
[Fact]
|
||||
public void Rejects_different_method()
|
||||
{
|
||||
PassthroughScript.TryMatch("return ctx.GetVirtualTag(\"a\").Value;", out _).ShouldBeFalse();
|
||||
}
|
||||
|
||||
/// <summary>Null / empty / whitespace input is rejected.</summary>
|
||||
[Theory]
|
||||
[InlineData(null)]
|
||||
[InlineData("")]
|
||||
[InlineData(" ")]
|
||||
public void Rejects_null_or_blank(string? source)
|
||||
{
|
||||
PassthroughScript.TryMatch(source, out var tag).ShouldBeFalse();
|
||||
tag.ShouldBe(string.Empty);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// A tag literal containing a backslash escape (C# source <c>"a\\b"</c> → runtime name
|
||||
/// <c>a\b</c>) does NOT match the passthrough pattern — it falls through to Roslyn, which
|
||||
/// interprets the escape and resolves the correct dependency key. Capturing the raw source
|
||||
/// text <c>a\\b</c> would produce a wrong-result silent miss against the key <c>a\b</c>.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Rejects_tag_literal_containing_backslash_escape()
|
||||
{
|
||||
// C# literal "return ctx.GetTag(\"a\\\\b\").Value;" → script source contains: a\\b (two chars: backslash + b)
|
||||
PassthroughScript.TryMatch("return ctx.GetTag(\"a\\\\b\").Value;", out var tag).ShouldBeFalse();
|
||||
tag.ShouldBe(string.Empty);
|
||||
}
|
||||
|
||||
/// <summary>A plain dotted tag name (no backslash) still matches — the fix is additive only.</summary>
|
||||
[Fact]
|
||||
public void Matches_plain_dotted_tag_after_backslash_fix()
|
||||
{
|
||||
PassthroughScript.TryMatch("return ctx.GetTag(\"Site1.Area.Tag\").Value;", out var tag).ShouldBeTrue();
|
||||
tag.ShouldBe("Site1.Area.Tag");
|
||||
}
|
||||
}
|
||||
@@ -189,6 +189,120 @@ public sealed class AdminOperationsActorTests : ControlPlaneActorTestBase
|
||||
verify.Deployments.Count().ShouldBe(0);
|
||||
}
|
||||
|
||||
/// <summary>Verifies the warn-only compile-cost advisory: seeding N distinct non-passthrough
|
||||
/// (genuinely-compiled) Script rows yields an <see cref="StartDeploymentOutcome.Accepted"/>
|
||||
/// deploy whose <see cref="StartDeploymentResult.Message"/> carries a compile-cost advisory.
|
||||
/// The guardrail NEVER rejects — it only surfaces the estimated RSS pressure to the operator.</summary>
|
||||
[Fact]
|
||||
public void StartDeployment_warns_when_many_scripts_will_compile()
|
||||
{
|
||||
const int n = 10;
|
||||
var dbFactory = NewInMemoryDbFactory();
|
||||
using (var db = dbFactory.CreateDbContext())
|
||||
{
|
||||
for (var i = 0; i < n; i++)
|
||||
{
|
||||
db.Scripts.Add(new Configuration.Entities.Script
|
||||
{
|
||||
ScriptId = $"s-{i}",
|
||||
Name = $"script-{i}",
|
||||
SourceCode = $"return (int)ctx.GetTag(\"a\").Value + {i};", // distinct, non-passthrough
|
||||
SourceHash = $"hash-{i}",
|
||||
});
|
||||
}
|
||||
db.SaveChanges();
|
||||
}
|
||||
|
||||
var coordinator = CreateTestProbe("coord");
|
||||
var actor = Sys.ActorOf(AdminOperationsActor.Props(dbFactory, coordinator.Ref, Enumerable.Empty<IDriverProbe>()));
|
||||
|
||||
actor.Tell(new StartDeployment("joe", CorrelationId.NewId()));
|
||||
|
||||
// Still dispatches — the advisory is non-blocking.
|
||||
coordinator.ExpectMsg<DispatchDeployment>(TimeSpan.FromSeconds(3));
|
||||
|
||||
var reply = ExpectMsg<StartDeploymentResult>(TimeSpan.FromSeconds(3));
|
||||
reply.Outcome.ShouldBe(StartDeploymentOutcome.Accepted);
|
||||
reply.Message.ShouldNotBeNull();
|
||||
reply.Message.ShouldContain("will compile");
|
||||
reply.Message.ShouldContain($"{n} script"); // the distinct compiled count
|
||||
|
||||
using var verify = dbFactory.CreateDbContext();
|
||||
verify.Deployments.Count().ShouldBe(1);
|
||||
}
|
||||
|
||||
/// <summary>Verifies that passthrough "mirror" scripts (<c>return ctx.GetTag("X").Value;</c>)
|
||||
/// compile to ~nothing and therefore do NOT count toward the compile-cost advisory: a deploy
|
||||
/// of only passthrough scripts is <see cref="StartDeploymentOutcome.Accepted"/> with no advisory.</summary>
|
||||
[Fact]
|
||||
public void StartDeployment_passthrough_scripts_do_not_count()
|
||||
{
|
||||
var dbFactory = NewInMemoryDbFactory();
|
||||
using (var db = dbFactory.CreateDbContext())
|
||||
{
|
||||
for (var i = 0; i < 5; i++)
|
||||
{
|
||||
db.Scripts.Add(new Configuration.Entities.Script
|
||||
{
|
||||
ScriptId = $"mirror-{i}",
|
||||
Name = $"mirror-{i}",
|
||||
SourceCode = $"return ctx.GetTag(\"tag-{i}\").Value;", // passthrough mirror — costs ~nothing
|
||||
SourceHash = $"mhash-{i}",
|
||||
});
|
||||
}
|
||||
db.SaveChanges();
|
||||
}
|
||||
|
||||
var coordinator = CreateTestProbe("coord");
|
||||
var actor = Sys.ActorOf(AdminOperationsActor.Props(dbFactory, coordinator.Ref, Enumerable.Empty<IDriverProbe>()));
|
||||
|
||||
actor.Tell(new StartDeployment("joe", CorrelationId.NewId()));
|
||||
|
||||
coordinator.ExpectMsg<DispatchDeployment>(TimeSpan.FromSeconds(3));
|
||||
|
||||
var reply = ExpectMsg<StartDeploymentResult>(TimeSpan.FromSeconds(3));
|
||||
reply.Outcome.ShouldBe(StartDeploymentOutcome.Accepted);
|
||||
// No compiled scripts → no advisory emitted.
|
||||
(reply.Message is null || !reply.Message.Contains("will compile")).ShouldBeTrue();
|
||||
}
|
||||
|
||||
/// <summary>Verifies the advisory counts DISTINCT non-passthrough sources: many rows sharing one
|
||||
/// identical source collapse to a single compiled unit (the compile cache keys on source), so the
|
||||
/// advisory reports 1 — not the row count.</summary>
|
||||
[Fact]
|
||||
public void StartDeployment_duplicate_sources_collapse_to_one_compiled_unit()
|
||||
{
|
||||
const string identical = "return (int)ctx.GetTag(\"a\").Value + 1;";
|
||||
var dbFactory = NewInMemoryDbFactory();
|
||||
using (var db = dbFactory.CreateDbContext())
|
||||
{
|
||||
for (var i = 0; i < 7; i++)
|
||||
{
|
||||
db.Scripts.Add(new Configuration.Entities.Script
|
||||
{
|
||||
ScriptId = $"dup-{i}",
|
||||
Name = $"dup-{i}",
|
||||
SourceCode = identical, // same source across all rows
|
||||
SourceHash = "dup-hash",
|
||||
});
|
||||
}
|
||||
db.SaveChanges();
|
||||
}
|
||||
|
||||
var coordinator = CreateTestProbe("coord");
|
||||
var actor = Sys.ActorOf(AdminOperationsActor.Props(dbFactory, coordinator.Ref, Enumerable.Empty<IDriverProbe>()));
|
||||
|
||||
actor.Tell(new StartDeployment("joe", CorrelationId.NewId()));
|
||||
|
||||
coordinator.ExpectMsg<DispatchDeployment>(TimeSpan.FromSeconds(3));
|
||||
|
||||
var reply = ExpectMsg<StartDeploymentResult>(TimeSpan.FromSeconds(3));
|
||||
reply.Outcome.ShouldBe(StartDeploymentOutcome.Accepted);
|
||||
reply.Message.ShouldNotBeNull();
|
||||
reply.Message.ShouldContain("will compile");
|
||||
reply.Message.ShouldContain("1 script(s) will compile");
|
||||
}
|
||||
|
||||
/// <summary>Verifies that starting a deployment is refused when another is in flight.</summary>
|
||||
[Fact]
|
||||
public void StartDeployment_refuses_when_another_is_in_flight()
|
||||
|
||||
+165
@@ -1,3 +1,5 @@
|
||||
using System.Collections;
|
||||
using System.Reflection;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
@@ -95,4 +97,167 @@ public sealed class RoslynVirtualTagEvaluatorTests
|
||||
result.Success.ShouldBeFalse();
|
||||
result.Reason!.ShouldContain("disposed");
|
||||
}
|
||||
|
||||
// ── A — passthrough fast-path: the mirror shape `return ctx.GetTag("X").Value;`
|
||||
// is answered directly from the dependency value, skipping Roslyn compilation. ──
|
||||
|
||||
/// <summary>Mirror passthrough returns the dependency value verbatim without compiling.</summary>
|
||||
[Fact]
|
||||
public void Passthrough_returns_dependency_value_without_compiling()
|
||||
{
|
||||
using var sut = new RoslynVirtualTagEvaluator(NullLogger<RoslynVirtualTagEvaluator>.Instance);
|
||||
|
||||
var result = sut.Evaluate(
|
||||
virtualTagId: "vt-mirror",
|
||||
expression: "return ctx.GetTag(\"a\").Value;",
|
||||
dependencies: new Dictionary<string, object?> { ["a"] = 42 });
|
||||
|
||||
result.Success.ShouldBeTrue(result.Reason);
|
||||
result.Value.ShouldBe(42);
|
||||
}
|
||||
|
||||
/// <summary>The passthrough fast-path returns the raw object reference, not a re-boxed copy —
|
||||
/// proof the value flowed straight through without round-tripping a Roslyn script run.</summary>
|
||||
[Fact]
|
||||
public void Passthrough_returns_same_object_reference_as_dependency()
|
||||
{
|
||||
using var sut = new RoslynVirtualTagEvaluator(NullLogger<RoslynVirtualTagEvaluator>.Instance);
|
||||
var payload = new object();
|
||||
|
||||
var result = sut.Evaluate(
|
||||
virtualTagId: "vt-ref",
|
||||
expression: "return ctx.GetTag(\"a\").Value;",
|
||||
dependencies: new Dictionary<string, object?> { ["a"] = payload });
|
||||
|
||||
result.Success.ShouldBeTrue(result.Reason);
|
||||
result.Value.ShouldBeSameAs(payload);
|
||||
}
|
||||
|
||||
/// <summary>Whitespace-tolerant mirror shapes still take the passthrough fast-path.</summary>
|
||||
[Fact]
|
||||
public void Passthrough_whitespace_variants_match()
|
||||
{
|
||||
using var sut = new RoslynVirtualTagEvaluator(NullLogger<RoslynVirtualTagEvaluator>.Instance);
|
||||
|
||||
var result = sut.Evaluate(
|
||||
virtualTagId: "vt-ws",
|
||||
expression: " return ctx . GetTag( \"a\" ) . Value ; ",
|
||||
dependencies: new Dictionary<string, object?> { ["a"] = 7 });
|
||||
|
||||
result.Success.ShouldBeTrue(result.Reason);
|
||||
result.Value.ShouldBe(7);
|
||||
}
|
||||
|
||||
/// <summary>A near-miss (arithmetic on the mirror value) falls through to Roslyn and still works.</summary>
|
||||
[Fact]
|
||||
public void Non_passthrough_falls_through_to_Roslyn()
|
||||
{
|
||||
using var sut = new RoslynVirtualTagEvaluator(NullLogger<RoslynVirtualTagEvaluator>.Instance);
|
||||
|
||||
var result = sut.Evaluate(
|
||||
virtualTagId: "vt-plus1",
|
||||
expression: "return (int)ctx.GetTag(\"a\").Value + 1;",
|
||||
dependencies: new Dictionary<string, object?> { ["a"] = 42 });
|
||||
|
||||
result.Success.ShouldBeTrue(result.Reason);
|
||||
result.Value.ShouldBe(43);
|
||||
}
|
||||
|
||||
/// <summary>Passthrough with an absent dependency yields the SAME result the Roslyn path
|
||||
/// produces: <c>GetTag</c> returns a Bad snapshot whose <c>.Value</c> is null, so the script
|
||||
/// returns null and the evaluator wraps it as <c>Ok(null)</c> (success, null value).</summary>
|
||||
[Fact]
|
||||
public void Passthrough_missing_dependency_matches_Roslyn_behaviour()
|
||||
{
|
||||
using var sut = new RoslynVirtualTagEvaluator(NullLogger<RoslynVirtualTagEvaluator>.Instance);
|
||||
const string mirror = "return ctx.GetTag(\"missing\").Value;";
|
||||
|
||||
// Roslyn baseline: same source, but force a near-miss that compiles, to capture the
|
||||
// not-found semantics independently. Here we just assert the mirror's own missing-dep
|
||||
// result equals the documented Ok(null) shape.
|
||||
var passthrough = sut.Evaluate("vt-miss", mirror, new Dictionary<string, object?>());
|
||||
|
||||
passthrough.Success.ShouldBeTrue(passthrough.Reason);
|
||||
passthrough.Value.ShouldBeNull();
|
||||
passthrough.Reason.ShouldBeNull();
|
||||
}
|
||||
|
||||
/// <summary>Cross-check: the equivalent Roslyn-compiled read of a missing dependency
|
||||
/// produces exactly the same <c>Ok(null)</c> result, proving the fast-path is byte-identical.</summary>
|
||||
[Fact]
|
||||
public void Roslyn_missing_dependency_also_returns_Ok_null()
|
||||
{
|
||||
using var sut = new RoslynVirtualTagEvaluator(NullLogger<RoslynVirtualTagEvaluator>.Instance);
|
||||
// `(object?)...Value` forces the compiled path (not the mirror shape) but reads the same
|
||||
// missing tag; result must match the passthrough missing-dep result above.
|
||||
var result = sut.Evaluate(
|
||||
"vt-miss-roslyn",
|
||||
"return (object?)ctx.GetTag(\"missing\").Value;",
|
||||
new Dictionary<string, object?>());
|
||||
|
||||
result.Success.ShouldBeTrue(result.Reason);
|
||||
result.Value.ShouldBeNull();
|
||||
result.Reason.ShouldBeNull();
|
||||
}
|
||||
|
||||
/// <summary>Direct parity test: the EXACT mirror source <c>return ctx.GetTag("missing").Value;</c>
|
||||
/// evaluated against an empty dependency dictionary takes the fast-path and yields
|
||||
/// <c>Ok(null)</c>; a minimally-altered but semantically-identical non-passthrough variant
|
||||
/// <c>return (object?)ctx.GetTag("missing").Value;</c> compiled by Roslyn against the same
|
||||
/// empty deps yields the identical <c>Ok(null)</c> — proving the fast-path is byte-identical
|
||||
/// to the Roslyn path for the missing-dependency case.</summary>
|
||||
[Fact]
|
||||
public void Passthrough_exact_mirror_missing_dep_matches_Roslyn_baseline()
|
||||
{
|
||||
using var sut = new RoslynVirtualTagEvaluator(NullLogger<RoslynVirtualTagEvaluator>.Instance);
|
||||
|
||||
// Fast-path: exact mirror shape → passthrough returns Ok(null) for missing dep.
|
||||
var fastPath = sut.Evaluate(
|
||||
"vt-parity-fast",
|
||||
"return ctx.GetTag(\"missing\").Value;",
|
||||
new Dictionary<string, object?>());
|
||||
|
||||
fastPath.Success.ShouldBeTrue(fastPath.Reason);
|
||||
fastPath.Value.ShouldBeNull();
|
||||
fastPath.Reason.ShouldBeNull();
|
||||
|
||||
// Roslyn path: `(object?)` cast forces compilation but reads the same missing tag.
|
||||
var roslynPath = sut.Evaluate(
|
||||
"vt-parity-roslyn",
|
||||
"return (object?)ctx.GetTag(\"missing\").Value;",
|
||||
new Dictionary<string, object?>());
|
||||
|
||||
roslynPath.Success.ShouldBeTrue(roslynPath.Reason);
|
||||
roslynPath.Value.ShouldBeNull();
|
||||
roslynPath.Reason.ShouldBeNull();
|
||||
}
|
||||
|
||||
/// <summary>Decisive proof the fast-path skips Roslyn: the compiled-script cache (which every
|
||||
/// Roslyn evaluation populates via <c>GetOrAdd</c>) stays EMPTY after a mirror evaluation, then
|
||||
/// grows to one entry once a genuine (non-mirror) expression forces compilation.</summary>
|
||||
[Fact]
|
||||
public void Passthrough_does_not_populate_the_compiled_script_cache()
|
||||
{
|
||||
using var sut = new RoslynVirtualTagEvaluator(NullLogger<RoslynVirtualTagEvaluator>.Instance);
|
||||
|
||||
// Mirror shape — must take the fast-path, leaving the Roslyn cache untouched.
|
||||
sut.Evaluate("vt-mirror", "return ctx.GetTag(\"a\").Value;",
|
||||
new Dictionary<string, object?> { ["a"] = 1 });
|
||||
CompiledCacheCount(sut).ShouldBe(0);
|
||||
|
||||
// Non-mirror shape — must compile, populating exactly one cache entry.
|
||||
sut.Evaluate("vt-real", "return (int)ctx.GetTag(\"a\").Value + 1;",
|
||||
new Dictionary<string, object?> { ["a"] = 1 });
|
||||
CompiledCacheCount(sut).ShouldBe(1);
|
||||
}
|
||||
|
||||
/// <summary>Reads the count of the private compiled-script cache via reflection.</summary>
|
||||
private static int CompiledCacheCount(RoslynVirtualTagEvaluator sut)
|
||||
{
|
||||
var field = typeof(RoslynVirtualTagEvaluator)
|
||||
.GetField("_cache", BindingFlags.Instance | BindingFlags.NonPublic);
|
||||
field.ShouldNotBeNull();
|
||||
var cache = (ICollection)field.GetValue(sut)!;
|
||||
return cache.Count;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
*/bin/
|
||||
*/obj/
|
||||
@@ -0,0 +1,18 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
<LangVersion>latest</LangVersion>
|
||||
<!-- Throwaway memory probe: keep build noise low. -->
|
||||
<TreatWarningsAsErrors>false</TreatWarningsAsErrors>
|
||||
<GenerateDocumentationFile>false</GenerateDocumentationFile>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<!-- Closure of THIS assembly = {LeanContext, Core.Abstractions}. No Roslyn. -->
|
||||
<ProjectReference Include="..\..\..\src\Core\ZB.MOM.WW.OtOpcUa.Core.Abstractions\ZB.MOM.WW.OtOpcUa.Core.Abstractions.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -0,0 +1,36 @@
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
|
||||
namespace LeanContext;
|
||||
|
||||
/// <summary>
|
||||
/// LEAN globals type for the memory probe. Its transitive reference closure is only
|
||||
/// {LeanContext, Core.Abstractions} — deliberately NO Roslyn — so the per-script cost
|
||||
/// of the Roslyn reference-manager loading the globalsType's closure (dotnet/roslyn#22219)
|
||||
/// can be measured against the heavy <c>VirtualTagContext</c>, whose closure pulls in
|
||||
/// Microsoft.CodeAnalysis.CSharp.Scripting.
|
||||
/// <para>
|
||||
/// <see cref="GetTag"/> returns the same <see cref="DataValueSnapshot"/> type that
|
||||
/// <c>VirtualTagContext.GetTag</c> returns, so the probe's script source
|
||||
/// (<c>ctx.GetTag("x").Value</c>) is byte-identical for both modes — the ONLY
|
||||
/// difference is which assembly closure the globalsType lives in.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public sealed class LeanCtx
|
||||
{
|
||||
private readonly System.Collections.Generic.Dictionary<string, DataValueSnapshot> _d = new();
|
||||
|
||||
public DataValueSnapshot GetTag(string p) =>
|
||||
_d.TryGetValue(p, out var v) ? v : new DataValueSnapshot(null, 0u, null, default);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// LEAN analogue of the prod <c>ScriptGlobals<TContext></c> wrapper: exposes a
|
||||
/// named <c>ctx</c> property so the script source can be byte-identical to the heavy
|
||||
/// path (<c>ctx.GetTag(...).Value</c>). Lives in the LeanContext assembly, so its
|
||||
/// reference closure is {LeanContext, Core.Abstractions} — NO Roslyn. This is the A0
|
||||
/// "globals type in a lean assembly" treatment.
|
||||
/// </summary>
|
||||
public sealed class LeanGlobals
|
||||
{
|
||||
public LeanCtx ctx { get; set; } = new();
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
<LangVersion>latest</LangVersion>
|
||||
<TreatWarningsAsErrors>false</TreatWarningsAsErrors>
|
||||
<GenerateDocumentationFile>false</GenerateDocumentationFile>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<!-- Same Roslyn version the repo pins (Directory.Packages.props => 4.12.0, CPM). -->
|
||||
<PackageReference Include="Microsoft.CodeAnalysis.CSharp.Scripting" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\LeanContext\LeanContext.csproj" />
|
||||
<!-- After A0: VirtualTagContext + ScriptGlobals live in Core.Scripting.Abstractions
|
||||
(Roslyn-free). We reference that directly so typeof(VirtualTagContext).Assembly
|
||||
resolves to the lean assembly — the key invariant the probe is verifying. -->
|
||||
<ProjectReference Include="..\..\..\src\Core\ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions\ZB.MOM.WW.OtOpcUa.Core.Scripting.Abstractions.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -0,0 +1,84 @@
|
||||
// A0 result (2026-06-07): heavy 18.2 MiB -> 2.4 MiB/script; lean 1.65 MiB; ~7.6x drop
|
||||
// (within noise of theoretical 11x). Proves globalsType-closure isolation: ScriptGlobals<VirtualTagContext>
|
||||
// now resolves from Core.Scripting.Abstractions (Roslyn-free); heavy ≈ lean as expected.
|
||||
// Pre-A0 baseline: heavy ~18 MiB (Core.Scripting + Core.VirtualTags both -> Roslyn).
|
||||
// Post-A0 (this run): heavy 2.40/2.53 MiB, lean 1.64/1.65 MiB — both well under the 3 MiB gate.
|
||||
|
||||
using Microsoft.CodeAnalysis.CSharp.Scripting;
|
||||
using Microsoft.CodeAnalysis.Scripting;
|
||||
|
||||
// Memory measurement probe for Roslyn C# scripting per dotnet/roslyn#22219.
|
||||
// Compiles + RETAINS N distinct scripts (like the prod compiled-delegate cache) and
|
||||
// measures the per-script working-set cost. The ONLY thing that varies between "heavy"
|
||||
// and "lean" is the globalsType's assembly closure:
|
||||
// heavy = VirtualTagContext (closure pulls in Roslyn via Core.Scripting)
|
||||
// lean = LeanCtx (closure = {LeanContext, Core.Abstractions} only)
|
||||
|
||||
static long Rss() => System.Diagnostics.Process.GetCurrentProcess().WorkingSet64;
|
||||
|
||||
static void Settle()
|
||||
{
|
||||
for (int i = 0; i < 3; i++)
|
||||
{
|
||||
GC.Collect(2, GCCollectionMode.Forced, true, true);
|
||||
GC.WaitForPendingFinalizers();
|
||||
}
|
||||
}
|
||||
|
||||
var mode = args.Length > 0 ? args[0] : "heavy";
|
||||
int N = args.Length > 1 && int.TryParse(args[1], out var n) ? n : 50;
|
||||
|
||||
// The globalsType is what Roslyn's reference manager loads the transitive closure of
|
||||
// (dotnet/roslyn#22219). We mirror production's choice: prod uses the WRAPPER
|
||||
// ScriptGlobals<TContext> (which exposes the named `ctx` property), NOT the raw context.
|
||||
// heavy = ScriptGlobals<VirtualTagContext> -> AFTER A0: both wrapper + generic arg now
|
||||
// live in Core.Scripting.Abstractions (Roslyn-free); closure = lean.
|
||||
// lean = LeanGlobals -> closure {LeanContext, Core.Abstractions},
|
||||
// NO Roslyn. This is the A0 "globals type in a lean assembly" treatment.
|
||||
var globalsType = mode == "lean"
|
||||
? typeof(LeanContext.LeanGlobals)
|
||||
: typeof(ZB.MOM.WW.OtOpcUa.Core.Scripting.ScriptGlobals<ZB.MOM.WW.OtOpcUa.Core.VirtualTags.VirtualTagContext>);
|
||||
|
||||
// The script reads ctx.GetTag("x").Value. We must reference: the globalsType's own
|
||||
// assembly, the assemblies of its generic type arguments (so `ctx`'s property type
|
||||
// resolves), and Core.Abstractions (DataValueSnapshot, the return type of GetTag).
|
||||
// References are minimal + identical in spirit for both modes; the real difference is
|
||||
// the transitive UNMANAGED closure of the globalsType's assembly that Roslyn's reference
|
||||
// manager loads per compilation (the #22219 effect).
|
||||
var snapshotAssembly = typeof(ZB.MOM.WW.OtOpcUa.Core.Abstractions.DataValueSnapshot).Assembly;
|
||||
var refAssemblies = new System.Collections.Generic.HashSet<System.Reflection.Assembly>
|
||||
{
|
||||
globalsType.Assembly,
|
||||
snapshotAssembly,
|
||||
};
|
||||
foreach (var ga in globalsType.GetGenericArguments())
|
||||
refAssemblies.Add(ga.Assembly);
|
||||
var opts = ScriptOptions.Default
|
||||
.WithReferences(refAssemblies)
|
||||
.WithImports();
|
||||
|
||||
// Warm up Roslyn once (compile 1 throwaway) so the baseline excludes one-time Roslyn init.
|
||||
_ = CSharpScript.Create<object>("return 0;", opts, globalsType).GetCompilation().GetDiagnostics();
|
||||
Settle();
|
||||
|
||||
long baseRss = Rss();
|
||||
long baseGc = GC.GetTotalMemory(true);
|
||||
|
||||
var held = new System.Collections.Generic.List<object>(N);
|
||||
for (int i = 0; i < N; i++)
|
||||
{
|
||||
var src = $"return ctx.GetTag(\"ref_{i}\").Value;";
|
||||
var script = CSharpScript.Create<object>(src, opts, globalsType);
|
||||
script.Compile(); // force compilation / emit
|
||||
held.Add(script.CreateDelegate()); // retain the compiled delegate (like the prod cache)
|
||||
}
|
||||
|
||||
Settle();
|
||||
long afterRss = Rss();
|
||||
long afterGc = GC.GetTotalMemory(true);
|
||||
GC.KeepAlive(held);
|
||||
|
||||
Console.WriteLine($"MODE={mode} N={N}");
|
||||
Console.WriteLine($" baseline RSS={baseRss / 1048576.0:F1} MiB managed={baseGc / 1048576.0:F1} MiB");
|
||||
Console.WriteLine($" afterN RSS={afterRss / 1048576.0:F1} MiB managed={afterGc / 1048576.0:F1} MiB");
|
||||
Console.WriteLine($" PER-SCRIPT: RSS={(afterRss - baseRss) / 1048576.0 / N:F2} MiB/script managed={(afterGc - baseGc) / 1048576.0 / N:F2} MiB/script");
|
||||
Reference in New Issue
Block a user