Second P1 slice of docs/plans/2026-11-09-out-of-process-script-runner-design.md; still opt-in behind
Scripting:Runner=OutOfProcess (InProcess unchanged).
- Compiled-PE cache (design §7, §14 #8): Scripting:Worker:PeCacheDir holds <dir>/pe/<id>.pe entries authenticated
with HMAC-SHA256 under a per-node random key (<dir>/pe-cache.key, created by the host, 0600/0700 on Unix), handed to
the worker in Hello. Entry id = SHA-256(compile fingerprint + kind + source); the fingerprint covers the Roslyn
version, the Core.Scripting build (analyzer, wrapper, stack guard), the sandbox reference set and the runtime. Edited,
truncated, other-key or renamed entries are rejected, deleted and recompiled; the worker never loads unverified bytes.
Pruned by recency at startup (30 d / 50 000). ScriptEvaluator splits into CompileToImage + FromImage. Measured on
40 scripts, restart to every first result: macOS 820 ms no cache / 840 ms empty / 85 ms warm;
windev (Windows) 9.5 s / 9.4 s / 1.0 s.
- Quarantine race: WorkerConnection runs exit attribution synchronously BEFORE completing the evaluations/loads it held
(and before Exited), and the shard passes it straight to the runner — a kill strike is in place before the killed
key's slot is released.
- Supervision: a dead worker restarts by itself after its backoff; Unavailable outcomes carry ScriptRunOutcome.Recovery
(-> VirtualTagEvalResult.RetryWhen) and the VT actor / Calculation driver re-evaluate once against current inputs when
it completes. Isolation worker stops after Scripting:Worker:IsolationIdleTimeout (10 min). Keys compiling when a
worker dies are suspects too (isolation finds a poison compile and quarantines it). Startup orphan sweep by launch
marker: kills only a process whose marker's host is gone and whose pid + start time + image match.
- Crash-loop circuit (Scripting:Worker:CircuitOpenAfter, 2 min): runner Unavailable, one Error, evaluations answered
at once while shards keep retrying at the cap.
- Observability (§10): otopcua.script.runner.state, worker.up/inflight/queue_depth/heap_bytes (shard),
ipc.roundtrip + eval.duration (kind), compile{cache}, isolation.suspects, exits reason=idle. `script-runner` health
check on the ready tier, at worst Degraded (§14 #12: degraded but still ready).
Left on #780: self-contained worker publish, installer (account, pipe ACL, firewall), docker-dev live drill,
scripted-alarm seam, the §4.5 per-seam batch submit, the §8 perf harness.
Tests: SupervisionTests, PeCacheStoreTests/PeCacheWorkerTests (real workers), ScriptRunnerHealthCheckTests, VT + Calc
recovery seam tests; 14 mutations each turned their test red.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Item 2: a tag's optional `write` block picks the statement the driver builds from its own
catalog-validated identifiers (no authored SQL; values always bound):
- Update (default, unchanged back-compat UPDATE ... WHERE key; 0 rows = BadNoData)
- Insert (key+value[+stamp]; append for a WideRow topByTimestamp tag)
- Upsert: SQL Server MERGE WITH (HOLDLOCK), PostgreSQL INSERT ... ON CONFLICT, MySQL
ON DUPLICATE KEY UPDATE (1-or-2 rows), Oracle MERGE from DUAL
- Procedure: named stored procedure resolved against the routine catalog by the catalog gate
(INFORMATION_SCHEMA.ROUTINES / ALL_PROCEDURES), value/key/timestamp params by name (MySQL positional)
- stampTimestamp sets the tag's timestamp column to the write time (UTC)
Odbc serves Update/Insert only (shared SqlWriteSupport table). Constraint failures map to
BadEntryExists (unique) / BadOutOfRange (other integrity, data exception); everything else,
permission denied included, stays BadCommunicationError. WriteIdempotent stays false.
The parser refuses malformed blocks (strict keys/enums/param charset); DraftValidator emits
blocking SqlWriteBlockInvalid / SqlWriteModeUnsupported from the driver's own parse; the AdminUI
editor surfaces the same message.
Items 1 + 4 (docs/drivers/Sql.md): ODBC stays the ANSI SQL:2008 profile (native providers for
SQL Server/MySQL, DB2 unsupported) with an ops note on driver manager + vendor driver on every
Odbc and AdminUI host; read-only drivers require a SELECT-only login, writable drivers a login
limited to their write targets.
Tests: SqlWriteModeTests (parse, every dialect's SQL, gate, status map, SQLite end-to-end),
DraftValidatorSqlWriteBlockTests, AdminUI round-trip; live SqlWriteModesLiveTests on SQL Server,
PostgreSQL, MySQL, Oracle (seeds gain write_targets + set_target) and an Odbc write leg.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Catalogue of what breaks moving from 1.5.378.106 to 2.0.1: restore
break (MEL 10.0.12), 42 -> 103 compile errors in the first two waves,
root-cause groups with API mappings, PeerAddress replacement for the
reflection-based remote endpoint resolver, effort and risks.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The docs said script authoring was Admin-gated (VirtualTags.md "Known
resource limits") and that a Designer "cannot publish" (security.md roles
table). The code admits Designer to both: Scripts, ScriptEdit, EquipmentPage
(virtual tags, scripted alarms), /api/script-analysis/* and Deployments all
carry AdminUiPolicies.ConfigEditor = RequireRole("Administrator",
"Designer"). That is a recorded decision, not drift: AdminUI-001 (98b27fc1)
loosened the analysis backend from FleetAdmin to Administrator,Designer to
match the page gate, and R2-05 converged the same pages on ConfigEditor
"with the same semantics". So the docs follow the code:
- VirtualTags.md: the primary control is ConfigEditor (Administrator or
Designer), which also gates deploy.
- security.md roles table: Designer = the whole ConfigEditor surface incl.
scripts and deploy; Administrator = Designer + the FleetAdmin-only surface
(role grants, API keys, certificate actions, manual failover) — the old
"publish, cluster/node CRUD" were ConfigEditor all along.
- 2026-06-09 Monaco plan: its FleetAdmin gate is marked superseded.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Reproduced first, out of process (ScriptRecursionContainmentTests re-launches
the xunit v3 test assembly to run one recursive script through
ScriptEvaluator + ScriptRunGuard, so the runner never executes one): before
this change every shape — local function, mutual local functions,
self-referencing Func<>, anonymous self-application, recursion through LINQ,
an expression-bodied void lambda, an anonymous method — ended the child with
exit 134 (SIGABRT, "Stack overflow."). A StackOverflowException cannot be
caught, so in production one script takes the whole node down.
Fix: ScriptEvaluator.Compile now rewrites the validated wrapper
(ScriptStackGuardInjector, after ForbiddenTypeAnalyzer) so the script body and
every lambda, anonymous method and local function starts with
ScriptStackGuard.Check(), which throws ScriptStackExhaustedException when
RuntimeHelpers.TryEnsureSufficientExecutionStack reports the stack nearly
spent. The script then fails like any throwing script (Bad, script log) and
the process lives.
Why a runtime check, not an analyzer rule: recursion is not decidable from
syntax (anonymous self-application never names itself), and bounded recursion
must keep working. Expression-bodied lambdas/local functions become blocks
(`return e;` or `e;`, chosen from the bound symbol, so the same overload
binds); expression-tree and async lambdas are left alone. Limits (documented
on ScriptStackGuard and in docs/VirtualTags.md): depth depends on the thread's
stack size; a recursion that never re-enters script code is not seen; a script
may catch the exception, which only unwinds it.
Tests: ScriptRecursionContainmentTests (7 shapes, child process each; all 7
fail with exit 134 when the rewrite is disabled), ScriptStackGuardInjectionTests
(bounded recursion and every rewritten lambda/local-function shape keep their
value; the expression-tree case fails if those lambdas are rewritten).
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
VirtualTagActor called the blocking IVirtualTagEvaluator.Evaluate, and
RoslynVirtualTagEvaluator blocked on the run guard with
.GetAwaiter().GetResult(), so every script run parked a dispatcher thread for
up to its 2 s budget (a first-use Roslyn compile did too). This is the #750 P0
refactor for virtual tags.
- IVirtualTagEvaluator.EvaluateAsync (default: a completed task over the
snapshot overload, so the null evaluator and fakes are unchanged).
- RoslynVirtualTagEvaluator implements it without blocking: passthrough and
compile failures complete synchronously, a first-use compile goes to the
pool, the run is awaited. The blocking overload now delegates to it.
- VirtualTagActor: a completed task is applied in the same turn, as before;
an incomplete one switches to an "evaluating" behaviour that stashes every
dependency/reassert message until a private completion message arrives,
then applies the result and unstashes. Each trigger is still evaluated
against its own inputs, results are applied/published in trigger order and
the dedup/Bad-transition state sees the same sequence. The dependency map is
not touched while a run reads it.
Calculation and scripted-alarm serialisation is NOT relaxed: neither blocks a
thread, and running independent scripts in parallel is not semantics-neutral
(pool consumption bound, cross-tag publish order, timer/pump exclusion for
Calc; state-machine/persist boundary, transition timestamps and the
store-failure path for alarms). Recorded in docs/VirtualTags.md and the #750
design's P0 entry.
Tests: VirtualTagActorAsyncEvaluationTests — on a one-thread dispatcher a slow
script no longer starves a sibling actor and the slow actor answers Identify
(fails when the actor calls the blocking Evaluate); triggers during a run are
evaluated afterwards in order against their own inputs; a reassert during a
run is applied after it; a faulted evaluation takes the evaluator-threw path
(the ordering/reassert tests fail with the stash removed).
RoslynVirtualTagEvaluatorAsyncTests — a slow script and a cold compile return
an incomplete task (fail without the override); cheap outcomes and timeouts
match the blocking overload.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Design only. Documents why today's in-process ScriptRunGuard can abandon but
not stop a runaway script (file:line evidence), and proposes a sharded
worker pool with an isolation worker under a ScriptRunnerActor, a
named-pipe + protobuf frame protocol carrying full input snapshots,
soft/hard deadlines with process kill, kill attribution and quarantine,
worker sandboxing (scrubbed env, pinned assemblies, Job Object, heap
limit), an HMAC'd content-addressed PE cache, performance budgets,
failure-quality semantics, a flag-gated InProcess/Shadow/OutOfProcess
rollout, a test plan, open decisions and an assumptions register.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
R06:M-5 made Sparkplug *Array leaves unbindable because the ingestor refused
them. The ingestor now decodes the packed bytes_value of all 13 array types
(new SparkplugArrayCodec: LE fixed widths, int64-ms DateTime, uint32-count +
MSB-first BooleanArray, NUL-terminated UTF-8 StringArray; golden vectors are
the spec's examples, reproduced byte-for-byte by SparkplugNet 1.3.10) and
publishes a typed 1-D array through the scalar coercion, all-or-nothing.
Shape uses the fleet's one array model: the tag's platform isArray/arrayLength
keys (ValueRank=1, ArrayDimensions=[arrayLength] or [0] = variable length).
The driver reads them with TagConfigIntent's exact semantics and never rejects
on them, so composer, driver, authoring model and deploy inspector agree
(MqttArrayShapeParityTests). Shape mismatch (array metric on a scalar tag, or
the reverse) is BadTypeMismatch - a String tag no longer gets the packed
buffer as base64; a malformed body or a length differing from a declared
arrayLength is BadDecodingError (MTConnect #627 verdict). Template/DataSet
members of an array type take the same path. Arrays stay read-only (encoder
refuses them; MQTT is not in DriverArrayWriteSupport).
Browser: array leaves are bindable again and state isArray in AddressFields;
the commit mapper writes "isArray": true (no arrayLength). Discovery reports
IsArray/ArrayDim. Docs: Mqtt.md "Array metrics", design table, drivers rule.
Tests: codec (golden + malformed), ingest through the real codec + Dispatch
(the project's MiniBroker never forwards a PUBLISH, so there is no in-process
broker harness), browse, discovery, commit mapper, parity corpus. Mutating the
fix back out fails 18 MQTT + 9 AdminUI tests.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Gate: Primary-only typed recording with real status, raw+UNS HistoryRead/Processed parity, HistoryUpdate
insert/duplicate/replace/delete + WriteOperate-only and anonymous denied, failover at 1x, Modbus device loss
stored as BadCommunicationError then a BadNoCommunication outage marker. Gate found the CLI hid the
node-level refusal code (fixed + pinned) and a pre-existing OpcUaClient comms-loss gap (filed separately).
docker-dev compose gains ServerHistorian__Backend / __SqlServer__ConnectionString (default Gateway).
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
A. #566 on a real CompactLogix (Z-35162, read-only): a class-3 connection idled 60 s past its
16 s timeout is closed at the TCP level, not with CIP 0x0107/0x0203; the next request fails on
its write (Broken pipe -> BadConnection) in 9 ms and the one after reopens. The transport already
handles this, so no production behaviour changed; the shape is pinned offline by
CipConnectionTests.A_target_that_resets_an_idle_session_... (FakeEipServer.ResetAllConnections),
proven red with a reconnect-skipping mutation. The diagnostics-only exchange tap now also sees
Forward Open replies (granted RPIs).
B. Native STRING write through the driver on Program:ES03_TemperatureControl._StrTmp: write,
read back, restored byte-exact (user-approved). Array write NOT run: the census's unreferenced
DINT[32] _SEQ_EP arrays are aliased sequencer step registers (raw L5X grep), so none qualifies.
#567 still impossible (a 128 B array / 88 B STRING never fragments).
New env-gated harness: AbCip.IntegrationTests/HardwareGate (IdleConnectionEvidenceTests,
StringWriteGateTests). Docs: gate results (hw5 section + raw output), AbCip.md, CLAUDE.md.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Review M1: round 1 refused only the rewritten status bits, so S:/O:/I: writes - which
never worked on libplctag because tag creation failed - would have started reaching
the PLC (outputs, status-file words) on the default transport. WriteRefusal now keys
on LibplctagLegacyName.IsTranslated: file number supplied (S/O/I, incl. bit writes
like O:0/3, refused ahead of the parent-word RMW) or status bit rewritten. Reads
unaffected. Docs/comments now true; gate doc notes R bit-number wire evidence is
non-discriminating (L1). 12 new cases fail with the round-1 predicate.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
libplctag 2.6.0 (bundled by libplctag 1.5.2 / NativeImport 1.0.41) pccc.c needs an
explicit file number (parse_pccc_file_num; S is only the status file when a digit
follows) and has no R-file status-bit mnemonics (sub_element_lookup: con/len/pos only;
a /bit is refused on a 6-byte element). So the canonical S:0 (the default probe),
O:0/3, I:0/0 and R6:0.DN failed with BadCommunicationError on a real MicroLogix 1400
while native read them.
LibplctagLegacyName (libplctag adapter only) now emits S2:/O0:/I1: (the native
encoder's implicit file numbers) and R6:0.CON/<bit> for status bits libplctag lacks
(R EN/EU/DN/EM/ER/UL/IN/FD, C UA), bit numbers taken from the native table. The
authored grammar is unchanged. Unit tests pin the translation per family and check,
through a model of the pccc.c parser, that the translated name addresses exactly
what the native encoder does (46 fail without the fix). Live: Z-33323 idle, reads
only - libplctag now equals native for S/O/I/R status bits; controls unchanged.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Pass 2 (2026-10-08 00:21Z, df776170): 17/17. Browse 3162=3162 tags, 20 UDT
templates identical, 200 tags read with 0 unexpected divergences (6 live
values compared structurally + every still element), 1 Forward Open and 1
acked Forward Close per native phase. Latency native p95 105.8 ms vs
libplctag 139.5 ms (limit 153.4) PASS. Pass 1 failures and their fixes are
recorded in the gate-results doc; pass 2 had no independent pcap.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Joseph Doherty