feat(scripts): read-only, allow-listed Files global for site scripts; Delmia test receiver parses recipe files
Script file access (design docs/plans/2026-09-28-script-file-access-design.md): Files.ReadTextAsync / ReadLinesAsync / ExistsAsync on site ScriptGlobals (+ Instance.Files), backed by the site-singleton ScriptFileReader. The one host-mediated exception to the System.IO ban - scripts still never name a System.IO type. OFF by default: ScadaBridge:SiteRuntime:ScriptFileAccess:AllowedRoots (empty = every call throws Disabled) + MaxFileBytes (1 MiB, validated 1..64 MiB, roots must be absolute). Guard: fully qualified, no device paths / ADS, canonical path strictly inside a root (directory-boundary match), no reparse point on the file or any directory below the root, size cap before and during the read. Errors are one Commons type, ScriptFileAccessException(+ScriptFileAccessFailure); Commons.Types.Scripts joins the default script imports on every surface so it binds by short name. Mirrored on ScriptCompileSurface (CompileFiles, parity-pinned) and the Test Run sandbox (binds, refuses Disabled). Inbound (central) scripts do not get it. Rig: docker/recipes (sample RecipeValueName,RecipeValue files incl. an invalid one) bind-mounted read-only at /app/recipes on both site-a nodes with AllowedRoots [/app/recipes]. The TestMesMachine SimDelmiaRecipe simulator now reads + parses the downloaded file, converts each value to its recipe tag's type and loads it all-or-nothing (14 new TestDelmiaReceiver attributes); verify-mes-test-machine.sh group A covers accept / re-download / invalid lines / NotFound / OutsideAllowedRoots, VERIFY_GROUPS selects groups (rig: AD).
This commit is contained in:
@@ -0,0 +1,55 @@
|
||||
# Script file access — the `Files` global (2026-09-28)
|
||||
|
||||
## Problem
|
||||
|
||||
A Delmia recipe download hands the machine a `DownloadPath`; in production the Galaxy `DelmiaReceiver`
|
||||
object's own script opens that text file, parses it and loads the values into its tags. ScadaBridge scripts
|
||||
cannot do the same thing: the trust policy forbids `System.IO` outright, and the deploy gate refuses any
|
||||
script that names a `System.IO` type. So an in-memory test machine (and any future ScadaBridge-side
|
||||
receiver) could exercise the handshake but never the file parse.
|
||||
|
||||
## Options considered
|
||||
|
||||
1. **Leave parsing in the Galaxy.** No change; test machines never parse.
|
||||
2. **A small file-reader HTTP service reached through `ExternalSystem.Call`.** No trust-model change, but a
|
||||
new service to deploy and secure on every site.
|
||||
3. **A host-mediated, allow-listed platform API.** *Chosen* (user decision). A deliberate, narrow exception
|
||||
to the trust boundary: the script never touches `System.IO`; the host reads on its behalf, only beneath
|
||||
directories an administrator lists.
|
||||
|
||||
Rejected outright: reading through SQL Server (`OPENROWSET(BULK …)` via `Database.Connection`) — needs
|
||||
elevated SQL rights, reads the SQL host's filesystem, and sidesteps the trust policy.
|
||||
|
||||
## Decisions
|
||||
|
||||
- **Surface.** `Files.ReadTextAsync(path)`, `Files.ReadLinesAsync(path)`, `Files.ExistsAsync(path)` on the
|
||||
site `ScriptGlobals` (+ `Instance.Files`), each taking an optional `CancellationToken`. Read-only — no
|
||||
write, list, delete or directory enumeration; add one only with its own use case.
|
||||
- **Off by default.** `ScadaBridge:SiteRuntime:ScriptFileAccess:AllowedRoots` (absolute local/UNC dirs,
|
||||
default empty) and `MaxFileBytes` (default 1 MiB, 1…64 MiB). No roots ⇒ every call throws `Disabled`, so
|
||||
upgrading a site changes nothing until an operator opts in. Validated at boot.
|
||||
- **Guard (host side, `ScriptFileReader`).** Fully-qualified path; no device paths or alternate data
|
||||
streams; canonicalize, then a directory-boundary prefix match against each root; no reparse point on the
|
||||
file or any directory between root and file; size cap enforced before and during the read.
|
||||
- **One exception type in Commons** — `ScriptFileAccessException` + `ScriptFileAccessFailure` — because
|
||||
scripts cannot name `System.IO` exceptions. `Commons.Types.Scripts` joins the default script imports on
|
||||
every surface so it binds by short name.
|
||||
- **Site scripts only.** Inbound API scripts run on central and have no `Files`. The Test Run sandbox
|
||||
mirrors the member (so scripts bind) and throws `Disabled` at run time. `ScriptCompileSurface.CompileFiles`
|
||||
mirrors it for the deploy gate; parity is pinned by `CompileSurfaceParityTests`.
|
||||
- **Not audited.** File reads are not one of the audit channels; refusals log a node Warning.
|
||||
|
||||
## Accepted residuals
|
||||
|
||||
- Time-of-check/time-of-use between the link check and the open: a writer inside an allowed root could swap
|
||||
a file for a link in between. Bounded by the allow-list (only administrator-chosen directories) and by the
|
||||
API being read-only.
|
||||
- The node's service account permissions still apply; on a site pair both nodes need the same roots and the
|
||||
same access (a script can run on either node after failover).
|
||||
|
||||
## First consumer
|
||||
|
||||
`docker/seed-mes-test-machine.sh`: the `SimDelmiaRecipe` simulator on `TestMesMachine` reads the downloaded
|
||||
recipe file (`RecipeValueName,RecipeValue` per line), types each value (int → double → bool → string) and
|
||||
loads it into the matching `TestDelmiaReceiver` recipe attributes, rejecting unknown names and malformed
|
||||
lines. Sample files: `docker/recipes/`.
|
||||
@@ -32,7 +32,7 @@ The following namespace/type prefixes are forbidden in all scripts:
|
||||
|
||||
| Scope | Rationale |
|
||||
|-------|-----------|
|
||||
| `System.IO` | File system access — forbidden entirely |
|
||||
| `System.IO` | File system access — forbidden entirely. The only file access a script has is the host-mediated, allow-listed, read-only `Files` global on site scripts (2026-09-28, `Component-SiteRuntime.md` → *File Access*) — the script never touches a `System.IO` type, so this scope is unchanged |
|
||||
| `System.Diagnostics.Process` | Process spawning — forbidden; `Stopwatch`, `Debug`, `Activity`, and other `System.Diagnostics` types are **allowed** |
|
||||
| `System.Threading` | Raw thread manipulation — forbidden, with the exceptions below |
|
||||
| `System.Reflection` | Reflection — forbidden entirely |
|
||||
@@ -92,7 +92,7 @@ The identifiers `dynamic` and `Activator` are forbidden at any scope, as they pr
|
||||
- For each identifier in the syntax tree, resolves the underlying symbol to its fully qualified containing namespace and type name.
|
||||
- Flags any symbol whose containing namespace or type matches a forbidden scope in `ScriptTrustPolicy.ForbiddenScopes`, taking `AllowedExceptions` into account.
|
||||
- Correctly handles aliases (`using X = System.IO.File`), `using static`, and `global::` prefixes — the resolved symbol is checked, not the spelling.
|
||||
- The validation compilation applies the **same default namespace imports** (`ScriptTrustPolicy.DefaultImports` — `using System;`, `System.Collections.Generic`, `System.Linq`, `System.Threading.Tasks`) that `RoslynScriptCompiler` compiles the real script with. Without them the semantic pass under-resolves **bare** identifiers the running script sees fully bound — e.g. `Environment.Exit(0)` or `GC.Collect()`, whose `Environment`/`GC` resolve to the forbidden `System.Environment`/`System.GC` types only when `using System;` is in scope. Matching the runtime import set closes that blind spot; it can only make more identifiers resolve to their true (forbidden) namespace, never produce a false allow.
|
||||
- The validation compilation applies the **same default namespace imports** (`ScriptTrustPolicy.DefaultImports` — `using System;`, `System.Collections.Generic`, `System.Linq`, `System.Threading.Tasks`, and since 2026-09-28 `ZB.MOM.WW.ScadaBridge.Commons.Types.Scripts`, so script-facing Commons types such as `ScriptFileAccessException` bind by short name) that `RoslynScriptCompiler` compiles the real script with. Without them the semantic pass under-resolves **bare** identifiers the running script sees fully bound — e.g. `Environment.Exit(0)` or `GC.Collect()`, whose `Environment`/`GC` resolve to the forbidden `System.Environment`/`System.GC` types only when `using System;` is in scope. Matching the runtime import set closes that blind spot; it can only make more identifiers resolve to their true (forbidden) namespace, never produce a false allow.
|
||||
- Because the full reference set is loaded, this pass also catches a forbidden type accessed inside an otherwise-allowed namespace (e.g., bare `Process` after `using System.Diagnostics;`).
|
||||
|
||||
##### `AnalysisReferences` vs `DefaultReferences`
|
||||
|
||||
@@ -520,6 +520,17 @@ Available to all script runs and alarm on-trigger runs:
|
||||
- The type lives in Commons so the runtime accessor and the Script Analysis compile-only surface (`ScriptCompileSurface.CompileAlarmsAccessor`) project to the **same** type — a script binding at the design-time gate binds identically at the site. Parity is pinned by `CompileSurfaceParityTests`.
|
||||
- Design record: `docs/plans/2026-06-30-mes-alarm-status-api.md` §5.2.
|
||||
|
||||
### File Access (`Files`, read-only, allow-listed — 2026-09-28)
|
||||
- `Files.ReadTextAsync(path)` → `string`, `Files.ReadLinesAsync(path)` → `IReadOnlyList<string>` (`\r\n`/`\n`/`\r`; a trailing line break adds no empty last line), `Files.ExistsAsync(path)` → `bool`. Also reachable as `Instance.Files`. The use case that motivated it: a recipe/NC file whose path a Delmia/MES call hands to the machine, which a script parses and loads into attributes.
|
||||
- **The one sanctioned exception to the `System.IO` ban, and it is host-mediated.** Scripts still cannot name any `System.IO` type — the trust policy is unchanged. The runtime `FilesAccessor` delegates to the site-singleton `ScriptFileReader` (`SiteRuntime/Scripts/`), which does every check on the host side.
|
||||
- **Off by default.** `ScadaBridge:SiteRuntime:ScriptFileAccess:AllowedRoots` (list of absolute local/UNC directories, default empty) enables it; with no roots every call throws `ScriptFileAccessException(Disabled)`, so an unconfigured site behaves exactly as before. `ScriptFileAccess:MaxFileBytes` (default 1 MiB, validated 1…64 MiB) caps one read. Both validated at boot (`SiteRuntimeOptionsValidator` — a relative root is rejected). Read per call through `IOptionsMonitor`.
|
||||
- **Guard, in order:** roots configured → path non-empty, fully qualified, not a device path (`\\?\`, `\\.\`), no NTFS alternate data stream → canonical path (`Path.GetFullPath`, so `..` is resolved first) **strictly inside** a root with a directory-boundary match (root `C:\recipes` does not admit `C:\recipes2\x`, and the root itself is not a file) → no symbolic link / junction / reparse point on the file **or on any directory between the root and the file** (the root itself may be a link — an administrator chose it) → size under the cap, re-checked while reading in case the file grows. Files are opened `FileShare.ReadWrite | Delete` so a file still being written by its producer can be read; text is BOM-detected, UTF-8 by default.
|
||||
- **Errors are one Commons type.** Every refusal and I/O failure is `ScriptFileAccessException` (`Commons.Types.Scripts`) with a `Failure` of `Disabled`, `InvalidPath`, `OutsideAllowedRoots`, `LinkNotAllowed`, `NotFound`, `TooLarge` or `IoError` — scripts cannot name `System.IO` exception types, so this is what they `catch`. `Commons.Types.Scripts` is now one of the **default script imports** on every surface (runtime, deploy gate, Test Run editor), so the short name binds.
|
||||
- **Bounded by the run.** Reads are cancelled by the script's own execution deadline as well as any token the script passes. Refusals (outside roots, links) log a Warning on the node; successful reads log at Debug. There is no audit row — file reads are not an audited channel (the `AuditLog` scope is outbound API/DB, notifications and inbound API).
|
||||
- **The node's service account still needs read permission** on the path; the allow-list only ever narrows access. A time-of-check/time-of-use window exists between the link check and the open (a writer inside an allowed root could swap a file for a link in between) — acceptable because only the directories an administrator lists are reachable at all, and read-only.
|
||||
- **Site scripts only.** Inbound API method scripts (central) have no `Files` global. The Central UI Test Run sandbox mirrors the member so scripts bind, but every call throws `ScriptFileAccessException(Disabled)` — central cannot reach a site's filesystem — so a script's own error handling runs as it would on an unconfigured site. The design-time `ScriptCompileSurface.CompileFiles` mirror is pinned by `CompileSurfaceParityTests`.
|
||||
- Design record: `docs/plans/2026-09-28-script-file-access-design.md`.
|
||||
|
||||
### Logging (`Log`)
|
||||
- `Log.Debug(message, exception?)` / `Log.Info` / `Log.Warn` / `Log.Error` — leveled, per-run logging available to **every** script, alongside the `Alarms` accessor above (design `docs/plans/2026-09-09-flow-debugger-step-logs-design.md` D4/D5). Synchronous, `void`, and it **never throws into the script**: a failing sink is swallowed, because a debug observer must not fail a production run.
|
||||
- **Every line always reaches the node's `ILogger`** (structured `Instance` / `Script` / `ExecutionId` / message), traced or not — an untraced run is an ordinary logging surface and pays nothing else. A `ScriptTrace` / `ScriptLog` **audit row** is emitted **only when the run is traced** (a Flow Debugger Live run, or a real call to a method with `TraceAllCalls`), so a timer or value-change script logging at high rate costs no audit volume. There is no site-event-log sink for `Log.*`.
|
||||
|
||||
Reference in New Issue
Block a user