ScadaBridge/docs/plans/2026-09-05-flow-debugger-design.md

# Flow Debugger — design

**Status:** implemented 2026-09-06 (branch `feat/flow-debugger`); plan `2026-09-05-flow-debugger.md` (24 tasks, all complete). Residual follow-ups: deferred-work register row 110. **v1.1 outbound calls** (a fourth *External* lane for API / DB / notification nodes, lifecycle folding, async/spawn edges, pending refresh) — see the addendum plan `2026-09-06-flow-debugger-outbound-calls.md`; §4's "three lanes fixed" is amended by its D2. **v1.2 follow-ups** (node Retry/Discard, `?correlationId=` deep links to the report pages, the prediction circuit-deadlock fix, Playwright coverage) — plan `2026-09-06-flow-debugger-followups.md`; register row 111 closed. **v1.3 script entry** (site → instance → script as a second entry point) — design `2026-09-08-flow-debugger-script-entry-design.md`. **v1.4 step logs and exceptions** (a `Log` global on every script surface, `ScriptTrace.ScriptLog` rows folded onto the step that produced them, the failing step's full exception on the node, per-run caps) — design `2026-09-09-flow-debugger-step-logs-design.md`, plan `2026-09-09-flow-debugger-step-logs.md`; register row 110's console-capture half closed by it.
**Mockup:** Claude Design canvas *Flow Debugger Mockup* — <https://claude.ai/code/artifact/3f3e81e5-8e5e-4333-bdf6-59256108494d> (Direction A swimlane boards + two low-fi alternates + decision notes).
**Owner components:** Central UI (#page), Inbound API, Site Runtime, Audit Log, Communication, Script Analysis, Configuration Database.

## 1. What it is

A Central UI page, **Deployment → Flow Debugger**, that runs one entry point with operator-supplied
inputs and renders every step the run passes through as a swimlane flowchart: lanes are *where* code
ran (Central, the Site the route landed on, the tag source), nodes are steps (request, script run,
route call, attribute write, wait, response) each carrying its input, output, duration and status.
Clicking a node opens a tabbed detail panel (Input · Output · Attribute writes · Console · Timing).

Worked example, and the acceptance flow for the whole feature: the Delmia recipe download
(`docs/api-specs/Delmia-Recipe-Download-API.md` §4) — inbound `POST /api/DelmiaRecipeDownload` →
central method script → `Route.To(MachineCode).Call("ProcessRecipeDownload")` → site Call script
on the `DelmiaReceiver` child → seven attribute writes → wait for Galaxy to raise
`RecipeProcessedFlag` → read result → response.

### Decisions taken (2026-09-05)

| # | Decision | Rejected |
|---|----------|----------|
| D1 | **Two modes: Live and Replay.** Live is a real run with real values, real writes, audited like any inbound call. Replay executes nothing — it rebuilds the flow from an existing ExecutionId's trace rows. | *Dry run* (execute scripts but capture writes/ES/DB/Notify instead of sending). Needs an interception seam in the site runtime that does not exist and would fork the semantics of every helper. |
| D2 | **Two entry points in v1:** an Inbound API method, and a value-change-triggered site script on a named instance. Both may land on a site script on instance X. | Alarm/timer trigger simulation, direct "call script X" — see §11. Superseded by `2026-09-08-flow-debugger-script-entry-design.md` (site → instance → script) as the page's second entry point. |
| D3 | **Value-change on a data-sourced attribute does a REAL tag write** through the instance actor's normal write path; the device echo fires the trigger. Correlation is by a short-lived *trace arm* on the instance actor (§5.3). | A debug-only "inject value" command that synthesises the change without touching the device — leaves the instance's cached value diverged from the device until the next real update. |
| D4 | **Live gets its trace inline; Replay reads audit rows.** Site audit telemetry forwards on a 5 s busy / 30 s idle cadence (`AuditLog:SiteTelemetry`), too slow to drive an interactive run, so the routed reply carries the site steps back and the same steps are *also* written as audit rows for Replay. | Live waiting on the audit pipeline. |
| D5 | **Tracing is gated, not always-on.** Sites run scripts at high rates from timers and value changes; a trace flag rides beside `ParentExecutionId` and only flagged runs emit trace rows. One exception is always-on: a central `RouteCall` row per `Route.To().Call` (inbound-driven volume, joins the two halves of a Replay). | Tracing every run (volume), or a purely time-window join. |
| D6 | **Live run is a Deployer act.** `RequireDeployment` gates the Run controls and the two run commands; the page and Replay are open to `OperationalAudit` roles + Deployer (`RequireFlowDebugger`, arch-review F3). | A separate `RequireOperator` gate for Live. |

## 2. What exists today (reused, not rebuilt)

- **Correlation ids already thread the chain.** `AuditWriteMiddleware` mints an `ExecutionId` per inbound request; `RouteHelper.Call` stamps it as `RouteToCallRequest.ParentExecutionId`; `InstanceActor` forwards it on `ScriptCallRequest.ParentExecutionId`; the site run mints its own id (`ScriptRuntimeContext._executionId`) and records the parent; a static write carries the writer's id (`SetStaticAttributeCommand.SourceExecutionId` → `AttributeValueChanged.SourceExecutionId`) into the value-change / alarm cascade. `IAuditLogRepository.GetExecutionTreeAsync` walks it.
- **Audit captures the ends.** `AuditKind.InboundRequest` stores the full request and response (up to `InboundMaxBytes`); `ApiCall`/`DbWrite`/`NotifySend` rows are per-execution.
- **Central already drives live instances.** `SiteCommandService` routing group: `RouteToCall`, `RouteToGetAttributes`, `RouteToSetAttributes`, `RouteToWaitForAttribute`; the Test Run sandbox's `SandboxInstanceGateway` runs a central script against a deployed instance over exactly this transport.
- **Roslyn compiles every script centrally** (`ScriptAnalysisService`, `ScriptTrustValidator`'s syntax walk), so the predicted graph is a sibling walker, not new infrastructure.
- **Attribute writes already have one path.** `WriteAttributeBatchRequest` → `InstanceActor.HandleWriteAttributeBatch` → `WriteTagBatchRequest` to the DCL for data-sourced attributes; `SetStaticAttributeCommand` for static ones.

## 3. What is missing (the build)

| Gap | Why it blocks | Closed by |
|-----|---------------|-----------|
| G1 No record that a site script ran — no `AuditKind` for a run; per-run site events are opt-in, aggregated, id-less. | Replay would show the inbound row and any outbound leaf rows with the site script itself absent. | §5.1 `ScriptRun` trace row. |
| G2 Attribute writes and waits are invisible — `WriteAttributeBatchRequest`/`WaitForAttributeRequest` carry no execution id; nothing records outcomes. | The Delmia flow's steps 5–7 cannot be drawn. | §5.1 `AttributeWrite`/`AttributeWait` rows + §6 additive fields. |
| G3 Route calls are not audited centrally and the routed reply does not return the child run's id. | Central and site halves of a Replay cannot be joined without guessing. | §5.1 `RouteCall` row (always-on) + `RouteToCallResponse.ChildExecutionId`. |
| G4 No trace gate. | Always-on would flood `AuditLog` from timer/value-change scripts. | §5.2 trace flag + per-method `TraceAllCalls`. |
| G5 A value-change entry needs a way to fire the trigger and correlate the echo. | Data-sourced echoes arrive as roots. | §5.3 trace arm. |
| G6 No page, no graph builder, no predictor. | — | §7, §8. |
| G7 Live runs have no API key, so no actor on the inbound row. | Audit would show an anonymous inbound call. | §9. |

## 4. User experience

Reference the mockup's two full-fidelity boards; this section only records what the mockup fixes.

**Toolbar.** Entry point (`Inbound API method` | `Value-change trigger`) + a picker for the method or
the instance + script; Mode (`Live` | `Replay`); `Edit inputs`, `Load from audit…`, `Run`.
Superseded by `2026-09-08-flow-debugger-script-entry-design.md`: the second entry point is
site → instance → script, not site → instance → attribute.

**Status strip.** State chip (`Not run` / `Running` / `Completed` / `Failed` / `Timed out`), run
number, wall-clock and node, total duration, counts (steps, boundaries crossed, attribute writes),
root `ExecutionId` linking to the existing Execution Chain page.

**Flow panel.** Three lanes fixed in v1: Central, Site (`{siteId} · {instance}`), Tags
(`{connection} → {source}`). Before a run the graph is *predicted* (§8) — nodes whose target
depends on an input value or on live site state render dashed. After a run every node is real;
predicted nodes that did not execute (an early-return branch not taken) render greyed. Node =
kind eyebrow + duration, title, one-line in/out summary; the selected node gets the accent border.
Connectors: solid = executed edge, accent = crossed a gRPC boundary, dashed = external actor
(the tag source responding). Zoom −/+/Fit only; no pan-by-drag in v1.

**Detail panel** (below the flow, tabbed): Input, Output, Attribute writes (attribute · before ·
written · quality · ack ms), Console, Timing; a right column *Where this ran* (site/node,
instance, composed child, connection, attribute kind, source line) with `Open in Debug View` and
`Audit rows (n)` drill-ins.

**Inputs form** (the `Step 1` detail before a run): generated from the method's
`ParameterDefinitions` (type, required) or, for a value-change entry, one field for the trigger
attribute's new value plus read-only current value/quality. `Fill from last real call` pre-fills
from the most recent `InboundRequest` row for that method.
Superseded by `2026-09-08-flow-debugger-script-entry-design.md`: the script entry point's Inputs
card is a `ParameterValueForm` from the script's own `ParameterDefinitions` for call kinds, and the
single value field (attribute derived from the trigger, not picked) for write kinds.

**Replay.** `Load from audit…` takes an ExecutionId (or arrives via a new `Open in Flow Debugger`
action on the Audit Log drawer and the Execution Chain page). The graph is built from trace rows
(§7.3); a run that was not traced degrades honestly — inbound row + `RouteCall` row + outbound
leaf rows, with the site interior rendered as one collapsed `not traced` node.

## 5. Trace model

### 5.1 Trace rows are audit rows

A new `AuditChannel.ScriptTrace` with kinds:

| Kind | Emitted by | Payload (`DetailsJson`) |
|------|-----------|-------------------------|
| `RouteCall` | Central `RouteHelper.Call` (**always-on**, not gated) | target instance, script, args (redacted per existing policy), return value, success/error, duration ms, `ChildExecutionId` |
| `ScriptRun` | Site `ScriptRunLauncher` on run completion (gated) | script, instance, trigger `{kind, attribute, value, source}` (a value-change run's trigger rides here — there is no separate `TriggerFired` kind), parameters, return value, success/error, duration ms, source node |
| `AttributeWrite` | Site `ScriptRuntimeContext.SetAttribute` / `WriteBatchAndWait` (gated) | one row per write or batch: writes[{attribute, written, success, error}], trigger attribute |
| `AttributeWait` | Site `ScriptRuntimeContext.WaitInternal` (gated) | attribute, predicate/target, timeout, outcome (satisfied/timeout/cancelled/error), elapsed ms, observed value |

**Who emits `AttributeWrite`/`AttributeWait` (planning correction 2026-09-05):** every accessor path
funnels into three `ScriptRuntimeContext` methods — `SetAttribute`, `WriteBatchAndWait` and
`WaitInternal` — and the context already holds `IAuditWriter`, `_executionId` and
`_parentExecutionId`, so the `InstanceActor` needs no new ids and no new request fields. Batch-write
echoes are **not** armed in v1 — only `SetStaticAttributeCommand` writes arm (§5.3).

Every row carries the run's `ExecutionId` and `ParentExecutionId` exactly as the existing channels
do, so the Execution Chain walk, `IX_AuditLog_ParentExecution`, telemetry forwarding, reconciliation
pulls, site 7-day retention and the central purge all apply unchanged. `RouteCall` is a
central-direct-write row (`ICentralAuditWriter`, `SourceNode = central-x`); the site kinds go
through the site `IAuditWriter` (SQLite hot path → telemetry → central).

Retention: `ScriptTrace` gets a per-channel override in `AuditLog:PerChannelRetentionDays`
(rig default 30 days; must stay under the 365-day global window per the existing rule).

### 5.2 The trace gate

A boolean `Trace` rides beside `ParentExecutionId` everywhere the parent id already travels:
`RouteToCallRequest`, `RouteToCallResponse` (echoed), `ScriptCallRequest`,
`SetStaticAttributeCommand`, and the new `SourceExecutionId` carriers in §6. Site-side the flag
lands in `ScriptRuntimeContext` as `IsTraced` and every helper consults it before emitting a
`ScriptTrace` row. Nested runs inherit it (`Scripts.Call` → `ScriptCallRequest.Trace`;
static-write cascades → `SetStaticAttributeCommand.Trace` → `AttributeValueChanged.Trace` →
the spawned run), so a traced root traces its whole tree and nothing else.

Two ways a run becomes traced:

1. **The debugger's Live run** sets it on the root.
2. **`ApiMethod.TraceAllCalls`** (new `bit NOT NULL DEFAULT 0` column, EF migration, CLI
   `api-method update --trace-all-calls`, Central UI checkbox on the method form): every real
   production call to that method is traced, so Replay of a real Delmia download is complete.
   Meant for low-volume integration methods; the method form shows the warning that it adds
   ~5 audit rows per call.

### 5.3 Value-change entry and the trace arm

The debugger writes the chosen attribute through the instance actor's normal path:

- **Static attribute:** `RouteToSetAttributes` → `SetStaticAttributeCommand` with
  `SourceExecutionId = <debugger root id>`, `Trace = true`. The existing cascade already stamps
  the resulting `AttributeValueChanged` and the spawned run's `ParentExecutionId`. Nothing new.
- **Data-sourced attribute:** the **same** `RouteToSetAttributes` → `SetStaticAttributeCommand`
  path — `InstanceActor` already dispatches data-sourced attributes to the DCL
  (`HandleSetDataAttribute` → `WriteTagRequest`); no `WriteAttributeBatchRequest` change is needed.
  The device echo comes back through the DCL as an ordinary `AttributeValueChanged` with no
  source. **Trace arm:** when `HandleSetDataAttribute` sees `command.Trace &&
  command.SourceExecutionId != null` it records `(attribute → sourceExecutionId, expiresAt =
  now + TraceArmWindow)` (default 30 s, `SiteRuntimeOptions`, one entry per attribute, newest
  wins). The next DCL value change for that attribute — whatever the value, so a device that
  clamps or rejects the write still shows what it actually did — is published with
  `SourceExecutionId` and `Trace` set (consumed in `HandleTagValueUpdate`), and the arm is
  consumed. An arm that expires unconsumed
  emits nothing; the debugger renders the trigger node as `no echo within 30 s`.

The arm exists only for traced writes, so production behaviour is unchanged. Alarm cascades that
the echo also fires inherit the parent id exactly as static cascades do today.

### 5.4 Live-run inline transport

**A trace step IS an `AuditEvent` in channel `ScriptTrace`** — there is no separate wire record.
`RouteToCallResponse` gains additive `ChildExecutionId` (`Guid?`) and `TraceEvents`
(`IReadOnlyList<AuditEvent>?`, the very events that became the site's audit rows, in emission
order); on the wire it is `repeated AuditEventDto trace_events` (the DTO `sitestream.proto`
already defines). The site collects events in a per-run `FlowTraceCollector` (Commons
`Types/Debugging/`, shared with the central `FlowDebugRunner`) held by `ScriptRuntimeContext` and
attaches the list to the `ScriptCallResult` → `RouteToCallResponse` when `Trace` is set. The
collector applies `IAuditRedactor` inside `Add`, so the inline record is the same filtered event
the writer persists, and it caps at `MaxTraceEventsPerRun` (500) **or** 1 MB of summed
`DetailsJson` (`DefaultMaxBytes`), whichever comes first; beyond either bound it keeps the earlier
events, sets `Truncated`, and the audit rows remain complete (arch-review F1). A site on an older build ignores
`Trace`, returns no `TraceEvents`, and the UI labels its lane `site not traced (older build)`.
`FlowStep` is a **central-side projection** (`Commons/Types/Debugging/FlowStepProjector`)
computed from `AuditEvent` + `AuditDetails`; nothing serializes it.

For the value-change entry there is no routed reply to ride on. The runner polls the site's
`PullTraceEvents` RPC by `ParentExecutionId` (1 s cadence, 2 s settle window, `WaitSeconds` bound) —
simpler than the debug-stream subscription originally planned. `AttributeValueUpdate.source_execution_id`/`trace`
(fields 7/8) are populated by the site and **consumed by nothing**; they are reserved for a future
debug-stream path and must not be reused. The RPC sits behind the same PSK interceptor as
`PullAuditEvents` and reads the site audit SQLite by `ExecutionId` / `ParentExecutionId` — one
additive RPC, no streaming change.

## 6. Message and wire changes (all additive)

| Contract | Change |
|----------|--------|
| `RouteToCallRequest` | `+ bool Trace = false` |
| `RouteToCallResponse` | `+ Guid? ChildExecutionId`, `+ IReadOnlyList<AuditEvent>? TraceEvents` |
| `ScriptCallRequest` / `ScriptCallResult` | `+ bool Trace`; `+ Guid? ExecutionId`, `+ IReadOnlyList<AuditEvent>? TraceEvents` on the result |
| `RouteToSetAttributesRequest` | `+ Guid? ParentExecutionId`, `+ bool Trace` |
| `SetStaticAttributeCommand`, `AttributeValueChanged` | `+ Guid? SourceExecutionId` (where missing), `+ bool Trace` |
| `site_command.proto` | `trace` on `RouteToCallRequestDto`/`RouteToSetAttributesRequestDto`; `child_execution_id` + `repeated AuditEventDto trace_events` on `RouteToCallResponseDto` (no `FlowStepDto`); regenerate with `docker/regen-proto.sh sitecommand` |
| `sitestream.proto` | `+ PullTraceEvents` RPC (`PullTraceEventsRequest { repeated execution_ids, since_utc, max_events }` → `PullTraceEventsResponse { repeated AuditEventDto events }`); `AttributeValueUpdate` `+ string source_execution_id = 7`, `+ bool trace = 8` so the debug stream can hand the debugger the spawned id |
| `ApiMethod` | `+ TraceAllCalls bit NOT NULL DEFAULT 0` (EF migration + manual SQL script) |
| `AuditChannel`/`AuditKind` | `ScriptTrace`; `RouteCall`, `ScriptRun`, `AttributeWrite`, `AttributeWait` |

`FlowStep` (Commons `Types/Debugging/FlowStep.cs`, **projection only — never on the wire**):
`EventId`, `ExecutionId`, `ParentExecutionId`, `Kind`, `Lane` (`Central`/`Site`/`Tags`), `Title`,
`Subtitle`, `OccurredAtUtc`, `DurationMs`, `Outcome`, `InputJson`, `OutputJson`, `ErrorMessage`,
`SiteId`/`Node`/`Instance`/`Target`, and a kind-specific `ScriptTraceExtra` parsed from
`AuditDetails.Extra`.
Field numbers never reused; proto3 defaults are the negotiation, as for stream batching.

## 7. Central components

### 7.1 `FlowDebugService` (Central UI, scoped)

- `PredictAsync(entry)` → §8.
- `RunMethodAsync` / `RunValueChangeAsync` / `ReplayAsync` → Live runs and Replay are
  `FlowDebugRunMethodCommand` / `FlowDebugRunValueChangeCommand` / `FlowDebugReplayCommand` on the
  `ManagementActor` (Deployer role, long-running timeout). The `ManagementActor` is a **per-node**
  actor (`AkkaHostedService`, `/user/management`), not a singleton. The run-method handler
  consults `IActiveNodeGate` and refuses on a standby with error code `STANDBY_NODE` naming the
  design rule (Component-InboundAPI: method scripts and `Route.To` never run on a standby);
  Traefik routes UI traffic to the active node so this is rare. The Central UI calls it through
  `ManagementActorHolder` (the `DataConnectionMoveService` pattern) and the CLI through
  `/management` — one path, one audit story. The inbound row for a Live run is written by the
  handler via `ICentralAuditWriter` (kind `InboundRequest`, actor = operator username,
  `Extra.source = "flow-debugger"`). Inbound entry: `InboundScriptExecutor.ExecuteAsync(method,
  inputs, route, timeout, parentExecutionId: root, trace: true)` with a `RouteHelper` whose
  `Trace` flag is set and whose inline trace sink collects the `RouteCall` row plus the site's
  `TraceEvents`. Value-change entry: §5.3 + §5.4.
- `ReplayAsync(executionId)` → `IAuditLogRepository.GetExecutionTreeAsync` + a new
  `GetTraceEventsAsync(rootExecutionId)` (all `ScriptTrace` rows in the tree, plus the
  `InboundRequest` row) → `FlowGraphBuilder`.

### 7.2 `FlowGraphBuilder` (pure, Commons or Central UI)

Steps → `FlowGraph { Lanes, Nodes, Edges }`. Lane assignment by `Where` (central node → Central;
site node → Site; `AttributeWrite` → Tags). Edges follow emission order within an
execution and parent→child across executions; an edge whose endpoints sit on different lanes is
marked `CrossesBoundary`. A predicted graph and a real graph share the model; the page overlays
them (predicted-but-not-executed → greyed).

### 7.3 Page and renderer

`Components/Pages/Deployment/FlowDebugger.razor` (+ `.razor.cs`), nav item under Deployment after
Debug View. Renderer is a custom SVG component `FlowGraphView` (no third-party charting or diagram
libraries, matching `KpiTrendChart`): fixed 110 px lanes, nodes placed by topological column,
connectors as orthogonal paths with arrow markers; zoom via a `viewBox` scale. Detail panel is a
plain Bootstrap tab strip. Persisted per-browser: last entry point and inputs (same
`localStorage` pattern Debug View uses for its selection).

## 8. Predicted graph (static analysis)

`FlowShapePredictor` in Script Analysis (#25), a `CSharpSyntaxWalker` beside
`ScriptTrustValidator.HardeningWalker`, over the entry script and every script it can reach:

- Recognised sites: `Route.To(x).Call(name, …)` / `.SetAttributes` / `.WaitForAttribute` /
  `.GetAttributes`; `Scripts.Call(name)` / `CallShared`; `Attributes["a"] = v`,
  `Children["c"].Attributes["a"] = v`; `WriteBatchAndWaitAsync(…)`, `WaitAsync(…)`,
  `WaitForAsync(…)`; `ExternalSystem.Call/CachedCall`; `Database.*`; `Notify.Send`; `return`.
- Constant string arguments resolve; anything else yields a dashed node (`target = input value`).
  `Route.To(constant)` resolves the instance → template → the named script's body, which is then
  walked too (one hop; depth cap 4, the same as the runtime call-depth cap).
- Branches: an `if … return` before a recognised site becomes an `early return` annotation on the
  edge, as the mockup shows; the predictor does not attempt path-sensitivity beyond that.
- Output is a `FlowGraph` with `Predicted = true`; the page renders it before the run and overlays
  the real graph after.

Compile is the existing cached compile (`CachingScriptMetadataResolver` attached like every other
surface); the walker is syntax-only and never executes anything.

## 9. Security, authorisation, audit

- **Page visibility:** `RequireFlowDebugger` — the `OperationalAudit` roles plus Deployer
  (`AuthorizationPolicies.FlowDebuggerAccessRoles` = Deployer, Administrator, Viewer), so an
  auditor can open the page and Replay a past execution (arch-review F3). **Live run:**
  `RequireDeployment` (same as Debug View) — decided 2026-09-05: a Live run writes real tags, but
  the Deployer already deploys and redeploys the scripts that do exactly that, so no separate
  Operator gate. The two run commands, the Live controls, the parked-node Retry/Discard actions and
  the `?methodId=`/`?instanceId=` drill-ins stay Deployer-only, and the page pins Replay mode for a
  non-Deployer. **Replay:** `FlowDebugReplayCommand` accepts the same `OperationalAudit` roles (it
  reads audit rows). No Verifier step — the debugger is a diagnostic, not a production write
  control, and the audit row is the safeguard.
- **Inbound audit row for a Live run:** the shared `AuditEvent.Actor` carries the operator's
  username (today it carries the API-key name); `Extra.source = "flow-debugger"`,
  `Extra.runNumber`; `Target` is the method as usual. API-key authentication is bypassed (there is
  no key), **parameter validation is not** — `ParameterValidator` runs exactly as for HTTP so a
  bad input fails the same way. The method's `TimeoutSeconds` applies. Site-scope (`SiteScopeService`)
  restricts which instances a value-change entry may target. The method entry enforces the
  operator's site scope at route time (`RouteHelper.WithSiteScope`, `SITE_SCOPE_DENIED` →
  `ManagementUnauthorized`, arch-review F2): the runner resolves the operator's permitted numeric
  site ids to site identifiers and hands them to `RouteHelper`, which refuses an out-of-scope target
  **before any site traffic**; the denial is rethrown as `SiteScopeViolationException` *after* the
  inbound audit row is written, so the attempt is recorded under the operator's name.
  Administrators and unscoped users are exempt, exactly as `EnforceSiteScope` treats them.
- Trace payloads pass through the existing redaction policy on BOTH copies: the stored row (writer)
  and the inline Live copy (`FlowTraceCollector` applies `IAuditRedactor` and a 1 MB byte budget
  beside the 500-event cap — arch-review F1). The row-level caps are unchanged (auth headers
  redacted, 8 KB cap, 64 KB on error rows). Console output is capped the same way.
- Nothing here touches the PSK boundary; `PullTraceEvents` sits behind `ControlPlaneAuthInterceptor`.

## 10. Volume and failure behaviour

- Untraced production is unchanged except one `RouteCall` row per routed call (inbound-driven).
  Every routed call adds one central `RouteCall` insert — at inbound-driven rates that doubles the
  per-call central audit write (accepted, D5).
- A traced Delmia call emits ≈ 1 `InboundRequest` + 1 `RouteCall` + 1 `ScriptRun` + 2
  `AttributeWrite` + 1 `AttributeWait` + 1 value-change `ScriptRun` ≈ **7–10 rows** (a cached DB
  write and a notification add 3 trace-only rows). `TraceAllCalls` on a 100-call/day method is
  ~700–1 000 rows/day.
- Trace emission is best-effort like all audit (a failed trace write never fails the run).
- Live run timeout = method timeout + `FlowDebugRunGrace` (10 s); a timeout renders whatever steps
  arrived and marks the run `Timed out`. The site's trace rows still land later, so Replay of the
  same ExecutionId is complete. The client waits longer than the server on purpose: the UI derives
  its Ask wait from the same bound (`RunAskTimeoutFor` = method timeout + grace + 15 s slack,
  floored at 30 s; `ValueChangeAskTimeoutFor` = the wait window + 15 s) and the CLI's
  `run-method --timeout-seconds` defaults to 120 while `run-value-change` waits
  `--wait-seconds + 25` (arch-review F4).
- **The Live run's inbound row is written AFTER the run completes**, so a Replay taken during a
  long Live run shows only the `RouteCall`.
- Older site build: no `TraceEvents`, lane labelled; older central build: ignores new fields; proto3
  defaults negotiate as for stream batching.
- **Mixed-version caveat:** an OLD central ingesting `ScriptTrace` rows from a NEW site parses
  unknown channel/kind names to the `AuditRowProjection` fallbacks (`ApiInbound`/`InboundRequest`);
  upgrade central first.

## 11. Out of scope (recorded so they are not rediscovered)

- Dry run (D1). Alarm-trigger and timer-trigger entry points. Step-into for `CallShared` inline
  shared scripts (they execute inline; only `Scripts.Call` spawns a traceable child). Live
  step-by-step / breakpoints. Editing a script from the debugger. Cross-run comparison. A
  `Vertical trace` list toggle (mockup Direction B) — cheap follow-up on the same `FlowGraph`.
  Line-level pinning (Direction C) — needs per-statement instrumentation. Console output capture
  on `ScriptRun` rows (no script console exists today).

## 12. Testing

- **Pure:** `FlowGraphBuilderTests` (lane/edge/boundary rules, truncation), `FlowShapePredictorTests`
  (each recognised site, constant vs input-dependent target, one-hop route resolution, depth cap,
  early-return annotation), `FlowTraceCollectorTests` (cap + `Truncated`).
- **Site runtime:** `ScriptRuntimeContext` traced write emits `AttributeWrite`; `InstanceActor` arms; echo consumes the
  arm and stamps `SourceExecutionId`/`Trace`; expiry emits nothing; untraced write emits nothing
  and arms nothing (the production-unchanged pin). `ScriptRunLauncher` emits `ScriptRun` only when
  traced. `ScriptRuntimeContext.WaitInternal` emits `AttributeWait` for satisfied/timeout.
- **Inbound API:** `RouteHelper.Call` always writes `RouteCall`; `ChildExecutionId` round-trips;
  `Trace` propagates to `RouteToCallRequest`.
- **Communication:** DTO mapper round-trips for every new field; an old-shape response (no
  `TraceEvents`) maps to null.
- **Audit:** new kinds accepted by ingest; per-channel retention deletes `ScriptTrace` rows early;
  `GetTraceEventsAsync` returns the tree's rows in order.
- **Central UI (bUnit):** page renders predicted graph, overlays real graph, Run button hidden
  without Deployer, Replay degrades to the collapsed `not traced` node.
- **Rig:** extend `docker/verify-delmia-notifier.sh` with step H — a Live run of
  `DelmiaRecipeDownload` against `DelmiaSim01` from the page's service, asserting 9 steps, and a
  Replay of the same ExecutionId producing the same graph.

## 13. Documents to update with the implementation

`Component-CentralUI.md` (new page), `Component-InboundAPI.md` (RouteCall row, TraceAllCalls,
Live-run actor), `Component-SiteRuntime.md` (trace collector, trace arm, gated emission),
`Component-AuditLog.md` (ScriptTrace channel, retention override, PullTraceEvents),
`Component-Communication.md` (proto changes), `Component-ScriptAnalysis.md` (predictor),
`Component-ConfigurationDatabase.md` (migration), `README.md` component table if a component is
added (none planned — the debugger is a Central UI page plus seams in existing components),
`CLAUDE.md` key-decisions bullet, CLI README (`--trace-all-calls`), deferred-work register row for
the Direction B list toggle.