Note the shared InstancePicker combobox (2026-09-08 design) replacing
the flat instance <select> on both pages, in Component-CentralUI.md's
Flow Debugger and Debug View sections and the CLAUDE.md Flow Debugger
bullet.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
The trigger-cascade (Spawn) edge used a lightning-bolt marker IN PLACE of the arrowhead. On the upward
Tags→Site hop the bolt sat on top of the riser and read as a broken arrow (rig screenshot 2026-09-08).
Every connector now ends in an ordinary arrowhead; the bolt is a badge (circle + bolt) centred on the
connector's vertical run (FlowGraphView.SpawnBadgePoint, kept in step with OrthogonalPath/AsyncPath).
Same screenshot, second defect: the cascade spawned by "RecipeDownloadFlag = false" was drawn from the
LAST write of the burst (RecipeProcessedFlag) because all four writes share a millisecond and the
spawner rule was time-only. FlowGraphBuilder now prefers the parent's last AttributeWrite of the attribute
the child's triggerAttribute names (at or before the child, else the last such write); the time/position
rules remain the fallback when the child names no attribute or the parent never wrote it.
Verified live on the rebuilt rig (DelmiaSim01 / DelmiaReceiverSimulator): both spawn edges show an
arrowhead + badge, the second one originates from the RecipeDownloadFlag write. bUnit + builder tests
updated/added; Playwright FlowDebuggerTests 4/4.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Replaces the value-change (site/instance/attribute) entry with a script picker.
ValueChange/Conditional scripts keep the write-and-poll path with the attribute
derived from the trigger config; every other kind is invoked directly through the
traced RouteToCall path with a parameter form from the script's ParameterDefinitions.
New additive FlowDebugRunScriptCommand; run-value-change stays as the CLI escape hatch.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Code review of the merged arch-review remediation (Tasks 1-13) found four items.
1. FlowDebugRunner returned its OWN synthesized rows un-redacted. RunMethodAsync's
inbound row and RunValueChangeAsync's root row were handed to WriteCentralAsync
(whose writer redacts a private copy) and then put RAW into the returned Events —
so a Live run's caller saw the unfiltered row while Replay of the same execution
showed the redacted one, defeating F1's "Live shows what Replay reads back". Both
methods now resolve IAuditRedactor once per run and place the filtered copy into
the result (including the early-return path on a failed attribute write); the raw
row still reaches the writer, which redacts itself.
2. FlowGraphBuilder's spawner search degenerated to "the last step of the parent".
Since executions are ordered as contiguous blocks, position[p] < childPosition
holds for EVERY parent step, so a run shaped ScriptRun -> Write(A) -> Wait ->
Write(B) whose Write(A) fired the cascade drew the Spawn edge from Write(B). New
rule, in order: (a) the last non-async, non-Response parent step occurring at or
before the child's first step — parent and child of a trigger cascade share the
site clock, so this picks the write that actually fired it; (b) the position rule
as fallback, which is the mixed-clock case F5 was written for (a child clock
behind its parent's, where no parent step satisfies (a)); (c) parentChain[0].
The class remark's stale "Replay order is unchanged" claim is replaced with what
actually happens to a central Database.* call made after Route.To().Call.
3. FlowTraceCollector's byte budget counts UTF-16 chars of DetailsJson only, so
"well under the 4 MB gRPC default" overstated it. Documented honestly: ~1 MB of
UTF-8 for ASCII payloads, ~3 MB worst case for non-ASCII, both under 4 MB before
framing — and raise it only against a real serialized-size measurement.
4. CLI --timeout-seconds help + README row now say the management endpoint caps
flow-debug run-method at ManagementService:LongRunningCommandTimeout (default
5 min), so a larger client value still 504s at the server's ceiling.
Deferred-work register rows 112 (site-side inline trace truncation is invisible to
the operator — needs a proto field) and 113 (the F2 method-entry site scope gates
routed traffic only; ApiMethods have no site affinity to scope against) record what
this review deliberately did not fix.
Tests: FlowGraphBuilderTests gains the same-clock Write(A)/Wait/Write(B) cascade
(red before the rule change, and the clock-behind test still passes on rule (b));
FlowDebugRunnerTests extends the redactor test to the inbound row and adds two
value-change cases (settled poll, failed write). Commons 793, ManagementService 615,
CentralUI 1264, CLI 462 — all green.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
The Flow Debugger design doc still described the pre-remediation shape: a
debug-stream subscription for the value-change entry, Deployer everywhere, an
unredacted inline trace copy and "~7 audit rows per call". Reconciled with what
shipped on this branch (Tasks 1-12):
- design D6 / §5.4 / §9 / §10: RequireFlowDebugger page policy with Deployer-only
execution (F3), the PullTraceEvents polling path with AttributeValueUpdate
fields 7/8 recorded as RESERVED and unconsumed (F12), the redacting + byte-
capped shared FlowTraceCollector (F1), route-time site scope (F2), the derived
client waits (F4), the 7-10 row estimate, the doubled central audit write per
routed call, and the inbound-row-after-the-run ordering.
- sitestream.proto: fields 7/8 marked reserved-but-populated, numbers not to be
reused (comment only; no regeneration needed).
- Component-InboundAPI: SkipBodyCapture on the RouteCall row (F11),
SchemaRefResolver's public visibility, WithSiteScope / SITE_SCOPE_DENIED (F2),
Replay's wider role set (F3).
- Component-CentralUI: page policy vs Deployer-only controls (F3), route-time
site scope (F2), client wait derivation (F4).
- "7-10 audit rows per call (3 of them trace-only)" everywhere the estimate is
quoted: the method form, both CLI --trace-all-calls help strings, the CLI
README, the ApiMethod entity doc, the manual migration script and the two
Flow Debugger plans.
- arch-review doc: new §7 Resolution listing F1-F13 with the fixing commit, plus
the tracking-store test race fixed alongside them (86451382).
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Full review of the Flow Debugger commit range (v1.0, v1.1 outbound calls, v1.2 follow-ups,
outcome-merge fix, two perf commits). Also records the exact 11 role-negative Playwright
tests that cannot pass on the login-disabled rig in the cluster-ops skill.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Measuring the tracking-store queue on the rig moved nothing: the DB-done -> Notify.Send
gap stayed at 108 ms. The cost was the audit half of CachedCallTelemetryForwarder:
SqliteAuditWriter.WriteAsync enqueues synchronously but returns the BATCH completion,
and the writer loop coalesces for FlushIntervalMs (50 ms) before it commits, so each
awaited audit row cost ~50 ms + fsync and a cached call awaited three (submit,
attempted, resolve) on the script thread. Every other script-path emitter already
fired-and-observed; only this forwarder awaited.
TryEmitAuditAsync now hands the row to the writer and observes the completion on a
fault-logging continuation (the FallbackAuditWriter ring-buffer fallback still runs
inside that task). Best-effort contract unchanged; per-store FIFO unchanged; nothing
joins the audit and tracking stores by commit order.
Rig, FlowOutboundDemo, 5-run medians: inbound 194 -> 25 ms, site script run 176 -> 17 ms,
CachedWrite submit->done 61 -> 11 ms, DB done -> Notify.Send 108 -> 0.1 ms. The tracking
queue stays (three fewer fsyncs per call, batched). Docs corrected to attribute the cost
to the flush wait, with the trap recorded in CLAUDE.md and Component-SiteRuntime.md.
Tests: ForwardAsync returns while the audit commit is still pending (never-completing
writer task) and still records tracking; a late commit fault is swallowed.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
A Flow Debugger live run (execution 1130c01f) spent 108 of a 176 ms site script run
between the cached DB write completing and Notify.Send starting: the immediate-
completion telemetry awaited two autocommit SQLite transactions on the consolidated
LocalDb file (RecordAttemptAsync + RecordTerminalAsync), on the script thread, each a
WAL fsync on a Docker volume; the submit row was a third. The audit half of the same
forwarder was already a channel write, so the tracking half was the whole cost.
OperationTrackingStore is now a single-writer queue: Record*Async enqueue an ordered
work item and return; one background pump drains the bounded channel (8192, wait
backpressure - tracking never drops) and applies each batch (<=256) in ONE transaction,
falling back to per-row apply when a batch fails so a poison row cannot take its
neighbours. Preserved invariants: FIFO across enqueue/attempt/terminal; read-your-
writes (GetStatusAsync and ReadChangedSinceAsync barrier on every write queued before
them, skipped when idle) so Tracking.Status, the telemetry actor's status join and
central's PullSiteCalls never see a stale row; timestamps stamped when recorded;
PurgeTerminalAsync returns after the purge lands; DisposeAsync drains the queue before
closing; sync Dispose keeps its documented in-flight-writes-fail semantics. New public
FlushAsync() is the explicit barrier. Accepted: a queued-but-unapplied row is lost on a
hard crash (ms window); S&F buffer and audit rows are unaffected.
No interface, forwarder or ScriptRuntimeContext change. Tests: write returns while
another connection holds the DB write lock and lands once released (fails on the old
store: 31 s); reads see 100 ordered attempts + terminal; DisposeAsync drains 200
queued rows; write after dispose throws ObjectDisposed. Existing tests that poke the
SQLite file right after a store call now FlushAsync first.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Found in a browser Live run on the rig: the run drew API, DB and Notify nodes from
its inline rows; the 5 s pending-refresh re-read the tree through Replay before the
site's cached-call telemetry (12 s) had reached central, swapped the graph wholesale,
and the resolved DB node vanished (7 -> 6 steps) until a manual refresh. Polling had
already stopped because nothing was Pending.
MergeRefreshedGraph now unions the re-read rows into the run's known rows by EventId
(fresh copy wins), stores the union back on _raw and rebuilds through
FlowGraphBuilder, so a refresh only ever adds nodes. The union is exact: the inline
rows are the same AuditEvent objects the site writer persists (FlowTraceCollector).
Tests: new PendingRefresh_KeepsStepsTheLiveRunAlreadyHad_WhenReplayLagsBehindTelemetry
(fails on the old code). Two existing fixtures were internally inconsistent and only
passed under the old replace semantics: ParkedThenResolved built its Resolved outcome
with the Parked raw rows, and OutboundOutcome minted fresh event/correlation ids per
call, so no row could ever match across live and replay. Both now share rows per run.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
An ephemeral inbound method (created and deleted through the CLI, with a never-taken
Route.To branch so prediction resolves a route target) is run live in a real browser through
the fixture's /auth/login; the reload test drives the OnAfterRenderAsync localStorage restore
and then proves the page is still interactive. Both fail on the pre-fix image at the
'Predicted' wait (the circuit deadlock) and pass on the fixed one.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
The page wires FlowStepDetail's OnRetry/OnDiscard to IFlowDebugService.RetryOutboundAsync /
DiscardOutboundAsync behind the same site-scope re-check and confirm wording the Site Calls /
Notification report pages use, toasts the relay outcome, and on success re-polls Replay (same
5 s / 2 min loop) until the acted node leaves Parked. Also drops a duplicated ?correlationId=
test block that a concurrent index-only commit left in QueryStringDrillInTests.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
FlowDebugService.PredictMethodAsync resolved route targets with a sync-over-async
GetAwaiter().GetResult() inside the predictor's synchronous resolver — on the circuit's
single-threaded renderer sync context that blocked the dispatcher forever the moment a
method whose script routes to a site script was chosen (every later click and JS-interop
reply queued behind it; the page looked inert with a healthy websocket and no error).
Misdiagnosed on 2026-09-06 as a login-disabled localStorage stall. Now a resolve-and-rewalk
loop: each pass records the targets it could not answer, they are awaited between passes,
and the walk converges once a pass asks for nothing new (bounded by MaxPredictionPasses).
Regression test drives the service under a single-thread pump sync context (fails in 10 s
on the old code, passes now) plus a nested Scripts.Call resolve-across-passes test.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Rig run 2026-09-06 (image built from this branch, containers recreated from
main's compose file): full drill A-H YES (active-central kill takeover 27 s,
rejoin as standby in 3 s), then with FLOW_ONLY=1:
H live+replay -> YES
live: YES root=8e628d0a-428b-48df-b937-f4c2ac631fc3 events=8 api=Delivered db=Delivered notify=Submitted
replay: YES root=8e628d0a-428b-48df-b937-f4c2ac631fc3 events=10 api=Delivered db=Delivered notify=Delivered
I outbound live+replay -> YES
Two rig facts the drill established: (1) with the DB reachable the site's
immediate-completion path resolves a CachedWrite INSIDE the run, so the DbCall
node is already Delivered inline (Submit/Attempted/Resolve share a tick — the
check now ranks statuses instead of taking list order); (2) cached-call
telemetry reaches central on its own ~20-30 s cadence, later than the 5 s audit
push, so replay needs the bounded retry. New FLOW_ONLY=1 switch runs H + I
without the notifier build or the docker kill.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
After a successful Live run whose graph has Pending outbound steps the page
re-reads the tree through the existing Replay command every 5 s for up to
2 min (page constants), merging by replacing the graph and keeping the
selection by event id; never in Replay mode; cancelled by a new run, a mode
switch or Dispose. Status strip gains a Pending count, a 'Refresh outcomes'
button (both modes) and an 'outcomes still arriving' note while polling. The
run's own timing and the Completed chip are left alone — pending outbound
work is not a failed run. Plan 2026-09-06 D5.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
A folded outbound step shows a Lifecycle tab (time · kind · status · node ·
error, oldest first; a Parked step adds the 'retry or discard from Site Calls /
Notification report' hint), replaces the never-populated Writes tab for those
steps, labels the Input tab Request / SQL + parameters / Subject + body by kind,
and 'Where this ran' names the System / Connection / List plus 'Delivered by'
(the terminal row's node). Pending/Parked outcome badges.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Four lanes (LaneCount 4; External sublabel = distinct targets, three then +n);
ApiCall/DbCall/Notify nodes carry a status pill (Ok/Pending/Parked/Failed —
pending pulses, reduced-motion aware) plus a sync/cached/notify mode; folded
lifecycles show '+4.1 s' submit→terminal wall time. Async edges are dashed
and routed through the inter-lane strip with a gutter tag; spawn edges use a
bolt marker. Predicted async edges (Task 6's PredictedEdge.Async) render the
same way. de-DE height pinned at 319.2 = (4×110+16)×0.7.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
FlowShapePredictor emits ExternalSystem.Call/CachedCall → ApiCall ('{s}.{m}'),
Database.Execute/Query/QuerySingle/CachedWrite → DbCall ('{conn} · {member}'),
Notify.Send(list) and Notify.To(list).Send() → Notify ('Notify {list}'), all on
FlowLane.External; Async-suffixed spellings map identically. PredictedEdge gains
an additive Async flag; an async edge (cached call, notify) does not advance the
chain, mirroring FlowGraphBuilder.IsAsync (D4/D8). Database.Connection() raw
handles no longer predict a node (command count is not statically knowable).
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
GetTraceEventsAsync widens its channel filter to ScriptTrace ∪ ApiOutbound ∪
DbOutbound ∪ Notification ∪ (ApiInbound & InboundRequest) (plan D6); SecuredWrite,
Cluster and InboundAuthFailure rows in the same tree stay excluded. The site
SqliteAuditWriter.ReadByExecutionIdsAsync already had no channel predicate — pinned
by a test. Interface doc updated to match.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Fold rule: outbound rows sharing (kind, CorrelationId) become one FlowStep with
a Lifecycle list — first row's identity/time/input, latest row's outcome/
subtitle/output/error, DurationMs = sync row's own or submit→terminal wall time
(null while pending). Chain skips async steps (Notify, or first row
CachedSubmit) and links them with Async edges from the preceding chain step.
Spawner edges from a non-RouteCall are Spawn: true. The Response links from the
last non-async step of the inbound execution (the RouteCall), not the globally
last cascaded step (D9). FlowGraphView places a Response at max column + 1 so
the D9 origin does not pull it left (pulled forward from Task 7 to keep the
layout test green).
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
ResolveKind maps ApiOutbound/DbOutbound/Notification to ApiCall/DbCall/Notify;
ResolveLane puts the three on FlowLane.External; outbound outcomes map
Delivered→Ok, Failed→Failed, Parked/Discarded→Parked, else Pending; subtitles
'{status} · sync|cached', '{op} · n rows' from a DbWrite Extra, '{status}' for
Notify. New FlowStepProjector.IsOutbound helper.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Gap verified 2026-09-06: the projector and predictor already know the three outbound
channels, but the live FlowTraceCollector is fed only by EmitTrace (writes/waits) and
the replay query filters to ScriptTrace + the inbound row, so a run leaves every
predicted outbound node as a ghost. Central method-script Database.* calls are not
audited at all.
Plan (11 tasks): three step kinds ApiCall/DbCall/Notify on a fourth External lane;
one node per operation with the cached/notify lifecycle folded in (Pending/Parked
outcomes, Lifecycle tab); sync calls in the chain, async ones on dashed branch edges;
site rows ride the existing collector, replay filter widens, central DB helper gets
DbOutbound rows; the page refreshes pending outcomes via the existing Replay command;
spawn edges styled and Response linked from the route call (the "two parallel paths"
question). Mockup: docs/plans/…-mockup.html, published as a Claude artifact.
Claude-Session: https://claude.ai/code/session_01GSLkkHKz9JrAtmJjmQK3cP
The swimlane's node text was clipped by character count (20/24), so dotted attribute
paths showed only their shared head ("DelmiaReceiver.Reci…" five times in a row) and
the subtitle spilled past the box. It was also unstyled: the SVG <text> elements are
emitted as raw markup (Razor reserves the <text> tag), so they never carried the CSS
isolation scope attribute and every .fg-* rule silently missed.
- Node body is now HTML inside an SVG foreignObject: the browser measures the text,
the title wraps to two lines and ellipsizes by width, the subtitle by width on one
line, nothing can overflow the box, and the div's title attribute carries the full
unstripped text as a tooltip. Node height 78 -> 84.
- A lane whose real titles share a dotted head (CommonDottedPrefix) shows it once in
the lane gutter ("DelmiaReceiver.*") and each node shows only the tail that tells
it apart; the write subtitle strips the same head per item. Predicted-only nodes
are never stripped.
- WrapHints inserts zero-width spaces at camel-case and ./_/ boundaries so a wrapped
title breaks at "DelmiaRecipe|Download" rather than mid-word.
- Gutter widened 130 -> 160 with the sublabel clip raised to 22 so "site-a · DelmiaSim01"
fits; the remaining SvgText gutter labels are styled through ::deep.
Verified live on the rig (replay of the Task 22 run): all three lanes readable, no
overflow, gutter shows the factored prefix. CentralUI suite 1199 pass.
Claude-Session: https://claude.ai/code/session_01GSLkkHKz9JrAtmJjmQK3cP
Rig recreated from the branch-built image through the main checkout's compose file
(deploy.sh refuses a worktree, register row 68). Fresh key delmia-flowdbg-0906 scoped to
DelmiaRecipeDownload. Full drill: A/B/E/F YES, C/G NO, takeover 29 s, rejoin 3 s, then
H live+replay -> YES
live: 11 events (InboundRequest, RouteCall, ScriptRun, 7x AttributeWrite, AttributeWait)
replay: 17 events on the first attempt — the six extra rows are the simulator's two
cascaded ValueChange runs (2x ScriptRun + 4x AttributeWrite): inline TraceEvents
carry the routed run's own steps, replay walks the whole execution tree.
Recorded as row H in Delmia-Recipe-Download-API.md §8; ledger marks Task 22 completed
(all 24 tasks done). Note: the rig still runs main's appsettings, so the ScriptTrace
30-day retention override lands only after merge + a deploy from main.
Claude-Session: https://claude.ai/code/session_01GSLkkHKz9JrAtmJjmQK3cP
Component-CentralUI (new Flow Debugger page section), Component-InboundAPI (TraceAllCalls
property, always-on RouteCall row + Trace flag under Routing Behavior, new 'Flow Debugger
runs' section), Component-SiteRuntime (Instance Actor trace arm; Script Actor traced runs;
CallScript flag inheritance), Component-AuditLog (ScriptTrace channel in Scope, the
PullTraceEvents point-read note + mixed-version caveat under Reconciliation, per-channel
retention example), Component-Communication (§4 trace flag + inline events with proto field
numbers; §8 PullTraceEvents; RPC list), Component-ScriptAnalysis (REQ-SA-7 FlowShapePredictor),
CLAUDE.md UI bullet, deferred-work register row 110 (vertical-trace list toggle + console
capture), design doc status → implemented. CLI README sections landed in Tasks 3/16.
Umbrella index (../scadaproj/CLAUDE.md) checked: its ScadaBridge row records stack/wire
relationships only, unchanged by a new page — no edit.
Claude-Session: https://claude.ai/code/session_01GSLkkHKz9JrAtmJjmQK3cP
Flow Debugger (design 2026-09-05 §7.1): FlowDebugRunMethodCommand / FlowDebugRunValueChangeCommand /
FlowDebugReplayCommand on the per-node ManagementActor (Deployer role; the two run commands
are long-running), all delegating to FlowDebugRunner. RunMethod refuses on a standby with
STANDBY_NODE (ManagementCommandException now carries an optional Code that replaces
COMMAND_FAILED), validates parameters through the real ParameterValidator/SchemaRefResolver
(made public), synthesizes the inbound row (actor = operator, Extra.source = flow-debugger)
and runs the method through InboundScriptExecutor with a trace-sink RouteHelper so the reply
carries the RouteCall row + the site's inline events. RunValueChange does a traced routed
write via IInstanceRouter, then polls IPullTraceEventsClient (interface moved to Commons so
ManagementService can consume it) until a ScriptRun parented to the root settles, else
NO_TRIGGER_WITHIN_WINDOW. Replay reads IAuditLogRepository.GetTraceEventsAsync.
Claude-Session: https://claude.ai/code/session_01GSLkkHKz9JrAtmJjmQK3cP
Flow Debugger (design 2026-09-05 §5.2): bit NOT NULL DEFAULT 0 column + EF migration
20260906034113 with its docs/plans/sql production twin. CreateApiMethodCommand gains
TraceAllCalls=false; UpdateApiMethodCommand gains bool? TraceAllCalls (null = unchanged so an
older CLI never resets it). Management result echoes the flag; CLI create/update expose
--trace-all-calls; README carries the ~7-rows-per-call warning.
Claude-Session: https://claude.ai/code/session_01GSLkkHKz9JrAtmJjmQK3cP
Live + Replay modes (no dry run); entry points = inbound API method and
value-change trigger; data-sourced value-change does a real tag write with a
30 s trace arm on the instance actor for echo correlation; gated trace rows in
a new ScriptTrace audit channel (RouteCall always-on); inline step return on
RouteToCallResponse for Live; Deployer gates the page and Live run.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Joseph Doherty