# Flow Debugger Implementation Plan > **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task. **Goal:** Ship the Flow Debugger — a Central UI page (plus CLI parity) that runs an inbound API method or a value-change trigger with operator inputs, or replays a past ExecutionId, and renders every step as a swimlane graph with per-step input/output. **Architecture:** Trace steps ARE audit events in a new `AuditChannel.ScriptTrace` (kinds `RouteCall`, `ScriptRun`, `AttributeWrite`, `AttributeWait`), emitted only when a `Trace` flag rides beside `ParentExecutionId`; the routed reply carries the site's events back inline so a Live run is instant, and the same events land in the audit pipeline for Replay. A per-node `FlowDebugRun*Command` on the `ManagementActor` (Deployer role, refuses on a standby with the active node named) is the single execution path for the UI and the CLI. Central projects events into a `FlowGraph` (lanes Central/Site/Tags) rendered by a custom SVG component; a Roslyn syntax walker predicts the graph before a run. **Tech Stack:** .NET 10, Akka.NET (actors, TestKit), gRPC/protobuf (vendored C#, `docker/regen-proto.sh`), EF Core (SQL Server), SQLite (site audit), Blazor Server + bUnit, System.CommandLine (CLI), Roslyn (`CSharpSyntaxWalker`), xunit + NSubstitute. **Design:** `docs/plans/2026-09-05-flow-debugger-design.md` (approved 2026-09-05). Task 0 amends it with four corrections found while planning. **Conventions for every task:** build with `dotnet build ZB.MOM.WW.ScadaBridge.slnx -nologo -v q`; run a test project with `dotnet test tests//.csproj -nologo -v q [--filter FullyQualifiedName~]`. Commit messages end with `Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29`. All record changes are **additive trailing optional parameters** — every constructor call site in `src/` and `tests/` keeps compiling unchanged. --- ## Task 0: Amend the design doc with the four planning corrections **Classification:** trivial **Estimated implement time:** ~4 min **Parallelizable with:** none (do first) **Files:** - Modify: `docs/plans/2026-09-05-flow-debugger-design.md` **Step 1: Apply the corrections** (edit the named sections in place; keep everything else): 1. **§5.1 / §5.4 / §6 — trace steps are audit events, not a new record.** Replace the `FlowStep` wire type with: "A trace step IS an `AuditEvent` in channel `ScriptTrace`. `RouteToCallResponse.TraceEvents` (`IReadOnlyList?`) carries the site's events inline; on the wire it is `repeated AuditEventDto trace_events` (the DTO `sitestream.proto` already defines). `FlowStep` is a **central-side projection** (`Commons/Types/Debugging/FlowStepProjector`) computed from `AuditEvent` + `AuditDetails`." Drop the `FlowStepDto` proto message and the `TriggerFired` kind — a value-change run's `ScriptRun` row carries `trigger: {attribute, value, source}` in `Extra`. 2. **§5.1 — who emits `AttributeWrite`/`AttributeWait`.** Replace "Site `InstanceActor` write handlers" with "`ScriptRuntimeContext.SetAttribute` / `WriteBatchAndWait` / `WaitInternal` — every accessor path funnels into these three methods, the context already holds `IAuditWriter`, `_executionId` and `_parentExecutionId`, so the actor needs no new ids." Delete the `WriteAttributeBatchRequest`/`WaitForAttributeRequest` rows from the §6 table. Note: batch-write echoes are **not** armed in v1 (only `SetStaticAttributeCommand` writes arm — §5.3). 3. **§5.3 — both attribute kinds use one command.** `RouteToSetAttributes` → `SetStaticAttributeCommand` already dispatches data-sourced attributes to the DCL (`InstanceActor.HandleSetDataAttribute`); the trace arm is set there when `command.Trace && command.SourceExecutionId != null`, consumed in `HandleTagValueUpdate`. 4. **§7.1 — execution path.** The `ManagementActor` is a **per-node** actor (`AkkaHostedService` line ~490, "/user/management"), not a singleton. Replace the paragraph with: "Live runs and Replay are `FlowDebugRunMethodCommand` / `FlowDebugRunValueChangeCommand` / `FlowDebugReplayCommand` on the `ManagementActor` (Deployer role, long-running timeout). The run-method handler consults `IActiveNodeGate` and refuses on a standby with error code `STANDBY_NODE` naming the design rule; Traefik routes UI traffic to the active node so this is rare. The Central UI calls it through `ManagementActorHolder` (the `DataConnectionMoveService` pattern) and the CLI through `/management` — one path, one audit story. The inbound row for a Live run is written by the handler via `ICentralAuditWriter` (kind `InboundRequest`, actor = operator username, `Extra.source = "flow-debugger"`)." 5. **§10 — mixed-version caveat.** Add: "An OLD central ingesting `ScriptTrace` rows from a NEW site parses unknown channel/kind names to the `AuditRowProjection` fallbacks (`ApiInbound`/`InboundRequest`); upgrade central first." 6. **§11** add "Console output capture on `ScriptRun` rows (no script console exists today)". **Step 2: Commit** ```bash git add docs/plans/2026-09-05-flow-debugger-design.md git commit -m "docs(plans): flow debugger design — trace steps are audit events; mgmt-command execution path; context-emitted write/wait rows" ``` --- ## Task 1: Audit enums — `ScriptTrace` channel and four kinds **Classification:** small **Estimated implement time:** ~3 min **Parallelizable with:** Task 2, Task 3, Task 5 **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Types/Enums/AuditChannel.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Types/Enums/AuditKind.cs` - Test: `tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Audit/ScriptTraceEnumTests.cs` **Step 1: Write the failing test** ```csharp using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit; using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums; namespace ZB.MOM.WW.ScadaBridge.Commons.Tests.Audit; public class ScriptTraceEnumTests { [Theory] [InlineData(AuditKind.RouteCall)] [InlineData(AuditKind.ScriptRun)] [InlineData(AuditKind.AttributeWrite)] [InlineData(AuditKind.AttributeWait)] public void ScriptTraceKinds_FitTheCentralColumns(AuditKind kind) { // AuditLog.Category / Kind are nvarchar(32) (AuditLogEntityTypeConfiguration). Assert.True(kind.ToString().Length <= 32); Assert.True(AuditChannel.ScriptTrace.ToString().Length <= 32); Assert.Equal($"ScriptTrace.{kind}", AuditFieldBuilders.BuildAction(AuditChannel.ScriptTrace, kind)); } [Fact] public void ScriptTrace_IsAppendedAfterCluster_SoExistingOrdinalsAreStable() { Assert.True((int)AuditChannel.ScriptTrace > (int)AuditChannel.Cluster); } } ``` **Step 2: Run it to make sure it fails** — `dotnet test tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/ZB.MOM.WW.ScadaBridge.Commons.Tests.csproj --filter FullyQualifiedName~ScriptTraceEnumTests` → compile error `ScriptTrace` not found. **Step 3: Implement** — append to `AuditChannel`: ```csharp /// /// Flow Debugger trace rows (design 2026-09-05): a script run, an attribute write, an /// attribute wait or a central route call, emitted only for traced executions /// (plus the always-on ). Appended last so stored /// ordinals of existing channels never move. /// ScriptTrace ``` and to `AuditKind` (after `ManualFailover`, keeping the file's explicit-value style if it uses one — check the last member and continue the sequence): ```csharp /// Central `Route.To().Call` round-trip (always-on; ScriptTrace channel). RouteCall, /// One site script run completed/failed/timed out (traced only). ScriptRun, /// One attribute write or batch write made by a traced script (traced only). AttributeWrite, /// One `WaitAsync`/`WaitForAsync` outcome in a traced script (traced only). AttributeWait, ``` **Step 4: Run the test** → PASS. Also run `dotnet test tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/...csproj` fully (some tests enumerate `Enum.GetValues()`; fix any that pin the count). **Step 5: Commit** — `git commit -am "feat(audit): ScriptTrace channel + RouteCall/ScriptRun/AttributeWrite/AttributeWait kinds"` --- ## Task 2: Additive `Trace` / trace-payload fields on the message records **Classification:** high-risk (data contracts) **Estimated implement time:** ~5 min **Parallelizable with:** Task 1, Task 3, Task 5 **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/InboundApi/RouteToInstanceRequest.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/ScriptExecution/ScriptCallRequest.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/ScriptExecution/ScriptCallResult.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Instance/SetStaticAttributeCommand.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Streaming/AttributeValueChanged.cs` - Test: `tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Messages/TraceFlagDefaultsTests.cs` **Step 1: Failing test** ```csharp using ZB.MOM.WW.Audit; using ZB.MOM.WW.ScadaBridge.Commons.Messages.InboundApi; using ZB.MOM.WW.ScadaBridge.Commons.Messages.Instance; using ZB.MOM.WW.ScadaBridge.Commons.Messages.ScriptExecution; using ZB.MOM.WW.ScadaBridge.Commons.Messages.Streaming; namespace ZB.MOM.WW.ScadaBridge.Commons.Tests.Messages; public class TraceFlagDefaultsTests { [Fact] public void ExistingPositionalConstructions_DefaultToUntraced() { var now = DateTimeOffset.UtcNow; Assert.False(new RouteToCallRequest("c", "i", "s", null, now).Trace); Assert.False(new RouteToSetAttributesRequest("c", "i", new Dictionary(), now).Trace); Assert.False(new ScriptCallRequest("s", null, 0, "c").Trace); Assert.False(new SetStaticAttributeCommand("c", "i", "a", "v", now).Trace); Assert.False(new AttributeValueChanged("i", "p", "a", 1, "Good", now).Trace); var reply = new RouteToCallResponse("c", true, null, null, now); Assert.Null(reply.ChildExecutionId); Assert.Null(reply.TraceEvents); var result = new ScriptCallResult("c", true, null, null); Assert.Null(result.ExecutionId); Assert.Null(result.TraceEvents); } [Fact] public void TraceEvents_AreAuditEvents() { var evt = new AuditEvent { EventId = Guid.NewGuid(), OccurredAtUtc = DateTimeOffset.UtcNow, Actor = "a", Action = "ScriptTrace.ScriptRun", Outcome = AuditOutcome.Success }; var result = new ScriptCallResult("c", true, null, null, Guid.NewGuid(), [evt]); Assert.Single(result.TraceEvents!); } } ``` (Check `AuditOutcome`'s member names in `~/Desktop/scadaproj/ZB.MOM.WW.Audit/src/ZB.MOM.WW.Audit/AuditOutcome.cs` and use a real one.) **Step 2: Run → fails to compile.** **Step 3: Implement** — append trailing optional parameters (doc-comment each with one line: "Flow Debugger trace flag — see design 2026-09-05 §5.2"): ```csharp // RouteToInstanceRequest.cs public record RouteToCallRequest(..., Guid? ParentExecutionId = null, bool Trace = false); public record RouteToCallResponse(string CorrelationId, bool Success, object? ReturnValue, string? ErrorMessage, DateTimeOffset Timestamp, Guid? ChildExecutionId = null, IReadOnlyList? TraceEvents = null); public record RouteToSetAttributesRequest(..., Guid? ParentExecutionId = null, bool Trace = false); // ScriptCallRequest.cs public record ScriptCallRequest(..., Guid? ParentExecutionId = null, bool Trace = false); // ScriptCallResult.cs public record ScriptCallResult(string CorrelationId, bool Success, object? ReturnValue, string? ErrorMessage, Guid? ExecutionId = null, IReadOnlyList? TraceEvents = null); // SetStaticAttributeCommand.cs public record SetStaticAttributeCommand(..., Guid? SourceExecutionId = null, bool Trace = false); // AttributeValueChanged.cs public record AttributeValueChanged(..., Guid? SourceExecutionId = null, bool Trace = false) : ISiteStreamEvent; ``` Add `using ZB.MOM.WW.Audit;` where `AuditEvent` is used (Commons already references the `ZB.MOM.WW.Audit` package — `IAuditWriter.cs` uses it). **Step 4: Build the whole solution** (`dotnet build ZB.MOM.WW.ScadaBridge.slnx`) — must be green with zero call-site edits. Run Commons.Tests → PASS. **Step 5: Commit** — `git commit -am "feat(commons): additive Trace flag + ChildExecutionId/TraceEvents on route/call/write/change records"` --- ## Task 3: `ApiMethod.TraceAllCalls` — entity, EF migration, management command, CLI flag **Classification:** high-risk (migration) **Estimated implement time:** ~5 min **Parallelizable with:** Task 1, Task 2, Task 5 **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Entities/InboundApi/ApiMethod.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Configurations/InboundApiConfiguration.cs` - Create: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Migrations/_AddApiMethodTraceAllCalls.cs` (+ Designer, snapshot update — generated) - Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Management/InboundApiCommands.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementActor.cs` (`HandleCreateApiMethod`, `HandleUpdateApiMethod`) - Modify: `src/ZB.MOM.WW.ScadaBridge.CLI/Commands/ApiMethodCommands.cs`, `src/ZB.MOM.WW.ScadaBridge.CLI/README.md` (§ `api-method create|update`) - Test: `tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests/ApiMethodTraceAllCallsTests.cs` **Step 1: Failing test** (follow the existing ManagementService test style for invoking `HandleUpdateApiMethod` — find a sibling test that exercises `UpdateApiMethodCommand` and copy its fixture; assert `method.TraceAllCalls` round-trips and defaults to `false` for the 5-arg constructor). **Step 2: Implement** - `ApiMethod`: `public bool TraceAllCalls { get; set; }` with summary "When true every production call to this method is fully traced (Flow Debugger design §5.2); intended for low-volume integration methods — 7-10 audit rows per call." - EF: `builder.Property(m => m.TraceAllCalls).IsRequired().HasDefaultValue(false);` - Migration: build first (CLAUDE.md gotcha), then `dotnet ef migrations add AddApiMethodTraceAllCalls --project src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase --startup-project src/ZB.MOM.WW.ScadaBridge.Host` (use the same `--startup-project` the previous migration used — check `git log -1 --stat -- src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Migrations/20260819201210_AddNotificationsListSortIndex.cs` for its commit message). Verify the generated `Up` contains exactly one `AddColumn("TraceAllCalls", "ApiMethods", nullable: false, defaultValue: false)`; delete and redo if empty. - Commands: `CreateApiMethodCommand(..., string? ReturnDefinition, bool TraceAllCalls = false)`, `UpdateApiMethodCommand(..., string? ReturnDefinition, bool? TraceAllCalls = null)` (null = leave unchanged, so old CLIs don't reset it). Handlers copy the value. - CLI: `--trace-all-calls` (`Option` on create; `Option` on update) → passed through. README: add the flag to both usage lines with the one-line warning. **Step 3: Run ManagementService.Tests + CLI.Tests → PASS. Step 4: Commit** — `git commit -am "feat(inbound-api): ApiMethod.TraceAllCalls (entity, migration, mgmt command, CLI --trace-all-calls)"` --- ## Task 4: `FlowStep` projection + `FlowGraph` model + `FlowGraphBuilder` (pure) **Classification:** standard **Estimated implement time:** ~5 min **Parallelizable with:** Task 5, Task 6, Task 8 **Files:** - Create: `src/ZB.MOM.WW.ScadaBridge.Commons/Types/Debugging/FlowStep.cs` - Create: `src/ZB.MOM.WW.ScadaBridge.Commons/Types/Debugging/FlowGraph.cs` - Create: `src/ZB.MOM.WW.ScadaBridge.Commons/Types/Debugging/FlowStepProjector.cs` - Create: `src/ZB.MOM.WW.ScadaBridge.Commons/Types/Debugging/FlowGraphBuilder.cs` - Create: `src/ZB.MOM.WW.ScadaBridge.Commons/Types/Debugging/ScriptTraceExtra.cs` - Test: `tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Debugging/FlowGraphBuilderTests.cs` **Step 1: Types** ```csharp namespace ZB.MOM.WW.ScadaBridge.Commons.Types.Debugging; public enum FlowLane { Central, Site, Tags } public enum FlowStepKind { InboundRequest, RouteCall, ScriptRun, AttributeWrite, AttributeWait, Response, External, Unknown } public enum FlowStepOutcome { Ok, Failed, TimedOut, Predicted, NotTraced } /// One node of the flow graph — a projection of a ScriptTrace/ApiInbound AuditEvent. public sealed record FlowStep( Guid EventId, Guid? ExecutionId, Guid? ParentExecutionId, FlowStepKind Kind, FlowLane Lane, string Title, string? Subtitle, DateTimeOffset OccurredAtUtc, int? DurationMs, FlowStepOutcome Outcome, string? InputJson, string? OutputJson, string? ErrorMessage, string? SiteId, string? Node, string? Instance, string? Target, ScriptTraceExtra? Extra); public sealed record FlowEdge(Guid FromEventId, Guid ToEventId, bool CrossesBoundary, bool External); public sealed record FlowGraph(IReadOnlyList Steps, IReadOnlyList Edges, bool Predicted, bool Truncated); /// Kind-specific detail carried in AuditDetails.Extra as JSON (camelCase). public sealed record ScriptTraceExtra { public string? Node { get; init; } // node-a / central-a public string? TriggerKind { get; init; } // call | valuechange | ... public string? TriggerAttribute { get; init; } public string? TriggerValue { get; init; } public string? TriggerSource { get; init; } // static-write | device-echo public Guid? ChildExecutionId { get; init; } // RouteCall public IReadOnlyList? Writes { get; init; } public string? WaitAttribute { get; init; } public string? WaitTarget { get; init; } public int? WaitTimeoutMs { get; init; } public string? WaitOutcome { get; init; } // satisfied | timeout | cancelled | error public string? ObservedValue { get; init; } public string? Source { get; init; } // "flow-debugger" on the synthesized inbound row } public sealed record AttributeWriteDetail(string Attribute, string? Written, bool Success, string? Error, bool IsStatic); ``` `FlowStepProjector.Project(AuditEvent evt)`: deserialize `AuditDetails` via `AuditDetailsCodec.Deserialize`, parse `Channel`/`Kind` (`ScriptTrace.*` and `ApiInbound.InboundRequest`), lane = `RouteCall|InboundRequest → Central`, `ScriptRun|AttributeWait → Site`, `AttributeWrite → Tags`; `Outcome` from `Status` (`Delivered→Ok`, `Failed→Failed`; a `ScriptRun` whose ErrorMessage contains "timed out" → `TimedOut`); Extra via `JsonSerializer.Deserialize(details.Extra, camelCase)`; Title = `Target` (script name, `{instance}.{script}`, attribute); Input/Output = `RequestSummary`/`ResponseSummary`. `FlowGraphBuilder.Build(IReadOnlyList events, bool truncated = false)`: project all; order by `OccurredAtUtc, EventId`; within one `ExecutionId` chain consecutive steps in time order; link a `RouteCall`'s node to the first step of the execution whose `ParentExecutionId` matches it (`CrossesBoundary = true`); link the last site step back to a synthesized `Response` step when an `InboundRequest` row has a `ResponseSummary` (append it as the final Central node); an `AttributeWait` that was `satisfied` gets an `External` edge from the preceding `AttributeWrite` in the same execution. Return `FlowGraph(Predicted:false)`. **Step 2: Failing test** — build a canned Delmia-shaped list of 7 `AuditEvent`s via `ScadaBridgeAuditEventFactory.Create(...)` (inbound, route call, script run, attribute write, attribute wait, and a second attribute write) with matching `ExecutionId`/`ParentExecutionId` and assert: 8 steps (incl. synthesized Response), lanes as expected, exactly two `CrossesBoundary` edges (route→site, site→response), one `External` edge, order by time. Add a `Truncated` pass-through test and an "unknown kind → `FlowStepKind.Unknown`, never throws" test. **Step 3: Run → fail → implement → pass. Step 4: Commit** — `git commit -am "feat(commons): FlowStep/FlowGraph projection + FlowGraphBuilder for the Flow Debugger"` --- ## Task 5: `FlowTraceCollector` (site, per run) **Classification:** small **Estimated implement time:** ~3 min **Parallelizable with:** Task 1, Task 2, Task 3, Task 4 **Files:** - Create: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Scripts/FlowTraceCollector.cs` - Test: `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Scripts/FlowTraceCollectorTests.cs` **Step 1: Failing test** — `Add` 501 events with `MaxEvents = 500` → `Snapshot()` has 500 and `Truncated == true`; concurrent adds from 8 tasks never throw and count ≤ cap; an untraced collector is `null`, not an object (callers null-check). **Step 2: Implement** ```csharp namespace ZB.MOM.WW.ScadaBridge.SiteRuntime.Scripts; /// Per-run sink for ScriptTrace audit events so a traced routed call can return them /// inline (design 2026-09-05 §5.4). Lock-based: a run emits tens of events, not thousands. public sealed class FlowTraceCollector { public const int DefaultMaxEvents = 500; private readonly List _events = new(); private readonly object _gate = new(); private readonly int _max; public bool Truncated { get; private set; } public FlowTraceCollector(int maxEvents = DefaultMaxEvents) => _max = maxEvents; public void Add(AuditEvent evt) { lock (_gate) { if (_events.Count >= _max) { Truncated = true; return; } _events.Add(evt); } } public IReadOnlyList Snapshot() { lock (_gate) return _events.ToArray(); } } ``` **Step 3: Pass. Commit** — `git commit -am "feat(site-runtime): FlowTraceCollector"` --- ## Task 6: `ScriptRuntimeContext` — traced flag, trace emission for writes and waits, flag inheritance **Classification:** high-risk (hot path) **Estimated implement time:** ~5 min **Parallelizable with:** Task 4, Task 8 **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Scripts/ScriptRuntimeContext.cs` (ctor ~line 205; `SetAttribute` ~513; `WriteBatchAndWait` ~550; `WaitInternal` ~471; `CallScript` ~596; the nested-context construction near line 630 that passes `_executionId, _auditWriter…`) - Test: `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Scripts/ScriptRuntimeContextTraceEmissionTests.cs` **Step 1: Failing tests** (TestKit; `IAuditWriter` = `Substitute.For()`; instance actor = `CreateTestProbe()` answering `SetStaticAttributeResponse(success)` / `WaitForAttributeResponse(Matched:true)`): - `SetAttribute_WhenTraced_EmitsAttributeWriteRow_AndPassesTraceOnCommand` — captured `SetStaticAttributeCommand.Trace == true`; writer received one event whose details `Channel == "ScriptTrace"`, `Kind == "AttributeWrite"`, `ExecutionId == ctx.ExecutionId`; collector snapshot has it. - `SetAttribute_WhenNotTraced_EmitsNothing_AndCommandTraceFalse` (the production-unchanged pin). - `WaitInternal_WhenTraced_EmitsAttributeWait_WithOutcomeSatisfied`. - `CallScript_WhenTraced_PropagatesTraceOnScriptCallRequest`. **Step 2: Implement** - Ctor: append `bool isTraced = false, FlowTraceCollector? traceCollector = null`; fields `_isTraced`, `_traceCollector`; `public bool IsTraced => _isTraced;` `public Guid ExecutionId => _executionId;` (if not already public — check; add only what's missing). - Private helper: ```csharp private void EmitTrace(AuditKind kind, AuditStatus status, string target, DateTime startedUtc, int durationMs, string? input, string? output, string? error, ScriptTraceExtra? extra) { if (!_isTraced) return; AuditEvent evt; try { evt = ScadaBridgeAuditEventFactory.Create( AuditChannel.ScriptTrace, kind, status, occurredAtUtc: startedUtc, actor: _sourceScript, target: target, sourceNode: _sourceNode, executionId: _executionId, parentExecutionId: _parentExecutionId, sourceSiteId: string.IsNullOrEmpty(_siteId) ? null : _siteId, sourceInstanceId: _instanceName, sourceScript: _sourceScript, durationMs: durationMs, errorMessage: error, requestSummary: input, responseSummary: output, extra: extra is null ? null : JsonSerializer.Serialize(extra, TraceJson)); } catch (Exception ex) { _logger.LogWarning(ex, "Flow trace event build failed for {Kind}", kind); return; } _traceCollector?.Add(evt); if (_auditWriter is null) return; var t = _auditWriter.WriteAsync(evt, CancellationToken.None); if (!t.IsCompleted) _ = t.ContinueWith(x => _logger.LogWarning(x.Exception, "Flow trace write failed"), TaskContinuationOptions.OnlyOnFaulted); } private static readonly JsonSerializerOptions TraceJson = new(JsonSerializerDefaults.Web); ``` - `SetAttribute`: `Stopwatch` around the Ask; pass `Trace: _isTraced` on the command; after the reply `EmitTrace(AuditKind.AttributeWrite, response.Success ? Delivered : Failed, attributeName, started, ms, input: JsonSerializer.Serialize(new { attributeName, value }), output: null, error: response.ErrorMessage, extra: new ScriptTraceExtra { Writes = [new(attributeName, value, response.Success, response.ErrorMessage, IsStatic: true)] })`. (`IsStatic` is unknown to the context — set `false` and name the field `IsStaticHint`… simpler: drop `IsStatic` from `AttributeWriteDetail` in Task 4 if you prefer; either way keep the tests consistent.) - `WriteBatchAndWait`: emit one `AttributeWrite` for the batch (all `encodedValues` + the trigger) after the batch Ask, then the wait emits through `WaitInternal`. - `WaitInternal`: emit `AttributeWait` with `WaitOutcome = response.Matched ? "satisfied" : response.TimedOut ? "timeout" : "error"`, `ObservedValue = AttributeValueCodec.Encode(response.Value)`, `WaitTimeoutMs`, status `Matched → Delivered`, else `Failed`. - `CallScript`: `Trace: _isTraced` on `ScriptCallRequest`. The inline-shared nested context construction (~line 630): pass `isTraced: _isTraced, traceCollector: _traceCollector`. - Payload caps: reuse the existing 8 KB truncation helper the context uses for `RequestSummary` (grep `Truncate`/`MaxPayload` in the file) so trace rows obey §9. **Step 3: Run the new tests + the whole SiteRuntime.Tests project (existing tests must stay green). Commit** — `git commit -am "feat(site-runtime): ScriptRuntimeContext emits ScriptTrace AttributeWrite/AttributeWait rows and inherits the Trace flag"` --- ## Task 7: `ScriptRunLauncher` + `ScriptActor` — traced runs emit `ScriptRun`, return `ExecutionId` + events, value-change inheritance **Classification:** high-risk (actor model) **Estimated implement time:** ~5 min **Parallelizable with:** Task 8 **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Scripts/ScriptRunLauncher.cs` (`LaunchScript` signature ~76, `RunScriptAsync` ~228–470) - Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/ScriptActor.cs` (`HandleScriptCallRequest`, `HandleAttributeValueChanged` ~283, `TrySpawnExecution`, `SpawnExecution` ~520) - Test: `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Actors/ScriptActorTraceTests.cs` **Step 1: Failing tests** (model on `AlarmCascadeParentExecutionTests` for fixture; `ScriptActor` construction — copy from an existing `ScriptActor` test in the project): - `Call_WithTrace_ReplyCarriesExecutionId_AndAScriptRunEvent` — `Tell(new ScriptCallRequest("s", null, 0, "cid", Trace: true))`; `ExpectMsg`; `ExecutionId != null`; `TraceEvents` contains one event with `Kind == "ScriptRun"`, `Status == "Delivered"`, `RequestSummary` = params JSON, `ResponseSummary` = return JSON. - `Call_WithoutTrace_ReplyHasNoTraceEvents` (and `ExecutionId` still set — cheap and useful for correlation). - `ValueChangeTrigger_TracedChange_RunParentsToSourceExecutionId` — script with `valuechange` trigger on `Status`; `Tell(new AttributeValueChanged("Pump1","Status","Status","x","Good",now, SourceExecutionId: g, Trace: true))`; the launched run's `ScriptRun` event (capture via `IAuditWriter` substitute registered in the `IServiceProvider` passed to the actor) has `ParentExecutionId == g` and `Extra.triggerSource == "static-write"`? — the actor cannot know static vs echo; set `TriggerSource` to `"value-change"` and `TriggerAttribute`/`TriggerValue`. - `ValueChangeTrigger_UntracedChange_RunStaysRoot` (pin). **Step 2: Implement** - `LaunchScript(..., ScriptExecutionScheduler? scheduler, bool trace = false, ScriptTraceExtra? triggerExtra = null)` → threaded into `RunScriptAsync`. - In `RunScriptAsync`: `var collector = trace ? new FlowTraceCollector() : null;` pass `isTraced: trace, traceCollector: collector` to the context. After success: emit `ScriptRun` (Delivered) via the same factory call shape as Task 6 but from the launcher (it has `auditWriter`, `siteId`, `sourceNode`, `context.ExecutionId`, `parentExecutionId`; write it through `collector` first so it is included, then `auditWriter`); reply `new ScriptCallResult(correlationId, true, state.ReturnValue, null, context.ExecutionId, collector?.Snapshot())`. Timeout/failure catches: same with `Failed` + error message. Put the emission in a small private static `EmitScriptRun(...)` to avoid triplication. Duration = `runStopwatch.ElapsedMilliseconds`; Input = params JSON (8 KB cap); Output = return value JSON. - `ScriptActor.HandleScriptCallRequest` → `SpawnExecution(..., request.ParentExecutionId, request.Trace)`; `SpawnExecution` gains `bool trace = false, ScriptTraceExtra? triggerExtra = null` → passes to `LaunchScript`. - `HandleAttributeValueChanged`: for the `ValueChangeTriggerConfig` branch call `TrySpawnExecution(null, changed.Trace ? changed.SourceExecutionId : null, changed.Trace, changed.Trace ? new ScriptTraceExtra { TriggerKind = "valuechange", TriggerAttribute = changed.AttributeName, TriggerValue = AttributeValueCodec.Encode(changed.Value), TriggerSource = "value-change" } : null)`. `TrySpawnExecution(parameters, Guid? parentExecutionId = null, bool trace = false, ScriptTraceExtra? triggerExtra = null)` → `SpawnExecution(parameters, 0, ActorRefs.NoSender!, Guid.NewGuid().ToString(), parentExecutionId, trace, triggerExtra)`. Other trigger kinds keep calling `TrySpawnExecution(null)` unchanged (roots by design — CLAUDE.md). **Step 3: Run SiteRuntime.Tests fully. Commit** — `git commit -am "feat(site-runtime): traced runs emit ScriptRun rows, return ExecutionId + trace events; traced value-change runs inherit the parent"` --- ## Task 8: `InstanceActor` — trace arm for data-sourced writes; `Trace` on published changes **Classification:** high-risk (actor model) **Estimated implement time:** ~5 min **Parallelizable with:** Task 4, Task 6, Task 7 **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/InstanceActor.cs` (`HandleSetStaticAttributeCore` ~538, `HandleSetDataAttribute` ~693, `HandleTagValueUpdate` ~1025, fields ~39–193) - Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/SiteRuntimeOptions.cs`, `SiteRuntimeOptionsValidator.cs` - Test: `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Actors/InstanceActorTraceArmTests.cs`, extend `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/SiteRuntimeOptionsValidatorTests.cs` **Step 1: Failing tests** (fixture = `InstanceActorWaitForAttributeTests.CreateInstanceActor` pattern with a DCL probe; config with one data-sourced attribute `Processed` bound to tag `ns=3;s=Recipe.Processed` on connection `PLC`, plus a child probe registered as an all-change listener — or simply subscribe the test probe through the existing `SubscribeDebugView`/stream path; simplest: pass a `SiteStreamManager` substitute and capture `PublishAttributeValueChanged`): - `TracedDataWrite_ThenEcho_PublishesChangeStampedWithSourceAndTrace` — `Tell(SetStaticAttributeCommand(..., SourceExecutionId: g, Trace: true))`; DCL probe `ExpectMsg` and reply `WriteTagResponse(success)`; then `Tell(new TagValueUpdate("PLC", tag, true, QualityCode.Good, now))`; captured `AttributeValueChanged.SourceExecutionId == g && Trace`. - `UntracedDataWrite_ThenEcho_PublishesUnstampedChange` (pin: production unchanged). - `Arm_IsConsumedOnce` — second echo is unstamped. - `Arm_Expired_IsIgnored` — construct with `TraceArmWindowSeconds = 0`… use an injected clock? Keep simple: options `TraceArmWindowSeconds` and a `Func`-free approach: test with window `1` and `Thread.Sleep(1100)` before the echo (acceptable in TestKit tests; mark `[Trait("Category","Slow")]`). - `StaticWrite_Traced_PublishesTraceFlag` — `HandleSetStaticAttributeCore` path. - Validator: `TraceArmWindowSeconds` must be `> 0`. **Step 2: Implement** - `SiteRuntimeOptions.TraceArmWindowSeconds { get; set; } = 30;` (doc: design §5.3). Validator rule. - Fields: `private readonly Dictionary _traceArms = new();` - `HandleSetDataAttribute`: before the DCL Ask, `if (command.Trace && command.SourceExecutionId is { } src) _traceArms[attributeName] = (src, DateTimeOffset.UtcNow.AddSeconds(_options.TraceArmWindowSeconds));` - `HandleTagValueUpdate`: for each `attrName`, resolve `Guid? src = null; var traced = false; if (_traceArms.Remove(attrName, out var arm) && arm.ExpiresAt > update.Timestamp) { src = arm.SourceExecutionId; traced = true; }` and pass `SourceExecutionId: src, Trace: traced` on every `new AttributeValueChanged(...)` in that loop (three sites). - `HandleSetStaticAttributeCore`: `Trace: command.Trace` on the published change. - No sweep needed: arms are removed on the next echo; a never-echoed arm is overwritten by the next traced write or removed when the actor stops (bounded by attribute count). **Step 3: Run SiteRuntime.Tests fully. Commit** — `git commit -am "feat(site-runtime): InstanceActor trace arm stamps the device echo of a traced write; Trace rides on published changes"` --- ## Task 9: `DeploymentManagerActor` routing — pass `Trace`, return child id + events **Classification:** standard **Estimated implement time:** ~3 min **Parallelizable with:** Task 10, Task 11 **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/DeploymentManagerActor.cs` (`RouteInboundApiCall` ~2866, `RouteInboundApiSetAttributes` ~3081) - Test: `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Actors/DeploymentManagerRouteTraceTests.cs` (find an existing test that drives `RouteInboundApiCall` with a probe instance actor and copy its fixture; if none exists, exercise the two private methods' behaviour through a `TestActorRef` — check how `_instanceActors` is populated in existing tests) **Implement:** `ScriptCallRequest(..., ParentExecutionId: request.ParentExecutionId, Trace: request.Trace)`; map `new RouteToCallResponse(request.CorrelationId, result.Success, NormalizeRoutedReturnValue(result.ReturnValue), result.ErrorMessage, DateTimeOffset.UtcNow, result.ExecutionId, result.TraceEvents)`; `SetStaticAttributeCommand(..., SourceExecutionId: request.ParentExecutionId, Trace: request.Trace)`. Tests assert the forwarded `ScriptCallRequest.Trace` and the mapped `ChildExecutionId`/`TraceEvents`. Commit — `git commit -am "feat(site-runtime): route handlers forward Trace and return child ExecutionId + trace events"` --- ## Task 10: Wire — `site_command.proto` trace fields, mapper, golden samples **Classification:** high-risk (wire contract) **Estimated implement time:** ~5 min **Parallelizable with:** Task 9, Task 11 **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.Communication/Protos/site_command.proto` (add `import "Protos/sitestream.proto";` if `AuditEventDto` is not visible — `central_control.proto` already imports sitestream the same way; copy its import line verbatim) - Regenerate: `src/ZB.MOM.WW.ScadaBridge.Communication/SiteCommandGrpc/*.cs` via `bash docker/regen-proto.sh sitecommand` - Modify: `src/ZB.MOM.WW.ScadaBridge.Communication/Grpc/SiteCommandDtoMapper.cs` (`ToProto/FromProto` for `RouteToCallRequest`, `RouteToCallResponse`, `RouteToSetAttributesRequest`) - Test: `tests/ZB.MOM.WW.ScadaBridge.Communication.Tests/SiteCommandDtoMapperGoldenTests.cs` + the `SiteCommandSamples` it reads (add traced samples), new `tests/.../SiteCommandDtoMapperTraceTests.cs` **Proto (additive, never reuse numbers):** ```proto message RouteToCallRequestDto { ...existing 1-6...; bool trace = 7; } message RouteToCallResponseDto { ...existing 1-5...; string child_execution_id = 6; repeated AuditEventDto trace_events = 7; } message RouteToSetAttributesRequestDto { ...existing 1-5...; bool trace = 6; } ``` **Mapper:** `Trace = request.Trace` / `dto.Trace`; `ChildExecutionId = response.ChildExecutionId?.ToString("D") ?? ""`; `TraceEvents.AddRange(response.TraceEvents?.Select(AuditEventDtoMapper.ToDto) ?? [])`; reverse with `Guid.TryParse` and `dto.TraceEvents.Count == 0 ? null : dto.TraceEvents.Select(AuditEventDtoMapper.FromDto).ToList()` (an old site sends none → `null`, which the UI labels "site not traced"). **Tests:** round-trip with two trace events preserves `ChildExecutionId` and event `Action`/`DetailsJson`; a DTO with no trace fields maps to `Trace=false`/`null`s; golden samples updated (the golden test enforces every record field is covered — follow its failure message). **Also:** confirm `docker/regen-proto.sh` restored the csproj (`git diff src/ZB.MOM.WW.ScadaBridge.Communication/*.csproj` must be empty). Commit — `git commit -am "feat(wire): route_to_call trace flag + child_execution_id + trace_events; route_to_set_attributes trace (additive proto + mapper)"` --- ## Task 11: Wire — `sitestream.proto` `source_execution_id`/`trace` on attribute updates + `PullTraceEvents` RPC; site SQLite read by execution ids **Classification:** high-risk (wire + storage) **Estimated implement time:** ~5 min **Parallelizable with:** Task 9, Task 10 **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.Communication/Protos/sitestream.proto` - Regenerate: `SiteStreamGrpc/*.cs` (and `CentralControlGrpc/*.cs` if the script requires — run `bash docker/regen-proto.sh sitestream`; if central_control's generated code references changed descriptors, run `all`) - Modify: `src/ZB.MOM.WW.ScadaBridge.Communication/Actors/StreamRelayActor.cs` (~43), `src/ZB.MOM.WW.ScadaBridge.Communication/Grpc/SiteStreamGrpcClient.cs` (~554), `src/ZB.MOM.WW.ScadaBridge.Communication/Grpc/SiteStreamGrpcServer.cs` (new override next to `PullAuditEvents`) - Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Interfaces/Services/ISiteAuditQueue.cs`, `src/ZB.MOM.WW.ScadaBridge.AuditLog/Site/SqliteAuditWriter.cs` (schema block ~233–266 + new read) - Create: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/IPullTraceEventsClient.cs`, `GrpcPullTraceEventsClient.cs` (model on `GrpcPullAuditEventsClient` incl. its invoker/PSK plumbing), register in `src/ZB.MOM.WW.ScadaBridge.AuditLog/ServiceCollectionExtensions.cs` beside `IPullAuditEventsClient` - Test: `tests/ZB.MOM.WW.ScadaBridge.Communication.Tests/SiteStreamPullTraceEventsTests.cs` (model on `SiteStreamPullAuditEventsTests`), `tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Site/SqliteAuditWriterReadByExecutionTests.cs`, `tests/ZB.MOM.WW.ScadaBridge.Communication.Tests/StreamRelayTraceStampTests.cs` **Proto:** ```proto message AttributeValueUpdate { ...1-6...; string source_execution_id = 7; bool trace = 8; } service SiteStreamService { ...; rpc PullTraceEvents(PullTraceEventsRequest) returns (PullTraceEventsResponse); } message PullTraceEventsRequest { repeated string execution_ids = 1; google.protobuf.Timestamp since_utc = 2; int32 max_events = 3; } message PullTraceEventsResponse { repeated AuditEventDto events = 1; } ``` **Site SQLite:** `ISiteAuditQueue.ReadByExecutionIdsAsync(IReadOnlyCollection executionIds, DateTime sinceUtc, int limit, CancellationToken ct)` — SQL: ```sql SELECT EventId, OccurredAtUtc, Actor, Action, Outcome, Category, Target, SourceNode, CorrelationId, DetailsJson FROM audit_event WHERE OccurredAtUtc >= @since AND (json_extract(DetailsJson,'$.ExecutionId') IN (...) OR json_extract(DetailsJson,'$.ParentExecutionId') IN (...)) ORDER BY OccurredAtUtc, EventId LIMIT @limit; ``` (check the exact property names `AuditDetailsCodec` writes — camelCase vs Pascal — by reading one serialized row in a test; the central computed column uses `'$.kind'`, so likely camelCase `'$.executionId'`). Add `CREATE INDEX IF NOT EXISTS IX_audit_event_occurred ON audit_event (OccurredAtUtc);` to the schema block (idempotent, additive). Parameterize the `IN` lists (`@e0..@eN`), cap ids at 64. **Server:** `PullTraceEvents` → validate `max_events > 0` and ≤ 2000 → queue read → `AuditEventDtoMapper.ToDto`. The `ControlPlaneAuthInterceptor` already gates the whole `/sitestream.SiteStreamService/` prefix — add a test asserting the new method name is under that prefix (a one-liner on `SiteStreamService.Descriptor.FullName`). **Relay + client:** `SourceExecutionId = msg.SourceExecutionId?.ToString("D") ?? "", Trace = msg.Trace` on the way out; parse back into the domain record on the way in. **Central client:** `Task> PullAsync(string siteId, string nodeAddress…)` following `GrpcPullAuditEventsClient`'s constructor/invoker shape exactly (it resolves the site PSK via `ISitePskProvider`). Commit — `git commit -am "feat(wire): attribute updates carry source_execution_id/trace; PullTraceEvents RPC + site read-by-execution-ids"` --- ## Task 12: `RouteHelper` — always-on `RouteCall` audit row, `WithTrace`, inline trace sink **Classification:** standard **Estimated implement time:** ~5 min **Parallelizable with:** Task 13, Task 14 **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.InboundApi/RouteHelper.cs` (`RouteHelper` ctor/With* ~36–125; `RouteTarget` ctor ~131; `Call` ~178; `SetAttributes` ~341) - Test: `tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/RouteHelperTests.cs` (extend) + `RouteHelperTraceTests.cs` **Implement:** - Public ctor becomes `RouteHelper(IInstanceLocator, IInstanceRouter, ICentralAuditWriter? auditWriter = null, INodeIdentityProvider? nodeIdentity = null)` — DI (`AddScoped()`) resolves both when registered on central; existing `new RouteHelper(locator, router)` test call sites compile. - Private state adds `bool _trace`, `Action? _traceSink`, plus `WithTrace(bool)` and `WithTraceSink(Action?)` (immutable-copy style like the existing `With*`). `RouteTarget` receives them. - `Call`: `Stopwatch`; `RouteToCallRequest(..., _parentExecutionId, Trace: _trace)`; after the response (or exception) build `ScadaBridgeAuditEventFactory.Create(AuditChannel.ScriptTrace, AuditKind.RouteCall, success ? Delivered : Failed, actor: "inbound-api", target: $"{_instanceCode}.{scriptName}", sourceNode: node, executionId: _parentExecutionId, parentExecutionId: null, durationMs, errorMessage, requestSummary: args JSON (8 KB cap), responseSummary: return JSON (8 KB), extra: ScriptTraceExtra { ChildExecutionId = response?.ChildExecutionId })`. Write via `_auditWriter` fire-and-forget (never throws into the script); also `_traceSink?.Invoke(evt)` and `foreach (var e in response.TraceEvents ?? []) _traceSink?.Invoke(e)`. **Always-on:** the row is emitted regardless of `_trace` when a writer is present. - `SetAttributes`: pass `Trace: _trace`. **Tests:** RouteCall row written with `ExecutionId == parent`, `Extra.childExecutionId` from the reply; sink receives RouteCall + the reply's `TraceEvents`; `Trace` propagates on both requests; no writer → no throw; failure path writes `Failed` with the message. Commit — `git commit -am "feat(inbound-api): RouteHelper writes an always-on RouteCall audit row, propagates Trace, exposes an inline trace sink"` --- ## Task 13: `InboundScriptExecutor.ExecuteAsync(trace)` + endpoint honours `TraceAllCalls` **Classification:** small **Estimated implement time:** ~3 min **Parallelizable with:** Task 12, Task 14 **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.InboundApi/InboundScriptExecutor.cs` (~324 signature, ~380 context) - Modify: `src/ZB.MOM.WW.ScadaBridge.InboundApi/EndpointExtensions.cs` (~276) - Test: `tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/InboundScriptExecutorTests.cs` (add one test) **Implement:** `ExecuteAsync(..., Guid? parentExecutionId = null, bool trace = false)` → `route.WithDeadline(...).WithRequestAborted(...).WithParentExecutionId(parentExecutionId).WithTrace(trace)`; endpoint passes `trace: method.TraceAllCalls`. Test: a registered handler that captures `ctx.Route` and asserts the built `RouteToCallRequest.Trace` via a substitute router (use `WithTrace(true)` end-to-end). Commit — `git commit -am "feat(inbound-api): executor threads the trace flag; endpoint honours ApiMethod.TraceAllCalls"` --- ## Task 14: `IAuditLogRepository.GetTraceEventsAsync(rootExecutionId)` **Classification:** standard **Estimated implement time:** ~4 min **Parallelizable with:** Task 12, Task 13 **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Interfaces/Repositories/IAuditLogRepository.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/AuditLogRepository.cs` (beside `GetExecutionTreeAsync`) - Test: `tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Integration/FlowTraceReplayQueryTests.cs` (uses the Integration infrastructure the sibling `ExecutionIdCorrelationTests` uses — same fixture/collection attributes) **Implement:** reuse the upward walk + `Chain` CTE from `GetExecutionTreeAsync` (refactor the chain SQL into a private `ChainCteSql` constant used by both), then `SELECT FROM dbo.AuditLog WHERE ExecutionId IN (SELECT ExecutionId FROM ChainIds) AND (Category = 'ScriptTrace' OR (Category = 'ApiInbound' AND Kind = 'InboundRequest')) ORDER BY OccurredAtUtc, EventId` mapped with the repository's existing row→`AuditEvent` mapper. Test: insert inbound + RouteCall + two site rows chained by parent ids, plus an unrelated row → exactly 4 back, ordered. Commit — `git commit -am "feat(audit): GetTraceEventsAsync returns the ScriptTrace chain for Replay"` --- ## Task 15: Management commands — `FlowDebugRunMethodCommand`, `FlowDebugRunValueChangeCommand`, `FlowDebugReplayCommand` **Classification:** high-risk (security, cross-cluster) **Estimated implement time:** ~5 min (split 15a/15b if it runs long) **Parallelizable with:** Task 16 (after the records exist), Task 17, Task 18 **Files:** - Create: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Management/FlowDebugCommands.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementActor.cs` (`GetRequiredRoles` → `DeployerOnly`; `DispatchCommand`; three handlers), `src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementEndpoints.cs` (`LongRunningCommands` += the two run commands) - Create: `src/ZB.MOM.WW.ScadaBridge.ManagementService/FlowDebug/FlowDebugRunner.cs` (the handlers delegate here so it is unit-testable without the actor) - Modify: `src/ZB.MOM.WW.ScadaBridge.ManagementService/ServiceCollectionExtensions.cs` (`AddSingleton()`) - Test: `tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests/FlowDebugRunnerTests.cs`, extend the existing role-gate test that enumerates `GetRequiredRoles` **Records** (namespace `ZB.MOM.WW.ScadaBridge.Commons.Messages.Management` — `ManagementCommandRegistry` auto-registers by simple name): ```csharp public record FlowDebugRunMethodCommand(int ApiMethodId, JsonElement? Parameters); public record FlowDebugRunValueChangeCommand(int InstanceId, string AttributeName, string Value, int WaitSeconds = 45); public record FlowDebugReplayCommand(Guid ExecutionId); public record FlowDebugRunResult( Guid RootExecutionId, bool Completed, string? Error, string? ErrorCode, IReadOnlyList Events, bool Truncated, DateTimeOffset StartedAtUtc, DateTimeOffset FinishedAtUtc); ``` **`FlowDebugRunner`** (ctor: `IServiceProvider`, `ILogger`): - `RunMethodAsync(cmd, user, ct)`: 1. `IActiveNodeGate gate = sp.GetService()`; if `gate is not null && !gate.IsActive` → `throw new ManagementCommandException("Live runs execute on the ACTIVE central node only (Component-InboundAPI: method scripts and Route.To never run on a standby). Open the Central UI or point the CLI at the active node.") { Code = "STANDBY_NODE" }` (check how `ManagementCommandException` carries a code; add an optional code if it does not). 2. Load the `ApiMethod` (`IInboundApiRepository`), `SchemaRefResolver.BuildAsync` + `ParameterValidator.Validate(cmd.Parameters, method.ParameterDefinitions, resolveRef)` → 400-style error on invalid. 3. `var root = Guid.NewGuid(); var events = new ConcurrentQueue();` 4. Synthesize and write the inbound row via `ICentralAuditWriter`: `Create(ApiInbound, InboundRequest, Submitted → later Delivered/Failed, actor: user.Username, target: method.Name, executionId: root, requestSummary: params JSON, extra: {"source":"flow-debugger"})` — write it once at the end with the final status and duration (one row, like the middleware). 5. `var route = sp.GetRequiredService().WithTraceSink(events.Enqueue)` (create a DI scope for scoped services); `executor.ExecuteAsync(method, parameters, route, timeout, ct, parentExecutionId: root, trace: true)`. 6. Return `FlowDebugRunResult(root, result.Success, result.ErrorMessage, result.ErrorCode, [inboundRow, ..events ordered by OccurredAtUtc], truncated: false, …)`. - `RunValueChangeAsync(cmd, user, ct)`: load instance + site (`ITemplateEngineRepository`, `ISiteRepository`), `EnforceSiteScope`; `root = Guid.NewGuid()`; `CommunicationService.RouteToSetAttributesAsync(site.SiteIdentifier, new RouteToSetAttributesRequest(cid, instance.UniqueName, {attr: value}, now, ParentExecutionId: root, Trace: true))`; then poll `IPullTraceEventsClient.PullAsync(site, [root], since: started, max: 500)` every 1 s until a `ScriptRun` row with `ParentExecutionId == root` exists **and** no new rows arrived in the last 2 s, or `WaitSeconds` elapses (then `Completed=false, ErrorCode="NO_TRIGGER_WITHIN_WINDOW"`). Prepend a synthesized `AttributeWrite`-kind central row (`actor: user`, `target: attr`, `executionId: root`, `extra.source = flow-debugger`) so the graph has a root node. (Pulling needs the site's gRPC node address — copy how `SiteAuditReconciliationActor` picks NodeA/NodeB for `IPullAuditEventsClient`.) - `ReplayAsync(cmd)`: `IAuditLogRepository.GetTraceEventsAsync(cmd.ExecutionId)` → root = the topmost row's `ExecutionId`; `Completed=true`. **Tests (runner, NSubstitute):** standby gate refuses with `STANDBY_NODE`; invalid parameters fail before any route call; happy path returns inbound row + sink events ordered; replay returns repository rows; role gate for all three = Deployer. Commit — `git commit -am "feat(management): FlowDebug run-method / run-value-change / replay commands (Deployer, active-node gated, inline trace)"` --- ## Task 16: CLI `flow-debug` command group **Classification:** small **Estimated implement time:** ~4 min **Parallelizable with:** Task 17, Task 18 **Files:** - Create: `src/ZB.MOM.WW.ScadaBridge.CLI/Commands/FlowDebugCommands.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.CLI/Program.cs` (register), `src/ZB.MOM.WW.ScadaBridge.CLI/README.md` (new `flow-debug` section modelled on `deploy`) - Test: `tests/ZB.MOM.WW.ScadaBridge.CLI.Tests/FlowDebugCommandsTests.cs` (model on an existing command test that asserts the built management command) **Commands:** `flow-debug run-method --method-id [--params ]`, `flow-debug run-value-change --instance-id --attribute --value [--wait-seconds 45]`, `flow-debug replay --execution-id `; all via `CommandHelpers.ExecuteCommandAsync(..., timeout: TimeSpan.FromMinutes(2))`; add a `tableProjector` that prints one line per event: `occurred | kind | lane-ish target | duration | status` (JSON format stays the raw `FlowDebugRunResult`). Commit — `git commit -am "feat(cli): flow-debug run-method|run-value-change|replay"` --- ## Task 17: `FlowShapePredictor` (Script Analysis) **Classification:** standard **Estimated implement time:** ~5 min **Parallelizable with:** Task 15, Task 16, Task 18 **Files:** - Create: `src/ZB.MOM.WW.ScadaBridge.ScriptAnalysis/FlowShapePredictor.cs` - Test: `tests/ZB.MOM.WW.ScadaBridge.ScriptAnalysis.Tests/FlowShapePredictorTests.cs` **API:** ```csharp public sealed record PredictedNode(string Id, FlowStepKind Kind, FlowLane Lane, string Title, bool InputDependent, string? Note); public sealed record PredictedEdge(string From, string To, bool CrossesBoundary); public sealed record PredictedFlow(IReadOnlyList Nodes, IReadOnlyList Edges); public static class FlowShapePredictor { /// (instanceCode, scriptName) → the site script body when the code is a constant /// and central can resolve instance → template → script; null otherwise. Keeps this project DB-free. public static PredictedFlow PredictInboundMethod(string methodName, string methodScript, Func resolveRouteScript, int maxDepth = 4); public static PredictedFlow PredictSiteScript(string scriptName, string scriptBody, int maxDepth = 4); } ``` **Walker** (`CSharpSyntaxWalker` over `CSharpSyntaxTree.ParseText(code, new CSharpParseOptions(kind: SourceCodeKind.Script))`): visit `InvocationExpressionSyntax`, match the member-access chains by text with whitespace stripped, exactly as `ScriptTrustValidator.HardeningWalker` does: `Route.To(X).Call(Y,…)` / `.SetAttributes` / `.WaitForAttribute` / `.GetAttributes`; `Scripts.Call(Y)`; `…Attributes.WriteBatchAndWaitAsync(…)`, `…Attributes.WaitAsync(…)`, `…Attributes.WaitForAsync(…)`, `…Attributes.SetAsync(…)`, `ExternalSystem.Call/CachedCall`, `Database.*`, `Notify.Send`; plus `AssignmentExpressionSyntax` whose left side is an `ElementAccessExpressionSyntax` on `…Attributes[...]`. Constant string arguments (`LiteralExpressionSyntax` string) resolve; anything else → `InputDependent = true`, title `Route.To()`. An `IfStatementSyntax` containing a `ReturnStatementSyntax` that precedes the first routed call becomes a `Note = "early return: "` on the edge into the next node. Nodes get ids `n1…`. `PredictInboundMethod` adds `InboundRequest` (Central) first and `Response` (Central) last, and for a constant route target recurses one hop into `resolveRouteScript(...)`, adding its nodes in the Site/Tags lanes. Depth cap `maxDepth`. **Tests:** Delmia method script + site script (verbatim from `docs/api-specs/Delmia-Recipe-Download-API.md` §4) → 8 nodes in the right lanes, route node `InputDependent` (target is `code`), site nodes present because the test resolver returns the site script for any code, early-return note present; a constant `Route.To("Z28061Sim")` is not input-dependent; a script with no recognised call yields only request+response; recursion stops at `maxDepth`; malformed code never throws (returns request+response with `Note = "parse error"`). Commit — `git commit -am "feat(script-analysis): FlowShapePredictor — predicted flow graph from script syntax"` --- ## Task 18: Central UI service — `IFlowDebugService` / `FlowDebugService` **Classification:** standard **Estimated implement time:** ~5 min **Parallelizable with:** Task 15 (after records), Task 16, Task 17 **Files:** - Create: `src/ZB.MOM.WW.ScadaBridge.CentralUI/Services/IFlowDebugService.cs`, `FlowDebugService.cs` - Modify: `src/ZB.MOM.WW.ScadaBridge.CentralUI/ServiceCollectionExtensions.cs` (`AddScoped()` — find where `IDataConnectionMoveService` is registered and add beside it) - Test: `tests/ZB.MOM.WW.ScadaBridge.CentralUI.Tests/Services/FlowDebugServiceTests.cs` **Interface:** ```csharp public interface IFlowDebugService { Task PredictMethodAsync(int apiMethodId, CancellationToken ct = default); Task PredictValueChangeAsync(int instanceId, string attributeName, CancellationToken ct = default); Task RunMethodAsync(int apiMethodId, JsonElement? parameters, CancellationToken ct = default); Task RunValueChangeAsync(int instanceId, string attributeName, string value, CancellationToken ct = default); Task ReplayAsync(Guid executionId, CancellationToken ct = default); } public sealed record FlowDebugOutcome(FlowGraph? Graph, FlowDebugRunResult? Raw, string? Error, string? ErrorCode); ``` **Implementation:** `ManagementActorHolder` + `AuthenticationStateProvider` exactly as `DataConnectionMoveService` (copy `SendAsync`/`BuildAuthenticatedUserAsync`; Ask timeout 2 min for the run commands); map `ManagementSuccess` payload (`FlowDebugRunResult`) → `FlowGraphBuilder.Build(result.Events, result.Truncated)`; `ManagementUnauthorized`/`ManagementError` → `Error`/`ErrorCode`. Predict: load method/instance via repositories (`IInboundApiRepository`, `ITemplateEngineRepository`) and give `FlowShapePredictor` a resolver that maps `instanceCode → Instance.UniqueName → template → TemplateScript(name).Code` (find the repository call the template editor uses to list scripts; `IFlatteningPipeline.PreviewAsync` is heavier than needed — use the template's own script list, inheritance-resolved if `ITemplateInheritanceQueryService` offers scripts, else the template's direct scripts with a `Note = "inherited scripts not resolved"`). **Tests:** holder null → `SERVICE_UNAVAILABLE`; `ManagementError` maps to `ErrorCode`; success maps to a graph with the right step count; predict wires the resolver (substitute repos). Commit — `git commit -am "feat(central-ui): FlowDebugService — management-command bridge + predicted graph"` --- ## Task 19: `FlowGraphView` — custom SVG swimlane renderer **Classification:** standard **Estimated implement time:** ~5 min **Parallelizable with:** Task 18, Task 20 **Files:** - Create: `src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Shared/FlowGraphView.razor`, `FlowGraphView.razor.cs`, `FlowGraphView.razor.css` - Test: `tests/ZB.MOM.WW.ScadaBridge.CentralUI.Tests/Shared/FlowGraphViewTests.cs` **Contract:** parameters `FlowGraph? Graph`, `PredictedFlow? Predicted`, `Guid? SelectedEventId`, `EventCallback OnSelect`, `double Zoom = 1.0`. Layout (pure, in the code-behind, unit-tested): three fixed lanes of 110 px (labels Central / Site / Tags from the steps' `SiteId`/`Instance`), columns assigned by topological order (a step's column = 1 + max column of its predecessors; siblings in the same lane share a column only if independent), node 150×78 at `x = 130 + col*170`, `y = 16 + lane*110`; connectors as orthogonal `` with `marker-end` arrows; `CrossesBoundary` edges use the accent colour; `External` edges dashed; predicted-only nodes dashed; predicted nodes not present in the real graph greyed. Node content = kind eyebrow + duration, title, subtitle (from `FlowStep`). Mirror the mockup's CSS values (see the design canvas): borders `#d2d2cb`, selected `2px #2f5fd0` on `#eef2fc`, eyebrow `0.6rem` uppercase, mono duration — but use the kit's CSS variables (`var(--rule-strong)`, `var(--accent)`, `var(--active-bg)`) so dark mode follows. `InvariantCulture` for every coordinate string (see `KpiTrendChart`). Click a node → `OnSelect`. `role="img"` + `aria-label` summary; each node a `` with Enter/Space selection. **Tests:** layout assigns Delmia-shaped graph to expected columns/lanes; selected node has the `selected` class; clicking raises `OnSelect` with the step; empty graph renders the lanes and a "No steps" note; coordinates are culture-invariant under `de-DE`. Commit — `git commit -am "feat(central-ui): FlowGraphView SVG swimlane renderer"` --- ## Task 20: Flow Debugger page — toolbar, inputs, predicted graph, run, detail panel **Classification:** standard **Estimated implement time:** ~5 min ×2 (split as 20a page skeleton + predicted graph + inputs; 20b run/replay + detail panel) **Parallelizable with:** Task 19 (20a can start with a stub `FlowGraphView`) **Files:** - Create: `src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Pages/Deployment/FlowDebugger.razor`, `FlowDebugger.razor.cs`, `FlowDebugger.razor.css` - Create: `src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Deployment/FlowStepDetail.razor` (tabs: Input · Output · Attribute writes · Timing; *Where this ran* column; drill-ins `Open in Debug View` → `/deployment/debug-view?instanceId=…` if that page reads a query param — check `DebugView.razor`; `Audit rows (n)` → `/audit/log?executionId=…`) - Modify: `src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Layout/NavMenu.razor` (Deployment section, after Debug View: ``) - Test: `tests/ZB.MOM.WW.ScadaBridge.CentralUI.Tests/Deployment/FlowDebuggerPageTests.cs` (bUnit, fixture copied from `DebugViewDisposalTests.RegisterServices` + `Substitute.For()`) **Page (`@page "/deployment/flow-debugger"`, `[Authorize(Policy = AuthorizationPolicies.RequireDeployment)]`):** query params `?methodId=`, `?instanceId=&attribute=`, `?executionId=` (replay drill-in). State machine: `Idle → Predicted → Running → Completed|Failed|TimedOut`. Toolbar/status/flow/detail exactly as the design §4 and the mockup; mode toggle `Live | Replay` (Replay shows the ExecutionId box instead of inputs). Inputs form generated from `ApiMethod.ParameterDefinitions` (reuse the JSON-schema parsing `ParameterValidator` uses — if it is internal, expose a small `ParameterDefinitionReader` in Commons or parse the schema's `properties`/`required` directly with `System.Text.Json`); values collected into a `JsonElement` object. Value-change entry: Site → Instance → Attribute pickers (instances from `ITemplateEngineRepository` filtered by `SiteScopeService`), current value shown via `IInstanceSnapshotClient` if cheap, else omitted. Persist last entry/inputs in `localStorage` via `IJSRuntime` (same pattern Debug View uses). Run button disabled while running; a `STANDBY_NODE` error renders as an inline alert with the message verbatim. After a run, the predicted graph is overlaid: predicted nodes with no real counterpart render greyed (matching by kind + title). Selecting a node fills `FlowStepDetail`. **Tests:** renders with Deployer claim; predicted graph appears after choosing a method (service substitute returns a 3-node prediction); Run calls `RunMethodAsync` with the typed inputs and renders the returned graph's step count; `STANDBY_NODE` shows the alert; `?executionId=` triggers `ReplayAsync` on first render; a graph whose site lane is empty and whose route call has no `ChildExecutionId` shows the "site not traced" label. Commit — `git commit -am "feat(central-ui): Flow Debugger page (predict, live run, replay, step detail) + nav entry"` --- ## Task 21: Drill-ins and the `TraceAllCalls` checkbox **Classification:** small **Estimated implement time:** ~4 min **Parallelizable with:** Task 22 **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Audit/AuditEventDetail.razor` (+ `.razor.cs`): button `Open in Flow Debugger` (`data-test="open-flow-debugger"`) beside *View execution chain* → `/deployment/flow-debugger?executionId={exec}`; rendered when `ExecutionId` is non-null. - Modify: `src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Pages/Audit/ExecutionTreePage.razor`: same button next to *View this execution in the Audit Log*. - Modify: `src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Pages/Design/ApiMethodForm.razor`: under the Timeout field a `form-check` "Trace all calls (Flow Debugger)" bound to `_traceAllCalls`, help text "Adds 7–10 audit rows per call; for low-volume integration methods", saved onto `_existing.TraceAllCalls` / the create path. - Test: `tests/ZB.MOM.WW.ScadaBridge.CentralUI.Tests/Audit/FlowDebuggerDrillinTests.cs`, extend the existing `ApiMethodForm` test file if one exists (grep `ApiMethodForm` in tests). Commit — `git commit -am "feat(central-ui): Flow Debugger drill-ins from Audit Log + Execution Chain; TraceAllCalls checkbox on the method form"` --- ## Task 22: Rig — retention override, redeploy, verification step H **Classification:** standard **Estimated implement time:** ~5 min (plus the rig rebuild wall time — run `bash docker/deploy.sh` in the background) **Parallelizable with:** Task 21 **Files:** - Modify: `docker/central-node-a/appsettings.Central.json`, `docker/central-node-b/appsettings.Central.json` — add `"AuditLog": { "PerChannelRetentionDays": { "ScriptTrace": 30 } }` at the top level (the file's root key today is `ScadaBridge` only; `AuditLog` is a sibling root section — confirm by grepping `GetSection("AuditLog")` in the Host, or place it wherever the existing rig sites configure `AuditLog:SiteWriter`). Validator requires 30 ≤ value ≤ global `RetentionDays` (365 default) — 30 is the floor. - Modify: `docker/verify-delmia-notifier.sh` — after G add **H**: `scadabridge --url "$(url_of "$ACT")" api-method list` to find `DelmiaRecipeDownload`'s id, then `flow-debug run-method --method-id --params '{"MachineCode":"DelmiaSim01","DownloadPath":"C:\\recipes\\wo1.nc","WorkOrderNumber":"WO-H","PartNumber":"PN-1","JobStepNumber":"0100","Username":"drill"}' --format json`; assert with `python3 -c` that `Completed == true` and the event kinds include `InboundRequest, RouteCall, ScriptRun, AttributeWrite, AttributeWait`; then `flow-debug replay --execution-id --format json` and assert the same kind multiset (Replay may lag the 5 s telemetry — retry up to 40 s). Print `H live+replay -> YES`. - Modify: `docker/README.md` one line for step H. **Steps:** `bash docker/deploy.sh` (background, ~minutes); when both centrals and site-a are `/health/ready`, run `DELMIA_API_KEY=… bash docker/verify-delmia-notifier.sh` (the key: mint a new-named one per the `delmia-rig-standin-state` memory) and paste the H line into the commit message. If the run fails, fix forward — do not mark the task done on a failing H. Commit — `git commit -am "chore(rig): ScriptTrace retention override; verify-delmia-notifier step H = Flow Debugger live run + replay"` --- ## Task 23: Documentation sweep **Classification:** trivial **Estimated implement time:** ~5 min **Parallelizable with:** none (last) **Files:** - `docs/requirements/Component-CentralUI.md` — new `### Flow Debugger (Deployment Role)` before `### Alarm Summary` (line ~218): page, modes, entry points, predicted graph, drill-ins, Deployer gate, `STANDBY_NODE` behaviour; add to the page list if the doc has one. - `docs/requirements/Component-InboundAPI.md` — §*API Method Definition → Properties*: `TraceAllCalls`; §*Request Flow*: the always-on `RouteCall` row and the `Trace` flag; new §*Flow Debugger runs* (management commands, inbound row synthesized with operator actor). - `docs/requirements/Component-SiteRuntime.md` — under *Instance Actor*: *Trace arm*; under *Script Actor → Script Run Launch*: traced runs, `ScriptRun` row, inline events, value-change parent inheritance; under *Handling `Instance.CallScript`*: flag inheritance. - `docs/requirements/Component-AuditLog.md` — §*Scope*: `ScriptTrace` channel (gated) + `RouteCall` (always-on); §*Retention*: per-channel override example; §*Reconciliation pull*: `PullTraceEvents` (point read, not a reconciliation path); the mixed-version caveat. - `docs/requirements/Component-Communication.md` — §8 *Remote Queries*: `PullTraceEvents`; §4 *Integration Routing*: trace flag + inline events on the reply; proto field list. - `docs/requirements/Component-ScriptAnalysis.md` — `REQ-SA-7: Flow shape predictor`. - `CLAUDE.md` — *UI & Monitoring*: one bullet after the Debug view bullet: Flow Debugger (Live + Replay; trace = audit events in `ScriptTrace`, gated by a `Trace` flag beside `ParentExecutionId`; always-on `RouteCall`; trace arm; management-command execution path, `STANDBY_NODE`; `TraceAllCalls`; design + plan links). - `src/ZB.MOM.WW.ScadaBridge.CLI/README.md` — confirm `flow-debug` + `--trace-all-calls` sections landed. - `docs/plans/2026-07-08-deferred-work-register.md` — new row 110: "Flow Debugger *Vertical trace* list toggle (mockup Direction B) and console capture on `ScriptRun`"; revisit trigger: first operator request. - `docs/plans/2026-09-05-flow-debugger-design.md` — status → "implemented ; plan `2026-09-05-flow-debugger.md`". - `../scadaproj/CLAUDE.md` — only if the ScadaBridge index row mentions the component summary in a way this changes (it lists stack/wire relationships; a new page does not change them — verify and skip if so). Commit — `git commit -am "docs: Flow Debugger — component specs, CLAUDE.md, CLI README, register row 110"` --- ## Dependency order ``` 0 → {1,2,3,5} → 4 → 6 → 7 → 8 → 9 → {10,11} → {12,13,14} → 15 → {16,17,18} → 19 → 20 → 21 → 22 → 23 ``` Parallel lanes once Task 2 is in: (6,7,8,9) site runtime · (10,11) wire · (12,13,14) central audit/inbound · (17) predictor · (16) CLI after 15's records. Task 20 needs 18 + 19. ## Verification before "done" `dotnet build ZB.MOM.WW.ScadaBridge.slnx` clean; `dotnet test ZB.MOM.WW.ScadaBridge.slnx` green; `git diff --stat src/ZB.MOM.WW.ScadaBridge.Communication/*.csproj` empty (no live `` item); rig step H prints YES twice (live + replay); the Central UI page renders the Delmia run with 9 nodes on `http://localhost:9001/deployment/flow-debugger` (Traefik or direct).