Files

T

Joseph Doherty 2e3eb94d38 docs(plans): Flow Debugger implementation plan — 24 tasks, TDD, additive contracts first, rig step H last

Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29

2026-09-05 23:22:55 -04:00

68 KiB

Raw Blame History

Flow Debugger Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task.

Goal: Ship the Flow Debugger — a Central UI page (plus CLI parity) that runs an inbound API method or a value-change trigger with operator inputs, or replays a past ExecutionId, and renders every step as a swimlane graph with per-step input/output.

Architecture: Trace steps ARE audit events in a new AuditChannel.ScriptTrace (kinds RouteCall, ScriptRun, AttributeWrite, AttributeWait), emitted only when a Trace flag rides beside ParentExecutionId; the routed reply carries the site's events back inline so a Live run is instant, and the same events land in the audit pipeline for Replay. A per-node FlowDebugRun*Command on the ManagementActor (Deployer role, refuses on a standby with the active node named) is the single execution path for the UI and the CLI. Central projects events into a FlowGraph (lanes Central/Site/Tags) rendered by a custom SVG component; a Roslyn syntax walker predicts the graph before a run.

Tech Stack: .NET 10, Akka.NET (actors, TestKit), gRPC/protobuf (vendored C#, docker/regen-proto.sh), EF Core (SQL Server), SQLite (site audit), Blazor Server + bUnit, System.CommandLine (CLI), Roslyn (CSharpSyntaxWalker), xunit + NSubstitute.

Design: docs/plans/2026-09-05-flow-debugger-design.md (approved 2026-09-05). Task 0 amends it with four corrections found while planning.

Conventions for every task: build with dotnet build ZB.MOM.WW.ScadaBridge.slnx -nologo -v q; run a test project with dotnet test tests/<Project>/<Project>.csproj -nologo -v q [--filter FullyQualifiedName~<Class>]. Commit messages end with Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29. All record changes are additive trailing optional parameters — every constructor call site in src/ and tests/ keeps compiling unchanged.

Task 0: Amend the design doc with the four planning corrections

Classification: trivial Estimated implement time: ~4 min Parallelizable with: none (do first)

Files:

Modify: docs/plans/2026-09-05-flow-debugger-design.md

Step 1: Apply the corrections (edit the named sections in place; keep everything else):

§5.1 / §5.4 / §6 — trace steps are audit events, not a new record. Replace the FlowStep wire type with: "A trace step IS an AuditEvent in channel ScriptTrace. RouteToCallResponse.TraceEvents (IReadOnlyList<AuditEvent>?) carries the site's events inline; on the wire it is repeated AuditEventDto trace_events (the DTO sitestream.proto already defines). FlowStep is a central-side projection (Commons/Types/Debugging/FlowStepProjector) computed from AuditEvent + AuditDetails." Drop the FlowStepDto proto message and the TriggerFired kind — a value-change run's ScriptRun row carries trigger: {attribute, value, source} in Extra.
§5.1 — who emits AttributeWrite/AttributeWait. Replace "Site InstanceActor write handlers" with "ScriptRuntimeContext.SetAttribute / WriteBatchAndWait / WaitInternal — every accessor path funnels into these three methods, the context already holds IAuditWriter, _executionId and _parentExecutionId, so the actor needs no new ids." Delete the WriteAttributeBatchRequest/WaitForAttributeRequest rows from the §6 table. Note: batch-write echoes are not armed in v1 (only SetStaticAttributeCommand writes arm — §5.3).
§5.3 — both attribute kinds use one command. RouteToSetAttributes → SetStaticAttributeCommand already dispatches data-sourced attributes to the DCL (InstanceActor.HandleSetDataAttribute); the trace arm is set there when command.Trace && command.SourceExecutionId != null, consumed in HandleTagValueUpdate.
§7.1 — execution path. The ManagementActor is a per-node actor (AkkaHostedService line ~490, "/user/management"), not a singleton. Replace the paragraph with: "Live runs and Replay are FlowDebugRunMethodCommand / FlowDebugRunValueChangeCommand / FlowDebugReplayCommand on the ManagementActor (Deployer role, long-running timeout). The run-method handler consults IActiveNodeGate and refuses on a standby with error code STANDBY_NODE naming the design rule; Traefik routes UI traffic to the active node so this is rare. The Central UI calls it through ManagementActorHolder (the DataConnectionMoveService pattern) and the CLI through /management — one path, one audit story. The inbound row for a Live run is written by the handler via ICentralAuditWriter (kind InboundRequest, actor = operator username, Extra.source = "flow-debugger")."
§10 — mixed-version caveat. Add: "An OLD central ingesting ScriptTrace rows from a NEW site parses unknown channel/kind names to the AuditRowProjection fallbacks (ApiInbound/InboundRequest); upgrade central first."
§11 add "Console output capture on ScriptRun rows (no script console exists today)".

Step 2: Commit

git add docs/plans/2026-09-05-flow-debugger-design.md
git commit -m "docs(plans): flow debugger design — trace steps are audit events; mgmt-command execution path; context-emitted write/wait rows"

Task 1: Audit enums — `ScriptTrace` channel and four kinds

Classification: small Estimated implement time: ~3 min Parallelizable with: Task 2, Task 3, Task 5

Files:

Modify: src/ZB.MOM.WW.ScadaBridge.Commons/Types/Enums/AuditChannel.cs
Modify: src/ZB.MOM.WW.ScadaBridge.Commons/Types/Enums/AuditKind.cs
Test: tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Audit/ScriptTraceEnumTests.cs

Step 1: Write the failing test

using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;

namespace ZB.MOM.WW.ScadaBridge.Commons.Tests.Audit;

public class ScriptTraceEnumTests
{
    [Theory]
    [InlineData(AuditKind.RouteCall)]
    [InlineData(AuditKind.ScriptRun)]
    [InlineData(AuditKind.AttributeWrite)]
    [InlineData(AuditKind.AttributeWait)]
    public void ScriptTraceKinds_FitTheCentralColumns(AuditKind kind)
    {
        // AuditLog.Category / Kind are nvarchar(32) (AuditLogEntityTypeConfiguration).
        Assert.True(kind.ToString().Length <= 32);
        Assert.True(AuditChannel.ScriptTrace.ToString().Length <= 32);
        Assert.Equal($"ScriptTrace.{kind}", AuditFieldBuilders.BuildAction(AuditChannel.ScriptTrace, kind));
    }

    [Fact]
    public void ScriptTrace_IsAppendedAfterCluster_SoExistingOrdinalsAreStable()
    {
        Assert.True((int)AuditChannel.ScriptTrace > (int)AuditChannel.Cluster);
    }
}

Step 2: Run it to make sure it fails — dotnet test tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/ZB.MOM.WW.ScadaBridge.Commons.Tests.csproj --filter FullyQualifiedName~ScriptTraceEnumTests → compile error ScriptTrace not found.

Step 3: Implement — append to AuditChannel:

    /// <summary>
    /// Flow Debugger trace rows (design 2026-09-05): a script run, an attribute write, an
    /// attribute wait or a central route call, emitted only for traced executions
    /// (plus the always-on <see cref="AuditKind.RouteCall"/>). Appended last so stored
    /// ordinals of existing channels never move.
    /// </summary>
    ScriptTrace

and to AuditKind (after ManualFailover, keeping the file's explicit-value style if it uses one — check the last member and continue the sequence):

    /// <summary>Central `Route.To().Call` round-trip (always-on; ScriptTrace channel).</summary>
    RouteCall,
    /// <summary>One site script run completed/failed/timed out (traced only).</summary>
    ScriptRun,
    /// <summary>One attribute write or batch write made by a traced script (traced only).</summary>
    AttributeWrite,
    /// <summary>One `WaitAsync`/`WaitForAsync` outcome in a traced script (traced only).</summary>
    AttributeWait,

Step 4: Run the test → PASS. Also run dotnet test tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/...csproj fully (some tests enumerate Enum.GetValues<AuditChannel>(); fix any that pin the count).

Step 5: Commit — git commit -am "feat(audit): ScriptTrace channel + RouteCall/ScriptRun/AttributeWrite/AttributeWait kinds"

Task 2: Additive `Trace` / trace-payload fields on the message records

Classification: high-risk (data contracts) Estimated implement time: ~5 min Parallelizable with: Task 1, Task 3, Task 5

Files:

Modify: src/ZB.MOM.WW.ScadaBridge.Commons/Messages/InboundApi/RouteToInstanceRequest.cs
Modify: src/ZB.MOM.WW.ScadaBridge.Commons/Messages/ScriptExecution/ScriptCallRequest.cs
Modify: src/ZB.MOM.WW.ScadaBridge.Commons/Messages/ScriptExecution/ScriptCallResult.cs
Modify: src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Instance/SetStaticAttributeCommand.cs
Modify: src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Streaming/AttributeValueChanged.cs
Test: tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Messages/TraceFlagDefaultsTests.cs

Step 1: Failing test

using ZB.MOM.WW.Audit;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.InboundApi;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Instance;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.ScriptExecution;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Streaming;

namespace ZB.MOM.WW.ScadaBridge.Commons.Tests.Messages;

public class TraceFlagDefaultsTests
{
    [Fact]
    public void ExistingPositionalConstructions_DefaultToUntraced()
    {
        var now = DateTimeOffset.UtcNow;
        Assert.False(new RouteToCallRequest("c", "i", "s", null, now).Trace);
        Assert.False(new RouteToSetAttributesRequest("c", "i", new Dictionary<string, string>(), now).Trace);
        Assert.False(new ScriptCallRequest("s", null, 0, "c").Trace);
        Assert.False(new SetStaticAttributeCommand("c", "i", "a", "v", now).Trace);
        Assert.False(new AttributeValueChanged("i", "p", "a", 1, "Good", now).Trace);

        var reply = new RouteToCallResponse("c", true, null, null, now);
        Assert.Null(reply.ChildExecutionId);
        Assert.Null(reply.TraceEvents);

        var result = new ScriptCallResult("c", true, null, null);
        Assert.Null(result.ExecutionId);
        Assert.Null(result.TraceEvents);
    }

    [Fact]
    public void TraceEvents_AreAuditEvents()
    {
        var evt = new AuditEvent { EventId = Guid.NewGuid(), OccurredAtUtc = DateTimeOffset.UtcNow,
            Actor = "a", Action = "ScriptTrace.ScriptRun", Outcome = AuditOutcome.Success };
        var result = new ScriptCallResult("c", true, null, null, Guid.NewGuid(), [evt]);
        Assert.Single(result.TraceEvents!);
    }
}

(Check AuditOutcome's member names in ~/Desktop/scadaproj/ZB.MOM.WW.Audit/src/ZB.MOM.WW.Audit/AuditOutcome.cs and use a real one.)

Step 2: Run → fails to compile.

Step 3: Implement — append trailing optional parameters (doc-comment each with one line: "Flow Debugger trace flag — see design 2026-09-05 §5.2"):

// RouteToInstanceRequest.cs
public record RouteToCallRequest(..., Guid? ParentExecutionId = null, bool Trace = false);
public record RouteToCallResponse(string CorrelationId, bool Success, object? ReturnValue, string? ErrorMessage,
    DateTimeOffset Timestamp, Guid? ChildExecutionId = null, IReadOnlyList<AuditEvent>? TraceEvents = null);
public record RouteToSetAttributesRequest(..., Guid? ParentExecutionId = null, bool Trace = false);

// ScriptCallRequest.cs
public record ScriptCallRequest(..., Guid? ParentExecutionId = null, bool Trace = false);

// ScriptCallResult.cs
public record ScriptCallResult(string CorrelationId, bool Success, object? ReturnValue, string? ErrorMessage,
    Guid? ExecutionId = null, IReadOnlyList<AuditEvent>? TraceEvents = null);

// SetStaticAttributeCommand.cs
public record SetStaticAttributeCommand(..., Guid? SourceExecutionId = null, bool Trace = false);

// AttributeValueChanged.cs
public record AttributeValueChanged(..., Guid? SourceExecutionId = null, bool Trace = false) : ISiteStreamEvent;

Add using ZB.MOM.WW.Audit; where AuditEvent is used (Commons already references the ZB.MOM.WW.Audit package — IAuditWriter.cs uses it).

Step 4: Build the whole solution (dotnet build ZB.MOM.WW.ScadaBridge.slnx) — must be green with zero call-site edits. Run Commons.Tests → PASS.

Step 5: Commit — git commit -am "feat(commons): additive Trace flag + ChildExecutionId/TraceEvents on route/call/write/change records"

Task 3: `ApiMethod.TraceAllCalls` — entity, EF migration, management command, CLI flag

Classification: high-risk (migration) Estimated implement time: ~5 min Parallelizable with: Task 1, Task 2, Task 5

Files:

Modify: src/ZB.MOM.WW.ScadaBridge.Commons/Entities/InboundApi/ApiMethod.cs
Modify: src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Configurations/InboundApiConfiguration.cs
Create: src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Migrations/<timestamp>_AddApiMethodTraceAllCalls.cs (+ Designer, snapshot update — generated)
Modify: src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Management/InboundApiCommands.cs
Modify: src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementActor.cs (HandleCreateApiMethod, HandleUpdateApiMethod)
Modify: src/ZB.MOM.WW.ScadaBridge.CLI/Commands/ApiMethodCommands.cs, src/ZB.MOM.WW.ScadaBridge.CLI/README.md (§ api-method create|update)
Test: tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests/ApiMethodTraceAllCallsTests.cs

Step 1: Failing test (follow the existing ManagementService test style for invoking HandleUpdateApiMethod — find a sibling test that exercises UpdateApiMethodCommand and copy its fixture; assert method.TraceAllCalls round-trips and defaults to false for the 5-arg constructor).

Step 2: Implement

ApiMethod: public bool TraceAllCalls { get; set; } with summary "When true every production call to this method is fully traced (Flow Debugger design §5.2); intended for low-volume integration methods — ~7 audit rows per call."
EF: builder.Property(m => m.TraceAllCalls).IsRequired().HasDefaultValue(false);
Migration: build first (CLAUDE.md gotcha), then dotnet ef migrations add AddApiMethodTraceAllCalls --project src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase --startup-project src/ZB.MOM.WW.ScadaBridge.Host (use the same --startup-project the previous migration used — check git log -1 --stat -- src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Migrations/20260819201210_AddNotificationsListSortIndex.cs for its commit message). Verify the generated Up contains exactly one AddColumn<bool>("TraceAllCalls", "ApiMethods", nullable: false, defaultValue: false); delete and redo if empty.
Commands: CreateApiMethodCommand(..., string? ReturnDefinition, bool TraceAllCalls = false), UpdateApiMethodCommand(..., string? ReturnDefinition, bool? TraceAllCalls = null) (null = leave unchanged, so old CLIs don't reset it). Handlers copy the value.
CLI: --trace-all-calls (Option<bool> on create; Option<bool?> on update) → passed through. README: add the flag to both usage lines with the one-line warning.

Step 3: Run ManagementService.Tests + CLI.Tests → PASS. Step 4: Commit — git commit -am "feat(inbound-api): ApiMethod.TraceAllCalls (entity, migration, mgmt command, CLI --trace-all-calls)"

Task 4: `FlowStep` projection + `FlowGraph` model + `FlowGraphBuilder` (pure)

Classification: standard Estimated implement time: ~5 min Parallelizable with: Task 5, Task 6, Task 8

Files:

Create: src/ZB.MOM.WW.ScadaBridge.Commons/Types/Debugging/FlowStep.cs
Create: src/ZB.MOM.WW.ScadaBridge.Commons/Types/Debugging/FlowGraph.cs
Create: src/ZB.MOM.WW.ScadaBridge.Commons/Types/Debugging/FlowStepProjector.cs
Create: src/ZB.MOM.WW.ScadaBridge.Commons/Types/Debugging/FlowGraphBuilder.cs
Create: src/ZB.MOM.WW.ScadaBridge.Commons/Types/Debugging/ScriptTraceExtra.cs
Test: tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Debugging/FlowGraphBuilderTests.cs

Step 1: Types

namespace ZB.MOM.WW.ScadaBridge.Commons.Types.Debugging;

public enum FlowLane { Central, Site, Tags }
public enum FlowStepKind { InboundRequest, RouteCall, ScriptRun, AttributeWrite, AttributeWait, Response, External, Unknown }
public enum FlowStepOutcome { Ok, Failed, TimedOut, Predicted, NotTraced }

/// <summary>One node of the flow graph — a projection of a ScriptTrace/ApiInbound AuditEvent.</summary>
public sealed record FlowStep(
    Guid EventId, Guid? ExecutionId, Guid? ParentExecutionId,
    FlowStepKind Kind, FlowLane Lane, string Title, string? Subtitle,
    DateTimeOffset OccurredAtUtc, int? DurationMs, FlowStepOutcome Outcome,
    string? InputJson, string? OutputJson, string? ErrorMessage,
    string? SiteId, string? Node, string? Instance, string? Target,
    ScriptTraceExtra? Extra);

public sealed record FlowEdge(Guid FromEventId, Guid ToEventId, bool CrossesBoundary, bool External);
public sealed record FlowGraph(IReadOnlyList<FlowStep> Steps, IReadOnlyList<FlowEdge> Edges, bool Predicted, bool Truncated);

/// <summary>Kind-specific detail carried in AuditDetails.Extra as JSON (camelCase).</summary>
public sealed record ScriptTraceExtra
{
    public string? Node { get; init; }                    // node-a / central-a
    public string? TriggerKind { get; init; }             // call | valuechange | ...
    public string? TriggerAttribute { get; init; }
    public string? TriggerValue { get; init; }
    public string? TriggerSource { get; init; }           // static-write | device-echo
    public Guid? ChildExecutionId { get; init; }          // RouteCall
    public IReadOnlyList<AttributeWriteDetail>? Writes { get; init; }
    public string? WaitAttribute { get; init; }
    public string? WaitTarget { get; init; }
    public int? WaitTimeoutMs { get; init; }
    public string? WaitOutcome { get; init; }             // satisfied | timeout | cancelled | error
    public string? ObservedValue { get; init; }
    public string? Source { get; init; }                  // "flow-debugger" on the synthesized inbound row
}
public sealed record AttributeWriteDetail(string Attribute, string? Written, bool Success, string? Error, bool IsStatic);

FlowStepProjector.Project(AuditEvent evt): deserialize AuditDetails via AuditDetailsCodec.Deserialize, parse Channel/Kind (ScriptTrace.* and ApiInbound.InboundRequest), lane = RouteCall|InboundRequest → Central, ScriptRun|AttributeWait → Site, AttributeWrite → Tags; Outcome from Status (Delivered→Ok, Failed→Failed; a ScriptRun whose ErrorMessage contains "timed out" → TimedOut); Extra via JsonSerializer.Deserialize<ScriptTraceExtra>(details.Extra, camelCase); Title = Target (script name, {instance}.{script}, attribute); Input/Output = RequestSummary/ResponseSummary.

FlowGraphBuilder.Build(IReadOnlyList<AuditEvent> events, bool truncated = false): project all; order by OccurredAtUtc, EventId; within one ExecutionId chain consecutive steps in time order; link a RouteCall's node to the first step of the execution whose ParentExecutionId matches it (CrossesBoundary = true); link the last site step back to a synthesized Response step when an InboundRequest row has a ResponseSummary (append it as the final Central node); an AttributeWait that was satisfied gets an External edge from the preceding AttributeWrite in the same execution. Return FlowGraph(Predicted:false).

Step 2: Failing test — build a canned Delmia-shaped list of 7 AuditEvents via ScadaBridgeAuditEventFactory.Create(...) (inbound, route call, script run, attribute write, attribute wait, and a second attribute write) with matching ExecutionId/ParentExecutionId and assert: 8 steps (incl. synthesized Response), lanes as expected, exactly two CrossesBoundary edges (route→site, site→response), one External edge, order by time. Add a Truncated pass-through test and an "unknown kind → FlowStepKind.Unknown, never throws" test.

Step 3: Run → fail → implement → pass. Step 4: Commit — git commit -am "feat(commons): FlowStep/FlowGraph projection + FlowGraphBuilder for the Flow Debugger"

Task 5: `FlowTraceCollector` (site, per run)

Classification: small Estimated implement time: ~3 min Parallelizable with: Task 1, Task 2, Task 3, Task 4

Files:

Create: src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Scripts/FlowTraceCollector.cs
Test: tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Scripts/FlowTraceCollectorTests.cs

Step 1: Failing test — Add 501 events with MaxEvents = 500 → Snapshot() has 500 and Truncated == true; concurrent adds from 8 tasks never throw and count ≤ cap; an untraced collector is null, not an object (callers null-check).

Step 2: Implement

namespace ZB.MOM.WW.ScadaBridge.SiteRuntime.Scripts;

/// <summary>Per-run sink for ScriptTrace audit events so a traced routed call can return them
/// inline (design 2026-09-05 §5.4). Lock-based: a run emits tens of events, not thousands.</summary>
public sealed class FlowTraceCollector
{
    public const int DefaultMaxEvents = 500;
    private readonly List<AuditEvent> _events = new();
    private readonly object _gate = new();
    private readonly int _max;
    public bool Truncated { get; private set; }
    public FlowTraceCollector(int maxEvents = DefaultMaxEvents) => _max = maxEvents;
    public void Add(AuditEvent evt)
    { lock (_gate) { if (_events.Count >= _max) { Truncated = true; return; } _events.Add(evt); } }
    public IReadOnlyList<AuditEvent> Snapshot() { lock (_gate) return _events.ToArray(); }
}

Step 3: Pass. Commit — git commit -am "feat(site-runtime): FlowTraceCollector"

Task 6: `ScriptRuntimeContext` — traced flag, trace emission for writes and waits, flag inheritance

Classification: high-risk (hot path) Estimated implement time: ~5 min Parallelizable with: Task 4, Task 8

Files:

Modify: src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Scripts/ScriptRuntimeContext.cs (ctor ~line 205; SetAttribute ~513; WriteBatchAndWait ~550; WaitInternal ~471; CallScript ~596; the nested-context construction near line 630 that passes _executionId, _auditWriter…)
Test: tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Scripts/ScriptRuntimeContextTraceEmissionTests.cs

Step 1: Failing tests (TestKit; IAuditWriter = Substitute.For<IAuditWriter>(); instance actor = CreateTestProbe() answering SetStaticAttributeResponse(success) / WaitForAttributeResponse(Matched:true)):

SetAttribute_WhenTraced_EmitsAttributeWriteRow_AndPassesTraceOnCommand — captured SetStaticAttributeCommand.Trace == true; writer received one event whose details Channel == "ScriptTrace", Kind == "AttributeWrite", ExecutionId == ctx.ExecutionId; collector snapshot has it.
SetAttribute_WhenNotTraced_EmitsNothing_AndCommandTraceFalse (the production-unchanged pin).
WaitInternal_WhenTraced_EmitsAttributeWait_WithOutcomeSatisfied.
CallScript_WhenTraced_PropagatesTraceOnScriptCallRequest.

Step 2: Implement

Ctor: append bool isTraced = false, FlowTraceCollector? traceCollector = null; fields _isTraced, _traceCollector; public bool IsTraced => _isTraced; public Guid ExecutionId => _executionId; (if not already public — check; add only what's missing).
Private helper:

private void EmitTrace(AuditKind kind, AuditStatus status, string target, DateTime startedUtc,
    int durationMs, string? input, string? output, string? error, ScriptTraceExtra? extra)
{
    if (!_isTraced) return;
    AuditEvent evt;
    try
    {
        evt = ScadaBridgeAuditEventFactory.Create(
            AuditChannel.ScriptTrace, kind, status, occurredAtUtc: startedUtc,
            actor: _sourceScript, target: target, sourceNode: _sourceNode,
            executionId: _executionId, parentExecutionId: _parentExecutionId,
            sourceSiteId: string.IsNullOrEmpty(_siteId) ? null : _siteId,
            sourceInstanceId: _instanceName, sourceScript: _sourceScript,
            durationMs: durationMs, errorMessage: error,
            requestSummary: input, responseSummary: output,
            extra: extra is null ? null : JsonSerializer.Serialize(extra, TraceJson));
    }
    catch (Exception ex) { _logger.LogWarning(ex, "Flow trace event build failed for {Kind}", kind); return; }
    _traceCollector?.Add(evt);
    if (_auditWriter is null) return;
    var t = _auditWriter.WriteAsync(evt, CancellationToken.None);
    if (!t.IsCompleted) _ = t.ContinueWith(x => _logger.LogWarning(x.Exception, "Flow trace write failed"),
        TaskContinuationOptions.OnlyOnFaulted);
}
private static readonly JsonSerializerOptions TraceJson = new(JsonSerializerDefaults.Web);

SetAttribute: Stopwatch around the Ask; pass Trace: _isTraced on the command; after the reply EmitTrace(AuditKind.AttributeWrite, response.Success ? Delivered : Failed, attributeName, started, ms, input: JsonSerializer.Serialize(new { attributeName, value }), output: null, error: response.ErrorMessage, extra: new ScriptTraceExtra { Writes = [new(attributeName, value, response.Success, response.ErrorMessage, IsStatic: true)] }). (IsStatic is unknown to the context — set false and name the field IsStaticHint… simpler: drop IsStatic from AttributeWriteDetail in Task 4 if you prefer; either way keep the tests consistent.)
WriteBatchAndWait: emit one AttributeWrite for the batch (all encodedValues + the trigger) after the batch Ask, then the wait emits through WaitInternal.
WaitInternal: emit AttributeWait with WaitOutcome = response.Matched ? "satisfied" : response.TimedOut ? "timeout" : "error", ObservedValue = AttributeValueCodec.Encode(response.Value), WaitTimeoutMs, status Matched → Delivered, else Failed.
CallScript: Trace: _isTraced on ScriptCallRequest. The inline-shared nested context construction (~line 630): pass isTraced: _isTraced, traceCollector: _traceCollector.
Payload caps: reuse the existing 8 KB truncation helper the context uses for RequestSummary (grep Truncate/MaxPayload in the file) so trace rows obey §9.

Step 3: Run the new tests + the whole SiteRuntime.Tests project (existing tests must stay green). Commit — git commit -am "feat(site-runtime): ScriptRuntimeContext emits ScriptTrace AttributeWrite/AttributeWait rows and inherits the Trace flag"

Task 7: `ScriptRunLauncher` + `ScriptActor` — traced runs emit `ScriptRun`, return `ExecutionId` + events, value-change inheritance

Classification: high-risk (actor model) Estimated implement time: ~5 min Parallelizable with: Task 8

Files:

Modify: src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Scripts/ScriptRunLauncher.cs (LaunchScript signature ~76, RunScriptAsync ~228–470)
Modify: src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/ScriptActor.cs (HandleScriptCallRequest, HandleAttributeValueChanged ~283, TrySpawnExecution, SpawnExecution ~520)
Test: tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Actors/ScriptActorTraceTests.cs

Step 1: Failing tests (model on AlarmCascadeParentExecutionTests for fixture; ScriptActor construction — copy from an existing ScriptActor test in the project):

Call_WithTrace_ReplyCarriesExecutionId_AndAScriptRunEvent — Tell(new ScriptCallRequest("s", null, 0, "cid", Trace: true)); ExpectMsg<ScriptCallResult>; ExecutionId != null; TraceEvents contains one event with Kind == "ScriptRun", Status == "Delivered", RequestSummary = params JSON, ResponseSummary = return JSON.
Call_WithoutTrace_ReplyHasNoTraceEvents (and ExecutionId still set — cheap and useful for correlation).
ValueChangeTrigger_TracedChange_RunParentsToSourceExecutionId — script with valuechange trigger on Status; Tell(new AttributeValueChanged("Pump1","Status","Status","x","Good",now, SourceExecutionId: g, Trace: true)); the launched run's ScriptRun event (capture via IAuditWriter substitute registered in the IServiceProvider passed to the actor) has ParentExecutionId == g and Extra.triggerSource == "static-write"? — the actor cannot know static vs echo; set TriggerSource to "value-change" and TriggerAttribute/TriggerValue.
ValueChangeTrigger_UntracedChange_RunStaysRoot (pin).

Step 2: Implement

LaunchScript(..., ScriptExecutionScheduler? scheduler, bool trace = false, ScriptTraceExtra? triggerExtra = null) → threaded into RunScriptAsync.
In RunScriptAsync: var collector = trace ? new FlowTraceCollector() : null; pass isTraced: trace, traceCollector: collector to the context. After success: emit ScriptRun (Delivered) via the same factory call shape as Task 6 but from the launcher (it has auditWriter, siteId, sourceNode, context.ExecutionId, parentExecutionId; write it through collector first so it is included, then auditWriter); reply new ScriptCallResult(correlationId, true, state.ReturnValue, null, context.ExecutionId, collector?.Snapshot()). Timeout/failure catches: same with Failed + error message. Put the emission in a small private static EmitScriptRun(...) to avoid triplication. Duration = runStopwatch.ElapsedMilliseconds; Input = params JSON (8 KB cap); Output = return value JSON.
ScriptActor.HandleScriptCallRequest → SpawnExecution(..., request.ParentExecutionId, request.Trace); SpawnExecution gains bool trace = false, ScriptTraceExtra? triggerExtra = null → passes to LaunchScript.
HandleAttributeValueChanged: for the ValueChangeTriggerConfig branch call TrySpawnExecution(null, changed.Trace ? changed.SourceExecutionId : null, changed.Trace, changed.Trace ? new ScriptTraceExtra { TriggerKind = "valuechange", TriggerAttribute = changed.AttributeName, TriggerValue = AttributeValueCodec.Encode(changed.Value), TriggerSource = "value-change" } : null). TrySpawnExecution(parameters, Guid? parentExecutionId = null, bool trace = false, ScriptTraceExtra? triggerExtra = null) → SpawnExecution(parameters, 0, ActorRefs.NoSender!, Guid.NewGuid().ToString(), parentExecutionId, trace, triggerExtra). Other trigger kinds keep calling TrySpawnExecution(null) unchanged (roots by design — CLAUDE.md).

Step 3: Run SiteRuntime.Tests fully. Commit — git commit -am "feat(site-runtime): traced runs emit ScriptRun rows, return ExecutionId + trace events; traced value-change runs inherit the parent"

Task 8: `InstanceActor` — trace arm for data-sourced writes; `Trace` on published changes

Classification: high-risk (actor model) Estimated implement time: ~5 min Parallelizable with: Task 4, Task 6, Task 7

Files:

Modify: src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/InstanceActor.cs (HandleSetStaticAttributeCore ~538, HandleSetDataAttribute ~693, HandleTagValueUpdate ~1025, fields ~39–193)
Modify: src/ZB.MOM.WW.ScadaBridge.SiteRuntime/SiteRuntimeOptions.cs, SiteRuntimeOptionsValidator.cs
Test: tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Actors/InstanceActorTraceArmTests.cs, extend tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/SiteRuntimeOptionsValidatorTests.cs

Step 1: Failing tests (fixture = InstanceActorWaitForAttributeTests.CreateInstanceActor pattern with a DCL probe; config with one data-sourced attribute Processed bound to tag ns=3;s=Recipe.Processed on connection PLC, plus a child probe registered as an all-change listener — or simply subscribe the test probe through the existing SubscribeDebugView/stream path; simplest: pass a SiteStreamManager substitute and capture PublishAttributeValueChanged):

TracedDataWrite_ThenEcho_PublishesChangeStampedWithSourceAndTrace — Tell(SetStaticAttributeCommand(..., SourceExecutionId: g, Trace: true)); DCL probe ExpectMsg<WriteTagRequest> and reply WriteTagResponse(success); then Tell(new TagValueUpdate("PLC", tag, true, QualityCode.Good, now)); captured AttributeValueChanged.SourceExecutionId == g && Trace.
UntracedDataWrite_ThenEcho_PublishesUnstampedChange (pin: production unchanged).
Arm_IsConsumedOnce — second echo is unstamped.
Arm_Expired_IsIgnored — construct with TraceArmWindowSeconds = 0… use an injected clock? Keep simple: options TraceArmWindowSeconds and a Func<DateTimeOffset>-free approach: test with window 1 and Thread.Sleep(1100) before the echo (acceptable in TestKit tests; mark [Trait("Category","Slow")]).
StaticWrite_Traced_PublishesTraceFlag — HandleSetStaticAttributeCore path.
Validator: TraceArmWindowSeconds must be > 0.

Step 2: Implement

SiteRuntimeOptions.TraceArmWindowSeconds { get; set; } = 30; (doc: design §5.3). Validator rule.
Fields: private readonly Dictionary<string, (Guid SourceExecutionId, DateTimeOffset ExpiresAt)> _traceArms = new();
HandleSetDataAttribute: before the DCL Ask, if (command.Trace && command.SourceExecutionId is { } src) _traceArms[attributeName] = (src, DateTimeOffset.UtcNow.AddSeconds(_options.TraceArmWindowSeconds));
HandleTagValueUpdate: for each attrName, resolve Guid? src = null; var traced = false; if (_traceArms.Remove(attrName, out var arm) && arm.ExpiresAt > update.Timestamp) { src = arm.SourceExecutionId; traced = true; } and pass SourceExecutionId: src, Trace: traced on every new AttributeValueChanged(...) in that loop (three sites).
HandleSetStaticAttributeCore: Trace: command.Trace on the published change.
No sweep needed: arms are removed on the next echo; a never-echoed arm is overwritten by the next traced write or removed when the actor stops (bounded by attribute count).

Step 3: Run SiteRuntime.Tests fully. Commit — git commit -am "feat(site-runtime): InstanceActor trace arm stamps the device echo of a traced write; Trace rides on published changes"

Task 9: `DeploymentManagerActor` routing — pass `Trace`, return child id + events

Classification: standard Estimated implement time: ~3 min Parallelizable with: Task 10, Task 11

Files:

Modify: src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/DeploymentManagerActor.cs (RouteInboundApiCall ~2866, RouteInboundApiSetAttributes ~3081)
Test: tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Actors/DeploymentManagerRouteTraceTests.cs (find an existing test that drives RouteInboundApiCall with a probe instance actor and copy its fixture; if none exists, exercise the two private methods' behaviour through a TestActorRef<DeploymentManagerActor> — check how _instanceActors is populated in existing tests)

Implement: ScriptCallRequest(..., ParentExecutionId: request.ParentExecutionId, Trace: request.Trace); map new RouteToCallResponse(request.CorrelationId, result.Success, NormalizeRoutedReturnValue(result.ReturnValue), result.ErrorMessage, DateTimeOffset.UtcNow, result.ExecutionId, result.TraceEvents); SetStaticAttributeCommand(..., SourceExecutionId: request.ParentExecutionId, Trace: request.Trace). Tests assert the forwarded ScriptCallRequest.Trace and the mapped ChildExecutionId/TraceEvents. Commit — git commit -am "feat(site-runtime): route handlers forward Trace and return child ExecutionId + trace events"

Task 10: Wire — `site_command.proto` trace fields, mapper, golden samples

Classification: high-risk (wire contract) Estimated implement time: ~5 min Parallelizable with: Task 9, Task 11

Files:

Modify: src/ZB.MOM.WW.ScadaBridge.Communication/Protos/site_command.proto (add import "Protos/sitestream.proto"; if AuditEventDto is not visible — central_control.proto already imports sitestream the same way; copy its import line verbatim)
Regenerate: src/ZB.MOM.WW.ScadaBridge.Communication/SiteCommandGrpc/*.cs via bash docker/regen-proto.sh sitecommand
Modify: src/ZB.MOM.WW.ScadaBridge.Communication/Grpc/SiteCommandDtoMapper.cs (ToProto/FromProto for RouteToCallRequest, RouteToCallResponse, RouteToSetAttributesRequest)
Test: tests/ZB.MOM.WW.ScadaBridge.Communication.Tests/SiteCommandDtoMapperGoldenTests.cs + the SiteCommandSamples it reads (add traced samples), new tests/.../SiteCommandDtoMapperTraceTests.cs

Proto (additive, never reuse numbers):

message RouteToCallRequestDto { ...existing 1-6...; bool trace = 7; }
message RouteToCallResponseDto { ...existing 1-5...; string child_execution_id = 6; repeated AuditEventDto trace_events = 7; }
message RouteToSetAttributesRequestDto { ...existing 1-5...; bool trace = 6; }

Mapper: Trace = request.Trace / dto.Trace; ChildExecutionId = response.ChildExecutionId?.ToString("D") ?? ""; TraceEvents.AddRange(response.TraceEvents?.Select(AuditEventDtoMapper.ToDto) ?? []); reverse with Guid.TryParse and dto.TraceEvents.Count == 0 ? null : dto.TraceEvents.Select(AuditEventDtoMapper.FromDto).ToList() (an old site sends none → null, which the UI labels "site not traced").

Tests: round-trip with two trace events preserves ChildExecutionId and event Action/DetailsJson; a DTO with no trace fields maps to Trace=false/nulls; golden samples updated (the golden test enforces every record field is covered — follow its failure message).

Also: confirm docker/regen-proto.sh restored the csproj (git diff src/ZB.MOM.WW.ScadaBridge.Communication/*.csproj must be empty). Commit — git commit -am "feat(wire): route_to_call trace flag + child_execution_id + trace_events; route_to_set_attributes trace (additive proto + mapper)"

Task 11: Wire — `sitestream.proto` `source_execution_id`/`trace` on attribute updates + `PullTraceEvents` RPC; site SQLite read by execution ids

Classification: high-risk (wire + storage) Estimated implement time: ~5 min Parallelizable with: Task 9, Task 10

Files:

Modify: src/ZB.MOM.WW.ScadaBridge.Communication/Protos/sitestream.proto
Regenerate: SiteStreamGrpc/*.cs (and CentralControlGrpc/*.cs if the script requires — run bash docker/regen-proto.sh sitestream; if central_control's generated code references changed descriptors, run all)
Modify: src/ZB.MOM.WW.ScadaBridge.Communication/Actors/StreamRelayActor.cs (~43), src/ZB.MOM.WW.ScadaBridge.Communication/Grpc/SiteStreamGrpcClient.cs (~554), src/ZB.MOM.WW.ScadaBridge.Communication/Grpc/SiteStreamGrpcServer.cs (new override next to PullAuditEvents)
Modify: src/ZB.MOM.WW.ScadaBridge.Commons/Interfaces/Services/ISiteAuditQueue.cs, src/ZB.MOM.WW.ScadaBridge.AuditLog/Site/SqliteAuditWriter.cs (schema block ~233–266 + new read)
Create: src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/IPullTraceEventsClient.cs, GrpcPullTraceEventsClient.cs (model on GrpcPullAuditEventsClient incl. its invoker/PSK plumbing), register in src/ZB.MOM.WW.ScadaBridge.AuditLog/ServiceCollectionExtensions.cs beside IPullAuditEventsClient
Test: tests/ZB.MOM.WW.ScadaBridge.Communication.Tests/SiteStreamPullTraceEventsTests.cs (model on SiteStreamPullAuditEventsTests), tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Site/SqliteAuditWriterReadByExecutionTests.cs, tests/ZB.MOM.WW.ScadaBridge.Communication.Tests/StreamRelayTraceStampTests.cs

Proto:

message AttributeValueUpdate { ...1-6...; string source_execution_id = 7; bool trace = 8; }
service SiteStreamService { ...; rpc PullTraceEvents(PullTraceEventsRequest) returns (PullTraceEventsResponse); }
message PullTraceEventsRequest { repeated string execution_ids = 1; google.protobuf.Timestamp since_utc = 2; int32 max_events = 3; }
message PullTraceEventsResponse { repeated AuditEventDto events = 1; }

Site SQLite: ISiteAuditQueue.ReadByExecutionIdsAsync(IReadOnlyCollection<Guid> executionIds, DateTime sinceUtc, int limit, CancellationToken ct) — SQL:

SELECT EventId, OccurredAtUtc, Actor, Action, Outcome, Category, Target, SourceNode, CorrelationId, DetailsJson
FROM audit_event
WHERE OccurredAtUtc >= @since
  AND (json_extract(DetailsJson,'$.ExecutionId') IN (...) OR json_extract(DetailsJson,'$.ParentExecutionId') IN (...))
ORDER BY OccurredAtUtc, EventId LIMIT @limit;

(check the exact property names AuditDetailsCodec writes — camelCase vs Pascal — by reading one serialized row in a test; the central computed column uses '$.kind', so likely camelCase '$.executionId'). Add CREATE INDEX IF NOT EXISTS IX_audit_event_occurred ON audit_event (OccurredAtUtc); to the schema block (idempotent, additive). Parameterize the IN lists (@e0..@eN), cap ids at 64.

Server: PullTraceEvents → validate max_events > 0 and ≤ 2000 → queue read → AuditEventDtoMapper.ToDto. The ControlPlaneAuthInterceptor already gates the whole /sitestream.SiteStreamService/ prefix — add a test asserting the new method name is under that prefix (a one-liner on SiteStreamService.Descriptor.FullName).

Relay + client: SourceExecutionId = msg.SourceExecutionId?.ToString("D") ?? "", Trace = msg.Trace on the way out; parse back into the domain record on the way in.

Central client: Task<IReadOnlyList<AuditEvent>> PullAsync(string siteId, string nodeAddress…) following GrpcPullAuditEventsClient's constructor/invoker shape exactly (it resolves the site PSK via ISitePskProvider).

Commit — git commit -am "feat(wire): attribute updates carry source_execution_id/trace; PullTraceEvents RPC + site read-by-execution-ids"

Task 12: `RouteHelper` — always-on `RouteCall` audit row, `WithTrace`, inline trace sink

Classification: standard Estimated implement time: ~5 min Parallelizable with: Task 13, Task 14

Files:

Modify: src/ZB.MOM.WW.ScadaBridge.InboundApi/RouteHelper.cs (RouteHelper ctor/With* ~36–125; RouteTarget ctor ~131; Call ~178; SetAttributes ~341)
Test: tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/RouteHelperTests.cs (extend) + RouteHelperTraceTests.cs

Implement:

Public ctor becomes RouteHelper(IInstanceLocator, IInstanceRouter, ICentralAuditWriter? auditWriter = null, INodeIdentityProvider? nodeIdentity = null) — DI (AddScoped<RouteHelper>()) resolves both when registered on central; existing new RouteHelper(locator, router) test call sites compile.
Private state adds bool _trace, Action<AuditEvent>? _traceSink, plus WithTrace(bool) and WithTraceSink(Action<AuditEvent>?) (immutable-copy style like the existing With*). RouteTarget receives them.
Call: Stopwatch; RouteToCallRequest(..., _parentExecutionId, Trace: _trace); after the response (or exception) build ScadaBridgeAuditEventFactory.Create(AuditChannel.ScriptTrace, AuditKind.RouteCall, success ? Delivered : Failed, actor: "inbound-api", target: $"{_instanceCode}.{scriptName}", sourceNode: node, executionId: _parentExecutionId, parentExecutionId: null, durationMs, errorMessage, requestSummary: args JSON (8 KB cap), responseSummary: return JSON (8 KB), extra: ScriptTraceExtra { ChildExecutionId = response?.ChildExecutionId }). Write via _auditWriter fire-and-forget (never throws into the script); also _traceSink?.Invoke(evt) and foreach (var e in response.TraceEvents ?? []) _traceSink?.Invoke(e). Always-on: the row is emitted regardless of _trace when a writer is present.
SetAttributes: pass Trace: _trace.

Tests: RouteCall row written with ExecutionId == parent, Extra.childExecutionId from the reply; sink receives RouteCall + the reply's TraceEvents; Trace propagates on both requests; no writer → no throw; failure path writes Failed with the message.

Commit — git commit -am "feat(inbound-api): RouteHelper writes an always-on RouteCall audit row, propagates Trace, exposes an inline trace sink"

Task 13: `InboundScriptExecutor.ExecuteAsync(trace)` + endpoint honours `TraceAllCalls`

Classification: small Estimated implement time: ~3 min Parallelizable with: Task 12, Task 14

Files:

Modify: src/ZB.MOM.WW.ScadaBridge.InboundApi/InboundScriptExecutor.cs (~324 signature, ~380 context)
Modify: src/ZB.MOM.WW.ScadaBridge.InboundApi/EndpointExtensions.cs (~276)
Test: tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/InboundScriptExecutorTests.cs (add one test)

Implement: ExecuteAsync(..., Guid? parentExecutionId = null, bool trace = false) → route.WithDeadline(...).WithRequestAborted(...).WithParentExecutionId(parentExecutionId).WithTrace(trace); endpoint passes trace: method.TraceAllCalls. Test: a registered handler that captures ctx.Route and asserts the built RouteToCallRequest.Trace via a substitute router (use WithTrace(true) end-to-end). Commit — git commit -am "feat(inbound-api): executor threads the trace flag; endpoint honours ApiMethod.TraceAllCalls"

Task 14: `IAuditLogRepository.GetTraceEventsAsync(rootExecutionId)`

Classification: standard Estimated implement time: ~4 min Parallelizable with: Task 12, Task 13

Files:

Modify: src/ZB.MOM.WW.ScadaBridge.Commons/Interfaces/Repositories/IAuditLogRepository.cs
Modify: src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/AuditLogRepository.cs (beside GetExecutionTreeAsync)
Test: tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Integration/FlowTraceReplayQueryTests.cs (uses the Integration infrastructure the sibling ExecutionIdCorrelationTests uses — same fixture/collection attributes)

Implement: reuse the upward walk + Chain CTE from GetExecutionTreeAsync (refactor the chain SQL into a private ChainCteSql constant used by both), then SELECT <all columns> FROM dbo.AuditLog WHERE ExecutionId IN (SELECT ExecutionId FROM ChainIds) AND (Category = 'ScriptTrace' OR (Category = 'ApiInbound' AND Kind = 'InboundRequest')) ORDER BY OccurredAtUtc, EventId mapped with the repository's existing row→AuditEvent mapper. Test: insert inbound + RouteCall + two site rows chained by parent ids, plus an unrelated row → exactly 4 back, ordered. Commit — git commit -am "feat(audit): GetTraceEventsAsync returns the ScriptTrace chain for Replay"

Task 15: Management commands — `FlowDebugRunMethodCommand`, `FlowDebugRunValueChangeCommand`, `FlowDebugReplayCommand`

Classification: high-risk (security, cross-cluster) Estimated implement time: ~5 min (split 15a/15b if it runs long) Parallelizable with: Task 16 (after the records exist), Task 17, Task 18

Files:

Create: src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Management/FlowDebugCommands.cs
Modify: src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementActor.cs (GetRequiredRoles → DeployerOnly; DispatchCommand; three handlers), src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementEndpoints.cs (LongRunningCommands += the two run commands)
Create: src/ZB.MOM.WW.ScadaBridge.ManagementService/FlowDebug/FlowDebugRunner.cs (the handlers delegate here so it is unit-testable without the actor)
Modify: src/ZB.MOM.WW.ScadaBridge.ManagementService/ServiceCollectionExtensions.cs (AddSingleton<FlowDebugRunner>())
Test: tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests/FlowDebugRunnerTests.cs, extend the existing role-gate test that enumerates GetRequiredRoles

Records (namespace ZB.MOM.WW.ScadaBridge.Commons.Messages.Management — ManagementCommandRegistry auto-registers by simple name):

public record FlowDebugRunMethodCommand(int ApiMethodId, JsonElement? Parameters);
public record FlowDebugRunValueChangeCommand(int InstanceId, string AttributeName, string Value, int WaitSeconds = 45);
public record FlowDebugReplayCommand(Guid ExecutionId);
public record FlowDebugRunResult(
    Guid RootExecutionId, bool Completed, string? Error, string? ErrorCode,
    IReadOnlyList<AuditEvent> Events, bool Truncated, DateTimeOffset StartedAtUtc, DateTimeOffset FinishedAtUtc);

FlowDebugRunner (ctor: IServiceProvider, ILogger):

RunMethodAsync(cmd, user, ct):
1. IActiveNodeGate gate = sp.GetService<IActiveNodeGate>(); if gate is not null && !gate.IsActive → throw new ManagementCommandException("Live runs execute on the ACTIVE central node only (Component-InboundAPI: method scripts and Route.To never run on a standby). Open the Central UI or point the CLI at the active node.") { Code = "STANDBY_NODE" } (check how ManagementCommandException carries a code; add an optional code if it does not).
2. Load the ApiMethod (IInboundApiRepository), SchemaRefResolver.BuildAsync + ParameterValidator.Validate(cmd.Parameters, method.ParameterDefinitions, resolveRef) → 400-style error on invalid.
3. var root = Guid.NewGuid(); var events = new ConcurrentQueue<AuditEvent>();
4. Synthesize and write the inbound row via ICentralAuditWriter: Create(ApiInbound, InboundRequest, Submitted → later Delivered/Failed, actor: user.Username, target: method.Name, executionId: root, requestSummary: params JSON, extra: {"source":"flow-debugger"}) — write it once at the end with the final status and duration (one row, like the middleware).
5. var route = sp.GetRequiredService<RouteHelper>().WithTraceSink(events.Enqueue) (create a DI scope for scoped services); executor.ExecuteAsync(method, parameters, route, timeout, ct, parentExecutionId: root, trace: true).
6. Return FlowDebugRunResult(root, result.Success, result.ErrorMessage, result.ErrorCode, [inboundRow, ..events ordered by OccurredAtUtc], truncated: false, …).
RunValueChangeAsync(cmd, user, ct): load instance + site (ITemplateEngineRepository, ISiteRepository), EnforceSiteScope; root = Guid.NewGuid(); CommunicationService.RouteToSetAttributesAsync(site.SiteIdentifier, new RouteToSetAttributesRequest(cid, instance.UniqueName, {attr: value}, now, ParentExecutionId: root, Trace: true)); then poll IPullTraceEventsClient.PullAsync(site, [root], since: started, max: 500) every 1 s until a ScriptRun row with ParentExecutionId == root exists and no new rows arrived in the last 2 s, or WaitSeconds elapses (then Completed=false, ErrorCode="NO_TRIGGER_WITHIN_WINDOW"). Prepend a synthesized AttributeWrite-kind central row (actor: user, target: attr, executionId: root, extra.source = flow-debugger) so the graph has a root node. (Pulling needs the site's gRPC node address — copy how SiteAuditReconciliationActor picks NodeA/NodeB for IPullAuditEventsClient.)
ReplayAsync(cmd): IAuditLogRepository.GetTraceEventsAsync(cmd.ExecutionId) → root = the topmost row's ExecutionId; Completed=true.

Tests (runner, NSubstitute): standby gate refuses with STANDBY_NODE; invalid parameters fail before any route call; happy path returns inbound row + sink events ordered; replay returns repository rows; role gate for all three = Deployer. Commit — git commit -am "feat(management): FlowDebug run-method / run-value-change / replay commands (Deployer, active-node gated, inline trace)"

Task 16: CLI `flow-debug` command group

Classification: small Estimated implement time: ~4 min Parallelizable with: Task 17, Task 18

Files:

Create: src/ZB.MOM.WW.ScadaBridge.CLI/Commands/FlowDebugCommands.cs
Modify: src/ZB.MOM.WW.ScadaBridge.CLI/Program.cs (register), src/ZB.MOM.WW.ScadaBridge.CLI/README.md (new flow-debug section modelled on deploy)
Test: tests/ZB.MOM.WW.ScadaBridge.CLI.Tests/FlowDebugCommandsTests.cs (model on an existing command test that asserts the built management command)

Commands: flow-debug run-method --method-id <int> [--params <json>], flow-debug run-value-change --instance-id <int> --attribute <name> --value <string> [--wait-seconds 45], flow-debug replay --execution-id <guid>; all via CommandHelpers.ExecuteCommandAsync(..., timeout: TimeSpan.FromMinutes(2)); add a tableProjector that prints one line per event: occurred | kind | lane-ish target | duration | status (JSON format stays the raw FlowDebugRunResult). Commit — git commit -am "feat(cli): flow-debug run-method|run-value-change|replay"

Task 17: `FlowShapePredictor` (Script Analysis)

Classification: standard Estimated implement time: ~5 min Parallelizable with: Task 15, Task 16, Task 18

Files:

Create: src/ZB.MOM.WW.ScadaBridge.ScriptAnalysis/FlowShapePredictor.cs
Test: tests/ZB.MOM.WW.ScadaBridge.ScriptAnalysis.Tests/FlowShapePredictorTests.cs

API:

public sealed record PredictedNode(string Id, FlowStepKind Kind, FlowLane Lane, string Title, bool InputDependent, string? Note);
public sealed record PredictedEdge(string From, string To, bool CrossesBoundary);
public sealed record PredictedFlow(IReadOnlyList<PredictedNode> Nodes, IReadOnlyList<PredictedEdge> Edges);

public static class FlowShapePredictor
{
    /// <param name="resolveRouteScript">(instanceCode, scriptName) → the site script body when the code is a constant
    /// and central can resolve instance → template → script; null otherwise. Keeps this project DB-free.</param>
    public static PredictedFlow PredictInboundMethod(string methodName, string methodScript,
        Func<string, string, string?> resolveRouteScript, int maxDepth = 4);
    public static PredictedFlow PredictSiteScript(string scriptName, string scriptBody, int maxDepth = 4);
}

Walker (CSharpSyntaxWalker over CSharpSyntaxTree.ParseText(code, new CSharpParseOptions(kind: SourceCodeKind.Script))): visit InvocationExpressionSyntax, match the member-access chains by text with whitespace stripped, exactly as ScriptTrustValidator.HardeningWalker does: Route.To(X).Call(Y,…) / .SetAttributes / .WaitForAttribute / .GetAttributes; Scripts.Call(Y); …Attributes.WriteBatchAndWaitAsync(…), …Attributes.WaitAsync(…), …Attributes.WaitForAsync(…), …Attributes.SetAsync(…), ExternalSystem.Call/CachedCall, Database.*, Notify.Send; plus AssignmentExpressionSyntax whose left side is an ElementAccessExpressionSyntax on …Attributes[...]. Constant string arguments (LiteralExpressionSyntax string) resolve; anything else → InputDependent = true, title Route.To(<expr>). An IfStatementSyntax containing a ReturnStatementSyntax that precedes the first routed call becomes a Note = "early return: <condition text>" on the edge into the next node. Nodes get ids n1…. PredictInboundMethod adds InboundRequest (Central) first and Response (Central) last, and for a constant route target recurses one hop into resolveRouteScript(...), adding its nodes in the Site/Tags lanes. Depth cap maxDepth.

Tests: Delmia method script + site script (verbatim from docs/api-specs/Delmia-Recipe-Download-API.md §4) → 8 nodes in the right lanes, route node InputDependent (target is code), site nodes present because the test resolver returns the site script for any code, early-return note present; a constant Route.To("Z28061Sim") is not input-dependent; a script with no recognised call yields only request+response; recursion stops at maxDepth; malformed code never throws (returns request+response with Note = "parse error"). Commit — git commit -am "feat(script-analysis): FlowShapePredictor — predicted flow graph from script syntax"

Task 18: Central UI service — `IFlowDebugService` / `FlowDebugService`

Classification: standard Estimated implement time: ~5 min Parallelizable with: Task 15 (after records), Task 16, Task 17

Files:

Create: src/ZB.MOM.WW.ScadaBridge.CentralUI/Services/IFlowDebugService.cs, FlowDebugService.cs
Modify: src/ZB.MOM.WW.ScadaBridge.CentralUI/ServiceCollectionExtensions.cs (AddScoped<IFlowDebugService, FlowDebugService>() — find where IDataConnectionMoveService is registered and add beside it)
Test: tests/ZB.MOM.WW.ScadaBridge.CentralUI.Tests/Services/FlowDebugServiceTests.cs

Interface:

public interface IFlowDebugService
{
    Task<PredictedFlow> PredictMethodAsync(int apiMethodId, CancellationToken ct = default);
    Task<PredictedFlow> PredictValueChangeAsync(int instanceId, string attributeName, CancellationToken ct = default);
    Task<FlowDebugOutcome> RunMethodAsync(int apiMethodId, JsonElement? parameters, CancellationToken ct = default);
    Task<FlowDebugOutcome> RunValueChangeAsync(int instanceId, string attributeName, string value, CancellationToken ct = default);
    Task<FlowDebugOutcome> ReplayAsync(Guid executionId, CancellationToken ct = default);
}
public sealed record FlowDebugOutcome(FlowGraph? Graph, FlowDebugRunResult? Raw, string? Error, string? ErrorCode);

Implementation: ManagementActorHolder + AuthenticationStateProvider exactly as DataConnectionMoveService (copy SendAsync/BuildAuthenticatedUserAsync; Ask timeout 2 min for the run commands); map ManagementSuccess payload (FlowDebugRunResult) → FlowGraphBuilder.Build(result.Events, result.Truncated); ManagementUnauthorized/ManagementError → Error/ErrorCode. Predict: load method/instance via repositories (IInboundApiRepository, ITemplateEngineRepository) and give FlowShapePredictor a resolver that maps instanceCode → Instance.UniqueName → template → TemplateScript(name).Code (find the repository call the template editor uses to list scripts; IFlatteningPipeline.PreviewAsync is heavier than needed — use the template's own script list, inheritance-resolved if ITemplateInheritanceQueryService offers scripts, else the template's direct scripts with a Note = "inherited scripts not resolved").

Tests: holder null → SERVICE_UNAVAILABLE; ManagementError maps to ErrorCode; success maps to a graph with the right step count; predict wires the resolver (substitute repos). Commit — git commit -am "feat(central-ui): FlowDebugService — management-command bridge + predicted graph"

Task 19: `FlowGraphView` — custom SVG swimlane renderer

Classification: standard Estimated implement time: ~5 min Parallelizable with: Task 18, Task 20

Files:

Create: src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Shared/FlowGraphView.razor, FlowGraphView.razor.cs, FlowGraphView.razor.css
Test: tests/ZB.MOM.WW.ScadaBridge.CentralUI.Tests/Shared/FlowGraphViewTests.cs

Contract: parameters FlowGraph? Graph, PredictedFlow? Predicted, Guid? SelectedEventId, EventCallback<FlowStep> OnSelect, double Zoom = 1.0. Layout (pure, in the code-behind, unit-tested): three fixed lanes of 110 px (labels Central / Site / Tags from the steps' SiteId/Instance), columns assigned by topological order (a step's column = 1 + max column of its predecessors; siblings in the same lane share a column only if independent), node 150×78 at x = 130 + col*170, y = 16 + lane*110; connectors as orthogonal <path> with marker-end arrows; CrossesBoundary edges use the accent colour; External edges dashed; predicted-only nodes dashed; predicted nodes not present in the real graph greyed. Node content = kind eyebrow + duration, title, subtitle (from FlowStep). Mirror the mockup's CSS values (see the design canvas): borders #d2d2cb, selected 2px #2f5fd0 on #eef2fc, eyebrow 0.6rem uppercase, mono duration — but use the kit's CSS variables (var(--rule-strong), var(--accent), var(--active-bg)) so dark mode follows. InvariantCulture for every coordinate string (see KpiTrendChart). Click a node → OnSelect. role="img" + aria-label summary; each node a <g role="button" tabindex="0"> with Enter/Space selection.

Tests: layout assigns Delmia-shaped graph to expected columns/lanes; selected node has the selected class; clicking raises OnSelect with the step; empty graph renders the lanes and a "No steps" note; coordinates are culture-invariant under de-DE. Commit — git commit -am "feat(central-ui): FlowGraphView SVG swimlane renderer"

Classification: standard Estimated implement time: ~5 min ×2 (split as 20a page skeleton + predicted graph + inputs; 20b run/replay + detail panel) Parallelizable with: Task 19 (20a can start with a stub FlowGraphView)

Files:

Create: src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Pages/Deployment/FlowDebugger.razor, FlowDebugger.razor.cs, FlowDebugger.razor.css
Create: src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Deployment/FlowStepDetail.razor (tabs: Input · Output · Attribute writes · Timing; Where this ran column; drill-ins Open in Debug View → /deployment/debug-view?instanceId=… if that page reads a query param — check DebugView.razor; Audit rows (n) → /audit/log?executionId=…)
Modify: src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Layout/NavMenu.razor (Deployment section, after Debug View: <NavRailItem Href="/deployment/flow-debugger" Text="Flow Debugger" />)
Test: tests/ZB.MOM.WW.ScadaBridge.CentralUI.Tests/Deployment/FlowDebuggerPageTests.cs (bUnit, fixture copied from DebugViewDisposalTests.RegisterServices + Substitute.For<IFlowDebugService>())

Page (@page "/deployment/flow-debugger", [Authorize(Policy = AuthorizationPolicies.RequireDeployment)]): query params ?methodId=, ?instanceId=&attribute=, ?executionId= (replay drill-in). State machine: Idle → Predicted → Running → Completed|Failed|TimedOut. Toolbar/status/flow/detail exactly as the design §4 and the mockup; mode toggle Live | Replay (Replay shows the ExecutionId box instead of inputs). Inputs form generated from ApiMethod.ParameterDefinitions (reuse the JSON-schema parsing ParameterValidator uses — if it is internal, expose a small ParameterDefinitionReader in Commons or parse the schema's properties/required directly with System.Text.Json); values collected into a JsonElement object. Value-change entry: Site → Instance → Attribute pickers (instances from ITemplateEngineRepository filtered by SiteScopeService), current value shown via IInstanceSnapshotClient if cheap, else omitted. Persist last entry/inputs in localStorage via IJSRuntime (same pattern Debug View uses). Run button disabled while running; a STANDBY_NODE error renders as an inline alert with the message verbatim. After a run, the predicted graph is overlaid: predicted nodes with no real counterpart render greyed (matching by kind + title). Selecting a node fills FlowStepDetail.

Tests: renders with Deployer claim; predicted graph appears after choosing a method (service substitute returns a 3-node prediction); Run calls RunMethodAsync with the typed inputs and renders the returned graph's step count; STANDBY_NODE shows the alert; ?executionId= triggers ReplayAsync on first render; a graph whose site lane is empty and whose route call has no ChildExecutionId shows the "site not traced" label. Commit — git commit -am "feat(central-ui): Flow Debugger page (predict, live run, replay, step detail) + nav entry"

Task 21: Drill-ins and the `TraceAllCalls` checkbox

Classification: small Estimated implement time: ~4 min Parallelizable with: Task 22

Files:

Modify: src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Audit/AuditEventDetail.razor (+ .razor.cs): button Open in Flow Debugger (data-test="open-flow-debugger") beside View execution chain → /deployment/flow-debugger?executionId={exec}; rendered when ExecutionId is non-null.
Modify: src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Pages/Audit/ExecutionTreePage.razor: same button next to View this execution in the Audit Log.
Modify: src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Pages/Design/ApiMethodForm.razor: under the Timeout field a form-check "Trace all calls (Flow Debugger)" bound to _traceAllCalls, help text "Adds ~7 audit rows per call; for low-volume integration methods", saved onto _existing.TraceAllCalls / the create path.
Test: tests/ZB.MOM.WW.ScadaBridge.CentralUI.Tests/Audit/FlowDebuggerDrillinTests.cs, extend the existing ApiMethodForm test file if one exists (grep ApiMethodForm in tests).

Commit — git commit -am "feat(central-ui): Flow Debugger drill-ins from Audit Log + Execution Chain; TraceAllCalls checkbox on the method form"

Task 22: Rig — retention override, redeploy, verification step H

Classification: standard Estimated implement time: ~5 min (plus the rig rebuild wall time — run bash docker/deploy.sh in the background) Parallelizable with: Task 21

Files:

Modify: docker/central-node-a/appsettings.Central.json, docker/central-node-b/appsettings.Central.json — add "AuditLog": { "PerChannelRetentionDays": { "ScriptTrace": 30 } } at the top level (the file's root key today is ScadaBridge only; AuditLog is a sibling root section — confirm by grepping GetSection("AuditLog") in the Host, or place it wherever the existing rig sites configure AuditLog:SiteWriter). Validator requires 30 ≤ value ≤ global RetentionDays (365 default) — 30 is the floor.
Modify: docker/verify-delmia-notifier.sh — after G add H: scadabridge --url "$(url_of "$ACT")" api-method list to find DelmiaRecipeDownload's id, then flow-debug run-method --method-id <id> --params '{"MachineCode":"DelmiaSim01","DownloadPath":"C:\\recipes\\wo1.nc","WorkOrderNumber":"WO-H","PartNumber":"PN-1","JobStepNumber":"0100","Username":"drill"}' --format json; assert with python3 -c that Completed == true and the event kinds include InboundRequest, RouteCall, ScriptRun, AttributeWrite, AttributeWait; then flow-debug replay --execution-id <RootExecutionId> --format json and assert the same kind multiset (Replay may lag the 5 s telemetry — retry up to 40 s). Print H live+replay -> YES.
Modify: docker/README.md one line for step H.

Steps: bash docker/deploy.sh (background, ~minutes); when both centrals and site-a are /health/ready, run DELMIA_API_KEY=… bash docker/verify-delmia-notifier.sh (the key: mint a new-named one per the delmia-rig-standin-state memory) and paste the H line into the commit message. If the run fails, fix forward — do not mark the task done on a failing H. Commit — git commit -am "chore(rig): ScriptTrace retention override; verify-delmia-notifier step H = Flow Debugger live run + replay"

Task 23: Documentation sweep

Classification: trivial Estimated implement time: ~5 min Parallelizable with: none (last)

Files:

docs/requirements/Component-CentralUI.md — new ### Flow Debugger (Deployment Role) before ### Alarm Summary (line ~218): page, modes, entry points, predicted graph, drill-ins, Deployer gate, STANDBY_NODE behaviour; add to the page list if the doc has one.
docs/requirements/Component-InboundAPI.md — §API Method Definition → Properties: TraceAllCalls; §Request Flow: the always-on RouteCall row and the Trace flag; new §Flow Debugger runs (management commands, inbound row synthesized with operator actor).
docs/requirements/Component-SiteRuntime.md — under Instance Actor: Trace arm; under Script Actor → Script Run Launch: traced runs, ScriptRun row, inline events, value-change parent inheritance; under Handling Instance.CallScript: flag inheritance.
docs/requirements/Component-AuditLog.md — §Scope: ScriptTrace channel (gated) + RouteCall (always-on); §Retention: per-channel override example; §Reconciliation pull: PullTraceEvents (point read, not a reconciliation path); the mixed-version caveat.
docs/requirements/Component-Communication.md — §8 Remote Queries: PullTraceEvents; §4 Integration Routing: trace flag + inline events on the reply; proto field list.
docs/requirements/Component-ScriptAnalysis.md — REQ-SA-7: Flow shape predictor.
CLAUDE.md — UI & Monitoring: one bullet after the Debug view bullet: Flow Debugger (Live + Replay; trace = audit events in ScriptTrace, gated by a Trace flag beside ParentExecutionId; always-on RouteCall; trace arm; management-command execution path, STANDBY_NODE; TraceAllCalls; design + plan links).
src/ZB.MOM.WW.ScadaBridge.CLI/README.md — confirm flow-debug + --trace-all-calls sections landed.
docs/plans/2026-07-08-deferred-work-register.md — new row 110: "Flow Debugger Vertical trace list toggle (mockup Direction B) and console capture on ScriptRun"; revisit trigger: first operator request.
docs/plans/2026-09-05-flow-debugger-design.md — status → "implemented ; plan 2026-09-05-flow-debugger.md".
../scadaproj/CLAUDE.md — only if the ScadaBridge index row mentions the component summary in a way this changes (it lists stack/wire relationships; a new page does not change them — verify and skip if so).

Commit — git commit -am "docs: Flow Debugger — component specs, CLAUDE.md, CLI README, register row 110"

Dependency order

0 → {1,2,3,5} → 4 → 6 → 7 → 8 → 9 → {10,11} → {12,13,14} → 15 → {16,17,18} → 19 → 20 → 21 → 22 → 23

Parallel lanes once Task 2 is in: (6,7,8,9) site runtime · (10,11) wire · (12,13,14) central audit/inbound · (17) predictor · (16) CLI after 15's records. Task 20 needs 18 + 19.

Verification before "done"

dotnet build ZB.MOM.WW.ScadaBridge.slnx clean; dotnet test ZB.MOM.WW.ScadaBridge.slnx green; git diff --stat src/ZB.MOM.WW.ScadaBridge.Communication/*.csproj empty (no live <Protobuf> item); rig step H prints YES twice (live + replay); the Central UI page renders the Delmia run with 9 nodes on http://localhost:9001/deployment/flow-debugger (Traefik or direct).

68 KiB Raw Blame History Unescape Escape

Flow Debugger Implementation Plan

Task 0: Amend the design doc with the four planning corrections

Task 1: Audit enums — ScriptTrace channel and four kinds

Task 2: Additive Trace / trace-payload fields on the message records

Task 3: ApiMethod.TraceAllCalls — entity, EF migration, management command, CLI flag

Task 4: FlowStep projection + FlowGraph model + FlowGraphBuilder (pure)

Task 5: FlowTraceCollector (site, per run)

Task 6: ScriptRuntimeContext — traced flag, trace emission for writes and waits, flag inheritance

Task 7: ScriptRunLauncher + ScriptActor — traced runs emit ScriptRun, return ExecutionId + events, value-change inheritance

Task 8: InstanceActor — trace arm for data-sourced writes; Trace on published changes

Task 9: DeploymentManagerActor routing — pass Trace, return child id + events

Task 10: Wire — site_command.proto trace fields, mapper, golden samples

Task 11: Wire — sitestream.proto source_execution_id/trace on attribute updates + PullTraceEvents RPC; site SQLite read by execution ids

Task 12: RouteHelper — always-on RouteCall audit row, WithTrace, inline trace sink

Task 13: InboundScriptExecutor.ExecuteAsync(trace) + endpoint honours TraceAllCalls

Task 14: IAuditLogRepository.GetTraceEventsAsync(rootExecutionId)

Task 15: Management commands — FlowDebugRunMethodCommand, FlowDebugRunValueChangeCommand, FlowDebugReplayCommand

Task 16: CLI flow-debug command group

Task 17: FlowShapePredictor (Script Analysis)

Task 18: Central UI service — IFlowDebugService / FlowDebugService

Task 19: FlowGraphView — custom SVG swimlane renderer

Task 20: Flow Debugger page — toolbar, inputs, predicted graph, run, detail panel

Task 21: Drill-ins and the TraceAllCalls checkbox

Task 22: Rig — retention override, redeploy, verification step H

Task 23: Documentation sweep

Dependency order

Verification before "done"

68 KiB

Raw Blame History

Task 1: Audit enums — `ScriptTrace` channel and four kinds

Task 2: Additive `Trace` / trace-payload fields on the message records

Task 3: `ApiMethod.TraceAllCalls` — entity, EF migration, management command, CLI flag

Task 4: `FlowStep` projection + `FlowGraph` model + `FlowGraphBuilder` (pure)

Task 5: `FlowTraceCollector` (site, per run)

Task 6: `ScriptRuntimeContext` — traced flag, trace emission for writes and waits, flag inheritance

Task 7: `ScriptRunLauncher` + `ScriptActor` — traced runs emit `ScriptRun`, return `ExecutionId` + events, value-change inheritance

Task 8: `InstanceActor` — trace arm for data-sourced writes; `Trace` on published changes

Task 9: `DeploymentManagerActor` routing — pass `Trace`, return child id + events

Task 10: Wire — `site_command.proto` trace fields, mapper, golden samples

Task 11: Wire — `sitestream.proto` `source_execution_id`/`trace` on attribute updates + `PullTraceEvents` RPC; site SQLite read by execution ids

Task 12: `RouteHelper` — always-on `RouteCall` audit row, `WithTrace`, inline trace sink

Task 13: `InboundScriptExecutor.ExecuteAsync(trace)` + endpoint honours `TraceAllCalls`

Task 14: `IAuditLogRepository.GetTraceEventsAsync(rootExecutionId)`

Task 15: Management commands — `FlowDebugRunMethodCommand`, `FlowDebugRunValueChangeCommand`, `FlowDebugReplayCommand`

Task 16: CLI `flow-debug` command group

Task 17: `FlowShapePredictor` (Script Analysis)

Task 18: Central UI service — `IFlowDebugService` / `FlowDebugService`

Task 19: `FlowGraphView` — custom SVG swimlane renderer

Task 21: Drill-ins and the `TraceAllCalls` checkbox