# ScadaBridge — architectural / code review (whole system)

**Date:** 2026-09-09 · **Status:** ALL 13 HIGH AND 33 MEDIUM FINDINGS COMPLETED (see §§9–10)
**Reviewed revision:** `main` @ `c078bc8f`. Its only difference from the first review's
`d8229916` is the original findings document; production source/configuration is identical.
**Scope:** all 26 `src/` projects, 32 test/support projects, both Docker rig configurations,
composition roots, contracts, and relevant design documentation.
**Method:** three parallel read-only subagents reviewed (1) Site Runtime / DCL / S&F / event
logging / CLI / notifier, (2) Management / Deployment / Templates / ScriptAnalysis / UI / Security,
and (3) Commons / ConfigurationDatabase / Transport / edge integrations / audit / health.
The primary reviewer covered Communication / ClusterInfrastructure / Host / configuration,
rechecked all High source paths, ran build/test/format/package checks, and executed isolated
probes. Each area revalidated its existing IDs and searched for additional defects. Source-traced
mechanisms, runtime probes, latent risks, and unproven operational impact are distinguished below.
This is a focused whole-system review, not a claim that every line or production topology was
exercised. The review phase ran no rig rebuild, device writes, failover drill, or remote QA/production inspection.
Sections 1–8 preserve that reviewed baseline; §§9–10 record subsequent implementation and local rig verification.

**Second-pass changes:** 11 new findings; UI-3 withdrawn as unsupported; XC-6 merged into CM-5;
UI-4/6/7 reclassified as Low hardening gaps. Existing IDs are preserved for traceability. The
first report counted the same dead RPCs twice and overstated several triggers and fixes; the
corrections here supersede that wording. §8 records the review decisions and validation limits.

Severity: **High** = data loss / security boundary / outage; **Medium** = wrong behaviour an operator
will hit, or a documented property that does not hold; **Low** = quality, consistency, narrow edge.

---

## 1. Verdict

The component structure remains intact. The twelve structural/runtime commitments catalogued
in the first pass retain their code anchors (§6.H — active-node definition, single ActorSystem name, UTC everywhere, no ClusterClient,
auto-down emitted, replication exclusions, metadata resolver on all four compile surfaces, no
health publisher, no central buffering for unreachable sites, single trust boundary, additive
proto evolution). The dependency graph is an acyclic DAG rooted at Commons with zero project
references; every mapped gRPC service is behind a fail-closed constant-time interceptor; all
contract changes since August are additive; all eight migrations are sound and the model snapshot
matches. The test suite is broad; fresh execution evidence and its limits are in §7.

The reviewed baseline contained **92 unique findings: 13 High, 33 Medium, 46 Low**.
**Remediation status: all 13 High and 33 Medium findings completed; 46 / 46 Low findings completed (see §11).**
Completion evidence and limits are recorded in §§9–10.
This includes low-priority test/documentation gaps and latent risks, not just reproduced failures.
The reviewed High-priority findings were:

| # | Sev | Finding | Area |
|---|---|---|---|
| DC-1 | **High** | **Nothing in production ever unsubscribes a tag or alarm source** — the DCL leaks device subscriptions for the life of the node; dead Instance Actor refs get a `TagValueUpdate` per change | DCL / Site Runtime |
| DC-2 | **High** | **Releasing a "redundant" MxGateway handle tears down the LIVE subscription** — an ordinary double-subscribe during resolution silently freezes the tag at its last Good value until the next reconnect | DCL |
| DC-3 | **High** | **Device writes are stashed across a reconnect and replayed minutes later**, after the script was told the write failed — the doc says there is no buffering of setpoints | DCL |
| SR-1 | **High** | **The synchronous `Attributes[key]` accessor blocks a script-pool worker on a continuation queued to the same pool** — saturation can stall the script pool until watchdog recovery (timing is configuration-dependent) | Site Runtime |
| CM-1 | **High** | **No explicit gRPC message-size configuration**; a 256-row site audit batch of 64 KB error rows exceeds the 4 MB default and, because the drain is deterministic oldest-first, the site's audit forwarding can repeatedly stall on the same oversized window and its audit DB grows unbounded | Communication / AuditLog |
| MG-1 | **High** | **Script trust deny-list omits `System.Runtime.CompilerServices` (`Unsafe`, `RuntimeHelpers`) and `System.Runtime.Loader`** — type-punning and constructor bypass pass the deploy gate and the site backstop | ScriptAnalysis |
| UI-1 | **High** | **Many Central UI configuration mutations bypass the configuration audit log** (16 page files save through repositories; service-mediated template/shared-script edits are audited) | Central UI |
| UI-2 | **High** | **Passive session pings keep an unattended tab authenticated** — sliding renewal and periodic role-refresh renewal advance the session | Security |
| SR-11 | **High** | **A raw transaction exposes the underlying database connection**, allowing commands to bypass the audit wrapper | Site Runtime |
| DC-12 | **High** | **OPC UA snapshot-end markers clear unrelated sources' active alarms** because per-feed boundaries are broadcast connection-wide | DCL / Site Runtime |
| CM-9 | **High** | **A site-A key can request site-B reconciliation tokens** — authenticated metadata is never bound to payload ownership | Communication / Security |
| CM-10 | **High** | **Cross-node retry treats every `Unavailable` as unsent**, repeating non-idempotent writes after an ambiguous response loss | Communication |
| UI-13 | **High** | **Open Blazor circuits retain revoked roles and site scopes**, even after the HTTP cookie has refreshed | Central UI / Security |

The other reviewed findings grouped into five themes: **audit completeness** (EI-1/2/3/5,
UI-1), **authorization scope** (MG-2, UI-6/7/8, XC-1, XC-8), **stale/duplicate-apply under
failover** (MG-3/4, EI-4/9, DL-1/6, ORCH-1), **subscription/stream lifecycle** (DC-4/5, SR-3/5,
CM-2), and **inert configuration** (XC-2/3/7). §5 groups everything into a remediation program.

Most changes fit the current component structure. Reliable script containment and safe retries
need explicit runtime/idempotency boundaries; they cannot be fixed by a timeout or status-code
check alone. The immediate sequence is revised in §5.

---

## 2. Findings by area

Each finding: where (file:line), what the code does, why it matters, fix, effort.

### 2.1 Site Runtime / StoreAndForward / SiteEventLogging (SR)

**SR-1 — Synchronous `Attributes[key]` blocks a script-pool worker on a continuation queued to that same pool `[High][CONFIRMED, orch-verified]`**
`Scripts/ScopeAccessors.cs:58-59` (`GetAwaiter().GetResult()`); `Scripts/ScriptRuntimeContext.cs:704, 914` (the Asks inside `GetAttribute`/`SetAttribute` have no `ConfigureAwait(false)`); `Scripts/ScriptRunLauncher.cs:110-123` (the run body is started with `Task.Factory.StartNew(..., executionScheduler)`). Inside the run, `TaskScheduler.Current` is the `ScriptExecutionScheduler`, so the awaiter captures it and the continuation that completes the Ask is `QueueTask`'d onto the same bounded pool whose worker is blocked in `GetResult()`. Pool floor is 8; eight concurrent runs each doing `Instance.Attributes["Setpoint"] = 42` (the documented, compile-gated API — the rig's own Delmia stand-in is a per-attribute write script) block all 8 workers; script/alarm execution can stall until the watchdog detaches/replaces blocked workers. The nominal 30 s timeout + 30 s grace is not a guaranteed recovery time; below saturation these synchronous paths consume additional worker capacity. The class comment at `ScopeAccessors.cs:12-21` asserts the opposite. **Fix:** `.ConfigureAwait(false)` on the awaits in `GetAttribute`, `SetAttribute`, `WaitInternal`, `WriteBatchAndWait` (AsyncLocal `TraceStepScope` flows with `ExecutionContext`, not the scheduler, so attribution is unaffected); correct the comment; add a test running `poolSize` concurrent bodies each doing one sync `Attributes["x"]` on a scheduler of that size. Small.

**SR-2 — Alarm on-trigger runs have no integration surface `[Medium][CONFIRMED]`**
`ScriptRunLauncher.cs:701-725` (`RunAlarmScriptAsync`) never creates a DI scope; `_externalSystemClient`, `_databaseGateway`, `_storeAndForward`, `_operationTrackingStore`, `_cachedForwarder`, `_auditWriter`, `_siteId`, `_sourceNode` are null, so `ExternalSystem`/`Database`/`Notify`/`Tracking` throw (`ScriptRuntimeContext.cs:1422/1485/2145/2195/2760`). The deploy gate compiles alarm on-trigger scripts against the full `ScriptCompileSurface`, so `Notify.To("Ops").Send(...)` deploys cleanly and throws the first time the alarm fires. `Component-SiteRuntime.md:482` says the full API is available to alarm on-trigger runs. **Fix:** wire the scope + dependencies exactly as `RunScriptAsync` does (small), or narrow the doc and gate it at design time (medium).

**SR-3 — `NativeAlarmActor` subscribe retry has no in-flight guard, no backoff, no correlation match `[Medium][CONFIRMED]`**
`Actors/NativeAlarmActor.cs:147-165, 320-340`: fresh correlation id per send, discarded; fixed-interval re-send; `HandleSubscribeResponse` checks only `Success`. This is the amplifier Lever 1 (2026-08-28) closed on `InstanceActor` and left as a follow-up; a late `Success` from a superseded request also cancels the timer armed for the current one. **Fix:** port `TagSubscribeState` (in-flight id + ceiling, `NextTagSubscribeInterval`, drop non-matching replies). Small.

**Second-pass clarification:** Lost replies retry at twice the configured interval; explicit failures use the configured interval. DCL does have `_alarmSubscribesInFlight` deduplication, so repeated requests are not proof that every retry opens another physical feed. Correlation and stale-response handling remain defective.

**SR-4 — `AuditingDbConnection` forwards a settable `ConnectionString` while audit rows keep the construction-time name `[Medium][CONFIRMED]`**
`Scripts/AuditingDbConnection.cs:113-120` vs `:35`; `AuditingDbCommand.cs:426`. A script can `Close()`, assign an arbitrary connection string, `Open()`, and every `DbWrite` row still says `target = "MES-Prod"`; with `Persist Security Info=true` the getter returns the password. The audit log is the compensating control for the raw-ADO escape hatch. **Fix:** reject connection retargeting and redact the getter, then close the raw-transaction escape in SR-11; the class is already sealed. Small plus SR-11.

**SR-5 — `InstanceActor.PostStop` blocks the default dispatcher for up to 5 s per instance `[Medium][CONFIRMED]`**
`Actors/InstanceActor.cs:129, 443-444, 693-708` (`flush.Wait(FinalStaticOverrideFlushTimeout)`); no `WithDispatcher` anywhere in SiteRuntime. A bulk redeploy landing during an `EventLogPurgeService` tick (1000-row `DELETE` + `incremental_vacuum` under `SiteEventLogger._writeLock`, `EventLogPurgeService.cs:519-552`) stops N actors concurrently, each burning a dispatcher thread on `SQLITE_BUSY` backoff — node-wide, since DCL, Deployment Manager, Script/Alarm actors and the stream relay share it. **Fix:** gate the redeploy clear on an explicit "flush committed" ack piped from the persist task instead of `Terminated` + a blocking wait; with bounded stop fanout. A shorter blocking timeout alone does not establish safe flush/reset ordering. Medium.

**SR-6 — Trace arm is consumed by any subsequent value change, not the traced write's echo `[Low][CONFIRMED]`** — `InstanceActor.cs:185, 775-780, 1085-1092`; a periodic device publish inside the 30 s window is stamped with the writer's execution id and the real echo goes untraced. Debug fidelity only. Fix: define explicit echo-matching semantics or mark linkage inferred; device updates carry no write-request id, and clamped/coerced values need deliberate handling. Narrowing the window alone is only a heuristic. Small.

**SR-7 — `EmitTrace`/`Collect` log their own failures through the decorated logger `[Low][CONFIRMED]`** — `ScriptRuntimeContext.cs:415-419, 551, 562, 568` use `_logger` (the `TraceCapturingLogger` on a traced run) where `EmitLog` (:590-645) correctly uses `_innerLogger`; each failed trace write becomes a `ScriptLog` row plus a second write attempt (bounded at depth 2). Fix: `_innerLogger`. Small.

**SR-8 — `ScriptExecutionScheduler.TryExecuteTaskInline` keys on the thread-name prefix, not identity `[Low][CONFIRMED]`** — `:385-393` vs naming at `:331-335`; co-resident schedulers (the #18 seam) cross-inline, defeating the bound and detach bookkeeping. Latent. Fix: `[ThreadStatic] CurrentScheduler` + `ReferenceEquals`. Small.

**SR-9 — Shared trigger gate does not recreate a disposed instance `[Low][CONFIRMED latent lifecycle gap]`**
`SiteRuntime/Scripts/TriggerEvalGate.cs:55-64` returns the existing shared instance after disposal,
unlike the scheduler's disposed guard. No production call disposing that shared gate was found;
this is not an established current node-outage path. **Fix:** either recreate after disposal or
make process-lifetime ownership explicit; test Shared→Dispose→Shared if that lifecycle is supported.
Small.

**SR-10 — Script helper properties allocate a fresh helper + guard closure per access `[Low][CONFIRMED]`** — `ScriptRuntimeContext.cs:1070, 1092, 1130, 1150, 1213`. Fix: lazily cached fields. Small.

**SR-11 — A raw transaction exposes the underlying connection and bypasses DB auditing `[High][CONFIRMED, isolated reproduction, orch-verified]`**
`src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Scripts/AuditingDbConnection.cs:149-150` returns the
provider's transaction. `AuditingDbCommand.cs:172-175` also exposes its raw transaction. A script
can call `connection.BeginTransaction().Connection`, create a command on that raw connection,
and execute it without `AuditingDbCommand`; abstract ADO types are permitted by the trust policy.
This also bypasses any ConnectionString/ChangeDatabase hardening proposed under SR-4.
An in-memory SQLite probe constructed the internal wrapper through a test harness, then used
ordinary public ADO APIs: normal command audit count **1**, transaction connection identity
**the original provider connection**, escaped write rows **1**, escaped audit count **0**.
Reflection was only harness setup, not part of the script escape. **Fix:** wrap transactions,
return the auditing connection from transaction.Connection, and unwrap only internally when
assigning a provider command transaction. Cover sync/async starts and Transaction readback.
Test exactly-one audit through every public connection/transaction/command path. Medium.

**SR-12 — Late native-alarm rehydration can resurrect already resolved conditions `[Medium][CONFIRMED source path]`**
`src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/NativeAlarmActor.cs:125-128` starts an asynchronous
storage read and immediately subscribes. `HandleRehydration:167-175` guards only current
`_alarms.ContainsKey`; :219 inserts old state. A resolved live transition (:297-306) or completed
empty snapshot (:251-282) removes that key. If the old storage read arrives afterward, the prior
active condition is restored and emitted. **Fix:** fence rehydration against the first authoritative
snapshot and track live-touched keys including deletions. Test delayed read completion after both
a resolved live event and an empty snapshot; neither may resurrect the old alarm. Small.

**SR-13 — An in-progress S&F sweep starts further deliveries after node demotion `[Medium][CONFIRMED source path]`**
`src/ZB.MOM.WW.ScadaBridge.StoreAndForward/StoreAndForwardService.cs:828-846` checks the active
gate once before loading a batch. Lanes wait at :864 and start each remaining delivery at :873
without rechecking; :917 calls the external handler. A demotion during the first slow delivery
leaves the old node starting additional work while the new active can drain replicated rows.
This exceeds a single already-in-flight delivery window. **Fix:** recheck a fail-closed gate before
each new attempt, including after lane waits, and abandon the remaining batch on demotion.
Already-sent operations still need idempotency/fencing for ambiguous outcomes. Test a blocked first
handler, flip the gate, release it, and assert no second attempt starts on the old node. Small.

### 2.2 Communication / ClusterInfrastructure / Host (CM)

**CM-1 — No explicit gRPC message-size configuration; the audit batch arithmetic can exceed the 4 MB defaults and can repeatedly stall the same oversized audit window `[High][CONFIRMED, orch-verified]`**
`Host/Program.cs:291-294` (central `AddGrpc`) and `:825-829` (site) set no `MaxReceiveMessageSize`; neither do `CentralChannelProvider.cs:111-120`, `SitePairChannelProvider.cs:424-437`, `SiteStreamGrpcClient.cs:87-95`; a repo-wide grep is empty. Defaults: receive 4 MB both ends, send unlimited. `SiteAuditTelemetryOptions.BatchSize = 256` × `AuditLogOptions.ErrorCapBytes = 65 536` = 16 MB possible; 64 error-only rows at the cap already exceed 4 MiB once protobuf framing is included. `SqliteAuditWriter.cs:659-661` serves a deterministic oldest-first window and `SiteCommunicationAuditClient.cs:78-82` deliberately does not catch, so an unchanged oversized window fails again on each tick. `PurgeExpiredAsync` (:1170-1176) never purges `Pending`, so the site audit DB on `/app/data` grows unbounded behind the stuck head. Same arithmetic on the reconciliation pull (`BatchSize = 256`, site → central, 4 MB client receive) and on `DeployArtifactsCommand` (unlimited central send, 4 MB site receive). **Fix:** explicit `MaxReceiveMessageSize` (e.g. 32 MB) on both `AddGrpc` calls and every channel, **and** a byte budget on the drain/pull mirroring LocalDb's `MaxBatchBytes`; at minimum treat `StatusCode.ResourceExhausted` in `SiteAuditTelemetryActor` as "halve the batch". Test: serialize a 256-row batch of `ErrorCapBytes` rows and assert it fits the configured limit. Medium.

**Second-pass clarification:** The isolated serializer probe measured **16,779,264 bytes** for 256 rows containing only a 65,536-byte ErrorDetail, versus the configured-by-framework **4,194,304-byte** receive default. Caps apply per field, not per whole row; account for all fields/framing and single oversize rows. Ordinary pending reads are at `AuditLog/Site/SqliteAuditWriter.cs:587-620`; the original :659 anchor is the cached lane. The stall persists while the same oversized windows remain and neither a smaller reconciliation window nor operator action drains them; no live disk-exhaustion experiment ran.

**CM-2 — `docker-env2` runs the same simultaneous `--force-recreate` deploy that split site pairs twice, with the bootstrap guard OFF `[Medium][CONFIRMED]`** — `docker-env2/deploy.sh:24` is byte-identical to `docker/deploy.sh:25`; env2's four node appsettings have no `BootstrapGuard` block (`ClusterOptions.cs:144` default false). A deploy there can produce two 1-node clusters that never merge, and env2 is the second real environment gating Transport work. Fix: add the block to all four files. Small.

**CM-3 — `PullAuditEvents`/`PullSiteCalls` accept an unbounded `batch_size` `[Low][CONFIRMED]`** — `SiteStreamGrpcServer.cs:727-734, 816-822` only check `> 0`; `PullTraceEvents` (:668-676) caps 1..2000; `SiteAuditReconciliationOptionsValidator.cs:23` only `> 0`. Feeds CM-1. Fix: same ceiling + validator upper bound. Small.

**CM-4 — `ClusterBootstrapCoordinator` can issue `JoinSeedNodes` during host shutdown `[Low][CONFIRMED]`** — `Host/Actors/ClusterBootstrapCoordinator.cs:186-188` swallows the shutdown cancellation as "partner unreachable", so `RunAsync` (:118-126) self-first-joins with no `ct` re-check. Fix: check `ct` before the join / rethrow. Small.

**CM-5 — Site-hosted `IngestAuditEvents` / `IngestCachedTelemetry` are permanently inert `[Low][CONFIRMED]`** — `SetAuditIngestActor` (`SiteStreamGrpcServer.cs:159-162`) is called only from `AkkaHostedService.cs:569-570` inside `RegisterCentralActors`, but `SiteStreamGrpcServer` is Site-branch-only (`Program.cs:830, 868`); null on central, never set on site. Both RPCs answer an empty ack + Warning on every real node, reachable on the PSK-gated listener; the comment at `AkkaHostedService.cs:565-568` reads as if live. (XC-6 is the same finding from the RPC-inventory side.) Fix: remove inert host wiring and return `Unimplemented` from retired handlers while preserving additive proto compatibility; production ingest uses `CentralControlService`. Small.

**CM-6 — Central Kestrel sets no HTTP/2 keepalive, and the optional metrics listener exposes the whole routing table `[Low][CONFIRMED]`** — `Program.cs:231-263` (no `Limits.Http2.KeepAlivePing*`, unlike `SiteKestrelSetup.cs:51-52`); `:254-262` binds `MetricsPort` as `Http1AndHttp2` on the shared pipeline. Defensible today (unary-only `CentralControlService`), undocumented. Fix: apply the site keepalive on central; document the metrics-port surface. Small. (See XC-1 for the larger form of this.)

**CM-7 — `ChannelDisposalGrace` (5 min) can be outlived by `RouteToWaitForAttribute` `[Low][PLAUSIBLE]`** — `SitePairChannelProvider.cs:56-62` sizes the grace to "the longest command deadline"; `GrpcSiteTransport.cs:224` resolves that request's deadline as caller-supplied `r.Timeout + IntegrationTimeout`, unclamped (`RouteToInstanceRequest.cs:119`, `DeploymentManagerActor.cs:3036`). A > 5-min wait crossing an address refresh gets `ObjectDisposed`/`Cancelled` (not `Unavailable`, so no cross-node retry). Fix: clamp, or grace = `max(5 min, longest resolvable)`. Small.

**CM-8 — `SiteStreamGrpcClient.ConnectedHeaderTimeout` is a mutable static test seam `[Low][CONFIRMED]`** — `:460`, mutated by `SiteStreamGrpcClientTests.cs:440, 474` while `SiteAlarmAggregatorActorTests` (:865) depends on the default; a parallel-run flake source of the register-row-118 kind. Fix: instance value from `CommunicationOptions`. Small.

**CM-9 — Per-site authentication is not bound to payload site ownership `[High][CONFIRMED source path and isolated gate probe, orch-verified]`**
`src/ZB.MOM.WW.ScadaBridge.Host/CentralControlAuthInterceptor.cs:149-188` authenticates the key
for the metadata `x-scadabridge-site`, but neither records an authoritative site principal nor
validates the decoded payload. `Communication/Grpc/CentralControlGrpcService.cs:258-268` forwards
`ReconcileSiteRequestDto` unchanged through `CentralControlDtoMapper.cs:381`.
`Communication/ReconcileService.cs:64-79,110-161` resolves the payload SiteIdentifier and returns
fresh/existing deployment fetch tokens for that site's snapshots. Thus a caller holding only
site-A's PSK can name site-B in its request and obtain B's deployment fetch capability, subject to
B having deployments. Health/audit source-site fields also retain untrusted payload attribution.
The isolated real interceptor+mapper probe accepted site-A metadata/key with a site-B reconciliation
payload. No production secrets were fetched. **Fix:** carry verified site identity into dispatch,
reject mismatched reconciliation/health/notification ownership, validate every audit/telemetry row
against authenticated site policy, and authorize status queries by stored ownership. Preserve
legitimate cross-site execution links without letting a caller forge the owning site. Test a
cross-site request through the actual interceptor+service+repository path and prove no token or
mutation is produced. Medium.

**CM-10 — `Unavailable` triggers blind replay of non-idempotent site commands `[High][CONFIRMED retry mechanism, orch-verified]`**
`src/ZB.MOM.WW.ScadaBridge.Communication/Grpc/SitePairChannelProvider.cs:226-237` retries every
`RpcException(Unavailable)` on the second node, asserting the call never reached a server.
`GrpcSiteTransport.cs:125-175` uses this for lifecycle, tag writes, and `RouteToCall` commands.
A response-side connection loss can occur after application; gRPC does not guarantee that
`Unavailable` means unsent ([gRPC error guide](https://grpc.io/docs/guides/error/),
[HTTP/2 protocol](https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md)).
An isolated provider probe applied a synthetic effect then threw Unavailable: the provider applied
it again on the peer and returned success (**2 applications**). This proves retry behavior under
an injected ambiguous outcome, not a live-device duplicate. `GrpcCentralTransport.cs:176-186`
shares the incorrect premise, although notification/audit identity dedup protects some operations.
**Fix:** classify operations and guarantee durable/shared deduplication before retrying mutations,
or return an explicit uncertain outcome after a possibly committed send. Restrict unprotected
retry to provable pre-dispatch failures; merely checking status is insufficient. Test response-loss
after commit and assert one effect across the pair. Medium, with cross-layer operation identity.

### 2.3 ManagementService / DeploymentManager / TemplateEngine / ScriptAnalysis (MG)

**MG-1 — Script trust deny-list omits `System.Runtime.CompilerServices` and `System.Runtime.Loader` `[High][CONFIRMED, orch-verified]`**
`ScriptAnalysis/ScriptTrustPolicy.cs:36-62` (`ForbiddenScopes`) blocks `System.Runtime.InteropServices` and nothing else under `System.Runtime.*`; Pass 2's `ReflectionGatewayMembers` (`ScriptTrustValidator.cs:162-172, 266-281`) has none of `Unsafe.As`/`Unsafe.Write`/`RuntimeHelpers.GetUninitializedObject`/`RunClassConstructor`/`LoadFromAssemblyPath`; `DefaultAssemblies` (:149-156) includes CoreLib, which hosts all of them on net10.0. `Unsafe.As<object,T>(ref o)` / `Unsafe.Write` need no `unsafe` context, so `AllowUnsafe=false` does not stop them; type-punning reaches the host's private state (the `ScriptRuntimeContext`, the DI graph, secrets). The deploy gate and the site backstop share the policy, so nothing catches it. `Component-ScriptAnalysis.md:38` claims "Reflection — forbidden entirely". **Fix:** add type-level entries `System.Runtime.CompilerServices.Unsafe`, `System.Runtime.CompilerServices.RuntimeHelpers`, `System.Runtime.Loader` (type-level so `AsyncMethodBuilder`/`CallerMemberName` and compiler-generated attributes stay allowed — verify against the clean-script corpus); negative test per entry; doc table. Small. Ranked High because it is the trust boundary itself, even though it needs the Designer role.

**MG-2 — `/api/audit/tree` and `FlowDebugReplayCommand` apply no site scope `[Medium][CONFIRMED, orch-verified]`**
`AuditEndpoints.cs:280-305` (`HandleTree`) authenticates and role-checks but never calls `ApplySiteScope` (which `HandleQuery`/`HandleExport` do at :584); `ManagementActor.cs:423` passes only `sp, cmd` to `FlowDebugRunner.ReplayAsync` (:538), which takes no `AuthenticatedUser`; `AuditLogRepository.cs:946-1077` walks the whole `(ExecutionId, ParentExecutionId)` chain with no `SourceSiteId` predicate. A site-A-scoped operator picks any `ExecutionId` from their (correctly scoped) query and receives site-B's `ScriptRun`/`AttributeWrite`/`DbOutbound` rows (SQL text + bound values) from a multi-site chain — the boundary rows 98/99 and P6-4/P6-5 hardened on every other read path. The three Live run commands DO enforce scope. **Fix:** optional permitted-`SourceSiteId` set on `GetExecutionTreeAsync`/`GetTraceEventsAsync` (NULL = central rows, always visible); thread the user into `ReplayAsync`; stop-gap: refuse both for scoped non-admins. Medium.

**MG-3 — Process-local stale-probe invalidation misses configuration edits on the other central `[Medium][CONFIRMED, corrected trigger]`**
`DeploymentManager/StaleInstanceProbe.cs:45,120-140` caches against an in-process watermark;
`ConfigurationDatabase/Repositories/TemplateEngineRepository.cs:772-774,888-897` advances only
that process's injected watermark after writes. `ManagementService/ManagementActor.cs:343-350`
accepts configuration handlers without an active-node gate. Warm A's memo, then edit the shared
configuration DB through an authenticated direct connection to B: A's cached revision remains
valid by its own unchanged watermark, so `DeploymentService.cs:1178` can skip a changed instance.
The earlier A→standby→A scenario was not established: planned failover leaves/restarts the process,
which clears static memory. **Fix:** authoritative shared revisions or reliable cross-node cache
invalidation; alternatively remove memoized hashes from deployment skip decisions. Activation-only
clearing is insufficient while the active process remains active. Test two independently owned
watermarks against a shared store and a peer-originated edit. Medium.

**MG-4 — `MgmtDeploySiteCommand`'s worst-case runtime exceeds its own Ask timeout by construction `[Medium][CONFIRMED]`**
`ManagementEndpoints.cs:44-77` (`LongRunningCommandTimeout` 5 min); `DeploymentManagerOptions.cs:34, 42` (parallelism 4 × 120 s per instance → `ceil(N/4) × 120 s`; N = 9 already needs three waves (360 s), before serial prepare and finalize). The caller gets `504` while the batch keeps applying; a redeploy is not idempotent at the site (resets overrides, clears native alarm state), so the retry re-applies live instances and its prepare loses the 5 s lock timeout on still-held locks. **Fix:** derive the bulk Ask timeout from the batch, or make bulk deploy submit-and-poll with stable operation identity. Propagate the overall budget through CLI/HTTP as well; a longer inner Ask alone cannot prevent caller timeout. Medium.

**MG-5 — `FlowDebugRunner.MaxWaitSeconds` equals the Ask timeout it must stay under, unvalidated `[Low][CONFIRMED]`** — `FlowDebugRunner.cs:56-63` (300 s) vs `ManagementServiceOptionsValidator.cs:26-28` (only `> 0`); a `--wait-seconds 300` run 504s the caller while the runner keeps a scoped provider and polls the site. Fix: establish a total deadline at command entry and pass remaining budget through the initial write, audit, and every pull; reserve response grace and validate configuration. A poll-only clamp ignores time already spent before polling. Small.

**MG-6 — `WriteAndPollAsync` re-fetches a fixed oldest page every second; register row 117 confirmed, plus a silent 64-id truncation `[Low][CONFIRMED]`** — `FlowDebugRunner.cs:501` (`since` never advances), `:510` (`ids.Take(64)` drops descendant ids beyond `MaxExecutionIdsPerRead` with no diagnostic); the site serves `ORDER BY OccurredAtUtc ASC LIMIT` (`SqliteAuditWriter.cs:899-908`), so > 2000 rows in the window makes the awaited run unreachable and up to 300 polls re-transfer the same page. Fix: use timestamp-plus-event-id keyset paging and explicit per-execution batching/truncation. Advancing only a timestamp can lose ties and older rows belonging to newly discovered descendants; test both plus more than 64 ids. Small.

**MG-7 — The value-change / write-kind Flow Debugger result bypasses the `FlowTraceCollector` caps `[Low][CONFIRMED]`** — `FlowDebugRunner.cs:525-526` returns all of `known` with `Truncated: false` hardcoded, vs the capped paths at `:190-197`/`:416-418`; 200 log lines × a cascade is an unbounded reply through `SerializeResult` and the Blazor circuit. Fix: feed `known` through a collector (null redactor — rows were redacted at the site). Small. (Same class as DL-3.)

**MG-8 — Shared scripts never reach the design-time trust/compile gate on create/update `[Low][CONFIRMED]`** — `TemplateEngine/SharedScriptService.cs:60, 109, 178-206` run only `CSharpDelimiterScanner`; `ValidationService.cs:276-293` compiles `configuration.Scripts` only; `ArtifactDeploymentService.cs:172-179` ships them fleet-wide; only `BundleImporter` (:912, 4658) gates them. A forbidden-API shared script is rejected at the site's `ScriptCompilationService.Compile`, at run time. Fix: `ScriptCompiler.TryCompile` in create/update (memoised by `ScriptCompileVerdictCache`). Small.

**MG-9 — `ManagementActor.Page` turns an out-of-range `Take` into unlimited `[Low][CONFIRMED]`** — `:685-691` (`take is > 0 and <= 1000 ? take : int.MaxValue`; comment says clamped). Fix: clamp positive takes to 1000 while explicitly preserving/documenting null and nonpositive semantics; test every boundary. Small.

**MG-10 — `Component-ManagementService.md` documents none of the four `FlowDebug*` commands `[Low][CONFIRMED]`** — zero occurrences, though `RequiredRoleMatrixTests.cs:16-18` names the catalog as its cross-check. Fix: add the subsection. Small.

### 2.4 Central UI / Security (UI)

**UI-1 — Many Central UI configuration mutations bypass configuration auditing `[High][CONFIRMED, corrected scope, orch-verified]`**
Sixteen page files call repository `SaveChangesAsync` directly: `NotificationLists`,
`NotificationListForm`, `SmtpConfiguration`, `SmsConfiguration`, `ExternalSystemForm`, `SiteForm`,
`LdapMappingForm`, `ApiMethodForm`, `TemplateEdit`, `LdapMappings`, `DbConnectionForm`,
`DataConnectionForm`, `ExternalSystems`, `Sites`, `DataConnections`, and `InstanceConfigure`.
Examples under `CentralUI/Components/Pages/`: `Admin/LdapMappingForm.razor:170-178,200-223`,
`Design/DataConnectionForm.razor:337-349`, `Notifications/NotificationLists.razor:168-169`, and
`Design/TemplateEdit.razor:1088-1119` (native alarms). Role/scope grants and connection changes
can commit without the configuration audit row that the corresponding management path writes.
Repository bypasses also miss application-level scrubber/validation/delete guards.
The original “every edit” claim was incorrect: TemplateEdit's ordinary template/script/attribute
changes call audited `TemplateService` (`TemplateService.cs:144,410,802`), and
`SharedScriptService.cs:77,118,141` audits its mutations. **Fix:** route the bypass paths through
audited application services/management envelopes. A temporary audit integration must use the
same transaction, scrub secrets, and handle generated IDs correctly. Test actual UI create/update/
delete success and rollback, including LDAP mappings; a source-string sweep alone is insufficient.
Large.

**UI-2 — Passive session checks keep an unattended tab authenticated `[High][CONFIRMED, orch-verified]`**
`CentralUI/Components/Shared/SessionExpiry.razor:37,49,64,80-100` pings every minute without an
interaction signal. Normal cookie sliding renewal applies, and
`Security/CookieSessionValidator.cs:177-210` periodically rebuilds the principal with a fresh
LastActivity anchor when role mapping refresh succeeds. `Security/ServiceCollectionExtensions.cs:274-275`
explicitly renews that replacement too. With healthy mapping, an unattended tab can remain logged
in indefinitely. **Fix:** distinguish passive validity checks from actual activity: disable passive
sliding and Replace-driven activity renewal, preserve role refresh without advancing idle time,
and supply real interaction tracking including Blazor input. Merely stopping ping renewal can also
expire actively used circuits because SignalR events are not new cookie-authenticated HTTP requests.
Test idle pings beyond the timeout and genuine interaction over the same interval; separately fix
UI-13's stale circuit authority. Medium.


**UI-3 — Alleged off-renderer Flow Debugger refresh mutation `[WITHDRAWN — unsupported]`**
`CentralUI/Components/Pages/Deployment/FlowDebugger.razor.cs:543-609,612-625` starts the refresh
from UI event paths and ordinarily awaits both Delay and ReplayAsync. Those caller awaits capture
the Blazor renderer synchronization context. A service's internal `ConfigureAwait(false)` does
not remove the caller's context capture. No background entry point establishing the alleged
cross-thread mutation was found. Explicit `InvokeAsync` could be defensive, but this is not a
confirmed defect and is excluded from the active tally.

**UI-4 — Poll-gate cleanup depends on a posted renderer callback running `[Low][CONFIRMED mechanism; live outage not established]`**
`CentralUI/Components/Pages/Monitoring/Health.razor:719-734` and `AlarmSummary.razor:394-411`
acquire the gate outside the posted callback but release inside it. A rejected callback can leave
the gate held and its unobserved task faulted. Rejection normally occurs during disposal, when
refresh should stop anyway; the previous continuing-component outage claim was not reproduced.
**Fix:** observe dispatch in an outer try/finally, release the gate there, and guard disposed/queued
callbacks. Test dispatcher rejection and disposal races. Small hardening change.

**UI-5 — Noncooperative Test Run scripts permanently occupy admission slots `[Medium][CONFIRMED; remediation corrected]`**
`CentralUI/ScriptAnalysis/ScriptAnalysisService.cs:207-214,471-481,524-527` has four static slots,
awaits actual Roslyn execution, and releases only when it completes. A cancellation-ignoring loop
retains one worker/slot indefinitely; four exhaust Test Run capacity. **Do not release the slot
merely because a logical timeout fired:** the work is still running, and admitting replacements
permits unbounded runaway CPU/memory and continued side effects through retained gateways.
**Fix:** a bounded pool of isolated, killable workers with confirmed termination before reuse.
Interim timeout responses must observe/quarantine unfinished work and retain its capacity charge
until actual exit. Test noncooperative code only in a disposable child process, including no further
operations after termination. Medium; robust containment is larger than the original small fix.

**UI-6 — Manual failover lacks service-level authorization `[Low][CONFIRMED defense-in-depth gap]`**
`Host/Health/AkkaManualFailoverService.cs:59,89` has no authoritative caller check; its current
component entry is protected by an Admin `AuthorizeView`. No unauthorized invocation of that
handler was demonstrated. **Fix:** an authenticated application facade/management command must
verify the Administrator role before calling the host service; respect scoped circuit versus
singleton service lifetimes. Test denial at that facade. UI-13 separately covers real stale-grant
access through an existing circuit. Small.

**UI-7 — No complete UI route-policy guard `[Low][CONFIRMED prevention gap]`**
`Security/AuthorizationPolicies.cs:195-244` defines no fallback policy, and no sweep freezes every
routable component's required role. Current application routes do carry authorization attributes;
no anonymous existing-page bypass was shown. **Fix:** reflection-based route/policy tests including
the external secrets assembly. A global HTTP fallback is useful only after preserving intended
auth/bootstrap endpoints and does not itself enforce each interactive component's role policy.
Small.

**UI-8 — Flow Debugger's parked-node site-scope check fails open on an unresolvable site `[Low][CONFIRMED]`** — `FlowDebugger.razor.cs:816-838` (lookup ultimately permits a null site); `FlowDebugService.RelayAsync` (:284-345) has no scope check of its own. Fix: deny for scoped principals; move the check into `RelayAsync` (shared with the Site Calls page). Small.

**UI-9 — `TemplateFolderTree` rebuilds the whole forest on every parameter set `[Low][CONFIRMED]`** — `:99-103` vs the `InstancePicker.razor.cs:157-166` reference-equality guard. Fix: reference guards only under an explicit immutable/reference-change contract covering every tree-affecting parameter; otherwise in-place changes would disappear. Small.

**UI-10 — Cookie login accepts cross-site form submission without antiforgery validation `[Low][CONFIRMED, narrowed]`**
`CentralUI/Auth/AuthEndpoints.cs:25-110` manually reads a form, sets the authentication cookie,
and disables antiforgery. An attacker with valid credentials can establish their session in a
victim browser, causing later actions to be attributed to that account. `/auth/token:112-171`
returns JSON and sets no cookie; the same browser-session claim does not apply there.
**Fix:** render an antiforgery token in Login and explicitly validate it or attach required
validation metadata to the manual form handler. Removing `DisableAntiforgery` alone is insufficient.
Test missing/invalid token rejection and retain legitimate CLI bearer-token exchange. Small.

**UI-11 — Topology loads every instance and template into the circuit, unpaged `[Low][CONFIRMED]`** — `Topology.razor:346-349`; the report pages page at 50. Fix: lazy per expanded site/area. Medium.

**UI-12 — Empty "Notifications" nav heading for a Viewer; Flow Debugger unreachable from the nav for the roles its policy admits `[Low][CONFIRMED]`** — `NavMenu.razor:55-58` (false premise: a Viewer holds none of Admin/Design/Deployment) and `:48` (link inside the Deployer section though `RequireFlowDebugger` admits Viewer — the F3 auditor intent is half-served). Fix: any-of gate; own `AuthorizeView`. Small.

**UI-13 — Existing circuits retain revoked roles and site scopes `[High][CONFIRMED source path, orch-verified]`**
`src/ZB.MOM.WW.ScadaBridge.CentralUI/Auth/CookieAuthenticationStateProvider.cs:42,46-47` serves
one captured principal for the circuit lifetime; `Auth/SiteScopeService.cs:95-98,128-129` caches
scope too. HTTP cookie refresh (`Security/ServiceCollectionExtensions.cs:274-275`) updates neither.
`CentralUI/Auth/AuthEndpoints.cs:214-218` returns 200 while the replacement principal is still
authenticated, so the ping never signals a role/site revocation. `Services/FlowDebugService.cs:581-591`
builds ManagementEnvelope authority from the frozen principal. An open circuit can therefore keep
issuing formerly allowed mutations after an administrator narrows its grants. Fixing UI-2 alone
does not revoke grants on an actively used circuit. **Fix:** server-side circuit revalidation and
permission-cache invalidation, with authoritative checks at mutation time; publish changed auth
state or terminate/re-establish the circuit on authorization-version changes. Repeatedly reading
IHttpContextAccessor is not a valid circuit-lifetime solution. Test revoke role and site while the
same circuit stays authenticated, then assert no site command is sent. Medium.

**UI-14 — Sandbox output is bounded only after unbounded capture allocation `[Medium][CONFIRMED source path]`**
`src/ZB.MOM.WW.ScadaBridge.CentralUI/ScriptAnalysis/ScriptAnalysisService.cs:460` uses an ordinary
StringWriter; :488 and error/timeout branches call `captured.ToString()` before :579-582 truncates
to the 32k display ceiling. `SandboxConsoleCapture.cs:117-129` forwards every write without a byte/
character budget. A finite or cancellation-cooperative script can allocate a large buffer, then a
second full string, in the central process. Four slots and a time limit are not memory bounds.
**Fix:** a synchronized bounded writer that discards/counts excess without materializing it, including
error/timeout paths. Test many writes of a reused large string and concurrent child output with
bounded retained memory. Review return-value serialization under the same output budget. Small.

### 2.5 InboundAPI / ESG / Notifications / SiteCallAudit / AuditLog / KpiHistory / HealthMonitoring (EI)

**EI-1 — Relay failure with all peers unreachable leaves no inbound audit row `[Medium][CONFIRMED, narrowed]`**
`InboundAPI/InboundApiEndpointFilter.cs:95-99` sets `RelayedToActiveItemKey` before forwarding;
`Middleware/AuditWriteMiddleware.cs:236,251` then suppresses the local row even when
`HttpStandbyRequestForwarder.cs:160-163` returns `ACTIVE_NODE_UNREACHABLE`. If every peer is
unreachable, none can audit. A reachable standby refusal may audit, so “no peer executes” does
not universally mean zero rows; multiple refusal attempts can also create multiple rows.
**Fix:** explicit relay-attempt/serving-peer audit ownership and a correlated local failure row when
no serving peer accepts. A response alone is not proof of persisted audit. Test all-unreachable,
all-standby, successful relay, and active method failure through middleware plus filter. Small.

**EI-2 — `SkipBodyCapture` does not suppress `errorDetail`, which routinely quotes the body `[Medium][CONFIRMED]`** — `AuditWriteMiddleware.cs:195, 243-249` vs `:335-341`; `InboundScriptExecutor.cs:601-604` (`ErrorDetail: ex.ToString()`); `EndpointExtensions.cs:288-291`. F11 closed this for the `RouteCall` row (`RouteHelper.cs:422`); the failure path re-opens it for every failing call, traced or not (an `ArgumentException` quoting a lot number, a `JsonException` quoting the payload, a `SqlException` echoing a key). Fix: if this option is intended to prevent body-derived persistence, suppress both free-text error message and detail or replace them with a fixed error category; either can quote the payload. The current option contract names request/response summaries only, so document the strengthened privacy semantics and test complete stored rows. Small.

**EI-3 — Per-target SQL-parameter redaction is not applied to `DbOutbound` error rows `[Medium][CONFIRMED]`** — `ScadaBridgeAuditRedactor.cs:217-231` redacts only the request summary; `InboundDatabaseHelper.cs:251` and `AuditingDbCommand.cs:516` stamp `errorDetail: thrown?.ToString()`, and SQL Server error text carries parameter values (`The duplicate key value is (…)`). Fix: the SQL regex matches parameter NAMES, so applying it to free-text errors cannot reliably remove VALUES. Suppress sensitive free-text SQL exceptions (retain error category/number), or redact the matched parameter values at emission plus configured body patterns. Include all error fields; see EI-11. Small.

**EI-4 — Relay timeout can expire before method deadline and cause ambiguous replay `[Medium][CONFIRMED mechanism; duplicate impact conditional]`**
`InboundAPI/HttpStandbyRequestForwarder.cs:112-113,147-152` times out at the relay budget and tries
another peer; method timeouts can be longer (`EndpointExtensions.cs:260-262`). The executor does
link `RequestAborted` (`InboundScriptExecutor.cs:342-349,525`), so a cooperative method need not
continue. Already-committed side effects, queued operations, and noncooperative code can survive;
replaying after a lost response may therefore repeat effects. **Fix:** align an end-to-end deadline
and use stable idempotency keys/result deduplication for retriable operations, or return an
explicit uncertain outcome instead of automatically replaying an ambiguous send. Increasing the
timeout alone does not establish at-most-once behavior. Test both cooperative cancellation and
commit-before-response-loss against two peers. Medium.

**EI-5 — Relayed audit rows lose verified caller IP `[Medium][CONFIRMED; proposed trust rule corrected]`**
`InboundAPI/HttpStandbyRequestForwarder.cs:166-192` supplies no authenticated address provenance;
`Middleware/AuditWriteMiddleware.cs:344` records the standby's remote address. The
`X-ScadaBridge-Forwarded` marker is caller-controlled loop prevention, not peer authentication.
**Fix:** have the standby overwrite verified client provenance and accept it only through a
configured trusted-proxy chain or authenticated peer protocol with bounded hops. Reuse the
trusted forwarded-header setup in `Host/Program.cs:669-675` where appropriate. Never trust XFF
solely because the loop marker exists. Test forged direct headers and a genuine trusted relay.
Small.

**EI-6 — Per-channel retention purge is an unbounded loop on the maintenance path `[Low][CONFIRMED]`** — `AuditLogRepository.cs:699-735` (`while (true) { DELETE TOP … }`) driven serially per channel from `AuditLogPurgeActor.cs:262-325`; a `ScriptTrace`@30 d backlog after `TraceAllCalls` holds the actor and starves later channels. Fix: per-tick row/time budget like `MaxPurgedRowsPerTick`. Small.

**EI-7 — A hung KPI source can stall KPI sampling until actor restart `[Low][CONFIRMED]`** — `KpiHistoryRecorderActor.cs:315-342` awaits `CollectAsync` with only the shutdown token; the in-flight guard stays raised forever. Fix: bound the actual wait, not just a cooperative cancellation token; keep abandoned work/scopes bounded and do not start unlimited replacement tasks. Test a cancellation-ignoring source and later healthy sources. No live production hang was observed. Small.

**EI-8 — The standby relay replays only status + `Content-Type`, while `Accept-Encoding` passes through `[Low][CONFIRMED]`** — `HttpStandbyRequestForwarder.cs:131-141, 206-236` vs `:59-63`; the day response compression is enabled, callers get gzip labelled `application/json`. Fix: strip `Accept-Encoding` or copy response headers minus hop-by-hop. Small.

**EI-9 — Outbox dispatch has no row-level claim; dual-active can repeatedly duplicate notifications `[Low][CONFIRMED]`** — `NotificationOutboxRepository.cs:140-158` (plain read), status written only after delivery (`NotificationOutboxActor.cs:601-637`); under the accepted auto-down dual-active state both singletons can pick overlapping batches and repeatedly deliver the same notifications. `Component-NotificationOutbox.md:112` describes "one notification, rare failover". Fix: widen the doc; optionally an `UPDATE`-claim with a lease. Small (doc) / medium.

**EI-10 — Audit body pre-read disables the configured chunked-request size limit `[Medium][CONFIRMED source path and isolated Kestrel reproduction]`**
`src/ZB.MOM.WW.ScadaBridge.Host/Program.cs:701-708` mounts audit middleware before endpoint
execution. `InboundAPI/Middleware/AuditWriteMiddleware.cs:203-205,495-496` reads body bytes before
`InboundApiEndpointFilter.cs:80-84` attempts to set IHttpMaxRequestBodySizeFeature. Kestrel has
already made it read-only, so chunked requests without Content-Length skip the intended limit and
fall back to the listener/proxy cap. The JSON handler and standby body copy then consume the rest.
An isolated real-Kestrel reproduction of this ordering used global limit 1000, desired limit 8,
and a 100-byte chunked request: **HTTP 200; IsReadOnly=True; actual cap=1000; accepted bytes=100**.
This demonstrates the ordering mechanism, not the entire application pipeline. **Fix:** install the
route-specific cap before any audit buffering/read. Test chunked and HTTP/2 unknown-length JSON
over the configured cap with audit on, asserting 413 and no handler/relay dispatch; include
SkipBodyCapture and declared-length variants. Small.

**EI-11 — Normal audit redaction leaves ErrorMessage unredacted and uncapped `[Medium][CONFIRMED source path, orch-verified]`**
`src/ZB.MOM.WW.ScadaBridge.AuditLog/Redaction/ScadaBridgeAuditRedactor.cs:192-214,235-247` redacts
and caps request, response, ErrorDetail, and Extra while preserving `d.ErrorMessage` unchanged.
`InboundAPI/InboundDatabaseHelper.cs:250` and other emitters put exception.Message there. A global
or target secret pattern can scrub the stack detail yet leave the same value in the short error
message; long exception messages also bypass the advertised field cap. This is broader than EI-3's
SQL-parameter-name policy. **Fix:** apply normal-path body redaction and caps to ErrorMessage too,
and verify the entire serialized event contains no secret marker. Test all error fields with
global/per-target patterns and oversized messages, not just ErrorDetail. Small.

### 2.6 Commons / ConfigurationDatabase / Transport (DL)

**DL-1 — `Instance.UniqueName` is assumed globally unique but the DB enforces only `(SiteId, UniqueName)` `[Medium][CONFIRMED, orch-verified]`** — `InstanceConfiguration.cs:61`; `InstanceLocator.cs:25-26` (what `Route.To` uses to pick the **site**) is `FirstOrDefault(i => i.UniqueName == name)` with no site qualifier or ordering; also `DeploymentManagerRepository.cs:516`, `TemplateEngineRepository.cs:468`. Global uniqueness is an app-side check-then-act (`InstanceService.cs:67-70`) on a per-node `ManagementActor`, so two operators on the two centrals can both commit; `BundleImporter` Rename (:4205-4215) does no check. A duplicate makes `Route.To` route setpoints to the wrong plant with nothing logged. Fix: inventory and resolve existing duplicates, then add a global unique index and translate concurrent/import collisions into domain conflicts. Wrong-site execution remains subject to the caller’s existing site-scope checks. Small.

**DL-2 — `DataConnection.Primary/BackupConfiguration` hold protocol credentials in plaintext at rest, unlike every sibling secret column `[Medium][CONFIRMED, orch-verified]`** — `ScadaBridgeDbContext.cs:334-353` encrypts exactly four columns (SMTP credentials, SMS auth token, ESG auth config, DB connection string); `ManagementActor.cs:1936` itself says the DataConnection JSON "can embed OPC UA user and certificate passwords"; `Component-Transport.md:98` puts them in the encrypted `SecretsBlock`; `GuardSecretWritesHaveAKeyRing` (:258-268) omits the entity. Caveat: the key ring is `PersistKeysToDbContext` in the same DB without `ProtectKeysWith*` (`ServiceCollectionExtensions.cs:109-110`), so the existing encryption is defence-in-depth, not a hard boundary — but this is the one column class an operator would assume covered. Fix: add the converter to both properties + the guard switch + a one-off re-encrypt migration; or record the deferral in the register and the doc. Medium.

**DL-3 — Flow Debugger Replay reads an unbounded audit-row set into memory and over the envelope `[Medium][CONFIRMED]`** — `AuditLogRepository.cs:1182-1230` (`GetTraceEventsAsync`, no `Take`); `FlowDebugRunner.cs:544-553` returns every row. `PullTraceEventsMax = 2000` governs only the site RPC; the Live path is capped 500 events / 1 MB. v1.4's 200 `ScriptLog` rows per execution × a tree, `ApiInbound` rows up to 1 MiB each, and the page polls Replay every 5 s for 2 min. Fix: bounded row paging AND a byte budget, plus truthful `FlowGraph.Truncated` and separately retained root metadata. A 2001-row limit alone can still produce a very large response. Small.

**DL-4 — `FlowGraphBuilder` emits two edges between a write and the wait it satisfied `[Low][CONFIRMED]`** — `:150-152` + `:219-224`; `FlowGraphView` draws both, obscuring the dashed echo edge; the test at `FlowGraphBuilderTests.cs:90` asserts `Single(External)`, not `(From, To)` uniqueness. Fix: coalesce by endpoints with explicit priority for the dashed External echo edge, or omit the ordinary chain edge when it is replaced. Test both unique endpoints and preserved echo semantics. Small.

**DL-5 — Refresh merge violates the builder's input-order contract for late-arriving executions `[Low][CONFIRMED mechanism / PLAUSIBLE impact]`** — `FlowDebugger.razor.cs:707-714` (known-first, then fresh) vs the F5 rule (`FlowGraphBuilder.cs:84-117`); a trigger-cascaded child whose rows arrive by telemetry lands at the tail regardless of when it ran. Fix: base on the (complete) Replay order, append live-only rows. Small.

**DL-6 — A bundle `Overwrite` silently reassigns a deployed instance to a different site `[Medium][CONFIRMED path / PLAUSIBLE operationally]`** — `BundleImporter.cs:4236-4247` rewrites `existing.SiteId` (+ `State = NotDeployed`, and `ReplaceInstanceChildrenAsync` :4419-4438 hard-deletes overrides/bindings). Transport never touches runtime nodes, so the instance keeps running at the old site with no central record binding it there — invisible on the Deployments page. `Component-Transport.md:306-315` never mentions the `SiteId` rewrite. Fix: block the site change on Overwrite when `State != NotDeployed`, or an explicit preview blocker; document. Small.

**DL-7 — Every audit row is JSON-deserialized 2–3× per graph build `[Low][CONFIRMED]`** — `FlowStepProjector.cs:20, 54, 63, 82` from `FlowGraphBuilder.cs:72-73, 386, 419`; ~4–6 k parses per 5 s poll on a 2 000-row tree. Fix: deserialize once into a tuple list. Small.

**DL-8 — Replay's root execution id is picked by timestamp order, which F5 declared untrustworthy `[Low][CONFIRMED]`** — `FlowDebugRunner.cs:552` (`events[0]`) though `ResolveChainAnchorAsync` (`AuditLogRepository.cs:945-1004`) already computes the true root. Fix: return `ChainAnchor.RootExecutionId`. Small.

**DL-9 — Commons documentation omits debugging types and collector ownership `[Low][CONFIRMED documentation gap]`**
`docs/requirements/Component-Commons.md:171-208` omits `Types/Debugging/`. Its REQ-COM-6
`:319-325` already permits self-contained helpers with owned state, so `FlowTraceCollector`
being mutable does not by itself violate the architecture. It collects bounded in-memory data
and accepts a redactor abstraction. **Fix:** document the namespace and per-run collector
ownership/redaction contract; no relocation is justified merely by stateful fields. Small.

**DL-10 — No enum lock-in tests for `FlowLane`/`FlowStepKind`/`FlowStepOutcome` `[Low][CONFIRMED]`** — `AuditEnumTests` covers only the four audit enums; `KindRank`'s display-order contract is unguarded. Fix: member/value contract assertions where serialization requires them, plus a separate `KindRank` display-order test. Existing functional tests already exercise these enums. Small.

**DL-11 — Folded outbound steps retain the first exception detail instead of the final one `[Low][CONFIRMED source path]`**
`src/ZB.MOM.WW.ScadaBridge.Commons/Types/Debugging/FlowGraphBuilder.cs:424-440` creates a folded
step from the first lifecycle row and replaces latest Outcome/ErrorMessage but omits ErrorDetail.
`FlowStepProjector.cs:46` does project each row's detail; lifecycle rows (:82-93) do not retain it.
A submit with null detail followed by a failed terminal event displays no terminal stack, while an
earlier failure's detail can remain after later success. **Fix:** carry latest ErrorDetail with
latest ErrorMessage, with explicit lifecycle detail semantics if needed. Test null-submit→failure
and failure→success. Small.

### 2.7 DataConnectionLayer / CLI / DelmiaNotifier / docker scripts (DC)

**DC-1 — Nothing in production ever sends `UnsubscribeTagsRequest` / `UnsubscribeAlarmsRequest`; the DCL leaks subscriptions for the life of the node `[High][CONFIRMED, orch-verified]`**
Handlers `Actors/DataConnectionActor.cs:1799, 3138`; routing `DataConnectionManagerActor.cs:153, 178`; a repo-wide grep finds senders only in tests — `InstanceActor.cs` and `NativeAlarmActor.cs` contain zero references, and `DataConnectionActor` has no `Context.Watch` at all. The connection actor is never stopped on redeploy (`HandleReconfigureConnection` :118 forwards; `HandleCreateConnection` :75 short-circuits on an existing name). Disable, delete, or redeploy an instance with a narrower attribute set and: the removed tags stay advised on the device (OPC UA monitored items / MxAccess advises never released), `ReSubscribeAll` re-issues them on every reconnect forever, `TotalSubscribedTagCount`/`ResolvedTagCount` never drop, `_subscribers[instanceName]` keeps a dead `IActorRef` that receives a `TagValueUpdate` per value change (dead-letter flood at tag rate), and `_alarmSourceSubscribers` never empties. On a churny site this is unbounded device-side subscription growth. `Component-DataConnectionLayer.md:447, 455` describe the unsubscribe as a live interaction. **Fix:** owner-aware cleanup on stop/undeploy plus DCL DeathWatch. Fence tag removal by actor identity/generation: the current handler at `DataConnectionActor.cs:1804` keys only by instance name, so a late old actor unsubscribe must not remove its same-name replacement. Preserve actual subscriber identity on alarm cleanup too. Test stop cleanup and old-owner cleanup arriving after replacement subscription. Medium.

**DC-2 — Releasing a "redundant" MxGateway handle tears down the LIVE subscription for that tag `[High][CONFIRMED, orch-verified code path]`**
`DataConnectionActor.cs:1642-1653` + `:1703` (`_ = UnsubscribeIdsAsync(msg.Adapter, redundantIds)`), same in `:2500, 2509`. On MxGateway the "subscription id" **is** the MXAccess item handle (`RealMxGatewayClient.cs:193`, `reply.ItemHandle.ToString(...)`), per tag, so the redundant id is normally the live handle — and both removals are identity-blind anyway (`MxGatewayDataConnection.cs:294-304` `_tagToSub.TryRemove(s.TagPath)`; `RealMxGatewayClient.cs:219` `_tagToHandle.TryRemove(tag)`), deleting the mapping the event loop routes on. The trigger is ordinary: `HandleSubscribe`'s dedup partition (:1151) checks `_subscriptionIds` and `_subscribesInFlight` but not `_resolutionInFlight`, so a second instance subscribing to an unresolved tag while its resolution probe (:2277) is in flight double-subscribes. Result: the tag silently stops updating — the attribute freezes at its last Good value, no quality change, nothing in the health report — until the next reconnect. Same class as the QA Z28061 frozen-value incident. **Fix:** (a) include `_resolutionInFlight` in the partition; (b) identity-checked removals (`if (cur == id)`); (c) skip the release when `result.SubscriptionId == _subscriptionIds[tag]`. Test the DCL duplicate-completion path: it must never issue an unsubscribe for the current live handle; separately verify atomic identity-checked reverse-map removal for distinct old/new handles. A direct unsubscribe of the live handle is supposed to remove it. Same-handle behavior was source-traced, not reproduced against the live gateway. Small.

**DC-3 — Device writes are stashed across a reconnect and applied minutes later, contradicting the "no buffered setpoints" invariant `[High][CONFIRMED, orch-verified]`**
`DataConnectionActor.cs:417-422` (Connecting) and `:793-796` (Reconnecting) stash `WriteTagRequest`/`WriteTagBatchRequest`; `:494` `UnstashAll` on `BecomeConnected`. `Component-DataConnectionLayer.md:421`: "There is **no store-and-forward for device writes** … buffering stale setpoints would be dangerous." The script's 30 s Ask (`InstanceActor.cs:787`) times out and the script is told the write failed; the reply dead-letters; the value is still written to the device when the link returns. A 10-minute outage ends with a burst of stale setpoints hitting equipment whose scripts may already have compensated. Secondarily the stash is unbounded (no `stash-capacity` configured) and Lever 1's `SubscribeTagsRequest` re-sends (every ≤ 60 s during the outage) are all stashed and replayed, each a full `SeedTagsAsync`. **Fix:** answer writes in Connecting/Reconnecting with an immediate failed response appropriate to single/batch requests (what the no-buffering contract requires); dedupe stashed subscribes per instance / set `stash-capacity`. Small.

**DC-4 — OPC UA silently downgrades to an unsecured endpoint when the requested `SecurityMode` is unavailable or discovery fails `[Medium][CONFIRMED]`** — `RealOpcUaClient.cs:151-158` (`?? endpoints.FirstOrDefault()`), `:386-392` (catch → `new EndpointDescription(url)`, mode None); no post-connect assertion. A `SignAndEncrypt` connection connects unencrypted and reports green; Verify has the same fallback. Fix: fail the connect when `preferred != None` and no match / discovery threw, naming requested vs available. Small.

**DC-5 — New alarm subscribers can miss standing-state snapshots on both adapters `[Medium][CONFIRMED, expanded]`**
`DataConnectionLayer/Adapters/MxGatewayDataConnection.cs:319-322` skips a fresh source snapshot
when the existing prefix covers it. Independently, `Actors/DataConnectionActor.cs:2881-2886`
returns Success to a new subscriber of an already active source without replaying its state;
that path affects OPC UA too. A newly deployed instance can remain blind to standing conditions
until a later transition/reconnect. The original “OPC UA unaffected” claim was incorrect.
**Fix:** deliver an initial snapshot to each subscriber, using a DCL state replay or a correctly
scoped refresh; handle the Mx prefix optimization separately. Fix DC-12's marker scoping before
increasing refresh traffic. Test distinct covered sources and a second instance sharing one
source on each adapter. Medium.

**DC-6 — `deploy site` exits 0 when its domain result reports failed instances `[Medium][CONFIRMED]`** — `CLI/Commands/CommandHelpers.cs:169-193` returns 0 for any 2xx; `DeployCommands.cs:68-84`. A CI `scadabridge deploy site … || rollback` never fires; other command-specific success payloads need separate contract checks; do not assume every JSON command has the same failure shape. Fix: an `onSuccess` handler inspecting `failureCount` → exit 1; document. Small.

**DC-7 — DelmiaNotifier treats a standby central's 503 as final and never fails over `[Medium][CONFIRMED]`** — `DelmiaNotifier/Notifier.cs:31-52` rolls over only on `ConnectFailed`. Its HA therefore depends entirely on `StandbyForwardPeerEndpoints` being configured on both centrals — which QA (`vq03`/`vq04`) and prod (`vd03`) still lack (CLAUDE.md). Fix: distinguish a known pre-execution standby rejection from an ambiguous post-send failure before retrying; generic retry-on-any-503 can duplicate effects. Document and test the notifier HA contract. QA/prod peer settings quoted here were not remotely rechecked in this pass. Small.

**DC-8 — `secured-write` is a shipping CLI group absent from the README and from `CommandTreeTests` `[Low][CONFIRMED]`** — `CLI/Program.cs:42`; `CommandTreeTests.cs:26-45, 63-71` asserts 18 groups vs 19 in `Program.cs` and builds `data-connection` without `ConnectionBrowseCommands.AddTo`, so browse/search/verify-endpoint/certs are uncovered too. Fix: add to the test (19), `AddTo` in the test build, README section. Small.

**DC-9 — Stale statements in `Component-DataConnectionLayer.md` `[Low][CONFIRMED]`** — `:412` (`volatile bool` → `int` + `Interlocked.Exchange`), `:318` (`UnsubscribeAlarmsRequest` is handled, not stashed, in Reconnecting at `:802`), `:382/384` vs `:421` contradiction on write stashing (DC-3), `:323` prefix restart "costs bandwidth, not correctness" (DC-5). Small.

**DC-10 — Per-item shard planning allocates a list per monitored item on the subscribe hot path `[Low][CONFIRMED]`** — `RealOpcUaClient.cs:765-772` from the loop at `:686`; ~75 k short-lived objects on a 37.5 k-tag reconnect. Fix: hoist / plan per chunk. Small.

**DC-11 — DelmiaNotifier aborts the whole failover loop on a malformed `BaseUrls` entry `[Low][CONFIRMED]`** — `HttpRecipeSender.cs:20-45` catches only `HttpRequestException`/`OperationCanceledException`; a `UriFormatException` on the first URL escapes to `Program`'s outer catch. Fix: validate every BaseUrls entry as an absolute HTTP(S) URI during config load, with a clear diagnostic or explicit invalid-endpoint skip policy. Do not map arbitrary exceptions/cancellation to connect failure. Small.

**DC-12 — OPC UA per-feed snapshot completion clears unrelated active alarms `[High][CONFIRMED source path, orch-verified; live multi-feed gate pending]`**
`src/ZB.MOM.WW.ScadaBridge.DataConnectionLayer/Adapters/RealOpcUaClient.cs:867-909` registers
per-source monitored callbacks. `HandleAlarmEvent:1165-1168` emits an unscoped SnapshotComplete
for each feed's RefreshEnd. `Actors/DataConnectionActor.cs:3021-3027` broadcasts each marker to
all alarm subscribers. `SiteRuntime/Actors/NativeAlarmActor.cs:251-282` swaps/clears its snapshot
buffer on every marker. After A commits its snapshot, B's subsequent completion reaches A with
an empty buffer, generating false return-to-normal/removal of still-active conditions. Multiple
feeds are ordinary configuration; the scope is lost inside the application. **Fix:** preserve
feed/source identity and refresh epoch on snapshot boundaries, keeping Mx connection-wide stream
semantics separate. Test interleaved A/B snapshot/end delivery and assert completing B cannot
change A, then validate two real OPC UA alarm sources on the rig. Medium.

### 2.8 Cross-cutting (XC) and orchestrator (ORCH)

**XC-1 — Central's entire HTTP endpoint graph is served on the site-facing h2c gRPC listener, bypassing Traefik `[Medium][CONFIRMED, orch-verified]`** — `Host/Program.cs:228-262` binds the HTTP ports (`Http1AndHttp2`) and `centralGrpcPort` (`Http2`) on one `WebApplication`; endpoint routing is not listener-scoped and there is no `RequireHost`/`MapWhen` anywhere in `src/`. Every endpoint mapped at `:711-748` — Central UI, `/management`, `/api/*`, `/api/audit`, `/metrics`, `/health/*`, the SignalR hub — is also served on `:8083`, the port every site dials and the rig publishes (`docker-compose.yml:131, 198`). `curl --http2-prior-knowledge http://central-a:8083/metrics` reaches metrics and health **anonymously**, and the authenticated surfaces with credentials, on a port not behind Traefik — so the `/health/active` active-node routing is bypassed. Register row 56 deliberately made central `/metrics` opt-in on a separate port for this reason. Fix: enforce listener ownership using the actual `HttpContext.Connection.LocalPort` or separate pipelines/listeners, rejecting non-control-plane endpoints before dispatch. `RequireHost` matches caller-supplied Host/authority and is not a port-isolation boundary. Test an h2c call to the gRPC socket with authority set to the HTTP port, and the inverse; preserve intentional metrics/health access. The original curl example is a reproduction recipe, not a live probe run in this pass. Small.

**XC-2 — Four SMTP keys in all four rig central appsettings bind to nothing `[Medium][CONFIRMED, orch-verified]`** — `docker/central-node-*/appsettings.Central.json:84-89` and env2 set `Notification:{SmtpServer, SmtpPort, AuthMode, FromAddress}`; `NotificationOptions` (`NotificationService/NotificationOptions.cs:13-25`) has only `ConnectionTimeoutSeconds` and `MaxConcurrentConnections`; zero code hits. Real SMTP config is the DB `SmtpConfiguration` row. An operator editing the host/port here changes nothing, silently. Fix: delete the keys, leave a `_comment` pointing at `notification smtp` / `/notifications/smtp`. Small.

**XC-3 — `SiteEventLog:PurgeScheduleCron` is set on all 8 site nodes and read by nothing `[Medium][CONFIRMED, orch-verified]`** — `docker/site-*/appsettings.Site.json:75` (`"0 2 * * *"`); no `Cron` in `src/`. The purge is a `PurgeInterval` delay loop anchored to the previous purge's end, so it drifts to whatever time the node last restarted, and a 2.7 M-row retention burst can land mid-shift while the file promises 02:00. Fix: remove the key (small) or implement it (medium).

**XC-4 — The vendored gRPC C# has no guard tying it to the `.proto` `[Medium][CONFIRMED]`** — `Communication.csproj:64-70` (commented-out `<Protobuf>`, correctly); no test reads a `.proto`. Editing a proto and forgetting `regen-proto.sh` builds green and ships the stale wire shape — invisible because both ends compile from the vendored file, until a third party generates from the real proto. Checked by hand: 565 fields, 0 mismatches today. Fix: a ~40-line test that parses `Protos/*.proto` for `name = n;` and asserts each `PascalCase(name) + "FieldNumber" == n` on the generated types. Small.

**XC-5 — README, HighLevelReqs, Component-ManagementService and the components skill still describe ClusterClient as the live transport `[Medium][CONFIRMED]`** — `README.md:83, 96`; `HighLevelReqs.md:49-51`; `Component-ManagementService.md:5, 16, 21, 48, 50, 301, 307` (a whole "ClusterClientReceptionist Registration" section, contradicted by line 27 of the same file); `Component-NotificationOutbox.md:31`; `Component-Commons.md:292`; `.claude/skills/scadabridge-components/SKILL.md` entries 5 and 18. These are the entry-point documents an agent loads first. Fix: rewrite to gRPC both directions + `SiteStreamService` + token-gated HTTP config fetch; delete the receptionist section. Small.

**XC-6 — Dead site-hosted ingest RPCs `[MERGED into CM-5 — not separately counted]`**
This was the same pair of handlers already recorded under CM-5, assigned a different severity by
a second reviewer. Retain CM-5 as the single Low-priority implementation finding. Preserve proto
compatibility when retiring handlers; return `Unimplemented` rather than silently acknowledging.

**XC-7 — Migration-only legacy keys still shipped in every rig site appsettings `[Low][CONFIRMED]`** — `ScadaBridge:Database:SiteDbPath` and `ScadaBridge:StoreAndForward:SqliteDbPath` in all 8 site files (both rigs), which CLAUDE.md says to delete once migrated; the other two named keys are already gone. Fix: delete. Small.

**XC-8 — Both gRPC auth interceptors are allow-by-default for unlisted service prefixes, with no coverage test `[Low][CONFIRMED]`** — `ControlPlaneAuthInterceptor.cs:164-168, 197-207`; `CentralControlAuthInterceptor.cs:50-51, 197-201` (`if (!IsGated(method)) return;` over a hard-coded allowlist). The existing tests assert the prefixes match descriptor names, not that every `MapGrpcService` is covered; a fourth service ships unauthenticated with no failing test. Fix: a test reflecting over the mapped services against the union of gated sets. Small.

**XC-9 — Package version skew: `Microsoft.Data.SqlClient` pinned below EF's floor, `SignalR.Client` one major behind, one prerelease `[Low][CONFIRMED]`** — `Directory.Packages.props:41` (6.0.2) with `VersionOverride="6.1.1"` in three test csprojs ("EF SqlServer 10.0.7 needs ≥ 6.1.1 … override is test-only"): ESG compiles against 6.0.2 while the Host resolves 6.1.1 at runtime; `SignalR.Client 9.0.3` in a 10.0.7 family; `bunit 2.0.33-preview`. Fix: raise the central pin to 6.1.1 and delete the overrides; evaluate SignalR 10. Small.

**XC-10 — `OpcUaGlobalOptions` / `MxGatewayGlobalOptions` are the only bound options with neither `ValidateOnStart` nor a validator `[Low][CONFIRMED]`** — `DataConnectionLayer/ServiceCollectionExtensions.cs:32-36`; every sibling chains both (arch-review 08 §1.5). A bad timeout surfaces as a hang at first connect, not at boot; masked today because neither section is set on either rig. Fix: validators + `ValidateOnStart`. Small.

**XC-11 — Umbrella index is two features behind, and `main` is 47 commits ahead of the local origin/main ref `[Low][CONFIRMED]`** — `scadaproj/CLAUDE.md:41` pins `main @ 789377ef` (2026-09-06); local `main` = `c078bc8f`; the instance picker and v1.4 step logs are unrecorded; `origin/main..HEAD` = 47 against the locally cached ref (no remote fetch in this review). Fix during a separately authorized integration/push: update the row (short, per the index's own rule). Small.

**XC-12 — `RequiredSingletonsHealthCheck`'s doc miscounts the central singletons `[Low][CONFIRMED]`** — `Host/Health/RequiredSingletonsHealthCheck.cs:32-38` says "all five … unconditional"; there are seven (`AkkaHostedService.cs:518, 551, 619, 664, 687, 718, 744`), two excluded as best-effort — a reason the comment's two-branch rule would get wrong. Fix: amend the remark. Small.

**ORCH-1 — `OperationTrackingStore.Dispose()` disposes the write connection under a live pump transaction `[Medium][CONFIRMED, observed in the run]`** — `SiteRuntime/Tracking/OperationTrackingStore.cs:616-634` completes the channel then `_writeConnection.Dispose()` immediately, while the `Task.Run(PumpAsync)` pump may be inside `using var tx = _writeConnection.BeginTransaction()` (:388, :407). `SqliteConnection.Dispose` rolls back the live transaction on the disposing thread while the pump concurrently completes it → `NullReferenceException` inside `SqliteTransaction.Complete()`. Recorded in the **first-pass** suite run: `OperationTrackingStoreTests.SR024_SyncDispose_DoesNotDeadlock_WhenInvokedFromFreshThread` failed with exactly that stack (§7). Production disposes the host singleton through `DisposeAsync` (which drains), so this is a latent hazard on the sync path plus a flaky test, not an outage. Fix: complete/cancel and join the pump before disposing its resources. If a bounded wait expires, defer disposal until completion; never close the connection/gate beneath a live pump. Pin the race with a controlled pump barrier and include sync-context safety; a passing timing-sensitive test does not disprove the source race. Small.

---

## 3. Verified strengths and retained first-pass inventories

These inventories originate in the first pass. Production source has not changed, and the second pass rechecked the implicated boundaries plus the dependency graph. This is not a second exhaustive rerun of every historical inventory check. Qualifications below and the current findings take precedence:

- **Site Runtime:** a stop does not cancel an in-flight run; deadlines armed at enqueue and shed at dequeue on every path; watchdog detach/replace bookkeeping (`ScriptExecutionScheduler.cs:275-379`); `MaxConcurrentRunsPerScript` counter cannot leak; **no awaited audit write on any script path** (eight SiteRuntime sites + the forwarder, all enqueue-and-observe on `TaskScheduler.Default`); `TraceCapturingLogger` double-wrap guard and the shared-script logger sharing; `TraceStepScope` restore; log caps + `FlushDropped` once + collector reserve arithmetic; Lever 1 in full; `OperationTrackingStore` read-your-writes barrier; `Notify.Send` `maxRetries: 0` + the `MaxRetries > 0` park test; purge pacing pure and faulted ticks don't arm catch-up; trigger eval off-pool with coalescing.
- **Communication / Host:** every mapped gRPC service is behind a fail-closed `FixedTimeEquals` interceptor (`CentralControlAuthInterceptor`, `SecretsHubAuthInterceptor` via the shared `UsesGrpcHub` predicate, `ControlPlaneAuthInterceptor` on stream + command, `LocalDbSyncAuthInterceptor`); proto diff vs `ee193cd2` is additive-only with vendored C# in sync and `regen-proto.sh` restoring the csproj on every exit path; batching negotiation via the proto3 default; `ActiveNodeEvaluator` is the sole active definition (`State.Leader` appears only in two "never use" comments); `BuildHocon` emits `AutoDowning`, `min-nr-of-members = 1` pinned, empty seed list under the guard; seed self-first enforced and true in all 14 appsettings; channel park-then-sweep, `(site, endpoint)` factory key, `TryGet` on teardown; reconnect on OK completion wired through both consumers; single-endpoint sites collapse correctly; bounded DropOldest channels; options validated on start everywhere but XC-10; correlation-id path-element guard; Traefik probes `/health/active`.
- **Management / Deployment / Templates:** deployment-config token boundary (existence → TTL → token, 256-bit CSPRNG, `FixedTimeEquals`); secured writes CAS on `Pending` for approve/reject with self-approval and scope checks before the CAS (no oracle); bulk deploy failure isolation end to end (P7-3/5/6), `Disabled` wins under `--force`, conservative `IsUpToDateAsync`; `OperationLockManager` ref-counted with CAS-idempotent release on exactly the four mutating commands; `PendingDeployment` per-instance under a 1-permit gate; revision-hash determinism guarded by test; acyclicity via both cycle detectors; all four Roslyn surfaces attach `CachingScriptMetadataResolver.Instance`; compile-surface parity pinned; no `Task.Run`/`.Result`/`.Wait()` in the four projects; `RequiredRoleMatrixTests` (145 entries) + `DispatchCoverageTests` cover the four `FlowDebug*` commands.
- **Central UI / Security:** no third-party component framework; no sync-over-async on any render/event/lifecycle path (the 4769e192 fix intact, `MaxPredictionPasses = 8`); `DisableLoginGuard` refuses outside Development, runs before `AddSecurity`; cookie hardening and the two validator inequalities; LDAP-failure semantics (`Keep` on refresh fault, DB-only refresh); JWT key ≥ 32 bytes, `ClockSkew = Zero`; `LoginThrottle` per user|ip and per-ip, bounded; forwarded headers opt-in, `ForwardLimit = 1`, mounted first; sandbox trust gate before execution and for nested `CallShared`; `InstancePicker`'s reference contract honoured by both callers; report pages server-paged; bundle upload size-capped. All 43 routable pages carry an authorization attribute (login `AllowAnonymous`; dashboard + health bare `Authorize`; admin/*, certificates, transport import, SMTP/SMS `RequireAdmin`; design/* + notification lists `RequireDesign`; deployment/*, monitoring/*, reports `RequireDeployment`; flow debugger `RequireFlowDebugger`; audit/* `OperationalAudit`; secured writes `RequireSecuredWriteAccess`; `/admin/secrets` external).
- **Edge / audit backbone:** enqueue-and-observe on all eleven script/request-thread audit writes (awaited only on actor paths); redactor on every persistence path including the inline Live copy, failing closed on a bad pattern; site retention SQL purges only `Forwarded`/`Reconciled`, `MarkForwarded` after the central ack; central append-only guard + per-channel purge validated to `[30, RetentionDays]`; SiteCallAudit insert-first then rank-guarded update with `COALESCE(SourceNode)`; inbound auth constant-time with single 401/403 bodies and the scope check before the DB; ESG single deadline, ordered catch filters, bounded read, verb allowlist, escaping, auth never logged, poison parks; EWS https at the sender, `CredentialRedactor.Scrub` on every adapter error, adapters resolved per delivery scope; exactly one `Meter` in `src/` and it plus `LocalDbMetrics` are allowlisted on both roots; health sequence numbers seeded from unix ms; the middleware rewinds the body in `finally` so the relay sees it; KPI per-source fault isolation.
- **Data layer:** Commons has zero project references and the graph is an acyclic DAG (§6.A); every changed record since August adds trailing `default(T)`-safe parameters only; `AuditKind.ScriptLog` and `AuditChannel.ScriptTrace` are last and stored as strings; all 8 migrations non-empty with faithful `Down` and no destructive statements; model snapshot == last Designer target model; hot-path index/query matches on SiteCalls, Notifications, KpiSample, DeploymentRecords; UTC converters on the audit read path; no FK on any audit-bearing table, `Restrict` on the instance/site/connection edges; idempotent ingest on all three central stores; bundle import inside the execution strategy with tracker clears and explicit rollback; the import trust gate is Pass 0, fail-closed, over all six script kinds; secrets presence-only in diffs; `LineDiffer` cap before Myers; `ScriptArtifactsChanged` covers `Add`; `DeploymentRecord` rowversion real including the delete stub path.
- **DCL / CLI / scripts:** adapter-generation fencing on every completion message and the P2-6 demotion; no overlapping reconnects; OPC UA item accounting leak-free under `AsyncSerialGate`; adapter dispose/reset order; health counts derived; the verify probe never trusts; `TrustServerCertCommand` fans to a per-node `CertStoreActor`, Admin-gated; `AckTime` semantics per spec; CLI timeout honesty; no secrets echoed anywhere; README ↔ code parity on 134 leaf commands (DC-8 the only gap); all ten docker scripts `set -euo pipefail`, rig test creds only, worktree guard, `DELMIA_API_KEY` required.

---

## 4. Doc drift (consolidated)

| Doc | Says | Actually | Finding |
|---|---|---|---|
| `README.md:83, 96`; `HighLevelReqs.md:49-51`; `Component-ManagementService.md:5,16,21,48-50,301,307`; `Component-NotificationOutbox.md:31`; `Component-Commons.md:292`; components skill entries 5, 18 | ClusterClient / ClusterClientReceptionist is the command/control transport | gRPC both directions since Phase 4 (2026-07-23); `Component-ManagementService.md` contradicts itself at line 27 | XC-5 |
| `Component-SiteRuntime.md:482` | full Script Runtime API available to alarm on-trigger runs | no integrations wired | SR-2 |
| `Component-StoreAndForward.md:79` | retry-exhausted notifications park | `maxRetries: 0`, park test `MaxRetries > 0`; contradicts line 52 | — |
| `Component-StoreAndForward.md` sweep section | ordered `created_at ASC` drain | per-`(Category, Target)` lanes at `SweepTargetParallelism = 4`, lane abandoned on first transient failure — undocumented | — |
| `Component-DataConnectionLayer.md:421` vs `:382/384` | no buffering of device writes / writes are stashed | stashed and replayed with no TTL | DC-3 |
| `Component-DataConnectionLayer.md:447, 455, 358` | Site Runtime sends unsubscribes | no production sender | DC-1 |
| `Component-DataConnectionLayer.md:412, 318, 323` | `volatile bool`; alarm requests stashed while Reconnecting; prefix restart costs bandwidth only | `int`+`Interlocked`; unsubscribe handled; also costs the snapshot | DC-9, DC-5 |
| `Component-ScriptAnalysis.md:38` | reflection forbidden entirely | `RuntimeHelpers`/`Unsafe`/`AssemblyLoadContext` allowed | MG-1 |
| `Component-ManagementService.md` | command catalog | missing all four `FlowDebug*` | MG-10 |
| `CLAUDE.md` | "frozen 138-command authorization matrix" | 145 entries | — |
| `Component-InboundAPI.md:366`; `CLAUDE.md` | relayed call audited once; peer response replayed unchanged | zero rows when no peer served; headers dropped | EI-1, EI-8 |
| `Component-NotificationOutbox.md:112` | one duplicate on a rare failover | continuous duplication under dual-active | EI-9 |
| `Component-CentralUI.md:89, 285, 311, 317` | data-connection pages Admin; audit pages Admin/Audit; LDAP mapping three roles | `RequireDesign`; `OperationalAudit` = {Admin, Viewer}; six roles | — |
| `Component-CentralUI.md:311-315` | configuration audit shows before/after per change | many direct UI mutations lack rows; audited service paths exist; only `AfterStateJson` exists | UI-1 |
| `Component-Security.md:54`; `SessionExpiry.razor:24-26` | idle session rejected; ping never keeps a session alive | ping slides the cookie every minute | UI-2 |
| `Component-CentralUI.md` | — | Test Run / sandbox surface undocumented | — |
| `Component-Commons.md:171-208, 309-327` | `Types/` subfolder list; pure helpers only | `Types/Debugging/` missing; `FlowTraceCollector` per-run state needs documentation, already compatible with the owned-state carve-out | DL-9 |
| `Component-ConfigurationDatabase.md` | — | encrypted vs plaintext secret columns never enumerated | DL-2 |
| `Component-Transport.md:298, 306-315` | `UniqueName` identity key; FK rewiring list | not DB-enforced globally; `SiteId` rewrite omitted | DL-1, DL-6 |
| `Component-Communication.md:114` | `IngestCachedTelemetry` live | dead | CM-5 |
| `docker/*/appsettings*` | SMTP keys; 02:00 cron purge; legacy DB paths | none bound | XC-2, XC-3, XC-7 |
| `RequiredSingletonsHealthCheck.cs:32-38`; `ScopeAccessors.cs:12-21`; `TriggerEvalGate.cs:25-26`; `StaleInstanceProbe.cs:28-33`; `FlowDebugRunner.cs:56-63`; `ManagementActor.cs:685-686`; `SitePairChannelProvider.cs:57-62`; `AkkaHostedService.cs:565-568`; `CommandTreeTests.cs:22-24` | code comments | each contradicted by the code beside it | XC-12, SR-1, SR-9, MG-3, MG-5, MG-9, CM-7, CM-5, DC-8 |
| `scadaproj/CLAUDE.md:41` | `main @ 789377ef` | `c078bc8f`, two features later, 47 commits ahead of cached origin/main | XC-11 |

---

## 5. Recommended remediation program

Grouped in the shape of `2026-08-14-arch-review-remediation-plan.md`. Packages now overlap
in shared wrappers, actors, and security paths; assign file ownership and sequence those changes
before parallel implementation. Package numbers remain stable, but the second-pass order is risk-based: start WP1/WP2 and
CM-9 (WP4), CM-10 (WP6), UI-13 (WP5) before lower-severity hygiene. Scope WP5 sandbox
containment explicitly; do not implement its withdrawn release-on-timeout shortcut.

| WP | Findings | Theme | Effort |
|---|---|---|---|
| **WP1 — DCL subscription lifecycle** | DC-1, DC-2, DC-3, DC-5, DC-12, SR-3, SR-5, SR-12, DC-4, DC-10 | owner-fenced unsubscribe + DeathWatch; source-scoped alarm snapshots + rehydration fence; async stop/flush ordering; profile shard-planning allocation; identity-checked handle release + `_resolutionInFlight` in the partition; fail writes while disconnected; snapshot on every new alarm source; port Lever 1 to `NativeAlarmActor`; no silent security downgrade | medium — needs a rig rebuild + the Lever-1-style live gate (subscribe count on a redeploy, frozen-tag probe) |
| **WP2 — Script pool + trust** | SR-1, SR-2, MG-1, SR-4, SR-11, MG-8, SR-7, SR-8, SR-9, SR-10 | `ConfigureAwait(false)` on the accessor paths; three deny-list entries; close connection/transaction escape surfaces; wire alarm integration scope; gate shared scripts at save | small/medium; unit and integration tests |
| **WP3 — gRPC sizing** | CM-1, CM-3, XC-4, XC-8 | explicit receive limits both roots + channels; byte budget on the audit drain/pull; batch-size ceilings; proto↔C# guard test; interceptor coverage test | medium |
| **WP4 — Audit completeness + scope** | UI-1, EI-1, EI-2, EI-3, EI-5, EI-11, MG-2, CM-9, UI-8 | UI mutations through the management envelope (or the `IAuditService` stop-gap); relay audit ownership; redact/suppress all sensitive error fields; trusted caller IP; bind PSK identity to payload ownership; site scope on tree/Replay/relay | large UI boundary work plus small/medium security fixes |
| **WP5 — Session + UI hygiene** | UI-2, UI-13, UI-14, UI-4, UI-5, UI-6, UI-7, UI-10, UI-12, UI-9, UI-11 | real activity tracking; circuit grant revalidation; bounded output; observed poll cleanup; isolated/quarantined sandbox work; service auth; route sweep; explicit login antiforgery | mixed; sandbox isolation requires its own design |
| **WP6 — Failover-safe apply** | MG-3, MG-4, CM-10, EI-4, SR-13, DL-1, DL-6, ORCH-1, EI-9, DL-2 | cross-node/shared revision invalidation; batch-derived bulk-deploy deadline or submit-and-poll; aligned deadlines + operation deduplication; recheck S&F gate per attempt; global `UniqueName` index; block cross-site Overwrite; sync-dispose ordering; outbox doc/claim; credential encryption migration or explicit DL-2 deferral | mixed; includes one migration and operation-idempotency work |
| **WP7 — Flow Debugger follow-ups** | DL-3, MG-7, MG-6, MG-5, DL-4, DL-5, DL-7, DL-8, DL-10, DL-11, SR-6 | cap Replay + write-kind results; page the site pull (closes register row 117); root id from anchor; preserve echo edge; order-aware merge; final exception detail; parse once; enum tests | small changes across the listed paths |
| **WP8 — Central listener + config hygiene** | XC-1, EI-10, XC-2, XC-3, XC-7, CM-2, CM-6, XC-10, XC-9, CM-5 (alias XC-6), CM-4, CM-7, CM-8, MG-9, EI-6, EI-7, EI-8, DC-6, DC-7, DC-11 | actual local-port enforcement + body cap before audit reads; delete inert keys; env2 bootstrap guard; retire dead handlers compatibly; validators; positive page-size clamp; SqlClient pin; CLI exit codes; notifier safe pre-dispatch failover | mostly small changes plus integration tests and a rig config sweep |
| **WP9 — Docs sweep** | XC-5, XC-11, XC-12, MG-10, DC-8, DC-9, DL-9, SR-2 doc half, every row of §4 | ClusterClient purge from the entry-point docs; register rows for anything deferred (DL-2 if deferred, EI-9); umbrella index/integration status review | small |

Phase gate after WP1–WP3 and WP6: full solution build + test, `bash docker/deploy.sh`, then the
existing live probes (`failover-drill.sh`, `verify-delmia-notifier.sh` A–J) plus these required new regression gates: narrowed redeploy drops subscriptions without removing
same-name replacement ownership; disconnected writes never replay; two OPC UA alarm feeds retain
independent snapshots; cross-site payload with another site's credential yields no fetch token;
response loss after write produces no second effect; revoked grants cannot act from an existing
circuit; transaction-derived commands remain audited; chunked oversized input is rejected before
handler/relay dispatch. These are remediation acceptance gates, **not tests claimed to have run
against the current rig**.

---

## 6. Inventories (from the cross-cutting pass)

### 6.A Project dependency graph (`ZB.MOM.WW.ScadaBridge.` stripped)

```
Commons               -> (none)                 [PackageRef: ZB.MOM.WW.Audit only]
DelmiaNotifier        -> (none)                 [standalone AOT exe]
CLI, ClusterInfrastructure, ConfigurationDatabase, KpiHistory, ScriptAnalysis,
Security, SiteEventLogging                      -> Commons
StoreAndForward       -> Commons, SiteEventLogging
HealthMonitoring      -> Commons, StoreAndForward
Communication         -> Commons, HealthMonitoring
TemplateEngine        -> Commons, ScriptAnalysis
NotificationService   -> Commons, StoreAndForward
ExternalSystemGateway -> Commons, StoreAndForward
DataConnectionLayer   -> Commons, HealthMonitoring, SiteEventLogging
AuditLog              -> Commons, Communication, ConfigurationDatabase
DeploymentManager     -> Commons, Communication, TemplateEngine
NotificationOutbox    -> Commons, KpiHistory, NotificationService
InboundAPI            -> AuditLog, Commons, Communication, ScriptAnalysis
Transport             -> Commons, ConfigurationDatabase, ScriptAnalysis, TemplateEngine
SiteCallAudit         -> AuditLog, Commons, Communication, ConfigurationDatabase, KpiHistory
SiteRuntime           -> Commons, Communication, DataConnectionLayer, HealthMonitoring,
                         ScriptAnalysis, SiteEventLogging, StoreAndForward
ManagementService     -> Commons, Communication, DeploymentManager, HealthMonitoring,
                         InboundAPI, Security, TemplateEngine
CentralUI             -> Commons, Communication, DeploymentManager, HealthMonitoring, KpiHistory,
                         ManagementService, NotificationOutbox, ScriptAnalysis, Security,
                         TemplateEngine, Transport
Host                  -> all 22 component projects
```
No cycles. Nothing but tests references Host. Two cross-role assembly loads are noted, not defects:
`AuditLog → ConfigurationDatabase` (central-only EF, loaded on sites via `AddAuditLog`) and
`Communication → HealthMonitoring → StoreAndForward` (site-only, loaded on central).

Shared-lib versions in use: Health 0.3.0 · Telemetry 0.1.0 · MxGateway.Client 0.2.1 ·
Configuration 0.1.0 · Auth.* 0.3.0 · Audit 0.1.0 · Theme 0.4.1 · Secrets.* 0.6.2 · LocalDb.* 0.3.4 —
all matching the umbrella index. Akka 1.5.62, Grpc 2.76.0 / Protobuf 3.34.1, Microsoft.* 10.0.7
uniform; skew only as XC-9.

### 6.B Composition roots

| | Central | Site |
|---|---|---|
| Cluster singletons | `notification-outbox`, `audit-log-ingest`, `site-call-audit`, `audit-log-purge`, `site-audit-reconciliation` (required-for-readiness), `kpi-history-recorder`, `pending-deployment-purge` (best-effort, excluded) | `deployment-manager`, `event-log-handler` (both `role: site-{SiteId}`) |
| Per-node actors | `dead-letter-monitor`, `central-communication`, `management` | `dead-letter-monitor`, `site-communication`, `dcl-manager`, `cert-store`, `parked-message-handler`, `SiteReconciliationActor`, `site-audit-telemetry` |
| Hosted services | Akka, bootstrap coordinator, inbound key import (SqlServer), script-artifact subscriber, audit partition maintenance, bundle session eviction, health aggregator, health report loop | Akka, bootstrap coordinator, health report sender, storage initializer, event-log purge, audit backlog/retention reporters, scheduler stats, stream drop reporter, run-summary flush, event-log failure counter, LocalDb replication status |
| gRPC | `CentralControlGrpcService` + secrets hub | `SiteStreamGrpcServer`, `SiteCommandGrpcService`, LocalDb sync |
| Health checks | database, akka-cluster, required-singletons (Ready); active-node + database (Active) | akka-cluster, localdb (Ready); `SitePairActiveNodeHealthCheck` (Active) |

No duplicate hosted services; `TryAddEnumerable` used correctly; only XC-10's two options classes lack validation.

### 6.C Cross-cluster RPC inventory

| Service (host) | RPCs | Interceptor | Caller |
|---|---|---|---|
| `SiteStreamService` (site) | `SubscribeInstance`, `SubscribeSite`, `PullAuditEvents`, `PullTraceEvents`, `PullSiteCalls` — and dead `IngestAuditEvents`, `IngestCachedTelemetry` | `ControlPlaneAuthInterceptor` | central stream client / audit pull clients |
| `SiteCommandService` (site) | `ExecuteLifecycle`/`Opcua`/`Query`/`Parked`/`Route`, `TriggerFailover` | `ControlPlaneAuthInterceptor` | `GrpcSiteTransport` |
| `CentralControlService` (central) | `SubmitNotification`, `QueryNotificationStatus`, `IngestAuditEvents`, `IngestCachedTelemetry`, `ReconcileSite`, `ReportSiteHealth`, `Heartbeat` | `CentralControlAuthInterceptor` | `GrpcCentralTransport` |
| `LocalDbSync` (site) | library | `LocalDbSyncAuthInterceptor` | peer site node |
| Secrets hub (central) | library | `SecretsHubAuthInterceptor` | site secrets sweep |

Zero removed field lines in `git log -p -- Protos/` since 2026-06-01; 565 fields, 0 proto↔C# mismatches.

### 6.D Config contract deltas

Read by code, set on no rig (silently defaulted): `ScadaBridge:{OpcUa, MxGateway, DeploymentManager, SiteCallAudit, ExternalSystemGateway, Sms, SiteRuntime, OperationTracking}`, `Telemetry:{Exporter, OtlpEndpoint}`, `AuditLog:{SiteTelemetry, Purge, Reconciliation, PartitionMaintenance, SiteRetention}`, `SiteEventLog:{PurgeInterval, MaxPurgedRowsPerTick, PurgeCatchUpInterval, QueryPageSize, WriteQueueCapacity}`.
Set on the rig, read by nothing: XC-2 (4 SMTP keys), XC-3 (`PurgeScheduleCron`), XC-7 (2 legacy paths).
Required keys not checked by `StartupValidator`: `LocalDb:Path` (delegated to `AddZbLocalDb`), `Node:CentralGrpcPort` (defaulted). Every credential literal on both rigs is the documented dev set with its `${secret:}` form named inline.

### 6.E Deferred-work register (rows 90–118)

Open: 92, 94, 107, 109, 110, 112, 113, 115, 116, 117, 118 (114 never issued). None should be closed; every open premise re-verified. Row 112 should be re-worded (central now has `Truncated` + `LogsTruncated`; the site half is still unflagged). Row 117 confirmed and extended by MG-6. No dangling row citations in code. New rows to mint from this review if deferred: DL-2, EI-9, XC-3 (if not implemented).

### 6.F Test-suite shape (first-pass inventory; fresh execution in §7)

≈ 7 579 `[Fact]`/`[Theory]`/skippable attributes across 32 test projects, all in the slnx; zero static `Skip=`; runtime `Skip.If` concentrated in the MSSQL-fixture and Playwright suites (environment-conditional, not visible in a count); no retry/flaky markers; 56 assertion-less candidates all sampled to throwing helpers — no genuine no-op test found.

### 6.H Design commitments

All twelve checked HOLD with a code anchor: active-node definition (`ActiveNodeEvaluator.cs:43`, no `State.Leader` use); single `"scadabridge"` ActorSystem (`AkkaHostedService.cs:190`); UTC (`DateTime.Now`/`DateTimeOffset.Now`/`Today` → 0 hits in `src/`); no ClusterClient; `notification_lists`/`smtp_configurations` unregistered (`SiteLocalDbSetup.cs:60-72`); auto-down emitted (`AkkaHostedService.cs:283-285`); resolver on all four compile surfaces; no `IHealthCheckPublisher`; central never buffers; single trust boundary at five call sites; additive protos; no active `<Protobuf>` item.

---

## 7. Verification evidence

### 7.A Fresh second-pass commands at `c078bc8f`

All commands ran from the repository root on 2026-09-09. Build/test used existing restored
assets; package listing queried the configured feeds. No production source, tests, package pins,
or rig configuration were changed by this review.

| Check | Exact command | Result |
|---|---|---|
| Build | `dotnet build ZB.MOM.WW.ScadaBridge.slnx --no-restore` | exit 0; **0 warnings, 0 errors**, 34.02 s |
| Selected solution tests | `dotnet test ZB.MOM.WW.ScadaBridge.slnx --no-build --no-restore --filter 'FullyQualifiedName!~PlaywrightTests' --logger trx --results-directory /tmp/scadabridge-review-tests` | exit 0; **9,232 passed, 0 failed, 0 skipped** in 29 nonempty assemblies; 30 TRX files include the empty filtered Playwright assembly |
| Formatting | `dotnet format ZB.MOM.WW.ScadaBridge.slnx --verify-no-changes --no-restore --verbosity quiet` | exit 2; **1,290 WHITESPACE diagnostics across 60 files**; no formatter mutations |
| Vulnerabilities | `dotnet package list --project ZB.MOM.WW.ScadaBridge.slnx --include-transitive --vulnerable --format json --no-restore` | exit 0; 58 projects inspected; one reported dependency: **AngleSharp 1.1.1, Moderate, test-only CentralUI.Tests** |
| Deprecations | `dotnet package list --project ZB.MOM.WW.ScadaBridge.slnx --deprecated --format json --no-restore` | exit 0; **xunit 2.9.3 marked Legacy in 30 test projects**; feed suggests xunit.v3 |
| Dependency graph | XML parsing of all source `.csproj` ProjectReference elements, DFS cycle check | **26 nodes, 91 edges, no cycles; Commons has zero project references** |
| Report integrity | parse finding headings, verify active tally/unique IDs, `git diff --check`, inspect final diff | 92 active IDs; 13 High / 33 Medium / 46 Low; UI-3 and XC-6 excluded |

AngleSharp's [advisory](https://github.com/advisories/GHSA-pgww-w46g-26qg) is already explicitly
suppressed only in `tests/ZB.MOM.WW.ScadaBridge.CentralUI.Tests/ZB.MOM.WW.ScadaBridge.CentralUI.Tests.csproj:64`,
with compatibility rationale recorded in that file and `Directory.Packages.props`. This pass
reports that accepted test-only exposure; it does not assert current patched bUnit compatibility
or recommend silently overriding its dependency. xUnit deprecation is a migration-planning signal,
not proof of a current runtime defect. No version changes were made.

The first pass recorded **9,231 passed / 1 failed**, with a NullReferenceException in
`OperationTrackingStoreTests.SR024_SyncDispose_DoesNotDeadlock_WhenInvokedFromFreshThread`.
That same test **passed in this fresh run**. ORCH-1 remains a source-confirmed race; this pass did
not reproduce the historical stack and does not claim it is fixed. Passing tests also do not
cover the newly identified unsafe paths unless explicitly noted below.

Raw evidence is local to this review session: `/tmp/scadabridge-review-build.log`,
`/tmp/scadabridge-review-test.log`, `/tmp/scadabridge-review-tests/`,
`/tmp/scadabridge-review-format.log`, `/tmp/scadabridge-review-vulnerable.json`, and
`/tmp/scadabridge-review-deprecated.json`. The results and reproduction recipes below are retained
in this document; temporary files are not durable repository artifacts.

### 7.B Isolated behavior probes

These probes used local compiled assemblies or real loopback Kestrel with synthetic data. They
were created under `/tmp`, not added as regression tests. They neither touched live devices nor
fetched production credentials.

| Finding | Executed probe / observation | Scope of proof |
|---|---|---|
| MG-1 | `ScriptTrustValidator.FindViolations` returned **0** for RuntimeHelpers.GetUninitializedObject, managed Unsafe.As, and new AssemblyLoadContext snippets | Real policy permits each referenced API; not a full sandbox escape exploit |
| CM-1 | `AuditEventBatch.CalculateSize()` for 256 ErrorDetail-only rows of 65,536 ASCII bytes returned **16,779,264**; actual `AddGrpc` options resolve receive default **4,194,304** | Real protobuf size and framework default; no disk-growth or end-to-end drain experiment |
| CM-9 | Real central auth interceptor with site-A metadata/key accepted reconciliation payload `SiteIdentifier=site-b`; real mapper preserved B | Auth/request binding gap, followed in source to token staging; no cross-site secret fetch executed |
| CM-10 | Provider callback applied a synthetic effect, threw Unavailable, then succeeded on peer: **applications=2**, returned `ok` | Fault-injected retry path, not physical network/device duplication |
| SR-11 | In-memory SQLite: normal wrapped write audit count **1**; `tx.Connection` was original provider, not wrapper; raw write affected **1** row with **0** audit emissions, persisted value 42 | Actual ADO escape and missing audit; reflection only instantiated internal wrapper in harness, not used by escape; not SQL Server/Roslyn end-to-end |
| EI-10 | Implemented; verification pending | Early bounded body reader applies even with auditing disabled; real Kestrel gate pending. |

Probe sources/output: `/tmp/scadabridge-review-probes/Program.cs` and
`/tmp/scadabridge-review-probes.log`; `/tmp/scadabridge-sr11-probe/Program.cs`;
`/tmp/scadabridge-cap-probe/Program.cs` and `/tmp/scadabridge-cap-probe/output.txt`.
Repeatable regression recipes are attached to each finding and the §5 acceptance gates.

## 8. Second-pass review decisions and architecture fitness

### 8.A Corrections that change remediation

| IDs | Decision |
|---|---|
| UI-3 | Withdraw: caller awaits retain renderer synchronization context; no demonstrated off-dispatcher entry |
| XC-6 / CM-5 | One defect, retain CM-5 Low; XC-6 is a historical alias |
| UI-1 | Many bypass paths in 16 files, not every UI edit; template/shared-script service paths already audit |
| MG-3 | Implemented; verification pending | Removed stale deployment probe memoization; peer-change regressions passed. |
| UI-4/6/7 | Low hardening/prevention gaps; no continuing-circuit refresh outage or unauthorized handler exploit demonstrated |
| UI-2 / UI-13 | Passive cookie renewal and stale circuit authority are separate failures; both need server-side handling |
| UI-5 / UI-14 | Timeout is not termination or a memory bound; never recycle an admission slot while runaway work remains |
| DC-1/2/5/12 | Cleanup must fence old owners and live handles; snapshots must cover every subscriber without broadcasting per-feed completion to unrelated sources |
| SR-4 / SR-11 | Retargeting restrictions must cover transaction-derived connections; the class is already sealed |
| EI-2/3/5/11 | Free-text messages may contain secrets; SQL name patterns do not redact values; caller-controlled forwarding markers are not identity proof |
| EI-4 / CM-10 / DC-7 | Align deadlines but distinguish safe pre-dispatch rejection from an uncertain already-applied request |
| ORCH-1 / EI-7 | A timeout cannot justify disposing resources still owned by unfinished work or admitting unlimited abandoned tasks |
| MG-6 / DL-3/4/8/11 | Stable keysets + bytes + true root + echo-edge priority + latest detail are separate correctness requirements |
| DL-9 | Document bounded owned state; statefulness alone is not a Commons architecture violation |
| XC-1 | Use actual listener/local port, not spoofable Host/authority matching, for port isolation |

All previous SR/DC/MG/UI/DL/EI/CM/XC IDs and ORCH-1 were source-revisited or, for inventory/doc
observations, checked against their current files. Historical production/QA statements, performance
estimates, and migration/deferred-register inventories are not newly measured simply because
source is unchanged. The original twelve design commitments describe implementation structure;
they are not evidence that the script trust gate is a complete security sandbox or that per-site
keys enforce payload ownership.

### 8.B CheckIt matrix

The intended design is a modular Akka hub-and-spoke system with a framework-light Commons core,
not a framework-free application layer. Checks are interpreted against that declared architecture.
A `pass` is limited to the cited check, not a blanket claim that the component is bug-free.

| Check | Status | Evidence / interpretation |
|---|---|---|
| CA-01 dependency direction | pass | Commons defines entities/ports, no project references; EF/UI/Host remain outside core; §6.A graph |
| CA-02 framework independence | pass (Commons scope) | Commons directly references only shared Audit; no EF/ASP.NET/Akka core imports found; actor application components intentionally depend on Akka |
| CA-03 ports/adapters | pass | External gateway/DB/device/notification ports and test doubles remain; SR-11 is an adapter escape defect, not absence of ports |
| CA-04 boundary data shape | pass (structural) | Typed message/proto DTO mapping and additive contracts retained; CM-9 shows shape alone does not validate ownership |
| CA-05 rule placement | fail | UI-1 bypasses audited application mutation boundaries and their guards |
| CA-06 cycles | pass | Fresh DFS: 26 source projects / 91 edges / zero cycles |
| CA-07 testability | pass | Runtime, management, adapter, and transport suites use probes/test doubles; selected solution tests run locally |
| BP-01 separation of concerns | fail | UI-1 duplicates mutation/audit responsibility outside audited service path |
| BP-02 explicit dependencies / lifetime | fail | MG-3 process-global memo, CM-8 mutable static seam, SR-2 omitted alarm dependencies, ORCH-1 resource lifetime race |
| BP-03 repeatable self-checking tests | fail | Current run passes; ORCH-1 historical failure + CM-8 shared static test seam still lack deterministic guards |
| BP-04 integration boundaries | pass | Separate Transport.IntegrationTests and host/communication boundary suites; new negative gates needed for reported defects |
| BP-05 static-analysis health | pass (executed build diagnostics only) | Warnings-as-errors configured; build 0/0; no separate security/static-analysis engine run, format fails below |
| BP-06 automated style | fail | No checked-in root .editorconfig or CI workflow found; formatter reports 1,290 diagnostics in 60 files |
| BP-07 dependency-risk process | pass with documented exception | Fresh vulnerable/deprecated queries completed; known test-only AngleSharp suppression has rationale; xUnit Legacy recorded; XC-9 version skew remains |
| BP-08 security baseline | fail | MG-1, MG-2, UI-1/2/13, SR-11, CM-9, EI-10/11 show concrete gaps despite existing auth/redaction controls |
| BP-09 continuous architecture fitness | fail | Role/dispatch/compile-contract/captive-dependency tests exist; whole-graph, mapped-service-auth, proto-source sync and route-policy guards are incomplete; no checked-in CI workflow found |
| LOG-01 organization logging baseline | fail | Host uses Serilog (`Program.cs:181,795`) and structured node/trace enrichment (`LoggerConfigurationFactory.cs:85-95`); actors also use Akka ILoggingAdapter (`CentralCommunicationActor.cs:27`). No application LogContext push/FromLogContext or request/user/operation scope setup found; only TraceCapturingLogger delegates BeginScope. Thus the stricter ILogger + Serilog + LogContext baseline is not fully implemented. Existing trace IDs are not absent, and this is not a new security defect count |

### 8.C Fitness functions to add

1. Actual mapped gRPC services must have an authentication gate; verified principal ownership must
   match every request/row site. Combine prefix coverage with adversarial metadata/payload tests.
2. Source project graph must stay acyclic and Commons must not acquire outer-layer references;
   freeze deliberate exceptions, not incidental transitive package names.
3. Generated descriptors must match proto messages, fields, types, oneofs, and service signatures.
   Field-number-only regex checking is a useful start but misses renamed/retyped/service drift.
4. Every UI mutation goes through an audited application boundary and uses current grants. Test
   representative create/update/delete rollback and revocation on a live circuit; don't rely only
   on file-name/string sweeps or role attributes.
5. Every bounded surface enforces its limit before allocation/read: audit serialized batches,
   Replay payloads, sandbox writers, and chunked HTTP bodies. Include single oversize rows.
6. Lifecycle tests control the race: owner replacement/unsubscribe, interleaved source snapshots,
   late rehydration, demotion between deliveries, pump shutdown, and response loss after commit.

Add these to the repository's chosen CI system with existing build/test gates; keep formatting
cleanup and framework/package migration in separate maintenance work so they do not obscure the
correctness/security remediation.


## 9. High-severity remediation completion

**Completed 2026-09-09. All 13 High findings are resolved and verified.** Implementation
started from `8bfe6551` on `codex/resolve-high-review-findings`, with parallel device/runtime,
transport, and UI/security agents plus primary-agent integration and independent reviews.
The findings above preserve the reviewed baseline; this section supersedes their open status.
Medium/Low findings remain outside this remediation unless a change was required for a High fix.

| Finding | Status | Implementation / verification |
|---|---|---|
| DC-1 | **Completed** | Owner-checked PostStop unsubscribe and DeathWatch release dead demand. Shared/replacement demand survives old-owner termination and in-flight completion. Deterministic lifecycle/race regressions pass. |
| DC-2 | **Completed** | Initial subscribe and resolution exclude each other’s in-flight tags; equal live handles are never released. Mx reverse-map removal is atomic and identity-checked. Adapter and controlled-race regressions pass. |
| DC-3 | **Completed** | Connecting/reconnecting reject single and batch writes immediately; subscription retries are deduplicated by owner/source. No stale setpoint replay after reconnect. Full DCL suite: 365 passed. |
| DC-12 | **Completed** | OPC snapshot start/entries/end retain source and epoch; NativeAlarmActor ignores unrelated/duplicate ends and drops partial snapshots on disconnect. Legacy Mx framing remains supported. Actor tests prove standing-alarm preservation; live two-boiler feed verifies distinct wire scopes and epochs. |
| CM-1 | **Completed** | Servers and all six application channels use explicit 32 MiB send/receive limits. Protobuf-exact 2 MiB audit budgets cover pending/cached drain and reconciliation paging. Both 256×64 KiB lanes drain without loss; real gRPC 17 MiB response passes. Oversize-row policy is explicit below. |
| CM-9 | **Completed** | Authenticated site identity guards RPC payloads, audit rows and notification status. SQL upsert rejects another site’s stored operation ID, including duplicate-insert retry. Real gRPC denial, outbox ownership and 29 SQL Server lifecycle/ownership tests pass. |
| CM-10 | **Completed** | Reads/idempotent ingests may retry; mutations retry only after explicit server pre-dispatch rejection. Ambiguous primary/secondary response loss returns `OUTCOME_UNKNOWN` without replay. Real transport and effect-count tests pass. |
| UI-1 | **Completed** | All 16 configuration repository-bypass pages use typed commands. One transaction spans entity saves and final scrubbed audit flush, including generated IDs; cache publication follows commit. Direct instance/area paths retain domain checks within the same boundary. LDAP CRUD and audit-failure rollback regressions pass; deployed browser CRUD passes. |
| UI-2 | **Completed** | Passive ping/Blazor validation neither slides cookies nor advances activity. Trusted browser input posts with antiforgery; per-login activity reaches existing circuits. Stable NameIdentifier preserves antiforgery across role refresh. Real HTTP fake-clock tests and deployed trusted/synthetic-input browser gate pass. |
| UI-13 | **Completed** | Circuit authority remaps stored groups in fresh scopes; mutations require current roles and persisted-site scope. API keys, reports, failover, parked messages, instances and Topology recheck before dispatch. Revoked-scope and authorized-positive tests pass, including nonnumeric site identifiers. Explicit server-configured auto-login retains synthetic grants under its existing startup guard. |
| SR-1 | **Completed** | Attribute get/set/wait/batch continuations use `ConfigureAwait(false)`. Saturated two-worker synchronous read/write regressions pass without watchdog assistance; wait/trace regression coverage remains green. |
| SR-11 | **Completed** | Sync/async transaction and command connection/transaction readback retain auditing wrappers. SQLite regressions prove persisted writes and exactly one audit through each former escape. |
| MG-1 | **Completed** | Trust gate denies Unsafe, RuntimeHelpers and Runtime.Loader, including alias/static imports, while preserving async and CallerMemberName. ScriptAnalysis suite: 81 passed. |

### Final verification

| Gate | Result / evidence |
|---|---|
| Final solution build | `dotnet build ZB.MOM.WW.ScadaBridge.slnx --no-restore`: **0 warnings, 0 errors**. `/tmp/high-final-build3.log` |
| Selected full solution | `dotnet test ZB.MOM.WW.ScadaBridge.slnx --no-build --no-restore --filter 'FullyQualifiedName!~PlaywrightTests' --logger trx --results-directory /tmp/scadabridge-high-final-green`: **9,310 passed, 0 failed, 0 skipped**; exit 0. `/tmp/high-final-tests2.log` |
| Final UI follow-up | After the final auto-login compatibility fix: **1,392 passed, 0 failed, 0 skipped** in the full CentralUI suite. This reruns the 1,391 UI cases included above plus the new regression; counts are not additive. `/tmp/high-ui-final.log` |
| Deployed browser | LDAP create/edit/delete round trip and trusted-input/passive-input/antiforgery/throttle scenario: **2 passed, 0 failed, 0 skipped**. `/tmp/high-browser-final.log` |
| Independent reviews | Runtime/trust, transport/ownership, and UI transaction/authorization reviews passed after corrections; final auto-login guard follow-up also passed. `/tmp/scadabridge-ui-independent-review.md` |
| Local rig | Final image `sha256:b9d6b60ab77093ee537470992fb891f78cc9888170d290021855c902c0115eb6` on **all eight nodes**. All ready (200), one active per pair (200) and four standbys (503), two members per pair, zero unreachable/restarts, all five central singletons reachable. Site-A replication connected with backlog 0. `/tmp/scadabridge-final-loginfix-postdeploy-health.json` |
| Change hygiene | `git diff --check` clean. Canonical original-repository config/log mounts and named data volumes preserved; env2 remains untouched. Component requirements updated with implementation. |

The local image was built with `bash docker/build.sh` from the implementation worktree, then
all eight nodes were recreated using the canonical original repository’s Compose file. This
keeps existing configuration, logs and data in place rather than rebinding them to the worktree.
No device writes, failover drill, remote deployment, commit or push was performed by this remediation.

### Regression evidence and limits

- SR-1/SR-11 reproduced **five failures before** their fixes; the runtime follow-up passed 39
  targeted tests. MG-1 reproduced five forbidden-call failures while two benign cases passed;
  all 81 ScriptAnalysis cases pass afterward. Full SiteRuntime: **774 passed**.
- CM-1’s 17 MiB response failed at the previous default limit. CM-9’s stored-owner SQL regression
  failed before its guard. Full Communication **756**, ConfigurationDatabase **411**, AuditLog
  **396**, Host **590**, and integration **117** tests pass. Two historical Host tests expected
  blind mutation retry on `Unavailable`; they now model explicit pre-dispatch rejection, with
  a separate real-transport test proving ambiguous failure never dispatches to the peer.
- Device tests control owner death, replacement, shared-tag demand, subscribe/resolve races,
  disconnected writes and interleaved snapshots. A stale failover test assertion was corrected:
  backup demand must be restored while an original subscribe remains pending. The full DCL
  suite subsequently passed. The local two-boiler snapshots were empty; standing-condition
  preservation is proven by deterministic actor tests, **not by live raised alarms**.
- UI revocation and Topology denied-mutation tests reproduced failures before correction.
  The first LDAP baseline test stopped at an incorrect seeded-row count, so it does **not**
  establish a before-fix missing-audit reproduction. Corrected real-SQLite CRUD/audit/rollback
  tests pass. A real HTTP regression found antiforgery failure after role refresh; stable user
  identity fixed it. The deployed browser found the synthetic auto-login role-loss regression;
  the explicit trusted-configuration guard passed its red/green test and repeat live CRUD gate.
- A single audit row above 2 MiB travels alone up to the 32 MiB wire ceiling. A larger row remains
  durable/unacknowledged with an explicit operator-action error, without truncation or silent
  loss. Stored LDAP groups are remapped live; directory group-membership changes still require
  login. Pre-upgrade cookies without stable NameIdentifier may require a fresh sign-in.
- No separate formatter cleanup or framework/package migration was included. The original
  Medium/Low register and baseline formatting debt remain. The local rig has an existing
  `zztest-conn-5d93ea81` test connection retrying unreachable `localhost:4840`; healthy cluster
  readiness does not assert that every configured device endpoint is reachable.

## 10. Medium-severity remediation completion

**All 33 Medium findings are completed and verified.** The High fixes were committed first as
`146a1323` (`Resolve all high-severity architecture review findings`). Medium implementation,
tests and documentation remain uncommitted on `codex/resolve-high-review-findings` in
`/Users/dohertj2/Desktop/ScadaBridge-high-findings`; no push was requested for this phase.
Parallel subagents handled runtime/devices, management/UI and edge integrations; the primary
agent handled persistence, listener/contracts, configuration and final integration. Independent
cross-reviews rechecked lifecycle, sandbox containment, relay provenance and migration safety.

| Finding | Status | Progress / evidence |
|---|---|---|
| SR-2 | Completed | Fresh alarm execution scope supplies database and tracking dependencies; runtime focused and full solution suites passed. |
| SR-3 | Completed | Correlated bounded retry restores failed alarm subscriptions; full solution suite passed. |
| SR-4 | Completed | Implemented; immutable audited target regression passed. |
| SR-5 | Completed | Ordered persistence drain and same-name stop barriers; independent review passed; runtime suite 102 passed including locked delete/disable and queued cancellation. |
| SR-12 | Completed | Implemented; delayed rehydration fencing regression passed. |
| SR-13 | Completed | Authority rechecked after each lane admission and before every delivery; StoreAndForward suite 38 passed. |
| CM-2 | Completed | Implemented; all four env2 node configurations enable deterministic bootstrap; parsed configuration validation passed. |
| MG-2 | Completed | Mandatory fresh caller scope filters audit trees and replay; management regressions and full UI suite passed. |
| MG-3 | Completed | Removed stale deployment probe memoization; peer-change regressions passed. |
| MG-4 | Completed | Server-owned bulk deadline stops subsequent dispatch and returns unfinished failures; targeted deployment tests passed. |
| UI-5 | Completed | Isolated child processes killed and joined before admission reuse; broker-deadline and runaway-worker tests included in 97 passing UI cases. Published Linux image contains worker DLL/deps/runtimeconfig; deployed browser termination/recovery check passed. |
| UI-14 | Completed | Console, return serialization and IPC bounded; oversized-output regressions passed in the full UI suite and deployed browser. |
| EI-1 | Completed | Relay ownership and uncertain-outcome audit completion implemented; InboundAPI suite passed. |
| EI-2 | Completed | Body-capture opt-out also suppresses free-form failure details and companion events; InboundAPI suite passed. |
| EI-3 | Completed | SQL exception details respect sensitive-parameter redaction policy; audit redaction regressions passed. |
| EI-4 | Completed | Single relay deadline and explicit pre-execution-only retry; ambiguous outcomes are not replayed; InboundAPI suite passed. |
| EI-5 | Completed | Authenticated request-bound relay provenance replaces trust in caller-supplied forwarding headers; independent review and InboundAPI suite passed. |
| EI-10 | Completed | Early body cap applies with auditing disabled; real Kestrel oversized/chunked cases included in six passing Host regressions. |
| EI-11 | Completed | All retained error-message/detail fields are capped and redacted; targeted audit suite passed. |
| DL-1 | Completed | Global unique index, domain conflict translation and duplicate preflight migration; real SQL concurrency/upgrade guard tests passed. |
| DL-2 | Completed | Protected primary/backup configuration with per-provider model cache and idempotent bounded startup upgrade; real SQL and SQLite regressions passed. |
| DL-3 | Completed | Eight-row keyset paging, 500-event ceiling, UTF-8 budget and preserved root/truncation metadata; all six real SQL regressions passed. |
| DL-6 | Completed | Deployed cross-site overwrite rejected; both real import cases passed after a valid failing baseline. |
| DC-4 | Completed | Exact requested OPC UA security mode enforced during discovery/connect; focused device suite passed. |
| DC-5 | Completed | Shared alarm cache merges live events with snapshots and fences stale epochs; independent review and 34 device regressions passed. |
| DC-6 | Completed | CLI returns failure for nonzero bulk failureCount and malformed responses; all three actual command/socket regressions passed. |
| DC-7 | Completed | Notifier retries only explicit standby rejection; notifier suite passed. |
| XC-1 | Completed | Actual socket and gRPC metadata enforce endpoint ownership despite forged authority; six combined Host/Kestrel regressions passed. |
| XC-2 | Completed | Implemented; removed four ignored SMTP keys from all four central rig settings; database configuration guidance replaces them. |
| XC-3 | Completed | Implemented; removed ignored cron key from all eight site settings; documents interval-based purge. |
| XC-4 | Completed | Actual protoc descriptors match vendored message/service graphs; four tests passed including three name/type/service drift controls. Implicit JSON names normalized using effective runtime descriptors. |
| XC-5 | Completed | Implemented; README, high-level/component requirements and component skill now describe actual gRPC/HTTP transport. |
| ORCH-1 | Completed | Synchronous disposal joins context-free persistence drain; runtime 102-case suite passed. |

### Final verification

| Gate | Result / evidence |
|---|---|
| Full solution build | **0 warnings, 0 errors**; `/tmp/medium-full-build2.log`. Final deployed-browser project build also passed 0/0 after its test-only update (`/tmp/medium-browser-build.log`). |
| Selected full solution | `dotnet test ZB.MOM.WW.ScadaBridge.slnx --no-restore --filter 'FullyQualifiedName!~PlaywrightTests' --logger trx --results-directory /tmp/scadabridge-medium-full`: **9,397 passed, 0 failed, 0 skipped**, exit 0, 30 result files. `/tmp/medium-full-tests.log`. Includes 1,399 UI, 596 Host, 125 Transport integration and 117 cross-component integration cases plus 13 performance checks. |
| Deployed browser | **2 passed, 0 failed, 0 skipped**: Linux sandbox normal execution, bounded console/return output, noncooperative infinite-loop timeout and subsequent successful execution; trusted/passive session input and antiforgery. `/tmp/medium-browser-final.log`. No child worker remained afterward on either central node. |
| Real persistence | Global-name concurrency and duplicate-upgrade rejection, primary/backup encryption and idempotent conversion passed against SQL Server; key-ring isolation and guards passed against SQLite. `/tmp/medium-persistence-final.log`. Final replay keyset/Unicode/oversized-root SQL checks also passed and are included in the full solution run. |
| Transport contract | Six real Kestrel listener/body-cap cases passed (`/tmp/medium-host-green.log`); actual protoc descriptor comparison and three drift controls passed (`/tmp/medium-proto-green2.log`). |
| Local rig | Image `sha256:b8d158309b5d850fc01a3d32e7325e2238611b039e37f5fa0d51e7a488ddee36` on all **eight nodes**; all ready, one active per pair, two members per pair, zero unreachable members or restarts. Site-A replication connected, backlog 0. `/tmp/scadabridge-medium-postdeploy-health.json`. |
| Upgrade | Coordinated central-pair stop, successful COPY_ONLY/CHECKSUM backup and RESTORE VERIFYONLY, then migration `20260909203046_EnforceGlobalInstanceNamesAndProtectProtocolConfiguration`. All 4 protocol rows retained; 3 non-null primary values protected, 0 unprotected values, existing 1-entry key ring preserved. `/tmp/medium-rig-backup.log`, `/tmp/medium-rig-before-counts.log`, `/tmp/medium-rig-after-counts.log`. Live Site-A OPC UA connection reconnected after upgrade. |
| Configuration/docs | All 12 rig JSON files parse; four env2 bootstrap guards enabled; obsolete SMTP/cron keys removed; transport entry-point documentation corrected. Component requirements and umbrella project index updated. |

### Operational notes and verification limits

- The SQL backup remains inside the existing SQL Server data volume at
  `/var/opt/mssql/backup/ScadaBridgeConfig_before_medium_20260909_211100.bak`.
  Protocol encryption is a coordinated central upgrade: old binaries cannot read the new values.
  A rollback requires the coordinated pre-upgrade database/key-ring backup, as documented in
  `Component-ConfigurationDatabase.md`; the migration refuses a downgrade over protected rows.
- The image was built from the implementation worktree and deployed using the original
  repository's canonical Compose file. Existing configuration/log bind mounts and named data
  volumes were preserved. Env2 bootstrap settings were verified statically; env2 containers
  remain untouched. No device writes, live failover drill or remote deployment was performed.
- Sandbox isolation provides a process resource/lifetime boundary, not a separate OS identity.
  The worker is killed at the deadline. If a parent-side broker ignores cancellation, its slot
  and scope remain owned until that operation actually completes; no orphan broker operation
  or early capacity reuse is introduced. Already accepted remote work cannot be undone by
  killing its client. Console retains 32,768 characters plus a fixed truncation marker; return
  serialization is capped at 32 KiB bytes, IPC at 1 MiB, managed worker heap at 512 MiB.
- Replay is capped at 500 events and a 1 MiB serialized UTF-8 budget, with eight-row keyset
  pages bounding EF buffering. Oversized events are omitted whole with truthful truncation
  and separately retained root metadata. Shared snapshot/lifecycle races are covered by
  deterministic tests, not a live raised-alarm scenario. The pre-existing unreachable
  `zztest-conn-5d93ea81` test connection remains outside this remediation.
- The first proto comparison exposed implicit JSON-name representation differences, not
  wire drift. It now compares effective runtime names against compiler output while preserving
  explicit field/service metadata. Earlier fixture/build failures were corrected without
  weakening production guards: audit test doubles accept caller scope, and protocol-writing
  fixtures use stable ephemeral key rings. These are not claimed as before-fix reproductions.
- Low findings are tracked in §11. Focused test counts overlap the full suite and are not additive.


## 11. Low-severity remediation progress

**Completed: 46 / 46 Low findings.** All 46 Low findings are in scope. This register supersedes the
baseline finding status only when a row is marked **Completed** with verification.
High and Medium completion records in §§9–10 remain authoritative.

Implementation follows the existing finding-specific fixes: preserve wire compatibility,
bound maintenance and debugging work, enforce authorization at application boundaries,
and correct documentation against current source. Independent UI, Flow Debugger, and
runtime/device groups run in parallel with exclusive file ownership. Transport,
maintenance, package/configuration work, and this register have one coordinating owner.
Behavior changes receive focused regression checks; documentation and configuration
changes receive source/structure validation. Final gates are full build, relevant/full
solution tests, diff review, and a local rig rebuild for shipping runtime changes.

| Finding | Status | Progress / evidence |
|---|---|---|
| SR-6 | **Completed** | Device-to-script linkage is explicitly inferred; alarm causal parents remain reserved for actual writes. Runtime/Commons regressions pass. |
| SR-7 | **Completed** | Trace failure diagnostics use the undecorated logger; recursive trace capture regression passes. |
| SR-8 | **Completed** | Thread-static scheduler identity prevents cross-scheduler inlining; saturation regression passes. |
| SR-9 | **Completed** | Shared trigger gate has explicit process ownership; consumer Dispose cannot poison later callers. Lifecycle regression passes. |
| SR-10 | **Completed** | Per-context helpers and guard closures are lazily cached; identity regression passes. |
| CM-3 | **Completed** | Raw audit/site-call pulls reject limits outside 1–2000 before reading; both reconciliation option validators enforce the same cap. Boundary regressions pass. |
| CM-4 | **Completed** | Bootstrap checks cancellation before resolving the actor system or joining seeds, and propagates canceled probes. Host suite 598 passed. |
| CM-5 | **Completed** | Retired site ingest RPCs always return Unimplemented, including empty payloads; dead actor wiring removed. Contract regressions pass. |
| CM-6 | **Completed** | Central and site HTTP/2 listeners share finite keepalive settings; metrics listener isolation remains enforced. Build and Host 598 tests pass. |
| CM-7 | **Completed** | Channel graveyard retains active calls beyond grace; lifecycle locking closes shutdown/requeue races. Completion and shutdown regressions pass. |
| CM-8 | **Completed** | Connected-header timeout is immutable per client, bound through validated CommunicationOptions. Parallel-instance regression passes. |
| MG-5 | **Completed** | Total live-command budget includes lookup/write/audit/pull, reserves response grace, and propagates audit cancellation. Management suite 658 passed. |
| MG-6 | **Completed** | Clock-independent keyset rescans capture late/backdated completion and descendant rows. Indexed key union bounds payload pages; real SQLite plans cover 1/64 IDs and malformed legacy JSON. Management 660 and AuditLog 407 tests pass. |
| MG-7 | **Completed** | Live write results use bounded collector with explicit event/log truncation; Management suite 658 passed. |
| MG-8 | **Completed** | Shared-script create/update uses cached compile/trust validation; TemplateEngine suite 483 passed. |
| MG-9 | **Completed** | Positive page sizes clamp to 1000; null/nonpositive semantics preserved and boundaries tested. |
| MG-10 | **Completed** | Management component catalog now documents all four FlowDebug commands, authority, paging and budget. |
| UI-4 | **Completed** | Poll cleanup runs in outer finally and observes rejected renderer work; full UI suite 1416 passed. |
| UI-6 | **Completed** | Scoped failover facade requires current Administrator authority before invoking the host backend; positive/negative tests pass. |
| UI-7 | **Completed** | Exact route-policy sweep covers UI and external Secrets components; full UI suite 1416 passed. |
| UI-8 | **Completed** | Shared parked-call relay resolves site ownership and denies unknown/out-of-scope targets for scoped principals; regressions pass. |
| UI-9 | **Completed** | Tree value snapshots skip unchanged forest rebuilds while preserving in-place edits, child metadata and filtered key/inheritance behavior. |
| UI-10 | **Completed** | Cookie login explicitly validates antiforgery; missing token rejected and valid real token issues cookie. CLI bearer-token endpoint preserved. |
| UI-11 | **Completed** | Lazy site/area queries retain one untracked 50-row instance page per loaded branch; shared async gate serializes branch/refresh/staleness. SQLite paging and controlled interleaving regressions pass. |
| UI-12 | **Completed** | Viewer sees no empty Notifications heading and Flow Debugger navigation matches its policy; regression passes. |
| EI-6 | **Completed** | Purge actor forwards per-channel 50000-row tick budget; repository caps final partial batch. Purge/options 20 and repository/guard 10 tests pass. |
| EI-7 | **Completed** | Root-DI admission registry retains one unfinished collection per registration across actor recreation; actual wait is bounded and task owns scope until completion. Full KPI suite 57 passed. |
| EI-8 | **Completed** | Relay strips Accept-Encoding because it forwards raw response bytes without Content-Encoding; regression and full InboundAPI suite 360 pass. |
| EI-9 | **Completed** | Outbox contract explicitly documents repeated duplicate batches during dual-active operation and lack of row claims. |
| DL-4 | **Completed** | Echo edges replace ordinary duplicate endpoint pairs; unique-edge and echo-semantic regressions pass. |
| DL-5 | **Completed** | Refresh bases graph order on authoritative Replay and appends live-only rows; order regression passes. |
| DL-7 | **Completed** | Graph build parses each audit payload once and reuses projections; Commons suite 851 passed. |
| DL-8 | **Completed** | Existing Medium chain-anchor fix revalidated by timestamp-skewed child/root regression. |
| DL-9 | **Completed** | Commons Debugging namespace, per-run bounded collector ownership and redaction documented. |
| DL-10 | **Completed** | Enum name/value serialization contracts and display ordering have separate regression coverage. |
| DL-11 | **Completed** | Folded steps/lifecycle records retain latest exception detail and clear stale details after success; regressions pass. |
| DC-8 | **Completed** | CLI harness covers all 19 groups plus browse/search/verify/certs; secured-write README added; CLI suite 475 passed. |
| DC-9 | **Completed** | DCL docs corrected against current unsubscribe, write rejection, atomic disconnect and snapshot behavior. |
| DC-10 | **Completed** | OPC shard plans are computed once per resolved chunk and reused for each monitored item; DCL suite 379 passed. |
| DC-11 | **Completed** | All notifier base URLs are validated as absolute HTTP(S) before delivery; notifier suite 36 passed. |
| XC-7 | **Completed** | Removed both migration-only sections from all eight rig site JSON configurations; parsed and verified each file. |
| XC-8 | **Completed** | Source-discovered service maps and external LocalDb/Secrets mappings are compared against generated method descriptors and auth gates, with gate-removal negative control. Host suite 598 passed. |
| XC-9 | **Completed** | Central SqlClient 6.1.1 removes per-test overrides; SignalR.Client 10.0.7 aligns with runtime; bUnit 2.7.2 uses stable syntax. Restore, zero-warning build and all solution suites pass. |
| XC-10 | **Completed** | Global OPC/Mx options validate required identity strings at startup, preserving optional path defaults; DCL suite 379 passed. |
| XC-11 | **Completed** | Umbrella index now includes instance picker/v1.4 logs and live remediation status. Fresh origin/main fetch confirms current main fef3e767 is already pushed (0 ahead/behind); no new push needed for the historical finding. |
| XC-12 | **Completed** | Health-check remarks now distinguish seven central singleton proxies, five required and two best-effort exclusions. |


### Final verification — completed 2026-09-09

- Full solution restore succeeded after centralizing Microsoft.Data.SqlClient 6.1.1,
  Microsoft.AspNetCore.SignalR.Client 10.0.7, and stable bUnit 2.7.2.
- Full solution build: **0 warnings, 0 errors**. Canonical `docker/regen-proto.sh sitestream`
  preserves the source descriptor path for the additive trace cursor; all 766 Communication
  tests pass, including the complete source/vendored descriptor comparison.
- Initial solution run (excluding deployed Playwright tests): 9474 passed, three failures.
  One was the generated descriptor path, corrected and verified above. Two login integration
  fixtures omitted the now-required antiforgery token; they now fetch the real rendered token
  and cookie, preserve the original login assertions, and add missing-token rejection coverage.
  Full Integration rerun: **118 passed, zero failed/skipped**. Across the full run with
  corrected Communication and Integration suite reruns: **9478 passed, zero outstanding failures
  or skips**. Counts replace earlier suite results rather than adding repeated tests. The deployed
  Playwright smoke adds **1 passed**: real antiforgery login, initially unloaded topology, bounded
  branch loading/page controls, and Flow Debugger navigation without browser errors.
- `bash docker/deploy.sh` succeeded from the canonical repository. All eight primary rig nodes run
  image `sha256:cfa919dd343ffa090995336f2854ac4d79d4c2bb919514aa7e72abf579b0b4ed`, with
  zero restarts, `/health/ready` 200, two members and zero unreachable members per pair, and
  exactly one `/health/active` 200 per pair (standby 503). Existing bind mounts and volumes
  were preserved. Env2 containers were not redeployed.
- Final completion audit verifies all 46 unique Low IDs have Completed rows, with no remaining
  Low findings. `git diff --check` passes. The verified remediation is committed directly on `main`.
- Validation logs: `/tmp/low-final-build2.log`, `/tmp/low-full-tests.log`,
  `/tmp/low-comm-final.log`, `/tmp/low-integration-final.log`, `/tmp/low-rig-deploy.log`,
  `/tmp/low-rig-health.json`, and `/tmp/low-smoke.log`. Focused regression evidence above overlaps
  the full suite and is not additive.

### Operational notes

- MG-6 intentionally rescans matching execution-ID batches on each poll to capture completion
  rows inserted late with earlier timestamps. Indexed SQLite key queries avoid scanning unrelated
  retained audit payloads. Discovery memory grows with the traversed tree during the bounded
  command; returned results have fixed count/byte caps. No central/site clock agreement is assumed.
- EI-7 bounds the caller wait and admits at most one unfinished source collection per registration
  across actor recreation on the same root provider. A source that ignores cancellation retains
  its task and scope until actual completion; subsequent ticks do not create duplicate work.
- EI-9 is resolved through the requested contract clarification: dual-active outbox writers can
  deliver repeated duplicate batches; database row claims are not implemented by this change.