docs(archreview): sync PLAN-04 tasks.json + tracker — Task 26 sweep GREEN, plan COMPLETE (26/26)

Full-solution build 0 warn/0 err; all 30 unit+integration test assemblies pass
after fixing one stale enum-count lock-in test (22d52390). Playwright E2E residue
is environmental (stale docker cluster, SQLite-open on site-container deploy;
non-deterministic set; zero PLAN-04 file overlap). Initiative total 104/192.

Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj

archreview/plans/00-MASTER-TRACKER.md

@@ -13,7 +13,7 @@ Every plan follows the TDD bite-sized-task format and ships a co-located `.tasks
 | [PLAN-01](PLAN-01-cluster-host-failover.md) | Cluster, Host & Failover | 23 | 23 | ✅ Complete (all 23 tasks; merged to `main` `ec47cb56`) | 28 findings: 20 fixed, 3 → other plans, 3 accepted, 2 roadmap-deferred |
 | [PLAN-02](PLAN-02-communication-store-and-forward.md) | Communication & S&F | 24 | 24 | ✅ Complete (all 24 tasks; merged to `main`) | 28 findings: 25 fixed, 1 → plan 01, 1 deferred (serializer swap), 1 won't-fix |
 | [PLAN-03](PLAN-03-site-runtime-dcl.md) | Site Runtime & DCL | 26 | 26 | ✅ Complete (all 26 tasks; merged to `main` `c7a0c922`) | 28 actionable: 25 fixed, 3 deferred with rationale |
-| [PLAN-04](PLAN-04-data-audit-backbone.md) | Data & Audit Backbone | 26 | 0 | ⬜ Not started | 27 items: 22 fixed, 4 deferred, 1 won't-fix (incl. NodeB reconcile from report 08) |
+| [PLAN-04](PLAN-04-data-audit-backbone.md) | Data & Audit Backbone | 26 | 26 | ✅ Complete (T26 sweep green; on branch `worktree-archreview-wave2-plan04` — Batches 1–3 merged to `main`, Batches 4–8 + T24 + T26 fix unmerged, pending finishing-a-development-branch) | 27 items: 22 fixed, 4 deferred, 1 won't-fix (incl. NodeB reconcile from report 08) |
 | [PLAN-05](PLAN-05-templates-deployment-transport.md) | Templates, Deployment & Transport | 25 | 1 | 🟨 In progress (P0: T1) | 33 line-items: 27 fixed, 6 deferred with rationale |
 | [PLAN-06](PLAN-06-edge-integrations.md) | Edge Integrations | 24 | 0 | ⬜ Not started | 24 findings: 21 fixed, 3 deferred/won't-fix |
 | [PLAN-07](PLAN-07-ui-management-security.md) | UI, Management & Security | 33 | 3 | 🟨 In progress (P0: T1, T2, T3) | 24 items: 20 fixed, 3 partial (deferrals documented), 1 won't-fix |
@@ -29,6 +29,45 @@ Status values: ⬜ Not started · 🟨 In progress · ✅ Complete · ⏸ Blocke
 
 **Wave 1 — PLAN-03 COMPLETE (2026-07-09):** all 26 tasks delivered and **merged to `main`** (`c7a0c922`). Site Runtime & DCL hardening across nine TDD batches: reconnect/backoff leak fixes and the `DeployCompileValidator` compile gate, DCL adapter lifecycle + alarm-filter overwrite warning (S10) + observed adapter dispose on stop (S9), site-node cert reconcile (CertStoreActor trusted-store export → `DeploymentManagerActor` MemberUp-driven additive push), replicated config-fetch retry (`ConfigFetchRetryCount`), invariant-culture conditional-trigger fallback (C6), bounded/DropOldest event-channel comment fix (C2), and the SiteRuntime/DCL requirements-doc sync. Three low-severity items deferred with rationale (see PLAN-03 coverage table). Full CentralUI/SiteRuntime/DCL suites green. Initiative total: **78 of 192** (sum of the Done column above). Two out-of-band fixes also landed post-merge: PLAN-07's SecuredWrites `IBrowseService` bUnit-harness gap (resolved 2026-07-09, `9049690b`) — a pre-existing failure surfaced during PLAN-03's full-suite run, not a plan task.
 
+**Wave 2 — PLAN-04 COMPLETE (2026-07-09):** all 26 tasks delivered on worktree branch `worktree-archreview-wave2-plan04` (Batches 1–3 fast-forwarded to `main`+`origin`; Batches 4–8 + T24 + the T26 fix still on the branch, pending `finishing-a-development-branch`). Site SQLite retention chain + partition-switch hardening + purger role-grants/index migrations (T1–T9), backlog KPI + SiteCalls freshness upsert + EF `EnableRetryOnFailure` (T10–T12), operator-identity audit on Retry/Discard relays + reconciliation composite-keyset cursor/abandoned-row audit/NodeB failover (T13–T18), time-sliced KpiSample purge + single-query NotificationOutbox KPI + SiteCalls `OPTION (RECOMPILE)` (T19–T21), short-first-tick purge timers + data-layer cleanups + bounded parallel notification delivery (T22, T23, T25), the design-doc reconciliation pass (T24). **T26 (final `dotnet build`/`dotnet test` sweep):** build 0 warn/0 err; all 30 unit + integration test assemblies green after fixing one stale enum-count lock-in test (`22d52390` — Task 17 added `AuditKind.ReconciliationAbandoned` but left `AuditEnumTests` asserting 14 members; the per-task filtered runs never hit that Commons guard, so the full sweep caught it). The Playwright E2E residue (8–9 fails) is environmental — a 17h-old docker cluster whose site containers fail `instance deploy` with `SQLite Error 14: unable to open database file`; the failure set is non-deterministic and PLAN-04 changed zero files in that project or the deployment-DB path. Initiative total: **104 of 192** (sum of the Done column).
+
+## Parallelization Map — Incomplete Tasks (2026-07-09)
+
+Covers the plans with open tasks: PLAN-05 (24), PLAN-06 (24), PLAN-07 (30), PLAN-08 (10). PLAN-01/02/03/04 are complete — nothing to parallelize there (PLAN-04 finished 2026-07-09; the section below is retained for its cross-plan mutex analysis, which still governs PLAN-05–08). **"Ready"** = every `blockedBy` entry is already `completed` in the plan's `.tasks.json`, so the task can start now. Ready sets are recomputed below from the live manifests; re-derive after each merge.
+
+### Cross-plan serialization mutexes (a task holding one must be the ONLY task touching that surface, anywhere)
+
+| Shared surface | Tasks that touch it (P0n = PLAN-0n) | Rule |
+|---|---|---|
+| `ManagementService/ManagementActor.cs` | P05 T13; P06 T2, T3, T11, T20, T22; P07 T5, T7, T8, T9, T10, T11, T12, T15, T16, T17, T22, T23, T28, T30, T31 | **Global single-writer.** At most one ManagementActor task in flight across ALL plans. PLAN-07 already serializes its 15 internally — extend the same mutex to P05 T13 and the five P06 tasks. |
+| EF migration model-snapshot (`ConfigurationDatabase/Migrations/*` + `*ModelSnapshot`) | P06 T8 (ready), T19 | **Never two migration tasks concurrently.** Build first, never `--no-build` (empty-scaffold gotcha). Only P06 T8 is ready now. |
+| `Host/Program.cs` | P06 T7; P08 T4 | Single-writer — don't run P06 T7 and P08 T4 at the same time. |
+| `DeploymentManager/ArtifactDeploymentService.cs` | P06 T10; P07 T4 | Single-writer (P06 T10 not yet ready; P07 T4 is). |
+| `SiteRuntime/Persistence/SiteStorageService.cs` | P06 T10; P08 T6 | Single-writer (neither ready yet). |
+| `NotificationOutbox/NotificationOutboxActor.cs` | P06 T18, T21 | Intra-PLAN-06 (PLAN-04's delivery work is done and no longer touches it). |
+| `CentralUI/.../Monitoring/AlarmSummary.razor` | P07 T29, T32 | Intra-PLAN-07 — serialize. |
+| Docs (low-risk, coordinate — pathspec-commit + last-writer reconcile): `CLAUDE.md` (P05 T25, P06 T18, P07 T15/T31/T33, P08 T10), `Component-ManagementService.md`, `Component-InboundAPI.md`, `Component-ExternalSystemGateway.md`, `Component-NotificationService.md` | multiple | Not a hard blocker; do not co-edit the same file in the same wall-clock window. |
+
+**Dedupe reminder:** **P07 T33 is a skip** — CLI.Tests is already in the slnx via P08 T1 (done). Mark it `skipped-duplicate` in `PLAN-07-…tasks.json`; do not execute it.
+
+### Ready set + concurrent lanes per plan
+
+- **PLAN-04** — ✅ COMPLETE (2026-07-09); all 26 tasks done, T26 sweep green. Pending only `finishing-a-development-branch` (merge Batches 4–8 + T24 + T26 fix). No longer a parallelization lane.
+- **PLAN-05** — ready: **{2, 7, 9, 13, 14, 15, 16, 21, 22}**. Lanes: *(a)* `BundleImporter.cs` mutex — **T2 ↔ T14** (one at a time); *(b)* global ManagementActor lane — **T13** (also edits `FlatteningService.cs`, so not concurrent with T9); *(c)* free-concurrent, file-disjoint — **T7, T9, T15, T16, T21, T22** (Export / FlatteningService / TemplateEngine-Validation / DeploymentManager-pipeline / ScriptAnalysis / LineDiffer). Up to ~6 concurrent implementers once the two mutex lanes are accounted for.
+- **PLAN-06** — ready: **{1, 6, 7, 8, 17, 23}**. Lanes: **T8** is the sole migration in flight; **T7** holds the `Host/Program.cs` mutex (coordinate with P08 T4); **T1, T6, T17, T23** are free-concurrent (InboundScriptExecutor / InboundApiEndpointFilter / SmsAdapter / DelmiaNotifier docs). The ExternalSystem (T9→…), Notification (T18/T19→…) and inbound-contract (T2→T3→T4→T5) chains stay blocked.
+- **PLAN-07** — ready non-ManagementActor (mutually concurrent, file-disjoint): **{4, 6, 14, 19, 21, 26, 29 | 32}** (ArtifactDeploymentService / ConfigSecretScrubber / SecuredWriteRepository / ManagementEndpoints+options / Commons Browse-Cert message records / Security LoginThrottle / CentralUI PollGate — T29 and T32 share `AlarmSummary.razor`, so serialize those two). Ready **ManagementActor** tasks (→ the ONE global Lane M): **{8, 9, 10, 12, 28, 30, 31}** — **start with T10** (it unblocks T11, T13, T22, T23). **T33 = skip (dedupe).**
+- **PLAN-08** — ready: **{2, 3, 4, 5, 7, 8, 9, 11}** — each is a new per-component validator file or an isolated test/doc, so **mutually parallel**, with one caveat: **T4 shares `Host/Program.cs` with P06 T7** (sequence them). This is the initiative's single highest-parallelism slice (independent validator files). T6, T10 stay blocked.
+
+### Recommended concurrent execution
+
+Run up to **five plan-lane executor sessions concurrently** (one each for PLAN-04/05/06/07/08 — PLAN-01/02/03 are done), with these coordination rules layered on top:
+
+1. **One ManagementActor task at a time, initiative-wide** — route P05 T13 and every ready P07 ManagementActor task ({8,9,10,12,28,30,31}) through a single serialized "Lane M"; start with **P07 T10**.
+2. **One migration at a time** — only **P06 T8** is a ready migration; hold the snapshot until it lands.
+3. **`Host/Program.cs` mutex** — **P06 T7 ⟂ P08 T4**.
+4. Inside each session, dispatch that plan's ready, file-disjoint tasks as concurrent implementers per the lanes above; use **pathspec-scoped commits and verify each landed on HEAD** (known repo failure mode — see Execution hygiene).
+5. Highest-value immediate fan-out: **PLAN-08 T2/T3/T5/T7/T8/T9/T11** (independent validator + test + doc files) and **PLAN-07's non-ManagementActor octet** can run as many concurrent implementers today; the throughput ceiling is Lane M (ManagementActor) and the single migration lane.
+
 ## P0 — Do These First (regardless of plan order)
 
 **✅ All six delivered 2026-07-08** on branch `archreview-p0` (worktree). TDD throughout; full solution builds clean; touched suites green (Host, Communication 244, Security 163, Transport 127+63, IntegrationTests cluster rig). One finding surfaced — see the active-node-crash SBR gap in the deferred registry below.

archreview/plans/PLAN-04-data-audit-backbone.md

@@ -8,6 +8,10 @@
 
 **Tech Stack:** C#/.NET 8, Akka.NET (actors/TestKit), EF Core + SQL Server (central), Microsoft.Data.Sqlite (site), xUnit. Build: `dotnet build ZB.MOM.WW.ScadaBridge.slnx`. Test per-project: `dotnet test tests/<project> --filter <name>`. MSSQL-backed integration tests need `cd infra && docker compose up -d`. **EF migrations gotcha (repo-documented):** always build first and run `dotnet ef migrations add <Name>` WITHOUT `--no-build`, or it scaffolds an empty migration off the stale DLL.
 
+## Parallelization (incomplete tasks) — 2026-07-09
+
+**Open:** only **T26** (full `dotnet build`/`dotnet test` sweep) — T24 (design-doc reconciliation) completed 2026-07-09. T26 has no intra-plan parallelism and **no cross-plan shared-file conflict** (build/test only), so a PLAN-04 executor can run concurrently with any other plan's session. Whole-initiative view: [00-MASTER-TRACKER.md § Parallelization Map](00-MASTER-TRACKER.md#parallelization-map--incomplete-tasks-2026-07-09).
+
 ## Findings Coverage
 
 | # | Report finding (severity) | Task(s) |

archreview/plans/PLAN-04-data-audit-backbone.md.tasks.json

@@ -10,23 +10,23 @@
     { "id": 7, "subject": "Task 7: Honest append-only enforcement story (docs)", "status": "completed", "blockedBy": [6] },
     { "id": 8, "subject": "Task 8: SiteCalls filtered non-terminal index (migration)", "status": "completed", "blockedBy": [6] },
     { "id": 9, "subject": "Task 9: Bound the execution-tree edge scan with a root time window", "status": "completed", "blockedBy": [] },
-    { "id": 10, "subject": "Task 10: Fix the hardcoded-zero backlogTotal KPI history", "status": "pending", "blockedBy": [] },
-    { "id": 11, "subject": "Task 11: SiteCalls same-rank freshness upsert (unfreeze RetryCount/LastError)", "status": "pending", "blockedBy": [] },
-    { "id": 12, "subject": "Task 12: EF Core connection resiliency (EnableRetryOnFailure)", "status": "pending", "blockedBy": [] },
-    { "id": 13, "subject": "Task 13: Audit row for operator Retry on parked notifications", "status": "pending", "blockedBy": [] },
-    { "id": 14, "subject": "Task 14: Operator identity on the SiteCalls Retry/Discard relay", "status": "pending", "blockedBy": [13] },
-    { "id": 15, "subject": "Task 15: Composite keyset in the SiteCalls reconciliation pull contract (site side)", "status": "pending", "blockedBy": [] },
-    { "id": 16, "subject": "Task 16: Central keyset cursor — un-pin the SiteCalls reconciliation loop", "status": "pending", "blockedBy": [15] },
-    { "id": 17, "subject": "Task 17: Durable record for permanently-abandoned reconciliation rows", "status": "pending", "blockedBy": [] },
-    { "id": 18, "subject": "Task 18: Reconciliation NodeB failover (owned from report 08)", "status": "pending", "blockedBy": [] },
-    { "id": 19, "subject": "Task 19: Batch the KpiSample purge", "status": "pending", "blockedBy": [] },
-    { "id": 20, "subject": "Task 20: NotificationOutbox KPI — single-query aggregation, bounded oldest", "status": "pending", "blockedBy": [] },
-    { "id": 21, "subject": "Task 21: OPTION (RECOMPILE) on the SiteCalls catch-all query", "status": "pending", "blockedBy": [11] },
-    { "id": 22, "subject": "Task 22: Short first-tick delay on the three purge timers", "status": "pending", "blockedBy": [5, 16] },
-    { "id": 23, "subject": "Task 23: Low-severity cleanup batch (param chunking, KPI catalog, computed-column guard)", "status": "pending", "blockedBy": [1] },
-    { "id": 24, "subject": "Task 24: Design-doc reconciliation pass", "status": "pending", "blockedBy": [7, 17] },
-    { "id": 25, "subject": "Task 25: Bounded parallel notification delivery", "status": "pending", "blockedBy": [13] },
-    { "id": 26, "subject": "Final verification: full solution build + test sweep (infra up for MSSQL suites)", "status": "pending", "blockedBy": [3, 8, 9, 10, 12, 14, 18, 19, 20, 21, 22, 23, 24, 25] }
+    { "id": 10, "subject": "Task 10: Fix the hardcoded-zero backlogTotal KPI history", "status": "completed", "blockedBy": [] },
+    { "id": 11, "subject": "Task 11: SiteCalls same-rank freshness upsert (unfreeze RetryCount/LastError)", "status": "completed", "blockedBy": [] },
+    { "id": 12, "subject": "Task 12: EF Core connection resiliency (EnableRetryOnFailure)", "status": "completed", "blockedBy": [] },
+    { "id": 13, "subject": "Task 13: Audit row for operator Retry on parked notifications", "status": "completed", "blockedBy": [] },
+    { "id": 14, "subject": "Task 14: Operator identity on the SiteCalls Retry/Discard relay", "status": "completed", "blockedBy": [13] },
+    { "id": 15, "subject": "Task 15: Composite keyset in the SiteCalls reconciliation pull contract (site side)", "status": "completed", "blockedBy": [] },
+    { "id": 16, "subject": "Task 16: Central keyset cursor — un-pin the SiteCalls reconciliation loop", "status": "completed", "blockedBy": [15] },
+    { "id": 17, "subject": "Task 17: Durable record for permanently-abandoned reconciliation rows", "status": "completed", "blockedBy": [] },
+    { "id": 18, "subject": "Task 18: Reconciliation NodeB failover (owned from report 08)", "status": "completed", "blockedBy": [] },
+    { "id": 19, "subject": "Task 19: Batch the KpiSample purge", "status": "completed", "blockedBy": [] },
+    { "id": 20, "subject": "Task 20: NotificationOutbox KPI — single-query aggregation, bounded oldest", "status": "completed", "blockedBy": [] },
+    { "id": 21, "subject": "Task 21: OPTION (RECOMPILE) on the SiteCalls catch-all query", "status": "completed", "blockedBy": [11] },
+    { "id": 22, "subject": "Task 22: Short first-tick delay on the three purge timers", "status": "completed", "blockedBy": [5, 16] },
+    { "id": 23, "subject": "Task 23: Low-severity cleanup batch (param chunking, KPI catalog, computed-column guard)", "status": "completed", "blockedBy": [1] },
+    { "id": 24, "subject": "Task 24: Design-doc reconciliation pass", "status": "completed", "blockedBy": [7, 17] },
+    { "id": 25, "subject": "Task 25: Bounded parallel notification delivery", "status": "completed", "blockedBy": [13] },
+    { "id": 26, "subject": "Final verification: full solution build + test sweep (infra up for MSSQL suites)", "status": "completed", "blockedBy": [3, 8, 9, 10, 12, 14, 18, 19, 20, 21, 22, 23, 24, 25] }
   ],
-  "lastUpdated": "2026-07-09 (Batch 3: Tasks 7-9 completed)"
+  "lastUpdated": "2026-07-09 (Task 26 sweep GREEN — 26/26, PLAN-04 COMPLETE. Build 0/0; all 30 unit+integration assemblies pass after fixing 1 stale enum-count lock-in test (22d52390, Task 17 gap). Playwright E2E residue (8-9 fails) is environmental: stale 17h docker cluster, SQLite Error 14 unable-to-open-db on site-container instance deploy, non-deterministic set, zero PLAN-04 file overlap. Next: finishing-a-development-branch.)"
 }

archreview/plans/PLAN-05-templates-deployment-transport.md

@@ -8,6 +8,17 @@
 
 **Tech Stack:** C#/.NET, EF Core (in-memory + MSSQL integration fixtures), Roslyn (`Microsoft.CodeAnalysis.CSharp.Scripting`), xUnit. Build: `dotnet build ZB.MOM.WW.ScadaBridge.slnx`. Test per project: `dotnet test tests/<project>`.
 
+## Parallelization (incomplete tasks) — 2026-07-09
+
+Only T1 is complete. **Ready now** (all `blockedBy` done): **{2, 7, 9, 13, 14, 15, 16, 21, 22}**.
+
+**Concurrent lanes (dispatch as separate implementers):**
+- **Free / file-disjoint:** T7 (Export/DependencyResolver), T9 (`FlatteningService.cs`), T15 (TemplateEngine/Validation compile-cache), T16 (DeploymentManager pipeline), T21 (ScriptAnalysis/`ScriptTrustPolicy`), T22 (Transport/`LineDiffer`). Up to ~6 at once.
+- **`Transport/Import/BundleImporter.cs` mutex:** T2 ↔ T14 — one at a time (the downstream import chain T3→T4→T5→T6→T8 and T19→T20→T23 also serialize through this file).
+- **Global ManagementActor lane:** T13 (`ManagementActor.cs` + `FlatteningService.cs`) — runs in the initiative-wide ManagementActor mutex, and because it also edits `FlatteningService.cs` it is **not** concurrent with T9.
+
+**Cross-plan cautions:** T13 shares the `ManagementActor.cs` global single-writer with PLAN-06/07; T25 (docs) later touches `CLAUDE.md`. See [00-MASTER-TRACKER.md § Parallelization Map](00-MASTER-TRACKER.md#parallelization-map--incomplete-tasks-2026-07-09) for the full mutex table.
+
 ## Findings Coverage
 
 | Report finding | Severity | Task(s) |

archreview/plans/PLAN-06-edge-integrations.md

@@ -10,6 +10,17 @@
 
 Build: `dotnet build ZB.MOM.WW.ScadaBridge.slnx`. Test per-project: `dotnet test tests/<project>`. EF migrations: **build first, never `--no-build`**; `dotnet ef migrations add <Name> --project src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase --startup-project src/ZB.MOM.WW.ScadaBridge.Host`. If the migration scaffolds empty, delete the files and rebuild first.
 
+## Parallelization (incomplete tasks) — 2026-07-09
+
+Nothing complete yet. **Ready now** (`blockedBy` empty): **{1, 6, 7, 8, 17, 23}**.
+
+**Concurrent lanes (dispatch as separate implementers):**
+- **Free / file-disjoint:** T1 (`InboundScriptExecutor.cs`, high-risk), T6 (`InboundApiEndpointFilter.cs`, trivial), T17 (`SmsNotificationDeliveryAdapter.cs`), T23 (DelmiaNotifier docs).
+- **Migration holder (single-writer, initiative-wide):** T8 (ExternalSystem `TimeoutSeconds` entity + EF migration) — never run alongside another migration task (PLAN-06 T19, etc.). Unblocks the ExternalSystem chain T9–T16, T19.
+- **`Host/Program.cs` mutex:** T7 — coordinate with PLAN-08 T4 (both edit `Program.cs`).
+
+**Downstream chains (blocked until their roots land):** inbound contract T2→T3→T4→T5; ExternalSystem T8→{T9,T10,T11,T12,T19}→…; Notification T18/T19→T20/T21→T22. Several of these (T2, T3, T11, T20, T22) touch **`ManagementActor.cs`** → they enter the initiative-wide ManagementActor mutex when ready. T18/T21 own `NotificationOutboxActor.cs`. See [00-MASTER-TRACKER.md § Parallelization Map](00-MASTER-TRACKER.md#parallelization-map--incomplete-tasks-2026-07-09).
+
 ## Findings Coverage
 
 | # | Finding (report section) | Severity | Task(s) |

archreview/plans/PLAN-07-ui-management-security.md

@@ -10,6 +10,19 @@
 
 ---
 
+## Parallelization (incomplete tasks) — 2026-07-09
+
+T1–T3 complete. **Ready now** (`blockedBy` ⊆ {1,2,3}): **{4, 6, 8, 9, 10, 12, 14, 19, 21, 26, 28, 29, 30, 31, 32, 33}**.
+
+**This plan's defining constraint:** 15 tasks edit **`ManagementActor.cs`** (T5, T7, T8, T9, T10, T11, T12, T15, T16, T17, T22, T23, T28, T30, T31) — they **serialize into one lane**, and that lane is shared initiative-wide with PLAN-05 T13 and PLAN-06 (T2/T3/T11/T20/T22).
+
+**Concurrent lanes:**
+- **Global ManagementActor lane (one at a time):** ready members = **{8, 9, 10, 12, 28, 30, 31}**. **Start with T10** (GetRequiredRoles refactor) — it unblocks T11, T13, T22, T23.
+- **Free / file-disjoint (run as concurrent implementers):** T4 (`ArtifactDeploymentService.cs` — coordinate later with PLAN-06 T10), T6 (`ConfigSecretScrubber.cs`), T14 (`SecuredWriteRepository`), T19 (`ManagementEndpoints`/options), T21 (Commons Browse/Verify/Cert message records), T26 (Security `LoginThrottle`), and **T29 ↔ T32** (both edit `AlarmSummary.razor` — serialize the pair).
+- **Skip:** **T33** — CLI.Tests is already in the slnx via PLAN-08 T1 (done); mark `skipped-duplicate` in the `.tasks.json`, don't execute.
+
+See [00-MASTER-TRACKER.md § Parallelization Map](00-MASTER-TRACKER.md#parallelization-map--incomplete-tasks-2026-07-09) for the initiative-wide mutex table.
+
 ## Findings Coverage
 
 | Finding | Severity | Disposition |

archreview/plans/PLAN-08-conventions-tests.md

@@ -10,6 +10,15 @@
 
 **Build:** `dotnet build ZB.MOM.WW.ScadaBridge.slnx`. Tests: `dotnet test ZB.MOM.WW.ScadaBridge.slnx` — after Task 1 this picks up CLI.Tests too.
 
+## Parallelization (incomplete tasks) — 2026-07-09
+
+T1 complete. **Ready now** (`blockedBy` ⊆ {1}): **{2, 3, 4, 5, 7, 8, 9, 11}**.
+
+This is the **highest-parallelism slice in the initiative** — T2–T5 each create per-component options-validator files, and T7/T8/T9/T11 are isolated test/props/doc files, so they are **mutually parallel** and can run as many concurrent implementers at once, with a single caveat:
+- **`Host/Program.cs` mutex:** T4 edits `Program.cs` (shared with PLAN-06 T7) — sequence those two. (T3 edits `Host/SiteServiceRegistration.cs`, a *different* Host file — no conflict.)
+
+**Blocked:** T6 (`blockedBy` T3 — removes the vestigial site notification surface, shares the SiteRuntime project), T10 (`blockedBy` 1, 11 — docs-drift cleanup, touches `CLAUDE.md` + `ManagementService.csproj`, coordinate with T4 which also edits that csproj). See [00-MASTER-TRACKER.md § Parallelization Map](00-MASTER-TRACKER.md#parallelization-map--incomplete-tasks-2026-07-09).
+
 ## Findings Coverage
 
 | Report finding (08-conventions-tests-underdeveloped.md) | Severity | Coverage |

docs/requirements/Component-AuditLog.md

@@ -138,6 +138,7 @@ row per lifecycle event across all channels.
 | `SecuredWriteApprove` | A verifier wins the approval CAS for a pending secured write. |
 | `SecuredWriteReject` | A verifier rejects a pending secured write (`Status=Discarded`). |
 | `SecuredWriteExecute` | The approved write was relayed to the site MxGateway — terminal outcome (`Delivered`-equivalent on success, `Failed` on error). |
+| `ReconciliationAbandoned` | Central-direct synthetic row (`Status=Failed`) written when a reconciliation pull row failed to insert on every retry up to the permanent-abandon threshold and central advanced its cursor past it. `Extra` carries the abandoned `EventId`, the source site, and the final error — so the permanent loss is queryable, not only in the Critical log line. |
 
 Inbound API is intentionally collapsed to a single `InboundRequest` (or
 `InboundAuthFailure` for auth rejections) row per request rather than a
@@ -278,6 +279,26 @@ non-draining (e.g., telemetry actor wedged), central issues a
 are flipped to `ForwardState = 'Reconciled'` site-side. Same self-healing
 pattern as Site Call Audit's reconciliation of `SiteCalls`.
 
+**Endpoint resolution & NodeB failover.** Each pull dials the site's `NodeA`
+gRPC address first; if `NodeA` is blank the site's `NodeB` address becomes the
+primary (a NodeB-only site is no longer skipped — only a site with *both*
+addresses blank is). When a distinct `NodeB` address also exists it is carried
+as a fallback: a transport fault against the primary (site offline, deadline
+exceeded, cancelled, connection error) fails over to `NodeB` ONCE before
+collapsing to an empty batch. So during a NodeA outage the loss-recovery net
+stays available. The failover is scoped to transport faults — a
+mapping/unexpected fault collapses to empty without a second dial (the other
+node would hit the same fault). The same resolution + failover applies to Site
+Call Audit's `PullSiteCalls`.
+
+> **Cursor keyset (tracked follow-up).** The `PullAuditEvents` cursor is still a
+> single `sinceUtc` timestamp. Site Call Audit's pull now uses a composite
+> `(UpdatedAtUtc, TrackedOperationId)` keyset to avoid a single-timestamp pin
+> (see Component-SiteCallAudit.md → Reconciliation). The same keyset should be
+> applied here, but it is lower urgency because the audit cursor already re-pulls
+> idempotently on `EventId` — a saturated single-timestamp window re-inserts
+> harmless no-ops rather than losing rows.
+
 ### Central direct-write (central-originated events)
 
 Events originating at central never touch site SQLite. Inbound API writes one
@@ -386,6 +407,15 @@ MS SQL for direct-write events). Unredacted secrets never persist.
 - **Reconciliation as fallback.** If two consecutive reconciliation cycles
   report a non-draining backlog, the supervisor restarts the telemetry actor
   and a `SiteAuditTelemetryStalled` event fires.
+- **Permanent-abandonment record.** A reconciliation pull row that fails to
+  insert on every retry up to the permanent-abandon threshold is dropped and the
+  cursor advances past it (so one broken row can't block the site forever). The
+  loss is not silent: alongside the Critical log line the actor writes ONE
+  synthetic `ReconciliationAbandoned` audit row (fresh `EventId`, `Status=Failed`,
+  channel preserved from the lost row where parseable) whose `Extra` carries the
+  abandoned `EventId`, the source site, and the final error — so the permanent
+  loss is queryable in the Audit Log itself. That write is best-effort in its own
+  try/catch and never blocks the cursor a second time.
 - **No dedup horizon.** `EventId` PK enforces uniqueness only while a row
   exists. A retry that arrives after the original row is purged inserts a "new"
   row — vanishingly rare and harmless.
@@ -422,7 +452,8 @@ MS SQL for direct-write events). Unredacted secrets never persist.
   `UX_AuditLog_EventId` until a later tick's CATCH branch rebuilt it.
 - **Per-channel retention overrides (M5.5 T3):** `AuditLog:PerChannelRetentionDays`
   is a dictionary keyed by canonical channel name (`ApiOutbound`, `DbOutbound`,
-  `Notification`, `ApiInbound`) whose value is a retention window in days that
+  `Notification`, `ApiInbound`, `SecuredWrite` — all five `AuditChannel` values are
+  accepted by the validator) whose value is a retention window in days that
   MUST be strictly shorter than the global `RetentionDays`. After the daily
   partition switch-out, the purge actor runs a bounded, batched row DELETE
   (`PurgeChannelOlderThanAsync`) for each channel whose override is shorter than

docs/requirements/Component-CentralUI.md

@@ -75,7 +75,7 @@ Central cluster only. Sites have no user interface.
 
 ### Notification List Management (Design Role)
 - Create, edit, and delete notification lists.
-- Each notification list has a **`Type`** — `Email` now, with `Teams` and other types planned. The type determines the type-specific targets a list carries.
+- Each notification list has a **`Type`** — `Email` or `Sms` (SMS via Twilio shipped 2026-06-19; a Teams adapter was evaluated and dropped). The type determines the type-specific targets a list carries.
 - Manage recipients (name + email) within each `Email` list.
 - Configure SMTP settings.
 

docs/requirements/Component-Commons.md

@@ -43,11 +43,12 @@ Commons must define shared primitive and utility types used across multiple comp
 - **`TrackedOperationId`**: A GUID identifying a tracked store-and-forward operation (`ExternalSystem.CachedCall`, `Database.CachedWrite`, `Notify.Send`). Generated caller-side at the site at call time, returned to the script as a tracking handle, and reused as the idempotency key for telemetry sent to central. The notification domain's existing `NotificationId` is the notification-specific name for this same concept.
 - **`TrackedOperationKind` enum**: ExternalCall, DatabaseWrite. Discriminates the two cached-call kinds carried by a tracked operation (notifications are tracked separately via the `NotificationType` enum).
 - **`TrackedOperationStatus` enum**: Pending, Retrying, Delivered, Parked, Failed, Discarded. The unified lifecycle state shared by all tracked store-and-forward operations. This is the operation's externally-observable lifecycle status in the site-local tracking table (the status record); it is related to but distinct from the S&F buffer's own `StoreAndForwardMessageStatus`, which tracks a buffered message's retry state within the buffer (the retry mechanism). `Failed` (permanent failure) has no notification analogue — notifications use only the other five states (the `NotificationStatus` enum omits `Failed`).
-- **`AuditChannel` enum**: ApiOutbound, DbOutbound, Notification, ApiInbound. Discriminates the script-trust-boundary channel that produced an `AuditEvent`. Owned by the Audit Log component.
-- **`AuditKind` enum**: ApiCall, ApiCallCached, DbWrite, DbWriteCached, NotifySend, NotifyDeliver, InboundRequest, InboundAuthFailure, CachedSubmit, CachedResolve. Channel-specific event kind — the valid `Kind` values for each `AuditChannel` are listed in the Audit Log component design (`Component-AuditLog.md`).
+- **`AuditChannel` enum**: ApiOutbound, DbOutbound, Notification, ApiInbound, SecuredWrite. Discriminates the script-trust-boundary channel that produced an `AuditEvent` (`SecuredWrite` extends the boundary to the two-person MxGateway secured-write flow). Owned by the Audit Log component.
+- **`AuditKind` enum**: ApiCall, ApiCallCached, DbWrite, DbWriteCached, NotifySend, NotifyDeliver, InboundRequest, InboundAuthFailure, CachedSubmit, CachedResolve, SecuredWriteSubmit, SecuredWriteApprove, SecuredWriteReject, SecuredWriteExecute, ReconciliationAbandoned. Channel-specific event kind — the valid `Kind` values for each `AuditChannel` are listed in the Audit Log component design (`Component-AuditLog.md`). (`SecuredWrite*` are the four secured-write lifecycle events; `ReconciliationAbandoned` is the durable record emitted when a reconciliation pull permanently fails to insert a lost row — arch-review 04.)
 - **`AuditStatus` enum**: Submitted, Forwarded, Attempted, Delivered, Failed, Parked, Discarded, Skipped. Lifecycle status of an audit event row; cached operations transit Submitted → Forwarded → Attempted → Delivered/Parked/Discarded. `Skipped` covers short-circuited (e.g. dry-run) actions that should still be audited.
 - **`AuditForwardState` enum**: Pending, Forwarded, Reconciled. Site-local SQLite flag governing the telemetry/reconciliation loop (set on a row but never sent to central).
 - **`AuditEvent`** (`ZB.MOM.WW.Audit.AuditEvent` — external NuGet package, not a Commons entity class): The canonical, transport-agnostic audit record from the `ZB.MOM.WW.Audit` package that Commons references. Carries the 10 canonical fields: `EventId` (GUID, idempotency key), `OccurredAtUtc`, `Actor`, `Action` (format `"{channel}.{kind}"`), `Outcome` (`AuditOutcome`: `Success`/`Failure`/`Denied`), `Category` (channel name for ScadaBridge), `Target`, `SourceNode`, `CorrelationId`, and `DetailsJson` (JSON extension bag). All ScadaBridge-specific fields — `Channel`, `Kind`, `Status`, `ExecutionId`, `ParentExecutionId`, `SourceSiteId`, `SourceInstanceId`, `SourceScript`, `HttpStatus`, `DurationMs`, `ErrorMessage`, `ErrorDetail`, `RequestSummary`, `ResponseSummary`, `PayloadTruncated`, `Extra`, `IngestedAtUtc` — are carried in `DetailsJson` and are not top-level properties of `AuditEvent`. The `AuditChannel`, `AuditKind`, `AuditStatus`, and `AuditForwardState` enums defined in Commons (under `Types/Enums/`) describe the vocabulary encoded in `DetailsJson` and in the `AuditLogRow` computed columns. The site-local `ForwardState` (`AuditForwardState`) flag used by the telemetry/reconciliation loop is stored separately in the site SQLite `AuditLog` table. See `Component-AuditLog.md` for the persistence schema and ingest semantics.
+  - **`AuditLogRow` deviation (documented, intentional):** the central `dbo.AuditLog` EF entity `AuditLogRow` is a **persistence-only projection** of the external `ZB.MOM.WW.Audit.AuditEvent` contract — it lives in the Configuration Database component, **not** as a Commons POCO. This is a deliberate exception to the "entities are persistence-ignorant POCOs in Commons" rule: the canonical record is owned by the external `ZB.MOM.WW.Audit` package, and `AuditLogRow` exists only to give EF a table shape (canonical columns + `DetailsJson`-derived computed columns) to read/write. Do not lift it into Commons.
 - **`SiteCall`**: A record carrying the central `SiteCalls` operational-mirror row — `TrackedOperationId`, `Channel` (the trust-boundary channel, `ApiOutbound` or `DbOutbound`), `Target` (human-readable call target), `SourceSite` (site id), `SourceNode` (emitting node, nullable), `Status` (string form of `AuditStatus`; monotonic), `RetryCount`, `LastError`, `HttpStatus`, `CreatedAtUtc`, `UpdatedAtUtc`, `TerminalAtUtc`, `IngestedAtUtc` — fed by site `CachedCallTelemetry` and the periodic reconciliation pull.
 
 Types defined here must be immutable and thread-safe.
@@ -86,7 +87,7 @@ Entity classes are organized by domain area:
 - **Deployment**: `DeploymentRecord`, `SystemArtifactDeploymentRecord`, `DeployedConfigSnapshot`.
 - **Audit**: `AuditLogEntry` (configuration-change audit, owned by Configuration Database), `SiteCall` (`SiteCalls` operational-mirror row). (`AuditEvent` is `ZB.MOM.WW.Audit.AuditEvent` from the external NuGet package — not a Commons entity class file; see REQ-COM-1.)
 
-The **`Notification`** entity is the persistence-ignorant POCO for a row of the central `Notifications` table — the durable notification queue owned by the Notification Outbox. It is a plain class with properties for `NotificationId` (GUID, the idempotency key), `Type` (`NotificationType` enum discriminator), `ListName`, `Subject`, `Body`, `TypeData` (a JSON string — the type-agnostic extensibility hook), `Status` (`NotificationStatus` enum), `RetryCount`, `LastError`, `ResolvedTargets`, the provenance fields `SourceSiteId` / `SourceInstanceId` / `SourceScript`, and the UTC timestamps `SiteEnqueuedAt`, `CreatedAt`, `LastAttemptAt`, `NextAttemptAt`, `DeliveredAt`. As with every entity class it has no EF dependency; the Configuration Database component supplies the Fluent API mapping, value conversions, and indexes. The `Type` and `Status` enums (`NotificationType`: `Email` — currently the only value; `Teams` and other channels are planned but not yet present in the enum; `NotificationStatus`: `Pending`, `Retrying`, `Delivered`, `Parked`, `Discarded`) are defined under `Types/Enums/` per REQ-COM-1.
+The **`Notification`** entity is the persistence-ignorant POCO for a row of the central `Notifications` table — the durable notification queue owned by the Notification Outbox. It is a plain class with properties for `NotificationId` (GUID, the idempotency key), `Type` (`NotificationType` enum discriminator), `ListName`, `Subject`, `Body`, `TypeData` (a JSON string — the type-agnostic extensibility hook), `Status` (`NotificationStatus` enum), `RetryCount`, `LastError`, `ResolvedTargets`, the provenance fields `SourceSiteId` / `SourceInstanceId` / `SourceScript`, and the UTC timestamps `SiteEnqueuedAt`, `CreatedAt`, `LastAttemptAt`, `NextAttemptAt`, `DeliveredAt`. As with every entity class it has no EF dependency; the Configuration Database component supplies the Fluent API mapping, value conversions, and indexes. The `Type` and `Status` enums (`NotificationType`: `Email`, `Sms` — SMS via Twilio shipped 2026-06-19; a Microsoft Teams adapter was evaluated and dropped, because an outbound-only Graph client cannot send 1:1 chat bodies without a Bot Framework inbound endpoint; `NotificationStatus`: `Pending`, `Retrying`, `Delivered`, `Parked`, `Discarded`) are defined under `Types/Enums/` per REQ-COM-1.
 
 ### REQ-COM-4: Per-Component Repository Interfaces
 

docs/requirements/Component-ConfigurationDatabase.md

@@ -55,13 +55,20 @@ The configuration database stores all central system data, organized by domain a
 - **Database Connection Definitions**: Named database connections (name, connection details, retry settings).
 
 ### Notifications
-- **Notification Lists**: List definitions (name, `Type` discriminator — `Email` / `Teams` / … — selecting the list's notification type and type-specific targets).
+- **Notification Lists**: List definitions (name, `Type` discriminator — `Email` / `Sms` — selecting the list's notification type and type-specific targets).
 - **Notification Recipients**: Recipients per list (name, email address).
 - **SMTP Configuration**: Email server settings.
-- **Notifications**: The durable central notification queue owned by the Notification Outbox — one row per notification, the single source of audit truth. The schema is **type-agnostic** so it records any notification type the system supports (email today, Microsoft Teams and others later): a `Type` discriminator selects the type, and a `TypeData` JSON column (`nvarchar(max)`) carries any future per-type fields without a schema change. Columns: `NotificationId` (GUID, primary key — generated at the site, used as the idempotency key), `Type`, `ListName`, `Subject`, `Body`, `TypeData`, `Status`, `RetryCount`, `LastError`, `ResolvedTargets`, `SourceSiteId`, `SourceInstanceId`, `SourceScript`, `SiteEnqueuedAt`, `CreatedAt`, `LastAttemptAt`, `NextAttemptAt`, `DeliveredAt`. `Status` is a `NotificationStatus` enum stored with values `Pending`, `Retrying`, `Delivered`, `Parked`, `Discarded` (the site-local `Forwarding` state is never persisted centrally). Indexed on `Status` and `NextAttemptAt` for efficient dispatcher polling of due rows, and on `SourceSiteId` and `CreatedAt` for KPI computation and the Central UI query page. Terminal rows are removed by a daily purge job — see Scheduled Maintenance below. See Component-NotificationOutbox.md for the full lifecycle.
+- **Notifications**: The durable central notification queue owned by the Notification Outbox — one row per notification, the single source of audit truth. The schema is **type-agnostic** so it records any notification type the system supports (email and SMS today; the type-agnostic shape absorbs future types without a migration): a `Type` discriminator selects the type, and a `TypeData` JSON column (`nvarchar(max)`) carries any future per-type fields without a schema change. Columns: `NotificationId` (GUID, primary key — generated at the site, used as the idempotency key), `Type`, `ListName`, `Subject`, `Body`, `TypeData`, `Status`, `RetryCount`, `LastError`, `ResolvedTargets`, `SourceSiteId`, `SourceInstanceId`, `SourceScript`, `SiteEnqueuedAt`, `CreatedAt`, `LastAttemptAt`, `NextAttemptAt`, `DeliveredAt`. `Status` is a `NotificationStatus` enum stored with values `Pending`, `Retrying`, `Delivered`, `Parked`, `Discarded` (the site-local `Forwarding` state is never persisted centrally). Indexed on `Status` and `NextAttemptAt` for efficient dispatcher polling of due rows, and on `SourceSiteId` and `CreatedAt` for KPI computation and the Central UI query page. Terminal rows are removed by a daily purge job — see Scheduled Maintenance below. See Component-NotificationOutbox.md for the full lifecycle.
 
 ### Site Calls
-- **SiteCalls**: The central audit table for cached site calls — `ExternalSystem.CachedCall()` and `Database.CachedWrite()` — owned by the Site Call Audit component and a sibling of the `Notifications` table. One row per cached operation. Columns: `TrackedOperationId` (`varchar(36)`, primary key — the GUID idempotency key stored in "D"-format string, matching the site SQLite and gRPC wire format), `Channel` (`varchar(32)` — `ApiOutbound` or `DbOutbound`; the trust-boundary channel that produced the call), `Target` (`varchar(256)` — human-readable target, e.g. `ERP.GetOrder` for an external call), `SourceSite` (`varchar(64)` — the site that submitted the cached call), `SourceNode` (`varchar(64)` NULL — cluster node that emitted the call; null for reconciled rows from retired nodes), `Status` (`varchar(32)` — `AuditStatus` enum name: `Submitted`, `Forwarded`, `Attempted`, `Delivered`, `Failed`, `Parked`, `Discarded`; lifecycle is monotonic), `RetryCount` (`int`), `LastError` (`nvarchar(1024)` NULL), `HttpStatus` (`int` NULL — last HTTP status code for API calls), `CreatedAtUtc` (`datetime2`), `UpdatedAtUtc` (`datetime2`), `TerminalAtUtc` (`datetime2` NULL), `IngestedAtUtc` (`datetime2` — central ingest timestamp). The table is populated **only** by Site Call Audit telemetry and reconciliation pulls — sites are the source of truth and the row is an eventually-consistent mirror, never written by a central dispatcher. Ingestion is **insert-if-not-exists** keyed on `TrackedOperationId`, then **upsert-on-newer-status**; the lifecycle is monotonic, so at-least-once and out-of-order telemetry are harmless. Indexes: `IX_SiteCalls_Source_Created (SourceSite, CreatedAtUtc DESC)` for per-site queries and `IX_SiteCalls_Status_Updated (Status, UpdatedAtUtc DESC)` for status-filtered views. Terminal rows are removed by a daily purge job — see Scheduled Maintenance below. See Component-SiteCallAudit.md for the full lifecycle.
+- **SiteCalls**: The central audit table for cached site calls — `ExternalSystem.CachedCall()` and `Database.CachedWrite()` — owned by the Site Call Audit component and a sibling of the `Notifications` table. One row per cached operation. Columns: `TrackedOperationId` (`varchar(36)`, primary key — the GUID idempotency key stored in "D"-format string, matching the site SQLite and gRPC wire format), `Channel` (`varchar(32)` — `ApiOutbound` or `DbOutbound`; the trust-boundary channel that produced the call), `Target` (`varchar(256)` — human-readable target, e.g. `ERP.GetOrder` for an external call), `SourceSite` (`varchar(64)` — the site that submitted the cached call), `SourceNode` (`varchar(64)` NULL — cluster node that emitted the call; null for reconciled rows from retired nodes), `Status` (`varchar(32)` — `AuditStatus` enum name: `Submitted`, `Forwarded`, `Attempted`, `Delivered`, `Failed`, `Parked`, `Discarded`; lifecycle is monotonic), `RetryCount` (`int`), `LastError` (`nvarchar(1024)` NULL), `HttpStatus` (`int` NULL — last HTTP status code for API calls), `CreatedAtUtc` (`datetime2`), `UpdatedAtUtc` (`datetime2`), `TerminalAtUtc` (`datetime2` NULL), `IngestedAtUtc` (`datetime2` — central ingest timestamp). The table is populated **only** by Site Call Audit telemetry and reconciliation pulls — sites are the source of truth and the row is an eventually-consistent mirror, never written by a central dispatcher. Ingestion is **insert-if-not-exists** keyed on `TrackedOperationId`, then **upsert-on-newer-status**; the lifecycle is monotonic, so at-least-once and out-of-order telemetry are harmless. Indexes: `IX_SiteCalls_Source_Created (SourceSite, CreatedAtUtc DESC)` for per-site queries, `IX_SiteCalls_Status_Updated (Status, UpdatedAtUtc DESC)` for status-filtered views, and the filtered `IX_SiteCalls_NonTerminal` (on `CreatedAtUtc`, `WHERE TerminalAtUtc IS NULL`, INCLUDE `SourceSite, SourceNode, Status`) that backs the live-queue KPI predicates. Terminal rows are removed by a daily purge job — see Scheduled Maintenance below. See Component-SiteCallAudit.md for the full lifecycle.
+
+### Data-layer conventions & tracked follow-ups (arch-review 04)
+
+- **New tables use UTC `DateTime` + the UTC value converter**, not `DateTimeOffset`. The `Notification` entity's `DateTimeOffset` timestamps are the **documented legacy exception** — on the SQLite test harness the `DateTimeOffset`→ISO-8601-string converter forced in-memory reductions in a few KPI paths (now pushed server-side where translatable). Prefer `datetime2` UTC columns for any new table.
+- **New repositories write MSSQL-only SQL with MSSQL-backed test fixtures** (`SkippableFact` gated on the live SQL Server container). The `NotificationOutboxRepository` dual SQLite/T-SQL dialect is the **documented legacy exception** (finding S8) — it predates the MSSQL-fixture convention; do not copy that pattern.
+- **AuditLog clustered key.** The `dbo.AuditLog` primary/clustered key is `(EventId, OccurredAtUtc)` — `OccurredAtUtc` is required by the monthly partition scheme (`ps_AuditLog_Month`), and `EventId` carries a separate unique index for `InsertIfNotExists` idempotency. **Tracked follow-up (P3):** benchmark whether flipping the clustered key to `(OccurredAtUtc, EventId)` improves range-scan locality once real volume exists.
+- **Tracked follow-ups (P4/P6):** the per-node KPI sampling cadence and the AuditLog reconciliation-pull keyset paging (the SiteCalls reconciliation pull already uses a composite keyset cursor; `PullAuditEvents` remains inclusive-`>=`, idempotent on `EventId`) are deferred, lower-urgency hardening items.
 
 ### Audit Log
 - **AuditLog**: The central, append-only audit table owned by the Audit Log component — one row per script-trust-boundary lifecycle event across all channels (outbound API calls, outbound DB writes/reads, notifications, and inbound API requests). Sibling of the `Notifications` and `SiteCalls` tables but distinct: `AuditLog` is the immutable history that observes the other subsystems, not an operational state store.

docs/requirements/Component-NotificationOutbox.md

@@ -20,7 +20,7 @@ Central cluster. The `NotificationOutboxActor` is a **singleton on the active ce
 - Compute delivery KPIs from the `Notifications` table for the Health Monitoring dashboard and the Central UI.
 - Purge terminal rows daily after a configurable retention window.
 
-SMTP and HTTP delivery is blocking I/O. Delivery work runs on a **dedicated blocking-I/O dispatcher**, the same pattern used by Script Execution Actors, so delivery never blocks the actor's dispatcher loop.
+SMTP and HTTP delivery is blocking I/O. A dispatch sweep runs **off the actor thread** on the thread pool (the sweep task pipes its completion back to the actor), so delivery never blocks the actor's message loop. Within a sweep, deliveries run with **bounded per-sweep parallelism** — a `SemaphoreSlim` caps the number of concurrent deliveries at `MaxParallelDeliveries` (default 4; set to 1 for strictly sequential delivery). Each concurrent delivery uses its own DI scope/repository, so there is no shared-`DbContext` contention.
 
 ## End-to-End Flow
 
@@ -117,7 +117,7 @@ The dispatcher loop runs on a fixed interval. On each tick the `NotificationOutb
 
 1. Polls the `Notifications` table for **due rows** — `Pending` rows, and `Retrying` rows whose `NextAttemptAt` has passed.
 2. Resolves the target notification list to its recipients/targets at central, at delivery time.
-3. Hands the notification to the delivery adapter registered for its `Type`, running on the dedicated blocking-I/O dispatcher.
+3. Hands the notification to the delivery adapter registered for its `Type`. Deliveries within the sweep run with bounded parallelism (`MaxParallelDeliveries`, default 4), each on its own DI scope/repository, off the actor thread.
 4. Applies the result:
    - **success** → `Delivered`, set `DeliveredAt`, snapshot `ResolvedTargets`.
    - **transient failure** → `Retrying`, increment `RetryCount`, set `NextAttemptAt`, record `LastError`; once retries are exhausted → `Parked`.
@@ -125,7 +125,7 @@ The dispatcher loop runs on a fixed interval. On each tick the `NotificationOutb
 
 Each delivery attempt also writes a `Notification.Attempt` row to the central `AuditLog` via `ICentralAuditWriter`; a transition to a terminal status (`Delivered` / `Parked` / `Discarded`) writes a `Notification.Terminal` row. Audit writes are **direct** (no telemetry — the dispatcher runs at central), insert-if-not-exists on `EventId`. The site-emitted `Notification.Enqueued` row arrives separately via the standard audit telemetry channel from the site's SQLite write-buffer, so the full per-notification audit trail is `Enqueued` (site-originated) → `Attempt` × N (central direct-write) → `Terminal` (central direct-write). See [Component-AuditLog.md](Component-AuditLog.md), Central direct-write (central-originated events).
 
-The operational `Notifications` table remains the **source of truth** for the dispatcher and for Retry/Discard actions; the `AuditLog` rows are immutable shadows. Operator Retry/Discard still mutates only the `Notifications` row, and each transition emits the corresponding `Notification.Attempt` / `Notification.Terminal` audit row.
+The operational `Notifications` table remains the **source of truth** for the dispatcher and for Retry/Discard actions; the `AuditLog` rows are immutable shadows. Operator Retry/Discard still mutates only the `Notifications` row, and each transition emits the corresponding audit row **carrying the operator identity as `Actor`** so an operator action is attributable: an operator **Retry** on a parked notification emits a `Notification`-channel `NotifyDeliver` row with status `Submitted` (recording *who un-parked it* — otherwise the trail reads `Parked → Attempt → Delivered` with no operator on record); a **Discard** emits the `Notification.Terminal` row with the operator as `Actor`. The username is captured at the Central UI and flows in on `RetryNotificationRequest` / `DiscardNotificationRequest` (`RequestedBy`); it is `null` (and the `Actor` falls back to `system`) only where no authenticated identity exists. These operator-attributed rows are best-effort like every other audit write — a writer failure never aborts the Retry/Discard.
 
 **Audit-write failure never affects delivery.** If the `ICentralAuditWriter` direct-write fails (transient DB error, schema lock, etc.) the dispatcher logs the failure and increments the `CentralAuditWriteFailures` health metric (see Health Monitoring #11), but the delivery attempt's outcome on the `Notifications` row stands. The audit row is recovered by re-emission on the next dispatcher tick or by the on-startup reconciliation sweep; central never aborts a notification because audit failed.
 

docs/requirements/Component-SiteCallAudit.md

@@ -32,28 +32,50 @@ central node. Registered as component #22 in the Host role configuration.
 ## The `SiteCalls` Table
 
 Lives in the central MS SQL configuration database — a sibling of the
-`Notifications` table. One row per `TrackedOperationId`:
+`Notifications` table. One row per `TrackedOperationId` (the shipped columns, as
+mapped by `SiteCallEntityTypeConfiguration` — the source of truth is
+`Component-ConfigurationDatabase.md` § Site Calls):
 
-- **TrackedOperationId** — GUID, primary key. Generated site-side at call time.
-- **SourceSite** — site that issued the call.
-- **SourceNode** — the cluster node on which the call was issued (`node-a` /
-  `node-b`, qualified by `SourceSite`). Nullable. Stamped site-side at submit
-  time and carried verbatim through the combined `CachedCallTelemetry` packet,
-  reconciliation pulls, and the central upsert.
-- **Kind** — `TrackedOperationKind` enum: `ExternalCall` or `DatabaseWrite`.
-- **TargetSummary** — external system + method name for an `ExternalCall`; for a
-  `DatabaseWrite`, just the database connection name — intentionally not the SQL
-  statement or table, a deliberate scoping choice.
-- **Status** — `Pending`, `Retrying`, `Delivered`, `Parked`, `Failed`, `Discarded`.
-- **RetryCount** — attempts so far.
-- **LastError** — most recent error detail, if any.
-- **Provenance** — source instance / script.
-- **CreatedAtUtc**, **UpdatedAtUtc**, **TerminalAtUtc** — key timestamps.
+- **TrackedOperationId** — GUID (`varchar(36)`, "D"-format), primary key. Generated
+  site-side at call time.
+- **Channel** — `varchar(32)`, the trust-boundary channel that produced the call:
+  `ApiOutbound` (`ExternalSystem.CachedCall()`) or `DbOutbound` (`Database.CachedWrite()`).
+- **Target** — `varchar(256)`, human-readable target (e.g. `ERP.GetOrder` for an
+  external call, or the database connection name for a cached write — intentionally
+  not the SQL statement or table, a deliberate scoping choice).
+- **SourceSite** — `varchar(64)`, site that issued the call.
+- **SourceNode** — `varchar(64)` NULL, the cluster node on which the call was issued
+  (`node-a` / `node-b`, qualified by `SourceSite`). Stamped site-side at submit time
+  and carried verbatim through the combined `CachedCallTelemetry` packet,
+  reconciliation pulls, and the central upsert; NULL for reconciled rows from a
+  retired node.
+- **Status** — `varchar(32)`, the `AuditStatus` enum name (**not** the tracking-lifecycle
+  names): `Submitted`, `Forwarded`, `Attempted`, `Delivered`, `Failed`, `Parked`,
+  `Discarded`. The lifecycle is monotonic, so out-of-order/at-least-once telemetry is
+  harmless.
+- **RetryCount** — `int`, attempts so far.
+- **LastError** — `nvarchar(1024)` NULL, most recent error detail, if any.
+- **HttpStatus** — `int` NULL, last HTTP status code for API calls.
+- **CreatedAtUtc**, **UpdatedAtUtc** (`datetime2`), **TerminalAtUtc** (`datetime2` NULL),
+  **IngestedAtUtc** (`datetime2`, central ingest timestamp) — key timestamps.
+
+There are **no `Kind`, `TargetSummary`, or provenance (instance/script) columns** —
+those were an earlier design that did not ship; provenance detail for a cached call
+lives on the site's own tracking store and in the AuditLog rows.
 
 ## Status Lifecycle
 
 `Pending → Retrying → Delivered / Parked / Failed / Discarded`
 
+> **Stored vs. tracking view.** This lifecycle is the operator-facing *tracking*
+> view (what the site's tracking store and `Tracking.Status()` express). The
+> persisted `SiteCalls.Status` **column** stores the monotonic `AuditStatus`-derived
+> string (`Submitted` / `Forwarded` / `Attempted` / `Delivered` / `Failed` / `Parked`
+> / `Discarded`) carried by the combined telemetry packet — the mirror records the
+> audit-event status, not the tracking enum name. The two agree on the terminal
+> outcomes; the non-terminal `Submitted`/`Forwarded`/`Attempted` strings are the
+> ingest-phase equivalents of `Pending`/`Retrying`.
+
 - **Pending** — non-terminal: buffered after a transient failure, awaiting its
   first retry.
 - **Retrying** — non-terminal: undergoing retry attempts.
@@ -73,9 +95,15 @@ mirror — never queried by scripts (`Tracking.Status()` is answered site-locall
 ## Ingest & Idempotency
 
 Telemetry ingestion is **insert-if-not-exists** keyed on `TrackedOperationId`,
-then **upsert-on-newer-status**. The lifecycle is monotonic, so status only
-advances and never regresses; at-least-once and out-of-order telemetry are
-therefore harmless.
+then **upsert-on-newer-status, with a newest-`UpdatedAtUtc` tiebreaker within
+equal non-terminal rank**. The lifecycle is monotonic on status rank, so status
+never regresses. Within an equal *non-terminal* rank (the `Attempted`/`Skipped`
+retry phase), the packet with the newest `UpdatedAtUtc` wins — so a retrying
+call's `RetryCount`/`LastError`/`HttpStatus` stay live instead of freezing at the
+first `Attempted` write. Equal *terminal* rank stays immutable (a later
+`Delivered` never overwrites an earlier `Parked`), equal stamps are an idempotent
+no-op, and a lower rank is always a no-op — so at-least-once and out-of-order
+telemetry remain harmless.
 
 From v1.x onward, the `CachedCallTelemetry` message additively carries the
 `AuditEvent` content alongside the existing operational fields. Central's
@@ -92,6 +120,21 @@ reconnect — pulls "all tracking rows changed since cursor X" from each site.
 Gaps left by lost telemetry self-heal. Central converges to the site; the site
 never depends on central.
 
+The per-site cursor is a **composite `(UpdatedAtUtc, TrackedOperationId)`
+keyset**, not a single timestamp. Each pull asks for rows strictly greater than
+the cursor pair and advances it to the maximum row seen; a burst of more rows
+than one batch all sharing one exact `UpdatedAtUtc` therefore drains via the
+`TrackedOperationId` tiebreak instead of pinning the timestamp forever. The
+`after_id` keyset field is additive on the pull contract — a first pull (or a
+**legacy** site that predates it) sends no `after_id` and keeps the inclusive
+`>=` timestamp behaviour. When such a legacy site keeps reporting
+`MoreAvailable` yet the composite cursor cannot advance, the actor latches the
+site as *pinned* and publishes `SiteCallReconciliationPinnedChanged(siteId,
+Pinned)` on the EventStream (transition-only, mirroring
+`SiteAuditTelemetryStalledChanged`) — the un-drainable tail is a
+health-observable condition rather than a silent log line, and the latch clears
+with `Pinned=false` once a later tick makes progress.
+
 ## Retry / Discard Relay
 
 Parked cached calls live in the owning site's S&F buffer. Operator Retry/Discard
@@ -101,6 +144,17 @@ applies the change and emits telemetry reflecting the new state; central never
 mutates the `SiteCalls` row directly. If the site is offline the command fails
 fast and the UI surfaces a "site unreachable" message.
 
+On a **successful** relay (the site acks `Applied`), `SiteCallAuditActor` emits
+one best-effort central **direct-write** audit row (`CachedResolve`, status
+`Submitted` for a Retry / `Discarded` for a Discard) carrying the authenticated
+operator as `Actor` and the `TrackedOperationId` as `CorrelationId` — recording
+*who asked*. The operator identity flows in on `RetrySiteCallRequest` /
+`DiscardSiteCallRequest` (`RequestedBy`, captured at the Central UI). This row
+only adds provenance: the **site remains the source of truth** for the state
+change itself, and central reads the stored `SiteCalls` row solely to enrich the
+audit row's channel/target (a benign read, never a mutation). Audit is
+best-effort — a writer fault never changes the relay outcome.
+
 Only `Parked` rows are operator-actionable. `Failed` rows offer no Retry or
 Discard: a permanent failure (e.g. HTTP 4xx) would simply fail again, and the
 error was already returned synchronously to the calling script — there is

docs/requirements/HighLevelReqs.md

@@ -329,7 +329,7 @@ Scripts can interact with databases in two modes:
 
 ### 6.1 Notification Lists
 - Notification lists are **system-wide**, managed by users with the **Design** role.
-- Each list has a **name** and a **`Type`** (`Email` now; `Teams` and other types planned). The type determines the type-specific targets the list carries.
+- Each list has a **name** and a **`Type`** (`Email` or `Sms`; SMS via Twilio shipped 2026-06-19, a Teams adapter was evaluated and dropped). The type determines the type-specific targets the list carries.
 - An `Email` list contains one or more **recipients**, each with a **name** and an **email address**.
 - Notification lists are defined and stored **centrally only** — they are **not deployed to sites** and have no local SQLite copy. Recipient resolution happens at central, at delivery time.
 

src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/AuditLogIngestActor.cs

@@ -1,4 +1,5 @@
 using Akka.Actor;
+using Microsoft.EntityFrameworkCore;
 using Microsoft.Extensions.DependencyInjection;
 using Microsoft.Extensions.Logging;
 using ZB.MOM.WW.Audit;
@@ -259,37 +260,51 @@ public class AuditLogIngestActor : ReceiveActor
             // skip the registration.
             var failureCounter = scope.ServiceProvider.GetService<ICentralAuditWriteFailureCounter>();
 
+            // Task 12 (arch-review 04 S5): retry the dual-write under the DbContext's
+            // configured resiliency strategy. A manual (user-initiated) transaction is
+            // NOT auto-retried by EnableRetryOnFailure — EF executes directly while a
+            // transaction is active — so the whole { BEGIN; insert; upsert; COMMIT }
+            // unit is wrapped here to become a single retriable operation. The unit is
+            // idempotent (InsertIfNotExists on EventId + monotonic upsert on
+            // TrackedOperationId), so replay after a rolled-back transient failure is
+            // safe. NoOpExecutionStrategy on non-configured/test contexts runs it once.
+            var strategy = dbContext.Database.CreateExecutionStrategy();
+
             foreach (var entry in cmd.Entries)
             {
                 try
                 {
-                    await using var tx = await dbContext.Database
-                        .BeginTransactionAsync()
-                        .ConfigureAwait(false);
+                    await strategy.ExecuteAsync(async () =>
+                    {
+                        await using var tx = await dbContext.Database
+                            .BeginTransactionAsync()
+                            .ConfigureAwait(false);
 
-                    // Stamp IngestedAtUtc on both rows from a single
-                    // central-side instant so a join on the two tables sees
-                    // matching timestamps (debugging convenience, not a
-                    // correctness invariant).
-                    var ingestedAt = DateTime.UtcNow;
-                    // Redact the audit half BEFORE the dual-write — only the
-                    // AuditLog row's payload columns are redactable; SiteCalls
-                    // carries operational state only (status, retry count) and
-                    // is left untouched. Null redactor falls back
-                    // to SafeDefault so header redaction always runs.
-                    // IngestedAtUtc is a DetailsJson field
-                    // on the canonical record, so stamp it via the projection helper.
-                    var safeRedactor = redactor ?? SafeDefaultAuditRedactor.Instance;
-                    var filteredAudit = safeRedactor.Apply(entry.Audit);
-                    var auditStamped = AuditRowProjection.WithIngestedAtUtc(filteredAudit, ingestedAt);
-                    var siteCallStamped = entry.SiteCall with { IngestedAtUtc = ingestedAt };
+                        // Stamp IngestedAtUtc on both rows from a single
+                        // central-side instant so a join on the two tables sees
+                        // matching timestamps (debugging convenience, not a
+                        // correctness invariant).
+                        var ingestedAt = DateTime.UtcNow;
+                        // Redact the audit half BEFORE the dual-write — only the
+                        // AuditLog row's payload columns are redactable; SiteCalls
+                        // carries operational state only (status, retry count) and
+                        // is left untouched. Null redactor falls back
+                        // to SafeDefault so header redaction always runs.
+                        // IngestedAtUtc is a DetailsJson field
+                        // on the canonical record, so stamp it via the projection helper.
+                        var safeRedactor = redactor ?? SafeDefaultAuditRedactor.Instance;
+                        var filteredAudit = safeRedactor.Apply(entry.Audit);
+                        var auditStamped = AuditRowProjection.WithIngestedAtUtc(filteredAudit, ingestedAt);
+                        var siteCallStamped = entry.SiteCall with { IngestedAtUtc = ingestedAt };
 
-                    await auditRepo.InsertIfNotExistsAsync(auditStamped)
-                        .ConfigureAwait(false);
-                    await siteCallRepo.UpsertAsync(siteCallStamped)
-                        .ConfigureAwait(false);
+                        await auditRepo.InsertIfNotExistsAsync(auditStamped)
+                            .ConfigureAwait(false);
+                        await siteCallRepo.UpsertAsync(siteCallStamped)
+                            .ConfigureAwait(false);
+
+                        await tx.CommitAsync().ConfigureAwait(false);
+                    }).ConfigureAwait(false);
 
-                    await tx.CommitAsync().ConfigureAwait(false);
                     accepted.Add(entry.Audit.EventId);
                 }
                 catch (Exception ex)

src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/AuditLogPurgeActor.cs

@@ -5,6 +5,7 @@ using Microsoft.Extensions.Logging;
 using Microsoft.Extensions.Options;
 using ZB.MOM.WW.ScadaBridge.AuditLog.Configuration;
 using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
+using ZB.MOM.WW.ScadaBridge.Commons.Types;
 
 namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central;
 
@@ -93,7 +94,10 @@ public class AuditLogPurgeActor : ReceiveActor
         base.PreStart();
         var interval = _purgeOptions.Interval;
         _timer = Context.System.Scheduler.ScheduleTellRepeatedlyCancelable(
-            initialDelay: interval,
+            // Short first tick (min(interval, 5min)): purges are idempotent, and using the
+            // full interval as the initial delay meant a daily-recycled node never purged
+            // (arch-review 04, S9). Fast test cadences (interval < 5min) are unchanged.
+            initialDelay: PurgeTimerSchedule.InitialDelay(interval),
             interval: interval,
             receiver: Self,
             message: PurgeTick.Instance,

src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/GrpcPullAuditEventsClient.cs

@@ -76,7 +76,7 @@ public sealed class GrpcPullAuditEventsClient : IPullAuditEventsClient
         int batchSize,
         CancellationToken ct)
     {
-        var endpoint = await ResolveEndpointAsync(siteId, ct).ConfigureAwait(false);
+        var (endpoint, fallback) = await ResolveEndpointsAsync(siteId, ct).ConfigureAwait(false);
         if (endpoint is null)
         {
             // No gRPC address registered for the site — absence of an address is
@@ -95,44 +95,27 @@ public sealed class GrpcPullAuditEventsClient : IPullAuditEventsClient
             BatchSize = batchSize,
         };
 
-        ProtoPullResponse reply;
-        try
+        var (reply, transportFault) = await TryInvokeAsync(endpoint, request, siteId, ct)
+            .ConfigureAwait(false);
+
+        // NodeB failover (Task 18): a transport fault against the primary (NodeA)
+        // dials the fallback (NodeB) ONCE before collapsing to empty, so a NodeA
+        // outage doesn't take the reconciliation loss-recovery net offline. Only
+        // the transport-fault set triggers this (not a mapping/unexpected fault),
+        // and not when the caller's token is already cancelled (host shutdown).
+        if (reply is null && transportFault
+            && !string.IsNullOrWhiteSpace(fallback)
+            && !string.Equals(fallback, endpoint, StringComparison.Ordinal)
+            && !ct.IsCancellationRequested)
         {
-            reply = await _invoker.InvokeAsync(endpoint, request, ct).ConfigureAwait(false);
+            _logger.LogInformation(
+                "PullAuditEvents failing over from primary {Primary} to NodeB {Fallback} for site {SiteId}.",
+                endpoint, fallback, siteId);
+            (reply, _) = await TryInvokeAsync(fallback, request, siteId, ct).ConfigureAwait(false);
         }
-        catch (RpcException ex) when (IsTolerable(ex.StatusCode))
+
+        if (reply is null)
         {
-            _logger.LogDebug(ex,
-                "PullAuditEvents tolerable transport fault for site {SiteId} ({Endpoint}): {Status}. Returning empty batch.",
-                siteId, endpoint, ex.StatusCode);
-            return Empty;
-        }
-        catch (Exception ex) when (ex is HttpRequestException or System.Net.Sockets.SocketException)
-        {
-            _logger.LogDebug(ex,
-                "PullAuditEvents connection-layer fault for site {SiteId} ({Endpoint}). Returning empty batch.",
-                siteId, endpoint);
-            return Empty;
-        }
-        catch (OperationCanceledException)
-        {
-            // Reconciliation tick was cancelled — either the caller's token
-            // (host shutdown / scope dispose) or an internal gRPC deadline /
-            // linked-CTS cancellation. Both are tolerable for a best-effort
-            // pull; collapse to empty rather than letting an internal
-            // cancellation land noisily in the catch-all below.
-            return Empty;
-        }
-        catch (Exception ex)
-        {
-            // Any other fault (e.g. a malformed reply that fails DTO mapping
-            // below would actually surface here only if mapping moved inline,
-            // but a non-RpcException transport fault wrapper lands here too).
-            // Audit reconciliation is best-effort; swallow to empty rather than
-            // throw — the actor's per-site guard would only re-catch it.
-            _logger.LogWarning(ex,
-                "PullAuditEvents unexpected fault for site {SiteId} ({Endpoint}). Returning empty batch.",
-                siteId, endpoint);
             return Empty;
         }
 
@@ -147,7 +130,60 @@ public sealed class GrpcPullAuditEventsClient : IPullAuditEventsClient
         return new PullAuditEventsResponse(events, reply.MoreAvailable);
     }
 
-    private async Task<string?> ResolveEndpointAsync(string siteId, CancellationToken ct)
+    /// <summary>
+    /// Issues one pull against <paramref name="endpoint"/>, classifying faults.
+    /// Returns <c>(reply, false)</c> on success; <c>(null, true)</c> on a
+    /// tolerable TRANSPORT fault (the failover-eligible set:
+    /// <see cref="StatusCode.Unavailable"/> / <see cref="StatusCode.DeadlineExceeded"/> /
+    /// <see cref="StatusCode.Cancelled"/> / <see cref="HttpRequestException"/> /
+    /// <c>SocketException</c> / <see cref="OperationCanceledException"/>); and
+    /// <c>(null, false)</c> on any other (mapping/unexpected) fault, which
+    /// collapses to empty WITHOUT a fallback dial.
+    /// </summary>
+    private async Task<(ProtoPullResponse? Reply, bool TransportFault)> TryInvokeAsync(
+        string endpoint, ProtoPullRequest request, string siteId, CancellationToken ct)
+    {
+        try
+        {
+            var reply = await _invoker.InvokeAsync(endpoint, request, ct).ConfigureAwait(false);
+            return (reply, false);
+        }
+        catch (RpcException ex) when (IsTolerable(ex.StatusCode))
+        {
+            _logger.LogDebug(ex,
+                "PullAuditEvents tolerable transport fault for site {SiteId} ({Endpoint}): {Status}.",
+                siteId, endpoint, ex.StatusCode);
+            return (null, true);
+        }
+        catch (Exception ex) when (ex is HttpRequestException or System.Net.Sockets.SocketException)
+        {
+            _logger.LogDebug(ex,
+                "PullAuditEvents connection-layer fault for site {SiteId} ({Endpoint}).",
+                siteId, endpoint);
+            return (null, true);
+        }
+        catch (OperationCanceledException)
+        {
+            // Reconciliation tick was cancelled — either the caller's token
+            // (host shutdown / scope dispose) or an internal gRPC deadline /
+            // linked-CTS cancellation. Tolerable for a best-effort pull.
+            return (null, true);
+        }
+        catch (Exception ex)
+        {
+            // Any other fault (e.g. a malformed reply that fails DTO mapping).
+            // Audit reconciliation is best-effort; swallow to empty rather than
+            // throw — and do NOT fail over, since a second node would hit the
+            // same non-transport fault.
+            _logger.LogWarning(ex,
+                "PullAuditEvents unexpected fault for site {SiteId} ({Endpoint}). Returning empty batch.",
+                siteId, endpoint);
+            return (null, false);
+        }
+    }
+
+    private async Task<(string? Primary, string? Fallback)> ResolveEndpointsAsync(
+        string siteId, CancellationToken ct)
     {
         var sites = await _sites.EnumerateAsync(ct).ConfigureAwait(false);
         foreach (var site in sites)
@@ -155,10 +191,10 @@ public sealed class GrpcPullAuditEventsClient : IPullAuditEventsClient
             if (string.Equals(site.SiteId, siteId, StringComparison.Ordinal) &&
                 !string.IsNullOrWhiteSpace(site.GrpcEndpoint))
             {
-                return site.GrpcEndpoint;
+                return (site.GrpcEndpoint, site.FallbackGrpcEndpoint);
             }
         }
-        return null;
+        return (null, null);
     }
 
     private static readonly PullAuditEventsResponse Empty =

src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/GrpcPullSiteCallsClient.cs

@@ -80,10 +80,11 @@ public sealed class GrpcPullSiteCallsClient : IPullSiteCallsClient
     public async Task<PullSiteCallsResponse> PullAsync(
         string siteId,
         DateTime sinceUtc,
+        string? afterId,
         int batchSize,
         CancellationToken ct)
     {
-        var endpoint = await ResolveEndpointAsync(siteId, ct).ConfigureAwait(false);
+        var (endpoint, fallback) = await ResolveEndpointsAsync(siteId, ct).ConfigureAwait(false);
         if (endpoint is null)
         {
             // No gRPC address registered for the site — a configuration decision
@@ -100,43 +101,33 @@ public sealed class GrpcPullSiteCallsClient : IPullSiteCallsClient
             // EnsureUtc keeps Timestamp.FromDateTime happy (it requires UTC kind).
             SinceUtc = Timestamp.FromDateTime(EnsureUtc(sinceUtc)),
             BatchSize = batchSize,
+            // Composite-keyset tiebreak (Task 16). proto3 has no nullable string —
+            // an unset/empty AfterId is the site's signal to keep the legacy
+            // inclusive-timestamp contract (also what a first pull sends).
+            AfterId = afterId ?? string.Empty,
         };
 
-        ProtoPullResponse reply;
-        try
+        var (reply, transportFault) = await TryInvokeAsync(endpoint, request, siteId, ct)
+            .ConfigureAwait(false);
+
+        // NodeB failover (Task 18): a transport fault against the primary (NodeA)
+        // dials the fallback (NodeB) ONCE before collapsing to empty, so a NodeA
+        // outage doesn't take the reconciliation loss-recovery net offline. Only
+        // the transport-fault set triggers this (not a mapping/unexpected fault),
+        // and not when the caller's token is already cancelled (host shutdown).
+        if (reply is null && transportFault
+            && !string.IsNullOrWhiteSpace(fallback)
+            && !string.Equals(fallback, endpoint, StringComparison.Ordinal)
+            && !ct.IsCancellationRequested)
         {
-            reply = await _invoker.InvokeAsync(endpoint, request, ct).ConfigureAwait(false);
+            _logger.LogInformation(
+                "PullSiteCalls failing over from primary {Primary} to NodeB {Fallback} for site {SiteId}.",
+                endpoint, fallback, siteId);
+            (reply, _) = await TryInvokeAsync(fallback, request, siteId, ct).ConfigureAwait(false);
         }
-        catch (RpcException ex) when (IsTolerable(ex.StatusCode))
+
+        if (reply is null)
         {
-            _logger.LogDebug(ex,
-                "PullSiteCalls tolerable transport fault for site {SiteId} ({Endpoint}): {Status}. Returning empty batch.",
-                siteId, endpoint, ex.StatusCode);
-            return Empty;
-        }
-        catch (Exception ex) when (ex is HttpRequestException or System.Net.Sockets.SocketException)
-        {
-            _logger.LogDebug(ex,
-                "PullSiteCalls connection-layer fault for site {SiteId} ({Endpoint}). Returning empty batch.",
-                siteId, endpoint);
-            return Empty;
-        }
-        catch (OperationCanceledException)
-        {
-            // Reconciliation tick cancelled — caller token (host shutdown) or an
-            // internal gRPC deadline / linked-CTS cancellation. Both tolerable for
-            // a best-effort pull; collapse to empty rather than landing noisily in
-            // the catch-all below.
-            return Empty;
-        }
-        catch (Exception ex)
-        {
-            // Any other fault. Reconciliation is best-effort; swallow to empty
-            // rather than throw — the (future) actor's per-site guard would only
-            // re-catch it.
-            _logger.LogWarning(ex,
-                "PullSiteCalls unexpected fault for site {SiteId} ({Endpoint}). Returning empty batch.",
-                siteId, endpoint);
             return Empty;
         }
 
@@ -168,7 +159,59 @@ public sealed class GrpcPullSiteCallsClient : IPullSiteCallsClient
         return new PullSiteCallsResponse(siteCalls, reply.MoreAvailable);
     }
 
-    private async Task<string?> ResolveEndpointAsync(string siteId, CancellationToken ct)
+    /// <summary>
+    /// Issues one pull against <paramref name="endpoint"/>, classifying faults.
+    /// Returns <c>(reply, false)</c> on success; <c>(null, true)</c> on a
+    /// tolerable TRANSPORT fault (the failover-eligible set:
+    /// <see cref="StatusCode.Unavailable"/> / <see cref="StatusCode.DeadlineExceeded"/> /
+    /// <see cref="StatusCode.Cancelled"/> / <see cref="HttpRequestException"/> /
+    /// <c>SocketException</c> / <see cref="OperationCanceledException"/>); and
+    /// <c>(null, false)</c> on any other (mapping/unexpected) fault, which
+    /// collapses to empty WITHOUT a fallback dial.
+    /// </summary>
+    private async Task<(ProtoPullResponse? Reply, bool TransportFault)> TryInvokeAsync(
+        string endpoint, ProtoPullRequest request, string siteId, CancellationToken ct)
+    {
+        try
+        {
+            var reply = await _invoker.InvokeAsync(endpoint, request, ct).ConfigureAwait(false);
+            return (reply, false);
+        }
+        catch (RpcException ex) when (IsTolerable(ex.StatusCode))
+        {
+            _logger.LogDebug(ex,
+                "PullSiteCalls tolerable transport fault for site {SiteId} ({Endpoint}): {Status}.",
+                siteId, endpoint, ex.StatusCode);
+            return (null, true);
+        }
+        catch (Exception ex) when (ex is HttpRequestException or System.Net.Sockets.SocketException)
+        {
+            _logger.LogDebug(ex,
+                "PullSiteCalls connection-layer fault for site {SiteId} ({Endpoint}).",
+                siteId, endpoint);
+            return (null, true);
+        }
+        catch (OperationCanceledException)
+        {
+            // Reconciliation tick cancelled — caller token (host shutdown) or an
+            // internal gRPC deadline / linked-CTS cancellation. Tolerable for a
+            // best-effort pull.
+            return (null, true);
+        }
+        catch (Exception ex)
+        {
+            // Any other fault. Reconciliation is best-effort; swallow to empty
+            // rather than throw — and do NOT fail over, since a second node would
+            // hit the same non-transport fault.
+            _logger.LogWarning(ex,
+                "PullSiteCalls unexpected fault for site {SiteId} ({Endpoint}). Returning empty batch.",
+                siteId, endpoint);
+            return (null, false);
+        }
+    }
+
+    private async Task<(string? Primary, string? Fallback)> ResolveEndpointsAsync(
+        string siteId, CancellationToken ct)
     {
         var sites = await _sites.EnumerateAsync(ct).ConfigureAwait(false);
         foreach (var site in sites)
@@ -176,10 +219,10 @@ public sealed class GrpcPullSiteCallsClient : IPullSiteCallsClient
             if (string.Equals(site.SiteId, siteId, StringComparison.Ordinal) &&
                 !string.IsNullOrWhiteSpace(site.GrpcEndpoint))
             {
-                return site.GrpcEndpoint;
+                return (site.GrpcEndpoint, site.FallbackGrpcEndpoint);
             }
         }
-        return null;
+        return (null, null);
     }
 
     private static readonly PullSiteCallsResponse Empty =

src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/IPullSiteCallsClient.cs

@@ -46,12 +46,22 @@ public interface IPullSiteCallsClient
     /// </summary>
     /// <param name="siteId">The identifier of the site to pull cached-call operational rows from.</param>
     /// <param name="sinceUtc">Only rows with an <c>UpdatedAtUtc</c> at or after this cursor time are returned.</param>
+    /// <param name="afterId">
+    /// The composite-keyset tiebreak cursor (Task 16). When non-null it is the
+    /// <c>TrackedOperationId</c> of the last row already consumed at
+    /// <paramref name="sinceUtc"/>; the site returns only rows strictly greater
+    /// than the composite <c>(UpdatedAtUtc, TrackedOperationId)</c> pair, so a
+    /// burst sharing one exact <c>UpdatedAtUtc</c> drains via the id tiebreak
+    /// instead of pinning the inclusive-timestamp cursor. Null on the first pull
+    /// (or against a legacy site) preserves the inclusive <c>&gt;=</c> contract.
+    /// </param>
     /// <param name="batchSize">Maximum number of rows to return per call.</param>
     /// <param name="ct">Cancellation token.</param>
     /// <returns>A task that resolves to the next reconciliation batch with a <c>MoreAvailable</c> flag.</returns>
     Task<PullSiteCallsResponse> PullAsync(
         string siteId,
         DateTime sinceUtc,
+        string? afterId,
         int batchSize,
         CancellationToken ct);
 }

src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/ISiteEnumerator.cs

@@ -8,13 +8,13 @@ namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central;
 /// </summary>
 /// <remarks>
 /// The production implementation wraps <c>ISiteRepository.GetAllSitesAsync</c>
-/// and projects each <c>Site</c> to a <see cref="SiteEntry"/> using the
-/// site's configured <c>GrpcNodeAAddress</c>. This is a NodeA-only first cut:
-/// sites with a blank <c>GrpcNodeAAddress</c> are silently SKIPPED — the
-/// reconciliation pull cannot reach them, but absence of an address is a
-/// configuration decision, not a runtime error. NodeB-fallback endpoint
-/// selection (dial NodeB when NodeA is unset/unreachable) is a follow-up
-/// (mirrors the comment in <c>SiteEnumerator.cs</c>).
+/// and projects each <c>Site</c> to a <see cref="SiteEntry"/>. The primary dial
+/// target is <c>GrpcNodeAAddress</c> when set, otherwise <c>GrpcNodeBAddress</c>;
+/// when both a distinct NodeA and NodeB address exist, NodeB rides along as
+/// <see cref="SiteEntry.FallbackGrpcEndpoint"/> so the reconciliation pull can
+/// fail over per call (Task 18). A site is skipped only when BOTH addresses are
+/// blank — the reconciliation pull cannot reach it, but absence of any address
+/// is a configuration decision, not a runtime error.
 /// </remarks>
 public interface ISiteEnumerator
 {
@@ -30,8 +30,19 @@ public interface ISiteEnumerator
 
 /// <summary>
 /// One reconciliation target: the site identifier the actor uses as the
-/// cursor key and the gRPC endpoint <see cref="IPullAuditEventsClient"/> dials
-/// to issue the pull. Endpoint is the bare authority (e.g. <c>http://siteA:8083</c>);
-/// transport selection (TLS, keepalive, etc.) is the client's concern.
+/// cursor key and the gRPC endpoint the pull clients dial to issue the pull.
+/// Endpoints are bare authorities (e.g. <c>http://siteA:8083</c>); transport
+/// selection (TLS, keepalive, etc.) is the client's concern.
 /// </summary>
-public sealed record SiteEntry(string SiteId, string GrpcEndpoint);
+/// <param name="SiteId">The site identifier used as the reconciliation cursor key.</param>
+/// <param name="GrpcEndpoint">The primary gRPC authority to dial (NodeA where configured, else NodeB).</param>
+/// <param name="FallbackGrpcEndpoint">
+/// Optional secondary gRPC authority (NodeB) dialed once when the primary raises
+/// a transport fault during a pull (Task 18). Null when no distinct second
+/// address is configured. Additive — absent value preserves the prior
+/// single-endpoint behaviour.
+/// </param>
+public sealed record SiteEntry(
+    string SiteId,
+    string GrpcEndpoint,
+    string? FallbackGrpcEndpoint = null);

src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/SiteAuditReconciliationActor.cs

@@ -1,9 +1,11 @@
+using System.Text.Json;
 using Akka.Actor;
 using Microsoft.Extensions.DependencyInjection;
 using Microsoft.Extensions.Logging;
 using Microsoft.Extensions.Options;
 using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
 using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
+using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
 
 namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central;
 
@@ -290,6 +292,13 @@ public class SiteAuditReconciliationActor : ReceiveActor
                         attempts);
                     _failedInsertAttempts.Remove(evt.EventId);
                     advanceForThisRow = true;
+
+                    // Task 17: the permanent loss must be queryable in the Audit Log
+                    // itself, not only in a rotating log file. Leave a durable
+                    // synthetic ReconciliationAbandoned row recording the lost id,
+                    // the site, and the final error.
+                    await EmitAbandonmentRecordAsync(repository, evt, site.SiteId, ex, nowUtc)
+                        .ConfigureAwait(false);
                 }
                 else
                 {
@@ -324,6 +333,68 @@ public class SiteAuditReconciliationActor : ReceiveActor
         UpdateStalledState(site.SiteId, draining: !nonDraining, eventStream);
     }
 
+    /// <summary>
+    /// Writes ONE synthetic <see cref="AuditKind.ReconciliationAbandoned"/> audit
+    /// row when a pulled event is permanently abandoned (Task 17) — so the loss is
+    /// queryable in the Audit Log rather than living only in the Critical log line.
+    /// The synthetic row is built through the same
+    /// <see cref="ScadaBridgeAuditEventFactory"/> the ingest path uses (fresh
+    /// <c>EventId</c>, <c>Actor="system"</c>, channel preserved from the lost row's
+    /// <c>Category</c> where parseable) and carries the abandoned <c>EventId</c>,
+    /// the source site, and the final error in <c>Extra</c>. It runs in its own
+    /// try/catch: the synthetic row is small and well-formed, so the row-specific
+    /// fault that killed the original will not recur; if it somehow does, log and
+    /// continue — the abandonment cursor must never be blocked a second time.
+    /// </summary>
+    private async Task EmitAbandonmentRecordAsync(
+        IAuditLogRepository repository,
+        ZB.MOM.WW.Audit.AuditEvent lost,
+        string siteId,
+        Exception error,
+        DateTime nowUtc)
+    {
+        try
+        {
+            // Category is the channel name (BuildCategory); preserve it where it
+            // parses, else ApiOutbound so the row still lands (mirrors the Task-14
+            // fallback convention).
+            var channel = Enum.TryParse<AuditChannel>(lost.Category, out var parsed)
+                ? parsed
+                : AuditChannel.ApiOutbound;
+
+            var extra = JsonSerializer.Serialize(new
+            {
+                abandonedEventId = lost.EventId,
+                sourceSiteId = siteId,
+                error = error.Message,
+            });
+
+            var synthetic = ScadaBridgeAuditEventFactory.Create(
+                channel: channel,
+                kind: AuditKind.ReconciliationAbandoned,
+                status: AuditStatus.Failed,
+                occurredAtUtc: nowUtc,
+                actor: "system",
+                sourceNode: lost.SourceNode,
+                sourceSiteId: siteId,
+                errorMessage: error.Message,
+                extra: extra,
+                ingestedAtUtc: nowUtc);
+
+            await repository.InsertIfNotExistsAsync(synthetic).ConfigureAwait(false);
+        }
+        catch (Exception ex)
+        {
+            _logger.LogError(
+                ex,
+                "Failed to write the synthetic ReconciliationAbandoned audit row for the "
+                + "permanently-abandoned AuditEvent {EventId} (site {SiteId}); the loss is recorded "
+                + "only in the Critical log line.",
+                lost.EventId,
+                siteId);
+        }
+    }
+
     /// <summary>
     /// Flips the per-site stalled flag based on whether this tick drained the
     /// queue. A "draining" cycle is one where the server reported no more rows

src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/SiteEnumerator.cs

@@ -21,17 +21,18 @@ namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central;
 /// per-tick scope pattern in <see cref="SiteAuditReconciliationActor.OnTickAsync"/>.
 /// </para>
 /// <para>
-/// <b>Blank-address skip.</b> Sites with no <c>GrpcNodeAAddress</c> configured
-/// are silently skipped: the reconciliation pull cannot dial them, but absence
-/// of an address is a configuration decision, not a runtime error (per the
+/// <b>Blank-address skip.</b> A site is skipped only when BOTH
+/// <c>GrpcNodeAAddress</c> and <c>GrpcNodeBAddress</c> are blank: the
+/// reconciliation pull cannot dial it, but absence of any address is a
+/// configuration decision, not a runtime error (per the
 /// <see cref="ISiteEnumerator"/> contract).
 /// </para>
 /// <para>
-/// <b>NodeA-only first cut.</b> This implementation always uses NodeA's gRPC
-/// address. NodeA/NodeB failover endpoint selection (dial NodeB when NodeA is
-/// unreachable) is a follow-up — the <see cref="SiteEntry"/> shape already
-/// carries a single endpoint, so failover will live in the puller/client, not
-/// here.
+/// <b>NodeA/NodeB failover (Task 18).</b> The primary dial target is NodeA when
+/// configured, otherwise NodeB (a site with only NodeB is no longer skipped).
+/// When a distinct NodeB address also exists it rides along as
+/// <see cref="SiteEntry.FallbackGrpcEndpoint"/> so the pull clients can fail over
+/// to it once per call when the primary raises a transport fault.
 /// </para>
 /// </remarks>
 public sealed class SiteEnumerator : ISiteEnumerator
@@ -60,16 +61,28 @@ public sealed class SiteEnumerator : ISiteEnumerator
         var entries = new List<SiteEntry>(sites.Count);
         foreach (var site in sites)
         {
-            // First cut: NodeA's gRPC address is the dial target. NodeA/NodeB
-            // failover endpoint selection is a follow-up.
-            if (string.IsNullOrWhiteSpace(site.GrpcNodeAAddress))
+            // Primary dial target: NodeA when configured, otherwise NodeB (Task 18) —
+            // a site with only a NodeB address is no longer skipped.
+            var nodeA = site.GrpcNodeAAddress;
+            var nodeB = site.GrpcNodeBAddress;
+            var primary = !string.IsNullOrWhiteSpace(nodeA) ? nodeA : nodeB;
+
+            if (string.IsNullOrWhiteSpace(primary))
             {
+                // Both addresses blank — nothing to dial (a configuration
+                // decision, not a runtime error).
                 continue;
             }
 
-            // The IsNullOrWhiteSpace guard above proves GrpcNodeAAddress is
-            // non-null here; explicit null-forgiving for clarity.
-            entries.Add(new SiteEntry(site.SiteIdentifier, site.GrpcNodeAAddress!));
+            // NodeB rides along as the failover endpoint only when it is a
+            // distinct, non-blank address (i.e. NodeA is the primary and NodeB
+            // differs); when NodeB IS the primary there is no separate fallback.
+            var fallback =
+                !string.IsNullOrWhiteSpace(nodeB) && !string.Equals(nodeB, primary, StringComparison.Ordinal)
+                    ? nodeB
+                    : null;
+
+            entries.Add(new SiteEntry(site.SiteIdentifier, primary!, fallback));
         }
 
         return entries;

src/ZB.MOM.WW.ScadaBridge.AuditLog/Kpi/AuditLogKpiSampleSource.cs

@@ -3,6 +3,12 @@ using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Kpi;
 using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
 using ZB.MOM.WW.ScadaBridge.Commons.Types.Kpi;
 
+// Task 10 (arch-review 04): the append-only repository leaves the snapshot's
+// BacklogTotal at zero; the real backlog is the cross-site health aggregate that
+// only the central node knows. An optional IAuditBacklogProvider supplies it so
+// the *history* sample stops recording a hardwired zero (the live tile already
+// filled this in). Sites/tests leave it unregistered → null → snapshot fallback.
+
 namespace ZB.MOM.WW.ScadaBridge.AuditLog.Kpi;
 
 /// <summary>
@@ -44,18 +50,30 @@ public sealed class AuditLogKpiSampleSource : IKpiSampleSource
     private const string BacklogTotalMetric = KpiMetrics.AuditLog.BacklogTotal;
 
     private readonly IAuditLogRepository _repository;
+    private readonly IAuditBacklogProvider? _backlogProvider;
 
     /// <summary>
     /// Initializes a new instance of the <see cref="AuditLogKpiSampleSource"/> class.
     /// </summary>
     /// <param name="repository">
     /// Append-only Audit Log repository — its <see cref="IAuditLogRepository.GetKpiSnapshotAsync"/>
-    /// supplies the volume, error, and backlog counts snapshotted here.
+    /// supplies the volume and error counts snapshotted here (its <c>BacklogTotal</c>
+    /// is always zero — see <paramref name="backlogProvider"/>).
     /// </param>
-    public AuditLogKpiSampleSource(IAuditLogRepository repository)
+    /// <param name="backlogProvider">
+    /// Optional cross-site backlog aggregate. When registered (central node), the
+    /// <c>backlogTotal</c> sample records its value; when absent (sites / unit tests,
+    /// where MS.DI resolves the default) the source falls back to the snapshot's own
+    /// <c>BacklogTotal</c> — i.e. zero, the pre-Task-10 behavior. Injecting the seam
+    /// keeps the append-only repo out of the in-memory health aggregator.
+    /// </param>
+    public AuditLogKpiSampleSource(
+        IAuditLogRepository repository,
+        IAuditBacklogProvider? backlogProvider = null)
     {
         ArgumentNullException.ThrowIfNull(repository);
         _repository = repository;
+        _backlogProvider = backlogProvider;
     }
 
     /// <inheritdoc/>
@@ -78,11 +96,15 @@ public sealed class AuditLogKpiSampleSource : IKpiSampleSource
             return [];
         }
 
+        // Backlog: the provider's cross-site aggregate when registered (central),
+        // otherwise the snapshot's own value (zero) as the site/test fallback.
+        var backlogTotal = _backlogProvider?.GetPendingBacklogTotal() ?? snapshot.BacklogTotal;
+
         return
         [
             Sample(TotalEventsLastHourMetric, snapshot.TotalEventsLastHour, capturedAtUtc),
             Sample(ErrorEventsLastHourMetric, snapshot.ErrorEventsLastHour, capturedAtUtc),
-            Sample(BacklogTotalMetric, snapshot.BacklogTotal, capturedAtUtc),
+            Sample(BacklogTotalMetric, backlogTotal, capturedAtUtc),
         ];
     }
 

src/ZB.MOM.WW.ScadaBridge.AuditLog/Site/SqliteAuditWriter.cs

@@ -55,6 +55,13 @@ public class SqliteAuditWriter : IAuditWriter, ISiteAuditQueue, IAsyncDisposable
     // transaction and never reached once audit_event's insert throws).
     private const int SqliteErrorConstraint = 19;
 
+    // Max EventIds bound into a single IN (...) UPDATE by MarkForwardedAsync /
+    // MarkReconciledAsync. SQLite caps bound parameters per statement (999 on older
+    // builds, SQLITE_MAX_VARIABLE_NUMBER); a large drain batch would otherwise blow the
+    // ceiling as batch sizes grow. Chunk the id list into ≤500-id slices, all inside one
+    // transaction so the batch stays atomic (arch-review 04, P8).
+    private const int MarkForwardStateChunkSize = 500;
+
     private readonly SqliteConnection _connection;
     // Dedicated read-only connection used by GetBacklogStatsAsync,
     // ReadPendingAsync, ReadPendingSinceAsync, and ReadForwardedAsync so a slow
@@ -625,13 +632,11 @@ public class SqliteAuditWriter : IAuditWriter, ISiteAuditQueue, IAsyncDisposable
         {
             ObjectDisposedException.ThrowIf(_disposed, this);
 
-            using var cmd = _connection.CreateCommand();
             // Flip the sidecar — UPDATE audit_forward_state, not the canonical
             // audit_event (which is append-only / write-once). Bump AttemptCount +
             // stamp LastAttemptUtc so operators can see how many drain passes a row
-            // took to forward. Build a single IN (...) parameter list so we issue
-            // one UPDATE per batch regardless of size. Each id is bound as its own
-            // parameter, so no string concatenation of user data ever enters the SQL.
+            // took to forward. Each id is bound as its own parameter, so no string
+            // concatenation of user data ever enters the SQL.
             //
             // Defensive state guard: only transition rows that are still Pending or
             // Forwarded (i.e. not yet Reconciled). Without this guard a mis-called
@@ -641,25 +646,41 @@ public class SqliteAuditWriter : IAuditWriter, ISiteAuditQueue, IAsyncDisposable
             //   WHERE ForwardState IN ($pending, $forwarded)
             // guard. Current callers only pass Pending IDs, so normal-path behaviour
             // is unchanged; the guard is purely defensive.
-            var sb = new System.Text.StringBuilder();
-            sb.Append("UPDATE audit_forward_state SET ForwardState = $forwarded, ")
-              .Append("AttemptCount = AttemptCount + 1, LastAttemptUtc = $now ")
-              .Append("WHERE ForwardState IN ($pending, $forwarded) AND EventId IN (");
-            for (int i = 0; i < eventIds.Count; i++)
-            {
-                if (i > 0) sb.Append(',');
-                var p = $"$id{i}";
-                sb.Append(p);
-                cmd.Parameters.AddWithValue(p, eventIds[i].ToString());
-            }
-            sb.Append(");");
-            cmd.CommandText = sb.ToString();
-            cmd.Parameters.AddWithValue("$forwarded", AuditForwardState.Forwarded.ToString());
-            cmd.Parameters.AddWithValue("$pending", AuditForwardState.Pending.ToString());
-            cmd.Parameters.AddWithValue("$now", DateTime.UtcNow.ToString(
-                "o", System.Globalization.CultureInfo.InvariantCulture));
+            //
+            // Stamp one $now for the whole call so every chunk records the same
+            // LastAttemptUtc (matching the prior single-statement behaviour).
+            var now = DateTime.UtcNow.ToString(
+                "o", System.Globalization.CultureInfo.InvariantCulture);
+            var forwarded = AuditForwardState.Forwarded.ToString();
+            var pending = AuditForwardState.Pending.ToString();
 
-            cmd.ExecuteNonQuery();
+            // Chunk into ≤MarkForwardStateChunkSize-id UPDATEs, all inside ONE
+            // transaction so the whole batch commits atomically (arch-review 04, P8).
+            using var transaction = _connection.BeginTransaction();
+            for (int start = 0; start < eventIds.Count; start += MarkForwardStateChunkSize)
+            {
+                var chunk = Math.Min(MarkForwardStateChunkSize, eventIds.Count - start);
+                using var cmd = _connection.CreateCommand();
+                cmd.Transaction = transaction;
+                var sb = new System.Text.StringBuilder();
+                sb.Append("UPDATE audit_forward_state SET ForwardState = $forwarded, ")
+                  .Append("AttemptCount = AttemptCount + 1, LastAttemptUtc = $now ")
+                  .Append("WHERE ForwardState IN ($pending, $forwarded) AND EventId IN (");
+                for (int i = 0; i < chunk; i++)
+                {
+                    if (i > 0) sb.Append(',');
+                    var p = $"$id{i}";
+                    sb.Append(p);
+                    cmd.Parameters.AddWithValue(p, eventIds[start + i].ToString());
+                }
+                sb.Append(");");
+                cmd.CommandText = sb.ToString();
+                cmd.Parameters.AddWithValue("$forwarded", forwarded);
+                cmd.Parameters.AddWithValue("$pending", pending);
+                cmd.Parameters.AddWithValue("$now", now);
+                cmd.ExecuteNonQuery();
+            }
+            transaction.Commit();
             return Task.CompletedTask;
         }
     }
@@ -728,27 +749,39 @@ public class SqliteAuditWriter : IAuditWriter, ISiteAuditQueue, IAsyncDisposable
         {
             ObjectDisposedException.ThrowIf(_disposed, this);
 
-            using var cmd = _connection.CreateCommand();
             // Flip the sidecar from Pending/Forwarded → Reconciled. Rows
             // already Reconciled are left untouched (idempotent re-call), and the
             // canonical audit_event row is never modified.
-            var sb = new System.Text.StringBuilder();
-            sb.Append("UPDATE audit_forward_state SET ForwardState = $reconciled ")
-              .Append("WHERE ForwardState IN ($pending, $forwarded) AND EventId IN (");
-            for (int i = 0; i < eventIds.Count; i++)
-            {
-                if (i > 0) sb.Append(',');
-                var p = $"$id{i}";
-                sb.Append(p);
-                cmd.Parameters.AddWithValue(p, eventIds[i].ToString());
-            }
-            sb.Append(");");
-            cmd.CommandText = sb.ToString();
-            cmd.Parameters.AddWithValue("$reconciled", AuditForwardState.Reconciled.ToString());
-            cmd.Parameters.AddWithValue("$pending", AuditForwardState.Pending.ToString());
-            cmd.Parameters.AddWithValue("$forwarded", AuditForwardState.Forwarded.ToString());
+            var reconciled = AuditForwardState.Reconciled.ToString();
+            var pending = AuditForwardState.Pending.ToString();
+            var forwarded = AuditForwardState.Forwarded.ToString();
 
-            cmd.ExecuteNonQuery();
+            // Chunk into ≤MarkForwardStateChunkSize-id UPDATEs, all inside ONE
+            // transaction so the whole batch commits atomically (arch-review 04, P8).
+            using var transaction = _connection.BeginTransaction();
+            for (int start = 0; start < eventIds.Count; start += MarkForwardStateChunkSize)
+            {
+                var chunk = Math.Min(MarkForwardStateChunkSize, eventIds.Count - start);
+                using var cmd = _connection.CreateCommand();
+                cmd.Transaction = transaction;
+                var sb = new System.Text.StringBuilder();
+                sb.Append("UPDATE audit_forward_state SET ForwardState = $reconciled ")
+                  .Append("WHERE ForwardState IN ($pending, $forwarded) AND EventId IN (");
+                for (int i = 0; i < chunk; i++)
+                {
+                    if (i > 0) sb.Append(',');
+                    var p = $"$id{i}";
+                    sb.Append(p);
+                    cmd.Parameters.AddWithValue(p, eventIds[start + i].ToString());
+                }
+                sb.Append(");");
+                cmd.CommandText = sb.ToString();
+                cmd.Parameters.AddWithValue("$reconciled", reconciled);
+                cmd.Parameters.AddWithValue("$pending", pending);
+                cmd.Parameters.AddWithValue("$forwarded", forwarded);
+                cmd.ExecuteNonQuery();
+            }
+            transaction.Commit();
             return Task.CompletedTask;
         }
     }

src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Pages/Notifications/NotificationReport.razor

@@ -10,6 +10,7 @@
 @inject SiteScopeService SiteScope
 @inject IDialogService Dialog
 @inject ILogger<NotificationReport> Logger
+@inject AuthenticationStateProvider AuthStateProvider
 
 <div class="container-fluid mt-3">
     <ToastNotification @ref="_toast" />
@@ -481,8 +482,9 @@
         _actionInProgress = true;
         try
         {
+            var requestedBy = await AuthStateProvider.GetCurrentUsernameAsync();
             var response = await CommunicationService.RetryNotificationAsync(
-                new RetryNotificationRequest(Guid.NewGuid().ToString("N"), n.NotificationId));
+                new RetryNotificationRequest(Guid.NewGuid().ToString("N"), n.NotificationId, requestedBy));
             if (response.Success)
             {
                 _toast.ShowSuccess($"Notification {ShortId(n.NotificationId)} re-queued for delivery.");
@@ -517,8 +519,9 @@
         _actionInProgress = true;
         try
         {
+            var requestedBy = await AuthStateProvider.GetCurrentUsernameAsync();
             var response = await CommunicationService.DiscardNotificationAsync(
-                new DiscardNotificationRequest(Guid.NewGuid().ToString("N"), n.NotificationId));
+                new DiscardNotificationRequest(Guid.NewGuid().ToString("N"), n.NotificationId, requestedBy));
             if (response.Success)
             {
                 _toast.ShowSuccess($"Notification {ShortId(n.NotificationId)} discarded.");

src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Pages/SiteCalls/SiteCallsReport.razor.cs

@@ -1,4 +1,5 @@
 using Microsoft.AspNetCore.Components;
+using Microsoft.AspNetCore.Components.Authorization;
 using Microsoft.AspNetCore.WebUtilities;
 using Microsoft.Extensions.Logging;
 using ZB.MOM.WW.ScadaBridge.CentralUI.Auth;
@@ -48,6 +49,7 @@ public partial class SiteCallsReport
 
     [Inject] private NavigationManager Navigation { get; set; } = null!;
     [Inject] private SiteScopeService SiteScope { get; set; } = null!;
+    [Inject] private AuthenticationStateProvider AuthStateProvider { get; set; } = null!;
 
     // The Status filter <select> options — the exact strings the dropdown binds and
     // the KPI tiles emit (e.g. ?status=Parked). A query-string status only seeds the
@@ -296,8 +298,9 @@ public partial class SiteCallsReport
         _actionInProgress = true;
         try
         {
+            var requestedBy = await AuthStateProvider.GetCurrentUsernameAsync();
             var response = await CommunicationService.RetrySiteCallAsync(
-                new RetrySiteCallRequest(Guid.NewGuid().ToString("N"), c.TrackedOperationId, c.SourceSite));
+                new RetrySiteCallRequest(Guid.NewGuid().ToString("N"), c.TrackedOperationId, c.SourceSite, requestedBy));
             ShowRelayOutcome(response.Outcome, response.SiteReachable, response.ErrorMessage,
                 appliedMessage: $"Retry of {ShortId(c.TrackedOperationId)} relayed to {SiteName(c.SourceSite)}.");
             if (response.Success)
@@ -330,8 +333,9 @@ public partial class SiteCallsReport
         _actionInProgress = true;
         try
         {
+            var requestedBy = await AuthStateProvider.GetCurrentUsernameAsync();
             var response = await CommunicationService.DiscardSiteCallAsync(
-                new DiscardSiteCallRequest(Guid.NewGuid().ToString("N"), c.TrackedOperationId, c.SourceSite));
+                new DiscardSiteCallRequest(Guid.NewGuid().ToString("N"), c.TrackedOperationId, c.SourceSite, requestedBy));
             ShowRelayOutcome(response.Outcome, response.SiteReachable, response.ErrorMessage,
                 appliedMessage: $"Discard of {ShortId(c.TrackedOperationId)} relayed to {SiteName(c.SourceSite)}.");
             if (response.Success)

src/ZB.MOM.WW.ScadaBridge.Commons/Interfaces/Kpi/IAuditBacklogProvider.cs

@@ -0,0 +1,33 @@
+namespace ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Kpi;
+
+/// <summary>
+/// Supplies the system-wide pending Audit Log backlog for the
+/// <c>backlogTotal</c> KPI history sample ("KPI History &amp; Trends").
+/// </summary>
+/// <remarks>
+/// <para>
+/// The append-only Audit Log repository leaves <c>AuditLogKpiSnapshot.BacklogTotal</c>
+/// at zero — the backlog is not a central-table count but the sum of the latest
+/// per-site <c>SiteAuditBacklog.PendingCount</c> health reports, which live only in
+/// the in-memory central health aggregator. The live Health-dashboard "Audit" tile
+/// already fills this in (<c>AuditLogQueryService.GetKpiSnapshotAsync</c>); this seam
+/// lets the KPI <em>history</em> sample source read the same aggregate instead of
+/// persisting a hardwired zero.
+/// </para>
+/// <para>
+/// Best-effort: sites that have not yet reported (or whose reporter is disabled)
+/// contribute zero — a missing snapshot is treated as "unknown, count as nothing",
+/// mirroring the live tile. Central-only: registered on the central node where the
+/// health aggregator lives; sites and unit tests leave it unregistered so the
+/// sample source falls back to the snapshot's own <c>BacklogTotal</c>.
+/// </para>
+/// </remarks>
+public interface IAuditBacklogProvider
+{
+    /// <summary>
+    /// Returns the current total pending (not-yet-forwarded) Audit Log backlog,
+    /// summed across every site's latest health report. Never negative; zero when
+    /// no site is reporting a backlog.
+    /// </summary>
+    long GetPendingBacklogTotal();
+}

src/ZB.MOM.WW.ScadaBridge.Commons/Interfaces/Repositories/IKpiHistoryRepository.cs

@@ -42,7 +42,9 @@ public interface IKpiHistoryRepository
 
     /// <summary>
     /// Bulk-deletes rows whose <see cref="KpiSample.CapturedAtUtc"/> is strictly older
-    /// than <paramref name="before"/> (retention purge).
+    /// than <paramref name="before"/> (retention purge). Implementations slice the
+    /// purge into time-windowed DELETE batches so a large catch-up (e.g. after an
+    /// outage) never runs as a single lock/log-heavy transaction (arch-review 04, P4).
     /// </summary>
     /// <param name="before">UTC cut-off; rows captured before this instant are deleted.</param>
     /// <param name="cancellationToken">Cancellation token.</param>

src/ZB.MOM.WW.ScadaBridge.Commons/Interfaces/Services/IOperationTrackingStore.cs

@@ -137,6 +137,19 @@ public interface IOperationTrackingStore
     /// the caller advance the cursor monotonically across follow-up pulls.
     /// </para>
     /// <para>
+    /// <b>Composite keyset (Task 15).</b> When <paramref name="afterId"/> is
+    /// supplied the read uses a <c>(UpdatedAtUtc, TrackedOperationId)</c> keyset:
+    /// it returns rows strictly after that cursor — <c>UpdatedAtUtc &gt; sinceUtc</c>,
+    /// or <c>UpdatedAtUtc = sinceUtc</c> with a <c>TrackedOperationId</c> greater
+    /// than <paramref name="afterId"/>. This un-pins a batch that would otherwise
+    /// stall: when more than <paramref name="batchSize"/> rows share one
+    /// <c>UpdatedAtUtc</c>, the plain inclusive <c>&gt;=</c> resume re-reads the
+    /// same page forever. When <paramref name="afterId"/> is <c>null</c>/empty the
+    /// legacy inclusive <c>&gt;=</c> contract is preserved exactly (an older
+    /// central that does not send the cursor is unaffected). Ordering is always
+    /// deterministic — <c>UpdatedAtUtc ASC, TrackedOperationId ASC</c>.
+    /// </para>
+    /// <para>
     /// <see cref="SiteCallOperational.SourceSite"/> is left as the empty string:
     /// the site id is not a tracking-store column, and the central client re-stamps
     /// it from the <c>siteId</c> it dialed (the only authority that knows which
@@ -148,10 +161,17 @@ public interface IOperationTrackingStore
     /// </remarks>
     /// <param name="sinceUtc">Inclusive lower bound on <c>UpdatedAtUtc</c>; <see cref="DateTime.MinValue"/> reads from the start.</param>
     /// <param name="batchSize">Maximum number of rows to return (oldest first).</param>
+    /// <param name="afterId">
+    /// Optional composite-keyset cursor — the <c>TrackedOperationId</c> of the last
+    /// row already consumed at <paramref name="sinceUtc"/>. When supplied the read
+    /// returns only rows strictly after <c>(sinceUtc, afterId)</c>; when
+    /// <c>null</c>/empty the legacy inclusive <c>&gt;= sinceUtc</c> behaviour applies.
+    /// </param>
     /// <param name="ct">Cancellation token.</param>
     /// <returns>The matching rows projected to <see cref="SiteCallOperational"/>, oldest-first, capped at <paramref name="batchSize"/>.</returns>
     Task<IReadOnlyList<SiteCallOperational>> ReadChangedSinceAsync(
         DateTime sinceUtc,
         int batchSize,
+        string? afterId = null,
         CancellationToken ct = default);
 }

src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Audit/SiteCallRelayMessages.cs

@@ -58,10 +58,16 @@ public enum SiteCallRelayOutcome
 /// <param name="SourceSite">
 /// The owning site (<c>SiteCall.SourceSite</c>) the relay is routed to.
 /// </param>
+/// <param name="RequestedBy">
+/// The authenticated operator who initiated the retry, stamped as the <c>Actor</c>
+/// on the central direct-write audit row the relay emits — recording *who asked*
+/// (Task 14). Additive; null where no identity is available.
+/// </param>
 public sealed record RetrySiteCallRequest(
     string CorrelationId,
     Guid TrackedOperationId,
-    string SourceSite);
+    string SourceSite,
+    string? RequestedBy = null);
 
 /// <summary>
 /// Site Call Audit → Central UI: result of a <see cref="RetrySiteCallRequest"/>.
@@ -99,7 +105,8 @@ public sealed record RetrySiteCallResponse(
 public sealed record DiscardSiteCallRequest(
     string CorrelationId,
     Guid TrackedOperationId,
-    string SourceSite);
+    string SourceSite,
+    string? RequestedBy = null);
 
 /// <summary>
 /// Site Call Audit → Central UI: result of a <see cref="DiscardSiteCallRequest"/>.

src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Notification/NotificationOutboxQueries.cs

@@ -51,9 +51,17 @@ public record NotificationOutboxQueryResponse(
 /// <summary>
 /// Outbox UI -> Central: request to immediately retry delivery of a notification.
 /// </summary>
+/// <param name="CorrelationId">Request/response correlation id.</param>
+/// <param name="NotificationId">The notification to re-queue.</param>
+/// <param name="RequestedBy">
+/// The authenticated operator who initiated the retry, stamped as the <c>Actor</c> on
+/// the emitted audit row so an un-park is attributable (Task 13). Additive; null where
+/// no identity is available (legacy senders compile unchanged).
+/// </param>
 public record RetryNotificationRequest(
     string CorrelationId,
-    string NotificationId);
+    string NotificationId,
+    string? RequestedBy = null);
 
 /// <summary>
 /// Central -> Outbox UI: result of a <see cref="RetryNotificationRequest"/>.
@@ -66,9 +74,16 @@ public record RetryNotificationResponse(
 /// <summary>
 /// Outbox UI -> Central: request to discard (cancel) a pending or stuck notification.
 /// </summary>
+/// <param name="CorrelationId">Request/response correlation id.</param>
+/// <param name="NotificationId">The notification to discard.</param>
+/// <param name="RequestedBy">
+/// The authenticated operator who initiated the discard, stamped as the <c>Actor</c> on
+/// the emitted terminal audit row (Task 13). Additive; null where no identity is available.
+/// </param>
 public record DiscardNotificationRequest(
     string CorrelationId,
-    string NotificationId);
+    string NotificationId,
+    string? RequestedBy = null);
 
 /// <summary>
 /// Central -> Outbox UI: result of a <see cref="DiscardNotificationRequest"/>.

src/ZB.MOM.WW.ScadaBridge.Commons/Types/Enums/AuditKind.cs

@@ -22,5 +22,15 @@ public enum AuditKind
     SecuredWriteSubmit,
     SecuredWriteApprove,
     SecuredWriteReject,
-    SecuredWriteExecute
+    SecuredWriteExecute,
+
+    /// <summary>
+    /// A reconciliation pull row that failed to insert on every retry up to the
+    /// permanent-abandon threshold, so central advanced its cursor past it and
+    /// the row is permanently lost from the mirror. The reconciliation actor
+    /// emits ONE synthetic audit row of this kind (carrying the abandoned
+    /// <c>EventId</c>, source site, and final error) so the loss is queryable in
+    /// the Audit Log itself, not only in a rotating log file. (arch-review 04, Task 17)
+    /// </summary>
+    ReconciliationAbandoned
 }

src/ZB.MOM.WW.ScadaBridge.Commons/Types/Kpi/KpiMetrics.cs

@@ -38,11 +38,17 @@ public static class KpiMetrics
         /// <summary>Pending-queue depth (rows awaiting delivery).</summary>
         public const string QueueDepth = "queueDepth";
 
+        /// <summary>Stuck-row count (non-terminal older than the stuck-age threshold).</summary>
+        public const string StuckCount = "stuckCount";
+
         /// <summary>Parked-row count.</summary>
         public const string ParkedCount = "parkedCount";
 
         /// <summary>Rows delivered in the last KPI interval.</summary>
         public const string DeliveredLastInterval = "deliveredLastInterval";
+
+        /// <summary>Age of the oldest pending row, in seconds.</summary>
+        public const string OldestPendingAgeSeconds = "oldestPendingAgeSeconds";
     }
 
     /// <summary>

src/ZB.MOM.WW.ScadaBridge.Commons/Types/PurgeTimerSchedule.cs

@@ -0,0 +1,33 @@
+namespace ZB.MOM.WW.ScadaBridge.Commons.Types;
+
+/// <summary>
+/// Shared initial-delay rule for the central retention-purge singletons
+/// (Audit Log partition purge, Site Call Audit terminal purge, KPI History
+/// sample purge). All three run on a long (up to 24 h) interval; scheduling the
+/// FIRST tick after a full interval meant a node that recycles more often than
+/// the interval — e.g. a daily container restart against a 24 h cadence — would
+/// never actually purge (arch-review 04, S9).
+/// </summary>
+/// <remarks>
+/// Purges are idempotent, so an early first sweep is always safe. Taking
+/// <c>min(interval, <see cref="ShortFirstTick"/>)</c> guarantees the first purge
+/// runs within five minutes of startup while leaving the steady-state cadence —
+/// and the millisecond intervals the actor test suites use — unchanged.
+/// </remarks>
+public static class PurgeTimerSchedule
+{
+    /// <summary>
+    /// Upper bound on the delay before the first purge tick. Short enough that a
+    /// frequently-recycled node still purges, long enough not to add startup churn.
+    /// </summary>
+    public static readonly TimeSpan ShortFirstTick = TimeSpan.FromMinutes(5);
+
+    /// <summary>
+    /// The initial delay a purge timer should use for its first tick given its
+    /// steady-state <paramref name="interval"/>: the interval itself when it is
+    /// already shorter than <see cref="ShortFirstTick"/> (preserving fast test
+    /// cadences), otherwise capped at <see cref="ShortFirstTick"/>.
+    /// </summary>
+    public static TimeSpan InitialDelay(TimeSpan interval) =>
+        interval < ShortFirstTick ? interval : ShortFirstTick;
+}

src/ZB.MOM.WW.ScadaBridge.Communication/Grpc/SiteStreamGrpcServer.cs

@@ -561,19 +561,24 @@ public class SiteStreamGrpcServer : SiteStreamService.SiteStreamServiceBase
             ? DateTime.SpecifyKind(request.SinceUtc.ToDateTime(), DateTimeKind.Utc)
             : DateTime.MinValue;
 
+        // Composite-keyset cursor (Task 15): proto3 defaults an unset string to
+        // "", which the store treats as "no cursor" (legacy inclusive >= behaviour)
+        // — so an older central that never sets after_id is unaffected.
+        var afterId = string.IsNullOrEmpty(request.AfterId) ? null : request.AfterId;
+
         IReadOnlyList<SiteCallOperational> operationals;
         try
         {
             operationals = await store.ReadChangedSinceAsync(
-                since, request.BatchSize, context.CancellationToken);
+                since, request.BatchSize, afterId, context.CancellationToken);
         }
         catch (Exception ex)
         {
             // Best-effort, like PullAuditEvents: a read fault must never abort
             // the reconciliation tick — central retries on its next cycle.
             _logger.LogError(ex,
-                "ReadChangedSinceAsync failed for since={Since} batch={Batch}; returning empty response.",
-                since, request.BatchSize);
+                "ReadChangedSinceAsync failed for since={Since} batch={Batch} afterId={AfterId}; returning empty response.",
+                since, request.BatchSize, afterId);
             return new PullSiteCallsResponse();
         }
 

src/ZB.MOM.WW.ScadaBridge.Communication/Protos/sitestream.proto

@@ -171,6 +171,14 @@ message PullAuditEventsResponse {
 message PullSiteCallsRequest {
   google.protobuf.Timestamp since_utc = 1;
   int32 batch_size = 2;
+  // Composite-keyset cursor (Task 15): the TrackedOperationId ("D" GUID form) of
+  // the last row already consumed at since_utc. When set, the site returns only
+  // rows strictly after (since_utc, after_id) under a deterministic
+  // (UpdatedAtUtc, TrackedOperationId) order — un-pinning a batch that would
+  // otherwise stall when more than batch_size rows share one since_utc instant.
+  // Empty (the proto3 string default) preserves the legacy inclusive >= behaviour,
+  // so an older central that never sets it is unaffected. Additive-only.
+  string after_id = 3;
 }
 
 message PullSiteCallsResponse {

src/ZB.MOM.WW.ScadaBridge.Communication/SiteStreamGrpc/Sitestream.cs

@@ -83,29 +83,30 @@ namespace ZB.MOM.WW.ScadaBridge.Communication.Grpc {
             "GAEgASgLMhouZ29vZ2xlLnByb3RvYnVmLlRpbWVzdGFtcBISCgpiYXRjaF9z",
             "aXplGAIgASgFIlwKF1B1bGxBdWRpdEV2ZW50c1Jlc3BvbnNlEikKBmV2ZW50",
             "cxgBIAMoCzIZLnNpdGVzdHJlYW0uQXVkaXRFdmVudER0bxIWCg5tb3JlX2F2",
-            "YWlsYWJsZRgCIAEoCCJZChRQdWxsU2l0ZUNhbGxzUmVxdWVzdBItCglzaW5j",
+            "YWlsYWJsZRgCIAEoCCJrChRQdWxsU2l0ZUNhbGxzUmVxdWVzdBItCglzaW5j",
             "ZV91dGMYASABKAsyGi5nb29nbGUucHJvdG9idWYuVGltZXN0YW1wEhIKCmJh",
-            "dGNoX3NpemUYAiABKAUiaQoVUHVsbFNpdGVDYWxsc1Jlc3BvbnNlEjgKDG9w",
-            "ZXJhdGlvbmFscxgBIAMoCzIiLnNpdGVzdHJlYW0uU2l0ZUNhbGxPcGVyYXRp",
-            "b25hbER0bxIWCg5tb3JlX2F2YWlsYWJsZRgCIAEoCCpcCgdRdWFsaXR5EhcK",
-            "E1FVQUxJVFlfVU5TUEVDSUZJRUQQABIQCgxRVUFMSVRZX0dPT0QQARIVChFR",
-            "VUFMSVRZX1VOQ0VSVEFJThACEg8KC1FVQUxJVFlfQkFEEAMqXQoOQWxhcm1T",
-            "dGF0ZUVudW0SGwoXQUxBUk1fU1RBVEVfVU5TUEVDSUZJRUQQABIWChJBTEFS",
-            "TV9TVEFURV9OT1JNQUwQARIWChJBTEFSTV9TVEFURV9BQ1RJVkUQAiqFAQoO",
-            "QWxhcm1MZXZlbEVudW0SFAoQQUxBUk1fTEVWRUxfTk9ORRAAEhMKD0FMQVJN",
-            "X0xFVkVMX0xPVxABEhcKE0FMQVJNX0xFVkVMX0xPV19MT1cQAhIUChBBTEFS",
-            "TV9MRVZFTF9ISUdIEAMSGQoVQUxBUk1fTEVWRUxfSElHSF9ISUdIEAQytwMK",
-            "EVNpdGVTdHJlYW1TZXJ2aWNlElUKEVN1YnNjcmliZUluc3RhbmNlEiEuc2l0",
-            "ZXN0cmVhbS5JbnN0YW5jZVN0cmVhbVJlcXVlc3QaGy5zaXRlc3RyZWFtLlNp",
-            "dGVTdHJlYW1FdmVudDABEkcKEUluZ2VzdEF1ZGl0RXZlbnRzEhsuc2l0ZXN0",
-            "cmVhbS5BdWRpdEV2ZW50QmF0Y2gaFS5zaXRlc3RyZWFtLkluZ2VzdEFjaxJQ",
-            "ChVJbmdlc3RDYWNoZWRUZWxlbWV0cnkSIC5zaXRlc3RyZWFtLkNhY2hlZFRl",
-            "bGVtZXRyeUJhdGNoGhUuc2l0ZXN0cmVhbS5Jbmdlc3RBY2sSWgoPUHVsbEF1",
-            "ZGl0RXZlbnRzEiIuc2l0ZXN0cmVhbS5QdWxsQXVkaXRFdmVudHNSZXF1ZXN0",
-            "GiMuc2l0ZXN0cmVhbS5QdWxsQXVkaXRFdmVudHNSZXNwb25zZRJUCg1QdWxs",
-            "U2l0ZUNhbGxzEiAuc2l0ZXN0cmVhbS5QdWxsU2l0ZUNhbGxzUmVxdWVzdBoh",
-            "LnNpdGVzdHJlYW0uUHVsbFNpdGVDYWxsc1Jlc3BvbnNlQiuqAihaQi5NT00u",
-            "V1cuU2NhZGFCcmlkZ2UuQ29tbXVuaWNhdGlvbi5HcnBjYgZwcm90bzM="));
+            "dGNoX3NpemUYAiABKAUSEAoIYWZ0ZXJfaWQYAyABKAkiaQoVUHVsbFNpdGVD",
+            "YWxsc1Jlc3BvbnNlEjgKDG9wZXJhdGlvbmFscxgBIAMoCzIiLnNpdGVzdHJl",
+            "YW0uU2l0ZUNhbGxPcGVyYXRpb25hbER0bxIWCg5tb3JlX2F2YWlsYWJsZRgC",
+            "IAEoCCpcCgdRdWFsaXR5EhcKE1FVQUxJVFlfVU5TUEVDSUZJRUQQABIQCgxR",
+            "VUFMSVRZX0dPT0QQARIVChFRVUFMSVRZX1VOQ0VSVEFJThACEg8KC1FVQUxJ",
+            "VFlfQkFEEAMqXQoOQWxhcm1TdGF0ZUVudW0SGwoXQUxBUk1fU1RBVEVfVU5T",
+            "UEVDSUZJRUQQABIWChJBTEFSTV9TVEFURV9OT1JNQUwQARIWChJBTEFSTV9T",
+            "VEFURV9BQ1RJVkUQAiqFAQoOQWxhcm1MZXZlbEVudW0SFAoQQUxBUk1fTEVW",
+            "RUxfTk9ORRAAEhMKD0FMQVJNX0xFVkVMX0xPVxABEhcKE0FMQVJNX0xFVkVM",
+            "X0xPV19MT1cQAhIUChBBTEFSTV9MRVZFTF9ISUdIEAMSGQoVQUxBUk1fTEVW",
+            "RUxfSElHSF9ISUdIEAQytwMKEVNpdGVTdHJlYW1TZXJ2aWNlElUKEVN1YnNj",
+            "cmliZUluc3RhbmNlEiEuc2l0ZXN0cmVhbS5JbnN0YW5jZVN0cmVhbVJlcXVl",
+            "c3QaGy5zaXRlc3RyZWFtLlNpdGVTdHJlYW1FdmVudDABEkcKEUluZ2VzdEF1",
+            "ZGl0RXZlbnRzEhsuc2l0ZXN0cmVhbS5BdWRpdEV2ZW50QmF0Y2gaFS5zaXRl",
+            "c3RyZWFtLkluZ2VzdEFjaxJQChVJbmdlc3RDYWNoZWRUZWxlbWV0cnkSIC5z",
+            "aXRlc3RyZWFtLkNhY2hlZFRlbGVtZXRyeUJhdGNoGhUuc2l0ZXN0cmVhbS5J",
+            "bmdlc3RBY2sSWgoPUHVsbEF1ZGl0RXZlbnRzEiIuc2l0ZXN0cmVhbS5QdWxs",
+            "QXVkaXRFdmVudHNSZXF1ZXN0GiMuc2l0ZXN0cmVhbS5QdWxsQXVkaXRFdmVu",
+            "dHNSZXNwb25zZRJUCg1QdWxsU2l0ZUNhbGxzEiAuc2l0ZXN0cmVhbS5QdWxs",
+            "U2l0ZUNhbGxzUmVxdWVzdBohLnNpdGVzdHJlYW0uUHVsbFNpdGVDYWxsc1Jl",
+            "c3BvbnNlQiuqAihaQi5NT00uV1cuU2NhZGFCcmlkZ2UuQ29tbXVuaWNhdGlv",
+            "bi5HcnBjYgZwcm90bzM="));
       descriptor = pbr::FileDescriptor.FromGeneratedCode(descriptorData,
           new pbr::FileDescriptor[] { global::Google.Protobuf.WellKnownTypes.TimestampReflection.Descriptor, global::Google.Protobuf.WellKnownTypes.WrappersReflection.Descriptor, },
           new pbr::GeneratedClrTypeInfo(new[] {typeof(global::ZB.MOM.WW.ScadaBridge.Communication.Grpc.Quality), typeof(global::ZB.MOM.WW.ScadaBridge.Communication.Grpc.AlarmStateEnum), typeof(global::ZB.MOM.WW.ScadaBridge.Communication.Grpc.AlarmLevelEnum), }, null, new pbr::GeneratedClrTypeInfo[] {
@@ -121,7 +122,7 @@ namespace ZB.MOM.WW.ScadaBridge.Communication.Grpc {
             new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.ScadaBridge.Communication.Grpc.CachedTelemetryBatch), global::ZB.MOM.WW.ScadaBridge.Communication.Grpc.CachedTelemetryBatch.Parser, new[]{ "Packets" }, null, null, null, null),
             new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.ScadaBridge.Communication.Grpc.PullAuditEventsRequest), global::ZB.MOM.WW.ScadaBridge.Communication.Grpc.PullAuditEventsRequest.Parser, new[]{ "SinceUtc", "BatchSize" }, null, null, null, null),
             new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.ScadaBridge.Communication.Grpc.PullAuditEventsResponse), global::ZB.MOM.WW.ScadaBridge.Communication.Grpc.PullAuditEventsResponse.Parser, new[]{ "Events", "MoreAvailable" }, null, null, null, null),
-            new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.ScadaBridge.Communication.Grpc.PullSiteCallsRequest), global::ZB.MOM.WW.ScadaBridge.Communication.Grpc.PullSiteCallsRequest.Parser, new[]{ "SinceUtc", "BatchSize" }, null, null, null, null),
+            new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.ScadaBridge.Communication.Grpc.PullSiteCallsRequest), global::ZB.MOM.WW.ScadaBridge.Communication.Grpc.PullSiteCallsRequest.Parser, new[]{ "SinceUtc", "BatchSize", "AfterId" }, null, null, null, null),
             new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.ScadaBridge.Communication.Grpc.PullSiteCallsResponse), global::ZB.MOM.WW.ScadaBridge.Communication.Grpc.PullSiteCallsResponse.Parser, new[]{ "Operationals", "MoreAvailable" }, null, null, null, null)
           }));
     }
@@ -5200,6 +5201,7 @@ namespace ZB.MOM.WW.ScadaBridge.Communication.Grpc {
     public PullSiteCallsRequest(PullSiteCallsRequest other) : this() {
       sinceUtc_ = other.sinceUtc_ != null ? other.sinceUtc_.Clone() : null;
       batchSize_ = other.batchSize_;
+      afterId_ = other.afterId_;
       _unknownFields = pb::UnknownFieldSet.Clone(other._unknownFields);
     }
 
@@ -5233,6 +5235,27 @@ namespace ZB.MOM.WW.ScadaBridge.Communication.Grpc {
       }
     }
 
+    /// <summary>Field number for the "after_id" field.</summary>
+    public const int AfterIdFieldNumber = 3;
+    private string afterId_ = "";
+    /// <summary>
+    /// Composite-keyset cursor (Task 15): the TrackedOperationId ("D" GUID form) of
+    /// the last row already consumed at since_utc. When set, the site returns only
+    /// rows strictly after (since_utc, after_id) under a deterministic
+    /// (UpdatedAtUtc, TrackedOperationId) order — un-pinning a batch that would
+    /// otherwise stall when more than batch_size rows share one since_utc instant.
+    /// Empty (the proto3 string default) preserves the legacy inclusive >= behaviour,
+    /// so an older central that never sets it is unaffected. Additive-only.
+    /// </summary>
+    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
+    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
+    public string AfterId {
+      get { return afterId_; }
+      set {
+        afterId_ = pb::ProtoPreconditions.CheckNotNull(value, "value");
+      }
+    }
+
     [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
     [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
     public override bool Equals(object other) {
@@ -5250,6 +5273,7 @@ namespace ZB.MOM.WW.ScadaBridge.Communication.Grpc {
       }
       if (!object.Equals(SinceUtc, other.SinceUtc)) return false;
       if (BatchSize != other.BatchSize) return false;
+      if (AfterId != other.AfterId) return false;
       return Equals(_unknownFields, other._unknownFields);
     }
 
@@ -5259,6 +5283,7 @@ namespace ZB.MOM.WW.ScadaBridge.Communication.Grpc {
       int hash = 1;
       if (sinceUtc_ != null) hash ^= SinceUtc.GetHashCode();
       if (BatchSize != 0) hash ^= BatchSize.GetHashCode();
+      if (AfterId.Length != 0) hash ^= AfterId.GetHashCode();
       if (_unknownFields != null) {
         hash ^= _unknownFields.GetHashCode();
       }
@@ -5285,6 +5310,10 @@ namespace ZB.MOM.WW.ScadaBridge.Communication.Grpc {
         output.WriteRawTag(16);
         output.WriteInt32(BatchSize);
       }
+      if (AfterId.Length != 0) {
+        output.WriteRawTag(26);
+        output.WriteString(AfterId);
+      }
       if (_unknownFields != null) {
         _unknownFields.WriteTo(output);
       }
@@ -5303,6 +5332,10 @@ namespace ZB.MOM.WW.ScadaBridge.Communication.Grpc {
         output.WriteRawTag(16);
         output.WriteInt32(BatchSize);
       }
+      if (AfterId.Length != 0) {
+        output.WriteRawTag(26);
+        output.WriteString(AfterId);
+      }
       if (_unknownFields != null) {
         _unknownFields.WriteTo(ref output);
       }
@@ -5319,6 +5352,9 @@ namespace ZB.MOM.WW.ScadaBridge.Communication.Grpc {
       if (BatchSize != 0) {
         size += 1 + pb::CodedOutputStream.ComputeInt32Size(BatchSize);
       }
+      if (AfterId.Length != 0) {
+        size += 1 + pb::CodedOutputStream.ComputeStringSize(AfterId);
+      }
       if (_unknownFields != null) {
         size += _unknownFields.CalculateSize();
       }
@@ -5340,6 +5376,9 @@ namespace ZB.MOM.WW.ScadaBridge.Communication.Grpc {
       if (other.BatchSize != 0) {
         BatchSize = other.BatchSize;
       }
+      if (other.AfterId.Length != 0) {
+        AfterId = other.AfterId;
+      }
       _unknownFields = pb::UnknownFieldSet.MergeFrom(_unknownFields, other._unknownFields);
     }
 
@@ -5370,6 +5409,10 @@ namespace ZB.MOM.WW.ScadaBridge.Communication.Grpc {
             BatchSize = input.ReadInt32();
             break;
           }
+          case 26: {
+            AfterId = input.ReadString();
+            break;
+          }
         }
       }
     #endif
@@ -5400,6 +5443,10 @@ namespace ZB.MOM.WW.ScadaBridge.Communication.Grpc {
             BatchSize = input.ReadInt32();
             break;
           }
+          case 26: {
+            AfterId = input.ReadString();
+            break;
+          }
         }
       }
     }

src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Configurations/AuditLogEntityTypeConfiguration.cs

@@ -155,6 +155,12 @@ public class AuditLogEntityTypeConfiguration : IEntityTypeConfiguration<AuditLog
         // rejected at CREATE. It is not indexed, so non-persistence costs nothing.
         // Routed through the nullable UTC converter so the materialised value
         // carries Kind=Utc.
+        //
+        // NEVER use IngestedAtUtc in a WHERE predicate — it is non-persisted
+        // (stored:false), so any filter on it forces a full-table JSON_VALUE parse of
+        // DetailsJson on every row (no index can back it). Read-side projection only
+        // (arch-review 04, P7). Filter on OccurredAtUtc (the partition-aligned clustered
+        // key) instead.
         builder.Property(e => e.IngestedAtUtc)
             .HasColumnType("datetime2(7)")
             .HasConversion(NullableUtcConverter)

src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/AuditLogRepository.cs

@@ -229,6 +229,13 @@ VALUES
     /// <inheritdoc />
     public async Task<long> SwitchOutPartitionAsync(DateTime monthBoundary, TimeSpan? commandTimeout = null, CancellationToken ct = default)
     {
+        // Task 12 (arch-review 04 S5) note: the drop-and-rebuild batch below runs via
+        // ExecuteSqlRaw with NO EF user-transaction — it carries its own server-side
+        // BEGIN TRANSACTION / TRY-CATCH / ROLLBACK — so the DbContext's retrying
+        // execution strategy (EnableRetryOnFailure) MAY auto-replay the whole batch on
+        // a transient fault. That replay is safe: every step is IF-EXISTS / IF-NOT-EXISTS
+        // guarded and the staging table is GUID-suffixed, so a re-run is idempotent.
+        //
         // Maintenance timeout in whole seconds (ADO.NET CommandTimeout unit). Null leaves the
         // provider default in place. See AuditLogPurgeOptions.MaintenanceCommandTimeoutMinutes /
         // arch-review 04 S2 for why the ~30s default is unsafe for the switch-out dance.

src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/KpiHistoryRepository.cs

@@ -64,9 +64,22 @@ public sealed class KpiHistoryRepository : IKpiHistoryRepository
     /// <inheritdoc />
     public async Task<int> PurgeOlderThanAsync(DateTime before, CancellationToken cancellationToken = default)
     {
-        // Set-based delete — no entity materialisation; returns the rows affected.
-        return await _context.KpiSamples
-            .Where(s => s.CapturedAtUtc < before)
-            .ExecuteDeleteAsync(cancellationToken);
+        // Time-sliced batches: each DELETE covers at most one hour of samples, capping the
+        // lock/log footprint per statement (arch-review 04, P4 — steady state is ~1 day of
+        // rows/day; after an outage the catch-up would otherwise be one giant transaction).
+        var total = 0;
+        var floor = await _context.KpiSamples.Where(s => s.CapturedAtUtc < before)
+            .MinAsync(s => (DateTime?)s.CapturedAtUtc, cancellationToken);
+        while (floor is not null && floor < before)
+        {
+            var ceiling = floor.Value.AddHours(1) < before ? floor.Value.AddHours(1) : before;
+            total += await _context.KpiSamples
+                .Where(s => s.CapturedAtUtc < ceiling)
+                .ExecuteDeleteAsync(cancellationToken);
+            floor = await _context.KpiSamples.Where(s => s.CapturedAtUtc < before)
+                .MinAsync(s => (DateTime?)s.CapturedAtUtc, cancellationToken);
+        }
+
+        return total;
     }
 }

src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/NotificationOutboxRepository.cs

@@ -241,45 +241,40 @@ VALUES
     {
         var now = DateTimeOffset.UtcNow;
 
-        var queueDepth = await _context.Notifications
-            .CountAsync(n => n.Status == NotificationStatus.Pending
-                || n.Status == NotificationStatus.Retrying, cancellationToken);
+        // One conditional-aggregation pass replaces four sequential COUNT round trips:
+        // each metric is a COUNT(CASE WHEN <predicate> THEN 1 END) over the same scan
+        // (arch-review 04). GroupBy(_ => 1) yields a single group (no rows → no group).
+        var counts = await _context.Notifications
+            .GroupBy(_ => 1)
+            .Select(g => new
+            {
+                QueueDepth = g.Count(n => n.Status == NotificationStatus.Pending
+                    || n.Status == NotificationStatus.Retrying),
+                StuckCount = g.Count(n => (n.Status == NotificationStatus.Pending
+                        || n.Status == NotificationStatus.Retrying)
+                    && n.CreatedAt < stuckCutoff),
+                ParkedCount = g.Count(n => n.Status == NotificationStatus.Parked),
+                DeliveredLastInterval = g.Count(n => n.Status == NotificationStatus.Delivered
+                    && n.DeliveredAt != null && n.DeliveredAt >= deliveredSince),
+            })
+            .FirstOrDefaultAsync(cancellationToken);
 
-        var stuckCount = await _context.Notifications
-            .CountAsync(n => (n.Status == NotificationStatus.Pending
-                    || n.Status == NotificationStatus.Retrying)
-                && n.CreatedAt < stuckCutoff, cancellationToken);
-
-        var parkedCount = await _context.Notifications
-            .CountAsync(n => n.Status == NotificationStatus.Parked, cancellationToken);
-
-        var deliveredLastInterval = await _context.Notifications
-            .CountAsync(n => n.Status == NotificationStatus.Delivered
-                && n.DeliveredAt != null
-                && n.DeliveredAt >= deliveredSince, cancellationToken);
-
-        // Oldest non-terminal CreatedAt. The DateTimeOffset value converter makes a SQL
-        // Min aggregate awkward, so order ascending and take the first instead.
-        var nonTerminal = _context.Notifications
+        // Oldest non-terminal CreatedAt — TOP(1) via ordered FirstOrDefault, so the
+        // live queue is never materialized. The nullable cast lets FirstOrDefault
+        // return null (no non-terminal rows) instead of throwing.
+        var oldestCreatedAt = await _context.Notifications
             .Where(n => n.Status == NotificationStatus.Pending
-                || n.Status == NotificationStatus.Retrying);
-
-        TimeSpan? oldestPendingAge = null;
-        if (await nonTerminal.AnyAsync(cancellationToken))
-        {
-            var oldestCreatedAt = await nonTerminal
-                .OrderBy(n => n.CreatedAt)
-                .Select(n => n.CreatedAt)
-                .FirstAsync(cancellationToken);
-            oldestPendingAge = now - oldestCreatedAt;
-        }
+                || n.Status == NotificationStatus.Retrying)
+            .OrderBy(n => n.CreatedAt)
+            .Select(n => (DateTimeOffset?)n.CreatedAt)
+            .FirstOrDefaultAsync(cancellationToken);
 
         return new NotificationKpiSnapshot(
-            QueueDepth: queueDepth,
-            StuckCount: stuckCount,
-            ParkedCount: parkedCount,
-            DeliveredLastInterval: deliveredLastInterval,
-            OldestPendingAge: oldestPendingAge);
+            QueueDepth: counts?.QueueDepth ?? 0,
+            StuckCount: counts?.StuckCount ?? 0,
+            ParkedCount: counts?.ParkedCount ?? 0,
+            DeliveredLastInterval: counts?.DeliveredLastInterval ?? 0,
+            OldestPendingAge: oldestCreatedAt is null ? null : now - oldestCreatedAt.Value);
     }
 
     /// <inheritdoc />
@@ -288,49 +283,47 @@ VALUES
     {
         var now = DateTimeOffset.UtcNow;
 
-        var queueDepth = await CountBySiteAsync(
-            n => n.Status == NotificationStatus.Pending || n.Status == NotificationStatus.Retrying,
-            cancellationToken);
+        // One grouped conditional-aggregation query per scope: counts AND the oldest
+        // non-terminal CreatedAt (server-side MIN over a CASE, so the live queue is
+        // never materialized) computed in a single scan. Pre-filter to rows that
+        // contribute to at least one metric so a site with only fully-uncounted rows
+        // (e.g. Discarded, or out-of-window Delivered) does not surface as an
+        // all-zero row — preserving the prior union-of-metric-keys semantics.
+        var rows = await _context.Notifications
+            .Where(n => n.Status == NotificationStatus.Pending
+                || n.Status == NotificationStatus.Retrying
+                || n.Status == NotificationStatus.Parked
+                || (n.Status == NotificationStatus.Delivered
+                    && n.DeliveredAt != null && n.DeliveredAt >= deliveredSince))
+            .GroupBy(n => n.SourceSiteId)
+            .Select(g => new
+            {
+                Site = g.Key,
+                QueueDepth = g.Count(n => n.Status == NotificationStatus.Pending
+                    || n.Status == NotificationStatus.Retrying),
+                StuckCount = g.Count(n => (n.Status == NotificationStatus.Pending
+                        || n.Status == NotificationStatus.Retrying)
+                    && n.CreatedAt < stuckCutoff),
+                ParkedCount = g.Count(n => n.Status == NotificationStatus.Parked),
+                DeliveredLastInterval = g.Count(n => n.Status == NotificationStatus.Delivered
+                    && n.DeliveredAt != null && n.DeliveredAt >= deliveredSince),
+                OldestCreatedAt = g.Min(n => (n.Status == NotificationStatus.Pending
+                        || n.Status == NotificationStatus.Retrying)
+                    ? (DateTimeOffset?)n.CreatedAt
+                    : null),
+            })
+            .ToListAsync(cancellationToken);
 
-        var stuck = await CountBySiteAsync(
-            n => (n.Status == NotificationStatus.Pending || n.Status == NotificationStatus.Retrying)
-                && n.CreatedAt < stuckCutoff,
-            cancellationToken);
-
-        var parked = await CountBySiteAsync(
-            n => n.Status == NotificationStatus.Parked, cancellationToken);
-
-        var delivered = await CountBySiteAsync(
-            n => n.Status == NotificationStatus.Delivered
-                && n.DeliveredAt != null && n.DeliveredAt >= deliveredSince,
-            cancellationToken);
-
-        // Oldest non-terminal CreatedAt per site. A SQL Min over the DateTimeOffset
-        // converter is awkward (see ComputeKpisAsync), so project the non-terminal
-        // (site, created) pairs — the live queue, which stays bounded — and reduce
-        // in memory.
-        var oldest = (await _context.Notifications
-                .Where(n => n.Status == NotificationStatus.Pending
-                    || n.Status == NotificationStatus.Retrying)
-                .Select(n => new { n.SourceSiteId, n.CreatedAt })
-                .ToListAsync(cancellationToken))
-            .GroupBy(x => x.SourceSiteId)
-            .ToDictionary(g => g.Key, g => g.Min(x => x.CreatedAt));
-
-        var siteIds = queueDepth.Keys
-            .Concat(stuck.Keys).Concat(parked.Keys).Concat(delivered.Keys)
-            .Distinct()
-            .OrderBy(s => s, StringComparer.Ordinal);
-
-        return siteIds.Select(site => new SiteNotificationKpiSnapshot(
-            SourceSiteId: site,
-            QueueDepth: queueDepth.GetValueOrDefault(site),
-            StuckCount: stuck.GetValueOrDefault(site),
-            ParkedCount: parked.GetValueOrDefault(site),
-            DeliveredLastInterval: delivered.GetValueOrDefault(site),
-            OldestPendingAge: oldest.TryGetValue(site, out var createdAt)
-                ? now - createdAt
-                : null)).ToList();
+        return rows
+            .OrderBy(r => r.Site, StringComparer.Ordinal)
+            .Select(r => new SiteNotificationKpiSnapshot(
+                SourceSiteId: r.Site,
+                QueueDepth: r.QueueDepth,
+                StuckCount: r.StuckCount,
+                ParkedCount: r.ParkedCount,
+                DeliveredLastInterval: r.DeliveredLastInterval,
+                OldestPendingAge: r.OldestCreatedAt is null ? null : now - r.OldestCreatedAt.Value))
+            .ToList();
     }
 
     /// <inheritdoc />
@@ -339,83 +332,45 @@ VALUES
     {
         var now = DateTimeOffset.UtcNow;
 
-        // Exclude rows with NULL SourceNode (legacy / unstamped) — per-node KPIs
-        // are only meaningful when the node identity is known.
-        var queueDepth = await CountByNodeAsync(
-            n => (n.Status == NotificationStatus.Pending || n.Status == NotificationStatus.Retrying)
-                && n.SourceNode != null,
-            cancellationToken);
-
-        var stuck = await CountByNodeAsync(
-            n => (n.Status == NotificationStatus.Pending || n.Status == NotificationStatus.Retrying)
-                && n.CreatedAt < stuckCutoff
-                && n.SourceNode != null,
-            cancellationToken);
-
-        var parked = await CountByNodeAsync(
-            n => n.Status == NotificationStatus.Parked && n.SourceNode != null,
-            cancellationToken);
-
-        var delivered = await CountByNodeAsync(
-            n => n.Status == NotificationStatus.Delivered
-                && n.DeliveredAt != null && n.DeliveredAt >= deliveredSince
-                && n.SourceNode != null,
-            cancellationToken);
-
-        // Oldest non-terminal CreatedAt per node — same in-memory reduction
-        // pattern as ComputePerSiteKpisAsync (DateTimeOffset converter makes
-        // a SQL Min awkward).
-        var oldest = (await _context.Notifications
-                .Where(n => (n.Status == NotificationStatus.Pending
-                    || n.Status == NotificationStatus.Retrying)
-                    && n.SourceNode != null)
-                .Select(n => new { n.SourceNode, n.CreatedAt })
-                .ToListAsync(cancellationToken))
-            .GroupBy(x => x.SourceNode!)
-            .ToDictionary(g => g.Key, g => g.Min(x => x.CreatedAt));
-
-        var nodeNames = queueDepth.Keys
-            .Concat(stuck.Keys).Concat(parked.Keys).Concat(delivered.Keys)
-            .Distinct()
-            .OrderBy(n => n, StringComparer.Ordinal);
-
-        return nodeNames.Select(node => new NodeNotificationKpiSnapshot(
-            SourceNode: node,
-            QueueDepth: queueDepth.GetValueOrDefault(node),
-            StuckCount: stuck.GetValueOrDefault(node),
-            ParkedCount: parked.GetValueOrDefault(node),
-            DeliveredLastInterval: delivered.GetValueOrDefault(node),
-            OldestPendingAge: oldest.TryGetValue(node, out var createdAt)
-                ? now - createdAt
-                : null)).ToList();
-    }
-
-    /// <summary>Counts notification rows matching <paramref name="predicate"/>, grouped by source site.</summary>
-    private async Task<Dictionary<string, int>> CountBySiteAsync(
-        System.Linq.Expressions.Expression<Func<Notification, bool>> predicate,
-        CancellationToken cancellationToken)
-    {
-        return await _context.Notifications
-            .Where(predicate)
-            .GroupBy(n => n.SourceSiteId)
-            .Select(g => new { Site = g.Key, Count = g.Count() })
-            .ToDictionaryAsync(x => x.Site, x => x.Count, cancellationToken);
-    }
-
-    /// <summary>
-    /// Counts notification rows matching <paramref name="predicate"/>, grouped by source node.
-    /// Only rows with a non-null <c>SourceNode</c> should be included; the predicate is
-    /// responsible for enforcing that guard.
-    /// </summary>
-    private async Task<Dictionary<string, int>> CountByNodeAsync(
-        System.Linq.Expressions.Expression<Func<Notification, bool>> predicate,
-        CancellationToken cancellationToken)
-    {
-        return await _context.Notifications
-            .Where(predicate)
+        // Same single-scan grouped aggregation as ComputePerSiteKpisAsync, keyed by
+        // SourceNode. Rows with a NULL SourceNode (legacy / unstamped) are excluded —
+        // per-node KPIs are only meaningful when node identity is known.
+        var rows = await _context.Notifications
+            .Where(n => n.SourceNode != null
+                && (n.Status == NotificationStatus.Pending
+                    || n.Status == NotificationStatus.Retrying
+                    || n.Status == NotificationStatus.Parked
+                    || (n.Status == NotificationStatus.Delivered
+                        && n.DeliveredAt != null && n.DeliveredAt >= deliveredSince)))
             .GroupBy(n => n.SourceNode!)
-            .Select(g => new { Node = g.Key, Count = g.Count() })
-            .ToDictionaryAsync(x => x.Node, x => x.Count, cancellationToken);
+            .Select(g => new
+            {
+                Node = g.Key,
+                QueueDepth = g.Count(n => n.Status == NotificationStatus.Pending
+                    || n.Status == NotificationStatus.Retrying),
+                StuckCount = g.Count(n => (n.Status == NotificationStatus.Pending
+                        || n.Status == NotificationStatus.Retrying)
+                    && n.CreatedAt < stuckCutoff),
+                ParkedCount = g.Count(n => n.Status == NotificationStatus.Parked),
+                DeliveredLastInterval = g.Count(n => n.Status == NotificationStatus.Delivered
+                    && n.DeliveredAt != null && n.DeliveredAt >= deliveredSince),
+                OldestCreatedAt = g.Min(n => (n.Status == NotificationStatus.Pending
+                        || n.Status == NotificationStatus.Retrying)
+                    ? (DateTimeOffset?)n.CreatedAt
+                    : null),
+            })
+            .ToListAsync(cancellationToken);
+
+        return rows
+            .OrderBy(r => r.Node, StringComparer.Ordinal)
+            .Select(r => new NodeNotificationKpiSnapshot(
+                SourceNode: r.Node,
+                QueueDepth: r.QueueDepth,
+                StuckCount: r.StuckCount,
+                ParkedCount: r.ParkedCount,
+                DeliveredLastInterval: r.DeliveredLastInterval,
+                OldestPendingAge: r.OldestCreatedAt is null ? null : now - r.OldestCreatedAt.Value))
+            .ToList();
     }
 
     /// <inheritdoc />

src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/SiteCallAuditRepository.cs

@@ -24,9 +24,17 @@ public class SiteCallAuditRepository : ISiteCallAuditRepository
     private const int SqlErrorUniqueIndexViolation = 2601;
     private const int SqlErrorPrimaryKeyViolation = 2627;
 
-    // Monotonic status ordering. Lower rank wins on tie (same-rank upserts are
-    // no-ops, including terminal-over-terminal):
+    // Monotonic status ordering:
     // Submitted < Forwarded < Attempted == Skipped < Delivered == Failed == Parked == Discarded.
+    // A higher incoming rank always wins. WITHIN an equal NON-terminal rank
+    // (Attempted/Skipped, rank 2 — and the transient Submitted/Forwarded ranks),
+    // the newest UpdatedAtUtc wins so a retrying call's live RetryCount/LastError
+    // no longer freezes at first-write (Task 11). Equal terminal ranks (rank 3)
+    // stay immutable — the freshness tiebreaker is deliberately scoped to
+    // rank < 3, so a later terminal NEVER flips an earlier one (Delivered cannot
+    // overwrite Parked). Still idempotent (equal stamps are inert) and still
+    // regression-proof (a lower rank is always a no-op).
+    private const int TerminalRank = 3;
     private static readonly Dictionary<string, int> StatusRank = new(StringComparer.Ordinal)
     {
         ["Submitted"] = 0,
@@ -95,10 +103,16 @@ VALUES
                 idText);
         }
 
-        // Step 2: monotonic update. The CASE expression maps the stored Status
-        // string to the same rank table the caller uses; we only mutate if the
-        // incoming rank is strictly greater. Same-rank (including
-        // terminal-over-terminal) is a no-op — first-write-wins at each rank.
+        // Step 2: monotonic update with a same-rank freshness tiebreaker. The
+        // CASE expression maps the stored Status string to the same rank table
+        // the caller uses. We mutate when EITHER the incoming rank is strictly
+        // greater, OR the incoming rank equals the stored rank AND that rank is
+        // non-terminal (< TerminalRank) AND the incoming UpdatedAtUtc is strictly
+        // newer than the stored one — so a retrying call's Attempted-phase
+        // RetryCount/LastError/HttpStatus stay live instead of freezing at the
+        // first Attempted packet (Task 11). Terminal ranks are excluded from the
+        // tiebreaker, so a later terminal NEVER overwrites an earlier one; equal
+        // stamps are inert (idempotent replay) and a lower rank is always a no-op.
         //
         // SourceNode-stamping: SourceNode is updated via
         // COALESCE(@SourceNode, SourceNode). The operator returns @SourceNode
@@ -125,7 +139,7 @@ SET Status = {siteCall.Status},
     IngestedAtUtc = {siteCall.IngestedAtUtc},
     SourceNode = COALESCE({siteCall.SourceNode}, SourceNode)
 WHERE TrackedOperationId = {idText}
-  AND {incomingRank} > (CASE Status
+  AND ( {incomingRank} > (CASE Status
       WHEN 'Submitted' THEN 0
       WHEN 'Forwarded' THEN 1
       WHEN 'Attempted' THEN 2
@@ -135,7 +149,20 @@ WHERE TrackedOperationId = {idText}
       WHEN 'Parked'    THEN 3
       WHEN 'Discarded' THEN 3
       ELSE -1
-  END);",
+  END)
+     OR ( {incomingRank} = (CASE Status
+      WHEN 'Submitted' THEN 0
+      WHEN 'Forwarded' THEN 1
+      WHEN 'Attempted' THEN 2
+      WHEN 'Skipped'   THEN 2
+      WHEN 'Delivered' THEN 3
+      WHEN 'Failed'    THEN 3
+      WHEN 'Parked'    THEN 3
+      WHEN 'Discarded' THEN 3
+      ELSE -1
+  END)
+          AND {incomingRank} < {TerminalRank}
+          AND UpdatedAtUtc < {siteCall.UpdatedAtUtc} ) );",
             ct);
     }
 
@@ -199,7 +226,14 @@ WHERE ({filter.Channel} IS NULL OR Channel = {filter.Channel})
   AND ({(hasCursor ? 1 : 0)} = 0
        OR CreatedAtUtc < {afterCreated}
        OR (CreatedAtUtc = {afterCreated} AND TrackedOperationId < {afterIdString}))
-ORDER BY CreatedAtUtc DESC, TrackedOperationId DESC;";
+ORDER BY CreatedAtUtc DESC, TrackedOperationId DESC
+-- Every filter above is the (@p IS NULL OR col = @p) optional-parameter shape, so a
+-- single cached plan would be parameter-sniffed for whichever filters happened to be
+-- non-null on first compile. RECOMPILE lets the optimizer prune the dead (@p IS NULL)
+-- predicates per invocation and pick IX_SiteCalls_Status_Updated / IX_SiteCalls_NonTerminal
+-- for the filters actually supplied (arch-review 04, P5). Per-invocation compile cost is
+-- negligible at UI-page cadence.
+OPTION (RECOMPILE);";
 
         var rows = await _context.Set<SiteCall>()
             .FromSqlInterpolated(sql)

src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/ServiceCollectionExtensions.cs

@@ -28,7 +28,22 @@ public static class ServiceCollectionExtensions
         // CreateProtector during model building.
         services.AddDbContext<ScadaBridgeDbContext>((serviceProvider, options) =>
         {
-            options.UseSqlServer(connectionString)
+            options.UseSqlServer(
+                    connectionString,
+                    // Task 12 (arch-review 04 S5): connection resiliency. A transient
+                    // SQL fault (failover, throttling, dropped connection) previously
+                    // surfaced raw to every read-side caller — KPI asks, outbox
+                    // queries, Tracking.Status, execution-tree walks. EnableRetryOnFailure
+                    // installs the SqlServerRetryingExecutionStrategy so those retry
+                    // transparently. Note: manual (user-initiated) transactions are NOT
+                    // auto-retried — EF executes them directly while a transaction is
+                    // active — so the combined-telemetry dual-write is wrapped in an
+                    // explicit CreateExecutionStrategy() to become retriable (see
+                    // AuditLogIngestActor.OnCachedTelemetryAsync).
+                    sql => sql.EnableRetryOnFailure(
+                        maxRetryCount: 5,
+                        maxRetryDelay: TimeSpan.FromSeconds(30),
+                        errorNumbersToAdd: null))
                 .ConfigureWarnings(w => w.Ignore(
                     Microsoft.EntityFrameworkCore.Diagnostics.RelationalEventId.PendingModelChangesWarning));
         });

src/ZB.MOM.WW.ScadaBridge.HealthMonitoring/Kpi/CentralHealthAuditBacklogProvider.cs

@@ -0,0 +1,52 @@
+using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Kpi;
+
+namespace ZB.MOM.WW.ScadaBridge.HealthMonitoring.Kpi;
+
+/// <summary>
+/// Central <see cref="IAuditBacklogProvider"/> — sums the latest per-site
+/// <c>SiteAuditBacklog.PendingCount</c> across the in-memory
+/// <see cref="ICentralHealthAggregator"/>, the same aggregate the live
+/// Health-dashboard "Audit" backlog tile shows
+/// (<c>AuditLogQueryService.GetKpiSnapshotAsync</c>).
+/// </summary>
+/// <remarks>
+/// Registered only on the central node (where the aggregator lives), so the
+/// <see cref="ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Kpi.IKpiSampleSource"/>
+/// backing the <c>backlogTotal</c> trend records the real backlog instead of the
+/// append-only repository's hardwired zero. Best-effort: sites with no report or
+/// a null <c>SiteAuditBacklog</c> contribute nothing.
+/// </remarks>
+public sealed class CentralHealthAuditBacklogProvider : IAuditBacklogProvider
+{
+    private readonly ICentralHealthAggregator _aggregator;
+
+    /// <summary>
+    /// Initializes a new instance of the <see cref="CentralHealthAuditBacklogProvider"/> class.
+    /// </summary>
+    /// <param name="aggregator">The central health aggregator whose per-site latest reports carry the backlog.</param>
+    public CentralHealthAuditBacklogProvider(ICentralHealthAggregator aggregator)
+    {
+        ArgumentNullException.ThrowIfNull(aggregator);
+        _aggregator = aggregator;
+    }
+
+    /// <inheritdoc />
+    public long GetPendingBacklogTotal()
+    {
+        // Ported verbatim from AuditLogQueryService.GetKpiSnapshotAsync: a null
+        // SiteAuditBacklog is "unknown" (contributes zero), and the `> 0` guard
+        // ignores any defensive negative/zero value so the tile and the history
+        // agree exactly.
+        long backlog = 0;
+        foreach (var state in _aggregator.GetAllSiteStates().Values)
+        {
+            var pending = state.LatestReport?.SiteAuditBacklog?.PendingCount;
+            if (pending is > 0)
+            {
+                backlog += pending.Value;
+            }
+        }
+
+        return backlog;
+    }
+}

src/ZB.MOM.WW.ScadaBridge.HealthMonitoring/ServiceCollectionExtensions.cs

@@ -63,6 +63,15 @@ public static class ServiceCollectionExtensions
         services.AddHostedService(sp => sp.GetRequiredService<CentralHealthAggregator>());
         services.AddHostedService<CentralHealthReportLoop>();
 
+        // Task 10 (arch-review 04): the central backlog aggregate for the Audit Log
+        // KPI *history*. Only the central node runs the aggregator, so this seam is
+        // registered here; the AuditLogKpiSampleSource resolves it (optional ctor
+        // param) and records the real cross-site backlog instead of the append-only
+        // repository's hardwired zero. Sites leave it unregistered → snapshot fallback.
+        services.AddSingleton<
+            Commons.Interfaces.Kpi.IAuditBacklogProvider,
+            Kpi.CentralHealthAuditBacklogProvider>();
+
         // "KPI History & Trends": per-site Site Health KPI sample source.
         // Reads the in-memory central aggregator (a singleton) rather than a
         // repository; registered Scoped to match the recorder's per-tick scope

src/ZB.MOM.WW.ScadaBridge.KpiHistory/KpiHistoryRecorderActor.cs

@@ -4,6 +4,7 @@ using Microsoft.Extensions.Logging;
 using ZB.MOM.WW.ScadaBridge.Commons.Entities.Kpi;
 using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Kpi;
 using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
+using ZB.MOM.WW.ScadaBridge.Commons.Types;
 
 namespace ZB.MOM.WW.ScadaBridge.KpiHistory;
 
@@ -119,12 +120,15 @@ public class KpiHistoryRecorderActor : ReceiveActor, IWithTimers
             initialDelay: TimeSpan.FromSeconds(5),
             interval: _options.SampleInterval);
 
-        // The purge is daily and idempotent — no initial fast tick; the first sweep
-        // fires after a full PurgeInterval.
+        // The purge is idempotent, so it takes a short first tick (min(PurgeInterval, 5min)):
+        // scheduling the first sweep a full PurgeInterval out meant a node recycled more often
+        // than the interval (e.g. daily restart vs. daily cadence) would never purge
+        // (arch-review 04, S9). Fast test cadences (interval < 5min) are unchanged.
         Timers.StartPeriodicTimer(
             PurgeTimerKey,
             PurgeTick.Instance,
-            _options.PurgeInterval);
+            initialDelay: PurgeTimerSchedule.InitialDelay(_options.PurgeInterval),
+            interval: _options.PurgeInterval);
     }
 
     /// <inheritdoc />

src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/Kpi/NotificationOutboxKpiSampleSource.cs

@@ -37,10 +37,10 @@ public sealed class NotificationOutboxKpiSampleSource : IKpiSampleSource
     // Charted metrics share the public Commons catalog so source + UI trend page key
     // off one symbol. The uncharted internal metrics stay private here.
     private const string MetricQueueDepth = KpiMetrics.NotificationOutbox.QueueDepth;
-    private const string MetricStuckCount = "stuckCount";
+    private const string MetricStuckCount = KpiMetrics.NotificationOutbox.StuckCount;
     private const string MetricParkedCount = KpiMetrics.NotificationOutbox.ParkedCount;
     private const string MetricDeliveredLastInterval = KpiMetrics.NotificationOutbox.DeliveredLastInterval;
-    private const string MetricOldestPendingAgeSeconds = "oldestPendingAgeSeconds";
+    private const string MetricOldestPendingAgeSeconds = KpiMetrics.NotificationOutbox.OldestPendingAgeSeconds;
 
     private readonly INotificationOutboxRepository _repository;
     private readonly NotificationOutboxOptions _options;

src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/NotificationOutboxActor.cs

@@ -318,33 +318,26 @@ public class NotificationOutboxActor : ReceiveActor, IWithTimers
 
             var (maxRetries, retryDelay) = await ResolveRetryPolicyAsync(notificationRepository);
 
+            // Deliver the claimed batch with bounded parallelism. Delivery is I/O-bound
+            // (multi-second SMTP/Twilio round trips), so a strictly sequential sweep caps
+            // throughput at ~1/attempt-latency — an alarm-storm backlog behind one slow
+            // host would drain far too slowly (arch-review 04). A SemaphoreSlim caps the
+            // number of in-flight deliveries; each delivery runs on its OWN DI scope/
+            // repository (the EF DbContext is not thread-safe and rows are distinct per
+            // notification, so concurrent deliveries never contend). The cached adapters
+            // and the lifetime-scoped audit writer are already safe for concurrent use.
+            // MaxParallelDeliveries = 1 preserves the strictly sequential behaviour.
+            using var deliveryGate = new SemaphoreSlim(
+                _options.ResolvedMaxParallelDeliveries, _options.ResolvedMaxParallelDeliveries);
+            var deliveries = new List<Task>(due.Count);
             foreach (var notification in due)
             {
-                // Between deliveries, observe shutdown so we don't kick off a fresh
-                // SMTP send when the actor is already tearing down.
-                if (cancellationToken.IsCancellationRequested)
-                {
-                    return;
-                }
-
-                try
-                {
-                    await DeliverOneAsync(
-                        notification, now, maxRetries, retryDelay, outboxRepository, adapters, cancellationToken);
-                }
-                catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
-                {
-                    // In-flight delivery interrupted by shutdown. Row remains in its
-                    // pre-attempt state; next active sweep retries.
-                    return;
-                }
-                catch (Exception ex)
-                {
-                    // Isolate per-notification failures so the remainder of the batch still runs.
-                    _logger.LogError(
-                        ex, "Dispatch failed for notification {NotificationId}.", notification.NotificationId);
-                }
+                deliveries.Add(DeliverGatedAsync(
+                    notification, now, maxRetries, retryDelay, adapters, deliveryGate, cancellationToken));
             }
+
+            // Each DeliverGatedAsync isolates its own faults, so WhenAll never throws.
+            await Task.WhenAll(deliveries);
         }
         catch (Exception ex)
         {
@@ -354,6 +347,67 @@ public class NotificationOutboxActor : ReceiveActor, IWithTimers
         }
     }
 
+    /// <summary>
+    /// Delivers one notification under the sweep's concurrency gate. Acquires a permit,
+    /// runs <see cref="DeliverOneAsync"/> on a fresh per-notification DI scope (so parallel
+    /// deliveries never share an EF <c>DbContext</c>), and always releases the permit.
+    /// Faults are isolated here — shutdown cancellation is swallowed (the row stays in its
+    /// pre-attempt state for the next active sweep) and any other exception is logged — so
+    /// one bad delivery never aborts the rest of the batch and <c>Task.WhenAll</c> never faults.
+    /// </summary>
+    private async Task DeliverGatedAsync(
+        Notification notification,
+        DateTimeOffset now,
+        int maxRetries,
+        TimeSpan retryDelay,
+        IReadOnlyDictionary<NotificationType, INotificationDeliveryAdapter> adapters,
+        SemaphoreSlim gate,
+        CancellationToken cancellationToken)
+    {
+        try
+        {
+            await gate.WaitAsync(cancellationToken).ConfigureAwait(false);
+        }
+        catch (OperationCanceledException)
+        {
+            // Shutdown cancelled before this delivery acquired a permit; the row stays
+            // Pending and the next active node's sweep re-claims it. No permit to release.
+            return;
+        }
+
+        try
+        {
+            if (cancellationToken.IsCancellationRequested)
+            {
+                return;
+            }
+
+            // Each delivery owns its DI scope + repository so parallel UpdateAsync calls
+            // never share a DbContext. The adapters map + audit writer are already safe
+            // for concurrent use.
+            using var scope = _serviceProvider.CreateScope();
+            var outboxRepository = scope.ServiceProvider.GetRequiredService<INotificationOutboxRepository>();
+            await DeliverOneAsync(
+                    notification, now, maxRetries, retryDelay, outboxRepository, adapters, cancellationToken)
+                .ConfigureAwait(false);
+        }
+        catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
+        {
+            // In-flight delivery interrupted by shutdown. Row remains in its pre-attempt
+            // state; next active sweep retries.
+        }
+        catch (Exception ex)
+        {
+            // Isolate per-notification failures so the remainder of the batch still runs.
+            _logger.LogError(
+                ex, "Dispatch failed for notification {NotificationId}.", notification.NotificationId);
+        }
+        finally
+        {
+            gate.Release();
+        }
+    }
+
     /// <summary>
     /// Resolves the retry policy from the first SMTP configuration row. When no SMTP
     /// configuration exists, falls back to a conservative default — delivery itself will
@@ -587,12 +641,14 @@ public class NotificationOutboxActor : ReceiveActor, IWithTimers
     private async Task EmitTerminalAuditAsync(
         Notification notification,
         DateTimeOffset now,
-        string? errorMessage)
+        string? errorMessage,
+        string? actorOverride = null)
     {
         try
         {
             var terminalStatus = MapNotificationStatusToAuditStatus(notification.Status);
-            var evt = BuildNotifyDeliverEvent(notification, now, terminalStatus, errorMessage);
+            var evt = BuildNotifyDeliverEvent(
+                notification, now, terminalStatus, errorMessage, actorOverride: actorOverride);
             await _auditWriter.WriteAsync(evt);
         }
         catch (Exception ex)
@@ -677,7 +733,8 @@ public class NotificationOutboxActor : ReceiveActor, IWithTimers
         DateTimeOffset now,
         AuditStatus status,
         string? errorMessage,
-        int? durationMs = null)
+        int? durationMs = null,
+        string? actorOverride = null)
     {
         Guid? correlationId = Guid.TryParse(notification.NotificationId, out var parsed)
             ? parsed
@@ -688,11 +745,12 @@ public class NotificationOutboxActor : ReceiveActor, IWithTimers
             kind: AuditKind.NotifyDeliver,
             status: status,
             occurredAtUtc: now.UtcDateTime,
-            // Central dispatch — a system identity per the Actor-column spec;
-            // there is no per-call authenticated user here. The originating
-            // script is still captured on SourceScript (and on the upstream
-            // NotifySend row).
-            actor: SystemActor,
+            // Central dispatch is a system identity per the Actor-column spec —
+            // there is no per-call authenticated user for automatic delivery. An
+            // operator-initiated transition (Retry/Discard) passes actorOverride
+            // so the row is attributable to the person who un-parked/cancelled it
+            // (Task 13). The originating script is still captured on SourceScript.
+            actor: actorOverride ?? SystemActor,
             target: notification.ListName,
             correlationId: correlationId,
             // ExecutionId: the originating script execution's id,
@@ -973,9 +1031,41 @@ public class NotificationOutboxActor : ReceiveActor, IWithTimers
         notification.LastError = null;
         await repository.UpdateAsync(notification);
 
+        // Operator re-queued a parked notification. Emit a Submitted NotifyDeliver
+        // row attributing the un-park to the operator — otherwise the lifecycle
+        // reads Parked → Attempted → Delivered with no record of who un-parked it
+        // (Task 13). Best-effort: audit failure never aborts the retry.
+        await EmitRetrySubmittedAuditAsync(notification, DateTimeOffset.UtcNow, request.RequestedBy);
+
         return new RetryNotificationResponse(request.CorrelationId, Success: true, ErrorMessage: null);
     }
 
+    /// <summary>
+    /// Emits a single <see cref="AuditKind.NotifyDeliver"/> row with
+    /// <see cref="AuditStatus.Submitted"/> recording that <paramref name="actorOverride"/>
+    /// re-queued a parked notification. Best-effort with the same
+    /// audit-never-affects-the-action shape as <see cref="EmitTerminalAuditAsync"/>.
+    /// </summary>
+    private async Task EmitRetrySubmittedAuditAsync(
+        Notification notification,
+        DateTimeOffset now,
+        string? actorOverride)
+    {
+        try
+        {
+            var evt = BuildNotifyDeliverEvent(
+                notification, now, AuditStatus.Submitted, errorMessage: null, actorOverride: actorOverride);
+            await _auditWriter.WriteAsync(evt);
+        }
+        catch (Exception ex)
+        {
+            _logger.LogWarning(
+                ex,
+                "Failed to emit Submitted (operator retry) audit row for notification {NotificationId}.",
+                notification.NotificationId);
+        }
+    }
+
     /// <summary>
     /// Handles a manual discard request. Only a <c>Parked</c> notification can be discarded;
     /// it is moved to the terminal <c>Discarded</c> status.
@@ -1018,7 +1108,10 @@ public class NotificationOutboxActor : ReceiveActor, IWithTimers
         // Emit a Discarded NotifyDeliver row to match the dispatcher's
         // Delivered/Parked emissions; the row carries no error message because
         // the discard is an operator-driven cancellation, not a delivery error.
-        await EmitTerminalAuditAsync(notification, DateTimeOffset.UtcNow, errorMessage: null);
+        // The operator identity is stamped as Actor so the cancellation is
+        // attributable (Task 13).
+        await EmitTerminalAuditAsync(
+            notification, DateTimeOffset.UtcNow, errorMessage: null, actorOverride: request.RequestedBy);
 
         return new DiscardNotificationResponse(request.CorrelationId, Success: true, ErrorMessage: null);
     }

src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/NotificationOutboxOptions.cs

@@ -48,4 +48,23 @@ public class NotificationOutboxOptions
     /// Default: 1 minute.
     /// </summary>
     public TimeSpan DeliveredKpiWindow { get; set; } = TimeSpan.FromMinutes(1);
+
+    /// <summary>
+    /// Maximum number of notifications delivered concurrently within a single dispatch
+    /// sweep. Delivery is I/O-bound (SMTP / Twilio round trips of seconds each), so a
+    /// purely sequential sweep caps throughput at ~1/attempt-latency regardless of batch
+    /// size — an alarm storm queued behind one slow SMTP host would drain far too slowly
+    /// (arch-review 04). Each concurrent delivery runs on its own DI scope/repository, so
+    /// there is no shared-DbContext contention. Set to <c>1</c> to preserve the strictly
+    /// sequential behaviour. Clamped to a minimum of 1 via <see cref="ResolvedMaxParallelDeliveries"/>.
+    /// Default: 4.
+    /// </summary>
+    public int MaxParallelDeliveries { get; set; } = 4;
+
+    /// <summary>
+    /// Resolves the effective delivery parallelism, clamped to at least 1 so a
+    /// misconfigured <c>0</c>/negative value cannot stall the dispatcher (a zero-count
+    /// semaphore would deadlock the sweep).
+    /// </summary>
+    public int ResolvedMaxParallelDeliveries => MaxParallelDeliveries < 1 ? 1 : MaxParallelDeliveries;
 }

src/ZB.MOM.WW.ScadaBridge.SiteCallAudit/SiteCallAuditActor.cs

@@ -4,10 +4,12 @@ using Microsoft.Extensions.Logging;
 using ZB.MOM.WW.ScadaBridge.AuditLog.Central;
 using ZB.MOM.WW.ScadaBridge.Commons.Entities.Audit;
 using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
+using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Services;
 using ZB.MOM.WW.ScadaBridge.Commons.Messages.Audit;
 using ZB.MOM.WW.ScadaBridge.Commons.Messages.RemoteQuery;
 using ZB.MOM.WW.ScadaBridge.Commons.Types;
 using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
+using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
 using ZB.MOM.WW.ScadaBridge.Communication;
 
 namespace ZB.MOM.WW.ScadaBridge.SiteCallAudit;
@@ -90,6 +92,17 @@ public class SiteCallAuditActor : ReceiveActor
     private readonly SiteCallAuditOptions _options;
     private readonly ILogger<SiteCallAuditActor> _logger;
 
+    /// <summary>
+    /// Central direct-write audit sink for operator Retry/Discard relay actions
+    /// (Task 14). Null when no writer is registered (minimal hosts / most unit
+    /// tests) — in that case the relay emits no audit row and never reads the
+    /// <c>SiteCalls</c> mirror, preserving the "central touches no mirror row on
+    /// the relay path" behaviour. When present, a successful relay emits ONE
+    /// best-effort <c>CachedResolve</c> row recording the operator identity; the
+    /// site remains the source of truth for the state change itself.
+    /// </summary>
+    private readonly ICentralAuditWriter? _auditWriter;
+
     /// <summary>
     /// Reconciliation collaborators (Piece A). The per-site self-heal pull
     /// (<see cref="IPullSiteCallsClient"/>) and the site list
@@ -118,17 +131,32 @@ public class SiteCallAuditActor : ReceiveActor
     private readonly bool _backgroundTimersEnabled;
 
     /// <summary>
-    /// Per-site reconciliation watermark — the highest
-    /// <see cref="SiteCall.UpdatedAtUtc"/> seen for that site on a previous
-    /// tick. The next tick asks for rows at or after this cursor; idempotent
-    /// monotonic <see cref="ISiteCallAuditRepository.UpsertAsync"/> swallows any
-    /// duplicate-with-same-timestamp rows. In-memory for the singleton's
-    /// lifetime — a failover / restart resets every cursor to
-    /// <see cref="DateTime.MinValue"/>, which is conservative but correct
-    /// (the next tick re-pulls and idempotent upsert dedupes). Mirrors
-    /// <c>SiteAuditReconciliationActor</c>.
+    /// Per-site reconciliation watermark — the composite
+    /// <c>(UpdatedAtUtc, TrackedOperationId)</c> keyset of the highest row seen
+    /// for that site on a previous tick (Task 16). The next tick asks for rows
+    /// strictly greater than this pair, so a burst sharing one exact
+    /// <see cref="SiteCall.UpdatedAtUtc"/> drains via the id tiebreak rather than
+    /// pinning the timestamp forever; idempotent monotonic
+    /// <see cref="ISiteCallAuditRepository.UpsertAsync"/> still swallows any
+    /// boundary duplicate. In-memory for the singleton's lifetime — a
+    /// failover / restart resets every cursor to
+    /// <c>(DateTime.MinValue, null)</c>, which is conservative but correct (the
+    /// next tick re-pulls from the start and idempotent upsert dedupes). Mirrors
+    /// <c>SiteAuditReconciliationActor</c>'s cursor (now composite).
     /// </summary>
-    private readonly Dictionary<string, DateTime> _reconciliationCursors = new();
+    private readonly Dictionary<string, (DateTime Since, string? AfterId)> _reconciliationCursors = new();
+
+    /// <summary>
+    /// Per-site "pinned" latch (Task 16) — <c>true</c> once a site's composite
+    /// cursor stopped advancing while the site still reported
+    /// <c>MoreAvailable=true</c> (in practice only a legacy site that ignores the
+    /// <c>after_id</c> keyset field). Tracks the latch so only <em>transitions</em>
+    /// publish <see cref="SiteCallReconciliationPinnedChanged"/> on the EventStream
+    /// (pinned on latch, unpinned when progress resumes), mirroring the sibling
+    /// <c>SiteAuditReconciliationActor</c>'s stalled-state machine — no flood of
+    /// redundant notifications.
+    /// </summary>
+    private readonly Dictionary<string, bool> _reconciliationPinned = new();
 
     private ICancelable? _reconciliationTimer;
     private ICancelable? _purgeTimer;
@@ -166,7 +194,8 @@ public class SiteCallAuditActor : ReceiveActor
     public SiteCallAuditActor(
         ISiteCallAuditRepository repository,
         ILogger<SiteCallAuditActor> logger,
-        SiteCallAuditOptions? options = null)
+        SiteCallAuditOptions? options = null,
+        ICentralAuditWriter? auditWriter = null)
     {
         ArgumentNullException.ThrowIfNull(repository);
         ArgumentNullException.ThrowIfNull(logger);
@@ -174,6 +203,7 @@ public class SiteCallAuditActor : ReceiveActor
         _injectedRepository = repository;
         _logger = logger;
         _options = options ?? new SiteCallAuditOptions();
+        _auditWriter = auditWriter;
 
         // Repo-only MSSQL test ctor: keep BOTH background timers off so the
         // read/upsert tests see no scheduled side effects.
@@ -253,6 +283,12 @@ public class SiteCallAuditActor : ReceiveActor
         _options = options;
         _logger = logger;
 
+        // Central direct-write audit sink for operator relay actions (Task 14).
+        // Resolved once from the root provider — ICentralAuditWriter is a central
+        // singleton (registered by AuditLog). Null-safe: a host that omits it just
+        // relays without emitting the operator-identity row.
+        _auditWriter = serviceProvider.GetService<ICentralAuditWriter>();
+
         // Reconciliation collaborators (Piece A) are central-only singletons
         // registered by AddAuditLogCentralReconciliationClient — always on the
         // central composition root (Program.cs). Resolve them once here (the
@@ -379,7 +415,10 @@ public class SiteCallAuditActor : ReceiveActor
 
         var interval = _options.ResolvedPurgeInterval;
         _purgeTimer = Context.System.Scheduler.ScheduleTellRepeatedlyCancelable(
-            initialDelay: interval,
+            // Short first tick (min(interval, 5min)): purges are idempotent, and a full-interval
+            // initial delay meant a daily-recycled node never purged (arch-review 04, S9). Fast
+            // test cadences (interval < 5min) are unchanged.
+            initialDelay: PurgeTimerSchedule.InitialDelay(interval),
             interval: interval,
             receiver: Self,
             message: PurgeTick.Instance,
@@ -560,52 +599,56 @@ public class SiteCallAuditActor : ReceiveActor
     /// not the source of truth, so a slow site simply lags rather than corrupts.
     /// </para>
     /// <para>
-    /// <b>Inclusive cursor boundary.</b> The cursor is advanced to the maximum
-    /// <see cref="SiteCall.UpdatedAtUtc"/> seen, and the pull asks for rows at or
-    /// after it (<c>since</c> is <c>&gt;=</c>, not <c>&gt;</c>). The row whose
-    /// timestamp equals the cursor is therefore re-pulled on the next tick and
-    /// deduplicated by the idempotent monotonic upsert — the same inclusive-boundary
-    /// contract as <c>SiteAuditReconciliationActor</c>'s cursor.
+    /// <b>Composite keyset cursor (Task 16).</b> The cursor is the
+    /// <c>(UpdatedAtUtc, TrackedOperationId)</c> pair of the maximum row seen, and
+    /// the pull asks for rows strictly greater than it. The boundary row is NOT
+    /// re-pulled (unlike the prior inclusive-timestamp cursor), and — critically —
+    /// a burst of more than <see cref="SiteCallAuditOptions.ReconciliationBatchSize"/>
+    /// rows sharing one exact <see cref="SiteCall.UpdatedAtUtc"/> now drains via the
+    /// id tiebreak instead of pinning the timestamp forever. The idempotent monotonic
+    /// upsert still dedupes any overlap. Matches the composite keyset the site's
+    /// <c>ReadChangedSinceAsync</c> honours (Task 15).
     /// </para>
     /// <para>
     /// <b>Consumes <see cref="PullSiteCallsResponse.MoreAvailable"/>
-    /// to guarantee forward progress.</b> Whereas the prior implementation ignored
-    /// the flag entirely and relied solely on the tick cadence, this method now
-    /// continues pulling within the same tick while the site reports
-    /// <c>MoreAvailable=true</c>, bounded by
-    /// <see cref="MaxReconciliationPagesPerTick"/>. This closes the
-    /// single-timestamp-saturation edge: if a backlog larger than
-    /// <see cref="SiteCallAuditOptions.ReconciliationBatchSize"/> all shares one
-    /// exact <see cref="SiteCall.UpdatedAtUtc"/>, the inclusive max-timestamp cursor
-    /// cannot advance, so the previous code re-pulled the identical window forever
-    /// across ticks and never drained the tail. Here, a saturated batch whose
-    /// observed max timestamp did NOT advance past <c>since</c> is detected as a
-    /// no-progress pin: the loop stops and logs a Warning (the same observability
-    /// intent as the sibling's stalled signal, without its EventStream state
-    /// machine), so the pathological site surfaces rather than spinning silently.
-    /// This diverges from <c>SiteAuditReconciliationActor</c>, which reads
-    /// <c>MoreAvailable</c> to drive a <c>SiteAuditTelemetryStalledChanged</c>
-    /// stalled-detection state machine instead of a within-tick continuation drain.
+    /// to guarantee forward progress.</b> The method continues pulling within the
+    /// same tick while the site reports <c>MoreAvailable=true</c>, bounded by
+    /// <see cref="MaxReconciliationPagesPerTick"/> so a misbehaving site can never
+    /// spin the dispatcher. Each page advances the in-flight composite cursor. If a
+    /// page fails to advance the cursor yet the site still reports more available,
+    /// the site is NOT honouring the <c>after_id</c> keyset (a legacy pre-Task-15
+    /// site): continuing would re-pull the identical window forever, so the loop
+    /// latches the site as <em>pinned</em> and publishes
+    /// <see cref="SiteCallReconciliationPinnedChanged"/> on the EventStream — the
+    /// dead-end becomes a health-observable condition rather than a silent log line.
+    /// When a later tick makes progress the latch clears with <c>Pinned=false</c>.
+    /// This aligns with <c>SiteAuditReconciliationActor</c>'s
+    /// <c>SiteAuditTelemetryStalledChanged</c> transition-only publication.
     /// </para>
     /// </remarks>
     private async Task ReconcileSiteAsync(
         SiteEntry site, IPullSiteCallsClient client, ISiteCallAuditRepository repository)
     {
-        var cursor = _reconciliationCursors.TryGetValue(site.SiteId, out var c) ? c : DateTime.MinValue;
+        var cursor = _reconciliationCursors.TryGetValue(site.SiteId, out var c)
+            ? c
+            : (Since: DateTime.MinValue, AfterId: (string?)null);
 
         // Drain within the tick while the site keeps reporting
         // MoreAvailable, bounded by MaxReconciliationPagesPerTick so a misbehaving
         // site can never spin the dispatcher. Each page advances the in-flight
-        // cursor; a saturated page that fails to advance the cursor is the
-        // single-timestamp no-progress pin — break and surface it.
+        // composite cursor; a page that fails to advance it while MoreAvailable is
+        // still true means the site ignores after_id (legacy) — latch + publish
+        // the pinned state and break.
         for (var page = 0; page < MaxReconciliationPagesPerTick; page++)
         {
-            var since = cursor;
+            var since = cursor.Since;
+            var afterId = cursor.AfterId;
             var response = await client
-                .PullAsync(site.SiteId, since, _options.ReconciliationBatchSize, CancellationToken.None)
+                .PullAsync(site.SiteId, since, afterId, _options.ReconciliationBatchSize, CancellationToken.None)
                 .ConfigureAwait(false);
 
             var maxUpdated = since;
+            var maxAfterId = afterId;
             var nowUtc = DateTime.UtcNow;
             foreach (var row in response.SiteCalls)
             {
@@ -616,40 +659,57 @@ public class SiteCallAuditActor : ReceiveActor
                 var siteCall = row with { IngestedAtUtc = nowUtc };
                 await repository.UpsertAsync(siteCall).ConfigureAwait(false);
 
-                if (row.UpdatedAtUtc > maxUpdated)
+                // Advance the composite max: greater timestamp wins; on a tie the
+                // greater TrackedOperationId (ordinal, matching the site's SQLite
+                // BINARY text collation) wins.
+                var rowId = row.TrackedOperationId.ToString();
+                if (row.UpdatedAtUtc > maxUpdated ||
+                    (row.UpdatedAtUtc == maxUpdated && string.CompareOrdinal(rowId, maxAfterId) > 0))
                 {
                     maxUpdated = row.UpdatedAtUtc;
+                    maxAfterId = rowId;
                 }
             }
 
+            // Did the composite cursor move? (timestamp advanced OR — at the same
+            // timestamp — the id tiebreak advanced). CompareOrdinal handles nulls
+            // (CompareOrdinal(x, null) > 0 for any non-null x).
+            var advanced = maxUpdated > since || string.CompareOrdinal(maxAfterId, afterId) != 0;
+
             // Persist the advanced cursor after every page so a fault on a later
             // page (caught per-site upstream) still keeps the rows already drained.
-            cursor = maxUpdated;
+            cursor = (maxUpdated, maxAfterId);
             _reconciliationCursors[site.SiteId] = cursor;
 
+            // Progress resumed → clear any prior pinned latch (publishes the
+            // Pinned=false transition exactly once).
+            if (advanced)
+            {
+                SetReconciliationPinned(site.SiteId, pinned: false);
+            }
+
             if (!response.MoreAvailable)
             {
                 // Backlog fully drained for this site this tick.
                 return;
             }
 
-            if (maxUpdated <= since)
+            if (!advanced)
             {
-                // No-progress pin: the site saturated the batch yet the max
-                // observed UpdatedAtUtc did not advance past the inclusive cursor
-                // (a burst of > batch-size rows sharing one exact timestamp).
-                // Continuing would re-pull the identical window forever, so stop
-                // and surface it — the inclusive max-timestamp cursor cannot make
-                // progress on this input without a composite (timestamp,id)
-                // keyset, which the pull contract does not yet support.
+                // The composite cursor did not advance yet the site still reports
+                // MoreAvailable — the site ignored after_id (a legacy pre-keyset
+                // site), so a continuation would re-pull the identical window
+                // forever. Latch + publish the pinned state and stop; the next tick
+                // re-checks and unpins if the site catches up.
+                SetReconciliationPinned(site.SiteId, pinned: true);
                 _logger.LogWarning(
-                    "SiteCallAudit reconciliation for site {SiteId} cannot make progress: a saturated "
-                    + "batch of more than {BatchSize} rows shares a single UpdatedAtUtc ({CursorUtc:o}), "
-                    + "so the inclusive cursor is pinned. The backlog tail beyond the batch ceiling will "
-                    + "not reconcile until those rows' timestamps differ.",
+                    "SiteCallAudit reconciliation for site {SiteId} is pinned: the composite "
+                    + "(UpdatedAtUtc, TrackedOperationId) cursor did not advance past ({CursorUtc:o}, {AfterId}) "
+                    + "yet the site reports more rows available — the site does not honour the after_id keyset "
+                    + "(legacy). The backlog tail will not reconcile until the site is upgraded.",
                     site.SiteId,
-                    _options.ReconciliationBatchSize,
-                    since);
+                    since,
+                    afterId ?? "(none)");
                 return;
             }
         }
@@ -666,6 +726,25 @@ public class SiteCallAuditActor : ReceiveActor
             MaxReconciliationPagesPerTick);
     }
 
+    /// <summary>
+    /// Latches the per-site reconciliation pinned state and publishes
+    /// <see cref="SiteCallReconciliationPinnedChanged"/> on the actor system
+    /// EventStream ONLY on a transition — mirroring
+    /// <c>SiteAuditReconciliationActor.UpdateStalledState</c> so a downstream
+    /// health subscriber never sees a flood of same-state notifications.
+    /// </summary>
+    private void SetReconciliationPinned(string siteId, bool pinned)
+    {
+        var was = _reconciliationPinned.TryGetValue(siteId, out var prior) && prior;
+        if (was == pinned)
+        {
+            return;
+        }
+
+        _reconciliationPinned[siteId] = pinned;
+        Context.System.EventStream.Publish(new SiteCallReconciliationPinnedChanged(siteId, pinned));
+    }
+
     // ── Piece B: daily terminal-row purge scheduler ──
 
     /// <summary>
@@ -1026,11 +1105,37 @@ public class SiteCallAuditActor : ReceiveActor
             request.CorrelationId, new TrackedOperationId(request.TrackedOperationId));
         var envelope = new SiteEnvelope(request.SourceSite, relay);
 
-        _centralCommunication.Ask<ParkedOperationActionAck>(envelope, _options.RelayTimeout)
-            .PipeTo(
-                sender,
-                success: ack => MapRetryResponse(request.CorrelationId, ack),
-                failure: ex => MapRetryFailure(request.CorrelationId, request.SourceSite, ex));
+        RelayRetryAsync(request, envelope).PipeTo(sender);
+    }
+
+    /// <summary>
+    /// Awaits the site's ack for a Retry relay, maps it onto the response, and —
+    /// on an <see cref="SiteCallRelayOutcome.Applied"/> outcome with an audit sink
+    /// present — emits ONE best-effort operator-identity audit row (Task 14). The
+    /// relay outcome is authoritative; the audit write never changes it.
+    /// </summary>
+    private async Task<RetrySiteCallResponse> RelayRetryAsync(RetrySiteCallRequest request, SiteEnvelope envelope)
+    {
+        ParkedOperationActionAck ack;
+        try
+        {
+            ack = await _centralCommunication!
+                .Ask<ParkedOperationActionAck>(envelope, _options.RelayTimeout)
+                .ConfigureAwait(false);
+        }
+        catch (Exception ex)
+        {
+            return MapRetryFailure(request.CorrelationId, request.SourceSite, ex);
+        }
+
+        var response = MapRetryResponse(request.CorrelationId, ack);
+        if (response.Outcome == SiteCallRelayOutcome.Applied)
+        {
+            await EmitRelayAuditAsync(
+                request.TrackedOperationId, request.RequestedBy, AuditStatus.Submitted).ConfigureAwait(false);
+        }
+
+        return response;
     }
 
     /// <summary>
@@ -1056,11 +1161,91 @@ public class SiteCallAuditActor : ReceiveActor
             request.CorrelationId, new TrackedOperationId(request.TrackedOperationId));
         var envelope = new SiteEnvelope(request.SourceSite, relay);
 
-        _centralCommunication.Ask<ParkedOperationActionAck>(envelope, _options.RelayTimeout)
-            .PipeTo(
-                sender,
-                success: ack => MapDiscardResponse(request.CorrelationId, ack),
-                failure: ex => MapDiscardFailure(request.CorrelationId, request.SourceSite, ex));
+        RelayDiscardAsync(request, envelope).PipeTo(sender);
+    }
+
+    /// <summary>
+    /// Awaits the site's ack for a Discard relay, maps it onto the response, and —
+    /// on an <see cref="SiteCallRelayOutcome.Applied"/> outcome with an audit sink
+    /// present — emits ONE best-effort operator-identity audit row (Task 14).
+    /// </summary>
+    private async Task<DiscardSiteCallResponse> RelayDiscardAsync(DiscardSiteCallRequest request, SiteEnvelope envelope)
+    {
+        ParkedOperationActionAck ack;
+        try
+        {
+            ack = await _centralCommunication!
+                .Ask<ParkedOperationActionAck>(envelope, _options.RelayTimeout)
+                .ConfigureAwait(false);
+        }
+        catch (Exception ex)
+        {
+            return MapDiscardFailure(request.CorrelationId, request.SourceSite, ex);
+        }
+
+        var response = MapDiscardResponse(request.CorrelationId, ack);
+        if (response.Outcome == SiteCallRelayOutcome.Applied)
+        {
+            await EmitRelayAuditAsync(
+                request.TrackedOperationId, request.RequestedBy, AuditStatus.Discarded).ConfigureAwait(false);
+        }
+
+        return response;
+    }
+
+    /// <summary>
+    /// Emits one best-effort central direct-write <see cref="AuditKind.CachedResolve"/>
+    /// row attributing an operator Retry/Discard relay to <paramref name="actor"/>. The
+    /// audited <see cref="AuditChannel"/>, target, source site and node are read from the
+    /// stored <c>SiteCalls</c> mirror row (a benign READ — central still never mutates the
+    /// mirror). No-op when no <see cref="ICentralAuditWriter"/> is registered. A missing
+    /// row or a writer fault is swallowed and logged — the relay outcome is authoritative.
+    /// </summary>
+    private async Task EmitRelayAuditAsync(Guid trackedOperationId, string? actor, AuditStatus status)
+    {
+        if (_auditWriter is null)
+        {
+            return;
+        }
+
+        var (scope, repository) = ResolveRepository();
+        try
+        {
+            var row = await repository
+                .GetAsync(new TrackedOperationId(trackedOperationId))
+                .ConfigureAwait(false);
+
+            // SiteCall.Channel is the string form of an AuditChannel ("ApiOutbound"/
+            // "DbOutbound"); fall back to ApiOutbound if the row is missing or the
+            // string is unrecognised so the row still records who asked.
+            var channel = row is not null && Enum.TryParse<AuditChannel>(row.Channel, out var parsed)
+                ? parsed
+                : AuditChannel.ApiOutbound;
+
+            var evt = ScadaBridgeAuditEventFactory.Create(
+                channel: channel,
+                kind: AuditKind.CachedResolve,
+                status: status,
+                occurredAtUtc: DateTime.UtcNow,
+                actor: actor,
+                target: row?.Target,
+                sourceNode: row?.SourceNode,
+                correlationId: trackedOperationId,
+                sourceSiteId: row?.SourceSite);
+
+            await _auditWriter.WriteAsync(evt).ConfigureAwait(false);
+        }
+        catch (Exception ex)
+        {
+            _logger.LogWarning(
+                ex,
+                "Failed to emit operator relay audit row for {TrackedOperationId}.",
+                trackedOperationId);
+        }
+        finally
+        {
+            scope?.Dispose();
+        }
     }
 
     /// <summary>

src/ZB.MOM.WW.ScadaBridge.SiteCallAudit/SiteCallReconciliationPinnedChanged.cs

@@ -0,0 +1,19 @@
+namespace ZB.MOM.WW.ScadaBridge.SiteCallAudit;
+
+/// <summary>
+/// Published on the actor system EventStream when a site's Site Call Audit
+/// reconciliation puller transitions into or out of the "pinned" state — the
+/// composite <c>(UpdatedAtUtc, TrackedOperationId)</c> keyset cursor could not
+/// advance yet the site still reports <c>MoreAvailable=true</c>, so the backlog
+/// tail cannot be drained. In practice this only happens against a legacy site
+/// that ignores the <c>after_id</c> keyset field (a modern site drains a
+/// single-timestamp burst via the id tiebreak). Mirrors
+/// <c>SiteAuditTelemetryStalledChanged</c>: only transitions are published
+/// (<c>Pinned=true</c> on latch, <c>Pinned=false</c> when progress resumes), so
+/// a downstream health subscriber never sees a flood of redundant notifications.
+/// The dead-end that the previous implementation only logged is now a
+/// health-observable condition.
+/// </summary>
+/// <param name="SiteId">The site whose reconciliation cursor pinned (or unpinned).</param>
+/// <param name="Pinned">True on entering the pinned state; false when the cursor makes progress again.</param>
+public sealed record SiteCallReconciliationPinnedChanged(string SiteId, bool Pinned);

src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Tracking/OperationTrackingStore.cs

@@ -365,6 +365,7 @@ public class OperationTrackingStore : IOperationTrackingStore, IAsyncDisposable,
     public async Task<IReadOnlyList<SiteCallOperational>> ReadChangedSinceAsync(
         DateTime sinceUtc,
         int batchSize,
+        string? afterId = null,
         CancellationToken ct = default)
     {
         ObjectDisposedException.ThrowIf(Volatile.Read(ref _disposeState) != 0, this);
@@ -379,18 +380,28 @@ public class OperationTrackingStore : IOperationTrackingStore, IAsyncDisposable,
         await readConnection.OpenAsync(ct).ConfigureAwait(false);
 
         await using var cmd = readConnection.CreateCommand();
-        // Inclusive lower bound on UpdatedAtUtc (>=) so a caller resuming from
-        // the last returned timestamp does not skip a row sharing that instant;
-        // central ingest is insert-if-not-exists + upsert-on-newer, so the
-        // boundary row re-read is a no-op. ORDER BY ... ASC + LIMIT yields the
-        // OLDEST matching rows so the cursor advances monotonically.
-        cmd.CommandText = """
+
+        // Composite (UpdatedAtUtc, TrackedOperationId) keyset (Task 15). Ordering
+        // is ALWAYS deterministic (UpdatedAtUtc ASC, TrackedOperationId ASC) so the
+        // cursor is well-defined even when many rows share one UpdatedAtUtc.
+        //   • afterId present → strict keyset: skip everything up to and including
+        //     (sinceUtc, afterId). Without this a page fully inside one UpdatedAtUtc
+        //     instant re-reads forever (the plain >= resume never advances past the
+        //     tie). TrackedOperationId is the "D" GUID form on both sides, so the
+        //     TEXT comparison is a stable total order.
+        //   • afterId null/empty → legacy inclusive >= sinceUtc (an older central
+        //     that does not send a cursor is byte-for-byte unaffected).
+        var useKeyset = !string.IsNullOrEmpty(afterId);
+        var predicate = useKeyset
+            ? "UpdatedAtUtc > $since OR (UpdatedAtUtc = $since AND TrackedOperationId > $afterId)"
+            : "UpdatedAtUtc >= $since";
+        cmd.CommandText = $"""
             SELECT TrackedOperationId, Kind, TargetSummary, Status,
                    RetryCount, LastError, HttpStatus,
                    CreatedAtUtc, UpdatedAtUtc, TerminalAtUtc, SourceNode
             FROM   OperationTracking
-            WHERE  UpdatedAtUtc >= $since
-            ORDER  BY UpdatedAtUtc ASC
+            WHERE  {predicate}
+            ORDER  BY UpdatedAtUtc ASC, TrackedOperationId ASC
             LIMIT  $batchSize;
             """;
         // Force UTC kind before formatting so the cursor's "o" text matches the
@@ -403,6 +414,10 @@ public class OperationTrackingStore : IOperationTrackingStore, IAsyncDisposable,
             .ToString("o", CultureInfo.InvariantCulture);
         cmd.Parameters.AddWithValue("$since", sinceText);
         cmd.Parameters.AddWithValue("$batchSize", batchSize);
+        if (useKeyset)
+        {
+            cmd.Parameters.AddWithValue("$afterId", afterId!);
+        }
 
         var rows = new List<SiteCallOperational>();
         await using var reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);

tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Central/GrpcPullAuditEventsClientTests.cs

@@ -71,6 +71,26 @@ public class GrpcPullAuditEventsClientTests
         }
     }
 
+    /// <summary>
+    /// Per-endpoint fake invoker (Task 18): routes dials by endpoint string so a
+    /// NodeA-down / NodeB-up failover can be scripted. Records dialed endpoints.
+    /// </summary>
+    private sealed class PerEndpointInvoker : GrpcPullAuditEventsClient.IPullAuditEventsInvoker
+    {
+        private readonly Dictionary<string, Func<ProtoPullResponse>> _byEndpoint;
+        public List<string> Dialed { get; } = new();
+
+        public PerEndpointInvoker(Dictionary<string, Func<ProtoPullResponse>> byEndpoint) =>
+            _byEndpoint = byEndpoint;
+
+        public Task<ProtoPullResponse> InvokeAsync(
+            string endpoint, ProtoPullRequest request, CancellationToken ct)
+        {
+            Dialed.Add(endpoint);
+            return Task.FromResult(_byEndpoint[endpoint]());
+        }
+    }
+
     private static AuditEventDto Dto(Guid id, DateTime occurredAtUtc) =>
         AuditEventDtoMapper.ToDto(ScadaBridgeAuditEventFactory.Create(
             eventId: id,
@@ -212,4 +232,48 @@ public class GrpcPullAuditEventsClientTests
         Assert.Empty(result.Events);
         Assert.False(result.MoreAvailable);
     }
+
+    [Fact]
+    public async Task PullAsync_fails_over_to_NodeB_when_primary_raises_transport_fault()
+    {
+        // Task 18: NodeA (primary) is unreachable, NodeB (fallback) answers — the
+        // pull must return NodeB's events rather than collapsing to empty.
+        var id = Guid.NewGuid();
+        var fallbackResponse = new ProtoPullResponse { MoreAvailable = false };
+        fallbackResponse.Events.Add(Dto(id, BaseTime));
+
+        var invoker = new PerEndpointInvoker(new Dictionary<string, Func<ProtoPullResponse>>
+        {
+            ["http://node-a:8083"] = () => throw new RpcException(new Status(StatusCode.Unavailable, "node A down")),
+            ["http://node-b:8083"] = () => fallbackResponse,
+        });
+        var sut = new GrpcPullAuditEventsClient(
+            new StaticEnumerator(new SiteEntry("site-a", "http://node-a:8083", "http://node-b:8083")),
+            invoker,
+            NullLogger<GrpcPullAuditEventsClient>.Instance);
+
+        var result = await sut.PullAsync("site-a", BaseTime, batchSize: 256, CancellationToken.None);
+
+        Assert.Equal(new[] { "http://node-a:8083", "http://node-b:8083" }, invoker.Dialed);
+        var evt = Assert.Single(result.Events);
+        Assert.Equal(id, evt.EventId);
+    }
+
+    [Fact]
+    public async Task PullAsync_does_not_fail_over_when_no_fallback_configured()
+    {
+        var invoker = new PerEndpointInvoker(new Dictionary<string, Func<ProtoPullResponse>>
+        {
+            ["http://node-a:8083"] = () => throw new RpcException(new Status(StatusCode.Unavailable, "node A down")),
+        });
+        var sut = new GrpcPullAuditEventsClient(
+            new StaticEnumerator(new SiteEntry("site-a", "http://node-a:8083")), // no fallback
+            invoker,
+            NullLogger<GrpcPullAuditEventsClient>.Instance);
+
+        var result = await sut.PullAsync("site-a", BaseTime, batchSize: 256, CancellationToken.None);
+
+        Assert.Empty(result.Events);
+        Assert.Equal(new[] { "http://node-a:8083" }, invoker.Dialed);
+    }
 }

tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Central/GrpcPullSiteCallsClientTests.cs

@@ -63,6 +63,28 @@ public class GrpcPullSiteCallsClientTests
         }
     }
 
+    /// <summary>
+    /// Per-endpoint fake invoker (Task 18): dials are routed by endpoint string so
+    /// a NodeA-down / NodeB-up failover can be scripted. Records every dialed
+    /// endpoint in order.
+    /// </summary>
+    private sealed class PerEndpointInvoker : GrpcPullSiteCallsClient.IPullSiteCallsInvoker
+    {
+        private readonly Dictionary<string, Func<ProtoPullResponse>> _byEndpoint;
+        public List<string> Dialed { get; } = new();
+
+        public PerEndpointInvoker(Dictionary<string, Func<ProtoPullResponse>> byEndpoint) =>
+            _byEndpoint = byEndpoint;
+
+        public Task<ProtoPullResponse> InvokeAsync(
+            string endpoint, ProtoPullRequest request, CancellationToken ct)
+        {
+            Dialed.Add(endpoint);
+            // The Func may throw (transport fault) — the client's try/catch handles it.
+            return Task.FromResult(_byEndpoint[endpoint]());
+        }
+    }
+
     // The site leaves SourceSite empty (it is not a tracking-store column); the
     // client re-stamps it from the dialed siteId. Mint DTOs with empty SourceSite
     // to prove that re-stamp.
@@ -98,7 +120,7 @@ public class GrpcPullSiteCallsClientTests
             invoker,
             NullLogger<GrpcPullSiteCallsClient>.Instance);
 
-        var result = await sut.PullAsync("site-a", BaseTime, batchSize: 256, CancellationToken.None);
+        var result = await sut.PullAsync("site-a", BaseTime, afterId: null, batchSize: 256, CancellationToken.None);
 
         // Endpoint resolution + request shaping.
         Assert.Equal("http://site-a:8083", invoker.Endpoint);
@@ -131,7 +153,7 @@ public class GrpcPullSiteCallsClientTests
             invoker,
             NullLogger<GrpcPullSiteCallsClient>.Instance);
 
-        var result = await sut.PullAsync("site-a", BaseTime, batchSize: 256, CancellationToken.None);
+        var result = await sut.PullAsync("site-a", BaseTime, afterId: null, batchSize: 256, CancellationToken.None);
 
         Assert.Empty(result.SiteCalls);
         Assert.False(result.MoreAvailable);
@@ -150,7 +172,7 @@ public class GrpcPullSiteCallsClientTests
             invoker,
             NullLogger<GrpcPullSiteCallsClient>.Instance);
 
-        var result = await sut.PullAsync("site-a", BaseTime, batchSize: 256, CancellationToken.None);
+        var result = await sut.PullAsync("site-a", BaseTime, afterId: null, batchSize: 256, CancellationToken.None);
 
         Assert.Empty(result.SiteCalls);
         Assert.False(result.MoreAvailable);
@@ -165,7 +187,7 @@ public class GrpcPullSiteCallsClientTests
             invoker,
             NullLogger<GrpcPullSiteCallsClient>.Instance);
 
-        var result = await sut.PullAsync("site-a", BaseTime, batchSize: 256, CancellationToken.None);
+        var result = await sut.PullAsync("site-a", BaseTime, afterId: null, batchSize: 256, CancellationToken.None);
 
         Assert.Empty(result.SiteCalls);
         Assert.False(result.MoreAvailable);
@@ -180,7 +202,7 @@ public class GrpcPullSiteCallsClientTests
             invoker,
             NullLogger<GrpcPullSiteCallsClient>.Instance);
 
-        var result = await sut.PullAsync("site-a", BaseTime, batchSize: 256, CancellationToken.None);
+        var result = await sut.PullAsync("site-a", BaseTime, afterId: null, batchSize: 256, CancellationToken.None);
 
         Assert.Empty(result.SiteCalls);
         Assert.False(result.MoreAvailable);
@@ -211,7 +233,7 @@ public class GrpcPullSiteCallsClientTests
             NullLogger<GrpcPullSiteCallsClient>.Instance);
 
         // Must NOT throw — the bad row is dropped, the good rows are returned.
-        var result = await sut.PullAsync("site-a", BaseTime, batchSize: 256, CancellationToken.None);
+        var result = await sut.PullAsync("site-a", BaseTime, afterId: null, batchSize: 256, CancellationToken.None);
 
         Assert.Equal(2, result.SiteCalls.Count);
         // Survivors are oldest-first and SourceSite re-stamped from the dialed siteId.
@@ -239,7 +261,7 @@ public class GrpcPullSiteCallsClientTests
             invoker,
             NullLogger<GrpcPullSiteCallsClient>.Instance);
 
-        var result = await sut.PullAsync("site-a", minUnspecified, batchSize: 256, CancellationToken.None);
+        var result = await sut.PullAsync("site-a", minUnspecified, afterId: null, batchSize: 256, CancellationToken.None);
 
         Assert.Equal(1, invoker.CallCount);
         Assert.Equal("http://site-a:8083", invoker.Endpoint);
@@ -248,4 +270,52 @@ public class GrpcPullSiteCallsClientTests
         Assert.Empty(result.SiteCalls);
         Assert.False(result.MoreAvailable);
     }
+
+    [Fact]
+    public async Task PullAsync_fails_over_to_NodeB_when_primary_raises_transport_fault()
+    {
+        // Task 18: NodeA (primary) is unreachable, NodeB (fallback) answers — the
+        // pull must return NodeB's rows rather than collapsing to empty.
+        var id = Guid.NewGuid();
+        var fallbackResponse = new ProtoPullResponse();
+        fallbackResponse.Operationals.Add(Dto(id, BaseTime.AddMinutes(1)));
+
+        var invoker = new PerEndpointInvoker(new Dictionary<string, Func<ProtoPullResponse>>
+        {
+            ["http://node-a:8083"] = () => throw new RpcException(new Status(StatusCode.Unavailable, "node A down")),
+            ["http://node-b:8083"] = () => fallbackResponse,
+        });
+        var sut = new GrpcPullSiteCallsClient(
+            new StaticEnumerator(new SiteEntry("site-a", "http://node-a:8083", "http://node-b:8083")),
+            invoker,
+            NullLogger<GrpcPullSiteCallsClient>.Instance);
+
+        var result = await sut.PullAsync("site-a", BaseTime, afterId: null, batchSize: 256, CancellationToken.None);
+
+        // Primary dialed first, then the NodeB fallback once.
+        Assert.Equal(new[] { "http://node-a:8083", "http://node-b:8083" }, invoker.Dialed);
+        var row = Assert.Single(result.SiteCalls);
+        Assert.Equal(id, row.TrackedOperationId.Value);
+        Assert.Equal("site-a", row.SourceSite);
+    }
+
+    [Fact]
+    public async Task PullAsync_does_not_fail_over_when_no_fallback_configured()
+    {
+        // A transport fault with no NodeB fallback collapses to empty and dials
+        // only the primary once (no phantom second dial).
+        var invoker = new PerEndpointInvoker(new Dictionary<string, Func<ProtoPullResponse>>
+        {
+            ["http://node-a:8083"] = () => throw new RpcException(new Status(StatusCode.Unavailable, "node A down")),
+        });
+        var sut = new GrpcPullSiteCallsClient(
+            new StaticEnumerator(new SiteEntry("site-a", "http://node-a:8083")), // no fallback
+            invoker,
+            NullLogger<GrpcPullSiteCallsClient>.Instance);
+
+        var result = await sut.PullAsync("site-a", BaseTime, afterId: null, batchSize: 256, CancellationToken.None);
+
+        Assert.Empty(result.SiteCalls);
+        Assert.Equal(new[] { "http://node-a:8083" }, invoker.Dialed);
+    }
 }

tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Central/SiteAuditReconciliationActorTests.cs

@@ -113,6 +113,69 @@ public class SiteAuditReconciliationActorTests : TestKit, IClassFixture<MsSqlMig
             Task.FromResult<IReadOnlyList<string>>(Array.Empty<string>());
     }
 
+    /// <summary>
+    /// Recording stub whose <see cref="InsertIfNotExistsAsync"/> throws for ONE
+    /// poison <see cref="AuditEvent.EventId"/> (every attempt) but records every
+    /// other row — including the synthetic ReconciliationAbandoned row the actor
+    /// emits once it gives up on the poison row (Task 17).
+    /// </summary>
+    private sealed class PoisonInsertRepo : IAuditLogRepository
+    {
+        private readonly Guid _poisonId;
+        private readonly Exception _fault;
+        public List<AuditEvent> Inserted { get; } = new();
+        private readonly HashSet<Guid> _seen = new();
+
+        public PoisonInsertRepo(Guid poisonId, Exception fault)
+        {
+            _poisonId = poisonId;
+            _fault = fault;
+        }
+
+        public Task InsertIfNotExistsAsync(AuditEvent evt, CancellationToken ct = default)
+        {
+            if (evt.EventId == _poisonId)
+            {
+                throw _fault;
+            }
+            if (_seen.Add(evt.EventId))
+            {
+                Inserted.Add(evt);
+            }
+            return Task.CompletedTask;
+        }
+
+        public Task<IReadOnlyList<AuditEvent>> QueryAsync(
+            AuditLogQueryFilter filter, AuditLogPaging paging, CancellationToken ct = default) =>
+            Task.FromResult<IReadOnlyList<AuditEvent>>(Inserted);
+
+        public Task<long> SwitchOutPartitionAsync(DateTime monthBoundary, TimeSpan? commandTimeout = null, CancellationToken ct = default) =>
+            Task.FromResult(0L);
+
+        public Task<long> PurgeChannelOlderThanAsync(
+            string channel, DateTime threshold, int batchSize, TimeSpan? commandTimeout = null, CancellationToken ct = default) =>
+            Task.FromResult(0L);
+
+        public Task<long> BackfillSourceNodeAsync(
+            string sentinel, DateTime before, int batchSize, CancellationToken ct = default) =>
+            Task.FromResult(0L);
+
+        public Task<IReadOnlyList<DateTime>> GetPartitionBoundariesOlderThanAsync(
+            DateTime threshold, CancellationToken ct = default) =>
+            Task.FromResult<IReadOnlyList<DateTime>>(Array.Empty<DateTime>());
+
+        public Task<ZB.MOM.WW.ScadaBridge.Commons.Types.AuditLogKpiSnapshot> GetKpiSnapshotAsync(
+            TimeSpan window, DateTime? nowUtc = null, CancellationToken ct = default) =>
+            Task.FromResult(new ZB.MOM.WW.ScadaBridge.Commons.Types.AuditLogKpiSnapshot(0L, 0L, 0L, nowUtc ?? DateTime.UtcNow));
+
+        public Task<IReadOnlyList<ExecutionTreeNode>> GetExecutionTreeAsync(
+            Guid executionId, CancellationToken ct = default) =>
+            Task.FromResult<IReadOnlyList<ExecutionTreeNode>>(Array.Empty<ExecutionTreeNode>());
+
+        public Task<IReadOnlyList<string>> GetDistinctSourceNodesAsync(CancellationToken ct = default) =>
+            Task.FromResult<IReadOnlyList<string>>(Array.Empty<string>());
+    }
+
     /// <summary>
     /// In-memory enumerator returning a static list of sites.
     /// </summary>
@@ -457,4 +520,56 @@ public class SiteAuditReconciliationActorTests : TestKit, IClassFixture<MsSqlMig
         Assert.False(second.Stalled);
         Assert.Equal("siteA", second.SiteId);
     }
+
+    // ---------------------------------------------------------------------
+    // 9. Task 17: a permanently-abandoned row leaves a durable
+    //    ReconciliationAbandoned audit record (not just a Critical log line),
+    //    and the cursor advances past it.
+    // ---------------------------------------------------------------------
+
+    [Fact]
+    public void PermanentAbandonment_EmitsReconciliationAbandonedRow_AndAdvancesCursor()
+    {
+        var siteId = "siteAbandon";
+        var poisonTime = new DateTime(2026, 5, 20, 10, 5, 0, DateTimeKind.Utc);
+        var poison = NewEvent(siteId, poisonTime);
+
+        var sites = new StaticEnumerator(new SiteEntry(siteId, "http://x:8083"));
+        // Five pulls each returning the same poison row so its per-EventId attempt
+        // counter reaches MaxPermanentInsertAttempts (5); afterwards the scripted
+        // client returns empty and the actor settles.
+        var responses = Enumerable.Range(0, 5)
+            .Select(_ => new PullAuditEventsResponse(new[] { poison }, MoreAvailable: false))
+            .ToArray();
+        var client = new ScriptedPullClient().Script(siteId, responses);
+        var repo = new PoisonInsertRepo(
+            poison.EventId, new InvalidOperationException("poison row will never insert"));
+
+        CreateActor(sites, client, repo, FastTickOptions());
+
+        // After the 5th failed attempt the actor abandons the row and writes ONE
+        // synthetic ReconciliationAbandoned row (fresh EventId) whose details carry
+        // the abandoned id + the site.
+        AwaitAssert(
+            () =>
+            {
+                var synthetic = repo.Inserted.SingleOrDefault(
+                    e => e.Action.EndsWith(".ReconciliationAbandoned", StringComparison.Ordinal));
+                Assert.NotNull(synthetic);
+                Assert.NotEqual(poison.EventId, synthetic!.EventId);
+                Assert.Contains(poison.EventId.ToString(), synthetic.DetailsJson);
+                Assert.Contains(siteId, synthetic.DetailsJson);
+            },
+            duration: TimeSpan.FromSeconds(8),
+            interval: TimeSpan.FromMilliseconds(100));
+
+        // The cursor advanced past MinValue to the abandoned row's timestamp — the
+        // poison row no longer pins the site's reconciliation forever. A later tick
+        // must pull with since = poisonTime (poll for it; the abandonment tick and
+        // the first advanced-cursor pull are separate ticks).
+        AwaitAssert(
+            () => Assert.Contains(client.Calls, c => c.SinceUtc == poisonTime),
+            duration: TimeSpan.FromSeconds(5),
+            interval: TimeSpan.FromMilliseconds(100));
+    }
 }

tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Central/SiteEnumeratorTests.cs

@@ -76,6 +76,44 @@ public class SiteEnumeratorTests
         Assert.Empty(result);
     }
 
+    [Fact]
+    public async Task EnumerateAsync_BothAddresses_ProjectsNodeBAsFallback()
+    {
+        // Task 18: a site with both NodeA and a DISTINCT NodeB yields
+        // SiteEntry(primary=NodeA, FallbackGrpcEndpoint=NodeB).
+        var repository = Substitute.For<ISiteRepository>();
+        repository.GetAllSitesAsync(Arg.Any<CancellationToken>()).Returns(new List<SiteEntity>
+        {
+            SiteWith("site-ab", grpcNodeA: "http://site-a:8083", grpcNodeB: "http://site-b:8083"),
+        });
+
+        var enumerator = new SiteEnumerator(BuildProvider(repository));
+
+        var entry = Assert.Single(await enumerator.EnumerateAsync());
+        Assert.Equal("site-ab", entry.SiteId);
+        Assert.Equal("http://site-a:8083", entry.GrpcEndpoint);
+        Assert.Equal("http://site-b:8083", entry.FallbackGrpcEndpoint);
+    }
+
+    [Fact]
+    public async Task EnumerateAsync_OnlyNodeB_IsNotSkipped_AndHasNoFallback()
+    {
+        // Task 18: a site with a blank NodeA but a valid NodeB is no longer
+        // skipped — NodeB becomes the primary and there is no separate fallback.
+        var repository = Substitute.For<ISiteRepository>();
+        repository.GetAllSitesAsync(Arg.Any<CancellationToken>()).Returns(new List<SiteEntity>
+        {
+            SiteWith("site-bonly", grpcNodeA: "   ", grpcNodeB: "http://site-b:8083"),
+        });
+
+        var enumerator = new SiteEnumerator(BuildProvider(repository));
+
+        var entry = Assert.Single(await enumerator.EnumerateAsync());
+        Assert.Equal("site-bonly", entry.SiteId);
+        Assert.Equal("http://site-b:8083", entry.GrpcEndpoint);
+        Assert.Null(entry.FallbackGrpcEndpoint);
+    }
+
     [Fact]
     public async Task EnumerateAsync_ReturnsEmpty_WhenNoSites()
     {

tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Kpi/AuditLogKpiSampleSourceTests.cs

@@ -1,5 +1,6 @@
 using NSubstitute;
 using ZB.MOM.WW.ScadaBridge.AuditLog.Kpi;
+using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Kpi;
 using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
 using ZB.MOM.WW.ScadaBridge.Commons.Types;
 using ZB.MOM.WW.ScadaBridge.Commons.Types.Kpi;
@@ -97,6 +98,57 @@ public class AuditLogKpiSampleSourceTests
         Assert.Empty(samples);
     }
 
+    [Fact]
+    public async Task CollectAsync_uses_backlog_provider_when_present()
+    {
+        // The repository's snapshot carries the hardwired zero the append-only
+        // repo always emits; the provider carries the real cross-site aggregate.
+        var snapshot = new AuditLogKpiSnapshot(
+            TotalEventsLastHour: 10,
+            ErrorEventsLastHour: 1,
+            BacklogTotal: 0,
+            AsOfUtc: CapturedAt);
+
+        var repo = Substitute.For<IAuditLogRepository>();
+        repo.GetKpiSnapshotAsync(Arg.Any<TimeSpan>(), Arg.Any<DateTime?>(), Arg.Any<CancellationToken>())
+            .Returns(snapshot);
+
+        var backlog = Substitute.For<IAuditBacklogProvider>();
+        backlog.GetPendingBacklogTotal().Returns(42L);
+
+        var sut = new AuditLogKpiSampleSource(repo, backlog);
+
+        var samples = await sut.CollectAsync(CapturedAt);
+
+        // Volume + error metrics still come from the snapshot; backlog is the
+        // provider's real aggregate, not the snapshot's zero.
+        AssertMetric(samples, "totalEventsLastHour", 10);
+        AssertMetric(samples, "errorEventsLastHour", 1);
+        AssertMetric(samples, "backlogTotal", 42);
+    }
+
+    [Fact]
+    public async Task CollectAsync_falls_back_to_snapshot_backlog_when_provider_absent()
+    {
+        // No provider (sites / unit tests): the sample source degrades to the
+        // snapshot's own BacklogTotal — unchanged pre-Task-10 behavior.
+        var snapshot = new AuditLogKpiSnapshot(
+            TotalEventsLastHour: 10,
+            ErrorEventsLastHour: 1,
+            BacklogTotal: 7,
+            AsOfUtc: CapturedAt);
+
+        var repo = Substitute.For<IAuditLogRepository>();
+        repo.GetKpiSnapshotAsync(Arg.Any<TimeSpan>(), Arg.Any<DateTime?>(), Arg.Any<CancellationToken>())
+            .Returns(snapshot);
+
+        var sut = new AuditLogKpiSampleSource(repo, backlogProvider: null);
+
+        var samples = await sut.CollectAsync(CapturedAt);
+
+        AssertMetric(samples, "backlogTotal", 7);
+    }
+
     private static void AssertMetric(
         IReadOnlyList<Commons.Entities.Kpi.KpiSample> samples,
         string metric,

tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Site/SqliteAuditWriterWriteTests.cs

@@ -590,6 +590,44 @@ public class SqliteAuditWriterWriteTests
         }
     }
 
+    /// <summary>
+    /// P8 (arch-review 04): a drain batch larger than SQLite's bound-parameter ceiling
+    /// (999 on older builds) must still flip every row. The writer chunks the id list
+    /// into ≤500-id UPDATEs inside one transaction, so 1200 ids in a single
+    /// <see cref="SqliteAuditWriter.MarkForwardedAsync"/> call flip atomically — and
+    /// <see cref="SqliteAuditWriter.MarkReconciledAsync"/> chunks the same way.
+    /// </summary>
+    [Fact]
+    public async Task MarkForwardedThenReconciled_LargeBatch_ChunksParameters_AllFlip()
+    {
+        var (writer, dataSource) = CreateWriter(nameof(MarkForwardedThenReconciled_LargeBatch_ChunksParameters_AllFlip));
+        await using var _ = writer;
+
+        const int count = 1200; // > 999 and > the 500 chunk size, so multiple chunks fire
+        var ids = new List<Guid>(count);
+        for (int i = 0; i < count; i++)
+        {
+            var id = Guid.NewGuid();
+            ids.Add(id);
+            await writer.WriteAsync(NewEvent(id));
+        }
+
+        // One MarkForwardedAsync call over all 1200 ids — must not throw on the
+        // parameter ceiling and must flip every sidecar row to Forwarded.
+        await writer.MarkForwardedAsync(ids);
+
+        var afterForward = ForwardStateCounts(dataSource);
+        Assert.Equal(count, afterForward[AuditForwardState.Forwarded.ToString()]);
+        Assert.False(afterForward.ContainsKey(AuditForwardState.Pending.ToString()));
+
+        // Same for reconciliation over the full batch.
+        await writer.MarkReconciledAsync(ids);
+
+        var afterReconcile = ForwardStateCounts(dataSource);
+        Assert.Equal(count, afterReconcile[AuditForwardState.Reconciled.ToString()]);
+        Assert.False(afterReconcile.ContainsKey(AuditForwardState.Forwarded.ToString()));
+    }
+
     // ----- ExecutionId (rides DetailsJson, recomposed via AsRow) ----- //
 
     [Fact]

tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Types/Enums/AuditEnumTests.cs

@@ -23,7 +23,7 @@ public class AuditEnumTests
     }
 
     [Fact]
-    public void AuditKind_HasExactlyFourteenExpectedMembers()
+    public void AuditKind_HasExactlyFifteenExpectedMembers()
     {
         var expected = new[]
         {
@@ -31,13 +31,14 @@ public class AuditEnumTests
             "NotifySend", "NotifyDeliver", "InboundRequest", "InboundAuthFailure",
             "CachedSubmit", "CachedResolve",
             "SecuredWriteSubmit", "SecuredWriteApprove", "SecuredWriteReject", "SecuredWriteExecute",
+            "ReconciliationAbandoned",
         };
         var actual = Enum.GetValues(typeof(AuditKind))
             .Cast<AuditKind>()
             .Select(x => x.ToString())
             .ToArray();
 
-        Assert.Equal(14, actual.Length);
+        Assert.Equal(15, actual.Length);
         Assert.Equal(expected, actual);
     }
 

tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Types/PurgeTimerScheduleTests.cs

@@ -0,0 +1,40 @@
+using ZB.MOM.WW.ScadaBridge.Commons.Types;
+
+namespace ZB.MOM.WW.ScadaBridge.Commons.Tests.Types;
+
+/// <summary>
+/// Covers the shared purge-timer initial-delay rule (arch-review 04, S9): the
+/// three central purge singletons must NOT wait a full (up to 24 h) interval
+/// before their first tick, or a node recycled daily would never purge. The
+/// initial delay is min(interval, 5 min).
+/// </summary>
+public class PurgeTimerScheduleTests
+{
+    [Fact]
+    public void InitialDelay_LongInterval_IsCappedAtShortFirstTick()
+    {
+        // A daily (24 h) cadence must still fire its first purge within 5 minutes.
+        Assert.Equal(PurgeTimerSchedule.ShortFirstTick, PurgeTimerSchedule.InitialDelay(TimeSpan.FromHours(24)));
+    }
+
+    [Fact]
+    public void InitialDelay_IntervalShorterThanShortFirstTick_IsUnchanged()
+    {
+        // A millisecond test cadence must be preserved verbatim (min picks the interval),
+        // so the existing fast-tick actor suites keep their sub-second timing.
+        var fast = TimeSpan.FromMilliseconds(100);
+        Assert.Equal(fast, PurgeTimerSchedule.InitialDelay(fast));
+    }
+
+    [Fact]
+    public void InitialDelay_IntervalEqualToShortFirstTick_IsThatValue()
+    {
+        Assert.Equal(PurgeTimerSchedule.ShortFirstTick, PurgeTimerSchedule.InitialDelay(PurgeTimerSchedule.ShortFirstTick));
+    }
+
+    [Fact]
+    public void ShortFirstTick_IsFiveMinutes()
+    {
+        Assert.Equal(TimeSpan.FromMinutes(5), PurgeTimerSchedule.ShortFirstTick);
+    }
+}

tests/ZB.MOM.WW.ScadaBridge.Communication.Tests/SiteStreamPullSiteCallsTests.cs

@@ -72,7 +72,7 @@ public class SiteStreamPullSiteCallsTests : TestKit
     {
         var store = Substitute.For<IOperationTrackingStore>();
         var rows = Enumerable.Range(0, 5).Select(_ => NewOperational()).ToList();
-        store.ReadChangedSinceAsync(Arg.Any<DateTime>(), Arg.Any<int>(), Arg.Any<CancellationToken>())
+        store.ReadChangedSinceAsync(Arg.Any<DateTime>(), Arg.Any<int>(), Arg.Any<string?>(), Arg.Any<CancellationToken>())
             .Returns((IReadOnlyList<SiteCallOperational>)rows);
 
         var server = CreateServer();
@@ -97,7 +97,7 @@ public class SiteStreamPullSiteCallsTests : TestKit
     {
         var store = Substitute.For<IOperationTrackingStore>();
         var capturedSince = DateTime.MinValue;
-        store.ReadChangedSinceAsync(Arg.Any<DateTime>(), Arg.Any<int>(), Arg.Any<CancellationToken>())
+        store.ReadChangedSinceAsync(Arg.Any<DateTime>(), Arg.Any<int>(), Arg.Any<string?>(), Arg.Any<CancellationToken>())
             .Returns(call =>
             {
                 capturedSince = call.ArgAt<DateTime>(0);
@@ -121,6 +121,65 @@ public class SiteStreamPullSiteCallsTests : TestKit
         Assert.Equal(since, capturedSince);
     }
 
+    [Fact]
+    public async Task PullSiteCalls_AfterIdSet_PassesCursorThroughToStore()
+    {
+        // Task 15: a non-empty after_id is forwarded verbatim to the store as the
+        // composite-keyset cursor.
+        var store = Substitute.For<IOperationTrackingStore>();
+        string? capturedAfterId = "SENTINEL";
+        store.ReadChangedSinceAsync(Arg.Any<DateTime>(), Arg.Any<int>(), Arg.Any<string?>(), Arg.Any<CancellationToken>())
+            .Returns(call =>
+            {
+                capturedAfterId = call.ArgAt<string?>(2);
+                return (IReadOnlyList<SiteCallOperational>)Array.Empty<SiteCallOperational>();
+            });
+
+        var server = CreateServer();
+        server.SetOperationTrackingStore(store);
+
+        var cursor = Guid.NewGuid().ToString("D");
+        var request = new PullSiteCallsRequest
+        {
+            SinceUtc = Timestamp.FromDateTime(DateTime.UtcNow.AddHours(-1)),
+            BatchSize = 50,
+            AfterId = cursor,
+        };
+
+        await server.PullSiteCalls(request, NewContext());
+
+        Assert.Equal(cursor, capturedAfterId);
+    }
+
+    [Fact]
+    public async Task PullSiteCalls_AfterIdUnset_PassesNullToStore()
+    {
+        // proto3 defaults an unset string to ""; the handler must normalise that to
+        // null so the store keeps its legacy inclusive >= behaviour (older central).
+        var store = Substitute.For<IOperationTrackingStore>();
+        string? capturedAfterId = "SENTINEL";
+        store.ReadChangedSinceAsync(Arg.Any<DateTime>(), Arg.Any<int>(), Arg.Any<string?>(), Arg.Any<CancellationToken>())
+            .Returns(call =>
+            {
+                capturedAfterId = call.ArgAt<string?>(2);
+                return (IReadOnlyList<SiteCallOperational>)Array.Empty<SiteCallOperational>();
+            });
+
+        var server = CreateServer();
+        server.SetOperationTrackingStore(store);
+
+        var request = new PullSiteCallsRequest
+        {
+            SinceUtc = Timestamp.FromDateTime(DateTime.UtcNow.AddHours(-1)),
+            BatchSize = 50,
+            // AfterId left unset → "" on the wire.
+        };
+
+        await server.PullSiteCalls(request, NewContext());
+
+        Assert.Null(capturedAfterId);
+    }
+
     [Fact]
     public async Task PullSiteCalls_SinceUtcUnset_PassesDateTimeMinValue()
     {
@@ -130,7 +189,7 @@ public class SiteStreamPullSiteCallsTests : TestKit
         // without a null-deref — this proves the very first cycle doesn't crash.
         var store = Substitute.For<IOperationTrackingStore>();
         var captured = new DateTime(2099, 1, 1, 0, 0, 0, DateTimeKind.Utc); // sentinel
-        store.ReadChangedSinceAsync(Arg.Any<DateTime>(), Arg.Any<int>(), Arg.Any<CancellationToken>())
+        store.ReadChangedSinceAsync(Arg.Any<DateTime>(), Arg.Any<int>(), Arg.Any<string?>(), Arg.Any<CancellationToken>())
             .Returns(call =>
             {
                 captured = call.ArgAt<DateTime>(0);
@@ -158,7 +217,7 @@ public class SiteStreamPullSiteCallsTests : TestKit
     {
         var store = Substitute.For<IOperationTrackingStore>();
         var rows = Enumerable.Range(0, 3).Select(_ => NewOperational()).ToList();
-        store.ReadChangedSinceAsync(Arg.Any<DateTime>(), Arg.Any<int>(), Arg.Any<CancellationToken>())
+        store.ReadChangedSinceAsync(Arg.Any<DateTime>(), Arg.Any<int>(), Arg.Any<string?>(), Arg.Any<CancellationToken>())
             .Returns((IReadOnlyList<SiteCallOperational>)rows);
 
         var server = CreateServer();
@@ -200,7 +259,7 @@ public class SiteStreamPullSiteCallsTests : TestKit
     {
         // Best-effort: a read fault must never abort the reconciliation tick.
         var store = Substitute.For<IOperationTrackingStore>();
-        store.ReadChangedSinceAsync(Arg.Any<DateTime>(), Arg.Any<int>(), Arg.Any<CancellationToken>())
+        store.ReadChangedSinceAsync(Arg.Any<DateTime>(), Arg.Any<int>(), Arg.Any<string?>(), Arg.Any<CancellationToken>())
             .ThrowsAsync(new InvalidOperationException("SQLite disposed mid-call"));
 
         var server = CreateServer();

tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/NotificationOutboxRepositoryKpiQueryShapeTests.cs

@@ -0,0 +1,112 @@
+using System.Data.Common;
+using Microsoft.EntityFrameworkCore.Diagnostics;
+using ZB.MOM.WW.ScadaBridge.Commons.Entities.Notifications;
+using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
+using ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Repositories;
+
+namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests;
+
+/// <summary>
+/// Query-shape coverage for the Notification Outbox KPI snapshots (arch-review 04,
+/// Task 20). Each snapshot must be a single-query conditional aggregation with a
+/// bounded oldest-pending — the old shape issued 5-7 sequential COUNT round trips
+/// and materialized the whole live queue to find the oldest row.
+/// </summary>
+public class NotificationOutboxRepositoryKpiQueryShapeTests
+{
+    private static Notification NewNotification(
+        string sourceSiteId, string? sourceNode, NotificationStatus status,
+        DateTimeOffset createdAt, DateTimeOffset? deliveredAt = null) =>
+        new(Guid.NewGuid().ToString(), NotificationType.Email, "Ops List", "Subject", "Body", sourceSiteId)
+        {
+            Status = status,
+            CreatedAt = createdAt,
+            DeliveredAt = deliveredAt,
+            SourceNode = sourceNode,
+        };
+
+    [Fact]
+    public async Task AllThreeKpiSnapshots_IssueAtMostTwoQueries_AndBoundTheOldestLookup()
+    {
+        var counter = new SelectCountingInterceptor();
+        await using var ctx = SqliteTestHelper.CreateInMemoryContext(counter);
+        var now = DateTimeOffset.UtcNow;
+
+        // Three non-terminal rows for one site/node — the "live queue" the old code
+        // pulled into memory to find the oldest. All older than the stuck cutoff.
+        ctx.Notifications.Add(NewNotification("plant-a", "node-a", NotificationStatus.Pending, now.AddMinutes(-30)));
+        ctx.Notifications.Add(NewNotification("plant-a", "node-a", NotificationStatus.Retrying, now.AddMinutes(-20)));
+        ctx.Notifications.Add(NewNotification("plant-a", "node-a", NotificationStatus.Pending, now.AddMinutes(-10)));
+        await ctx.SaveChangesAsync();
+
+        var repo = new NotificationOutboxRepository(ctx);
+        var stuckCutoff = now.AddMinutes(-5);
+        var deliveredSince = now.AddMinutes(-60);
+
+        // Global: one aggregation query + one bounded oldest query.
+        counter.Reset();
+        var global = await repo.ComputeKpisAsync(stuckCutoff, deliveredSince);
+        Assert.True(counter.Count <= 2,
+            $"ComputeKpisAsync issued {counter.Count} queries against Notifications; expected <= 2");
+        // The oldest lookup must be bounded (LIMIT/TOP), never a full non-terminal SELECT.
+        Assert.Contains(counter.Commands, sql => sql.Contains("LIMIT", StringComparison.OrdinalIgnoreCase));
+        Assert.Equal(3, global.QueueDepth);
+        Assert.NotNull(global.OldestPendingAge);
+
+        // Per-site: a single grouped conditional-aggregation query (counts + MIN).
+        counter.Reset();
+        var perSite = await repo.ComputePerSiteKpisAsync(stuckCutoff, deliveredSince);
+        Assert.True(counter.Count <= 2,
+            $"ComputePerSiteKpisAsync issued {counter.Count} queries against Notifications; expected <= 2");
+        var siteA = Assert.Single(perSite);
+        Assert.Equal(3, siteA.QueueDepth);
+        Assert.NotNull(siteA.OldestPendingAge);
+
+        // Per-node: same single grouped aggregation, keyed by SourceNode.
+        counter.Reset();
+        var perNode = await repo.ComputePerNodeKpisAsync(stuckCutoff, deliveredSince);
+        Assert.True(counter.Count <= 2,
+            $"ComputePerNodeKpisAsync issued {counter.Count} queries against Notifications; expected <= 2");
+        var nodeA = Assert.Single(perNode);
+        Assert.Equal(3, nodeA.QueueDepth);
+        Assert.NotNull(nodeA.OldestPendingAge);
+    }
+
+    /// <summary>
+    /// Records the text of every reader command touching the Notifications table so a
+    /// test can count the round trips a KPI snapshot makes and inspect their shape.
+    /// </summary>
+    private sealed class SelectCountingInterceptor : DbCommandInterceptor
+    {
+        private readonly List<string> _commands = new();
+
+        public IReadOnlyList<string> Commands => _commands;
+
+        public int Count => _commands.Count;
+
+        public void Reset() => _commands.Clear();
+
+        private void Record(DbCommand command)
+        {
+            if (command.CommandText.Contains("Notifications", StringComparison.OrdinalIgnoreCase))
+            {
+                _commands.Add(command.CommandText);
+            }
+        }
+
+        public override InterceptionResult<DbDataReader> ReaderExecuting(
+            DbCommand command, CommandEventData eventData, InterceptionResult<DbDataReader> result)
+        {
+            Record(command);
+            return base.ReaderExecuting(command, eventData, result);
+        }
+
+        public override ValueTask<InterceptionResult<DbDataReader>> ReaderExecutingAsync(
+            DbCommand command, CommandEventData eventData, InterceptionResult<DbDataReader> result,
+            CancellationToken cancellationToken = default)
+        {
+            Record(command);
+            return base.ReaderExecutingAsync(command, eventData, result, cancellationToken);
+        }
+    }
+}

tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/Repositories/KpiHistoryRepositoryTests.cs

@@ -1,3 +1,5 @@
+using System.Data.Common;
+using Microsoft.EntityFrameworkCore.Diagnostics;
 using ZB.MOM.WW.ScadaBridge.Commons.Entities.Kpi;
 using ZB.MOM.WW.ScadaBridge.ConfigurationDatabase;
 using ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Repositories;
@@ -154,4 +156,79 @@ public class KpiHistoryRepositoryTests
         Assert.Equal(2, remaining.Count);
         Assert.Equal(new[] { 3d, 4d }, remaining.Select(p => p.Value).ToArray());
     }
+
+    [Fact]
+    public async Task PurgeOlderThanAsync_MultiWindow_IssuesMoreThanOneDelete_AndReturnsTotal()
+    {
+        // Rows span several days, each in its own hour-bucket, all older than the
+        // cutoff. The batched purge must slice them into per-hour DELETE windows
+        // (arch-review 04, P4) rather than one giant DELETE — so more than one
+        // DELETE statement is issued, and the return value still equals the total.
+        var counter = new DeleteCountingInterceptor();
+        await using var ctx = SqliteTestHelper.CreateInMemoryContext(counter);
+        var repo = new KpiHistoryRepository(ctx);
+
+        await repo.RecordSamplesAsync(new[]
+        {
+            // Three distinct hour-buckets strictly older than the cutoff — purged.
+            Sample("NotificationOutbox", "queueDepth", "Global", null, value: 1, capturedAtUtc: Base.AddDays(-5)),
+            Sample("NotificationOutbox", "queueDepth", "Global", null, value: 2, capturedAtUtc: Base.AddDays(-4)),
+            Sample("NotificationOutbox", "queueDepth", "Global", null, value: 3, capturedAtUtc: Base.AddDays(-3)),
+            // Newer than the cutoff — kept, and never scanned into a delete window.
+            Sample("NotificationOutbox", "queueDepth", "Global", null, value: 4, capturedAtUtc: Base.AddDays(-1)),
+            Sample("NotificationOutbox", "queueDepth", "Global", null, value: 5, capturedAtUtc: Base),
+        });
+
+        // Only count DELETEs the purge itself issues (RecordSamples above emits none).
+        counter.Reset();
+
+        var deleted = await repo.PurgeOlderThanAsync(Base.AddDays(-2));
+
+        // All three old rows gone; the batched total is the sum across windows.
+        Assert.Equal(3, deleted);
+        Assert.True(counter.DeleteCount > 1,
+            $"expected the batched purge to issue more than one DELETE, got {counter.DeleteCount}");
+
+        var remaining = await repo.GetRawSeriesAsync(
+            "NotificationOutbox", "queueDepth", "Global", scopeKey: null,
+            fromUtc: Base.AddDays(-30), toUtc: Base.AddDays(30));
+        Assert.Equal(new[] { 4d, 5d }, remaining.Select(p => p.Value).ToArray());
+    }
+
+    /// <summary>
+    /// EF command interceptor that counts the DELETE statements actually issued
+    /// to the store, so a test can prove the purge is sliced into multiple
+    /// windowed DELETEs rather than one mega-statement. Threads both the sync and
+    /// async non-query entry points (<c>ExecuteDeleteAsync</c> routes through the
+    /// async path).
+    /// </summary>
+    private sealed class DeleteCountingInterceptor : DbCommandInterceptor
+    {
+        public int DeleteCount { get; private set; }
+
+        public void Reset() => DeleteCount = 0;
+
+        private void CountIfDelete(DbCommand command)
+        {
+            if (command.CommandText.Contains("DELETE", StringComparison.OrdinalIgnoreCase))
+            {
+                DeleteCount++;
+            }
+        }
+
+        public override InterceptionResult<int> NonQueryExecuting(
+            DbCommand command, CommandEventData eventData, InterceptionResult<int> result)
+        {
+            CountIfDelete(command);
+            return base.NonQueryExecuting(command, eventData, result);
+        }
+
+        public override ValueTask<InterceptionResult<int>> NonQueryExecutingAsync(
+            DbCommand command, CommandEventData eventData, InterceptionResult<int> result,
+            CancellationToken cancellationToken = default)
+        {
+            CountIfDelete(command);
+            return base.NonQueryExecutingAsync(command, eventData, result, cancellationToken);
+        }
+    }
 }

tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/Repositories/SiteCallAuditRepositoryTests.cs

@@ -95,19 +95,23 @@ public class SiteCallAuditRepositoryTests : IClassFixture<MsSqlMigrationFixture>
     }
 
     [SkippableFact]
-    public async Task UpsertAsync_SameStatus_IsNoOp()
+    public async Task UpsertAsync_SameStatus_EqualUpdatedAt_IsNoOp()
     {
         Skip.IfNot(_fixture.Available, _fixture.SkipReason);
 
         var id = TrackedOperationId.New();
+        var stamp = new DateTime(2026, 6, 1, 9, 0, 0, DateTimeKind.Utc);
         await using var context = CreateContext();
         var repo = new SiteCallAuditRepository(context);
 
-        await repo.UpsertAsync(NewRow(id, status: "Attempted", retryCount: 1, lastError: "first"));
+        await repo.UpsertAsync(NewRow(id, status: "Attempted", retryCount: 1, lastError: "first", updatedAtUtc: stamp));
         var snapshot = await repo.GetAsync(id);
 
-        // Same rank (2) — repository must treat this as a no-op (no fields move).
-        await repo.UpsertAsync(NewRow(id, status: "Attempted", retryCount: 2, lastError: "second"));
+        // Same rank (2) AND an EQUAL UpdatedAtUtc — an idempotent replay of the
+        // same telemetry packet. The same-rank freshness tiebreaker only fires
+        // for a STRICTLY-newer stamp, so a true duplicate stays a no-op (no
+        // fields move). Task 11: newest-within-rank wins, equal stamps are inert.
+        await repo.UpsertAsync(NewRow(id, status: "Attempted", retryCount: 2, lastError: "second", updatedAtUtc: stamp));
         var afterDuplicate = await repo.GetAsync(id);
 
         Assert.NotNull(afterDuplicate);
@@ -117,21 +121,92 @@ public class SiteCallAuditRepositoryTests : IClassFixture<MsSqlMigrationFixture>
         Assert.Equal(snapshot!.UpdatedAtUtc, afterDuplicate.UpdatedAtUtc);
     }
 
+    [SkippableFact]
+    public async Task UpsertAsync_SameRank_NewerUpdatedAt_RefreshesProgressFields()
+    {
+        Skip.IfNot(_fixture.Available, _fixture.SkipReason);
+
+        var id = TrackedOperationId.New();
+        var t0 = new DateTime(2026, 6, 1, 10, 0, 0, DateTimeKind.Utc);
+        await using var context = CreateContext();
+        var repo = new SiteCallAuditRepository(context);
+
+        // Two Attempted packets (rank 2) from one retrying call; the second
+        // carries a strictly-newer UpdatedAtUtc, so its live RetryCount/LastError
+        // must land — the row no longer freezes at first-Attempted (Task 11).
+        await repo.UpsertAsync(NewRow(id, status: "Attempted", retryCount: 1, lastError: "timeout #1", updatedAtUtc: t0));
+        await repo.UpsertAsync(NewRow(id, status: "Attempted", retryCount: 5, lastError: "timeout #5", updatedAtUtc: t0.AddMinutes(5)));
+
+        var row = await repo.GetAsync(id);
+        Assert.NotNull(row);
+        Assert.Equal(5, row!.RetryCount);
+        Assert.Equal("timeout #5", row.LastError);
+        Assert.Equal(t0.AddMinutes(5), row.UpdatedAtUtc);
+    }
+
+    [SkippableFact]
+    public async Task UpsertAsync_SameRank_OlderUpdatedAt_IsNoOp()
+    {
+        Skip.IfNot(_fixture.Available, _fixture.SkipReason);
+
+        var id = TrackedOperationId.New();
+        var t0 = new DateTime(2026, 6, 1, 11, 0, 0, DateTimeKind.Utc);
+        await using var context = CreateContext();
+        var repo = new SiteCallAuditRepository(context);
+
+        // Land retry 5 (stamp t0+5), then an out-of-order retry-2 packet with an
+        // OLDER stamp (t0). Same rank, older UpdatedAtUtc → rejected; stays at 5.
+        await repo.UpsertAsync(NewRow(id, status: "Attempted", retryCount: 5, lastError: "timeout #5", updatedAtUtc: t0.AddMinutes(5)));
+        await repo.UpsertAsync(NewRow(id, status: "Attempted", retryCount: 2, lastError: "timeout #2", updatedAtUtc: t0));
+
+        var row = await repo.GetAsync(id);
+        Assert.NotNull(row);
+        Assert.Equal(5, row!.RetryCount);
+        Assert.Equal("timeout #5", row.LastError);
+        Assert.Equal(t0.AddMinutes(5), row.UpdatedAtUtc);
+    }
+
+    [SkippableFact]
+    public async Task UpsertAsync_LowerRank_NewerUpdatedAt_IsStillNoOp()
+    {
+        Skip.IfNot(_fixture.Available, _fixture.SkipReason);
+
+        var id = TrackedOperationId.New();
+        var t0 = new DateTime(2026, 6, 1, 12, 0, 0, DateTimeKind.Utc);
+        await using var context = CreateContext();
+        var repo = new SiteCallAuditRepository(context);
+
+        // Terminal Delivered (rank 3), then a late lower-rank Attempted (rank 2)
+        // with a NEWER stamp. Rank monotonicity dominates the freshness
+        // tiebreaker — the row stays Delivered (regression-proof).
+        await repo.UpsertAsync(NewRow(id, status: "Delivered", retryCount: 0, updatedAtUtc: t0, terminal: true, terminalAtUtc: t0));
+        await repo.UpsertAsync(NewRow(id, status: "Attempted", retryCount: 9, lastError: "late", updatedAtUtc: t0.AddMinutes(5)));
+
+        var row = await repo.GetAsync(id);
+        Assert.NotNull(row);
+        Assert.Equal("Delivered", row!.Status);
+        Assert.Equal(0, row.RetryCount);
+    }
+
     [SkippableFact]
     public async Task UpsertAsync_TerminalOverTerminal_IsNoOp()
     {
         Skip.IfNot(_fixture.Available, _fixture.SkipReason);
 
         // Bundle B3 plan: terminal statuses share rank 3 and are mutually
-        // exclusive — Delivered cannot overwrite Parked.
+        // exclusive — Delivered cannot overwrite Parked. Task 11 note: the
+        // Delivered packet carries a STRICTLY-NEWER UpdatedAtUtc, yet the row
+        // stays Parked — the same-rank freshness tiebreaker is scoped to
+        // NON-terminal ranks (< 3), so terminal immutability is preserved.
         var id = TrackedOperationId.New();
+        var t0 = new DateTime(2026, 6, 1, 13, 0, 0, DateTimeKind.Utc);
         await using var context = CreateContext();
         var repo = new SiteCallAuditRepository(context);
 
-        await repo.UpsertAsync(NewRow(id, status: "Parked", retryCount: 3, lastError: "parked-reason", terminal: true));
+        await repo.UpsertAsync(NewRow(id, status: "Parked", retryCount: 3, lastError: "parked-reason", updatedAtUtc: t0, terminal: true, terminalAtUtc: t0));
         var afterPark = await repo.GetAsync(id);
 
-        await repo.UpsertAsync(NewRow(id, status: "Delivered", retryCount: 4, lastError: null, terminal: true));
+        await repo.UpsertAsync(NewRow(id, status: "Delivered", retryCount: 4, lastError: null, updatedAtUtc: t0.AddMinutes(5), terminal: true, terminalAtUtc: t0.AddMinutes(5)));
         var afterDeliveredAttempt = await repo.GetAsync(id);
 
         Assert.NotNull(afterDeliveredAttempt);

tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/ServiceCollectionExtensionsTests.cs

@@ -1,4 +1,5 @@
 using System.Reflection;
+using Microsoft.EntityFrameworkCore;
 using Microsoft.Extensions.DependencyInjection;
 using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
 using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Services;
@@ -13,7 +14,7 @@ public class ServiceCollectionExtensionsTests
     {
         var services = new ServiceCollection();
 
-        services.AddConfigurationDatabase("DataSource=:memory:");
+        services.AddConfigurationDatabase("Server=unused;Database=unused;");
 
         Assert.Contains(services, d => d.ServiceType == typeof(ITemplateEngineRepository));
         Assert.Contains(services, d => d.ServiceType == typeof(IAuditService));
@@ -21,6 +22,32 @@ public class ServiceCollectionExtensionsTests
         Assert.Contains(services, d => d.ServiceType == typeof(IAuditLogRepository));
     }
 
+    [Fact]
+    public void AddConfigurationDatabase_Configures_RetryingExecutionStrategy()
+    {
+        // Task 12 (arch-review 04 S5): the central DbContext must survive transient
+        // SQL faults. EnableRetryOnFailure installs a non-null ExecutionStrategyFactory
+        // on the SqlServer options extension — covering every read-side path (KPI asks,
+        // outbox queries, Tracking.Status, execution trees) that previously surfaced
+        // transient faults raw.
+        var services = new ServiceCollection();
+        services.AddConfigurationDatabase("Server=unused;Database=unused;");
+        using var sp = services.BuildServiceProvider();
+
+        var options = sp.GetRequiredService<DbContextOptions<ScadaBridgeDbContext>>();
+
+        // EF1001: SqlServerOptionsExtension is an internal EF surface, but it is the
+        // only place the retrying-strategy factory is observable — asserting it is the
+        // point of this test. Scoped suppression, not a project-wide one.
+#pragma warning disable EF1001
+        var sqlExt = options.Extensions
+            .OfType<Microsoft.EntityFrameworkCore.SqlServer.Infrastructure.Internal.SqlServerOptionsExtension>()
+            .Single();
+
+        Assert.NotNull(sqlExt.ExecutionStrategyFactory);
+#pragma warning restore EF1001
+    }
+
     // The no-arg overload is [Obsolete(error: true)], so it cannot be referenced directly
     // from source — that is the compile-time guard. Invoke it via reflection to verify the
     // runtime defence-in-depth behaviour.

tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/SqliteTestHelper.cs

@@ -87,6 +87,25 @@ public static class SqliteTestHelper
         return context;
     }
 
+    /// <summary>
+    /// In-memory SQLite context with EF command interceptors attached — lets a
+    /// test observe the actual SQL a repository emits (e.g. count DELETE statements
+    /// issued by a batched purge).
+    /// </summary>
+    public static ScadaBridgeDbContext CreateInMemoryContext(params IInterceptor[] interceptors)
+    {
+        var options = new DbContextOptionsBuilder<ScadaBridgeDbContext>()
+            .UseSqlite("DataSource=:memory:")
+            .ConfigureWarnings(w => w.Ignore(RelationalEventId.PendingModelChangesWarning))
+            .AddInterceptors(interceptors)
+            .Options;
+
+        var context = new SqliteTestDbContext(options);
+        context.Database.OpenConnection();
+        context.Database.EnsureCreated();
+        return context;
+    }
+
     public static ScadaBridgeDbContext CreateFileContext(string dbPath)
     {
         var options = new DbContextOptionsBuilder<ScadaBridgeDbContext>()

tests/ZB.MOM.WW.ScadaBridge.HealthMonitoring.Tests/Kpi/CentralHealthAuditBacklogProviderTests.cs

@@ -0,0 +1,100 @@
+using ZB.MOM.WW.ScadaBridge.Commons.Messages.Health;
+using ZB.MOM.WW.ScadaBridge.Commons.Types;
+using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
+using ZB.MOM.WW.ScadaBridge.HealthMonitoring.Kpi;
+
+namespace ZB.MOM.WW.ScadaBridge.HealthMonitoring.Tests.Kpi;
+
+/// <summary>
+/// Task 10 (arch-review 04) coverage for <see cref="CentralHealthAuditBacklogProvider"/>.
+/// The provider sums the latest per-site <c>SiteAuditBacklog.PendingCount</c> across the
+/// in-memory <see cref="ICentralHealthAggregator"/> — the same aggregate the live Audit
+/// backlog tile shows — so the <c>backlogTotal</c> KPI history stops recording a hardwired
+/// zero. Sites with no report or a null backlog snapshot contribute nothing.
+/// </summary>
+public class CentralHealthAuditBacklogProviderTests
+{
+    private static readonly DateTime At = new(2026, 6, 17, 12, 0, 0, DateTimeKind.Utc);
+
+    [Fact]
+    public void GetPendingBacklogTotal_Sums_PendingCount_Across_Sites()
+    {
+        var aggregator = new StubAggregator
+        {
+            States =
+            {
+                ["site-a"] = StateWithBacklog("site-a", 9),
+                ["site-b"] = StateWithBacklog("site-b", 4),
+                ["site-c"] = new SiteHealthState { SiteId = "site-c", LatestReport = null }, // heartbeat-only
+            },
+        };
+
+        var provider = new CentralHealthAuditBacklogProvider(aggregator);
+
+        // 9 + 4; the null-report site contributes nothing.
+        Assert.Equal(13, provider.GetPendingBacklogTotal());
+    }
+
+    [Fact]
+    public void GetPendingBacklogTotal_Returns_Zero_When_No_Backlog_Reported()
+    {
+        var aggregator = new StubAggregator
+        {
+            States =
+            {
+                ["site-a"] = new SiteHealthState { SiteId = "site-a", LatestReport = MinimalReport("site-a") with { SiteAuditBacklog = null } },
+                ["site-b"] = new SiteHealthState { SiteId = "site-b", LatestReport = null },
+            },
+        };
+
+        var provider = new CentralHealthAuditBacklogProvider(aggregator);
+
+        Assert.Equal(0, provider.GetPendingBacklogTotal());
+    }
+
+    private static SiteHealthState StateWithBacklog(string siteId, int pending) =>
+        new()
+        {
+            SiteId = siteId,
+            LatestReport = MinimalReport(siteId) with
+            {
+                SiteAuditBacklog = new SiteAuditBacklogSnapshot(
+                    PendingCount: pending, OldestPendingUtc: null, OnDiskBytes: 0),
+            },
+        };
+
+    private static SiteHealthReport MinimalReport(string siteId) =>
+        new(
+            SiteId: siteId,
+            SequenceNumber: 1,
+            ReportTimestamp: At,
+            DataConnectionStatuses: new Dictionary<string, ConnectionHealth>(),
+            TagResolutionCounts: new Dictionary<string, TagResolutionStatus>(),
+            ScriptErrorCount: 0,
+            AlarmEvaluationErrorCount: 0,
+            StoreAndForwardBufferDepths: new Dictionary<string, int>(),
+            DeadLetterCount: 0,
+            DeployedInstanceCount: 0,
+            EnabledInstanceCount: 0,
+            DisabledInstanceCount: 0);
+
+    /// <summary>
+    /// Hand-rolled <see cref="ICentralHealthAggregator"/> stub — the HealthMonitoring.Tests
+    /// project has no mocking library. Only <see cref="GetAllSiteStates"/> is exercised.
+    /// </summary>
+    private sealed class StubAggregator : ICentralHealthAggregator
+    {
+        public Dictionary<string, SiteHealthState> States { get; } = new();
+
+        public IReadOnlyDictionary<string, SiteHealthState> GetAllSiteStates() => States;
+
+        public SiteHealthState? GetSiteState(string siteId) =>
+            States.TryGetValue(siteId, out var state) ? state : null;
+
+        public void ProcessReport(SiteHealthReport report) => throw new NotSupportedException();
+
+        public void MarkHeartbeat(string siteId, DateTimeOffset receivedAt) => throw new NotSupportedException();
+
+        public void PruneUnknownSites(IReadOnlyCollection<string> knownSiteIds) => throw new NotSupportedException();
+    }
+}

tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests/Kpi/NotificationOutboxKpiSampleSourceTests.cs

@@ -33,6 +33,20 @@ public class NotificationOutboxKpiSampleSourceTests
         Assert.Equal(KpiSources.NotificationOutbox, source.Source);
     }
 
+    // C4 (arch-review 04): the two metric names that were inline string literals now
+    // live in the shared KpiMetrics catalog. Their VALUES are persisted in the KpiSample
+    // store and rendered by the trend UI, so a rename would silently orphan history —
+    // pin the exact historical strings here.
+    [Fact]
+    public void KpiMetricCatalog_NotificationOutbox_PreservesHistoricalStringValues()
+    {
+        Assert.Equal("queueDepth", KpiMetrics.NotificationOutbox.QueueDepth);
+        Assert.Equal("stuckCount", KpiMetrics.NotificationOutbox.StuckCount);
+        Assert.Equal("parkedCount", KpiMetrics.NotificationOutbox.ParkedCount);
+        Assert.Equal("deliveredLastInterval", KpiMetrics.NotificationOutbox.DeliveredLastInterval);
+        Assert.Equal("oldestPendingAgeSeconds", KpiMetrics.NotificationOutbox.OldestPendingAgeSeconds);
+    }
+
     [Fact]
     public async Task CollectAsync_PassesCutoffsAnchoredOnCapturedAt()
     {

tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests/NotificationOutboxActorDispatchTests.cs

@@ -567,4 +567,121 @@ public class NotificationOutboxActorDispatchTests : TestKit
                 Arg.Any<DateTimeOffset>(), Arg.Any<int>(), Arg.Any<CancellationToken>()),
             duration: TimeSpan.FromSeconds(2));
     }
+
+    // ── Task 25: bounded parallel delivery ───────────────────────────────────
+
+    /// <summary>
+    /// Adapter that records the concurrent-invocation high-water mark. Each
+    /// <see cref="DeliverAsync"/> increments a live counter, notes the peak, then blocks
+    /// on a shared gate until <paramref name="releaseWhen"/> deliveries are concurrently
+    /// in flight — proving real overlap. A safety timeout releases the gate so a
+    /// parallelism-1 run (which can never reach <c>releaseWhen</c> concurrently) still
+    /// drains instead of deadlocking.
+    /// </summary>
+    private sealed class ConcurrencyHighWaterAdapter : INotificationDeliveryAdapter
+    {
+        private readonly int _releaseWhen;
+        private readonly TaskCompletionSource _gate =
+            new(TaskCreationOptions.RunContinuationsAsynchronously);
+        private int _current;
+        private int _highWater;
+        private int _entered;
+
+        public ConcurrencyHighWaterAdapter(int releaseWhen) => _releaseWhen = releaseWhen;
+
+        public NotificationType Type => NotificationType.Email;
+
+        public int HighWater => Volatile.Read(ref _highWater);
+
+        public async Task<DeliveryOutcome> DeliverAsync(
+            Notification notification, CancellationToken cancellationToken = default)
+        {
+            var current = Interlocked.Increment(ref _current);
+            int hw;
+            while ((hw = Volatile.Read(ref _highWater)) < current
+                && Interlocked.CompareExchange(ref _highWater, current, hw) != hw)
+            {
+                // retry until our peak is recorded
+            }
+
+            if (Interlocked.Increment(ref _entered) >= _releaseWhen)
+            {
+                _gate.TrySetResult();
+            }
+
+            // Overlap window: wait until enough deliveries are concurrently in flight,
+            // or a short safety timeout (so the strictly-sequential config still drains).
+            await Task.WhenAny(_gate.Task, Task.Delay(TimeSpan.FromMilliseconds(400), cancellationToken))
+                .ConfigureAwait(false);
+
+            Interlocked.Decrement(ref _current);
+            return DeliveryOutcome.Success("ops@example.com");
+        }
+    }
+
+    private void SetupDueBatchOnce(int count)
+    {
+        var batch = Enumerable.Range(0, count).Select(_ => MakeNotification()).ToArray();
+        var claimed = 0;
+        _outboxRepository.GetDueAsync(Arg.Any<DateTimeOffset>(), Arg.Any<int>(), Arg.Any<CancellationToken>())
+            .Returns(_ => Interlocked.Increment(ref claimed) == 1
+                ? batch
+                : Array.Empty<Notification>());
+    }
+
+    [Fact]
+    public void Dispatch_WithMaxParallelFour_DeliversConcurrently_AndAllReachTerminal()
+    {
+        SetupSmtpRetryPolicy(maxRetries: 5, retryDelay: TimeSpan.FromMinutes(1));
+        SetupDueBatchOnce(8);
+        var adapter = new ConcurrencyHighWaterAdapter(releaseWhen: 2);
+        var actor = CreateActor(
+            [adapter],
+            new NotificationOutboxOptions
+            {
+                DispatchInterval = TimeSpan.FromHours(1),
+                MaxParallelDeliveries = 4,
+            });
+
+        actor.Tell(InternalMessages.DispatchTick.Instance);
+
+        AwaitAssert(
+            () =>
+            {
+                // All 8 notifications reached a terminal Delivered update.
+                _outboxRepository.Received(8).UpdateAsync(
+                    Arg.Is<Notification>(n => n.Status == NotificationStatus.Delivered),
+                    Arg.Any<CancellationToken>());
+                // At least two deliveries overlapped — the sweep is no longer sequential.
+                Assert.True(adapter.HighWater >= 2,
+                    $"expected concurrent deliveries (high-water >= 2), got {adapter.HighWater}");
+            },
+            duration: TimeSpan.FromSeconds(10));
+    }
+
+    [Fact]
+    public void Dispatch_WithMaxParallelOne_DeliversSequentially_AndAllReachTerminal()
+    {
+        SetupSmtpRetryPolicy(maxRetries: 5, retryDelay: TimeSpan.FromMinutes(1));
+        SetupDueBatchOnce(8);
+        var adapter = new ConcurrencyHighWaterAdapter(releaseWhen: 2);
+        var actor = CreateActor(
+            [adapter],
+            new NotificationOutboxOptions
+            {
+                DispatchInterval = TimeSpan.FromHours(1),
+                MaxParallelDeliveries = 1,
+            });
+
+        actor.Tell(InternalMessages.DispatchTick.Instance);
+
+        AwaitAssert(
+            () => _outboxRepository.Received(8).UpdateAsync(
+                Arg.Is<Notification>(n => n.Status == NotificationStatus.Delivered),
+                Arg.Any<CancellationToken>()),
+            duration: TimeSpan.FromSeconds(10));
+
+        // Strictly sequential: never more than one delivery in flight at once.
+        Assert.Equal(1, adapter.HighWater);
+    }
 }

tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests/NotificationOutboxActorRetryEmissionTests.cs

@@ -0,0 +1,155 @@
+using Akka.Actor;
+using Akka.TestKit.Xunit2;
+using Microsoft.Extensions.DependencyInjection;
+using Microsoft.Extensions.Logging.Abstractions;
+using NSubstitute;
+using ZB.MOM.WW.Audit;
+using ZB.MOM.WW.ScadaBridge.Commons.Entities.Notifications;
+using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
+using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Services;
+using ZB.MOM.WW.ScadaBridge.Commons.Messages.Notification;
+using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
+using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
+using ZB.MOM.WW.ScadaBridge.NotificationOutbox.Delivery;
+
+namespace ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests;
+
+/// <summary>
+/// Task 13 (arch-review 04) — verifies that an operator-initiated Retry on a parked
+/// notification emits a <see cref="AuditChannel.Notification"/>/<see cref="AuditKind.NotifyDeliver"/>
+/// audit row with status <see cref="AuditStatus.Submitted"/> and the operator as
+/// <c>Actor</c>, closing the forensic gap (previously the lifecycle read
+/// <c>Parked → Attempted → Delivered</c> with no record of who un-parked it). Also
+/// asserts the audit write is best-effort — a throwing writer never fails the retry.
+/// </summary>
+public class NotificationOutboxActorRetryEmissionTests : TestKit
+{
+    private readonly INotificationOutboxRepository _outboxRepository =
+        Substitute.For<INotificationOutboxRepository>();
+
+    private readonly RecordingCentralAuditWriter _auditWriter = new();
+
+    private sealed class RecordingCentralAuditWriter : ICentralAuditWriter
+    {
+        public List<AuditRowProjection.AuditRowValues> Events { get; } = new();
+        public Func<AuditEvent, Task>? OnWrite { get; set; }
+
+        public Task WriteAsync(AuditEvent evt, CancellationToken ct = default)
+        {
+            lock (Events)
+            {
+                Events.Add(evt.AsRow());
+            }
+
+            return OnWrite?.Invoke(evt) ?? Task.CompletedTask;
+        }
+    }
+
+    private IServiceProvider BuildServiceProvider()
+    {
+        var services = new ServiceCollection();
+        services.AddScoped(_ => _outboxRepository);
+        return services.BuildServiceProvider();
+    }
+
+    private IActorRef CreateActor()
+    {
+        return Sys.ActorOf(Props.Create(() => new NotificationOutboxActor(
+            BuildServiceProvider(),
+            new NotificationOutboxOptions { DispatchInterval = TimeSpan.FromHours(1) },
+            (ICentralAuditWriter)_auditWriter,
+            NullLogger<NotificationOutboxActor>.Instance)));
+    }
+
+    private static Notification MakeParked(Guid? notificationId = null)
+    {
+        return new Notification(
+            (notificationId ?? Guid.NewGuid()).ToString("D"),
+            NotificationType.Email,
+            "ops-team",
+            "Tank overflow",
+            "Tank 3 level critical",
+            "site-1")
+        {
+            Status = NotificationStatus.Parked,
+            RetryCount = 5,
+            CreatedAt = DateTimeOffset.UtcNow,
+        };
+    }
+
+    private List<AuditRowProjection.AuditRowValues> EventsByStatus(AuditStatus status)
+    {
+        lock (_auditWriter.Events)
+        {
+            return _auditWriter.Events.Where(e => e.Status == status).ToList();
+        }
+    }
+
+    [Fact]
+    public void Retry_OnParked_Emits_Submitted_NotifyDeliver_AuditRow_With_Operator()
+    {
+        var notificationId = Guid.NewGuid();
+        var notification = MakeParked(notificationId);
+        _outboxRepository.GetByIdAsync(notification.NotificationId, Arg.Any<CancellationToken>())
+            .Returns(notification);
+        var actor = CreateActor();
+
+        actor.Tell(new RetryNotificationRequest(
+            CorrelationId: "test-corr", NotificationId: notification.NotificationId, RequestedBy: "jdoe"));
+
+        ExpectMsg<RetryNotificationResponse>(r => r.Success);
+        AwaitAssert(() =>
+        {
+            var submitted = EventsByStatus(AuditStatus.Submitted);
+            Assert.Single(submitted);
+            var evt = submitted[0];
+            Assert.Equal(AuditChannel.Notification, evt.Channel);
+            Assert.Equal(AuditKind.NotifyDeliver, evt.Kind);
+            Assert.Equal("jdoe", evt.Actor);
+            Assert.Equal(notificationId, evt.CorrelationId);
+        });
+    }
+
+    [Fact]
+    public void Discard_OnParked_Emits_Discarded_AuditRow_With_Operator()
+    {
+        var notification = MakeParked();
+        _outboxRepository.GetByIdAsync(notification.NotificationId, Arg.Any<CancellationToken>())
+            .Returns(notification);
+        var actor = CreateActor();
+
+        actor.Tell(new DiscardNotificationRequest(
+            CorrelationId: "test-corr", NotificationId: notification.NotificationId, RequestedBy: "jdoe"));
+
+        ExpectMsg<DiscardNotificationResponse>(r => r.Success);
+        AwaitAssert(() =>
+        {
+            var discarded = EventsByStatus(AuditStatus.Discarded);
+            Assert.Single(discarded);
+            Assert.Equal("jdoe", discarded[0].Actor);
+        });
+    }
+
+    [Fact]
+    public void Retry_AuditFailure_DoesNotFail_TheRetry()
+    {
+        // Audit is best-effort: a throwing writer must NOT abort the retry — the
+        // row is still flipped back to Pending and the response reports Success.
+        var notification = MakeParked();
+        _outboxRepository.GetByIdAsync(notification.NotificationId, Arg.Any<CancellationToken>())
+            .Returns(notification);
+        _auditWriter.OnWrite = _ => throw new InvalidOperationException("audit dead");
+        var actor = CreateActor();
+
+        actor.Tell(new RetryNotificationRequest(
+            CorrelationId: "test-corr", NotificationId: notification.NotificationId, RequestedBy: "jdoe"));
+
+        ExpectMsg<RetryNotificationResponse>(r => r.Success);
+        AwaitAssert(() =>
+        {
+            _outboxRepository.Received(1).UpdateAsync(
+                Arg.Is<Notification>(n => n.Status == NotificationStatus.Pending),
+                Arg.Any<CancellationToken>());
+        });
+    }
+}

tests/ZB.MOM.WW.ScadaBridge.SiteCallAudit.Tests/SiteCallAuditPurgeTests.cs

@@ -42,7 +42,7 @@ public class SiteCallAuditPurgeTests : TestKit
     private sealed class NoOpPullClient : IPullSiteCallsClient
     {
         public Task<PullSiteCallsResponse> PullAsync(
-            string siteId, DateTime sinceUtc, int batchSize, CancellationToken ct) =>
+            string siteId, DateTime sinceUtc, string? afterId, int batchSize, CancellationToken ct) =>
             Task.FromResult(new PullSiteCallsResponse(Array.Empty<SiteCall>(), MoreAvailable: false));
     }
 

tests/ZB.MOM.WW.ScadaBridge.SiteCallAudit.Tests/SiteCallAuditReconciliationTests.cs

@@ -74,7 +74,7 @@ public class SiteCallAuditReconciliationTests : TestKit
     /// </summary>
     private sealed class ScriptedPullClient : IPullSiteCallsClient
     {
-        public List<(string SiteId, DateTime SinceUtc, int BatchSize)> Calls { get; } = new();
+        public List<(string SiteId, DateTime SinceUtc, string? AfterId, int BatchSize)> Calls { get; } = new();
         private readonly Dictionary<string, Queue<PullSiteCallsResponse>> _scripted = new();
         private readonly Dictionary<string, Exception> _throwOnSite = new();
 
@@ -91,9 +91,9 @@ public class SiteCallAuditReconciliationTests : TestKit
         }
 
         public Task<PullSiteCallsResponse> PullAsync(
-            string siteId, DateTime sinceUtc, int batchSize, CancellationToken ct)
+            string siteId, DateTime sinceUtc, string? afterId, int batchSize, CancellationToken ct)
         {
-            Calls.Add((siteId, sinceUtc, batchSize));
+            Calls.Add((siteId, sinceUtc, afterId, batchSize));
             if (_throwOnSite.TryGetValue(siteId, out var ex))
             {
                 throw ex;
@@ -107,6 +107,67 @@ public class SiteCallAuditReconciliationTests : TestKit
         }
     }
 
+    /// <summary>
+    /// Keyset-honouring in-memory pull client (Task 16). Holds one static set of
+    /// rows and answers each pull as a real site would: returns the next
+    /// <c>batchSize</c> rows strictly greater than the composite
+    /// <c>(sinceUtc, afterId)</c> cursor under the deterministic
+    /// <c>(UpdatedAtUtc, TrackedOperationId ordinal)</c> ordering, with
+    /// <c>MoreAvailable=true</c> when unseen rows remain. This is the modern site
+    /// that DOES honour <c>after_id</c>, so a burst sharing one exact
+    /// <c>UpdatedAtUtc</c> drains fully via the id tiebreak — no single-timestamp
+    /// pin.
+    /// </summary>
+    private sealed class KeysetHonoringPullClient : IPullSiteCallsClient
+    {
+        private readonly IReadOnlyList<SiteCall> _rows;
+        public int CallCount { get; private set; }
+
+        public KeysetHonoringPullClient(IEnumerable<SiteCall> rows) =>
+            _rows = rows
+                .OrderBy(r => r.UpdatedAtUtc)
+                .ThenBy(r => r.TrackedOperationId.ToString(), StringComparer.Ordinal)
+                .ToList();
+
+        public Task<PullSiteCallsResponse> PullAsync(
+            string siteId, DateTime sinceUtc, string? afterId, int batchSize, CancellationToken ct)
+        {
+            CallCount++;
+            var greater = _rows
+                .Where(r =>
+                    r.UpdatedAtUtc > sinceUtc ||
+                    (r.UpdatedAtUtc == sinceUtc &&
+                     string.CompareOrdinal(r.TrackedOperationId.ToString(), afterId) > 0))
+                .ToList();
+            var page = greater.Take(batchSize).ToList();
+            var more = greater.Count > page.Count;
+            return Task.FromResult(
+                new PullSiteCallsResponse(page, MoreAvailable: more));
+        }
+    }
+
+    /// <summary>
+    /// Legacy pull client (Task 16) that IGNORES <c>afterId</c> and always
+    /// returns the same saturated page (<c>MoreAvailable=true</c>) — models a
+    /// pre-keyset site the central actor cannot drain past. The composite cursor
+    /// cannot advance, so the actor must latch and publish
+    /// <see cref="SiteCallReconciliationPinnedChanged"/> instead of spinning.
+    /// </summary>
+    private sealed class LegacyIgnoresAfterIdPullClient : IPullSiteCallsClient
+    {
+        private readonly IReadOnlyList<SiteCall> _rows;
+        public int CallCount { get; private set; }
+
+        public LegacyIgnoresAfterIdPullClient(IReadOnlyList<SiteCall> rows) => _rows = rows;
+
+        public Task<PullSiteCallsResponse> PullAsync(
+            string siteId, DateTime sinceUtc, string? afterId, int batchSize, CancellationToken ct)
+        {
+            CallCount++;
+            return Task.FromResult(new PullSiteCallsResponse(_rows, MoreAvailable: true));
+        }
+    }
+
     /// <summary>
     /// Pull client that ALWAYS returns the same saturated response
     /// (<c>MoreAvailable=true</c>) regardless of the <c>since</c> cursor —
@@ -124,7 +185,7 @@ public class SiteCallAuditReconciliationTests : TestKit
         public SaturatedPinPullClient(IReadOnlyList<SiteCall> rows) => _rows = rows;
 
         public Task<PullSiteCallsResponse> PullAsync(
-            string siteId, DateTime sinceUtc, int batchSize, CancellationToken ct)
+            string siteId, DateTime sinceUtc, string? afterId, int batchSize, CancellationToken ct)
         {
             CallCount++;
             return Task.FromResult(new PullSiteCallsResponse(_rows, MoreAvailable: true));
@@ -436,4 +497,89 @@ public class SiteCallAuditReconciliationTests : TestKit
             $"a single-timestamp saturation pin must break the within-tick drain, not spin to the "
             + $"page ceiling; got {client.CallCount} pulls (an unbounded within-tick loop would be 50+)");
     }
+
+    // ---------------------------------------------------------------------
+    // 7. Task 16: a keyset-honouring site drains a single-timestamp burst
+    //    LARGER than the batch size in ONE tick — the composite (UpdatedAtUtc,
+    //    TrackedOperationId) cursor advances by id even when every row shares
+    //    one exact timestamp, so the tail no longer pins forever.
+    // ---------------------------------------------------------------------
+
+    [Fact]
+    public void ReconciliationTick_KeysetCursor_DrainsSingleTimestampBurstBeyondBatchSize()
+    {
+        var siteId = "siteA";
+        // 2 × batchSize rows all sharing ONE exact UpdatedAtUtc: the pre-keyset
+        // inclusive-timestamp cursor could never advance past this window.
+        var ts = new DateTime(2026, 5, 20, 10, 0, 0, DateTimeKind.Utc);
+        var rows = Enumerable.Range(0, 4)
+            .Select(_ => NewRow(TrackedOperationId.New(), siteId, updatedAtUtc: ts))
+            .ToList();
+
+        var sites = new StaticEnumerator(new SiteEntry(siteId, "http://siteA:8083"));
+        var client = new KeysetHonoringPullClient(rows);
+        var repo = new RecordingRepo();
+
+        // batchSize = 2 → the 4-row burst needs the id tiebreak to page through.
+        // Slow tick so the full drain must be the within-tick continuation loop,
+        // not multiple ticks.
+        var options = new SiteCallAuditOptions
+        {
+            ReconciliationIntervalOverride = TimeSpan.FromSeconds(2),
+            ReconciliationBatchSize = 2,
+        };
+
+        CreateActor(sites, client, repo, options);
+
+        AwaitAssert(
+            () =>
+            {
+                foreach (var r in rows)
+                {
+                    Assert.True(repo.Upserted.ContainsKey(r.TrackedOperationId),
+                        "every row in the single-timestamp burst must reconcile via the "
+                        + "composite keyset cursor within one tick");
+                }
+            },
+            duration: TimeSpan.FromSeconds(3),
+            interval: TimeSpan.FromMilliseconds(50));
+
+        // Two pages (2 + 2) drained the burst; well under the 50-page ceiling.
+        Assert.True(client.CallCount is >= 2 and < 10,
+            $"expected the burst to drain in a couple of keyset pages, got {client.CallCount}");
+    }
+
+    // ---------------------------------------------------------------------
+    // 8. Task 16: a LEGACY site that ignores after_id can never be drained, so
+    //    the actor latches and publishes SiteCallReconciliationPinnedChanged
+    //    instead of silently logging (the dead-end is now health-observable).
+    // ---------------------------------------------------------------------
+
+    [Fact]
+    public void ReconciliationTick_LegacySiteIgnoresAfterId_PublishesPinnedChanged()
+    {
+        var siteId = "siteLegacy";
+        var ts = new DateTime(2026, 5, 20, 10, 0, 0, DateTimeKind.Utc);
+        var r1 = NewRow(TrackedOperationId.New(), siteId, updatedAtUtc: ts);
+        var r2 = NewRow(TrackedOperationId.New(), siteId, updatedAtUtc: ts);
+
+        var sites = new StaticEnumerator(new SiteEntry(siteId, "http://siteLegacy:8083"));
+        var client = new LegacyIgnoresAfterIdPullClient(new[] { r1, r2 });
+        var repo = new RecordingRepo();
+
+        // Subscribe BEFORE the actor starts ticking so the transition isn't missed.
+        Sys.EventStream.Subscribe(TestActor, typeof(SiteCallReconciliationPinnedChanged));
+
+        var options = new SiteCallAuditOptions
+        {
+            ReconciliationIntervalOverride = TimeSpan.FromMilliseconds(100),
+            ReconciliationBatchSize = 2,
+        };
+
+        CreateActor(sites, client, repo, options);
+
+        var evt = ExpectMsg<SiteCallReconciliationPinnedChanged>(TimeSpan.FromSeconds(5));
+        Assert.Equal(siteId, evt.SiteId);
+        Assert.True(evt.Pinned, "a legacy site that ignores after_id must publish Pinned=true");
+    }
 }

tests/ZB.MOM.WW.ScadaBridge.SiteCallAudit.Tests/SiteCallRelayTests.cs

@@ -213,4 +213,159 @@ public class SiteCallRelayTests : TestKit
         Assert.Equal(SiteCallRelayOutcome.SiteUnreachable, response.Outcome);
         Assert.False(response.SiteReachable);
     }
+
+    // ── Task 14: operator-identity audit on a successful relay ──
+
+    /// <summary>Captures the central direct-write audit events emitted by the relay.</summary>
+    private sealed class RecordingCentralAuditWriter : ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Services.ICentralAuditWriter
+    {
+        public List<AuditRowProjection.AuditRowValues> Events { get; } = new();
+
+        public Task WriteAsync(ZB.MOM.WW.Audit.AuditEvent evt, CancellationToken ct = default)
+        {
+            lock (Events)
+            {
+                Events.Add(evt.AsRow());
+            }
+
+            return Task.CompletedTask;
+        }
+    }
+
+    /// <summary>
+    /// A repository whose <see cref="GetAsync"/> returns one fixed row (so the relay
+    /// can read its Channel for the audit event) but still throws on every MUTATING
+    /// call — central never writes the mirror row on the relay path.
+    /// </summary>
+    private sealed class RowReturningRepository : ISiteCallAuditRepository
+    {
+        private readonly SiteCall _row;
+        public RowReturningRepository(SiteCall row) { _row = row; }
+
+        public Task<SiteCall?> GetAsync(TrackedOperationId id, CancellationToken ct = default) =>
+            Task.FromResult<SiteCall?>(_row.TrackedOperationId == id ? _row : null);
+
+        public Task UpsertAsync(SiteCall siteCall, CancellationToken ct = default) =>
+            throw new InvalidOperationException("relay must not write the SiteCalls row");
+
+        public Task<IReadOnlyList<SiteCall>> QueryAsync(
+            SiteCallQueryFilter filter, SiteCallPaging paging, CancellationToken ct = default) =>
+            throw new InvalidOperationException("relay must not query the SiteCalls table");
+
+        public Task<int> PurgeTerminalAsync(DateTime olderThanUtc, CancellationToken ct = default) =>
+            throw new InvalidOperationException("relay must not purge");
+
+        public Task<SiteCallKpiSnapshot> ComputeKpisAsync(
+            DateTime stuckCutoff, DateTime intervalSince, CancellationToken ct = default) =>
+            throw new InvalidOperationException("relay must not compute KPIs");
+
+        public Task<IReadOnlyList<SiteCallSiteKpiSnapshot>> ComputePerSiteKpisAsync(
+            DateTime stuckCutoff, DateTime intervalSince, CancellationToken ct = default) =>
+            throw new InvalidOperationException("relay must not compute per-site KPIs");
+
+        public Task<IReadOnlyList<SiteCallNodeKpiSnapshot>> ComputePerNodeKpisAsync(
+            DateTime stuckCutoff, DateTime intervalSince, CancellationToken ct = default) =>
+            throw new InvalidOperationException("relay must not compute per-node KPIs");
+    }
+
+    private static SiteCall MakeRow(Guid id, string channel = "ApiOutbound") => new()
+    {
+        TrackedOperationId = new TrackedOperationId(id),
+        Channel = channel,
+        Target = "ERP.GetOrder",
+        SourceSite = "site-north",
+        SourceNode = "node-a",
+        Status = "Parked",
+        RetryCount = 3,
+        CreatedAtUtc = DateTime.UtcNow,
+        UpdatedAtUtc = DateTime.UtcNow,
+        IngestedAtUtc = DateTime.UtcNow,
+    };
+
+    private (IActorRef Actor, RecordingCentralAuditWriter Writer) CreateActorWithAudit(IActorRef centralComm, SiteCall row)
+    {
+        var writer = new RecordingCentralAuditWriter();
+        var options = new SiteCallAuditOptions { RelayTimeout = TimeSpan.FromMilliseconds(500) };
+        var actor = Sys.ActorOf(Props.Create(() => new SiteCallAuditActor(
+            new RowReturningRepository(row),
+            NullLogger<SiteCallAuditActor>.Instance,
+            options,
+            (ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Services.ICentralAuditWriter)writer)));
+        actor.Tell(new RegisterCentralCommunication(centralComm));
+        return (actor, writer);
+    }
+
+    [Fact]
+    public void RetrySiteCall_WhenApplied_EmitsCachedResolveSubmitted_WithOperator()
+    {
+        var central = CreateTestProbe();
+        var id = Guid.NewGuid();
+        var (actor, writer) = CreateActorWithAudit(central.Ref, MakeRow(id, channel: "ApiOutbound"));
+
+        actor.Tell(new RetrySiteCallRequest("corr-a", id, "site-north", RequestedBy: "jdoe"));
+
+        var envelope = central.ExpectMsg<SiteEnvelope>();
+        var relay = (RetryParkedOperation)envelope.Message;
+        central.Reply(new ParkedOperationActionAck(relay.CorrelationId, Applied: true));
+
+        ExpectMsg<RetrySiteCallResponse>(r => r.Success);
+        AwaitAssert(() =>
+        {
+            List<AuditRowProjection.AuditRowValues> events;
+            lock (writer.Events) { events = writer.Events.ToList(); }
+            Assert.Single(events);
+            var evt = events[0];
+            Assert.Equal(Commons.Types.Enums.AuditChannel.ApiOutbound, evt.Channel);
+            Assert.Equal(Commons.Types.Enums.AuditKind.CachedResolve, evt.Kind);
+            Assert.Equal(Commons.Types.Enums.AuditStatus.Submitted, evt.Status);
+            Assert.Equal("jdoe", evt.Actor);
+            Assert.Equal(id, evt.CorrelationId);
+        });
+    }
+
+    [Fact]
+    public void RetrySiteCall_WhenNotApplied_EmitsNoAuditRow()
+    {
+        var central = CreateTestProbe();
+        var id = Guid.NewGuid();
+        var (actor, writer) = CreateActorWithAudit(central.Ref, MakeRow(id));
+
+        actor.Tell(new RetrySiteCallRequest("corr-b", id, "site-north", RequestedBy: "jdoe"));
+
+        var envelope = central.ExpectMsg<SiteEnvelope>();
+        var relay = (RetryParkedOperation)envelope.Message;
+        // Nothing was parked at the site — no state change, so no operator audit row.
+        central.Reply(new ParkedOperationActionAck(relay.CorrelationId, Applied: false));
+
+        ExpectMsg<RetrySiteCallResponse>(r => r.Outcome == SiteCallRelayOutcome.NotParked);
+        // Give any erroneous emission a chance to land, then assert none did.
+        ExpectNoMsg(TimeSpan.FromMilliseconds(300));
+        lock (writer.Events) { Assert.Empty(writer.Events); }
+    }
+
+    [Fact]
+    public void DiscardSiteCall_WhenApplied_EmitsCachedResolveDiscarded_WithOperator()
+    {
+        var central = CreateTestProbe();
+        var id = Guid.NewGuid();
+        var (actor, writer) = CreateActorWithAudit(central.Ref, MakeRow(id, channel: "DbOutbound"));
+
+        actor.Tell(new DiscardSiteCallRequest("corr-c", id, "site-north", RequestedBy: "jdoe"));
+
+        var envelope = central.ExpectMsg<SiteEnvelope>();
+        var relay = (DiscardParkedOperation)envelope.Message;
+        central.Reply(new ParkedOperationActionAck(relay.CorrelationId, Applied: true));
+
+        ExpectMsg<DiscardSiteCallResponse>(r => r.Success);
+        AwaitAssert(() =>
+        {
+            List<AuditRowProjection.AuditRowValues> events;
+            lock (writer.Events) { events = writer.Events.ToList(); }
+            Assert.Single(events);
+            var evt = events[0];
+            Assert.Equal(Commons.Types.Enums.AuditChannel.DbOutbound, evt.Channel);
+            Assert.Equal(Commons.Types.Enums.AuditStatus.Discarded, evt.Status);
+            Assert.Equal("jdoe", evt.Actor);
+        });
+    }
 }

tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Tracking/OperationTrackingStoreTests.cs

@@ -466,7 +466,7 @@ public class OperationTrackingStoreTests
 
         // Cursor at the middle row's UpdatedAtUtc: inclusive lower bound, so
         // middle + newer come back, older is excluded.
-        var result = await store.ReadChangedSinceAsync(t0.AddMinutes(10), batchSize: 100, CancellationToken.None);
+        var result = await store.ReadChangedSinceAsync(t0.AddMinutes(10), batchSize: 100, ct: CancellationToken.None);
 
         Assert.Equal(2, result.Count);
         Assert.Equal(middle, result[0].TrackedOperationId);
@@ -483,7 +483,7 @@ public class OperationTrackingStoreTests
         await store.RecordEnqueueAsync(TrackedOperationId.New(), nameof(AuditKind.ApiCallCached), "A", null, null, null);
         await store.RecordEnqueueAsync(TrackedOperationId.New(), nameof(AuditKind.ApiCallCached), "B", null, null, null);
 
-        var result = await store.ReadChangedSinceAsync(DateTime.MinValue, batchSize: 100, CancellationToken.None);
+        var result = await store.ReadChangedSinceAsync(DateTime.MinValue, batchSize: 100, ct: CancellationToken.None);
 
         Assert.Equal(2, result.Count);
     }
@@ -504,7 +504,7 @@ public class OperationTrackingStoreTests
             SetUpdatedAt(dataSource, id, t0.AddMinutes(i));
         }
 
-        var result = await store.ReadChangedSinceAsync(DateTime.MinValue, batchSize: 3, CancellationToken.None);
+        var result = await store.ReadChangedSinceAsync(DateTime.MinValue, batchSize: 3, ct: CancellationToken.None);
 
         // Capped to 3 — and the cap takes the OLDEST 3 (asc order) so the
         // caller can advance the cursor monotonically across follow-up pulls.
@@ -527,7 +527,7 @@ public class OperationTrackingStoreTests
         await store.RecordAttemptAsync(apiId, nameof(AuditStatus.Attempted), 2, "HTTP 503", 503);
         await store.RecordTerminalAsync(dbId, nameof(AuditStatus.Parked), "max retries", null);
 
-        var result = await store.ReadChangedSinceAsync(DateTime.MinValue, batchSize: 100, CancellationToken.None);
+        var result = await store.ReadChangedSinceAsync(DateTime.MinValue, batchSize: 100, ct: CancellationToken.None);
         var api = result.Single(r => r.TrackedOperationId == apiId);
         var db = result.Single(r => r.TrackedOperationId == dbId);
 
@@ -720,4 +720,92 @@ public class OperationTrackingStoreTests
 
         Assert.Equal(2, rows.Count);
     }
+
+    // ── Task 15: composite (UpdatedAtUtc, TrackedOperationId) keyset ──
+
+    /// <summary>
+    /// Seeds <c>batchSize + 2</c> rows that all share ONE <c>UpdatedAtUtc</c> and
+    /// pages through them with the composite keyset. Page 1 (no cursor) returns the
+    /// first <c>batchSize</c>; page 2 resumes from <c>(sinceUtc, lastId)</c> and
+    /// returns the REMAINING rows with no overlap — the stall the plain inclusive
+    /// <c>&gt;=</c> resume can never escape.
+    /// </summary>
+    [Fact]
+    public async Task ReadChangedSince_CompositeKeyset_AdvancesPastSharedTimestamp()
+    {
+        const int batchSize = 3;
+        var (store, dataSource) = CreateStore(nameof(ReadChangedSince_CompositeKeyset_AdvancesPastSharedTimestamp));
+        await using var _store = store;
+
+        for (var i = 0; i < batchSize + 2; i++)
+        {
+            await store.RecordEnqueueAsync(TrackedOperationId.New(),
+                kind: nameof(AuditKind.ApiCallCached), targetSummary: $"ERP.{i}",
+                sourceInstanceId: "I", sourceScript: "S", sourceNode: "node-a");
+        }
+
+        var boundary = ForceSharedUpdatedAt(dataSource);
+
+        // Page 1: no cursor → the first batchSize rows, ordered by TrackedOperationId.
+        var page1 = await store.ReadChangedSinceAsync(boundary, batchSize);
+        Assert.Equal(batchSize, page1.Count);
+
+        // Page 2: resume strictly after the last id of page 1.
+        var afterId = page1[^1].TrackedOperationId.ToString();
+        var page2 = await store.ReadChangedSinceAsync(boundary, batchSize, afterId);
+
+        // The remaining two rows, none seen on page 1, all strictly greater than the cursor.
+        Assert.Equal(2, page2.Count);
+        var page1Ids = page1.Select(r => r.TrackedOperationId).ToHashSet();
+        Assert.All(page2, r => Assert.DoesNotContain(r.TrackedOperationId, page1Ids));
+        Assert.All(page2, r => Assert.True(
+            string.CompareOrdinal(r.TrackedOperationId.ToString(), afterId) > 0));
+
+        // Union covers every seeded row exactly once — no gap, no dup.
+        Assert.Equal(batchSize + 2, page1Ids.Concat(page2.Select(r => r.TrackedOperationId)).Distinct().Count());
+    }
+
+    /// <summary>
+    /// Regression guard for the legacy contract: with NO cursor a resume at the
+    /// shared timestamp re-reads the same page (the exact stall the keyset fixes) —
+    /// an older central that never sends <c>after_id</c> is byte-for-byte unaffected.
+    /// </summary>
+    [Fact]
+    public async Task ReadChangedSince_WithoutKeyset_ReReadsSamePage_LegacyPreserved()
+    {
+        const int batchSize = 3;
+        var (store, dataSource) = CreateStore(nameof(ReadChangedSince_WithoutKeyset_ReReadsSamePage_LegacyPreserved));
+        await using var _store = store;
+
+        for (var i = 0; i < batchSize + 2; i++)
+        {
+            await store.RecordEnqueueAsync(TrackedOperationId.New(),
+                kind: nameof(AuditKind.ApiCallCached), targetSummary: $"ERP.{i}",
+                sourceInstanceId: "I", sourceScript: "S", sourceNode: "node-a");
+        }
+
+        var boundary = ForceSharedUpdatedAt(dataSource);
+
+        var page1 = await store.ReadChangedSinceAsync(boundary, batchSize);
+        // Legacy resume: same sinceUtc, no cursor → identical page (the stall).
+        var pageAgain = await store.ReadChangedSinceAsync(boundary, batchSize);
+
+        Assert.Equal(
+            page1.Select(r => r.TrackedOperationId).ToList(),
+            pageAgain.Select(r => r.TrackedOperationId).ToList());
+    }
+
+    /// <summary>Forces every OperationTracking row to one exact UpdatedAtUtc instant; returns it.</summary>
+    private DateTime ForceSharedUpdatedAt(string dataSource)
+    {
+        var boundary = DateTime.UtcNow;
+        var boundaryText = DateTime.SpecifyKind(boundary, DateTimeKind.Utc)
+            .ToString("o", System.Globalization.CultureInfo.InvariantCulture);
+        using var conn = OpenVerifierConnection(dataSource);
+        using var cmd = conn.CreateCommand();
+        cmd.CommandText = "UPDATE OperationTracking SET UpdatedAtUtc = $ts";
+        cmd.Parameters.AddWithValue("$ts", boundaryText);
+        cmd.ExecuteNonQuery();
+        return boundary;
+    }
 }