perf(audit-log): cached-call forwarder enqueues the audit row instead of awaiting its commit (194 ms -> 25 ms)
Measuring the tracking-store queue on the rig moved nothing: the DB-done -> Notify.Send gap stayed at 108 ms. The cost was the audit half of CachedCallTelemetryForwarder: SqliteAuditWriter.WriteAsync enqueues synchronously but returns the BATCH completion, and the writer loop coalesces for FlushIntervalMs (50 ms) before it commits, so each awaited audit row cost ~50 ms + fsync and a cached call awaited three (submit, attempted, resolve) on the script thread. Every other script-path emitter already fired-and-observed; only this forwarder awaited. TryEmitAuditAsync now hands the row to the writer and observes the completion on a fault-logging continuation (the FallbackAuditWriter ring-buffer fallback still runs inside that task). Best-effort contract unchanged; per-store FIFO unchanged; nothing joins the audit and tracking stores by commit order. Rig, FlowOutboundDemo, 5-run medians: inbound 194 -> 25 ms, site script run 176 -> 17 ms, CachedWrite submit->done 61 -> 11 ms, DB done -> Notify.Send 108 -> 0.1 ms. The tracking queue stays (three fewer fsyncs per call, batched). Docs corrected to attribute the cost to the flush wait, with the trap recorded in CLAUDE.md and Component-SiteRuntime.md. Tests: ForwardAsync returns while the audit commit is still pending (never-completing writer task) and still records tracking; a late commit fault is swallowed. Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
This commit is contained in:
@@ -53,7 +53,7 @@ For notifications, "delivery" means forwarding the message to the central cluste
|
||||
|
||||
There are now exactly two ways a notification parks. First, a **corrupt buffered payload** (a row whose stored JSON no longer deserialises to a `NotificationSubmit`, or deserialises to null): the forwarder returns the delivery-handler contract's permanent-failure signal (`false`) so the engine parks the row, preserving the payload for operator forensics. This **supersedes `StoreAndForward-018`**, which discarded such rows by reporting them as delivered — a silent data loss with no parked row, no central `Notifications` row, and no audit trail. Retrying a corrupt payload is pointless (it can never deserialise), so parking, not retrying, is the correct home for it. Second, retry exhaustion under the `DefaultMaxRetries` cap — but because `Notify.Send` enqueues with `maxRetries: 0` (its unbounded-retry contract, above), that second cause is **off for script-produced notifications**, leaving corrupt payload as their only parking cause. (Retry-exhaustion parking remains reachable only for a hypothetical notification enqueued with a positive `maxRetries`.)
|
||||
|
||||
For the cached-call categories (`ExternalCall` and `DatabaseWrite`), the operation tracking table is the status record and the S&F buffer is purely the retry mechanism. A cached call that succeeds on its first immediate attempt is written directly as a terminal `Delivered` tracking row and never enters the S&F buffer. When immediate delivery fails transiently, the message is buffered and its tracking row moves to `Pending`/`Retrying`; the buffered message carries its `TrackedOperationId` so the tracking row and the retry record stay linked. When immediate delivery fails **permanently** (e.g. HTTP 4xx), the message is not buffered — the error is returned synchronously to the calling script as before — but the tracking row is written directly as a terminal `Failed` row capturing the error. On every tracking-table status transition the site emits `CachedCallTelemetry` to central. The tracking row itself is written through a single-writer queue (Site Runtime, 2026-09-06): the script's `await` on each lifecycle emission returns once the row is queued, not committed, and reads of the table wait for earlier queued writes so status stays read-your-writes — see `Component-SiteRuntime.md` → *Operation Tracking*.
|
||||
For the cached-call categories (`ExternalCall` and `DatabaseWrite`), the operation tracking table is the status record and the S&F buffer is purely the retry mechanism. A cached call that succeeds on its first immediate attempt is written directly as a terminal `Delivered` tracking row and never enters the S&F buffer. When immediate delivery fails transiently, the message is buffered and its tracking row moves to `Pending`/`Retrying`; the buffered message carries its `TrackedOperationId` so the tracking row and the retry record stay linked. When immediate delivery fails **permanently** (e.g. HTTP 4xx), the message is not buffered — the error is returned synchronously to the calling script as before — but the tracking row is written directly as a terminal `Failed` row capturing the error. On every tracking-table status transition the site emits `CachedCallTelemetry` to central. Neither the audit row nor the tracking row of that telemetry is awaited to commit on the script thread (Site Runtime, 2026-09-06): the audit row is enqueued on the site writer and its completion observed off-thread, and the tracking row goes through a single-writer queue whose reads wait for earlier queued writes so `Tracking.Status` stays read-your-writes — see `Component-SiteRuntime.md` → *Operation Tracking*.
|
||||
|
||||
Every cached-call outcome maps to a tracking-table state: immediate success → `Delivered`; transient failure → `Pending`/`Retrying`, eventually `Delivered` or `Parked`; permanent failure → terminal `Failed`; operator discard of a parked row → terminal `Discarded`.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user