perf(central): set-based ingest, aligned partition purge, KPI query shapes, EF hygiene
This commit is contained in:
@@ -291,6 +291,27 @@ stay `Pending` for the next sweep. Cadence is short (default 5 s) when
|
||||
non-empty, longer (default 30 s) when idle; telemetry runs on a dedicated
|
||||
dispatcher.
|
||||
|
||||
**Central-side ingest is set-based.** `AuditLogIngestActor` writes a whole
|
||||
packet with ONE `InsertManyIfNotExistsAsync` statement rather than one
|
||||
`IF NOT EXISTS … INSERT` round trip per event; the cached-telemetry dual-write
|
||||
similarly runs the whole packet in ONE transaction (set-based audit insert plus
|
||||
one single-statement `SiteCalls` upsert per entry) instead of a transaction per
|
||||
entry. Idempotency is unchanged: duplicates that repeat *within* a packet
|
||||
collapse first-write-wins before the statement is built, duplicates *across*
|
||||
packets are eliminated by the anti-semi-join, and any failure falls back to the
|
||||
per-row / per-entry path — so the documented invariant that one bad row cannot
|
||||
sink the rest of the batch still holds, it is simply no longer paid for on the
|
||||
healthy path.
|
||||
|
||||
**Timeout ladder.** The ingest budget is deliberately the smallest on the path:
|
||||
the site's Ask and the central gRPC handler's Ask are both 30 s, the actor's
|
||||
own database budget is 20 s and the per-statement SQL timeout is 15 s. Before
|
||||
this the three were identical, so they expired at the same instant and the
|
||||
caller learned nothing but "it took 30 s" — no partial ack, no way to tell a
|
||||
slow database from a wedged singleton. With the inner budgets strictly smaller,
|
||||
a slow batch is abandoned by the actor first and the accepted-so-far ids are
|
||||
still replied while the outer Asks are still waiting.
|
||||
|
||||
### Reconciliation pull (self-healing for missed telemetry)
|
||||
|
||||
A central `SiteAuditReconciliationActor` periodically (default 5 min per site)
|
||||
@@ -490,12 +511,19 @@ MS SQL for direct-write events). Unredacted secrets never persist.
|
||||
silently failing retention job would otherwise be invisible until the table grew
|
||||
unbounded. Per-boundary/per-channel error isolation is unchanged — a single failure
|
||||
still never abandons the rest of the tick.
|
||||
- **Maintenance command timeout:** the switch-out drop-and-rebuild dance and each
|
||||
- **Maintenance command timeout:** the switch-out staging batch and each
|
||||
per-channel `DELETE TOP` batch run with an explicit command timeout
|
||||
(`AuditLog:Purge:MaintenanceCommandTimeoutMinutes`, default 30, floor 1 min) rather
|
||||
than the ~30 s ADO.NET default, which could abort the metadata-only SWITCH mid-dance
|
||||
on a large or contended partition and leave the live table without
|
||||
`UX_AuditLog_EventId` until a later tick's CATCH branch rebuilt it.
|
||||
than the ~30 s ADO.NET default, which could abort the metadata-only SWITCH mid-batch
|
||||
on a large or contended partition and leave an orphaned staging table for the next
|
||||
tick's CATCH branch to clean up.
|
||||
- **Partition-aligned uniqueness:** the switch no longer drops and rebuilds an index.
|
||||
`EventId` uniqueness rides the clustered `PK_AuditLog (EventId, OccurredAtUtc)`,
|
||||
which is aligned on `ps_AuditLog_Month(OccurredAtUtc)`, so `SWITCH PARTITION` has no
|
||||
non-aligned unique index to object to. The predecessor `UX_AuditLog_EventId` forced
|
||||
an offline whole-table index build inside the switch transaction — blocking every
|
||||
audit writer for its duration — and left a window in which the idempotency-supporting
|
||||
index did not exist at all. See migration `AlignAuditLogEventIdUniqueness`.
|
||||
- **Per-channel retention overrides (M5.5 T3):** `AuditLog:PerChannelRetentionDays`
|
||||
is a dictionary keyed by canonical channel name (`ApiOutbound`, `DbOutbound`,
|
||||
`Notification`, `ApiInbound`, `SecuredWrite` — all five `AuditChannel` values are
|
||||
|
||||
Reference in New Issue
Block a user