perf(historian): SQL Server retention deletes only for tags with eligible rows (lmxopcua#746)
The SqlServer purge read every TagId and issued at least one DELETE TOP per table per tag on every pass, even when nothing was eligible: 50000 tags made 100000 statements per hourly purge, serialised on one connection under the applock. One pre-filter query per table (EXISTS seek on the clustered key) now picks the tags holding a row older than the cutoff; only those get deletes. A pass with nothing eligible sends just the Event delete. The mid-command cancellation test now accepts the pre-filter SELECT as the blocked command, since it is the first statement to touch Sample. Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22 in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
@@ -1842,6 +1842,10 @@ equivalent TTL (`QuestDb:RetentionDays`); the Gateway backend's retention is the
|
||||
`DELETE TOP (@n) FROM SampleModified WHERE TagId = @id AND SourceTimeUtc < @cutoff` (#668; same **source-time** cutoff, clustered seek; the
|
||||
purge itself never writes a trail row) — then events with `DELETE TOP (@n) ... WHERE EventTimeUtc < @cutoff` (seek on `CX_Event_Time`), each looped
|
||||
until a batch comes back short. Cancellation is honoured between batches. The `hist.Tag` catalogue is never purged.
|
||||
**Only eligible tags get deletes:** one pre-filter query per table (`SELECT TagId FROM Tag t WHERE EXISTS (SELECT 1 FROM Sample s
|
||||
WHERE s.TagId = t.TagId AND s.SourceTimeUtc < @cutoff)`, a key seek per tag inside one statement) picks the tags holding an
|
||||
eligible row; the per-tag loop used to send two `DELETE TOP` statements for every tag on every pass — 100 000 for 50 000 tags with
|
||||
nothing to delete, all under the applock.
|
||||
- **Shared databases.** Several pairs may point at one database; a session `sp_getapplock` (`OtOpcUa.Historian.Retention.<schema>`,
|
||||
timeout 0) lets one purger run and the others skip that tick (Debug log).
|
||||
- **Observability.** Counter `otopcua.historian.retention.purged{table=Sample|SampleModified|Event}`; a failed pass logs a Warning and retries next
|
||||
|
||||
Reference in New Issue
Block a user