perf(historian): SQL Server retention deletes only for tags with eligible rows (lmxopcua#746)

The SqlServer purge read every TagId and issued at least one DELETE TOP per
table per tag on every pass, even when nothing was eligible: 50000 tags made
100000 statements per hourly purge, serialised on one connection under the
applock.

One pre-filter query per table (EXISTS seek on the clustered key) now picks
the tags holding a row older than the cutoff; only those get deletes. A pass
with nothing eligible sends just the Event delete. The mid-command
cancellation test now accepts the pre-filter SELECT as the blocked command,
since it is the first statement to touch Sample.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-10-22 16:56:57 -04:00
parent 74afc3f90a
commit d44e58dcd6
3 changed files with 75 additions and 16 deletions
+4
View File
@@ -1842,6 +1842,10 @@ equivalent TTL (`QuestDb:RetentionDays`); the Gateway backend's retention is the
`DELETE TOP (@n) FROM SampleModified WHERE TagId = @id AND SourceTimeUtc < @cutoff` (#668; same **source-time** cutoff, clustered seek; the
purge itself never writes a trail row) — then events with `DELETE TOP (@n) ... WHERE EventTimeUtc < @cutoff` (seek on `CX_Event_Time`), each looped
until a batch comes back short. Cancellation is honoured between batches. The `hist.Tag` catalogue is never purged.
**Only eligible tags get deletes:** one pre-filter query per table (`SELECT TagId FROM Tag t WHERE EXISTS (SELECT 1 FROM Sample s
WHERE s.TagId = t.TagId AND s.SourceTimeUtc < @cutoff)`, a key seek per tag inside one statement) picks the tags holding an
eligible row; the per-tag loop used to send two `DELETE TOP` statements for every tag on every pass — 100 000 for 50 000 tags with
nothing to delete, all under the applock.
- **Shared databases.** Several pairs may point at one database; a session `sp_getapplock` (`OtOpcUa.Historian.Retention.<schema>`,
timeout 0) lets one purger run and the others skip that tick (Debug log).
- **Observability.** Counter `otopcua.historian.retention.purged{table=Sample|SampleModified|Event}`; a failed pass logs a Warning and retries next