fix(historian): #683 live gate 9 F1 — force an exact-key seek for every StartEnd pick

The window-bounded pick joins (4a3b1256) let the optimizer choose, under
understated statistics, a nested-loops RANGE seek over the window with the pick
time as a residual, once per bucket (O(buckets x window rows)); the cached plan
was then reused for wide reads: gate 9 1d@1min 20 s, here 47 s / 3.6M reads.
Each pick join now carries WITH (FORCESEEK (PK_Sample (TagId, SourceTimeUtc))):
an exact-key seek on both columns per bucket under any statistics, and no
hash/merge join over the tag's history can exist (forcing one is error 8622), so
the window bound is dropped from the join. Output identical (EXCEPT both ways,
fwd/rev, incl. a mixed Double/Int64/Boolean/String/DateTime/outage tag).

Measured (6M-row tag + 86k mixed-kinds tag; ms / logical reads), understated
stats: 1d@1min 47,223/3.6M -> 253/25.6k; mixed 1d@1min 20,164/1.66M -> 137/24k;
accurate stats: 1d@1min 121 -> 160 ms, 1d@1h 79 -> 117 ms. Rejected: OUTER
APPLY TOP(1) FORCESEEK (identical), OPTION (RECOMPILE) (still 20 s under the bad
stats, and a compile per read).

StartEndJoinBackTests rewritten: understated statistics + compile-small/reuse-wide
(18,645 reads; the merged query read 1,645,845 / 23 s and failed it), accurate
statistics with 300k history rows either side (22,616), forced HASH/MERGE
refused (8622 — also failed against the merged query).

Docs: perf table + explanation; SDK-parity N2 (*ActualTime Calculated bit) and
N3 (Good String => BadAggregateInvalidInputs, SDK skips it).
This commit is contained in:
Joseph Doherty
2026-10-12 13:14:24 -04:00
parent fe644d7eda
commit ce3369ebca
3 changed files with 143 additions and 95 deletions
+38 -20
View File
@@ -1337,6 +1337,12 @@ bounding values back to the source integer type, so its `Interpolative` returns
`TimeAverage`/`TimeAverage2`/`Total`/`Total2` differ from ours by the truncated bound values (~0.002–0.02 on the gate's data). Ours
always computes and returns `Double` (as stated above for every aggregate but Count and Start/End), which is the exact Part 13
value; the parity datasets are all `Double`, so this one is not masked there but recorded here.
Two more, found by live gate 9 (both kept deliberately): **Minimum/MaximumActualTime carry the Calculated bit on every interval**
(`0x…401`, `0x…405` with Partial) where the SDK returns the picked raw sample's own status (`0x…000`, `0x…404`) — the same value and
timestamp, and the same treatment as the Min/Max-at-interval-start case above; masked in `SdkAggregateParityTests` (class remarks).
And an interval holding a **Good String (or DateTime) sample** is `BadAggregateInvalidInputs` for Average/Minimum/Maximum/Range/
*ActualTime here (the documented #684 refusal rule — a Good non-numeric value is not a number to aggregate), while the SDK skips the
string and serves the numeric samples (e.g. Average 4 Good); an Uncertain non-numeric sample refuses nothing on either side.
**Pushdown trade-off.** Unlike the per-sample aggregates, the time-weighted family is **not** pushed down to SQL: the backend streams
the window's rows plus the four bounds (one batch of five result sets on SQL Server; a bounds `UNION ALL` + a window select on QuestDB)
@@ -1606,28 +1612,40 @@ Every query filters `NOT deleted`; every value goes through `StoredSample`, so t
into a windowed pass + conditional `MAX`; **#683 replaced that** — its four `ROW_NUMBER` sorts spilled and it measured *slower* than
the five-reference form on long windows. It is now **aggregate, then join back on the key**: one `GROUP BY` computes the counts and each
bucket's four pick *times* (`MIN`/`MAX(SourceTimeUtc)`, any quality and usable), and each pick's columns are fetched by a seek on the
clustered key `(TagId, SourceTimeUtc)`. Output is identical (`EXCEPT` both ways empty on 6 M rows, forward and reverse); measured on
6 M rows / 30 days, one tag (so a 1 d or 7 d window leaves up to 5.8 M rows of the tag's history outside it), average of 3 runs
over 2 rounds, plans compiled by the 30 d read and reused (sniffed) by the rest:
clustered key `(TagId, SourceTimeUtc)` — **forced** with `WITH (FORCESEEK (PK_Sample (TagId, SourceTimeUtc)))`, so every pick is a
nested-loops seek on both key columns, one row per bucket per pick, whatever the statistics say. The optimizer's own choices were
both pathological, and plans are cached and reused across reads and tags: keyed on the pick time alone, a hash/merge join whose inner
input was the tag's **whole history** (#683 review: 43,657 logical reads vs 361 for a 1 d @ 1 h read of a tag with 3 M rows outside the
window); bounded to the window as well, a nested-loops **range** seek over the window with the pick time as a residual, once per
bucket — O(buckets × window rows) — compiled whenever the statistics understate the table (a fresh or freshly bulk-loaded
historian, ascending-key estimates for recent windows) and then reused for a wide read (**live gate 9 F1**: 1 d @ 1 min 20 s). With
the hint neither shape exists: a forced `HASH`/`MERGE JOIN` is refused (error 8622). ⚠️ The hint names the primary key `PK_Sample`
created by `001_initial.sql`; a hand-built schema must keep that name.
| Window | windowed form (before) | join-back |
|---|---|---|
| 30 d @ 1 h (720 buckets) | 19.0–21.7 s | 3.4–3.5 s |
| 30 d @ 1 h, reverse | 24.7–30.3 s | 3.4 s |
| 7 d @ 1 d (7 buckets) | 5.5–6.0 s | 0.78–0.81 s |
| 7 d @ 1 min (10,080 buckets) | 2.1–2.3 s | 1.08–1.11 s |
| 1 d @ 1 h (24 buckets) | 0.55–0.60 s | 0.12–0.13 s |
| 1 d @ 1 h, reverse | 0.52–0.55 s | 0.11 s |
| 1 d @ 1 min (1440 buckets) | 0.29–0.32 s | 0.16 s |
Output is identical to every earlier form (`EXCEPT` both ways empty, forward and reverse, including a tag of mixed Double/Int64/
Boolean/String/DateTime/outage samples). Measured on 6 M rows / 30 days for one tag (a 1 d or 7 d window leaves up to 5.8 M rows of
that tag outside it) plus an 86 k-row mixed-kinds tag; elapsed ms / session logical reads, each variant's plan compiled on its first
read and reused by the rest:
⚠️ **Every pick join is bounded to the window** (`@start`/`@end`, the same predicate as the window CTE), not only keyed on the pick
time. Keyed alone, the optimizer may choose a hash/merge/adaptive join whose inner input is the tag's **whole history**, and
parameter sniffing then reuses that plan for every tag — a review measured 43,657 logical reads vs 361 for a 1 d @ 1 h read of a tag
with 3 M rows outside the window. Forcing that shape here, 1 d @ 1 h: **152,524** session logical reads unbounded vs **6,756** bounded
under `OPTION (HASH JOIN)` (6,655 under `MERGE JOIN`; the unforced plan reads 1,846 either way). `StartEndJoinBackTests` pins it:
a tag with 300 k rows either side of a 1 h window, forced hash and merge joins, < 1,000 reads (183 bounded, 8,055 with the bound
removed). Wider windows trade reads for time — 7 d @ 1 min reads 174 k pages (four key seeks per bucket) against the windowed form's
46 k, yet runs in half the time. (The dev SQL container is slow in absolute terms — the ratios are the point.)
| Window | before #683 (windowed) | #683 as merged, accurate stats | #683 as merged, understated stats ‡ | **now (FORCESEEK)**, accurate | **now**, understated ‡ |
|---|---|---|---|---|---|
| 1 d @ 1 min (1440 buckets) | 329 / 6.7 k | 121 / 14.9 k | **47,223 / 3.6 M** | 160 / 25.0 k | 253 / 25.6 k |
| 1 d @ 1 min, reverse | 371 / 6.6 k | 129 / 14.9 k | **46,806 / 3.6 M** | 167 / 25.0 k | 246 / 25.6 k |
| 1 d @ 1 h (24 buckets) | 578 / 5.4 k | 79 / 5.5 k | 984 / 63 k | 117 / 1.8 k | 206 / 3.6 k |
| 7 d @ 1 min (10,080 buckets) | 3,426 / 103 k | 1,019 / 92 k | (hours — not run) | 1,472 / 180 k | 1,819 / 179 k |
| 30 d @ 1 h (720 buckets) † | 35,339 / 322 k | 1,487 / 191 k | (hours — not run) | 5,720 / 73 k | 6,921 / 101 k |
| mixed kinds 1 d @ 1 min | 190 / 5.9 k | 92 / 12.0 k | **20,164 / 1.66 M** | 107 / 24.4 k | 137 / 24.0 k |
| mixed kinds 1 d @ 7 min, reverse | 199 / 1.5 k | 71 / 3.9 k | 2,977 / 238 k | 59 / 4.2 k | 93 / 4.7 k |
‡ `UPDATE STATISTICS … WITH ROWCOUNT = 34, PAGECOUNT = 1`, then the plan compiled on a 1 h @ 1 min read and reused — gate 9's
recipe. † The 30 d row is dominated by the window `GROUP BY`, not the picks: compiled on the 30 d read itself, the merged query and
the hinted one both take ~6.9 s (serial, a small memory grant, spills); the merged query's 1.5 s above is a plan compiled on the
1 d read and reused. Alternatives measured and rejected: the same seek written as `OUTER APPLY (SELECT TOP (1) … WITH (FORCESEEK
…))` (identical plans and numbers — the join form is shorter); `OPTION (RECOMPILE)` on the merged query (compiles under the bad
statistics on every call: still 20.4 s / 1.66 M reads on the mixed-kinds tag, and a compile per read). `StartEndJoinBackTests`
pins it — one sample a second over a day with 300 k rows of history either side: under understated statistics 18,645 reads
(the merged query: **1,645,845 reads, 23 s**, so the test failed against it), with accurate statistics 22,616, and forced
`HASH`/`MERGE` joins refused with 8622. (The dev SQL container is slow in absolute terms — the ratios are the point.)
⚠️ The seeks are separate reads of the table within one statement, so a sample **deleted** by HistoryUpdate between the aggregate
and its seek leaves that pick NULL while the counts include it: the null-tolerant reader below answers that bucket's Start/End/Delta