perf(historian): #683 SqlServer StartEnd join-back on PK
Start/End/Delta: one GROUP BY computes the counts and each bucket's four pick
times (MIN/MAX SourceTimeUtc, any quality and usable); each pick's columns are
then fetched by a seek on the clustered key (TagId, SourceTimeUtc). Replaces the
windowed ROW_NUMBER form (live-gate 8 F1), whose four window sorts spilled.
Output identical (EXCEPT both ways empty, 6M rows, forward + reverse incl. a
sub-microsecond reverse start). 6M rows / 30 days, avg of 3 x 2 rounds:
30d@1h 18.6-31.7 s -> 3.4 s (reverse 18.1-29.9 s -> 3.4-5.2 s)
7d@1d 5.2-6.1 s -> 0.8 s
1d@1min 0.31-0.40 s -> 0.16 s (1d@7min reverse 0.46 -> 0.12 s)
The seeks are separate reads within one statement: under READ COMMITTED a
HistoryUpdate delete between the aggregate and the seek nulls that pick; the
null-tolerant reader (2ceed561) makes that bucket's Start/End/Delta BadNoData,
transiently. Unit test pins it; documented in the query remarks + Historian.md.
QuestDB single-scan StartEnd unchanged.
This commit is contained in:
+19
@@ -76,6 +76,25 @@ public sealed class BucketRowMappingTests
|
||||
AggregateEvaluator.Evaluate(agg, b, T, T).StatusCode.ShouldBe(NoDataCalc);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_pick_nulled_by_a_delete_racing_the_key_join_is_no_data_for_that_end_only()
|
||||
{
|
||||
// #683 join-back: the bucket's first sample (usable, so both first picks) was deleted between the GROUP BY and its
|
||||
// key seek — the counts still include it, its pick columns are NULL; the last picks joined normally.
|
||||
var t = StartEndTable();
|
||||
var row = Counts(t, 0, 2);
|
||||
row["k1"] = (byte)1; row["n1"] = 20d; row["t1"] = T.AddSeconds(1);
|
||||
row["at1"] = T.AddSeconds(1); row["ak1"] = (byte)1; row["an1"] = 20d; row["st1"] = 0;
|
||||
t.Rows.Add(row);
|
||||
|
||||
var b = Map(t, AggregateFamily.StartEnd);
|
||||
b.FirstAny.ShouldBeNull();
|
||||
b.First.Kind.ShouldBe(HistorianValueKind.Empty);
|
||||
AggregateEvaluator.Evaluate(HistoryAggregateType.Start, b, T, T).StatusCode.ShouldBe(NoDataCalc);
|
||||
AggregateEvaluator.Evaluate(HistoryAggregateType.Delta, b, T, T).StatusCode.ShouldBe(NoDataCalc);
|
||||
AggregateEvaluator.Evaluate(HistoryAggregateType.End, b, T, T).Value.ShouldBe(20d);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_consistent_row_still_maps_every_pick()
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user