feat(historian): Part 13 time-weighted aggregates on SqlServer + QuestDb (#671)
New AggregateFamily.TimeWeighted serving TimeAverage, TimeAverage2, Total, Total2 and Interpolative, evaluated by the shared Sql.Common TimeWeightedAggregator for both SQL backends (identical results by construction). HistoryAggregateType gains TimeAverage2 + Total2 (appended); OtOpcUaNodeManager.MapAggregate maps their NodeIds. Gateway's mapper already refuses unknown defined members => BadAggregateNotSupported per node. Semantics (server defaults, client AggregateConfiguration ignored): TreatUncertainAsBad=false, PercentDataBad=PercentDataGood=100, UseSlopedExtrapolation=false. Sloped for Double/Int64, stepped for Boolean; a usable sample followed by a Bad one is held up to the outage (AtTime rule); stepped Uncertain extrapolation past the last sample. TimeAverage/Total use interpolated bounding (bridge outages); TimeAverage2/Total2 exclude Bad time. Bounding values: latest usable + latest any before start, first any + first usable at/after end (four TOP 1 / LIMIT 1 seeks). CLIENT-VISIBLE: Total on SqlServer/QuestDb was SUM of samples; it is now the time-weighted integral (TimeAverage x interval seconds), matching Gateway's Integral. Release note in docs/Historian.md. Trade-off (deviation from the preferred hybrid pushdown, allowed by the design): the time-weighted family is NOT pushed down. Each read streams the window's rows plus the four bounds (SQL Server: one five-result-set batch; QuestDB: bounds UNION ALL + window select) into the C# evaluator. Network is O(samples in window), memory O(buckets). Writing the per-segment interpolation/edge splitting twice in two SQL dialects and keeping them bit-identical was judged the bigger risk; this also makes the QuestDB lag/lead spike moot. Tests: 35 hand-computed TimeWeightedAggregator unit cases (each of 10 implementation mutations goes red); served-aggregate pin 13 -> 17; conformance sparse fixture (intervals wider and narrower than sample gaps) + an outage straddling both window edges (each of the four SQL seeks disabled in turn goes red); node-manager NodeId mapping test.
This commit is contained in:
@@ -91,6 +91,36 @@ internal static class SqlAggregateQueries
|
||||
"SELECT st.*, r.NumValue, r.SourceTimeUtc " +
|
||||
"FROM st LEFT JOIN r ON r.b = st.b AND r.rn = 1 ORDER BY st.b;";
|
||||
|
||||
/// <summary>
|
||||
/// Time-weighted family (#671: TimeAverage / TimeAverage2 / Total / Total2 / Interpolative) — <b>not</b> pushed down.
|
||||
/// One batch of five result sets, each <c>SourceTimeUtc, ValueKind, NumValue, IntValue, StrValue, StatusCode</c>:
|
||||
/// (1) the latest usable sample before <c>@start</c>, (2) the latest sample of any quality before <c>@start</c>,
|
||||
/// (3) every sample in <c>[@start, @end)</c> ascending, (4) the first sample of any quality at or after
|
||||
/// <c>@end</c>, (5) the first usable one at or after <c>@end</c>. Each bound is a <c>TOP 1</c> seek on the
|
||||
/// <c>(TagId, SourceTimeUtc)</c> key. The caller streams them, in that order, into
|
||||
/// <see cref="TimeWeightedAggregator"/> (shared with QuestDB, so both backends agree exactly).
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Trade-off: the window's rows travel to the server instead of one row per bucket. Part 13's per-segment
|
||||
/// interpolation, bucket-edge splitting and the two bounding-value definitions would otherwise have to be written
|
||||
/// twice (T-SQL and QuestDB SQL) and kept bit-identical; evaluating once in C# keeps the backends equal by
|
||||
/// construction. Memory stays O(buckets): the rows are streamed, never materialised.
|
||||
/// </remarks>
|
||||
/// <param name="sample">Schema-qualified <c>Sample</c> table name.</param>
|
||||
/// <returns>The SQL batch.</returns>
|
||||
public static string TimeWeighted(string sample)
|
||||
{
|
||||
const string cols = "SourceTimeUtc, ValueKind, NumValue, IntValue, StrValue, StatusCode";
|
||||
const string usable = " AND StatusCode >= 0 AND ValueKind <> 0";
|
||||
var key = $"FROM {sample} WHERE TagId = @id AND ";
|
||||
return
|
||||
$"SELECT TOP 1 {cols} {key}SourceTimeUtc < @start{usable} ORDER BY SourceTimeUtc DESC; " +
|
||||
$"SELECT TOP 1 {cols} {key}SourceTimeUtc < @start ORDER BY SourceTimeUtc DESC; " +
|
||||
$"SELECT {cols} {key}SourceTimeUtc >= @start AND SourceTimeUtc < @end ORDER BY SourceTimeUtc ASC; " +
|
||||
$"SELECT TOP 1 {cols} {key}SourceTimeUtc >= @end ORDER BY SourceTimeUtc ASC; " +
|
||||
$"SELECT TOP 1 {cols} {key}SourceTimeUtc >= @end{usable} ORDER BY SourceTimeUtc ASC;";
|
||||
}
|
||||
|
||||
/// <summary>The <c>s</c> CTE: the window's samples with bucket index and the usable/good flags.</summary>
|
||||
private static string Classified(string sample) =>
|
||||
"WITH s AS (SELECT (DATEDIFF_BIG(microsecond, @start, SourceTimeUtc) * 10 " +
|
||||
|
||||
Reference in New Issue
Block a user