feat(historian): Part 13 time-weighted aggregates on SqlServer + QuestDb (#671)

New AggregateFamily.TimeWeighted serving TimeAverage, TimeAverage2, Total,
Total2 and Interpolative, evaluated by the shared Sql.Common
TimeWeightedAggregator for both SQL backends (identical results by
construction). HistoryAggregateType gains TimeAverage2 + Total2 (appended);
OtOpcUaNodeManager.MapAggregate maps their NodeIds. Gateway's mapper already
refuses unknown defined members => BadAggregateNotSupported per node.

Semantics (server defaults, client AggregateConfiguration ignored):
TreatUncertainAsBad=false, PercentDataBad=PercentDataGood=100,
UseSlopedExtrapolation=false. Sloped for Double/Int64, stepped for Boolean;
a usable sample followed by a Bad one is held up to the outage (AtTime rule);
stepped Uncertain extrapolation past the last sample. TimeAverage/Total use
interpolated bounding (bridge outages); TimeAverage2/Total2 exclude Bad time.
Bounding values: latest usable + latest any before start, first any + first
usable at/after end (four TOP 1 / LIMIT 1 seeks).

CLIENT-VISIBLE: Total on SqlServer/QuestDb was SUM of samples; it is now the
time-weighted integral (TimeAverage x interval seconds), matching Gateway's
Integral. Release note in docs/Historian.md.

Trade-off (deviation from the preferred hybrid pushdown, allowed by the
design): the time-weighted family is NOT pushed down. Each read streams the
window's rows plus the four bounds (SQL Server: one five-result-set batch;
QuestDB: bounds UNION ALL + window select) into the C# evaluator. Network is
O(samples in window), memory O(buckets). Writing the per-segment
interpolation/edge splitting twice in two SQL dialects and keeping them
bit-identical was judged the bigger risk; this also makes the QuestDB
lag/lead spike moot.

Tests: 35 hand-computed TimeWeightedAggregator unit cases (each of 10
implementation mutations goes red); served-aggregate pin 13 -> 17;
conformance sparse fixture (intervals wider and narrower than sample gaps)
+ an outage straddling both window edges (each of the four SQL seeks
disabled in turn goes red); node-manager NodeId mapping test.
This commit is contained in:
Joseph Doherty
2026-10-11 08:44:26 -04:00
parent db6a53036b
commit 1f88669304
17 changed files with 1118 additions and 37 deletions
@@ -105,6 +105,41 @@ internal static class QuestDbAggregateQueries
"SELECT st.*, r.num_value, r.ts " +
"FROM st LEFT JOIN (SELECT * FROM r WHERE rn = 1) r ON r.b = st.b ORDER BY st.b";
/// <summary>
/// Time-weighted family (#671), bounds statement — <b>not</b> pushed down (see <see cref="TimeWeightedWindow"/>).
/// Four <c>LIMIT 1</c> seeks <c>UNION ALL</c>-ed, each tagged with <c>k</c>: 0 = latest usable sample before the
/// start, 1 = latest sample of any quality before the start, 2 = first sample of any quality at or after the end,
/// 3 = first usable one at or after the end. Parameters: <c>$1</c> tag, <c>$2</c>/<c>$3</c> the forward window
/// start / end. Columns: <c>k</c>, then <c>ts, kind, num_value, int_value, str_value, status, server_ts</c>.
/// </summary>
/// <param name="sample">Prefixed sample table name.</param>
/// <returns>The SQL text.</returns>
public static string TimeWeightedBounds(string sample)
{
var usable = $" AND status < {NotBadLimit} AND kind <> {EmptyKind}";
string Seek(int k, string cmp, string filter, string order) =>
$"SELECT * FROM (SELECT {k.ToString(CultureInfo.InvariantCulture)} AS k, {SampleColumns} FROM {sample} " +
$"WHERE tag = $1 AND ts {cmp}{filter} AND NOT deleted ORDER BY ts {order} LIMIT 1)";
return Seek(0, "< $2", usable, "DESC") + " UNION ALL " + Seek(1, "< $2", "", "DESC") + " UNION ALL " +
Seek(2, ">= $3", "", "ASC") + " UNION ALL " + Seek(3, ">= $3", usable, "ASC");
}
/// <summary>
/// Time-weighted family (#671), window statement: every live sample in <c>[$2, $3)</c> for tag <c>$1</c>, ascending.
/// Columns: <c>ts, kind, num_value, int_value, str_value, status, server_ts</c>. The caller feeds the before-bounds,
/// these rows and the after-bounds, in that order, into <see cref="TimeWeightedAggregator"/> — the same evaluator
/// the SQL Server backend uses, so the two agree exactly. Trade-off: the window's rows travel to the server rather
/// than one row per bucket; Part 13's segment interpolation and bucket-edge splitting are then written once
/// instead of twice in two SQL dialects that must stay bit-identical.
/// </summary>
/// <param name="sample">Prefixed sample table name.</param>
/// <returns>The SQL text.</returns>
public static string TimeWeightedWindow(string sample) =>
$"SELECT {SampleColumns} FROM {sample} WHERE tag = $1 AND ts >= $2 AND ts < $3 AND NOT deleted ORDER BY ts ASC";
/// <summary>The row columns the time-weighted statements return (after <c>k</c> for the bounds).</summary>
private const string SampleColumns = "ts, kind, num_value, int_value, str_value, status, server_ts";
/// <summary>The bucket-index expression (exposed for tests).</summary>
internal const string BucketExpression = "(cast(ts AS LONG) * 10 - $1) / $2";