fix(historian): live-gate 8 F1 — per-sample counts and picks from one table version (#680 follow-up)
QuestDB Start/End/Delta reads of a tag being written intermittently failed BadHistoryOperationUnsupported (InvalidCastException: Column 'ts_3' is null): the StartEnd query referenced its CTE four times and QuestDB opens a table reader per reference, so a row committed between two of them was counted but not picked. MinimumActualTime/MaximumActualTime had the same race and returned a value-less Good. (a) SQL - QuestDB StartEnd: one scan, one GROUP BY — first()/last() for the any-quality picks, arg_min/arg_max keyed on the usable rows' time for the usable picks (no str_value: arg_min has no VARCHAR form; only Delta reads them). Byte-identical output to the old query on a 1M-row tag, ~15% faster. - SqlServer StartEnd: one windowed pass + conditional MAX (was five CTE references); identical output, 1.4-3x faster. - ActualTime kept on both backends: every single-scan form measured 1.2-2.4x slower; its inconsistency is absorbed by (b). (b) Defence in depth on both backends - ReadBucket takes DbDataReader and reads absent picks as null (ReadStoredOrNull) instead of throwing. - AggregateEvaluator: Delta with a missing first/last usable pick and ActualTime with a missing extreme are BadNoData (were BadAggregateInvalidInputs / a value-less Good); Start/End with no pick already were. - Unit tests (Sql.Common evaluator, QuestDb + SqlServer mappers over a DataTableReader) fail with the guards removed. LiveWriteReadRaceStressTests (QuestDB integration, opt-in via HISTORIAN_QUESTDB_STRESS_SECONDS, Category=SlowUnit): single-row writer vs 4 tight-loop readers. Old code: 6 faults in ~105k reads; new: 0 in 106k. Docs: Historian.md QuestDB processed-read note; F2 (Min/Max on the interval start: SDK raw, ours Calculated) and F3 (integer tags: SDK casts interpolated values to the integer type, ours Double) added to the SDK-parity deviations.
This commit is contained in:
@@ -25,6 +25,12 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common;
|
||||
/// sample is not usable.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Inconsistent rows</b> (live-gate 8 F1): a row whose counts say a bucket holds samples while a picked sample
|
||||
/// (first/last of any quality, first/last usable, the extreme) is absent is answered from what is present —
|
||||
/// Start/End/Delta/ActualTime without the sample they need are <c>BadNoData</c> — never an exception. The backends'
|
||||
/// queries read counts and picks in one scan so this should not arise; the evaluator does not rely on it.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Values: Count is <see cref="int"/>; Start/End keep the sample's type; every other aggregate is
|
||||
/// <see cref="double"/> computed over the numeric value (Boolean = 0/1). These are the simple per-sample
|
||||
/// definitions; the Part 13 time-weighted aggregates (TimeAverage, TimeAverage2, Total, Total2, Interpolative)
|
||||
@@ -76,12 +82,20 @@ public static class AggregateEvaluator
|
||||
case HistoryAggregateType.Delta:
|
||||
// #680 (SDK ComputeDelta): last usable − first usable; Uncertain only when the interval's first or last
|
||||
// sample was skipped (not usable) — a Bad sample in the middle does not degrade it.
|
||||
// Live-gate 8 F1: a row whose counts promise a usable sample its picks do not carry (a backend that read
|
||||
// the counts and the picks from two table versions) has no consistent first/last — no data, not a throw
|
||||
// and not a wrong-input verdict.
|
||||
if (r.First.Kind == HistorianValueKind.Empty || r.Last.Kind == HistorianValueKind.Empty)
|
||||
return NoData(bucketStart, now);
|
||||
if (!r.First.TryGetDouble(out var a) || !r.Last.TryGetDouble(out var b))
|
||||
return Calc(null, HistorianStatusCodes.BadAggregateInvalidInputs);
|
||||
var bounded = r.FirstAny is { IsUsable: true } && r.LastAny is { IsUsable: true };
|
||||
return Calc(b - a, bounded ? HistorianStatusCodes.Good : HistorianStatusCodes.UncertainDataSubNormal);
|
||||
case HistoryAggregateType.MinimumActualTime:
|
||||
case HistoryAggregateType.MaximumActualTime:
|
||||
// Live-gate 8 F1: usable numeric samples counted but no extreme picked (an inconsistent row) is no data,
|
||||
// never a value-less Good.
|
||||
if (r.Extreme is null || r.ExtremeTime is null) return NoData(bucketStart, now);
|
||||
return Calc(r.Extreme, status, r.ExtremeTime);
|
||||
default:
|
||||
throw new HistoryAggregateNotSupportedException(aggregate);
|
||||
|
||||
Reference in New Issue
Block a user