fix(historian): live-gate 8 F1 — per-sample counts and picks from one table version (#680 follow-up)

QuestDB Start/End/Delta reads of a tag being written intermittently failed
BadHistoryOperationUnsupported (InvalidCastException: Column 'ts_3' is null):
the StartEnd query referenced its CTE four times and QuestDB opens a table
reader per reference, so a row committed between two of them was counted but
not picked. MinimumActualTime/MaximumActualTime had the same race and returned
a value-less Good.

(a) SQL
- QuestDB StartEnd: one scan, one GROUP BY — first()/last() for the
  any-quality picks, arg_min/arg_max keyed on the usable rows' time for the
  usable picks (no str_value: arg_min has no VARCHAR form; only Delta reads
  them). Byte-identical output to the old query on a 1M-row tag, ~15% faster.
- SqlServer StartEnd: one windowed pass + conditional MAX (was five CTE
  references); identical output, 1.4-3x faster.
- ActualTime kept on both backends: every single-scan form measured 1.2-2.4x
  slower; its inconsistency is absorbed by (b).

(b) Defence in depth on both backends
- ReadBucket takes DbDataReader and reads absent picks as null
  (ReadStoredOrNull) instead of throwing.
- AggregateEvaluator: Delta with a missing first/last usable pick and
  ActualTime with a missing extreme are BadNoData (were BadAggregateInvalidInputs
  / a value-less Good); Start/End with no pick already were.
- Unit tests (Sql.Common evaluator, QuestDb + SqlServer mappers over a
  DataTableReader) fail with the guards removed.

LiveWriteReadRaceStressTests (QuestDB integration, opt-in via
HISTORIAN_QUESTDB_STRESS_SECONDS, Category=SlowUnit): single-row writer vs
4 tight-loop readers. Old code: 6 faults in ~105k reads; new: 0 in 106k.

Docs: Historian.md QuestDB processed-read note; F2 (Min/Max on the interval
start: SDK raw, ours Calculated) and F3 (integer tags: SDK casts interpolated
values to the integer type, ours Double) added to the SDK-parity deviations.
This commit is contained in:
Joseph Doherty
2026-10-11 18:28:29 -04:00
parent 11f792538e
commit 2ceed561ad
13 changed files with 549 additions and 54 deletions
@@ -106,7 +106,7 @@ internal sealed class SqlHistorianConnection
/// <param name="reader">Reader positioned on a row.</param>
/// <param name="ordinal">Column ordinal.</param>
/// <returns>UTC timestamp.</returns>
public static DateTime GetUtc(SqlDataReader reader, int ordinal) =>
public static DateTime GetUtc(System.Data.Common.DbDataReader reader, int ordinal) =>
DateTime.SpecifyKind(reader.GetDateTime(ordinal), DateTimeKind.Utc);
/// <summary>True for connection/timeout/deadlock/throttling errors worth retrying.</summary>