Historian aggregates: Average/Minimum/Maximum include Uncertain samples (Part 13 / SDK use Good only) #684

New Issue

2026-10-11T16:38:14-04:00

dohertj2 commented

2026-10-11 16:38:14 -04:00

Left out of scope in #680. On SqlServer/QuestDB the per-sample Average/Minimum/Maximum (and Range, *ActualTime) value columns use Good and Uncertain samples (u in both SQL builders), consistent with treating TreatUncertainAsBad=false. Part 13 says these aggregates use "Good values" and the SDK AggregateCalculator uses Good only, so results differ whenever an interval holds an Uncertain sample. SdkAggregateParityTests would need an Uncertain-bearing dataset to pin it.

Decide: align (switch the value columns from u to g in SqlAggregateQueries/QuestDbAggregateQueries, status rules per the spec tables) or document as intentional (and say which AggregateConfiguration it corresponds to — the server rejects non-default client configurations since #677).

Left out of scope in #680. On SqlServer/QuestDB the per-sample Average/Minimum/Maximum (and Range, *ActualTime) value columns use Good **and** Uncertain samples (`u` in both SQL builders), consistent with treating `TreatUncertainAsBad=false`. Part 13 says these aggregates use "Good values" and the SDK `AggregateCalculator` uses Good only, so results differ whenever an interval holds an Uncertain sample. `SdkAggregateParityTests` would need an Uncertain-bearing dataset to pin it. Decide: align (switch the value columns from `u` to `g` in `SqlAggregateQueries`/`QuestDbAggregateQueries`, status rules per the spec tables) or document as intentional (and say which `AggregateConfiguration` it corresponds to — the server rejects non-default client configurations since #677).

dohertj2 referenced this issue

2026-10-12 08:38:37 -04:00

Time-weighted aggregates: forward differences from the SDK AggregateCalculator (TimeAverage/Total status, Total at start of data, Partial on truncated) #680

dohertj2 commented

2026-10-12 13:59:26 -04:00

Fixed (aligned) — merged dbdf29c8. Part 13 defines Average/Minimum/Maximum/Range (and *ActualTime) over Good raw values; §4.2.1.2's TreatUncertainAsBad=false applies "unless the Aggregate definition says otherwise", and these definitions do (confirmed by the OPC Foundation on its forum, and by the SDK AggregateCalculator). So the Good+Uncertain value columns were a defect, not a configuration choice; the advertised TreatUncertainAsBad=false stays accurate (it governs the time-weighted family and status calculations).

Now on SqlServer, QuestDB (and Gateway via #681):

values use Good samples only; an interval with only Uncertain/Bad samples ⇒ BadNoData;
Minimum/Maximum/*ActualTime/Range status per the Part 13 tables: Uncertain_DataSubNormal if a Bad value exists or an Uncertain value lies beyond the Good extreme (deliberate spec-over-SDK deviation — the SDK computes those extremes and never reads them);
Average keeps the SDK value-based status (Good only if every sample is Good);
an Uncertain String/DateTime sample no longer refuses the interval (only a Good one does, ⇒ BadAggregateInvalidInputs).

⚠️ Client-visible: results change whenever an interval holds Uncertain samples.

Verification: DenseUncertain/UncertainBeyond/UncertainText parity datasets vs the SDK (unmasked except the documented deviation), real-SQL conformance (Uncertain above/below the Good range, Uncertain-only interval, Uncertain string), falsified; live gate 9 G2 on both backends (byte-identical between them) — every SDK difference is a documented one.

**Fixed (aligned) — merged `dbdf29c8`.** Part 13 defines Average/Minimum/Maximum/Range (and *ActualTime) over **Good** raw values; §4.2.1.2's TreatUncertainAsBad=false applies "unless the Aggregate definition says otherwise", and these definitions do (confirmed by the OPC Foundation on its forum, and by the SDK `AggregateCalculator`). So the Good+Uncertain value columns were a defect, not a configuration choice; the advertised `TreatUncertainAsBad=false` stays accurate (it governs the time-weighted family and status calculations). Now on SqlServer, QuestDB (and Gateway via #681): - values use Good samples only; an interval with only Uncertain/Bad samples ⇒ `BadNoData`; - Minimum/Maximum/*ActualTime/Range status per the Part 13 tables: `Uncertain_DataSubNormal` if a Bad value exists **or an Uncertain value lies beyond the Good extreme** (deliberate spec-over-SDK deviation — the SDK computes those extremes and never reads them); - Average keeps the SDK value-based status (Good only if every sample is Good); - an Uncertain String/DateTime sample no longer refuses the interval (only a Good one does, ⇒ `BadAggregateInvalidInputs`). ⚠️ Client-visible: results change whenever an interval holds Uncertain samples. Verification: `DenseUncertain`/`UncertainBeyond`/`UncertainText` parity datasets vs the SDK (unmasked except the documented deviation), real-SQL conformance (Uncertain above/below the Good range, Uncertain-only interval, Uncertain string), falsified; live gate 9 G2 on both backends (byte-identical between them) — every SDK difference is a documented one.

dohertj2 referenced this issue from a commit

2026-10-12 13:59:26 -04:00

fix(historian): per-sample Average/Min/Max/Range/ActualTime use Good samples only, Part 13 min/max status (#684)

dohertj2 referenced this issue from a commit

2026-10-12 13:59:26 -04:00

fix(historian): #684 review — Uncertain non-numeric samples do not refuse the Good-value aggregates; real-SQL Uncertain-above-max test

dohertj2 referenced this issue from a commit

2026-10-12 13:59:26 -04:00

Merge fix/684-uncertain-samples: Average/Min/Max/Range/*ActualTime use Good samples only; Min/Max status per Part 13 (#684)

dohertj2 referenced this issue from a commit

2026-10-12 13:59:26 -04:00

fix(historian): #681 on #682/#684 — Gateway served set is the 12-member ServedAggregates; raw buckets use Good-only values