dohertj2/lmxopcua
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(historian): per-sample Average/Min/Max/Range/ActualTime use Good samples only, Part 13 min/max status (#684)

Browse Source 
Part 13 defines Average/Minimum/Maximum/Range/*ActualTime over Good raw values
(independent of TreatUncertainAsBad) and the SDK's calculators skip Uncertain values
under the server's TreatUncertainAsBad=false. SqlServer/QuestDB value columns now use
the good flag (g) instead of usable (u); new UncMin/UncMax columns feed the Part 13
Minimum/Maximum/Range status rule (Uncertain on a Bad sample or an Uncertain value
beyond the Good extreme -- the SDK applies only the Bad half). An Uncertain-only
interval is BadNoData. Uncertain-bearing parity datasets (in-memory + real SQL) match
the SDK unmasked; the beyond-extreme divergence is pinned explicitly.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Joseph Doherty
2026-10-12 09:03:47 -04:00
parent 113f1eaf7c
commit a11ab0dc08
14 changed files with 533 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/Historian.md
+28 -6
View File
@@ -996,7 +996,27 @@ point (see [Continuation-point paging (Events)](#continuation-point-paging-event
> so a "live" tag's latest interval is now flagged.
>
> A client that read `Count` as "usable samples", keyed `Start`/`End` on the interval start, or treated `Count = BadNoData` as "no
> samples" must adjust. Not changed: `Average`/`Minimum`/`Maximum` still use Good + Uncertain samples (the SDK uses Good only).
> samples" must adjust. (`Average`/`Minimum`/`Maximum` were left on Good + Uncertain samples here; #684, below, moved them to Good only.)
> ⚠️ **#684 (client-visible): `Average`, `Minimum`, `Maximum`, `Range` and `Minimum`/`MaximumActualTime` use GOOD samples only** on
> the `SqlServer` and `QuestDb` backends (they used every *usable* sample — Good **and** Uncertain). Part 13 defines each over "Good
> raw values" (§5.4.3.5, .10–.14), independent of `TreatUncertainAsBad` (only `Minimum2`/`Maximum2`, not served, consult it), and the
> SDK's `AverageAggregateCalculator`/`MinMaxAggregateCalculator` skip Uncertain values under the server's own `TreatUncertainAsBad = false`.
> - An Uncertain sample is **never part of the value** (an interval `Good 6, Good 8, Uncertain 4` now averages 7, was 6; its Minimum is
> 6, was 4), and an interval holding **only** Uncertain (and Bad) samples is now **`BadNoData`** (it used to return their value,
> Uncertain).
> - **Status:** `Average` is value-based (Good only when every sample is Good, else `UncertainDataSubNormal` — unchanged, the SDK's
> rule). `Minimum`/`Maximum`/`Range`/`*ActualTime` follow their Part 13 tables: `UncertainDataSubNormal` when a **Bad** sample
> exists, or an **Uncertain value lies beyond the reported Good extreme** (below the minimum for Minimum/MinimumActualTime, above the
> maximum for Maximum/MaximumActualTime, either for Range; strictly — equal does not count); otherwise **Good**. So an Uncertain
> sample inside the Good range no longer makes Minimum/Maximum Uncertain (it used to).
> - The SDK applies only the Bad half of that rule (its `ComputeMinMax` collects the Uncertain extremes and never reads them) and answers
> Good for an Uncertain value beyond the extreme; ours follows Part 13 there — the one remaining per-sample Uncertain difference, pinned
> by `SdkAggregateParityTests`.
> - `Count` (Good only, #680), `Start`/`End`/`Delta` (usable / any-quality picks) and the time-weighted family (Uncertain time counts as
> Good time under `TreatUncertainAsBad = false`, as Part 13 §5.4.3.2 says) are unchanged. The advertised `TreatUncertainAsBad = false`
> stays correct: it governs the time-weighted and status calculations; the per-sample aggregates above are defined over Good values
> whatever its setting.
**Bucketing.** Forward: one result per interval of `[start, end)`; bucket *i* = `[start + i·interval, start + (i+1)·interval)`,
start-inclusive / end-exclusive to the tick, the last bucket truncated at `end`, each stamped with its start. Reverse (`end < start`,
@@ -1043,17 +1063,18 @@ included); a per-sample `Average` read skips them.
| Aggregate | Computed over | Value timestamp |
|---|---|---|
| `Average`, `Minimum`, `Maximum` | `AVG` / `MIN` / `MAX` of `NumValue` over usable samples | bucket start (reverse read: its later edge, as every row marked "bucket start" here) |
| `Average`, `Minimum`, `Maximum` | `AVG` / `MIN` / `MAX` of `NumValue` over **Good** samples (#684) | bucket start (reverse read: its later edge, as every row marked "bucket start" here) |
| `Count` | number of **Good** samples (`Int32`; #680) | bucket start |
| `Range` | `Maximum − Minimum` | bucket start |
| `Range` | `Maximum − Minimum` (Good samples) | bucket start |
| `Start`, `End` | the **raw** first / last sample of any quality (its type, its own status; value null when Bad — #680) | **the sample's own time** |
| `Delta` | last usable − first usable value (numeric samples) | bucket start |
| `MinimumActualTime`, `MaximumActualTime` | the usable sample with the smallest / largest `NumValue`; ties ⇒ earliest | **the sample's own time** |
| `MinimumActualTime`, `MaximumActualTime` | the **Good** sample (#684) with the smallest / largest `NumValue`; ties ⇒ earliest | **the sample's own time** |
Status and edge rules:
- An **empty interval** (no samples) ⇒ `BadNoData` — except **`Count`**, which is `0`, Good, when the interval lies inside the tag's data range `[first sample, last sample]` (both edges inclusive, either direction; #680). **Every value aggregate over an interval whose samples are all Bad ⇒ `BadNoData`**; `Count` there is the generic **Bad** (value null).
- Value aggregates are `Good` when every sample in the bucket is Good and `UncertainDataSubNormal` when the bucket also holds any Bad **or Uncertain** sample. `Count`'s status is value-based (all Good ⇒ Good, all Bad ⇒ Bad, else Uncertain); `Start`/`End` carry the raw sample's own status; `Delta` is Good unless the interval's first or last sample is not usable (then Uncertain) — #680.
- An interval with samples but **no Good numeric value** (only Uncertain / Bad) ⇒ `BadNoData` for `Average`/`Minimum`/`Maximum`/`Range`/`*ActualTime` (#684).
- `Average` is `Good` when every sample in the bucket is Good, else `UncertainDataSubNormal` (value-based, the SDK's rule). `Minimum`/`Maximum`/`Range`/`*ActualTime` are `UncertainDataSubNormal` when the bucket holds a Bad sample or an Uncertain value beyond the reported Good extreme (Part 13 Tables 57–62; strictly beyond), else `Good` (#684). `Count`'s status is value-based (all Good ⇒ Good, all Bad ⇒ Bad, else Uncertain); `Start`/`End` carry the raw sample's own status; `Delta` is Good unless the interval's first or last sample is not usable (then Uncertain) — #680.
- `Count`, `Start`, `End` accept any value kind; the other aggregates (and `Delta`) on a bucket that holds a usable `String`/`DateTime` sample ⇒ `BadAggregateInvalidInputs` for that interval.
- `PercentGood` / `PercentBad` are **duration-based since #680** — see [Time-weighted aggregates](#time-weighted-aggregates-part-13-671).
- Everything except `Count` and `Start`/`End` is a `double`; `Int64` samples aggregate as doubles and `Boolean` as 0/1.
@@ -1198,7 +1219,8 @@ Start, End, Delta and Minimum/MaximumActualTime; the time-weighted one TimeAvera
PercentGood/Bad (#680), including datasets whose data ends inside the window and with an interval wholly inside an outage; reverse
reads over data that starts inside the window are compared too. Known differences that remain (tracked as
#680) — documented and masked in the tests, deliberately not changed here because forward results were to stay as they are — are:
the SDK's per-sample Average/Minimum/Maximum use Good samples only, ours Good + Uncertain; the SDK sets Partial on Bad per-sample
an Uncertain value beyond the Good extreme makes Minimum/Maximum/Range/*ActualTime Uncertain here (Part 13), Good in the SDK
(#684 aligned the values — Good samples only — and every other Uncertain case); the SDK sets Partial on Bad per-sample
results too, ours never (#680 closed the Start/End, Delta, Count and end-of-data Partial differences). Time-weighted differences (#680 closed
the TimeAverage/Total status, `Total` coverage, end-of-data and PercentGood/Bad ones), each deliberate: ours flags a truncated interval
Partial (Part 13 §5.3.3.2 Case 3), the SDK does not; ours carries the Calculated bit on a `BadNoData`; `TimeAverage2`/`Total2` over an