fix(historian): #680 review — recovery/only-sample TA2/Total2 value, outage-last + Boolean parity, docs

TimeAverage2/Total2: the interval the data ends inside with no non-Bad time
before its last sample (comms-loss recovery, a single sample) answers that
sample's value (Total2 0), Uncertain + Partial, instead of Bad/BadNoData —
as the SDK. Parity datasets RecoveryEnds, OutageLast, BoolSwitch with
documented masks; ProcessedConformance's reverse TA/Total mask narrowed to
the later-edge-sample interval. Docs: report-by-exception consequence,
outage-last and Boolean differences; CLAUDE.md historian bullet.
This commit is contained in:
Joseph Doherty
2026-10-11 16:02:22 -04:00
parent b61975f7d7
commit 0d206a0884
6 changed files with 218 additions and 16 deletions
+23 -3
View File
@@ -1002,6 +1002,16 @@ and the raw-row cap below). Bucketing (edges, truncated interval, reverse anchor
> two samples now has a value instead of `BadNoData`.
> - **Partial:** set on the forward end-of-data interval (`TimeAverage2`/`Total2`/percentages) and, on a reverse read, on the interval
> holding the latest sample when nothing follows it (all of this family but `Interpolative`).
> - ⚠️ **Report-by-exception tags (a slowly changing or deadband-filtered value) are the visible case.** A tag whose last stored
> sample lies *before* the window and that has nothing after it has, by Part 13, **ended its data**: every interval of
> `TimeAverage2` / `Total2` / `PercentGood` / `PercentBad` is **`BadNoData`**, and every interval of `TimeAverage` / `Total` is the
> held value, **Uncertain**. When the next sample arrives, a re-read of the same window returns values / `Good` for those intervals
> (the held value is then interpolated between real samples, not extrapolated). So a dashboard, a ScadaBridge consumer or any
> client that caches processed results for "closed" intervals can see an interval go `BadNoData` → value, or Uncertain → Good, on a
> later read — re-read recent intervals, or use `TimeAverage`/`Total` (which always carry a value) for a report-by-exception tag. The
> SDK's own calculators behave the same way. Likewise the interval holding a tag's latest sample is Partial and, until a later
> sample arrives, `TimeAverage2`/`Total2` there cover only up to it — the comms-loss recovery shape (a single Good sample after an
> outage marker, nothing after) answers that sample's value (`Total2` 0), Uncertain + Partial, never Bad/null.
**Server defaults, fixed:** `TreatUncertainAsBad = false`, `PercentDataBad = PercentDataGood = 100`, `UseSlopedExtrapolation = false`
(`ServerAggregateConfiguration` in the OPC UA server project — the SDK's own `AggregateManager` defaults, which a test pins). **A
@@ -1049,8 +1059,10 @@ are used", and the SDK's `AverageAggregateCalculator`):
value held up to an outage). The duration is the interval's, cut at the end of the data:
- After the end of the data ⇒ `BadNoData` (Part 13 "After End of Data"); the interval the data ends inside covers only up to the last
sample and is **Partial** (forward); when the data ends exactly on its start there is nothing to cover and the value is the last
sample's (`Total2` 0), with its status.
sample and is **Partial** (forward). When it has no non-Bad time before the last sample — the data ends exactly on its start, or the
last sample is the first usable one after an outage (comms-loss recovery), or the tag's only sample — there is nothing to cover and
the value is the last sample's (`Total2` 0): `Good` only when the data ends exactly on the interval start with a Good sample,
otherwise `UncertainDataSubNormal` (the SDK answers the same, modulo its 1 ms artifact).
- Every instant of the (covered) interval non-Bad ⇒ `Good`.
- Some non-Bad time but not all (an outage, or time before the first sample) ⇒ `UncertainDataSubNormal`.
- No non-Bad time but some outage time (an interval wholly inside an outage) ⇒ generic **`Bad`** (`0x80000000`), value null — Part 13's
@@ -1118,7 +1130,15 @@ ours follow Part 13: `TimeAverage2`/`Total2` stop at the last sample, the percen
read the SDK's backward slice files a sample exactly on an interval's later edge as its early bound — so a reverse `Interpolative` at
such a point comes back Interpolated (Uncertain if a Bad sample follows) and a reverse `TimeAverage`/`Total` interval ending on it takes
the status of the segment beyond it — where ours, like the SDK forward, returns the raw sample and keeps the interval's own segments
(an SDK defect: its forward and reverse answers differ).
(an SDK defect: its forward and reverse answers differ). Three more, kept deliberately (#680 review): an **outage marker as the last
sample** (nothing after it) runs Bad to the window end here — `TimeAverage2`/`Total2` generic `Bad` and the percentages 100 % Bad
after it, no end-of-data Partial on the interval holding it, and `TimeAverage`/`Total` Uncertain over the span the last usable value is
held up to the marker (no Good value follows ⇒ extrapolated) — where the SDK treats the data as ended at the marker (`BadNoData` after
it, Partial on that interval, the held span Good): the tag is in an outage, not out of data. **Booleans** are stepped here (a
non-analog value is held; the simple bound next to an Uncertain sample takes the earlier sample's status), while the SDK interpolates
them as numbers — e.g. an interval that is `false` for its first half and `true` for its second averages 0.5 here, 1 in the SDK, and a
PercentGood region ahead of an Uncertain Boolean is Good here, Uncertain in the SDK. Each is pinned by `TimeWeightedAggregatorTests` and
by a parity dataset (`OutageLast`, `BoolSwitch`) whose masks are documented in `SdkAggregateParityTests`.
**Pushdown trade-off.** Unlike the per-sample aggregates, the time-weighted family is **not** pushed down to SQL: the backend streams
the window's rows plus the four bounds (one batch of five result sets on SQL Server; a bounds `UNION ALL` + a window select on QuestDB)