fix(historian): #680 lane A — time-weighted aggregates follow Part 13
TimeAverage/Total: Uncertain when a Bad value is skipped, an Uncertain value is used, the value is extrapolated or the interval is not covered; an interval wholly inside an outage bridges (Uncertain); Total integrates only the covered part. TimeAverage2/Total2 stop at the end of the data (Partial, then BadNoData). PercentGood/PercentBad become duration-based and move to the time-weighted family (now subject to MaxTimeWeightedRawRows). Reverse reads set Partial on the interval holding the latest sample when nothing follows (SDK m_startOfData). Parity tests gain end-of-data and outage datasets; pre-existing #680 masks removed.
This commit is contained in:
Joseph Doherty
+102
-46
@@ -144,7 +144,7 @@ and all HistoryRead calls on historized nodes return `GoodNoData` (empty, not an
|
||||
| `MaxTieClusterOverfetch` | int | `65536` | (Node-manager paging; all backends — a `SqlServer` PK rules out tie clusters, so it is moot there.) Maximum samples the server will fetch in one shot to page through a tie cluster (multiple samples sharing one `SourceTimestamp`). A cluster larger than this ceiling fails `BadHistoryOperationUnsupported`. Raise to handle abnormally large tie clusters; the default covers all normal-data cases. |
|
||||
| `HistoryReadBatchParallelism` | int | `4` | Max historized nodes served **concurrently** within one HistoryRead batch (arch-review 03/S3). The SDK's `HistoryRead*` override surface is synchronous, so each arm bridges once at its boundary; this turns per-node serving from sequential (`N × CallTimeout` worst case) into `⌈N / P⌉ × CallTimeout`. Also caps the gateway-load multiplication a single client can command. `≤ 1` serves sequentially. |
|
||||
| `MaxConcurrentHistoryReadBatches` | int | `16` | Process-wide limit on HistoryRead (and HistoryUpdate) batches served **concurrently** (arch-review 03/S3). A flood of history clients degrades to `BadTooManyOperations` rather than exhausting the SDK request-thread pool. With `HistoryReadBatchParallelism` this caps total in-flight gateway reads at `MaxConcurrentHistoryReadBatches × HistoryReadBatchParallelism` (default 16 × 4 = 64). A batch that cannot get a slot within `min(HistoryReadDeadline, 5 s)` fails every handle `BadTooManyOperations`. |
|
||||
| `MaxTimeWeightedRawRows` | int | `1000000` | `SqlServer` / `QuestDb` only (#671). The most raw in-window samples one [time-weighted aggregate](#time-weighted-aggregates-part-13-671) read (TimeAverage, TimeAverage2, **Total**, Total2, Interpolative) may stream into the server-side evaluator — that family is **not** pushed down to SQL. The window query is capped at this + 1 rows; one row past the cap fails **that node** with `BadTooManyOperations` (not counted as a backend fault) instead of holding a pooled connection until `HistoryReadDeadline` and returning `BadTimeout`. Bound rows outside the window do not count; per-sample aggregates are not capped. 1 000 000 ≈ 11.5 days of a 1 Hz tag. Non-positive = unbounded (a `Validate()` warning). |
|
||||
| `MaxTimeWeightedRawRows` | int | `1000000` | `SqlServer` / `QuestDb` only (#671). The most raw in-window samples one [time-weighted aggregate](#time-weighted-aggregates-part-13-671) read (TimeAverage, TimeAverage2, **Total**, Total2, Interpolative, and — since #680 — **PercentGood/PercentBad**) may stream into the server-side evaluator — that family is **not** pushed down to SQL. The window query is capped at this + 1 rows; one row past the cap fails **that node** with `BadTooManyOperations` (not counted as a backend fault) instead of holding a pooled connection until `HistoryReadDeadline` and returning `BadTimeout`. Bound rows outside the window do not count; per-sample aggregates are not capped. 1 000 000 ≈ 11.5 days of a 1 Hz tag. Non-positive = unbounded (a `Validate()` warning). |
|
||||
| `HistoryReadDeadline` | TimeSpan | `00:01:00` | Per-request wall-clock deadline for a HistoryRead (and HistoryUpdate) batch (arch-review 03/S3). A node still in flight at the deadline surfaces `BadTimeout`, so one request can never hold an SDK request thread longer than this regardless of node count. Non-positive = unbounded (a `Validate()` warning). |
|
||||
| `HistoryUpdateMaxFutureSkew` | TimeSpan | `1.00:00:00` | #674. How far past the server clock a HistoryUpdate (Insert / Replace / Update) item's `SourceTimestamp` may lie; an item beyond `now + skew` is refused **`BadOutOfRange`** by the node manager before any backend sees it (every backend). Guards against a far-future typo (a 2062 timestamp). The live recorder is not bounded. Non-positive disables the bound (no warning — a deliberate choice). See [HistoryUpdate](#historyupdate). |
|
||||
|
||||
@@ -908,10 +908,10 @@ point (see [Continuation-point paging (Events)](#continuation-point-paging-event
|
||||
|
||||
### Aggregates (Processed reads)
|
||||
|
||||
> **Two kinds.** The *per-sample* aggregates (Average, Minimum, Maximum, Count, Range, Start/End/Delta, Min/MaxActualTime,
|
||||
> PercentGood/Bad) are simple per-sample definitions — no time-weighting, no interval-boundary values — pushed down to SQL. The
|
||||
> *time-weighted* aggregates (TimeAverage, TimeAverage2, Total, Total2, Interpolative; #671) follow OPC UA Part 13 with bounding values
|
||||
> — see [Time-weighted aggregates](#time-weighted-aggregates-part-13-671). Any `HistoryAggregateType` not in the tables throws
|
||||
> **Two kinds.** The *per-sample* aggregates (Average, Minimum, Maximum, Count, Range, Start/End/Delta, Min/MaxActualTime) are
|
||||
> simple per-sample definitions — no time-weighting, no interval-boundary values — pushed down to SQL. The *time-weighted* aggregates
|
||||
> (TimeAverage, TimeAverage2, Total, Total2, Interpolative; #671 — and the duration-based PercentGood/PercentBad since #680) follow
|
||||
> OPC UA Part 13 with bounding values — see [Time-weighted aggregates](#time-weighted-aggregates-part-13-671). Any `HistoryAggregateType` not in the tables throws
|
||||
> `HistoryAggregateNotSupportedException` ⇒ `BadAggregateNotSupported` for that node (e.g. `StartBound`, `DurationGood`,
|
||||
> `StandardDeviationPopulation`).
|
||||
>
|
||||
@@ -951,7 +951,9 @@ returned status carries the historian *Calculated* info bit, `BadNoData` include
|
||||
result for the interval the tag's data **starts inside** — its earliest stored sample (any quality) lies strictly between the interval's
|
||||
edges, so there is no bounding value before it and the data does not cover the whole interval. This follows the SDK's `m_startOfData`
|
||||
rule, which the SDK turns off for `Average` and in effect applies to forward reads only (a reverse read is fed latest-first, so its
|
||||
"start of data" is the latest sample, past the window). A reverse read therefore sets no start-of-data Partial — matching the SDK. Only a
|
||||
"start of data" is the latest sample, past the window). A reverse read therefore sets no start-of-data Partial — matching the SDK
|
||||
(the time-weighted family does set it, on a reverse read, on the interval holding the *latest* sample when nothing follows it — #680,
|
||||
see [Time-weighted aggregates](#time-weighted-aggregates-part-13-671)). Only a
|
||||
forward per-sample read of an aggregate other than `Average` fetches the tag's earliest sample (one seek, bounded by the window end);
|
||||
every other read skips that query. An earliest sample exactly on an edge, or before the interval, sets nothing. The time-weighted
|
||||
aggregates also keep setting Partial on the truncated interval (below).
|
||||
@@ -967,23 +969,39 @@ aggregates also keep setting Partial on the truncated interval (below).
|
||||
| `Start`, `End` | first / last usable sample by time (keeps the sample's type) | bucket start |
|
||||
| `Delta` | `End − Start` (numeric samples) | bucket start |
|
||||
| `MinimumActualTime`, `MaximumActualTime` | the usable sample with the smallest / largest `NumValue`; ties ⇒ earliest | **the sample's own time** |
|
||||
| `PercentGood` | `100 × (Good samples) / (all samples)` | bucket start |
|
||||
| `PercentBad` | `100 × (Bad samples) / (all samples)` | bucket start |
|
||||
|
||||
Status and edge rules:
|
||||
|
||||
- An **empty interval** (no samples) ⇒ `BadNoData`. **`Count` — and every value aggregate — over an interval whose samples are all Bad ⇒ `BadNoData`**, not `0`.
|
||||
- Value aggregates are `Good` when every sample in the bucket is Good and `UncertainDataSubNormal` when the bucket also holds any Bad **or Uncertain** sample.
|
||||
- `Count`, `Start`, `End`, `PercentGood`, `PercentBad` accept any value kind; the other aggregates (and `Delta`) on a bucket that holds a usable `String`/`DateTime` sample ⇒ `BadAggregateInvalidInputs` for that interval.
|
||||
- **`PercentGood` / `PercentBad` and Uncertain samples:** Uncertain samples are counted in the denominator but are **neither Good nor Bad**, so `PercentGood + PercentBad < 100` whenever a bucket holds Uncertain samples. Both report status `Good` for any non-empty bucket.
|
||||
- `Count`, `Start`, `End` accept any value kind; the other aggregates (and `Delta`) on a bucket that holds a usable `String`/`DateTime` sample ⇒ `BadAggregateInvalidInputs` for that interval.
|
||||
- `PercentGood` / `PercentBad` are **duration-based since #680** — see [Time-weighted aggregates](#time-weighted-aggregates-part-13-671).
|
||||
- Everything except `Count` and `Start`/`End` is a `double`; `Int64` samples aggregate as doubles and `Boolean` as 0/1.
|
||||
|
||||
### Time-weighted aggregates (Part 13, #671)
|
||||
|
||||
`TimeAverage`, `TimeAverage2`, `Total`, `Total2` and `Interpolative`, evaluated by the shared
|
||||
`Driver.Historian.Sql.Common/TimeWeightedAggregator.cs` for **both** SQL backends (so `SqlServer` and `QuestDb` agree exactly).
|
||||
**This whole family — `Total` included — is evaluated in C#, not pushed down to SQL** (see *Pushdown trade-off* and the raw-row cap
|
||||
below). Bucketing (edges, truncated interval, reverse anchoring, the 100 000-interval cap) is the same as above.
|
||||
`TimeAverage`, `TimeAverage2`, `Total`, `Total2`, `Interpolative` and (since #680) `PercentGood` / `PercentBad`, evaluated by the
|
||||
shared `Driver.Historian.Sql.Common/TimeWeightedAggregator.cs` for **both** SQL backends (so `SqlServer` and `QuestDb` agree exactly).
|
||||
**This whole family — `Total` and the percentages included — is evaluated in C#, not pushed down to SQL** (see *Pushdown trade-off*
|
||||
and the raw-row cap below). Bucketing (edges, truncated interval, reverse anchoring, the 100 000-interval cap) is the same as above.
|
||||
|
||||
> ⚠️ **#680 (client-visible): time-weighted results now follow Part 13 more closely** (`SqlServer` / `QuestDb`; the SDK's own
|
||||
> calculators agree except where noted under *SDK parity*):
|
||||
> - **`TimeAverage` / `Total` status** is `UncertainDataSubNormal` whenever the interval uses an Uncertain value, a segment bridged
|
||||
> across a Bad value, the stepped extrapolation past the last sample, or does not cover the interval — no longer the time-based
|
||||
> rule. **The latest interval of a live tag now reads Uncertain, not Good**, and so does a fully-covered interval bridged across an
|
||||
> outage.
|
||||
> - **`TimeAverage` / `Total` over an interval wholly inside an outage** (data on both sides) now return the bridged value,
|
||||
> Uncertain — they used to return generic `Bad`, value null.
|
||||
> - **`Total` over an interval the data starts inside** integrates only the covered part (it used to be `TimeAverage × interval`).
|
||||
> - **`TimeAverage2` / `Total2` no longer extrapolate past the end of the data** (the last sample, when nothing at or after the window
|
||||
> end exists and it is usable): the interval holding it covers only up to it and is **Partial**; later intervals are **`BadNoData`**
|
||||
> (they used to hold the last value, Good). A one-interval read over the whole history therefore averages the stored data only.
|
||||
> - **`PercentGood` / `PercentBad` are duration-based** (they counted samples), moved to this family — so they are now subject to
|
||||
> `ServerHistorian:MaxTimeWeightedRawRows` (`BadTooManyOperations` past the cap) — and an interval with no sample of its own between
|
||||
> two samples now has a value instead of `BadNoData`.
|
||||
> - **Partial:** set on the forward end-of-data interval (`TimeAverage2`/`Total2`/percentages) and, on a reverse read, on the interval
|
||||
> holding the latest sample when nothing follows it (all of this family but `Interpolative`).
|
||||
|
||||
**Server defaults, fixed:** `TreatUncertainAsBad = false`, `PercentDataBad = PercentDataGood = 100`, `UseSlopedExtrapolation = false`
|
||||
(`ServerAggregateConfiguration` in the OPC UA server project — the SDK's own `AggregateManager` defaults, which a test pins). **A
|
||||
@@ -1001,40 +1019,71 @@ still gets a value.
|
||||
|
||||
**The signal.** Between a usable sample `p` and the next sample `s`: both `Double`/`Int64` ⇒ **sloped** (linear), segment quality the
|
||||
worse of the two; either `Boolean` ⇒ **stepped** (`p` held), quality `p`'s; `s` Bad ⇒ `p` held up to the outage (never interpolated into
|
||||
it — the AtTime rule), Uncertain; no later sample at all ⇒ `p` held to the window end (stepped extrapolation), Uncertain. From a Bad
|
||||
sample (incl. an `Empty` outage marker) to the next sample the signal is **Bad** (outage); before the first sample there is **no
|
||||
data**. That is the
|
||||
*simple-bounding* signal. The *interpolated-bounding* signal skips Bad samples: it joins consecutive usable samples (sloped / stepped)
|
||||
straight across an outage.
|
||||
it — the AtTime rule), Uncertain. From a Bad sample (incl. an `Empty` outage marker) to the next sample the signal is **Bad**
|
||||
(outage) — an outage marker with nothing after it runs to the window end; before the first sample there is **no data**; after a
|
||||
*usable* last sample with nothing at or after the window end the data has **ended** (#680, Part 13 "After End of Data": no
|
||||
extrapolation). That is the *simple-bounding* signal. The *interpolated-bounding* signal skips Bad samples: it joins consecutive usable
|
||||
samples (sloped / stepped) straight across an outage, and holds the last usable value to the window end (stepped extrapolation).
|
||||
|
||||
| Aggregate | Value | Bad intervals |
|
||||
|---|---|---|
|
||||
| `TimeAverage` | ∫ interpolated-bounding signal ÷ its defined duration | interpolated across (Part 13) |
|
||||
| `Total` | `TimeAverage × interval seconds` (value·s; the truncated last bucket uses its own length) | interpolated across |
|
||||
| `Total` | ∫ interpolated-bounding signal over its defined part (value·s) — `TimeAverage × interval seconds` on a covered interval; only the covered part when the data starts inside (#680) | interpolated across |
|
||||
| `TimeAverage2` | ∫ simple-bounding signal over non-Bad time ÷ non-Bad duration | excluded |
|
||||
| `Total2` | ∫ simple-bounding signal over non-Bad time (value·s) | excluded |
|
||||
| `Interpolative` | interpolated-bounding value **at the interval start** (reverse read: see below) | interpolated across |
|
||||
| `PercentGood` / `PercentBad` | `100 ×` Good / Bad time ÷ interval duration (#680, Part 13 §5.4.3.31-34) — see below | counted as Bad time |
|
||||
|
||||
**Status, the four time-weighted values** — aligned with the OPC Foundation SDK's `AggregateCalculator.GetTimeBasedStatusCode`
|
||||
(inspected in `Opc.Ua.Server` 1.5.378.106) under the fixed defaults. With `TreatUncertainAsBad = false`, **Uncertain time counts as
|
||||
Good time**: Uncertain samples, a value held up to an outage and the stepped extrapolation past the last sample all count toward
|
||||
PercentDataGood.
|
||||
**Status, `TimeAverage` / `Total`** (interpolated bounding, #680 — Part 13 "Uncertain if Bad values are skipped or Uncertain values
|
||||
are used", and the SDK's `AverageAggregateCalculator`):
|
||||
|
||||
- Every instant of the interval non-Bad ⇒ `Good`.
|
||||
- No defined value anywhere in the interval (before the first usable sample) ⇒ `BadNoData`.
|
||||
- Any part of the interval uses an Uncertain value (either end of a sloped segment, the start of a stepped one), lies on a segment
|
||||
bridged across a Bad sample, is the stepped extrapolation past the last usable sample, or is not covered (the data starts inside it)
|
||||
⇒ `UncertainDataSubNormal`. A segment bridged across an outage marks **every** interval it touches, not just the one holding the Bad
|
||||
sample.
|
||||
- Otherwise ⇒ `Good`.
|
||||
|
||||
**Status, `TimeAverage2` / `Total2`** — the SDK's `AggregateCalculator.GetTimeBasedStatusCode` (inspected in `Opc.Ua.Server`
|
||||
1.5.378.106) under the fixed defaults. With `TreatUncertainAsBad = false`, **Uncertain time counts as Good time** (Uncertain samples, a
|
||||
value held up to an outage). The duration is the interval's, cut at the end of the data:
|
||||
|
||||
- After the end of the data ⇒ `BadNoData` (Part 13 "After End of Data"); the interval the data ends inside covers only up to the last
|
||||
sample and is **Partial** (forward); when the data ends exactly on its start there is nothing to cover and the value is the last
|
||||
sample's (`Total2` 0), with its status.
|
||||
- Every instant of the (covered) interval non-Bad ⇒ `Good`.
|
||||
- Some non-Bad time but not all (an outage, or time before the first sample) ⇒ `UncertainDataSubNormal`.
|
||||
- No non-Bad time but some outage time ⇒ generic **`Bad`** (`0x80000000`) — the SDK's all-Bad result, value null.
|
||||
- No non-Bad time but some outage time (an interval wholly inside an outage) ⇒ generic **`Bad`** (`0x80000000`), value null — Part 13's
|
||||
PercentDataBad rule. (The SDK answers `BadNoData` there; see *SDK parity*.)
|
||||
- No data at all ⇒ `BadNoData`.
|
||||
- A usable `String`/`DateTime` sample feeding the interval ⇒ `BadAggregateInvalidInputs`.
|
||||
|
||||
All carry the Calculated bit; a non-Bad result for the **truncated interval** (shorter than the processing interval — the latest one
|
||||
forward, the earliest one reverse) or for the interval the tag's **data starts inside** (#677, see *Partial bit* above) also carries the
|
||||
**Partial** bit (`0x4`, `AggregateBits.Partial`). Timestamp = interval start (reverse: its later edge); value `double` (Boolean 0/1).
|
||||
For all four, a usable `String`/`DateTime` sample feeding the interval ⇒ `BadAggregateInvalidInputs`.
|
||||
|
||||
> **A live tag's latest interval.** Past the tag's last stored sample the value is held (stepped extrapolation, Uncertain). Under
|
||||
> the SDK rule that Uncertain time is Good time, so the latest interval of a live tag normally reads **`Good`** — not Uncertain —
|
||||
> with the last value held to the interval end. It reads Uncertain only if the interval also contains outage time or starts
|
||||
> before the tag's first sample. Do not take a Good latest interval as proof that data arrived up to its end. If it was truncated
|
||||
> by the request's end time it also carries **Partial**.
|
||||
**`PercentGood` / `PercentBad`** (#680, Part 13 §5.4.3.31-34 — they used to count samples). The interval is cut into **stepped
|
||||
regions**, each with the status of the sample that starts it: Good regions count toward `PercentGood`, Bad ones (incl. outage markers)
|
||||
toward `PercentBad`, Uncertain ones toward neither (so the two sum to < 100 when Uncertain data is present). The region that starts an
|
||||
interval before its first own sample takes the status of the *simple bounding value* there (the SDK's `GetSimpleBound`): Bad after a
|
||||
Bad sample, Uncertain on a segment that runs into an outage or slopes to/from an Uncertain sample, else the earlier sample's. Time
|
||||
before the tag's first sample is **Bad**. When the data ends (a usable last sample, nothing at or after the window end) its region runs
|
||||
to the **end of its own interval** (Part 13 — the SDK gives it 1 ms) and later intervals are `BadNoData`; an outage marker with nothing
|
||||
after it is Bad to the window end. An interval that ends before the first sample ⇒ `BadNoData`. Any value kind is accepted; the status
|
||||
is `Good` (plus Calculated / Partial). Value = `100 × region time ÷ interval duration`.
|
||||
|
||||
All carry the Calculated bit. A non-Bad result also carries the **Partial** bit (`0x4`, `AggregateBits.Partial`) — never on a Bad
|
||||
result — for: the **truncated interval** (shorter than the processing interval — the latest one forward, the earliest one reverse;
|
||||
Part 13 §5.3.3.2 Case 3 requires it — the SDK does not set it, a deliberate difference); the interval the tag's **data starts inside**
|
||||
(#677, forward, see *Partial bit* above); on a **reverse** read, the interval the **latest** sample lies strictly inside when nothing
|
||||
follows it (where a reverse read's data starts — the SDK's `m_startOfData`, keyed on the first value it is fed, #680); and, forward,
|
||||
the **end-of-data** interval of `TimeAverage2` / `Total2` / the percentages (the SDK's `m_endOfData` rule, which it skips for an
|
||||
extrapolated `TimeAverage` / `Total`). Timestamp = interval start (reverse: its later edge); value `double` (Boolean 0/1).
|
||||
|
||||
> **A live tag's latest interval (#680 — changed, client-visible).** Past the tag's last stored sample `TimeAverage` / `Total` hold the
|
||||
> last value to the interval end (stepped extrapolation), and that interval now reads **`UncertainDataSubNormal`** — the extrapolated
|
||||
> part is not data. (It used to read `Good`, because the time-based rule counted the extrapolated time as Good time.) `TimeAverage2` /
|
||||
> `Total2` do not extrapolate: they average / integrate only up to the last sample and set **Partial**; any later interval is
|
||||
> `BadNoData`. A `Good` latest interval therefore now does mean the data covered it — for these aggregates, the latest interval of a
|
||||
> tag that has not yet received a sample at or after the request's end time is never `Good` (`TimeAverage`/`Total`) or always
|
||||
> `Partial` (`TimeAverage2`/`Total2`, forward). If it was truncated by the request's end time it also carries **Partial**.
|
||||
|
||||
**Status, `Interpolative`** (point-based, not the PercentDataGood rule):
|
||||
- A usable sample exactly at the point ⇒ that value with **its own stored status** (no historian bit).
|
||||
@@ -1051,19 +1100,25 @@ value at **10:00**, not 09:00, and no point ever falls before the window. The in
|
||||
**SDK parity (#677).** `SdkAggregateParityTests` (unit, `Sql.Common.Tests`) and
|
||||
`ProcessedConformance.Processed_reads_match_the_SDK_aggregate_calculators` (both SQL backends, through the real SQL) run the pinned SDK's
|
||||
own calculators (`AverageAggregateCalculator`, `MinMaxAggregateCalculator`, `CountAggregateCalculator`, the base `AggregateCalculator`
|
||||
for Interpolative) over the same samples, forward and reverse, and compare every interval's timestamp, value, status code and Partial
|
||||
bit. The per-sample comparison covers Average, Minimum, Maximum, Count, Range, Start, End, Delta, Minimum/MaximumActualTime and
|
||||
PercentGood/Bad; reverse reads over data that starts inside the window are compared too. Known differences that remain (tracked as
|
||||
for Interpolative, `StatusAggregateCalculator` for PercentGood/Bad) over the same samples, forward and reverse, and compare every
|
||||
interval's timestamp, value, status code and Partial bit. The per-sample comparison covers Average, Minimum, Maximum, Count, Range,
|
||||
Start, End, Delta and Minimum/MaximumActualTime; the time-weighted one TimeAverage, TimeAverage2, Total, Total2, Interpolative and
|
||||
PercentGood/Bad (#680), including datasets whose data ends inside the window and with an interval wholly inside an outage; reverse
|
||||
reads over data that starts inside the window are compared too. Known differences that remain (tracked as
|
||||
#680) — documented and masked in the tests, deliberately not changed here because forward results were to stay as they are — are:
|
||||
`Start`/`End` (the SDK returns the raw first/last sample, of any quality, with its own timestamp and stored status; ours returns the
|
||||
first/last usable value stamped with the interval, under the per-sample status rule — the values agree wherever the SDK's is not Bad);
|
||||
`Delta` status (the SDK's stays Good with a Bad sample in the interval, ours is Uncertain); `PercentGood`/`PercentBad` (ours count
|
||||
samples, the SDK measures duration — different definitions, so only timestamp and status are compared); the SDK reports `TimeAverage`/`Total` Uncertain whenever any region of the interval is non-Good, where ours applies the
|
||||
time-based rule (a fully-covered interval bridged across a Bad sample is Good here); `Total` over an interval the data starts inside is
|
||||
`TimeAverage × interval` here but the integral of the covered part in the SDK; ours flags a truncated interval Partial, the SDK does
|
||||
not; the SDK's forward end-of-data Partial rule is not implemented; the SDK's per-sample Count answers 0/Good for an empty interval
|
||||
inside the data and its per-sample aggregates use Good samples only; and on a reverse `Interpolative` read the SDK marks a sample exactly
|
||||
at a point Interpolated (Uncertain if a Bad sample follows) where ours — like the SDK forward — returns the raw sample.
|
||||
`Delta` status (the SDK's stays Good with a Bad sample in the interval, ours is Uncertain); the SDK's per-sample Count answers
|
||||
0/Good for an empty interval inside the data and its per-sample aggregates use Good samples only. Time-weighted differences (#680 closed
|
||||
the TimeAverage/Total status, `Total` coverage, end-of-data and PercentGood/Bad ones), each deliberate: ours flags a truncated interval
|
||||
Partial (Part 13 §5.3.3.2 Case 3), the SDK does not; ours carries the Calculated bit on a `BadNoData`; `TimeAverage2`/`Total2` over an
|
||||
interval wholly inside an outage are generic `Bad` here (Part 13's PercentDataBad rule), `BadNoData` in the SDK; in the end-of-data
|
||||
interval the SDK gives the region after the last sample 1 ms (so `Total2` there differs by ~1 ms × value and the percentages by a lot —
|
||||
ours follow Part 13: `TimeAverage2`/`Total2` stop at the last sample, the percentages' region runs to the interval end); and on a reverse
|
||||
read the SDK's backward slice files a sample exactly on an interval's later edge as its early bound — so a reverse `Interpolative` at
|
||||
such a point comes back Interpolated (Uncertain if a Bad sample follows) and a reverse `TimeAverage`/`Total` interval ending on it takes
|
||||
the status of the segment beyond it — where ours, like the SDK forward, returns the raw sample and keeps the interval's own segments
|
||||
(an SDK defect: its forward and reverse answers differ).
|
||||
|
||||
**Pushdown trade-off.** Unlike the per-sample aggregates, the time-weighted family is **not** pushed down to SQL: the backend streams
|
||||
the window's rows plus the four bounds (one batch of five result sets on SQL Server; a bounds `UNION ALL` + a window select on QuestDB)
|
||||
@@ -1077,7 +1132,8 @@ judged the larger risk. For very long, densely sampled windows prefer the pushed
|
||||
SQL backends; `TOP (@maxRows)` / `LIMIT $4`). When that extra row arrives the read stops and the node fails with
|
||||
**`BadTooManyOperations`** — for that node only; it is not counted in the backend's health failures. Without the cap a one-year
|
||||
`Total` on a 1 Hz tag (≈31 M rows) would hold a pooled connection until `HistoryReadDeadline` and then return `BadTimeout`. The four
|
||||
bound rows do not count toward the cap, and the per-sample aggregates are never capped. Raise the cap, or read in shorter windows.
|
||||
bound rows do not count toward the cap, and the per-sample aggregates are never capped. Since #680 `PercentGood`/`PercentBad` are in
|
||||
this family and **are** capped (they used to be pushed down). Raise the cap, or read in shorter windows.
|
||||
|
||||
### Health
|
||||
|
||||
|
||||
Reference in New Issue
Block a user