Merge feat/680-time-weighted: time-weighted aggregates SDK/Part 13 parity — TA/Total status, covered-part Total, outage bridging, duration Percent*, TA2/Total2 end of data (#680)
This commit is contained in:
Joseph Doherty
@@ -1350,7 +1350,7 @@ historian tagname, which is the tag's driver `FullName`.
|
||||
### Pluggable backends, SQL Server and HistoryUpdate (2026-10)
|
||||
|
||||
- **Selection:** `ServerHistorian:Backend` (`Gateway` default | `SqlServer` | `QuestDb`; unknown fails host start). `HistorianBackendRegistry` → `IHistorianBackendProvider` builds all five seams; the gateway keys are only required for `Gateway`. `SqlServer` uses `ServerHistorian:SqlServer:{ConnectionString (required), Schema=hist, CommandTimeout=30s, AutoCreateSchema=true}`.
|
||||
- **SQL Server** (`Driver.Historian.SqlServer`): `hist.Sample` PK `(TagId, SourceTimeUtc)` — writes are insert-if-absent, so outbox redelivery is idempotent. Stores every value kind with its **real status**; per-sample aggregates (12, simple definitions) are pushed down to T-SQL with bucket math in **100 ns ticks**; the Part 13 time-weighted five (TimeAverage/TimeAverage2/Total/Total2/Interpolative, #671 — ⚠️ `Total` is now the time-weighted integral, no longer `SUM`) stream window + 4 bound rows into the shared `TimeWeightedAggregator` (both SQL backends, identical results; client `AggregateConfiguration` ignored; status = the SDK `AggregateCalculator` time-based rule, Uncertain time counts as Good; reverse `Interpolative` anchored at the later time; window capped by `ServerHistorian:MaxTimeWeightedRawRows` ⇒ `BadTooManyOperations`). Docker-dev recipe, schema and aggregate semantics: `docs/Historian.md` §"SQL Server backend". Integration tests need `HISTORIAN_SQL_TEST_CONNECTION`. **Retention (#667):** opt-in `SqlServer:RetentionDays` (+ `PurgeInterval` 1h, `PurgeBatchSize` 4000) -> `HistorianRetentionService` purges old `hist.Sample` (per tag) / `hist.Event` in `DELETE TOP(n)` batches, Primary-gated on `IHistorizationRoleView`, one purger per DB via `sp_getapplock`; counter `otopcua.historian.retention.purged{table}`. Optional `IHistorianBackendProvider.CreateRetention` (default null). `docs/Historian.md` §Retention.
|
||||
- **SQL Server** (`Driver.Historian.SqlServer`): `hist.Sample` PK `(TagId, SourceTimeUtc)` — writes are insert-if-absent, so outbox redelivery is idempotent. Stores every value kind with its **real status**; per-sample aggregates (10, simple definitions) are pushed down to T-SQL with bucket math in **100 ns ticks**; the Part 13 time-weighted seven (TimeAverage/TimeAverage2/Total/Total2/Interpolative, #671 — ⚠️ `Total` is now the time-weighted integral, no longer `SUM` — plus the **duration-based** PercentGood/PercentBad, #680) stream window + 4 bound rows into the shared `TimeWeightedAggregator` (both SQL backends, identical results; reverse `Interpolative` anchored at the later time; window capped by `ServerHistorian:MaxTimeWeightedRawRows` ⇒ `BadTooManyOperations`, percentages included). ⚠️ **#680 status rules (client-visible):** TimeAverage/Total are Uncertain if any part of the interval uses an Uncertain value, is bridged across a Bad sample, is extrapolated past the last sample (⇒ a live tag's latest interval is Uncertain) or is not covered; TimeAverage2/Total2 keep the SDK time-based rule (Uncertain time counts as Good) but never extrapolate past the last sample — that interval is Partial, later ones `BadNoData` (a report-by-exception tag with nothing after its last sample reads `BadNoData` until its next sample arrives). `docs/Historian.md` §"Time-weighted aggregates". Docker-dev recipe, schema and aggregate semantics: `docs/Historian.md` §"SQL Server backend". Integration tests need `HISTORIAN_SQL_TEST_CONNECTION`. **Retention (#667):** opt-in `SqlServer:RetentionDays` (+ `PurgeInterval` 1h, `PurgeBatchSize` 4000) -> `HistorianRetentionService` purges old `hist.Sample` (per tag) / `hist.Event` in `DELETE TOP(n)` batches, Primary-gated on `IHistorizationRoleView`, one purger per DB via `sp_getapplock`; counter `otopcua.historian.retention.purged{table}`. Optional `IHistorianBackendProvider.CreateRetention` (default null). `docs/Historian.md` §Retention.
|
||||
- **QuestDB** (`Driver.Historian.QuestDb`, `ServerHistorian:QuestDb:*`, Npgsql over PG wire; docker-dev overlay `docker-compose.questdb.yml`): same kinds/status/aggregates/HistoryUpdate as SqlServer, soft delete by tombstone (INSERT only — never `UPDATE`/`DELETE`; every read filters `NOT deleted`). ⚠️ **Timestamps are µs** (`QuestDbConnection.Round`/`BindTimestamp`; `IHistorianDataSource.TimestampResolutionTicks` = 10 steps HistoryRead paging), **pre-1970 cannot be stored** (writer drops, HistoryUpdate `BadOutOfRange`), every write waits for WAL visibility, TTL is enforced on write commits only, set Npgsql `Maximum Pool Size` < QuestDB's 64. `docs/Historian.md` §"QuestDB backend".
|
||||
- ⚠️ **Reverse processed reads are Part 13/SDK-anchored (#677, client-visible):** every aggregate on SqlServer/QuestDB now anchors a reverse read at its later time — window `(end, start]`, partial interval earliest, each stamped with its later edge (was: earlier-anchored, stamped earlier edge); on forward reads Partial also marks the interval the data starts inside (not Average — the SDK's rule, forward-only); a non-default client `AggregateConfiguration` ⇒ `BadAggregateConfigurationRejected`; Gateway provisioning sends `IntegralDivisor = 1`. SDK-parity tests: `SdkAggregateParityTests`. `docs/Historian.md` §Aggregates.
|
||||
- **Real status fix:** the recorder used to stamp every sample Good (`192`); it now carries the driver's status. ⚠️ On the **Gateway** backend this changes nothing stored: the gateway discards quality (C-002), so AVEVA still stores Good.
|
||||
@@ -1381,7 +1381,7 @@ authorization uses the standard `AccessLevels.HistoryRead` bit set at materializ
|
||||
| `MaxTieClusterOverfetch` | `65536` | Bounded over-fetch the HistoryRead-Raw paging uses to page within an oversized same-timestamp tie cluster (retained from the prior backend) |
|
||||
| `HistoryReadBatchParallelism` | `4` | Max nodes served concurrently within one HistoryRead batch (03/S3); `⌈N/P⌉ × CallTimeout` worst case instead of sequential |
|
||||
| `MaxConcurrentHistoryReadBatches` | `16` | Process-wide concurrent HistoryRead batch cap (03/S3); flood degrades to `BadTooManyOperations`. Total in-flight reads capped at 16 × 4 = 64 |
|
||||
| `MaxTimeWeightedRawRows` | `1000000` | `SqlServer`/`QuestDb` only (#671): raw in-window rows one time-weighted aggregate read (incl. `Total`) may stream into the C# evaluator; window query capped at this + 1, past it ⇒ `BadTooManyOperations` for that node. ≤ 0 = unbounded |
|
||||
| `MaxTimeWeightedRawRows` | `1000000` | `SqlServer`/`QuestDb` only (#671): raw in-window rows one time-weighted aggregate read (incl. `Total` and, since #680, PercentGood/PercentBad) may stream into the C# evaluator; window query capped at this + 1, past it ⇒ `BadTooManyOperations` for that node. ≤ 0 = unbounded |
|
||||
| `HistoryReadDeadline` | `00:01:00` | Per-request HistoryRead wall-clock deadline (03/S3); a node still in flight surfaces `BadTimeout`. Non-positive = unbounded |
|
||||
|
||||
### Alarm-history path (`AlarmHistorian` section)
|
||||
|
||||
+122
-46
@@ -144,7 +144,7 @@ and all HistoryRead calls on historized nodes return `GoodNoData` (empty, not an
|
||||
| `MaxTieClusterOverfetch` | int | `65536` | (Node-manager paging; all backends — a `SqlServer` PK rules out tie clusters, so it is moot there.) Maximum samples the server will fetch in one shot to page through a tie cluster (multiple samples sharing one `SourceTimestamp`). A cluster larger than this ceiling fails `BadHistoryOperationUnsupported`. Raise to handle abnormally large tie clusters; the default covers all normal-data cases. |
|
||||
| `HistoryReadBatchParallelism` | int | `4` | Max historized nodes served **concurrently** within one HistoryRead batch (arch-review 03/S3). The SDK's `HistoryRead*` override surface is synchronous, so each arm bridges once at its boundary; this turns per-node serving from sequential (`N × CallTimeout` worst case) into `⌈N / P⌉ × CallTimeout`. Also caps the gateway-load multiplication a single client can command. `≤ 1` serves sequentially. |
|
||||
| `MaxConcurrentHistoryReadBatches` | int | `16` | Process-wide limit on HistoryRead (and HistoryUpdate) batches served **concurrently** (arch-review 03/S3). A flood of history clients degrades to `BadTooManyOperations` rather than exhausting the SDK request-thread pool. With `HistoryReadBatchParallelism` this caps total in-flight gateway reads at `MaxConcurrentHistoryReadBatches × HistoryReadBatchParallelism` (default 16 × 4 = 64). A batch that cannot get a slot within `min(HistoryReadDeadline, 5 s)` fails every handle `BadTooManyOperations`. |
|
||||
| `MaxTimeWeightedRawRows` | int | `1000000` | `SqlServer` / `QuestDb` only (#671). The most raw in-window samples one [time-weighted aggregate](#time-weighted-aggregates-part-13-671) read (TimeAverage, TimeAverage2, **Total**, Total2, Interpolative) may stream into the server-side evaluator — that family is **not** pushed down to SQL. The window query is capped at this + 1 rows; one row past the cap fails **that node** with `BadTooManyOperations` (not counted as a backend fault) instead of holding a pooled connection until `HistoryReadDeadline` and returning `BadTimeout`. Bound rows outside the window do not count; per-sample aggregates are not capped. 1 000 000 ≈ 11.5 days of a 1 Hz tag. Non-positive = unbounded (a `Validate()` warning). |
|
||||
| `MaxTimeWeightedRawRows` | int | `1000000` | `SqlServer` / `QuestDb` only (#671). The most raw in-window samples one [time-weighted aggregate](#time-weighted-aggregates-part-13-671) read (TimeAverage, TimeAverage2, **Total**, Total2, Interpolative, and — since #680 — **PercentGood/PercentBad**) may stream into the server-side evaluator — that family is **not** pushed down to SQL. The window query is capped at this + 1 rows; one row past the cap fails **that node** with `BadTooManyOperations` (not counted as a backend fault) instead of holding a pooled connection until `HistoryReadDeadline` and returning `BadTimeout`. Bound rows outside the window do not count; per-sample aggregates are not capped. 1 000 000 ≈ 11.5 days of a 1 Hz tag. Non-positive = unbounded (a `Validate()` warning). |
|
||||
| `HistoryReadDeadline` | TimeSpan | `00:01:00` | Per-request wall-clock deadline for a HistoryRead (and HistoryUpdate) batch (arch-review 03/S3). A node still in flight at the deadline surfaces `BadTimeout`, so one request can never hold an SDK request thread longer than this regardless of node count. Non-positive = unbounded (a `Validate()` warning). |
|
||||
| `HistoryUpdateMaxFutureSkew` | TimeSpan | `1.00:00:00` | #674. How far past the server clock a HistoryUpdate (Insert / Replace / Update) item's `SourceTimestamp` may lie; an item beyond `now + skew` is refused **`BadOutOfRange`** by the node manager before any backend sees it (every backend). Guards against a far-future typo (a 2062 timestamp). The live recorder is not bounded. Non-positive disables the bound (no warning — a deliberate choice). See [HistoryUpdate](#historyupdate). |
|
||||
|
||||
@@ -908,10 +908,10 @@ point (see [Continuation-point paging (Events)](#continuation-point-paging-event
|
||||
|
||||
### Aggregates (Processed reads)
|
||||
|
||||
> **Two kinds.** The *per-sample* aggregates (Average, Minimum, Maximum, Count, Range, Start/End/Delta, Min/MaxActualTime,
|
||||
> PercentGood/Bad) are simple per-sample definitions — no time-weighting, no interval-boundary values — pushed down to SQL. The
|
||||
> *time-weighted* aggregates (TimeAverage, TimeAverage2, Total, Total2, Interpolative; #671) follow OPC UA Part 13 with bounding values
|
||||
> — see [Time-weighted aggregates](#time-weighted-aggregates-part-13-671). Any `HistoryAggregateType` not in the tables throws
|
||||
> **Two kinds.** The *per-sample* aggregates (Average, Minimum, Maximum, Count, Range, Start/End/Delta, Min/MaxActualTime) are
|
||||
> simple per-sample definitions — no time-weighting, no interval-boundary values — pushed down to SQL. The *time-weighted* aggregates
|
||||
> (TimeAverage, TimeAverage2, Total, Total2, Interpolative; #671 — and the duration-based PercentGood/PercentBad since #680) follow
|
||||
> OPC UA Part 13 with bounding values — see [Time-weighted aggregates](#time-weighted-aggregates-part-13-671). Any `HistoryAggregateType` not in the tables throws
|
||||
> `HistoryAggregateNotSupportedException` ⇒ `BadAggregateNotSupported` for that node (e.g. `StartBound`, `DurationGood`,
|
||||
> `StandardDeviationPopulation`).
|
||||
>
|
||||
@@ -951,7 +951,9 @@ returned status carries the historian *Calculated* info bit, `BadNoData` include
|
||||
result for the interval the tag's data **starts inside** — its earliest stored sample (any quality) lies strictly between the interval's
|
||||
edges, so there is no bounding value before it and the data does not cover the whole interval. This follows the SDK's `m_startOfData`
|
||||
rule, which the SDK turns off for `Average` and in effect applies to forward reads only (a reverse read is fed latest-first, so its
|
||||
"start of data" is the latest sample, past the window). A reverse read therefore sets no start-of-data Partial — matching the SDK. Only a
|
||||
"start of data" is the latest sample, past the window). A reverse read therefore sets no start-of-data Partial — matching the SDK
|
||||
(the time-weighted family does set it, on a reverse read, on the interval holding the *latest* sample when nothing follows it — #680,
|
||||
see [Time-weighted aggregates](#time-weighted-aggregates-part-13-671)). Only a
|
||||
forward per-sample read of an aggregate other than `Average` fetches the tag's earliest sample (one seek, bounded by the window end);
|
||||
every other read skips that query. An earliest sample exactly on an edge, or before the interval, sets nothing. The time-weighted
|
||||
aggregates also keep setting Partial on the truncated interval (below).
|
||||
@@ -967,23 +969,49 @@ aggregates also keep setting Partial on the truncated interval (below).
|
||||
| `Start`, `End` | first / last usable sample by time (keeps the sample's type) | bucket start |
|
||||
| `Delta` | `End − Start` (numeric samples) | bucket start |
|
||||
| `MinimumActualTime`, `MaximumActualTime` | the usable sample with the smallest / largest `NumValue`; ties ⇒ earliest | **the sample's own time** |
|
||||
| `PercentGood` | `100 × (Good samples) / (all samples)` | bucket start |
|
||||
| `PercentBad` | `100 × (Bad samples) / (all samples)` | bucket start |
|
||||
|
||||
Status and edge rules:
|
||||
|
||||
- An **empty interval** (no samples) ⇒ `BadNoData`. **`Count` — and every value aggregate — over an interval whose samples are all Bad ⇒ `BadNoData`**, not `0`.
|
||||
- Value aggregates are `Good` when every sample in the bucket is Good and `UncertainDataSubNormal` when the bucket also holds any Bad **or Uncertain** sample.
|
||||
- `Count`, `Start`, `End`, `PercentGood`, `PercentBad` accept any value kind; the other aggregates (and `Delta`) on a bucket that holds a usable `String`/`DateTime` sample ⇒ `BadAggregateInvalidInputs` for that interval.
|
||||
- **`PercentGood` / `PercentBad` and Uncertain samples:** Uncertain samples are counted in the denominator but are **neither Good nor Bad**, so `PercentGood + PercentBad < 100` whenever a bucket holds Uncertain samples. Both report status `Good` for any non-empty bucket.
|
||||
- `Count`, `Start`, `End` accept any value kind; the other aggregates (and `Delta`) on a bucket that holds a usable `String`/`DateTime` sample ⇒ `BadAggregateInvalidInputs` for that interval.
|
||||
- `PercentGood` / `PercentBad` are **duration-based since #680** — see [Time-weighted aggregates](#time-weighted-aggregates-part-13-671).
|
||||
- Everything except `Count` and `Start`/`End` is a `double`; `Int64` samples aggregate as doubles and `Boolean` as 0/1.
|
||||
|
||||
### Time-weighted aggregates (Part 13, #671)
|
||||
|
||||
`TimeAverage`, `TimeAverage2`, `Total`, `Total2` and `Interpolative`, evaluated by the shared
|
||||
`Driver.Historian.Sql.Common/TimeWeightedAggregator.cs` for **both** SQL backends (so `SqlServer` and `QuestDb` agree exactly).
|
||||
**This whole family — `Total` included — is evaluated in C#, not pushed down to SQL** (see *Pushdown trade-off* and the raw-row cap
|
||||
below). Bucketing (edges, truncated interval, reverse anchoring, the 100 000-interval cap) is the same as above.
|
||||
`TimeAverage`, `TimeAverage2`, `Total`, `Total2`, `Interpolative` and (since #680) `PercentGood` / `PercentBad`, evaluated by the
|
||||
shared `Driver.Historian.Sql.Common/TimeWeightedAggregator.cs` for **both** SQL backends (so `SqlServer` and `QuestDb` agree exactly).
|
||||
**This whole family — `Total` and the percentages included — is evaluated in C#, not pushed down to SQL** (see *Pushdown trade-off*
|
||||
and the raw-row cap below). Bucketing (edges, truncated interval, reverse anchoring, the 100 000-interval cap) is the same as above.
|
||||
|
||||
> ⚠️ **#680 (client-visible): time-weighted results now follow Part 13 more closely** (`SqlServer` / `QuestDb`; the SDK's own
|
||||
> calculators agree except where noted under *SDK parity*):
|
||||
> - **`TimeAverage` / `Total` status** is `UncertainDataSubNormal` whenever the interval uses an Uncertain value, a segment bridged
|
||||
> across a Bad value, the stepped extrapolation past the last sample, or does not cover the interval — no longer the time-based
|
||||
> rule. **The latest interval of a live tag now reads Uncertain, not Good**, and so does a fully-covered interval bridged across an
|
||||
> outage.
|
||||
> - **`TimeAverage` / `Total` over an interval wholly inside an outage** (data on both sides) now return the bridged value,
|
||||
> Uncertain — they used to return generic `Bad`, value null.
|
||||
> - **`Total` over an interval the data starts inside** integrates only the covered part (it used to be `TimeAverage × interval`).
|
||||
> - **`TimeAverage2` / `Total2` no longer extrapolate past the end of the data** (the last sample, when nothing at or after the window
|
||||
> end exists and it is usable): the interval holding it covers only up to it and is **Partial**; later intervals are **`BadNoData`**
|
||||
> (they used to hold the last value, Good). A one-interval read over the whole history therefore averages the stored data only.
|
||||
> - **`PercentGood` / `PercentBad` are duration-based** (they counted samples), moved to this family — so they are now subject to
|
||||
> `ServerHistorian:MaxTimeWeightedRawRows` (`BadTooManyOperations` past the cap) — and an interval with no sample of its own between
|
||||
> two samples now has a value instead of `BadNoData`.
|
||||
> - **Partial:** set on the forward end-of-data interval (`TimeAverage2`/`Total2`/percentages) and, on a reverse read, on the interval
|
||||
> holding the latest sample when nothing follows it (all of this family but `Interpolative`).
|
||||
> - ⚠️ **Report-by-exception tags (a slowly changing or deadband-filtered value) are the visible case.** A tag whose last stored
|
||||
> sample lies *before* the window and that has nothing after it has, by Part 13, **ended its data**: every interval of
|
||||
> `TimeAverage2` / `Total2` / `PercentGood` / `PercentBad` is **`BadNoData`**, and every interval of `TimeAverage` / `Total` is the
|
||||
> held value, **Uncertain**. When the next sample arrives, a re-read of the same window returns values / `Good` for those intervals
|
||||
> (the held value is then interpolated between real samples, not extrapolated). So a dashboard, a ScadaBridge consumer or any
|
||||
> client that caches processed results for "closed" intervals can see an interval go `BadNoData` → value, or Uncertain → Good, on a
|
||||
> later read — re-read recent intervals, or use `TimeAverage`/`Total` (which always carry a value) for a report-by-exception tag. The
|
||||
> SDK's own calculators behave the same way. Likewise the interval holding a tag's latest sample is Partial and, until a later
|
||||
> sample arrives, `TimeAverage2`/`Total2` there cover only up to it — the comms-loss recovery shape (a single Good sample after an
|
||||
> outage marker, nothing after) answers that sample's value (`Total2` 0), Uncertain + Partial, never Bad/null.
|
||||
|
||||
**Server defaults, fixed:** `TreatUncertainAsBad = false`, `PercentDataBad = PercentDataGood = 100`, `UseSlopedExtrapolation = false`
|
||||
(`ServerAggregateConfiguration` in the OPC UA server project — the SDK's own `AggregateManager` defaults, which a test pins). **A
|
||||
@@ -1001,40 +1029,73 @@ still gets a value.
|
||||
|
||||
**The signal.** Between a usable sample `p` and the next sample `s`: both `Double`/`Int64` ⇒ **sloped** (linear), segment quality the
|
||||
worse of the two; either `Boolean` ⇒ **stepped** (`p` held), quality `p`'s; `s` Bad ⇒ `p` held up to the outage (never interpolated into
|
||||
it — the AtTime rule), Uncertain; no later sample at all ⇒ `p` held to the window end (stepped extrapolation), Uncertain. From a Bad
|
||||
sample (incl. an `Empty` outage marker) to the next sample the signal is **Bad** (outage); before the first sample there is **no
|
||||
data**. That is the
|
||||
*simple-bounding* signal. The *interpolated-bounding* signal skips Bad samples: it joins consecutive usable samples (sloped / stepped)
|
||||
straight across an outage.
|
||||
it — the AtTime rule), Uncertain. From a Bad sample (incl. an `Empty` outage marker) to the next sample the signal is **Bad**
|
||||
(outage) — an outage marker with nothing after it runs to the window end; before the first sample there is **no data**; after a
|
||||
*usable* last sample with nothing at or after the window end the data has **ended** (#680, Part 13 "After End of Data": no
|
||||
extrapolation). That is the *simple-bounding* signal. The *interpolated-bounding* signal skips Bad samples: it joins consecutive usable
|
||||
samples (sloped / stepped) straight across an outage, and holds the last usable value to the window end (stepped extrapolation).
|
||||
|
||||
| Aggregate | Value | Bad intervals |
|
||||
|---|---|---|
|
||||
| `TimeAverage` | ∫ interpolated-bounding signal ÷ its defined duration | interpolated across (Part 13) |
|
||||
| `Total` | `TimeAverage × interval seconds` (value·s; the truncated last bucket uses its own length) | interpolated across |
|
||||
| `Total` | ∫ interpolated-bounding signal over its defined part (value·s) — `TimeAverage × interval seconds` on a covered interval; only the covered part when the data starts inside (#680) | interpolated across |
|
||||
| `TimeAverage2` | ∫ simple-bounding signal over non-Bad time ÷ non-Bad duration | excluded |
|
||||
| `Total2` | ∫ simple-bounding signal over non-Bad time (value·s) | excluded |
|
||||
| `Interpolative` | interpolated-bounding value **at the interval start** (reverse read: see below) | interpolated across |
|
||||
| `PercentGood` / `PercentBad` | `100 ×` Good / Bad time ÷ interval duration (#680, Part 13 §5.4.3.31-34) — see below | counted as Bad time |
|
||||
|
||||
**Status, the four time-weighted values** — aligned with the OPC Foundation SDK's `AggregateCalculator.GetTimeBasedStatusCode`
|
||||
(inspected in `Opc.Ua.Server` 1.5.378.106) under the fixed defaults. With `TreatUncertainAsBad = false`, **Uncertain time counts as
|
||||
Good time**: Uncertain samples, a value held up to an outage and the stepped extrapolation past the last sample all count toward
|
||||
PercentDataGood.
|
||||
**Status, `TimeAverage` / `Total`** (interpolated bounding, #680 — Part 13 "Uncertain if Bad values are skipped or Uncertain values
|
||||
are used", and the SDK's `AverageAggregateCalculator`):
|
||||
|
||||
- Every instant of the interval non-Bad ⇒ `Good`.
|
||||
- No defined value anywhere in the interval (before the first usable sample) ⇒ `BadNoData`.
|
||||
- Any part of the interval uses an Uncertain value (either end of a sloped segment, the start of a stepped one), lies on a segment
|
||||
bridged across a Bad sample, is the stepped extrapolation past the last usable sample, or is not covered (the data starts inside it)
|
||||
⇒ `UncertainDataSubNormal`. A segment bridged across an outage marks **every** interval it touches, not just the one holding the Bad
|
||||
sample.
|
||||
- Otherwise ⇒ `Good`.
|
||||
|
||||
**Status, `TimeAverage2` / `Total2`** — the SDK's `AggregateCalculator.GetTimeBasedStatusCode` (inspected in `Opc.Ua.Server`
|
||||
1.5.378.106) under the fixed defaults. With `TreatUncertainAsBad = false`, **Uncertain time counts as Good time** (Uncertain samples, a
|
||||
value held up to an outage). The duration is the interval's, cut at the end of the data:
|
||||
|
||||
- After the end of the data ⇒ `BadNoData` (Part 13 "After End of Data"); the interval the data ends inside covers only up to the last
|
||||
sample and is **Partial** (forward). When it has no non-Bad time before the last sample — the data ends exactly on its start, or the
|
||||
last sample is the first usable one after an outage (comms-loss recovery), or the tag's only sample — there is nothing to cover and
|
||||
the value is the last sample's (`Total2` 0): `Good` only when the data ends exactly on the interval start with a Good sample,
|
||||
otherwise `UncertainDataSubNormal` (the SDK answers the same, modulo its 1 ms artifact).
|
||||
- Every instant of the (covered) interval non-Bad ⇒ `Good`.
|
||||
- Some non-Bad time but not all (an outage, or time before the first sample) ⇒ `UncertainDataSubNormal`.
|
||||
- No non-Bad time but some outage time ⇒ generic **`Bad`** (`0x80000000`) — the SDK's all-Bad result, value null.
|
||||
- No non-Bad time but some outage time (an interval wholly inside an outage) ⇒ generic **`Bad`** (`0x80000000`), value null — Part 13's
|
||||
PercentDataBad rule. (The SDK answers `BadNoData` there; see *SDK parity*.)
|
||||
- No data at all ⇒ `BadNoData`.
|
||||
- A usable `String`/`DateTime` sample feeding the interval ⇒ `BadAggregateInvalidInputs`.
|
||||
|
||||
All carry the Calculated bit; a non-Bad result for the **truncated interval** (shorter than the processing interval — the latest one
|
||||
forward, the earliest one reverse) or for the interval the tag's **data starts inside** (#677, see *Partial bit* above) also carries the
|
||||
**Partial** bit (`0x4`, `AggregateBits.Partial`). Timestamp = interval start (reverse: its later edge); value `double` (Boolean 0/1).
|
||||
For all four, a usable `String`/`DateTime` sample feeding the interval ⇒ `BadAggregateInvalidInputs`.
|
||||
|
||||
> **A live tag's latest interval.** Past the tag's last stored sample the value is held (stepped extrapolation, Uncertain). Under
|
||||
> the SDK rule that Uncertain time is Good time, so the latest interval of a live tag normally reads **`Good`** — not Uncertain —
|
||||
> with the last value held to the interval end. It reads Uncertain only if the interval also contains outage time or starts
|
||||
> before the tag's first sample. Do not take a Good latest interval as proof that data arrived up to its end. If it was truncated
|
||||
> by the request's end time it also carries **Partial**.
|
||||
**`PercentGood` / `PercentBad`** (#680, Part 13 §5.4.3.31-34 — they used to count samples). The interval is cut into **stepped
|
||||
regions**, each with the status of the sample that starts it: Good regions count toward `PercentGood`, Bad ones (incl. outage markers)
|
||||
toward `PercentBad`, Uncertain ones toward neither (so the two sum to < 100 when Uncertain data is present). The region that starts an
|
||||
interval before its first own sample takes the status of the *simple bounding value* there (the SDK's `GetSimpleBound`): Bad after a
|
||||
Bad sample, Uncertain on a segment that runs into an outage or slopes to/from an Uncertain sample, else the earlier sample's. Time
|
||||
before the tag's first sample is **Bad**. When the data ends (a usable last sample, nothing at or after the window end) its region runs
|
||||
to the **end of its own interval** (Part 13 — the SDK gives it 1 ms) and later intervals are `BadNoData`; an outage marker with nothing
|
||||
after it is Bad to the window end. An interval that ends before the first sample ⇒ `BadNoData`. Any value kind is accepted; the status
|
||||
is `Good` (plus Calculated / Partial). Value = `100 × region time ÷ interval duration`.
|
||||
|
||||
All carry the Calculated bit. A non-Bad result also carries the **Partial** bit (`0x4`, `AggregateBits.Partial`) — never on a Bad
|
||||
result — for: the **truncated interval** (shorter than the processing interval — the latest one forward, the earliest one reverse;
|
||||
Part 13 §5.3.3.2 Case 3 requires it — the SDK does not set it, a deliberate difference); the interval the tag's **data starts inside**
|
||||
(#677, forward, see *Partial bit* above); on a **reverse** read, the interval the **latest** sample lies strictly inside when nothing
|
||||
follows it (where a reverse read's data starts — the SDK's `m_startOfData`, keyed on the first value it is fed, #680); and, forward,
|
||||
the **end-of-data** interval of `TimeAverage2` / `Total2` / the percentages (the SDK's `m_endOfData` rule, which it skips for an
|
||||
extrapolated `TimeAverage` / `Total`). Timestamp = interval start (reverse: its later edge); value `double` (Boolean 0/1).
|
||||
|
||||
> **A live tag's latest interval (#680 — changed, client-visible).** Past the tag's last stored sample `TimeAverage` / `Total` hold the
|
||||
> last value to the interval end (stepped extrapolation), and that interval now reads **`UncertainDataSubNormal`** — the extrapolated
|
||||
> part is not data. (It used to read `Good`, because the time-based rule counted the extrapolated time as Good time.) `TimeAverage2` /
|
||||
> `Total2` do not extrapolate: they average / integrate only up to the last sample and set **Partial**; any later interval is
|
||||
> `BadNoData`. A `Good` latest interval therefore now does mean the data covered it — for these aggregates, the latest interval of a
|
||||
> tag that has not yet received a sample at or after the request's end time is never `Good` (`TimeAverage`/`Total`) or always
|
||||
> `Partial` (`TimeAverage2`/`Total2`, forward). If it was truncated by the request's end time it also carries **Partial**.
|
||||
|
||||
**Status, `Interpolative`** (point-based, not the PercentDataGood rule):
|
||||
- A usable sample exactly at the point ⇒ that value with **its own stored status** (no historian bit).
|
||||
@@ -1051,19 +1112,33 @@ value at **10:00**, not 09:00, and no point ever falls before the window. The in
|
||||
**SDK parity (#677).** `SdkAggregateParityTests` (unit, `Sql.Common.Tests`) and
|
||||
`ProcessedConformance.Processed_reads_match_the_SDK_aggregate_calculators` (both SQL backends, through the real SQL) run the pinned SDK's
|
||||
own calculators (`AverageAggregateCalculator`, `MinMaxAggregateCalculator`, `CountAggregateCalculator`, the base `AggregateCalculator`
|
||||
for Interpolative) over the same samples, forward and reverse, and compare every interval's timestamp, value, status code and Partial
|
||||
bit. The per-sample comparison covers Average, Minimum, Maximum, Count, Range, Start, End, Delta, Minimum/MaximumActualTime and
|
||||
PercentGood/Bad; reverse reads over data that starts inside the window are compared too. Known differences that remain (tracked as
|
||||
for Interpolative, `StatusAggregateCalculator` for PercentGood/Bad) over the same samples, forward and reverse, and compare every
|
||||
interval's timestamp, value, status code and Partial bit. The per-sample comparison covers Average, Minimum, Maximum, Count, Range,
|
||||
Start, End, Delta and Minimum/MaximumActualTime; the time-weighted one TimeAverage, TimeAverage2, Total, Total2, Interpolative and
|
||||
PercentGood/Bad (#680), including datasets whose data ends inside the window and with an interval wholly inside an outage; reverse
|
||||
reads over data that starts inside the window are compared too. Known differences that remain (tracked as
|
||||
#680) — documented and masked in the tests, deliberately not changed here because forward results were to stay as they are — are:
|
||||
`Start`/`End` (the SDK returns the raw first/last sample, of any quality, with its own timestamp and stored status; ours returns the
|
||||
first/last usable value stamped with the interval, under the per-sample status rule — the values agree wherever the SDK's is not Bad);
|
||||
`Delta` status (the SDK's stays Good with a Bad sample in the interval, ours is Uncertain); `PercentGood`/`PercentBad` (ours count
|
||||
samples, the SDK measures duration — different definitions, so only timestamp and status are compared); the SDK reports `TimeAverage`/`Total` Uncertain whenever any region of the interval is non-Good, where ours applies the
|
||||
time-based rule (a fully-covered interval bridged across a Bad sample is Good here); `Total` over an interval the data starts inside is
|
||||
`TimeAverage × interval` here but the integral of the covered part in the SDK; ours flags a truncated interval Partial, the SDK does
|
||||
not; the SDK's forward end-of-data Partial rule is not implemented; the SDK's per-sample Count answers 0/Good for an empty interval
|
||||
inside the data and its per-sample aggregates use Good samples only; and on a reverse `Interpolative` read the SDK marks a sample exactly
|
||||
at a point Interpolated (Uncertain if a Bad sample follows) where ours — like the SDK forward — returns the raw sample.
|
||||
`Delta` status (the SDK's stays Good with a Bad sample in the interval, ours is Uncertain); the SDK's per-sample Count answers
|
||||
0/Good for an empty interval inside the data and its per-sample aggregates use Good samples only. Time-weighted differences (#680 closed
|
||||
the TimeAverage/Total status, `Total` coverage, end-of-data and PercentGood/Bad ones), each deliberate: ours flags a truncated interval
|
||||
Partial (Part 13 §5.3.3.2 Case 3), the SDK does not; ours carries the Calculated bit on a `BadNoData`; `TimeAverage2`/`Total2` over an
|
||||
interval wholly inside an outage are generic `Bad` here (Part 13's PercentDataBad rule), `BadNoData` in the SDK; in the end-of-data
|
||||
interval the SDK gives the region after the last sample 1 ms (so `Total2` there differs by ~1 ms × value and the percentages by a lot —
|
||||
ours follow Part 13: `TimeAverage2`/`Total2` stop at the last sample, the percentages' region runs to the interval end); and on a reverse
|
||||
read the SDK's backward slice files a sample exactly on an interval's later edge as its early bound — so a reverse `Interpolative` at
|
||||
such a point comes back Interpolated (Uncertain if a Bad sample follows) and a reverse `TimeAverage`/`Total` interval ending on it takes
|
||||
the status of the segment beyond it — where ours, like the SDK forward, returns the raw sample and keeps the interval's own segments
|
||||
(an SDK defect: its forward and reverse answers differ). Three more, kept deliberately (#680 review): an **outage marker as the last
|
||||
sample** (nothing after it) runs Bad to the window end here — `TimeAverage2`/`Total2` generic `Bad` and the percentages 100 % Bad
|
||||
after it, no end-of-data Partial on the interval holding it, and `TimeAverage`/`Total` Uncertain over the span the last usable value is
|
||||
held up to the marker (no Good value follows ⇒ extrapolated) — where the SDK treats the data as ended at the marker (`BadNoData` after
|
||||
it, Partial on that interval, the held span Good): the tag is in an outage, not out of data. **Booleans** are stepped here (a
|
||||
non-analog value is held; the simple bound next to an Uncertain sample takes the earlier sample's status), while the SDK interpolates
|
||||
them as numbers — e.g. an interval that is `false` for its first half and `true` for its second averages 0.5 here, 1 in the SDK, and a
|
||||
PercentGood region ahead of an Uncertain Boolean is Good here, Uncertain in the SDK. Each is pinned by `TimeWeightedAggregatorTests` and
|
||||
by a parity dataset (`OutageLast`, `BoolSwitch`) whose masks are documented in `SdkAggregateParityTests`.
|
||||
|
||||
**Pushdown trade-off.** Unlike the per-sample aggregates, the time-weighted family is **not** pushed down to SQL: the backend streams
|
||||
the window's rows plus the four bounds (one batch of five result sets on SQL Server; a bounds `UNION ALL` + a window select on QuestDB)
|
||||
@@ -1077,7 +1152,8 @@ judged the larger risk. For very long, densely sampled windows prefer the pushed
|
||||
SQL backends; `TOP (@maxRows)` / `LIMIT $4`). When that extra row arrives the read stops and the node fails with
|
||||
**`BadTooManyOperations`** — for that node only; it is not counted in the backend's health failures. Without the cap a one-year
|
||||
`Total` on a 1 Hz tag (≈31 M rows) would hold a pooled connection until `HistoryReadDeadline` and then return `BadTimeout`. The four
|
||||
bound rows do not count toward the cap, and the per-sample aggregates are never capped. Raise the cap, or read in shorter windows.
|
||||
bound rows do not count toward the cap, and the per-sample aggregates are never capped. Since #680 `PercentGood`/`PercentBad` are in
|
||||
this family and **are** capped (they used to be pushed down). Raise the cap, or read in shorter windows.
|
||||
|
||||
### Health
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common;
|
||||
/// <summary>The query shape that serves an aggregate. One query per family.</summary>
|
||||
public enum AggregateFamily
|
||||
{
|
||||
/// <summary>Average / Minimum / Maximum / Count / Range / PercentGood / PercentBad: one GROUP BY.</summary>
|
||||
/// <summary>Average / Minimum / Maximum / Count / Range: one GROUP BY.</summary>
|
||||
Simple,
|
||||
|
||||
/// <summary>Start / End / Delta: first + last usable sample per bucket.</summary>
|
||||
@@ -15,7 +15,8 @@ public enum AggregateFamily
|
||||
ActualTime,
|
||||
|
||||
/// <summary>
|
||||
/// TimeAverage / TimeAverage2 / Total / Total2 / Interpolative: Part 13 time-weighted, with bounding values.
|
||||
/// TimeAverage / TimeAverage2 / Total / Total2 / Interpolative, and the duration-based PercentGood / PercentBad
|
||||
/// (#680): Part 13 time-weighted, with bounding values.
|
||||
/// The backend streams the window's samples plus the bounding rows and <see cref="TimeWeightedAggregator"/>
|
||||
/// evaluates them (identically for every backend).
|
||||
/// </summary>
|
||||
@@ -31,12 +32,12 @@ public static class AggregateFamilies
|
||||
public static AggregateFamily? FamilyOf(HistoryAggregateType aggregate) => aggregate switch
|
||||
{
|
||||
HistoryAggregateType.Average or HistoryAggregateType.Minimum or HistoryAggregateType.Maximum
|
||||
or HistoryAggregateType.Count or HistoryAggregateType.Range
|
||||
or HistoryAggregateType.PercentGood or HistoryAggregateType.PercentBad => AggregateFamily.Simple,
|
||||
or HistoryAggregateType.Count or HistoryAggregateType.Range => AggregateFamily.Simple,
|
||||
HistoryAggregateType.Start or HistoryAggregateType.End or HistoryAggregateType.Delta => AggregateFamily.StartEnd,
|
||||
HistoryAggregateType.MinimumActualTime or HistoryAggregateType.MaximumActualTime => AggregateFamily.ActualTime,
|
||||
HistoryAggregateType.TimeAverage or HistoryAggregateType.TimeAverage2 or HistoryAggregateType.Total
|
||||
or HistoryAggregateType.Total2 or HistoryAggregateType.Interpolative => AggregateFamily.TimeWeighted,
|
||||
or HistoryAggregateType.Total2 or HistoryAggregateType.Interpolative
|
||||
or HistoryAggregateType.PercentGood or HistoryAggregateType.PercentBad => AggregateFamily.TimeWeighted,
|
||||
_ => null,
|
||||
};
|
||||
}
|
||||
|
||||
+204
-42
@@ -4,8 +4,8 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common;
|
||||
|
||||
/// <summary>
|
||||
/// The OPC UA Part 13 time-weighted aggregates (<see cref="AggregateFamily.TimeWeighted"/>): <c>TimeAverage</c>,
|
||||
/// <c>TimeAverage2</c>, <c>Total</c>, <c>Total2</c> and <c>Interpolative</c>, evaluated over a time-ordered stream of
|
||||
/// stored samples. Both SQL backends fetch the same rows and feed them here, so the two produce identical results
|
||||
/// <c>TimeAverage2</c>, <c>Total</c>, <c>Total2</c>, <c>Interpolative</c> and the duration-based <c>PercentGood</c> /
|
||||
/// <c>PercentBad</c> (#680), evaluated over a time-ordered stream of stored samples. Both SQL backends fetch the same rows and feed them here, so the two produce identical results
|
||||
/// by construction.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
@@ -24,10 +24,12 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common;
|
||||
/// <item><c>s</c> usable, both Double/Int64 ⇒ <b>sloped</b> (linear <c>p → s</c>); quality = worse of the two.</item>
|
||||
/// <item><c>s</c> usable, either Boolean ⇒ <b>stepped</b> (<c>p</c> held); quality = <c>p</c>'s.</item>
|
||||
/// <item><c>s</c> Bad ⇒ <c>p</c> held up to the outage (never interpolated into it, as AtTime does); Uncertain.</item>
|
||||
/// <item>no later sample at all ⇒ <c>p</c> held to the window end (stepped extrapolation); Uncertain.</item>
|
||||
/// <item>no later sample at all (nothing at or after the window end) ⇒ the data has ENDED at <c>p</c>: no
|
||||
/// extrapolation (Part 13 "After End of Data", #680).</item>
|
||||
/// </list>
|
||||
/// From a Bad sample (incl. an <c>Empty</c> outage marker) to the next sample, and before the first sample, the
|
||||
/// signal is Bad (no data). This is the <i>simple bounding</i> signal.
|
||||
/// From a Bad sample (incl. an <c>Empty</c> outage marker) to the next sample the signal is Bad (outage) — to the
|
||||
/// window end when nothing follows it; before the first sample there is no data. This is the <i>simple bounding</i>
|
||||
/// signal.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// The <i>interpolated bounding</i> signal (Part 13 "interpolated bounding values") skips Bad samples: it joins
|
||||
@@ -36,32 +38,58 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common;
|
||||
/// </para>
|
||||
/// <list type="table">
|
||||
/// <item><term>TimeAverage</term><description>∫ interpolated-bounding signal ÷ its defined duration in the interval.</description></item>
|
||||
/// <item><term>Total</term><description><c>TimeAverage × interval seconds</c> (value·s) — Part 13 §5.4.3.8.</description></item>
|
||||
/// <item><term>Total</term><description>∫ interpolated-bounding signal over its defined part, in value·s — Part 13
|
||||
/// §5.4.3.8 (= <c>TimeAverage × interval seconds</c> on a covered interval; only the covered part when the data
|
||||
/// starts inside it, #680).</description></item>
|
||||
/// <item><term>TimeAverage2</term><description>∫ simple-bounding signal over its non-Bad parts ÷ non-Bad duration (Bad intervals excluded).</description></item>
|
||||
/// <item><term>Total2</term><description>∫ simple-bounding signal over its non-Bad parts, in value·s (= TimeAverage2 × non-Bad seconds).</description></item>
|
||||
/// <item><term>PercentGood / PercentBad</term><description><c>100 ×</c> Good / Bad region time ÷ interval duration
|
||||
/// (Part 13 §5.4.3.31-34): stepped regions, each with the status of the sample that starts it; the region before an
|
||||
/// interval's first own sample has the simple bounding value's status (Bad after a Bad sample; Uncertain on a
|
||||
/// segment running into an outage or sloping to/from an Uncertain sample; else the earlier sample's — the SDK's
|
||||
/// <c>GetSimpleBound</c>); time before the first sample is Bad; at the end of the data the last region runs to the
|
||||
/// end of its own interval (Part 13; the SDK gives it 1 ms) and later intervals are <c>BadNoData</c>; an interval
|
||||
/// ending before the first sample ⇒ <c>BadNoData</c>. Status Good; any value kind.</description></item>
|
||||
/// <item><term>Interpolative</term><description>The interpolated-bounding signal's value at each interval start —
|
||||
/// for a reverse read, at <c>End − i·interval</c> (Part 13 anchors a reverse read at its later time), emitted
|
||||
/// latest first.</description></item>
|
||||
/// </list>
|
||||
/// <para>
|
||||
/// <b>Status (time-weighted four)</b> — the SDK <c>AggregateCalculator.GetTimeBasedStatusCode</c> rule with
|
||||
/// TreatUncertainAsBad = false, so Uncertain time (Uncertain samples, a value held up to an outage, stepped
|
||||
/// extrapolation past the last sample) counts as Good time: every instant of the interval non-Bad ⇒
|
||||
/// <c>Good</c>; no non-Bad time but some Bad (outage) time ⇒ <c>Bad</c>; no data at all ⇒ <c>BadNoData</c>;
|
||||
/// otherwise (PercentDataGood = PercentDataBad = 100) <c>UncertainDataSubNormal</c>. Time before the first sample
|
||||
/// counts as Bad (no data) time. A usable String/DateTime sample contributing to the interval ⇒
|
||||
/// <c>BadAggregateInvalidInputs</c>. Every status carries the Calculated bit, plus the Partial bit on a non-Bad
|
||||
/// result of the truncated interval or of the interval the data starts inside (the first sample fed lies strictly
|
||||
/// within it — forward reads only, as in the SDK, #677); the timestamp is the interval's <see cref="BucketPlan.BucketTimestamp"/> (its start forward, its
|
||||
/// later edge reverse); the value is a
|
||||
/// <see cref="double"/> (Boolean = 0/1, Int64 as double).
|
||||
/// <b>Status (TimeAverage / Total, #680)</b> — Part 13 ("Uncertain if Bad values are skipped or Uncertain values are
|
||||
/// used") and the SDK's <c>AverageAggregateCalculator</c>: no defined value in the interval ⇒ <c>BadNoData</c>; any
|
||||
/// part of it on a bridged segment with an Uncertain end (or Uncertain start when stepped), bridged across a Bad
|
||||
/// sample, extrapolated past the last usable sample, or not covered at all ⇒ <c>UncertainDataSubNormal</c> (so the
|
||||
/// latest interval of a live tag is Uncertain, and an interval wholly inside an outage has the bridged value,
|
||||
/// Uncertain); else <c>Good</c>.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Status (TimeAverage2 / Total2)</b> — the SDK <c>AggregateCalculator.GetTimeBasedStatusCode</c> rule with
|
||||
/// TreatUncertainAsBad = false, so Uncertain time (Uncertain samples, a value held up to an outage) counts as Good
|
||||
/// time, over the interval cut at the end of the data: after the end of the data ⇒ <c>BadNoData</c>; the data
|
||||
/// ending exactly on the interval start ⇒ the last sample's value (Total2 0) with its status; every covered instant
|
||||
/// non-Bad ⇒ <c>Good</c>; no non-Bad time but some Bad (outage) time ⇒ <c>Bad</c> (Part 13's PercentDataBad rule;
|
||||
/// the SDK answers <c>BadNoData</c>); no data at all ⇒ <c>BadNoData</c>; otherwise (PercentDataGood = PercentDataBad
|
||||
/// = 100) <c>UncertainDataSubNormal</c>. Time before the first sample counts as Bad (no data) time.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// For the four, a usable String/DateTime sample contributing to the interval ⇒ <c>BadAggregateInvalidInputs</c>.
|
||||
/// Every status of this family but Interpolative's carries the Calculated bit, plus — on a non-Bad result only — the
|
||||
/// Partial bit on: the truncated interval (Part 13 §5.3.3.2 Case 3; the SDK does not set it — deliberate); the
|
||||
/// interval the data starts inside in the read direction (forward: the first sample fed lies strictly within it,
|
||||
/// #677; reverse: the latest sample fed lies strictly within it, nothing after — the SDK's <c>m_startOfData</c>,
|
||||
/// keyed on the first value it is fed, #680); and, forward, the end-of-data interval of TimeAverage2 / Total2 /
|
||||
/// PercentGood / PercentBad (the SDK's <c>m_endOfData</c>, which it skips for an extrapolated TimeAverage / Total).
|
||||
/// The timestamp is the interval's <see cref="BucketPlan.BucketTimestamp"/> (its start forward, its later edge
|
||||
/// reverse); the value is a <see cref="double"/> (Boolean = 0/1, Int64 as double).
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Status (Interpolative).</b> A usable sample exactly at the point ⇒ its value with its own status
|
||||
/// (raw, no historian bit). Otherwise no usable sample at or before ⇒ <c>BadNoData</c>; else the interpolated
|
||||
/// value with the Interpolated bit and the status of the simple-bounding signal at that instant: <c>Good</c> on
|
||||
/// a Good segment, <c>UncertainDataSubNormal</c> on an Uncertain one, inside an outage (interpolated across it)
|
||||
/// or past the last sample (extrapolated).
|
||||
/// or past the last sample (extrapolated). A reverse read returns the raw sample at a point exactly on a sample
|
||||
/// too — the SDK's reverse answer (Interpolated, from the segment beyond) differs from its own forward one; that
|
||||
/// asymmetry is the SDK's defect, deliberately not copied.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
public sealed class TimeWeightedAggregator
|
||||
@@ -108,13 +136,19 @@ public sealed class TimeWeightedAggregator
|
||||
// Interpolated-bounding (bridged) signal, per bucket.
|
||||
private readonly double[] _bridgedArea;
|
||||
private readonly long[] _bridgedTicks;
|
||||
// A bridged segment touching the bucket was non-Good: an Uncertain end, a Bad sample skipped, or extrapolated (#680).
|
||||
private readonly bool[] _bridgedNonGood;
|
||||
private readonly bool[] _invalid;
|
||||
// PercentGood / PercentBad (#680): stepped status regions, per bucket.
|
||||
private readonly long[] _goodRegionTicks;
|
||||
private readonly long[] _badRegionTicks;
|
||||
// Interpolative, per point (index = ascending time; see PointTicks).
|
||||
private readonly DataValueSnapshot?[] _points;
|
||||
|
||||
private StoredSample? _prevAny;
|
||||
private StoredSample? _prevUsable;
|
||||
private DateTime? _firstSampleUtc; // the earliest sample fed (any quality): where the data starts
|
||||
private long? _dataEndTicks; // the last sample fed, when usable and nothing at/after the end followed
|
||||
private long _nextPoint; // next point (ascending time) not yet classified
|
||||
private readonly List<PendingPoint> _pending = []; // points awaiting the next usable sample
|
||||
private bool _finished;
|
||||
@@ -146,7 +180,10 @@ public sealed class TimeWeightedAggregator
|
||||
_badTicks = new long[n];
|
||||
_bridgedArea = new double[n];
|
||||
_bridgedTicks = new long[n];
|
||||
_bridgedNonGood = new bool[n];
|
||||
_invalid = new bool[n];
|
||||
_goodRegionTicks = new long[n];
|
||||
_badRegionTicks = new long[n];
|
||||
_points = new DataValueSnapshot?[n];
|
||||
}
|
||||
|
||||
@@ -172,7 +209,11 @@ public sealed class TimeWeightedAggregator
|
||||
if (_finished) throw new InvalidOperationException("The aggregation is already finished.");
|
||||
var t = s.TimeUtc.Ticks;
|
||||
if (_prevAny is { } last && t <= last.TimeUtc.Ticks) return;
|
||||
_firstSampleUtc ??= s.TimeUtc;
|
||||
if (_firstSampleUtc is null)
|
||||
{
|
||||
_firstSampleUtc = s.TimeUtc;
|
||||
AddRegion(_badRegionTicks, _start, t); // PercentBad: the time before the first sample is Bad (no data)
|
||||
}
|
||||
|
||||
// Interpolative: classify every bucket start before this sample by the simple-bounding signal there.
|
||||
ClassifyPointsBefore(t);
|
||||
@@ -188,6 +229,7 @@ public sealed class TimeWeightedAggregator
|
||||
{
|
||||
AddBad(bad, t);
|
||||
}
|
||||
if (_prevAny is { } r) AddStatusRegion(r, s, t);
|
||||
|
||||
if (!s.IsUsable) DegradePending(); // the segment the pending points sit on runs into an outage
|
||||
|
||||
@@ -196,7 +238,10 @@ public sealed class TimeWeightedAggregator
|
||||
if (_prevUsable is { } pu)
|
||||
{
|
||||
var sloped = IsSloped(pu, s);
|
||||
AddBridged(pu, t, sloped ? s.Value.Number : pu.Value.Number);
|
||||
// Part 13: Uncertain when a Bad value was skipped or an Uncertain value is used (the end sample only
|
||||
// contributes its value to a sloped segment).
|
||||
var nonGood = !pu.IsGood || (sloped && !s.IsGood) || _prevAny is { IsUsable: false };
|
||||
AddBridged(pu, t, sloped ? s.Value.Number : pu.Value.Number, nonGood);
|
||||
}
|
||||
ResolvePending(_prevUsable, s);
|
||||
_prevUsable = s;
|
||||
@@ -215,11 +260,21 @@ public sealed class TimeWeightedAggregator
|
||||
_finished = true;
|
||||
ClassifyPointsBefore(long.MaxValue);
|
||||
if (_prevAny is { IsUsable: true } p && p.TimeUtc.Ticks < _end)
|
||||
AddRaw(p, _end, p.Value.Number); // stepped extrapolation
|
||||
{
|
||||
// No sample at or after the window end: the data ENDS here (Part 13 "After End of Data"). The simple-
|
||||
// bounding signal is not extrapolated (TimeAverage2/Total2 cover up to it; later intervals BadNoData), and
|
||||
// the status region it starts runs to the end of its own interval only (PercentGood/PercentBad).
|
||||
_dataEndTicks = p.TimeUtc.Ticks;
|
||||
if (p.TimeUtc.Ticks >= _start && _plan.BucketCount > 0)
|
||||
AddRegion(RegionTicks(p), p.TimeUtc.Ticks, BucketEnd(FirstBucketAt(p.TimeUtc.Ticks)));
|
||||
}
|
||||
else if (_prevAny is { } bad && bad.TimeUtc.Ticks < _end)
|
||||
AddBad(bad, _end); // the outage runs to the window end
|
||||
{
|
||||
AddBad(bad, _end); // an outage marker last: the outage runs to the window end
|
||||
AddRegion(_badRegionTicks, bad.TimeUtc.Ticks, _end);
|
||||
}
|
||||
if (_prevUsable is { } pu && pu.TimeUtc.Ticks < _end)
|
||||
AddBridged(pu, _end, pu.Value.Number);
|
||||
AddBridged(pu, _end, pu.Value.Number, nonGood: true); // stepped extrapolation ⇒ Uncertain
|
||||
ResolvePending(_prevUsable, next: null);
|
||||
}
|
||||
|
||||
@@ -239,30 +294,82 @@ public sealed class TimeWeightedAggregator
|
||||
: new DataValueSnapshot(null, HistorianStatusCodes.BadNoData, PointTime(i), now);
|
||||
|
||||
var stamp = _plan.BucketTimestamp(i);
|
||||
var duration = BucketEnd(i) - BucketStartTicks(i);
|
||||
// Partial: the truncated interval (the latest forward, the earliest reverse), or the one the data starts inside
|
||||
// (no bounding value before it — the SDK's m_startOfData rule, #677), which the SDK applies to forward reads only.
|
||||
var partial = _plan.IsTruncated(i) || (!_plan.Reversed && _plan.DataStartsInside(i, _firstSampleUtc));
|
||||
DataValueSnapshot Calc(object? value, uint status)
|
||||
var lo = BucketStartTicks(i);
|
||||
var hi = BucketEnd(i);
|
||||
var duration = hi - lo;
|
||||
// Partial: the truncated interval (the latest forward, the earliest reverse — Part 13 §5.3.3.2 Case 3, which the
|
||||
// SDK does not implement), or the one the data starts inside IN THE READ DIRECTION (no bounding value on that
|
||||
// side — the SDK's m_startOfData rule, #677): forward, the earliest sample strictly inside; reverse, the latest
|
||||
// sample strictly inside (nothing after it — the SDK keys m_startOfData on the first value it is fed, #680).
|
||||
// Forward, the end-of-data interval (the latest sample inside it, nothing after) is Partial too — never on an
|
||||
// extrapolated TimeAverage/Total (the SDK's m_endOfData rule, #680). Never on a Bad result.
|
||||
var latestInside = _plan.Reversed && _prevAny is { } latest
|
||||
&& latest.TimeUtc.Ticks > lo && latest.TimeUtc.Ticks < hi;
|
||||
var partial = _plan.IsTruncated(i) || latestInside
|
||||
|| (!_plan.Reversed && _plan.DataStartsInside(i, _firstSampleUtc));
|
||||
var endOfData = !_plan.Reversed && _dataEndTicks is { } de && de >= lo && de < hi;
|
||||
DataValueSnapshot Calc(object? value, uint status, bool endPartial = false)
|
||||
{
|
||||
var code = HistorianInfoBits.WithCalculated(status);
|
||||
if (partial && !StatusBits.IsBad(status)) code |= HistorianInfoBits.HistorianPartial;
|
||||
if ((partial || endPartial) && !StatusBits.IsBad(status)) code |= HistorianInfoBits.HistorianPartial;
|
||||
return new DataValueSnapshot(value, code, stamp, now);
|
||||
}
|
||||
|
||||
if (_invalid[i]) return Calc(null, HistorianStatusCodes.BadAggregateInvalidInputs);
|
||||
if (_usableTicks[i] == 0)
|
||||
return _badTicks[i] > 0 ? Calc(null, HistorianStatusCodes.Bad) : AggregateEvaluator.NoData(stamp, now);
|
||||
|
||||
var status = _usableTicks[i] == duration ? HistorianStatusCodes.Good : HistorianStatusCodes.UncertainDataSubNormal;
|
||||
return _aggregate switch
|
||||
switch (_aggregate)
|
||||
{
|
||||
HistoryAggregateType.TimeAverage => Calc(_bridgedArea[i] / _bridgedTicks[i], status),
|
||||
HistoryAggregateType.Total => Calc(_bridgedArea[i] / _bridgedTicks[i] * (duration / TicksPerSecond), status),
|
||||
HistoryAggregateType.TimeAverage2 => Calc(_rawArea[i] / _usableTicks[i], status),
|
||||
HistoryAggregateType.Total2 => Calc(_rawArea[i] / TicksPerSecond, status),
|
||||
_ => throw new HistoryAggregateNotSupportedException(_aggregate),
|
||||
};
|
||||
case HistoryAggregateType.TimeAverage or HistoryAggregateType.Total:
|
||||
{
|
||||
// Interpolated bounding: Uncertain when any part of the interval was bridged across a Bad value, used an
|
||||
// Uncertain value, was extrapolated, or is not covered at all (Part 13; the SDK's AverageAggregateCalculator).
|
||||
if (_invalid[i]) return Calc(null, HistorianStatusCodes.BadAggregateInvalidInputs);
|
||||
if (_bridgedTicks[i] == 0) return AggregateEvaluator.NoData(stamp, now);
|
||||
var status = _bridgedNonGood[i] || _bridgedTicks[i] < duration
|
||||
? HistorianStatusCodes.UncertainDataSubNormal
|
||||
: HistorianStatusCodes.Good;
|
||||
return _aggregate == HistoryAggregateType.TimeAverage
|
||||
? Calc(_bridgedArea[i] / _bridgedTicks[i], status)
|
||||
: Calc(_bridgedArea[i] / TicksPerSecond, status); // ∫ over the covered part only (Part 13 partial width)
|
||||
}
|
||||
case HistoryAggregateType.TimeAverage2 or HistoryAggregateType.Total2:
|
||||
{
|
||||
if (_invalid[i]) return Calc(null, HistorianStatusCodes.BadAggregateInvalidInputs);
|
||||
if (_dataEndTicks is { } dataEnd)
|
||||
{
|
||||
// After the end of the data: nothing to aggregate (Part 13 "After End of Data: Bad_NoData").
|
||||
if (lo > dataEnd) return AggregateEvaluator.NoData(stamp, now);
|
||||
if (_usableTicks[i] == 0 && dataEnd < hi && _prevAny is { } last)
|
||||
{
|
||||
// The data ends inside this interval and no non-Bad time precedes its last sample here (the data
|
||||
// ends on the interval's start; or the last sample is the first usable one after an outage — the
|
||||
// comms-loss recovery shape — or the tag's only sample): nothing covered, so the value at that
|
||||
// instant (Total2 0). Good only when the data ends exactly on the interval start with a Good
|
||||
// sample (nothing before it in the interval); otherwise the interval also holds Bad / no-data time
|
||||
// ⇒ Uncertain (the SDK answers the same, modulo its 1 ms artifact).
|
||||
if (!IsNumeric(last)) return Calc(null, HistorianStatusCodes.BadAggregateInvalidInputs);
|
||||
var st = lo == dataEnd && last.IsGood ? HistorianStatusCodes.Good : HistorianStatusCodes.UncertainDataSubNormal;
|
||||
return Calc(_aggregate == HistoryAggregateType.TimeAverage2 ? last.Value.Number : 0d, st, endOfData);
|
||||
}
|
||||
duration = Math.Min(hi, dataEnd) - lo;
|
||||
}
|
||||
if (_usableTicks[i] == 0)
|
||||
return _badTicks[i] > 0 ? Calc(null, HistorianStatusCodes.Bad) : AggregateEvaluator.NoData(stamp, now);
|
||||
var status = _usableTicks[i] == duration ? HistorianStatusCodes.Good : HistorianStatusCodes.UncertainDataSubNormal;
|
||||
return _aggregate == HistoryAggregateType.TimeAverage2
|
||||
? Calc(_rawArea[i] / _usableTicks[i], status, endOfData)
|
||||
: Calc(_rawArea[i] / TicksPerSecond, status, endOfData);
|
||||
}
|
||||
case HistoryAggregateType.PercentGood or HistoryAggregateType.PercentBad:
|
||||
{
|
||||
// No data in or around the interval (it ends before the first sample — the SDK's out-of-range test — or
|
||||
// starts after the end of the data) ⇒ BadNoData; otherwise the share of the interval's duration.
|
||||
if (_firstSampleUtc is not { } first || hi < first.Ticks || (_dataEndTicks is { } de2 && lo > de2))
|
||||
return AggregateEvaluator.NoData(stamp, now);
|
||||
var ticks = _aggregate == HistoryAggregateType.PercentGood ? _goodRegionTicks[i] : _badRegionTicks[i];
|
||||
return Calc(100d * ticks / duration, HistorianStatusCodes.Good, endOfData);
|
||||
}
|
||||
default:
|
||||
throw new HistoryAggregateNotSupportedException(_aggregate);
|
||||
}
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------------------------------ Segments
|
||||
@@ -305,13 +412,68 @@ public sealed class TimeWeightedAggregator
|
||||
}
|
||||
}
|
||||
|
||||
private void AddBridged(StoredSample from, long toTicks, double toValue) =>
|
||||
private void AddBridged(StoredSample from, long toTicks, double toValue, bool nonGood) =>
|
||||
Distribute(from, toTicks, toValue, (i, area, ticks) =>
|
||||
{
|
||||
_bridgedArea[i] += area;
|
||||
_bridgedTicks[i] += ticks;
|
||||
if (nonGood) _bridgedNonGood[i] = true;
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------------------------------------ PercentGood / PercentBad
|
||||
|
||||
/// <summary>The region array a stepped region of <paramref name="s"/>'s status counts toward (null: Uncertain, neither).</summary>
|
||||
private long[]? RegionTicks(StoredSample s) =>
|
||||
s.IsGood ? _goodRegionTicks : !s.IsUsable ? _badRegionTicks : null;
|
||||
|
||||
/// <summary>
|
||||
/// The status regions between sample <paramref name="p"/> and the next sample <paramref name="s"/> (at
|
||||
/// <paramref name="t"/>). Inside the interval <paramref name="p"/> lies in, the region has <paramref name="p"/>'s
|
||||
/// status (stepped, Part 13 §5.4.3.31-34). In a later interval the region starts at that interval's simple bounding
|
||||
/// value, whose status is the simple-bounding signal's: Bad after a Bad <paramref name="p"/>; Uncertain when the
|
||||
/// segment runs into an outage or a sloped one has a non-Good end; else <paramref name="p"/>'s (the SDK's
|
||||
/// <c>GetSimpleBound</c>).
|
||||
/// </summary>
|
||||
private void AddStatusRegion(StoredSample p, StoredSample s, long t)
|
||||
{
|
||||
var lo = Math.Max(p.TimeUtc.Ticks, _start);
|
||||
var hi = Math.Min(t, _end);
|
||||
if (lo >= hi || _plan.BucketCount == 0) return;
|
||||
long[]? bound;
|
||||
if (!p.IsUsable) bound = _badRegionTicks;
|
||||
else if (!s.IsUsable) bound = null; // held into an outage ⇒ Uncertain
|
||||
else if (IsSloped(p, s)) bound = p.IsGood && s.IsGood ? _goodRegionTicks : null;
|
||||
else bound = RegionTicks(p);
|
||||
var own = RegionTicks(p);
|
||||
for (var i = FirstBucketAt(lo); i < _plan.BucketCount; i++)
|
||||
{
|
||||
var bs = BucketStartTicks(i);
|
||||
if (bs >= hi) break;
|
||||
var a = Math.Max(lo, bs);
|
||||
var b = Math.Min(hi, BucketEnd(i));
|
||||
if (a >= b) continue;
|
||||
var target = p.TimeUtc.Ticks >= bs ? own : bound;
|
||||
if (target is not null) target[i] += b - a;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Adds <c>[from, to)</c>, clipped to the window, to <paramref name="target"/> per bucket.</summary>
|
||||
private void AddRegion(long[]? target, long from, long to)
|
||||
{
|
||||
if (target is null) return;
|
||||
var lo = Math.Max(from, _start);
|
||||
var hi = Math.Min(to, _end);
|
||||
if (lo >= hi || _plan.BucketCount == 0) return;
|
||||
for (var i = FirstBucketAt(lo); i < _plan.BucketCount; i++)
|
||||
{
|
||||
var bs = BucketStartTicks(i);
|
||||
if (bs >= hi) break;
|
||||
var a = Math.Max(lo, bs);
|
||||
var b = Math.Min(hi, BucketEnd(i));
|
||||
if (a < b) target[i] += b - a;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Splits the linear segment <c>[from.t, toTicks)</c> (value <c>from.v → toValue</c>) at the bucket edges, clipped to
|
||||
/// the window, and hands each bucket its trapezoid area (value·ticks) and duration. A non-numeric start sample
|
||||
|
||||
@@ -68,13 +68,17 @@ public abstract class ProcessedConformance : ConformanceBase
|
||||
[Theory]
|
||||
[InlineData(HistoryAggregateType.PercentGood, 100d, 50d, 100d)]
|
||||
[InlineData(HistoryAggregateType.PercentBad, 0d, 50d, 0d)]
|
||||
public async Task Percent_aggregates_count_every_sample(HistoryAggregateType agg, double b0, double b1, double b2)
|
||||
public async Task Percent_aggregates_are_shares_of_the_interval_duration(HistoryAggregateType agg, double b0, double b1,
|
||||
double b2)
|
||||
{
|
||||
// #680 (Part 13 §5.4.3.31-34): stepped regions, each with the status of the sample starting it. B1: [20,30) Bad,
|
||||
// [30,40) Good. B2 holds the end of the data (+40, nothing after) ⇒ its region runs to the interval end, Partial;
|
||||
// B3 is after the end of the data ⇒ BadNoData.
|
||||
await ConformanceData.SeedAAsync(B);
|
||||
var s = (await ReadA(agg)).Samples;
|
||||
s.Select(x => x.Value).ShouldBe([b0, b1, b2, null]);
|
||||
// A percentage is a statement about quality, so it is not itself degraded by the Bad input.
|
||||
s.Select(x => x.StatusCode).ShouldBe([GoodCalc, GoodCalc, GoodCalc, NoDataCalc]);
|
||||
s.Select(x => x.StatusCode).ShouldBe([GoodCalc, GoodCalc, GoodCalc | 0x4u, NoDataCalc]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -169,9 +173,10 @@ public abstract class ProcessedConformance : ConformanceBase
|
||||
// An interval wider than the window (here TimeSpan.MaxValue) is the same single interval, no overflow.
|
||||
(await src.ReadProcessedAsync("A", start, end, TimeSpan.MaxValue, HistoryAggregateType.Count, Ct))
|
||||
.Samples.Single().Value.ShouldBe(4);
|
||||
// The time-weighted path over the whole range: ≈ the extrapolated 50 that dominates eight millennia.
|
||||
// The time-weighted path over the whole range. TimeAverage2 is not extrapolated past the end of the data (+40, #680):
|
||||
// (150 + 200 + 450) value·s over the 30 non-Bad seconds = 80/3.
|
||||
var tw = (await src.ReadProcessedAsync("A", start, end, TimeSpan.Zero, HistoryAggregateType.TimeAverage2, Ct)).Samples.Single();
|
||||
((double)tw.Value!).ShouldBe(50d, 1e-6);
|
||||
((double)tw.Value!).ShouldBe(80d / 3d, 1e-6);
|
||||
// The data starts (+0) strictly inside the one interval ⇒ Partial as well (#677, the SDK's m_startOfData rule).
|
||||
tw.StatusCode.ShouldBe(UncertainCalc | 0x4u);
|
||||
src.GetHealthSnapshot().TotalFailures.ShouldBe(0);
|
||||
@@ -347,15 +352,29 @@ public abstract class ProcessedConformance : ConformanceBase
|
||||
}
|
||||
}
|
||||
var reverseInterpolative = agg == HistoryAggregateType.Interpolative && from > to;
|
||||
SdkAggregateOracle.ShouldMatch(ours, SdkAggregateOracle.Run(agg, start, end, interval, rows),
|
||||
var sdk = SdkAggregateOracle.Run(agg, start, end, interval, rows);
|
||||
if (from > to && agg is HistoryAggregateType.TimeAverage or HistoryAggregateType.Total)
|
||||
{
|
||||
// Documented SDK divergence (#680, see SdkAggregateParityTests.MaskDocumented): on a reverse read the SDK files
|
||||
// a sample exactly on an interval's later edge as its early bound, so a TimeAverage/Total interval ending on a
|
||||
// sample takes the status of the segment beyond it (Dense: +20 next to the Bad +22.5). Only that interval,
|
||||
// only Good (ours) vs Uncertain (SDK), is aligned.
|
||||
var plan = BucketPlan.Create(start, end, interval);
|
||||
for (long i = 0; i < plan.BucketCount; i++)
|
||||
{
|
||||
var k = (int)(plan.BucketCount - 1 - i);
|
||||
if (k < sdk.Count && rows.Any(r => r.Item1.Ticks == plan.UpperTicks(i))
|
||||
&& (ours[k].StatusCode & 0xFFFF0000u) == 0
|
||||
&& (sdk[k].StatusCode & 0xFFFF0000u) == 0x40A40000u)
|
||||
ours[k] = ours[k] with { StatusCode = ours[k].StatusCode | 0x40A40000u };
|
||||
}
|
||||
}
|
||||
SdkAggregateOracle.ShouldMatch(ours, sdk,
|
||||
$"{agg} {from}→{to}/{intervalS}s",
|
||||
compareCalculatedInterpolatedBits: tw && !reverseInterpolative,
|
||||
// pre-existing, #680: TimeAverage/Total status rule; Start/End raw-sample timestamp + status; Delta status;
|
||||
// PercentGood/Bad per-sample vs per-duration values (see SdkAggregateParityTests).
|
||||
compareCodeBits: agg is not (HistoryAggregateType.TimeAverage or HistoryAggregateType.Total
|
||||
or HistoryAggregateType.Delta) && !reverseInterpolative && !startOrEnd,
|
||||
compareTimestamps: !startOrEnd,
|
||||
compareValues: agg is not (HistoryAggregateType.PercentGood or HistoryAggregateType.PercentBad));
|
||||
// pre-existing, #680: Start/End raw-sample timestamp + status; Delta status (see SdkAggregateParityTests).
|
||||
compareCodeBits: agg != HistoryAggregateType.Delta && !reverseInterpolative && !startOrEnd,
|
||||
compareTimestamps: !startOrEnd);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -383,7 +402,8 @@ public abstract class ProcessedConformance : ConformanceBase
|
||||
avg.StatusCode.ShouldBe(UncertainCalc);
|
||||
(await One(HistoryAggregateType.Count)).Value.ShouldBe(2);
|
||||
(await One(HistoryAggregateType.Start)).Value.ShouldBe(10d);
|
||||
((double)(await One(HistoryAggregateType.PercentBad)).Value!).ShouldBe(33.333333333333336, 1e-9); // 100 × 1/3
|
||||
// #680: duration-based — [0,5) Good, [5,10) Bad, [10,20) Good (no end bound: the last region runs to the interval end).
|
||||
((double)(await One(HistoryAggregateType.PercentBad)).Value!).ShouldBe(25d, 1e-9);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -611,6 +631,9 @@ public abstract class ProcessedConformance : ConformanceBase
|
||||
var ex = await Should.ThrowAsync<HistoryReadLimitExceededException>(
|
||||
() => under.ReadProcessedAsync("A", At(0), At(80), Twenty, HistoryAggregateType.Total, Ct));
|
||||
ex.Limit.ShouldBe(4);
|
||||
// #680: PercentGood/PercentBad are duration-based, served by the same window stream ⇒ capped too.
|
||||
(await Should.ThrowAsync<HistoryReadLimitExceededException>(
|
||||
() => under.ReadProcessedAsync("A", At(0), At(80), Twenty, HistoryAggregateType.PercentGood, Ct))).Limit.ShouldBe(4);
|
||||
// Pushed-down (per-sample) aggregates are not capped; the bound rows outside the window do not count.
|
||||
(await under.ReadProcessedAsync("A", At(0), At(80), Twenty, HistoryAggregateType.Average, Ct)).Samples.Count.ShouldBe(4);
|
||||
(await B.CreateDataSource(3).ReadProcessedAsync("W", At(0), At(80), Twenty, HistoryAggregateType.TimeAverage, Ct))
|
||||
|
||||
+4
-2
@@ -11,10 +11,12 @@ public sealed class ProcessedReadTests
|
||||
public void Every_enum_member_is_either_served_or_refused()
|
||||
{
|
||||
// Seventeen served (#671 added TimeAverage, TimeAverage2, Interpolative and Total2, and moved Total to the
|
||||
// time-weighted family); anything appended to the enum later falls to the refusal (total default).
|
||||
// time-weighted family; #680 moved PercentGood/PercentBad there too — they are duration-based); anything appended to
|
||||
// the enum later falls to the refusal (total default).
|
||||
Enum.GetValues<HistoryAggregateType>().Count(a => AggregateFamilies.FamilyOf(a) is not null).ShouldBe(17);
|
||||
Enum.GetValues<HistoryAggregateType>().Where(a => AggregateFamilies.FamilyOf(a) == AggregateFamily.TimeWeighted)
|
||||
.ShouldBe([HistoryAggregateType.Total, HistoryAggregateType.Interpolative, HistoryAggregateType.TimeAverage,
|
||||
HistoryAggregateType.TimeAverage2, HistoryAggregateType.Total2]);
|
||||
HistoryAggregateType.PercentGood, HistoryAggregateType.PercentBad, HistoryAggregateType.TimeAverage2,
|
||||
HistoryAggregateType.Total2]);
|
||||
}
|
||||
}
|
||||
|
||||
+192
-37
@@ -21,26 +21,37 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common.Tests;
|
||||
/// ours every usable (Good + Uncertain) sample (TreatUncertainAsBad = false applied consistently);</item>
|
||||
/// <item>the per-sample <b>Calculated / MultipleValues</b> info bits (the SDK's MinMax sets Calculated only when the
|
||||
/// extreme is not at the interval start, and flags ties) — those comparisons check code bits + Partial only;</item>
|
||||
/// <item>a <b>truncated</b> interval: ours sets Partial on a non-Bad time-weighted result (the SDK does not), so the
|
||||
/// non-multiple windows mask that one bucket's Partial bit;</item>
|
||||
/// <item>the SDK's <b>end-of-data</b> Partial rule (forward, the interval holding the last sample when nothing follows
|
||||
/// and nothing was extrapolated) is not implemented: every dataset carries data past the window end.</item>
|
||||
/// </list>
|
||||
/// Pre-existing forward divergences this parity test surfaced (#671 behaviour, unchanged here — forward results were
|
||||
/// to stay as they are; recorded for a follow-up rather than fixed in place):
|
||||
/// <list type="bullet">
|
||||
/// <item><b>TimeAverage / Total status:</b> the SDK (interpolated bounds) reports <c>UncertainDataSubNormal</c> when
|
||||
/// <i>any</i> region of the interval is non-Good (e.g. a segment bridged across a Bad sample); ours applies the
|
||||
/// time-based rule (Uncertain time counts as Good) to these two as well, so a bridged-but-fully-covered interval
|
||||
/// is Good. Values and timestamps agree; the status code bits are not compared for these two.</item>
|
||||
/// <item><b>Total over an interval the data starts inside:</b> the SDK integrates only the defined part
|
||||
/// (value·s over the covered seconds); ours is <c>TimeAverage × interval seconds</c>. The Partial bit and timestamp
|
||||
/// agree; that value is not compared.</item>
|
||||
/// <item>a <b>truncated</b> interval: ours sets Partial on a non-Bad time-weighted result (Part 13 §5.3.3.2 Case 3
|
||||
/// requires it; the SDK does not), so the non-multiple windows mask that one bucket's Partial bit;</item>
|
||||
/// <item>a time-weighted <c>BadNoData</c> carries the Calculated bit in ours (every processed status does), not in the
|
||||
/// SDK's — masked by <see cref="MaskDocumented"/>;</item>
|
||||
/// <item><b>TimeAverage2 / Total2 over an interval wholly inside an outage:</b> ours generic <c>Bad</c> (data, but no
|
||||
/// non-Bad time — Part 13's PercentData rule), the SDK <c>BadNoData</c> — masked by <see cref="MaskDocumented"/>;</item>
|
||||
/// <item><b>the end-of-data interval</b> (the last sample inside it, nothing after) of TimeAverage2 / Total2 /
|
||||
/// PercentGood / PercentBad: the SDK gives the region after the last sample a 1 ms duration (an artifact of its
|
||||
/// <c>GetRegionsInValueSet</c>); Part 13 ends TimeAverage2/Total2 at the last sample and runs the PercentGood/Bad
|
||||
/// region to the interval end, which ours does (#680; pinned by <c>TimeWeightedAggregatorTests</c>). That
|
||||
/// interval's value is not compared; its status, Partial bit and timestamp are.</item>
|
||||
/// <item><b>an outage marker as the last sample</b> (<see cref="OutageLast"/>): ours runs the outage to the window
|
||||
/// end — TimeAverage2/Total2 <c>Bad</c>, the percentages 100 % Bad, no end-of-data Partial on the interval holding
|
||||
/// it; the SDK treats the data as ended (<c>BadNoData</c> after it, Partial on that interval); and TimeAverage/Total
|
||||
/// over the span the last usable value is held up to the marker are Uncertain here (no Good value follows ⇒
|
||||
/// extrapolated, Part 13), Good in the SDK — masked by
|
||||
/// <see cref="MaskDocumented"/>;</item>
|
||||
/// <item><b>Booleans</b> (<see cref="BoolSwitch"/>): ours steps them (Part 13: a non-analog value is held, and the
|
||||
/// bound next to an Uncertain sample takes the earlier sample's status); the SDK interpolates them as numbers — its
|
||||
/// values (e.g. 1 vs our 0.5 over an interval switching half-way) and the Good/Uncertain choice are not compared,
|
||||
/// Bad vs non-Bad, timestamps and Partial are;</item>
|
||||
/// <item><b>TimeAverage / Total, reverse, a sample exactly on an interval's later edge:</b> the SDK's backward
|
||||
/// <c>UpdateSlice</c> files it as the early bound (below), so the interval takes the status of the segment beyond
|
||||
/// it (Uncertain when that segment skips a Bad sample); ours keeps the interval's own segments — masked by
|
||||
/// <see cref="MaskDocumented"/>;</item>
|
||||
/// <item><b>Interpolative, reverse, a sample exactly at the point:</b> the SDK's backward <c>UpdateSlice</c> files
|
||||
/// a sample AT the point as the early bound (<c>>= 0</c>), so it interpolates from it to the next Good sample —
|
||||
/// same value, but the Interpolated bit, and Uncertain when a Bad sample sits between. Ours returns the raw
|
||||
/// sample with its own status in both directions (as the SDK itself does forward). Reverse Interpolative
|
||||
/// therefore compares timestamps and values (the anchoring question) but not status.</item>
|
||||
/// therefore compares timestamps and values (the anchoring question) but not status. (The SDK's asymmetry —
|
||||
/// forward it returns the raw sample — is its defect; ours is the same in both directions.)</item>
|
||||
/// </list>
|
||||
/// </remarks>
|
||||
public sealed class SdkAggregateParityTests
|
||||
@@ -64,6 +75,74 @@ public sealed class SdkAggregateParityTests
|
||||
S(100, HistorianValue.FromDouble(60), Good),
|
||||
];
|
||||
|
||||
/// <summary>
|
||||
/// #680: data that ENDS inside every window (nothing after +52.5): every 2.5 s from −10, values cycling, a Bad
|
||||
/// sample at +22.5 and an Uncertain one at +40 — the end-of-data rules (TimeAverage2/Total2 stop at the last
|
||||
/// sample, Partial, then BadNoData; TimeAverage/Total extrapolate, Uncertain).
|
||||
/// </summary>
|
||||
private static readonly StoredSample[] EndsInside = Enumerable.Range(0, 26)
|
||||
.Select(i => (t: -10 + i * 2.5, i))
|
||||
.Select(x => S(x.t, HistorianValue.FromDouble(10 + (x.i * 7 % 13)),
|
||||
x.t == 22.5 ? Bad : x.t == 40 ? Uncertain : Good))
|
||||
.ToArray();
|
||||
|
||||
/// <summary>
|
||||
/// #680: an outage +18 … +45 that wholly contains an interval of every window ([20,40), [20,35), [23,43), (20,40]):
|
||||
/// TimeAverage/Total bridge it (Uncertain); TimeAverage2/Total2 have no non-Bad time there.
|
||||
/// </summary>
|
||||
private static readonly StoredSample[] OutageInside =
|
||||
[
|
||||
S(-10, HistorianValue.FromDouble(5), Good),
|
||||
S(0, HistorianValue.FromDouble(10), Good),
|
||||
S(15, HistorianValue.FromDouble(20), Good),
|
||||
S(18, HistorianValue.Empty, BadNoCommunication),
|
||||
S(45, HistorianValue.FromDouble(30), Good),
|
||||
S(60, HistorianValue.FromDouble(40), Good),
|
||||
S(100, HistorianValue.FromDouble(40), Good),
|
||||
];
|
||||
|
||||
/// <summary>
|
||||
/// #680 review: the comms-loss recovery shape — a Good sample, an outage marker, then ONE recovery sample and nothing
|
||||
/// after (+27.5, inside an interval of every window, never on an edge): the interval holding the recovery sample has
|
||||
/// no non-Bad time before it, so TimeAverage2/Total2 answer the sample's value (Total2 0), Uncertain + Partial.
|
||||
/// </summary>
|
||||
private static readonly StoredSample[] RecoveryEnds =
|
||||
[
|
||||
S(-10, HistorianValue.FromDouble(10), Good),
|
||||
S(5, HistorianValue.Empty, BadNoCommunication),
|
||||
S(27.5, HistorianValue.FromDouble(20), Good),
|
||||
];
|
||||
|
||||
/// <summary>
|
||||
/// #680 review: an outage marker as the LAST sample (+31, nothing after) — ours: the outage runs to the window end
|
||||
/// (TimeAverage2/Total2 Bad, the percentages 100 % Bad); the SDK: the data ended (BadNoData, Partial on the interval
|
||||
/// holding the marker). Also TimeAverage/Total over the span the last usable value is held up to the marker: ours
|
||||
/// Uncertain (Part 13 — no Good value after it, so extrapolated), the SDK Good. Documented, deliberate differences.
|
||||
/// </summary>
|
||||
private static readonly StoredSample[] OutageLast =
|
||||
[
|
||||
S(-10, HistorianValue.FromDouble(10), Good),
|
||||
S(25, HistorianValue.FromDouble(20), Good),
|
||||
S(31, HistorianValue.Empty, BadNoCommunication),
|
||||
];
|
||||
|
||||
/// <summary>
|
||||
/// #680 review: Boolean (stepped) data with an Uncertain and a Bad sample. Ours steps Booleans (Part 13: a
|
||||
/// non-analog value is held); the SDK interpolates them as numbers and bounds them sloped — so only timestamps,
|
||||
/// status and Partial are compared for the values it computes that way (see <see cref="MaskDocumented"/>).
|
||||
/// </summary>
|
||||
private static readonly StoredSample[] BoolSwitch =
|
||||
[
|
||||
S(-10, HistorianValue.FromBoolean(true), Good),
|
||||
S(10, HistorianValue.FromBoolean(false), Good),
|
||||
S(27.5, HistorianValue.FromBoolean(true), Uncertain),
|
||||
S(32, HistorianValue.FromBoolean(false), Good),
|
||||
S(47.5, HistorianValue.FromBoolean(true), Good),
|
||||
S(62, HistorianValue.FromBoolean(true), Bad),
|
||||
S(71, HistorianValue.FromBoolean(false), Good),
|
||||
S(100, HistorianValue.FromBoolean(true), Good),
|
||||
];
|
||||
|
||||
/// <summary>Data that starts inside the window (no sample before +7) and runs past its end.</summary>
|
||||
private static readonly StoredSample[] LateStart = Enumerable.Range(0, 20)
|
||||
.Select(i => S(7 + i * 5, HistorianValue.FromDouble(i % 2 == 0 ? 20 : 40), Good))
|
||||
@@ -110,21 +189,109 @@ public sealed class SdkAggregateParityTests
|
||||
return list;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Applies the documented divergences listed on the class (other than truncation): strips the Calculated bit from our
|
||||
/// <c>BadNoData</c>; maps our generic-Bad TimeAverage2/Total2 outage interval to the SDK's <c>BadNoData</c>; takes the
|
||||
/// SDK's value for the end-of-data interval of TimeAverage2/Total2/PercentGood/PercentBad (its 1 ms artifact).
|
||||
/// </summary>
|
||||
private static IReadOnlyList<DataValueSnapshot> MaskDocumented(HistoryAggregateType agg, IReadOnlyList<DataValueSnapshot> ours,
|
||||
IReadOnlyList<SdkAggregateOracle.Result> sdk, double from, double to, double intervalS, StoredSample[] data)
|
||||
{
|
||||
var plan = BucketPlan.Create(At(from), At(to), TimeSpan.FromSeconds(intervalS));
|
||||
var list = ours.ToList();
|
||||
var last = data.Max(x => x.TimeUtc).Ticks;
|
||||
var endsInWindow = last < Math.Max(At(from).Ticks, At(to).Ticks);
|
||||
var dataEnds = endsInWindow && data.Single(x => x.TimeUtc.Ticks == last).IsUsable;
|
||||
var outageEnds = endsInWindow && !dataEnds;
|
||||
var boolean = data.Any(x => x.Value.Kind == HistorianValueKind.Boolean);
|
||||
var percent = agg is HistoryAggregateType.PercentGood or HistoryAggregateType.PercentBad;
|
||||
for (long i = 0; i < plan.BucketCount; i++)
|
||||
{
|
||||
var k = (int)(plan.Reversed ? plan.BucketCount - 1 - i : i);
|
||||
if (k >= sdk.Count) continue; // the count mismatch is reported by ShouldMatch
|
||||
var code = list[k].StatusCode & StatusBits.CodeBitsMask;
|
||||
if (code == HistorianStatusCodes.BadNoData)
|
||||
list[k] = list[k] with { StatusCode = list[k].StatusCode & ~StatusBits.HistorianBitsMask };
|
||||
var twoVariant = agg is HistoryAggregateType.TimeAverage2 or HistoryAggregateType.Total2;
|
||||
if (twoVariant && code == HistorianStatusCodes.Bad && (sdk[k].StatusCode & StatusBits.CodeBitsMask) == HistorianStatusCodes.BadNoData)
|
||||
list[k] = list[k] with { StatusCode = HistorianStatusCodes.BadNoData };
|
||||
// Reverse TimeAverage/Total: the SDK's backward UpdateSlice files a sample exactly on the interval's later edge
|
||||
// as the early bound, so the interval inherits the status of the segment beyond it (the root cause of the
|
||||
// reverse Interpolative divergence). Ours keeps the interval's own segments.
|
||||
if (plan.Reversed && agg is HistoryAggregateType.TimeAverage or HistoryAggregateType.Total
|
||||
&& data.Any(x => x.TimeUtc.Ticks == plan.UpperTicks(i))
|
||||
&& code == HistorianStatusCodes.Good
|
||||
&& (sdk[k].StatusCode & StatusBits.CodeBitsMask) == HistorianStatusCodes.UncertainDataSubNormal)
|
||||
list[k] = list[k] with { StatusCode = (list[k].StatusCode & ~StatusBits.CodeBitsMask) | HistorianStatusCodes.UncertainDataSubNormal };
|
||||
if (dataEnds && (twoVariant || agg is HistoryAggregateType.PercentGood or HistoryAggregateType.PercentBad)
|
||||
&& last >= plan.LowerTicks(i) && last < plan.UpperTicks(i) && sdk[k].Value is not null)
|
||||
list[k] = list[k] with { Value = sdk[k].Value };
|
||||
// An outage marker as the last sample: ours runs the outage to the window end (no end-of-data Partial on the
|
||||
// interval holding it; Bad / 100 % Bad after it); the SDK treats the data as ended there.
|
||||
if (outageEnds && (twoVariant || percent))
|
||||
{
|
||||
if (last >= plan.LowerTicks(i) && last < plan.UpperTicks(i))
|
||||
list[k] = list[k] with
|
||||
{
|
||||
StatusCode = (list[k].StatusCode & ~HistorianInfoBits.HistorianPartial)
|
||||
| (sdk[k].StatusCode & HistorianInfoBits.HistorianPartial),
|
||||
};
|
||||
else if (plan.LowerTicks(i) > last && (sdk[k].StatusCode & StatusBits.CodeBitsMask) == HistorianStatusCodes.BadNoData)
|
||||
list[k] = list[k] with { Value = null, StatusCode = HistorianStatusCodes.BadNoData };
|
||||
}
|
||||
// ...and TimeAverage/Total hold the last usable value up to that marker: Part 13 calls that extrapolation (no
|
||||
// Good value after it) ⇒ Uncertain; the SDK, which sees a later (Bad) sample, reports the held span Good.
|
||||
var lastUsable = data.Where(x => x.IsUsable).Max(x => x.TimeUtc).Ticks;
|
||||
if (outageEnds && agg is HistoryAggregateType.TimeAverage or HistoryAggregateType.Total
|
||||
&& plan.LowerTicks(i) < last && plan.UpperTicks(i) > lastUsable
|
||||
&& (list[k].StatusCode & StatusBits.CodeBitsMask) == HistorianStatusCodes.UncertainDataSubNormal
|
||||
&& (sdk[k].StatusCode & StatusBits.CodeBitsMask) == HistorianStatusCodes.Good)
|
||||
list[k] = list[k] with { StatusCode = list[k].StatusCode & ~StatusBits.CodeBitsMask };
|
||||
// Booleans: the SDK slopes them (values, and the status of a bound next to an Uncertain sample); ours steps.
|
||||
if (boolean && sdk[k].Value is not null && list[k].Value is not null)
|
||||
list[k] = list[k] with { Value = sdk[k].Value };
|
||||
if (boolean && (StatusBits.IsGood(list[k].StatusCode) || StatusBits.IsGood(sdk[k].StatusCode))
|
||||
&& !StatusBits.IsBad(list[k].StatusCode) && !StatusBits.IsBad(sdk[k].StatusCode))
|
||||
list[k] = list[k] with
|
||||
{
|
||||
StatusCode = (list[k].StatusCode & ~StatusBits.CodeBitsMask) | (sdk[k].StatusCode & StatusBits.CodeBitsMask),
|
||||
};
|
||||
}
|
||||
return list;
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[MemberData(nameof(Windows))]
|
||||
public void Time_weighted_aggregates_match_the_SDK(double from, double to, double intervalS)
|
||||
{
|
||||
var failures = new List<string>();
|
||||
foreach (var agg in new[]
|
||||
{
|
||||
HistoryAggregateType.TimeAverage, HistoryAggregateType.TimeAverage2, HistoryAggregateType.Total,
|
||||
HistoryAggregateType.Total2, HistoryAggregateType.Interpolative,
|
||||
HistoryAggregateType.Total2, HistoryAggregateType.Interpolative, HistoryAggregateType.PercentGood,
|
||||
HistoryAggregateType.PercentBad,
|
||||
})
|
||||
foreach (var (name, data) in new[] { ("Bounded", Bounded), ("Dense", Dense) })
|
||||
SdkAggregateOracle.ShouldMatch(MaskTruncation(Ours(agg, from, to, intervalS, data), from, to, intervalS),
|
||||
Sdk(agg, from, to, intervalS, data), $"{agg} {name} {from}→{to}/{intervalS}s",
|
||||
compareCalculatedInterpolatedBits: !(agg == HistoryAggregateType.Interpolative && from > to),
|
||||
compareCodeBits: agg is not (HistoryAggregateType.TimeAverage or HistoryAggregateType.Total)
|
||||
&& !(agg == HistoryAggregateType.Interpolative && from > to));
|
||||
foreach (var (name, data) in new[]
|
||||
{
|
||||
("Bounded", Bounded), ("Dense", Dense), ("EndsInside", EndsInside), ("OutageInside", OutageInside),
|
||||
("RecoveryEnds", RecoveryEnds), ("OutageLast", OutageLast), ("BoolSwitch", BoolSwitch),
|
||||
})
|
||||
{
|
||||
try
|
||||
{
|
||||
var sdk = Sdk(agg, from, to, intervalS, data);
|
||||
var ours = MaskDocumented(agg, MaskTruncation(Ours(agg, from, to, intervalS, data), from, to, intervalS),
|
||||
sdk, from, to, intervalS, data);
|
||||
SdkAggregateOracle.ShouldMatch(ours, sdk, $"{agg} {name} {from}→{to}/{intervalS}s",
|
||||
compareCalculatedInterpolatedBits: !(agg == HistoryAggregateType.Interpolative && from > to),
|
||||
compareCodeBits: !(agg == HistoryAggregateType.Interpolative && from > to));
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
failures.Add(ex.Message);
|
||||
}
|
||||
}
|
||||
failures.ShouldBeEmpty(string.Join("\n", failures));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
@@ -138,7 +305,6 @@ public sealed class SdkAggregateParityTests
|
||||
{
|
||||
SdkAggregateOracle.ShouldMatch(Ours(agg, from, to, intervalS, Dense), Sdk(agg, from, to, intervalS, Dense),
|
||||
$"{agg} Dense {from}→{to}/{intervalS}s", compareCalculatedInterpolatedBits: false,
|
||||
compareValues: !PercentAggregate(agg),
|
||||
// pre-existing, #680: the SDK's Start/End return the raw first/last sample (any quality) with its own
|
||||
// timestamp and stored status; ours return the first/last USABLE value stamped with the interval, under the
|
||||
// per-sample status rule. The SDK's Delta is Good even when a Bad sample sits in the interval; ours is
|
||||
@@ -159,20 +325,11 @@ public sealed class SdkAggregateParityTests
|
||||
HistoryAggregateType.Average, HistoryAggregateType.Minimum, HistoryAggregateType.Maximum,
|
||||
HistoryAggregateType.Count, HistoryAggregateType.Range, HistoryAggregateType.Start, HistoryAggregateType.End,
|
||||
HistoryAggregateType.Delta, HistoryAggregateType.MinimumActualTime, HistoryAggregateType.MaximumActualTime,
|
||||
HistoryAggregateType.PercentGood, HistoryAggregateType.PercentBad,
|
||||
];
|
||||
|
||||
/// <summary>
|
||||
/// pre-existing, #680: ours is the per-SAMPLE percentage (100 × Good samples / all samples); the SDK's
|
||||
/// <c>StatusAggregateCalculator</c> is the per-DURATION one (Good time / interval time). Different definitions, so the
|
||||
/// values are not compared; timestamp, status and Partial still are.
|
||||
/// </summary>
|
||||
private static bool StartOrEnd(HistoryAggregateType agg) =>
|
||||
agg is HistoryAggregateType.Start or HistoryAggregateType.End;
|
||||
|
||||
private static bool PercentAggregate(HistoryAggregateType agg) =>
|
||||
agg is HistoryAggregateType.PercentGood or HistoryAggregateType.PercentBad;
|
||||
|
||||
[Theory]
|
||||
[InlineData(HistoryAggregateType.TimeAverage)]
|
||||
[InlineData(HistoryAggregateType.TimeAverage2)]
|
||||
@@ -194,9 +351,7 @@ public sealed class SdkAggregateParityTests
|
||||
var ours = Ours(agg, 40, 0, 10, LateStart);
|
||||
SdkAggregateOracle.ShouldMatch(ours, Sdk(agg, 40, 0, 10, LateStart), $"{agg} reverse LateStart",
|
||||
compareCalculatedInterpolatedBits: tw,
|
||||
// pre-existing, #680: Total over the interval the data starts inside (TimeAverage × interval vs the SDK's
|
||||
// integral of the covered part); Start's raw-sample timestamp/status (see Per_sample_aggregates_match_the_SDK).
|
||||
compareValues: agg != HistoryAggregateType.Total,
|
||||
// pre-existing, #680: Start's raw-sample timestamp/status (see Per_sample_aggregates_match_the_SDK).
|
||||
compareTimestamps: !StartOrEnd(agg),
|
||||
compareCodeBits: !StartOrEnd(agg) && agg != HistoryAggregateType.Delta);
|
||||
ours.ShouldAllBe(x => (x.StatusCode & HistorianInfoBits.HistorianPartial) == 0);
|
||||
@@ -218,7 +373,7 @@ public sealed class SdkAggregateParityTests
|
||||
var tw = AggregateFamilies.FamilyOf(agg) == AggregateFamily.TimeWeighted;
|
||||
var ours = Ours(agg, 0, 40, 10, LateStart);
|
||||
SdkAggregateOracle.ShouldMatch(ours, Sdk(agg, 0, 40, 10, LateStart), $"{agg} LateStart",
|
||||
compareCalculatedInterpolatedBits: tw, compareValues: agg != HistoryAggregateType.Total);
|
||||
compareCalculatedInterpolatedBits: tw);
|
||||
(ours[0].StatusCode & HistorianInfoBits.HistorianPartial).ShouldBe(
|
||||
agg == HistoryAggregateType.Average ? 0u : HistorianInfoBits.HistorianPartial);
|
||||
ours.Skip(1).ShouldAllBe(x => (x.StatusCode & HistorianInfoBits.HistorianPartial) == 0);
|
||||
|
||||
+285
-30
@@ -17,6 +17,7 @@ public sealed class TimeWeightedAggregatorTests
|
||||
private const uint InvalidInputs = 0x80D60000u;
|
||||
private const uint BadCalc = 0x80000401u; // generic Bad | Calculated (an all-outage interval)
|
||||
private const uint GoodCalcPartial = 0x00000405u; // Good | Calculated | Partial (truncated last interval)
|
||||
private const uint UncertainCalcPartial = 0x40A40405u; // UncertainDataSubNormal | Calculated | Partial
|
||||
|
||||
private static IReadOnlyList<DataValueSnapshot> Run(HistoryAggregateType agg, double from, double to, double intervalS,
|
||||
params StoredSample[] samples) =>
|
||||
@@ -33,43 +34,286 @@ public sealed class TimeWeightedAggregatorTests
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- tag A: +0=10 G, +10=20 G, +20=Bad, +30=40 G, +40=50 G
|
||||
// Window [0,80), 20 s buckets. Raw: [0,10) 10→20 Good (150); [10,20) 20 held into the outage, Uncertain (200);
|
||||
// [20,30) Bad; [30,40) 40→50 Good (450); [40,80) 50 extrapolated, Uncertain (2000).
|
||||
// Bridged: [0,10) 150; [10,30) 20→40 = 250 in B0 + 350 in B1; [30,40) 450; [40,80) 50 held.
|
||||
// Window [0,80), 20 s buckets; nothing after +40 (no end bound: the data ENDS at +40). Raw: [0,10) 10→20 Good (150);
|
||||
// [10,20) 20 held into the outage, Uncertain (200); [20,30) Bad; [30,40) 40→50 Good (450); after +40 nothing (#680:
|
||||
// Part 13 "After End of Data" — the simple-bounding signal is not extrapolated).
|
||||
// Bridged: [0,10) 150; [10,30) 20→40 = 250 in B0 + 350 in B1; [30,40) 450; [40,80) 50 held (extrapolated).
|
||||
|
||||
[Fact]
|
||||
public void TimeAverage_bridges_the_outage_on_tag_A()
|
||||
{
|
||||
// B0 (150+250)/20 = 20; B1 (350+450)/20 = 40; B2, B3 = 50. Status (SDK rule, Uncertain time counts as Good):
|
||||
// B0 is non-Bad throughout (held-into-outage time is Uncertain) ⇒ Good; B1 has the Bad [20,30) ⇒ Uncertain;
|
||||
// B2, B3 are extrapolated (Uncertain) throughout ⇒ Good.
|
||||
// B0 (150+250)/20 = 20; B1 (350+450)/20 = 40; B2, B3 = 50. Status (#680, Part 13 + the SDK's interpolated-bounds
|
||||
// rule): Uncertain when a Bad value was skipped or the value was extrapolated — B0 and B1 hold the segment bridged
|
||||
// across the Bad +20; B2 and B3 are extrapolated past the last sample.
|
||||
var s = Run(HistoryAggregateType.TimeAverage, 0, 80, 20, A);
|
||||
Values(s, 20, 40, 50, 50);
|
||||
s.Select(x => x.StatusCode).ShouldBe([GoodCalc, UncertainCalc, GoodCalc, GoodCalc]);
|
||||
s.Select(x => x.StatusCode).ShouldBe([UncertainCalc, UncertainCalc, UncertainCalc, UncertainCalc]);
|
||||
s.Select(x => x.SourceTimestampUtc).ShouldBe([At(0), At(20), At(40), At(60)]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Total_is_TimeAverage_times_interval_seconds_on_tag_A()
|
||||
{
|
||||
// 20×20, 40×20, 50×20, 50×20.
|
||||
// Fully covered intervals: 20×20, 40×20, 50×20, 50×20 (the integral of the interpolated-bounding signal).
|
||||
Values(Run(HistoryAggregateType.Total, 0, 80, 20, A), 400, 800, 1000, 1000);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void TimeAverage2_excludes_the_bad_interval_on_tag_A()
|
||||
public void TimeAverage2_excludes_the_bad_interval_and_stops_at_the_end_of_data_on_tag_A()
|
||||
{
|
||||
// B0 350/20 = 17.5; B1 450/10 = 45 (the Bad [20,30) excluded); B2, B3 = 50.
|
||||
// B0 350/20 = 17.5; B1 450/10 = 45 (the Bad [20,30) excluded). B2 [40,60) holds the end of the data (+40, on its
|
||||
// start): no covered time, so the value is the last sample's (50), Good, Partial (#680; the SDK answers the same).
|
||||
// B3 is after the end of the data ⇒ BadNoData (Part 13 "After End of Data").
|
||||
var s = Run(HistoryAggregateType.TimeAverage2, 0, 80, 20, A);
|
||||
Values(s, 17.5, 45, 50, 50);
|
||||
s.Select(x => x.StatusCode).ShouldBe([GoodCalc, UncertainCalc, GoodCalc, GoodCalc]);
|
||||
Values(s, 17.5, 45, 50, null);
|
||||
s.Select(x => x.StatusCode).ShouldBe([GoodCalc, UncertainCalc, GoodCalcPartial, NoDataCalc]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Total2_integrates_only_the_non_bad_data_on_tag_A()
|
||||
{
|
||||
// B0 150+200 = 350; B1 450; B2, B3 1000.
|
||||
Values(Run(HistoryAggregateType.Total2, 0, 80, 20, A), 350, 450, 1000, 1000);
|
||||
// B0 150+200 = 350; B1 450; B2 nothing covered after the last sample ⇒ 0, Partial; B3 BadNoData.
|
||||
var s = Run(HistoryAggregateType.Total2, 0, 80, 20, A);
|
||||
Values(s, 350, 450, 0, null);
|
||||
s.Select(x => x.StatusCode).ShouldBe([GoodCalc, UncertainCalc, GoodCalcPartial, NoDataCalc]);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- #680: end of data, live tags, outages, Total coverage
|
||||
|
||||
/// <summary>−5 = 1, +3 = 2, +25 = 3 (all Good), nothing later: the data ends at +25.</summary>
|
||||
private static readonly StoredSample[] EndsInside =
|
||||
[
|
||||
S(-5, HistorianValue.FromDouble(1), Good), S(3, HistorianValue.FromDouble(2), Good),
|
||||
S(25, HistorianValue.FromDouble(3), Good),
|
||||
];
|
||||
|
||||
[Fact]
|
||||
public void TimeAverage2_and_Total2_stop_at_the_end_of_data_with_Partial_then_BadNoData()
|
||||
{
|
||||
// Window [0,40), 10 s. Segment +3..+25 slopes 2→3 (1/22 per s). [20,30): covered only [20,25): 2+17/22 → 3, area
|
||||
// 5 × (2.772727… + 3)/2 = 14.431818…; TimeAverage2 = 2.886363…, Good + Partial (the SDK: 14.4348 for Total2 —
|
||||
// its last region carries a 1 ms artifact). [30,40) is after the end of the data ⇒ BadNoData.
|
||||
var t2 = Run(HistoryAggregateType.Total2, 0, 40, 10, EndsInside);
|
||||
((double)t2[2].Value!).ShouldBe(5 * ((2 + 17d / 22) + 3) / 2, 1e-9);
|
||||
t2[2].StatusCode.ShouldBe(GoodCalcPartial);
|
||||
t2[3].StatusCode.ShouldBe(NoDataCalc);
|
||||
t2[3].Value.ShouldBeNull();
|
||||
var ta2 = Run(HistoryAggregateType.TimeAverage2, 0, 40, 10, EndsInside);
|
||||
((double)ta2[2].Value!).ShouldBe(((2 + 17d / 22) + 3) / 2, 1e-9);
|
||||
ta2.Select(x => x.StatusCode).ShouldBe([GoodCalc, GoodCalc, GoodCalcPartial, NoDataCalc]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void TimeAverage_and_Total_extrapolate_past_the_end_of_data_as_Uncertain_without_Partial()
|
||||
{
|
||||
// The interpolated-bounding signal holds the last value to the window end (stepped extrapolation) — Part 13 and
|
||||
// the SDK make that Uncertain; the SDK sets no end-of-data Partial on an extrapolated result.
|
||||
// [20,30): [20,25) 2.772727…→3 + [25,30) 3 ⇒ (14.431818… + 15) / 10; [30,40): 3.
|
||||
var ta = Run(HistoryAggregateType.TimeAverage, 0, 40, 10, EndsInside);
|
||||
((double)ta[2].Value!).ShouldBe((5 * ((2 + 17d / 22) + 3) / 2 + 15) / 10, 1e-9);
|
||||
((double)ta[3].Value!).ShouldBe(3d);
|
||||
ta.Select(x => x.StatusCode).ShouldBe([GoodCalc, GoodCalc, UncertainCalc, UncertainCalc]);
|
||||
Values(Run(HistoryAggregateType.Total, 30, 40, 0, EndsInside), 30);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void The_latest_interval_of_a_live_tag_is_Uncertain()
|
||||
{
|
||||
// A live tag: −10 = 10, +5 = 20, +12 = 30, nothing after yet. [10,20) is extrapolated from +12 ⇒ Uncertain
|
||||
// (#680 — it used to read Good). [0,10) is bridged Good→Good with no skip ⇒ Good. Values as the SDK:
|
||||
// [0,10) 20.952380…, [10,20) 29.714285….
|
||||
StoredSample[] live =
|
||||
[
|
||||
S(-10, HistorianValue.FromDouble(10), Good), S(5, HistorianValue.FromDouble(20), Good),
|
||||
S(12, HistorianValue.FromDouble(30), Good),
|
||||
];
|
||||
var s = Run(HistoryAggregateType.TimeAverage, 0, 20, 10, live);
|
||||
Values(s, 20.952380952380953, 29.71428571428571);
|
||||
s.Select(x => x.StatusCode).ShouldBe([GoodCalc, UncertainCalc]);
|
||||
Run(HistoryAggregateType.Total, 0, 20, 10, live)[1].StatusCode.ShouldBe(UncertainCalc);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void An_uncertain_sample_makes_TimeAverage_and_Total_uncertain()
|
||||
{
|
||||
// Part 13: "Uncertain if ... Uncertain values are used". −10 = 10 G, +5 = 20 U, +15 = 30 G, +40 = 30 G; [0,20):
|
||||
// the value 24.583333… (SDK), Uncertain.
|
||||
StoredSample[] unc =
|
||||
[
|
||||
S(-10, HistorianValue.FromDouble(10), Good), S(5, HistorianValue.FromDouble(20), Uncertain),
|
||||
S(15, HistorianValue.FromDouble(30), Good), S(40, HistorianValue.FromDouble(30), Good),
|
||||
];
|
||||
var ta = Run(HistoryAggregateType.TimeAverage, 0, 20, 0, unc).Single();
|
||||
((double)ta.Value!).ShouldBe(24.583333333333332, 1e-9);
|
||||
ta.StatusCode.ShouldBe(UncertainCalc);
|
||||
Run(HistoryAggregateType.Total, 0, 20, 0, unc).Single().StatusCode.ShouldBe(UncertainCalc);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_segment_bridged_across_a_bad_sample_is_uncertain_in_every_interval_it_touches()
|
||||
{
|
||||
// −10 = 10 G, +10 Bad, +15 = 20 G, +40 = 20 G: the bridged segment −10..+15 skips the Bad +10, so [0,10) — which
|
||||
// holds no Bad sample itself — is Uncertain too (the SDK agrees).
|
||||
StoredSample[] edge =
|
||||
[
|
||||
S(-10, HistorianValue.FromDouble(10), Good), S(10, HistorianValue.FromDouble(30), Bad),
|
||||
S(15, HistorianValue.FromDouble(20), Good), S(40, HistorianValue.FromDouble(20), Good),
|
||||
];
|
||||
var s = Run(HistoryAggregateType.TimeAverage, 0, 20, 10, edge);
|
||||
Values(s, 16, 19.5);
|
||||
s.Select(x => x.StatusCode).ShouldBe([UncertainCalc, UncertainCalc]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Total_over_an_interval_the_data_starts_inside_integrates_only_the_covered_part()
|
||||
{
|
||||
// +5 = 10, +15 = 10, +40 = 10, nothing before: [0,20) is covered from +5 only ⇒ Total = 10 × 15 s = 150 (Part 13
|
||||
// partial-interval width; the SDK agrees). TimeAverage is still 10. Uncertain (no data for [0,5)) + Partial.
|
||||
StoredSample[] late =
|
||||
[
|
||||
S(5, HistorianValue.FromDouble(10), Good), S(15, HistorianValue.FromDouble(10), Good),
|
||||
S(40, HistorianValue.FromDouble(10), Good),
|
||||
];
|
||||
var total = Run(HistoryAggregateType.Total, 0, 20, 0, late).Single();
|
||||
((double)total.Value!).ShouldBe(150d);
|
||||
total.StatusCode.ShouldBe(UncertainCalcPartial);
|
||||
((double)Run(HistoryAggregateType.TimeAverage, 0, 20, 0, late).Single().Value!).ShouldBe(10d);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_recovery_sample_after_an_outage_that_ends_the_data_is_its_own_value_Uncertain_and_Partial()
|
||||
{
|
||||
// #680 review (the comms-loss recovery shape): −10 = 10 G, +5 outage, +25 = 20 G (the recovery), nothing after.
|
||||
// [0,20): 10 held into the outage over [0,5) ⇒ TimeAverage2 10 (Total2 50), Uncertain. [20,40) holds the end of the
|
||||
// data but no non-Bad time before it ⇒ the sample's value 20 (Total2 0), Uncertain + Partial — never Bad/null (the
|
||||
// SDK answers the same). [40,60) is after the end of the data ⇒ BadNoData.
|
||||
StoredSample[] s =
|
||||
[
|
||||
S(-10, HistorianValue.FromDouble(10), Good), S(5, HistorianValue.Empty, BadNoCommunication),
|
||||
S(25, HistorianValue.FromDouble(20), Good),
|
||||
];
|
||||
var ta2 = Run(HistoryAggregateType.TimeAverage2, 0, 60, 20, s);
|
||||
Values(ta2, 10, 20, null);
|
||||
ta2.Select(x => x.StatusCode).ShouldBe([UncertainCalc, UncertainCalcPartial, NoDataCalc]);
|
||||
var t2 = Run(HistoryAggregateType.Total2, 0, 60, 20, s);
|
||||
Values(t2, 50, 0, null);
|
||||
t2.Select(x => x.StatusCode).ShouldBe([UncertainCalc, UncertainCalcPartial, NoDataCalc]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_single_sample_is_its_own_value_Uncertain_and_Partial()
|
||||
{
|
||||
// #680 review: the tag's only sample, +25 = 20. One interval [0,60) ⇒ 20 (Total2 0), Uncertain + Partial (time
|
||||
// before the data is Bad). At 20 s: [0,20) before the data ⇒ BadNoData; [20,40) the sample ⇒ 20, Uncertain +
|
||||
// Partial; [40,60) after the end of the data ⇒ BadNoData.
|
||||
StoredSample[] one = [S(25, HistorianValue.FromDouble(20), Good)];
|
||||
var whole = Run(HistoryAggregateType.TimeAverage2, 0, 60, 0, one).Single();
|
||||
((double)whole.Value!).ShouldBe(20d);
|
||||
whole.StatusCode.ShouldBe(UncertainCalcPartial);
|
||||
var total = Run(HistoryAggregateType.Total2, 0, 60, 0, one).Single();
|
||||
((double)total.Value!).ShouldBe(0d);
|
||||
total.StatusCode.ShouldBe(UncertainCalcPartial);
|
||||
var split = Run(HistoryAggregateType.TimeAverage2, 0, 60, 20, one);
|
||||
Values(split, null, 20, null);
|
||||
split.Select(x => x.StatusCode).ShouldBe([NoDataCalc, UncertainCalcPartial, NoDataCalc]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void An_outage_marker_as_the_last_sample_runs_Bad_to_the_window_end()
|
||||
{
|
||||
// #680 review, a deliberate difference from the SDK (which treats the data as ended at the marker): −10 = 10 G,
|
||||
// +25 = 20 G, +30 outage, nothing after. TimeAverage2 [20,40): [20,25) 18.5714…→20 (96.43) + [25,30) 20 held into
|
||||
// the outage (100) over 10 s = 19.642857…, Uncertain, NOT Partial (the data did not end — the tag is in an
|
||||
// outage); [40,60) inside the still-open outage ⇒ generic Bad. PercentBad [40,60) = 100, Good. TimeAverage holds
|
||||
// 20 to the window end — extrapolated (no Good value follows) ⇒ Uncertain.
|
||||
StoredSample[] s =
|
||||
[
|
||||
S(-10, HistorianValue.FromDouble(10), Good), S(25, HistorianValue.FromDouble(20), Good),
|
||||
S(30, HistorianValue.Empty, BadNoCommunication),
|
||||
];
|
||||
var ta2 = Run(HistoryAggregateType.TimeAverage2, 0, 60, 20, s);
|
||||
((double)ta2[1].Value!).ShouldBe(19.642857142857142, 1e-9);
|
||||
ta2.Select(x => x.StatusCode).Skip(1).ShouldBe([UncertainCalc, BadCalc]);
|
||||
var pb = Run(HistoryAggregateType.PercentBad, 0, 60, 20, s);
|
||||
Values(pb, 0, 50, 100);
|
||||
pb.ShouldAllBe(x => x.StatusCode == GoodCalc);
|
||||
Run(HistoryAggregateType.TimeAverage, 0, 60, 20, s).Select(x => x.StatusCode).Skip(1)
|
||||
.ShouldBe([UncertainCalc, UncertainCalc]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Boolean_percent_regions_take_the_stepped_bound_status()
|
||||
{
|
||||
// #680 review: −10 = true G, +15 = true U, +30 = false G, +50 = false G. [0,20): the bound at +0 is stepped from the
|
||||
// Good −10 (a Boolean is held, not sloped toward the Uncertain +15) ⇒ [0,15) Good, [15,20) Uncertain ⇒ PercentGood
|
||||
// 75. (The SDK slopes Booleans, so it calls the bound Uncertain and answers 0.) [20,40): [20,30) Uncertain (bound
|
||||
// stepped from the Uncertain +15), [30,40) Good ⇒ 50.
|
||||
StoredSample[] s =
|
||||
[
|
||||
S(-10, HistorianValue.FromBoolean(true), Good), S(15, HistorianValue.FromBoolean(true), Uncertain),
|
||||
S(30, HistorianValue.FromBoolean(false), Good), S(50, HistorianValue.FromBoolean(false), Good),
|
||||
];
|
||||
Values(Run(HistoryAggregateType.PercentGood, 0, 40, 20, s), 75, 50);
|
||||
// TimeAverage over the same Booleans is stepped: [20,40) = 1 over [20,30), 0 over [30,40) ⇒ 0.5 (the SDK: 1).
|
||||
Values(Run(HistoryAggregateType.TimeAverage, 0, 40, 20, s), 1, 0.5);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- #680: PercentGood / PercentBad are duration-based
|
||||
|
||||
[Fact]
|
||||
public void Percent_aggregates_are_durations_on_tag_A()
|
||||
{
|
||||
// Stepped regions, each with the status of the sample that starts it. B0 [0,20) Good throughout; B1 [20,30) Bad
|
||||
// (the Bad +20), [30,40) Good ⇒ 50 / 50; B2 holds the end of the data (+40, Good) ⇒ its region runs to the interval
|
||||
// end (Part 13), Partial (forward end-of-data, as the SDK); B3 is after the end of the data ⇒ BadNoData.
|
||||
var good = Run(HistoryAggregateType.PercentGood, 0, 80, 20, A);
|
||||
Values(good, 100, 50, 100, null);
|
||||
good.Select(x => x.StatusCode).ShouldBe([GoodCalc, GoodCalc, GoodCalcPartial, NoDataCalc]);
|
||||
var bad = Run(HistoryAggregateType.PercentBad, 0, 80, 20, A);
|
||||
Values(bad, 0, 50, 0, null);
|
||||
bad.Select(x => x.StatusCode).ShouldBe([GoodCalc, GoodCalc, GoodCalcPartial, NoDataCalc]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void With_no_end_bound_the_last_region_runs_to_the_interval_end()
|
||||
{
|
||||
// 0 = 10 G, +5 outage, +10 = 30 G, nothing after; [0,20) one interval: [0,5) Good, [5,10) Bad, [10,20) Good
|
||||
// (Part 13: the last region extends to the interval end — the SDK instead gives it 1 ms, ≈ 49.995 % Bad).
|
||||
StoredSample[] s =
|
||||
[
|
||||
S(0, HistorianValue.FromDouble(10), Good), S(5, HistorianValue.Empty, BadNoCommunication),
|
||||
S(10, HistorianValue.FromDouble(30), Good),
|
||||
];
|
||||
var bad = Run(HistoryAggregateType.PercentBad, 0, 20, 0, s).Single();
|
||||
((double)bad.Value!).ShouldBe(25d, 1e-9);
|
||||
bad.StatusCode.ShouldBe(GoodCalcPartial);
|
||||
((double)Run(HistoryAggregateType.PercentGood, 0, 20, 0, s).Single().Value!).ShouldBe(75d, 1e-9);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Percent_regions_use_the_simple_bound_at_the_interval_start_and_count_time_before_the_data_as_Bad()
|
||||
{
|
||||
// −3 = 10 G, +4 outage, +6 = 30 U, +12 = 3 G, +30 = 3 G; [0,20), 10 s. The simple bound at +0 lies on a segment
|
||||
// that runs into the outage ⇒ Uncertain for [0,4) (the SDK's simple bounding value); [4,6) Bad; [6,10) Uncertain.
|
||||
// B0: Good 0 %, Bad 20 %. B1: [10,12) Uncertain (bound between the Uncertain +6 and +12), [12,20) Good ⇒ 80 / 0.
|
||||
StoredSample[] s =
|
||||
[
|
||||
S(-3, HistorianValue.FromDouble(10), Good), S(4, HistorianValue.Empty, BadNoCommunication),
|
||||
S(6, HistorianValue.FromDouble(30), Uncertain), S(12, HistorianValue.FromDouble(3), Good),
|
||||
S(30, HistorianValue.FromDouble(3), Good),
|
||||
];
|
||||
Values(Run(HistoryAggregateType.PercentGood, 0, 20, 10, s), 0, 80);
|
||||
Values(Run(HistoryAggregateType.PercentBad, 0, 20, 10, s), 20, 0);
|
||||
// The data starts at +10, exactly on the end of [0,10): all of [0,10) is before the data ⇒ 100 % Bad (the SDK
|
||||
// agrees); data starting at +5 ⇒ [0,5) Bad, Partial.
|
||||
StoredSample[] late = [S(10, HistorianValue.FromDouble(1), Good), S(30, HistorianValue.FromDouble(2), Good)];
|
||||
Values(Run(HistoryAggregateType.PercentBad, 0, 20, 10, late), 100, 0);
|
||||
StoredSample[] mid = [S(5, HistorianValue.FromDouble(1), Good), S(30, HistorianValue.FromDouble(2), Good)];
|
||||
var m = Run(HistoryAggregateType.PercentBad, 0, 20, 0, mid).Single();
|
||||
((double)m.Value!).ShouldBe(25d, 1e-9);
|
||||
m.StatusCode.ShouldBe(GoodCalcPartial);
|
||||
// Nothing at all ⇒ BadNoData.
|
||||
Run(HistoryAggregateType.PercentGood, 0, 20, 10).ShouldAllBe(x => x.StatusCode == NoDataCalc);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -146,12 +390,13 @@ public sealed class TimeWeightedAggregatorTests
|
||||
[Fact]
|
||||
public void Past_the_last_sample_the_value_is_extrapolated_stepped()
|
||||
{
|
||||
// -5 = 7 and nothing later: held at 7 over [0,10). The extrapolated time is Uncertain, which the SDK rule
|
||||
// (TreatUncertainAsBad = false) counts as Good time ⇒ the interval is Good; the Interpolative point is Uncertain.
|
||||
// -5 = 7 and nothing later: TimeAverage holds 7 over [0,10), Uncertain (extrapolated, #680). TimeAverage2 does not
|
||||
// extrapolate: [0,10) is after the end of the data ⇒ BadNoData. The Interpolative point is Uncertain.
|
||||
StoredSample[] one = [S(-5, HistorianValue.FromDouble(7), Good)];
|
||||
var s = Run(HistoryAggregateType.TimeAverage2, 0, 10, 0, one).Single();
|
||||
var s = Run(HistoryAggregateType.TimeAverage, 0, 10, 0, one).Single();
|
||||
((double)s.Value!).ShouldBe(7d);
|
||||
s.StatusCode.ShouldBe(GoodCalc);
|
||||
s.StatusCode.ShouldBe(UncertainCalc);
|
||||
Run(HistoryAggregateType.TimeAverage2, 0, 10, 0, one).Single().StatusCode.ShouldBe(NoDataCalc);
|
||||
var i = Run(HistoryAggregateType.Interpolative, 0, 10, 0, one).Single();
|
||||
((double)i.Value!).ShouldBe(7d);
|
||||
i.StatusCode.ShouldBe(UncertainInterpolated);
|
||||
@@ -208,13 +453,17 @@ public sealed class TimeWeightedAggregatorTests
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void An_interval_entirely_inside_an_outage_is_Bad_but_Interpolative_bridges_it()
|
||||
public void An_interval_entirely_inside_an_outage_bridges_for_TimeAverage_but_is_Bad_for_TimeAverage2()
|
||||
{
|
||||
// Window [10,15): raw is Bad throughout ⇒ data but no non-Bad time ⇒ generic Bad (the SDK rule; BadNoData is
|
||||
// reserved for no data at all). Interpolative at +10 = 10 + 20·(10/15).
|
||||
// Window [10,15): the interpolated-bounding signal (10→30 over [0,15)) bridges it — 23.333… → 30, mean 26.666…,
|
||||
// Uncertain (a Bad value skipped; #680, Part 13 + the SDK). Total = 26.666… × 5 s. The simple-bounding signal is
|
||||
// Bad throughout ⇒ TimeAverage2/Total2 generic Bad (Part 13's PercentData rule; the SDK answers BadNoData).
|
||||
// Interpolative at +10 = 10 + 20·(10/15).
|
||||
var ta = Run(HistoryAggregateType.TimeAverage, 10, 15, 0, Outage).Single();
|
||||
ta.StatusCode.ShouldBe(BadCalc);
|
||||
ta.Value.ShouldBeNull();
|
||||
((double)ta.Value!).ShouldBe((10d + 20d * 10d / 15d + 30d) / 2, 1e-9);
|
||||
ta.StatusCode.ShouldBe(UncertainCalc);
|
||||
((double)Run(HistoryAggregateType.Total, 10, 15, 0, Outage).Single().Value!)
|
||||
.ShouldBe((10d + 20d * 10d / 15d + 30d) / 2 * 5, 1e-9);
|
||||
Run(HistoryAggregateType.TimeAverage2, 10, 15, 0, Outage).Single().StatusCode.ShouldBe(BadCalc);
|
||||
Run(HistoryAggregateType.Total2, 10, 15, 0, Outage).Single().StatusCode.ShouldBe(BadCalc);
|
||||
var i = Run(HistoryAggregateType.Interpolative, 10, 15, 0, Outage).Single();
|
||||
@@ -324,11 +573,14 @@ public sealed class TimeWeightedAggregatorTests
|
||||
[Fact]
|
||||
public void A_reverse_window_is_returned_latest_first_stamped_with_later_edges()
|
||||
{
|
||||
// #677: the four whole intervals are the same spans as forward, but each is stamped with its later edge.
|
||||
// #677: the four whole intervals are the same spans as forward, but each is stamped with its later edge. The data
|
||||
// ends at +40 (#680): the latest interval is after it ⇒ BadNoData; the next holds it on its earlier edge ⇒ 50, with
|
||||
// no end-of-data Partial on a reverse read (the SDK answers the same).
|
||||
var s = TimeWeightedAggregator.Evaluate(HistoryAggregateType.TimeAverage2,
|
||||
BucketPlan.Create(At(80), At(0), TimeSpan.FromSeconds(20)), A, Now);
|
||||
s.Select(x => x.SourceTimestampUtc).ShouldBe([At(80), At(60), At(40), At(20)]);
|
||||
Values(s, 50, 50, 45, 17.5);
|
||||
Values(s, null, 50, 45, 17.5);
|
||||
s.Select(x => x.StatusCode).ShouldBe([NoDataCalc, GoodCalc, UncertainCalc, GoodCalc]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -336,13 +588,15 @@ public sealed class TimeWeightedAggregatorTests
|
||||
{
|
||||
// Tag A, At(50) → At(0), 20 s ⇒ (30,50], (10,30], (0,10]. Raw signal: [0,10) 10→20 Good (150); [10,20) 20 held
|
||||
// into the outage, Uncertain (200); [20,30) Bad; [30,40) 40→50 Good (450); [40,50) 50 extrapolated (500).
|
||||
// (30,50]: 950/20 = 47.5 Good; (10,30]: 200/10 = 20, the Bad 10 s ⇒ Uncertain; (0,10]: 150/10 = 15, truncated ⇒
|
||||
// (30,50]: the data ends at +40 (#680: no extrapolation) ⇒ 450/10 = 45, Good; the latest sample lies strictly
|
||||
// inside it, where the reverse read's data starts ⇒ Partial (the SDK's m_startOfData keyed on the first value it
|
||||
// is fed); (10,30]: 200/10 = 20, the Bad 10 s ⇒ Uncertain; (0,10]: 150/10 = 15, truncated ⇒
|
||||
// Partial. The old layout would have answered [0,20), [20,40), [40,50) stamped 0/20/40, partial at the LATEST end.
|
||||
var s = TimeWeightedAggregator.Evaluate(HistoryAggregateType.TimeAverage2,
|
||||
BucketPlan.Create(At(50), At(0), TimeSpan.FromSeconds(20)), A, Now);
|
||||
s.Select(x => x.SourceTimestampUtc).ShouldBe([At(50), At(30), At(10)]);
|
||||
Values(s, 47.5, 20, 15);
|
||||
s.Select(x => x.StatusCode).ShouldBe([GoodCalc, UncertainCalc, GoodCalcPartial]);
|
||||
Values(s, 45, 20, 15);
|
||||
s.Select(x => x.StatusCode).ShouldBe([GoodCalcPartial, UncertainCalc, GoodCalcPartial]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -368,8 +622,9 @@ public sealed class TimeWeightedAggregatorTests
|
||||
// Window [0,50), 20 s: B2 = [40,50) is 10 s of the extrapolated 50 ⇒ TimeAverage 50, Total 500.
|
||||
var s = Run(HistoryAggregateType.Total, 0, 50, 20, A);
|
||||
Values(s, 400, 800, 500);
|
||||
// Only the truncated interval carries the Partial bit.
|
||||
s.Select(x => x.StatusCode).ShouldBe([GoodCalc, UncertainCalc, GoodCalcPartial]);
|
||||
// Only the truncated interval carries the Partial bit (Part 13 §5.3.3.2 Case 3 — the SDK does not set it; ours
|
||||
// does, deliberately). Every interval is Uncertain (bridged across the Bad +20, or extrapolated).
|
||||
s.Select(x => x.StatusCode).ShouldBe([UncertainCalc, UncertainCalc, UncertainCalcPartial]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
||||
Reference in New Issue
Block a user