fix(historian): #687 review — one bounded re-read instead of failing, raw budget unclamped, exclusive forward end
- M1/M2: when the 1 ms pre-window margin (boundary row + stored samples, e.g. a >16-sample tie cluster at start-1ms / lastTs-1ms) uses up the cap + 16 budget, re-read ONCE asking for cap + m + 16 (m = pre-window rows seen); fail BadTooManyOperations only if the re-read is still short because of them. The raw budget is no longer clamped at the gateway default ceiling (AVEVA always emits the boundary row, so page sizes 999,985-1,000,000 always came back short); the #688 clamp stays on the per-sample aggregate path only. - Merge trap: NumValuesPerNode 0 => int.MaxValue, never through the clamp. - L3: the forward end is exclusive ([start, end), the IHistorianDataSource contract); a point read (start == end, the tie-cluster over-fetch) inclusive. - RequestStartFor never goes before the Unix epoch (#686's walk-back floor). - L4 (docs): an Uncertain native row for an all-NULL interval is unverified live.
This commit is contained in:
+25
-12
@@ -682,18 +682,28 @@ data.
|
||||
> `18:06:21 v=30 Good`), a window before the tag's history answered `v=0 Bad` instead of `GoodNoData`, and a start-only read led
|
||||
> with the fabricated row. It is now dropped; such windows read `GoodNoData`.
|
||||
|
||||
- **How.** The same classification the raw-evaluated aggregates use (#681, `RawAggregateReader.RequestStartFor` /
|
||||
`IsWindowSample` — one authority): the gateway is asked from **1 ms before** the window, so the boundary row lands before it, and
|
||||
only rows stamped at or after StartTime are returned. The decision is by **timestamp**, not by value or quality — a stored NULL
|
||||
sample looks exactly like the no-data boundary row on the wire (value 0, Bad), and a stored NULL inside the window is still
|
||||
returned (Bad), as it is a stored sample.
|
||||
- **Paging (#669) is unaffected.** The gateway is asked for `NumValuesPerNode + 16` rows (`NumValuesPerNode = 0` stays unlimited),
|
||||
so the boundary row cannot shorten a page: a full page still holds exactly `NumValuesPerNode` stored samples and carries a
|
||||
continuation point, and a resumed page (which re-reads from the last returned timestamp inclusive, and the tie-cluster over-fetch
|
||||
`start == end` read) no longer gets a fabricated row at its resume point. If more than 16 rows lie in the 1 ms before the window
|
||||
and exhaust that allowance, the page cannot be read completely and the node fails (`HistoryReadLimitExceededException`, not
|
||||
counted as a backend fault) rather than answering a short page that would end the client's paging early. Every gateway stream is
|
||||
still drained to completion.
|
||||
- **How.** The same request start the raw-evaluated aggregates use (#681, `RawAggregateReader.RequestStartFor` — one authority):
|
||||
the gateway is asked from **1 ms before** the window (never before the Unix epoch, the start of the historian's time encoding),
|
||||
so the boundary row lands before it, and only rows `RawAggregateReader.IsWindowSample` are returned. The decision is by
|
||||
**timestamp**, not by value or quality — a stored NULL sample looks exactly like the no-data boundary row on the wire (value 0,
|
||||
Bad), and a stored NULL inside the window is still returned (Bad), as it is a stored sample.
|
||||
- **The forward end is exclusive** (`[StartTime, EndTime)`, the `IHistorianDataSource` contract and what the SQL backends return;
|
||||
the gateway's own `ReadRaw` end is inclusive, so a sample stored exactly at EndTime used to be returned). A point read
|
||||
(`StartTime == EndTime` — the server's tie-cluster over-fetch) still returns the samples at that instant. The #686 reverse
|
||||
walk-back is exact either way.
|
||||
- **Row budget.** `NumValuesPerNode = 0` ("all values") asks the gateway for an unbounded `max_values` (`int.MaxValue` — the client
|
||||
package and the gateway reject 0, #686). A cap asks for `NumValuesPerNode + 16` rows, **not** clamped at the gateway's default
|
||||
ceiling (#688 clamps only the per-sample aggregate budget): AVEVA always emits the boundary row, so a clamped budget would make
|
||||
every page size from 999,985 to 1,000,000 come back short. A gateway with `AllowStreamingBeyondCeiling=false` refuses an
|
||||
over-ceiling budget instead ⇒ `BadTooManyOperations` (not a backend fault); a default gateway serves it.
|
||||
- **Paging (#669) is unaffected.** A full page still holds exactly `NumValuesPerNode` stored samples and carries a continuation
|
||||
point, and a resumed page (which re-reads from the last returned timestamp inclusive, and the tie-cluster over-fetch
|
||||
`start == end` read) no longer gets a fabricated row at its resume point. A read is complete when the stream came back short,
|
||||
filled the page, or reached EndTime. If rows **before** the window — the boundary row plus stored samples inside the 1 ms margin,
|
||||
e.g. a tie cluster at `StartTime − 1 ms` or at a resume point `− 1 ms` — used up the budget, the window is re-read **once**
|
||||
asking for `NumValuesPerNode + m + 16` (m = the pre-window rows the first read saw, unclamped). Only if that re-read is still
|
||||
short because of pre-window rows does the node fail `BadTooManyOperations` (not a backend fault) rather than answer a short page
|
||||
that would end the client's paging early. Every gateway stream is still drained to completion.
|
||||
- **`ReturnBounds` is not honoured** — on any backend: the server does not pass it to the data source, so no bounding values are
|
||||
returned (see [Known limitations](#known-limitations)). The AVEVA boundary row is not used as a bound either: it is stamped at the
|
||||
query start rather than carrying the time of the real prior sample, so it could only ever be a Part 11 *interpolated* bound, which
|
||||
@@ -845,6 +855,9 @@ fixed:
|
||||
- ⚠️ **End-to-end correctness also needs the gateway fix historiangw#27:** the gateway sends the aggregate interval to AVEVA in
|
||||
ticks where AVEVA reads milliseconds (10,000× too large), so today a native read returns one row for the whole window (plus the
|
||||
pre-window row, now dropped). OtOpcUa does not work around it; per-interval values return once the gateway is fixed.
|
||||
- **Not verified live:** what AVEVA answers for an interval holding **only** NULL samples (no good time at all). If it marks such a
|
||||
row Uncertain (`opc_quality` 64–191) with a placeholder value, that value is now passed through as Uncertain; the gate only
|
||||
observed Uncertain rows that also held good time. Re-check on the VPN once historiangw#27 is fixed.
|
||||
|
||||
**Integral unit (#677).** AVEVA's integral is `∫ value dt` (seconds) ÷ the tag's `IntegralDivisor`. The proto3 `double` default for
|
||||
`HistorianTagDefinition.IntegralDivisor` is `0`, so `GatewayTagProvisioner` now sends **`IntegralDivisor = 1`** explicitly on every
|
||||
|
||||
Reference in New Issue
Block a user