fix(historian): refuse native Gateway TimeAverage/Total until historiangw#27 (lmxopcua#746)

AVEVA's native TimeAverage / Total are answered with Good-status wrong values
while the gateway sends the interval in the wrong unit (historiangw#27): one
whole-span value on the earliest interval, the rest absent. Nothing on the
wire marks the answer as degraded.

Both are now refused BadAggregateNotSupported (unsent, not counted as a
backend failure, not advertised in AggregateFunctions) unless
ServerHistorian:Gateway:EnableNativeTimeWeighted is true. The default
Gateway set is the ten raw-evaluated aggregates
(GatewayHistorianDataSource.RawEvaluatedAggregates); ServedAggregates stays
the 12 the backend can serve with the flag on. The native-path tests opt in.

Also documents the ServerHistorian:Gateway sub-section (EventDrainBudget from
the previous commit) and the events drain in docs/Historian.md.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-10-22 16:30:19 -04:00
parent 24609ececa
commit 0b85c499e3
11 changed files with 157 additions and 16 deletions
+19 -3
View File
@@ -156,6 +156,13 @@ Processed read is `BadAggregateNotSupported` and no history aggregate functions
| `HistoryReadDeadline` | TimeSpan | `00:01:00` | Per-request wall-clock deadline for a HistoryRead (and HistoryUpdate) batch (arch-review 03/S3). A node still in flight at the deadline surfaces `BadTimeout`, so one request can never hold an SDK request thread longer than this regardless of node count. Non-positive = unbounded (a `Validate()` warning). |
| `HistoryUpdateMaxFutureSkew` | TimeSpan | `1.00:00:00` | #674. How far past the server clock a HistoryUpdate (Insert / Replace / Update) item's `SourceTimestamp` may lie; an item beyond `now + skew` is refused **`BadOutOfRange`** by the node manager before any backend sees it (every backend). Guards against a far-future typo (a 2062 timestamp). The live recorder is not bounded. Non-positive disables the bound (no warning — a deliberate choice). See [HistoryUpdate](#historyupdate). |
**`ServerHistorian:Gateway`** (read only when `Backend=Gateway`): knobs that exist because of the HistorianGateway's wire contract.
| Key | Type | Default | Description |
|---|---|---|---|
| `Gateway:EnableNativeTimeWeighted` | bool | `false` | Serve HistoryRead-Processed `TimeAverage` / `Total` through AVEVA's native retrieval modes. ⚠️ **Off by default (client-visible):** the gateway sends AVEVA the interval in the wrong unit (historiangw#27), so AVEVA answers one Good-status whole-span value on the earliest interval and nothing for the rest — wrong values a client cannot tell from right ones. Off, both are `BadAggregateNotSupported` and are **not advertised** (`AggregateFunctions`), like the other five time-weighted aggregates on this backend. Turn it on only after a fixed gateway is live-verified. |
| `Gateway:EventDrainBudget` | int | `10000` | Stream rows a capped HistoryReadEvents read consumes (and discards) past its page cap before it abandons the gateway stream. The gateway's `ReadEvents` has no max-events field, so the cap is applied client-side; an **abandoned** stream cancels the RPC and the gateway evicts its pooled historian session (the next read reconnects; concurrent reads on that session fail). The usual small overshoot is drained to the stream's end; past the budget the stream is abandoned, so a page over a window far larger than the page does not stream the whole window. Rows of another source never count before the "more events" signal is seen, so an abandoned stream never answers a page as complete. `0` = abandon at the cap (the previous behaviour); negative counts as `0` (a `Validate()` warning). |
**`ServerHistorian:SqlServer`** (read only when `Backend=SqlServer`; bound by `HistorianBackendRegistry.BindSqlServer`):
| Key | Type | Default | Validation | Description |
@@ -987,6 +994,10 @@ says is complete carries no continuation point. `NumValuesPerNode == 0` stays un
- **Tie exactness depends on the backend.** SqlServer and QuestDB order by `(EventTimeUtc, EventId)`, so the skip count is exact.
The Gateway backend has no stable intra-timestamp order (and only a truncation flag), so ties at a page boundary of a
**forward** read are best-effort there; a walked-back **reverse** read (below) sorts them by `EventId` itself and is exact.
- **Gateway streams are drained, not abandoned.** The gateway's `ReadEvents` carries no max-events field, so a capped page is cut
client-side; the rest of the stream is consumed (uncollected) up to `ServerHistorian:Gateway:EventDrainBudget` rows (default
10 000) so the gateway does not evict its historian session for a cancelled stream. A page over a window much larger than the
budget still abandons its stream. A caller cancellation (client abort, `HistoryReadDeadline`) is not counted as a backend failure.
- **Reverse event reads page newest-first (#675).** `StartTime > EndTime` reads the Part 11 reverse window `(EndTime, StartTime]`
with events in DESCENDING `(EventTimeUtc, EventId)` order, so a cap keeps the **latest** N and a continuation point is issued
when the source flags more. The direction is stored **in the cursor**: a reverse point resumes reverse (and a forward one
@@ -1171,6 +1182,8 @@ gateway refuses `start > end` and `interval <= 0`):
intervals (e.g. 1 s over 2 days) is refused up front with `BadHistoryOperationUnsupported`, as on the SQL backends — it used to be
served up to the gateway's own `MaxAggregateIntervals` (1,000,000). Not counted as a backend failure. A clipped interval shorter than
1 ms (a sub-ms window end) is sent as a sub-ms interval request; re-check it in the post-historiangw#27 live leg.
- ⚠️ **Native TimeAverage / Total are off by default** (`ServerHistorian:Gateway:EnableNativeTimeWeighted=false` ⇒
`BadAggregateNotSupported`, unsent, unadvertised) because of the next point; everything in this section describes the opted-in path.
- ⚠️ **Until historiangw#27 is fixed, native values are wrong in BOTH directions with a Good status:** AVEVA answers one row per request
(the whole span's value), which lands on that request's first interval as a Good, Calculated value while the other intervals are
absent. Pinned against a native-mode fake only — re-run the forward and reverse native legs live once the gateway is fixed. Since #681 every native bucket is stamped with its interval **start**
@@ -1192,9 +1205,12 @@ field (`tail + 43`) its own reverse-engineering notes still mark unverified (His
> statuses may differ from AVEVA's own modes; every per-sample result now carries the Calculated info bits; a per-sample read is
> bounded by **`ServerHistorian:MaxTimeWeightedRawRows`** (in-window raw rows; past it ⇒ `BadTooManyOperations` for that node — read
> a long window over a dense tag in shorter windows). `TimeAverage` and `Total` stay AVEVA-native (`TimeWeightedAverage` /
> `Integral`), now stamped with the interval start (was its end) and carrying the Calculated bit; `TimeAverage2`, `Total2`,
> `Interpolative`, `PercentGood` and `PercentBad` remain `BadAggregateNotSupported` on Gateway. The served set is the single
> static `GatewayHistorianDataSource.ServedAggregates`.
> `Integral`) — ⚠️ **and are refused `BadAggregateNotSupported` unless `ServerHistorian:Gateway:EnableNativeTimeWeighted` is
> `true`** (default `false`, until historiangw#27 is live-verified; the default set is the ten raw-evaluated aggregates,
> `GatewayHistorianDataSource.RawEvaluatedAggregates`) — now stamped with the interval start (was its end) and carrying the Calculated bit; `TimeAverage2`, `Total2`,
> `Interpolative`, `PercentGood` and `PercentBad` remain `BadAggregateNotSupported` on Gateway. The most the backend can serve is
> `GatewayHistorianDataSource.ServedAggregates` (12); an instance serves and advertises `SupportedAggregates` — those 12 with the
> flag on, the 10 raw-evaluated ones without it.
**How the Gateway raw evaluation works (#681).** Forward `ReadRaw` reads only (the gateway streams oldest-first and refuses
`end < start`; `BucketPlan` assembles a reverse result newest-first), O(buckets) memory, and **every gateway stream is consumed to