fix(historian): refuse aggregates not valid for the node's DataType up front (#897)
v2-ci / unit-tests (push) Has been cancelled

TimeAverage / Total on a Boolean node over the Gateway backend (AVEVA discrete tag, live
SysPulse) went to the gateway since native TimeAverage/Total became default-on (#694). AVEVA
refuses both modes at StartQuery; the gateway reports that only as Internal /
HistorianServerOperationException (the same answer as a historian storage fault), so the
node answered BadHistoryOperationUnsupported and the read counted as a backend failure.

HistoryRead-Processed now checks the node's scalar DataType before calling any backend and
answers BadAggregateNotSupported (Part 11 6.5.4: "If the Aggregate is not valid for the Node
then the StatusCode shall be Bad_AggregateNotSupported"). Nothing is sent, nothing is counted.
- Shared, every backend: HistoryAggregateDataTypes (Core.Abstractions) applies Part 13
  5.4.2.3 Table 50: the numeric-only aggregates (Average, TimeAverage(2), Total(2), Min/Max,
  ActualTime, Range, Delta, DeltaBounds, Interpolative, stddev/variance) on a String or
  DateTime node are refused. Client-visible: the SQL backends used to answer every interval
  BadAggregateInvalidInputs.
- Per backend: IHistorianDataSource.SupportsAggregateFor(aggregate, TypeCode?) (default true;
  every production backend declares it, guard-tested). Gateway: false for native
  TimeAverage/Total on a Boolean node. SqlServer/QuestDb: true, since they serve Boolean as
  0/1 (stepped), so Boolean stays served there, as do the Gateway's raw-evaluated aggregates.

The residual case (a node declared numeric whose AVEVA tag is discrete) cannot be told apart
from a historian fault on the wire and still counts; that needs the gateway to surface
AVEVA's native error code as a deterministic status.

Tests: HistoryAggregateDataTypesTests, guard test, node-manager String/DateTime and per-type
refusals, Gateway capability matrix, DiscreteTagTimeWeightedWireTests (production Gateway
backend over an AVEVA-shaped fake: unsent, 0 health failures, positive controls). Each
mutation (no check, no shared rule, no backend hint, Gateway hint true, shared rule always
true, SQL inheriting the default) fails at least one. Live gate re-run read-only against
AVEVA 2023 R2 via a private HistorianGateway fa9cf31 container: SysPulse TimeAverage on the
wire = BadAggregateNotSupported, health failures unchanged; #801 gate 244 cases pass.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
2026-11-13 00:41:32 -05:00
parent 2e537dc6ca
commit ee39c169df
17 changed files with 506 additions and 6 deletions
+32 -2
View File
@@ -200,7 +200,7 @@ Processed read is `BadAggregateNotSupported` and no history aggregate functions
| Key | Type | Default | Description |
|---|---|---|---|
| `Gateway:EnableNativeTimeWeighted` | bool | `true` | Serve HistoryRead-Processed `TimeAverage` / `Total` through AVEVA's native retrieval modes. ⚠️ **On by default since #694 (client-visible: both are advertised and served again).** The gateway interval fix historiangw#27 (gateway server `9f711c1`+) is live-verified: forward and reverse per-interval values, clipped, sub-millisecond clipped and fractional-millisecond intervals against a step-held reference, and all-NULL intervals ⇒ `BadNoData` ([evidence](plans/evidence/2026-11-12-historian-694-700-746/aveva-gate.md)). `false` is the kill switch for a gateway older than the fix (it answers one Good-status whole-span value on the earliest interval): both are then `BadAggregateNotSupported` and **not advertised**. On a discrete (Boolean) AVEVA tag AVEVA refuses both modes and the node answers `BadHistoryOperationUnsupported` (counted as a backend failure; #897). |
| `Gateway:EnableNativeTimeWeighted` | bool | `true` | Serve HistoryRead-Processed `TimeAverage` / `Total` through AVEVA's native retrieval modes. ⚠️ **On by default since #694 (client-visible: both are advertised and served again).** The gateway interval fix historiangw#27 (gateway server `9f711c1`+) is live-verified: forward and reverse per-interval values, clipped, sub-millisecond clipped and fractional-millisecond intervals against a step-held reference, and all-NULL intervals ⇒ `BadNoData` ([evidence](plans/evidence/2026-11-12-historian-694-700-746/aveva-gate.md)). `false` is the kill switch for a gateway older than the fix (it answers one Good-status whole-span value on the earliest interval): both are then `BadAggregateNotSupported` and **not advertised**. On a **Boolean** node both are refused `BadAggregateNotSupported` up front (AVEVA refuses both modes on a discrete tag; #897 — before, the read went to the gateway, answered `BadHistoryOperationUnsupported` and counted as a backend failure). See [Aggregates by node DataType](#aggregates-by-node-datatype-897). |
| `Gateway:ProbeInterval` | TimeSpan | `00:00:30` | How often the read client calls the gateway's `ProbeAsync` for the historian health snapshot (`ProcessConnectionOpen` / `EventConnectionOpen`, shown on `/hosts`). The first probe runs at start-up, so a node that has served no HistoryRead yet — or only writes (continuous historization) — reports a healthy gateway as open; it used to read closed until some client happened to read. The latest observation (probe or read outcome) wins; probes never count as queries or failures. Zero/negative = no probe (read outcomes only, the previous behaviour). |
| `Gateway:EventDrainBudget` | int | `10000` | Since #746 a capped HistoryReadEvents read sends cap + 1 as the server-side `max_events` and the gateway completes the stream there, so this budget applies only to the uncapped fallback read (an ambiguous server-capped answer — foreign rows filtered out of a stream the server cut — or a gateway refusing the cap) and to a stream that runs past the `max_events` it was sent. It is the number of stream rows such a read consumes (and discards) past its page cap before it abandons the gateway stream; an **abandoned** stream cancels the RPC and the gateway evicts its pooled historian session (the next read reconnects; concurrent reads on that session fail). The usual small overshoot is drained to the stream's end; past the budget the stream is abandoned, so a page over a window far larger than the page does not stream the whole window. Rows of another source never count before the "more events" signal is seen, so an abandoned stream never answers a page as complete. `0` = abandon at the cap (the previous behaviour); negative counts as `0` (a `Validate()` warning). |
@@ -1508,7 +1508,37 @@ two things:
- **What HistoryRead-Processed refuses up front.** An aggregate outside the set is `BadAggregateNotSupported` for that node
without a backend call (the backend still throws `HistoryAggregateNotSupportedException` for one as defence in depth —
derived from the same table, never a second list). A source whose `SupportedAggregates` is `null` (out-of-tree, against
the contract) is treated as the interface default — **all** — by both consumers.
the contract) is treated as the interface default — **all** — by both consumers. An aggregate in the set that is not
valid for the node's DataType is refused the same way — see the next section.
### Aggregates by node DataType (#897)
⚠️ **Client-visible.** HistoryRead-Processed checks the node's scalar DataType before calling any backend, and an aggregate
that does not apply to it is **`BadAggregateNotSupported`** for that node — Part 11 §6.5.4: *"If the Aggregate is not valid
for the Node then the StatusCode shall be Bad_AggregateNotSupported."* Nothing is sent and no backend failure is counted.
Two layers, in order:
1. **Shared, every backend** (`HistoryAggregateDataTypes`, Core.Abstractions): Part 13 §5.4.2.3 Table 50 lists Average,
TimeAverage, TimeAverage2, Total, Total2, Minimum, Maximum, the ActualTime pair, Range, Delta, DeltaBounds, Interpolative and
the standard-deviation / variance family as **numeric only** ("dates, strings, arrays, etc. are not supported"). A
**String** or **DateTime** node asked for one of them is refused. Count, Start, End, DurationGood/Bad and PercentGood/Bad
apply to any type. (The SQL backends used to answer each interval of such a read `BadAggregateInvalidInputs`; that is
still the answer for a usable String/DateTime sample inside a numeric node's interval.)
2. **Per backend** (`IHistorianDataSource.SupportsAggregateFor(aggregate, valueType)`, declared by every production backend,
guard-tested): only the **Gateway** narrows it — native **TimeAverage / Total on a Boolean node** are refused, because AVEVA
refuses its `TimeWeightedAverage` / `Integral` modes on a discrete tag at `StartQuery`. The gateway reports that only as
`Internal` / `HistorianServerOperationException`, the same answer as a historian storage fault, so it cannot be told apart
on the wire; before #897 such a read (live: `SysPulse`) answered `BadHistoryOperationUnsupported` and counted as a backend
failure in the historian health snapshot.
Boolean is deliberately **not** refused by the shared rule although Table 50 lists it apart from numeric: the SQL backends
serve a Boolean node's numeric aggregates as 0/1 (stepped for the time-weighted ones — TimeAverage is the fraction of time
true), and the Gateway's raw-evaluated Average/Min/Max/… do the same. So on a Boolean node TimeAverage / Total are served by
`SqlServer` / `QuestDb` and refused by `Gateway`. An unknown node type (BaseDataType, an array) is not refused.
The residual case — a node declared numeric whose AVEVA tag is discrete — still reaches AVEVA and answers
`BadHistoryOperationUnsupported`, counted as a failure, until the gateway surfaces AVEVA's native error code as a
deterministic status.
⚠️ **`Server/ServerCapabilities/AggregateFunctions` (`i=2997`) is deliberately NOT trimmed** — it keeps the SDK's full 37 in
every configuration, including with no historian. Part 5 defines it as every aggregate the *server* supports, and the server
@@ -84,6 +84,13 @@ deterministic `GatewayAlarmWriterBatchTests`.
gateway answers `Internal`, and the node answers **`BadHistoryOperationUnsupported`**. That is a Bad status, never a value,
but the read counts as a backend failure in the health snapshot. Before `4c8e977e` the pair was refused up front with
`BadAggregateNotSupported`. Follow-up: [#897](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/897).
**Fixed by #897:** a Boolean node's TimeAverage / Total are now refused `BadAggregateNotSupported` up front on the
Gateway backend (nothing sent, no failure counted); see docs/Historian.md §Aggregates by node DataType.
Live re-run 2026-11-13 (followups gate + #801 gate, both passed, strict fixture not run) against a private
HistorianGateway `fa9cf31` container (auth disabled, live writes off, removed afterwards): the wire answers
`BadAggregateNotSupported` for `SysPulse` TimeAverage with the health failure count unchanged, while the gate's own
direct backend call still gets `HistorianServerOperationException`, so AVEVA still refuses the mode
([report](aveva-gate-followups-fix-897.json.gz), 65 cases, sha256 of the JSON `6b7d136029ea…`).
4. AVEVA integrates an integer tag (`SysTimeSec`) as a step (TimeAverage of 0..19 over 20 s = 9.5). Float tags are sloped
(see docs/Historian.md). The step reference above is therefore exact for `SysTimeSec`.