dohertj2/lmxopcua
1
0
Fork 0
Code Issues 5 Pull Requests Actions Packages Projects Releases Wiki Activity

Gateway historian: TimeAverage/Total on a discrete AVEVA tag answers BadHistoryOperationUnsupported and counts as a backend failure #897

New Issue
Closed
opened 2026-11-13 00:09:16 -05:00 by dohertj2 · 1 comment
dohertj2 commented 2026-11-13 00:09:16 -05:00
Owner
Copy Link
Copy Source

Found by the #694 live gate (2026-11-12, 4c8e977e, local HistorianGateway dffb5e2 → AVEVA 2023 R2 on wonder-sql-vd03). Evidence: docs/plans/evidence/2026-11-12-historian-694-700-746/aveva-gate.md, Finding 3.

Native TimeAverage / Total are on by default again since #694. On a discrete AVEVA tag (SysPulse, a Boolean node), AVEVA refuses TimeWeightedAverage / Integral at StartQuery (UnknownException 85). The gateway answers Internal (HistorianServerOperationException), and the Gateway backend reports that as a read failure. The node answers BadHistoryOperationUnsupported, which is a Bad status and never a wrong value. But:

  • the read counts as a backend failure in the historian health snapshot (one client's request taints /hosts health until the next good read or probe), and
  • before #694 the pair was refused up front with BadAggregateNotSupported (not sent to the gateway, not a failure).

Options:

  1. Refuse TimeAverage / Total for Boolean (and String) nodes on the Gateway backend before the gateway call. The node manager knows the DataType; the data source would need a per-type capability hint. This answers BadAggregateNotSupported and nothing is counted.
  2. Detect a discrete tag from GetTagMetadata: it reports no data_type, i.e. INT1, which a real Int1 analog tag would report too, so this is a heuristic.
  3. Leave it as it is and document it (done in docs/Historian.md).

Recommendation: option 1, since it is exact and needs no extra gateway call. The SQL backends are not affected.

Found by the #694 live gate (2026-11-12, `4c8e977e`, local HistorianGateway `dffb5e2` → AVEVA 2023 R2 on wonder-sql-vd03). Evidence: `docs/plans/evidence/2026-11-12-historian-694-700-746/aveva-gate.md`, Finding 3. Native TimeAverage / Total are on by default again since #694. On a **discrete** AVEVA tag (`SysPulse`, a Boolean node), AVEVA refuses `TimeWeightedAverage` / `Integral` at `StartQuery` (UnknownException 85). The gateway answers `Internal` (`HistorianServerOperationException`), and the Gateway backend reports that as a read failure. The node answers **`BadHistoryOperationUnsupported`**, which is a Bad status and never a wrong value. But: - the read counts as a backend failure in the historian health snapshot (one client's request taints `/hosts` health until the next good read or probe), and - before #694 the pair was refused up front with `BadAggregateNotSupported` (not sent to the gateway, not a failure). Options: 1. Refuse TimeAverage / Total for Boolean (and String) nodes on the Gateway backend before the gateway call. The node manager knows the DataType; the data source would need a per-type capability hint. This answers `BadAggregateNotSupported` and nothing is counted. 2. Detect a discrete tag from `GetTagMetadata`: it reports no `data_type`, i.e. INT1, which a real Int1 analog tag would report too, so this is a heuristic. 3. Leave it as it is and document it (done in docs/Historian.md). Recommendation: option 1, since it is exact and needs no extra gateway call. The SQL backends are not affected.
dohertj2 added the type:followuppriority:p3 labels 2026-11-13 00:09:16 -05:00
dohertj2 commented 2026-11-13 00:42:55 -05:00
Author
Owner
Copy Link
Copy Source

Fixed in ee39c169 on master.

What changed. HistoryRead-Processed now checks the node's scalar DataType before it calls any backend. An aggregate that is not valid for the node gets BadAggregateNotSupported. Nothing is sent to the gateway and no backend failure is counted.

Why this status. Part 11 §6.5.4 says: "If the Aggregate is not valid for the Node then the StatusCode shall be Bad_AggregateNotSupported." Part 13 §5.4.2.3 Table 50 lists TimeAverage, Total, Average, Min/Max and the rest of that group as numeric only. Bad_AggregateInvalidInputs is the status for a single interval whose data can't be aggregated, not for a whole node.

Where the check lives. It has two layers, both applied in OtOpcUaNodeManager.HistoryReadProcessed:

  1. Shared, every backend: HistoryAggregateDataTypes in Core.Abstractions applies Table 50. A numeric-only aggregate on a String or DateTime node is refused. This changes what clients see on SqlServer/QuestDb: they used to answer every interval BadAggregateInvalidInputs.
  2. Per backend: the new IHistorianDataSource.SupportsAggregateFor(aggregate, TypeCode?). Every production backend declares it, and a guard test checks that. Only Gateway restricts it: native TimeAverage / Total on a Boolean node are refused. SqlServer/QuestDb keep serving Boolean as stepped 0/1 (TimeAverage = fraction of time true), and so do the Gateway's raw-evaluated Average/Min/Max. Boolean is therefore not refused by the shared rule.

Health ("unsupported for this data" vs "backend down"). For a Boolean node the refusal now happens before anything is sent, so it never reaches health. The leftover case is a node declared numeric whose AVEVA tag is discrete. That read still reaches AVEVA and still counts as a failure. The gateway reports AVEVA's refusal only as Internal with the redacted detail HistorianServerOperationException, which is exactly what it sends for a historian storage-engine fault (the gateway's own drain path treats that as retryable). OtOpcUa can't tell the two apart on the wire, so it would need the gateway to send AVEVA's native error code (85) as a deterministic status (e.g. HistorianServerRejectionException → FailedPrecondition). That fix belongs in historiangw. Transport outages were already kept separate (BadNoCommunication vs BadHistoryOperationUnsupported).

Tests (each one fails under a mutation of the fix: check removed, shared rule removed, backend hint removed, Gateway hint forced true, shared rule always true, SQL backend inheriting the default):

  • HistoryAggregateDataTypesTests: the Table 50 sets.
  • Guard test Every_implementation_declares_its_per_type_aggregate_support_itself.
  • NodeManagerHistoryReadTests: String/DateTime refused without a backend call; backend per-type refusal; positive controls.
  • GatewayNativeTimeWeightedGateTests.The_native_pair_is_declared_unsupported_on_a_Boolean_node_only.
  • DiscreteTagTimeWeightedWireTests: the production Gateway backend over an AVEVA-shaped fake gateway, through a real SDK session. Boolean TimeAverage/Total give BadAggregateNotSupported, the gateway is never called, 0 health failures. Int32 TimeAverage and Boolean Average are still served.

Suites green: Core.Abstractions 774, Historian.Gateway 583, QuestDb 88, Sql.Common 300, SqlServer 45, Runtime 1037, OpcUaServer 1284. Release build: 0 warnings.

Live, read-only (2026-11-13): I ran a private HistorianGateway fa9cf31 container with auth disabled and Historian__LiveWritesEnabled=false, then removed it. It talked to AVEVA 2023 R2 on wonder-sql-vd03, and an in-process OtOpcUa server read through a real SDK session. Result: 700.existing.processed.TimeAverage.boolean (SysPulse) = BadAggregateNotSupported, with the health failure count unchanged across the wire read. The gate's own direct backend call still gets HistorianServerOperationException, which shows AVEVA still refuses the mode. Boolean Average was Good and both TimeAverage and Total are still advertised. The followups gate (65 cases) and the #801 gate (244 cases) passed; I didn't run the strict fixture. Report: docs/plans/evidence/2026-11-12-historian-694-700-746/aveva-gate-followups-fix-897.json.gz. No writes, no tags created, nothing ran on a wonder box.

Docs: docs/Historian.md §"Aggregates by node DataType (#897)", .claude/rules/historian.md, evidence Finding 3.

Fixed in `ee39c169` on master. **What changed.** HistoryRead-Processed now checks the node's scalar DataType before it calls any backend. An aggregate that is not valid for the node gets **`BadAggregateNotSupported`**. Nothing is sent to the gateway and no backend failure is counted. **Why this status.** Part 11 §6.5.4 says: *"If the Aggregate is not valid for the Node then the StatusCode shall be Bad_AggregateNotSupported."* Part 13 §5.4.2.3 Table 50 lists TimeAverage, Total, Average, Min/Max and the rest of that group as *numeric only*. `Bad_AggregateInvalidInputs` is the status for a single interval whose data can't be aggregated, not for a whole node. **Where the check lives.** It has two layers, both applied in `OtOpcUaNodeManager.HistoryReadProcessed`: 1. **Shared, every backend:** `HistoryAggregateDataTypes` in Core.Abstractions applies Table 50. A numeric-only aggregate on a **String or DateTime** node is refused. This changes what clients see on SqlServer/QuestDb: they used to answer every interval `BadAggregateInvalidInputs`. 2. **Per backend:** the new `IHistorianDataSource.SupportsAggregateFor(aggregate, TypeCode?)`. Every production backend declares it, and a guard test checks that. Only **Gateway** restricts it: native TimeAverage / Total on a **Boolean** node are refused. SqlServer/QuestDb keep serving Boolean as stepped 0/1 (TimeAverage = fraction of time true), and so do the Gateway's raw-evaluated Average/Min/Max. Boolean is therefore not refused by the shared rule. **Health ("unsupported for this data" vs "backend down").** For a Boolean node the refusal now happens before anything is sent, so it never reaches health. The leftover case is a node declared numeric whose AVEVA tag is discrete. That read still reaches AVEVA and still counts as a failure. The gateway reports AVEVA's refusal only as `Internal` with the redacted detail `HistorianServerOperationException`, which is exactly what it sends for a historian storage-engine fault (the gateway's own drain path treats that as retryable). OtOpcUa can't tell the two apart on the wire, so it would need the gateway to send AVEVA's native error code (85) as a deterministic status (e.g. `HistorianServerRejectionException` → `FailedPrecondition`). That fix belongs in historiangw. Transport outages were already kept separate (`BadNoCommunication` vs `BadHistoryOperationUnsupported`). **Tests** (each one fails under a mutation of the fix: check removed, shared rule removed, backend hint removed, Gateway hint forced true, shared rule always true, SQL backend inheriting the default): - `HistoryAggregateDataTypesTests`: the Table 50 sets. - Guard test `Every_implementation_declares_its_per_type_aggregate_support_itself`. - `NodeManagerHistoryReadTests`: String/DateTime refused without a backend call; backend per-type refusal; positive controls. - `GatewayNativeTimeWeightedGateTests.The_native_pair_is_declared_unsupported_on_a_Boolean_node_only`. - `DiscreteTagTimeWeightedWireTests`: the production Gateway backend over an AVEVA-shaped fake gateway, through a real SDK session. Boolean TimeAverage/Total give `BadAggregateNotSupported`, the gateway is never called, 0 health failures. Int32 TimeAverage and Boolean Average are still served. Suites green: Core.Abstractions 774, Historian.Gateway 583, QuestDb 88, Sql.Common 300, SqlServer 45, Runtime 1037, OpcUaServer 1284. Release build: 0 warnings. **Live, read-only** (2026-11-13): I ran a private HistorianGateway `fa9cf31` container with auth disabled and `Historian__LiveWritesEnabled=false`, then removed it. It talked to AVEVA 2023 R2 on wonder-sql-vd03, and an in-process OtOpcUa server read through a real SDK session. Result: `700.existing.processed.TimeAverage.boolean` (`SysPulse`) = **`BadAggregateNotSupported`**, with the health failure count unchanged across the wire read. The gate's own direct backend call still gets `HistorianServerOperationException`, which shows AVEVA still refuses the mode. Boolean Average was Good and both TimeAverage and Total are still advertised. The followups gate (65 cases) and the #801 gate (244 cases) passed; I didn't run the strict fixture. Report: `docs/plans/evidence/2026-11-12-historian-694-700-746/aveva-gate-followups-fix-897.json.gz`. No writes, no tags created, nothing ran on a wonder box. Docs: `docs/Historian.md` §"Aggregates by node DataType (#897)", `.claude/rules/historian.md`, evidence Finding 3.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
auto-managed
queue management
 blocked
Blocked by dependency or external hold
 cross-driver
queue management
 cross-repo
Needs coordinated change in another family repo (wire contract etc.)
 driver/abcip
queue management
 driver/ablegacy
queue management
 driver/focas
queue management
 driver/mqtt
MQTT / Sparkplug B broker-subscribe driver
 driver/opcuaclient
queue management
 driver/s7
queue management
 driver/twincat
queue management
 migrated
Migrated from a markdown backlog (2026-11 standardization)
 phase/1
queue management
 phase/2
queue management
 phase/3
queue management
 phase/4
queue management
 phase/5
queue management
 phase/6
queue management
 priority:p0
Must-have / blocking
 priority:p1
High priority
 priority:p2
Normal priority
 priority:p3
Low priority / nice to have
 queue/blocked
queue management
 queue/done
queue management
 queue/failed
queue management
 queue/in-progress
queue management
 queue/queued
queue management
 type:bug
Defect in shipped behaviour
 type:debt
Technical debt / hygiene
 type:deploy-gate
Rollout/deploy/migration step not yet done on an environment
 type:docs
Documentation work
 type:feature
Feature implementation
 type:followup
Follow-up deferred from a plan, PR or gate
 type:infra
Infrastructure and scaffolding
 type:review-finding
Unfixed finding from an architecture/code review
 type:test
Test work
No Label priority:p3 type:followup
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
dohertj2 (dohertj2)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dohertj2/lmxopcua#897
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?