dohertj2/lmxopcua
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity

Gateway historian: remaining Part 11/13 gaps after #685–#688 (ReturnBounds, reverse native aggregates, strict-ceiling uncapped reads, reverse events) #691

New Issue
Closed
opened 2026-10-12 16:03:37 -04:00 by dohertj2 · 1 comment
dohertj2 commented 2026-10-12 16:03:37 -04:00
Owner
Copy Link
Copy Source

Collected from the #685–#688 lanes, reviews and live gate (2026-10-12, master 1f5d3100). Low priority; tracked so they don't get lost.

  1. ReturnBounds is ignored on every backend. Part 11 bounding values are not implemented in the node manager or IHistorianDataSource. AVEVA's synthetic boundary row cannot serve as a bound (it is stamped at the query start, not at the prior sample's time).
  2. Reverse native TimeAverage/Total on Gateway are still refused by the gateway; the #677 reverse anchoring was not added for the native modes.
  3. Strict gateway ceiling: with AllowStreamingBeyondCeiling=false, every Gateway raw read with NumValuesPerNode = 0 is refused (BadTooManyOperations), even a 10-row window, because unlimited is sent as int.MaxValue. Consider sending the ceiling and treating a full result as truncated. Also a gateway configured with a ceiling below 1,000,000 is not handled specially.
  4. Reverse events on Gateway: end-only events reads stay forward and explicit reverse events reads are refused (#675), unlike raw reads which now walk back (#686). The same walk could serve events.
  5. Deep reverse page-through to the epoch: the last pages of a deep reverse page-through each walk down to 1970 (about 14–16 small reads, ~2.5 s live). Bounded, but could carry an "exhausted below T" hint in the continuation point.
  6. #685 end to end still needs historiangw#27 (aggregate interval sent as ticks). Once it lands, re-run the native TimeAverage/Total leg, including the all-NULL interval (observed today as quality 1, value 0 ⇒ BadNoData).
Collected from the #685–#688 lanes, reviews and live gate (2026-10-12, master `1f5d3100`). Low priority; tracked so they don't get lost. 1. **ReturnBounds is ignored on every backend.** Part 11 bounding values are not implemented in the node manager or `IHistorianDataSource`. AVEVA's synthetic boundary row cannot serve as a bound (it is stamped at the query start, not at the prior sample's time). 2. **Reverse native TimeAverage/Total on Gateway** are still refused by the gateway; the #677 reverse anchoring was not added for the native modes. 3. **Strict gateway ceiling:** with `AllowStreamingBeyondCeiling=false`, every Gateway raw read with `NumValuesPerNode = 0` is refused (`BadTooManyOperations`), even a 10-row window, because unlimited is sent as `int.MaxValue`. Consider sending the ceiling and treating a full result as truncated. Also a gateway configured with a ceiling below 1,000,000 is not handled specially. 4. **Reverse events on Gateway:** end-only events reads stay forward and explicit reverse events reads are refused (#675), unlike raw reads which now walk back (#686). The same walk could serve events. 5. **Deep reverse page-through to the epoch:** the last pages of a deep reverse page-through each walk down to 1970 (about 14–16 small reads, ~2.5 s live). Bounded, but could carry an "exhausted below T" hint in the continuation point. 6. **#685 end to end still needs historiangw#27** (aggregate interval sent as ticks). Once it lands, re-run the native TimeAverage/Total leg, including the all-NULL interval (observed today as quality 1, value 0 ⇒ BadNoData).
dohertj2 commented 2026-10-13 00:11:26 -04:00
Author
Owner
Copy Link
Copy Source

All items addressed on master 15ccd9b7 except item 6, which is blocked on historiangw#27 and carried to #694.

  1. ReturnBounds (merge 942ca7bc): Part 11 bounding values on every backend from one authority (HistoryBounds), checked against every row of Table 1; Gateway start bound = the real prior sample via the walk-back (never AVEVA's synthetic row), end-bound lookup clamped to now + 1 day; no data + no bounds ⇒ GoodNoData; modified read + bounds ⇒ Bad_InvalidArgument; kill switch ServerHistorian:HonorReturnBounds (default true). ⚠️ Breaking for SDK clients: new ReadRawModifiedDetails() defaults ReturnBounds AND IsReadModified to true (documented).
  2. Reverse native TimeAverage/Total (merge f0d064a6): served via forward requests on the SQL backends' grid in both directions; clipped interval is its own exact-span read marked Partial; forward reads no longer average past EndTime; >100k intervals refused up front.
  3. Strict ceiling (f0d064a6): the gateway's ceiling is learned from its refusal and remembered per data source; a read past it is BadTooManyOperations, never truncated.
  4. Reverse events on Gateway (88141d6e): served by the walk (EventId tie order); uncapped event reads (forward too) no longer throw.
  5. Deep page-through: the proven-empty stretch rides the continuation point; last pages ≤ ~1–3 reads.

Live AVEVA gate PASS for 1–5 (requests forward-only; one ceiling refusal across repeated reads; reverse events newest-first with a 3-way tie split correctly across pages; last deep pages ≤ 3 reads/<0.5 s). Values of item 2 remain wrong end to end until historiangw#27. Follow-ups: #692, #693, #694.

All items addressed on master `15ccd9b7` except item 6, which is blocked on historiangw#27 and carried to #694. 1. **ReturnBounds** (merge `942ca7bc`): Part 11 bounding values on every backend from one authority (`HistoryBounds`), checked against every row of Table 1; Gateway start bound = the real prior sample via the walk-back (never AVEVA's synthetic row), end-bound lookup clamped to now + 1 day; no data + no bounds ⇒ `GoodNoData`; modified read + bounds ⇒ `Bad_InvalidArgument`; kill switch `ServerHistorian:HonorReturnBounds` (default true). ⚠️ Breaking for SDK clients: `new ReadRawModifiedDetails()` defaults `ReturnBounds` AND `IsReadModified` to true (documented). 2. **Reverse native TimeAverage/Total** (merge `f0d064a6`): served via forward requests on the SQL backends' grid in both directions; clipped interval is its own exact-span read marked Partial; forward reads no longer average past EndTime; >100k intervals refused up front. 3. **Strict ceiling** (`f0d064a6`): the gateway's ceiling is learned from its refusal and remembered per data source; a read past it is `BadTooManyOperations`, never truncated. 4. **Reverse events on Gateway** (`88141d6e`): served by the walk (EventId tie order); uncapped event reads (forward too) no longer throw. 5. **Deep page-through**: the proven-empty stretch rides the continuation point; last pages ≤ ~1–3 reads. **Live AVEVA gate PASS** for 1–5 (requests forward-only; one ceiling refusal across repeated reads; reverse events newest-first with a 3-way tie split correctly across pages; last deep pages ≤ 3 reads/<0.5 s). Values of item 2 remain wrong end to end until historiangw#27. Follow-ups: #692, #693, #694.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
auto-managed
queue management
 cross-driver
queue management
 driver/abcip
queue management
 driver/ablegacy
queue management
 driver/focas
queue management
 driver/mqtt
MQTT / Sparkplug B broker-subscribe driver
 driver/opcuaclient
queue management
 driver/s7
queue management
 driver/twincat
queue management
 phase/1
queue management
 phase/2
queue management
 phase/3
queue management
 phase/4
queue management
 phase/5
queue management
 phase/6
queue management
 queue/blocked
queue management
 queue/done
queue management
 queue/failed
queue management
 queue/in-progress
queue management
 queue/queued
queue management
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
dohertj2 (dohertj2)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dohertj2/lmxopcua#691
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?