Merge feat/675-reverse-event-paging: reverse event HistoryRead paging, end-only backward, Gateway MaxValue pin (#675)

This commit is contained in:
Joseph Doherty
2026-10-11 11:23:51 -04:00
20 changed files with 752 additions and 60 deletions
+26 -5
View File
@@ -632,13 +632,31 @@ says is complete carries no continuation point. `NumValuesPerNode == 0` stays un
- The cursor is `(source, next start, end, boundary skip, cap)` in the same session-bound store as Raw (flagged as an events
cursor — a Raw point presented to an events read, or the reverse, is `BadContinuationPointInvalid`). The next page reads from
the last event's `EventTimeUtc` **inclusive** and drops the events at that timestamp already emitted.
the last event's `EventTimeUtc` **inclusive** and drops the events at that timestamp already emitted — and asks the source for
that many more rows than the cap, so every page short of the last is a FULL `NumValuesPerNode` page (#675; resumed pages used
to come back short by the skip).
- A tie cluster larger than the cap is paged within by over-fetching the half-open window `[T, T + resolution)` up to
`MaxTieClusterOverfetch`; beyond that bound the node answers `BadHistoryOperationUnsupported` (raise the bound).
- **Tie exactness depends on the backend.** SqlServer and QuestDB order by `(EventTimeUtc, EventId)`, so the skip count is exact.
The Gateway backend has no stable intra-timestamp order (and only a truncation flag), so ties at a page boundary are
best-effort there.
- Reverse event reads (`start > end`) are not paged server-side; no continuation point is issued for them.
- **Reverse event reads page newest-first (#675).** `StartTime > EndTime` reads the Part 11 reverse window `(EndTime, StartTime]`
with events in DESCENDING `(EventTimeUtc, EventId)` order, so a cap keeps the **latest** N and a continuation point is issued
when the source flags more. The direction is stored **in the cursor**: a reverse point resumes reverse (and a forward one
forward) whatever the resume request's details say. Reverse tie clusters are over-fetched forward and sliced from the newest
end, exactly as Raw does. Before #675 a capped reverse read silently returned the EARLIEST N, ascending, Good, with no
continuation point.
- **End-only events reads read backward (#675).** `StartTime` unset (`DateTime.MinValue`) with an `EndTime` and
`NumValuesPerNode > 0` pages backward from `EndTime` over `(MinValue, EndTime]` — the same rule as Raw. Start-only reads still
page forward to an open upper bound (`DateTime.MaxValue`, UTC). Without a cap an end-only read keeps its forward
`[MinValue, EndTime)` meaning.
- **Backends.** The server calls `IHistorianDataSource.ReadEventsAsync(…, bool reverse, …)`. SqlServer and QuestDB serve reverse
natively (`… > lo AND … <= hi ORDER BY time DESC, id DESC`). **Gateway and Null refuse a reverse read** (the interface default
throws `NotSupportedException`) and the node answers `BadHistoryOperationUnsupported` — the gateway streams ascending only and
rejects `end < start`, and draining the whole window to keep its tail would be exactly the unbounded read the cap prevents.
- **Gateway open upper bound.** The `DateTime.MaxValue` (UTC) bound of a start-only forward read is pinned offline through the
published client's proto `Timestamp` conversion (captured by an in-process gRPC server) and the FILETIME encoding
(`GatewayEventBoundsTests`). What a live AVEVA Historian does with a year-9999 bound is still unverified (needs the VPN).
- No session on the request (in-process callers) degrades to single-shot, as for Raw.
### Modified-value history (Part 11)
@@ -861,8 +879,10 @@ result never has both.
### Events
`[start, end)` by `EventTimeUtc`; **a reversed window (`end < start`) is swapped and returned ascending**. A null `sourceName` means all
sources, otherwise `SourceName = @source`. Ordered `(EventTimeUtc, EventId)` ascending. A bounded read fetches `maxEvents + 1` rows and
`[start, end)` by `EventTimeUtc`; through the five-argument read **a reversed window (`end < start`) is swapped and returned
ascending**. The direction-aware overload with `reverse: true` (what the node manager calls for a reverse read, #675) reads
`(lo, hi]` ordered `(EventTimeUtc, EventId)` **descending** — the clustered index scanned backwards. A null `sourceName` means all
sources, otherwise `SourceName = @source`. Forward is ordered `(EventTimeUtc, EventId)` ascending. A bounded read fetches `maxEvents + 1` rows and
returns a non-null continuation point when the extra row exists; the node manager turns that flag into an OPC UA continuation
point (see [Continuation-point paging (Events)](#continuation-point-paging-events-669)).
@@ -1205,7 +1225,8 @@ Every query filters `NOT deleted`; every value goes through `StoredSample`, so t
from the last timestamp ± `TimestampResolutionTicks`.
- **AtTime:** previous-or-equal and next neighbour of each time by `LIMIT 1` seeks, `UNION ALL`-ed, **50 times per statement** (QuestDB 10.0.1 refuses
more than 126 branches). `AtTimeEvaluator` decides each value, as on `SqlServer`.
- **Events:** `[start, end)` by `ts`, ordered `(ts, event_id)`, `maxEvents + 1` fetched to produce the continuation point.
- **Events:** `[start, end)` by `ts`, ordered `(ts, event_id)`, `maxEvents + 1` fetched to produce the continuation point; a
reverse read (#675) is `(lo, hi]` ordered `(ts, event_id)` descending.
- **Processed:** one query per node; bucket index `(cast(ts AS LONG) * 10 - startUnixTicks) / intervalTicks` (never `SAMPLE BY`, whose alignment
differs), so bucket edges are identical to `SqlServer` (start-inclusive, end-exclusive, partial last bucket, interval 0 ⇒ one bucket, 100,000-bucket cap).
Min/max times use `row_number()` windows with the earliest-timestamp tie-break.