fix(historian): #691.4 walk reverse events reads back on forward-only backends (Gateway)
A reverse or end-only capped events read on a backend without SupportsReverseEvents (Gateway, Null) was refused BadHistoryOperationUnsupported (explicit reverse) or answered with the OLDEST N ascending (end-only, #675). It is now served newest-first by the same ReverseWalk as raw reads: page-sized ms-aligned segments, never answered from a read the source flagged truncated, ties sorted by EventId (the gateway orders by event time only - the fake shuffles them per call), one extra event walked to signal 'more', uncapped = one forward read reversed, and the width + proven-empty stretch carried in the events cursor. The oversized-cluster over-fetch keeps exactly the events at T in the same order. A tie cluster over the budget is BadTooManyOperations. Docs note the gateway's EventReadMaxRows must be unlimited (it truncates silently).
This commit is contained in:
+31
-19
@@ -677,8 +677,7 @@ backend:
|
||||
> ⚠️ **Client-visible (#686).** Since #669 an end-only capped raw read, and any explicit reverse raw read, on the Gateway
|
||||
> backend failed the node (`BadHistoryOperationUnsupported`) and was counted as a historian backend failure (the gateway
|
||||
> refused `start > end`). Before #669 an end-only read on Gateway returned the **oldest** N values ascending. Both now return
|
||||
> the Part 11 answer — the latest values at or before `EndTime`, newest first. **Events differ:** an end-only events read on
|
||||
> Gateway still keeps its forward meaning and an explicit reverse events read is refused (see below).
|
||||
> the Part 11 answer — the latest values at or before `EndTime`, newest first. **Events** follow since #691.4 (see below).
|
||||
>
|
||||
> ⚠️ **Also fixed with #686: uncapped raw reads on Gateway.** `NumValuesPerNode = 0` ("all values") used to be forwarded as
|
||||
> `max_values = 0`, which the client package and the gateway reject — every uncapped raw read (forward too) failed the node
|
||||
@@ -763,30 +762,43 @@ says is complete carries no continuation point. `NumValuesPerNode == 0` stays un
|
||||
that many more rows than the cap, so every page short of the last is a FULL `NumValuesPerNode` page (#675; resumed pages used
|
||||
to come back short by the skip).
|
||||
- A tie cluster larger than the cap is paged within by over-fetching the half-open window `[T, T + resolution)` up to
|
||||
`MaxTieClusterOverfetch`; beyond that bound the node answers `BadHistoryOperationUnsupported` (raise the bound).
|
||||
`MaxTieClusterOverfetch`; beyond that bound the node answers `BadTooManyOperations` (raise the bound).
|
||||
- **Tie exactness depends on the backend.** SqlServer and QuestDB order by `(EventTimeUtc, EventId)`, so the skip count is exact.
|
||||
The Gateway backend has no stable intra-timestamp order (and only a truncation flag), so ties at a page boundary are
|
||||
best-effort there.
|
||||
The Gateway backend has no stable intra-timestamp order (and only a truncation flag), so ties at a page boundary of a
|
||||
**forward** read are best-effort there; a walked-back **reverse** read (below) sorts them by `EventId` itself and is exact.
|
||||
- **Reverse event reads page newest-first (#675).** `StartTime > EndTime` reads the Part 11 reverse window `(EndTime, StartTime]`
|
||||
with events in DESCENDING `(EventTimeUtc, EventId)` order, so a cap keeps the **latest** N and a continuation point is issued
|
||||
when the source flags more. The direction is stored **in the cursor**: a reverse point resumes reverse (and a forward one
|
||||
forward) whatever the resume request's details say. Reverse tie clusters are over-fetched forward and sliced from the newest
|
||||
end, exactly as Raw does. Before #675 a capped reverse read silently returned the EARLIEST N, ascending, Good, with no
|
||||
continuation point.
|
||||
- **End-only events reads: backward on SqlServer/QuestDB, forward on Gateway (#675).** `StartTime` unset
|
||||
(`DateTime.MinValue`) with an `EndTime` and `NumValuesPerNode > 0` pages backward from `EndTime` over `(MinValue, EndTime]` —
|
||||
the same rule as Raw — **only on a backend that reports `SupportsReverseEvents`** (SqlServer, QuestDB). On Gateway (the
|
||||
default backend) and Null it keeps its pre-#675 forward `[MinValue, EndTime)` meaning, paged oldest-first, rather than
|
||||
becoming a refused reverse read. Start-only reads page forward to an open upper bound (`DateTime.MaxValue`, UTC) everywhere.
|
||||
Without a cap an end-only read is forward `[MinValue, EndTime)` everywhere.
|
||||
- **Backends.** The server checks `IHistorianDataSource.SupportsReverseEvents` and then calls
|
||||
`ReadEventsAsync(…, bool reverse, …)`. SqlServer and QuestDB serve reverse natively (`… > lo AND … <= hi ORDER BY time DESC,
|
||||
id DESC`) and report `true`. **Gateway and Null report `false`**: an explicit reverse read (`StartTime > EndTime`) is answered
|
||||
`BadHistoryOperationUnsupported` **without calling the backend** — the gateway streams ascending only and rejects `end < start`,
|
||||
and draining the whole window to keep its tail would be exactly the unbounded read the cap prevents. Because the refusal is a
|
||||
capability check, not a caught exception, a `NotSupportedException` thrown from inside a backend (a driver library) is logged
|
||||
as a backend fault like any other. Every implementation (and any future wrapper) must declare both members itself — the
|
||||
interface defaults (`false` / throw) would silently disable reverse reads; `HistorianDataSourceReverseEventsGuardTests` pins it.
|
||||
- **End-only events reads page backward on every backend (#675, #691.4).** `StartTime` unset (`DateTime.MinValue`) with an
|
||||
`EndTime` and `NumValuesPerNode > 0` pages backward from `EndTime` over `(MinValue, EndTime]` — the same rule as Raw.
|
||||
Start-only reads page forward to an open upper bound (`DateTime.MaxValue`, UTC) everywhere. Without a cap an end-only read
|
||||
is forward `[MinValue, EndTime)` everywhere.
|
||||
- **Backends.** The server checks `IHistorianDataSource.SupportsReverseEvents`. SqlServer and QuestDB serve reverse natively
|
||||
(`… > lo AND … <= hi ORDER BY time DESC, id DESC`, the `ReadEventsAsync(…, bool reverse, …)` overload) and report `true`.
|
||||
**Gateway and Null report `false`** — the gateway streams ascending only and rejects `end < start` — and are never sent a
|
||||
reverse read: since **#691.4** the server walks it back from bounded **forward** reads with the raw walk (#686/#689/#691:
|
||||
page-sized ms-aligned segments filtered to `(segLo, segHi]`, a read the source flagged truncated is never answered from,
|
||||
the now + 1 day / 1970 bounds, the remainder probe and the proven-empty stretch carried in the cursor). Event specifics:
|
||||
- **Ties** are sorted by `EventId` (ordinal) on every read — the gateway orders by `EventTimeUtc` only, in any order within a
|
||||
timestamp — so the page is DESCENDING `(EventTimeUtc, EventId)` and a resumed page's boundary skip lines up; the
|
||||
oversized-cluster over-fetch keeps exactly the events at `T` (the gateway's end is inclusive) in the same order.
|
||||
- **"More exists"** is found by walking for one event past the page (the walk has no source truncation flag to hand on).
|
||||
- **Uncapped** reverse reads are one forward read of the window, sorted and reversed.
|
||||
- A timestamp holding more than `MaxTieClusterOverfetch` events ⇒ **`BadTooManyOperations`** for that node.
|
||||
- ⚠️ **Gateway prerequisite:** the gateway's `RuntimeDb:EventReadMaxRows` must be `0` (its default, unlimited) or larger than
|
||||
the walk's biggest read (`2N + 18` rows, a discovery read 1,026, a tie-cluster read up to `2·MaxTieClusterOverfetch + 4`).
|
||||
That cap truncates **silently** (the stream just ends), so a segment it cut would look complete and drop its newest events —
|
||||
the same blindness a forward read has to it, but on the end a reverse read is asking for.
|
||||
|
||||
⚠️ **Client-visible (#691.4).** On Gateway (the default backend) an explicit reverse events read used to be refused
|
||||
`BadHistoryOperationUnsupported` (#675), and an end-only capped events read returned the **oldest** N events ascending. Both
|
||||
now return the latest events newest-first. Because the capability is checked, not a caught exception, a
|
||||
`NotSupportedException` thrown from inside a backend (a driver library) is still logged as a backend fault. Every
|
||||
implementation (and any future wrapper) must declare both members itself — the interface defaults (`false` / throw) would
|
||||
silently send a native-reverse backend through the walk; `HistorianDataSourceReverseEventsGuardTests` pins it.
|
||||
- **Gateway open upper bound.** The `DateTime.MaxValue` (UTC) bound of a start-only forward read is pinned offline through the
|
||||
published client's proto `Timestamp` conversion (captured by an in-process gRPC server) and the FILETIME encoding
|
||||
(`GatewayEventBoundsTests`). What a live AVEVA Historian does with a year-9999 bound is still unverified (needs the VPN).
|
||||
|
||||
Reference in New Issue
Block a user