fix(historian): #692 review — a drained tie cluster never resumes onto the exclusive EndTime

F1 (pre-existing): SliceTieCluster continued a forward read while next <= endUtc, but the
forward window is end-exclusive; an oversized cluster one step below EndTime resumed AT
EndTime, the resume became a start == end point read and returned the EndTime samples as
data (microsecond: EndTime 8.1 us truncated to 8.0). Continue only while next is strictly
before EndTime truncated to the backend resolution.

Tests: A_tie_cluster_just_below_the_end_time_never_resumes_onto_the_end_time (native +
microsecond; both fail on the old rule) and
A_cursor_inside_a_tie_cluster_over_fetches_without_a_normal_read_first (pins the
WithinTieCluster shortcut by per-page read count; fails with the shortcut disabled).
Docs: the oversized-cluster exception to 'every page full / no lookahead => last' (F2),
and the N + skip + 1 strict-ceiling edge note referencing #694 (F4).
This commit is contained in:
Joseph Doherty
2026-10-13 01:02:12 -04:00
parent 3d7eea6280
commit e04e1bf127
3 changed files with 83 additions and 11 deletions
+13 -4
View File
@@ -587,14 +587,20 @@ paging time-based:
returns exactly `NumValuesPerNode` values, and the server stores a resume cursor and returns an opaque continuation
point (16 bytes). The client hands it back to fetch the next page.
- When it does not, the page is the **last** — no continuation point, even when it is exactly full. (#692: a total that
is an exact multiple of the cap used to end with an extra empty `GoodNoData` page.)
is an exact multiple of the cap used to end with an extra empty `GoodNoData` page.) The exception is an oversized tie
cluster (below): its pages are sliced from the over-fetched cluster, its last slice is `min(N, remaining)` values and
still carries a continuation point when the window goes on — so stepping past a cluster at the very end of the data can
yield one empty final page.
- `NumValuesPerNode == 0` ("all values, no limit") is never paged; the whole window returns in one
shot.
- The resume cursor is **tie-safe**: the next page resumes from the last returned sample's
SourceTimestamp *inclusive* and drops the boundary samples already emitted, so samples sharing
the boundary timestamp are neither duplicated nor skipped. Because it re-reads them, a resumed page asks for
`NumValuesPerNode + skip + 1` rows, so it still carries a full page.
- ⚠️ **#692 (client-visible): every page is full.** A resumed page used to ask for only `NumValuesPerNode` rows, so after
`NumValuesPerNode + skip + 1` rows, so it still carries a full page. On a strict Gateway
(`AllowStreamingBeyondCeiling=false`) that moves the row-ceiling edge by `skip + 1`: a `NumValuesPerNode` just under the
learned ceiling may now cost one learning refusal (#691.3 re-asks at the ceiling). A page clamp to the learned ceiling is
being added separately (#694).
- ⚠️ **#692 (client-visible): every page is full** (outside an oversized tie cluster — see above). A resumed page used to ask for only `NumValuesPerNode` rows, so after
the trim it netted `N − skip` values — and at `NumValuesPerNode = 1` **nothing**: the lone re-read boundary sample was
trimmed, the empty "full" page was taken for a stalled tie cluster, and a `MaxTieClusterOverfetch + 1` (65,537-row)
over-fetch stepped one tick to return an **empty page with a continuation point between every value** (live, AVEVA:
@@ -610,7 +616,10 @@ paging time-based:
> `ServerHistorian:MaxTieClusterOverfetch` (default **65 536**), then serves the cluster
> `NumValuesPerNode` samples at a time across successive pages, advancing the cursor one tick past
> the timestamp once the cluster is fully drained. A cursor left inside the cluster is marked so (`WithinTieCluster`), and
> its next page over-fetches the cluster directly instead of first re-reading rows it knows are ties.
> its next page over-fetches the cluster directly instead of first re-reading rows it knows are ties. A drained cluster
> resumes only while the next step is strictly before `EndTime` (truncated to the backend's resolution) — the forward
> window is end-exclusive; resuming AT `EndTime` used to make the next read a point read that returned the samples at
> `EndTime` as data (#692 review).
>
> **Reverse reads** (`start > end`, newest first) page the same way in the opposite direction: a drained
> cluster steps the cursor one tick **down**, the read ends when that would reach the (excluded) `end`, and