fix(historian): #689 size reverse-walk segments from the page, not the tie budget
ReverseRawReader aimed every segment at half of MaxTieClusterOverfetch (~32k rows by default) and carried that width page to page, so a paged 'latest N' read on a dense Gateway tag fetched ~32,770 rows per page (2-5 s live) to return N. The walk now lives in a generic ReverseWalk core: segments are aimed at ~N+8 rows (page budget 2N+16), a discovery segment (first without a hint / after an empty one) may read 1,024, and the tie budget is only the ceiling a single-timestamp cluster escalates to. A truncated read never re-enters the stretch below its newest row until the gap above it is bisected. Tests: per-page row counts on a dense tag (node manager + walk), randomized exactness fuzz (walk and node-manager paging), dense-tag-below-a-long-gap read bound.
This commit is contained in:
+20
-9
@@ -639,15 +639,26 @@ backend:
|
||||
`[⌊segLo⌋, ⌈segHi + 1 tick⌉]`, both rounded **outward to the millisecond**, and filtered on the samples' own
|
||||
timestamps — exact for an inclusive or exclusive forward end, and the row AVEVA fabricates at a query's (possibly
|
||||
rounded-up) start can never land inside the segment. The first segment is **1 minute** (or the width the previous
|
||||
page ended on — it rides the continuation point); each next width is aimed at about half the row budget from the
|
||||
density just observed, growing at most ×4. So "the last N values" of a live tag is normally one read, a dense tag
|
||||
costs about one read per page, and sparse history is found in a few dozen tiny ones.
|
||||
- **Never answered from a truncated read.** The row budget is `M = max(1, MaxTieClusterOverfetch)`; a segment read asks
|
||||
for `M + 2` rows (room for the fabricated row and one to see past the segment). Rows stream ascending, so a read is
|
||||
complete when it came back short or its last row is past the segment; otherwise it is discarded — re-read once with
|
||||
room for the rows before the segment that crowded it (a neighbouring tie cluster, at most `2M + 4` rows), or narrowed
|
||||
to the density it showed. Only **one timestamp holding more than `M` samples** is refused, **`BadTooManyOperations`**
|
||||
(a request limit, not a backend fault); a cluster of exactly `M`, or neighbouring clusters each within `M`, is served.
|
||||
page ended on — it rides the continuation point); each next width is aimed from the density just observed at about
|
||||
half the **page budget** `2N + 16` (≈ N + 8 rows), growing at most ×4. So "the last N values" of a live tag is
|
||||
normally one read of about N rows, a dense tag costs about one N-sized read per page, and sparse history is found in
|
||||
a few dozen tiny ones.
|
||||
- **Read sizes (#689).** A segment read asks for its budget + 2 rows (room for the fabricated row and one to see past
|
||||
the segment). The budget is the page budget `2N + 16`, except a **discovery** segment — the first of a page with no
|
||||
carried width, or one after an empty segment, whose ×4 growth may overshoot into dense data — which may read up to
|
||||
**1,024** rows. The tie budget `M = max(1, MaxTieClusterOverfetch)` is only a **ceiling**: a single timestamp holding
|
||||
more rows than the segment budget escalates to `M`.
|
||||
⚠️ Before #689 every segment was aimed at half of `M` (≈ 32k rows by default) and that width was carried page to
|
||||
page, so on a 1 Hz tag from about page 5 every page of a "latest N" read fetched ~32,770 rows and took 2–5 s to
|
||||
return N.
|
||||
- **Never answered from a truncated read.** Rows stream ascending, so a read is complete when it came back short or its
|
||||
last row is past the segment; otherwise it is discarded — re-read once with room for the rows before the segment
|
||||
that crowded it (a neighbouring tie cluster, at most twice the budget + 4 rows), or narrowed: to the density it
|
||||
showed, and never again below the newest row it saw until what lies above that row is read (the gap above it is
|
||||
bisected — a dense tag that went quiet days ago is found in about 20 reads rather than re-reading the same oldest
|
||||
rows of its burst over and over). Only **one timestamp holding more than `M` samples** is refused,
|
||||
**`BadTooManyOperations`** (a request limit, not a backend fault); a cluster of exactly `M`, or neighbouring
|
||||
clusters each within `M`, is served.
|
||||
- **Bounds.** The walk starts no later than **now + 1 day** (a `StartTime = DateTime.MaxValue` read does not send
|
||||
year-9999 query starts) and never reads before the **Unix epoch** (AVEVA holds nothing earlier): a walked-back read
|
||||
returns only samples strictly after 1970-01-01T00:00:00Z and no later than now + 1 day.
|
||||
|
||||
Reference in New Issue
Block a user