fix(historian): #691.4 re-review — never narrow a walk segment below the 1 ms request granularity

A tie cluster's truncated read left the width at one tick; once the gap
above it was under two milliseconds nothing floored it, so segHi
stepped down one tick per read: 30 rows at one ms 0.3 s below hi with a
carried 1 s width took 10,006 identical reads (S8). (a) The top-anchored
width is floored at the granularity, Min kept; (b) a segment no wider
than the granularity is the unit that escalates to
MaxTieClusterOverfetch and then refuses - per millisecond, exact for the
ms-resolution Gateway (documented). Both are needed: the floor alone
loops on crowded reads with a small tie budget. S8 is a repo test
(4 shapes, reads <= 30, 0 repeated requests); random-data tests now
measure the refusal unit as rows in any 1 ms window. Reviewer harness,
5,000 seeds x default/MSONLY/CROWD: worst page 70 reads (default/MSONLY),
63 (CROWD) at M=65536, p99.9 27-37; no page over 500 reads at any tie
budget.
This commit is contained in:
Joseph Doherty
2026-10-12 17:50:39 -04:00
parent 574c3aa1b0
commit 9bb1363cd6
8 changed files with 109 additions and 20 deletions
+6 -2
View File
@@ -659,9 +659,13 @@ backend:
that row (`(row + d, top]`, `d` ×4 per step, following each truncated read) until a read comes back empty, then a
**bisection** of the bracket — never narrower than the 1 ms request granularity, which only re-sent the same request.
Measured: a dense tag that stopped two days ago ≈ **26** reads, a burst of just over 1,024 rows that ended 100 days ago
≈ **18** (was 46 bisecting the whole gap) — rather than re-reading the oldest rows of a burst over and over. Only **one timestamp holding more than `M` samples** is refused,
≈ **18** (was 46 bisecting the whole gap) — rather than re-reading the oldest rows of a burst over and over. Only **more than `M` samples within one millisecond** are refused,
**`BadTooManyOperations`** (a request limit, not a backend fault); a cluster of exactly `M`, or neighbouring
clusters each within `M`, is served.
clusters each within `M` and a millisecond or more apart, is served. ⚠️ The unit is the **millisecond**, not the
timestamp (#691.4 re-review): requests are rounded to the millisecond, so a narrower segment is the same request —
letting the width fall below it once stepped a page through one millisecond a tick at a time (~10,000 reads). Exact
for the Gateway, which stores milliseconds; a sub-millisecond backend walked this way would refuse more than `M`
samples inside one millisecond even when no single timestamp holds that many.
- **Proving the rest empty (#691).** Walking down through empty history to the floor used to cost one read per ×4
growth step — ~14–16 reads (~2.5 s live) on **every** last page of a deep page-through. Now, after two consecutive
empty segments, the walk asks **once per page** for the whole remainder down to the floor at the page budget: a