fix(historian): #691.4 review — search the gap above a truncated read by gallop then bisect, never below 1 ms
Bisecting the gap above a truncated read halved below the 1 ms request granularity (requests are ms-rounded, so it re-sent identical requests) and bisected the WHOLE gap even when the burst ended right above the row it saw: a 1,026-row burst ending 100 days ago took 46 reads. After an empty top-anchored segment the walk now gallops up from the known row ((known + d, top], d x4, following each truncated read), then bisects the bracket, with halving floored at the granularity (the Min kept; a crowded gallop falls back to top-anchored halving). Burst: 18 reads, 0 repeated requests; dense tag stopped 2 days ago: 26 (gallop alone crawled: 94). Walk fuzzed at 4,000 seeds locally (400 in-tree). Docs: measured counts; CLAUDE.md gateway prerequisites list RuntimeDb:EventReadMaxRows.
This commit is contained in:
+9
-6
@@ -654,9 +654,12 @@ backend:
|
||||
- **Never answered from a truncated read.** Rows stream ascending, so a read is complete when it came back short or its
|
||||
last row is past the segment; otherwise it is discarded — re-read once with room for the rows before the segment
|
||||
that crowded it (a neighbouring tie cluster, at most twice the budget + 4 rows), or narrowed: to the density it
|
||||
showed, and never again below the newest row it saw until what lies above that row is read (the gap above it is
|
||||
bisected — a dense tag that went quiet days ago is found in about 20 reads rather than re-reading the same oldest
|
||||
rows of its burst over and over). Only **one timestamp holding more than `M` samples** is refused,
|
||||
showed, and never again below the newest row it saw until what lies above that row is read. The gap above it is
|
||||
searched: a segment at that density ending on the top (data running right up to it); if empty, a **gallop** up from
|
||||
that row (`(row + d, top]`, `d` ×4 per step, following each truncated read) until a read comes back empty, then a
|
||||
**bisection** of the bracket — never narrower than the 1 ms request granularity, which only re-sent the same request.
|
||||
Measured: a dense tag that stopped two days ago ≈ **26** reads, a burst of just over 1,024 rows that ended 100 days ago
|
||||
≈ **18** (was 46 bisecting the whole gap) — rather than re-reading the oldest rows of a burst over and over. Only **one timestamp holding more than `M` samples** is refused,
|
||||
**`BadTooManyOperations`** (a request limit, not a backend fault); a cluster of exactly `M`, or neighbouring
|
||||
clusters each within `M`, is served.
|
||||
- **Proving the rest empty (#691).** Walking down through empty history to the floor used to cost one read per ×4
|
||||
@@ -778,8 +781,8 @@ says is complete carries no continuation point. `NumValuesPerNode == 0` stays un
|
||||
is forward `[MinValue, EndTime)` everywhere.
|
||||
- **Backends.** The server checks `IHistorianDataSource.SupportsReverseEvents`. SqlServer and QuestDB serve reverse natively
|
||||
(`… > lo AND … <= hi ORDER BY time DESC, id DESC`, the `ReadEventsAsync(…, bool reverse, …)` overload) and report `true`.
|
||||
**Gateway and Null report `false`** — the gateway streams ascending only and rejects `end < start` — and are never sent a
|
||||
reverse read: since **#691.4** the server walks it back from bounded **forward** reads with the raw walk (#686/#689/#691:
|
||||
**Gateway reports `false`** — the gateway streams ascending only and rejects `end < start` — and is never sent a reverse
|
||||
read (Null reports `true` and answers empty either way, as for raw): since **#691.4** the server walks it back from bounded **forward** reads with the raw walk (#686/#689/#691:
|
||||
page-sized ms-aligned segments filtered to `(segLo, segHi]`, a read the source flagged truncated is never answered from,
|
||||
the now + 1 day / 1970 bounds, the remainder probe and the proven-empty stretch carried in the cursor). Event specifics:
|
||||
- **Ties** are sorted by `EventId` (ordinal) on every read — the gateway orders by `EventTimeUtc` only, in any order within a
|
||||
@@ -796,7 +799,7 @@ says is complete carries no continuation point. `NumValuesPerNode == 0` stays un
|
||||
That cap truncates **silently** (the stream just ends), so a segment it cut would look complete and drop its newest events —
|
||||
the same blindness a forward read has to it, but on the end a reverse read is asking for.
|
||||
|
||||
⚠️ **Client-visible (#691.4).** On Gateway (the default backend) an explicit reverse events read used to be refused
|
||||
⚠️ **Client-visible (#691.4).** On Gateway (the default backend) and Null an explicit reverse events read used to be refused
|
||||
`BadHistoryOperationUnsupported` (#675), and an end-only capped events read returned the **oldest** N events ascending. Both
|
||||
now return the latest events newest-first. Because the capability is checked, not a caught exception, a
|
||||
`NotSupportedException` thrown from inside a backend (a driver library) is still logged as a backend fault. Every
|
||||
|
||||
Reference in New Issue
Block a user