fix(historian): #691.3 review — remember a strict gateway's ceiling per data source; re-probe once when a bounded read comes back full

This commit is contained in:
Joseph Doherty
2026-10-12 16:50:32 -04:00
parent ab5030174a
commit 7b86059daf
7 changed files with 190 additions and 38 deletions
+8 -2
View File
@@ -705,8 +705,14 @@ data.
every page size from 999,985 to 1,000,000 come back short. A default gateway serves any budget.
- ⚠️ **Strict gateway (#691.3, client-visible).** A gateway with `AllowStreamingBeyondCeiling=false` refuses a budget over its
`MaxValuesCeiling` (before streaming) and names the ceiling in the refusal; the gateway has no RPC that reports it. The read is
then **re-issued once with the ceiling as the budget** — nothing is cached, so a reconfigured gateway is honoured on the next
read. So an uncapped read (`NumValuesPerNode = 0`, which used to be refused outright on a strict gateway, even for a 10-row
then **re-issued once with the ceiling as the budget**, and the ceiling is **remembered per data source** (`GatewayRowCeiling`,
shared by raw and per-sample reads): every refused attempt counts as an error in the gateway's own outcome metrics, so after the
first refusal later reads ask for at most the learned ceiling and are not refused. A **lowered** ceiling is re-learned from the
refusal of the remembered one. A **raised** ceiling (or a switch to streaming) is noticed by the only reads it matters to: when a
read bounded by the remembered ceiling comes back full without proving completeness, it is re-probed **once** at its full budget
— served ⇒ the ceiling is forgotten; refused naming a larger ceiling ⇒ that one is learned and the read re-issued at it; refused
naming the same ceiling ⇒ the bounded result stands. No timer: a TTL would only add refusals for reads that already fit; the cost
of the rule is one refused re-probe per genuinely over-ceiling read (which fails either way). So an uncapped read (`NumValuesPerNode = 0`, which used to be refused outright on a strict gateway, even for a 10-row
window) and a page size within 16 of the ceiling are now served when the window fits. A ceiling-bounded read is answered only
when it is **provably complete** — the stream came back short of the ceiling, or reached `EndTime` (or, capped, filled the page);
a stream that delivered the whole ceiling short of `EndTime` may be truncated and fails **`BadTooManyOperations`** (not a backend