fix(historian): fail fast for missing tags and reduce gateway read cost
v2-ci / build (pull_request) Successful in 5m6s
v2-ci / unit-tests (pull_request) Failing after 29m54s

This commit is contained in:
Joseph Doherty
2026-10-13 04:31:52 -04:00
parent 97668d26d3
commit 5897764024
15 changed files with 439 additions and 18 deletions
+16 -2
View File
@@ -765,8 +765,11 @@ data.
refusal of the remembered one. A **raised** ceiling (or a switch to streaming) is noticed by the only reads it matters to: when a
read bounded by the remembered ceiling comes back full without proving completeness, it is re-probed **once** at its full budget
— served ⇒ the ceiling is forgotten; refused naming a larger ceiling ⇒ that one is learned and the read re-issued at it; refused
naming the same ceiling ⇒ the bounded result stands. No timer: a TTL would only add refusals for reads that already fit; the cost
of the rule is one refused re-probe per genuinely over-ceiling read (which fails either way). So an uncapped read (`NumValuesPerNode = 0`, which used to be refused outright on a strict gateway, even for a 10-row
naming the same ceiling ⇒ the bounded result stands. Raw and AtTime re-probe immediately. Per-sample aggregates (#696) reuse
a refusal for **30 seconds** before re-probing a full stream; within that period an oversized window still fails honestly with
`BadTooManyOperations`. A lowered ceiling is still re-learned immediately; a raise becomes visible on the next incomplete
aggregate read after the cooldown. So an uncapped read (`NumValuesPerNode = 0`, which used to be refused outright on a strict
gateway, even for a 10-row
window) and a page size within 16 of the ceiling are now served when the window fits. A ceiling-bounded read is answered only
when it is **provably complete** — the stream came back short of the ceiling, or reached `EndTime` (or, capped, filled the page);
a stream that delivered the whole ceiling short of `EndTime` may be truncated and fails **`BadTooManyOperations`** (not a backend
@@ -828,6 +831,17 @@ data.
refusing; doubling the look-back allowance does not skip that final usable budget. Reverse paging retries a crowded segment
with a smaller in-window budget derived from the learned ceiling and the observed pre-window rows.
**Missing tags (#653).** Gateway Raw, Processed and AtTime first check the tag catalog. An empty exact-name catalog result or an explicit
`NotFound` returns no data without a history RPC: Raw is empty; AtTime and Processed return null/`BadNoData` at the
requested times or bucket timestamps. Unsupported aggregate validation still runs first. A bounded per-data-source cache
holds at most 4,096 tag names, with **5 seconds** for absence and **60 seconds** for presence, so newly provisioned tags become
visible after the short negative expiry. Catalog faults and cancellation never become missing-tag cache entries. The adapter drains the filtered catalog stream and checks exact names, avoiding older gateways that misparse a missing `GetTagMetadata` response as `Internal`. Empty
AtTime/Processed requests need no check. Events and modified-history capability checks are unaffected.
**Stopped-tag reverse discovery (#696).** On a strict Gateway, a truncated remainder probe retains its oldest-row evidence
and density. Gap discovery uses the learned ceiling's available budget; ordinary page reads keep their page-sized budget.
Only complete segments contribute to a page, preserving newest-first order, bounds, ties and continuation hints.
### Gateway AtTime is evaluated from raw reads (#693)
> ⚠️ **Client-visible (#693).** Gateway AtTime used to pass AVEVA's own interpolated answer (the gateway's `ReadAtTime`) through,