fix(historian): respect gateway ceilings and history value contracts

docs/Historian.md

@@ -573,7 +573,8 @@ node without `isHistorized` has no HistoryRead access bit and answers
 | Historized node, historian configured, time range empty | `GoodNoData` |
 | Historized node, historian NOT configured (`Enabled=false` / Null source) | `GoodNoData` (empty) for Raw / AtTime; Processed ⇒ `BadAggregateNotSupported` (#682) |
 | Non-historized node | `BadHistoryOperationUnsupported` |
-| Backend timeout or exception | `BadHistoryOperationUnsupported` per node; other nodes in the same batch are unaffected |
+| Backend transport outage (#697) | `BadNoCommunication` per node across Raw, Modified, Processed, AtTime and Events; cancelled/deadline-limited server reads retain `BadTimeout` |
+| Other backend exception / unsupported operation | `BadHistoryOperationUnsupported` per node; other nodes in the same batch are unaffected |
 | `Backend=SqlServer`, `AutoCreateSchema=false`, schema missing/behind | every read/write fails with a clear error naming the missing schema version (health `LastError`); reads answer `BadHistoryOperationUnsupported` per node; no host crash |
 
 A historized node with no historian configured never returns an error status — it returns
@@ -823,6 +824,10 @@ data.
   it could only ever be a Part 11 *interpolated* bound. The start bound on the Gateway is the real prior sample, found by the
   #686 walk-back.
 
+**Warm crowding ceiling (#695).** A raw crowding re-read spends the remaining room below a learned gateway ceiling before
+refusing; doubling the look-back allowance does not skip that final usable budget. Reverse paging retries a crowded segment
+with a smaller in-window budget derived from the learned ceiling and the observed pre-window rows.
+
 ### Gateway AtTime is evaluated from raw reads (#693)
 
 > ⚠️ **Client-visible (#693).** Gateway AtTime used to pass AVEVA's own interpolated answer (the gateway's `ReadAtTime`) through,
@@ -880,9 +885,14 @@ data.
   here — the SQL backends store DateTime as its own kind and return a `DateTime`. Under an integer hint a NaN, ±∞ or
   out-of-Int64-range stored value is no integer: it is treated as unusable (as `BadNoData`), so a neighbour like that is never
   interpolated or saturated into a Good value.
-- **What differs between backends (by design, not changed by #693).** Only the **Gateway** converts an AtTime result to the node's
-  integer type (`Int32` ⇒ `int`): the SQL backends store and return integer nodes as `Int64` (and Boolean as Boolean). Gateway
-  **Raw** reads still return a Boolean node's samples as the stored Double 0/1 (the hint is passed only on AtTime).
+- **Client value types (#698, all backends).** Before returning Raw, Modified or AtTime history, the node manager converts non-null
+  scalar values to the node's declared DataType (including Boolean, integer widths and Float). Arrays retain their stored values.
+  Conversion failures produce a null value with `BadTypeMismatch` for that sample, preserving timestamps and modification metadata.
+  DateTime requires an actual DateTime value: the gateway's numeric DateTime encoding is unknown and cannot be inferred.
+  Processed results retain each aggregate's result type, so Count and averages can remain Double on a Boolean or integer node.
+- **Future timestamps (#696).** A bounded raw probe checks for stored samples between now and the requested future time. An empty
+  future interval starts the prior-sample search near now; a full probe seeds the existing walk-back with a stored future sample.
+  The result still carries the requested timestamp and the evaluator's Uncertain status when no later sample exists.
 - ⚠️ **Shared-evaluator fix (all backends that use it):** `AtTimeEvaluator` returned every interpolated `Int64` boxed as a
   **Double** (a C# `cond ? (long)x : double` conditional widens the long); its "keeping the Int64 type" tests passed because
   Shouldly's `ShouldBe` compares numbers across types. SqlServer / QuestDB interpolated `Int64` AtTime values are now `Int64`,
@@ -1764,7 +1774,7 @@ this family and **are** capped (they used to be pushed down). Raise the cap, or
 ### Health
 
 The read client reports `GetHealthSnapshot()` (query/failure counters, consecutive failures, `LastError` — e.g. a missing schema under
-`AutoCreateSchema=false`). A SQL error on a read surfaces as `BadHistoryOperationUnsupported` for **that node only**; a cancelled read
+`AutoCreateSchema=false`). A SQL connection outage on a read surfaces as `BadNoCommunication`; other SQL errors retain `BadHistoryOperationUnsupported` for **that node only**; a cancelled read
 (deadline, client abort) is not counted as a backend fault.
 
 ### Retention (`RetentionDays`)

docs/plans/2026-10-13-historian-695-698-results.md

@@ -0,0 +1,28 @@
+# Historian follow-up results — 2026-10-13
+
+Scope: approved bounded fixes for #695–#698, plus live measurement of #653. Base: `66a2b0c3`.
+
+- #695: spend the remaining learned-ceiling room on crowding reads; reduce crowded reverse-read windows using the observed pre-window count. Regression cases cover ceilings 1,709 and 1,369, and reverse paging through a crowded millisecond.
+- #696 item 1: probe the future interval before starting the prior-sample search near now. Preserve stored future samples, requested timestamps and Uncertain status. Dense future probes still use the existing bounded walk. Items 2 and 3 remain open (stopped-tag strict-ceiling cost and aggregate ceiling re-probes).
+- #697: actual Gateway, SQL Server and QuestDB transport failures become `BadNoCommunication` on every HistoryRead arm. Cancellation remains `BadTimeout`; unsupported operations retain `BadHistoryOperationUnsupported`. Tests include real refused database connections, repeated reads and nested gRPC failures.
+- #698: normalize Raw, Modified and AtTime scalar values to the node DataType. Conversion failure gives per-sample null/`BadTypeMismatch`; arrays, audit metadata and timestamps survive. Processed results retain aggregate-specific types.
+- #694 item 3 remains blocked on historiangw#27. No upstream code was changed.
+
+## Live #653 measurement
+
+Five sequential calls per operation and tag, using client 0.3.0 against the running local gateway (`localhost:5221`) with its existing remote AVEVA connection. Commissioned a two-hour `historian:read` key, then revoked it after measurement. No historian writes or provisioning performed.
+
+| Operation | SysTimeSec, elapsed ms | Nonexistent tag, elapsed ms |
+|---|---|---|
+| Metadata | 299, 85, 81, 82, 48 (success) | 89, 53, 41, 77, 76 (gateway exception) |
+| Raw | 143, 99, 145, 133, 159 (success) | 30260, 30634, 30675, 30754, 30628 (gateway exception) |
+| Aggregate | 271, 275, 309, 249, 304 (success) | 30564, 30605, 30583, 30575, 30806 (gateway exception) |
+| AtTime | 140, 88, 89, 127, 92 (success) | 61074, 61330, 61601, 61249, 61271 (gateway exception) |
+
+Nonexistent tag: `OtOpcUa.Issue653.Nonexistent.20261013`. The delay is still present. Recommend a metadata pre-check with a short negative cache, caching only explicit missing-tag outcomes and never authentication, communication or arbitrary errors. This is follow-up implementation; #653 remains open. Direct gateway AtTime above differs from OtOpcUa's raw-based AtTime evaluator.
+
+## Consumer and validation notes
+
+ScadaBridge source and tests contain no HistoryRead consumer; no consumer change required. Independent review found a future-stored-sample edge case, which was fixed and covered before final verification. No deployment or live end-to-end OPC UA verification of these new fixes is claimed; the live run measures #653 through the gateway client.
+
+Full solution: 14,581 passed, 609 skipped, 0 failed across 62 test suites. Command: `dotnet test ZB.MOM.WW.OtOpcUa.slnx --verbosity quiet`, exit 0. Skips include environment-gated integration suites.