fix(historian): #694 page raw reads at a strict gateway's learned ceiling; opt-in EventReadHorizon

Item 1: IHistorianDataSource.MaxRawValuesPerRead (Gateway: learned ceiling - 16 = L; 0 elsewhere, guard-tested).
RawPageLimit plans every raw page on top of #692's lookahead: with L > 0 the read (page P + boundary ties k
+ lookahead) is <= L, P = min(N, L - k - 1) (uncapped: L - k - 1); no room => straight to the tie-cluster
over-fetch (never an uncapped or underflowing request). One re-plan + re-read when the read taught a new
limit or was refused for crowding pre-window rows (HistoryReadLimitExceededException.PreWindowRows): the
page shrinks by them. Over-fetch bound min(MaxTieClusterOverfetch, L - 1); slice size never 0.
Review fixes: F1 a window starting at/before the epoch counts AVEVA's synthetic row as pre-window (a
ceiling-truncated read was reported complete); F2 the page path disables ReverseRawReader's own re-run
(cold end-only ceiling 20/N=3000 cost 3,023 reads, now <= 40); F3 crowded look-back re-read around, not
refused; F4 the #687 crowding re-read room at least doubles (up to 8 re-reads), so a cluster of hundreds
just before a resume point no longer fails on a default gateway; a crowding refusal that leaves no room
is BadTooManyOperations, never a tick-by-tick cursor.

Item 2: ServerHistorian:EventReadHorizon (default 00:00:00 = exact, unchanged). When set, every fresh
HistoryReadEvents window's earlier bound is raised to now - horizon (both directions, all backends).
Lossy by design; documented.

Item 3 (native TimeAverage/Total re-check) stays blocked on historiangw#27.
This commit is contained in:
Joseph Doherty
2026-10-13 00:52:12 -04:00
parent a5a0496176
commit 3c3513ea3e
23 changed files with 1461 additions and 79 deletions
@@ -0,0 +1,97 @@
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer;
/// <summary>How one HistoryRead-Raw page reads (<see cref="RawPageLimit.Plan"/>).</summary>
/// <param name="Limit">The backend's <see cref="IHistorianDataSource.MaxRawValuesPerRead"/> the plan was made under (0 = none).</param>
/// <param name="PageCap">The page: at most this many values are returned (0 = all values — only without a limit).</param>
/// <param name="RequestCap">What the read asks for: page + the boundary ties it will trim + one lookahead row (0 = all).</param>
/// <param name="NoRoom">The boundary ties alone fill a read — nothing can be read past them; serve the tie cluster.</param>
internal readonly record struct RawPagePlan(int Limit, uint PageCap, uint RequestCap, bool NoRoom);
/// <summary>
/// Gitea #694: sizes a HistoryRead-Raw page to what the backend can return in one read
/// (<see cref="IHistorianDataSource.MaxRawValuesPerRead"/> = L — a strict HistorianGateway's learned row ceiling, less
/// its boundary-row allowance). With a limit the read — page P + boundary ties k + the #692 lookahead row — is at most L,
/// so P = min(want, L − k − 1) (all values: P = L − k − 1); a full page carries a continuation point like any other, as
/// Part 11 allows fewer values than <c>NumValuesPerNode</c> with one. Before #694 a cap above a strict gateway's ceiling
/// (or an uncapped read of a window holding more) failed <c>BadTooManyOperations</c>.
/// </summary>
/// <remarks>
/// <para>
/// <b>Never a silent truncation.</b> The backend still answers a read in full or refuses it; only the server's page is
/// smaller, and a page is judged full against what was asked for. A read the backend cannot serve at all (a tie
/// cluster larger than one read) stays <c>BadTooManyOperations</c>.
/// </para>
/// <para>
/// <b>One re-read, planned again.</b> A refused read (<see cref="HistoryReadLimitExceededException"/>) is re-planned and
/// re-issued ONCE when (a) the backend learned a different limit while it ran — the first read after start-up teaches
/// the ceiling — or (b) it was refused because rows before the window crowded the budget
/// (<see cref="HistoryReadLimitExceededException.PreWindowRows"/> m): the page is then shrunk by m so that the
/// backend's look-back fits beside it. Anything else is the answer.
/// </para>
/// </remarks>
internal static class RawPageLimit
{
/// <summary>Plans one page.</summary>
/// <param name="want">The page the client asked for (N, less a start bound; 0 = all values).</param>
/// <param name="boundaryTies">Ties at the resume timestamp already returned, which the read re-reads and trims (k).</param>
/// <param name="limit">The backend's limit L (&lt;= 0 = none).</param>
/// <param name="sizeLimit">The limit to size the page by (L, or L − m after a crowded refusal); defaults to <paramref name="limit"/>.</param>
/// <returns>The plan.</returns>
public static RawPagePlan Plan(uint want, int boundaryTies, int limit, int? sizeLimit = null)
{
var k = Math.Max(0, boundaryTies);
if (limit <= 0)
return new RawPagePlan(0, want, want == 0 ? 0u : (uint)Math.Min(uint.MaxValue, (long)want + k + 1L), false);
var room = (long)(sizeLimit ?? limit) - k - 1;
if (room < 1) return new RawPagePlan(limit, 0, 0, true);
var page = want == 0 ? room : Math.Min(want, room);
return new RawPagePlan(limit, (uint)page, (uint)(page + k + 1), false);
}
/// <summary>
/// Reads one page under the backend's current limit, re-planning and re-reading ONCE as described in the remarks.
/// A <see cref="RawPagePlan.NoRoom"/> plan is returned without a read.
/// </summary>
/// <typeparam name="T">The page.</typeparam>
/// <param name="source">The backend.</param>
/// <param name="want">The page the client asked for (0 = all values).</param>
/// <param name="boundaryTies">The boundary ties the read re-reads and trims.</param>
/// <param name="read">One read with the given plan (it asks for <see cref="RawPagePlan.RequestCap"/>).</param>
/// <returns>The page (null when <see cref="RawPagePlan.NoRoom"/>) and the plan it was read with.</returns>
public static async Task<(T? Page, RawPagePlan Plan)> ReadAsync<T>(
IHistorianDataSource source, uint want, int boundaryTies, Func<RawPagePlan, Task<T>> read)
{
var before = source.MaxRawValuesPerRead;
var plan = Plan(want, boundaryTies, before);
if (plan.NoRoom) return (default, plan);
HistoryReadLimitExceededException refused;
try
{
return (await read(plan).ConfigureAwait(false), plan);
}
catch (HistoryReadLimitExceededException ex) when (source.MaxRawValuesPerRead is var now and > 0
&& (now != before || ex.PreWindowRows > 0))
{
refused = ex; // re-read outside the catch: the second refusal is the answer
}
var limit = source.MaxRawValuesPerRead;
var shrink = Math.Max(0, refused.PreWindowRows);
plan = Plan(want, boundaryTies, limit, (int)Math.Max(0L, (long)limit - shrink));
// No room left once the crowding rows are allowed for: those rows are not (necessarily) the boundary ties, so this is
// NOT "the cursor sits in a tie cluster" — the refusal is the answer (stepping the cursor through it a tick per page
// would never end).
// (A limit that changed under an un-crowded read leaves the boundary ties alone filling a read: that IS the tie
// cluster at the cursor, served by the over-fetch.)
if (plan.NoRoom)
{
if (shrink > 0) throw refused;
return (default, plan);
}
return (await read(plan).ConfigureAwait(false), plan);
}
}