Files
lmxopcua/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/RawPageLimit.cs
T
Joseph Doherty 3c3513ea3e fix(historian): #694 page raw reads at a strict gateway's learned ceiling; opt-in EventReadHorizon
Item 1: IHistorianDataSource.MaxRawValuesPerRead (Gateway: learned ceiling - 16 = L; 0 elsewhere, guard-tested).
RawPageLimit plans every raw page on top of #692's lookahead: with L > 0 the read (page P + boundary ties k
+ lookahead) is <= L, P = min(N, L - k - 1) (uncapped: L - k - 1); no room => straight to the tie-cluster
over-fetch (never an uncapped or underflowing request). One re-plan + re-read when the read taught a new
limit or was refused for crowding pre-window rows (HistoryReadLimitExceededException.PreWindowRows): the
page shrinks by them. Over-fetch bound min(MaxTieClusterOverfetch, L - 1); slice size never 0.
Review fixes: F1 a window starting at/before the epoch counts AVEVA's synthetic row as pre-window (a
ceiling-truncated read was reported complete); F2 the page path disables ReverseRawReader's own re-run
(cold end-only ceiling 20/N=3000 cost 3,023 reads, now <= 40); F3 crowded look-back re-read around, not
refused; F4 the #687 crowding re-read room at least doubles (up to 8 re-reads), so a cluster of hundreds
just before a resume point no longer fails on a default gateway; a crowding refusal that leaves no room
is BadTooManyOperations, never a tick-by-tick cursor.

Item 2: ServerHistorian:EventReadHorizon (default 00:00:00 = exact, unchanged). When set, every fresh
HistoryReadEvents window's earlier bound is raised to now - horizon (both directions, all backends).
Lossy by design; documented.

Item 3 (native TimeAverage/Total re-check) stays blocked on historiangw#27.
2026-10-13 01:39:32 -04:00

98 lines
5.9 KiB
C#
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer;
/// <summary>How one HistoryRead-Raw page reads (<see cref="RawPageLimit.Plan"/>).</summary>
/// <param name="Limit">The backend's <see cref="IHistorianDataSource.MaxRawValuesPerRead"/> the plan was made under (0 = none).</param>
/// <param name="PageCap">The page: at most this many values are returned (0 = all values — only without a limit).</param>
/// <param name="RequestCap">What the read asks for: page + the boundary ties it will trim + one lookahead row (0 = all).</param>
/// <param name="NoRoom">The boundary ties alone fill a read — nothing can be read past them; serve the tie cluster.</param>
internal readonly record struct RawPagePlan(int Limit, uint PageCap, uint RequestCap, bool NoRoom);
/// <summary>
/// Gitea #694: sizes a HistoryRead-Raw page to what the backend can return in one read
/// (<see cref="IHistorianDataSource.MaxRawValuesPerRead"/> = L — a strict HistorianGateway's learned row ceiling, less
/// its boundary-row allowance). With a limit the read — page P + boundary ties k + the #692 lookahead row — is at most L,
/// so P = min(want, L − k − 1) (all values: P = L − k − 1); a full page carries a continuation point like any other, as
/// Part 11 allows fewer values than <c>NumValuesPerNode</c> with one. Before #694 a cap above a strict gateway's ceiling
/// (or an uncapped read of a window holding more) failed <c>BadTooManyOperations</c>.
/// </summary>
/// <remarks>
/// <para>
/// <b>Never a silent truncation.</b> The backend still answers a read in full or refuses it; only the server's page is
/// smaller, and a page is judged full against what was asked for. A read the backend cannot serve at all (a tie
/// cluster larger than one read) stays <c>BadTooManyOperations</c>.
/// </para>
/// <para>
/// <b>One re-read, planned again.</b> A refused read (<see cref="HistoryReadLimitExceededException"/>) is re-planned and
/// re-issued ONCE when (a) the backend learned a different limit while it ran — the first read after start-up teaches
/// the ceiling — or (b) it was refused because rows before the window crowded the budget
/// (<see cref="HistoryReadLimitExceededException.PreWindowRows"/> m): the page is then shrunk by m so that the
/// backend's look-back fits beside it. Anything else is the answer.
/// </para>
/// </remarks>
internal static class RawPageLimit
{
/// <summary>Plans one page.</summary>
/// <param name="want">The page the client asked for (N, less a start bound; 0 = all values).</param>
/// <param name="boundaryTies">Ties at the resume timestamp already returned, which the read re-reads and trims (k).</param>
/// <param name="limit">The backend's limit L (&lt;= 0 = none).</param>
/// <param name="sizeLimit">The limit to size the page by (L, or L − m after a crowded refusal); defaults to <paramref name="limit"/>.</param>
/// <returns>The plan.</returns>
public static RawPagePlan Plan(uint want, int boundaryTies, int limit, int? sizeLimit = null)
{
var k = Math.Max(0, boundaryTies);
if (limit <= 0)
return new RawPagePlan(0, want, want == 0 ? 0u : (uint)Math.Min(uint.MaxValue, (long)want + k + 1L), false);
var room = (long)(sizeLimit ?? limit) - k - 1;
if (room < 1) return new RawPagePlan(limit, 0, 0, true);
var page = want == 0 ? room : Math.Min(want, room);
return new RawPagePlan(limit, (uint)page, (uint)(page + k + 1), false);
}
/// <summary>
/// Reads one page under the backend's current limit, re-planning and re-reading ONCE as described in the remarks.
/// A <see cref="RawPagePlan.NoRoom"/> plan is returned without a read.
/// </summary>
/// <typeparam name="T">The page.</typeparam>
/// <param name="source">The backend.</param>
/// <param name="want">The page the client asked for (0 = all values).</param>
/// <param name="boundaryTies">The boundary ties the read re-reads and trims.</param>
/// <param name="read">One read with the given plan (it asks for <see cref="RawPagePlan.RequestCap"/>).</param>
/// <returns>The page (null when <see cref="RawPagePlan.NoRoom"/>) and the plan it was read with.</returns>
public static async Task<(T? Page, RawPagePlan Plan)> ReadAsync<T>(
IHistorianDataSource source, uint want, int boundaryTies, Func<RawPagePlan, Task<T>> read)
{
var before = source.MaxRawValuesPerRead;
var plan = Plan(want, boundaryTies, before);
if (plan.NoRoom) return (default, plan);
HistoryReadLimitExceededException refused;
try
{
return (await read(plan).ConfigureAwait(false), plan);
}
catch (HistoryReadLimitExceededException ex) when (source.MaxRawValuesPerRead is var now and > 0
&& (now != before || ex.PreWindowRows > 0))
{
refused = ex; // re-read outside the catch: the second refusal is the answer
}
var limit = source.MaxRawValuesPerRead;
var shrink = Math.Max(0, refused.PreWindowRows);
plan = Plan(want, boundaryTies, limit, (int)Math.Max(0L, (long)limit - shrink));
// No room left once the crowding rows are allowed for: those rows are not (necessarily) the boundary ties, so this is
// NOT "the cursor sits in a tie cluster" — the refusal is the answer (stepping the cursor through it a tick per page
// would never end).
// (A limit that changed under an un-crowded read leaves the boundary ties alone filling a read: that IS the tie
// cluster at the cursor, served by the over-fetch.)
if (plan.NoRoom)
{
if (shrink > 0) throw refused;
return (default, plan);
}
return (await read(plan).ConfigureAwait(false), plan);
}
}