Files
lmxopcua/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/RawPageLimit.cs
T
Joseph Doherty 7e8941fbdc test(historian): paged-vs-unpaged raw-cap fuzz, MaxReturnDataValues advertisement, cap floor of 3
- NodeManagerHistoryReadRawCapFuzzTests: 10k seeds (env OTOPCUA_PAGING_FUZZ_ITERATIONS)
  through a real SDK node manager against wire-faithful fakes of SqlServer / QuestDB /
  Gateway that enforce the backend half of the cap; random series with ties, random
  windows (empty, on/past a tie, between samples, forward/reverse/end-only, bounds),
  N in {0,1,2,small,huge}, cap in {off,3..1000}, tie budget {3,8,65536}. Pages must
  concatenate to the cap-off one-shot answer (checked against the series), terminate
  within values + 2 (+ one empty step per oversized boundary cluster), never exceed
  the cap, never ask the backend for 0 / over-cap; refusals only past the budget.
- HistoryPaging.AdvertiseMaxReturnDataValues writes HistoryServerCapabilities/
  MaxReturnDataValues (i=11273), read back through the SDK master node manager.
- MinMaxRawValuesPerRead raised to 3: at 2 a resumed page has no room (value + its
  boundary tie + lookahead) and every value cost an empty page + CP (seed 15).
- RawPageLimit rows pin the default-cap arithmetic (9 999 first page, 9 998 resumed).

Fuzz findings recorded as follow-ups: a bounded read whose data fits one page
returns cap + 1 (end bound fitted against N, not the cap; node manager, F2);
the Gateway walk-back's cost on a sparse series under a tiny tie budget is
pre-existing (seed 320: 82 reads uncapped, 93 capped).

Refs #736

Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
2026-10-22 10:15:21 -04:00

103 lines
6.5 KiB
C#
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer;
/// <summary>How one HistoryRead-Raw page reads (<see cref="RawPageLimit.Plan"/>).</summary>
/// <param name="Limit">The backend's <see cref="IHistorianDataSource.MaxRawValuesPerRead"/> the plan was made under (0 = none).</param>
/// <param name="PageCap">The page: at most this many values are returned (0 = all values — only without a limit).</param>
/// <param name="RequestCap">What the read asks for: page + the boundary ties it will trim + one lookahead row (0 = all).</param>
/// <param name="NoRoom">The boundary ties alone fill a read — nothing can be read past them; serve the tie cluster.</param>
internal readonly record struct RawPagePlan(int Limit, uint PageCap, uint RequestCap, bool NoRoom);
/// <summary>
/// sizes a HistoryRead-Raw page to what the backend can return in one read
/// (<see cref="IHistorianDataSource.MaxRawValuesPerRead"/> = L — the server raw cap
/// <c>ServerHistorian:MaxRawValuesPerRead</c>, default 10 000, which every backend reports (review R08:1.1, #736),
/// and/or a strict HistorianGateway's learned row ceiling less its boundary-row allowance, whichever is lower). With
/// a limit the read — page P + boundary ties k + the lookahead row — is at most L, so P = min(want, L − k − 1) (all
/// values: P = L − k − 1, i.e. a first page of 9 999 at the default cap); a full page carries a continuation point like
/// any other, as Part 11 allows fewer values than <c>NumValuesPerNode</c> with one. Previously a cap above a strict
/// gateway's ceiling (or an uncapped read of a window holding more) failed <c>BadTooManyOperations</c>, and before
/// #736 an uncapped read on a backend without a ceiling read the whole window in one shot.
/// </summary>
/// <remarks>
/// <para>
/// <b>Never a silent truncation.</b> The backend still answers a read in full or refuses it; only the server's page is
/// smaller, and a page is judged full against what was asked for. A read the backend cannot serve at all (a tie
/// cluster larger than one read) stays <c>BadTooManyOperations</c>. The backends enforce the same cap themselves
/// (a read asking for more than L, or for everything, is refused when the window holds more than L rows), so this
/// planner is what keeps the server's own reads inside it: with L &gt; 0 no read it plans ever asks for 0 or more than L.
/// </para>
/// <para>
/// <b>One re-read, planned again.</b> A refused read (<see cref="HistoryReadLimitExceededException"/>) is re-planned and
/// re-issued ONCE when (a) the backend learned a different limit while it ran — the first read after start-up teaches
/// the ceiling — or (b) it was refused because rows before the window crowded the budget
/// (<see cref="HistoryReadLimitExceededException.PreWindowRows"/> m): the page is then shrunk by m so that the
/// backend's look-back fits beside it. Anything else is the answer.
/// </para>
/// </remarks>
internal static class RawPageLimit
{
/// <summary>Plans one page.</summary>
/// <param name="want">The page the client asked for (N, less a start bound; 0 = all values).</param>
/// <param name="boundaryTies">Ties at the resume timestamp already returned, which the read re-reads and trims (k).</param>
/// <param name="limit">The backend's limit L (&lt;= 0 = none).</param>
/// <param name="sizeLimit">The limit to size the page by (L, or L − m after a crowded refusal); defaults to <paramref name="limit"/>.</param>
/// <returns>The plan.</returns>
public static RawPagePlan Plan(uint want, int boundaryTies, int limit, int? sizeLimit = null)
{
var k = Math.Max(0, boundaryTies);
if (limit <= 0)
return new RawPagePlan(0, want, want == 0 ? 0u : (uint)Math.Min(uint.MaxValue, (long)want + k + 1L), false);
var room = (long)(sizeLimit ?? limit) - k - 1;
if (room < 1) return new RawPagePlan(limit, 0, 0, true);
var page = want == 0 ? room : Math.Min(want, room);
return new RawPagePlan(limit, (uint)page, (uint)(page + k + 1), false);
}
/// <summary>
/// Reads one page under the backend's current limit, re-planning and re-reading ONCE as described in the remarks.
/// A <see cref="RawPagePlan.NoRoom"/> plan is returned without a read.
/// </summary>
/// <typeparam name="T">The page.</typeparam>
/// <param name="source">The backend.</param>
/// <param name="want">The page the client asked for (0 = all values).</param>
/// <param name="boundaryTies">The boundary ties the read re-reads and trims.</param>
/// <param name="read">One read with the given plan (it asks for <see cref="RawPagePlan.RequestCap"/>).</param>
/// <returns>The page (null when <see cref="RawPagePlan.NoRoom"/>) and the plan it was read with.</returns>
public static async Task<(T? Page, RawPagePlan Plan)> ReadAsync<T>(
IHistorianDataSource source, uint want, int boundaryTies, Func<RawPagePlan, Task<T>> read)
{
var before = source.MaxRawValuesPerRead;
var plan = Plan(want, boundaryTies, before);
if (plan.NoRoom) return (default, plan);
HistoryReadLimitExceededException refused;
try
{
return (await read(plan).ConfigureAwait(false), plan);
}
catch (HistoryReadLimitExceededException ex) when (source.MaxRawValuesPerRead is var now and > 0
&& (now != before || ex.PreWindowRows > 0))
{
refused = ex; // re-read outside the catch: the second refusal is the answer
}
var limit = source.MaxRawValuesPerRead;
var shrink = Math.Max(0, refused.PreWindowRows);
plan = Plan(want, boundaryTies, limit, (int)Math.Max(0L, (long)limit - shrink));
// No room left once the crowding rows are allowed for: those rows are not (necessarily) the boundary ties, so this is
// NOT "the cursor sits in a tie cluster" — the refusal is the answer (stepping the cursor through it a tick per page
// would never end).
// (A limit that changed under an un-crowded read leaves the boundary ties alone filling a read: that IS the tie
// cluster at the cursor, served by the over-fetch.)
if (plan.NoRoom)
{
if (shrink > 0) throw refused;
return (default, plan);
}
return (await read(plan).ConfigureAwait(false), plan);
}
}