dohertj2/lmxopcua
1
0
Fork 0
Code Issues 43 Pull Requests Actions Packages Projects Releases Wiki Activity

test(historian): paged-vs-unpaged raw-cap fuzz, MaxReturnDataValues advertisement, cap floor of 3

Browse Source 
- NodeManagerHistoryReadRawCapFuzzTests: 10k seeds (env OTOPCUA_PAGING_FUZZ_ITERATIONS)
  through a real SDK node manager against wire-faithful fakes of SqlServer / QuestDB /
  Gateway that enforce the backend half of the cap; random series with ties, random
  windows (empty, on/past a tie, between samples, forward/reverse/end-only, bounds),
  N in {0,1,2,small,huge}, cap in {off,3..1000}, tie budget {3,8,65536}. Pages must
  concatenate to the cap-off one-shot answer (checked against the series), terminate
  within values + 2 (+ one empty step per oversized boundary cluster), never exceed
  the cap, never ask the backend for 0 / over-cap; refusals only past the budget.
- HistoryPaging.AdvertiseMaxReturnDataValues writes HistoryServerCapabilities/
  MaxReturnDataValues (i=11273), read back through the SDK master node manager.
- MinMaxRawValuesPerRead raised to 3: at 2 a resumed page has no room (value + its
  boundary tie + lookahead) and every value cost an empty page + CP (seed 15).
- RawPageLimit rows pin the default-cap arithmetic (9 999 first page, 9 998 resumed).

Fuzz findings recorded as follow-ups: a bounded read whose data fits one page
returns cap + 1 (end bound fitted against N, not the cap; node manager, F2);
the Gateway walk-back's cost on a sparse series under a tiny tie budget is
pre-existing (seed 320: 82 reads uncapped, 93 capped).

Refs #736

Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
This commit is contained in:
Joseph Doherty
2026-10-22 10:15:21 -04:00
parent 6669fae7e5
commit 7e8941fbdc
7 changed files with 576 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/HistoryPaging.cs
+43 -6
View File
@@ -1,3 +1,5 @@
using Opc.Ua;
using Opc.Ua.Server;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer;
@@ -110,16 +112,51 @@ internal enum HistoryCursorKind
/// <c>NumValuesPerNode</c> samples with a null continuation point — so paging is synthesised here,
/// time-based:
/// <list type="bullet">
/// <item>A page that returns EXACTLY the requested cap (<c>NumValuesPerNode &gt; 0</c>) MAY have
/// more behind it ⇒ emit a continuation point.</item>
/// <item>A short page (fewer than the cap) is the last page ⇒ no continuation point.</item>
/// <item><c>NumValuesPerNode == 0</c> means "all values, no limit" (OPC UA Part 11) ⇒ never page;
/// return everything in one shot.</item>
/// <item>A page whose read returned the lookahead row past it MAY have more behind it ⇒ emit a continuation
/// point (<see cref="IsFullPage"/> judges a read against what was asked for).</item>
/// <item>A short read (no lookahead row) is the last page ⇒ no continuation point.</item>
/// <item><c>NumValuesPerNode == 0</c> means "all values" (OPC UA Part 11) — but not "in one shot": the page is
/// sized to the backend's <see cref="IHistorianDataSource.MaxRawValuesPerRead"/> by <see cref="RawPageLimit"/>
/// (the server raw cap <c>ServerHistorian:MaxRawValuesPerRead</c>, default 10 000, and/or a strict gateway's
/// learned ceiling; review R08:1.1, #736), and a full page carries a continuation point like any other. Only
/// with no limit at all (<c>0</c>, the opt-out) is everything returned in one shot.</item>
/// </list>
/// All methods are static + pure so they unit-test without a server, a session, or the SDK.
/// All paging methods are static + pure so they unit-test without a server, a session, or the SDK;
/// <see cref="AdvertiseMaxReturnDataValues"/> is the one SDK-facing member.
/// </summary>
internal static class HistoryPaging
{
/// <summary>
/// Advertise the server raw cap as <c>HistoryServerCapabilities/MaxReturnDataValues</c> (i=11273; Part 11 §5.4.2:
/// "the maximum number of values that can be returned by the Server for each Node", 0 = no limit). The cap is
/// the most values one HistoryRead-Raw page returns (<c>ServerHistorian:MaxRawValuesPerRead</c>), so a client can
/// size its <c>NumValuesPerNode</c> and expect a continuation point past it. Idempotent; a re-sync with a
/// different cap overwrites. Never throws: a missing diagnostics manager or node leaves the SDK default (0) and
/// returns <c>false</c> — HistoryRead itself still enforces the cap.
/// </summary>
/// <param name="server">The running server.</param>
/// <param name="maxRawValuesPerRead">The cap (&lt;= 0 advertises 0 = no limit).</param>
/// <returns><c>true</c> when the property was written.</returns>
public static bool AdvertiseMaxReturnDataValues(IServerInternal? server, int maxRawValuesPerRead)
{
if (server?.DiagnosticsNodeManager is not DiagnosticsNodeManager diagnostics) return false;
try
{
lock (diagnostics.Lock)
{
var capabilities = diagnostics.GetDefaultHistoryCapabilities();
if (capabilities?.MaxReturnDataValues is not { } property) return false;
property.Value = (uint)Math.Max(0, maxRawValuesPerRead);
property.ClearChangeMasks(diagnostics.SystemContext, false);
return true;
}
}
catch (ServiceResultException)
{
return false;
}
}
/// <summary>
/// Decide whether a just-returned page is "full" and therefore MAY be followed by more data —
/// the signal to emit a continuation point. A page is full when the client asked for a finite
src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/RawPageLimit.cs
+11 -6
View File
@@ -11,17 +11,22 @@ internal readonly record struct RawPagePlan(int Limit, uint PageCap, uint Reques
/// <summary>
/// sizes a HistoryRead-Raw page to what the backend can return in one read
/// (<see cref="IHistorianDataSource.MaxRawValuesPerRead"/> = L — a strict HistorianGateway's learned row ceiling, less
/// its boundary-row allowance). With a limit the read — page P + boundary ties k + the lookahead row — is at most L,
/// so P = min(want, L − k − 1) (all values: P = L − k − 1); a full page carries a continuation point like any other, as
/// Part 11 allows fewer values than <c>NumValuesPerNode</c> with one. Previously a cap above a strict gateway's ceiling
/// (or an uncapped read of a window holding more) failed <c>BadTooManyOperations</c>.
/// (<see cref="IHistorianDataSource.MaxRawValuesPerRead"/> = L — the server raw cap
/// <c>ServerHistorian:MaxRawValuesPerRead</c>, default 10 000, which every backend reports (review R08:1.1, #736),
/// and/or a strict HistorianGateway's learned row ceiling less its boundary-row allowance, whichever is lower). With
/// a limit the read — page P + boundary ties k + the lookahead row — is at most L, so P = min(want, L − k − 1) (all
/// values: P = L − k − 1, i.e. a first page of 9 999 at the default cap); a full page carries a continuation point like
/// any other, as Part 11 allows fewer values than <c>NumValuesPerNode</c> with one. Previously a cap above a strict
/// gateway's ceiling (or an uncapped read of a window holding more) failed <c>BadTooManyOperations</c>, and before
/// #736 an uncapped read on a backend without a ceiling read the whole window in one shot.
/// </summary>
/// <remarks>
/// <para>
/// <b>Never a silent truncation.</b> The backend still answers a read in full or refuses it; only the server's page is
/// smaller, and a page is judged full against what was asked for. A read the backend cannot serve at all (a tie
/// cluster larger than one read) stays <c>BadTooManyOperations</c>.
/// cluster larger than one read) stays <c>BadTooManyOperations</c>. The backends enforce the same cap themselves
/// (a read asking for more than L, or for everything, is refused when the window holds more than L rows), so this
/// planner is what keeps the server's own reads inside it: with L &gt; 0 no read it plans ever asks for 0 or more than L.
/// </para>
/// <para>
/// <b>One re-read, planned again.</b> A refused read (<see cref="HistoryReadLimitExceededException"/>) is re-planned and
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ServerHistorianOptions.cs
+10 -6
View File
@@ -158,10 +158,12 @@ public sealed class ServerHistorianOptions
public const int DefaultMaxRawValuesPerRead = 10_000;
/// <summary>
/// The smallest usable <see cref="MaxRawValuesPerRead"/>: a read must carry a page of at least one value AND the
/// lookahead row that tells the server more data follows, so a cap of 1 is raised to 2 (<see cref="EffectiveMaxRawValuesPerRead"/>).
/// The smallest usable <see cref="MaxRawValuesPerRead"/>: a resumed page's read must carry at least one value, the
/// boundary tie it re-reads and trims, AND the lookahead row that tells the server more data follows. At 2 a resumed
/// page has no room at all and every value costs an empty page with a continuation point (the #692 defect, re-created
/// by configuration), so 1 and 2 are raised to 3 (<see cref="EffectiveMaxRawValuesPerRead"/>).
/// </summary>
public const int MinMaxRawValuesPerRead = 2;
public const int MinMaxRawValuesPerRead = 3;
/// <summary>
/// Every backend: the most raw values the server returns on ONE HistoryRead-Raw page, and the most rows one
@@ -175,8 +177,10 @@ public sealed class ServerHistorianOptions
/// and is REFUSED (<c>BadTooManyOperations</c>) when the window holds more — never answered truncated — so a caller that
/// bypasses the node manager's paging cannot bypass the bound. On the Gateway backend a strict gateway's learned row
/// ceiling (less its boundary-row allowance) still applies when lower. The tie-cluster bound of a resumed page is also
/// at most cap − 1 (as under a gateway ceiling; <see cref="MaxTieClusterOverfetch"/> above that is inert). Default
/// 10 000. Non-positive = unbounded, the pre-#736 behaviour (a <see cref="Validate"/> warning); 1 is raised to 2.
/// at most cap − 1 (as under a gateway ceiling; <see cref="MaxTieClusterOverfetch"/> above that is inert), and a
/// page whose boundary tie cluster is that large steps past it with one empty page. Default 10 000. Non-positive =
/// unbounded, the pre-#736 behaviour (a <see cref="Validate"/> warning); below <see cref="MinMaxRawValuesPerRead"/>
/// (3) is raised to it.
/// </summary>
public int MaxRawValuesPerRead { get; init; } = DefaultMaxRawValuesPerRead;
@@ -243,7 +247,7 @@ public sealed class ServerHistorianOptions
if (MaxRawValuesPerRead <= 0)
warnings.Add($"ServerHistorian:MaxRawValuesPerRead is {MaxRawValuesPerRead} — must be > 0; HistoryRead-Raw pages and backend raw reads are unbounded, so a client asking for a whole window (NumValuesPerNode = 0) materialises every sample in memory.");
else if (MaxRawValuesPerRead < MinMaxRawValuesPerRead)
warnings.Add($"ServerHistorian:MaxRawValuesPerRead is {MaxRawValuesPerRead} — must be at least {MinMaxRawValuesPerRead} (a page plus its lookahead row); treated as {MinMaxRawValuesPerRead}.");
warnings.Add($"ServerHistorian:MaxRawValuesPerRead is {MaxRawValuesPerRead} — must be at least {MinMaxRawValuesPerRead} (a value, its boundary tie and the lookahead row); treated as {MinMaxRawValuesPerRead}.");
if (EventReadHorizon < TimeSpan.Zero)
warnings.Add($"ServerHistorian:EventReadHorizon is {EventReadHorizon} — negative is treated as no horizon (00:00:00); HistoryReadEvents reads every window in full.");
if (HistoryReadDeadline <= TimeSpan.Zero)