fix(historian): #687 review — one bounded re-read instead of failing, raw budget unclamped, exclusive forward end

- M1/M2: when the 1 ms pre-window margin (boundary row + stored samples, e.g. a
  >16-sample tie cluster at start-1ms / lastTs-1ms) uses up the cap + 16 budget,
  re-read ONCE asking for cap + m + 16 (m = pre-window rows seen); fail
  BadTooManyOperations only if the re-read is still short because of them. The
  raw budget is no longer clamped at the gateway default ceiling (AVEVA always
  emits the boundary row, so page sizes 999,985-1,000,000 always came back
  short); the #688 clamp stays on the per-sample aggregate path only.
- Merge trap: NumValuesPerNode 0 => int.MaxValue, never through the clamp.
- L3: the forward end is exclusive ([start, end), the IHistorianDataSource
  contract); a point read (start == end, the tie-cluster over-fetch) inclusive.
- RequestStartFor never goes before the Unix epoch (#686's walk-back floor).
- L4 (docs): an Uncertain native row for an all-NULL interval is unverified live.
This commit is contained in:
Joseph Doherty
2026-10-12 15:19:34 -04:00
parent 6796ba4126
commit 8114f13a28
5 changed files with 202 additions and 63 deletions
@@ -80,25 +80,32 @@ internal static class RawAggregateReader
/// earlier, so the AVEVA start-boundary row — stamped exactly at the QUERY start, carrying the value in effect there
/// (<c>Quality = 133</c>) or, with no earlier data, a NULL row (value 0, Bad OPC quality) — lands BEFORE the window
/// (<see cref="IsWindowSample"/>). The one place both the raw-evaluated aggregates (#681) and a raw HistoryRead
/// (#687) decide what the request start is. No margin when the window starts within one margin of
/// <see cref="DateTime.MinValue"/>.
/// (#687) decide what the request start is. Never before <see cref="ProbeFloor"/> (the Unix epoch — the start of
/// the historian's time encoding, the same floor the server's reverse walk-back uses, #686): a window starting less
/// than one margin after it is requested from the epoch itself.
/// </summary>
/// <param name="windowStartUtc">The window's (inclusive) start.</param>
/// <returns>The start to request.</returns>
public static DateTime RequestStartFor(DateTime windowStartUtc) =>
windowStartUtc.Ticks > BoundaryMargin.Ticks ? windowStartUtc - BoundaryMargin : windowStartUtc;
windowStartUtc - ProbeFloor > BoundaryMargin ? windowStartUtc - BoundaryMargin
: windowStartUtc > ProbeFloor ? ProbeFloor
: windowStartUtc;
/// <summary>
/// True when a row of a stream requested from <see cref="RequestStartFor"/> is a stored sample of the window that
/// begins at <paramref name="windowStartUtc"/>: it is stamped at or after the window start. A row before it is the
/// AVEVA boundary row (whatever its value or quality — a stored NULL looks the same on the wire, so the timestamp is
/// the evidence) or a stored sample inside the margin; a row without a timestamp is never a sample.
/// True when a row of a forward raw stream requested from <see cref="RequestStartFor"/> is a stored sample of the raw
/// window <c>[windowStartUtc, windowEndUtc)</c> — the <see cref="IHistorianDataSource.ReadRawAsync"/> contract
/// (exclusive end, as on the SQL backends; the gateway's own <c>ReadRaw</c> end is inclusive) — or, for a point read
/// (<c>start == end</c>, the server's tie-cluster over-fetch), of that one instant. A row before the start is the AVEVA
/// boundary row (whatever its value or quality — a stored NULL looks the same on the wire, so the timestamp is the
/// evidence) or a stored sample inside the margin; a row without a timestamp is never a sample.
/// </summary>
/// <param name="raw">The gateway row.</param>
/// <param name="windowStartUtc">The window's (inclusive) start.</param>
/// <param name="windowEndUtc">The window's end: exclusive, or inclusive when it equals the start.</param>
/// <returns>Whether the row belongs to the window.</returns>
public static bool IsWindowSample(HistorianSample raw, DateTime windowStartUtc) =>
raw.Timestamp is { } ts && ts.ToDateTime() >= windowStartUtc;
public static bool IsWindowSample(HistorianSample raw, DateTime windowStartUtc, DateTime windowEndUtc) =>
raw.Timestamp is { } ts && ts.ToDateTime() is var t && t >= windowStartUtc
&& (t < windowEndUtc || (t == windowEndUtc && windowStartUtc == windowEndUtc));
/// <summary>True for the aggregates this reader serves: the per-sample families of <see cref="AggregateFamilies"/>.</summary>
/// <param name="aggregate">Requested aggregate.</param>