fix(historian): #694 page raw reads at a strict gateway's learned ceiling; opt-in EventReadHorizon
Item 1: IHistorianDataSource.MaxRawValuesPerRead (Gateway: learned ceiling - 16 = L; 0 elsewhere, guard-tested). RawPageLimit plans every raw page on top of #692's lookahead: with L > 0 the read (page P + boundary ties k + lookahead) is <= L, P = min(N, L - k - 1) (uncapped: L - k - 1); no room => straight to the tie-cluster over-fetch (never an uncapped or underflowing request). One re-plan + re-read when the read taught a new limit or was refused for crowding pre-window rows (HistoryReadLimitExceededException.PreWindowRows): the page shrinks by them. Over-fetch bound min(MaxTieClusterOverfetch, L - 1); slice size never 0. Review fixes: F1 a window starting at/before the epoch counts AVEVA's synthetic row as pre-window (a ceiling-truncated read was reported complete); F2 the page path disables ReverseRawReader's own re-run (cold end-only ceiling 20/N=3000 cost 3,023 reads, now <= 40); F3 crowded look-back re-read around, not refused; F4 the #687 crowding re-read room at least doubles (up to 8 re-reads), so a cluster of hundreds just before a resume point no longer fails on a default gateway; a crowding refusal that leaves no room is BadTooManyOperations, never a tick-by-tick cursor. Item 2: ServerHistorian:EventReadHorizon (default 00:00:00 = exact, unchanged). When set, every fresh HistoryReadEvents window's earlier bound is raised to now - horizon (both directions, all backends). Lossy by design; documented. Item 3 (native TimeAverage/Total re-check) stays blocked on historiangw#27.
This commit is contained in:
Joseph Doherty
@@ -320,6 +320,9 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl
|
||||
// Gitea #691: the ReturnBounds kill switch (default on).
|
||||
_server.NodeManager.HonorReturnBounds = HonorReturnBounds;
|
||||
|
||||
// Gitea #694: the opt-in HistoryReadEvents horizon (default off = exact to each read's own window).
|
||||
_server.NodeManager.EventReadHorizon = EventReadHorizon;
|
||||
|
||||
// Propagate the S3 HistoryRead concurrency bounds (arch-review 03/S3): bounded per-node fan-out
|
||||
// within a batch, the process-wide concurrent-batch limiter, and the per-request deadline. Defaults
|
||||
// (4 / 16 / 60 s) survive when the ServerHistorian section is absent.
|
||||
@@ -356,6 +359,12 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl
|
||||
/// </summary>
|
||||
internal bool HonorReturnBounds => _serverHistorianOptions.HonorReturnBounds;
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #694 — the <c>ServerHistorian:EventReadHorizon</c> <c>StartAsync</c> applies to the node manager (default
|
||||
/// <see cref="TimeSpan.Zero"/> = none). Exposed so the wiring is testable without booting the SDK.
|
||||
/// </summary>
|
||||
internal TimeSpan EventReadHorizon => _serverHistorianOptions.EventReadHorizon;
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #674 — what <c>StartAsync</c> wires onto the node manager for HistoryUpdate: the Primary gate (read from
|
||||
/// the <see cref="IHistoryUpdateRoleView"/> on every call; null when none is registered, i.e. allow), the
|
||||
|
||||
@@ -35,7 +35,9 @@
|
||||
"AllowUntrustedServerCertificate": false,
|
||||
"CaCertificatePath": null,
|
||||
"CallTimeout": "00:00:30",
|
||||
"MaxTieClusterOverfetch": 65536
|
||||
"MaxTieClusterOverfetch": 65536,
|
||||
"EventReadHorizon": "00:00:00",
|
||||
"_EventReadHorizonComment": "Gitea #694, opt-in. 00:00:00 = no horizon: HistoryReadEvents is exact to each read's window (on the Gateway a reverse/end-only events read proves 'nothing older' with one read down to 1970 - seconds on a large Runtime.dbo.Events). A positive span (e.g. 365.00:00:00) never reads events older than now minus it, in either direction - LOSSY: older events become invisible to HistoryReadEvents."
|
||||
},
|
||||
"ContinuousHistorization": {
|
||||
"_comment": "Continuous historization of driver (non-Galaxy) tag values: a crash-safe FasterLog outbox + recorder draining to the ServerHistorian gateway's WriteLiveValues. Disabled => no recorder is spawned. Requires ServerHistorian to be configured; the gateway connection (endpoint/key/TLS) is sourced from the ServerHistorian section, not here.",
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer;
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #694 item 2: the opt-in <c>ServerHistorian:EventReadHorizon</c> — HistoryReadEvents never reads events older than
|
||||
/// <c>now − horizon</c>. Off (<see cref="TimeSpan.Zero"/>, the default) a read is exact down to whatever its window says,
|
||||
/// which on the Gateway means a reverse / end-only read walked back with forward reads proves "nothing older" with one
|
||||
/// read down to the Unix epoch (#691) — ~3.8 s live on <c>Runtime.dbo.Events</c>, and an unfiltered one can hit the
|
||||
/// gateway's 30 s SQL timeout. A forward read whose StartTime is far in the past costs the same on that table. The
|
||||
/// horizon bounds both, at a stated price: events older than it are not returned, in either direction, on any backend.
|
||||
/// </summary>
|
||||
internal static class EventHorizon
|
||||
{
|
||||
/// <summary>
|
||||
/// Raises the EARLIER bound of a fresh events window to the horizon's floor (<c>now − horizon</c>, inclusive): the
|
||||
/// start of a forward <c>[start, end)</c> window, the exclusive lower end of a reverse <c>(end, start]</c> one.
|
||||
/// </summary>
|
||||
/// <param name="startUtc">The window start (the later bound of a reverse read).</param>
|
||||
/// <param name="endUtc">The window end (the exclusive earlier bound of a reverse read).</param>
|
||||
/// <param name="reverse">Whether the read is newest-first (<c>start > end</c>).</param>
|
||||
/// <param name="horizon">The horizon; non-positive = none (nothing changes).</param>
|
||||
/// <param name="nowUtc">Now.</param>
|
||||
/// <returns><c>false</c> when the whole window lies before the floor — the read is empty and must not be sent.</returns>
|
||||
public static bool TryClamp(ref DateTime startUtc, ref DateTime endUtc, bool reverse, TimeSpan horizon, DateTime nowUtc)
|
||||
{
|
||||
if (horizon <= TimeSpan.Zero) return true;
|
||||
var floor = nowUtc.Ticks - DateTime.MinValue.Ticks > horizon.Ticks
|
||||
? new DateTime(nowUtc.Ticks - horizon.Ticks, DateTimeKind.Utc)
|
||||
: new DateTime(0, DateTimeKind.Utc);
|
||||
if (floor.Ticks == 0) return true;
|
||||
|
||||
if (reverse)
|
||||
{
|
||||
if (startUtc < floor) return false; // (end, start] lies wholly before the floor
|
||||
var below = floor.AddTicks(-1); // exclusive lower end ⇒ the floor instant itself is read
|
||||
if (endUtc < below) endUtc = below;
|
||||
return true;
|
||||
}
|
||||
|
||||
if (endUtc <= floor) return false; // [start, end) lies wholly before the floor
|
||||
if (startUtc < floor) startUtc = floor;
|
||||
return true;
|
||||
}
|
||||
}
|
||||
@@ -552,6 +552,13 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
/// </summary>
|
||||
public bool HonorReturnBounds { get; set; } = true;
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #694: <c>ServerHistorian:EventReadHorizon</c> — HistoryReadEvents never reads events older than
|
||||
/// <c>now − EventReadHorizon</c> (<see cref="EventHorizon"/>). <see cref="TimeSpan.Zero"/> (the default) = no horizon:
|
||||
/// every read is exact down to its own window (a walked Gateway reverse read proves "nothing older" down to 1970).
|
||||
/// </summary>
|
||||
public TimeSpan EventReadHorizon { get; set; } = TimeSpan.Zero;
|
||||
|
||||
/// <summary>
|
||||
/// Maximum historized nodes served concurrently within one HistoryRead batch (arch-review 03/S3).
|
||||
/// Mirrors <c>ServerHistorianOptions.HistoryReadBatchParallelism</c>; the Host sets it at
|
||||
@@ -5339,6 +5346,19 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
|
||||
// Part 11: StartTime later than EndTime is a reverse read — events newest-first over (end, start].
|
||||
reverse = startUtc > endUtc;
|
||||
|
||||
// #694: the opt-in events horizon raises the window's earlier bound (the cursor then carries it). A window
|
||||
// wholly older than it is empty and is not sent (a forward [start, end) clamped past its end would be refused).
|
||||
if (!EventHorizon.TryClamp(ref startUtc, ref endUtc, reverse, EventReadHorizon, Clock.GetUtcNow().UtcDateTime))
|
||||
{
|
||||
results[handle.Index] = new SdkHistoryReadResult
|
||||
{
|
||||
StatusCode = StatusCodes.GoodNoData,
|
||||
HistoryData = new ExtensionObject(ProjectEvents([], selectClauses)),
|
||||
};
|
||||
errors[handle.Index] = ServiceResult.Good;
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// A reverse read on a backend that cannot read newest-first (the Gateway) is never sent there (#675 refused it
|
||||
@@ -6450,58 +6470,77 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
// "full" empty page was taken for a stalled tie cluster, and a MaxTieClusterOverfetch + 1 over-fetch stepped one
|
||||
// tick to return an EMPTY page with a continuation point — between every value. The lookahead row is what says
|
||||
// "more data follows": a page whose read found no row past it is the last, so the read no longer ends with an
|
||||
// empty trailing page either.
|
||||
// empty trailing page either. (RawPageLimit.Plan computes the request: page + boundary ties + 1.)
|
||||
var resumed = inboundCp is { Length: > 0 };
|
||||
var requestCap = readCap == 0
|
||||
? 0u
|
||||
: (uint)Math.Min(uint.MaxValue, (long)readCap + (resumed ? Math.Max(0, boundarySkip) : 0) + 1L);
|
||||
var boundaryTies = resumed ? Math.Max(0, boundarySkip) : 0;
|
||||
var want = readCap; // N, less the start bound on a bounded first page; 0 = all values
|
||||
|
||||
// HistoryRead is NOT under the node-manager Lock — awaiting the async source is safe.
|
||||
// A reverse window (explicit, resumed, or the end-only read above) goes to the backend only when it declares it
|
||||
// can read newest-first; otherwise it is walked back from bounded forward reads (#686 — the Gateway rejects
|
||||
// start > end, which failed the node and counted as a backend fault).
|
||||
// A cursor left inside an oversized tie cluster (#692) skips this read: every row it could return is a tie.
|
||||
// #694: a backend with a row ceiling (MaxRawValuesPerRead = L > 0, a strict gateway) sizes the page to it — the
|
||||
// read asks for page + boundary ties + lookahead ≤ L, and a full page carries a continuation point as always
|
||||
// (uncapped reads included). No room left after the boundary ties ⇒ the cursor sits in a tie cluster as large as
|
||||
// a read: straight to the over-fetch. See RawPageLimit.
|
||||
var source = HistorianDataSource;
|
||||
IReadOnlyList<DataValueSnapshot> samples = [];
|
||||
var backendFull = false;
|
||||
int pageLimit;
|
||||
if (!withinTieCluster)
|
||||
{
|
||||
HistorianRead sourceResult;
|
||||
if (startUtc > endUtc && !source.SupportsReverseRaw)
|
||||
var read = await RawPageLimit.ReadAsync(source, want, boundaryTies, async plan =>
|
||||
{
|
||||
var page = await ReverseRawReader.ReadAsync(
|
||||
source, tagname, startUtc, endUtc, requestCap, MaxTieClusterOverfetch,
|
||||
nowUtc, walkHint, ct)
|
||||
.ConfigureAwait(false);
|
||||
walkHint = page.Hint;
|
||||
sourceResult = new HistorianRead(page.Samples, ContinuationPoint: null);
|
||||
if (startUtc > endUtc && !source.SupportsReverseRaw)
|
||||
{
|
||||
var page = await ReverseRawReader.ReadAsync(
|
||||
source, tagname, startUtc, endUtc, plan.RequestCap, MaxTieClusterOverfetch,
|
||||
nowUtc, walkHint, ct, retryOnLearnedLimit: false)
|
||||
.ConfigureAwait(false);
|
||||
walkHint = page.Hint;
|
||||
return new HistorianRead(page.Samples, ContinuationPoint: null);
|
||||
}
|
||||
|
||||
return await source.ReadRawAsync(tagname, startUtc, endUtc, plan.RequestCap, ct).ConfigureAwait(false);
|
||||
}).ConfigureAwait(false);
|
||||
pageLimit = read.Plan.Limit;
|
||||
|
||||
if (read.Plan.NoRoom)
|
||||
{
|
||||
if (!resumed)
|
||||
throw new HistoryReadLimitExceededException(pageLimit,
|
||||
$"The backend's row ceiling ({pageLimit} values per read) leaves no room for a raw page of '{tagname}'.");
|
||||
withinTieCluster = true; // every row a read could return is a boundary tie already emitted
|
||||
}
|
||||
else
|
||||
{
|
||||
sourceResult = await source
|
||||
.ReadRawAsync(tagname, startUtc, endUtc, requestCap, ct)
|
||||
.ConfigureAwait(false);
|
||||
}
|
||||
readCap = read.Plan.PageCap;
|
||||
var sourceResult = read.Page!;
|
||||
backendFull = HistoryPaging.IsFullPage(sourceResult.Samples.Count, read.Plan.RequestCap);
|
||||
|
||||
backendFull = HistoryPaging.IsFullPage(sourceResult.Samples.Count, requestCap);
|
||||
// On a resume read, drop the boundary ties already returned on the prior page.
|
||||
samples = resumed
|
||||
? HistoryPaging.TrimBoundaryDuplicates(sourceResult.Samples, startUtc, boundarySkip)
|
||||
: sourceResult.Samples;
|
||||
|
||||
// On a resume read, drop the boundary ties already returned on the prior page.
|
||||
samples = resumed
|
||||
? HistoryPaging.TrimBoundaryDuplicates(sourceResult.Samples, startUtc, boundarySkip)
|
||||
: sourceResult.Samples;
|
||||
|
||||
// #691: a bounded read's data window excludes the end time (a sample there is the end bound). A backend whose
|
||||
// forward end is inclusive (the gateway's own) returns it as data; dropping it also means the data is drained.
|
||||
if (endBound is not null)
|
||||
{
|
||||
var inside = samples.Where(x => HistoryBounds.InDataWindow(x, endBound, resolution)).ToList();
|
||||
if (inside.Count != samples.Count)
|
||||
// #691: a bounded read's data window excludes the end time (a sample there is the end bound). A backend whose
|
||||
// forward end is inclusive (the gateway's own) returns it as data; dropping it also means the data is drained.
|
||||
if (endBound is not null)
|
||||
{
|
||||
samples = inside;
|
||||
backendFull = false;
|
||||
var inside = samples.Where(x => HistoryBounds.InDataWindow(x, endBound, resolution)).ToList();
|
||||
if (inside.Count != samples.Count)
|
||||
{
|
||||
samples = inside;
|
||||
backendFull = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
pageLimit = source.MaxRawValuesPerRead;
|
||||
}
|
||||
|
||||
// Oversized tie cluster: a resume read whose FULL backend read does not advance past the resume boundary
|
||||
// timestamp `startUtc` — every row it returned past the trimmed ties, the lookahead row included, still sits on
|
||||
@@ -6523,7 +6562,12 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
// lets us DETECT a cluster strictly larger than the bound (the absurd-burst backstop below).
|
||||
// Compute in uint so +1 can never wrap: Math.Max(1,…) also guards against a zero/negative
|
||||
// config value slipping through (Validate() warns but does not block startup).
|
||||
var overfetchCap = (uint)Math.Max(1, MaxTieClusterOverfetch) + 1u;
|
||||
// #694: on a backend with a row ceiling the bound is also at most L − 1, so the over-fetch (bound + 1 rows)
|
||||
// fits one read and an oversized cluster is refused here rather than by a gateway refusal.
|
||||
var overfetchBound = pageLimit > 0
|
||||
? Math.Min(Math.Max(1, MaxTieClusterOverfetch), Math.Max(1, pageLimit - 1))
|
||||
: Math.Max(1, MaxTieClusterOverfetch);
|
||||
var overfetchCap = (uint)overfetchBound + 1u;
|
||||
IReadOnlyList<DataValueSnapshot> cluster = (await HistorianDataSource
|
||||
.ReadRawAsync(tagname, startUtc, startUtc, overfetchCap, ct)
|
||||
.ConfigureAwait(false)).Samples;
|
||||
@@ -6537,12 +6581,13 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
// Absurd burst: more ties than we're willing to buffer in memory. Preserve today's loud-fail
|
||||
// for that node rather than over-fetch an unbounded cluster; the operator's remedy is a
|
||||
// larger ServerHistorian:MaxTieClusterOverfetch (or NumValuesPerNode).
|
||||
if (cluster.Count > MaxTieClusterOverfetch)
|
||||
if (cluster.Count > overfetchBound)
|
||||
{
|
||||
_logger.LogError(
|
||||
"OtOpcUaNodeManager: HistoryReadRaw tie cluster at {StartUtc:O} for tag '{Tagname}' has {Count} samples, " +
|
||||
"exceeding MaxTieClusterOverfetch={Max}; cannot page within it. Increase MaxTieClusterOverfetch.",
|
||||
startUtc, tagname, cluster.Count, MaxTieClusterOverfetch);
|
||||
"exceeding {Max} (MaxTieClusterOverfetch, or the backend's row ceiling less one); cannot page within it. " +
|
||||
"Increase MaxTieClusterOverfetch (or the HistorianGateway MaxValuesCeiling).",
|
||||
startUtc, tagname, cluster.Count, overfetchBound);
|
||||
errors[handle.Index] = StatusCodes.BadTooManyOperations;
|
||||
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadTooManyOperations };
|
||||
return;
|
||||
@@ -6550,8 +6595,15 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
|
||||
// Direction comes from the window itself (Part 11: start > end is a reverse read). A resumed reverse
|
||||
// read keeps start > end, because a reverse resume point is always strictly after endUtc.
|
||||
// #694: an uncapped read pages only at the backend's limit, so it slices at that page size — never 0 (a zero
|
||||
// slice hands back the same cursor forever; a limit that reverted to 0 under an uncapped cursor in a cluster
|
||||
// falls back to the cluster's own size).
|
||||
var sliceSize = numValuesPerNode > 0 ? numValuesPerNode
|
||||
: readCap > 0 ? readCap
|
||||
: pageLimit > 0 ? (uint)pageLimit
|
||||
: (uint)Math.Max(1, cluster.Count);
|
||||
HistoryPaging.SliceTieCluster(
|
||||
cluster.Count, boundarySkip, numValuesPerNode, startUtc, endUtc,
|
||||
cluster.Count, boundarySkip, sliceSize, startUtc, endUtc,
|
||||
out var sliceStart, out var sliceCount, out var nextStartUtc, out var nextSkip,
|
||||
reverse: reverse, resolutionTicks: Math.Max(1L, HistorianDataSource.TimestampResolutionTicks));
|
||||
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer;
|
||||
|
||||
/// <summary>How one HistoryRead-Raw page reads (<see cref="RawPageLimit.Plan"/>).</summary>
|
||||
/// <param name="Limit">The backend's <see cref="IHistorianDataSource.MaxRawValuesPerRead"/> the plan was made under (0 = none).</param>
|
||||
/// <param name="PageCap">The page: at most this many values are returned (0 = all values — only without a limit).</param>
|
||||
/// <param name="RequestCap">What the read asks for: page + the boundary ties it will trim + one lookahead row (0 = all).</param>
|
||||
/// <param name="NoRoom">The boundary ties alone fill a read — nothing can be read past them; serve the tie cluster.</param>
|
||||
internal readonly record struct RawPagePlan(int Limit, uint PageCap, uint RequestCap, bool NoRoom);
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #694: sizes a HistoryRead-Raw page to what the backend can return in one read
|
||||
/// (<see cref="IHistorianDataSource.MaxRawValuesPerRead"/> = L — a strict HistorianGateway's learned row ceiling, less
|
||||
/// its boundary-row allowance). With a limit the read — page P + boundary ties k + the #692 lookahead row — is at most L,
|
||||
/// so P = min(want, L − k − 1) (all values: P = L − k − 1); a full page carries a continuation point like any other, as
|
||||
/// Part 11 allows fewer values than <c>NumValuesPerNode</c> with one. Before #694 a cap above a strict gateway's ceiling
|
||||
/// (or an uncapped read of a window holding more) failed <c>BadTooManyOperations</c>.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// <b>Never a silent truncation.</b> The backend still answers a read in full or refuses it; only the server's page is
|
||||
/// smaller, and a page is judged full against what was asked for. A read the backend cannot serve at all (a tie
|
||||
/// cluster larger than one read) stays <c>BadTooManyOperations</c>.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>One re-read, planned again.</b> A refused read (<see cref="HistoryReadLimitExceededException"/>) is re-planned and
|
||||
/// re-issued ONCE when (a) the backend learned a different limit while it ran — the first read after start-up teaches
|
||||
/// the ceiling — or (b) it was refused because rows before the window crowded the budget
|
||||
/// (<see cref="HistoryReadLimitExceededException.PreWindowRows"/> m): the page is then shrunk by m so that the
|
||||
/// backend's look-back fits beside it. Anything else is the answer.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
internal static class RawPageLimit
|
||||
{
|
||||
/// <summary>Plans one page.</summary>
|
||||
/// <param name="want">The page the client asked for (N, less a start bound; 0 = all values).</param>
|
||||
/// <param name="boundaryTies">Ties at the resume timestamp already returned, which the read re-reads and trims (k).</param>
|
||||
/// <param name="limit">The backend's limit L (<= 0 = none).</param>
|
||||
/// <param name="sizeLimit">The limit to size the page by (L, or L − m after a crowded refusal); defaults to <paramref name="limit"/>.</param>
|
||||
/// <returns>The plan.</returns>
|
||||
public static RawPagePlan Plan(uint want, int boundaryTies, int limit, int? sizeLimit = null)
|
||||
{
|
||||
var k = Math.Max(0, boundaryTies);
|
||||
if (limit <= 0)
|
||||
return new RawPagePlan(0, want, want == 0 ? 0u : (uint)Math.Min(uint.MaxValue, (long)want + k + 1L), false);
|
||||
|
||||
var room = (long)(sizeLimit ?? limit) - k - 1;
|
||||
if (room < 1) return new RawPagePlan(limit, 0, 0, true);
|
||||
var page = want == 0 ? room : Math.Min(want, room);
|
||||
return new RawPagePlan(limit, (uint)page, (uint)(page + k + 1), false);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reads one page under the backend's current limit, re-planning and re-reading ONCE as described in the remarks.
|
||||
/// A <see cref="RawPagePlan.NoRoom"/> plan is returned without a read.
|
||||
/// </summary>
|
||||
/// <typeparam name="T">The page.</typeparam>
|
||||
/// <param name="source">The backend.</param>
|
||||
/// <param name="want">The page the client asked for (0 = all values).</param>
|
||||
/// <param name="boundaryTies">The boundary ties the read re-reads and trims.</param>
|
||||
/// <param name="read">One read with the given plan (it asks for <see cref="RawPagePlan.RequestCap"/>).</param>
|
||||
/// <returns>The page (null when <see cref="RawPagePlan.NoRoom"/>) and the plan it was read with.</returns>
|
||||
public static async Task<(T? Page, RawPagePlan Plan)> ReadAsync<T>(
|
||||
IHistorianDataSource source, uint want, int boundaryTies, Func<RawPagePlan, Task<T>> read)
|
||||
{
|
||||
var before = source.MaxRawValuesPerRead;
|
||||
var plan = Plan(want, boundaryTies, before);
|
||||
if (plan.NoRoom) return (default, plan);
|
||||
|
||||
HistoryReadLimitExceededException refused;
|
||||
try
|
||||
{
|
||||
return (await read(plan).ConfigureAwait(false), plan);
|
||||
}
|
||||
catch (HistoryReadLimitExceededException ex) when (source.MaxRawValuesPerRead is var now and > 0
|
||||
&& (now != before || ex.PreWindowRows > 0))
|
||||
{
|
||||
refused = ex; // re-read outside the catch: the second refusal is the answer
|
||||
}
|
||||
|
||||
var limit = source.MaxRawValuesPerRead;
|
||||
var shrink = Math.Max(0, refused.PreWindowRows);
|
||||
plan = Plan(want, boundaryTies, limit, (int)Math.Max(0L, (long)limit - shrink));
|
||||
// No room left once the crowding rows are allowed for: those rows are not (necessarily) the boundary ties, so this is
|
||||
// NOT "the cursor sits in a tie cluster" — the refusal is the answer (stepping the cursor through it a tick per page
|
||||
// would never end).
|
||||
// (A limit that changed under an un-crowded read leaves the boundary ties alone filling a read: that IS the tie
|
||||
// cluster at the cursor, served by the over-fetch.)
|
||||
if (plan.NoRoom)
|
||||
{
|
||||
if (shrink > 0) throw refused;
|
||||
return (default, plan);
|
||||
}
|
||||
return (await read(plan).ConfigureAwait(false), plan);
|
||||
}
|
||||
}
|
||||
@@ -58,25 +58,72 @@ internal static class ReverseRawReader
|
||||
/// <param name="ct">Cancellation.</param>
|
||||
/// <returns>The page, newest first, and the hint for the next page.</returns>
|
||||
/// <exception cref="HistoryReadLimitExceededException">One timestamp holds more than the tie budget.</exception>
|
||||
/// <remarks>
|
||||
/// Gitea #694: every segment read fits the backend's <see cref="IHistorianDataSource.MaxRawValuesPerRead"/> (a strict
|
||||
/// gateway's learned ceiling) — the walk's tie budget is bounded by it so its segments are sized for it, and a
|
||||
/// crowded re-read asking more is cut to it (and counts as truncated, never as complete). A walk that TEACHES the
|
||||
/// backend its limit (the first read after start-up) and is refused for it is re-run once with segments sized for the
|
||||
/// new limit — unless <paramref name="retryOnLearnedLimit"/> is false: the HistoryRead page path passes false, so the
|
||||
/// refusal reaches <see cref="RawPageLimit"/>, which shrinks the PAGE to the limit as well (re-running the walk for a
|
||||
/// page of N far above the limit cost thousands of segment reads). The bound lookups (a page of one) keep the re-run.
|
||||
/// The page size itself is never cut here: a short answer would read as the last page. A segment refused because rows
|
||||
/// just before it crowded the backend's budget (<see cref="HistoryReadLimitExceededException.PreWindowRows"/>) is
|
||||
/// re-read once that many rows smaller and counts as truncated when full. An uncapped walk (one forward read of the
|
||||
/// whole window) is never cut: it is answered in full or refused.
|
||||
/// </remarks>
|
||||
/// <param name="retryOnLearnedLimit">Re-run the walk once when it taught the backend its limit (see the remarks).</param>
|
||||
public static async Task<ReverseRawPage> ReadAsync(
|
||||
IHistorianDataSource source, string tagname, DateTime hi, DateTime lo, uint maxValues, int maxTieCluster,
|
||||
DateTime nowUtc, ReverseWalkHint hint, CancellationToken ct)
|
||||
DateTime nowUtc, ReverseWalkHint hint, CancellationToken ct, bool retryOnLearnedLimit = true)
|
||||
{
|
||||
var budget = Math.Max(1, maxTieCluster);
|
||||
var (rows, next) = await ReverseWalk.WalkAsync(
|
||||
async (start, end, cap, token) =>
|
||||
{
|
||||
var samples = (await source.ReadRawAsync(tagname, start, end, (uint)cap, token).ConfigureAwait(false)).Samples;
|
||||
return new ForwardSegmentRead<DataValueSnapshot>(samples, cap > 0 && samples.Count >= cap);
|
||||
},
|
||||
Ticks, tieOrder: null, hi, lo, maxValues, budget, source.TimestampResolutionTicks, nowUtc, hint,
|
||||
at => new HistoryReadLimitExceededException(budget,
|
||||
$"A reverse raw read of '{tagname}' at {at:O} found more than {budget} samples within one millisecond; the " +
|
||||
"backend reads forward only, so the latest of them cannot be selected. Increase " +
|
||||
"ServerHistorian:MaxTieClusterOverfetch."),
|
||||
ct)
|
||||
.ConfigureAwait(false);
|
||||
return new ReverseRawPage(rows, next.WidthTicks) { Hint = next };
|
||||
var before = source.MaxRawValuesPerRead;
|
||||
try
|
||||
{
|
||||
return await WalkAsync(before).ConfigureAwait(false);
|
||||
}
|
||||
catch (HistoryReadLimitExceededException) when (retryOnLearnedLimit && maxValues > 0
|
||||
&& source.MaxRawValuesPerRead is var now and > 0 && now != before)
|
||||
{
|
||||
// fall through (outside the catch): the walk taught the backend its limit — re-run it sized for that limit once.
|
||||
// Not an uncapped walk: it is one read of the whole window, which the limit does not change.
|
||||
}
|
||||
|
||||
return await WalkAsync(source.MaxRawValuesPerRead).ConfigureAwait(false);
|
||||
|
||||
async Task<ReverseRawPage> WalkAsync(int limit)
|
||||
{
|
||||
// A segment asks for at most its budget + 2 rows (ReverseWalk), so the budget is bounded 2 under the limit.
|
||||
var budget = Math.Max(1, maxTieCluster);
|
||||
if (limit > 0) budget = Math.Min(budget, Math.Max(1, limit - 2));
|
||||
var (rows, next) = await ReverseWalk.WalkAsync(
|
||||
async (start, end, cap, token) =>
|
||||
{
|
||||
var asked = limit > 0 && cap > limit ? limit : cap; // cap 0 (the whole window) is never cut
|
||||
IReadOnlyList<DataValueSnapshot> samples;
|
||||
try
|
||||
{
|
||||
samples = (await source.ReadRawAsync(tagname, start, end, (uint)asked, token).ConfigureAwait(false)).Samples;
|
||||
}
|
||||
catch (HistoryReadLimitExceededException ex) when (asked > 0 && ex.PreWindowRows > 0 && asked - ex.PreWindowRows >= 1)
|
||||
{
|
||||
// #694: rows just before the segment crowded the backend's budget — read it smaller; a full answer
|
||||
// then counts as truncated (the walk narrows), never as complete.
|
||||
asked -= ex.PreWindowRows;
|
||||
samples = (await source.ReadRawAsync(tagname, start, end, (uint)asked, token).ConfigureAwait(false)).Samples;
|
||||
}
|
||||
return new ForwardSegmentRead<DataValueSnapshot>(samples, asked > 0 && samples.Count >= asked);
|
||||
},
|
||||
Ticks, tieOrder: null, hi, lo, maxValues, budget, source.TimestampResolutionTicks, nowUtc, hint,
|
||||
at => new HistoryReadLimitExceededException(budget,
|
||||
$"A reverse raw read of '{tagname}' at {at:O} found more than {budget} samples within one millisecond; the " +
|
||||
"backend reads forward only, so the latest of them cannot be selected. Increase " +
|
||||
(budget < Math.Max(1, maxTieCluster)
|
||||
? "the backend's row ceiling (HistorianGateway Historian:ReadLimits:MaxValuesCeiling)."
|
||||
: "ServerHistorian:MaxTieClusterOverfetch.")),
|
||||
ct)
|
||||
.ConfigureAwait(false);
|
||||
return new ReverseRawPage(rows, next.WidthTicks) { Hint = next };
|
||||
}
|
||||
}
|
||||
|
||||
private static long Ticks(DataValueSnapshot s) => (s.SourceTimestampUtc ?? DateTime.MinValue).Ticks;
|
||||
|
||||
@@ -106,6 +106,18 @@ public sealed class ServerHistorianOptions
|
||||
/// </summary>
|
||||
public bool GatewayAtTimeFromRaw { get; init; } = true;
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #694 (opt-in): HistoryReadEvents never reads events older than <c>now − EventReadHorizon</c> — the earlier
|
||||
/// bound of every fresh events window is raised to it (both directions, every backend; a window wholly older is
|
||||
/// <c>GoodNoData</c>). Default <see cref="TimeSpan.Zero"/> = no horizon: every read is exact to its own window. It
|
||||
/// exists for the Gateway, where proving "no older events" costs a read down to the Unix epoch on
|
||||
/// <c>Runtime.dbo.Events</c> (~3.8 s live per walked reverse / end-only read; an unfiltered one can hit the
|
||||
/// gateway's 30 s SQL timeout) and a forward read from a far-past StartTime costs the same. ⚠️ Lossy by design:
|
||||
/// events older than the horizon are invisible to HistoryReadEvents while it is set. Negative = none (a
|
||||
/// <see cref="Validate"/> warning).
|
||||
/// </summary>
|
||||
public TimeSpan EventReadHorizon { get; init; } = TimeSpan.Zero;
|
||||
|
||||
/// <summary>
|
||||
/// Maximum number of historized nodes served CONCURRENTLY within one HistoryRead batch (arch-review
|
||||
/// 03/S3). The SDK's <c>HistoryRead*</c> override surface is synchronous, so each arm block-bridges
|
||||
@@ -196,6 +208,8 @@ public sealed class ServerHistorianOptions
|
||||
warnings.Add($"ServerHistorian:MaxConcurrentHistoryReadBatches is {MaxConcurrentHistoryReadBatches} — must be > 0; the process-wide HistoryRead batch limiter is disabled.");
|
||||
if (MaxTimeWeightedRawRows <= 0)
|
||||
warnings.Add($"ServerHistorian:MaxTimeWeightedRawRows is {MaxTimeWeightedRawRows} — must be > 0; time-weighted aggregate reads on the SqlServer/QuestDb backends stream an unbounded number of raw samples.");
|
||||
if (EventReadHorizon < TimeSpan.Zero)
|
||||
warnings.Add($"ServerHistorian:EventReadHorizon is {EventReadHorizon} — negative is treated as no horizon (00:00:00); HistoryReadEvents reads every window in full.");
|
||||
if (HistoryReadDeadline <= TimeSpan.Zero)
|
||||
warnings.Add($"ServerHistorian:HistoryReadDeadline is {HistoryReadDeadline} — must be > 0; HistoryRead requests run unbounded (no per-request deadline).");
|
||||
return warnings;
|
||||
|
||||
Reference in New Issue
Block a user