dohertj2/lmxopcua
1
0
Fork 0
Code Issues 32 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(historian): #689 size reverse-walk segments from the page, not the tie budget

Browse Source 
ReverseRawReader aimed every segment at half of MaxTieClusterOverfetch
(~32k rows by default) and carried that width page to page, so a paged
'latest N' read on a dense Gateway tag fetched ~32,770 rows per page
(2-5 s live) to return N. The walk now lives in a generic ReverseWalk
core: segments are aimed at ~N+8 rows (page budget 2N+16), a discovery
segment (first without a hint / after an empty one) may read 1,024, and
the tie budget is only the ceiling a single-timestamp cluster escalates
to. A truncated read never re-enters the stretch below its newest row
until the gap above it is bisected. Tests: per-page row counts on a
dense tag (node manager + walk), randomized exactness fuzz (walk and
node-manager paging), dense-tag-below-a-long-gap read bound.
This commit is contained in:
Joseph Doherty
2026-10-12 16:25:07 -04:00
parent 71d031fcd0
commit d33992034d
8 changed files with 663 additions and 183 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/ReverseWalk.cs
+297
View File
@@ -0,0 +1,297 @@
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer;
/// <summary>What one walked-back page tells the next page of the same read (carried in the continuation point).</summary>
/// <param name="WidthTicks">The segment width the walk ended on, aimed at the page's density — the next page's first width (0 = none).</param>
internal readonly record struct ReverseWalkHint(long WidthTicks);
/// <summary>One forward read of a segment: the rows (ascending by time) and whether a cap cut the read short.</summary>
/// <typeparam name="T">Row type.</typeparam>
/// <param name="Rows">Rows in ascending time order (ties in any order).</param>
/// <param name="Truncated">A cap stopped the read — more rows may follow the last one.</param>
internal readonly record struct ForwardSegmentRead<T>(IReadOnlyList<T> Rows, bool Truncated);
/// <summary>
/// Gitea #686/#689: the walk-back that serves a Part 11 reverse read — <c>(lo, hi]</c> newest-first, capped — from a
/// backend that can only read FORWARD. Shared by raw values (<see cref="ReverseRawReader"/>) and events; the row type
/// only has to expose a timestamp.
/// </summary>
/// <remarks>
/// <para>
/// <b>Uncapped</b> (<c>want = 0</c>): ONE forward read of the whole window, filtered and reversed in memory.
/// </para>
/// <para>
/// <b>Capped: walk back in segments.</b> Starting at <c>hi</c>, read one segment <c>(segLo, segHi]</c> FORWARD, keep
/// it newest-first, and step to the next older segment until the cap is met or <c>lo</c> is reached. The first
/// segment is <see cref="InitialSegment"/> wide, or the width the previous page of the same read ended on (carried in
/// the continuation point). After each read the next width is re-aimed from the density just observed at about half
/// the PAGE budget <c>2N + 16</c> (≈ N + 8 rows), growing at most ×4 per step (#689 — it used to aim at half the tie
/// budget, ~32k rows, whatever N was) — so a live tag's "latest N" is usually one read of about N rows, and sparse
/// history is found in a few dozen tiny reads.
/// </para>
/// <para>
/// <b>Read sizes.</b> A segment read asks for its budget + 2 rows (room for a row an AVEVA read fabricates at its query
/// start, and one to see past the segment). The budget is the page budget, except a <b>discovery</b> segment — the
/// first of a page with no carried width, or one after an empty segment (whose ×4 growth may overshoot into dense
/// data) — which may read up to <see cref="DiscoveryRows"/>. The tie budget <c>M = max(1, MaxTieClusterOverfetch)</c>
/// is only a CEILING: a single timestamp holding more rows than the segment budget escalates to <c>M</c>.
/// </para>
/// <para>
/// <b>Never answer from a truncated read.</b> Rows stream ascending, so a read is complete for its segment when it was
/// not cut short OR its last row is already past <c>segHi</c>. An incomplete read is discarded: when rows BEFORE the
/// segment (a neighbouring tie cluster inside the alignment slack, a fabricated row) crowded it out it is re-read once
/// with room for them (at most twice the budget + 4); otherwise the segment is narrowed — to the density the truncated
/// read showed, and never again below the newest row it saw (<c>known</c>: rows exist there, so walking past it before
/// finding what lies above it would re-read the same dense stretch over and over; the gap above it is bisected
/// instead). Only a single timestamp holding more than <c>M</c> rows — a one-resolution segment that still does not
/// fit at the tie budget — is refused.
/// </para>
/// <para>
/// <b>Edges.</b> A segment is requested as <c>[⌊segLo⌋, ⌈segHi + 1 tick⌉]</c>, both rounded outward to the millisecond
/// (the gateway's / AVEVA's resolution), and filtered to <c>(segLo, segHi]</c> on the rows' own timestamps — exact
/// whether the backend's forward end is exclusive or inclusive and whether it truncates its bounds; a row fabricated at
/// the (millisecond) query start can never land inside the segment (#687). Adjacent segments share no timestamp.
/// </para>
/// <para>
/// <b>Ties</b> come back in the backend's forward order reversed, or — with a tie order given (events, whose gateway
/// order within a timestamp is arbitrary) — sorted by it and reversed, so a resumed page's boundary-tie skip lines up.
/// </para>
/// <para>
/// <b>Bounds.</b> The walk never asks for data after now + <see cref="LookAhead"/> nor before <see cref="Floor"/> (the
/// Unix epoch — AVEVA holds nothing earlier). A walked-back read returns only rows strictly after the epoch and no
/// later than now + one day.
/// </para>
/// </remarks>
internal static class ReverseWalk
{
/// <summary>The first segment's width when no hint is carried.</summary>
internal static readonly TimeSpan InitialSegment = TimeSpan.FromMinutes(1);
/// <summary>The walk's exclusive lower limit.</summary>
internal static readonly DateTime Floor = DateTime.UnixEpoch;
/// <summary>How far past "now" the walk starts at most.</summary>
internal static readonly TimeSpan LookAhead = TimeSpan.FromDays(1);
/// <summary>Slack added to a page's row budget (#689): room for density variation between segments.</summary>
internal const int PageSlack = 16;
/// <summary>The most rows a discovery segment (see the remarks) may read before it counts as truncated (#689).</summary>
internal const int DiscoveryRows = 1024;
/// <summary>Request bounds are rounded outward to this (the gateway / AVEVA timestamp resolution).</summary>
private const long Alignment = TimeSpan.TicksPerMillisecond;
/// <summary>The rows a segment is sized for: <c>2N + </c><see cref="PageSlack"/>, never above the tie budget.</summary>
/// <param name="want">The page cap N (&gt; 0).</param>
/// <param name="tieBudget">The tie budget M (&gt;= 1).</param>
/// <returns>The page budget.</returns>
internal static int PageBudget(uint want, int tieBudget) => (int)Math.Clamp(2L * want + PageSlack, 1L, Math.Max(1, tieBudget));
/// <summary>Reads <c>(lo, hi]</c> newest-first from forward reads.</summary>
/// <typeparam name="T">Row type.</typeparam>
/// <param name="read">One forward read <c>[start, end]</c> asking for at most <c>cap</c> rows (<c>0</c> = all).</param>
/// <param name="ticksOf">A row's timestamp, in ticks.</param>
/// <param name="tieOrder">Order within one timestamp (null = the backend's forward order).</param>
/// <param name="hi">The later (inclusive) bound.</param>
/// <param name="lo">The earlier (exclusive) bound.</param>
/// <param name="want">Cap; <c>0</c> = every row in the window (one forward read).</param>
/// <param name="tieBudget">The tie budget M (values below 1 count as 1).</param>
/// <param name="resolutionTicks">The backend's timestamp resolution (&gt;= 1).</param>
/// <param name="nowUtc">Now, for the look-ahead clamp.</param>
/// <param name="hint">What the previous page of the same read carried (default = none).</param>
/// <param name="tooManyAtOneTimestamp">Builds the exception for a timestamp holding more than M rows.</param>
/// <param name="ct">Cancellation.</param>
/// <returns>At most <paramref name="want"/> rows, newest first, and the hint for the next page.</returns>
public static async Task<(List<T> Rows, ReverseWalkHint Next)> WalkAsync<T>(
Func<DateTime, DateTime, int, CancellationToken, Task<ForwardSegmentRead<T>>> read,
Func<T, long> ticksOf,
IComparer<T>? tieOrder,
DateTime hi, DateTime lo, uint want, int tieBudget, long resolutionTicks, DateTime nowUtc,
ReverseWalkHint hint, Func<DateTime, Exception> tooManyAtOneTimestamp, CancellationToken ct)
{
var resolution = Math.Max(1L, resolutionTicks);
var granularity = Math.Max(Alignment, resolution);
var budget = Math.Max(1, tieBudget);
var result = new List<T>();
var horizon = nowUtc.Ticks > DateTime.MaxValue.Ticks - LookAhead.Ticks ? DateTime.MaxValue.Ticks : nowUtc.Ticks + LookAhead.Ticks;
var hiTicks = Math.Min(hi.Ticks, horizon);
var stopTicks = Math.Max(lo.Ticks, Floor.Ticks);
if (hiTicks <= stopTicks) return (result, hint);
if (want == 0)
{
// One forward read of the whole window, reversed — the uncapped forward read's own cost and bound.
var all = (await read(Utc(AlignDown(stopTicks, granularity)), Utc(AlignUp(hiTicks + 1, granularity)), 0, ct)
.ConfigureAwait(false)).Rows;
AppendNewestFirst(all, ticksOf, tieOrder, stopTicks, hiTicks, result, int.MaxValue);
return (result, hint);
}
var maxWidth = DateTime.MaxValue.Ticks;
var discovering = hint.WidthTicks <= 0;
var width = discovering ? Math.Max(resolution, InitialSegment.Ticks) : Math.Clamp(hint.WidthTicks, resolution, maxWidth);
var pageBudget = PageBudget(want, budget);
var discoveryBudget = Math.Min(budget, Math.Max(pageBudget, DiscoveryRows));
// `known` > 0: a truncated read saw rows up to this timestamp (below segHi), and nothing is known of (known, segHi].
// Segments stay above it (bisecting the gap) until that stretch is read; then `knownWidth` (the density the
// truncated read showed) sizes the segment that ends on it.
var known = 0L;
var knownWidth = 0L;
var segHi = hiTicks;
while (segHi > stopTicks && result.Count < want)
{
var w = width;
if (known > 0) w = Math.Min(w, Math.Max(1, (segHi - known) / 2));
var segLo = segHi - stopTicks <= w ? stopTicks : segHi - w;
var start = Utc(AlignDown(segLo, granularity));
var end = Utc(AlignUp(segHi + 1, granularity));
var segBudget = discovering ? discoveryBudget : pageBudget;
var cap = segBudget + 2;
IReadOnlyList<T> rows;
var narrowed = false;
while (true)
{
ct.ThrowIfCancellationRequested();
var segment = await read(start, end, cap, ct).ConfigureAwait(false);
rows = segment.Rows;
if (!segment.Truncated || rows.Count == 0 || ticksOf(rows[^1]) > segHi) break; // complete for (segLo, segHi]
var leading = 0;
var inner = 0;
var firstInner = 0L;
var lastInner = 0L;
foreach (var r in rows)
{
var ts = ticksOf(r);
if (ts <= segLo) leading++;
else if (ts <= segHi)
{
if (inner++ == 0) firstInner = ts;
lastInner = ts;
}
}
// Crowded out by rows before the segment, with the segment itself possibly within budget: re-read once
// with room for them.
var roomier = leading + segBudget + 2;
if (leading > 0 && inner <= segBudget && roomier > cap && roomier <= 2 * segBudget + 4)
{
cap = roomier;
continue;
}
if (segHi - segLo <= resolution)
{
// A one-resolution segment still over its budget: one timestamp's tie cluster. Escalate to the tie
// budget M before refusing — a cluster of up to M rows is served (#686).
if (segBudget < budget)
{
segBudget = budget;
cap = segBudget + 2;
continue;
}
throw tooManyAtOneTimestamp(Utc(segHi));
}
// Narrow — the segment shrinks every time, so this terminates.
if (inner > 0 && lastInner == segHi)
{
width = resolution; // a tie cluster at the top of the segment: read that one timestamp
}
else if (inner > 0)
{
// Rows exist at lastInner; what lies above it is unknown. Aim at about half the page budget at the
// density the truncated rows showed (their span, not the segment's — it may open on an empty stretch).
var aimed = inner > 1 && lastInner > firstInner
? (long)Math.Min((double)(lastInner - firstInner) * (pageBudget / 2.0) / (inner - 1), maxWidth)
: resolution;
known = lastInner;
knownWidth = Math.Max(resolution, aimed);
width = Math.Max(resolution, Math.Min(aimed, segHi - segLo - 1));
}
else
{
width = Math.Max(resolution, (segHi - segLo) / 4); // only rows before the segment: shrink it
}
discovering = false;
narrowed = true;
break;
}
if (narrowed) continue; // re-read the same segHi, narrower
var count = AppendNewestFirst(rows, ticksOf, tieOrder, segLo, segHi, result, (int)Math.Min(int.MaxValue, want));
var grown = width > maxWidth / 4 ? maxWidth : width * 4;
if (count == 0)
{
// ×4 growth may overshoot into dense data; above a known row, bisect the rest of the gap instead.
width = known > 0 ? maxWidth : grown;
discovering = true;
}
else
{
// Re-aim at about half the page budget from the density just seen.
width = (long)Math.Clamp((double)(segHi - segLo) * Math.Max(1.0, pageBudget / 2.0) / count, resolution, grown);
discovering = false;
known = 0;
}
segHi = segLo;
if (known > 0 && segHi <= known)
{
// Everything above the newest row the truncated read saw is read: the next segment ends on it, at its density.
known = 0;
width = knownWidth;
discovering = false;
}
}
return (result, new ReverseWalkHint(width));
}
/// <summary>Appends the rows in <c>(segLo, segHi]</c> newest-first (until <paramref name="limit"/>) and counts them.</summary>
private static int AppendNewestFirst<T>(
IReadOnlyList<T> rows, Func<T, long> ticksOf, IComparer<T>? tieOrder, long segLo, long segHi, List<T> result, int limit)
{
IReadOnlyList<T> ordered = rows;
if (tieOrder is not null)
{
var copy = rows.ToList();
copy.Sort((a, b) =>
{
var c = ticksOf(a).CompareTo(ticksOf(b));
return c != 0 ? c : tieOrder.Compare(a, b);
});
ordered = copy;
}
var count = 0;
for (var i = ordered.Count - 1; i >= 0; i--)
{
var ts = ticksOf(ordered[i]);
if (ts <= segLo || ts > segHi) continue;
count++;
if (result.Count < limit) result.Add(ordered[i]);
}
return count;
}
private static DateTime Utc(long ticks) => new(ticks, DateTimeKind.Utc);
private static long AlignDown(long ticks, long step) => ticks - ticks % step;
private static long AlignUp(long ticks, long step)
{
var down = AlignDown(ticks, step);
if (down == ticks) return ticks;
return down > DateTime.MaxValue.Ticks - step ? DateTime.MaxValue.Ticks : down + step;
}
}