feat(historian): server-side continuation-point paging for HistoryRead-Raw
The Wonderware historian backend is single-shot — it returns up to NumValuesPerNode samples with a null continuation point — so paging is synthesised server-side, time-based, for the only count-capped arm (Raw): - A full page (count == NumValuesPerNode, NumValuesPerNode > 0) emits an opaque 16-byte continuation point and stores a resume cursor; a short page (or NumValuesPerNode == 0 "all values") emits none. - A resume read takes the stored cursor, reads the next page from the boundary forward, and emits a fresh CP only if that page is also full. - The resume cursor is tie-safe (HistoryPaging.ComputeResumeCursor / TrimBoundaryDuplicates): the next page resumes from the boundary timestamp INCLUSIVE and drops the head ties already returned, so samples sharing the boundary SourceTimestamp are neither duplicated nor skipped. Continuation points are bound to the OPC UA session via the SDK's ISession.SaveHistoryContinuationPoint / RestoreHistoryContinuationPoint store (SessionHistoryContinuationStore) — capped by ServerConfiguration. MaxHistoryContinuationPoints (default 100, oldest-evicted) and disposed on session close. releaseContinuationPoints is honoured via an override of HistoryReleaseContinuationPoints (the base dispatcher routes release-only reads there, never to the per-details arms). An unknown / evicted / released point resumes to BadContinuationPointInvalid. Processed and AtTime stay single-shot: neither details type carries a client count cap, so the single-shot backend returns the complete result in one read and there is no "full page" signal to page on (spec-conformant). Modified-value history remains out of scope. The pure paging decisions + CP store contract are unit-tested via HistoryPaging + InMemoryHistoryContinuationStore; the full multi-page round trip is driven end-to-end through the node manager with an in-memory store + a series-backed fake historian (the in-process harness is session-less).
This commit is contained in:
Joseph Doherty
@@ -0,0 +1,167 @@
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer;
|
||||
|
||||
/// <summary>
|
||||
/// The kind of variable-history read a continuation point resumes. Only the two count-capped,
|
||||
/// time-range arms page server-side (see <see cref="HistoryPaging"/>); AtTime is single-shot
|
||||
/// (no client count cap, so there is never a "full page" signal to page on) and never produces a
|
||||
/// continuation point, so it has no entry here.
|
||||
/// </summary>
|
||||
internal enum HistoryReadKind
|
||||
{
|
||||
/// <summary>HistoryRead-Raw — resumes via <see cref="IHistorianDataSource.ReadRawAsync"/>.</summary>
|
||||
Raw,
|
||||
|
||||
/// <summary>HistoryRead-Processed — resumes via <see cref="IHistorianDataSource.ReadProcessedAsync"/>.</summary>
|
||||
Processed,
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The server-side resume state stored behind an opaque continuation point for a single
|
||||
/// paged variable-history read. Captures exactly enough to continue the SAME logical read from
|
||||
/// where the previous page stopped: the read kind + tagname, the original (inclusive) end of the
|
||||
/// window, the next start of the window, and — for Processed — the aggregate + interval.
|
||||
/// <para>
|
||||
/// The boundary fields (<see cref="NextStartUtc"/> + <see cref="BoundarySkipCount"/>) encode a
|
||||
/// tie-safe resume cursor: the next page reads from <see cref="NextStartUtc"/> INCLUSIVE and
|
||||
/// then drops the first <see cref="BoundarySkipCount"/> samples whose SourceTimestamp equals
|
||||
/// <see cref="NextStartUtc"/>. This guarantees that samples sharing the page-boundary timestamp
|
||||
/// are neither re-returned (duplicate) nor skipped — see <see cref="HistoryPaging"/>.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// This record carries no SDK types so the whole paging decision surface is a pure, allocation-
|
||||
/// cheap value that unit tests can drive directly.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
/// <param name="Kind">Which variable-history arm this state resumes.</param>
|
||||
/// <param name="Tagname">The resolved historian tagname (NOT the NodeId) to read from.</param>
|
||||
/// <param name="NextStartUtc">
|
||||
/// Inclusive lower bound for the next page — the boundary timestamp the previous page stopped on.
|
||||
/// </param>
|
||||
/// <param name="EndUtc">The original (inclusive) upper bound of the read window; unchanged across pages.</param>
|
||||
/// <param name="BoundarySkipCount">
|
||||
/// How many samples whose SourceTimestamp equals <see cref="NextStartUtc"/> were already returned on
|
||||
/// prior pages and must be dropped from the head of the next page (tie de-dup).
|
||||
/// </param>
|
||||
/// <param name="NumValuesPerNode">The client's per-page cap; re-applied to every resumed page.</param>
|
||||
/// <param name="Aggregate">The aggregate for a Processed read; ignored for Raw.</param>
|
||||
/// <param name="IntervalTicks">The Processed bucketing interval in ticks; ignored for Raw.</param>
|
||||
internal sealed record HistoryContinuationState(
|
||||
HistoryReadKind Kind,
|
||||
string Tagname,
|
||||
DateTime NextStartUtc,
|
||||
DateTime EndUtc,
|
||||
int BoundarySkipCount,
|
||||
uint NumValuesPerNode,
|
||||
HistoryAggregateType Aggregate,
|
||||
long IntervalTicks);
|
||||
|
||||
/// <summary>
|
||||
/// Pure server-side continuation-point paging decisions for the count-capped variable-history arms
|
||||
/// (Raw / Processed). The backend (Wonderware sidecar) does NOT page — it returns up to
|
||||
/// <c>NumValuesPerNode</c> samples with a null continuation point — so paging is synthesised here,
|
||||
/// time-based:
|
||||
/// <list type="bullet">
|
||||
/// <item>A page that returns EXACTLY the requested cap (<c>NumValuesPerNode > 0</c>) MAY have
|
||||
/// more behind it ⇒ emit a continuation point.</item>
|
||||
/// <item>A short page (fewer than the cap) is the last page ⇒ no continuation point.</item>
|
||||
/// <item><c>NumValuesPerNode == 0</c> means "all values, no limit" (OPC UA Part 11) ⇒ never page;
|
||||
/// return everything in one shot.</item>
|
||||
/// </list>
|
||||
/// All methods are static + pure so they unit-test without a server, a session, or the SDK.
|
||||
/// </summary>
|
||||
internal static class HistoryPaging
|
||||
{
|
||||
/// <summary>
|
||||
/// Decide whether a just-returned page is "full" and therefore MAY be followed by more data —
|
||||
/// the signal to emit a continuation point. A page is full when the client asked for a finite
|
||||
/// cap (<paramref name="numValuesPerNode"/> > 0) and the backend returned exactly that many
|
||||
/// samples. A short page (or an unlimited <c>0</c> request) is terminal.
|
||||
/// </summary>
|
||||
/// <param name="returnedCount">The number of samples the backend returned for this page.</param>
|
||||
/// <param name="numValuesPerNode">The client's per-page cap; <c>0</c> means "no limit".</param>
|
||||
/// <returns><c>true</c> when a continuation point should be emitted; otherwise <c>false</c>.</returns>
|
||||
public static bool IsFullPage(int returnedCount, uint numValuesPerNode) =>
|
||||
numValuesPerNode > 0 && returnedCount >= numValuesPerNode;
|
||||
|
||||
/// <summary>
|
||||
/// Build the resume cursor (next-start + boundary skip count) from the last sample of a full
|
||||
/// page, tie-safe against samples that share the boundary SourceTimestamp.
|
||||
/// <para>
|
||||
/// The next page resumes from the LAST returned sample's SourceTimestamp <em>inclusive</em>
|
||||
/// (NOT advanced by a tick), and the returned <paramref name="boundarySkipCount"/> counts how
|
||||
/// many samples in the page already carry that exact boundary timestamp. Resuming inclusively
|
||||
/// + dropping that many head samples guarantees:
|
||||
/// <list type="bullet">
|
||||
/// <item>no sample is re-returned (the ones already emitted at the boundary are skipped), and</item>
|
||||
/// <item>no sample is skipped (any un-emitted ties at the boundary are still read because we
|
||||
/// start AT the boundary, not after it).</item>
|
||||
/// </list>
|
||||
/// A naive "+1 tick" advance would skip un-emitted ties; this carry-offset strategy does not.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
/// <param name="page">The page just returned (chronological, non-empty — guaranteed by the caller,
|
||||
/// which only pages a full page and a full page implies <c>NumValuesPerNode > 0</c> samples).</param>
|
||||
/// <param name="nextStartUtc">The boundary timestamp the next page resumes from (inclusive).</param>
|
||||
/// <param name="boundarySkipCount">How many head samples at <paramref name="nextStartUtc"/> the next
|
||||
/// page must drop (samples already emitted at the boundary timestamp).</param>
|
||||
public static void ComputeResumeCursor(
|
||||
IReadOnlyList<DataValueSnapshot> page,
|
||||
out DateTime nextStartUtc,
|
||||
out int boundarySkipCount)
|
||||
{
|
||||
// The boundary is the last returned sample's SourceTimestamp. A sample whose SourceTimestamp is
|
||||
// null (Bad/unset) cannot anchor a time cursor; fall back to MinValue so the next read covers the
|
||||
// whole remaining window rather than silently dropping data — duplicates are then de-duped by the
|
||||
// skip count below.
|
||||
var last = page[^1];
|
||||
nextStartUtc = last.SourceTimestampUtc ?? DateTime.MinValue;
|
||||
|
||||
// Count how many trailing samples in THIS page share the boundary timestamp — those are the ties
|
||||
// already emitted at the boundary that the next page must drop from its head.
|
||||
var skip = 0;
|
||||
for (var i = page.Count - 1; i >= 0; i--)
|
||||
{
|
||||
if ((page[i].SourceTimestampUtc ?? DateTime.MinValue) == nextStartUtc) skip++;
|
||||
else break;
|
||||
}
|
||||
|
||||
boundarySkipCount = skip;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Drop the first <paramref name="boundarySkipCount"/> samples of a freshly-read resumed page
|
||||
/// whose SourceTimestamp equals the boundary <paramref name="boundaryUtc"/> — the ties already
|
||||
/// emitted on the previous page. Samples past the boundary timestamp are always kept (only the
|
||||
/// exact-boundary head is trimmed), so a backend that returns fewer boundary ties than expected
|
||||
/// (data pruned between pages) still yields a correct, monotonic result.
|
||||
/// </summary>
|
||||
/// <param name="resumedPage">The page returned by the resumed backend read (chronological).</param>
|
||||
/// <param name="boundaryUtc">The boundary timestamp the resume read started at (inclusive).</param>
|
||||
/// <param name="boundarySkipCount">How many head samples at <paramref name="boundaryUtc"/> to drop.</param>
|
||||
/// <returns>The page with the already-emitted boundary ties trimmed from the head.</returns>
|
||||
public static IReadOnlyList<DataValueSnapshot> TrimBoundaryDuplicates(
|
||||
IReadOnlyList<DataValueSnapshot> resumedPage,
|
||||
DateTime boundaryUtc,
|
||||
int boundarySkipCount)
|
||||
{
|
||||
if (boundarySkipCount <= 0 || resumedPage.Count == 0) return resumedPage;
|
||||
|
||||
var dropped = 0;
|
||||
var i = 0;
|
||||
while (i < resumedPage.Count
|
||||
&& dropped < boundarySkipCount
|
||||
&& (resumedPage[i].SourceTimestampUtc ?? DateTime.MinValue) == boundaryUtc)
|
||||
{
|
||||
i++;
|
||||
dropped++;
|
||||
}
|
||||
|
||||
if (i == 0) return resumedPage;
|
||||
// Slice off the trimmed head; return a copy so the caller owns a plain list (no SDK coupling).
|
||||
var trimmed = new List<DataValueSnapshot>(resumedPage.Count - i);
|
||||
for (var j = i; j < resumedPage.Count; j++) trimmed.Add(resumedPage[j]);
|
||||
return trimmed;
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user