53978ca638
#694 item 3 — historiangw#27 is fixed and now live-verified through OtOpcUa: - ServerHistorian:Gateway:EnableNativeTimeWeighted defaults to true (false is the kill switch for a gateway older than 9f711c1). TimeAverage / Total are served and advertised again (12 aggregates). - The gateway sends the interval as max(1, round(ms)) and AVEVA clips its last interval at the request end (live). NativeAggregatePlan therefore sends a multi-interval grid only for whole-ms intervals; a fractional interval is read one interval per request (at most 1,000, else BadTooManyOperations), and every one-interval request goes out rounded UP to whole ms so AVEVA answers one clipped row. Live, a 3.0004 s clipped interval sent at its own width lost its last 0.4 ms (Total 123 vs 123.0172). #746 — HistorianGateway Contracts/Client 1.0.0 surface: - Alarm history: GatewayAlarmHistorianWriter sends SendEvents batches of <=200. A failed per-event result is re-sent alone via SendEvent (classified as before); a refused batch (InvalidArgument etc.) goes event by event so a poison event dead-letters alone; a systemic fault retries the rest without sending more; Unimplemented (older gateway) falls back to SendEvent for good. IHistorianGatewayClient.SendEventsAsync defaults to Unimplemented. - Events reads: a capped page sends cap + 1 as ReadEventsRequest.max_events, so the gateway completes the stream instead of the client cancelling it. An ambiguous capped answer (foreign rows filtered out of the server's cut) or a refused cap is re-read uncapped with the existing EventDrainBudget drain. Tests: GatewayServerCappedEventsTests, GatewayAlarmWriterBatchTests, GatewayNativeWholeMillisecondGridTests (gateway-faithful rounding fake), gate tests flipped to the new default; every new test mutation-checked (each mutation of the production change fails at least one). Live harness AvevaGatewayHistorianFollowupsLiveGateTests (read-only) for #694/#700/#746. Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
160 lines
11 KiB
C#
160 lines
11 KiB
C#
using ZB.MOM.WW.HistorianGateway.Contracts.Grpc;
|
|
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
|
using ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Mapping;
|
|
using ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common;
|
|
|
|
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway;
|
|
|
|
/// <summary>
|
|
/// Serves a processed read of the AVEVA-native <c>TimeAverage</c> / <c>Total</c> on the SQL backends' interval grid
|
|
/// (<see cref="BucketPlan"/>), in either direction: forward — window <c>[start, end)</c>, intervals stepping forward
|
|
/// from <c>start</c>, the clipped interval the LATEST, each stamped with its earlier edge, oldest first; reverse — window <c>(end, start]</c>, intervals stepping back from <c>start</c>, the clipped interval the EARLIEST,
|
|
/// each stamped with its later edge, newest first. The gateway refuses <c>start > end</c> and <c>interval <= 0</c>,
|
|
/// so every request it sees is forward with a positive interval.
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// <para>
|
|
/// <b>Forward requests on the plan's grid.</b> AVEVA anchors a native read's interval grid at the request start (live,
|
|
/// 2026-10-12: rows at <c>start + j·interval</c>, plus one extra row for the interval before it). So the plan's
|
|
/// whole intervals are requested as ONE forward read spanning exactly them with the interval <c>I</c>, and the
|
|
/// clipped interval (when the window is not a whole number of intervals) as a second forward read over exactly its
|
|
/// span with its own width as the interval — AVEVA never averages time outside the window (before it on a reverse
|
|
/// read, past <c>EndTime</c> on a forward one). An interval wider than the window is sent as the window, and
|
|
/// <c>interval <= 0</c> as one interval over the whole window (it used to reach the gateway as 0 and be refused).
|
|
/// </para>
|
|
/// <para>
|
|
/// <b>Soundness of reverse.</b> A time-weighted average / integral over <c>[a, b)</c> and over <c>(a, b]</c> integrate
|
|
/// the same function over sets differing by two instants, so a forward native row IS the value of the reverse
|
|
/// interval with the same edges. The one place the half-open side could matter is AVEVA's status for an interval
|
|
/// whose only non-good sample lies exactly on an edge — accepted, as on every native row.
|
|
/// </para>
|
|
/// <para>
|
|
/// <b>Mapping.</b> Each row is mapped as before (<see cref="SampleMapper.ToAggregateSnapshot"/>: Uncertain keeps its
|
|
/// value, no data ⇒ <c>BadNoData</c>, Calculated bits), placed by its interval start (else end), restamped with
|
|
/// <see cref="BucketPlan.BucketTimestamp"/>, and the clipped interval carries the Partial bit unless Bad (Part 13
|
|
/// §5.3.3.2 Case 3, as the SQL backends' time-weighted aggregates — which additionally mark the intervals the data
|
|
/// starts/ends inside; AVEVA's rows carry no data range, so those are not reproduced). An interval AVEVA answers no
|
|
/// row for is absent (not a fabricated <c>BadNoData</c>, unlike the SQL backends).
|
|
/// </para>
|
|
/// <para>
|
|
/// <b>Whole milliseconds only on a shared grid.</b> The gateway sends AVEVA the interval in whole milliseconds
|
|
/// (<c>max(1, round(ms))</c>), and AVEVA clips its last interval at the request's end (live, lmxopcua#694, after
|
|
/// historiangw#27: a 0.4 ms window answers one row over exactly that 0.4 ms — <c>Integral</c> = value·0.0004 s — and a
|
|
/// 1.4 ms window read at "1.4 ms" answers 1 ms rows). So a grid of several intervals is sent as one request only when
|
|
/// the interval is a whole number of milliseconds; otherwise every interval is its own request over exactly its span
|
|
/// (<see cref="MaxPerIntervalRequests"/> at most — more is <see cref="HistoryReadLimitExceededException"/>). A request
|
|
/// for ONE interval (the clipped one, a window narrower than one interval, or each of those per-interval requests) is
|
|
/// sent with its width rounded UP to whole milliseconds as the interval, so AVEVA answers exactly one row, clipped at
|
|
/// the request's end: sent at its own width, a 3.0004 s interval reached AVEVA as 3,000 ms and answered a 3 s row plus
|
|
/// a 0.4 ms one (live, lmxopcua#694 — the clipped interval's Total lost the 0.4 ms).
|
|
/// </para>
|
|
/// </remarks>
|
|
internal static class NativeAggregatePlan
|
|
{
|
|
/// <summary>
|
|
/// The most intervals a native read whose interval is not a whole number of milliseconds may cover: each is its own
|
|
/// gateway request.
|
|
/// </summary>
|
|
public const int MaxPerIntervalRequests = 1000;
|
|
|
|
/// <summary>One forward native request and the ascending bucket index of its first interval.</summary>
|
|
/// <param name="StartUtc">Forward request start (inclusive).</param>
|
|
/// <param name="EndUtc">Forward request end.</param>
|
|
/// <param name="Interval">The request's interval (> 0).</param>
|
|
/// <param name="FirstBucket">Ascending plan bucket of the request's first interval.</param>
|
|
/// <param name="BucketCount">Number of plan buckets the request covers.</param>
|
|
internal readonly record struct Request(DateTime StartUtc, DateTime EndUtc, TimeSpan Interval, long FirstBucket, long BucketCount);
|
|
|
|
/// <summary>The forward requests serving a plan: the whole intervals first, then the clipped one (if any).</summary>
|
|
/// <param name="plan">A plan with at least one bucket.</param>
|
|
/// <returns>One or two forward requests.</returns>
|
|
/// <exception cref="HistoryReadLimitExceededException">
|
|
/// The interval is not a whole number of milliseconds and the window holds more than
|
|
/// <see cref="MaxPerIntervalRequests"/> intervals.
|
|
/// </exception>
|
|
public static IReadOnlyList<Request> Requests(BucketPlan plan)
|
|
{
|
|
var interval = TimeSpan.FromTicks(plan.IntervalTicks);
|
|
var n = plan.BucketCount;
|
|
if (n > 1 && plan.IntervalTicks % TimeSpan.TicksPerMillisecond != 0)
|
|
{
|
|
// The gateway would round the interval to whole ms and AVEVA would answer on that grid, not the plan's: read
|
|
// each interval over exactly its span instead (one interval per request is exact at any width).
|
|
if (n > MaxPerIntervalRequests)
|
|
throw new HistoryReadLimitExceededException(MaxPerIntervalRequests,
|
|
$"A native TimeAverage/Total read whose interval is not a whole number of milliseconds is read one " +
|
|
$"interval per gateway request, at most {MaxPerIntervalRequests}; this window holds {n}. Use a whole-millisecond " +
|
|
"ProcessingInterval or a shorter window.");
|
|
var each = new List<Request>((int)n);
|
|
for (long i = 0; i < n; i++)
|
|
{
|
|
var lo = new DateTime(plan.LowerTicks(i), DateTimeKind.Utc);
|
|
var hi = new DateTime(plan.UpperTicks(i), DateTimeKind.Utc);
|
|
each.Add(new Request(lo, hi, OneInterval(hi - lo), i, 1));
|
|
}
|
|
return each;
|
|
}
|
|
var clipped = plan.IsTruncated(0) ? 0L : plan.IsTruncated(n - 1) ? n - 1 : -1L;
|
|
if (clipped < 0) return [new Request(plan.StartUtc, plan.EndUtc, interval, 0, n)];
|
|
|
|
var clipLo = new DateTime(plan.LowerTicks(clipped), DateTimeKind.Utc);
|
|
var clipHi = new DateTime(plan.UpperTicks(clipped), DateTimeKind.Utc);
|
|
var requests = new List<Request>(2);
|
|
if (n > 1)
|
|
requests.Add(clipped == 0
|
|
? new Request(clipHi, plan.EndUtc, interval, 1, n - 1) // reverse: the clipped one is the earliest
|
|
: new Request(plan.StartUtc, clipLo, interval, 0, n - 1)); // forward: the clipped one is the latest
|
|
requests.Add(new Request(clipLo, clipHi, OneInterval(clipHi - clipLo), clipped, 1));
|
|
return requests;
|
|
}
|
|
|
|
/// <summary>
|
|
/// The interval to send for a request covering exactly one interval of <paramref name="width"/>: the width rounded UP
|
|
/// to whole milliseconds (at least 1 ms), which the gateway sends unchanged, so AVEVA's single interval spans the whole
|
|
/// request and is clipped at its end — never a whole-ms interval plus a sub-ms remainder row.
|
|
/// </summary>
|
|
/// <param name="width">The request's span (> 0).</param>
|
|
/// <returns>The interval to send.</returns>
|
|
internal static TimeSpan OneInterval(TimeSpan width)
|
|
{
|
|
var ms = Math.Max(1, (width.Ticks + TimeSpan.TicksPerMillisecond - 1) / TimeSpan.TicksPerMillisecond);
|
|
return TimeSpan.FromTicks(ms * TimeSpan.TicksPerMillisecond);
|
|
}
|
|
|
|
/// <summary>
|
|
/// Maps a request's rows onto the plan's buckets: rows outside the request window (AVEVA's pre-window row)
|
|
/// are dropped; each in-window row is placed by its interval start (or, without one, its end).
|
|
/// </summary>
|
|
/// <param name="plan">The plan.</param>
|
|
/// <param name="request">The forward request the rows answer.</param>
|
|
/// <param name="rows">The drained rows.</param>
|
|
/// <param name="byBucket">Results by ascending bucket index (first row per bucket wins).</param>
|
|
public static void Place(BucketPlan plan, Request request, IEnumerable<HistorianAggregateSample> rows,
|
|
Dictionary<long, DataValueSnapshot> byBucket)
|
|
{
|
|
var interval = request.Interval.Ticks;
|
|
foreach (var row in rows)
|
|
{
|
|
if (!SampleMapper.IsInWindow(row, request.StartUtc, request.EndUtc)) continue;
|
|
long offset = row.StartTime is { } s
|
|
? (s.ToDateTime().Ticks - request.StartUtc.Ticks) / interval
|
|
: (row.EndTime.ToDateTime().Ticks - request.StartUtc.Ticks - 1) / interval; // IsInWindow ⇒ end in (lo, hi]
|
|
if (offset < 0 || offset >= request.BucketCount) continue;
|
|
var index = request.FirstBucket + offset;
|
|
if (byBucket.ContainsKey(index)) continue;
|
|
|
|
var snapshot = SampleMapper.ToAggregateSnapshot(row);
|
|
var status = snapshot.StatusCode;
|
|
if (plan.IsTruncated(index) && !StatusBits.IsBad(status)) status |= HistorianInfoBits.HistorianPartial;
|
|
byBucket[index] = snapshot with { StatusCode = status, SourceTimestampUtc = plan.BucketTimestamp(index) };
|
|
}
|
|
}
|
|
|
|
/// <summary>The placed results in the read's order (ascending forward, descending reverse).</summary>
|
|
/// <param name="plan">The plan.</param>
|
|
/// <param name="byBucket">Results by ascending bucket index.</param>
|
|
/// <returns>The results.</returns>
|
|
public static IReadOnlyList<DataValueSnapshot> Assemble(BucketPlan plan, IReadOnlyDictionary<long, DataValueSnapshot> byBucket) =>
|
|
(plan.Reversed ? byBucket.OrderByDescending(kv => kv.Key) : byBucket.OrderBy(kv => kv.Key)).Select(kv => kv.Value).ToList();
|
|
}
|