feat(historian-gateway): native TimeAverage/Total on by default, SendEvents batches, server-side max_events (#694, #746)

#694 item 3 — historiangw#27 is fixed and now live-verified through OtOpcUa:
- ServerHistorian:Gateway:EnableNativeTimeWeighted defaults to true (false is
  the kill switch for a gateway older than 9f711c1). TimeAverage / Total are
  served and advertised again (12 aggregates).
- The gateway sends the interval as max(1, round(ms)) and AVEVA clips its last
  interval at the request end (live). NativeAggregatePlan therefore sends a
  multi-interval grid only for whole-ms intervals; a fractional interval is read
  one interval per request (at most 1,000, else BadTooManyOperations), and every
  one-interval request goes out rounded UP to whole ms so AVEVA answers one
  clipped row. Live, a 3.0004 s clipped interval sent at its own width lost its
  last 0.4 ms (Total 123 vs 123.0172).

#746 — HistorianGateway Contracts/Client 1.0.0 surface:
- Alarm history: GatewayAlarmHistorianWriter sends SendEvents batches of <=200.
  A failed per-event result is re-sent alone via SendEvent (classified as
  before); a refused batch (InvalidArgument etc.) goes event by event so a
  poison event dead-letters alone; a systemic fault retries the rest without
  sending more; Unimplemented (older gateway) falls back to SendEvent for good.
  IHistorianGatewayClient.SendEventsAsync defaults to Unimplemented.
- Events reads: a capped page sends cap + 1 as ReadEventsRequest.max_events, so
  the gateway completes the stream instead of the client cancelling it. An
  ambiguous capped answer (foreign rows filtered out of the server's cut) or a
  refused cap is re-read uncapped with the existing EventDrainBudget drain.

Tests: GatewayServerCappedEventsTests, GatewayAlarmWriterBatchTests,
GatewayNativeWholeMillisecondGridTests (gateway-faithful rounding fake), gate
tests flipped to the new default; every new test mutation-checked (each
mutation of the production change fails at least one). Live harness
AvevaGatewayHistorianFollowupsLiveGateTests (read-only) for #694/#700/#746.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
2026-11-12 23:57:38 -05:00
parent 09b373e4db
commit 53978ca638
22 changed files with 1847 additions and 176 deletions
@@ -37,13 +37,26 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway;
/// row for is absent (not a fabricated <c>BadNoData</c>, unlike the SQL backends).
/// </para>
/// <para>
/// ⚠️ Today historiangw#27 (the interval reaches AVEVA in ticks where it reads milliseconds) makes AVEVA answer ONE
/// Good row per request — the whole-intervals read lands on its earliest interval with a whole-span value and the rest
/// are absent — in both directions. The mapping needs no change once the gateway is fixed.
/// <b>Whole milliseconds only on a shared grid.</b> The gateway sends AVEVA the interval in whole milliseconds
/// (<c>max(1, round(ms))</c>), and AVEVA clips its last interval at the request's end (live, lmxopcua#694, after
/// historiangw#27: a 0.4 ms window answers one row over exactly that 0.4 ms — <c>Integral</c> = value·0.0004 s — and a
/// 1.4 ms window read at "1.4 ms" answers 1 ms rows). So a grid of several intervals is sent as one request only when
/// the interval is a whole number of milliseconds; otherwise every interval is its own request over exactly its span
/// (<see cref="MaxPerIntervalRequests"/> at most — more is <see cref="HistoryReadLimitExceededException"/>). A request
/// for ONE interval (the clipped one, a window narrower than one interval, or each of those per-interval requests) is
/// sent with its width rounded UP to whole milliseconds as the interval, so AVEVA answers exactly one row, clipped at
/// the request's end: sent at its own width, a 3.0004 s interval reached AVEVA as 3,000 ms and answered a 3 s row plus
/// a 0.4 ms one (live, lmxopcua#694 — the clipped interval's Total lost the 0.4 ms).
/// </para>
/// </remarks>
internal static class NativeAggregatePlan
{
/// <summary>
/// The most intervals a native read whose interval is not a whole number of milliseconds may cover: each is its own
/// gateway request.
/// </summary>
public const int MaxPerIntervalRequests = 1000;
/// <summary>One forward native request and the ascending bucket index of its first interval.</summary>
/// <param name="StartUtc">Forward request start (inclusive).</param>
/// <param name="EndUtc">Forward request end.</param>
@@ -55,10 +68,32 @@ internal static class NativeAggregatePlan
/// <summary>The forward requests serving a plan: the whole intervals first, then the clipped one (if any).</summary>
/// <param name="plan">A plan with at least one bucket.</param>
/// <returns>One or two forward requests.</returns>
/// <exception cref="HistoryReadLimitExceededException">
/// The interval is not a whole number of milliseconds and the window holds more than
/// <see cref="MaxPerIntervalRequests"/> intervals.
/// </exception>
public static IReadOnlyList<Request> Requests(BucketPlan plan)
{
var interval = TimeSpan.FromTicks(plan.IntervalTicks);
var n = plan.BucketCount;
if (n > 1 && plan.IntervalTicks % TimeSpan.TicksPerMillisecond != 0)
{
// The gateway would round the interval to whole ms and AVEVA would answer on that grid, not the plan's: read
// each interval over exactly its span instead (one interval per request is exact at any width).
if (n > MaxPerIntervalRequests)
throw new HistoryReadLimitExceededException(MaxPerIntervalRequests,
$"A native TimeAverage/Total read whose interval is not a whole number of milliseconds is read one " +
$"interval per gateway request, at most {MaxPerIntervalRequests}; this window holds {n}. Use a whole-millisecond " +
"ProcessingInterval or a shorter window.");
var each = new List<Request>((int)n);
for (long i = 0; i < n; i++)
{
var lo = new DateTime(plan.LowerTicks(i), DateTimeKind.Utc);
var hi = new DateTime(plan.UpperTicks(i), DateTimeKind.Utc);
each.Add(new Request(lo, hi, OneInterval(hi - lo), i, 1));
}
return each;
}
var clipped = plan.IsTruncated(0) ? 0L : plan.IsTruncated(n - 1) ? n - 1 : -1L;
if (clipped < 0) return [new Request(plan.StartUtc, plan.EndUtc, interval, 0, n)];
@@ -69,10 +104,23 @@ internal static class NativeAggregatePlan
requests.Add(clipped == 0
? new Request(clipHi, plan.EndUtc, interval, 1, n - 1) // reverse: the clipped one is the earliest
: new Request(plan.StartUtc, clipLo, interval, 0, n - 1)); // forward: the clipped one is the latest
requests.Add(new Request(clipLo, clipHi, clipHi - clipLo, clipped, 1));
requests.Add(new Request(clipLo, clipHi, OneInterval(clipHi - clipLo), clipped, 1));
return requests;
}
/// <summary>
/// The interval to send for a request covering exactly one interval of <paramref name="width"/>: the width rounded UP
/// to whole milliseconds (at least 1 ms), which the gateway sends unchanged, so AVEVA's single interval spans the whole
/// request and is clipped at its end — never a whole-ms interval plus a sub-ms remainder row.
/// </summary>
/// <param name="width">The request's span (&gt; 0).</param>
/// <returns>The interval to send.</returns>
internal static TimeSpan OneInterval(TimeSpan width)
{
var ms = Math.Max(1, (width.Ticks + TimeSpan.TicksPerMillisecond - 1) / TimeSpan.TicksPerMillisecond);
return TimeSpan.FromTicks(ms * TimeSpan.TicksPerMillisecond);
}
/// <summary>
/// Maps a request's rows onto the plan's buckets: rows outside the request window (AVEVA's pre-window row)
/// are dropped; each in-window row is placed by its interval start (or, without one, its end).