feat(historian): Gateway serves the per-sample aggregates from raw reads with the shared Sql.Common evaluator (#681)

Average, Count, Minimum, Maximum, Range, Start, End, Delta and Minimum/MaximumActualTime are evaluated
on the Gateway backend from forward ReadRaw reads with BucketPlan + AggregateEvaluator, so Gateway
matches SqlServer/QuestDB exactly (values, status bits, interval-start stamps, #677 reverse).

- Window read [start - 1 ms, end] so AVEVA's initial-value row never lands in a bucket; a pre-window
  row proves earlier data only with a non-Bad OPC quality (histsdk puts a value on every NULL row).
- Earlier data otherwise from a 2-row [1970, start) probe; later data from a 2-row [end, now+1d]
  look-ahead; both ignore rows stamped at their query start/end.
- No gateway stream is ever abandoned (a client-side cancel makes the gateway evict its shared
  historian session); past ServerHistorian:MaxTimeWeightedRawRows the bounded stream is drained, then
  BadTooManyOperations. The gateway's max_values-ceiling InvalidArgument maps to the same limit; a
  caller cancellation is not a backend fault.
- TimeAverage/Total stay AVEVA-native (no prior-sample read for exact Part 13 bounds), now stamped with
  the interval start (PENDING the live leg: histsdk's tail+43 start_time field is unverified) and
  carrying the Calculated bits.
- Served set: GatewayHistorianDataSource.ServedAggregates (12). Live AVEVA leg open (VPN).
This commit is contained in:
Joseph Doherty
2026-10-12 09:55:32 -04:00
parent dbdf29c8fc
commit 78f604e0cd
16 changed files with 1187 additions and 77 deletions
@@ -1,5 +1,6 @@
using ZB.MOM.WW.HistorianGateway.Contracts.Grpc;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common;
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Mapping;
@@ -30,16 +31,23 @@ internal static class SampleMapper
else
value = null;
return new DataValueSnapshot(
Value: value,
StatusCode: ToStatus(sample),
SourceTimestampUtc: sample.Timestamp?.ToDateTime(), // Utc kind
ServerTimestampUtc: DateTime.UtcNow);
}
/// <summary>The OPC UA status of a raw sample (shared by raw reads and the #681 raw-evaluated aggregates).</summary>
/// <param name="sample">The gateway raw sample.</param>
/// <returns>The mapped status code.</returns>
public static uint ToStatus(HistorianSample sample)
{
// Prefer the OPC DA quality byte (opc_quality); the gateway populates it directly from the
// SDK's OpcQuality, so it is the authoritative byte for GatewayQualityMapper. Fall back to
// the historian quality field only when opc_quality is unset (0).
byte qualityByte = sample.OpcQuality != 0 ? (byte)sample.OpcQuality : (byte)sample.Quality;
return new DataValueSnapshot(
Value: value,
StatusCode: GatewayQualityMapper.Map(qualityByte),
SourceTimestampUtc: sample.Timestamp?.ToDateTime(), // Utc kind
ServerTimestampUtc: DateTime.UtcNow);
return GatewayQualityMapper.Map(qualityByte);
}
/// <summary>Maps a batch of raw samples to value snapshots, in order.</summary>
@@ -57,20 +65,28 @@ internal static class SampleMapper
/// <param name="aggregate">The gateway aggregate sample.</param>
/// <returns>The driver-agnostic snapshot.</returns>
/// <remarks>
/// <para>
/// Unlike the legacy Wonderware DTO (a nullable <c>double?</c>), the gateway proto carries a
/// non-optional <c>double value</c>, so an unavailable (no-data) bucket cannot be signalled by a
/// null value. Instead it is signalled by a non-Good <c>opc_quality</c>: a Good bucket
/// (<c>opc_quality &gt;= 192</c>) yields its value with a Good status, anything else maps to
/// <c>BadNoData</c> with a null value — preserving the Wonderware aggregate contract (binary
/// Good-with-value / BadNoData-null).
/// </para>
/// <para>
/// #681: both statuses carry the Calculated info bits (<c>0x401</c>) like every other backend's processed value, and
/// the bucket is stamped with its interval START (Part 13; it was the interval end), falling back to the end only when
/// the start is absent. Pending the live leg: histsdk decodes <c>start_time</c> from a row field (<c>tail + 43</c>) its
/// reverse-engineering notes still mark unverified.
/// </para>
/// </remarks>
public static DataValueSnapshot ToAggregateSnapshot(HistorianAggregateSample aggregate)
{
bool hasData = aggregate.OpcQuality >= GoodQualityFloor;
return new DataValueSnapshot(
Value: hasData ? aggregate.Value : null, // boxes as System.Double when present
StatusCode: hasData ? StatusGood : StatusBadNoData,
SourceTimestampUtc: (aggregate.EndTime ?? aggregate.StartTime)?.ToDateTime(), // bucket timestamp
StatusCode: HistorianInfoBits.WithCalculated(hasData ? StatusGood : StatusBadNoData),
SourceTimestampUtc: (aggregate.StartTime ?? aggregate.EndTime)?.ToDateTime(), // bucket timestamp = interval start
ServerTimestampUtc: DateTime.UtcNow);
}