Joseph Doherty
8a0032bfdf
When the data starts inside an interval (no start bound) and its first value is
Uncertain, the frozen 1.5.378.182 AverageAggregateCalculator integrated the
uncovered span as a ramp from 0 and divided TimeAverage by the whole interval:
GetRegionsInValueSet (ignoreBadData, sloped) overwrites the BadNoData start
region's status with UncertainDataSubNormal in its one branch without a not-Bad
guard. Live AggregateFilter therefore disagreed with HistoryRead and Part 13
§5.4.3.6/§5.4.3.8 ("No Start Bound: Calculate Partial Interval"); QP: Total
188.96 vs 163.96, TimeAverage 9.448 vs 10.931. SDK 2.0.1 stock does the same.
Owner-approved 2026-11-11.
- New app-owned CoveredTimeAverageAggregateCalculator subclasses the baseline
and recomputes only TimeAverage/Total, restoring the Bad status of a region
that starts at a Bad point (only the start bound can). Only such an interval's
value changes; status, timestamps, type and bits are the baseline's. Stepped
and TreatUncertainAsBad=true never reached the defect. CompatibilityAggregate-
Factory routes exactly those two IDs; copied SDK files untouched.
- Golden test: 0 fixture vectors change (every data set has a usable value at
-10 s), asserted. New sweep over late-start variants of every data set (cut
+5 s / +25 s, first non-Bad sample Uncertain, all directions/stepping/configs/
incremental): 150 / 142 of 384 change, each pinned to a baseline-derived
oracle (untouched baseline's value with that sample Good, its status/bits on
the variant itself) - never our calculator. Plus a hand-computed case.
- Live-vs-HistoryRead parity (TimeWeightedAggregator) forward + reverse, no
masks, with discrimination against the pre-#821 baseline; QualityMix now
compares QP's TimeAverage/Total with the live oracle on both SQL backends;
booted-runtime facts test expects 163.96 / 10.93.
- README recorded decision, ServerAggregateConfiguration remarks, oracle
remarks, docs/Historian.md and .claude/rules/historian.md updated.
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
232 lines
15 KiB
C#
232 lines
15 KiB
C#
using Opc.Ua;
|
||
using Opc.Ua.Server;
|
||
using ZB.MOM.WW.OtOpcUa.OpcUaServer.Compatibility.Aggregates;
|
||
using BaselineAggregateCalculator = ZB.MOM.WW.OtOpcUa.OpcUaServer.Compatibility.Aggregates.AggregateCalculator;
|
||
using BaselineAggregators = ZB.MOM.WW.OtOpcUa.OpcUaServer.Compatibility.Aggregates.Aggregators;
|
||
using Shouldly;
|
||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||
|
||
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Conformance;
|
||
|
||
/// <summary>
|
||
/// Drives the application's versioned 1.5.378.182 Part 13 compatibility calculators — the same factories
|
||
/// the upgraded server registers for live aggregates in-process over a sample set, so our processed reads can be compared with the
|
||
/// reference implementation bucket by bucket (#677). The calculators run under the server's default
|
||
/// <see cref="AggregateConfiguration"/> (TreatUncertainAsBad = false, PercentDataBad = PercentDataGood = 100,
|
||
/// UseSlopedExtrapolation = false), sloped interpolation, source timestamps.
|
||
/// </summary>
|
||
/// <remarks>
|
||
/// <para>Shared (source-linked) by <c>Sql.Common.Tests</c> and both SQL backends' integration suites.</para>
|
||
/// <para>
|
||
/// <b>PercentGood / DurationGood (#796, #820, #809).</b> The 1.5.378.182 baseline changed
|
||
/// <c>StatusAggregateCalculator.ComputeDurationGoodBad</c> (1.5.378.106 and earlier counted Good regions only):
|
||
/// with TreatUncertainAsBad = false it counts Uncertain regions as <b>Good</b> time (as Bad when true). Part 13
|
||
/// keeps Uncertain out of both (§5.4.3.31 DurationGood counts Good regions; §5.4.3.33 PercentGood =
|
||
/// DurationGood ÷ interval; Mantis 0006166, closed "no change required": an all-Uncertain interval is
|
||
/// PercentGood 0 <i>and</i> PercentBad 0), and so does ours. Since #809 the LIVE calculators follow Part 13 too:
|
||
/// <c>CompatibilityAggregateFactory</c> serves those two through the application's
|
||
/// <c>GoodOnlyStatusAggregateCalculator</c> (the same baseline region machinery — simple bounds, end-of-data 1 ms
|
||
/// region, status and Partial bits — counting only Good regions), so <see cref="Run"/> / <see cref="RunLive"/> are
|
||
/// exactly what a live AggregateFilter computes, for every aggregate. PercentBad / DurationBad count Bad regions only
|
||
/// (unchanged by .182). <see cref="RunSdkAsIs"/> runs the untouched baseline (what live served before #809 / #821),
|
||
/// for the tests that pin those divergences.
|
||
/// </para>
|
||
/// <para>
|
||
/// <b>TimeAverage / Total with no start bound (#821).</b> When the data starts inside an interval with an
|
||
/// <b>Uncertain</b> first value, the baseline integrates the uncovered span before it as a ramp from 0 (its region
|
||
/// machinery overwrites that region's Bad status with Uncertain). Part 13 ("No Start Bound: Calculate Partial
|
||
/// Interval") and ours integrate only the covered part, as the baseline itself does after a Good first value. Since
|
||
/// #821 the live calculators follow Part 13 here too (<c>CoveredTimeAverageAggregateCalculator</c>).
|
||
/// </para>
|
||
/// <para>
|
||
/// Not <c>100 − PercentBad</c> under TreatUncertainAsBad = true: that flag also changes the region machinery
|
||
/// (<c>IsGood</c> drives the end-of-data 1 ms rule and the simple bounds), so after an Uncertain last sample the
|
||
/// region runs to the interval end instead of 1 ms and the total differs from the one PercentBad is computed
|
||
/// over. Pinned by <c>SdkAggregateParityTests</c>.
|
||
/// </para>
|
||
/// </remarks>
|
||
public static class SdkAggregateOracle
|
||
{
|
||
private static readonly ITelemetryContext Telemetry = DefaultTelemetry.Create(_ => { });
|
||
/// <summary>One processed value: value (double / int / null), full status code, timestamp.</summary>
|
||
public readonly record struct Result(object? Value, uint StatusCode, DateTime Timestamp);
|
||
|
||
/// <summary>The server-default aggregate configuration (what <c>AggregateManager.GetDefaultConfiguration</c> returns).</summary>
|
||
public static AggregateConfiguration DefaultConfiguration => new()
|
||
{
|
||
TreatUncertainAsBad = false,
|
||
PercentDataBad = 100,
|
||
PercentDataGood = 100,
|
||
UseSlopedExtrapolation = false,
|
||
UseServerCapabilitiesDefaults = false,
|
||
};
|
||
|
||
/// <summary>
|
||
/// Runs the Part 13 oracle for <paramref name="aggregate"/> over <paramref name="samples"/>: the live calculator
|
||
/// (<see cref="RunLive"/>) — the baseline, except PercentGood / DurationGood, which count Good regions only, and
|
||
/// TimeAverage / Total, which leave the span before an Uncertain first value out (see the remarks, #796/#820/#809/#821).
|
||
/// </summary>
|
||
/// <param name="aggregate">The aggregate.</param>
|
||
/// <param name="start">Request start (later than <paramref name="end"/> for a reverse read).</param>
|
||
/// <param name="end">Request end.</param>
|
||
/// <param name="interval">Processing interval.</param>
|
||
/// <param name="samples">Every stored sample of the tag, any order (sorted here).</param>
|
||
/// <returns>The oracle's processed values, in the order it returns them.</returns>
|
||
public static IReadOnlyList<Result> Run(HistoryAggregateType aggregate, DateTime start, DateTime end, TimeSpan interval,
|
||
IEnumerable<(DateTime Time, HistorianValue Value, uint Status)> samples) =>
|
||
RunLive(aggregate, start, end, interval, samples);
|
||
|
||
/// <summary>
|
||
/// Runs exactly the calculator the server registers for live AggregateFilter monitored items
|
||
/// (<c>CompatibilityAggregateFactory.Create</c>, the #809 Good-only PercentGood / DurationGood and the #821
|
||
/// covered-only TimeAverage / Total included).
|
||
/// </summary>
|
||
/// <param name="aggregate">The aggregate.</param>
|
||
/// <param name="start">Request start (later than <paramref name="end"/> for a reverse read).</param>
|
||
/// <param name="end">Request end.</param>
|
||
/// <param name="interval">Processing interval.</param>
|
||
/// <param name="samples">Every stored sample of the tag, any order (sorted here).</param>
|
||
/// <param name="configuration">The configuration (default <see cref="DefaultConfiguration"/>).</param>
|
||
/// <returns>The calculator's processed values, in the order it returns them.</returns>
|
||
public static IReadOnlyList<Result> RunLive(HistoryAggregateType aggregate, DateTime start, DateTime end,
|
||
TimeSpan interval, IEnumerable<(DateTime Time, HistorianValue Value, uint Status)> samples,
|
||
AggregateConfiguration? configuration = null) =>
|
||
RunCore(CompatibilityAggregateFactory.Create(AggregateId(aggregate), start, end, interval.TotalMilliseconds, false,
|
||
configuration ?? DefaultConfiguration, Telemetry), start, end, samples);
|
||
|
||
/// <summary>
|
||
/// Runs the untouched baseline (1.5.378.182) calculator for <paramref name="aggregate"/> exactly as it computes it,
|
||
/// under <see cref="DefaultConfiguration"/> — including its PercentGood / DurationGood rule (Uncertain counted as
|
||
/// Good) and its TimeAverage / Total ramp before an Uncertain first value, which neither HistoryRead nor, since
|
||
/// #809 / #821, the live calculators use.
|
||
/// </summary>
|
||
/// <param name="aggregate">The aggregate.</param>
|
||
/// <param name="start">Request start (later than <paramref name="end"/> for a reverse read).</param>
|
||
/// <param name="end">Request end.</param>
|
||
/// <param name="interval">Processing interval.</param>
|
||
/// <param name="samples">Every stored sample of the tag, any order (sorted here).</param>
|
||
/// <param name="configuration">The configuration (default <see cref="DefaultConfiguration"/>).</param>
|
||
/// <returns>The calculator's processed values, in the order it returns them.</returns>
|
||
public static IReadOnlyList<Result> RunSdkAsIs(HistoryAggregateType aggregate, DateTime start, DateTime end,
|
||
TimeSpan interval, IEnumerable<(DateTime Time, HistorianValue Value, uint Status)> samples,
|
||
AggregateConfiguration? configuration = null) =>
|
||
RunCore(new LegacyCalculatorAdapter(BaselineAggregators.CreateStandardCalculator(AggregateId(aggregate), start, end,
|
||
interval.TotalMilliseconds, false, configuration ?? DefaultConfiguration, Telemetry)
|
||
?? throw new ArgumentOutOfRangeException(nameof(aggregate))), start, end, samples);
|
||
|
||
private static IReadOnlyList<Result> RunCore(IAggregateCalculator calc, DateTime start, DateTime end,
|
||
IEnumerable<(DateTime Time, HistorianValue Value, uint Status)> samples)
|
||
{
|
||
|
||
// The SDK expects values in the direction of the read: ascending forward, descending reverse.
|
||
var ordered = samples.OrderBy(s => s.Time).ToList();
|
||
if (end < start) ordered.Reverse();
|
||
foreach (var s in ordered) calc.QueueRawValue(ToDataValue(s.Time, s.Value, s.Status));
|
||
|
||
var results = new List<Result>();
|
||
for (var guard = 0; guard < 100_000; guard++)
|
||
{
|
||
if (!calc.TryGetProcessedValue(true, out var dv)) break;
|
||
results.Add(new Result(dv.WrappedValue.AsBoxedObject(Variant.BoxingBehavior.Legacy), dv.StatusCode.Code, (DateTime)dv.SourceTimestamp));
|
||
}
|
||
return results;
|
||
}
|
||
|
||
/// <summary>
|
||
/// Asserts our snapshots equal the SDK's: same count, same timestamps, same status code bits (severity + sub-code)
|
||
/// and Partial bit (and optionally the Calculated / Interpolated bits), same value (doubles to 1e-9; ints exactly;
|
||
/// null on a Bad status). A failure lists every differing interval side by side.
|
||
/// </summary>
|
||
/// <param name="ours">Our processed values.</param>
|
||
/// <param name="sdk">The SDK's.</param>
|
||
/// <param name="label">Context for failure messages.</param>
|
||
/// <param name="compareCalculatedInterpolatedBits">Also compare the Calculated / Interpolated aggregate bits.</param>
|
||
/// <param name="compareCodeBits">Compare the severity + sub-code (false only for a documented status divergence).</param>
|
||
/// <param name="compareValues">Compare the values (false only for a documented value divergence).</param>
|
||
/// <param name="compareTimestamps">Compare the timestamps (false only for a documented timestamp divergence).</param>
|
||
/// <param name="comparePartialBit">Compare the Partial bit (false only for a documented divergence).</param>
|
||
public static void ShouldMatch(IReadOnlyList<DataValueSnapshot> ours, IReadOnlyList<Result> sdk, string label,
|
||
bool compareCalculatedInterpolatedBits = true, bool compareCodeBits = true, bool compareValues = true,
|
||
bool compareTimestamps = true, bool comparePartialBit = true)
|
||
{
|
||
var problems = new List<string>();
|
||
if (ours.Count != sdk.Count) problems.Add($"interval count: ours {ours.Count}, sdk {sdk.Count}");
|
||
for (var i = 0; i < Math.Min(ours.Count, sdk.Count); i++)
|
||
{
|
||
var o = ours[i];
|
||
var s = sdk[i];
|
||
var bitMask = (comparePartialBit ? 0x4u : 0u) | (compareCodeBits ? 0xFFFF0000u : 0u)
|
||
| (compareCalculatedInterpolatedBits ? 0x3u : 0u);
|
||
var same = (!compareTimestamps || o.SourceTimestampUtc == s.Timestamp)
|
||
&& (o.StatusCode & bitMask) == (s.StatusCode & bitMask);
|
||
if (!compareValues || (!compareCodeBits && StatusCode.IsBad(s.StatusCode)))
|
||
{
|
||
// Values not compared (or the SDK answered Bad under a status this comparison does not check).
|
||
}
|
||
else if (same && !StatusCode.IsBad(s.StatusCode))
|
||
{
|
||
same = s.Value switch
|
||
{
|
||
null => o.Value is null,
|
||
int n => o.Value is not null && Convert.ToInt32(o.Value) == n,
|
||
_ => o.Value is not null && Math.Abs(Convert.ToDouble(o.Value) - Convert.ToDouble(s.Value)) <= 1e-9,
|
||
};
|
||
}
|
||
else if (same && compareCodeBits) same = o.Value is null;
|
||
if (!same)
|
||
problems.Add($"[{i}] ours {o.SourceTimestampUtc:HH:mm:ss.fffffff} {o.Value ?? "null"} 0x{o.StatusCode:X8}" +
|
||
$" | sdk {s.Timestamp:HH:mm:ss.fffffff} {s.Value ?? "null"} 0x{s.StatusCode:X8}");
|
||
}
|
||
problems.ShouldBeEmpty($"{label} differs from the SDK:\n" + string.Join("\n", problems));
|
||
}
|
||
|
||
/// <summary>The <see cref="CompatibilityAggregateFactory"/> adapter, for a baseline calculator built here.</summary>
|
||
private sealed class LegacyCalculatorAdapter(BaselineAggregateCalculator calculator) : IAggregateCalculator
|
||
{
|
||
public NodeId AggregateId => calculator.AggregateId;
|
||
public bool QueueRawValue(DataValue value) => calculator.QueueRawValue(LegacyDataValue.FromSdk(value));
|
||
public bool HasEndTimePassed(DateTimeUtc currentTime) => calculator.HasEndTimePassed((DateTime)currentTime);
|
||
public bool TryGetProcessedValue(bool returnPartial, out DataValue value)
|
||
{
|
||
var processed = calculator.GetProcessedValue(returnPartial);
|
||
value = processed is null ? default : processed.ToSdk();
|
||
return processed is not null;
|
||
}
|
||
}
|
||
|
||
private static DataValue ToDataValue(DateTime t, HistorianValue v, uint status) => new(
|
||
v.Kind switch
|
||
{
|
||
HistorianValueKind.Empty => Variant.Null,
|
||
HistorianValueKind.Int64 => Variant.From(v.Integer),
|
||
HistorianValueKind.Boolean => Variant.From(v.Number != 0),
|
||
HistorianValueKind.String => v.Text is { } text ? Variant.From(text) : Variant.Null,
|
||
_ => Variant.From(v.Number),
|
||
},
|
||
status, t, t);
|
||
|
||
private static NodeId AggregateId(HistoryAggregateType a) => a switch
|
||
{
|
||
HistoryAggregateType.Average => ObjectIds.AggregateFunction_Average,
|
||
HistoryAggregateType.Minimum => ObjectIds.AggregateFunction_Minimum,
|
||
HistoryAggregateType.Maximum => ObjectIds.AggregateFunction_Maximum,
|
||
HistoryAggregateType.Range => ObjectIds.AggregateFunction_Range,
|
||
HistoryAggregateType.MinimumActualTime => ObjectIds.AggregateFunction_MinimumActualTime,
|
||
HistoryAggregateType.MaximumActualTime => ObjectIds.AggregateFunction_MaximumActualTime,
|
||
HistoryAggregateType.Count => ObjectIds.AggregateFunction_Count,
|
||
HistoryAggregateType.PercentGood => ObjectIds.AggregateFunction_PercentGood,
|
||
HistoryAggregateType.PercentBad => ObjectIds.AggregateFunction_PercentBad,
|
||
HistoryAggregateType.DurationGood => ObjectIds.AggregateFunction_DurationGood,
|
||
HistoryAggregateType.DurationBad => ObjectIds.AggregateFunction_DurationBad,
|
||
HistoryAggregateType.Start => ObjectIds.AggregateFunction_Start,
|
||
HistoryAggregateType.End => ObjectIds.AggregateFunction_End,
|
||
HistoryAggregateType.Delta => ObjectIds.AggregateFunction_Delta,
|
||
HistoryAggregateType.TimeAverage => ObjectIds.AggregateFunction_TimeAverage,
|
||
HistoryAggregateType.TimeAverage2 => ObjectIds.AggregateFunction_TimeAverage2,
|
||
HistoryAggregateType.Total => ObjectIds.AggregateFunction_Total,
|
||
HistoryAggregateType.Total2 => ObjectIds.AggregateFunction_Total2,
|
||
HistoryAggregateType.Interpolative => ObjectIds.AggregateFunction_Interpolative,
|
||
_ => throw new ArgumentOutOfRangeException(nameof(a), a, "no SDK aggregate id mapped"),
|
||
};
|
||
}
|