Files
lmxopcua/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Compatibility/Aggregates/README.md
T
Joseph Doherty ed05d50795 fix(aggregates): live PercentGood/DurationGood count Good time only (#809)
The frozen 1.5.378.182 StatusAggregateCalculator counts Uncertain regions as
Good under TreatUncertainAsBad=false, so live AggregateFilter PercentGood /
DurationGood disagreed with HistoryRead (Part 13 §5.4.3.31/§5.4.3.33, Mantis
0006166) whenever Uncertain data was present. Owner-approved 2026-11-11.

- New app-owned GoodOnlyStatusAggregateCalculator subclasses the baseline and
  replaces only the PercentGood/DurationGood computation (Good regions only);
  CompatibilityAggregateFactory routes exactly those two IDs to it. The copied
  SDK files are untouched; the other 35 aggregates are byte-identical.
- Golden test: the 64 changed vectors (38 PercentGood + 26 DurationGood) are
  exempted exactly — those whose baseline output differs from the same case's
  baseline TreatUncertainAsBad=true twin — and pinned to that twin (baseline
  evidence, not regenerated from the implementation); plus a full Good-only
  pin over all 352 PercentGood/DurationGood vectors and a hand-computed case.
- Live-vs-history parity over Good/Uncertain/Bad mixes (no masks), with a
  discrimination check against the pre-#809 baseline; the booted-runtime facts
  test now expects 8000 ms / 40 %.
- SdkAggregateOracle: Run/RunLive = the registered factory; RunSdkAsIs = the
  untouched baseline; its Part13GoodCalculator duplicate is removed.
- README (recorded decision), ServerAggregateConfiguration remarks,
  docs/Historian.md and .claude/rules/historian.md updated.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:05:34 -05:00

5.5 KiB

Preserved aggregate calculations

The SDK 2.0 upgrade keeps the application's existing historian and live AggregateFilter results (owner decision, 2026-11-10, #782). SDK 2.0.1 changes more than the default configuration: uncertain-sample selection, value-based quality, reverse data edges, partial results and interpolation differ. Changing TreatUncertainAsBad alone does not preserve the contract.

The eight upstream algorithm/factory files here originate at OPCFoundation/UA-.NETStandard tag 1.5.378.182. Original MIT license headers remain. baseline-sources.json records each original URL and SHA-256. They are application-private source, not an old SDK runtime or migration shim.

Representation adaptations:

  • Private LegacyDataValue retains mutable intermediate work values and CLR DateTime timestamps; the adapter accepts/emits immutable SDK 2 values, including status and picoseconds.
  • Nullable reference annotations describe the existing optional bounds/slices; no warning policy is disabled.
  • Immutable StatusCode methods replace the old setters. NodeId extraction and QualifiedName construction use supported typed APIs.
  • Removed TypeInfo.Cast/Variant(object, TypeInfo) operations use the SDK 2 typed conversion surface; baseline vectors verify value types, values, timestamps and all status bits.
  • Interval, quality, sample selection, queue trimming, interpolation and partial-result algorithms remain the baseline algorithms. The server creates a fresh SDK AggregateManager and registers these 37 IDs exactly once through the public API. It must not call the base registration first: replacing an existing factory also adds a duplicate diagnostic node in SDK 2.0.1. No extra capability is advertised.

The isolated scripts/validation/opcua-aggregate-baseline process pins only SDK 1.5.378.182 and generates the checked-in golden fixture. It is deliberately outside the solution and does not inherit the candidate package graph. The golden file records the baseline assembly hash, shared inputs and 6,144 outputs/cases across all 37 aggregate IDs: the original 4,144 cases plus 2,000 numeric-width, nonfinite/conversion, timestamp tie/picosecond and configuration-edge cases. Forward/reverse, stepped/sloped interpolation, uncertain-as-bad flags and incremental/partial processing are retained. The candidate tests exercise the same independent expectations both directly and through the booted runtime manager. Unset SDK 2 FILETIME timestamps map explicitly to the original CLR unset sentinel when observing results. No baseline binary is loaded by the candidate tests or production host.

Run the generator with a new output path, then compare it with tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/Fixtures/opcua-1.5.378.182-aggregates.json. It refuses to overwrite evidence. Never regenerate expectations from the compatibility implementation. Existing historian parity tests retain their application-specific documented divergences from the baseline SDK, while new compatibility tests check the complete old-calculator output.

This preserves older behavior deliberately; it does not claim that all older SDK algorithms implement the newest OPC UA interpretation. A future semantic migration needs its own consumer-impact decision and versioned tests.

Recorded semantic decisions

#809 — PercentGood / DurationGood count Good time only (owner-approved 2026-11-11)

The baseline StatusAggregateCalculator.ComputeDurationGoodBad counts Uncertain regions as Good when TreatUncertainAsBad = false (1.5.378.106 and SDK 2.0.1 count Good only). Part 13 §5.4.3.31/§5.4.3.33 (Mantis 0006166) and the HistoryRead evaluators (TimeWeightedAggregator, SqlServer + QuestDB) count Good regions only, so live AggregateFilter results disagreed with HistoryRead whenever Uncertain data was present. ScadaBridge uses neither AggregateFilter nor these aggregates.

  • The copied upstream files above are unchanged (their SHA-256s still match baseline-sources.json).
  • GoodOnlyStatusAggregateCalculator.cs (application code, not a baseline copy) subclasses the baseline StatusAggregateCalculator and replaces only the PercentGood / DurationGood computation: the same slice values, simple bounds, stepped regions, empty-slice BadNoData, timestamps, value type and Calculated bit, with a region counted as Good only when its status is Good — whatever TreatUncertainAsBad says (when true, Uncertain is Bad, so it was never Good). CompatibilityAggregateFactory.Create routes exactly those two IDs to it; the other 35 IDs, including DurationBad / PercentBad / WorstQuality, are the baseline unchanged.
  • Golden test: the 64 vectors this changes (38 PercentGood + 26 DurationGood, all with an effective TreatUncertainAsBad = false over data with Uncertain regions) are exempted exactly — the PercentGood / DurationGood vectors whose baseline output differs from the same case's baseline vector with TreatUncertainAsBad = true, where 1.5.378.182 already counted Good regions only — and each is pinned to that twin's output. The pin is therefore baseline evidence, never regenerated from the application's calculator; any further drift in any of the 6,144 cases fails. Sdk2AggregateCompatibilityTests.Issue809_* also pins a hand-computed case.
  • Live-vs-history parity: SdkAggregateParityTests.Live_AggregateFilter_PercentGood_and_DurationGood_equal_HistoryRead_over_Uncertain_samples compares the registered calculator with TimeWeightedAggregator over Good / Uncertain / Bad mixes with no masks, and checks that the data discriminates against the pre-#809 baseline.