dohertj2/lmxopcua
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity

Time-weighted aggregates (#671) follow-ups: reverse anchoring, Partial bit, integral unit, AggregateConfiguration, CLI list #677

New Issue
Closed
opened 2026-10-11 10:18:14 -04:00 by dohertj2 · 2 comments
dohertj2 commented 2026-10-11 10:18:14 -04:00
Owner
Copy Link
Copy Source

Follow-ups from #671 (time-weighted Part 13 aggregates) and live gate 2.

  1. Reverse anchoring: reverse reads of TimeAverage/TimeAverage2/Total/Total2/Average/Count put the partial interval at the latest end (BucketPlan.cs:9-10, documented). The SDK's AggregateCalculator.CreateSlice anchors a reverse read at its later time — only Interpolative follows the SDK rule today. Decide and align.
  2. Per-sample Partial bit (the Partial bit is set per bucket; Part 13 also defines per-sample cases).
  3. AVEVA/Gateway Total integral unit — confirm the Gateway's integral unit matches the SQL backends (seconds).
  4. Client-supplied AggregateConfiguration is ignored (server defaults always) — support or reject explicitly.
  5. Client.CLI aggregate list does not offer TimeAverage2 / Total2 / Interpolative.
Follow-ups from #671 (time-weighted Part 13 aggregates) and live gate 2. 1. **Reverse anchoring:** reverse reads of TimeAverage/TimeAverage2/Total/Total2/Average/Count put the partial interval at the latest end (`BucketPlan.cs:9-10`, documented). The SDK's `AggregateCalculator.CreateSlice` anchors a reverse read at its later time — only Interpolative follows the SDK rule today. Decide and align. 2. Per-sample Partial bit (the Partial bit is set per bucket; Part 13 also defines per-sample cases). 3. AVEVA/Gateway `Total` integral unit — confirm the Gateway's integral unit matches the SQL backends (seconds). 4. Client-supplied `AggregateConfiguration` is ignored (server defaults always) — support or reject explicitly. 5. Client.CLI aggregate list does not offer TimeAverage2 / Total2 / Interpolative.
dohertj2 commented 2026-10-11 13:20:33 -04:00
Author
Owner
Copy Link
Copy Source

Items 1, 2, 4, 5 and the offline part of 3 done — merged to master (945ec2c0; client list in 3067146b).

  1. Reverse processed reads (all aggregates, SqlServer + QuestDB) now follow the SDK / Part 13: window (EndTime, StartTime], buckets anchored at the later time stepping back, the partial bucket is the earliest, each stamped with its later edge, latest-first. ⚠️ Client-visible for reverse reads only; forward reads unchanged. Pinned by SdkAggregateParityTests, which drive the SDK's own AggregateCalculators in-process (unit + through real SQL on both backends).
  2. Start-of-data Partial: the bucket the tag's data starts inside carries Partial (forward reads, every aggregate except Average — as the SDK). The earliest-sample lookup is skipped for reverse/Average and capped at the window end.
  3. GatewayTagProvisioner now sends IntegralDivisor = 1.0 (proto3 default was 0).
  4. AggregateConfiguration: UseServerCapabilitiesDefaults=true or values equal to the server defaults ⇒ served; anything else ⇒ per-node BadAggregateConfigurationRejected (defaults are named constants, tested equal to the SDK's).
  5. Client.CLI / Client.UI aggregate lists gain TimeAverage, TimeAverage2, Total, Total2, Interpolative.

Live gate (docker-dev, livegate-6) PASS on SqlServer + QuestDB: reverse reads latest-first, later-edge stamps, partial bucket earliest; forward unchanged; Average/Min/Count match an offline calculation; 192 rows compared across backends — identical times/status, values within 7.5e-9. Start-of-data Partial forward only. AggregateConfiguration rejection on all backends.

Found by the parity test and filed separately: #680 (pre-existing forward differences from the SDK) and #679 (Gateway Count/Average mode mapping).

Left open: live check of the AVEVA Total (Integral) magnitude with a constant sandbox tag — needs the VPN.

Items 1, 2, 4, 5 and the offline part of 3 done — merged to master (`945ec2c0`; client list in `3067146b`). 1. **Reverse processed reads** (all aggregates, SqlServer + QuestDB) now follow the SDK / Part 13: window `(EndTime, StartTime]`, buckets anchored at the later time stepping back, the partial bucket is the **earliest**, each stamped with its **later** edge, latest-first. ⚠️ Client-visible for reverse reads only; forward reads unchanged. Pinned by `SdkAggregateParityTests`, which drive the SDK's own `AggregateCalculator`s in-process (unit + through real SQL on both backends). 2. **Start-of-data Partial**: the bucket the tag's data starts inside carries Partial (forward reads, every aggregate except Average — as the SDK). The earliest-sample lookup is skipped for reverse/Average and capped at the window end. 3. `GatewayTagProvisioner` now sends `IntegralDivisor = 1.0` (proto3 default was 0). 4. **AggregateConfiguration**: `UseServerCapabilitiesDefaults=true` or values equal to the server defaults ⇒ served; anything else ⇒ per-node `BadAggregateConfigurationRejected` (defaults are named constants, tested equal to the SDK's). 5. Client.CLI / Client.UI aggregate lists gain TimeAverage, TimeAverage2, Total, Total2, Interpolative. **Live gate (docker-dev, livegate-6) PASS on SqlServer + QuestDB:** reverse reads latest-first, later-edge stamps, partial bucket earliest; forward unchanged; Average/Min/Count match an offline calculation; 192 rows compared across backends — identical times/status, values within 7.5e-9. Start-of-data Partial forward only. AggregateConfiguration rejection on all backends. Found by the parity test and filed separately: **#680** (pre-existing forward differences from the SDK) and **#679** (Gateway Count/Average mode mapping). **Left open:** live check of the AVEVA `Total` (Integral) magnitude with a constant sandbox tag — needs the VPN.
dohertj2 commented 2026-10-12 14:21:06 -04:00
Author
Owner
Copy Link
Copy Source

Live AVEVA leg (2026-10-12, VPN). Integral units with IntegralDivisor=1: PASS at the AVEVA level — a constant 1 over 1 s integrates to 1 (value·seconds; constant 1 over 20 s ⇒ 20; a 10→20→30 ramp ⇒ 450, the trapezoidal area), the same unit as the SqlServer/QuestDB Total.

End to end through OtOpcUa the Gateway Total is currently wrong for an unrelated reason: the gateway sends the aggregate interval 10,000× too large (historiangw#27), and OtOpcUa passes through AVEVA's leading pre-window row / drops Uncertain rows (#685). Those are tracked there; the #677 legs themselves are verified. Closing.

**Live AVEVA leg (2026-10-12, VPN).** Integral units with `IntegralDivisor=1`: **PASS at the AVEVA level** — a constant 1 over 1 s integrates to 1 (value·seconds; constant 1 over 20 s ⇒ 20; a 10→20→30 ramp ⇒ 450, the trapezoidal area), the same unit as the SqlServer/QuestDB `Total`. End to end through OtOpcUa the Gateway `Total` is currently wrong for an unrelated reason: the gateway sends the aggregate interval 10,000× too large (historiangw#27), and OtOpcUa passes through AVEVA's leading pre-window row / drops Uncertain rows (#685). Those are tracked there; the #677 legs themselves are verified. Closing.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
auto-managed
queue management
 cross-driver
queue management
 driver/abcip
queue management
 driver/ablegacy
queue management
 driver/focas
queue management
 driver/mqtt
MQTT / Sparkplug B broker-subscribe driver
 driver/opcuaclient
queue management
 driver/s7
queue management
 driver/twincat
queue management
 phase/1
queue management
 phase/2
queue management
 phase/3
queue management
 phase/4
queue management
 phase/5
queue management
 phase/6
queue management
 queue/blocked
queue management
 queue/done
queue management
 queue/failed
queue management
 queue/in-progress
queue management
 queue/queued
queue management
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
dohertj2 (dohertj2)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dohertj2/lmxopcua#677
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?