dohertj2/lmxopcua
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity

Gateway AtTime returns AVEVA interpolation unchanged (NULL handling, drift, no Interpolated bit) #693

New Issue
Closed
opened 2026-10-13 00:11:06 -04:00 by dohertj2 · 1 comment
dohertj2 commented 2026-10-13 00:11:06 -04:00
Owner
Copy Link
Copy Source

Found by the #689–#691 live AVEVA gate (2026-10-13). Pre-existing.

Gateway HistoryRead AtTime passes AVEVA's interpolated answer through, which breaks the AtTime rules the SQL backends follow (Part 11 §6.5.4, stepped/interpolated with the Interpolated bit):

  • At a stored NULL's own time it returns the PREVIOUS value as Good (HistGW.LiveTest.Gate685 19:00:03 ⇒ 107).
  • At a stored Good sample right after a NULL it returns null + Bad (19:00:05, stored 108).
  • At an exact stored sample the value drifts (99.99999999944 for a stored 100).
  • Interpolated results carry no Interpolated info bit.

Suggested: serve Gateway AtTime from raw reads (prior/next sample via the #686/#691.1 bound lookups) with the shared evaluator the SQL backends use, as #681 did for per-sample aggregates; or at minimum return exact stored samples as stored and set the Interpolated bit.

Evidence: scratchpad livegate-aveva-689.md (F2).

Found by the #689–#691 live AVEVA gate (2026-10-13). Pre-existing. Gateway HistoryRead AtTime passes AVEVA's interpolated answer through, which breaks the AtTime rules the SQL backends follow (Part 11 §6.5.4, stepped/interpolated with the Interpolated bit): - At a stored NULL's own time it returns the PREVIOUS value as Good (`HistGW.LiveTest.Gate685` 19:00:03 ⇒ 107). - At a stored Good sample right after a NULL it returns null + Bad (19:00:05, stored 108). - At an exact stored sample the value drifts (99.99999999944 for a stored 100). - Interpolated results carry no Interpolated info bit. Suggested: serve Gateway AtTime from raw reads (prior/next sample via the #686/#691.1 bound lookups) with the shared evaluator the SQL backends use, as #681 did for per-sample aggregates; or at minimum return exact stored samples as stored and set the Interpolated bit. Evidence: scratchpad `livegate-aveva-689.md` (F2).
dohertj2 commented 2026-10-13 02:12:14 -04:00
Author
Owner
Copy Link
Copy Source

Fixed on master 66a2b0c3 (merge a5a04961; commits 37390139, 0d249f3e, 68a1f313, 2609c546). Live AVEVA gate PASS (2026-10-13).

Fix: Gateway AtTime is now worked out from raw reads by the same AtTimeEvaluator the SQL backends use (GatewayAtTimeReader):

  • one window read for the batch;
  • a walk-back for the earlier sample: steps back ×4, then reads the whole remainder, then bisects at ms granularity — a dense tag that stopped a year ago costs ≤ 31 reads;
  • a short probe for the later sample.

Live (Gate685):

Time Now AVEVA (before)
exact 19:00:00 100 99.99999999944
stored NULL, and the span after it null + Bad previous value, Good
19:00:05 (stored, after the NULL) 108 Good null + Bad
just before the NULL 107 Uncertain + Interpolated —
between Good samples linear + Interpolated (100.5, 108.5) no Interpolated bit
before the first sample BadNoData —

Ten dense timestamps cost 1 read when exact, 3 when interpolated. Kill switch ServerHistorian:GatewayAtTimeFromRaw=false reproduces AVEVA's native answers exactly.

Type hint: the node manager passes the node's scalar type (new default-interface overload ReadAtTimeAsync(…, TypeCode? valueType, …)). Gateway rows are converted before evaluation:

  • Boolean is stepped;
  • integers are interpolated, rounded half away from zero and returned as the node's type (live: Int32 4.6→5, 11.5→12, 108.5→109);
  • DateTime is stepped;
  • NaN, ±∞ or out-of-range values under an integer hint are unusable.

Also fixed (client-visible, SQL backends too): the shared evaluator returned interpolated Int64 AtTime values as Double (cond ? (long)x : v widens to double). They are now Int64. The conformance test asserts the type, because Shouldly ShouldBe compares across numeric types and hid it.

Known differences (documented): SqlServer/QuestDB return Int64 for any integer node (they ignore the hint); Gateway Raw still returns Double 0/1 for Boolean nodes; DateTime nodes on Gateway step but keep the gateway's Double value.

Read cap: 64 + 16 reads per distinct time (runaway guard; HistoryReadDeadline bounds wall-clock); a strict ceiling below 4 rows is refused with a message naming it.

Tests: 45 GatewayAtTimeTests against a wire-faithful fake. Differential fuzz vs the SQL evaluator: 0 diffs over 16,500 cases.

Follow-ups: #696 (future-time prior-search cost), #698 (value types across backends).

Fixed on master `66a2b0c3` (merge `a5a04961`; commits `37390139`, `0d249f3e`, `68a1f313`, `2609c546`). Live AVEVA gate PASS (2026-10-13). **Fix:** Gateway AtTime is now worked out from raw reads by the same `AtTimeEvaluator` the SQL backends use (`GatewayAtTimeReader`): - one window read for the batch; - a walk-back for the earlier sample: steps back ×4, then reads the whole remainder, then bisects at ms granularity — a dense tag that stopped a year ago costs ≤ 31 reads; - a short probe for the later sample. **Live (Gate685):** | Time | Now | AVEVA (before) | |---|---|---| | exact 19:00:00 | 100 | 99.99999999944 | | stored NULL, and the span after it | null + Bad | previous value, Good | | 19:00:05 (stored, after the NULL) | 108 Good | null + Bad | | just before the NULL | 107 Uncertain + Interpolated | — | | between Good samples | linear + Interpolated (100.5, 108.5) | no Interpolated bit | | before the first sample | BadNoData | — | Ten dense timestamps cost 1 read when exact, 3 when interpolated. Kill switch `ServerHistorian:GatewayAtTimeFromRaw=false` reproduces AVEVA's native answers exactly. **Type hint:** the node manager passes the node's scalar type (new default-interface overload `ReadAtTimeAsync(…, TypeCode? valueType, …)`). Gateway rows are converted before evaluation: - Boolean is stepped; - integers are interpolated, rounded half away from zero and returned as the node's type (live: Int32 4.6→5, 11.5→12, 108.5→109); - DateTime is stepped; - NaN, ±∞ or out-of-range values under an integer hint are unusable. **Also fixed (client-visible, SQL backends too):** the shared evaluator returned interpolated Int64 AtTime values as **Double** (`cond ? (long)x : v` widens to double). They are now Int64. The conformance test asserts the type, because Shouldly `ShouldBe` compares across numeric types and hid it. **Known differences (documented):** SqlServer/QuestDB return Int64 for any integer node (they ignore the hint); Gateway *Raw* still returns Double 0/1 for Boolean nodes; DateTime nodes on Gateway step but keep the gateway's Double value. **Read cap:** 64 + 16 reads per distinct time (runaway guard; `HistoryReadDeadline` bounds wall-clock); a strict ceiling below 4 rows is refused with a message naming it. **Tests:** 45 `GatewayAtTimeTests` against a wire-faithful fake. Differential fuzz vs the SQL evaluator: 0 diffs over 16,500 cases. Follow-ups: #696 (future-time prior-search cost), #698 (value types across backends).
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
auto-managed
queue management
 cross-driver
queue management
 driver/abcip
queue management
 driver/ablegacy
queue management
 driver/focas
queue management
 driver/mqtt
MQTT / Sparkplug B broker-subscribe driver
 driver/opcuaclient
queue management
 driver/s7
queue management
 driver/twincat
queue management
 phase/1
queue management
 phase/2
queue management
 phase/3
queue management
 phase/4
queue management
 phase/5
queue management
 phase/6
queue management
 queue/blocked
queue management
 queue/done
queue management
 queue/failed
queue management
 queue/in-progress
queue management
 queue/queued
queue management
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
dohertj2 (dohertj2)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dohertj2/lmxopcua#693
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?