Gateway AtTime: prior-sample search starts at the requested time, not min(T, now); strict-ceiling stopped-tag end-only reads costly #696

New Issue

2026-10-13T02:12:07-04:00

dohertj2 commented

2026-10-13 02:12:07 -04:00

Follow-ups from the #692–#694 live AVEVA gate (2026-10-13, master 66a2b0c3). Low; results are correct, cost only.

AtTime in the future on a live tag (gate F1): AtTime at now + 1 h on SysTimeSec (1 Hz) costs 24 reads, ~15k rows and 3.7 s. GatewayAtTimeReader's walk-back for the earlier sample starts at the requested time T rather than min(T, now). The empty future stretch makes it fall through to a whole-remainder read and a bisection over ~3 years. Start the search at min(T, now + small margin); the future-time rule (Uncertain hold) is unchanged.
Strict ceiling, end-only read over a dense tag that stopped (gate F3): 23 reads / 3.25 s against 3 reads on a streaming gateway. Correct, but each walk segment is capped at the ceiling. Consider aiming the first segments from the learned ceiling and density, or reusing the walk hint.
Info: after the ceiling is learned, a per-sample aggregate still makes one refused 1,000,000-row re-probe (#691.3 rule). Consider skipping the re-probe when the ceiling was learned recently.

Evidence: scratchpad livegate-aveva-692.md (F1, F3), logs g692/l693*.log, g692/l694*.log.

Follow-ups from the #692–#694 live AVEVA gate (2026-10-13, master `66a2b0c3`). Low; results are correct, cost only. 1. **AtTime in the future on a live tag (gate F1):** AtTime at now + 1 h on `SysTimeSec` (1 Hz) costs 24 reads, ~15k rows and 3.7 s. `GatewayAtTimeReader`'s walk-back for the earlier sample starts at the requested time T rather than min(T, now). The empty future stretch makes it fall through to a whole-remainder read and a bisection over ~3 years. Start the search at min(T, now + small margin); the future-time rule (Uncertain hold) is unchanged. 2. **Strict ceiling, end-only read over a dense tag that stopped (gate F3):** 23 reads / 3.25 s against 3 reads on a streaming gateway. Correct, but each walk segment is capped at the ceiling. Consider aiming the first segments from the learned ceiling and density, or reusing the walk hint. 3. **Info:** after the ceiling is learned, a per-sample aggregate still makes one refused 1,000,000-row re-probe (#691.3 rule). Consider skipping the re-probe when the ceiling was learned recently. Evidence: scratchpad `livegate-aveva-692.md` (F1, F3), logs `g692/l693*.log`, `g692/l694*.log`.

dohertj2 referenced this issue

2026-10-13 02:12:14 -04:00

Gateway AtTime returns AVEVA interpolation unchanged (NULL handling, drift, no Interpolated bit) #693

dohertj2 referenced this issue

2026-10-13 02:12:15 -04:00

Gateway historian: strict lowered ceiling refuses ordinary pages; deep event reads slow (follow-ups from #689–#691 gate) #694

dohertj2 referenced this issue

2026-10-13 03:51:22 -04:00

Fix historian missing-tag latency, paging limits and response contracts #699

dohertj2 commented

2026-10-13 03:52:23 -04:00

Implemented item 1: a bounded future-interval probe preserves stored future samples, then starts prior-sample search near now when the interval is empty. Requested timestamps and Uncertain hold status remain intact. Tests cover dense live history (at most four raw reads), stored future samples and a full future probe.

Items 2 (stopped-tag strict-ceiling end-only cost) and 3 (per-sample aggregate ceiling re-probe) are not changed. Leaving issue open for those remaining performance items.

Pushed commit 97668d26; PR: #699.

Full solution verified: 14,581 passed, 609 skipped, 0 failed across 62 suites (dotnet test ZB.MOM.WW.OtOpcUa.slnx --verbosity quiet, exit 0). Independent final review found no important defects. New fixes have not been deployed or live-tested end to end through OPC UA.

Implemented item 1: a bounded future-interval probe preserves stored future samples, then starts prior-sample search near now when the interval is empty. Requested timestamps and Uncertain hold status remain intact. Tests cover dense live history (at most four raw reads), stored future samples and a full future probe. Items 2 (stopped-tag strict-ceiling end-only cost) and 3 (per-sample aggregate ceiling re-probe) are not changed. Leaving issue open for those remaining performance items. Pushed commit `97668d26`; PR: https://gitea.dohertylan.com/dohertj2/lmxopcua/pulls/699. Full solution verified: 14,581 passed, 609 skipped, 0 failed across 62 suites (`dotnet test ZB.MOM.WW.OtOpcUa.slnx --verbosity quiet`, exit 0). Independent final review found no important defects. New fixes have not been deployed or live-tested end to end through OPC UA.

dohertj2 commented

2026-10-13 04:34:35 -04:00

Remaining items fixed:

Strict-ceiling stopped-tag reverse discovery retains truncated remainder-probe rows/density and uses the available discovery budget for gaps, rather than throwing that evidence away. Only complete segments enter a page; order/ties/bounds/continuation hints remain exact. Regression: 7,200 stopped 1 Hz rows, ceiling 1,000, latest three requested one day later, baseline 17 calls; now at most 12 with exact values 7199/7198/7197. The Gateway suite covers existing paging/tie cases as well.
Per-sample aggregates reuse a recently refused ceiling for 30 seconds before re-probing an incomplete stream. A raised ceiling is noticed after expiry; a lowered ceiling is learned immediately. Oversized reads remain BadTooManyOperations during the cooldown, never truncated answers. Raw/AtTime immediate re-probes remain intact. Regression checks no second refusal in the cooldown and success after a raised limit at expiry.

Item 1 was fixed in 97668d26 (bounded future probe and near-now prior search). All three items are now handled. Closing this issue.

Pushed commit 58977640 on fix/historian-653-followups, included in PR #699: #699 (awaiting merge).

Full final-tree solution: 14,600 passed, 609 skipped, 0 failed across 62 suites (dotnet test ZB.MOM.WW.OtOpcUa.slnx --verbosity quiet, exit 0). Final Gateway suite: 434 passed / 6 environment-gated skips. Independent final review found no important defects.

Evidence: docs/plans/2026-10-13-historian-653-696-results.md.

Remaining items fixed: 2. Strict-ceiling stopped-tag reverse discovery retains truncated remainder-probe rows/density and uses the available discovery budget for gaps, rather than throwing that evidence away. Only complete segments enter a page; order/ties/bounds/continuation hints remain exact. Regression: 7,200 stopped 1 Hz rows, ceiling 1,000, latest three requested one day later, baseline 17 calls; now at most 12 with exact values 7199/7198/7197. The Gateway suite covers existing paging/tie cases as well. 3. Per-sample aggregates reuse a recently refused ceiling for 30 seconds before re-probing an incomplete stream. A raised ceiling is noticed after expiry; a lowered ceiling is learned immediately. Oversized reads remain BadTooManyOperations during the cooldown, never truncated answers. Raw/AtTime immediate re-probes remain intact. Regression checks no second refusal in the cooldown and success after a raised limit at expiry. Item 1 was fixed in 97668d26 (bounded future probe and near-now prior search). All three items are now handled. Closing this issue. Pushed commit `58977640` on `fix/historian-653-followups`, included in PR #699: https://gitea.dohertylan.com/dohertj2/lmxopcua/pulls/699 (awaiting merge). Full final-tree solution: 14,600 passed, 609 skipped, 0 failed across 62 suites (`dotnet test ZB.MOM.WW.OtOpcUa.slnx --verbosity quiet`, exit 0). Final Gateway suite: 434 passed / 6 environment-gated skips. Independent final review found no important defects. Evidence: `docs/plans/2026-10-13-historian-653-696-results.md`.

dohertj2 closed this issue

2026-10-13 04:34:54 -04:00

dohertj2 commented

2026-10-13 04:49:10 -04:00

Merged PR #699 into master as 618d98e822. The merged tree exactly matches tested 58977640: full solution 14,600 passed, 609 skipped, zero failed across 62 suites. #653 and #695–#698 are closed; #694 remains open for historiangw#27.

Pending findings are now tracked explicitly:

lmxopcua#700: deployed OPC UA/live historian verification, after historiangw#29 restores the existing-tag control.
lmxopcua#701: optional concurrent cold/expiry cache coverage.
historiangw#29: live existing-tag Raw deadline failures.
historiangw#30: deploy/verify the existing missing-metadata sentinel fix.
historiangw#31: fail on a premature empty native catalog page (source-inspection finding; not reproduced live).
historiangw#32: correct API-key rotation flags and verify deployed unknown-ID revoke reporting.

No deployment changes were made. All temporary test keys were independently verified revoked.

Merged PR #699 into master as 618d98e8224ce1989677590094627e924c644c91. The merged tree exactly matches tested 58977640: full solution 14,600 passed, 609 skipped, zero failed across 62 suites. #653 and #695–#698 are closed; #694 remains open for historiangw#27. Pending findings are now tracked explicitly: - lmxopcua#700: deployed OPC UA/live historian verification, after historiangw#29 restores the existing-tag control. - lmxopcua#701: optional concurrent cold/expiry cache coverage. - historiangw#29: live existing-tag Raw deadline failures. - historiangw#30: deploy/verify the existing missing-metadata sentinel fix. - historiangw#31: fail on a premature empty native catalog page (source-inspection finding; not reproduced live). - historiangw#32: correct API-key rotation flags and verify deployed unknown-ID revoke reporting. No deployment changes were made. All temporary test keys were independently verified revoked.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dohertj2/lmxopcua#696