Gateway raw: warm reads refuse a tie cluster that fits under the learned ceiling (look-back below a reverse segment or bound lookup) #695

New Issue

2026-10-13T02:12:07-04:00

dohertj2 commented

2026-10-13 02:12:07 -04:00

Follow-up from #694 (review R1, master 66a2b0c3). Low–medium; not a regression: master before #694 refused more.

Symptom: once the strict ceiling is learned (warm), a tie cluster of roughly 1,024 rows up to the ceiling, sitting in the 1 ms look-back just below a reverse-walk segment or a ReturnBounds bound lookup, is refused BadTooManyOperations. The cold read of the same window succeeds.

Traced cases (reviewer harness ZzReviewFuzz694.cs, FUZZ694_ONLY=<seed>):

seed 1000939: ceiling 1709, cluster 1366, end-only N=249 — cold OK, warm refused;
bounds run seed 1501191: ceiling 1369, cluster 1292 — the start-bound lookup fails after 3 reads.

Cause:

A segment asks 1,026 rows and all 1,042 rows returned are the cluster.
The crowding loop (GatewayHistorianDataSource, ~:222) breaks as soon as the next doubled budget exceeds the known ceiling, without trying the largest budget that still fits.
ReverseRawReader (~:107) retries only when asked − PreWindowRows ≥ 1, measured against the current request rather than the ceiling.
The bounds paths go through neither page-path retry.

Suggested fix:

Before breaking, clamp the room to known − cap − 16 and try once more.
In the reverse reader, shrink to limit − m − 1 from the ceiling, and allow one more shrink if m grew.
Pin both traced seeds.

Follow-up from #694 (review R1, master `66a2b0c3`). Low–medium; not a regression: master before #694 refused more. **Symptom:** once the strict ceiling is learned (warm), a tie cluster of roughly 1,024 rows up to the ceiling, sitting in the 1 ms look-back just below a reverse-walk segment or a ReturnBounds bound lookup, is refused `BadTooManyOperations`. The cold read of the same window succeeds. **Traced cases** (reviewer harness `ZzReviewFuzz694.cs`, `FUZZ694_ONLY=<seed>`): - seed 1000939: ceiling 1709, cluster 1366, end-only N=249 — cold OK, warm refused; - bounds run seed 1501191: ceiling 1369, cluster 1292 — the start-bound lookup fails after 3 reads. **Cause:** - A segment asks 1,026 rows and all 1,042 rows returned are the cluster. - The crowding loop (`GatewayHistorianDataSource`, ~:222) breaks as soon as the next doubled budget exceeds the known ceiling, without trying the largest budget that still fits. - `ReverseRawReader` (~:107) retries only when `asked − PreWindowRows ≥ 1`, measured against the current request rather than the ceiling. - The bounds paths go through neither page-path retry. **Suggested fix:** - Before breaking, clamp the room to `known − cap − 16` and try once more. - In the reverse reader, shrink to `limit − m − 1` from the ceiling, and allow one more shrink if m grew. - Pin both traced seeds.

dohertj2 referenced this issue

2026-10-13 02:12:15 -04:00

Gateway historian: strict lowered ceiling refuses ordinary pages; deep event reads slow (follow-ups from #689–#691 gate) #694

~~dohertj2 referenced this issue 2026-10-13 03:51:22 -04:00~~

Fix historian missing-tag latency, paging limits and response contracts #699

dohertj2 commented

2026-10-13 03:52:23 -04:00

Implemented: crowding re-reads clamp the allowance to the remaining learned-ceiling room and spend that final usable budget before refusing. Crowded reverse segments retry with a reduced in-window budget derived from the learned ceiling and observed pre-window rows. Regression tests cover ceilings 1709/1369 and reverse paging through a crowded millisecond. Ready in the pushed branch; this issue closes when PR #699 merges.

Pushed commit 97668d26; PR: #699.

Full solution verified: 14,581 passed, 609 skipped, 0 failed across 62 suites (dotnet test ZB.MOM.WW.OtOpcUa.slnx --verbosity quiet, exit 0). Independent final review found no important defects. New fixes have not been deployed or live-tested end to end through OPC UA.

Implemented: crowding re-reads clamp the allowance to the remaining learned-ceiling room and spend that final usable budget before refusing. Crowded reverse segments retry with a reduced in-window budget derived from the learned ceiling and observed pre-window rows. Regression tests cover ceilings 1709/1369 and reverse paging through a crowded millisecond. Ready in the pushed branch; this issue closes when PR #699 merges. Pushed commit `97668d26`; PR: https://gitea.dohertylan.com/dohertj2/lmxopcua/pulls/699. Full solution verified: 14,581 passed, 609 skipped, 0 failed across 62 suites (`dotnet test ZB.MOM.WW.OtOpcUa.slnx --verbosity quiet`, exit 0). Independent final review found no important defects. New fixes have not been deployed or live-tested end to end through OPC UA.

dohertj2 closed this issue

2026-10-13 03:53:42 -04:00

dohertj2 referenced this issue

2026-10-13 04:33:35 -04:00

Fix historian missing-tag latency, paging limits and response contracts #699

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dohertj2/lmxopcua#695