dohertj2/lmxopcua
1
0
Fork 0
Code Issues 43 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(historian): ServerHistorian:MaxRawValuesPerRead caps raw pages and backend reads

Browse Source 
Review R08:1.1: an uncapped HistoryRead-Raw (NumValuesPerNode = 0) read a
tag's whole window into memory on every backend (SQL TOP(int.MaxValue),
QuestDB LIMIT, Gateway max_values = int.MaxValue + a second snapshot copy).

New ServerHistorian:MaxRawValuesPerRead (default 10 000; <= 0 unbounded with
a Validate() warning; 1 raised to 2). Each backend data source takes it,
reports it as IHistorianDataSource.MaxRawValuesPerRead (the Gateway: min
with a strict gateway's learned ceiling), so the node manager's existing
RawPageLimit paging serves N = 0 and N > cap as pages of at most the cap
with a continuation point. Independently, a backend raw read asking for 0
or more than the cap reads cap + 1 rows and is REFUSED
(HistoryReadLimitExceededException => BadTooManyOperations) when the window
holds more - never answered truncated, since the server's paging reads a
short answer as the last page. Bound through the Host registry, the
SqlServer/QuestDb providers and GatewayHistorian.CreateDataSource.

Tests: options (default/binding/warnings), provider binding, Gateway unit +
through a real SDK node manager (uncapped -> 999/998/998/605 + CPs), and a
conformance case for the SQL backends.

Refs #736

Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
This commit is contained in:
Joseph Doherty
2026-10-22 06:29:10 -04:00
parent ad2b5c4044
commit 6669fae7e5
17 changed files with 621 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/Drivers/Historian.Conformance/IConformanceBackend.cs
+8 -3
View File
@@ -22,9 +22,14 @@ public interface IConformanceBackend : IAsyncDisposable
/// <param name="clock">Its "now"; null means the system clock.</param>
IHistorianDataSource CreateDataSource(TimeProvider? clock = null);
/// <summary>A fresh data source whose time-weighted raw-row cap (<c>ServerHistorian:MaxTimeWeightedRawRows</c>) is <paramref name="maxTimeWeightedRawRows"/>.</summary>
/// <param name="maxTimeWeightedRawRows">The cap (&lt;= 0 = unbounded).</param>
IHistorianDataSource CreateDataSource(int maxTimeWeightedRawRows);
/// <summary>
/// A fresh data source whose time-weighted raw-row cap (<c>ServerHistorian:MaxTimeWeightedRawRows</c>) is
/// <paramref name="maxTimeWeightedRawRows"/> and whose server raw cap (<c>ServerHistorian:MaxRawValuesPerRead</c>,
/// R08:1.1 / #736) is <paramref name="maxRawValuesPerRead"/>.
/// </summary>
/// <param name="maxTimeWeightedRawRows">The time-weighted cap (&lt;= 0 = unbounded).</param>
/// <param name="maxRawValuesPerRead">The raw cap (&lt;= 0 = unbounded, the default).</param>
IHistorianDataSource CreateDataSource(int maxTimeWeightedRawRows, int maxRawValuesPerRead = 0);
/// <summary>A fresh live-value writer over this backend's store.</summary>
/// <param name="utcNow">The clock used to stamp values that carry no timestamp; null means the system clock.</param>
tests/Drivers/Historian.Conformance/RawReadConformance.cs
+37
View File
@@ -25,6 +25,43 @@ public abstract class RawReadConformance : ConformanceBase
r.ContinuationPoint.ShouldBeNull();
}
/// <summary>
/// R08:1.1 (#736): the server raw cap bounds the backend's own <c>TOP</c> / <c>LIMIT</c>. A read asking for everything
/// (or for more than the cap) reads at most cap + 1 rows and is refused when the window holds more — never answered
/// truncated (a short answer is what the server's paging reads as "the last page") — and is served in full when it fits.
/// A read at or under the cap is unchanged. The refusal is a client-request limit, not a backend fault.
/// </summary>
[Fact]
public async Task The_server_raw_cap_bounds_every_read_and_refuses_an_over_cap_window_rather_than_truncating()
{
await ConformanceData.SeedAAsync(B); // five samples at +0, +10, +20, +30, +40
var src = B.CreateDataSource(maxTimeWeightedRawRows: 1_000_000, maxRawValuesPerRead: 3);
src.MaxRawValuesPerRead.ShouldBe(3);
// Five rows under a cap of three: everything / above the cap ⇒ refused; at the cap ⇒ the first three; under ⇒ as asked.
var refused = await Should.ThrowAsync<HistoryReadLimitExceededException>(() => src.ReadRawAsync("A", At(-1), At(100), 0, Ct));
refused.Limit.ShouldBe(3);
await Should.ThrowAsync<HistoryReadLimitExceededException>(() => src.ReadRawAsync("A", At(-1), At(100), 4, Ct));
await Should.ThrowAsync<HistoryReadLimitExceededException>(() => src.ReadRawAsync("A", At(-1), At(100), uint.MaxValue, Ct));
Values(await src.ReadRawAsync("A", At(-1), At(100), 3, Ct)).ShouldBe([10d, 20d, 30d]);
Values(await src.ReadRawAsync("A", At(-1), At(100), 2, Ct)).ShouldBe([10d, 20d]);
// Three rows fit: everything / above the cap are served in full.
Values(await src.ReadRawAsync("A", At(10), At(40), 0, Ct)).ShouldBe([20d, 30d, 40d]);
Values(await src.ReadRawAsync("A", At(10), At(40), 4, Ct)).ShouldBe([20d, 30d, 40d]);
// Reverse reads are bounded the same way, keeping the LATEST rows.
await Should.ThrowAsync<HistoryReadLimitExceededException>(() => src.ReadRawAsync("A", At(100), At(-1), 0, Ct));
Values(await src.ReadRawAsync("A", At(100), At(-1), 3, Ct)).ShouldBe([50d, 40d, 30d]);
Values(await src.ReadRawAsync("A", At(40), At(10), 0, Ct)).ShouldBe([50d, 40d, 30d]);
// A point read and an empty window are unaffected.
Values(await src.ReadRawAsync("A", At(10), At(10), 0, Ct)).ShouldBe([20d]);
(await src.ReadRawAsync("A", At(50), At(60), 0, Ct)).Samples.ShouldBeEmpty();
src.GetHealthSnapshot().TotalFailures.ShouldBe(0, "a refused over-cap read is a client-request limit, not a backend fault");
}
[Fact]
public async Task Forward_read_honours_the_limit()
{