Merge feat/691-return-bounds: Part 11 ReturnBounds on every backend (#691.1)

This commit is contained in:
Joseph Doherty
2026-10-12 18:14:06 -04:00
21 changed files with 1336 additions and 17 deletions
+2
View File
@@ -1355,6 +1355,7 @@ historian tagname, which is the tag's driver `FullName`.
- ⚠️ **Reverse processed reads are Part 13/SDK-anchored (#677, client-visible):** every aggregate on SqlServer/QuestDB now anchors a reverse read at its later time — window `(end, start]`, partial interval earliest, each stamped with its later edge (was: earlier-anchored, stamped earlier edge); on forward reads Partial also marks the interval the data starts inside (not Average — the SDK's rule, forward-only); a non-default client `AggregateConfiguration` ⇒ `BadAggregateConfigurationRejected`; Gateway provisioning sends `IntegralDivisor = 1`. SDK-parity tests: `SdkAggregateParityTests`. `docs/Historian.md` §Aggregates.
- ⚠️ **Reverse / end-only RAW reads work on Gateway (#686, client-visible):** `IHistorianDataSource.SupportsReverseRaw` (SqlServer/QuestDB/Null `true`, Gateway `false`, interface default `false`, guard-tested) — the node manager sends `start > end` only to a backend that declares it; otherwise `ReverseRawReader` serves it from FORWARD reads: uncapped ⇒ ONE forward read reversed in memory; capped ⇒ a bounded walk back in segments (ms-aligned request bounds, width aimed from observed density at ≈ N + 8 rows and carried page to page in the continuation point — #689: it used to aim at half `MaxTieClusterOverfetch`, ~32k rows per page on a dense tag; a read asks for `2N + 18` rows (a discovery segment ≤ 1,026), `MaxTieClusterOverfetch` is only the tie-cluster ceiling — a full read is re-read, never answered truncated; only more than max(1, `MaxTieClusterOverfetch`) samples within ONE MILLISECOND ⇒ `BadTooManyOperations` (per ms, not per timestamp — requests are ms-rounded, and letting a segment narrow below 1 ms stepped a page through one ms a tick per read, ~10k reads); walks no later than now + 1 day and never before the Unix epoch; #691: after two empty segments it asks once for the whole remainder, and the stretch a page proved empty rides the continuation point — the last pages of a deep page-through no longer each walk ~14 reads to 1970). The raw/events tie-path oversized-cluster refusal is now `BadTooManyOperations` too (was `BadHistoryOperationUnsupported`). Uncapped Gateway raw reads (forward too) used to send `max_values = 0`, which the client/gateway reject — now unbounded (`int.MaxValue`; a ceiling refusal ⇒ `BadTooManyOperations`), and a cancelled read is no longer a backend failure. Since #669 such reads failed the node on Gateway and counted as a backend fault (the gateway rejects start > end — `GatewayHistorianDataSource.ReadRawAsync` now refuses one unsent); before #669 an end-only Gateway read returned the OLDEST N ascending — now the latest N at/before EndTime, newest first. Events follow since #691.4: a reverse/end-only capped events read on Gateway (`SupportsReverseEvents` false; Null now reports `true` and answers empty) is walked back the same way (ties sorted by EventId — the gateway orders by time only; one extra event found ⇒ continuation point) — it used to be refused `BadHistoryOperationUnsupported` (explicit reverse) or answered with the OLDEST N ascending (end-only); needs the gateway's `RuntimeDb:EventReadMaxRows` 0/unlimited (it truncates silently). Uncapped Gateway events reads (forward too) used to throw "Array dimensions exceeded" (result list pre-sized to `int.MaxValue`) — fixed. Above a truncated read the walk gallops up from the newest row seen, then bisects (never below 1 ms): a burst that ended 100 days ago ≈ 18 reads, a dense tag that stopped days ago ≈ 26. `docs/Historian.md` §"Open-ended and reverse raw reads".
- **Advertised aggregates (#682):** `IHistorianDataSource.SupportedAggregates` (Gateway 4 via `GatewayHistorianDataSource.ServedAggregates`, SqlServer/QuestDB 17 via `AggregateFamilies.Served`, Null **none**; a null set ⇒ all) is the one authority for `HistoryServerCapabilities/AggregateFunctions` (i=11201 — only its folder references are trimmed) and the up-front `BadAggregateNotSupported` in HistoryRead-Processed. ⚠️ `ServerCapabilities/AggregateFunctions` (i=2997) is deliberately NOT trimmed: it keeps the SDK's 37 because live `AggregateFilter` subscriptions still compute them all. ⚠️ Historian disabled ⇒ Processed is now `BadAggregateNotSupported`, not `GoodNoData`. `docs/Historian.md` §Advertised aggregates.
- **Raw `ReturnBounds` is honoured on every backend (#691, client-visible):** Part 11 bounding values (§4.6 Table 1 is the test oracle) are composed in the server by `HistoryBounds` from each backend's raw reads — "at or before" = capped reverse read (native on SQL backends, the #686 `ReverseRawReader` walk-back on Gateway, never AVEVA's synthetic boundary row), "at or after" = capped forward read. Bounds are stored samples of any quality, count toward `NumValuesPerNode`, start bound on the first page / end bound on the last (pending end bound rides the continuation point); missing ⇒ `Bad_BoundNotFound` (null, at the bound time; a one-sided read's open side ±1 s from the previous value). ⚠️ **Breaking for SDK-default clients:** the .NET SDK's `new ReadRawModifiedDetails()` defaults BOTH `ReturnBounds = true` AND `IsReadModified = true` — a client that cleared only `IsReadModified` now gets bounds, and one that set neither is a modified read with bounds ⇒ `Bad_InvalidArgument` (§6.5.3.3); set both explicitly (Client.Shared does; tests building details must too). No data AND no bound found ⇒ `GoodNoData`, no values (the pre-bounds answer). On Gateway the at-or-after lookup ends at now + 1 day (never a year-9999 end to AVEVA). Kill switch `ServerHistorian:HonorReturnBounds` (default `true`; `false` = pre-#691). `docs/Historian.md` §Bounding values.
- **Real status fix:** the recorder used to stamp every sample Good (`192`); it now carries the driver's status. ⚠️ On the **Gateway** backend this changes nothing stored: the gateway discards quality (C-002), so AVEVA still stores Good.
- **Comms loss is historized as an outage marker** (`Empty` + `BadNoCommunication`, one per tag per outage) on SqlServer; dropped (`DroppedUnsupportedKind`) on Gateway. Outbox records are **format v2** — drain or delete `ContinuousHistorization:OutboxPath` before downgrading a node.
- **HistoryUpdate** (SqlServer + QuestDb; Gateway ⇒ `BadHistoryOperationUnsupported`): Insert/Replace/Update/DeleteRaw/DeleteAtTime, **Primary-only** since #674 (off-Primary ⇒ `BadInvalidState`, after authorization AND after planning — an unservable request (Remove / `IsDeleteModified` / not historized) stays `BadHistoryOperationUnsupported` off-Primary; the verdict is `IHistoryUpdateRoleView` = the data-plane verdict but **closed until this node's first redundancy snapshot**, so a pair cannot both accept in the boot window; admin-only nodes register no view ⇒ no gate; reads ungated), items with a `SourceTimestamp` past now + `ServerHistorian:HistoryUpdateMaxFutureSkew` (default 1 day, ≤0 disables) ⇒ `BadOutOfRange` in the node manager (every backend), gated in both ACL modes — flag off needs the `HistoryUpdate` role from a **Cluster-scoped** grant (+ key re-check), flag on uses `DataPlaneAuthorizer`'s HistoryUpdate operation; `WriteOperate` never implies it. The `HistoryWrite` AccessLevel bit is set only when an updater is wired at materialise time. `docs/Historian.md` §HistoryUpdate, `docs/security.md`. ⚠️ **SqlServer schema v1 is accepted (#674)** with `AutoCreateSchema=false`: writes/reads/events/retention work; modified reads + HistoryUpdate ⇒ node-level `BadHistoryOperationUnsupported` on every arm (the updater throws `NotSupportedException`, nothing sent; the `HistoryWrite` bit is still advertised — not knowable at materialise time), retention skips `SampleModified`, one Warning per boot (`SqlServerSchemaBootstrapper.SupportsModifiedTrail`; `RequiredVersion` = 1, `ModifiedTrailVersion` = 2). QuestDB: keep `cairo.ttl.use.wall.clock=true`; a QuestDB trail INSERT retried after a client timeout duplicates (no DEDUP key).
@@ -1381,6 +1382,7 @@ authorization uses the standard `AccessLevels.HistoryRead` bit set at materializ
| `CaCertificatePath` | `null` | PEM CA file pinning the gateway TLS chain; null/empty uses the OS trust store |
| `CallTimeout` | `00:00:30` | Per-call deadline for each unary gateway read |
| `MaxTieClusterOverfetch` | `65536` | Bounded over-fetch the HistoryRead-Raw paging uses to page within an oversized same-timestamp tie cluster (retained from the prior backend) |
| `HonorReturnBounds` | `true` | #691 kill switch: `false` ignores HistoryRead-Raw `ReturnBounds` as before (no bounding values, no modified-read `Bad_InvalidArgument`) |
| `HistoryReadBatchParallelism` | `4` | Max nodes served concurrently within one HistoryRead batch (03/S3); `⌈N/P⌉ × CallTimeout` worst case instead of sequential |
| `MaxConcurrentHistoryReadBatches` | `16` | Process-wide concurrent HistoryRead batch cap (03/S3); flood degrades to `BadTooManyOperations`. Total in-flight reads capped at 16 × 4 = 64 |
| `MaxTimeWeightedRawRows` | `1000000` | `SqlServer`/`QuestDb` (#671): raw in-window rows one time-weighted aggregate read (incl. `Total` and, since #680, PercentGood/PercentBad) may stream into the C# evaluator — and, since #681, every `Gateway` per-sample read (raw-evaluated); window query capped at this + 1, past it ⇒ `BadTooManyOperations` for that node. ≤ 0 = unbounded. Gateway (#688): the `cap + 18` budget is clamped at the gateway's default `MaxValuesCeiling` (1,000,000) while the cap is ≤ it, so the default no longer trips a strict (`AllowStreamingBeyondCeiling=false`) gateway |
+58 -6
View File
@@ -111,6 +111,7 @@ Processed read is `BadAggregateNotSupported` and no history aggregate functions
"CaCertificatePath": null,
"CallTimeout": "00:00:30",
"MaxTieClusterOverfetch": 65536,
"HonorReturnBounds": true,
"HistoryReadBatchParallelism": 4,
"MaxConcurrentHistoryReadBatches": 16,
"HistoryReadDeadline": "00:01:00",
@@ -144,6 +145,7 @@ Processed read is `BadAggregateNotSupported` and no history aggregate functions
| `CaCertificatePath` | string\|null | `null` | **Gateway backend only.** PEM CA file pinning the gateway's TLS chain. Null/empty uses the OS trust store. |
| `CallTimeout` | TimeSpan | `00:00:30` | **Gateway backend only.** Per-call deadline applied to each unary gateway read. A read for a tag the historian does not yet know about — e.g. a freshly-historized tag that has **not** been provisioned (see [Tag auto-provisioning](#tag-auto-provisioning-ensuretags)) — can run to this **full** deadline before the gateway errors, so an unprovisioned-tag HistoryRead can block for ~30 s. Lower this to fail faster, at the cost of truncating legitimately long reads. |
| `MaxTieClusterOverfetch` | int | `65536` | (Node-manager paging; all backends — a `SqlServer` PK rules out tie clusters, so it is moot there.) Maximum samples the server will fetch in one shot to page through a tie cluster (multiple samples sharing one `SourceTimestamp`). A cluster larger than this ceiling fails `BadTooManyOperations` (#686; was `BadHistoryOperationUnsupported`). Also the per-read row budget of a Gateway reverse raw read (walked back from forward reads, #686): one timestamp holding more samples is refused `BadTooManyOperations`. Values below 1 count as 1. Raise to handle abnormally large tie clusters; the default covers all normal-data cases. |
| `HonorReturnBounds` | bool | `true` | (#691, all backends.) Whether HistoryRead-Raw honours `ReturnBounds` (Part 11 bounding values, [Bounding values](#bounding-values-returnbounds-691)). `false` is the kill switch back to the pre-#691 behaviour: the flag is ignored (stored samples only) and a modified read carrying it is not refused `Bad_InvalidArgument`. Note the .NET SDK defaults both `ReturnBounds` and `IsReadModified` to `true`. |
| `HistoryReadBatchParallelism` | int | `4` | Max historized nodes served **concurrently** within one HistoryRead batch (arch-review 03/S3). The SDK's `HistoryRead*` override surface is synchronous, so each arm bridges once at its boundary; this turns per-node serving from sequential (`N × CallTimeout` worst case) into `⌈N / P⌉ × CallTimeout`. Also caps the gateway-load multiplication a single client can command. `≤ 1` serves sequentially. |
| `MaxConcurrentHistoryReadBatches` | int | `16` | Process-wide limit on HistoryRead (and HistoryUpdate) batches served **concurrently** (arch-review 03/S3). A flood of history clients degrades to `BadTooManyOperations` rather than exhausting the SDK request-thread pool. With `HistoryReadBatchParallelism` this caps total in-flight gateway reads at `MaxConcurrentHistoryReadBatches × HistoryReadBatchParallelism` (default 16 × 4 = 64). A batch that cannot get a slot within `min(HistoryReadDeadline, 5 s)` fails every handle `BadTooManyOperations`. |
| `MaxTimeWeightedRawRows` | int | `1000000` | `SqlServer` / `QuestDb` (#671), and the `Gateway` per-sample aggregates (#681 — every per-sample read there streams raw samples, so the cap applies to the window's raw rows; see [Aggregate set](#aggregate-set-and-badaggregatenotsupported-285)). The most raw in-window samples one [time-weighted aggregate](#time-weighted-aggregates-part-13-671) read (TimeAverage, TimeAverage2, **Total**, Total2, Interpolative, and — since #680 — **PercentGood/PercentBad**) may stream into the server-side evaluator — that family is **not** pushed down to SQL. The window query is capped at this + 1 rows; one row past the cap fails **that node** with `BadTooManyOperations` (not counted as a backend fault) instead of holding a pooled connection until `HistoryReadDeadline` and returning `BadTimeout`. Bound rows outside the window do not count; per-sample aggregates are not capped on the SQL backends (pushed down). 1 000 000 ≈ 11.5 days of a 1 Hz tag. Non-positive = unbounded (a `Validate()` warning). `Gateway` (#688): the row budget it asks the gateway for (`cap + 18`) is clamped at the gateway's default `MaxValuesCeiling` (1,000,000) while the cap is at or under it, so the default works against a gateway with `AllowStreamingBeyondCeiling=false`; against a gateway with a *lower* ceiling the refused budget is retried once at the ceiling the gateway names (#691.3), so no change is needed (`ceiling − 18` only saves that round-trip). |
@@ -752,10 +754,60 @@ data.
asking for `NumValuesPerNode + m + 16` (m = the pre-window rows the first read saw, unclamped). Only if that re-read is still
short because of pre-window rows does the node fail `BadTooManyOperations` (not a backend fault) rather than answer a short page
that would end the client's paging early. Every gateway stream is still drained to completion.
- **`ReturnBounds` is not honoured** — on any backend: the server does not pass it to the data source, so no bounding values are
returned (see [Known limitations](#known-limitations)). The AVEVA boundary row is not used as a bound either: it is stamped at the
query start rather than carrying the time of the real prior sample, so it could only ever be a Part 11 *interpolated* bound, which
this backend does not compute.
- **`ReturnBounds` is served by the server, not by this row** (#691, [Bounding values](#bounding-values-returnbounds-691)). The AVEVA
boundary row is never used as a bound: it is stamped at the query start rather than carrying the time of the real prior sample, so
it could only ever be a Part 11 *interpolated* bound. The start bound on the Gateway is the real prior sample, found by the
#686 walk-back.
### Bounding values (`ReturnBounds`, #691)
⚠️ **Client-visible — a breaking change for .NET clients that rely on SDK defaults.** Until #691
`ReadRawModifiedDetails.ReturnBounds` was ignored on every backend. It is now honoured on **every** backend (SqlServer, QuestDB,
Gateway, and none), following OPC UA Part 11 §4.6 / §6.5.3.2 and checked against every row of Part 11 Table 1. **The OPC Foundation
.NET SDK's `new ReadRawModifiedDetails()` defaults BOTH `ReturnBounds = true` AND `IsReadModified = true`** (verified on
1.5.378.106), so:
- a client that clears `IsReadModified` but never sets `ReturnBounds` used to get stored values only and **now gets bounding values**;
- a client that sets neither (a bare `new ReadRawModifiedDetails { StartTime, EndTime }`) is a *modified* read with bounds, which
used to be served (or `BadHistoryOperationUnsupported` on a backend with no trail) and is **now `Bad_InvalidArgument`** (§6.5.3.3).
Set both explicitly. `Client.Shared` (Client.CLI / Client.UI) already does, and ScadaBridge does not issue HistoryRead. The kill
switch `ServerHistorian:HonorReturnBounds = false` restores the pre-#691 behaviour (both changes) fleet-wide.
- **What a bound is.** Forward (`StartTime < EndTime`): the start bound is the stored sample **at or immediately before**
StartTime, the end bound the one **at or immediately after** EndTime. Reverse mirrors it (start bound at or after StartTime, end
bound at or before EndTime), newest first. A sample exactly **on** StartTime is the bound and is already the data's first value
(nothing extra is added); a sample exactly on EndTime is added as the end bound (the data window excludes EndTime).
`StartTime == EndTime` reads forward: the data is the samples at that instant, the end bound the first sample strictly after it.
- **A bound is a stored sample of any quality** — a Bad sample or a comms-loss outage marker comes back as stored. Inside a tie
cluster the bound is the cluster's first sample in the read's direction.
- **Nothing at all ⇒ `GoodNoData`.** A read with no data in the window **and** neither bound found returns no values and
`GoodNoData` — the answer it gave before bounds (§6.5.3.2: bounds make the result a success only "if one or both exist"). This is
judged over the whole read (it only arises on a single, final first page); once any bound or value exists, a missing bound is
returned as its placeholder.
- **Missing bound:** `Bad_BoundNotFound`, null value, stamped with StartTime / EndTime. A one-sided read's open side (only a
StartTime, or only an EndTime) is never looked up; it ends the read as `Bad_BoundNotFound` stamped one second after (forward) or
before (reverse) the previous returned value (Table 1 footnotes a/b). `Bad_BoundNotSupported` is never returned — every backend
can find a bound.
- **Counting and paging.** Bounds **count toward `NumValuesPerNode`** (§6.5.3.2: "any Bounding Values returned are included in the
numValuesPerNode count"; `NumValuesPerNode = 1` returns the start bound alone). The start bound leads the **first** page and the
end bound ends the **last**: the pending end bound rides the continuation point, and when a page is exactly full the end bound is
the whole next page. Paging a bounded read at any page size concatenates to the unpaged answer.
- **How it is served — one authority, `HistoryBounds` in the OPC UA server.** "At or before T" is a newest-first read of `(0, T]`
capped at one — native on SqlServer/QuestDB, the #686 walk-back of bounded forward reads on the Gateway (a tag with no sample
before StartTime is proven empty down to 1970 by the #689 walk — after two empty segments it asks once for the whole remainder —
so a missing start bound costs a few small reads, not one per ×4 step). "At or after T" is a forward read
of `[T, MaxValue)` capped at one — on the **Gateway** it ends at **now + 1 day** instead (the walk-back's horizon; the
HistorianGateway only checks `start ≤ end`, so a year-9999 end would reach AVEVA), and an end bound beyond that horizon is
`Bad_BoundNotFound` without a read. SqlServer/QuestDB keep `MaxValue` (they bind it safely and may hold samples further ahead).
"Strictly after" starts one storage resolution later. On QuestDB, whose window bounds truncate to
the microsecond, "on the bound" compares at that resolution, so a sample at ⌊StartTime⌋ is never returned twice. A first page
therefore costs up to three backend reads (start bound, data, end bound). The `BoundingValueConformance` suite pins those exact
lookup reads on SqlServer and QuestDB.
- **Modified reads:** `IsReadModified = true` with `ReturnBounds = true` is `Bad_InvalidArgument` per node (Part 11 §6.5.3.3).
- **Kill switch:** `ServerHistorian:HonorReturnBounds` (default `true`). `false` ignores `ReturnBounds` exactly as before #691 —
stored samples only, and no modified-read rejection.
- **Not affected:** Processed, AtTime and Events reads (they have no `ReturnBounds`).
### Continuation-point paging (Events) (#669)
@@ -1095,8 +1147,8 @@ whatever the SDK sets — #682 touches only the i=11201 folder (follow-up candid
sample per requested timestamp, so the single-shot backend returns the complete result in one
read and there is no "full page ⇒ maybe more" signal to page on. Returning the full result with
no continuation point is spec-conformant.
- **`ReturnBounds` is ignored** (every backend): Raw reads return stored values only; no bounding values are added at the window
edges. The node manager does not pass the flag to `IHistorianDataSource`.
- **Bounds are raw only:** `ReturnBounds` (#691, [Bounding values](#bounding-values-returnbounds-691)) never interpolates a bound —
it is always a stored sample or `Bad_BoundNotFound`.
- **No modified-value history on the `Gateway` backend.** Modified reads (`IsReadModified = true`) are served by
`SqlServer` and `QuestDb` ([Modified-value history](#modified-value-history-part-11), #668); the HistorianGateway exposes no
modified/edited-history surface, so on `Gateway` (and with no historian) they return `BadHistoryOperationUnsupported`.
@@ -317,6 +317,9 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl
// section is absent; NodeManager is non-null here (guarded above).
_server.NodeManager.MaxTieClusterOverfetch = _serverHistorianOptions.MaxTieClusterOverfetch;
// Gitea #691: the ReturnBounds kill switch (default on).
_server.NodeManager.HonorReturnBounds = HonorReturnBounds;
// Propagate the S3 HistoryRead concurrency bounds (arch-review 03/S3): bounded per-node fan-out
// within a batch, the process-wide concurrent-batch limiter, and the per-request deadline. Defaults
// (4 / 16 / 60 s) survive when the ServerHistorian section is absent.
@@ -347,6 +350,12 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl
_logger.LogInformation("OtOpcUaServerHostedService: SDK started, address-space + ServiceLevel sinks bound");
}
/// <summary>
/// Gitea #691 — the <c>ServerHistorian:HonorReturnBounds</c> kill switch <c>StartAsync</c> applies to the node manager
/// (default <c>true</c> when the section or key is absent). Exposed so the wiring is testable without booting the SDK.
/// </summary>
internal bool HonorReturnBounds => _serverHistorianOptions.HonorReturnBounds;
/// <summary>
/// Gitea #674 — what <c>StartAsync</c> wires onto the node manager for HistoryUpdate: the Primary gate (read from
/// the <see cref="IHistoryUpdateRoleView"/> on every call; null when none is registered, i.e. allow), the
@@ -0,0 +1,219 @@
using Opc.Ua;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer;
/// <summary>
/// The end bound still owed to a HistoryRead-Raw read with <c>ReturnBounds = true</c> (Gitea #691). It rides the
/// continuation point, because Part 11 returns the end bound with the LAST page only.
/// </summary>
/// <param name="AtUtc">The request's end time (the later bound of a forward read, the earlier of a reverse one).</param>
/// <param name="Reverse">The read is newest-first (<c>StartTime &gt; EndTime</c>, including an end-only read).</param>
/// <param name="Inclusive">
/// <c>false</c> only for <c>StartTime == EndTime</c>: the samples AT that instant are the data, so the end bound is the
/// first sample strictly after it (Part 11 Table 1 footnote c). Otherwise the end bound is the sample at or beyond the
/// end time, and the data window excludes the end time.
/// </param>
/// <param name="Open">The end time was not given (a one-sided read's sentinel): there is no end bound to look up.</param>
/// <param name="DataExhausted">The data is drained and the end bound did not fit on the previous page.</param>
/// <param name="LastReturnedUtc">The last value's time returned on an earlier page (stamps an open bound's placeholder).</param>
internal sealed record HistoryEndBound(
DateTime AtUtc, bool Reverse, bool Inclusive, bool Open, bool DataExhausted = false, DateTime? LastReturnedUtc = null);
/// <summary>
/// Gitea #691: OPC UA Part 11 bounding values for HistoryRead-Raw (<c>ReadRawModifiedDetails.ReturnBounds</c>), composed
/// in the server from every backend's existing raw read — the ONE authority, so SqlServer, QuestDB, Gateway and Null
/// answer identically.
/// </summary>
/// <remarks>
/// <para>
/// <b>What a bound is</b> (Part 11 §4.6, Table 1). Forward: the start bound is the stored sample AT or immediately
/// BEFORE StartTime, the end bound the one AT or immediately AFTER EndTime. Reverse mirrors it: the start bound is at
/// or after StartTime, the end bound at or before EndTime. A sample exactly on a bound IS the bound: on the start it
/// is already the first sample of the data (the window includes StartTime), so nothing extra is added; on the end it is
/// added (the window excludes EndTime). <c>StartTime == EndTime</c> reads forward, and its end bound is the first
/// sample strictly after the instant. A bound is a stored sample of any quality — a Bad sample is returned as stored.
/// </para>
/// <para>
/// <b>Missing bound.</b> <c>Bad_BoundNotFound</c>, a null value, stamped with the bound's time. A one-sided read's
/// missing side (no EndTime / no StartTime) is never looked up; it is <c>Bad_BoundNotFound</c> stamped one second after
/// (forward) or before (reverse) the previous returned value (Table 1 footnotes a and b).
/// </para>
/// <para>
/// <b>Lookups.</b> "At or before T" is a newest-first read of <c>(0, T]</c> capped at one — native on a backend that
/// declares <see cref="IHistorianDataSource.SupportsReverseRaw"/>, otherwise <see cref="ReverseRawReader"/>'s walk-back
/// from bounded forward reads (the Gateway: NOT AVEVA's synthetic boundary row, which is stamped at the query start
/// rather than at the prior sample's own time). "At or after T" is a forward read of <c>[T, MaxValue)</c> capped at one (on a
/// forward-only backend ending at now + <see cref="ReverseRawReader.LookAhead"/> instead — #691 review);
/// "strictly after T" starts one storage resolution later (lossless: nothing is stored in between). Inside a tie
/// cluster the bound is the cluster's first sample in the read's direction — the forward read's first, the reverse
/// read's first (the forward order's last).
/// </para>
/// <para>
/// <b>Resolution.</b> "On the bound" compares at the backend's storage resolution, because a microsecond backend
/// (QuestDB) truncates the window bounds it is sent: a sample at <c>⌊T⌋</c> is inside a window starting at <c>T</c>
/// there, so it must not also be returned as the start bound.
/// </para>
/// </remarks>
internal static class HistoryBounds
{
private static readonly TimeSpan OpenBoundStep = TimeSpan.FromSeconds(1);
/// <summary>The end bound owed by a fresh read over the normalised window.</summary>
/// <param name="startUtc">The (normalised) start: the later time of a reverse read.</param>
/// <param name="endUtc">The (normalised) end; a one-sided read's sentinel (tick 0 / MaxValue) is an open bound.</param>
/// <returns>The pending end bound.</returns>
public static HistoryEndBound For(DateTime startUtc, DateTime endUtc) =>
new(endUtc, Reverse: startUtc > endUtc, Inclusive: startUtc != endUtc, Open: IsOpen(endUtc));
/// <summary>Whether a returned value is a <see cref="NotFound"/> placeholder rather than a stored sample.</summary>
/// <param name="value">The value.</param>
/// <returns><c>true</c> for a <c>Bad_BoundNotFound</c> placeholder.</returns>
public static bool IsPlaceholder(DataValueSnapshot value) =>
value.StatusCode == StatusCodes.BadBoundNotFound && value.Value is null;
/// <summary>The pending end bound to store behind a continuation point, remembering the page's last returned time.</summary>
/// <param name="bound">The pending end bound (null = bounds not requested).</param>
/// <param name="page">The page being returned.</param>
/// <returns>The bound to carry, or null.</returns>
public static HistoryEndBound? Carry(HistoryEndBound? bound, IReadOnlyList<DataValueSnapshot> page) =>
bound is null || page.Count == 0 || page[^1].SourceTimestampUtc is not { } last
? bound
: bound with { LastReturnedUtc = last };
/// <summary>The <c>Bad_BoundNotFound</c> placeholder (null value) stamped at <paramref name="atUtc"/>.</summary>
/// <param name="atUtc">The bound's time.</param>
/// <returns>The placeholder.</returns>
public static DataValueSnapshot NotFound(DateTime atUtc) =>
new(null, StatusCodes.BadBoundNotFound, atUtc, atUtc);
/// <summary>
/// The start bound to return ahead of the data, or <c>null</c> when a stored sample sits on StartTime (that sample is
/// the bound, and it is already the data's first sample).
/// </summary>
public static async Task<DataValueSnapshot?> StartBoundAsync(
IHistorianDataSource source, string tagname, DateTime startUtc, bool reverse, int maxTieCluster, DateTime nowUtc,
CancellationToken ct)
{
if (IsOpen(startUtc)) return NotFound(startUtc);
var found = reverse
? await EarliestAtOrAfterAsync(source, tagname, startUtc, nowUtc, ct).ConfigureAwait(false)
: await LatestAtOrBeforeAsync(source, tagname, startUtc, maxTieCluster, nowUtc, ct).ConfigureAwait(false);
if (found is null) return NotFound(startUtc);
return SameInstant(Ticks(found), startUtc.Ticks, Resolution(source)) ? null : found;
}
/// <summary>The end bound: the stored sample on or beyond the end time, else the placeholder.</summary>
/// <param name="source">The backend.</param>
/// <param name="tagname">Historian tag.</param>
/// <param name="bound">The pending end bound.</param>
/// <param name="previousReturnedUtc">The last value's time already returned (stamps an open bound's placeholder).</param>
/// <param name="maxTieCluster">The walk-back's row budget (forward-only backends).</param>
/// <param name="nowUtc">Now (the walk-back's look-ahead clamp).</param>
/// <param name="ct">Cancellation.</param>
public static async Task<DataValueSnapshot> EndBoundAsync(
IHistorianDataSource source, string tagname, HistoryEndBound bound, DateTime previousReturnedUtc,
int maxTieCluster, DateTime nowUtc, CancellationToken ct)
{
if (bound.Open)
{
var ticks = bound.Reverse
? Math.Max(0L, previousReturnedUtc.Ticks - OpenBoundStep.Ticks)
: Math.Min(DateTime.MaxValue.Ticks, previousReturnedUtc.Ticks + OpenBoundStep.Ticks);
return NotFound(new DateTime(ticks, DateTimeKind.Utc));
}
DataValueSnapshot? found;
if (bound.Reverse)
found = await LatestAtOrBeforeAsync(source, tagname, bound.AtUtc, maxTieCluster, nowUtc, ct).ConfigureAwait(false);
else if (bound.Inclusive)
found = await EarliestAtOrAfterAsync(source, tagname, bound.AtUtc, nowUtc, ct).ConfigureAwait(false);
else
{
var resolution = Resolution(source);
found = bound.AtUtc.Ticks > DateTime.MaxValue.Ticks - resolution
? null
: await EarliestAtOrAfterAsync(source, tagname, bound.AtUtc.AddTicks(resolution), nowUtc, ct).ConfigureAwait(false);
}
return found ?? NotFound(bound.AtUtc);
}
/// <summary>
/// Whether a data sample lies inside the data window of a bounded read — a forward window excludes the end time (a
/// sample there is the end bound), compared at the storage resolution. A backend whose forward end is inclusive, or a
/// resumed read that steps onto the end time, would otherwise return the end bound twice.
/// </summary>
public static bool InDataWindow(DataValueSnapshot sample, HistoryEndBound bound, long resolution)
{
if (!bound.Inclusive || bound.Open) return true;
var ts = Align(Ticks(sample), resolution);
var end = Align(bound.AtUtc.Ticks, resolution);
return bound.Reverse ? ts > end : ts < end;
}
/// <summary>Whether a resumed read's next start is already past the data window (only the end bound remains).</summary>
public static bool DataWindowDrained(DateTime nextStartUtc, HistoryEndBound bound, long resolution)
{
if (!bound.Inclusive || bound.Open) return false;
var next = Align(nextStartUtc.Ticks, resolution);
var end = Align(bound.AtUtc.Ticks, resolution);
return bound.Reverse ? next <= end : next >= end;
}
/// <summary>The latest stored sample at or before <paramref name="atUtc"/>, or null.</summary>
internal static async Task<DataValueSnapshot?> LatestAtOrBeforeAsync(
IHistorianDataSource source, string tagname, DateTime atUtc, int maxTieCluster, DateTime nowUtc, CancellationToken ct)
{
if (atUtc.Ticks == 0) return null;
var floor = new DateTime(0, DateTimeKind.Utc);
IReadOnlyList<DataValueSnapshot> rows = source.SupportsReverseRaw
? (await source.ReadRawAsync(tagname, atUtc, floor, 1, ct).ConfigureAwait(false)).Samples
: (await ReverseRawReader.ReadAsync(source, tagname, atUtc, floor, 1, maxTieCluster, nowUtc, 0, ct)
.ConfigureAwait(false)).Samples;
var limit = Align(atUtc.Ticks, Resolution(source));
foreach (var row in rows)
if (Align(Ticks(row), Resolution(source)) <= limit)
return row;
return null;
}
/// <summary>The earliest stored sample at or after <paramref name="atUtc"/>, or null.</summary>
/// <remarks>
/// #691 review: on a backend that reads forward only (the Gateway) the read's end is clamped to now +
/// <see cref="ReverseRawReader.LookAhead"/> — the same horizon the walk-back uses — and a time beyond it is "not found"
/// without a read. The HistorianGateway only checks <c>start &lt;= end</c>, so a <c>DateTime.MaxValue</c> end would
/// reach AVEVA as a year-9999 query end (unverified: a rejection, or a fabricated row). The clamp is NOT applied to a
/// backend that reads newest-first natively (SqlServer / QuestDB): they bind <c>MaxValue</c> safely (pinned by the
/// <c>Extreme_bounds</c> conformance test) and can legitimately hold samples further ahead (HistoryUpdate's
/// <c>HistoryUpdateMaxFutureSkew</c> is configurable, and driver-stamped live values carry device time) — clamping
/// there would turn a real end bound into <c>Bad_BoundNotFound</c>.
/// </remarks>
internal static async Task<DataValueSnapshot?> EarliestAtOrAfterAsync(
IHistorianDataSource source, string tagname, DateTime atUtc, DateTime nowUtc, CancellationToken ct)
{
var endTicks = DateTime.MaxValue.Ticks;
if (!source.SupportsReverseRaw)
endTicks = nowUtc.Ticks > DateTime.MaxValue.Ticks - ReverseRawReader.LookAhead.Ticks
? DateTime.MaxValue.Ticks
: nowUtc.Ticks + ReverseRawReader.LookAhead.Ticks;
if (atUtc.Ticks >= endTicks) return null;
var rows = (await source.ReadRawAsync(tagname, atUtc, new DateTime(endTicks, DateTimeKind.Utc), 1, ct)
.ConfigureAwait(false)).Samples;
var limit = Align(atUtc.Ticks, Resolution(source));
foreach (var row in rows)
if (Align(Ticks(row), Resolution(source)) >= limit)
return row;
return null;
}
private static bool IsOpen(DateTime t) => t.Ticks == 0 || t.Ticks == DateTime.MaxValue.Ticks;
private static bool SameInstant(long a, long b, long resolution) => Align(a, resolution) == Align(b, resolution);
private static long Resolution(IHistorianDataSource source) => Math.Max(1L, source.TimestampResolutionTicks);
private static long Ticks(DataValueSnapshot s) => (s.SourceTimestampUtc ?? DateTime.MinValue).Ticks;
private static long Align(long ticks, long resolution) => ticks - ticks % resolution;
}
@@ -62,6 +62,10 @@ namespace ZB.MOM.WW.OtOpcUa.OpcUaServer;
/// serialized), so a new field with a "none" default is all the versioning it needs.
/// </param>
/// <param name="EmptyToTicks">The exclusive upper end of that stretch.</param>
/// <param name="PendingEndBound">
/// <see cref="HistoryCursorKind.Raw"/> with <c>ReturnBounds = true</c> only (Gitea #691): the end bound still owed —
/// Part 11 returns it with the last page. <c>null</c> = bounds not requested.
/// </param>
internal sealed record HistoryContinuationState(
string Tagname,
DateTime NextStartUtc,
@@ -74,7 +78,8 @@ internal sealed record HistoryContinuationState(
bool Reverse = false,
long SegmentWidthTicks = 0,
long EmptyFromTicks = 0,
long EmptyToTicks = 0)
long EmptyToTicks = 0,
HistoryEndBound? PendingEndBound = null)
{
/// <summary>The walk-back hint this cursor carries (#686/#689/#691).</summary>
public ReverseWalkHint WalkHint => new(SegmentWidthTicks, EmptyFromTicks, EmptyToTicks);
@@ -544,6 +544,13 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
/// </summary>
public int MaxTieClusterOverfetch { get; set; } = 65536;
/// <summary>
/// Gitea #691: whether HistoryRead-Raw honours <c>ReturnBounds</c> (Part 11 bounding values, <see cref="HistoryBounds"/>)
/// — <c>ServerHistorian:HonorReturnBounds</c>, default <c>true</c>. <c>false</c> ignores the flag, as before #691,
/// including the modified-read <c>Bad_InvalidArgument</c>.
/// </summary>
public bool HonorReturnBounds { get; set; } = true;
/// <summary>
/// Maximum historized nodes served concurrently within one HistoryRead batch (arch-review 03/S3).
/// Mirrors <c>ServerHistorianOptions.HistoryReadBatchParallelism</c>; the Host sets it at
@@ -5056,6 +5063,15 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
var work = new List<Func<Task>>(nodesToProcess.Count);
foreach (var handle in nodesToProcess)
{
if (details.IsReadModified && details.ReturnBounds && HonorReturnBounds)
{
// Part 11 §6.5.3.3: returnBounds "shall be set to FALSE" for a modified read, "otherwise the Server returns
// a Bad_InvalidArgument" (#691). Off with the ServerHistorian:HonorReturnBounds kill switch.
errors[handle.Index] = StatusCodes.BadInvalidArgument;
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadInvalidArgument };
continue;
}
if (details.IsReadModified)
{
// Part 11 modified-value history (#668): the prior values every HistoryUpdate recorded. Gated by the
@@ -5070,7 +5086,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
work.Add(() => ServeRawPagedAsync(
handle, session, nodesToRead, results, errors,
details.StartTime, details.EndTime, details.NumValuesPerNode,
ct));
ct, details.ReturnBounds && HonorReturnBounds));
}
RunBounded(work, HistoryReadBatchParallelism);
@@ -6197,6 +6213,11 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
/// <param name="startTimeUtc">The request window's (inclusive) lower bound, used for a fresh read.</param>
/// <param name="endUtc">The (inclusive) upper bound of the read window; unchanged across pages.</param>
/// <param name="numValuesPerNode">The client's per-page cap; <c>0</c> means "all values, no paging".</param>
/// <param name="ct">Cancellation (the per-request deadline).</param>
/// <param name="returnBounds">
/// Part 11 bounding values (#691, <see cref="HistoryBounds"/>): the start bound leads the first page, the end bound
/// ends the last, and both count toward <paramref name="numValuesPerNode"/>. A resumed read takes it from the cursor.
/// </param>
private async Task ServeRawPagedAsync(
NodeHandle handle,
ISession? session,
@@ -6206,7 +6227,8 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
DateTime startTimeUtc,
DateTime endUtc,
uint numValuesPerNode,
CancellationToken ct)
CancellationToken ct,
bool returnBounds = false)
{
var inboundCp = nodesToRead[handle.Index].ContinuationPoint;
@@ -6215,6 +6237,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
DateTime startUtc;
var boundarySkip = 0;
var walkHint = default(ReverseWalkHint); // #686/#689/#691: the walk-back's width + proven-empty stretch, page to page
HistoryEndBound? endBound = null; // #691: the end bound still owed (null = bounds not requested)
string tagname;
if (inboundCp is { Length: > 0 })
@@ -6236,6 +6259,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
endUtc = state.EndUtc;
numValuesPerNode = state.NumValuesPerNode;
walkHint = state.WalkHint;
endBound = state.PendingEndBound;
}
else
{
@@ -6261,6 +6285,107 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
else if (startUtc == DateTime.MinValue && endUtc != DateTime.MinValue)
(startUtc, endUtc) = (endUtc, new DateTime(0, DateTimeKind.Utc));
}
if (returnBounds) endBound = HistoryBounds.For(startUtc, endUtc);
}
var nowUtc = Clock.GetUtcNow().UtcDateTime;
var resolution = Math.Max(1L, HistorianDataSource.TimestampResolutionTicks);
// #691: the data is drained — the end bound ends this page when it has room (Part 11: bounds count toward
// NumValuesPerNode), else it is the whole next page. Returns the continuation point to emit, if any.
async Task<byte[]?> AppendEndBoundAsync(List<DataValueSnapshot> page, DateTime resumedAtUtc)
{
// The previous returned value stamps an open bound's placeholder (Part 11 Table 1 footnotes a/b): this
// page's last, else the last of an earlier page (carried in the cursor), else where the read resumed.
var previous = (page.Count > 0 ? page[^1].SourceTimestampUtc : null)
?? endBound!.LastReturnedUtc ?? resumedAtUtc;
if (numValuesPerNode == 0 || page.Count < numValuesPerNode)
{
page.Add(await HistoryBounds.EndBoundAsync(
HistorianDataSource, tagname, endBound!, previous, MaxTieClusterOverfetch, nowUtc, ct).ConfigureAwait(false));
return null;
}
return _historyContinuationStore.Save(session, new HistoryContinuationState(
tagname, previous, endUtc, 0, numValuesPerNode, SegmentWidthTicks: walkHint.WidthTicks,
EmptyFromTicks: walkHint.EmptyFromTicks, EmptyToTicks: walkHint.EmptyToTicks,
PendingEndBound: HistoryBounds.Carry(endBound!, page)! with { DataExhausted = true }));
}
// #691 review: whether a bounded read whose start bound was NOT found has any data or a found end bound — only
// asked when a one-value page would otherwise be the start-bound placeholder alone.
async Task<bool> BoundedReadHasAnythingAsync()
{
var src = HistorianDataSource;
IReadOnlyList<DataValueSnapshot> first = startUtc > endUtc && !src.SupportsReverseRaw
? (await ReverseRawReader.ReadAsync(src, tagname, startUtc, endUtc, 1, MaxTieClusterOverfetch, nowUtc, 0, ct)
.ConfigureAwait(false)).Samples
: (await src.ReadRawAsync(tagname, startUtc, endUtc, 1, ct).ConfigureAwait(false)).Samples;
if (first.Any(x => HistoryBounds.InDataWindow(x, endBound!, resolution))) return true;
var end = await HistoryBounds.EndBoundAsync(
src, tagname, endBound!, startUtc, MaxTieClusterOverfetch, nowUtc, ct).ConfigureAwait(false);
return !HistoryBounds.IsPlaceholder(end);
}
// #691: a resumed bounded read whose data is drained (the end bound did not fit on the previous page, or the
// resume point already sits on the end time) owes only the end bound.
if (endBound is not null && inboundCp is { Length: > 0 }
&& (endBound.DataExhausted || HistoryBounds.DataWindowDrained(startUtc, endBound, resolution)))
{
var onlyBound = new List<DataValueSnapshot>(1);
var afterCp = await AppendEndBoundAsync(onlyBound, startUtc).ConfigureAwait(false);
results[handle.Index] = new SdkHistoryReadResult
{
StatusCode = StatusCodes.Good,
HistoryData = new ExtensionObject(ToHistoryDataFromSamples(onlyBound)),
ContinuationPoint = afterCp,
};
errors[handle.Index] = ServiceResult.Good;
return;
}
// #691: the start bound leads the first page and counts toward the cap; null when a sample sits on StartTime
// (that sample is the bound, and it is the data's first).
DataValueSnapshot? startBound = null;
var readCap = numValuesPerNode;
if (endBound is not null && inboundCp is not { Length: > 0 })
{
startBound = await HistoryBounds.StartBoundAsync(
HistorianDataSource, tagname, startUtc, endBound.Reverse, MaxTieClusterOverfetch, nowUtc, ct)
.ConfigureAwait(false);
if (startBound is not null && numValuesPerNode == 1
&& (!HistoryBounds.IsPlaceholder(startBound) || await BoundedReadHasAnythingAsync().ConfigureAwait(false)))
{
// The page is the start bound alone; the data starts at StartTime on the next page.
var boundOnlyCp = _historyContinuationStore.Save(session, new HistoryContinuationState(
tagname, startUtc, endUtc, 0, numValuesPerNode, SegmentWidthTicks: walkHint.WidthTicks,
EmptyFromTicks: walkHint.EmptyFromTicks, EmptyToTicks: walkHint.EmptyToTicks,
PendingEndBound: HistoryBounds.Carry(endBound, [startBound])));
results[handle.Index] = new SdkHistoryReadResult
{
StatusCode = StatusCodes.Good,
HistoryData = new ExtensionObject(ToHistoryDataFromSamples([startBound])),
ContinuationPoint = boundOnlyCp,
};
errors[handle.Index] = ServiceResult.Good;
return;
}
if (startBound is not null && numValuesPerNode == 1)
{
// Nothing at all — no data and no bound found (review of #691): the pre-bounds GoodNoData answer.
results[handle.Index] = new SdkHistoryReadResult
{
StatusCode = StatusCodes.GoodNoData,
HistoryData = new ExtensionObject(ToHistoryDataFromSamples([])),
ContinuationPoint = null,
};
errors[handle.Index] = ServiceResult.Good;
return;
}
if (startBound is not null && numValuesPerNode > 0) readCap = numValuesPerNode - 1;
}
// HistoryRead is NOT under the node-manager Lock — awaiting the async source is safe.
@@ -6272,8 +6397,8 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
if (startUtc > endUtc && !source.SupportsReverseRaw)
{
var page = await ReverseRawReader.ReadAsync(
source, tagname, startUtc, endUtc, numValuesPerNode, MaxTieClusterOverfetch,
Clock.GetUtcNow().UtcDateTime, walkHint, ct)
source, tagname, startUtc, endUtc, readCap, MaxTieClusterOverfetch,
nowUtc, walkHint, ct)
.ConfigureAwait(false);
walkHint = page.Hint;
sourceResult = new HistorianRead(page.Samples, ContinuationPoint: null);
@@ -6281,17 +6406,29 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
else
{
sourceResult = await source
.ReadRawAsync(tagname, startUtc, endUtc, numValuesPerNode, ct)
.ReadRawAsync(tagname, startUtc, endUtc, readCap, ct)
.ConfigureAwait(false);
}
var backendFull = HistoryPaging.IsFullPage(sourceResult.Samples.Count, numValuesPerNode);
var backendFull = HistoryPaging.IsFullPage(sourceResult.Samples.Count, readCap);
// On a resume read, drop the boundary ties already returned on the prior page.
var samples = inboundCp is { Length: > 0 }
? HistoryPaging.TrimBoundaryDuplicates(sourceResult.Samples, startUtc, boundarySkip)
: sourceResult.Samples;
// #691: a bounded read's data window excludes the end time (a sample there is the end bound). A backend whose
// forward end is inclusive (the gateway's own) returns it as data; dropping it also means the data is drained.
if (endBound is not null)
{
var inside = samples.Where(x => HistoryBounds.InDataWindow(x, endBound, resolution)).ToList();
if (inside.Count != samples.Count)
{
samples = inside;
backendFull = false;
}
}
// Oversized tie cluster: a resume read whose FULL backend page does not advance past the resume
// boundary timestamp `startUtc`. That happens when more samples share `startUtc` than the page
// cap — a tie cluster larger than NumValuesPerNode. The fixed-(start,end,cap) backend can only
@@ -6355,9 +6492,15 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
{
var clusterState = new HistoryContinuationState(
tagname, resumeAt, endUtc, nextSkip, numValuesPerNode, SegmentWidthTicks: walkHint.WidthTicks,
EmptyFromTicks: walkHint.EmptyFromTicks, EmptyToTicks: walkHint.EmptyToTicks);
EmptyFromTicks: walkHint.EmptyFromTicks, EmptyToTicks: walkHint.EmptyToTicks,
PendingEndBound: HistoryBounds.Carry(endBound, slice));
clusterCp = _historyContinuationStore.Save(session, clusterState);
}
else if (endBound is not null)
{
// #691: the data is drained — the end bound ends this page, or is the next one when this one is full.
clusterCp = await AppendEndBoundAsync(slice, startUtc).ConfigureAwait(false);
}
results[handle.Index] = new SdkHistoryReadResult
{
@@ -6369,6 +6512,14 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
return;
}
// #691: the start bound (first page only) leads the data.
if (startBound is not null)
{
var led = new List<DataValueSnapshot>(samples.Count + 2) { startBound };
led.AddRange(samples);
samples = led;
}
// The "full page" test is against the RAW backend count (before trimming): the backend honoured
// the cap, so a full backend page ⇒ there may be more even if we trimmed some boundary ties.
var historyData = ToHistoryDataFromSamples(samples);
@@ -6379,11 +6530,30 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
HistoryPaging.ComputeResumeCursor(samples, out var nextStart, out var skip);
var nextState = new HistoryContinuationState(
tagname, nextStart, endUtc, skip, numValuesPerNode, SegmentWidthTicks: walkHint.WidthTicks,
EmptyFromTicks: walkHint.EmptyFromTicks, EmptyToTicks: walkHint.EmptyToTicks);
EmptyFromTicks: walkHint.EmptyFromTicks, EmptyToTicks: walkHint.EmptyToTicks,
PendingEndBound: HistoryBounds.Carry(endBound, samples));
// Save may return null (no session on this request) ⇒ degrade to single-shot for this node.
// Built AFTER historyData so a failure projecting samples can never orphan a stored cursor.
outboundCp = _historyContinuationStore.Save(session, nextState);
}
else if (endBound is not null)
{
// #691: the data is drained — the end bound ends this page, or is the next one when this one is full.
var page = new List<DataValueSnapshot>(samples.Count + 1);
page.AddRange(samples);
outboundCp = await AppendEndBoundAsync(page, startUtc).ConfigureAwait(false);
samples = page;
historyData = ToHistoryDataFromSamples(samples);
}
// #691 review: a whole bounded read with no data and no bound found (one page, every value a placeholder) is the
// pre-bounds answer — GoodNoData, no values (§6.5.3.2: bounds make it a success only "if one or both exist").
if (endBound is not null && inboundCp is not { Length: > 0 } && outboundCp is null
&& samples.All(HistoryBounds.IsPlaceholder))
{
samples = [];
historyData = ToHistoryDataFromSamples(samples);
}
results[handle.Index] = new SdkHistoryReadResult
{
@@ -89,6 +89,14 @@ public sealed class ServerHistorianOptions
/// </summary>
public int MaxTieClusterOverfetch { get; init; } = 65536;
/// <summary>
/// Gitea #691: whether HistoryRead-Raw honours <c>ReadRawModifiedDetails.ReturnBounds</c> (OPC UA Part 11 bounding
/// values). Default <c>true</c>. <c>false</c> is the kill switch back to the pre-#691 answer: the flag is ignored
/// (stored samples only) and a modified read carrying it is not refused <c>Bad_InvalidArgument</c>. Note the .NET
/// SDK's <c>new ReadRawModifiedDetails()</c> defaults <c>ReturnBounds</c> (and <c>IsReadModified</c>) to <c>true</c>.
/// </summary>
public bool HonorReturnBounds { get; init; } = true;
/// <summary>
/// Maximum number of historized nodes served CONCURRENTLY within one HistoryRead batch (arch-review
/// 03/S3). The SDK's <c>HistoryRead*</c> override surface is synchronous, so each arm block-bridges
@@ -0,0 +1,100 @@
using Shouldly;
using Xunit;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Conformance;
/// <summary>
/// Gitea #691: the server composes Part 11 bounding values (HistoryRead-Raw <c>ReturnBounds</c>) from exactly these raw
/// reads on a backend that reads newest-first natively (<c>HistoryBounds</c> in the OPC UA server) — so every backend must
/// answer them precisely: "at or before T" = a reverse read of <c>(0, T]</c> capped at one, "at or after T" = a forward read
/// of <c>[T, MaxValue)</c> capped at one, "strictly after T" = the same from <c>T + resolution</c>. A bound is a stored
/// sample of any quality, so a Bad row or an outage marker comes back as the bound.
/// </summary>
public abstract class BoundingValueConformance : ConformanceBase
{
private static readonly CancellationToken Ct = CancellationToken.None;
private static readonly DateTime Floor = new(0, DateTimeKind.Utc);
private static readonly DateTime Ceiling = new(DateTime.MaxValue.Ticks, DateTimeKind.Utc);
private async Task<DataValueSnapshot?> AtOrBeforeAsync(IHistorianDataSource src, string tag, DateTime t) =>
(await src.ReadRawAsync(tag, t, Floor, 1, Ct)).Samples.SingleOrDefault();
private async Task<DataValueSnapshot?> AtOrAfterAsync(IHistorianDataSource src, string tag, DateTime t) =>
(await src.ReadRawAsync(tag, t, Ceiling, 1, Ct)).Samples.SingleOrDefault();
[Fact]
public async Task The_sample_at_or_before_a_time_is_the_latest_one_on_or_before_it()
{
await ConformanceData.SeedAAsync(B);
var src = B.CreateDataSource();
(await AtOrBeforeAsync(src, "A", At(10)))!.Value.ShouldBe(20d); // exactly on it
(await AtOrBeforeAsync(src, "A", At(15)))!.Value.ShouldBe(20d); // between samples
(await AtOrBeforeAsync(src, "A", At(1000)))!.Value.ShouldBe(50d); // past the last
(await AtOrBeforeAsync(src, "A", At(-1))).ShouldBeNull(); // before the first: no bound
(await AtOrBeforeAsync(src, "A", At(15)))!.SourceTimestampUtc!.Value.Ticks.ShouldBe(B.Round(At(10)).Ticks);
}
[Fact]
public async Task The_sample_at_or_after_a_time_is_the_earliest_one_on_or_after_it()
{
await ConformanceData.SeedAAsync(B);
var src = B.CreateDataSource();
(await AtOrAfterAsync(src, "A", At(10)))!.Value.ShouldBe(20d); // exactly on it
(await AtOrAfterAsync(src, "A", At(15)))!.Value.ShouldBe(30d); // between samples
(await AtOrAfterAsync(src, "A", At(-100)))!.Value.ShouldBe(10d); // before the first
(await AtOrAfterAsync(src, "A", At(41))).ShouldBeNull(); // past the last: no bound
}
[Fact]
public async Task The_sample_strictly_after_a_time_starts_one_storage_resolution_later()
{
await ConformanceData.SeedAAsync(B);
var src = B.CreateDataSource();
(await AtOrAfterAsync(src, "A", At(10).AddTicks(B.ResolutionTicks)))!.Value.ShouldBe(30d);
(await AtOrAfterAsync(src, "A", At(40).AddTicks(B.ResolutionTicks))).ShouldBeNull();
}
[Fact]
public async Task A_time_between_storage_ticks_finds_the_sample_the_backend_stores_at_its_truncation()
{
// The server compares "on the bound" at the storage resolution, so the lookups must agree with the window reads:
// a time a fraction of a resolution step after a stored sample is that sample's instant on a coarse backend.
await ConformanceData.SeedAAsync(B);
var src = B.CreateDataSource();
var t = At(10).AddTicks(B.ResolutionTicks - 1); // == At(10) on a 1-tick backend
(await AtOrBeforeAsync(src, "A", t))!.Value.ShouldBe(20d);
var after = await AtOrAfterAsync(src, "A", t);
after!.SourceTimestampUtc!.Value.Ticks.ShouldBeGreaterThanOrEqualTo(B.Round(t).Ticks);
after.Value.ShouldBe(B.Round(t) == B.Round(At(10)) ? 20d : 30d);
}
[Fact]
public async Task A_bad_sample_or_an_outage_marker_is_a_bound_as_stored()
{
await B.SeedAsync("M",
(At(0), HistorianValue.FromDouble(10), Good),
(At(5), HistorianValue.Empty, BadNoCommunication),
(At(7), HistorianValue.FromDouble(99), Bad));
var src = B.CreateDataSource();
var before = await AtOrBeforeAsync(src, "M", At(6));
before!.Value.ShouldBeNull();
before.StatusCode.ShouldBe(0x80310000u);
var after = await AtOrAfterAsync(src, "M", At(6));
after!.Value.ShouldBe(99d);
after.StatusCode.ShouldBe(0x80000000u);
}
[Fact]
public async Task An_unknown_tag_has_no_bounds()
{
var src = B.CreateDataSource();
(await AtOrBeforeAsync(src, "nope", At(10))).ShouldBeNull();
(await AtOrAfterAsync(src, "nope", At(10))).ShouldBeNull();
}
}
@@ -0,0 +1,174 @@
using Google.Protobuf.WellKnownTypes;
using Microsoft.Extensions.Logging.Abstractions;
using Shouldly;
using Xunit;
using ZB.MOM.WW.HistorianGateway.Contracts.Grpc;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using ZB.MOM.WW.OtOpcUa.OpcUaServer;
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests;
/// <summary>
/// Gitea #691: Part 11 bounding values on the Gateway, composed by the server's <see cref="HistoryBounds"/> over the REAL
/// source and an AVEVA-shaped fake (the live gate's wire shape). The start bound of a forward read must be the stored
/// sample before StartTime, at ITS OWN time — never AVEVA's synthetic boundary row, which is stamped at the query start
/// and carries the value in effect there (<c>Quality = 133</c>). The Gateway reads forward only, so "at or before" is
/// served by the walk-back (<see cref="ReverseRawReader"/>), and no reverse window is ever sent.
/// </summary>
public sealed class GatewayBoundingValueTests
{
private static CancellationToken Ct => TestContext.Current.CancellationToken;
private static readonly DateTime T0 = new(2026, 10, 12, 18, 6, 0, DateTimeKind.Utc);
private static readonly DateTime Now = T0.AddHours(1);
/// <summary>The live gate's tag <c>HistGW.LiveTest.Gate681</c> (18:06:40 is a stored NULL).</summary>
private static readonly (double Sec, double Value, bool Null)[] Gate681 =
[
(0, 10, false), (5, 20, false), (20, 30, false), (40, 0, true), (50, 5, false), (60, 1, false), (90, 1, false),
(100, 7, false),
];
private static DateTime At(double seconds) => T0.AddTicks((long)(seconds * TimeSpan.TicksPerSecond));
private static HistorianSample Stored(DateTime t, double v, bool isNull) => new()
{
Tag = "T",
Timestamp = Timestamp.FromDateTime(t),
NumericValue = isNull ? 0 : v,
OpcQuality = isNull ? 0u : 192u,
Quality = isNull ? 1u : 0u,
};
/// <summary>The latest query end any ReadRaw was sent (a far-future end is the #691 review's risk).</summary>
private sealed class EndProbe
{
public DateTime MaxEnd = DateTime.MinValue;
}
/// <summary>
/// ReadRaw over <c>[start, end]</c> ascending, capped, led by AVEVA's boundary row at the query start. A query end more
/// than two days past <see cref="Now"/> is REFUSED (<c>InvalidArgument</c>), modelling the unverified AVEVA behaviour
/// for a year-9999 end (the HistorianGateway only checks <c>start &lt;= end</c>, so it reaches AVEVA unclamped).
/// </summary>
private static FakeHistorianGatewayClient Aveva(EndProbe? probe = null)
{
var sorted = Gate681.Select(x => (T: At(x.Sec), V: x.Value, x.Null)).ToList();
return new FakeHistorianGatewayClient
{
RawSource = (start, end, max) =>
{
if (probe is not null && end > probe.MaxEnd) probe.MaxEnd = end;
if (end > Now.AddDays(2))
throw new Grpc.Core.RpcException(new Grpc.Core.Status(Grpc.Core.StatusCode.InvalidArgument,
"end_time is outside the historian's time range"));
var rows = new List<HistorianSample>();
if (sorted.All(d => d.T != start))
{
var prior = sorted.Where(d => d.T < start).Select(d => ((DateTime T, double V, bool Null)?)d).LastOrDefault();
HistorianSample boundary;
if (prior is { } p)
{
boundary = Stored(start, p.V, p.Null);
if (!p.Null) boundary.Quality = 133;
}
else
{
boundary = new HistorianSample { Tag = "T", Timestamp = Timestamp.FromDateTime(start), NumericValue = 0, OpcQuality = 0, Quality = 1 };
}
rows.Add(boundary);
}
rows.AddRange(sorted.Where(d => d.T >= start && d.T <= end).Select(d => Stored(d.T, d.V, d.Null)));
return max > 0 ? rows.Take(max) : rows;
},
};
}
private static GatewayHistorianDataSource Source(FakeHistorianGatewayClient fake) =>
new(fake, NullLogger<GatewayHistorianDataSource>.Instance);
[Fact]
public async Task The_start_bound_is_the_stored_prior_sample_at_its_own_time_not_the_synthetic_row()
{
var bound = await HistoryBounds.StartBoundAsync(Source(Aveva()), "T", At(21), reverse: false, 65536, Now, Ct);
bound.ShouldNotBeNull();
bound.SourceTimestampUtc.ShouldBe(At(20)); // the synthetic row would be stamped At(21)
bound.Value.ShouldBe(30d);
bound.StatusCode.ShouldBe(0u);
}
[Fact]
public async Task A_stored_NULL_prior_sample_is_the_start_bound_as_the_Bad_sample_it_is()
{
var bound = await HistoryBounds.StartBoundAsync(Source(Aveva()), "T", At(45), reverse: false, 65536, Now, Ct);
bound!.SourceTimestampUtc.ShouldBe(At(40));
bound.StatusCode.ShouldBe(0x80000000u);
}
[Fact]
public async Task Before_the_first_sample_the_start_bound_is_not_found_at_the_start_time()
{
var bound = await HistoryBounds.StartBoundAsync(Source(Aveva()), "T", At(-30), reverse: false, 65536, Now, Ct);
bound!.StatusCode.ShouldBe(Opc.Ua.StatusCodes.BadBoundNotFound);
bound.SourceTimestampUtc.ShouldBe(At(-30));
bound.Value.ShouldBeNull();
}
[Fact]
public async Task A_sample_exactly_on_the_start_time_is_the_bound_and_already_the_data()
{
(await HistoryBounds.StartBoundAsync(Source(Aveva()), "T", At(20), reverse: false, 65536, Now, Ct)).ShouldBeNull();
}
[Fact]
public async Task The_end_bound_is_the_first_stored_sample_at_or_after_the_end_time()
{
var source = Source(Aveva());
var after = await HistoryBounds.EndBoundAsync(source, "T", HistoryBounds.For(At(0), At(41)), At(40), 65536, Now, Ct);
after.SourceTimestampUtc.ShouldBe(At(50));
after.Value.ShouldBe(5d);
var missing = await HistoryBounds.EndBoundAsync(source, "T", HistoryBounds.For(At(0), At(101)), At(100), 65536, Now, Ct);
missing.StatusCode.ShouldBe(Opc.Ua.StatusCodes.BadBoundNotFound);
missing.SourceTimestampUtc.ShouldBe(At(101));
}
/// <summary>
/// #691 review: the at-or-after read never sends an end beyond now + <see cref="ReverseRawReader.LookAhead"/> (the
/// historian stores nothing further ahead), and an end bound wanted beyond that horizon is not found without a read.
/// </summary>
[Fact]
public async Task The_at_or_after_read_never_asks_past_now_plus_one_day()
{
var probe = new EndProbe();
var fake = Aveva(probe);
var source = Source(fake);
var found = await HistoryBounds.EndBoundAsync(source, "T", HistoryBounds.For(At(0), At(41)), At(40), 65536, Now, Ct);
found.SourceTimestampUtc.ShouldBe(At(50));
probe.MaxEnd.ShouldBeLessThanOrEqualTo(Now + ReverseRawReader.LookAhead.Add(TimeSpan.FromMilliseconds(1)));
var calls = fake.ReadRawCallCount;
var beyond = Now.AddDays(3);
var missing = await HistoryBounds.EndBoundAsync(source, "T", HistoryBounds.For(At(0), beyond), At(100), 65536, Now, Ct);
missing.StatusCode.ShouldBe(Opc.Ua.StatusCodes.BadBoundNotFound);
missing.SourceTimestampUtc.ShouldBe(beyond);
fake.ReadRawCallCount.ShouldBe(calls, "a bound beyond the horizon is not found without a read");
source.GetHealthSnapshot().TotalFailures.ShouldBe(0);
}
[Fact]
public async Task A_reverse_read_end_bound_walks_back_to_the_stored_sample_at_or_before_the_end_time()
{
var fake = Aveva();
var bound = await HistoryBounds.EndBoundAsync(Source(fake), "T", HistoryBounds.For(At(100), At(55)), At(60), 65536, Now, Ct);
bound.SourceTimestampUtc.ShouldBe(At(50));
bound.Value.ShouldBe(5d);
}
}
@@ -67,3 +67,10 @@ public sealed class QuestDbEventReadConformance(QuestDbFixture fx) : EventReadCo
/// <inheritdoc/>
protected override Task<IConformanceBackend> CreateBackendAsync() => QuestDbConformanceBackend.CreateAsync(fx);
}
[Collection(QuestDbCollection.Name)]
public sealed class QuestDbBoundingValueConformance(QuestDbFixture fx) : BoundingValueConformance
{
/// <inheritdoc/>
protected override Task<IConformanceBackend> CreateBackendAsync() => QuestDbConformanceBackend.CreateAsync(fx);
}
@@ -67,3 +67,10 @@ public sealed class SqlServerEventReadConformance(SqlHistorianFixture fx) : Even
/// <inheritdoc/>
protected override Task<IConformanceBackend> CreateBackendAsync() => SqlServerConformanceBackend.CreateAsync(fx);
}
[Collection(SqlHistorianCollection.Name)]
public sealed class SqlServerBoundingValueConformance(SqlHistorianFixture fx) : BoundingValueConformance
{
/// <inheritdoc/>
protected override Task<IConformanceBackend> CreateBackendAsync() => SqlServerConformanceBackend.CreateAsync(fx);
}
@@ -72,6 +72,18 @@ public sealed class OtOpcUaServerHostedServiceHistoryUpdateTests
Service(view: null).BuildHistoryUpdatePosture().MaxFutureSkew.ShouldBe(TimeSpan.FromDays(1));
}
/// <summary>#691: the ReturnBounds kill switch is bound from <c>ServerHistorian:HonorReturnBounds</c>, default on.</summary>
[Fact]
public void The_return_bounds_kill_switch_comes_from_options_and_defaults_on()
{
Service(view: null).HonorReturnBounds.ShouldBeTrue();
var config = new ConfigurationBuilder().AddInMemoryCollection(new Dictionary<string, string?>
{
["ServerHistorian:HonorReturnBounds"] = "false",
}).Build();
Service(view: null, config: config).HonorReturnBounds.ShouldBeFalse();
}
private static OtOpcUaServerHostedService Service(
IHistoryUpdateRoleView? view, IDriverPrimaryRoleSource? primary = null, IConfiguration? config = null,
TimeProvider? clock = null) => new(
@@ -97,6 +97,7 @@ public sealed class HistoryUpdateOverTheWireTests
var details = new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = T1.AddHours(-1), EndTime = T1.AddHours(1), NumValuesPerNode = 100, IsReadModified = true,
};
var response = await session.HistoryReadAsync(
@@ -384,6 +384,7 @@ public sealed class NodeManagerAclEnforcementTests : IDisposable
context,
new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = DateTime.UtcNow.AddHours(-1),
EndTime = DateTime.UtcNow,
NumValuesPerNode = 10,
@@ -0,0 +1,523 @@
using Microsoft.Extensions.Logging.Abstractions;
using Opc.Ua;
using Opc.Ua.Server;
using Shouldly;
using Xunit;
using ZB.MOM.WW.OtOpcUa.Commons.OpcUa;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using HistorianRead = ZB.MOM.WW.OtOpcUa.Core.Abstractions.HistoryReadResult;
using SdkHistoryReadResult = Opc.Ua.HistoryReadResult;
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests;
/// <summary>Boots ONE real SDK server for the whole bounds suite (the cases swap the historian source per read).</summary>
public sealed class BoundsServerFixture : IAsyncLifetime
{
private readonly string _pkiRoot = Path.Combine(Path.GetTempPath(), $"otopcua-bounds-{Guid.NewGuid():N}");
private OpcUaApplicationHost? _host;
/// <summary>The node manager under test.</summary>
public OtOpcUaNodeManager Nm { get; private set; } = null!;
/// <summary>The historized variable every case reads.</summary>
public NodeId NodeId { get; private set; } = null!;
/// <summary>Serialises the cases (they share the node manager's source).</summary>
public SemaphoreSlim Gate { get; } = new(1, 1);
public async ValueTask InitializeAsync()
{
var started = await OpcUaHostStart.StartAsync(
port => new OpcUaApplicationHost(
new OpcUaApplicationHostOptions
{
ApplicationName = "OtOpcUa.BoundsTest",
ApplicationUri = $"urn:OtOpcUa.BoundsTest:{Guid.NewGuid():N}",
OpcUaPort = port,
PublicHostname = "localhost",
PkiStoreRoot = _pkiRoot,
},
NullLogger<OpcUaApplicationHost>.Instance),
() => new OtOpcUaSdkServer(), CancellationToken.None);
_host = started.Host;
Nm = started.Server.NodeManager!;
Nm.HistoryContinuationStore = new InMemoryHistoryContinuationStore();
Nm.EnsureVariable("eq-1/bounds", parentFolderNodeId: null, displayName: "Bounds", dataType: "Double",
writable: false, historianTagname: "WW.Bounds", realm: AddressSpaceRealm.Uns);
NodeId = Nm.TryGetVariable("eq-1/bounds")!.NodeId;
}
public async ValueTask DisposeAsync()
{
if (_host is not null) await _host.DisposeAsync();
if (Directory.Exists(_pkiRoot))
{
try { Directory.Delete(_pkiRoot, recursive: true); }
catch { /* best-effort */ }
}
}
}
/// <summary>
/// Gitea #691: HistoryRead-Raw <c>ReturnBounds</c> (OPC UA Part 11 §4.6 bounding values, §6.5.3.2). The oracle is Part 11
/// Table 1 itself — every row, over its data set (samples at 5:00, 5:02, 5:03, 5:05, 5:06) — run against three backend
/// shapes: one that reads newest-first natively (SqlServer-like), one that reads forward only (Gateway-like, walked back
/// by <see cref="ReverseRawReader"/>) and a microsecond one (QuestDB-like,
/// truncating its window bounds).
/// </summary>
public sealed class NodeManagerHistoryReadBoundsTests(BoundsServerFixture fx) : IClassFixture<BoundsServerFixture>
{
private static readonly DateTime Day = new(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc);
private static readonly DateTime Unspecified = DateTime.MinValue;
private static DateTime T(string hhmm)
{
var parts = hhmm.Split(':');
return Day.AddHours(int.Parse(parts[0])).AddMinutes(int.Parse(parts[1]));
}
/// <summary>Part 11 Table 1's data: value = minutes past 5:00.</summary>
private static DataValueSnapshot[] TableData() =>
new[] { "5:00", "5:02", "5:03", "5:05", "5:06" }
.Select(s => new DataValueSnapshot((double)(T(s) - T("5:00")).TotalMinutes, StatusCodes.Good, T(s), T(s)))
.ToArray();
public enum Shape { Reverse, ForwardOnly, Microsecond }
private static IHistorianDataSource Source(Shape shape, IReadOnlyList<DataValueSnapshot> series) => shape switch
{
Shape.Reverse => new SeriesSource(series, reverse: true, inclusiveForwardEnd: false, resolution: 1),
Shape.ForwardOnly => new SeriesSource(series, reverse: false, inclusiveForwardEnd: false, resolution: 1),
_ => new SeriesSource(series, reverse: true, inclusiveForwardEnd: false, resolution: 10),
};
/// <summary>
/// Part 11 Table 1, every row, verbatim. "FIRST"/"LAST" are <c>Bad_BoundNotFound</c> placeholders (null value) at the
/// StartTime/EndTime; a/b footnotes: one second after/before the previous returned value when that side is unspecified.
/// </summary>
public static TheoryData<string, string, uint, bool, string> Table1 => new()
{
{ "5:00", "5:05", 0, true, "5:00, 5:02, 5:03, 5:05" },
{ "5:00", "5:05", 0, false, "5:00, 5:02, 5:03" },
{ "5:01", "5:04", 0, true, "5:00, 5:02, 5:03, 5:05" },
{ "5:01", "5:04", 0, false, "5:02, 5:03" },
{ "5:05", "5:00", 0, true, "5:05, 5:03, 5:02, 5:00" },
{ "5:05", "5:00", 0, false, "5:05, 5:03, 5:02" },
{ "5:04", "5:01", 0, true, "5:05, 5:03, 5:02, 5:00" },
{ "5:04", "5:01", 0, false, "5:03, 5:02" },
{ "4:59", "5:05", 0, true, "FIRST, 5:00, 5:02, 5:03, 5:05" },
{ "4:59", "5:05", 0, false, "5:00, 5:02, 5:03" },
{ "5:01", "5:07", 0, true, "5:00, 5:02, 5:03, 5:05, 5:06, LAST" },
{ "5:01", "5:07", 0, false, "5:02, 5:03, 5:05, 5:06" },
{ "5:00", "5:05", 3, true, "5:00, 5:02, 5:03" },
{ "5:00", "5:05", 3, false, "5:00, 5:02, 5:03" },
{ "5:01", "5:04", 3, true, "5:00, 5:02, 5:03" },
{ "5:01", "5:04", 3, false, "5:02, 5:03" },
{ "5:05", "5:00", 3, true, "5:05, 5:03, 5:02" },
{ "5:05", "5:00", 3, false, "5:05, 5:03, 5:02" },
{ "5:04", "5:01", 3, true, "5:05, 5:03, 5:02" },
{ "5:04", "5:01", 3, false, "5:03, 5:02" },
{ "4:59", "5:05", 3, true, "FIRST, 5:00, 5:02" },
{ "4:59", "5:05", 3, false, "5:00, 5:02, 5:03" },
{ "5:01", "5:07", 3, true, "5:00, 5:02, 5:03" },
{ "5:01", "5:07", 3, false, "5:02, 5:03, 5:05" },
{ "5:00", "", 3, true, "5:00, 5:02, 5:03" },
{ "5:00", "", 3, false, "5:00, 5:02, 5:03" },
{ "5:00", "", 6, true, "5:00, 5:02, 5:03, 5:05, 5:06, LAST+" },
{ "5:00", "", 6, false, "5:00, 5:02, 5:03, 5:05, 5:06" },
{ "5:07", "", 6, true, "5:06, LAST+" },
{ "5:07", "", 6, false, "" },
{ "", "5:06", 3, true, "5:06, 5:05, 5:03" },
{ "", "5:06", 3, false, "5:06, 5:05, 5:03" },
{ "", "5:06", 6, true, "5:06, 5:05, 5:03, 5:02, 5:00, FIRST-" },
{ "", "5:06", 6, false, "5:06, 5:05, 5:03, 5:02, 5:00" },
{ "", "4:48", 6, true, "5:00, FIRST-" },
{ "", "4:48", 6, false, "" },
{ "4:48", "4:48", 0, true, "FIRST, 5:00" },
{ "4:48", "4:48", 0, false, "" },
{ "4:48", "4:48", 1, true, "FIRST" },
{ "4:48", "4:48", 1, false, "" },
{ "4:48", "4:48", 2, true, "FIRST, 5:00" },
{ "5:00", "5:00", 0, true, "5:00, 5:02" },
{ "5:00", "5:00", 0, false, "5:00" },
{ "5:00", "5:00", 1, true, "5:00" },
{ "5:00", "5:00", 1, false, "5:00" },
{ "5:01", "5:01", 0, true, "5:00, 5:02" },
{ "5:01", "5:01", 0, false, "" },
{ "5:01", "5:01", 1, true, "5:00" },
{ "5:01", "5:01", 1, false, "" },
};
[Theory]
[MemberData(nameof(Table1))]
public async Task Part11_table1_first_page_on_a_reverse_capable_backend(string start, string end, uint max, bool bounds, string expected) =>
await AssertFirstPage(Shape.Reverse, start, end, max, bounds, expected);
[Theory]
[MemberData(nameof(Table1))]
public async Task Part11_table1_first_page_on_a_forward_only_backend(string start, string end, uint max, bool bounds, string expected) =>
await AssertFirstPage(Shape.ForwardOnly, start, end, max, bounds, expected);
[Theory]
[MemberData(nameof(Table1))]
public async Task Part11_table1_first_page_on_a_microsecond_backend(string start, string end, uint max, bool bounds, string expected) =>
await AssertFirstPage(Shape.Microsecond, start, end, max, bounds, expected);
private async Task AssertFirstPage(Shape shape, string start, string end, uint max, bool bounds, string expected)
{
var s = start.Length == 0 ? Unspecified : T(start);
var e = end.Length == 0 ? Unspecified : T(end);
var source = Source(shape, TableData());
var (values, error, _) = await ReadAsync(source, s, e, max, bounds, cp: null);
error.StatusCode.Code.ShouldBe(StatusCodes.Good);
Render(values, s, e).ShouldBe(expected);
(source as SeriesSource)?.ReverseWindowsOnForwardOnly.ShouldBe(0);
}
/// <summary>
/// Part 11: bounds count toward NumValuesPerNode, the start bound comes with the first page and the end bound with the
/// last — so paging a bounded two-sided read at any page size concatenates to the unpaged (N = 0) answer.
/// </summary>
[Theory]
[InlineData(Shape.Reverse)]
[InlineData(Shape.ForwardOnly)]
[InlineData(Shape.Microsecond)]
public async Task Paging_a_bounded_read_concatenates_to_the_unpaged_answer(Shape shape)
{
var windows = new[]
{
("5:00", "5:05"), ("5:01", "5:04"), ("5:05", "5:00"), ("5:04", "5:01"), ("4:59", "5:05"), ("5:01", "5:07"),
("4:48", "4:48"), ("5:00", "5:00"), ("5:01", "5:01"), ("5:07", "4:00"), ("5:06", "5:06"),
};
foreach (var (start, end) in windows)
{
var source = Source(shape, TableData());
var all = await ReadAsync(source, T(start), T(end), 0, true, cp: null);
all.Cp.ShouldBeNull();
var expected = Render(all.Values, T(start), T(end));
for (uint n = 1; n <= 6; n++)
{
var paged = await DrainAsync(source, T(start), T(end), n, bounds: true);
Render(paged, T(start), T(end)).ShouldBe(expected, $"{start}->{end} paged by {n}");
}
}
}
/// <summary>One-sided reads page to the open placeholder exactly once (footnotes a/b), never past it.</summary>
[Theory]
[InlineData(Shape.Reverse)]
[InlineData(Shape.ForwardOnly)]
[InlineData(Shape.Microsecond)]
public async Task Paging_a_one_sided_bounded_read_ends_with_the_open_placeholder(Shape shape)
{
for (uint n = 1; n <= 7; n++)
{
var fwd = await DrainAsync(Source(shape, TableData()), T("5:01"), Unspecified, n, bounds: true);
Render(fwd, T("5:01"), Unspecified).ShouldBe("5:00, 5:02, 5:03, 5:05, 5:06, LAST+", $"forward by {n}");
var rev = await DrainAsync(Source(shape, TableData()), Unspecified, T("5:04"), n, bounds: true);
Render(rev, Unspecified, T("5:04")).ShouldBe("5:05, 5:03, 5:02, 5:00, FIRST-", $"reverse by {n}");
}
}
/// <summary>A bound is a stored sample of any quality: a Bad sample before StartTime is the start bound, as stored.</summary>
[Theory]
[InlineData(Shape.Reverse)]
[InlineData(Shape.ForwardOnly)]
[InlineData(Shape.Microsecond)]
public async Task A_bad_sample_is_returned_as_the_bound_as_stored(Shape shape)
{
var series = TableData();
series[0] = new DataValueSnapshot(null, StatusCodes.BadNoCommunication, T("5:00"), T("5:00"));
var (values, _, _) = await ReadAsync(Source(shape, series), T("5:01"), T("5:04"), 0, true, cp: null);
values[0].StatusCode.Code.ShouldBe(StatusCodes.BadNoCommunication);
values[0].SourceTimestamp.ShouldBe(T("5:00"));
values.Count.ShouldBe(4);
}
/// <summary>
/// Ties on a bound: the start bound before a forward window is the LAST of a tie cluster (the sample immediately
/// before StartTime in the read's order); the end bound after it is the FIRST of one; and ties ON StartTime are all
/// data (the bound is the first of them, so nothing extra is added).
/// </summary>
[Theory]
[InlineData(Shape.Reverse)]
[InlineData(Shape.ForwardOnly)]
[InlineData(Shape.Microsecond)]
public async Task Tie_clusters_on_either_side_yield_one_bound_each(Shape shape)
{
var series = new List<DataValueSnapshot>();
void Add(string t, double v) => series.Add(new DataValueSnapshot(v, StatusCodes.Good, T(t), T(t)));
Add("5:00", 1); Add("5:00", 2); Add("5:00", 3);
Add("5:02", 4);
Add("5:05", 5); Add("5:05", 6);
var (fwd, _, _) = await ReadAsync(Source(shape, series), T("5:01"), T("5:04"), 0, true, cp: null);
fwd.Select(v => (double)v.Value).ShouldBe([3d, 4d, 5d]);
var (rev, _, _) = await ReadAsync(Source(shape, series), T("5:04"), T("5:01"), 0, true, cp: null);
rev.Select(v => (double)v.Value).ShouldBe([5d, 4d, 3d]);
var (onStart, _, _) = await ReadAsync(Source(shape, series), T("5:00"), T("5:03"), 0, true, cp: null);
onStart.Select(v => (double)v.Value).ShouldBe([1d, 2d, 3d, 4d, 5d]);
}
/// <summary>
/// QuestDB truncates the window bounds it is sent to the microsecond: a sample at ⌊StartTime⌋ is inside a window that
/// starts a few ticks later, so it is data — never returned a second time as the start bound.
/// </summary>
[Fact]
public async Task A_microsecond_backend_does_not_return_a_sample_at_the_truncated_start_twice()
{
var series = TableData();
var (values, _, _) = await ReadAsync(Source(Shape.Microsecond, series), T("5:00").AddTicks(7), T("5:04"), 0, true, cp: null);
values.Select(v => (double)v.Value).ShouldBe([0d, 2d, 3d, 5d]);
}
/// <summary>
/// A backend whose forward end is inclusive (the gateway's own <c>ReadRaw</c>), or a resumed read that steps onto
/// EndTime, returns the sample AT EndTime as data — it must come back once, as the end bound.
/// </summary>
[Theory]
[InlineData(0u)]
[InlineData(1u)]
[InlineData(2u)]
[InlineData(3u)]
public async Task An_inclusive_forward_end_does_not_return_the_end_bound_twice(uint max)
{
var source = new SeriesSource(TableData(), reverse: false, inclusiveForwardEnd: true, resolution: 1);
var values = await DrainAsync(source, T("5:00"), T("5:05"), max, bounds: true);
Render(values, T("5:00"), T("5:05")).ShouldBe("5:00, 5:02, 5:03, 5:05");
}
/// <summary>
/// #691 review: no data AND no bound found ⇒ <c>GoodNoData</c> with no values — the answer a tag with no history (and a
/// disabled historian) gave before bounds (§6.5.3.2: bounds make the result Good only "if one or both exist"). Only
/// when the WHOLE read is empty: a single bound found keeps the placeholders (Table 1: "4:48 4:48 1 ⇒ FIRST").
/// </summary>
[Theory]
[InlineData(Shape.Reverse)]
[InlineData(Shape.ForwardOnly)]
[InlineData(Shape.Microsecond)]
public async Task No_data_and_no_bound_found_is_GoodNoData_with_no_values(Shape shape)
{
var windows = new (string S, string E)[] { ("5:00", "5:05"), ("5:05", "5:00"), ("5:00", "5:00"), ("5:00", ""), ("", "5:05") };
foreach (var (start, end) in windows)
{
var s = start.Length == 0 ? Unspecified : T(start);
var e = end.Length == 0 ? Unspecified : T(end);
foreach (var n in new uint[] { 0, 1, 2, 3 })
{
if (n == 0 && (s == Unspecified || e == Unspecified)) continue; // a one-sided read needs a cap
var (values, error, cp) = await ReadResultAsync(Source(shape, []), s, e, n);
values.ShouldBeEmpty($"{start}->{end} by {n}");
cp.ShouldBeNull();
error.StatusCode.Code.ShouldBe(StatusCodes.Good);
LastNodeStatus.Code.ShouldBe(StatusCodes.GoodNoData, $"{start}->{end} by {n}");
}
}
// Positive control: one bound found keeps the missing one's placeholder.
var single = await DrainAsync(Source(shape, TableData()), T("4:48"), T("4:48"), 1, bounds: true);
Render(single, T("4:48"), T("4:48")).ShouldBe("FIRST, 5:00");
}
/// <summary>
/// #691 review: <c>ServerHistorian:HonorReturnBounds = false</c> restores the pre-#691 answer — ReturnBounds is ignored
/// (Table 1's "No" rows for every "Yes" row) and a modified read with ReturnBounds is not refused for it.
/// </summary>
[Fact]
public async Task The_kill_switch_ignores_return_bounds()
{
await fx.Gate.WaitAsync(TestContext.Current.CancellationToken);
try
{
fx.Nm.HonorReturnBounds = false;
fx.Nm.HistorianDataSource = Source(Shape.Reverse, TableData());
var (values, _, _) = Read(T("5:01"), T("5:07"), 0, bounds: true, cp: null, modified: false);
Render(values, T("5:01"), T("5:07")).ShouldBe("5:02, 5:03, 5:05, 5:06");
var (_, modErr, _) = Read(T("5:00"), T("5:05"), 0, bounds: true, cp: null, modified: true);
modErr.StatusCode.Code.ShouldBe(StatusCodes.BadHistoryOperationUnsupported, "no trail on this fake — not InvalidArgument");
}
finally
{
fx.Nm.HonorReturnBounds = true;
fx.Gate.Release();
}
}
[Fact]
public async Task Bounds_are_honoured_by_default()
{
await fx.Gate.WaitAsync(TestContext.Current.CancellationToken);
try { fx.Nm.HonorReturnBounds.ShouldBeTrue(); }
finally { fx.Gate.Release(); }
}
/// <summary>Part 11 §6.5.3.3: a modified read with ReturnBounds is <c>Bad_InvalidArgument</c>.</summary>
[Fact]
public async Task A_modified_read_with_return_bounds_is_an_invalid_argument()
{
await fx.Gate.WaitAsync(TestContext.Current.CancellationToken);
try
{
fx.Nm.HistorianDataSource = Source(Shape.Reverse, TableData());
var (_, error, cp) = Read(T("5:00"), T("5:05"), 0, bounds: true, cp: null, modified: true);
error.StatusCode.Code.ShouldBe(StatusCodes.BadInvalidArgument);
cp.ShouldBeNull();
}
finally { fx.Gate.Release(); }
}
// --- helpers --------------------------------------------------------------------------------
private static string Render(IReadOnlyList<DataValue> values, DateTime start, DateTime end)
{
var parts = new List<string>();
for (var i = 0; i < values.Count; i++)
{
var v = values[i];
if (v.StatusCode.Code != StatusCodes.BadBoundNotFound)
{
v.Value.ShouldNotBeNull();
parts.Add($"{v.SourceTimestamp.Hour}:{v.SourceTimestamp.Minute:00}");
continue;
}
v.Value.ShouldBeNull("a missing bound carries a null value");
var isFirst = i == 0 && start != Unspecified && v.SourceTimestamp == start;
var isLast = end != Unspecified && v.SourceTimestamp == end;
if (isFirst) { parts.Add("FIRST"); continue; }
if (isLast) { parts.Add("LAST"); continue; }
// Open side (footnotes a/b): one second from the previous returned value.
i.ShouldBeGreaterThan(0);
var prev = values[i - 1].SourceTimestamp;
if (v.SourceTimestamp == prev.AddSeconds(1)) parts.Add("LAST+");
else if (v.SourceTimestamp == prev.AddSeconds(-1)) parts.Add("FIRST-");
else parts.Add($"?{v.SourceTimestamp:O}");
}
return string.Join(", ", parts);
}
private async Task<List<DataValue>> DrainAsync(IHistorianDataSource source, DateTime start, DateTime end, uint max, bool bounds)
{
var all = new List<DataValue>();
byte[]? cp = null;
var pages = 0;
do
{
var (values, error, next) = await ReadAsync(source, start, end, max, bounds, cp);
error.StatusCode.Code.ShouldBe(StatusCodes.Good);
if (max > 0) values.Count.ShouldBeLessThanOrEqualTo((int)max, "bounds count toward NumValuesPerNode");
all.AddRange(values);
cp = next;
(++pages).ShouldBeLessThan(50, "paging must terminate");
}
while (cp is not null);
return all;
}
/// <summary>The node-level StatusCode of the last <see cref="Read"/> (GoodNoData vs Good).</summary>
private StatusCode LastNodeStatus;
private async Task<(IReadOnlyList<DataValue> Values, ServiceResult Error, byte[]? Cp)> ReadResultAsync(
IHistorianDataSource source, DateTime start, DateTime end, uint max) =>
await ReadAsync(source, start, end, max, bounds: true, cp: null);
private async Task<(IReadOnlyList<DataValue> Values, ServiceResult Error, byte[]? Cp)> ReadAsync(
IHistorianDataSource source, DateTime start, DateTime end, uint max, bool bounds, byte[]? cp)
{
await fx.Gate.WaitAsync(TestContext.Current.CancellationToken);
try
{
fx.Nm.HistorianDataSource = source;
return Read(start, end, max, bounds, cp, modified: false);
}
finally { fx.Gate.Release(); }
}
private (IReadOnlyList<DataValue> Values, ServiceResult Error, byte[]? Cp) Read(
DateTime start, DateTime end, uint max, bool bounds, byte[]? cp, bool modified)
{
var context = new OperationContext(new RequestHeader(), secureChannelContext: null, RequestType.HistoryRead, identity: null);
var details = new ReadRawModifiedDetails
{
StartTime = start, EndTime = end, NumValuesPerNode = max, ReturnBounds = bounds, IsReadModified = modified,
};
var nodesToRead = new List<HistoryReadValueId> { new() { NodeId = fx.NodeId, ContinuationPoint = cp } };
var results = new List<SdkHistoryReadResult> { null! };
var errors = new List<ServiceResult> { null! };
fx.Nm.HistoryRead(context, details, TimestampsToReturn.Both, releaseContinuationPoints: false, nodesToRead, results, errors);
LastNodeStatus = results[0]?.StatusCode ?? StatusCodes.Bad;
var values = new List<DataValue>();
if (results[0]?.HistoryData is { } ho && !ExtensionObject.IsNull(ho))
values.AddRange(((HistoryData)ExtensionObject.ToEncodeable(ho)).DataValues);
return (values, errors[0], results[0]?.ContinuationPoint);
}
/// <summary>
/// A series-backed historian. <paramref name="reverse"/>: serves <c>start &gt; end</c> as <c>(end, start]</c>
/// newest-first (SqlServer/QuestDB); otherwise rejects it (the Gateway) and counts that. The forward read is
/// <c>[start, end)</c>, or <c>[start, end]</c> when <paramref name="inclusiveForwardEnd"/> (the gateway's own end);
/// a point read is the instant. <paramref name="resolution"/> &gt; 1 truncates every window bound to it first (QuestDB
/// µs); the series must already be stored at that resolution.
/// </summary>
private sealed class SeriesSource(IReadOnlyList<DataValueSnapshot> series, bool reverse, bool inclusiveForwardEnd, long resolution)
: IHistorianDataSource
{
public int ReverseWindowsOnForwardOnly;
public bool SupportsReverseRaw => reverse;
public long TimestampResolutionTicks => resolution;
private DateTime Trunc(DateTime t) => new(t.Ticks - t.Ticks % resolution, DateTimeKind.Utc);
public Task<HistorianRead> ReadRawAsync(
string fullReference, DateTime startUtc, DateTime endUtc, uint maxValuesPerNode, CancellationToken cancellationToken)
{
static DateTime Ts(DataValueSnapshot s) => s.SourceTimestampUtc ?? DateTime.MinValue;
startUtc = Trunc(startUtc);
endUtc = Trunc(endUtc);
List<DataValueSnapshot> window;
if (startUtc > endUtc)
{
if (!reverse)
{
Interlocked.Increment(ref ReverseWindowsOnForwardOnly);
throw new InvalidOperationException("start_time must be less than or equal to end_time. (InvalidArgument)");
}
window = series.Where(s => Ts(s) > endUtc && Ts(s) <= startUtc).Reverse().ToList();
}
else if (startUtc == endUtc) window = series.Where(s => Ts(s) == startUtc).ToList();
else window = series.Where(s => Ts(s) >= startUtc && (inclusiveForwardEnd ? Ts(s) <= endUtc : Ts(s) < endUtc)).ToList();
var page = maxValuesPerNode == 0 ? window : window.Take((int)maxValuesPerNode).ToList();
return Task.FromResult(new HistorianRead(page, null));
}
public Task<HistorianRead> ReadProcessedAsync(
string fullReference, DateTime startUtc, DateTime endUtc, TimeSpan interval,
HistoryAggregateType aggregate, CancellationToken cancellationToken) =>
Task.FromResult(new HistorianRead(Array.Empty<DataValueSnapshot>(), null));
public Task<HistorianRead> ReadAtTimeAsync(
string fullReference, IReadOnlyList<DateTime> timestampsUtc, CancellationToken cancellationToken) =>
Task.FromResult(new HistorianRead(Array.Empty<DataValueSnapshot>(), null));
public Task<HistoricalEventsResult> ReadEventsAsync(
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents, CancellationToken cancellationToken) =>
Task.FromResult(new HistoricalEventsResult(Array.Empty<HistoricalEvent>(), null));
public HistorianHealthSnapshot GetHealthSnapshot() => NullHistorianDataSource.Instance.GetHealthSnapshot();
public void Dispose() { }
}
}
@@ -45,6 +45,7 @@ public sealed class NodeManagerHistoryReadConcurrencyTests : IDisposable
var details = new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = DateTime.UtcNow.AddHours(-1),
EndTime = DateTime.UtcNow,
NumValuesPerNode = 10,
@@ -78,6 +79,7 @@ public sealed class NodeManagerHistoryReadConcurrencyTests : IDisposable
var nodeIds = MaterializeHistorizedNodes(nm, count: 12);
var details = new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = DateTime.UtcNow.AddHours(-1),
EndTime = DateTime.UtcNow,
NumValuesPerNode = 10,
@@ -115,6 +117,7 @@ public sealed class NodeManagerHistoryReadConcurrencyTests : IDisposable
var nodeIds = MaterializeHistorizedNodes(nm, count: 8);
var details = new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = DateTime.UtcNow.AddHours(-1),
EndTime = DateTime.UtcNow,
NumValuesPerNode = 10,
@@ -157,6 +160,7 @@ public sealed class NodeManagerHistoryReadConcurrencyTests : IDisposable
var nodeIds = MaterializeHistorizedNodes(nm, count: 4);
var details = new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = DateTime.UtcNow.AddHours(-1),
EndTime = DateTime.UtcNow,
NumValuesPerNode = 10,
@@ -190,6 +194,7 @@ public sealed class NodeManagerHistoryReadConcurrencyTests : IDisposable
var ids = MaterializeHistorizedNodes(nm, count: 2);
var details = new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = DateTime.UtcNow.AddHours(-1),
EndTime = DateTime.UtcNow,
NumValuesPerNode = 10,
@@ -160,7 +160,7 @@ public sealed class NodeManagerHistoryReadEventsPagingTests : IDisposable
var ctx = new OperationContext(new RequestHeader(), secureChannelContext: null, RequestType.HistoryRead, identity: null);
var res = new List<SdkHistoryReadResult> { null! };
var errs = new List<ServiceResult> { null! };
nm.HistoryRead(ctx, new ReadRawModifiedDetails { StartTime = Epoch, EndTime = Epoch.AddHours(1), NumValuesPerNode = 2, IsReadModified = false },
nm.HistoryRead(ctx, new ReadRawModifiedDetails { ReturnBounds = false, StartTime = Epoch, EndTime = Epoch.AddHours(1), NumValuesPerNode = 2, IsReadModified = false },
TimestampsToReturn.Both, false,
new List<HistoryReadValueId> { new() { NodeId = n, ContinuationPoint = cp } }, res, errs);
err = errs[0];
@@ -271,7 +271,7 @@ public sealed class NodeManagerHistoryReadModifiedTests : IDisposable
var results = new List<SdkHistoryReadResult> { null! };
var errors = new List<ServiceResult> { null! };
nm.HistoryRead(context,
new ReadRawModifiedDetails { StartTime = start, EndTime = end, NumValuesPerNode = max, IsReadModified = modified },
new ReadRawModifiedDetails { ReturnBounds = false, StartTime = start, EndTime = end, NumValuesPerNode = max, IsReadModified = modified },
TimestampsToReturn.Both, false,
new List<HistoryReadValueId> { new() { NodeId = node, ContinuationPoint = cp } }, results, errors);
@@ -967,6 +967,7 @@ public sealed class NodeManagerHistoryReadPagingTests : IDisposable
var details = new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = start,
EndTime = end,
NumValuesPerNode = max,
@@ -1000,6 +1001,7 @@ public sealed class NodeManagerHistoryReadPagingTests : IDisposable
new RequestHeader(), secureChannelContext: null, RequestType.HistoryRead, identity: null);
var details = new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = Epoch, EndTime = Epoch.AddHours(1), NumValuesPerNode = 100, IsReadModified = false,
};
var nodesToRead = new List<HistoryReadValueId> { new() { NodeId = nodeId, ContinuationPoint = cp } };
@@ -52,6 +52,7 @@ public sealed class NodeManagerHistoryReadTests : IDisposable
var end = DateTime.UtcNow;
var details = new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = start,
EndTime = end,
NumValuesPerNode = 100,
@@ -97,6 +98,7 @@ public sealed class NodeManagerHistoryReadTests : IDisposable
var details = new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = DateTime.UtcNow.AddHours(-1),
EndTime = DateTime.UtcNow,
NumValuesPerNode = 10,
@@ -129,6 +131,7 @@ public sealed class NodeManagerHistoryReadTests : IDisposable
var details = new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = DateTime.UtcNow.AddHours(-1),
EndTime = DateTime.UtcNow,
NumValuesPerNode = 10,
@@ -162,6 +165,7 @@ public sealed class NodeManagerHistoryReadTests : IDisposable
var details = new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = DateTime.UtcNow.AddHours(-1),
EndTime = DateTime.UtcNow,
NumValuesPerNode = 10,
@@ -195,6 +199,7 @@ public sealed class NodeManagerHistoryReadTests : IDisposable
var details = new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = DateTime.UtcNow.AddHours(-1),
EndTime = DateTime.UtcNow,
NumValuesPerNode = 10,
@@ -613,6 +618,7 @@ public sealed class NodeManagerHistoryReadTests : IDisposable
var details = new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = DateTime.UtcNow.AddHours(-1),
EndTime = DateTime.UtcNow,
NumValuesPerNode = 10,
@@ -658,6 +664,7 @@ public sealed class NodeManagerHistoryReadTests : IDisposable
var details = new ReadRawModifiedDetails
{
ReturnBounds = false,
StartTime = DateTime.UtcNow.AddHours(-1),
EndTime = DateTime.UtcNow,
NumValuesPerNode = 10,
@@ -104,6 +104,21 @@ public sealed class ServerHistorianOptionsTests
/// #674: the HistoryUpdate future-skew bound defaults to one day and binds from configuration as a TimeSpan;
/// a non-positive value is a deliberate "disabled", not a misconfiguration, so it does not warn.
/// </summary>
/// <summary>#691: the ReturnBounds kill switch defaults on and binds off; neither value warns.</summary>
[Fact]
public void HonorReturnBounds_defaults_to_true_and_binds_false_silently()
{
Assert.True(new ServerHistorianOptions().HonorReturnBounds);
var bound = new ConfigurationBuilder()
.AddInMemoryCollection(new Dictionary<string, string?> { ["ServerHistorian:HonorReturnBounds"] = "false" })
.Build()
.GetSection(ServerHistorianOptions.SectionName)
.Get<ServerHistorianOptions>();
Assert.False(bound!.HonorReturnBounds);
Assert.Empty(new ServerHistorianOptions
{ Enabled = true, Endpoint = "https://h:5222", ApiKey = "histgw_x_y", HonorReturnBounds = false }.Validate());
}
[Fact]
public void HistoryUpdateMaxFutureSkew_defaults_to_one_day_binds_and_zero_is_silent()
{