diff --git a/CLAUDE.md b/CLAUDE.md index 1b49cc92..9d70fe72 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1355,6 +1355,7 @@ historian tagname, which is the tag's driver `FullName`. - ⚠️ **Reverse processed reads are Part 13/SDK-anchored (#677, client-visible):** every aggregate on SqlServer/QuestDB now anchors a reverse read at its later time — window `(end, start]`, partial interval earliest, each stamped with its later edge (was: earlier-anchored, stamped earlier edge); on forward reads Partial also marks the interval the data starts inside (not Average — the SDK's rule, forward-only); a non-default client `AggregateConfiguration` ⇒ `BadAggregateConfigurationRejected`; Gateway provisioning sends `IntegralDivisor = 1`. SDK-parity tests: `SdkAggregateParityTests`. `docs/Historian.md` §Aggregates. - ⚠️ **Reverse / end-only RAW reads work on Gateway (#686, client-visible):** `IHistorianDataSource.SupportsReverseRaw` (SqlServer/QuestDB/Null `true`, Gateway `false`, interface default `false`, guard-tested) — the node manager sends `start > end` only to a backend that declares it; otherwise `ReverseRawReader` serves it from FORWARD reads: uncapped ⇒ ONE forward read reversed in memory; capped ⇒ a bounded walk back in segments (ms-aligned request bounds, width aimed from observed density at ≈ N + 8 rows and carried page to page in the continuation point — #689: it used to aim at half `MaxTieClusterOverfetch`, ~32k rows per page on a dense tag; a read asks for `2N + 18` rows (a discovery segment ≤ 1,026), `MaxTieClusterOverfetch` is only the tie-cluster ceiling — a full read is re-read, never answered truncated; only more than max(1, `MaxTieClusterOverfetch`) samples within ONE MILLISECOND ⇒ `BadTooManyOperations` (per ms, not per timestamp — requests are ms-rounded, and letting a segment narrow below 1 ms stepped a page through one ms a tick per read, ~10k reads); walks no later than now + 1 day and never before the Unix epoch; #691: after two empty segments it asks once for the whole remainder, and the stretch a page proved empty rides the continuation point — the last pages of a deep page-through no longer each walk ~14 reads to 1970). The raw/events tie-path oversized-cluster refusal is now `BadTooManyOperations` too (was `BadHistoryOperationUnsupported`). Uncapped Gateway raw reads (forward too) used to send `max_values = 0`, which the client/gateway reject — now unbounded (`int.MaxValue`; a ceiling refusal ⇒ `BadTooManyOperations`), and a cancelled read is no longer a backend failure. Since #669 such reads failed the node on Gateway and counted as a backend fault (the gateway rejects start > end — `GatewayHistorianDataSource.ReadRawAsync` now refuses one unsent); before #669 an end-only Gateway read returned the OLDEST N ascending — now the latest N at/before EndTime, newest first. Events follow since #691.4: a reverse/end-only capped events read on Gateway (`SupportsReverseEvents` false; Null now reports `true` and answers empty) is walked back the same way (ties sorted by EventId — the gateway orders by time only; one extra event found ⇒ continuation point) — it used to be refused `BadHistoryOperationUnsupported` (explicit reverse) or answered with the OLDEST N ascending (end-only); needs the gateway's `RuntimeDb:EventReadMaxRows` 0/unlimited (it truncates silently). Uncapped Gateway events reads (forward too) used to throw "Array dimensions exceeded" (result list pre-sized to `int.MaxValue`) — fixed. Above a truncated read the walk gallops up from the newest row seen, then bisects (never below 1 ms): a burst that ended 100 days ago ≈ 18 reads, a dense tag that stopped days ago ≈ 26. `docs/Historian.md` §"Open-ended and reverse raw reads". - **Advertised aggregates (#682):** `IHistorianDataSource.SupportedAggregates` (Gateway 4 via `GatewayHistorianDataSource.ServedAggregates`, SqlServer/QuestDB 17 via `AggregateFamilies.Served`, Null **none**; a null set ⇒ all) is the one authority for `HistoryServerCapabilities/AggregateFunctions` (i=11201 — only its folder references are trimmed) and the up-front `BadAggregateNotSupported` in HistoryRead-Processed. ⚠️ `ServerCapabilities/AggregateFunctions` (i=2997) is deliberately NOT trimmed: it keeps the SDK's 37 because live `AggregateFilter` subscriptions still compute them all. ⚠️ Historian disabled ⇒ Processed is now `BadAggregateNotSupported`, not `GoodNoData`. `docs/Historian.md` §Advertised aggregates. +- **Raw `ReturnBounds` is honoured on every backend (#691, client-visible):** Part 11 bounding values (§4.6 Table 1 is the test oracle) are composed in the server by `HistoryBounds` from each backend's raw reads — "at or before" = capped reverse read (native on SQL backends, the #686 `ReverseRawReader` walk-back on Gateway, never AVEVA's synthetic boundary row), "at or after" = capped forward read. Bounds are stored samples of any quality, count toward `NumValuesPerNode`, start bound on the first page / end bound on the last (pending end bound rides the continuation point); missing ⇒ `Bad_BoundNotFound` (null, at the bound time; a one-sided read's open side ±1 s from the previous value). ⚠️ **Breaking for SDK-default clients:** the .NET SDK's `new ReadRawModifiedDetails()` defaults BOTH `ReturnBounds = true` AND `IsReadModified = true` — a client that cleared only `IsReadModified` now gets bounds, and one that set neither is a modified read with bounds ⇒ `Bad_InvalidArgument` (§6.5.3.3); set both explicitly (Client.Shared does; tests building details must too). No data AND no bound found ⇒ `GoodNoData`, no values (the pre-bounds answer). On Gateway the at-or-after lookup ends at now + 1 day (never a year-9999 end to AVEVA). Kill switch `ServerHistorian:HonorReturnBounds` (default `true`; `false` = pre-#691). `docs/Historian.md` §Bounding values. - **Real status fix:** the recorder used to stamp every sample Good (`192`); it now carries the driver's status. ⚠️ On the **Gateway** backend this changes nothing stored: the gateway discards quality (C-002), so AVEVA still stores Good. - **Comms loss is historized as an outage marker** (`Empty` + `BadNoCommunication`, one per tag per outage) on SqlServer; dropped (`DroppedUnsupportedKind`) on Gateway. Outbox records are **format v2** — drain or delete `ContinuousHistorization:OutboxPath` before downgrading a node. - **HistoryUpdate** (SqlServer + QuestDb; Gateway ⇒ `BadHistoryOperationUnsupported`): Insert/Replace/Update/DeleteRaw/DeleteAtTime, **Primary-only** since #674 (off-Primary ⇒ `BadInvalidState`, after authorization AND after planning — an unservable request (Remove / `IsDeleteModified` / not historized) stays `BadHistoryOperationUnsupported` off-Primary; the verdict is `IHistoryUpdateRoleView` = the data-plane verdict but **closed until this node's first redundancy snapshot**, so a pair cannot both accept in the boot window; admin-only nodes register no view ⇒ no gate; reads ungated), items with a `SourceTimestamp` past now + `ServerHistorian:HistoryUpdateMaxFutureSkew` (default 1 day, ≤0 disables) ⇒ `BadOutOfRange` in the node manager (every backend), gated in both ACL modes — flag off needs the `HistoryUpdate` role from a **Cluster-scoped** grant (+ key re-check), flag on uses `DataPlaneAuthorizer`'s HistoryUpdate operation; `WriteOperate` never implies it. The `HistoryWrite` AccessLevel bit is set only when an updater is wired at materialise time. `docs/Historian.md` §HistoryUpdate, `docs/security.md`. ⚠️ **SqlServer schema v1 is accepted (#674)** with `AutoCreateSchema=false`: writes/reads/events/retention work; modified reads + HistoryUpdate ⇒ node-level `BadHistoryOperationUnsupported` on every arm (the updater throws `NotSupportedException`, nothing sent; the `HistoryWrite` bit is still advertised — not knowable at materialise time), retention skips `SampleModified`, one Warning per boot (`SqlServerSchemaBootstrapper.SupportsModifiedTrail`; `RequiredVersion` = 1, `ModifiedTrailVersion` = 2). QuestDB: keep `cairo.ttl.use.wall.clock=true`; a QuestDB trail INSERT retried after a client timeout duplicates (no DEDUP key). @@ -1381,6 +1382,7 @@ authorization uses the standard `AccessLevels.HistoryRead` bit set at materializ | `CaCertificatePath` | `null` | PEM CA file pinning the gateway TLS chain; null/empty uses the OS trust store | | `CallTimeout` | `00:00:30` | Per-call deadline for each unary gateway read | | `MaxTieClusterOverfetch` | `65536` | Bounded over-fetch the HistoryRead-Raw paging uses to page within an oversized same-timestamp tie cluster (retained from the prior backend) | +| `HonorReturnBounds` | `true` | #691 kill switch: `false` ignores HistoryRead-Raw `ReturnBounds` as before (no bounding values, no modified-read `Bad_InvalidArgument`) | | `HistoryReadBatchParallelism` | `4` | Max nodes served concurrently within one HistoryRead batch (03/S3); `⌈N/P⌉ × CallTimeout` worst case instead of sequential | | `MaxConcurrentHistoryReadBatches` | `16` | Process-wide concurrent HistoryRead batch cap (03/S3); flood degrades to `BadTooManyOperations`. Total in-flight reads capped at 16 × 4 = 64 | | `MaxTimeWeightedRawRows` | `1000000` | `SqlServer`/`QuestDb` (#671): raw in-window rows one time-weighted aggregate read (incl. `Total` and, since #680, PercentGood/PercentBad) may stream into the C# evaluator — and, since #681, every `Gateway` per-sample read (raw-evaluated); window query capped at this + 1, past it ⇒ `BadTooManyOperations` for that node. ≤ 0 = unbounded. Gateway (#688): the `cap + 18` budget is clamped at the gateway's default `MaxValuesCeiling` (1,000,000) while the cap is ≤ it, so the default no longer trips a strict (`AllowStreamingBeyondCeiling=false`) gateway | diff --git a/docs/Historian.md b/docs/Historian.md index 40aae51a..083360ed 100644 --- a/docs/Historian.md +++ b/docs/Historian.md @@ -111,6 +111,7 @@ Processed read is `BadAggregateNotSupported` and no history aggregate functions "CaCertificatePath": null, "CallTimeout": "00:00:30", "MaxTieClusterOverfetch": 65536, + "HonorReturnBounds": true, "HistoryReadBatchParallelism": 4, "MaxConcurrentHistoryReadBatches": 16, "HistoryReadDeadline": "00:01:00", @@ -144,6 +145,7 @@ Processed read is `BadAggregateNotSupported` and no history aggregate functions | `CaCertificatePath` | string\|null | `null` | **Gateway backend only.** PEM CA file pinning the gateway's TLS chain. Null/empty uses the OS trust store. | | `CallTimeout` | TimeSpan | `00:00:30` | **Gateway backend only.** Per-call deadline applied to each unary gateway read. A read for a tag the historian does not yet know about — e.g. a freshly-historized tag that has **not** been provisioned (see [Tag auto-provisioning](#tag-auto-provisioning-ensuretags)) — can run to this **full** deadline before the gateway errors, so an unprovisioned-tag HistoryRead can block for ~30 s. Lower this to fail faster, at the cost of truncating legitimately long reads. | | `MaxTieClusterOverfetch` | int | `65536` | (Node-manager paging; all backends — a `SqlServer` PK rules out tie clusters, so it is moot there.) Maximum samples the server will fetch in one shot to page through a tie cluster (multiple samples sharing one `SourceTimestamp`). A cluster larger than this ceiling fails `BadTooManyOperations` (#686; was `BadHistoryOperationUnsupported`). Also the per-read row budget of a Gateway reverse raw read (walked back from forward reads, #686): one timestamp holding more samples is refused `BadTooManyOperations`. Values below 1 count as 1. Raise to handle abnormally large tie clusters; the default covers all normal-data cases. | +| `HonorReturnBounds` | bool | `true` | (#691, all backends.) Whether HistoryRead-Raw honours `ReturnBounds` (Part 11 bounding values, [Bounding values](#bounding-values-returnbounds-691)). `false` is the kill switch back to the pre-#691 behaviour: the flag is ignored (stored samples only) and a modified read carrying it is not refused `Bad_InvalidArgument`. Note the .NET SDK defaults both `ReturnBounds` and `IsReadModified` to `true`. | | `HistoryReadBatchParallelism` | int | `4` | Max historized nodes served **concurrently** within one HistoryRead batch (arch-review 03/S3). The SDK's `HistoryRead*` override surface is synchronous, so each arm bridges once at its boundary; this turns per-node serving from sequential (`N × CallTimeout` worst case) into `⌈N / P⌉ × CallTimeout`. Also caps the gateway-load multiplication a single client can command. `≤ 1` serves sequentially. | | `MaxConcurrentHistoryReadBatches` | int | `16` | Process-wide limit on HistoryRead (and HistoryUpdate) batches served **concurrently** (arch-review 03/S3). A flood of history clients degrades to `BadTooManyOperations` rather than exhausting the SDK request-thread pool. With `HistoryReadBatchParallelism` this caps total in-flight gateway reads at `MaxConcurrentHistoryReadBatches × HistoryReadBatchParallelism` (default 16 × 4 = 64). A batch that cannot get a slot within `min(HistoryReadDeadline, 5 s)` fails every handle `BadTooManyOperations`. | | `MaxTimeWeightedRawRows` | int | `1000000` | `SqlServer` / `QuestDb` (#671), and the `Gateway` per-sample aggregates (#681 — every per-sample read there streams raw samples, so the cap applies to the window's raw rows; see [Aggregate set](#aggregate-set-and-badaggregatenotsupported-285)). The most raw in-window samples one [time-weighted aggregate](#time-weighted-aggregates-part-13-671) read (TimeAverage, TimeAverage2, **Total**, Total2, Interpolative, and — since #680 — **PercentGood/PercentBad**) may stream into the server-side evaluator — that family is **not** pushed down to SQL. The window query is capped at this + 1 rows; one row past the cap fails **that node** with `BadTooManyOperations` (not counted as a backend fault) instead of holding a pooled connection until `HistoryReadDeadline` and returning `BadTimeout`. Bound rows outside the window do not count; per-sample aggregates are not capped on the SQL backends (pushed down). 1 000 000 ≈ 11.5 days of a 1 Hz tag. Non-positive = unbounded (a `Validate()` warning). `Gateway` (#688): the row budget it asks the gateway for (`cap + 18`) is clamped at the gateway's default `MaxValuesCeiling` (1,000,000) while the cap is at or under it, so the default works against a gateway with `AllowStreamingBeyondCeiling=false`; against a gateway with a *lower* ceiling the refused budget is retried once at the ceiling the gateway names (#691.3), so no change is needed (`ceiling − 18` only saves that round-trip). | @@ -752,10 +754,60 @@ data. asking for `NumValuesPerNode + m + 16` (m = the pre-window rows the first read saw, unclamped). Only if that re-read is still short because of pre-window rows does the node fail `BadTooManyOperations` (not a backend fault) rather than answer a short page that would end the client's paging early. Every gateway stream is still drained to completion. -- **`ReturnBounds` is not honoured** — on any backend: the server does not pass it to the data source, so no bounding values are - returned (see [Known limitations](#known-limitations)). The AVEVA boundary row is not used as a bound either: it is stamped at the - query start rather than carrying the time of the real prior sample, so it could only ever be a Part 11 *interpolated* bound, which - this backend does not compute. +- **`ReturnBounds` is served by the server, not by this row** (#691, [Bounding values](#bounding-values-returnbounds-691)). The AVEVA + boundary row is never used as a bound: it is stamped at the query start rather than carrying the time of the real prior sample, so + it could only ever be a Part 11 *interpolated* bound. The start bound on the Gateway is the real prior sample, found by the + #686 walk-back. + +### Bounding values (`ReturnBounds`, #691) + +⚠️ **Client-visible — a breaking change for .NET clients that rely on SDK defaults.** Until #691 +`ReadRawModifiedDetails.ReturnBounds` was ignored on every backend. It is now honoured on **every** backend (SqlServer, QuestDB, +Gateway, and none), following OPC UA Part 11 §4.6 / §6.5.3.2 and checked against every row of Part 11 Table 1. **The OPC Foundation +.NET SDK's `new ReadRawModifiedDetails()` defaults BOTH `ReturnBounds = true` AND `IsReadModified = true`** (verified on +1.5.378.106), so: + +- a client that clears `IsReadModified` but never sets `ReturnBounds` used to get stored values only and **now gets bounding values**; +- a client that sets neither (a bare `new ReadRawModifiedDetails { StartTime, EndTime }`) is a *modified* read with bounds, which + used to be served (or `BadHistoryOperationUnsupported` on a backend with no trail) and is **now `Bad_InvalidArgument`** (§6.5.3.3). + +Set both explicitly. `Client.Shared` (Client.CLI / Client.UI) already does, and ScadaBridge does not issue HistoryRead. The kill +switch `ServerHistorian:HonorReturnBounds = false` restores the pre-#691 behaviour (both changes) fleet-wide. + +- **What a bound is.** Forward (`StartTime < EndTime`): the start bound is the stored sample **at or immediately before** + StartTime, the end bound the one **at or immediately after** EndTime. Reverse mirrors it (start bound at or after StartTime, end + bound at or before EndTime), newest first. A sample exactly **on** StartTime is the bound and is already the data's first value + (nothing extra is added); a sample exactly on EndTime is added as the end bound (the data window excludes EndTime). + `StartTime == EndTime` reads forward: the data is the samples at that instant, the end bound the first sample strictly after it. +- **A bound is a stored sample of any quality** — a Bad sample or a comms-loss outage marker comes back as stored. Inside a tie + cluster the bound is the cluster's first sample in the read's direction. +- **Nothing at all ⇒ `GoodNoData`.** A read with no data in the window **and** neither bound found returns no values and + `GoodNoData` — the answer it gave before bounds (§6.5.3.2: bounds make the result a success only "if one or both exist"). This is + judged over the whole read (it only arises on a single, final first page); once any bound or value exists, a missing bound is + returned as its placeholder. +- **Missing bound:** `Bad_BoundNotFound`, null value, stamped with StartTime / EndTime. A one-sided read's open side (only a + StartTime, or only an EndTime) is never looked up; it ends the read as `Bad_BoundNotFound` stamped one second after (forward) or + before (reverse) the previous returned value (Table 1 footnotes a/b). `Bad_BoundNotSupported` is never returned — every backend + can find a bound. +- **Counting and paging.** Bounds **count toward `NumValuesPerNode`** (§6.5.3.2: "any Bounding Values returned are included in the + numValuesPerNode count"; `NumValuesPerNode = 1` returns the start bound alone). The start bound leads the **first** page and the + end bound ends the **last**: the pending end bound rides the continuation point, and when a page is exactly full the end bound is + the whole next page. Paging a bounded read at any page size concatenates to the unpaged answer. +- **How it is served — one authority, `HistoryBounds` in the OPC UA server.** "At or before T" is a newest-first read of `(0, T]` + capped at one — native on SqlServer/QuestDB, the #686 walk-back of bounded forward reads on the Gateway (a tag with no sample + before StartTime is proven empty down to 1970 by the #689 walk — after two empty segments it asks once for the whole remainder — + so a missing start bound costs a few small reads, not one per ×4 step). "At or after T" is a forward read + of `[T, MaxValue)` capped at one — on the **Gateway** it ends at **now + 1 day** instead (the walk-back's horizon; the + HistorianGateway only checks `start ≤ end`, so a year-9999 end would reach AVEVA), and an end bound beyond that horizon is + `Bad_BoundNotFound` without a read. SqlServer/QuestDB keep `MaxValue` (they bind it safely and may hold samples further ahead). + "Strictly after" starts one storage resolution later. On QuestDB, whose window bounds truncate to + the microsecond, "on the bound" compares at that resolution, so a sample at ⌊StartTime⌋ is never returned twice. A first page + therefore costs up to three backend reads (start bound, data, end bound). The `BoundingValueConformance` suite pins those exact + lookup reads on SqlServer and QuestDB. +- **Modified reads:** `IsReadModified = true` with `ReturnBounds = true` is `Bad_InvalidArgument` per node (Part 11 §6.5.3.3). +- **Kill switch:** `ServerHistorian:HonorReturnBounds` (default `true`). `false` ignores `ReturnBounds` exactly as before #691 — + stored samples only, and no modified-read rejection. +- **Not affected:** Processed, AtTime and Events reads (they have no `ReturnBounds`). ### Continuation-point paging (Events) (#669) @@ -1095,8 +1147,8 @@ whatever the SDK sets — #682 touches only the i=11201 folder (follow-up candid sample per requested timestamp, so the single-shot backend returns the complete result in one read and there is no "full page ⇒ maybe more" signal to page on. Returning the full result with no continuation point is spec-conformant. -- **`ReturnBounds` is ignored** (every backend): Raw reads return stored values only; no bounding values are added at the window - edges. The node manager does not pass the flag to `IHistorianDataSource`. +- **Bounds are raw only:** `ReturnBounds` (#691, [Bounding values](#bounding-values-returnbounds-691)) never interpolates a bound — + it is always a stored sample or `Bad_BoundNotFound`. - **No modified-value history on the `Gateway` backend.** Modified reads (`IsReadModified = true`) are served by `SqlServer` and `QuestDb` ([Modified-value history](#modified-value-history-part-11), #668); the HistorianGateway exposes no modified/edited-history surface, so on `Gateway` (and with no historian) they return `BadHistoryOperationUnsupported`. diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs b/src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs index 32af5abd..9ad9c414 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs @@ -317,6 +317,9 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl // section is absent; NodeManager is non-null here (guarded above). _server.NodeManager.MaxTieClusterOverfetch = _serverHistorianOptions.MaxTieClusterOverfetch; + // Gitea #691: the ReturnBounds kill switch (default on). + _server.NodeManager.HonorReturnBounds = HonorReturnBounds; + // Propagate the S3 HistoryRead concurrency bounds (arch-review 03/S3): bounded per-node fan-out // within a batch, the process-wide concurrent-batch limiter, and the per-request deadline. Defaults // (4 / 16 / 60 s) survive when the ServerHistorian section is absent. @@ -347,6 +350,12 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl _logger.LogInformation("OtOpcUaServerHostedService: SDK started, address-space + ServiceLevel sinks bound"); } + /// + /// Gitea #691 — the ServerHistorian:HonorReturnBounds kill switch StartAsync applies to the node manager + /// (default true when the section or key is absent). Exposed so the wiring is testable without booting the SDK. + /// + internal bool HonorReturnBounds => _serverHistorianOptions.HonorReturnBounds; + /// /// Gitea #674 — what StartAsync wires onto the node manager for HistoryUpdate: the Primary gate (read from /// the on every call; null when none is registered, i.e. allow), the diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/HistoryBounds.cs b/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/HistoryBounds.cs new file mode 100644 index 00000000..2dbf03a7 --- /dev/null +++ b/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/HistoryBounds.cs @@ -0,0 +1,219 @@ +using Opc.Ua; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.OpcUaServer; + +/// +/// The end bound still owed to a HistoryRead-Raw read with ReturnBounds = true (Gitea #691). It rides the +/// continuation point, because Part 11 returns the end bound with the LAST page only. +/// +/// The request's end time (the later bound of a forward read, the earlier of a reverse one). +/// The read is newest-first (StartTime > EndTime, including an end-only read). +/// +/// false only for StartTime == EndTime: the samples AT that instant are the data, so the end bound is the +/// first sample strictly after it (Part 11 Table 1 footnote c). Otherwise the end bound is the sample at or beyond the +/// end time, and the data window excludes the end time. +/// +/// The end time was not given (a one-sided read's sentinel): there is no end bound to look up. +/// The data is drained and the end bound did not fit on the previous page. +/// The last value's time returned on an earlier page (stamps an open bound's placeholder). +internal sealed record HistoryEndBound( + DateTime AtUtc, bool Reverse, bool Inclusive, bool Open, bool DataExhausted = false, DateTime? LastReturnedUtc = null); + +/// +/// Gitea #691: OPC UA Part 11 bounding values for HistoryRead-Raw (ReadRawModifiedDetails.ReturnBounds), composed +/// in the server from every backend's existing raw read — the ONE authority, so SqlServer, QuestDB, Gateway and Null +/// answer identically. +/// +/// +/// +/// What a bound is (Part 11 §4.6, Table 1). Forward: the start bound is the stored sample AT or immediately +/// BEFORE StartTime, the end bound the one AT or immediately AFTER EndTime. Reverse mirrors it: the start bound is at +/// or after StartTime, the end bound at or before EndTime. A sample exactly on a bound IS the bound: on the start it +/// is already the first sample of the data (the window includes StartTime), so nothing extra is added; on the end it is +/// added (the window excludes EndTime). StartTime == EndTime reads forward, and its end bound is the first +/// sample strictly after the instant. A bound is a stored sample of any quality — a Bad sample is returned as stored. +/// +/// +/// Missing bound. Bad_BoundNotFound, a null value, stamped with the bound's time. A one-sided read's +/// missing side (no EndTime / no StartTime) is never looked up; it is Bad_BoundNotFound stamped one second after +/// (forward) or before (reverse) the previous returned value (Table 1 footnotes a and b). +/// +/// +/// Lookups. "At or before T" is a newest-first read of (0, T] capped at one — native on a backend that +/// declares , otherwise 's walk-back +/// from bounded forward reads (the Gateway: NOT AVEVA's synthetic boundary row, which is stamped at the query start +/// rather than at the prior sample's own time). "At or after T" is a forward read of [T, MaxValue) capped at one (on a +/// forward-only backend ending at now + instead — #691 review); +/// "strictly after T" starts one storage resolution later (lossless: nothing is stored in between). Inside a tie +/// cluster the bound is the cluster's first sample in the read's direction — the forward read's first, the reverse +/// read's first (the forward order's last). +/// +/// +/// Resolution. "On the bound" compares at the backend's storage resolution, because a microsecond backend +/// (QuestDB) truncates the window bounds it is sent: a sample at ⌊T⌋ is inside a window starting at T +/// there, so it must not also be returned as the start bound. +/// +/// +internal static class HistoryBounds +{ + private static readonly TimeSpan OpenBoundStep = TimeSpan.FromSeconds(1); + + /// The end bound owed by a fresh read over the normalised window. + /// The (normalised) start: the later time of a reverse read. + /// The (normalised) end; a one-sided read's sentinel (tick 0 / MaxValue) is an open bound. + /// The pending end bound. + public static HistoryEndBound For(DateTime startUtc, DateTime endUtc) => + new(endUtc, Reverse: startUtc > endUtc, Inclusive: startUtc != endUtc, Open: IsOpen(endUtc)); + + /// Whether a returned value is a placeholder rather than a stored sample. + /// The value. + /// true for a Bad_BoundNotFound placeholder. + public static bool IsPlaceholder(DataValueSnapshot value) => + value.StatusCode == StatusCodes.BadBoundNotFound && value.Value is null; + + /// The pending end bound to store behind a continuation point, remembering the page's last returned time. + /// The pending end bound (null = bounds not requested). + /// The page being returned. + /// The bound to carry, or null. + public static HistoryEndBound? Carry(HistoryEndBound? bound, IReadOnlyList page) => + bound is null || page.Count == 0 || page[^1].SourceTimestampUtc is not { } last + ? bound + : bound with { LastReturnedUtc = last }; + + /// The Bad_BoundNotFound placeholder (null value) stamped at . + /// The bound's time. + /// The placeholder. + public static DataValueSnapshot NotFound(DateTime atUtc) => + new(null, StatusCodes.BadBoundNotFound, atUtc, atUtc); + + /// + /// The start bound to return ahead of the data, or null when a stored sample sits on StartTime (that sample is + /// the bound, and it is already the data's first sample). + /// + public static async Task StartBoundAsync( + IHistorianDataSource source, string tagname, DateTime startUtc, bool reverse, int maxTieCluster, DateTime nowUtc, + CancellationToken ct) + { + if (IsOpen(startUtc)) return NotFound(startUtc); + var found = reverse + ? await EarliestAtOrAfterAsync(source, tagname, startUtc, nowUtc, ct).ConfigureAwait(false) + : await LatestAtOrBeforeAsync(source, tagname, startUtc, maxTieCluster, nowUtc, ct).ConfigureAwait(false); + if (found is null) return NotFound(startUtc); + return SameInstant(Ticks(found), startUtc.Ticks, Resolution(source)) ? null : found; + } + + /// The end bound: the stored sample on or beyond the end time, else the placeholder. + /// The backend. + /// Historian tag. + /// The pending end bound. + /// The last value's time already returned (stamps an open bound's placeholder). + /// The walk-back's row budget (forward-only backends). + /// Now (the walk-back's look-ahead clamp). + /// Cancellation. + public static async Task EndBoundAsync( + IHistorianDataSource source, string tagname, HistoryEndBound bound, DateTime previousReturnedUtc, + int maxTieCluster, DateTime nowUtc, CancellationToken ct) + { + if (bound.Open) + { + var ticks = bound.Reverse + ? Math.Max(0L, previousReturnedUtc.Ticks - OpenBoundStep.Ticks) + : Math.Min(DateTime.MaxValue.Ticks, previousReturnedUtc.Ticks + OpenBoundStep.Ticks); + return NotFound(new DateTime(ticks, DateTimeKind.Utc)); + } + + DataValueSnapshot? found; + if (bound.Reverse) + found = await LatestAtOrBeforeAsync(source, tagname, bound.AtUtc, maxTieCluster, nowUtc, ct).ConfigureAwait(false); + else if (bound.Inclusive) + found = await EarliestAtOrAfterAsync(source, tagname, bound.AtUtc, nowUtc, ct).ConfigureAwait(false); + else + { + var resolution = Resolution(source); + found = bound.AtUtc.Ticks > DateTime.MaxValue.Ticks - resolution + ? null + : await EarliestAtOrAfterAsync(source, tagname, bound.AtUtc.AddTicks(resolution), nowUtc, ct).ConfigureAwait(false); + } + + return found ?? NotFound(bound.AtUtc); + } + + /// + /// Whether a data sample lies inside the data window of a bounded read — a forward window excludes the end time (a + /// sample there is the end bound), compared at the storage resolution. A backend whose forward end is inclusive, or a + /// resumed read that steps onto the end time, would otherwise return the end bound twice. + /// + public static bool InDataWindow(DataValueSnapshot sample, HistoryEndBound bound, long resolution) + { + if (!bound.Inclusive || bound.Open) return true; + var ts = Align(Ticks(sample), resolution); + var end = Align(bound.AtUtc.Ticks, resolution); + return bound.Reverse ? ts > end : ts < end; + } + + /// Whether a resumed read's next start is already past the data window (only the end bound remains). + public static bool DataWindowDrained(DateTime nextStartUtc, HistoryEndBound bound, long resolution) + { + if (!bound.Inclusive || bound.Open) return false; + var next = Align(nextStartUtc.Ticks, resolution); + var end = Align(bound.AtUtc.Ticks, resolution); + return bound.Reverse ? next <= end : next >= end; + } + + /// The latest stored sample at or before , or null. + internal static async Task LatestAtOrBeforeAsync( + IHistorianDataSource source, string tagname, DateTime atUtc, int maxTieCluster, DateTime nowUtc, CancellationToken ct) + { + if (atUtc.Ticks == 0) return null; + var floor = new DateTime(0, DateTimeKind.Utc); + IReadOnlyList rows = source.SupportsReverseRaw + ? (await source.ReadRawAsync(tagname, atUtc, floor, 1, ct).ConfigureAwait(false)).Samples + : (await ReverseRawReader.ReadAsync(source, tagname, atUtc, floor, 1, maxTieCluster, nowUtc, 0, ct) + .ConfigureAwait(false)).Samples; + var limit = Align(atUtc.Ticks, Resolution(source)); + foreach (var row in rows) + if (Align(Ticks(row), Resolution(source)) <= limit) + return row; + return null; + } + + /// The earliest stored sample at or after , or null. + /// + /// #691 review: on a backend that reads forward only (the Gateway) the read's end is clamped to now + + /// — the same horizon the walk-back uses — and a time beyond it is "not found" + /// without a read. The HistorianGateway only checks start <= end, so a DateTime.MaxValue end would + /// reach AVEVA as a year-9999 query end (unverified: a rejection, or a fabricated row). The clamp is NOT applied to a + /// backend that reads newest-first natively (SqlServer / QuestDB): they bind MaxValue safely (pinned by the + /// Extreme_bounds conformance test) and can legitimately hold samples further ahead (HistoryUpdate's + /// HistoryUpdateMaxFutureSkew is configurable, and driver-stamped live values carry device time) — clamping + /// there would turn a real end bound into Bad_BoundNotFound. + /// + internal static async Task EarliestAtOrAfterAsync( + IHistorianDataSource source, string tagname, DateTime atUtc, DateTime nowUtc, CancellationToken ct) + { + var endTicks = DateTime.MaxValue.Ticks; + if (!source.SupportsReverseRaw) + endTicks = nowUtc.Ticks > DateTime.MaxValue.Ticks - ReverseRawReader.LookAhead.Ticks + ? DateTime.MaxValue.Ticks + : nowUtc.Ticks + ReverseRawReader.LookAhead.Ticks; + if (atUtc.Ticks >= endTicks) return null; + var rows = (await source.ReadRawAsync(tagname, atUtc, new DateTime(endTicks, DateTimeKind.Utc), 1, ct) + .ConfigureAwait(false)).Samples; + var limit = Align(atUtc.Ticks, Resolution(source)); + foreach (var row in rows) + if (Align(Ticks(row), Resolution(source)) >= limit) + return row; + return null; + } + + private static bool IsOpen(DateTime t) => t.Ticks == 0 || t.Ticks == DateTime.MaxValue.Ticks; + + private static bool SameInstant(long a, long b, long resolution) => Align(a, resolution) == Align(b, resolution); + + private static long Resolution(IHistorianDataSource source) => Math.Max(1L, source.TimestampResolutionTicks); + + private static long Ticks(DataValueSnapshot s) => (s.SourceTimestampUtc ?? DateTime.MinValue).Ticks; + + private static long Align(long ticks, long resolution) => ticks - ticks % resolution; +} diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/HistoryPaging.cs b/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/HistoryPaging.cs index 595affbb..b618dbf9 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/HistoryPaging.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/HistoryPaging.cs @@ -62,6 +62,10 @@ namespace ZB.MOM.WW.OtOpcUa.OpcUaServer; /// serialized), so a new field with a "none" default is all the versioning it needs. /// /// The exclusive upper end of that stretch. +/// +/// with ReturnBounds = true only (Gitea #691): the end bound still owed — +/// Part 11 returns it with the last page. null = bounds not requested. +/// internal sealed record HistoryContinuationState( string Tagname, DateTime NextStartUtc, @@ -74,7 +78,8 @@ internal sealed record HistoryContinuationState( bool Reverse = false, long SegmentWidthTicks = 0, long EmptyFromTicks = 0, - long EmptyToTicks = 0) + long EmptyToTicks = 0, + HistoryEndBound? PendingEndBound = null) { /// The walk-back hint this cursor carries (#686/#689/#691). public ReverseWalkHint WalkHint => new(SegmentWidthTicks, EmptyFromTicks, EmptyToTicks); diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs b/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs index a939ed3c..eb0c81d8 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs @@ -544,6 +544,13 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 /// public int MaxTieClusterOverfetch { get; set; } = 65536; + /// + /// Gitea #691: whether HistoryRead-Raw honours ReturnBounds (Part 11 bounding values, ) + /// — ServerHistorian:HonorReturnBounds, default true. false ignores the flag, as before #691, + /// including the modified-read Bad_InvalidArgument. + /// + public bool HonorReturnBounds { get; set; } = true; + /// /// Maximum historized nodes served concurrently within one HistoryRead batch (arch-review 03/S3). /// Mirrors ServerHistorianOptions.HistoryReadBatchParallelism; the Host sets it at @@ -5056,6 +5063,15 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 var work = new List>(nodesToProcess.Count); foreach (var handle in nodesToProcess) { + if (details.IsReadModified && details.ReturnBounds && HonorReturnBounds) + { + // Part 11 §6.5.3.3: returnBounds "shall be set to FALSE" for a modified read, "otherwise the Server returns + // a Bad_InvalidArgument" (#691). Off with the ServerHistorian:HonorReturnBounds kill switch. + errors[handle.Index] = StatusCodes.BadInvalidArgument; + results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadInvalidArgument }; + continue; + } + if (details.IsReadModified) { // Part 11 modified-value history (#668): the prior values every HistoryUpdate recorded. Gated by the @@ -5070,7 +5086,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 work.Add(() => ServeRawPagedAsync( handle, session, nodesToRead, results, errors, details.StartTime, details.EndTime, details.NumValuesPerNode, - ct)); + ct, details.ReturnBounds && HonorReturnBounds)); } RunBounded(work, HistoryReadBatchParallelism); @@ -6197,6 +6213,11 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 /// The request window's (inclusive) lower bound, used for a fresh read. /// The (inclusive) upper bound of the read window; unchanged across pages. /// The client's per-page cap; 0 means "all values, no paging". + /// Cancellation (the per-request deadline). + /// + /// Part 11 bounding values (#691, ): the start bound leads the first page, the end bound + /// ends the last, and both count toward . A resumed read takes it from the cursor. + /// private async Task ServeRawPagedAsync( NodeHandle handle, ISession? session, @@ -6206,7 +6227,8 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 DateTime startTimeUtc, DateTime endUtc, uint numValuesPerNode, - CancellationToken ct) + CancellationToken ct, + bool returnBounds = false) { var inboundCp = nodesToRead[handle.Index].ContinuationPoint; @@ -6215,6 +6237,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 DateTime startUtc; var boundarySkip = 0; var walkHint = default(ReverseWalkHint); // #686/#689/#691: the walk-back's width + proven-empty stretch, page to page + HistoryEndBound? endBound = null; // #691: the end bound still owed (null = bounds not requested) string tagname; if (inboundCp is { Length: > 0 }) @@ -6236,6 +6259,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 endUtc = state.EndUtc; numValuesPerNode = state.NumValuesPerNode; walkHint = state.WalkHint; + endBound = state.PendingEndBound; } else { @@ -6261,6 +6285,107 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 else if (startUtc == DateTime.MinValue && endUtc != DateTime.MinValue) (startUtc, endUtc) = (endUtc, new DateTime(0, DateTimeKind.Utc)); } + + if (returnBounds) endBound = HistoryBounds.For(startUtc, endUtc); + } + + var nowUtc = Clock.GetUtcNow().UtcDateTime; + var resolution = Math.Max(1L, HistorianDataSource.TimestampResolutionTicks); + + // #691: the data is drained — the end bound ends this page when it has room (Part 11: bounds count toward + // NumValuesPerNode), else it is the whole next page. Returns the continuation point to emit, if any. + async Task AppendEndBoundAsync(List page, DateTime resumedAtUtc) + { + // The previous returned value stamps an open bound's placeholder (Part 11 Table 1 footnotes a/b): this + // page's last, else the last of an earlier page (carried in the cursor), else where the read resumed. + var previous = (page.Count > 0 ? page[^1].SourceTimestampUtc : null) + ?? endBound!.LastReturnedUtc ?? resumedAtUtc; + if (numValuesPerNode == 0 || page.Count < numValuesPerNode) + { + page.Add(await HistoryBounds.EndBoundAsync( + HistorianDataSource, tagname, endBound!, previous, MaxTieClusterOverfetch, nowUtc, ct).ConfigureAwait(false)); + return null; + } + + return _historyContinuationStore.Save(session, new HistoryContinuationState( + tagname, previous, endUtc, 0, numValuesPerNode, SegmentWidthTicks: walkHint.WidthTicks, + EmptyFromTicks: walkHint.EmptyFromTicks, EmptyToTicks: walkHint.EmptyToTicks, + PendingEndBound: HistoryBounds.Carry(endBound!, page)! with { DataExhausted = true })); + } + + // #691 review: whether a bounded read whose start bound was NOT found has any data or a found end bound — only + // asked when a one-value page would otherwise be the start-bound placeholder alone. + async Task BoundedReadHasAnythingAsync() + { + var src = HistorianDataSource; + IReadOnlyList first = startUtc > endUtc && !src.SupportsReverseRaw + ? (await ReverseRawReader.ReadAsync(src, tagname, startUtc, endUtc, 1, MaxTieClusterOverfetch, nowUtc, 0, ct) + .ConfigureAwait(false)).Samples + : (await src.ReadRawAsync(tagname, startUtc, endUtc, 1, ct).ConfigureAwait(false)).Samples; + if (first.Any(x => HistoryBounds.InDataWindow(x, endBound!, resolution))) return true; + var end = await HistoryBounds.EndBoundAsync( + src, tagname, endBound!, startUtc, MaxTieClusterOverfetch, nowUtc, ct).ConfigureAwait(false); + return !HistoryBounds.IsPlaceholder(end); + } + + // #691: a resumed bounded read whose data is drained (the end bound did not fit on the previous page, or the + // resume point already sits on the end time) owes only the end bound. + if (endBound is not null && inboundCp is { Length: > 0 } + && (endBound.DataExhausted || HistoryBounds.DataWindowDrained(startUtc, endBound, resolution))) + { + var onlyBound = new List(1); + var afterCp = await AppendEndBoundAsync(onlyBound, startUtc).ConfigureAwait(false); + results[handle.Index] = new SdkHistoryReadResult + { + StatusCode = StatusCodes.Good, + HistoryData = new ExtensionObject(ToHistoryDataFromSamples(onlyBound)), + ContinuationPoint = afterCp, + }; + errors[handle.Index] = ServiceResult.Good; + return; + } + + // #691: the start bound leads the first page and counts toward the cap; null when a sample sits on StartTime + // (that sample is the bound, and it is the data's first). + DataValueSnapshot? startBound = null; + var readCap = numValuesPerNode; + if (endBound is not null && inboundCp is not { Length: > 0 }) + { + startBound = await HistoryBounds.StartBoundAsync( + HistorianDataSource, tagname, startUtc, endBound.Reverse, MaxTieClusterOverfetch, nowUtc, ct) + .ConfigureAwait(false); + if (startBound is not null && numValuesPerNode == 1 + && (!HistoryBounds.IsPlaceholder(startBound) || await BoundedReadHasAnythingAsync().ConfigureAwait(false))) + { + // The page is the start bound alone; the data starts at StartTime on the next page. + var boundOnlyCp = _historyContinuationStore.Save(session, new HistoryContinuationState( + tagname, startUtc, endUtc, 0, numValuesPerNode, SegmentWidthTicks: walkHint.WidthTicks, + EmptyFromTicks: walkHint.EmptyFromTicks, EmptyToTicks: walkHint.EmptyToTicks, + PendingEndBound: HistoryBounds.Carry(endBound, [startBound]))); + results[handle.Index] = new SdkHistoryReadResult + { + StatusCode = StatusCodes.Good, + HistoryData = new ExtensionObject(ToHistoryDataFromSamples([startBound])), + ContinuationPoint = boundOnlyCp, + }; + errors[handle.Index] = ServiceResult.Good; + return; + } + + if (startBound is not null && numValuesPerNode == 1) + { + // Nothing at all — no data and no bound found (review of #691): the pre-bounds GoodNoData answer. + results[handle.Index] = new SdkHistoryReadResult + { + StatusCode = StatusCodes.GoodNoData, + HistoryData = new ExtensionObject(ToHistoryDataFromSamples([])), + ContinuationPoint = null, + }; + errors[handle.Index] = ServiceResult.Good; + return; + } + + if (startBound is not null && numValuesPerNode > 0) readCap = numValuesPerNode - 1; } // HistoryRead is NOT under the node-manager Lock — awaiting the async source is safe. @@ -6272,8 +6397,8 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 if (startUtc > endUtc && !source.SupportsReverseRaw) { var page = await ReverseRawReader.ReadAsync( - source, tagname, startUtc, endUtc, numValuesPerNode, MaxTieClusterOverfetch, - Clock.GetUtcNow().UtcDateTime, walkHint, ct) + source, tagname, startUtc, endUtc, readCap, MaxTieClusterOverfetch, + nowUtc, walkHint, ct) .ConfigureAwait(false); walkHint = page.Hint; sourceResult = new HistorianRead(page.Samples, ContinuationPoint: null); @@ -6281,17 +6406,29 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 else { sourceResult = await source - .ReadRawAsync(tagname, startUtc, endUtc, numValuesPerNode, ct) + .ReadRawAsync(tagname, startUtc, endUtc, readCap, ct) .ConfigureAwait(false); } - var backendFull = HistoryPaging.IsFullPage(sourceResult.Samples.Count, numValuesPerNode); + var backendFull = HistoryPaging.IsFullPage(sourceResult.Samples.Count, readCap); // On a resume read, drop the boundary ties already returned on the prior page. var samples = inboundCp is { Length: > 0 } ? HistoryPaging.TrimBoundaryDuplicates(sourceResult.Samples, startUtc, boundarySkip) : sourceResult.Samples; + // #691: a bounded read's data window excludes the end time (a sample there is the end bound). A backend whose + // forward end is inclusive (the gateway's own) returns it as data; dropping it also means the data is drained. + if (endBound is not null) + { + var inside = samples.Where(x => HistoryBounds.InDataWindow(x, endBound, resolution)).ToList(); + if (inside.Count != samples.Count) + { + samples = inside; + backendFull = false; + } + } + // Oversized tie cluster: a resume read whose FULL backend page does not advance past the resume // boundary timestamp `startUtc`. That happens when more samples share `startUtc` than the page // cap — a tie cluster larger than NumValuesPerNode. The fixed-(start,end,cap) backend can only @@ -6355,9 +6492,15 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 { var clusterState = new HistoryContinuationState( tagname, resumeAt, endUtc, nextSkip, numValuesPerNode, SegmentWidthTicks: walkHint.WidthTicks, - EmptyFromTicks: walkHint.EmptyFromTicks, EmptyToTicks: walkHint.EmptyToTicks); + EmptyFromTicks: walkHint.EmptyFromTicks, EmptyToTicks: walkHint.EmptyToTicks, + PendingEndBound: HistoryBounds.Carry(endBound, slice)); clusterCp = _historyContinuationStore.Save(session, clusterState); } + else if (endBound is not null) + { + // #691: the data is drained — the end bound ends this page, or is the next one when this one is full. + clusterCp = await AppendEndBoundAsync(slice, startUtc).ConfigureAwait(false); + } results[handle.Index] = new SdkHistoryReadResult { @@ -6369,6 +6512,14 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 return; } + // #691: the start bound (first page only) leads the data. + if (startBound is not null) + { + var led = new List(samples.Count + 2) { startBound }; + led.AddRange(samples); + samples = led; + } + // The "full page" test is against the RAW backend count (before trimming): the backend honoured // the cap, so a full backend page ⇒ there may be more even if we trimmed some boundary ties. var historyData = ToHistoryDataFromSamples(samples); @@ -6379,11 +6530,30 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 HistoryPaging.ComputeResumeCursor(samples, out var nextStart, out var skip); var nextState = new HistoryContinuationState( tagname, nextStart, endUtc, skip, numValuesPerNode, SegmentWidthTicks: walkHint.WidthTicks, - EmptyFromTicks: walkHint.EmptyFromTicks, EmptyToTicks: walkHint.EmptyToTicks); + EmptyFromTicks: walkHint.EmptyFromTicks, EmptyToTicks: walkHint.EmptyToTicks, + PendingEndBound: HistoryBounds.Carry(endBound, samples)); // Save may return null (no session on this request) ⇒ degrade to single-shot for this node. // Built AFTER historyData so a failure projecting samples can never orphan a stored cursor. outboundCp = _historyContinuationStore.Save(session, nextState); } + else if (endBound is not null) + { + // #691: the data is drained — the end bound ends this page, or is the next one when this one is full. + var page = new List(samples.Count + 1); + page.AddRange(samples); + outboundCp = await AppendEndBoundAsync(page, startUtc).ConfigureAwait(false); + samples = page; + historyData = ToHistoryDataFromSamples(samples); + } + + // #691 review: a whole bounded read with no data and no bound found (one page, every value a placeholder) is the + // pre-bounds answer — GoodNoData, no values (§6.5.3.2: bounds make it a success only "if one or both exist"). + if (endBound is not null && inboundCp is not { Length: > 0 } && outboundCp is null + && samples.All(HistoryBounds.IsPlaceholder)) + { + samples = []; + historyData = ToHistoryDataFromSamples(samples); + } results[handle.Index] = new SdkHistoryReadResult { diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ServerHistorianOptions.cs b/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ServerHistorianOptions.cs index 7c04110d..37c04091 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ServerHistorianOptions.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ServerHistorianOptions.cs @@ -89,6 +89,14 @@ public sealed class ServerHistorianOptions /// public int MaxTieClusterOverfetch { get; init; } = 65536; + /// + /// Gitea #691: whether HistoryRead-Raw honours ReadRawModifiedDetails.ReturnBounds (OPC UA Part 11 bounding + /// values). Default true. false is the kill switch back to the pre-#691 answer: the flag is ignored + /// (stored samples only) and a modified read carrying it is not refused Bad_InvalidArgument. Note the .NET + /// SDK's new ReadRawModifiedDetails() defaults ReturnBounds (and IsReadModified) to true. + /// + public bool HonorReturnBounds { get; init; } = true; + /// /// Maximum number of historized nodes served CONCURRENTLY within one HistoryRead batch (arch-review /// 03/S3). The SDK's HistoryRead* override surface is synchronous, so each arm block-bridges diff --git a/tests/Drivers/Historian.Conformance/BoundingValueConformance.cs b/tests/Drivers/Historian.Conformance/BoundingValueConformance.cs new file mode 100644 index 00000000..d8394d74 --- /dev/null +++ b/tests/Drivers/Historian.Conformance/BoundingValueConformance.cs @@ -0,0 +1,100 @@ +using Shouldly; +using Xunit; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Conformance; + +/// +/// Gitea #691: the server composes Part 11 bounding values (HistoryRead-Raw ReturnBounds) from exactly these raw +/// reads on a backend that reads newest-first natively (HistoryBounds in the OPC UA server) — so every backend must +/// answer them precisely: "at or before T" = a reverse read of (0, T] capped at one, "at or after T" = a forward read +/// of [T, MaxValue) capped at one, "strictly after T" = the same from T + resolution. A bound is a stored +/// sample of any quality, so a Bad row or an outage marker comes back as the bound. +/// +public abstract class BoundingValueConformance : ConformanceBase +{ + private static readonly CancellationToken Ct = CancellationToken.None; + private static readonly DateTime Floor = new(0, DateTimeKind.Utc); + private static readonly DateTime Ceiling = new(DateTime.MaxValue.Ticks, DateTimeKind.Utc); + + private async Task AtOrBeforeAsync(IHistorianDataSource src, string tag, DateTime t) => + (await src.ReadRawAsync(tag, t, Floor, 1, Ct)).Samples.SingleOrDefault(); + + private async Task AtOrAfterAsync(IHistorianDataSource src, string tag, DateTime t) => + (await src.ReadRawAsync(tag, t, Ceiling, 1, Ct)).Samples.SingleOrDefault(); + + [Fact] + public async Task The_sample_at_or_before_a_time_is_the_latest_one_on_or_before_it() + { + await ConformanceData.SeedAAsync(B); + var src = B.CreateDataSource(); + + (await AtOrBeforeAsync(src, "A", At(10)))!.Value.ShouldBe(20d); // exactly on it + (await AtOrBeforeAsync(src, "A", At(15)))!.Value.ShouldBe(20d); // between samples + (await AtOrBeforeAsync(src, "A", At(1000)))!.Value.ShouldBe(50d); // past the last + (await AtOrBeforeAsync(src, "A", At(-1))).ShouldBeNull(); // before the first: no bound + (await AtOrBeforeAsync(src, "A", At(15)))!.SourceTimestampUtc!.Value.Ticks.ShouldBe(B.Round(At(10)).Ticks); + } + + [Fact] + public async Task The_sample_at_or_after_a_time_is_the_earliest_one_on_or_after_it() + { + await ConformanceData.SeedAAsync(B); + var src = B.CreateDataSource(); + + (await AtOrAfterAsync(src, "A", At(10)))!.Value.ShouldBe(20d); // exactly on it + (await AtOrAfterAsync(src, "A", At(15)))!.Value.ShouldBe(30d); // between samples + (await AtOrAfterAsync(src, "A", At(-100)))!.Value.ShouldBe(10d); // before the first + (await AtOrAfterAsync(src, "A", At(41))).ShouldBeNull(); // past the last: no bound + } + + [Fact] + public async Task The_sample_strictly_after_a_time_starts_one_storage_resolution_later() + { + await ConformanceData.SeedAAsync(B); + var src = B.CreateDataSource(); + + (await AtOrAfterAsync(src, "A", At(10).AddTicks(B.ResolutionTicks)))!.Value.ShouldBe(30d); + (await AtOrAfterAsync(src, "A", At(40).AddTicks(B.ResolutionTicks))).ShouldBeNull(); + } + + [Fact] + public async Task A_time_between_storage_ticks_finds_the_sample_the_backend_stores_at_its_truncation() + { + // The server compares "on the bound" at the storage resolution, so the lookups must agree with the window reads: + // a time a fraction of a resolution step after a stored sample is that sample's instant on a coarse backend. + await ConformanceData.SeedAAsync(B); + var src = B.CreateDataSource(); + var t = At(10).AddTicks(B.ResolutionTicks - 1); // == At(10) on a 1-tick backend + + (await AtOrBeforeAsync(src, "A", t))!.Value.ShouldBe(20d); + var after = await AtOrAfterAsync(src, "A", t); + after!.SourceTimestampUtc!.Value.Ticks.ShouldBeGreaterThanOrEqualTo(B.Round(t).Ticks); + after.Value.ShouldBe(B.Round(t) == B.Round(At(10)) ? 20d : 30d); + } + + [Fact] + public async Task A_bad_sample_or_an_outage_marker_is_a_bound_as_stored() + { + await B.SeedAsync("M", + (At(0), HistorianValue.FromDouble(10), Good), + (At(5), HistorianValue.Empty, BadNoCommunication), + (At(7), HistorianValue.FromDouble(99), Bad)); + var src = B.CreateDataSource(); + + var before = await AtOrBeforeAsync(src, "M", At(6)); + before!.Value.ShouldBeNull(); + before.StatusCode.ShouldBe(0x80310000u); + var after = await AtOrAfterAsync(src, "M", At(6)); + after!.Value.ShouldBe(99d); + after.StatusCode.ShouldBe(0x80000000u); + } + + [Fact] + public async Task An_unknown_tag_has_no_bounds() + { + var src = B.CreateDataSource(); + (await AtOrBeforeAsync(src, "nope", At(10))).ShouldBeNull(); + (await AtOrAfterAsync(src, "nope", At(10))).ShouldBeNull(); + } +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests/GatewayBoundingValueTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests/GatewayBoundingValueTests.cs new file mode 100644 index 00000000..0b855923 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests/GatewayBoundingValueTests.cs @@ -0,0 +1,174 @@ +using Google.Protobuf.WellKnownTypes; +using Microsoft.Extensions.Logging.Abstractions; +using Shouldly; +using Xunit; +using ZB.MOM.WW.HistorianGateway.Contracts.Grpc; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; +using ZB.MOM.WW.OtOpcUa.OpcUaServer; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests; + +/// +/// Gitea #691: Part 11 bounding values on the Gateway, composed by the server's over the REAL +/// source and an AVEVA-shaped fake (the live gate's wire shape). The start bound of a forward read must be the stored +/// sample before StartTime, at ITS OWN time — never AVEVA's synthetic boundary row, which is stamped at the query start +/// and carries the value in effect there (Quality = 133). The Gateway reads forward only, so "at or before" is +/// served by the walk-back (), and no reverse window is ever sent. +/// +public sealed class GatewayBoundingValueTests +{ + private static CancellationToken Ct => TestContext.Current.CancellationToken; + + private static readonly DateTime T0 = new(2026, 10, 12, 18, 6, 0, DateTimeKind.Utc); + private static readonly DateTime Now = T0.AddHours(1); + + /// The live gate's tag HistGW.LiveTest.Gate681 (18:06:40 is a stored NULL). + private static readonly (double Sec, double Value, bool Null)[] Gate681 = + [ + (0, 10, false), (5, 20, false), (20, 30, false), (40, 0, true), (50, 5, false), (60, 1, false), (90, 1, false), + (100, 7, false), + ]; + + private static DateTime At(double seconds) => T0.AddTicks((long)(seconds * TimeSpan.TicksPerSecond)); + + private static HistorianSample Stored(DateTime t, double v, bool isNull) => new() + { + Tag = "T", + Timestamp = Timestamp.FromDateTime(t), + NumericValue = isNull ? 0 : v, + OpcQuality = isNull ? 0u : 192u, + Quality = isNull ? 1u : 0u, + }; + + /// The latest query end any ReadRaw was sent (a far-future end is the #691 review's risk). + private sealed class EndProbe + { + public DateTime MaxEnd = DateTime.MinValue; + } + + /// + /// ReadRaw over [start, end] ascending, capped, led by AVEVA's boundary row at the query start. A query end more + /// than two days past is REFUSED (InvalidArgument), modelling the unverified AVEVA behaviour + /// for a year-9999 end (the HistorianGateway only checks start <= end, so it reaches AVEVA unclamped). + /// + private static FakeHistorianGatewayClient Aveva(EndProbe? probe = null) + { + var sorted = Gate681.Select(x => (T: At(x.Sec), V: x.Value, x.Null)).ToList(); + return new FakeHistorianGatewayClient + { + RawSource = (start, end, max) => + { + if (probe is not null && end > probe.MaxEnd) probe.MaxEnd = end; + if (end > Now.AddDays(2)) + throw new Grpc.Core.RpcException(new Grpc.Core.Status(Grpc.Core.StatusCode.InvalidArgument, + "end_time is outside the historian's time range")); + var rows = new List(); + if (sorted.All(d => d.T != start)) + { + var prior = sorted.Where(d => d.T < start).Select(d => ((DateTime T, double V, bool Null)?)d).LastOrDefault(); + HistorianSample boundary; + if (prior is { } p) + { + boundary = Stored(start, p.V, p.Null); + if (!p.Null) boundary.Quality = 133; + } + else + { + boundary = new HistorianSample { Tag = "T", Timestamp = Timestamp.FromDateTime(start), NumericValue = 0, OpcQuality = 0, Quality = 1 }; + } + + rows.Add(boundary); + } + + rows.AddRange(sorted.Where(d => d.T >= start && d.T <= end).Select(d => Stored(d.T, d.V, d.Null))); + return max > 0 ? rows.Take(max) : rows; + }, + }; + } + + private static GatewayHistorianDataSource Source(FakeHistorianGatewayClient fake) => + new(fake, NullLogger.Instance); + + [Fact] + public async Task The_start_bound_is_the_stored_prior_sample_at_its_own_time_not_the_synthetic_row() + { + var bound = await HistoryBounds.StartBoundAsync(Source(Aveva()), "T", At(21), reverse: false, 65536, Now, Ct); + + bound.ShouldNotBeNull(); + bound.SourceTimestampUtc.ShouldBe(At(20)); // the synthetic row would be stamped At(21) + bound.Value.ShouldBe(30d); + bound.StatusCode.ShouldBe(0u); + } + + [Fact] + public async Task A_stored_NULL_prior_sample_is_the_start_bound_as_the_Bad_sample_it_is() + { + var bound = await HistoryBounds.StartBoundAsync(Source(Aveva()), "T", At(45), reverse: false, 65536, Now, Ct); + + bound!.SourceTimestampUtc.ShouldBe(At(40)); + bound.StatusCode.ShouldBe(0x80000000u); + } + + [Fact] + public async Task Before_the_first_sample_the_start_bound_is_not_found_at_the_start_time() + { + var bound = await HistoryBounds.StartBoundAsync(Source(Aveva()), "T", At(-30), reverse: false, 65536, Now, Ct); + + bound!.StatusCode.ShouldBe(Opc.Ua.StatusCodes.BadBoundNotFound); + bound.SourceTimestampUtc.ShouldBe(At(-30)); + bound.Value.ShouldBeNull(); + } + + [Fact] + public async Task A_sample_exactly_on_the_start_time_is_the_bound_and_already_the_data() + { + (await HistoryBounds.StartBoundAsync(Source(Aveva()), "T", At(20), reverse: false, 65536, Now, Ct)).ShouldBeNull(); + } + + [Fact] + public async Task The_end_bound_is_the_first_stored_sample_at_or_after_the_end_time() + { + var source = Source(Aveva()); + var after = await HistoryBounds.EndBoundAsync(source, "T", HistoryBounds.For(At(0), At(41)), At(40), 65536, Now, Ct); + after.SourceTimestampUtc.ShouldBe(At(50)); + after.Value.ShouldBe(5d); + + var missing = await HistoryBounds.EndBoundAsync(source, "T", HistoryBounds.For(At(0), At(101)), At(100), 65536, Now, Ct); + missing.StatusCode.ShouldBe(Opc.Ua.StatusCodes.BadBoundNotFound); + missing.SourceTimestampUtc.ShouldBe(At(101)); + } + + /// + /// #691 review: the at-or-after read never sends an end beyond now + (the + /// historian stores nothing further ahead), and an end bound wanted beyond that horizon is not found without a read. + /// + [Fact] + public async Task The_at_or_after_read_never_asks_past_now_plus_one_day() + { + var probe = new EndProbe(); + var fake = Aveva(probe); + var source = Source(fake); + + var found = await HistoryBounds.EndBoundAsync(source, "T", HistoryBounds.For(At(0), At(41)), At(40), 65536, Now, Ct); + found.SourceTimestampUtc.ShouldBe(At(50)); + probe.MaxEnd.ShouldBeLessThanOrEqualTo(Now + ReverseRawReader.LookAhead.Add(TimeSpan.FromMilliseconds(1))); + + var calls = fake.ReadRawCallCount; + var beyond = Now.AddDays(3); + var missing = await HistoryBounds.EndBoundAsync(source, "T", HistoryBounds.For(At(0), beyond), At(100), 65536, Now, Ct); + missing.StatusCode.ShouldBe(Opc.Ua.StatusCodes.BadBoundNotFound); + missing.SourceTimestampUtc.ShouldBe(beyond); + fake.ReadRawCallCount.ShouldBe(calls, "a bound beyond the horizon is not found without a read"); + source.GetHealthSnapshot().TotalFailures.ShouldBe(0); + } + + [Fact] + public async Task A_reverse_read_end_bound_walks_back_to_the_stored_sample_at_or_before_the_end_time() + { + var fake = Aveva(); + var bound = await HistoryBounds.EndBoundAsync(Source(fake), "T", HistoryBounds.For(At(100), At(55)), At(60), 65536, Now, Ct); + + bound.SourceTimestampUtc.ShouldBe(At(50)); + bound.Value.ShouldBe(5d); + } +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformance.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformance.cs index a9987d1a..d84c755e 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformance.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformance.cs @@ -67,3 +67,10 @@ public sealed class QuestDbEventReadConformance(QuestDbFixture fx) : EventReadCo /// protected override Task CreateBackendAsync() => QuestDbConformanceBackend.CreateAsync(fx); } + +[Collection(QuestDbCollection.Name)] +public sealed class QuestDbBoundingValueConformance(QuestDbFixture fx) : BoundingValueConformance +{ + /// + protected override Task CreateBackendAsync() => QuestDbConformanceBackend.CreateAsync(fx); +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/SqlServerConformance.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/SqlServerConformance.cs index 73b9862f..c08d50ee 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/SqlServerConformance.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/SqlServerConformance.cs @@ -67,3 +67,10 @@ public sealed class SqlServerEventReadConformance(SqlHistorianFixture fx) : Even /// protected override Task CreateBackendAsync() => SqlServerConformanceBackend.CreateAsync(fx); } + +[Collection(SqlHistorianCollection.Name)] +public sealed class SqlServerBoundingValueConformance(SqlHistorianFixture fx) : BoundingValueConformance +{ + /// + protected override Task CreateBackendAsync() => SqlServerConformanceBackend.CreateAsync(fx); +} diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/OpcUa/OtOpcUaServerHostedServiceHistoryUpdateTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/OpcUa/OtOpcUaServerHostedServiceHistoryUpdateTests.cs index 9b3e9dc5..37461249 100644 --- a/tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/OpcUa/OtOpcUaServerHostedServiceHistoryUpdateTests.cs +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/OpcUa/OtOpcUaServerHostedServiceHistoryUpdateTests.cs @@ -72,6 +72,18 @@ public sealed class OtOpcUaServerHostedServiceHistoryUpdateTests Service(view: null).BuildHistoryUpdatePosture().MaxFutureSkew.ShouldBe(TimeSpan.FromDays(1)); } + /// #691: the ReturnBounds kill switch is bound from ServerHistorian:HonorReturnBounds, default on. + [Fact] + public void The_return_bounds_kill_switch_comes_from_options_and_defaults_on() + { + Service(view: null).HonorReturnBounds.ShouldBeTrue(); + var config = new ConfigurationBuilder().AddInMemoryCollection(new Dictionary + { + ["ServerHistorian:HonorReturnBounds"] = "false", + }).Build(); + Service(view: null, config: config).HonorReturnBounds.ShouldBeFalse(); + } + private static OtOpcUaServerHostedService Service( IHistoryUpdateRoleView? view, IDriverPrimaryRoleSource? primary = null, IConfiguration? config = null, TimeProvider? clock = null) => new( diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/HistoryUpdateOverTheWireTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/HistoryUpdateOverTheWireTests.cs index 16f9ccc4..43fa05e2 100644 --- a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/HistoryUpdateOverTheWireTests.cs +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/HistoryUpdateOverTheWireTests.cs @@ -97,6 +97,7 @@ public sealed class HistoryUpdateOverTheWireTests var details = new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = T1.AddHours(-1), EndTime = T1.AddHours(1), NumValuesPerNode = 100, IsReadModified = true, }; var response = await session.HistoryReadAsync( diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerAclEnforcementTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerAclEnforcementTests.cs index 9dedcab8..a81e663d 100644 --- a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerAclEnforcementTests.cs +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerAclEnforcementTests.cs @@ -384,6 +384,7 @@ public sealed class NodeManagerAclEnforcementTests : IDisposable context, new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = DateTime.UtcNow.AddHours(-1), EndTime = DateTime.UtcNow, NumValuesPerNode = 10, diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadBoundsTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadBoundsTests.cs new file mode 100644 index 00000000..8b618b9e --- /dev/null +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadBoundsTests.cs @@ -0,0 +1,523 @@ +using Microsoft.Extensions.Logging.Abstractions; +using Opc.Ua; +using Opc.Ua.Server; +using Shouldly; +using Xunit; +using ZB.MOM.WW.OtOpcUa.Commons.OpcUa; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; +using HistorianRead = ZB.MOM.WW.OtOpcUa.Core.Abstractions.HistoryReadResult; +using SdkHistoryReadResult = Opc.Ua.HistoryReadResult; + +namespace ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests; + +/// Boots ONE real SDK server for the whole bounds suite (the cases swap the historian source per read). +public sealed class BoundsServerFixture : IAsyncLifetime +{ + private readonly string _pkiRoot = Path.Combine(Path.GetTempPath(), $"otopcua-bounds-{Guid.NewGuid():N}"); + private OpcUaApplicationHost? _host; + + /// The node manager under test. + public OtOpcUaNodeManager Nm { get; private set; } = null!; + + /// The historized variable every case reads. + public NodeId NodeId { get; private set; } = null!; + + /// Serialises the cases (they share the node manager's source). + public SemaphoreSlim Gate { get; } = new(1, 1); + + public async ValueTask InitializeAsync() + { + var started = await OpcUaHostStart.StartAsync( + port => new OpcUaApplicationHost( + new OpcUaApplicationHostOptions + { + ApplicationName = "OtOpcUa.BoundsTest", + ApplicationUri = $"urn:OtOpcUa.BoundsTest:{Guid.NewGuid():N}", + OpcUaPort = port, + PublicHostname = "localhost", + PkiStoreRoot = _pkiRoot, + }, + NullLogger.Instance), + () => new OtOpcUaSdkServer(), CancellationToken.None); + _host = started.Host; + Nm = started.Server.NodeManager!; + Nm.HistoryContinuationStore = new InMemoryHistoryContinuationStore(); + Nm.EnsureVariable("eq-1/bounds", parentFolderNodeId: null, displayName: "Bounds", dataType: "Double", + writable: false, historianTagname: "WW.Bounds", realm: AddressSpaceRealm.Uns); + NodeId = Nm.TryGetVariable("eq-1/bounds")!.NodeId; + } + + public async ValueTask DisposeAsync() + { + if (_host is not null) await _host.DisposeAsync(); + if (Directory.Exists(_pkiRoot)) + { + try { Directory.Delete(_pkiRoot, recursive: true); } + catch { /* best-effort */ } + } + } +} + +/// +/// Gitea #691: HistoryRead-Raw ReturnBounds (OPC UA Part 11 §4.6 bounding values, §6.5.3.2). The oracle is Part 11 +/// Table 1 itself — every row, over its data set (samples at 5:00, 5:02, 5:03, 5:05, 5:06) — run against three backend +/// shapes: one that reads newest-first natively (SqlServer-like), one that reads forward only (Gateway-like, walked back +/// by ) and a microsecond one (QuestDB-like, +/// truncating its window bounds). +/// +public sealed class NodeManagerHistoryReadBoundsTests(BoundsServerFixture fx) : IClassFixture +{ + private static readonly DateTime Day = new(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc); + private static readonly DateTime Unspecified = DateTime.MinValue; + + private static DateTime T(string hhmm) + { + var parts = hhmm.Split(':'); + return Day.AddHours(int.Parse(parts[0])).AddMinutes(int.Parse(parts[1])); + } + + /// Part 11 Table 1's data: value = minutes past 5:00. + private static DataValueSnapshot[] TableData() => + new[] { "5:00", "5:02", "5:03", "5:05", "5:06" } + .Select(s => new DataValueSnapshot((double)(T(s) - T("5:00")).TotalMinutes, StatusCodes.Good, T(s), T(s))) + .ToArray(); + + public enum Shape { Reverse, ForwardOnly, Microsecond } + + private static IHistorianDataSource Source(Shape shape, IReadOnlyList series) => shape switch + { + Shape.Reverse => new SeriesSource(series, reverse: true, inclusiveForwardEnd: false, resolution: 1), + Shape.ForwardOnly => new SeriesSource(series, reverse: false, inclusiveForwardEnd: false, resolution: 1), + _ => new SeriesSource(series, reverse: true, inclusiveForwardEnd: false, resolution: 10), + }; + + /// + /// Part 11 Table 1, every row, verbatim. "FIRST"/"LAST" are Bad_BoundNotFound placeholders (null value) at the + /// StartTime/EndTime; a/b footnotes: one second after/before the previous returned value when that side is unspecified. + /// + public static TheoryData Table1 => new() + { + { "5:00", "5:05", 0, true, "5:00, 5:02, 5:03, 5:05" }, + { "5:00", "5:05", 0, false, "5:00, 5:02, 5:03" }, + { "5:01", "5:04", 0, true, "5:00, 5:02, 5:03, 5:05" }, + { "5:01", "5:04", 0, false, "5:02, 5:03" }, + { "5:05", "5:00", 0, true, "5:05, 5:03, 5:02, 5:00" }, + { "5:05", "5:00", 0, false, "5:05, 5:03, 5:02" }, + { "5:04", "5:01", 0, true, "5:05, 5:03, 5:02, 5:00" }, + { "5:04", "5:01", 0, false, "5:03, 5:02" }, + { "4:59", "5:05", 0, true, "FIRST, 5:00, 5:02, 5:03, 5:05" }, + { "4:59", "5:05", 0, false, "5:00, 5:02, 5:03" }, + { "5:01", "5:07", 0, true, "5:00, 5:02, 5:03, 5:05, 5:06, LAST" }, + { "5:01", "5:07", 0, false, "5:02, 5:03, 5:05, 5:06" }, + { "5:00", "5:05", 3, true, "5:00, 5:02, 5:03" }, + { "5:00", "5:05", 3, false, "5:00, 5:02, 5:03" }, + { "5:01", "5:04", 3, true, "5:00, 5:02, 5:03" }, + { "5:01", "5:04", 3, false, "5:02, 5:03" }, + { "5:05", "5:00", 3, true, "5:05, 5:03, 5:02" }, + { "5:05", "5:00", 3, false, "5:05, 5:03, 5:02" }, + { "5:04", "5:01", 3, true, "5:05, 5:03, 5:02" }, + { "5:04", "5:01", 3, false, "5:03, 5:02" }, + { "4:59", "5:05", 3, true, "FIRST, 5:00, 5:02" }, + { "4:59", "5:05", 3, false, "5:00, 5:02, 5:03" }, + { "5:01", "5:07", 3, true, "5:00, 5:02, 5:03" }, + { "5:01", "5:07", 3, false, "5:02, 5:03, 5:05" }, + { "5:00", "", 3, true, "5:00, 5:02, 5:03" }, + { "5:00", "", 3, false, "5:00, 5:02, 5:03" }, + { "5:00", "", 6, true, "5:00, 5:02, 5:03, 5:05, 5:06, LAST+" }, + { "5:00", "", 6, false, "5:00, 5:02, 5:03, 5:05, 5:06" }, + { "5:07", "", 6, true, "5:06, LAST+" }, + { "5:07", "", 6, false, "" }, + { "", "5:06", 3, true, "5:06, 5:05, 5:03" }, + { "", "5:06", 3, false, "5:06, 5:05, 5:03" }, + { "", "5:06", 6, true, "5:06, 5:05, 5:03, 5:02, 5:00, FIRST-" }, + { "", "5:06", 6, false, "5:06, 5:05, 5:03, 5:02, 5:00" }, + { "", "4:48", 6, true, "5:00, FIRST-" }, + { "", "4:48", 6, false, "" }, + { "4:48", "4:48", 0, true, "FIRST, 5:00" }, + { "4:48", "4:48", 0, false, "" }, + { "4:48", "4:48", 1, true, "FIRST" }, + { "4:48", "4:48", 1, false, "" }, + { "4:48", "4:48", 2, true, "FIRST, 5:00" }, + { "5:00", "5:00", 0, true, "5:00, 5:02" }, + { "5:00", "5:00", 0, false, "5:00" }, + { "5:00", "5:00", 1, true, "5:00" }, + { "5:00", "5:00", 1, false, "5:00" }, + { "5:01", "5:01", 0, true, "5:00, 5:02" }, + { "5:01", "5:01", 0, false, "" }, + { "5:01", "5:01", 1, true, "5:00" }, + { "5:01", "5:01", 1, false, "" }, + }; + + [Theory] + [MemberData(nameof(Table1))] + public async Task Part11_table1_first_page_on_a_reverse_capable_backend(string start, string end, uint max, bool bounds, string expected) => + await AssertFirstPage(Shape.Reverse, start, end, max, bounds, expected); + + [Theory] + [MemberData(nameof(Table1))] + public async Task Part11_table1_first_page_on_a_forward_only_backend(string start, string end, uint max, bool bounds, string expected) => + await AssertFirstPage(Shape.ForwardOnly, start, end, max, bounds, expected); + + [Theory] + [MemberData(nameof(Table1))] + public async Task Part11_table1_first_page_on_a_microsecond_backend(string start, string end, uint max, bool bounds, string expected) => + await AssertFirstPage(Shape.Microsecond, start, end, max, bounds, expected); + + private async Task AssertFirstPage(Shape shape, string start, string end, uint max, bool bounds, string expected) + { + var s = start.Length == 0 ? Unspecified : T(start); + var e = end.Length == 0 ? Unspecified : T(end); + var source = Source(shape, TableData()); + var (values, error, _) = await ReadAsync(source, s, e, max, bounds, cp: null); + + error.StatusCode.Code.ShouldBe(StatusCodes.Good); + Render(values, s, e).ShouldBe(expected); + (source as SeriesSource)?.ReverseWindowsOnForwardOnly.ShouldBe(0); + } + + /// + /// Part 11: bounds count toward NumValuesPerNode, the start bound comes with the first page and the end bound with the + /// last — so paging a bounded two-sided read at any page size concatenates to the unpaged (N = 0) answer. + /// + [Theory] + [InlineData(Shape.Reverse)] + [InlineData(Shape.ForwardOnly)] + [InlineData(Shape.Microsecond)] + public async Task Paging_a_bounded_read_concatenates_to_the_unpaged_answer(Shape shape) + { + var windows = new[] + { + ("5:00", "5:05"), ("5:01", "5:04"), ("5:05", "5:00"), ("5:04", "5:01"), ("4:59", "5:05"), ("5:01", "5:07"), + ("4:48", "4:48"), ("5:00", "5:00"), ("5:01", "5:01"), ("5:07", "4:00"), ("5:06", "5:06"), + }; + foreach (var (start, end) in windows) + { + var source = Source(shape, TableData()); + var all = await ReadAsync(source, T(start), T(end), 0, true, cp: null); + all.Cp.ShouldBeNull(); + var expected = Render(all.Values, T(start), T(end)); + for (uint n = 1; n <= 6; n++) + { + var paged = await DrainAsync(source, T(start), T(end), n, bounds: true); + Render(paged, T(start), T(end)).ShouldBe(expected, $"{start}->{end} paged by {n}"); + } + } + } + + /// One-sided reads page to the open placeholder exactly once (footnotes a/b), never past it. + [Theory] + [InlineData(Shape.Reverse)] + [InlineData(Shape.ForwardOnly)] + [InlineData(Shape.Microsecond)] + public async Task Paging_a_one_sided_bounded_read_ends_with_the_open_placeholder(Shape shape) + { + for (uint n = 1; n <= 7; n++) + { + var fwd = await DrainAsync(Source(shape, TableData()), T("5:01"), Unspecified, n, bounds: true); + Render(fwd, T("5:01"), Unspecified).ShouldBe("5:00, 5:02, 5:03, 5:05, 5:06, LAST+", $"forward by {n}"); + var rev = await DrainAsync(Source(shape, TableData()), Unspecified, T("5:04"), n, bounds: true); + Render(rev, Unspecified, T("5:04")).ShouldBe("5:05, 5:03, 5:02, 5:00, FIRST-", $"reverse by {n}"); + } + } + + /// A bound is a stored sample of any quality: a Bad sample before StartTime is the start bound, as stored. + [Theory] + [InlineData(Shape.Reverse)] + [InlineData(Shape.ForwardOnly)] + [InlineData(Shape.Microsecond)] + public async Task A_bad_sample_is_returned_as_the_bound_as_stored(Shape shape) + { + var series = TableData(); + series[0] = new DataValueSnapshot(null, StatusCodes.BadNoCommunication, T("5:00"), T("5:00")); + var (values, _, _) = await ReadAsync(Source(shape, series), T("5:01"), T("5:04"), 0, true, cp: null); + + values[0].StatusCode.Code.ShouldBe(StatusCodes.BadNoCommunication); + values[0].SourceTimestamp.ShouldBe(T("5:00")); + values.Count.ShouldBe(4); + } + + /// + /// Ties on a bound: the start bound before a forward window is the LAST of a tie cluster (the sample immediately + /// before StartTime in the read's order); the end bound after it is the FIRST of one; and ties ON StartTime are all + /// data (the bound is the first of them, so nothing extra is added). + /// + [Theory] + [InlineData(Shape.Reverse)] + [InlineData(Shape.ForwardOnly)] + [InlineData(Shape.Microsecond)] + public async Task Tie_clusters_on_either_side_yield_one_bound_each(Shape shape) + { + var series = new List(); + void Add(string t, double v) => series.Add(new DataValueSnapshot(v, StatusCodes.Good, T(t), T(t))); + Add("5:00", 1); Add("5:00", 2); Add("5:00", 3); + Add("5:02", 4); + Add("5:05", 5); Add("5:05", 6); + + var (fwd, _, _) = await ReadAsync(Source(shape, series), T("5:01"), T("5:04"), 0, true, cp: null); + fwd.Select(v => (double)v.Value).ShouldBe([3d, 4d, 5d]); + + var (rev, _, _) = await ReadAsync(Source(shape, series), T("5:04"), T("5:01"), 0, true, cp: null); + rev.Select(v => (double)v.Value).ShouldBe([5d, 4d, 3d]); + + var (onStart, _, _) = await ReadAsync(Source(shape, series), T("5:00"), T("5:03"), 0, true, cp: null); + onStart.Select(v => (double)v.Value).ShouldBe([1d, 2d, 3d, 4d, 5d]); + } + + /// + /// QuestDB truncates the window bounds it is sent to the microsecond: a sample at ⌊StartTime⌋ is inside a window that + /// starts a few ticks later, so it is data — never returned a second time as the start bound. + /// + [Fact] + public async Task A_microsecond_backend_does_not_return_a_sample_at_the_truncated_start_twice() + { + var series = TableData(); + var (values, _, _) = await ReadAsync(Source(Shape.Microsecond, series), T("5:00").AddTicks(7), T("5:04"), 0, true, cp: null); + + values.Select(v => (double)v.Value).ShouldBe([0d, 2d, 3d, 5d]); + } + + /// + /// A backend whose forward end is inclusive (the gateway's own ReadRaw), or a resumed read that steps onto + /// EndTime, returns the sample AT EndTime as data — it must come back once, as the end bound. + /// + [Theory] + [InlineData(0u)] + [InlineData(1u)] + [InlineData(2u)] + [InlineData(3u)] + public async Task An_inclusive_forward_end_does_not_return_the_end_bound_twice(uint max) + { + var source = new SeriesSource(TableData(), reverse: false, inclusiveForwardEnd: true, resolution: 1); + var values = await DrainAsync(source, T("5:00"), T("5:05"), max, bounds: true); + Render(values, T("5:00"), T("5:05")).ShouldBe("5:00, 5:02, 5:03, 5:05"); + } + + /// + /// #691 review: no data AND no bound found ⇒ GoodNoData with no values — the answer a tag with no history (and a + /// disabled historian) gave before bounds (§6.5.3.2: bounds make the result Good only "if one or both exist"). Only + /// when the WHOLE read is empty: a single bound found keeps the placeholders (Table 1: "4:48 4:48 1 ⇒ FIRST"). + /// + [Theory] + [InlineData(Shape.Reverse)] + [InlineData(Shape.ForwardOnly)] + [InlineData(Shape.Microsecond)] + public async Task No_data_and_no_bound_found_is_GoodNoData_with_no_values(Shape shape) + { + var windows = new (string S, string E)[] { ("5:00", "5:05"), ("5:05", "5:00"), ("5:00", "5:00"), ("5:00", ""), ("", "5:05") }; + foreach (var (start, end) in windows) + { + var s = start.Length == 0 ? Unspecified : T(start); + var e = end.Length == 0 ? Unspecified : T(end); + foreach (var n in new uint[] { 0, 1, 2, 3 }) + { + if (n == 0 && (s == Unspecified || e == Unspecified)) continue; // a one-sided read needs a cap + var (values, error, cp) = await ReadResultAsync(Source(shape, []), s, e, n); + values.ShouldBeEmpty($"{start}->{end} by {n}"); + cp.ShouldBeNull(); + error.StatusCode.Code.ShouldBe(StatusCodes.Good); + LastNodeStatus.Code.ShouldBe(StatusCodes.GoodNoData, $"{start}->{end} by {n}"); + } + } + + // Positive control: one bound found keeps the missing one's placeholder. + var single = await DrainAsync(Source(shape, TableData()), T("4:48"), T("4:48"), 1, bounds: true); + Render(single, T("4:48"), T("4:48")).ShouldBe("FIRST, 5:00"); + } + + /// + /// #691 review: ServerHistorian:HonorReturnBounds = false restores the pre-#691 answer — ReturnBounds is ignored + /// (Table 1's "No" rows for every "Yes" row) and a modified read with ReturnBounds is not refused for it. + /// + [Fact] + public async Task The_kill_switch_ignores_return_bounds() + { + await fx.Gate.WaitAsync(TestContext.Current.CancellationToken); + try + { + fx.Nm.HonorReturnBounds = false; + fx.Nm.HistorianDataSource = Source(Shape.Reverse, TableData()); + var (values, _, _) = Read(T("5:01"), T("5:07"), 0, bounds: true, cp: null, modified: false); + Render(values, T("5:01"), T("5:07")).ShouldBe("5:02, 5:03, 5:05, 5:06"); + + var (_, modErr, _) = Read(T("5:00"), T("5:05"), 0, bounds: true, cp: null, modified: true); + modErr.StatusCode.Code.ShouldBe(StatusCodes.BadHistoryOperationUnsupported, "no trail on this fake — not InvalidArgument"); + } + finally + { + fx.Nm.HonorReturnBounds = true; + fx.Gate.Release(); + } + } + + [Fact] + public async Task Bounds_are_honoured_by_default() + { + await fx.Gate.WaitAsync(TestContext.Current.CancellationToken); + try { fx.Nm.HonorReturnBounds.ShouldBeTrue(); } + finally { fx.Gate.Release(); } + } + + /// Part 11 §6.5.3.3: a modified read with ReturnBounds is Bad_InvalidArgument. + [Fact] + public async Task A_modified_read_with_return_bounds_is_an_invalid_argument() + { + await fx.Gate.WaitAsync(TestContext.Current.CancellationToken); + try + { + fx.Nm.HistorianDataSource = Source(Shape.Reverse, TableData()); + var (_, error, cp) = Read(T("5:00"), T("5:05"), 0, bounds: true, cp: null, modified: true); + error.StatusCode.Code.ShouldBe(StatusCodes.BadInvalidArgument); + cp.ShouldBeNull(); + } + finally { fx.Gate.Release(); } + } + + // --- helpers -------------------------------------------------------------------------------- + + private static string Render(IReadOnlyList values, DateTime start, DateTime end) + { + var parts = new List(); + for (var i = 0; i < values.Count; i++) + { + var v = values[i]; + if (v.StatusCode.Code != StatusCodes.BadBoundNotFound) + { + v.Value.ShouldNotBeNull(); + parts.Add($"{v.SourceTimestamp.Hour}:{v.SourceTimestamp.Minute:00}"); + continue; + } + + v.Value.ShouldBeNull("a missing bound carries a null value"); + var isFirst = i == 0 && start != Unspecified && v.SourceTimestamp == start; + var isLast = end != Unspecified && v.SourceTimestamp == end; + if (isFirst) { parts.Add("FIRST"); continue; } + if (isLast) { parts.Add("LAST"); continue; } + // Open side (footnotes a/b): one second from the previous returned value. + i.ShouldBeGreaterThan(0); + var prev = values[i - 1].SourceTimestamp; + if (v.SourceTimestamp == prev.AddSeconds(1)) parts.Add("LAST+"); + else if (v.SourceTimestamp == prev.AddSeconds(-1)) parts.Add("FIRST-"); + else parts.Add($"?{v.SourceTimestamp:O}"); + } + + return string.Join(", ", parts); + } + + private async Task> DrainAsync(IHistorianDataSource source, DateTime start, DateTime end, uint max, bool bounds) + { + var all = new List(); + byte[]? cp = null; + var pages = 0; + do + { + var (values, error, next) = await ReadAsync(source, start, end, max, bounds, cp); + error.StatusCode.Code.ShouldBe(StatusCodes.Good); + if (max > 0) values.Count.ShouldBeLessThanOrEqualTo((int)max, "bounds count toward NumValuesPerNode"); + all.AddRange(values); + cp = next; + (++pages).ShouldBeLessThan(50, "paging must terminate"); + } + while (cp is not null); + + return all; + } + + /// The node-level StatusCode of the last (GoodNoData vs Good). + private StatusCode LastNodeStatus; + + private async Task<(IReadOnlyList Values, ServiceResult Error, byte[]? Cp)> ReadResultAsync( + IHistorianDataSource source, DateTime start, DateTime end, uint max) => + await ReadAsync(source, start, end, max, bounds: true, cp: null); + + private async Task<(IReadOnlyList Values, ServiceResult Error, byte[]? Cp)> ReadAsync( + IHistorianDataSource source, DateTime start, DateTime end, uint max, bool bounds, byte[]? cp) + { + await fx.Gate.WaitAsync(TestContext.Current.CancellationToken); + try + { + fx.Nm.HistorianDataSource = source; + return Read(start, end, max, bounds, cp, modified: false); + } + finally { fx.Gate.Release(); } + } + + private (IReadOnlyList Values, ServiceResult Error, byte[]? Cp) Read( + DateTime start, DateTime end, uint max, bool bounds, byte[]? cp, bool modified) + { + var context = new OperationContext(new RequestHeader(), secureChannelContext: null, RequestType.HistoryRead, identity: null); + var details = new ReadRawModifiedDetails + { + StartTime = start, EndTime = end, NumValuesPerNode = max, ReturnBounds = bounds, IsReadModified = modified, + }; + var nodesToRead = new List { new() { NodeId = fx.NodeId, ContinuationPoint = cp } }; + var results = new List { null! }; + var errors = new List { null! }; + fx.Nm.HistoryRead(context, details, TimestampsToReturn.Both, releaseContinuationPoints: false, nodesToRead, results, errors); + LastNodeStatus = results[0]?.StatusCode ?? StatusCodes.Bad; + + var values = new List(); + if (results[0]?.HistoryData is { } ho && !ExtensionObject.IsNull(ho)) + values.AddRange(((HistoryData)ExtensionObject.ToEncodeable(ho)).DataValues); + return (values, errors[0], results[0]?.ContinuationPoint); + } + + /// + /// A series-backed historian. : serves start > end as (end, start] + /// newest-first (SqlServer/QuestDB); otherwise rejects it (the Gateway) and counts that. The forward read is + /// [start, end), or [start, end] when (the gateway's own end); + /// a point read is the instant. > 1 truncates every window bound to it first (QuestDB + /// µs); the series must already be stored at that resolution. + /// + private sealed class SeriesSource(IReadOnlyList series, bool reverse, bool inclusiveForwardEnd, long resolution) + : IHistorianDataSource + { + public int ReverseWindowsOnForwardOnly; + + public bool SupportsReverseRaw => reverse; + + public long TimestampResolutionTicks => resolution; + + private DateTime Trunc(DateTime t) => new(t.Ticks - t.Ticks % resolution, DateTimeKind.Utc); + + public Task ReadRawAsync( + string fullReference, DateTime startUtc, DateTime endUtc, uint maxValuesPerNode, CancellationToken cancellationToken) + { + static DateTime Ts(DataValueSnapshot s) => s.SourceTimestampUtc ?? DateTime.MinValue; + startUtc = Trunc(startUtc); + endUtc = Trunc(endUtc); + List window; + if (startUtc > endUtc) + { + if (!reverse) + { + Interlocked.Increment(ref ReverseWindowsOnForwardOnly); + throw new InvalidOperationException("start_time must be less than or equal to end_time. (InvalidArgument)"); + } + + window = series.Where(s => Ts(s) > endUtc && Ts(s) <= startUtc).Reverse().ToList(); + } + else if (startUtc == endUtc) window = series.Where(s => Ts(s) == startUtc).ToList(); + else window = series.Where(s => Ts(s) >= startUtc && (inclusiveForwardEnd ? Ts(s) <= endUtc : Ts(s) < endUtc)).ToList(); + + var page = maxValuesPerNode == 0 ? window : window.Take((int)maxValuesPerNode).ToList(); + return Task.FromResult(new HistorianRead(page, null)); + } + + public Task ReadProcessedAsync( + string fullReference, DateTime startUtc, DateTime endUtc, TimeSpan interval, + HistoryAggregateType aggregate, CancellationToken cancellationToken) => + Task.FromResult(new HistorianRead(Array.Empty(), null)); + + public Task ReadAtTimeAsync( + string fullReference, IReadOnlyList timestampsUtc, CancellationToken cancellationToken) => + Task.FromResult(new HistorianRead(Array.Empty(), null)); + + public Task ReadEventsAsync( + string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents, CancellationToken cancellationToken) => + Task.FromResult(new HistoricalEventsResult(Array.Empty(), null)); + + public HistorianHealthSnapshot GetHealthSnapshot() => NullHistorianDataSource.Instance.GetHealthSnapshot(); + + public void Dispose() { } + } +} diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadConcurrencyTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadConcurrencyTests.cs index 93153acb..22a64e41 100644 --- a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadConcurrencyTests.cs +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadConcurrencyTests.cs @@ -45,6 +45,7 @@ public sealed class NodeManagerHistoryReadConcurrencyTests : IDisposable var details = new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = DateTime.UtcNow.AddHours(-1), EndTime = DateTime.UtcNow, NumValuesPerNode = 10, @@ -78,6 +79,7 @@ public sealed class NodeManagerHistoryReadConcurrencyTests : IDisposable var nodeIds = MaterializeHistorizedNodes(nm, count: 12); var details = new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = DateTime.UtcNow.AddHours(-1), EndTime = DateTime.UtcNow, NumValuesPerNode = 10, @@ -115,6 +117,7 @@ public sealed class NodeManagerHistoryReadConcurrencyTests : IDisposable var nodeIds = MaterializeHistorizedNodes(nm, count: 8); var details = new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = DateTime.UtcNow.AddHours(-1), EndTime = DateTime.UtcNow, NumValuesPerNode = 10, @@ -157,6 +160,7 @@ public sealed class NodeManagerHistoryReadConcurrencyTests : IDisposable var nodeIds = MaterializeHistorizedNodes(nm, count: 4); var details = new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = DateTime.UtcNow.AddHours(-1), EndTime = DateTime.UtcNow, NumValuesPerNode = 10, @@ -190,6 +194,7 @@ public sealed class NodeManagerHistoryReadConcurrencyTests : IDisposable var ids = MaterializeHistorizedNodes(nm, count: 2); var details = new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = DateTime.UtcNow.AddHours(-1), EndTime = DateTime.UtcNow, NumValuesPerNode = 10, diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadEventsPagingTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadEventsPagingTests.cs index dc0ecf85..7d4ec80f 100644 --- a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadEventsPagingTests.cs +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadEventsPagingTests.cs @@ -160,7 +160,7 @@ public sealed class NodeManagerHistoryReadEventsPagingTests : IDisposable var ctx = new OperationContext(new RequestHeader(), secureChannelContext: null, RequestType.HistoryRead, identity: null); var res = new List { null! }; var errs = new List { null! }; - nm.HistoryRead(ctx, new ReadRawModifiedDetails { StartTime = Epoch, EndTime = Epoch.AddHours(1), NumValuesPerNode = 2, IsReadModified = false }, + nm.HistoryRead(ctx, new ReadRawModifiedDetails { ReturnBounds = false, StartTime = Epoch, EndTime = Epoch.AddHours(1), NumValuesPerNode = 2, IsReadModified = false }, TimestampsToReturn.Both, false, new List { new() { NodeId = n, ContinuationPoint = cp } }, res, errs); err = errs[0]; diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadModifiedTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadModifiedTests.cs index 164638c1..48b10136 100644 --- a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadModifiedTests.cs +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadModifiedTests.cs @@ -271,7 +271,7 @@ public sealed class NodeManagerHistoryReadModifiedTests : IDisposable var results = new List { null! }; var errors = new List { null! }; nm.HistoryRead(context, - new ReadRawModifiedDetails { StartTime = start, EndTime = end, NumValuesPerNode = max, IsReadModified = modified }, + new ReadRawModifiedDetails { ReturnBounds = false, StartTime = start, EndTime = end, NumValuesPerNode = max, IsReadModified = modified }, TimestampsToReturn.Both, false, new List { new() { NodeId = node, ContinuationPoint = cp } }, results, errors); diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadPagingTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadPagingTests.cs index 747ac185..b881d737 100644 --- a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadPagingTests.cs +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadPagingTests.cs @@ -967,6 +967,7 @@ public sealed class NodeManagerHistoryReadPagingTests : IDisposable var details = new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = start, EndTime = end, NumValuesPerNode = max, @@ -1000,6 +1001,7 @@ public sealed class NodeManagerHistoryReadPagingTests : IDisposable new RequestHeader(), secureChannelContext: null, RequestType.HistoryRead, identity: null); var details = new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = Epoch, EndTime = Epoch.AddHours(1), NumValuesPerNode = 100, IsReadModified = false, }; var nodesToRead = new List { new() { NodeId = nodeId, ContinuationPoint = cp } }; diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadTests.cs index e8a231ed..a3c8a266 100644 --- a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadTests.cs +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadTests.cs @@ -52,6 +52,7 @@ public sealed class NodeManagerHistoryReadTests : IDisposable var end = DateTime.UtcNow; var details = new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = start, EndTime = end, NumValuesPerNode = 100, @@ -97,6 +98,7 @@ public sealed class NodeManagerHistoryReadTests : IDisposable var details = new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = DateTime.UtcNow.AddHours(-1), EndTime = DateTime.UtcNow, NumValuesPerNode = 10, @@ -129,6 +131,7 @@ public sealed class NodeManagerHistoryReadTests : IDisposable var details = new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = DateTime.UtcNow.AddHours(-1), EndTime = DateTime.UtcNow, NumValuesPerNode = 10, @@ -162,6 +165,7 @@ public sealed class NodeManagerHistoryReadTests : IDisposable var details = new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = DateTime.UtcNow.AddHours(-1), EndTime = DateTime.UtcNow, NumValuesPerNode = 10, @@ -195,6 +199,7 @@ public sealed class NodeManagerHistoryReadTests : IDisposable var details = new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = DateTime.UtcNow.AddHours(-1), EndTime = DateTime.UtcNow, NumValuesPerNode = 10, @@ -613,6 +618,7 @@ public sealed class NodeManagerHistoryReadTests : IDisposable var details = new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = DateTime.UtcNow.AddHours(-1), EndTime = DateTime.UtcNow, NumValuesPerNode = 10, @@ -658,6 +664,7 @@ public sealed class NodeManagerHistoryReadTests : IDisposable var details = new ReadRawModifiedDetails { + ReturnBounds = false, StartTime = DateTime.UtcNow.AddHours(-1), EndTime = DateTime.UtcNow, NumValuesPerNode = 10, diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Historian/ServerHistorianOptionsTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Historian/ServerHistorianOptionsTests.cs index 767ec21f..40bea597 100644 --- a/tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Historian/ServerHistorianOptionsTests.cs +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Historian/ServerHistorianOptionsTests.cs @@ -104,6 +104,21 @@ public sealed class ServerHistorianOptionsTests /// #674: the HistoryUpdate future-skew bound defaults to one day and binds from configuration as a TimeSpan; /// a non-positive value is a deliberate "disabled", not a misconfiguration, so it does not warn. /// + /// #691: the ReturnBounds kill switch defaults on and binds off; neither value warns. + [Fact] + public void HonorReturnBounds_defaults_to_true_and_binds_false_silently() + { + Assert.True(new ServerHistorianOptions().HonorReturnBounds); + var bound = new ConfigurationBuilder() + .AddInMemoryCollection(new Dictionary { ["ServerHistorian:HonorReturnBounds"] = "false" }) + .Build() + .GetSection(ServerHistorianOptions.SectionName) + .Get(); + Assert.False(bound!.HonorReturnBounds); + Assert.Empty(new ServerHistorianOptions + { Enabled = true, Endpoint = "https://h:5222", ApiKey = "histgw_x_y", HonorReturnBounds = false }.Validate()); + } + [Fact] public void HistoryUpdateMaxFutureSkew_defaults_to_one_day_binds_and_zero_is_silent() {