diff --git a/CLAUDE.md b/CLAUDE.md index ce8f2845..e8a24343 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1353,7 +1353,7 @@ historian tagname, which is the tag's driver `FullName`. - **SQL Server** (`Driver.Historian.SqlServer`): `hist.Sample` PK `(TagId, SourceTimeUtc)` — writes are insert-if-absent, so outbox redelivery is idempotent. Stores every value kind with its **real status**; per-sample aggregates (10, simple definitions) are pushed down to T-SQL with bucket math in **100 ns ticks**; the Part 13 time-weighted seven (TimeAverage/TimeAverage2/Total/Total2/Interpolative, #671 — ⚠️ `Total` is now the time-weighted integral, no longer `SUM` — plus the **duration-based** PercentGood/PercentBad, #680) stream window + 4 bound rows into the shared `TimeWeightedAggregator` (both SQL backends, identical results; reverse `Interpolative` anchored at the later time; window capped by `ServerHistorian:MaxTimeWeightedRawRows` ⇒ `BadTooManyOperations`, percentages included). ⚠️ **#680 status rules (client-visible):** TimeAverage/Total are Uncertain if any part of the interval uses an Uncertain value, is bridged across a Bad sample, is extrapolated past the last sample (⇒ a live tag's latest interval is Uncertain) or is not covered; TimeAverage2/Total2 keep the SDK time-based rule (Uncertain time counts as Good) but never extrapolate past the last sample — that interval is Partial, later ones `BadNoData` (a report-by-exception tag with nothing after its last sample reads `BadNoData` until its next sample arrives). `docs/Historian.md` §"Time-weighted aggregates". ⚠️ **#680 per-sample (client-visible):** `Count` counts **Good** samples only (value-based status; an all-Bad interval is generic Bad, value null) and an empty interval **inside the tag's data range** is `0`/Good (outside it still `BadNoData`); `Start`/`End` return the **raw** first/last sample of any quality — its own timestamp and status, value null when Bad; `Delta` is Uncertain only when the interval's first or last sample is unusable; ⚠️ #684: `Average`/`Minimum`/`Maximum`/`Range`/`*ActualTime` use **Good samples only** (was Good + Uncertain; an Uncertain-only interval is `BadNoData`), Min/Max-family status Uncertain on a Bad sample, an Uncertain value beyond the Good extreme (Part 13 — the SDK ignores that) or an Uncertain non-numeric sample, and only a **Good** String/DateTime sample refuses them (`BadAggregateInvalidInputs`; Delta still refuses any usable one); and every per-sample aggregate but `Average` sets **end-of-data Partial** on the interval holding the last sample (forward incl. its earlier edge, reverse strictly inside), never on a Bad result. The data-range lookup is a key seek on SQL Server and a `tag` symbol-index seek on QuestDB (#683 — `{p}sample.tag` is `SYMBOL INDEX`; the bootstrapper ADDs it to a pre-#683 table, idempotent), read BEFORE the bucket statement so a racing append can never yield a Good `Count` of 0 (a concurrent HistoryUpdate delete is the mirror case and still can, transiently). `docs/Historian.md` §"Aggregates". Docker-dev recipe, schema and aggregate semantics: `docs/Historian.md` §"SQL Server backend". Integration tests need `HISTORIAN_SQL_TEST_CONNECTION`. **Retention (#667):** opt-in `SqlServer:RetentionDays` (+ `PurgeInterval` 1h, `PurgeBatchSize` 4000) -> `HistorianRetentionService` purges old `hist.Sample` (per tag) / `hist.Event` in `DELETE TOP(n)` batches, Primary-gated on `IHistorizationRoleView`, one purger per DB via `sp_getapplock`; counter `otopcua.historian.retention.purged{table}`. Optional `IHistorianBackendProvider.CreateRetention` (default null). `docs/Historian.md` §Retention. - **QuestDB** (`Driver.Historian.QuestDb`, `ServerHistorian:QuestDb:*`, Npgsql over PG wire; docker-dev overlay `docker-compose.questdb.yml`): same kinds/status/aggregates/HistoryUpdate as SqlServer, soft delete by tombstone (INSERT only — never `UPDATE`/`DELETE`; every read filters `NOT deleted`). ⚠️ **Timestamps are µs** (`QuestDbConnection.Round`/`BindTimestamp`; `IHistorianDataSource.TimestampResolutionTicks` = 10 steps HistoryRead paging), **pre-1970 cannot be stored** (writer drops, HistoryUpdate `BadOutOfRange`), every write waits for WAL visibility, TTL is enforced on write commits only, set Npgsql `Maximum Pool Size` < QuestDB's 64. `docs/Historian.md` §"QuestDB backend". - ⚠️ **Reverse processed reads are Part 13/SDK-anchored (#677, client-visible):** every aggregate on SqlServer/QuestDB now anchors a reverse read at its later time — window `(end, start]`, partial interval earliest, each stamped with its later edge (was: earlier-anchored, stamped earlier edge); on forward reads Partial also marks the interval the data starts inside (not Average — the SDK's rule, forward-only); a non-default client `AggregateConfiguration` ⇒ `BadAggregateConfigurationRejected`; Gateway provisioning sends `IntegralDivisor = 1`. SDK-parity tests: `SdkAggregateParityTests`. `docs/Historian.md` §Aggregates. -- ⚠️ **Reverse / end-only RAW reads work on Gateway (#686, client-visible):** `IHistorianDataSource.SupportsReverseRaw` (SqlServer/QuestDB/Null `true`, Gateway `false`, interface default `false`, guard-tested) — the node manager sends `start > end` only to a backend that declares it; otherwise `ReverseRawReader` serves it from FORWARD reads: uncapped ⇒ ONE forward read reversed in memory; capped ⇒ a bounded walk back in segments (ms-aligned request bounds, width aimed from observed density at ≈ N + 8 rows and carried page to page in the continuation point — #689: it used to aim at half `MaxTieClusterOverfetch`, ~32k rows per page on a dense tag; a read asks for `2N + 18` rows (a discovery segment ≤ 1,026), `MaxTieClusterOverfetch` is only the tie-cluster ceiling — a full read is re-read, never answered truncated; only one timestamp holding more than max(1, `MaxTieClusterOverfetch`) samples ⇒ `BadTooManyOperations`; walks no later than now + 1 day and never before the Unix epoch; #691: after two empty segments it asks once for the whole remainder, and the stretch a page proved empty rides the continuation point — the last pages of a deep page-through no longer each walk ~14 reads to 1970). The raw/events tie-path oversized-cluster refusal is now `BadTooManyOperations` too (was `BadHistoryOperationUnsupported`). Uncapped Gateway raw reads (forward too) used to send `max_values = 0`, which the client/gateway reject — now unbounded (`int.MaxValue`; a ceiling refusal ⇒ `BadTooManyOperations`), and a cancelled read is no longer a backend failure. Since #669 such reads failed the node on Gateway and counted as a backend fault (the gateway rejects start > end — `GatewayHistorianDataSource.ReadRawAsync` now refuses one unsent); before #669 an end-only Gateway read returned the OLDEST N ascending — now the latest N at/before EndTime, newest first. Events still differ (end-only forward / explicit reverse refused on Gateway, #675). `docs/Historian.md` §"Open-ended and reverse raw reads". +- ⚠️ **Reverse / end-only RAW reads work on Gateway (#686, client-visible):** `IHistorianDataSource.SupportsReverseRaw` (SqlServer/QuestDB/Null `true`, Gateway `false`, interface default `false`, guard-tested) — the node manager sends `start > end` only to a backend that declares it; otherwise `ReverseRawReader` serves it from FORWARD reads: uncapped ⇒ ONE forward read reversed in memory; capped ⇒ a bounded walk back in segments (ms-aligned request bounds, width aimed from observed density at ≈ N + 8 rows and carried page to page in the continuation point — #689: it used to aim at half `MaxTieClusterOverfetch`, ~32k rows per page on a dense tag; a read asks for `2N + 18` rows (a discovery segment ≤ 1,026), `MaxTieClusterOverfetch` is only the tie-cluster ceiling — a full read is re-read, never answered truncated; only one timestamp holding more than max(1, `MaxTieClusterOverfetch`) samples ⇒ `BadTooManyOperations`; walks no later than now + 1 day and never before the Unix epoch; #691: after two empty segments it asks once for the whole remainder, and the stretch a page proved empty rides the continuation point — the last pages of a deep page-through no longer each walk ~14 reads to 1970). The raw/events tie-path oversized-cluster refusal is now `BadTooManyOperations` too (was `BadHistoryOperationUnsupported`). Uncapped Gateway raw reads (forward too) used to send `max_values = 0`, which the client/gateway reject — now unbounded (`int.MaxValue`; a ceiling refusal ⇒ `BadTooManyOperations`), and a cancelled read is no longer a backend failure. Since #669 such reads failed the node on Gateway and counted as a backend fault (the gateway rejects start > end — `GatewayHistorianDataSource.ReadRawAsync` now refuses one unsent); before #669 an end-only Gateway read returned the OLDEST N ascending — now the latest N at/before EndTime, newest first. Events follow since #691.4: a reverse/end-only capped events read on Gateway/Null (`SupportsReverseEvents` false) is walked back the same way (ties sorted by EventId — the gateway orders by time only; one extra event found ⇒ continuation point) — it used to be refused `BadHistoryOperationUnsupported` (explicit reverse) or answered with the OLDEST N ascending (end-only); needs the gateway's `RuntimeDb:EventReadMaxRows` 0/unlimited (it truncates silently). `docs/Historian.md` §"Open-ended and reverse raw reads". - **Advertised aggregates (#682):** `IHistorianDataSource.SupportedAggregates` (Gateway 4 via `GatewayHistorianDataSource.ServedAggregates`, SqlServer/QuestDB 17 via `AggregateFamilies.Served`, Null **none**; a null set ⇒ all) is the one authority for `HistoryServerCapabilities/AggregateFunctions` (i=11201 — only its folder references are trimmed) and the up-front `BadAggregateNotSupported` in HistoryRead-Processed. ⚠️ `ServerCapabilities/AggregateFunctions` (i=2997) is deliberately NOT trimmed: it keeps the SDK's 37 because live `AggregateFilter` subscriptions still compute them all. ⚠️ Historian disabled ⇒ Processed is now `BadAggregateNotSupported`, not `GoodNoData`. `docs/Historian.md` §Advertised aggregates. - **Real status fix:** the recorder used to stamp every sample Good (`192`); it now carries the driver's status. ⚠️ On the **Gateway** backend this changes nothing stored: the gateway discards quality (C-002), so AVEVA still stores Good. - **Comms loss is historized as an outage marker** (`Empty` + `BadNoCommunication`, one per tag per outage) on SqlServer; dropped (`DroppedUnsupportedKind`) on Gateway. Outbox records are **format v2** — drain or delete `ContinuousHistorization:OutboxPath` before downgrading a node. diff --git a/docs/Historian.md b/docs/Historian.md index 82b68546..c14a98ff 100644 --- a/docs/Historian.md +++ b/docs/Historian.md @@ -677,8 +677,7 @@ backend: > ⚠️ **Client-visible (#686).** Since #669 an end-only capped raw read, and any explicit reverse raw read, on the Gateway > backend failed the node (`BadHistoryOperationUnsupported`) and was counted as a historian backend failure (the gateway > refused `start > end`). Before #669 an end-only read on Gateway returned the **oldest** N values ascending. Both now return -> the Part 11 answer — the latest values at or before `EndTime`, newest first. **Events differ:** an end-only events read on -> Gateway still keeps its forward meaning and an explicit reverse events read is refused (see below). +> the Part 11 answer — the latest values at or before `EndTime`, newest first. **Events** follow since #691.4 (see below). > > ⚠️ **Also fixed with #686: uncapped raw reads on Gateway.** `NumValuesPerNode = 0` ("all values") used to be forwarded as > `max_values = 0`, which the client package and the gateway reject — every uncapped raw read (forward too) failed the node @@ -763,30 +762,43 @@ says is complete carries no continuation point. `NumValuesPerNode == 0` stays un that many more rows than the cap, so every page short of the last is a FULL `NumValuesPerNode` page (#675; resumed pages used to come back short by the skip). - A tie cluster larger than the cap is paged within by over-fetching the half-open window `[T, T + resolution)` up to - `MaxTieClusterOverfetch`; beyond that bound the node answers `BadHistoryOperationUnsupported` (raise the bound). + `MaxTieClusterOverfetch`; beyond that bound the node answers `BadTooManyOperations` (raise the bound). - **Tie exactness depends on the backend.** SqlServer and QuestDB order by `(EventTimeUtc, EventId)`, so the skip count is exact. - The Gateway backend has no stable intra-timestamp order (and only a truncation flag), so ties at a page boundary are - best-effort there. + The Gateway backend has no stable intra-timestamp order (and only a truncation flag), so ties at a page boundary of a + **forward** read are best-effort there; a walked-back **reverse** read (below) sorts them by `EventId` itself and is exact. - **Reverse event reads page newest-first (#675).** `StartTime > EndTime` reads the Part 11 reverse window `(EndTime, StartTime]` with events in DESCENDING `(EventTimeUtc, EventId)` order, so a cap keeps the **latest** N and a continuation point is issued when the source flags more. The direction is stored **in the cursor**: a reverse point resumes reverse (and a forward one forward) whatever the resume request's details say. Reverse tie clusters are over-fetched forward and sliced from the newest end, exactly as Raw does. Before #675 a capped reverse read silently returned the EARLIEST N, ascending, Good, with no continuation point. -- **End-only events reads: backward on SqlServer/QuestDB, forward on Gateway (#675).** `StartTime` unset - (`DateTime.MinValue`) with an `EndTime` and `NumValuesPerNode > 0` pages backward from `EndTime` over `(MinValue, EndTime]` — - the same rule as Raw — **only on a backend that reports `SupportsReverseEvents`** (SqlServer, QuestDB). On Gateway (the - default backend) and Null it keeps its pre-#675 forward `[MinValue, EndTime)` meaning, paged oldest-first, rather than - becoming a refused reverse read. Start-only reads page forward to an open upper bound (`DateTime.MaxValue`, UTC) everywhere. - Without a cap an end-only read is forward `[MinValue, EndTime)` everywhere. -- **Backends.** The server checks `IHistorianDataSource.SupportsReverseEvents` and then calls - `ReadEventsAsync(…, bool reverse, …)`. SqlServer and QuestDB serve reverse natively (`… > lo AND … <= hi ORDER BY time DESC, - id DESC`) and report `true`. **Gateway and Null report `false`**: an explicit reverse read (`StartTime > EndTime`) is answered - `BadHistoryOperationUnsupported` **without calling the backend** — the gateway streams ascending only and rejects `end < start`, - and draining the whole window to keep its tail would be exactly the unbounded read the cap prevents. Because the refusal is a - capability check, not a caught exception, a `NotSupportedException` thrown from inside a backend (a driver library) is logged - as a backend fault like any other. Every implementation (and any future wrapper) must declare both members itself — the - interface defaults (`false` / throw) would silently disable reverse reads; `HistorianDataSourceReverseEventsGuardTests` pins it. +- **End-only events reads page backward on every backend (#675, #691.4).** `StartTime` unset (`DateTime.MinValue`) with an + `EndTime` and `NumValuesPerNode > 0` pages backward from `EndTime` over `(MinValue, EndTime]` — the same rule as Raw. + Start-only reads page forward to an open upper bound (`DateTime.MaxValue`, UTC) everywhere. Without a cap an end-only read + is forward `[MinValue, EndTime)` everywhere. +- **Backends.** The server checks `IHistorianDataSource.SupportsReverseEvents`. SqlServer and QuestDB serve reverse natively + (`… > lo AND … <= hi ORDER BY time DESC, id DESC`, the `ReadEventsAsync(…, bool reverse, …)` overload) and report `true`. + **Gateway and Null report `false`** — the gateway streams ascending only and rejects `end < start` — and are never sent a + reverse read: since **#691.4** the server walks it back from bounded **forward** reads with the raw walk (#686/#689/#691: + page-sized ms-aligned segments filtered to `(segLo, segHi]`, a read the source flagged truncated is never answered from, + the now + 1 day / 1970 bounds, the remainder probe and the proven-empty stretch carried in the cursor). Event specifics: + - **Ties** are sorted by `EventId` (ordinal) on every read — the gateway orders by `EventTimeUtc` only, in any order within a + timestamp — so the page is DESCENDING `(EventTimeUtc, EventId)` and a resumed page's boundary skip lines up; the + oversized-cluster over-fetch keeps exactly the events at `T` (the gateway's end is inclusive) in the same order. + - **"More exists"** is found by walking for one event past the page (the walk has no source truncation flag to hand on). + - **Uncapped** reverse reads are one forward read of the window, sorted and reversed. + - A timestamp holding more than `MaxTieClusterOverfetch` events ⇒ **`BadTooManyOperations`** for that node. + - ⚠️ **Gateway prerequisite:** the gateway's `RuntimeDb:EventReadMaxRows` must be `0` (its default, unlimited) or larger than + the walk's biggest read (`2N + 18` rows, a discovery read 1,026, a tie-cluster read up to `2·MaxTieClusterOverfetch + 4`). + That cap truncates **silently** (the stream just ends), so a segment it cut would look complete and drop its newest events — + the same blindness a forward read has to it, but on the end a reverse read is asking for. + + ⚠️ **Client-visible (#691.4).** On Gateway (the default backend) an explicit reverse events read used to be refused + `BadHistoryOperationUnsupported` (#675), and an end-only capped events read returned the **oldest** N events ascending. Both + now return the latest events newest-first. Because the capability is checked, not a caught exception, a + `NotSupportedException` thrown from inside a backend (a driver library) is still logged as a backend fault. Every + implementation (and any future wrapper) must declare both members itself — the interface defaults (`false` / throw) would + silently send a native-reverse backend through the walk; `HistorianDataSourceReverseEventsGuardTests` pins it. - **Gateway open upper bound.** The `DateTime.MaxValue` (UTC) bound of a start-only forward read is pinned offline through the published client's proto `Timestamp` conversion (captured by an in-process gRPC server) and the FILETIME encoding (`GatewayEventBoundsTests`). What a live AVEVA Historian does with a year-9999 bound is still unverified (needs the VPN). diff --git a/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianDataSource.cs b/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianDataSource.cs index 926881c1..3b9c1b97 100644 --- a/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianDataSource.cs +++ b/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianDataSource.cs @@ -120,9 +120,9 @@ public interface IHistorianDataSource : IDisposable /// /// Whether this backend can serve a reverse (newest-first) events read — the reverse overload of /// (Gitea #675). The server - /// checks it BEFORE calling the source: an explicit reverse read (StartTime > EndTime) on a backend that - /// cannot is answered BadHistoryOperationUnsupported without a call, and an end-only capped read, which Part 11 - /// reads backward, keeps its forward [MinValue, EndTime) meaning there. Default false; an + /// checks it BEFORE calling the source: a reverse read (explicit StartTime > EndTime, or an end-only capped + /// read, which Part 11 reads backward) on a backend that cannot is never sent the reverse overload — since Gitea + /// #691.4 the server walks it back from bounded FORWARD reads instead (#675 refused it). Default false; an /// implementation (or a wrapper) declares it itself — a guard test pins that. /// bool SupportsReverseEvents => false; @@ -144,9 +144,9 @@ public interface IHistorianDataSource : IDisposable /// /// The default forwards a forward read to the five-argument overload and throws /// for a reverse one — a backend that cannot read newest-first (Gateway, - /// Null). The server never relies on that throw: it consults first and - /// answers BadHistoryOperationUnsupported without calling, rather than returning the earliest events in - /// ascending order as if they were the answer. A backend that serves reverse must override BOTH members. + /// Null). The server never relies on that throw: it consults first and walks + /// such a read back from forward reads (#691.4), never returning the earliest events in ascending order as if + /// they were the answer. A backend that serves reverse natively must override BOTH members. /// /// /// The source name to filter events, or null to return events from all sources. diff --git a/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/NullHistorianDataSource.cs b/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/NullHistorianDataSource.cs index 769f1d4e..9d97b317 100644 --- a/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/NullHistorianDataSource.cs +++ b/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/NullHistorianDataSource.cs @@ -61,8 +61,8 @@ public sealed class NullHistorianDataSource : IHistorianDataSource public bool SupportsReverseRaw => true; /// - /// false (Gitea #675): there is no history to read in either direction, and an explicit reverse read - /// is answered BadHistoryOperationUnsupported like any backend that cannot read newest-first. + /// false (Gitea #675): there is no history to read in either direction; the server walks a reverse + /// read back from (empty) forward reads like any backend that cannot read newest-first (#691.4). public bool SupportsReverseEvents => false; /// diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/GatewayHistorianDataSource.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/GatewayHistorianDataSource.cs index 5685ed32..b25790be 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/GatewayHistorianDataSource.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/GatewayHistorianDataSource.cs @@ -393,8 +393,9 @@ public sealed class GatewayHistorianDataSource : IHistorianDataSource, IAsyncDis /// /// - /// false: the gateway's ReadEvents streams ascending only (Gitea #675). The server therefore refuses an - /// explicit reverse events read without calling, and keeps an end-only capped read forward. + /// false: the gateway's ReadEvents streams ascending only (Gitea #675). The server never sends it a + /// reverse read; since #691.4 it walks one back from bounded forward reads (ties sorted by EventId — the gateway + /// orders by event time only). /// public bool SupportsReverseEvents => false; @@ -404,10 +405,9 @@ public sealed class GatewayHistorianDataSource : IHistorianDataSource, IAsyncDis /// /// - /// A reverse (newest-first) read is refused with — the server reports - /// BadHistoryOperationUnsupported (Gitea #675). The gateway's ReadEvents streams ascending only and - /// rejects end < start, so the only way to serve "the latest N" would be to drain the whole window and - /// keep its tail, an unbounded read the cap exists to prevent. Nothing is sent. + /// A reverse (newest-first) read is refused with (Gitea #675) — the gateway's + /// ReadEvents streams ascending only and rejects end < start. The server never calls this with + /// reverse: it walks a reverse read back from bounded forward reads (#691.4). Nothing is sent. /// public Task ReadEventsAsync( string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents, bool reverse, diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs b/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs index 6b6c5dca..57ab3fc8 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs @@ -5248,6 +5248,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 var endUtc = details.EndTime; var numValuesPerNode = details.NumValuesPerNode; var boundarySkip = 0; + var walkHint = default(ReverseWalkHint); // #691.4: a walked-back read's width + proven-empty stretch bool reverse; if (resumed) @@ -5271,35 +5272,30 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 // The direction is the CURSOR's (Gitea #675), never re-derived from the resume request's details: a // reverse read resumes reverse, a forward one forward. reverse = state.Reverse; + walkHint = state.WalkHint; } else { // Part 11: with one bound unset, an unset EndTime (MinValue) with a StartTime means "page FORWARD from // StartTime" — normalise it to an open upper bound, or the cursor would store MinValue and the read would // look reversed. An unset StartTime with an EndTime and a count cap means "page BACKWARD from EndTime" - // (#675, mirroring raw): the reverse window (MinValue, EndTime] — but only on a backend that can read - // newest-first. Elsewhere (Gateway, the default backend) and without a count cap an end-only read keeps its - // historical forward [MinValue, EndTime) meaning rather than becoming a refused reverse read. + // (#675, mirroring raw): the reverse window (MinValue, EndTime] — on every backend since #691.4 (a backend + // that cannot read newest-first is walked back below). Without a count cap an end-only read keeps its + // forward [MinValue, EndTime) meaning. if (endUtc == DateTime.MinValue && startUtc != DateTime.MinValue) endUtc = new DateTime(DateTime.MaxValue.Ticks, DateTimeKind.Utc); // Utc kind: a proto Timestamp conversion rejects Unspecified - else if (numValuesPerNode > 0 && startUtc == DateTime.MinValue && endUtc != DateTime.MinValue - && source.SupportsReverseEvents) + else if (numValuesPerNode > 0 && startUtc == DateTime.MinValue && endUtc != DateTime.MinValue) (startUtc, endUtc) = (endUtc, new DateTime(0, DateTimeKind.Utc)); // Part 11: StartTime later than EndTime is a reverse read — events newest-first over (end, start]. reverse = startUtc > endUtc; } - // An explicit reverse read (or a reverse cursor resumed after the source was swapped) on a backend that cannot - // read newest-first: refuse it BEFORE calling the source — never the earliest events in ascending order as if - // they were the answer. Checked here rather than by catching an exception, so a NotSupportedException thrown - // from inside a backend (a driver library, the projection) stays a backend fault. - if (reverse && !source.SupportsReverseEvents) - { - errors[handle.Index] = StatusCodes.BadHistoryOperationUnsupported; - results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadHistoryOperationUnsupported }; - return; - } + // A reverse read on a backend that cannot read newest-first (Gateway, Null) is never sent there (#675 refused it + // here); since #691.4 it is walked back from bounded forward reads — the raw walk (#686), with ties ordered by + // EventId because the gateway orders by event time only. Decided by the capability, not by catching an + // exception, so a NotSupportedException thrown from inside a backend stays a backend fault. + var walked = reverse && !source.SupportsReverseEvents; // NOT under the node-manager Lock — awaiting the async source is safe here. // NumValuesPerNode==0 means "no limit — return ALL values" per OPC UA Part 4/11, so it is translated @@ -5309,8 +5305,39 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 var fetch = EventMaxEvents(numValuesPerNode); if (numValuesPerNode > 0 && boundarySkip > 0) fetch = (int)Math.Min(int.MaxValue - 1L, (long)fetch + boundarySkip); - var sourceResult = await source.ReadEventsAsync( - sourceName, startUtc, endUtc, fetch, reverse, ct).ConfigureAwait(false); + HistoricalEventsResult sourceResult; + if (walked) + { + // Walk for one more than the fetch: finding it is the "more events exist" signal a source gives with its + // continuation point (the walk itself cannot flag truncation). + var want = numValuesPerNode == 0 ? 0u : (uint)Math.Min(uint.MaxValue - 1L, (long)fetch + 1); + var budget = Math.Max(1, MaxTieClusterOverfetch); + var walkSource = sourceName; + var (rows, next) = await ReverseWalk.WalkAsync( + async (s, e, cap, token) => + { + var r = await source.ReadEventsAsync(walkSource, s, e, cap == 0 ? int.MaxValue : cap, reverse: false, token) + .ConfigureAwait(false); + return new ForwardSegmentRead(r.Events, r.ContinuationPoint is not null); + }, + ev => ev.EventTimeUtc.Ticks, EventIdTieOrder.Instance, startUtc, endUtc, want, budget, + source.TimestampResolutionTicks, Clock.GetUtcNow().UtcDateTime, walkHint, + at => new HistoryReadLimitExceededException(budget, + $"A reverse events read of '{walkSource}' at {at:O} found more than {budget} events at one " + + "timestamp; the backend reads forward only, so the latest of them cannot be selected. Increase " + + "ServerHistorian:MaxTieClusterOverfetch."), + ct) + .ConfigureAwait(false); + walkHint = next; + var more = want > 0 && rows.Count > fetch; + if (more) rows.RemoveRange(fetch, rows.Count - fetch); + sourceResult = new HistoricalEventsResult(rows, more ? [0x01] : null); + } + else + { + sourceResult = await source.ReadEventsAsync( + sourceName, startUtc, endUtc, fetch, reverse, ct).ConfigureAwait(false); + } // Truncation is the SOURCE's word (non-null continuation point) — never inferred from the count, so a // page that happens to be exactly cap-sized but complete never emits a point. @@ -5350,6 +5377,12 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 sourceName, startUtc, startUtc.AddTicks(resolution), overfetchCap, reverse: false, ct) .ConfigureAwait(false)).Events; + // A walked read (#691.4) emitted its ties sorted by EventId; the gateway returns them in any order and its + // end is inclusive — keep exactly the cluster, in the walk's order, so the skip indexes the same events. + if (walked) + cluster = cluster.Where(e => e.EventTimeUtc == startUtc) + .OrderBy(e => e, EventIdTieOrder.Instance).ToList(); + // The over-fetch reads the cluster FORWARD (ascending EventId), but a reverse read emitted its ties — and // counted boundarySkip — in descending order. Slice in the read's own direction, as raw does. if (reverse) cluster = cluster.Reverse().ToList(); @@ -5378,7 +5411,8 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 { pendingCursor = new HistoryContinuationState( sourceName ?? string.Empty, resumeAt, endUtc, nextSkip, numValuesPerNode, HistoryCursorKind.Events, - Reverse: reverse); + Reverse: reverse, SegmentWidthTicks: walkHint.WidthTicks, EmptyFromTicks: walkHint.EmptyFromTicks, + EmptyToTicks: walkHint.EmptyToTicks); } } else if (truncated && events.Count > 0) @@ -5390,7 +5424,8 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 for (var i = events.Count - 1; i >= 0 && events[i].EventTimeUtc == boundary; i--) skip++; pendingCursor = new HistoryContinuationState( sourceName ?? string.Empty, boundary, endUtc, skip, numValuesPerNode, HistoryCursorKind.Events, - Reverse: reverse); + Reverse: reverse, SegmentWidthTicks: walkHint.WidthTicks, EmptyFromTicks: walkHint.EmptyFromTicks, + EmptyToTicks: walkHint.EmptyToTicks); } var historyEvent = ProjectEvents(events, selectClauses); @@ -5411,6 +5446,14 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2 errors[handle.Index] = StatusCodes.BadTimeout; results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadTimeout }; } + catch (HistoryReadLimitExceededException ex) + { + // A request limit (#691.4: a walked-back reverse read met a tie cluster over the budget) — BadTooManyOperations + // for THIS node, as on the raw path; not a backend fault. + _logger.LogWarning("OtOpcUaNodeManager: HistoryReadEvents limit exceeded for node {NodeId}: {Message}", handle.NodeId, ex.Message); + errors[handle.Index] = StatusCodes.BadTooManyOperations; + results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadTooManyOperations }; + } catch (Exception ex) { // One node's backend failure must not throw out of the batch — surface Bad for THIS node diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/ReverseWalk.cs b/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/ReverseWalk.cs index f68f167e..738c6de5 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/ReverseWalk.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/ReverseWalk.cs @@ -1,3 +1,5 @@ +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + namespace ZB.MOM.WW.OtOpcUa.OpcUaServer; /// What one walked-back page tells the next page of the same read (carried in the continuation point). @@ -364,3 +366,16 @@ internal static class ReverseWalk return down > DateTime.MaxValue.Ticks - step ? DateTime.MaxValue.Ticks : down + step; } } + +/// +/// Orders events within one timestamp by (ordinal) — the tie order a walked-back +/// events read (#691.4) and its oversized-cluster over-fetch share, because the Gateway orders by event time only. +/// +internal sealed class EventIdTieOrder : IComparer +{ + /// The shared instance. + public static readonly EventIdTieOrder Instance = new(); + + /// + public int Compare(HistoricalEvent? x, HistoricalEvent? y) => string.CompareOrdinal(x?.EventId, y?.EventId); +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests/GatewayEventBoundsTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests/GatewayEventBoundsTests.cs index 5f8ad206..548b4f14 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests/GatewayEventBoundsTests.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests/GatewayEventBoundsTests.cs @@ -35,7 +35,7 @@ public sealed class GatewayEventBoundsTests { var fake = new FakeHistorianGatewayClient(); IHistorianDataSource ds = new GatewayHistorianDataSource(fake, NullLogger.Instance); - ds.SupportsReverseEvents.ShouldBeFalse("the server must keep end-only reads forward on the default backend"); + ds.SupportsReverseEvents.ShouldBeFalse("the server must walk reverse reads back from forward reads on the default backend (#691.4)"); await Should.ThrowAsync(() => ds.ReadEventsAsync("Pump1", DateTime.UtcNow, DateTime.UtcNow.AddHours(-1), 10, reverse: true, Ct)); diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadEventsPagingTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadEventsPagingTests.cs index 17c446ac..89be1c67 100644 --- a/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadEventsPagingTests.cs +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/NodeManagerHistoryReadEventsPagingTests.cs @@ -333,12 +333,16 @@ public sealed class NodeManagerHistoryReadEventsPagingTests : IDisposable } [Fact] - public async Task A_backend_without_reverse_support_answers_BadHistoryOperationUnsupported_not_the_earliest_events() + public async Task A_backend_without_reverse_support_walks_a_reverse_read_back_never_answering_the_earliest_events() { + // #691.4 (was refused BadHistoryOperationUnsupported since #675): served newest-first from forward reads. var (host, nm, node, src) = await SetupAsync(Distinct(10)); - nm.HistorianDataSource = new ForwardOnlyEventSource(src); - ReadEvents(nm, node, max: 3, inboundCp: null, end: Early, start: Late).Error.StatusCode.Code - .ShouldBe(StatusCodes.BadHistoryOperationUnsupported); + var fwd = new ForwardOnlyEventSource(src); + nm.HistorianDataSource = fwd; + var r = ReadEvents(nm, node, max: 3, inboundCp: null, end: Early, start: Late); + r.Error.StatusCode.Code.ShouldBe(StatusCodes.Good); + r.Ids.ShouldBe(["e0009", "e0008", "e0007"]); + r.Cp.ShouldNotBeNull(); // Forward reads on the same backend are unaffected. DrainAsync(nm, node, max: 4).ShouldBe(src.All.Select(e => e.EventId)); await host.DisposeAsync(); @@ -349,30 +353,115 @@ public sealed class NodeManagerHistoryReadEventsPagingTests : IDisposable // --- #675 review follow-ups: the reverse-events capability ------------------------------------------------ + // --- #691.4: reverse events on a backend that reads forward only (the Gateway) are walked back ------------------ + [Fact] - public async Task End_only_capped_read_on_a_backend_without_reverse_support_pages_forward_as_before() + public async Task End_only_capped_read_on_a_forward_only_backend_pages_backward_from_EndTime() { - // Gateway (the default backend) cannot read newest-first: an end-only read keeps its pre-#675 forward - // [MinValue, EndTime) meaning there rather than turning into a refused reverse read. - var (host, nm, node, src) = await SetupAsync(Distinct(25)); - var fwd = new NonReverseSource(src); - nm.HistorianDataSource = fwd; + // Part 11 (as raw, #686): the latest events at or before EndTime, newest first — on Gateway this used to keep its + // pre-#675 forward [MinValue, EndTime) meaning, i.e. the OLDEST events. The backend is never sent a reverse read. + var (host, nm, node, _) = await SetupAsync([]); + var gw = new GatewayShapedEventSource(Distinct(25)); + nm.HistorianDataSource = gw; var ids = DrainAsync(nm, node, max: 10, start: DateTime.MinValue, end: Epoch.AddSeconds(20)); - ids.ShouldBe(Enumerable.Range(0, 20).Select(i => $"e{i:0000}")); - fwd.ReverseCalls.ShouldBe(0); + ids.ShouldBe(Enumerable.Range(0, 21).Reverse().Select(i => $"e{i:0000}")); + gw.ReverseCalls.ShouldBe(0); + gw.BadRequests.ShouldBe(0); await host.DisposeAsync(); } [Fact] - public async Task Explicit_reverse_read_on_a_backend_without_reverse_support_is_refused_without_calling_it() + public async Task Explicit_reverse_read_on_a_forward_only_backend_pages_the_reverse_window_newest_first() { - var (host, nm, node, src) = await SetupAsync(Distinct(10)); - var fwd = new NonReverseSource(src); - nm.HistorianDataSource = fwd; - ReadEvents(nm, node, max: 3, inboundCp: null, end: Early, start: Late).Error.StatusCode.Code - .ShouldBe(StatusCodes.BadHistoryOperationUnsupported); - fwd.ReverseCalls.ShouldBe(0, "the capability is checked BEFORE the source is called"); - fwd.ForwardCalls.ShouldBe(0); + // (EndTime, StartTime]: StartTime inclusive, EndTime exclusive. + var (host, nm, node, _) = await SetupAsync([]); + var gw = new GatewayShapedEventSource(Distinct(30)); + nm.HistorianDataSource = gw; + var ids = DrainAsync(nm, node, max: 4, start: Epoch.AddSeconds(25), end: Epoch.AddSeconds(5)); + ids.ShouldBe(Enumerable.Range(6, 20).Reverse().Select(i => $"e{i:0000}")); + gw.ReverseCalls.ShouldBe(0); + await host.DisposeAsync(); + } + + [Fact] + public async Task Reverse_ties_on_a_forward_only_backend_page_in_a_stable_order_although_the_backend_shuffles_them() + { + // The gateway orders events by EventTimeUtc only, so ties come back in any order — this fake shuffles them on every + // call. The walk sorts ties by EventId (descending, newest-first) on every read, and the oversized-cluster over-fetch + // does the same, so a boundary-tie skip always lines up. Cap 3 cuts through a 7-way cluster. + var events = new List { Ev("a0", 0) }; + for (var i = 0; i < 7; i++) events.Add(Ev($"t{i}", 5)); + events.Add(Ev("z0", 9)); + var (host, nm, node, _) = await SetupAsync([]); + nm.HistorianDataSource = new GatewayShapedEventSource(events, shuffleSeed: 7); + DrainAsync(nm, node, max: 3, start: DateTime.MinValue, end: Epoch.AddSeconds(30)).ShouldBe( + ["z0", "t6", "t5", "t4", "t3", "t2", "t1", "t0", "a0"]); + await host.DisposeAsync(); + } + + [Fact] + public async Task Uncapped_reverse_read_on_a_forward_only_backend_is_one_forward_read() + { + var (host, nm, node, _) = await SetupAsync([]); + var gw = new GatewayShapedEventSource(Distinct(30)); + nm.HistorianDataSource = gw; + var r = ReadEvents(nm, node, max: 0, inboundCp: null, end: Epoch.AddSeconds(5), start: Epoch.AddSeconds(25)); + r.Ids.ShouldBe(Enumerable.Range(6, 20).Reverse().Select(i => $"e{i:0000}")); + r.Cp.ShouldBeNull(); + gw.ForwardCalls.ShouldBe(1); + await host.DisposeAsync(); + } + + [Fact] + public async Task Reverse_paging_on_a_forward_only_backend_is_exact_on_random_data() + { + var (host, nm, node, _) = await SetupAsync([]); + for (var seed = 0; seed < 80; seed++) + { + var rng = new Random(seed); + var events = new List(); + var t = Epoch.Ticks; + var n = rng.Next(1, 120); + while (events.Count < n) + { + t += rng.Next(4) switch + { + 0 => rng.Next(1, 10), + 1 => rng.Next(1, 5) * TimeSpan.TicksPerMillisecond, + 2 => rng.Next(1, 90) * TimeSpan.TicksPerMinute, + _ => rng.Next(1, 2000) * TimeSpan.TicksPerMillisecond, + }; + var ties = rng.Next(5) == 0 ? rng.Next(2, 8) : 1; + for (var k = 0; k < ties; k++) + { + var at = new DateTime(t, DateTimeKind.Utc); + events.Add(new HistoricalEvent($"{Guid.NewGuid():N}", "src", at, at, "m", 500)); + } + } + + nm.HistorianDataSource = new GatewayShapedEventSource(events, shuffleSeed: seed); + nm.MaxTieClusterOverfetch = rng.Next(2) == 0 ? 8 : 65536; + var max = (uint)rng.Next(1, 9); + var end = new DateTime(t + rng.Next(0, 3) * TimeSpan.TicksPerSecond, DateTimeKind.Utc); + var expected = events.OrderByDescending(e => e.EventTimeUtc).ThenByDescending(e => e.EventId, StringComparer.Ordinal) + .Select(e => e.EventId).ToList(); + + var all = new List(); + byte[]? cp = null; + var pages = 0; + do + { + var r = ReadEvents(nm, node, max, cp, end, DateTime.MinValue); + r.Error.StatusCode.Code.ShouldBe(StatusCodes.Good, $"seed {seed}"); + all.AddRange(r.Ids); + cp = r.Cp; + (++pages).ShouldBeLessThan(2000); + } + while (cp is not null); + + all.ShouldBe(expected, $"seed {seed}, N = {max}"); + } + await host.DisposeAsync(); } @@ -391,29 +480,39 @@ public sealed class NodeManagerHistoryReadEventsPagingTests : IDisposable await host.DisposeAsync(); } - /// Declares the reverse overload (so a call would be visible) but reports no reverse capability. - private sealed class NonReverseSource(IHistorianDataSource inner) : IHistorianDataSource + /// + /// Shaped like the Gateway's SQL event read (#691.4): forward only ( false — a + /// reverse call is counted and refused), [start, end] INCLUSIVE, ordered by event time only with ties in an + /// arbitrary order (shuffled on every call when a seed is given), capped with a truncation flag. + /// + private sealed class GatewayShapedEventSource(List events, int? shuffleSeed = null) : IHistorianDataSource { + private readonly Random? _rng = shuffleSeed is { } s ? new Random(s) : null; public int ReverseCalls { get; private set; } public int ForwardCalls { get; private set; } + public int BadRequests { get; private set; } public bool SupportsReverseEvents => false; public Task ReadEventsAsync(string? n, DateTime s, DateTime e, int m, CancellationToken ct) { ForwardCalls++; - return inner.ReadEventsAsync(n, s, e, m, ct); + if (s > e || s < DateTime.UnixEpoch) BadRequests++; + var window = events.Where(x => x.EventTimeUtc >= s && x.EventTimeUtc <= e) + .OrderBy(x => x.EventTimeUtc).ThenBy(_ => _rng?.Next() ?? 0).ToList(); + var cap = m <= 0 ? int.MaxValue : m; + return Task.FromResult(new HistoricalEventsResult(window.Take(cap).ToList(), window.Count > cap ? new byte[] { 1 } : null)); } public Task ReadEventsAsync(string? n, DateTime s, DateTime e, int m, bool reverse, CancellationToken ct) { - if (reverse) { ReverseCalls++; return inner.ReadEventsAsync(n, s, e, m, true, ct); } + if (reverse) { ReverseCalls++; throw new NotSupportedException("forward only"); } return ReadEventsAsync(n, s, e, m, ct); } public Task ReadRawAsync(string r, DateTime s, DateTime e, uint m, CancellationToken ct) => - inner.ReadRawAsync(r, s, e, m, ct); + NullHistorianDataSource.Instance.ReadRawAsync(r, s, e, m, ct); public Task ReadProcessedAsync(string r, DateTime s, DateTime e, TimeSpan i, HistoryAggregateType a, CancellationToken ct) => - inner.ReadProcessedAsync(r, s, e, i, a, ct); + NullHistorianDataSource.Instance.ReadProcessedAsync(r, s, e, i, a, ct); public Task ReadAtTimeAsync(string r, IReadOnlyList t, CancellationToken ct) => - inner.ReadAtTimeAsync(r, t, ct); - public HistorianHealthSnapshot GetHealthSnapshot() => inner.GetHealthSnapshot(); + NullHistorianDataSource.Instance.ReadAtTimeAsync(r, t, ct); + public HistorianHealthSnapshot GetHealthSnapshot() => NullHistorianDataSource.Instance.GetHealthSnapshot(); public void Dispose() { } }