feat(historian-gateway): native TimeAverage/Total on by default, SendEvents batches, server-side max_events (#694, #746)
#694 item 3 — historiangw#27 is fixed and now live-verified through OtOpcUa: - ServerHistorian:Gateway:EnableNativeTimeWeighted defaults to true (false is the kill switch for a gateway older than 9f711c1). TimeAverage / Total are served and advertised again (12 aggregates). - The gateway sends the interval as max(1, round(ms)) and AVEVA clips its last interval at the request end (live). NativeAggregatePlan therefore sends a multi-interval grid only for whole-ms intervals; a fractional interval is read one interval per request (at most 1,000, else BadTooManyOperations), and every one-interval request goes out rounded UP to whole ms so AVEVA answers one clipped row. Live, a 3.0004 s clipped interval sent at its own width lost its last 0.4 ms (Total 123 vs 123.0172). #746 — HistorianGateway Contracts/Client 1.0.0 surface: - Alarm history: GatewayAlarmHistorianWriter sends SendEvents batches of <=200. A failed per-event result is re-sent alone via SendEvent (classified as before); a refused batch (InvalidArgument etc.) goes event by event so a poison event dead-letters alone; a systemic fault retries the rest without sending more; Unimplemented (older gateway) falls back to SendEvent for good. IHistorianGatewayClient.SendEventsAsync defaults to Unimplemented. - Events reads: a capped page sends cap + 1 as ReadEventsRequest.max_events, so the gateway completes the stream instead of the client cancelling it. An ambiguous capped answer (foreign rows filtered out of the server's cut) or a refused cap is re-read uncapped with the existing EventDrainBudget drain. Tests: GatewayServerCappedEventsTests, GatewayAlarmWriterBatchTests, GatewayNativeWholeMillisecondGridTests (gateway-faithful rounding fake), gate tests flipped to the new default; every new test mutation-checked (each mutation of the production change fails at least one). Live harness AvevaGatewayHistorianFollowupsLiveGateTests (read-only) for #694/#700/#746. Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
@@ -50,7 +50,7 @@ historian tagname, which is the tag's driver `FullName`.
|
||||
The server dispatches all OPC UA HistoryRead to the registered `IHistorianDataSource` — the
|
||||
`GatewayHistorianDataSource` read client when enabled, else the `NullHistorianDataSource` default
|
||||
(historized nodes return `GoodNoData` for Raw/AtTime — Processed is `BadAggregateNotSupported` since #682). Supported variants: Raw (⚠️ #687, client-visible: Gateway raw reads no longer return AVEVA's synthetic start-boundary row — asked from 1 ms early (epoch-floored) with `NumValuesPerNode + 16` rows (unclamped; 0 ⇒ `int.MaxValue`) and filtered by timestamp — so an empty window is `GoodNoData`, not a fabricated value at StartTime; the forward end is now exclusive (a point read inclusive); pre-window rows that crowd the budget trigger ONE re-read with room for them, else `BadTooManyOperations`; `ReturnBounds` is ignored on every backend; ⚠️ #690: a Gateway Bad row — a stored NULL, which histsdk sends as value 0 — now reads value **null** + Bad on Raw and AtTime, as on the SQL backends; ⚠️ #691.3: a strict gateway (`AllowStreamingBeyondCeiling=false`) that refuses a raw budget is re-asked ONCE at the ceiling its refusal names — raw and per-sample reads alike; the ceiling is then remembered per data source (`GatewayRowCeiling`: refusals count as gateway errors), re-learned from a refusal when lowered, and re-probed once at the full budget when a ceiling-bounded read comes back full (raised / streaming) — so an uncapped (`NumValuesPerNode = 0`) read of a small window is served; a ceiling-bounded stream that is not provably complete is `BadTooManyOperations`, never a truncated answer), Processed
|
||||
(backend-dependent: SqlServer/QuestDb serve nineteen; Gateway twelve = `GatewayHistorianDataSource.ServedAggregates` — ⚠️ #681, client-visible: the ten per-sample ones (Average, Count, Min/Max, Range, Start/End/Delta, Min/MaxActualTime) are evaluated from RAW gateway reads by the shared `Sql.Common` evaluator — identical to the SQL backends, Calculated bits, interval-start stamps, #677 reverse; Min/Max are no longer AVEVA modes, the read moves every raw sample of the window (a month of 1 Hz ⇒ `BadTooManyOperations` at the default `MaxTimeWeightedRawRows` cap), and every gateway stream is consumed to completion (an abandoned one makes the gateway evict its shared historian session) — plus TimeAverage/Total via AVEVA modes (interval-start stamped + Calculated; ⚠️ #685: AVEVA's extra pre-window row is dropped ⇒ one row per interval in `[Start, End)`, and an Uncertain (opc 64) interval keeps its value as `UncertainDataSubNormal` — AVEVA's TWA excludes NULL-gap time where the SQL backends bridge it; per-interval values still need gateway fix historiangw#27; ⚠️ #691.2: native reads use the SQL backends' grid in both directions via forward requests only — a REVERSE read (refused by the gateway before) is anchored at its later time (#677), and the clipped interval (earliest reverse, LATEST forward — which used to be averaged past EndTime) is its own exact-span read marked Partial unless Bad; an interval wider than the window is sent as the window, interval 0 as one interval; more than 100,000 intervals is refused up front (`BadHistoryOperationUnsupported`, as on the SQL backends — forward used to be served up to the gateway's 1,000,000); until historiangw#27 lands both directions return Good-status wrong values); the other time-weighted five stay `BadAggregateNotSupported` because the gateway has no prior-sample read for Part 13 bounds; `docs/Historian.md` §Aggregate set), and AtTime over historized variable nodes (⚠️ #693, client-visible: Gateway AtTime is evaluated from RAW reads by the shared `Sql.Common` `AtTimeEvaluator` — exact stored value at a stored time, null + Bad at/after a stored NULL, Uncertain hold before a Bad neighbour, linear + Interpolated bit, `BadNoData` before the first sample — never AVEVA's own interpolation (which answered 107 Good at a NULL, null+Bad at a Good sample after it, 99.99999999944 for 100, no Interpolated bit); `GatewayAtTimeReader`: one ≤1,024-row window read for the batch (jumps to far-apart times use 64-row reads) + a walk-back for the first time's prev (gallop ×4, whole-remainder read after two empties, then real BISECTION seeded by any full read's last row — a dense tag stopped a year ago ≈ 25–31 reads; the first cut regrew segments after a full read and gave up at 65) + an 18-row probe for the last time's next — 10 dense times = 1 read (exact) / 3 (interpolated), 10 hourly = 12; read cap 64 + 16 per distinct time (a runaway guard: a time in a gap of a bursty tag needs ≈ 6 reads; 4/time refused a 1 h/day tag asked daily ×30) and a strict ceiling < 4 rows ⇒ `BadTooManyOperations` with a message naming the limit; DateTime hints stepped, non-finite/out-of-range values under an integer hint unusable; all request bounds whole ms, rows counted only strictly inside a request; the node manager passes the node's scalar type (`ReadAtTimeAsync(…, TypeCode? valueType, …)` default-interface overload, ignored by SqlServer/QuestDB/Null) and the Gateway converts raw rows to it before evaluating — Boolean stepped, integers interpolated + rounded half-away and returned as the node's integer type, floats Double; no hint ⇒ Double; ⚠️ the shared evaluator used to box interpolated Int64 as Double (`cond ? (long)x : double` widens) — fixed for SQL backends too, and Shouldly `ShouldBe` hides numeric-type bugs (assert `ShouldBeOfType`); kill switch `ServerHistorian:GatewayAtTimeFromRaw` (default `true`); `docs/Historian.md` §Gateway AtTime); Events over
|
||||
(backend-dependent: SqlServer/QuestDb serve nineteen; Gateway twelve = `GatewayHistorianDataSource.ServedAggregates` — ⚠️ #681, client-visible: the ten per-sample ones (Average, Count, Min/Max, Range, Start/End/Delta, Min/MaxActualTime) are evaluated from RAW gateway reads by the shared `Sql.Common` evaluator — identical to the SQL backends, Calculated bits, interval-start stamps, #677 reverse; Min/Max are no longer AVEVA modes, the read moves every raw sample of the window (a month of 1 Hz ⇒ `BadTooManyOperations` at the default `MaxTimeWeightedRawRows` cap), and every gateway stream is consumed to completion (an abandoned one makes the gateway evict its shared historian session) — plus TimeAverage/Total via AVEVA modes (interval-start stamped + Calculated; ⚠️ #685: AVEVA's extra pre-window row is dropped ⇒ one row per interval in `[Start, End)`, and an Uncertain (opc 64) interval keeps its value as `UncertainDataSubNormal` — AVEVA's TWA excludes NULL-gap time where the SQL backends bridge it; per-interval values live-verified after gateway fix historiangw#27 (#694 — on by default again, `Gateway:EnableNativeTimeWeighted=false` is the kill switch; a non-whole-ms interval is read one interval per request, a one-interval request is sent rounded UP to whole ms so AVEVA answers one clipped row); ⚠️ #691.2: native reads use the SQL backends' grid in both directions via forward requests only — a REVERSE read (refused by the gateway before) is anchored at its later time (#677), and the clipped interval (earliest reverse, LATEST forward — which used to be averaged past EndTime) is its own exact-span read marked Partial unless Bad; an interval wider than the window is sent as the window, interval 0 as one interval; more than 100,000 intervals is refused up front (`BadHistoryOperationUnsupported`, as on the SQL backends — forward used to be served up to the gateway's 1,000,000); until historiangw#27 lands both directions return Good-status wrong values); the other time-weighted five stay `BadAggregateNotSupported` because the gateway has no prior-sample read for Part 13 bounds; `docs/Historian.md` §Aggregate set), and AtTime over historized variable nodes (⚠️ #693, client-visible: Gateway AtTime is evaluated from RAW reads by the shared `Sql.Common` `AtTimeEvaluator` — exact stored value at a stored time, null + Bad at/after a stored NULL, Uncertain hold before a Bad neighbour, linear + Interpolated bit, `BadNoData` before the first sample — never AVEVA's own interpolation (which answered 107 Good at a NULL, null+Bad at a Good sample after it, 99.99999999944 for 100, no Interpolated bit); `GatewayAtTimeReader`: one ≤1,024-row window read for the batch (jumps to far-apart times use 64-row reads) + a walk-back for the first time's prev (gallop ×4, whole-remainder read after two empties, then real BISECTION seeded by any full read's last row — a dense tag stopped a year ago ≈ 25–31 reads; the first cut regrew segments after a full read and gave up at 65) + an 18-row probe for the last time's next — 10 dense times = 1 read (exact) / 3 (interpolated), 10 hourly = 12; read cap 64 + 16 per distinct time (a runaway guard: a time in a gap of a bursty tag needs ≈ 6 reads; 4/time refused a 1 h/day tag asked daily ×30) and a strict ceiling < 4 rows ⇒ `BadTooManyOperations` with a message naming the limit; DateTime hints stepped, non-finite/out-of-range values under an integer hint unusable; all request bounds whole ms, rows counted only strictly inside a request; the node manager passes the node's scalar type (`ReadAtTimeAsync(…, TypeCode? valueType, …)` default-interface overload, ignored by SqlServer/QuestDB/Null) and the Gateway converts raw rows to it before evaluating — Boolean stepped, integers interpolated + rounded half-away and returned as the node's integer type, floats Double; no hint ⇒ Double; ⚠️ the shared evaluator used to box interpolated Int64 as Double (`cond ? (long)x : double` widens) — fixed for SQL backends too, and Shouldly `ShouldBe` hides numeric-type bugs (assert `ShouldBeOfType`); kill switch `ServerHistorian:GatewayAtTimeFromRaw` (default `true`); `docs/Historian.md` §Gateway AtTime); Events over
|
||||
alarm-owning equipment-folder event-notifier nodes. Reads are ungated (served from any redundancy node);
|
||||
authorization uses the standard `AccessLevels.HistoryRead` bit set at materialization.
|
||||
|
||||
@@ -76,7 +76,7 @@ authorization uses the standard `AccessLevels.HistoryRead` bit set at materializ
|
||||
|
||||
### Alarm-history path (`AlarmHistorian` section)
|
||||
|
||||
Alarm events are written through `GatewayAlarmHistorianWriter` (the gateway **`SendEvent`** path) behind
|
||||
Alarm events are written through `GatewayAlarmHistorianWriter` (gateway **`SendEvents`** batches of ≤200 since #746; `SendEvent` per event for a failed event, a refused batch or an older gateway) behind
|
||||
the durable **`LocalDbStoreAndForwardSink`** — `AlarmHistorian:Enabled=true` swaps the `NullAlarmHistorianSink`
|
||||
default for the store-and-forward queue, which buffers into the node's consolidated LocalDb as the
|
||||
replicated `alarm_sf_events` table (see the LocalDb section) and whose drain worker forwards batches to
|
||||
@@ -186,6 +186,13 @@ dotnet test --filter "Category=LiveIntegration"
|
||||
The live suite **skips cleanly** when these env vars are absent (safe to run offline on macOS). It is the
|
||||
gate the operator runs on the VPN before trusting the cutover.
|
||||
|
||||
⚠️ That `LiveIntegration` suite **writes** (sandbox tag, `SendEvent`). The **read-only** HistoryRead gates — run these
|
||||
when historian writes are not allowed — are `AvevaGatewayHistoryLiveGateTests` (#801) and
|
||||
`AvevaGatewayHistorianFollowupsLiveGateTests` (#694/#700/#746: native TimeAverage/Total vs a step reference, typed values,
|
||||
outage statuses, server-capped event paging, and a strict-ceiling fixture) in `OpcUaServer.IntegrationTests`. A local
|
||||
read-only gateway is enough: publish HistorianGateway `main` and run it with `ApiKeys__Mode=Disabled`,
|
||||
`Historian__LiveWritesEnabled=false` (recipe + evidence: `docs/plans/evidence/2026-11-12-historian-694-700-746/aveva-gate.md`).
|
||||
|
||||
### Continuous-historization value capture — wired and live-gated (was "KNOWN LIMITATION 2"; #491 CLOSED 2026-08-04)
|
||||
|
||||
The `ContinuousHistorizationRecorder` is fully wired (actor + FasterLog outbox + gateway value-writer +
|
||||
|
||||
@@ -53,7 +53,9 @@ unless noted.
|
||||
- **`IAlarmHistorianWriter`** — what the drain worker delegates writes to.
|
||||
`WriteBatchAsync(batch, ct)` returns one `HistorianWriteOutcome` per event,
|
||||
in order. Production binds this to `GatewayAlarmHistorianWriter` (the
|
||||
HistorianGateway `SendEvent` path).
|
||||
HistorianGateway `SendEvents` batch path — up to 200 events per RPC, #746 —
|
||||
with `SendEvent` as the per-event fallback for a failed event, a refused
|
||||
batch, or a gateway without the batch RPC).
|
||||
- **`HistorianWriteOutcome`** — per-event drain result: `Ack` (persisted,
|
||||
remove from queue), `RetryPlease` (transient failure — leave queued, retry
|
||||
after backoff), `PermanentFail` (malformed/unrecoverable — move to
|
||||
@@ -241,7 +243,7 @@ writes across a redundant pair. `AlarmTransitionEvent` carries `AlarmTypeName`
|
||||
(the Part 9 subtype string) and `Comment` (the operator comment from the
|
||||
originating ack/shelve command) that populate the corresponding fields of
|
||||
`AlarmHistorianEvent`. `GatewayAlarmHistorianWriter` is the `IAlarmHistorianWriter`
|
||||
the drain worker delegates to (the gateway `SendEvent` path). See
|
||||
the drain worker delegates to (gateway `SendEvents` batches, `SendEvent` fallback). See
|
||||
[ServiceHosting.md](ServiceHosting.md) for the (external) HistorianGateway setup.
|
||||
|
||||
**Scope:** scripted alarms only. Galaxy-native alarms historize via System
|
||||
@@ -285,7 +287,7 @@ node's consolidated `LocalDb:Path` database (see
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|---|---|---|---|
|
||||
| `Enabled` | bool | `false` | Enable the durable store-and-forward sink (drains to the HistorianGateway `SendEvent` path). `false` → `NullAlarmHistorianSink`. Requires LocalDb to be registered — an enabled sink with no `ILocalDb` fails at resolution rather than silently discarding events. |
|
||||
| `Enabled` | bool | `false` | Enable the durable store-and-forward sink (drains to the HistorianGateway `SendEvents` / `SendEvent` path). `false` → `NullAlarmHistorianSink`. Requires LocalDb to be registered — an enabled sink with no `ILocalDb` fails at resolution rather than silently discarding events. |
|
||||
| `BatchSize` | int | `100` | Max rows per drain cycle handed to `IAlarmHistorianWriter.WriteBatchAsync`. |
|
||||
| `DrainIntervalSeconds` | int | `5` | Seconds between drain-worker ticks. |
|
||||
| `Capacity` | long | `1000000` | Max queued rows before the sink evicts the oldest (data-loss signal via `EvictedCount`). |
|
||||
|
||||
+40
-27
@@ -108,14 +108,14 @@ consumes the historian. The existing enable gates (`ServerHistorian:Enabled`, `A
|
||||
| Value kinds recorded | `Double`, `Int64`, `Boolean` (numeric analog only) | all: `Double`, `Int64`, `Boolean`, `String`, `DateTime`, plus `Empty` (a Bad sample with no value) | same as `SqlServer` |
|
||||
| Status code stored | Sent as DA quality but **discarded** by the gateway (C-002); AVEVA stores every sample Good | Stored bit-for-bit (`StatusCode` int column) | Stored bit-for-bit (unsigned, in a `LONG`) |
|
||||
| Comms-loss / driver-Bad publishes | Dropped, counted in `DroppedUnsupportedKind` | Stored as an `Empty` outage-marker sample carrying the Bad code (see [below](#typed-values-real-status-and-outage-markers)) | Stored as an `Empty` outage-marker sample, as on `SqlServer` |
|
||||
| Processed aggregates | **Ten** by default: the per-sample ones (Average, Count, Minimum, Maximum, Range, Start, End, Delta, Min/MaxActualTime) evaluated from raw reads with the SQL backends' evaluator — identical results (#681). TimeAverage and Total from AVEVA's native modes **only with `ServerHistorian:Gateway:EnableNativeTimeWeighted=true`** (default off: Good-status wrong values until historiangw#27). TimeAverage2 / Total2 / Interpolative / PercentGood / PercentBad / DurationGood / DurationBad: never | The nineteen in the [aggregate table](#aggregates-processed-reads) (incl. Part 13 time-weighted, #671, and DurationGood/DurationBad, #820) | The same nineteen, identical results |
|
||||
| Processed aggregates | **Twelve** by default: the per-sample ones (Average, Count, Minimum, Maximum, Range, Start, End, Delta, Min/MaxActualTime) evaluated from raw reads with the SQL backends' evaluator — identical results (#681) — plus TimeAverage and Total from AVEVA's native modes (on by default again since #694, after historiangw#27 was live-verified; `ServerHistorian:Gateway:EnableNativeTimeWeighted=false` is the kill switch). TimeAverage2 / Total2 / Interpolative / PercentGood / PercentBad / DurationGood / DurationBad: never | The nineteen in the [aggregate table](#aggregates-processed-reads) (incl. Part 13 time-weighted, #671, and DurationGood/DurationBad, #820) | The same nineteen, identical results |
|
||||
| Reverse / end-only Raw and Events reads | Emulated: walked back from bounded forward reads by the server (`ReverseRawReader` / `ReverseEventsReader`) — correct, but many round trips on a sparse tag | Native newest-first | Native newest-first |
|
||||
| Capped Events read | The cap is applied client-side (the gateway RPC has no max-events field); the stream past it is drained up to `Gateway:EventDrainBudget` rows | `TOP` | `LIMIT` |
|
||||
| Capped Events read | Server-side: cap + 1 sent as `ReadEventsRequest.max_events` (#746); the gateway completes the stream at it. Uncapped fallback (drained up to `Gateway:EventDrainBudget` rows) only for an ambiguous or refused cap | `TOP` | `LIMIT` |
|
||||
| Retention | AVEVA Historian's own | `SqlServer:RetentionDays` purge (opt-in) | `QuestDb:RetentionDays` TTL (opt-in) |
|
||||
| Health connection flags | `Gateway:ProbeInterval` probe + read outcomes | Read outcomes | Read outcomes |
|
||||
| HistoryUpdate | No (`BadHistoryOperationUnsupported`) | Yes (Insert / Replace / Update / DeleteRaw / DeleteAtTime) | Yes — soft delete (tombstones), see [below](#historyupdate-soft-delete) |
|
||||
| Modified-value history (`IsReadModified`, #668) | No (`BadHistoryOperationUnsupported`) | Yes — `SampleModified`, atomic with the change | Yes — `{prefix}sample_modified`, written before the change, marked applied after it (an orphan is not served) |
|
||||
| Alarm history | gateway `SendEvent`, one RPC per event, up to 8 in flight (no batch RPC yet) | `Event` table (insert-if-absent), multi-row statements of 150 events | `{prefix}event` table (dedup upsert on `(ts, event_id)`), multi-row INSERTs of 500 events |
|
||||
| Alarm history | gateway `SendEvents` batches of up to 200 events (#746); a failed event, a refused batch or a gateway without the RPC falls back to `SendEvent`, up to 8 in flight | `Event` table (insert-if-absent), multi-row statements of 150 events | `{prefix}event` table (dedup upsert on `(ts, event_id)`), multi-row INSERTs of 500 events |
|
||||
| Provisioning (`EnsureTags`) | numeric types only; others skipped | `MERGE` upsert of every request, all types | Upsert into `{prefix}tag`, all types |
|
||||
| Connection | `Endpoint` + `ApiKey` + TLS keys (top-level `ServerHistorian:*`) | `ServerHistorian:SqlServer:*` | `ServerHistorian:QuestDb:*` |
|
||||
|
||||
@@ -200,9 +200,9 @@ Processed read is `BadAggregateNotSupported` and no history aggregate functions
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|---|---|---|---|
|
||||
| `Gateway:EnableNativeTimeWeighted` | bool | `false` | Serve HistoryRead-Processed `TimeAverage` / `Total` through AVEVA's native retrieval modes. ⚠️ **Off by default (client-visible):** the gateway sends AVEVA the interval in the wrong unit (historiangw#27), so AVEVA answers one Good-status whole-span value on the earliest interval and nothing for the rest — wrong values a client cannot tell from right ones. Off, both are `BadAggregateNotSupported` and are **not advertised** (`AggregateFunctions`), like the other five time-weighted aggregates on this backend. Turn it on only after a fixed gateway is live-verified. |
|
||||
| `Gateway:EnableNativeTimeWeighted` | bool | `true` | Serve HistoryRead-Processed `TimeAverage` / `Total` through AVEVA's native retrieval modes. ⚠️ **On by default since #694 (client-visible: both are advertised and served again).** The gateway interval fix historiangw#27 (gateway server `9f711c1`+) is live-verified: forward and reverse per-interval values, clipped, sub-millisecond clipped and fractional-millisecond intervals against a step-held reference, and all-NULL intervals ⇒ `BadNoData` ([evidence](plans/evidence/2026-11-12-historian-694-700-746/aveva-gate.md)). `false` is the kill switch for a gateway older than the fix (it answers one Good-status whole-span value on the earliest interval): both are then `BadAggregateNotSupported` and **not advertised**. On a discrete (Boolean) AVEVA tag AVEVA refuses both modes and the node answers `BadHistoryOperationUnsupported` (counted as a backend failure). |
|
||||
| `Gateway:ProbeInterval` | TimeSpan | `00:00:30` | How often the read client calls the gateway's `ProbeAsync` for the historian health snapshot (`ProcessConnectionOpen` / `EventConnectionOpen`, shown on `/hosts`). The first probe runs at start-up, so a node that has served no HistoryRead yet — or only writes (continuous historization) — reports a healthy gateway as open; it used to read closed until some client happened to read. The latest observation (probe or read outcome) wins; probes never count as queries or failures. Zero/negative = no probe (read outcomes only, the previous behaviour). |
|
||||
| `Gateway:EventDrainBudget` | int | `10000` | Stream rows a capped HistoryReadEvents read consumes (and discards) past its page cap before it abandons the gateway stream. The gateway's `ReadEvents` has no max-events field, so the cap is applied client-side; an **abandoned** stream cancels the RPC and the gateway evicts its pooled historian session (the next read reconnects; concurrent reads on that session fail). The usual small overshoot is drained to the stream's end; past the budget the stream is abandoned, so a page over a window far larger than the page does not stream the whole window. Rows of another source never count before the "more events" signal is seen, so an abandoned stream never answers a page as complete. `0` = abandon at the cap (the previous behaviour); negative counts as `0` (a `Validate()` warning). |
|
||||
| `Gateway:EventDrainBudget` | int | `10000` | Since #746 a capped HistoryReadEvents read sends cap + 1 as the server-side `max_events` and the gateway completes the stream there, so this budget applies only to the uncapped fallback read (an ambiguous server-capped answer — foreign rows filtered out of a stream the server cut — or a gateway refusing the cap) and to a stream that runs past the `max_events` it was sent. It is the number of stream rows such a read consumes (and discards) past its page cap before it abandons the gateway stream; an **abandoned** stream cancels the RPC and the gateway evicts its pooled historian session (the next read reconnects; concurrent reads on that session fail). The usual small overshoot is drained to the stream's end; past the budget the stream is abandoned, so a page over a window far larger than the page does not stream the whole window. Rows of another source never count before the "more events" signal is seen, so an abandoned stream never answers a page as complete. `0` = abandon at the cap (the previous behaviour); negative counts as `0` (a `Validate()` warning). |
|
||||
|
||||
**`ServerHistorian:SqlServer`** (read only when `Backend=SqlServer`; bound by `HistorianBackendRegistry.BindSqlServer`):
|
||||
|
||||
@@ -1189,10 +1189,16 @@ says is complete carries no continuation point. `NumValuesPerNode == 0` stays un
|
||||
- **Tie exactness depends on the backend.** SqlServer and QuestDB order by `(EventTimeUtc, EventId)`, so the skip count is exact.
|
||||
The Gateway backend has no stable intra-timestamp order (and only a truncation flag), so ties at a page boundary of a
|
||||
**forward** read are best-effort there; a walked-back **reverse** read (below) sorts them by `EventId` itself and is exact.
|
||||
- **Gateway streams are drained, not abandoned.** The gateway's `ReadEvents` carries no max-events field, so a capped page is cut
|
||||
client-side; the rest of the stream is consumed (uncollected) up to `ServerHistorian:Gateway:EventDrainBudget` rows (default
|
||||
10 000) so the gateway does not evict its historian session for a cancelled stream. A page over a window much larger than the
|
||||
budget still abandons its stream. A caller cancellation (client abort, `HistoryReadDeadline`) is not counted as a backend failure.
|
||||
- **Gateway streams complete, they are not abandoned (#746).** A capped page sends cap + 1 as `ReadEventsRequest.max_events`
|
||||
(HistorianGateway Contracts/Client 0.4.0+): the gateway stops there and completes the stream normally, so it never evicts its
|
||||
historian session for a cancelled stream, and the (cap + 1)-th event is the "more events" signal. Live: 1-, 7- and reverse 5-event
|
||||
pages over 72 `DevWinPlatform` events with `EventDrainBudget = 0` abandoned **0** streams; the same run with the cap not sent
|
||||
abandoned 81 ([evidence](plans/evidence/2026-11-12-historian-694-700-746/aveva-gate.md)). If the server cut the stream at cap + 1 but the client-side source filter dropped some of those rows (so
|
||||
whether more matching events exist is unknown), or the gateway refuses the cap (`InvalidArgument`, a `MaxEventsCeiling` below
|
||||
it), the page is re-read without a server cap and drained as before: the rest of the stream is consumed (uncollected) up to
|
||||
`ServerHistorian:Gateway:EventDrainBudget` rows (default 10 000), past which the stream is abandoned. A gateway older than
|
||||
`max_events` ignores it and the package client stops at cap + 1 itself (which cancels the stream). A caller cancellation
|
||||
(client abort, `HistoryReadDeadline`) is not counted as a backend failure.
|
||||
- **Reverse event reads page newest-first (#675).** `StartTime > EndTime` reads the Part 11 reverse window `(EndTime, StartTime]`
|
||||
with events in DESCENDING `(EventTimeUtc, EventId)` order, so a cap keeps the **latest** N and a continuation point is issued
|
||||
when the source flags more. The direction is stored **in the cursor**: a reverse point resumes reverse (and a forward one
|
||||
@@ -1354,12 +1360,12 @@ fixed:
|
||||
bounding — 6.08 Uncertain for the same samples, see [Time-weighted aggregates](#time-weighted-aggregates-part-13-671)).
|
||||
Extrapolation past the last sample is `Good` on AVEVA and `Uncertain` on the SQL backends. On all-Good spans the magnitudes
|
||||
match exactly (both sloped/trapezoidal).
|
||||
- ⚠️ **End-to-end correctness also needs the gateway fix historiangw#27:** the gateway sends the aggregate interval to AVEVA in
|
||||
ticks where AVEVA reads milliseconds (10,000× too large), so today a native read returns one row for the whole window (plus the
|
||||
pre-window row, now dropped). OtOpcUa does not work around it; per-interval values return once the gateway is fixed.
|
||||
- **Not verified live:** what AVEVA answers for an interval holding **only** NULL samples (no good time at all). If it marks such a
|
||||
row Uncertain (`opc_quality` 64–191) with a placeholder value, that value is now passed through as Uncertain; the gate only
|
||||
observed Uncertain rows that also held good time. Re-check on the VPN once historiangw#27 is fixed.
|
||||
- **End-to-end correctness needed the gateway fix historiangw#27** (the interval reached AVEVA in ticks where it reads
|
||||
milliseconds, so a native read returned one row for the whole window). Fixed on the gateway (`9f711c1`) and live-verified through
|
||||
OtOpcUa in #694 ([evidence](plans/evidence/2026-11-12-historian-694-700-746/aveva-gate.md)): per-interval values in both directions match a step-held integral of the tag's own raw rows exactly.
|
||||
- **All-NULL intervals (verified live, #694).** An interval holding no good time at all (a float tag stored NULL at 11:32:36 with
|
||||
nothing until 11:38:34) is answered by AVEVA `opc_quality = 0` with NaN, mapped `BadNoData | Calculated` (`0x809B0401`), null
|
||||
value; the intervals that hold the NULL *and* some good time are Uncertain (`0x40A40401`) with AVEVA's value.
|
||||
|
||||
**Integral unit (#677).** AVEVA's integral is `∫ value dt` (seconds) ÷ the tag's `IntegralDivisor`. The proto3 `double` default for
|
||||
`HistorianTagDefinition.IntegralDivisor` is `0`, so `GatewayTagProvisioner` now sends **`IntegralDivisor = 1`** explicitly on every
|
||||
@@ -1384,13 +1390,20 @@ gateway refuses `start > end` and `interval <= 0`):
|
||||
(#685). An interval AVEVA answers no row for is absent (the SQL backends answer it `BadNoData`).
|
||||
- ⚠️ **Interval cap (client-visible):** forward native reads now go through `BucketPlan` too, so a window of more than 100,000
|
||||
intervals (e.g. 1 s over 2 days) is refused up front with `BadHistoryOperationUnsupported`, as on the SQL backends — it used to be
|
||||
served up to the gateway's own `MaxAggregateIntervals` (1,000,000). Not counted as a backend failure. A clipped interval shorter than
|
||||
1 ms (a sub-ms window end) is sent as a sub-ms interval request; re-check it in the post-historiangw#27 live leg.
|
||||
- ⚠️ **Native TimeAverage / Total are off by default** (`ServerHistorian:Gateway:EnableNativeTimeWeighted=false` ⇒
|
||||
`BadAggregateNotSupported`, unsent, unadvertised) because of the next point; everything in this section describes the opted-in path.
|
||||
- ⚠️ **Until historiangw#27 is fixed, native values are wrong in BOTH directions with a Good status:** AVEVA answers one row per request
|
||||
(the whole span's value), which lands on that request's first interval as a Good, Calculated value while the other intervals are
|
||||
absent. Pinned against a native-mode fake only — re-run the forward and reverse native legs live once the gateway is fixed. Since #681 every native bucket is stamped with its interval **start**
|
||||
served up to the gateway's own `MaxAggregateIntervals` (1,000,000). Not counted as a backend failure.
|
||||
- ⚠️ **Whole milliseconds on the wire (#694, client-visible).** The gateway sends AVEVA the interval as `max(1, round(ms))` and AVEVA
|
||||
clips its last interval at the request's end (both observed live). So: (1) a grid of several intervals is one request only when the
|
||||
interval is a whole number of milliseconds — a fractional one (e.g. `ProcessingInterval = 1.5`) is read **one interval per
|
||||
request**, at most 1,000 intervals (more ⇒ `BadTooManyOperations`, not a backend failure); before, AVEVA answered it on the rounded
|
||||
grid and the buckets were misplaced. (2) A request for one interval — the clipped one, a window narrower than one interval, each
|
||||
per-interval request — is sent with its width rounded **up** to whole milliseconds, so AVEVA answers one row clipped at the
|
||||
request's end: sent at its own width, a 3.0004 s clipped interval reached AVEVA as 3,000 ms and its Total lost the last 0.4 ms
|
||||
(123 instead of 123.0172 live). A sub-millisecond clipped interval is exact (live: 0.4 ms at value 40 ⇒ Total 0.016).
|
||||
- **Native TimeAverage / Total are on by default** since #694 (`ServerHistorian:Gateway:EnableNativeTimeWeighted=false` ⇒
|
||||
`BadAggregateNotSupported`, unsent, unadvertised — the kill switch for a gateway older than historiangw#27).
|
||||
- **Live-verified (#694, [evidence](plans/evidence/2026-11-12-historian-694-700-746/aveva-gate.md)):** forward and reverse, whole, clipped, sub-ms clipped and fractional-ms intervals on `SysTimeSec`
|
||||
(an integer tag AVEVA integrates as a step) equal a step-held integral of its raw rows, stamps and Partial bits included. Before
|
||||
the gateway fix AVEVA answered one row per request (the whole span's value) on the first interval. Since #681 every native bucket is stamped with its interval **start**
|
||||
(Part 13; it was the interval end) and carries the Calculated info bits (`Good` ⇒ `0x00000401`, no data ⇒ `BadNoData | 0x401`).
|
||||
The live AVEVA gate (2026-10-12) confirmed `start_time` is the interval start (and the `value·s` unit). Previously noted as
|
||||
pending: it reads the aggregate row's `start_time`, which histsdk decodes from a row
|
||||
@@ -1409,12 +1422,12 @@ field (`tail + 43`) its own reverse-engineering notes still mark unverified (His
|
||||
> statuses may differ from AVEVA's own modes; every per-sample result now carries the Calculated info bits; a per-sample read is
|
||||
> bounded by **`ServerHistorian:MaxTimeWeightedRawRows`** (in-window raw rows; past it ⇒ `BadTooManyOperations` for that node — read
|
||||
> a long window over a dense tag in shorter windows). `TimeAverage` and `Total` stay AVEVA-native (`TimeWeightedAverage` /
|
||||
> `Integral`) — ⚠️ **and are refused `BadAggregateNotSupported` unless `ServerHistorian:Gateway:EnableNativeTimeWeighted` is
|
||||
> `true`** (default `false`, until historiangw#27 is live-verified; the default set is the ten raw-evaluated aggregates,
|
||||
> `Integral`) — served by default since #694 (refused `BadAggregateNotSupported` with the
|
||||
> `ServerHistorian:Gateway:EnableNativeTimeWeighted=false` kill switch, which leaves the ten raw-evaluated aggregates,
|
||||
> `GatewayHistorianDataSource.RawEvaluatedAggregates`) — now stamped with the interval start (was its end) and carrying the Calculated bit; `TimeAverage2`, `Total2`,
|
||||
> `Interpolative`, `PercentGood`, `PercentBad`, `DurationGood` and `DurationBad` remain `BadAggregateNotSupported` on Gateway. The most the backend can serve is
|
||||
> `GatewayHistorianDataSource.ServedAggregates` (12); an instance serves and advertises `SupportedAggregates` — those 12 with the
|
||||
> flag on, the 10 raw-evaluated ones without it.
|
||||
> `GatewayHistorianDataSource.ServedAggregates` (12); an instance serves and advertises `SupportedAggregates` — those 12 by
|
||||
> default, the 10 raw-evaluated ones with the kill switch.
|
||||
|
||||
**How the Gateway raw evaluation works (#681).** Forward `ReadRaw` reads only (the gateway streams oldest-first and refuses
|
||||
`end < start`; `BucketPlan` assembles a reverse result newest-first), O(buckets) memory, and **every gateway stream is consumed to
|
||||
@@ -2681,7 +2694,7 @@ phases and are recorded here so future audits don't re-flag them.
|
||||
## See also
|
||||
|
||||
- [docs/plans/2026-06-14-galaxy-phase-c-historian-design.md](plans/2026-06-14-galaxy-phase-c-historian-design.md) — full design and implementation notes
|
||||
- [AlarmHistorian.md](AlarmHistorian.md) — alarm write path; drains to the same HistorianGateway (`SendEvent`)
|
||||
- [AlarmHistorian.md](AlarmHistorian.md) — alarm write path; drains to the same HistorianGateway (`SendEvents` batches, `SendEvent` fallback)
|
||||
- [AlarmTracking.md](AlarmTracking.md) — OPC UA Part 9 alarm surface (event history source)
|
||||
- [Client.CLI.md](Client.CLI.md) — full `historyread` flag reference
|
||||
- [ScriptedAlarms.md](ScriptedAlarms.md) §"Native driver alarms" — the Phase B `alarm` object in `TagConfig` (parallel carrier)
|
||||
|
||||
+158
-24
@@ -8,20 +8,35 @@ using ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Mapping;
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway;
|
||||
|
||||
/// <summary>
|
||||
/// <see cref="IAlarmHistorianWriter"/> backed by the HistorianGateway <c>SendEvent</c> path. The
|
||||
/// drain worker behind <c>LocalDbStoreAndForwardSink</c> calls
|
||||
/// <see cref="WriteBatchAsync"/> and uses the returned per-event
|
||||
/// <see cref="HistorianWriteOutcome"/> to decide retry vs. dead-letter, so this writer maps every
|
||||
/// gateway result — success ack, the published client's typed exception hierarchy, raw
|
||||
/// <see cref="RpcException"/> (defensive), and any unexpected error — onto exactly one outcome per
|
||||
/// event and <b>never throws</b>.
|
||||
/// <see cref="IAlarmHistorianWriter"/> backed by the HistorianGateway <c>SendEvents</c> batch RPC (Contracts/Client
|
||||
/// 0.4.0+), with the single-event <c>SendEvent</c> path as its fallback. The drain worker behind
|
||||
/// <c>LocalDbStoreAndForwardSink</c> calls <see cref="WriteBatchAsync"/> and uses the returned per-event
|
||||
/// <see cref="HistorianWriteOutcome"/> to decide retry vs. dead-letter, so this writer maps every gateway result — a
|
||||
/// batch result, a success ack, the published client's typed exception hierarchy, raw <see cref="RpcException"/>
|
||||
/// (defensive), and any unexpected error — onto exactly one outcome per event and <b>never throws</b>.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// Each event is sent individually so one poison event cannot fail the whole batch: a permanent
|
||||
/// failure on event N is dead-lettered while its siblings continue. The sends are pipelined — at most
|
||||
/// <see cref="MaxSendsInFlight"/> in flight — because the gateway has no batch <c>SendEvent</c> RPC yet.
|
||||
/// Outcomes are returned in input order, one per event; an empty batch yields an empty list with no gateway call.
|
||||
/// <b>Batches (lmxopcua#746).</b> A drain batch is sent as <c>SendEvents</c> calls of at most
|
||||
/// <see cref="MaxEventsPerBatch"/> events, one round trip each. Per event result: committed / queued ⇒
|
||||
/// <see cref="HistorianWriteOutcome.Ack"/>; failed ⇒ that event is re-sent ALONE through <c>SendEvent</c> so it is
|
||||
/// classified exactly as before (a historian rejection dead-letters, a transient fault retries) — a failed result
|
||||
/// was not committed, so the re-send cannot duplicate it. A whole-call failure:
|
||||
/// </para>
|
||||
/// <list type="bullet">
|
||||
/// <item>transient / systemic (<c>Unavailable</c>, auth, back-pressure, deadline, cancellation) ⇒ that call's and
|
||||
/// every later call's events are <see cref="HistorianWriteOutcome.RetryPlease"/>, nothing more is sent (every
|
||||
/// later call would hit it too). Events before the fault may already be committed, so the retry can duplicate
|
||||
/// them — <c>SendEvent</c> is at-least-once too;</item>
|
||||
/// <item><c>Unimplemented</c> (a gateway older than the RPC) ⇒ the writer stops trying batches for its lifetime and
|
||||
/// sends every event individually;</item>
|
||||
/// <item>any other refusal (<c>InvalidArgument</c> — a poison event or an over-limit batch, nothing sent — and the
|
||||
/// other permanent codes) ⇒ that call's events are sent individually, so the poison event dead-letters alone.</item>
|
||||
/// </list>
|
||||
/// <para>
|
||||
/// <b>Single sends</b> are pipelined — at most <see cref="MaxSendsInFlight"/> in flight — and each gets its own
|
||||
/// outcome. Outcomes are returned in input order, one per event; an empty batch yields an empty list with no gateway
|
||||
/// call.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Outcome mapping.</b> Success (or store-forward-queued) ack ⇒ <see cref="HistorianWriteOutcome.Ack"/>.
|
||||
@@ -42,8 +57,11 @@ public sealed class GatewayAlarmHistorianWriter : IAlarmHistorianWriter, IDispos
|
||||
private readonly IHistorianGatewayClient _client;
|
||||
private readonly ILogger<GatewayAlarmHistorianWriter> _logger;
|
||||
|
||||
/// <summary>Set once the gateway answers <c>SendEvents</c> with <c>Unimplemented</c>: single sends from then on.</summary>
|
||||
private volatile bool _batchUnsupported;
|
||||
|
||||
/// <summary>Creates the writer over a gateway client seam.</summary>
|
||||
/// <param name="client">The gateway client used for the <c>SendEvent</c> write path.</param>
|
||||
/// <param name="client">The gateway client used for the <c>SendEvents</c> / <c>SendEvent</c> write paths.</param>
|
||||
/// <param name="logger">Logger for per-event outcome diagnostics (never logs event content).</param>
|
||||
public GatewayAlarmHistorianWriter(IHistorianGatewayClient client, ILogger<GatewayAlarmHistorianWriter> logger)
|
||||
{
|
||||
@@ -51,6 +69,15 @@ public sealed class GatewayAlarmHistorianWriter : IAlarmHistorianWriter, IDispos
|
||||
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
||||
}
|
||||
|
||||
/// <summary>The most events one <c>SendEvents</c> call carries (the gateway's own default limit is 10,000 / 16 MiB).</summary>
|
||||
public const int MaxEventsPerBatch = 200;
|
||||
|
||||
/// <summary>The most single <c>SendEvent</c> RPCs one batch has in flight at once (the fallback path).</summary>
|
||||
public const int MaxSendsInFlight = 8;
|
||||
|
||||
/// <summary>Whether the writer has fallen back to single sends for good (the gateway has no <c>SendEvents</c>) — diagnostic.</summary>
|
||||
internal bool BatchUnsupported => _batchUnsupported;
|
||||
|
||||
/// <inheritdoc />
|
||||
public async Task<IReadOnlyList<HistorianWriteOutcome>> WriteBatchAsync(
|
||||
IReadOnlyList<AlarmHistorianEvent> batch, CancellationToken cancellationToken)
|
||||
@@ -62,21 +89,123 @@ public sealed class GatewayAlarmHistorianWriter : IAlarmHistorianWriter, IDispos
|
||||
return Array.Empty<HistorianWriteOutcome>();
|
||||
}
|
||||
|
||||
var outcomes = new HistorianWriteOutcome[batch.Count];
|
||||
var outcomes = new HistorianWriteOutcome?[batch.Count];
|
||||
var singles = new List<int>();
|
||||
|
||||
// The gateway has no batch SendEvent RPC, so the sends are PIPELINED: up to MaxSendsInFlight at once, each still
|
||||
// its own RPC with its own outcome (so one poison event cannot fail the batch), outcomes kept in input order. A
|
||||
// sequential drain cost one round trip per event — an alarm storm over a WAN took minutes.
|
||||
using var window = new SemaphoreSlim(MaxSendsInFlight);
|
||||
var sends = new Task[batch.Count];
|
||||
for (var i = 0; i < batch.Count; i++)
|
||||
for (var offset = 0; offset < batch.Count; offset += MaxEventsPerBatch)
|
||||
{
|
||||
var index = i;
|
||||
sends[i] = SendInWindowAsync(index);
|
||||
var count = Math.Min(MaxEventsPerBatch, batch.Count - offset);
|
||||
if (_batchUnsupported)
|
||||
{
|
||||
for (var i = 0; i < count; i++) singles.Add(offset + i);
|
||||
continue;
|
||||
}
|
||||
if (cancellationToken.IsCancellationRequested)
|
||||
{
|
||||
// Shutdown mid-drain: an event never sent stays queued for next startup — never dead-lettered.
|
||||
for (var i = offset; i < batch.Count; i++) outcomes[i] = HistorianWriteOutcome.RetryPlease;
|
||||
break;
|
||||
}
|
||||
|
||||
if (!await SendChunkAsync(batch, offset, count, outcomes, singles, cancellationToken).ConfigureAwait(false))
|
||||
{
|
||||
// A systemic fault: every later call would hit it too — retry them all, send nothing more.
|
||||
for (var i = offset + count; i < batch.Count; i++) outcomes[i] = HistorianWriteOutcome.RetryPlease;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (singles.Count > 0) await SendSinglesAsync(batch, singles, outcomes, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
var result = new HistorianWriteOutcome[batch.Count];
|
||||
for (var i = 0; i < result.Length; i++) result[i] = outcomes[i] ?? HistorianWriteOutcome.RetryPlease;
|
||||
return result;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Sends events [<paramref name="offset"/>, +<paramref name="count"/>) as one <c>SendEvents</c> call, recording each
|
||||
/// event's outcome or queueing it for a single send. Returns <c>false</c> on a systemic fault (the chunk's events are
|
||||
/// then <see cref="HistorianWriteOutcome.RetryPlease"/> and the caller sends nothing more).
|
||||
/// </summary>
|
||||
private async Task<bool> SendChunkAsync(IReadOnlyList<AlarmHistorianEvent> batch, int offset, int count,
|
||||
HistorianWriteOutcome?[] outcomes, List<int> singles, CancellationToken cancellationToken)
|
||||
{
|
||||
var events = new List<HistorianEvent>(count);
|
||||
var indexes = new List<int>(count);
|
||||
for (var i = offset; i < offset + count; i++)
|
||||
{
|
||||
try
|
||||
{
|
||||
events.Add(AlarmEventMapper.ToHistorianEvent(batch[i]));
|
||||
indexes.Add(i);
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
// An event that cannot even be mapped is poison: classified exactly as a single send would classify it.
|
||||
outcomes[i] = Classify(exception);
|
||||
_logger.LogWarning("Alarm event could not be mapped for SendEvents ({Exception}); dead-lettering it.",
|
||||
exception.GetType().Name);
|
||||
}
|
||||
}
|
||||
if (events.Count == 0) return true;
|
||||
|
||||
SendEventsReply reply;
|
||||
try
|
||||
{
|
||||
reply = await _client.SendEventsAsync(events, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
|
||||
{
|
||||
_logger.LogDebug("Alarm SendEvents cancelled at shutdown; will retry.");
|
||||
foreach (var i in indexes) outcomes[i] = HistorianWriteOutcome.RetryPlease;
|
||||
return false;
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
if (StatusOf(exception) == StatusCode.Unimplemented)
|
||||
{
|
||||
_batchUnsupported = true;
|
||||
_logger.LogInformation(
|
||||
"HistorianGateway has no SendEvents RPC (older than Contracts 0.4.0); alarm-history writes fall back to one SendEvent per event.");
|
||||
singles.AddRange(indexes);
|
||||
return true;
|
||||
}
|
||||
if (Classify(exception) == HistorianWriteOutcome.RetryPlease)
|
||||
{
|
||||
_logger.LogDebug("Alarm SendEvents transiently failed ({Exception}); the batch will retry.", exception.GetType().Name);
|
||||
foreach (var i in indexes) outcomes[i] = HistorianWriteOutcome.RetryPlease;
|
||||
return false;
|
||||
}
|
||||
// Refused up front (a poison event, an over-limit batch, …): nothing was sent — send them one by one so a poison
|
||||
// event dead-letters alone and its siblings still land.
|
||||
_logger.LogDebug("Alarm SendEvents refused the batch ({Exception}); sending its events individually.", exception.GetType().Name);
|
||||
singles.AddRange(indexes);
|
||||
return true;
|
||||
}
|
||||
|
||||
for (var k = 0; k < indexes.Count; k++)
|
||||
{
|
||||
if (k < reply.Results.Count && reply.Results[k] is { } r && (r.Success || r.Queued))
|
||||
outcomes[indexes[k]] = HistorianWriteOutcome.Ack;
|
||||
else
|
||||
singles.Add(indexes[k]); // failed (not committed) or missing from the reply: classify it by a single send
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/// <summary>Sends the given events one <c>SendEvent</c> each, at most <see cref="MaxSendsInFlight"/> in flight.</summary>
|
||||
private async Task SendSinglesAsync(IReadOnlyList<AlarmHistorianEvent> batch, List<int> indexes,
|
||||
HistorianWriteOutcome?[] outcomes, CancellationToken cancellationToken)
|
||||
{
|
||||
using var window = new SemaphoreSlim(MaxSendsInFlight);
|
||||
var sends = new Task[indexes.Count];
|
||||
for (var k = 0; k < indexes.Count; k++)
|
||||
{
|
||||
var index = indexes[k];
|
||||
sends[k] = SendInWindowAsync(index);
|
||||
}
|
||||
// Every task completes (none throws), so the window is never disposed under a waiter.
|
||||
await Task.WhenAll(sends).ConfigureAwait(false);
|
||||
return outcomes;
|
||||
|
||||
async Task SendInWindowAsync(int index)
|
||||
{
|
||||
@@ -104,8 +233,13 @@ public sealed class GatewayAlarmHistorianWriter : IAlarmHistorianWriter, IDispos
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>The most <c>SendEvent</c> RPCs one batch has in flight at once (bounded pipelining until a batch RPC exists).</summary>
|
||||
public const int MaxSendsInFlight = 8;
|
||||
/// <summary>The gRPC status of a gateway failure (raw, or wrapped by the package client); null when it has none.</summary>
|
||||
private static StatusCode? StatusOf(Exception exception)
|
||||
{
|
||||
for (Exception? e = exception; e is not null; e = e.InnerException)
|
||||
if (e is RpcException rpc) return rpc.StatusCode;
|
||||
return null;
|
||||
}
|
||||
|
||||
private async Task<HistorianWriteOutcome> SendOneAsync(AlarmHistorianEvent evt, CancellationToken cancellationToken)
|
||||
{
|
||||
|
||||
+144
-64
@@ -39,10 +39,9 @@ public sealed class GatewayHistorianDataSource : IHistorianDataSource, IAsyncDis
|
||||
/// up-front <c>BadAggregateNotSupported</c> read it too.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// The two native members are served only with <c>ServerHistorian:Gateway:EnableNativeTimeWeighted</c>: until the
|
||||
/// gateway's interval fix (historiangw#27) is live-verified AVEVA answers them with Good-status wrong values, so by
|
||||
/// default an instance serves (and advertises) <see cref="RawEvaluatedAggregates"/> only — see
|
||||
/// <see cref="SupportedAggregates"/>.
|
||||
/// The two native members are served only with <c>ServerHistorian:Gateway:EnableNativeTimeWeighted</c> (default
|
||||
/// <c>true</c> since lmxopcua#694, once the gateway's interval fix historiangw#27 was live-verified); with it off an
|
||||
/// instance serves (and advertises) <see cref="RawEvaluatedAggregates"/> only — see <see cref="SupportedAggregates"/>.
|
||||
/// </remarks>
|
||||
public static IReadOnlySet<HistoryAggregateType> ServedAggregates { get; } =
|
||||
Enum.GetValues<HistoryAggregateType>()
|
||||
@@ -50,8 +49,8 @@ public sealed class GatewayHistorianDataSource : IHistorianDataSource, IAsyncDis
|
||||
.ToFrozenSet();
|
||||
|
||||
/// <summary>
|
||||
/// The per-sample aggregates this backend evaluates from raw reads — what an instance serves without
|
||||
/// <c>ServerHistorian:Gateway:EnableNativeTimeWeighted</c> (the default).
|
||||
/// The per-sample aggregates this backend evaluates from raw reads — what an instance serves with
|
||||
/// <c>ServerHistorian:Gateway:EnableNativeTimeWeighted=false</c> (the kill switch).
|
||||
/// </summary>
|
||||
public static IReadOnlySet<HistoryAggregateType> RawEvaluatedAggregates { get; } =
|
||||
Enum.GetValues<HistoryAggregateType>().Where(RawAggregateReader.Serves).ToFrozenSet();
|
||||
@@ -163,8 +162,8 @@ public sealed class GatewayHistorianDataSource : IHistorianDataSource, IAsyncDis
|
||||
/// </param>
|
||||
/// <param name="enableNativeTimeWeighted">
|
||||
/// Serve TimeAverage / Total through AVEVA's native retrieval modes (<c>ServerHistorian:Gateway:EnableNativeTimeWeighted</c>,
|
||||
/// default <c>false</c>): until historiangw#27 is live-verified they answer Good-status wrong values, so by default they
|
||||
/// are refused <see cref="HistoryAggregateNotSupportedException"/> (<c>BadAggregateNotSupported</c>) and not advertised.
|
||||
/// default <c>true</c> since lmxopcua#694). <c>false</c> refuses both <see cref="HistoryAggregateNotSupportedException"/>
|
||||
/// (<c>BadAggregateNotSupported</c>) and stops advertising them — for a gateway older than the historiangw#27 fix.
|
||||
/// </param>
|
||||
/// <param name="probeInterval">
|
||||
/// Cadence of the <c>ProbeAsync</c> health loop (<c>ServerHistorian:Gateway:ProbeInterval</c>); the first probe runs
|
||||
@@ -178,7 +177,7 @@ public sealed class GatewayHistorianDataSource : IHistorianDataSource, IAsyncDis
|
||||
public GatewayHistorianDataSource(IHistorianGatewayClient client, ILogger<GatewayHistorianDataSource> logger,
|
||||
int maxRawRows = TimeWeightedAggregator.DefaultMaxWindowRows, TimeProvider? timeProvider = null,
|
||||
bool atTimeFromRaw = true, int maxPreWindowRows = DefaultMaxPreWindowRows, int maxRawValuesPerRead = 0,
|
||||
int eventDrainBudget = DefaultEventDrainBudget, bool enableNativeTimeWeighted = false,
|
||||
int eventDrainBudget = DefaultEventDrainBudget, bool enableNativeTimeWeighted = true,
|
||||
TimeSpan? probeInterval = null)
|
||||
{
|
||||
_enableNativeTimeWeighted = enableNativeTimeWeighted;
|
||||
@@ -494,7 +493,7 @@ public sealed class GatewayHistorianDataSource : IHistorianDataSource, IAsyncDis
|
||||
|
||||
// Mapped OUTSIDE the failure-recording try: an aggregate the gateway has no mode for is a client
|
||||
// request the server answers BadAggregateNotSupported, not a backend fault to count. The native pair is refused
|
||||
// the same way unless opted in (Good-status wrong values until historiangw#27).
|
||||
// the same way when EnableNativeTimeWeighted is off (the kill switch for a gateway older than historiangw#27).
|
||||
var mode = AggregateModeMapper.ToRetrievalMode(aggregate);
|
||||
if (!_enableNativeTimeWeighted) throw new HistoryAggregateNotSupportedException(aggregate);
|
||||
// Planned OUTSIDE the failure-recording try: too many intervals is a client request limit, not a backend fault.
|
||||
@@ -510,6 +509,8 @@ public sealed class GatewayHistorianDataSource : IHistorianDataSource, IAsyncDis
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
if (plan.BucketCount == 0) return new HistoryReadResult([], ContinuationPoint: null);
|
||||
// Planned OUTSIDE the failure-recording try: a non-whole-ms interval over too many intervals is a client request limit.
|
||||
var requests = NativeAggregatePlan.Requests(plan);
|
||||
try
|
||||
{
|
||||
if (!await _tags.ExistsAsync(fullReference, cancellationToken).ConfigureAwait(false))
|
||||
@@ -518,7 +519,7 @@ public sealed class GatewayHistorianDataSource : IHistorianDataSource, IAsyncDis
|
||||
return new HistoryReadResult(plan.Assemble(new Dictionary<long, DataValueSnapshot>(), t => AggregateEvaluator.NoData(t, _time.GetUtcNow().UtcDateTime)), ContinuationPoint: null);
|
||||
}
|
||||
var byBucket = new Dictionary<long, DataValueSnapshot>();
|
||||
foreach (var request in NativeAggregatePlan.Requests(plan))
|
||||
foreach (var request in requests)
|
||||
{
|
||||
var buckets = new List<HistorianAggregateSample>();
|
||||
await foreach (var bucket in _client
|
||||
@@ -738,72 +739,61 @@ public sealed class GatewayHistorianDataSource : IHistorianDataSource, IAsyncDis
|
||||
"The HistorianGateway backend reads events oldest-first only; a reverse (newest-first) events read is not supported.")
|
||||
: ReadEventsAsync(sourceName, startUtc, endUtc, maxEvents, cancellationToken);
|
||||
|
||||
/// <summary>
|
||||
/// The largest page cap sent as the server-side <c>max_events</c> (as cap + 1): the gateway's default
|
||||
/// <c>Historian:ReadLimits:MaxEventsCeiling</c> is 1,000,000 and a larger <c>max_events</c> is <c>InvalidArgument</c>.
|
||||
/// A larger cap (an uncapped read arrives as <see cref="int.MaxValue"/>) is read without a server cap.
|
||||
/// </summary>
|
||||
internal const int MaxServerCappedEvents = 999_999;
|
||||
|
||||
/// <inheritdoc />
|
||||
// The gateway's ReadEvents has no max-events field (the 0.3.0 wire contract), so the cap is applied here. The stream
|
||||
// is DRAINED past the cap — counted, never collected — rather than broken out of: abandoning a gateway stream cancels
|
||||
// the RPC, which the gateway classifies as a connection fault and answers by evicting its pooled historian session
|
||||
// (the next read of any kind reconnects; concurrent reads on that session fail). The drain is bounded by the
|
||||
// EventDrainBudget (ServerHistorian:Gateway:EventDrainBudget): past cap + budget events the stream is abandoned after
|
||||
// all (a window far larger than the page — streaming all of it for every page would cost O(window²/N)), logged at
|
||||
// Debug and counted in EventStreamsAbandoned. 0 = abandon at the cap (the previous behaviour).
|
||||
// A capped read asks the gateway for cap + 1 events through ReadEventsRequest.max_events (HistorianGateway
|
||||
// Contracts/Client 0.4.0+, lmxopcua#746): the gateway stops there and COMPLETES the stream normally — nothing is cancelled,
|
||||
// so it never evicts its pooled historian session for this read — and the (cap + 1)-th matching event is the "more
|
||||
// events exist" signal. When that answer is ambiguous — the server stopped at cap + 1 rows but the defensive client-side
|
||||
// source filter dropped some of them, so whether more matching events exist is unknown — or a gateway refuses the cap
|
||||
// (InvalidArgument: a MaxEventsCeiling below cap + 1), the read is repeated without a server cap by the drain below.
|
||||
//
|
||||
// Uncapped (no server cap) the stream is DRAINED past the page cap — counted, never collected — rather than broken out
|
||||
// of: abandoning a gateway stream cancels the RPC, which the gateway classifies as a connection fault and answers by
|
||||
// evicting its pooled historian session. The drain is bounded by the EventDrainBudget
|
||||
// (ServerHistorian:Gateway:EventDrainBudget): past cap + budget events the stream is abandoned after all, logged at Debug
|
||||
// and counted in EventStreamsAbandoned. 0 = abandon at the cap.
|
||||
public async Task<HistoricalEventsResult> ReadEventsAsync(
|
||||
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
try
|
||||
{
|
||||
var hasCap = maxEvents > 0;
|
||||
// Bounded initial capacity: an uncapped read arrives as int.MaxValue ("Array dimensions exceeded").
|
||||
var collected = new List<HistoricalEvent>(hasCap ? Math.Min(maxEvents, 1024) : 0);
|
||||
var truncated = false;
|
||||
long pastCap = 0; // stream rows consumed after the cap filled (matching or not)
|
||||
var abandoned = false;
|
||||
|
||||
await foreach (var wireEvent in _client
|
||||
.ReadEventsAsync(sourceName, startUtc, endUtc, maxEvents, cancellationToken)
|
||||
.ConfigureAwait(false))
|
||||
var serverCap = maxEvents is > 0 and <= MaxServerCappedEvents ? maxEvents + 1 : 0;
|
||||
EventDrain drain;
|
||||
if (serverCap > 0)
|
||||
{
|
||||
if (hasCap && collected.Count == maxEvents)
|
||||
try
|
||||
{
|
||||
// The page is full: drain the rest of the stream without collecting it, within the budget.
|
||||
// One more MATCHING event is the "more events exist" signal; a foreign source's row is not, and
|
||||
// until the signal is seen no row counts toward the budget — abandoning before it would answer
|
||||
// "complete" for a page that may not be (and a spurious continuation point on a stretch of foreign
|
||||
// rows would make the server crawl it one resolution step per page).
|
||||
if (!truncated)
|
||||
{
|
||||
if (!EventSourceMatch.Matches(sourceName, wireEvent.SourceName))
|
||||
continue;
|
||||
truncated = true;
|
||||
}
|
||||
if (++pastCap > _eventDrainBudget)
|
||||
{
|
||||
abandoned = true;
|
||||
break;
|
||||
}
|
||||
continue;
|
||||
drain = await DrainEventsAsync(sourceName, startUtc, endUtc, maxEvents, serverCap, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
}
|
||||
|
||||
var mapped = EventMapper.ToHistoricalEvent(wireEvent);
|
||||
|
||||
// Defensive client-side source filter: the gateway's SQL ReadEvents source filter
|
||||
// may not be present, so drop any event whose source does not match the request. The request's own
|
||||
// events and its attributes' (<source>.<attribute>, the name the gateway reports an AVEVA attribute
|
||||
// alarm under) match; a sibling that merely shares the prefix does not (#882, EventSourceMatch).
|
||||
if (!EventSourceMatch.Matches(sourceName, mapped.SourceName))
|
||||
catch (Exception ex) when (!cancellationToken.IsCancellationRequested && IsInvalidArgument(ex))
|
||||
{
|
||||
continue;
|
||||
_logger.LogDebug("HistorianGateway refused the events read's max_events {Cap}; re-reading it without a server cap.",
|
||||
serverCap);
|
||||
drain = await DrainEventsAsync(sourceName, startUtc, endUtc, maxEvents, 0, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
}
|
||||
// Exactly cap + 1 rows and no signal: the server cut the stream, but foreign rows were filtered out of it.
|
||||
// (More rows than asked = a gateway that ignores max_events streamed everything: the drain is the answer.)
|
||||
if (drain.Received == serverCap && !drain.Truncated)
|
||||
{
|
||||
Interlocked.Increment(ref _eventServerCapReReads);
|
||||
drain = await DrainEventsAsync(sourceName, startUtc, endUtc, maxEvents, 0, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
}
|
||||
|
||||
collected.Add(mapped);
|
||||
}
|
||||
|
||||
if (abandoned)
|
||||
else
|
||||
{
|
||||
Interlocked.Increment(ref _eventStreamsAbandoned);
|
||||
_logger.LogDebug(
|
||||
"HistorianGateway events read abandoned its stream {Budget} rows past the page cap (EventDrainBudget); " +
|
||||
"the gateway may evict its historian session.", _eventDrainBudget);
|
||||
drain = await DrainEventsAsync(sourceName, startUtc, endUtc, maxEvents, 0, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
}
|
||||
|
||||
RecordOutcome(success: true, error: null);
|
||||
@@ -811,7 +801,7 @@ public sealed class GatewayHistorianDataSource : IHistorianDataSource, IAsyncDis
|
||||
// The gateway has no resumable cursor, so the token's contents carry no paging state —
|
||||
// its presence alone is the "more events exist" signal. A fresh array per call keeps it
|
||||
// from being shared/mutated.
|
||||
return new HistoricalEventsResult(collected, truncated ? new byte[] { 0x01 } : null);
|
||||
return new HistoricalEventsResult(drain.Collected, drain.Truncated ? new byte[] { 0x01 } : null);
|
||||
}
|
||||
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
|
||||
{
|
||||
@@ -825,6 +815,96 @@ public sealed class GatewayHistorianDataSource : IHistorianDataSource, IAsyncDis
|
||||
}
|
||||
}
|
||||
|
||||
private long _eventServerCapReReads;
|
||||
|
||||
/// <summary>
|
||||
/// Server-capped events reads repeated without the cap because the capped answer was ambiguous (foreign rows filtered
|
||||
/// out of a stream the server cut at cap + 1) — test/diagnostic.
|
||||
/// </summary>
|
||||
internal long EventServerCapReReads => Interlocked.Read(ref _eventServerCapReReads);
|
||||
|
||||
/// <summary>One events stream's outcome: the collected page, the "more exist" signal and the stream rows received.</summary>
|
||||
private readonly record struct EventDrain(List<HistoricalEvent> Collected, bool Truncated, long Received);
|
||||
|
||||
/// <summary>
|
||||
/// Reads one events stream: collects up to <paramref name="maxEvents"/> matching events, then (cap reached) looks for
|
||||
/// one more matching event as the "more exist" signal and drains the rest within the EventDrainBudget.
|
||||
/// <paramref name="serverCap"/> > 0 is sent as <c>max_events</c>; 0 = no server cap.
|
||||
/// </summary>
|
||||
private async Task<EventDrain> DrainEventsAsync(string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents,
|
||||
int serverCap, CancellationToken cancellationToken)
|
||||
{
|
||||
var hasCap = maxEvents > 0;
|
||||
// Bounded initial capacity: an uncapped read arrives as int.MaxValue ("Array dimensions exceeded").
|
||||
var collected = new List<HistoricalEvent>(hasCap ? Math.Min(maxEvents, 1024) : 0);
|
||||
var truncated = false;
|
||||
long pastCap = 0; // stream rows consumed after the cap filled (matching or not)
|
||||
long received = 0;
|
||||
var abandoned = false;
|
||||
|
||||
await foreach (var wireEvent in _client
|
||||
.ReadEventsAsync(sourceName, startUtc, endUtc, serverCap, cancellationToken)
|
||||
.ConfigureAwait(false))
|
||||
{
|
||||
received++;
|
||||
if (hasCap && collected.Count == maxEvents)
|
||||
{
|
||||
// The page is full: drain the rest of the stream without collecting it, within the budget.
|
||||
// One more MATCHING event is the "more events exist" signal; a foreign source's row is not, and
|
||||
// until the signal is seen no row counts toward the budget — abandoning before it would answer
|
||||
// "complete" for a page that may not be (and a spurious continuation point on a stretch of foreign
|
||||
// rows would make the server crawl it one resolution step per page).
|
||||
if (!truncated)
|
||||
{
|
||||
if (!EventSourceMatch.Matches(sourceName, wireEvent.SourceName))
|
||||
continue;
|
||||
truncated = true;
|
||||
}
|
||||
// Server-capped, the stream is already bounded at cap + 1 (by the gateway, or by the package client against an
|
||||
// older gateway): read it to its end so the call completes normally instead of being cancelled. Only a stream
|
||||
// running past the max_events it was sent is drained within the budget.
|
||||
if (serverCap > 0 && received <= serverCap) continue;
|
||||
if (++pastCap > _eventDrainBudget)
|
||||
{
|
||||
abandoned = true;
|
||||
break;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
var mapped = EventMapper.ToHistoricalEvent(wireEvent);
|
||||
|
||||
// Defensive client-side source filter: the gateway's SQL ReadEvents source filter
|
||||
// may not be present, so drop any event whose source does not match the request. The request's own
|
||||
// events and its attributes' (<source>.<attribute>, the name the gateway reports an AVEVA attribute
|
||||
// alarm under) match; a sibling that merely shares the prefix does not (#882, EventSourceMatch).
|
||||
if (!EventSourceMatch.Matches(sourceName, mapped.SourceName))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
collected.Add(mapped);
|
||||
}
|
||||
|
||||
if (abandoned)
|
||||
{
|
||||
Interlocked.Increment(ref _eventStreamsAbandoned);
|
||||
_logger.LogDebug(
|
||||
"HistorianGateway events read abandoned its stream {Budget} rows past the page cap (EventDrainBudget); " +
|
||||
"the gateway may evict its historian session.", _eventDrainBudget);
|
||||
}
|
||||
|
||||
return new EventDrain(collected, truncated, received);
|
||||
}
|
||||
|
||||
/// <summary>Whether a gateway failure is an <c>InvalidArgument</c> refusal (raw, or wrapped by the package client).</summary>
|
||||
private static bool IsInvalidArgument(Exception ex)
|
||||
{
|
||||
for (Exception? e = ex; e is not null; e = e.InnerException)
|
||||
if (e is RpcException { StatusCode: Grpc.Core.StatusCode.InvalidArgument }) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public HistorianHealthSnapshot GetHealthSnapshot()
|
||||
{
|
||||
|
||||
+1
-1
@@ -49,7 +49,7 @@ public static class GatewayHistorian
|
||||
maxRawValuesPerRead: options.EffectiveMaxRawValuesPerRead,
|
||||
// How far a capped events read drains its stream past the cap before abandoning it.
|
||||
eventDrainBudget: options.Gateway.EventDrainBudget,
|
||||
// AVEVA's native TimeAverage / Total — off until historiangw#27 is live-verified.
|
||||
// AVEVA's native TimeAverage / Total — on by default since historiangw#27 was live-verified (lmxopcua#694).
|
||||
enableNativeTimeWeighted: options.Gateway.EnableNativeTimeWeighted,
|
||||
// A slow ProbeAsync timer so the health snapshot reflects the gateway without waiting for a read.
|
||||
probeInterval: options.Gateway.ProbeInterval);
|
||||
|
||||
+6
-1
@@ -135,7 +135,8 @@ public sealed class HistorianGatewayClientAdapter : IHistorianGatewayClient, IDi
|
||||
Value = sourceName,
|
||||
};
|
||||
|
||||
return _inner.ReadEventsAsync(startUtc, endUtc, filter, ct);
|
||||
// max_events rides on the request (server-side cap, completes the stream normally); 0 = unlimited.
|
||||
return _inner.ReadEventsAsync(startUtc, endUtc, filter, Math.Max(0, maxEvents), ct);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
@@ -147,6 +148,10 @@ public sealed class HistorianGatewayClientAdapter : IHistorianGatewayClient, IDi
|
||||
public Task<WriteAck> SendEventAsync(HistorianEvent evt, CancellationToken ct) =>
|
||||
_inner.SendEventAsync(evt, ct);
|
||||
|
||||
/// <inheritdoc />
|
||||
public Task<SendEventsReply> SendEventsAsync(IReadOnlyList<HistorianEvent> events, CancellationToken ct) =>
|
||||
_inner.SendEventsAsync(events, ct);
|
||||
|
||||
/// <inheritdoc />
|
||||
public Task<TagOperationResults> EnsureTagsAsync(
|
||||
IReadOnlyList<HistorianTagDefinition> definitions, CancellationToken ct) =>
|
||||
|
||||
@@ -65,10 +65,11 @@ public interface IHistorianGatewayClient : IAsyncDisposable
|
||||
/// <param name="startUtc">Inclusive start of the time window (UTC).</param>
|
||||
/// <param name="endUtc">Exclusive end of the time window (UTC).</param>
|
||||
/// <param name="maxEvents">
|
||||
/// Caps the number of events returned. The gateway wire contract (<c>ReadEventsRequest</c>) has
|
||||
/// no per-call server cap, so this limit is enforced client-side by early stream termination:
|
||||
/// <c>0</c> or negative means no client-side limit (the gateway may still apply its configured
|
||||
/// <c>RuntimeDb:EventReadMaxRows</c>); a positive value stops draining after that many events.
|
||||
/// The most events to stream, sent as the server-side <c>ReadEventsRequest.max_events</c> (HistorianGateway
|
||||
/// Contracts/Client 0.4.0+): the gateway stops at that many and COMPLETES the stream normally, so a bounded read
|
||||
/// needs no cancellation. <c>0</c> or negative = unlimited (the gateway still applies its
|
||||
/// <c>Historian:ReadLimits:MaxEventsCeiling</c>). A gateway older than the field ignores it; the package client then
|
||||
/// stops reading at the cap itself, which abandons (cancels) the stream.
|
||||
/// </param>
|
||||
/// <param name="ct">Cancellation token.</param>
|
||||
/// <returns>An async stream of historian events within the window.</returns>
|
||||
@@ -95,6 +96,23 @@ public interface IHistorianGatewayClient : IAsyncDisposable
|
||||
/// <returns>The write acknowledgement returned by the gateway.</returns>
|
||||
Task<WriteAck> SendEventAsync(HistorianEvent evt, CancellationToken ct);
|
||||
|
||||
/// <summary>
|
||||
/// Sends a batch of historian events in one <c>SendEvents</c> RPC (HistorianGateway Contracts/Client 0.4.0+): one
|
||||
/// <see cref="EventWriteResult"/> per event, in request order. A systemic fault fails the whole call (events before it
|
||||
/// may already be committed); a request the gateway refuses up front (an event without <c>event_time</c>, a non-GUID
|
||||
/// id, a batch over its limits) is <c>InvalidArgument</c> with nothing sent.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// The default implementation answers <c>Unimplemented</c> — what a gateway older than the RPC answers — so a seam
|
||||
/// implementation without a batch path (test doubles) makes callers fall back to <see cref="SendEventAsync"/>.
|
||||
/// </remarks>
|
||||
/// <param name="events">The events to send (non-empty).</param>
|
||||
/// <param name="ct">Cancellation token.</param>
|
||||
/// <returns>The per-event results, in request order.</returns>
|
||||
Task<SendEventsReply> SendEventsAsync(IReadOnlyList<HistorianEvent> events, CancellationToken ct) =>
|
||||
Task.FromException<SendEventsReply>(new Grpc.Core.RpcException(
|
||||
new Grpc.Core.Status(Grpc.Core.StatusCode.Unimplemented, "SendEvents is not implemented by this gateway client.")));
|
||||
|
||||
/// <summary>Ensures the supplied tag definitions exist (create-or-update).</summary>
|
||||
/// <param name="definitions">The tag definitions to create or update.</param>
|
||||
/// <param name="ct">Cancellation token.</param>
|
||||
|
||||
@@ -37,13 +37,26 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway;
|
||||
/// row for is absent (not a fabricated <c>BadNoData</c>, unlike the SQL backends).
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// ⚠️ Today historiangw#27 (the interval reaches AVEVA in ticks where it reads milliseconds) makes AVEVA answer ONE
|
||||
/// Good row per request — the whole-intervals read lands on its earliest interval with a whole-span value and the rest
|
||||
/// are absent — in both directions. The mapping needs no change once the gateway is fixed.
|
||||
/// <b>Whole milliseconds only on a shared grid.</b> The gateway sends AVEVA the interval in whole milliseconds
|
||||
/// (<c>max(1, round(ms))</c>), and AVEVA clips its last interval at the request's end (live, lmxopcua#694, after
|
||||
/// historiangw#27: a 0.4 ms window answers one row over exactly that 0.4 ms — <c>Integral</c> = value·0.0004 s — and a
|
||||
/// 1.4 ms window read at "1.4 ms" answers 1 ms rows). So a grid of several intervals is sent as one request only when
|
||||
/// the interval is a whole number of milliseconds; otherwise every interval is its own request over exactly its span
|
||||
/// (<see cref="MaxPerIntervalRequests"/> at most — more is <see cref="HistoryReadLimitExceededException"/>). A request
|
||||
/// for ONE interval (the clipped one, a window narrower than one interval, or each of those per-interval requests) is
|
||||
/// sent with its width rounded UP to whole milliseconds as the interval, so AVEVA answers exactly one row, clipped at
|
||||
/// the request's end: sent at its own width, a 3.0004 s interval reached AVEVA as 3,000 ms and answered a 3 s row plus
|
||||
/// a 0.4 ms one (live, lmxopcua#694 — the clipped interval's Total lost the 0.4 ms).
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
internal static class NativeAggregatePlan
|
||||
{
|
||||
/// <summary>
|
||||
/// The most intervals a native read whose interval is not a whole number of milliseconds may cover: each is its own
|
||||
/// gateway request.
|
||||
/// </summary>
|
||||
public const int MaxPerIntervalRequests = 1000;
|
||||
|
||||
/// <summary>One forward native request and the ascending bucket index of its first interval.</summary>
|
||||
/// <param name="StartUtc">Forward request start (inclusive).</param>
|
||||
/// <param name="EndUtc">Forward request end.</param>
|
||||
@@ -55,10 +68,32 @@ internal static class NativeAggregatePlan
|
||||
/// <summary>The forward requests serving a plan: the whole intervals first, then the clipped one (if any).</summary>
|
||||
/// <param name="plan">A plan with at least one bucket.</param>
|
||||
/// <returns>One or two forward requests.</returns>
|
||||
/// <exception cref="HistoryReadLimitExceededException">
|
||||
/// The interval is not a whole number of milliseconds and the window holds more than
|
||||
/// <see cref="MaxPerIntervalRequests"/> intervals.
|
||||
/// </exception>
|
||||
public static IReadOnlyList<Request> Requests(BucketPlan plan)
|
||||
{
|
||||
var interval = TimeSpan.FromTicks(plan.IntervalTicks);
|
||||
var n = plan.BucketCount;
|
||||
if (n > 1 && plan.IntervalTicks % TimeSpan.TicksPerMillisecond != 0)
|
||||
{
|
||||
// The gateway would round the interval to whole ms and AVEVA would answer on that grid, not the plan's: read
|
||||
// each interval over exactly its span instead (one interval per request is exact at any width).
|
||||
if (n > MaxPerIntervalRequests)
|
||||
throw new HistoryReadLimitExceededException(MaxPerIntervalRequests,
|
||||
$"A native TimeAverage/Total read whose interval is not a whole number of milliseconds is read one " +
|
||||
$"interval per gateway request, at most {MaxPerIntervalRequests}; this window holds {n}. Use a whole-millisecond " +
|
||||
"ProcessingInterval or a shorter window.");
|
||||
var each = new List<Request>((int)n);
|
||||
for (long i = 0; i < n; i++)
|
||||
{
|
||||
var lo = new DateTime(plan.LowerTicks(i), DateTimeKind.Utc);
|
||||
var hi = new DateTime(plan.UpperTicks(i), DateTimeKind.Utc);
|
||||
each.Add(new Request(lo, hi, OneInterval(hi - lo), i, 1));
|
||||
}
|
||||
return each;
|
||||
}
|
||||
var clipped = plan.IsTruncated(0) ? 0L : plan.IsTruncated(n - 1) ? n - 1 : -1L;
|
||||
if (clipped < 0) return [new Request(plan.StartUtc, plan.EndUtc, interval, 0, n)];
|
||||
|
||||
@@ -69,10 +104,23 @@ internal static class NativeAggregatePlan
|
||||
requests.Add(clipped == 0
|
||||
? new Request(clipHi, plan.EndUtc, interval, 1, n - 1) // reverse: the clipped one is the earliest
|
||||
: new Request(plan.StartUtc, clipLo, interval, 0, n - 1)); // forward: the clipped one is the latest
|
||||
requests.Add(new Request(clipLo, clipHi, clipHi - clipLo, clipped, 1));
|
||||
requests.Add(new Request(clipLo, clipHi, OneInterval(clipHi - clipLo), clipped, 1));
|
||||
return requests;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The interval to send for a request covering exactly one interval of <paramref name="width"/>: the width rounded UP
|
||||
/// to whole milliseconds (at least 1 ms), which the gateway sends unchanged, so AVEVA's single interval spans the whole
|
||||
/// request and is clipped at its end — never a whole-ms interval plus a sub-ms remainder row.
|
||||
/// </summary>
|
||||
/// <param name="width">The request's span (> 0).</param>
|
||||
/// <returns>The interval to send.</returns>
|
||||
internal static TimeSpan OneInterval(TimeSpan width)
|
||||
{
|
||||
var ms = Math.Max(1, (width.Ticks + TimeSpan.TicksPerMillisecond - 1) / TimeSpan.TicksPerMillisecond);
|
||||
return TimeSpan.FromTicks(ms * TimeSpan.TicksPerMillisecond);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Maps a request's rows onto the plan's buckets: rows outside the request window (AVEVA's pre-window row)
|
||||
/// are dropped; each in-window row is placed by its interval start (or, without one, its end).
|
||||
|
||||
@@ -284,13 +284,15 @@ public sealed class GatewayHistorianSettings
|
||||
public int EventDrainBudget { get; init; } = DefaultEventDrainBudget;
|
||||
|
||||
/// <summary>
|
||||
/// Serve HistoryRead-Processed <c>TimeAverage</c> / <c>Total</c> through AVEVA's native retrieval modes. Default
|
||||
/// <c>false</c>: the gateway hands AVEVA the interval in the wrong unit (historiangw#27), so AVEVA answers ONE
|
||||
/// Good-status whole-span value on the earliest interval and nothing for the rest — wrong values a client cannot
|
||||
/// tell from right ones. Off, both are <c>BadAggregateNotSupported</c> and are not advertised (like the other five
|
||||
/// time-weighted aggregates on this backend). Turn it on only once a fixed gateway is live-verified.
|
||||
/// Serve HistoryRead-Processed <c>TimeAverage</c> / <c>Total</c> through AVEVA's native retrieval modes
|
||||
/// (<c>TimeWeightedAverage</c> / <c>Integral</c>). Default <c>true</c> since lmxopcua#694 (2026-11-13): the gateway
|
||||
/// interval fix (historiangw#27, gateway server <c>9f711c1</c> or later) is live-verified — forward and reverse
|
||||
/// per-interval values, all-NULL intervals and sub-millisecond clipped intervals. <c>false</c> is the kill switch:
|
||||
/// both are then <c>BadAggregateNotSupported</c> and are not advertised (like the other five time-weighted aggregates
|
||||
/// on this backend). Set it <c>false</c> when the gateway is older than the #27 fix (HistorianGateway <c>9f711c1</c>):
|
||||
/// such a gateway makes AVEVA answer ONE Good-status whole-span value on the earliest interval.
|
||||
/// </summary>
|
||||
public bool EnableNativeTimeWeighted { get; init; }
|
||||
public bool EnableNativeTimeWeighted { get; init; } = true;
|
||||
|
||||
/// <summary>
|
||||
/// How often the read client probes the gateway (<c>ProbeAsync</c>) for the historian health snapshot's
|
||||
|
||||
+42
-1
@@ -211,7 +211,48 @@ public sealed class FakeHistorianGatewayClient : IHistorianGatewayClient
|
||||
LastReadEventsEndUtc = endUtc;
|
||||
LastReadEventsMaxEvents = maxEvents;
|
||||
ReadEventsCallCount++;
|
||||
return CountingEventStream(ToAsyncStream(EventsSource is null ? Events : EventsSource(startUtc, endUtc).ToList(), ThrowOnRead, ct));
|
||||
lock (EventReadCaps) EventReadCaps.Add(maxEvents);
|
||||
if (MaxEventsCeiling is { } ceiling && maxEvents > ceiling)
|
||||
{
|
||||
// The real gateway's refusal (HistorianReadService.ReadEvents: max_events over MaxEventsCeiling), raised before
|
||||
// any row, as the client package surfaces it.
|
||||
var rpc = new Grpc.Core.RpcException(new Grpc.Core.Status(Grpc.Core.StatusCode.InvalidArgument,
|
||||
$"max_events exceeds the server ceiling of {ceiling}."));
|
||||
return CountingEventStream(ToAsyncStream(Array.Empty<HistorianEvent>(),
|
||||
new ZB.MOM.WW.HistorianGateway.Client.HistorianGatewayException(rpc.Status.Detail, rpc), ct));
|
||||
}
|
||||
IReadOnlyList<HistorianEvent> items = EventsSource is null ? Events : EventsSource(startUtc, endUtc).ToList();
|
||||
// A current gateway stops at max_events and completes the stream normally (historiangw#53); IgnoreMaxEvents
|
||||
// simulates an older gateway that streams everything.
|
||||
if (maxEvents > 0 && !IgnoreMaxEvents && items.Count > maxEvents) items = items.Take(maxEvents).ToList();
|
||||
return CountingEventStream(ToAsyncStream(items, ThrowOnRead, ct));
|
||||
}
|
||||
|
||||
/// <summary>Simulates a gateway older than <c>max_events</c>: every event in the window is streamed.</summary>
|
||||
public bool IgnoreMaxEvents;
|
||||
|
||||
/// <summary>When set, a ReadEvents whose max_events exceeds it is refused <c>InvalidArgument</c> (MaxEventsCeiling).</summary>
|
||||
public int? MaxEventsCeiling;
|
||||
|
||||
/// <summary>Every ReadEvents call's maxEvents (the server-side max_events sent), in order.</summary>
|
||||
public readonly List<int> EventReadCaps = new();
|
||||
|
||||
// ---- SendEvents ----------------------------------------------------------------------------
|
||||
/// <summary>
|
||||
/// When set, SendEvents is served by it. Unset, SendEvents answers <c>Unimplemented</c> (the seam's default — an
|
||||
/// older gateway), so the writer falls back to SendEvent.
|
||||
/// </summary>
|
||||
public Func<IReadOnlyList<HistorianEvent>, CancellationToken, Task<SendEventsReply>>? SendEventsSource;
|
||||
|
||||
/// <summary>Every SendEvents call's events, in order.</summary>
|
||||
public readonly List<IReadOnlyList<HistorianEvent>> SendEventsCalls = new();
|
||||
|
||||
public Task<SendEventsReply> SendEventsAsync(IReadOnlyList<HistorianEvent> events, CancellationToken ct)
|
||||
{
|
||||
lock (SendEventsCalls) SendEventsCalls.Add(events);
|
||||
if (SendEventsSource is { } source) return source(events, ct);
|
||||
return Task.FromException<SendEventsReply>(new Grpc.Core.RpcException(
|
||||
new Grpc.Core.Status(Grpc.Core.StatusCode.Unimplemented, "Method is unimplemented.")));
|
||||
}
|
||||
|
||||
public Task<WriteAck> WriteLiveValuesAsync(
|
||||
|
||||
+184
@@ -0,0 +1,184 @@
|
||||
using Grpc.Core;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.HistorianGateway.Client;
|
||||
using ZB.MOM.WW.HistorianGateway.Contracts.Grpc;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.AlarmHistorian;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// lmxopcua#746: alarm-history writes go to the gateway as <c>SendEvents</c> batches (Contracts/Client 0.4.0+) — one
|
||||
/// round trip per <see cref="GatewayAlarmHistorianWriter.MaxEventsPerBatch"/> events — with per-event outcomes in input
|
||||
/// order. A failed per-event result is classified by a single <c>SendEvent</c> re-send; a refused batch is sent event by
|
||||
/// event (a poison event dead-letters alone); a systemic fault retries the rest without sending more; an older gateway
|
||||
/// (<c>Unimplemented</c>) falls back to single sends for good.
|
||||
/// </summary>
|
||||
public sealed class GatewayAlarmWriterBatchTests
|
||||
{
|
||||
private static CancellationToken Ct => TestContext.Current.CancellationToken;
|
||||
|
||||
private static AlarmHistorianEvent Evt(int i) => new(
|
||||
$"alarm-{i}", "Area/Pump", "N", "LimitAlarm", AlarmSeverity.High, "Activated", $"m{i}", "u", null,
|
||||
new DateTime(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc).AddSeconds(i));
|
||||
|
||||
private static int MessageIndex(HistorianEvent e) => int.Parse(e.Properties["Message"][1..]);
|
||||
|
||||
private static SendEventsReply AllOk(IReadOnlyList<HistorianEvent> events)
|
||||
{
|
||||
var reply = new SendEventsReply();
|
||||
foreach (var e in events) reply.Results.Add(new EventWriteResult { Id = e.Id, Success = true });
|
||||
return reply;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_batch_goes_out_as_SendEvents_calls_of_at_most_the_batch_size_with_no_single_sends()
|
||||
{
|
||||
var fake = new FakeHistorianGatewayClient { SendEventsSource = (events, _) => Task.FromResult(AllOk(events)) };
|
||||
using var writer = new GatewayAlarmHistorianWriter(fake, NullLogger<GatewayAlarmHistorianWriter>.Instance);
|
||||
const int n = 407;
|
||||
|
||||
var outcomes = await writer.WriteBatchAsync(Enumerable.Range(0, n).Select(Evt).ToList(), Ct);
|
||||
|
||||
outcomes.ShouldAllBe(o => o == HistorianWriteOutcome.Ack);
|
||||
outcomes.Count.ShouldBe(n);
|
||||
// 200 per call: well inside the gateway's 10,000-event / 16 MiB request limits and the 4 MiB gRPC message default.
|
||||
fake.SendEventsCalls.Select(c => c.Count).ShouldBe([200, 200, 7]);
|
||||
fake.SendEventsCalls.SelectMany(c => c).Select(MessageIndex).ShouldBe(Enumerable.Range(0, n), "request order = input order");
|
||||
fake.SendEventCallCount.ShouldBe(0, "no per-event round trips when the batch RPC answers");
|
||||
writer.BatchUnsupported.ShouldBeFalse();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Queued_results_ack_and_a_failed_result_is_classified_by_a_single_re_send()
|
||||
{
|
||||
var fake = new FakeHistorianGatewayClient
|
||||
{
|
||||
SendEventsSource = (events, _) =>
|
||||
{
|
||||
var reply = new SendEventsReply();
|
||||
foreach (var e in events)
|
||||
reply.Results.Add(MessageIndex(e) switch
|
||||
{
|
||||
1 => new EventWriteResult { Id = e.Id, Success = true, Queued = true },
|
||||
2 or 3 => new EventWriteResult { Id = e.Id, Success = false, Error = "Historian did not accept the write." },
|
||||
_ => new EventWriteResult { Id = e.Id, Success = true },
|
||||
});
|
||||
return Task.FromResult(reply);
|
||||
},
|
||||
// The single re-send: event 2 is rejected by the historian (FailedPrecondition, as SendEvent answers a Failed
|
||||
// write), event 3 is transient.
|
||||
SendEventSource = (e, _) => MessageIndex(e) == 2
|
||||
? Task.FromException<WriteAck>(new RpcException(new Status(StatusCode.FailedPrecondition, "rejected")))
|
||||
: Task.FromException<WriteAck>(new HistorianGatewayUnavailableException("down", new RpcException(new Status(StatusCode.Unavailable, "down")))),
|
||||
};
|
||||
using var writer = new GatewayAlarmHistorianWriter(fake, NullLogger<GatewayAlarmHistorianWriter>.Instance);
|
||||
|
||||
var outcomes = await writer.WriteBatchAsync(Enumerable.Range(0, 5).Select(Evt).ToList(), Ct);
|
||||
|
||||
outcomes.ShouldBe([
|
||||
HistorianWriteOutcome.Ack, HistorianWriteOutcome.Ack, HistorianWriteOutcome.PermanentFail,
|
||||
HistorianWriteOutcome.RetryPlease, HistorianWriteOutcome.Ack,
|
||||
]);
|
||||
fake.SendEventsCalls.Count.ShouldBe(1);
|
||||
fake.SendEventCallCount.ShouldBe(2, "only the two failed events are re-sent");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_refused_batch_is_sent_event_by_event_so_a_poison_event_dead_letters_alone()
|
||||
{
|
||||
var fake = new FakeHistorianGatewayClient
|
||||
{
|
||||
// The gateway validates the whole request first: one bad event refuses the batch, nothing sent.
|
||||
SendEventsSource = (_, _) => Task.FromException<SendEventsReply>(
|
||||
new HistorianGatewayException("events[2].id must be empty or a GUID.",
|
||||
new RpcException(new Status(StatusCode.InvalidArgument, "events[2].id must be empty or a GUID.")))),
|
||||
SendEventSource = (e, _) => MessageIndex(e) == 2
|
||||
? Task.FromException<WriteAck>(new RpcException(new Status(StatusCode.InvalidArgument, "id must be empty or a GUID.")))
|
||||
: Task.FromResult(new WriteAck { Success = true }),
|
||||
};
|
||||
using var writer = new GatewayAlarmHistorianWriter(fake, NullLogger<GatewayAlarmHistorianWriter>.Instance);
|
||||
|
||||
var outcomes = await writer.WriteBatchAsync(Enumerable.Range(0, 4).Select(Evt).ToList(), Ct);
|
||||
|
||||
outcomes.ShouldBe([HistorianWriteOutcome.Ack, HistorianWriteOutcome.Ack, HistorianWriteOutcome.PermanentFail, HistorianWriteOutcome.Ack]);
|
||||
fake.SendEventCallCount.ShouldBe(4);
|
||||
writer.BatchUnsupported.ShouldBeFalse("a refused batch is not an older gateway — the next drain batches again");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_systemic_fault_retries_the_rest_and_sends_nothing_more()
|
||||
{
|
||||
var calls = 0;
|
||||
var fake = new FakeHistorianGatewayClient
|
||||
{
|
||||
SendEventsSource = (events, _) => Interlocked.Increment(ref calls) == 1
|
||||
? Task.FromResult(AllOk(events))
|
||||
: Task.FromException<SendEventsReply>(new HistorianGatewayUnavailableException("down",
|
||||
new RpcException(new Status(StatusCode.Unavailable, "down")))),
|
||||
};
|
||||
using var writer = new GatewayAlarmHistorianWriter(fake, NullLogger<GatewayAlarmHistorianWriter>.Instance);
|
||||
var size = GatewayAlarmHistorianWriter.MaxEventsPerBatch;
|
||||
|
||||
var outcomes = await writer.WriteBatchAsync(Enumerable.Range(0, size * 3).Select(Evt).ToList(), Ct);
|
||||
|
||||
outcomes.Take(size).ShouldAllBe(o => o == HistorianWriteOutcome.Ack);
|
||||
outcomes.Skip(size).ShouldAllBe(o => o == HistorianWriteOutcome.RetryPlease);
|
||||
calls.ShouldBe(2, "the third chunk would hit the same outage");
|
||||
fake.SendEventCallCount.ShouldBe(0, "a systemic fault must not fan out into per-event sends");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task An_older_gateway_falls_back_to_single_sends_for_good()
|
||||
{
|
||||
// No SendEventsSource: the fake answers Unimplemented, as a gateway older than the RPC.
|
||||
var fake = new FakeHistorianGatewayClient();
|
||||
using var writer = new GatewayAlarmHistorianWriter(fake, NullLogger<GatewayAlarmHistorianWriter>.Instance);
|
||||
|
||||
(await writer.WriteBatchAsync(Enumerable.Range(0, 5).Select(Evt).ToList(), Ct)).ShouldAllBe(o => o == HistorianWriteOutcome.Ack);
|
||||
(await writer.WriteBatchAsync(Enumerable.Range(5, 5).Select(Evt).ToList(), Ct)).ShouldAllBe(o => o == HistorianWriteOutcome.Ack);
|
||||
|
||||
writer.BatchUnsupported.ShouldBeTrue();
|
||||
fake.SendEventsCalls.Count.ShouldBe(1, "Unimplemented is remembered — the second drain batch goes straight to single sends");
|
||||
fake.SendEventCallCount.ShouldBe(10);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_cancellation_during_the_batch_call_retries_and_dead_letters_nothing()
|
||||
{
|
||||
using var cts = new CancellationTokenSource();
|
||||
var fake = new FakeHistorianGatewayClient
|
||||
{
|
||||
SendEventsSource = async (_, ct) =>
|
||||
{
|
||||
await cts.CancelAsync();
|
||||
ct.ThrowIfCancellationRequested();
|
||||
return new SendEventsReply();
|
||||
},
|
||||
};
|
||||
using var writer = new GatewayAlarmHistorianWriter(fake, NullLogger<GatewayAlarmHistorianWriter>.Instance);
|
||||
|
||||
var outcomes = await writer.WriteBatchAsync(Enumerable.Range(0, GatewayAlarmHistorianWriter.MaxEventsPerBatch + 3).Select(Evt).ToList(), cts.Token);
|
||||
|
||||
outcomes.ShouldAllBe(o => o == HistorianWriteOutcome.RetryPlease);
|
||||
fake.SendEventsCalls.Count.ShouldBe(1);
|
||||
fake.SendEventCallCount.ShouldBe(0);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_short_reply_classifies_the_missing_events_by_single_sends()
|
||||
{
|
||||
var fake = new FakeHistorianGatewayClient
|
||||
{
|
||||
SendEventsSource = (events, _) => Task.FromResult(AllOk(events.Take(2).ToList())),
|
||||
};
|
||||
using var writer = new GatewayAlarmHistorianWriter(fake, NullLogger<GatewayAlarmHistorianWriter>.Instance);
|
||||
|
||||
var outcomes = await writer.WriteBatchAsync(Enumerable.Range(0, 4).Select(Evt).ToList(), Ct);
|
||||
|
||||
outcomes.ShouldAllBe(o => o == HistorianWriteOutcome.Ack);
|
||||
fake.SendEventCallCount.ShouldBe(2);
|
||||
}
|
||||
}
|
||||
+11
-9
@@ -8,9 +8,11 @@ using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// The Gateway events read drains its stream past the cap (within a budget) instead of abandoning it — an abandoned
|
||||
/// gateway stream cancels the RPC and the gateway evicts its pooled historian session — and a caller cancellation on
|
||||
/// the events / native-AtTime arms is not a backend failure.
|
||||
/// Against a stream that runs past the <c>max_events</c> it was sent (a seam that ignores it), or the uncapped fallback
|
||||
/// read, the Gateway events read drains its stream past the cap (within a budget) instead of abandoning it — an abandoned
|
||||
/// gateway stream cancels the RPC and the gateway evicts its pooled historian session — and a caller cancellation on the
|
||||
/// events / native-AtTime arms is not a backend failure. The server-side <c>max_events</c> path is
|
||||
/// <see cref="GatewayServerCappedEventsTests"/>.
|
||||
/// </summary>
|
||||
public sealed class GatewayEventDrainTests
|
||||
{
|
||||
@@ -20,7 +22,7 @@ public sealed class GatewayEventDrainTests
|
||||
[Fact]
|
||||
public async Task A_capped_read_drains_the_stream_to_its_end_and_still_flags_truncation()
|
||||
{
|
||||
var fake = new FakeHistorianGatewayClient { Events = Series(50) };
|
||||
var fake = new FakeHistorianGatewayClient { Events = Series(50), IgnoreMaxEvents = true };
|
||||
var ds = new GatewayHistorianDataSource(fake, NullLogger<GatewayHistorianDataSource>.Instance);
|
||||
|
||||
var r = await ds.ReadEventsAsync(null, T0, T0.AddHours(1), maxEvents: 10, Ct);
|
||||
@@ -36,7 +38,7 @@ public sealed class GatewayEventDrainTests
|
||||
[Fact]
|
||||
public async Task Past_the_drain_budget_the_stream_is_abandoned_but_the_page_is_unchanged()
|
||||
{
|
||||
var fake = new FakeHistorianGatewayClient { Events = Series(100) };
|
||||
var fake = new FakeHistorianGatewayClient { Events = Series(100), IgnoreMaxEvents = true };
|
||||
var ds = new GatewayHistorianDataSource(fake, NullLogger<GatewayHistorianDataSource>.Instance, eventDrainBudget: 5);
|
||||
|
||||
var r = await ds.ReadEventsAsync(null, T0, T0.AddHours(1), maxEvents: 10, Ct);
|
||||
@@ -44,21 +46,21 @@ public sealed class GatewayEventDrainTests
|
||||
r.Events.Count.ShouldBe(10);
|
||||
r.ContinuationPoint.ShouldNotBeNull();
|
||||
fake.EventStreamsAbandoned.ShouldBe(1);
|
||||
fake.EventsYieldedCount.ShouldBe(10 + 5 + 1, "cap + budget, then the row that overran it");
|
||||
fake.EventsYieldedCount.ShouldBe(10 + 1 + 5 + 1, "the max_events it was sent (cap + 1), the budget, then the row that overran it");
|
||||
ds.EventStreamsAbandoned.ShouldBe(1);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Budget_zero_is_the_previous_abandon_at_the_cap()
|
||||
{
|
||||
var fake = new FakeHistorianGatewayClient { Events = Series(20) };
|
||||
var fake = new FakeHistorianGatewayClient { Events = Series(20), IgnoreMaxEvents = true };
|
||||
var ds = new GatewayHistorianDataSource(fake, NullLogger<GatewayHistorianDataSource>.Instance, eventDrainBudget: 0);
|
||||
|
||||
var r = await ds.ReadEventsAsync(null, T0, T0.AddHours(1), maxEvents: 3, Ct);
|
||||
|
||||
r.Events.Count.ShouldBe(3);
|
||||
r.ContinuationPoint.ShouldNotBeNull();
|
||||
fake.EventsYieldedCount.ShouldBe(4);
|
||||
fake.EventsYieldedCount.ShouldBe(5, "the max_events it was sent (cap + 1), then the row past it");
|
||||
fake.EventStreamsAbandoned.ShouldBe(1);
|
||||
}
|
||||
|
||||
@@ -67,7 +69,7 @@ public sealed class GatewayEventDrainTests
|
||||
{
|
||||
// The page is exactly the matching events; everything after it belongs to another source (an unfiltered gateway).
|
||||
var events = Series(3, "Pump1").Concat(Series(5, "Pump2", offsetSeconds: 10)).ToArray();
|
||||
var fake = new FakeHistorianGatewayClient { Events = events };
|
||||
var fake = new FakeHistorianGatewayClient { Events = events, IgnoreMaxEvents = true };
|
||||
var ds = new GatewayHistorianDataSource(fake, NullLogger<GatewayHistorianDataSource>.Instance);
|
||||
|
||||
var r = await ds.ReadEventsAsync("Pump1", T0, T0.AddHours(1), maxEvents: 3, Ct);
|
||||
|
||||
+3
-2
@@ -15,8 +15,9 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests;
|
||||
/// first); a forward read's clipped LATEST interval is read over exactly its span (review). The fake is a native-mode AVEVA
|
||||
/// model: a step-held stored series; for a forward request <c>[s, e)</c> with interval <c>I</c> it answers one row per
|
||||
/// interval of the grid anchored at <c>s</c> (plus AVEVA's extra row for the interval before <c>s</c>, #685), each the
|
||||
/// time-weighted average / integral of exactly that interval — the per-interval behaviour the gateway will have once
|
||||
/// historiangw#27 is fixed.
|
||||
/// time-weighted average / integral of exactly that interval — the per-interval behaviour the gateway has since
|
||||
/// historiangw#27 (live-verified, lmxopcua#694). The gateway's whole-millisecond interval rounding is
|
||||
/// <see cref="GatewayNativeWholeMillisecondGridTests"/>.
|
||||
/// </summary>
|
||||
public sealed class GatewayNativeAggregateGridTests
|
||||
{
|
||||
|
||||
+18
-13
@@ -9,9 +9,9 @@ using ZB.MOM.WW.OtOpcUa.Runtime.Historian;
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// AVEVA's native TimeAverage / Total answer Good-status wrong values until the gateway's interval fix
|
||||
/// (historiangw#27) is live-verified, so they are refused <c>BadAggregateNotSupported</c> unless
|
||||
/// <c>ServerHistorian:Gateway:EnableNativeTimeWeighted</c> opts back in.
|
||||
/// AVEVA's native TimeAverage / Total are served by default since the gateway's interval fix (historiangw#27) was
|
||||
/// live-verified (lmxopcua#694); <c>ServerHistorian:Gateway:EnableNativeTimeWeighted=false</c> is the kill switch that
|
||||
/// refuses them <c>BadAggregateNotSupported</c> (for a gateway older than the fix).
|
||||
/// </summary>
|
||||
public sealed class GatewayNativeTimeWeightedGateTests
|
||||
{
|
||||
@@ -21,10 +21,10 @@ public sealed class GatewayNativeTimeWeightedGateTests
|
||||
[Theory]
|
||||
[InlineData(HistoryAggregateType.TimeAverage)]
|
||||
[InlineData(HistoryAggregateType.Total)]
|
||||
public async Task By_default_the_native_time_weighted_pair_is_refused_unsent_and_unadvertised(HistoryAggregateType a)
|
||||
public async Task The_kill_switch_refuses_the_native_time_weighted_pair_unsent_and_unadvertised(HistoryAggregateType a)
|
||||
{
|
||||
var fake = new FakeHistorianGatewayClient();
|
||||
var ds = new GatewayHistorianDataSource(fake, NullLogger<GatewayHistorianDataSource>.Instance);
|
||||
var ds = new GatewayHistorianDataSource(fake, NullLogger<GatewayHistorianDataSource>.Instance, enableNativeTimeWeighted: false);
|
||||
|
||||
ds.SupportedAggregates.ShouldNotContain(a);
|
||||
await Should.ThrowAsync<HistoryAggregateNotSupportedException>(
|
||||
@@ -43,10 +43,10 @@ public sealed class GatewayNativeTimeWeightedGateTests
|
||||
[Theory]
|
||||
[InlineData(HistoryAggregateType.TimeAverage)]
|
||||
[InlineData(HistoryAggregateType.Total)]
|
||||
public async Task Opting_in_serves_and_advertises_the_native_pair(HistoryAggregateType a)
|
||||
public async Task By_default_the_native_pair_is_served_and_advertised(HistoryAggregateType a)
|
||||
{
|
||||
var fake = new FakeHistorianGatewayClient();
|
||||
var ds = new GatewayHistorianDataSource(fake, NullLogger<GatewayHistorianDataSource>.Instance, enableNativeTimeWeighted: true);
|
||||
var ds = new GatewayHistorianDataSource(fake, NullLogger<GatewayHistorianDataSource>.Instance);
|
||||
|
||||
ds.SupportedAggregates.ShouldBe(GatewayHistorianDataSource.ServedAggregates);
|
||||
await ds.ReadProcessedAsync("T", T0, T0.AddHours(1), TimeSpan.FromMinutes(1), a, Ct);
|
||||
@@ -54,17 +54,22 @@ public sealed class GatewayNativeTimeWeightedGateTests
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void The_default_set_is_the_ten_raw_evaluated_aggregates()
|
||||
public void The_default_set_is_all_twelve_and_the_kill_switch_leaves_the_ten_raw_evaluated()
|
||||
{
|
||||
var ds = new GatewayHistorianDataSource(new FakeHistorianGatewayClient(), NullLogger<GatewayHistorianDataSource>.Instance);
|
||||
ds.SupportedAggregates.Count.ShouldBe(10);
|
||||
ds.SupportedAggregates.ShouldBe(GatewayHistorianDataSource.RawEvaluatedAggregates);
|
||||
GatewayHistorianDataSource.ServedAggregates.Except(ds.SupportedAggregates).OrderBy(x => x)
|
||||
var on = new GatewayHistorianDataSource(new FakeHistorianGatewayClient(), NullLogger<GatewayHistorianDataSource>.Instance);
|
||||
on.SupportedAggregates.Count.ShouldBe(12);
|
||||
on.SupportedAggregates.ShouldBe(GatewayHistorianDataSource.ServedAggregates);
|
||||
|
||||
var off = new GatewayHistorianDataSource(new FakeHistorianGatewayClient(), NullLogger<GatewayHistorianDataSource>.Instance,
|
||||
enableNativeTimeWeighted: false);
|
||||
off.SupportedAggregates.Count.ShouldBe(10);
|
||||
off.SupportedAggregates.ShouldBe(GatewayHistorianDataSource.RawEvaluatedAggregates);
|
||||
GatewayHistorianDataSource.ServedAggregates.Except(off.SupportedAggregates).OrderBy(x => x)
|
||||
.ShouldBe([HistoryAggregateType.Total, HistoryAggregateType.TimeAverage]);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(null, false)]
|
||||
[InlineData(null, true)]
|
||||
[InlineData("false", false)]
|
||||
[InlineData("true", true)]
|
||||
public void The_flag_binds_from_configuration_and_reaches_the_data_source(string? configured, bool expected)
|
||||
|
||||
+180
@@ -0,0 +1,180 @@
|
||||
using Google.Protobuf.WellKnownTypes;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.HistorianGateway.Contracts.Grpc;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// lmxopcua#694: the gateway sends AVEVA the native interval in WHOLE milliseconds (<c>max(1, round(ms))</c>) and AVEVA
|
||||
/// clips its last interval at the request's end — both observed live (a 1.5 ms interval answers on a 2 ms grid; a 0.4 ms
|
||||
/// window answers one row over exactly 0.4 ms). So a multi-interval request is only correct for a whole-millisecond
|
||||
/// interval: a fractional one is read one interval per request, up to
|
||||
/// <see cref="NativeAggregatePlan.MaxPerIntervalRequests"/>, and every one-interval request is sent with its width
|
||||
/// rounded UP to whole milliseconds so AVEVA answers it as ONE clipped row. The fake models exactly that gateway + AVEVA
|
||||
/// behaviour (the gateway's round-half-to-even included) over a step-held series.
|
||||
/// </summary>
|
||||
public sealed class GatewayNativeWholeMillisecondGridTests
|
||||
{
|
||||
private static readonly CancellationToken Ct = CancellationToken.None;
|
||||
private static readonly DateTime T0 = new(2026, 11, 11, 14, 2, 3, DateTimeKind.Utc);
|
||||
private const uint GoodCalc = 0x00000401u;
|
||||
private const uint Partial = 0x4u;
|
||||
|
||||
/// <summary>A value that changes every 0.5 ms (1, 2, 3, …): any wrong grid shows in the values.</summary>
|
||||
private static double ValueAt(DateTime t) => 1 + Math.Floor((t - T0).Ticks / 5000.0);
|
||||
|
||||
private static double Integral(DateTime a, DateTime b)
|
||||
{
|
||||
double sum = 0;
|
||||
for (var t = a; t < b;)
|
||||
{
|
||||
var next = T0.AddTicks(((t - T0).Ticks / 5000 + 1) * 5000);
|
||||
if (next > b) next = b;
|
||||
sum += ValueAt(t) * (next - t).TotalSeconds;
|
||||
t = next;
|
||||
}
|
||||
return sum;
|
||||
}
|
||||
|
||||
private static double Expected(RetrievalMode mode, DateTime a, DateTime b) =>
|
||||
mode == RetrievalMode.Integral ? Integral(a, b) : Integral(a, b) / (b - a).TotalSeconds;
|
||||
|
||||
/// <summary>The gateway (whole-ms interval) in front of AVEVA (grid anchored at the request start, last interval clipped).</summary>
|
||||
private static FakeHistorianGatewayClient GatewayAndAveva() => new()
|
||||
{
|
||||
AggregateSource = (s, e, mode, interval) =>
|
||||
{
|
||||
var wire = TimeSpan.FromMilliseconds(Math.Max(1, Math.Round(interval.TotalMilliseconds)));
|
||||
var rows = new List<HistorianAggregateSample>();
|
||||
for (var lo = s - wire; lo < e; lo += wire) // AVEVA's extra pre-window row first
|
||||
{
|
||||
var hi = lo + wire > e && lo >= s ? e : lo + wire;
|
||||
rows.Add(new HistorianAggregateSample
|
||||
{
|
||||
Tag = "T",
|
||||
StartTime = Timestamp.FromDateTime(lo),
|
||||
EndTime = Timestamp.FromDateTime(hi),
|
||||
Value = Expected(mode, lo, hi),
|
||||
OpcQuality = 192,
|
||||
RetrievalMode = mode,
|
||||
});
|
||||
}
|
||||
return rows;
|
||||
},
|
||||
};
|
||||
|
||||
private static GatewayHistorianDataSource Source(FakeHistorianGatewayClient fake) =>
|
||||
new(fake, NullLogger<GatewayHistorianDataSource>.Instance);
|
||||
|
||||
[Theory]
|
||||
[InlineData(HistoryAggregateType.TimeAverage, RetrievalMode.TimeWeightedAverage)]
|
||||
[InlineData(HistoryAggregateType.Total, RetrievalMode.Integral)]
|
||||
public async Task A_fractional_millisecond_interval_is_read_one_interval_per_request(HistoryAggregateType agg, RetrievalMode mode)
|
||||
{
|
||||
var fake = GatewayAndAveva();
|
||||
var interval = TimeSpan.FromTicks(15_000); // 1.5 ms — the gateway would send 2 ms
|
||||
var s = (await Source(fake).ReadProcessedAsync("T", T0, T0.AddMilliseconds(6), interval, agg, Ct)).Samples;
|
||||
|
||||
s.Select(x => x.SourceTimestampUtc).ShouldBe(new DateTime?[] { T0, T0.AddTicks(15_000), T0.AddTicks(30_000), T0.AddTicks(45_000) });
|
||||
s.Select(x => (double)x.Value!).ShouldBe(
|
||||
Enumerable.Range(0, 4).Select(i => Expected(mode, T0.AddTicks(i * 15_000L), T0.AddTicks((i + 1) * 15_000L))).ToList(),
|
||||
tolerance: 1e-9);
|
||||
s.ShouldAllBe(x => x.StatusCode == GoodCalc);
|
||||
fake.AggregateCalls.Select(c => (c.Start, c.End)).ShouldBe(
|
||||
Enumerable.Range(0, 4).Select(i => (T0.AddTicks(i * 15_000L), T0.AddTicks((i + 1) * 15_000L))).ToList());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_fractional_interval_reverse_read_is_exact_too()
|
||||
{
|
||||
var fake = GatewayAndAveva();
|
||||
var m = RetrievalMode.TimeWeightedAverage;
|
||||
var s = (await Source(fake).ReadProcessedAsync("T", T0.AddTicks(40_000), T0, TimeSpan.FromTicks(15_000),
|
||||
HistoryAggregateType.TimeAverage, Ct)).Samples;
|
||||
|
||||
// (0, 4 ms] stepping back from 4 ms: (2.5, 4], (1, 2.5], and the clipped (0, 1] — newest first, later edges.
|
||||
s.Select(x => x.SourceTimestampUtc).ShouldBe(new DateTime?[] { T0.AddTicks(40_000), T0.AddTicks(25_000), T0.AddTicks(10_000) });
|
||||
s.Select(x => (double)x.Value!).ShouldBe(
|
||||
[Expected(m, T0.AddTicks(25_000), T0.AddTicks(40_000)), Expected(m, T0.AddTicks(10_000), T0.AddTicks(25_000)),
|
||||
Expected(m, T0, T0.AddTicks(10_000))],
|
||||
tolerance: 1e-9);
|
||||
s.Select(x => x.StatusCode).ShouldBe([GoodCalc, GoodCalc, GoodCalc | Partial]);
|
||||
fake.ReadAggregateCallCount.ShouldBe(3);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_whole_millisecond_interval_stays_one_request_with_a_sub_millisecond_clipped_interval()
|
||||
{
|
||||
var fake = GatewayAndAveva();
|
||||
var m = RetrievalMode.Integral;
|
||||
var s = (await Source(fake).ReadProcessedAsync("T", T0, T0.AddTicks(30_004), TimeSpan.FromMilliseconds(1),
|
||||
HistoryAggregateType.Total, Ct)).Samples;
|
||||
|
||||
s.Count.ShouldBe(4);
|
||||
s.Select(x => (double)x.Value!).ShouldBe(
|
||||
[Expected(m, T0, T0.AddMilliseconds(1)), Expected(m, T0.AddMilliseconds(1), T0.AddMilliseconds(2)),
|
||||
Expected(m, T0.AddMilliseconds(2), T0.AddMilliseconds(3)), Expected(m, T0.AddMilliseconds(3), T0.AddTicks(30_004))],
|
||||
tolerance: 1e-12);
|
||||
s[^1].StatusCode.ShouldBe(GoodCalc | Partial);
|
||||
fake.AggregateCalls.Select(c => (c.Start, c.End, c.Interval)).ShouldBe([
|
||||
(T0, T0.AddMilliseconds(3), TimeSpan.FromMilliseconds(1)),
|
||||
(T0.AddMilliseconds(3), T0.AddTicks(30_004), TimeSpan.FromMilliseconds(1)), // the 0.4 µs width, rounded UP to 1 ms
|
||||
]);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(HistoryAggregateType.TimeAverage, RetrievalMode.TimeWeightedAverage)]
|
||||
[InlineData(HistoryAggregateType.Total, RetrievalMode.Integral)]
|
||||
public async Task A_clipped_interval_that_is_not_whole_milliseconds_is_one_row_over_its_whole_span(HistoryAggregateType agg,
|
||||
RetrievalMode mode)
|
||||
{
|
||||
// Live (lmxopcua#694): [40 s, 43.0004 s) sent at its own width reached AVEVA as 3,000 ms and answered a 3 s row plus a
|
||||
// 0.4 ms one; the first won, so the clipped Total lost its last 0.4 ms. Here: 4 ms intervals, a 2.0004 ms clipped one.
|
||||
var fake = GatewayAndAveva();
|
||||
var end = T0.AddTicks(60_004);
|
||||
var s = (await Source(fake).ReadProcessedAsync("T", T0, end, TimeSpan.FromMilliseconds(4), agg, Ct)).Samples;
|
||||
|
||||
s.Count.ShouldBe(2);
|
||||
((double)s[1].Value!).ShouldBe(Expected(mode, T0.AddMilliseconds(4), end), 1e-12);
|
||||
s[1].StatusCode.ShouldBe(GoodCalc | Partial);
|
||||
fake.AggregateCalls[^1].ShouldBe((T0.AddMilliseconds(4), end, mode, TimeSpan.FromMilliseconds(3)));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_half_millisecond_interval_rounding_to_even_still_reads_each_interval_whole()
|
||||
{
|
||||
// 2.5 ms rounds (to even) to 2 ms on the gateway: sent at its width, each interval would answer a 2 ms row + 0.5 ms.
|
||||
var fake = GatewayAndAveva();
|
||||
var m = RetrievalMode.Integral;
|
||||
var s = (await Source(fake).ReadProcessedAsync("T", T0, T0.AddMilliseconds(5), TimeSpan.FromTicks(25_000),
|
||||
HistoryAggregateType.Total, Ct)).Samples;
|
||||
|
||||
s.Select(x => (double)x.Value!).ShouldBe(
|
||||
[Expected(m, T0, T0.AddTicks(25_000)), Expected(m, T0.AddTicks(25_000), T0.AddMilliseconds(5))], tolerance: 1e-12);
|
||||
fake.AggregateCalls.Select(c => c.Interval).ShouldAllBe(i => i == TimeSpan.FromMilliseconds(3));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Too_many_fractional_intervals_are_a_request_limit_unsent_and_not_a_backend_failure()
|
||||
{
|
||||
var fake = GatewayAndAveva();
|
||||
var ds = Source(fake);
|
||||
var interval = TimeSpan.FromTicks(15_000);
|
||||
var end = T0.AddTicks(15_000L * (NativeAggregatePlan.MaxPerIntervalRequests + 1));
|
||||
|
||||
var ex = await Should.ThrowAsync<HistoryReadLimitExceededException>(
|
||||
() => ds.ReadProcessedAsync("T", T0, end, interval, HistoryAggregateType.TimeAverage, Ct));
|
||||
ex.Limit.ShouldBe(NativeAggregatePlan.MaxPerIntervalRequests);
|
||||
fake.ReadAggregateCallCount.ShouldBe(0);
|
||||
fake.TagExistsCallCount.ShouldBe(0);
|
||||
ds.GetHealthSnapshot().TotalFailures.ShouldBe(0);
|
||||
|
||||
// Positive control: exactly the limit is served.
|
||||
var ok = await ds.ReadProcessedAsync("T", T0, T0.AddTicks(15_000L * NativeAggregatePlan.MaxPerIntervalRequests), interval,
|
||||
HistoryAggregateType.TimeAverage, Ct);
|
||||
ok.Samples.Count.ShouldBe(NativeAggregatePlan.MaxPerIntervalRequests);
|
||||
}
|
||||
}
|
||||
+129
@@ -0,0 +1,129 @@
|
||||
using Google.Protobuf.WellKnownTypes;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.HistorianGateway.Contracts.Grpc;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// lmxopcua#746: a capped Gateway events read sends cap + 1 as the server-side <c>ReadEventsRequest.max_events</c>, so a
|
||||
/// current gateway completes the stream normally at the cap (no cancelled RPC, no evicted historian session) and the
|
||||
/// (cap + 1)-th event is the "more exist" signal. An ambiguous capped answer (foreign rows filtered out of a stream the
|
||||
/// server cut) and a refused cap (InvalidArgument) are re-read without a server cap. The fake honours
|
||||
/// <c>max_events</c> the way the gateway does (stop at it, complete normally).
|
||||
/// </summary>
|
||||
public sealed class GatewayServerCappedEventsTests
|
||||
{
|
||||
private static readonly DateTime T0 = new(2026, 11, 12, 8, 0, 0, DateTimeKind.Utc);
|
||||
private static CancellationToken Ct => TestContext.Current.CancellationToken;
|
||||
|
||||
[Fact]
|
||||
public async Task A_capped_read_asks_the_server_for_cap_plus_one_and_never_abandons_the_stream()
|
||||
{
|
||||
// Budget 0 = abandon at the cap on the drain path: only the server cap keeps this stream from being abandoned.
|
||||
var fake = new FakeHistorianGatewayClient { Events = Series(50) };
|
||||
var ds = new GatewayHistorianDataSource(fake, NullLogger<GatewayHistorianDataSource>.Instance, eventDrainBudget: 0);
|
||||
|
||||
var r = await ds.ReadEventsAsync(null, T0, T0.AddHours(1), maxEvents: 10, Ct);
|
||||
|
||||
r.Events.Select(e => e.EventId).ShouldBe(Enumerable.Range(0, 10).Select(i => $"E{i:000}"));
|
||||
r.ContinuationPoint.ShouldNotBeNull("the 11th event is the 'more exist' signal");
|
||||
fake.EventReadCaps.ShouldBe([11]);
|
||||
fake.EventsYieldedCount.ShouldBe(11, "the server stops at max_events — the window's other 39 events never cross the wire");
|
||||
fake.EventStreamsCompleted.ShouldBe(1);
|
||||
fake.EventStreamsAbandoned.ShouldBe(0);
|
||||
ds.EventStreamsAbandoned.ShouldBe(0);
|
||||
ds.EventServerCapReReads.ShouldBe(0);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_window_holding_exactly_the_cap_is_complete_in_one_read()
|
||||
{
|
||||
var fake = new FakeHistorianGatewayClient { Events = Series(10) };
|
||||
var ds = new GatewayHistorianDataSource(fake, NullLogger<GatewayHistorianDataSource>.Instance);
|
||||
|
||||
var r = await ds.ReadEventsAsync(null, T0, T0.AddHours(1), maxEvents: 10, Ct);
|
||||
|
||||
r.Events.Count.ShouldBe(10);
|
||||
r.ContinuationPoint.ShouldBeNull();
|
||||
fake.EventReadCaps.ShouldBe([11]);
|
||||
fake.ReadEventsCallCount.ShouldBe(1);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Foreign_rows_inside_the_server_cut_are_re_read_without_the_cap()
|
||||
{
|
||||
// The server filter let foreign rows through (the client-side filter drops them): the capped stream ends at
|
||||
// cap + 1 rows without a matching overflow row, so whether more matching events exist is unknown.
|
||||
var events = Series(3, "Pump1").Concat(Series(5, "Pump2", offsetSeconds: 10)).Append(Ev(99, "Pump1")).ToArray();
|
||||
var fake = new FakeHistorianGatewayClient { Events = events };
|
||||
var ds = new GatewayHistorianDataSource(fake, NullLogger<GatewayHistorianDataSource>.Instance);
|
||||
|
||||
var r = await ds.ReadEventsAsync("Pump1", T0, T0.AddHours(1), maxEvents: 3, Ct);
|
||||
|
||||
r.Events.Count.ShouldBe(3);
|
||||
r.ContinuationPoint.ShouldNotBeNull("the late Pump1 event exists past the server's cut");
|
||||
fake.EventReadCaps.ShouldBe([4, 0]);
|
||||
ds.EventServerCapReReads.ShouldBe(1);
|
||||
|
||||
// Negative: without the late matching event the re-read answers complete.
|
||||
fake.Events = events[..^1];
|
||||
fake.EventReadCaps.Clear();
|
||||
(await ds.ReadEventsAsync("Pump1", T0, T0.AddHours(1), maxEvents: 3, Ct)).ContinuationPoint.ShouldBeNull();
|
||||
fake.EventReadCaps.ShouldBe([4, 0]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_refused_server_cap_is_re_read_uncapped_and_is_not_a_backend_failure()
|
||||
{
|
||||
var fake = new FakeHistorianGatewayClient { Events = Series(20), MaxEventsCeiling = 5 };
|
||||
var ds = new GatewayHistorianDataSource(fake, NullLogger<GatewayHistorianDataSource>.Instance);
|
||||
|
||||
var r = await ds.ReadEventsAsync(null, T0, T0.AddHours(1), maxEvents: 10, Ct);
|
||||
|
||||
r.Events.Count.ShouldBe(10);
|
||||
r.ContinuationPoint.ShouldNotBeNull();
|
||||
fake.EventReadCaps.ShouldBe([11, 0]);
|
||||
ds.GetHealthSnapshot().TotalFailures.ShouldBe(0);
|
||||
|
||||
// Positive control: a cap under the ceiling goes out as the server cap and is not re-read.
|
||||
fake.EventReadCaps.Clear();
|
||||
(await ds.ReadEventsAsync(null, T0, T0.AddHours(1), maxEvents: 3, Ct)).Events.Count.ShouldBe(3);
|
||||
fake.EventReadCaps.ShouldBe([4]);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(0)]
|
||||
[InlineData(int.MaxValue)]
|
||||
[InlineData(GatewayHistorianDataSource.MaxServerCappedEvents + 1)]
|
||||
public async Task Uncapped_and_over_ceiling_reads_send_no_server_cap(int maxEvents)
|
||||
{
|
||||
var fake = new FakeHistorianGatewayClient { Events = Series(5) };
|
||||
var ds = new GatewayHistorianDataSource(fake, NullLogger<GatewayHistorianDataSource>.Instance);
|
||||
|
||||
(await ds.ReadEventsAsync(null, T0, T0.AddHours(1), maxEvents, Ct)).Events.Count.ShouldBe(5);
|
||||
fake.EventReadCaps.ShouldBe([0]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task The_largest_server_capped_page_sends_the_gateway_default_ceiling()
|
||||
{
|
||||
var fake = new FakeHistorianGatewayClient { Events = Series(5), MaxEventsCeiling = 1_000_000 };
|
||||
var ds = new GatewayHistorianDataSource(fake, NullLogger<GatewayHistorianDataSource>.Instance);
|
||||
|
||||
await ds.ReadEventsAsync(null, T0, T0.AddHours(1), GatewayHistorianDataSource.MaxServerCappedEvents, Ct);
|
||||
fake.EventReadCaps.ShouldBe([1_000_000], "cap + 1 must stay within the gateway's default MaxEventsCeiling");
|
||||
}
|
||||
|
||||
private static HistorianEvent[] Series(int count, string source = "Pump1", int offsetSeconds = 0) =>
|
||||
Enumerable.Range(0, count).Select(i => Ev(i + offsetSeconds, source, $"E{i:000}")).ToArray();
|
||||
|
||||
private static HistorianEvent Ev(int seconds, string source, string? id = null) => new()
|
||||
{
|
||||
Id = id ?? $"X{seconds}",
|
||||
SourceName = source,
|
||||
EventTime = Timestamp.FromDateTime(T0.AddSeconds(seconds)),
|
||||
ReceivedTime = Timestamp.FromDateTime(T0.AddSeconds(seconds)),
|
||||
};
|
||||
}
|
||||
+1
-1
@@ -51,7 +51,7 @@ public sealed class AggregateModeMapperTests
|
||||
}
|
||||
Assert.Equal(12, GatewayHistorianDataSource.ServedAggregates.Count);
|
||||
|
||||
// The native pair is opt-in (ServerHistorian:Gateway:EnableNativeTimeWeighted) — see GatewayNativeTimeWeightedGateTests.
|
||||
// The native pair follows ServerHistorian:Gateway:EnableNativeTimeWeighted (default on) — see GatewayNativeTimeWeightedGateTests.
|
||||
var source = new GatewayHistorianDataSource(
|
||||
new FakeHistorianGatewayClient(), Microsoft.Extensions.Logging.Abstractions.NullLogger<GatewayHistorianDataSource>.Instance,
|
||||
enableNativeTimeWeighted: true);
|
||||
|
||||
+9
-8
@@ -150,24 +150,25 @@ public sealed class HistorianBackendRegistryTests
|
||||
IHistorianDataSource Source(ServerHistorianOptions o, IConfiguration c) =>
|
||||
HistorianBackendRegistry.Resolve(o, c).CreateDataSource(sp);
|
||||
|
||||
// #681: the ten per-sample aggregates (raw-evaluated). The two AVEVA-native ones (TimeAverage / Total) are
|
||||
// refused and not advertised by default until historiangw#27 is fixed (#746, R08:2.1) ...
|
||||
// #681: the ten per-sample aggregates (raw-evaluated), plus — by default since historiangw#27 was live-verified
|
||||
// (lmxopcua#694) — the two AVEVA-native ones (TimeAverage / Total) ...
|
||||
HistoryAggregateType[] gatewayRawEvaluated =
|
||||
[
|
||||
HistoryAggregateType.Average, HistoryAggregateType.Count, HistoryAggregateType.Minimum,
|
||||
HistoryAggregateType.Maximum, HistoryAggregateType.Range, HistoryAggregateType.Start, HistoryAggregateType.End,
|
||||
HistoryAggregateType.Delta, HistoryAggregateType.MinimumActualTime, HistoryAggregateType.MaximumActualTime,
|
||||
];
|
||||
Source(Gateway(), Cfg).SupportedAggregates.ShouldBe(gatewayRawEvaluated, ignoreOrder: true);
|
||||
Source(Gateway(), Cfg).SupportedAggregates.ShouldBe(
|
||||
[.. gatewayRawEvaluated, HistoryAggregateType.TimeAverage, HistoryAggregateType.Total], ignoreOrder: true);
|
||||
|
||||
// ... and served again, through the Host's resolve path, once ServerHistorian:Gateway:EnableNativeTimeWeighted is on.
|
||||
var nativeOn = new ServerHistorianOptions
|
||||
// ... and refused + unadvertised again, through the Host's resolve path, with the
|
||||
// ServerHistorian:Gateway:EnableNativeTimeWeighted=false kill switch (a gateway older than the #27 fix).
|
||||
var nativeOff = new ServerHistorianOptions
|
||||
{
|
||||
Backend = "Gateway", Enabled = true, Endpoint = "http://localhost:5222", ApiKey = "histgw_id_secret", UseTls = false,
|
||||
Gateway = new GatewayHistorianSettings { EnableNativeTimeWeighted = true },
|
||||
Gateway = new GatewayHistorianSettings { EnableNativeTimeWeighted = false },
|
||||
};
|
||||
Source(nativeOn, Cfg).SupportedAggregates.ShouldBe(
|
||||
[.. gatewayRawEvaluated, HistoryAggregateType.TimeAverage, HistoryAggregateType.Total], ignoreOrder: true);
|
||||
Source(nativeOff, Cfg).SupportedAggregates.ShouldBe(gatewayRawEvaluated, ignoreOrder: true);
|
||||
Source(new ServerHistorianOptions { Backend = "SqlServer", Enabled = true }, SqlCfg())
|
||||
.SupportedAggregates.ShouldBe(SqlServed, ignoreOrder: true);
|
||||
Source(new ServerHistorianOptions { Backend = "QuestDb", Enabled = true }, QuestDbCfg())
|
||||
|
||||
+3
-2
@@ -20,11 +20,12 @@ namespace ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests;
|
||||
/// </summary>
|
||||
public sealed class AdvertisedAggregateFunctionsWireTests
|
||||
{
|
||||
// #681: the ten per-sample aggregates (raw-evaluated). AVEVA-native TimeAverage / Total are no longer advertised by
|
||||
// default (lmxopcua#746 refuses them until historiangw#27 is fixed and verified; EnableNativeTimeWeighted=true restores them).
|
||||
// #681: the ten per-sample aggregates (raw-evaluated) plus the AVEVA-native TimeAverage / Total, advertised by default
|
||||
// again since historiangw#27 was live-verified (lmxopcua#694; EnableNativeTimeWeighted=false is the kill switch).
|
||||
private static readonly string[] GatewayNames =
|
||||
[
|
||||
"Average", "Count", "Delta", "End", "Maximum", "MaximumActualTime", "Minimum", "MinimumActualTime", "Range", "Start",
|
||||
"TimeAverage", "Total",
|
||||
];
|
||||
|
||||
// #820 added DurationBad / DurationGood (the PercentBad / PercentGood region times, in ms).
|
||||
|
||||
+817
@@ -0,0 +1,817 @@
|
||||
using System.Diagnostics;
|
||||
using System.Globalization;
|
||||
using System.Text.Json;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Opc.Ua;
|
||||
using Opc.Ua.Client;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.OtOpcUa.Commons.OpcUa;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
using ZB.MOM.WW.OtOpcUa.Host.Configuration;
|
||||
using ZB.MOM.WW.OtOpcUa.Runtime.Historian;
|
||||
using static ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests.ServerEventRig;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests;
|
||||
|
||||
/// <summary>
|
||||
/// Live gate for the historian follow-ups lmxopcua#694 (item 3), #700 and #746 (item 1): the booted OtOpcUa server with
|
||||
/// the <b>production</b> Gateway backend (<see cref="HistorianBackendRegistry"/>, default <see cref="ServerHistorianOptions"/>)
|
||||
/// in front of a REAL HistorianGateway and the real AVEVA Historian, read over a real SDK client session.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para><b>Read-only.</b> Only HistoryRead (raw, processed, at-time, events) is sent; the Gateway backend has no updater,
|
||||
/// and no provisioner, recorder or alarm writer is built.</para>
|
||||
/// <para><b>What it checks.</b></para>
|
||||
/// <list type="bullet">
|
||||
/// <item>#694.3 — native <c>TimeAverage</c> / <c>Total</c> (now on by default): forward and reverse over whole
|
||||
/// intervals, a clipped interval, a sub-millisecond clipped interval and a fractional-millisecond interval on
|
||||
/// <c>SysTimeSec</c>, each bucket checked against an independent step-held integral of the tag's own raw rows; and
|
||||
/// an outage window on a float tag whose middle intervals hold no good time at all (all-NULL ⇒ <c>BadNoData</c>).</item>
|
||||
/// <item>#700 — missing and existing tag Raw / Processed / AtTime; history values typed as the node's DataType
|
||||
/// (Int32, Boolean, Float); future AtTime keeps the requested timestamps and order; a transport outage answers
|
||||
/// <c>BadNoCommunication</c> while unsupported operations keep their own status.</item>
|
||||
/// <item>#746.1 — events paging over the wire equals the whole read, through a data source whose
|
||||
/// <c>EventDrainBudget</c> is 0: with the server-side <c>max_events</c> no page ever abandons its gateway stream.</item>
|
||||
/// </list>
|
||||
/// <para><b>Env:</b> <c>HISTGW_GATEWAY_ENDPOINT</c> and <c>HISTGW_GATEWAY_APIKEY</c> (any value against a gateway with
|
||||
/// API-key auth disabled) — skips when unset; optional <c>OTOPCUA_AVEVA_GATE_OUT</c>, <c>OTOPCUA_AVEVA_GATE_LABEL</c>.
|
||||
/// The strict-ceiling fixture (<see cref="Strict_ceiling_fixture_stopped_tag_reverse_cost_and_raised_lowered_limits"/>)
|
||||
/// additionally needs <c>OTOPCUA_AVEVA_GATE_STRICT_ENDPOINT</c> (a gateway with <c>AllowStreamingBeyondCeiling=false</c>)
|
||||
/// and <c>OTOPCUA_AVEVA_GATE_STRICT_SET_CEILING</c> (a script that restarts that gateway with the ceiling given as its
|
||||
/// argument). The windows are fixed past DEV (wonder-sql-vd03) windows.</para>
|
||||
/// </remarks>
|
||||
public sealed class AvevaGatewayHistorianFollowupsLiveGateTests
|
||||
{
|
||||
private const string SysTimeSec = "SysTimeSec";
|
||||
private const string SysPulse = "SysPulse";
|
||||
private const string FloatTag = "ZIPSEN.WTC1";
|
||||
private const string MissingTag = "OtOpcUa.Gate694.DoesNotExist";
|
||||
private const string EventSource = "DevWinPlatform";
|
||||
|
||||
private static readonly DateTime S0 = Utc("2026-11-11T14:00:00Z");
|
||||
// W1 on DEV: ZIPSEN.WTC1 is stored NULL at 11:32:36.073 (platform stop), then nothing until 11:38:34.853.
|
||||
private static readonly DateTime OutageStart = Utc("2026-11-10T11:31:00Z");
|
||||
private static readonly DateTime OutageEnd = Utc("2026-11-10T11:40:00Z");
|
||||
private static readonly (DateTime Start, DateTime End) EventWindow = (Utc("2026-11-05T15:00:00Z"), Utc("2026-11-12T00:00:00Z"));
|
||||
|
||||
private const uint GoodCalculated = 0x00000401u;
|
||||
private const uint PartialBit = 0x00000004u;
|
||||
|
||||
[Fact]
|
||||
public async Task Native_time_weighted_typed_history_outage_and_event_paging_over_the_wire()
|
||||
{
|
||||
var endpoint = Env("HISTGW_GATEWAY_ENDPOINT");
|
||||
var apiKey = Env("HISTGW_GATEWAY_APIKEY");
|
||||
if (endpoint is null || apiKey is null)
|
||||
Assert.Skip("Set HISTGW_GATEWAY_ENDPOINT and HISTGW_GATEWAY_APIKEY to run the #694/#700/#746 AVEVA gate.");
|
||||
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
var options = Options(endpoint!, apiKey!);
|
||||
options.Gateway.EnableNativeTimeWeighted.ShouldBeTrue("native TimeAverage / Total are on by default since #694");
|
||||
var provider = HistorianBackendRegistry.Resolve(options, new ConfigurationBuilder().Build());
|
||||
provider.CreateUpdater(new ServiceCollection().BuildServiceProvider()).ShouldBeNull();
|
||||
using var source = provider.CreateDataSource(new ServiceCollection().BuildServiceProvider());
|
||||
|
||||
var report = new Report(Env("OTOPCUA_AVEVA_GATE_LABEL") ?? "unlabelled", endpoint!);
|
||||
var failures = new List<string>();
|
||||
|
||||
await using var rig = await StartAsync("AvevaGate694", encrypted: false, ct);
|
||||
rig.Nm.HistorianDataSource = source;
|
||||
rig.Nm.MaxTieClusterOverfetch = options.MaxTieClusterOverfetch;
|
||||
rig.Nm.HonorReturnBounds = options.HonorReturnBounds;
|
||||
rig.Nm.EventReadHorizon = options.EventReadHorizon;
|
||||
rig.Nm.HistoryReadBatchParallelism = options.HistoryReadBatchParallelism;
|
||||
rig.Nm.MaxConcurrentHistoryReadBatches = options.MaxConcurrentHistoryReadBatches;
|
||||
rig.Nm.HistoryReadDeadline = options.HistoryReadDeadline;
|
||||
rig.Enforce(false, [Key()]);
|
||||
|
||||
var intNode = rig.EnsureRawVariable("HistInt", "Int32", writable: false, SysTimeSec);
|
||||
var boolNode = rig.EnsureRawVariable("HistBool", "Boolean", writable: false, SysPulse);
|
||||
var floatNode = rig.EnsureRawVariable("HistFloat", "Float", writable: false, FloatTag);
|
||||
var dblNode = rig.EnsureRawVariable("HistDbl", "Double", writable: false, SysTimeSec);
|
||||
var missingNode = rig.EnsureRawVariable("HistMissing", "Double", writable: false, MissingTag);
|
||||
rig.Sink.EnsureFolder(EventSource, parentNodeId: null, displayName: EventSource, realm: AddressSpaceRealm.Raw);
|
||||
rig.Sink.MaterialiseAlarmCondition($"{EventSource}/gate694", EventSource, "gate694", "OffNormalAlarm", 500,
|
||||
AddressSpaceRealm.Raw, isNative: true);
|
||||
|
||||
using var session = await rig.ConnectAsync(anonymous: true, ct);
|
||||
var ns = (ushort)session.NamespaceUris.GetIndex(V3NodeIds.RawNamespaceUri);
|
||||
NodeId Node(string id) => new(id, ns);
|
||||
|
||||
// ================================================================ #694.3 — native TimeAverage / Total
|
||||
var caps = await AggregateFunctionsAsync(session, ct);
|
||||
report.Add(new Case("694.advertised", null, null, "wire", "Good", caps, null));
|
||||
Check(failures, caps.Split(',').Contains("TimeAverage") && caps.Split(',').Contains("Total"),
|
||||
$"TimeAverage and Total must be advertised by default, got {caps}");
|
||||
|
||||
// The step-held reference: SysTimeSec is an integer tag, which AVEVA integrates as a step (live: [14:00:00, 14:00:20) of
|
||||
// 0..19 answers TimeWeightedAverage 9.5 / Integral 190). Raw rows straddling every case window.
|
||||
var refRows = (await source.ReadRawAsync(SysTimeSec, S0.AddSeconds(-5), S0.AddMinutes(4), 0, ct)).Samples
|
||||
.Where(s => s.Value is not null).Select(s => (T: s.SourceTimestampUtc!.Value, V: Convert.ToDouble(s.Value, CultureInfo.InvariantCulture)))
|
||||
.ToList();
|
||||
Check(failures, refRows.Count >= 240, $"SysTimeSec reference: expected ~1 Hz rows, got {refRows.Count}");
|
||||
double StepIntegral(DateTime a, DateTime b)
|
||||
{
|
||||
double sum = 0;
|
||||
for (var i = 0; i < refRows.Count; i++)
|
||||
{
|
||||
var from = refRows[i].T;
|
||||
var to = i + 1 < refRows.Count ? refRows[i + 1].T : DateTime.MaxValue;
|
||||
var lo = from > a ? from : a;
|
||||
var hi = to < b ? to : b;
|
||||
if (hi > lo) sum += refRows[i].V * (hi - lo).TotalSeconds;
|
||||
}
|
||||
return sum;
|
||||
}
|
||||
|
||||
var nativeCases = new (string Name, DateTime Start, DateTime End, double IntervalMs)[]
|
||||
{
|
||||
("forward.whole", S0, S0.AddMinutes(2), 20_000),
|
||||
("reverse.whole", S0.AddMinutes(2), S0, 20_000),
|
||||
("forward.clipped7s", S0, S0.AddSeconds(127), 20_000),
|
||||
("reverse.clipped7s", S0.AddSeconds(127), S0, 20_000),
|
||||
("forward.subMsClipped", S0, S0.AddSeconds(40).AddTicks(4_000), 20_000), // last interval 0.4 ms at value 40
|
||||
("forward.nonWholeMsClipped", S0, S0.AddSeconds(43).AddTicks(4_000), 20_000), // last interval 3.0004 s
|
||||
("reverse.subMsClipped", S0.AddSeconds(43).AddTicks(4_000), S0.AddSeconds(3), 20_000), // earliest (3, 3.0004] at value 3
|
||||
("forward.fractionalMs", S0.AddSeconds(3).AddTicks(9_970_000), S0.AddSeconds(4).AddTicks(30_000), 1.5), // straddles :04
|
||||
("reverse.fractionalMs", S0.AddSeconds(4).AddTicks(30_000), S0.AddSeconds(3).AddTicks(9_970_000), 1.5),
|
||||
};
|
||||
foreach (var (name, start, end, intervalMs) in nativeCases)
|
||||
{
|
||||
foreach (var (aggName, aggId, aggType) in new[]
|
||||
{
|
||||
("TimeAverage", ObjectIds.AggregateFunction_TimeAverage, HistoryAggregateType.TimeAverage),
|
||||
("Total", ObjectIds.AggregateFunction_Total, HistoryAggregateType.Total),
|
||||
})
|
||||
{
|
||||
var w = $"{start:o}/{end:o}@{intervalMs}ms";
|
||||
var wire = await ReadProcessedAsync(session, Node(dblNode), start, end, intervalMs, aggId, ct);
|
||||
var direct = await source.ReadProcessedAsync(SysTimeSec, start, end, TimeSpan.FromTicks((long)(intervalMs * TimeSpan.TicksPerMillisecond)), aggType, ct);
|
||||
report.Add(new Case($"694.{aggName}.{name}", SysTimeSec, w, "wire", wire.Status, null, wire.Rows));
|
||||
CheckSame(failures, $"694.{aggName}.{name}", wire.Rows, Rows(direct.Samples));
|
||||
|
||||
// The independent reference, interval by interval on the Part 13 grid (#677: reverse steps back from StartTime).
|
||||
var expected = Grid(start, end, TimeSpan.FromTicks((long)(intervalMs * TimeSpan.TicksPerMillisecond)));
|
||||
Check(failures, wire.Rows.Count == expected.Count, $"694.{aggName}.{name}: {wire.Rows.Count} buckets vs {expected.Count} intervals");
|
||||
var reference = new List<string>();
|
||||
for (var i = 0; i < Math.Min(wire.Rows.Count, expected.Count); i++)
|
||||
{
|
||||
var (lo, hi, stamp, clipped) = expected[i];
|
||||
var integral = StepIntegral(lo, hi);
|
||||
var want = aggType == HistoryAggregateType.Total ? integral : integral / (hi - lo).TotalSeconds;
|
||||
var row = wire.Rows[i];
|
||||
var got = double.Parse(row.V!, CultureInfo.InvariantCulture);
|
||||
reference.Add($"[{lo:HH:mm:ss.ffffff},{hi:HH:mm:ss.ffffff}) want={want.ToString("R", CultureInfo.InvariantCulture)}");
|
||||
Check(failures, Math.Abs(got - want) <= 1e-9 * Math.Max(1, Math.Abs(want)),
|
||||
$"694.{aggName}.{name}[{i}] [{lo:o},{hi:o}): {got} vs step reference {want}");
|
||||
Check(failures, row.Src == stamp, $"694.{aggName}.{name}[{i}]: stamp {row.Src:o} vs {stamp:o}");
|
||||
Check(failures, row.Status == (clipped ? GoodCalculated | PartialBit : GoodCalculated),
|
||||
$"694.{aggName}.{name}[{i}]: status 0x{row.Status:X8} (clipped={clipped})");
|
||||
}
|
||||
report.Add(new Case($"694.{aggName}.{name}", SysTimeSec, w, "reference", "Good", string.Join("; ", reference), null));
|
||||
}
|
||||
}
|
||||
|
||||
// All-NULL intervals: the float tag's outage (stored NULL at 11:32:36, nothing until 11:38:34).
|
||||
foreach (var (aggName, aggId, aggType) in new[]
|
||||
{
|
||||
("TimeAverage", ObjectIds.AggregateFunction_TimeAverage, HistoryAggregateType.TimeAverage),
|
||||
("Total", ObjectIds.AggregateFunction_Total, HistoryAggregateType.Total),
|
||||
})
|
||||
{
|
||||
foreach (var (dir, start, end) in new[] { ("forward", OutageStart, OutageEnd), ("reverse", OutageEnd, OutageStart) })
|
||||
{
|
||||
var w = $"{start:o}/{end:o}@60000ms";
|
||||
var wire = await ReadProcessedAsync(session, Node(floatNode), start, end, 60_000, aggId, ct);
|
||||
var direct = await source.ReadProcessedAsync(FloatTag, start, end, TimeSpan.FromMinutes(1), aggType, ct);
|
||||
report.Add(new Case($"694.{aggName}.allNull.{dir}", FloatTag, w, "wire", wire.Status, null, wire.Rows));
|
||||
CheckSame(failures, $"694.{aggName}.allNull.{dir}", wire.Rows, Rows(direct.Samples));
|
||||
Check(failures, wire.Rows.Count == 9, $"694.{aggName}.allNull.{dir}: {wire.Rows.Count} buckets vs 9");
|
||||
// Interval start minute → expected class. 11:33–11:37 hold no good time at all; 11:32 and 11:38 hold some.
|
||||
foreach (var row in wire.Rows)
|
||||
{
|
||||
var minuteStart = dir == "forward" ? row.Src!.Value : row.Src!.Value.AddMinutes(-1);
|
||||
var m = minuteStart.Minute;
|
||||
var cls = m is >= 33 and <= 37 ? "BadNoData" : m is 32 or 38 ? "Uncertain" : "Good";
|
||||
var ok = cls switch
|
||||
{
|
||||
"BadNoData" => (row.Status & 0xFFFF0000u) == StatusCodes.BadNoData && row.V is null,
|
||||
"Uncertain" => (row.Status & 0xC0000000u) == 0x40000000u && row.V is not null,
|
||||
_ => (row.Status & 0xC0000000u) == 0 && row.V is not null,
|
||||
};
|
||||
Check(failures, ok, $"694.{aggName}.allNull.{dir} {minuteStart:HH:mm}: expected {cls}, got 0x{row.Status:X8} value {row.V ?? "null"}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================ #700 — missing / existing tag, DataTypes, AtTime
|
||||
{
|
||||
var (start, end) = (S0, S0.AddSeconds(30));
|
||||
var w = $"{start:o}/{end:o}";
|
||||
var raw = await ReadRawAsync(session, Node(missingNode), start, end, 0, ct);
|
||||
var avg = await ReadProcessedAsync(session, Node(missingNode), start, end, 10_000, ObjectIds.AggregateFunction_Average, ct);
|
||||
var tavg = await ReadProcessedAsync(session, Node(missingNode), start, end, 10_000, ObjectIds.AggregateFunction_TimeAverage, ct);
|
||||
var at = await ReadAtTimeAsync(session, Node(missingNode), [start, start.AddSeconds(1.5)], ct);
|
||||
report.Add(new Case("700.missing.raw", MissingTag, w, "wire", raw.Status, null, raw.Rows));
|
||||
report.Add(new Case("700.missing.average", MissingTag, w, "wire", avg.Status, null, avg.Rows));
|
||||
report.Add(new Case("700.missing.timeAverage", MissingTag, w, "wire", tavg.Status, null, tavg.Rows));
|
||||
report.Add(new Case("700.missing.atTime", MissingTag, w, "wire", at.Status, null, at.Rows));
|
||||
Check(failures, raw.Status == "GoodNoData" && raw.Rows.Count == 0, $"700.missing.raw: {raw.Status} rows={raw.Rows.Count}");
|
||||
foreach (var (n, r) in new[] { ("average", avg), ("timeAverage", tavg) })
|
||||
{
|
||||
Check(failures, r.Rows.Count == 3, $"700.missing.{n}: {r.Rows.Count} buckets vs 3");
|
||||
Check(failures, r.Rows.All(x => (x.Status & 0xFFFF0000u) == StatusCodes.BadNoData && x.V is null),
|
||||
$"700.missing.{n}: every bucket must be BadNoData, got {string.Join(",", r.Rows.Select(x => $"0x{x.Status:X8}"))}");
|
||||
}
|
||||
Check(failures, at.Rows.Count == 2 && at.Rows.All(x => StatusCode.IsBad(x.Status)),
|
||||
$"700.missing.atTime: expected two Bad values, got {at.Status} {string.Join(",", at.Rows.Select(x => $"0x{x.Status:X8}"))}");
|
||||
}
|
||||
|
||||
foreach (var (label, node, tag, type, typeCode) in new[]
|
||||
{
|
||||
("int32", intNode, SysTimeSec, "Int32", TypeCode.Int32),
|
||||
("boolean", boolNode, SysPulse, "Boolean", TypeCode.Boolean),
|
||||
("float", floatNode, FloatTag, "Single", TypeCode.Single),
|
||||
})
|
||||
{
|
||||
var (start, end) = label == "float"
|
||||
? (Utc("2026-11-10T11:31:00Z"), Utc("2026-11-10T11:31:20Z"))
|
||||
: label == "boolean" ? (S0, S0.AddMinutes(5)) : (S0, S0.AddSeconds(10));
|
||||
var w = $"{start:o}/{end:o}";
|
||||
var raw = await ReadRawAsync(session, Node(node), start, end, 0, ct);
|
||||
var direct = await source.ReadRawAsync(tag, start, end, 0, ct);
|
||||
report.Add(new Case($"700.existing.raw.{label}", tag, w, "wire", raw.Status, Types(raw.Rows), raw.Rows));
|
||||
Check(failures, raw.Rows.Count > 0 && raw.Rows.Count == direct.Samples.Count,
|
||||
$"700.existing.raw.{label}: wire {raw.Rows.Count} vs direct {direct.Samples.Count}");
|
||||
Check(failures, raw.Rows.All(r => r.Type == type), $"700.existing.raw.{label}: value types {Types(raw.Rows)}, node DataType {type}");
|
||||
for (var i = 0; i < Math.Min(raw.Rows.Count, direct.Samples.Count); i++)
|
||||
Check(failures, raw.Rows[i].Src == direct.Samples[i].SourceTimestampUtc && raw.Rows[i].Status == direct.Samples[i].StatusCode,
|
||||
$"700.existing.raw.{label}[{i}]: wire ({raw.Rows[i].Src:o}, 0x{raw.Rows[i].Status:X8}) vs direct");
|
||||
|
||||
// AtTime: a stored timestamp and a midpoint, typed as the node.
|
||||
var times = new List<DateTime> { raw.Rows[0].Src!.Value };
|
||||
if (raw.Rows.Count > 1) times.Add(raw.Rows[0].Src!.Value + (raw.Rows[1].Src!.Value - raw.Rows[0].Src!.Value) / 2);
|
||||
var at = await ReadAtTimeAsync(session, Node(node), times, ct);
|
||||
var atDirect = await source.ReadAtTimeAsync(tag, times, typeCode, ct);
|
||||
report.Add(new Case($"700.existing.atTime.{label}", tag, string.Join(",", times.Select(t => t.ToString("o"))), "wire", at.Status,
|
||||
Types(at.Rows), at.Rows));
|
||||
CheckSame(failures, $"700.existing.atTime.{label}", at.Rows, Rows(atDirect.Samples));
|
||||
Check(failures, at.Rows.Select(r => r.Src).SequenceEqual(times.Select(t => (DateTime?)t)),
|
||||
$"700.existing.atTime.{label}: timestamps must be the requested ones, in order");
|
||||
Check(failures, at.Rows.All(r => r.V is null || r.Type == type), $"700.existing.atTime.{label}: types {Types(at.Rows)} vs {type}");
|
||||
|
||||
// Processed: a per-sample and a native aggregate over the same window, wire vs the backend.
|
||||
foreach (var (aggName, aggId, aggType) in new[]
|
||||
{
|
||||
("Average", ObjectIds.AggregateFunction_Average, HistoryAggregateType.Average),
|
||||
("TimeAverage", ObjectIds.AggregateFunction_TimeAverage, HistoryAggregateType.TimeAverage),
|
||||
})
|
||||
{
|
||||
var intervalMs = (end - start).TotalMilliseconds / 2;
|
||||
var p = await ReadProcessedAsync(session, Node(node), start, end, intervalMs, aggId, ct);
|
||||
ZB.MOM.WW.OtOpcUa.Core.Abstractions.HistoryReadResult? d = null;
|
||||
string? directError = null;
|
||||
try { d = await source.ReadProcessedAsync(tag, start, end, TimeSpan.FromMilliseconds(intervalMs), aggType, ct); }
|
||||
catch (Exception ex) { directError = $"{ex.GetType().Name}: {ex.Message}"; }
|
||||
report.Add(new Case($"700.existing.processed.{aggName}.{label}", tag, w, "wire", p.Status,
|
||||
$"{Types(p.Rows)} direct={directError ?? "ok"}", p.Rows));
|
||||
if (label == "boolean" && aggType == HistoryAggregateType.TimeAverage)
|
||||
{
|
||||
// AVEVA refuses its native time-weighted modes on a discrete tag (StartQuery fails): recorded; the node
|
||||
// must answer Bad, never a value.
|
||||
Check(failures, p.Status.StartsWith("Bad", StringComparison.Ordinal) && p.Rows.Count == 0,
|
||||
$"700.existing.processed.TimeAverage.boolean: {p.Status} rows={p.Rows.Count} (direct: {directError ?? "served"})");
|
||||
continue;
|
||||
}
|
||||
if (d is null) { failures.Add($"700.existing.processed.{aggName}.{label}: direct read failed: {directError}"); continue; }
|
||||
CheckSame(failures, $"700.existing.processed.{aggName}.{label}", p.Rows, Rows(d.Samples));
|
||||
Check(failures, p.Rows.Count == 2, $"700.existing.processed.{aggName}.{label}: {p.Rows.Count} buckets");
|
||||
}
|
||||
}
|
||||
|
||||
// Future AtTime: requested timestamps and order preserved; cost recorded.
|
||||
{
|
||||
var now = DateTime.UtcNow;
|
||||
var times = new[] { now.AddHours(1), now.AddMinutes(30), now.AddHours(2) };
|
||||
var sw = Stopwatch.StartNew();
|
||||
var at = await ReadAtTimeAsync(session, Node(intNode), times, ct);
|
||||
sw.Stop();
|
||||
report.Add(new Case("700.atTime.future", SysTimeSec, string.Join(",", times.Select(t => t.ToString("o"))), "wire", at.Status,
|
||||
$"elapsedMs={sw.ElapsedMilliseconds} {Types(at.Rows)}", at.Rows));
|
||||
Check(failures, at.Rows.Select(r => r.Src).SequenceEqual(times.Select(t => (DateTime?)t)),
|
||||
"700.atTime.future: timestamps must be the requested ones, in request order");
|
||||
Check(failures, at.Rows.All(r => !StatusCode.IsGood(r.Status)), "700.atTime.future: a future value is never Good");
|
||||
}
|
||||
|
||||
// ================================================================ #746.1 — events paging, server-side max_events
|
||||
{
|
||||
// A second production data source whose EventDrainBudget is 0: without the server cap every truncated page would
|
||||
// abandon (cancel) its gateway stream at the cap.
|
||||
var budget0 = Options(endpoint!, apiKey!);
|
||||
var eventsOptions = new ServerHistorianOptions
|
||||
{
|
||||
Backend = budget0.Backend, Enabled = true, Endpoint = budget0.Endpoint, ApiKey = budget0.ApiKey, UseTls = budget0.UseTls,
|
||||
AllowUntrustedServerCertificate = budget0.AllowUntrustedServerCertificate,
|
||||
Gateway = new GatewayHistorianSettings { EventDrainBudget = 0 },
|
||||
};
|
||||
using var eventsSource = HistorianBackendRegistry.Resolve(eventsOptions, new ConfigurationBuilder().Build())
|
||||
.CreateDataSource(new ServiceCollection().BuildServiceProvider());
|
||||
rig.Nm.HistorianDataSource = eventsSource;
|
||||
var (start, end) = EventWindow;
|
||||
var w = $"{start:o}/{end:o}";
|
||||
var whole = await ReadEventsAsync(session, Node(EventSource), start, end, 0, ct);
|
||||
var sw = Stopwatch.StartNew();
|
||||
var paged1 = await ReadEventsAsync(session, Node(EventSource), start, end, 1, ct);
|
||||
var paged7 = await ReadEventsAsync(session, Node(EventSource), start, end, 7, ct);
|
||||
var reverse5 = await ReadEventsAsync(session, Node(EventSource), end, start, 5, ct);
|
||||
sw.Stop();
|
||||
var abandoned = Internal<long>(eventsSource, "EventStreamsAbandoned");
|
||||
var reReads = Internal<long>(eventsSource, "EventServerCapReReads");
|
||||
report.Add(new Case("746.events.whole", EventSource, w, "wire", whole.Status, $"events={whole.Rows.Count}", null, null, whole.Rows));
|
||||
report.Add(new Case("746.events.paged1", EventSource, w, "wire", paged1.Status, $"pages={paged1.Pages}", null, null, paged1.Rows));
|
||||
report.Add(new Case("746.events.paged7", EventSource, w, "wire", paged7.Status, $"pages={paged7.Pages}", null, null, paged7.Rows));
|
||||
report.Add(new Case("746.events.reverse5", EventSource, w, "wire", reverse5.Status, $"pages={reverse5.Pages}", null, null, reverse5.Rows));
|
||||
report.Add(new Case("746.events.streams", EventSource, w, "direct", "Good",
|
||||
$"abandonedStreams={abandoned} serverCapReReads={reReads} pagedElapsedMs={sw.ElapsedMilliseconds}", null));
|
||||
Check(failures, whole.Rows.Count > 7, $"746.events: expected more than one 7-event page, got {whole.Rows.Count}");
|
||||
var ids = whole.Rows.Select(r => r.Id).ToList();
|
||||
Check(failures, paged1.Rows.Select(r => r.Id).SequenceEqual(ids), "746.events: 1-event pages must equal the whole read");
|
||||
Check(failures, paged7.Rows.Select(r => r.Id).SequenceEqual(ids), "746.events: 7-event pages must equal the whole read");
|
||||
var reverseIds = whole.Rows.OrderByDescending(r => r.Time).ThenByDescending(r => r.Id, StringComparer.Ordinal).Select(r => r.Id);
|
||||
Check(failures, reverse5.Rows.Select(r => r.Id).SequenceEqual(reverseIds), "746.events: 5-event reverse pages, newest first");
|
||||
Check(failures, abandoned == 0, $"746.events: {abandoned} gateway streams abandoned with EventDrainBudget=0 — max_events not honoured");
|
||||
rig.Nm.HistorianDataSource = source;
|
||||
}
|
||||
|
||||
// ================================================================ #700 — transport outage vs unsupported
|
||||
{
|
||||
var dead = new ServerHistorianOptions
|
||||
{
|
||||
Backend = "Gateway", Enabled = true, Endpoint = "http://127.0.0.1:9", ApiKey = apiKey!, UseTls = false,
|
||||
};
|
||||
using var deadSource = HistorianBackendRegistry.Resolve(dead, new ConfigurationBuilder().Build())
|
||||
.CreateDataSource(new ServiceCollection().BuildServiceProvider());
|
||||
rig.Nm.HistorianDataSource = deadSource;
|
||||
var (start, end) = (S0, S0.AddSeconds(30));
|
||||
var w = $"{start:o}/{end:o}";
|
||||
var results = new Dictionary<string, string>
|
||||
{
|
||||
["raw"] = (await ReadRawAsync(session, Node(intNode), start, end, 0, ct)).Status,
|
||||
["processed.Average"] = (await ReadProcessedAsync(session, Node(intNode), start, end, 10_000, ObjectIds.AggregateFunction_Average, ct)).Status,
|
||||
["processed.TimeAverage"] = (await ReadProcessedAsync(session, Node(intNode), start, end, 10_000, ObjectIds.AggregateFunction_TimeAverage, ct)).Status,
|
||||
["atTime"] = (await ReadAtTimeAsync(session, Node(intNode), [start], ct)).Status,
|
||||
["events"] = (await ReadEventsAsync(session, Node(EventSource), EventWindow.Start, EventWindow.End, 0, ct)).Status,
|
||||
["modified"] = await ReadModifiedAsync(session, Node(intNode), start, end, ct),
|
||||
["processed.TimeAverage2"] = (await ReadProcessedAsync(session, Node(intNode), start, end, 10_000, ObjectIds.AggregateFunction_TimeAverage2, ct)).Status,
|
||||
};
|
||||
report.Add(new Case("700.outage", SysTimeSec, w, "wire", "see note",
|
||||
string.Join(" ", results.Select(kv => $"{kv.Key}={kv.Value}")), null));
|
||||
foreach (var k in new[] { "raw", "processed.Average", "processed.TimeAverage", "atTime", "events" })
|
||||
Check(failures, results[k] == "BadNoCommunication", $"700.outage.{k}: {results[k]} (expected BadNoCommunication)");
|
||||
Check(failures, results["modified"] == "BadHistoryOperationUnsupported", $"700.outage.modified: {results["modified"]}");
|
||||
Check(failures, results["processed.TimeAverage2"] == "BadAggregateNotSupported", $"700.outage.TimeAverage2: {results["processed.TimeAverage2"]}");
|
||||
|
||||
// Positive control: the same session, back on the live source, reads again.
|
||||
rig.Nm.HistorianDataSource = source;
|
||||
var back = await ReadRawAsync(session, Node(intNode), start, end, 0, ct);
|
||||
report.Add(new Case("700.outage.recovered", SysTimeSec, w, "wire", back.Status, $"rows={back.Rows.Count}", null));
|
||||
Check(failures, back.Rows.Count > 0 && !back.Status.StartsWith("Bad", StringComparison.Ordinal), $"700.outage.recovered: {back.Status}");
|
||||
}
|
||||
|
||||
await WriteReportAsync(report, failures, "followups", ct);
|
||||
failures.ShouldBeEmpty();
|
||||
await session.CloseAsync(ct);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// #700, controlled fixture: a strict gateway (<c>AllowStreamingBeyondCeiling=false</c>) whose ceiling the test changes
|
||||
/// by restarting it. Checks the strict-ceiling stopped-tag reverse read (order, ties, completeness, timestamps; cost in
|
||||
/// gateway calls against #696's unit bound) and that one long-lived data source follows a lowered and a raised
|
||||
/// ceiling, including the per-sample aggregate's 30 s cooldown.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task Strict_ceiling_fixture_stopped_tag_reverse_cost_and_raised_lowered_limits()
|
||||
{
|
||||
var endpoint = Env("OTOPCUA_AVEVA_GATE_STRICT_ENDPOINT");
|
||||
var setCeiling = Env("OTOPCUA_AVEVA_GATE_STRICT_SET_CEILING");
|
||||
var apiKey = Env("HISTGW_GATEWAY_APIKEY");
|
||||
var referenceEndpoint = Env("HISTGW_GATEWAY_ENDPOINT");
|
||||
if (endpoint is null || setCeiling is null || apiKey is null || referenceEndpoint is null)
|
||||
Assert.Skip("Set OTOPCUA_AVEVA_GATE_STRICT_ENDPOINT, OTOPCUA_AVEVA_GATE_STRICT_SET_CEILING, HISTGW_GATEWAY_ENDPOINT (a " +
|
||||
"streaming gateway for the reference reads) and HISTGW_GATEWAY_APIKEY to run the strict-ceiling fixture.");
|
||||
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
var report = new Report(Env("OTOPCUA_AVEVA_GATE_LABEL") ?? "unlabelled", endpoint!);
|
||||
var failures = new List<string>();
|
||||
var calls = new GatewayCallCounter(Env("OTOPCUA_AVEVA_GATE_STRICT_CONTAINER"));
|
||||
|
||||
await SetCeilingAsync(setCeiling!, 1000, ct);
|
||||
var options = Options(endpoint!, apiKey!);
|
||||
using var source = HistorianBackendRegistry.Resolve(options, new ConfigurationBuilder().Build())
|
||||
.CreateDataSource(new ServiceCollection().BuildServiceProvider());
|
||||
// References come from a streaming gateway in front of the same historian (the strict one would refuse them).
|
||||
using var reference = HistorianBackendRegistry.Resolve(Options(referenceEndpoint!, apiKey!), new ConfigurationBuilder().Build())
|
||||
.CreateDataSource(new ServiceCollection().BuildServiceProvider());
|
||||
|
||||
await using var rig = await StartAsync("AvevaGate700Strict", encrypted: false, ct);
|
||||
rig.Nm.HistorianDataSource = source;
|
||||
rig.Nm.MaxTieClusterOverfetch = options.MaxTieClusterOverfetch;
|
||||
rig.Nm.HonorReturnBounds = options.HonorReturnBounds;
|
||||
rig.Nm.HistoryReadDeadline = options.HistoryReadDeadline;
|
||||
rig.Enforce(false, [Key()]);
|
||||
var floatNode = rig.EnsureRawVariable("HistFloat", "Float", writable: false, FloatTag);
|
||||
var dblNode = rig.EnsureRawVariable("HistSec", "Double", writable: false, SysTimeSec);
|
||||
using var session = await rig.ConnectAsync(anonymous: true, ct);
|
||||
var ns = (ushort)session.NamespaceUris.GetIndex(V3NodeIds.RawNamespaceUri);
|
||||
NodeId Node(string id) => new(id, ns);
|
||||
|
||||
// ---- Stopped-tag reverse read: the float tag stopped at 11:32:36 (NULL) and resumed at 11:38:34. Read the latest 3
|
||||
// before 11:38:00 (end-only: StartTime unset, EndTime 11:38:00 — Part 11 reads back from EndTime), on the strict
|
||||
// 1,000-row gateway.
|
||||
{
|
||||
var at = Utc("2026-11-10T11:38:00Z");
|
||||
var stored = (await reference.ReadRawAsync(FloatTag, at.AddHours(-1), at, 0, ct)).Samples;
|
||||
var expected = stored.Reverse().Take(3).Select(s => s.SourceTimestampUtc).ToList();
|
||||
await calls.MarkAsync("ReadRaw", ct);
|
||||
var sw = Stopwatch.StartNew();
|
||||
var r = await ReadRawAsync(session, Node(floatNode), DateTime.MinValue, at, 3, ct);
|
||||
sw.Stop();
|
||||
var n = await calls.CountAsync("ReadRaw", ct);
|
||||
report.Add(new Case("700.strict.stoppedTagReverse3", FloatTag, $"(end-only)/{at:o}", "wire", r.Status,
|
||||
$"gatewayReadRawCalls={n} elapsedMs={sw.ElapsedMilliseconds} unitBound=12 (#696: 7,200 stopped 1 Hz rows, ceiling 1,000, one day later)",
|
||||
r.Rows));
|
||||
Check(failures, r.Rows.Select(x => x.Src).SequenceEqual(expected),
|
||||
$"700.strict.stoppedTagReverse3: {string.Join(",", r.Rows.Select(x => x.Src?.ToString("o")))} vs {string.Join(",", expected.Select(x => x?.ToString("o")))}");
|
||||
Check(failures, n is > 0 and <= 12, $"700.strict.stoppedTagReverse3: {n} gateway ReadRaw calls (unit bound 12)");
|
||||
|
||||
// A reverse read across the whole outage: newest first, paged 37, equal to the forward reference reversed.
|
||||
var (s0, s1) = (Utc("2026-11-10T11:30:00Z"), Utc("2026-11-10T11:41:00Z"));
|
||||
var fwd = (await reference.ReadRawAsync(FloatTag, s0.AddTicks(1), s1.AddTicks(1), 0, ct)).Samples;
|
||||
var paged = await ReadRawPagedAsync(session, Node(floatNode), s1, s0, 37, ct);
|
||||
report.Add(new Case("700.strict.reversePaged37", FloatTag, $"{s1:o}/{s0:o}", "wire", paged.Status, $"pages={paged.Pages}", paged.Rows));
|
||||
CheckSame(failures, "700.strict.reversePaged37", paged.Rows, Rows(fwd).AsEnumerable().Reverse().ToList());
|
||||
}
|
||||
|
||||
// ---- Lowered then raised limits, one long-lived data source. A 30-minute SysTimeSec raw read (1,800 rows) pages at
|
||||
// the learned ceiling; an Average over it (one 30-minute interval) needs more rows than a 1,000 / 500 ceiling allows.
|
||||
// The Average goes FIRST in each probe: a raw read that finds the gateway raised forgets the learned ceiling, which
|
||||
// would end the per-sample aggregate's cooldown early.
|
||||
var (a, b) = (S0, S0.AddMinutes(30));
|
||||
async Task<(WireRead Avg, WireRead Raw)> Probe(string phase)
|
||||
{
|
||||
var avg = await ReadProcessedAsync(session, Node(dblNode), a, b, 30 * 60_000, ObjectIds.AggregateFunction_Average, ct);
|
||||
var raw = await ReadRawPagedAsync(session, Node(dblNode), a, b, 0, ct);
|
||||
report.Add(new Case($"700.limits.{phase}.average", SysTimeSec, $"{a:o}/{b:o}", "wire", avg.Status,
|
||||
string.Join(",", avg.Rows.Select(r => $"0x{r.Status:X8}")), avg.Rows));
|
||||
report.Add(new Case($"700.limits.{phase}.raw", SysTimeSec, $"{a:o}/{b:o}", "wire", raw.Status, $"pages={raw.Pages} rows={raw.Rows.Count}", null));
|
||||
return (avg, raw);
|
||||
}
|
||||
static bool Refused(WireRead r) =>
|
||||
r.Status == "BadTooManyOperations" || (r.Rows.Count == 1 && r.Rows[0].Status == StatusCodes.BadTooManyOperations);
|
||||
|
||||
var at1000 = await Probe("ceiling1000");
|
||||
Check(failures, at1000.Raw.Rows.Count == 1800 && at1000.Raw.Pages >= 2, $"700.limits.1000: raw {at1000.Raw.Rows.Count} rows / {at1000.Raw.Pages} pages");
|
||||
Check(failures, Refused(at1000.Avg), $"700.limits.1000: an Average over 1,800 rows on a 1,000-row strict gateway must be BadTooManyOperations, got {at1000.Avg.Status}");
|
||||
|
||||
await SetCeilingAsync(setCeiling!, 500, ct); // lowered: learned at once, from the next refusal
|
||||
var lastRefusal = DateTime.UtcNow;
|
||||
var at500 = await Probe("ceiling500");
|
||||
Check(failures, Refused(at500.Avg), $"700.limits.500: Average must be BadTooManyOperations, got {at500.Avg.Status}");
|
||||
Check(failures, at500.Raw.Rows.Count == 1800 && at500.Raw.Pages > at1000.Raw.Pages, $"700.limits.500: raw {at500.Raw.Rows.Count} rows / {at500.Raw.Pages} pages");
|
||||
|
||||
await SetCeilingAsync(setCeiling!, 5000, ct); // raised: the per-sample aggregate reuses the 500 refusal for 30 s
|
||||
var sinceRefusal = DateTime.UtcNow - lastRefusal;
|
||||
var inCooldown = await ReadProcessedAsync(session, Node(dblNode), a, b, 30 * 60_000, ObjectIds.AggregateFunction_Average, ct);
|
||||
report.Add(new Case("700.limits.ceiling5000.averageInCooldown", SysTimeSec, $"{a:o}/{b:o}", "wire", inCooldown.Status,
|
||||
$"secondsSinceLastRefusal={sinceRefusal.TotalSeconds:F1} {string.Join(",", inCooldown.Rows.Select(r => $"0x{r.Status:X8}"))}", inCooldown.Rows));
|
||||
if (sinceRefusal < TimeSpan.FromSeconds(25))
|
||||
Check(failures, Refused(inCooldown), $"700.limits.5000: inside the 30 s cooldown the Average stays a request limit, got {inCooldown.Status}");
|
||||
var wait = lastRefusal.AddSeconds(31) - DateTime.UtcNow;
|
||||
if (wait > TimeSpan.Zero) await Task.Delay(wait, ct);
|
||||
var after = await Probe("ceiling5000.afterCooldown");
|
||||
Check(failures, after.Avg.Rows.Count == 1 && StatusCode.IsGood(after.Avg.Rows[0].Status),
|
||||
$"700.limits.5000: after the cooldown the Average is served, got {after.Avg.Status} {string.Join(",", after.Avg.Rows.Select(r => $"0x{r.Status:X8}"))}");
|
||||
Check(failures, after.Raw.Rows.Count == 1800 && after.Raw.Pages == 1, $"700.limits.5000: raw {after.Raw.Rows.Count} rows / {after.Raw.Pages} pages");
|
||||
|
||||
await SetCeilingAsync(setCeiling!, 1000, ct);
|
||||
await WriteReportAsync(report, failures, "strict", ct);
|
||||
failures.ShouldBeEmpty();
|
||||
await session.CloseAsync(ct);
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------ fixture helpers
|
||||
|
||||
private static ServerHistorianOptions Options(string endpoint, string apiKey) => new()
|
||||
{
|
||||
Backend = "Gateway",
|
||||
Enabled = true,
|
||||
Endpoint = endpoint,
|
||||
ApiKey = apiKey,
|
||||
UseTls = endpoint.StartsWith("https://", StringComparison.OrdinalIgnoreCase),
|
||||
AllowUntrustedServerCertificate = string.Equals(Env("HISTGW_GATEWAY_ALLOW_UNTRUSTED"), "true", StringComparison.OrdinalIgnoreCase),
|
||||
};
|
||||
|
||||
private static async Task SetCeilingAsync(string script, int ceiling, CancellationToken ct)
|
||||
{
|
||||
using var p = Process.Start(new ProcessStartInfo("bash", [script, ceiling.ToString(CultureInfo.InvariantCulture)])
|
||||
{
|
||||
RedirectStandardOutput = true, RedirectStandardError = true,
|
||||
})!;
|
||||
await p.WaitForExitAsync(ct);
|
||||
p.ExitCode.ShouldBe(0, $"{script} {ceiling}: {await p.StandardError.ReadToEndAsync(ct)}");
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Counts a gateway container's gRPC calls by method from its request log: the calls since the last
|
||||
/// <see cref="MarkAsync"/> (a delta of whole-log counts, so the Docker VM's clock does not matter).
|
||||
/// </summary>
|
||||
private sealed class GatewayCallCounter(string? container)
|
||||
{
|
||||
private readonly Dictionary<string, int> _marks = new(StringComparer.Ordinal);
|
||||
|
||||
public async Task MarkAsync(string method, CancellationToken ct) => _marks[method] = await TotalAsync(method, ct);
|
||||
|
||||
public async Task<int> CountAsync(string method, CancellationToken ct)
|
||||
{
|
||||
if (container is null) return -1;
|
||||
await Task.Delay(TimeSpan.FromSeconds(2), ct); // let the container's request log reach `docker logs`
|
||||
return await TotalAsync(method, ct) - _marks.GetValueOrDefault(method);
|
||||
}
|
||||
|
||||
private async Task<int> TotalAsync(string method, CancellationToken ct)
|
||||
{
|
||||
if (container is null) return 0;
|
||||
using var p = Process.Start(new ProcessStartInfo("docker", ["logs", container])
|
||||
{
|
||||
RedirectStandardOutput = true, RedirectStandardError = true,
|
||||
})!;
|
||||
var stdout = p.StandardOutput.ReadToEndAsync(ct);
|
||||
var stderr = p.StandardError.ReadToEndAsync(ct);
|
||||
await p.WaitForExitAsync(ct);
|
||||
var text = await stdout + await stderr;
|
||||
return text.Split('\n').Count(l => l.Contains("Request starting", StringComparison.Ordinal)
|
||||
&& l.Contains($".HistorianRead/{method} ", StringComparison.Ordinal));
|
||||
}
|
||||
}
|
||||
|
||||
private static T Internal<T>(object instance, string property) =>
|
||||
(T)instance.GetType().GetProperty(property, System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.NonPublic)!
|
||||
.GetValue(instance)!;
|
||||
|
||||
/// <summary>
|
||||
/// The Part 13 interval grid (#677): forward — from StartTime, the clipped interval the latest, each stamped with its
|
||||
/// earlier edge; reverse — stepping back from StartTime, the clipped interval the earliest, stamped with its later edge,
|
||||
/// newest first. Each entry is (lo, hi, stamp, clipped).
|
||||
/// </summary>
|
||||
private static List<(DateTime Lo, DateTime Hi, DateTime Stamp, bool Clipped)> Grid(DateTime start, DateTime end, TimeSpan interval)
|
||||
{
|
||||
var list = new List<(DateTime, DateTime, DateTime, bool)>();
|
||||
if (start <= end)
|
||||
{
|
||||
for (var lo = start; lo < end; lo += interval)
|
||||
{
|
||||
var hi = lo + interval > end ? end : lo + interval;
|
||||
list.Add((lo, hi, lo, hi - lo < interval));
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
for (var hi = start; hi > end; hi -= interval)
|
||||
{
|
||||
var lo = hi - interval < end ? end : hi - interval;
|
||||
list.Add((lo, hi, hi, hi - lo < interval));
|
||||
}
|
||||
}
|
||||
return list;
|
||||
}
|
||||
|
||||
private static async Task WriteReportAsync(Report report, List<string> failures, string kind, CancellationToken ct)
|
||||
{
|
||||
var outDir = Env("OTOPCUA_AVEVA_GATE_OUT") ?? Path.GetTempPath();
|
||||
Directory.CreateDirectory(outDir);
|
||||
var file = Path.Combine(outDir, $"aveva-gate-{kind}-{Sanitize(report.Label)}.json");
|
||||
report.Failures = failures;
|
||||
await File.WriteAllTextAsync(file, JsonSerializer.Serialize(report, new JsonSerializerOptions { WriteIndented = true }), ct);
|
||||
TestContext.Current.TestOutputHelper?.WriteLine($"AVEVA gate report: {file} ({report.Cases.Count} cases, {failures.Count} failures)");
|
||||
foreach (var f in failures) TestContext.Current.TestOutputHelper?.WriteLine($"FAIL {f}");
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------ wire helpers
|
||||
|
||||
private static async Task<WireRead> ReadRawAsync(ISession session, NodeId node, DateTime start, DateTime end, uint max, CancellationToken ct)
|
||||
{
|
||||
var details = new ReadRawModifiedDetails { StartTime = start, EndTime = end, NumValuesPerNode = max, IsReadModified = false, ReturnBounds = false };
|
||||
var (status, rows, cp) = await HistoryReadAsync(session, details, node, default, ct);
|
||||
await ReleaseAsync(session, details, node, cp, ct);
|
||||
return new WireRead(status, rows, 1);
|
||||
}
|
||||
|
||||
private static async Task<WireRead> ReadRawPagedAsync(ISession session, NodeId node, DateTime start, DateTime end, uint page, CancellationToken ct)
|
||||
{
|
||||
var details = new ReadRawModifiedDetails { StartTime = start, EndTime = end, NumValuesPerNode = page, IsReadModified = false, ReturnBounds = false };
|
||||
return await PageAsync(session, details, node, ct);
|
||||
}
|
||||
|
||||
private static async Task<string> ReadModifiedAsync(ISession session, NodeId node, DateTime start, DateTime end, CancellationToken ct)
|
||||
{
|
||||
var details = new ReadRawModifiedDetails { StartTime = start, EndTime = end, NumValuesPerNode = 0, IsReadModified = true, ReturnBounds = false };
|
||||
var (status, _, cp) = await HistoryReadAsync(session, details, node, default, ct);
|
||||
await ReleaseAsync(session, details, node, cp, ct);
|
||||
return status;
|
||||
}
|
||||
|
||||
private static async Task<WireRead> ReadProcessedAsync(ISession session, NodeId node, DateTime start, DateTime end, double intervalMs,
|
||||
NodeId aggregate, CancellationToken ct)
|
||||
{
|
||||
var details = new ReadProcessedDetails
|
||||
{
|
||||
StartTime = start, EndTime = end, ProcessingInterval = intervalMs,
|
||||
AggregateType = new List<NodeId> { aggregate },
|
||||
AggregateConfiguration = new AggregateConfiguration { UseServerCapabilitiesDefaults = true },
|
||||
};
|
||||
return await PageAsync(session, details, node, ct);
|
||||
}
|
||||
|
||||
private static async Task<WireRead> PageAsync(ISession session, IEncodeable details, NodeId node, CancellationToken ct)
|
||||
{
|
||||
var all = new List<Row>();
|
||||
var pages = 0;
|
||||
ByteString cp = default;
|
||||
string status;
|
||||
do
|
||||
{
|
||||
(status, var rows, cp) = await HistoryReadAsync(session, details, node, cp, ct);
|
||||
all.AddRange(rows);
|
||||
pages++;
|
||||
if (status.StartsWith("Bad", StringComparison.Ordinal)) break;
|
||||
}
|
||||
while (!cp.IsEmpty && pages < 10_000);
|
||||
return new WireRead(status, all, pages);
|
||||
}
|
||||
|
||||
private static async Task<WireRead> ReadAtTimeAsync(ISession session, NodeId node, IReadOnlyList<DateTime> times, CancellationToken ct)
|
||||
{
|
||||
var details = new ReadAtTimeDetails { ReqTimes = times.Select(t => (DateTimeUtc)t).ToList(), UseSimpleBounds = true };
|
||||
var (status, rows, cp) = await HistoryReadAsync(session, details, node, default, ct);
|
||||
await ReleaseAsync(session, details, node, cp, ct);
|
||||
return new WireRead(status, rows, 1);
|
||||
}
|
||||
|
||||
private static async Task<(string Status, List<Row> Rows, ByteString Cp)> HistoryReadAsync(
|
||||
ISession session, IEncodeable details, NodeId node, ByteString cp, CancellationToken ct)
|
||||
{
|
||||
var response = await session.HistoryReadAsync(null, new ExtensionObject(details), TimestampsToReturn.Both, false,
|
||||
new HistoryReadValueId[] { new() { NodeId = node, ContinuationPoint = cp } }, ct);
|
||||
var result = response.Results.ToList().ShouldHaveSingleItem();
|
||||
var rows = new List<Row>();
|
||||
if (!result.HistoryData.IsNull && ExtensionObject.ToEncodeable(result.HistoryData) is HistoryData data)
|
||||
rows.AddRange(data.DataValues.ToList().Select(dv =>
|
||||
{
|
||||
var v = dv.ClrValue();
|
||||
return new Row(Fmt(v), dv.StatusCode.Code, Ts(dv.SourceTimestamp), v?.GetType().Name);
|
||||
}));
|
||||
return (Name(result.StatusCode), rows, result.ContinuationPoint);
|
||||
}
|
||||
|
||||
private static async Task ReleaseAsync(ISession session, IEncodeable details, NodeId node, ByteString cp, CancellationToken ct)
|
||||
{
|
||||
if (cp.IsEmpty) return;
|
||||
await session.HistoryReadAsync(null, new ExtensionObject(details), TimestampsToReturn.Both, true,
|
||||
new HistoryReadValueId[] { new() { NodeId = node, ContinuationPoint = cp } }, ct);
|
||||
}
|
||||
|
||||
private static async Task<EventRead> ReadEventsAsync(ISession session, NodeId node, DateTime start, DateTime end, uint page, CancellationToken ct)
|
||||
{
|
||||
var filter = new EventFilter();
|
||||
foreach (var field in new[] { "EventId", "Time", "SourceName", "Severity" })
|
||||
filter.SelectClauses += new SimpleAttributeOperand(ObjectTypeIds.BaseEventType, new QualifiedName(field)) { AttributeId = Attributes.Value };
|
||||
var details = new ReadEventDetails { StartTime = start, EndTime = end, NumValuesPerNode = page, Filter = filter };
|
||||
var rows = new List<EventRow>();
|
||||
var pages = 0;
|
||||
ByteString cp = default;
|
||||
string status;
|
||||
do
|
||||
{
|
||||
var response = await session.HistoryReadAsync(null, new ExtensionObject(details), TimestampsToReturn.Both, false,
|
||||
new HistoryReadValueId[] { new() { NodeId = node, ContinuationPoint = cp } }, ct);
|
||||
var result = response.Results.ToList().ShouldHaveSingleItem();
|
||||
status = Name(result.StatusCode);
|
||||
cp = result.ContinuationPoint;
|
||||
pages++;
|
||||
if (!result.HistoryData.IsNull && ExtensionObject.ToEncodeable(result.HistoryData) is HistoryEvent he)
|
||||
foreach (var e in he.Events.ToList())
|
||||
{
|
||||
var f = e.EventFields.ToList();
|
||||
rows.Add(new EventRow(
|
||||
f[0].ClrValue() is byte[] id ? System.Text.Encoding.UTF8.GetString(id) : Fmt(f[0].ClrValue()),
|
||||
f[1].ClrValue() as DateTime? ?? default, f[2].ClrValue() as string,
|
||||
Convert.ToUInt16(f[3].ClrValue() ?? 0, CultureInfo.InvariantCulture)));
|
||||
}
|
||||
if (status.StartsWith("Bad", StringComparison.Ordinal)) break;
|
||||
}
|
||||
while (!cp.IsEmpty && pages < 10_000);
|
||||
return new EventRead(status, rows, pages);
|
||||
}
|
||||
|
||||
private static async Task<string> AggregateFunctionsAsync(ISession session, CancellationToken ct)
|
||||
{
|
||||
var response = await session.BrowseAsync(null, null, 0,
|
||||
new BrowseDescription[]
|
||||
{
|
||||
new()
|
||||
{
|
||||
NodeId = ObjectIds.HistoryServerCapabilities_AggregateFunctions, BrowseDirection = BrowseDirection.Forward,
|
||||
ReferenceTypeId = ReferenceTypeIds.Organizes, IncludeSubtypes = true, NodeClassMask = 0,
|
||||
ResultMask = (uint)BrowseResultMask.All,
|
||||
},
|
||||
}, ct);
|
||||
return string.Join(",", response.Results[0].References.ToList().Select(r => r.BrowseName.Name).OrderBy(n => n, StringComparer.Ordinal));
|
||||
}
|
||||
|
||||
private static DateTime? Ts(DateTimeUtc t) => (DateTime)t == DateTime.MinValue ? null : DateTime.SpecifyKind((DateTime)t, DateTimeKind.Utc);
|
||||
|
||||
private static string Name(StatusCode code) => code.SymbolicId is { Length: > 0 } s ? s : $"0x{code.Code:X8}";
|
||||
|
||||
private static List<Row> Rows(IReadOnlyList<DataValueSnapshot> samples) =>
|
||||
samples.Select(s => new Row(Fmt(s.Value), s.StatusCode, s.SourceTimestampUtc, s.Value?.GetType().Name)).ToList();
|
||||
|
||||
private static string Types(IEnumerable<Row> rows) =>
|
||||
"types=" + string.Join("/", rows.Select(r => r.Type ?? "null").Distinct(StringComparer.Ordinal));
|
||||
|
||||
private static string? Fmt(object? value) => value switch
|
||||
{
|
||||
null => null,
|
||||
double d => d.ToString("R", CultureInfo.InvariantCulture),
|
||||
float f => ((double)f).ToString("R", CultureInfo.InvariantCulture),
|
||||
bool b => b ? "true" : "false",
|
||||
DateTime t => t.ToUniversalTime().ToString("o", CultureInfo.InvariantCulture),
|
||||
IFormattable f => f.ToString(null, CultureInfo.InvariantCulture),
|
||||
_ => value.ToString(),
|
||||
};
|
||||
|
||||
/// <summary>Wire vs reference: value text, status and source timestamp (types may differ: the backend answers doubles).</summary>
|
||||
private static void CheckSame(List<string> failures, string what, IReadOnlyList<Row> wire, IReadOnlyList<Row> reference)
|
||||
{
|
||||
if (wire.Count != reference.Count)
|
||||
{
|
||||
failures.Add($"{what}: wire {wire.Count} rows vs reference {reference.Count}");
|
||||
return;
|
||||
}
|
||||
for (var i = 0; i < wire.Count; i++)
|
||||
{
|
||||
var (x, y) = (wire[i], reference[i]);
|
||||
if (!SameValue(x.V, y.V) || x.Status != y.Status || x.Src != y.Src)
|
||||
{
|
||||
failures.Add($"{what}[{i}]: wire ({x.V}, 0x{x.Status:X8}, {x.Src:o}) vs reference ({y.V}, 0x{y.Status:X8}, {y.Src:o})");
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static bool SameValue(string? a, string? b)
|
||||
{
|
||||
if (a == b) return true;
|
||||
if (a is null || b is null) return false;
|
||||
if (double.TryParse(a, NumberStyles.Float, CultureInfo.InvariantCulture, out var x)
|
||||
&& double.TryParse(b, NumberStyles.Float, CultureInfo.InvariantCulture, out var y))
|
||||
return x == y || Math.Abs(x - y) <= 1e-6 * Math.Max(1, Math.Abs(y)); // a Float node narrows the backend's double
|
||||
return (a, b) is ("true", "1") or ("false", "0") or ("1", "true") or ("0", "false");
|
||||
}
|
||||
|
||||
private static void Check(List<string> failures, bool ok, string message)
|
||||
{
|
||||
if (!ok) failures.Add(message);
|
||||
}
|
||||
|
||||
private static DateTime Utc(string s) =>
|
||||
DateTime.Parse(s, CultureInfo.InvariantCulture, DateTimeStyles.AdjustToUniversal | DateTimeStyles.AssumeUniversal);
|
||||
|
||||
private static string? Env(string name) => Environment.GetEnvironmentVariable(name) is { Length: > 0 } v ? v.Trim() : null;
|
||||
|
||||
private static string Sanitize(string s) => string.Concat(s.Select(c => char.IsLetterOrDigit(c) || c is '.' or '-' ? c : '_'));
|
||||
|
||||
private sealed record Row(string? V, uint Status, DateTime? Src, string? Type);
|
||||
|
||||
private sealed record EventRow(string? Id, DateTime Time, string? Source, ushort Severity);
|
||||
|
||||
private sealed record WireRead(string Status, List<Row> Rows, int Pages);
|
||||
|
||||
private sealed record EventRead(string Status, List<EventRow> Rows, int Pages);
|
||||
|
||||
private sealed record Case(
|
||||
string Name, string? Tag, string? Window, string Path, string Status, string? Note, List<Row>? Rows,
|
||||
Dictionary<string, string?>? Attributes = null, List<EventRow>? Events = null);
|
||||
|
||||
private sealed class Report(string label, string endpoint)
|
||||
{
|
||||
public string Label { get; } = label;
|
||||
public string Endpoint { get; } = endpoint;
|
||||
public string SdkServerAssembly { get; } = typeof(Opc.Ua.Server.StandardServer).Assembly.GetName().Version?.ToString() ?? "?";
|
||||
public DateTime RunUtc { get; } = DateTime.UtcNow;
|
||||
public List<Case> Cases { get; } = [];
|
||||
public List<string> Failures { get; set; } = [];
|
||||
public void Add(Case c) => Cases.Add(c);
|
||||
}
|
||||
}
|
||||
+5
-4
@@ -64,7 +64,8 @@ public sealed class AvevaGatewayHistoryLiveGateTests
|
||||
private const string MissingTag = "OtOpcUa.Gate801.DoesNotExist";
|
||||
private const double IntervalMs = 60_000;
|
||||
|
||||
// The ten per-sample aggregates the Gateway backend serves (#681) and a set it must refuse (time-weighted / never).
|
||||
// The aggregates the Gateway backend serves: the ten per-sample ones (#681) plus AVEVA's native TimeAverage / Total (on by
|
||||
// default since lmxopcua#694), and a set it must refuse (the other time-weighted ones / never).
|
||||
private static readonly (string Name, NodeId Id, HistoryAggregateType? Domain)[] Served =
|
||||
[
|
||||
("Average", ObjectIds.AggregateFunction_Average, HistoryAggregateType.Average),
|
||||
@@ -77,12 +78,12 @@ public sealed class AvevaGatewayHistoryLiveGateTests
|
||||
("Delta", ObjectIds.AggregateFunction_Delta, HistoryAggregateType.Delta),
|
||||
("MinimumActualTime", ObjectIds.AggregateFunction_MinimumActualTime, HistoryAggregateType.MinimumActualTime),
|
||||
("MaximumActualTime", ObjectIds.AggregateFunction_MaximumActualTime, HistoryAggregateType.MaximumActualTime),
|
||||
("TimeAverage", ObjectIds.AggregateFunction_TimeAverage, HistoryAggregateType.TimeAverage),
|
||||
("Total", ObjectIds.AggregateFunction_Total, HistoryAggregateType.Total),
|
||||
];
|
||||
|
||||
private static readonly (string Name, NodeId Id)[] Refused =
|
||||
[
|
||||
("TimeAverage", ObjectIds.AggregateFunction_TimeAverage),
|
||||
("Total", ObjectIds.AggregateFunction_Total),
|
||||
("TimeAverage2", ObjectIds.AggregateFunction_TimeAverage2),
|
||||
("Interpolative", ObjectIds.AggregateFunction_Interpolative),
|
||||
("PercentGood", ObjectIds.AggregateFunction_PercentGood),
|
||||
@@ -186,7 +187,7 @@ public sealed class AvevaGatewayHistoryLiveGateTests
|
||||
var caps = await ReadHistoryCapabilitiesAsync(session, ct);
|
||||
report.Add(new Case("metadata.historyServerCapabilities", null, null, "wire", "Good", null, caps));
|
||||
Check(failures, caps["AggregateFunctions"] == string.Join(",", Served.Select(a => a.Name).OrderBy(n => n, StringComparer.Ordinal)),
|
||||
$"advertised history aggregates must be exactly the Gateway's ten, got {caps["AggregateFunctions"]}");
|
||||
$"advertised history aggregates must be exactly the Gateway's twelve, got {caps["AggregateFunctions"]}");
|
||||
// lmxopcua#881: the Boolean capabilities follow the Gateway backend — data + event reads, no HistoryUpdate path, no
|
||||
// stored server timestamps, never event / annotation updates (this gate recorded all-false before the fix).
|
||||
foreach (var flag in new[] { "AccessHistoryDataCapability", "AccessHistoryEventsCapability" })
|
||||
|
||||
Reference in New Issue
Block a user