docs(evidence): #801 real-AVEVA history gate — SDK 2.0.1 identical to 1.5.378.182
v2-ci / unit-tests (push) Waiting to run
v2-ci / unit-tests (push) Waiting to run
Both builds served HistoryRead from the real AVEVA Historian via the DEV HistorianGateway (vd03, ea9f21c): 225 cases / 3,855 rows / events, 0 differences. Raw also matches direct SQL on Runtime.dbo.History (6/6) and histgw-cli; simple aggregates match a recomputation from raw (336/336). Modified, HistoryUpdate, seven refused aggregates and HistoryServerCapabilities answer as before. No write reached the historian. Findings (pre-existing, not SDK): all HistoryServerCapabilities booleans false; AVEVA attribute-alarm events unreachable per source. Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
@@ -0,0 +1,78 @@
|
||||
# #801: real AVEVA history through HistorianGateway, SDK 2.0.1 vs 1.5.378.182 (2026-11-12)
|
||||
|
||||
**Scope.** This covers the "real AVEVA historian" part of [#801](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/801) only.
|
||||
Consumer compatibility, pair/failover and #800 are separate gates and are not covered here.
|
||||
|
||||
**Result.** SDK 2.0.1 serves HistoryRead from the real AVEVA Historian with output identical to SDK 1.5.378.182. Both builds
|
||||
ran the same queries over the same fixed past windows, through the same gateway. The comparison covered 225 cases,
|
||||
3,855 rows and the event rows, and found 0 differences.
|
||||
|
||||
Raw rows also match two independent sources:
|
||||
- direct SQL on `Runtime.dbo.History`, read-only (6/6 tag-windows);
|
||||
- HistorianGateway's own CLI.
|
||||
|
||||
The simple aggregates also match a recomputation from the raw rows (336/336).
|
||||
|
||||
Unsupported capabilities answer the same way on both builds: modified history, HistoryUpdate, the seven refused
|
||||
aggregates and the HistoryServerCapabilities flags.
|
||||
|
||||
## Setup
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Server under test | Local `d46044f5` = master `438484fd` + this harness (SDK **2.0.1**). It was pushed as `f607ee75` after a rebase onto `c266971d`, which adds only #789 docs and a ScriptRunner perf test, so no server or historian code changed. Baseline: `efb9197a`, master's first parent before the SDK 2 merge `b36e688f` (SDK **1.5.378.182**). Both use HistorianGateway Client/Contracts **1.0.0**. |
|
||||
| How | `AvevaGatewayHistoryLiveGateTests` boots the OtOpcUa server in-process on the Mac. It wires the **production** Gateway backend (`HistorianBackendRegistry` → `GatewayHistorianDataSource`, default `ServerHistorianOptions`) and reads over a real SDK client session (`None`, anonymous, ACL off). The SDK 1 build runs the same file with [`sdk1-baseline-port.patch`](../../../../scripts/validation/aveva-history-801/sdk1-baseline-port.patch), which changes API spelling only. |
|
||||
| Gateway | wonder-app-vd03 (DEV) `http://…:5221` (h2c), HistorianGateway **`ea9f21c`**. The SHA was read from the deployed `Server.dll` over SMB. The gateway talks gRPC to the AVEVA 2023 R2 historian on wonder-sql-vd03. The gateway config and service were not touched. |
|
||||
| Credential | vd03 OtOpcUa's existing `ServerHistorian__ApiKey` (`OtOpcUa-wonder`, from the box's env backup, read over SMB). It was used only through env/a 0600 file and was never printed. A bogus key gets `Unauthenticated` (negative control). |
|
||||
| Tags | `ZIPSEN.WTC1` (float, ~2 s), `ZIPSEN.OvenPressureTransducer` (float, sparse), `SysPerfCPUTotal` (5 s). The tag `Z28061` was not used. |
|
||||
| Windows (UTC) | W1 `2026-11-10 11:30–11:41`: brackets the DevWinPlatform System.Stop at 11:32:35. It has a stored NULL/Bad row at 11:32:36, a gap, and a recovery row at 11:38:34 (QualityDetail 248). W2 `2026-11-11 14:00–14:10`: steady. Events: `2026-11-05 15:00` → `2026-11-12 00:00`. |
|
||||
| Writes | **None reached the historian.** The Gateway provider has no updater (asserted). The only HistoryUpdate went to a node bound to the designated sandbox `HistGW.LiveTest.Sandbox` (`HISTORIAN_WRITE_SANDBOX_TAG`), and the server refused it. No provisioner, recorder or alarm writer was built. |
|
||||
| Runs | SDK 2 run at 2026-11-12 15:00:06Z, SDK 1 run straight after. Both passed with every in-test assertion green. |
|
||||
|
||||
## Capability table
|
||||
|
||||
"Reference" means the same backend's direct `IHistorianDataSource` answer (asserted in-test, row by row: value, StatusCode,
|
||||
SourceTimestamp), plus the SDK 1 run (diffed) and, for raw, SQL and histgw-cli.
|
||||
|
||||
| Capability | Query | SDK 2 result | Reference | Verdict |
|
||||
|---|---|---|---|---|
|
||||
| Raw | `ReadRawModifiedDetails`, `NumValuesPerNode=0`, `ReturnBounds=false`, 3 tags × 2 windows | 80 / 300 / 14 / 35 / 51 / 120 rows. W1's stored NULL is **null + `Bad` (0x80000000)**. QD 248 and QD 44 rows are Good. AVEVA's synthetic Q133 start row is dropped (#687). | Backend: identical. SDK 1: identical. SQL `History` (Full, UTC): 6/6 tag-windows match on value, ms timestamp and Good/Bad class. histgw-cli: 4/6 complete matches. On the 2 W1 windows of the ZIPSEN tags, histgw-cli stops at the NULL, but the prefix matches. | **PASS** |
|
||||
| Raw paging | same, page 37 via continuation points | Concatenated pages (1–9 pages) equal the unpaged read. | identical on SDK 1 | **PASS** |
|
||||
| Raw reverse | `StartTime > EndTime`, 25 | Newest first. The Part 11 window `(start, end]` includes a sample stored exactly at the end (SysPerfCPUTotal 11:41:00). | Backend `(start, end]`, reversed: identical. SDK 1: identical. | **PASS** |
|
||||
| Raw bounds | `ReturnBounds=true` | Real prior and next samples are added at each end (e.g. 11:29:59.62 / 11:44:09.107). | SDK 1: identical | **PASS** |
|
||||
| Modified | `IsReadModified=true` | `BadHistoryOperationUnsupported` | SDK 1: identical (Gateway has no modified history) | **PASS** (unsupported preserved) |
|
||||
| Processed | 60 s interval, Average, Count, Minimum, Maximum, Range, Start, End, Delta, MinimumActualTime, MaximumActualTime | Values match. Statuses: Good+Calculated `0x00000401`. The outage interval is `UncertainDataSubNormal` `0x40A40401`. Empty intervals are `BadNoData` `0x809B0401`. Start/End carry the raw sample's own time and status (End of 11:32 = the NULL: null + Bad). | Backend: identical (asserted). SDK 1: identical (application aggregate semantics unchanged, `TreatUncertainAsBad=false`). Recomputed from raw: 336/336 non-empty all-Good intervals. | **PASS** |
|
||||
| Processed, refused | TimeAverage, Total, TimeAverage2, Interpolative, PercentGood, DurationGood, StandardDeviationSample | `BadAggregateNotSupported` | SDK 1: identical. Native time-weighted is still off; no string backfill. | **PASS** (no new aggregates) |
|
||||
| AtTime | window start, a stored time, a midpoint | Stored value exact (Good). Midpoints are linear + Interpolated `0x00000402`. | Backend: identical. SDK 1: identical. | **PASS** |
|
||||
| Events | `ReadEventDetails` on folder `DevWinPlatform`: all, 1-event pages, reverse | 2 events, `System.Stop` 11:32:35.095Z / `System.Start` 11:43:46.563Z, with EventId, Time, ReceiveTime, SourceName, Message and Severity. Paging takes 2 pages. Reverse returns the newest first. | Backend: identical. SDK 1: identical. | **PASS** |
|
||||
| Events, attribute alarm | same, folder `DevWinPlatform.CPULoadAvg` | `GoodNoData`. The gateway reports these 70 events with SourceName `DevWinPlatform.CPULoadAvg` in an unfiltered read, but a per-source read for that name returns none (Source_Object is `DevWinPlatform`). | SDK 1: identical | **PASS** (pre-existing gap, see Findings) |
|
||||
| HistoryUpdate | Insert and DeleteRaw on the `HistGW.LiveTest.Sandbox`-bound node | `BadHistoryOperationUnsupported` on both | SDK 1: identical. No updater, no backend call. | **PASS** (unsupported preserved) |
|
||||
| Node metadata | Historizing, AccessLevel, UserAccessLevel | `true`, `5` (CurrentRead + HistoryRead, **no HistoryWrite**), `5`, DataType `i=11` | SDK 1: identical | **PASS** |
|
||||
| History capabilities | `HistoryServerCapabilities` children + AggregateFunctions | AggregateFunctions = the Gateway's ten. Every boolean capability flag is `false` (incl. AccessHistoryData/Events). `MaxReturnDataValues=10000`, `MaxReturnEventValues=0`. | SDK 1: identical | **PASS** (existing metadata preserved, see Findings) |
|
||||
| Missing tag | raw on `OtOpcUa.Gate801.DoesNotExist` | `GoodNoData`, 0 rows (#653) | SDK 1: identical | **PASS** |
|
||||
| SysTimeSec raw | 10 s window | Good, 10 rows, 69 ms | SDK 1: identical | **PASS**, informational. historiangw#29's timeout did **not** reproduce on `ea9f21c`. |
|
||||
|
||||
ServerTimestamp on history values is the read time (`now`) on both builds. Its presence is compared; its value is not.
|
||||
|
||||
## Findings (none caused by the SDK)
|
||||
|
||||
1. **The boolean flags under `HistoryServerCapabilities` are all `false`** on both SDKs, including AccessHistoryDataCapability and
|
||||
AccessHistoryEventsCapability, even though the server serves data and event history. Clients that check these flags before
|
||||
calling HistoryRead would skip it. This is existing behaviour and the upgrade preserves it. It is a candidate follow-up, not filed.
|
||||
2. **AVEVA attribute-alarm events cannot be read per source.** An unfiltered read reports SourceName = `Source_ProcessVariable`
|
||||
(e.g. `DevWinPlatform.CPULoadAvg`), but a per-source read with that name returns nothing. Reading by `Source_Object` returns
|
||||
only the rows whose SourceName equals it: OtOpcUa's client-side filter drops the rest. Native AVEVA alarms are therefore
|
||||
invisible through OtOpcUa's event history unless their object-level name matches. This is a gateway/consumer contract gap,
|
||||
identical on SDK 1. It is a candidate follow-up, not filed.
|
||||
3. **historiangw `histgw-cli read-raw` aborts on a stored NULL.** Its JSON writer cannot emit NaN. This is a CLI-only issue: OtOpcUa
|
||||
maps the same row to null + Bad.
|
||||
4. **historiangw#29 (SysTimeSec raw timeout) did not reproduce** against the `ea9f21c` gateway.
|
||||
|
||||
## Files
|
||||
|
||||
- [`aveva-history-801/aveva-gate-sdk2-d46044f5.json.gz`](aveva-history-801/aveva-gate-sdk2-d46044f5.json.gz), sha256 of the JSON `00b5ecb6…fc27`.
|
||||
- [`aveva-history-801/aveva-gate-sdk1-efb9197a.json.gz`](aveva-history-801/aveva-gate-sdk1-efb9197a.json.gz), sha256 `21fe4985…751d`.
|
||||
- [`aveva-history-801/compare.txt`](aveva-history-801/compare.txt): diff, SQL, recompute and histgw-cli outputs.
|
||||
[`histgw-cli-read-raw.tar.gz`](aveva-history-801/histgw-cli-read-raw.tar.gz) holds the CLI rows.
|
||||
[`test-runs.txt`](aveva-history-801/test-runs.txt) holds the two test-run summaries.
|
||||
- Reproduction: [`scripts/validation/aveva-history-801/README.md`](../../../../scripts/validation/aveva-history-801/README.md).
|
||||
BIN
Binary file not shown.
BIN
Binary file not shown.
@@ -0,0 +1,36 @@
|
||||
$ diff_runs.py sdk2 sdk1
|
||||
A=sdk2-d46044f5 sdk=2.0.1.0 cases=225 failures=0
|
||||
B=sdk1-efb9197a sdk=1.5.378.0 cases=225 failures=0
|
||||
compared 225 cases: identical=225 differing=0; rows compared=3855, events compared=8 (ServerTimestamp compared for presence only: it is the read time)
|
||||
|
||||
$ sql_raw_check.py sdk2-d46044f5
|
||||
ZIPSEN.WTC1 2026-11-10T11:30:00 sql_stored=80 (boundary rows dropped=1) wire=80 quality(Q,QD,OPC)={('0', '192', '192'): 78, ('1', '24', '0'): 1, ('0', '248', '192'): 1} -> MATCH
|
||||
ZIPSEN.WTC1 2026-11-11T14:00:00 sql_stored=300 (boundary rows dropped=1) wire=300 quality(Q,QD,OPC)={('0', '192', '192'): 300} -> MATCH
|
||||
ZIPSEN.OvenPressureTransducer 2026-11-10T11:30:00 sql_stored=14 (boundary rows dropped=1) wire=14 quality(Q,QD,OPC)={('0', '192', '192'): 12, ('1', '24', '0'): 1, ('0', '248', '192'): 1} -> MATCH
|
||||
ZIPSEN.OvenPressureTransducer 2026-11-11T14:00:00 sql_stored=35 (boundary rows dropped=1) wire=35 quality(Q,QD,OPC)={('0', '192', '192'): 35} -> MATCH
|
||||
SysPerfCPUTotal 2026-11-10T11:30:00 sql_stored=51 (boundary rows dropped=0) wire=51 quality(Q,QD,OPC)={('0', '192', '192'): 50, ('0', '44', '192'): 1} -> MATCH
|
||||
SysPerfCPUTotal 2026-11-11T14:00:00 sql_stored=120 (boundary rows dropped=0) wire=120 quality(Q,QD,OPC)={('0', '192', '192'): 120} -> MATCH
|
||||
6 tag-windows compared, 0 mismatching
|
||||
|
||||
$ aggregate_recompute.py sdk2-d46044f5
|
||||
336 non-empty interval values recomputed from raw, 0 mismatching
|
||||
|
||||
$ sql_raw_check.py sdk1-efb9197a
|
||||
ZIPSEN.WTC1 2026-11-10T11:30:00 sql_stored=80 (boundary rows dropped=1) wire=80 quality(Q,QD,OPC)={('0', '192', '192'): 78, ('1', '24', '0'): 1, ('0', '248', '192'): 1} -> MATCH
|
||||
ZIPSEN.WTC1 2026-11-11T14:00:00 sql_stored=300 (boundary rows dropped=1) wire=300 quality(Q,QD,OPC)={('0', '192', '192'): 300} -> MATCH
|
||||
ZIPSEN.OvenPressureTransducer 2026-11-10T11:30:00 sql_stored=14 (boundary rows dropped=1) wire=14 quality(Q,QD,OPC)={('0', '192', '192'): 12, ('1', '24', '0'): 1, ('0', '248', '192'): 1} -> MATCH
|
||||
ZIPSEN.OvenPressureTransducer 2026-11-11T14:00:00 sql_stored=35 (boundary rows dropped=1) wire=35 quality(Q,QD,OPC)={('0', '192', '192'): 35} -> MATCH
|
||||
SysPerfCPUTotal 2026-11-10T11:30:00 sql_stored=51 (boundary rows dropped=0) wire=51 quality(Q,QD,OPC)={('0', '192', '192'): 50, ('0', '44', '192'): 1} -> MATCH
|
||||
SysPerfCPUTotal 2026-11-11T14:00:00 sql_stored=120 (boundary rows dropped=0) wire=120 quality(Q,QD,OPC)={('0', '192', '192'): 120} -> MATCH
|
||||
6 tag-windows compared, 0 mismatching
|
||||
|
||||
$ aggregate_recompute.py sdk1-efb9197a
|
||||
336 non-empty interval values recomputed from raw, 0 mismatching
|
||||
|
||||
$ hgcli_raw_check.py sdk2
|
||||
SysPerfCPUTotal 2026-11-10T11:30:00 histgw-cli=51 wire=51 compared=51 MATCH
|
||||
SysPerfCPUTotal 2026-11-11T14:00:00 histgw-cli=120 wire=120 compared=120 MATCH
|
||||
ZIPSEN.OvenPressureTransducer 2026-11-10T11:30:00 histgw-cli=11 wire=14 compared=11 MATCH (histgw-cli stopped at the stored NULL: its JSON writer cannot emit NaN)
|
||||
ZIPSEN.OvenPressureTransducer 2026-11-11T14:00:00 histgw-cli=35 wire=35 compared=35 MATCH
|
||||
ZIPSEN.WTC1 2026-11-10T11:30:00 histgw-cli=77 wire=80 compared=77 MATCH (histgw-cli stopped at the stored NULL: its JSON writer cannot emit NaN)
|
||||
ZIPSEN.WTC1 2026-11-11T14:00:00 histgw-cli=300 wire=300 compared=300 MATCH
|
||||
Binary file not shown.
@@ -0,0 +1,2 @@
|
||||
Passed! - Failed: 0, Passed: 1, Skipped: 0, Total: 1, Duration: 23 s - ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests.dll (net10.0)
|
||||
Passed! - Failed: 0, Passed: 1, Skipped: 0, Total: 1, Duration: 17 s - ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests.dll (net10.0)
|
||||
Reference in New Issue
Block a user