diff --git a/CLAUDE.md b/CLAUDE.md index 92f1e90..e4331e2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -71,7 +71,7 @@ Three layered subsystems, intentionally decoupled so protocol parsing can be uni - **`Wcf/`** — managed WCF/MDAS layer. The Historian uses Net.TCP on port `32568` with a custom `application/x-mdas` content type wrapping a binary SOAP 1.2 / WS-Addressing 1.0 envelope. `MdasMessageEncoder` + `MdasMessageEncodingBindingElement` implement that wrapper. `HistorianWcfBindingFactory` produces three flavors: plain MDAS, MDAS+Windows transport (used for `/Hist-Integrated`), and MDAS+certificate (used for `/HistCert`). Service paths live in `HistorianWcfServiceNames`. WCF data contracts (`Wcf/Contracts/`) are reproduced from server-side static analysis and are versioned per native interface (e.g., `IRetrievalServiceContract2..4`). - **`Protocol/`** — binary frame layer (`HistorianFrameReader`/`Writer`, `HistorianBinaryPrimitives`, `HistorianMessageType`). `Historian2020ProtocolDialect` is the version-anchored bridge between `HistorianClient` and the frame layer; methods without sufficient evidence throw `ProtocolEvidenceMissingException` rather than guessing wire bytes. - **`Transport/`** — pluggable `IHistorianTransport` (default: TCP). Tests inject a fake transport. -- **`Grpc/`** — 2023 R2 gRPC transport (`HistorianTransport.RemoteGrpc`). The recovered protobuf contract lives in `Grpc/Protos/*.proto` and is compiled to client stubs at build time by `Grpc.Tools`. `HistorianGrpcChannelFactory` builds a gRPC-Web/HTTP-1.1 channel (default port `32565`, optional TLS, gzip) matching the stock 2023 R2 client. `HistorianGrpcReadOrchestrator` mirrors `HistorianWcfReadOrchestrator` but over gRPC: it reuses the exact native serializers/parsers — the same Open2 buffer, SSPI/NTLM tokens, and `DataQueryRequest`/result buffers travel inside protobuf `bytes` fields. The 2020→gRPC op map: `Hist.ValCl`→`HistoryService.ExchangeKey`, `Hist.Open2`→`HistoryService.OpenConnection`, `Retr.StartQuery2`→`RetrievalService.StartQuery`, `Retr.GetNextQueryResultBuffer2`→`RetrievalService.GetNextQueryResultBuffer`. The transport-agnostic handshake (Open2 request builder + SSPI token loop + response decode) is shared via `Wcf/HistorianNativeHandshake`. **Not yet live-verified against a 2023 R2 server** — the auth handshake op (`ExchangeKey`) is the first thing to revisit if a live server rejects it; the byte payloads are the proven 2020 protocol. Gated live test: set `HISTORIAN_GRPC_HOST` (+ `HISTORIAN_TEST_TAG`, optional `HISTORIAN_GRPC_PORT`/`HISTORIAN_GRPC_TLS`/`HISTORIAN_GRPC_DNSID`). +- **`Grpc/`** — 2023 R2 gRPC transport (`HistorianTransport.RemoteGrpc`). The recovered protobuf contract lives in `Grpc/Protos/*.proto` and is compiled to client stubs at build time by `Grpc.Tools`. `HistorianGrpcChannelFactory` builds a gRPC-Web/HTTP-1.1 channel (default port `32565`, optional TLS, gzip) matching the stock 2023 R2 client. `HistorianGrpcReadOrchestrator` mirrors `HistorianWcfReadOrchestrator` but over gRPC: it reuses the exact native serializers/parsers — the same Open2 buffer, SSPI/NTLM tokens, and `DataQueryRequest`/result buffers travel inside protobuf `bytes` fields. The 2020→gRPC op map: `Hist.ValCl`→`StorageService.ValidateClientCredential` (the SSPI/Negotiate token loop), `Hist.Open2`→`HistoryService.OpenConnection`, `Retr.StartQuery2`→`RetrievalService.StartQuery`, `Retr.GetNextQueryResultBuffer2`→`RetrievalService.GetNextQueryResultBuffer`. The transport-agnostic handshake (Open2 request builder + SSPI token loop + response decode) is shared via `Wcf/HistorianNativeHandshake`. **Live-verified 2026-06-21 against a real 2023 R2 server** (interface versions History=12, Retrieval=4, Storage=4): the full read chain returns rows. NOTE: `HistoryService.ExchangeKey` is a SEPARATE key-exchange/cert-path op, NOT the Negotiate loop — an earlier revision wrongly routed the token loop there and it was rejected at round 0 regardless of credentials; the loop belongs on `StorageService.ValidateClientCredential` (which kept the 2020 inBuff/outBuff token framing). The byte payloads are the proven 2020 protocol and transfer unchanged; only the History interface integer differs (12 vs 11) and is buffer-compatible, so `VerifyServerInterfaceVersion=false` is currently required against a v12 server (the gate still pins History=11). Gated live test: set `HISTORIAN_GRPC_HOST` (+ `HISTORIAN_TEST_TAG`, optional `HISTORIAN_GRPC_PORT`/`HISTORIAN_GRPC_TLS`/`HISTORIAN_GRPC_DNSID`); reach the live 2023 R2 box via [[reference_2023r2_live_server_access]]. - **`Models/`** — public DTOs and enums (`HistorianSample`, `RetrievalMode`, etc.). `HistorianDataValue` represents the discriminated value type. `InternalsVisibleTo` exposes internals to the test assembly and the reverse-engineering tool. diff --git a/docs/plans/hcal-roadmap.md b/docs/plans/hcal-roadmap.md index 5df3ba1..77525de 100644 --- a/docs/plans/hcal-roadmap.md +++ b/docs/plans/hcal-roadmap.md @@ -9,15 +9,54 @@ HCAL replacement, built on the **2023 R2 gRPC transport**. Derived from > protocol serializer/parser + golden-byte unit test + an env-gated live integration > test against the local Historian. -## Progress (updated 2026-06-19) +## Progress (updated 2026-06-21) - ✅ **R0.6 version gate** — `HistorianServerVersionGate` + `HistorianClientOptions.VerifyServerInterfaceVersion`; fail-closed on connect, wired into both WCF and gRPC paths. Supported versions are - evidence-based (Hist=11, Retr=4, Trx=2; Status reachability-only), captured from the - live server. 10 unit tests. + evidence-based (Hist=11/12, Retr=4, Trx=2; Status reachability-only), captured from the + live server. History 12 (2023 R2 gRPC) accepted alongside 11 (buffer-compatible). - ✅ **CW-1 capture pipeline** — `ProtocolCaptureSanitizer` + `ProtocolFixtureWriter` + `capture-tag-info` CLI command; produces sanitized `fixtures/protocol//` golden files. 11 unit tests. First fixture: `get-tag-info/analog-*.json`. +- ✅ **gRPC auth handshake (read chain)** — LIVE-VERIFIED 2026-06-21 against a real 2023 R2 + server: `ReadRawAsync` over `RemoteGrpc` returns rows. Token loop routes to + `StorageService.ValidateClientCredential`. Shared handshake extracted to + `Grpc/HistorianGrpcHandshake` for reuse by the status/browse/metadata paths. +- ✅ **R0.4 Probe over gRPC** — `Grpc/HistorianGrpcProbe` (History/Retrieval/Status + `GetInterfaceVersion`); `ProbeAsync` routes over gRPC when `Transport==RemoteGrpc`. + **LIVE-VERIFIED 2026-06-21** (no credentials required — runs before the auth loop). +- ✅ **R0.3 System parameter over gRPC** — `Grpc/HistorianGrpcStatusClient.GetSystemParameterAsync` + (`StatusService.GetSystemParameter`); routed in the dialect. Built + unit-tested + **LIVE-VERIFIED + 2026-06-21** against a real 2023 R2 server (returned `HistorianVersion`). Code path is the proven + handshake + a single string-in/string-out RPC. +- ✅ **R0.2 Tag metadata over gRPC** — `Grpc/HistorianGrpcTagClient.GetTagMetadataAsync` + (`RetrievalService.GetTagInfosFromName`, the plural **string-handle** op). `GetTagMetadataAsync` + routes over gRPC when `Transport==RemoteGrpc`. Request `btTagNames` = `uint count + per-name(uint + charCount + UTF-16LE)` (golden-byte unit-tested); response `btTagInfos` = `uint count + CTagMetadata` + records (reuses `ParseGetTagInfoResponse`); string handle = uppercase Open2 storage GUID. The 2020 + WCF string-handle wall does **not** apply on the gRPC front door (as predicted). **LIVE-VERIFIED + 2026-06-21** — `GetTagMetadataAsync` returned the requested tag + a valid data type. +- ✅ **R0.1 Browse over gRPC** — DONE, **LIVE-VERIFIED 2026-06-21**. + `HistorianClient.BrowseTagNamesAsync` routes over gRPC via + `Grpc/HistorianGrpcTagClient.BrowseTagNamesAsync`: StartTagQuery(**OData** filter) → paged + **QueryTag** (`btRequest` = `u16 0x6752 + u16 1 + u16 queryType + u32 startIndex + u32 count`) → + EndTagQuery; response = `u32 count + per-name(u32 charCount + UTF-16LE) + trailer`. The SDK glob + filter is translated by `GlobToODataFilter` (`Pre*`→`startswith`, `*suf`→`endswith`, `*mid*`→ + `contains`, exact→`eq`). The QueryTag packet-id `0x6752` was recovered from a `.rdata` + packet-descriptor table (`{0x6751,1}`=StartTagQuery, `{0x6752,1}`=QueryTag) — no Ghidra needed. + Golden-byte + glob unit tests + gated live test. Full finding: + `docs/reverse-engineering/grpc-tag-query-odata.md`. + +> ✅ **Milestone 0 (gRPC parity) is COMPLETE** — probe, system-param, metadata, and browse all run +> over `RemoteGrpc` and are live-verified against a real 2023 R2 server, alongside the read chain. + +> ℹ️ **Auth note (2026-06-21, resolved):** an apparent NTLM round-1 `SEC_E_LOGON_DENIED` blocker +> turned out to be a **test-harness credential-parsing bug**, not a server/account/SDK issue — the +> gitignored creds file stores **quoted** values (`"nam\user"`, `"pass"`), and the env-setup must +> **strip surrounding quotes** before exporting `HISTORIAN_USER`/`HISTORIAN_PASSWORD`. With quotes +> stripped, the domain account authenticates and the full read + system-param + probe chain passes +> live. The round-failure diagnostic added during the hunt is kept +> (`HistorianNativeHandshake.DescribeError` decodes the native error + hex/ASCII preview). > ⚠️ **Live-verification constraint:** the local Historian is **2020** (WCF, port 32568) — the > 2023 R2 gRPC endpoint (32565) is absent. M0's gRPC routing (R0.1–R0.4) can be built and diff --git a/docs/reverse-engineering/grpc-tag-query-odata.md b/docs/reverse-engineering/grpc-tag-query-odata.md new file mode 100644 index 0000000..4fc28c8 --- /dev/null +++ b/docs/reverse-engineering/grpc-tag-query-odata.md @@ -0,0 +1,57 @@ +# R0.1 browse over gRPC — StartTagQuery takes an OData filter (2026-06-21) + +Live-probed `RetrievalService.StartTagQuery` / `QueryTag` against a real **2023 R2** server over the +gRPC front door (string-handle = uppercase Open2 storage GUID). Key result: **browse is feasible on +2023 R2** — the 2020 WCF "metadata-server pipe" wall does **not** block here. + +## StartTagQuery — CRACKED + +`StartTagQuery(strHandle, btRequest)` where `btRequest` = the native +`marker(26449) + version(1) + WriteHistorianString(filter)` buffer +(`HistorianTagQueryProtocol.CreateStartTagQueryAttempt`). The server runs +`CMdServer::StartTagQuery::StartActiveTagnamesQuery` over `\\.\pipe\aahMetadataServer\console` and +**parses the filter string as OData** (not SQL-LIKE). Swept filters: + +| filter | result | +|---|---| +| `startswith(TagName,'Sys')` | ✅ success, 8-byte response | +| `contains(TagName,'Sys')` | ✅ success | +| `TagName eq 'SysTimeSec'` | ✅ success | +| `` (empty) | ✅ success (all tags) | +| `Sys*` / `*` | ❌ `ODataFilter ... bad token` | +| `TagName like 'Sys%'` / `Name like 'Sys%'` | ❌ rejected | + +Success response `btResponse` is the 8-byte `(queryHandle:uint, tagCount:uint)` pair +(`ParseStartTagQueryResponse`). Live: `startswith(TagName,'Sys')` → tagCount = 220. + +**Implication for the public API:** browse must translate the SDK's glob filter to OData — +`*` → empty, `Pre*` → `startswith(TagName,'Pre')`, `*sub*` → `contains(TagName,'sub')`, +exact → `TagName eq '...'`. (Escaping single-quotes in names still TBD.) + +## QueryTag — CRACKED (2026-06-21), browse SHIPPED + +`QueryTag(strHandle, uiQueryHandle, btRequest)` is the paging call that returns the tag-name rows. +The blocker was the packet id: every guessed `btRequest` returned native error **type 4 / code 72 = +`InvalidPacketId`** (`ArchestrA.CloudHistorian.Contract.ErrorCode.InvalidPacketId`). The generic +`0x6751` header that StartTagQuery accepts is the **wrong** id for QueryTag. + +**How it was found (no Ghidra needed):** a `.rdata` **packet-descriptor table** in +`aahClientManaged.dll` lists consecutive `{uint marker, uint version}` entries — +`{0x6751, 1}` (StartTagQuery) immediately followed by **`{0x6752, 1}`** (the paired op). Found by +`pefile` byte-scan of `.rdata` for `51 67 00 00` and dumping the surrounding dwords. Testing `0x6752` +live confirmed it. + +**QueryTag wire format (live-verified):** +- request `btRequest` = `u16 marker(0x6752) + u16 version(1) + u16 queryType + u32 startIndex + u32 count` + — `queryType = 1` returns tag-name rows (`queryType = 0` returns an empty/count-only page). +- response `btResonse` = `u32 count + per-name(u32 charCount + UTF-16LE) + trailer` + (the trailer is the CloudHistorian `NextIndex`/`TagMetadataBuffer` region — ignored by + `HistorianTagQueryProtocol.ParseTagNameQueryPage`). +- Semantic fields match `ArchestrA.CloudHistorian.Contract.QueryTagRequest` + (`QueryType/StartIndex/TagCount`; the QueryHandle travels in the protobuf `uiQueryHandle`). + +**Browse is shipped:** `HistorianClient.BrowseTagNamesAsync` routes over gRPC when +`Transport==RemoteGrpc` via `Grpc/HistorianGrpcTagClient.BrowseTagNamesAsync` +(StartTagQuery(OData) → paged QueryTag(0x6752) → EndTagQuery), with the SDK glob filter translated by +`GlobToODataFilter`. Golden-byte + glob unit tests and a gated live test +(`BrowseTagNamesAsync_OverGrpc_ReturnsSystemTags`) cover it. **M0 gRPC parity is complete.** diff --git a/src/AVEVA.Historian.Client/Grpc/HistorianGrpcHandshake.cs b/src/AVEVA.Historian.Client/Grpc/HistorianGrpcHandshake.cs new file mode 100644 index 0000000..787adf0 --- /dev/null +++ b/src/AVEVA.Historian.Client/Grpc/HistorianGrpcHandshake.cs @@ -0,0 +1,94 @@ +using Google.Protobuf; +using Grpc.Core; +using AVEVA.Historian.Client.Wcf; +using GrpcHistory = ArchestrA.Grpc.Contract.History; +using GrpcStorage = ArchestrA.Grpc.Contract.Storage; + +namespace AVEVA.Historian.Client.Grpc; + +/// +/// Shared 2023 R2 gRPC authentication handshake. Opens an authenticated History session over an +/// existing and returns the transient client handle used by +/// the Retrieval/Status services. Extracted from so the +/// read, status, and (future) browse/metadata gRPC paths all drive the identical chain: +/// HistoryService.GetInterfaceVersion → StorageService.ValidateClientCredential (token loop) → +/// HistoryService.OpenConnection. The byte payloads (OpenConnection3 v6 request, NTLM token +/// framing) are the proven 2020 protocol and transfer unchanged inside protobuf bytes fields. +/// +/// See for the op-routing rationale (the Negotiate loop +/// belongs on StorageService.ValidateClientCredential, NOT HistoryService.ExchangeKey). +/// +internal static class HistorianGrpcHandshake +{ + /// + /// The handles produced by a successful OpenConnection. is the + /// transient uint session token used by StartQuery/GetSystemParameter and the other + /// uint-handle ops. is the storage-session GUID used (formatted + /// uppercase via ) by the string-handle ops + /// (GetTagInfosFromName, GetTagExtendedPropertiesFromName, ExecuteSqlCommand, ...). + /// + internal readonly record struct Session(uint ClientHandle, Guid StorageSessionId) + { + /// The storage GUID in the uppercase "D" form the native string-handle ops require. + public string StringHandle => StorageSessionId.ToString("D").ToUpperInvariant(); + } + + /// Convenience overload for callers that only need the uint client handle. + public static uint OpenAuthenticatedConnection( + HistorianGrpcConnection connection, + HistorianClientOptions options, + CancellationToken cancellationToken) + => OpenSession(connection, options, cancellationToken).ClientHandle; + + public static Session OpenSession( + HistorianGrpcConnection connection, + HistorianClientOptions options, + CancellationToken cancellationToken) + { + DateTime Deadline() => DateTime.UtcNow.Add(options.RequestTimeout); + + Guid contextKey = Guid.NewGuid(); + var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel); + + GrpcHistory.GetInterfaceVersionResponse historyVersion = historyClient.GetInterfaceVersion( + new GrpcHistory.GetInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken); + HistorianServerVersionGate.Validate(HistorianServiceInterface.History, historyVersion.UiVersion, options); + + var storageClient = new GrpcStorage.StorageService.StorageServiceClient(connection.Channel); + HistorianNativeHandshake.RunTokenRounds( + (handle, wrapped, _) => + { + GrpcStorage.ValidateClientCredentialResponse response = storageClient.ValidateClientCredential( + new GrpcStorage.ValidateClientCredentialRequest { Handle = handle, InBuff = ByteString.CopyFrom(wrapped) }, + connection.Metadata, + Deadline(), + cancellationToken); + byte[] serverOutput = response.OutBuff?.ToByteArray() ?? []; + byte[] error = response.Status?.BtError?.ToByteArray() ?? []; + bool success = response.Status?.BSuccess ?? false; + return new HistorianNativeHandshake.TokenExchangeResult(success, serverOutput, error); + }, + contextKey, + options, + cancellationToken); + + byte[] open2Request = HistorianNativeHandshake.BuildOpenConnection3Request( + options.Host, contextKey, HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode); + + GrpcHistory.OpenConnectionResponse open2 = historyClient.OpenConnection( + new GrpcHistory.OpenConnectionRequest { BtConnectionRequest = ByteString.CopyFrom(open2Request) }, + connection.Metadata, + Deadline(), + cancellationToken); + + byte[] open2Response = open2.BtConnectionResponse?.ToByteArray() ?? []; + if (!(open2.Status?.BSuccess ?? false)) + { + byte[] err = open2.Status?.BtError?.ToByteArray() ?? []; + throw new InvalidOperationException($"gRPC OpenConnection failed (errorLen={err.Length}, responseLen={open2Response.Length})."); + } + + (uint clientHandle, Guid storageSessionId) = HistorianNativeHandshake.ParseOpenConnectionResponse(open2Response); + return new Session(clientHandle, storageSessionId); + } +} diff --git a/src/AVEVA.Historian.Client/Grpc/HistorianGrpcProbe.cs b/src/AVEVA.Historian.Client/Grpc/HistorianGrpcProbe.cs new file mode 100644 index 0000000..1886987 --- /dev/null +++ b/src/AVEVA.Historian.Client/Grpc/HistorianGrpcProbe.cs @@ -0,0 +1,48 @@ +using Grpc.Core; +using GrpcHistory = ArchestrA.Grpc.Contract.History; +using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval; +using GrpcStatus = ArchestrA.Grpc.Contract.Status; + +namespace AVEVA.Historian.Client.Grpc; + +/// +/// 2023 R2 gRPC reachability probe (roadmap item R0.4). Mirrors +/// over the gRPC transport: it calls the unauthenticated GetInterfaceVersion RPC on the +/// History, Retrieval, and Status services and applies the same success criteria. No credentials +/// are required — these RPCs run before the SSPI/Negotiate token loop — so the probe works even +/// when authentication is unavailable. +/// +internal static class HistorianGrpcProbe +{ + public static async Task ProbeAsync(HistorianClientOptions options, CancellationToken cancellationToken) + { + ArgumentNullException.ThrowIfNull(options); + cancellationToken.ThrowIfCancellationRequested(); + + return await Task.Run(() => Probe(options, cancellationToken), cancellationToken).ConfigureAwait(false); + } + + private static bool Probe(HistorianClientOptions options, CancellationToken cancellationToken) + { + using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options); + DateTime deadline = DateTime.UtcNow.Add(options.ConnectTimeout > TimeSpan.Zero ? options.ConnectTimeout : TimeSpan.FromSeconds(5)); + + var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel); + GrpcHistory.GetInterfaceVersionResponse history = historyClient.GetInterfaceVersion( + new GrpcHistory.GetInterfaceVersionRequest(), connection.Metadata, deadline, cancellationToken); + + var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel); + GrpcRetrieval.GetRetrievalInterfaceVersionResponse retrieval = retrievalClient.GetRetrievalInterfaceVersion( + new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), connection.Metadata, deadline, cancellationToken); + + var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel); + GrpcStatus.GetStatusInterfaceVersionResponse status = statusClient.GetStatusInterfaceVersion( + new GrpcStatus.GetStatusInterfaceVersionRequest(), connection.Metadata, deadline, cancellationToken); + + return history.UiError == 0 + && history.UiVersion > 0 + && retrieval.UiError == 0 + && retrieval.UiVersion > 0 + && status.UiError == 0; + } +} diff --git a/src/AVEVA.Historian.Client/Grpc/HistorianGrpcReadOrchestrator.cs b/src/AVEVA.Historian.Client/Grpc/HistorianGrpcReadOrchestrator.cs index 72d2d5d..74d0bad 100644 --- a/src/AVEVA.Historian.Client/Grpc/HistorianGrpcReadOrchestrator.cs +++ b/src/AVEVA.Historian.Client/Grpc/HistorianGrpcReadOrchestrator.cs @@ -3,7 +3,6 @@ using Google.Protobuf; using Grpc.Core; using AVEVA.Historian.Client.Models; using AVEVA.Historian.Client.Wcf; -using GrpcHistory = ArchestrA.Grpc.Contract.History; using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval; namespace AVEVA.Historian.Client.Grpc; @@ -16,17 +15,23 @@ namespace AVEVA.Historian.Client.Grpc; /// /// Operation mapping (2020 WCF → 2023 R2 gRPC): /// Hist.GetInterfaceVersion → HistoryService.GetInterfaceVersion -/// Hist.ValidateClientCredential (loop) → HistoryService.ExchangeKey (loop) +/// Hist.ValidateClientCredential (loop) → StorageService.ValidateClientCredential (loop) /// Hist.OpenConnection2 → HistoryService.OpenConnection /// Retr.StartQuery2 → RetrievalService.StartQuery /// Retr.GetNextQueryResultBuffer2 (loop) → RetrievalService.GetNextQueryResultBuffer (loop) /// Retr.EndQuery2 → RetrievalService.EndQuery /// -/// NOTE: not yet live-verified against a 2023 R2 server. The auth handshake uses -/// HistoryService.ExchangeKey because the gRPC HistoryService dropped ValidateClientCredential -/// (it now lives only on StorageService) and gained ExchangeKey with the identical -/// handle+token→token shape. If a live server rejects this, the handshake op is the first thing -/// to revisit — everything else is the proven 2020 byte protocol. +/// LIVE-VERIFIED 2026-06-21 against a real 2023 R2 server (interface versions: History=12, +/// Retrieval=4, Storage=4). The SSPI/Negotiate token loop maps to +/// StorageService.ValidateClientCredential(Handle, InBuff)→(status, OutBuff) — the op that +/// kept the 2020 inBuff/outBuff token framing. The gRPC HistoryService dropped +/// ValidateClientCredential and gained ExchangeKey, but ExchangeKey is a SEPARATE +/// key-exchange/cert-path op, NOT the Negotiate loop: feeding it an NTLM token is rejected at +/// round 0 regardless of credentials. An earlier revision wrongly routed the loop to ExchangeKey; +/// routing it to StorageService.ValidateClientCredential completes the full read chain. The byte +/// payloads (OpenConnection3 v6, token framing, DataQueryRequest, row buffers) are the proven 2020 +/// protocol and transfer unchanged — only the History interface integer differs (12 vs the 2020 +/// value 11), and that version is buffer-compatible (a live read returns rows). /// internal sealed class HistorianGrpcReadOrchestrator { @@ -159,50 +164,7 @@ internal sealed class HistorianGrpcReadOrchestrator } private uint OpenAuthenticatedConnection(HistorianGrpcConnection connection, CancellationToken cancellationToken) - { - Guid contextKey = Guid.NewGuid(); - var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel); - - GrpcHistory.GetInterfaceVersionResponse historyVersion = historyClient.GetInterfaceVersion( - new GrpcHistory.GetInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken); - HistorianServerVersionGate.Validate(HistorianServiceInterface.History, historyVersion.UiVersion, _options); - - HistorianNativeHandshake.RunTokenRounds( - (handle, wrapped, _) => - { - GrpcHistory.ExchangeKeyResponse response = historyClient.ExchangeKey( - new GrpcHistory.ExchangeKeyRequest { StrHandle = handle, BtInput = ByteString.CopyFrom(wrapped) }, - connection.Metadata, - Deadline(), - cancellationToken); - byte[] serverOutput = response.BtOutput?.ToByteArray() ?? []; - byte[] error = response.Status?.BtError?.ToByteArray() ?? []; - bool success = response.Status?.BSuccess ?? false; - return new HistorianNativeHandshake.TokenExchangeResult(success, serverOutput, error); - }, - contextKey, - _options, - cancellationToken); - - byte[] open2Request = HistorianNativeHandshake.BuildOpenConnection3Request( - _options.Host, contextKey, HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode); - - GrpcHistory.OpenConnectionResponse open2 = historyClient.OpenConnection( - new GrpcHistory.OpenConnectionRequest { BtConnectionRequest = ByteString.CopyFrom(open2Request) }, - connection.Metadata, - Deadline(), - cancellationToken); - - byte[] open2Response = open2.BtConnectionResponse?.ToByteArray() ?? []; - if (!(open2.Status?.BSuccess ?? false)) - { - byte[] err = open2.Status?.BtError?.ToByteArray() ?? []; - throw new InvalidOperationException($"gRPC OpenConnection failed (errorLen={err.Length}, responseLen={open2Response.Length})."); - } - - (uint clientHandle, _) = HistorianNativeHandshake.ParseOpenConnectionResponse(open2Response); - return clientHandle; - } + => HistorianGrpcHandshake.OpenAuthenticatedConnection(connection, _options, cancellationToken); private List RunQuery( HistorianGrpcConnection connection, diff --git a/src/AVEVA.Historian.Client/Grpc/HistorianGrpcStatusClient.cs b/src/AVEVA.Historian.Client/Grpc/HistorianGrpcStatusClient.cs new file mode 100644 index 0000000..c0538f1 --- /dev/null +++ b/src/AVEVA.Historian.Client/Grpc/HistorianGrpcStatusClient.cs @@ -0,0 +1,38 @@ +using Grpc.Core; +using GrpcStatus = ArchestrA.Grpc.Contract.Status; + +namespace AVEVA.Historian.Client.Grpc; + +/// +/// 2023 R2 gRPC status client (roadmap item R0.3). Mirrors +/// over the gRPC transport: it opens an authenticated +/// History session via and queries the StatusService for the +/// resulting client handle. GetSystemParameter carries the parameter name as a protobuf +/// string and returns the value string directly — there is no opaque native buffer to decode. +/// +internal static class HistorianGrpcStatusClient +{ + public static Task GetSystemParameterAsync( + HistorianClientOptions options, + string parameterName, + CancellationToken cancellationToken) + { + ArgumentException.ThrowIfNullOrWhiteSpace(parameterName); + return Task.Run(() => GetSystemParameter(options, parameterName, cancellationToken), cancellationToken); + } + + private static string? GetSystemParameter(HistorianClientOptions options, string parameterName, CancellationToken cancellationToken) + { + using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options); + uint clientHandle = HistorianGrpcHandshake.OpenAuthenticatedConnection(connection, options, cancellationToken); + + var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel); + GrpcStatus.GetSystemParameterResponse response = statusClient.GetSystemParameter( + new GrpcStatus.GetSystemParameterRequest { UiHandle = clientHandle, StrParameterName = parameterName }, + connection.Metadata, + DateTime.UtcNow.Add(options.RequestTimeout), + cancellationToken); + + return (response.Status?.BSuccess ?? false) ? response.StrParameterValue : null; + } +} diff --git a/src/AVEVA.Historian.Client/Grpc/HistorianGrpcTagClient.cs b/src/AVEVA.Historian.Client/Grpc/HistorianGrpcTagClient.cs new file mode 100644 index 0000000..7600d80 --- /dev/null +++ b/src/AVEVA.Historian.Client/Grpc/HistorianGrpcTagClient.cs @@ -0,0 +1,283 @@ +using System.Text; +using Google.Protobuf; +using Grpc.Core; +using AVEVA.Historian.Client.Models; +using AVEVA.Historian.Client.Wcf; +using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval; + +namespace AVEVA.Historian.Client.Grpc; + +/// +/// 2023 R2 gRPC tag-metadata + browse client (roadmap items R0.2 metadata, R0.1 browse). +/// Browse drives StartTagQuery (OData filter) → paged QueryTag → EndTagQuery +/// (see and docs/reverse-engineering/grpc-tag-query-odata.md). +/// Unlike the WCF singular +/// GetTagInfoFromName (a uint-handle op), the gRPC front door exposes the plural +/// RetrievalService.GetTagInfosFromName — a string-handle op keyed off the Open2 +/// storage-session GUID (uppercase). The request btTagNames buffer and response +/// btTagInfos buffer carry the proven native encodings: +/// +/// request btTagNames = uint count + per-name(uint charCount + UTF-16LE) +/// response btTagInfos = uint tagCount + per-tag CTagMetadata record +/// (the same record decodes) +/// +/// The string-handle "wall" that blocks this op family on the 2020 WCF transport does not apply on +/// the gRPC front door (different envelope/registration) — see +/// docs/reverse-engineering/wcf-string-handle-wall.md. +/// +internal static class HistorianGrpcTagClient +{ + public static Task GetTagMetadataAsync( + HistorianClientOptions options, + string tag, + CancellationToken cancellationToken) + { + ArgumentException.ThrowIfNullOrWhiteSpace(tag); + return Task.Run(() => GetTagMetadata(options, tag, cancellationToken), cancellationToken); + } + + private static HistorianTagMetadata? GetTagMetadata(HistorianClientOptions options, string tag, CancellationToken cancellationToken) + { + byte[] tagInfos = GetTagInfosRaw(options, [tag], cancellationToken); + if (tagInfos.Length < 4) + { + return null; + } + + IReadOnlyList parsed = HistorianTagQueryProtocol.ParseGetTagInfoResponse(tagInfos); + if (parsed.Count == 0) + { + return null; + } + + HistorianTagInfoResponse info = parsed[0]; + return new HistorianTagMetadata( + Name: info.TagName, + Key: info.TagKey, + DataType: HistorianWcfTagClient.MapDataType(info.NativeDataTypeDescriptor), + Description: info.Description ?? info.MetadataProvider, + EngineeringUnit: info.EngineeringUnit ?? string.Empty, + MinRaw: info.MinEU, + MaxRaw: info.MaxEU); + } + + /// + /// Issues a single GetTagInfosFromName call and returns the raw native btTagInfos + /// response buffer. Internal so reverse-engineering probes can capture the framing. + /// + internal static byte[] GetTagInfosRaw(HistorianClientOptions options, IReadOnlyList tags, CancellationToken cancellationToken) + { + using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options); + HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken); + + var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel); + byte[] requestBuffer = BuildTagNamesBuffer(tags); + GrpcRetrieval.GetTagInfosFromNameResponse response = retrievalClient.GetTagInfosFromName( + new GrpcRetrieval.GetTagInfosFromNameRequest + { + StrHandle = session.StringHandle, + BtTagNames = ByteString.CopyFrom(requestBuffer), + UiSequence = 0 + }, + connection.Metadata, + DateTime.UtcNow.Add(options.RequestTimeout), + cancellationToken); + + if (!(response.Status?.BSuccess ?? false)) + { + byte[] error = response.Status?.BtError?.ToByteArray() ?? []; + throw new InvalidOperationException($"gRPC GetTagInfosFromName failed (errorLen={error.Length})."); + } + + return response.BtTagInfos?.ToByteArray() ?? []; + } + + // QueryTag (browse paging) request framing, recovered from the .rdata packet-descriptor table + // in aahClientManaged.dll (entries {0x6751,1}=StartTagQuery, {0x6752,1}=QueryTag) and confirmed + // live: btRequest = u16 marker(0x6752) + u16 version(1) + u16 queryType + u32 startIndex + u32 count. + private const ushort QueryTagPacketMarker = 0x6752; + private const ushort TagQueryHeaderVersion = 1; + private const ushort QueryTagModeNames = 1; // queryType 1 returns tag-name rows + private const uint BrowsePageSize = 1000; + + /// + /// Browses tag names over gRPC (roadmap item R0.1). Drives + /// StartTagQuery (OData filter) → paged QueryTag → EndTagQuery on the + /// RetrievalService. The 2023 R2 metadata-server parses the filter as OData, so the SDK's + /// glob filter is translated via . Each QueryTag page returns + /// uint count + per-name(uint charCount + UTF-16LE), decoded by + /// . + /// + public static async IAsyncEnumerable BrowseTagNamesAsync( + HistorianClientOptions options, + string filter, + [System.Runtime.CompilerServices.EnumeratorCancellation] CancellationToken cancellationToken) + { + IReadOnlyList names = await Task.Run(() => BrowseTagNames(options, filter, cancellationToken), cancellationToken).ConfigureAwait(false); + foreach (string name in names) + { + cancellationToken.ThrowIfCancellationRequested(); + yield return name; + } + } + + private static List BrowseTagNames(HistorianClientOptions options, string filter, CancellationToken cancellationToken) + { + using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options); + HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken); + var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel); + DateTime Deadline() => DateTime.UtcNow.Add(options.RequestTimeout); + + byte[] startRequest = HistorianTagQueryProtocol.CreateStartTagQueryAttempt(GlobToODataFilter(filter)).RequestBuffer; + GrpcRetrieval.StartTagQueryResponse start = retrievalClient.StartTagQuery( + new GrpcRetrieval.StartTagQueryRequest { StrHandle = session.StringHandle, BtRequest = ByteString.CopyFrom(startRequest) }, + connection.Metadata, Deadline(), cancellationToken); + if (!(start.Status?.BSuccess ?? false)) + { + byte[] error = start.Status?.BtError?.ToByteArray() ?? []; + throw new InvalidOperationException($"gRPC StartTagQuery failed (errorLen={error.Length})."); + } + + HistorianTagQueryStartResponse parsed = HistorianTagQueryProtocol.ParseStartTagQueryResponse(start.BtResponse?.ToByteArray() ?? []); + List names = new(checked((int)parsed.TagCount)); + try + { + uint startIndex = 0; + while (names.Count < parsed.TagCount) + { + cancellationToken.ThrowIfCancellationRequested(); + uint page = Math.Min(BrowsePageSize, parsed.TagCount - (uint)names.Count); + GrpcRetrieval.QueryTagResponse query = retrievalClient.QueryTag( + new GrpcRetrieval.QueryTagRequest + { + StrHandle = session.StringHandle, + UiQueryHandle = parsed.QueryHandle, + BtRequest = ByteString.CopyFrom(BuildQueryTagRequest(QueryTagModeNames, startIndex, page)) + }, + connection.Metadata, Deadline(), cancellationToken); + if (!(query.Status?.BSuccess ?? false)) + { + byte[] error = query.Status?.BtError?.ToByteArray() ?? []; + throw new InvalidOperationException($"gRPC QueryTag failed (errorLen={error.Length})."); + } + + IReadOnlyList pageNames = HistorianTagQueryProtocol.ParseTagNameQueryPage(query.BtResonse?.ToByteArray() ?? []); + if (pageNames.Count == 0) + { + break; + } + + names.AddRange(pageNames); + startIndex += (uint)pageNames.Count; + } + } + finally + { + try + { + retrievalClient.EndTagQuery( + new GrpcRetrieval.EndTagQueryRequest { StrHandle = session.StringHandle, UiQueryHandle = parsed.QueryHandle }, + connection.Metadata, Deadline(), CancellationToken.None); + } + catch { /* best-effort cleanup */ } + } + + return names; + } + + /// Builds the QueryTag paging request: u16 marker(0x6752) + u16 version + u16 queryType + u32 startIndex + u32 count. + internal static byte[] BuildQueryTagRequest(ushort queryType, uint startIndex, uint count) + { + using MemoryStream stream = new(); + using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true); + writer.Write(QueryTagPacketMarker); + writer.Write(TagQueryHeaderVersion); + writer.Write(queryType); + writer.Write(startIndex); + writer.Write(count); + return stream.ToArray(); + } + + /// + /// Translates the SDK's glob filter (* wildcard) into the OData filter the 2023 R2 + /// metadata-server's StartActiveTagnamesQuery expects. Single-quotes are OData-escaped. + /// + /// * / empty → no filter (all tags) + /// Pre* → startswith(TagName,'Pre') + /// *suf → endswith(TagName,'suf') + /// *mid* → contains(TagName,'mid') + /// a*b → startswith(TagName,'a') and endswith(TagName,'b') + /// Exact → TagName eq 'Exact' + /// + /// + internal static string GlobToODataFilter(string filter) + { + if (string.IsNullOrEmpty(filter) || filter == "*") + { + return string.Empty; + } + + static string Esc(string s) => s.Replace("'", "''"); + + bool starStart = filter.StartsWith('*'); + bool starEnd = filter.EndsWith('*'); + string core = filter.Trim('*'); + if (core.Length == 0) + { + return string.Empty; // "**" etc. + } + + if (filter.IndexOf('*') < 0) + { + return $"TagName eq '{Esc(filter)}'"; + } + + if (starStart && starEnd && !core.Contains('*')) + { + return $"contains(TagName,'{Esc(core)}')"; + } + + if (starEnd && !core.Contains('*') && !starStart) + { + return $"startswith(TagName,'{Esc(core)}')"; + } + + if (starStart && !core.Contains('*') && !starEnd) + { + return $"endswith(TagName,'{Esc(core)}')"; + } + + // Internal wildcard(s): anchor on the prefix before the first '*' and the suffix after the last. + string prefix = filter[..filter.IndexOf('*')]; + string suffix = filter[(filter.LastIndexOf('*') + 1)..]; + List parts = []; + if (prefix.Length > 0) + { + parts.Add($"startswith(TagName,'{Esc(prefix)}')"); + } + if (suffix.Length > 0) + { + parts.Add($"endswith(TagName,'{Esc(suffix)}')"); + } + return parts.Count > 0 ? string.Join(" and ", parts) : string.Empty; + } + + /// Builds the native tag-names request buffer: uint count + per-name(uint charCount + UTF-16LE). + internal static byte[] BuildTagNamesBuffer(IReadOnlyList tags) + { + using MemoryStream stream = new(); + using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true); + + writer.Write((uint)tags.Count); + foreach (string tag in tags) + { + writer.Write((uint)tag.Length); + if (tag.Length > 0) + { + writer.Write(Encoding.Unicode.GetBytes(tag)); + } + } + + return stream.ToArray(); + } +} diff --git a/src/AVEVA.Historian.Client/HistorianClient.cs b/src/AVEVA.Historian.Client/HistorianClient.cs index 97781d9..b3bc40f 100644 --- a/src/AVEVA.Historian.Client/HistorianClient.cs +++ b/src/AVEVA.Historian.Client/HistorianClient.cs @@ -25,7 +25,9 @@ public sealed class HistorianClient : IAsyncDisposable public async Task ProbeAsync(CancellationToken cancellationToken = default) { - return await HistorianWcfProbe.ProbeAsync(_options, cancellationToken).ConfigureAwait(false); + return _options.Transport == HistorianTransport.RemoteGrpc + ? await Grpc.HistorianGrpcProbe.ProbeAsync(_options, cancellationToken).ConfigureAwait(false) + : await HistorianWcfProbe.ProbeAsync(_options, cancellationToken).ConfigureAwait(false); } public IAsyncEnumerable ReadRawAsync( @@ -129,13 +131,17 @@ public sealed class HistorianClient : IAsyncDisposable public IAsyncEnumerable BrowseTagNamesAsync(string filter = "*", CancellationToken cancellationToken = default) { ArgumentException.ThrowIfNullOrWhiteSpace(filter); - return HistorianWcfTagClient.BrowseTagNamesAsync(_options, filter, cancellationToken); + return _options.Transport == HistorianTransport.RemoteGrpc + ? Grpc.HistorianGrpcTagClient.BrowseTagNamesAsync(_options, filter, cancellationToken) + : HistorianWcfTagClient.BrowseTagNamesAsync(_options, filter, cancellationToken); } public Task GetTagMetadataAsync(string tag, CancellationToken cancellationToken = default) { ArgumentException.ThrowIfNullOrWhiteSpace(tag); - return HistorianWcfTagClient.GetTagMetadataAsync(_options, tag, cancellationToken); + return _options.Transport == HistorianTransport.RemoteGrpc + ? Grpc.HistorianGrpcTagClient.GetTagMetadataAsync(_options, tag, cancellationToken) + : HistorianWcfTagClient.GetTagMetadataAsync(_options, tag, cancellationToken); } public Task GetConnectionStatusAsync(CancellationToken cancellationToken = default) diff --git a/src/AVEVA.Historian.Client/HistorianServerVersionGate.cs b/src/AVEVA.Historian.Client/HistorianServerVersionGate.cs index aa05f3c..ba9f984 100644 --- a/src/AVEVA.Historian.Client/HistorianServerVersionGate.cs +++ b/src/AVEVA.Historian.Client/HistorianServerVersionGate.cs @@ -43,6 +43,15 @@ internal static class HistorianServerVersionGate public const uint RetrievalInterfaceVersion = 4; public const uint TransactionInterfaceVersion = 2; + /// + /// The 2023 R2 gRPC HistoryService reports interface version 12. It is buffer-compatible with + /// the 2020 version 11 — the OpenConnection3 v6 / token / DataQueryRequest / row buffers are + /// byte-identical — confirmed by a live end-to-end gRPC read against a real 2023 R2 server + /// (2026-06-21). So both 11 and 12 are accepted for History. (Retrieval reported 4, matching + /// the 2020 value, so it needs no widening.) + /// + public const uint HistoryInterfaceVersionGrpc2023R2 = 12; + /// /// True when the service interface reports a meaningful version that should be matched. /// Status is reachability-only (its GetInterfaceVersion returns 0). @@ -56,7 +65,7 @@ internal static class HistorianServerVersionGate _ => false }; - /// The interface version this SDK's serializers target for a value-gated service. + /// The canonical interface version this SDK's serializers target for a value-gated service. public static uint ExpectedVersion(HistorianServiceInterface service) => service switch { HistorianServiceInterface.History => HistoryInterfaceVersion, @@ -65,6 +74,18 @@ internal static class HistorianServerVersionGate _ => throw new ArgumentOutOfRangeException(nameof(service), service, "Service interface is not value-gated.") }; + /// + /// All interface versions accepted for a value-gated service. Usually a single value, but + /// History accepts both the 2020 value (11) and the buffer-compatible 2023 R2 gRPC value (12). + /// + public static uint[] AcceptedVersions(HistorianServiceInterface service) => service switch + { + HistorianServiceInterface.History => [HistoryInterfaceVersion, HistoryInterfaceVersionGrpc2023R2], + HistorianServiceInterface.Retrieval => [RetrievalInterfaceVersion], + HistorianServiceInterface.Transaction => [TransactionInterfaceVersion], + _ => throw new ArgumentOutOfRangeException(nameof(service), service, "Service interface is not value-gated.") + }; + /// /// Throws when version verification is enabled /// and the server's reported interface version differs from the version this SDK targets. @@ -80,14 +101,15 @@ internal static class HistorianServerVersionGate return; } - uint expected = ExpectedVersion(service); - if (reportedVersion == expected) + uint[] accepted = AcceptedVersions(service); + if (Array.IndexOf(accepted, reportedVersion) >= 0) { return; } + string acceptedList = string.Join(", ", accepted); throw new ProtocolEvidenceMissingException( - $"{service} interface version {reportedVersion} (this SDK's serializers target version {expected}); " + + $"{service} interface version {reportedVersion} (this SDK's serializers target version {acceptedList}); " + $"set {nameof(HistorianClientOptions)}.{nameof(HistorianClientOptions.VerifyServerInterfaceVersion)}=false to bypass at your own risk"); } } diff --git a/src/AVEVA.Historian.Client/Protocol/Historian2020ProtocolDialect.cs b/src/AVEVA.Historian.Client/Protocol/Historian2020ProtocolDialect.cs index d3ae8af..079a2ff 100644 --- a/src/AVEVA.Historian.Client/Protocol/Historian2020ProtocolDialect.cs +++ b/src/AVEVA.Historian.Client/Protocol/Historian2020ProtocolDialect.cs @@ -64,7 +64,9 @@ internal sealed class Historian2020ProtocolDialect { cancellationToken.ThrowIfCancellationRequested(); ArgumentException.ThrowIfNullOrWhiteSpace(name); - return Wcf.HistorianWcfStatusClient.GetSystemParameterAsync(_options, name, cancellationToken); + return UseGrpc + ? HistorianGrpcStatusClient.GetSystemParameterAsync(_options, name, cancellationToken) + : Wcf.HistorianWcfStatusClient.GetSystemParameterAsync(_options, name, cancellationToken); } public Task GetRuntimeParameterAsync(string name, CancellationToken cancellationToken) diff --git a/src/AVEVA.Historian.Client/Wcf/HistorianNativeHandshake.cs b/src/AVEVA.Historian.Client/Wcf/HistorianNativeHandshake.cs index e4760a1..fcec618 100644 --- a/src/AVEVA.Historian.Client/Wcf/HistorianNativeHandshake.cs +++ b/src/AVEVA.Historian.Client/Wcf/HistorianNativeHandshake.cs @@ -33,7 +33,7 @@ internal static class HistorianNativeHandshake /// upper-case context-key GUID, is the AVEVA-wrapped SSPI /// token (round byte + length + token). The WCF path maps this to /// Hist.ValidateClientCredential; the gRPC path maps it to - /// HistoryService.ExchangeKey (the renamed handshake op). + /// StorageService.ValidateClientCredential (the op that kept the 2020 token framing). /// internal delegate TokenExchangeResult TokenExchange(string handle, byte[] wrappedToken, int round); @@ -70,7 +70,8 @@ internal static class HistorianNativeHandshake if (!result.Success) { - throw new InvalidOperationException($"Credential token round {round} rejected (errorLen={error.Length})."); + throw new InvalidOperationException( + $"Credential token round {round} rejected (errorLen={error.Length}).{DescribeError(error)}"); } ValidateClientCredentialResponse? response = HistorianWcfAuthenticationProtocol.TryReadValidateClientCredentialResponse(serverOutput); @@ -162,4 +163,32 @@ internal static class HistorianNativeHandshake int slash = userName.IndexOf('\\'); return slash > 0 ? userName[(slash + 1)..] : userName; } + + /// + /// Renders a diagnostic suffix for a rejected credential round: the decoded native error + /// (type/code/name) plus a short hex + printable-ASCII preview of the server error buffer. + /// Keeps secrets out — error buffers carry server status codes/messages, not credentials. + /// + private static string DescribeError(byte[] error) + { + if (error.Length == 0) + { + return string.Empty; + } + + HistorianNativeError? native = HistorianOpen2Protocol.TryReadNativeError(error); + string nativePart = native is null + ? string.Empty + : $" native(type={native.Type}, code={native.Code}{(native.Name is null ? string.Empty : $", {native.Name}")})"; + + ReadOnlySpan preview = error.AsSpan(0, Math.Min(error.Length, 64)); + string hex = Convert.ToHexString(preview); + char[] ascii = new char[preview.Length]; + for (int i = 0; i < preview.Length; i++) + { + ascii[i] = preview[i] is >= 0x20 and < 0x7F ? (char)preview[i] : '.'; + } + + return $"{nativePart} hex={hex} ascii=\"{new string(ascii)}\""; + } } diff --git a/src/AVEVA.Historian.Client/Wcf/HistorianTagQueryProtocol.cs b/src/AVEVA.Historian.Client/Wcf/HistorianTagQueryProtocol.cs index d14a362..d118c33 100644 --- a/src/AVEVA.Historian.Client/Wcf/HistorianTagQueryProtocol.cs +++ b/src/AVEVA.Historian.Client/Wcf/HistorianTagQueryProtocol.cs @@ -102,6 +102,34 @@ internal static class HistorianTagQueryProtocol return tagNames; } + /// + /// Parses one page of a gRPC QueryTag tag-name response: uint count + per-name(uint + /// charCount + UTF-16LE), then a trailing region (NextIndex + optional metadata buffer) that + /// is intentionally ignored. Unlike this tolerates the + /// trailer rather than requiring the buffer to end exactly after the names. + /// + public static IReadOnlyList ParseTagNameQueryPage(ReadOnlySpan response) + { + if (response.Length < 4) + { + return []; + } + + int cursor = 0; + uint count = ReadUInt32(response, ref cursor); + List tagNames = new(checked((int)count)); + for (uint index = 0; index < count; index++) + { + uint charLength = ReadUInt32(response, ref cursor); + int byteLength = checked((int)charLength * 2); + EnsureAvailable(response, cursor, byteLength); + tagNames.Add(Encoding.Unicode.GetString(response.Slice(cursor, byteLength))); + cursor += byteLength; + } + + return tagNames; + } + private static void WriteHistorianString(BinaryWriter writer, string value) { writer.Write((uint)value.Length); diff --git a/tests/AVEVA.Historian.Client.Tests/HistorianGrpcHandshakeRoutingTests.cs b/tests/AVEVA.Historian.Client.Tests/HistorianGrpcHandshakeRoutingTests.cs new file mode 100644 index 0000000..fee9f3f --- /dev/null +++ b/tests/AVEVA.Historian.Client.Tests/HistorianGrpcHandshakeRoutingTests.cs @@ -0,0 +1,155 @@ +using System.Reflection; +using System.Reflection.Emit; + +namespace AVEVA.Historian.Client.Tests; + +/// +/// Structural guardrail pinning the 2023 R2 gRPC auth-handshake op routing. The SSPI/Negotiate +/// token loop MUST be carried by StorageService.ValidateClientCredential (the op that kept +/// the 2020 inBuff/outBuff token framing), NOT by HistoryService.ExchangeKey — ExchangeKey +/// is a separate key-exchange/cert-path op that rejects an NTLM token at round 0 regardless of +/// credentials (live-confirmed against a real 2023 R2 server, 2026-06-21). An earlier revision +/// routed the loop to ExchangeKey; this test fails if that regression returns. +/// +/// It works by disassembling the IL of HistorianGrpcHandshake (and its +/// compiler-generated nested closure types — the token-loop call lives inside a lambda) and +/// collecting every method invoked. +/// +public sealed class HistorianGrpcHandshakeRoutingTests +{ + [Fact] + public void Handshake_UsesValidateClientCredential_NotExchangeKey() + { + // The auth token loop lives in the shared handshake helper (reused by the read, status, + // and future browse/metadata gRPC paths). + HashSet calledMethods = CollectCalledMethodNames( + "AVEVA.Historian.Client.Grpc.HistorianGrpcHandshake"); + + Assert.Contains("ValidateClientCredential", calledMethods); + Assert.DoesNotContain("ExchangeKey", calledMethods); + } + + private static HashSet CollectCalledMethodNames(string typeFullName) + { + Assembly sdk = typeof(HistorianClientOptions).Assembly; + Type orchestrator = sdk.GetType(typeFullName, throwOnError: true)!; + Module module = orchestrator.Module; + + // The orchestrator type plus its compiler-generated nested types (lambda closures). + IEnumerable types = new[] { orchestrator } + .Concat(orchestrator.GetNestedTypes(BindingFlags.Public | BindingFlags.NonPublic)); + + var names = new HashSet(StringComparer.Ordinal); + foreach (Type t in types) + { + Type[] typeArgs = t.IsGenericType ? t.GetGenericArguments() : Type.EmptyTypes; + foreach (MethodInfo m in t.GetMethods( + BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Static | BindingFlags.DeclaredOnly)) + { + byte[]? il = m.GetMethodBody()?.GetILAsByteArray(); + if (il is null) + { + continue; + } + + Type[] methodArgs = m.IsGenericMethodDefinition ? m.GetGenericArguments() : Type.EmptyTypes; + foreach (int token in EnumerateMethodTokens(il)) + { + try + { + MethodBase? resolved = module.ResolveMethod(token, typeArgs, methodArgs); + if (resolved is not null) + { + names.Add(resolved.Name); + } + } + catch (Exception ex) when (ex is ArgumentException or BadImageFormatException) + { + // vararg / MethodSpec tokens that don't resolve cleanly — irrelevant here. + } + } + } + } + + return names; + } + + /// + /// Walks an IL byte stream, yielding the 4-byte metadata token of every call/callvirt/newobj + /// (any opcode). Uses the reflection-emit opcode table so + /// operands of other instructions are skipped correctly rather than misread as opcodes. + /// + private static IEnumerable EnumerateMethodTokens(byte[] il) + { + int pos = 0; + while (pos < il.Length) + { + OpCode op; + if (il[pos] == 0xFE && pos + 1 < il.Length) + { + op = TwoByteOpCodes[il[pos + 1]]; + pos += 2; + } + else + { + op = OneByteOpCodes[il[pos]]; + pos += 1; + } + + switch (op.OperandType) + { + case OperandType.InlineMethod: + yield return BitConverter.ToInt32(il, pos); + pos += 4; + break; + case OperandType.InlineNone: + break; + case OperandType.ShortInlineBrTarget: + case OperandType.ShortInlineI: + case OperandType.ShortInlineVar: + pos += 1; + break; + case OperandType.InlineVar: + pos += 2; + break; + case OperandType.InlineI8: + case OperandType.InlineR: + pos += 8; + break; + case OperandType.InlineSwitch: + int count = BitConverter.ToInt32(il, pos); + pos += 4 + (4 * count); + break; + default: + // InlineBrTarget, InlineField, InlineI, InlineSig, InlineString, InlineTok, + // InlineType, ShortInlineR — all 4-byte operands. + pos += 4; + break; + } + } + } + + private static readonly OpCode[] OneByteOpCodes = BuildOpCodeTable(twoByte: false); + private static readonly OpCode[] TwoByteOpCodes = BuildOpCodeTable(twoByte: true); + + private static OpCode[] BuildOpCodeTable(bool twoByte) + { + var table = new OpCode[256]; + foreach (FieldInfo f in typeof(OpCodes).GetFields(BindingFlags.Public | BindingFlags.Static)) + { + if (f.GetValue(null) is not OpCode op) + { + continue; + } + + ushort value = unchecked((ushort)op.Value); + bool isTwoByte = (value & 0xFF00) == 0xFE00; + if (isTwoByte == twoByte) + { + table[value & 0xFF] = op; + } + } + + return table; + } +} diff --git a/tests/AVEVA.Historian.Client.Tests/HistorianGrpcIntegrationTests.cs b/tests/AVEVA.Historian.Client.Tests/HistorianGrpcIntegrationTests.cs index dcd71d6..e932476 100644 --- a/tests/AVEVA.Historian.Client.Tests/HistorianGrpcIntegrationTests.cs +++ b/tests/AVEVA.Historian.Client.Tests/HistorianGrpcIntegrationTests.cs @@ -12,6 +12,21 @@ namespace AVEVA.Historian.Client.Tests; /// public sealed class HistorianGrpcIntegrationTests { + [Fact] + public async Task ProbeAsync_OverGrpc_ReturnsTrue() + { + string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST"); + if (string.IsNullOrWhiteSpace(host)) + { + return; + } + + // ProbeAsync calls the unauthenticated GetInterfaceVersion RPCs, so it succeeds even when + // credentials are unavailable — no HISTORIAN_USER/PASSWORD required. + HistorianClient client = new(BuildOptions(host)); + Assert.True(await client.ProbeAsync(CancellationToken.None)); + } + [Fact] public async Task ReadRawAsync_OverGrpc_ReturnsAtLeastOneRow() { @@ -37,6 +52,61 @@ public sealed class HistorianGrpcIntegrationTests Assert.All(samples, s => Assert.Equal(testTag, s.TagName)); } + [Fact] + public async Task GetSystemParameterAsync_OverGrpc_ReturnsValue() + { + string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST"); + if (string.IsNullOrWhiteSpace(host) || string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER"))) + { + return; + } + + HistorianClient client = new(BuildOptions(host)); + string? version = await client.GetSystemParameterAsync("HistorianVersion", CancellationToken.None); + Assert.False(string.IsNullOrWhiteSpace(version)); + } + + [Fact] + public async Task GetTagMetadataAsync_OverGrpc_ReturnsRequestedTag() + { + string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST"); + string? tag = Environment.GetEnvironmentVariable("HISTORIAN_TEST_TAG"); + if (string.IsNullOrWhiteSpace(host) || string.IsNullOrWhiteSpace(tag) + || string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER"))) + { + return; + } + + HistorianClient client = new(BuildOptions(host)); + HistorianTagMetadata? metadata = await client.GetTagMetadataAsync(tag, CancellationToken.None); + + Assert.NotNull(metadata); + Assert.Equal(tag, metadata!.Name); + // A real metadata record decodes to a known data type (descriptor passed MapDataType). + Assert.True(Enum.IsDefined(metadata.DataType)); + } + + [Fact] + public async Task BrowseTagNamesAsync_OverGrpc_ReturnsSystemTags() + { + string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST"); + if (string.IsNullOrWhiteSpace(host) || string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER"))) + { + return; + } + + // Full R0.1 browse over gRPC: StartTagQuery(OData) -> paged QueryTag(0x6752) -> EndTagQuery. + HistorianClient client = new(BuildOptions(host)); + List names = []; + await foreach (string name in client.BrowseTagNamesAsync("Sys*", CancellationToken.None)) + { + names.Add(name); + } + + Assert.NotEmpty(names); + Assert.All(names, n => Assert.StartsWith("Sys", n, StringComparison.Ordinal)); + } + private static HistorianClientOptions BuildOptions(string host) { string? user = Environment.GetEnvironmentVariable("HISTORIAN_USER"); diff --git a/tests/AVEVA.Historian.Client.Tests/HistorianGrpcTransportTests.cs b/tests/AVEVA.Historian.Client.Tests/HistorianGrpcTransportTests.cs index a67de94..f50a0d8 100644 --- a/tests/AVEVA.Historian.Client.Tests/HistorianGrpcTransportTests.cs +++ b/tests/AVEVA.Historian.Client.Tests/HistorianGrpcTransportTests.cs @@ -100,6 +100,88 @@ public sealed class HistorianGrpcTransportTests Assert.Equal((uint)HistorianDataQueryProtocol.QueryRequestTypeData, decoded.UiQueryRequestType); } + [Fact] + public void InterfaceVersionResponses_ExposeErrorAndVersion_AsProbeExpects() + { + // R0.4 ProbeAsync reads uiError/uiVersion off each service's GetInterfaceVersion response. + // Pin that field mapping (success = uiError 0 + uiVersion > 0) via a protobuf round-trip. + var history = GrpcHistory.GetInterfaceVersionResponse.Parser.ParseFrom( + new GrpcHistory.GetInterfaceVersionResponse { UiError = 0, UiVersion = 12 }.ToByteArray()); + var retrieval = GetRetrievalInterfaceVersionResponse.Parser.ParseFrom( + new GetRetrievalInterfaceVersionResponse { UiError = 0, UiVersion = 4 }.ToByteArray()); + + Assert.Equal(0u, history.UiError); + Assert.Equal(12u, history.UiVersion); + Assert.Equal(0u, retrieval.UiError); + Assert.Equal(4u, retrieval.UiVersion); + } + + [Fact] + public void GetSystemParameterMessages_CarryHandleNameAndValue_AsStatusClientExpects() + { + // R0.3 sends {uiHandle, strParameterName} and reads strParameterValue when status succeeds. + var request = ArchestrA.Grpc.Contract.Status.GetSystemParameterRequest.Parser.ParseFrom( + new ArchestrA.Grpc.Contract.Status.GetSystemParameterRequest { UiHandle = 9, StrParameterName = "HistorianVersion" }.ToByteArray()); + Assert.Equal(9u, request.UiHandle); + Assert.Equal("HistorianVersion", request.StrParameterName); + + var response = ArchestrA.Grpc.Contract.Status.GetSystemParameterResponse.Parser.ParseFrom( + new ArchestrA.Grpc.Contract.Status.GetSystemParameterResponse + { + Status = new ArchestrA.Grpc.Contract.RequestStatus.Status { BSuccess = true }, + StrParameterValue = "20.0.000" + }.ToByteArray()); + Assert.True(response.Status.BSuccess); + Assert.Equal("20.0.000", response.StrParameterValue); + } + + [Fact] + public void BuildTagNamesBuffer_EncodesCountThenLengthPrefixedUtf16Names() + { + // R0.2 request framing: uint count + per-name(uint charCount + UTF-16LE). Golden bytes. + byte[] buffer = AVEVA.Historian.Client.Grpc.HistorianGrpcTagClient.BuildTagNamesBuffer(["AB", "C"]); + + byte[] expected = + [ + 0x02, 0x00, 0x00, 0x00, // count = 2 + 0x02, 0x00, 0x00, 0x00, // "AB" char count = 2 + 0x41, 0x00, 0x42, 0x00, // 'A','B' UTF-16LE + 0x01, 0x00, 0x00, 0x00, // "C" char count = 1 + 0x43, 0x00 // 'C' UTF-16LE + ]; + Assert.Equal(expected, buffer); + } + + [Fact] + public void BuildQueryTagRequest_EncodesMarkerVersionTypeStartCount() + { + // R0.1 QueryTag paging request: u16 0x6752 + u16 1 + u16 queryType + u32 startIndex + u32 count. + byte[] buffer = AVEVA.Historian.Client.Grpc.HistorianGrpcTagClient.BuildQueryTagRequest(1, 0, 50); + byte[] expected = + [ + 0x52, 0x67, // marker 0x6752 + 0x01, 0x00, // version 1 + 0x01, 0x00, // queryType 1 (names) + 0x00, 0x00, 0x00, 0x00, // startIndex 0 + 0x32, 0x00, 0x00, 0x00 // count 50 + ]; + Assert.Equal(expected, buffer); + } + + [Theory] + [InlineData("*", "")] + [InlineData("", "")] + [InlineData("Sys*", "startswith(TagName,'Sys')")] + [InlineData("*Total", "endswith(TagName,'Total')")] + [InlineData("*Alarm*", "contains(TagName,'Alarm')")] + [InlineData("Exact.Tag", "TagName eq 'Exact.Tag'")] + [InlineData("Pre*Suf", "startswith(TagName,'Pre') and endswith(TagName,'Suf')")] + [InlineData("O'Brien*", "startswith(TagName,'O''Brien')")] + public void GlobToODataFilter_TranslatesWildcards(string glob, string expected) + { + Assert.Equal(expected, AVEVA.Historian.Client.Grpc.HistorianGrpcTagClient.GlobToODataFilter(glob)); + } + [Fact] public void OpenConnectionRequest_CarriesNativeOpen2BufferUnchanged() { diff --git a/tests/AVEVA.Historian.Client.Tests/HistorianServerVersionGateTests.cs b/tests/AVEVA.Historian.Client.Tests/HistorianServerVersionGateTests.cs index 9c3a98e..cbc7539 100644 --- a/tests/AVEVA.Historian.Client.Tests/HistorianServerVersionGateTests.cs +++ b/tests/AVEVA.Historian.Client.Tests/HistorianServerVersionGateTests.cs @@ -29,6 +29,20 @@ public sealed class HistorianServerVersionGateTests HistorianServerVersionGate.Validate(HistorianServiceInterface.Transaction, HistorianServerVersionGate.TransactionInterfaceVersion, Options()); } + [Fact] + public void Validate_History_AcceptsBoth2020And2023R2Versions() + { + // History 11 (2020 WCF) and 12 (2023 R2 gRPC) are both buffer-compatible — a live gRPC + // read against a real 2023 R2 server (interface version 12) returns rows. Both must pass. + HistorianServerVersionGate.Validate(HistorianServiceInterface.History, 11u, Options()); + HistorianServerVersionGate.Validate(HistorianServiceInterface.History, HistorianServerVersionGate.HistoryInterfaceVersionGrpc2023R2, Options()); + Assert.Equal(12u, HistorianServerVersionGate.HistoryInterfaceVersionGrpc2023R2); + Assert.Contains(11u, HistorianServerVersionGate.AcceptedVersions(HistorianServiceInterface.History)); + Assert.Contains(12u, HistorianServerVersionGate.AcceptedVersions(HistorianServiceInterface.History)); + // Retrieval reported 4 on the live 2023 R2 server — matches 2020, so it is NOT widened. + Assert.DoesNotContain(5u, HistorianServerVersionGate.AcceptedVersions(HistorianServiceInterface.Retrieval)); + } + [Fact] public void Validate_MismatchedVersion_ThrowsProtocolEvidenceMissing() { @@ -36,7 +50,7 @@ public sealed class HistorianServerVersionGateTests (HistorianServiceInterface Service, uint Version)[] cases = [ (HistorianServiceInterface.History, 10u), - (HistorianServiceInterface.History, 12u), + (HistorianServiceInterface.History, 13u), (HistorianServiceInterface.Retrieval, 3u), (HistorianServiceInterface.Retrieval, 5u), (HistorianServiceInterface.Transaction, 1u),