histsdk/docs/plans/grpc-transport.md

AVEVA Historian SDK 2023 R2 — gRPC Transport Analysis

Scope: Documents the new gRPC transport that 2023 R2 adds to the Historian Client Access Layer (HCAL). Kept deliberately separate from the main histsdk reverse-engineering docs — this is 2023 R2 evidence, not the 2020/WCF protocol the production SDK currently targets.

Source: the 2023 R2 HistorianSDK installer (not installed). SDKSetup.msi was laid out with msiexec /a (administrative extract, no registration) into a local msi-extract staging dir, then the managed assemblies were decompiled with ilspycmd.

Assembly versions analysed: 2023.1219.4004.5 (Archestra.Grpc.Contract.dll, Archestra.Historian.GrpcClient.dll, aahClientManaged.dll).

---

1. Headline finding

The 2023 R2 gRPC transport is a transport swap, not a protocol redesign. Every gRPC request/response wraps the same opaque native binary buffers that the 2020/WCF-MDAS path already carries — OpenConnection3 v6 buffer, NTLM/SSPI ValCl tokens, DataQueryRequest, GetNextQueryResultBuffer row buffers, the Status err blob, etc. — inside protobuf bytes fields.

Concrete proof, from Archestra.Historian.GrpcClient:

// History/OpenConnection — same byte[] openParameters the WCF path built
OpenConnectionRequest request = new OpenConnectionRequest {
    BtConnectionRequest = ByteString.CopyFrom(openParameters)
};
// Retrieval/StartQuery — same queryType + DataQueryRequest bytes + handle
StartQueryRequest request = new StartQueryRequest {
    UiHandle = handle,
    UiQueryRequestType = queryRequestType,
    BtRequestBuffer = ByteString.CopyFrom(requestBuffer)
};

Implication for the histsdk project: all of the hard-won payload serializers (HistorianOpen2Protocol, HistorianDataQueryProtocol, HistorianEventRowProtocol, the SSPI ValCl token framing, the EnsT2 CTagMetadata layout) transfer unchanged. Only the envelope around them changes: protobuf-over-gRPC instead of binary-SOAP-over-application/x-mdas. The WCF [MessageParameter(Name=…)] guessing that dominated the 2020 work is gone — field names and numbers are explicit in the protobuf contract.

---

2. Transport stack

From GrpcClientBase.InitializeBase(target, portNumber, securedConnection, certificateName, trusted):

Aspect	Value / behaviour
Library	Grpc.Net.Client + Grpc.Net.Client.Web (GrpcWebHandler)
Mode	gRPC-Web, GrpcWebMode.GrpcWeb (binary application/grpc-web, not -text)
HTTP version	HTTP/1.1 (GrpcWebHandler.HttpVersion = new Version(1,1)) — not HTTP/2
Address	http://{target}:{port} insecure, or https://{certificateName}:{port} secure
Inner handler	HttpClientHandler with custom ServerCertificateCustomValidationCallback
Compression	gzip on by default; request header grpc-internal-encoding-request: gzip; custom CustomCompressionProvider / CustomGZipStream used for bandwidth accounting
Default timeout	60 s per call (m_timeoutInSeconds = 60, sent as gRPC deadline)
Interceptor	ClientInterceptor (logging hook, currently a no-op LogCall)

Because it is gRPC-Web over HTTP/1.1, the transport is proxy/firewall friendly and does not require HTTP/2 negotiation — note HistorianConnectionArgs.ProxyServer (e.g. http://host:9480) in the public API.

Port

• Default port 32565 — HistorianConnectionArgs.TcpPort, "the TCP port of the Historian Client Access Point." (Note this differs from the 2020 WCF port 32568 the production SDK uses.)
• All services reach the same host:port; gRPC multiplexes by service path (/HistoryService/OpenConnection, /RetrievalService/StartQuery, …).

Channel topology

Five service stubs grouped into four wrapper clients, each constructing its own GrpcChannel to the same endpoint:

Wrapper (GrpcClientBase subclass)	gRPC service stub(s)
GrpcHistoryClient	HistoryService + TransactionService (one channel)
GrpcRetrievalClient	RetrievalService
GrpcStatusClient	StatusService
GrpcStorageClient	StorageService

---

3. Authentication model (unchanged in substance)

Auth is still the native session handshake, carried over gRPC instead of WCF. There is no per-call bearer/auth token in gRPC metadata — the only metadata sent is the gzip-encoding hint. Methods pass m_metadata (gzip) or null; neither carries credentials. The server keys the session off the handle GUID established by the handshake, exactly as the 2020 path does.

Handshake operations (same byte payloads as 2020):

• HistoryService.GetInterfaceVersion → version probe.
• StorageService.ValidateClientCredential { string Handle; bytes InBuff } → { Status; bytes OutBuff }. InBuff/OutBuff carry the NTLM/SSPI tokens — same multi-round Negotiate exchange, same field names the 2020 ildasm revealed (inBuff/outBuff), now first-class protobuf fields.
• HistoryService.ExchangeKey { string StrHandle; bytes BtInput } → { Status; bytes BtOutput } (key-exchange / cert path).
• HistoryService.OpenConnection { bytes BtConnectionRequest } → { Status; bytes BtConnectionResponse } — same OpenConnection3 v6 request buffer in, same 42-byte session blob out.

Public-API security knobs (aahClientManaged.xml):

• HistorianConnectionArgs.ConnectionMode — "whether GRPC connection to the Historian Server. Default is true (GRPC)." This is the master switch selecting gRPC vs legacy.
• HistorianSecurityMode: None, Disabled, TransportWindows (Windows creds), TransportCertificate (server cert).
• AllowUnTrustedConnection → maps to the trusted arg; when false the client bypasses X509 chain validation (ValidateServerCertificate returns true early). Equivalent to the production SDK's AllowUntrustedServerCertificate.
• AuthenticationMode default HistorianNative; CertificateInfo.CertificateName supplies the https://{certificateName}:{port} SNI/host identity.

---

4. gRPC service surface (full RPC list)

All methods are unary (MethodType 0). Names map 1:1 onto the 2020 WCF operations the production SDK already understands.

HistoryService (/HistoryService/…)

GetInterfaceVersion, ExchangeKey, OpenConnection, CloseConnection, UpdateClientStatus, RegisterTags, EnsureTags, AddStreamValues, AddTagExtendedPropertyGroups, AddTagExtendedProperties, StartJob, GetJobStatus, DeleteTagExtendedProperties, DeleteTags, AddTagLocalizedProperties, DeleteTagLocalizedProperties

RetrievalService (/RetrievalService/…)

GetRetrievalInterfaceVersion, StartQuery, GetNextQueryResultBuffer, EndQuery, GetShardTagidsByTagnameAndSource, GetTagInfosFromName, GetTagExtendedPropertiesFromName, ExecuteSqlCommand, StartEventQuery, GetNextEventQueryResultBuffer, EndEventQuery, StartTagQuery, QueryTag, EndTagQuery, GetTagLocalizedPropertiesFromName

StatusService (/StatusService/…)

GetStatusInterfaceVersion, GetSystemParameter, SendInfo, RequestInfo, DeleteInfo, GetHistorianInfo, StartProcess, StopProcess, PingServer, PingPipe, ConfigureAutoStartProcess, GetHistorianConsoleStatus, GetRuntimeParameter, GetSystemTimeZoneName, SetHistorianConsoleStatus, CanUpdateAreaHierarchy, UpdateAreaHierarchy, UpdateObjectHierarchy

StorageService (/StorageService/…)

GetInterfaceVersion, OpenStorageConnection, OpenStorageConnection2, CloseStorageConnection, Ping, AddTags, RegisterTags, AddStreamValues, AddStreamValues2, GetTagIds, GetTags, FlushMetadata, FlushData, LoadBlocks, GetSnapshots, StartQuerySnapshot, NextQuerySnapshot, EndSnapshot, Stop, ClearTagidPairs, AddTagidPairs, GetSFParameter, SetSFParameter, SendSnapshotBegin, SendSnapshotEnd, SendSnapshot, DeleteSnapshot, ClearShardTagids, AddShardTagids, SplitUnknownShards, GetRemainingSnapshotsSize, DeleteTags, OpenStorageConnection2, ValidateClientCredential, GetInfo

TransactionService (/TransactionService/…)

ForwardSnapshot, ForwardSnapshotBegin, ForwardSnapshotEnd, GetTransactionInterfaceVersion, AddNonStreamValuesBegin, AddNonStreamValues, AddNonStreamValuesEnd

A separate ArchestrA.CloudHistorian.Contract assembly defines a parallel cloud-ingest contract (AddHistorianValues, CreateTags, EnqueueTagDataPacket, Enqueue…, etc.) used by aahCloudConfigurator / online.wonderware.com. Out of scope here; noted for completeness.

---

5. Representative message shapes (protobuf field numbers)

The universal result wrapper and the auth/query messages — note how thin they are; the real structure lives inside the bytes fields.

// Common result wrapper (ArchestrA.Grpc.Contract.RequestStatus)
message Status {
  bool  bSuccess = 1;   // success flag (replaces WCF return-bool)
  bytes btError  = 2;   // native error buffer (same type/code blob as WCF err)
}

// HistoryService
message OpenConnectionRequest  { bytes btConnectionRequest  = 1; } // OpenConnection3 v6 buffer
message OpenConnectionResponse { Status status = 1; bytes btConnectionResponse = 2; } // 42-byte session blob
message ExchangeKeyRequest     { string strHandle = 1; bytes btInput = 2; }

// StorageService — Negotiate/NTLM handshake
message ValidateClientCredentialRequest  { string handle = 1; bytes inBuff  = 2; }
message ValidateClientCredentialResponse { Status status = 1; bytes outBuff = 2; }

// RetrievalService
message StartQueryRequest {
  uint32 uiHandle           = 1;
  uint32 uiQueryRequestType = 2;   // RetrievalMode → QueryType, same mapping as 2020
  bytes  btRequestBuffer    = 3;   // DataQueryRequest bytes, byte-identical to WCF
}

Across the contract the recurring pattern is { Status status; bytes <payload> } for responses and { [string handle][uint …] bytes <payload> } for requests.

The full canonical IDL has been recovered. All six .proto files were rendered from the embedded FileDescriptors and protoc-validated — see ../out/proto/*.proto, the portable ../out/archestra_grpc.fileset.pb (FileDescriptorSet), and the per-message field dump ../out/grpc-contract-dump.md. ../out/README.md explains the contract quirks (global proto package, cross-file name collisions, all-unary RPCs) and the 2020→gRPC read-path mapping. Regenerate with the protodump/ tool.

---

6. What this means for histsdk (if a gRPC transport is ever added)

This is not a request to implement anything — recording the path:

1. Add a transport enum value (e.g. RemoteGrpc) alongside LocalPipe / RemoteTcpIntegrated / RemoteTcpCertificate.
2. Reference Grpc.Net.Client + Grpc.Net.Client.Web; build a GrpcChannel.ForAddress("http(s)://host:32565", { HttpHandler = GrpcWebHandler(GrpcWeb, HttpClientHandler), … }) with HTTP/1.1.
3. Reuse every existing payload serializer unchanged — feed the same byte buffers into the protobuf bytes fields instead of MDAS bodies. The orchestrator call order (GetV → ValCl×N → Open2 → Retr.GetV → IsOriginalAllowed → StartQuery → GetNextQueryResultBuffer…) is identical.
4. Auth: still the SSPI/Negotiate token loop via ValidateClientCredential, carried in inBuff/outBuff. No per-call gRPC auth metadata needed.
5. Biggest win: no WCF [MessageParameter] reverse-engineering — the protobuf field numbers are authoritative and stable.

Caveat: this is the 2023 R2 server contract. The production SDK targets a 2020-era server; whether that server exposes the gRPC HCAP endpoint at all is a server-version question, not a client one. Treat this as forward-looking.

---

7. Artifacts (all under the separate analysis folder, none committed to histsdk)

histsdk-2023r2-analysis/
  msi-extract/                         # msiexec /a layout of SDKSetup.msi
  bin/                                 # copied key assemblies + aahClientManaged.xml
  decompiled/
    Archestra.Grpc.Contract/           # full protobuf contract (services + messages)
    Archestra.Historian.GrpcClient/    # transport wrappers (channel/auth/calls)
    ArchestrA.CloudHistorian.Contract/ # cloud-ingest contract (out of scope)
  protodump/                           # .NET 10 tool: descriptor graph -> .proto / dump
  out/
    proto/*.proto                      # recovered, protoc-validated IDL (6 files)
    archestra_grpc.fileset.pb          # portable FileDescriptorSet (grpcurl/buf/protoc)
    grpc-contract-dump.md              # per-message field dump + service tables
    README.md                          # artifact guide + contract quirks + read-path map
  docs/grpc-transport.md               # this file

gRPC redist proof in the installer: Redist/HistorianSDK 2023 R2/x64/{GRPCCore,GRPCNetClient,HistorianGRPCClient,HistorianGRPCContract,Protobuf}.msm plus shipped Grpc.Net.Client*.dll, Grpc.Core.Api.dll, Google.Protobuf.dll.