Pursued hypothesis #3 (connection-frame capture). Built a TLS-terminating tee proxy
(artifacts/.../httpcap/, gitignored: self-signed server cert, forwards through the
loopback tunnel, logs decrypted HTTP/1.1 + gRPC-Web both directions) and ran a native
capture-event (returns 50 rows) and our SDK diagnostic (0 rows) through the SAME
proxy/upstream for a clean A/B.
Findings:
- The stock client is gRPC-Web/HTTP-1.1 (alpn empty), and clientCert=none on every
connection — confirming (with the decompile) that hypothesis #2 (TLS client cert) is
moot: the native presents no client cert.
- Connection topology differs: the native opens 5 TLS connections, one per service, and
runs the event query (StartEventQuery/GetNext/EndEventQuery) on a DEDICATED
RetrievalService connection, separate from the HistoryService connection that opened and
registered the session. Our SDK collapses every service onto one connection. (Matches the
decompile: the stock client has a separate GrpcClientBase per service.)
- Framing differs benignly: native uses content-length + Expect: 100-continue; SDK uses
transfer-encoding: chunked. The server accepts both (StartEventQuery returns a valid
handle), so framing is not the gate. No hidden header on either side.
Tested the topology hypothesis with a new env-gated switch
(HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL=1): run StartEventQuery/GetNext/EndEventQuery on a
dedicated RetrievalService connection (no re-handshake, reusing the session handle —
mirroring native conn4), registration staying on the main connection. Result: still
0B00000000001E000000 (0 rows), QH=1063. Splitting the event query onto its own connection
does not make rows flow — the server correlates by session handle, not connection, so
topology is not the row-scoping gate.
Every angle is now exhausted (payload, transport, metadata/interceptor/cert, topology,
data store). The gate is a server-internal per-connection retrieval working-set in the
native HistorianClient C++ core, unreachable from a pure-managed client. Conclusion
unchanged: auth-solved / retrieval-server-gated; ReadEventsAsync over gRPC keeps the
no-row throw, event reads use WCF. 56 offline gRPC/event tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Tested grpc-event-query-capture.md's leading next-session hypothesis — that the
native client's Grpc.Core HTTP/2 transport (vs our Grpc.Net.Client + GrpcWebHandler
gRPC-Web) is why event reads return zero rows. Added HistorianGrpcChannelFactory
.CreateHttp2 (plain HTTP/2 over SocketsHttpHandler, no gRPC-Web wrap) and an
HISTORIAN_GRPC_EVENT_HTTP2 switch on the event orchestrator (event path only; reads
stay gRPC-Web).
Live side-by-side against the event-bearing 2023 R2 server, everything else held
constant: the full v8 chain (ExchangeKey auth, CM_EVENT RegisterTags/EnsureTags=True,
StartEventQuery with a valid handle) runs end-to-end over BOTH native HTTP/2 and
gRPC-Web, and the server returns the byte-identical version-11 rowCount-0 terminal
(0B00000000001E000000) on both transports. Transport choice makes no difference —
the leading hypothesis is disproven and the zero-row scoping sits above the gRPC
transport layer.
Also confirmed the native capture-event harness queries a 30-day historical window
(returns 50 rows), so the native read is connection-scoped historical retrieval,
not a live subscription.
CreateHttp2 + the env switch + the EventChannelMode diagnostic are retained for
further connection-level probing. 44 offline tests pass; orchestrator stays on the
no-row throw.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Extends the instrument-grpc rewrite to log string (strHandle) + uint (uiHandle /
queryRequestType) params, not just byte[], and captures our SDK's live v8
openParameters for a byte-diff against the native.
Result of the exhaustive comparison (all live-confirmed via the opt-in
EventReadDiagnostic test):
- StartEventQuery request: byte-identical to the native (v6 layout)
- v8 OpenConnection openParameters: byte-identical to the native (302B) once
ClientNodeName matches — every control byte/ConnectionType/token/ShardId
- handle usage identical: ExchangeKey->contextKey, registration->storage GUID
(strHandle), query->client uint (uiHandle); handles valid (RTag/EnsT=True)
- queryRequestType=3, registration order, gzip metadata header — all match
- window has events (native returns 50 now); eventCount not it
Every observable client-side byte matches the native, yet the server scopes 0
events to our connection. The event RPCs succeed over our transport and return a
valid EMPTY result (not a transport error), so this is a connection/server-level
difference (session affinity tied to the native Grpc.Core HTTP/2 connection or a
connection identity used to scope events) — invisible to and unfixable by client
payload matching. Needs server-side insight, not more wire RE.
Added opt-in diagnostics (RegistrationDiag, LastResultBufferHex,
LastEventOpenRequestHex). 326/326 offline; gated test still pins the no-row throw.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Continues closing the event-row gap after the v8 ExchangeKey/RC4 auth breakthrough.
- HistorianGrpcHandshake: the v8 EVENT path skips StorageService.ValidateClientCredential
(the native event connection authenticates purely via ExchangeKey + the RC4 token;
running the Negotiate loop establishes a different session scope).
- HistorianGrpcEventOrchestrator.RegisterCmEventTag: simplified to the exact native
gRPC event sequence (UpdateClientStatus -> RegisterTags -> EnsureTags -> GetHistorianInfo
-> GetSystemParameter x7), dropping the 2020-WCF-era cross-service GetV probes and
params-before-register that the gRPC event flow does not use. eventCount 5 -> 100.
- Opt-in diagnostics (RegistrationDiag, LastResultBufferHex/LastErrorBufferHex; gated
EventReadDiagnostic test) for the continued investigation.
STATUS: auth + StartEventQuery + registration all succeed live (RTag/EnsT=True, valid
query handle), but GetNext returns version-11 rowCount-0 while the native returns 50 for
a BYTE-IDENTICAL request. Every observable wire element matches the native. The remaining
unknown is the string/uint HANDLE field VALUES the native uses per event RPC — the
instrument-grpc capture logged only byte[] params, not the handle fields. Next: extend
the IL rewrite to log strHandle/uiHandle/queryRequestType, re-capture, and match.
326/326 offline; gated test still pins the no-row throw.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Implements the reverse-engineered v8 credential token in pure managed code and
wires the full event-connection auth chain. Live result: the v8 OpenConnection
now AUTHENTICATES against the 2023 R2 server (past the 132/171 AuthenticationFailed
wall) — the crypto is solved.
- HistorianNativeHandshake.DeriveExchangeKeyClientKey: client key = SHA256(ECDH
shared secret) via ECDiffieHellman.DeriveKeyFromHash(SHA256), matching the native
ECDiffieHellmanCng{Hash,SHA256}.DeriveKeyMaterial.
- BuildExchangeKeyCredentialToken + Rc4: token = RC4(password-UTF16LE, key=MD5(clientKey)).
Reproduces a live-captured token EXACTLY (verified offline) — the native
HistorianCrypto.NRC4_V2.aahCryptV2 scheme (MD5-keyed RC4). Pure managed; nothing
AVEVA shipped. RC4 pinned by the standard test vector.
- OpenSession(eventConnection:true): ExchangeKey -> derive client key -> token ->
v8 OpenConnection with ConnectionType=Event + the token. Orchestrator re-armed.
- HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc: the 86-byte native gRPC
CM_EVENT EnsureTags (8-byte header + ...2f27 event-type GUID), replacing the
2020 WCF 83-byte CTagMetadata on the gRPC event registration.
Goldens: RC4 standard vector + token construction. 326/326 offline.
KNOWN REMAINING: the event query still returns zero rows (GetNext yields a 10-byte
zero-row buffer). Auth + StartEventQuery succeed; the query-layer detail (vs the
native row-returning capture) is the last step. Gated test still pins the no-row
throw; opt-in diagnostic (HISTORIAN_GRPC_EVENT_DIAG) surfaces the journey.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Implements HistoryService.ExchangeKey as a pure-managed P-256 ECDH key exchange
and wires it ahead of the v8 Event OpenConnection.
- HistorianNativeHandshake.BuildExchangeKeyClientHello / DeriveExchangeKeySecret:
.NET ECDiffieHellman (nistP256); wire format "ECK1" + u32(32) + X(32) + Y(32),
decoded from the live capture. No native AVEVA dependency.
- HistorianGrpcHandshake.OpenSession(eventConnection: true): runs ExchangeKey on
the context-key handle before the v8 OpenConnection.
- Guardrail HistorianGrpcHandshakeRoutingTests scoped to the token-loop closure:
still pins that the Negotiate token loop routes to ValidateClientCredential (not
ExchangeKey), while allowing the legitimate ExchangeKey call in OpenSession.
Live result: ExchangeKey succeeds (server accepts our public key) and the v8
OpenConnection error advances from 132/34 "Failed to get client key" to 132/171
AuthenticationFailed — the ECDH cleared the client-key layer. The remaining
blocker is the 26-byte v8 credential token, which must be derived from the ECDH
shared secret (token KDF, not yet recovered). Orchestrator stays on v6 (set
eventConnection: true to re-arm once the KDF lands). 323/323 offline.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Builds the native 2023 R2 version-8 OpenConnection format, which (unlike v6)
carries a ConnectionType byte (Event vs Process) — required because the 2023 R2
server returns event rows only on an Event connection.
- HistorianOpen2Protocol.SerializeNativeOpenConnectionVersion8: reproduces the
302-byte v8 layout decoded from a live capture (version 8, markers, client-key
GUID, username HString, length-prefixed credential token, ClientType /
ConnectionType / flag / constant word / compact metadata / two empty strings;
the tail reuses WriteClientCommonInfo). Golden-tested
(Version8EventSerializerReproducesCapturedNativeStructure).
- HistorianNativeHandshake.BuildEventOpenConnectionVersion8Request: ConnectionType=
Event, zeroed credential token (mirroring how v6 zeros its credential block and
relies on the separate ValidateClientCredential handshake).
- HistorianGrpcHandshake.OpenSession: optional eventConnection switch; the
OpenConnection failure path now decodes the native error (type/code/ASCII).
Path A (reuse ValidateClientCredential + zeroed token) was live-tested and
DISPROVEN: the server parses the v8 buffer but rejects it at the auth check with
native 132/34 "EstablishConnection Failed to get client key" — the v8 path looks
up the client key in the registry HistoryService.ExchangeKey (ECDH) populates,
not the one ValidateClientCredential does. The event orchestrator is therefore
reverted to the v6 session (gated test still pins the no-row throw). The v8
serializer/builder are retained for Path B (implement ExchangeKey). 323/323
offline tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Captured the stock 2023 R2 client doing a gRPC event read (50 rows flowed) to
resolve the open "gRPC event ROW retrieval returns zero rows" item. Two captured
differences from our SDK's path; this lands the first (necessary) one plus the
capture tooling.
- HistorianEventQueryProtocol.CreateStartEventQueryAttempts: add a `version`
parameter (default 5 = the 2020 WCF format, unchanged). The gRPC event
orchestrator now opts into version 6 — the leading `06` plus a 5-byte trailing
zero pad — which is the envelope the stock 2023 R2 client sends. The two
buffers are otherwise byte-identical (filter block, UTC string, metadata
namespace). Golden test Version6EmptyFilterMatchesCapturedGrpcEnvelope pins it.
- Grpc2023CaptureHarness: new `capture-event` scenario drives HistorianAccess
over an Event-type gRPC connection (CreateEventQuery -> EventQueryArgs ->
StartQuery -> MoveNext) so the wide-net instrument-grpc-nonstream rewrite dumps
StartEventQuery.requestBuffer + the row result. Hostname defaults sanitized to
HISTORIAN_GRPC_HOST / "localhost" (removed hardcoded server name).
NECESSARY BUT NOT SUFFICIENT: live validation shows v6 alone does not make rows
flow — the read also requires an Event-type connection, which our SDK's v6 Open2
format cannot express (see the companion docs commit). The gated
ReadEventsAsync_OverGrpc_* test correctly still pins the no-row throw. 322/322
offline tests pass; WCF event path unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Adds HistorianGrpcEventOrchestrator: opens a read-only gRPC session, replays the
CM_EVENT registration (UpdateClientStatus -> 6 GetSystemParameter -> RegisterTags
-> cross-service version probes -> EnsureTags), then StartEventQuery -> loop
GetNextEventQueryResultBuffer -> EndEventQuery, reusing the WCF query builder and
row parser verbatim. Routed in Historian2020ProtocolDialect on UseGrpc.
The captured registration buffers (CmEventTagId, UpdC3 blob, RTag2 buffer, GETHI
builder, pre-register param list, native-error decode) are extracted into a shared
HistorianEventRegistrationProtocol so the WCF and gRPC paths can't drift; the WCF
orchestrator is refactored onto it with no behavior change.
Live finding (2026-06-22): the chain runs and StartEventQuery succeeds, but the
gRPC server long-polls GetNextEventQueryResultBuffer on no data (it blocks to the
deadline instead of returning the synchronous 5-byte code-85 terminal the WCF op
returns). Per-call gRPC-Web deadlines proved unreliable over a tunnel, so the read
is hard-bounded by an overall linked-CTS budget (<=30s; gRPC honors token
cancellation). On the no-row path it throws ProtocolEvidenceMissingException rather
than assert a false-empty list. Row-level retrieval awaits an event-bearing 2023 R2
server (the dev box holds no events).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Joseph Doherty