gRPC events: capture decrypted HTTP/1.1 frames native vs ours — topology found + tested null

Pursued hypothesis #3 (connection-frame capture). Built a TLS-terminating tee proxy
(artifacts/.../httpcap/, gitignored: self-signed server cert, forwards through the
loopback tunnel, logs decrypted HTTP/1.1 + gRPC-Web both directions) and ran a native
capture-event (returns 50 rows) and our SDK diagnostic (0 rows) through the SAME
proxy/upstream for a clean A/B.

Findings:
- The stock client is gRPC-Web/HTTP-1.1 (alpn empty), and clientCert=none on every
  connection — confirming (with the decompile) that hypothesis #2 (TLS client cert) is
  moot: the native presents no client cert.
- Connection topology differs: the native opens 5 TLS connections, one per service, and
  runs the event query (StartEventQuery/GetNext/EndEventQuery) on a DEDICATED
  RetrievalService connection, separate from the HistoryService connection that opened and
  registered the session. Our SDK collapses every service onto one connection. (Matches the
  decompile: the stock client has a separate GrpcClientBase per service.)
- Framing differs benignly: native uses content-length + Expect: 100-continue; SDK uses
  transfer-encoding: chunked. The server accepts both (StartEventQuery returns a valid
  handle), so framing is not the gate. No hidden header on either side.

Tested the topology hypothesis with a new env-gated switch
(HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL=1): run StartEventQuery/GetNext/EndEventQuery on a
dedicated RetrievalService connection (no re-handshake, reusing the session handle —
mirroring native conn4), registration staying on the main connection. Result: still
0B00000000001E000000 (0 rows), QH=1063. Splitting the event query onto its own connection
does not make rows flow — the server correlates by session handle, not connection, so
topology is not the row-scoping gate.

Every angle is now exhausted (payload, transport, metadata/interceptor/cert, topology,
data store). The gate is a server-internal per-connection retrieval working-set in the
native HistorianClient C++ core, unreachable from a pure-managed client. Conclusion
unchanged: auth-solved / retrieval-server-gated; ReadEventsAsync over gRPC keeps the
no-row throw, event reads use WCF. 56 offline gRPC/event tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
This commit is contained in:
Joseph Doherty
2026-06-23 13:43:21 -04:00
parent 6cf4dd13fe
commit 8199dde452
2 changed files with 67 additions and 38 deletions
@@ -281,9 +281,21 @@ internal sealed class HistorianGrpcEventOrchestrator
HistorianEventFilter? filter,
CancellationToken cancellationToken)
{
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
// HTTP/2-frame capture (grpc-event-query-capture.md #3) showed the stock client runs the event
// query on a DEDICATED RetrievalService TLS connection, separate from the HistoryService
// connection that opened+registered the session (correlated only by the session handle); our SDK
// collapses every service onto one connection. Opt in via HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL=1 to
// run StartEventQuery/GetNext/EndEventQuery on their own connection (mirrors native conn4: no
// re-handshake, just the existing handle), to test whether topology is the row-scoping gate.
bool splitChannel = string.Equals(
Environment.GetEnvironmentVariable("HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL"), "1", StringComparison.Ordinal);
HistorianGrpcConnection rconn = splitChannel ? HistorianGrpcChannelFactory.Create(_options) : connection;
try
{
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(rconn.Channel);
GrpcRetrieval.GetRetrievalInterfaceVersionResponse retrievalVersion = retrievalClient.GetRetrievalInterfaceVersion(
new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken);
new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), rconn.Metadata, Deadline(), cancellationToken);
HistorianServerVersionGate.Validate(HistorianServiceInterface.Retrieval, retrievalVersion.UiVersion, _options);
// Version 6 envelope: the stock 2023 R2 client sends v6 (the WCF path's v5 request is accepted
@@ -306,7 +318,7 @@ internal sealed class HistorianGrpcEventOrchestrator
UiQueryRequestType = HistorianEventQueryProtocol.QueryRequestTypeEvent,
BtRequest = ByteString.CopyFrom(requestBuffer)
},
connection.Metadata,
rconn.Metadata,
Deadline(),
cancellationToken);
@@ -331,7 +343,7 @@ internal sealed class HistorianGrpcEventOrchestrator
{
nextResponse = retrievalClient.GetNextEventQueryResultBuffer(
new GrpcRetrieval.GetNextEventQueryResultBufferRequest { UiHandle = session.ClientHandle, UiQueryHandle = queryHandle },
connection.Metadata,
rconn.Metadata,
EventPollDeadline(),
cancellationToken);
}
@@ -384,7 +396,12 @@ internal sealed class HistorianGrpcEventOrchestrator
}
finally
{
EndEventQuerySafely(retrievalClient, connection, session.ClientHandle, queryHandle);
EndEventQuerySafely(retrievalClient, rconn, session.ClientHandle, queryHandle);
}
}
finally
{
if (splitChannel) { rconn.Dispose(); }
}
}