Extends the instrument-grpc rewrite to log string (strHandle) + uint (uiHandle /
queryRequestType) params, not just byte[], and captures our SDK's live v8
openParameters for a byte-diff against the native.
Result of the exhaustive comparison (all live-confirmed via the opt-in
EventReadDiagnostic test):
- StartEventQuery request: byte-identical to the native (v6 layout)
- v8 OpenConnection openParameters: byte-identical to the native (302B) once
ClientNodeName matches — every control byte/ConnectionType/token/ShardId
- handle usage identical: ExchangeKey->contextKey, registration->storage GUID
(strHandle), query->client uint (uiHandle); handles valid (RTag/EnsT=True)
- queryRequestType=3, registration order, gzip metadata header — all match
- window has events (native returns 50 now); eventCount not it
Every observable client-side byte matches the native, yet the server scopes 0
events to our connection. The event RPCs succeed over our transport and return a
valid EMPTY result (not a transport error), so this is a connection/server-level
difference (session affinity tied to the native Grpc.Core HTTP/2 connection or a
connection identity used to scope events) — invisible to and unfixable by client
payload matching. Needs server-side insight, not more wire RE.
Added opt-in diagnostics (RegistrationDiag, LastResultBufferHex,
LastEventOpenRequestHex). 326/326 offline; gated test still pins the no-row throw.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Continues closing the event-row gap after the v8 ExchangeKey/RC4 auth breakthrough.
- HistorianGrpcHandshake: the v8 EVENT path skips StorageService.ValidateClientCredential
(the native event connection authenticates purely via ExchangeKey + the RC4 token;
running the Negotiate loop establishes a different session scope).
- HistorianGrpcEventOrchestrator.RegisterCmEventTag: simplified to the exact native
gRPC event sequence (UpdateClientStatus -> RegisterTags -> EnsureTags -> GetHistorianInfo
-> GetSystemParameter x7), dropping the 2020-WCF-era cross-service GetV probes and
params-before-register that the gRPC event flow does not use. eventCount 5 -> 100.
- Opt-in diagnostics (RegistrationDiag, LastResultBufferHex/LastErrorBufferHex; gated
EventReadDiagnostic test) for the continued investigation.
STATUS: auth + StartEventQuery + registration all succeed live (RTag/EnsT=True, valid
query handle), but GetNext returns version-11 rowCount-0 while the native returns 50 for
a BYTE-IDENTICAL request. Every observable wire element matches the native. The remaining
unknown is the string/uint HANDLE field VALUES the native uses per event RPC — the
instrument-grpc capture logged only byte[] params, not the handle fields. Next: extend
the IL rewrite to log strHandle/uiHandle/queryRequestType, re-capture, and match.
326/326 offline; gated test still pins the no-row throw.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Implements the reverse-engineered v8 credential token in pure managed code and
wires the full event-connection auth chain. Live result: the v8 OpenConnection
now AUTHENTICATES against the 2023 R2 server (past the 132/171 AuthenticationFailed
wall) — the crypto is solved.
- HistorianNativeHandshake.DeriveExchangeKeyClientKey: client key = SHA256(ECDH
shared secret) via ECDiffieHellman.DeriveKeyFromHash(SHA256), matching the native
ECDiffieHellmanCng{Hash,SHA256}.DeriveKeyMaterial.
- BuildExchangeKeyCredentialToken + Rc4: token = RC4(password-UTF16LE, key=MD5(clientKey)).
Reproduces a live-captured token EXACTLY (verified offline) — the native
HistorianCrypto.NRC4_V2.aahCryptV2 scheme (MD5-keyed RC4). Pure managed; nothing
AVEVA shipped. RC4 pinned by the standard test vector.
- OpenSession(eventConnection:true): ExchangeKey -> derive client key -> token ->
v8 OpenConnection with ConnectionType=Event + the token. Orchestrator re-armed.
- HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc: the 86-byte native gRPC
CM_EVENT EnsureTags (8-byte header + ...2f27 event-type GUID), replacing the
2020 WCF 83-byte CTagMetadata on the gRPC event registration.
Goldens: RC4 standard vector + token construction. 326/326 offline.
KNOWN REMAINING: the event query still returns zero rows (GetNext yields a 10-byte
zero-row buffer). Auth + StartEventQuery succeed; the query-layer detail (vs the
native row-returning capture) is the last step. Gated test still pins the no-row
throw; opt-in diagnostic (HISTORIAN_GRPC_EVENT_DIAG) surfaces the journey.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Implements HistoryService.ExchangeKey as a pure-managed P-256 ECDH key exchange
and wires it ahead of the v8 Event OpenConnection.
- HistorianNativeHandshake.BuildExchangeKeyClientHello / DeriveExchangeKeySecret:
.NET ECDiffieHellman (nistP256); wire format "ECK1" + u32(32) + X(32) + Y(32),
decoded from the live capture. No native AVEVA dependency.
- HistorianGrpcHandshake.OpenSession(eventConnection: true): runs ExchangeKey on
the context-key handle before the v8 OpenConnection.
- Guardrail HistorianGrpcHandshakeRoutingTests scoped to the token-loop closure:
still pins that the Negotiate token loop routes to ValidateClientCredential (not
ExchangeKey), while allowing the legitimate ExchangeKey call in OpenSession.
Live result: ExchangeKey succeeds (server accepts our public key) and the v8
OpenConnection error advances from 132/34 "Failed to get client key" to 132/171
AuthenticationFailed — the ECDH cleared the client-key layer. The remaining
blocker is the 26-byte v8 credential token, which must be derived from the ECDH
shared secret (token KDF, not yet recovered). Orchestrator stays on v6 (set
eventConnection: true to re-arm once the KDF lands). 323/323 offline.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Builds the native 2023 R2 version-8 OpenConnection format, which (unlike v6)
carries a ConnectionType byte (Event vs Process) — required because the 2023 R2
server returns event rows only on an Event connection.
- HistorianOpen2Protocol.SerializeNativeOpenConnectionVersion8: reproduces the
302-byte v8 layout decoded from a live capture (version 8, markers, client-key
GUID, username HString, length-prefixed credential token, ClientType /
ConnectionType / flag / constant word / compact metadata / two empty strings;
the tail reuses WriteClientCommonInfo). Golden-tested
(Version8EventSerializerReproducesCapturedNativeStructure).
- HistorianNativeHandshake.BuildEventOpenConnectionVersion8Request: ConnectionType=
Event, zeroed credential token (mirroring how v6 zeros its credential block and
relies on the separate ValidateClientCredential handshake).
- HistorianGrpcHandshake.OpenSession: optional eventConnection switch; the
OpenConnection failure path now decodes the native error (type/code/ASCII).
Path A (reuse ValidateClientCredential + zeroed token) was live-tested and
DISPROVEN: the server parses the v8 buffer but rejects it at the auth check with
native 132/34 "EstablishConnection Failed to get client key" — the v8 path looks
up the client key in the registry HistoryService.ExchangeKey (ECDH) populates,
not the one ValidateClientCredential does. The event orchestrator is therefore
reverted to the v6 session (gated test still pins the no-row throw). The v8
serializer/builder are retained for Path B (implement ExchangeKey). 323/323
offline tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
The D2 storage-engine-pipe wall is WCF-transport-specific. On the 2023 R2 gRPC
front door, TransactionService is a first-class service AND the gateway to the
storage engine, so the Open2 storage-session GUID (uppercase) is accepted
directly as strHandle with no legacy pipe.
Live-verified against the real 2023 R2 server over a write-enabled (0x401) gRPC
session: AddNonStreamValuesBegin returns a real strTransactionId, and
AddNonStreamValuesEnd(bCommit=false) discards it cleanly (no data written). On
2020 WCF the same op returns UnknownClient(51) for every handle + priming chain.
- HistorianGrpcRevisionProbe + grpc-revision-probe CLI command + gated test
NonStreamedWriteTransaction_OverGrpc_BeginsAndDiscards (live pass).
- HistorianGrpcHandshake.OpenSession gains an optional connectionMode param
(default read-only 0x402; pass 0x401 for write-enabled) — non-breaking.
- Docs: revision-write-path.md "the wall is gone" section; roadmap M3 section,
R3.1-R3.3 rows, one-glance table, and status note updated honestly.
Not yet shipped: the AddNonStreamValues btInput VTQ buffer is uncaptured (never
guess wire bytes), so no value-commit is implemented. Scope is non-streamed
ORIGINAL backfill; revision EDITS (R4.2) remain pipe-only even on gRPC.
272 unit tests pass; sanitization scan clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Routes HistorianClient.GetTagMetadataAsync over gRPC when Transport==RemoteGrpc,
via the new Grpc/HistorianGrpcTagClient calling RetrievalService.GetTagInfosFromName
(the plural string-handle metadata op).
- String handle = the Open2 storage-session GUID formatted uppercase (the format
that resolves the native string-handle path); threaded out of the shared handshake
via a new HistorianGrpcHandshake.Session { ClientHandle, StorageSessionId, StringHandle }.
- Request btTagNames = uint count + per-name(uint charCount + UTF-16LE) — golden-byte
unit-tested (BuildTagNamesBuffer).
- Response btTagInfos = uint count + CTagMetadata records — decoded by the existing
HistorianTagQueryProtocol.ParseGetTagInfoResponse; data type via the shared MapDataType.
The 2020 WCF string-handle wall does NOT apply on the gRPC front door, as the
string-handle-wall RE note predicted. LIVE-VERIFIED against a real 2023 R2 server:
GetTagMetadataAsync returns the requested tag with a valid decoded data type.
216 unit tests pass. Captured framing confirmed live then discarded; no tag names
or identities committed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Roadmap docs/plans/hcal-roadmap.md, milestone M0 (gRPC parity for the DONE
surface). Now unblocked for live verification by a reachable 2023 R2 server.
- R0.4 Probe over gRPC: new HistorianGrpcProbe calls History/Retrieval/Status
GetInterfaceVersion (unauthenticated). ProbeAsync routes over gRPC when
Transport==RemoteGrpc. LIVE-VERIFIED against a real 2023 R2 server — needs no
credentials (runs before the auth loop), so it works despite the auth blocker.
- R0.3 System parameter over gRPC: new HistorianGrpcStatusClient calls
StatusService.GetSystemParameter over the authenticated session; routed in the
dialect. Built + unit-tested (request/response field mapping pinned).
Live-verification pending an auth fix (see below).
- Extracted the proven auth handshake from HistorianGrpcReadOrchestrator into
shared Grpc/HistorianGrpcHandshake (reused by read + status + future
browse/metadata). Repointed the IL structural guardrail test to it.
- Diagnostics: round-failure now decodes the native server error + hex/ASCII
preview (HistorianNativeHandshake.DescribeError). This surfaced the live auth
blocker as SEC_E_LOGON_DENIED (0x8009030C) at NTLM round 1 — framing is correct,
the credential did not validate. Probable cause: stale file password or NAM-domain
NTLM restriction (Kerberos/RDP works, NTLM denied; no SPN path over the tunnel).
216 unit tests pass; live gRPC probe passes. Sanitization scan clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Joseph Doherty