diff --git a/docs/reverse-engineering/grpc-event-query-capture.md b/docs/reverse-engineering/grpc-event-query-capture.md index 17de437..cb56466 100644 --- a/docs/reverse-engineering/grpc-event-query-capture.md +++ b/docs/reverse-engineering/grpc-event-query-capture.md @@ -139,3 +139,93 @@ as the captured-correct request format** for when the open is rebuilt. Capture artifacts (gitignored): `artifacts/reverse-engineering/grpc-event-capture/` — `event-capture.ndjson` (Event), `process-connect-2.ndjson` (Process). + +## v8 `openParameters` fully decoded (2026-06-23) + the ECDH ExchangeKey finding + +Full byte map of the native Event-connection `openParameters` (302 bytes; identity values +redacted — they are session-specific and sit in the gitignored capture): + +``` +[0] byte 0x08 format version = 8 +[1] byte 0xf0 constant marker +[2..20] 19 × 0x00 +[21] byte 0x01 constant marker +[22..37] 16B GUID per-session client key +[38..41] u32 username length (chars) +[42..N] UTF-16 username (HistorianString) +[..+1] u16 credential-token length (= 26 in the capture) +[..] 26B token ECDH-derived credential token <-- see below +[94] byte 0x04 ClientType (= our NativeClientType 4) +[95] byte ConnectionType 01 = Event / 02 = Process <-- THE GATE +[96] byte flag 01 (Event) / 00 (Process) +[97..] control bytes (0x03 ... small region, not fully named) +[~114..117]u32 FormatVersion=3 +[..] HistorianString machine/server node name +[..] HistorianString client node name "()" +[..] u32 session-variable (process-ish) +[..] u32 / zeros +[..] u32 datasource len +[..] UTF-16 datasource id e.g. "2023.1219.4004.5" +[270..285] 16 × 0xff ShardId (all-FF = unset; our v6 sends Empty) +[286..289] u32 client/hcal version int +[290..297] i64 FILETIME ClientTimestamp +[298..301] u32 0 +``` + +The tail (`FormatVersion` → machine → clientNode → datasource → ShardId → version → timestamp) +is the **same `ClientCommonInfo` our v6 already emits**. The new/different parts are: version byte, +the `[1]`/`[21]` markers, the GUID position, the **26-byte credential token** (vs v6's fixed-size +block), the **`ConnectionType` byte**, and ShardId=FF. + +**The auth is ECDH, not Negotiate.** The capture's `ExchangeKey` buffers begin `45 43 4b 31` = +ASCII **`"ECK1"`** + a 64-byte EC public-key point — a Diffie-Hellman key exchange — and the 26-byte +`openParameters` token is derived from it. `HistorianSecurityMode` offers only `Disabled` / `None` / +`TransportCertificate`; the harness used `TransportCertificate`, which is what drives the ECDH +`ExchangeKey`. There is **no TLS+Negotiate mode** on the native client (it couples TLS with the cert +ECDH path), so a Negotiate-auth v8 capture cannot be produced from the native client. + +**Key de-risking insight:** our SDK's v6 `OpenConnection` sends a **fully zeroed** 1026-byte +credential block (`credentialBlock: new byte[1026]`) and reads still work — because authentication is +actually carried by the separate `StorageService.ValidateClientCredential` (Negotiate) handshake, not +by the bytes inside `openParameters`. By analogy the v8 `[68..93]` token may likewise be **ignorable** +once `ValidateClientCredential` has run. So the first build hypothesis (cheapest, read-only to test): + +> Reuse the SDK's existing `ValidateClientCredential` handshake, then send a **v8 `OpenConnection` +> with `ConnectionType=Event` and a zeroed credential token**, and see whether the 2023 R2 server +> returns event rows. + +If that works, the ECDH ExchangeKey RE is unnecessary. If it fails, the fallback is full reproduction +of the ECDH `ExchangeKey` handshake (curve/KDF/cipher) — a much larger crypto-RE effort. Build path: +add `SerializeNativeOpenConnectionVersion8(connectionType)` to `HistorianOpen2Protocol`, wire the gRPC +event handshake to use it (events only; reads stay on v6), live-test (non-destructive). Full hex in +the gitignored capture. + +### Path A built + live-tested 2026-06-23 — DISPROVEN (v8 is coupled to ExchangeKey) + +Built `HistorianOpen2Protocol.SerializeNativeOpenConnectionVersion8` (golden-tested, +`Version8EventSerializerReproducesCapturedNativeStructure` — reproduces the captured 302-byte +structure exactly) + `HistorianNativeHandshake.BuildEventOpenConnectionVersion8Request` (zeroed +credential token) + an `eventConnection` switch on `HistorianGrpcHandshake.OpenSession`, and live-ran +the event read against the server. Result: the v8 `OpenConnection` was **parsed by the server** (got +past the byte format) but **rejected at the auth check** with native error + +``` +type=132 code=34 "aahHcapLib::HistoryService::EstablishConnection — Failed to get client key" +``` + +i.e. `EstablishConnection` could not find a server-side **client key** for our session. In the v6 +path that key is established by `StorageService.ValidateClientCredential` (which is why v6 reads +work); the v8 path looks it up in the registry that **`HistoryService.ExchangeKey` (ECDH)** populates, +and there is **no `ValidateClientCredential` on `HistoryService`** in the gRPC contract. So the server +branches on the OpenConnection version: v6 accepts the Negotiate-established key, **v8 requires the +ExchangeKey-established key**. The zeroed-token hypothesis is therefore disproven — not because of the +token bytes, but because the whole v8 path is gated on `ExchangeKey` having run first. + +**Status:** the v8 serializer/builder are correct and retained (golden-tested), plus the +`OpenConnection` failure now decodes the native error (type/code/ASCII). The event orchestrator is +reverted to the v6 session (gated test still pins the no-row throw). The remaining route is **Path B: +implement `HistoryService.ExchangeKey`** — `"ECK1"` + a 64-byte EC public-key point (P-256 X‖Y, by the +size) — using .NET `ECDiffieHellman`, establish the client key, then reissue the v8 `OpenConnection`. +Open question for Path B: whether merely *completing* the ECDH key agreement registers the client key +(so the zeroed openParameters token still rides through), or whether the token must also be derived +from the shared secret (full KDF/cipher RE).