R0.1 browse over gRPC SHIPPED — QueryTag cracked, M0 gRPC parity complete

Wires HistorianClient.BrowseTagNamesAsync over gRPC (Transport==RemoteGrpc) via
Grpc/HistorianGrpcTagClient.BrowseTagNamesAsync: StartTagQuery(OData) -> paged
QueryTag -> EndTagQuery. Live-verified against a real 2023 R2 server (returns Sys* tags).

QueryTag packet-id recovered WITHOUT native disassembly: a .rdata packet-descriptor
table in aahClientManaged.dll lists {0x6751,1}=StartTagQuery immediately followed by
{0x6752,1}=QueryTag (found via pefile byte-scan of .rdata), confirmed live.

Wire format (live-verified):
- request btRequest = u16 0x6752 + u16 version(1) + u16 queryType(1=names) + u32 startIndex + u32 count
- response btResonse = u32 count + per-name(u32 charCount + UTF-16LE) + trailer (NextIndex/metadata, ignored)
- new HistorianTagQueryProtocol.ParseTagNameQueryPage tolerates the trailer
- GlobToODataFilter translates the SDK glob filter to OData (Pre*->startswith, *suf->endswith,
  *mid*->contains, exact->eq); the 2023 R2 metadata-server parses filters as OData.

Replaces the earlier RE probe helpers with the shipped browse path. Adds golden-byte
(BuildQueryTagRequest) + 8 glob-translation unit tests + gated live browse test.
226 unit tests pass; 5/5 live gRPC tests pass (read, probe, system-param, metadata, browse).

Milestone 0 (full gRPC parity) is complete.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
This commit is contained in:
Joseph Doherty
2026-06-21 16:01:15 -04:00
parent 630295bd18
commit 85ff1b48df
7 changed files with 244 additions and 115 deletions
+13 -8
View File
@@ -36,14 +36,19 @@ HCAL replacement, built on the **2023 R2 gRPC transport**. Derived from
records (reuses `ParseGetTagInfoResponse`); string handle = uppercase Open2 storage GUID. The 2020
WCF string-handle wall does **not** apply on the gRPC front door (as predicted). **LIVE-VERIFIED
2026-06-21** — `GetTagMetadataAsync` returned the requested tag + a valid data type.
- 🟡 **R0.1 Browse over gRPC**PARTIAL. Probed live 2026-06-21: the 2023 R2 gRPC front door does
**not** hit the 2020 metadata-server-pipe wall. `RetrievalService.StartTagQuery` is **cracked** — it
parses the filter as **OData** (`startswith(TagName,'Sys')`/`contains`/`eq`/empty succeed, returning
the 8-byte `(queryHandle, tagCount)`; SQL-LIKE `%`/glob `*``ODataFilter: bad token`). Live: 220
Sys* tags counted. **Remaining:** the `QueryTag` paging request format (constant native error
type 4 / code 72 across guessed `btRequest` shapes) — needs a native capture, not guessing. Probe
helpers (`HistorianGrpcTagClient.ProbeStartTagQuery`/`ProbeTagQuerySequence`) + a gated StartTagQuery
test are committed. Full finding: `docs/reverse-engineering/grpc-tag-query-odata.md`.
- **R0.1 Browse over gRPC**DONE, **LIVE-VERIFIED 2026-06-21**.
`HistorianClient.BrowseTagNamesAsync` routes over gRPC via
`Grpc/HistorianGrpcTagClient.BrowseTagNamesAsync`: StartTagQuery(**OData** filter) → paged
**QueryTag** (`btRequest` = `u16 0x6752 + u16 1 + u16 queryType + u32 startIndex + u32 count`) →
EndTagQuery; response = `u32 count + per-name(u32 charCount + UTF-16LE) + trailer`. The SDK glob
filter is translated by `GlobToODataFilter` (`Pre*``startswith`, `*suf``endswith`, `*mid*`
`contains`, exact→`eq`). The QueryTag packet-id `0x6752` was recovered from a `.rdata`
packet-descriptor table (`{0x6751,1}`=StartTagQuery, `{0x6752,1}`=QueryTag) — no Ghidra needed.
Golden-byte + glob unit tests + gated live test. Full finding:
`docs/reverse-engineering/grpc-tag-query-odata.md`.
> ✅ **Milestone 0 (gRPC parity) is COMPLETE** — probe, system-param, metadata, and browse all run
> over `RemoteGrpc` and are live-verified against a real 2023 R2 server, alongside the read chain.
> ️ **Auth note (2026-06-21, resolved):** an apparent NTLM round-1 `SEC_E_LOGON_DENIED` blocker
> turned out to be a **test-harness credential-parsing bug**, not a server/account/SDK issue — the
@@ -28,45 +28,30 @@ Success response `btResponse` is the 8-byte `(queryHandle:uint, tagCount:uint)`
`*` → empty, `Pre*` → `startswith(TagName,'Pre')`, `*sub*` → `contains(TagName,'sub')`,
exact → `TagName eq '...'`. (Escaping single-quotes in names still TBD.)
## QueryTag — OPEN (needs native RE of aahClient.dll)
## QueryTag — CRACKED (2026-06-21), browse SHIPPED
`QueryTag(strHandle, uiQueryHandle, btRequest)` is the paging call that returns the tag-name rows.
Every `btRequest` shape tried returns the constant native error **type 4 / code 72 =
`InvalidPacketId`** (`ArchestrA.CloudHistorian.Contract.ErrorCode.InvalidPacketId = 72`; the empty
buffer alone gives a different code). So the `btRequest` must carry a **packet-id header specific to
QueryTag** that we don't have — the generic `0x6751`/version-1 header (which StartTagQuery accepts) is
rejected here.
The blocker was the packet id: every guessed `btRequest` returned native error **type 4 / code 72 =
`InvalidPacketId`** (`ArchestrA.CloudHistorian.Contract.ErrorCode.InvalidPacketId`). The generic
`0x6751` header that StartTagQuery accepts is the **wrong** id for QueryTag.
**Semantic fields are known** from `ArchestrA.CloudHistorian.Contract`:
- request `QueryTagRequest` = `QueryHandle:uint("q") + QueryType:ushort("t") + StartIndex:uint("s") + TagCount:uint("c")`
- response `QueryTagResponse` = `QueryHandle:uint + TagNames:string[]("t") + NextIndex:uint("i") + TagMetadataBuffer:byte[]("tb")`
— so QueryTag returns the names directly (plus an optional metadata buffer), and pages via StartIndex/NextIndex.
**How it was found (no Ghidra needed):** a `.rdata` **packet-descriptor table** in
`aahClientManaged.dll` lists consecutive `{uint marker, uint version}` entries —
`{0x6751, 1}` (StartTagQuery) immediately followed by **`{0x6752, 1}`** (the paired op). Found by
`pefile` byte-scan of `.rdata` for `51 67 00 00` and dumping the surrounding dwords. Testing `0x6752`
live confirmed it.
What's missing is the **binary `btRequest` packet framing** (the QueryTag packet id + how those fields
are laid out). That serializer lives in **native `aahClient.dll`** — `aahClientManaged.dll` is
**mixed-mode (C++/CLI)** so ilspycmd cannot decompile it, and no managed assembly builds the buffer.
Completing QueryTag therefore requires **native RE (Ghidra/IDA on `aahClient.dll`)** or a **live gRPC
capture** of the stock 2023 R2 client browsing. Do not ship a guessed QueryTag request (project
discipline: no guessed wire bytes).
**QueryTag wire format (live-verified):**
- request `btRequest` = `u16 marker(0x6752) + u16 version(1) + u16 queryType + u32 startIndex + u32 count`
— `queryType = 1` returns tag-name rows (`queryType = 0` returns an empty/count-only page).
- response `btResonse` = `u32 count + per-name(u32 charCount + UTF-16LE) + trailer`
(the trailer is the CloudHistorian `NextIndex`/`TagMetadataBuffer` region — ignored by
`HistorianTagQueryProtocol.ParseTagNameQueryPage`).
- Semantic fields match `ArchestrA.CloudHistorian.Contract.QueryTagRequest`
(`QueryType/StartIndex/TagCount`; the QueryHandle travels in the protobuf `uiQueryHandle`).
Probe helpers live in `Grpc/HistorianGrpcTagClient` (`ProbeStartTagQuery`, `ProbeTagQuerySequence`)
and the gated `StartTagQuery_OverGrpc_AcceptsODataFilter` test pins the StartTagQuery+OData result.
### Native-RE attempt (2026-06-21) — needs Ghidra/IDA
The QueryTag packet-id is built in native code (the C++ HCAL inside the **mixed-mode**
`aahClientManaged.dll`). Lightweight tooling was exhausted and is insufficient:
- **ilspycmd** cannot decompile or even list the mixed-mode assembly (throws).
- **capstone** byte-search: the StartTagQuery marker `0x6751` is **not** a plain `mov` immediate in
`.text` (no `66 C7 … 51 67` and no clean `B8 51 67 00 00`); the three `51 67 00 00` hits in `.text`
are coincidental jump-table data (disassembly around them is garbage). The constant lives in the
`.rdata` pool (which holds the `51 67` bytes) and is loaded RIP-relative — so the serializer can't be
found by immediate-scanning without cross-reference analysis.
- Managed metadata (`ArchestrA.CloudHistorian.Contract`) gives the field semantics —
`QueryTagRequest{SessionId, QueryHandle:uint, QueryType:ushort, StartIndex:uint, TagCount:uint}`,
DataContract code `QTLQ` (StartTagQuery = `TLQR`) — but **not** the binary packet-id/framing.
Finishing QueryTag therefore requires **Ghidra/IDA** (decompiler + xref to the `.rdata` marker constant
to find the serializer and read the QueryTag packet-id + byte layout), or the **live capture** option
(IL-rewrite `Archestra.Historian.GrpcClient.QueryTag` to log `requestBuffer` while a real 2023 R2
client browses). StartTagQuery (the hard OData part) remains cracked and live-verified.
**Browse is shipped:** `HistorianClient.BrowseTagNamesAsync` routes over gRPC when
`Transport==RemoteGrpc` via `Grpc/HistorianGrpcTagClient.BrowseTagNamesAsync`
(StartTagQuery(OData) → paged QueryTag(0x6752) → EndTagQuery), with the SDK glob filter translated by
`GlobToODataFilter`. Golden-byte + glob unit tests and a gated live test
(`BrowseTagNamesAsync_OverGrpc_ReturnsSystemTags`) cover it. **M0 gRPC parity is complete.**