dohertj2/histsdk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

M3 R3.1 decode: AddNonStreamValues reaches server StoreNonStreamValues (storage-engine console pipe)

Browse Source 
Empirically decoded the AddNonStreamValues btInput framing against the live 2023
R2 server (grpc-nonstream-decode command + ProbeNonStreamedBuffersAsync driver).
Every transaction rolled back (bCommit=false) — no data written.

Finding: the btInput is assembled native-C++-side (not in any decompile), so 6
evidence-based framings (44-54B, packed HISTORIAN_VALUE2 variants) were probed.
All 6 returned the IDENTICAL server error while an empty buffer returned a
different InvalidParameter — so non-empty buffers pass parameter validation into
CHistStorageConnection::StoreNonStreamValues, which routes to the
\.\pipe\aahStorageEngine\console pipe server-side. Identical-across-framings =>
the blocker is NOT the btInput layout but a missing storage-engine console
session / tag-registration precondition for the connection.

Next step (untested): StorageService.OpenStorageConnection + tag registration
(RegisterTags/AddTagidPairs/AddShardTagids) before AddNonStreamValues, then
commit + read-back on a sandbox tag. Documented in revision-write-path.md
(R3.1 decode section); raw artifact gitignored.

272 unit tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
This commit is contained in:
Joseph Doherty
2026-06-21 18:08:27 -04:00
parent 23798db1ef
commit 8fbb868813
3 changed files with 272 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/revision-write-path.md
+34
View File
@@ -56,6 +56,40 @@ path uses. Proto: `src/AVEVA.Historian.Client/Grpc/Protos/TransactionService.pro
matrix confirms they still ride the storage-engine pipe. The gRPC unlock here is original backfill,
not after-the-fact edits.
### R3.1 decode probe (2026-06-21): `AddNonStreamValues` reaches the server-side storage-engine console pipe
The `btInput` VTQ buffer is assembled in native C++ (`SendNonStreamedValues(batchID)` → a vtable
call after values are pooled via native `AddNonStreamedValueAsync(&HISTORIAN_VALUE2)`) and is **not
visible in any decompile** — only the 44-byte packed `HISTORIAN_VALUE2` struct is (TagKey@0,
FILETIME@4, OpcQuality@20, Type@24=7 numeric, value@33, bVersioned@41, VersionStatus@42). So the
framing was probed empirically against the live server with `grpc-nonstream-decode` (every
transaction `bCommit=false` → rolled back, nothing written; tag key from `SysTimeSec`).
**Result — the failure is NOT a buffer-format problem:** six different framings (44–54 bytes:
count-prefixed packed struct, struct-only, version+count, OS-wrapped) all returned the **identical**
`AddNonStreamValues` error, while an empty buffer returned a *different* error (`04 01 00 00 00`,
InvalidParameter). The shared error is a nested `SError` whose detail strings are decisive:
```
aahClientAccessPoint::CHistStorageConnection::StoreNonStreamValues::StoreNonStreamValues
\\.\pipe\aahStorageEngine\console,sid(<server storage-engine session GUID>)
```
So non-empty buffers get **past parameter validation into `StoreNonStreamValues`**, which routes to
the **`aahStorageEngine` console named pipe** server-side (the same storage engine as D2 — but the
gRPC *server* now holds the pipe, not the client). Because the error is identical across every
framing, the blocker is **not** the `btInput` layout — it is a **missing storage-engine console
session / tag-registration precondition** for the connection.
**Next step to finish M3 (untested):** establish the StorageService side **before**
`AddNonStreamValues` — `StorageService.OpenStorageConnection`/`OpenStorageConnection2` to open the
console session, then register the tag→storage mapping (`RegisterTags` / `AddTagidPairs` /
`AddShardTagids`), then retry `AddNonStreamValues` and finally `End(bCommit=true)` + SQL read-back on
a sandbox tag. Each of those StorageService ops has its own buffer format to RE. Raw decode artifact:
`artifacts/reverse-engineering/grpc-nonstream-decode/batch1-decode.txt` (gitignored). Probe command:
`grpc-nonstream-decode`; driver: `HistorianGrpcRevisionProbe.ProbeNonStreamedBuffersAsync` (candidate
guess-bytes live in the RE tool, not `src/`).
---
## Legacy WCF analysis (preserved — still accurate for the 2020 WCF transport)