Joseph Doherty
34e352ba28
Pushed on recovering the summary query params. Findings: - Added `enum-dump` to the RE CLI (dumps a managed enum's literal members). Confirmed INSQL_QUERYTYPE / HISTORIAN_SUMMARYTYPE are value__-only in the managed metadata — their named members are native-side constants, so they can't be recovered statically. Same for CColumnNameMap.LoadColumnNameMap (column->bit built from native string/const data, not IL ldstr/ldc). - Live-probed StartQuery2 against SysTimeSec sweeping QueryType/SummaryType/ ColumnSelectorFlags. The server ACCEPTS SummaryType 1/2/4/5 (returns a valid version-9 buffer) but yields 0 rows; column flags don't change that; QueryType 15/16 don't exist. So a summary query is NOT Full+SummaryType+ flags — the config lives in the AutoSummaryParameters trailer (currently zeroed) and/or a native summary QueryType. Conclusion recorded in the plan: the request shape needs a NATIVE capture (instrument-wcf-writemessage on a real summary query), not managed-metadata recovery or blind probing. Decode targets remain located. No guessed code in src/; probe scaffolding removed. 208 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
93 lines
6.6 KiB
Markdown
93 lines
6.6 KiB
Markdown
# R1.8 / R1.9 — Analog-summary & State-summary queries (implementation plan)
|
||
|
||
**Status (2026-06-20): scoped + de-risked. Reachable on 2020 WCF. Decode targets located in
|
||
the native dll. Ready to implement; not yet started in `src/` (no guessed code shipped).**
|
||
|
||
Unlike the M1 *read* items gated by the [string-handle wall](../reverse-engineering/wcf-string-handle-wall.md),
|
||
summary queries ride the **proven `uint`-handle `StartQuery2`** path — the same call the working
|
||
raw/aggregate reads use. So they are genuinely reachable here; the only work is (a) the right
|
||
request parameters and (b) decoding the summary row buffer.
|
||
|
||
## What's already in place
|
||
|
||
`HistorianDataQueryRequest` + `SerializeFullHistoryRequest`
|
||
(`Wcf/HistorianDataQueryProtocol.cs`) already serialize every field a summary query needs:
|
||
`QueryType` (INSQL_QUERYTYPE), `SummaryType` (HISTORIAN_SUMMARYTYPE), `AggregationType`,
|
||
`ColumnSelectorFlags`, `Resolution`. Normal reads send `SummaryType=0` and
|
||
`ColumnSelectorFlags=0x0000_8182_0007_82FF`. A summary query is the **same request with summary
|
||
values in those three fields**, then a different row parser on the result buffer.
|
||
|
||
## Decode targets recovered from `current/aahClientManaged.dll`
|
||
|
||
Found via `methods … Summary` + `dnlib-method`:
|
||
|
||
| Native artifact | Token | Use |
|
||
|---|---|---|
|
||
| `CAnalogSummaryValue.UnpackFromValueBuffer` | `0x06000394` | **the analog-summary row decoder** — a chain of buffer-reader calls (not literal offsets), so decode empirically against a captured buffer |
|
||
| `CAnalogSummaryValue.PackToVtq` | `0x06000395` | inverse (for a future write path) |
|
||
| `CAnalogSummaryValue` setters | `0x0600038A‑92` | wire field set: **StartDateTime, Min, Max, First, Last, ValueCount, TimeGood, Integral, IntegralOfSquares** |
|
||
| `CAnalogSummaryStruct` setters | `0x06000369‑77` | fuller field set: adds **MinDateTime, MaxDateTime, FirstDateTime, LastDateTime, FirstNullDateTime, LastNullFlag, LinearIntegral** |
|
||
| `CStateSummaryStruct` setters | `0x0600039B‑A0` | **state-summary fields: MinContained, MaxContained, TotalContained, PartialStart, PartialEnd, StateEntryCount** |
|
||
| `QueryColumnSelector.SelectAnalogSummaryColumns` | `0x0600004B` | builds `ColumnSelectorFlags` for analog summary via `CColumnNameMap.GetColumnFlag(name)` per column |
|
||
| `QueryColumnSelector.SelectStateSummaryColumns` | `0x0600004C` | same, state summary |
|
||
| `QueryColumnSelector.SelectNonSummaryColumns` | `0x0600004D` | the default (matches the `0x…82FF` flags reads already send) |
|
||
| `CTypeMetadata.IsAnalogSummary` / `IsStateSummary` | `0x060001A4/A5` | server-side type gating |
|
||
| `INSQL_QUERYTYPE` / `HISTORIAN_SUMMARYTYPE` | enums `0200013F` / `02000191` | the `QueryType` / `SummaryType` values to send |
|
||
|
||
## Empirical probe results (2026-06-20, live `SysTimeSec` over `StartQuery2`)
|
||
|
||
Swept `QueryType`/`SummaryType`/`ColumnSelectorFlags` against the live 2020 server:
|
||
|
||
- `QueryType=2 (Full)`, `SummaryType ∈ {0,3,6}` → normal 109-byte version-9 data buffer.
|
||
- `QueryType=2`, `SummaryType ∈ {1,2,4,5}` → **valid version-9 buffer with 0 rows** (`09 00 00 00 00 00`).
|
||
The server **accepts** these summary types but yields no rows.
|
||
- The 0-row result is **unchanged** by `ColumnSelectorFlags` (tried default, all-bits
|
||
`0xFFFF…FFFF`, high-dword, low-48). So column flags are *not* the unlock.
|
||
- `QueryType ∈ {15,16}` → `GetNext` blocks/times out (no such INSQL_QUERYTYPE ordinal).
|
||
|
||
**Conclusion:** a summary query is *not* `Full + SummaryType + column flags`. The summary
|
||
configuration lives elsewhere in the request — almost certainly the **`AutoSummaryParameters`
|
||
trailer** (`SerializeFullHistoryRequest` currently writes it all-zero via
|
||
`WriteAutoSummaryParameters`) and/or a native summary `QueryType`. Both are **native-side
|
||
constants** (`HISTORIAN_SUMMARYTYPE` / `INSQL_QUERYTYPE` are `value__`-only in managed metadata;
|
||
`CColumnNameMap.LoadColumnNameMap` builds column→bit via native string/const data, not IL
|
||
`ldstr`/`ldc`). So they cannot be recovered from managed metadata, and blind probing of the
|
||
obvious fields returns empty.
|
||
|
||
**Therefore the right next step is a native request capture, not more probing:** drive the native
|
||
client (NativeTraceHarness / a real summary query) and capture the `pRequestBuff` bytes via the
|
||
existing `instrument-wcf-writemessage` pipeline — the same method that produced every other proven
|
||
request shape (reads, events, EnsT2). Diff that buffer against a normal Full request to read off
|
||
the exact `QueryType` + `SummaryType` + `AutoSummaryParameters` layout, then implement against it.
|
||
|
||
## Open questions (nail these next, in order)
|
||
|
||
1. **Request params.** Recover the exact `QueryType` + `SummaryType` (+ whether `ColumnSelectorFlags`
|
||
must change) for analog vs state summary. Source: decompile `INSQL_QUERYTYPE`/`HISTORIAN_SUMMARYTYPE`
|
||
enum members and `Select{Analog,State}SummaryColumns` (the `GetColumnFlag` column-name `ldstr`
|
||
operands → the flag set). The standard QueryType map (Cyclic=0 … EndBound=14) is already verified;
|
||
summary is expected to be a `SummaryType`≠0 with an existing base `QueryType`, **not** a new mode
|
||
ordinal — confirm.
|
||
2. **Row layout.** Capture a real `GetNextQueryResultBuffer2` buffer for an analog summary of a
|
||
data-bearing tag (`SysTimeSec`) over a multi-hour window with an interval, then decode against the
|
||
`CAnalogSummaryValue` field set. Likely each row = StartDateTime FILETIME + the 8 typed fields.
|
||
|
||
## Implementation steps (per the project's two-tests discipline)
|
||
|
||
1. Add request params to `HistorianDataQueryRequest` builders (a `BuildAnalogSummaryRequest` /
|
||
`BuildStateSummaryRequest` alongside `BuildAggregateQueryRequest`).
|
||
2. **Live-probe** `SysTimeSec` via a gated diagnostic; sanitize the response into
|
||
`fixtures/protocol/analog-summary/` using the CW-1 pipeline.
|
||
3. Write `TryParseGetNextQueryResultBufferAnalogSummaryRows` (+ state variant) against the fixture.
|
||
4. Public API: `ReadAnalogSummaryAsync` / `ReadStateSummaryAsync` returning new models
|
||
`HistorianAnalogSummary` (Min/Max/First/Last/Avg=Integral÷TimeGood/ValueCount/…) and
|
||
`HistorianStateSummary` (per-state contained/partial/entry-count). Reuse `RunQuery` plumbing.
|
||
5. Golden-byte test on the parser + gated live test on `localhost` (assert non-empty, fields sane).
|
||
|
||
## Why stop here this session
|
||
|
||
`UnpackFromValueBuffer` is reader-call-based (no literal offset table), so a correct parser needs a
|
||
**captured real buffer** to decode against — that's the next concrete action, not a guess. Per
|
||
project rule ("never guess wire bytes; leave throwing until evidence supports it") no summary code
|
||
was added to `src/` yet. Everything above is the evidence needed to implement directly.
|