Merge feat/r1.8-r1.9-summary-scope: summary-query scoping + native-capture finding
Scoped R1.8/R1.9 (analog/state summary). Reachable on 2020 WCF via the proven uint-handle StartQuery2; all decode targets located in aahClientManaged.dll (CAnalogSummaryValue.UnpackFromValueBuffer + struct field sets). Live probing showed the server accepts SummaryType 1/2/4/5 but returns 0 rows with the obvious params — the summary config is native-side (AutoSummaryParameters trailer / native QueryType), not recoverable from managed metadata. Next step is a native pRequestBuff capture via instrument-wcf-writemessage. Adds the enum-dump RE CLI command. No guessed code in src/; 208 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Joseph Doherty
@@ -105,8 +105,8 @@ read/browse/status surface is Windows-free and the gRPC stack is the default pat
|
||||
| ~~R1.5~~ | Extended-property **read** | `Retrieval.GetTagExtendedPropertiesFromName` | ⛔ **string-handle wall** (GetTepByNm takes `string handle`). TEP result buffer. | string-handle RE |
|
||||
| ~~R1.6~~ | Localized-property **read** | `Retrieval.GetTagLocalizedPropertiesFromName` | ⛔ **string-handle wall** (same family). | string-handle RE |
|
||||
| R1.7 | Event **filters** | filter bytes in `Retrieval.StartEventQuery` | filter predicate encoding (name/op/value) — **`uint`-handle**, reachable | R0.5 |
|
||||
| R1.8 | Analog-summary query | `Retrieval.StartQuery` (summary mode) | summary row layout — **`uint`-handle**, reachable | — |
|
||||
| R1.9 | State-summary query | `Retrieval.StartQuery` (state mode) | state-summary row layout — **`uint`-handle**, reachable | — |
|
||||
| R1.8 | Analog-summary query | `Retrieval.StartQuery` (summary mode) | summary row layout — **`uint`-handle, reachable. Scoped + decode targets located** (`CAnalogSummaryValue.UnpackFromValueBuffer`, fields Min/Max/First/Last/ValueCount/Integral/…). Plan: [`r1.8-r1.9-summary-queries.md`](r1.8-r1.9-summary-queries.md) | — |
|
||||
| R1.9 | State-summary query | `Retrieval.StartQuery` (state mode) | state-summary row layout — **`uint`-handle, reachable. Scoped** (`CStateSummaryStruct`: MinContained/MaxContained/TotalContained/PartialStart/PartialEnd/StateEntryCount). Plan: [`r1.8-r1.9-summary-queries.md`](r1.8-r1.9-summary-queries.md) | — |
|
||||
|
||||
### 1c. Bounded config writes (S–M each)
|
||||
| ID | Capability | gRPC op | Payload | Notes |
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
# R1.8 / R1.9 — Analog-summary & State-summary queries (implementation plan)
|
||||
|
||||
**Status (2026-06-20): scoped + de-risked. Reachable on 2020 WCF. Decode targets located in
|
||||
the native dll. Ready to implement; not yet started in `src/` (no guessed code shipped).**
|
||||
|
||||
Unlike the M1 *read* items gated by the [string-handle wall](../reverse-engineering/wcf-string-handle-wall.md),
|
||||
summary queries ride the **proven `uint`-handle `StartQuery2`** path — the same call the working
|
||||
raw/aggregate reads use. So they are genuinely reachable here; the only work is (a) the right
|
||||
request parameters and (b) decoding the summary row buffer.
|
||||
|
||||
## What's already in place
|
||||
|
||||
`HistorianDataQueryRequest` + `SerializeFullHistoryRequest`
|
||||
(`Wcf/HistorianDataQueryProtocol.cs`) already serialize every field a summary query needs:
|
||||
`QueryType` (INSQL_QUERYTYPE), `SummaryType` (HISTORIAN_SUMMARYTYPE), `AggregationType`,
|
||||
`ColumnSelectorFlags`, `Resolution`. Normal reads send `SummaryType=0` and
|
||||
`ColumnSelectorFlags=0x0000_8182_0007_82FF`. A summary query is the **same request with summary
|
||||
values in those three fields**, then a different row parser on the result buffer.
|
||||
|
||||
## Decode targets recovered from `current/aahClientManaged.dll`
|
||||
|
||||
Found via `methods … Summary` + `dnlib-method`:
|
||||
|
||||
| Native artifact | Token | Use |
|
||||
|---|---|---|
|
||||
| `CAnalogSummaryValue.UnpackFromValueBuffer` | `0x06000394` | **the analog-summary row decoder** — a chain of buffer-reader calls (not literal offsets), so decode empirically against a captured buffer |
|
||||
| `CAnalogSummaryValue.PackToVtq` | `0x06000395` | inverse (for a future write path) |
|
||||
| `CAnalogSummaryValue` setters | `0x0600038A‑92` | wire field set: **StartDateTime, Min, Max, First, Last, ValueCount, TimeGood, Integral, IntegralOfSquares** |
|
||||
| `CAnalogSummaryStruct` setters | `0x06000369‑77` | fuller field set: adds **MinDateTime, MaxDateTime, FirstDateTime, LastDateTime, FirstNullDateTime, LastNullFlag, LinearIntegral** |
|
||||
| `CStateSummaryStruct` setters | `0x0600039B‑A0` | **state-summary fields: MinContained, MaxContained, TotalContained, PartialStart, PartialEnd, StateEntryCount** |
|
||||
| `QueryColumnSelector.SelectAnalogSummaryColumns` | `0x0600004B` | builds `ColumnSelectorFlags` for analog summary via `CColumnNameMap.GetColumnFlag(name)` per column |
|
||||
| `QueryColumnSelector.SelectStateSummaryColumns` | `0x0600004C` | same, state summary |
|
||||
| `QueryColumnSelector.SelectNonSummaryColumns` | `0x0600004D` | the default (matches the `0x…82FF` flags reads already send) |
|
||||
| `CTypeMetadata.IsAnalogSummary` / `IsStateSummary` | `0x060001A4/A5` | server-side type gating |
|
||||
| `INSQL_QUERYTYPE` / `HISTORIAN_SUMMARYTYPE` | enums `0200013F` / `02000191` | the `QueryType` / `SummaryType` values to send |
|
||||
|
||||
## Empirical probe results (2026-06-20, live `SysTimeSec` over `StartQuery2`)
|
||||
|
||||
Swept `QueryType`/`SummaryType`/`ColumnSelectorFlags` against the live 2020 server:
|
||||
|
||||
- `QueryType=2 (Full)`, `SummaryType ∈ {0,3,6}` → normal 109-byte version-9 data buffer.
|
||||
- `QueryType=2`, `SummaryType ∈ {1,2,4,5}` → **valid version-9 buffer with 0 rows** (`09 00 00 00 00 00`).
|
||||
The server **accepts** these summary types but yields no rows.
|
||||
- The 0-row result is **unchanged** by `ColumnSelectorFlags` (tried default, all-bits
|
||||
`0xFFFF…FFFF`, high-dword, low-48). So column flags are *not* the unlock.
|
||||
- `QueryType ∈ {15,16}` → `GetNext` blocks/times out (no such INSQL_QUERYTYPE ordinal).
|
||||
|
||||
**Conclusion:** a summary query is *not* `Full + SummaryType + column flags`. The summary
|
||||
configuration lives elsewhere in the request — almost certainly the **`AutoSummaryParameters`
|
||||
trailer** (`SerializeFullHistoryRequest` currently writes it all-zero via
|
||||
`WriteAutoSummaryParameters`) and/or a native summary `QueryType`. Both are **native-side
|
||||
constants** (`HISTORIAN_SUMMARYTYPE` / `INSQL_QUERYTYPE` are `value__`-only in managed metadata;
|
||||
`CColumnNameMap.LoadColumnNameMap` builds column→bit via native string/const data, not IL
|
||||
`ldstr`/`ldc`). So they cannot be recovered from managed metadata, and blind probing of the
|
||||
obvious fields returns empty.
|
||||
|
||||
**Therefore the right next step is a native request capture, not more probing:** drive the native
|
||||
client (NativeTraceHarness / a real summary query) and capture the `pRequestBuff` bytes via the
|
||||
existing `instrument-wcf-writemessage` pipeline — the same method that produced every other proven
|
||||
request shape (reads, events, EnsT2). Diff that buffer against a normal Full request to read off
|
||||
the exact `QueryType` + `SummaryType` + `AutoSummaryParameters` layout, then implement against it.
|
||||
|
||||
## Open questions (nail these next, in order)
|
||||
|
||||
1. **Request params.** Recover the exact `QueryType` + `SummaryType` (+ whether `ColumnSelectorFlags`
|
||||
must change) for analog vs state summary. Source: decompile `INSQL_QUERYTYPE`/`HISTORIAN_SUMMARYTYPE`
|
||||
enum members and `Select{Analog,State}SummaryColumns` (the `GetColumnFlag` column-name `ldstr`
|
||||
operands → the flag set). The standard QueryType map (Cyclic=0 … EndBound=14) is already verified;
|
||||
summary is expected to be a `SummaryType`≠0 with an existing base `QueryType`, **not** a new mode
|
||||
ordinal — confirm.
|
||||
2. **Row layout.** Capture a real `GetNextQueryResultBuffer2` buffer for an analog summary of a
|
||||
data-bearing tag (`SysTimeSec`) over a multi-hour window with an interval, then decode against the
|
||||
`CAnalogSummaryValue` field set. Likely each row = StartDateTime FILETIME + the 8 typed fields.
|
||||
|
||||
## Implementation steps (per the project's two-tests discipline)
|
||||
|
||||
1. Add request params to `HistorianDataQueryRequest` builders (a `BuildAnalogSummaryRequest` /
|
||||
`BuildStateSummaryRequest` alongside `BuildAggregateQueryRequest`).
|
||||
2. **Live-probe** `SysTimeSec` via a gated diagnostic; sanitize the response into
|
||||
`fixtures/protocol/analog-summary/` using the CW-1 pipeline.
|
||||
3. Write `TryParseGetNextQueryResultBufferAnalogSummaryRows` (+ state variant) against the fixture.
|
||||
4. Public API: `ReadAnalogSummaryAsync` / `ReadStateSummaryAsync` returning new models
|
||||
`HistorianAnalogSummary` (Min/Max/First/Last/Avg=Integral÷TimeGood/ValueCount/…) and
|
||||
`HistorianStateSummary` (per-state contained/partial/entry-count). Reuse `RunQuery` plumbing.
|
||||
5. Golden-byte test on the parser + gated live test on `localhost` (assert non-empty, fields sane).
|
||||
|
||||
## Why stop here this session
|
||||
|
||||
`UnpackFromValueBuffer` is reader-call-based (no literal offset table), so a correct parser needs a
|
||||
**captured real buffer** to decode against — that's the next concrete action, not a guess. Per
|
||||
project rule ("never guess wire bytes; leave throwing until evidence supports it") no summary code
|
||||
was added to `src/` yet. Everything above is the evidence needed to implement directly.
|
||||
@@ -35,6 +35,7 @@ try
|
||||
"dnlib-method" => InspectDnlibMethod(args),
|
||||
"method-refs" => PrintMethodReferences(args),
|
||||
"field-constant" => PrintFieldConstantReferences(args),
|
||||
"enum-dump" => DumpEnumMembers(args),
|
||||
"instrument-startdataquery" => InstrumentStartDataQuery(args),
|
||||
"instrument-getnextrow" => InstrumentGetNextRow(args),
|
||||
"instrument-wcf-readquery" => InstrumentWcfReadQuery(args),
|
||||
@@ -80,6 +81,53 @@ catch (Exception ex)
|
||||
return 1;
|
||||
}
|
||||
|
||||
static int DumpEnumMembers(string[] args)
|
||||
{
|
||||
string path = args.Length > 1 ? args[1] : Path.Combine("current", "aahClientManaged.dll");
|
||||
string filter = args.Length > 2 ? args[2] : throw new ArgumentException("Usage: enum-dump [dll-path] [type-name-substring]");
|
||||
|
||||
dnlib.DotNet.ModuleDefMD module = dnlib.DotNet.ModuleDefMD.Load(path);
|
||||
int matchedTypes = 0;
|
||||
foreach (dnlib.DotNet.TypeDef type in module.GetTypes())
|
||||
{
|
||||
if (type.Name.String.IndexOf(filter, StringComparison.OrdinalIgnoreCase) < 0)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
matchedTypes++;
|
||||
Console.WriteLine($"Type: {type.FullName} (enum={type.IsEnum}, fields={type.Fields.Count})");
|
||||
foreach (dnlib.DotNet.FieldDef field in type.Fields)
|
||||
{
|
||||
object? value = field.Constant?.Value;
|
||||
if (value is null)
|
||||
{
|
||||
Console.WriteLine($" {field.Name,-40} [literal={field.IsLiteral} static={field.IsStatic} sig={field.FieldType?.TypeName}] (no const)");
|
||||
continue;
|
||||
}
|
||||
string hex = value switch
|
||||
{
|
||||
int i => $"0x{i:X}",
|
||||
uint u => $"0x{u:X}",
|
||||
short s => $"0x{s:X}",
|
||||
ushort us => $"0x{us:X}",
|
||||
long l => $"0x{l:X}",
|
||||
byte b => $"0x{b:X}",
|
||||
sbyte sb => $"0x{sb:X}",
|
||||
_ => "?"
|
||||
};
|
||||
Console.WriteLine($" {field.Name,-40} = {value} ({hex})");
|
||||
}
|
||||
}
|
||||
|
||||
if (matchedTypes == 0)
|
||||
{
|
||||
Console.WriteLine($"No type matched '{filter}'.");
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int PrintExports(string[] args)
|
||||
{
|
||||
string path = args.Length > 1 ? args[1] : Path.Combine("current", "aahClient.dll");
|
||||
|
||||
Reference in New Issue
Block a user