R0.1 browse over gRPC SHIPPED — QueryTag cracked, M0 gRPC parity complete

Wires HistorianClient.BrowseTagNamesAsync over gRPC (Transport==RemoteGrpc) via
Grpc/HistorianGrpcTagClient.BrowseTagNamesAsync: StartTagQuery(OData) -> paged
QueryTag -> EndTagQuery. Live-verified against a real 2023 R2 server (returns Sys* tags).

QueryTag packet-id recovered WITHOUT native disassembly: a .rdata packet-descriptor
table in aahClientManaged.dll lists {0x6751,1}=StartTagQuery immediately followed by
{0x6752,1}=QueryTag (found via pefile byte-scan of .rdata), confirmed live.

Wire format (live-verified):
- request btRequest = u16 0x6752 + u16 version(1) + u16 queryType(1=names) + u32 startIndex + u32 count
- response btResonse = u32 count + per-name(u32 charCount + UTF-16LE) + trailer (NextIndex/metadata, ignored)
- new HistorianTagQueryProtocol.ParseTagNameQueryPage tolerates the trailer
- GlobToODataFilter translates the SDK glob filter to OData (Pre*->startswith, *suf->endswith,
  *mid*->contains, exact->eq); the 2023 R2 metadata-server parses filters as OData.

Replaces the earlier RE probe helpers with the shipped browse path. Adds golden-byte
(BuildQueryTagRequest) + 8 glob-translation unit tests + gated live browse test.
226 unit tests pass; 5/5 live gRPC tests pass (read, probe, system-param, metadata, browse).

Milestone 0 (full gRPC parity) is complete.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
This commit is contained in:
Joseph Doherty
2026-06-21 16:01:15 -04:00
parent 630295bd18
commit 85ff1b48df
7 changed files with 244 additions and 115 deletions
@@ -8,7 +8,10 @@ using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// 2023 R2 gRPC tag-metadata client (roadmap item R0.2). Unlike the WCF singular
/// 2023 R2 gRPC tag-metadata + browse client (roadmap items R0.2 metadata, R0.1 browse).
/// Browse drives <c>StartTagQuery</c> (OData filter) → paged <c>QueryTag</c> → <c>EndTagQuery</c>
/// (see <see cref="BrowseTagNamesAsync"/> and <c>docs/reverse-engineering/grpc-tag-query-odata.md</c>).
/// Unlike the WCF singular
/// <c>GetTagInfoFromName</c> (a <c>uint</c>-handle op), the gRPC front door exposes the plural
/// <c>RetrievalService.GetTagInfosFromName</c> — a <b>string-handle</b> op keyed off the Open2
/// storage-session GUID (uppercase). The request <c>btTagNames</c> buffer and response
@@ -89,84 +92,84 @@ internal static class HistorianGrpcTagClient
return response.BtTagInfos?.ToByteArray() ?? [];
}
/// <summary>Outcome of a StartTagQuery probe — whether the gRPC front door accepts the op.</summary>
internal readonly record struct StartTagQueryProbeResult(bool Success, int ResponseLength, byte[] Response, int ErrorLength, byte[] Error);
// QueryTag (browse paging) request framing, recovered from the .rdata packet-descriptor table
// in aahClientManaged.dll (entries {0x6751,1}=StartTagQuery, {0x6752,1}=QueryTag) and confirmed
// live: btRequest = u16 marker(0x6752) + u16 version(1) + u16 queryType + u32 startIndex + u32 count.
private const ushort QueryTagPacketMarker = 0x6752;
private const ushort TagQueryHeaderVersion = 1;
private const ushort QueryTagModeNames = 1; // queryType 1 returns tag-name rows
private const uint BrowsePageSize = 1000;
/// <summary>
/// Reverse-engineering probe for R0.1 (browse): drives <c>RetrievalService.StartTagQuery</c> with
/// the native filter request and reports whether the 2023 R2 front door accepts it (on 2020 WCF this
/// op fails server-side on the aahMetadataServer pipe regardless of handle format).
/// Browses tag names over gRPC (roadmap item R0.1). Drives
/// <c>StartTagQuery</c> (OData filter) → paged <c>QueryTag</c> → <c>EndTagQuery</c> on the
/// RetrievalService. The 2023 R2 metadata-server parses the filter as <b>OData</b>, so the SDK's
/// glob filter is translated via <see cref="GlobToODataFilter"/>. Each QueryTag page returns
/// <c>uint count + per-name(uint charCount + UTF-16LE)</c>, decoded by
/// <see cref="HistorianTagQueryProtocol.ParseGetLikeTagNamesResponse"/>.
/// </summary>
internal static StartTagQueryProbeResult ProbeStartTagQuery(HistorianClientOptions options, string filter, CancellationToken cancellationToken)
public static async IAsyncEnumerable<string> BrowseTagNamesAsync(
HistorianClientOptions options,
string filter,
[System.Runtime.CompilerServices.EnumeratorCancellation] CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
byte[] requestBuffer = HistorianTagQueryProtocol.CreateStartTagQueryAttempt(filter).RequestBuffer;
GrpcRetrieval.StartTagQueryResponse response = retrievalClient.StartTagQuery(
new GrpcRetrieval.StartTagQueryRequest
{
StrHandle = session.StringHandle,
BtRequest = ByteString.CopyFrom(requestBuffer)
},
connection.Metadata,
DateTime.UtcNow.Add(options.RequestTimeout),
cancellationToken);
bool success = response.Status?.BSuccess ?? false;
byte[] resp = response.BtResponse?.ToByteArray() ?? [];
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
return new StartTagQueryProbeResult(success, resp.Length, resp, error.Length, error);
IReadOnlyList<string> names = await Task.Run(() => BrowseTagNames(options, filter, cancellationToken), cancellationToken).ConfigureAwait(false);
foreach (string name in names)
{
cancellationToken.ThrowIfCancellationRequested();
yield return name;
}
}
/// <summary>Outcome of a full StartTagQuery → QueryTag → EndTagQuery probe.</summary>
internal readonly record struct TagQuerySequenceResult(
bool StartSuccess, uint QueryHandle, uint TagCount,
bool QuerySuccess, byte[] QueryResponse, byte[] QueryError);
/// <summary>
/// Reverse-engineering probe for R0.1 (browse): runs the full StartTagQuery (OData filter) →
/// QueryTag (paging) → EndTagQuery sequence and returns the raw QueryTag response buffer so its
/// tag-name framing can be discovered. <paramref name="queryRequest"/> is the candidate
/// QueryTag <c>btRequest</c> buffer (format unknown — swept by the probe).
/// </summary>
internal static TagQuerySequenceResult ProbeTagQuerySequence(
HistorianClientOptions options, string odataFilter, byte[] queryRequest, CancellationToken cancellationToken)
private static List<string> BrowseTagNames(HistorianClientOptions options, string filter, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
DateTime Deadline() => DateTime.UtcNow.Add(options.RequestTimeout);
byte[] startRequest = HistorianTagQueryProtocol.CreateStartTagQueryAttempt(odataFilter).RequestBuffer;
byte[] startRequest = HistorianTagQueryProtocol.CreateStartTagQueryAttempt(GlobToODataFilter(filter)).RequestBuffer;
GrpcRetrieval.StartTagQueryResponse start = retrievalClient.StartTagQuery(
new GrpcRetrieval.StartTagQueryRequest { StrHandle = session.StringHandle, BtRequest = ByteString.CopyFrom(startRequest) },
connection.Metadata, Deadline(), cancellationToken);
if (!(start.Status?.BSuccess ?? false))
{
return new TagQuerySequenceResult(false, 0, 0, false, [], start.Status?.BtError?.ToByteArray() ?? []);
byte[] error = start.Status?.BtError?.ToByteArray() ?? [];
throw new InvalidOperationException($"gRPC StartTagQuery failed (errorLen={error.Length}).");
}
byte[] startResp = start.BtResponse?.ToByteArray() ?? [];
HistorianTagQueryStartResponse parsed = HistorianTagQueryProtocol.ParseStartTagQueryResponse(startResp);
HistorianTagQueryStartResponse parsed = HistorianTagQueryProtocol.ParseStartTagQueryResponse(start.BtResponse?.ToByteArray() ?? []);
List<string> names = new(checked((int)parsed.TagCount));
try
{
GrpcRetrieval.QueryTagResponse query = retrievalClient.QueryTag(
new GrpcRetrieval.QueryTagRequest
uint startIndex = 0;
while (names.Count < parsed.TagCount)
{
cancellationToken.ThrowIfCancellationRequested();
uint page = Math.Min(BrowsePageSize, parsed.TagCount - (uint)names.Count);
GrpcRetrieval.QueryTagResponse query = retrievalClient.QueryTag(
new GrpcRetrieval.QueryTagRequest
{
StrHandle = session.StringHandle,
UiQueryHandle = parsed.QueryHandle,
BtRequest = ByteString.CopyFrom(BuildQueryTagRequest(QueryTagModeNames, startIndex, page))
},
connection.Metadata, Deadline(), cancellationToken);
if (!(query.Status?.BSuccess ?? false))
{
StrHandle = session.StringHandle,
UiQueryHandle = parsed.QueryHandle,
BtRequest = ByteString.CopyFrom(queryRequest)
},
connection.Metadata, Deadline(), cancellationToken);
byte[] error = query.Status?.BtError?.ToByteArray() ?? [];
throw new InvalidOperationException($"gRPC QueryTag failed (errorLen={error.Length}).");
}
return new TagQuerySequenceResult(
true, parsed.QueryHandle, parsed.TagCount,
query.Status?.BSuccess ?? false,
query.BtResonse?.ToByteArray() ?? [],
query.Status?.BtError?.ToByteArray() ?? []);
IReadOnlyList<string> pageNames = HistorianTagQueryProtocol.ParseTagNameQueryPage(query.BtResonse?.ToByteArray() ?? []);
if (pageNames.Count == 0)
{
break;
}
names.AddRange(pageNames);
startIndex += (uint)pageNames.Count;
}
}
finally
{
@@ -178,6 +181,85 @@ internal static class HistorianGrpcTagClient
}
catch { /* best-effort cleanup */ }
}
return names;
}
/// <summary>Builds the QueryTag paging request: u16 marker(0x6752) + u16 version + u16 queryType + u32 startIndex + u32 count.</summary>
internal static byte[] BuildQueryTagRequest(ushort queryType, uint startIndex, uint count)
{
using MemoryStream stream = new();
using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true);
writer.Write(QueryTagPacketMarker);
writer.Write(TagQueryHeaderVersion);
writer.Write(queryType);
writer.Write(startIndex);
writer.Write(count);
return stream.ToArray();
}
/// <summary>
/// Translates the SDK's glob filter (<c>*</c> wildcard) into the OData filter the 2023 R2
/// metadata-server's <c>StartActiveTagnamesQuery</c> expects. Single-quotes are OData-escaped.
/// <list type="bullet">
/// <item><c>*</c> / empty → no filter (all tags)</item>
/// <item><c>Pre*</c> → <c>startswith(TagName,'Pre')</c></item>
/// <item><c>*suf</c> → <c>endswith(TagName,'suf')</c></item>
/// <item><c>*mid*</c> → <c>contains(TagName,'mid')</c></item>
/// <item><c>a*b</c> → <c>startswith(TagName,'a') and endswith(TagName,'b')</c></item>
/// <item><c>Exact</c> → <c>TagName eq 'Exact'</c></item>
/// </list>
/// </summary>
internal static string GlobToODataFilter(string filter)
{
if (string.IsNullOrEmpty(filter) || filter == "*")
{
return string.Empty;
}
static string Esc(string s) => s.Replace("'", "''");
bool starStart = filter.StartsWith('*');
bool starEnd = filter.EndsWith('*');
string core = filter.Trim('*');
if (core.Length == 0)
{
return string.Empty; // "**" etc.
}
if (filter.IndexOf('*') < 0)
{
return $"TagName eq '{Esc(filter)}'";
}
if (starStart && starEnd && !core.Contains('*'))
{
return $"contains(TagName,'{Esc(core)}')";
}
if (starEnd && !core.Contains('*') && !starStart)
{
return $"startswith(TagName,'{Esc(core)}')";
}
if (starStart && !core.Contains('*') && !starEnd)
{
return $"endswith(TagName,'{Esc(core)}')";
}
// Internal wildcard(s): anchor on the prefix before the first '*' and the suffix after the last.
string prefix = filter[..filter.IndexOf('*')];
string suffix = filter[(filter.LastIndexOf('*') + 1)..];
List<string> parts = [];
if (prefix.Length > 0)
{
parts.Add($"startswith(TagName,'{Esc(prefix)}')");
}
if (suffix.Length > 0)
{
parts.Add($"endswith(TagName,'{Esc(suffix)}')");
}
return parts.Count > 0 ? string.Join(" and ", parts) : string.Empty;
}
/// <summary>Builds the native tag-names request buffer: uint count + per-name(uint charCount + UTF-16LE).</summary>
@@ -98,7 +98,9 @@ public sealed class HistorianClient : IAsyncDisposable
public IAsyncEnumerable<string> BrowseTagNamesAsync(string filter = "*", CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(filter);
return HistorianWcfTagClient.BrowseTagNamesAsync(_options, filter, cancellationToken);
return _options.Transport == HistorianTransport.RemoteGrpc
? Grpc.HistorianGrpcTagClient.BrowseTagNamesAsync(_options, filter, cancellationToken)
: HistorianWcfTagClient.BrowseTagNamesAsync(_options, filter, cancellationToken);
}
public Task<HistorianTagMetadata?> GetTagMetadataAsync(string tag, CancellationToken cancellationToken = default)
@@ -102,6 +102,34 @@ internal static class HistorianTagQueryProtocol
return tagNames;
}
/// <summary>
/// Parses one page of a gRPC <c>QueryTag</c> tag-name response: <c>uint count + per-name(uint
/// charCount + UTF-16LE)</c>, then a trailing region (NextIndex + optional metadata buffer) that
/// is intentionally ignored. Unlike <see cref="ParseGetLikeTagNamesResponse"/> this tolerates the
/// trailer rather than requiring the buffer to end exactly after the names.
/// </summary>
public static IReadOnlyList<string> ParseTagNameQueryPage(ReadOnlySpan<byte> response)
{
if (response.Length < 4)
{
return [];
}
int cursor = 0;
uint count = ReadUInt32(response, ref cursor);
List<string> tagNames = new(checked((int)count));
for (uint index = 0; index < count; index++)
{
uint charLength = ReadUInt32(response, ref cursor);
int byteLength = checked((int)charLength * 2);
EnsureAvailable(response, cursor, byteLength);
tagNames.Add(Encoding.Unicode.GetString(response.Slice(cursor, byteLength)));
cursor += byteLength;
}
return tagNames;
}
private static void WriteHistorianString(BinaryWriter writer, string value)
{
writer.Write((uint)value.Length);