Merge grpc-event-decompile-confirm: stock-client decompile + frame capture + event-parser fix
Three steps verifying the gRPC event read against the provided 2023 R2 client: - Decompiled the stock managed client (Archestra.Historian.GrpcClient, HistorianAccess): confirms no hidden client-side difference. The stock client is gRPC-Web/HTTP-1.1 (same transport as ours), m_metadata is gzip-only, the ClientInterceptor is a no-op, and it presents no TLS client cert. The event-query orchestration lives in the native C++ core. - Captured decrypted HTTP/1.1 frames of a native capture-event (50 rows) vs our SDK (0 rows) through a TLS-terminating tee proxy. Found the native splits services across 5 connections and runs the event query on a dedicated RetrievalService connection; tested replicating that (HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL) -> still 0 rows, so the server correlates by session handle, not connection. Topology is not the gate. - Verified the parse path against the provided client's real result buffer (50 events) and fixed a latent bug: the event-row buffer is version + rowCount + a one-time 0x1E header field then MARKERLESS rows; the parser wrongly treated 0x1E as a per-row marker and decoded only the first row of any multi-row buffer. This also affected the shipped WCF event read (identical v9 header). Fixed to a 10-byte header + markerless rows, accepting version 9 (WCF) and 11 (gRPC). The real 50-row buffer now decodes to exactly 50 events. Net: every client-side angle for the gRPC zero-rows is exhausted (payload, transport, metadata/cert, topology, data store) -> the gate is a server-internal per-connection retrieval working-set. The parse path is now verified against real 2023 R2 event data on both transports, and WCF event reads now correctly return all rows of a multi-row buffer. gRPC event-row retrieval stays auth-solved / parse-verified / retrieval-server-gated. 328 offline tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
This commit is contained in:
Joseph Doherty
@@ -370,29 +370,34 @@ live-server access reference.
|
||||
retained for future connection-level probing. This eliminates the leading hypothesis and tightens the
|
||||
conclusion: the server scopes 0 events to our connection at a layer **above** the gRPC transport.
|
||||
|
||||
2. **TLS client identity / certificate.** The native used `SecurityMode=TransportCertificate`. Determine
|
||||
whether it presents a **client certificate** the server uses to scope events (our SDK presents none —
|
||||
`AllowUntrustedServerCertificate=true`, server cert only). TEST: capture the TLS handshake (e.g.
|
||||
`SSLKEYLOGFILE` + Wireshark, or a decrypting proxy) for a native `capture-event` run and check the
|
||||
Certificate message; if a client cert is presented, replicate it.
|
||||
2. ~~**TLS client identity / certificate.**~~ **DISPROVEN 2026-06-23 (decompile + capture).** The stock
|
||||
client's `GrpcClientBase.InitializeBase` creates a bare `HttpClientHandler` and sets only
|
||||
`ServerCertificateCustomValidationCallback` — it **never adds a client certificate**. The TLS-tee
|
||||
capture (below) confirms `clientCert=none` on every native connection. So the native presents no client
|
||||
cert; this is not the gate.
|
||||
|
||||
3. **HTTP/2-level capture.** The byte[]/handle capture is RPC-payload only. Capture the actual HTTP/2
|
||||
frames (HEADERS/SETTINGS/stream IDs, connection reuse) for the native run vs ours — via a
|
||||
TLS-decrypting mitm on the loopback forward — to see any connection-level header/affinity our capture
|
||||
can't see.
|
||||
|
||||
2. **TLS client identity / certificate.** The native used `SecurityMode=TransportCertificate`. Determine
|
||||
whether it presents a **client certificate** the server uses to scope events (our SDK presents none —
|
||||
`AllowUntrustedServerCertificate=true`, server cert only). TEST: capture the TLS handshake (e.g.
|
||||
`SSLKEYLOGFILE` + Wireshark, or a decrypting proxy) for a native `capture-event` run and check the
|
||||
Certificate message; if a client cert is presented, replicate it. **Lower-probability after #1: the
|
||||
plain-HTTP/2 path presents no client cert either, yet auth + registration still succeed and the gate
|
||||
persists — so the gate is not at the TLS-identity layer the cert would affect.**
|
||||
|
||||
3. **HTTP/2-level capture.** The byte[]/handle capture is RPC-payload only. Capture the actual HTTP/2
|
||||
frames (HEADERS/SETTINGS/stream IDs, connection reuse) for the native run vs ours — via a
|
||||
TLS-decrypting mitm on the loopback forward — to see any connection-level header/affinity our capture
|
||||
can't see.
|
||||
3. ~~**HTTP/2-level / connection-frame capture.**~~ **DONE 2026-06-23 — topology difference found, tested,
|
||||
NULL.** Built a TLS-terminating tee proxy (`artifacts/.../httpcap/`, gitignored: self-signed server
|
||||
cert, forwards through the loopback tunnel, logs decrypted HTTP/1.1 + gRPC-Web both ways) and ran a
|
||||
**native `capture-event` (returns 50 rows) and our SDK diagnostic (0 rows) through the same
|
||||
proxy/upstream**. Note: the stock client is gRPC-Web/HTTP-1.1 (not HTTP/2 — `alpn` empty), so the
|
||||
capture is HTTP/1.1 framing. Findings:
|
||||
- **Connection topology differs.** The native opens **5 TLS connections, one per service** —
|
||||
`HistoryService` (ExchangeKey/OpenConnection/Register/EnsureTags), `StatusService` (×2), and
|
||||
**`RetrievalService` (the event query: GetRetrievalInterfaceVersion → StartEventQuery → GetNext →
|
||||
EndEventQuery) on its own dedicated connection**. Our SDK collapses **every service onto one
|
||||
connection**. (Matches the decompile: stock has a separate `GrpcClientBase` per service.)
|
||||
- **Framing differs** (benign): native uses `content-length` + `Expect: 100-continue`; SDK uses
|
||||
`transfer-encoding: chunked`. The server accepts both (our `StartEventQuery` returns a valid handle),
|
||||
so framing is not the gate. No extra/hidden header on either side; `clientCert=none` throughout.
|
||||
- **TESTED the topology hypothesis (`HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL=1`):** ran
|
||||
`StartEventQuery`/`GetNext`/`EndEventQuery` on a **dedicated RetrievalService connection** (no
|
||||
re-handshake, reusing the session handle — exactly mirroring native conn4), registration staying on
|
||||
the main connection. **Result: still `0B00000000001E000000` (0 rows), `QH=1063`.** Splitting the
|
||||
event query onto its own connection — the one concrete structural difference the capture revealed —
|
||||
**does not make rows flow.** So the server correlates by session handle, not by connection, and the
|
||||
topology is **not** the row-scoping gate. The `CreateHttp2`/`SPLIT_CHANNEL` switches + the
|
||||
`httpcap` proxy are retained as diagnostics.
|
||||
|
||||
4. ~~**Server-side ground truth.**~~ **ANSWERED 2026-06-23 (DISPROVES the data-scoping premise).** Via
|
||||
the SOCKS→SQL relay (read-only; `artifacts/.../sqlschema/`, gitignored), dumped the full event schema
|
||||
@@ -420,15 +425,77 @@ live-server access reference.
|
||||
`Microsoft.Data.SqlClient`; authenticate with the server's SQL login, not the domain Historian acct —
|
||||
creds in the gitignored creds file).
|
||||
|
||||
**Conclusion (after #1 disproven + #4 answered).** Three independent angles are now exhausted: client
|
||||
payload (byte-identical), transport (native HTTP/2 == gRPC-Web, both 0 rows), and data store (global,
|
||||
unscoped, 71,332 events the engine serves via INSQL but withholds from our gRPC connection). The gate is
|
||||
a **server-internal per-connection retrieval working-set** that a pure-managed client cannot reconstruct
|
||||
by matching wire bytes, transport, or data. The remaining angles (#2 client-cert, #3 HTTP/2-frame
|
||||
capture) are low-probability — #1 showed auth+registration succeed with no client cert over plain HTTP/2
|
||||
and the gate still holds. **gRPC event-row retrieval stands documented as auth-solved /
|
||||
retrieval-server-gated**; `ReadEventsAsync` over gRPC keeps the honest no-row throw, and event reads use
|
||||
the WCF transport.
|
||||
### Stock managed client decompiled (2026-06-23) — confirms no hidden client-side difference
|
||||
|
||||
Closing the gap that prior cycles left: the zero-rows conclusion had leaned on **wire capture**
|
||||
(`instrument-grpc-nonstream`, which only hooks `byte[]` params on `Grpc*Client` methods) — blind to gRPC
|
||||
metadata/headers, interceptors, channel options, and any non-`byte[]` call. Read the **stock managed
|
||||
client source directly** (`histsdk-2023r2-analysis/decompiled/Archestra.Historian.GrpcClient` +
|
||||
`HistorianAccess`; the pure-managed assemblies decompile cleanly even though the mixed-mode
|
||||
`aahClientManaged.dll` crashes ILSpy). Findings:
|
||||
|
||||
- **`GrpcClientBase.InitializeBase` builds the same channel we do.** `GrpcWebHandler((GrpcWebMode)0,
|
||||
HttpClientHandler)` with `HttpVersion = 1.1` — i.e. **the stock client speaks gRPC-Web over HTTP/1.1,
|
||||
the same transport as our SDK.** This *corrects the premise of hypothesis #1*: there was never a native
|
||||
`Grpc.Core` HTTP/2 path to differ from — the stock client that returns 50 rows is itself gRPC-Web. The
|
||||
HTTP/2 disproof's *conclusion* stands (and is reinforced: identical transport on both sides).
|
||||
- **`m_metadata` passed to every RPC (incl. `StartEventQuery`/`GetNextEventQueryResultBuffer`) is only
|
||||
`grpc-internal-encoding-request: gzip`** — exactly our header set. No connection-id, session token, or
|
||||
auth header rides in gRPC metadata. The **`ClientInterceptor` is a no-op** (`LogCall` is empty; both
|
||||
unary overloads just invoke the continuation). So the "invisible per-connection metadata/header" blind
|
||||
spot is **confirmed empty** — there is no hidden client-side identity the `byte[]` capture missed.
|
||||
- **The event-read query orchestration is genuinely not in managed code.** `CreateEventQuery` /
|
||||
`EventQuery.StartQuery` / `MoveNext` are not in the managed `HistorianAccess`; the managed
|
||||
`GrpcRetrievalClient.StartEventQuery` is a thin one-RPC stub. The query logic lives in the native
|
||||
C++/CLI `HistorianClient` core (the mixed-mode part ILSpy can't decompile) — consistent with the
|
||||
working-set being native/server-side, not a managed step we could read and replicate.
|
||||
|
||||
So **every client-controllable layer is now confirmed identical by reading the stock source**, not just
|
||||
by wire match: request bytes, transport, channel options, gRPC metadata, interceptor. The remaining
|
||||
difference is below the managed surface (native core) / server-side.
|
||||
|
||||
**Conclusion (after #1–#4 + stock client decompiled + TLS-tee capture).** Every angle is now exhausted:
|
||||
- **client payload** — byte-identical (IL capture + decompile);
|
||||
- **transport** — stock client is *also* gRPC-Web/HTTP-1.1; native HTTP/2 makes no difference, both 0 rows;
|
||||
- **client metadata/interceptor/channel** — decompiled: identical gzip-only header, no-op interceptor, no
|
||||
client cert; the TLS-tee capture confirms no hidden header and `clientCert=none`;
|
||||
- **connection topology** — the native splits services across 5 connections and queries on a dedicated
|
||||
RetrievalService connection; replicating that (`SPLIT_CHANNEL`) still returns 0 rows → the server
|
||||
correlates by session handle, not connection;
|
||||
- **data store** — global, unscoped; 71,332 events the engine serves via INSQL but withholds from our
|
||||
gRPC connection.
|
||||
|
||||
The gate is a **server-internal per-connection retrieval working-set** that a pure-managed client cannot
|
||||
reconstruct by matching wire bytes, transport, metadata, topology, or data — and the establishing logic is
|
||||
in the native `HistorianClient` C++ core, not in any decompilable managed step or observable on the wire.
|
||||
**gRPC event-row retrieval stands documented as auth-solved / retrieval-server-gated**; `ReadEventsAsync`
|
||||
over gRPC keeps the honest no-row throw, and event reads use the WCF transport. Diagnostics retained for
|
||||
any future server-side investigation: the `httpcap` TLS-tee proxy, the `CreateHttp2` / `SPLIT_CHANNEL`
|
||||
switches, the `EventReadDiagnostic` test, and the `capture-event` harness (native, returns rows).
|
||||
|
||||
### Verify the parse path against the provided client's real data (2026-06-23) — found + fixed a latent bug
|
||||
|
||||
Used the provided 2023 R2 client as an **oracle**: the `capture-event` harness returns 50 real events
|
||||
(verified live + through the `httpcap` proxy), and the `instrument-grpc-nonstream` rewrite captured the
|
||||
exact `GetNextEventQueryResultBuffer.result` buffer the stock client received — **63,192 bytes, version
|
||||
`0x0B` (11), rowCount 50** (25 `Alarm.Set` + 25 `Alarm.Clear`). Fed that real buffer through our
|
||||
`HistorianEventRowProtocol.Parse` to verify the read path decodes genuine gRPC event data, and it
|
||||
**exposed a latent parser bug**:
|
||||
|
||||
- The real row buffer is `version(2) + rowCount(4) + headerField(4, =0x1E)` then **markerless rows**
|
||||
(`rowFormat(2)=7 + filetime(8) + 8×u16 slots + compact-ascii type + propCount + props`). Our parser
|
||||
wrongly treated the one-time `0x1E` field as a **per-row marker** and re-consumed `[marker+format]`
|
||||
every row — so it parsed only the **first** row of any multi-row buffer and stopped. This is **not
|
||||
gRPC-specific**: the captured **WCF v9** buffer has the identical `0900 <rowCount> 1E000000 0700 …`
|
||||
header, so the shipped WCF event read had the same latent multi-row truncation.
|
||||
- **Fix:** read a 10-byte buffer header (skip the `0x1E` field once) and parse markerless rows; accept
|
||||
container version **9 (WCF) and 11 (gRPC)**. Verified: the real 50-row buffer now decodes to exactly 50
|
||||
events, ending cleanly at end-of-buffer (`Parse_RealStockClientCapture_DecodesAllEvents`, gated on
|
||||
`HISTORIAN_EVENT_CAPTURE_NDJSON`); plus a synthetic v11 golden test. 328 offline tests pass.
|
||||
|
||||
So the **parse path is now verified against the provided client's real event data** — the one remaining
|
||||
gap is strictly the server delivering rows to our gRPC connection (the working-set gate above). If that
|
||||
were ever opened, the decoded events would now flow through correctly on both transports.
|
||||
|
||||
**2 of 3 layers cleared** (key exchange + client key); the 3rd (token construction) is localized to a
|
||||
specific managed method, pending dnlib extraction. ExchangeKey + the v8 serializer are committed; the
|
||||
|
||||
@@ -281,9 +281,21 @@ internal sealed class HistorianGrpcEventOrchestrator
|
||||
HistorianEventFilter? filter,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
|
||||
// HTTP/2-frame capture (grpc-event-query-capture.md #3) showed the stock client runs the event
|
||||
// query on a DEDICATED RetrievalService TLS connection, separate from the HistoryService
|
||||
// connection that opened+registered the session (correlated only by the session handle); our SDK
|
||||
// collapses every service onto one connection. Opt in via HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL=1 to
|
||||
// run StartEventQuery/GetNext/EndEventQuery on their own connection (mirrors native conn4: no
|
||||
// re-handshake, just the existing handle), to test whether topology is the row-scoping gate.
|
||||
bool splitChannel = string.Equals(
|
||||
Environment.GetEnvironmentVariable("HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL"), "1", StringComparison.Ordinal);
|
||||
HistorianGrpcConnection rconn = splitChannel ? HistorianGrpcChannelFactory.Create(_options) : connection;
|
||||
try
|
||||
{
|
||||
|
||||
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(rconn.Channel);
|
||||
GrpcRetrieval.GetRetrievalInterfaceVersionResponse retrievalVersion = retrievalClient.GetRetrievalInterfaceVersion(
|
||||
new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken);
|
||||
new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), rconn.Metadata, Deadline(), cancellationToken);
|
||||
HistorianServerVersionGate.Validate(HistorianServiceInterface.Retrieval, retrievalVersion.UiVersion, _options);
|
||||
|
||||
// Version 6 envelope: the stock 2023 R2 client sends v6 (the WCF path's v5 request is accepted
|
||||
@@ -306,7 +318,7 @@ internal sealed class HistorianGrpcEventOrchestrator
|
||||
UiQueryRequestType = HistorianEventQueryProtocol.QueryRequestTypeEvent,
|
||||
BtRequest = ByteString.CopyFrom(requestBuffer)
|
||||
},
|
||||
connection.Metadata,
|
||||
rconn.Metadata,
|
||||
Deadline(),
|
||||
cancellationToken);
|
||||
|
||||
@@ -331,7 +343,7 @@ internal sealed class HistorianGrpcEventOrchestrator
|
||||
{
|
||||
nextResponse = retrievalClient.GetNextEventQueryResultBuffer(
|
||||
new GrpcRetrieval.GetNextEventQueryResultBufferRequest { UiHandle = session.ClientHandle, UiQueryHandle = queryHandle },
|
||||
connection.Metadata,
|
||||
rconn.Metadata,
|
||||
EventPollDeadline(),
|
||||
cancellationToken);
|
||||
}
|
||||
@@ -384,7 +396,12 @@ internal sealed class HistorianGrpcEventOrchestrator
|
||||
}
|
||||
finally
|
||||
{
|
||||
EndEventQuerySafely(retrievalClient, connection, session.ClientHandle, queryHandle);
|
||||
EndEventQuerySafely(retrievalClient, rconn, session.ClientHandle, queryHandle);
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
if (splitChannel) { rconn.Dispose(); }
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -10,10 +10,10 @@ namespace AVEVA.Historian.Client.Wcf;
|
||||
/// native event read (instrument-wcf-readmessage record 24, two rows for Alarm.Set + Alarm.Clear):
|
||||
///
|
||||
/// <code>
|
||||
/// UInt16 version = 9
|
||||
/// UInt16 version = 9 (WCF) | 11 (2023 R2 gRPC)
|
||||
/// UInt32 rowCount
|
||||
/// UInt32 headerField = 0x1E // ONE buffer-level field (NOT a per-row marker)
|
||||
/// rowCount × Row {
|
||||
/// UInt32 rowMarker = 0x1E
|
||||
/// UInt16 rowFormat = 7
|
||||
/// Int64 eventTimeUtcFiletime
|
||||
/// UInt16 × 8 // purpose unclear (slot offsets?)
|
||||
@@ -42,10 +42,23 @@ namespace AVEVA.Historian.Client.Wcf;
|
||||
internal static class HistorianEventRowProtocol
|
||||
{
|
||||
public const ushort EventRowProtocolVersion = 9;
|
||||
public const uint RowMarker = 0x0000001Eu;
|
||||
public const ushort RowFormatV9 = 7;
|
||||
private const int HeaderSize = 6;
|
||||
private const int RowFixedHeaderSize = 4 + 2 + 8 + 16;
|
||||
|
||||
/// <summary>
|
||||
/// 2023 R2 gRPC returns the event-row buffer with container version <c>11</c> instead of the
|
||||
/// 2020 WCF <c>9</c>. The row layout is otherwise <b>byte-identical</b> (verified against a captured
|
||||
/// stock-client read: header <c>0B00 <rowCount> 1E000000</c> then markerless rows, 50
|
||||
/// Alarm.Set/Alarm.Clear rows decoded clean to end-of-buffer; the WCF v9 capture has the same
|
||||
/// <c>0900 <rowCount> 1E000000</c> header). Accept both, exactly as the interface-version gate
|
||||
/// accepts History 11 and 12.
|
||||
/// </summary>
|
||||
public const ushort EventRowProtocolVersionGrpc = 11;
|
||||
|
||||
/// <summary>Constant buffer-level field following <c>rowCount</c> (observed <c>0x1E</c>). NOT a
|
||||
/// per-row marker — it appears exactly once, before the first row.</summary>
|
||||
public const uint BufferHeaderField = 0x0000001Eu;
|
||||
public const ushort RowFormat = 7;
|
||||
private const int BufferHeaderSize = 2 + 4 + 4; // version + rowCount + headerField
|
||||
private const int RowFixedHeaderSize = 2 + 8 + 16; // rowFormat + filetime + 8×UInt16 slots
|
||||
|
||||
private const byte ValueTypeBool = 0x02;
|
||||
private const byte ValueTypeGuid = 0x10;
|
||||
@@ -55,25 +68,26 @@ internal static class HistorianEventRowProtocol
|
||||
|
||||
public static IReadOnlyList<HistorianEvent> Parse(ReadOnlySpan<byte> buffer)
|
||||
{
|
||||
if (buffer.Length < HeaderSize)
|
||||
if (buffer.Length < 6)
|
||||
{
|
||||
return [];
|
||||
}
|
||||
|
||||
ushort version = BinaryPrimitives.ReadUInt16LittleEndian(buffer[..2]);
|
||||
if (version != EventRowProtocolVersion)
|
||||
if (version != EventRowProtocolVersion && version != EventRowProtocolVersionGrpc)
|
||||
{
|
||||
return [];
|
||||
}
|
||||
|
||||
uint rowCount = BinaryPrimitives.ReadUInt32LittleEndian(buffer.Slice(2, 4));
|
||||
if (rowCount == 0)
|
||||
if (rowCount == 0 || buffer.Length < BufferHeaderSize)
|
||||
{
|
||||
return [];
|
||||
}
|
||||
|
||||
List<HistorianEvent> events = new(checked((int)rowCount));
|
||||
int cursor = HeaderSize;
|
||||
// Skip the single buffer-level header field (0x1E); rows follow with NO per-row marker.
|
||||
int cursor = BufferHeaderSize;
|
||||
for (uint rowIndex = 0; rowIndex < rowCount; rowIndex++)
|
||||
{
|
||||
if (!TryReadRow(buffer, ref cursor, out HistorianEvent? row))
|
||||
@@ -95,19 +109,13 @@ internal static class HistorianEventRowProtocol
|
||||
return false;
|
||||
}
|
||||
|
||||
uint marker = BinaryPrimitives.ReadUInt32LittleEndian(buffer.Slice(cursor, 4));
|
||||
if (marker != RowMarker)
|
||||
ushort format = BinaryPrimitives.ReadUInt16LittleEndian(buffer.Slice(cursor, 2));
|
||||
if (format != RowFormat)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
ushort format = BinaryPrimitives.ReadUInt16LittleEndian(buffer.Slice(cursor + 4, 2));
|
||||
if (format != RowFormatV9)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
long filetime = BinaryPrimitives.ReadInt64LittleEndian(buffer.Slice(cursor + 6, 8));
|
||||
long filetime = BinaryPrimitives.ReadInt64LittleEndian(buffer.Slice(cursor + 2, 8));
|
||||
DateTime eventTimeUtc = DateTime.FromFileTimeUtc(filetime);
|
||||
int afterFixedHeader = cursor + RowFixedHeaderSize;
|
||||
|
||||
|
||||
@@ -90,6 +90,69 @@ public sealed class HistorianEventRowProtocolTests
|
||||
Assert.Equal(500, evt.Properties["priority"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Parse_Version11GrpcHeader_ParsesRowsIdenticalToV9()
|
||||
{
|
||||
// 2023 R2 gRPC returns the event-row buffer with container version 11; the per-row layout is
|
||||
// byte-identical to the WCF v9 format. The parser must accept both (verified against a captured
|
||||
// stock-client read of 50 Alarm.Set/Alarm.Clear rows whose header began 0B00 .. 1E000000 0700).
|
||||
DateTime t1 = new(2026, 6, 23, 13, 34, 14, DateTimeKind.Utc);
|
||||
DateTime t2 = t1.AddSeconds(10);
|
||||
|
||||
byte[] header = BuildHeader(2u, HistorianEventRowProtocol.EventRowProtocolVersionGrpc); // version 11
|
||||
byte[] buffer = Concat(header, BuildRow(t1, "Alarm.Set", []), BuildRow(t2, "Alarm.Clear", []));
|
||||
IReadOnlyList<HistorianEvent> events = HistorianEventRowProtocol.Parse(buffer);
|
||||
|
||||
Assert.Equal(2, events.Count);
|
||||
Assert.Equal("Alarm.Set", events[0].Type);
|
||||
Assert.Equal(t1, events[0].EventTimeUtc);
|
||||
Assert.Equal("Alarm.Clear", events[1].Type);
|
||||
Assert.Equal(t2, events[1].EventTimeUtc);
|
||||
}
|
||||
|
||||
// Verification against the PROVIDED 2023 R2 client: parse the real GetNextEventQueryResultBuffer
|
||||
// result the stock client received (50 events), proving our read path decodes genuine gRPC event
|
||||
// data. The capture carries customer identity so it is gitignored — point HISTORIAN_EVENT_CAPTURE_NDJSON
|
||||
// at the captured ndjson to run; the test skips cleanly otherwise (no fixture committed).
|
||||
[Fact]
|
||||
public void Parse_RealStockClientCapture_DecodesAllEvents()
|
||||
{
|
||||
string? ndjson = Environment.GetEnvironmentVariable("HISTORIAN_EVENT_CAPTURE_NDJSON");
|
||||
if (string.IsNullOrWhiteSpace(ndjson) || !File.Exists(ndjson))
|
||||
{
|
||||
return; // gated: no capture available
|
||||
}
|
||||
|
||||
byte[]? resultBuffer = null;
|
||||
foreach (string line in File.ReadLines(ndjson))
|
||||
{
|
||||
if (!line.Contains("GetNextEventQueryResultBuffer.result.out")) continue;
|
||||
int i = line.IndexOf("\"Base64\":\"", StringComparison.Ordinal);
|
||||
if (i < 0) continue;
|
||||
i += "\"Base64\":\"".Length;
|
||||
int j = line.IndexOf('"', i);
|
||||
resultBuffer = Convert.FromBase64String(line.Substring(i, j - i));
|
||||
break;
|
||||
}
|
||||
|
||||
Assert.NotNull(resultBuffer);
|
||||
ushort version = BinaryPrimitives.ReadUInt16LittleEndian(resultBuffer.AsSpan(0, 2));
|
||||
uint rowCount = BinaryPrimitives.ReadUInt32LittleEndian(resultBuffer.AsSpan(2, 4));
|
||||
Assert.Equal(HistorianEventRowProtocol.EventRowProtocolVersionGrpc, version); // real gRPC buffer is v11
|
||||
|
||||
IReadOnlyList<HistorianEvent> events = HistorianEventRowProtocol.Parse(resultBuffer);
|
||||
|
||||
// Our parser decodes every row the stock client received.
|
||||
Assert.Equal((int)rowCount, events.Count);
|
||||
Assert.All(events, e =>
|
||||
{
|
||||
Assert.False(string.IsNullOrEmpty(e.Type));
|
||||
Assert.NotEqual(default, e.EventTimeUtc);
|
||||
});
|
||||
// Sanitized cross-check: only the generic AVEVA event types (no customer fields asserted).
|
||||
Assert.All(events, e => Assert.Contains(e.Type, new[] { "Alarm.Set", "Alarm.Clear" }));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Parse_UnknownTypeMarker_KeepsRawBytesInPropertyBag()
|
||||
{
|
||||
@@ -120,11 +183,15 @@ public sealed class HistorianEventRowProtocolTests
|
||||
Assert.Equal("Alarm.Set", events[0].Type);
|
||||
}
|
||||
|
||||
private static byte[] BuildHeader(uint rowCount)
|
||||
private static byte[] BuildHeader(uint rowCount) => BuildHeader(rowCount, HistorianEventRowProtocol.EventRowProtocolVersion);
|
||||
|
||||
private static byte[] BuildHeader(uint rowCount, ushort version)
|
||||
{
|
||||
byte[] header = new byte[6];
|
||||
BinaryPrimitives.WriteUInt16LittleEndian(header.AsSpan(0, 2), HistorianEventRowProtocol.EventRowProtocolVersion);
|
||||
// version(2) + rowCount(4) + the single buffer-level header field (0x1E). Rows are markerless.
|
||||
byte[] header = new byte[10];
|
||||
BinaryPrimitives.WriteUInt16LittleEndian(header.AsSpan(0, 2), version);
|
||||
BinaryPrimitives.WriteUInt32LittleEndian(header.AsSpan(2, 4), rowCount);
|
||||
BinaryPrimitives.WriteUInt32LittleEndian(header.AsSpan(6, 4), HistorianEventRowProtocol.BufferHeaderField);
|
||||
return header;
|
||||
}
|
||||
|
||||
@@ -141,13 +208,13 @@ public sealed class HistorianEventRowProtocolTests
|
||||
propertyBlockSize += propertyBlocks[i].Length;
|
||||
}
|
||||
|
||||
byte[] row = new byte[4 + 2 + 8 + 16 + eventTypeBytes.Length + 2 + propertyBlockSize];
|
||||
// Markerless row: rowFormat(2) + filetime(8) + 8×UInt16 slots(16) + type + propCount + props.
|
||||
byte[] row = new byte[2 + 8 + 16 + eventTypeBytes.Length + 2 + propertyBlockSize];
|
||||
Span<byte> span = row;
|
||||
BinaryPrimitives.WriteUInt32LittleEndian(span[..4], HistorianEventRowProtocol.RowMarker);
|
||||
BinaryPrimitives.WriteUInt16LittleEndian(span.Slice(4, 2), HistorianEventRowProtocol.RowFormatV9);
|
||||
BinaryPrimitives.WriteInt64LittleEndian(span.Slice(6, 8), eventTimeUtc.ToFileTimeUtc());
|
||||
BinaryPrimitives.WriteUInt16LittleEndian(span[..2], HistorianEventRowProtocol.RowFormat);
|
||||
BinaryPrimitives.WriteInt64LittleEndian(span.Slice(2, 8), eventTimeUtc.ToFileTimeUtc());
|
||||
// 16 bytes of zeroed slot ushorts left as-is.
|
||||
int eventTypeOffset = 4 + 2 + 8 + 16;
|
||||
int eventTypeOffset = 2 + 8 + 16;
|
||||
eventTypeBytes.CopyTo(span[eventTypeOffset..]);
|
||||
BinaryPrimitives.WriteUInt16LittleEndian(span.Slice(eventTypeOffset + eventTypeBytes.Length, 2), propertyCount);
|
||||
int cursor = eventTypeOffset + eventTypeBytes.Length + 2;
|
||||
|
||||
Reference in New Issue
Block a user