From b05063b19526c46ea94a9cf6c00f98243fc6db38 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Mon, 4 May 2026 15:35:48 -0400 Subject: [PATCH] Document the trailing 35 bytes of GetNextQueryResultBuffer rows MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Investigation finding only — no behavior change, no new public fields. Captured a fresh GetNextQueryResultBufferResponse for SysTimeSec via instrument-wcf-readmessage and compared against the canonical 4-row OtOpcUaParityTest_001.Counter fixture. Trailing-block structure is tag-independent: bytes 0-2 constant 0x00 0x00 0x01 (sample-format marker) bytes 3-10 Int64 FILETIME UTC (duplicate of startTime for raw rows; already used by the aggregate parser as the interval start) bytes 11-18 zeros (likely end-time slot — populated by aggregate variants) bytes 19-26 varies row-to-row even with identical Quality/Value; looks like a storage block sequence ID or snapshot offset bytes 27,29 flag bytes (0/1 and 0/4 observed); semantics undecoded bytes 28, 30-34 zeros None of bytes 19-34 have a clear user-facing meaning; they appear to be server-internal storage metadata. Updated the TryParseGetNextQueryResultBufferRows remarks block with the byte map and a note that surfacing them as new HistorianSample fields should wait until a customer actually asks. CLAUDE.md "Remaining gaps" entry updated to reflect the new partial decode. Co-Authored-By: Claude Opus 4.7 (1M context) --- CLAUDE.md | 2 +- .../Wcf/HistorianDataQueryProtocol.cs | 22 +++++++++++++++---- 2 files changed, 19 insertions(+), 5 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 4e3939b..314f9c8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -95,7 +95,7 @@ Smaller, isolated items — none block the production read surface: - Remote TCP transports (`RemoteTcpIntegrated`, `RemoteTcpCertificate`) untested against an actual remote Historian (tests skip without `HISTORIAN_REMOTE_TCP_HOST`). - Explicit username/password tag-metadata path is wired (validator only blocks no-auth-at-all), but live-verification requires `HISTORIAN_USER`+`HISTORIAN_PASSWORD` set; gated test `GetTagMetadataAsync_ExplicitCredentials_AgainstLocalHistorian` skips otherwise. -- Per-row trailing ~24 bytes of `GetNextQueryResultBuffer` are not decoded (likely per-sample value/source/state metadata). +- Per-row trailing 35 bytes of `GetNextQueryResultBuffer` are now mapped (see `HistorianDataQueryProtocol.TryParseGetNextQueryResultBufferRows` doc comment) — bytes 3-10 = duplicate FILETIME (already used by aggregate parser), bytes 0-2 + 19-34 = server-internal sample/storage metadata with no clear user-facing meaning. No new public fields added; revisit if a customer asks for storage metadata exposure. - `EnsureTagAsync` distinct `MinRaw`/`MaxRaw` persistence requires `ApplyScaling=true` + a follow-up `UpdateTags` call — not yet wired (no API user has asked). ### Tools Layer diff --git a/src/AVEVA.Historian.Client/Wcf/HistorianDataQueryProtocol.cs b/src/AVEVA.Historian.Client/Wcf/HistorianDataQueryProtocol.cs index 64ad32d..f512643 100644 --- a/src/AVEVA.Historian.Client/Wcf/HistorianDataQueryProtocol.cs +++ b/src/AVEVA.Historian.Client/Wcf/HistorianDataQueryProtocol.cs @@ -18,10 +18,24 @@ internal static class HistorianDataQueryProtocol /// the canonical OtOpcUaParityTest_001.Counter capture, 4 rows × 141 bytes inside a 570-byte body): /// header is UInt16 version=9 + UInt32 rowCount; each row is UInt32 tagKey + UInt32 tagNameLen + /// (tagNameLen × 2) UTF-16 chars + UInt32 sampleCount + Int64 startUtc FILETIME + UInt32 quality + - /// UInt32 qualityDetail + UInt32 opcQuality + Double numericValue + Double percentGood + 1-byte - /// marker + 34 trailing bytes whose meaning is undecoded for raw rows. The 5-byte error/terminal - /// buffer accompanying the result decodes as `04 1E 00 00 00` = type 4, code 30 = "no more data"; - /// any other shape leaves true. + /// UInt32 qualityDetail + UInt32 opcQuality + Double numericValue + Double percentGood + 35-byte + /// trailing block. The 5-byte error/terminal buffer accompanying the result decodes as + /// `04 1E 00 00 00` = type 4, code 30 = "no more data"; any other shape leaves + /// true. + /// + /// Trailing 35 bytes (cross-tag verified 2026-05-04 against SysTimeSec — structure is + /// tag-independent, server-internal sample metadata): + /// bytes 0-2 constant 0x00 0x00 0x01 (sample-format marker) + /// bytes 3-10 Int64 FILETIME UTC — duplicate of startTime for raw rows; + /// aggregate parser reads it as the interval start (offset row+tail+43) + /// bytes 11-18 zeros (reserved — likely end-time slot, populated by aggregate variants) + /// bytes 19-26 varies row-to-row even for identical Quality/Value; likely a storage + /// block sequence ID or snapshot offset. No user-facing meaning surfaced. + /// bytes 27,29 flag bytes (0/1 and 0/4 observed); semantics undecoded + /// bytes 28, 30-34 zeros (reserved) + /// No public HistorianSample fields map to bytes 19-34 — they look like server-internal + /// storage metadata. If a customer ever needs them surfaced, capture more rows with + /// known-distinct properties (force-store, backfill, version-replace) to narrow down. /// public static bool TryParseGetNextQueryResultBufferRows( ReadOnlySpan result,