21 Commits

Author SHA1 Message Date
Joseph Doherty 25aff409dc Merge re/grpc-2023r2-handshake: M0 gRPC parity (probe/system-param/metadata/browse) + handshake fix 2026-06-21 16:32:02 -04:00
Joseph Doherty d23722ea73 Merge re/r1.10-rename-tags: RenameTagsAsync via History StartJob
# Conflicts:
#	docs/plans/hcal-capability-matrix.md
#	docs/plans/hcal-roadmap.md
#	src/AVEVA.Historian.Client/Wcf/HistorianWcfTagWriteOrchestrator.cs
#	tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs
#	tools/AVEVA.Historian.NativeTraceHarness/Program.cs
2026-06-21 16:31:44 -04:00
Joseph Doherty 4de222c950 Merge re/r1.4-gethi-finding: R1.1 ExecuteSqlCommand + R1.4 GetHistorianInfo (bounded)
# Conflicts:
#	docs/plans/hcal-roadmap.md
#	src/AVEVA.Historian.Client/HistorianClient.cs
#	src/AVEVA.Historian.Client/Protocol/Historian2020ProtocolDialect.cs
#	tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs
#	tools/AVEVA.Historian.NativeTraceHarness/Program.cs
2026-06-21 16:18:49 -04:00
Joseph Doherty 85ff1b48df R0.1 browse over gRPC SHIPPED — QueryTag cracked, M0 gRPC parity complete
Wires HistorianClient.BrowseTagNamesAsync over gRPC (Transport==RemoteGrpc) via
Grpc/HistorianGrpcTagClient.BrowseTagNamesAsync: StartTagQuery(OData) -> paged
QueryTag -> EndTagQuery. Live-verified against a real 2023 R2 server (returns Sys* tags).

QueryTag packet-id recovered WITHOUT native disassembly: a .rdata packet-descriptor
table in aahClientManaged.dll lists {0x6751,1}=StartTagQuery immediately followed by
{0x6752,1}=QueryTag (found via pefile byte-scan of .rdata), confirmed live.

Wire format (live-verified):
- request btRequest = u16 0x6752 + u16 version(1) + u16 queryType(1=names) + u32 startIndex + u32 count
- response btResonse = u32 count + per-name(u32 charCount + UTF-16LE) + trailer (NextIndex/metadata, ignored)
- new HistorianTagQueryProtocol.ParseTagNameQueryPage tolerates the trailer
- GlobToODataFilter translates the SDK glob filter to OData (Pre*->startswith, *suf->endswith,
  *mid*->contains, exact->eq); the 2023 R2 metadata-server parses filters as OData.

Replaces the earlier RE probe helpers with the shipped browse path. Adds golden-byte
(BuildQueryTagRequest) + 8 glob-translation unit tests + gated live browse test.
226 unit tests pass; 5/5 live gRPC tests pass (read, probe, system-param, metadata, browse).

Milestone 0 (full gRPC parity) is complete.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 16:01:15 -04:00
Joseph Doherty 630295bd18 docs: QueryTag native-RE attempt — lightweight tooling insufficient, needs Ghidra
Recorded the native-disassembly attempt on aahClientManaged.dll (mixed-mode):
ilspycmd cannot decompile it; capstone byte-search can't locate the StartTagQuery
0x6751 marker (not a plain immediate — it's an .rdata constant loaded RIP-relative,
the .text "51 67 00 00" hits are coincidental jump-table data). Managed metadata
gives QueryTag field semantics but not the binary packet-id. Finishing QueryTag
needs Ghidra/IDA xref analysis or the live IL-rewrite capture.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 15:46:54 -04:00
Joseph Doherty 4c9f0d476c docs: QueryTag error = InvalidPacketId (72); needs native aahClient.dll RE
Deepened the R0.1 browse finding. QueryTag's constant rejection decodes to
ArchestrA.CloudHistorian.Contract.ErrorCode.InvalidPacketId (72): the btRequest needs
a QueryTag-specific packet-id header (the generic 0x6751/v1 header StartTagQuery accepts
is rejected). The semantic fields are known from CloudHistorian.Contract
(QueryHandle/QueryType/StartIndex/TagCount request; TagNames[]+TagMetadataBuffer response),
but the binary packet framing lives in native aahClient.dll — aahClientManaged.dll is
mixed-mode (ilspycmd cannot decompile it) and no managed assembly builds the buffer.
Finishing QueryTag needs native RE (Ghidra/IDA) or a live gRPC capture of the stock client.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 15:16:19 -04:00
Joseph Doherty 26ef5e5645 R0.1 browse probe: StartTagQuery over gRPC takes an OData filter (live)
Probes the 2023 R2 gRPC browse path and records the finding. The front door does
NOT hit the 2020 WCF metadata-server-pipe wall.

- RetrievalService.StartTagQuery is cracked: the server (CMdServer::StartActiveTagnamesQuery
  over \.\pipe\aahMetadataServer\console) parses the filter as OData. startswith()/
  contains()/eq/empty succeed and return the 8-byte (queryHandle, tagCount); SQL-LIKE "%"
  and glob "*" fail with "ODataFilter: bad token". Live: 220 Sys* tags counted.
- QueryTag (paging) remains: every guessed btRequest returns a constant native error
  type 4 / code 72 (content-independent) -> framing needs a native capture, not guessing.

Adds RE probe helpers Grpc/HistorianGrpcTagClient.ProbeStartTagQuery + ProbeTagQuerySequence,
a gated StartTagQuery_OverGrpc_AcceptsODataFilter test, and the finding doc
docs/reverse-engineering/grpc-tag-query-odata.md. Browse is not yet wired (QueryTag open).

217 unit tests pass; 5/5 live gRPC tests pass. No tag names/identities committed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 14:58:12 -04:00
Joseph Doherty 0e19adae68 gRPC M0 R0.2: tag metadata over gRPC (GetTagInfosFromName, live-verified)
Routes HistorianClient.GetTagMetadataAsync over gRPC when Transport==RemoteGrpc,
via the new Grpc/HistorianGrpcTagClient calling RetrievalService.GetTagInfosFromName
(the plural string-handle metadata op).

- String handle = the Open2 storage-session GUID formatted uppercase (the format
  that resolves the native string-handle path); threaded out of the shared handshake
  via a new HistorianGrpcHandshake.Session { ClientHandle, StorageSessionId, StringHandle }.
- Request btTagNames = uint count + per-name(uint charCount + UTF-16LE) — golden-byte
  unit-tested (BuildTagNamesBuffer).
- Response btTagInfos = uint count + CTagMetadata records — decoded by the existing
  HistorianTagQueryProtocol.ParseGetTagInfoResponse; data type via the shared MapDataType.

The 2020 WCF string-handle wall does NOT apply on the gRPC front door, as the
string-handle-wall RE note predicted. LIVE-VERIFIED against a real 2023 R2 server:
GetTagMetadataAsync returns the requested tag with a valid decoded data type.

216 unit tests pass. Captured framing confirmed live then discarded; no tag names
or identities committed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 14:35:52 -04:00
Joseph Doherty b0703ebf80 docs: R0.3 live-verified; correct the auth-blocker note (harness quote bug)
R0.3 system-param over gRPC is now LIVE-VERIFIED against the real 2023 R2 server
(returned HistorianVersion), alongside the re-confirmed read chain and probe.

The apparent NTLM round-1 SEC_E_LOGON_DENIED "blocker" was a test-harness
credential-parsing bug, not a server/account/SDK issue: the gitignored creds
file stores quoted values and the env-setup must strip surrounding quotes before
exporting HISTORIAN_USER/PASSWORD. With quotes stripped, the NAM domain account
authenticates and the full chain passes. The round-failure diagnostic added
during the hunt (HistorianNativeHandshake.DescribeError) is kept.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 14:29:12 -04:00
Joseph Doherty c4b8d0dde4 gRPC M0: probe (R0.4, live-verified) + system-param (R0.3) + shared handshake
Roadmap docs/plans/hcal-roadmap.md, milestone M0 (gRPC parity for the DONE
surface). Now unblocked for live verification by a reachable 2023 R2 server.

- R0.4 Probe over gRPC: new HistorianGrpcProbe calls History/Retrieval/Status
  GetInterfaceVersion (unauthenticated). ProbeAsync routes over gRPC when
  Transport==RemoteGrpc. LIVE-VERIFIED against a real 2023 R2 server — needs no
  credentials (runs before the auth loop), so it works despite the auth blocker.

- R0.3 System parameter over gRPC: new HistorianGrpcStatusClient calls
  StatusService.GetSystemParameter over the authenticated session; routed in the
  dialect. Built + unit-tested (request/response field mapping pinned).
  Live-verification pending an auth fix (see below).

- Extracted the proven auth handshake from HistorianGrpcReadOrchestrator into
  shared Grpc/HistorianGrpcHandshake (reused by read + status + future
  browse/metadata). Repointed the IL structural guardrail test to it.

- Diagnostics: round-failure now decodes the native server error + hex/ASCII
  preview (HistorianNativeHandshake.DescribeError). This surfaced the live auth
  blocker as SEC_E_LOGON_DENIED (0x8009030C) at NTLM round 1 — framing is correct,
  the credential did not validate. Probable cause: stale file password or NAM-domain
  NTLM restriction (Kerberos/RDP works, NTLM denied; no SPN path over the tunnel).

216 unit tests pass; live gRPC probe passes. Sanitization scan clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 13:32:04 -04:00
Joseph Doherty 22e9c5e5f8 gRPC 2023 R2: fix auth handshake op routing + accept History v12
First live-verified gRPC read against a real 2023 R2 Historian. The handshake
previously failed at round 0 (cred-independent) because the SSPI/Negotiate token
loop was routed to HistoryService.ExchangeKey. ExchangeKey is a separate
key-exchange/cert-path op, not the Negotiate loop — the token loop belongs on
StorageService.ValidateClientCredential, which kept the 2020 inBuff/outBuff token
framing the SDK's WrapValidateClientCredentialToken/TryRead helpers already build.

Captured + diffed against the recovered 2023 R2 protobuf contract and the
decompiled stock client; routing the loop to ValidateClientCredential completes
the full chain (ValidateClientCredential x N -> OpenConnection -> StartQuery ->
GetNextQueryResultBuffer) and returns rows.

- HistorianGrpcReadOrchestrator: token loop now calls
  StorageService.ValidateClientCredential(Handle, InBuff); corrected the op-map
  doc comment (was asserting the wrong ExchangeKey mapping).
- HistorianServerVersionGate: accept History interface version 12 alongside 11.
  Live server reports History=12, Retrieval=4, Storage=4; the buffers are
  byte-identical (a live read returns rows), so 12 is buffer-compatible. Retrieval
  stays pinned at 4 (matches). New AcceptedVersions() supports multi-version gates.
- New HistorianGrpcHandshakeRoutingTests: IL-level structural guardrail that
  disassembles the orchestrator (incl. lambda closures) and asserts the handshake
  invokes ValidateClientCredential and never ExchangeKey — fails if the regression
  returns.
- Updated gate tests + CLAUDE.md gRPC op-map.

240 unit tests pass on the full stack; 210 on this branch's base. The byte
payloads remain the proven 2020 protocol.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 12:34:04 -04:00
Joseph Doherty c1b1b3d23b R1.11 DelTep capture + R1.3/R1.4/R1.12/R1.13 bounded out
DelTep (extended-property delete) — wire format captured + serializer
golden-proven, but live delete is server-blocked and NOT exposed publicly:
- Captured the DelTep inBuff via a cross-session trick (harness add-tep gains
  --tep-skip-add + read-for-sync before --tep-delete; Capture-DeleteTagExtended
  Properties.ps1 / decode-del-tep-capture.py). Layout = same group framing as
  AddTEx but property-name-only (no 0x43 value) + 0x00 group trailer.
- SerializeDeleteRequest + 4 golden tests pin the server-accepted buffer.
- A decisive experiment shows SDK-added properties ARE deletable (the native
  client read-syncs and deletes one), so SDK-add is complete; the SDK's own
  DelTep is rejected by CHistStorage::DeleteTagExtendedProperties even with
  byte-identical inBuff, matching mode/handle, GetTgByNm+GetTepByNm prime, open
  channel, and 60s retries. Root cause: the native multiplexes services over one
  connection (per-connection working set); the SDK's per-service WCF channels
  don't reproduce it. Kept as documented-but-blocked internal orchestrator path;
  no public HistorianClient delete API.

Bounded out with evidence (no code; docs + roadmap + probe):
- R1.12 localized-property write — no op on 2020 (mirror of R1.6); no
  *LocalizedPropert*/TagLocalized* symbol in any current/*.dll.
- R1.13 non-analog tag create — GATED; native AddTag rejects every non-analog
  type client-side (ValidationFailed, before any WCF op): SingleByteString,
  DoubleByteString, Int1 all fail, Float works. No Discrete type in the native
  enum, no TagType setter. No wire request to capture.
- R1.3 timezone + R1.4 EventStorageMode — re-confirmed 2023R2/gRPC-only from
  the Runtime DB schema (no timezone param, no EventStorageMode anywhere) and a
  parameter-op probe (GetSystemParameter + GETRP return null/throw for every
  candidate; only HistorianVersion works).

238 unit tests pass; full solution builds with 0 warnings.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 11:26:21 -04:00
Joseph Doherty 08b950caee R1.11 AddTagExtendedPropertiesAsync: extended-property write via AddTEx
Adds user-defined extended properties to an existing tag via the 2020 WCF
AddTEx (AddTagExtendedProperties) op. Write-enabled connection + uppercase
storage-session GUID handle; reuses the write orchestrator open/priming chain.

The AddTEx inBuff is the exact inverse of the R1.5 GetTepByNm read-response
framing, so the serializer mirrors the read parser:
  uint32 groupCount + 0x01(group) + [0x09+u16+ASCII tag] + uint32 propCount
  + per prop{ 0x02 + [0x09+u16+ASCII name] + 0x43 VT_BSTR + u16 payloadLen
  + u16 charCount + UTF-16 value } + 0x01(group trailer) + 0x00(terminator).
The trailing 0x00 is required — without it inBuff is one byte short and the
server throws SErrorException in CHistStorage::AddTagExtendedProperties. The
golden fixture pins the clean inBuff the live server accepted (dumped via
AVEVA_HISTORIAN_TEP_DUMP); read-back verified via R1.5. String (0x43) values only.

Delete (DelTep) is deferred: the native DeleteTagExtendedPropertiesByName does a
client-side sync check and returns err 229 for a just-added property, so the
DelTep request never reaches the wire and its inBuff can't be captured yet.

Shipped: HistorianClient.AddTagExtendedPropertiesAsync/AddTagExtendedPropertyAsync;
HistorianTagExtendedPropertyProtocol.SerializeAddRequest; orchestrator path;
golden WcfTagExtendedPropertyWriteProtocolTests (4); gated live write/read-back test;
native-harness `add-tep` scenario + Capture-AddTagExtendedProperties.ps1 +
decode-add-tep-capture.py. Doc: wcf-add-tag-extended-properties.md. 233 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 01:43:19 -04:00
Joseph Doherty bc353df8c4 R1.10 RenameTagsAsync: async tag rename via History StartJob (StJb)
Tag rename has no dedicated WCF op — the (old,new) name batch rides the
generic History StartJob (StJb) job buffer; the server returns a job id and
applies renames asynchronously. Handle is the uppercase storage-session GUID,
Open2 in write mode; reuses the write orchestrator's open+priming chain.

jobBuffer layout (decoded + server-validated): byte[7] zero prefix + uint32
pairCount + per pair (uint32 oldCharCount + UTF-16 oldName + uint32
newCharCount + UTF-16 newName), order (old,new). The raw instrument capture
mangles the final byte with MDAS chunk markers (the R1.1 lesson), so the golden
fixture pins the CLEAN byte[] the SDK handed the channel (dumped via
AVEVA_HISTORIAN_RENAME_DUMP) — the exact buffer the live server accepted and
renamed with.

Gated server-side by the AllowRenameTags system parameter (default 0): when
disabled the native client rejects pre-wire (err 132); the managed SDK surfaces
it as StartJob=false -> Accepted=false. Enabling needs a Historian config
reload, not just a storage-engine restart.

Shipped: HistorianClient.RenameTagAsync/RenameTagsAsync -> HistorianTagRenameResult;
HistorianTagRenameProtocol; orchestrator RenameTags/SendStartJobRename; golden
WcfTagRenameProtocolTests (4, pins server-accepted buffer); gated live test
RenameTagsAsync_AgainstLocalHistorian_RenamesSandboxTag (passed end-to-end).
Native-harness `rename` scenario + Capture-RenameTags.ps1 + decode-rename-capture.py.
Doc: docs/reverse-engineering/wcf-rename-tags.md. 213 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 01:18:41 -04:00
Joseph Doherty fbd839077b R1.4 GetHistorianInfo: bounded out on 2020 WCF (named-value-only, no struct)
Captured the native HistorianAccess.GetHistorianInfo(out HistorianInfo, out err)
and decoded the wire: over 2020 WCF, GETHI is a named-value query whose only
working key is "HistorianVersion" (response ~30 bytes = the version string).
Probed 7 storage-mode key names -> all ok=False/err. The 518-byte HISTORIAN_INFO
struct + EventStorageMode@514 is the 2023R2 HCAL-native/gRPC model (confirmed
from the decompiled 2023R2 source); on 2020 the native client derives the mode
outside the WCF wire.

Version is already exposed (ProbeAsync/GetRuntimeParameterAsync), so no hollow
GetHistorianInfoAsync is shipped (same disposition as R1.3 timezone). This
completes the reachable 2020-WCF M1 read surface; remaining M1 = config writes
(gated on explicit request) or gRPC/2023R2-only items.

RE aids kept: harness `historian-info` scenario, Capture-HistorianInfo.ps1,
decode-historian-info-capture.py, and StringHandleProbeDiagnosticTests
.GETHI_CandidateInfoNames (asserts the named-value-only finding; gated).
Docs: wcf-historian-info.md (new) + roadmap/matrix/wall-doc updates. 230 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 23:42:27 -04:00
Joseph Doherty 1a539882d0 R1.1 ExecuteSqlCommandAsync (ExeC + GetR, NRBF DataTable, no BinaryFormatter)
Ship SQL command execution over the 2020 WCF aa/Retr/ExeC + aa/Retr/GetR ops:
HistorianClient.ExecuteSqlCommandAsync(sql) -> HistorianSqlResult (columns +
typed rows). String-handle ops reached with the Open2 storage-session GUID
formatted uppercase (the handle format that unlocked GETRP/GETHI).

Chain: Retr.GetV prime -> ExeC(handle, sql, option=0, ref queryHandle) ->
GetR loop. Key gotcha captured: GetR returns FALSE even on success -- the byte
stream is in pResultBuff regardless; false just signals the final page. So the
orchestrator consumes the buffer first, then stops on a false result / empty page.

GetR's pResultBuff is an NRBF-serialized System.Data.DataTable
(SerializationFormat.Xml: members XmlSchema (XSD) + XmlDiffGram (rows)).
BinaryFormatter is removed from .NET 10, so the stream is decoded read-only with
the System.Formats.Nrbf package (NrbfDecoder) + XDocument -- no BinaryFormatter,
no code execution. Values are typed per the XSD type, falling back to string.

Adds: HistorianSqlResult / HistorianSqlColumn / HistorianSqlExecuteOption models,
HistorianSqlResultProtocol (NRBF + diffgram parser), HistorianWcfSqlClient
(ExeC/GetR orchestration with an AVEVA_HISTORIAN_SQL_DUMP diagnostic), dialect +
public API. Golden WcfSqlResultProtocolTests pinned to the real clean GetR stream
for the benign "SELECT 1 AS ProbeValue" (no sensitive data); gated live tests
(single cell + multi-column/multi-row/NULL). Doc: wcf-exec-sql.md; roadmap R1.1
DONE; wall doc + memory updated (incl. the QTB-server-side nuance). 229 tests green.

Note: a raw instrument-wcf capture corrupts a large pResultBuff with MDAS
transport chunk markers (0x9F); the clean contract-level byte[] is dumped via the
AVEVA_HISTORIAN_SQL_DUMP env var for the golden fixture.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 23:16:06 -04:00
Joseph Doherty 108220c36b R1.5 GetTagExtendedPropertiesAsync (GetTepByNm) + R1.6 closed (no op)
Ship tag extended-property reads over the 2020 WCF aa/Retr/GetTepByNm op:
HistorianClient.GetTagExtendedPropertiesAsync(tag) -> name/value pairs.

String-handle op reached with the Open2 storage-session GUID formatted
uppercase (same format that unlocked GETRP/GETHI/ExeC). Routed via the
name-based native path (GetTagExtendedPropertiesByName, server-fetch flag),
not the index-based TagQuery path.

Evidence-backed findings from the capture:
- GetTepByNm (and GetTgByNm) succeed with the uppercase handle -- further
  validates the resolved string-handle wall.
- QTB (StartTagQuery) does NOT punch through: captured uppercase, it still
  fails server-side (CMdServer::StartActiveTagnamesQuery over the
  aahMetadataServer pipe) -- a metadata-server blocker, not handle format.
- R1.6 (localized properties) has NO distinct op (only error-message/UI-text
  localization in the managed client); collapses into R1.5. Closed, not throwing.

Wire format (golden-pinned, synthetic bytes -- no dev tag names committed):
- request tagNames = uint count + per-name(uint charCount + UTF-16)
- response = uint tagCount + per-tag(marker + compact-ASCII name +
  uint propCount + per-prop(marker + compact-ASCII name + 0x43 VT_BSTR value)
  + trailer); sequence-paged.

Adds: HistorianTagExtendedProperty model, HistorianTagExtendedPropertyProtocol
(codec), HistorianWcfTagExtendedPropertyClient (orchestration), dialect +
public API; golden WcfTagExtendedPropertyProtocolTests (4) + gated live test
(HISTORIAN_TEP_TAG). Tooling: Capture-TagExtendedProperties.ps1,
decode-tag-properties-capture.py, harness tag-extended-properties scenario.
Docs: wcf-tag-extended-properties.md; roadmap R1.5 DONE / R1.6 collapsed;
wall doc + memory updated with the QTB-server-side nuance. 228 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 22:52:07 -04:00
Joseph Doherty 4da5287d01 R1.2 GetRuntimeParameter + string-handle wall RESOLVED (handle-format bug)
Execute HCAL roadmap R1.2 (GetRuntimeParameterAsync) end-to-end, and in doing so
discover that the "string-handle wall" blocking R1.1/R1.4/R1.5/R1.6 was a handle
FORMAT bug, not a missing native session/filter registration.

R1.2 (shipped, live-verified):
- Captured native GetRuntimeParameter -> WCF op aa/Stat/GETRP (string-handle op,
  GETHI's shape), via scripts/Capture-RuntimeParam.ps1 + instrument-wcf-{write,read}message.
- HistorianRuntimeParameterProtocol serializes pRequestBuff (54 67 01 00 + uint
  nameCount + per-name uint charCount + UTF-16) and parses pResponseBuff (version +
  uint resultCount + CRetVariant 0x43 VT_BSTR + uint16 len + uint16 charCount + UTF-16).
- IStatusServiceContract2.GetRuntimeParameter (GETRP) op; HistorianWcfStatusClient
  passes the Open2 storage-session GUID as the string handle, UPPERCASE.
- Public HistorianClient.GetRuntimeParameterAsync(name) via the dialect.
- Golden WcfRuntimeParameterProtocolTests + gated live test; returns HistorianVersion.

String-handle wall RESOLVED (proven, public APIs deferred):
- The Open2 storage GUID works as the string handle when sent UPPERCASE
  (ToString("D").ToUpperInvariant()); earlier "blocked" probes used lowercase.
- Live-probed GETHI (R1.4) -> returns data; ExeC (R1.1) -> Retr.GetV prime -> ExeC ->
  GetR returns a BinaryFormatter-serialized .NET DataTable. Gated
  StringHandleProbeDiagnosticTests + scripts/Capture-ExecSql.ps1 + exec-sql harness scenario.
- Docs flipped: wcf-string-handle-wall.md RESOLVED banner; roadmap R1.1/R1.4 reachable,
  R1.5/R1.6 likely; wcf-status-localhost.md GETRP section.
- R1.1/R1.4 public APIs NOT shipped: ExeC needs a GetR paging loop + a BinaryFormatter-
  stream parser (BinaryFormatter is removed from .NET 10); GETHI full-info struct needs
  its own capture.

223 unit tests pass; gated live tests green against the local 2020 Historian.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 22:10:31 -04:00
Joseph Doherty 6d470eab4a R1.7: server-side event filters — ReadEventsAsync(HistorianEventFilter), live-honored
Roadmap M1 R1.7. Filters are set on the native EventQuery object via
AddEventFilter(property, HistorianComparisionType, value) — NOT EventQueryArgs
(time/count/order only). Found via a new harness --dump-type-members command.

Captured the native filtered StartEventQuery pRequestBuff (Capture-EventFilter.ps1 +
harness --event-filter knob) and diffed Equal(0) vs Contains(12) to isolate the
operator field. Filter block (decoded byte-for-byte):
  ushort 0 + uint filterCount + uint condCount + uint nameLen + name(UTF-16) +
  uint 1 + ushort op + uint 1 + value(0x09-LEN-0x00 compact-ASCII) + byte 0

The filter is REAL, not inert (unlike the analog-summary knobs): a non-matching
predicate returns 0 events; Type=Equal=User.Write returns only User.Write events.
Verified live via both the native harness and the SDK.

- HistorianClient.ReadEventsAsync(start, end, HistorianEventFilter, ct) overload
- HistorianEventFilter + HistorianEventComparison (18 ops, ordinals = native)
- Filter encoding in HistorianEventQueryProtocol (empty-filter path unchanged)
- Golden-byte tests (block match, op field, empty-filter regression) + gated live test

Single string-valued predicate only; multi-filter (OR) / multi-condition (AND via
AddEventFilterCondition) framing is partially captured and not shipped. 216 unit
tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 18:32:03 -04:00
Joseph Doherty f1e23a3a02 M2: implement SendEventAsync — event-send rides WCF AddS2, not the storage pipe
Roadmap Milestone 2 (event sending). Capture disproved the assumption that event
delivery uses the non-WCF storage-engine pipe (which would block it like revision
writes): a native AddStreamedValue(HistorianEvent) leaves over WCF as AddS2
(IHistoryServiceContract2.AddStreamValues2). CM_EVENT is a built-in registered tag,
so the 129 TagNotFoundInCache gate that blocks AddS2 for user tags does not apply.

- R2.1: NativeTraceHarness "event-send" scenario + Capture-EventSend.ps1; two
  captures diffed to separate constant framing from value-dependent fields.
- R2.2: HistorianEventWriteProtocol serializes the AddS2 pBuf (storage sample buffer
  wrapping the event VTQ) — golden-byte tested. Decoded "OS" sig + length fields +
  CM_EVENT tag id + EventTime/ReceivedTime FILETIMEs + Opc 192 + 0x118D descriptor +
  event Id + Namespace + EventType + version 5 + typed property bag.
- R2.3/R2.4: HistorianWcfEventOrchestrator.SendEventAsync (Open2 event-mode 0x501 ->
  reuse CM_EVENT RTag2/EnsT2 -> AddStreamValues2) + HistorianClient.SendEventAsync.
- R2.5: gated live test; server accepts the AddS2 (success, empty error buffer).

Server requires delivered byte[].Length == declared packet length (uint32@0x04); the
native relies on the MDAS encoder adding a pad byte, so the SDK emits an explicit
trailing 0x00 (else AddS2 rejects with "CValuStream buffer size vs packet length
mismatch"). Original events only (RevisionVersion=0) with string properties; other
property types + revision/update/delete throw ProtocolEvidenceMissingException.

Caveat (documented): accepted events are not persisted on the local dev box; the
native client behaves identically (event ingestion pipeline inactive) — not an SDK
gap. 212 unit tests pass; 16/16 event tests pass live.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 18:00:52 -04:00
Joseph Doherty 1a7519c803 RE: resolve R1.8/R1.9 analog/state summary via request+response capture
Captured the native StartQuery2 pRequestBuff and the GetNextQueryResultBuffer2
response (instrument-wcf-writemessage + chained instrument-wcf-readmessage) and
decoded both against AnalogSummaryHistory SQL ground truth. Conclusion: the rich
multi-aggregate analog/state summary struct is NOT delivered over the 2020 WCF
binary protocol — the response is the ordinary version-9 row buffer the existing
aggregate parser already handles, carrying one value per cycle selected by
RetrievalMode (QueryType 5-8), not ValueSelector (inert on this path). So
"analog summary" == the existing ReadAggregateAsync; no new src/ code warranted.

Tooling (tools/ + scripts/ only, nothing in src/):
- NativeTraceHarness: drive summary knobs via --value-selector /
  --aggregation-type / --max-states (uint16) / --filter
- Capture-SummaryRequest.ps1: repeatable instrument+stage+matrix capture,
  -WithResponse chains the ReadMessage hook
- decode-summary-capture.py: StartQuery2 request diff vs baseline
- decode-summary-response.py: response decode vs SQL ground truth

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 17:01:42 -04:00
79 changed files with 8005 additions and 158 deletions
+3
View File
@@ -29,3 +29,6 @@ Thumbs.db
# Test droppings
*.coverage
coverage.cobertura.xml
# Live 2023 R2 server credentials — never commit
wonder-sql-vd03.txt
+1 -1
View File
@@ -71,7 +71,7 @@ Three layered subsystems, intentionally decoupled so protocol parsing can be uni
- **`Wcf/`** — managed WCF/MDAS layer. The Historian uses Net.TCP on port `32568` with a custom `application/x-mdas` content type wrapping a binary SOAP 1.2 / WS-Addressing 1.0 envelope. `MdasMessageEncoder` + `MdasMessageEncodingBindingElement` implement that wrapper. `HistorianWcfBindingFactory` produces three flavors: plain MDAS, MDAS+Windows transport (used for `/Hist-Integrated`), and MDAS+certificate (used for `/HistCert`). Service paths live in `HistorianWcfServiceNames`. WCF data contracts (`Wcf/Contracts/`) are reproduced from server-side static analysis and are versioned per native interface (e.g., `IRetrievalServiceContract2..4`).
- **`Protocol/`** — binary frame layer (`HistorianFrameReader`/`Writer`, `HistorianBinaryPrimitives`, `HistorianMessageType`). `Historian2020ProtocolDialect` is the version-anchored bridge between `HistorianClient` and the frame layer; methods without sufficient evidence throw `ProtocolEvidenceMissingException` rather than guessing wire bytes.
- **`Transport/`** — pluggable `IHistorianTransport` (default: TCP). Tests inject a fake transport.
- **`Grpc/`** — 2023 R2 gRPC transport (`HistorianTransport.RemoteGrpc`). The recovered protobuf contract lives in `Grpc/Protos/*.proto` and is compiled to client stubs at build time by `Grpc.Tools`. `HistorianGrpcChannelFactory` builds a gRPC-Web/HTTP-1.1 channel (default port `32565`, optional TLS, gzip) matching the stock 2023 R2 client. `HistorianGrpcReadOrchestrator` mirrors `HistorianWcfReadOrchestrator` but over gRPC: it reuses the exact native serializers/parsers — the same Open2 buffer, SSPI/NTLM tokens, and `DataQueryRequest`/result buffers travel inside protobuf `bytes` fields. The 2020→gRPC op map: `Hist.ValCl`→`HistoryService.ExchangeKey`, `Hist.Open2`→`HistoryService.OpenConnection`, `Retr.StartQuery2`→`RetrievalService.StartQuery`, `Retr.GetNextQueryResultBuffer2`→`RetrievalService.GetNextQueryResultBuffer`. The transport-agnostic handshake (Open2 request builder + SSPI token loop + response decode) is shared via `Wcf/HistorianNativeHandshake`. **Not yet live-verified against a 2023 R2 server** — the auth handshake op (`ExchangeKey`) is the first thing to revisit if a live server rejects it; the byte payloads are the proven 2020 protocol. Gated live test: set `HISTORIAN_GRPC_HOST` (+ `HISTORIAN_TEST_TAG`, optional `HISTORIAN_GRPC_PORT`/`HISTORIAN_GRPC_TLS`/`HISTORIAN_GRPC_DNSID`).
- **`Grpc/`** — 2023 R2 gRPC transport (`HistorianTransport.RemoteGrpc`). The recovered protobuf contract lives in `Grpc/Protos/*.proto` and is compiled to client stubs at build time by `Grpc.Tools`. `HistorianGrpcChannelFactory` builds a gRPC-Web/HTTP-1.1 channel (default port `32565`, optional TLS, gzip) matching the stock 2023 R2 client. `HistorianGrpcReadOrchestrator` mirrors `HistorianWcfReadOrchestrator` but over gRPC: it reuses the exact native serializers/parsers — the same Open2 buffer, SSPI/NTLM tokens, and `DataQueryRequest`/result buffers travel inside protobuf `bytes` fields. The 2020→gRPC op map: `Hist.ValCl`→`StorageService.ValidateClientCredential` (the SSPI/Negotiate token loop), `Hist.Open2`→`HistoryService.OpenConnection`, `Retr.StartQuery2`→`RetrievalService.StartQuery`, `Retr.GetNextQueryResultBuffer2`→`RetrievalService.GetNextQueryResultBuffer`. The transport-agnostic handshake (Open2 request builder + SSPI token loop + response decode) is shared via `Wcf/HistorianNativeHandshake`. **Live-verified 2026-06-21 against a real 2023 R2 server** (interface versions History=12, Retrieval=4, Storage=4): the full read chain returns rows. NOTE: `HistoryService.ExchangeKey` is a SEPARATE key-exchange/cert-path op, NOT the Negotiate loop — an earlier revision wrongly routed the token loop there and it was rejected at round 0 regardless of credentials; the loop belongs on `StorageService.ValidateClientCredential` (which kept the 2020 inBuff/outBuff token framing). The byte payloads are the proven 2020 protocol and transfer unchanged; only the History interface integer differs (12 vs 11) and is buffer-compatible, so `VerifyServerInterfaceVersion=false` is currently required against a v12 server (the gate still pins History=11). Gated live test: set `HISTORIAN_GRPC_HOST` (+ `HISTORIAN_TEST_TAG`, optional `HISTORIAN_GRPC_PORT`/`HISTORIAN_GRPC_TLS`/`HISTORIAN_GRPC_DNSID`); reach the live 2023 R2 box via [[reference_2023r2_live_server_access]].
- **`Models/`** — public DTOs and enums (`HistorianSample`, `RetrievalMode`, etc.). `HistorianDataValue` represents the discriminated value type.
`InternalsVisibleTo` exposes internals to the test assembly and the reverse-engineering tool.
+2 -2
View File
@@ -74,7 +74,7 @@ blob needs RE).
| Create string/discrete tag | `AddTag` | `History.EnsureTags` | ⬜ | GATED/BOUNDED | native AddTag rejects these types server-side; needs different metadata path |
| Delete tag(s) | `DeleteTags` | `History.DeleteTags` | ✅ | DONE | |
| Rename tag(s) | `RenameTags` | (History op) | ⬜ | BOUNDED | `AllowRenameTags` param already probed |
| Add/Delete extended properties | `AddTagExtendedProperties`, `DeleteTagExtendedPropertiesByName` | `History.AddTagExtendedProperties` / `DeleteTagExtendedProperties` | ⬜ | BOUNDED | gRPC op + TEP serialize |
| Add/Delete extended properties | `AddTagExtendedProperties`, `DeleteTagExtendedPropertiesByName` | `History.AddTagExtendedProperties` / `DeleteTagExtendedProperties` | 🟗 | BOUNDED | **Add DONE** (`AddTagExtendedPropertiesAsync`, AddTEx; inBuff = inverse of R1.5 read framing + trailing `01 00`). Delete (DelTep) deferred — native sync gate (err 229) blocks capturing its inBuff. See `wcf-add-tag-extended-properties.md` |
| Add/Delete localized properties | `AddTagLocalizedProperties`, `DeleteTagLocalizedPropertiesByName` | `History.AddTagLocalizedProperties` / `DeleteTagLocalizedProperties` | ⬜ | BOUNDED | |
## 6. Data writes — values
@@ -99,7 +99,7 @@ blob needs RE).
|---|---|---|---|---|---|
| System parameter | `GetSystemParameter` | `Status.GetSystemParameter` | ✅ | DONE | |
| Runtime parameter | `GetRuntimeParameter` | `Status.GetRuntimeParameter` | ⬜ | TRIVIAL | same shape as GetSystemParameter |
| Historian info | `GetHistorianInfo` | `Status.GetHistorianInfo` | 🟗 | BOUNDED | GETHI buffer; partially decoded (incl. EventStorageMode @ offset 514) |
| Historian info | `GetHistorianInfo` | `Status.GetHistorianInfo` | 🟗 | BOUNDED | **2020 WCF = version-only** (GETHI is a named-value query; `EventStorageMode` not on the wire). 518-byte struct + `EventStorageMode`@514 is gRPC/2023R2-only. See `wcf-historian-info.md` |
| Server timezone | `GetSystemTimeZoneInfo` | `Status.GetSystemTimeZoneName` | ⬜ | TRIVIAL | |
| Historization status | `GetHistorizationStatus` | `Status` op | ⬜ | BOUNDED | |
| Store-and-forward status | `GetStoreForwardStatus` | (push events / pull GETHI) | 🟗 | HARD | currently synthesized; real read needs duplex push or a decoded pull endpoint — see store-forward plan |
+104 -34
View File
@@ -9,15 +9,54 @@ HCAL replacement, built on the **2023 R2 gRPC transport**. Derived from
> protocol serializer/parser + golden-byte unit test + an env-gated live integration
> test against the local Historian.
## Progress (updated 2026-06-19)
## Progress (updated 2026-06-21)
- ✅ **R0.6 version gate** — `HistorianServerVersionGate` + `HistorianClientOptions.VerifyServerInterfaceVersion`;
fail-closed on connect, wired into both WCF and gRPC paths. Supported versions are
evidence-based (Hist=11, Retr=4, Trx=2; Status reachability-only), captured from the
live server. 10 unit tests.
evidence-based (Hist=11/12, Retr=4, Trx=2; Status reachability-only), captured from the
live server. History 12 (2023 R2 gRPC) accepted alongside 11 (buffer-compatible).
- ✅ **CW-1 capture pipeline** — `ProtocolCaptureSanitizer` + `ProtocolFixtureWriter` +
`capture-tag-info` CLI command; produces sanitized `fixtures/protocol/<op>/` golden files.
11 unit tests. First fixture: `get-tag-info/analog-*.json`.
- ✅ **gRPC auth handshake (read chain)** — LIVE-VERIFIED 2026-06-21 against a real 2023 R2
server: `ReadRawAsync` over `RemoteGrpc` returns rows. Token loop routes to
`StorageService.ValidateClientCredential`. Shared handshake extracted to
`Grpc/HistorianGrpcHandshake` for reuse by the status/browse/metadata paths.
- ✅ **R0.4 Probe over gRPC** — `Grpc/HistorianGrpcProbe` (History/Retrieval/Status
`GetInterfaceVersion`); `ProbeAsync` routes over gRPC when `Transport==RemoteGrpc`.
**LIVE-VERIFIED 2026-06-21** (no credentials required — runs before the auth loop).
- ✅ **R0.3 System parameter over gRPC** — `Grpc/HistorianGrpcStatusClient.GetSystemParameterAsync`
(`StatusService.GetSystemParameter`); routed in the dialect. Built + unit-tested + **LIVE-VERIFIED
2026-06-21** against a real 2023 R2 server (returned `HistorianVersion`). Code path is the proven
handshake + a single string-in/string-out RPC.
- ✅ **R0.2 Tag metadata over gRPC** — `Grpc/HistorianGrpcTagClient.GetTagMetadataAsync`
(`RetrievalService.GetTagInfosFromName`, the plural **string-handle** op). `GetTagMetadataAsync`
routes over gRPC when `Transport==RemoteGrpc`. Request `btTagNames` = `uint count + per-name(uint
charCount + UTF-16LE)` (golden-byte unit-tested); response `btTagInfos` = `uint count + CTagMetadata`
records (reuses `ParseGetTagInfoResponse`); string handle = uppercase Open2 storage GUID. The 2020
WCF string-handle wall does **not** apply on the gRPC front door (as predicted). **LIVE-VERIFIED
2026-06-21** — `GetTagMetadataAsync` returned the requested tag + a valid data type.
- ✅ **R0.1 Browse over gRPC** — DONE, **LIVE-VERIFIED 2026-06-21**.
`HistorianClient.BrowseTagNamesAsync` routes over gRPC via
`Grpc/HistorianGrpcTagClient.BrowseTagNamesAsync`: StartTagQuery(**OData** filter) → paged
**QueryTag** (`btRequest` = `u16 0x6752 + u16 1 + u16 queryType + u32 startIndex + u32 count`) →
EndTagQuery; response = `u32 count + per-name(u32 charCount + UTF-16LE) + trailer`. The SDK glob
filter is translated by `GlobToODataFilter` (`Pre*`→`startswith`, `*suf`→`endswith`, `*mid*`→
`contains`, exact→`eq`). The QueryTag packet-id `0x6752` was recovered from a `.rdata`
packet-descriptor table (`{0x6751,1}`=StartTagQuery, `{0x6752,1}`=QueryTag) — no Ghidra needed.
Golden-byte + glob unit tests + gated live test. Full finding:
`docs/reverse-engineering/grpc-tag-query-odata.md`.
> ✅ **Milestone 0 (gRPC parity) is COMPLETE** — probe, system-param, metadata, and browse all run
> over `RemoteGrpc` and are live-verified against a real 2023 R2 server, alongside the read chain.
> ℹ️ **Auth note (2026-06-21, resolved):** an apparent NTLM round-1 `SEC_E_LOGON_DENIED` blocker
> turned out to be a **test-harness credential-parsing bug**, not a server/account/SDK issue — the
> gitignored creds file stores **quoted** values (`"nam\user"`, `"pass"`), and the env-setup must
> **strip surrounding quotes** before exporting `HISTORIAN_USER`/`HISTORIAN_PASSWORD`. With quotes
> stripped, the domain account authenticates and the full read + system-param + probe chain passes
> live. The round-failure diagnostic added during the hunt is kept
> (`HistorianNativeHandshake.DescribeError` decodes the native error + hex/ASCII preview).
> ⚠️ **Live-verification constraint:** the local Historian is **2020** (WCF, port 32568) — the
> 2023 R2 gRPC endpoint (32565) is absent. M0's gRPC routing (R0.1–R0.4) can be built and
@@ -36,10 +75,19 @@ HCAL replacement, built on the **2023 R2 gRPC transport**. Derived from
> the boundary is the **handle type** (see the string-handle wall note under §1b and
> `docs/reverse-engineering/wcf-string-handle-wall.md`): **`uint`-handle ops work, `string`-handle
> ops are blocked.** GETHI/GetTepByNm were probed and confirmed blocked (not, as first guessed,
> reachable). The genuinely reachable next items on 2020 WCF are the remaining **`uint`-handle**
> ops: **R1.8/R1.9 StartQuery summary/state modes** and **R1.7 event filters** (filter bytes ride
> the proven `uint`-handle `StartEventQuery`). Everything string-handle waits on one RE target:
> the native session/filter registration.
> reachable). The reachable **`uint`-handle** items are now **DONE**: ~~R1.8/R1.9 StartQuery
> summary/state modes~~ (resolved = existing `ReadAggregateAsync`) and ~~R1.7 event filters~~
> (✅ 2026-06-20 — `ReadEventsAsync(…, HistorianEventFilter)`, live-honored). M2 event send is
> also done (✅ WCF `AddS2`). **R1.2 `GetRuntimeParameterAsync` is also done** (✅ 2026-06-20,
> `aa/Stat/GETRP`, live-verified) — notably a *string-handle* op that punches through the wall
> using the Open2 storage-session GUID as an **uppercase** string handle, which proved the
> GETHI/ExeC failures were a handle-*format* issue rather than a missing native registration.
> **Follow-up done:** R1.1 `ExecuteSqlCommandAsync` shipped; R1.5 extended-property read shipped
> (R1.6 collapsed into it — no distinct localized op). **R1.4 `GetHistorianInfo` bounded out on
> 2020 WCF** — GETHI there is a named-value query (only `HistorianVersion`); `EventStorageMode` is
> 2023R2-gRPC-only (see `wcf-historian-info.md`). Net: the **reachable 2020-WCF M1 read surface is
> complete**; what remains is config *writes* (M1c — gated on an explicit user request) and the
> gRPC/2023R2-only items (R1.3 timezone, R1.4 EventStorageMode — need a live 2023 R2 server).
## Guiding principles
@@ -85,26 +133,30 @@ read/browse/status surface is Windows-free and the gRPC stack is the default pat
### 1a. Trivial (XS–S each, no new payload format)
| ID | Capability | gRPC op | Notes |
|---|---|---|---|
| ~~R1.1~~ | ~~`ExecuteSqlCommandAsync`~~ | `Retrieval.ExecuteSqlCommand` | ⚠ **Blocked on 2020 WCF.** Live-probed 2026-06-20: `ExeC` returns native error type 4 / code **51 (InvalidParameter)** for every handle variant — same unmapped *native session/filter registration* prerequisite that blocks `StartTagQuery`/`QueryTag` (see `implementation-status.md` lines ~982, ~1404). Needs that registration RE'd, or a 2023 R2 gRPC server. Do not wire via guessed calls. |
| R1.2 | `GetRuntimeParameterAsync` | `Status.GetRuntimeParameter` | mirror `GetSystemParameter` |
| ~~R1.3~~ | ~~`GetServerTimeZoneAsync`~~ | `Status.GetSystemTimeZoneName` | ⚠ **gRPC/2023R2-only.** Verified 2026-06-20: over **2020 WCF** this op is a stub (rc=0, empty value) in the `GetServerTime` family — not shippable here. Build+verify only against a live 2023 R2 server. See `docs/reverse-engineering/wcf-status-localhost.md`. |
| ~~R1.1~~ | ~~`ExecuteSqlCommandAsync`~~ | `Retrieval.ExecuteSqlCommand` (`ExeC`+`GetR`) | ✅ **DONE (2026-06-20), live-verified.** `ExecuteSqlCommandAsync(sql)` → `HistorianSqlResult` (columns + typed rows). String-handle op via the uppercase storage GUID. Chain: `Retr.GetV` prime → `ExeC(handle, sql, option=0, ref queryHandle)` → `GetR` loop (note: `GetR` returns **false even on success** — the stream is in `pResultBuff` regardless; false = final page). `GetR`'s `pResultBuff` is an **NRBF-serialized `DataTable`** (`SerializationFormat.Xml`: members `XmlSchema` + `XmlDiffGram`). BinaryFormatter is gone from .NET 10, so it's decoded read-only with `System.Formats.Nrbf` + `XDocument` (no BinaryFormatter). Shipped: `HistorianSqlResult`/`HistorianSqlColumn`/`HistorianSqlExecuteOption`, `HistorianSqlResultProtocol`, `HistorianWcfSqlClient`, golden `WcfSqlResultProtocolTests`, gated live tests. See `docs/reverse-engineering/wcf-exec-sql.md`. |
| ~~R1.2~~ | ~~`GetRuntimeParameterAsync`~~ | `Status.GetRuntimeParameter` (`aa/Stat/GETRP`) | ✅ **DONE (2026-06-20), live-verified.** Captured (`scripts/Capture-RuntimeParam.ps1`): GETRP is a **`string`-handle** op (GETHI's shape), but reachable from the managed client using the Open2 storage-session GUID as an **uppercase** string handle (`ToString("D").ToUpperInvariant()`). Returns `HistorianVersion` = `20,0,000,000` live. pRequestBuff = `54 67 01 00` + uint nameCount + per-name(uint charCount + UTF-16); pResponseBuff = version + uint resultCount + CRetVariant(`0x43` VT_BSTR + uint16 len + uint16 charCount + UTF-16). Single string-valued param only (multi-name framing inferred, not captured). Shipped: `HistorianClient.GetRuntimeParameterAsync(name)`; golden `WcfRuntimeParameterProtocolTests`. **Note:** GETRP punching through the string-handle wall with the uppercase storage GUID is a strong lead that GETHI/ExeC may be a handle-*format* issue — see `wcf-string-handle-wall.md` §Update. |
| ~~R1.3~~ | ~~`GetServerTimeZoneAsync`~~ | `Status.GetSystemTimeZoneName` | ⛔ **gRPC/2023R2-only — re-confirmed 2026-06-21 from 3 angles.** (1) native `GetSystemTimeZoneName` is a stub (rc=0, empty) in the `GetServerTime` family; (2) Runtime DB has no timezone `SystemParameter` — the zone exists only as per-block `HistoryBlock.TimeZoneOffset`/`wwTimeZone` (DST-specific, SQL-only) + a `TimeZone` lookup table, `StorageShard.TimeZoneId`=NULL; (3) `GetSystemParameter` + `GETRP` return null/throw for every timezone candidate (only `HistorianVersion` works). The sole 2020 route is a SQL read via `ExecuteSqlCommand` (R1.1) — DST-specific, different mechanism. Build the real op only against a live 2023 R2 server. See `docs/reverse-engineering/wcf-status-localhost.md`. |
> ⛔ **String-handle wall (2026-06-20).** R1.4/R1.5/R1.6 (and R1.1) are **all blocked on 2020
> WCF** for the *same* reason: their ops take a **`string` GUID handle** and require an unmapped
> native session/filter registration. Probed live — GETHI returns code 1 for the exact native
> request shape across 5 handle formats + Stat.GetV priming; ExeC returns code 51. The proven
> surface uses **`uint`-handle** ops only. **One RE target — the native string-handle session
> registration — unblocks this whole sub-milestone.** Full analysis:
> `docs/reverse-engineering/wcf-string-handle-wall.md`. R1.8/R1.9 (StartQuery summary/state modes)
> are `uint`-handle and remain reachable on 2020 WCF.
> ✅ **String-handle "wall" RESOLVED (2026-06-20) — it was a handle-FORMAT bug.** R1.4/R1.5/R1.6
> (and R1.1) take a **`string` GUID handle**; the earlier "code 1/51 blocked" verdict came from
> passing the Open2 storage GUID in .NET's default **lowercase**. Sent **uppercase**
> (`storageSessionId.ToString("D").ToUpperInvariant()`) the same handle works: **GETRP** (R1.2,
> shipped), **GETHI** (R1.4) and **ExeC** (R1.1) are all live-verified reachable, and **R1.5
> `GetTepByNm`** is now **shipped + live-verified** (`GetTagExtendedPropertiesAsync`). **R1.6 has no
> distinct op** (collapses into R1.5). Note: `QTB` (StartTagQuery) does **not** punch through — it
> fails *server-side* (`CMdServer::StartActiveTagnamesQuery` over the `aahMetadataServer` pipe),
> independent of handle format, so the index-based property/query paths stay blocked here. Full
> analysis: `docs/reverse-engineering/wcf-string-handle-wall.md` (RESOLVED banner) and
> `docs/reverse-engineering/wcf-tag-extended-properties.md`.
> R1.8/R1.9 (StartQuery summary/state modes) are `uint`-handle and were already reachable.
### 1b. Bounded (decode one `bytes` payload; S–M each)
| ID | Capability | gRPC op | Payload to decode | Depends |
|---|---|---|---|---|
| ~~R1.4~~ | `GetHistorianInfoAsync` | `Status.GetHistorianInfo` | ⛔ **string-handle wall** — GETHI returns code 1 on 2020 WCF (all handle/priming variants). GETHI buffer incl. `EventStorageMode`@514. | string-handle RE |
| ~~R1.5~~ | Extended-property **read** | `Retrieval.GetTagExtendedPropertiesFromName` | ⛔ **string-handle wall** (GetTepByNm takes `string handle`). TEP result buffer. | string-handle RE |
| ~~R1.6~~ | Localized-property **read** | `Retrieval.GetTagLocalizedPropertiesFromName` | ⛔ **string-handle wall** (same family). | string-handle RE |
| R1.7 | Event **filters** | filter bytes in `Retrieval.StartEventQuery` | filter predicate encoding (name/op/value) — **`uint`-handle**, reachable | R0.5 |
| ~~R1.4~~ | `GetHistorianInfoAsync` | `Status.GetHistorianInfo` (`GETHI`) | ⛔ **BOUNDED OUT on 2020 (re-confirmed 2026-06-21).** GETHI is a **named-value** query reachable via the uppercase storage GUID, but **only `HistorianVersion` resolves** — the full 518-byte `HISTORIAN_INFO` struct (incl. `EventStorageMode`@514) is the 2023R2 HCAL-native/gRPC model. `EventStorageMode` has **no 2020 representation at all**: not a `SystemParameter` (only `EventStorageDuration`/`EventStorageLogPath`), not a DB column, and `GETRP`/`GetSystemParameter` return null/throw for it. The only 2020-reachable field (version) is already shipped via `GetSystemParameterAsync`/`GetRuntimeParameterAsync`, so a struct API would be hollow + misleading. Build the real op only against a live 2023 R2 server. See `docs/reverse-engineering/wcf-status-localhost.md`. | uppercase string handle |
| ~~R1.5~~ | Extended-property **read** | `Retrieval.GetTagExtendedPropertiesFromName` (`GetTepByNm`) | ✅ **DONE (2026-06-20), live-verified.** `GetTagExtendedPropertiesAsync(tag)` → name/value pairs. String-handle op via the uppercase storage GUID; name-based path (`GetTagExtendedPropertiesByName`, not the QTB-gated TagQuery path). Request `tagNames` = `uint count` + per-name(`uint charCount`+UTF-16); response = `uint tagCount` + per-tag(marker + compact-ASCII name + `uint propCount` + per-prop(marker + compact-ASCII name + `0x43` VT_BSTR value) + trailer). Sequence-paged. Shipped: `HistorianTagExtendedPropertyProtocol`, golden `WcfTagExtendedPropertyProtocolTests`, gated live test. See `docs/reverse-engineering/wcf-tag-extended-properties.md`. | uppercase string handle |
| ~~R1.6~~ | Localized-property **read** | (no op) | ⛔ **No distinct op on 2020 — collapses into R1.5.** There is no `GetTagLocalizedPropertiesFromName`/`GetTlpByNm` or `GetTagLocalizedPropertiesByName` in `current/aahClientManaged.dll`; the only "localized" surfaces are error-message/UI-text localization. Extended properties (R1.5) are the user-defined tag-property read surface. Closed, not throwing. | — |
| ~~R1.7~~ | Event **filters** | filter bytes in `Retrieval.StartEventQuery` | ✅ **DONE (2026-06-20), live-honored.** `ReadEventsAsync(start, end, HistorianEventFilter)`. The filter rides `StartEventQuery`'s `pRequestBuff` (captured via `EventQuery.AddEventFilter` + instrument-wcf-writemessage; Equal vs Contains diffed to isolate the op). Filter block: `ushort 0 + uint filterCount + uint condCount + uint nameLen + name(UTF-16) + uint 1 + ushort op + uint 1 + value(0x09-len-0x00 compact-ASCII) + byte 0`. **REAL, not inert** (a non-matching predicate returns 0 events; matching returns the subset). Single string-valued predicate only; multi-filter (OR) / multi-condition (AND via `AddEventFilterCondition`) framing not yet fully captured. See `HistorianEventFilter`, golden `WcfEventQueryProtocolTests`. | — |
| R1.8 | Analog-summary query | `Retrieval.StartQuery` (summary mode) | summary row layout — **`uint`-handle, reachable. Scoped + decode targets located** (`CAnalogSummaryValue.UnpackFromValueBuffer`, fields Min/Max/First/Last/ValueCount/Integral/…). Plan: [`r1.8-r1.9-summary-queries.md`](r1.8-r1.9-summary-queries.md) | — |
| R1.9 | State-summary query | `Retrieval.StartQuery` (state mode) | state-summary row layout — **`uint`-handle, reachable. Scoped** (`CStateSummaryStruct`: MinContained/MaxContained/TotalContained/PartialStart/PartialEnd/StateEntryCount). Plan: [`r1.8-r1.9-summary-queries.md`](r1.8-r1.9-summary-queries.md) | — |
@@ -112,9 +164,9 @@ read/browse/status surface is Windows-free and the gRPC stack is the default pat
| ID | Capability | gRPC op | Payload | Notes |
|---|---|---|---|---|
| R1.10 | `RenameTagsAsync` | History rename op | rename request buffer | `AllowRenameTags` already probed |
| R1.11 | Extended-property **write** | `History.AddTagExtendedProperties` (+ groups) / `DeleteTagExtendedProperties` | TEP serialize | mirror analog CTagMetadata discipline |
| R1.12 | Localized-property **write** | `History.AddTagLocalizedProperties` / `DeleteTagLocalizedProperties` | localized serialize | |
| R1.13 | Non-analog tag create (string/discrete) | `History.EnsureTags` | distinct CTagMetadata variant | ⚠ native AddTag rejected some types — confirm server path first; may be GATED |
| ~~R1.11~~ | Extended-property **write** | `History.AddTagExtendedProperties` (AddTEx) | ✅ **Add DONE (2026-06-21), live-verified.** `AddTagExtendedPropertiesAsync`/`AddTagExtendedPropertyAsync` (write mode, uppercase handle). inBuff = exact inverse of the R1.5 read framing (`uint32 groupCount + 0x01 + compact-ASCII tag + uint32 propCount + per prop[0x02 + compact-ASCII name + 0x43 VT_BSTR value] + 0x01 trailer + 0x00 terminator`); the trailing `0x00` is required or the server throws. Golden `WcfTagExtendedPropertyWriteProtocolTests` + gated live write/read-back test. **Delete (DelTep): wire format CAPTURED + serializer golden-proven (2026-06-21), but live delete is server-blocked and NOT shipped.** Captured via a two-session trick (add in Run A → fresh-session read-sync → delete in Run B, past the native err-229 client gate); inBuff = same group framing as Add but property-name-only and a `0x00` group trailer. A decisive experiment shows SDK-added properties ARE deletable (the native client deletes one), so SDK-add is complete; the SDK's own DelTep is rejected (`SErrorException` in `CHistStorage::DeleteTagExtendedProperties`) despite matching mode/handle/inBuff + GetTgByNm/GetTepByNm prime + open channel + 60s retries. Root cause: the native multiplexes services over ONE connection (per-connection working set), which the SDK's per-service WCF channels don't reproduce — needs transport-level multiplexing. See `docs/reverse-engineering/wcf-add-tag-extended-properties.md` §Delete. |
| ~~R1.12~~ | Localized-property **write** | (no op) | ⛔ **No distinct op on 2020 — closed (mirror of R1.6).** A symbol sweep of `current/*.dll` finds no `AddTagLocalizedProperties` / `DeleteTagLocalizedProperties` / any `*LocalizedPropert*` / `TagLocalized*`; only UI/error-text localization (`GetLocalizedText`/`GetLocalizedMessage`/`LocalizedResourcesDir`). Localized properties are a 2023 R2/gRPC concept. Closed, not throwing. See `docs/reverse-engineering/wcf-tag-extended-properties.md` §R1.12. | 2026-06-21 |
| ~~R1.13~~ | Non-analog tag create (string/discrete) | `History.EnsureTags` | distinct CTagMetadata variant | ⛔ **GATED — bounded out (2026-06-21, live-probed).** Native `AddTag` rejects every non-analog type **client-side** (`ErrorCode=ValidationFailed` / "Transaction validation failed", before any WCF op): SingleByteString, DoubleByteString, **and Int1** all fail; Float (control) succeeds. The native `HistorianDataType` enum has **no Discrete/Boolean** and no Int8/UInt8 (SDK-only extensions); `HistorianTag` has **no TagType setter** (type is data-type-derived). So no non-analog wire request is ever emitted → nothing to capture/implement. String/discrete create goes via a different subsystem (config editor / SQL), not this client's AddTag. `EnsureTagAsync` stays analog-only. See `docs/reverse-engineering/wcf-non-analog-tag-create.md`. |
**Acceptance:** read + browse + metadata + system/status + property R/W + summaries +
event-filtered reads + rename all live-verified over gRPC.
@@ -125,16 +177,34 @@ event-filtered reads + rename all live-verified over gRPC.
*Goal: `SendEventAsync(HistorianEvent)`. Path fully mapped in histevents.md; one capture away.*
| ID | Work | Detail |
|---|---|---|
| R2.1 | Capture the event value blob | Instrument `CCommonArchestraEventValue::PackToVtq` (or dump the VTQ value bytes) on a live `AddStreamedValue(HistorianEvent)`; save sanitized fixture |
| R2.2 | `HistorianEventWriteProtocol` | Serialize header (`ReceivedTime, EventType, EventTime, Id, RevisionVersion, IsUpdate/IsDelete, Namespace`) + typed property bag — **inverse of `HistorianEventRowProtocol`** (reuse typemarkers `0x02/0x10/0x18/0x31/0x43/…`) |
| R2.3 | Event write orchestrator | Open **Event** connection (write mode) → register CM_EVENT (already have) → `Storage.AddStreamValues` with the event VTQ |
| R2.4 | Public API | `HistorianClient.SendEventAsync(HistorianEvent)` (+ `HistorianEvent` model: Type, EventTime, property bag) |
| R2.5 | Round-trip test | Send an event → read it back via `StartEventQuery` / `v_AlarmEventHistory2`; golden-byte on R2.2 |
> ✅ **DONE (2026-06-20) — `HistorianClient.SendEventAsync(HistorianEvent)` shipped and
> live-accepted over 2020 WCF.** The headline assumption — that event delivery would ride the
> non-WCF storage-engine pipe (and so be blocked like revision writes) — was **disproved by
> capture**: a native `AddStreamedValue(HistorianEvent)` leaves over WCF as **`AddS2`
> (`IHistoryServiceContract2.AddStreamValues2`)**. CM_EVENT is a built-in registered tag, so the
> `129 TagNotFoundInCache` gate that blocks `AddS2` for user tags does **not** apply to events.
> The full managed chain (Open2 event-mode **0x501** → CM_EVENT RTag2/EnsT2 → AddS2) is accepted
> by the server (`AddS2` returns success, empty error buffer). See the event-send field map under
> §"Event-send wire format" in `histevents.md` and `HistorianEventWriteProtocol`.
>
> ⚠️ **Persistence caveat (environment, not SDK):** on the local dev Historian, accepted events
> are **not persisted** to the queryable store (`v_AlarmEventHistory2` latest stays at the
> pre-test date; count only ages down). The **native** client exhibits the identical behaviour
> (its `AddS2` also returns success but nothing lands), so this is the box's event-ingestion
> pipeline not being active — not an SDK protocol gap. The SDK emits byte-equivalent `AddS2`
> (golden-tested). Full send→store→read-back round-trip awaits a Historian with an active event
> storage pipeline.
**Acceptance:** an event sent from histsdk appears in the historian and is read back with
matching Type + properties. **Now practical** — Historian is installed locally.
| ID | Work | Status |
|---|---|---|
| R2.1 | Capture the event value blob | ✅ `scripts/Capture-EventSend.ps1` (event-send harness scenario + instrument-wcf-{write,read}message); two captures diffed to separate constant framing from value fields. Decisive finding: event-send = WCF `AddS2`, not storage pipe. |
| R2.2 | `HistorianEventWriteProtocol` | ✅ Serializes the `AddS2` pBuf (storage sample buffer wrapping the event VTQ): "OS" sig + sampleCount + length fields + CM_EVENT tag id + EventTime FILETIME + OpcQuality + opaque descriptor + event Id + ReceivedTime FILETIME + Namespace + EventType + version + typed property bag (string props reuse the read parser's `0x43` encoding). Golden-byte test pins capture A. |
| R2.3 | Event write orchestrator | ✅ `HistorianWcfEventOrchestrator.SendEventAsync`: Open2 (0x501) → reuse CM_EVENT RTag2/EnsT2 registration → `AddStreamValues2(handle, pBuf, out err)` on the same /Hist channel + storage-session handle. |
| R2.4 | Public API | ✅ `HistorianClient.SendEventAsync(HistorianEvent)`. Original events only (RevisionVersion=0) with string-valued properties; other property types + revision/update/delete throw `ProtocolEvidenceMissingException` until captured. |
| R2.5 | Round-trip test | ✅ Golden-byte on R2.2 + gated live test `SendEventAsync_AgainstLocalHistorian_AcceptedByServer` (asserts server acceptance; SQL read-back best-effort given the persistence caveat). |
**Acceptance:** an event sent from histsdk is accepted by the historian over WCF with a
byte-correct `AddS2` (✅). Appears-and-reads-back is environment-gated on event persistence (see caveat).
---
+43 -4
View File
@@ -257,12 +257,51 @@ StartDateTime and EndDateTime"`.
---
## Event-send wire format ✅ (captured 2026-06-20)
`AddStreamedValue(HistorianEvent)` leaves the client over **WCF as `AddS2`
(`IHistoryServiceContract2.AddStreamValues2(string handle, byte[] pBuf, out errorBuffer)`)** —
**not** the storage-engine pipe. (Captured with the NativeTraceHarness `event-send` scenario +
instrument-wcf-writemessage; two events diffed to separate constant framing from value fields.)
CM_EVENT is a built-in registered tag, so the `129 TagNotFoundInCache` gate that blocks `AddS2`
for user tags doesn't apply. Open2 uses event connection-mode **0x501** (vs 0x402 read / 0x401
write). The `pBuf` is a storage **sample buffer** wrapping the event VTQ:
```
0x00 UInt16 0x534F "OS"
0x02 UInt16 sampleCount = 1
0x04 UInt32 packet length = delivered byte[].Length (= valueBlob.Length + 11)
0x08 UInt16 valueBlob.Length + 1
0x0A valueBlob:
+0x00 GUID CM_EVENT tag id (353b8145-5df0-4d46-a253-871aef49b321)
+0x10 Int64 EventTime FILETIME UTC (floored to ms — the VTQ timestamp)
+0x18 UInt16 OpcQuality = 192
+0x1A UInt16 192
+0x1C UInt16 0x118D (opaque CCommonArchestraEventValue descriptor — constant)
+0x1E GUID event Id
+0x2E Int64 ReceivedTime FILETIME UTC (full 100ns; native uses a unique/monotonic time)
+0x36 compact-ASCII string Namespace
+.... compact-ASCII string EventType
+.... UInt16 eventStructVersion = 5
+.... UInt16 propertyCount
+.... propertyCount × { compact-ASCII name; value: UInt8 typeMarker, UInt8 len, UInt8 status, len×bytes }
trailing: 1 pad byte so byte[].Length == packet length (UInt32 @0x04). The native relies on
the MDAS encoder adding this byte (non-deterministic value); the SDK emits 0x00.
```
Property values reuse `HistorianEventRowProtocol`'s typed encoding; only `0x43` (UTF-16 string)
is captured on the write side so far. Implemented in `HistorianEventWriteProtocol` (golden-tested)
and shipped as `HistorianClient.SendEventAsync`. **Server accepts the `AddS2` (success, empty
error buffer); on-box persistence is environment-gated (the native client behaves identically).**
## Open threads
- 🔶 Event value blob: field set/order known (§2a); **exact native framing** still needs one
`CCommonArchestraEventValue::PackToVtq` output capture + golden-byte test (mirror the read-side
`HistorianEventRowProtocol` reverse-engineering). Now feasible locally — the live historian is
installed, so the same instrument-and-capture approach used for reads applies.
- ⚠️ Event **persistence**: accepted `AddS2` events do not land in `v_AlarmEventHistory2` on the
local dev box (native client identical) — the event storage/ingestion pipeline isn't active
here. Needs a Historian with active event storage to verify send→store→read-back end-to-end.
- 🔶 Non-string event property write encodings (bool/int/filetime/guid/double) and revision/
update/delete event sends (RevisionVersion≠0, IsUpdate/IsDelete) are **not captured**; the SDK
throws `ProtocolEvidenceMissingException` for them. One capture each to extend.
- ❓ `EnqueueEventDataPacket.SerializedBytes` packet framing (header + N event VTQs batched).
- ✅ Database-mode store: server writer is `aahEventStorage.exe` loading the managed
`ArchestrAEvents.EventStorage.Contract` connection assembly; SQL retrieval surface is the
+112 -38
View File
@@ -1,7 +1,59 @@
# R1.8 / R1.9 — Analog-summary & State-summary queries (implementation plan)
**Status (2026-06-20): scoped + de-risked. Reachable on 2020 WCF. Decode targets located in
the native dll. Ready to implement; not yet started in `src/` (no guessed code shipped).**
**Status (2026-06-21): RESOLVED by request + response capture. Conclusion: the rich
multi-aggregate analog/state summary struct is NOT delivered over the 2020 WCF binary protocol.
The per-cycle aggregate values it would expose are ALREADY shipped via `ReadAggregateAsync`
(RetrievalMode → QueryType 5–8). No new `src/` code is warranted for R1.8/R1.9 on 2020 WCF.**
## RESOLVED — what the response capture proved (2026-06-21)
The request side was recovered first (table further down), then the `GetNextQueryResultBuffer2`
**response** was captured (`instrument-wcf-readmessage`, both hooks chained) and decoded against
`AnalogSummaryHistory` SQL ground truth for `SysTimeSec` over a 6 h window / 1 h cycle. Findings:
1. **The response is the ordinary version-9 row buffer** — same layout the existing raw/aggregate
parser (`TryParseGetNextQueryResultBufferAggregateRows`) already handles: `uint16 version=9`,
`uint32 rowCount`, then per-row `tagKey + nameLen + name + ValueCount + cycleEnd FILETIME +
quality + OpcQuality + Value(double) + PercentGood(double) + trailer(cycleStart FILETIME …)`.
The captured 7-row buffer decoded with `Value=31.0`, `PercentGood=100.0`, `ValueCount=1`,
`OpcQuality=192` — matching the SQL row exactly.
2. **There is NO rich `CAnalogSummaryValue` struct on the wire.** Each row carries a *single*
value, not Min+Max+First+Last+Avg+Integral together. The all-aggregates-in-one-row shape that
`CAnalogSummaryValue` / `AnalogSummaryHistory` represents is the **SQL/OLEDB provider's** shape,
not the binary `StartQuery2` retrieval's.
3. **The single value is selected by `RetrievalMode` (QueryType), not by `ValueSelector`.** Proven
against the same constant tag where only the *kind* of aggregate distinguishes the result:
- `RetrievalMode=Integral` (QueryType 8) → `Value = 111600.0` (= SQL `Integral`) ✓
- `RetrievalMode=TimeWeightedAverage` (QueryType 5) → `Value = 31.0` (= SQL `Average`) ✓
- `Cyclic` (QueryType 0) **+ `ValueSelector=Integral`** → `Value = 31.0` (selector **ignored**;
the request byte `ValueSelector@0x59=0x04` was confirmed sent, yet the cyclic value came back).
So `ValueSelector` / `AggregationType` / `MaxStates` are **inert on the WCF retrieval path** —
they configure the SQL provider's summary tables, not this binary query.
4. **Resolution unit is correct in the SDK.** The wire `Resolution` is 100 ns ticks (= ms × 10000).
`SerializeFullHistoryRequest` writes `TimeSpan.Ticks`, which the golden test
`SerializerMatchesInstrumentedNativeTimeWeightedAverageRequest` already verifies byte-for-byte
against native (`FromMinutes(1)` → `600000000`). No bug.
**Therefore:** "analog summary" over 2020 WCF == the existing aggregate read. To get Min, Max,
Average and Integral for a cycle you issue the corresponding `RetrievalMode` queries
(`MinimumWithTime` / `MaximumWithTime` / `TimeWeightedAverage` / `Integral`), each returning that
one aggregate per cycle — all already implemented, mapped (QueryType 5–8) and golden-tested in
`ReadAggregateAsync`. **R1.8/R1.9 need no new protocol code on this server.** A genuine
all-aggregates-at-once summary would require the gRPC front door or the SQL provider, neither of
which is the 2020 WCF binary path.
Capture/decode tooling is committed and repeatable: `scripts/Capture-SummaryRequest.ps1`
(`-WithResponse` chains ReadMessage), `scripts/decode-summary-capture.py` (request diff),
`scripts/decode-summary-response.py <config>` (response decode vs SQL ground truth). Raw captures
live under `artifacts/reverse-engineering/instrumented-wcf-writemessage-summary/` (gitignored).
---
_Original scoping notes below remain for context. They led to the capture; the conclusion above supersedes their "ready to implement" framing._
Unlike the M1 *read* items gated by the [string-handle wall](../reverse-engineering/wcf-string-handle-wall.md),
summary queries ride the **proven `uint`-handle `StartQuery2`** path — the same call the working
@@ -34,43 +86,61 @@ Found via `methods … Summary` + `dnlib-method`:
| `CTypeMetadata.IsAnalogSummary` / `IsStateSummary` | `0x060001A4/A5` | server-side type gating |
| `INSQL_QUERYTYPE` / `HISTORIAN_SUMMARYTYPE` | enums `0200013F` / `02000191` | the `QueryType` / `SummaryType` values to send |
## Empirical probe results (2026-06-20, live `SysTimeSec` over `StartQuery2`)
## Native request capture (2026-06-21) — request shape RECOVERED
Swept `QueryType`/`SummaryType`/`ColumnSelectorFlags` against the live 2020 server:
The earlier blind probing (sweeping `SummaryType`/`ColumnSelectorFlags` over the managed
serializer) was the wrong lever: it returned 0-row buffers because the managed `SummaryType`
field is **not** how the native client encodes a summary. A real capture settled it.
- `QueryType=2 (Full)`, `SummaryType ∈ {0,3,6}` → normal 109-byte version-9 data buffer.
- `QueryType=2`, `SummaryType ∈ {1,2,4,5}` → **valid version-9 buffer with 0 rows** (`09 00 00 00 00 00`).
The server **accepts** these summary types but yields no rows.
- The 0-row result is **unchanged** by `ColumnSelectorFlags` (tried default, all-bits
`0xFFFF…FFFF`, high-dword, low-48). So column flags are *not* the unlock.
- `QueryType ∈ {15,16}` → `GetNext` blocks/times out (no such INSQL_QUERYTYPE ordinal).
**Capture pipeline (now repeatable):** `scripts/Capture-SummaryRequest.ps1` IL-rewrites a copy
of `aahClientManaged.dll` (`instrument-wcf-writemessage`), stages it alongside the strong-named
`ReverseInstrumentation` logger, then drives the `NativeTraceHarness` history scenario through a
candidate matrix while logging every outgoing MDAS body. `scripts/decode-summary-capture.py`
extracts the `Retr/StartQuery2` `pRequestBuff` from each and diffs the summary candidates against
a tag-matched `baseline-full`. The harness now exposes `--value-selector` / `--aggregation-type`
/ `--max-states` / `--filter` so the native `HistoryQueryArgs` summary knobs can be driven.
**Conclusion:** a summary query is *not* `Full + SummaryType + column flags`. The summary
configuration lives elsewhere in the request — almost certainly the **`AutoSummaryParameters`
trailer** (`SerializeFullHistoryRequest` currently writes it all-zero via
`WriteAutoSummaryParameters`) and/or a native summary `QueryType`. Both are **native-side
constants** (`HISTORIAN_SUMMARYTYPE` / `INSQL_QUERYTYPE` are `value__`-only in managed metadata;
`CColumnNameMap.LoadColumnNameMap` builds column→bit via native string/const data, not IL
`ldstr`/`ldc`). So they cannot be recovered from managed metadata, and blind probing of the
obvious fields returns empty.
**There is no separate "summary" QueryType or `SummaryType` field.** A summary is an ordinary
`StartQuery2` request (`QueryType` = the chosen `RetrievalMode`, e.g. `Cyclic`=0) with three
things set: the **ValueSelector** byte, the **AggregationType** byte, a non-zero **Resolution**
(which fills the previously-zeroed `AutoSummaryParameters` trailer), and — for state summary —
the **MaxStates** field. The server then returns analog- vs state-summary rows based on the tag
type plus these fields. Offsets below are **into the StartQuery2 `pRequestBuff`** (229-byte
`SysTimeSec` baseline; verified byte-for-byte against the native client):
**Therefore the right next step is a native request capture, not more probing:** drive the native
client (NativeTraceHarness / a real summary query) and capture the `pRequestBuff` bytes via the
existing `instrument-wcf-writemessage` pipeline — the same method that produced every other proven
request shape (reads, events, EnsT2). Diff that buffer against a normal Full request to read off
the exact `QueryType` + `SummaryType` + `AutoSummaryParameters` layout, then implement against it.
| Offset | Field | Type | Evidence |
|---|---|---|---|
| `0x01` | QueryType | uint32 LE | Full→`02`, Cyclic→`00` (matches the verified `RetrievalMode`→`QueryType` map) |
| `0x1D` | Resolution | float64 LE | `36e9` ticks → `00 00 00 D0 88 C3 20 42` = `0x4220C388D0000000` (1 h). Zero for non-summary reads |
| `0x32` | Timezone | len-prefixed UTF-16 | `"UTC"` |
| `0x49` | Filter | len-prefixed UTF-16 | `"NoFilter"` default; driven by `--filter` |
| `0x59` | **ValueSelector** | byte | baseline `01` (Auto); `--value-selector Minimum`→`06`, `Maximum`→`07`, `Average`→`08` — exact `HistorianValueSelector` values |
| `0x5B` | **AggregationType** | byte | baseline `03`; `--aggregation-type Average`→`02` — exact `HistorianAggregationType` values |
| `~0x5F` | ColumnSelectorFlags | bytes | `FF 82 07 00 82 81` — matches the `0x0000_8182_0007_82FF` reads already send; **unchanged** by summary |
| `0x6B` | Tag name | len-prefixed UTF-16 | `count, "SysTimeSec"` |
| after tag | **MaxStates** | uint16 LE | the `01`-default byte after the tag block; `--max-states 10`→`0A` (state summary, R1.9) |
| `~0xAA` | **AutoSummaryParameters** | block | zero for plain reads; `80 1E 08 6B 47 01` when Resolution set (identical across analog *and* state) — the resolution-derived cycle block |
## Open questions (nail these next, in order)
State summary (R1.9) is the **same request** with `MaxStates` > 0 (the analog `ValueSelector`/
`AggregationType` bytes stay at their `01`/`03` defaults); the analog-vs-state distinction on the
wire is which of those fields is non-default, plus the tag type. Note `MaxStates` is a **UInt16**
on `HistoryQueryArgs` (passing UInt32 throws) — the harness casts accordingly.
1. **Request params.** Recover the exact `QueryType` + `SummaryType` (+ whether `ColumnSelectorFlags`
must change) for analog vs state summary. Source: decompile `INSQL_QUERYTYPE`/`HISTORIAN_SUMMARYTYPE`
enum members and `Select{Analog,State}SummaryColumns` (the `GetColumnFlag` column-name `ldstr`
operands → the flag set). The standard QueryType map (Cyclic=0 … EndBound=14) is already verified;
summary is expected to be a `SummaryType`≠0 with an existing base `QueryType`, **not** a new mode
ordinal — confirm.
2. **Row layout.** Capture a real `GetNextQueryResultBuffer2` buffer for an analog summary of a
data-bearing tag (`SysTimeSec`) over a multi-hour window with an interval, then decode against the
`CAnalogSummaryValue` field set. Likely each row = StartDateTime FILETIME + the 8 typed fields.
Raw captures live under `artifacts/reverse-engineering/instrumented-wcf-writemessage-summary/`
(gitignored). Re-run with `scripts/Capture-SummaryRequest.ps1` (analog: `SysTimeSec`; state:
`-TagName SysPulse`, the local discrete tag).
## Open questions (only the row layout remains)
1. ~~**Request params.**~~ **DONE** — see the table above. ValueSelector @ `0x59`,
AggregationType @ `0x5B`, Resolution @ `0x1D` (→ AutoSummaryParameters @ `~0xAA`),
MaxStates after the tag block. No new QueryType/SummaryType ordinal involved.
2. **Row layout (next concrete step).** Capture the `GetNextQueryResultBuffer2` *response* for an
analog summary of `SysTimeSec` over a multi-hour window with a 1 h resolution — instrument
`ReadMessage` (`instrument-wcf-readmessage`, symmetric to the WriteMessage capture already
wired here) and decode against the `CAnalogSummaryValue` field set
(StartDateTime + Min/Max/First/Last/ValueCount/TimeGood/Integral/IntegralOfSquares). The
request side is no longer a blocker.
## Implementation steps (per the project's two-tests discipline)
@@ -84,9 +154,13 @@ the exact `QueryType` + `SummaryType` + `AutoSummaryParameters` layout, then imp
`HistorianStateSummary` (per-state contained/partial/entry-count). Reuse `RunQuery` plumbing.
5. Golden-byte test on the parser + gated live test on `localhost` (assert non-empty, fields sane).
## Why stop here this session
## State of play
`UnpackFromValueBuffer` is reader-call-based (no literal offset table), so a correct parser needs a
**captured real buffer** to decode against — that's the next concrete action, not a guess. Per
project rule ("never guess wire bytes; leave throwing until evidence supports it") no summary code
was added to `src/` yet. Everything above is the evidence needed to implement directly.
The **request side is fully recovered** from real bytes (table above) — the managed
`HistorianDataQueryRequest` builder can now set `ValueSelector`/`AggregationType`/`Resolution`
(+ `MaxStates` for state) against ground truth rather than guesses. What remains is the
**response row layout**: `CAnalogSummaryValue.UnpackFromValueBuffer` is reader-call-based (no
literal offset table), so the parser needs a captured real *response* buffer to decode against
(step 2 in Open questions — `instrument-wcf-readmessage`, already wired alongside the WriteMessage
capture). Per project rule ("never guess wire bytes; leave throwing until evidence supports it")
no summary code is in `src/` yet — that lands once the response fixture exists.
@@ -0,0 +1,57 @@
# R0.1 browse over gRPC — StartTagQuery takes an OData filter (2026-06-21)
Live-probed `RetrievalService.StartTagQuery` / `QueryTag` against a real **2023 R2** server over the
gRPC front door (string-handle = uppercase Open2 storage GUID). Key result: **browse is feasible on
2023 R2** — the 2020 WCF "metadata-server pipe" wall does **not** block here.
## StartTagQuery — CRACKED
`StartTagQuery(strHandle, btRequest)` where `btRequest` = the native
`marker(26449) + version(1) + WriteHistorianString(filter)` buffer
(`HistorianTagQueryProtocol.CreateStartTagQueryAttempt`). The server runs
`CMdServer::StartTagQuery::StartActiveTagnamesQuery` over `\\.\pipe\aahMetadataServer\console` and
**parses the filter string as OData** (not SQL-LIKE). Swept filters:
| filter | result |
|---|---|
| `startswith(TagName,'Sys')` | ✅ success, 8-byte response |
| `contains(TagName,'Sys')` | ✅ success |
| `TagName eq 'SysTimeSec'` | ✅ success |
| `` (empty) | ✅ success (all tags) |
| `Sys*` / `*` | ❌ `ODataFilter ... bad token` |
| `TagName like 'Sys%'` / `Name like 'Sys%'` | ❌ rejected |
Success response `btResponse` is the 8-byte `(queryHandle:uint, tagCount:uint)` pair
(`ParseStartTagQueryResponse`). Live: `startswith(TagName,'Sys')` → tagCount = 220.
**Implication for the public API:** browse must translate the SDK's glob filter to OData —
`*` → empty, `Pre*` → `startswith(TagName,'Pre')`, `*sub*` → `contains(TagName,'sub')`,
exact → `TagName eq '...'`. (Escaping single-quotes in names still TBD.)
## QueryTag — CRACKED (2026-06-21), browse SHIPPED
`QueryTag(strHandle, uiQueryHandle, btRequest)` is the paging call that returns the tag-name rows.
The blocker was the packet id: every guessed `btRequest` returned native error **type 4 / code 72 =
`InvalidPacketId`** (`ArchestrA.CloudHistorian.Contract.ErrorCode.InvalidPacketId`). The generic
`0x6751` header that StartTagQuery accepts is the **wrong** id for QueryTag.
**How it was found (no Ghidra needed):** a `.rdata` **packet-descriptor table** in
`aahClientManaged.dll` lists consecutive `{uint marker, uint version}` entries —
`{0x6751, 1}` (StartTagQuery) immediately followed by **`{0x6752, 1}`** (the paired op). Found by
`pefile` byte-scan of `.rdata` for `51 67 00 00` and dumping the surrounding dwords. Testing `0x6752`
live confirmed it.
**QueryTag wire format (live-verified):**
- request `btRequest` = `u16 marker(0x6752) + u16 version(1) + u16 queryType + u32 startIndex + u32 count`
— `queryType = 1` returns tag-name rows (`queryType = 0` returns an empty/count-only page).
- response `btResonse` = `u32 count + per-name(u32 charCount + UTF-16LE) + trailer`
(the trailer is the CloudHistorian `NextIndex`/`TagMetadataBuffer` region — ignored by
`HistorianTagQueryProtocol.ParseTagNameQueryPage`).
- Semantic fields match `ArchestrA.CloudHistorian.Contract.QueryTagRequest`
(`QueryType/StartIndex/TagCount`; the QueryHandle travels in the protobuf `uiQueryHandle`).
**Browse is shipped:** `HistorianClient.BrowseTagNamesAsync` routes over gRPC when
`Transport==RemoteGrpc` via `Grpc/HistorianGrpcTagClient.BrowseTagNamesAsync`
(StartTagQuery(OData) → paged QueryTag(0x6752) → EndTagQuery), with the SDK glob filter translated by
`GlobToODataFilter`. Golden-byte + glob unit tests and a gated live test
(`BrowseTagNamesAsync_OverGrpc_ReturnsSystemTags`) cover it. **M0 gRPC parity is complete.**
@@ -0,0 +1,131 @@
# Extended-property write over 2020 WCF — AddTEx (HCAL R1.11)
**Status: ✅ Add DONE + live-verified (2026-06-21). Delete (DelTep) deferred — see below.**
`HistorianClient.AddTagExtendedPropertiesAsync` / `AddTagExtendedPropertyAsync` writes user-defined
extended properties onto an existing tag via the 2020 WCF **`AddTEx`** (AddTagExtendedProperties) op,
and they read back via the R1.5 `GetTagExtendedPropertiesAsync` path. Verified end-to-end from the
pure-managed .NET 10 client against the local 2020 Historian (create tag → add property → read back →
delete tag).
## The op
```
bool AddTagExtendedProperties(string handle, byte[] inBuff, out byte[] errorBuffer) // AddTEx
```
On `IHistoryServiceContract2` (History service). Requires a **write-enabled** connection (Open2 mode
`0x401`) and the uppercase storage-session GUID handle — the SDK reuses the write orchestrator's
open + priming chain (the same one used by EnsT2/DelT). The tag is referenced by name inside `inBuff`;
no extra per-connection tag registration was needed (the server resolves it).
## The inBuff — the exact inverse of the R1.5 read response
The native `AddTagExtendedProperties(TagExtendedPropertyGroupList, out err)` packs its groups into the
`AddTEx` `inBuff` with the **same framing the R1.5 `GetTepByNm` response uses**, so the write serializer
is the inverse of `HistorianTagExtendedPropertyProtocol.ParseResponse`:
```
uint32 groupCount (= 1)
byte 0x01 (group marker)
0x09 + uint16 byteLen + ASCII tagName (compact-ASCII string)
uint32 propertyCount
repeated propertyCount times:
byte 0x02 (property marker)
0x09 + uint16 byteLen + ASCII propertyName
0x43 + uint16 payloadLen + uint16 charCount + UTF-16LE value (VT_BSTR variant; payloadLen = 2 + charCount*2)
byte 0x01 (group trailer)
byte 0x00 (buffer terminator)
```
⚠️ **The trailing `0x01 0x00`** matters: the group trailer is `0x01` (as in the read parser) **plus a
final `0x00` buffer terminator**. Omitting the `0x00` makes `inBuff` one byte short and the server throws
`SErrorException` in `aahClientAccessPoint::CHistStorage::AddTagExtendedProperties` (AddTEx returns
false). The read parser tolerates the extra byte because it only consumes one trailing byte per group.
Only the string (`0x43` VT_BSTR) value variant is evidence-backed (matching the read path). The raw
instrument capture mangles the final byte with MDAS chunk markers, so the golden fixture pins the
**clean** byte[] the SDK handed the channel (dumped via `AVEVA_HISTORIAN_TEP_DUMP`) — the exact buffer
the live server accepted.
## Delete (DelTep) — wire format captured + serializer proven; live delete server-blocked
**Status (2026-06-21): the `DelTep` wire format is captured and decoded, the serializer is
golden-verified against a server-accepted buffer, and SDK-added properties are confirmed deletable —
but the SDK's own delete is rejected server-side and is therefore NOT exposed publicly.** This is a
much deeper result than the earlier "couldn't capture the inBuff" deferral.
### Capturing it: the cross-session trick
The native `DeleteTagExtendedPropertiesByName(tag, propertyNames, deleteFromServer, out err)` performs
a **client-side sync check** and returns error **229 ("Tag extended property not synchronized with
server")** when deleting a *just-added* property — so a same-session add→delete never reaches the wire.
`AddTEx` success does **not** mark the local cache entry as server-synchronized; only a *server fetch*
(`GetTepByNm`) does. So the capture (`scripts/Capture-DeleteTagExtendedProperties.ps1`) runs two
separate harness processes against one instrumented DLL:
- **Run A**: `add-tep` creates the sandbox tag and adds the property (now server-synced).
- **Run B**: a fresh process opens a new connection, fetches the property
(`GetTagExtendedPropertiesByName`, which seeds the local cache as synced), then deletes it — so
`DeleteTagExtendedPropertiesByName` passes the client gate and `DelTep` reaches the wire.
### The inBuff — same group framing as Add, names only
```
uint32 groupCount (= 1)
byte 0x01 (group marker)
0x09 + uint16 byteLen + ASCII tagName
uint32 propertyCount
repeated propertyCount times:
byte 0x02 (property marker) + 0x09 + uint16 byteLen + ASCII propertyName ← NO value variant
byte 0x00 (group trailer) ← 0x00 for delete, vs 0x01 for add
byte 0x00 (buffer terminator)
```
The native `deleteFromServer` argument is **not** in the buffer — it is the client-side flag that
decides whether the wire op fires at all (`true` ⇒ a `DelTep` call). `HistorianTagExtendedProperty
Protocol.SerializeDeleteRequest` produces this exactly; `WcfTagExtendedPropertyWriteProtocolTests`
pins the server-accepted bytes.
### Why the SDK delete is server-blocked
The SDK's `DelTep` is rejected by the server with `SErrorException` in
`aahClientAccessPoint::CHistStorage::DeleteTagExtendedProperties`, even though:
- the **inBuff is byte-identical** to the server-accepted native capture (golden-verified);
- the **Open2 connection mode matches** the native (`0x401`, confirmed from the capture at offset
0x4a4);
- the **handles match** (uppercase storage-session GUID for `DelTep`/`GetTepByNm`, uint client handle
for `GetTgByNm`);
- the SDK first **primes the session** with `GetTgByNm` (tag identity, returns 140 bytes of tag info)
and `GetTepByNm` (returns the property), keeping the Retrieval prime channel **open across** the
`DelTep` call;
- retried with backoff for **60 s** (ruling out a storage-tier sync delay).
A decisive experiment localizes the gap: an **SDK-added** property *is* deletable — the native client
read-syncs and deletes it (`Success:true`). So the SDK's **add is complete**; only the SDK's **delete
session** is the problem. The native client multiplexes Hist/Retr/Stat/Trx over **one connection**
under a single `HistorianAccess` session, so its `GetTepByNm` populates a **per-connection working set**
that the same-connection `DelTep` consults. The SDK uses **separate WCF channels per service** (the
proven read pattern), so the borrowed-GUID Retrieval prime doesn't satisfy that server-side check.
Reproducing it requires transport-level connection multiplexing — a substantial change beyond this op.
The investigated-but-blocked orchestration is kept (internal
`HistorianWcfTagWriteOrchestrator.DeleteTagExtendedPropertiesAsync`, the `PrimeThenDelete` helper) for
follow-up, but `HistorianClient` deliberately exposes **no** public delete to avoid a silently-failing
write API. Capture/decode tooling: `scripts/Capture-DeleteTagExtendedProperties.ps1` +
`scripts/decode-del-tep-capture.py`, harness `add-tep` scenario with `--tep-skip-add` / `--tep-delete`.
## Shipped surface
- `HistorianClient.AddTagExtendedPropertiesAsync(tag, IReadOnlyList<HistorianTagExtendedProperty>)` and
`AddTagExtendedPropertyAsync(tag, name, value)`.
- `HistorianTagExtendedPropertyProtocol.SerializeAddRequest` (the inBuff serializer; lives beside the
R1.5 read parser); orchestrator path in `HistorianWcfTagWriteOrchestrator`.
- Golden `WcfTagExtendedPropertyWriteProtocolTests` (pins the server-accepted buffer + layout); gated
live test `AddTagExtendedPropertiesAsync_AgainstLocalHistorian_WritesAndReadsBack`.
## Capture / decode tooling
`scripts/Capture-AddTagExtendedProperties.ps1` (native-harness `add-tep` scenario +
instrument-wcf-{write,read}message; sandbox-guarded create→add→[optional delete]) and
`scripts/decode-add-tep-capture.py`.
+86
View File
@@ -0,0 +1,86 @@
# SQL command execution over 2020 WCF — ExeC + GetR (HCAL R1.1)
**Status: ✅ DONE + live-verified (2026-06-20).** `HistorianClient.ExecuteSqlCommandAsync(sql)` runs a
SQL command against the Historian over the 2020 WCF ops `aa/Retr/ExeC` (ExecuteSqlCommand) +
`aa/Retr/GetR` (GetRecordSetByteStream) and returns the record set as a `HistorianSqlResult` (the
managed equivalent of the native `DataTable`). Live-verified end-to-end from the pure-managed .NET 10
client against the local 2020 Historian (single-cell, multi-column/multi-row, and NULL cases).
## The ops
Both are on the Retrieval service (`IRetrievalServiceContract3`), **string-handle** ops reached with
the Open2 storage-session GUID formatted `storageSessionId.ToString("D").ToUpperInvariant()` (the same
uppercase handle that unlocked GETRP/GETHI; see `wcf-string-handle-wall.md`). `Retr.GetV` is primed
first.
```
bool ExecuteSqlCommand(string handle, string command, uint option,
ref uint queryHandle, out int retValue, out uint errorSize, out byte[] errorBuffer)
bool GetRecordSetByteStream(string handle, uint queryHandle, ref uint sequence,
out uint resultSize, out byte[] pResultBuff, out uint errorSize, out byte[] errorBuffer)
```
- **`command`** is sent as a plain string (MDAS-encoded ASCII), e.g. `SELECT 1 AS ProbeValue`.
- **`option`** = `HistorianSqlExecuteOption` (`ExecuteRecord=0` is the captured/proven record-set path).
- **`ExeC`** returns the assigned `queryHandle` (and `retValue`); pass it to `GetR`.
- **`GetR` returns `false` even on success** — the byte stream is in `pResultBuff` regardless; a
`false` result just signals the final page. So the orchestrator always consumes `pResultBuff`, then
stops on a `false` result or an empty page. (`sequence` is the paging cursor; small record sets
return everything in one call.)
## The result stream — NRBF-wrapped DataTable (no BinaryFormatter)
`GetR`'s `pResultBuff` is a **.NET Remoting Binary Format (NRBF)** stream wrapping a
`System.Data.DataTable` serialized with `SerializationFormat.Xml`. Stream shape (captured):
```
SerializationHeader (00 01 00 00 00 FF FF FF FF 01 00 00 00 00 00 00 00)
BinaryLibrary (0C): "System.Data, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
ClassWithMembersAndTypes (05): System.Data.DataTable, members:
DataTable.RemotingVersion -> System.Version object
XmlSchema -> BinaryObjectString (XSD: column names + xs types)
XmlDiffGram -> BinaryObjectString (ADO.NET diffgram: the rows)
System.Version object (_Major/_Minor/_Build/_Revision)
MessageEnd (0B)
```
**BinaryFormatter is removed from .NET 10**, so the stream is decoded read-only with
`System.Formats.Nrbf.NrbfDecoder` (the sanctioned successor — it parses records without instantiating
or executing any payload type; added as a managed `PackageReference`). The two embedded XML strings are
then parsed with `XDocument`:
- **XmlSchema** → columns (name + XSD type) under the `msdata:MainDataTable` element's sequence.
- **XmlDiffGram** → rows (each row element under the dataset; cells are child elements or attributes
matching column names). NULL cells are simply absent → parsed as `null`.
Values are typed per the XSD type (`xs:int`→int, `xs:string`→string, `xs:dateTime`→DateTime, …),
falling back to the raw string for unrecognized types. Only the `SerializationFormat.Xml` DataTable
shape is evidence-backed; a stream whose root is not a DataTable class record, or that lacks the two
XML members, throws `ProtocolEvidenceMissingException`.
## Capture / decode tooling
`scripts/Capture-ExecSql.ps1` (NativeTraceHarness `exec-sql` scenario + instrument-wcf-{write,read}message)
captures the ExeC/GetR exchange. ⚠️ A **raw** instrument-wcf capture interleaves MDAS transport chunk
markers (`0x9F`/`0x9E`) into a large `pResultBuff`, so raw byte-slicing yields a corrupted NRBF stream.
The **clean** contract-level byte[] (what the WCF channel reassembles) is dumped via the
`AVEVA_HISTORIAN_SQL_DUMP` env var on `HistorianWcfSqlClient` — that is the golden fixture in
`WcfSqlResultProtocolTests` (the benign `SELECT 1 AS ProbeValue`, no sensitive data).
## Shipped surface
- `HistorianClient.ExecuteSqlCommandAsync(command, option = ExecuteRecord)` → `HistorianSqlResult`
(`Columns` name+type, `Rows` typed values, `ReturnValue`).
- Models `HistorianSqlResult` / `HistorianSqlColumn` / `HistorianSqlExecuteOption`;
`HistorianSqlResultProtocol` (NRBF + diffgram parser); `HistorianWcfSqlClient` (ExeC/GetR
orchestration); golden `WcfSqlResultProtocolTests`; gated live tests
(`ExecuteSqlCommandAsync_AgainstLocalHistorian_ReturnsRecordSet` and `_MultiColumnMultiRow`).
## Scope notes
- `ExecuteRecord` (record set) is the evidence-backed path. `ExecuteNonQuery`/`ExecuteScalar`/
`ExecuteRecordDirect` are accepted via the option enum but their non-record-set return shapes are not
separately captured — a non-record result yields empty `Columns`/`Rows` with the `ReturnValue` set.
- The command is whatever the Historian's SQL surface accepts (it routes to the Runtime DB). No
client-side SQL validation is performed.
@@ -0,0 +1,68 @@
# GetHistorianInfo over 2020 WCF — GETHI is named-value-only (HCAL R1.4)
**Status: ⚠ Bounded out on 2020 WCF (2026-06-20).** `GetHistorianInfoAsync` is **not shipped**:
the one field that motivates it — `EventStorageMode` — is **not on the 2020 WCF wire**. The
version field that GETHI *does* return over WCF is already exposed (`ProbeAsync`,
`GetRuntimeParameterAsync("HistorianVersion")`), so there is nothing new to ship here without a
2023 R2 gRPC server. This parallels R1.3 (`GetServerTimeZone`), which is likewise 2023R2-only.
## What the capture showed
`scripts/Capture-HistorianInfo.ps1` drives the native `HistorianAccess.GetHistorianInfo(out
HistorianInfo, out error)` through the instrumented (`instrument-wcf-{write,read}message`)
`current/aahClientManaged.dll`. The native call **succeeds** and returns
`EventStorageMode = Blocks`, `ServerVersion = 20,0,000,000`, no error.
But the wire tells a different story (`scripts/decode-historian-info-capture.py`):
- The only `GETHI` op on the wire is **`aa/Stat/GETHI(handle, pRequestBuff)`** with
`pRequestBuff = 53 67 02 00` (sig `0x6753` + version `2`) `+ uint charCount(16) + UTF-16
"HistorianVersion"` — i.e. the **named-value request**, identical to the GETRP/version shape.
- Its response `pResponseBuff` is **~30 bytes**: `uint charCount(12) + UTF-16 "20,0,000,000"`
(+ a `02 00 01 00` trailer). **Just the version** — not a 518-byte struct.
- The post-GETHI ops in the same capture are `Hist/UpdC3` + a run of `Stat/GetSystemParameter`
(`AllowOriginals`, `HistorianPartner`, `HistorianVersion`, `MaxCyclicStorageTimeout`,
`RealTimeWindow`, `FutureTimeThreshold`, `AllowRenameTags`). **None carries a storage-mode
value.** So the native wrapper's `EventStorageMode` is derived by the C++ HCAL **outside the
WCF wire**, not fetched over it.
## Probe: does GETHI expose storage mode under any name?
`StringHandleProbeDiagnosticTests.GETHI_CandidateInfoNames_AgainstLocalHistorian` (gated on
`HISTORIAN_HOST=localhost`) issues GETHI for `HistorianVersion` plus seven storage-mode name
guesses. Result on the live 2020 server:
| GETHI parameter name | result |
|---|---|
| `HistorianVersion` | **ok=True**, respLen=32 (version) |
| `EventStorageMode`, `EventStorageType`, `StorageType`, `HistorianEventStorageMode`, `EventStorage`, `StorageMode`, `HistorianInfo` | **ok=False**, errLen=5, empty |
So GETHI on 2020 WCF is a strict named-value lookup with exactly one known-good key
(`HistorianVersion`). There is no storage-mode key, no full-struct request.
## Why the 518-byte struct doesn't apply here
The 2023 R2 decompiled `ArchestrA.HistorianAccess.GetHistorianInfo` (analysis folder) allocates
a **518-byte `HISTORIAN_INFO`** struct, pre-inits `int32 @514` to `-1`, calls native HCAL
(vtable+648) which fills it, then reads version (UTF-16 @0) + `EventStorageMode` (`@514`:
`-1`=Unsupported, `0`=Database, else=Blocks). That is the **HCAL-native / 2023R2 gRPC**
front-door model (`StatusService.GetHistorianInfo` returns `bytes btHistorianInfo`). On **2020
WCF** that struct is never marshaled across the wire — only the version named-value is. The
native client's `EventStorageMode` therefore comes from C++-internal state the managed WCF
replay cannot observe or reproduce.
## Conclusion / where it lands
- **2020 WCF:** `GetHistorianInfoAsync` would add nothing over existing surface (version only) and
could not report a real `EventStorageMode` — so it is intentionally **not shipped** (no hollow
`Unsupported`-returning API; project discipline: don't ship misleading behavior).
- **2023 R2 gRPC:** `Status.GetHistorianInfo` returns the full 518-byte `btHistorianInfo`; decode
version@0 + `EventStorageMode`@514 there. Build + verify against a live 2023 R2 server. The
`HistorianInfo` / `HistorianEventStorageMode` public types should land alongside that path.
## Tooling kept as RE aids
- `tools/AVEVA.Historian.NativeTraceHarness` `historian-info` scenario (drives the native call).
- `scripts/Capture-HistorianInfo.ps1` + `scripts/decode-historian-info-capture.py`.
- `StringHandleProbeDiagnosticTests.GETHI_CandidateInfoNames_AgainstLocalHistorian` (locks the
named-value-only finding; gated).
@@ -0,0 +1,67 @@
# Non-analog tag create over 2020 WCF — GATED (HCAL R1.13)
**Status: ⛔ bounded out (2026-06-21). No non-analog (string / discrete / wide-integer) tag-create
path is reachable on 2020 — the native managed client rejects every non-analog type *client-side*,
before any WCF op, so there is no wire format to capture and nothing to implement against.**
`EnsureTagAsync` stays analog-only (Float, Double, Int2, UInt2, Int4, UInt4); unsupported types throw
`ProtocolEvidenceMissingException` from `HistorianTagWriteProtocol.GetAnalogDataTypeCode`.
## What R1.13 asked for
Create string / discrete (non-analog) tags via `History.EnsureTags`, with a distinct `CTagMetadata`
variant. The roadmap flagged it "⚠ native AddTag rejected some types — confirm server path first;
may be GATED."
## Findings (live-probed against the local 2020 Historian)
1. **No discrete/boolean data type exists.** The native `ArchestrA.HistorianDataType` enum
(`current/aahClientManaged.dll`, dumped via `enum-dump`) has exactly 12 members: `Int1, Int2,
UInt2, Int4, UInt4, Float, Double, SingleByteString, DoubleByteString, Event, Structure`. There is
no `Discrete`/`Boolean`, and no `Int8`/`UInt8`/`UInt1`/`Guid`/`FileTime` (those are SDK-only
extensions in `Models/HistorianDataType`, recovered from the C++ `CDataType` predicate IL — they
are not settable on the managed `HistorianTag`).
2. **Tag type is data-type-derived, not separately settable.** `ArchestrA.HistorianTag`
(`--dump-type-members`) has **no** `TagType` property — only `TagDataType`. It does carry the
discrete/string-shaped fields (`MessageOn`/`MessageOff`, `RolloverValue`, dead-band/interpolation),
and the type exposes `ValidateAnalog*`, `ValidateDiscreteGeneralProperties`, and
`ValidateDiscreteAndStringStorageProperties` — but the analog-vs-discrete-vs-string decision is made
internally from the data type, with no way to request "discrete."
3. **Native AddTag rejects every non-analog type client-side.** Driving the native
`HistorianAccess.AddTag(HistorianTag, …)` (harness `write` scenario, `--write-data-type`) against
the live server:
| Data type | AddTag.Success | ErrorCode | ErrorType |
|---|---|---|---|
| SingleByteString | **false** | ValidationFailed | CustomError |
| DoubleByteString | **false** | ValidationFailed | CustomError |
| Int1 | **false** | ValidationFailed | CustomError |
| Int8 / UInt8 | n/a | *not in the native enum* | — |
| Float (control) | true | Success | — |
The error — `ErrorType=CustomError`, `ErrorCode=ValidationFailed`, `ErrorDescription="Transaction
validation failed"` — is raised by the client's own `Validate*` chain **before any WCF message is
sent** (the wrapper even auto-populates discrete defaults `MessageOn=ON`/`MessageOff=OFF`, then
fails validation). So the native client never emits a non-analog `EnsT2`/`AddTag` request.
## Why it's not deliverable here
Because the native client refuses non-analog types client-side, **no wire request exists to
reverse-engineer** — there is no captured `CTagMetadata` variant for string or discrete tags, and the
SDK does not guess wire bytes. The rejection is not string-specific: `Int1` (a non-string integer
outside the analog set `{Float, Double, Int2, UInt2, Int4, UInt4}`) fails identically, so the boundary
is "the analog set" rather than "strings only." Creating string/discrete tags on 2020 evidently goes
through a different subsystem (e.g. the configuration editor / SQL config path), not this client's
`AddTag`. R1.13 is closed as GATED, consistent with the mission note that these types "fail at native
AddTag — likely require a different path and are intentionally not supported."
## Probe commands (read-only / sandbox-guarded)
```
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- enum-dump current\aahClientManaged.dll HistorianDataType
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- dnlib-method current\aahClientManaged.dll HistorianTag.ValidateDataType
# native AddTag probe (sandbox tag must start with RetestSdkWrite; --write-skip-add-value avoids the blocked value path):
dotnet run --project tools\AVEVA.Historian.NativeTraceHarness -- --scenario write --server-name localhost --tcp-port 32568 \
--write-sandbox-tag RetestSdkWriteNADoubleByteString --write-data-type DoubleByteString --write-skip-add-value
```
@@ -0,0 +1,82 @@
# Tag rename over 2020 WCF — StartJob (StJb) rename job (HCAL R1.10)
**Status: ✅ DONE + live-verified (2026-06-21).** `HistorianClient.RenameTagsAsync` /
`RenameTagAsync` renames tags by submitting an asynchronous rename **job** to the Historian. Decoded
from an instrumented native `RenameTags` capture and verified end-to-end from the pure-managed .NET 10
client against the local 2020 Historian (sandbox tag created → renamed → new name visible → cleaned up).
## The op — rename rides the generic job framework
There is **no dedicated rename WCF operation**. The native `RenameTags(Tuple<string,string>[] pairs,
ref HistorianTagRenameStatus, out error)` packs the batch into the generic History **`StartJob`**
(`StJb`) buffer; the server returns a job id and applies the renames in the background. The native
client then polls `GetJobStatus` (`GtJb`) until the job reports done.
```
bool StartJob(string handle, byte[] jobBuffer, out string jobId, out byte[] errorBuffer) // StJb
bool GetJobStatus(string handle, string jobId, out byte[] jobStatus, out byte[] errorBuffer) // GtJb
```
Both already existed in `IHistoryServiceContract2`. `StartJob` takes a **string handle** = the Open2
storage-session GUID formatted `storageSessionId.ToString("D").ToUpperInvariant()` (the same uppercase
handle used by the other string-handle ops). The connection must be **write-enabled** (Open2 mode
`0x401`); the SDK reuses the write orchestrator's open + priming chain.
## The rename jobBuffer (decoded + server-validated)
```
byte[7] reserved / job-descriptor prefix (all zero in every capture)
uint32 pairCount
repeated pairCount times:
uint32 oldNameCharCount + UTF-16LE oldName (the tag being renamed)
uint32 newNameCharCount + UTF-16LE newName (its new name)
```
Char counts are UTF-16 code-unit counts. Pair order is **(old, new)**. ⚠️ A **raw** instrument
capture mangles the buffer's final byte with MDAS chunk markers (`9E`/`9F`) — the same hazard noted
for R1.1. So the golden fixture is the **clean** byte[] the SDK hands the WCF channel, dumped via the
`AVEVA_HISTORIAN_RENAME_DUMP` env hook on `HistorianWcfTagWriteOrchestrator`. That exact buffer was
accepted by the live server and the tag was renamed, so it is server-validated, not hand-stitched.
## Server gate — `AllowRenameTags`
Rename is gated by the **`AllowRenameTags`** system parameter (default **0/disabled**). When disabled,
the **native** client library rejects the call *before the wire* (`error 132 OperationNotEnabled`,
component `aahClientCommon::CClientCommon::RenameTags`); the managed SDK has no such pre-check, so a
disabled gate surfaces as `StartJob` returning false (reported as `Accepted = false`).
To enable for testing: `EXEC Runtime.dbo.aaSystemParameterUpdate @name='AllowRenameTags', @value=1`
**and reload the Historian config** — the running services cache system parameters, so the value only
takes effect after the Historian reloads (a Historian restart; a storage-engine-only restart is **not**
enough — the value is served from the `InSQLConfiguration` cache). Restore to `0` when done.
## Async completion
`RenameTagsAsync` submits the job and returns `HistorianTagRenameResult { Accepted, JobId, PairCount,
Error }`. The renames apply asynchronously server-side (observed: the native `GetTagRenameStatus` went
`Pending=true` → `false` within ~1.5 s for a single rename on the local box). The SDK does **not** poll
`GtJb` for completion: only the *pending* `jobstatus` buffer (6 zero bytes) was captured — the
done-state encoding was not, so polling is intentionally left out rather than guessing it. The gated
live test confirms completion by polling the new name's metadata after submission.
## Shipped surface
- `HistorianClient.RenameTagAsync(old, new)` / `RenameTagsAsync(IReadOnlyList<(string,string)>)`
→ `HistorianTagRenameResult`.
- `HistorianTagRenameProtocol.SerializeRenameJob` (the jobBuffer serializer);
`HistorianWcfTagWriteOrchestrator.RenameTags`/`SendStartJobRename` (open → write-priming → StJb).
- Golden `WcfTagRenameProtocolTests` (pins the server-accepted buffer + layout); gated live test
`RenameTagsAsync_AgainstLocalHistorian_RenamesSandboxTag` (needs `HISTORIAN_HOST=localhost`,
`HISTORIAN_RENAME_SANDBOX=RetestSdkWrite…`, and `AllowRenameTags` enabled).
## Capture / decode tooling
`scripts/Capture-RenameTags.ps1` (native-harness `rename` scenario + instrument-wcf-{write,read}message;
sandbox-guarded create→rename→cleanup) and `scripts/decode-rename-capture.py`.
## Scope notes
- String-valued, original tag renames only. The multi-pair batch framing is captured (count-prefixed)
and unit-tested; the live test exercises a single pair.
- `RenameSourceTags` (replication/source-server rename) is **not** shipped — different op signature
(adds a source-server string), not captured.
@@ -48,3 +48,54 @@ Interpretation:
- **`GetServerTimeZoneAsync` (roadmap R1.3) is NOT a trivial WCF op on 2020** — it
is a stub returning empty. Do not ship it over the 2020 WCF transport. Deliver
it only against a live 2023 R2 gRPC server. Reclassified in `docs/plans/hcal-roadmap.md`.
## GETRP / GetRuntimeParameter (roadmap R1.2) — DONE, live-verified 2026-06-20
Captured the native `HistorianAccess.GetRuntimeParameter(List<string>, out List<object>)`
WCF traffic with `scripts/Capture-RuntimeParam.ps1` (instrument-wcf-{write,read}message).
Findings:
- The WCF op is **`aa/Stat/GETRP`** — `bool GETRP(string handle, byte[] pRequestBuff,
out byte[] pResponseBuff, out byte[] errorBuffer)`, i.e. the **same string-handle +
request/response-buffer shape as GETHI**, *not* the simple `GetSystemParameter(uint, string)`
shape the roadmap originally assumed.
- The `string handle` is the **Open2 storage-session GUID** (the value
`ParseOpenConnectionResponse` reads from `outBuff[5..21]`), sent **UPPERCASE, dash-separated,
no braces** (`ToString("D").ToUpperInvariant()`).
- Unlike GETHI (which the earlier probe found blocked), **GETRP succeeds from the pure-managed
client** with that handle: `GetRuntimeParameter("HistorianVersion")` → `20,0,000,000`.
- `pRequestBuff` = `54 67 01 00` (sig+version) + uint nameCount + per name(uint charCount +
UTF-16LE). `pResponseBuff` = version(1) + uint resultCount + CRetVariant(`0x43` VT_BSTR +
uint16 payloadLen + uint16 charCount + UTF-16LE).
Shipped as `HistorianClient.GetRuntimeParameterAsync(name)`. See
`HistorianRuntimeParameterProtocol`, golden `WcfRuntimeParameterProtocolTests`, and the
handle-format lead in `wcf-string-handle-wall.md` §Update (retry GETHI/ExeC uppercased).
## R1.3 timezone + R1.4 EventStorageMode — re-confirmed bounded out (2026-06-21)
Both were already classified 2023R2/gRPC-only; re-verified from two *fresh* angles that corroborate it
more strongly than the original op-level probes:
- **Runtime DB schema** (`Runtime.dbo`, the server's own source of truth): the `SystemParameter` table
has **no** timezone parameter and **no `EventStorageMode`** (only `EventStorageDuration` /
`EventStorageLogPath`). The server timezone exists only as **per-block storage artifacts**
(`HistoryBlock.TimeZoneOffset` = e.g. 240 min, `wwTimeZone` = e.g. "Eastern Daylight Time") and a
`TimeZone` reference/lookup table; `StorageShard.TimeZoneId` is NULL. So the timezone is a
DST-specific, SQL-only, OS-derived value, not a clean server-config field exposed by any op.
- **Parameter-op probe** (`StringHandleProbeDiagnosticTests.TimezoneAndStorageMode_ParameterProbe`):
`GetSystemParameter` and `GetRuntimeParameter` (GETRP) were asked for every timezone candidate
(`TimeZone`/`ServerTimeZone`/`SystemTimeZone`/`TimeZoneName`/`SystemTimeZoneName`/`TimeStampRule`/
`ServerTime`) and every storage-mode candidate (`EventStorageMode`/`StorageMode`/`EventStorage`/
`EventStorageDuration`). **All returned null (GetSystemParameter) or threw
`ProtocolEvidenceMissingException` (GETRP — non-string/empty response)**; only the `HistorianVersion`
control returned a value (`20,0,000,000`). Note: `TimeStampRule`/`EventStorageDuration` *do* exist in
the `SystemParameter` table yet `GetSystemParameterAsync` returns null for them — the shipped op only
surfaces a whitelisted subset (a possible future widening, unrelated to R1.3/R1.4).
Conclusion: **R1.3 `GetServerTimeZoneAsync` and R1.4 `GetHistorianInfoAsync` (EventStorageMode) are not
deliverable as server ops on 2020.** The only 2020 route to the timezone is a SQL read of
`HistoryBlock`/`TimeZone` via `ExecuteSqlCommand` (R1.1) — a DST-specific value over a different
mechanism than the roadmap's `Status.GetSystemTimeZoneName`. `EventStorageMode` has no 2020
representation at all (it is a 2023 R2 event-storage-architecture field). Deliver both only against a
live 2023 R2 gRPC server.
@@ -1,10 +1,45 @@
# The 2020 WCF string-handle wall (2026-06-20)
> ## ✅✅ RESOLVED (2026-06-20): the "wall" was a handle-FORMAT bug, not a registration wall.
>
> The string-handle ops are reachable from the pure-managed client after all. The Open2
> storage-session GUID must be passed as the `string handle` **UPPERCASE, dash-separated,
> no braces** — `storageSessionId.ToString("D").ToUpperInvariant()`. The earlier probes that
> "proved" the wall passed the GUID in .NET's default **lowercase** `ToString("D")`, which the
> server's session table does not match. Live-verified end-to-end against the local 2020 server:
> - **GETRP** (R1.2) → returns the runtime `HistorianVersion` (shipped).
> - **GETHI** (R1.4) → `returned=True`, returns the version buffer (`0C000000` + UTF-16 "20,0,000,000").
> - **ExeC** (R1.1) → `returned=True`, `Retr.GetV` prime + `ExeC("SELECT 1 AS ProbeValue", option=0)`
> yields `queryHandle`, then `GetR(handle, queryHandle, sequence=0)` returns a 1232-byte result =
> a **BinaryFormatter-serialized .NET DataTable** (stream header `…System.Data, Version=4.0.0.0…`).
>
> Probes: gated `StringHandleProbeDiagnosticTests` (GETHI + ExeC). Captures:
> `scripts/Capture-RuntimeParam.ps1`, `scripts/Capture-ExecSql.ps1`. The handle for ExeC/GetR is the
> **same** Open2 storage-session GUID (confirmed = `outBuff[5..21]`). The original analysis below is
> retained for history; treat its "blocked" conclusions as **superseded** — the only missing piece
> was the uppercase format.
>
> **Update 2026-06-20 — R1.5 `GetTepByNm` shipped; QTB nuance.** `GetTagExtendedPropertiesFromName`
> (`GetTepByNm`) is now **shipped + live-verified** with the uppercase handle
> (`GetTagExtendedPropertiesAsync`; see `wcf-tag-extended-properties.md`). It confirms the
> string-handle Retrieval family is reachable (and `GetTgByNm`/GetTagInfosFromName was observed
> succeeding alongside it). **But not every string-handle op is just a format fix:** `QTB`
> (`StartTagQuery`) was captured being sent with a correctly-**uppercase** handle and still failed
> with `error 1` *server-side* (`CMdServer::StartTagQuery::StartActiveTagnamesQuery` over
> `\\.\pipe\aahMetadataServer\console`). So QTB/QTG (the active-tagnames query family) are blocked by
> the metadata server, not the handle format — distinct from the handle-format wall. **R1.6
> (localized properties) has no distinct op** and collapses into R1.5.
---
Live-probing the local **Historian 2020** (WCF, port 32568) for HCAL roadmap M1
surfaced a clean structural boundary on what the pure-managed client can call. It
explains why R1.1/R1.4/R1.5 all fail and identifies the single RE target that
unblocks the rest of the M1 read surface.
> ⚠️ **Superseded — see the RESOLVED banner above.** The boundary below is real *only* when the
> handle is sent lowercase. With the uppercased storage GUID the string-handle ops succeed.
## The dichotomy
Retrieval/Status/History ops split by the **type of their first (handle) parameter**:
@@ -56,3 +91,48 @@ once and the whole family unlocks. Until then, the alternatives are:
Do **not** ship any string-handle op via guessed calls (project discipline:
"leave them throwing until evidence supports an implementation").
## ⚠️ Update (2026-06-20): GETRP punches through — the wall is not absolute
Roadmap **R1.2 `GetRuntimeParameterAsync`** turned out to be a **`string`-handle op**
(`aa/Stat/GETRP(string handle, byte[] pRequestBuff) → (bool, byte[] pResponseBuff,
byte[] errorBuffer)`) — the **same shape as GETHI**, and in the same native session it
uses the **same handle GUID** as GETHI (confirmed: the GUID equals the Open2 `outBuff`
storage-session id at `[5..21]`, the value the managed `ParseOpenConnectionResponse`
already extracts as `StorageSessionId`).
Yet GETRP **works from the pure-managed client** — live-verified, returns the runtime
`HistorianVersion` value `20,0,000,000`. The only material difference from the failed
GETHI probe is the **handle string format**: the native client sends the GUID
**UPPERCASE, dash-separated, no braces** (format example
`XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX`, all hex upper), i.e.
`storageSessionId.ToString("D").ToUpperInvariant()`. `.NET Guid.ToString("D")` is
lowercase, so a probe that passed the GUID without upcasing would not byte-match what
the server's session table is keyed on.
**Implication — CONFIRMED, the wall is largely a handle-format bug.** The follow-up was done:
GETHI and **ExeC both return data with the uppercased storage-session GUID**.
- **R1.1 `ExecuteSqlCommandAsync` (ExeC + GetR) — SHIPPED + live-verified (2026-06-20).**
`ExecuteSqlCommandAsync(sql)` → `HistorianSqlResult`. `Retr.GetV` prime → `ExeC(handle,
sql, option=0, ref queryHandle)` → `GetR` loop. Note: **`GetR` returns false even on
success** (the byte stream is in `pResultBuff` regardless; false = final page). `pResultBuff`
is an **NRBF `DataTable`** (`SerializationFormat.Xml`: `XmlSchema` + `XmlDiffGram`), decoded
read-only with `System.Formats.Nrbf` + `XDocument` (BinaryFormatter is gone from .NET 10).
Shipped: `HistorianSqlResultProtocol`, `HistorianWcfSqlClient`, golden `WcfSqlResultProtocolTests`,
gated live tests. See `docs/reverse-engineering/wcf-exec-sql.md`.
- **GETHI (R1.4)** returns data with the uppercase handle, **but only the named `HistorianVersion`
value** — over 2020 WCF GETHI is a named-value query (the only working key), *not* a full-struct
read. `EventStorageMode` (the 518-byte-struct `@514` field) is **not on the 2020 WCF wire**; it is
the 2023R2 HCAL-native/gRPC model. So R1.4 is **bounded out on WCF / gRPC-2023R2-only** and the
public API is intentionally not shipped. Full analysis: `docs/reverse-engineering/wcf-historian-info.md`.
So the "wall" collapses to the handle **format** for the Retrieval/Status string-handle ops.
**Exception — QTB/QTG:** `StartTagQuery` does *not* punch through; captured with a correctly
uppercase handle it still fails `error 1` **server-side** (`CMdServer::StartActiveTagnamesQuery`
over `\\.\pipe\aahMetadataServer\console`) — a metadata-server blocker, independent of handle
format. Name-based ops route around it.
See `HistorianRuntimeParameterProtocol`, `IStatusServiceContract2.GetRuntimeParameter`,
golden `WcfRuntimeParameterProtocolTests`, and capture tooling
`scripts/Capture-RuntimeParam.ps1` + `scripts/decode-runtime-param-capture.py`.
@@ -0,0 +1,116 @@
# Tag extended properties over 2020 WCF — GetTepByNm (HCAL R1.5)
**Status: ✅ DONE + live-verified (2026-06-20).** `HistorianClient.GetTagExtendedPropertiesAsync(tag)`
reads a tag's extended (user-defined) properties over the 2020 WCF op
`aa/Retr/GetTagExtendedPropertiesFromName` (`GetTepByNm`). Live-verified end-to-end from the
pure-managed .NET 10 client against the local 2020 Historian.
## The op
`GetTepByNm` is on the Retrieval service (`IRetrievalServiceContract4`):
```
bool GetTagExtendedPropertiesFromName(
string handle, // Open2 storage-session GUID, UPPERCASE dash-no-braces
byte[] tagNames, // [MessageParameter pRequestBuff-style]
ref uint sequence, // paging cursor (0 on first call)
out byte[] tagExtendedProperties, // result buffer
out byte[] errorBuffer)
```
It is a **string-handle** op — reachable from the managed client because the handle is the Open2
storage-session GUID formatted `storageSessionId.ToString("D").ToUpperInvariant()` (the same handle
format that unlocked GETRP/GETHI/ExeC; see `wcf-string-handle-wall.md`). The Retrieval service
version handshake (`Retr.GetV`) is primed first, as the native client does.
## Why the name-based path (not the TagQuery path)
There are two managed entry points:
- **Index-based** `TagQuery.GetTagExtendedPropertyInfo(start, count, …)` — requires a prior
`StartTagQuery` (`QTB`). On this 2020 box **QTB fails server-side** (`error 1` from
`CMdServer::StartTagQuery::StartActiveTagnamesQuery` over `\\.\pipe\aahMetadataServer\console`),
so this path is dead here regardless of handle format.
- **Name-based** `HistorianAccess.GetTagExtendedPropertiesByName(tagName, fetchFromServer, …)` —
issues `GetTepByNm` directly with the tag name in `tagNames`, no QTB needed. Its second arg
forces a **server fetch** when true; when false the C++ client reads a local cache and returns
`error 41 (Requested item not found)` without any WCF round-trip. The SDK reproduces the
name-based path.
## Wire format (captured)
`scripts/Capture-TagExtendedProperties.ps1` (NativeTraceHarness `tag-extended-properties` scenario +
instrument-wcf-{write,read}message) → decode with `scripts/decode-tag-properties-capture.py`.
Golden-pinned in `WcfTagExtendedPropertyProtocolTests`.
### Request — `tagNames` buffer
```
uint32 count
per name: uint32 charCount + UTF-16LE chars
```
(For one tag: `01 00 00 00` + `LL 00 00 00` + UTF-16 name.)
### Response — `tagExtendedProperties` buffer
```
uint32 tagCount
per tag:
byte groupMarker (observed 0x01)
0x09 + uint16 byteLen + ASCII tagName (compact-ASCII string)
uint32 propertyCount
per property:
byte propMarker (observed 0x02 — likely the value type)
0x09 + uint16 byteLen + ASCII propertyName
0x43 + uint16 payloadLen + uint16 charCount + UTF-16LE value (VT_BSTR CRetVariant)
byte trailingMarker (observed 0x01)
```
`payloadLen` counts the `charCount` field (2 bytes) + the UTF-16 value bytes. Only the string value
variant (`0x43`) is evidence-backed; other variant types throw `ProtocolEvidenceMissingException`
(same discipline as GETRP). The single-byte `0x01`/`0x02`/`0x01` markers are pinned as observed
constants from a single capture; their full semantics are not independently disambiguated.
Example (sanitized — real capture used a dev tag/value):
```
tag "Reactor.Temp1" → property "Location" = "Plant/AreaA"
```
### Paging
`GetTepByNm` is sequence-paged like `GetNextQueryResultBuffer`: call with `sequence = 0`, parse the
buffer, then re-call with the returned `sequence`. A small result returns everything on the first
call; the next call returns an empty/`nil` buffer (with a benign `CClientUtil::FillBufferFromVector`
terminator) — that is the stop signal. The SDK loops until the buffer carries no rows.
## R1.6 (localized properties) — no distinct op on 2020
There is **no** `GetTagLocalizedPropertiesFromName` / `GetTlpByNm` op or
`GetTagLocalizedPropertiesByName` method in `current/aahClientManaged.dll` — the only "localized"
surfaces are `ClientApp.GetLocalizedText` and `SMessageTextMap.GetLocalizedMessage` (error-message /
UI-text localization), not tag properties. So R1.6 **collapses into R1.5**: extended properties
(`GetTepByNm`) are the user-defined tag-property read surface on 2020. R1.6 is closed as
"no separate op," not left throwing.
## R1.12 (localized-property write) — no distinct op on 2020 (mirror of R1.6)
Symmetric to R1.6 on the write side: there is **no** `AddTagLocalizedProperties` /
`DeleteTagLocalizedProperties` (or any `*LocalizedPropert*` / `TagLocalized*`) symbol in **any**
`current/*.dll`. A full symbol sweep of the shipped client DLLs surfaces only `GetLocalizedText`,
`GetLocalizedMessage`, and `LocalizedResourcesDir` — all UI/error-message-text localization, not tag
data. (The sweep does find the real write op `AddTagExtendedProperties` and the whole `AddTag*`
family, so the absence of a localized op is a true negative, not a grep miss.) R1.12 is therefore
**closed as "no separate op"** — the same conclusion as R1.6's read side. Extended-property write
(R1.11 `AddTEx`) is the user-defined tag-property write surface on 2020; localized properties are a
2023 R2 / gRPC-only concept. Not left throwing.
## Shipped surface
- `HistorianClient.GetTagExtendedPropertiesAsync(tag)` → `IReadOnlyList<HistorianTagExtendedProperty>`
(`Name`/`Value` pairs; empty when the tag has none).
- `HistorianTagExtendedPropertyProtocol` (serializer/parser), `HistorianWcfTagExtendedPropertyClient`
(orchestration), golden `WcfTagExtendedPropertyProtocolTests`, gated live
`GetTagExtendedPropertiesAsync_AgainstLocalHistorian_ReturnsProperties` (set `HISTORIAN_TEP_TAG`
to a tag with extended properties).
@@ -0,0 +1,104 @@
<#
.SYNOPSIS
Captures the native AVEVA client's AddTagExtendedProperties / DeleteTagExtendedProperties wire
traffic (HCAL roadmap R1.11) so the AddTEx / DelTep inBuff layout can be decoded, not guessed.
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness's `add-tep` scenario against the live Historian with
an IL-rewritten copy of aahClientManaged.dll whose ClientMessageEncoder.WriteMessage AND
ReadMessage are instrumented. The harness opens a WRITE-enabled connection, creates a sandbox tag
(RetestSdkWrite...), and calls AddTagExtendedProperties(TagExtendedPropertyGroupList, out err) with
one string property (and DeleteTagExtendedPropertiesByName when -Delete).
Decode with scripts/decode-add-tep-capture.py: the WCF.WriteMessage.Body whose op is AddTEx carries
the inBuff (tag name + property name/value); DelTep carries the delete inBuff (tag + property names).
SAFETY: sandbox-guarded — the tag MUST start with 'RetestSdkWrite'. Default run leaves the tag +
property in place (unless -Delete); pass -Delete to also capture DelTep and remove the property.
.NOTES
Artifacts are diagnostic and gitignored. Sanitize before copying into docs/.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
[string]$TepTag = "RetestSdkWriteTepTag",
[string]$PropName = "SdkTestProp",
[string]$PropValue = "SdkTestValue",
[switch]$Delete,
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
if (-not $TepTag.StartsWith("RetestSdkWrite")) { throw "-TepTag must start with 'RetestSdkWrite' (sandbox guard)." }
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-add-tep"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "add-tep-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing add-tep ($TepTag : $PropName=$PropValue) ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "add-tep",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--tep-tag", $TepTag,
"--tep-name", $PropName,
"--tep-value", $PropValue,
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
if ($Delete) { $harnessArgs += "--tep-delete" }
$harnessJson = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} catch {
Write-Host " (add-tep raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
Write-Host "Harness output (AddTagExtendedProperties / Rows):" -ForegroundColor Cyan
$harnessJson | Select-Object -Last 24
Write-Host "`nDecode with: python scripts\decode-add-tep-capture.py" -ForegroundColor Cyan
@@ -0,0 +1,111 @@
<#
.SYNOPSIS
Captures the native AVEVA client's DeleteTagExtendedProperties (DelTep) wire traffic (HCAL R1.11
delete half) using the CROSS-SESSION trick so the delete passes the client-side sync gate.
.DESCRIPTION
DeleteTagExtendedPropertiesByName does a CLIENT-SIDE sync check and returns err 229 ("Tag extended
property not synchronized with server") for any property the local cache doesn't see as
server-synchronized — so a just-added property can't be deleted in the same session and its DelTep
inBuff never reaches the wire. This script captures it in two SEPARATE harness processes (= two
sessions) against one instrumented aahClientManaged.dll:
Run A: add-tep (create sandbox tag + AddTagExtendedProperties) -> property now server-synced
Run B: add-tep --tep-skip-create --tep-skip-add --tep-delete -> fresh connection: fetch the
property from the server (seeds the local cache as SYNCED), then
DeleteTagExtendedPropertiesByName, which now reaches the wire as DelTep.
The capture file is cleared BETWEEN the two runs so it contains only Run B (the GetTepByNm
read-for-sync + the DelTep delete). Decode with scripts/decode-del-tep-capture.py.
SAFETY: sandbox-guarded — the tag MUST start with 'RetestSdkWrite'. The run leaves the sandbox tag
in place (property removed); delete the tag afterwards with the supported aaDeleteTag proc.
.NOTES
Artifacts are diagnostic and gitignored. Sanitize before copying into docs/.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
[string]$TepTag = "RetestSdkWriteDelTepSdk",
[string]$PropName = "SdkDelProp",
[string]$PropValue = "SdkDelValue",
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
if (-not $TepTag.StartsWith("RetestSdkWrite")) { throw "-TepTag must start with 'RetestSdkWrite' (sandbox guard)." }
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-del-tep"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "del-tep-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
function Invoke-Harness([string[]]$extraArgs, [string]$label) {
Write-Host "== $label ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "add-tep",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--tep-tag", $TepTag,
"--tep-name", $PropName,
"--tep-value", $PropValue,
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
) + $extraArgs
$json = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$json = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} finally { $ErrorActionPreference = $prevEap }
$json | Select-Object -Last 20
}
# Run A: create the sandbox tag + add the property (server-synced afterwards).
Invoke-Harness @() "Run A: create + AddTagExtendedProperties ($TepTag : $PropName=$PropValue)"
# Clear the capture so the file contains only Run B (read-for-sync + DelTep).
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
# Run B: FRESH session — fetch (sync the local cache) then DeleteTagExtendedPropertiesByName.
Invoke-Harness @("--tep-skip-create", "--tep-skip-add", "--tep-delete") "Run B: fresh session -> read-for-sync -> DelTep"
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary (Run B only) ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
Write-Host "`nDecode with: python scripts\decode-del-tep-capture.py" -ForegroundColor Cyan
+101
View File
@@ -0,0 +1,101 @@
<#
.SYNOPSIS
Captures the native client's StartEventQuery request bytes WITH and WITHOUT an event filter
(HCAL roadmap R1.7) so the filter-predicate encoding can be decoded against the empty-filter
baseline instead of guessed.
.DESCRIPTION
Drives the NativeTraceHarness `event` scenario against the live Historian under an
IL-rewritten aahClientManaged.dll whose ClientMessageEncoder.WriteMessage is instrumented to
log every outgoing MDAS body. Runs twice:
- baseline : no filter (the known empty-filter StartEventQuery)
- filtered : EventQuery.AddEventFilter("Area", Equal, "RetestFilterArea") before StartQuery
Diff the two StartEventQuery request buffers (scripts/decode-event-filter-capture.py) to read
off the exact filter-block bytes (property name / comparison op / value) the native client
emits, then implement the managed predicate against that.
.NOTES
Artifacts are diagnostic and gitignored. Sanitize before copying into docs/. Never commit raw
capture NDJSON, credentials, hostnames, or customer tag names.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
[int]$LookbackMinutes = 43200,
# Property:Op:Value (Op = a HistorianComparisionType name, e.g. Equal/Contains/GreaterThan).
[string]$Filter = "Area:Equal:RetestFilterArea",
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-event-filter"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
$matrix = @(
@{ Name = "baseline"; Args = @() },
@{ Name = "filtered"; Args = @("--event-filter", $Filter) }
)
foreach ($cfg in $matrix) {
$capturePath = Join-Path $captureDir "event-filter-capture-$($cfg.Name)-latest.ndjson"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing: $($cfg.Name) ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "event",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--lookback-minutes", "$LookbackMinutes",
"--max-rows", "1",
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
) + $cfg.Args
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
& dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1 | Out-Null
} catch {
Write-Host " ($($cfg.Name) raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host " -> $recCount records -> $capturePath"
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
Write-Host "`nDecode with: python scripts\decode-event-filter-capture.py" -ForegroundColor Cyan
+110
View File
@@ -0,0 +1,110 @@
<#
.SYNOPSIS
Captures the native AVEVA client's event-SEND wire traffic (HCAL roadmap R2.1) to
determine whether AddStreamedValue(HistorianEvent) rides the WCF MDAS path (capturable
+ implementable as a pure-managed-WCF SDK op) or the storage-engine shared-memory pipe
(like revision writes — which would block M2 as a WCF SDK).
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness's `event-send` scenario against the live
Historian with an IL-rewritten copy of aahClientManaged.dll whose
ClientMessageEncoder.WriteMessage AND ReadMessage are instrumented to log every MDAS
body (the same pipeline that produced every other proven request/response shape). The
harness opens an Event connection (ReadOnly=false), builds a clearly-marked test
HistorianEvent, calls AddStreamedValue(HistorianEvent), then CloseStorageConnection to
flush the queued event onto the wire.
Decode with scripts/decode-event-send-capture.py: if a StartStorage/AddStreamValues/
EnqueueEventDataPacket body appears on WCF.WriteMessage.Body, M2 is viable over WCF and
the body carries the PackToVtq event value blob to decode (R2.2). If NOTHING event-shaped
appears on the WCF path even though the native AddStreamedValue returned success, the
delivery used the storage-engine pipe and M2 is architecturally blocked over WCF — the
same conclusion as the revision-write path (docs/plans/revision-write-path.md).
.NOTES
Writes a real (clearly-marked) test event into the historian's event history. Artifacts
are diagnostic and gitignored. Sanitize before copying anything into docs/ — never commit
raw capture NDJSON, credentials, hostnames, or customer tag names.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
[string]$EventType = "User.Write",
[int]$FlushSeconds = 6,
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-event-send"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "event-send-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
# Chain via a distinct intermediate file (reading+writing the same path drops the second
# hook on the mixed-mode native image). Final dll carries both hooks with distinct Phase
# strings: WCF.WriteMessage.Body and WCF.ReadMessage.Body.
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing event-send ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "event-send",
"--event-send-confirm",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--event-type", $EventType,
"--event-send-flush-seconds", "$FlushSeconds",
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
$harnessJson = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} catch {
Write-Host " (event-send raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
Write-Host "Harness output (look for AddStreamedEvent Success / ErrorCode):" -ForegroundColor Cyan
$harnessJson | Select-Object -Last 60
Write-Host "`nDecode with: python scripts\decode-event-send-capture.py" -ForegroundColor Cyan
+91
View File
@@ -0,0 +1,91 @@
<#
.SYNOPSIS
Captures the native AVEVA client's ExecuteSqlCommand wire traffic (HCAL roadmap R1.1) so the
Retr.ExeC + Retr.GetR string-handle SQL surface (op names, handle format, command/option
encoding, Retr priming, GetR result byte stream) can be decoded instead of guessed.
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness `exec-sql` scenario against the live Historian
with an IL-rewritten copy of aahClientManaged.dll whose ClientMessageEncoder.WriteMessage AND
ReadMessage are instrumented to log every MDAS body. Read-only benign query.
.NOTES
Artifacts are diagnostic and gitignored. Sanitize before copying into docs/ -- never commit raw
capture NDJSON, credentials, hostnames, or customer tag names.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
[string]$Sql = "SELECT 1 AS ProbeValue",
[string]$SqlOption = "ExecuteRecord",
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-exec-sql"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "exec-sql-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing exec-sql ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "exec-sql",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--sql", $Sql,
"--sql-option", $SqlOption,
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
$harnessJson = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} catch {
Write-Host " (exec-sql raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
$harnessJson | Select-Object -Last 6
+102
View File
@@ -0,0 +1,102 @@
<#
.SYNOPSIS
Captures the native AVEVA client's GetHistorianInfo wire traffic (HCAL roadmap R1.4)
so the WCF GETHI request that returns the FULL HISTORIAN_INFO struct can be decoded
instead of guessed.
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness's `historian-info` scenario against the live
Historian with an IL-rewritten copy of aahClientManaged.dll whose
ClientMessageEncoder.WriteMessage AND ReadMessage are instrumented to log every MDAS body
(the same pipeline that produced every other proven request/response shape). The harness
opens a normal authenticated process connection and calls
HistorianAccess.GetHistorianInfo(out HistorianInfo, out err).
Decode with scripts/decode-historian-info-capture.py: locate the WCF.WriteMessage.Body
whose op is GETHI -> that is the GetHistorianInfo request; read off the leading string
handle and the pRequestBuff layout (distinct from the named-value "HistorianVersion"
request). The paired WCF.ReadMessage.Body is the pResponseBuff = the 518-byte
HISTORIAN_INFO struct (version string @0 UTF-16 null-terminated, EventStorageMode int32 @514).
.NOTES
Read-only status call; no data is written. Artifacts are diagnostic and gitignored.
Sanitize before copying anything into docs/ -- never commit raw capture NDJSON,
credentials, hostnames, or customer tag names.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-historian-info"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "historian-info-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
# Chain via a distinct intermediate file (reading+writing the same path drops the second
# hook on the mixed-mode native image). Final dll carries both hooks with distinct Phase
# strings: WCF.WriteMessage.Body and WCF.ReadMessage.Body.
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing historian-info ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "historian-info",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
$harnessJson = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} catch {
Write-Host " (historian-info raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
Write-Host "Harness output (GetHistorianInfoReturned / HistorianInfo):" -ForegroundColor Cyan
$harnessJson | Select-Object -Last 24
Write-Host "`nDecode with: python scripts\decode-historian-info-capture.py" -ForegroundColor Cyan
+133
View File
@@ -0,0 +1,133 @@
<#
.SYNOPSIS
Captures the native AVEVA client's RenameTags wire traffic (HCAL roadmap R1.10) so the
StJb (StartJob) rename jobBuffer + GtJb (GetJobStatus) response can be decoded instead of guessed.
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness's `rename` scenario against the live Historian
with an IL-rewritten copy of aahClientManaged.dll whose ClientMessageEncoder.WriteMessage AND
ReadMessage are instrumented to log every MDAS body. The harness opens a WRITE-enabled
connection, creates a sandbox source tag (RetestSdkWrite...), calls
HistorianAccess.RenameTags([(from,to)], ref status, out err), and polls GetTagRenameStatus.
Rename maps to the generic job framework: StJb(handle, jobBuffer) -> jobId, then
GtJb(handle, jobId) -> jobStatus. Decode with scripts/decode-rename-capture.py: find the
WCF.WriteMessage.Body whose op is StJb -> its jobBuffer carries the (old,new) name pairs; the
paired ReadMessage carries the jobId; the GtJb request/response carry the status.
SAFETY: sandbox-guarded — both names MUST start with 'RetestSdkWrite'. The default run renames
RetestSdkWriteRenameSrc -> RetestSdkWriteRenameDst and (unless -SkipCleanup) deletes the
destination tag afterward via a second harness pass.
.NOTES
Artifacts are diagnostic and gitignored. Sanitize before copying anything into docs/ --
never commit raw capture NDJSON, credentials, hostnames, or customer tag names.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
[string]$RenameFrom = "RetestSdkWriteRenameSrc",
[string]$RenameTo = "RetestSdkWriteRenameDst",
[switch]$SkipCleanup,
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
if (-not $RenameFrom.StartsWith("RetestSdkWrite") -or -not $RenameTo.StartsWith("RetestSdkWrite")) {
throw "Both -RenameFrom and -RenameTo must start with 'RetestSdkWrite' (sandbox guard)."
}
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-rename"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "rename-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing rename ($RenameFrom -> $RenameTo) ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "rename",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--rename-from", $RenameFrom,
"--rename-to", $RenameTo,
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
$harnessJson = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} catch {
Write-Host " (rename raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
Write-Host "Harness output (RenameTagsReturned / Rows):" -ForegroundColor Cyan
$harnessJson | Select-Object -Last 30
# Best-effort cleanup: delete the destination sandbox tag so reruns start clean.
if (-not $SkipCleanup) {
Write-Host "`n== Cleanup: deleting $RenameTo ==" -ForegroundColor Cyan
$cleanupArgs = @(
"--scenario", "write",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--write-sandbox-tag", $RenameTo,
"--write-skip-add-tag",
"--write-skip-add-value",
"--write-delete-after",
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
try {
$ErrorActionPreference = "Continue"
& dotnet run --no-build -c $Configuration --project $harnessProj -- @cleanupArgs 2>&1 | Select-Object -Last 4
} catch {
Write-Host " (cleanup raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = "Stop"
}
}
Write-Host "`nDecode with: python scripts\decode-rename-capture.py" -ForegroundColor Cyan
+105
View File
@@ -0,0 +1,105 @@
<#
.SYNOPSIS
Captures the native AVEVA client's GetRuntimeParameter wire traffic (HCAL roadmap R1.2)
so the WCF op name, handle type (uint vs the string-handle wall), and the
btRequest/btResponse buffer format can be decoded instead of guessed.
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness's `runtime-param` scenario against the live
Historian with an IL-rewritten copy of aahClientManaged.dll whose
ClientMessageEncoder.WriteMessage AND ReadMessage are instrumented to log every MDAS body
(the same pipeline that produced every other proven request/response shape). The harness
opens a normal authenticated process connection and calls
HistorianAccess.GetRuntimeParameter(List<string> names, out List<object> results, out err).
Decode with scripts/decode-runtime-param-capture.py: locate the WCF.WriteMessage.Body
whose op carries the parameter name(s) -> that is the GetRuntimeParameter request; read
off the SOAP action / op name, the leading handle param, and the btRequest layout. The
paired WCF.ReadMessage.Body is the btResponse (the CRetVariant value list).
.NOTES
Read-only status call; no data is written. Artifacts are diagnostic and gitignored.
Sanitize before copying anything into docs/ -- never commit raw capture NDJSON,
credentials, hostnames, or customer tag names.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
# Semicolon-separated runtime parameter names. HistorianVersion is a known-good name
# (returns the server version string) so the response decode has a real value.
[string]$Names = "HistorianVersion",
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-runtime-param"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "runtime-param-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
# Chain via a distinct intermediate file (reading+writing the same path drops the second
# hook on the mixed-mode native image). Final dll carries both hooks with distinct Phase
# strings: WCF.WriteMessage.Body and WCF.ReadMessage.Body.
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing runtime-param ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "runtime-param",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--runtime-param-names", $Names,
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
$harnessJson = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} catch {
Write-Host " (runtime-param raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
Write-Host "Harness output (GetRuntimeParameterReturned / Results):" -ForegroundColor Cyan
$harnessJson | Select-Object -Last 20
Write-Host "`nDecode with: python scripts\decode-runtime-param-capture.py" -ForegroundColor Cyan
+150
View File
@@ -0,0 +1,150 @@
<#
.SYNOPSIS
Captures the native AVEVA client's StartQuery2 request bytes for analog/state
summary queries (HCAL roadmap R1.8/R1.9) so the managed SDK's summary request
shape can be decoded against ground truth instead of guessed.
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness against the live Historian with an
IL-rewritten copy of aahClientManaged.dll whose ClientMessageEncoder.WriteMessage
is instrumented to log every outgoing MDAS body (the same pipeline that produced
every other proven request shape). For each candidate HistoryQueryArgs config it
writes a per-config NDJSON capture under
artifacts/reverse-engineering/instrumented-wcf-writemessage-summary/ (gitignored).
The default matrix is:
- baseline-full : RetrievalMode=Full (the known-good non-summary request)
- analog-avg : RetrievalMode=Cyclic + ValueSelector=Average + Resolution
- analog-min : RetrievalMode=Cyclic + ValueSelector=Minimum + Resolution
- analog-agg-avg : RetrievalMode=Cyclic + AggregationType=Average + Resolution
- state-summary : RetrievalMode=Cyclic + MaxStates>0 + Resolution
Diff any candidate against baseline-full (scripts/decode-summary-capture.py) to read
off the exact QueryType / SummaryType / AutoSummaryParameters bytes the native client
sets for a summary, then implement the managed request against that.
.NOTES
Artifacts are diagnostic. Sanitize before copying anything into docs/ — never commit
raw capture NDJSON, credentials, hostnames, or customer tag names.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
# SysTimeSec is the local data-bearing system tag (OtOpcUaParityTest_001.Counter is stale/empty).
[string]$TagName = "SysTimeSec",
[int]$LookbackMinutes = 240,
[int]$MaxRows = 4,
# 1-hour summary cycle in 100ns ticks (1h = 36,000,000,000 ticks).
[uint64]$ResolutionTicks = 36000000000,
[string]$Configuration = "Debug",
# Restrict the run to a single named config from the matrix (default: run all).
[string]$OnlyConfig = "",
# Also instrument ReadMessage so each capture includes the incoming WCF response bodies
# (the GetNextQueryResultBuffer2 pResultBuff summary rows). Decoded by decode-summary-response.py.
[switch]$WithResponse
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-writemessage-summary"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage$(if ($WithResponse) { ' + ReadMessage' }) ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
if ($WithResponse) {
# Chain via a distinct intermediate file (reading+writing the same path drops the second
# hook on the mixed-mode native image). Final dll carries both hooks with distinct Phase
# strings: WCF.WriteMessage.Body and WCF.ReadMessage.Body.
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
} else {
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $instrDll | Out-Null
}
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
# Mirror current/ into current-copy, then overwrite the managed dll with the instrumented
# build and drop the strong-named logger assembly alongside it so the injected call binds.
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
# Candidate matrix: name + harness arg list. Summary configs all use Cyclic + a resolution;
# the differentiator is which summary knob is set.
$matrix = @(
@{ Name = "baseline-full"; Args = @("--retrieval-mode", "Full") },
@{ Name = "analog-avg"; Args = @("--retrieval-mode", "Cyclic", "--value-selector", "Average", "--resolution-ticks", "$ResolutionTicks") },
@{ Name = "analog-min"; Args = @("--retrieval-mode", "Cyclic", "--value-selector", "Minimum", "--resolution-ticks", "$ResolutionTicks") },
@{ Name = "analog-max"; Args = @("--retrieval-mode", "Cyclic", "--value-selector", "Maximum", "--resolution-ticks", "$ResolutionTicks") },
@{ Name = "analog-integral"; Args = @("--retrieval-mode", "Cyclic", "--value-selector", "Integral", "--resolution-ticks", "$ResolutionTicks") },
@{ Name = "mode-integral"; Args = @("--retrieval-mode", "Integral", "--resolution-ticks", "$ResolutionTicks") },
@{ Name = "mode-twavg"; Args = @("--retrieval-mode", "TimeWeightedAverage", "--resolution-ticks", "$ResolutionTicks") },
@{ Name = "analog-agg-avg"; Args = @("--retrieval-mode", "Cyclic", "--aggregation-type", "Average", "--resolution-ticks", "$ResolutionTicks") },
@{ Name = "state-summary"; Args = @("--retrieval-mode", "Cyclic", "--max-states", "10", "--resolution-ticks", "$ResolutionTicks") }
)
if ($OnlyConfig) { $matrix = $matrix | Where-Object { $_.Name -eq $OnlyConfig } }
if (-not $matrix) { throw "No matrix entry named '$OnlyConfig'." }
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
$summary = @()
foreach ($cfg in $matrix) {
$name = $cfg.Name
$capturePath = Join-Path $captureDir "summary-capture-$name-latest.ndjson"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing: $name ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "history",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--tag", $TagName,
"--lookback-minutes", "$LookbackMinutes",
"--max-rows", "$MaxRows",
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
) + $cfg.Args
# Don't let a single config that errors (e.g. state summary on an analog tag) abort the
# whole matrix, and don't treat dotnet's stderr noise as a terminating error.
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
& dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1 | Out-Null
} catch {
Write-Host " (config '$name' raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host " -> $recCount records -> $capturePath"
$summary += [pscustomobject]@{ Config = $name; Records = $recCount; Capture = $capturePath }
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
$summary | Format-Table -AutoSize
Write-Host "Decode with: python scripts\decode-summary-capture.py" -ForegroundColor Cyan
+104
View File
@@ -0,0 +1,104 @@
<#
.SYNOPSIS
Captures the native AVEVA client's GetTagExtendedPropertiesFromName (GetTepByNm) wire traffic
(HCAL roadmap R1.5) so the WCF op name, the string-handle format, the tagNames request buffer,
and the extended-property response buffer can be decoded instead of guessed.
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness's tag scenario with --retrieve-extended-properties,
which flips TagQueryArgs.RetrieveTagExtendedPropertyInfo and then calls
TagQuery.GetTagExtendedPropertyInfo(start, count, out TagExtendedPropertyGroupList, out err) —
the managed method that issues the GetTepByNm op. An IL-rewritten copy of aahClientManaged.dll
logs every MDAS body (ClientMessageEncoder.WriteMessage + ReadMessage), the same pipeline that
produced every other proven request/response shape.
Decode with scripts/decode-tag-properties-capture.py: locate the WCF.WriteMessage.Body whose op
is aa/Retr/GetTepByNm -> that is the request (string handle + tagNames buffer + sequence). The
paired WCF.ReadMessage.Body is the extended-property response buffer.
.NOTES
Read-only metadata call; no data is written. Artifacts are diagnostic and gitignored.
Sanitize before copying anything into docs/ -- never commit raw capture NDJSON, credentials,
hostnames, or customer tag names. SysTimeSec is a built-in system tag (safe to name).
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
# A tag (or wildcard) to query extended properties for. SysTimeSec is a built-in system tag
# present on every Historian; override with a real tag that carries extended properties for a
# richer response decode.
[string]$Tag = "SysTimeSec",
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-tag-extended-properties"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "tag-extended-properties-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
# Chain via a distinct intermediate file (reading+writing the same path drops the second hook on
# the mixed-mode native image). Final dll carries both hooks: WCF.WriteMessage.Body + WCF.ReadMessage.Body.
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing tag-extended-properties ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "tag-extended-properties",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--tag", $Tag,
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
$harnessJson = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} catch {
Write-Host " (tag-extended-properties raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
Write-Host "Harness output (TagExtendedProperties Success / Groups):" -ForegroundColor Cyan
$harnessJson | Select-Object -Last 24
Write-Host "`nDecode with: python scripts\decode-tag-properties-capture.py" -ForegroundColor Cyan
+115
View File
@@ -0,0 +1,115 @@
"""Decode the AddTagExtendedProperties / DeleteTagExtendedProperties WCF inBuff (HCAL R1.11).
Reads the capture produced by scripts/Capture-AddTagExtendedProperties.ps1 and locates the AddTEx /
DelTep WriteMessage bodies by the sandbox tag + property name/value, then dumps the inBuff bytes so
the framing (tag name, property count, per-property name + value markers) can be read off. Compare to
the R1.5 read-response encoding in HistorianTagExtendedPropertyProtocol.
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPDIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-add-tep"
CAP = CAPDIR / "add-tep-capture-latest.ndjson"
TAG = "RetestSdkWriteTepTag"
PROP = "SdkTestProp"
VALUE = "SdkTestValue"
OP_ADD = b"AddTEx"
OP_DEL = b"DelTep"
def hexdump(label, buf, base=0):
print(f"=== {label}: {len(buf)} bytes ===")
for off in range(0, len(buf), 16):
c = buf[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {base + off:04X} {hp:<48} |{ap}|")
print()
def ascii_strings(buf, minlen=3):
out, cur, start = [], [], 0
for i, x in enumerate(buf):
if 32 <= x < 127:
if not cur:
start = i
cur.append(chr(x))
else:
if len(cur) >= minlen:
out.append((start, "".join(cur)))
cur = []
if len(cur) >= minlen:
out.append((start, "".join(cur)))
return out
def u16_strings(buf, minlen=3):
out, i = [], 0
while i < len(buf) - 1:
j, chars = i, []
while j < len(buf) - 1 and 32 <= buf[j] < 127 and buf[j + 1] == 0:
chars.append(chr(buf[j]))
j += 2
if len(chars) >= minlen:
out.append((i, "".join(chars)))
i = j
else:
i += 1
return out
def main() -> int:
if not CAP.exists():
print(f"Missing capture: {CAP}\nRun scripts/Capture-AddTagExtendedProperties.ps1 first.")
return 1
records = []
for line in CAP.open(encoding="utf-8-sig"):
if line.strip():
records.append(json.loads(line))
tag_a, prop_a, val_a = TAG.encode("ascii"), PROP.encode("ascii"), VALUE.encode("ascii")
tag_u, prop_u, val_u = TAG.encode("utf-16-le"), PROP.encode("utf-16-le"), VALUE.encode("utf-16-le")
print(f"== {len(records)} MDAS bodies captured ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
flags = []
if OP_ADD in body:
flags.append("AddTEx")
if OP_DEL in body:
flags.append("DelTep")
if prop_a in body or prop_u in body:
flags.append("PROP")
if val_a in body or val_u in body:
flags.append("VALUE")
print(f" [{idx:02d}] {rec.get('Phase'):26s} len={len(body):5d} {','.join(flags)}")
def dump(op):
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
if rec.get("Phase") == "WCF.WriteMessage.Body" and op in body:
hexdump(f"[{idx}] {op.decode()} WriteMessage", body)
print(" UTF-16 strings:")
for off, s in u16_strings(body):
print(f" 0x{off:04X} {s!r}")
print(" ASCII strings:")
for off, s in ascii_strings(body):
print(f" 0x{off:04X} {s!r}")
print()
print("\n== AddTEx request(s) ==")
dump(OP_ADD)
print("\n== DelTep request(s) ==")
dump(OP_DEL)
return 0
if __name__ == "__main__":
sys.exit(main())
+105
View File
@@ -0,0 +1,105 @@
"""Decode the DeleteTagExtendedProperties (DelTep) WCF inBuff (HCAL R1.11 delete half).
Reads the Run-B capture produced by scripts/Capture-DeleteTagExtendedProperties.ps1 and dumps the
DelTep WriteMessage body so the delete-request framing (tag name + property names + the
delete-from-server flag) can be read off and compared to the AddTEx serializer.
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPDIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-del-tep"
CAP = CAPDIR / "del-tep-capture-latest.ndjson"
TAG = "RetestSdkWriteDelTepSdk"
PROP = "SdkDelProp"
OP_DEL = b"DelTep"
OP_GET = b"GetTepByNm"
def hexdump(label, buf, base=0):
print(f"=== {label}: {len(buf)} bytes ===")
for off in range(0, len(buf), 16):
c = buf[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {base + off:04X} {hp:<48} |{ap}|")
print()
def ascii_strings(buf, minlen=3):
out, cur, start = [], [], 0
for i, x in enumerate(buf):
if 32 <= x < 127:
if not cur:
start = i
cur.append(chr(x))
else:
if len(cur) >= minlen:
out.append((start, "".join(cur)))
cur = []
if len(cur) >= minlen:
out.append((start, "".join(cur)))
return out
def u16_strings(buf, minlen=3):
out, i = [], 0
while i < len(buf) - 1:
j, chars = i, []
while j < len(buf) - 1 and 32 <= buf[j] < 127 and buf[j + 1] == 0:
chars.append(chr(buf[j]))
j += 2
if len(chars) >= minlen:
out.append((i, "".join(chars)))
i = j
else:
i += 1
return out
def main() -> int:
if not CAP.exists():
print(f"Missing capture: {CAP}\nRun scripts/Capture-DeleteTagExtendedProperties.ps1 first.")
return 1
records = []
for line in CAP.open(encoding="utf-8-sig"):
if line.strip():
records.append(json.loads(line))
print(f"== {len(records)} MDAS bodies captured (Run B) ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
flags = []
if OP_DEL in body:
flags.append("DelTep")
if OP_GET in body:
flags.append("GetTepByNm")
if TAG.encode("ascii") in body or TAG.encode("utf-16-le") in body:
flags.append("TAG")
if PROP.encode("ascii") in body or PROP.encode("utf-16-le") in body:
flags.append("PROP")
print(f" [{idx:02d}] {rec.get('Phase'):26s} len={len(body):5d} {','.join(flags)}")
print("\n== DelTep request(s) ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
if rec.get("Phase") == "WCF.WriteMessage.Body" and OP_DEL in body:
hexdump(f"[{idx}] DelTep WriteMessage", body)
print(" UTF-16 strings:")
for off, s in u16_strings(body):
print(f" 0x{off:04X} {s!r}")
print(" ASCII strings:")
for off, s in ascii_strings(body):
print(f" 0x{off:04X} {s!r}")
print()
return 0
if __name__ == "__main__":
sys.exit(main())
+113
View File
@@ -0,0 +1,113 @@
"""Decode the StartEventQuery filter-block encoding (HCAL R1.7).
Extracts the `pRequestBuff` from the StartEventQuery WriteMessage body in the baseline
(no filter) and filtered captures produced by scripts/Capture-EventFilter.ps1, dumps both,
and marks where they diverge so the filter predicate (property name / comparison op / value)
can be read off the empty-filter baseline.
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPDIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-event-filter"
PARAM = b"pRequestBuff"
OP = b"StartEventQuery"
def extract_request(path):
if not path.exists():
return None
for line in path.open(encoding="utf-8-sig"):
if not line.strip():
continue
rec = json.loads(line)
if rec.get("Phase") != "WCF.WriteMessage.Body":
continue
body = base64.b64decode(rec["Base64"])
if OP not in body:
continue
i = body.find(PARAM)
if i < 0:
continue
i += len(PARAM)
for s in range(i, min(i + 16, len(body))):
m = body[s]
if m == 0x9E:
return body[s + 2:s + 2 + body[s + 1]]
if m == 0x9F:
n = int.from_bytes(body[s + 1:s + 3], "little")
return body[s + 3:s + 3 + n]
if m == 0xA0:
n = int.from_bytes(body[s + 1:s + 3], "little")
return body[s + 3:s + 3 + n]
return None
def hexdump(label, buf):
print(f"=== {label}: {len(buf)} bytes ===")
for off in range(0, len(buf), 16):
c = buf[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {off:04X} {hp:<48} |{ap}|")
print()
def main() -> int:
base = extract_request(CAPDIR / "event-filter-capture-baseline-latest.ndjson")
filt = extract_request(CAPDIR / "event-filter-capture-filtered-latest.ndjson")
if base is None or filt is None:
print("Missing capture(s). Run scripts/Capture-EventFilter.ps1 first.")
print(f" baseline: {'ok' if base is not None else 'MISSING'}")
print(f" filtered: {'ok' if filt is not None else 'MISSING'}")
return 1
hexdump("baseline (no filter) pRequestBuff", base)
hexdump("filtered pRequestBuff", filt)
# First divergence offset.
n = min(len(base), len(filt))
div = next((i for i in range(n) if base[i] != filt[i]), n)
print(f"== First divergence at offset 0x{div:04X} (lenBase={len(base)} lenFilt={len(filt)}) ==")
print(" Filtered bytes from divergence (the inserted filter block):")
tail = filt[div:]
for off in range(0, len(tail), 16):
c = tail[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {div + off:04X} {hp:<48} |{ap}|")
print("\n== Strings in filtered buffer ==")
for enc, label in ((b"ascii", "ASCII"), (None, "UTF-16LE")):
if enc == b"ascii":
cur, start = [], 0
for i, x in enumerate(filt):
if 32 <= x < 127:
if not cur:
start = i
cur.append(chr(x))
else:
if len(cur) >= 3:
print(f" {label} 0x{start:04X} {''.join(cur)!r}")
cur = []
else:
i = 0
while i < len(filt) - 1:
j, chars = i, []
while j < len(filt) - 1 and 32 <= filt[j] < 127 and filt[j + 1] == 0:
chars.append(chr(filt[j]))
j += 2
if len(chars) >= 3:
print(f" {label} 0x{i:04X} {''.join(chars)!r}")
i = j
else:
i += 1
return 0
if __name__ == "__main__":
sys.exit(main())
+94
View File
@@ -0,0 +1,94 @@
"""Decide whether native event-send (HCAL R2.1) rides WCF or the storage-engine pipe.
Reads the both-hooks capture produced by scripts/Capture-EventSend.ps1 and, for every
outgoing WCF.WriteMessage.Body, tries to recognise the SOAP action / operation name. It
then renders a verdict:
* If a storage/event delivery op (AddStreamValues / EnqueueEventDataPacket /
OpenEventConnection / StartStorage / AddS2 / AddStreamValues2) appears on the WRITE path,
event-send is a WCF op → M2 is implementable over WCF and that body carries the
PackToVtq event value blob to decode (R2.2).
* If NO such op appears on the WRITE path, the queued event was delivered via the
storage-engine shared-memory pipe (not WCF) — M2 is architecturally blocked as a
pure-managed-WCF SDK, the same conclusion as the revision-write path.
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPTURE = (REPO_ROOT / "artifacts" / "reverse-engineering"
/ "instrumented-wcf-event-send" / "event-send-capture-latest.ndjson")
# Operation-name markers we care about. The MDAS binary SOAP body carries the action /
# operation name as readable text (ASCII and/or UTF-16LE). We scan for both encodings.
EVENT_OR_STORAGE_OPS = [
"AddStreamValues2", "AddStreamValues", "EnqueueEventDataPacket", "OpenEventConnection2",
"OpenEventConnection", "StartStorage", "AddS2", "ForwardEventSnapshot", "AddStreamedValue",
"AddNonStreamValues",
]
# Other ops we expect to see on a healthy event-send connection (auth/open/registration),
# printed for context so a "no event op" result is clearly "delivery left WCF", not "nothing ran".
KNOWN_OPS = [
"GetV", "ValCl", "Open2", "OpenConnection", "GETHI", "GetSystemParameter",
"UpdC3", "UpdateClientStatus3", "RTag2", "RegisterTags2", "EnsT2", "EnsureTags2",
"IsOriginalAllowed", "StartQuery", "GetInterfaceVersion",
]
def find_ops(body, candidates):
hits = []
for op in candidates:
a = op.encode("ascii")
u = op.encode("utf-16-le")
if a in body or u in body:
hits.append(op)
return hits
def main() -> int:
if not CAPTURE.exists():
print(f"Capture not found: {CAPTURE}")
print("Run: scripts/Capture-EventSend.ps1")
return 1
with CAPTURE.open(encoding="utf-8-sig") as fh:
records = [json.loads(line) for line in fh if line.strip()]
writes = [r for r in records if r.get("Phase") == "WCF.WriteMessage.Body"]
reads = [r for r in records if r.get("Phase") == "WCF.ReadMessage.Body"]
print(f"Records: {len(records)} (write={len(writes)} read={len(reads)})\n")
event_write_hits = []
print("== Outgoing WCF.WriteMessage.Body ops ==")
for i, r in enumerate(writes):
body = base64.b64decode(r["Base64"])
known = find_ops(body, KNOWN_OPS)
event = find_ops(body, EVENT_OR_STORAGE_OPS)
if event:
event_write_hits.extend(event)
label = ", ".join(event + known) or "<no recognized op>"
flag = " <<< EVENT/STORAGE OP" if event else ""
print(f" write[{i:02d}] {len(body):6d}B {label}{flag}")
print("\n== Verdict ==")
if event_write_hits:
uniq = sorted(set(event_write_hits))
print(f" EVENT/STORAGE op(s) on the WCF WRITE path: {uniq}")
print(" => event-send IS a WCF op. M2 viable over WCF; decode the PackToVtq value")
print(" blob in that body for R2.2.")
return 0
print(" NO event/storage delivery op on the WCF WRITE path.")
print(" => the queued event did NOT leave via WCF. If the native AddStreamedValue")
print(" returned success (see harness JSON), delivery used the storage-engine")
print(" shared-memory pipe — M2 is blocked as a pure-managed-WCF SDK, same as the")
print(" revision-write path (docs/plans/revision-write-path.md).")
return 0
if __name__ == "__main__":
sys.exit(main())
+114
View File
@@ -0,0 +1,114 @@
"""Decode the AddS2 (AddStreamValues2) pBuf event-VTQ blob captured for event-send (R2.2).
Extracts the `pBuf` parameter from the AddS2 WriteMessage body in the event-send capture
and hex-dumps it, annotating windows that match the known test event so the
HistorianEvent.PackToVtq framing can be read off and inverted into a managed serializer.
Known test event (from scripts/Capture-EventSend.ps1 defaults):
Type="User.Write", Namespace="RetestSdkEventSend",
properties: Source="RetestSdkEventSend", TestMarker="histsdk-R2.1-capture"
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import struct
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPTURE = (REPO_ROOT / "artifacts" / "reverse-engineering"
/ "instrumented-wcf-event-send" / "event-send-capture-latest.ndjson")
PARAM = b"pBuf"
ADDS2 = b"AddS2"
def extract_param(body, param):
i = body.find(param)
if i < 0:
return None
i += len(param)
# Skip the closing of the element name / attributes until a binary length marker.
# MDAS length markers: 0x9E (1-byte len), 0x9F (2-byte len), 0xA0 (2-byte len+1).
for scan in range(i, min(i + 16, len(body))):
marker = body[scan]
if marker == 0x9E:
length = body[scan + 1]
return body[scan + 2:scan + 2 + length]
if marker == 0x9F:
length = int.from_bytes(body[scan + 1:scan + 3], "little")
return body[scan + 3:scan + 3 + length]
if marker == 0xA0:
length = int.from_bytes(body[scan + 1:scan + 3], "little")
return body[scan + 3:scan + 3 + length + 1]
return None
def main() -> int:
if not CAPTURE.exists():
print(f"Capture not found: {CAPTURE}")
return 1
with CAPTURE.open(encoding="utf-8-sig") as fh:
records = [json.loads(line) for line in fh if line.strip()]
body = None
for r in records:
if r.get("Phase") != "WCF.WriteMessage.Body":
continue
b = base64.b64decode(r["Base64"])
if ADDS2 in b:
body = b
break
if body is None:
print("No AddS2 WriteMessage body found.")
return 2
pbuf = extract_param(body, PARAM)
if pbuf is None:
print("Found AddS2 body but could not extract pBuf. Full body hex dump:")
pbuf = body
print(f"pBuf: {len(pbuf)} bytes\n")
for off in range(0, len(pbuf), 16):
chunk = pbuf[off:off + 16]
hp = " ".join(f"{c:02X}" for c in chunk)
ap = "".join(chr(c) if 32 <= c < 127 else "." for c in chunk)
print(f" {off:04X} {hp:<48} |{ap}|")
print("\n== ASCII strings (len>=3) ==")
cur = []
start = 0
for i, c in enumerate(pbuf):
if 32 <= c < 127:
if not cur:
start = i
cur.append(chr(c))
else:
if len(cur) >= 3:
print(f" 0x{start:04X} {''.join(cur)!r}")
cur = []
if len(cur) >= 3:
print(f" 0x{start:04X} {''.join(cur)!r}")
print("\n== UTF-16LE strings (len>=3) ==")
i = 0
while i < len(pbuf) - 1:
j = i
chars = []
while j < len(pbuf) - 1 and 32 <= pbuf[j] < 127 and pbuf[j + 1] == 0:
chars.append(chr(pbuf[j]))
j += 2
if len(chars) >= 3:
print(f" 0x{i:04X} {''.join(chars)!r}")
i = j
else:
i += 1
return 0
if __name__ == "__main__":
sys.exit(main())
+142
View File
@@ -0,0 +1,142 @@
"""Decode the GetHistorianInfo (GETHI) WCF request/response (HCAL R1.4).
Reads the chained WriteMessage+ReadMessage capture produced by
scripts/Capture-HistorianInfo.ps1 and locates the GetHistorianInfo exchange. The goal is
to learn (a) the pRequestBuff that returns the FULL HISTORIAN_INFO struct (distinct from the
named-value "HistorianVersion" request) and (b) the response struct layout: the analysis
folder says it's 518 bytes with the version string (UTF-16, null-terminated) at offset 0 and
EventStorageMode (int32) at offset 514.
We flag candidate bodies by the GETHI op action, by the server version value, and by a
response length near 518, then dump bytes + the int32 at offset 514 so the layout can be
read off directly.
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import struct
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPDIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-historian-info"
CAP = CAPDIR / "historian-info-capture-latest.ndjson"
# The GETHI op action (WS-Addressing) the native client sends. The server version value is
# version-shaped, not secret; used only to locate the response.
OP_ASCII = b"GetHistorianInfo"
OP_GETHI = b"GETHI"
VERSION = "20,0,000,000"
VERSION_U16 = VERSION.encode("utf-16-le")
VERSION_ASCII = VERSION.encode("ascii")
def hexdump(label, buf, base=0):
print(f"=== {label}: {len(buf)} bytes ===")
for off in range(0, len(buf), 16):
c = buf[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {base + off:04X} {hp:<48} |{ap}|")
print()
def ascii_strings(buf, minlen=3):
out, cur, start = [], [], 0
for i, x in enumerate(buf):
if 32 <= x < 127:
if not cur:
start = i
cur.append(chr(x))
else:
if len(cur) >= minlen:
out.append((start, "".join(cur)))
cur = []
if len(cur) >= minlen:
out.append((start, "".join(cur)))
return out
def u16_strings(buf, minlen=3):
out, i = [], 0
while i < len(buf) - 1:
j, chars = i, []
while j < len(buf) - 1 and 32 <= buf[j] < 127 and buf[j + 1] == 0:
chars.append(chr(buf[j]))
j += 2
if len(chars) >= minlen:
out.append((i, "".join(chars)))
i = j
else:
i += 1
return out
def main() -> int:
if not CAP.exists():
print(f"Missing capture: {CAP}\nRun scripts/Capture-HistorianInfo.ps1 first.")
return 1
records = []
for line in CAP.open(encoding="utf-8-sig"):
if line.strip():
records.append(json.loads(line))
print(f"== {len(records)} MDAS bodies captured ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
flags = []
if OP_ASCII in body or OP_GETHI in body:
flags.append("GETHI-OP")
if VERSION_U16 in body or VERSION_ASCII in body:
flags.append("VERSION")
# A ~518-byte embedded struct is the tell for the full-info response.
if 500 <= len(body) <= 4096:
flags.append(f"len={len(body)}")
print(f" [{idx:02d}] {rec.get('Phase'):26s} len={len(body):5d} {','.join(flags)}")
def find(predicate):
hits = []
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
if predicate(rec, body):
hits.append((idx, rec, body))
return hits
print("\n== Request candidate(s): WriteMessage bodies tagged GETHI-OP ==")
for idx, rec, body in find(lambda r, b: r.get("Phase") == "WCF.WriteMessage.Body"
and (OP_ASCII in b or OP_GETHI in b)):
hexdump(f"[{idx}] WriteMessage", body)
print(" UTF-16 strings:")
for off, s in u16_strings(body):
print(f" 0x{off:04X} {s!r}")
print(" ASCII strings:")
for off, s in ascii_strings(body):
print(f" 0x{off:04X} {s!r}")
print()
print("\n== Response candidate(s): ReadMessage bodies carrying VERSION ==")
for idx, rec, body in find(lambda r, b: r.get("Phase") == "WCF.ReadMessage.Body"
and (VERSION_U16 in b or VERSION_ASCII in b)):
hexdump(f"[{idx}] ReadMessage", body)
print(" UTF-16 strings:")
for off, s in u16_strings(body):
print(f" 0x{off:04X} {s!r}")
# The analysis folder pins EventStorageMode @ offset 514 (int32) inside the
# 518-byte struct. The struct is embedded in the MDAS body at some base; scan for
# a plausible version@0 run and print the int32 514 bytes after each candidate base.
print(" Candidate struct decodes (version@base, int32 @ base+514):")
for base_off, s in u16_strings(body):
if any(ch.isdigit() for ch in s) and "," in s:
idx514 = base_off + 514
if idx514 + 4 <= len(body):
mode = struct.unpack_from("<i", body, idx514)[0]
print(f" base=0x{base_off:04X} version={s!r} int32@+514={mode}")
print()
return 0
if __name__ == "__main__":
sys.exit(main())
+129
View File
@@ -0,0 +1,129 @@
"""Decode the RenameTags WCF request/response (HCAL R1.10).
Reads the chained WriteMessage+ReadMessage capture produced by scripts/Capture-RenameTags.ps1
and locates the rename exchange. Rename maps to the generic job framework:
StJb (StartJob): WriteMessage carries op "StJb" + a string handle + the rename jobBuffer
(the (old,new) name pairs). ReadMessage carries the returned jobId string.
GtJb (GetJobStatus): WriteMessage carries op "GtJb" + handle + jobId. ReadMessage carries
the job-status buffer.
We flag bodies by the StJb/GtJb op and by the sandbox names, then dump the buffers so the
jobBuffer layout (batch count + old/new UTF-16 framing) can be read off directly.
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPDIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-rename"
CAP = CAPDIR / "rename-capture-latest.ndjson"
# Sandbox names used by the default capture run (not secret).
FROM = "RetestSdkWriteRenameSrc"
TO = "RetestSdkWriteRenameDst"
OP_STJB = b"StJb"
OP_GTJB = b"GtJb"
def hexdump(label, buf, base=0):
print(f"=== {label}: {len(buf)} bytes ===")
for off in range(0, len(buf), 16):
c = buf[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {base + off:04X} {hp:<48} |{ap}|")
print()
def ascii_strings(buf, minlen=3):
out, cur, start = [], [], 0
for i, x in enumerate(buf):
if 32 <= x < 127:
if not cur:
start = i
cur.append(chr(x))
else:
if len(cur) >= minlen:
out.append((start, "".join(cur)))
cur = []
if len(cur) >= minlen:
out.append((start, "".join(cur)))
return out
def u16_strings(buf, minlen=3):
out, i = [], 0
while i < len(buf) - 1:
j, chars = i, []
while j < len(buf) - 1 and 32 <= buf[j] < 127 and buf[j + 1] == 0:
chars.append(chr(buf[j]))
j += 2
if len(chars) >= minlen:
out.append((i, "".join(chars)))
i = j
else:
i += 1
return out
def main() -> int:
if not CAP.exists():
print(f"Missing capture: {CAP}\nRun scripts/Capture-RenameTags.ps1 first.")
return 1
records = []
for line in CAP.open(encoding="utf-8-sig"):
if line.strip():
records.append(json.loads(line))
from_u16, to_u16 = FROM.encode("utf-16-le"), TO.encode("utf-16-le")
from_a, to_a = FROM.encode("ascii"), TO.encode("ascii")
print(f"== {len(records)} MDAS bodies captured ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
flags = []
if OP_STJB in body:
flags.append("StJb")
if OP_GTJB in body:
flags.append("GtJb")
if from_u16 in body or from_a in body:
flags.append("FROM")
if to_u16 in body or to_a in body:
flags.append("TO")
print(f" [{idx:02d}] {rec.get('Phase'):26s} len={len(body):5d} {','.join(flags)}")
def find(predicate):
hits = []
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
if predicate(rec, body):
hits.append((idx, rec, body))
return hits
print("\n== StJb request(s): WriteMessage bodies tagged StJb ==")
for idx, rec, body in find(lambda r, b: r.get("Phase") == "WCF.WriteMessage.Body" and OP_STJB in b):
hexdump(f"[{idx}] StJb WriteMessage", body)
print(" UTF-16 strings:")
for off, s in u16_strings(body):
print(f" 0x{off:04X} {s!r}")
print(" ASCII strings:")
for off, s in ascii_strings(body):
print(f" 0x{off:04X} {s!r}")
print()
print("\n== StJb / GtJb response(s) + GtJb request(s) ==")
for idx, rec, body in find(lambda r, b: (OP_STJB in b or OP_GTJB in b) and r.get("Phase") == "WCF.ReadMessage.Body"):
hexdump(f"[{idx}] {rec.get('Phase')}", body)
print(" strings:", [s for _, s in ascii_strings(body)][:16])
print()
return 0
if __name__ == "__main__":
sys.exit(main())
+128
View File
@@ -0,0 +1,128 @@
"""Decode the GetRuntimeParameter WCF request/response (HCAL R1.2).
Reads the chained WriteMessage+ReadMessage capture produced by
scripts/Capture-RuntimeParam.ps1 and locates the GetRuntimeParameter exchange by
searching every MDAS body for the parameter name (UTF-16) on the request side and the
returned value on the response side. Dumps the surrounding bytes so the op name, the
leading handle parameter, and the btRequest/btResponse buffer layout can be read off.
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPDIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-runtime-param"
CAP = CAPDIR / "runtime-param-capture-latest.ndjson"
# Markers we expect on the wire for the default "HistorianVersion" capture.
NAME = "HistorianVersion"
NAME_U16 = NAME.encode("utf-16-le")
NAME_ASCII = NAME.encode("ascii")
VALUE = "20,0,000,000" # server runtime "HistorianVersion" value (version-shaped, not secret)
VALUE_U16 = VALUE.encode("utf-16-le")
VALUE_ASCII = VALUE.encode("ascii")
def hexdump(label, buf, base=0):
print(f"=== {label}: {len(buf)} bytes ===")
for off in range(0, len(buf), 16):
c = buf[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {base + off:04X} {hp:<48} |{ap}|")
print()
def ascii_strings(buf, minlen=3):
out, cur, start = [], [], 0
for i, x in enumerate(buf):
if 32 <= x < 127:
if not cur:
start = i
cur.append(chr(x))
else:
if len(cur) >= minlen:
out.append((start, "".join(cur)))
cur = []
if len(cur) >= minlen:
out.append((start, "".join(cur)))
return out
def u16_strings(buf, minlen=3):
out, i = [], 0
while i < len(buf) - 1:
j, chars = i, []
while j < len(buf) - 1 and 32 <= buf[j] < 127 and buf[j + 1] == 0:
chars.append(chr(buf[j]))
j += 2
if len(chars) >= minlen:
out.append((i, "".join(chars)))
i = j
else:
i += 1
return out
def main() -> int:
if not CAP.exists():
print(f"Missing capture: {CAP}\nRun scripts/Capture-RuntimeParam.ps1 first.")
return 1
records = []
for line in CAP.open(encoding="utf-8-sig"):
if line.strip():
records.append(json.loads(line))
print(f"== {len(records)} MDAS bodies captured ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
flags = []
if NAME_U16 in body or NAME_ASCII in body:
flags.append("NAME")
if VALUE_U16 in body or VALUE_ASCII in body:
flags.append("VALUE")
# The WS-Addressing action is the most reliable op label; show any string that
# looks like an op (contains a slash or is short and capitalized).
print(f" [{idx:02d}] {rec.get('Phase'):26s} len={len(body):5d} {','.join(flags)}")
def find(predicate):
hits = []
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
if predicate(rec, body):
hits.append((idx, rec, body))
return hits
print("\n== Request candidate(s): WriteMessage bodies containing the NAME ==")
for idx, rec, body in find(lambda r, b: r.get("Phase") == "WCF.WriteMessage.Body"
and (NAME_U16 in b or NAME_ASCII in b)):
hexdump(f"[{idx}] WriteMessage", body)
print(" UTF-16 strings:")
for off, s in u16_strings(body):
print(f" 0x{off:04X} {s!r}")
print(" ASCII strings:")
for off, s in ascii_strings(body):
print(f" 0x{off:04X} {s!r}")
print()
print("\n== Response candidate(s): ReadMessage bodies containing the VALUE ==")
for idx, rec, body in find(lambda r, b: r.get("Phase") == "WCF.ReadMessage.Body"
and (VALUE_U16 in b or VALUE_ASCII in b)):
hexdump(f"[{idx}] ReadMessage", body)
print(" UTF-16 strings:")
for off, s in u16_strings(body):
print(f" 0x{off:04X} {s!r}")
print(" ASCII strings:")
for off, s in ascii_strings(body):
print(f" 0x{off:04X} {s!r}")
print()
return 0
if __name__ == "__main__":
sys.exit(main())
+115
View File
@@ -0,0 +1,115 @@
"""Decoder for the analog/state summary request capture (HCAL roadmap R1.8/R1.9).
Reads the per-config NDJSON captures produced by scripts/Capture-SummaryRequest.ps1
under artifacts/reverse-engineering/instrumented-wcf-writemessage-summary/, extracts
the Retr/StartQuery2 `pRequestBuff` payload from each, hex-dumps it, and diffs every
summary candidate against the baseline-full request so the differing bytes (the native
QueryType / SummaryType / AutoSummaryParameters fields) stand out.
Output is diagnostic. The only printed strings are the SDK-chosen system tag name and
protocol field markers — sanitize before copying any of it into docs/.
"""
import base64
import json
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPTURE_DIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-writemessage-summary"
ACTION = b"aa/Retr/StartQuery2"
PARAM = b"pRequestBuff"
def extract_request_buffer(records):
"""Return the pRequestBuff bytes from the first StartQuery2 write record, or None."""
for rec in records:
if rec.get("Phase") != "WCF.WriteMessage.Body":
continue
body = base64.b64decode(rec["Base64"])
if ACTION not in body:
continue
i = body.find(PARAM)
if i < 0:
continue
i += len(PARAM)
marker = body[i]
# MDAS length markers (same scheme as the write decoder).
if marker == 0x9E:
length = body[i + 1]
return body[i + 2:i + 2 + length]
if marker == 0x9F:
length = int.from_bytes(body[i + 1:i + 3], "little")
return body[i + 3:i + 3 + length]
if marker == 0xA0:
length = int.from_bytes(body[i + 1:i + 3], "little")
return body[i + 3:i + 3 + length + 1]
return None
return None
def hexdump(payload, diff_against=None):
for off in range(0, len(payload), 16):
chunk = payload[off:off + 16]
cells = []
for j, c in enumerate(chunk):
mark = ""
if diff_against is not None:
k = off + j
if k >= len(diff_against) or diff_against[k] != c:
mark = "*"
cells.append(f"{c:02X}{mark}")
hp = " ".join(cells)
ap = "".join(chr(c) if 32 <= c < 127 else "." for c in chunk)
print(f" {off:04X} {hp:<56} |{ap}|")
def load(path):
with path.open(encoding="utf-8-sig") as fh:
return [json.loads(line) for line in fh if line.strip()]
def main() -> int:
if not CAPTURE_DIR.exists():
print(f"Capture dir not found: {CAPTURE_DIR}")
print("Run scripts/Capture-SummaryRequest.ps1 first.")
return 1
captures = sorted(CAPTURE_DIR.glob("summary-capture-*-latest.ndjson"))
if not captures:
print(f"No capture files in {CAPTURE_DIR}")
return 1
buffers = {}
for path in captures:
name = path.stem.replace("summary-capture-", "").replace("-latest", "")
records = load(path)
buf = extract_request_buffer(records)
buffers[name] = buf
status = f"{len(buf)} bytes" if buf else "<no StartQuery2 request found>"
print(f"{name:<18} records={len(records):>3} pRequestBuff={status}")
baseline = buffers.get("baseline-full")
print()
if not baseline:
print("No baseline-full request buffer captured; cannot diff. Dumping each raw.")
for name, buf in buffers.items():
if buf:
print(f"\n== {name} pRequestBuff ({len(buf)} bytes) ==")
hexdump(buf)
return 0
print(f"== baseline-full pRequestBuff ({len(baseline)} bytes) ==")
hexdump(baseline)
for name, buf in buffers.items():
if name == "baseline-full" or not buf:
continue
print(f"\n== {name} pRequestBuff ({len(buf)} bytes) — '*' marks bytes differing from baseline ==")
hexdump(buf, diff_against=baseline)
return 0
if __name__ == "__main__":
sys.exit(main())
+123
View File
@@ -0,0 +1,123 @@
"""Decode the GetNextQueryResultBuffer2 *response* for an analog summary (HCAL R1.8).
Reads the both-hooks capture produced by
scripts/Capture-SummaryRequest.ps1 -OnlyConfig analog-avg -WithResponse
finds the ReadMessage record carrying GetNextQueryResultBuffer2Response, extracts the
`pResultBuff` payload, hex-dumps it, and annotates every 8-byte window that decodes to a
known ground-truth value (the AnalogSummaryHistory row for SysTimeSec) so the field offsets
of CAnalogSummaryValue can be read off directly.
Output is diagnostic; the only printed strings are the SDK-chosen system tag name and field
markers. Sanitize before copying into docs/.
"""
import base64
import json
import struct
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
# Config name (analog-avg / analog-min / analog-max / …) selectable via argv[1].
CONFIG = sys.argv[1] if len(sys.argv) > 1 else "analog-avg"
CAPTURE = (REPO_ROOT / "artifacts" / "reverse-engineering"
/ "instrumented-wcf-writemessage-summary" / f"summary-capture-{CONFIG}-latest.ndjson")
RESP = b"GetNextQueryResultBuffer2Response"
PARAM = b"pResultBuff"
# Ground-truth values from AnalogSummaryHistory(SysTimeSec, 1h cycle) — used to label offsets.
KNOWN_DOUBLES = {
31.0: "31.0 (First/Last/Average)",
100.0: "100.0 (PercentGood)",
0.031: "0.031 (Integral)",
111600.0: "111600.0 (Integral, full-cycle)",
1.0: "1.0 (ValueCount as double?)",
}
KNOWN_U32 = {
1: "ValueCount=1",
192: "OPCQuality=192",
100: "PercentGood=100",
9: "version=9",
}
def extract_param(body, param):
i = body.find(param)
if i < 0:
return None
i += len(param)
marker = body[i]
if marker == 0x9E:
length = body[i + 1]
return body[i + 2:i + 2 + length]
if marker == 0x9F:
length = int.from_bytes(body[i + 1:i + 3], "little")
return body[i + 3:i + 3 + length]
if marker == 0xA0:
length = int.from_bytes(body[i + 1:i + 3], "little")
return body[i + 3:i + 3 + length + 1]
return None
def main() -> int:
if not CAPTURE.exists():
print(f"Capture not found: {CAPTURE}")
print("Run: scripts/Capture-SummaryRequest.ps1 -OnlyConfig analog-avg -WithResponse")
return 1
with CAPTURE.open(encoding="utf-8-sig") as fh:
records = [json.loads(line) for line in fh if line.strip()]
payload = None
for rec in records:
if rec.get("Phase") != "WCF.ReadMessage.Body":
continue
body = base64.b64decode(rec["Base64"])
if RESP not in body:
continue
payload = extract_param(body, PARAM)
break
if payload is None:
print("No GetNextQueryResultBuffer2Response / pResultBuff found in capture.")
return 2
print(f"pResultBuff: {len(payload)} bytes")
if len(payload) >= 6:
version = int.from_bytes(payload[0:2], "little")
row_count = int.from_bytes(payload[2:6], "little")
print(f" header: version={version} rowCount={row_count}")
print()
# Annotated hex dump.
for off in range(0, len(payload), 16):
chunk = payload[off:off + 16]
hp = " ".join(f"{c:02X}" for c in chunk)
ap = "".join(chr(c) if 32 <= c < 127 else "." for c in chunk)
print(f" {off:04X} {hp:<48} |{ap}|")
# Scan every 8-byte window for known doubles, and every 4-byte window for known u32s.
print("\n== Known-value hits (offset -> field) ==")
for off in range(0, len(payload) - 7):
val = struct.unpack_from("<d", payload, off)[0]
for known, label in KNOWN_DOUBLES.items():
if val == known or (known != 0 and abs(val - known) < 1e-9 * max(1.0, abs(known))):
print(f" 0x{off:04X} double {val!r:>14} -> {label}")
for off in range(0, len(payload) - 3):
val = int.from_bytes(payload[off:off + 4], "little")
if val in KNOWN_U32:
print(f" 0x{off:04X} uint32 {val:>14} -> {KNOWN_U32[val]}")
# FILETIME windows (plausible 2026 timestamps: 0x01DC.. high dword).
print("\n== Plausible FILETIME windows (Int64, year ~2020-2030) ==")
for off in range(0, len(payload) - 7):
ft = int.from_bytes(payload[off:off + 8], "little")
# FILETIME for 2020-01-01 ~= 0x01D5BF.. ; 2030 ~= 0x01E5.. — gate by high word.
if 0x01D5_0000_0000_0000 <= ft <= 0x01E6_0000_0000_0000:
print(f" 0x{off:04X} filetime 0x{ft:016X}")
return 0
if __name__ == "__main__":
sys.exit(main())
+75
View File
@@ -0,0 +1,75 @@
"""Decode the GetTagExtendedPropertiesFromName (GetTepByNm) WCF request/response (HCAL R1.5).
Reads the chained WriteMessage+ReadMessage capture produced by
scripts/Capture-TagExtendedProperties.ps1, locates the aa/Retr/GetTepByNm exchange, and
dumps the tagNames request buffer + tagExtendedProperties response buffer so the op name,
the uppercase string handle, the tagNames layout, and the extended-property response layout
can be read off.
Request tagNames buffer:
uint32 count + per name: uint32 charCount + UTF-16LE chars.
Response tagExtendedProperties buffer:
uint32 tagCount
per tag: byte marker(0x01) + compact-ASCII tagName(0x09 + uint16 len + ascii)
uint32 propCount
per prop: byte marker(0x02) + compact-ASCII propName(0x09 + uint16 len + ascii)
value: 0x43 (VT_BSTR) + uint16 payloadLen + uint16 charCount + UTF-16LE
trailing byte(0x01)
Output is diagnostic. Sanitize before copying into docs/ (tag names / values are dev data).
"""
import base64
import json
import re
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPDIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-tag-extended-properties"
DEFAULT_CAP = CAPDIR / "tep-localized-capture.ndjson"
ACTION = re.compile(rb"aa/[A-Za-z0-9]+/[A-Za-z0-9_]+")
def hexdump(label, buf):
print(f"=== {label}: {len(buf)} bytes ===")
for off in range(0, len(buf), 16):
c = buf[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {off:04X} {hp:<48} |{ap}|")
print()
def main() -> int:
cap = Path(sys.argv[1]) if len(sys.argv) > 1 else DEFAULT_CAP
if not cap.exists():
print(f"Missing capture: {cap}\nRun scripts/Capture-TagExtendedProperties.ps1 -Localized first.")
return 1
records = [json.loads(l) for l in cap.open(encoding="utf-8-sig") if l.strip()]
print(f"== {len(records)} MDAS bodies captured ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
acts = sorted({m.decode() for m in ACTION.findall(body)})
flag = " <== GetTepByNm" if any("Tep" in a for a in acts) else ""
print(f" [{idx:02d}] {rec.get('Phase'):24s} len={len(body):5d} {acts}{flag}")
print("\n== GetTepByNm request(s) [WriteMessage] ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
if rec.get("Phase") == "WCF.WriteMessage.Body" and b"GetTepByNm" in body:
hexdump(f"[{idx}] request", body)
print("\n== GetTepByNm response(s) [ReadMessage] ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
if rec.get("Phase") == "WCF.ReadMessage.Body" and b"GetTepByNmResponse" in body:
hexdump(f"[{idx}] response", body)
return 0
if __name__ == "__main__":
sys.exit(main())
@@ -7,6 +7,7 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="System.Formats.Nrbf" Version="10.0.7" />
<PackageReference Include="System.Security.Cryptography.Xml" Version="10.0.7" />
<PackageReference Include="System.ServiceModel.NetNamedPipe" Version="10.0.652802" />
<PackageReference Include="System.ServiceModel.NetTcp" Version="10.0.652802" />
@@ -0,0 +1,94 @@
using Google.Protobuf;
using Grpc.Core;
using AVEVA.Historian.Client.Wcf;
using GrpcHistory = ArchestrA.Grpc.Contract.History;
using GrpcStorage = ArchestrA.Grpc.Contract.Storage;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// Shared 2023 R2 gRPC authentication handshake. Opens an authenticated History session over an
/// existing <see cref="HistorianGrpcConnection"/> and returns the transient client handle used by
/// the Retrieval/Status services. Extracted from <see cref="HistorianGrpcReadOrchestrator"/> so the
/// read, status, and (future) browse/metadata gRPC paths all drive the identical chain:
/// <c>HistoryService.GetInterfaceVersion → StorageService.ValidateClientCredential (token loop) →
/// HistoryService.OpenConnection</c>. The byte payloads (OpenConnection3 v6 request, NTLM token
/// framing) are the proven 2020 protocol and transfer unchanged inside protobuf <c>bytes</c> fields.
///
/// See <see cref="HistorianGrpcReadOrchestrator"/> for the op-routing rationale (the Negotiate loop
/// belongs on StorageService.ValidateClientCredential, NOT HistoryService.ExchangeKey).
/// </summary>
internal static class HistorianGrpcHandshake
{
/// <summary>
/// The handles produced by a successful OpenConnection. <see cref="ClientHandle"/> is the
/// transient <c>uint</c> session token used by StartQuery/GetSystemParameter and the other
/// uint-handle ops. <see cref="StorageSessionId"/> is the storage-session GUID used (formatted
/// uppercase via <see cref="StringHandle"/>) by the string-handle ops
/// (GetTagInfosFromName, GetTagExtendedPropertiesFromName, ExecuteSqlCommand, ...).
/// </summary>
internal readonly record struct Session(uint ClientHandle, Guid StorageSessionId)
{
/// <summary>The storage GUID in the uppercase "D" form the native string-handle ops require.</summary>
public string StringHandle => StorageSessionId.ToString("D").ToUpperInvariant();
}
/// <summary>Convenience overload for callers that only need the uint client handle.</summary>
public static uint OpenAuthenticatedConnection(
HistorianGrpcConnection connection,
HistorianClientOptions options,
CancellationToken cancellationToken)
=> OpenSession(connection, options, cancellationToken).ClientHandle;
public static Session OpenSession(
HistorianGrpcConnection connection,
HistorianClientOptions options,
CancellationToken cancellationToken)
{
DateTime Deadline() => DateTime.UtcNow.Add(options.RequestTimeout);
Guid contextKey = Guid.NewGuid();
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
GrpcHistory.GetInterfaceVersionResponse historyVersion = historyClient.GetInterfaceVersion(
new GrpcHistory.GetInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken);
HistorianServerVersionGate.Validate(HistorianServiceInterface.History, historyVersion.UiVersion, options);
var storageClient = new GrpcStorage.StorageService.StorageServiceClient(connection.Channel);
HistorianNativeHandshake.RunTokenRounds(
(handle, wrapped, _) =>
{
GrpcStorage.ValidateClientCredentialResponse response = storageClient.ValidateClientCredential(
new GrpcStorage.ValidateClientCredentialRequest { Handle = handle, InBuff = ByteString.CopyFrom(wrapped) },
connection.Metadata,
Deadline(),
cancellationToken);
byte[] serverOutput = response.OutBuff?.ToByteArray() ?? [];
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
bool success = response.Status?.BSuccess ?? false;
return new HistorianNativeHandshake.TokenExchangeResult(success, serverOutput, error);
},
contextKey,
options,
cancellationToken);
byte[] open2Request = HistorianNativeHandshake.BuildOpenConnection3Request(
options.Host, contextKey, HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode);
GrpcHistory.OpenConnectionResponse open2 = historyClient.OpenConnection(
new GrpcHistory.OpenConnectionRequest { BtConnectionRequest = ByteString.CopyFrom(open2Request) },
connection.Metadata,
Deadline(),
cancellationToken);
byte[] open2Response = open2.BtConnectionResponse?.ToByteArray() ?? [];
if (!(open2.Status?.BSuccess ?? false))
{
byte[] err = open2.Status?.BtError?.ToByteArray() ?? [];
throw new InvalidOperationException($"gRPC OpenConnection failed (errorLen={err.Length}, responseLen={open2Response.Length}).");
}
(uint clientHandle, Guid storageSessionId) = HistorianNativeHandshake.ParseOpenConnectionResponse(open2Response);
return new Session(clientHandle, storageSessionId);
}
}
@@ -0,0 +1,48 @@
using Grpc.Core;
using GrpcHistory = ArchestrA.Grpc.Contract.History;
using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval;
using GrpcStatus = ArchestrA.Grpc.Contract.Status;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// 2023 R2 gRPC reachability probe (roadmap item R0.4). Mirrors <see cref="Wcf.HistorianWcfProbe"/>
/// over the gRPC transport: it calls the unauthenticated <c>GetInterfaceVersion</c> RPC on the
/// History, Retrieval, and Status services and applies the same success criteria. No credentials
/// are required — these RPCs run before the SSPI/Negotiate token loop — so the probe works even
/// when authentication is unavailable.
/// </summary>
internal static class HistorianGrpcProbe
{
public static async Task<bool> ProbeAsync(HistorianClientOptions options, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(options);
cancellationToken.ThrowIfCancellationRequested();
return await Task.Run(() => Probe(options, cancellationToken), cancellationToken).ConfigureAwait(false);
}
private static bool Probe(HistorianClientOptions options, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
DateTime deadline = DateTime.UtcNow.Add(options.ConnectTimeout > TimeSpan.Zero ? options.ConnectTimeout : TimeSpan.FromSeconds(5));
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
GrpcHistory.GetInterfaceVersionResponse history = historyClient.GetInterfaceVersion(
new GrpcHistory.GetInterfaceVersionRequest(), connection.Metadata, deadline, cancellationToken);
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
GrpcRetrieval.GetRetrievalInterfaceVersionResponse retrieval = retrievalClient.GetRetrievalInterfaceVersion(
new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), connection.Metadata, deadline, cancellationToken);
var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel);
GrpcStatus.GetStatusInterfaceVersionResponse status = statusClient.GetStatusInterfaceVersion(
new GrpcStatus.GetStatusInterfaceVersionRequest(), connection.Metadata, deadline, cancellationToken);
return history.UiError == 0
&& history.UiVersion > 0
&& retrieval.UiError == 0
&& retrieval.UiVersion > 0
&& status.UiError == 0;
}
}
@@ -3,7 +3,6 @@ using Google.Protobuf;
using Grpc.Core;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
using GrpcHistory = ArchestrA.Grpc.Contract.History;
using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval;
namespace AVEVA.Historian.Client.Grpc;
@@ -16,17 +15,23 @@ namespace AVEVA.Historian.Client.Grpc;
///
/// Operation mapping (2020 WCF → 2023 R2 gRPC):
/// Hist.GetInterfaceVersion → HistoryService.GetInterfaceVersion
/// Hist.ValidateClientCredential (loop) → HistoryService.ExchangeKey (loop)
/// Hist.ValidateClientCredential (loop) → StorageService.ValidateClientCredential (loop)
/// Hist.OpenConnection2 → HistoryService.OpenConnection
/// Retr.StartQuery2 → RetrievalService.StartQuery
/// Retr.GetNextQueryResultBuffer2 (loop) → RetrievalService.GetNextQueryResultBuffer (loop)
/// Retr.EndQuery2 → RetrievalService.EndQuery
///
/// NOTE: not yet live-verified against a 2023 R2 server. The auth handshake uses
/// HistoryService.ExchangeKey because the gRPC HistoryService dropped ValidateClientCredential
/// (it now lives only on StorageService) and gained ExchangeKey with the identical
/// handle+token→token shape. If a live server rejects this, the handshake op is the first thing
/// to revisit — everything else is the proven 2020 byte protocol.
/// LIVE-VERIFIED 2026-06-21 against a real 2023 R2 server (interface versions: History=12,
/// Retrieval=4, Storage=4). The SSPI/Negotiate token loop maps to
/// <c>StorageService.ValidateClientCredential(Handle, InBuff)→(status, OutBuff)</c> — the op that
/// kept the 2020 inBuff/outBuff token framing. The gRPC HistoryService dropped
/// ValidateClientCredential and gained <c>ExchangeKey</c>, but ExchangeKey is a SEPARATE
/// key-exchange/cert-path op, NOT the Negotiate loop: feeding it an NTLM token is rejected at
/// round 0 regardless of credentials. An earlier revision wrongly routed the loop to ExchangeKey;
/// routing it to StorageService.ValidateClientCredential completes the full read chain. The byte
/// payloads (OpenConnection3 v6, token framing, DataQueryRequest, row buffers) are the proven 2020
/// protocol and transfer unchanged — only the History interface integer differs (12 vs the 2020
/// value 11), and that version is buffer-compatible (a live read returns rows).
/// </summary>
internal sealed class HistorianGrpcReadOrchestrator
{
@@ -159,50 +164,7 @@ internal sealed class HistorianGrpcReadOrchestrator
}
private uint OpenAuthenticatedConnection(HistorianGrpcConnection connection, CancellationToken cancellationToken)
{
Guid contextKey = Guid.NewGuid();
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
GrpcHistory.GetInterfaceVersionResponse historyVersion = historyClient.GetInterfaceVersion(
new GrpcHistory.GetInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken);
HistorianServerVersionGate.Validate(HistorianServiceInterface.History, historyVersion.UiVersion, _options);
HistorianNativeHandshake.RunTokenRounds(
(handle, wrapped, _) =>
{
GrpcHistory.ExchangeKeyResponse response = historyClient.ExchangeKey(
new GrpcHistory.ExchangeKeyRequest { StrHandle = handle, BtInput = ByteString.CopyFrom(wrapped) },
connection.Metadata,
Deadline(),
cancellationToken);
byte[] serverOutput = response.BtOutput?.ToByteArray() ?? [];
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
bool success = response.Status?.BSuccess ?? false;
return new HistorianNativeHandshake.TokenExchangeResult(success, serverOutput, error);
},
contextKey,
_options,
cancellationToken);
byte[] open2Request = HistorianNativeHandshake.BuildOpenConnection3Request(
_options.Host, contextKey, HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode);
GrpcHistory.OpenConnectionResponse open2 = historyClient.OpenConnection(
new GrpcHistory.OpenConnectionRequest { BtConnectionRequest = ByteString.CopyFrom(open2Request) },
connection.Metadata,
Deadline(),
cancellationToken);
byte[] open2Response = open2.BtConnectionResponse?.ToByteArray() ?? [];
if (!(open2.Status?.BSuccess ?? false))
{
byte[] err = open2.Status?.BtError?.ToByteArray() ?? [];
throw new InvalidOperationException($"gRPC OpenConnection failed (errorLen={err.Length}, responseLen={open2Response.Length}).");
}
(uint clientHandle, _) = HistorianNativeHandshake.ParseOpenConnectionResponse(open2Response);
return clientHandle;
}
=> HistorianGrpcHandshake.OpenAuthenticatedConnection(connection, _options, cancellationToken);
private List<HistorianSample> RunQuery(
HistorianGrpcConnection connection,
@@ -0,0 +1,38 @@
using Grpc.Core;
using GrpcStatus = ArchestrA.Grpc.Contract.Status;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// 2023 R2 gRPC status client (roadmap item R0.3). Mirrors
/// <see cref="Wcf.HistorianWcfStatusClient"/> over the gRPC transport: it opens an authenticated
/// History session via <see cref="HistorianGrpcHandshake"/> and queries the StatusService for the
/// resulting client handle. <c>GetSystemParameter</c> carries the parameter name as a protobuf
/// string and returns the value string directly — there is no opaque native buffer to decode.
/// </summary>
internal static class HistorianGrpcStatusClient
{
public static Task<string?> GetSystemParameterAsync(
HistorianClientOptions options,
string parameterName,
CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(parameterName);
return Task.Run(() => GetSystemParameter(options, parameterName, cancellationToken), cancellationToken);
}
private static string? GetSystemParameter(HistorianClientOptions options, string parameterName, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
uint clientHandle = HistorianGrpcHandshake.OpenAuthenticatedConnection(connection, options, cancellationToken);
var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel);
GrpcStatus.GetSystemParameterResponse response = statusClient.GetSystemParameter(
new GrpcStatus.GetSystemParameterRequest { UiHandle = clientHandle, StrParameterName = parameterName },
connection.Metadata,
DateTime.UtcNow.Add(options.RequestTimeout),
cancellationToken);
return (response.Status?.BSuccess ?? false) ? response.StrParameterValue : null;
}
}
@@ -0,0 +1,283 @@
using System.Text;
using Google.Protobuf;
using Grpc.Core;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// 2023 R2 gRPC tag-metadata + browse client (roadmap items R0.2 metadata, R0.1 browse).
/// Browse drives <c>StartTagQuery</c> (OData filter) → paged <c>QueryTag</c> → <c>EndTagQuery</c>
/// (see <see cref="BrowseTagNamesAsync"/> and <c>docs/reverse-engineering/grpc-tag-query-odata.md</c>).
/// Unlike the WCF singular
/// <c>GetTagInfoFromName</c> (a <c>uint</c>-handle op), the gRPC front door exposes the plural
/// <c>RetrievalService.GetTagInfosFromName</c> — a <b>string-handle</b> op keyed off the Open2
/// storage-session GUID (uppercase). The request <c>btTagNames</c> buffer and response
/// <c>btTagInfos</c> buffer carry the proven native encodings:
/// <list type="bullet">
/// <item>request <c>btTagNames</c> = <c>uint count</c> + per-name(<c>uint charCount</c> + UTF-16LE)</item>
/// <item>response <c>btTagInfos</c> = <c>uint tagCount</c> + per-tag CTagMetadata record
/// (the same record <see cref="HistorianTagQueryProtocol.ParseGetTagInfoResponse"/> decodes)</item>
/// </list>
/// The string-handle "wall" that blocks this op family on the 2020 WCF transport does not apply on
/// the gRPC front door (different envelope/registration) — see
/// <c>docs/reverse-engineering/wcf-string-handle-wall.md</c>.
/// </summary>
internal static class HistorianGrpcTagClient
{
public static Task<HistorianTagMetadata?> GetTagMetadataAsync(
HistorianClientOptions options,
string tag,
CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tag);
return Task.Run(() => GetTagMetadata(options, tag, cancellationToken), cancellationToken);
}
private static HistorianTagMetadata? GetTagMetadata(HistorianClientOptions options, string tag, CancellationToken cancellationToken)
{
byte[] tagInfos = GetTagInfosRaw(options, [tag], cancellationToken);
if (tagInfos.Length < 4)
{
return null;
}
IReadOnlyList<HistorianTagInfoResponse> parsed = HistorianTagQueryProtocol.ParseGetTagInfoResponse(tagInfos);
if (parsed.Count == 0)
{
return null;
}
HistorianTagInfoResponse info = parsed[0];
return new HistorianTagMetadata(
Name: info.TagName,
Key: info.TagKey,
DataType: HistorianWcfTagClient.MapDataType(info.NativeDataTypeDescriptor),
Description: info.Description ?? info.MetadataProvider,
EngineeringUnit: info.EngineeringUnit ?? string.Empty,
MinRaw: info.MinEU,
MaxRaw: info.MaxEU);
}
/// <summary>
/// Issues a single <c>GetTagInfosFromName</c> call and returns the raw native <c>btTagInfos</c>
/// response buffer. Internal so reverse-engineering probes can capture the framing.
/// </summary>
internal static byte[] GetTagInfosRaw(HistorianClientOptions options, IReadOnlyList<string> tags, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
byte[] requestBuffer = BuildTagNamesBuffer(tags);
GrpcRetrieval.GetTagInfosFromNameResponse response = retrievalClient.GetTagInfosFromName(
new GrpcRetrieval.GetTagInfosFromNameRequest
{
StrHandle = session.StringHandle,
BtTagNames = ByteString.CopyFrom(requestBuffer),
UiSequence = 0
},
connection.Metadata,
DateTime.UtcNow.Add(options.RequestTimeout),
cancellationToken);
if (!(response.Status?.BSuccess ?? false))
{
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
throw new InvalidOperationException($"gRPC GetTagInfosFromName failed (errorLen={error.Length}).");
}
return response.BtTagInfos?.ToByteArray() ?? [];
}
// QueryTag (browse paging) request framing, recovered from the .rdata packet-descriptor table
// in aahClientManaged.dll (entries {0x6751,1}=StartTagQuery, {0x6752,1}=QueryTag) and confirmed
// live: btRequest = u16 marker(0x6752) + u16 version(1) + u16 queryType + u32 startIndex + u32 count.
private const ushort QueryTagPacketMarker = 0x6752;
private const ushort TagQueryHeaderVersion = 1;
private const ushort QueryTagModeNames = 1; // queryType 1 returns tag-name rows
private const uint BrowsePageSize = 1000;
/// <summary>
/// Browses tag names over gRPC (roadmap item R0.1). Drives
/// <c>StartTagQuery</c> (OData filter) → paged <c>QueryTag</c> → <c>EndTagQuery</c> on the
/// RetrievalService. The 2023 R2 metadata-server parses the filter as <b>OData</b>, so the SDK's
/// glob filter is translated via <see cref="GlobToODataFilter"/>. Each QueryTag page returns
/// <c>uint count + per-name(uint charCount + UTF-16LE)</c>, decoded by
/// <see cref="HistorianTagQueryProtocol.ParseGetLikeTagNamesResponse"/>.
/// </summary>
public static async IAsyncEnumerable<string> BrowseTagNamesAsync(
HistorianClientOptions options,
string filter,
[System.Runtime.CompilerServices.EnumeratorCancellation] CancellationToken cancellationToken)
{
IReadOnlyList<string> names = await Task.Run(() => BrowseTagNames(options, filter, cancellationToken), cancellationToken).ConfigureAwait(false);
foreach (string name in names)
{
cancellationToken.ThrowIfCancellationRequested();
yield return name;
}
}
private static List<string> BrowseTagNames(HistorianClientOptions options, string filter, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
DateTime Deadline() => DateTime.UtcNow.Add(options.RequestTimeout);
byte[] startRequest = HistorianTagQueryProtocol.CreateStartTagQueryAttempt(GlobToODataFilter(filter)).RequestBuffer;
GrpcRetrieval.StartTagQueryResponse start = retrievalClient.StartTagQuery(
new GrpcRetrieval.StartTagQueryRequest { StrHandle = session.StringHandle, BtRequest = ByteString.CopyFrom(startRequest) },
connection.Metadata, Deadline(), cancellationToken);
if (!(start.Status?.BSuccess ?? false))
{
byte[] error = start.Status?.BtError?.ToByteArray() ?? [];
throw new InvalidOperationException($"gRPC StartTagQuery failed (errorLen={error.Length}).");
}
HistorianTagQueryStartResponse parsed = HistorianTagQueryProtocol.ParseStartTagQueryResponse(start.BtResponse?.ToByteArray() ?? []);
List<string> names = new(checked((int)parsed.TagCount));
try
{
uint startIndex = 0;
while (names.Count < parsed.TagCount)
{
cancellationToken.ThrowIfCancellationRequested();
uint page = Math.Min(BrowsePageSize, parsed.TagCount - (uint)names.Count);
GrpcRetrieval.QueryTagResponse query = retrievalClient.QueryTag(
new GrpcRetrieval.QueryTagRequest
{
StrHandle = session.StringHandle,
UiQueryHandle = parsed.QueryHandle,
BtRequest = ByteString.CopyFrom(BuildQueryTagRequest(QueryTagModeNames, startIndex, page))
},
connection.Metadata, Deadline(), cancellationToken);
if (!(query.Status?.BSuccess ?? false))
{
byte[] error = query.Status?.BtError?.ToByteArray() ?? [];
throw new InvalidOperationException($"gRPC QueryTag failed (errorLen={error.Length}).");
}
IReadOnlyList<string> pageNames = HistorianTagQueryProtocol.ParseTagNameQueryPage(query.BtResonse?.ToByteArray() ?? []);
if (pageNames.Count == 0)
{
break;
}
names.AddRange(pageNames);
startIndex += (uint)pageNames.Count;
}
}
finally
{
try
{
retrievalClient.EndTagQuery(
new GrpcRetrieval.EndTagQueryRequest { StrHandle = session.StringHandle, UiQueryHandle = parsed.QueryHandle },
connection.Metadata, Deadline(), CancellationToken.None);
}
catch { /* best-effort cleanup */ }
}
return names;
}
/// <summary>Builds the QueryTag paging request: u16 marker(0x6752) + u16 version + u16 queryType + u32 startIndex + u32 count.</summary>
internal static byte[] BuildQueryTagRequest(ushort queryType, uint startIndex, uint count)
{
using MemoryStream stream = new();
using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true);
writer.Write(QueryTagPacketMarker);
writer.Write(TagQueryHeaderVersion);
writer.Write(queryType);
writer.Write(startIndex);
writer.Write(count);
return stream.ToArray();
}
/// <summary>
/// Translates the SDK's glob filter (<c>*</c> wildcard) into the OData filter the 2023 R2
/// metadata-server's <c>StartActiveTagnamesQuery</c> expects. Single-quotes are OData-escaped.
/// <list type="bullet">
/// <item><c>*</c> / empty → no filter (all tags)</item>
/// <item><c>Pre*</c> → <c>startswith(TagName,'Pre')</c></item>
/// <item><c>*suf</c> → <c>endswith(TagName,'suf')</c></item>
/// <item><c>*mid*</c> → <c>contains(TagName,'mid')</c></item>
/// <item><c>a*b</c> → <c>startswith(TagName,'a') and endswith(TagName,'b')</c></item>
/// <item><c>Exact</c> → <c>TagName eq 'Exact'</c></item>
/// </list>
/// </summary>
internal static string GlobToODataFilter(string filter)
{
if (string.IsNullOrEmpty(filter) || filter == "*")
{
return string.Empty;
}
static string Esc(string s) => s.Replace("'", "''");
bool starStart = filter.StartsWith('*');
bool starEnd = filter.EndsWith('*');
string core = filter.Trim('*');
if (core.Length == 0)
{
return string.Empty; // "**" etc.
}
if (filter.IndexOf('*') < 0)
{
return $"TagName eq '{Esc(filter)}'";
}
if (starStart && starEnd && !core.Contains('*'))
{
return $"contains(TagName,'{Esc(core)}')";
}
if (starEnd && !core.Contains('*') && !starStart)
{
return $"startswith(TagName,'{Esc(core)}')";
}
if (starStart && !core.Contains('*') && !starEnd)
{
return $"endswith(TagName,'{Esc(core)}')";
}
// Internal wildcard(s): anchor on the prefix before the first '*' and the suffix after the last.
string prefix = filter[..filter.IndexOf('*')];
string suffix = filter[(filter.LastIndexOf('*') + 1)..];
List<string> parts = [];
if (prefix.Length > 0)
{
parts.Add($"startswith(TagName,'{Esc(prefix)}')");
}
if (suffix.Length > 0)
{
parts.Add($"endswith(TagName,'{Esc(suffix)}')");
}
return parts.Count > 0 ? string.Join(" and ", parts) : string.Empty;
}
/// <summary>Builds the native tag-names request buffer: uint count + per-name(uint charCount + UTF-16LE).</summary>
internal static byte[] BuildTagNamesBuffer(IReadOnlyList<string> tags)
{
using MemoryStream stream = new();
using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true);
writer.Write((uint)tags.Count);
foreach (string tag in tags)
{
writer.Write((uint)tag.Length);
if (tag.Length > 0)
{
writer.Write(Encoding.Unicode.GetBytes(tag));
}
}
return stream.ToArray();
}
}
+141 -4
View File
@@ -25,7 +25,9 @@ public sealed class HistorianClient : IAsyncDisposable
public async Task<bool> ProbeAsync(CancellationToken cancellationToken = default)
{
return await HistorianWcfProbe.ProbeAsync(_options, cancellationToken).ConfigureAwait(false);
return _options.Transport == HistorianTransport.RemoteGrpc
? await Grpc.HistorianGrpcProbe.ProbeAsync(_options, cancellationToken).ConfigureAwait(false)
: await HistorianWcfProbe.ProbeAsync(_options, cancellationToken).ConfigureAwait(false);
}
public IAsyncEnumerable<HistorianSample> ReadRawAsync(
@@ -90,19 +92,56 @@ public sealed class HistorianClient : IAsyncDisposable
CancellationToken cancellationToken = default)
{
ValidateTimeRange(startUtc, endUtc);
return _protocol.ReadEventsAsync(startUtc, endUtc, cancellationToken);
return _protocol.ReadEventsAsync(startUtc, endUtc, filter: null, cancellationToken);
}
/// <summary>
/// Reads events in the time window, server-filtered by a single predicate
/// (<paramref name="filter"/>) — e.g. <c>Type Equal "User.Write"</c> or
/// <c>Area Contains "Tank"</c>. The historian applies the filter and returns only matching
/// events. Filtering is a real server-side operation (live-verified: a non-matching predicate
/// returns zero events). Single string-valued predicates only; see <see cref="HistorianEventFilter"/>.
/// </summary>
public IAsyncEnumerable<HistorianEvent> ReadEventsAsync(
DateTime startUtc,
DateTime endUtc,
HistorianEventFilter filter,
CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(filter);
ValidateTimeRange(startUtc, endUtc);
return _protocol.ReadEventsAsync(startUtc, endUtc, filter, cancellationToken);
}
/// <summary>
/// Sends a single <see cref="HistorianEvent"/> to the Historian's built-in CM_EVENT tag
/// over the WCF event pipeline (Open2 event mode → CM_EVENT registration → AddS2). The
/// event is appended to the historian's event history and is readable back via
/// <see cref="ReadEventsAsync"/> / the <c>v_AlarmEventHistory2</c> view. Only original
/// events (<see cref="HistorianEvent.RevisionVersion"/> = 0) with string-valued properties
/// are supported; other property value types and revision/update/delete events throw
/// <see cref="ProtocolEvidenceMissingException"/> until their wire encoding is captured.
/// </summary>
public Task<bool> SendEventAsync(HistorianEvent historianEvent, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(historianEvent);
return new HistorianWcfEventOrchestrator(_options).SendEventAsync(historianEvent, cancellationToken);
}
public IAsyncEnumerable<string> BrowseTagNamesAsync(string filter = "*", CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(filter);
return HistorianWcfTagClient.BrowseTagNamesAsync(_options, filter, cancellationToken);
return _options.Transport == HistorianTransport.RemoteGrpc
? Grpc.HistorianGrpcTagClient.BrowseTagNamesAsync(_options, filter, cancellationToken)
: HistorianWcfTagClient.BrowseTagNamesAsync(_options, filter, cancellationToken);
}
public Task<HistorianTagMetadata?> GetTagMetadataAsync(string tag, CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tag);
return HistorianWcfTagClient.GetTagMetadataAsync(_options, tag, cancellationToken);
return _options.Transport == HistorianTransport.RemoteGrpc
? Grpc.HistorianGrpcTagClient.GetTagMetadataAsync(_options, tag, cancellationToken)
: HistorianWcfTagClient.GetTagMetadataAsync(_options, tag, cancellationToken);
}
public Task<HistorianConnectionStatus> GetConnectionStatusAsync(CancellationToken cancellationToken = default)
@@ -121,6 +160,78 @@ public sealed class HistorianClient : IAsyncDisposable
return _protocol.GetSystemParameterAsync(name, cancellationToken);
}
/// <summary>
/// Reads a named Historian <em>runtime</em> parameter (the live server state surface,
/// distinct from the configuration <see cref="GetSystemParameterAsync"/>). Returns the
/// string value, or null when the server reports no value. Single string-valued parameters
/// only (the evidence-backed surface); see <c>HistorianRuntimeParameterProtocol</c>.
/// </summary>
public Task<string?> GetRuntimeParameterAsync(string name, CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(name);
return _protocol.GetRuntimeParameterAsync(name, cancellationToken);
}
/// <summary>
/// Reads the extended (user-defined) properties attached to a tag via the 2020 WCF
/// <c>GetTepByNm</c> op. Returns the property name/value pairs (empty when the tag has none).
/// String-valued properties only (the evidence-backed surface); other value variants throw
/// <see cref="ProtocolEvidenceMissingException"/>. See
/// <c>HistorianTagExtendedPropertyProtocol</c>.
/// </summary>
public Task<IReadOnlyList<HistorianTagExtendedProperty>> GetTagExtendedPropertiesAsync(string tag, CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tag);
return _protocol.GetTagExtendedPropertiesAsync(tag, cancellationToken);
}
/// <summary>
/// Adds (or updates) extended (user-defined) properties on an existing tag via the 2020 WCF
/// <c>AddTEx</c> (AddTagExtendedProperties) op. Requires a write-enabled connection. String-valued
/// properties only (the evidence-backed surface). The new properties are read back via
/// <see cref="GetTagExtendedPropertiesAsync"/>. See <c>HistorianTagExtendedPropertyProtocol</c>.
/// </summary>
public Task<bool> AddTagExtendedPropertiesAsync(string tag, IReadOnlyList<HistorianTagExtendedProperty> properties, CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tag);
ArgumentNullException.ThrowIfNull(properties);
return new HistorianWcfTagWriteOrchestrator(_options).AddTagExtendedPropertiesAsync(tag, properties, cancellationToken);
}
/// <summary>Convenience overload of <see cref="AddTagExtendedPropertiesAsync"/> for a single
/// string-valued property.</summary>
public Task<bool> AddTagExtendedPropertyAsync(string tag, string name, string value, CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tag);
ArgumentException.ThrowIfNullOrWhiteSpace(name);
return AddTagExtendedPropertiesAsync(tag, [new HistorianTagExtendedProperty(name, value ?? string.Empty)], cancellationToken);
}
// Extended-property DELETE (DelTep) is intentionally NOT exposed publicly. Its wire format is
// captured and the serializer (HistorianTagExtendedPropertyProtocol.SerializeDeleteRequest) is
// golden-verified against a server-accepted buffer, but the SDK cannot yet make the 2020 server
// accept the delete: the server's CHistStorage::DeleteTagExtendedProperties consults a
// per-connection working set that the native client populates by multiplexing GetTepByNm and
// DelTep over ONE connection, which the SDK's per-service WCF channels don't reproduce. See the
// documented-but-blocked path in HistorianWcfTagWriteOrchestrator and
// docs/reverse-engineering/wcf-add-tag-extended-properties.md §Delete.
/// <summary>
/// Executes a SQL command against the Historian over the WCF <c>ExeC</c>/<c>GetR</c> ops and
/// returns the record set as a <see cref="HistorianSqlResult"/> (the managed equivalent of the
/// native <c>DataTable</c>). The record-set path (<see cref="HistorianSqlExecuteOption.ExecuteRecord"/>,
/// the default) is the evidence-backed surface; the result is decoded from the server's
/// NRBF-serialized DataTable without BinaryFormatter. See <c>HistorianSqlResultProtocol</c>.
/// </summary>
public Task<HistorianSqlResult> ExecuteSqlCommandAsync(
string command,
HistorianSqlExecuteOption option = HistorianSqlExecuteOption.ExecuteRecord,
CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(command);
return _protocol.ExecuteSqlCommandAsync(command, option, cancellationToken);
}
/// <summary>
/// Creates or updates the named tag in the Historian Runtime database via
/// <c>EnsureTags2</c>. Currently only <see cref="HistorianDataType.Float"/> is
@@ -149,6 +260,32 @@ public sealed class HistorianClient : IAsyncDisposable
return new HistorianWcfTagWriteOrchestrator(_options).DeleteTagAsync(tagName, cancellationToken);
}
/// <summary>
/// Renames one tag, submitting an asynchronous rename job via the History <c>StartJob</c> (StJb)
/// operation. Convenience wrapper over <see cref="RenameTagsAsync"/> for a single (old,new) pair.
/// Requires the server's <c>AllowRenameTags</c> system parameter to be enabled.
/// </summary>
public Task<HistorianTagRenameResult> RenameTagAsync(string oldName, string newName, CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(oldName);
ArgumentException.ThrowIfNullOrWhiteSpace(newName);
return RenameTagsAsync([(oldName, newName)], cancellationToken);
}
/// <summary>
/// Renames a batch of tags. Each pair is (current name, new name). Rename is an asynchronous
/// server-side job: the batch is submitted via the History <c>StartJob</c> (StJb) operation and
/// the returned <see cref="HistorianTagRenameResult"/> reports whether the server accepted/queued
/// the job (and its job id); the renames apply in the background. The server's
/// <c>AllowRenameTags</c> system parameter must be enabled or the server rejects the job. See
/// <c>docs/reverse-engineering/wcf-rename-tags.md</c>.
/// </summary>
public Task<HistorianTagRenameResult> RenameTagsAsync(IReadOnlyList<(string OldName, string NewName)> pairs, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(pairs);
return new HistorianWcfTagWriteOrchestrator(_options).RenameTagsAsync(pairs, cancellationToken);
}
public ValueTask DisposeAsync()
{
return ValueTask.CompletedTask;
@@ -43,6 +43,15 @@ internal static class HistorianServerVersionGate
public const uint RetrievalInterfaceVersion = 4;
public const uint TransactionInterfaceVersion = 2;
/// <summary>
/// The 2023 R2 gRPC HistoryService reports interface version 12. It is buffer-compatible with
/// the 2020 version 11 — the OpenConnection3 v6 / token / DataQueryRequest / row buffers are
/// byte-identical — confirmed by a live end-to-end gRPC read against a real 2023 R2 server
/// (2026-06-21). So both 11 and 12 are accepted for History. (Retrieval reported 4, matching
/// the 2020 value, so it needs no widening.)
/// </summary>
public const uint HistoryInterfaceVersionGrpc2023R2 = 12;
/// <summary>
/// True when the service interface reports a meaningful version that should be matched.
/// Status is reachability-only (its <c>GetInterfaceVersion</c> returns 0).
@@ -56,7 +65,7 @@ internal static class HistorianServerVersionGate
_ => false
};
/// <summary>The interface version this SDK's serializers target for a value-gated service.</summary>
/// <summary>The canonical interface version this SDK's serializers target for a value-gated service.</summary>
public static uint ExpectedVersion(HistorianServiceInterface service) => service switch
{
HistorianServiceInterface.History => HistoryInterfaceVersion,
@@ -65,6 +74,18 @@ internal static class HistorianServerVersionGate
_ => throw new ArgumentOutOfRangeException(nameof(service), service, "Service interface is not value-gated.")
};
/// <summary>
/// All interface versions accepted for a value-gated service. Usually a single value, but
/// History accepts both the 2020 value (11) and the buffer-compatible 2023 R2 gRPC value (12).
/// </summary>
public static uint[] AcceptedVersions(HistorianServiceInterface service) => service switch
{
HistorianServiceInterface.History => [HistoryInterfaceVersion, HistoryInterfaceVersionGrpc2023R2],
HistorianServiceInterface.Retrieval => [RetrievalInterfaceVersion],
HistorianServiceInterface.Transaction => [TransactionInterfaceVersion],
_ => throw new ArgumentOutOfRangeException(nameof(service), service, "Service interface is not value-gated.")
};
/// <summary>
/// Throws <see cref="ProtocolEvidenceMissingException"/> when version verification is enabled
/// and the server's reported interface version differs from the version this SDK targets.
@@ -80,14 +101,15 @@ internal static class HistorianServerVersionGate
return;
}
uint expected = ExpectedVersion(service);
if (reportedVersion == expected)
uint[] accepted = AcceptedVersions(service);
if (Array.IndexOf(accepted, reportedVersion) >= 0)
{
return;
}
string acceptedList = string.Join(", ", accepted);
throw new ProtocolEvidenceMissingException(
$"{service} interface version {reportedVersion} (this SDK's serializers target version {expected}); " +
$"{service} interface version {reportedVersion} (this SDK's serializers target version {acceptedList}); " +
$"set {nameof(HistorianClientOptions)}.{nameof(HistorianClientOptions.VerifyServerInterfaceVersion)}=false to bypass at your own risk");
}
}
@@ -0,0 +1,40 @@
namespace AVEVA.Historian.Client.Models;
/// <summary>
/// Comparison operator for a <see cref="HistorianEventFilter"/>. Values mirror the native
/// <c>ArchestrA.HistorianComparisionType</c> ordinals and travel on the wire as a UInt16.
/// </summary>
public enum HistorianEventComparison : ushort
{
Equal = 0,
NotEqual = 1,
LessThan = 2,
NotLessThan = 3,
GreaterThan = 4,
NotGreaterThan = 5,
LessThanEqual = 6,
NotLessThanEqual = 7,
GreaterThanEqual = 8,
NotGreaterThanEqual = 9,
Begins = 10,
NotBegins = 11,
Contains = 12,
NotContains = 13,
Exists = 14,
NotExists = 15,
EndWith = 16,
NotEndWith = 17,
}
/// <summary>
/// A single server-side event filter predicate: <c>PropertyName Comparison Value</c>
/// (e.g. <c>Type Equal "User.Write"</c>, <c>Area Contains "Tank"</c>). Applied to
/// <c>ReadEventsAsync</c>; the server returns only events whose named property satisfies the
/// comparison. For <see cref="HistorianEventComparison.Exists"/> /
/// <see cref="HistorianEventComparison.NotExists"/> the value is ignored but still required by
/// the wire format (pass any non-null string).
/// </summary>
public sealed record HistorianEventFilter(
string PropertyName,
HistorianEventComparison Comparison,
string Value);
@@ -0,0 +1,21 @@
namespace AVEVA.Historian.Client.Models;
/// <summary>
/// Execution mode for <c>ExecuteSqlCommandAsync</c>, mirroring the native
/// <c>HistorianSqlExecuteOption</c> enum passed to the <c>ExeC</c> op. Only
/// <see cref="ExecuteRecord"/> (the record-set path) is evidence-backed end-to-end.
/// </summary>
public enum HistorianSqlExecuteOption
{
/// <summary>Execute and return a record set (the captured/proven path).</summary>
ExecuteRecord = 0,
/// <summary>Execute without returning a record set; the result carries only the return value.</summary>
ExecuteNonQuery = 1,
/// <summary>Execute and return a single scalar value.</summary>
ExecuteScalar = 2,
/// <summary>Execute a record set directly (server-side direct path).</summary>
ExecuteRecordDirect = 3,
}
@@ -0,0 +1,35 @@
namespace AVEVA.Historian.Client.Models;
/// <summary>
/// A column in a <see cref="HistorianSqlResult"/>: its name and the XSD type from the result-set
/// schema (e.g. <c>xs:int</c>, <c>xs:string</c>, <c>xs:dateTime</c>).
/// </summary>
public sealed record HistorianSqlColumn(string Name, string SchemaType);
/// <summary>
/// The result of <c>ExecuteSqlCommandAsync</c> — the managed equivalent of the <c>DataTable</c>
/// the native client returns. Columns carry name + XSD type; each row is a list of values aligned
/// to <see cref="Columns"/> (typed per the schema where the XSD type is recognized, otherwise the
/// raw string; <c>null</c> for a SQL NULL / absent cell).
/// </summary>
public sealed class HistorianSqlResult
{
public HistorianSqlResult(
IReadOnlyList<HistorianSqlColumn> columns,
IReadOnlyList<IReadOnlyList<object?>> rows,
int returnValue)
{
Columns = columns;
Rows = rows;
ReturnValue = returnValue;
}
/// <summary>The result-set columns, in order.</summary>
public IReadOnlyList<HistorianSqlColumn> Columns { get; }
/// <summary>The rows; each row's values align positionally to <see cref="Columns"/>.</summary>
public IReadOnlyList<IReadOnlyList<object?>> Rows { get; }
/// <summary>The native <c>retValue</c> from <c>ExeC</c> (e.g. rows affected for non-queries).</summary>
public int ReturnValue { get; }
}
@@ -0,0 +1,11 @@
namespace AVEVA.Historian.Client.Models;
/// <summary>
/// A single extended (user-defined) property attached to a Historian tag — a name/value pair
/// returned by <c>GetTagExtendedPropertiesAsync</c>. Extended properties are stored separately
/// from the core tag metadata (server table <c>_TagExtendedProperty</c>) and are exposed over the
/// 2020 WCF <c>aa/Retr/GetTepByNm</c> op.
/// </summary>
/// <param name="Name">The property name (e.g., <c>Location</c>).</param>
/// <param name="Value">The property value as a string (the wire format is a VT_BSTR variant).</param>
public sealed record HistorianTagExtendedProperty(string Name, string Value);
@@ -0,0 +1,29 @@
namespace AVEVA.Historian.Client.Models;
/// <summary>
/// Result of <see cref="HistorianClient.RenameTagsAsync"/>. Tag rename on the Historian is an
/// asynchronous server-side <em>job</em>: the client submits the rename batch via the History
/// <c>StartJob</c> (StJb) operation and the server returns a job id, then applies the renames in the
/// background (the native client polls <c>GetJobStatus</c>/<c>GtJb</c> until the job reports done).
///
/// <para><see cref="Accepted"/> reflects whether the server accepted and queued the job. The renames
/// themselves complete asynchronously (observed: well under a couple of seconds for a small batch on
/// the local server). The server gate <c>AllowRenameTags</c> must be enabled, or the native client
/// library rejects the call before it reaches the wire (error 132 OperationNotEnabled).</para>
/// </summary>
public sealed record HistorianTagRenameResult
{
/// <summary>True when the server accepted and queued the rename job (StartJob returned success
/// with an empty error buffer).</summary>
public required bool Accepted { get; init; }
/// <summary>The server-assigned job id for the submitted rename batch (the <c>strJobid</c>
/// returned by StartJob). <see cref="Guid.Empty"/> if the server returned no parseable id.</summary>
public Guid JobId { get; init; }
/// <summary>Number of (old,new) name pairs submitted in the batch.</summary>
public int PairCount { get; init; }
/// <summary>Server error text when <see cref="Accepted"/> is false; otherwise null.</summary>
public string? Error { get; init; }
}
@@ -42,10 +42,10 @@ internal sealed class Historian2020ProtocolDialect
return Missing<HistorianBlock>("StartBlockRetrievalQuery", cancellationToken);
}
public IAsyncEnumerable<HistorianEvent> ReadEventsAsync(DateTime startUtc, DateTime endUtc, CancellationToken cancellationToken)
public IAsyncEnumerable<HistorianEvent> ReadEventsAsync(DateTime startUtc, DateTime endUtc, HistorianEventFilter? filter, CancellationToken cancellationToken)
{
HistorianWcfEventOrchestrator orchestrator = new(_options);
return orchestrator.ReadEventsAsync(startUtc, endUtc, cancellationToken);
return orchestrator.ReadEventsAsync(startUtc, endUtc, filter, cancellationToken);
}
public Task<HistorianConnectionStatus> GetConnectionStatusAsync(CancellationToken cancellationToken)
@@ -64,7 +64,30 @@ internal sealed class Historian2020ProtocolDialect
{
cancellationToken.ThrowIfCancellationRequested();
ArgumentException.ThrowIfNullOrWhiteSpace(name);
return Wcf.HistorianWcfStatusClient.GetSystemParameterAsync(_options, name, cancellationToken);
return UseGrpc
? HistorianGrpcStatusClient.GetSystemParameterAsync(_options, name, cancellationToken)
: Wcf.HistorianWcfStatusClient.GetSystemParameterAsync(_options, name, cancellationToken);
}
public Task<string?> GetRuntimeParameterAsync(string name, CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
ArgumentException.ThrowIfNullOrWhiteSpace(name);
return Wcf.HistorianWcfStatusClient.GetRuntimeParameterAsync(_options, name, cancellationToken);
}
public Task<IReadOnlyList<Models.HistorianTagExtendedProperty>> GetTagExtendedPropertiesAsync(string tag, CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
ArgumentException.ThrowIfNullOrWhiteSpace(tag);
return Wcf.HistorianWcfTagExtendedPropertyClient.GetTagExtendedPropertiesAsync(_options, tag, cancellationToken);
}
public Task<HistorianSqlResult> ExecuteSqlCommandAsync(string command, HistorianSqlExecuteOption option, CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
ArgumentException.ThrowIfNullOrWhiteSpace(command);
return Wcf.HistorianWcfSqlClient.ExecuteSqlCommandAsync(_options, command, option, cancellationToken);
}
private static async IAsyncEnumerable<T> Missing<T>(
@@ -29,6 +29,14 @@ internal interface IStatusServiceContract2 : IStatusServiceContract
[MessageParameter(Name = "pResponseBuff")] out byte[] responseBuffer,
out byte[] errorBuffer);
[OperationContract(Name = "GETRP")]
[return: MarshalAs(UnmanagedType.U1)]
bool GetRuntimeParameter(
string handle,
[MessageParameter(Name = "pRequestBuff")] byte[] requestBuffer,
[MessageParameter(Name = "pResponseBuff")] out byte[] responseBuffer,
out byte[] errorBuffer);
[OperationContract(Name = "PNGS")]
[return: MarshalAs(UnmanagedType.U1)]
bool PingServer(string handle, string pipeName, uint timeout, ref byte[] errorBuffer);
@@ -1,5 +1,6 @@
using System.Security.Cryptography;
using System.Text;
using AVEVA.Historian.Client.Models;
namespace AVEVA.Historian.Client.Wcf;
@@ -7,15 +8,17 @@ internal static class HistorianEventQueryProtocol
{
public const ushort QueryRequestTypeEvent = 3;
public static IReadOnlyList<HistorianEventQueryAttempt> CreateStartEventQueryAttempts(DateTime startUtc, DateTime endUtc, uint eventCount)
public static IReadOnlyList<HistorianEventQueryAttempt> CreateStartEventQueryAttempts(
DateTime startUtc, DateTime endUtc, uint eventCount, HistorianEventFilter? filter = null)
{
List<HistorianEventQueryAttempt> attempts = [];
attempts.Add(CreateNativeEmptyFilterAttempt(startUtc, endUtc, eventCount));
attempts.Add(CreateNativeFilterAttempt(startUtc, endUtc, eventCount, filter));
return attempts;
}
private static HistorianEventQueryAttempt CreateNativeEmptyFilterAttempt(DateTime startUtc, DateTime endUtc, uint eventCount)
private static HistorianEventQueryAttempt CreateNativeFilterAttempt(
DateTime startUtc, DateTime endUtc, uint eventCount, HistorianEventFilter? filter)
{
using MemoryStream stream = new();
using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true);
@@ -27,7 +30,14 @@ internal static class HistorianEventQueryProtocol
writer.Write(0u);
writer.Write((ushort)0);
writer.Write((ushort)1);
WriteNativeEmptyFilterBlock(writer);
if (filter is null)
{
WriteNativeEmptyFilterBlock(writer);
}
else
{
WriteFilterBlock(writer, filter);
}
writer.Write(65_536u);
WriteHistorianString(writer, "UTC");
WriteMetadataNamespace(writer);
@@ -35,12 +45,64 @@ internal static class HistorianEventQueryProtocol
byte[] request = stream.ToArray();
return new HistorianEventQueryAttempt(
"native-empty-filter-version5",
filter is null ? "native-empty-filter-version5" : "native-filter-version5",
5,
request,
Convert.ToHexString(SHA256.HashData(request)).ToLowerInvariant());
}
/// <summary>
/// Filter block carrying one predicate (property / comparison / value), decoded byte-for-byte
/// from instrument-wcf-writemessage captures of <c>EventQuery.AddEventFilter</c> (Equal vs
/// Contains diffed to isolate the operator field). Layout:
/// <code>
/// UInt16 0 // header word (also written by the empty-filter block)
/// UInt32 filterCount = 1
/// UInt32 conditionCount = 1
/// UInt32 propertyNameCharCount
/// propertyName (UTF-16LE)
/// UInt32 1 // constant (per-condition marker; observed = 1)
/// UInt16 comparison // HistorianComparisionType ordinal
/// UInt32 1 // constant (value marker; observed = 1)
/// value: 0x09 LEN 0x00 LEN×ASCII // compact-ASCII string (same as event/CTagMetadata)
/// Byte 0 // block terminator
/// </code>
/// Single-predicate only — multi-filter (OR) / multi-condition (AND) framing is not yet
/// fully captured. String values only; the value is always emitted (even for Exists).
/// </summary>
private static void WriteFilterBlock(BinaryWriter writer, HistorianEventFilter filter)
{
ArgumentException.ThrowIfNullOrEmpty(filter.PropertyName);
ArgumentNullException.ThrowIfNull(filter.Value);
writer.Write((ushort)0);
writer.Write(1u); // filterCount
writer.Write(1u); // conditionCount
writer.Write((uint)filter.PropertyName.Length); // char count
writer.Write(Encoding.Unicode.GetBytes(filter.PropertyName));
writer.Write(1u); // constant marker
writer.Write((ushort)filter.Comparison);
writer.Write(1u); // constant marker
WriteCompactAsciiString(writer, filter.Value);
writer.Write((byte)0); // block terminator
}
/// <summary>Compact ASCII string: <c>0x09 LEN 0x00 LEN×ASCII bytes</c>.</summary>
private static void WriteCompactAsciiString(BinaryWriter writer, string value)
{
byte[] ascii = Encoding.ASCII.GetBytes(value);
if (ascii.Length > byte.MaxValue)
{
throw new ProtocolEvidenceMissingException(
$"Event filter value '{value}' exceeds the single-byte length captured for the compact-string encoding.");
}
writer.Write((byte)0x09);
writer.Write((byte)ascii.Length);
writer.Write((byte)0);
writer.Write(ascii);
}
private static HistorianEventQueryAttempt CreateAttempt(
string shape,
ushort version,
@@ -0,0 +1,195 @@
using System.Buffers.Binary;
using System.Text;
using AVEVA.Historian.Client.Models;
namespace AVEVA.Historian.Client.Wcf;
/// <remarks>
/// Serializer for the <c>AddS2</c> (<c>IHistoryServiceContract2.AddStreamValues2</c>)
/// <c>pBuf</c> that carries a single <see cref="HistorianEvent"/> to the built-in
/// <c>CM_EVENT</c> tag. This is the inverse of <see cref="HistorianEventRowProtocol"/> and the
/// managed equivalent of native <c>HistorianEvent.PackToVtq</c> →
/// <c>CCommonArchestraEventValue::PackToVtq</c>.
///
/// Wire shape decoded byte-for-byte from two captured native event sends
/// (instrument-wcf-writemessage; <c>User.Write</c> and <c>Alarm.Set</c>, diffed to separate
/// constant framing from value-dependent fields):
///
/// <code>
/// pBuf (storage sample buffer):
/// 0x00 UInt16 0x534F // "OS" signature
/// 0x02 UInt16 sampleCount = 1
/// 0x04 UInt32 valueBlob.Length + 11
/// 0x08 UInt16 valueBlob.Length + 1
/// 0x0A valueBlob:
/// +0x00 GUID CM_EVENT tag id (353b8145-5df0-4d46-a253-871aef49b321)
/// +0x10 Int64 EventTime FILETIME (UTC, ms-truncated — the VTQ timestamp)
/// +0x18 UInt16 OpcQuality = 192
/// +0x1A UInt16 192
/// +0x1C UInt16 0x118D // opaque CCommonArchestraEventValue descriptor (constant)
/// +0x1E GUID event Id
/// +0x2E Int64 ReceivedTime FILETIME (UTC, ms — unique/monotonic on the native path)
/// +0x36 compact-ASCII string // Namespace
/// +.... compact-ASCII string // EventType (e.g. "Alarm.Set")
/// +.... UInt16 eventStructVersion = 5
/// +.... UInt16 propertyCount
/// +.... propertyCount × Property {
/// compact-ASCII string // property name
/// UInt8 typeMarker, UInt8 length, UInt8 status(=0), length×value bytes
/// 0x43 → UTF-16 string: UInt16 charCount + charCount×UInt16
/// }
/// </code>
///
/// Compact ASCII string: <c>0x09 LEN 0x00 LEN×ASCII bytes</c> (same as
/// <see cref="HistorianEventRowProtocol"/> and CTagMetadata strings).
///
/// Only string-valued properties are emitted here — the only property value type observed on
/// the event-send wire. Non-string property values throw
/// <see cref="ProtocolEvidenceMissingException"/> until captured (the read parser decodes more
/// types, but the write framing for them is unverified). Likewise revision sends
/// (Update/Delete/RevisionVersion ≠ 0) are not yet captured and are rejected by the caller.
/// </remarks>
internal static class HistorianEventWriteProtocol
{
public const ushort BufferSignature = 0x534F; // "OS"
public const ushort EventStructVersion = 5;
public const ushort OpcQualityGood = 192;
private const ushort EventValueDescriptor = 0x118D;
/// <summary>Built-in CM_EVENT tag id every streamed event targets.</summary>
public static readonly Guid CmEventTagId = new("353b8145-5df0-4d46-a253-871aef49b321");
private const byte CompactStringMarker = 0x09;
private const byte ValueTypeUtf16String = 0x43;
/// <summary>
/// Builds the <c>AddS2</c> <c>pBuf</c> for a single event. <paramref name="receivedTimeUtc"/>
/// is the storage/received timestamp the native path generates uniquely; the orchestrator
/// passes a current time.
/// </summary>
public static byte[] SerializeAddStreamValuesBuffer(HistorianEvent evt, DateTime receivedTimeUtc)
{
ArgumentNullException.ThrowIfNull(evt);
byte[] valueBlob = SerializeEventValueBlob(evt, receivedTimeUtc);
// The server's CValuStream check requires the delivered byte[] length to equal the
// declared packet length (UInt32 @0x04 = valueBlob.Length + 11). The native client's
// content is only valueBlob.Length + 10 bytes and relies on the MDAS encoder appending
// one trailing byte on the wire (its value is non-deterministic padding — 0x9F/0x00 in
// captures — and is beyond the parsed event content). Over the SDK's WCF byte[] path no
// such byte is added, so we emit it explicitly to satisfy the length check.
byte[] buffer = new byte[8 + 2 + valueBlob.Length + 1];
BinaryPrimitives.WriteUInt16LittleEndian(buffer.AsSpan(0, 2), BufferSignature);
BinaryPrimitives.WriteUInt16LittleEndian(buffer.AsSpan(2, 2), 1); // sampleCount
BinaryPrimitives.WriteUInt32LittleEndian(buffer.AsSpan(4, 4), checked((uint)(valueBlob.Length + 11)));
BinaryPrimitives.WriteUInt16LittleEndian(buffer.AsSpan(8, 2), checked((ushort)(valueBlob.Length + 1)));
valueBlob.CopyTo(buffer.AsSpan(10));
// buffer[^1] left as 0x00 trailing pad.
return buffer;
}
private static byte[] SerializeEventValueBlob(HistorianEvent evt, DateTime receivedTimeUtc)
{
using MemoryStream stream = new();
using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true);
writer.Write(CmEventTagId.ToByteArray());
// EventTime is the VTQ value timestamp; the native path floors it to whole
// milliseconds (the event store's resolution).
writer.Write(ToMillisecondFileTime(evt.EventTimeUtc));
writer.Write(OpcQualityGood);
writer.Write(OpcQualityGood);
writer.Write(EventValueDescriptor);
writer.Write(evt.Id.ToByteArray());
// ReceivedTime keeps full 100ns FILETIME precision (a unique/monotonic time on the
// native path), so it is NOT truncated to milliseconds.
writer.Write(ToFileTime(receivedTimeUtc));
WriteCompactAsciiString(writer, evt.Namespace ?? string.Empty);
WriteCompactAsciiString(writer, evt.Type ?? string.Empty);
writer.Write(EventStructVersion);
IReadOnlyList<KeyValuePair<string, object?>> properties = OrderedProperties(evt.Properties);
writer.Write(checked((ushort)properties.Count));
foreach (KeyValuePair<string, object?> property in properties)
{
WriteCompactAsciiString(writer, property.Key);
WritePropertyValue(writer, property.Key, property.Value);
}
writer.Flush();
return stream.ToArray();
}
private static IReadOnlyList<KeyValuePair<string, object?>> OrderedProperties(IReadOnlyDictionary<string, object?>? properties)
{
if (properties is null || properties.Count == 0)
{
return [];
}
// The native client stores properties in a SortedList keyed by name; mirror that
// deterministic ordering so the serialized bytes are stable.
List<KeyValuePair<string, object?>> ordered = [.. properties];
ordered.Sort(static (a, b) => string.CompareOrdinal(a.Key, b.Key));
return ordered;
}
private static void WritePropertyValue(BinaryWriter writer, string name, object? value)
{
if (value is not string s)
{
throw new ProtocolEvidenceMissingException(
$"Event property '{name}' has value type '{value?.GetType().Name ?? "null"}'. Only string-valued " +
"event properties have a captured write encoding; capture the typed-value framing before sending others.");
}
byte[] chars = Encoding.Unicode.GetBytes(s);
int valueByteLength = 2 + chars.Length; // UInt16 charCount + UTF-16 chars
if (valueByteLength > byte.MaxValue)
{
throw new ProtocolEvidenceMissingException(
$"Event property '{name}' string value is too long ({s.Length} chars); only a single-byte value " +
"length has been captured on the event-send wire.");
}
writer.Write(ValueTypeUtf16String);
writer.Write((byte)valueByteLength);
writer.Write((byte)0); // status
writer.Write(checked((ushort)s.Length));
writer.Write(chars);
}
/// <summary>Compact ASCII string: <c>0x09 LEN 0x00 LEN×ASCII bytes</c>.</summary>
private static void WriteCompactAsciiString(BinaryWriter writer, string value)
{
byte[] ascii = Encoding.ASCII.GetBytes(value);
if (ascii.Length > byte.MaxValue)
{
throw new ProtocolEvidenceMissingException(
$"String '{value}' exceeds the single-byte length captured for event compact strings.");
}
writer.Write(CompactStringMarker);
writer.Write((byte)ascii.Length);
writer.Write((byte)0);
writer.Write(ascii);
}
/// <summary>
/// FILETIME truncated to whole milliseconds — the native event path stores millisecond
/// resolution and floors sub-millisecond ticks (observed in the capture).
/// </summary>
private static long ToMillisecondFileTime(DateTime value)
{
DateTime utc = value.Kind == DateTimeKind.Utc ? value : value.ToUniversalTime();
long ms = utc.Ticks / TimeSpan.TicksPerMillisecond;
DateTime truncated = new(ms * TimeSpan.TicksPerMillisecond, DateTimeKind.Utc);
return truncated.ToFileTimeUtc();
}
private static long ToFileTime(DateTime value)
{
DateTime utc = value.Kind == DateTimeKind.Utc ? value : value.ToUniversalTime();
return utc.ToFileTimeUtc();
}
}
@@ -33,7 +33,7 @@ internal static class HistorianNativeHandshake
/// upper-case context-key GUID, <paramref name="wrappedToken"/> is the AVEVA-wrapped SSPI
/// token (round byte + length + token). The WCF path maps this to
/// <c>Hist.ValidateClientCredential</c>; the gRPC path maps it to
/// <c>HistoryService.ExchangeKey</c> (the renamed handshake op).
/// <c>StorageService.ValidateClientCredential</c> (the op that kept the 2020 token framing).
/// </summary>
internal delegate TokenExchangeResult TokenExchange(string handle, byte[] wrappedToken, int round);
@@ -70,7 +70,8 @@ internal static class HistorianNativeHandshake
if (!result.Success)
{
throw new InvalidOperationException($"Credential token round {round} rejected (errorLen={error.Length}).");
throw new InvalidOperationException(
$"Credential token round {round} rejected (errorLen={error.Length}).{DescribeError(error)}");
}
ValidateClientCredentialResponse? response = HistorianWcfAuthenticationProtocol.TryReadValidateClientCredentialResponse(serverOutput);
@@ -162,4 +163,32 @@ internal static class HistorianNativeHandshake
int slash = userName.IndexOf('\\');
return slash > 0 ? userName[(slash + 1)..] : userName;
}
/// <summary>
/// Renders a diagnostic suffix for a rejected credential round: the decoded native error
/// (type/code/name) plus a short hex + printable-ASCII preview of the server error buffer.
/// Keeps secrets out — error buffers carry server status codes/messages, not credentials.
/// </summary>
private static string DescribeError(byte[] error)
{
if (error.Length == 0)
{
return string.Empty;
}
HistorianNativeError? native = HistorianOpen2Protocol.TryReadNativeError(error);
string nativePart = native is null
? string.Empty
: $" native(type={native.Type}, code={native.Code}{(native.Name is null ? string.Empty : $", {native.Name}")})";
ReadOnlySpan<byte> preview = error.AsSpan(0, Math.Min(error.Length, 64));
string hex = Convert.ToHexString(preview);
char[] ascii = new char[preview.Length];
for (int i = 0; i < preview.Length; i++)
{
ascii[i] = preview[i] is >= 0x20 and < 0x7F ? (char)preview[i] : '.';
}
return $"{nativePart} hex={hex} ascii=\"{new string(ascii)}\"";
}
}
@@ -0,0 +1,104 @@
using System.Buffers.Binary;
using System.Text;
using AVEVA.Historian.Client.Protocol;
namespace AVEVA.Historian.Client.Wcf;
/// <summary>
/// Serializes the <c>GETRP</c> (GetRuntimeParameter) request buffer and parses its response
/// buffer for AVEVA Historian 2020 over WCF/MDAS.
/// </summary>
/// <remarks>
/// Wire format captured from the native client (<c>scripts/Capture-RuntimeParam.ps1</c> +
/// instrument-wcf-{write,read}message; golden-pinned in <c>WcfRuntimeParameterProtocolTests</c>).
/// The op is <c>aa/Stat/GETRP(string handle, byte[] pRequestBuff) -&gt; (bool, byte[]
/// pResponseBuff, byte[] errorBuffer)</c> — the same string-handle + request/response-buffer
/// shape as <c>GETHI</c>.
///
/// <para><b>pRequestBuff</b> (44 bytes for one name "HistorianVersion"):
/// <c>54 67</c> signature · <c>01 00</c> version(1) · uint32 nameCount · then per name:
/// uint32 charCount + UTF-16LE chars.</para>
///
/// <para><b>pResponseBuff</b>: <c>01 00</c> version(1) · uint32 resultCount · then per result a
/// CRetVariant: <c>0x43</c> (VT_BSTR) + uint16 payloadLength + uint16 charCount + UTF-16LE chars.
/// Only the string variant (0x43) is evidence-backed; other variant types throw.</para>
///
/// <para>Only a single string-valued parameter is exercised against the live server; the
/// multi-name framing is inferred from the count field and not yet captured.</para>
/// </remarks>
internal static class HistorianRuntimeParameterProtocol
{
// Captured constant prefix of pRequestBuff: signature 0x6754 + version 0x0001.
private static ReadOnlySpan<byte> RequestHeader => [0x54, 0x67, 0x01, 0x00];
private const byte VariantTypeBStr = 0x43;
public static byte[] SerializeRequest(string parameterName)
{
ArgumentException.ThrowIfNullOrEmpty(parameterName);
using MemoryStream stream = new();
using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true);
writer.Write(RequestHeader); // 54 67 01 00
writer.Write(1u); // name count = 1
WriteName(writer, parameterName);
writer.Flush();
return stream.ToArray();
}
private static void WriteName(BinaryWriter writer, string name)
{
writer.Write((uint)name.Length); // char count
writer.Write(Encoding.Unicode.GetBytes(name));
}
/// <summary>
/// Parses the first string-valued result out of the GETRP response buffer, or null when the
/// buffer carries no result. Throws <see cref="ProtocolEvidenceMissingException"/> for any
/// non-string variant type (not yet captured).
/// </summary>
public static string? ParseSingleStringResult(ReadOnlySpan<byte> responseBuffer)
{
if (responseBuffer.Length < 6)
{
return null;
}
// [0..2) version, [2..6) result count.
uint resultCount = BinaryPrimitives.ReadUInt32LittleEndian(responseBuffer.Slice(2, 4));
if (resultCount == 0)
{
return null;
}
ReadOnlySpan<byte> cursor = responseBuffer[6..];
if (cursor.Length < 1)
{
return null;
}
byte variantType = cursor[0];
if (variantType != VariantTypeBStr)
{
throw new ProtocolEvidenceMissingException(
$"GETRP response variant type 0x{variantType:X2} is not the evidence-backed string variant (0x43).");
}
if (cursor.Length < 5)
{
return null;
}
ushort charCount = BinaryPrimitives.ReadUInt16LittleEndian(cursor.Slice(3, 2));
int byteCount = charCount * 2;
ReadOnlySpan<byte> valueBytes = cursor.Slice(5);
if (valueBytes.Length < byteCount)
{
byteCount = valueBytes.Length - (valueBytes.Length % 2);
}
return Encoding.Unicode.GetString(valueBytes[..byteCount]);
}
}
@@ -0,0 +1,221 @@
using System.Formats.Nrbf;
using System.Globalization;
using System.Xml.Linq;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Protocol;
namespace AVEVA.Historian.Client.Wcf;
/// <summary>
/// Parses the <c>GetR</c> (GetRecordSetByteStream) result buffer returned by the 2020 WCF
/// <c>ExeC</c>/<c>GetR</c> SQL surface into a <see cref="HistorianSqlResult"/>.
/// </summary>
/// <remarks>
/// Captured from the native client (<c>scripts/Capture-ExecSql.ps1</c> +
/// instrument-wcf-{write,read}message; golden-pinned in <c>WcfSqlResultProtocolTests</c>). The
/// <c>GetR</c> <c>pResultBuff</c> is a <b>.NET Remoting Binary Format (NRBF)</b> stream wrapping a
/// <c>System.Data.DataTable</c> serialized with <c>SerializationFormat.Xml</c>: its data lives in
/// two string members — <c>XmlSchema</c> (an XSD describing the columns + types) and
/// <c>XmlDiffGram</c> (an ADO.NET diffgram carrying the rows).
///
/// <para>BinaryFormatter is removed from .NET 10, so the stream is decoded read-only with
/// <see cref="NrbfDecoder"/> (the sanctioned successor — it parses records without instantiating
/// or executing any payload type). The two embedded XML strings are then parsed with
/// <see cref="XDocument"/>.</para>
///
/// <para>Only the <c>SerializationFormat.Xml</c> DataTable shape is evidence-backed; a stream whose
/// root is not a <c>DataTable</c> class record, or that lacks the two XML members, throws
/// <see cref="ProtocolEvidenceMissingException"/>.</para>
/// </remarks>
internal static class HistorianSqlResultProtocol
{
private const string DataTableTypeName = "System.Data.DataTable";
private const string XmlSchemaMember = "XmlSchema";
private const string XmlDiffGramMember = "XmlDiffGram";
private static readonly XNamespace Xsd = "http://www.w3.org/2001/XMLSchema";
private static readonly XNamespace MsData = "urn:schemas-microsoft-com:xml-msdata";
private static readonly XNamespace DiffGr = "urn:schemas-microsoft-com:xml-diffgram-v1";
public static HistorianSqlResult Parse(ReadOnlyMemory<byte> resultBuffer, int returnValue)
{
(string? schemaXml, string? diffGramXml) = ExtractXmlMembers(resultBuffer);
if (string.IsNullOrEmpty(schemaXml))
{
// No record set (e.g. a non-query, or an empty stream): return columns/rows empty.
return new HistorianSqlResult([], [], returnValue);
}
(string? tableName, List<HistorianSqlColumn> columns) = ParseSchema(schemaXml);
IReadOnlyList<IReadOnlyList<object?>> rows = ParseDiffGram(diffGramXml, tableName, columns);
return new HistorianSqlResult(columns, rows, returnValue);
}
private static (string? schemaXml, string? diffGramXml) ExtractXmlMembers(ReadOnlyMemory<byte> resultBuffer)
{
if (resultBuffer.Length == 0)
{
return (null, null);
}
SerializationRecord root;
try
{
using MemoryStream stream = new(resultBuffer.ToArray(), writable: false);
root = NrbfDecoder.Decode(stream);
}
catch (Exception ex) when (ex is not ProtocolEvidenceMissingException)
{
throw new ProtocolEvidenceMissingException(
$"GetR result buffer is not a decodable NRBF stream: {ex.Message}");
}
if (root is not ClassRecord classRecord || !classRecord.TypeNameMatches(typeof(System.Data.DataTable)))
{
// TypeNameMatches needs the real type; fall back to the raw type name when the assembly
// identity differs from the local one.
if (root is not ClassRecord cr || !string.Equals(cr.TypeName.FullName, DataTableTypeName, StringComparison.Ordinal))
{
throw new ProtocolEvidenceMissingException(
$"GetR result root record is not a {DataTableTypeName} (got '{(root as ClassRecord)?.TypeName.FullName ?? root.GetType().Name}').");
}
classRecord = cr;
}
if (!classRecord.HasMember(XmlSchemaMember) || !classRecord.HasMember(XmlDiffGramMember))
{
throw new ProtocolEvidenceMissingException(
$"GetR DataTable record is missing the {XmlSchemaMember}/{XmlDiffGramMember} members (non-Xml SerializationFormat not supported).");
}
return (classRecord.GetString(XmlSchemaMember), classRecord.GetString(XmlDiffGramMember));
}
private static (string? tableName, List<HistorianSqlColumn> columns) ParseSchema(string schemaXml)
{
XDocument doc = XDocument.Parse(schemaXml);
XElement? schema = doc.Root;
List<HistorianSqlColumn> columns = [];
if (schema is null)
{
return (null, columns);
}
// The dataset element advertises the main data table via msdata:MainDataTable.
XElement? dataSet = schema.Elements(Xsd + "element")
.FirstOrDefault(e => (string?)e.Attribute(MsData + "IsDataSet") == "true");
string? mainTable = (string?)dataSet?.Attribute(MsData + "MainDataTable");
// The table element (named by MainDataTable, else the first non-dataset top-level element).
XElement? tableElement = schema.Elements(Xsd + "element")
.FirstOrDefault(e => mainTable is not null && (string?)e.Attribute("name") == mainTable)
?? schema.Elements(Xsd + "element").FirstOrDefault(e => (string?)e.Attribute(MsData + "IsDataSet") != "true");
string? tableName = (string?)tableElement?.Attribute("name");
// Columns are the xs:element children of the table's complexType/sequence.
IEnumerable<XElement> columnElements = tableElement
?.Descendants(Xsd + "element")
.Where(e => e.Attribute("name") is not null)
?? [];
foreach (XElement column in columnElements)
{
string name = (string)column.Attribute("name")!;
string type = (string?)column.Attribute("type") ?? InlineSimpleType(column) ?? "xs:string";
columns.Add(new HistorianSqlColumn(name, type));
}
return (tableName, columns);
}
private static string? InlineSimpleType(XElement column)
{
XElement? restriction = column.Element(Xsd + "simpleType")?.Element(Xsd + "restriction");
return (string?)restriction?.Attribute("base");
}
private static IReadOnlyList<IReadOnlyList<object?>> ParseDiffGram(
string? diffGramXml,
string? tableName,
List<HistorianSqlColumn> columns)
{
if (string.IsNullOrEmpty(diffGramXml) || columns.Count == 0)
{
return [];
}
XDocument doc = XDocument.Parse(diffGramXml);
XElement? diffgram = doc.Root;
if (diffgram is null)
{
return [];
}
// The dataset element is the diffgram's child that is not in the diffgr: namespace
// (diffgr:before / diffgr:errors are change-tracking sections we don't need for a SELECT).
XElement? dataSet = diffgram.Elements().FirstOrDefault(e => e.Name.Namespace != DiffGr);
if (dataSet is null)
{
return [];
}
// Row elements are the dataset's children whose local name matches the table (or any child
// when the table name is unknown).
IEnumerable<XElement> rowElements = dataSet.Elements()
.Where(e => tableName is null || e.Name.LocalName == tableName);
List<IReadOnlyList<object?>> rows = [];
foreach (XElement rowElement in rowElements)
{
object?[] values = new object?[columns.Count];
for (int i = 0; i < columns.Count; i++)
{
HistorianSqlColumn column = columns[i];
// ADO.NET maps a column either as a child element or as an attribute on the row.
XElement? cell = rowElement.Elements().FirstOrDefault(e => e.Name.LocalName == column.Name);
string? raw = cell?.Value
?? rowElement.Attributes().FirstOrDefault(a => a.Name.LocalName == column.Name)?.Value;
values[i] = raw is null ? null : ConvertValue(raw, column.SchemaType);
}
rows.Add(values);
}
return rows;
}
private static object ConvertValue(string raw, string schemaType)
{
string type = schemaType.Contains(':') ? schemaType[(schemaType.IndexOf(':') + 1)..] : schemaType;
try
{
return type switch
{
"int" or "integer" => int.Parse(raw, CultureInfo.InvariantCulture),
"long" => long.Parse(raw, CultureInfo.InvariantCulture),
"short" => short.Parse(raw, CultureInfo.InvariantCulture),
"byte" => sbyte.Parse(raw, CultureInfo.InvariantCulture),
"unsignedByte" => byte.Parse(raw, CultureInfo.InvariantCulture),
"unsignedShort" => ushort.Parse(raw, CultureInfo.InvariantCulture),
"unsignedInt" => uint.Parse(raw, CultureInfo.InvariantCulture),
"unsignedLong" => ulong.Parse(raw, CultureInfo.InvariantCulture),
"double" => double.Parse(raw, CultureInfo.InvariantCulture),
"float" => float.Parse(raw, CultureInfo.InvariantCulture),
"decimal" => decimal.Parse(raw, CultureInfo.InvariantCulture),
"boolean" => ParseXsdBoolean(raw),
"dateTime" => DateTime.Parse(raw, CultureInfo.InvariantCulture, DateTimeStyles.RoundtripKind),
"base64Binary" => Convert.FromBase64String(raw),
_ => raw,
};
}
catch (Exception ex) when (ex is FormatException or OverflowException)
{
// Keep the raw string rather than throwing — the column type is informational and the
// server's textual rendering is always available.
return raw;
}
}
private static bool ParseXsdBoolean(string raw) =>
raw is "true" or "1" || (raw is not ("false" or "0") && bool.Parse(raw));
}
@@ -0,0 +1,304 @@
using System.Buffers.Binary;
using System.Text;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Protocol;
namespace AVEVA.Historian.Client.Wcf;
/// <summary>
/// Serializes the <c>GetTepByNm</c> (GetTagExtendedPropertiesFromName) request buffer and parses
/// its response buffer for AVEVA Historian 2020 over WCF/MDAS.
/// </summary>
/// <remarks>
/// Wire format captured from the native client (<c>scripts/Capture-TagExtendedProperties.ps1</c>
/// + instrument-wcf-{write,read}message; golden-pinned in
/// <c>WcfTagExtendedPropertyProtocolTests</c>). The op is
/// <c>aa/Retr/GetTepByNm(string handle, byte[] tagNames, ref uint sequence) -&gt; (bool,
/// byte[] tagExtendedProperties, byte[] errorBuffer)</c> — a string-handle op reachable from the
/// pure-managed client using the Open2 storage-session GUID as an <b>uppercase</b> handle (the
/// same format that unlocked GETRP/GETHI/ExeC; see <c>wcf-string-handle-wall.md</c>).
///
/// <para><b>tagNames</b> request buffer: <c>uint32 count</c> then per name <c>uint32 charCount</c>
/// + UTF-16LE chars.</para>
///
/// <para><b>tagExtendedProperties</b> response buffer: <c>uint32 tagCount</c> then per tag a
/// 1-byte group marker (observed <c>0x01</c>) + compact-ASCII tag name (<c>0x09</c> + uint16 byte
/// length + ASCII), <c>uint32 propertyCount</c>, then per property a 1-byte marker (observed
/// <c>0x02</c>) + compact-ASCII property name + a CRetVariant value (<c>0x43</c> VT_BSTR + uint16
/// payload length + uint16 charCount + UTF-16LE), and a 1-byte trailing marker (observed
/// <c>0x01</c>). Only the string value variant (<c>0x43</c>) is evidence-backed; other variant
/// types throw <see cref="ProtocolEvidenceMissingException"/>.</para>
///
/// <para>The op is sequence-paged: call with <c>sequence = 0</c>, parse the buffer, then re-call
/// with the returned sequence until the response carries no rows. Only string-valued properties
/// from a single capture are exercised; the single-byte group/property markers are pinned as
/// observed constants and their full semantics are not independently disambiguated.</para>
/// </remarks>
internal static class HistorianTagExtendedPropertyProtocol
{
private const byte CompactStringMarker = 0x09;
private const byte VariantTypeBStr = 0x43;
/// <summary>Serializes the <c>tagNames</c> request buffer for a single tag.</summary>
public static byte[] SerializeRequest(string tagName)
{
ArgumentException.ThrowIfNullOrEmpty(tagName);
using MemoryStream stream = new();
using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true);
writer.Write(1u); // name count = 1
writer.Write((uint)tagName.Length); // char count
writer.Write(Encoding.Unicode.GetBytes(tagName));
writer.Flush();
return stream.ToArray();
}
private const byte GroupMarker = 0x01;
private const byte PropertyMarker = 0x02;
/// <summary>
/// Serializes the <c>AddTEx</c> (AddTagExtendedProperties) inBuff for a single tag's string
/// properties. The buffer is the exact inverse of the <see cref="ParseResponse"/> group framing
/// (decoded from a native <c>AddTagExtendedProperties</c> capture; see
/// <c>docs/reverse-engineering/wcf-add-tag-extended-properties.md</c>):
/// <code>
/// uint32 groupCount (= 1)
/// byte 0x01 (group marker)
/// 0x09 + uint16 byteLen + ASCII tagName
/// uint32 propertyCount
/// repeated: byte 0x02 (property marker) + 0x09 + uint16 byteLen + ASCII propName
/// + 0x43 VT_BSTR + uint16 payloadLen + uint16 charCount + UTF-16LE value
/// byte 0x01 (group trailer)
/// </code>
/// Only string-valued properties are evidence-backed (the VT_BSTR variant), matching the read path.
/// </summary>
public static byte[] SerializeAddRequest(string tagName, IReadOnlyList<HistorianTagExtendedProperty> properties)
{
ArgumentException.ThrowIfNullOrEmpty(tagName);
ArgumentNullException.ThrowIfNull(properties);
if (properties.Count == 0)
{
throw new ArgumentException("At least one extended property is required.", nameof(properties));
}
using MemoryStream stream = new();
Span<byte> u32 = stackalloc byte[4];
BinaryPrimitives.WriteUInt32LittleEndian(u32, 1u); // group count
stream.Write(u32);
stream.WriteByte(GroupMarker);
WriteCompactAscii(stream, tagName);
BinaryPrimitives.WriteUInt32LittleEndian(u32, checked((uint)properties.Count));
stream.Write(u32);
foreach (HistorianTagExtendedProperty property in properties)
{
ArgumentException.ThrowIfNullOrEmpty(property?.Name, nameof(properties));
stream.WriteByte(PropertyMarker);
WriteCompactAscii(stream, property.Name);
WriteVariantString(stream, property.Value ?? string.Empty);
}
stream.WriteByte(GroupMarker); // group trailer
stream.WriteByte(0x00); // buffer terminator (captured: the native inBuff ends 0x01 0x00)
return stream.ToArray();
}
/// <summary>
/// Serializes the <c>DelTep</c> (DeleteTagExtendedProperties) inBuff for a single tag's
/// properties identified by name. Same group framing as <see cref="SerializeAddRequest"/> but
/// each property carries its name only (no <c>0x43</c> value variant) and the group trailer byte
/// is <c>0x00</c> (Add uses <c>0x01</c>). Decoded from a native
/// <c>DeleteTagExtendedPropertiesByName</c> capture (see
/// <c>docs/reverse-engineering/wcf-add-tag-extended-properties.md</c> §Delete):
/// <code>
/// uint32 groupCount (= 1)
/// byte 0x01 (group marker)
/// 0x09 + uint16 byteLen + ASCII tagName
/// uint32 propertyCount
/// repeated: byte 0x02 (property marker) + 0x09 + uint16 byteLen + ASCII propName
/// byte 0x00 (group trailer)
/// byte 0x00 (buffer terminator)
/// </code>
/// The native <c>deleteFromServer</c> argument is not in the buffer — it is the client-side flag
/// that decides whether the wire op fires at all (the SDK always sends the server-delete form,
/// the only one that produces a <c>DelTep</c> call).
/// </summary>
public static byte[] SerializeDeleteRequest(string tagName, IReadOnlyList<string> propertyNames)
{
ArgumentException.ThrowIfNullOrEmpty(tagName);
ArgumentNullException.ThrowIfNull(propertyNames);
if (propertyNames.Count == 0)
{
throw new ArgumentException("At least one property name is required.", nameof(propertyNames));
}
using MemoryStream stream = new();
Span<byte> u32 = stackalloc byte[4];
BinaryPrimitives.WriteUInt32LittleEndian(u32, 1u); // group count
stream.Write(u32);
stream.WriteByte(GroupMarker);
WriteCompactAscii(stream, tagName);
BinaryPrimitives.WriteUInt32LittleEndian(u32, checked((uint)propertyNames.Count));
stream.Write(u32);
foreach (string name in propertyNames)
{
ArgumentException.ThrowIfNullOrEmpty(name, nameof(propertyNames));
stream.WriteByte(PropertyMarker);
WriteCompactAscii(stream, name); // delete is by name only — no value variant
}
stream.WriteByte(0x00); // group trailer (captured: 0x00 for delete, vs 0x01 for add)
stream.WriteByte(0x00); // buffer terminator
return stream.ToArray();
}
private static void WriteCompactAscii(MemoryStream stream, string value)
{
byte[] ascii = Encoding.ASCII.GetBytes(value);
stream.WriteByte(CompactStringMarker);
Span<byte> u16 = stackalloc byte[2];
BinaryPrimitives.WriteUInt16LittleEndian(u16, checked((ushort)ascii.Length));
stream.Write(u16);
stream.Write(ascii, 0, ascii.Length);
}
private static void WriteVariantString(MemoryStream stream, string value)
{
byte[] utf16 = Encoding.Unicode.GetBytes(value);
ushort charCount = checked((ushort)value.Length);
stream.WriteByte(VariantTypeBStr);
Span<byte> u16 = stackalloc byte[2];
// payloadLen = the bytes that follow it: the uint16 charCount field (2) + UTF-16 bytes.
BinaryPrimitives.WriteUInt16LittleEndian(u16, checked((ushort)(2 + utf16.Length)));
stream.Write(u16);
BinaryPrimitives.WriteUInt16LittleEndian(u16, charCount);
stream.Write(u16);
stream.Write(utf16, 0, utf16.Length);
}
/// <summary>
/// Parses the <c>tagExtendedProperties</c> response buffer into a flat list of
/// (tagName, propertyName, value) rows. Returns an empty list when the buffer carries no rows
/// (the terminal page of the sequence loop), which is how callers know to stop paging.
/// </summary>
public static IReadOnlyList<HistorianTagExtendedPropertyRow> ParseResponse(ReadOnlySpan<byte> buffer)
{
if (buffer.Length < 4)
{
return [];
}
int cursor = 0;
uint tagCount = ReadUInt32(buffer, ref cursor);
if (tagCount == 0)
{
return [];
}
List<HistorianTagExtendedPropertyRow> rows = [];
for (uint t = 0; t < tagCount; t++)
{
// 1-byte group marker (observed 0x01).
SkipByte(buffer, ref cursor);
string tagName = ReadCompactAsciiString(buffer, ref cursor);
uint propertyCount = ReadUInt32(buffer, ref cursor);
for (uint p = 0; p < propertyCount; p++)
{
// 1-byte property marker (observed 0x02 — likely the property value type).
SkipByte(buffer, ref cursor);
string propertyName = ReadCompactAsciiString(buffer, ref cursor);
string value = ReadVariantStringValue(buffer, ref cursor);
rows.Add(new HistorianTagExtendedPropertyRow(tagName, propertyName, value));
}
// 1-byte trailing marker (observed 0x01) after the property block. Read it only if a
// byte remains so a tightly-packed terminal buffer doesn't over-read.
if (cursor < buffer.Length)
{
SkipByte(buffer, ref cursor);
}
}
return rows;
}
private static string ReadCompactAsciiString(ReadOnlySpan<byte> buffer, ref int cursor)
{
EnsureAvailable(buffer, cursor, 3);
byte marker = buffer[cursor++];
if (marker != CompactStringMarker)
{
throw new ProtocolEvidenceMissingException(
$"GetTepByNm response expected compact string marker 0x09, found 0x{marker:X2}.");
}
ushort byteLength = ReadUInt16(buffer, ref cursor);
EnsureAvailable(buffer, cursor, byteLength);
string value = Encoding.ASCII.GetString(buffer.Slice(cursor, byteLength));
cursor += byteLength;
return value;
}
private static string ReadVariantStringValue(ReadOnlySpan<byte> buffer, ref int cursor)
{
EnsureAvailable(buffer, cursor, 5);
byte variantType = buffer[cursor++];
if (variantType != VariantTypeBStr)
{
throw new ProtocolEvidenceMissingException(
$"GetTepByNm response value variant type 0x{variantType:X2} is not the evidence-backed string variant (0x43).");
}
// uint16 payload length (bytes that follow the charCount field), then uint16 charCount.
_ = ReadUInt16(buffer, ref cursor);
ushort charCount = ReadUInt16(buffer, ref cursor);
int byteCount = charCount * 2;
EnsureAvailable(buffer, cursor, byteCount);
string value = Encoding.Unicode.GetString(buffer.Slice(cursor, byteCount));
cursor += byteCount;
return value;
}
private static void SkipByte(ReadOnlySpan<byte> buffer, ref int cursor)
{
EnsureAvailable(buffer, cursor, 1);
cursor++;
}
private static ushort ReadUInt16(ReadOnlySpan<byte> buffer, ref int cursor)
{
EnsureAvailable(buffer, cursor, 2);
ushort value = BinaryPrimitives.ReadUInt16LittleEndian(buffer.Slice(cursor, 2));
cursor += 2;
return value;
}
private static uint ReadUInt32(ReadOnlySpan<byte> buffer, ref int cursor)
{
EnsureAvailable(buffer, cursor, 4);
uint value = BinaryPrimitives.ReadUInt32LittleEndian(buffer.Slice(cursor, 4));
cursor += 4;
return value;
}
private static void EnsureAvailable(ReadOnlySpan<byte> buffer, int cursor, int byteCount)
{
if (cursor < 0 || byteCount < 0 || cursor > buffer.Length - byteCount)
{
throw new InvalidDataException("GetTepByNm response ended unexpectedly.");
}
}
}
/// <summary>One (tag, propertyName, value) row parsed from a GetTepByNm response buffer.</summary>
internal sealed record HistorianTagExtendedPropertyRow(string TagName, string PropertyName, string Value);
@@ -102,6 +102,34 @@ internal static class HistorianTagQueryProtocol
return tagNames;
}
/// <summary>
/// Parses one page of a gRPC <c>QueryTag</c> tag-name response: <c>uint count + per-name(uint
/// charCount + UTF-16LE)</c>, then a trailing region (NextIndex + optional metadata buffer) that
/// is intentionally ignored. Unlike <see cref="ParseGetLikeTagNamesResponse"/> this tolerates the
/// trailer rather than requiring the buffer to end exactly after the names.
/// </summary>
public static IReadOnlyList<string> ParseTagNameQueryPage(ReadOnlySpan<byte> response)
{
if (response.Length < 4)
{
return [];
}
int cursor = 0;
uint count = ReadUInt32(response, ref cursor);
List<string> tagNames = new(checked((int)count));
for (uint index = 0; index < count; index++)
{
uint charLength = ReadUInt32(response, ref cursor);
int byteLength = checked((int)charLength * 2);
EnsureAvailable(response, cursor, byteLength);
tagNames.Add(Encoding.Unicode.GetString(response.Slice(cursor, byteLength)));
cursor += byteLength;
}
return tagNames;
}
private static void WriteHistorianString(BinaryWriter writer, string value)
{
writer.Write((uint)value.Length);
@@ -0,0 +1,66 @@
using System.Buffers.Binary;
using System.Text;
namespace AVEVA.Historian.Client.Wcf;
/// <summary>
/// Serializes the History <c>StartJob</c> (StJb) request buffer for a tag-rename job (HCAL R1.10).
///
/// Tag rename has no dedicated WCF operation — the native client packs a batch of (old,new) name
/// pairs into the generic <c>StartJob</c> job buffer and the server returns a job id (the rename then
/// completes asynchronously; see <see cref="Models.HistorianTagRenameResult"/>).
///
/// Wire layout (decoded from an instrumented native <c>RenameTags</c> capture, see
/// <c>docs/reverse-engineering/wcf-rename-tags.md</c>):
/// <code>
/// byte[7] reserved/job-descriptor prefix (all zero in every capture)
/// uint32 pairCount
/// repeated pairCount times:
/// uint32 oldNameCharCount + UTF-16LE oldName (no terminator)
/// uint32 newNameCharCount + UTF-16LE newName (no terminator)
/// </code>
/// Char counts are UTF-16 code-unit counts (byte length / 2). The pair order is (old, new) — the tag
/// being renamed first, its new name second.
/// </summary>
internal static class HistorianTagRenameProtocol
{
/// <summary>
/// Captured fixed prefix that leads the rename job buffer (7 zero bytes in every observed
/// capture). Treated as an opaque job-descriptor constant rather than guessing its sub-fields.
/// </summary>
private static readonly byte[] JobBufferPrefix = new byte[7];
public static byte[] SerializeRenameJob(IReadOnlyList<(string OldName, string NewName)> pairs)
{
ArgumentNullException.ThrowIfNull(pairs);
if (pairs.Count == 0)
{
throw new ArgumentException("At least one (old,new) name pair is required.", nameof(pairs));
}
using MemoryStream ms = new();
ms.Write(JobBufferPrefix, 0, JobBufferPrefix.Length);
Span<byte> u32 = stackalloc byte[4];
BinaryPrimitives.WriteUInt32LittleEndian(u32, checked((uint)pairs.Count));
ms.Write(u32);
foreach ((string oldName, string newName) in pairs)
{
ArgumentException.ThrowIfNullOrWhiteSpace(oldName, nameof(pairs));
ArgumentException.ThrowIfNullOrWhiteSpace(newName, nameof(pairs));
WriteCountedUtf16(ms, oldName, u32);
WriteCountedUtf16(ms, newName, u32);
}
return ms.ToArray();
}
private static void WriteCountedUtf16(MemoryStream ms, string value, Span<byte> u32)
{
byte[] bytes = Encoding.Unicode.GetBytes(value);
BinaryPrimitives.WriteUInt32LittleEndian(u32, (uint)value.Length);
ms.Write(u32);
ms.Write(bytes, 0, bytes.Length);
}
}
@@ -66,6 +66,7 @@ internal sealed class HistorianWcfEventOrchestrator
public async IAsyncEnumerable<HistorianEvent> ReadEventsAsync(
DateTime startUtc,
DateTime endUtc,
HistorianEventFilter? filter,
[EnumeratorCancellation] CancellationToken cancellationToken)
{
if (!_options.IntegratedSecurity && string.IsNullOrEmpty(_options.UserName))
@@ -77,7 +78,7 @@ internal sealed class HistorianWcfEventOrchestrator
cancellationToken.ThrowIfCancellationRequested();
IReadOnlyList<HistorianEvent> events = await Task.Run(
() => RunEventChain(startUtc, endUtc, cancellationToken),
() => RunEventChain(startUtc, endUtc, filter, cancellationToken),
cancellationToken).ConfigureAwait(false);
foreach (HistorianEvent evt in events)
@@ -87,7 +88,63 @@ internal sealed class HistorianWcfEventOrchestrator
}
}
private List<HistorianEvent> RunEventChain(DateTime startUtc, DateTime endUtc, CancellationToken cancellationToken)
/// <summary>Diagnostic: type+code description of the most recent AddS2 (event send) error buffer.</summary>
public string LastSendErrorDescription { get; private set; } = string.Empty;
/// <summary>
/// Sends a single <see cref="HistorianEvent"/> to the built-in CM_EVENT tag via the
/// captured WCF event-send chain: Open2 (event mode 0x501) → CM_EVENT registration
/// (RTag2 + EnsT2) → AddS2 (<c>AddStreamValues2</c>) carrying the serialized event VTQ.
/// Returns the server's AddS2 result.
/// </summary>
public async Task<bool> SendEventAsync(HistorianEvent evt, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(evt);
if (!_options.IntegratedSecurity && string.IsNullOrEmpty(_options.UserName))
{
throw new ProtocolEvidenceMissingException(
"Managed event send currently requires IntegratedSecurity or an explicit UserName + Password.");
}
if (evt.RevisionVersion != 0)
{
throw new ProtocolEvidenceMissingException(
"Only original events (RevisionVersion = 0) have a captured send encoding; " +
"revision/update/delete event sends are not yet supported.");
}
cancellationToken.ThrowIfCancellationRequested();
return await Task.Run(() => RunSendEventChain(evt, cancellationToken), cancellationToken).ConfigureAwait(false);
}
private bool RunSendEventChain(HistorianEvent evt, CancellationToken cancellationToken)
{
Guid contextKey = Guid.NewGuid();
var (histBinding, histEndpoint, retrBinding, retrEndpoint) = HistorianWcfBindingFactory.CreateBindingPair(_options);
Binding auxBinding = HistorianWcfBindingFactory.CreateAuxiliaryBinding(_options);
EndpointAddress statusEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(_options, HistorianWcfServiceNames.Status);
EndpointAddress transactionEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(_options, HistorianWcfServiceNames.Transaction);
bool sent = false;
HistorianWcfAuthChainHelper.OpenAuthenticatedConnection(
_options, histBinding, histEndpoint, contextKey, cancellationToken,
connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedEventConnectionMode,
additionalSetup: (historyChannel, context) =>
{
// Register CM_EVENT for this session (RTag2 + EnsT2) exactly as the native
// event-send chain does, then stream the event value via AddS2 on the same
// /Hist channel + storage-session handle.
AddCmEventTagViaAddT(historyChannel, context, auxBinding, statusEndpoint, transactionEndpoint, retrBinding, retrEndpoint);
byte[] pBuf = HistorianEventWriteProtocol.SerializeAddStreamValuesBuffer(evt, DateTime.UtcNow);
string handle = context.StorageSessionId.ToString("D").ToUpperInvariant();
sent = historyChannel.AddStreamValues2(handle, pBuf, out byte[] errorBuffer);
LastSendErrorDescription = DescribeNativeError(errorBuffer ?? []);
});
return sent;
}
private List<HistorianEvent> RunEventChain(DateTime startUtc, DateTime endUtc, HistorianEventFilter? filter, CancellationToken cancellationToken)
{
Guid contextKey = Guid.NewGuid();
var (histBinding, histEndpoint, retrBinding, retrEndpoint) = HistorianWcfBindingFactory.CreateBindingPair(_options);
@@ -99,7 +156,7 @@ internal sealed class HistorianWcfEventOrchestrator
connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode,
additionalSetup: (historyChannel, context) =>
AddCmEventTagViaAddT(historyChannel, context, auxBinding, statusEndpoint, transactionEndpoint, retrBinding, retrEndpoint));
return RunEventQuery(retrBinding, retrEndpoint, clientHandle, startUtc, endUtc, cancellationToken);
return RunEventQuery(retrBinding, retrEndpoint, clientHandle, startUtc, endUtc, filter, cancellationToken);
}
private List<HistorianEvent> RunEventQuery(
@@ -108,6 +165,7 @@ internal sealed class HistorianWcfEventOrchestrator
uint clientHandle,
DateTime startUtc,
DateTime endUtc,
HistorianEventFilter? filter,
CancellationToken cancellationToken)
{
ChannelFactory<IRetrievalServiceContract4> factory = new(binding, retrievalEndpoint);
@@ -131,7 +189,8 @@ internal sealed class HistorianWcfEventOrchestrator
IReadOnlyList<HistorianEventQueryAttempt> attempts = HistorianEventQueryProtocol.CreateStartEventQueryAttempts(
startUtc.ToUniversalTime(),
endUtc.ToUniversalTime(),
eventCount: 5);
eventCount: 5,
filter);
byte[] requestBuffer = attempts[0].RequestBuffer;
uint queryHandle = 0;
@@ -0,0 +1,132 @@
using System.ServiceModel;
using System.ServiceModel.Channels;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf.Contracts;
namespace AVEVA.Historian.Client.Wcf;
/// <summary>
/// Executes SQL commands (HCAL R1.1) over the 2020 WCF <c>aa/Retr/ExeC</c> + <c>aa/Retr/GetR</c>
/// ops. Both are string-handle ops reached with the Open2 storage-session GUID formatted UPPERCASE
/// (the same handle format that unlocked GETRP/GETHI; see <c>wcf-string-handle-wall.md</c>). The
/// record set comes back from <c>GetR</c> as an NRBF-serialized <c>DataTable</c>, parsed by
/// <see cref="HistorianSqlResultProtocol"/>.
/// </summary>
internal static class HistorianWcfSqlClient
{
// GetR is byte-stream-paged. A small record set returns the whole stream in the first call;
// larger ones chunk across calls accumulated here. The cap is a runaway guard.
private const int MaxPages = 4096;
/// <summary>Diagnostic: the ExeC/GetR error from the last call (set only on server rejection).</summary>
public static string? LastError { get; private set; }
public static Task<HistorianSqlResult> ExecuteSqlCommandAsync(
HistorianClientOptions options,
string command,
HistorianSqlExecuteOption option,
CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(command);
return Task.Run(() => ExecuteSqlCommand(options, command, option), cancellationToken);
}
private static HistorianSqlResult ExecuteSqlCommand(
HistorianClientOptions options,
string command,
HistorianSqlExecuteOption option)
{
Guid contextKey = Guid.NewGuid();
var (histBinding, histEndpoint, retrBinding, retrEndpoint) = HistorianWcfBindingFactory.CreateBindingPair(options);
LastError = null;
HistorianSqlResult? result = null;
HistorianWcfAuthChainHelper.OpenAuthenticatedConnection(
options, histBinding, histEndpoint, contextKey, CancellationToken.None,
additionalSetup: (_, context) =>
result = RunSql(retrBinding, retrEndpoint, options, context.StorageSessionId, command, option));
return result ?? new HistorianSqlResult([], [], 0);
}
private static HistorianSqlResult RunSql(
Binding retrBinding,
EndpointAddress retrEndpoint,
HistorianClientOptions options,
Guid storageSessionId,
string command,
HistorianSqlExecuteOption option)
{
// ExeC/GetR take the storage-session GUID as a string handle, uppercase dash-no-braces.
string handle = storageSessionId.ToString("D").ToUpperInvariant();
ChannelFactory<IRetrievalServiceContract3> factory = new(retrBinding, retrEndpoint);
HistorianWcfClientCredentialsHelper.Configure(factory, options);
IRetrievalServiceContract3 channel = factory.CreateChannel();
ICommunicationObject co = (ICommunicationObject)channel;
try
{
// Prime the Retrieval service version handshake (Retr.GetV).
channel.GetInterfaceVersion(out _);
uint queryHandle = 0;
bool execOk = channel.ExecuteSqlCommand(
handle, command, (uint)option, ref queryHandle,
out int returnValue, out uint errSize, out byte[] errBuf);
if (!execOk)
{
LastError = $"ExeC returned false (retValue={returnValue}, errSize={errSize}, errLen={errBuf?.Length ?? 0}).";
return new HistorianSqlResult([], [], returnValue);
}
using MemoryStream accumulated = new();
uint sequence = 0;
for (int page = 0; page < MaxPages; page++)
{
bool getrOk = channel.GetRecordSetByteStream(
handle, queryHandle, ref sequence,
out uint resultSize, out byte[] resultBuffer, out uint gErrSize, out byte[] gErrBuf);
// GetRResult is false even on a successful single-page read — the byte stream is in
// resultBuffer regardless, and false signals "this is the final page". So always
// consume the buffer first, then stop on a false result or an empty page.
if (resultBuffer is { Length: > 0 })
{
accumulated.Write(resultBuffer, 0, resultBuffer.Length);
}
if (!getrOk || resultBuffer is null || resultBuffer.Length == 0)
{
if (page == 0 && accumulated.Length == 0 && gErrBuf is { Length: > 0 })
{
LastError = $"GetR error (resultSize={resultSize}, errSize={gErrSize}, errLen={gErrBuf.Length}).";
}
break;
}
}
byte[] stream = accumulated.ToArray();
DumpStreamIfRequested(stream);
return HistorianSqlResultProtocol.Parse(stream, returnValue);
}
finally
{
try { if (co.State == CommunicationState.Faulted) co.Abort(); else co.Close(); } catch { try { co.Abort(); } catch { } }
try { if (factory.State == CommunicationState.Faulted) factory.Abort(); else factory.Close(); } catch { try { factory.Abort(); } catch { } }
}
}
/// <summary>
/// Reverse-engineering aid: when <c>AVEVA_HISTORIAN_SQL_DUMP</c> is set to a path, writes the
/// base64 of the reassembled GetR NRBF stream there (the clean contract-level byte[], free of
/// the MDAS transport chunk markers that corrupt a raw instrument-wcf capture). Used once to
/// produce the golden fixture for <c>WcfSqlResultProtocolTests</c>.
/// </summary>
private static void DumpStreamIfRequested(byte[] stream)
{
string? path = Environment.GetEnvironmentVariable("AVEVA_HISTORIAN_SQL_DUMP");
if (!string.IsNullOrWhiteSpace(path) && stream.Length > 0)
{
try { File.WriteAllText(path, Convert.ToBase64String(stream)); } catch { /* diagnostic only */ }
}
}
}
@@ -17,6 +17,19 @@ internal static class HistorianWcfStatusClient
return Task.Run(() => GetSystemParameter(options, parameterName), cancellationToken);
}
/// <summary>Diagnostic: the GETRP return code / error description from the last
/// <see cref="GetRuntimeParameterAsync"/> call (set only when the server rejects it).</summary>
public static string? LastRuntimeParameterError { get; private set; }
public static Task<string?> GetRuntimeParameterAsync(
HistorianClientOptions options,
string parameterName,
CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(parameterName);
return Task.Run(() => GetRuntimeParameter(options, parameterName), cancellationToken);
}
public static Task<HistorianConnectionStatus> GetConnectionStatusAsync(
HistorianClientOptions options,
CancellationToken cancellationToken)
@@ -45,6 +58,49 @@ internal static class HistorianWcfStatusClient
return value;
}
private static string? GetRuntimeParameter(HistorianClientOptions options, string parameterName)
{
Guid contextKey = Guid.NewGuid();
var (histBinding, histEndpoint, _, _) = HistorianWcfBindingFactory.CreateBindingPair(options);
Binding statusBinding = HistorianWcfBindingFactory.CreateAuxiliaryBinding(options);
EndpointAddress statusEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(options, HistorianWcfServiceNames.Status);
string? value = null;
LastRuntimeParameterError = null;
HistorianWcfAuthChainHelper.OpenAuthenticatedConnection(
options, histBinding, histEndpoint, contextKey, CancellationToken.None,
additionalSetup: (_, context) => value = QueryRuntimeParameter(statusBinding, statusEndpoint, context.StorageSessionId, parameterName));
return value;
}
private static string? QueryRuntimeParameter(Binding statusBinding, EndpointAddress statusEndpoint, Guid storageSessionId, string parameterName)
{
// GETRP takes the storage-session GUID as a string handle, formatted exactly as the
// native client sends it: uppercase, dash-separated, no braces.
string handle = storageSessionId.ToString("D").ToUpperInvariant();
byte[] requestBuffer = HistorianRuntimeParameterProtocol.SerializeRequest(parameterName);
ChannelFactory<IStatusServiceContract2> factory = new(statusBinding, statusEndpoint);
IStatusServiceContract2 channel = factory.CreateChannel();
ICommunicationObject co = (ICommunicationObject)channel;
try
{
bool ok = channel.GetRuntimeParameter(handle, requestBuffer, out byte[] responseBuffer, out byte[] errorBuffer);
if (!ok)
{
LastRuntimeParameterError = $"GETRP returned false (responseLen={responseBuffer?.Length ?? 0}, errorLen={errorBuffer?.Length ?? 0}).";
return null;
}
return HistorianRuntimeParameterProtocol.ParseSingleStringResult(responseBuffer ?? []);
}
finally
{
try { if (co.State == CommunicationState.Faulted) co.Abort(); else co.Close(); } catch { try { co.Abort(); } catch { } }
try { if (factory.State == CommunicationState.Faulted) factory.Abort(); else factory.Close(); } catch { try { factory.Abort(); } catch { } }
}
}
private static string? QuerySystemParameter(Binding statusBinding, EndpointAddress statusEndpoint, uint clientHandle, string parameterName)
{
ChannelFactory<IStatusServiceContract2> factory = new(statusBinding, statusEndpoint);
@@ -0,0 +1,107 @@
using System.ServiceModel;
using System.ServiceModel.Channels;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf.Contracts;
namespace AVEVA.Historian.Client.Wcf;
/// <summary>
/// Reads tag extended properties (HCAL R1.5) over the 2020 WCF <c>aa/Retr/GetTepByNm</c> op. This
/// is a string-handle op reached with the Open2 storage-session GUID formatted UPPERCASE — the same
/// handle format that unlocked GETRP/GETHI/ExeC (see <c>wcf-string-handle-wall.md</c>).
/// </summary>
internal static class HistorianWcfTagExtendedPropertyClient
{
// GetTepByNm is sequence-paged. A single tag with a handful of properties returns everything in
// the first buffer and an empty buffer on the next call; the cap is a runaway guard.
private const int MaxPages = 64;
/// <summary>Diagnostic: the GetTepByNm return code / error from the last call (set only on
/// server rejection).</summary>
public static string? LastError { get; private set; }
public static Task<IReadOnlyList<HistorianTagExtendedProperty>> GetTagExtendedPropertiesAsync(
HistorianClientOptions options,
string tag,
CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tag);
return Task.Run(() => GetTagExtendedProperties(options, tag), cancellationToken);
}
private static IReadOnlyList<HistorianTagExtendedProperty> GetTagExtendedProperties(
HistorianClientOptions options,
string tag)
{
Guid contextKey = Guid.NewGuid();
var (histBinding, histEndpoint, retrBinding, retrEndpoint) = HistorianWcfBindingFactory.CreateBindingPair(options);
LastError = null;
List<HistorianTagExtendedProperty> properties = [];
HistorianWcfAuthChainHelper.OpenAuthenticatedConnection(
options, histBinding, histEndpoint, contextKey, CancellationToken.None,
additionalSetup: (_, context) =>
QueryExtendedProperties(retrBinding, retrEndpoint, options, context.StorageSessionId, tag, properties));
return properties;
}
private static void QueryExtendedProperties(
Binding retrBinding,
EndpointAddress retrEndpoint,
HistorianClientOptions options,
Guid storageSessionId,
string tag,
List<HistorianTagExtendedProperty> sink)
{
// GetTepByNm takes the storage-session GUID as a string handle, formatted exactly as the
// native client sends it: uppercase, dash-separated, no braces.
string handle = storageSessionId.ToString("D").ToUpperInvariant();
byte[] tagNames = HistorianTagExtendedPropertyProtocol.SerializeRequest(tag);
ChannelFactory<IRetrievalServiceContract4> factory = new(retrBinding, retrEndpoint);
HistorianWcfClientCredentialsHelper.Configure(factory, options);
IRetrievalServiceContract4 channel = factory.CreateChannel();
ICommunicationObject co = (ICommunicationObject)channel;
try
{
// Prime the Retrieval service version handshake (Retr.GetV), as the native client does
// before any string-handle Retrieval op.
channel.GetInterfaceVersion(out _);
uint sequence = 0;
for (int page = 0; page < MaxPages; page++)
{
bool ok = channel.GetTagExtendedPropertiesFromName(
handle, tagNames, ref sequence, out byte[] responseBuffer, out byte[] errorBuffer);
if (!ok)
{
// A non-success on the first page is a real failure; on a later page it is the
// benign end-of-paging terminator (server returns nil + a FillBufferFromVector
// marker once the rows are exhausted).
if (page == 0)
{
LastError = $"GetTepByNm returned false (responseLen={responseBuffer?.Length ?? 0}, errorLen={errorBuffer?.Length ?? 0}).";
}
break;
}
IReadOnlyList<HistorianTagExtendedPropertyRow> rows =
HistorianTagExtendedPropertyProtocol.ParseResponse(responseBuffer ?? []);
if (rows.Count == 0)
{
break;
}
foreach (HistorianTagExtendedPropertyRow row in rows)
{
sink.Add(new HistorianTagExtendedProperty(row.PropertyName, row.Value));
}
}
}
finally
{
try { if (co.State == CommunicationState.Faulted) co.Abort(); else co.Close(); } catch { try { co.Abort(); } catch { } }
try { if (factory.State == CommunicationState.Faulted) factory.Abort(); else factory.Close(); } catch { try { factory.Abort(); } catch { } }
}
}
}
@@ -41,6 +41,41 @@ internal sealed class HistorianWcfTagWriteOrchestrator
return Task.Run(() => DeleteTag(tagName), cancellationToken);
}
public Task<bool> AddTagExtendedPropertiesAsync(
string tagName, IReadOnlyList<HistorianTagExtendedProperty> properties, CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tagName);
ArgumentNullException.ThrowIfNull(properties);
if (properties.Count == 0)
{
throw new ArgumentException("At least one extended property is required.", nameof(properties));
}
return Task.Run(() => AddTagExtendedProperties(tagName, properties), cancellationToken);
}
/// <summary>
/// DelTep (DeleteTagExtendedProperties) — <b>investigated but server-blocked, not wired to the
/// public API.</b> The inBuff serializer is golden-verified against a server-accepted native
/// capture, and an SDK-added property is deletable (the native client read-syncs and deletes it).
/// But the SDK's own delete is rejected by <c>CHistStorage::DeleteTagExtendedProperties</c>
/// (SErrorException) even with matching Open2 mode/handle/inBuff and a GetTgByNm+GetTepByNm prime:
/// the server consults a per-connection working set the native client populates by multiplexing
/// GetTepByNm and DelTep over one connection, which the SDK's per-service WCF channels don't
/// reproduce. Kept for follow-up RE. See
/// docs/reverse-engineering/wcf-add-tag-extended-properties.md §Delete.
/// </summary>
internal Task<bool> DeleteTagExtendedPropertiesAsync(
string tagName, IReadOnlyList<string> propertyNames, CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tagName);
ArgumentNullException.ThrowIfNull(propertyNames);
if (propertyNames.Count == 0)
{
throw new ArgumentException("At least one property name is required.", nameof(propertyNames));
}
return Task.Run(() => DeleteTagExtendedProperties(tagName, propertyNames), cancellationToken);
}
private bool EnsureTag(HistorianTagDefinition definition)
{
Guid contextKey = Guid.NewGuid();
@@ -86,6 +121,136 @@ internal sealed class HistorianWcfTagWriteOrchestrator
return result;
}
private bool AddTagExtendedProperties(string tagName, IReadOnlyList<HistorianTagExtendedProperty> properties)
{
Guid contextKey = Guid.NewGuid();
var (histBinding, histEndpoint, _, _) = HistorianWcfBindingFactory.CreateBindingPair(_options);
Binding auxBinding = HistorianWcfBindingFactory.CreateAuxiliaryBinding(_options);
EndpointAddress statusEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(_options, HistorianWcfServiceNames.Status);
EndpointAddress transactionEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(_options, HistorianWcfServiceNames.Transaction);
EndpointAddress retrievalEndpoint = _options.Transport == HistorianTransport.LocalPipe
? HistorianWcfBindingFactory.CreatePipeEndpointAddress(_options.Host, HistorianWcfServiceNames.Retrieval)
: HistorianWcfBindingFactory.CreateEndpointAddress(_options.Host, _options.Port, HistorianWcfServiceNames.Retrieval);
bool result = false;
HistorianWcfAuthChainHelper.OpenAuthenticatedConnection(
_options, histBinding, histEndpoint, contextKey, CancellationToken.None,
connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode,
additionalSetup: (historyChannel, context) =>
{
RunWritePriming(historyChannel, context, auxBinding, statusEndpoint, transactionEndpoint, retrievalEndpoint);
string handle = context.StorageSessionId.ToString("D").ToUpperInvariant();
byte[] inBuff = HistorianTagExtendedPropertyProtocol.SerializeAddRequest(tagName, properties);
DumpTepIfRequested(inBuff);
bool ok = historyChannel.AddTagExtendedProperties(handle, inBuff, out byte[] errorBuffer);
WriteDiag("AddTEx", $"Returned={ok} Tag={tagName} PropCount={properties.Count} InLen={inBuff.Length} ErrLen={errorBuffer?.Length ?? -1} ErrHex={(errorBuffer is null ? "<null>" : Convert.ToHexString(errorBuffer))}");
result = ok;
});
return result;
}
private bool DeleteTagExtendedProperties(string tagName, IReadOnlyList<string> propertyNames)
{
Guid contextKey = Guid.NewGuid();
var (histBinding, histEndpoint, _, _) = HistorianWcfBindingFactory.CreateBindingPair(_options);
Binding auxBinding = HistorianWcfBindingFactory.CreateAuxiliaryBinding(_options);
EndpointAddress statusEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(_options, HistorianWcfServiceNames.Status);
EndpointAddress transactionEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(_options, HistorianWcfServiceNames.Transaction);
EndpointAddress retrievalEndpoint = _options.Transport == HistorianTransport.LocalPipe
? HistorianWcfBindingFactory.CreatePipeEndpointAddress(_options.Host, HistorianWcfServiceNames.Retrieval)
: HistorianWcfBindingFactory.CreateEndpointAddress(_options.Host, _options.Port, HistorianWcfServiceNames.Retrieval);
bool result = false;
HistorianWcfAuthChainHelper.OpenAuthenticatedConnection(
_options, histBinding, histEndpoint, contextKey, CancellationToken.None,
connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode,
additionalSetup: (historyChannel, context) =>
{
RunWritePriming(historyChannel, context, auxBinding, statusEndpoint, transactionEndpoint, retrievalEndpoint);
string handle = context.StorageSessionId.ToString("D").ToUpperInvariant();
byte[] inBuff = HistorianTagExtendedPropertyProtocol.SerializeDeleteRequest(tagName, propertyNames);
DumpTepIfRequested(inBuff);
// The server's CHistStorage::DeleteTagExtendedProperties resolves each property from
// the storage session's working set — so the tag identity (GetTgByNm) and its
// extended properties (GetTepByNm) must be fetched on THIS session first, exactly as
// the native client does (register → read → delete). The Retrieval prime channel is
// kept OPEN across the DelTep call (the registration is bound to the live channel
// session); without this prime the server throws SErrorException and DelTep returns
// false. See docs/reverse-engineering/wcf-add-tag-extended-properties.md §Delete.
result = PrimeThenDelete(auxBinding, retrievalEndpoint, handle, context.ClientHandle, tagName, () =>
{
bool ok = historyChannel.DeleteTagExtendedProperties(handle, inBuff, out byte[] errorBuffer);
WriteDiag("DelTep", $"Returned={ok} Tag={tagName} PropCount={propertyNames.Count} InLen={inBuff.Length} ErrLen={errorBuffer?.Length ?? -1} ErrHex={(errorBuffer is null ? "<null>" : Convert.ToHexString(errorBuffer))}");
return ok;
});
});
return result;
}
/// <summary>
/// Primes the Retrieval session before a <c>DelTep</c> delete exactly as the native client does:
/// register the tag identity (<c>GetTgByNm</c>) then fetch its extended properties
/// (<c>GetTepByNm</c>) so the server loads both into the storage session's working set, then runs
/// <paramref name="deleteAction"/> (the DelTep call) WHILE the Retrieval channel is still open.
/// Without this two-step prime — or if the channel is closed before DelTep — the server's
/// <c>CHistStorage::DeleteTagExtendedProperties</c> can't resolve the property and throws
/// <c>SErrorException</c> (DelTep returns false). The fetched rows are discarded — the calls are
/// for their server-side registration effect. See
/// docs/reverse-engineering/wcf-add-tag-extended-properties.md §Delete.
/// </summary>
private bool PrimeThenDelete(
Binding retrievalBinding, EndpointAddress retrievalEndpoint, string handle, uint clientHandle,
string tagName, Func<bool> deleteAction)
{
ChannelFactory<IRetrievalServiceContract4> factory = new(retrievalBinding, retrievalEndpoint);
HistorianWcfClientCredentialsHelper.Configure(factory, _options);
IRetrievalServiceContract4 channel = factory.CreateChannel();
try
{
TryRun(() => channel.GetInterfaceVersion(out _));
byte[] tagNames = HistorianTagExtendedPropertyProtocol.SerializeRequest(tagName);
// Register the tag identity on this session (GetTgByNm, uint clientHandle). tagNames uses
// the same count+charCount+UTF-16 framing as GetTepByNm's request buffer.
try
{
uint tgSequence = 0;
uint tgRet = channel.GetTagInfosFromName(clientHandle, (uint)tagNames.Length, tagNames, ref tgSequence, out uint tgInfosSize, out byte[] tgInfos);
WriteDiag("DelTepPrime", $"GetTgByNm ret={tgRet} clientHandle={clientHandle} seq={tgSequence} infosSize={tgInfosSize} infosLen={tgInfos?.Length ?? -1}");
}
catch (Exception ex) { WriteDiag("DelTepPrime", $"GetTgByNm threw: {ex.GetType().Name}: {ex.Message}"); }
uint sequence = 0;
for (int page = 0; page < 64; page++)
{
bool ok;
byte[] responseBuffer;
try { ok = channel.GetTagExtendedPropertiesFromName(handle, tagNames, ref sequence, out responseBuffer, out byte[] errBuf); }
catch (Exception ex) { WriteDiag("DelTepPrime", $"GetTepByNm threw page={page}: {ex.GetType().Name}: {ex.Message}"); break; }
int rows = HistorianTagExtendedPropertyProtocol.ParseResponse(responseBuffer ?? []).Count;
WriteDiag("DelTepPrime", $"GetTepByNm page={page} ok={ok} respLen={responseBuffer?.Length ?? -1} rows={rows}");
if (!ok) break;
if (rows == 0) break;
}
// DelTep runs here, while the Retrieval prime channel is still open.
return deleteAction();
}
finally
{
CloseSafely(channel, factory);
}
}
/// <summary>Env-gated dump of the clean AddTEx / DelTep inBuff (base64) for golden-fixture
/// capture, mirroring the rename/SQL dump hooks — avoids hand-stitching MDAS chunk markers.</summary>
private static void DumpTepIfRequested(byte[] inBuff)
{
string? path = Environment.GetEnvironmentVariable("AVEVA_HISTORIAN_TEP_DUMP");
if (string.IsNullOrWhiteSpace(path)) return;
try { File.AppendAllText(path, Convert.ToBase64String(inBuff) + Environment.NewLine); } catch { }
}
private static bool SendEnsureTags2(
IHistoryServiceContract2 historyChannel,
HistorianWcfAuthChainHelper.OpenConnectionContext context,
@@ -252,4 +417,87 @@ internal sealed class HistorianWcfTagWriteOrchestrator
try { if (channel is ICommunicationObject co) { if (co.State == CommunicationState.Faulted) co.Abort(); else co.Close(); } } catch { }
try { if (factory.State == CommunicationState.Faulted) factory.Abort(); else factory.Close(); } catch { }
}
public Task<HistorianTagRenameResult> RenameTagsAsync(
IReadOnlyList<(string OldName, string NewName)> pairs, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(pairs);
if (pairs.Count == 0)
{
throw new ArgumentException("At least one (old,new) name pair is required.", nameof(pairs));
}
foreach ((string oldName, string newName) in pairs)
{
ArgumentException.ThrowIfNullOrWhiteSpace(oldName, nameof(pairs));
ArgumentException.ThrowIfNullOrWhiteSpace(newName, nameof(pairs));
}
return Task.Run(() => RenameTags(pairs), cancellationToken);
}
private HistorianTagRenameResult RenameTags(IReadOnlyList<(string OldName, string NewName)> pairs)
{
Guid contextKey = Guid.NewGuid();
var (histBinding, histEndpoint, _, _) = HistorianWcfBindingFactory.CreateBindingPair(_options);
Binding auxBinding = HistorianWcfBindingFactory.CreateAuxiliaryBinding(_options);
EndpointAddress statusEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(_options, HistorianWcfServiceNames.Status);
EndpointAddress transactionEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(_options, HistorianWcfServiceNames.Transaction);
EndpointAddress retrievalEndpoint = _options.Transport == HistorianTransport.LocalPipe
? HistorianWcfBindingFactory.CreatePipeEndpointAddress(_options.Host, HistorianWcfServiceNames.Retrieval)
: HistorianWcfBindingFactory.CreateEndpointAddress(_options.Host, _options.Port, HistorianWcfServiceNames.Retrieval);
HistorianTagRenameResult result = new() { Accepted = false, PairCount = pairs.Count };
HistorianWcfAuthChainHelper.OpenAuthenticatedConnection(
_options, histBinding, histEndpoint, contextKey, CancellationToken.None,
connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode,
additionalSetup: (historyChannel, context) =>
{
RunWritePriming(historyChannel, context, auxBinding, statusEndpoint, transactionEndpoint, retrievalEndpoint);
result = SendStartJobRename(historyChannel, context, pairs);
});
return result;
}
/// <summary>
/// Submits the rename batch via the History <c>StartJob</c> (StJb) op with the uppercase
/// storage-session GUID handle. The server queues a job and returns its id; the renames apply
/// asynchronously. Requires the server gate <c>AllowRenameTags</c> to be enabled (otherwise the
/// native path would reject with err 132 before the wire — here the managed client has no such
/// pre-check, so a disabled gate surfaces as StartJob returning false).
/// </summary>
private static HistorianTagRenameResult SendStartJobRename(
IHistoryServiceContract2 historyChannel,
HistorianWcfAuthChainHelper.OpenConnectionContext context,
IReadOnlyList<(string OldName, string NewName)> pairs)
{
string handle = context.StorageSessionId.ToString("D").ToUpperInvariant();
byte[] jobBuffer = HistorianTagRenameProtocol.SerializeRenameJob(pairs);
DumpRenameJobIfRequested(jobBuffer);
bool ok = historyChannel.StartJob(handle, jobBuffer, out string jobId, out byte[] errorBuffer);
WriteDiag("StJb", $"Returned={ok} JobId={jobId} JobBufferLen={jobBuffer.Length} ErrLen={errorBuffer?.Length ?? -1} ErrHex={(errorBuffer is null ? "<null>" : Convert.ToHexString(errorBuffer))}");
Guid parsedJobId = Guid.Empty;
if (!string.IsNullOrWhiteSpace(jobId))
{
Guid.TryParse(jobId.Trim().Trim('$', '{', '}'), out parsedJobId);
}
return new HistorianTagRenameResult
{
Accepted = ok,
JobId = parsedJobId,
PairCount = pairs.Count,
Error = ok ? null : "Server rejected the rename job (StartJob returned false). Check that the 'AllowRenameTags' system parameter is enabled.",
};
}
/// <summary>Env-gated dump of the clean rename job buffer (base64) for golden-fixture capture,
/// mirroring the <c>AVEVA_HISTORIAN_SQL_DUMP</c> hook — avoids hand-stitching MDAS chunk markers
/// out of a raw instrument capture.</summary>
private static void DumpRenameJobIfRequested(byte[] jobBuffer)
{
string? path = Environment.GetEnvironmentVariable("AVEVA_HISTORIAN_RENAME_DUMP");
if (string.IsNullOrWhiteSpace(path)) return;
try { File.AppendAllText(path, Convert.ToBase64String(jobBuffer) + Environment.NewLine); } catch { }
}
}
@@ -36,7 +36,7 @@ public sealed class EventChainDiagnosticTests
int observed = 0;
AVEVA.Historian.Client.Models.HistorianEvent? firstEvent = null;
await foreach (var evt in orchestrator.ReadEventsAsync(startUtc, endUtc, CancellationToken.None))
await foreach (var evt in orchestrator.ReadEventsAsync(startUtc, endUtc, filter: null, CancellationToken.None))
{
observed++;
firstEvent ??= evt;
@@ -273,6 +273,123 @@ public sealed class HistorianClientIntegrationTests
Assert.False(string.IsNullOrWhiteSpace(value));
}
[Fact]
public async Task GetRuntimeParameterAsync_AgainstLocalHistorian_ReturnsHistorianVersion()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_HOST");
if (string.IsNullOrWhiteSpace(host) || !string.Equals(host, "localhost", StringComparison.OrdinalIgnoreCase) || !OperatingSystem.IsWindows())
{
return;
}
HistorianClient client = new(new HistorianClientOptions
{
Host = host,
IntegratedSecurity = true,
Transport = HistorianTransport.LocalPipe
});
// GETRP rides the storage-session GUID as an uppercase string handle. HistorianVersion is
// a known-good runtime parameter returning the server version (e.g. "20,0,000,000").
string? value = await client.GetRuntimeParameterAsync("HistorianVersion", CancellationToken.None);
Assert.False(string.IsNullOrWhiteSpace(value));
}
[Fact]
public async Task GetTagExtendedPropertiesAsync_AgainstLocalHistorian_ReturnsProperties()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_HOST");
// A tag that carries at least one extended property. Gated on its own env var so the test
// skips cleanly when no such tag is configured (no tag name is hardcoded).
string? tepTag = Environment.GetEnvironmentVariable("HISTORIAN_TEP_TAG");
if (string.IsNullOrWhiteSpace(host)
|| !string.Equals(host, "localhost", StringComparison.OrdinalIgnoreCase)
|| !OperatingSystem.IsWindows()
|| string.IsNullOrWhiteSpace(tepTag))
{
return;
}
HistorianClient client = new(new HistorianClientOptions
{
Host = host,
IntegratedSecurity = true,
Transport = HistorianTransport.LocalPipe
});
// GetTepByNm rides the storage-session GUID as an uppercase string handle. The configured
// tag has at least one string-valued extended property (e.g. Location).
IReadOnlyList<AVEVA.Historian.Client.Models.HistorianTagExtendedProperty> properties =
await client.GetTagExtendedPropertiesAsync(tepTag, CancellationToken.None);
Assert.NotEmpty(properties);
Assert.All(properties, p =>
{
Assert.False(string.IsNullOrWhiteSpace(p.Name));
Assert.NotNull(p.Value);
});
}
[Fact]
public async Task ExecuteSqlCommandAsync_AgainstLocalHistorian_ReturnsRecordSet()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_HOST");
if (string.IsNullOrWhiteSpace(host) || !string.Equals(host, "localhost", StringComparison.OrdinalIgnoreCase) || !OperatingSystem.IsWindows())
{
return;
}
HistorianClient client = new(new HistorianClientOptions
{
Host = host,
IntegratedSecurity = true,
Transport = HistorianTransport.LocalPipe
});
// ExeC/GetR ride the storage-session GUID as an uppercase string handle. A constant SELECT
// returns a single int column; the DataTable is decoded from the NRBF stream (no BinaryFormatter).
AVEVA.Historian.Client.Models.HistorianSqlResult result =
await client.ExecuteSqlCommandAsync("SELECT 1 AS ProbeValue", cancellationToken: CancellationToken.None);
AVEVA.Historian.Client.Models.HistorianSqlColumn column = Assert.Single(result.Columns);
Assert.Equal("ProbeValue", column.Name);
IReadOnlyList<object?> row = Assert.Single(result.Rows);
Assert.Equal(1, Assert.Single(row));
}
[Fact]
public async Task ExecuteSqlCommandAsync_AgainstLocalHistorian_MultiColumnMultiRow()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_HOST");
if (string.IsNullOrWhiteSpace(host) || !string.Equals(host, "localhost", StringComparison.OrdinalIgnoreCase) || !OperatingSystem.IsWindows())
{
return;
}
HistorianClient client = new(new HistorianClientOptions
{
Host = host,
IntegratedSecurity = true,
Transport = HistorianTransport.LocalPipe
});
// Fully synthetic query (no server data): two columns (int + string), two rows, one NULL —
// exercises the schema/diffgram parser beyond the single-cell case.
AVEVA.Historian.Client.Models.HistorianSqlResult result = await client.ExecuteSqlCommandAsync(
"SELECT 10 AS Num, 'alpha' AS Word UNION ALL SELECT 20, NULL",
cancellationToken: CancellationToken.None);
Assert.Equal(2, result.Columns.Count);
Assert.Equal("Num", result.Columns[0].Name);
Assert.Equal("Word", result.Columns[1].Name);
Assert.Equal(2, result.Rows.Count);
Assert.Equal(10, result.Rows[0][0]);
Assert.Equal("alpha", result.Rows[0][1]);
Assert.Equal(20, result.Rows[1][0]);
Assert.Null(result.Rows[1][1]);
}
[Fact]
public async Task GetConnectionStatusAsync_AgainstLocalHistorian_ReportsConnectedToServer()
{
@@ -447,6 +564,72 @@ public sealed class HistorianClientIntegrationTests
Assert.True(deleted, "DeleteTagAsync returned false against the live Historian.");
}
[Fact]
public async Task AddTagExtendedPropertiesAsync_AgainstLocalHistorian_WritesAndReadsBack()
{
// Safety: localhost only, sandbox tag must start with "RetestSdkWrite". Creates the tag,
// adds an extended property, reads it back via R1.5, and deletes the tag. Gated on
// HISTORIAN_WRITE_SANDBOX_TAG.
string? host = Environment.GetEnvironmentVariable("HISTORIAN_HOST");
string? sandboxTag = Environment.GetEnvironmentVariable("HISTORIAN_WRITE_SANDBOX_TAG");
if (string.IsNullOrWhiteSpace(host) || !string.Equals(host, "localhost", StringComparison.OrdinalIgnoreCase) || !OperatingSystem.IsWindows())
{
return;
}
if (string.IsNullOrWhiteSpace(sandboxTag) || !sandboxTag.StartsWith("RetestSdkWrite", StringComparison.Ordinal))
{
return; // safety gate
}
HistorianClient client = new(new HistorianClientOptions
{
Host = host,
IntegratedSecurity = true,
Transport = HistorianTransport.LocalPipe
});
await client.EnsureTagAsync(new AVEVA.Historian.Client.Models.HistorianTagDefinition
{
TagName = sandboxTag,
Description = "SDK ext-property write live test",
EngineeringUnit = "test",
DataType = AVEVA.Historian.Client.Models.HistorianDataType.Float,
MinEU = 0.0,
MaxEU = 100.0,
}, CancellationToken.None);
try
{
const string propName = "SdkLiveTestProp";
const string propValue = "SdkLiveTestValue";
bool added = await client.AddTagExtendedPropertyAsync(sandboxTag, propName, propValue, CancellationToken.None);
Assert.True(added, "AddTagExtendedPropertyAsync returned false against the live Historian.");
// Read back via R1.5 (server may take a moment to surface the new property).
bool found = false;
for (int i = 0; i < 10 && !found; i++)
{
await Task.Delay(500);
var props = await client.GetTagExtendedPropertiesAsync(sandboxTag, CancellationToken.None);
found = props.Any(p =>
string.Equals(p.Name, propName, StringComparison.OrdinalIgnoreCase) &&
string.Equals(p.Value, propValue, StringComparison.Ordinal));
}
Assert.True(found, $"Extended property '{propName}={propValue}' was not read back after the write.");
}
finally
{
try { await client.DeleteTagAsync(sandboxTag, CancellationToken.None); } catch { }
}
}
// Extended-property DELETE (DelTep) has no live integration test: the wire format is captured and
// the serializer is golden-verified (WcfTagExtendedPropertyWriteProtocolTests), but the SDK
// cannot yet make the server accept the delete (the native single-connection working-set
// requirement isn't reproduced by per-service WCF channels). See
// docs/reverse-engineering/wcf-add-tag-extended-properties.md §Delete.
// Round-trip every live-verified analog data type + the non-default-range case. The
// sandbox tag name is suffixed per case so the runs don't collide. Always cleans up.
[Theory]
@@ -744,4 +927,184 @@ public sealed class HistorianClientIntegrationTests
Assert.True(metadata.MaxRaw is > 0 and <= 1e15);
Assert.False(string.IsNullOrWhiteSpace(metadata.EngineeringUnit));
}
[Fact]
public async Task ReadEventsAsync_WithFilter_IsHonoredByServer()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_HOST");
if (string.IsNullOrWhiteSpace(host) || !string.Equals(host, "localhost", StringComparison.OrdinalIgnoreCase) || !OperatingSystem.IsWindows())
{
return;
}
HistorianClient client = new(new HistorianClientOptions
{
Host = host,
IntegratedSecurity = true,
Transport = HistorianTransport.LocalPipe
});
DateTime endUtc = DateTime.UtcNow;
DateTime startUtc = endUtc - TimeSpan.FromDays(30);
// A predicate that matches nothing must return zero events — proving the server applies
// the filter (not inert), unlike e.g. the analog-summary knobs.
List<AVEVA.Historian.Client.Models.HistorianEvent> noMatch = [];
await foreach (var evt in client.ReadEventsAsync(startUtc, endUtc,
new AVEVA.Historian.Client.Models.HistorianEventFilter("Type",
AVEVA.Historian.Client.Models.HistorianEventComparison.Equal, "ZZZ_NoSuchEventType"),
CancellationToken.None))
{
noMatch.Add(evt);
}
Assert.Empty(noMatch);
// A matching predicate returns events, all of the filtered Type.
List<AVEVA.Historian.Client.Models.HistorianEvent> matched = [];
await foreach (var evt in client.ReadEventsAsync(startUtc, endUtc,
new AVEVA.Historian.Client.Models.HistorianEventFilter("Type",
AVEVA.Historian.Client.Models.HistorianEventComparison.Equal, "User.Write"),
CancellationToken.None))
{
matched.Add(evt);
}
// Requires User.Write events in the window (present on a working Historian). If the store
// is empty in the window this asserts nothing was wrongly returned; otherwise every row
// must match the filtered type.
Assert.All(matched, evt => Assert.Equal("User.Write", evt.Type));
}
[Fact]
public async Task SendEventAsync_AgainstLocalHistorian_AcceptedByServer()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_HOST");
if (string.IsNullOrWhiteSpace(host) || !string.Equals(host, "localhost", StringComparison.OrdinalIgnoreCase) || !OperatingSystem.IsWindows())
{
return;
}
HistorianClient client = new(new HistorianClientOptions
{
Host = host,
IntegratedSecurity = true,
Transport = HistorianTransport.LocalPipe
});
Guid eventId = Guid.NewGuid();
DateTime eventTime = DateTime.UtcNow;
AVEVA.Historian.Client.Models.HistorianEvent evt = new(
Id: eventId,
EventTimeUtc: eventTime,
ReceivedTimeUtc: eventTime,
Type: "User.Write",
SourceName: string.Empty,
Namespace: "RetestSdkEventSend",
RevisionVersion: 0,
Properties: new Dictionary<string, object?>
{
["Source"] = "RetestSdkEventSend",
["TestMarker"] = "histsdk-R2.5-roundtrip",
});
// The full managed event-send chain (Open2 event-mode 0x501 → CM_EVENT RTag2/EnsT2 →
// AddS2) reaches the server and the server accepts the AddS2 delivery. NOTE: whether the
// event is then persisted to the queryable store depends on the historian's event
// ingestion pipeline being active — on this dev box new events are accepted but not
// persisted (the native client behaves identically), so this asserts acceptance, which
// is the SDK-level signal. Round-trip read-back is best-effort below.
bool accepted = await client.SendEventAsync(evt, CancellationToken.None);
Assert.True(accepted);
// Best-effort round-trip: if the event store persisted it, it should be readable in a
// tight time window. Not asserted hard because event persistence is environment-gated.
try
{
using Microsoft.Data.SqlClient.SqlConnection sql = new("Server=.;Database=Runtime;Integrated Security=SSPI;Encrypt=False;TrustServerCertificate=True");
await sql.OpenAsync();
using Microsoft.Data.SqlClient.SqlCommand cmd = sql.CreateCommand();
cmd.CommandText =
"SELECT COUNT(*) FROM v_AlarmEventHistory2 " +
"WHERE EventStampUTC BETWEEN @s AND @e AND Type = @t";
cmd.Parameters.AddWithValue("@s", eventTime.AddMinutes(-2));
cmd.Parameters.AddWithValue("@e", eventTime.AddMinutes(2));
cmd.Parameters.AddWithValue("@t", "User.Write");
int count = Convert.ToInt32(await cmd.ExecuteScalarAsync());
// If persistence is active the event is present; if not, count is 0 (env limitation).
Assert.True(count >= 0);
}
catch
{
// SQL read-back is diagnostic only; never fail the send test on a query issue.
}
}
[Fact]
public async Task RenameTagsAsync_AgainstLocalHistorian_RenamesSandboxTag()
{
// Safety: localhost only, names must start with "RetestSdkWrite". Requires the server's
// AllowRenameTags system parameter to be enabled (otherwise StartJob returns false). Gated
// on HISTORIAN_RENAME_SANDBOX so it stays skipped unless explicitly enabled.
string? host = Environment.GetEnvironmentVariable("HISTORIAN_HOST");
string? sandbox = Environment.GetEnvironmentVariable("HISTORIAN_RENAME_SANDBOX");
if (string.IsNullOrWhiteSpace(host) || !string.Equals(host, "localhost", StringComparison.OrdinalIgnoreCase) || !OperatingSystem.IsWindows())
{
return;
}
if (string.IsNullOrWhiteSpace(sandbox) || !sandbox.StartsWith("RetestSdkWrite", StringComparison.Ordinal))
{
return; // safety gate
}
string src = sandbox + "Src";
string dst = sandbox + "Dst";
HistorianClient client = new(new HistorianClientOptions
{
Host = host,
IntegratedSecurity = true,
Transport = HistorianTransport.LocalPipe
});
// Fresh source tag.
await client.EnsureTagAsync(new AVEVA.Historian.Client.Models.HistorianTagDefinition
{
TagName = src,
Description = "SDK rename live test",
EngineeringUnit = "test",
DataType = AVEVA.Historian.Client.Models.HistorianDataType.Float,
MinEU = 0.0,
MaxEU = 100.0,
}, CancellationToken.None);
try
{
AVEVA.Historian.Client.Models.HistorianTagRenameResult result =
await client.RenameTagAsync(src, dst, CancellationToken.None);
Assert.True(result.Accepted, "RenameTagsAsync was not accepted by the server (is AllowRenameTags enabled?).");
Assert.NotEqual(Guid.Empty, result.JobId);
Assert.Equal(1, result.PairCount);
// Rename completes asynchronously; poll the new name's metadata briefly.
bool renamed = false;
for (int i = 0; i < 10 && !renamed; i++)
{
await Task.Delay(500);
try
{
var md = await client.GetTagMetadataAsync(dst, CancellationToken.None);
renamed = md is not null && string.Equals(md.Name, dst, StringComparison.OrdinalIgnoreCase);
}
catch { /* not yet visible */ }
}
Assert.True(renamed, $"Renamed tag '{dst}' did not become visible after the job completed.");
}
finally
{
// Clean up whichever name ended up in the DB.
try { await client.DeleteTagAsync(dst, CancellationToken.None); } catch { }
try { await client.DeleteTagAsync(src, CancellationToken.None); } catch { }
}
}
}
@@ -0,0 +1,155 @@
using System.Reflection;
using System.Reflection.Emit;
namespace AVEVA.Historian.Client.Tests;
/// <summary>
/// Structural guardrail pinning the 2023 R2 gRPC auth-handshake op routing. The SSPI/Negotiate
/// token loop MUST be carried by <c>StorageService.ValidateClientCredential</c> (the op that kept
/// the 2020 inBuff/outBuff token framing), NOT by <c>HistoryService.ExchangeKey</c> — ExchangeKey
/// is a separate key-exchange/cert-path op that rejects an NTLM token at round 0 regardless of
/// credentials (live-confirmed against a real 2023 R2 server, 2026-06-21). An earlier revision
/// routed the loop to ExchangeKey; this test fails if that regression returns.
///
/// It works by disassembling the IL of <c>HistorianGrpcHandshake</c> (and its
/// compiler-generated nested closure types — the token-loop call lives inside a lambda) and
/// collecting every method invoked.
/// </summary>
public sealed class HistorianGrpcHandshakeRoutingTests
{
[Fact]
public void Handshake_UsesValidateClientCredential_NotExchangeKey()
{
// The auth token loop lives in the shared handshake helper (reused by the read, status,
// and future browse/metadata gRPC paths).
HashSet<string> calledMethods = CollectCalledMethodNames(
"AVEVA.Historian.Client.Grpc.HistorianGrpcHandshake");
Assert.Contains("ValidateClientCredential", calledMethods);
Assert.DoesNotContain("ExchangeKey", calledMethods);
}
private static HashSet<string> CollectCalledMethodNames(string typeFullName)
{
Assembly sdk = typeof(HistorianClientOptions).Assembly;
Type orchestrator = sdk.GetType(typeFullName, throwOnError: true)!;
Module module = orchestrator.Module;
// The orchestrator type plus its compiler-generated nested types (lambda closures).
IEnumerable<Type> types = new[] { orchestrator }
.Concat(orchestrator.GetNestedTypes(BindingFlags.Public | BindingFlags.NonPublic));
var names = new HashSet<string>(StringComparer.Ordinal);
foreach (Type t in types)
{
Type[] typeArgs = t.IsGenericType ? t.GetGenericArguments() : Type.EmptyTypes;
foreach (MethodInfo m in t.GetMethods(
BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Static | BindingFlags.DeclaredOnly))
{
byte[]? il = m.GetMethodBody()?.GetILAsByteArray();
if (il is null)
{
continue;
}
Type[] methodArgs = m.IsGenericMethodDefinition ? m.GetGenericArguments() : Type.EmptyTypes;
foreach (int token in EnumerateMethodTokens(il))
{
try
{
MethodBase? resolved = module.ResolveMethod(token, typeArgs, methodArgs);
if (resolved is not null)
{
names.Add(resolved.Name);
}
}
catch (Exception ex) when (ex is ArgumentException or BadImageFormatException)
{
// vararg / MethodSpec tokens that don't resolve cleanly — irrelevant here.
}
}
}
}
return names;
}
/// <summary>
/// Walks an IL byte stream, yielding the 4-byte metadata token of every call/callvirt/newobj
/// (any <see cref="OperandType.InlineMethod"/> opcode). Uses the reflection-emit opcode table so
/// operands of other instructions are skipped correctly rather than misread as opcodes.
/// </summary>
private static IEnumerable<int> EnumerateMethodTokens(byte[] il)
{
int pos = 0;
while (pos < il.Length)
{
OpCode op;
if (il[pos] == 0xFE && pos + 1 < il.Length)
{
op = TwoByteOpCodes[il[pos + 1]];
pos += 2;
}
else
{
op = OneByteOpCodes[il[pos]];
pos += 1;
}
switch (op.OperandType)
{
case OperandType.InlineMethod:
yield return BitConverter.ToInt32(il, pos);
pos += 4;
break;
case OperandType.InlineNone:
break;
case OperandType.ShortInlineBrTarget:
case OperandType.ShortInlineI:
case OperandType.ShortInlineVar:
pos += 1;
break;
case OperandType.InlineVar:
pos += 2;
break;
case OperandType.InlineI8:
case OperandType.InlineR:
pos += 8;
break;
case OperandType.InlineSwitch:
int count = BitConverter.ToInt32(il, pos);
pos += 4 + (4 * count);
break;
default:
// InlineBrTarget, InlineField, InlineI, InlineSig, InlineString, InlineTok,
// InlineType, ShortInlineR — all 4-byte operands.
pos += 4;
break;
}
}
}
private static readonly OpCode[] OneByteOpCodes = BuildOpCodeTable(twoByte: false);
private static readonly OpCode[] TwoByteOpCodes = BuildOpCodeTable(twoByte: true);
private static OpCode[] BuildOpCodeTable(bool twoByte)
{
var table = new OpCode[256];
foreach (FieldInfo f in typeof(OpCodes).GetFields(BindingFlags.Public | BindingFlags.Static))
{
if (f.GetValue(null) is not OpCode op)
{
continue;
}
ushort value = unchecked((ushort)op.Value);
bool isTwoByte = (value & 0xFF00) == 0xFE00;
if (isTwoByte == twoByte)
{
table[value & 0xFF] = op;
}
}
return table;
}
}
@@ -12,6 +12,21 @@ namespace AVEVA.Historian.Client.Tests;
/// </summary>
public sealed class HistorianGrpcIntegrationTests
{
[Fact]
public async Task ProbeAsync_OverGrpc_ReturnsTrue()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST");
if (string.IsNullOrWhiteSpace(host))
{
return;
}
// ProbeAsync calls the unauthenticated GetInterfaceVersion RPCs, so it succeeds even when
// credentials are unavailable — no HISTORIAN_USER/PASSWORD required.
HistorianClient client = new(BuildOptions(host));
Assert.True(await client.ProbeAsync(CancellationToken.None));
}
[Fact]
public async Task ReadRawAsync_OverGrpc_ReturnsAtLeastOneRow()
{
@@ -37,6 +52,61 @@ public sealed class HistorianGrpcIntegrationTests
Assert.All(samples, s => Assert.Equal(testTag, s.TagName));
}
[Fact]
public async Task GetSystemParameterAsync_OverGrpc_ReturnsValue()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST");
if (string.IsNullOrWhiteSpace(host) || string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER")))
{
return;
}
HistorianClient client = new(BuildOptions(host));
string? version = await client.GetSystemParameterAsync("HistorianVersion", CancellationToken.None);
Assert.False(string.IsNullOrWhiteSpace(version));
}
[Fact]
public async Task GetTagMetadataAsync_OverGrpc_ReturnsRequestedTag()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST");
string? tag = Environment.GetEnvironmentVariable("HISTORIAN_TEST_TAG");
if (string.IsNullOrWhiteSpace(host) || string.IsNullOrWhiteSpace(tag)
|| string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER")))
{
return;
}
HistorianClient client = new(BuildOptions(host));
HistorianTagMetadata? metadata = await client.GetTagMetadataAsync(tag, CancellationToken.None);
Assert.NotNull(metadata);
Assert.Equal(tag, metadata!.Name);
// A real metadata record decodes to a known data type (descriptor passed MapDataType).
Assert.True(Enum.IsDefined(metadata.DataType));
}
[Fact]
public async Task BrowseTagNamesAsync_OverGrpc_ReturnsSystemTags()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST");
if (string.IsNullOrWhiteSpace(host) || string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER")))
{
return;
}
// Full R0.1 browse over gRPC: StartTagQuery(OData) -> paged QueryTag(0x6752) -> EndTagQuery.
HistorianClient client = new(BuildOptions(host));
List<string> names = [];
await foreach (string name in client.BrowseTagNamesAsync("Sys*", CancellationToken.None))
{
names.Add(name);
}
Assert.NotEmpty(names);
Assert.All(names, n => Assert.StartsWith("Sys", n, StringComparison.Ordinal));
}
private static HistorianClientOptions BuildOptions(string host)
{
string? user = Environment.GetEnvironmentVariable("HISTORIAN_USER");
@@ -100,6 +100,88 @@ public sealed class HistorianGrpcTransportTests
Assert.Equal((uint)HistorianDataQueryProtocol.QueryRequestTypeData, decoded.UiQueryRequestType);
}
[Fact]
public void InterfaceVersionResponses_ExposeErrorAndVersion_AsProbeExpects()
{
// R0.4 ProbeAsync reads uiError/uiVersion off each service's GetInterfaceVersion response.
// Pin that field mapping (success = uiError 0 + uiVersion > 0) via a protobuf round-trip.
var history = GrpcHistory.GetInterfaceVersionResponse.Parser.ParseFrom(
new GrpcHistory.GetInterfaceVersionResponse { UiError = 0, UiVersion = 12 }.ToByteArray());
var retrieval = GetRetrievalInterfaceVersionResponse.Parser.ParseFrom(
new GetRetrievalInterfaceVersionResponse { UiError = 0, UiVersion = 4 }.ToByteArray());
Assert.Equal(0u, history.UiError);
Assert.Equal(12u, history.UiVersion);
Assert.Equal(0u, retrieval.UiError);
Assert.Equal(4u, retrieval.UiVersion);
}
[Fact]
public void GetSystemParameterMessages_CarryHandleNameAndValue_AsStatusClientExpects()
{
// R0.3 sends {uiHandle, strParameterName} and reads strParameterValue when status succeeds.
var request = ArchestrA.Grpc.Contract.Status.GetSystemParameterRequest.Parser.ParseFrom(
new ArchestrA.Grpc.Contract.Status.GetSystemParameterRequest { UiHandle = 9, StrParameterName = "HistorianVersion" }.ToByteArray());
Assert.Equal(9u, request.UiHandle);
Assert.Equal("HistorianVersion", request.StrParameterName);
var response = ArchestrA.Grpc.Contract.Status.GetSystemParameterResponse.Parser.ParseFrom(
new ArchestrA.Grpc.Contract.Status.GetSystemParameterResponse
{
Status = new ArchestrA.Grpc.Contract.RequestStatus.Status { BSuccess = true },
StrParameterValue = "20.0.000"
}.ToByteArray());
Assert.True(response.Status.BSuccess);
Assert.Equal("20.0.000", response.StrParameterValue);
}
[Fact]
public void BuildTagNamesBuffer_EncodesCountThenLengthPrefixedUtf16Names()
{
// R0.2 request framing: uint count + per-name(uint charCount + UTF-16LE). Golden bytes.
byte[] buffer = AVEVA.Historian.Client.Grpc.HistorianGrpcTagClient.BuildTagNamesBuffer(["AB", "C"]);
byte[] expected =
[
0x02, 0x00, 0x00, 0x00, // count = 2
0x02, 0x00, 0x00, 0x00, // "AB" char count = 2
0x41, 0x00, 0x42, 0x00, // 'A','B' UTF-16LE
0x01, 0x00, 0x00, 0x00, // "C" char count = 1
0x43, 0x00 // 'C' UTF-16LE
];
Assert.Equal(expected, buffer);
}
[Fact]
public void BuildQueryTagRequest_EncodesMarkerVersionTypeStartCount()
{
// R0.1 QueryTag paging request: u16 0x6752 + u16 1 + u16 queryType + u32 startIndex + u32 count.
byte[] buffer = AVEVA.Historian.Client.Grpc.HistorianGrpcTagClient.BuildQueryTagRequest(1, 0, 50);
byte[] expected =
[
0x52, 0x67, // marker 0x6752
0x01, 0x00, // version 1
0x01, 0x00, // queryType 1 (names)
0x00, 0x00, 0x00, 0x00, // startIndex 0
0x32, 0x00, 0x00, 0x00 // count 50
];
Assert.Equal(expected, buffer);
}
[Theory]
[InlineData("*", "")]
[InlineData("", "")]
[InlineData("Sys*", "startswith(TagName,'Sys')")]
[InlineData("*Total", "endswith(TagName,'Total')")]
[InlineData("*Alarm*", "contains(TagName,'Alarm')")]
[InlineData("Exact.Tag", "TagName eq 'Exact.Tag'")]
[InlineData("Pre*Suf", "startswith(TagName,'Pre') and endswith(TagName,'Suf')")]
[InlineData("O'Brien*", "startswith(TagName,'O''Brien')")]
public void GlobToODataFilter_TranslatesWildcards(string glob, string expected)
{
Assert.Equal(expected, AVEVA.Historian.Client.Grpc.HistorianGrpcTagClient.GlobToODataFilter(glob));
}
[Fact]
public void OpenConnectionRequest_CarriesNativeOpen2BufferUnchanged()
{
@@ -29,6 +29,20 @@ public sealed class HistorianServerVersionGateTests
HistorianServerVersionGate.Validate(HistorianServiceInterface.Transaction, HistorianServerVersionGate.TransactionInterfaceVersion, Options());
}
[Fact]
public void Validate_History_AcceptsBoth2020And2023R2Versions()
{
// History 11 (2020 WCF) and 12 (2023 R2 gRPC) are both buffer-compatible — a live gRPC
// read against a real 2023 R2 server (interface version 12) returns rows. Both must pass.
HistorianServerVersionGate.Validate(HistorianServiceInterface.History, 11u, Options());
HistorianServerVersionGate.Validate(HistorianServiceInterface.History, HistorianServerVersionGate.HistoryInterfaceVersionGrpc2023R2, Options());
Assert.Equal(12u, HistorianServerVersionGate.HistoryInterfaceVersionGrpc2023R2);
Assert.Contains(11u, HistorianServerVersionGate.AcceptedVersions(HistorianServiceInterface.History));
Assert.Contains(12u, HistorianServerVersionGate.AcceptedVersions(HistorianServiceInterface.History));
// Retrieval reported 4 on the live 2023 R2 server — matches 2020, so it is NOT widened.
Assert.DoesNotContain(5u, HistorianServerVersionGate.AcceptedVersions(HistorianServiceInterface.Retrieval));
}
[Fact]
public void Validate_MismatchedVersion_ThrowsProtocolEvidenceMissing()
{
@@ -36,7 +50,7 @@ public sealed class HistorianServerVersionGateTests
(HistorianServiceInterface Service, uint Version)[] cases =
[
(HistorianServiceInterface.History, 10u),
(HistorianServiceInterface.History, 12u),
(HistorianServiceInterface.History, 13u),
(HistorianServiceInterface.Retrieval, 3u),
(HistorianServiceInterface.Retrieval, 5u),
(HistorianServiceInterface.Transaction, 1u),
@@ -0,0 +1,269 @@
using System.ServiceModel;
using System.ServiceModel.Channels;
using System.Text;
using System.Runtime.Versioning;
using AVEVA.Historian.Client.Wcf;
using AVEVA.Historian.Client.Wcf.Contracts;
using Xunit.Abstractions;
namespace AVEVA.Historian.Client.Tests;
/// <summary>
/// Diagnostic: retest the "string-handle wall" ops (GETHI / ExeC) using the Open2
/// storage-session GUID formatted UPPERCASE (the format the native client sends, and the
/// one that made GETRP punch through). Not an assertion test — it prints the server's
/// return code / buffer lengths so we can judge whether the wall is a handle-format issue.
/// </summary>
[SupportedOSPlatform("windows")]
public sealed class StringHandleProbeDiagnosticTests
{
private readonly ITestOutputHelper _output;
public StringHandleProbeDiagnosticTests(ITestOutputHelper output)
{
_output = output;
}
private static bool ShouldRun(out string host)
{
host = Environment.GetEnvironmentVariable("HISTORIAN_HOST") ?? string.Empty;
return !string.IsNullOrWhiteSpace(host)
&& string.Equals(host, "localhost", StringComparison.OrdinalIgnoreCase)
&& OperatingSystem.IsWindows();
}
[Fact]
public void GETHI_WithUppercaseStorageGuid_AgainstLocalHistorian()
{
if (!ShouldRun(out string host)) return;
HistorianClientOptions options = new()
{
Host = host,
IntegratedSecurity = true,
Transport = HistorianTransport.LocalPipe
};
// Native GETHI pRequestBuff: 53 67 02 00 (sig 0x6753 + version 2) + uint charCount + UTF-16 name.
const string name = "HistorianVersion";
using MemoryStream ms = new();
using (BinaryWriter w = new(ms, Encoding.Unicode, leaveOpen: true))
{
w.Write(new byte[] { 0x53, 0x67, 0x02, 0x00 });
w.Write((uint)name.Length);
w.Write(Encoding.Unicode.GetBytes(name));
}
byte[] requestBuffer = ms.ToArray();
ProbeOnStatusChannel(options, (channel, handle) =>
{
bool ok = channel.GetHistorianInfo(handle, requestBuffer, out byte[] resp, out byte[] err);
_output.WriteLine($"GETHI returned={ok} respLen={resp?.Length ?? 0} errLen={err?.Length ?? 0}");
if (resp is { Length: > 0 })
{
_output.WriteLine(" resp[0..64]=" + Convert.ToHexString(resp.AsSpan(0, Math.Min(64, resp.Length))));
}
});
}
/// <summary>
/// R1.4 probe: GETHI on 2020 WCF is a named-value query (capture showed the native client
/// only ever asks for "HistorianVersion"; the 518-byte struct + EventStorageMode@514 is the
/// 2023R2 HCAL-native/gRPC model). This probes whether the 2020 server exposes any
/// storage-mode / extended info value through GETHI under other parameter names, so we can
/// decide honestly what GetHistorianInfoAsync can return over WCF.
/// </summary>
[Fact]
public void GETHI_CandidateInfoNames_AgainstLocalHistorian()
{
if (!ShouldRun(out string host)) return;
HistorianClientOptions options = new()
{
Host = host,
IntegratedSecurity = true,
Transport = HistorianTransport.LocalPipe
};
string[] candidates =
{
"HistorianVersion", "EventStorageMode", "EventStorageType", "StorageType",
"HistorianEventStorageMode", "EventStorage", "StorageMode", "HistorianInfo",
};
Dictionary<string, (bool ok, int respLen)> results = new();
ProbeOnStatusChannel(options, (channel, handle) =>
{
foreach (string name in candidates)
{
using MemoryStream ms = new();
using (BinaryWriter w = new(ms, Encoding.Unicode, leaveOpen: true))
{
w.Write(new byte[] { 0x53, 0x67, 0x02, 0x00 });
w.Write((uint)name.Length);
w.Write(Encoding.Unicode.GetBytes(name));
}
byte[] req = ms.ToArray();
byte[]? resp = null;
byte[]? err = null;
bool ok;
try
{
ok = channel.GetHistorianInfo(handle, req, out resp, out err);
}
catch (Exception ex)
{
_output.WriteLine($" {name,-26} EXCEPTION {ex.GetType().Name}: {ex.Message}");
results[name] = (false, -1);
continue;
}
string respHex = resp is { Length: > 0 }
? Convert.ToHexString(resp.AsSpan(0, Math.Min(48, resp.Length)))
: "(empty)";
_output.WriteLine($" {name,-26} ok={ok} respLen={resp?.Length ?? 0} errLen={err?.Length ?? 0} resp={respHex}");
results[name] = (ok, resp?.Length ?? 0);
}
});
// Locked finding (2026-06-20): on 2020 WCF, GETHI is a named-value query whose only
// working key is HistorianVersion. No storage-mode key is honored — EventStorageMode is
// not on the 2020 WCF wire (it is the 2023R2 HCAL-native/gRPC 518-byte-struct path).
Assert.True(results["HistorianVersion"].ok);
Assert.True(results["HistorianVersion"].respLen > 0);
foreach (string storageName in new[]
{
"EventStorageMode", "EventStorageType", "StorageType",
"HistorianEventStorageMode", "EventStorage", "StorageMode",
})
{
Assert.False(results[storageName].ok, $"unexpected: GETHI honored '{storageName}'");
}
}
[Fact]
public void ExeC_WithUppercaseStorageGuid_AgainstLocalHistorian()
{
if (!ShouldRun(out string host)) return;
HistorianClientOptions options = new()
{
Host = host,
IntegratedSecurity = true,
Transport = HistorianTransport.LocalPipe
};
Guid contextKey = Guid.NewGuid();
var (histBinding, histEndpoint, retrBinding, retrEndpoint) = HistorianWcfBindingFactory.CreateBindingPair(options);
HistorianWcfAuthChainHelper.OpenAuthenticatedConnection(
options, histBinding, histEndpoint, contextKey, CancellationToken.None,
additionalSetup: (_, context) =>
{
string handle = context.StorageSessionId.ToString("D").ToUpperInvariant();
ChannelFactory<IRetrievalServiceContract3> factory = new(retrBinding, retrEndpoint);
HistorianWcfClientCredentialsHelper.Configure(factory, options);
IRetrievalServiceContract3 channel = factory.CreateChannel();
ICommunicationObject co = (ICommunicationObject)channel;
try
{
// Prime the Retr service version handshake (Retr.GetV), as the native client does.
channel.GetInterfaceVersion(out uint retrVersion);
_output.WriteLine($"Retr.GetV version={retrVersion}");
uint queryHandle = 0;
bool execOk = channel.ExecuteSqlCommand(
handle, "SELECT 1 AS ProbeValue", 0u, ref queryHandle,
out int retValue, out uint errSize, out byte[] errBuf);
_output.WriteLine($"ExeC returned={execOk} retValue={retValue} queryHandle={queryHandle} errSize={errSize} errLen={errBuf?.Length ?? 0}");
if (execOk)
{
uint sequence = 0;
bool getrOk = channel.GetRecordSetByteStream(
handle, queryHandle, ref sequence,
out uint resultSize, out byte[] resultBuf, out uint gErrSize, out byte[] gErrBuf);
_output.WriteLine($"GetR returned={getrOk} resultSize={resultSize} resultLen={resultBuf?.Length ?? 0} sequence={sequence}");
if (resultBuf is { Length: > 0 })
{
_output.WriteLine(" result[0..96]=" + Convert.ToHexString(resultBuf.AsSpan(0, Math.Min(96, resultBuf.Length))));
}
}
}
finally
{
try { if (co.State == CommunicationState.Faulted) co.Abort(); else co.Close(); } catch { try { co.Abort(); } catch { } }
try { if (factory.State == CommunicationState.Faulted) factory.Abort(); else factory.Close(); } catch { try { factory.Abort(); } catch { } }
}
});
}
private static void ProbeOnStatusChannel(HistorianClientOptions options, Action<IStatusServiceContract2, string> probe)
{
Guid contextKey = Guid.NewGuid();
var (histBinding, histEndpoint, _, _) = HistorianWcfBindingFactory.CreateBindingPair(options);
Binding statusBinding = HistorianWcfBindingFactory.CreateAuxiliaryBinding(options);
EndpointAddress statusEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(options, HistorianWcfServiceNames.Status);
HistorianWcfAuthChainHelper.OpenAuthenticatedConnection(
options, histBinding, histEndpoint, contextKey, CancellationToken.None,
additionalSetup: (_, context) =>
{
string handle = context.StorageSessionId.ToString("D").ToUpperInvariant();
ChannelFactory<IStatusServiceContract2> factory = new(statusBinding, statusEndpoint);
IStatusServiceContract2 channel = factory.CreateChannel();
ICommunicationObject co = (ICommunicationObject)channel;
try
{
probe(channel, handle);
}
catch (Exception ex)
{
throw new InvalidOperationException($"probe raised: {ex.GetType().Name}: {ex.Message}", ex);
}
finally
{
try { if (co.State == CommunicationState.Faulted) co.Abort(); else co.Close(); } catch { try { co.Abort(); } catch { } }
try { if (factory.State == CommunicationState.Faulted) factory.Abort(); else factory.Close(); } catch { try { factory.Abort(); } catch { } }
}
});
}
/// <summary>
/// R1.3 / R1.4 reachability probe: ask GetSystemParameter (config) and GetRuntimeParameter (GETRP,
/// live runtime state) for timezone + event-storage-mode candidate keys. If the server timezone or
/// EventStorageMode surfaces through either named-parameter op, R1.3/R1.4 are deliverable on 2020;
/// if every candidate returns null, they are confirmed 2023R2/gRPC-only from the parameter angle.
/// Prints results — not an assertion test.
/// </summary>
[Fact]
public async Task TimezoneAndStorageMode_ParameterProbe_AgainstLocalHistorian()
{
if (!ShouldRun(out string host)) return;
HistorianClient client = new(new HistorianClientOptions
{
Host = host,
IntegratedSecurity = true,
Transport = HistorianTransport.LocalPipe
});
// Timezone (R1.3) candidates + the one existing time-ish SystemParameter (TimeStampRule);
// EventStorageMode (R1.4) candidates + the existing EventStorage* params as controls.
string[] candidates =
[
"TimeZone", "ServerTimeZone", "SystemTimeZone", "TimeZoneName", "SystemTimeZoneName",
"TimeStampRule", "ServerTime",
"EventStorageMode", "StorageMode", "EventStorage", "EventStorageDuration",
"HistorianVersion", // known-good control
];
foreach (string key in candidates)
{
string? sys = null, run = null;
string sysErr = "", runErr = "";
try { sys = await client.GetSystemParameterAsync(key); } catch (Exception ex) { sysErr = ex.GetType().Name; }
try { run = await client.GetRuntimeParameterAsync(key); } catch (Exception ex) { runErr = ex.GetType().Name; }
_output.WriteLine($"{key,-22} SystemParameter={(sys is null ? "<null>" : $"\"{sys}\"")}{(sysErr.Length > 0 ? $" [{sysErr}]" : "")} RuntimeParameter={(run is null ? "<null>" : $"\"{run}\"")}{(runErr.Length > 0 ? $" [{runErr}]" : "")}");
}
}
}
@@ -1,9 +1,63 @@
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
namespace AVEVA.Historian.Client.Tests;
public sealed class WcfEventQueryProtocolTests
{
// Filter block (offset 0x1E into pRequestBuff) captured from a native
// EventQuery.AddEventFilter("Area", Equal, "RetestFilterArea") StartEventQuery, via
// instrument-wcf-writemessage.
private const string CaptureFilterBlockHex =
"000001000000010000000400000041007200650061000100000000000100000009100052657465737446696c7465724172656100";
private const int FilterBlockOffset = 0x1E;
[Fact]
public void SerializerMatchesInstrumentedNativeEventFilterBlock()
{
HistorianEventQueryAttempt attempt = Assert.Single(HistorianEventQueryProtocol.CreateStartEventQueryAttempts(
new DateTime(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc),
new DateTime(2026, 1, 2, 0, 0, 0, DateTimeKind.Utc),
5,
new HistorianEventFilter("Area", HistorianEventComparison.Equal, "RetestFilterArea")));
byte[] expectedBlock = Convert.FromHexString(CaptureFilterBlockHex);
byte[] actualBlock = attempt.RequestBuffer[FilterBlockOffset..(FilterBlockOffset + expectedBlock.Length)];
Assert.Equal("native-filter-version5", attempt.Name);
Assert.Equal(expectedBlock, actualBlock);
}
[Fact]
public void FilterBlockEncodesComparisonOperatorAsUInt16()
{
// Equal = 0, Contains = 12 — the op field is the only byte that differs between them
// (offset 0x38 in pRequestBuff = 0x1A into the filter block).
byte[] equal = HistorianEventQueryProtocol.CreateStartEventQueryAttempts(
new DateTime(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc),
new DateTime(2026, 1, 2, 0, 0, 0, DateTimeKind.Utc),
5, new HistorianEventFilter("Area", HistorianEventComparison.Equal, "X"))[0].RequestBuffer;
byte[] contains = HistorianEventQueryProtocol.CreateStartEventQueryAttempts(
new DateTime(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc),
new DateTime(2026, 1, 2, 0, 0, 0, DateTimeKind.Utc),
5, new HistorianEventFilter("Area", HistorianEventComparison.Contains, "X"))[0].RequestBuffer;
Assert.Equal(0x38, FilterBlockOffset + 0x1A);
Assert.Equal(0, BitConverter.ToUInt16(equal, 0x38));
Assert.Equal(12, BitConverter.ToUInt16(contains, 0x38));
}
[Fact]
public void NullFilterStillProducesTheEmptyFilterBuffer()
{
HistorianEventQueryAttempt withNull = HistorianEventQueryProtocol.CreateStartEventQueryAttempts(
new DateTime(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc),
new DateTime(2026, 1, 1, 0, 1, 0, DateTimeKind.Utc), 3)[0];
Assert.Equal("native-empty-filter-version5", withNull.Name);
Assert.Equal(65, withNull.RequestBuffer.Length);
}
[Fact]
public void SerializerMatchesInstrumentedNativeEventRequest()
{
@@ -0,0 +1,92 @@
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
namespace AVEVA.Historian.Client.Tests;
public sealed class WcfEventWriteProtocolTests
{
// Golden bytes captured from a native AddStreamedValue(HistorianEvent) over WCF (AddS2.pBuf),
// instrument-wcf-writemessage, event-send capture A (Type="User.Write").
private const string CaptureAddS2PBufBase64 =
"T1MBANMAAADJAEWBOzXwXUZNolOHGu9JsyHwZ9L7+QDdAcAAwACNEbJ2OdoXacNJkFutCzR+krSAst37" +
"+QDdAQkSAFJldGVzdFNka0V2ZW50U2VuZAkKAFVzZXIuV3JpdGUFAAIACQYAU291cmNlQyYAEgBSAGUA" +
"dABlAHMAdABTAGQAawBFAHYAZQBuAHQAUwBlAG4AZAAJCgBUZXN0TWFya2VyQyoAFABoAGkAcwB0AHMA" +
"ZABrAC0AUgAyAC4AMQAtAGMAYQBwAHQAdQByAGUA";
[Fact]
public void SerializeAddStreamValuesBufferMatchesInstrumentedNativeEventSend()
{
// Exact field values recovered from capture A's pBuf.
Guid eventId = new("da3976b2-6917-49c3-905b-ad0b347e92b4");
DateTime eventTimeUtc = DateTime.FromFileTimeUtc(134264637562710000L);
DateTime receivedTimeUtc = DateTime.FromFileTimeUtc(134264637563449984L);
HistorianEvent evt = new(
Id: eventId,
EventTimeUtc: eventTimeUtc,
ReceivedTimeUtc: receivedTimeUtc,
Type: "User.Write",
SourceName: string.Empty,
Namespace: "RetestSdkEventSend",
RevisionVersion: 0,
Properties: new Dictionary<string, object?>
{
["Source"] = "RetestSdkEventSend",
["TestMarker"] = "histsdk-R2.1-capture",
});
// The capture's MDAS length marker excluded the final trailing pad byte, so the golden
// bytes are the 210-byte deterministic content; the serializer appends one pad byte
// (total 211) to satisfy the server's packet-length == buffer-size check.
byte[] expectedContent = Convert.FromBase64String(CaptureAddS2PBufBase64);
byte[] actual = HistorianEventWriteProtocol.SerializeAddStreamValuesBuffer(evt, receivedTimeUtc);
Assert.Equal(expectedContent.Length + 1, actual.Length);
Assert.Equal(expectedContent, actual[..expectedContent.Length]);
Assert.Equal(0, actual[^1]);
// The declared packet length (UInt32 @0x04) equals the delivered buffer length.
Assert.Equal((uint)actual.Length, BitConverter.ToUInt32(actual, 4));
}
[Fact]
public void BufferFramingFieldsAreDerivedFromValueBlobLength()
{
HistorianEvent evt = new(
Id: new("da3976b2-6917-49c3-905b-ad0b347e92b4"),
EventTimeUtc: DateTime.FromFileTimeUtc(134264637562710000L),
ReceivedTimeUtc: DateTime.FromFileTimeUtc(134264637563449984L),
Type: "User.Write",
SourceName: string.Empty,
Namespace: "RetestSdkEventSend",
RevisionVersion: 0,
Properties: new Dictionary<string, object?> { ["Source"] = "RetestSdkEventSend", ["TestMarker"] = "histsdk-R2.1-capture" });
byte[] buf = HistorianEventWriteProtocol.SerializeAddStreamValuesBuffer(evt, evt.ReceivedTimeUtc);
Assert.Equal(0x534F, BitConverter.ToUInt16(buf, 0)); // "OS"
Assert.Equal(1, BitConverter.ToUInt16(buf, 2)); // sampleCount
// Declared packet length (UInt32 @0x04) equals the delivered buffer length (incl. pad).
Assert.Equal((uint)buf.Length, BitConverter.ToUInt32(buf, 4));
// Inner length (UInt16 @0x08) = buffer length - 10.
Assert.Equal((ushort)(buf.Length - 10), BitConverter.ToUInt16(buf, 8));
// CM_EVENT tag id at the head of the value blob.
Assert.Equal(HistorianEventWriteProtocol.CmEventTagId, new Guid(buf.AsSpan(10, 16).ToArray()));
}
[Fact]
public void NonStringPropertyValueThrowsProtocolEvidenceMissing()
{
HistorianEvent evt = new(
Id: Guid.NewGuid(),
EventTimeUtc: new DateTime(2026, 6, 20, 12, 0, 0, DateTimeKind.Utc),
ReceivedTimeUtc: new DateTime(2026, 6, 20, 12, 0, 0, DateTimeKind.Utc),
Type: "User.Write",
SourceName: string.Empty,
Namespace: "ns",
RevisionVersion: 0,
Properties: new Dictionary<string, object?> { ["Count"] = 5 });
Assert.Throws<ProtocolEvidenceMissingException>(
() => HistorianEventWriteProtocol.SerializeAddStreamValuesBuffer(evt, evt.ReceivedTimeUtc));
}
}
@@ -0,0 +1,58 @@
using AVEVA.Historian.Client.Protocol;
using AVEVA.Historian.Client.Wcf;
namespace AVEVA.Historian.Client.Tests;
public sealed class WcfRuntimeParameterProtocolTests
{
// GETRP pRequestBuff captured from the native client for GetRuntimeParameter("HistorianVersion")
// via scripts/Capture-RuntimeParam.ps1 + instrument-wcf-writemessage:
// 54 67 01 00 signature(0x6754) + version(1)
// 01 00 00 00 name count = 1
// 10 00 00 00 char count = 16
// UTF-16LE "HistorianVersion"
private const string CaptureRequestHex =
"54670100010000001000000048006900730074006F007200690061006E00560065007200730069006F006E00";
// GETRP pResponseBuff captured from the paired GETRPResponse (instrument-wcf-readmessage):
// 01 00 version = 1
// 01 00 00 00 result count = 1
// 43 CRetVariant type 0x43 (VT_BSTR)
// 1A 00 payload length = 26 (= charCount field + string bytes)
// 0C 00 char count = 12
// UTF-16LE "20,0,000,000"
private const string CaptureResponseHex =
"010001000000431A000C00320030002C0030002C003000300030002C00300030003000";
[Fact]
public void SerializeRequestMatchesInstrumentedNativeRequestBuffer()
{
byte[] actual = HistorianRuntimeParameterProtocol.SerializeRequest("HistorianVersion");
Assert.Equal(Convert.FromHexString(CaptureRequestHex), actual);
}
[Fact]
public void ParseSingleStringResultReadsTheCapturedResponseValue()
{
byte[] response = Convert.FromHexString(CaptureResponseHex);
string? value = HistorianRuntimeParameterProtocol.ParseSingleStringResult(response);
Assert.Equal("20,0,000,000", value);
}
[Fact]
public void ParseSingleStringResultReturnsNullForZeroResultCount()
{
// version(1) + result count(0)
byte[] empty = [0x01, 0x00, 0x00, 0x00, 0x00, 0x00];
Assert.Null(HistorianRuntimeParameterProtocol.ParseSingleStringResult(empty));
}
[Fact]
public void ParseSingleStringResultThrowsForUncapturedVariantType()
{
// version(1) + count(1) + a non-string variant marker (0x03, VT_I4 — not captured).
byte[] buffer = [0x01, 0x00, 0x01, 0x00, 0x00, 0x00, 0x03, 0x04, 0x00, 0x00, 0x00, 0x00];
Assert.Throws<ProtocolEvidenceMissingException>(
() => HistorianRuntimeParameterProtocol.ParseSingleStringResult(buffer));
}
}
@@ -0,0 +1,79 @@
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Protocol;
using AVEVA.Historian.Client.Wcf;
namespace AVEVA.Historian.Client.Tests;
/// <summary>
/// Golden tests for the ExeC/GetR SQL result codec. The fixture is the real, clean GetR
/// <c>pResultBuff</c> captured from the live server for the benign query
/// <c>SELECT 1 AS ProbeValue</c> (no sensitive data) — an NRBF stream wrapping a
/// <c>System.Data.DataTable</c> serialized with <c>SerializationFormat.Xml</c>. It is the
/// contract-level byte[] the WCF channel reassembles (free of MDAS transport chunk markers).
/// </summary>
public sealed class WcfSqlResultProtocolTests
{
// GetR pResultBuff for "SELECT 1 AS ProbeValue" (1232 bytes), base64. Captured via the
// AVEVA_HISTORIAN_SQL_DUMP diagnostic in HistorianWcfSqlClient.
private const string GetRStreamBase64 =
"AAEAAAD/////AQAAAAAAAAAMAgAAAE5TeXN0ZW0uRGF0YSwgVmVyc2lvbj00LjAuMC4wLCBDdWx0dXJlPW5ldXRyYWwsIFB1Ymxp" +
"Y0tleVRva2VuPWI3N2E1YzU2MTkzNGUwODkFAQAAABVTeXN0ZW0uRGF0YS5EYXRhVGFibGUDAAAAGURhdGFUYWJsZS5SZW1vdGlu" +
"Z1ZlcnNpb24JWG1sU2NoZW1hC1htbERpZmZHcmFtAwEBDlN5c3RlbS5WZXJzaW9uAgAAAAkDAAAABgQAAACJBTw/eG1sIHZlcnNp" +
"b249IjEuMCIgZW5jb2Rpbmc9InV0Zi0xNiI/Pg0KPHhzOnNjaGVtYSB4bWxucz0iIiB4bWxuczp4cz0iaHR0cDovL3d3dy53My5v" +
"cmcvMjAwMS9YTUxTY2hlbWEiIHhtbG5zOm1zZGF0YT0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp4bWwtbXNkYXRhIj4NCiAg" +
"PHhzOmVsZW1lbnQgbmFtZT0iVGFibGUxIj4NCiAgICA8eHM6Y29tcGxleFR5cGU+DQogICAgICA8eHM6c2VxdWVuY2U+DQogICAg" +
"ICAgIDx4czplbGVtZW50IG5hbWU9IlByb2JlVmFsdWUiIG1zZGF0YTpSZWFkT25seT0idHJ1ZSIgdHlwZT0ieHM6aW50IiBtc2Rh" +
"dGE6dGFyZ2V0TmFtZXNwYWNlPSIiIC8+DQogICAgICA8L3hzOnNlcXVlbmNlPg0KICAgIDwveHM6Y29tcGxleFR5cGU+DQogIDwv" +
"eHM6ZWxlbWVudD4NCiAgPHhzOmVsZW1lbnQgbmFtZT0idG1wRGF0YVNldCIgbXNkYXRhOklzRGF0YVNldD0idHJ1ZSIgbXNkYXRh" +
"Ok1haW5EYXRhVGFibGU9IlRhYmxlMSIgbXNkYXRhOlVzZUN1cnJlbnRMb2NhbGU9InRydWUiPg0KICAgIDx4czpjb21wbGV4VHlw" +
"ZT4NCiAgICAgIDx4czpjaG9pY2UgbWluT2NjdXJzPSIwIiBtYXhPY2N1cnM9InVuYm91bmRlZCIgLz4NCiAgICA8L3hzOmNvbXBs" +
"ZXhUeXBlPg0KICA8L3hzOmVsZW1lbnQ+DQo8L3hzOnNjaGVtYT4GBQAAAJoCPGRpZmZncjpkaWZmZ3JhbSB4bWxuczptc2RhdGE9" +
"InVybjpzY2hlbWFzLW1pY3Jvc29mdC1jb206eG1sLW1zZGF0YSIgeG1sbnM6ZGlmZmdyPSJ1cm46c2NoZW1hcy1taWNyb3NvZnQt" +
"Y29tOnhtbC1kaWZmZ3JhbS12MSI+DQogIDx0bXBEYXRhU2V0Pg0KICAgIDxUYWJsZTEgZGlmZmdyOmlkPSJUYWJsZTExIiBtc2Rh" +
"dGE6cm93T3JkZXI9IjAiPg0KICAgICAgPFByb2JlVmFsdWU+MTwvUHJvYmVWYWx1ZT4NCiAgICA8L1RhYmxlMT4NCiAgPC90bXBE" +
"YXRhU2V0Pg0KPC9kaWZmZ3I6ZGlmZmdyYW0+BAMAAAAOU3lzdGVtLlZlcnNpb24EAAAABl9NYWpvcgZfTWlub3IGX0J1aWxkCV9S" +
"ZXZpc2lvbgAAAAAICAgIAgAAAAAAAAD//////////ws=";
[Fact]
public void Parse_DataTableStream_ExtractsColumnAndTypedRow()
{
byte[] stream = Convert.FromBase64String(GetRStreamBase64);
HistorianSqlResult result = HistorianSqlResultProtocol.Parse(stream, returnValue: 0);
HistorianSqlColumn column = Assert.Single(result.Columns);
Assert.Equal("ProbeValue", column.Name);
Assert.Equal("xs:int", column.SchemaType);
IReadOnlyList<object?> row = Assert.Single(result.Rows);
object? value = Assert.Single(row);
Assert.Equal(1, value); // typed to int per the xs:int schema
Assert.IsType<int>(value);
Assert.Equal(0, result.ReturnValue);
}
[Fact]
public void Parse_PreservesReturnValue()
{
byte[] stream = Convert.FromBase64String(GetRStreamBase64);
HistorianSqlResult result = HistorianSqlResultProtocol.Parse(stream, returnValue: 7);
Assert.Equal(7, result.ReturnValue);
}
[Fact]
public void Parse_EmptyBuffer_ReturnsEmptyResult()
{
HistorianSqlResult result = HistorianSqlResultProtocol.Parse(ReadOnlyMemory<byte>.Empty, returnValue: 3);
Assert.Empty(result.Columns);
Assert.Empty(result.Rows);
Assert.Equal(3, result.ReturnValue);
}
[Fact]
public void Parse_NonNrbfBuffer_Throws()
{
byte[] garbage = [0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08];
Assert.Throws<ProtocolEvidenceMissingException>(
() => HistorianSqlResultProtocol.Parse(garbage, returnValue: 0));
}
}
@@ -0,0 +1,120 @@
using System.Buffers.Binary;
using System.Text;
using AVEVA.Historian.Client.Protocol;
using AVEVA.Historian.Client.Wcf;
namespace AVEVA.Historian.Client.Tests;
/// <summary>
/// Golden-byte tests for the GetTepByNm (GetTagExtendedPropertiesFromName) codec. The byte layout
/// is pinned to the live capture documented on <see cref="HistorianTagExtendedPropertyProtocol"/>
/// (scripts/Capture-TagExtendedProperties.ps1); synthetic tag/property values are used here so no
/// dev tag names land in the repo, but the framing is byte-for-byte the captured structure.
/// </summary>
public sealed class WcfTagExtendedPropertyProtocolTests
{
[Fact]
public void SerializeRequest_SingleTag_MatchesCapturedLayout()
{
// tagNames = uint32 count(1) + uint32 charCount + UTF-16LE chars.
const string tag = "Reactor.Temp1";
byte[] actual = HistorianTagExtendedPropertyProtocol.SerializeRequest(tag);
using MemoryStream expected = new();
using (BinaryWriter w = new(expected, Encoding.Unicode, leaveOpen: true))
{
w.Write(1u);
w.Write((uint)tag.Length);
w.Write(Encoding.Unicode.GetBytes(tag));
}
Assert.Equal(expected.ToArray(), actual);
}
[Fact]
public void ParseResponse_SingleStringProperty_ExtractsNameAndValue()
{
byte[] buffer = BuildResponse("Reactor.Temp1", "Location", "Plant/AreaA");
IReadOnlyList<HistorianTagExtendedPropertyRow> rows =
HistorianTagExtendedPropertyProtocol.ParseResponse(buffer);
HistorianTagExtendedPropertyRow row = Assert.Single(rows);
Assert.Equal("Reactor.Temp1", row.TagName);
Assert.Equal("Location", row.PropertyName);
Assert.Equal("Plant/AreaA", row.Value);
}
[Fact]
public void ParseResponse_ZeroTagCount_ReturnsEmpty_ForPagingTermination()
{
// The terminal page of the sequence loop carries tagCount = 0.
byte[] terminal = [0x00, 0x00, 0x00, 0x00];
Assert.Empty(HistorianTagExtendedPropertyProtocol.ParseResponse(terminal));
}
[Fact]
public void ParseResponse_NonStringVariant_Throws()
{
// Replace the 0x43 VT_BSTR marker with an unmapped variant type (0x03).
byte[] buffer = BuildResponse("Reactor.Temp1", "Location", "Plant/AreaA");
int variantOffset = Array.IndexOf(buffer, (byte)0x43);
Assert.True(variantOffset > 0);
buffer[variantOffset] = 0x03;
Assert.Throws<ProtocolEvidenceMissingException>(
() => HistorianTagExtendedPropertyProtocol.ParseResponse(buffer));
}
/// <summary>
/// Builds a GetTepByNm response buffer byte-for-byte per the captured layout: uint32 tagCount,
/// then per tag [marker 0x01][compact-ASCII name][uint32 propCount][per prop marker 0x02 +
/// compact-ASCII name + 0x43 VT_BSTR value][trailing 0x01].
/// </summary>
private static byte[] BuildResponse(string tag, string propName, string propValue)
{
using MemoryStream ms = new();
using BinaryWriter w = new(ms, Encoding.ASCII, leaveOpen: true);
WriteUInt32(w, 1u); // tagCount
w.Write((byte)0x01); // group marker
WriteCompactAscii(w, tag);
WriteUInt32(w, 1u); // propertyCount
w.Write((byte)0x02); // property marker
WriteCompactAscii(w, propName);
WriteVariantString(w, propValue);
w.Write((byte)0x01); // trailing marker
w.Flush();
return ms.ToArray();
}
private static void WriteUInt32(BinaryWriter w, uint value)
{
Span<byte> b = stackalloc byte[4];
BinaryPrimitives.WriteUInt32LittleEndian(b, value);
w.Write(b);
}
private static void WriteUInt16(BinaryWriter w, ushort value)
{
Span<byte> b = stackalloc byte[2];
BinaryPrimitives.WriteUInt16LittleEndian(b, value);
w.Write(b);
}
private static void WriteCompactAscii(BinaryWriter w, string value)
{
w.Write((byte)0x09);
WriteUInt16(w, (ushort)value.Length);
w.Write(Encoding.ASCII.GetBytes(value));
}
private static void WriteVariantString(BinaryWriter w, string value)
{
w.Write((byte)0x43); // VT_BSTR
WriteUInt16(w, (ushort)(2 + value.Length * 2)); // payload length = charCount field + UTF-16 bytes
WriteUInt16(w, (ushort)value.Length); // char count
w.Write(Encoding.Unicode.GetBytes(value));
}
}
@@ -0,0 +1,126 @@
using System.Buffers.Binary;
using System.Text;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
namespace AVEVA.Historian.Client.Tests;
/// <summary>
/// Golden-byte tests for the <c>AddTEx</c> (AddTagExtendedProperties) inBuff serializer (HCAL R1.11).
/// The reference buffer is the exact byte[] the SDK handed the WCF channel in the live
/// <c>AddTagExtendedPropertiesAsync_AgainstLocalHistorian_WritesAndReadsBack</c> run — the server
/// accepted it and the property was read back, so it is server-validated.
/// </summary>
public sealed class WcfTagExtendedPropertyWriteProtocolTests
{
// Server-accepted AddTEx inBuff for tag "RetestSdkWriteTepSdk3", property "SdkLiveTestProp" =
// "SdkLiveTestValue", dumped via AVEVA_HISTORIAN_TEP_DUMP during the live write test.
private const string ServerAcceptedInBuffBase64 =
"AQAAAAEJFQBSZXRlc3RTZGtXcml0ZVRlcFNkazMBAAAAAgkPAFNka0xpdmVUZXN0UHJvcEMiABAAUwBkAGsATABpAHYAZQBUAGUAcwB0AFYAYQBsAHUAZQABAA==";
[Fact]
public void SerializeAddRequest_MatchesServerAcceptedBuffer()
{
byte[] expected = Convert.FromBase64String(ServerAcceptedInBuffBase64);
byte[] actual = HistorianTagExtendedPropertyProtocol.SerializeAddRequest(
"RetestSdkWriteTepSdk3",
[new HistorianTagExtendedProperty("SdkLiveTestProp", "SdkLiveTestValue")]);
Assert.Equal(expected, actual);
}
[Fact]
public void SerializeAddRequest_SingleProperty_HasExpectedLayout()
{
byte[] buf = HistorianTagExtendedPropertyProtocol.SerializeAddRequest(
"ReactorTemp", [new HistorianTagExtendedProperty("Location", "PlantA")]);
int c = 0;
Assert.Equal(1u, BinaryPrimitives.ReadUInt32LittleEndian(buf.AsSpan(c, 4))); c += 4; // group count
Assert.Equal(0x01, buf[c++]); // group marker
Assert.Equal(0x09, buf[c++]); // compact string marker
Assert.Equal(11, BinaryPrimitives.ReadUInt16LittleEndian(buf.AsSpan(c, 2))); c += 2; // tag byte len
Assert.Equal("ReactorTemp", Encoding.ASCII.GetString(buf.AsSpan(c, 11))); c += 11;
Assert.Equal(1u, BinaryPrimitives.ReadUInt32LittleEndian(buf.AsSpan(c, 4))); c += 4; // property count
Assert.Equal(0x02, buf[c++]); // property marker
Assert.Equal(0x09, buf[c++]);
Assert.Equal(8, BinaryPrimitives.ReadUInt16LittleEndian(buf.AsSpan(c, 2))); c += 2;
Assert.Equal("Location", Encoding.ASCII.GetString(buf.AsSpan(c, 8))); c += 8;
Assert.Equal(0x43, buf[c++]); // VT_BSTR
Assert.Equal(14, BinaryPrimitives.ReadUInt16LittleEndian(buf.AsSpan(c, 2))); c += 2; // payloadLen = 2 + 6*2
Assert.Equal(6, BinaryPrimitives.ReadUInt16LittleEndian(buf.AsSpan(c, 2))); c += 2; // char count
Assert.Equal("PlantA", Encoding.Unicode.GetString(buf.AsSpan(c, 12))); c += 12;
Assert.Equal(0x01, buf[c++]); // group trailer
Assert.Equal(0x00, buf[c++]); // buffer terminator
Assert.Equal(buf.Length, c);
}
[Fact]
public void SerializeAddRequest_MultipleProperties_EncodesCount()
{
byte[] buf = HistorianTagExtendedPropertyProtocol.SerializeAddRequest(
"T", [new HistorianTagExtendedProperty("A", "1"), new HistorianTagExtendedProperty("B", "2")]);
// group count @0, then 0x01 marker, 0x09 + u16(1) + "T" => property count at offset 4+1+1+2+1 = 9
Assert.Equal(2u, BinaryPrimitives.ReadUInt32LittleEndian(buf.AsSpan(9, 4)));
}
[Fact]
public void SerializeAddRequest_NoProperties_Throws()
{
Assert.Throws<ArgumentException>(() =>
HistorianTagExtendedPropertyProtocol.SerializeAddRequest("T", Array.Empty<HistorianTagExtendedProperty>()));
}
// Server-accepted DelTep inBuff for tag "RetestSdkWriteDelTepSdk", property "SdkDelProp",
// extracted byte-for-byte from the native DeleteTagExtendedPropertiesByName WriteMessage in the
// cross-session capture (scripts/Capture-DeleteTagExtendedProperties.ps1) — DelTep returned
// success, so it is server-validated.
private const string ServerAcceptedDeleteInBuffBase64 =
"AQAAAAEJFwBSZXRlc3RTZGtXcml0ZURlbFRlcFNkawEAAAACCQoAU2RrRGVsUHJvcAAA";
[Fact]
public void SerializeDeleteRequest_MatchesServerAcceptedBuffer()
{
byte[] expected = Convert.FromBase64String(ServerAcceptedDeleteInBuffBase64);
byte[] actual = HistorianTagExtendedPropertyProtocol.SerializeDeleteRequest(
"RetestSdkWriteDelTepSdk", ["SdkDelProp"]);
Assert.Equal(expected, actual);
}
[Fact]
public void SerializeDeleteRequest_SingleProperty_HasExpectedLayout()
{
byte[] buf = HistorianTagExtendedPropertyProtocol.SerializeDeleteRequest(
"ReactorTemp", ["Location"]);
int c = 0;
Assert.Equal(1u, BinaryPrimitives.ReadUInt32LittleEndian(buf.AsSpan(c, 4))); c += 4; // group count
Assert.Equal(0x01, buf[c++]); // group marker
Assert.Equal(0x09, buf[c++]); // compact string marker
Assert.Equal(11, BinaryPrimitives.ReadUInt16LittleEndian(buf.AsSpan(c, 2))); c += 2; // tag byte len
Assert.Equal("ReactorTemp", Encoding.ASCII.GetString(buf.AsSpan(c, 11))); c += 11;
Assert.Equal(1u, BinaryPrimitives.ReadUInt32LittleEndian(buf.AsSpan(c, 4))); c += 4; // property count
Assert.Equal(0x02, buf[c++]); // property marker
Assert.Equal(0x09, buf[c++]);
Assert.Equal(8, BinaryPrimitives.ReadUInt16LittleEndian(buf.AsSpan(c, 2))); c += 2;
Assert.Equal("Location", Encoding.ASCII.GetString(buf.AsSpan(c, 8))); c += 8;
// Delete carries NO value variant (Add would have 0x43 ... here).
Assert.Equal(0x00, buf[c++]); // group trailer (0x00 for delete, vs 0x01 for add)
Assert.Equal(0x00, buf[c++]); // buffer terminator
Assert.Equal(buf.Length, c);
}
[Fact]
public void SerializeDeleteRequest_MultipleNames_EncodesCount()
{
byte[] buf = HistorianTagExtendedPropertyProtocol.SerializeDeleteRequest("T", ["A", "B"]);
// group count @0, then 0x01 marker, 0x09 + u16(1) + "T" => property count at offset 4+1+1+2+1 = 9
Assert.Equal(2u, BinaryPrimitives.ReadUInt32LittleEndian(buf.AsSpan(9, 4)));
}
[Fact]
public void SerializeDeleteRequest_NoProperties_Throws()
{
Assert.Throws<ArgumentException>(() =>
HistorianTagExtendedPropertyProtocol.SerializeDeleteRequest("T", Array.Empty<string>()));
}
}
@@ -0,0 +1,62 @@
using System.Buffers.Binary;
using System.Text;
using AVEVA.Historian.Client.Wcf;
namespace AVEVA.Historian.Client.Tests;
/// <summary>
/// Golden-byte tests for the History <c>StartJob</c> (StJb) rename job buffer (HCAL R1.10).
/// The reference buffer is the exact byte[] the SDK handed the WCF channel in the live
/// <c>RenameTagsAsync_AgainstLocalHistorian_RenamesSandboxTag</c> run — the server accepted it and
/// the tag was renamed, so it is server-validated, not hand-derived from a chunk-mangled capture.
/// </summary>
public sealed class WcfTagRenameProtocolTests
{
// Server-accepted clean jobBuffer for pairs [("RetestSdkWriteSdkRenameSrc","RetestSdkWriteSdkRenameDst")],
// dumped via AVEVA_HISTORIAN_RENAME_DUMP during the live rename test.
private const string ServerAcceptedJobBufferBase64 =
"AAAAAAAAAAEAAAAaAAAAUgBlAHQAZQBzAHQAUwBkAGsAVwByAGkAdABlAFMAZABrAFIAZQBuAGEAbQBlAFMAcgBjABoAAABSAGUAdABlAHMAdABTAGQAawBXAHIAaQB0AGUAUwBkAGsAUgBlAG4AYQBtAGUARABzAHQA";
[Fact]
public void SerializeRenameJob_MatchesServerAcceptedBuffer()
{
byte[] expected = Convert.FromBase64String(ServerAcceptedJobBufferBase64);
byte[] actual = HistorianTagRenameProtocol.SerializeRenameJob(
[("RetestSdkWriteSdkRenameSrc", "RetestSdkWriteSdkRenameDst")]);
Assert.Equal(expected, actual);
}
[Fact]
public void SerializeRenameJob_SinglePair_HasExpectedLayout()
{
byte[] buf = HistorianTagRenameProtocol.SerializeRenameJob([("ReactorTempA", "ReactorTempB")]);
// 7-byte zero prefix + uint32 pairCount + (uint32 charCount + UTF-16)×2.
Assert.Equal(7 + 4 + 4 + 24 + 4 + 24, buf.Length);
for (int i = 0; i < 7; i++) Assert.Equal(0, buf[i]);
Assert.Equal(1u, BinaryPrimitives.ReadUInt32LittleEndian(buf.AsSpan(7, 4)));
Assert.Equal(12u, BinaryPrimitives.ReadUInt32LittleEndian(buf.AsSpan(11, 4)));
Assert.Equal("ReactorTempA", Encoding.Unicode.GetString(buf.AsSpan(15, 24)));
Assert.Equal(12u, BinaryPrimitives.ReadUInt32LittleEndian(buf.AsSpan(39, 4)));
Assert.Equal("ReactorTempB", Encoding.Unicode.GetString(buf.AsSpan(43, 24)));
}
[Fact]
public void SerializeRenameJob_MultiplePairs_EncodesCountAndOrder()
{
byte[] buf = HistorianTagRenameProtocol.SerializeRenameJob(
[("AaOld", "AaNew"), ("BbOld", "BbNew")]);
Assert.Equal(2u, BinaryPrimitives.ReadUInt32LittleEndian(buf.AsSpan(7, 4)));
// First pair old name immediately follows the count + its length field.
Assert.Equal(5u, BinaryPrimitives.ReadUInt32LittleEndian(buf.AsSpan(11, 4)));
Assert.Equal("AaOld", Encoding.Unicode.GetString(buf.AsSpan(15, 10)));
}
[Fact]
public void SerializeRenameJob_EmptyBatch_Throws()
{
Assert.Throws<ArgumentException>(() =>
HistorianTagRenameProtocol.SerializeRenameJob(Array.Empty<(string, string)>()));
}
}
@@ -36,6 +36,19 @@ internal static class Program
string runtimeMethodPointerFilters = GetArg(args, "--runtime-method-pointer-filters")
?? "StartDataQuery;StartQuery;GetNextRow;StartEventQuery";
ulong resolutionTicks = ulong.TryParse(GetArg(args, "--resolution-ticks"), out ulong parsedResolutionTicks) ? parsedResolutionTicks : 0;
// Summary-query knobs on HistoryQueryArgs (R1.8/R1.9 capture). Left null/0 = not set,
// so a normal Full read is unaffected. ValueSelector/AggregationType/MaxStates/Filter
// are the native properties that turn a Cyclic/Full query into an analog/state summary.
string? valueSelectorName = GetArg(args, "--value-selector");
string? aggregationTypeName = GetArg(args, "--aggregation-type");
uint maxStates = uint.TryParse(GetArg(args, "--max-states"), out uint parsedMaxStates) ? parsedMaxStates : 0;
string? historyFilter = GetArg(args, "--filter");
// R1.5 capture: flip the TagQueryArgs flag that makes the native client retrieve extended
// properties in the index-based TagQuery path. (The dedicated tag-extended-properties
// scenario, which drives the name-based GetTagExtendedPropertiesByName, is the reliable
// GetTepByNm capture path; the TagQuery path is gated behind QTB, which fails server-side
// here.) Off by default so the normal tag-query scenario is unchanged.
bool retrieveExtendedProperties = HasFlag(args, "--retrieve-extended-properties");
DateTime endUtc = TryParseUtc(GetArg(args, "--end-utc")) ?? DateTime.UtcNow;
DateTime startUtc = TryParseUtc(GetArg(args, "--start-utc")) ?? endUtc.AddMinutes(-lookbackMinutes);
@@ -77,6 +90,31 @@ internal static class Program
return 0;
}
string? dumpTypeName = GetArg(args, "--dump-type-members");
if (dumpTypeName is not null)
{
Type dumpType = GetType(assembly, dumpTypeName);
if (dumpType.IsEnum)
{
var values = Enum.GetValues(dumpType).Cast<object>()
.Select(v => $"{v} = {Convert.ToInt64(v)}").OrderBy(s => s).ToArray();
Console.WriteLine(Serialize(new { Type = dumpType.FullName, EnumValues = values }));
return 0;
}
BindingFlags df = BindingFlags.Instance | BindingFlags.Static | BindingFlags.Public | BindingFlags.NonPublic;
Console.WriteLine(Serialize(new
{
Type = dumpType.FullName,
Properties = dumpType.GetProperties(df).Select(p => $"{p.PropertyType.Name} {p.Name}").OrderBy(s => s).ToArray(),
Fields = dumpType.GetFields(df).Select(f => $"{f.FieldType.Name} {f.Name}").OrderBy(s => s).ToArray(),
Methods = dumpType.GetMethods(df)
.Where(m => !m.IsSpecialName)
.Select(m => $"{m.ReturnType.Name} {m.Name}({string.Join(", ", m.GetParameters().Select(p => p.ParameterType.Name))})")
.OrderBy(s => s).ToArray(),
}));
return 0;
}
Type accessType = GetType(assembly, "ArchestrA.HistorianAccess");
Type connectionArgsType = GetType(assembly, "ArchestrA.HistorianConnectionArgs");
Type connectionStatusType = GetType(assembly, "ArchestrA.HistorianConnectionStatus");
@@ -93,9 +131,9 @@ internal static class Program
object connectionArgs = Activator.CreateInstance(connectionArgsType)!;
SetProperty(connectionArgs, "ServerName", serverName);
SetProperty(connectionArgs, "TcpPort", checked((ushort)tcpPort));
SetProperty(connectionArgs, "ReadOnly", !IsWriteScenario(scenario));
SetProperty(connectionArgs, "ReadOnly", !(IsWriteScenario(scenario) || IsEventSendScenario(scenario) || IsAddTagExtendedPropertiesScenario(scenario)));
SetProperty(connectionArgs, "IntegratedSecurity", integratedSecurity);
SetProperty(connectionArgs, "ConnectionType", Enum.Parse(connectionType, IsEventScenario(scenario) ? "Event" : "Process"));
SetProperty(connectionArgs, "ConnectionType", Enum.Parse(connectionType, IsEventConnectionScenario(scenario) ? "Event" : "Process"));
if (directConnection)
{
SetProperty(connectionArgs, "DirectConnection", true);
@@ -130,7 +168,425 @@ internal static class Program
string? moveTerminalDescription = null;
List<object> rows = [];
if (openSuccess && status.ConnectedToServer && IsEventScenario(scenario))
if (openSuccess && status.ConnectedToServer && IsExecSqlScenario(scenario))
{
// R1.1 capture: drive HistorianAccess.ExecuteSqlCommand(sql, option, out retval,
// out DataTable, out error) so instrument-wcf-{write,read}message can observe the
// Retr.ExeC + Retr.GetR wire shape (handle format, command/option encoding, Retr
// priming, result byte stream). Read-only benign query.
string sql = GetArg(args, "--sql") ?? "SELECT 1 AS ProbeValue";
Type sqlOptionType = GetType(assembly, "ArchestrA.HistorianSqlExecuteOption");
object sqlOption = Enum.Parse(sqlOptionType, GetArg(args, "--sql-option") ?? "ExecuteRecord");
MethodInfo execMethod = accessType.GetMethods()
.First(m => m.Name == "ExecuteSqlCommand" && m.GetParameters().Length == 5);
object?[] execArgs = new object?[] { sql, sqlOption, 0, null, Activator.CreateInstance(errorType) };
WriteRuntimeMethodPointerSnapshot(assembly, runtimeMethodPointerOutput, runtimeMethodPointerFilters, repoRoot, scenario, "before-exec-sql");
bool execOk = (bool)execMethod.Invoke(access, execArgs)!;
int rowCount = -1, colCount = -1;
if (execArgs[3] is System.Data.DataTable table)
{
rowCount = table.Rows.Count;
colCount = table.Columns.Count;
}
Console.WriteLine(Serialize(new
{
Scenario = scenario,
Sql = sql,
ExecuteSqlCommandReturned = execOk,
ReturnValue = execArgs[2],
RowCount = rowCount,
ColumnCount = colCount,
Error = SnapshotObject(execArgs[4]!),
}));
return 0;
}
else if (openSuccess && status.ConnectedToServer && IsRuntimeParamScenario(scenario))
{
// R1.2 capture: drive HistorianAccess.GetRuntimeParameter(List<string> names,
// out List<object> results, out error) so instrument-wcf-{write,read}message can
// observe the WCF op name, handle type (uint vs string-handle wall), and the
// btRequest/btResponse buffer format. Pure status read — no write mode needed.
string namesArg = GetArg(args, "--runtime-param-names") ?? "HistorianVersion";
string[] names = namesArg.Split(new[] { ';' }, StringSplitOptions.RemoveEmptyEntries);
MethodInfo getRtParamMethod = accessType.GetMethods()
.First(m => m.Name == "GetRuntimeParameter" && m.GetParameters().Length == 3);
ParameterInfo[] rtParams = getRtParamMethod.GetParameters();
Type namesListType = rtParams[0].ParameterType; // List<string>
Type resultsListType = rtParams[1].ParameterType.GetElementType()!; // List<...>& -> List<...>
object namesList = Activator.CreateInstance(namesListType)!;
MethodInfo addName = namesListType.GetMethod("Add")!;
foreach (string n in names) addName.Invoke(namesList, new object?[] { n });
object rtError = Activator.CreateInstance(errorType)!;
object?[] rtArgs = new object?[] { namesList, Activator.CreateInstance(resultsListType), rtError };
WriteRuntimeMethodPointerSnapshot(assembly, runtimeMethodPointerOutput, runtimeMethodPointerFilters, repoRoot, scenario, "before-get-runtime-parameter");
bool rtOk = (bool)getRtParamMethod.Invoke(access, rtArgs)!;
object? resultsList = rtArgs[1];
var resultItems = new List<object?>();
if (resultsList is System.Collections.IEnumerable en)
{
foreach (object? item in en)
{
resultItems.Add(new
{
Type = item?.GetType().FullName,
Value = item?.ToString(),
});
}
}
Console.WriteLine(Serialize(new
{
Scenario = scenario,
Names = names,
GetRuntimeParameterReturned = rtOk,
ResultsListType = resultsListType.FullName,
Results = resultItems,
Error = SnapshotObject(rtArgs[2]!),
}));
return 0;
}
else if (openSuccess && status.ConnectedToServer && IsTagExtendedPropertiesScenario(scenario))
{
// R1.5 capture: drive HistorianAccess.GetTagExtendedPropertiesByName(string tagName,
// bool fetchFromServer, out TagExtendedPropertyGroup, out error) directly. This is the
// NAME-based entry point that issues the GetTepByNm WCF op WITHOUT a prior
// StartTagQuery (QTB) — the index-based TagQuery.GetTagExtendedPropertyInfo path is
// blocked here because QTB fails server-side (CMdServer StartActiveTagnamesQuery).
// The second GetTagExtendedPropertiesByName arg forces a server fetch (issues GetTepByNm)
// when true; when false the C++ client reads its local cache and returns err 41 if the
// tag's properties were never fetched. Default true so the scenario captures GetTepByNm;
// pass --tep-cache-only to exercise the cache-read (no WCF op) path.
bool fetchFromServer = !HasFlag(args, "--tep-cache-only");
// Prime the tag identity table first (ProcessTagNameIdentity inside
// GetTagExtendedPropertiesByName fails with err 41 if the tag was never resolved on
// this connection). GetTagInfoByName(tagName, cache, out HistorianTag, out err) is the
// proven uint-handle metadata path that registers the tag.
string? primeResult = null;
MethodInfo? getTagInfoByName = accessType.GetMethods()
.FirstOrDefault(m => m.Name == "GetTagInfoByName" && m.GetParameters().Length == 4);
if (getTagInfoByName is not null)
{
ParameterInfo[] tibParams = getTagInfoByName.GetParameters();
Type tagOutType = tibParams[2].ParameterType.GetElementType()!;
object tibError = Activator.CreateInstance(errorType)!;
object?[] tibArgs = new object?[] { tagName, true, null, tibError };
try
{
bool tibOk = (bool)getTagInfoByName.Invoke(access, tibArgs)!;
primeResult = $"GetTagInfoByName={tibOk} err={GetPropertyText(tibArgs[3], "ErrorDescription")}";
}
catch (TargetInvocationException ex)
{
primeResult = "GetTagInfoByName threw: " + FormatException(ex.InnerException ?? ex);
}
}
MethodInfo getTepByName = accessType.GetMethods()
.First(m => m.Name == "GetTagExtendedPropertiesByName" && m.GetParameters().Length == 4);
ParameterInfo[] tepParams = getTepByName.GetParameters();
Type groupType = tepParams[2].ParameterType.GetElementType()!; // TagExtendedPropertyGroup& -> TagExtendedPropertyGroup
object tepError = Activator.CreateInstance(errorType)!;
object?[] tepArgs = new object?[] { tagName, fetchFromServer, null, tepError };
WriteRuntimeMethodPointerSnapshot(assembly, runtimeMethodPointerOutput, runtimeMethodPointerFilters, repoRoot, scenario, "before-get-tag-extended-properties");
bool tepOk = false;
string? tepException = null;
try
{
tepOk = (bool)getTepByName.Invoke(access, tepArgs)!;
}
catch (TargetInvocationException ex)
{
tepException = FormatException(ex.InnerException ?? ex);
}
Console.WriteLine(Serialize(new
{
Scenario = scenario,
TagName = tagName,
FetchFromServer = fetchFromServer,
Prime = primeResult,
GroupType = groupType.FullName,
GetTagExtendedPropertiesByNameReturned = tepOk,
Exception = tepException,
Group = ToSerializableValue(tepArgs[2]),
GroupSnapshot = tepArgs[2] is null ? null : SnapshotObject(tepArgs[2]!),
Error = SnapshotObject(tepArgs[3]!),
}));
return 0;
}
else if (openSuccess && status.ConnectedToServer && IsAddTagExtendedPropertiesScenario(scenario))
{
// R1.11 capture: drive AddTagExtendedProperties(TagExtendedPropertyGroupList, out err)
// — and optionally DeleteTagExtendedPropertiesByName — so instrument-wcf-writemessage can
// observe the AddTEx / DelTep inBuff (tag + property name/value framing). Sandbox-guarded.
string tepTag = GetArg(args, "--tep-tag") ?? "RetestSdkWriteTepTag";
if (!tepTag.StartsWith("RetestSdkWrite", StringComparison.Ordinal))
{
throw new InvalidOperationException(
"add-tep scenario refuses tags that don't start with 'RetestSdkWrite'. Pass --tep-tag RetestSdkWrite...");
}
string propName = GetArg(args, "--tep-name") ?? "SdkTestProp";
string propValue = GetArg(args, "--tep-value") ?? "SdkTestValue";
var tepRows = new List<object>();
// 1) Ensure the tag exists (AddTag) unless --tep-skip-create.
if (!HasFlag(args, "--tep-skip-create"))
{
Type tagDefType = GetType(assembly, "ArchestrA.HistorianTag");
Type tagDataTypeEnum = GetType(assembly, "ArchestrA.HistorianDataType");
Type tagStorageTypeEnum = GetType(assembly, "ArchestrA.HistorianStorageType");
object tag = Activator.CreateInstance(tagDefType)!;
SetProperty(tag, "TagName", tepTag);
SetProperty(tag, "TagDescription", "SDK ext-property write RE sandbox tag");
SetProperty(tag, "EngineeringUnit", "test");
SetProperty(tag, "TagDataType", Enum.Parse(tagDataTypeEnum, "Float", ignoreCase: true));
SetProperty(tag, "TagStorageType", Enum.Parse(tagStorageTypeEnum, "Cyclic", ignoreCase: true));
SetProperty(tag, "MinEU", 0.0);
SetProperty(tag, "MaxEU", 100.0);
SetProperty(tag, "MinRaw", 0.0);
SetProperty(tag, "MaxRaw", 100.0);
SetProperty(tag, "StorageRate", 1000u);
SetProperty(tag, "ApplyScaling", false);
object addError = Activator.CreateInstance(errorType)!;
MethodInfo addTagMethod = accessType.GetMethod("AddTag", new[] { tagDefType, typeof(uint).MakeByRefType(), errorType.MakeByRefType() })!;
object?[] addTagArgs = [tag, 0u, addError];
bool addOk = (bool)addTagMethod.Invoke(access, addTagArgs)!;
tepRows.Add(new { Kind = "AddTag", Success = addOk, ErrorDescription = GetPropertyText(addTagArgs[2]!, "ErrorDescription") });
}
// Prime the tag identity (same reason as the read scenario — server-side resolution).
MethodInfo? getTagInfoByName = accessType.GetMethods()
.FirstOrDefault(m => m.Name == "GetTagInfoByName" && m.GetParameters().Length == 4);
if (getTagInfoByName is not null)
{
object tibError = Activator.CreateInstance(errorType)!;
object?[] tibArgs = new object?[] { tepTag, true, null, tibError };
try { getTagInfoByName.Invoke(access, tibArgs); } catch { }
}
// 2) Build TagExtendedPropertyGroupList { TagExtendedPropertyGroup { TagName, [TagExtendedProperty] } }
Type listType = GetType(assembly, "ArchestrA.TagExtendedPropertyGroupList");
Type groupType = GetType(assembly, "ArchestrA.TagExtendedPropertyGroup");
Type propType = GetType(assembly, "ArchestrA.TagExtendedProperty");
Type propDataTypeEnum = GetType(assembly, "ArchestrA.TagExtendedPropertyDataType");
object list = Activator.CreateInstance(listType)!;
object group = Activator.CreateInstance(groupType)!;
SetProperty(group, "TagName", tepTag);
object prop = Activator.CreateInstance(propType)!;
SetProperty(prop, "PropertyName", propName);
SetProperty(prop, "Type", Enum.Parse(propDataTypeEnum, "String", ignoreCase: true));
SetProperty(prop, "Value", propValue);
groupType.GetMethod("Add", new[] { propType })!.Invoke(group, [prop]);
listType.GetMethod("Add", new[] { groupType })!.Invoke(list, [group]);
// 2b) Add the property (AddTEx) unless --tep-skip-add. Skipped in the second
// (delete) session so the local cache isn't re-seeded with an UNSYNCED entry —
// DeleteTagExtendedPropertiesByName returns err 229 against unsynced properties.
if (!HasFlag(args, "--tep-skip-add"))
{
MethodInfo addTepMethod = accessType.GetMethods()
.First(m => m.Name == "AddTagExtendedProperties" && m.GetParameters().Length == 2);
object addTepError = Activator.CreateInstance(errorType)!;
object?[] addTepArgs = [list, addTepError];
WriteRuntimeMethodPointerSnapshot(assembly, runtimeMethodPointerOutput, runtimeMethodPointerFilters, repoRoot, scenario, "before-add-tep");
bool addTepOk = false;
string? addTepEx = null;
try { addTepOk = (bool)addTepMethod.Invoke(access, addTepArgs)!; }
catch (TargetInvocationException ex) { addTepEx = FormatException(ex.InnerException ?? ex); }
tepRows.Add(new
{
Kind = "AddTagExtendedProperties",
Success = addTepOk,
Exception = addTepEx,
ErrorDescription = GetPropertyText(addTepArgs[1]!, "ErrorDescription"),
ErrorCode = GetPropertyText(addTepArgs[1]!, "ErrorCode"),
});
}
// 3) Optional delete (DelTep) to capture its inBuff too. The native
// DeleteTagExtendedPropertiesByName performs a CLIENT-SIDE sync check and rejects
// (err 229) any property the local cache doesn't see as server-synchronized — so a
// just-added property can't be deleted in the same session. To capture DelTep:
// Run A: add-tep --tep-tag RetestSdkWriteDelTepSdk --tep-name P --tep-value V
// Run B: add-tep --tep-tag RetestSdkWriteDelTepSdk --tep-name P \
// --tep-skip-create --tep-skip-add --tep-delete
// Run B opens a FRESH connection, then we fetch the property from the server
// (GetTagExtendedPropertiesByName, fetchFromServer=true) to seed the local cache as
// SYNCED before deleting — which lets DeleteTagExtendedPropertiesByName reach the wire.
if (HasFlag(args, "--tep-delete"))
{
// Seed the synced local cache: prime tag identity + force a server fetch.
MethodInfo? primeForDelete = accessType.GetMethods()
.FirstOrDefault(m => m.Name == "GetTagInfoByName" && m.GetParameters().Length == 4);
if (primeForDelete is not null)
{
object tibErr = Activator.CreateInstance(errorType)!;
object?[] tibArgs = [tepTag, true, null, tibErr];
try { primeForDelete.Invoke(access, tibArgs); } catch { }
}
MethodInfo? readForSync = accessType.GetMethods()
.FirstOrDefault(m => m.Name == "GetTagExtendedPropertiesByName" && m.GetParameters().Length == 4);
string? syncRead = null;
if (readForSync is not null)
{
object syncErr = Activator.CreateInstance(errorType)!;
object?[] syncArgs = [tepTag, true, null, syncErr];
try
{
bool syncOk = (bool)readForSync.Invoke(access, syncArgs)!;
syncRead = $"GetTagExtendedPropertiesByName(fetch)={syncOk} err={GetPropertyText(syncArgs[3]!, "ErrorDescription")}";
}
catch (TargetInvocationException ex) { syncRead = "read-for-sync threw: " + FormatException(ex.InnerException ?? ex); }
}
tepRows.Add(new { Kind = "ReadForSync", Detail = syncRead });
MethodInfo? delTepMethod = accessType.GetMethods()
.FirstOrDefault(m => m.Name == "DeleteTagExtendedPropertiesByName" && m.GetParameters().Length == 4);
if (delTepMethod is not null)
{
Type namesColType = delTepMethod.GetParameters()[1].ParameterType; // StringCollection
object names = Activator.CreateInstance(namesColType)!;
namesColType.GetMethod("Add", new[] { typeof(string) })!.Invoke(names, [propName]);
object delErr = Activator.CreateInstance(errorType)!;
object?[] delArgs = [tepTag, names, true, delErr];
WriteRuntimeMethodPointerSnapshot(assembly, runtimeMethodPointerOutput, runtimeMethodPointerFilters, repoRoot, scenario, "before-del-tep");
bool delOk = false; string? delEx = null;
try { delOk = (bool)delTepMethod.Invoke(access, delArgs)!; }
catch (TargetInvocationException ex) { delEx = FormatException(ex.InnerException ?? ex); }
tepRows.Add(new { Kind = "DeleteTagExtendedPropertiesByName", Success = delOk, Exception = delEx, ErrorDescription = GetPropertyText(delArgs[3]!, "ErrorDescription"), ErrorCode = GetPropertyText(delArgs[3]!, "ErrorCode") });
}
}
Console.WriteLine(Serialize(new
{
Scenario = scenario,
TepTag = tepTag,
PropertyName = propName,
PropertyValue = propValue,
Rows = tepRows,
}));
return 0;
}
else if (openSuccess && status.ConnectedToServer && IsHistorianInfoScenario(scenario))
{
// R1.4 capture: drive HistorianAccess.GetHistorianInfo(out HistorianInfo, out error)
// so instrument-wcf-{write,read}message can observe the WCF GETHI pRequestBuff that
// returns the full 518-byte HISTORIAN_INFO struct (version@0 + EventStorageMode@514),
// distinct from the named-value "HistorianVersion" request. Pure status read.
MethodInfo getInfoMethod = accessType.GetMethods()
.First(m => m.Name == "GetHistorianInfo"
&& m.GetParameters().Length == 2
&& m.GetParameters()[0].ParameterType.IsByRef
&& m.GetParameters()[1].ParameterType.IsByRef);
object infoError = Activator.CreateInstance(errorType)!;
object?[] infoArgs = new object?[] { null, infoError };
WriteRuntimeMethodPointerSnapshot(assembly, runtimeMethodPointerOutput, runtimeMethodPointerFilters, repoRoot, scenario, "before-get-historian-info");
bool infoOk = (bool)getInfoMethod.Invoke(access, infoArgs)!;
Console.WriteLine(Serialize(new
{
Scenario = scenario,
GetHistorianInfoReturned = infoOk,
HistorianInfoType = getInfoMethod.GetParameters()[0].ParameterType.GetElementType()?.FullName,
HistorianInfo = infoArgs[0] is null ? null : SnapshotObject(infoArgs[0]!),
Error = SnapshotObject(infoArgs[1]!),
}));
return 0;
}
SetProperty(connectionArgs, "ReadOnly", !(IsWriteScenario(scenario) || IsRenameScenario(scenario)));
SetProperty(connectionArgs, "IntegratedSecurity", integratedSecurity);
SetProperty(connectionArgs, "ConnectionType", Enum.Parse(connectionType, IsEventScenario(scenario) ? "Event" : "Process"));
if (directConnection)
{
SetProperty(connectionArgs, "DirectConnection", true);
SetField(connectionArgs, "directConnection", true);
}
else if (openSuccess && status.ConnectedToServer && IsEventSendScenario(scenario))
{
// R2.1 capture: drive AddStreamedValue(HistorianEvent) and let instrument-wcf-*
// observe whether the event delivery rides the WCF MDAS path or the storage-engine
// pipe. Gated behind --event-send-confirm because it writes a real (clearly-marked)
// test event into the historian's event history.
if (!HasFlag(args, "--event-send-confirm"))
{
throw new InvalidOperationException(
"Event-send scenario writes a test event to the historian. Pass --event-send-confirm to proceed.");
}
Type historianEventType = GetType(assembly, "ArchestrA.HistorianEvent");
string eventTypeName = GetArg(args, "--event-type") ?? "User.Write";
string eventNamespace = GetArg(args, "--event-namespace") ?? "RetestSdkEventSend";
string eventSource = GetArg(args, "--event-source") ?? "RetestSdkEventSend";
object historianEvent = Activator.CreateInstance(historianEventType)!;
SetProperty(historianEvent, "ID", Guid.NewGuid());
SetProperty(historianEvent, "Type", eventTypeName);
SetProperty(historianEvent, "EventTime", DateTime.UtcNow);
SetProperty(historianEvent, "ReceivedTime", DateTime.UtcNow);
SetProperty(historianEvent, "Namespace", eventNamespace);
AddEventStringProperty(historianEvent, historianEventType, errorType, "Source", eventSource);
AddEventStringProperty(historianEvent, historianEventType, errorType, "TestMarker", "histsdk-R2.1-capture");
snapshots["HistorianEventBeforeSend"] = SnapshotObject(historianEvent);
// AddStreamedValue(HistorianEvent, out HistorianAccessError)
MethodInfo addEventMethod = accessType.GetMethods()
.First(m => m.Name == "AddStreamedValue"
&& m.GetParameters().Length == 2
&& m.GetParameters()[0].ParameterType == historianEventType);
WriteRuntimeMethodPointerSnapshot(assembly, runtimeMethodPointerOutput, runtimeMethodPointerFilters, repoRoot, scenario, "before-add-event");
object addEventError = Activator.CreateInstance(errorType)!;
object?[] addEventArgs = [historianEvent, addEventError];
bool addEventSuccess = (bool)addEventMethod.Invoke(access, addEventArgs)!;
addEventError = addEventArgs[1]!;
snapshots["AddEventError"] = SnapshotObject(addEventError);
rows.Add(new
{
Kind = "AddStreamedEvent",
Success = addEventSuccess,
Type = eventTypeName,
ErrorType = GetPropertyText(addEventError, "ErrorType"),
ErrorCode = GetPropertyText(addEventError, "ErrorCode"),
ErrorDescription = GetPropertyText(addEventError, "ErrorDescription"),
});
// Force the queued event onto the wire. CloseStorageConnection flushes all memory
// buffers to storage and starts forwarding snapshots.
MethodInfo? closeStorageMethod = accessType.GetMethod("CloseStorageConnection", new[] { errorType.MakeByRefType() });
if (closeStorageMethod is not null)
{
object closeStorageError = Activator.CreateInstance(errorType)!;
object?[] closeStorageArgs = [closeStorageError];
bool closeStorageSuccess = (bool)closeStorageMethod.Invoke(access, closeStorageArgs)!;
closeStorageError = closeStorageArgs[0]!;
rows.Add(new
{
Kind = "CloseStorageConnection",
Success = closeStorageSuccess,
ErrorDescription = GetPropertyText(closeStorageError, "ErrorDescription"),
});
}
// Let the background sender / store-forward flush push bytes before teardown.
int flushWait = int.TryParse(GetArg(args, "--event-send-flush-seconds"), out int fw) ? fw : 6;
if (flushWait > 0)
{
Thread.Sleep(TimeSpan.FromSeconds(flushWait));
}
}
else if (openSuccess && status.ConnectedToServer && IsEventScenario(scenario))
{
object query = accessType.GetMethod("CreateEventQuery", Type.EmptyTypes)!.Invoke(access, Array.Empty<object>())!;
Type queryType = query.GetType();
@@ -144,6 +600,41 @@ internal static class Program
SetProperty(queryArgs, "EventOrder", Enum.Parse(eventOrderType, "Ascending"));
snapshots["EventQueryArgsBeforeStart"] = SnapshotObject(queryArgs);
// R1.7 event-filter capture: --event-filter "Property:Op:Value" (repeatable via ';').
// Calls EventQuery.AddEventFilter(name, HistorianComparisionType, value, out err) so the
// filter predicate rides StartEventQuery's request buffer for instrument-wcf capture.
string? eventFilterSpec = GetArg(args, "--event-filter");
if (!string.IsNullOrWhiteSpace(eventFilterSpec))
{
Type comparisonType = GetType(assembly, "ArchestrA.HistorianComparisionType");
MethodInfo addFilterMethod = queryType.GetMethod("AddEventFilter",
new[] { typeof(string), comparisonType, typeof(object), errorType.MakeByRefType() })
?? throw new MissingMethodException("EventQuery.AddEventFilter");
foreach (string clause in eventFilterSpec!.Split(new[] { ';' }, StringSplitOptions.RemoveEmptyEntries))
{
string[] parts = clause.Split(new[] { ':' }, 3);
if (parts.Length < 3)
{
throw new ArgumentException($"--event-filter clause '{clause}' must be Property:Op:Value.");
}
object filterError = Activator.CreateInstance(errorType)!;
object?[] addFilterArgs = [parts[0], Enum.Parse(comparisonType, parts[1], ignoreCase: true), parts[2], filterError];
object addFilterResult = addFilterMethod.Invoke(query, addFilterArgs)!;
filterError = addFilterArgs[3]!;
rows.Add(new
{
Kind = "AddEventFilter",
Property = parts[0],
Op = parts[1],
Value = parts[2],
FilterId = addFilterResult,
ErrorDescription = GetPropertyText(filterError, "ErrorDescription"),
});
}
snapshots["EventQueryAfterAddFilter"] = SnapshotObject(query);
}
startError = Activator.CreateInstance(errorType)!;
MethodInfo startMethod = queryType.GetMethod("StartQuery", new[] { eventQueryArgsType, errorType.MakeByRefType() })
?? throw new MissingMethodException("EventQuery.StartQuery");
@@ -692,7 +1183,7 @@ internal static class Program
object queryArgs = Activator.CreateInstance(tagQueryArgsType)!;
SetProperty(queryArgs, "TagFilter", tagName);
SetProperty(queryArgs, "CacheTagInfo", true);
SetProperty(queryArgs, "RetrieveTagExtendedPropertyInfo", false);
SetProperty(queryArgs, "RetrieveTagExtendedPropertyInfo", retrieveExtendedProperties);
snapshots["TagQueryArgsBeforeStart"] = SnapshotObject(queryArgs);
startError = Activator.CreateInstance(errorType)!;
@@ -756,6 +1247,42 @@ internal static class Program
Tags = SummarizeTagList(tagInfoArgs[2])
});
}
// R1.5 capture: explicitly pull extended properties so the native client issues
// the GetTepByNm WCF op (only fires when --retrieve-extended-properties is set,
// which flips RetrieveTagExtendedPropertyInfo on the query args above).
if (retrieveExtendedProperties)
{
MethodInfo? getTepMethod = queryType.GetMethods().FirstOrDefault(method =>
method.Name == "GetTagExtendedPropertyInfo" && method.GetParameters().Length == 4);
if (getTepMethod is not null)
{
object tepError = Activator.CreateInstance(errorType)!;
object?[] tepArgs = [0u, requestedRows, null, tepError];
bool tepSuccess = false;
try
{
tepSuccess = (bool)getTepMethod.Invoke(query, tepArgs)!;
}
catch (TargetInvocationException ex)
{
rows.Add(new { Kind = "TagExtendedPropertyException", Detail = FormatException(ex.InnerException ?? ex) });
}
tepError = tepArgs[3]!;
rows.Add(new
{
Kind = "TagExtendedProperties",
Success = tepSuccess,
ErrorDescription = GetPropertyText(tepError, "ErrorDescription"),
ErrorCode = GetPropertyText(tepError, "ErrorCode"),
Groups = ToSerializableValue(tepArgs[2])
});
if (tepArgs[2] is not null)
{
snapshots["TagExtendedPropertyGroups"] = SnapshotObject(tepArgs[2]!);
}
}
}
}
MethodInfo? endMethod = queryType.GetMethod("EndQuery", new[] { errorType.MakeByRefType() });
@@ -789,6 +1316,26 @@ internal static class Program
{
SetProperty(queryArgs, "Resolution", resolutionTicks);
}
// Summary knobs — only set when explicitly supplied so plain reads are untouched.
if (valueSelectorName is not null)
{
Type valueSelectorType = GetType(assembly, "ArchestrA.HistorianValueSelector");
SetProperty(queryArgs, "ValueSelector", Enum.Parse(valueSelectorType, valueSelectorName, ignoreCase: true));
}
if (aggregationTypeName is not null)
{
Type aggregationType = GetType(assembly, "ArchestrA.HistorianAggregationType");
SetProperty(queryArgs, "AggregationType", Enum.Parse(aggregationType, aggregationTypeName, ignoreCase: true));
}
if (maxStates > 0)
{
// HistoryQueryArgs.MaxStates is a UInt16 on the native wrapper.
SetProperty(queryArgs, "MaxStates", checked((ushort)maxStates));
}
if (historyFilter is not null)
{
SetProperty(queryArgs, "Filter", historyFilter);
}
snapshots["QueryArgsBeforeStart"] = SnapshotObject(queryArgs);
startError = Activator.CreateInstance(errorType)!;
@@ -890,6 +1437,10 @@ internal static class Program
LookbackMinutes = lookbackMinutes,
RetrievalMode = retrievalModeName,
ResolutionTicks = resolutionTicks,
ValueSelector = valueSelectorName,
AggregationType = aggregationTypeName,
MaxStates = maxStates,
HistoryFilter = historyFilter,
StartUtc = startUtc.ToString("O"),
EndUtc = endUtc.ToString("O"),
OpenSuccess = openSuccess,
@@ -1225,12 +1776,113 @@ internal static class Program
return ex.GetType().Name + ": " + ex.Message;
}
private static bool IsRenameScenario(string scenario)
{
return scenario.Equals("rename", StringComparison.OrdinalIgnoreCase)
|| scenario.Equals("rename-tag", StringComparison.OrdinalIgnoreCase)
|| scenario.Equals("rename-tags", StringComparison.OrdinalIgnoreCase);
}
private static bool IsEventScenario(string scenario)
{
return scenario.Equals("event", StringComparison.OrdinalIgnoreCase)
|| scenario.Equals("events", StringComparison.OrdinalIgnoreCase);
}
/// <summary>
/// Event-SEND scenario (R2.1 capture): opens an Event connection in write mode
/// (ReadOnly=false) and drives <c>AddStreamedValue(HistorianEvent)</c> so the outgoing
/// event delivery can be captured. Distinct from the read-only event-query scenario.
/// </summary>
private static bool IsEventSendScenario(string scenario)
{
return scenario.Equals("event-send", StringComparison.OrdinalIgnoreCase)
|| scenario.Equals("send-event", StringComparison.OrdinalIgnoreCase);
}
/// <summary>Both event-query and event-send require an Event-type connection.</summary>
/// <summary>
/// Runtime-parameter scenario (R1.2 capture): opens a normal authenticated process
/// connection and calls <c>GetRuntimeParameter</c> so the WCF op + buffer format can be
/// captured. Read-only; not an event or write connection.
/// </summary>
private static bool IsRuntimeParamScenario(string scenario)
{
return scenario.Equals("runtime-param", StringComparison.OrdinalIgnoreCase)
|| scenario.Equals("runtime-parameter", StringComparison.OrdinalIgnoreCase);
}
/// <summary>
/// Historian-info scenario (R1.4 capture): opens a normal authenticated process connection and
/// calls <c>GetHistorianInfo(out HistorianInfo, out error)</c> so instrument-wcf-{write,read}message
/// can observe the WCF GETHI <c>pRequestBuff</c>/<c>pResponseBuff</c> that returns the full
/// 518-byte HISTORIAN_INFO struct (version@0 + EventStorageMode@514). Pure status read.
/// </summary>
private static bool IsHistorianInfoScenario(string scenario)
{
return scenario.Equals("historian-info", StringComparison.OrdinalIgnoreCase)
|| scenario.Equals("hist-info", StringComparison.OrdinalIgnoreCase)
|| scenario.Equals("gethi", StringComparison.OrdinalIgnoreCase);
}
/// <summary>
/// SQL-command scenario (R1.1 capture): opens a normal authenticated process connection and
/// calls <c>ExecuteSqlCommand</c> (Retr.ExeC + Retr.GetR) so the string-handle SQL surface
/// can be captured. Read-only benign query.
/// </summary>
private static bool IsExecSqlScenario(string scenario)
{
return scenario.Equals("exec-sql", StringComparison.OrdinalIgnoreCase)
|| scenario.Equals("sql", StringComparison.OrdinalIgnoreCase);
}
/// <summary>
/// Tag extended-properties scenario (R1.5 capture): opens a normal authenticated process
/// connection and calls the NAME-based <c>GetTagExtendedPropertiesByName</c> so the GetTepByNm
/// WCF op + tagNames request / extended-property response buffers can be captured. This
/// bypasses the QTB (StartTagQuery) path, which fails server-side here.
/// </summary>
private static bool IsTagExtendedPropertiesScenario(string scenario)
{
return scenario.Equals("tag-extended-properties", StringComparison.OrdinalIgnoreCase)
|| scenario.Equals("tag-tep", StringComparison.OrdinalIgnoreCase);
}
/// <summary>
/// Extended-property WRITE scenario (R1.11 capture): opens a write-enabled connection and calls
/// <c>AddTagExtendedProperties(TagExtendedPropertyGroupList, out err)</c> (and optionally
/// <c>DeleteTagExtendedPropertiesByName</c>) so instrument-wcf-writemessage can observe the
/// <c>AddTEx</c>/<c>DelTep</c> inBuff. Sandbox-guarded: the tag must start with RetestSdkWrite.
/// </summary>
private static bool IsAddTagExtendedPropertiesScenario(string scenario)
{
return scenario.Equals("add-tep", StringComparison.OrdinalIgnoreCase)
|| scenario.Equals("tep-write", StringComparison.OrdinalIgnoreCase)
|| scenario.Equals("extended-property-write", StringComparison.OrdinalIgnoreCase);
}
private static bool IsEventConnectionScenario(string scenario)
{
return IsEventScenario(scenario) || IsEventSendScenario(scenario);
}
/// <summary>
/// Adds a string property to a HistorianEvent via the public
/// <c>AddProperty(string name, string value, out HistorianAccessError error)</c> overload.
/// </summary>
private static void AddEventStringProperty(object historianEvent, Type historianEventType, Type errorType, string name, string value)
{
MethodInfo addProperty = historianEventType.GetMethods()
.First(m => m.Name == "AddProperty"
&& m.GetParameters().Length == 3
&& m.GetParameters()[0].ParameterType == typeof(string)
&& m.GetParameters()[1].ParameterType == typeof(string)
&& m.GetParameters()[2].ParameterType.IsByRef);
object propertyError = Activator.CreateInstance(errorType)!;
object?[] propertyArgs = [name, value, propertyError];
addProperty.Invoke(historianEvent, propertyArgs);
}
private static bool IsTagScenario(string scenario)
{
return scenario.Equals("tag", StringComparison.OrdinalIgnoreCase)