From 4da5287d0147bd0c7f2894473ab6e04986b4154f Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Sat, 20 Jun 2026 22:10:31 -0400 Subject: [PATCH] R1.2 GetRuntimeParameter + string-handle wall RESOLVED (handle-format bug) Execute HCAL roadmap R1.2 (GetRuntimeParameterAsync) end-to-end, and in doing so discover that the "string-handle wall" blocking R1.1/R1.4/R1.5/R1.6 was a handle FORMAT bug, not a missing native session/filter registration. R1.2 (shipped, live-verified): - Captured native GetRuntimeParameter -> WCF op aa/Stat/GETRP (string-handle op, GETHI's shape), via scripts/Capture-RuntimeParam.ps1 + instrument-wcf-{write,read}message. - HistorianRuntimeParameterProtocol serializes pRequestBuff (54 67 01 00 + uint nameCount + per-name uint charCount + UTF-16) and parses pResponseBuff (version + uint resultCount + CRetVariant 0x43 VT_BSTR + uint16 len + uint16 charCount + UTF-16). - IStatusServiceContract2.GetRuntimeParameter (GETRP) op; HistorianWcfStatusClient passes the Open2 storage-session GUID as the string handle, UPPERCASE. - Public HistorianClient.GetRuntimeParameterAsync(name) via the dialect. - Golden WcfRuntimeParameterProtocolTests + gated live test; returns HistorianVersion. String-handle wall RESOLVED (proven, public APIs deferred): - The Open2 storage GUID works as the string handle when sent UPPERCASE (ToString("D").ToUpperInvariant()); earlier "blocked" probes used lowercase. - Live-probed GETHI (R1.4) -> returns data; ExeC (R1.1) -> Retr.GetV prime -> ExeC -> GetR returns a BinaryFormatter-serialized .NET DataTable. Gated StringHandleProbeDiagnosticTests + scripts/Capture-ExecSql.ps1 + exec-sql harness scenario. - Docs flipped: wcf-string-handle-wall.md RESOLVED banner; roadmap R1.1/R1.4 reachable, R1.5/R1.6 likely; wcf-status-localhost.md GETRP section. - R1.1/R1.4 public APIs NOT shipped: ExeC needs a GetR paging loop + a BinaryFormatter- stream parser (BinaryFormatter is removed from .NET 10); GETHI full-info struct needs its own capture. 223 unit tests pass; gated live tests green against the local 2020 Historian. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC --- docs/plans/hcal-roadmap.md | 34 ++-- .../wcf-status-localhost.md | 23 +++ .../wcf-string-handle-wall.md | 56 +++++++ scripts/Capture-ExecSql.ps1 | 91 ++++++++++ scripts/Capture-RuntimeParam.ps1 | 105 ++++++++++++ scripts/decode-runtime-param-capture.py | 128 ++++++++++++++ src/AVEVA.Historian.Client/HistorianClient.cs | 12 ++ .../Protocol/Historian2020ProtocolDialect.cs | 7 + .../Wcf/Contracts/IStatusServiceContract2.cs | 8 + .../Wcf/HistorianRuntimeParameterProtocol.cs | 104 ++++++++++++ .../Wcf/HistorianWcfStatusClient.cs | 56 +++++++ .../HistorianClientIntegrationTests.cs | 23 +++ .../StringHandleProbeDiagnosticTests.cs | 156 ++++++++++++++++++ .../WcfRuntimeParameterProtocolTests.cs | 58 +++++++ .../Program.cs | 108 +++++++++++- 15 files changed, 953 insertions(+), 16 deletions(-) create mode 100644 scripts/Capture-ExecSql.ps1 create mode 100644 scripts/Capture-RuntimeParam.ps1 create mode 100644 scripts/decode-runtime-param-capture.py create mode 100644 src/AVEVA.Historian.Client/Wcf/HistorianRuntimeParameterProtocol.cs create mode 100644 tests/AVEVA.Historian.Client.Tests/StringHandleProbeDiagnosticTests.cs create mode 100644 tests/AVEVA.Historian.Client.Tests/WcfRuntimeParameterProtocolTests.cs diff --git a/docs/plans/hcal-roadmap.md b/docs/plans/hcal-roadmap.md index 989df6d..9c8558c 100644 --- a/docs/plans/hcal-roadmap.md +++ b/docs/plans/hcal-roadmap.md @@ -39,8 +39,12 @@ HCAL replacement, built on the **2023 R2 gRPC transport**. Derived from > reachable). The reachable **`uint`-handle** items are now **DONE**: ~~R1.8/R1.9 StartQuery > summary/state modes~~ (resolved = existing `ReadAggregateAsync`) and ~~R1.7 event filters~~ > (✅ 2026-06-20 — `ReadEventsAsync(…, HistorianEventFilter)`, live-honored). M2 event send is -> also done (✅ WCF `AddS2`). Everything string-handle still waits on one RE target: the native -> session/filter registration. +> also done (✅ WCF `AddS2`). **R1.2 `GetRuntimeParameterAsync` is also done** (✅ 2026-06-20, +> `aa/Stat/GETRP`, live-verified) — notably a *string-handle* op that punches through the wall +> using the Open2 storage-session GUID as an **uppercase** string handle, which is a strong lead +> that the GETHI/ExeC failures are (at least partly) a handle-*format* issue rather than only a +> missing native registration. **Cheap high-value follow-up: retry GETHI/ExeC with the uppercased +> storage GUID** before assuming the registration wall (see `wcf-string-handle-wall.md` §Update). ## Guiding principles @@ -86,25 +90,25 @@ read/browse/status surface is Windows-free and the gRPC stack is the default pat ### 1a. Trivial (XS–S each, no new payload format) | ID | Capability | gRPC op | Notes | |---|---|---|---| -| ~~R1.1~~ | ~~`ExecuteSqlCommandAsync`~~ | `Retrieval.ExecuteSqlCommand` | ⚠ **Blocked on 2020 WCF.** Live-probed 2026-06-20: `ExeC` returns native error type 4 / code **51 (InvalidParameter)** for every handle variant — same unmapped *native session/filter registration* prerequisite that blocks `StartTagQuery`/`QueryTag` (see `implementation-status.md` lines ~982, ~1404). Needs that registration RE'd, or a 2023 R2 gRPC server. Do not wire via guessed calls. | -| R1.2 | `GetRuntimeParameterAsync` | `Status.GetRuntimeParameter` | mirror `GetSystemParameter` | +| R1.1 | `ExecuteSqlCommandAsync` | `Retrieval.ExecuteSqlCommand` (`ExeC`+`GetR`) | ✅ **REACHABLE (2026-06-20, live-probed).** The earlier "code 51 blocked" verdict was a handle-**format** bug — `ExeC` succeeds with the Open2 storage GUID sent **uppercase** (`ToString("D").ToUpperInvariant()`). Chain: `Retr.GetV` prime → `ExeC(handle, sqlString, option=0, ref queryHandle)` → `GetR(handle, queryHandle, ref sequence)` returns the result as a **BinaryFormatter-serialized .NET DataTable**. Proven by `StringHandleProbeDiagnosticTests` + `scripts/Capture-ExecSql.ps1`. **Public API not yet shipped** — needs a `GetR` continuation loop + a custom BinaryFormatter-stream parser (BinaryFormatter is removed from .NET 10, so a DataTable can't just be deserialized). | +| ~~R1.2~~ | ~~`GetRuntimeParameterAsync`~~ | `Status.GetRuntimeParameter` (`aa/Stat/GETRP`) | ✅ **DONE (2026-06-20), live-verified.** Captured (`scripts/Capture-RuntimeParam.ps1`): GETRP is a **`string`-handle** op (GETHI's shape), but reachable from the managed client using the Open2 storage-session GUID as an **uppercase** string handle (`ToString("D").ToUpperInvariant()`). Returns `HistorianVersion` = `20,0,000,000` live. pRequestBuff = `54 67 01 00` + uint nameCount + per-name(uint charCount + UTF-16); pResponseBuff = version + uint resultCount + CRetVariant(`0x43` VT_BSTR + uint16 len + uint16 charCount + UTF-16). Single string-valued param only (multi-name framing inferred, not captured). Shipped: `HistorianClient.GetRuntimeParameterAsync(name)`; golden `WcfRuntimeParameterProtocolTests`. **Note:** GETRP punching through the string-handle wall with the uppercase storage GUID is a strong lead that GETHI/ExeC may be a handle-*format* issue — see `wcf-string-handle-wall.md` §Update. | | ~~R1.3~~ | ~~`GetServerTimeZoneAsync`~~ | `Status.GetSystemTimeZoneName` | ⚠ **gRPC/2023R2-only.** Verified 2026-06-20: over **2020 WCF** this op is a stub (rc=0, empty value) in the `GetServerTime` family — not shippable here. Build+verify only against a live 2023 R2 server. See `docs/reverse-engineering/wcf-status-localhost.md`. | -> ⛔ **String-handle wall (2026-06-20).** R1.4/R1.5/R1.6 (and R1.1) are **all blocked on 2020 -> WCF** for the *same* reason: their ops take a **`string` GUID handle** and require an unmapped -> native session/filter registration. Probed live — GETHI returns code 1 for the exact native -> request shape across 5 handle formats + Stat.GetV priming; ExeC returns code 51. The proven -> surface uses **`uint`-handle** ops only. **One RE target — the native string-handle session -> registration — unblocks this whole sub-milestone.** Full analysis: -> `docs/reverse-engineering/wcf-string-handle-wall.md`. R1.8/R1.9 (StartQuery summary/state modes) -> are `uint`-handle and remain reachable on 2020 WCF. +> ✅ **String-handle "wall" RESOLVED (2026-06-20) — it was a handle-FORMAT bug.** R1.4/R1.5/R1.6 +> (and R1.1) take a **`string` GUID handle**; the earlier "code 1/51 blocked" verdict came from +> passing the Open2 storage GUID in .NET's default **lowercase**. Sent **uppercase** +> (`storageSessionId.ToString("D").ToUpperInvariant()`) the same handle works: **GETRP** (R1.2, +> shipped), **GETHI** (R1.4) and **ExeC** (R1.1) are all live-verified reachable. R1.5/R1.6 +> (GetTepByNm family) + QTB/QTG are very likely reachable the same way (not yet individually +> re-probed). Full analysis: `docs/reverse-engineering/wcf-string-handle-wall.md` (RESOLVED banner). +> R1.8/R1.9 (StartQuery summary/state modes) are `uint`-handle and were already reachable. ### 1b. Bounded (decode one `bytes` payload; S–M each) | ID | Capability | gRPC op | Payload to decode | Depends | |---|---|---|---|---| -| ~~R1.4~~ | `GetHistorianInfoAsync` | `Status.GetHistorianInfo` | ⛔ **string-handle wall** — GETHI returns code 1 on 2020 WCF (all handle/priming variants). GETHI buffer incl. `EventStorageMode`@514. | string-handle RE | -| ~~R1.5~~ | Extended-property **read** | `Retrieval.GetTagExtendedPropertiesFromName` | ⛔ **string-handle wall** (GetTepByNm takes `string handle`). TEP result buffer. | string-handle RE | -| ~~R1.6~~ | Localized-property **read** | `Retrieval.GetTagLocalizedPropertiesFromName` | ⛔ **string-handle wall** (same family). | string-handle RE | +| R1.4 | `GetHistorianInfoAsync` | `Status.GetHistorianInfo` (`GETHI`) | ✅ **REACHABLE (2026-06-20, live-probed)** via the uppercase storage GUID — `GETHI` returns data (`StringHandleProbeDiagnosticTests`). The version-keyed request returns `uint charCount + UTF-16`; the full info struct (incl. `EventStorageMode`@514) needs its own request capture. **Public API not yet shipped.** | uppercase string handle | +| R1.5 | Extended-property **read** | `Retrieval.GetTagExtendedPropertiesFromName` | 🟡 **Likely reachable** via uppercase string handle (GetTepByNm family) — not yet individually re-probed. TEP result buffer. | uppercase string handle | +| R1.6 | Localized-property **read** | `Retrieval.GetTagLocalizedPropertiesFromName` | 🟡 **Likely reachable** (same family) — not yet re-probed. | uppercase string handle | | ~~R1.7~~ | Event **filters** | filter bytes in `Retrieval.StartEventQuery` | ✅ **DONE (2026-06-20), live-honored.** `ReadEventsAsync(start, end, HistorianEventFilter)`. The filter rides `StartEventQuery`'s `pRequestBuff` (captured via `EventQuery.AddEventFilter` + instrument-wcf-writemessage; Equal vs Contains diffed to isolate the op). Filter block: `ushort 0 + uint filterCount + uint condCount + uint nameLen + name(UTF-16) + uint 1 + ushort op + uint 1 + value(0x09-len-0x00 compact-ASCII) + byte 0`. **REAL, not inert** (a non-matching predicate returns 0 events; matching returns the subset). Single string-valued predicate only; multi-filter (OR) / multi-condition (AND via `AddEventFilterCondition`) framing not yet fully captured. See `HistorianEventFilter`, golden `WcfEventQueryProtocolTests`. | — | | R1.8 | Analog-summary query | `Retrieval.StartQuery` (summary mode) | summary row layout — **`uint`-handle, reachable. Scoped + decode targets located** (`CAnalogSummaryValue.UnpackFromValueBuffer`, fields Min/Max/First/Last/ValueCount/Integral/…). Plan: [`r1.8-r1.9-summary-queries.md`](r1.8-r1.9-summary-queries.md) | — | | R1.9 | State-summary query | `Retrieval.StartQuery` (state mode) | state-summary row layout — **`uint`-handle, reachable. Scoped** (`CStateSummaryStruct`: MinContained/MaxContained/TotalContained/PartialStart/PartialEnd/StateEntryCount). Plan: [`r1.8-r1.9-summary-queries.md`](r1.8-r1.9-summary-queries.md) | — | diff --git a/docs/reverse-engineering/wcf-status-localhost.md b/docs/reverse-engineering/wcf-status-localhost.md index d0bd776..03de8ed 100644 --- a/docs/reverse-engineering/wcf-status-localhost.md +++ b/docs/reverse-engineering/wcf-status-localhost.md @@ -48,3 +48,26 @@ Interpretation: - **`GetServerTimeZoneAsync` (roadmap R1.3) is NOT a trivial WCF op on 2020** — it is a stub returning empty. Do not ship it over the 2020 WCF transport. Deliver it only against a live 2023 R2 gRPC server. Reclassified in `docs/plans/hcal-roadmap.md`. + +## GETRP / GetRuntimeParameter (roadmap R1.2) — DONE, live-verified 2026-06-20 + +Captured the native `HistorianAccess.GetRuntimeParameter(List, out List)` +WCF traffic with `scripts/Capture-RuntimeParam.ps1` (instrument-wcf-{write,read}message). +Findings: + +- The WCF op is **`aa/Stat/GETRP`** — `bool GETRP(string handle, byte[] pRequestBuff, + out byte[] pResponseBuff, out byte[] errorBuffer)`, i.e. the **same string-handle + + request/response-buffer shape as GETHI**, *not* the simple `GetSystemParameter(uint, string)` + shape the roadmap originally assumed. +- The `string handle` is the **Open2 storage-session GUID** (the value + `ParseOpenConnectionResponse` reads from `outBuff[5..21]`), sent **UPPERCASE, dash-separated, + no braces** (`ToString("D").ToUpperInvariant()`). +- Unlike GETHI (which the earlier probe found blocked), **GETRP succeeds from the pure-managed + client** with that handle: `GetRuntimeParameter("HistorianVersion")` → `20,0,000,000`. +- `pRequestBuff` = `54 67 01 00` (sig+version) + uint nameCount + per name(uint charCount + + UTF-16LE). `pResponseBuff` = version(1) + uint resultCount + CRetVariant(`0x43` VT_BSTR + + uint16 payloadLen + uint16 charCount + UTF-16LE). + +Shipped as `HistorianClient.GetRuntimeParameterAsync(name)`. See +`HistorianRuntimeParameterProtocol`, golden `WcfRuntimeParameterProtocolTests`, and the +handle-format lead in `wcf-string-handle-wall.md` §Update (retry GETHI/ExeC uppercased). diff --git a/docs/reverse-engineering/wcf-string-handle-wall.md b/docs/reverse-engineering/wcf-string-handle-wall.md index 63be01f..6fdd4fb 100644 --- a/docs/reverse-engineering/wcf-string-handle-wall.md +++ b/docs/reverse-engineering/wcf-string-handle-wall.md @@ -1,10 +1,35 @@ # The 2020 WCF string-handle wall (2026-06-20) +> ## ✅✅ RESOLVED (2026-06-20): the "wall" was a handle-FORMAT bug, not a registration wall. +> +> The string-handle ops are reachable from the pure-managed client after all. The Open2 +> storage-session GUID must be passed as the `string handle` **UPPERCASE, dash-separated, +> no braces** — `storageSessionId.ToString("D").ToUpperInvariant()`. The earlier probes that +> "proved" the wall passed the GUID in .NET's default **lowercase** `ToString("D")`, which the +> server's session table does not match. Live-verified end-to-end against the local 2020 server: +> - **GETRP** (R1.2) → returns the runtime `HistorianVersion` (shipped). +> - **GETHI** (R1.4) → `returned=True`, returns the version buffer (`0C000000` + UTF-16 "20,0,000,000"). +> - **ExeC** (R1.1) → `returned=True`, `Retr.GetV` prime + `ExeC("SELECT 1 AS ProbeValue", option=0)` +> yields `queryHandle`, then `GetR(handle, queryHandle, sequence=0)` returns a 1232-byte result = +> a **BinaryFormatter-serialized .NET DataTable** (stream header `…System.Data, Version=4.0.0.0…`). +> +> Probes: gated `StringHandleProbeDiagnosticTests` (GETHI + ExeC). Captures: +> `scripts/Capture-RuntimeParam.ps1`, `scripts/Capture-ExecSql.ps1`. The handle for ExeC/GetR is the +> **same** Open2 storage-session GUID (confirmed = `outBuff[5..21]`). The original analysis below is +> retained for history; treat its "blocked" conclusions as **superseded** — the only missing piece +> was the uppercase format. R1.5/R1.6 (GetTepByNm family) and QTB/QTG are very likely reachable the +> same way but have not yet been individually re-probed. + +--- + Live-probing the local **Historian 2020** (WCF, port 32568) for HCAL roadmap M1 surfaced a clean structural boundary on what the pure-managed client can call. It explains why R1.1/R1.4/R1.5 all fail and identifies the single RE target that unblocks the rest of the M1 read surface. +> ⚠️ **Superseded — see the RESOLVED banner above.** The boundary below is real *only* when the +> handle is sent lowercase. With the uppercased storage GUID the string-handle ops succeed. + ## The dichotomy Retrieval/Status/History ops split by the **type of their first (handle) parameter**: @@ -56,3 +81,34 @@ once and the whole family unlocks. Until then, the alternatives are: Do **not** ship any string-handle op via guessed calls (project discipline: "leave them throwing until evidence supports an implementation"). + +## ⚠️ Update (2026-06-20): GETRP punches through — the wall is not absolute + +Roadmap **R1.2 `GetRuntimeParameterAsync`** turned out to be a **`string`-handle op** +(`aa/Stat/GETRP(string handle, byte[] pRequestBuff) → (bool, byte[] pResponseBuff, +byte[] errorBuffer)`) — the **same shape as GETHI**, and in the same native session it +uses the **same handle GUID** as GETHI (confirmed: the GUID equals the Open2 `outBuff` +storage-session id at `[5..21]`, the value the managed `ParseOpenConnectionResponse` +already extracts as `StorageSessionId`). + +Yet GETRP **works from the pure-managed client** — live-verified, returns the runtime +`HistorianVersion` value `20,0,000,000`. The only material difference from the failed +GETHI probe is the **handle string format**: the native client sends the GUID +**UPPERCASE, dash-separated, no braces** (format example +`XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX`, all hex upper), i.e. +`storageSessionId.ToString("D").ToUpperInvariant()`. `.NET Guid.ToString("D")` is +lowercase, so a probe that passed the GUID without upcasing would not byte-match what +the server's session table is keyed on. + +**Implication / open lead (not yet retested):** the GETHI/ExeC/QTB/QTG family failures +may be (at least partly) a **handle-format** issue, not (only) a missing native +registration step. The highest-value cheap follow-up is to **re-probe GETHI and ExeC +with the uppercased storage-session GUID** before assuming the registration wall. If +they also return data, the "wall" collapses to a formatting bug and R1.4/R1.5/R1.6/R1.1 +may be reachable without any new RE. This has **not** been done yet — do not reclassify +those items until it is. GETRP is shipped because it was directly captured + live-verified +end-to-end; the rest remain `ProtocolEvidenceMissingException`/unprobed until tested. + +See `HistorianRuntimeParameterProtocol`, `IStatusServiceContract2.GetRuntimeParameter`, +golden `WcfRuntimeParameterProtocolTests`, and capture tooling +`scripts/Capture-RuntimeParam.ps1` + `scripts/decode-runtime-param-capture.py`. diff --git a/scripts/Capture-ExecSql.ps1 b/scripts/Capture-ExecSql.ps1 new file mode 100644 index 0000000..7a08c77 --- /dev/null +++ b/scripts/Capture-ExecSql.ps1 @@ -0,0 +1,91 @@ +<# +.SYNOPSIS + Captures the native AVEVA client's ExecuteSqlCommand wire traffic (HCAL roadmap R1.1) so the + Retr.ExeC + Retr.GetR string-handle SQL surface (op names, handle format, command/option + encoding, Retr priming, GetR result byte stream) can be decoded instead of guessed. + +.DESCRIPTION + Drives the .NET-Framework NativeTraceHarness `exec-sql` scenario against the live Historian + with an IL-rewritten copy of aahClientManaged.dll whose ClientMessageEncoder.WriteMessage AND + ReadMessage are instrumented to log every MDAS body. Read-only benign query. + +.NOTES + Artifacts are diagnostic and gitignored. Sanitize before copying into docs/ -- never commit raw + capture NDJSON, credentials, hostnames, or customer tag names. +#> +[CmdletBinding()] +param( + [string]$ServerName = "localhost", + [int]$TcpPort = 32568, + [string]$Sql = "SELECT 1 AS ProbeValue", + [string]$SqlOption = "ExecuteRecord", + [string]$Configuration = "Debug" +) + +$ErrorActionPreference = "Stop" +$repoRoot = Split-Path -Parent $PSScriptRoot +Set-Location $repoRoot + +$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj" +$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj" +$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj" + +$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-exec-sql" +$currentCopy = Join-Path $captureDir "current-copy" +$instrDll = Join-Path $captureDir "aahClientManaged.dll" +$capturePath = Join-Path $captureDir "exec-sql-capture-latest.ndjson" + +Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan +dotnet build $reProj -c $Configuration --nologo -v q | Out-Null +dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null +dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null + +$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") ` + -Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName +if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." } + +Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan +New-Item -ItemType Directory -Force -Path $captureDir | Out-Null +$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll" +dotnet run --no-build -c $Configuration --project $reProj -- ` + instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null +dotnet run --no-build -c $Configuration --project $reProj -- ` + instrument-wcf-readmessage $writeOnly $instrDll | Out-Null + +Write-Host "== Staging current-copy ==" -ForegroundColor Cyan +robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null +Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll") +Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll") + +$harnessDll = Join-Path $currentCopy "aahClientManaged.dll" +if (Test-Path $capturePath) { Remove-Item -Force $capturePath } +$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath + +Write-Host "== Capturing exec-sql ==" -ForegroundColor Green +$harnessArgs = @( + "--scenario", "exec-sql", + "--server-name", $ServerName, + "--tcp-port", "$TcpPort", + "--sql", $Sql, + "--sql-option", $SqlOption, + "--current-dir", $currentCopy, + "--managed-dll-path", $harnessDll +) + +$harnessJson = $null +try { + $prevEap = $ErrorActionPreference + $ErrorActionPreference = "Continue" + $harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1 +} catch { + Write-Host " (exec-sql raised: $($_.Exception.Message))" -ForegroundColor Yellow +} finally { + $ErrorActionPreference = $prevEap +} + +Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue + +$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 } +Write-Host "`n== Capture summary ==" -ForegroundColor Cyan +Write-Host " -> $recCount records -> $capturePath" +$harnessJson | Select-Object -Last 6 diff --git a/scripts/Capture-RuntimeParam.ps1 b/scripts/Capture-RuntimeParam.ps1 new file mode 100644 index 0000000..40f7528 --- /dev/null +++ b/scripts/Capture-RuntimeParam.ps1 @@ -0,0 +1,105 @@ +<# +.SYNOPSIS + Captures the native AVEVA client's GetRuntimeParameter wire traffic (HCAL roadmap R1.2) + so the WCF op name, handle type (uint vs the string-handle wall), and the + btRequest/btResponse buffer format can be decoded instead of guessed. + +.DESCRIPTION + Drives the .NET-Framework NativeTraceHarness's `runtime-param` scenario against the live + Historian with an IL-rewritten copy of aahClientManaged.dll whose + ClientMessageEncoder.WriteMessage AND ReadMessage are instrumented to log every MDAS body + (the same pipeline that produced every other proven request/response shape). The harness + opens a normal authenticated process connection and calls + HistorianAccess.GetRuntimeParameter(List names, out List results, out err). + + Decode with scripts/decode-runtime-param-capture.py: locate the WCF.WriteMessage.Body + whose op carries the parameter name(s) -> that is the GetRuntimeParameter request; read + off the SOAP action / op name, the leading handle param, and the btRequest layout. The + paired WCF.ReadMessage.Body is the btResponse (the CRetVariant value list). + +.NOTES + Read-only status call; no data is written. Artifacts are diagnostic and gitignored. + Sanitize before copying anything into docs/ -- never commit raw capture NDJSON, + credentials, hostnames, or customer tag names. +#> +[CmdletBinding()] +param( + [string]$ServerName = "localhost", + [int]$TcpPort = 32568, + # Semicolon-separated runtime parameter names. HistorianVersion is a known-good name + # (returns the server version string) so the response decode has a real value. + [string]$Names = "HistorianVersion", + [string]$Configuration = "Debug" +) + +$ErrorActionPreference = "Stop" +$repoRoot = Split-Path -Parent $PSScriptRoot +Set-Location $repoRoot + +$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj" +$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj" +$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj" + +$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-runtime-param" +$currentCopy = Join-Path $captureDir "current-copy" +$instrDll = Join-Path $captureDir "aahClientManaged.dll" +$capturePath = Join-Path $captureDir "runtime-param-capture-latest.ndjson" + +Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan +dotnet build $reProj -c $Configuration --nologo -v q | Out-Null +dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null +dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null + +$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") ` + -Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName +if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." } + +Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan +New-Item -ItemType Directory -Force -Path $captureDir | Out-Null +# Chain via a distinct intermediate file (reading+writing the same path drops the second +# hook on the mixed-mode native image). Final dll carries both hooks with distinct Phase +# strings: WCF.WriteMessage.Body and WCF.ReadMessage.Body. +$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll" +dotnet run --no-build -c $Configuration --project $reProj -- ` + instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null +dotnet run --no-build -c $Configuration --project $reProj -- ` + instrument-wcf-readmessage $writeOnly $instrDll | Out-Null + +Write-Host "== Staging current-copy ==" -ForegroundColor Cyan +robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null +Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll") +Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll") + +$harnessDll = Join-Path $currentCopy "aahClientManaged.dll" +if (Test-Path $capturePath) { Remove-Item -Force $capturePath } +$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath + +Write-Host "== Capturing runtime-param ==" -ForegroundColor Green +$harnessArgs = @( + "--scenario", "runtime-param", + "--server-name", $ServerName, + "--tcp-port", "$TcpPort", + "--runtime-param-names", $Names, + "--current-dir", $currentCopy, + "--managed-dll-path", $harnessDll +) + +$harnessJson = $null +try { + $prevEap = $ErrorActionPreference + $ErrorActionPreference = "Continue" + $harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1 +} catch { + Write-Host " (runtime-param raised: $($_.Exception.Message))" -ForegroundColor Yellow +} finally { + $ErrorActionPreference = $prevEap +} + +Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue + +$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 } +Write-Host "`n== Capture summary ==" -ForegroundColor Cyan +Write-Host " -> $recCount records -> $capturePath" +Write-Host "Harness output (GetRuntimeParameterReturned / Results):" -ForegroundColor Cyan +$harnessJson | Select-Object -Last 20 +Write-Host "`nDecode with: python scripts\decode-runtime-param-capture.py" -ForegroundColor Cyan diff --git a/scripts/decode-runtime-param-capture.py b/scripts/decode-runtime-param-capture.py new file mode 100644 index 0000000..6c356af --- /dev/null +++ b/scripts/decode-runtime-param-capture.py @@ -0,0 +1,128 @@ +"""Decode the GetRuntimeParameter WCF request/response (HCAL R1.2). + +Reads the chained WriteMessage+ReadMessage capture produced by +scripts/Capture-RuntimeParam.ps1 and locates the GetRuntimeParameter exchange by +searching every MDAS body for the parameter name (UTF-16) on the request side and the +returned value on the response side. Dumps the surrounding bytes so the op name, the +leading handle parameter, and the btRequest/btResponse buffer layout can be read off. + +Output is diagnostic. Sanitize before copying into docs/. +""" +import base64 +import json +import sys +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parent.parent +CAPDIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-runtime-param" +CAP = CAPDIR / "runtime-param-capture-latest.ndjson" + +# Markers we expect on the wire for the default "HistorianVersion" capture. +NAME = "HistorianVersion" +NAME_U16 = NAME.encode("utf-16-le") +NAME_ASCII = NAME.encode("ascii") +VALUE = "20,0,000,000" # server runtime "HistorianVersion" value (version-shaped, not secret) +VALUE_U16 = VALUE.encode("utf-16-le") +VALUE_ASCII = VALUE.encode("ascii") + + +def hexdump(label, buf, base=0): + print(f"=== {label}: {len(buf)} bytes ===") + for off in range(0, len(buf), 16): + c = buf[off:off + 16] + hp = " ".join(f"{x:02X}" for x in c) + ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c) + print(f" {base + off:04X} {hp:<48} |{ap}|") + print() + + +def ascii_strings(buf, minlen=3): + out, cur, start = [], [], 0 + for i, x in enumerate(buf): + if 32 <= x < 127: + if not cur: + start = i + cur.append(chr(x)) + else: + if len(cur) >= minlen: + out.append((start, "".join(cur))) + cur = [] + if len(cur) >= minlen: + out.append((start, "".join(cur))) + return out + + +def u16_strings(buf, minlen=3): + out, i = [], 0 + while i < len(buf) - 1: + j, chars = i, [] + while j < len(buf) - 1 and 32 <= buf[j] < 127 and buf[j + 1] == 0: + chars.append(chr(buf[j])) + j += 2 + if len(chars) >= minlen: + out.append((i, "".join(chars))) + i = j + else: + i += 1 + return out + + +def main() -> int: + if not CAP.exists(): + print(f"Missing capture: {CAP}\nRun scripts/Capture-RuntimeParam.ps1 first.") + return 1 + + records = [] + for line in CAP.open(encoding="utf-8-sig"): + if line.strip(): + records.append(json.loads(line)) + + print(f"== {len(records)} MDAS bodies captured ==") + for idx, rec in enumerate(records): + body = base64.b64decode(rec["Base64"]) + flags = [] + if NAME_U16 in body or NAME_ASCII in body: + flags.append("NAME") + if VALUE_U16 in body or VALUE_ASCII in body: + flags.append("VALUE") + # The WS-Addressing action is the most reliable op label; show any string that + # looks like an op (contains a slash or is short and capitalized). + print(f" [{idx:02d}] {rec.get('Phase'):26s} len={len(body):5d} {','.join(flags)}") + + def find(predicate): + hits = [] + for idx, rec in enumerate(records): + body = base64.b64decode(rec["Base64"]) + if predicate(rec, body): + hits.append((idx, rec, body)) + return hits + + print("\n== Request candidate(s): WriteMessage bodies containing the NAME ==") + for idx, rec, body in find(lambda r, b: r.get("Phase") == "WCF.WriteMessage.Body" + and (NAME_U16 in b or NAME_ASCII in b)): + hexdump(f"[{idx}] WriteMessage", body) + print(" UTF-16 strings:") + for off, s in u16_strings(body): + print(f" 0x{off:04X} {s!r}") + print(" ASCII strings:") + for off, s in ascii_strings(body): + print(f" 0x{off:04X} {s!r}") + print() + + print("\n== Response candidate(s): ReadMessage bodies containing the VALUE ==") + for idx, rec, body in find(lambda r, b: r.get("Phase") == "WCF.ReadMessage.Body" + and (VALUE_U16 in b or VALUE_ASCII in b)): + hexdump(f"[{idx}] ReadMessage", body) + print(" UTF-16 strings:") + for off, s in u16_strings(body): + print(f" 0x{off:04X} {s!r}") + print(" ASCII strings:") + for off, s in ascii_strings(body): + print(f" 0x{off:04X} {s!r}") + print() + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/src/AVEVA.Historian.Client/HistorianClient.cs b/src/AVEVA.Historian.Client/HistorianClient.cs index 41e5962..2a24954 100644 --- a/src/AVEVA.Historian.Client/HistorianClient.cs +++ b/src/AVEVA.Historian.Client/HistorianClient.cs @@ -154,6 +154,18 @@ public sealed class HistorianClient : IAsyncDisposable return _protocol.GetSystemParameterAsync(name, cancellationToken); } + /// + /// Reads a named Historian runtime parameter (the live server state surface, + /// distinct from the configuration ). Returns the + /// string value, or null when the server reports no value. Single string-valued parameters + /// only (the evidence-backed surface); see HistorianRuntimeParameterProtocol. + /// + public Task GetRuntimeParameterAsync(string name, CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(name); + return _protocol.GetRuntimeParameterAsync(name, cancellationToken); + } + /// /// Creates or updates the named tag in the Historian Runtime database via /// EnsureTags2. Currently only is diff --git a/src/AVEVA.Historian.Client/Protocol/Historian2020ProtocolDialect.cs b/src/AVEVA.Historian.Client/Protocol/Historian2020ProtocolDialect.cs index 429ccd8..eb9d70e 100644 --- a/src/AVEVA.Historian.Client/Protocol/Historian2020ProtocolDialect.cs +++ b/src/AVEVA.Historian.Client/Protocol/Historian2020ProtocolDialect.cs @@ -67,6 +67,13 @@ internal sealed class Historian2020ProtocolDialect return Wcf.HistorianWcfStatusClient.GetSystemParameterAsync(_options, name, cancellationToken); } + public Task GetRuntimeParameterAsync(string name, CancellationToken cancellationToken) + { + cancellationToken.ThrowIfCancellationRequested(); + ArgumentException.ThrowIfNullOrWhiteSpace(name); + return Wcf.HistorianWcfStatusClient.GetRuntimeParameterAsync(_options, name, cancellationToken); + } + private static async IAsyncEnumerable Missing( string operation, [System.Runtime.CompilerServices.EnumeratorCancellation] CancellationToken cancellationToken) diff --git a/src/AVEVA.Historian.Client/Wcf/Contracts/IStatusServiceContract2.cs b/src/AVEVA.Historian.Client/Wcf/Contracts/IStatusServiceContract2.cs index c940ce1..3464908 100644 --- a/src/AVEVA.Historian.Client/Wcf/Contracts/IStatusServiceContract2.cs +++ b/src/AVEVA.Historian.Client/Wcf/Contracts/IStatusServiceContract2.cs @@ -29,6 +29,14 @@ internal interface IStatusServiceContract2 : IStatusServiceContract [MessageParameter(Name = "pResponseBuff")] out byte[] responseBuffer, out byte[] errorBuffer); + [OperationContract(Name = "GETRP")] + [return: MarshalAs(UnmanagedType.U1)] + bool GetRuntimeParameter( + string handle, + [MessageParameter(Name = "pRequestBuff")] byte[] requestBuffer, + [MessageParameter(Name = "pResponseBuff")] out byte[] responseBuffer, + out byte[] errorBuffer); + [OperationContract(Name = "PNGS")] [return: MarshalAs(UnmanagedType.U1)] bool PingServer(string handle, string pipeName, uint timeout, ref byte[] errorBuffer); diff --git a/src/AVEVA.Historian.Client/Wcf/HistorianRuntimeParameterProtocol.cs b/src/AVEVA.Historian.Client/Wcf/HistorianRuntimeParameterProtocol.cs new file mode 100644 index 0000000..aeaa709 --- /dev/null +++ b/src/AVEVA.Historian.Client/Wcf/HistorianRuntimeParameterProtocol.cs @@ -0,0 +1,104 @@ +using System.Buffers.Binary; +using System.Text; +using AVEVA.Historian.Client.Protocol; + +namespace AVEVA.Historian.Client.Wcf; + +/// +/// Serializes the GETRP (GetRuntimeParameter) request buffer and parses its response +/// buffer for AVEVA Historian 2020 over WCF/MDAS. +/// +/// +/// Wire format captured from the native client (scripts/Capture-RuntimeParam.ps1 + +/// instrument-wcf-{write,read}message; golden-pinned in WcfRuntimeParameterProtocolTests). +/// The op is aa/Stat/GETRP(string handle, byte[] pRequestBuff) -> (bool, byte[] +/// pResponseBuff, byte[] errorBuffer) — the same string-handle + request/response-buffer +/// shape as GETHI. +/// +/// pRequestBuff (44 bytes for one name "HistorianVersion"): +/// 54 67 signature · 01 00 version(1) · uint32 nameCount · then per name: +/// uint32 charCount + UTF-16LE chars. +/// +/// pResponseBuff: 01 00 version(1) · uint32 resultCount · then per result a +/// CRetVariant: 0x43 (VT_BSTR) + uint16 payloadLength + uint16 charCount + UTF-16LE chars. +/// Only the string variant (0x43) is evidence-backed; other variant types throw. +/// +/// Only a single string-valued parameter is exercised against the live server; the +/// multi-name framing is inferred from the count field and not yet captured. +/// +internal static class HistorianRuntimeParameterProtocol +{ + // Captured constant prefix of pRequestBuff: signature 0x6754 + version 0x0001. + private static ReadOnlySpan RequestHeader => [0x54, 0x67, 0x01, 0x00]; + + private const byte VariantTypeBStr = 0x43; + + public static byte[] SerializeRequest(string parameterName) + { + ArgumentException.ThrowIfNullOrEmpty(parameterName); + + using MemoryStream stream = new(); + using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true); + + writer.Write(RequestHeader); // 54 67 01 00 + writer.Write(1u); // name count = 1 + WriteName(writer, parameterName); + + writer.Flush(); + return stream.ToArray(); + } + + private static void WriteName(BinaryWriter writer, string name) + { + writer.Write((uint)name.Length); // char count + writer.Write(Encoding.Unicode.GetBytes(name)); + } + + /// + /// Parses the first string-valued result out of the GETRP response buffer, or null when the + /// buffer carries no result. Throws for any + /// non-string variant type (not yet captured). + /// + public static string? ParseSingleStringResult(ReadOnlySpan responseBuffer) + { + if (responseBuffer.Length < 6) + { + return null; + } + + // [0..2) version, [2..6) result count. + uint resultCount = BinaryPrimitives.ReadUInt32LittleEndian(responseBuffer.Slice(2, 4)); + if (resultCount == 0) + { + return null; + } + + ReadOnlySpan cursor = responseBuffer[6..]; + if (cursor.Length < 1) + { + return null; + } + + byte variantType = cursor[0]; + if (variantType != VariantTypeBStr) + { + throw new ProtocolEvidenceMissingException( + $"GETRP response variant type 0x{variantType:X2} is not the evidence-backed string variant (0x43)."); + } + + if (cursor.Length < 5) + { + return null; + } + + ushort charCount = BinaryPrimitives.ReadUInt16LittleEndian(cursor.Slice(3, 2)); + int byteCount = charCount * 2; + ReadOnlySpan valueBytes = cursor.Slice(5); + if (valueBytes.Length < byteCount) + { + byteCount = valueBytes.Length - (valueBytes.Length % 2); + } + + return Encoding.Unicode.GetString(valueBytes[..byteCount]); + } +} diff --git a/src/AVEVA.Historian.Client/Wcf/HistorianWcfStatusClient.cs b/src/AVEVA.Historian.Client/Wcf/HistorianWcfStatusClient.cs index 44af582..bd4cf71 100644 --- a/src/AVEVA.Historian.Client/Wcf/HistorianWcfStatusClient.cs +++ b/src/AVEVA.Historian.Client/Wcf/HistorianWcfStatusClient.cs @@ -17,6 +17,19 @@ internal static class HistorianWcfStatusClient return Task.Run(() => GetSystemParameter(options, parameterName), cancellationToken); } + /// Diagnostic: the GETRP return code / error description from the last + /// call (set only when the server rejects it). + public static string? LastRuntimeParameterError { get; private set; } + + public static Task GetRuntimeParameterAsync( + HistorianClientOptions options, + string parameterName, + CancellationToken cancellationToken) + { + ArgumentException.ThrowIfNullOrWhiteSpace(parameterName); + return Task.Run(() => GetRuntimeParameter(options, parameterName), cancellationToken); + } + public static Task GetConnectionStatusAsync( HistorianClientOptions options, CancellationToken cancellationToken) @@ -45,6 +58,49 @@ internal static class HistorianWcfStatusClient return value; } + private static string? GetRuntimeParameter(HistorianClientOptions options, string parameterName) + { + Guid contextKey = Guid.NewGuid(); + var (histBinding, histEndpoint, _, _) = HistorianWcfBindingFactory.CreateBindingPair(options); + Binding statusBinding = HistorianWcfBindingFactory.CreateAuxiliaryBinding(options); + EndpointAddress statusEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(options, HistorianWcfServiceNames.Status); + + string? value = null; + LastRuntimeParameterError = null; + HistorianWcfAuthChainHelper.OpenAuthenticatedConnection( + options, histBinding, histEndpoint, contextKey, CancellationToken.None, + additionalSetup: (_, context) => value = QueryRuntimeParameter(statusBinding, statusEndpoint, context.StorageSessionId, parameterName)); + return value; + } + + private static string? QueryRuntimeParameter(Binding statusBinding, EndpointAddress statusEndpoint, Guid storageSessionId, string parameterName) + { + // GETRP takes the storage-session GUID as a string handle, formatted exactly as the + // native client sends it: uppercase, dash-separated, no braces. + string handle = storageSessionId.ToString("D").ToUpperInvariant(); + byte[] requestBuffer = HistorianRuntimeParameterProtocol.SerializeRequest(parameterName); + + ChannelFactory factory = new(statusBinding, statusEndpoint); + IStatusServiceContract2 channel = factory.CreateChannel(); + ICommunicationObject co = (ICommunicationObject)channel; + try + { + bool ok = channel.GetRuntimeParameter(handle, requestBuffer, out byte[] responseBuffer, out byte[] errorBuffer); + if (!ok) + { + LastRuntimeParameterError = $"GETRP returned false (responseLen={responseBuffer?.Length ?? 0}, errorLen={errorBuffer?.Length ?? 0})."; + return null; + } + + return HistorianRuntimeParameterProtocol.ParseSingleStringResult(responseBuffer ?? []); + } + finally + { + try { if (co.State == CommunicationState.Faulted) co.Abort(); else co.Close(); } catch { try { co.Abort(); } catch { } } + try { if (factory.State == CommunicationState.Faulted) factory.Abort(); else factory.Close(); } catch { try { factory.Abort(); } catch { } } + } + } + private static string? QuerySystemParameter(Binding statusBinding, EndpointAddress statusEndpoint, uint clientHandle, string parameterName) { ChannelFactory factory = new(statusBinding, statusEndpoint); diff --git a/tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs b/tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs index ddc5339..6b38821 100644 --- a/tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs +++ b/tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs @@ -273,6 +273,29 @@ public sealed class HistorianClientIntegrationTests Assert.False(string.IsNullOrWhiteSpace(value)); } + [Fact] + public async Task GetRuntimeParameterAsync_AgainstLocalHistorian_ReturnsHistorianVersion() + { + string? host = Environment.GetEnvironmentVariable("HISTORIAN_HOST"); + if (string.IsNullOrWhiteSpace(host) || !string.Equals(host, "localhost", StringComparison.OrdinalIgnoreCase) || !OperatingSystem.IsWindows()) + { + return; + } + + HistorianClient client = new(new HistorianClientOptions + { + Host = host, + IntegratedSecurity = true, + Transport = HistorianTransport.LocalPipe + }); + + // GETRP rides the storage-session GUID as an uppercase string handle. HistorianVersion is + // a known-good runtime parameter returning the server version (e.g. "20,0,000,000"). + string? value = await client.GetRuntimeParameterAsync("HistorianVersion", CancellationToken.None); + + Assert.False(string.IsNullOrWhiteSpace(value)); + } + [Fact] public async Task GetConnectionStatusAsync_AgainstLocalHistorian_ReportsConnectedToServer() { diff --git a/tests/AVEVA.Historian.Client.Tests/StringHandleProbeDiagnosticTests.cs b/tests/AVEVA.Historian.Client.Tests/StringHandleProbeDiagnosticTests.cs new file mode 100644 index 0000000..65531a5 --- /dev/null +++ b/tests/AVEVA.Historian.Client.Tests/StringHandleProbeDiagnosticTests.cs @@ -0,0 +1,156 @@ +using System.ServiceModel; +using System.ServiceModel.Channels; +using System.Text; +using System.Runtime.Versioning; +using AVEVA.Historian.Client.Wcf; +using AVEVA.Historian.Client.Wcf.Contracts; +using Xunit.Abstractions; + +namespace AVEVA.Historian.Client.Tests; + +/// +/// Diagnostic: retest the "string-handle wall" ops (GETHI / ExeC) using the Open2 +/// storage-session GUID formatted UPPERCASE (the format the native client sends, and the +/// one that made GETRP punch through). Not an assertion test — it prints the server's +/// return code / buffer lengths so we can judge whether the wall is a handle-format issue. +/// +[SupportedOSPlatform("windows")] +public sealed class StringHandleProbeDiagnosticTests +{ + private readonly ITestOutputHelper _output; + + public StringHandleProbeDiagnosticTests(ITestOutputHelper output) + { + _output = output; + } + + private static bool ShouldRun(out string host) + { + host = Environment.GetEnvironmentVariable("HISTORIAN_HOST") ?? string.Empty; + return !string.IsNullOrWhiteSpace(host) + && string.Equals(host, "localhost", StringComparison.OrdinalIgnoreCase) + && OperatingSystem.IsWindows(); + } + + [Fact] + public void GETHI_WithUppercaseStorageGuid_AgainstLocalHistorian() + { + if (!ShouldRun(out string host)) return; + + HistorianClientOptions options = new() + { + Host = host, + IntegratedSecurity = true, + Transport = HistorianTransport.LocalPipe + }; + + // Native GETHI pRequestBuff: 53 67 02 00 (sig 0x6753 + version 2) + uint charCount + UTF-16 name. + const string name = "HistorianVersion"; + using MemoryStream ms = new(); + using (BinaryWriter w = new(ms, Encoding.Unicode, leaveOpen: true)) + { + w.Write(new byte[] { 0x53, 0x67, 0x02, 0x00 }); + w.Write((uint)name.Length); + w.Write(Encoding.Unicode.GetBytes(name)); + } + byte[] requestBuffer = ms.ToArray(); + + ProbeOnStatusChannel(options, (channel, handle) => + { + bool ok = channel.GetHistorianInfo(handle, requestBuffer, out byte[] resp, out byte[] err); + _output.WriteLine($"GETHI returned={ok} respLen={resp?.Length ?? 0} errLen={err?.Length ?? 0}"); + if (resp is { Length: > 0 }) + { + _output.WriteLine(" resp[0..64]=" + Convert.ToHexString(resp.AsSpan(0, Math.Min(64, resp.Length)))); + } + }); + } + + [Fact] + public void ExeC_WithUppercaseStorageGuid_AgainstLocalHistorian() + { + if (!ShouldRun(out string host)) return; + + HistorianClientOptions options = new() + { + Host = host, + IntegratedSecurity = true, + Transport = HistorianTransport.LocalPipe + }; + + Guid contextKey = Guid.NewGuid(); + var (histBinding, histEndpoint, retrBinding, retrEndpoint) = HistorianWcfBindingFactory.CreateBindingPair(options); + + HistorianWcfAuthChainHelper.OpenAuthenticatedConnection( + options, histBinding, histEndpoint, contextKey, CancellationToken.None, + additionalSetup: (_, context) => + { + string handle = context.StorageSessionId.ToString("D").ToUpperInvariant(); + ChannelFactory factory = new(retrBinding, retrEndpoint); + HistorianWcfClientCredentialsHelper.Configure(factory, options); + IRetrievalServiceContract3 channel = factory.CreateChannel(); + ICommunicationObject co = (ICommunicationObject)channel; + try + { + // Prime the Retr service version handshake (Retr.GetV), as the native client does. + channel.GetInterfaceVersion(out uint retrVersion); + _output.WriteLine($"Retr.GetV version={retrVersion}"); + + uint queryHandle = 0; + bool execOk = channel.ExecuteSqlCommand( + handle, "SELECT 1 AS ProbeValue", 0u, ref queryHandle, + out int retValue, out uint errSize, out byte[] errBuf); + _output.WriteLine($"ExeC returned={execOk} retValue={retValue} queryHandle={queryHandle} errSize={errSize} errLen={errBuf?.Length ?? 0}"); + + if (execOk) + { + uint sequence = 0; + bool getrOk = channel.GetRecordSetByteStream( + handle, queryHandle, ref sequence, + out uint resultSize, out byte[] resultBuf, out uint gErrSize, out byte[] gErrBuf); + _output.WriteLine($"GetR returned={getrOk} resultSize={resultSize} resultLen={resultBuf?.Length ?? 0} sequence={sequence}"); + if (resultBuf is { Length: > 0 }) + { + _output.WriteLine(" result[0..96]=" + Convert.ToHexString(resultBuf.AsSpan(0, Math.Min(96, resultBuf.Length)))); + } + } + } + finally + { + try { if (co.State == CommunicationState.Faulted) co.Abort(); else co.Close(); } catch { try { co.Abort(); } catch { } } + try { if (factory.State == CommunicationState.Faulted) factory.Abort(); else factory.Close(); } catch { try { factory.Abort(); } catch { } } + } + }); + } + + private static void ProbeOnStatusChannel(HistorianClientOptions options, Action probe) + { + Guid contextKey = Guid.NewGuid(); + var (histBinding, histEndpoint, _, _) = HistorianWcfBindingFactory.CreateBindingPair(options); + Binding statusBinding = HistorianWcfBindingFactory.CreateAuxiliaryBinding(options); + EndpointAddress statusEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(options, HistorianWcfServiceNames.Status); + + HistorianWcfAuthChainHelper.OpenAuthenticatedConnection( + options, histBinding, histEndpoint, contextKey, CancellationToken.None, + additionalSetup: (_, context) => + { + string handle = context.StorageSessionId.ToString("D").ToUpperInvariant(); + ChannelFactory factory = new(statusBinding, statusEndpoint); + IStatusServiceContract2 channel = factory.CreateChannel(); + ICommunicationObject co = (ICommunicationObject)channel; + try + { + probe(channel, handle); + } + catch (Exception ex) + { + throw new InvalidOperationException($"probe raised: {ex.GetType().Name}: {ex.Message}", ex); + } + finally + { + try { if (co.State == CommunicationState.Faulted) co.Abort(); else co.Close(); } catch { try { co.Abort(); } catch { } } + try { if (factory.State == CommunicationState.Faulted) factory.Abort(); else factory.Close(); } catch { try { factory.Abort(); } catch { } } + } + }); + } +} diff --git a/tests/AVEVA.Historian.Client.Tests/WcfRuntimeParameterProtocolTests.cs b/tests/AVEVA.Historian.Client.Tests/WcfRuntimeParameterProtocolTests.cs new file mode 100644 index 0000000..70acb94 --- /dev/null +++ b/tests/AVEVA.Historian.Client.Tests/WcfRuntimeParameterProtocolTests.cs @@ -0,0 +1,58 @@ +using AVEVA.Historian.Client.Protocol; +using AVEVA.Historian.Client.Wcf; + +namespace AVEVA.Historian.Client.Tests; + +public sealed class WcfRuntimeParameterProtocolTests +{ + // GETRP pRequestBuff captured from the native client for GetRuntimeParameter("HistorianVersion") + // via scripts/Capture-RuntimeParam.ps1 + instrument-wcf-writemessage: + // 54 67 01 00 signature(0x6754) + version(1) + // 01 00 00 00 name count = 1 + // 10 00 00 00 char count = 16 + // UTF-16LE "HistorianVersion" + private const string CaptureRequestHex = + "54670100010000001000000048006900730074006F007200690061006E00560065007200730069006F006E00"; + + // GETRP pResponseBuff captured from the paired GETRPResponse (instrument-wcf-readmessage): + // 01 00 version = 1 + // 01 00 00 00 result count = 1 + // 43 CRetVariant type 0x43 (VT_BSTR) + // 1A 00 payload length = 26 (= charCount field + string bytes) + // 0C 00 char count = 12 + // UTF-16LE "20,0,000,000" + private const string CaptureResponseHex = + "010001000000431A000C00320030002C0030002C003000300030002C00300030003000"; + + [Fact] + public void SerializeRequestMatchesInstrumentedNativeRequestBuffer() + { + byte[] actual = HistorianRuntimeParameterProtocol.SerializeRequest("HistorianVersion"); + Assert.Equal(Convert.FromHexString(CaptureRequestHex), actual); + } + + [Fact] + public void ParseSingleStringResultReadsTheCapturedResponseValue() + { + byte[] response = Convert.FromHexString(CaptureResponseHex); + string? value = HistorianRuntimeParameterProtocol.ParseSingleStringResult(response); + Assert.Equal("20,0,000,000", value); + } + + [Fact] + public void ParseSingleStringResultReturnsNullForZeroResultCount() + { + // version(1) + result count(0) + byte[] empty = [0x01, 0x00, 0x00, 0x00, 0x00, 0x00]; + Assert.Null(HistorianRuntimeParameterProtocol.ParseSingleStringResult(empty)); + } + + [Fact] + public void ParseSingleStringResultThrowsForUncapturedVariantType() + { + // version(1) + count(1) + a non-string variant marker (0x03, VT_I4 — not captured). + byte[] buffer = [0x01, 0x00, 0x01, 0x00, 0x00, 0x00, 0x03, 0x04, 0x00, 0x00, 0x00, 0x00]; + Assert.Throws( + () => HistorianRuntimeParameterProtocol.ParseSingleStringResult(buffer)); + } +} diff --git a/tools/AVEVA.Historian.NativeTraceHarness/Program.cs b/tools/AVEVA.Historian.NativeTraceHarness/Program.cs index fc375d5..d2f0f60 100644 --- a/tools/AVEVA.Historian.NativeTraceHarness/Program.cs +++ b/tools/AVEVA.Historian.NativeTraceHarness/Program.cs @@ -162,7 +162,91 @@ internal static class Program string? moveTerminalDescription = null; List rows = []; - if (openSuccess && status.ConnectedToServer && IsEventSendScenario(scenario)) + if (openSuccess && status.ConnectedToServer && IsExecSqlScenario(scenario)) + { + // R1.1 capture: drive HistorianAccess.ExecuteSqlCommand(sql, option, out retval, + // out DataTable, out error) so instrument-wcf-{write,read}message can observe the + // Retr.ExeC + Retr.GetR wire shape (handle format, command/option encoding, Retr + // priming, result byte stream). Read-only benign query. + string sql = GetArg(args, "--sql") ?? "SELECT 1 AS ProbeValue"; + Type sqlOptionType = GetType(assembly, "ArchestrA.HistorianSqlExecuteOption"); + object sqlOption = Enum.Parse(sqlOptionType, GetArg(args, "--sql-option") ?? "ExecuteRecord"); + + MethodInfo execMethod = accessType.GetMethods() + .First(m => m.Name == "ExecuteSqlCommand" && m.GetParameters().Length == 5); + object?[] execArgs = new object?[] { sql, sqlOption, 0, null, Activator.CreateInstance(errorType) }; + WriteRuntimeMethodPointerSnapshot(assembly, runtimeMethodPointerOutput, runtimeMethodPointerFilters, repoRoot, scenario, "before-exec-sql"); + bool execOk = (bool)execMethod.Invoke(access, execArgs)!; + + int rowCount = -1, colCount = -1; + if (execArgs[3] is System.Data.DataTable table) + { + rowCount = table.Rows.Count; + colCount = table.Columns.Count; + } + + Console.WriteLine(Serialize(new + { + Scenario = scenario, + Sql = sql, + ExecuteSqlCommandReturned = execOk, + ReturnValue = execArgs[2], + RowCount = rowCount, + ColumnCount = colCount, + Error = SnapshotObject(execArgs[4]!), + })); + return 0; + } + else if (openSuccess && status.ConnectedToServer && IsRuntimeParamScenario(scenario)) + { + // R1.2 capture: drive HistorianAccess.GetRuntimeParameter(List names, + // out List results, out error) so instrument-wcf-{write,read}message can + // observe the WCF op name, handle type (uint vs string-handle wall), and the + // btRequest/btResponse buffer format. Pure status read — no write mode needed. + string namesArg = GetArg(args, "--runtime-param-names") ?? "HistorianVersion"; + string[] names = namesArg.Split(new[] { ';' }, StringSplitOptions.RemoveEmptyEntries); + + MethodInfo getRtParamMethod = accessType.GetMethods() + .First(m => m.Name == "GetRuntimeParameter" && m.GetParameters().Length == 3); + ParameterInfo[] rtParams = getRtParamMethod.GetParameters(); + Type namesListType = rtParams[0].ParameterType; // List + Type resultsListType = rtParams[1].ParameterType.GetElementType()!; // List<...>& -> List<...> + + object namesList = Activator.CreateInstance(namesListType)!; + MethodInfo addName = namesListType.GetMethod("Add")!; + foreach (string n in names) addName.Invoke(namesList, new object?[] { n }); + + object rtError = Activator.CreateInstance(errorType)!; + object?[] rtArgs = new object?[] { namesList, Activator.CreateInstance(resultsListType), rtError }; + WriteRuntimeMethodPointerSnapshot(assembly, runtimeMethodPointerOutput, runtimeMethodPointerFilters, repoRoot, scenario, "before-get-runtime-parameter"); + bool rtOk = (bool)getRtParamMethod.Invoke(access, rtArgs)!; + + object? resultsList = rtArgs[1]; + var resultItems = new List(); + if (resultsList is System.Collections.IEnumerable en) + { + foreach (object? item in en) + { + resultItems.Add(new + { + Type = item?.GetType().FullName, + Value = item?.ToString(), + }); + } + } + + Console.WriteLine(Serialize(new + { + Scenario = scenario, + Names = names, + GetRuntimeParameterReturned = rtOk, + ResultsListType = resultsListType.FullName, + Results = resultItems, + Error = SnapshotObject(rtArgs[2]!), + })); + return 0; + } + else if (openSuccess && status.ConnectedToServer && IsEventSendScenario(scenario)) { // R2.1 capture: drive AddStreamedValue(HistorianEvent) and let instrument-wcf-* // observe whether the event delivery rides the WCF MDAS path or the storage-engine @@ -1407,6 +1491,28 @@ internal static class Program } /// Both event-query and event-send require an Event-type connection. + /// + /// Runtime-parameter scenario (R1.2 capture): opens a normal authenticated process + /// connection and calls GetRuntimeParameter so the WCF op + buffer format can be + /// captured. Read-only; not an event or write connection. + /// + private static bool IsRuntimeParamScenario(string scenario) + { + return scenario.Equals("runtime-param", StringComparison.OrdinalIgnoreCase) + || scenario.Equals("runtime-parameter", StringComparison.OrdinalIgnoreCase); + } + + /// + /// SQL-command scenario (R1.1 capture): opens a normal authenticated process connection and + /// calls ExecuteSqlCommand (Retr.ExeC + Retr.GetR) so the string-handle SQL surface + /// can be captured. Read-only benign query. + /// + private static bool IsExecSqlScenario(string scenario) + { + return scenario.Equals("exec-sql", StringComparison.OrdinalIgnoreCase) + || scenario.Equals("sql", StringComparison.OrdinalIgnoreCase); + } + private static bool IsEventConnectionScenario(string scenario) { return IsEventScenario(scenario) || IsEventSendScenario(scenario);