Ship SQL command execution over the 2020 WCF aa/Retr/ExeC + aa/Retr/GetR ops:
HistorianClient.ExecuteSqlCommandAsync(sql) -> HistorianSqlResult (columns +
typed rows). String-handle ops reached with the Open2 storage-session GUID
formatted uppercase (the handle format that unlocked GETRP/GETHI).
Chain: Retr.GetV prime -> ExeC(handle, sql, option=0, ref queryHandle) ->
GetR loop. Key gotcha captured: GetR returns FALSE even on success -- the byte
stream is in pResultBuff regardless; false just signals the final page. So the
orchestrator consumes the buffer first, then stops on a false result / empty page.
GetR's pResultBuff is an NRBF-serialized System.Data.DataTable
(SerializationFormat.Xml: members XmlSchema (XSD) + XmlDiffGram (rows)).
BinaryFormatter is removed from .NET 10, so the stream is decoded read-only with
the System.Formats.Nrbf package (NrbfDecoder) + XDocument -- no BinaryFormatter,
no code execution. Values are typed per the XSD type, falling back to string.
Adds: HistorianSqlResult / HistorianSqlColumn / HistorianSqlExecuteOption models,
HistorianSqlResultProtocol (NRBF + diffgram parser), HistorianWcfSqlClient
(ExeC/GetR orchestration with an AVEVA_HISTORIAN_SQL_DUMP diagnostic), dialect +
public API. Golden WcfSqlResultProtocolTests pinned to the real clean GetR stream
for the benign "SELECT 1 AS ProbeValue" (no sensitive data); gated live tests
(single cell + multi-column/multi-row/NULL). Doc: wcf-exec-sql.md; roadmap R1.1
DONE; wall doc + memory updated (incl. the QTB-server-side nuance). 229 tests green.
Note: a raw instrument-wcf capture corrupts a large pResultBuff with MDAS
transport chunk markers (0x9F); the clean contract-level byte[] is dumped via the
AVEVA_HISTORIAN_SQL_DUMP env var for the golden fixture.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Execute HCAL roadmap R1.2 (GetRuntimeParameterAsync) end-to-end, and in doing so
discover that the "string-handle wall" blocking R1.1/R1.4/R1.5/R1.6 was a handle
FORMAT bug, not a missing native session/filter registration.
R1.2 (shipped, live-verified):
- Captured native GetRuntimeParameter -> WCF op aa/Stat/GETRP (string-handle op,
GETHI's shape), via scripts/Capture-RuntimeParam.ps1 + instrument-wcf-{write,read}message.
- HistorianRuntimeParameterProtocol serializes pRequestBuff (54 67 01 00 + uint
nameCount + per-name uint charCount + UTF-16) and parses pResponseBuff (version +
uint resultCount + CRetVariant 0x43 VT_BSTR + uint16 len + uint16 charCount + UTF-16).
- IStatusServiceContract2.GetRuntimeParameter (GETRP) op; HistorianWcfStatusClient
passes the Open2 storage-session GUID as the string handle, UPPERCASE.
- Public HistorianClient.GetRuntimeParameterAsync(name) via the dialect.
- Golden WcfRuntimeParameterProtocolTests + gated live test; returns HistorianVersion.
String-handle wall RESOLVED (proven, public APIs deferred):
- The Open2 storage GUID works as the string handle when sent UPPERCASE
(ToString("D").ToUpperInvariant()); earlier "blocked" probes used lowercase.
- Live-probed GETHI (R1.4) -> returns data; ExeC (R1.1) -> Retr.GetV prime -> ExeC ->
GetR returns a BinaryFormatter-serialized .NET DataTable. Gated
StringHandleProbeDiagnosticTests + scripts/Capture-ExecSql.ps1 + exec-sql harness scenario.
- Docs flipped: wcf-string-handle-wall.md RESOLVED banner; roadmap R1.1/R1.4 reachable,
R1.5/R1.6 likely; wcf-status-localhost.md GETRP section.
- R1.1/R1.4 public APIs NOT shipped: ExeC needs a GetR paging loop + a BinaryFormatter-
stream parser (BinaryFormatter is removed from .NET 10); GETHI full-info struct needs
its own capture.
223 unit tests pass; gated live tests green against the local 2020 Historian.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Probed R1.4 GetHistorianInfo (GETHI) live against the local 2020 server.
GETHI returns native error type 4 / code 1 for the exact native request shape
across 5 handle formats (storage GUID, context GUID, uint decimal/X8/0x-hex)
even with Stat.GetV ×2 priming. Its result is discarded (TryRun) in the only
place it's used, so it was never actually verified to return data managed-side.
This confirms a structural boundary on the 2020 WCF surface: ops taking a uint
client handle work (the proven read/browse/metadata/status/event surface);
ops taking a string GUID handle (ExeC, QTB, QTG, GETHI, GetTepByNm, ...) are
blocked behind an unmapped native session/filter registration. Every remaining
M1 *read* item (R1.1/R1.4/R1.5/R1.6) is string-handle -> all gated on that one
RE target. Reachable uint-handle items: R1.7 event filters, R1.8/R1.9 summary
modes.
New: docs/reverse-engineering/wcf-string-handle-wall.md (full dichotomy + table).
Roadmap R1.4/R1.5/R1.6 struck through; reachable items re-pointed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Live-probed both against the local Historian 2020 (WCF):
- R1.3 GetServerTimeZoneAsync: Status.GetSystemTimeZoneName returns rc=0 with
an empty value under a real authenticated handle — a client-side stub in the
GetServerTime family. gRPC/2023R2-only. Reverted the implementation.
- R1.1 ExecuteSqlCommandAsync: Retrieval.ExeC returns native error type 4 /
code 51 (InvalidParameter); the contract-3 string-handle ops require an
unmapped native session/filter registration step (the StartTagQuery wall).
Adds an M1a re-classification note steering future work toward proven
uint-handle / already-wired ops (R1.4 GETHI next) over string-handle ops.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
HistorianSspiClient rewritten on top of System.Net.Security.NegotiateAuthentication
in place of P/Invoke into secur32.dll's InitializeSecurityContextW. The class
keeps the same Next() / Dispose() / two-constructor surface so callers don't
change. RequiredProtectionLevel=EncryptAndSign + AllowedImpersonationLevel=
Identification produces a request-flag set equivalent to the captured native
0x2081C / 0x81C bitmasks (still preserved as constants for the existing unit
tests). Removes the only Windows P/Invoke in the production SDK; the
[SupportedOSPlatform("windows")] gating elsewhere stays in place pending a
separate sweep.
HistorianStorageType (Cyclic = 1, Delta = 2):
Captured 2026-05-04 via --write-storage-type on the harness. Delta differs
from Cyclic in three places — header byte 10 (0x02 -> 0x06), flag-block
byte 1 (0x01 -> 0x02), and 4 zero bytes inserted after StorageRate before
the FILETIME. Server persists Tag.StorageType=1/2 accordingly. Plumbed
through HistorianTagDefinition.StorageType + serializer + orchestrator + 2
new tests (golden bytes + live SQL persistence verification).
Docs polish:
CLAUDE.md no longer claims "no P/Invoke" (HistorianSspiClient is the one
allowed P/Invoke surface); updated test count to 169+; AGENTS.md Required
SDK Surface and Repository Layout brought up to date with the live state
including the write surface; handoff.md "not a git working tree" obsolete
note removed.
171/171 tests pass with the NegotiateAuthentication replacement (was 169;
+2 new tests for StorageType).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ApplyScaling (HistorianTagDefinition.ApplyScaling):
The EnsT2 trailer's second byte controls server-side scaling — `FE 00`
mirrors MinRaw to MinEU and sets AnalogTag.Scaling=0; `FE 01` persists
distinct MinRaw/MaxRaw and sets Scaling=1. Decoded by toggling
set_ApplyScaling on the native harness and capturing the wire bytes for
both values with identical inputs. The earlier docs claimed
EnsureTagAsync needed a follow-up "UpdateTags" call; the WCF surface has
no such operation — toggling that one byte is the whole fix.
StorageRate (HistorianTagDefinition.StorageRateMs):
Serializer accepts a non-default rate, validated empirically against
the live server which only accepts quantized values
(1000/5000/10000/60000/300000 ms).
EnsureTagAsync upsert semantics:
Second call on the same tag name with different fields succeeds and
updates Description, MinEU, MaxEU, MinRaw, MaxRaw, Scaling in place
(verified by direct SQL inspection in a live test).
Plan + doc closeout:
write-commands-reverse-engineering.md rewritten as a current-state
plan with three workstreams (A doc closeout / B idempotency / C1
StorageRate) and a parallelism table; prior phase notes preserved as
appendix. handoff.md, implementation-status.md, wcf-contract-evidence.md,
README.md updated to remove "writes are out of scope" / non-existent
UpdateTags references and document the actual EnsT2 wire format
including the `FE xx` trailer.
Reverse-engineering harness gains --write-apply-scaling and a SQL
post-check that prints the persisted AnalogTag bounds so future RE
sessions can verify wire→DB causality without leaving the harness.
169/169 tests pass (was 165; +4 new tests covering ApplyScaling,
StorageRate golden bytes, StorageRate live persistence, and
EnsureTagAsync upsert semantics).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Full read-only SDK (src/AVEVA.Historian.Client) implementing the CLAUDE.md required
surface against AVEVA Historian's binary WCF protocol — no native AVEVA runtime
dependency. All operations live-verified against a local Historian:
- ProbeAsync, ReadRawAsync, ReadAggregateAsync, ReadAtTimeAsync, ReadEventsAsync
- BrowseTagNamesAsync, GetTagMetadataAsync (17 native data-type codes mapped)
- GetConnectionStatusAsync, GetStoreForwardStatusAsync, GetSystemParameterAsync
- 108/108 unit + integration tests pass
Includes the reverse-engineering toolkit (tools/AVEVA.Historian.ReverseEngineering)
used to decode the protocol: WCF probes, IL inspection via dnlib, and IL-rewrite
instrumentation (instrument-wcf-{write,read}message etc.) plus the .NET Framework
trace harness (tools/AVEVA.Historian.NativeTraceHarness) for parity testing.
Sanitized handoff evidence under docs/reverse-engineering/. Native AVEVA binaries
(current/, aveva-install-x64/, aveva-install-x86/) are gitignored — fetch separately
from the AVEVA installer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Joseph Doherty