histsdk/docs/reverse-engineering/wcf-exec-sql.md

SQL command execution over 2020 WCF — ExeC + GetR (HCAL R1.1)

Status: ✅ DONE + live-verified (2026-06-20). HistorianClient.ExecuteSqlCommandAsync(sql) runs a SQL command against the Historian over the 2020 WCF ops aa/Retr/ExeC (ExecuteSqlCommand) + aa/Retr/GetR (GetRecordSetByteStream) and returns the record set as a HistorianSqlResult (the managed equivalent of the native DataTable). Live-verified end-to-end from the pure-managed .NET 10 client against the local 2020 Historian (single-cell, multi-column/multi-row, and NULL cases).

The ops

Both are on the Retrieval service (IRetrievalServiceContract3), string-handle ops reached with the Open2 storage-session GUID formatted storageSessionId.ToString("D").ToUpperInvariant() (the same uppercase handle that unlocked GETRP/GETHI; see wcf-string-handle-wall.md). Retr.GetV is primed first.

bool ExecuteSqlCommand(string handle, string command, uint option,
                       ref uint queryHandle, out int retValue, out uint errorSize, out byte[] errorBuffer)

bool GetRecordSetByteStream(string handle, uint queryHandle, ref uint sequence,
                            out uint resultSize, out byte[] pResultBuff, out uint errorSize, out byte[] errorBuffer)

• command is sent as a plain string (MDAS-encoded ASCII), e.g. SELECT 1 AS ProbeValue.
• option = HistorianSqlExecuteOption (ExecuteRecord=0 is the captured/proven record-set path).
• ExeC returns the assigned queryHandle (and retValue); pass it to GetR.
• GetR returns false even on success — the byte stream is in pResultBuff regardless; a false result just signals the final page. So the orchestrator always consumes pResultBuff, then stops on a false result or an empty page. (sequence is the paging cursor; small record sets return everything in one call.)

The result stream — NRBF-wrapped DataTable (no BinaryFormatter)

GetR's pResultBuff is a .NET Remoting Binary Format (NRBF) stream wrapping a System.Data.DataTable serialized with SerializationFormat.Xml. Stream shape (captured):

SerializationHeader (00 01 00 00 00 FF FF FF FF 01 00 00 00 00 00 00 00)
BinaryLibrary (0C): "System.Data, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
ClassWithMembersAndTypes (05): System.Data.DataTable, members:
    DataTable.RemotingVersion  -> System.Version object
    XmlSchema                  -> BinaryObjectString (XSD: column names + xs types)
    XmlDiffGram                -> BinaryObjectString (ADO.NET diffgram: the rows)
System.Version object (_Major/_Minor/_Build/_Revision)
MessageEnd (0B)

BinaryFormatter is removed from .NET 10, so the stream is decoded read-only with System.Formats.Nrbf.NrbfDecoder (the sanctioned successor — it parses records without instantiating or executing any payload type; added as a managed PackageReference). The two embedded XML strings are then parsed with XDocument:

• XmlSchema → columns (name + XSD type) under the msdata:MainDataTable element's sequence.
• XmlDiffGram → rows (each row element under the dataset; cells are child elements or attributes matching column names). NULL cells are simply absent → parsed as null.

Values are typed per the XSD type (xs:int→int, xs:string→string, xs:dateTime→DateTime, …), falling back to the raw string for unrecognized types. Only the SerializationFormat.Xml DataTable shape is evidence-backed; a stream whose root is not a DataTable class record, or that lacks the two XML members, throws ProtocolEvidenceMissingException.

Capture / decode tooling

scripts/Capture-ExecSql.ps1 (NativeTraceHarness exec-sql scenario + instrument-wcf-{write,read}message) captures the ExeC/GetR exchange. ⚠️ A raw instrument-wcf capture interleaves MDAS transport chunk markers (0x9F/0x9E) into a large pResultBuff, so raw byte-slicing yields a corrupted NRBF stream. The clean contract-level byte[] (what the WCF channel reassembles) is dumped via the AVEVA_HISTORIAN_SQL_DUMP env var on HistorianWcfSqlClient — that is the golden fixture in WcfSqlResultProtocolTests (the benign SELECT 1 AS ProbeValue, no sensitive data).

Shipped surface

• HistorianClient.ExecuteSqlCommandAsync(command, option = ExecuteRecord) → HistorianSqlResult (Columns name+type, Rows typed values, ReturnValue).
• Models HistorianSqlResult / HistorianSqlColumn / HistorianSqlExecuteOption; HistorianSqlResultProtocol (NRBF + diffgram parser); HistorianWcfSqlClient (ExeC/GetR orchestration); golden WcfSqlResultProtocolTests; gated live tests (ExecuteSqlCommandAsync_AgainstLocalHistorian_ReturnsRecordSet and _MultiColumnMultiRow).

Scope notes

• ExecuteRecord (record set) is the evidence-backed path. ExecuteNonQuery/ExecuteScalar/ ExecuteRecordDirect are accepted via the option enum but their non-record-set return shapes are not separately captured — a non-record result yields empty Columns/Rows with the ReturnValue set.
• The command is whatever the Historian's SQL surface accepts (it routes to the Runtime DB). No client-side SQL validation is performed.