Wire the config operations that previously only worked over WCF onto RemoteGrpc,
reusing the proven 2020 byte serializers verbatim inside the protobuf bytes fields
(keyed by the Open2 session handle). Live-verified against a real 2023 R2 server
where noted.
Read ops (live-verified):
- GetRuntimeParameterAsync -> StatusService.GetRuntimeParameter (GETRP serializer)
- GetTagExtendedPropertiesAsync -> RetrievalService.GetTagExtendedPropertiesFromName
(GetTepByNm serializer + sequence paging; page-0 FillBufferFromVector is the
benign no-data terminator, matched to the WCF break-and-return-empty semantics)
Server-walled (bounded with captured evidence):
- ExecuteSqlCommandAsync -> RetrievalService.ExecuteSqlCommand. The request rides
the RPC but the server-side CSrvDbConnection.ExecuteSqlCommand faults
(IndexOutOfRange / native err 38) on a DB-connection precondition the pure
managed gRPC session doesn't establish (same class as OpenStorageConnection).
Surfaced as ProtocolEvidenceMissingException.
Write ops (tooled + routed, sandbox-gated — not run destructively live):
- EnsureTagAsync / DeleteTagAsync / RenameTagsAsync / AddTagExtendedPropertiesAsync
via HistoryService.EnsureTags / DeleteTags / StartJob / AddTagExtendedProperties
on a write-enabled (0x401) session, reusing the WCF golden serializers. The WCF
priming discovery-dance is omitted (the M3 gRPC write probe worked without it);
add it first if a live sandbox run is rejected.
Routed in Historian2020ProtocolDialect / HistorianClient on the RemoteGrpc branch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Route GetStoreForwardStatusAsync to a gRPC path that actually contacts the
server (StatusService.GetHistorianConsoleStatus) instead of synthesizing an
all-false result blind. On a reachable/normal server it returns the
not-storing baseline but MEASURED; when the server is unreachable or the
console-status call fails it reports ErrorOccurred with the underlying error
(the old synthesis never contacted the server). The active-SF buffer
magnitude (Storing/Pending/DataStored) stays false because it lives behind
the D2 storage-engine console wall.
Non-gRPC transports keep the synthesized fallback. Live-verified against the
2023 R2 server; gated integration test
GetStoreForwardStatusAsync_OverGrpc_ReturnsMeasuredIdleState added. README
operation table updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Live-probed both R1.3 and R1.4 against a real 2023 R2 server over the gRPC
StatusService; implemented the one that carries an evidence-backed value.
R1.3 GetServerTimeZoneAsync — SHIPPED:
- StatusService.GetSystemTimeZoneName(uiHandle) returns the real server zone
over RemoteGrpc (the 2020 WCF op is a client-side stub returning empty).
- HistorianGrpcStatusClient.GetSystemTimeZoneNameAsync -> dialect routing ->
public HistorianClient.GetServerTimeZoneAsync. Non-gRPC transports fail
closed with ProtocolEvidenceMissingException (no empty-string lie).
- Golden message-shape unit test + non-gRPC guardrail unit test + gated live
test. 271 unit tests pass.
R1.4 GetHistorianInfoAsync (EventStorageMode) — bounded out on gRPC too:
- gRPC GetHistorianInfo is the same named-value query as 2020 WCF (only
HistorianVersion resolves); EventStorageMode + 7 variants fail on both
GetHistorianInfo and GetSystemParameter. The 518-byte struct is filled by a
native vtable+648 HCAL call, not the gRPC op (per the 2023 R2 decompile), so
the field is never on the wire. Not shipped on any transport. Closes the
roadmap's open "build against a live 2023 R2 server" caveat.
Also correct the stale M3 roadmap section: D2 already proved
Transaction.AddNonStreamValues* rides the storage-engine pipe (STransactPipeClient2
-> aaStorageEngine), not WCF — same wall as R4.2 — so M3-over-WCF is blocked, not
"the path that is NOT the gated cache push".
Docs: hcal-roadmap.md, wcf-historian-info.md, wcf-status-localhost.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Roadmap docs/plans/hcal-roadmap.md, milestone M0 (gRPC parity for the DONE
surface). Now unblocked for live verification by a reachable 2023 R2 server.
- R0.4 Probe over gRPC: new HistorianGrpcProbe calls History/Retrieval/Status
GetInterfaceVersion (unauthenticated). ProbeAsync routes over gRPC when
Transport==RemoteGrpc. LIVE-VERIFIED against a real 2023 R2 server — needs no
credentials (runs before the auth loop), so it works despite the auth blocker.
- R0.3 System parameter over gRPC: new HistorianGrpcStatusClient calls
StatusService.GetSystemParameter over the authenticated session; routed in the
dialect. Built + unit-tested (request/response field mapping pinned).
Live-verification pending an auth fix (see below).
- Extracted the proven auth handshake from HistorianGrpcReadOrchestrator into
shared Grpc/HistorianGrpcHandshake (reused by read + status + future
browse/metadata). Repointed the IL structural guardrail test to it.
- Diagnostics: round-failure now decodes the native server error + hex/ASCII
preview (HistorianNativeHandshake.DescribeError). This surfaced the live auth
blocker as SEC_E_LOGON_DENIED (0x8009030C) at NTLM round 1 — framing is correct,
the credential did not validate. Probable cause: stale file password or NAM-domain
NTLM restriction (Kerberos/RDP works, NTLM denied; no SPN path over the tunnel).
216 unit tests pass; live gRPC probe passes. Sanitization scan clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Ship SQL command execution over the 2020 WCF aa/Retr/ExeC + aa/Retr/GetR ops:
HistorianClient.ExecuteSqlCommandAsync(sql) -> HistorianSqlResult (columns +
typed rows). String-handle ops reached with the Open2 storage-session GUID
formatted uppercase (the handle format that unlocked GETRP/GETHI).
Chain: Retr.GetV prime -> ExeC(handle, sql, option=0, ref queryHandle) ->
GetR loop. Key gotcha captured: GetR returns FALSE even on success -- the byte
stream is in pResultBuff regardless; false just signals the final page. So the
orchestrator consumes the buffer first, then stops on a false result / empty page.
GetR's pResultBuff is an NRBF-serialized System.Data.DataTable
(SerializationFormat.Xml: members XmlSchema (XSD) + XmlDiffGram (rows)).
BinaryFormatter is removed from .NET 10, so the stream is decoded read-only with
the System.Formats.Nrbf package (NrbfDecoder) + XDocument -- no BinaryFormatter,
no code execution. Values are typed per the XSD type, falling back to string.
Adds: HistorianSqlResult / HistorianSqlColumn / HistorianSqlExecuteOption models,
HistorianSqlResultProtocol (NRBF + diffgram parser), HistorianWcfSqlClient
(ExeC/GetR orchestration with an AVEVA_HISTORIAN_SQL_DUMP diagnostic), dialect +
public API. Golden WcfSqlResultProtocolTests pinned to the real clean GetR stream
for the benign "SELECT 1 AS ProbeValue" (no sensitive data); gated live tests
(single cell + multi-column/multi-row/NULL). Doc: wcf-exec-sql.md; roadmap R1.1
DONE; wall doc + memory updated (incl. the QTB-server-side nuance). 229 tests green.
Note: a raw instrument-wcf capture corrupts a large pResultBuff with MDAS
transport chunk markers (0x9F); the clean contract-level byte[] is dumped via the
AVEVA_HISTORIAN_SQL_DUMP env var for the golden fixture.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Ship tag extended-property reads over the 2020 WCF aa/Retr/GetTepByNm op:
HistorianClient.GetTagExtendedPropertiesAsync(tag) -> name/value pairs.
String-handle op reached with the Open2 storage-session GUID formatted
uppercase (same format that unlocked GETRP/GETHI/ExeC). Routed via the
name-based native path (GetTagExtendedPropertiesByName, server-fetch flag),
not the index-based TagQuery path.
Evidence-backed findings from the capture:
- GetTepByNm (and GetTgByNm) succeed with the uppercase handle -- further
validates the resolved string-handle wall.
- QTB (StartTagQuery) does NOT punch through: captured uppercase, it still
fails server-side (CMdServer::StartActiveTagnamesQuery over the
aahMetadataServer pipe) -- a metadata-server blocker, not handle format.
- R1.6 (localized properties) has NO distinct op (only error-message/UI-text
localization in the managed client); collapses into R1.5. Closed, not throwing.
Wire format (golden-pinned, synthetic bytes -- no dev tag names committed):
- request tagNames = uint count + per-name(uint charCount + UTF-16)
- response = uint tagCount + per-tag(marker + compact-ASCII name +
uint propCount + per-prop(marker + compact-ASCII name + 0x43 VT_BSTR value)
+ trailer); sequence-paged.
Adds: HistorianTagExtendedProperty model, HistorianTagExtendedPropertyProtocol
(codec), HistorianWcfTagExtendedPropertyClient (orchestration), dialect +
public API; golden WcfTagExtendedPropertyProtocolTests (4) + gated live test
(HISTORIAN_TEP_TAG). Tooling: Capture-TagExtendedProperties.ps1,
decode-tag-properties-capture.py, harness tag-extended-properties scenario.
Docs: wcf-tag-extended-properties.md; roadmap R1.5 DONE / R1.6 collapsed;
wall doc + memory updated with the QTB-server-side nuance. 228 tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Execute HCAL roadmap R1.2 (GetRuntimeParameterAsync) end-to-end, and in doing so
discover that the "string-handle wall" blocking R1.1/R1.4/R1.5/R1.6 was a handle
FORMAT bug, not a missing native session/filter registration.
R1.2 (shipped, live-verified):
- Captured native GetRuntimeParameter -> WCF op aa/Stat/GETRP (string-handle op,
GETHI's shape), via scripts/Capture-RuntimeParam.ps1 + instrument-wcf-{write,read}message.
- HistorianRuntimeParameterProtocol serializes pRequestBuff (54 67 01 00 + uint
nameCount + per-name uint charCount + UTF-16) and parses pResponseBuff (version +
uint resultCount + CRetVariant 0x43 VT_BSTR + uint16 len + uint16 charCount + UTF-16).
- IStatusServiceContract2.GetRuntimeParameter (GETRP) op; HistorianWcfStatusClient
passes the Open2 storage-session GUID as the string handle, UPPERCASE.
- Public HistorianClient.GetRuntimeParameterAsync(name) via the dialect.
- Golden WcfRuntimeParameterProtocolTests + gated live test; returns HistorianVersion.
String-handle wall RESOLVED (proven, public APIs deferred):
- The Open2 storage GUID works as the string handle when sent UPPERCASE
(ToString("D").ToUpperInvariant()); earlier "blocked" probes used lowercase.
- Live-probed GETHI (R1.4) -> returns data; ExeC (R1.1) -> Retr.GetV prime -> ExeC ->
GetR returns a BinaryFormatter-serialized .NET DataTable. Gated
StringHandleProbeDiagnosticTests + scripts/Capture-ExecSql.ps1 + exec-sql harness scenario.
- Docs flipped: wcf-string-handle-wall.md RESOLVED banner; roadmap R1.1/R1.4 reachable,
R1.5/R1.6 likely; wcf-status-localhost.md GETRP section.
- R1.1/R1.4 public APIs NOT shipped: ExeC needs a GetR paging loop + a BinaryFormatter-
stream parser (BinaryFormatter is removed from .NET 10); GETHI full-info struct needs
its own capture.
223 unit tests pass; gated live tests green against the local 2020 Historian.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Roadmap M1 R1.7. Filters are set on the native EventQuery object via
AddEventFilter(property, HistorianComparisionType, value) — NOT EventQueryArgs
(time/count/order only). Found via a new harness --dump-type-members command.
Captured the native filtered StartEventQuery pRequestBuff (Capture-EventFilter.ps1 +
harness --event-filter knob) and diffed Equal(0) vs Contains(12) to isolate the
operator field. Filter block (decoded byte-for-byte):
ushort 0 + uint filterCount + uint condCount + uint nameLen + name(UTF-16) +
uint 1 + ushort op + uint 1 + value(0x09-LEN-0x00 compact-ASCII) + byte 0
The filter is REAL, not inert (unlike the analog-summary knobs): a non-matching
predicate returns 0 events; Type=Equal=User.Write returns only User.Write events.
Verified live via both the native harness and the SDK.
- HistorianClient.ReadEventsAsync(start, end, HistorianEventFilter, ct) overload
- HistorianEventFilter + HistorianEventComparison (18 ops, ordinals = native)
- Filter encoding in HistorianEventQueryProtocol (empty-filter path unchanged)
- Golden-byte tests (block match, op field, empty-filter regression) + gated live test
Single string-valued predicate only; multi-filter (OR) / multi-condition (AND via
AddEventFilterCondition) framing is partially captured and not shipped. 216 unit
tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Stands up HistorianTransport.RemoteGrpc end-to-end for the read path,
built on the recovered 2023 R2 gRPC contract (gRPC-Web/HTTP-1.1, port
32565, gzip). The opaque protobuf `bytes` fields carry the SAME native
binary payloads as the 2020 WCF/MDAS path, so the proven serializers and
parsers are reused unchanged.
- Grpc/Protos/*.proto: 6 protoc-validated contracts recovered from
embedded FileDescriptors (authoritative, not guessed).
- Grpc/HistorianGrpcChannelFactory: GrpcWebHandler/HTTP-1.1 channel,
ResolvePort/ResolveAddress, optional TLS + gzip.
- Grpc/HistorianGrpcReadOrchestrator: mirrors the WCF read chain over
gRPC; auth uses HistoryService.ExchangeKey (the gRPC ValCl op).
- Wcf/HistorianNativeHandshake: transport-agnostic Open2 request builder
+ SSPI/Negotiate token loop + response decode, shared by WCF and gRPC.
- Op map (2020 -> gRPC): ValCl->ExchangeKey, Open2->OpenConnection,
StartQuery2->StartQuery, GetNextQueryResultBuffer2->GetNextQueryResultBuffer.
- HistorianClientOptions: DefaultGrpcPort=32565, GrpcUseTls.
- csproj: Google.Protobuf, Grpc.Net.Client(.Web), Grpc.Tools codegen.
Not yet live-verified against a 2023 R2 server: ExchangeKey is the first
thing to revisit if a live server rejects the handshake; the inner byte
payloads are the proven 2020 protocol. Gated live test via
HISTORIAN_GRPC_HOST. 188 unit tests green; build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The dialect / orchestrators were defensively gated on Windows because
HistorianSspiClient previously P/Invoked InitializeSecurityContextW. With
that replaced by NegotiateAuthentication (cross-platform), the gates are
unnecessary. Removed them from:
- Historian2020ProtocolDialect (4 read paths + 3 status helpers)
- HistorianClient.EnsureTagAsync / DeleteTagAsync
- HistorianWcf{Auth,Read,Event,Status,TagWrite}Orchestrator/Helper
- HistorianWcf{HistAddressing,MessageCapture}Behavior
- HistorianWcfBindingFactory (with #pragma on the Named-Pipe builder
which still requires Windows at the BCL level)
Runtime constraint: LocalPipe and RemoteTcpIntegrated transports still
require Windows because NetNamedPipeBinding and the Windows transport
security binding are Windows-only at the BCL level. RemoteTcpCertificate
is now usable from Linux, and ProbeAsync is verified working from a
Debian client (10.100.0.35) against the Windows Historian (10.100.0.48).
171/171 tests still pass on Windows.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two cleanups from the post-EnsureTagAsync punch list — both isolated, no
protocol discovery required.
#89 dead code in Historian2020ProtocolDialect:
- BrowseTagNamesAsync and GetTagMetadataAsync on the dialect both threw
ProtocolEvidenceMissingException, but HistorianClient routes those calls
directly to HistorianWcfTagClient — the dialect overrides were never
reached. Removed both methods. ReadBlocksAsync stays (it's a deliberate
guardrailed entry on the public surface).
#90 explicit-creds tag-metadata path:
- HistorianWcfTagClient.WcfRetrievalSession.ValidateSupportedAuth threw
ProtocolEvidenceMissingException whenever IntegratedSecurity=false AND
UserName/Password were supplied. But the surrounding code already wires
those creds through ApplyWindowsCredential ->
factory.Credentials.Windows.ClientCredential — the validator was just
being conservative about an untested combination.
- Inverted the check: now only rejects the no-auth-at-all combination
(IntegratedSecurity=false + no UserName + no Password). The other three
valid auth shapes pass through to WCF.
Tests: 161 -> 163 (+2). New unit test verifies the no-auth case still
throws; new gated live integration test
GetTagMetadataAsync_ExplicitCredentials_AgainstLocalHistorian exercises the
explicit-creds path when HISTORIAN_USER+HISTORIAN_PASSWORD are set, skips
cleanly otherwise.
CLAUDE.md updated: removed the two now-resolved entries from "Remaining
gaps"; explicit-creds line refined to note the live-verification env-var
requirement.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Executes docs/plans/tcp-connection-validation.md. Full read-only SDK
surface now works against a remote AVEVA Historian over Net.TCP with
Windows transport authentication. 124/124 tests pass; the +10 new live
integration tests in RemoteTcpIntegrationTests.cs are gated by
HISTORIAN_REMOTE_TCP_HOST + HISTORIAN_REMOTE_TCP_TAG.
Two SDK bugs found while executing the plan:
1. Historian2020ProtocolDialect.ReadRawAsync / ReadAggregateAsync /
ReadAtTimeAsync / ReadEventsAsync had explicit
`if (_options.Transport != HistorianTransport.LocalPipe) return Missing<T>`
guards. These were a guardrail from before the orchestrators handled
TCP; the orchestrators have always used CreateBindingPair(options)
which dispatches on transport correctly. Gates removed.
2. HistorianWcfStatusClient and HistorianWcfEventOrchestrator hardcoded
HistorianWcfBindingFactory.CreatePipeEndpointAddress for the auxiliary
services (Stat, Trx, Retr). Worked for LocalPipe; for TCP it produced
an EndpointAddress with scheme net.pipe attached to a TCP binding
(channel factory rejected the URI). Worse, when only the endpoint was
transport-aware, the binding still requested a Windows-transport-
security upgrade that the Stat endpoint over TCP doesn't support
(auxiliaries don't repeat the auth — the Hist session is already
authenticated). Added two helpers:
- HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(options, name)
-> net.pipe for LocalPipe, net.tcp for remote
- HistorianWcfBindingFactory.CreateAuxiliaryBinding(options)
-> NamedPipe for LocalPipe, plain MdasNetTcpBinding for remote
Both call sites updated.
Live verification against the remote (probed previously in prior
sessions; reachability re-confirmed today):
- ProbeAsync over RemoteTcpIntegrated and RemoteTcpCertificate
- ReadRawAsync (8 samples returned for SysTimeSec)
- ReadAggregateAsync (TimeWeightedAverage, 1-min cycle, 10-min window)
- ReadAtTimeAsync (3 timestamps)
- BrowseTagNamesAsync (finds the test tag)
- GetTagMetadataAsync (full metadata populated)
- ReadEventsAsync (chain runs without throwing)
- GetConnectionStatusAsync (ConnectedToServer=true)
- GetSystemParameterAsync (HistorianVersion="20,0,000,000")
The default 'NT SERVICE\aahClientAccessPoint' SPN turned out to work
for the remote too — discovery workstream A (SPN-finding) was not
needed in practice.
README and the TCP plan doc updated to reflect the executed status.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Full read-only SDK (src/AVEVA.Historian.Client) implementing the CLAUDE.md required
surface against AVEVA Historian's binary WCF protocol — no native AVEVA runtime
dependency. All operations live-verified against a local Historian:
- ProbeAsync, ReadRawAsync, ReadAggregateAsync, ReadAtTimeAsync, ReadEventsAsync
- BrowseTagNamesAsync, GetTagMetadataAsync (17 native data-type codes mapped)
- GetConnectionStatusAsync, GetStoreForwardStatusAsync, GetSystemParameterAsync
- 108/108 unit + integration tests pass
Includes the reverse-engineering toolkit (tools/AVEVA.Historian.ReverseEngineering)
used to decode the protocol: WCF probes, IL inspection via dnlib, and IL-rewrite
instrumentation (instrument-wcf-{write,read}message etc.) plus the .NET Framework
trace harness (tools/AVEVA.Historian.NativeTraceHarness) for parity testing.
Sanitized handoff evidence under docs/reverse-engineering/. Native AVEVA binaries
(current/, aveva-install-x64/, aveva-install-x86/) are gitignored — fetch separately
from the AVEVA installer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Joseph Doherty