histsdk/README.md

histsdk

Pure managed .NET 10 client for AVEVA Historian's binary WCF protocol. The production SDK has no dependency on aahClientManaged.dll, aahClient.dll, or any other AVEVA native runtime — the wire protocol is reverse-engineered and re-implemented in C#.

The supported surface (per CLAUDE.md):

Operation	Status
ProbeAsync	live-verified
ReadRawAsync	live-verified
ReadAggregateAsync	live-verified across all 16 retrieval modes
ReadAtTimeAsync	live-verified
ReadEventsAsync	live-verified (typed event + 31-property property bag)
BrowseTagNamesAsync	live-verified
GetTagMetadataAsync	live-verified for 17 distinct native data-type codes
GetConnectionStatusAsync	synthesized from authenticated probe (matches native semantic)
GetStoreForwardStatusAsync	gRPC: measured idle-state (live-verified — contacts server, reports ErrorOccurred when unreachable; active-SF magnitude is D2-gated). Non-gRPC: synthesized defaults
GetSystemParameterAsync	live-verified via Stat/GetSystemParameter
EnsureTagAsync	live-verified for analog Float/Double/Int2/Int4/UInt4; ApplyScaling=true persists distinct MinRaw/MaxRaw
DeleteTagAsync	live-verified

Out of scope: writing samples (AddS2 is architecturally blocked — the server's runtime cache only ingests from configured IOServer / Application Server pipelines), store-forward write, configuration changes, discrete/string tag creation (native AddTag rejects them).

Quick start

using AVEVA.Historian.Client;
using AVEVA.Historian.Client.Models;

await using HistorianClient client = new(new HistorianClientOptions
{
    Host = "localhost",
    IntegratedSecurity = true,
    Transport = HistorianTransport.LocalPipe,
});

DateTime endUtc   = DateTime.UtcNow;
DateTime startUtc = endUtc - TimeSpan.FromMinutes(10);

await foreach (HistorianSample sample in client.ReadRawAsync(
    "SysTimeSec", startUtc, endUtc, maxValues: 100))
{
    Console.WriteLine($"{sample.TimestampUtc:o}  {sample.NumericValue}  Q={sample.Quality}");
}

For a remote Historian over Net.TCP set Transport = HistorianTransport.RemoteTcpIntegrated and Host to the server hostname. Both RemoteTcpIntegrated (Windows transport auth) and RemoteTcpCertificate (server-cert TLS) are now live-verified for ProbeAsync; RemoteTcpIntegrated is additionally live-verified for the full read / browse / metadata / event / status surface.

Transport matrix (WCF vs gRPC)

The SDK speaks two wire protocols. WCF is the 2020 binary MDAS protocol over Net.TCP 32568 (transports LocalPipe, RemoteTcpIntegrated, RemoteTcpCertificate); gRPC is the 2023 R2 HCAL transport over HTTP/2 32565 (transport RemoteGrpc). The core read chain reuses the same native Open2 buffers and SSPI tokens on both — on gRPC they simply ride inside protobuf bytes fields — so reads are at parity. The surfaces diverge at the edges.

Legend: ✅ tooled + live-verified · ⚠️ tooled, partial/synthesized · 🧪 tooled + routed but sandbox-gated (mutates server state, not yet run destructively against a live box) · 🔌 the gRPC server exposes the RPC (recovered in Grpc/Protos/*.proto) but the SDK doesn't drive it yet — untooled/uncaptured, not a protocol gap · ⛔ tooled but server-walled (the request rides the RPC but the server faults on an unmet precondition) · ❌ unavailable on that transport.

Operation	WCF	gRPC	Notes
ProbeAsync	✅	✅
ReadRawAsync	✅	✅
ReadAggregateAsync	✅	✅	all 16 retrieval modes
ReadAtTimeAsync	✅	✅
BrowseTagNamesAsync	✅	✅
GetTagMetadataAsync	✅	✅
GetSystemParameterAsync	✅	✅
AddHistoricalValuesAsync	❌	✅	historical/backfill writes ride HistoryService.AddStreamValues; non-gRPC throws ProtocolEvidenceMissingException
GetServerTimeZoneAsync	❌	✅	2020 GetSystemTimeZoneName is a client-side stub (empty); WCF throws
GetStoreForwardStatusAsync	⚠️	✅	gRPC contacts the server (measured idle-state, reports ErrorOccurred); WCF returns synthesized all-false. Active-SF magnitude is D2-gated on both
GetRuntimeParameterAsync	✅	✅	tooled + live-verified over gRPC (StatusService.GetRuntimeParameter, the 2020 GETRP buffers ride unchanged)
GetTagExtendedPropertiesAsync	✅	✅	tooled + live-verified over gRPC (RetrievalService.GetTagExtendedPropertiesFromName, the GetTepByNm buffers ride unchanged)
ExecuteSqlCommandAsync	✅	⛔	gRPC request rides RetrievalService.ExecuteSqlCommand, but the server-side CSrvDbConnection.ExecuteSqlCommand faults (IndexOutOfRange, native err 38) — an unmet DB-connection precondition; bounded behind ProtocolEvidenceMissingException. Use WCF
ReadEventsAsync	✅	🔌	gRPC StartEventQuery/GetNextEventQueryResultBuffer/EndEventQuery recovered, but the read needs the full CM_EVENT registration state machine (RTag2+EnsT2) ported — not yet tooled
SendEventAsync	✅	🔌	rides AddStreamValues family; no distinct event-send RPC, framing uncaptured over gRPC
EnsureTagAsync / DeleteTagAsync / RenameTagsAsync	✅	🧪	tooled + routed over gRPC (HistoryService.EnsureTags / DeleteTags / StartJob, write-enabled 0x401 session, WCF serializers reused); sandbox-gated — not yet run destructively against a live box
AddTagExtendedPropertiesAsync	✅	🧪	tooled + routed over gRPC (HistoryService.AddTagExtendedProperties, write-enabled session); sandbox-gated. gRPC also exposes DeleteTagExtendedProperties (WCF delete was server-blocked)
GetConnectionStatusAsync	✅	❌	synthesized from an authenticated probe — no dedicated RPC on either transport (gRPC PingServer/GetHistorianConsoleStatus could synthesize it)
ReadBlocksAsync	❌	❌	StartBlockRetrievalQuery never captured on either transport — throws ProtocolEvidenceMissingException

In short: WCF is the broad, mature surface (every config write, events, SQL, and all reads), while gRPC is the narrower tooled surface — but the 2023 R2 gRPC contract is actually a superset of WCF. The recovered config RPCs carry the same opaque bytes buffers the existing WCF serializers already emit, keyed by the same strHandle/uiHandle session handle the read path obtains — confirmed by tooling the read-side config ops (GetRuntimeParameter, GetTagExtendedProperties) live: the WCF buffers ride the gRPC RPC unchanged and the server accepts them. Two caveats surfaced when capturing the rest: ExecuteSqlCommand is server-walled (the front-door CSrvDbConnection faults on a DB-connection precondition the managed session doesn't establish — the same class of wall as OpenStorageConnection), and ReadEvents needs the CM_EVENT registration state machine ported. The remaining 🔌 rows are capture-and-wire items (route the existing serializer into a gRPC orchestrator + live-capture), not protocol-discovery — but per "capture first, never guess wire bytes" they stay untooled until each is verified live. The natural production pattern today remains WCF for config/writes and RemoteGrpc for reads + AddHistoricalValuesAsync.

A 2023 R2 server reports History interface version 12 (vs. 11 on 2020). The connect-time version gate accepts both — they are byte-compatible — so gRPC works against a v12 server with the default VerifyServerInterfaceVersion=true; no opt-out is required.

Resilience subsystems (M4)

Two pure client-side subsystems layered on top of HistorianClient. They use only the public SDK surface — no extra reverse-engineering, no server-side protocol — and are fully unit-tested without a live server.

Store-and-forward (AVEVA.Historian.Client.StoreForward)

A durable local outbox that buffers writes when the Historian is unreachable and replays them on reconnect. HistorianStoreForwardWriter wraps a HistorianClient (or any IHistorianWriteSink) and persists each enqueued write to an IHistorianOutboxStore — FileHistorianOutboxStore (crash-durable, atomic JSON-per-entry, FIFO by filename sequence, corrupt-file quarantine) or InMemoryHistorianOutboxStore. A background drain loop retries with FIFO head-of-line ordering, optional MaxDeliveryAttempts dead-lettering, and a DropOldest/Reject overflow policy.

using AVEVA.Historian.Client.StoreForward;

await using HistorianStoreForwardWriter writer = new(
    client,
    new FileHistorianOutboxStore(@"C:\ProgramData\histsdk\outbox"));
await writer.StartAsync();                       // background drain on reconnect

await writer.EnqueueHistoricalValuesAsync("MyTag",
    [new HistorianHistoricalValue(DateTime.UtcNow, 42.0)]);   // returns immediately, durable

HistorianStoreForwardStatusSnapshot status = await writer.GetStatusAsync();
// status.PendingCount / .Storing / .ErrorOccurred — mirrors the server SF semantics

This is a pragmatic outbox, not the bit-faithful native SF cache (Forward*Snapshot decode), which stays deferred.

Multi-historian redundancy (AVEVA.Historian.Client.Redundancy)

HistorianRedundantClient fronts N members as one logical client. Reads fail over to the next healthy member in priority order (streaming reads fail over only before the first row — mid-stream failures propagate to avoid duplicated/skipped rows); writes fan out (AllMembers/PreferredOnly) under an All/Any acknowledgement policy and return a per-member HistorianRedundantWriteResult. A failure-threshold demotion + background watchdog restores recovered members.

using AVEVA.Historian.Client.Redundancy;

await using HistorianRedundantClient cluster = new(
[
    new HistorianClientMember("primary",   primaryClient),
    new HistorianClientMember("secondary", secondaryClient),
]);
await cluster.StartAsync();                       // health watchdog

await foreach (HistorianSample s in cluster.ReadRawAsync("MyTag", startUtc, endUtc, 100))
{
    // served by the first healthy member; transparently fails over on connect
}

HistorianRedundantWriteResult w = await cluster.AddHistoricalValuesAsync("MyTag",
    [new HistorianHistoricalValue(DateTime.UtcNow, 42.0)]);   // fanned out across members

Backing a member's writes with a HistorianStoreForwardWriter gives the pragmatic equivalent of native ReSyncTags: a down member buffers locally and replays on recovery.

Build & test

dotnet build  .\Histsdk.slnx --no-restore
dotnet test   .\Histsdk.slnx --no-build --logger "console;verbosity=minimal"

Run a single test class:

dotnet test .\Histsdk.slnx --no-build --filter "FullyQualifiedName~WcfDataQueryProtocolTests"

Live integration tests (tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs) are gated and skip cleanly without these env vars:

$env:HISTORIAN_HOST       = 'localhost'
$env:HISTORIAN_TEST_TAG   = 'SysTimeSec'      # any tag the server has data for
# Optional for non-integrated auth:
$env:HISTORIAN_USER, $env:HISTORIAN_PASSWORD
$env:HISTORIAN_TAG_FILTER = 'Sys*'             # or any LIKE-pattern

The 2023 R2 RemoteGrpc transport has its own gated live suite (HistorianGrpcIntegrationTests) covering the full tooled gRPC surface — probe, raw / aggregate (incl. multiple retrieval modes) / at-time reads, browse, metadata, system-parameter, server time-zone, and measured store-forward status. It skips cleanly unless HISTORIAN_GRPC_HOST is set:

$env:HISTORIAN_GRPC_HOST  = 'my-2023r2-host'  # gates the gRPC suite
$env:HISTORIAN_TEST_TAG   = 'SysTimeSec'
$env:HISTORIAN_USER, $env:HISTORIAN_PASSWORD  # required for the authenticated ops
# Optional:
$env:HISTORIAN_GRPC_PORT     = '32565'        # default 32565
$env:HISTORIAN_GRPC_TLS      = 'true'         # gRPC over TLS
$env:HISTORIAN_GRPC_DNSID    = 'my-2023r2-host'  # cert DNS name when connecting by IP
$env:HISTORIAN_GRPC_TIMEOUT  = '120'          # per-call deadline (s); raise for slow links
$env:HISTORIAN_WRITE_SANDBOX_TAG = 'MyFloatTag'  # gates the AddHistoricalValues write test
$env:HISTORIAN_GRPC_WRITE_SANDBOX_TAG = 'SandboxTag'  # gates the DESTRUCTIVE tag create/rename/delete lifecycle test

The aggregate tests self-calibrate their query window from a real raw sample, so they pass against an idle 2023 R2 server (no recent collection) as well as a live-collecting one.

Repository layout

src/AVEVA.Historian.Client/    Production SDK — pure managed .NET 10. No native AVEVA dependency.
tests/                          Unit tests (golden-byte / round-trip) + gated live integration tests.
tools/                          Reverse-engineering tooling (NOT shipped):
  AVEVA.Historian.ReverseEngineering/  .NET 10 CLI: WCF probes, dnlib IL inspection,
                                       IL-rewrite instrumentation (instrument-wcf-{write,read}message,
                                       instrument-openconnection*, instrument-startdataquery, etc.).
  AVEVA.Historian.NativeTraceHarness/  .NET Framework harness that loads aahClientManaged.dll for
                                       byte-for-byte parity testing against the native wrapper.
  AVEVA.Historian.NetFxWcfProbe/       .NET Framework WCF probe for ruling out .NET 10-only differences.
  AVEVA.Historian.ReverseInstrumentation/  Helper assembly injected into IL-rewritten wrapper copies.
  AVEVA.Historian.WcfCaptureServer/    Fake server for endpoint experiments.
scripts/                        PowerShell + Frida runners + Python decoders for capture analysis.
fixtures/protocol/              Sanitized golden-byte fixtures.
docs/reverse-engineering/       Sanitized handoff evidence; commit-safe summaries only.

Native AVEVA binaries (current/, aveva-install-x64/, aveva-install-x86/) are gitignored. Each developer fetches their own copy from the AVEVA installer; we neither modify nor redistribute them.

Documentation

Read in order before non-trivial work:

Doc	Purpose
AGENTS.md	Standing project constraints and safety rules.
CLAUDE.md	Build commands, code architecture, the active protocol blocker (now resolved) and SDK surface.
docs/reverse-engineering/handoff.md	Decision record + decoded protocol evidence (binding shapes, descriptor layouts, Open2 v6 response, event-row wire format, property-bag value encoding).
instructions.md	Original plan + decision record.

Architecture

Three intentionally decoupled subsystems under src/AVEVA.Historian.Client/:

• HistorianClient + HistorianClientOptions — public façade. Validates inputs; delegates reads to Historian2020ProtocolDialect; delegates probe / tag metadata / browse / status helpers to the WCF layer.
• Wcf/ — managed WCF/MDAS layer. Custom MdasMessageEncoder wraps SOAP 1.2 + WS-Addressing 1.0 in AVEVA's application/x-mdas framing. Three binding flavors via HistorianWcfBindingFactory: plain MDAS over Net.NamedPipe (local), MDAS + Windows transport (remote /Hist-Integrated), MDAS + certificate (remote /HistCert). Service contracts in Wcf/Contracts/ mirror the server-side WCF surface (versioned per native interface — IHistoryServiceContract, IRetrievalServiceContract2..4, IStatusServiceContract2, etc.).
• Protocol/ — binary frame layer. Historian2020ProtocolDialect is the version-anchored bridge between the public façade and the WCF + frame layers. Methods without protocol evidence throw ProtocolEvidenceMissingException rather than guessing wire bytes.
• Models/ — public DTOs and enums. HistorianSample, HistorianAggregateSample, HistorianEvent, HistorianTagMetadata, HistorianDataType, RetrievalMode, HistorianTransport, etc.

Read flow end-to-end (live-verified against localhost):

Hist.GetV → Hist.ValCl × N → Hist.Open2 → Retr.GetV → Retr.IsOriginalAllowed →
Retr.StartQuery2 → loop Retr.GetNextQueryResultBuffer2 → typed HistorianSample rows

Event flow end-to-end (live-verified):

Hist.GetV → Hist.ValCl × N → Hist.Open2 → Hist.UpdC3 → 6× Stat.GetSystemParameter →
Hist.RTag2 → Stat.GetSystemParameter(AllowRenameTags) → Trx.GetV → Stat.GetV →
Retr.GetV → Hist.EnsT2(CmEventTagId) → Retr.StartEventQuery →
loop Retr.GetNextEventQueryResultBuffer → typed HistorianEvent rows with
property dictionary → Retr.EndEventQuery → Hist.Close2

Safety

• Production code under src/ must remain pure managed .NET 10 with no native AVEVA reference. Reverse-engineering harnesses under tools/ may reference native binaries.
• Never commit credentials, hostnames, user names, customer tag names, or raw packet captures. *.ndjson, current/, aveva-install-*/, and artifacts/ are gitignored precisely because they accumulate identity-bearing runtime data.
• Methods without protocol evidence throw ProtocolEvidenceMissingException. Do not stub fake behavior — leave them throwing until evidence supports an implementation.

Status

313 unit + live integration tests pass (dotnet test --logger "console;verbosity=minimal"). Full SDK surface — reads, browse, metadata, status, plus the two write ops (EnsureTagAsync / DeleteTagAsync) — verified end-to-end against both a local Historian (LocalPipe) and a remote Historian (RemoteTcpIntegrated over Net.TCP with Windows transport auth). RemoteTcpCertificate ProbeAsync is live-verified; deeper coverage over the cert transport plus the explicit-credentials path await additional verification.

The 2023 R2 RemoteGrpc transport's full tooled surface is live-verified against a real 2023 R2 server: probe, raw / aggregate (TimeWeightedAverage + Minimum/MaximumWithTime + BestFit) / at-time reads, browse, metadata, system-parameter, server time-zone, and measured store-forward status — plus the gRPC-only AddHistoricalValuesAsync backfill write. The remaining gRPC config ops are exposed by the server but untooled (see the transport matrix above).