# histsdk Pure managed .NET 10 client for AVEVA Historian's binary WCF protocol. The production SDK has no dependency on `aahClientManaged.dll`, `aahClient.dll`, or any other AVEVA native runtime — the wire protocol is reverse-engineered and re-implemented in C#. The supported surface (per [`CLAUDE.md`](CLAUDE.md)): | Operation | Status | |---|---| | `ProbeAsync` | live-verified | | `ReadRawAsync` | live-verified | | `ReadAggregateAsync` | live-verified across all 16 retrieval modes | | `ReadAtTimeAsync` | live-verified | | `ReadEventsAsync` | live-verified (typed event + 31-property property bag) | | `BrowseTagNamesAsync` | live-verified | | `GetTagMetadataAsync` | live-verified for 17 distinct native data-type codes | | `GetConnectionStatusAsync` | synthesized from authenticated probe (matches native semantic) | | `GetStoreForwardStatusAsync` | gRPC: measured idle-state (live-verified — contacts server, reports `ErrorOccurred` when unreachable; active-SF magnitude is D2-gated). Non-gRPC: synthesized defaults | | `GetSystemParameterAsync` | live-verified via `Stat/GetSystemParameter` | | `EnsureTagAsync` | live-verified for analog Float/Double/Int2/Int4/UInt4; `ApplyScaling=true` persists distinct MinRaw/MaxRaw | | `DeleteTagAsync` | live-verified | Out of scope: writing samples (`AddS2` is architecturally blocked — the server's runtime cache only ingests from configured IOServer / Application Server pipelines), store-forward write, configuration changes, discrete/string tag creation (native `AddTag` rejects them). ## Quick start ```csharp using AVEVA.Historian.Client; using AVEVA.Historian.Client.Models; await using HistorianClient client = new(new HistorianClientOptions { Host = "localhost", IntegratedSecurity = true, Transport = HistorianTransport.LocalPipe, }); DateTime endUtc = DateTime.UtcNow; DateTime startUtc = endUtc - TimeSpan.FromMinutes(10); await foreach (HistorianSample sample in client.ReadRawAsync( "SysTimeSec", startUtc, endUtc, maxValues: 100)) { Console.WriteLine($"{sample.TimestampUtc:o} {sample.NumericValue} Q={sample.Quality}"); } ``` For a remote Historian over Net.TCP set `Transport = HistorianTransport.RemoteTcpIntegrated` and `Host` to the server hostname. Both `RemoteTcpIntegrated` (Windows transport auth) and `RemoteTcpCertificate` (server-cert TLS) are now live-verified for `ProbeAsync`; `RemoteTcpIntegrated` is additionally live-verified for the full read / browse / metadata / event / status surface. ## Transport matrix (WCF vs gRPC) The SDK speaks two wire protocols. **WCF** is the 2020 binary MDAS protocol over Net.TCP `32568` (transports `LocalPipe`, `RemoteTcpIntegrated`, `RemoteTcpCertificate`); **gRPC** is the 2023 R2 HCAL transport over HTTP/2 `32565` (transport `RemoteGrpc`). The core read chain reuses the *same* native Open2 buffers and SSPI tokens on both — on gRPC they simply ride inside protobuf `bytes` fields — so reads are at parity. The surfaces diverge at the edges. Legend: ✅ tooled + live-verified · ⚠️ tooled, partial/synthesized · 🧪 tooled + routed but **sandbox-gated** (mutates server state, not yet run destructively against a live box) · 🔌 **the gRPC server exposes the RPC (recovered in `Grpc/Protos/*.proto`) but the SDK doesn't drive it yet** — untooled/uncaptured, *not* a protocol gap · ⛔ tooled but **server-walled** (the request rides the RPC but the server faults on an unmet precondition) · ❌ unavailable on that transport. | Operation | WCF | gRPC | Notes | |---|:---:|:---:|---| | `ProbeAsync` | ✅ | ✅ | | | `ReadRawAsync` | ✅ | ✅ | | | `ReadAggregateAsync` | ✅ | ✅ | all 16 retrieval modes | | `ReadAtTimeAsync` | ✅ | ✅ | | | `BrowseTagNamesAsync` | ✅ | ✅ | | | `GetTagMetadataAsync` | ✅ | ✅ | | | `GetSystemParameterAsync` | ✅ | ✅ | | | `AddHistoricalValuesAsync` | ❌ | ✅ | historical/backfill writes ride `HistoryService.AddStreamValues`; non-gRPC throws `ProtocolEvidenceMissingException` | | `GetServerTimeZoneAsync` | ❌ | ✅ | 2020 `GetSystemTimeZoneName` is a client-side stub (empty); WCF throws | | `GetStoreForwardStatusAsync` | ⚠️ | ✅ | gRPC contacts the server (measured idle-state, reports `ErrorOccurred`); WCF returns synthesized all-false. Active-SF magnitude is D2-gated on both | | `GetRuntimeParameterAsync` | ✅ | ✅ | tooled + live-verified over gRPC (`StatusService.GetRuntimeParameter`, the 2020 `GETRP` buffers ride unchanged) | | `GetTagExtendedPropertiesAsync` | ✅ | ✅ | tooled + live-verified over gRPC (`RetrievalService.GetTagExtendedPropertiesFromName`, the `GetTepByNm` buffers ride unchanged) | | `ExecuteSqlCommandAsync` | ✅ | ⛔ | gRPC request rides `RetrievalService.ExecuteSqlCommand`, but the server-side `CSrvDbConnection.ExecuteSqlCommand` faults (`IndexOutOfRange`, native err 38) — an unmet DB-connection precondition; bounded behind `ProtocolEvidenceMissingException`. Use WCF | | `ReadEventsAsync` | ✅ | ⚠️ | tooled + routed over gRPC: the full CM_EVENT registration replay (`UpdateClientStatus`→`RegisterTags`→`EnsureTags` + discovery probes) runs and `StartEventQuery` succeeds, but `GetNextEventQueryResultBuffer` **long-polls** on no data (it blocks to the deadline rather than returning the synchronous 5-byte code-85 terminal the WCF op gives). The read is **hard-bounded** (≤30s) and throws `ProtocolEvidenceMissingException` on the no-row path rather than assert a false empty. Row-level retrieval is **not yet live-verified** — the dev box holds no events; pending a capture against an event-bearing 2023 R2 server. Use WCF for event reads | | `SendEventAsync` | ✅ | 🔌 | rides `AddStreamValues` family; no distinct event-send RPC, framing uncaptured over gRPC | | `EnsureTagAsync` / `DeleteTagAsync` / `RenameTagsAsync` | ✅ | ✅ | live-verified 2026-06-22 over gRPC (`HistoryService.EnsureTags` / `DeleteTags` / `StartJob`, write-enabled 0x401 session, WCF serializers reused) via a self-cleaning sandbox-tag lifecycle. Rename is an async StartJob — transiently rejectable right after create, so callers should retry | | `AddTagExtendedPropertiesAsync` | ✅ | ✅ | live-verified 2026-06-22 over gRPC (`HistoryService.AddTagExtendedProperties`, write-enabled session). NOTE: reading a written prop back via `GetTagExtendedPropertiesAsync` can hit a shared-parser evidence gap (value marker `0x01` vs the captured compact-string `0x09`); the write itself is confirmed. gRPC also exposes `DeleteTagExtendedProperties` (WCF delete was server-blocked) | | `GetConnectionStatusAsync` | ✅ | ❌ | synthesized from an authenticated probe — no dedicated RPC on either transport (gRPC `PingServer`/`GetHistorianConsoleStatus` could synthesize it) | | `ReadBlocksAsync` | ❌ | ❌ | `StartBlockRetrievalQuery` never captured on either transport — throws `ProtocolEvidenceMissingException` | In short: **WCF is the broad, mature surface** (every config write, events, SQL, and all reads), while **gRPC is the narrower *tooled* surface** — but the 2023 R2 gRPC *contract* is actually a **superset** of WCF. The recovered config RPCs carry the **same opaque `bytes` buffers the existing WCF serializers already emit**, keyed by the same `strHandle`/`uiHandle` session handle the read path obtains — confirmed by tooling the read-side config ops (`GetRuntimeParameter`, `GetTagExtendedProperties`) live: the WCF buffers ride the gRPC RPC unchanged and the server accepts them. Two caveats surfaced when capturing the rest: `ExecuteSqlCommand` is **server-walled** (the front-door `CSrvDbConnection` faults on a DB-connection precondition the managed session doesn't establish — the same *class* of wall as `OpenStorageConnection`), and `ReadEvents` is now tooled over gRPC (the CM_EVENT registration state machine is ported and `StartEventQuery` succeeds) but its row retrieval is not yet live-verified: the gRPC server long-polls `GetNextEventQueryResultBuffer` on no data instead of returning the WCF code-85 terminal, so on the idle dev box the bounded read throws `ProtocolEvidenceMissingException` rather than fabricate an empty result — confirming rows awaits an event-bearing 2023 R2 server. The remaining 🔌 row (`SendEventAsync`) is a **capture-and-wire** item (route the existing serializer into a gRPC orchestrator + live-capture), not protocol-discovery — but per "capture first, never guess wire bytes" it stays untooled until verified live. The natural production pattern today: `RemoteGrpc` now covers reads, `AddHistoricalValuesAsync`, and the tag-config writes (create/delete/rename/extended properties, live-verified) — use WCF for SQL, events, and reading extended properties back until those gRPC gaps close. > A 2023 R2 server reports History interface version 12 (vs. 11 on 2020). The > connect-time version gate accepts both — they are byte-compatible — so gRPC > works against a v12 server with the default `VerifyServerInterfaceVersion=true`; > no opt-out is required. ## Resilience subsystems (M4) Two **pure client-side** subsystems layered on top of `HistorianClient`. They use only the public SDK surface — no extra reverse-engineering, no server-side protocol — and are fully unit-tested without a live server. ### Store-and-forward (`AVEVA.Historian.Client.StoreForward`) A durable local outbox that buffers writes when the Historian is unreachable and replays them on reconnect. `HistorianStoreForwardWriter` wraps a `HistorianClient` (or any `IHistorianWriteSink`) and persists each enqueued write to an `IHistorianOutboxStore` — `FileHistorianOutboxStore` (crash-durable, atomic JSON-per-entry, FIFO by filename sequence, corrupt-file quarantine) or `InMemoryHistorianOutboxStore`. A background drain loop retries with FIFO head-of-line ordering, optional `MaxDeliveryAttempts` dead-lettering, and a `DropOldest`/`Reject` overflow policy. ```csharp using AVEVA.Historian.Client.StoreForward; await using HistorianStoreForwardWriter writer = new( client, new FileHistorianOutboxStore(@"C:\ProgramData\histsdk\outbox")); await writer.StartAsync(); // background drain on reconnect await writer.EnqueueHistoricalValuesAsync("MyTag", [new HistorianHistoricalValue(DateTime.UtcNow, 42.0)]); // returns immediately, durable HistorianStoreForwardStatusSnapshot status = await writer.GetStatusAsync(); // status.PendingCount / .Storing / .ErrorOccurred — mirrors the server SF semantics ``` This is a pragmatic outbox, **not** the bit-faithful native SF cache (`Forward*Snapshot` decode), which stays deferred. ### Multi-historian redundancy (`AVEVA.Historian.Client.Redundancy`) `HistorianRedundantClient` fronts N members as one logical client. Reads fail over to the next healthy member in priority order (streaming reads fail over only *before the first row* — mid-stream failures propagate to avoid duplicated/skipped rows); writes fan out (`AllMembers`/`PreferredOnly`) under an `All`/`Any` acknowledgement policy and return a per-member `HistorianRedundantWriteResult`. A failure-threshold demotion + background watchdog restores recovered members. ```csharp using AVEVA.Historian.Client.Redundancy; await using HistorianRedundantClient cluster = new( [ new HistorianClientMember("primary", primaryClient), new HistorianClientMember("secondary", secondaryClient), ]); await cluster.StartAsync(); // health watchdog await foreach (HistorianSample s in cluster.ReadRawAsync("MyTag", startUtc, endUtc, 100)) { // served by the first healthy member; transparently fails over on connect } HistorianRedundantWriteResult w = await cluster.AddHistoricalValuesAsync("MyTag", [new HistorianHistoricalValue(DateTime.UtcNow, 42.0)]); // fanned out across members ``` Backing a member's writes with a `HistorianStoreForwardWriter` gives the pragmatic equivalent of native ReSyncTags: a down member buffers locally and replays on recovery. ## Build & test ```powershell dotnet build .\Histsdk.slnx --no-restore dotnet test .\Histsdk.slnx --no-build --logger "console;verbosity=minimal" ``` Run a single test class: ```powershell dotnet test .\Histsdk.slnx --no-build --filter "FullyQualifiedName~WcfDataQueryProtocolTests" ``` Live integration tests (`tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs`) are gated and skip cleanly without these env vars: ```powershell $env:HISTORIAN_HOST = 'localhost' $env:HISTORIAN_TEST_TAG = 'SysTimeSec' # any tag the server has data for # Optional for non-integrated auth: $env:HISTORIAN_USER, $env:HISTORIAN_PASSWORD $env:HISTORIAN_TAG_FILTER = 'Sys*' # or any LIKE-pattern ``` The 2023 R2 `RemoteGrpc` transport has its own gated live suite (`HistorianGrpcIntegrationTests`) covering the full tooled gRPC surface — probe, raw / aggregate (incl. multiple retrieval modes) / at-time reads, browse, metadata, system-parameter, server time-zone, and measured store-forward status. It skips cleanly unless `HISTORIAN_GRPC_HOST` is set: ```powershell $env:HISTORIAN_GRPC_HOST = 'my-2023r2-host' # gates the gRPC suite $env:HISTORIAN_TEST_TAG = 'SysTimeSec' $env:HISTORIAN_USER, $env:HISTORIAN_PASSWORD # required for the authenticated ops # Optional: $env:HISTORIAN_GRPC_PORT = '32565' # default 32565 $env:HISTORIAN_GRPC_TLS = 'true' # gRPC over TLS $env:HISTORIAN_GRPC_DNSID = 'my-2023r2-host' # cert DNS name when connecting by IP $env:HISTORIAN_GRPC_TIMEOUT = '120' # per-call deadline (s); raise for slow links $env:HISTORIAN_WRITE_SANDBOX_TAG = 'MyFloatTag' # gates the AddHistoricalValues write test $env:HISTORIAN_GRPC_WRITE_SANDBOX_TAG = 'SandboxTag' # gates the DESTRUCTIVE tag create/rename/delete lifecycle test ``` The aggregate tests self-calibrate their query window from a real raw sample, so they pass against an idle 2023 R2 server (no recent collection) as well as a live-collecting one. ## Repository layout ``` src/AVEVA.Historian.Client/ Production SDK — pure managed .NET 10. No native AVEVA dependency. tests/ Unit tests (golden-byte / round-trip) + gated live integration tests. tools/ Reverse-engineering tooling (NOT shipped): AVEVA.Historian.ReverseEngineering/ .NET 10 CLI: WCF probes, dnlib IL inspection, IL-rewrite instrumentation (instrument-wcf-{write,read}message, instrument-openconnection*, instrument-startdataquery, etc.). AVEVA.Historian.NativeTraceHarness/ .NET Framework harness that loads aahClientManaged.dll for byte-for-byte parity testing against the native wrapper. AVEVA.Historian.NetFxWcfProbe/ .NET Framework WCF probe for ruling out .NET 10-only differences. AVEVA.Historian.ReverseInstrumentation/ Helper assembly injected into IL-rewritten wrapper copies. AVEVA.Historian.WcfCaptureServer/ Fake server for endpoint experiments. scripts/ PowerShell + Frida runners + Python decoders for capture analysis. fixtures/protocol/ Sanitized golden-byte fixtures. docs/reverse-engineering/ Sanitized handoff evidence; commit-safe summaries only. ``` Native AVEVA binaries (`current/`, `aveva-install-x64/`, `aveva-install-x86/`) are **gitignored**. Each developer fetches their own copy from the AVEVA installer; we neither modify nor redistribute them. ## Documentation Read in order before non-trivial work: | Doc | Purpose | |---|---| | [`AGENTS.md`](AGENTS.md) | Standing project constraints and safety rules. | | [`CLAUDE.md`](CLAUDE.md) | Build commands, code architecture, the active protocol blocker (now resolved) and SDK surface. | | [`docs/reverse-engineering/handoff.md`](docs/reverse-engineering/handoff.md) | Decision record + decoded protocol evidence (binding shapes, descriptor layouts, Open2 v6 response, event-row wire format, property-bag value encoding). | | [`instructions.md`](instructions.md) | Original plan + decision record. | ## Architecture Three intentionally decoupled subsystems under `src/AVEVA.Historian.Client/`: - **`HistorianClient` + `HistorianClientOptions`** — public façade. Validates inputs; delegates reads to `Historian2020ProtocolDialect`; delegates probe / tag metadata / browse / status helpers to the WCF layer. - **`Wcf/`** — managed WCF/MDAS layer. Custom `MdasMessageEncoder` wraps SOAP 1.2 + WS-Addressing 1.0 in AVEVA's `application/x-mdas` framing. Three binding flavors via `HistorianWcfBindingFactory`: plain MDAS over Net.NamedPipe (local), MDAS + Windows transport (remote `/Hist-Integrated`), MDAS + certificate (remote `/HistCert`). Service contracts in `Wcf/Contracts/` mirror the server-side WCF surface (versioned per native interface — `IHistoryServiceContract`, `IRetrievalServiceContract2..4`, `IStatusServiceContract2`, etc.). - **`Protocol/`** — binary frame layer. `Historian2020ProtocolDialect` is the version-anchored bridge between the public façade and the WCF + frame layers. Methods without protocol evidence throw `ProtocolEvidenceMissingException` rather than guessing wire bytes. - **`Models/`** — public DTOs and enums. `HistorianSample`, `HistorianAggregateSample`, `HistorianEvent`, `HistorianTagMetadata`, `HistorianDataType`, `RetrievalMode`, `HistorianTransport`, etc. Read flow end-to-end (live-verified against `localhost`): ``` Hist.GetV → Hist.ValCl × N → Hist.Open2 → Retr.GetV → Retr.IsOriginalAllowed → Retr.StartQuery2 → loop Retr.GetNextQueryResultBuffer2 → typed HistorianSample rows ``` Event flow end-to-end (live-verified): ``` Hist.GetV → Hist.ValCl × N → Hist.Open2 → Hist.UpdC3 → 6× Stat.GetSystemParameter → Hist.RTag2 → Stat.GetSystemParameter(AllowRenameTags) → Trx.GetV → Stat.GetV → Retr.GetV → Hist.EnsT2(CmEventTagId) → Retr.StartEventQuery → loop Retr.GetNextEventQueryResultBuffer → typed HistorianEvent rows with property dictionary → Retr.EndEventQuery → Hist.Close2 ``` ## Safety - Production code under `src/` must remain pure managed .NET 10 with no native AVEVA reference. Reverse-engineering harnesses under `tools/` may reference native binaries. - Never commit credentials, hostnames, user names, customer tag names, or raw packet captures. `*.ndjson`, `current/`, `aveva-install-*/`, and `artifacts/` are gitignored precisely because they accumulate identity-bearing runtime data. - Methods without protocol evidence throw `ProtocolEvidenceMissingException`. Do not stub fake behavior — leave them throwing until evidence supports an implementation. ## Status 313 unit + live integration tests pass (`dotnet test --logger "console;verbosity=minimal"`). Full SDK surface — reads, browse, metadata, status, plus the two write ops (`EnsureTagAsync` / `DeleteTagAsync`) — verified end-to-end against both a local Historian (`LocalPipe`) and a remote Historian (`RemoteTcpIntegrated` over Net.TCP with Windows transport auth). `RemoteTcpCertificate` ProbeAsync is live-verified; deeper coverage over the cert transport plus the explicit-credentials path await additional verification. The 2023 R2 `RemoteGrpc` transport's full tooled surface is live-verified against a real 2023 R2 server: probe, raw / aggregate (TimeWeightedAverage + Minimum/MaximumWithTime + BestFit) / at-time reads, browse, metadata, system-parameter, server time-zone, and measured store-forward status — plus the gRPC-only `AddHistoricalValuesAsync` backfill write. The remaining gRPC config ops are exposed by the server but untooled (see the transport matrix above).