histsdk/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Mission

Build a fully managed .NET 10 replacement for AVEVA Historian's aahClientManaged / aahClient.dll stack by reverse-engineering the proprietary binary protocol. The production SDK under src/AVEVA.Historian.Client/ has no native AVEVA runtime dependency and no REST surface. The one P/Invoke is into Windows SSPI (HistorianSspiClient → InitializeSecurityContextW) for integrated-auth NTLM/Negotiate token generation; this gates the SDK to Windows-only execution today. See the RemoteTcpCertificate transport for a Windows-free auth path. Tools under tools/ and scripts under scripts/ are reverse-engineering aids only.

Read AGENTS.md (standing constraints), instructions.md (decision record), and docs/reverse-engineering/handoff.md (current evidence + active blocker) before starting non-trivial work. The handoff doc is the entry point — it tracks the live blocker, next pickup steps, and the canonical list of primary reference docs.

Required SDK Surface

Reads (the original required surface, all working live as of 2026-05-04):

• ProbeAsync, ReadRawAsync, ReadAggregateAsync, ReadAtTimeAsync, ReadEventsAsync
• BrowseTagNamesAsync, GetTagMetadataAsync
• Status helpers: GetConnectionStatusAsync, GetStoreForwardStatusAsync, GetSystemParameterAsync

Writes (added 2026-05-04 by explicit user request — do not extend further without one):

• EnsureTagAsync for analog types: Float, Double, Int2, Int4, UInt4 (live-verified end-to-end). Other types (SingleByteString/DoubleByteString/Int1/Int8/UInt8) fail at native AddTag — likely require a different path and are intentionally not supported. MinEU/MaxEU/MinRaw/MaxRaw all round-trip into the DB. By default ApplyScaling=false and the server mirrors MinRaw→MinEU and sets AnalogTag.Scaling=0; set ApplyScaling=true on the definition to persist distinct raw bounds with AnalogTag.Scaling=1. The wire encoding is the trailer's second byte (FE 00 vs FE 01).
• DeleteTagAsync

AddS2 (write samples) is architecturally blocked — server cache only ingests from configured IOServers/ApplicationServer pipelines. Do not add write-samples support.

Methods without protocol evidence currently throw ProtocolEvidenceMissingException from Historian2020ProtocolDialect. Do not stub fake behavior — leave them throwing until evidence supports an implementation.

Build & Test

dotnet build .\Histsdk.slnx --no-restore
dotnet test .\Histsdk.slnx --no-build --logger "console;verbosity=minimal"

Run a single test:

dotnet test .\Histsdk.slnx --no-build --filter "FullyQualifiedName~WcfDataQueryProtocolTests"

Live integration tests in tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs are gated and skip cleanly without these env vars:

$env:HISTORIAN_HOST, $env:HISTORIAN_PORT (32568), $env:HISTORIAN_USER, $env:HISTORIAN_PASSWORD,
$env:HISTORIAN_TEST_TAG, $env:HISTORIAN_TAG_FILTER

Never write real credentials, hostnames, user names, or customer tag names into docs, scripts, captures, or commit messages.

Reverse-Engineering CLI

tools/AVEVA.Historian.ReverseEngineering is the .NET 10 CLI for static inspection, WCF probes, and IL-rewrite instrumentation. Common entry points:

dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- wcf-probe $env:HISTORIAN_HOST 32568
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- wcf-cert-probe $env:HISTORIAN_HOST 32568 localhost
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- wcf-like-tag-browse $env:HISTORIAN_HOST 32568 $env:HISTORIAN_TAG_FILTER
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- wcf-start-query $env:HISTORIAN_HOST 32568 $env:HISTORIAN_TEST_TAG --max-attempts 1 --timeout-seconds 3
dotnet run --project tools\AVEVA.Historian.NativeTraceHarness -- --scenario history --tag $env:HISTORIAN_TEST_TAG --lookback-minutes 1440

The wcf-start-query matrix is expensive — always pass --max-attempts / --timeout-seconds for negative probes. See docs/reverse-engineering/capture-workflow.md for the full repeatable capture sequence (manifest, mark, exports, Frida winsock attach, etc.).

Code Architecture

Production SDK (src/AVEVA.Historian.Client/)

Three layered subsystems, intentionally decoupled so protocol parsing can be unit-tested without a live server:

• HistorianClient + HistorianClientOptions — public façade. Validates inputs, delegates reads to Historian2020ProtocolDialect, delegates probe/tag-metadata/browse to the WCF layer.
• Wcf/ — managed WCF/MDAS layer. The Historian uses Net.TCP on port 32568 with a custom application/x-mdas content type wrapping a binary SOAP 1.2 / WS-Addressing 1.0 envelope. MdasMessageEncoder + MdasMessageEncodingBindingElement implement that wrapper. HistorianWcfBindingFactory produces three flavors: plain MDAS, MDAS+Windows transport (used for /Hist-Integrated), and MDAS+certificate (used for /HistCert). Service paths live in HistorianWcfServiceNames. WCF data contracts (Wcf/Contracts/) are reproduced from server-side static analysis and are versioned per native interface (e.g., IRetrievalServiceContract2..4).
• Protocol/ — binary frame layer (HistorianFrameReader/Writer, HistorianBinaryPrimitives, HistorianMessageType). Historian2020ProtocolDialect is the version-anchored bridge between HistorianClient and the frame layer; methods without sufficient evidence throw ProtocolEvidenceMissingException rather than guessing wire bytes.
• Transport/ — pluggable IHistorianTransport (default: TCP). Tests inject a fake transport.
• Grpc/ — 2023 R2 gRPC transport (HistorianTransport.RemoteGrpc). The recovered protobuf contract lives in Grpc/Protos/*.proto and is compiled to client stubs at build time by Grpc.Tools. HistorianGrpcChannelFactory builds a gRPC-Web/HTTP-1.1 channel (default port 32565, optional TLS, gzip) matching the stock 2023 R2 client. HistorianGrpcReadOrchestrator mirrors HistorianWcfReadOrchestrator but over gRPC: it reuses the exact native serializers/parsers — the same Open2 buffer, SSPI/NTLM tokens, and DataQueryRequest/result buffers travel inside protobuf bytes fields. The 2020→gRPC op map: Hist.ValCl→HistoryService.ExchangeKey, Hist.Open2→HistoryService.OpenConnection, Retr.StartQuery2→RetrievalService.StartQuery, Retr.GetNextQueryResultBuffer2→RetrievalService.GetNextQueryResultBuffer. The transport-agnostic handshake (Open2 request builder + SSPI token loop + response decode) is shared via Wcf/HistorianNativeHandshake. Not yet live-verified against a 2023 R2 server — the auth handshake op (ExchangeKey) is the first thing to revisit if a live server rejects it; the byte payloads are the proven 2020 protocol. Gated live test: set HISTORIAN_GRPC_HOST (+ HISTORIAN_TEST_TAG, optional HISTORIAN_GRPC_PORT/HISTORIAN_GRPC_TLS/HISTORIAN_GRPC_DNSID).
• Models/ — public DTOs and enums (HistorianSample, RetrievalMode, etc.). HistorianDataValue represents the discriminated value type.

InternalsVisibleTo exposes internals to the test assembly and the reverse-engineering tool.

Read-path status (resolved 2026-05-04)

The original blocker — Open2 reaching server logic but Retr.StartQuery2 returning false with empty buffers — is resolved. Root causes were:

1. WCF parameter-name mismatches — server contracts use inBuff/outBuff/pRequestBuff/etc.; the SDK's default C#-derived names made the deserializer ignore the body. Fixed via [MessageParameter(Name = "...")] attributes across IHistoryServiceContract2, IRetrievalServiceContract2..4, IStatusServiceContract2, ITransactionServiceContract.
2. Native SSPI request flags — round 0 = 0x2081C (adds IDENTIFY + REPLAY_DETECT + SEQUENCE_DETECT); rounds 1+ = 0x81C. Without REPLAY_DETECT|SEQUENCE_DETECT, NTLM MIC generation is skipped and AcceptSecurityContext rejects round 1. Implemented in HistorianSspiClient via P/Invoke InitializeSecurityContextW.
3. Cross-service version probes (Trx/GetV, Stat/GetV, Retr/GetV) between RTag2 and EnsT2 in the event flow — required to register the client with each service's session table.

End-to-end chain working from a pure managed .NET 10 client: Hist.GetV → Hist.ValCl × N → Hist.Open2 → /Retr.GetV → Retr.IsOriginalAllowed → Retr.StartQuery2 → loop Retr.GetNextQueryResultBuffer2. 169 unit + live integration tests against localhost cover all required reads, the two write ops, and the RemoteTcpIntegrated / RemoteTcpCertificate transports.

Write-path notes (added 2026-05-04)

EnsureTagAsync and DeleteTagAsync chain follow the same pattern as reads but require Open2 with NativeIntegratedWriteEnabledConnectionMode = 0x401 (Process | Write | IntegratedSecurity) — the read-path's 0x402 (read-only) makes the server return err 132 OperationNotEnabled silently. The analog Float CTagMetadata payload is 144 bytes with a leading 0x4E marker byte and a 2-byte trailer FE xx where the second byte is the ApplyScaling flag (00 for false / 01 for true). The IHistoryServiceContract2 surface has no UpdateTags operation — distinct MinRaw/MaxRaw persistence is achieved entirely by toggling that one byte in the EnsT2 payload, not via a follow-up call. See docs/reverse-engineering/handoff.md and the WriteDiag env-gated diagnostic helper in HistorianWcfTagWriteOrchestrator for capture details.

Cross-platform status (verified 2026-05-04)

The SDK builds and runs on Linux (Debian 13, .NET 10 SDK 10.0.203). HistorianSspiClient was rewritten on top of System.Net.Security.NegotiateAuthentication so the only remaining Windows-only surface is in WCF itself:

• ✅ Build — clean on Linux (no platform-specific compile errors after the P/Invoke removal).
• ✅ ProbeAsync over RemoteTcpCertificate from a Debian client (10.100.0.35) against the Windows Historian (10.100.0.48) — TLS handshake succeeds, server returns its version.
• ⚠️ RemoteTcpIntegrated fails on Linux at the WCF transport layer (SecurityNegotiationException → AuthenticationException). NetTcpBinding with SecurityMode.Transport + TcpClientCredentialType.Windows requires Windows-only auth code in WCF that isn't ported to .NET on Linux. This is a hard WCF limitation, not a HistorianSspiClient issue. The HistorianWcfBindingFactory.CreateMdasNetNamedPipeBinding and CreateMdasNetTcpWindowsBinding methods carry a #pragma warning disable CA1416 documenting this.
• ✅ Authenticated WCF calls via NegotiateAuthentication GSSAPI/NTLM from Linux — verified end-to-end with explicit credentials: GetTagMetadataAsync returned correct fields, BrowseTagNamesAsync returned matching tags. Confirms the SDK's auth chain (Open2 → ValCl × N → service call) works cross-platform.
• ✅ Cert-binding calls from Linux verified end-to-end with the two new HistorianClientOptions knobs: AllowUntrustedServerCertificate=true (skips X509 chain validation — needed because .NET WCF on Linux ignores the system CA bundle) plus ServerDnsIdentity="localhost" (matches the installer-generated cert's DNS claim when reaching the server by IP). ReadRawAsync, GetSystemParameterAsync, BrowseTagNamesAsync, and GetTagMetadataAsync all succeed from Debian 13 against the Windows Historian over RemoteTcpCertificate with explicit Windows credentials.

Remaining gaps

Smaller, isolated items — none block the production read surface:

• Remote TCP transports verified by pointing HISTORIAN_REMOTE_TCP_HOST (and HISTORIAN_REMOTE_TCPCERT_HOST for the cert variant) at the host's own LAN IP — exercises the MdasNetTcpWindows / MdasNetTcpCertificate binding branches and SSPI/TLS handshake against a hostname rather than the loopback fast path. RemoteTcpIntegrated: 9 tests (Probe + full read surface + status helpers). RemoteTcpCertificate: Probe only; deeper coverage awaits an explicit-creds setup. True off-box verification (e.g. Linux client) would require porting HistorianSspiClient off InitializeSecurityContextW to managed NegotiateAuthentication + GSSAPI.
• Explicit username/password tag-metadata path is wired (validator only blocks no-auth-at-all), but live-verification requires HISTORIAN_USER+HISTORIAN_PASSWORD set; gated test GetTagMetadataAsync_ExplicitCredentials_AgainstLocalHistorian skips otherwise.
• Per-row trailing 35 bytes of GetNextQueryResultBuffer are now mapped (see HistorianDataQueryProtocol.TryParseGetNextQueryResultBufferRows doc comment) — bytes 3-10 = duplicate FILETIME (already used by aggregate parser), bytes 0-2 + 19-34 = server-internal sample/storage metadata with no clear user-facing meaning. No new public fields added; revisit if a customer asks for storage metadata exposure.
• (No remaining gaps in the write surface — ApplyScaling is now wired, see Required SDK Surface above.)

Tools Layer

• tools/AVEVA.Historian.NativeTraceHarness/ — .NET Framework (not .NET 10) harness that loads current/aahClientManaged.dll and records sanitized reflection snapshots around OpenConnection, StartQuery, MoveNext. Exists specifically to parity-test against the native wrapper.
• tools/AVEVA.Historian.NetFxWcfProbe/ — .NET Framework WCF probe to rule out .NET 10-only WCF behavior differences.
• tools/AVEVA.Historian.ReverseInstrumentation/ — assembly injected into IL-rewritten copies of aahClientManaged.dll for sanitized logging. Rewrites land in docs/reverse-engineering/dnlib-write-copy/, never in current/.
• tools/AVEVA.Historian.WcfCaptureServer/ — fake server for endpoint experiments.
• scripts/ — PowerShell + Frida runners for native attach captures (winsock, system boundary, runtime pointers, ValCl SSPI context).

Evidence & Artifacts

• docs/reverse-engineering/ — sanitized Markdown summaries + small JSON evidence. Always commit-safe.
• artifacts/reverse-engineering/ — raw / identity-bearing runtime output. Never committed; never copy contents into docs/ without sanitizing.
• fixtures/protocol/ — sanitized golden byte fixtures, named to match manifest scenarios.
• current/ and aveva-install-{x64,x86}/ — AVEVA binaries. Never modify, delete, or redistribute. Use current/ first because it matches the deployed sidecar.

Testing Conventions

Unit tests are golden-byte and round-trip oriented — WcfDataQueryProtocolTests, WcfEventQueryProtocolTests, WcfTagQueryProtocolTests, WcfOpen2ProtocolTests, FrameTests, BinaryPrimitiveTests. ProtocolGuardrailTests enforces that unimplemented methods throw ProtocolEvidenceMissingException rather than returning empty results. When adding a new protocol code path, add a golden-byte fixture before/alongside the implementation.

Safety

• Never commit credentials, hostnames, user names, customer tag names, or raw packet captures. Use placeholders in docs.
• Run a sanitization scan after touching auth/capture docs (the rg pattern is in handoff.md "Next Pickup Steps").
• Production code under src/ must remain pure managed .NET 10 with no native AVEVA reference. The one allowed P/Invoke is into the Windows SSPI surface (HistorianSspiClient) for integrated-auth tokens; do not add unrelated P/Invokes. Reverse-engineering harnesses under tools/ may reference native binaries.
• This workspace IS a Git working tree (origin: gitea.dohertylan.com). Use normal git workflow; the prior note about "no working tree, track via timestamps" is obsolete.