histsdk/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Mission

Build a fully managed .NET 10 replacement for AVEVA Historian's `aahClientManaged` / `aahClient.dll` stack by reverse-engineering the proprietary binary protocol. The production SDK under `src/AVEVA.Historian.Client/` has no native AVEVA runtime dependency and no REST surface. The one P/Invoke is into Windows SSPI (`HistorianSspiClient` → `InitializeSecurityContextW`) for integrated-auth NTLM/Negotiate token generation; this gates the SDK to Windows-only execution today. See the `RemoteTcpCertificate` transport for a Windows-free auth path. Tools under `tools/` and scripts under `scripts/` are reverse-engineering aids only.

Read `AGENTS.md` (standing constraints), `instructions.md` (decision record), and `docs/reverse-engineering/handoff.md` (current evidence + active blocker) before starting non-trivial work. The handoff doc is the entry point — it tracks the live blocker, next pickup steps, and the canonical list of primary reference docs.

## Required SDK Surface

Reads (the original required surface, all working live as of 2026-05-04):

- `ProbeAsync`, `ReadRawAsync`, `ReadAggregateAsync`, `ReadAtTimeAsync`, `ReadEventsAsync`
- `BrowseTagNamesAsync`, `GetTagMetadataAsync`
- Status helpers: `GetConnectionStatusAsync`, `GetStoreForwardStatusAsync`, `GetSystemParameterAsync`

Writes (added 2026-05-04 by explicit user request — do not extend further without one):

- `EnsureTagAsync` for analog types: Float, Double, Int2, Int4, UInt4 (live-verified end-to-end). Other types (SingleByteString/DoubleByteString/Int1/Int8/UInt8) fail at native AddTag — likely require a different path and are intentionally not supported. `MinEU`/`MaxEU`/`MinRaw`/`MaxRaw` all round-trip into the DB. By default `ApplyScaling=false` and the server mirrors MinRaw→MinEU and sets `AnalogTag.Scaling=0`; set `ApplyScaling=true` on the definition to persist distinct raw bounds with `AnalogTag.Scaling=1`. The wire encoding is the trailer's second byte (`FE 00` vs `FE 01`).
- `DeleteTagAsync`

`AddS2` (write samples) is architecturally blocked — server cache only ingests from configured IOServers/ApplicationServer pipelines. Do not add write-samples support.

Methods without protocol evidence currently throw `ProtocolEvidenceMissingException` from `Historian2020ProtocolDialect`. Do not stub fake behavior — leave them throwing until evidence supports an implementation.

## Build & Test

```powershell
dotnet build .\Histsdk.slnx --no-restore
dotnet test .\Histsdk.slnx --no-build --logger "console;verbosity=minimal"
```

Run a single test:

```powershell
dotnet test .\Histsdk.slnx --no-build --filter "FullyQualifiedName~WcfDataQueryProtocolTests"
```

Live integration tests in `tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs` are gated and skip cleanly without these env vars:

```powershell
$env:HISTORIAN_HOST, $env:HISTORIAN_PORT (32568), $env:HISTORIAN_USER, $env:HISTORIAN_PASSWORD,
$env:HISTORIAN_TEST_TAG, $env:HISTORIAN_TAG_FILTER
```

Never write real credentials, hostnames, user names, or customer tag names into docs, scripts, captures, or commit messages.

## Reverse-Engineering CLI

`tools/AVEVA.Historian.ReverseEngineering` is the .NET 10 CLI for static inspection, WCF probes, and IL-rewrite instrumentation. Common entry points:

```powershell
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- wcf-probe $env:HISTORIAN_HOST 32568
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- wcf-cert-probe $env:HISTORIAN_HOST 32568 localhost
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- wcf-like-tag-browse $env:HISTORIAN_HOST 32568 $env:HISTORIAN_TAG_FILTER
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- wcf-start-query $env:HISTORIAN_HOST 32568 $env:HISTORIAN_TEST_TAG --max-attempts 1 --timeout-seconds 3
dotnet run --project tools\AVEVA.Historian.NativeTraceHarness -- --scenario history --tag $env:HISTORIAN_TEST_TAG --lookback-minutes 1440
```

The `wcf-start-query` matrix is expensive — always pass `--max-attempts` / `--timeout-seconds` for negative probes. See `docs/reverse-engineering/capture-workflow.md` for the full repeatable capture sequence (manifest, mark, exports, Frida winsock attach, etc.).

## Code Architecture

### Production SDK (`src/AVEVA.Historian.Client/`)

Three layered subsystems, intentionally decoupled so protocol parsing can be unit-tested without a live server:

- **`HistorianClient` + `HistorianClientOptions`** — public façade. Validates inputs, delegates reads to `Historian2020ProtocolDialect`, delegates probe/tag-metadata/browse to the WCF layer.
- **`Wcf/`** — managed WCF/MDAS layer. The Historian uses Net.TCP on port `32568` with a custom `application/x-mdas` content type wrapping a binary SOAP 1.2 / WS-Addressing 1.0 envelope. `MdasMessageEncoder` + `MdasMessageEncodingBindingElement` implement that wrapper. `HistorianWcfBindingFactory` produces three flavors: plain MDAS, MDAS+Windows transport (used for `/Hist-Integrated`), and MDAS+certificate (used for `/HistCert`). Service paths live in `HistorianWcfServiceNames`. WCF data contracts (`Wcf/Contracts/`) are reproduced from server-side static analysis and are versioned per native interface (e.g., `IRetrievalServiceContract2..4`).
- **`Protocol/`** — binary frame layer (`HistorianFrameReader`/`Writer`, `HistorianBinaryPrimitives`, `HistorianMessageType`). `Historian2020ProtocolDialect` is the version-anchored bridge between `HistorianClient` and the frame layer; methods without sufficient evidence throw `ProtocolEvidenceMissingException` rather than guessing wire bytes.
- **`Transport/`** — pluggable `IHistorianTransport` (default: TCP). Tests inject a fake transport.
- **`Models/`** — public DTOs and enums (`HistorianSample`, `RetrievalMode`, etc.). `HistorianDataValue` represents the discriminated value type.

`InternalsVisibleTo` exposes internals to the test assembly and the reverse-engineering tool.

### Read-path status (resolved 2026-05-04)

The original blocker — `Open2` reaching server logic but `Retr.StartQuery2` returning false with empty buffers — is **resolved**. Root causes were:

1. WCF parameter-name mismatches — server contracts use `inBuff`/`outBuff`/`pRequestBuff`/etc.; the SDK's default C#-derived names made the deserializer ignore the body. Fixed via `[MessageParameter(Name = "...")]` attributes across `IHistoryServiceContract2`, `IRetrievalServiceContract2..4`, `IStatusServiceContract2`, `ITransactionServiceContract`.
2. Native SSPI request flags — round 0 = `0x2081C` (adds `IDENTIFY` + `REPLAY_DETECT` + `SEQUENCE_DETECT`); rounds 1+ = `0x81C`. Without `REPLAY_DETECT|SEQUENCE_DETECT`, NTLM MIC generation is skipped and `AcceptSecurityContext` rejects round 1. Implemented in `HistorianSspiClient` via P/Invoke `InitializeSecurityContextW`.
3. Cross-service version probes (`Trx/GetV`, `Stat/GetV`, `Retr/GetV`) between RTag2 and EnsT2 in the event flow — required to register the client with each service's session table.

End-to-end chain working from a pure managed .NET 10 client: `Hist.GetV → Hist.ValCl × N → Hist.Open2 → /Retr.GetV → Retr.IsOriginalAllowed → Retr.StartQuery2 → loop Retr.GetNextQueryResultBuffer2`. 169 unit + live integration tests against `localhost` cover all required reads, the two write ops, and the `RemoteTcpIntegrated` / `RemoteTcpCertificate` transports.

### Write-path notes (added 2026-05-04)

`EnsureTagAsync` and `DeleteTagAsync` chain follow the same pattern as reads but require Open2 with `NativeIntegratedWriteEnabledConnectionMode = 0x401` (Process | Write | IntegratedSecurity) — the read-path's `0x402` (read-only) makes the server return err 132 `OperationNotEnabled` silently. The analog Float `CTagMetadata` payload is 144 bytes with a leading `0x4E` marker byte and a 2-byte trailer `FE xx` where the second byte is the ApplyScaling flag (`00` for false / `01` for true). The `IHistoryServiceContract2` surface has no `UpdateTags` operation — distinct MinRaw/MaxRaw persistence is achieved entirely by toggling that one byte in the EnsT2 payload, not via a follow-up call. See `docs/reverse-engineering/handoff.md` and the `WriteDiag` env-gated diagnostic helper in `HistorianWcfTagWriteOrchestrator` for capture details.

### Cross-platform status (verified 2026-05-04)

The SDK builds and runs on Linux (Debian 13, .NET 10 SDK 10.0.203). `HistorianSspiClient` was rewritten on top of `System.Net.Security.NegotiateAuthentication` so the only remaining Windows-only surface is in WCF itself:

- ✅ **Build** — clean on Linux (no platform-specific compile errors after the
  P/Invoke removal).
- ✅ **`ProbeAsync` over `RemoteTcpCertificate`** from a Debian client
  (10.100.0.35) against the Windows Historian (10.100.0.48) — TLS handshake
  succeeds, server returns its version.
- ⚠️ **`RemoteTcpIntegrated`** fails on Linux at the WCF transport layer
  (`SecurityNegotiationException → AuthenticationException`). `NetTcpBinding`
  with `SecurityMode.Transport` + `TcpClientCredentialType.Windows` requires
  Windows-only auth code in WCF that isn't ported to .NET on Linux. This is
  a hard WCF limitation, not a `HistorianSspiClient` issue. The
  `HistorianWcfBindingFactory.CreateMdasNetNamedPipeBinding` and
  `CreateMdasNetTcpWindowsBinding` methods carry a `#pragma warning disable
  CA1416` documenting this.
- ✅ **Authenticated WCF calls via NegotiateAuthentication GSSAPI/NTLM**
  from Linux — verified end-to-end with explicit credentials:
  `GetTagMetadataAsync` returned correct fields, `BrowseTagNamesAsync`
  returned matching tags. Confirms the SDK's auth chain (Open2 → ValCl × N
  → service call) works cross-platform.
- ✅ **Cert-binding calls from Linux** verified end-to-end with the two
  new `HistorianClientOptions` knobs: `AllowUntrustedServerCertificate=true`
  (skips X509 chain validation — needed because .NET WCF on Linux ignores
  the system CA bundle) plus `ServerDnsIdentity="localhost"` (matches the
  installer-generated cert's DNS claim when reaching the server by IP).
  `ReadRawAsync`, `GetSystemParameterAsync`, `BrowseTagNamesAsync`, and
  `GetTagMetadataAsync` all succeed from Debian 13 against the Windows
  Historian over `RemoteTcpCertificate` with explicit Windows credentials.

### Remaining gaps

Smaller, isolated items — none block the production read surface:

- Remote TCP transports verified by pointing `HISTORIAN_REMOTE_TCP_HOST` (and `HISTORIAN_REMOTE_TCPCERT_HOST` for the cert variant) at the host's own LAN IP — exercises the `MdasNetTcpWindows` / `MdasNetTcpCertificate` binding branches and SSPI/TLS handshake against a hostname rather than the loopback fast path. `RemoteTcpIntegrated`: 9 tests (Probe + full read surface + status helpers). `RemoteTcpCertificate`: Probe only; deeper coverage awaits an explicit-creds setup. True off-box verification (e.g. Linux client) would require porting `HistorianSspiClient` off `InitializeSecurityContextW` to managed `NegotiateAuthentication` + GSSAPI.
- Explicit username/password tag-metadata path is wired (validator only blocks no-auth-at-all), but live-verification requires `HISTORIAN_USER`+`HISTORIAN_PASSWORD` set; gated test `GetTagMetadataAsync_ExplicitCredentials_AgainstLocalHistorian` skips otherwise.
- Per-row trailing 35 bytes of `GetNextQueryResultBuffer` are now mapped (see `HistorianDataQueryProtocol.TryParseGetNextQueryResultBufferRows` doc comment) — bytes 3-10 = duplicate FILETIME (already used by aggregate parser), bytes 0-2 + 19-34 = server-internal sample/storage metadata with no clear user-facing meaning. No new public fields added; revisit if a customer asks for storage metadata exposure.
- (No remaining gaps in the write surface — `ApplyScaling` is now wired, see Required SDK Surface above.)

### Tools Layer

- `tools/AVEVA.Historian.NativeTraceHarness/` — **.NET Framework** (not .NET 10) harness that loads `current/aahClientManaged.dll` and records sanitized reflection snapshots around `OpenConnection`, `StartQuery`, `MoveNext`. Exists specifically to parity-test against the native wrapper.
- `tools/AVEVA.Historian.NetFxWcfProbe/` — .NET Framework WCF probe to rule out .NET 10-only WCF behavior differences.
- `tools/AVEVA.Historian.ReverseInstrumentation/` — assembly injected into IL-rewritten copies of `aahClientManaged.dll` for sanitized logging. Rewrites land in `docs/reverse-engineering/dnlib-write-copy/`, never in `current/`.
- `tools/AVEVA.Historian.WcfCaptureServer/` — fake server for endpoint experiments.
- `scripts/` — PowerShell + Frida runners for native attach captures (winsock, system boundary, runtime pointers, ValCl SSPI context).

### Evidence & Artifacts

- `docs/reverse-engineering/` — sanitized Markdown summaries + small JSON evidence. Always commit-safe.
- `artifacts/reverse-engineering/` — raw / identity-bearing runtime output. Never committed; never copy contents into `docs/` without sanitizing.
- `fixtures/protocol/` — sanitized golden byte fixtures, named to match `manifest` scenarios.
- `current/` and `aveva-install-{x64,x86}/` — AVEVA binaries. **Never modify, delete, or redistribute.** Use `current/` first because it matches the deployed sidecar.

## Testing Conventions

Unit tests are golden-byte and round-trip oriented — `WcfDataQueryProtocolTests`, `WcfEventQueryProtocolTests`, `WcfTagQueryProtocolTests`, `WcfOpen2ProtocolTests`, `FrameTests`, `BinaryPrimitiveTests`. `ProtocolGuardrailTests` enforces that unimplemented methods throw `ProtocolEvidenceMissingException` rather than returning empty results. When adding a new protocol code path, add a golden-byte fixture before/alongside the implementation.

## Safety

- Never commit credentials, hostnames, user names, customer tag names, or raw packet captures. Use placeholders in docs.
- Run a sanitization scan after touching auth/capture docs (the rg pattern is in handoff.md "Next Pickup Steps").
- Production code under `src/` must remain pure managed .NET 10 with no native AVEVA reference. The one allowed P/Invoke is into the Windows SSPI surface (`HistorianSspiClient`) for integrated-auth tokens; do not add unrelated P/Invokes. Reverse-engineering harnesses under `tools/` may reference native binaries.
- This workspace IS a Git working tree (origin: gitea.dohertylan.com). Use normal git workflow; the prior note about "no working tree, track via timestamps" is obsolete.