histsdk/instructions.md

# Building a fully-managed .NET 10 AVEVA Historian SDK

Replacement plan for the `aahClientManaged` SDK — the goal is to drop the
.NET 4.8 / native-`aahClient.dll` sidecar in `OtOpcUa.Driver.Historian.Wonderware`
and call the Historian directly from the .NET 10 OPC UA server process.

## Why this is hard

`aahClientManaged.dll` (this folder, `current\`) is a thin managed wrapper around
`aahClient.dll`, which is **native C++** that speaks AVEVA's proprietary on-the-wire
binary protocol to the Historian server's `aahCfgSvc` / `aahDataSetSvc` /
`aahEventSvc` listeners. The protocol is undocumented; AVEVA only ships it as
the `aahClient*` family (this folder, `aveva-install-x64\`).

**There is no AVEVA-shipped managed (.NET-Standard / net6+) replacement.** The
2023 R2 Historian Client SDK still targets .NET Framework 4.8 — verified
against AVEVA's official patch readmes through 2023 R2 P01 + 2023 P04.

So a fully-managed .NET 10 client has to come from one of three angles:

1. **Talk to a different surface** the Historian already exposes (REST, OLE DB,
   SQL backend) — recommended, lowest risk.
2. **Reimplement the binary wire protocol** from scratch — highest risk,
   highest performance ceiling, requires Wireshark + ILSpy reverse engineering.
3. **Build a thin ManagedShim across the existing native SDK via P/Invoke**
   to `aahClient.dll` — this just relocates the .NET 4.8 wrapper into our code
   and still bitness-locks us to the native DLL's architecture; the only win is
   we control the API surface. Not really "fully managed."

The rest of this doc plans **option 1** as the practical path, with option 2
as the fallback if REST proves inadequate.

## What the SDK actually has to do

The minimum surface our existing `HistorianDataSource` consumes is small:

| Operation | aahClientManaged today | Used for |
|---|---|---|
| `ReadRawAsync(tag, start, end, max)` | `Connection.CreateHistoryQuery() + HistoryQueryArgs{ RetrievalMode=Full }` | `IHistoryProvider.ReadRawAsync` (OPC UA HA-Read raw) |
| `ReadAggregateAsync(tag, start, end, mode, interval)` | same path with `RetrievalMode={Average, Min, Max, ...}` | OPC UA HA-Read processed |
| `ReadAtTimeAsync(tag, timestamps[])` | per-timestamp queries with `RetrievalMode=Interpolated` and a tight window | OPC UA HA-Read at-time |
| `ReadEventsAsync(start, end)` | event-stream connection (separate from history connection) | alarm-historian replay |
| Connection health probe | open/close + ping a known tag | `IHostConnectivityProbe` |

That's the entire required surface for what runs in production today —
read-only history + events.

### The dormant write path

The sidecar's IPC contract (`Ipc/Contracts.cs:153` —
`WriteAlarmEventsRequest` / `Reply`) and dispatcher
(`Ipc/HistorianFrameHandler.cs:153` — `HandleWriteAlarmEventsAsync`) both
include a fifth operation: writing alarm events back into the historian. There
is also an `IAlarmEventWriter` abstraction.

**That write path is currently inert.** `Program.cs:57` builds the frame
handler without supplying an `alarmWriter`, so every `WriteAlarmEvents` RPC
short-circuits to `Success=false, Error="Sidecar not configured with an
alarm-event writer."`. The architecture anticipated alarm-event write-back
(probably via AVEVA's `aaInsert*` C++ entry points) but the implementation
never landed.

For the managed SDK design this means:

- **Day 1 — read-only is fine.** Shipping just the four read operations
  matches every code path the OPC UA server actually executes today, with no
  functional regression.
- **Day 2 — if write-back becomes a requirement**, AVEVA's REST API exposes
  `POST /api/v1/Events` (and `POST /api/v1/HistorianValues` for tag samples).
  Add a `WriteEventsAsync(IEnumerable<HistorianEvent>, ct)` method on
  `HistorianClient`, wire `IAlarmEventWriter` to a real implementation in
  `Core.AlarmHistorian` consumers, and the dormant slot lights up. No native
  code or wire-protocol work required — same REST transport as the reads.

## Option 1 — REST API (recommended)

AVEVA Historian 2023+ exposes a REST API. Endpoints documented at
[`docs.aveva.com/bundle/insight/page/273809.html`](https://docs.aveva.com/bundle/insight/page/273809.html).
Same data, different transport, no native code involved.

### Steps

1. **Verify the REST endpoint is reachable on the Historian server we'll talk
   to.** Older 2020 deployments may not have it on by default — toggle in
   `aaConfigurator` under "Historian → Client APIs → Enable REST endpoint".
   The REST listener typically binds `https://<historian-host>/Historian` on
   port 32568 (or the configured TLS port). Issue a `GET /api/v1/Tags?$top=1`
   to confirm.

2. **Authentication.** REST supports either ArchestrA Galaxy auth (token from
   `/api/v1/auth/galaxy`) or Insight token auth. Pick one and persist the
   bearer token; refresh on 401.

3. **Map the four read operations onto REST endpoints.** From the public REST
   docs:
   - `ReadRawAsync` → `GET /api/v1/HistorianValues?TagNames=&StartDateTime=&EndDateTime=&RetrievalMode=Full&MaxResults=`
   - `ReadAggregateAsync` → same endpoint with `RetrievalMode=Average|Min|Max|Counter|Delta|Range|Stddev|Total|Integral|Slope|Interpolated|BestFit` and a `Resolution=<ms>` parameter.
   - `ReadAtTimeAsync` → `RetrievalMode=ValueState` with a 1-ms window per requested timestamp, or use the bulk
     `POST /api/v1/HistorianValues/AtTime` with `[{ "TagName":"...", "Time":"..." }, ...]`.
   - `ReadEventsAsync` → `GET /api/v1/Events?StartDateTime=&EndDateTime=` plus filter options.

4. **Build the SDK as a single .NET 10 class library:**

   ```
   src/AVEVA.Historian.Client/                   ← new project (this repo)
       ├── HistorianClient.cs                    ← public surface (see below)
       ├── Auth/
       │   ├── BearerTokenProvider.cs
       │   └── GalaxyTokenProvider.cs            ← /api/v1/auth/galaxy
       ├── Models/
       │   ├── HistorianSample.cs                ← Value, TimestampUtc, Quality
       │   ├── HistorianAggregateSample.cs       ← + Min/Max/Avg + Quality flags
       │   ├── HistorianEvent.cs
       │   └── RetrievalMode.cs                  ← enum mirroring REST values
       ├── Transport/
       │   ├── HttpHistorianTransport.cs         ← HttpClient, retry, timeout
       │   └── ResponseParser.cs                 ← System.Text.Json + DateTime parsing
       └── Internal/
           ├── ClusterEndpointPicker.cs          ← port from existing sidecar
           └── BackoffPolicy.cs                  ← Polly v8 ResiliencePipeline
   ```

   Public surface:

   ```csharp
   public sealed class HistorianClient : IDisposable
   {
       public HistorianClient(HistorianClientOptions options);
       public Task<bool> ProbeAsync(CancellationToken ct = default);
       public IAsyncEnumerable<HistorianSample> ReadRawAsync(
           string tag, DateTime startUtc, DateTime endUtc, int maxValues, CancellationToken ct = default);
       public IAsyncEnumerable<HistorianAggregateSample> ReadAggregateAsync(
           string tag, DateTime startUtc, DateTime endUtc, RetrievalMode mode,
           TimeSpan interval, CancellationToken ct = default);
       public Task<IReadOnlyList<HistorianSample>> ReadAtTimeAsync(
           string tag, IReadOnlyList<DateTime> timestampsUtc, CancellationToken ct = default);
       public IAsyncEnumerable<HistorianEvent> ReadEventsAsync(
           DateTime startUtc, DateTime endUtc, CancellationToken ct = default);
   }

   public sealed record HistorianClientOptions(
       Uri BaseUri,
       ITokenProvider Tokens,
       TimeSpan RequestTimeout,
       int MaxParallelStreams,
       string? UserAgent = null);
   ```

   The shape mirrors the four operations the OPC UA server already calls; no
   wider surface needed.

5. **Wire it into the OPC UA server.** Replace the named-pipe call sites in
   `OtOpcUa.Driver.Historian.Wonderware.Client` with direct calls to
   `HistorianClient`. Delete the sidecar project once the test matrix is green.

### Test strategy

- **Contract tests** — feed canned JSON responses (captured from a real
  Historian) through `ResponseParser`. No network, no Historian needed.
- **Integration tests** — hit a live Historian REST endpoint (the dev VM has
  one once the 2023 R2 install on the desktop completes). Same scenarios as
  the existing `Driver.Historian.Wonderware.Tests` (`HistorianQualityMapper`,
  raw / aggregate / events) but rewired to the new client.
- **Cluster failover** — port `HistorianClusterEndpointPicker` unchanged; the
  REST layer fails the same way the SDK did (TCP error / 5xx → mark node bad).

### Why this is the recommended path

- Zero native code. Pure System.Net.Http.
- AVEVA-supported surface — the wire protocol won't change underneath us at the
  next Historian patch.
- Bitness-agnostic — runs in any .NET 10 process, in-process with the OPC UA
  server. The sidecar disappears.
- The four operations we need are exactly what the REST API was designed for.
  No impedance mismatch.

### What it doesn't get us

- **Sub-second polling latency may regress.** REST round-trips will be slower
  than a persistent binary socket on a hot LAN. Measure first; if the OPC UA
  HA-Read latency budget allows, this is fine.
- **No streaming / subscription.** The native SDK supports a "live values"
  push mode (`aahDataSetSvc`); the REST API is request-response. Not a
  blocker for `IHistoryProvider`, would be for an `ISubscribable` over the
  Historian — which we don't need today.

## Option 2 — Reverse-engineer the binary wire protocol

Only worth pursuing if REST latency is unacceptable.

### Tools

- **dnSpy** / **ILSpy** — decompile `aahClientManaged.dll` (this folder,
  `current\aahClientManaged.dll`) to see the marshalling shape it expects from
  the native side. The managed wrapper is small enough to read end-to-end —
  it'll tell you the connection-establishment handshake, the message framing,
  and the packed types of returned samples.
- **API Monitor** or **Detours hooks** on `aahClient.dll` — instrument calls
  out from the wrapper to map the native ABI surface.
- **Wireshark** with a custom dissector — capture a real Historian session
  (against this dev box's Historian 2020 install, port 32568) while running
  `OtOpcUa.Driver.Historian.Wonderware` against it. Diff the captures across
  query types to identify message kinds (raw vs aggregate vs at-time vs
  events).
- **`aahDbDump.exe`** (in the AVEVA install) — dumps Historian's internal
  block format. Useful for cross-checking what types you're getting on the
  wire against what's stored on disk.

### Risk

The wire protocol is undocumented and AVEVA can change it across versions
without warning. A reimplementation pins us to specific Historian versions
and creates a maintenance debt that probably outweighs sidecar elimination.

This is the path you take only if (a) REST is verified inadequate AND (b)
you're willing to own a private AVEVA-protocol stack indefinitely.

## Option 3 — P/Invoke shim (rejected)

You could build a `HistorianClient` that P/Invokes directly into
`aahClient.dll`, bypassing `aahClientManaged.dll`. That technically lets the
wrapper code be net10, but:

- The bitness lock-in stays (`aahClient.dll` is x86 OR x64, no AnyCPU).
- COM-style HRESULT returns + raw `BSTR` marshalling makes the boundary
  brittle.
- You learn the same things you'd learn for option 2 anyway, with no wire
  format won.

Unless there's a specific feature in `aahClient` that `aahClientManaged`
doesn't expose, this path strictly loses to option 1.

## Decision matrix

| Option | Pure managed? | net10 in-proc? | Risk | Effort | Recommendation |
|---|---|---|---|---|---|
| 1 — REST | yes | yes | low | ~1–2 weeks | **default** |
| 2 — Wire-protocol reimpl | yes | yes | high | months | only if REST inadequate |
| 3 — P/Invoke shim | no (native dep) | yes (with bitness lock) | medium | ~1 week | not recommended |

## Folder layout in this directory

```
histsdk\
  current\                      ← the 7 SDK DLLs the sidecar links against today
  aveva-install-x86\            ← full client-side DLL set from C:\Program Files (x86)\Wonderware\Historian
  aveva-install-x64\            ← same, x64 build (used by our sidecar after PR 80104ca)
  instructions.md               ← this file
```

Use `current\` to feed dnSpy when you want to read the existing wrapper.
Use `aveva-install-*\` to inspect what other client-side surfaces AVEVA ships
that we might pivot onto (`aahClientConfig.dll` for tag-metadata browse,
`aahDataSetClient.dll` for live values, etc.).