Initial commit: managed .NET 10 AVEVA Historian SDK + reverse-engineering toolkit
Full read-only SDK (src/AVEVA.Historian.Client) implementing the CLAUDE.md required
surface against AVEVA Historian's binary WCF protocol — no native AVEVA runtime
dependency. All operations live-verified against a local Historian:
- ProbeAsync, ReadRawAsync, ReadAggregateAsync, ReadAtTimeAsync, ReadEventsAsync
- BrowseTagNamesAsync, GetTagMetadataAsync (17 native data-type codes mapped)
- GetConnectionStatusAsync, GetStoreForwardStatusAsync, GetSystemParameterAsync
- 108/108 unit + integration tests pass
Includes the reverse-engineering toolkit (tools/AVEVA.Historian.ReverseEngineering)
used to decode the protocol: WCF probes, IL inspection via dnlib, and IL-rewrite
instrumentation (instrument-wcf-{write,read}message etc.) plus the .NET Framework
trace harness (tools/AVEVA.Historian.NativeTraceHarness) for parity testing.
Sanitized handoff evidence under docs/reverse-engineering/. Native AVEVA binaries
(current/, aveva-install-x64/, aveva-install-x86/) are gitignored — fetch separately
from the AVEVA installer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
dohertj2
+254
@@ -0,0 +1,254 @@
|
||||
# Building a fully-managed .NET 10 AVEVA Historian SDK
|
||||
|
||||
Replacement plan for the `aahClientManaged` SDK — the goal is to drop the
|
||||
.NET 4.8 / native-`aahClient.dll` sidecar in `OtOpcUa.Driver.Historian.Wonderware`
|
||||
and call the Historian directly from the .NET 10 OPC UA server process.
|
||||
|
||||
## Why this is hard
|
||||
|
||||
`aahClientManaged.dll` (this folder, `current\`) is a thin managed wrapper around
|
||||
`aahClient.dll`, which is **native C++** that speaks AVEVA's proprietary on-the-wire
|
||||
binary protocol to the Historian server's `aahCfgSvc` / `aahDataSetSvc` /
|
||||
`aahEventSvc` listeners. The protocol is undocumented; AVEVA only ships it as
|
||||
the `aahClient*` family (this folder, `aveva-install-x64\`).
|
||||
|
||||
**There is no AVEVA-shipped managed (.NET-Standard / net6+) replacement.** The
|
||||
2023 R2 Historian Client SDK still targets .NET Framework 4.8 — verified
|
||||
against AVEVA's official patch readmes through 2023 R2 P01 + 2023 P04.
|
||||
|
||||
So a fully-managed .NET 10 client has to come from one of three angles:
|
||||
|
||||
1. **Talk to a different surface** the Historian already exposes (REST, OLE DB,
|
||||
SQL backend) — recommended, lowest risk.
|
||||
2. **Reimplement the binary wire protocol** from scratch — highest risk,
|
||||
highest performance ceiling, requires Wireshark + ILSpy reverse engineering.
|
||||
3. **Build a thin ManagedShim across the existing native SDK via P/Invoke**
|
||||
to `aahClient.dll` — this just relocates the .NET 4.8 wrapper into our code
|
||||
and still bitness-locks us to the native DLL's architecture; the only win is
|
||||
we control the API surface. Not really "fully managed."
|
||||
|
||||
The rest of this doc plans **option 1** as the practical path, with option 2
|
||||
as the fallback if REST proves inadequate.
|
||||
|
||||
## What the SDK actually has to do
|
||||
|
||||
The minimum surface our existing `HistorianDataSource` consumes is small:
|
||||
|
||||
| Operation | aahClientManaged today | Used for |
|
||||
|---|---|---|
|
||||
| `ReadRawAsync(tag, start, end, max)` | `Connection.CreateHistoryQuery() + HistoryQueryArgs{ RetrievalMode=Full }` | `IHistoryProvider.ReadRawAsync` (OPC UA HA-Read raw) |
|
||||
| `ReadAggregateAsync(tag, start, end, mode, interval)` | same path with `RetrievalMode={Average, Min, Max, ...}` | OPC UA HA-Read processed |
|
||||
| `ReadAtTimeAsync(tag, timestamps[])` | per-timestamp queries with `RetrievalMode=Interpolated` and a tight window | OPC UA HA-Read at-time |
|
||||
| `ReadEventsAsync(start, end)` | event-stream connection (separate from history connection) | alarm-historian replay |
|
||||
| Connection health probe | open/close + ping a known tag | `IHostConnectivityProbe` |
|
||||
|
||||
That's the entire required surface for what runs in production today —
|
||||
read-only history + events.
|
||||
|
||||
### The dormant write path
|
||||
|
||||
The sidecar's IPC contract (`Ipc/Contracts.cs:153` —
|
||||
`WriteAlarmEventsRequest` / `Reply`) and dispatcher
|
||||
(`Ipc/HistorianFrameHandler.cs:153` — `HandleWriteAlarmEventsAsync`) both
|
||||
include a fifth operation: writing alarm events back into the historian. There
|
||||
is also an `IAlarmEventWriter` abstraction.
|
||||
|
||||
**That write path is currently inert.** `Program.cs:57` builds the frame
|
||||
handler without supplying an `alarmWriter`, so every `WriteAlarmEvents` RPC
|
||||
short-circuits to `Success=false, Error="Sidecar not configured with an
|
||||
alarm-event writer."`. The architecture anticipated alarm-event write-back
|
||||
(probably via AVEVA's `aaInsert*` C++ entry points) but the implementation
|
||||
never landed.
|
||||
|
||||
For the managed SDK design this means:
|
||||
|
||||
- **Day 1 — read-only is fine.** Shipping just the four read operations
|
||||
matches every code path the OPC UA server actually executes today, with no
|
||||
functional regression.
|
||||
- **Day 2 — if write-back becomes a requirement**, AVEVA's REST API exposes
|
||||
`POST /api/v1/Events` (and `POST /api/v1/HistorianValues` for tag samples).
|
||||
Add a `WriteEventsAsync(IEnumerable<HistorianEvent>, ct)` method on
|
||||
`HistorianClient`, wire `IAlarmEventWriter` to a real implementation in
|
||||
`Core.AlarmHistorian` consumers, and the dormant slot lights up. No native
|
||||
code or wire-protocol work required — same REST transport as the reads.
|
||||
|
||||
## Option 1 — REST API (recommended)
|
||||
|
||||
AVEVA Historian 2023+ exposes a REST API. Endpoints documented at
|
||||
[`docs.aveva.com/bundle/insight/page/273809.html`](https://docs.aveva.com/bundle/insight/page/273809.html).
|
||||
Same data, different transport, no native code involved.
|
||||
|
||||
### Steps
|
||||
|
||||
1. **Verify the REST endpoint is reachable on the Historian server we'll talk
|
||||
to.** Older 2020 deployments may not have it on by default — toggle in
|
||||
`aaConfigurator` under "Historian → Client APIs → Enable REST endpoint".
|
||||
The REST listener typically binds `https://<historian-host>/Historian` on
|
||||
port 32568 (or the configured TLS port). Issue a `GET /api/v1/Tags?$top=1`
|
||||
to confirm.
|
||||
|
||||
2. **Authentication.** REST supports either ArchestrA Galaxy auth (token from
|
||||
`/api/v1/auth/galaxy`) or Insight token auth. Pick one and persist the
|
||||
bearer token; refresh on 401.
|
||||
|
||||
3. **Map the four read operations onto REST endpoints.** From the public REST
|
||||
docs:
|
||||
- `ReadRawAsync` → `GET /api/v1/HistorianValues?TagNames=&StartDateTime=&EndDateTime=&RetrievalMode=Full&MaxResults=`
|
||||
- `ReadAggregateAsync` → same endpoint with `RetrievalMode=Average|Min|Max|Counter|Delta|Range|Stddev|Total|Integral|Slope|Interpolated|BestFit` and a `Resolution=<ms>` parameter.
|
||||
- `ReadAtTimeAsync` → `RetrievalMode=ValueState` with a 1-ms window per requested timestamp, or use the bulk
|
||||
`POST /api/v1/HistorianValues/AtTime` with `[{ "TagName":"...", "Time":"..." }, ...]`.
|
||||
- `ReadEventsAsync` → `GET /api/v1/Events?StartDateTime=&EndDateTime=` plus filter options.
|
||||
|
||||
4. **Build the SDK as a single .NET 10 class library:**
|
||||
|
||||
```
|
||||
src/AVEVA.Historian.Client/ ← new project (this repo)
|
||||
├── HistorianClient.cs ← public surface (see below)
|
||||
├── Auth/
|
||||
│ ├── BearerTokenProvider.cs
|
||||
│ └── GalaxyTokenProvider.cs ← /api/v1/auth/galaxy
|
||||
├── Models/
|
||||
│ ├── HistorianSample.cs ← Value, TimestampUtc, Quality
|
||||
│ ├── HistorianAggregateSample.cs ← + Min/Max/Avg + Quality flags
|
||||
│ ├── HistorianEvent.cs
|
||||
│ └── RetrievalMode.cs ← enum mirroring REST values
|
||||
├── Transport/
|
||||
│ ├── HttpHistorianTransport.cs ← HttpClient, retry, timeout
|
||||
│ └── ResponseParser.cs ← System.Text.Json + DateTime parsing
|
||||
└── Internal/
|
||||
├── ClusterEndpointPicker.cs ← port from existing sidecar
|
||||
└── BackoffPolicy.cs ← Polly v8 ResiliencePipeline
|
||||
```
|
||||
|
||||
Public surface:
|
||||
|
||||
```csharp
|
||||
public sealed class HistorianClient : IDisposable
|
||||
{
|
||||
public HistorianClient(HistorianClientOptions options);
|
||||
public Task<bool> ProbeAsync(CancellationToken ct = default);
|
||||
public IAsyncEnumerable<HistorianSample> ReadRawAsync(
|
||||
string tag, DateTime startUtc, DateTime endUtc, int maxValues, CancellationToken ct = default);
|
||||
public IAsyncEnumerable<HistorianAggregateSample> ReadAggregateAsync(
|
||||
string tag, DateTime startUtc, DateTime endUtc, RetrievalMode mode,
|
||||
TimeSpan interval, CancellationToken ct = default);
|
||||
public Task<IReadOnlyList<HistorianSample>> ReadAtTimeAsync(
|
||||
string tag, IReadOnlyList<DateTime> timestampsUtc, CancellationToken ct = default);
|
||||
public IAsyncEnumerable<HistorianEvent> ReadEventsAsync(
|
||||
DateTime startUtc, DateTime endUtc, CancellationToken ct = default);
|
||||
}
|
||||
|
||||
public sealed record HistorianClientOptions(
|
||||
Uri BaseUri,
|
||||
ITokenProvider Tokens,
|
||||
TimeSpan RequestTimeout,
|
||||
int MaxParallelStreams,
|
||||
string? UserAgent = null);
|
||||
```
|
||||
|
||||
The shape mirrors the four operations the OPC UA server already calls; no
|
||||
wider surface needed.
|
||||
|
||||
5. **Wire it into the OPC UA server.** Replace the named-pipe call sites in
|
||||
`OtOpcUa.Driver.Historian.Wonderware.Client` with direct calls to
|
||||
`HistorianClient`. Delete the sidecar project once the test matrix is green.
|
||||
|
||||
### Test strategy
|
||||
|
||||
- **Contract tests** — feed canned JSON responses (captured from a real
|
||||
Historian) through `ResponseParser`. No network, no Historian needed.
|
||||
- **Integration tests** — hit a live Historian REST endpoint (the dev VM has
|
||||
one once the 2023 R2 install on the desktop completes). Same scenarios as
|
||||
the existing `Driver.Historian.Wonderware.Tests` (`HistorianQualityMapper`,
|
||||
raw / aggregate / events) but rewired to the new client.
|
||||
- **Cluster failover** — port `HistorianClusterEndpointPicker` unchanged; the
|
||||
REST layer fails the same way the SDK did (TCP error / 5xx → mark node bad).
|
||||
|
||||
### Why this is the recommended path
|
||||
|
||||
- Zero native code. Pure System.Net.Http.
|
||||
- AVEVA-supported surface — the wire protocol won't change underneath us at the
|
||||
next Historian patch.
|
||||
- Bitness-agnostic — runs in any .NET 10 process, in-process with the OPC UA
|
||||
server. The sidecar disappears.
|
||||
- The four operations we need are exactly what the REST API was designed for.
|
||||
No impedance mismatch.
|
||||
|
||||
### What it doesn't get us
|
||||
|
||||
- **Sub-second polling latency may regress.** REST round-trips will be slower
|
||||
than a persistent binary socket on a hot LAN. Measure first; if the OPC UA
|
||||
HA-Read latency budget allows, this is fine.
|
||||
- **No streaming / subscription.** The native SDK supports a "live values"
|
||||
push mode (`aahDataSetSvc`); the REST API is request-response. Not a
|
||||
blocker for `IHistoryProvider`, would be for an `ISubscribable` over the
|
||||
Historian — which we don't need today.
|
||||
|
||||
## Option 2 — Reverse-engineer the binary wire protocol
|
||||
|
||||
Only worth pursuing if REST latency is unacceptable.
|
||||
|
||||
### Tools
|
||||
|
||||
- **dnSpy** / **ILSpy** — decompile `aahClientManaged.dll` (this folder,
|
||||
`current\aahClientManaged.dll`) to see the marshalling shape it expects from
|
||||
the native side. The managed wrapper is small enough to read end-to-end —
|
||||
it'll tell you the connection-establishment handshake, the message framing,
|
||||
and the packed types of returned samples.
|
||||
- **API Monitor** or **Detours hooks** on `aahClient.dll` — instrument calls
|
||||
out from the wrapper to map the native ABI surface.
|
||||
- **Wireshark** with a custom dissector — capture a real Historian session
|
||||
(against this dev box's Historian 2020 install, port 32568) while running
|
||||
`OtOpcUa.Driver.Historian.Wonderware` against it. Diff the captures across
|
||||
query types to identify message kinds (raw vs aggregate vs at-time vs
|
||||
events).
|
||||
- **`aahDbDump.exe`** (in the AVEVA install) — dumps Historian's internal
|
||||
block format. Useful for cross-checking what types you're getting on the
|
||||
wire against what's stored on disk.
|
||||
|
||||
### Risk
|
||||
|
||||
The wire protocol is undocumented and AVEVA can change it across versions
|
||||
without warning. A reimplementation pins us to specific Historian versions
|
||||
and creates a maintenance debt that probably outweighs sidecar elimination.
|
||||
|
||||
This is the path you take only if (a) REST is verified inadequate AND (b)
|
||||
you're willing to own a private AVEVA-protocol stack indefinitely.
|
||||
|
||||
## Option 3 — P/Invoke shim (rejected)
|
||||
|
||||
You could build a `HistorianClient` that P/Invokes directly into
|
||||
`aahClient.dll`, bypassing `aahClientManaged.dll`. That technically lets the
|
||||
wrapper code be net10, but:
|
||||
|
||||
- The bitness lock-in stays (`aahClient.dll` is x86 OR x64, no AnyCPU).
|
||||
- COM-style HRESULT returns + raw `BSTR` marshalling makes the boundary
|
||||
brittle.
|
||||
- You learn the same things you'd learn for option 2 anyway, with no wire
|
||||
format won.
|
||||
|
||||
Unless there's a specific feature in `aahClient` that `aahClientManaged`
|
||||
doesn't expose, this path strictly loses to option 1.
|
||||
|
||||
## Decision matrix
|
||||
|
||||
| Option | Pure managed? | net10 in-proc? | Risk | Effort | Recommendation |
|
||||
|---|---|---|---|---|---|
|
||||
| 1 — REST | yes | yes | low | ~1–2 weeks | **default** |
|
||||
| 2 — Wire-protocol reimpl | yes | yes | high | months | only if REST inadequate |
|
||||
| 3 — P/Invoke shim | no (native dep) | yes (with bitness lock) | medium | ~1 week | not recommended |
|
||||
|
||||
## Folder layout in this directory
|
||||
|
||||
```
|
||||
histsdk\
|
||||
current\ ← the 7 SDK DLLs the sidecar links against today
|
||||
aveva-install-x86\ ← full client-side DLL set from C:\Program Files (x86)\Wonderware\Historian
|
||||
aveva-install-x64\ ← same, x64 build (used by our sidecar after PR 80104ca)
|
||||
instructions.md ← this file
|
||||
```
|
||||
|
||||
Use `current\` to feed dnSpy when you want to read the existing wrapper.
|
||||
Use `aveva-install-*\` to inspect what other client-side surfaces AVEVA ships
|
||||
that we might pivot onto (`aahClientConfig.dll` for tag-metadata browse,
|
||||
`aahDataSetClient.dll` for live values, etc.).
|
||||
Reference in New Issue
Block a user