Joseph Doherty
5ce62a5900
ApplyScaling (HistorianTagDefinition.ApplyScaling): The EnsT2 trailer's second byte controls server-side scaling — `FE 00` mirrors MinRaw to MinEU and sets AnalogTag.Scaling=0; `FE 01` persists distinct MinRaw/MaxRaw and sets Scaling=1. Decoded by toggling set_ApplyScaling on the native harness and capturing the wire bytes for both values with identical inputs. The earlier docs claimed EnsureTagAsync needed a follow-up "UpdateTags" call; the WCF surface has no such operation — toggling that one byte is the whole fix. StorageRate (HistorianTagDefinition.StorageRateMs): Serializer accepts a non-default rate, validated empirically against the live server which only accepts quantized values (1000/5000/10000/60000/300000 ms). EnsureTagAsync upsert semantics: Second call on the same tag name with different fields succeeds and updates Description, MinEU, MaxEU, MinRaw, MaxRaw, Scaling in place (verified by direct SQL inspection in a live test). Plan + doc closeout: write-commands-reverse-engineering.md rewritten as a current-state plan with three workstreams (A doc closeout / B idempotency / C1 StorageRate) and a parallelism table; prior phase notes preserved as appendix. handoff.md, implementation-status.md, wcf-contract-evidence.md, README.md updated to remove "writes are out of scope" / non-existent UpdateTags references and document the actual EnsT2 wire format including the `FE xx` trailer. Reverse-engineering harness gains --write-apply-scaling and a SQL post-check that prints the persisted AnalogTag bounds so future RE sessions can verify wire→DB causality without leaving the harness. 169/169 tests pass (was 165; +4 new tests covering ApplyScaling, StorageRate golden bytes, StorageRate live persistence, and EnsureTagAsync upsert semantics). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
175 lines
8.2 KiB
Markdown
175 lines
8.2 KiB
Markdown
# histsdk
|
||
|
||
Pure managed .NET 10 client for AVEVA Historian's binary WCF protocol. The
|
||
production SDK has no dependency on `aahClientManaged.dll`, `aahClient.dll`, or
|
||
any other AVEVA native runtime — the wire protocol is reverse-engineered and
|
||
re-implemented in C#.
|
||
|
||
The supported surface (per [`CLAUDE.md`](CLAUDE.md)):
|
||
|
||
| Operation | Status |
|
||
|---|---|
|
||
| `ProbeAsync` | live-verified |
|
||
| `ReadRawAsync` | live-verified |
|
||
| `ReadAggregateAsync` | live-verified across all 16 retrieval modes |
|
||
| `ReadAtTimeAsync` | live-verified |
|
||
| `ReadEventsAsync` | live-verified (typed event + 31-property property bag) |
|
||
| `BrowseTagNamesAsync` | live-verified |
|
||
| `GetTagMetadataAsync` | live-verified for 17 distinct native data-type codes |
|
||
| `GetConnectionStatusAsync` | synthesized from authenticated probe (matches native semantic) |
|
||
| `GetStoreForwardStatusAsync` | synthesized defaults (no SF sidecar to probe) |
|
||
| `GetSystemParameterAsync` | live-verified via `Stat/GetSystemParameter` |
|
||
| `EnsureTagAsync` | live-verified for analog Float/Double/Int2/Int4/UInt4; `ApplyScaling=true` persists distinct MinRaw/MaxRaw |
|
||
| `DeleteTagAsync` | live-verified |
|
||
|
||
Out of scope: writing samples (`AddS2` is architecturally blocked — the server's
|
||
runtime cache only ingests from configured IOServer / Application Server
|
||
pipelines), store-forward write, configuration changes, discrete/string tag
|
||
creation (native `AddTag` rejects them).
|
||
|
||
## Quick start
|
||
|
||
```csharp
|
||
using AVEVA.Historian.Client;
|
||
using AVEVA.Historian.Client.Models;
|
||
|
||
await using HistorianClient client = new(new HistorianClientOptions
|
||
{
|
||
Host = "localhost",
|
||
IntegratedSecurity = true,
|
||
Transport = HistorianTransport.LocalPipe,
|
||
});
|
||
|
||
DateTime endUtc = DateTime.UtcNow;
|
||
DateTime startUtc = endUtc - TimeSpan.FromMinutes(10);
|
||
|
||
await foreach (HistorianSample sample in client.ReadRawAsync(
|
||
"SysTimeSec", startUtc, endUtc, maxValues: 100))
|
||
{
|
||
Console.WriteLine($"{sample.TimestampUtc:o} {sample.NumericValue} Q={sample.Quality}");
|
||
}
|
||
```
|
||
|
||
For a remote Historian over Net.TCP set `Transport = HistorianTransport.RemoteTcpIntegrated`
|
||
and `Host` to the server hostname. Both `RemoteTcpIntegrated` (Windows transport
|
||
auth) and `RemoteTcpCertificate` (server-cert TLS) are now live-verified for
|
||
`ProbeAsync`; `RemoteTcpIntegrated` is additionally live-verified for the full
|
||
read / browse / metadata / event / status surface.
|
||
|
||
## Build & test
|
||
|
||
```powershell
|
||
dotnet build .\Histsdk.slnx --no-restore
|
||
dotnet test .\Histsdk.slnx --no-build --logger "console;verbosity=minimal"
|
||
```
|
||
|
||
Run a single test class:
|
||
|
||
```powershell
|
||
dotnet test .\Histsdk.slnx --no-build --filter "FullyQualifiedName~WcfDataQueryProtocolTests"
|
||
```
|
||
|
||
Live integration tests (`tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs`)
|
||
are gated and skip cleanly without these env vars:
|
||
|
||
```powershell
|
||
$env:HISTORIAN_HOST = 'localhost'
|
||
$env:HISTORIAN_TEST_TAG = 'SysTimeSec' # any tag the server has data for
|
||
# Optional for non-integrated auth:
|
||
$env:HISTORIAN_USER, $env:HISTORIAN_PASSWORD
|
||
$env:HISTORIAN_TAG_FILTER = 'Sys*' # or any LIKE-pattern
|
||
```
|
||
|
||
## Repository layout
|
||
|
||
```
|
||
src/AVEVA.Historian.Client/ Production SDK — pure managed .NET 10. No native AVEVA dependency.
|
||
tests/ Unit tests (golden-byte / round-trip) + gated live integration tests.
|
||
tools/ Reverse-engineering tooling (NOT shipped):
|
||
AVEVA.Historian.ReverseEngineering/ .NET 10 CLI: WCF probes, dnlib IL inspection,
|
||
IL-rewrite instrumentation (instrument-wcf-{write,read}message,
|
||
instrument-openconnection*, instrument-startdataquery, etc.).
|
||
AVEVA.Historian.NativeTraceHarness/ .NET Framework harness that loads aahClientManaged.dll for
|
||
byte-for-byte parity testing against the native wrapper.
|
||
AVEVA.Historian.NetFxWcfProbe/ .NET Framework WCF probe for ruling out .NET 10-only differences.
|
||
AVEVA.Historian.ReverseInstrumentation/ Helper assembly injected into IL-rewritten wrapper copies.
|
||
AVEVA.Historian.WcfCaptureServer/ Fake server for endpoint experiments.
|
||
scripts/ PowerShell + Frida runners + Python decoders for capture analysis.
|
||
fixtures/protocol/ Sanitized golden-byte fixtures.
|
||
docs/reverse-engineering/ Sanitized handoff evidence; commit-safe summaries only.
|
||
```
|
||
|
||
Native AVEVA binaries (`current/`, `aveva-install-x64/`, `aveva-install-x86/`) are
|
||
**gitignored**. Each developer fetches their own copy from the AVEVA installer; we
|
||
neither modify nor redistribute them.
|
||
|
||
## Documentation
|
||
|
||
Read in order before non-trivial work:
|
||
|
||
| Doc | Purpose |
|
||
|---|---|
|
||
| [`AGENTS.md`](AGENTS.md) | Standing project constraints and safety rules. |
|
||
| [`CLAUDE.md`](CLAUDE.md) | Build commands, code architecture, the active protocol blocker (now resolved) and SDK surface. |
|
||
| [`docs/reverse-engineering/handoff.md`](docs/reverse-engineering/handoff.md) | Decision record + decoded protocol evidence (binding shapes, descriptor layouts, Open2 v6 response, event-row wire format, property-bag value encoding). |
|
||
| [`instructions.md`](instructions.md) | Original plan + decision record. |
|
||
|
||
## Architecture
|
||
|
||
Three intentionally decoupled subsystems under `src/AVEVA.Historian.Client/`:
|
||
|
||
- **`HistorianClient` + `HistorianClientOptions`** — public façade. Validates inputs;
|
||
delegates reads to `Historian2020ProtocolDialect`; delegates probe / tag metadata /
|
||
browse / status helpers to the WCF layer.
|
||
- **`Wcf/`** — managed WCF/MDAS layer. Custom `MdasMessageEncoder` wraps SOAP 1.2 +
|
||
WS-Addressing 1.0 in AVEVA's `application/x-mdas` framing. Three binding flavors
|
||
via `HistorianWcfBindingFactory`: plain MDAS over Net.NamedPipe (local), MDAS +
|
||
Windows transport (remote `/Hist-Integrated`), MDAS + certificate (remote
|
||
`/HistCert`). Service contracts in `Wcf/Contracts/` mirror the server-side WCF
|
||
surface (versioned per native interface — `IHistoryServiceContract`,
|
||
`IRetrievalServiceContract2..4`, `IStatusServiceContract2`, etc.).
|
||
- **`Protocol/`** — binary frame layer. `Historian2020ProtocolDialect` is the
|
||
version-anchored bridge between the public façade and the WCF + frame layers.
|
||
Methods without protocol evidence throw `ProtocolEvidenceMissingException` rather
|
||
than guessing wire bytes.
|
||
- **`Models/`** — public DTOs and enums. `HistorianSample`, `HistorianAggregateSample`,
|
||
`HistorianEvent`, `HistorianTagMetadata`, `HistorianDataType`, `RetrievalMode`,
|
||
`HistorianTransport`, etc.
|
||
|
||
Read flow end-to-end (live-verified against `localhost`):
|
||
|
||
```
|
||
Hist.GetV → Hist.ValCl × N → Hist.Open2 → Retr.GetV → Retr.IsOriginalAllowed →
|
||
Retr.StartQuery2 → loop Retr.GetNextQueryResultBuffer2 → typed HistorianSample rows
|
||
```
|
||
|
||
Event flow end-to-end (live-verified):
|
||
|
||
```
|
||
Hist.GetV → Hist.ValCl × N → Hist.Open2 → Hist.UpdC3 → 6× Stat.GetSystemParameter →
|
||
Hist.RTag2 → Stat.GetSystemParameter(AllowRenameTags) → Trx.GetV → Stat.GetV →
|
||
Retr.GetV → Hist.EnsT2(CmEventTagId) → Retr.StartEventQuery →
|
||
loop Retr.GetNextEventQueryResultBuffer → typed HistorianEvent rows with
|
||
property dictionary → Retr.EndEventQuery → Hist.Close2
|
||
```
|
||
|
||
## Safety
|
||
|
||
- Production code under `src/` must remain pure managed .NET 10 with no native AVEVA
|
||
reference. Reverse-engineering harnesses under `tools/` may reference native binaries.
|
||
- Never commit credentials, hostnames, user names, customer tag names, or raw packet
|
||
captures. `*.ndjson`, `current/`, `aveva-install-*/`, and `artifacts/` are gitignored
|
||
precisely because they accumulate identity-bearing runtime data.
|
||
- Methods without protocol evidence throw `ProtocolEvidenceMissingException`. Do not
|
||
stub fake behavior — leave them throwing until evidence supports an implementation.
|
||
|
||
## Status
|
||
|
||
165 unit + live integration tests pass (`dotnet test --logger "console;verbosity=minimal"`).
|
||
Full SDK surface — reads, browse, metadata, status, plus the two write ops
|
||
(`EnsureTagAsync` / `DeleteTagAsync`) — verified end-to-end against both a
|
||
local Historian (`LocalPipe`) and a remote Historian (`RemoteTcpIntegrated`
|
||
over Net.TCP with Windows transport auth). `RemoteTcpCertificate` ProbeAsync
|
||
is live-verified; deeper coverage over the cert transport plus the
|
||
explicit-credentials path await additional verification.
|