Files
Joseph Doherty 5ce62a5900 Wire ApplyScaling, StorageRate; close out write-commands plan
ApplyScaling (HistorianTagDefinition.ApplyScaling):
The EnsT2 trailer's second byte controls server-side scaling — `FE 00`
mirrors MinRaw to MinEU and sets AnalogTag.Scaling=0; `FE 01` persists
distinct MinRaw/MaxRaw and sets Scaling=1. Decoded by toggling
set_ApplyScaling on the native harness and capturing the wire bytes for
both values with identical inputs. The earlier docs claimed
EnsureTagAsync needed a follow-up "UpdateTags" call; the WCF surface has
no such operation — toggling that one byte is the whole fix.

StorageRate (HistorianTagDefinition.StorageRateMs):
Serializer accepts a non-default rate, validated empirically against
the live server which only accepts quantized values
(1000/5000/10000/60000/300000 ms).

EnsureTagAsync upsert semantics:
Second call on the same tag name with different fields succeeds and
updates Description, MinEU, MaxEU, MinRaw, MaxRaw, Scaling in place
(verified by direct SQL inspection in a live test).

Plan + doc closeout:
write-commands-reverse-engineering.md rewritten as a current-state
plan with three workstreams (A doc closeout / B idempotency / C1
StorageRate) and a parallelism table; prior phase notes preserved as
appendix. handoff.md, implementation-status.md, wcf-contract-evidence.md,
README.md updated to remove "writes are out of scope" / non-existent
UpdateTags references and document the actual EnsT2 wire format
including the `FE xx` trailer.

Reverse-engineering harness gains --write-apply-scaling and a SQL
post-check that prints the persisted AnalogTag bounds so future RE
sessions can verify wire→DB causality without leaving the harness.

169/169 tests pass (was 165; +4 new tests covering ApplyScaling,
StorageRate golden bytes, StorageRate live persistence, and
EnsureTagAsync upsert semantics).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 22:04:27 -04:00

1471 lines
89 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Implementation Status
## Completed
- Production SDK targets `net10.0` and has no AVEVA binary references.
- Public API includes the full intended parity surface:
- TCP probe
- raw, aggregate, at-time, and block history reads
- event reads
- tag browse and metadata calls
- connection, store-forward, and system-parameter status calls
- **`EnsureTagAsync`** for analog Float/Double/Int2/Int4/UInt4 with
optional `ApplyScaling=true` for distinct MinRaw/MaxRaw persistence
(live-verified end-to-end against `localhost`; SQL post-check confirms
`AnalogTag.Scaling=1` and distinct raw bounds when the flag is set)
- **`DeleteTagAsync`** (live-verified)
- **AddS2 (write samples) is architecturally blocked** — server runtime
cache only ingests from configured IOServers / Application Server
pipelines, not from `HistorianAccess.AddTag`-only flows. Three
independent reproduction attempts confirmed the same
`129 "Tag not found in cache"` failure even with the real wwTagKey,
fresh sessions, and 8s settle waits. Not a protocol gap.
- Internal protocol scaffolding exists:
- `HistorianConnection`
- `HistorianFrameReader`
- `HistorianFrameWriter`
- `Historian2020ProtocolDialect`
- Evidence-backed WCF scaffolding exists:
- WCF service names for `Hist`, `Retr`, `Storage`, `Stat`, and `Trx`
- contract interfaces for history, history extensions, retrieval, retrieval
extensions, and status
- `application/x-mdas` message encoder wrapper
- Net.TCP binding and endpoint factory for port `32568`
- bounded `wcf-start-query` harness options (`--max-attempts` and
`--timeout-seconds`) so negative query probes do not have to run the full
matrix
- live local and remote `GetV` evidence for `/Hist`, `/Retr`, `/Stat`, and
`/Trx`
- live `GetV` evidence for `/HistCert` using MDAS over WCF Net.TCP
transport security
- `ValidateClientCredential` token wrapping and the native NTLM negotiate
`VERSION` flag adjustment as isolated, tested protocol primitives
- `ProbeAsync` now uses fully managed WCF/MDAS `GetV` calls instead of a raw
TCP socket check.
- Managed `Hist.OpenConnection` reaches server logic, but the older scalar
operation expects the native password/session packet.
- Managed `Hist.Open2` now reaches server logic with a version-1 byte buffer.
Empty credentials return custom native error `171` (`AuthenticationFailed`),
confirming the basic byte-buffer framing and UTF-16 string encoding.
- Managed integrated Windows `Open2` succeeds when the same version-1 buffer is
sent to `/Hist-Integrated` with WCF Windows transport security. `/Hist` with
that binding fails before the operation call because the upgrade is not
supported on the plain history endpoint.
- The same managed integrated Windows `Open2` flow succeeds remotely against
`<historian-host>:32568/Hist-Integrated`; the returned handle is accepted by
`Retr.IsOriginalAllowed`. The stored artifact redacts the session output
buffer and transient handle value.
- The native `aahClientManaged.dll` path is confirmed to open successfully with
integrated Windows auth after polling `GetConnectionStatus` until pending
clears.
- The native integrated read path is confirmed to reach query execution: a
deliberately missing tag returns native `TagNotFound` (`127`) instead of
connection/authentication failure.
- `scripts\Find-GalaxyHistorizedTags.ps1` queries the local Galaxy Repository
(`localhost`, database `ZB`, Windows auth) for non-array dynamic attributes
with `HistoryExtension`.
- `OtOpcUaParityTest_001.Counter` is confirmed as a live native read fixture
candidate. A 1440-minute native wrapper read returned one row with timestamp
`2026-04-30T11:00:29.4340342Z`, `Value = 0`, `Quality = 133`,
`OpcQuality = 192`, and `QualityDetail = 248`.
- A Frida attach pass can see `aahClientManaged.dll` and install hooks at
candidate mixed-mode RVAs, but those `base + RVA` hooks do not fire during the
successful read. The next capture approach must intercept CLR/WCF managed
byte-array calls rather than raw method RVAs. See
`frida-aahclientmanaged-hook-pass.md`.
- `tools\AVEVA.Historian.NativeTraceHarness` is a reverse-engineering-only
.NET Framework harness for native integrated reads. It records sanitized
reflection snapshots around `OpenConnection`, `StartQuery`, and `MoveNext`.
The latest run confirms `StartQuery` assigned `queryHandle = 1` and returned
tag key `238`, value `0`, quality `133`, OPC quality `192`, and quality
detail `248` for `OtOpcUaParityTest_001.Counter`. It now supports fixed
UTC windows plus configurable retrieval mode and resolution; latest artifacts
cover `Full`, `Cyclic`, `Interpolated`, and `TimeWeightedAverage`.
- The native trace harness also supports `--scenario event` with an event
connection. The latest event artifact confirms `EventQuery.StartQuery`
succeeds and returns three sanitized local-dev event rows for a seven-day
window.
- `scripts\Attach-NativeTraceHarnessWinsockCapture.ps1` and
`scripts\frida\aahclientmanaged-winsock.js` can attach before native
`OpenConnection` and hook Winsock plus common file/pipe APIs. Localhost,
`127.0.0.1`, and the machine LAN IP all completed successful native reads
without observed `connect`/`send`/`recv`, `CreateFile`/`ReadFile`/`WriteFile`,
or `NtCreateFile`/`NtReadFile`/`NtWriteFile` payload events. This is negative
evidence that the installed local Historian path uses an in-process/local
optimization rather than the remote Net.TCP transport path.
- An expanded Frida method pass added decompiled `QueryColumnSelector` and
`HistorianClient.StartDataQuery`/`GetNextRow` MethodDef RVAs. Hooks installed
where Frida allowed them, but no enter/leave events fired during successful
native reads. MethodDef RVAs from the mixed-mode assembly are not the actual
CLR/native dispatch targets.
- The native trace harness can now dump prepared CLR runtime method pointers for
mixed-mode `<Module>` methods such as `HistorianClient.StartDataQuery`,
`CRetrievalConnectionWCF.StartQuery2`, `HistorianClient.GetNextRow`, and
`HistorianClient.StartEventQuery`. The corrected artifacts show these
pointers are process-specific CLR/JIT entry addresses outside the loaded
`aahClientManaged.dll` image, not stable DLL RVAs.
- `scripts\Attach-NativeTraceHarnessRuntimePointerCapture.ps1` automates a
same-process Frida pass against those runtime pointers. It successfully
generated a pre-`StartQuery` pointer snapshot and installed 37 absolute hooks
while the native direct history read was paused. The read then succeeded, but
no hook `enter`/`leave` callbacks fired. This is negative evidence for using
`MethodHandle.GetFunctionPointer()` addresses as direct Frida hook targets.
- `scripts\Attach-NativeTraceHarnessAahClientExportCapture.ps1` and
`scripts\frida\aahclient-exports.js` attempted to hook the procedural
`mdas_*` exports from `aahClient.dll`. A successful local direct history read
did not load `aahClient.dll`, and `dumpbin /dependents` confirms
`aahClientManaged.dll` does not import `aahClient.dll` or
`aahClientCommon.dll`. A separate `-DumpLoadedModules` run showed only
`aahClientManaged.dll` among the current AVEVA DLLs. The exported `mdas_*`
ABI is likely a separate native client surface, not the active C++/CLI wrapper
boundary used by this harness.
- `scripts\Attach-NativeTraceHarnessSystemBoundaryCapture.ps1` and
`scripts\frida\aahclientmanaged-system-boundary.js` now hook file I/O,
`Nt*File`, `NtDeviceIoControlFile`, DNS, exported Winsock calls, `WSAIoctl`,
`mswsock` extension exports, Secur32, Crypt32, and NetAPI. Local direct and
same-machine remote-IP reads produce no boundary callbacks beyond hook
installation. A Debian relay run confirms the relay TCP connection is owned
by the harness PID, but the same exported user-mode hooks still see no
connect/send/recv/device-control callbacks before the security reset.
- `scripts\Run-PktmonDebianRelayCapture.ps1` records pktmon metadata for the
Debian relay path without retaining raw payload bytes. The latest history run
captured TCP packet metadata for `<relay-host>:32568`, converted the ETL to a
text report, deleted the ETL, and wrote a summary confirming
`PayloadBytesCaptured = false` and `RawEtlDeleted = true`.
- Focused `ildasm` excerpts now confirm `ArchestrA.HistoryQuery.StartQuery` has
an IL call at `IL_02d2` to `HistorianClient.StartDataQuery` token
`060055E4`, and `HistoryQuery.MoveNext` has an IL call at `IL_0054` to
`HistorianClient.GetNextRow<class DataQueryResultRow>` token `0600588D`.
IL-rewrite instrumentation is now validated: a dnlib-written wrapper copy can
run in a disposable DLL folder, and an instrumented
`<Module>.Query.StartDataQuery` captured a 251-byte native
`DataQueryRequest` buffer during a successful local direct read.
- The captured full-history buffer corrected the managed
`HistorianDataQueryProtocol` serializer. It now matches the native wrapper
byte-for-byte for the deterministic `OtOpcUaParityTest_001.Counter` fixture.
The buffer hash is
`3581ef3b42b59b46503d1aa0127fa60fe4b40943e419aeab99e47e4683888851`.
- The same instrumentation path now captures the native
`DataQueryResultRow*` memory after `GetNextRow`. The first full-history row
confirms tag key, timestamp, quality, OPC quality, quality detail, and
percent-good offsets; the artifact hash is
`2c2cb06988187c1bd7793a52a71f33599542a69d5e83885c583de8bf3df5d43b`.
- The read-boundary instrumentation now logs `GetNextRow` `arg.1` as an
explicit `queryHandle` scalar before dumping `arg.2` row memory. A combined
local read captured `StartDataQuery.Request` SHA
`543ea11af87607044067a0274b1da423cef2acbb7b4f4fab137af023a7153d7f`,
`GetNextRow.QueryHandle = 1`, and `GetNextRow.DataQueryResultRow` SHA
`702f5248cf8319e3e02da33678ed97dfaa43666bddb88c42101d5290990a4198`
in the same session.
- A follow-up combined IL instrumentation pass correlated the native open,
query, WCF retrieval, and row-read handles. `HistorianClient.OpenConnection`
wrote legacy handle `2`, and `Query.StartDataQuery` read the same value as
its `ClientHandleCandidate` immediately before entering the common retrieval
layer. The successful `CRetrievalConnectionWCF.StartQuery2` call then used a
different transient WCF retrieval client handle. This proves the missing read
parity state is the native client/session mapping below the legacy
`ClientApp` handle, not the 251-byte `DataQueryRequest` payload.
- Instrumenting `aahClientCommon.CServerClient.GetHandle` token `060017F9`
produced no records during a successful local history read, even while the
surrounding open/query/WCF hooks fired. Direct metadata-token scanning also
found no IL references to that accessor. The transient WCF retrieval handle is
not obtained through `GetHandle` on this path.
- Instrumenting `aahClientCommon.CRetrieval.StartQuery2`,
`aahClientCommon.CSrvRetrievalConnection.StartQuery`, and
`CRetrievalConsoleClient.StartQuery` also produced no records in the
successful WCF read path. The active path is instead
`aahClientCommon.CClientCommon.StartQuery` token `06002E86`.
- The `CClientCommon.StartQuery` instrumentation captured the missing handle
boundary: legacy handle `2` enters `Query.StartDataQuery`, then
`CClientCommon.StartQuery` calls a `CClient` vtable function at IL offset
`0x01A3`. The returned transient value exactly equals the client handle used
by `CRetrievalConnectionWCF.StartQuery2` and
`GetNextQueryResultBuffer2`. The WCF server query handle returned by
`StartQuery2` is copied back through the `queryHandle` pointer, while the
public managed `HistoryQuery.queryHandle` remains the wrapper value `1`.
Sanitized evidence is in `cclientcommon-startquery-correlation-latest.json`.
- `CClientBase.OpenConnection` confirms that same vtable offset `24` is the
handle accessor: the initial value is `0`, then the secondary open branch
succeeds and the post-open value exactly matches the later
`CClientCommon.StartQuery` and WCF `StartQuery2` client handle. The primary
open branch did not emit records on this local integrated path. Sanitized
evidence is in `cclientbase-open-correlation-latest.json`; the next target is
the secondary open vtable call at IL offset `0x06D4`.
- The secondary open branch resolves to
`CHistoryConnectionWCF.OpenConnection3` token `06004059`. It calls
`IHistoryServiceContract2.OpenConnection2` with a 1346-byte request and gets a
42-byte response with empty error. The deserialized response initializes the
vtable offset `24` handle later used by `CClientCommon.StartQuery` and
`/Retr.StartQuery2`. In the captured 42-byte response, byte `0` is `0x03` and
the transient `/Retr` client handle is UInt32 little-endian at offset `1`.
`CClientInfo.DeserializeOpenConnectionOutParams` token `06004008` confirms
the response layout: byte protocol version, `uint32` client handle, 16 session
bytes, one FILETIME, and for version `3` an additional FILETIME. The observed
five-byte tail is preserved as opaque trailing data until a caller or native
field assignment is identified.
`CClientInfo.SerializeOpenConnectionInParams4` token `06004003` confirms the
observed request starts with protocol byte `6`, a 16-byte client key, and a
boolean content selector. The observed selector is `0`, so the active content
branch is `SerializeOpenConnectionInParams2Content` token `06004005`, whose
field order matches the existing native version-3 content serializer: host
string, UInt16 secret length plus secret bytes, client type, connection mode,
metadata namespace, two strings, and client common info.
`CMetadataNamespace.Save` uses compact/scrambled empty strings in this
envelope, so the empty metadata namespace is 10 bytes rather than the older
13-byte plain layout. With that correction, managed replay with the observed
1026-byte zero credential block reaches server credential validation instead
of packet-version rejection. Both `/Hist-Integrated` Windows transport and
`/HistCert` certificate transport now return native error `171` with an
extended message saying the server context could not be found. That makes the
leading 16-byte OpenConnection3 key the next evidence target. A same-run
memory-correlation pass now proves request bytes `1..16` exactly match
`CClientInfo +1240`, and request byte `17` matches `CClientInfo +1608`.
`CClientInfo.SetContextKey` token `0600386E` copies its GUID argument to the
same `CClientInfo +1240` field, but no direct IL caller exists. The actual
source is now narrowed to `CClientBase.ConfigureOpenConnection` token
`0600388C`: it calls native `CClientContext.AuthenticateClient` token
`06005DCB` on `CClientBase +2112`, then copies 16 bytes from
`CClientBase +2176` (`CClientContext +64`, the `GetContextKey` location) to
`CClientBase +1480`. Since this aligns with `CClientInfo +1240`, the prefix is
the context key that native `AuthenticateClient` validates before
OpenConnection3, not an arbitrary fresh managed replay GUID. Runtime
instrumentation around `AuthenticateClient` confirms the field equals the
generated client key before authentication, changes during the native auth
path, then matches the copied `CClientInfo` field and the OpenConnection3
request prefix. A direct managed `Hist-Integrated.ValidateClient2` probe reaches the
service but fails with native error type `4`/code `51` before `ExchangeKey`.
Sanitized IL instrumentation of `CHistoryConnectionWCF.ValidateClient`
(`06004044`) and `CHistoryConnectionWCF.GetClientKey` (`06004041`) was present
in an isolated wrapper copy and the logger smoke test passed, but a successful
native integrated history read emitted no `ValidateClient2` or `ExchangeKey`
records. That negative result rules out the obvious managed WCF auth methods
for this path. Native disassembly of `CClientContext.AuthenticateClient`
(`06005DCB`, RVA `0x298BA0`) shows it uses Secur32
`AcquireCredentialsHandleW` with package `Negotiate`, creates a context GUID
at `CClientContext +64`, and loops through
`CHistoryConnectionWCF.ValidateClientCredential` / WCF `ValCl`.
Instrumenting `ValCl` confirms two successful native rounds: a 69-byte client
input to 239-byte server output, then a 93-byte client input to a one-byte
terminal output. The client input envelope is one round byte, a 4-byte
little-endian token length, then an NTLMSSP token; the first native input
prefix is `01400000004e544c4d5353500001000000b7b218e209000900370000000f000f`.
System-boundary Frida evidence now confirms that these wrapped payloads come
directly from SSPI: native calls `AcquireCredentialsHandleW` with package
`Negotiate`, then two `InitializeSecurityContextW` calls for
`NT SERVICE\aahClientAccessPoint`. The raw SSPI output token lengths are 64
and 88 bytes, matching the 69- and 93-byte `ValCl` bodies after the 5-byte
AVEVA wrapper is added. The first call returns `SEC_I_CONTINUE_NEEDED`
(`0x90312`) and the second returns success.
Native disassembly of `CClientContext.AuthenticateClient` now maps the loop
directly: it is native-only, calls `UuidCreate`, stores the context GUID at
`CClientContext +64`, calls `AcquireCredentialsHandleW`, and enters a loop at
VA `0x180298DE0`. The internal helper at VA `0x180298F30` calls
`InitializeSecurityContextW`, uses request flags `0x2081C` on the first round
and `0x81C` on later rounds, then writes the `ValCl` stream envelope as round
byte, UInt32 token length, and token bytes. The outer loop copies
`CClientContext +64` as the context key and calls the connection's
`ValidateClientCredential` virtual method; in the captured local read,
`CClientContext +0x50` is zero, so the normal connection-object virtual path
at VA `0x180298E95` is selected.
A stable IL-side memory window around `CClientContext.AuthenticateClient`
shows the embedded `CClientContext` mutates only four regions in the first
256 bytes during successful auth: pointer-sized fields at offsets `+8`,
`+16`, and `+24`, plus the context key at `+64..+79`.
Dereferencing the `+8` target shows SSPI/security-package metadata strings
such as `Schannel`, `Microsoft Kerberos V1.0`, `TSSSP`, `System.Core`, and
`Default TLS SSP`. The `+16` target is partially decoded as pointer-rich
native state: nested targets at offsets `0` and `8` have the same
pointer/count shape and no readable ASCII or UTF-16 Historian payload strings,
while the nested target at offset `64` is all zero. The `+24` value is not
safe to treat as a directly readable buffer. This narrows the missing managed
replay state to native `CClientContext` object state and server acceptance of
that client context key, rather than the SSPI token bytes alone.
A managed `NegotiateAuthentication` probe can reproduce that first wrapped
input exactly after setting the NTLM `VERSION` negotiate flag, but standalone
`ValCl` still fails with native error type `4`/code `1` on both `/HistCert`
and `/Hist-Integrated`. That means the active blocker is not the first token
envelope; it is native connection/session prerequisite state around proxy
initialization or server-side context registration.
A follow-up instrumented local read shows the native path constructs
`CHistoryConnectionWCF`, then calls `GetInterfaceVersion` before auth. The
first `GetInterfaceVersion` `InitializeProxy` path succeeds and
`SetManagedPtr` sets the ready flag to `1`; by both successful `ValCl` rounds,
`COperation.Start2` succeeds with error type/code `0`, the existing proxy is
not faulted, and the reconnect flag is `0`. Branch instrumentation now shows
the local native path uses connection mode `1`, enters
`CWcfConfig.ConfigurePipeProxy`, builds the uncompressed local `/Hist`
named-pipe endpoint shape, and does not use `ConfigureTcpProxy`. Static IL
inspection of `ConfigurePipeProxy` confirms this path builds a
`NetNamedPipeBinding` with `Security.Mode = None`, `TransactionFlow = false`,
native-sized `MaxBufferSize`/`ReaderQuotas.MaxArrayLength`, and the
`aahMDASEncoder.ClientBinding` wrapper. It then creates the channel through
the static `ChannelFactory<T>.CreateChannel(binding, endpoint)` helper and
sets `IContextChannel.OperationTimeout` from the native timeout-minutes
argument. Managed pipe probes using the same static-factory channel shape,
with and without eager channel open, still fail in the same way. A managed
named-pipe `ValCl` replay reaches `GetInterfaceVersion` version `11` and
reproduces the first wrapped token hash, but is still rejected at round `0`
with native error type `4`/code `1`; explicitly running the managed calls under
the current Windows token does not change that result. Native handle summaries
show uppercase GUID text, and managed lowercase-handle replay still fails, so
handle casing is not the mismatch. Skipping
`GetInterfaceVersion` and avoiding explicit WCF channel open in the managed TCP
probe also returns native error type `4`/code `1`, so the mismatch is not just
transport or token bytes. Because native `GetInterfaceVersion` creates its
proxy while impersonating the stored Windows identity, the managed pipe probe
was extended to create/open the channel inside the same current-token
impersonation scope; that still fails at `ValCl` round `0`, with and without
also wrapping the operation calls. A local System Platform log check shows the managed
pipe `ValCl` failures correspond to `aahClientAccessPoint` warnings:
`ValidateClientCredential caught exception: System.NullReferenceException` at
`HistoryService.ValidateClientCredential` line `1593`. Enabling named-pipe
transport security is rejected as a binding mismatch, which confirms the native
uncompressed pipe binding shape. A new direct .NET Framework WCF probe using
the same `aa`/`Hist` contract, MDAS content type, uncompressed named pipe, and
SSPI-generated first token also fails with native error type `4`/code `1` and
the same server log exception. That rules out a simple .NET 10 WCF regression:
the success condition belongs to AVEVA's mixed/native wrapper state, not a
plain full-framework WCF client. Static IL inspection of `COperation.Start2`
shows it is only a local gate: it checks operation-priority and bandwidth
controls and can set local gate-failure error codes `243` or `150`, but it
does not call a WCF service operation.
Static inspection of the local `aahClientAccessPoint.exe` service now maps
the receiving side of this failure. `HistoryService.ValidateClientCredential`
parses the WCF `handle` as a GUID, allocates a `CServerBuffer`, copies the
`ValCl` byte array into that buffer, and calls native
`CServerNode.ProcessServerToken`. That native method parses exactly the AVEVA
token envelope already observed on the client side: one round byte, a
4-byte little-endian token length, then SSPI token bytes. On the first round,
`ProcessServerToken` calls helper `0x0050FFC0`, which locks the server
context collection and inserts or refreshes a keyed native context object.
It then calls helper `0x00517AB0` to look up that object. If no context object
is returned, the server sets custom error code `0x29` and `ValCl` fails before
any successful context registration. With a context object, helper
`0x00505C00` calls Secur32 `AcceptSecurityContext` through the service import
table at `0x005A0340`, treating both success and `SEC_I_CONTINUE_NEEDED`
(`0x90312`) as valid protocol progress. Only after the terminal round does
`HistoryService.ValidateClientCredential` add the context GUID to its managed
context-handle collection. This confirms the remaining managed replay gap is
before or inside server `ProcessServerToken` context setup/lookup, not
`OpenConnection3` itself.
A tighter IL window confirms the line number reported in System Platform logs
is the catch/log path, not the exact null-reference instruction. The normal
path is: `Guid.TryParse` on the WCF handle at IL `0x012A`, `CServerBuffer`
allocation through a vtable call at `0x0183`, byte-array pointer/length copy
into buffer offsets `+72/+76`, and `CServerNode.ProcessServerToken` at
`0x01DC`. Only when the native call returns success with `continue == false`
does IL `0x0311` add the parsed context GUID to `m_contextHandles`; when
`continue == true`, the method returns the server token without final handle
registration. This keeps the runtime server helper probe as the most direct
remaining evidence target.
Additional server disassembly and string decoding identify the native object
as `aahClientAccessPoint::CServerContext`. The first-round setup helper uses
the server lock at `CServerNode +0xE80` and keyed context map at
`CServerNode +0xE98`, logs `Adding ServerContext 0x%p`, constructs a
0x3c-byte context object through helper `0x00505100`, and inserts the new
node through a red-black-tree insertion helper at `0x0042F590`. The lookup
helper reads the same map and returns the context object from map node offsets
`+0x20/+0x24`; a null result is what drives the `0x29` custom error path.
The credential helper calls `AcquireCredentialsHandleW` with the UTF-16
package string `Negotiate`. The token-processing helper is logged as
`aahClientAccessPoint::CServerContext::ProcessClientToken`; its failure log
string still says `InitializeSecurityContext`, but the import actually called
by this server helper is Secur32 `AcceptSecurityContext`.
`HistoryService.ValidateIntegratedCredentials` is a separate server path: its
first instructions read `ServiceSecurityContext.Current.WindowsIdentity`.
That explains the earlier `OpenConnection2`/`OpenConnection3` null-reference
failures on bindings where `ServiceSecurityContext.Current` is absent. Those
errors are evidence of selecting the wrong integrated-credential path, not a
user/password validation result.
A focused object-window capture now shows the successful native
`GetInterfaceVersion` path populates the history proxy managed-pointer slot at
`CHistoryConnectionWCF +608` and the ready flag at `+669`; between
`GetInterfaceVersion` completion and `ValCl` entry, a second managed-pointer
slot at `+616` is populated for the binding wrapper. Across both successful
`ValCl` rounds the parent `CHistoryConnectionWCF` object window is stable, but
the `+608` history proxy target mutates at bytes `96..101`; the `+616` binding
target and `+640` Windows identity target remain stable. This points at
native-managed proxy wrapper state as the next concrete evidence target.
Static IL inspection of `CHistoryConnectionWCF.Initialize` narrows the
meaning of the AVEVA log line that says `Initialize: DataSourceId()`. It is
client-side connection/proxy setup, not a separate WCF `Initialize` operation:
the method logs `Initialize`, retrieves or creates the managed `/Hist` proxy
and binding through `InitializeProxy<IHistoryServiceContract2>`, stores those
pointers at the same `+608`/`+616` managed-pointer slots seen at runtime, then
does the same for the `/Trx` proxy. The history proxy initializer has only the
previously mapped WCF setup branches: `ConfigurePipeProxy` at IL `0x0098` and
`ConfigureTcpProxy` at IL `0x038E`, followed by `SetProxyString`. The
decompiled `HistoryServiceContract` interfaces contain no `Initialize`
operation contract. This makes the log line supporting evidence for local
proxy setup state, but not a missing service call that a managed replay can
simply add before `ValCl`.
A focused server-side Frida probe was added at
`scripts\frida\aahclientaccesspoint-valcl-context.js` with runner
`scripts\Capture-AahClientAccessPointValClContext.ps1`. It hooks
`ProcessServerToken`, the first-round context setup and lookup helpers, and
the `AcceptSecurityContext` wrapper while logging only sanitized pointer,
GUID, round, length, and return-value metadata. The script writes a
`.frida.log` sidecar.
An elevated PowerShell session (Admin, High Mandatory Label,
`SeDebugPrivilege` enabled, `SeImpersonatePrivilege` enabled) ran both
scenarios on `2026-05-03` and Frida attach was still rejected with the
CLI message `Failed to attach: process with pid <pid> either refused to
load frida-agent, or terminated during injection`. Direct
`frida.attach(<pid>)` from the Python API reveals the actual exception
class is `frida.ProcessNotRespondingError`, which means the agent
injection handshake did not complete in time, not that the OS refused
the DLL load. The original suspected cause (mitigation policy
`MicrosoftSignedOnly` / `BlockNonMicrosoftBinaries`) is now disproven:
`Get-ProcessMitigation -Id <pid>` reports every category OFF for this
process, including `BinarySignature.MicrosoftSignedOnly`,
`DynamicCode.BlockDynamicCode`, `Cfg.Enable`,
`ImageLoad.BlockRemoteImageLoads`, `ExtensionPoint.DisableExtensionPoints`,
and `UserShadowStack.*`. Process access from the elevated token also
succeeds at `PROCESS_ALL_ACCESS`, including `PROCESS_VM_OPERATION`,
`PROCESS_VM_WRITE`, and `PROCESS_CREATE_THREAD`, so the DACL is not
blocking injection. Cross-bitness Frida (64-bit Python attaching to a
fresh `C:\Windows\SysWOW64\cmd.exe`) attaches and runs scripts cleanly,
so the WOW64 path itself is not broken. Defender real-time protection,
behavior monitoring, and on-access protection are all OFF, no
third-party AV/EDR product is registered with `SecurityCenter2`, no
EDR-style filter driver is active, no `frida` modules appear in the
target's loaded module list before or after a failed attempt, no IFEO
debugger entry exists for `aahClientAccessPoint.exe`, and
`AppInit_DLLs` is empty in both 64-bit and WOW64 hives. Attach attempts
with `realm='native'`, `realm='emulated'`, and `persist_timeout=30` all
fail identically. The remaining likely cause is service-internal:
`aahClientAccessPoint.exe` runs 152 threads, many in `EventPairLow`
ALPC/SCM waits, and Frida's in-memory manual-mapper agent does not get
a cooperative thread for its RPC bootstrap. This is consistent with
`ProcessNotRespondingError` rather than a load-time rejection. The
NativeRead probe still completed end-to-end with the canonical fixture row
(`TagKey=238`, `Value=0`, `Quality=133`, `OpcQuality=192`,
`QualityDetail=248`) but emitted no server-side helper events. The
ManagedValCl probe ran the .NET Framework named-pipe ValCl path against
`net.pipe://localhost/Hist`, reproduced the canonical first wrapped NTLM
envelope (raw outgoing 64 bytes, wrapped 69 bytes, wrapped prefix
`01400000004e544c4d5353500001000000b7b218e209000900370000000f000f`), and
again returned `ServerSuccess=false`, `ServerOutputLength=0`, `ErrorLength=5`
with `NativeError {Type:4, Code:1, Name:Failure}` — matching prior managed
named-pipe ValCl failures and confirming the failure shape is reproducible
but providing no new server-side helper visibility. None of the five
diagnostic questions (whether `0x0050FFC0` ran, whether `0x00517AB0`
returned a context, whether `AcquireCredentialsHandleW` succeeded, whether
`AcceptSecurityContext` was reached, whether failures are pre- or
post-context-map insertion) can be answered from these captures.
ETW SSPI tracing on `2026-05-03` produced server-helper-boundary
evidence without injection. A `logman` trace session capturing the
`LsaSrv {199FE037-2B82-40A9-82AC-E1D46C792B99}`,
`LSA {CC85922F-DB41-11D2-9244-006008269001}`,
`Microsoft-Windows-NTLM {AC43300D-5FCC-4800-8E99-1BD3F85F0320}`,
`NTLM Security Protocol {C92CF544-91B3-4DC0-8E11-C580339A0BF8}`, and
`Security: NTLM Authentication {5BBB6C18-AA45-49B1-A15F-085F7ED0AA90}`
providers at level `0xFF` and keywords `0xFFFFFFFFFFFFFFFF` recorded:
| Run | Total events | aahClientAccessPoint events | lsass events |
|---|---|---|---|
| NativeRead (success) | 5610 | **10** | 4330 |
| ManagedValCl (fail) | 133 | **0** | 121 |
Successful native server-side activity is a 47-millisecond burst of
legacy MOF events (Ids `30, 34, 35, 40, 84, 10, 12, 16, 17, 86`) inside
PID `aahClientAccessPoint`. The failing managed run produces zero
events from the server PID at all — the server never reaches any SSPI
helper invocation. lsass activity is also 35x lower in the failing
run, consistent with auth never completing the first round end-to-end.
This pins the previously logged
`HistoryService.ValidateClientCredential caught
System.NullReferenceException at line 1593` to a code path **before**
`CServerNode.ProcessServerToken` at IL `0x01DC`.
Static IL inspection of the full `HistoryService.ValidateClientCredential`
body (token `0x06000774` in mixed-mode `aahClientAccessPoint.exe`,
779 instructions, dnlib output preserved at
`artifacts/reverse-engineering/server-historyservice-validateclientcredential-il-latest.json`)
enumerates every NRE-capable instruction reached on the straight-line
happy path before the ProcessServerToken call:
- **Prologue (IL `0x0000..0x0129`).** Constructs a `std::wstring`,
case-insensitively compares two wide strings via `_wcsicmp`
(`0x00A0`), and calls
`CServerNode.LogHistorianMessage(this, _, CServerClient*, _, _, _, _)`
at `0x00ED`. The third argument is a `CServerClient*`. If that
pointer is derived from `OperationContext.Current` or related WCF
context state and is null for the calling binding shape, the call
site itself is an NRE candidate before `Guid.TryParse` is reached.
- **Guid parse and recover (IL `0x012A..0x015E`).** `Guid.TryParse`
on `arg.1` at `0x012A`. False branch raises custom error code `28`
via `SError.SetToCustomError` and returns. True branch calls
`<Module>::PtrToStringChars` at `0x0150` then `<Module>::GuidFromString`
at `0x0159` to recover a native `_GUID` for ProcessServerToken.
`Guid.TryParse` cannot NRE on a non-null string; the recovery path
only NREs if the string is null, which a successful `TryParse`
rules out.
- **Allocator vtable chain (IL `0x0160..0x0183`).** Loads
`&g_ClientAccessPoint + 2328`, dereferences once at `0x0178` to get
the allocator pointer `pAllocator`, then derefs `*pAllocator` at
`0x017E` (vtable pointer) and `*(vtable + 40)` at `0x0182` (slot for
the allocator method), and `calli` at `0x0183` returns
`CServerBuffer*`. **NRE candidates: `0x017E` if the field at
`g_ClientAccessPoint + 2328` is null/uninitialised, or `0x0182` if
the vtable is malformed.** Both target a process-wide static, so
they would fail identically for the successful native path; rule
out unless the field is initialised lazily per-binding.
- **Allocator-null branch (IL `0x0189..0x01A3`).** `brtrue.s` on the
returned `CServerBuffer*`. Null path raises custom error code `204`
("No more buffer") and returns false. So a null allocator result is
handled and not the NRE source.
- **Byte-array copy (IL `0x01A8..0x01C6`).** `ldelema System.Byte` on
`arg.2` at `0x01AA` and `ldlen` on `arg.2` at `0x01B2`. **Both NRE
if the WCF-deserialised `inputBuffer` parameter is null.** The two
pointer/length values are then `stind.i4`'d into
`*(CServerBuffer + 72)` and `*(CServerBuffer + 76)`, matching the
documented `+72/+76` offsets.
- **ProcessServerToken call (IL `0x01CA..0x01DC`).** Loads
`&(g_ClientAccessPoint + 2144)` as the `CServerNode*` `this`, the
parsed `_GUID`, the `CServerBuffer*`, a `ref bool continue`, and a
`ref SError`, then calls `ProcessServerToken` (token `0x0600064F`).
The IL slice does not include exception-handler metadata (current
`dnlib-method` output covers instructions only), so the precise
source-line `1593` reported by the System Platform log catch handler
cannot be mapped to one specific instruction from static IL alone.
The structural narrowing is: the throw must happen at a `ldelema`,
`ldlen`, `ldind`, or `calli` instruction reached before `0x01DC`. The
shortlist is `0x00ED` (LogHistorianMessage with `CServerClient*`),
`0x017E` and `0x0182` (allocator vtable derefs), and `0x01AA` /
`0x01B2` (byte-array deref). All other instructions in the window
either operate on managed strings already proven non-null or on
static-field addresses.
Differential analysis against the successful native path narrows
this further. `g_ClientAccessPoint` is a process-wide singleton, so
the `+2328` field has the same value for both runs; the vtable chain
is therefore unlikely to be the differentiator. The native wrapper's
successful local read uses the same `Security.Mode = None`
uncompressed pipe binding the managed probe uses, so
`ServiceSecurityContext.Current.WindowsIdentity` is identical in both
paths and the prologue `CServerClient*` derivation is also unlikely
to be the differentiator. **The remaining structural difference is
the WCF parameter binding: if the managed probe's SOAP body schema
causes WCF to deserialise `inputBuffer` as null even though a 69-byte
wrapped token is on the wire, `ldelema` at `0x01AA` would NRE.** The
managed probe sends 69 bytes per its sanitised log, but the schema
expectation on the server side has not been verified end-to-end.
The next concrete evidence target is therefore not more static IL,
but a SOAP-body comparison: dump the actual `<inputBuffer>` element
the .NET Framework probe writes versus the wire shape the native
wrapper writes for `/Hist-Integrated.ValCl`. If the schemas differ,
the WCF service deserialises `arg.2` as null and the IL window
decisively fails at `0x01AA`. If the schemas match, the throw is
earlier (the prologue's `CServerClient*` log argument becomes the
prime suspect) and runtime confirmation needs PsExec SYSTEM
injection or a signed Detours stub at IL `0x00ED`.
SOAP-body comparison on `2026-05-04` resolved this. Enabling
`<system.serviceModel><diagnostics><messageLogging
logEntireMessage="true" logMessagesAtTransportLevel="true" .../>`
in `AVEVA.Historian.NetFxWcfProbe.exe.config` and capturing the
on-the-wire SOAP body for the failing `aa/Hist/ValCl` request
produced:
```xml
<ValCl xmlns="aa">
<handle>...GUID...</handle>
<inputBuffer>AUAAAABOVExNU1NQAAEAAAC3...</inputBuffer>
</ValCl>
```
The 69-byte wrapped NTLM token IS on the wire as base64 inside
`<inputBuffer>`. The matching server response, however, used
`<outBuff b:nil="true" .../>` rather than the expected `<outputBuffer
.../>` shape, exposing a parameter-name mismatch. `ildasm` against
`aahClientAccessPoint.exe` confirmed the actual server contract is
```il
ValidateClientCredential(string handle,
uint8[] inBuff,
[out] uint8[]& outBuff,
[out] uint8[]& errorBuffer)
```
not `inputBuffer` / `outputBuffer`. WCF builds the request body
element name from the C# parameter name, so the probe sent
`<inputBuffer>` and the server's WCF deserialiser ignored that
unknown element, leaving `arg.2 = inBuff = null`. IL `0x01AA`
`ldelema System.Byte` then NREs, which the C++/CLI catch handler at
HistoryService.cpp line 1593 maps to native error `Type=4 Code=1
Failure` with an empty error buffer.
Adding `[MessageParameter(Name = "inBuff")]` and
`[MessageParameter(Name = "outBuff")]` to the probe's
`ValidateClientCredential` declaration and re-running unblocks the
request: round 0 returns `ServerSuccess=true`,
`ServerOutputLength=239`, `ServerContinue=true`, with
`ServerOutputPrefixHex` `014e544c4d535350000200...` (a `0x01`
continue byte followed by NTLMSSP type-2 challenge). This matches
the previously documented native-success "69-byte client input to
239-byte server output" exactly. Round 1 then sends the 88/93-byte
NTLMSSP type-3 token and the server returns
`Type=129 Code=0x80090308` (`SEC_E_INVALID_TOKEN`) with a 100-byte
error buffer whose ASCII payload includes
`aahClientAccessPoint::CServerContext::ProcessClientToken` and
`InitializeSecurityContext`. That is a real SSPI-level rejection
inside `AcceptSecurityContext`, not the previous parameter-binding
NRE — the original blocker is gone and the next layer of failure is
exposed.
The same fix is now applied to the production SDK contracts
`IHistoryServiceContract2.ValidateClientCredential` and
`IStorageServiceContract.ValidateClientCredential` via
`[MessageParameter(Name = "inBuff" | "outBuff")]`, preserving the
conventional C# parameter names while making WCF emit the
server-correct element names. `ildasm` against `aahClientAccessPoint`
also revealed several other operations the SDK does not yet need
(`EnsT` `InBuff/OutBuff`, `EnsT2` `InBuff/OutBuff`, `RTag2`
`pInBuff/outBuff`, `ExKey` `inBuff/OutBuff`, `StJb` `strJobid`,
`GtJb` `strJobid/jobstatus`) carry the same parameter-naming
mismatch with their current SDK declarations. Those are intentionally
left alone for this read-only pass; audit them when their flows
become required.
The next concrete evidence target is now `SEC_E_INVALID_TOKEN` on
`ValCl` round 1: native traces showed `InitializeSecurityContextW`
with request flags `0x2081C` for the first round and `0x81C` for
later rounds. The .NET Framework probe uses default flags through
the SSPI wrapper. Replicating those exact flags (and confirming the
Negotiate target name matches the wrapper's `NT
SERVICE\aahClientAccessPoint`) is the next testable hypothesis.
Native SSPI flag replication on `2026-05-04` resolved this. Decoding
the documented native flags:
- `0x2081C` (round 0) = `ISC_REQ_IDENTIFY (0x20000) |
ISC_REQ_CONNECTION (0x800) | ISC_REQ_CONFIDENTIALITY (0x10) |
ISC_REQ_SEQUENCE_DETECT (0x8) | ISC_REQ_REPLAY_DETECT (0x4)`
- `0x81C` (round 1+) = same minus `ISC_REQ_IDENTIFY`
The probe's `SspiClient` previously used `ISC_REQ_ALLOCATE_MEMORY |
ISC_REQ_CONFIDENTIALITY | ISC_REQ_INTEGRITY | ISC_REQ_CONNECTION =
0x10910`, which is missing `ISC_REQ_REPLAY_DETECT`,
`ISC_REQ_SEQUENCE_DETECT`, and round-0 `ISC_REQ_IDENTIFY`. The
REPLAY/SEQUENCE pair gates NTLM MIC (Message Integrity Code)
generation in the type-3 response message; without them the type-3
message has no MIC and the server's `AcceptSecurityContext` rejects
it with `SEC_E_INVALID_TOKEN`.
Adding `ISC_REQ_REPLAY_DETECT`, `ISC_REQ_SEQUENCE_DETECT`, and
round-0-only `ISC_REQ_IDENTIFY` (keeping `ISC_REQ_ALLOCATE_MEMORY`
for buffer convenience and tracking the round count internally in
`SspiClient`) reproduces the documented native ValCl sequence
byte-for-byte from a fully managed client:
| Round | Outgoing wrapped | Server output | ServerContinue | NativeError |
|---|---|---|---|---|
| 0 | 69 bytes | 239 bytes (NTLM type-2 challenge) | true | none |
| 1 | 93 bytes | **1 byte (`0x00` terminal)** | false | **none** |
`FinalServerSuccess: true`, `FinalNativeError: null`. This matches
the previously documented "successful native two `ValCl` rounds:
69-byte client input to 239-byte server output, then 93-byte client
input to one-byte terminal output" exactly.
The long-standing managed `ValCl` blocker is therefore resolved.
The chain that successful native reads use is now reproducible from
a managed client end-to-end:
1. `Hist-Integrated.GetV` → version `11`
2. `Hist-Integrated.ValCl` round 0 (69 → 239 bytes) ✓
3. `Hist-Integrated.ValCl` round 1 (93 → 1 byte terminal) ✓
The next steps in the chain — `OpenConnection3` (with the now-known
context key), `Retr.IsOriginalAllowed`, and `Retr.StartQuery2` —
should now be exercisable without server-side helper failures,
because the prerequisite native context-map registration that
`ProcessServerToken` performs has finally been completed by a
managed client.
The production SDK currently has no SSPI client (only the wrap/unwrap
helpers in `HistorianWcfAuthenticationProtocol`). When the SDK auth
flow is wired up for the production read path, it must use the same
native-equivalent flags. .NET 10's `System.Net.Security.NegotiateAuthentication`
does not expose `ISC_REQ_*` directly, so the SDK will likely need to
P/Invoke `InitializeSecurityContextW` (or equivalent) to set
`IDENTIFY` + `REPLAY_DETECT` + `SEQUENCE_DETECT` exactly. Sample
reference implementation in
`tools/AVEVA.Historian.NetFxWcfProbe/Program.cs` `SspiClient`.
End-to-end chain verification on `2026-05-04`. With the WCF parameter
fix and SSPI flag fix in place, the .NET Framework probe was
extended to chain `Hist.Open2` (replaying the captured 1346-byte v6
request with the leading 16 context-key bytes spliced to match the
managed `ValCl` GUID), then `Retr.IsOriginalAllowed`, then
`Retr.StartQuery2` (replaying the captured 251-byte
`OtOpcUaParityTest_001.Counter` `DataQueryRequest`). The result:
| Step | Outcome |
|---|---|
| `Hist.GetV` | version `11` |
| `Hist.ValCl` round 0 | 239-byte server response, NTLM type-2 challenge |
| `Hist.ValCl` round 1 | 1-byte terminal response |
| `Hist.Open2` | **42 bytes, version `0x03`, transient `/Retr` client handle decoded** |
| `Retr.GetV` | version `4` |
| `Retr.IsOriginalAllowed(handle)` | return code `0`, `isAllowed = true` |
| `Retr.StartQuery2(handle, 1, 251 bytes, ...)` | **`Success=true`, response 31 bytes, `QueryHandlePresent=true`, no error** |
The 31-byte `StartQuery2` response SHA-256
`4c062b5ce8181308f0f46bfd8c6088acb52e6ade94401651b7d3ccc8952edfb5`
is **byte-for-byte identical** to the previously captured native
success response (recorded in the existing `Wcf.StartQuery2` block
of `Current Hard Blocker` and in
`instrumented-openconnection3-correlation/capture.ndjson`). The
full AVEVA Historian native wire protocol chain through `StartQuery2`
is now reproducible end-to-end from a fully managed client.
This required one additional contract fix: `IRetrievalServiceContract2`
also had the parameter-name mismatch class of bug. The actual server
contract uses `pRequestBuff` / `pResponseBuff` / `errSize` / `err`
on `StartQuery2` (and `pResultBuff` / `errSize` / `err` on
`GetNextQueryResultBuffer2`, `errSize` / `err` on `EndQuery2`); the
SDK declared them as `requestBuffer` / `responseBuffer` /
`errorSize` / `errorBuffer`. `[MessageParameter(Name = ...)]`
attributes added to `src/AVEVA.Historian.Client/Wcf/Contracts/IRetrievalServiceContract2.cs`.
Replay-only details: the `Open2` body was constructed by reading the
captured 1346-byte v6 native request from
`artifacts/reverse-engineering/openconnection3-request-replay.bin`
and overwriting bytes `1..16` with the new managed-side context-key
GUID; the 251-byte data query was loaded as-is from
`artifacts/reverse-engineering/startdataquery-request-replay.bin`.
Both inputs and the captured native fields they contain (machine
name, process name, etc.) are local to the dev host. The probe's
stdout JSON only echoes lengths, SHAs, version bytes, and prefix
hex; it does not echo identity payloads.
The next concrete step is the production-SDK pass to wire the
managed auth chain: implement an SSPI client that emits the native
flags, replace the captured-replay `Open2` payload with a
schema-driven serialiser using `HistorianOpen2Protocol.SerializeNativeOpenConnection3Version6`
(already in the SDK), and chain `ValCl → Open2 → /Retr.StartQuery2 →
/Retr.GetNextQueryResultBuffer2` for the canonical read fixture.
The reverse-engineered protocol is now fully understood end-to-end
for the read path; remaining work is plumbing.
Production SDK plumbing landed on `2026-05-04`. The full managed
read path is now wired and verified end-to-end against the live
local Historian:
- `src/AVEVA.Historian.Client/AVEVA.Historian.Client.csproj` —
added `System.ServiceModel.NetNamedPipe 10.0.652802` package.
- `src/AVEVA.Historian.Client/Wcf/HistorianWcfBindingFactory.cs` —
added `CreateMdasNetNamedPipeBinding` (Security.Mode = None +
MDAS encoder wrapper) and `CreatePipeEndpointAddress`. Marked
`[SupportedOSPlatform("windows")]` since named pipes are
Windows-only.
- `src/AVEVA.Historian.Client/Wcf/HistorianSspiClient.cs` (new) —
P/Invoke `InitializeSecurityContextW` /
`AcquireCredentialsHandleW` / `DeleteSecurityContext` /
`FreeCredentialsHandle` with internal round counter and the
canonical native flag bitmasks (`0x2081C` round 0 / `0x81C`
later, plus `ALLOCATE_MEMORY` for buffer convenience). Constants
exposed as `internal` for test verification.
- `src/AVEVA.Historian.Client/Wcf/HistorianDataQueryProtocol.cs` —
added `TryParseGetNextQueryResultBufferRows` for the raw/Full row
layout: 6-byte buffer header (`UInt16 version=9`, `UInt32
rowCount`) followed by `rowCount` self-describing rows of `UInt32
tagKey + UInt32 tagNameChars + tagName UTF-16 + UInt32
sampleBlockCount + Int64 startUtcFileTime + UInt32 quality +
UInt32 qualityDetail + UInt32 opcQuality + Double numericValue +
Double percentGood + 1-byte marker + 34 trailing bytes`. The
5-byte error/terminal `04 1E 00 00 00` (type 4, code 30 = "no
more data") is recognised as the "stop looping" signal.
- `src/AVEVA.Historian.Client/Wcf/HistorianWcfReadOrchestrator.cs`
(new) — chains `Hist.GetV → Hist.ValCl × N → Hist.Open2 → /Retr
channel → Retr.GetV → Retr.IsOriginalAllowed → Retr.StartQuery2 →
loop Retr.GetNextQueryResultBuffer2`. Builds the OpenConnection3
v6 request through `HistorianOpen2Protocol.SerializeNativeOpenConnection3Version6`
with the documented native constants (`ClientType=4`,
`ConnectionMode=0x402`, `FormatVersion=4`, `HcalVersion=17`,
`DataSourceId=ClientDllVersion="2020.406.2652.2"`,
`ClientCommonInfo.ClientVersion=999_999`, `ShardId=Guid.Empty`)
and a 1026-byte zero credential block.
- `src/AVEVA.Historian.Client/HistorianClientOptions.cs` — added
`Transport` (defaults to `LocalPipe`) and `TargetSpn` (defaults
to `NT SERVICE\aahClientAccessPoint`).
- `src/AVEVA.Historian.Client/HistorianTransport.cs` (new) —
enum `LocalPipe` / `RemoteTcpIntegrated` / `RemoteTcpCertificate`;
only `LocalPipe` is implemented in this pass.
- `src/AVEVA.Historian.Client/Models/HistorianSample.cs` — added
`PercentGood` positional property.
- `src/AVEVA.Historian.Client/Protocol/Historian2020ProtocolDialect.cs` —
constructor now takes `HistorianClientOptions`; `ReadRawAsync`
delegates to `HistorianWcfReadOrchestrator` on
Windows + `Transport.LocalPipe`, throws
`ProtocolEvidenceMissingException` otherwise.
- `tests/AVEVA.Historian.Client.Tests/` — new
`HistorianSspiClientTests` (5 flag-selection tests),
`WcfBindingFactoryTests` (3), `WcfDataQueryResultBufferTests` (5
golden-byte parser tests using the captured 570-byte fixture).
`HistorianClientIntegrationTests.ReadRawAsync_AgainstLocalHistorian_ReturnsAtLeastOneRow`
(gated on `HISTORIAN_HOST=localhost` + `HISTORIAN_TEST_TAG`)
exercises the full managed chain against the live local
Historian.
Test results with `HISTORIAN_HOST=localhost` and
`HISTORIAN_TEST_TAG=OtOpcUaParityTest_001.Counter`: **69/69 pass**,
including the live read. Without the env vars, the integration
test skips cleanly and 64/64 pass.
The reverse-engineering phase for the read path is complete. The
production SDK now reads history end-to-end against the live local
Historian using only managed code — no `aahClientManaged.dll` or
`aahClient.dll` loaded in the consuming process. Aggregate
(`Interpolated` / `TimeWeightedAverage`) read modes, remote TCP
transport, explicit username/password auth, event reads, and other
contracts (`EnsT`, `RTag2`, `ExKey`, `StJb`, `GtJb` — all of which
ildasm shows have the same parameter-naming mismatch as the
resolved `ValCl` / `StartQuery2` operations) remain follow-up work
but no longer face protocol-discovery blockers — only the
parameter-rename audit + per-mode row-layout decoding +
transport-binding additions.
All listed follow-up work landed on `2026-05-04`. The SDK now
supports: `ReadRawAsync`, `ReadAggregateAsync`, `ReadAtTimeAsync`,
and `ReadEventsAsync` — all verified against the live local
Historian (72/72 tests pass with the integration env vars set).
Specifically:
- **`[MessageParameter]` audit (Phase B2):** `ildasm` against
`aahClientAccessPoint.exe` was used to verify server parameter
names for every operation across `IHistoryServiceContract`,
`IHistoryServiceContract2`, `IRetrievalServiceContract`,
`IRetrievalServiceContract3`, and `IRetrievalServiceContract4`.
`[MessageParameter(Name = ...)]` attributes were applied to ~30
parameter-name mismatches: `EnsT`, `EnsT2`, `RTag2`, `ExKey`,
`AddTEx`, `DelTep`, `StJb`, `GtJb`, `AddS2`, `UpdC3`, `DelT`,
`VldC2`, `OpenConnection`, `AddTags`, `RegisterTags`,
`AddStreamValues`, `ValidateClient`, `UpdateClientStatus`,
`SetClientTimeOut`, `CloseConnection`, `StartQuery`,
`GetNextQueryResultBuffer`, `ExecuteSqlCommand`,
`GetRecordSetByteStream`, `StartTagQuery`, `QueryTag`,
`StartEventQuery`, `GetNextEventQueryResultBuffer`,
`EndEventQuery`, `GetShardTagidsByTagnameAndSource`. Build clean,
72/72 tests pass.
- **Aggregate row parser (Phase B4):**
`HistorianDataQueryProtocol.TryParseGetNextQueryResultBufferAggregateRows`
parses the same wire shape as raw rows but interprets FILETIME
#1 (at row offset `8 + tagNameLen*2 + 4`) as the interval
EndTimeUtc and the FILETIME at trailer offset 2 (row offset
`8 + tagNameLen*2 + 43`) as StartTimeUtc — derived from native
struct evidence (`+0x28 = EndDateTime`, `+0x150 = StartDateTime`)
and the captured raw fixture where Start == End. The orientation
was verified by the live `ReadAggregateAsync` test against
`OtOpcUaParityTest_001.Counter` returning consistent
`TimeWeightedAverage` rows.
- **Aggregate + at-time wiring (Phase B5):**
`HistorianWcfReadOrchestrator.ReadAggregateAsync` and
`ReadAtTimeAsync` chain `Hist.GetV → ValCl × N → Hist.Open2 →
Retr.IsOriginalAllowed → Retr.StartQuery2 → loop
Retr.GetNextQueryResultBuffer2`. The aggregate request maps the
public `RetrievalMode` enum to the documented
`HistorianDataQueryRequest.QueryType` values
(`Full → 2`, `Interpolated → 3`, `TimeWeightedAverage → 5`,
`Cyclic → 4`); other modes throw
`ProtocolEvidenceMissingException` until they have a fixture-
backed mapping. `ReadAtTimeAsync` issues a one-tick
`Interpolated` window per requested timestamp and folds each
aggregate result into a `HistorianSample`.
- **Event flow (Phase B6+B7+B8):**
`HistorianWcfEventOrchestrator` mirrors the read orchestrator but
targets `IRetrievalServiceContract4.StartEventQuery` and
`GetNextEventQueryResultBuffer`. The chain reaches `StartEventQuery`
successfully — a real victory, since the previous probe attempts
failed at this exact call site. `GetNextEventQueryResultBuffer`
then returns native error type=4 code=85 (0x55), which is a NEW
server response (not the canonical `code=30` "no more data").
The orchestrator treats any 5-byte type=4 error buffer as a
soft terminal and surfaces the full code via the
`LastErrorBufferDescription` diagnostic. Likely investigation
targets for code 85: the existing notes describe a native
`CreateDefaultEventTag` step that calls `RegisterTags2` to
register a synthetic `CM_EVENT` tag before any event read can
return rows; we currently skip that prerequisite. Event-row
WCF wire format also remains undecoded (only native struct
snapshots are captured), so even if rows came back we'd need a
fresh capture to parse them. Both of these are documented as
open follow-ups.
- **Remote TCP transport (Phase B1 + B9):**
`HistorianWcfBindingFactory.CreateBindingPair(options)` selects
binding + endpoint pairs by `HistorianTransport`:
`LocalPipe` → `net.pipe://host/Hist` + `/Retr` (existing pipe
binding); `RemoteTcpIntegrated` → `net.tcp://host:port/Hist-Integrated`
(NetTcpBinding + Windows transport security) for the auth chain
plus plain MDAS Net.TCP `/Retr` for queries (per existing
evidence that `/Retr` rejects Windows transport security);
`RemoteTcpCertificate` → `/HistCert` over MDAS+certificate +
plain `/Retr`. The orchestrators now consume the binding pair
transport-agnostically. Untested against a live remote Historian
on this host, but the auth chain is ready to fire.
- **Explicit username/password auth (Phase B3):**
`HistorianSspiClient` has a second constructor overload that
builds a `SEC_WINNT_AUTH_IDENTITY` Unicode struct and passes it
as `pAuthData` to `AcquireCredentialsHandleW`. The auth chain
helper picks the constructor based on
`HistorianClientOptions.IntegratedSecurity`: when `false` and
`UserName` is set, it parses `DOMAIN\user` (or treats the value
as a bare user) and forwards it with `Password`. Untested
against a live remote Historian; reserved for the explicit-creds
production path.
Verified test results with `HISTORIAN_HOST=localhost` and
`HISTORIAN_TEST_TAG=OtOpcUaParityTest_001.Counter`:
**72/72 pass (15-second total)**, including the four live
integration tests (`ProbeAsync`, `BrowseTagNamesAsync`,
`GetTagMetadataAsync`, `ReadRawAsync_AgainstLocalHistorian`,
`ReadAggregateAsync_AgainstLocalHistorian`,
`ReadAtTimeAsync_AgainstLocalHistorian`,
`ReadEventsAsync_AgainstLocalHistorian`). Without env vars, all
tests skip cleanly to 72/72.
Remaining open work (no protocol discovery — pure plumbing or
fresh capture):
- Decode the event-row WCF wire format (no captured fixture yet).
Most direct path: instrument the native trace harness to capture
`Wcf.GetNextEventQueryResultBuffer.ResultBytes` while running the
successful native event read, base-64-encode into the existing
capture.ndjson, then write a parser using the same approach as
the data-query parser.
- Investigate code 85 (`0x55`) terminal from
`GetNextEventQueryResultBuffer`. Most likely the missing
`RegisterTags2(CM_EVENT)` prerequisite. See
`wcf-register-event-tag-latest.json` for the prior probe attempts
and the documented `ConvertEventTagToTagMetadata` GUID values
(`353b8145-5df0-4d46-a253-871aef49b321` event tag id,
`5f59ae42-3bb6-4760-91a5-ab0be01f2f27` event type id).
- Verify remote TCP transports against an actual remote Historian.
Both `RemoteTcpIntegrated` (use `/Hist-Integrated`) and
`RemoteTcpCertificate` (use `/HistCert`) are wired but unverified
on this host.
- Verify explicit username/password against a live Historian with
a non-current user account.
- Add `RetrievalMode` mappings beyond the four currently supported
(`Cyclic`, `Delta`, `BestFit`, `MinimumWithTime`,
`MaximumWithTime`, `Integral`, `Slope`, `Counter`, `ValueState`,
`RoundTrip`, `StartBound`, `EndBound`) when corresponding native
QueryType constants are documented.
- Decode the trailing 24 bytes of each row body (after the
EndTime/Quality/NumericValue/PercentGood/marker/StartTime stack);
they're zero in the captured fixture but vary in some rows
(e.g. trailer bytes at row+125..+132 differ across the 4
captured rows for the same tag), suggesting a per-sample value
or source identifier we haven't decoded.
Event-flow follow-up investigation on `2026-05-04`. The event
orchestrator now uses `ConnectionMode = 0x501` (Event) so the
chain reaches `Retr.GetNextEventQueryResultBuffer`. The server
returns native error type=4 code=85 (`0x55`) with zero rows. SQL
ground truth (`SELECT * FROM Runtime.dbo.Events`) confirms 5+
events ARE available in the test window — they're just not
flowing because the session is not subscribed.
Attempted fix: call `IHistoryServiceContract2.RegisterTags2` with
a `count=1 + 16-byte CM_EVENT GUID` body before `StartEventQuery`.
Tested with both `storageSessionId` and `contextKey` as the handle
parameter, in both upper- and lower-case GUID-D format. Every
variant returned native error code=51 (InvalidParameter). Reverted.
Per existing notes (lines 673728 in this doc), the actual native
prerequisite is `IHistoryServiceContract.AddTags` (`AddT`) with a
`CTagMetadata` payload (`version=1 byte + optional-mask=2 bytes +
data-type-byte=10 + tag-id=16 bytes + compact UTF-16 strings`),
NOT `RegisterTags2`. Documented CM_EVENT identity:
tag id `353b8145-5df0-4d46-a253-871aef49b321`,
event type id `5f59ae42-3bb6-4760-91a5-ab0be01f2f27`,
`CDataType=10`, storage type `2`.
The remaining concrete next step for live event reads:
1. Instrument `Wcf.AddT.Request` on a running native event harness
to capture the exact `CTagMetadata` wire bytes. The existing
reverse-engineering CLI has IL-rewrite instrumentation that
captures other WCF request/response bodies — extend the same
approach to AddT.
2. Wire the captured payload into `HistorianWcfEventOrchestrator`
as the `additionalSetup` callback for the event chain.
3. Once `AddT(CTagMetadata)` succeeds, capture the resulting
`Wcf.GetNextEventQueryResultBuffer.ResultBytes` and write a
parser similar to the data-query row parser.
Until step 1 lands, `ReadEventsAsync` reaches the chain layer
successfully but returns empty results. The diagnostic helper
`EventChainDiagnosticTests.EventOrchestrator_DiagnosticDump_AgainstLocalHistorian`
surfaces `LastResultBufferLength` and `LastErrorBufferDescription`
via `ITestOutputHelper` for iteration.
Raw ETL files contain SSPI tokens, machine names, and identity
metadata; they stay under
`artifacts/reverse-engineering/etw-sspi-{nativeread,managedvalcl}-*.etl`
and are not committed.
Therefore "rerun from elevated PowerShell" is no longer a viable next
step on this host. Practical alternative evidence paths for the server
helper layer:
- SYSTEM-level injection from an interactive PsExec session
(`PsExec64 -accepteula -s -i frida -p <pid> -l <script>`). Requires user
consent because it spawns a SYSTEM-token shell.
- Signed instrumentation DLL: build a Detours-based agent with a code
signature trusted by the local mitigation policy, then `LoadLibraryA`
via a signed loader.
- ETW SSPI provider trace (`Microsoft-Windows-Security-SSPICli`,
`Microsoft-Windows-Security-Auditing`,
`Microsoft-Windows-Security-Negotiate`, `Microsoft-Windows-Security-Kerberos`)
filtered to PID `<pid>`. Captures `AcceptSecurityContext` calls and
return statuses without injection, at the cost of helper-call granularity.
- Inspect/temporarily relax the mitigation policy on this dev host
(`Get-ProcessMitigation -Name aahClientAccessPoint.exe`). This is a
user-consent-required change to a shared service binary.
Until one of those produces server-side helper evidence, the active blocker
remains "missing native wrapper/proxy/session state that lets the server
accept the client-generated context key during ValCl before
OpenConnection3."
Static IL inspection now confirms both managed auth helper methods use the
same handle source: `CHistoryConnectionWCF.GetClientKey` converts its
`contextKey` argument to 36-character GUID text before `ExchangeKey`, and
`CHistoryConnectionWCF.ValidateClientCredential` converts its `contextKey`
argument the same way before `ValCl`. A managed named-pipe probe was extended
to call `ExchangeKey` with that same handle style before replaying `ValCl`.
It reaches `GetInterfaceVersion` version `11`, but `ExchangeKey` itself fails
with native error type `4`/code `1` across default, static-channel, and lazy
static-channel variants. That rules out a simple preceding `ExchangeKey` call
as the missing registration step for the successful native `ValCl` path.
Static IL inspection also maps the client-side integrated identity state:
`CHistoryConnectionWCF.SetIntegratedSecurity(true)` sets flag
`CHistoryConnection +540`, captures `WindowsIdentity.GetCurrent()`, and stores
it with `CServiceUtility.SetManagedPtr<WindowsIdentity>` through the tuple at
offsets `+640`, `+600`, and `+664`. `GetInterfaceVersion` reads the same tuple
and calls `WindowsIdentity.Impersonate()` before proxy setup. Because this is
client-side wrapper state and direct managed probes already run under the
current Windows token, the remaining gap is the native wrapper/proxy state
that makes the server accept the `ValCl` context key around
`GetInterfaceVersion` proxy setup and
`CClientContext.AuthenticateClient`, not normal .NET WCF binding setup or
`COperation.Start2`.
Raw request bytes contain local identity metadata and stay under ignored
artifacts; sanitized hashes are in
`openconnection3-correlation-latest.json`.
- A `TimeWeightedAverage` aggregate request fixture is also captured and
covered by a byte-for-byte serializer test. It confirms query type `5`,
double resolution ticks in the first resolution field, and scaled resolution
ticks in the later field. The artifact hash is
`954874bf851bdea6333b8a8159f036e19b124b7a5febefb0cb9c9a8564b20981`.
- An `Interpolated` request fixture is captured and covered by a byte-for-byte
serializer test. It confirms query type `3` and the same resolution encoding
as aggregate requests. The artifact hash is
`fc3a2fcc28d1926d2bd1de477e306cb0930e80a3327be6309b6e834e2951ca26`.
- `TimeWeightedAverage` `DataQueryResultRow*` memory is captured. Aggregate
rows use offset `0x28` for managed `EndDateTime` and offset `0x150` for
managed `StartDateTime`, unlike the initial raw-row assumption where the
timestamp at `0x28` was sufficient for both bounds.
- `Interpolated` `DataQueryResultRow*` memory is captured and follows the same
time-bound offsets as aggregate rows in the one-minute-resolution fixture.
- `StartEventQuery` request bytes are captured from the native wrapper and now
back a byte-for-byte managed serializer test. The successful local event
capture returned three event rows and emitted a 65-byte request with SHA-256
`6b955b02087047a3199a8c74f3eee85c3b49aaa29b05de12eff2dd536f2da0d5`.
- Native event row memory is captured through the same IL-rewrite path. Three
sanitized `EventQueryResultRow*` snapshots confirm managed `EventTime` is a
direct FILETIME at offset `0x18`; `ReceivedTime` and event property/string
payload storage remain unresolved.
- `StartTagQuery` request bytes are captured from the native wrapper's WCF path
and now back a byte-for-byte managed serializer test. The deterministic OData
tag filter request is 92 bytes with SHA-256
`af1dbcdd3eb0ad91a18882c22252aa74aff82998e96a39b63415ab4792a962ac`; the
successful response is 8 bytes with SHA-256
`db49223a2cf9616171322e5325816a7a579582ebdce91c2f89df8df7aa8aac01` and
parses as `uint32 queryHandle`, `uint32 tagCount`. Local native execution
succeeds and returns tag name
`OtOpcUaParityTest_001.Counter` plus metadata summary `TagKey = 238`,
`TagDataType = 4`, `TagStorageType = 3`, and `EngineeringUnit = None`.
Wildcard tag browse is a separate `StartLikeTagNameSearch` /
`GetLikeTagnames` path and has not been captured yet.
- `TagQuery.GetTagInfo` response bytes are captured from the native wrapper
before `Load<CTagMetadata,SByteStream<SCrtMemFile>>` deserializes them. The
deterministic one-tag response is 106 bytes with SHA-256
`77b2bf720d8888f08a1499a8162e706c2cef567a1f6d74d7e92efe0cd3e3e34b` and now
backs a managed parser test for tag count, native data-type descriptor, type
GUID, tag key, compact ASCII tag name, metadata provider, storage type,
deadband type, and interpolation type. The post-deserialization
`CTagMetadata` vector is also captured with 224-byte elements and hash
`6df4a5a837d06df9332391eb7350af17804137b87541dad8556ca04d015e2995`.
- The separate wildcard browse path is now proven through fully managed WCF:
`StartLikeTagNameSearch` with filter `OtOpcUaParityTest%` returns `0`, and
`GetLikeTagnames` returns one 66-byte buffer with SHA-256
`2d450a55f392aed0026e9a957fefa3b116aab6ec81912c5d824c6b9a1ff5a4a1`.
The response parser is covered by a unit test and confirms layout
`uint32 count`, then repeated `uint32 UTF-16 char length` plus UTF-16LE tag
name bytes.
- `HistorianClient.BrowseTagNamesAsync` now uses the fully managed WCF tag
browse path instead of the evidence-missing guard. Public `*` wildcards are
normalized to the Historian `LIKE` wildcard `%`. Explicit username/password
Open2 for tag browse remains guarded because the successful evidence is
integrated Windows auth.
- `HistorianClient.GetTagMetadataAsync` now uses the fully managed WCF direct
metadata path instead of the evidence-missing guard. It opens the same
`/Hist-Integrated` session and calls `/Retr` `GetTagInfoFromName`, then
parses the observed 98-byte tag metadata record. The current data-type map is
intentionally conservative: descriptor `03 C3 00 31` maps to
`HistorianDataType.Int4`, and unknown descriptors still raise
`ProtocolEvidenceMissingException` until fixture-backed.
- Passwordless SSH to the Debian relay host works for the configured test
account, and the host can reach the Windows Historian on
`<historian-host>:32568`.
- `scripts\Run-DebianHistorianRelayCapture.ps1` starts a temporary Python relay
on the Debian box and forwards `<relay-host>:32568` to
`<historian-host>:32568`. The native Windows client then emits observable remote
Net.TCP/WCF traffic when pointed at `<relay-host>`.
- The relay capture confirms:
- initial WCF Net.TCP preamble bytes beginning with
`00 01 00 01 02 02 ... net.tcp:/`
- server preamble acknowledgement `0A`
- TLS-style `16 03 03 ...` exchange on one connection
- NTLMSSP type 1/2/3 messages on subsequent connections
- server rejection/reset before `OpenConnection` reaches connected state
- A fully managed `wcf-cert-probe` now reproduces the `/HistCert` transport
shape directly. Local `HistCert.GetV` returns interface version `11`, and
remote `<historian-host>:32568/HistCert` also returns version `11` when the client
endpoint identity is set to DNS `localhost`, matching the certificate claim.
- A fully managed remote `wcf-probe` also confirms the plain service endpoints
on `<historian-host>:32568`: `/Hist` version `11`, `/Retr` version `4`, `/Stat`
version `0`, and `/Trx` version `2`. The certificate DNS identity issue is
isolated to `/HistCert`.
- A fully managed remote `wcf-tag-info` probe confirms the same integrated
session handle works for scalar retrieval metadata calls. For
`OtOpcUaParityTest_001.Counter`, `GetTagTypeFromName` returns type `1`,
`IsManualTag` returns `false`, and legacy `GetTagInfoFromName` returns `238`
with no metadata buffer. Initial byte-buffer probes for `GetTgByNm` and
`GetTg` also return `238` with zero output bytes across tag-name and tag-id
encodings, so the remaining metadata gap is likely the exact WCF contract
signature or a richer request envelope rather than the session handle.
- The relay path provides transport evidence but not query buffers yet, because
the relayed authentication/session is rejected before a query can start.
- Server-side ArchestrA logs for the relayed run confirm the native client
initialized against `Server(<relay-host>)` from
`AVEVA.Historian.NativeTraceHarness(20.0.000)` and selected security type
`Transport with Certificate`. This means the relay failure is identity/security
state related; rewriting only the plaintext Net.TCP preamble host is not
sufficient to make a relay transparent.
- Forcing the private `HistorianConnectionArgs.directConnection` field to
`true` lets the native trace harness complete a successful read even with
`ServerName = <relay-host>`, and the Debian relay records no traffic. That
makes `DirectConnection` a native/local parity aid, not a way to capture the
remote protocol.
- The same relay path has now been run for native event connections with
endpoint-host rewriting enabled. Event mode opens the same initial
`/HistCert` Net.TCP preamble using `application/ssl-tls`, then retries
`/Hist-Integrated` with `application/negotiate` and NTLMSSP type 1/2/3
messages. The server still returns a 13-byte rejection/reset before the
harness reaches connected state. Setting `--direct-connection` does not
bypass this event path; it emits the same relay traffic and still fails to
connect. This differs from history direct mode, which completed locally and
avoided the relay.
- The same trace harness attempted WCF diagnostics/message logging, but no
`.svclog` file was produced. Ordinary WCF config tracing is not sufficient to
capture the native wrapper's byte-array calls in this process.
- The Open2 serializer and observed five-byte native error decoder are shared
production internals with unit coverage. The reverse-engineering CLI reports
native error type, code, and known name beside the sanitized base64 buffer.
- A reverse-engineering-only WCF capture server exists under
`tools\AVEVA.Historian.WcfCaptureServer`.
- Uncaptured protocol operations throw `ProtocolEvidenceMissingException`
instead of pretending to implement proprietary frames.
- Reverse-engineering CLI exists under `tools\AVEVA.Historian.ReverseEngineering`.
It can:
- inventory PE exports
- list mixed-mode MethodDef tokens/RVAs from `aahClientManaged.dll`
- list direct IL references to a metadata token
- hash native DLLs
- write capture manifests
- emit timestamp markers for capture notes
- probe WCF/MDAS endpoints
- attempt scalar `Open` and byte-buffer `Open2` session opens
- probe `Stat` WCF operations and optional system-parameter calls
- probe `Retr.StartEventQuery` with reconstructed event request buffers
- Unit tests cover confirmed binary facts:
- FILETIME UTC conversion
- UTF-16 null-terminated strings
- enum numeric compatibility
- frame container round trips
- legacy `Hist.Open2` version-1 buffer layout
- native `OpenConnection3` version-6 request prefix and content branch layout
- first-pass `DataQueryRequest.Save` field order for empty metadata and
auto-summary defaults
- observed native error buffer decoding
- native `OpenConnection3` version-3 response parsing from the decoded
`CClientInfo.DeserializeOpenConnectionOutParams` layout
- Windows `SYSTEMTIME` byte parsing for status evidence
- native `StartTagQuery` request/response layout and first `GetTagInfo`
response stream fields
- `GetLikeTagnames` browse response buffer layout
- public browse wildcard normalization
- evidence guardrails
## Current Hard Blocker
The 2020 client transport is now known to be WCF Net.TCP using a custom MDAS
message encoder, not just an opaque raw socket frame format. The `Hist.Open2`
legacy version-1 buffer is decoded far enough to make `/Hist-Integrated`
return success under fully managed integrated Windows auth. The 32-byte response
is decoded for the legacy path as:
- `uint32` client handle
- `Guid` storage session id
- `int64` connect time FILETIME UTC
- `uint32` server status
The decoded handle is accepted by `Retr.IsOriginalAllowed`, while hard-coded
handle `1` is rejected. That confirms the session handle is usable for at least
some retrieval contract calls.
The native wrapper WCF read boundary is now captured from a successful local
read. Reverse-only IL instrumentation of
`CRetrievalConnectionWCF.StartQuery2` and
`CRetrievalConnectionWCF.GetNextQueryResultBuffer2` shows:
- `StartQuery2` succeeds with `clientHandle = 256524568`,
`queryRequestType = 1`, request size `251`, response size `31`, response
SHA-256 `4c062b5ce8181308f0f46bfd8c6088acb52e6ade94401651b7d3ccc8952edfb5`,
server query handle `2967`, and zero error size.
- `GetNextQueryResultBuffer2` succeeds with the same client handle and server
query handle, result size `570`, result SHA-256
`f4126e0610a4c63b3dff0e41e8d15e51d75fdbc736f3ec490e7c66d1bb31638d`, and
a five-byte terminal/error buffer SHA-256
`7db15e1972ced8a44dae4d75f7a1f0cd74858c7d0deb8b3522d6a05904778bf7`.
- The public `HistoryQuery.queryHandle = 1` is a client-side wrapper handle.
The server WCF query handle on `/Retr` is a separate value (`2967` in this
run).
Direct managed `Retr.StartQuery2` probes remain negative even with byte-matched
request serialization, so the remaining gap is no longer the data-query request
payload. The latest correlation pass shows the native read path starts with
legacy native handle `2`, then the common retrieval/session layer supplies the
separate `/Retr` WCF client handle accepted by `StartQuery2`. Instrumenting
`CHistoryConnectionWCF.OpenConnection2`, `CServerClient.GetHandle`,
`aahClientCommon.CRetrieval.StartQuery2`,
`CSrvRetrievalConnection.StartQuery`, and
`CRetrievalConsoleClient.StartQuery` did not emit records in this path. The
current evidence target is the `CClient` object reached by
`CClientCommon.StartQuery`: the vtable method at offset `24` returns the `/Retr`
WCF client handle, and `CHistoryConnectionWCF.OpenConnection3` creates it by
calling `IHistoryServiceContract2.OpenConnection2`. The 1346-byte
OpenConnection3 request is now decoded far enough to show the remaining
prerequisite: reproduce `CClientContext.AuthenticateClient` so the managed
client can register/validate the client-generated context key with the server
before sending OpenConnection3. The first `ValCl` token and local named-pipe
setup are now mapped; next inspect or instrument the native
`GetInterfaceVersion` and wrapper/proxy side effects that make the same token
accepted when sent by the wrapper.
Current negative probe evidence:
- exact native `QueryColumnSelector.SelectNonSummaryColumns` state
`0x00008182000782FF`
- corrected packed `CQTIFlags` value for ordinary full reads (`0x01FF`)
- query type values `0..14`
- option `""` and `"NoOption"`
- selector flags `0xFFFF`, `0x3FFFF`, and `ulong.MaxValue`
- empty MDS/storage redundant endpoints
- local-machine MDS/storage redundant endpoint variants
- corrected `NoFilter` filter text
- corrected empty metadata namespace and default auto-summary byte blocks
- Windows transport security on `/Retr`, which fails before the operation with
`ProtocolException: The requested upgrade is not supported`; `/Retr` expects
the plain MDAS Net.TCP binding even when `/Hist-Integrated` uses Windows
transport security for `Open2`.
- remote `<historian-host>` WCF start-query parity: all 22 reconstructed request
variants open `/Hist-Integrated` successfully and pass
`Retr.IsOriginalAllowed`, but `StartQuery2` returns `false` with zero
response/error sizes; legacy `StartQuery` returns code `238` for each. The
legacy call also returns zero response size and no response buffer, so code
`238` is not a discarded response payload. The stored artifact redacts
transient handle values.
- a bounded current-session replay of the first byte-matched full-history
candidate again opened `/Hist-Integrated`, passed `Retr.IsOriginalAllowed`,
and returned `StartQuery2 = false` with zero response/error bytes. The legacy
`StartQuery` operation now faults with a server null-reference instead of
returning a useful response. This reinforces that the blocker is the native
session/client-handle state from `OpenConnection3`, not the 251-byte
`DataQueryRequest` payload.
The Galaxy DB query path also confirmed alternate historized tags, including
`TestMachine_001.TestHistoryValue`. Native direct local reads succeed for that
tag, while native non-direct local history reads now report an
`SEHException` from `DataQueryResponse.Load` after successful open/connect.
Native event queries still succeed. This means the current local server remains
useful for direct native value parity and event flow evidence, but it is not yet
a clean oracle for WCF `StartQuery2` history response bytes.
Running the native harness executable without its WCF diagnostics `.config`
does not change the non-direct history failure, so message logging is not the
cause.
Event request serialization has one stronger fact set than history now:
`EventQueryRequest.Save<SByteStream<SCrtMemFile>>` writes version `5`,
start/end FILETIME, event count, skip count, order, query type, the empty filter
block, result buffer size `0x10000`, timezone `UTC`, empty metadata namespace,
and empty selected-property count. The managed `wcf-start-event-query` probe
serializes that 65-byte request and records the SHA-256 in
`docs\reverse-engineering\wcf-start-event-query-attempts-latest.json`.
The WCF `queryRequestType` for event starts is now confirmed as `3`.
`CRetrievalConnectionWCF.StartEventQuery` also contains a version dispatch:
server interface versions `<= 2` use `/Retr.StartQuery2`, while newer versions
use `/Retr.StartEventQuery`. The latest direct managed probe tries both
operations with the same 65-byte request.
Event Open2 setup is also partially mapped. `SetConnectionMode(Event,
integratedSecurity: true)` yields connection mode `1281` (`0x501`), while the
process read-only integrated mode is `1026` (`0x402`). The latest managed probe
tries both modes and three handle candidates: returned Open2 handle,
`clientHandlerArray[1] == 1`, and native `eventTagHandle == 10000000`. All
currently return `false` with empty error buffers through both
`StartEventQuery` and `StartQuery2`.
Native event open performs one extra setup step that the managed probe does not
yet reproduce: `CreateDefaultEventTag` creates `CM_EVENT` with description
`AnE Event`, engineering unit `NONE`, and data type `10`; it then calls
`AddTagInternal`, which reaches `HistorianClient.AddHistorianTag`.
The immediate `aahClientCommon.CClient.RegisterTag` continuation is client-local
state: it inserts the tag id into local sets, marks the cached tag info as
registered, and flips synchronization flags. The `RTag2` WCF edge now has one
negative probe: its input buffer is a serialized GUID vector (`uint32 count`
followed by 16-byte GUIDs), but `wcf-register-event-tag` failed for empty,
zero-GUID, and event-handle-shaped vectors across Open2-derived GUID handle
candidates. First-16 handle candidates return native error `51`; the bytes 4-19
.NET GUID candidate reaches deeper and returns native error `1`. The richer
`AddHistorianTag` / `CTagMetadata` server path remains the leading candidate for
why native event queries start successfully while direct managed event-start
calls do not.
The default event tag metadata path is now partially concrete:
`ConvertEventTagToTagMetadata` maps `CM_EVENT` to event tag id
`353b8145-5df0-4d46-a253-871aef49b321`, common event type id
`5f59ae42-3bb6-4760-91a5-ab0be01f2f27`, `CDataType = 5`, and storage type `2`.
`CTagMetadata.Save` writes a one-byte version, two-byte optional-field mask,
the data-type byte, tag id, and compact UTF-16 strings. The current managed
`wcf-add-event-tag` probe uses these facts to try two payload variants through
`IHistoryServiceContract.AddTags` (`AddT`), but both return `AddReturnCode = 4`
with no output buffer and event query start remains `false`. This is useful
negative evidence: the current approximation is not the missing server state.
Two capture routes were also ruled out for this event setup. First, a fake WCF
server bound to an alternate port saw only startup and no native calls while the
native event harness still queried the real local Historian, so the native WCF
endpoint is not redirected simply by changing `HistorianConnectionArgs.TcpPort`.
Second, an expanded Frida pass installed hooks for
`CreateDefaultEventTag`, `AddTagInternal`, `AddHistorianTag`,
`ConvertEventTagToTagMetadata`, and `CTagMetadata.Save` before open; the event
query succeeded, but no hook enter/leave callbacks fired. MethodDef RVAs are
not viable runtime interception points for this mixed-mode path.
Third, the native harness now has a `--pre-load-sleep-seconds` option so Frida
can attach before `aahClientManaged.dll` is loaded. A preload Winsock/IPC pass
for the successful local event scenario (`winsock-event-preload-localhost-latest.ndjson`)
still recorded only hook installation plus the native success snapshot, with no
Historian-port socket traffic and no tracked named-pipe/file payloads. The local
event success path is therefore below or outside the current Winsock/pipe hook
set.
The following are not yet known:
- native version-3/version-4 open buffer differences
- exact native client/session field that maps legacy handle `2` to the `/Retr`
WCF `clientHandle` consumed by successful native `StartQuery2`.
- why direct managed `Retr.StartQuery2` fails with handles decoded from
managed `Open2` even though the native wrapper's WCF call succeeds with a
handle from the same operation family.
- whether `GetNextQueryResultBuffer2`'s five-byte error buffer on a successful
row batch is a terminal-status marker, a warning, or a harmless native error
scratch buffer.
- query result buffer layouts
- event default tag registration (`CM_EVENT`) and result buffer layout
- broader wildcard browse behavior for multi-batch and empty-result cases
- non-Open2 native error buffer variants
- write request/result buffer layouts
New tag-query evidence narrows the metadata path. The latest combined IL
instrumentation shows successful native `CRetrievalConnectionWCF.StartTagQuery`
uses a 4-byte WCF request (`51 67 01 00`) and a native retrieval-session GUID
string, not the 92-byte filter-bearing request shape. Direct managed replay of
both shapes with numeric, GUID, hex, base64, fresh-GUID, and captured-GUID handle
candidates failed with native errors `51` or `1`. This means
`StartTagQuery` depends on earlier native session/filter registration. Do not
wire metadata through guessed `StartTagQuery`; the SDK now uses the proven
`GetTagInfoFromName` direct WCF metadata path. Instrument the earlier
metadata-layer registration call only if `StartTagQuery` pagination becomes
necessary later.
Until these buffers are decoded and fixture-backed under `fixtures\protocol\2020`,
the SDK cannot truthfully perform real Historian reads or writes.
## Next Capture Pass
Run these first:
```powershell
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- manifest
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- exports current\aahClient.dll
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- mark connect-process
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Test-AahClientManagedOpen.ps1 -IntegratedSecurity
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Test-AahClientManagedReadIntegrated.ps1
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Find-GalaxyHistorizedTags.ps1 -Limit 25
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Attach-AahClientManagedFridaCapture.ps1 -TagName OtOpcUaParityTest_001.Counter
.\tools\AVEVA.Historian.NativeTraceHarness\bin\Debug\net481\AVEVA.Historian.NativeTraceHarness.exe --tag OtOpcUaParityTest_001.Counter --lookback-minutes 1440 --max-rows 1
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Attach-NativeTraceHarnessWinsockCapture.ps1 -ServerName <historian-host> -Scenario history -RetrievalMode Full -TagName OtOpcUaParityTest_001.Counter -LookbackMinutes 1440 -MaxRows 1
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Run-DebianHistorianRelayCapture.ps1 -SshUser <relay-user> -SshHost <relay-host> -TargetHost <historian-host> -OutputPath .\docs\reverse-engineering\debian-relay-history-latest.ndjson -HarnessOutputPath .\docs\reverse-engineering\native-trace-harness-via-debian-relay-latest.json
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Run-DebianHistorianRelayCapture.ps1 -SshUser <relay-user> -SshHost <relay-host> -TargetHost <historian-host> -OutputPath .\docs\reverse-engineering\debian-relay-history-direct-latest.ndjson -HarnessOutputPath .\docs\reverse-engineering\native-trace-harness-via-debian-relay-direct-latest.json -HarnessExtraArgs @("--direct-connection")
```
Then execute the matching native SDK scenario while recording API Monitor,
Detours-style lower-boundary hooks, or a CLR profiler/managed method rewrite.
Do not use MethodDef RVAs as Frida hook targets, and do not treat
`MethodHandle.GetFunctionPointer()` as sufficient by itself; the latest
same-process runtime-pointer Frida pass installed hooks but saw no callbacks
during a successful direct history read. Also do not assume `aahClient.dll`
exports are on the wrapper path; the latest export-hook pass did not observe
that DLL being loaded. Windows built-in packet capture produced empty PCAPNG
files for local loopback on this machine, and classic WCF diagnostics did not
produce message logs in the native harness process. A remote Historian server
target would be better for packet/Winsock capture because the installed local
server path appears to bypass observable client-process network and pipe IO.
Add only sanitized decoded buffers or fixture bytes to the repo.
Current Frida evidence is now broad enough that repeating export-level hooks is
unlikely to pay off. The next practical evidence path is one of:
- API Monitor/Detours with deeper mixed-mode or WCF-native call interception.
- CLR profiler/IL instrumentation around the C++/CLI wrapper's managed entry
and WCF contract calls.
- ETW/WFP/kernel-level network tracing for the relay path, then correlate to
the sanitized relay transcript and harness state.
The pktmon path now covers metadata/timing. It still cannot provide query bytes
because the relayed session fails before `ConnectedToServer = true`; it is a
correlation aid, not the protocol parser input.
The local WCF capture server cannot currently intercept the native `localhost`
path: the native wrapper ignores `TcpPort` for local machine targets and still
uses the installed Historian endpoint. See `native-open-capture-server.md`.
Additional confirmed status evidence is in `wcf-status-localhost.md`: the
`Stat` endpoint is reachable, but WCF `GetServerTime` is a native no-op stub and
handle-dependent status calls return failures with handle `0`.