77 Commits

Author SHA1 Message Date
Joseph Doherty f2297315b9 docs(c2): correct the WCF finding — transport+auth viable, row-retrieval server-gated
Overturns the earlier wrong "WCF not served on 2023 R2" conclusion (that was a
test error: wrong port/transport for the reverse tunnel). Corrected: the cert
(TLS) transport + NegotiateAuthentication auth reach the 2023 R2 historian
cross-platform; the 0x501 event connection mode makes CM_EVENT RegisterTags
succeed; yet StartEventQuery returns a 0-row buffer + long-polls over a window
that has events. Registration and window ruled out -> the same server-side
per-connection row gate as gRPC. Event reads stay server-gated over BOTH
transports; not client-fixable. Evidence doc rewritten; gRPC + WCF orchestrator
gating messages corrected.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-26 04:41:21 -04:00
Joseph Doherty de8d5e91ce feat(wcf): EventReadConnectionModeOverride + cross-platform/bounded C2 spike
Live investigation (direct from a VPN host to the 2023 R2 historian's real WCF
port) showed the certificate transport + NegotiateAuthentication auth work
cross-platform, and that the event-read chain needs the 0x501 event connection
mode for CM_EVENT RegisterTags to succeed (0x402/0x401 fail). Even with
registration succeeding over a window that has events, StartEventQuery returns a
0-row header and long-polls — the same server-side per-connection row gate proven
for gRPC. Adds: EventReadConnectionModeOverride (diagnostic), and spike knobs —
cross-platform cert gate, version-check bypass, per-call timeout, overall budget
with phase-diagnostic dump, connection-mode override.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-26 04:38:58 -04:00
Joseph Doherty 8777c0b816 fix(wcf): set Via via CreateChannel(address, via) — ClientViaBehavior absent in .NET WCF
ClientViaBehavior is a .NET Framework type not present in the System.ServiceModel
client libraries. Use the portable ChannelFactory.CreateChannel(EndpointAddress, Uri)
overload instead, via a CreateChannel helper applied at the history-open and
retrieval-query sites (the critical event path). Fixes the build break in 954b9cc.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 20:37:38 -04:00
Joseph Doherty 954b9cc9cc feat(wcf): add ConnectViaAddress (WCF Via) for tunneled historian access + wire into C2 spike
When the historian is reached through a port-forward whose local port differs
from the server's real service port, WCF's server-side AddressFilter rejects the
message (To = tunnel port != server port). ConnectViaAddress lets the channel
connect to the tunnel while addressing the SOAP To the real Host/Port endpoint.
Applied in HistorianWcfClientCredentialsHelper.Configure (the critical event
factories already call it). The C2 spike reads HISTORIAN_WCF_EVENT_VIA.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 20:35:46 -04:00
Joseph Doherty 7992e43908 test(c2): make WCF spike transport-selectable (integrated|certificate) + opt-in verbose
The first live run used the wrong port (32568 direct vs the 42568 WCF tunnel) and
hardcoded RemoteTcpIntegrated; via the tunnel the error advanced from socket-RST
to ProtocolException (binding/security mismatch). Add HISTORIAN_WCF_EVENT_TRANSPORT
(certificate), _DNSID, _ALLOW_UNTRUSTED, and an opt-in _VERBOSE for live binding
diagnosis. Default output stays sanitized; still Windows-only, never fails the suite.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 20:18:23 -04:00
Joseph Doherty 64c9793b91 docs(c2): correct event-read gating messages — WCF not served on 2023 R2
The gRPC ReadEvents throws no longer advise "use the WCF transport for event
reads": that path is moot on 2023 R2 (net.tcp is reset at the framing layer,
live-disproven 2026-06-25). Messages now state event reads are auth-solved but
server-gated over gRPC and have no WCF fallback on 2023 R2, citing the two
evidence docs. WCF orchestrator remarks scoped to legacy 2020 historians; row
layout noted as decoded. String/comment only; throw behavior unchanged.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 17:11:19 -04:00
Joseph Doherty f1c57f7149 docs(c2): record WCF event-read spike live result (RED — transport not served on 2023 R2)
WCF net.tcp (RemoteTcpIntegrated) against the live 2023 R2 historian is reset
at the socket-write/framing layer before any auth — both the event spike and a
basic Probe/ReadRaw throw the identical CommunicationException/SocketException
("forcibly closed by the remote host"). The 2023 R2 box does not serve the
legacy WCF transport; C2's "route via WCF" unblock is moot on this server class.
Sanitized: counts + native return codes + buffer lengths only.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 16:58:56 -04:00
Joseph Doherty 85f3bd4b4e test(c2): env-gated WCF event-read diagnostic spike (RemoteTcpIntegrated)
Drives HistorianWcfEventOrchestrator over RemoteTcpIntegrated and dumps the
native chain (UpdC3/RTag2/EnsT2 return codes, result-buffer length, row count)
to settle whether WCF event reads return rows on an event-bearing 2023 R2 box.
Windows-only, gated by HISTORIAN_WCF_EVENT_HOST, never fails the suite, inert
off Windows. Sanitized: counts + return codes + buffer lengths + sha256 only.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 16:40:21 -04:00
Joseph Doherty 5a7a28872b Merge feat/c1-numeric-writes: Int8/UInt8 live writes + C3a evidence (UInt1 server-blocked)
Int8/UInt8 analog tag-create + historical-value write encoders un-gated and
live-proven; golden-pinned. UInt1 attempted but re-gated — the historian stores a
degenerate UInt1 analog tag. 2023 R2 gRPC interface-version integers captured (C3a).

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 16:07:24 -04:00
Joseph Doherty 100b44a365 docs(write): drop UInt1 from value-layout table; mark Int8/UInt8 live-proven
Removes the stale UInt1 → UInt8(1) entry from the EncodeNativeValue layout
table (UInt1 is re-gated; the prose already said so). Int8/UInt8 layout note
updated from "pending" to live-proven.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 15:42:50 -04:00
Joseph Doherty aa8ca2f6ad docs: Int8/UInt8 analog writes supported (live-proven); UInt1 server-degenerate
EnsureTagAsync + AddHistoricalValues now cover Int8/UInt8 (codes 0x19/0x39, native
LE int64/uint64 value). UInt1 documented as not-supported: the server stores a
degenerate analog tag (GetTagInfo stub, type byte 0x00).

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 15:34:34 -04:00
Joseph Doherty 95b924cdbe fix(write): re-gate UInt1 — historian creates degenerate UInt1 analog tags
Live evidence: EnsureTags(UInt1) returns success but the server stores a
degenerate tag (descriptor type byte 0x00, no GUID/name), so GetTagInfo
truncates and writes fail. Not client-fixable on the analog path. Int8/UInt8
stay GREEN (live-proven). UInt1 reverts to ProtocolEvidenceMissingException
(fail-closed); golden rows removed; negative-gate tests added. Removed the
one-off capture diagnostic.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 15:27:04 -04:00
Joseph Doherty aa56d2d81b docs(re): correct Status interface-version comments (4 on gRPC, not 0)
Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 15:03:22 -04:00
Joseph Doherty dea9107b4b docs(re): capture + record 2023 R2 gRPC interface-version integers (C3a)
Env-gated live evidence test reads History/Retrieval/Transaction/Status
GetInterfaceVersion over gRPC; integers recorded in grpc-interface-versions.md.
Stale not-yet-captured comment fixed; gate XML-doc notes live confirmation.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 14:58:45 -04:00
Joseph Doherty 79bb1d9e06 test(write): golden-pin UInt1/Int8/UInt8 tag-create + value buffers
New InlineData rows derived from the captured Double baselines (type-code /
value bytes swapped). Negative-gate tests retained for still-gated types.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 14:49:21 -04:00
Joseph Doherty 43c2587498 feat(write): un-gate UInt1/Int8/UInt8 tag-create + historical-value encoders
GetAnalogDataTypeCode gains UInt1=0x08/Int8=0x19/UInt8=0x39 (read-side
MapDataType already maps these); EncodeNativeValue gains 1-byte UInt1 +
8-byte LE Int8/UInt8 (mirrors the captured Double value layout). Value API
stays double; 2^53 exact-magnitude ceiling documented. Golden tests + live
round-trip follow.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 14:38:27 -04:00
Joseph Doherty e04eb539f7 Merge feat/amortization-broadening: event-reuse spike + HistorianEventSession + browse/metadata seams
Track A (tag-client …OnSession seams + HistorianSession browse/metadata), the GREEN
event-session reuse spike (B0), and the HistorianEventSession primitive. Live-validated.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 12:58:45 -04:00
Joseph Doherty 2687b2b6d2 feat: HistorianEventSession primitive + OpenEventSessionAsync (v8 Event reuse)
Reusable v8 Event session wrapping the B0a seams (OpenAndRegisterEventSession once +
SendEventOnSession per op) with a GetSystemParameter keepalive; idempotent dispose.
Mirrors HistorianSession (v6 sibling). pending.md A1 broadening, Stage B1.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 11:54:09 -04:00
Joseph Doherty 5f949a86e2 docs(spike): event-session reuse spike results — GREEN
v8 Event session reuses across SendEvent (~10-16x amortization); register-once
sufficient; session survived 25s idle; event reads stay gated (C2). Live-validated
against wonder-sql-vd03. Gate decision: GREEN -> HistorianGateway Stage B1 (separate
event-session pool for SendEvent) warranted.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 11:34:34 -04:00
Joseph Doherty 81da404c5d test(spike): bound event read-after-send so the spike can't hang on C2 long-poll
The Q3 read-after-send probe (ReusedEventSession_ServesReadAfterSend_BestEffort)
long-polled GetNext to the no-data terminal with no tight bound and ran past the
5-min suite timeout on the live run. Bound it two ways: a read-only options copy
with a 5s RequestTimeout (so each GetNext RPC deadlines fast) and an 8s
CancellationToken passed as ct. Either fuse returns the method in ~10s; the
timeout/cancellation is logged as the expected C2-gated outcome (still no assert).
Other three spike methods unchanged.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 11:32:09 -04:00
Joseph Doherty 777a7700b4 test(spike): event-session reuse spike harness (env-gated, B0b)
Opens one v8 Event session and measures SendEvent reuse (register-once, send-many)
+ best-effort read-after-send + optional idle sweep. Skips offline; run live in B0c
to gate event amortization (pending.md A1 broadening, Stage B0).

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 11:02:34 -04:00
Joseph Doherty dc4141e718 feat(grpc): event-on-session seam for the reuse spike (SendEvent[+ReadEvents])
Extract SendEventOnSession (and best-effort RunEventQueryOnSession) so the B0b spike
can run multiple event ops on one already-opened v8 Event session. RegisterCmEventTag
made independently callable. Behaviour-preserving (pending.md A1 broadening, Stage B0).

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 10:39:40 -04:00
Joseph Doherty 81aff03748 feat: HistorianSession browse + metadata on the reused session (+ env-gated round-trip)
Browse mirrors ReadRawAsync (collect-then-yield); metadata mirrors ReadAtTimeAsync
(unary). Delegates to the HistorianGrpcTagClient …OnSession seams so a leased session
browses + reads metadata without re-handshaking (pending.md A1 broadening).

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 10:28:44 -04:00
Joseph Doherty 2f689cbe71 feat(grpc): …OnSession seams on HistorianGrpcTagClient (browse + metadata)
DRY split mirroring HistorianGrpcReadOrchestrator.RunRawQueryOnSession: browse +
GetTagInfos(metadata) gain externally-supplied connection+session seams; per-call
wrappers delegate. Behaviour-preserving (pending.md A1 broadening).

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 10:23:40 -04:00
Joseph Doherty be60d0b8d9 test: HistorianSession end-to-end round-trip (write+read+status+ping on one session, env-gated)
Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 03:09:18 -04:00
Joseph Doherty 15da8516ba feat: HistorianClient.OpenSessionAsync(kind) — open a reusable HistorianSession 2026-06-25 03:07:02 -04:00
Joseph Doherty d2b00fcda5 fix: idempotent HistorianSession.DisposeAsync + upfront cancellation check 2026-06-25 03:05:10 -04:00
Joseph Doherty ad2c02eb42 feat: HistorianSession primitive — reusable authenticated session over the …OnSession seams 2026-06-25 02:59:36 -04:00
Joseph Doherty d42019f481 feat(grpc): HistorianSessionKind + …OnSession seams (read/status/tag-write) 2026-06-25 02:52:32 -04:00
Joseph Doherty a0b5d35e48 docs: write-spike results — write-reuse GREEN, ONE-KIND pool confirmed 2026-06-25 02:11:10 -04:00
Joseph Doherty 9909921e87 test(grpc): write-reuse + read-on-write-session probe (one-vs-two-kind decider) 2026-06-25 02:08:23 -04:00
Joseph Doherty f8db01fd7f feat(grpc): add RunWriteOnSession seam for write-reuse spike 2026-06-25 02:04:52 -04:00
Joseph Doherty 3849f17746 docs: handshake-reuse spike results + verdict (GREEN, idle timeout ~20-25s) 2026-06-25 01:19:50 -04:00
Joseph Doherty 899f9ccf6b test(grpc): env-gated handshake-reuse spike (validity + latency + idle sweep) 2026-06-25 01:10:19 -04:00
Joseph Doherty 96f10372de feat(grpc): add RunRawQueryOnSession seam for handshake-reuse spike 2026-06-25 01:06:03 -04:00
Joseph Doherty 7d8be48d89 Merge handoff-sendevent-persistence: record SendEvent gRPC persistence finding
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 17:36:36 -04:00
Joseph Doherty f2c442bbaf handoff: record SendEvent gRPC persistence finding
Item 3: the gRPC read-back against the live 2023 R2 server proves the SDK-sent
event PERSISTS (independently read back from event history), resolving the older
WCF/M2 caveat ("server accepts AddS2 but the local dev box does not persist to
v_AlarmEventHistory2") as an environment artifact, not an SDK gap. Also retire the
stale "fresh native capture (SendEvent gRPC framing)" next-step note — SendEvent
over gRPC is now shipped + live-validated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 17:35:09 -04:00
Joseph Doherty 32d508eed4 Merge doc-sendevent-claudemd: document SendEventAsync in CLAUDE.md
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 17:30:27 -04:00
Joseph Doherty df9751f066 CLAUDE.md: document SendEventAsync (both transports)
Add SendEventAsync to the write surface: M2 event-send, now on both transports.
WCF runs Open2 event-mode -> CM_EVENT registration -> AddS2; gRPC runs v8 Event
OpenConnection -> registration -> HistoryService.AddStreamValues. Both carry the
same "OS" event VTQ buffer (no distinct event RPC, not the "ON" historical buffer);
gRPC path live-validated end-to-end. Only original events with string properties.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 15:55:13 -04:00
Joseph Doherty 5ad1adb429 Merge handoff-refresh-counts: SendEvent over gRPC + 2023 R2 binary-dive verdicts
- SendEvent over gRPC SHIPPED + live-validated (HistorianGrpcEventWriteOrchestrator;
  reuses the WCF "OS" event buffer verbatim on HistoryService.AddStreamValues).
- 2023 R2 stock-client binary dive: sharpened every pending item to evidence-based
  verdicts; DeleteTagExtendedProperties confirmed walled via native capture.
- Event registration (RegisterTags/EnsureTags) golden-tested vs live capture.
- Handoff date/test-count refresh; revision-probe scope comment fix.
331 offline tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 15:52:56 -04:00
Joseph Doherty dd57d212f8 DeleteTagExtendedProperties: confirm walled via native capture (do not ship)
Tested the binary-dive "use deleteFromServer=true" hypothesis directly against
the native client (local 2020 WCF box, Capture-DeleteTagExtendedProperties.ps1
cross-session sync trick). Result: the native DeleteTagExtendedPropertiesByName
with deleteFromServer=true returns Success=true, but the property is re-fetchable
and re-deletable across repeated fresh sessions — it is NEVER durably removed.

So the native client itself only performs an optimistic client-side cache delete
the server does not durably honor (the HCAL cache-sync model the decompile shows).
This supersedes the earlier "code=1, prop survives" note (that was the same-session
sync-gate failure; with proper cross-session sync it returns Success yet still does
not durably delete). A managed DeleteTagExtendedPropertiesAsync would return a
misleading success, so it correctly stays unshipped. Handoff item 7 updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 15:44:53 -04:00
Joseph Doherty afc7c4bf96 SendEvent over gRPC: implement + live-validate (was capture-gated)
Captured the native 2023 R2 client's gRPC event send (new capture-send-event
harness scenario): it rides HistoryService.AddStreamValues with the SAME "OS"
(0x534F) storage-sample buffer the WCF path already uses (HistorianEventWrite-
Protocol) — confirming "no distinct RPC", and that it is NOT the historical
write's "ON" buffer. Diffed the write-enabled vs read-only v8 Event open: byte-
identical apart from per-session crypto, so the existing OpenSession event path
is reused unchanged.

So SendEvent-over-gRPC was pure assembly of proven parts:
- HistorianGrpcEventWriteOrchestrator = v8 Event open + CM_EVENT registration
  (UpdC3/RegisterTags/EnsureTags) + AddStreamValues(OS buffer).
- HistorianClient.SendEventAsync now routes to it for RemoteGrpc (WCF otherwise).

Live-validated end-to-end against the 2023 R2 server: pure-managed SDK send →
AddStreamValues BSuccess=true → the event reads back from the server (markers
confirmed in returned event rows). The native gRPC RegisterTags(24B) +
EnsureTags(86B) byte-match our serializers (new GrpcEventSendProtocolTests
golden, closing the 83-vs-86 EnsureTags question). Gated live test
SendEventAsync_OverGrpc_AcceptsEvent (opt-in HISTORIAN_GRPC_EVENT_SEND=1).
331 offline tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 15:37:22 -04:00
Joseph Doherty ae536bb4b8 Record 2023 R2 binary-dive verdicts; fix revision-probe scope comment
Mined the full decompiled stock 2023 R2 managed client as the oracle for every
still-pending gRPC item. Governing fact: ArchestrA.HistorianAccess is a C++/CLI
shim into native HistorianClient; the managed Grpc*Client wrappers have zero
call sites, so buffer-building/dispatch for the pending items is native (absent
from the binaries). Sharpened verdicts into handoff.md:

- Items 4/5/6 + OpenStorageConnection: hard-confirmed walled, with real reasons
  (SQL gated out client-side via IsManagedHistorian; no Revision RPC in the gRPC
  contract; LoadBlocks response is a native blob behind the storage console handle).
- Items 3 (SendEvent) and 7 (DeleteTEP): moved from vague to precise, LOCAL-box
  capturable targets (PackToVtq btValues / DeleteTagExtendedPropertiesByName
  BtInput with deleteFromServer=true).

Also correct the HistorianGrpcRevisionProbe doc comment: it probes the
non-streamed ORIGINAL-insert path (AddNonStreamValues), a distinct capability
from revision EDITS (native AddRevisionValues trio, no gRPC RPC) — the prior
comment conflated them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 15:13:11 -04:00
Joseph Doherty cac81c7e5c handoff: refresh date + offline test count after event-parser merge
Bring the doc header current with the merged event-row parser fix: bump
"Last updated" to 2026-06-23 and correct the Build And Test known-good
result from 321/321 to the actual 328/328 (build clean 0/0), noting the
+7 are the HistorianEventRowProtocolTests golden + gated-capture coverage.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 14:40:57 -04:00
Joseph Doherty 0037ab8ca9 Merge handoff-event-parser-update: record event-row parser fix + mark gRPC event angles exhausted
Docs-only. Updates handoff.md item 1 to reflect that all four next-session angles for the
gRPC event zero-rows are tested and ruled out (transport, metadata/cert, topology, data
store) and that the parse path is verified against the provided client with a latent
multi-row bug fixed. Corrects two historical event-row-layout skeletons that described
0x1E as a per-row marker (it is a one-time buffer header field; rows are markerless).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 14:13:39 -04:00
Joseph Doherty 6a67a8366c handoff: record the event-row parser fix + mark the gRPC event angles exhausted
Updated the live-blocker item (1) to reflect that all four next-session angles for the
gRPC event zero-rows are now tested and ruled out (transport, metadata/cert, topology,
data store) and that the parse path is verified against the provided client with a latent
multi-row bug fixed. Corrected the two historical event-row-layout skeletons that wrongly
described 0x1E as a per-row marker: it is a one-time buffer header field and rows are
markerless (which is why the old parser returned only the first row of any multi-row
buffer, on WCF too).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 14:12:45 -04:00
Joseph Doherty 6faf8a5f30 Merge grpc-event-decompile-confirm: stock-client decompile + frame capture + event-parser fix
Three steps verifying the gRPC event read against the provided 2023 R2 client:

- Decompiled the stock managed client (Archestra.Historian.GrpcClient, HistorianAccess):
  confirms no hidden client-side difference. The stock client is gRPC-Web/HTTP-1.1 (same
  transport as ours), m_metadata is gzip-only, the ClientInterceptor is a no-op, and it
  presents no TLS client cert. The event-query orchestration lives in the native C++ core.
- Captured decrypted HTTP/1.1 frames of a native capture-event (50 rows) vs our SDK (0
  rows) through a TLS-terminating tee proxy. Found the native splits services across 5
  connections and runs the event query on a dedicated RetrievalService connection; tested
  replicating that (HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL) -> still 0 rows, so the server
  correlates by session handle, not connection. Topology is not the gate.
- Verified the parse path against the provided client's real result buffer (50 events) and
  fixed a latent bug: the event-row buffer is version + rowCount + a one-time 0x1E header
  field then MARKERLESS rows; the parser wrongly treated 0x1E as a per-row marker and
  decoded only the first row of any multi-row buffer. This also affected the shipped WCF
  event read (identical v9 header). Fixed to a 10-byte header + markerless rows, accepting
  version 9 (WCF) and 11 (gRPC). The real 50-row buffer now decodes to exactly 50 events.

Net: every client-side angle for the gRPC zero-rows is exhausted (payload, transport,
metadata/cert, topology, data store) -> the gate is a server-internal per-connection
retrieval working-set. The parse path is now verified against real 2023 R2 event data on
both transports, and WCF event reads now correctly return all rows of a multi-row buffer.
gRPC event-row retrieval stays auth-solved / parse-verified / retrieval-server-gated.
328 offline tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 14:08:10 -04:00
Joseph Doherty 8f4a188f78 Event-row parser: verify against the provided 2023 R2 client; fix latent multi-row bug
Used the provided stock client as an oracle to verify the event read path. The
capture-event harness returns 50 real events, and the instrument-grpc-nonstream rewrite
captured the exact GetNextEventQueryResultBuffer.result buffer (63,192 bytes, version
0x0B=11, rowCount 50 = 25 Alarm.Set + 25 Alarm.Clear). Feeding that real buffer through
HistorianEventRowProtocol.Parse exposed a latent parser bug.

The real buffer layout is: version(2) + rowCount(4) + headerField(4, =0x1E) followed by
MARKERLESS rows (rowFormat(2)=7 + filetime(8) + 8x u16 slots + compact-ascii type +
propCount + props). The parser wrongly treated the one-time 0x1E field as a per-row
marker and re-consumed [marker+format] for every row, so it decoded only the FIRST row
of any multi-row buffer and stopped. This is not gRPC-specific: the captured WCF v9
buffer has the identical 0900 <rowCount> 1E000000 0700 header, so the shipped WCF event
read had the same latent multi-row truncation.

Fix: read a 10-byte buffer header (skip the 0x1E field once) and parse markerless rows;
accept container version 9 (WCF) and 11 (gRPC), mirroring the interface-version gate that
accepts History 11 and 12.

Verified: the real 50-row buffer now decodes to exactly 50 events, ending cleanly at
end-of-buffer (Parse_RealStockClientCapture_DecodesAllEvents, gated on
HISTORIAN_EVENT_CAPTURE_NDJSON so it skips without the gitignored capture), plus a
synthetic v11 golden test. 328 offline tests pass.

The parse path is now verified against the provided client's real event data on both
transports; the only remaining gap for gRPC events is the server delivering rows to our
connection (the documented retrieval-server-gate).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 14:05:35 -04:00
Joseph Doherty 8199dde452 gRPC events: capture decrypted HTTP/1.1 frames native vs ours — topology found + tested null
Pursued hypothesis #3 (connection-frame capture). Built a TLS-terminating tee proxy
(artifacts/.../httpcap/, gitignored: self-signed server cert, forwards through the
loopback tunnel, logs decrypted HTTP/1.1 + gRPC-Web both directions) and ran a native
capture-event (returns 50 rows) and our SDK diagnostic (0 rows) through the SAME
proxy/upstream for a clean A/B.

Findings:
- The stock client is gRPC-Web/HTTP-1.1 (alpn empty), and clientCert=none on every
  connection — confirming (with the decompile) that hypothesis #2 (TLS client cert) is
  moot: the native presents no client cert.
- Connection topology differs: the native opens 5 TLS connections, one per service, and
  runs the event query (StartEventQuery/GetNext/EndEventQuery) on a DEDICATED
  RetrievalService connection, separate from the HistoryService connection that opened and
  registered the session. Our SDK collapses every service onto one connection. (Matches the
  decompile: the stock client has a separate GrpcClientBase per service.)
- Framing differs benignly: native uses content-length + Expect: 100-continue; SDK uses
  transfer-encoding: chunked. The server accepts both (StartEventQuery returns a valid
  handle), so framing is not the gate. No hidden header on either side.

Tested the topology hypothesis with a new env-gated switch
(HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL=1): run StartEventQuery/GetNext/EndEventQuery on a
dedicated RetrievalService connection (no re-handshake, reusing the session handle —
mirroring native conn4), registration staying on the main connection. Result: still
0B00000000001E000000 (0 rows), QH=1063. Splitting the event query onto its own connection
does not make rows flow — the server correlates by session handle, not connection, so
topology is not the row-scoping gate.

Every angle is now exhausted (payload, transport, metadata/interceptor/cert, topology,
data store). The gate is a server-internal per-connection retrieval working-set in the
native HistorianClient C++ core, unreachable from a pure-managed client. Conclusion
unchanged: auth-solved / retrieval-server-gated; ReadEventsAsync over gRPC keeps the
no-row throw, event reads use WCF. 56 offline gRPC/event tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 13:43:21 -04:00
Joseph Doherty 6cf4dd13fe gRPC events: decompile the stock managed client — confirms no hidden client-side difference
Closed the blind spot the zero-rows conclusion had leaned on: prior cycles used wire
capture (instrument-grpc-nonstream hooks only byte[] params), blind to gRPC metadata,
interceptors, channel options. Read the stock managed source directly
(histsdk-2023r2-analysis/decompiled/Archestra.Historian.GrpcClient + HistorianAccess;
the pure-managed assemblies decompile cleanly though mixed-mode aahClientManaged crashes
ILSpy).

Findings:
- GrpcClientBase.InitializeBase uses GrpcWebHandler (GrpcWebMode, HttpVersion 1.1) — the
  stock client speaks gRPC-Web over HTTP/1.1, the SAME transport as our SDK. This corrects
  the premise of hypothesis #1: there was never a native Grpc.Core HTTP/2 path to differ
  from; the stock client returning 50 rows is itself gRPC-Web. The HTTP/2 disproof's
  conclusion stands and is reinforced (identical transport on both sides).
- m_metadata on every RPC (incl. StartEventQuery/GetNextEventQueryResultBuffer) is only
  grpc-internal-encoding-request: gzip — exactly our header set. The ClientInterceptor is
  a no-op (empty LogCall). So the "invisible per-connection metadata/header" blind spot is
  confirmed empty — no hidden client-side identity the byte[] capture missed.
- CreateEventQuery/StartQuery/MoveNext are not in managed code; the managed
  GrpcRetrievalClient.StartEventQuery is a thin one-RPC stub. The query logic lives in the
  native C++/CLI HistorianClient core — consistent with the working-set being native/server-side.

Every client-controllable layer is now confirmed identical by reading the stock source,
not just by wire match: request bytes, transport, channel options, gRPC metadata,
interceptor. The remaining difference is below the managed surface / server-side.
Conclusion unchanged: gRPC event-row retrieval is auth-solved / retrieval-server-gated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 13:20:39 -04:00
Joseph Doherty e091965d59 Merge grpc-event-http2-disproof: gRPC event row retrieval — transport + SQL angles exhausted
Pursued the two server-side/connection hypotheses from grpc-event-query-capture.md
for the gRPC event zero-rows blocker:

- #1 transport (DISPROVEN): native HTTP/2 (no GrpcWebHandler) returns the byte-identical
  version-11 rowCount-0 terminal as gRPC-Web; the full v8 chain (ExchangeKey auth,
  CM_EVENT registration, StartEventQuery) runs end-to-end over both. Added
  HistorianGrpcChannelFactory.CreateHttp2 + HISTORIAN_GRPC_EVENT_HTTP2 switch.
- #4 SQL ground truth (ANSWERED): the event store has no per-connection column; the
  rich Events view is served live by the Historian engine via the INSQL OLE DB
  provider. Same engine + same window: 71,332 events via INSQL vs 0 via gRPC for our
  connection. The data is global/unscoped — the gate is the gRPC RetrievalService's
  per-connection in-process execution state, not data scoping or transport.

Three independent angles now exhausted (client payload byte-identical, transport
HTTP/2 == gRPC-Web, data store global). gRPC event-row retrieval stands documented as
auth-solved / retrieval-server-gated; ReadEventsAsync over gRPC keeps the no-row throw
and event reads use WCF. 326 offline tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 13:10:32 -04:00
Joseph Doherty f19eb3b821 gRPC events: answer hypothesis #4 (SQL ground truth) — event store is global, not connection-scoped
Pursued the server-side SQL angle for the gRPC event zero-rows. Built a read-only
SOCKS5-relay + Microsoft.Data.SqlClient probe (gitignored, artifacts/.../sqlschema/)
and dumped the live Runtime event schema. Findings:

- No per-connection / per-client / per-session column exists anywhere in the event
  store. The only scoping-like columns on Events/EventHistory/snapshots are event
  content (Source_* origin, User_* acker, Provider_NodeName, SourceServer replication).
- The rich Events view is not a relational table — it is served live by the Historian
  engine via the INSQL OLE DB provider (linked servers INSQL/INSQLD; encrypted remote
  view). The SQL EventHistory base table holds only 168 rows / 1 tag.
- Decisive: for the SAME -90d..now window the gRPC StartEventQuery diagnostic returned
  0 rows, the Events view via INSQL returns 71,332 events (most recent Alarm.Set firing
  seconds before the probe). Same engine, same window — INSQL serves the data, gRPC
  withholds it from our connection.

So there is nothing in the data to scope by: the zero-row gate is the gRPC
RetrievalService's per-connection in-process execution state, not data scoping or
transport (the same class of wall as DeleteTagExtendedProperties). Combined with the
transport disproof, three independent angles are now exhausted — client payload
(byte-identical), transport (HTTP/2 == gRPC-Web), and data store (global, unscoped).
gRPC event-row retrieval stands documented as auth-solved / retrieval-server-gated;
ReadEventsAsync over gRPC keeps the no-row throw and event reads use WCF.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 13:05:19 -04:00
Joseph Doherty b0388e7a40 gRPC events: disprove transport hypothesis (native HTTP/2 also returns zero rows)
Tested grpc-event-query-capture.md's leading next-session hypothesis — that the
native client's Grpc.Core HTTP/2 transport (vs our Grpc.Net.Client + GrpcWebHandler
gRPC-Web) is why event reads return zero rows. Added HistorianGrpcChannelFactory
.CreateHttp2 (plain HTTP/2 over SocketsHttpHandler, no gRPC-Web wrap) and an
HISTORIAN_GRPC_EVENT_HTTP2 switch on the event orchestrator (event path only; reads
stay gRPC-Web).

Live side-by-side against the event-bearing 2023 R2 server, everything else held
constant: the full v8 chain (ExchangeKey auth, CM_EVENT RegisterTags/EnsureTags=True,
StartEventQuery with a valid handle) runs end-to-end over BOTH native HTTP/2 and
gRPC-Web, and the server returns the byte-identical version-11 rowCount-0 terminal
(0B00000000001E000000) on both transports. Transport choice makes no difference —
the leading hypothesis is disproven and the zero-row scoping sits above the gRPC
transport layer.

Also confirmed the native capture-event harness queries a 30-day historical window
(returns 50 rows), so the native read is connection-scoped historical retrieval,
not a live subscription.

CreateHttp2 + the env switch + the EventChannelMode diagnostic are retained for
further connection-level probing. 44 offline tests pass; orchestrator stays on the
no-row throw.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 12:55:09 -04:00
Joseph Doherty 1161c40fd3 Merge docs-event-server-angle: server-side/connection angle documented for next session
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 12:42:48 -04:00
Joseph Doherty 88287a8c66 docs(grpc-events): document the server-side/connection angle for next session
Records the row-retrieval pickup now that the v8 ExchangeKey auth is solved and the
gap is proven connection-level (not client payload):
- grpc-event-query-capture.md: a "NEXT SESSION — the server-side / connection angle"
  section — what's already proven (don't redo), the in-place tooling, and ordered,
  testable hypotheses (HTTP/2 vs gRPC-Web transport [leading], TLS client cert,
  HTTP/2 frame capture, SQL event-store scoping).
- handoff.md item 1: updated to "v8 auth solved; row retrieval connection-gated",
  pointing at the NEXT SESSION section; the "to move any item" summary updated.

Doc-only; sanitization scan clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 12:37:22 -04:00
Joseph Doherty 9a25fa4ef7 Merge grpc-event-v8: v8 Event-connection ExchangeKey auth (ECDH+SHA256+MD5-keyed RC4) live-verified; row retrieval proven connection-gated
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 12:33:26 -04:00
Joseph Doherty 0921e21bdb feat(grpc-events): handle-capture cycle — event-row gap proven NOT a client payload issue
Extends the instrument-grpc rewrite to log string (strHandle) + uint (uiHandle /
queryRequestType) params, not just byte[], and captures our SDK's live v8
openParameters for a byte-diff against the native.

Result of the exhaustive comparison (all live-confirmed via the opt-in
EventReadDiagnostic test):
- StartEventQuery request: byte-identical to the native (v6 layout)
- v8 OpenConnection openParameters: byte-identical to the native (302B) once
  ClientNodeName matches — every control byte/ConnectionType/token/ShardId
- handle usage identical: ExchangeKey->contextKey, registration->storage GUID
  (strHandle), query->client uint (uiHandle); handles valid (RTag/EnsT=True)
- queryRequestType=3, registration order, gzip metadata header — all match
- window has events (native returns 50 now); eventCount not it

Every observable client-side byte matches the native, yet the server scopes 0
events to our connection. The event RPCs succeed over our transport and return a
valid EMPTY result (not a transport error), so this is a connection/server-level
difference (session affinity tied to the native Grpc.Core HTTP/2 connection or a
connection identity used to scope events) — invisible to and unfixable by client
payload matching. Needs server-side insight, not more wire RE.

Added opt-in diagnostics (RegistrationDiag, LastResultBufferHex,
LastEventOpenRequestHex). 326/326 offline; gated test still pins the no-row throw.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 12:31:33 -04:00
Joseph Doherty 32ae301050 feat(grpc-events): native-match event registration + skip ValidateClientCredential; diagnostics
Continues closing the event-row gap after the v8 ExchangeKey/RC4 auth breakthrough.

- HistorianGrpcHandshake: the v8 EVENT path skips StorageService.ValidateClientCredential
  (the native event connection authenticates purely via ExchangeKey + the RC4 token;
  running the Negotiate loop establishes a different session scope).
- HistorianGrpcEventOrchestrator.RegisterCmEventTag: simplified to the exact native
  gRPC event sequence (UpdateClientStatus -> RegisterTags -> EnsureTags -> GetHistorianInfo
  -> GetSystemParameter x7), dropping the 2020-WCF-era cross-service GetV probes and
  params-before-register that the gRPC event flow does not use. eventCount 5 -> 100.
- Opt-in diagnostics (RegistrationDiag, LastResultBufferHex/LastErrorBufferHex; gated
  EventReadDiagnostic test) for the continued investigation.

STATUS: auth + StartEventQuery + registration all succeed live (RTag/EnsT=True, valid
query handle), but GetNext returns version-11 rowCount-0 while the native returns 50 for
a BYTE-IDENTICAL request. Every observable wire element matches the native. The remaining
unknown is the string/uint HANDLE field VALUES the native uses per event RPC — the
instrument-grpc capture logged only byte[] params, not the handle fields. Next: extend
the IL rewrite to log strHandle/uiHandle/queryRequestType, re-capture, and match.
326/326 offline; gated test still pins the no-row throw.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 12:12:35 -04:00
Joseph Doherty 6d0f5c4b8f feat(grpc-events): implement aahCryptV2 token — v8 ExchangeKey auth now passes live
Implements the reverse-engineered v8 credential token in pure managed code and
wires the full event-connection auth chain. Live result: the v8 OpenConnection
now AUTHENTICATES against the 2023 R2 server (past the 132/171 AuthenticationFailed
wall) — the crypto is solved.

- HistorianNativeHandshake.DeriveExchangeKeyClientKey: client key = SHA256(ECDH
  shared secret) via ECDiffieHellman.DeriveKeyFromHash(SHA256), matching the native
  ECDiffieHellmanCng{Hash,SHA256}.DeriveKeyMaterial.
- BuildExchangeKeyCredentialToken + Rc4: token = RC4(password-UTF16LE, key=MD5(clientKey)).
  Reproduces a live-captured token EXACTLY (verified offline) — the native
  HistorianCrypto.NRC4_V2.aahCryptV2 scheme (MD5-keyed RC4). Pure managed; nothing
  AVEVA shipped. RC4 pinned by the standard test vector.
- OpenSession(eventConnection:true): ExchangeKey -> derive client key -> token ->
  v8 OpenConnection with ConnectionType=Event + the token. Orchestrator re-armed.
- HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc: the 86-byte native gRPC
  CM_EVENT EnsureTags (8-byte header + ...2f27 event-type GUID), replacing the
  2020 WCF 83-byte CTagMetadata on the gRPC event registration.

Goldens: RC4 standard vector + token construction. 326/326 offline.

KNOWN REMAINING: the event query still returns zero rows (GetNext yields a 10-byte
zero-row buffer). Auth + StartEventQuery succeed; the query-layer detail (vs the
native row-returning capture) is the last step. Gated test still pins the no-row
throw; opt-in diagnostic (HISTORIAN_GRPC_EVENT_DIAG) surfaces the journey.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 11:46:00 -04:00
Joseph Doherty c45f1a957b docs(grpc-events): token scheme fully RE'd via dnlib — aahCryptV2 (MD5-keyed RC4 + prefix)
Loaded dnlib in PowerShell (ILSpy crashes on the mixed-mode assembly) and scanned
the IL to recover the entire v8 token construction:

- <Module>::CHistoryConnectionGrpc.GetClientKey drives the ECDH: ECDiffieHellmanCng
  {KeyDerivationFunction=Hash, HashAlgorithm=SHA256, KeySize=256} -> ExchangeKey ->
  CngKey.Import(serverPub, EccPublicBlob) -> DeriveKeyMaterial = SHA256(shared secret),
  the 32-byte client key.
- aahClientCommon.CClientBase.ConfigureOpenConnection (the lone GetClientKey caller)
  builds the 26-byte token via HistorianCrypto.NRC4_V2.aahCryptV2 = a custom MD5-keyed
  RC4 stream cipher with a version prefix:
    * body/HashData = MD5 (verified by the round constants 0xd76aa478... + shifts 7/12/17/22)
    * prepare_key = RC4 KSA from a 16-byte MD5 key
    * enc_buffer = MD5 -> key, then rc4encrypt; enc prepends PrefixV2/InnerPrefixV2
      (the constant 0x8e token marker)
  So token = prefix + RC4(plaintext, key=MD5(keyMaterial)), keyMaterial tied to the
  SHA256(ECDH secret) client key. 100% reproducible in pure managed code (RC4+MD5).

Remaining (next cycle): read ConfigureOpenConnection's exact key/plaintext/prefix bytes,
implement aahCryptV2 managed-side, set the v8 token, live-test. Frida CNG + dnlib are
the RE path; nothing AVEVA is shipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 11:21:55 -04:00
Joseph Doherty b2ac35b98e docs(grpc-events): trace the ExchangeKey token crypto — KDF=SHA256(secret); token construction localized
Frida-hooked Windows CNG (scripts/frida/aahclientmanaged-cng-exchangekey.js) during
a real native ExchangeKey to recover the token derivation:

- The ECDH + KDF are standard CNG driven by managed System.Security.Cryptography
  .ECDiffieHellmanCng: NCryptSecretAgreement (P-256) -> NCryptDeriveKey(KDF=HASH,
  SHA256, 32 bytes). So the derived key = SHA256(ECDH shared secret).
- "ECK1" is the standard CNG BCRYPT_ECCPUBLIC_BLOB magic (P-256), confirming our
  BuildExchangeKeyClientHello wire format.
- The 26-byte token (constant 0x8e marker) is a custom construction over the
  derived key: a 528-candidate offline cracker (HMAC/SHA/AES-GCM/CBC/CTR over the
  derived key x request slices x creds) found no match, and it matches none of the
  traced hash digests. It is built in aahClientManaged's C++/CLI <Module> code
  between the DeriveKeyMaterial call and the openParameters assembly.

Next: ILSpy cannot decompile the mixed-mode assembly (crashes, exit 70); use dnlib
(IL-level) to dump the <Module> method referencing DeriveKeyMaterial and read the
post-derive token construction. 2 of 3 layers cleared (key exchange + client key);
the 3rd (token) is localized, pending dnlib extraction. Orchestrator stays on v6.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 11:11:21 -04:00
Joseph Doherty 3fd522fa10 docs(grpc-events): Path B — ExchangeKey ECDH clears 2 of 3 layers
Records that the pure-managed P-256 ExchangeKey works (cleared the v8 client-key
check; error advanced to 132/171 AuthenticationFailed). The remaining layer is the
26-byte credential-token KDF, which requires recovering the native key derivation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 10:31:37 -04:00
Joseph Doherty d67f6f5e96 feat(grpc-events): ExchangeKey ECDH (Path B) — clears the v8 client-key check
Implements HistoryService.ExchangeKey as a pure-managed P-256 ECDH key exchange
and wires it ahead of the v8 Event OpenConnection.

- HistorianNativeHandshake.BuildExchangeKeyClientHello / DeriveExchangeKeySecret:
  .NET ECDiffieHellman (nistP256); wire format "ECK1" + u32(32) + X(32) + Y(32),
  decoded from the live capture. No native AVEVA dependency.
- HistorianGrpcHandshake.OpenSession(eventConnection: true): runs ExchangeKey on
  the context-key handle before the v8 OpenConnection.
- Guardrail HistorianGrpcHandshakeRoutingTests scoped to the token-loop closure:
  still pins that the Negotiate token loop routes to ValidateClientCredential (not
  ExchangeKey), while allowing the legitimate ExchangeKey call in OpenSession.

Live result: ExchangeKey succeeds (server accepts our public key) and the v8
OpenConnection error advances from 132/34 "Failed to get client key" to 132/171
AuthenticationFailed — the ECDH cleared the client-key layer. The remaining
blocker is the 26-byte v8 credential token, which must be derived from the ECDH
shared secret (token KDF, not yet recovered). Orchestrator stays on v6 (set
eventConnection: true to re-arm once the KDF lands). 323/323 offline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 10:31:37 -04:00
Joseph Doherty 7284fdc976 docs(grpc-events): Path A disproven — v8 OpenConnection coupled to ExchangeKey
Records the full v8 openParameters byte map, the ECDH ExchangeKey finding, and
the Path A live result: the v8 OpenConnection on a ValidateClientCredential
session is rejected with native 132/34 "EstablishConnection Failed to get client
key". The v8 path requires the client key established by HistoryService.ExchangeKey
(ECDH), so the next route is Path B — implement ExchangeKey ("ECK1" + 64-byte
P-256 point) via .NET ECDiffieHellman, then reissue the v8 open.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 09:46:07 -04:00
Joseph Doherty 0b1e9d0a7f feat(grpc-events): v8 OpenConnection serializer + native error decode (Path A disproven)
Builds the native 2023 R2 version-8 OpenConnection format, which (unlike v6)
carries a ConnectionType byte (Event vs Process) — required because the 2023 R2
server returns event rows only on an Event connection.

- HistorianOpen2Protocol.SerializeNativeOpenConnectionVersion8: reproduces the
  302-byte v8 layout decoded from a live capture (version 8, markers, client-key
  GUID, username HString, length-prefixed credential token, ClientType /
  ConnectionType / flag / constant word / compact metadata / two empty strings;
  the tail reuses WriteClientCommonInfo). Golden-tested
  (Version8EventSerializerReproducesCapturedNativeStructure).
- HistorianNativeHandshake.BuildEventOpenConnectionVersion8Request: ConnectionType=
  Event, zeroed credential token (mirroring how v6 zeros its credential block and
  relies on the separate ValidateClientCredential handshake).
- HistorianGrpcHandshake.OpenSession: optional eventConnection switch; the
  OpenConnection failure path now decodes the native error (type/code/ASCII).

Path A (reuse ValidateClientCredential + zeroed token) was live-tested and
DISPROVEN: the server parses the v8 buffer but rejects it at the auth check with
native 132/34 "EstablishConnection Failed to get client key" — the v8 path looks
up the client key in the registry HistoryService.ExchangeKey (ECDH) populates,
not the one ValidateClientCredential does. The event orchestrator is therefore
reverted to the v6 session (gated test still pins the no-row throw). The v8
serializer/builder are retained for Path B (implement ExchangeKey). 323/323
offline tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 09:45:52 -04:00
Joseph Doherty ea85ea248d Merge handoff-event-gate-refresh: event-row item reflects captured + v8 gate
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 09:15:03 -04:00
Joseph Doherty 876cbc5d94 docs(handoff): refresh event-row item — captured + v8 connection-type gate
Item 1 is no longer capture-gated (the capture is done, merged 8ad160b). Update it
to: the capture-event run read 50 events from the stock client; the v6 request is
shipped; the remaining gate is the native v8 Event-type OpenConnection (a
ConnectionType field the SDK's v6 Open2 format lacks), a scoped RE+impl follow-on.
Points at docs/reverse-engineering/grpc-event-query-capture.md. Also corrected the
"to move any item" summary so item 1 no longer reads as needing a fresh capture.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 09:06:05 -04:00
Joseph Doherty 8ad160b843 Merge grpc-event-v6-capture: v6 StartEventQuery request + capture-event tooling; v8 connection-type gate diagnosed
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 09:01:37 -04:00
Joseph Doherty c6752804ee docs(grpc-events): event-query capture finding + v8 connection-type gate
Records the 2026-06-22 capture of the stock 2023 R2 gRPC event read and the
diagnosis of why row retrieval is gated:

1. The working StartEventQuery request is version 6 (vs the SDK's v5) — shipped in
   the companion code commit.
2. Rows additionally require an EVENT-type connection. Decoding the captured
   OpenConnection.openParameters (native format v8) shows a ConnectionType byte
   (Event=01 / Process=02) right after ClientType — a field the SDK's v6 Open2
   format does not have (it writes ClientType then ConnectionMode back-to-back).
   So the v6 buffer the SDK sends (accepted for reads) cannot mark the connection
   as Event, and the 2023 R2 server returns event rows only on an Event
   connection. The native client also used the ExchangeKey cert auth path.

Conclusion: making event rows flow over gRPC requires the SDK to emit the native
v8 OpenConnection format with ConnectionType=Event (a larger RE+implementation
effort), scoped as a follow-on. v6 is retained as the captured-correct request.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 10:41:29 -04:00
Joseph Doherty dbb5c99c53 feat(grpc-events): v6 StartEventQuery request + capture-event harness scenario
Captured the stock 2023 R2 client doing a gRPC event read (50 rows flowed) to
resolve the open "gRPC event ROW retrieval returns zero rows" item. Two captured
differences from our SDK's path; this lands the first (necessary) one plus the
capture tooling.

- HistorianEventQueryProtocol.CreateStartEventQueryAttempts: add a `version`
  parameter (default 5 = the 2020 WCF format, unchanged). The gRPC event
  orchestrator now opts into version 6 — the leading `06` plus a 5-byte trailing
  zero pad — which is the envelope the stock 2023 R2 client sends. The two
  buffers are otherwise byte-identical (filter block, UTC string, metadata
  namespace). Golden test Version6EmptyFilterMatchesCapturedGrpcEnvelope pins it.
- Grpc2023CaptureHarness: new `capture-event` scenario drives HistorianAccess
  over an Event-type gRPC connection (CreateEventQuery -> EventQueryArgs ->
  StartQuery -> MoveNext) so the wide-net instrument-grpc-nonstream rewrite dumps
  StartEventQuery.requestBuffer + the row result. Hostname defaults sanitized to
  HISTORIAN_GRPC_HOST / "localhost" (removed hardcoded server name).

NECESSARY BUT NOT SUFFICIENT: live validation shows v6 alone does not make rows
flow — the read also requires an Event-type connection, which our SDK's v6 Open2
format cannot express (see the companion docs commit). The gated
ReadEventsAsync_OverGrpc_* test correctly still pins the no-row throw. 322/322
offline tests pass; WCF event path unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 10:41:15 -04:00
Joseph Doherty d9051ba890 Merge fix-event-gate-characterization: gRPC event-row item is capture-gated (SQL ground truth)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 08:12:31 -04:00
Joseph Doherty 73f66cbf27 docs(handoff): re-characterize gRPC event-row gate — capture-gated, not server-gated
Live SQL ground truth (user-authorized one-time read via SOCKS->SQL relay)
disproves the gate on the open gRPC event-row item. The live 2023 R2 server
IS event-bearing — Runtime.dbo.Events holds 19,356 rows in the last 30 days
(90,944 in 365) — yet the empty-filter gRPC event query still returns zero rows
and long-polls to the deadline over that same window.

So GetNextEventQueryResultBuffer returning nothing is NOT "no events on the
server"; the empty-filter request shape (filter / namespace / event-tag
registration) doesn't match existing rows. The remaining work is a fresh native
gRPC event-query capture of the stock client, not access to a different server.

- handoff.md: rewrite open-item #1 with the SQL numbers + capture-gated framing;
  update the "to move any item" summary to match.
- HistorianGrpcIntegrationTests: correct the event-read test comment (drop the
  false "idle dev box holds no events" rationale; document 19k-events-yet-zero-rows).

No behavior change (test edit is comment-only). Sanitization scan clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 08:11:27 -04:00
Joseph Doherty 941e11d292 Merge docs-handoff-refresh: handoff doc reflects exhausted roadmap
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 07:54:29 -04:00
Joseph Doherty 8b966f3d80 docs(handoff): refresh to current state — roadmap exhausted
The handoff doc was anchored at 2026-05-04 (read-path blocker era). Add a
"Current Status (2026-06-22)" section at the top summarizing the full shipped
read/write/config/client-side surface across WCF + gRPC, the 8 remaining gated
items (event-row retrieval, active-SF magnitude, SendEvent capture, SQL wall,
R4.2 edits, ReadBlocks, the disproven DelTep gRPC probe, deferred-by-design),
and what would unblock each. Reframe the old "Active Blocker" as a historical
read-path record; fix the stale 55/55 test count to 321/321 offline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 07:54:29 -04:00
Joseph Doherty c88260c973 Merge grpc-deltep-probe: DelTep multiplexed-channel probe (disproven)
gRPC's single shared channel does NOT lift the WCF per-connection working-set
wall for DeleteTagExtendedProperties — probed live 2026-06-22, primes succeed
but the delete is still rejected (native code=1). Stays server-blocked on both
transports, unshipped; pinned by a gated negative test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 07:06:39 -04:00
Joseph Doherty b3417c2f6a docs(grpc): record DelTep multiplexed-channel probe as disproven
README transport matrix + grpc-tooling-completion.md §Out-of-scope: the gRPC
multiplexed-channel hypothesis for DeleteTagExtendedProperties was probed live
2026-06-22 and disproven — primes succeed on the shared channel but DelTep is
still rejected (native code=1), property survives. Stays server-blocked on both
transports, not shipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 06:55:05 -04:00
Joseph Doherty 2bd86e4e83 probe(grpc): DeleteTagExtendedProperties multiplexed-channel — disproven
Adds an internal RE probe (HistorianGrpcTagWriteOrchestrator.
ProbeDeleteTagExtendedPropertiesAsync) testing whether gRPC's single shared
channel lifts the WCF per-connection working-set wall that blocks DelTep.

Live result (2023 R2, History iface 12): both GetTgByNm + GetTepByNm primes
succeed on the one shared channel, yet DelTep is still rejected (native code=1)
and the property survives. So the working set is populated by the native
client's in-process registration state, not the wire session — neither WCF's
per-service channels nor gRPC's shared channel reproduce it. DelTep stays
server-blocked on BOTH transports and remains unshipped.

Gated negative test DeleteTagExtendedProperties_OverGrpc_ProbeMultiplexedChannel
pins this (primes succeed, delete rejected, prop survives) and flips if a future
server/registration lifts the wall. Comment in HistorianClient records the
probe. 321 offline tests pass; live test passes bounded at ~11s.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 06:55:05 -04:00
55 changed files with 4595 additions and 232 deletions
+3 -2
View File
@@ -18,9 +18,10 @@ Reads (the original required surface, all working live as of 2026-05-04):
Writes (added 2026-05-04 by explicit user request — do not extend further without one):
- `EnsureTagAsync` for analog types: Float, Double, Int2, Int4, UInt4 (live-verified end-to-end). Other types (SingleByteString/DoubleByteString/Int1/Int8/UInt8) fail at native AddTag — likely require a different path and are intentionally not supported. `MinEU`/`MaxEU`/`MinRaw`/`MaxRaw` all round-trip into the DB. By default `ApplyScaling=false` and the server mirrors MinRaw→MinEU and sets `AnalogTag.Scaling=0`; set `ApplyScaling=true` on the definition to persist distinct raw bounds with `AnalogTag.Scaling=1`. The wire encoding is the trailer's second byte (`FE 00` vs `FE 01`).
- `EnsureTagAsync` for analog types: Float, Double, Int2, Int4, UInt4, **Int8, UInt8** (all live-verified end-to-end; `Int8`/`UInt8` added 2026-06-25 — same analog `CTagMetadata` layout, type codes `0x19`/`0x39`). **`UInt1` is NOT supported**: the server accepts `EnsureTags(UInt1)` but stores a *degenerate* analog tag (`GetTagInfosFromName` returns a 31-byte stub — descriptor type byte `0x00`, no GUID), so the write fails on the `GetTagInfo` path; re-gated fail-closed. SingleByteString/DoubleByteString and special/event types require a different (non-analog) path and are intentionally not supported. `MinEU`/`MaxEU`/`MinRaw`/`MaxRaw` all round-trip into the DB. By default `ApplyScaling=false` and the server mirrors MinRaw→MinEU and sets `AnalogTag.Scaling=0`; set `ApplyScaling=true` on the definition to persist distinct raw bounds with `AnalogTag.Scaling=1`. The wire encoding is the trailer's second byte (`FE 00` vs `FE 01`).
- `DeleteTagAsync`
- `AddHistoricalValuesAsync` (added 2026-06-21 by explicit user request — M3 historical/backfill writes). **gRPC-only** (`HistorianTransport.RemoteGrpc`); non-gRPC transports throw `ProtocolEvidenceMissingException`. Reverse-engineered by capturing the native 2023 R2 client: the historical write rides `HistoryService.AddStreamValues` with an "ON" storage-sample buffer (`HistorianHistoricalWriteProtocol`, golden-tested), NOT the TransactionService `AddNonStreamValues` path the static decompile suggested. Orchestrator (`HistorianGrpcHistoricalWriteOrchestrator`): write-enabled session → `GetTagInfosFromName` (resolves the per-tag GUID = the tag-info `TypeId`, and maps the data type via `MapDataType`) → `AddStreamValues`. Tag must pre-exist (`EnsureTagAsync`). Supports all five analog types `EnsureTagAsync` does — **Float, Double, Int2, Int4, UInt4** (all captured live + golden-tested + write/read-back validated). The 4-byte value descriptor is constant (`C0 10 01 00`); the value is `u32(0) + native-width value` (float32 / double64 / int16 / int32 / uint32) selected by the tag's declared type. Other tag types throw `ProtocolEvidenceMissingException`. Live-validated end-to-end against the 2023 R2 server. The D2/`AddS2` cache gate (err 129) does NOT block the primed 2023 R2 client. See `docs/plans/revision-write-path.md` §"R3.1 CAPTURED".
- `AddHistoricalValuesAsync` (added 2026-06-21 by explicit user request — M3 historical/backfill writes). **gRPC-only** (`HistorianTransport.RemoteGrpc`); non-gRPC transports throw `ProtocolEvidenceMissingException`. Reverse-engineered by capturing the native 2023 R2 client: the historical write rides `HistoryService.AddStreamValues` with an "ON" storage-sample buffer (`HistorianHistoricalWriteProtocol`, golden-tested), NOT the TransactionService `AddNonStreamValues` path the static decompile suggested. Orchestrator (`HistorianGrpcHistoricalWriteOrchestrator`): write-enabled session → `GetTagInfosFromName` (resolves the per-tag GUID = the tag-info `TypeId`, and maps the data type via `MapDataType`) → `AddStreamValues`. Tag must pre-exist (`EnsureTagAsync`). Supports the analog types `EnsureTagAsync` does — **Float, Double, Int2, Int4, UInt4, Int8, UInt8** (all captured live + golden-tested + write/read-back validated; `Int8`/`UInt8` added 2026-06-25, value = native-width LE int64/uint64). The 4-byte value descriptor is constant (`C0 10 01 00`); the value is `u32(0) + native-width value` (float32 / double64 / int16 / int32 / uint32 / int64 / uint64) selected by the tag's declared type. Other tag types throw `ProtocolEvidenceMissingException` (incl. `UInt1` — server-degenerate, see `EnsureTagAsync` above). Live-validated end-to-end against the 2023 R2 server. The D2/`AddS2` cache gate (err 129) does NOT block the primed 2023 R2 client. See `docs/plans/revision-write-path.md` §"R3.1 CAPTURED".
- `SendEventAsync` (M2 event-send; added by explicit user request). Appends a single `HistorianEvent` to the built-in `CM_EVENT` tag, readable back via `ReadEventsAsync` / `v_AlarmEventHistory2`. Works on **both transports**, routed by `HistorianClientOptions.Transport`: WCF runs Open2 event-mode (`0x501`) → CM_EVENT registration (RTag2 + EnsT2) → `AddS2` (`AddStreamValues2`); gRPC (`RemoteGrpc`, `HistorianGrpcEventWriteOrchestrator`, added 2026-06-23) runs the v8 Event `OpenConnection` (ExchangeKey ECDH) → CM_EVENT registration → `HistoryService.AddStreamValues`. **Both carry the same `"OS"` (0x534F) event VTQ buffer** (`HistorianEventWriteProtocol`, the managed `PackToVtq` equivalent) — there is NO distinct event-send RPC and it is NOT the historical write's `"ON"` buffer (captured live from the native 2023 R2 client; the write-enabled Event open is byte-identical to the read-only one). The gRPC path is **live-validated end-to-end** (send → `BSuccess` → event reads back from the server). Only **original events** (`RevisionVersion = 0`) with **string-valued properties** have a captured encoding; revision/update/delete events and non-string property values throw `ProtocolEvidenceMissingException`. Registration buffers are golden-tested against the live capture (`GrpcEventSendProtocolTests`); gated live test `SendEventAsync_OverGrpc_AcceptsEvent` (opt-in `HISTORIAN_GRPC_EVENT_SEND=1`).
`AddS2` (streaming process-sample writes for user tags) remains architecturally blocked — the server cache only ingests from configured IOServers/ApplicationServer pipelines. Do not add streaming write-samples support. (`AddHistoricalValuesAsync` is the distinct *non-streamed original/backfill* path and is supported.)
+1 -1
View File
@@ -91,7 +91,7 @@ request rides the RPC but the server faults on an unmet precondition) ·
| `ReadEventsAsync` | ✅ | ⚠️ | tooled + routed over gRPC: the full CM_EVENT registration replay (`UpdateClientStatus`→`RegisterTags`→`EnsureTags` + discovery probes) runs and `StartEventQuery` succeeds, but `GetNextEventQueryResultBuffer` **long-polls** on no data (it blocks to the deadline rather than returning the synchronous 5-byte code-85 terminal the WCF op gives). The read is **hard-bounded** (≤30s) and throws `ProtocolEvidenceMissingException` on the no-row path rather than assert a false empty. Row-level retrieval is **not yet live-verified** — the dev box holds no events; pending a capture against an event-bearing 2023 R2 server. Use WCF for event reads |
| `SendEventAsync` | ✅ | 🔌 | rides `AddStreamValues` family; no distinct event-send RPC, framing uncaptured over gRPC |
| `EnsureTagAsync` / `DeleteTagAsync` / `RenameTagsAsync` | ✅ | ✅ | live-verified 2026-06-22 over gRPC (`HistoryService.EnsureTags` / `DeleteTags` / `StartJob`, write-enabled 0x401 session, WCF serializers reused) via a self-cleaning sandbox-tag lifecycle. Rename is an async StartJob — transiently rejectable right after create, so callers should retry |
| `AddTagExtendedPropertiesAsync` | ✅ | ✅ | live-verified 2026-06-22 over gRPC (`HistoryService.AddTagExtendedProperties`, write-enabled session); a written prop now round-trips through `GetTagExtendedPropertiesAsync` (the multi-property parser fix above). gRPC also exposes `DeleteTagExtendedProperties` (WCF delete was server-blocked) |
| `AddTagExtendedPropertiesAsync` | ✅ | ✅ | live-verified 2026-06-22 over gRPC (`HistoryService.AddTagExtendedProperties`, write-enabled session); a written prop now round-trips through `GetTagExtendedPropertiesAsync` (the multi-property parser fix above). `DeleteTagExtendedProperties` stays unshipped: probed over gRPC 2026-06-22 (prime `GetTgByNm`+`GetTepByNm` then `DelTep`, all on the one shared channel) — the server still rejects the delete (native code=1) and the property survives, so gRPC's multiplexed channel does **not** lift the WCF per-connection working-set wall |
| `GetConnectionStatusAsync` | ✅ | ✅ | live-verified 2026-06-22 over gRPC — measured from the handshake (`OpenConnection` yields a storage-session GUID ⇒ connected). No dedicated RPC on either transport; store-forward connectivity stays false (D2-gated) |
| `ReadBlocksAsync` | ❌ | ❌ | `StartBlockRetrievalQuery` never captured on either transport — throws `ProtocolEvidenceMissingException` |
+13 -2
View File
@@ -140,8 +140,19 @@ _Original notes (still the reference for the registration replay):_
### Out of scope
- `ReadBlocks` (`StartBlockRetrievalQuery`) — never captured on either transport; leave
throwing `ProtocolEvidenceMissingException`.
- `DeleteTagExtendedProperties` — server-blocked on WCF (per-connection working set);
gRPC's single multiplexed channel *might* fix it — opportunistic probe only.
- `DeleteTagExtendedProperties` — ❌ **PROBED 2026-06-22, multiplexed-channel hypothesis DISPROVEN.**
The WCF block (server resolves the property from a per-connection working set the SDK's separate
per-service channels can't populate) is NOT lifted by gRPC. The probe
(`HistorianGrpcTagWriteOrchestrator.ProbeDeleteTagExtendedPropertiesAsync`) runs the native
`GetTgByNm` → `GetTepByNm` → `DelTep` sequence over ONE write-enabled (0x401) session on gRPC's
single shared channel. Live against the 2023 R2 server (History iface 12): both primes succeed on the
shared channel (`TgPrimeBytes=98`, `TepPrimePages=1`) yet `DelTep` is still rejected with native
**code=1** (the 5-byte error buffer's byte0=132 is the universal `0x84` marker, not a code) and the
property survives. Conclusion: the working set the server consults is populated by something the SDK
can't reproduce even over one connection — most likely the native client's in-process registration
object, not the wire session. Stays server-blocked on BOTH transports; not shipped publicly. Pinned
by the gated negative test `DeleteTagExtendedProperties_OverGrpc_ProbeMultiplexedChannel` (flips if a
future server/registration lifts the wall).
## Live verification setup (every live run)
@@ -0,0 +1,111 @@
# Event-session reuse spike — live results
> **Question:** does the 2023 R2 historian honor REUSING one authenticated **v8 Event**
> session (ECDH `ExchangeKey` → RC4 token → `ConnectionType=Event`, then `RegisterCmEventTag`)
> across multiple `SendEvent` ops, instead of the per-op open+register the SDK does today?
> This is the precondition for amortizing the EVENT path (HistorianGateway `pending.md` A1
> broadening, Stage B0 / B1).
>
> **Verdict: GREEN — a v8 Event session reuses across sends, register-once is sufficient,
> and the amortization is ~10–16×. Event READS stay gated (C2) and are not a reuse signal.**
**Date:** 2026-06-25
**Branch:** `feat/amortization-broadening`
**Server:** live 2023 R2 (`wonder-sql-vd03`), RemoteGrpc transport.
**Sandbox identity:** `HISTORIAN_EVENT_SANDBOX_TAG=HistGW.LiveTest.EventSpike` — the CM_EVENT send
buffer has **no per-tag routing field** (it registers against a fixed system tag), so the sandbox
value is stamped into the event `Type`/`SourceName`/`Namespace` + a `SpikeMarker` property as an
**identity marker**; no real tag is written or overwritten.
**Harness:** `tests/AVEVA.Historian.Client.Tests/EventSessionReuseSpikeTests.cs` driving the B0a
seams `HistorianGrpcEventWriteOrchestrator.OpenAndRegisterEventSession` (open v8 Event session +
`RegisterCmEventTag` ONCE) and `SendEventOnSession` (send only — no open/register).
---
## 1. Send reuse — GREEN
`ReusedEventSession_SendsTwice_SecondSkipsHandshake` **passed** (both runs): one
`OpenAndRegisterEventSession` then **two `SendEventOnSession` on the same v8 Event session** — both
accepted (`AddStreamValues` `BSuccess=true`).
```
open+register (ECDH handshake + RegisterCmEventTag) = 242 ms (run 1: 350 ms)
registration diag: RTag=True; EnsT=True
reused-send[0] = 23 ms, ok=True
reused-send[1] = 22 ms, ok=True
```
The server accepts the same v8 Event client handle across back-to-back sends. The session handle is
an immutable `readonly record struct (uint ClientHandle, Guid StorageSessionId)`; the send is
stateless on the client side (each call reserializes a fresh `"OS"` buffer), so nothing per-op is
baked into the handle.
## 2. Amortization — ~10–16×
The open+register (P-256 ECDH `ExchangeKey` → RC4 credential token → v8 `OpenConnection` →
`RegisterCmEventTag`) costs ~242–350 ms and is paid **once**; a reused send is ~22 ms. So over a
burst of N sends the per-send cost collapses from ~(265 ms open + 22 ms) to ~22 ms — a ~10–16× win
on the send path, same shape as the v6 read/write amortization (`handshake-reuse-spike-results.md`).
## 3. Register-once is sufficient — GREEN
`ReusedEventSession_RegisterOnce_ThenSendMany` **passed**: `RegisterCmEventTag` run **once** (inside
`OpenAndRegisterEventSession`), then **three** sends, all accepted.
```
register-once send[0] = 25 ms, ok=True
register-once send[1] = 22 ms, ok=True
register-once send[2] = 22 ms, ok=True
```
CM_EVENT registration is **session-scoped, not per-send** — the server holds the registration for the
session's lifetime. A reuse pool registers once per warm session, not per op.
## 4. Idle tolerance — survived ≥25 s (best-effort, single sample)
`ReusedEventSession_IdleSweep_BestEffort` (log-only): after a send, a **25 s idle gap**, then another
send — **the second send succeeded** (`session SURVIVED the idle gap`). Notable: the v6 read session
idle-expires at a ≥25 s gap (`handshake-reuse-spike-results.md` §3), but this v8 Event session
survived 25 s. This is a single-sample best-effort observation — a keepalive should still ping under
the ~20 s floor for safety margin until the v8 Event idle boundary is characterized more finely.
## 5. Read-after-send — GATED (C2), not a reuse signal
`ReusedEventSession_ServesReadAfterSend_BestEffort` (log-only, hard-bounded by a 5 s gRPC deadline +
an 8 s cancellation): the read-after-send on the same session **did not return data** — it cancels at
the bound:
```
read-after-send -> swallowed (RpcException Cancelled / OperationCanceled)
=> read gated/unverified over gRPC (expected)
```
This matches the pre-existing C2 gate: event **reads** over gRPC long-poll `GetNext` to a no-data
terminal and are unverified. So the spike did **not** prove a one-session-serves-both-kinds property
for reads — `SendEvent` is the only trustworthy reuse signal. (An unbounded read hung the first run;
the harness now bounds it so the spike is a clean, re-runnable record.)
---
## 6. Implications for Stage B1 (the event-pool build)
GREEN → a **separate event-session pool** (the approved B1 approach) is warranted and high-value:
1. **Amortize `SendEvent` through a bounded event-session pool.** Open+register a v8 Event session
once per warm session; lease it per send op (exclusive, like the v6 pool); reuse across a burst.
~10–16× on the send path.
2. **Keep the event pool SEPARATE from the v6 pool** (B1, as approved) — different auth (ECDH/v8),
heavier re-handshake on drop, and its own idle characteristics.
3. **`ReadEvents` stays PER-CALL / gated (C2).** Reads are unverified over gRPC regardless of reuse,
so the event pool amortizes **sends only**; `ReadEvents` is unaffected by B1 and stays on the
per-call path. (This refines the design's "route SendEvent + ReadEvents through the pool": only
`SendEvent` is routed; `ReadEvents` remains per-call because it is gated, not because of reuse.)
4. **Keepalive:** ping the warm event session under the idle floor. The cheap keepalive op for the
event channel is TBD in B1 (the v6 pool uses `GetSystemParameter`; the event session's equivalent
warm-touch needs picking — likely a no-op send or a lightweight event-channel status op).
5. **Reactive re-auth:** on an expiry-looking failure, evict + full v8 re-handshake (heavier than the
v6 re-auth — one ECDH + register penalty).
**Gate decision: GREEN → HistorianGateway A1 Stage B1 (a bounded `HistorianEventSessionPool` for
`SendEvent`, default-on, parallel to the v6 `HistorianSessionPool`) is warranted and earns its own
re-planned design + plan.**
@@ -0,0 +1,505 @@
# gRPC event-query capture (2026-06-22) — the StartEventQuery request that returns rows
Captured the stock 2023 R2 client performing a **gRPC event read** that returns rows, to resolve
the open item "gRPC event ROW retrieval returns zero rows" (handoff §Current Status item 1). This
closes the capture-gate: the working request shape is now known.
## How it was captured
`tools/AVEVA.Historian.Grpc2023CaptureHarness` gained a `capture-event` scenario. It loads the
self-contained mixed-mode 2023 R2 `aahClientManaged.dll` and drives `HistorianAccess`:
```
OpenConnection(ConnectionMode=Historian /*gRPC*/, ConnectionType=Event, ReadOnly=true)
-> CreateEventQuery() // NON-null only on an Event connection
-> EventQueryArgs { StartDateTime, EndDateTime, EventCount }
-> EventQuery.StartQuery(args) // => GrpcRetrievalClient.StartEventQuery(requestBuffer)
-> loop EventQuery.MoveNext() / QueryResult// => GrpcRetrievalClient.GetNextEventQueryResultBuffer
-> EventQuery.EndQuery() -> CloseConnection
```
The existing wide-net `instrument-grpc-nonstream` IL rewrite (every `Grpc*Client` `byte[]` method)
already covers `GrpcRetrievalClient.StartEventQuery.requestBuffer` (entry) and
`GetNextEventQueryResultBuffer.result` (exit) — no new instrument command was needed. Run read-only
(non-destructive) against the live 2023 R2 server over the loopback tunnel; the rewrite + capture
NDJSON stay under `artifacts/reverse-engineering/grpc-event-capture/` (gitignored — the result
buffer carries event identity data).
Result: **50 events returned over gRPC** (Alarm.Set / Alarm.Clear rows), proving the path works when
driven through an Event connection.
## Two findings
### 1. The event read needs an **Event-type connection** (`ConnectionIndex 1`)
`HistorianAccess.CreateEventQuery()` returns `null` unless `IsEventConnectionRequested()` — i.e. the
connection was opened with `ConnectionType=Event`, which the native client routes to a *separate*
connection (ConnectionIndex 1) from the process/data path. The full captured pre-query sequence on
that connection: `OpenConnection` → `ExchangeKey` → `UpdateClientStatus` → `RegisterTags`(CM_EVENT) →
`EnsureTags`(CM_EVENT) → `GetHistorianInfo` + 7×`GetSystemParameter` (Stat priming) →
`StartEventQuery` → `GetNextEventQueryResultBuffer` (rows) → `EndEventQuery` → `CloseConnection`.
### 2. The working `StartEventQuery` request is **version 6**, not 5
Our SDK's `HistorianEventQueryProtocol.CreateNativeFilterAttempt` builds a **version-5** empty-filter
buffer; the stock 2023 R2 client sends **version 6**. Diffed byte-for-byte (same query window +
eventCount), the two buffers are **identical except**:
- **byte 0: version `06` vs `05`**
- **5 additional trailing zero bytes** (stock = 70 bytes, SDK v5 = 65 bytes)
The server returns rows for v6 and **zero rows for v5** (the v5 request is *accepted* —
`StartEventQuery` succeeds and yields a query handle — but `GetNextEventQueryResultBuffer` then
matches nothing). Everything else is shared: the two query-window FILETIMEs, `UInt32 eventCount`,
the `UInt32 65536` buffer hint, the `"UTC"` `HistorianString`, and the `01 01000001000001 0000`
metadata-namespace block.
Captured v6 request layout (70 bytes; the FILETIMEs below are just the harness query window — no
identity data):
```
[0..1] UInt16 version = 6 // SDK currently sends 5
[2..9] Int64 startUtc (FILETIME)
[10..17] Int64 endUtc (FILETIME)
[18..21] UInt32 eventCount
[22..25] UInt32 0
[26..27] UInt16 0
[28..29] UInt16 1
[30..36] 7 bytes 0 // empty-filter block
[37..40] UInt32 65536 // buffer-size hint
[41..50] HistorianString "UTC" (UInt32 len=3 + UTF-16LE)
[51..60] 01 01 00 00 01 00 00 01 00 00 // metadata-namespace block (marker + 3 empty)
[61..69] 9 bytes 0 // terminal (SDK v5 writes only 4 here)
```
## Fix part 1 — v6 request (DONE, necessary)
`HistorianEventQueryProtocol.CreateStartEventQueryAttempts` gained a `version` parameter (default 5 =
WCF/2020; the gRPC orchestrator passes 6). v6 emits the leading `06` and the 5-byte trailing pad. The
WCF path is unchanged (v5). Golden test `Version6EmptyFilterMatchesCapturedGrpcEnvelope` pins the
envelope; 322/322 offline tests pass.
## Fix part 2 — EVENT connection (the remaining gate, NOT yet implemented)
Live validation 2026-06-22: with the orchestrator now sending v6 against the event-bearing live
server, `GetNextEventQueryResultBuffer` **still long-polls and returns zero rows** (the gated test
still throws). So **v6 is necessary but not sufficient** — the read also requires an **Event-type
connection**, which our SDK does not open.
Isolated by diffing the captured `OpenConnection.openParameters` (302 bytes, native format v8) for a
**Process** connection (`connect` scenario) vs the **Event** connection (`capture-event`): aside from
the per-session auth GUID/credential-hash regions ([22..37], [68..93], which vary between any two
sessions), the connection differs in **two clean structural bytes**:
| offset | Process | Event |
|--------|---------|-------|
| 95 | `02` | `01` |
| 96 | `00` | `01` |
These correspond to `HistorianConnectionType` (Process vs Event; the native event path runs on
`ConnectionIndex 1`). The problem: our SDK opens the session with the **2020 OpenConnection3 v6**
buffer (`HistorianNativeHandshake.BuildOpenConnection3Request`, `connectionMode 0x402`), which the
2023 R2 server accepts for reads but which carries no event-connection-type marker. `connectionMode`
is NOT the discriminator (2020 WCF event reads work with `0x402`); the native client distinguishes
event vs process via this separate `ConnectionType` field in its v8 `openParameters`.
### Diagnosis (2026-06-22): the v6 Open2 format cannot express an event connection
Decoded the native `openParameters` (302 bytes): **byte 0 = `08` (format version 8)**, then a
context GUID, username, a 26-byte session-derived region ([68..93]), machine/client-node/datasource
strings, and at **[94] `ClientType=04`** immediately followed by **[95] `ConnectionType`
(`01`=Event / `02`=Process)** + **[96] a flag (`01`/`00`)**, then the rest.
Our SDK builds the **v6** buffer (`HistorianOpen2Protocol.SerializeNativeOpenConnection3Version6`,
byte 0 = `06`): it writes `ClientType` (1 byte) **immediately followed by `ConnectionMode` (uint)** —
there is **no `ConnectionType` byte at all**. The v8 format *inserts* `ConnectionType` (+flag) between
`ClientType` and the rest. So the v6 buffer the SDK sends (accepted by the 2023 R2 server for *reads*)
structurally cannot mark the connection as Event, and the server returns event rows only for an Event
connection.
Two further obstacles to simply emitting v8:
- the native client authenticated via **`ExchangeKey`** (cert path; 72-byte `btInput`/`btOutput` in
the capture) whereas the SDK's gRPC handshake uses **`ValidateClientCredential`** (Negotiate). The
v8 `openParameters` [68..93] region is session-derived and tied to that auth flow.
- `ConnectionMode` is NOT the lever (2020 WCF event reads work at `0x402`); `ConnectionType` is a
distinct field that only exists from format v8.
Also confirmed a secondary format gap: the native gRPC `EnsureTags` CM_EVENT payload is **86 bytes**
vs the SDK's `SerializeCmEventCTagMetadata` **83 bytes** (a 3-byte 2023 R2 bump, parallel to the
event-query v5→v6). This is likely benign on its own (CM_EVENT pre-exists; 2020 EnsT2 returns
benign-false yet events flow) but should be matched if the event open is ever rebuilt.
**Conclusion — the event-connection gate is NOT a tweak.** Making event rows flow over gRPC requires
the SDK to emit the native **v8 `OpenConnection` format** with `ConnectionType=Event` (a 302-byte
buffer whose layout differs from the v6 buffer and includes a session-derived auth region), and
likely to adopt the `ExchangeKey` cert auth path. That is a substantial RE+implementation effort
comparable to the original Open2 work — scoped as a follow-on, not a quick fix. Until then the gated
`ReadEventsAsync_OverGrpc_*` test correctly still pins the no-row throw, and **v6 (part 1) is retained
as the captured-correct request format** for when the open is rebuilt.
Capture artifacts (gitignored): `artifacts/reverse-engineering/grpc-event-capture/` —
`event-capture.ndjson` (Event), `process-connect-2.ndjson` (Process).
## v8 `openParameters` fully decoded (2026-06-23) + the ECDH ExchangeKey finding
Full byte map of the native Event-connection `openParameters` (302 bytes; identity values
redacted — they are session-specific and sit in the gitignored capture):
```
[0] byte 0x08 format version = 8
[1] byte 0xf0 constant marker
[2..20] 19 × 0x00
[21] byte 0x01 constant marker
[22..37] 16B GUID per-session client key
[38..41] u32 username length (chars)
[42..N] UTF-16 username (HistorianString)
[..+1] u16 credential-token length (= 26 in the capture)
[..] 26B token ECDH-derived credential token <-- see below
[94] byte 0x04 ClientType (= our NativeClientType 4)
[95] byte ConnectionType 01 = Event / 02 = Process <-- THE GATE
[96] byte flag 01 (Event) / 00 (Process)
[97..] control bytes (0x03 ... small region, not fully named)
[~114..117]u32 FormatVersion=3
[..] HistorianString machine/server node name
[..] HistorianString client node name "(<ver>)"
[..] u32 session-variable (process-ish)
[..] u32 / zeros
[..] u32 datasource len
[..] UTF-16 datasource id e.g. "2023.1219.4004.5"
[270..285] 16 × 0xff ShardId (all-FF = unset; our v6 sends Empty)
[286..289] u32 client/hcal version int
[290..297] i64 FILETIME ClientTimestamp
[298..301] u32 0
```
The tail (`FormatVersion` → machine → clientNode → datasource → ShardId → version → timestamp)
is the **same `ClientCommonInfo` our v6 already emits**. The new/different parts are: version byte,
the `[1]`/`[21]` markers, the GUID position, the **26-byte credential token** (vs v6's fixed-size
block), the **`ConnectionType` byte**, and ShardId=FF.
**The auth is ECDH, not Negotiate.** The capture's `ExchangeKey` buffers begin `45 43 4b 31` =
ASCII **`"ECK1"`** + a 64-byte EC public-key point — a Diffie-Hellman key exchange — and the 26-byte
`openParameters` token is derived from it. `HistorianSecurityMode` offers only `Disabled` / `None` /
`TransportCertificate`; the harness used `TransportCertificate`, which is what drives the ECDH
`ExchangeKey`. There is **no TLS+Negotiate mode** on the native client (it couples TLS with the cert
ECDH path), so a Negotiate-auth v8 capture cannot be produced from the native client.
**Key de-risking insight:** our SDK's v6 `OpenConnection` sends a **fully zeroed** 1026-byte
credential block (`credentialBlock: new byte[1026]`) and reads still work — because authentication is
actually carried by the separate `StorageService.ValidateClientCredential` (Negotiate) handshake, not
by the bytes inside `openParameters`. By analogy the v8 `[68..93]` token may likewise be **ignorable**
once `ValidateClientCredential` has run. So the first build hypothesis (cheapest, read-only to test):
> Reuse the SDK's existing `ValidateClientCredential` handshake, then send a **v8 `OpenConnection`
> with `ConnectionType=Event` and a zeroed credential token**, and see whether the 2023 R2 server
> returns event rows.
If that works, the ECDH ExchangeKey RE is unnecessary. If it fails, the fallback is full reproduction
of the ECDH `ExchangeKey` handshake (curve/KDF/cipher) — a much larger crypto-RE effort. Build path:
add `SerializeNativeOpenConnectionVersion8(connectionType)` to `HistorianOpen2Protocol`, wire the gRPC
event handshake to use it (events only; reads stay on v6), live-test (non-destructive). Full hex in
the gitignored capture.
### Path A built + live-tested 2026-06-23 — DISPROVEN (v8 is coupled to ExchangeKey)
Built `HistorianOpen2Protocol.SerializeNativeOpenConnectionVersion8` (golden-tested,
`Version8EventSerializerReproducesCapturedNativeStructure` — reproduces the captured 302-byte
structure exactly) + `HistorianNativeHandshake.BuildEventOpenConnectionVersion8Request` (zeroed
credential token) + an `eventConnection` switch on `HistorianGrpcHandshake.OpenSession`, and live-ran
the event read against the server. Result: the v8 `OpenConnection` was **parsed by the server** (got
past the byte format) but **rejected at the auth check** with native error
```
type=132 code=34 "aahHcapLib::HistoryService::EstablishConnection — Failed to get client key"
```
i.e. `EstablishConnection` could not find a server-side **client key** for our session. In the v6
path that key is established by `StorageService.ValidateClientCredential` (which is why v6 reads
work); the v8 path looks it up in the registry that **`HistoryService.ExchangeKey` (ECDH)** populates,
and there is **no `ValidateClientCredential` on `HistoryService`** in the gRPC contract. So the server
branches on the OpenConnection version: v6 accepts the Negotiate-established key, **v8 requires the
ExchangeKey-established key**. The zeroed-token hypothesis is therefore disproven — not because of the
token bytes, but because the whole v8 path is gated on `ExchangeKey` having run first.
**Status:** the v8 serializer/builder are correct and retained (golden-tested), plus the
`OpenConnection` failure now decodes the native error (type/code/ASCII). The event orchestrator is
reverted to the v6 session (gated test still pins the no-row throw). The remaining route is **Path B:
implement `HistoryService.ExchangeKey`** — `"ECK1"` + a 64-byte EC public-key point (P-256 X‖Y, by the
size) — using .NET `ECDiffieHellman`, establish the client key, then reissue the v8 `OpenConnection`.
Open question for Path B: whether merely *completing* the ECDH key agreement registers the client key
(so the zeroed openParameters token still rides through), or whether the token must also be derived
from the shared secret (full KDF/cipher RE).
### Path B started 2026-06-23 — ExchangeKey ECDH works; cleared 2 of 3 layers
Implemented `HistoryService.ExchangeKey` as a **pure-managed P-256 ECDH** key exchange
(`HistorianNativeHandshake.BuildExchangeKeyClientHello` / `DeriveExchangeKeySecret`, .NET
`ECDiffieHellman` over `nistP256`; wire format `"ECK1" + u32(32) + X(32) + Y(32)`) and wired it into
`HistorianGrpcHandshake.OpenSession(eventConnection: true)` ahead of the v8 `OpenConnection`,
on the same context-key handle. Live result against the server: the **`ExchangeKey` RPC succeeds**
(the server accepted our public key), and the v8 `OpenConnection` error **moved one layer deeper**:
```
Path A (no ExchangeKey): 132/34 "Failed to get client key"
Path B (ExchangeKey ECDH): 132/171 AuthenticationFailed "EstablishConnection — Authentication failed"
```
So the ECDH cleared the client-key check; the remaining blocker is **authentication**: the 26-byte
v8 credential token must be a *valid* value derived from the ECDH shared secret (not zeros).
### Token crypto traced 2026-06-23 (Frida → Windows CNG) — KDF found, token construction still open
Hooked Windows CNG (`bcrypt.dll`/`ncrypt.dll`) while the native harness ran a real ExchangeKey
(`scripts/frida/aahclientmanaged-cng-exchangekey.js` + `artifacts/.../cng-trace.py`). Findings:
- **The ECDH + KDF are standard CNG, driven by managed `System.Security.Cryptography.ECDiffieHellmanCng`**
(backtrace top frame = `System.Core.ni.dll`; the caller is aahClientManaged's C++/CLI `<Module>`):
`NCryptSecretAgreement` (P-256) → `NCryptDeriveKey(KDF=HASH, HASH_ALGORITHM=SHA256, 32 bytes)`. So the
derived key = **SHA256(ECDH shared secret)** — exactly `ECDiffieHellmanCng{ KeyDerivationFunction=Hash,
HashAlgorithm=SHA256 }.DeriveKeyMaterial(...)`. Our managed `DeriveExchangeKeySecret` should switch to
this (SHA256 of the raw agreement) to match.
- **`"ECK1"` is NOT AVEVA-custom** — it is the standard Windows CNG `BCRYPT_ECCPUBLIC_BLOB` magic for
P-256 (`NCryptExportKey`/`ImportKey` emit exactly `ECK1 + len(32) + X(32) + Y(32)`), confirming our
`BuildExchangeKeyClientHello` wire format is correct.
- **The 26-byte token is a custom construction that is not yet reproduced.** Correlated one run's
derived key (`SHA256(secret)`) with that run's token (from the IL openParameters capture): a
528-candidate offline cracker (HMAC/SHA/AES-GCM/CBC/CTR over the derived key × request slices ×
creds) found **no match**, and the token matches **none** of the traced hash digests. The token
starts with a constant `0x8e` marker in both captured runs (so it is structured, not raw cipher
output). It is built in managed code between the `DeriveKeyMaterial` call and the openParameters
assembly.
**dnlib IL extraction 2026-06-23 — the token scheme is fully reverse-engineered.** ILSpy can't
decompile the mixed-mode assembly (crashes), but loading `dnlib` in PowerShell and scanning the IL
recovered the whole construction:
- **`<Module>::CHistoryConnectionGrpc.GetClientKey`** is the ECDH driver: `new ECDiffieHellmanCng()`
→ `KeyDerivationFunction = Hash`, `HashAlgorithm = SHA256`, `KeySize = 256` →
`GrpcHistoryClient.ExchangeKey(strHandle, ourPubKey.ToByteArray(), out serverPub, out err)` →
`CngKey.Import(serverPub, CngKeyBlobFormat.EccPublicBlob)` → **`DeriveKeyMaterial`** = the 32-byte
client key = **`SHA256(ECDH shared secret)`**. (So our managed side should derive the key the same
way — `ECDiffieHellman` raw agreement then SHA256, or equivalently `DeriveKeyFromHash(..., SHA256)`.)
- **The 26-byte token is built by `aahClientCommon.CClientBase.ConfigureOpenConnection`** (the lone
caller of `GetClientKey`) using the **`HistorianCrypto.NRC4_V2.aahCryptV2`** scheme — a custom
**MD5-keyed RC4 stream cipher with a version prefix**:
- `aahCryptV2.body`/`HashData` = **MD5** (verified: the IL loads MD5 round constants `0xd76aa478`…
and rotates 7/12/17/22).
- `aahCryptV2.prepare_key` = standard **RC4 KSA** seeding the 256-byte S-box from a **16-byte (MD5)**
key (`std.array<unsigned char,16>`).
- `aahCryptV2.enc_buffer` = `MD5(...)` → key, then **`rc4encrypt`** the body; `enc` prepends a
scheme **prefix** (`NRC4_V2.PrefixV2` / `InnerPrefixV2`) — the constant `0x8e` token marker.
- `from_GUID` keys the cipher from a GUID string.
So the token = `prefix + RC4(plaintext, key = MD5(keyMaterial))`, where the key material ties back to
the `SHA256(ECDH secret)` client key. **This is 100% reproducible in pure managed code** (RC4 + MD5
are ~40 lines; nothing AVEVA ships).
**Remaining to finish (next cycle):** read `ConfigureOpenConnection`'s exact wiring (which value is
MD5'd for the RC4 key, what plaintext is encrypted, the exact prefix bytes — a little more dnlib IL),
implement `aahCryptV2` (RC4+MD5+prefix) managed-side, set the v8 token = that, and live-test
(non-destructive). The offline correlation data (one run's derived key + token + openParameters) is
captured under `artifacts/.../` to validate the managed reproduction before going live.
### Token implemented + auth WORKS live (2026-06-23); row retrieval still 0 — proven NOT a payload issue
`token = RC4(password-UTF16LE, key = MD5(SHA256(ECDH secret)))` was implemented in pure managed C#
(`HistorianNativeHandshake.BuildExchangeKeyCredentialToken` + `Rc4`; client key via
`DeriveKeyFromHash(SHA256)`), golden-tested (RC4 standard vector + token construction), and
**live-verified**: the v8 `OpenConnection` now **authenticates** against the 2023 R2 server (past the
`132/171 AuthenticationFailed` wall). Auth is solved.
The event **query** still returns `version-11 rowCount-0` while the native returns 50 for an
**identical** request. Exhaustively ruled out as the cause (all confirmed live, opt-in
`EventReadDiagnostic` test + the IL rewrite extended to log string/uint handle fields):
- `StartEventQuery` request: **byte-identical** to the native (v6 layout)
- v8 `OpenConnection` `openParameters`: **byte-identical** to the native (302 bytes) once ClientNodeName
is matched — every control byte, ConnectionType, token framing, ShardId, etc.
- Handle usage: identical — `ExchangeKey`→contextKey, registration→storage-session GUID (`strHandle`),
query→client uint (`uiHandle`); our parsed handles are valid (registration `RTag/EnsT=True`, valid
`queryHandle`)
- `queryRequestType = 3`, registration sequence/order, gzip metadata header — all match
- window (events exist; native returns 50 *now*), eventCount — not it
So **every observable client-side byte matches the native**, yet the server scopes 0 events to our
connection. The event RPCs succeed over our transport and return a valid *empty* result (not a
transport error), so it is **not a payload or transport-incompatibility issue** — it is a
connection/server-level difference (e.g. session affinity tied to the native `Grpc.Core` HTTP/2
connection or a connection-identity the server uses to scope events) that is **invisible to, and
unfixable by, client payload matching.** Closing it needs server-side insight or a different angle
(e.g. compare the full HTTP/2 connection setup / TLS identity), not more wire-payload RE.
**Shipped this effort:** the complete ExchangeKey crypto (ECDH + SHA256 + MD5-keyed RC4 token) — the
hard wall — pure managed, golden-tested, auth live-verified. Orchestrator stays on the no-row throw;
gated test unchanged.
### NEXT SESSION — the server-side / connection angle (row retrieval pickup)
Client payloads are exhausted (byte-identical to the native, proven above). The next investigation is
**connection-level**, not wire-payload. Pursue in roughly this order; each is concrete and testable.
**Already proven — do NOT redo:** auth works (ExchangeKey ECDH + RC4 token, live-verified); v8
`openParameters`, all handles (str/uint), `StartEventQuery` request, registration (`RTag/EnsT=True` +
order), `queryRequestType=3`, gzip header — all byte-match the native. Events exist (native returns 50
*now*). The event RPCs succeed over our transport and return a valid version-11 **rowCount-0** (not a
transport error). So the server scopes 0 events to *our* connection specifically.
**Tooling already in place:** opt-in diagnostic test `EventReadDiagnostic_OverGrpc_PrintsJourney`
(env `HISTORIAN_GRPC_EVENT_DIAG=1`, prints registration outcomes, handles, result hex, v8 buffer);
the `capture-event` harness scenario (native, returns rows); `instrument-grpc-nonstream` now logs
string/uint handle fields too; the CNG Frida hook. Live recipe: set `HISTORIAN_GRPC_HOST`/`_PORT
32565`/`_TLS true`/`_DNSID` to the 2023 R2 server + domain creds (strip quotes); reach the box per the
live-server access reference.
1. ~~**Transport: native `Grpc.Core` HTTP/2 vs our `Grpc.Net.Client` + `GrpcWebHandler` (gRPC-Web).**~~
**DISPROVEN 2026-06-23.** Built `HistorianGrpcChannelFactory.CreateHttp2` (plain HTTP/2 over a
`SocketsHttpHandler`, no `GrpcWebHandler` wrap, ALPN `h2` to the TLS server) and wired it into the
event orchestrator behind `HISTORIAN_GRPC_EVENT_HTTP2=1` (event path only; reads stay gRPC-Web). Live
side-by-side against the event-bearing server, **everything else held constant**:
| channel | auth | registration | queryHandle | result buffer |
|---------|------|--------------|-------------|---------------|
| `http2` (native HTTP/2) | ✓ | `RTag=True EnsT=True` | 1057 | `0B00000000001E000000` |
| `grpc-web` (default) | ✓ | `RTag=True EnsT=True` | 1058 | `0B00000000001E000000` |
The complete v8 chain — ExchangeKey ECDH auth, CM_EVENT `RegisterTags`/`EnsureTags`, `StartEventQuery`
(valid handle) — runs end-to-end over **plain native HTTP/2**, and the server returns the
**byte-identical** version-11 (`0x0B`) rowCount-0 terminal on both transports. So gRPC-Web vs native
HTTP/2 is **not** the discriminator — the zero-row scoping is identical regardless of transport. The
`CreateHttp2` factory + the `HISTORIAN_GRPC_EVENT_HTTP2` switch + the `EventChannelMode` diagnostic are
retained for future connection-level probing. This eliminates the leading hypothesis and tightens the
conclusion: the server scopes 0 events to our connection at a layer **above** the gRPC transport.
2. ~~**TLS client identity / certificate.**~~ **DISPROVEN 2026-06-23 (decompile + capture).** The stock
client's `GrpcClientBase.InitializeBase` creates a bare `HttpClientHandler` and sets only
`ServerCertificateCustomValidationCallback` — it **never adds a client certificate**. The TLS-tee
capture (below) confirms `clientCert=none` on every native connection. So the native presents no client
cert; this is not the gate.
3. ~~**HTTP/2-level / connection-frame capture.**~~ **DONE 2026-06-23 — topology difference found, tested,
NULL.** Built a TLS-terminating tee proxy (`artifacts/.../httpcap/`, gitignored: self-signed server
cert, forwards through the loopback tunnel, logs decrypted HTTP/1.1 + gRPC-Web both ways) and ran a
**native `capture-event` (returns 50 rows) and our SDK diagnostic (0 rows) through the same
proxy/upstream**. Note: the stock client is gRPC-Web/HTTP-1.1 (not HTTP/2 — `alpn` empty), so the
capture is HTTP/1.1 framing. Findings:
- **Connection topology differs.** The native opens **5 TLS connections, one per service** —
`HistoryService` (ExchangeKey/OpenConnection/Register/EnsureTags), `StatusService` (×2), and
**`RetrievalService` (the event query: GetRetrievalInterfaceVersion → StartEventQuery → GetNext →
EndEventQuery) on its own dedicated connection**. Our SDK collapses **every service onto one
connection**. (Matches the decompile: stock has a separate `GrpcClientBase` per service.)
- **Framing differs** (benign): native uses `content-length` + `Expect: 100-continue`; SDK uses
`transfer-encoding: chunked`. The server accepts both (our `StartEventQuery` returns a valid handle),
so framing is not the gate. No extra/hidden header on either side; `clientCert=none` throughout.
- **TESTED the topology hypothesis (`HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL=1`):** ran
`StartEventQuery`/`GetNext`/`EndEventQuery` on a **dedicated RetrievalService connection** (no
re-handshake, reusing the session handle — exactly mirroring native conn4), registration staying on
the main connection. **Result: still `0B00000000001E000000` (0 rows), `QH=1063`.** Splitting the
event query onto its own connection — the one concrete structural difference the capture revealed —
**does not make rows flow.** So the server correlates by session handle, not by connection, and the
topology is **not** the row-scoping gate. The `CreateHttp2`/`SPLIT_CHANNEL` switches + the
`httpcap` proxy are retained as diagnostics.
4. ~~**Server-side ground truth.**~~ **ANSWERED 2026-06-23 (DISPROVES the data-scoping premise).** Via
the SOCKS→SQL relay (read-only; `artifacts/.../sqlschema/`, gitignored), dumped the full event schema
on the live `Runtime` DB. Findings:
- **No per-connection / per-client / per-session column exists anywhere in the event store.** The only
"scoping-like" columns on `Events`/`EventHistory`/snapshots are event *content* — `Source_*` (event
origin area/object/PV), `User_*` (who acknowledged), `Provider_NodeName` (alarm provider node),
`SourceServer`/`SourceTag` (cross-server replication). None is "which client connection requested
this."
- **The rich `Events` view is not a relational table — it is served live by the Historian engine via
the `INSQL` OLE DB provider** (`sys.servers` shows linked servers `INSQL` + `INSQLD`;
`OBJECT_DEFINITION('dbo.Events')` is `NULL` = encrypted remote view). The Historian's own
`EventHistory` base table holds just 168 rows / 1 tag (the internal event-tag detector log); the
alarm/event journal the gRPC query reads lives in the engine, surfaced through INSQL.
- **Decisive: same engine, same `-90d..now` window, two paths diverge.** The `Events` view (via INSQL)
returns **71,332 events** for that window — most recent `Alarm.Set` firing seconds before the probe
(live, every few seconds) — while gRPC `StartEventQuery` for **our** connection returns **0**. The
data is global, abundant, recent, and identical-window-addressable; the engine simply does not hand
it to our gRPC connection.
→ There is **nothing in the data to scope by**, so the zero-row gate is **not** data scoping. It is the
gRPC RetrievalService's **per-connection in-process execution state** — the same class of wall as
`DeleteTagExtendedProperties` (server-side native in-process working-set, not reconstructable from
byte-identical wire requests). Reproduce: `artifacts/.../sqlschema/` (Program.cs = SOCKS5 relay +
`Microsoft.Data.SqlClient`; authenticate with the server's SQL login, not the domain Historian acct —
creds in the gitignored creds file).
### Stock managed client decompiled (2026-06-23) — confirms no hidden client-side difference
Closing the gap that prior cycles left: the zero-rows conclusion had leaned on **wire capture**
(`instrument-grpc-nonstream`, which only hooks `byte[]` params on `Grpc*Client` methods) — blind to gRPC
metadata/headers, interceptors, channel options, and any non-`byte[]` call. Read the **stock managed
client source directly** (`histsdk-2023r2-analysis/decompiled/Archestra.Historian.GrpcClient` +
`HistorianAccess`; the pure-managed assemblies decompile cleanly even though the mixed-mode
`aahClientManaged.dll` crashes ILSpy). Findings:
- **`GrpcClientBase.InitializeBase` builds the same channel we do.** `GrpcWebHandler((GrpcWebMode)0,
HttpClientHandler)` with `HttpVersion = 1.1` — i.e. **the stock client speaks gRPC-Web over HTTP/1.1,
the same transport as our SDK.** This *corrects the premise of hypothesis #1*: there was never a native
`Grpc.Core` HTTP/2 path to differ from — the stock client that returns 50 rows is itself gRPC-Web. The
HTTP/2 disproof's *conclusion* stands (and is reinforced: identical transport on both sides).
- **`m_metadata` passed to every RPC (incl. `StartEventQuery`/`GetNextEventQueryResultBuffer`) is only
`grpc-internal-encoding-request: gzip`** — exactly our header set. No connection-id, session token, or
auth header rides in gRPC metadata. The **`ClientInterceptor` is a no-op** (`LogCall` is empty; both
unary overloads just invoke the continuation). So the "invisible per-connection metadata/header" blind
spot is **confirmed empty** — there is no hidden client-side identity the `byte[]` capture missed.
- **The event-read query orchestration is genuinely not in managed code.** `CreateEventQuery` /
`EventQuery.StartQuery` / `MoveNext` are not in the managed `HistorianAccess`; the managed
`GrpcRetrievalClient.StartEventQuery` is a thin one-RPC stub. The query logic lives in the native
C++/CLI `HistorianClient` core (the mixed-mode part ILSpy can't decompile) — consistent with the
working-set being native/server-side, not a managed step we could read and replicate.
So **every client-controllable layer is now confirmed identical by reading the stock source**, not just
by wire match: request bytes, transport, channel options, gRPC metadata, interceptor. The remaining
difference is below the managed surface (native core) / server-side.
**Conclusion (after #1–#4 + stock client decompiled + TLS-tee capture).** Every angle is now exhausted:
- **client payload** — byte-identical (IL capture + decompile);
- **transport** — stock client is *also* gRPC-Web/HTTP-1.1; native HTTP/2 makes no difference, both 0 rows;
- **client metadata/interceptor/channel** — decompiled: identical gzip-only header, no-op interceptor, no
client cert; the TLS-tee capture confirms no hidden header and `clientCert=none`;
- **connection topology** — the native splits services across 5 connections and queries on a dedicated
RetrievalService connection; replicating that (`SPLIT_CHANNEL`) still returns 0 rows → the server
correlates by session handle, not connection;
- **data store** — global, unscoped; 71,332 events the engine serves via INSQL but withholds from our
gRPC connection.
The gate is a **server-internal per-connection retrieval working-set** that a pure-managed client cannot
reconstruct by matching wire bytes, transport, metadata, topology, or data — and the establishing logic is
in the native `HistorianClient` C++ core, not in any decompilable managed step or observable on the wire.
**gRPC event-row retrieval stands documented as auth-solved / retrieval-server-gated**; `ReadEventsAsync`
over gRPC keeps the honest no-row throw, and event reads use the WCF transport. Diagnostics retained for
any future server-side investigation: the `httpcap` TLS-tee proxy, the `CreateHttp2` / `SPLIT_CHANNEL`
switches, the `EventReadDiagnostic` test, and the `capture-event` harness (native, returns rows).
### Verify the parse path against the provided client's real data (2026-06-23) — found + fixed a latent bug
Used the provided 2023 R2 client as an **oracle**: the `capture-event` harness returns 50 real events
(verified live + through the `httpcap` proxy), and the `instrument-grpc-nonstream` rewrite captured the
exact `GetNextEventQueryResultBuffer.result` buffer the stock client received — **63,192 bytes, version
`0x0B` (11), rowCount 50** (25 `Alarm.Set` + 25 `Alarm.Clear`). Fed that real buffer through our
`HistorianEventRowProtocol.Parse` to verify the read path decodes genuine gRPC event data, and it
**exposed a latent parser bug**:
- The real row buffer is `version(2) + rowCount(4) + headerField(4, =0x1E)` then **markerless rows**
(`rowFormat(2)=7 + filetime(8) + 8×u16 slots + compact-ascii type + propCount + props`). Our parser
wrongly treated the one-time `0x1E` field as a **per-row marker** and re-consumed `[marker+format]`
every row — so it parsed only the **first** row of any multi-row buffer and stopped. This is **not
gRPC-specific**: the captured **WCF v9** buffer has the identical `0900 <rowCount> 1E000000 0700 …`
header, so the shipped WCF event read had the same latent multi-row truncation.
- **Fix:** read a 10-byte buffer header (skip the `0x1E` field once) and parse markerless rows; accept
container version **9 (WCF) and 11 (gRPC)**. Verified: the real 50-row buffer now decodes to exactly 50
events, ending cleanly at end-of-buffer (`Parse_RealStockClientCapture_DecodesAllEvents`, gated on
`HISTORIAN_EVENT_CAPTURE_NDJSON`); plus a synthetic v11 golden test. 328 offline tests pass.
So the **parse path is now verified against the provided client's real event data** — the one remaining
gap is strictly the server delivering rows to our gRPC connection (the working-set gate above). If that
were ever opened, the decoded events would now flow through correctly on both transports.
**2 of 3 layers cleared** (key exchange + client key); the 3rd (token construction) is localized to a
specific managed method, pending dnlib extraction. ExchangeKey + the v8 serializer are committed; the
orchestrator stays on v6 (set `eventConnection: true` to re-arm once the token construction lands). The
token-loop routing guardrail (`HistorianGrpcHandshakeRoutingTests`) was scoped to the closure so the
legitimate ExchangeKey call is allowed while still pinning that the Negotiate token loop never routes
there.
@@ -0,0 +1,45 @@
# 2023 R2 gRPC Interface-Version Integers (C3a)
**Captured:** 2026-06-25
**Transport:** 2023 R2 gRPC (h2c, unauthenticated `GetInterfaceVersion` RPCs — no credentials required)
**Status:** LIVE — integers captured from a real AVEVA Historian 2023 R2 server.
## Captured Values
| Service | UiVersion / Version | UiError / Error | Notes |
|---------------|---------------------|-----------------|----------------------------------------------------|
| History | 12 | 0 | Matches `HistoryInterfaceVersionGrpc2023R2 = 12` |
| Retrieval | 4 | 0 | Matches `RetrievalInterfaceVersion = 4` (unchanged from 2020) |
| Transaction | 2 | 0 | Matches `TransactionInterfaceVersion = 2` (unchanged from 2020) |
| Status | 4 | 0 | Reachability-only; version integer is not version-gated (see note) |
> **Field-name note:** The History, Retrieval, and Status proto responses use `UiError`/`UiVersion` fields.
> The Transaction response uses `Error`/`Version` (different naming convention in the proto). Both are
> captured correctly; the table uses a unified column header for readability.
> **Status note:** `StatusService.GetStatusInterfaceVersion` returned UiVersion=4, UiError=0 on the live
> 2023 R2 server. This differs from the historical 0 observed on 2020 WCF — both are reachability-only.
> Status is classified as reachability-only: its version integer carries no semantic meaning for the
> SDK's byte serializers, so its UiVersion is not gated and not asserted in tests.
## Evidence Test
`tests/AVEVA.Historian.Client.Tests/GrpcInterfaceVersionEvidenceTests.cs` —
`GrpcInterfaceVersions_LiveServer_MatchAcceptedSet` reads these four RPCs live and asserts:
- `history.UiError == 0` and `history.UiVersion ∈ {11, 12}`
- `retrieval.UiError == 0` and `retrieval.UiVersion == 4`
- `transaction.Error == 0` and `transaction.Version == 2`
- `status.UiError == 0` (version not asserted)
The test skips silently when `HISTORIAN_GRPC_HOST` is absent.
## Gap Closed
This document closes the **C3a** gap: "2023 R2 gRPC server-version integers not yet captured."
Prior to this capture, the `HistorianServerVersionGate` accepted History=12, Retrieval=4, and
Transaction=2 on the basis that they were inferred/expected-to-be-unchanged. All four integers are
now confirmed from a live 2023 R2 server over the gRPC transport; no widening of `AcceptedVersions`
is required (all captured values were already accepted).
The 2020 WCF baseline (History=11, Retrieval=4, Transaction=2) was captured earlier via the
`wcf-probe` command and is documented in `wcf-probe-remote-latest.json` and `wcf-contract-evidence.md`.
+224 -20
View File
@@ -1,6 +1,193 @@
# AVEVA Historian Managed Driver Handoff
Last updated: 2026-05-04 (write surface live: EnsT2 + DelT + ApplyScaling)
Last updated: 2026-06-23 (event-row parser fix merged; roadmap still exhausted — no actionable pure-code tasks remain)
> **Current status supersedes the historical blocker narrative below.** The
> sections from "Active Blocker" onward are a preserved reverse-engineering
> record of how the 2020 WCF read/write/event paths were cracked (2026-05-04).
> They are kept for provenance; they are **not** the live state. Start with
> "Current Status" immediately below.
## Current Status (2026-06-22) — roadmap exhausted
`docs/plans/hcal-roadmap.md` reachable surface is **complete**, and every plan
under `docs/plans/` is either DONE or has only gated items left. There are
**zero actionable pure-code tasks remaining**. Memory anchor:
`project_roadmap_exhausted_2020wcf`.
**Shipped + live-verified across both transports:**
- **Reads** (WCF + gRPC): `ProbeAsync`, `ReadRawAsync`, `ReadAggregateAsync`,
`ReadAtTimeAsync`, `ReadEventsAsync`, `BrowseTagNamesAsync`,
`GetTagMetadataAsync`, status helpers (`GetConnectionStatusAsync`,
`GetStoreForwardStatusAsync`, `GetSystemParameterAsync`).
- **Writes**: `EnsureTagAsync` (analog Float/Double/Int2/Int4/UInt4, `ApplyScaling`),
`DeleteTagAsync`, `RenameTagsAsync`, `AddTagExtendedPropertiesAsync`, and the
M3 historical/backfill `AddHistoricalValuesAsync` (**gRPC-only**, all five
analog types golden-tested + live write/read-back).
- **Config reads** (mostly gRPC): `GetRuntimeParameterAsync`,
`GetTagExtendedPropertiesAsync`, `ExecuteSqlCommandAsync` (WCF; gRPC
server-walled), `GetServerTimeZoneAsync` (gRPC-only).
- **Client-side**: M4 R4.1 store-and-forward outbox, R4.4 redundancy,
R4.3 measured-idle SF status.
The 2023 R2 **gRPC transport** (`HistorianTransport.RemoteGrpc`, port 32565)
reuses the proven 2020 WCF byte serializers/parsers unchanged inside protobuf
`bytes` fields, keyed by the Open2 session handle. Live-verified against a real
2023 R2 server (History interface v12) — see `reference_2023r2_live_server_access`.
**Everything still open is gated — none is a pure-code task:**
1. **gRPC event ROW retrieval** (`ReadEventsAsync` #2) — **AUTH-SOLVED / PARSE-VERIFIED /
RETRIEVAL-SERVER-GATED (every client-side angle exhausted 2026-06-23, merged `6faf8a5`).**
The v8 OpenConnection crypto wall is fully cracked + live-verified: the event connection
authenticates via **`HistoryService.ExchangeKey` (P-256 ECDH) → client key =
`SHA256(shared secret)` → credential token = `RC4(password-UTF16LE, key=MD5(client key))`**
(the native `HistorianCrypto.NRC4_V2.aahCryptV2` MD5-keyed RC4 scheme). RE'd via Frida CNG
hooks + dnlib IL extraction + an offline cracker; implemented pure-managed, golden-tested,
auth live-PASSES. The `StartEventQuery` v6 request and the Event-type v8 `OpenConnection`
(`ConnectionType=Event`) are shipped. **BUT** the query still returns **rowCount-0** while
the native returns 50 for a byte-identical request — and **all four next-session angles are
now tested and ruled out** (`grpc-event-query-capture.md`):
- **transport** — the stock client is *also* gRPC-Web/HTTP-1.1 (decompiled); plain native
HTTP/2 (`CreateHttp2`) returns the same 0 rows;
- **client metadata/cert** — decompiled + TLS-tee captured: gzip-only metadata, no-op
interceptor, **no TLS client cert** on either side;
- **connection topology** — the native splits services across 5 connections and queries on
a dedicated RetrievalService connection; replicating that (`HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL=1`)
still returns 0 rows → the server correlates by **session handle, not connection**;
- **data store** — via SOCKS→SQL: the event store is global/unscoped (no per-connection
column); the `Events` view (served by the engine via the INSQL provider) returns 71,332
events for the same window the gRPC query gets 0.
So the gate is a **server-internal per-connection retrieval working-set** in the native
`HistorianClient` C++ core — not reconstructable from a pure-managed client. **PARSE PATH NOW
VERIFIED + a latent bug FIXED:** fed the provided stock client's real captured result buffer
(63,192 B, 50 events) through `HistorianEventRowProtocol.Parse` — it exposed that the parser
treated the one-time `0x1E` buffer header field as a per-row marker, decoding only the **first
row of any multi-row buffer**. This also hit the **shipped WCF event read** (identical
`0900 <rowCount> 1E000000 0700` header). Fixed to a 10-byte buffer header + **markerless rows**,
accepting container version 9 (WCF) and 11 (gRPC); the real 50-row buffer now decodes to exactly
50 events (`Parse_RealStockClientCapture_DecodesAllEvents`, gated on `HISTORIAN_EVENT_CAPTURE_NDJSON`).
So if the server gate ever opens, decoded events flow through correctly on both transports; until
then the orchestrator stays on the no-row throw (`eventConnection: true` wired; opt-in
`EventReadDiagnostic` test, `HISTORIAN_GRPC_EVENT_DIAG=1`). Diagnostics retained: the `httpcap`
TLS-tee proxy, `CreateHttp2`/`SPLIT_CHANNEL` switches, the `sqlschema` SOCKS→SQL probe, the
`capture-event` harness (native, returns rows).
2. **R4.3 active-SF magnitude** — needs an **SF-active server** (D2 storage-engine
console handle).
3. **SendEvent over gRPC** — ✅ **SHIPPED + LIVE-VALIDATED 2026-06-23.** `SendEventAsync`
now routes over `RemoteGrpc` (`HistorianGrpcEventWriteOrchestrator`). Captured the native
client live (`capture-send-event` harness scenario): the send rides
`HistoryService.AddStreamValues` with the **same "OS" (0x534F) buffer the WCF path uses**
(`HistorianEventWriteProtocol` — "no distinct RPC" confirmed true), on a v8 Event session +
CM_EVENT registration. The write-enabled Event open is **byte-identical** to the read-only one
(diffed live — only per-session crypto differs), so the existing event-open path is reused
unchanged. End-to-end: pure-managed SDK send → `BSuccess=true` → event read back from the live
server (markers `SdkSendProbe`/`SdkCaptureProbe` confirmed in returned rows). Golden-tested
(`GrpcEventSendProtocolTests`) + gated live test (`SendEventAsync_OverGrpc_AcceptsEvent`,
opt-in `HISTORIAN_GRPC_EVENT_SEND=1`).
**PERSISTENCE RESOLVED:** the earlier WCF/M2 caveat was "server accepts AddS2 but the local
dev box does NOT persist the event to `v_AlarmEventHistory2`" — we couldn't tell if that was an
SDK gap or the environment. The gRPC read-back against the live 2023 R2 server **proves the event
persists** (sent via the pure-managed SDK, then independently read back from the server's event
history). So non-persistence on the local box was purely the dev-box event-ingestion environment,
**not an SDK gap** — the event-send path (WCF and gRPC, same "OS" buffer) is durably correct.
4. **ExecuteSqlCommand over gRPC** — **server-walled** (`CSrvDbConnection`;
RegisterTags prime doesn't help). Use WCF for SQL.
5. **R4.2 revision EDITS** — storage-engine-pipe-only on BOTH transports (the D2 wall).
6. **ReadBlocks** (`StartBlockRetrievalQuery`) — never captured on either transport.
7. **DeleteTagExtendedProperties** — server-blocked; **confirmed walled by capturing the NATIVE
client 2026-06-23.** The agent's "use `deleteFromServer=true`" angle is moot: the native
`HistorianAccess.DeleteTagExtendedPropertiesByName(...,deleteFromServer:true)`, driven with the
cross-session sync trick that gets it past the client-side err-229 sync gate
(`Capture-DeleteTagExtendedProperties.ps1`), returns **`Success=true` / ErrorCode=Success** —
yet across repeated fresh sessions the property is **re-fetchable and re-deletable every time**,
i.e. it is **never durably removed**. So the native client itself only performs an optimistic
client-side cache delete; the server does not durably honor it (matches the HCAL cache-sync
model the decompile shows). Shipping a `DeleteTagExtendedPropertiesAsync` would return a
misleading success while the property persists, so it correctly stays **unshipped**. (Earlier
gRPC multiplexed-channel hypothesis also PROBED + DISPROVEN 2026-06-22, merge `c88260c`; pinned
by `DeleteTagExtendedProperties_OverGrpc_ProbeMultiplexedChannel`.)
8. **Deferred-by-design** items (`write-commands` D1–D3, non-analog tag create,
etc.) — bounded out until an explicit customer/user demand signal.
To move any remaining item you need a **server-side / connection-level angle**
(item 1 — v8 event auth is solved; row retrieval is connection-gated, see the
NEXT SESSION section of `grpc-event-query-capture.md`), a **different server**
(SF-active for item 2), or a **demand signal** to unlock a deferred item.
(SendEvent over gRPC — formerly item 3 — is now SHIPPED + live-validated.)
Live-server gRPC probe recipe: set
`HISTORIAN_GRPC_HOST`/`_PORT 32565`/`_TLS true`/`_DNSID` + domain creds (strip
quotes — `reference_wonder_sql_vd03_credentials`) and run the gated
`HistorianGrpcIntegrationTests`.
### 2023 R2 stock-client binary dive (2026-06-23) — sharpened verdicts
Re-read the full decompiled stock 2023 R2 managed client
(`histsdk-2023r2-analysis/decompiled/`: `Archestra.Historian.GrpcClient`,
`ArchestrA.HistorianAccess`, `Archestra.Grpc.Contract`, `HistorianEvent`,
`HistorianAccessUtil`) as the oracle for every still-pending item. **Governing
fact:** `ArchestrA.HistorianAccess.dll` is a C++/CLI mixed assembly — every
data/config/write method is a thin shim into native `<Module>.HistorianClient.*`,
and the managed `Grpc*Client` wrappers are instantiated by **nothing** in the
decompiled set (`new Grpc*Client(` → zero call sites). So the buffer-building and
RPC-dispatch sequencing for these items lives in native C++ not present in the
binaries. That confirms the "gated" calls were not from missing managed steps —
with these refinements:
- **Item 1 (gRPC event rows)** — **confirmed native/server-side.** Stock event
call graph is provably identical to ours (transport, per-service channels,
gzip-only metadata, CM_EVENT registration, v8 ECDH Event-open, `StartEventQuery`
request bytes). `EventQuery.StartQuery`/`MoveNext` dispatch straight into native
`HistorianClient.StartEventQuery`/`GetNextRow`; the query orchestration that
would differ is native and not on the wire. One untested low-effort check
remains: byte-diff a captured **Event-connection** EnsureTags/RegisterTags
against our replay (the 83-vs-86-byte EnsT gap was never actually compared).
- **Item 3 (SendEvent over gRPC)** — ✅ **SHIPPED + LIVE-VALIDATED 2026-06-23** (was
"capturable"). RPC confirmed = `HistoryService.AddStreamValues` (the "no distinct RPC"
note is TRUE). The `btValues` VTQ buffer turned out to be already-owned: our M2
`HistorianEventWriteProtocol.SerializeAddStreamValuesBuffer` ("OS" buffer, decoded from
the WCF event-send) is the transport-independent `PackToVtq` equivalent and the gRPC send
uses it **verbatim** (live capture: sig `OS`/0x534F, CM_EVENT GUID, identical framing — NOT
the historical write's "ON" buffer). The write-enabled Event open is byte-identical to the
read-only one (live diff). So SendEvent-over-gRPC was pure assembly:
`HistorianGrpcEventWriteOrchestrator` = existing v8 Event open + existing CM_EVENT
registration + `AddStreamValues`(OS buffer). End-to-end live-validated (send → `BSuccess`
→ read back from the live server). Golden-tested + gated live test.
- **Item 4 (ExecuteSql over gRPC)** — **confirmed walled + explained.** The stock
client gates SQL **out client-side**: `HistorianAccess.ExecuteSqlCommand` returns
`OperationNotSupported` when `IsManagedHistorian(node)` or `!IsProcessConnectionRequested()`
(decompile ~:6198/:6214) and never sends the RPC. SQL-over-gRPC is unsupported by
design on a managed/gRPC historian; our `ProtocolEvidenceMissingException` is correct.
- **Item 5 (R4.2 revision edits)** — **confirmed HARD.** There is **no Revision RPC
in the gRPC contract** (zero "Revision" message types); the stock client reaches a
revision edit only via the native `HistorianClient.AddRevisionValuesBegin/AddRevisionValue/
AddRevisionValuesEnd` transaction trio over the storage-engine channel. NOTE: this is
a **distinct capability** from `AddNonStreamValues` (non-streamed original insert) —
`HistorianGrpcRevisionProbe` probes the latter; its doc comment was corrected to say so.
- **Item 6 (ReadBlocks/LoadBlocks)** — `LoadBlocks` request is a trivial
handle+sequence cursor but the `historyBlocks` response is a native blob with no
managed decoder, and it needs the D2-blocked `OpenStorageConnection` console handle.
Walled.
- **Item 7 (DeleteTagExtendedProperties)** — **capture done 2026-06-23; CONFIRMED WALLED, don't
ship.** RPC + string handle are correct; ADD/DELETE are structurally identical and neither uses
`StartJob`. The `deleteFromServer`-flag hypothesis is now tested and moot: the native
`DeleteTagExtendedPropertiesByName(...,deleteFromServer:true)`, driven past the err-229 client
sync gate with the cross-session trick (`Capture-DeleteTagExtendedProperties.ps1`), returns
`Success=true` — yet the property is **re-fetchable + re-deletable across repeated fresh sessions
(never durably removed)**. So the native client only does an optimistic client-side cache delete
the server doesn't durably honor (the HCAL cache-sync model). Shipping
`DeleteTagExtendedPropertiesAsync` would return a misleading success, so it stays unshipped.
- **SF/snapshot/shard/ForwardSnapshot ops** — only `Get/SetSFParameter` are managed-built
(typed strings); all others carry opaque native buffers and need the storage console
handle. Walled / tooling-internal.
**Net:** 3 items hard-confirmed walled with real explanations (4, 5, 6 + OpenStorageConnection),
and **2 moved to a precise, local-box-capturable target**: **SendEvent** (`PackToVtq` output)
and **DeleteTEP** (`BtInput` with `deleteFromServer=true`). Both need native instrumentation of
`aahClientManaged.dll` (Frida / IL-rewrite — repo tooling exists under
`tools/AVEVA.Historian.NativeTraceHarness` + `scripts/frida/`), not a special server.
## Project Direction
@@ -73,10 +260,14 @@ dotnet build .\Histsdk.slnx --no-restore
dotnet test .\Histsdk.slnx --no-build --logger "console;verbosity=minimal"
```
Current known-good result:
Current known-good result (2026-06-23):
- Build succeeds.
- Unit tests pass: 55/55.
- Build succeeds (0 warnings / 0 errors).
- Offline tests pass: 328/328 (live gRPC/integration tests skip cleanly without
their env vars). Gated live tests add to this when `HISTORIAN_*` /
`HISTORIAN_GRPC_*` are set. The +7 over the prior 321 are the event-row parser
fix's golden + gated-capture coverage (`HistorianEventRowProtocolTests`:
markerless multi-row, the v11 gRPC header, and the 50-event stock-client capture).
The workspace is a Git working tree (origin: gitea.dohertylan.com). Use
normal git workflow for change tracking; the prior "no working tree, use
@@ -227,7 +418,10 @@ Negative evidence:
- Running the same managed `ValCl` path through .NET Framework also fails, so
this is not just a .NET 10 WCF behavior difference.
## Active Blocker
## Historical: Read-Path Blocker (resolved 2026-05-04)
> Preserved RE record. This was the *original* active blocker; it is long
> resolved and is not the live state — see "Current Status" at the top.
**Resolved on `2026-05-04`.** The previous blocker — managed `ValCl`
rejected by the server — had two causes, both now fixed:
@@ -951,27 +1145,35 @@ record 5 of `instrumented-wcf-readmessage/readmessage-capture-event-latest.ndjso
### Event-row parser
`Wcf/HistorianEventRowProtocol.Parse(ReadOnlySpan<byte>)` parses the
version-9 row buffer:
> **CORRECTED 2026-06-23 (merged `6faf8a5`).** The skeleton below mis-read the `0x1E`
> as a *per-row* marker. Verifying the parser against the provided stock client's real
> 50-event buffer proved `0x1E` is a **one-time buffer-level header field**, and the rows
> are **markerless** — so the original parser silently returned only the **first** row of
> any multi-row buffer (on WCF too). The corrected layout and behaviour are below.
`Wcf/HistorianEventRowProtocol.Parse(ReadOnlySpan<byte>)` parses the row buffer
(container version 9 for WCF, 11 for 2023 R2 gRPC — both accepted):
```text
UInt16 version = 9
UInt16 version = 9 (WCF) | 11 (gRPC)
UInt32 rowCount
N rows, each:
UInt32 rowMarker = 0x1E
UInt32 headerField = 0x1E // ONE buffer-level field, NOT a per-row marker
N rows, each (MARKERLESS):
UInt16 rowFormat = 7
Int64 filetimeUtc (event time)
UInt16 × 8 fieldOffsets (opaque — purpose not fully decoded)
Property bag (sequence of name=value pairs; first name is the event type)
UInt16 propertyCount
Property bag (propertyCount × name=value pairs; first field is the event type)
```
The parser extracts `EventTimeUtc` and `Type` (the first compact-ASCII-string
in the property bag) for each row, and seeks forward to the next row by
scanning for the next `1E 00 00 00 07 00` marker. Property-bag value
encoding is partially decoded (compact ASCII `09 LEN 00 …`, UTF-16 strings
`43 UInt32 LEN × UInt16`, integers with markers in the `0x88–0x8B` range,
8-byte FILETIMEs) but **value parsing is intentionally not implemented yet**
— it requires more reverse-engineering and would need sanitized fixtures.
The parser reads the 10-byte buffer header (skipping the `0x1E` field once), then walks
each markerless row by length: `rowFormat(2) + filetime(8) + 8×UInt16 slots + compact-ASCII
type + propertyCount + propertyCount × (name + value)`. Value encoding **is** implemented
(compact ASCII `09 LEN 00 …`, Boolean `0x02`, GUID `0x10`, FILETIME `0x18`, Int32 `0x31`,
UTF-16 `0x43`; unknown markers preserve raw bytes). Verified against the provided client's
real buffer: `Parse_RealStockClientCapture_DecodesAllEvents` decodes all 50 events (25
Alarm.Set + 25 Alarm.Clear) to end-of-buffer (gated on `HISTORIAN_EVENT_CAPTURE_NDJSON`),
plus a synthetic v11 golden test.
5 unit tests in `HistorianEventRowProtocolTests.cs` cover empty buffer,
zero-row, wrong-version, two-row synthetic, and missing-marker. Test count
@@ -1069,10 +1271,12 @@ Typemarker dispatch:
Unknown markers preserve the raw `length` value bytes as a `byte[]` in
the property dictionary.
Each row layout (refines the earlier skeleton):
Each row layout (**corrected 2026-06-23** — see the "Event-row parser" note above; the
`0x1E` is a one-time buffer header field, NOT a per-row marker, and rows are markerless):
```text
UInt32 rowMarker = 0x1E
buffer header: UInt16 version (9|11) + UInt32 rowCount + UInt32 headerField (0x1E)
each row (markerless):
UInt16 rowFormat = 7
Int64 eventTimeUtcFiletime
UInt16 × 8 // purpose unclear
@@ -0,0 +1,126 @@
# Handshake / session-reuse spike — live results
> **Question:** does the 2023 R2 historian honor REUSING one authenticated session
> (channel + `OpenConnection` client handle) across multiple operations, instead of
> the per-operation Create+handshake the SDK does today? This is the precondition for
> "handshake amortization" (HistorianGateway `pending.md` A1).
>
> **Verdict: GREEN — reuse works and the win is large — but the server idle-expires a
> session in ~20–25 s, so a reuse pool must keep sessions warm.**
**Date:** 2026-06-25
**Branch:** `spike/handshake-reuse`
**Server:** live 2023 R2 (`wonder-sql-vd03`), RemoteGrpc transport, read-only test tag.
**Harness:** `tests/AVEVA.Historian.Client.Tests/HandshakeReuseSpikeTests.cs` driving the new
internal seam `HistorianGrpcReadOrchestrator.RunRawQueryOnSession(connection, clientHandle, …)`
(runs a raw query against an externally-supplied, already-authenticated connection + handle —
no Create, no handshake).
---
## 1. Reuse validity — GREEN
`ReusedSession_RunsManyReads_AllSucceed` **passed**: one `HistorianGrpcChannelFactory.Create`
+ one `HistorianGrpcHandshake.OpenSession`, then **5 consecutive `RunRawQueryOnSession` reads on
the same `ClientHandle`** — all returned rows.
```
open-session (handshake) = 325 ms
reused-read[0] = 96 ms, rows=8
reused-read[1] = 101 ms, rows=8
reused-read[2] = 179 ms, rows=8
reused-read[3] = 92 ms, rows=8
reused-read[4] = 95 ms, rows=8
```
The server accepts the same client handle across back-to-back `StartQuery`/`GetNextQueryResultBuffer`/
`EndQuery` cycles. Per-query handles are opened/closed each op; the **session** handle is the reused
artifact.
## 2. Win magnitude — large (~4.7×)
`ReusedSession_VsPerCallPath_LogsLatencyDelta` (logged, not asserted):
```
per-call (5 ops) = 2626 ms # fresh Create + full handshake + query, ×5
amortized (5 ops) = 561 ms # one handshake + 5 reused reads
saving over 5 ops = 2065 ms
```
The handshake (`GetInterfaceVersion` → `ValidateClientCredential` NTLM token loop →
`OpenConnection`, ~325 ms) dominates per-call cost. Amortized, a read is ~110 ms vs ~525 ms
per-call. **Amortization is clearly worth the refactor for any burst of activity.**
## 3. Expiry — idle timeout ~20–25 s (NOT an absolute TTL)
`ReusedSession_IdleSweep_SurfacesExpiryTier` rethrows at the first idle gap the server rejects.
Coarse sweep `[0, 30]`: `idle 0s → OK`, `idle 30s → BROKE`.
Fine sweep `[0,5,10,15,20,25,30]`:
```
idle 0s -> OK (rows=8)
idle 5s -> OK
idle 10s -> OK
idle 15s -> OK
idle 20s -> OK # session age here ≈ 50 s cumulative, still alive
idle 25s -> BROKE (InvalidOperationException: gRPC StartQuery (raw) failed, errorLen=5)
```
**Key inference — it's an idle timeout, not a fixed session lifetime.** The reads at gaps of
5/10/15/20 s kept succeeding even though the cumulative session age reached ~50 s by the 20 s-gap
read. The session only died after a **≥25 s idle gap**. So a session survives indefinitely as long
as operations are spaced under ~20 s apart; a quiet gap of ≥25 s invalidates it.
Expired-session failure mode on the wire: `StartQuery` returns `BSuccess=false` with a 5-byte error
buffer, surfaced by the SDK as `InvalidOperationException: gRPC StartQuery (raw) failed
(errorLen=5)`.
---
## 4. Implications for Phase 1 (the full amortization refactor)
A reuse pool is viable and high-value, with two requirements driven by §3:
1. **Keep sessions warm.** Ping each pooled session well under the ~20 s idle floor (e.g. a
~10–15 s keepalive — a cheap handle-using op such as `GetSystemParameter`) so a steady-state
session never crosses the idle timeout. Without a keepalive, amortization only helps within a
<~20 s activity burst.
2. **Reactive re-auth on expiry.** Treat `StartQuery failed (errorLen=5)` (and the equivalent on
other handle ops) as an expired-session signal: evict the session and re-handshake on next use
(one handshake penalty). In HistorianGateway this maps onto the existing
`IHistorianConnectionPool.ReportFaulted` eviction seam.
**Concurrency note (unchanged guidance):** lease a session exclusively per-op from a bounded pool —
this validity test only exercised *sequential* reuse, so concurrent use of one handle (esp.
streaming cursors) remains unproven and should be avoided by exclusive leasing.
**Gate decision:** GREEN → HistorianGateway A1 Phase 1 (HistorianSession primitive + orchestrator
acquire/execute split + re-vendor + leased-session pool with keepalive) is warranted and earns its
own design + plan.
---
## 5. Write-spike addendum (Phase 1 Stage 0) — 2026-06-25
Extends the harness to the write path via the `RunWriteOnSession` seam on
`HistorianGrpcHistoricalWriteOrchestrator`. Read + bounded writes to `HISTORIAN_WRITE_SANDBOX_TAG`
only.
```
reused-write[0] = 377 ms, ok=True
reused-write[1] = 76 ms, ok=True # 2nd write reuses the same 0x401 session — no handshake
read-on-0x401 -> OK (rows=3) # a WRITE-enabled session ALSO serves reads
```
**Findings:**
- **Write-session reuse — GREEN.** Two historical writes on one reused `0x401` (write-enabled)
session both succeed; the 2nd skips the Create+handshake.
- **One-kind pool — CONFIRMED.** A `0x401` session served a `StartQuery` read (`session.ClientHandle`)
successfully. So a single **write-enabled** session serves both reads and writes — the gateway pool
needs **one session kind**, not two. (`0x401` "unlocks write capability" and is a superset of the
`0x402` read-only mode, as the vendored comment hinted.)
**Decision for Phase 1 Stage 3:** the gateway always opens `WriteEnabled` sessions; the
`HistorianSessionPool` is a **single warm pool** (no per-kind keying). `HistorianSessionKind` still
exists upstream for API clarity, but the gateway uses only `WriteEnabled`.
@@ -0,0 +1,75 @@
# WCF event-read spike — live result (2026-06-25/26): transport+auth viable, row-retrieval server-gated
Settles the open question behind **C2** ("event reads over gRPC are gated; the only listed unblock is
*route event reads via WCF*"). The gRPC event-read path is a proven server-side dead-end
(`grpc-event-query-capture.md`: auth fully solved, every client-controllable layer byte-matched to the
stock client, yet the server scopes 0 rows to our connection). This spike resolved the **WCF** leg.
> **Correction to an earlier draft of this doc.** A first pass concluded "the 2023 R2 historian does not
> serve the legacy WCF transport (connection reset at framing)." **That was a test error, not a server
> fact.** It connected to the historian's real WCF port `32568` *directly* and used the Windows-integrated
> transport. In this environment the historian is reached through a **reverse SSH tunnel** (local
> `42568` → historian `32568`), and integrated/Kerberos auth does not work through that tunnel. The
> socket-RST was the tunnel/transport mismatch, not an absent listener. Corrected below.
## What was run
A Windows-only-by-default, env-gated diagnostic (`tests/AVEVA.Historian.Client.Tests/WcfEventReadSpikeTests.cs`)
drives `HistorianWcfEventOrchestrator.ReadEventsAsync` directly. The decisive run was **cross-platform,
direct** (no tunnel): from the VPN-holding host straight to the historian's real WCF endpoint
`net.tcp://<historian>:32568/HistCert`, using the **certificate transport** (`RemoteTcpCertificate`,
TLS, `AllowUntrustedServerCertificate`) and `NegotiateAuthentication` (cross-platform, explicit domain
credentials). The SDK's interface-version gate was bypassed (`VerifyServerInterfaceVersion=false`) —
the 2023 R2 WCF **History interface reports version 13** (this SDK's serializers target 11/12).
## Result — transport+auth viable; row-retrieval server-gated (sanitized)
Progression of the live errors as the addressing/transport was corrected:
| attempt | error |
|---|---|
| direct `:32568`, integrated | `SocketException` "forcibly closed" (wrong port + transport for the tunnel) |
| tunnel `:42568`, integrated | `ProtocolException` at the security UpgradeResponse (integrated can't negotiate through the tunnel) |
| tunnel `:42568`, certificate | reached the WCF dispatcher → `AddressFilter` mismatch (tunnel rewrites the port) |
| **direct `:32568`, certificate, cross-platform** | **past auth** → `ProtocolEvidenceMissingException`: History interface version **13** |
| + `VerifyServerInterfaceVersion=false` | **full chain runs**; query returns a 10-byte **0-row** header, then `GetNext` long-polls |
Connection-mode experiment (certificate transport, direct, version-bypassed, a 1-day window that holds
events), comparing the native OpenConnection mode used for the event-read chain:
| connMode | RegisterTags (RTag2) | EnsureTags (EnsT2) | result buffer | events |
|---|---|---|---|---|
| `0x501` (event) | **0 — success** | 1 (benign-false, as in the 2020 flow) | 10 bytes (0-row header) | **0** |
| `0x401` (write) | 1 (fail) | 1 | 10 bytes | 0 |
| `0x402` (read-only, default) | 1 (fail) | 1 | 10 bytes | 0 |
## Conclusion
1. **WCF transport + auth ARE viable on 2023 R2.** The certificate (TLS) transport negotiates and the
`NegotiateAuthentication` app-level handshake authenticates — **cross-platform** (proven from a
non-Windows VPN host). The earlier "WCF not served" conclusion was wrong. (Integrated/Windows
transport security is not usable through the reverse tunnel — `net.tcp` Kerberos does not tunnel.)
2. **The event-read chain needs the `0x501` event connection mode.** With it, CM_EVENT `RegisterTags`
**succeeds** (it fails on `0x402`/`0x401`). `EnsureTags` returns false, but that is documented as
benign in the 2020 flow that *did* return rows.
3. **Row retrieval is server-gated — same as gRPC.** Even with auth solved and `RegisterTags` succeeding,
over a window that holds events, `StartEventQuery` succeeds but `GetNextEventQueryResultBuffer` returns
a **0-row** header (10 bytes) and long-polls. Registration and window are ruled out as the cause; the
server simply does not scope event rows to a managed connection. This is the **identical** server-side
per-connection retrieval working-set gate proven for gRPC in `grpc-event-query-capture.md`.
**Therefore event reads do not return rows on the 2023 R2 historian over either transport** — gRPC
(retrieval-server-gated) and WCF (transport+auth work, but the same server-side row gate). The only
remaining theoretical unblock is server-side (AVEVA exposing event-row retrieval to a managed
connection) — not client-fixable. **C2 stays closed won't-fix**, for this (corrected) reason.
## SDK additions from this investigation (retained, build-clean, golden where applicable)
- `HistorianClientOptions.ConnectViaAddress` — WCF `Via` (connect to a tunnel/proxy while addressing the
SOAP `To` the real endpoint), so a port-forward whose local port differs from the server's real port
satisfies the server-side WCF AddressFilter.
- `HistorianClientOptions.EventReadConnectionModeOverride` — diagnostic override of the event-read
OpenConnection mode (the `0x501` finding above).
- The C2 spike is now transport-selectable (integrated|certificate), cross-platform for the cert
transport, bounded (per-call timeout + overall budget with a phase-diagnostic dump), and version-gate
bypassable. Output stays sanitized (counts, native return codes, buffer lengths, sha256).
@@ -0,0 +1,173 @@
// Frida hook for the native ExchangeKey credential-token crypto (Windows CNG / bcrypt.dll).
// Traces the ECDH secret agreement, the KDF (with its parameter list), symmetric-key import, and
// encrypt/hash so the 26-byte v8 credential-token derivation can be reconstructed in managed code.
// Reverse-engineering aid only — observes the native client; nothing is shipped from here.
'use strict';
function resolve(modName, fnName) {
let m = null;
try { m = Process.getModuleByName(modName); } catch (e) {
try { m = Module.load(modName); } catch (e2) { return null; }
}
try { return m.findExportByName(fnName); } catch (e) { return null; }
}
function dump(label, ptr, len) {
if (ptr.isNull() || len <= 0) { console.log(label + ' <empty>'); return; }
const n = Math.min(len, 256);
console.log(label + ' (' + len + ' bytes)\n' + hexdump(ptr, { length: n, header: false, ansi: false }));
}
function hook(modName, fnName, onEnter, onLeave) {
const addr = resolve(modName, fnName);
if (!addr) { console.log('[skip] ' + modName + '!' + fnName + ' not found'); return; }
Interceptor.attach(addr, { onEnter: onEnter, onLeave: onLeave });
console.log('[hooked] ' + modName + '!' + fnName);
}
// BCryptOpenAlgorithmProvider(phAlgorithm, pszAlgId, pszImplementation, dwFlags) — names every algo used.
hook('bcrypt.dll', 'BCryptOpenAlgorithmProvider', function (a) {
console.log('[OpenAlgorithmProvider] algId=' + (a[1].isNull() ? '?' : a[1].readUtf16String()));
});
// BCryptSecretAgreement(hPrivKey, hPubKey, *phAgreedSecret, flags)
hook('bcrypt.dll', 'BCryptSecretAgreement', function (a) {
console.log('[SecretAgreement] hPriv=' + a[0] + ' hPub=' + a[1]);
});
// Decode a BCryptBufferDesc parameter list (used by BCryptDeriveKey) into (type -> bytes).
function dumpParamList(pParamList) {
if (pParamList.isNull()) { console.log(' paramList <null>'); return; }
const cBuffers = pParamList.add(4).readU32(); // ULONG ulVersion; ULONG cBuffers;
const pBuffers = pParamList.add(8).readPointer(); // BCryptBuffer* pBuffers;
const names = { 0: 'HASH_ALGORITHM', 1: 'SECRET_PREPEND', 2: 'SECRET_APPEND', 3: 'HMAC_KEY',
4: 'TLS_PRF_LABEL', 5: 'TLS_PRF_SEED', 6: 'SECRET_HANDLE', 8: 'SP80056A_CONCAT',
0xD: 'LABEL', 0xE: 'CONTEXT', 0xF: 'SALT', 0x10: 'ITERATION_COUNT' };
console.log(' paramList cBuffers=' + cBuffers);
for (let i = 0; i < cBuffers; i++) {
const b = pBuffers.add(i * 16); // { ULONG cbBuffer; ULONG BufferType; PVOID pvBuffer; }
const cb = b.readU32();
const type = b.add(4).readU32();
const pv = b.add(8).readPointer();
const tn = names[type] || ('0x' + type.toString(16));
if (type === 0 || type === 4 || type === 0xD) { // string-ish (hash alg name / label)
console.log(' [' + tn + '] ' + (pv.isNull() ? '?' : pv.readUtf16String()));
} else {
dump(' [' + tn + ']', pv, cb);
}
}
}
// BCryptDeriveKey(hSecret, pwszKDF, *pParamList, pbDerivedKey, cbDerivedKey, *pcbResult, flags)
hook('bcrypt.dll', 'BCryptDeriveKey', function (a) {
this.kdf = a[1].isNull() ? '?' : a[1].readUtf16String();
this.outKey = a[3]; this.pcb = a[5];
console.log('[DeriveKey] KDF=' + this.kdf + ' cbDerivedKey=' + a[4].toInt32());
dumpParamList(a[2]);
}, function () {
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
dump('[DeriveKey] derived', this.outKey, n);
});
hook('bcrypt.dll', 'BCryptDeriveKeyPBKDF2', function (a) {
console.log('[PBKDF2] cbPassword=' + a[2].toInt32() + ' cbSalt=' + a[4].toInt32() + ' iter=' + a[5]);
dump(' password', a[1], a[2].toInt32());
dump(' salt', a[3], a[4].toInt32());
});
// BCryptGenerateSymmetricKey(hAlg, *phKey, pbKeyObject, cbKeyObject, pbSecret, cbSecret, flags) — the actual key bytes.
hook('bcrypt.dll', 'BCryptGenerateSymmetricKey', function (a) {
dump('[GenerateSymmetricKey] keyBytes', a[4], a[5].toInt32());
});
// BCryptEncrypt(hKey, pbIn, cbIn, *pPad, pbIV, cbIV, pbOut, cbOut, *pcbResult, flags)
hook('bcrypt.dll', 'BCryptEncrypt', function (a) {
this.out = a[6]; this.pcb = a[8];
dump('[Encrypt] plaintext', a[1], a[2].toInt32());
dump('[Encrypt] IV', a[4], a[5].toInt32());
}, function () {
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
dump('[Encrypt] ciphertext', this.out, n);
});
// Hash path (in case the token is a keyed hash rather than a cipher).
hook('bcrypt.dll', 'BCryptHashData', function (a) {
dump('[HashData] input', a[1], a[2].toInt32());
});
hook('bcrypt.dll', 'BCryptFinishHash', function (a) {
this.out = a[1]; this.cb = a[2].toInt32();
}, function () {
dump('[FinishHash] digest', this.out, this.cb);
});
// ---- NCrypt (CNG key-storage layer) — the likely home of the ECDH ExchangeKey + token crypto ----
// NCryptSecretAgreement(hPrivKey, hPubKey, *phAgreedSecret, dwFlags)
hook('ncrypt.dll', 'NCryptSecretAgreement', function (a) {
console.log('[NCryptSecretAgreement] hPriv=' + a[0] + ' hPub=' + a[1]);
console.log(' backtrace (addr -> module+offset):');
Thread.backtrace(this.context, Backtracer.ACCURATE).slice(0, 14).forEach(function (addr) {
const m = Process.findModuleByAddress(addr);
if (m) {
console.log(' ' + addr + ' ' + m.name + '+0x' + addr.sub(m.base).toString(16));
} else {
console.log(' ' + addr + ' <JIT/unknown>');
}
});
});
// NCryptDeriveKey(hSharedSecret, pwszKDF, *pParameterList, pbDerivedKey, cbDerivedKey, *pcbResult, dwFlags)
hook('ncrypt.dll', 'NCryptDeriveKey', function (a) {
this.kdf = a[1].isNull() ? '?' : a[1].readUtf16String();
this.outKey = a[3]; this.pcb = a[5];
console.log('[NCryptDeriveKey] KDF=' + this.kdf + ' cbDerivedKey=' + a[4].toInt32());
dumpParamList(a[2]);
}, function () {
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
dump('[NCryptDeriveKey] derived', this.outKey, n);
});
// NCryptEncrypt(hKey, pbInput, cbInput, *pPaddingInfo, pbOutput, cbOutput, *pcbResult, dwFlags)
hook('ncrypt.dll', 'NCryptEncrypt', function (a) {
this.out = a[4]; this.pcb = a[6];
dump('[NCryptEncrypt] plaintext', a[1], a[2].toInt32());
}, function () {
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
dump('[NCryptEncrypt] ciphertext', this.out, n);
});
// NCryptImportKey(hProvider, hImportKey, pszBlobType, *pParameterList, *phKey, pbData, cbData, dwFlags)
hook('ncrypt.dll', 'NCryptImportKey', function (a) {
console.log('[NCryptImportKey] blobType=' + (a[2].isNull() ? '?' : a[2].readUtf16String()));
dump(' blob', a[5], a[6].toInt32());
});
// NCryptExportKey(hKey, hExportKey, pszBlobType, *pParameterList, pbOutput, cbOutput, *pcbResult, dwFlags)
hook('ncrypt.dll', 'NCryptExportKey', function (a) {
this.blobType = a[2].isNull() ? '?' : a[2].readUtf16String();
this.out = a[4]; this.pcb = a[6];
}, function () {
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
console.log('[NCryptExportKey] blobType=' + this.blobType);
dump(' blob', this.out, n);
});
hook('ncrypt.dll', 'NCryptOpenStorageProvider', function (a) {
console.log('[NCryptOpenStorageProvider] ' + (a[1].isNull() ? '?' : a[1].readUtf16String()));
});
// BCrypt EC key operations (in case the ECDH is bcrypt but uses import/export rather than DeriveKey).
hook('bcrypt.dll', 'BCryptImportKeyPair', function (a) {
console.log('[BCryptImportKeyPair] blobType=' + (a[2].isNull() ? '?' : a[2].readUtf16String()) + ' cb=' + a[5].toInt32());
dump(' blob', a[4], a[5].toInt32());
});
hook('bcrypt.dll', 'BCryptExportKey', function (a) {
this.blobType = a[2].isNull() ? '?' : a[2].readUtf16String();
this.out = a[3]; this.pcb = a[5];
}, function () {
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
console.log('[BCryptExportKey] blobType=' + this.blobType);
dump(' blob', this.out, n);
});
console.log('=== CNG ExchangeKey crypto hooks installed ===');
@@ -1,3 +1,4 @@
using System.Net.Http;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;
using Grpc.Core;
@@ -63,6 +64,50 @@ internal static class HistorianGrpcChannelFactory
GrpcChannel channel = GrpcChannel.ForAddress(address, channelOptions);
return new HistorianGrpcConnection(channel, BuildMetadata(options));
}
/// <summary>
/// Builds an event-path channel that speaks <b>native HTTP/2 gRPC</b> (no <see cref="GrpcWebHandler"/>
/// wrap) — the leading hypothesis for why gRPC-Web event reads return zero rows while the native
/// <c>Grpc.Core</c> HTTP/2 client returns rows for a byte-identical request. The stock 2023 R2 client
/// uses native <c>Grpc.Core</c> (HTTP/2); reads happen to work over gRPC-Web too, but the
/// connection-scoped event query may require a true HTTP/2 connection. Over TLS this depends on the
/// server negotiating the <c>h2</c> ALPN protocol; <see cref="SocketsHttpHandler"/> is pinned to
/// HTTP/2 exact so the channel does not silently fall back to HTTP/1.1.
/// </summary>
public static HistorianGrpcConnection CreateHttp2(HistorianClientOptions options)
{
string address = ResolveAddress(options);
var socketsHandler = new SocketsHttpHandler
{
EnableMultipleHttp2Connections = true
};
if (options.AllowUntrustedServerCertificate && options.GrpcUseTls)
{
socketsHandler.SslOptions = new SslClientAuthenticationOptions
{
RemoteCertificateValidationCallback = (_, _, _, _) => true
};
}
var channelOptions = new GrpcChannelOptions
{
HttpHandler = socketsHandler
};
// GrpcChannel over a SocketsHttpHandler already issues requests as HTTP/2 with
// RequestVersionExact (no GrpcWebHandler means no HTTP/1.1 fallback to mask a failed h2
// negotiation — it surfaces instead).
GrpcChannel channel = GrpcChannel.ForAddress(address, channelOptions);
return new HistorianGrpcConnection(channel, BuildMetadata(options));
}
private static Metadata BuildMetadata(HistorianClientOptions options)
{
// The stock client always advertises gzip request encoding; honour the option so
// bandwidth-limited links can disable it.
var metadata = new Metadata();
@@ -71,7 +116,7 @@ internal static class HistorianGrpcChannelFactory
metadata.Add("grpc-internal-encoding-request", "gzip");
}
return new HistorianGrpcConnection(channel, metadata);
return metadata;
}
private static bool AcceptAnyCertificate(
@@ -34,13 +34,16 @@ namespace AVEVA.Historian.Client.Grpc;
/// (the read path proved the front-door session is sufficient over gRPC).
/// </para>
/// <para>
/// <b>Live status (2026-06-22):</b> the chain runs end-to-end and <c>StartEventQuery</c> succeeds, but
/// <c>GetNextEventQueryResultBuffer</c> <b>long-polls</b> when the query has no rows — it blocks to the
/// call deadline instead of returning the synchronous 5-byte code-85 terminal the 2020 WCF op returns.
/// A poll-deadline expiry is therefore treated as the no-data terminal (see the loop). The idle dev box
/// holds no events, so <b>row-level retrieval is not yet live-verified</b>; verifying parsed rows over
/// gRPC awaits an event-bearing 2023 R2 server. This is tooled + completes cleanly, NOT proven to
/// return rows.
/// <b>Live status — server-gated (settled 2026-06-25):</b> the chain runs end-to-end and
/// <c>StartEventQuery</c> succeeds, but <c>GetNextEventQueryResultBuffer</c> <b>long-polls</b> to the
/// no-data terminal (instead of the synchronous 5-byte code-85 terminal the 2020 WCF op returns); a
/// poll-deadline expiry is treated as that terminal (see the loop). This is <b>not</b> an empty-box
/// artifact: the live 2023 R2 server holds tens of thousands of events yet scopes <b>0 rows</b> to a
/// managed connection. Every client-controllable layer was byte-matched to the stock client that returns
/// rows (see <c>docs/reverse-engineering/grpc-event-query-capture.md</c>) — the gate is a server-internal
/// per-connection retrieval working-set, <b>not client-fixable</b>. The legacy WCF transport is not a
/// fallback on 2023 R2 (<c>docs/reverse-engineering/wcf-event-read-spike-results.md</c>). Tooled +
/// completes cleanly, but proven NOT to return rows over a managed connection.
/// </para>
/// </summary>
internal sealed class HistorianGrpcEventOrchestrator
@@ -58,6 +61,15 @@ internal sealed class HistorianGrpcEventOrchestrator
/// <summary>Diagnostic: type+code description of the most recent error/terminal buffer.</summary>
public string LastErrorBufferDescription { get; private set; } = string.Empty;
/// <summary>Diagnostic: which transport the event channel used (<c>grpc-web</c> or <c>http2</c>).</summary>
public string EventChannelMode { get; private set; } = string.Empty;
/// <summary>Diagnostic: hex of the most recent result buffer (first 48 bytes).</summary>
public string LastResultBufferHex { get; private set; } = string.Empty;
/// <summary>Diagnostic: hex of the most recent GetNext error buffer.</summary>
public string LastErrorBufferHex { get; private set; } = string.Empty;
public async IAsyncEnumerable<HistorianEvent> ReadEventsAsync(
DateTime startUtc,
DateTime endUtc,
@@ -92,9 +104,12 @@ internal sealed class HistorianGrpcEventOrchestrator
{
throw new ProtocolEvidenceMissingException(
$"ReadEvents over gRPC did not return rows within {OverallBudget.TotalSeconds:0}s: StartEventQuery " +
"succeeds but the CM_EVENT registration replay stalls and GetNextEventQueryResultBuffer long-polls " +
"(no synchronous code-85 terminal over gRPC). Row-level retrieval is not yet verified over gRPC " +
"(the dev box holds no events) — use the WCF transport for event reads.");
"succeeds but GetNextEventQueryResultBuffer long-polls to the no-data terminal. Event-row retrieval is " +
"auth-solved but SERVER-GATED on 2023 R2 over both transports — the server scopes 0 rows to a managed " +
"connection (gRPC: docs/reverse-engineering/grpc-event-query-capture.md). The WCF transport reaches the " +
"2023 R2 historian (certificate transport + auth work, CM_EVENT registration succeeds on the 0x501 event " +
"connection) but hits the SAME server-side row gate — 0-row buffer + long-poll (see " +
"docs/reverse-engineering/wcf-event-read-spike-results.md). Not client-fixable on either transport.");
}
foreach (HistorianEvent evt in events)
@@ -137,27 +152,44 @@ internal sealed class HistorianGrpcEventOrchestrator
private List<HistorianEvent> RunEventChain(DateTime startUtc, DateTime endUtc, HistorianEventFilter? filter, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken);
// Hypothesis #1 (server-side/connection angle, grpc-event-query-capture.md): the native client
// uses Grpc.Core native HTTP/2, while our default channel wraps gRPC-Web over HTTP/1.1. Reads
// work over gRPC-Web, but the connection-scoped event query may require a true HTTP/2 connection.
// Opt in via HISTORIAN_GRPC_EVENT_HTTP2=1 to use a plain HTTP/2 channel for the event path only.
bool useHttp2 = string.Equals(
Environment.GetEnvironmentVariable("HISTORIAN_GRPC_EVENT_HTTP2"), "1", StringComparison.Ordinal);
EventChannelMode = useHttp2 ? "http2" : "grpc-web";
using HistorianGrpcConnection connection = useHttp2
? HistorianGrpcChannelFactory.CreateHttp2(_options)
: HistorianGrpcChannelFactory.Create(_options);
// Event reads need an Event-type (v8) connection. OpenSession(eventConnection: true) runs the
// full v8 path: HistoryService.ExchangeKey (P-256 ECDH) -> client key = SHA256(secret) -> v8
// OpenConnection with ConnectionType=Event and the credential token RC4(password, MD5(clientKey)).
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken, eventConnection: true);
RegisterCmEventTag(connection, session, cancellationToken);
List<HistorianEvent> events = RunEventQuery(connection, session, startUtc, endUtc, filter, cancellationToken);
List<HistorianEvent> events = RunEventQueryOnSession(connection, session, startUtc, endUtc, filter, cancellationToken);
// Honest no-data handling: when the query returns real rows, hand them back. When it instead
// reaches the no-data terminal with ZERO rows (the gRPC server long-polls GetNext rather than
// returning the WCF code-85 terminal), we cannot distinguish "genuinely no events in range"
// from "the CM_EVENT registration replay didn't fully land over gRPC" — so we refuse to return
// a possibly-false empty list and surface the unverified state instead. An event-bearing 2023 R2
// server will return rows here and exercise the parse path; flip this once that is captured.
// a possibly-false empty list and surface the gated state instead. Proven server-gated: the live
// 2023 R2 server holds tens of thousands of events yet scopes 0 to a managed connection
// (grpc-event-query-capture.md). WCF reaches the same historian (cert transport + auth work,
// CM_EVENT registers on the 0x501 event connection) but hits the SAME row gate — not a fallback
// (wcf-event-read-spike-results.md).
if (events.Count == 0)
{
throw new ProtocolEvidenceMissingException(
"ReadEvents over gRPC: the chain completes and StartEventQuery succeeds, but " +
"GetNextEventQueryResultBuffer returns no rows (it long-polls to the no-data terminal " +
$"after the CM_EVENT registration replay; last={LastErrorBufferDescription}). Row-level " +
"retrieval is not yet verified over gRPC (the dev box holds no events) — use the WCF " +
"transport for event reads until a capture against an event-bearing 2023 R2 server confirms it.");
$"after the CM_EVENT registration replay; last={LastErrorBufferDescription}). Event-row retrieval is " +
"auth-solved but SERVER-GATED on 2023 R2 over both transports — the server scopes 0 rows to a managed " +
"connection (gRPC: docs/reverse-engineering/grpc-event-query-capture.md; WCF reaches the historian and " +
"registers on the 0x501 event connection yet hits the same row gate: " +
"docs/reverse-engineering/wcf-event-read-spike-results.md). Not client-fixable on either transport.");
}
return events;
@@ -203,57 +235,64 @@ internal sealed class HistorianGrpcEventOrchestrator
{
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel);
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
var transactionClient = new GrpcTransaction.TransactionService.TransactionServiceClient(connection.Channel);
// Discovery dance the native event flow runs between Open2 and EnsT2. All bounded by the
// short RegistrationDeadline (several stall server-side on the remote box).
TryRun(() => statusClient.GetStatusInterfaceVersion(new GrpcStatus.GetStatusInterfaceVersionRequest(), connection.Metadata, RegistrationDeadline(), cancellationToken));
TryRun(() => statusClient.GetStatusInterfaceVersion(new GrpcStatus.GetStatusInterfaceVersionRequest(), connection.Metadata, RegistrationDeadline(), cancellationToken));
byte[] historianVersionRequest = HistorianEventRegistrationProtocol.BuildGetHistorianInfoRequest("HistorianVersion");
TryRun(() => statusClient.GetHistorianInfo(
new GrpcStatus.GetHistorianInfoRequest { StrHandle = session.StringHandle, BtRequest = ByteString.CopyFrom(historianVersionRequest) },
connection.Metadata, RegistrationDeadline(), cancellationToken));
TryRun(() => statusClient.GetHistorianInfo(
new GrpcStatus.GetHistorianInfoRequest { StrHandle = session.StringHandle, BtRequest = ByteString.CopyFrom(historianVersionRequest) },
connection.Metadata, RegistrationDeadline(), cancellationToken));
// Native 2023 R2 gRPC event-connection registration sequence (captured order):
// UpdateClientStatus -> RegisterTags(CM_EVENT) -> EnsureTags(CM_EVENT) -> GetHistorianInfo
// -> GetSystemParameter x7. (StartEventQuery follows in RunEventQuery.) The 2020-WCF-era extra
// probes (cross-service GetV, params-before-register) are NOT in the gRPC event flow.
byte[] clientStatus = HistorianEventRegistrationProtocol.BuildUpdateClientStatusBlob();
TryRun(() => historyClient.UpdateClientStatus(
new GrpcHistory.UpdateClientStatusRequest { StrHandle = session.StringHandle, BtClientStatus = ByteString.CopyFrom(clientStatus) },
connection.Metadata, RegistrationDeadline(), cancellationToken));
// Records 11-16: 6 system-parameter queries before RTag2.
foreach (string parameterName in HistorianEventRegistrationProtocol.StatusParametersBeforeRegister)
byte[] registerBuffer = HistorianEventRegistrationProtocol.BuildRegisterCmEventInputBuffer();
try
{
GrpcHistory.RegisterTagsResponse rt = historyClient.RegisterTags(
new GrpcHistory.RegisterTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(registerBuffer) },
connection.Metadata, RegistrationDeadline(), cancellationToken);
RegistrationDiag += $"RTag={rt.Status?.BSuccess} e={Convert.ToHexString(rt.Status?.BtError?.ToByteArray() ?? [])}; ";
}
catch (Exception ex) { RegistrationDiag += $"RTag=EX:{ex.GetType().Name}; "; }
// gRPC CM_EVENT EnsureTags uses the 86-byte native format (8-byte header + the …2f27 event-type
// GUID), NOT the 2020 WCF CTagMetadata.
byte[] payload = HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc(DateTime.UtcNow);
try
{
GrpcHistory.EnsureTagsResponse et = historyClient.EnsureTags(
new GrpcHistory.EnsureTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(payload), ElementCount = 1 },
connection.Metadata, RegistrationDeadline(), cancellationToken);
RegistrationDiag += $"EnsT={et.Status?.BSuccess} e={Convert.ToHexString(et.Status?.BtError?.ToByteArray() ?? [])} out={Convert.ToHexString(et.BtTagStatus?.ToByteArray() ?? [])}; ";
}
catch (Exception ex) { RegistrationDiag += $"EnsT=EX:{ex.GetType().Name}; "; }
byte[] historianVersionRequest = HistorianEventRegistrationProtocol.BuildGetHistorianInfoRequest("HistorianVersion");
TryRun(() => statusClient.GetHistorianInfo(
new GrpcStatus.GetHistorianInfoRequest { StrHandle = session.StringHandle, BtRequest = ByteString.CopyFrom(historianVersionRequest) },
connection.Metadata, RegistrationDeadline(), cancellationToken));
string[] eventParams = ["AllowOriginals", "HistorianPartner", "HistorianVersion", "MaxCyclicStorageTimeout", "RealTimeWindow", "FutureTimeThreshold", "AllowRenameTags"];
foreach (string parameterName in eventParams)
{
TryRun(() => statusClient.GetSystemParameter(
new GrpcStatus.GetSystemParameterRequest { UiHandle = session.ClientHandle, StrParameterName = parameterName },
connection.Metadata, RegistrationDeadline(), cancellationToken));
}
byte[] registerBuffer = HistorianEventRegistrationProtocol.BuildRegisterCmEventInputBuffer();
TryRun(() => historyClient.RegisterTags(
new GrpcHistory.RegisterTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(registerBuffer) },
connection.Metadata, RegistrationDeadline(), cancellationToken));
// Record 18: one more system-parameter query after RTag2 before EnsT2.
TryRun(() => statusClient.GetSystemParameter(
new GrpcStatus.GetSystemParameterRequest { UiHandle = session.ClientHandle, StrParameterName = "AllowRenameTags" },
connection.Metadata, RegistrationDeadline(), cancellationToken));
// Records 19-21: cross-service version probes between RTag2 and EnsT2 (session-table registration).
TryRun(() => transactionClient.GetTransactionInterfaceVersion(new GrpcTransaction.GetTransactionInterfaceVersionRequest(), connection.Metadata, RegistrationDeadline(), cancellationToken));
TryRun(() => statusClient.GetStatusInterfaceVersion(new GrpcStatus.GetStatusInterfaceVersionRequest(), connection.Metadata, RegistrationDeadline(), cancellationToken));
TryRun(() => retrievalClient.GetRetrievalInterfaceVersion(new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), connection.Metadata, RegistrationDeadline(), cancellationToken));
byte[] payload = HistorianAddTagsProtocol.SerializeCmEventCTagMetadata(DateTime.UtcNow);
TryRun(() => historyClient.EnsureTags(
new GrpcHistory.EnsureTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(payload), ElementCount = 1 },
connection.Metadata, RegistrationDeadline(), cancellationToken));
}
private List<HistorianEvent> RunEventQuery(
/// <summary>Diagnostic: outcomes of the key CM_EVENT registration RPCs.</summary>
public string RegistrationDiag { get; private set; } = string.Empty;
// Spike seam (pending.md A1 broadening, Stage B0b): run ONLY the event query (StartEventQuery →
// GetNext loop → EndEventQuery) against an EXTERNALLY-supplied, already-opened + CM_EVENT-registered
// v8 Event connection + session — NO Create()/OpenSession/RegisterCmEventTag here. The per-call
// RunEventChain delegates to this so the per-call read and the B0b reuse spike share one query
// implementation (DRY). NOTE: event reads are otherwise GATED (C2) — the gRPC server long-polls
// GetNext to the no-data terminal and row-level retrieval is not yet verified over gRPC (see class
// remarks); the SEND seam is the spike's primary reuse signal. The split-channel opt-in
// (HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL) is preserved inside, unchanged.
internal List<HistorianEvent> RunEventQueryOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
DateTime startUtc,
@@ -261,16 +300,34 @@ internal sealed class HistorianGrpcEventOrchestrator
HistorianEventFilter? filter,
CancellationToken cancellationToken)
{
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
// HTTP/2-frame capture (grpc-event-query-capture.md #3) showed the stock client runs the event
// query on a DEDICATED RetrievalService TLS connection, separate from the HistoryService
// connection that opened+registered the session (correlated only by the session handle); our SDK
// collapses every service onto one connection. Opt in via HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL=1 to
// run StartEventQuery/GetNext/EndEventQuery on their own connection (mirrors native conn4: no
// re-handshake, just the existing handle), to test whether topology is the row-scoping gate.
bool splitChannel = string.Equals(
Environment.GetEnvironmentVariable("HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL"), "1", StringComparison.Ordinal);
HistorianGrpcConnection rconn = splitChannel ? HistorianGrpcChannelFactory.Create(_options) : connection;
try
{
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(rconn.Channel);
GrpcRetrieval.GetRetrievalInterfaceVersionResponse retrievalVersion = retrievalClient.GetRetrievalInterfaceVersion(
new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken);
new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), rconn.Metadata, Deadline(), cancellationToken);
HistorianServerVersionGate.Validate(HistorianServiceInterface.Retrieval, retrievalVersion.UiVersion, _options);
// Version 6 envelope: the stock 2023 R2 client sends v6 (the WCF path's v5 request is accepted
// here but is the legacy format). NECESSARY but not alone sufficient — live validation 2026-06-22
// showed rows still don't flow on v6 because the read also requires an EVENT-type connection
// (the stock client opens ConnectionType=Event; our OpenSession opens a Process-style 0x402
// session). See docs/reverse-engineering/grpc-event-query-capture.md "remaining gate".
IReadOnlyList<HistorianEventQueryAttempt> attempts = HistorianEventQueryProtocol.CreateStartEventQueryAttempts(
startUtc.ToUniversalTime(),
endUtc.ToUniversalTime(),
eventCount: 5,
filter);
eventCount: 100,
filter,
version: 6);
byte[] requestBuffer = attempts[0].RequestBuffer;
GrpcRetrieval.StartEventQueryResponse startResponse = retrievalClient.StartEventQuery(
@@ -280,7 +337,7 @@ internal sealed class HistorianGrpcEventOrchestrator
UiQueryRequestType = HistorianEventQueryProtocol.QueryRequestTypeEvent,
BtRequest = ByteString.CopyFrom(requestBuffer)
},
connection.Metadata,
rconn.Metadata,
Deadline(),
cancellationToken);
@@ -292,6 +349,7 @@ internal sealed class HistorianGrpcEventOrchestrator
}
uint queryHandle = startResponse.UiQueryHandle;
RegistrationDiag += $"QH={queryHandle} clientH={session.ClientHandle} strH={session.StringHandle}; ";
try
{
List<HistorianEvent> events = [];
@@ -304,7 +362,7 @@ internal sealed class HistorianGrpcEventOrchestrator
{
nextResponse = retrievalClient.GetNextEventQueryResultBuffer(
new GrpcRetrieval.GetNextEventQueryResultBufferRequest { UiHandle = session.ClientHandle, UiQueryHandle = queryHandle },
connection.Metadata,
rconn.Metadata,
EventPollDeadline(),
cancellationToken);
}
@@ -327,6 +385,8 @@ internal sealed class HistorianGrpcEventOrchestrator
LastResultBufferLength = resultBuffer.Length;
LastErrorBufferDescription = HistorianEventRegistrationProtocol.DescribeNativeError(errorBuffer);
LastResultBufferHex = Convert.ToHexString(resultBuffer.Length <= 48 ? resultBuffer : resultBuffer[..48]);
LastErrorBufferHex = Convert.ToHexString(errorBuffer);
// Any 5-byte type=4 error is a soft terminal (code 30 NoMoreData is canonical; code
// 85 / 0x55 is the missing-registration signal seen on early runs). Mirror the WCF
@@ -355,7 +415,12 @@ internal sealed class HistorianGrpcEventOrchestrator
}
finally
{
EndEventQuerySafely(retrievalClient, connection, session.ClientHandle, queryHandle);
EndEventQuerySafely(retrievalClient, rconn, session.ClientHandle, queryHandle);
}
}
finally
{
if (splitChannel) { rconn.Dispose(); }
}
}
@@ -0,0 +1,165 @@
using Google.Protobuf;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
using GrpcHistory = ArchestrA.Grpc.Contract.History;
using GrpcStatus = ArchestrA.Grpc.Contract.Status;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// 2023 R2 gRPC orchestrator for the event SEND (<see cref="HistorianClient.SendEventAsync"/>).
/// Captured live from the native 2023 R2 client (<c>capture-send-event</c> scenario,
/// 2026-06-23): an event send rides <c>HistoryService.AddStreamValues</c> with the SAME
/// <c>"OS"</c> (0x534F) storage-sample buffer the WCF AddS2 path uses
/// (<see cref="HistorianEventWriteProtocol"/>) — NOT a distinct event RPC and NOT the historical
/// write's <c>"ON"</c> buffer. The native client's write-enabled Event <c>OpenConnection</c>
/// request is byte-identical to the read-only event open (the ReadOnly arg does not change the v8
/// open buffer; diffed live — only the per-session client key + credential token differ), so the
/// existing <see cref="HistorianGrpcHandshake.OpenSession"/> event path is reused unchanged. The
/// chain on a single Event session:
/// <list type="number">
/// <item>OpenConnection (v8 Event, ExchangeKey ECDH auth) → string storage handle</item>
/// <item>CM_EVENT registration: UpdateClientStatus → RegisterTags → EnsureTags (the same
/// buffers the gRPC event READ replays — verified byte-identical to the capture)</item>
/// <item><c>HistoryService.AddStreamValues</c>(strHandle, "OS" event buffer)</item>
/// </list>
/// Only original events (<see cref="HistorianEvent.RevisionVersion"/> = 0) with string-valued
/// properties have a captured encoding; others throw <see cref="ProtocolEvidenceMissingException"/>
/// from <see cref="HistorianEventWriteProtocol"/>.
/// </summary>
internal sealed class HistorianGrpcEventWriteOrchestrator
{
private readonly HistorianClientOptions _options;
public HistorianGrpcEventWriteOrchestrator(HistorianClientOptions options)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
}
/// <summary>Diagnostic: type+code description of the most recent AddStreamValues error buffer.</summary>
public string LastSendErrorDescription { get; private set; } = string.Empty;
/// <summary>Diagnostic: outcomes of the CM_EVENT registration RPCs.</summary>
public string RegistrationDiag { get; private set; } = string.Empty;
public Task<bool> SendEventAsync(HistorianEvent evt, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(evt);
if (!_options.IntegratedSecurity && string.IsNullOrEmpty(_options.UserName))
{
throw new ProtocolEvidenceMissingException(
"Managed gRPC event send currently requires IntegratedSecurity or an explicit UserName + Password.");
}
if (evt.RevisionVersion != 0)
{
throw new ProtocolEvidenceMissingException(
"Only original events (RevisionVersion = 0) have a captured send encoding; " +
"revision/update/delete event sends are not yet supported.");
}
return Task.Run(() => Run(evt, cancellationToken), cancellationToken);
}
private bool Run(HistorianEvent evt, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
// The event SEND uses the same v8 Event connection as the event READ. The write-enabled
// open buffer is byte-identical to the read-only one (verified live), so OpenSession's
// event path is reused unchanged. Per-call: open + register + send on a fresh session.
HistorianGrpcHandshake.Session session = OpenAndRegisterEventSession(connection, cancellationToken);
return SendEventOnSession(connection, session, evt, cancellationToken);
}
// Spike seam (pending.md A1 broadening, Stage B0b): open a v8 Event connection and drive the
// CM_EVENT registration ONCE, returning the warm (connection, session). The per-call Run() uses
// it for a single send; the B0b reuse spike calls it once and then issues MULTIPLE
// SendEventOnSession ops against the returned session to measure whether a v8 Event session can
// be reused across sends (it has NEVER been proven reusable — that is exactly what B0b measures).
// The caller owns the connection's lifetime (dispose it).
internal HistorianGrpcHandshake.Session OpenAndRegisterEventSession(
HistorianGrpcConnection connection,
CancellationToken cancellationToken)
{
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(
connection, _options, cancellationToken, eventConnection: true);
RegisterCmEventTag(connection, session, cancellationToken);
return session;
}
// Spike seam (pending.md A1 broadening, Stage B0b): perform ONLY the event send against an
// EXTERNALLY-supplied, already-opened + CM_EVENT-registered v8 Event connection + session —
// i.e. NO Create(), NO OpenSession(eventConnection:true), NO RegisterCmEventTag inside it. The
// per-call Run() path delegates here so the per-call send and the B0b reuse-spike send share one
// implementation (DRY) and stay byte-identical. The spike drives this repeatedly on one warm
// session to measure whether the server honors a reused v8 Event session for multiple sends.
internal bool SendEventOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
HistorianEvent evt,
CancellationToken cancellationToken)
{
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
byte[] pBuf = HistorianEventWriteProtocol.SerializeAddStreamValuesBuffer(evt, DateTime.UtcNow);
GrpcHistory.AddStreamValuesResponse response = historyClient.AddStreamValues(
new GrpcHistory.AddStreamValuesRequest
{
StrHandle = session.StringHandle,
BtValues = ByteString.CopyFrom(pBuf),
},
connection.Metadata,
DateTime.UtcNow.Add(_options.RequestTimeout),
cancellationToken);
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
LastSendErrorDescription = HistorianEventRegistrationProtocol.DescribeNativeError(error);
return response.Status?.BSuccess ?? false;
}
/// <summary>
/// Replays the CM_EVENT registration the native event connection performs before a send:
/// UpdateClientStatus → RegisterTags(CM_EVENT) → EnsureTags(CM_EVENT). The buffers are shared
/// with the gRPC event READ path (<see cref="HistorianEventRegistrationProtocol"/> +
/// <see cref="HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc"/>) and were verified
/// byte-identical to the live capture. Best-effort: an individual rejection does not abort the
/// send (the server may already hold CM_EVENT registered for the session).
/// </summary>
private void RegisterCmEventTag(HistorianGrpcConnection connection, HistorianGrpcHandshake.Session session, CancellationToken cancellationToken)
{
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
DateTime Deadline() => DateTime.UtcNow.Add(_options.RequestTimeout);
byte[] clientStatus = HistorianEventRegistrationProtocol.BuildUpdateClientStatusBlob();
try
{
historyClient.UpdateClientStatus(
new GrpcHistory.UpdateClientStatusRequest { StrHandle = session.StringHandle, BtClientStatus = ByteString.CopyFrom(clientStatus) },
connection.Metadata, Deadline(), cancellationToken);
}
catch { /* best-effort */ }
byte[] registerBuffer = HistorianEventRegistrationProtocol.BuildRegisterCmEventInputBuffer();
try
{
GrpcHistory.RegisterTagsResponse rt = historyClient.RegisterTags(
new GrpcHistory.RegisterTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(registerBuffer) },
connection.Metadata, Deadline(), cancellationToken);
RegistrationDiag += $"RTag={rt.Status?.BSuccess}; ";
}
catch (Exception ex) { RegistrationDiag += $"RTag=EX:{ex.GetType().Name}; "; }
byte[] payload = HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc(DateTime.UtcNow);
try
{
GrpcHistory.EnsureTagsResponse et = historyClient.EnsureTags(
new GrpcHistory.EnsureTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(payload), ElementCount = 1 },
connection.Metadata, Deadline(), cancellationToken);
RegistrationDiag += $"EnsT={et.Status?.BSuccess}; ";
}
catch (Exception ex) { RegistrationDiag += $"EnsT=EX:{ex.GetType().Name}; "; }
}
}
@@ -1,3 +1,4 @@
using System.Security.Cryptography;
using Google.Protobuf;
using Grpc.Core;
using AVEVA.Historian.Client.Wcf;
@@ -20,6 +21,9 @@ namespace AVEVA.Historian.Client.Grpc;
/// </summary>
internal static class HistorianGrpcHandshake
{
/// <summary>Diagnostic: hex of the most recent v8 event-connection OpenConnection request.</summary>
internal static string LastEventOpenRequestHex { get; private set; } = string.Empty;
/// <summary>
/// The handles produced by a successful OpenConnection. <see cref="ClientHandle"/> is the
/// transient <c>uint</c> session token used by StartQuery/GetSystemParameter and the other
@@ -50,7 +54,8 @@ internal static class HistorianGrpcHandshake
HistorianGrpcConnection connection,
HistorianClientOptions options,
CancellationToken cancellationToken,
uint connectionMode = HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode)
uint connectionMode = HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode,
bool eventConnection = false)
{
DateTime Deadline() => DateTime.UtcNow.Add(options.RequestTimeout);
@@ -61,26 +66,65 @@ internal static class HistorianGrpcHandshake
new GrpcHistory.GetInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken);
HistorianServerVersionGate.Validate(HistorianServiceInterface.History, historyVersion.UiVersion, options);
var storageClient = new GrpcStorage.StorageService.StorageServiceClient(connection.Channel);
HistorianNativeHandshake.RunTokenRounds(
(handle, wrapped, _) =>
{
GrpcStorage.ValidateClientCredentialResponse response = storageClient.ValidateClientCredential(
new GrpcStorage.ValidateClientCredentialRequest { Handle = handle, InBuff = ByteString.CopyFrom(wrapped) },
connection.Metadata,
Deadline(),
cancellationToken);
byte[] serverOutput = response.OutBuff?.ToByteArray() ?? [];
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
bool success = response.Status?.BSuccess ?? false;
return new HistorianNativeHandshake.TokenExchangeResult(success, serverOutput, error);
},
contextKey,
options,
cancellationToken);
// The v6 (read/write) path authenticates via StorageService.ValidateClientCredential (Negotiate).
// The v8 EVENT path authenticates entirely via ExchangeKey (ECDH) + the RC4 credential token —
// the native client does NOT run ValidateClientCredential for an event connection, and doing so
// establishes a different session scope under which the event query returns zero rows. So skip it.
if (!eventConnection)
{
var storageClient = new GrpcStorage.StorageService.StorageServiceClient(connection.Channel);
HistorianNativeHandshake.RunTokenRounds(
(handle, wrapped, _) =>
{
GrpcStorage.ValidateClientCredentialResponse response = storageClient.ValidateClientCredential(
new GrpcStorage.ValidateClientCredentialRequest { Handle = handle, InBuff = ByteString.CopyFrom(wrapped) },
connection.Metadata,
Deadline(),
cancellationToken);
byte[] serverOutput = response.OutBuff?.ToByteArray() ?? [];
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
bool success = response.Status?.BSuccess ?? false;
return new HistorianNativeHandshake.TokenExchangeResult(success, serverOutput, error);
},
contextKey,
options,
cancellationToken);
}
byte[] open2Request = HistorianNativeHandshake.BuildOpenConnection3Request(
options.Host, contextKey, connectionMode);
// Event reads require an Event-type connection (ConnectionType=Event), which only the native
// v8 OpenConnection format carries — the v6 buffer has no such field. The v8 path authenticates
// via HistoryService.ExchangeKey (P-256 ECDH): the shared secret -> SHA256 = the client key, and
// the v8 credential token = RC4(password-UTF16LE, key=MD5(clientKey)) (the native HistorianCrypto
// aahCryptV2 scheme). The server shares the secret and RC4-decrypts the token to validate the
// password. See docs/reverse-engineering/grpc-event-query-capture.md.
byte[] eventToken = [];
if (eventConnection)
{
using ECDiffieHellman ecdh = ECDiffieHellman.Create(ECCurve.NamedCurves.nistP256);
byte[] clientHello = HistorianNativeHandshake.BuildExchangeKeyClientHello(ecdh);
string xkHandle = contextKey.ToString("D").ToUpperInvariant();
GrpcHistory.ExchangeKeyResponse xk = historyClient.ExchangeKey(
new GrpcHistory.ExchangeKeyRequest { StrHandle = xkHandle, BtInput = ByteString.CopyFrom(clientHello) },
connection.Metadata,
Deadline(),
cancellationToken);
if (!(xk.Status?.BSuccess ?? false))
{
byte[] xkErr = xk.Status?.BtError?.ToByteArray() ?? [];
HistorianNativeError? xkDecoded = HistorianOpen2Protocol.TryReadNativeError(xkErr);
string xkAscii = new(xkErr.Where(b => b is >= 0x20 and < 0x7F).Select(b => (char)b).ToArray());
throw new InvalidOperationException(
$"gRPC ExchangeKey failed (errorLen={xkErr.Length}, native={xkDecoded?.Type}/{xkDecoded?.Code}, ascii='{xkAscii}').");
}
byte[] clientKey = HistorianNativeHandshake.DeriveExchangeKeyClientKey(ecdh, xk.BtOutput?.ToByteArray() ?? []);
eventToken = HistorianNativeHandshake.BuildExchangeKeyCredentialToken(clientKey, options.Password);
}
byte[] open2Request = eventConnection
? HistorianNativeHandshake.BuildEventOpenConnectionVersion8Request(contextKey, options.UserName, eventToken)
: HistorianNativeHandshake.BuildOpenConnection3Request(options.Host, contextKey, connectionMode);
if (eventConnection) { LastEventOpenRequestHex = Convert.ToHexString(open2Request); }
GrpcHistory.OpenConnectionResponse open2 = historyClient.OpenConnection(
new GrpcHistory.OpenConnectionRequest { BtConnectionRequest = ByteString.CopyFrom(open2Request) },
@@ -92,7 +136,11 @@ internal static class HistorianGrpcHandshake
if (!(open2.Status?.BSuccess ?? false))
{
byte[] err = open2.Status?.BtError?.ToByteArray() ?? [];
throw new InvalidOperationException($"gRPC OpenConnection failed (errorLen={err.Length}, responseLen={open2Response.Length}).");
HistorianNativeError? decoded = HistorianOpen2Protocol.TryReadNativeError(err);
string ascii = new(err.Where(b => b is >= 0x20 and < 0x7F).Select(b => (char)b).ToArray());
throw new InvalidOperationException(
$"gRPC OpenConnection failed (errorLen={err.Length}, responseLen={open2Response.Length}, " +
$"native={decoded?.Type}/{decoded?.Code}{(decoded?.Name is { } n ? $" {n}" : "")}, ascii='{ascii}').");
}
(uint clientHandle, Guid storageSessionId) = HistorianNativeHandshake.ParseOpenConnectionResponse(open2Response);
@@ -47,6 +47,24 @@ internal sealed class HistorianGrpcHistoricalWriteOrchestrator
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(
connection, _options, cancellationToken,
connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode);
return RunWriteOnSession(connection, session, tag, values, cancellationToken);
}
// Spike/Phase-1 seam (A1): run the historical write against an EXTERNALLY-supplied, already-
// authenticated write-enabled (0x401) connection + session — NO Create()/handshake here. Run()
// delegates so the per-call path and the reuse path share one write implementation (DRY).
internal bool RunWriteOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
string tag,
IReadOnlyList<HistorianHistoricalValue> values,
CancellationToken cancellationToken)
{
if (values.Count == 0)
{
return true;
}
string handle = session.StringHandle;
DateTime Deadline() => DateTime.UtcNow.Add(_options.RequestTimeout);
@@ -102,12 +102,45 @@ internal sealed class HistorianGrpcReadOrchestrator
}
}
// Spike/Phase-1 seam (pending.md A1): run a raw query against an EXTERNALLY-supplied, already-
// authenticated connection + client handle — i.e. NO Create()/handshake here. RunRawChain delegates
// to this so the per-call path and the reuse path share one query implementation (DRY). The handshake
// reuse-probe test drives this directly to measure whether the server honors a reused session.
internal List<HistorianSample> RunRawQueryOnSession(
HistorianGrpcConnection connection,
uint clientHandle,
string tag,
DateTime startUtc,
DateTime endUtc,
int maxValues,
CancellationToken cancellationToken)
{
HistorianDataQueryRequest request = HistorianWcfReadOrchestrator.BuildDataQueryRequest(tag, startUtc, endUtc, maxValues);
return RunQuery(connection, clientHandle, request, maxValues, cancellationToken);
}
private List<HistorianSample> RunRawChain(string tag, DateTime startUtc, DateTime endUtc, int maxValues, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
uint clientHandle = OpenAuthenticatedConnection(connection, cancellationToken);
HistorianDataQueryRequest request = HistorianWcfReadOrchestrator.BuildDataQueryRequest(tag, startUtc, endUtc, maxValues);
return RunQuery(connection, clientHandle, request, maxValues, cancellationToken);
return RunRawQueryOnSession(connection, clientHandle, tag, startUtc, endUtc, maxValues, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): run an aggregate query against an EXTERNALLY-supplied,
// already-authenticated connection + client handle — i.e. NO Create()/handshake here.
// RunAggregateChain delegates to this so the per-call path and the reuse path share one query
// implementation (DRY).
internal List<HistorianAggregateSample> RunAggregateQueryOnSession(
HistorianGrpcConnection connection,
uint clientHandle,
string tag,
DateTime startUtc,
DateTime endUtc,
RetrievalMode mode,
TimeSpan interval,
CancellationToken ct)
{
return RunAggregateQuery(connection, clientHandle, tag, startUtc, endUtc, mode, interval, ct);
}
private List<HistorianAggregateSample> RunAggregateChain(
@@ -115,23 +148,29 @@ internal sealed class HistorianGrpcReadOrchestrator
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
uint clientHandle = OpenAuthenticatedConnection(connection, cancellationToken);
return RunAggregateQuery(connection, clientHandle, tag, startUtc, endUtc, mode, interval, cancellationToken);
return RunAggregateQueryOnSession(connection, clientHandle, tag, startUtc, endUtc, mode, interval, cancellationToken);
}
private List<HistorianSample> RunAtTimeChain(string tag, IReadOnlyList<DateTime> timestampsUtc, CancellationToken cancellationToken)
// Spike/Phase-1 seam (pending.md A1): run an at-time query against an EXTERNALLY-supplied,
// already-authenticated connection + client handle — i.e. NO Create()/handshake here.
// RunAtTimeChain delegates to this so the per-call path and the reuse path share one
// implementation (DRY).
internal List<HistorianSample> RunAtTimeOnSession(
HistorianGrpcConnection connection,
uint clientHandle,
string tag,
IReadOnlyList<DateTime> timestampsUtc,
CancellationToken ct)
{
if (timestampsUtc.Count == 0)
{
return [];
}
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
uint clientHandle = OpenAuthenticatedConnection(connection, cancellationToken);
List<HistorianSample> results = new(timestampsUtc.Count);
foreach (DateTime ts in timestampsUtc)
{
cancellationToken.ThrowIfCancellationRequested();
ct.ThrowIfCancellationRequested();
DateTime tsUtc = ts.ToUniversalTime();
List<HistorianAggregateSample> aggregates = RunAggregateQuery(
connection,
@@ -141,7 +180,7 @@ internal sealed class HistorianGrpcReadOrchestrator
tsUtc + TimeSpan.FromTicks(1),
RetrievalMode.Interpolated,
TimeSpan.FromTicks(2),
cancellationToken);
ct);
if (aggregates.Count == 0)
{
@@ -163,6 +202,18 @@ internal sealed class HistorianGrpcReadOrchestrator
return results;
}
private List<HistorianSample> RunAtTimeChain(string tag, IReadOnlyList<DateTime> timestampsUtc, CancellationToken cancellationToken)
{
if (timestampsUtc.Count == 0)
{
return [];
}
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
uint clientHandle = OpenAuthenticatedConnection(connection, cancellationToken);
return RunAtTimeOnSession(connection, clientHandle, tag, timestampsUtc, cancellationToken);
}
private uint OpenAuthenticatedConnection(HistorianGrpcConnection connection, CancellationToken cancellationToken)
=> HistorianGrpcHandshake.OpenAuthenticatedConnection(connection, _options, cancellationToken);
@@ -20,6 +20,18 @@ namespace AVEVA.Historian.Client.Grpc;
/// write-enabled session and calls <c>AddNonStreamValuesBegin</c>, surfacing whatever the server
/// returns. It writes NO data — if Begin succeeds it immediately calls <c>AddNonStreamValuesEnd</c>
/// with <c>bCommit=false</c> to discard the transaction.
///
/// <para><b>Scope note (corrected 2026-06-23 after a 2023 R2 binary re-read).</b> Despite the type
/// name, this probes the <i>non-streamed ORIGINAL / backfill insert</i> capability
/// (<c>AddNonStreamValues</c>), which is a <b>distinct capability from a revision EDIT</b>
/// (overwriting an existing historized value with a new revision). The stock high-level client
/// reaches a revision edit via a separate native transaction trio
/// <c>HistorianClient.AddRevisionValuesBegin/AddRevisionValue/AddRevisionValuesEnd</c>
/// (<c>ArchestrA.HistorianAccess.AddRevisionValues</c>, REVISION_MODE ∈ InsertLatest/UpdateSingle/
/// UpdateMultiple). That trio has <b>NO corresponding RPC in the gRPC contract</b> (no "Revision"
/// message type exists in <c>Archestra.Grpc.Contract</c>) — it rides the native storage-engine
/// transaction channel only. So R4.2 revision edits remain unreachable over gRPC regardless of this
/// probe's outcome; this probe neither covers nor unblocks them.</para>
/// </summary>
internal sealed class HistorianGrpcRevisionProbe
{
@@ -28,7 +28,19 @@ internal static class HistorianGrpcStatusClient
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
uint clientHandle = HistorianGrpcHandshake.OpenAuthenticatedConnection(connection, options, cancellationToken);
return GetSystemParameterOnSession(connection, clientHandle, options, parameterName, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): run GetSystemParameter against an EXTERNALLY-supplied,
// already-authenticated connection + client handle — NO Create()/handshake here. GetSystemParameter
// delegates so the per-call path and the reuse path share one RPC implementation (DRY).
internal static string? GetSystemParameterOnSession(
HistorianGrpcConnection connection,
uint clientHandle,
HistorianClientOptions options,
string parameterName,
CancellationToken cancellationToken)
{
var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel);
GrpcStatus.GetSystemParameterResponse response = statusClient.GetSystemParameter(
new GrpcStatus.GetSystemParameterRequest { UiHandle = clientHandle, StrParameterName = parameterName },
@@ -82,23 +94,7 @@ internal static class HistorianGrpcStatusClient
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel);
GrpcStatus.GetHistorianConsoleStatusResponse response = statusClient.GetHistorianConsoleStatus(
new GrpcStatus.GetHistorianConsoleStatusRequest { StrHandle = session.StringHandle },
connection.Metadata,
DateTime.UtcNow.Add(options.RequestTimeout),
cancellationToken);
if (response.Status?.BSuccess ?? false)
{
// Measured: server reachable, storage console reporting normally → not-storing baseline.
return NotStoring(errorOccurred: false, error: null);
}
byte[] err = response.Status?.BtError?.ToByteArray() ?? [];
string detail = err.Length == 0 ? "GetHistorianConsoleStatus returned failure." : Convert.ToHexString(err);
return NotStoring(errorOccurred: true, error: $"GetHistorianConsoleStatus failed: {detail}");
return GetStoreForwardStatusOnSession(connection, session.StringHandle, options, NotStoring, cancellationToken);
}
catch (OperationCanceledException)
{
@@ -111,6 +107,36 @@ internal static class HistorianGrpcStatusClient
}
}
// Spike/Phase-1 seam (pending.md A1): run GetHistorianConsoleStatus against an EXTERNALLY-supplied,
// already-authenticated connection + string handle — NO Create()/handshake here. GetStoreForwardStatus
// delegates so the per-call path and the reuse path share one RPC implementation (DRY). The
// unreachable/auth-failure try/catch (which must also cover the handshake) stays with the per-call
// method; this seam runs only the RPC + result mapping against the supplied session.
internal static HistorianStoreForwardStatus GetStoreForwardStatusOnSession(
HistorianGrpcConnection connection,
string stringHandle,
HistorianClientOptions options,
Func<bool, string?, HistorianStoreForwardStatus> notStoring,
CancellationToken cancellationToken)
{
var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel);
GrpcStatus.GetHistorianConsoleStatusResponse response = statusClient.GetHistorianConsoleStatus(
new GrpcStatus.GetHistorianConsoleStatusRequest { StrHandle = stringHandle },
connection.Metadata,
DateTime.UtcNow.Add(options.RequestTimeout),
cancellationToken);
if (response.Status?.BSuccess ?? false)
{
// Measured: server reachable, storage console reporting normally → not-storing baseline.
return notStoring(false, null);
}
byte[] err = response.Status?.BtError?.ToByteArray() ?? [];
string detail = err.Length == 0 ? "GetHistorianConsoleStatus returned failure." : Convert.ToHexString(err);
return notStoring(true, $"GetHistorianConsoleStatus failed: {detail}");
}
/// <summary>
/// Returns a <em>measured</em> connection status over the 2023 R2 gRPC transport (plan #5). Mirrors
/// <see cref="Wcf.HistorianWcfStatusClient"/>'s synthesize-from-handshake approach: it opens an
@@ -135,11 +161,7 @@ internal static class HistorianGrpcStatusClient
// A successful OpenConnection yields a non-empty storage-session GUID — proof the server and
// its storage session are reachable, the gRPC analog of the WCF handshake probe.
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
connected = session.StorageSessionId != Guid.Empty;
if (!connected)
{
error = "OpenConnection returned an empty storage-session handle.";
}
(connected, error) = EvaluateConnectionStatusOnSession(connection, session);
}
catch (OperationCanceledException)
{
@@ -162,6 +184,21 @@ internal static class HistorianGrpcStatusClient
ConnectionKind: HistorianConnectionKind.Process);
}
// Spike/Phase-1 seam (pending.md A1): evaluate connection status against an EXTERNALLY-supplied,
// already-authenticated connection + session — NO Create()/handshake here. GetConnectionStatus
// delegates so the per-call path and the reuse path share one evaluation (DRY). Unlike the other
// status seams there is no follow-on RPC: connectivity is derived entirely from the handshake's own
// storage-session GUID (a successful OpenConnection yields a non-empty GUID). The unreachable/auth
// try/catch (which must also cover the handshake) stays with the per-call method.
internal static (bool Connected, string? Error) EvaluateConnectionStatusOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session)
{
bool connected = session.StorageSessionId != Guid.Empty;
string? error = connected ? null : "OpenConnection returned an empty storage-session handle.";
return (connected, error);
}
/// <summary>
/// Reads the Historian server's system time-zone name (roadmap item R1.3,
/// <c>StatusService.GetSystemTimeZoneName</c>). Unlike the 2020 WCF surface — where the native
@@ -39,6 +39,27 @@ internal static class HistorianGrpcTagClient
private static HistorianTagMetadata? GetTagMetadata(HistorianClientOptions options, string tag, CancellationToken cancellationToken)
{
byte[] tagInfos = GetTagInfosRaw(options, [tag], cancellationToken);
return ParseTagMetadata(tagInfos);
}
// Spike/Phase-1 seam (pending.md A1): resolve tag metadata against an EXTERNALLY-supplied,
// already-authenticated connection + session — i.e. NO Create()/handshake here. The per-call
// GetTagMetadata and this seam share the parse tail (ParseTagMetadata) so neither duplicates the
// decode logic (DRY).
internal static HistorianTagMetadata? GetTagMetadataOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
string tag,
HistorianClientOptions options,
CancellationToken cancellationToken)
{
byte[] tagInfos = GetTagInfosRawOnSession(connection, session, [tag], options, cancellationToken);
return ParseTagMetadata(tagInfos);
}
// Shared parse tail for both the per-call GetTagMetadata and the reuse-path GetTagMetadataOnSession.
private static HistorianTagMetadata? ParseTagMetadata(byte[] tagInfos)
{
if (tagInfos.Length < 4)
{
return null;
@@ -69,7 +90,19 @@ internal static class HistorianGrpcTagClient
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
return GetTagInfosRawOnSession(connection, session, tags, options, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): issue GetTagInfosFromName against an EXTERNALLY-supplied,
// already-authenticated connection + session — i.e. NO Create()/handshake here. GetTagInfosRaw
// delegates to this so the per-call path and the reuse path share one query implementation (DRY).
internal static byte[] GetTagInfosRawOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
IReadOnlyList<string> tags,
HistorianClientOptions options,
CancellationToken cancellationToken)
{
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
byte[] requestBuffer = BuildTagNamesBuffer(tags);
GrpcRetrieval.GetTagInfosFromNameResponse response = retrievalClient.GetTagInfosFromName(
@@ -112,6 +145,8 @@ internal static class HistorianGrpcTagClient
return Task.Run(() => GetTagExtendedProperties(options, tag, cancellationToken), cancellationToken);
}
// No …OnSession seam: extended-properties browse stays per-call (not amortized through the session
// pool — out of A1-broadening scope). Add a seam here only if the pool ever needs to route it.
/// <summary>
/// Issues a single page-0 <c>GetTagExtendedPropertiesFromName</c> call and returns the raw native
/// <c>btTeps</c> response buffer (empty when the server reports no rows / non-success). Internal so
@@ -222,6 +257,20 @@ internal static class HistorianGrpcTagClient
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
return BrowseTagNamesOnSession(connection, session, filter, options, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): drive StartTagQuery → paged QueryTag → EndTagQuery against an
// EXTERNALLY-supplied, already-authenticated connection + session — i.e. NO Create()/handshake here.
// BrowseTagNames delegates to this so the per-call path and the reuse path share one browse
// implementation (DRY).
internal static List<string> BrowseTagNamesOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
string filter,
HistorianClientOptions options,
CancellationToken cancellationToken)
{
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
DateTime Deadline() => DateTime.UtcNow.Add(options.RequestTimeout);
@@ -2,6 +2,7 @@ using Google.Protobuf;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
using GrpcHistory = ArchestrA.Grpc.Contract.History;
using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval;
namespace AVEVA.Historian.Client.Grpc;
@@ -55,7 +56,18 @@ internal sealed class HistorianGrpcTagWriteOrchestrator
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken, WriteEnabledConnectionMode);
return EnsureTagOnSession(connection, session, definition, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): run EnsureTags against an EXTERNALLY-supplied, already-
// authenticated write-enabled (0x401) connection + session — NO Create()/handshake here. EnsureTag
// delegates so the per-call path and the reuse path share one op implementation (DRY).
internal bool EnsureTagOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
HistorianTagDefinition definition,
CancellationToken cancellationToken)
{
byte[] payload = HistorianTagWriteProtocol.SerializeAnalogCTagMetadata(
tagName: definition.TagName,
description: definition.Description,
@@ -96,7 +108,18 @@ internal sealed class HistorianGrpcTagWriteOrchestrator
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken, WriteEnabledConnectionMode);
return DeleteTagOnSession(connection, session, tagName, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): run DeleteTags against an EXTERNALLY-supplied, already-
// authenticated write-enabled (0x401) connection + session — NO Create()/handshake here. DeleteTag
// delegates so the per-call path and the reuse path share one op implementation (DRY).
internal bool DeleteTagOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
string tagName,
CancellationToken cancellationToken)
{
// DeleteTags takes the transient uint client handle (not the string handle), per the WCF wire capture.
byte[] tagNames = HistorianTagWriteProtocol.SerializeDeleteTagNames([tagName]);
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
@@ -130,7 +153,20 @@ internal sealed class HistorianGrpcTagWriteOrchestrator
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken, WriteEnabledConnectionMode);
return AddTagExtendedPropertiesOnSession(connection, session, tagName, properties, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): run AddTagExtendedProperties against an EXTERNALLY-supplied,
// already-authenticated write-enabled (0x401) connection + session — NO Create()/handshake here.
// AddTagExtendedProperties delegates so the per-call path and the reuse path share one op
// implementation (DRY).
internal bool AddTagExtendedPropertiesOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
string tagName,
IReadOnlyList<HistorianTagExtendedProperty> properties,
CancellationToken cancellationToken)
{
byte[] inBuff = HistorianTagExtendedPropertyProtocol.SerializeAddRequest(tagName, properties);
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
GrpcHistory.AddTagExtendedPropertiesResponse response = historyClient.AddTagExtendedProperties(
@@ -146,6 +182,114 @@ internal sealed class HistorianGrpcTagWriteOrchestrator
return response.Status?.BSuccess ?? false;
}
/// <summary>Outcome of the <see cref="ProbeDeleteTagExtendedPropertiesAsync"/> single-channel delete probe.</summary>
/// <param name="Accepted">True if the server's <c>DelTep</c> returned success.</param>
/// <param name="ErrorDescription">Decoded native error (byte0 0x84 + LE code + facility/file/message) when rejected.</param>
/// <param name="TagInfoPrimeBytes">Bytes returned by the GetTgByNm prime (tag-identity working-set load).</param>
/// <param name="ExtPropPrimePages">GetTepByNm prime pages that returned success (extended-property working-set load).</param>
internal readonly record struct DeleteTagExtendedPropertiesProbeResult(
bool Accepted, string? ErrorDescription, int TagInfoPrimeBytes, int ExtPropPrimePages);
/// <summary>
/// <b>Reverse-engineering probe (not a public op).</b> Tests whether <c>DelTep</c>
/// (DeleteTagExtendedProperties) — server-blocked on the 2020 WCF transport — succeeds over gRPC.
/// The WCF failure is structural: the server's <c>CHistStorage::DeleteTagExtendedProperties</c>
/// resolves each property from a <i>per-connection working set</i> the native client populates by
/// multiplexing <c>GetTgByNm</c> + <c>GetTepByNm</c> + <c>DelTep</c> over ONE physical connection.
/// The WCF SDK uses a separate channel per service, so the prime and the delete never share a
/// connection and the working set is empty at delete time (SErrorException). Over gRPC every service
/// client is built on the SAME <see cref="HistorianGrpcConnection.Channel"/>, so this probe runs the
/// identical native sequence — GetTgByNm prime, GetTepByNm prime, then DelTep — on one write-enabled
/// (0x401) session/channel, to see whether the multiplexed channel satisfies the working-set check.
/// Returns the decoded outcome rather than throwing so the caller can record a positive or negative
/// result. See docs/reverse-engineering/wcf-add-tag-extended-properties.md §Delete.
/// </summary>
internal Task<DeleteTagExtendedPropertiesProbeResult> ProbeDeleteTagExtendedPropertiesAsync(
string tagName, IReadOnlyList<string> propertyNames, CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tagName);
ArgumentNullException.ThrowIfNull(propertyNames);
if (propertyNames.Count == 0)
{
throw new ArgumentException("At least one property name is required.", nameof(propertyNames));
}
return Task.Run(() => ProbeDeleteTagExtendedProperties(tagName, propertyNames, cancellationToken), cancellationToken);
}
private DeleteTagExtendedPropertiesProbeResult ProbeDeleteTagExtendedProperties(
string tagName, IReadOnlyList<string> propertyNames, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken, WriteEnabledConnectionMode);
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
DateTime Deadline() => DateTime.UtcNow.Add(_options.RequestTimeout);
// Prime 1 — GetTgByNm: load the tag identity into the storage session's working set (same channel).
int tagInfoPrimeBytes = 0;
try
{
GrpcRetrieval.GetTagInfosFromNameResponse tg = retrievalClient.GetTagInfosFromName(
new GrpcRetrieval.GetTagInfosFromNameRequest
{
StrHandle = session.StringHandle,
BtTagNames = ByteString.CopyFrom(HistorianGrpcTagClient.BuildTagNamesBuffer([tagName])),
UiSequence = 0
},
connection.Metadata, Deadline(), cancellationToken);
tagInfoPrimeBytes = tg.BtTagInfos?.Length ?? 0;
}
catch
{
// Best-effort prime; the delete still runs so the error buffer is captured.
}
// Prime 2 — GetTepByNm: load the tag's extended properties into the working set (same channel),
// exactly as the native register->read->delete sequence does on its single connection.
int extPropPrimePages = 0;
byte[] tepRequest = HistorianTagExtendedPropertyProtocol.SerializeRequest(tagName);
uint sequence = 0;
for (int page = 0; page < 64; page++)
{
cancellationToken.ThrowIfCancellationRequested();
GrpcRetrieval.GetTagExtendedPropertiesFromNameResponse tep = retrievalClient.GetTagExtendedPropertiesFromName(
new GrpcRetrieval.GetTagExtendedPropertiesFromNameRequest
{
StrHandle = session.StringHandle,
BtTagNames = ByteString.CopyFrom(tepRequest),
UiSequence = sequence
},
connection.Metadata, Deadline(), cancellationToken);
if (!(tep.Status?.BSuccess ?? false))
{
break;
}
extPropPrimePages++;
if (HistorianTagExtendedPropertyProtocol.ParseResponse(tep.BtTeps?.ToByteArray() ?? []).Count == 0)
{
break;
}
sequence = tep.UiSequence;
}
// DelTep on the SAME channel/session, while the priming reads are part of the same working set.
byte[] inBuff = HistorianTagExtendedPropertyProtocol.SerializeDeleteRequest(tagName, propertyNames);
GrpcHistory.DeleteTagExtendedPropertiesResponse delete = historyClient.DeleteTagExtendedProperties(
new GrpcHistory.DeleteTagExtendedPropertiesRequest
{
StrHandle = session.StringHandle,
BtInput = ByteString.CopyFrom(inBuff)
},
connection.Metadata, Deadline(), cancellationToken);
bool accepted = delete.Status?.BSuccess ?? false;
string? errorDescription = accepted
? null
: HistorianEventRegistrationProtocol.DescribeNativeError(delete.Status?.BtError?.ToByteArray() ?? []);
return new DeleteTagExtendedPropertiesProbeResult(accepted, errorDescription, tagInfoPrimeBytes, extPropPrimePages);
}
public Task<HistorianTagRenameResult> RenameTagsAsync(
IReadOnlyList<(string OldName, string NewName)> pairs, CancellationToken cancellationToken)
{
@@ -166,7 +310,18 @@ internal sealed class HistorianGrpcTagWriteOrchestrator
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken, WriteEnabledConnectionMode);
return RenameTagsOnSession(connection, session, pairs, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): run StartJob (rename) against an EXTERNALLY-supplied, already-
// authenticated write-enabled (0x401) connection + session — NO Create()/handshake here. RenameTags
// delegates so the per-call path and the reuse path share one op implementation (DRY).
internal HistorianTagRenameResult RenameTagsOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
IReadOnlyList<(string OldName, string NewName)> pairs,
CancellationToken cancellationToken)
{
byte[] jobBuffer = HistorianTagRenameProtocol.SerializeRenameJob(pairs);
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
GrpcHistory.StartJobResponse response = historyClient.StartJob(
+88 -10
View File
@@ -114,18 +114,24 @@ public sealed class HistorianClient : IAsyncDisposable
}
/// <summary>
/// Sends a single <see cref="HistorianEvent"/> to the Historian's built-in CM_EVENT tag
/// over the WCF event pipeline (Open2 event mode → CM_EVENT registration → AddS2). The
/// event is appended to the historian's event history and is readable back via
/// <see cref="ReadEventsAsync"/> / the <c>v_AlarmEventHistory2</c> view. Only original
/// events (<see cref="HistorianEvent.RevisionVersion"/> = 0) with string-valued properties
/// are supported; other property value types and revision/update/delete events throw
/// Sends a single <see cref="HistorianEvent"/> to the Historian's built-in CM_EVENT tag.
/// Over WCF this runs Open2 event mode → CM_EVENT registration → AddS2; over the 2023 R2
/// <see cref="HistorianTransport.RemoteGrpc"/> transport it runs the captured-equivalent
/// v8 Event OpenConnection → CM_EVENT registration → <c>HistoryService.AddStreamValues</c>
/// with the same "OS" event buffer (live-captured 2026-06-23 — the send rides the same RPC
/// and buffer as the WCF path, not a distinct event RPC). The event is appended to the
/// historian's event history and is readable back via <see cref="ReadEventsAsync"/> /
/// the <c>v_AlarmEventHistory2</c> view. Only original events
/// (<see cref="HistorianEvent.RevisionVersion"/> = 0) with string-valued properties are
/// supported; other property value types and revision/update/delete events throw
/// <see cref="ProtocolEvidenceMissingException"/> until their wire encoding is captured.
/// </summary>
public Task<bool> SendEventAsync(HistorianEvent historianEvent, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(historianEvent);
return new HistorianWcfEventOrchestrator(_options).SendEventAsync(historianEvent, cancellationToken);
return _options.Transport == HistorianTransport.RemoteGrpc
? new Grpc.HistorianGrpcEventWriteOrchestrator(_options).SendEventAsync(historianEvent, cancellationToken)
: new HistorianWcfEventOrchestrator(_options).SendEventAsync(historianEvent, cancellationToken);
}
/// <summary>
@@ -256,9 +262,14 @@ public sealed class HistorianClient : IAsyncDisposable
// golden-verified against a server-accepted buffer, but the SDK cannot yet make the 2020 server
// accept the delete: the server's CHistStorage::DeleteTagExtendedProperties consults a
// per-connection working set that the native client populates by multiplexing GetTepByNm and
// DelTep over ONE connection, which the SDK's per-service WCF channels don't reproduce. See the
// documented-but-blocked path in HistorianWcfTagWriteOrchestrator and
// docs/reverse-engineering/wcf-add-tag-extended-properties.md §Delete.
// DelTep over ONE connection, which the SDK's per-service WCF channels don't reproduce. The gRPC
// transport — where every service client shares ONE channel — was probed 2026-06-22 to test that
// multiplexing hypothesis (GetTgByNm + GetTepByNm prime then DelTep on one write-enabled session,
// HistorianGrpcTagWriteOrchestrator.ProbeDeleteTagExtendedPropertiesAsync): both primes succeed on
// the shared channel yet the server STILL rejects the delete (native code=1), so gRPC does not lift
// the wall either. The working set is evidently populated by the native client's in-process
// registration state, not the wire session. See the documented-but-blocked path in
// HistorianWcfTagWriteOrchestrator and docs/reverse-engineering/wcf-add-tag-extended-properties.md §Delete.
/// <summary>
/// Executes a SQL command against the Historian over the WCF <c>ExeC</c>/<c>GetR</c> ops and
@@ -336,6 +347,73 @@ public sealed class HistorianClient : IAsyncDisposable
: new HistorianWcfTagWriteOrchestrator(_options).RenameTagsAsync(pairs, cancellationToken);
}
/// <summary>
/// Opens a reusable authenticated <see cref="HistorianSession"/> over the 2023 R2 gRPC transport.
/// The caller owns the session and must dispose it. Reusing the session across ops amortizes the auth
/// handshake; the server idle-expires it in ~20-25s, so keep it warm (HistorianSession.PingAsync) or
/// re-open. RemoteGrpc only.
/// </summary>
public async Task<HistorianSession> OpenSessionAsync(HistorianSessionKind kind, CancellationToken cancellationToken = default)
{
if (_options.Transport != HistorianTransport.RemoteGrpc)
{
throw new ProtocolEvidenceMissingException(
"HistorianSession is only supported over the 2023 R2 RemoteGrpc transport.");
}
return await Task.Run(() =>
{
uint mode = kind == HistorianSessionKind.WriteEnabled
? HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode
: HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode;
Grpc.HistorianGrpcConnection connection = Grpc.HistorianGrpcChannelFactory.Create(_options);
try
{
Grpc.HistorianGrpcHandshake.Session session =
Grpc.HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken, connectionMode: mode);
return new HistorianSession(connection, session, _options, kind);
}
catch
{
connection.Dispose(); // don't leak the channel if the handshake fails
throw;
}
}, cancellationToken).ConfigureAwait(false);
}
/// <summary>
/// Opens a reusable v8 EVENT session (ECDH + RegisterCmEventTag ONCE) over the 2023 R2 gRPC
/// transport. The caller owns the session and must dispose it. Reusing the session across sends
/// amortizes the ECDH+register cost (~10-16×, spike-proven); the server idle-expires it in ~25s,
/// so keep it warm (HistorianEventSession.PingAsync) or re-open. For SendEvent amortization only —
/// event reads are gated (C2) and not exposed here. RemoteGrpc only.
/// </summary>
public async Task<HistorianEventSession> OpenEventSessionAsync(CancellationToken cancellationToken = default)
{
if (_options.Transport != HistorianTransport.RemoteGrpc)
{
throw new ProtocolEvidenceMissingException(
"HistorianEventSession is only supported over the 2023 R2 RemoteGrpc transport.");
}
return await Task.Run(() =>
{
Grpc.HistorianGrpcConnection connection = Grpc.HistorianGrpcChannelFactory.Create(_options);
try
{
var orch = new Grpc.HistorianGrpcEventWriteOrchestrator(_options);
Grpc.HistorianGrpcHandshake.Session session = orch.OpenAndRegisterEventSession(connection, cancellationToken);
return new HistorianEventSession(connection, session, _options);
}
catch
{
connection.Dispose(); // don't leak the channel if the handshake fails
throw;
}
}, cancellationToken).ConfigureAwait(false);
}
public ValueTask DisposeAsync()
{
return ValueTask.CompletedTask;
@@ -53,6 +53,25 @@ public sealed class HistorianClientOptions
/// </summary>
public string? ServerDnsIdentity { get; init; }
/// <summary>
/// Optional WCF "Via" address (e.g. <c>net.tcp://host:42568</c>). When set, the SDK's WCF
/// channel factories <b>connect</b> to this address while still addressing the SOAP message
/// <c>To</c> the logical endpoint built from <see cref="Host"/>/<see cref="Port"/>. Use this when
/// the Historian is reached through a port-forwarding tunnel or proxy whose local port differs
/// from the server's real service port: point <see cref="Host"/>/<see cref="Port"/> at the
/// server's real endpoint (so the server's WCF AddressFilter matches) and set this to the tunnel
/// endpoint. Has no effect on the gRPC transport. Default null (connect == address).
/// </summary>
public string? ConnectViaAddress { get; init; }
/// <summary>
/// Diagnostic override for the native OpenConnection mode the WCF event-read chain uses (default
/// <c>0x402</c>, read-only process). Set to e.g. <c>0x501</c> (event) or <c>0x401</c> (write-enabled)
/// to probe whether CM_EVENT registration / event-row retrieval needs a different connection type on a
/// 2023 R2 server. Null = the default read-only process mode. Intended for protocol investigation.
/// </summary>
public uint? EventReadConnectionModeOverride { get; init; }
/// <summary>
/// For <see cref="HistorianTransport.RemoteGrpc"/>: when true the channel uses TLS
/// (<c>https://</c>); when false it uses plaintext (<c>http://</c>). Matches the stock
@@ -0,0 +1,66 @@
using AVEVA.Historian.Client.Grpc;
using AVEVA.Historian.Client.Models;
namespace AVEVA.Historian.Client;
/// <summary>A live, reusable authenticated v8 EVENT session: holds one event gRPC connection + one
/// open+registered Event handle and runs SendEvent on it WITHOUT re-handshaking. Reuse amortizes the
/// ECDH+register cost (~10-16×, spike-proven). SendEvent only — event READS are gated (C2) and stay
/// per-call. Keep in sync with <see cref="HistorianSession"/> (the v6 sibling).</summary>
public sealed class HistorianEventSession : IAsyncDisposable
{
private readonly HistorianGrpcConnection _connection;
private readonly HistorianGrpcHandshake.Session _session;
private readonly HistorianClientOptions _options;
private int _disposed;
internal HistorianEventSession(
HistorianGrpcConnection connection, HistorianGrpcHandshake.Session session, HistorianClientOptions options)
{
_connection = connection;
_session = session;
_options = options;
}
/// <summary>Exposes the held event gRPC connection for internal callers (e.g. the round-trip test
/// verifying the keepalive op directly). Not part of the public surface.</summary>
internal HistorianGrpcConnection Connection => _connection;
/// <summary>Exposes the held open+registered Event session handle for internal callers (e.g. the
/// round-trip test verifying the keepalive op directly). Not part of the public surface.</summary>
internal HistorianGrpcHandshake.Session Session => _session;
/// <summary>Sends one event on the held (open+registered) v8 Event session.</summary>
public Task<bool> SendEventAsync(HistorianEvent evt, CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(evt);
if (evt.RevisionVersion != 0)
{
throw new ProtocolEvidenceMissingException(
"Only original events (RevisionVersion = 0) have a captured send encoding; " +
"revision/update/delete event sends are not yet supported.");
}
var orch = new HistorianGrpcEventWriteOrchestrator(_options);
return Task.Run(() => orch.SendEventOnSession(_connection, _session, evt, ct), ct);
}
/// <summary>Keepalive via a lightweight <c>GetSystemParameter</c> status read on the event session's
/// <see cref="HistorianGrpcHandshake.Session.ClientHandle"/> (the same status op the native pre-query
/// sequence issues against an authenticated Event session), under the server idle floor. Mirrors
/// <see cref="HistorianSession.PingAsync"/>. The op's effectiveness on a v8 Event handle is
/// live-verified by the round-trip test.</summary>
public Task PingAsync(CancellationToken ct = default)
=> Task.Run(() => HistorianGrpcStatusClient.GetSystemParameterOnSession(
_connection, _session.ClientHandle, _options, "HistorianVersion", ct), ct);
/// <summary>Disposes the underlying event connection (idempotent).</summary>
public ValueTask DisposeAsync()
{
if (Interlocked.Exchange(ref _disposed, 1) == 0)
{
_connection.Dispose();
}
return ValueTask.CompletedTask;
}
}
@@ -29,8 +29,9 @@ internal enum HistorianServiceInterface
/// <item>Retrieval (<c>Retr</c>) interface version = 4</item>
/// <item>Transaction (<c>Trx</c>) interface version = 2</item>
/// </list>
/// The Status (<c>Stat</c>) service's <c>GetInterfaceVersion</c> returns 0 (not a real
/// version), so the Status interface is validated for reachability only, never value.
/// The Status (<c>Stat</c>) service's <c>GetInterfaceVersion</c> is not a real version (0 on
/// 2020 WCF, 4 on 2023 R2 gRPC) — it carries no meaning for the byte serializers either way — so
/// the Status interface is validated for reachability only, never value.
///
/// A 2023 R2 gRPC server reports History interface version 12 even though it carries the
/// same proven 2020 native buffers. That value is captured and accepted (see
@@ -48,14 +49,19 @@ internal static class HistorianServerVersionGate
/// The 2023 R2 gRPC HistoryService reports interface version 12. It is buffer-compatible with
/// the 2020 version 11 — the OpenConnection3 v6 / token / DataQueryRequest / row buffers are
/// byte-identical — confirmed by a live end-to-end gRPC read against a real 2023 R2 server
/// (2026-06-21). So both 11 and 12 are accepted for History. (Retrieval reported 4, matching
/// the 2020 value, so it needs no widening.)
/// (2026-06-21). So both 11 and 12 are accepted for History.
///
/// Retrieval=4, Transaction=2, and Status UiError=0 are now confirmed captured live over the
/// 2023 R2 gRPC transport (2026-06-25, unauthenticated GetInterfaceVersion RPCs); see
/// <c>docs/reverse-engineering/grpc-interface-versions.md</c>. All captured values were already
/// accepted — no widening of <see cref="AcceptedVersions"/> was required.
/// </summary>
public const uint HistoryInterfaceVersionGrpc2023R2 = 12;
/// <summary>
/// True when the service interface reports a meaningful version that should be matched.
/// Status is reachability-only (its <c>GetInterfaceVersion</c> returns 0).
/// Status is reachability-only (its <c>GetInterfaceVersion</c> is not a real version —
/// 0 on 2020 WCF, 4 on 2023 R2 gRPC).
/// </summary>
public static bool IsValueGated(HistorianServiceInterface service) => service switch
{
@@ -0,0 +1,214 @@
using System.Runtime.CompilerServices;
using AVEVA.Historian.Client.Grpc;
using AVEVA.Historian.Client.Models;
namespace AVEVA.Historian.Client;
/// <summary>A live, reusable authenticated Historian session: holds one gRPC connection + one
/// OpenConnection handle and runs ops on them WITHOUT re-handshaking. Reuse across ops amortizes the
/// auth handshake. Idle-expires server-side in ~20-25s — callers keep it warm (PingAsync) or re-open.
/// Reads, browse/metadata, historical-write, tag-write and status; events are NOT exposed (separate channel+auth).</summary>
public sealed class HistorianSession : IAsyncDisposable
{
private readonly HistorianGrpcConnection _connection;
private readonly HistorianGrpcHandshake.Session _session;
private readonly HistorianClientOptions _options;
private int _disposed;
/// <summary>Whether this session was opened read-only or write-enabled (the Open2 connection mode).</summary>
public HistorianSessionKind Kind { get; }
internal HistorianSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
HistorianClientOptions options,
HistorianSessionKind kind)
{
_connection = connection;
_session = session;
_options = options;
Kind = kind;
}
// --- reads (mirror the orchestrators' async-enumerable wrapping; call the …OnSession seams,
// which take the transient uint client handle) ---
/// <summary>Raw values for <paramref name="tag"/> in [<paramref name="startUtc"/>,
/// <paramref name="endUtc"/>], capped at <paramref name="maxValues"/>, on the held session.</summary>
public async IAsyncEnumerable<HistorianSample> ReadRawAsync(
string tag,
DateTime startUtc,
DateTime endUtc,
int maxValues,
[EnumeratorCancellation] CancellationToken ct = default)
{
ct.ThrowIfCancellationRequested();
var orch = new HistorianGrpcReadOrchestrator(_options);
IReadOnlyList<HistorianSample> rows = await Task.Run(
() => orch.RunRawQueryOnSession(_connection, _session.ClientHandle, tag, startUtc, endUtc, maxValues, ct), ct)
.ConfigureAwait(false);
foreach (HistorianSample sample in rows)
{
ct.ThrowIfCancellationRequested();
yield return sample;
}
}
/// <summary>Aggregate values for <paramref name="tag"/> over [<paramref name="startUtc"/>,
/// <paramref name="endUtc"/>] in <paramref name="interval"/> buckets, using the supplied
/// retrieval <paramref name="mode"/>, on the held session.</summary>
public async IAsyncEnumerable<HistorianAggregateSample> ReadAggregateAsync(
string tag,
DateTime startUtc,
DateTime endUtc,
RetrievalMode mode,
TimeSpan interval,
[EnumeratorCancellation] CancellationToken ct = default)
{
ct.ThrowIfCancellationRequested();
var orch = new HistorianGrpcReadOrchestrator(_options);
IReadOnlyList<HistorianAggregateSample> rows = await Task.Run(
() => orch.RunAggregateQueryOnSession(_connection, _session.ClientHandle, tag, startUtc, endUtc, mode, interval, ct), ct)
.ConfigureAwait(false);
foreach (HistorianAggregateSample sample in rows)
{
ct.ThrowIfCancellationRequested();
yield return sample;
}
}
/// <summary>Interpolated values for <paramref name="tag"/> at each of the supplied
/// <paramref name="timestampsUtc"/>, on the held session.</summary>
public Task<IReadOnlyList<HistorianSample>> ReadAtTimeAsync(
string tag,
IReadOnlyList<DateTime> timestampsUtc,
CancellationToken ct = default)
{
var orch = new HistorianGrpcReadOrchestrator(_options);
return Task.Run<IReadOnlyList<HistorianSample>>(
() => orch.RunAtTimeOnSession(_connection, _session.ClientHandle, tag, timestampsUtc, ct), ct);
}
// --- browse / metadata (call the …OnSession seams, which take the full Session for the string handle) ---
/// <summary>Browses tag names matching <paramref name="filter"/> on the held session.</summary>
public async IAsyncEnumerable<string> BrowseTagNamesAsync(
string filter = "*",
[EnumeratorCancellation] CancellationToken ct = default)
{
ct.ThrowIfCancellationRequested();
List<string> names = await Task.Run(
() => HistorianGrpcTagClient.BrowseTagNamesOnSession(_connection, _session, filter, _options, ct), ct)
.ConfigureAwait(false);
foreach (string name in names)
{
ct.ThrowIfCancellationRequested();
yield return name;
}
}
/// <summary>Reads metadata for <paramref name="tag"/> on the held session (null if unknown).</summary>
public Task<HistorianTagMetadata?> GetTagMetadataAsync(string tag, CancellationToken ct = default)
=> Task.Run(() => HistorianGrpcTagClient.GetTagMetadataOnSession(_connection, _session, tag, _options, ct), ct);
// --- writes (the …OnSession seams take the full Session, since the historical write keys on the
// string handle + tag GUID and the tag-config ops mix string/uint handles) ---
/// <summary>Writes historical/backfill values for <paramref name="tag"/> on the held (write-enabled) session.</summary>
public Task<bool> AddHistoricalValuesAsync(
string tag,
IReadOnlyList<HistorianHistoricalValue> values,
CancellationToken ct = default)
{
var orch = new HistorianGrpcHistoricalWriteOrchestrator(_options);
return Task.Run(() => orch.RunWriteOnSession(_connection, _session, tag, values, ct), ct);
}
/// <summary>Ensures the tag described by <paramref name="definition"/> exists on the held (write-enabled) session.</summary>
public Task<bool> EnsureTagAsync(HistorianTagDefinition definition, CancellationToken ct = default)
{
var orch = new HistorianGrpcTagWriteOrchestrator(_options);
return Task.Run(() => orch.EnsureTagOnSession(_connection, _session, definition, ct), ct);
}
/// <summary>Deletes <paramref name="tagName"/> on the held (write-enabled) session.</summary>
public Task<bool> DeleteTagAsync(string tagName, CancellationToken ct = default)
{
var orch = new HistorianGrpcTagWriteOrchestrator(_options);
return Task.Run(() => orch.DeleteTagOnSession(_connection, _session, tagName, ct), ct);
}
/// <summary>Renames the supplied (old,new) tag-name <paramref name="pairs"/> on the held (write-enabled) session.</summary>
public Task<HistorianTagRenameResult> RenameTagsAsync(
IReadOnlyList<(string OldName, string NewName)> pairs,
CancellationToken ct = default)
{
var orch = new HistorianGrpcTagWriteOrchestrator(_options);
return Task.Run(() => orch.RenameTagsOnSession(_connection, _session, pairs, ct), ct);
}
/// <summary>Adds extended <paramref name="properties"/> to <paramref name="tagName"/> on the held (write-enabled) session.</summary>
public Task<bool> AddTagExtendedPropertiesAsync(
string tagName,
IReadOnlyList<HistorianTagExtendedProperty> properties,
CancellationToken ct = default)
{
var orch = new HistorianGrpcTagWriteOrchestrator(_options);
return Task.Run(() => orch.AddTagExtendedPropertiesOnSession(_connection, _session, tagName, properties, ct), ct);
}
// --- status + keepalive (the status seams are static; handle shape differs per op) ---
/// <summary>Reads the named system parameter (e.g. "HistorianVersion") on the held session.</summary>
public Task<string?> GetSystemParameterAsync(string parameterName, CancellationToken ct = default)
=> Task.Run(() => HistorianGrpcStatusClient.GetSystemParameterOnSession(
_connection, _session.ClientHandle, _options, parameterName, ct), ct);
/// <summary>Reports connection status derived from the held session's storage handle (no follow-on RPC).</summary>
public Task<HistorianConnectionStatus> GetConnectionStatusAsync(CancellationToken ct = default)
=> Task.Run(() =>
{
(bool connected, string? error) = HistorianGrpcStatusClient.EvaluateConnectionStatusOnSession(_connection, _session);
return new HistorianConnectionStatus(
ServerName: _options.Host,
Pending: false,
ErrorOccurred: !connected,
Error: error,
ConnectedToServer: connected,
ConnectedToServerStorage: connected,
ConnectedToStoreForward: false,
ConnectionKind: HistorianConnectionKind.Process);
}, ct);
/// <summary>Reports a measured store-forward status (GetHistorianConsoleStatus) on the held session.</summary>
public Task<HistorianStoreForwardStatus> GetStoreForwardStatusAsync(CancellationToken ct = default)
=> Task.Run(() =>
{
HistorianStoreForwardStatus NotStoring(bool errorOccurred, string? error) => new(
ServerName: _options.Host,
Pending: false,
ErrorOccurred: errorOccurred,
Error: error,
DataStored: false,
Storing: false,
ConnectionKind: HistorianConnectionKind.Process);
return HistorianGrpcStatusClient.GetStoreForwardStatusOnSession(
_connection, _session.StringHandle, _options, NotStoring, ct);
}, ct);
/// <summary>Keeps the session warm against the server's ~20-25s idle expiry by issuing a cheap
/// system-parameter read. Call periodically (or before a latency-sensitive op) to avoid re-handshaking.</summary>
public Task PingAsync(CancellationToken ct = default) => GetSystemParameterAsync("HistorianVersion", ct);
/// <summary>Disposes the underlying gRPC connection (closes the channel). The server-side session
/// also idle-expires on its own; this releases the local channel resources immediately.</summary>
public ValueTask DisposeAsync()
{
if (Interlocked.Exchange(ref _disposed, 1) == 0)
{
_connection.Dispose();
}
return ValueTask.CompletedTask;
}
}
@@ -0,0 +1,5 @@
namespace AVEVA.Historian.Client;
/// <summary>Connection mode for an authenticated session. WriteEnabled (0x401) is a superset that
/// also serves reads (live-verified 2026-06-25); ReadOnly (0x402) is read-only.</summary>
public enum HistorianSessionKind { ReadOnly, WriteEnabled }
@@ -44,6 +44,40 @@ internal static class HistorianAddTagsProtocol
/// </remarks>
public static readonly Guid CommonArchestraEventTypeId = new("5f59ae42-3bb6-4760-91a5-ab0be01f9f02");
/// <summary>
/// The CM_EVENT event-type GUID used by the 2023 R2 <b>gRPC</b> EnsureTags (captured ending
/// <c>…e0 1f 2f 27</c>) — distinct from the 2020 WCF capture's <see cref="CommonArchestraEventTypeId"/>
/// (<c>…9f02</c>).
/// </summary>
public static readonly Guid CommonArchestraEventTypeIdGrpc = new("5f59ae42-3bb6-4760-91a5-ab0be01f2f27");
/// <summary>
/// Builds the native 2023 R2 <b>gRPC</b> CM_EVENT <c>EnsureTags.tagInfos</c> buffer (86 bytes,
/// captured byte-for-byte). Differs from the 2020 WCF <see cref="SerializeCmEventCTagMetadata"/>:
/// it is wrapped in an 8-byte EnsureTags header (<c>4E 67 03 00 01 00 00 00</c>), uses the
/// <see cref="CommonArchestraEventTypeIdGrpc"/> event-type GUID, and has no trailing bytes after it.
/// Used by the gRPC event registration so the server actually establishes CM_EVENT and the event
/// query returns rows.
/// </summary>
public static byte[] SerializeCmEventEnsureTagsGrpc(DateTime createdUtc)
{
using MemoryStream stream = new();
using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true);
writer.Write(new byte[] { 0x4E, 0x67, 0x03, 0x00, 0x01, 0x00, 0x00, 0x00 }); // EnsureTags header (count 1)
writer.Write((byte)3);
writer.Write((ushort)0x0086);
writer.Write((byte)5);
writer.Write(CmEventTagId.ToByteArray());
WriteCompressedHistorianString(writer, "CM_EVENT");
WriteCompressedHistorianString(writer, "AnE Event");
writer.Write(new byte[] { 0x02, 0x02, 0x01, 0x00, 0x00, 0x00, 0x01 });
writer.Write(0u);
writer.Write(createdUtc.ToUniversalTime().ToFileTimeUtc());
writer.Write(CommonArchestraEventTypeIdGrpc.ToByteArray());
return stream.ToArray();
}
public static byte[] SerializeCmEventCTagMetadata(DateTime createdUtc)
{
using MemoryStream stream = new();
@@ -8,22 +8,31 @@ internal static class HistorianEventQueryProtocol
{
public const ushort QueryRequestTypeEvent = 3;
/// <summary>
/// Builds the <c>StartEventQuery</c> <c>pRequestBuff</c>. <paramref name="version"/> selects the
/// envelope revision: <b>5</b> (default) is the native 2020 WCF format used by the WCF event
/// orchestrator; <b>6</b> is the 2023 R2 gRPC format. The two envelopes are byte-identical except
/// the leading version word and a 5-byte trailing zero pad — captured 2026-06-22 from the stock
/// 2023 R2 client (see <c>docs/reverse-engineering/grpc-event-query-capture.md</c>). The 2023 R2
/// server returns rows only for v6; v5 is accepted (StartEventQuery succeeds) but matches no rows.
/// The filter block in the middle is unchanged across versions.
/// </summary>
public static IReadOnlyList<HistorianEventQueryAttempt> CreateStartEventQueryAttempts(
DateTime startUtc, DateTime endUtc, uint eventCount, HistorianEventFilter? filter = null)
DateTime startUtc, DateTime endUtc, uint eventCount, HistorianEventFilter? filter = null, ushort version = 5)
{
List<HistorianEventQueryAttempt> attempts = [];
attempts.Add(CreateNativeFilterAttempt(startUtc, endUtc, eventCount, filter));
attempts.Add(CreateNativeFilterAttempt(startUtc, endUtc, eventCount, filter, version));
return attempts;
}
private static HistorianEventQueryAttempt CreateNativeFilterAttempt(
DateTime startUtc, DateTime endUtc, uint eventCount, HistorianEventFilter? filter)
DateTime startUtc, DateTime endUtc, uint eventCount, HistorianEventFilter? filter, ushort version)
{
using MemoryStream stream = new();
using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true);
writer.Write((ushort)5);
writer.Write(version);
writer.Write(startUtc.ToFileTimeUtc());
writer.Write(endUtc.ToFileTimeUtc());
writer.Write(eventCount);
@@ -43,10 +52,19 @@ internal static class HistorianEventQueryProtocol
WriteMetadataNamespace(writer);
writer.Write(0u);
// Version 6 (2023 R2 gRPC) appends a 5-byte trailing zero pad after the v5 terminal — the only
// envelope delta from v5 besides the version word. Captured live: the v6 buffer is the v5 buffer
// (byte 0 = 6) plus these 5 bytes, and is the form the 2023 R2 server returns event rows for.
if (version >= 6)
{
writer.Write(0u);
writer.Write((byte)0);
}
byte[] request = stream.ToArray();
return new HistorianEventQueryAttempt(
filter is null ? "native-empty-filter-version5" : "native-filter-version5",
5,
filter is null ? $"native-empty-filter-version{version}" : $"native-filter-version{version}",
version,
request,
Convert.ToHexString(SHA256.HashData(request)).ToLowerInvariant());
}
@@ -10,10 +10,10 @@ namespace AVEVA.Historian.Client.Wcf;
/// native event read (instrument-wcf-readmessage record 24, two rows for Alarm.Set + Alarm.Clear):
///
/// <code>
/// UInt16 version = 9
/// UInt16 version = 9 (WCF) | 11 (2023 R2 gRPC)
/// UInt32 rowCount
/// UInt32 headerField = 0x1E // ONE buffer-level field (NOT a per-row marker)
/// rowCount × Row {
/// UInt32 rowMarker = 0x1E
/// UInt16 rowFormat = 7
/// Int64 eventTimeUtcFiletime
/// UInt16 × 8 // purpose unclear (slot offsets?)
@@ -42,10 +42,23 @@ namespace AVEVA.Historian.Client.Wcf;
internal static class HistorianEventRowProtocol
{
public const ushort EventRowProtocolVersion = 9;
public const uint RowMarker = 0x0000001Eu;
public const ushort RowFormatV9 = 7;
private const int HeaderSize = 6;
private const int RowFixedHeaderSize = 4 + 2 + 8 + 16;
/// <summary>
/// 2023 R2 gRPC returns the event-row buffer with container version <c>11</c> instead of the
/// 2020 WCF <c>9</c>. The row layout is otherwise <b>byte-identical</b> (verified against a captured
/// stock-client read: header <c>0B00 &lt;rowCount&gt; 1E000000</c> then markerless rows, 50
/// Alarm.Set/Alarm.Clear rows decoded clean to end-of-buffer; the WCF v9 capture has the same
/// <c>0900 &lt;rowCount&gt; 1E000000</c> header). Accept both, exactly as the interface-version gate
/// accepts History 11 and 12.
/// </summary>
public const ushort EventRowProtocolVersionGrpc = 11;
/// <summary>Constant buffer-level field following <c>rowCount</c> (observed <c>0x1E</c>). NOT a
/// per-row marker — it appears exactly once, before the first row.</summary>
public const uint BufferHeaderField = 0x0000001Eu;
public const ushort RowFormat = 7;
private const int BufferHeaderSize = 2 + 4 + 4; // version + rowCount + headerField
private const int RowFixedHeaderSize = 2 + 8 + 16; // rowFormat + filetime + 8×UInt16 slots
private const byte ValueTypeBool = 0x02;
private const byte ValueTypeGuid = 0x10;
@@ -55,25 +68,26 @@ internal static class HistorianEventRowProtocol
public static IReadOnlyList<HistorianEvent> Parse(ReadOnlySpan<byte> buffer)
{
if (buffer.Length < HeaderSize)
if (buffer.Length < 6)
{
return [];
}
ushort version = BinaryPrimitives.ReadUInt16LittleEndian(buffer[..2]);
if (version != EventRowProtocolVersion)
if (version != EventRowProtocolVersion && version != EventRowProtocolVersionGrpc)
{
return [];
}
uint rowCount = BinaryPrimitives.ReadUInt32LittleEndian(buffer.Slice(2, 4));
if (rowCount == 0)
if (rowCount == 0 || buffer.Length < BufferHeaderSize)
{
return [];
}
List<HistorianEvent> events = new(checked((int)rowCount));
int cursor = HeaderSize;
// Skip the single buffer-level header field (0x1E); rows follow with NO per-row marker.
int cursor = BufferHeaderSize;
for (uint rowIndex = 0; rowIndex < rowCount; rowIndex++)
{
if (!TryReadRow(buffer, ref cursor, out HistorianEvent? row))
@@ -95,19 +109,13 @@ internal static class HistorianEventRowProtocol
return false;
}
uint marker = BinaryPrimitives.ReadUInt32LittleEndian(buffer.Slice(cursor, 4));
if (marker != RowMarker)
ushort format = BinaryPrimitives.ReadUInt16LittleEndian(buffer.Slice(cursor, 2));
if (format != RowFormat)
{
return false;
}
ushort format = BinaryPrimitives.ReadUInt16LittleEndian(buffer.Slice(cursor + 4, 2));
if (format != RowFormatV9)
{
return false;
}
long filetime = BinaryPrimitives.ReadInt64LittleEndian(buffer.Slice(cursor + 6, 8));
long filetime = BinaryPrimitives.ReadInt64LittleEndian(buffer.Slice(cursor + 2, 8));
DateTime eventTimeUtc = DateTime.FromFileTimeUtc(filetime);
int afterFixedHeader = cursor + RowFixedHeaderSize;
@@ -29,9 +29,12 @@ namespace AVEVA.Historian.Client.Wcf;
/// +0x2A value bytes, native width by tag type:
/// Float → Float32(4) · Double → Float64(8) · Int2 → Int16(2)
/// Int4 → Int32(4) · UInt4 → UInt32(4)
/// Int8 → Int64(8) · UInt8 → UInt64(8)
/// </code>
///
/// Captured for the five analog types <c>EnsureTagAsync</c> supports (Float/Double/Int2/Int4/UInt4).
/// Live-captured for Float/Double/Int2/Int4/UInt4; Int8/UInt8 mirror the captured
/// Double value layout (8 LE bytes) and are live-proven (2026-06-25). UInt1 is re-gated:
/// the historian accepts EnsureTags(UInt1) but stores a degenerate tag — writes would fail.
/// Other tag types have no captured value encoding and are rejected.
/// </remarks>
internal static class HistorianHistoricalWriteProtocol
@@ -52,7 +55,10 @@ internal static class HistorianHistoricalWriteProtocol
/// <paramref name="tagGuid"/> is the per-tag GUID (from the gRPC tag-info read),
/// <paramref name="dataType"/> is the tag's declared analog type (selects the value width), and
/// <paramref name="receivedTimeUtc"/> is the storage/received timestamp the orchestrator stamps.
/// Throws <see cref="ProtocolEvidenceMissingException"/> for tag types without a captured encoding.
/// Throws <see cref="ProtocolEvidenceMissingException"/> for tag types without a captured encoding
/// (including UInt1, which is re-gated — the historian creates a degenerate UInt1 analog tag).
/// Int8/UInt8 carry exact magnitude only up to 2^53 — the value API is <see langword="double"/>;
/// full 64-bit range is a separate follow-on.
/// </summary>
public static byte[] SerializeAddStreamValuesBuffer(
Guid tagGuid,
@@ -117,10 +123,22 @@ internal static class HistorianHistoricalWriteProtocol
BinaryPrimitives.WriteUInt32LittleEndian(b, checked((uint)value));
return b;
}
case HistorianDataType.Int8:
{
byte[] b = new byte[8];
BinaryPrimitives.WriteInt64LittleEndian(b, checked((long)value));
return b;
}
case HistorianDataType.UInt8:
{
byte[] b = new byte[8];
BinaryPrimitives.WriteUInt64LittleEndian(b, checked((ulong)value));
return b;
}
default:
throw new ProtocolEvidenceMissingException(
$"AddHistoricalValuesAsync has no captured value encoding for tag data type '{dataType}'. " +
"Captured types: Float, Double, Int2, Int4, UInt4.");
"Captured types: Float, Double, Int2, Int4, UInt4, Int8, UInt8.");
}
}
@@ -1,5 +1,6 @@
using System.Buffers.Binary;
using System.Diagnostics;
using System.Security.Cryptography;
namespace AVEVA.Historian.Client.Wcf;
@@ -28,6 +29,111 @@ internal static class HistorianNativeHandshake
/// <summary>Result of one transport-level credential-token exchange.</summary>
internal readonly record struct TokenExchangeResult(bool Success, byte[] ServerOutput, byte[] Error);
// HistoryService.ExchangeKey wire format (decoded from a live 2023 R2 capture): the ASCII magic
// "ECK1", a uint32 coordinate length (32 = P-256), then the raw EC public-key point X || Y. Used by
// the v8 (event) connection path to establish the client key the v8 OpenConnection requires.
private static readonly byte[] ExchangeKeyMagic = "ECK1"u8.ToArray();
private const int ExchangeKeyCoordinateBytes = 32; // P-256 / secp256r1
/// <summary>
/// Builds the <c>ExchangeKey</c> client hello: <c>"ECK1" + u32(32) + X(32) + Y(32)</c> for the
/// supplied P-256 key. Pure managed (<see cref="ECDiffieHellman"/>); no native AVEVA dependency.
/// </summary>
public static byte[] BuildExchangeKeyClientHello(ECDiffieHellman ecdh)
{
ECParameters parameters = ecdh.ExportParameters(includePrivateParameters: false);
byte[] x = LeftPadCoordinate(parameters.Q.X!);
byte[] y = LeftPadCoordinate(parameters.Q.Y!);
using MemoryStream stream = new();
using BinaryWriter writer = new(stream);
writer.Write(ExchangeKeyMagic);
writer.Write((uint)ExchangeKeyCoordinateBytes);
writer.Write(x);
writer.Write(y);
return stream.ToArray();
}
/// <summary>
/// Parses the server's <c>ExchangeKey</c> hello (same <c>"ECK1" + u32 + X + Y</c> shape) and
/// derives the 32-byte client key = <b>SHA256(ECDH shared secret)</b>. This matches the native
/// client, which uses <c>ECDiffieHellmanCng { KeyDerivationFunction=Hash, HashAlgorithm=SHA256 }
/// .DeriveKeyMaterial(...)</c> — i.e. the hash KDF over the raw agreement.
/// </summary>
public static byte[] DeriveExchangeKeyClientKey(ECDiffieHellman ecdh, byte[] serverHello)
{
const int headerLength = 8; // 4-byte magic + 4-byte coordinate length
int needed = headerLength + (2 * ExchangeKeyCoordinateBytes);
if (serverHello.Length < needed)
{
throw new InvalidOperationException($"ExchangeKey server hello too short (len={serverHello.Length}, need>={needed}).");
}
byte[] x = serverHello[headerLength..(headerLength + ExchangeKeyCoordinateBytes)];
byte[] y = serverHello[(headerLength + ExchangeKeyCoordinateBytes)..(headerLength + (2 * ExchangeKeyCoordinateBytes))];
ECParameters serverParameters = new()
{
Curve = ECCurve.NamedCurves.nistP256,
Q = new ECPoint { X = x, Y = y },
};
using ECDiffieHellman serverKey = ECDiffieHellman.Create(serverParameters);
return ecdh.DeriveKeyFromHash(serverKey.PublicKey, HashAlgorithmName.SHA256);
}
/// <summary>
/// Builds the v8 credential token: <c>RC4(password-UTF16LE, key = MD5(clientKey))</c>, where
/// <paramref name="clientKey"/> is the <see cref="DeriveExchangeKeyClientKey"/> result
/// (SHA256 of the ECDH secret). Reverse-engineered from the native <c>HistorianCrypto.NRC4_V2
/// .aahCryptV2</c> scheme (MD5-keyed RC4) and verified to reproduce a live-captured token exactly.
/// The server, sharing the ECDH secret, RC4-decrypts this to recover and validate the password.
/// Pure managed; nothing AVEVA is shipped.
/// </summary>
public static byte[] BuildExchangeKeyCredentialToken(byte[] clientKey, string password)
{
byte[] rc4Key = MD5.HashData(clientKey);
byte[] plaintext = System.Text.Encoding.Unicode.GetBytes(password ?? string.Empty);
return Rc4(rc4Key, plaintext);
}
internal static byte[] Rc4(byte[] key, byte[] data)
{
int[] s = new int[256];
for (int i = 0; i < 256; i++)
{
s[i] = i;
}
for (int i = 0, j = 0; i < 256; i++)
{
j = (j + s[i] + key[i % key.Length]) & 0xFF;
(s[i], s[j]) = (s[j], s[i]);
}
byte[] output = new byte[data.Length];
for (int i = 0, j = 0, n = 0; n < data.Length; n++)
{
i = (i + 1) & 0xFF;
j = (j + s[i]) & 0xFF;
(s[i], s[j]) = (s[j], s[i]);
output[n] = (byte)(data[n] ^ s[(s[i] + s[j]) & 0xFF]);
}
return output;
}
private static byte[] LeftPadCoordinate(byte[] coordinate)
{
if (coordinate.Length == ExchangeKeyCoordinateBytes)
{
return coordinate;
}
byte[] padded = new byte[ExchangeKeyCoordinateBytes];
Array.Copy(coordinate, 0, padded, ExchangeKeyCoordinateBytes - coordinate.Length, coordinate.Length);
return padded;
}
/// <summary>
/// Performs a single credential-token round on the wire. <paramref name="handle"/> is the
/// upper-case context-key GUID, <paramref name="wrappedToken"/> is the AVEVA-wrapped SSPI
@@ -134,6 +240,56 @@ internal static class HistorianNativeHandshake
credentialBlock: new byte[CredentialBlockSizeBytes]);
}
// v8 OpenConnection constants (2023 R2), decoded from the native Event-connection capture.
private const byte NativeConnectionTypeEvent = 1;
private const byte NativeConnectionFlagEvent = 1;
private const ushort NativeHcalVersionV8 = 18;
private const string ClientDataSourceIdV8 = "2023.1219.4004.5";
private const byte NativeClientCommonInfoFormatVersionV8 = 3;
private const int CredentialTokenSizeBytes = 26;
private static readonly Guid UnsetShardId = new(new byte[]
{
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
});
/// <summary>
/// Builds the native 2023 R2 <b>version-8</b> OpenConnection request for an <b>Event</b> connection
/// (<c>ConnectionType=Event</c>). The 2023 R2 server returns event rows only on an event connection,
/// and the v6 OpenConnection buffer has no ConnectionType field. The credential token is zeroed: the
/// session is authenticated by the preceding <c>ValidateClientCredential</c> handshake, exactly as
/// the v6 path already relies on (it sends a zeroed credential block). See
/// docs/reverse-engineering/grpc-event-query-capture.md.
/// </summary>
public static byte[] BuildEventOpenConnectionVersion8Request(Guid contextKey, string userName, byte[] credentialToken)
{
Process current = Process.GetCurrentProcess();
string machineName = Environment.MachineName;
string processName = string.IsNullOrEmpty(current.ProcessName) ? ClientNodeNameFallback : current.ProcessName;
HistorianClientCommonInfo commonInfo = new(
FormatVersion: NativeClientCommonInfoFormatVersionV8,
ServerNodeName: machineName,
ClientNodeName: processName,
ProcessId: checked((uint)current.Id),
HcalVersion: NativeHcalVersionV8,
ProcessName: string.Empty,
Proxy: string.Empty,
DataSourceId: ClientDataSourceIdV8,
ShardId: UnsetShardId,
ClientVersion: NativeClientVersionInt,
ClientTimestamp: (ulong)DateTime.UtcNow.ToFileTimeUtc(),
ClientDllVersion: string.Empty);
return HistorianOpen2Protocol.SerializeNativeOpenConnectionVersion8(
commonInfo,
contextKey,
NativeConnectionTypeEvent,
NativeConnectionFlagEvent,
userName ?? string.Empty,
credentialToken ?? []);
}
/// <summary>
/// Decodes the OpenConnection response blob: byte 0 = protocol version, bytes 1..4 =
/// transient /Retr client handle (UInt32 LE), bytes 5..20 = storage session GUID.
@@ -50,6 +50,49 @@ internal static class HistorianOpen2Protocol
return stream.ToArray();
}
/// <summary>
/// Builds the native 2023 R2 <b>version-8</b> OpenConnection request — the format the stock client
/// uses, which (unlike v6) carries a <paramref name="connectionType"/> byte (Event vs Process). The
/// tail is the same <see cref="HistorianClientCommonInfo"/> v6 emits; the head is the v8 layout
/// decoded from a live capture (docs/reverse-engineering/grpc-event-query-capture.md): version 8,
/// two marker bytes, the client-key GUID, a username <c>HistorianString</c>, a length-prefixed
/// credential token, then ClientType / ConnectionType / flag / a constant word / compact metadata
/// namespace / two empty strings. The credential token is normally an ECDH-derived blob in the
/// native cert path; this serializer sends it as supplied (zeroed for the Negotiate-auth path,
/// mirroring how the v6 request already sends a zeroed credential block and relies on the separate
/// ValidateClientCredential handshake for authentication).
/// </summary>
public static byte[] SerializeNativeOpenConnectionVersion8(
HistorianClientCommonInfo commonInfo,
Guid clientKey,
byte connectionType,
byte connectionFlag,
string userName,
byte[] credentialToken)
{
using MemoryStream stream = new();
using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true);
writer.Write((byte)8); // [0] format version 8
writer.Write((byte)0xF0); // [1] constant marker
writer.Write(new byte[19]); // [2..20] zero padding
writer.Write((byte)1); // [21] constant marker
writer.Write(clientKey.ToByteArray()); // [22..37] per-session client key
WriteHistorianString(writer, userName); // username (UTF-16 HistorianString)
writer.Write((ushort)credentialToken.Length); // credential-token length (u16)
writer.Write(credentialToken); // credential token (zeroed for the Negotiate path)
writer.Write((byte)4); // ClientType = 4
writer.Write(connectionType); // ConnectionType (Event = 1 / Process = 2)
writer.Write(connectionFlag); // type flag (Event = 1 / Process = 0)
writer.Write((ushort)3); // constant word observed at [97..98]
WriteCompactMetadataNamespace(writer, HistorianMetadataNamespace.Empty);
WriteHistorianString(writer, string.Empty);
WriteHistorianString(writer, string.Empty);
WriteClientCommonInfo(writer, commonInfo); // tail: FormatVersion 3 => no ClientDllVersion
writer.Write(0u); // trailing terminator
return stream.ToArray();
}
private static void WriteNativeOpenConnectionContent(
BinaryWriter writer,
HistorianOpen2Request request,
@@ -58,9 +58,13 @@ internal static class HistorianTagWriteProtocol
];
/// <summary>
/// Native CDataType wire codes per data type — captured 2026-05-04 by probing
/// every type via instrument-wcf-writemessage. Matches the codes already documented
/// in <see cref="HistorianWcfTagClient"/> MapDataType for the read path.
/// Native CDataType wire codes per data type — captured by probing every type via
/// instrument-wcf-writemessage. Matches the codes already documented in
/// <see cref="HistorianWcfTagClient"/> MapDataType for the read path; Int8/UInt8
/// reuse the same read-side codes (0x19/0x39).
/// UInt1 is excluded: live evidence shows the historian accepts EnsureTags(UInt1)
/// with success but stores a degenerate tag (null type descriptor) that subsequent
/// GetTagInfo cannot parse — not client-fixable on the analog path.
/// </summary>
public static byte GetAnalogDataTypeCode(Models.HistorianDataType dataType) => dataType switch
{
@@ -70,8 +74,10 @@ internal static class HistorianTagWriteProtocol
Models.HistorianDataType.UInt4 => 0x11,
Models.HistorianDataType.Int2 => 0x29,
Models.HistorianDataType.Int4 => 0x31,
Models.HistorianDataType.Int8 => 0x19,
Models.HistorianDataType.UInt8 => 0x39,
_ => throw new ProtocolEvidenceMissingException(
$"EnsureTagAsync data type {dataType} has no captured CTagMetadata wire code; supported: Float, Double, UInt2, UInt4, Int2, Int4."),
$"EnsureTagAsync data type {dataType} has no captured CTagMetadata wire code; supported: Float, Double, UInt2, UInt4, Int2, Int4, Int8, UInt8."),
};
private static readonly byte[] AnalogPadding16 = new byte[16];
@@ -118,8 +124,10 @@ internal static class HistorianTagWriteProtocol
/// <summary>
/// Serializes a CTagMetadata payload for an analog tag. Live-verified for Float,
/// Double, Int2, Int4, UInt4 — see <see cref="GetAnalogDataTypeCode"/> for the
/// type-code mapping. Output matches the byte-for-byte capture for the same inputs.
/// Double, Int2, Int4, UInt4; Int8/UInt8 supported via the read-side type codes
/// (pending live round-trip); UInt1 re-gated (historian creates a degenerate tag) —
/// see <see cref="GetAnalogDataTypeCode"/> for the type-code mapping. Output matches
/// the byte-for-byte capture for the same inputs.
/// When MinEU/MaxEU/MinRaw/MaxRaw are all defaults (0/100/0/100) emits the compact
/// `1A 03` scaling marker; otherwise emits `1F` + 4 doubles in order.
/// </summary>
@@ -41,7 +41,7 @@ internal static class HistorianWcfAuthChainHelper
try
{
IHistoryServiceContract2 historyChannel = historyFactory.CreateChannel();
IHistoryServiceContract2 historyChannel = HistorianWcfClientCredentialsHelper.CreateChannel(historyFactory, options);
ICommunicationObject historyChannelCo = (ICommunicationObject)historyChannel;
try
{
@@ -30,6 +30,24 @@ internal static class HistorianWcfClientCredentialsHelper
}
}
/// <summary>
/// Creates a channel from <paramref name="factory"/>, honoring
/// <see cref="HistorianClientOptions.ConnectViaAddress"/> when set: the channel <b>connects</b> to
/// the Via address while still addressing the SOAP message <c>To</c> the factory's logical endpoint.
/// This lets a port-forward whose local port differs from the server's real service port satisfy the
/// server-side WCF AddressFilter (which validates the To header). Use this in place of
/// <c>factory.CreateChannel()</c> at every WCF event/read channel-creation site.
/// </summary>
public static TChannel CreateChannel<TChannel>(ChannelFactory<TChannel> factory, HistorianClientOptions options)
{
ArgumentNullException.ThrowIfNull(factory);
ArgumentNullException.ThrowIfNull(options);
return string.IsNullOrWhiteSpace(options.ConnectViaAddress)
? factory.CreateChannel()
: factory.CreateChannel(factory.Endpoint.Address, new Uri(options.ConnectViaAddress));
}
private sealed class AcceptAnyCertificateValidator : X509CertificateValidator
{
public static readonly AcceptAnyCertificateValidator Instance = new();
@@ -8,10 +8,16 @@ using AVEVA.Historian.Client.Wcf.Contracts;
namespace AVEVA.Historian.Client.Wcf;
/// <remarks>
/// Mirrors HistorianWcfReadOrchestrator but targets IRetrievalServiceContract4 for the event flow.
/// Event row buffer layout is undecoded as of this pass — when StartEventQuery succeeds, this
/// orchestrator returns an empty enumeration but logs the row-buffer length via the
/// <see cref="LastResultBufferLength"/> diagnostic so a follow-up capture can decode the wire shape.
/// Mirrors HistorianWcfReadOrchestrator but targets IRetrievalServiceContract4 for the event flow. The
/// event row-buffer layout is decoded (<see cref="HistorianEventRowProtocol"/>; verified against real
/// captured rows). A <b>2023 R2</b> historian <i>does</i> serve this transport via the <b>certificate</b>
/// (TLS) endpoint (the cert transport + <c>NegotiateAuthentication</c> auth work cross-platform; the
/// integrated/Windows transport does not tunnel). With the <c>0x501</c> event connection mode CM_EVENT
/// registration succeeds — but <c>StartEventQuery</c> still returns a 0-row buffer and long-polls: event
/// rows are <b>server-gated</b> per connection on 2023 R2, the same wall as the gRPC path, and not
/// client-fixable (see <c>docs/reverse-engineering/wcf-event-read-spike-results.md</c> and
/// <c>grpc-event-query-capture.md</c>). The native return codes 76/85 noted below were 2020-historian
/// observations.
/// </remarks>
internal sealed class HistorianWcfEventOrchestrator
{
@@ -143,7 +149,7 @@ internal sealed class HistorianWcfEventOrchestrator
EndpointAddress transactionEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(_options, HistorianWcfServiceNames.Transaction);
uint clientHandle = HistorianWcfAuthChainHelper.OpenAuthenticatedConnection(
_options, histBinding, histEndpoint, contextKey, cancellationToken,
connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode,
connectionMode: _options.EventReadConnectionModeOverride ?? HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode,
additionalSetup: (historyChannel, context) =>
AddCmEventTagViaAddT(historyChannel, context, auxBinding, statusEndpoint, transactionEndpoint, retrBinding, retrEndpoint));
return RunEventQuery(retrBinding, retrEndpoint, clientHandle, startUtc, endUtc, filter, cancellationToken);
@@ -163,7 +169,7 @@ internal sealed class HistorianWcfEventOrchestrator
try
{
IRetrievalServiceContract4 channel = factory.CreateChannel();
IRetrievalServiceContract4 channel = HistorianWcfClientCredentialsHelper.CreateChannel(factory, _options);
ICommunicationObject channelCo = (ICommunicationObject)channel;
try
{
@@ -0,0 +1,51 @@
using System.Reflection;
using AVEVA.Historian.Client.Grpc;
using Xunit;
namespace AVEVA.Historian.Client.Tests;
/// <summary>
/// Reflection guard for the event-on-session seams the B0b reuse spike drives (pending.md A1
/// broadening, Stage B0). Mirrors <see cref="TagClientOnSessionSeamTests"/>: the seam runs ONLY the
/// op against an externally-supplied (connection, session), so the spike can run MULTIPLE event ops
/// on one already-opened + registered v8 Event session to measure reuse.
/// </summary>
public class EventOnSessionSeamTests
{
private static MethodInfo RequireMethod(Type owner, string name)
{
MethodInfo? m = owner.GetMethod(
name, BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Public | BindingFlags.Static);
Assert.NotNull(m);
return m!;
}
[Fact]
public void SendEventOnSession_ExposesSeam_WithConnectionAndSessionFirst()
{
MethodInfo m = RequireMethod(typeof(HistorianGrpcEventWriteOrchestrator), "SendEventOnSession");
ParameterInfo[] ps = m.GetParameters();
Assert.Equal("HistorianGrpcConnection", ps[0].ParameterType.Name);
Assert.Equal("Session", ps[1].ParameterType.Name);
}
[Fact]
public void OpenAndRegisterEventSession_ExposesRegisterOnceSeam()
{
// The spike registers CM_EVENT ONCE via this helper, then issues many SendEventOnSession ops.
MethodInfo m = RequireMethod(typeof(HistorianGrpcEventWriteOrchestrator), "OpenAndRegisterEventSession");
ParameterInfo[] ps = m.GetParameters();
Assert.Equal("HistorianGrpcConnection", ps[0].ParameterType.Name);
Assert.Equal("CancellationToken", ps[1].ParameterType.Name);
Assert.Equal("Session", m.ReturnType.Name);
}
[Fact]
public void RunEventQueryOnSession_ExposesSeam_WithConnectionAndSessionFirst()
{
MethodInfo m = RequireMethod(typeof(HistorianGrpcEventOrchestrator), "RunEventQueryOnSession");
ParameterInfo[] ps = m.GetParameters();
Assert.Equal("HistorianGrpcConnection", ps[0].ParameterType.Name);
Assert.Equal("Session", ps[1].ParameterType.Name);
}
}
@@ -0,0 +1,255 @@
using System.Diagnostics;
using Grpc.Core;
using AVEVA.Historian.Client.Grpc;
using AVEVA.Historian.Client.Models;
using Xunit;
using Xunit.Abstractions;
namespace AVEVA.Historian.Client.Tests;
/// <summary>
/// SPIKE (pending.md A1 broadening, Stage B0b): can ONE v8 Event session be REUSED across many event
/// ops without re-handshaking — the precondition for broadening handshake amortization to the event
/// path? Env-gated exactly like <see cref="HandshakeReuseSpikeTests"/> (silent early-return skip
/// without HISTORIAN_GRPC_HOST + HISTORIAN_USER + HISTORIAN_PASSWORD + HISTORIAN_EVENT_SANDBOX_TAG).
///
/// This is the B0b HARNESS only — it is RUN LIVE by a human over VPN in B0c. It SKIPS cleanly offline
/// (no historian contacted, no event sent). It drives the B0a internal seams directly:
/// <see cref="HistorianGrpcEventWriteOrchestrator.OpenAndRegisterEventSession"/> (open v8 Event session
/// + RegisterCmEventTag ONCE) and <see cref="HistorianGrpcEventWriteOrchestrator.SendEventOnSession"/>
/// (send-only, on the externally-supplied warm session).
///
/// Spike questions (priority order), mapped to the test methods below:
/// (1) Does a v8 Event session survive REUSE? — <see cref="ReusedEventSession_SendsTwice_SecondSkipsHandshake"/>
/// (PRIMARY GREEN/RED signal: two sends on one session both succeed; the 2nd skips ECDH+register).
/// (2) Does REGISTER-ONCE work? — <see cref="ReusedEventSession_RegisterOnce_ThenSendMany"/>
/// (OpenAndRegister once, then SendEventOnSession N× — no per-send re-registration).
/// (3) ONE-KIND best-effort — <see cref="ReusedEventSession_ServesReadAfterSend_BestEffort"/>
/// (can the same session also serve a ReadEvents after a send? LOGGED, never asserted — reads are gated C2).
/// (4) IDLE expiry best-effort — <see cref="ReusedEventSession_IdleSweep_BestEffort"/>
/// (how long can the session sit idle before a send breaks? LOGGED, never asserted).
///
/// SAFETY: every send targets the env var HISTORIAN_EVENT_SANDBOX_TAG ONLY (carried as the event
/// SourceName/Type so the appended events are unambiguously attributable to the sandbox identity, never
/// a production tag). Success is ASSERTED for (1)/(2); latency is LOGGED only (no flaky perf gates).
/// </summary>
public sealed class EventSessionReuseSpikeTests
{
private const int SendMany = 3;
private readonly ITestOutputHelper _output;
public EventSessionReuseSpikeTests(ITestOutputHelper output) => _output = output;
// (1) REUSE VALIDITY — PRIMARY signal. Open+register ONE v8 Event session, then SendEventOnSession
// TWICE on it with NO re-handshake/re-register between sends. If the server rejects reusing a v8
// Event session, send #2 fails (false / throws) -> RED finding. Both succeed -> GREEN (event-session
// reuse is sound, the precondition for event amortization). Latency LOGGED so B0c sees the win
// (open+register cost vs the two reused sends).
[Fact]
public void ReusedEventSession_SendsTwice_SecondSkipsHandshake()
{
if (!TryGetEnv(out string host, out string sandboxTag)) return;
HistorianClientOptions options = BuildOptions(host);
var orchestrator = new HistorianGrpcEventWriteOrchestrator(options);
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
var swOpen = Stopwatch.StartNew();
HistorianGrpcHandshake.Session session = orchestrator.OpenAndRegisterEventSession(connection, CancellationToken.None);
swOpen.Stop();
_output.WriteLine($"open+register (ECDH handshake + RegisterCmEventTag) = {swOpen.ElapsedMilliseconds} ms");
_output.WriteLine($"registration diag: {orchestrator.RegistrationDiag}");
for (int i = 0; i < 2; i++)
{
HistorianEvent evt = BuildSandboxEvent(sandboxTag, attempt: i);
var sw = Stopwatch.StartNew();
bool ok = orchestrator.SendEventOnSession(connection, session, evt, CancellationToken.None);
sw.Stop();
_output.WriteLine($"reused-send[{i}] = {sw.ElapsedMilliseconds} ms, ok={ok}, lastErr='{orchestrator.LastSendErrorDescription}'");
Assert.True(ok, $"reused v8 Event session send[{i}] should be accepted (AddStreamValues BSuccess).");
}
}
// (2) REGISTER-ONCE. Open+register ONCE, then SendEventOnSession N× — proving RegisterCmEventTag does
// NOT need re-running per send (the seam's whole point). All sends must succeed.
[Fact]
public void ReusedEventSession_RegisterOnce_ThenSendMany()
{
if (!TryGetEnv(out string host, out string sandboxTag)) return;
HistorianClientOptions options = BuildOptions(host);
var orchestrator = new HistorianGrpcEventWriteOrchestrator(options);
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = orchestrator.OpenAndRegisterEventSession(connection, CancellationToken.None);
for (int i = 0; i < SendMany; i++)
{
HistorianEvent evt = BuildSandboxEvent(sandboxTag, attempt: i);
var sw = Stopwatch.StartNew();
bool ok = orchestrator.SendEventOnSession(connection, session, evt, CancellationToken.None);
sw.Stop();
_output.WriteLine($"register-once send[{i}] = {sw.ElapsedMilliseconds} ms, ok={ok}");
Assert.True(ok, $"register-once send[{i}] should be accepted without per-send re-registration.");
}
}
// (3) ONE-KIND PROBE (best-effort). After a send on the warm session, try a ReadEvents on the SAME
// session. Event reads are GATED (C2 — the gRPC server long-polls GetNext to the no-data terminal and
// row-level retrieval is not verified over gRPC), so the outcome (rows or exception) is LOGGED, never
// asserted: the test passes as long as the catch swallows any failure. Records the one-kind finding
// (can one Event session serve both send and read?) for B0c.
[Fact]
public void ReusedEventSession_ServesReadAfterSend_BestEffort()
{
if (!TryGetEnv(out string host, out string sandboxTag)) return;
HistorianClientOptions options = BuildOptions(host);
var writeOrch = new HistorianGrpcEventWriteOrchestrator(options);
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = writeOrch.OpenAndRegisterEventSession(connection, CancellationToken.None);
bool sent = writeOrch.SendEventOnSession(connection, session, BuildSandboxEvent(sandboxTag, attempt: 0), CancellationToken.None);
_output.WriteLine($"seed-send before read-probe ok={sent}");
// HARD bound so this probe CANNOT hang on the known C2 event-read long-poll (GetNext blocks to the
// no-data terminal on an idle box). Two independent fuses: (1) a read-only options copy with a 5s
// RequestTimeout so each underlying GetNext RPC deadlines quickly (the read orchestrator caps its
// poll deadline at min(10s, RequestTimeout)); (2) an 8s CancellationToken passed as ct so the chain
// is cancelled even if a per-RPC deadline is not honored over a tunnel. Whichever fires first, the
// method returns in ~10s max even when the read never returns data.
HistorianClientOptions readOptions = BuildOptions(host, requestTimeoutOverride: TimeSpan.FromSeconds(5));
var readOrch = new HistorianGrpcEventOrchestrator(readOptions);
using var readCts = new CancellationTokenSource(TimeSpan.FromSeconds(8));
(DateTime startUtc, DateTime endUtc) = LastSevenDays();
try
{
List<HistorianEvent> rows = readOrch.RunEventQueryOnSession(
connection, session, startUtc, endUtc, filter: null, readCts.Token);
_output.WriteLine($"read-after-send -> OK (rows={rows.Count}) => ONE-KIND (an Event session serves send AND read)");
}
catch (Exception ex) when (ex is OperationCanceledException
|| (ex is RpcException rpc && rpc.StatusCode == StatusCode.DeadlineExceeded))
{
// EXPECTED outcome: the read hit its bound (CTS timeout or per-RPC deadline) without returning —
// consistent with the C2 event-read gating (GetNext long-polls to the no-data terminal). This is
// the recorded one-kind finding, NOT a failure.
_output.WriteLine($"read-after-send did not return within the bound (consistent with C2 event-read gating): {ex.GetType().Name}");
}
catch (Exception ex)
{
// Any other rejection on the reused session is also the finding, not a failure.
_output.WriteLine($"read-after-send -> swallowed ({ex.GetType().Name}: {ex.Message}) => read gated/unverified over gRPC (expected)");
}
// No assertion: this method's job is to RECORD the one-kind outcome for B0c, not gate on it.
}
// (4) IDLE-EXPIRY SWEEP (best-effort, log-only). Send, sit idle for a gap, send again; LOG whether the
// 2nd send broke (and after how long). Bounds how long a warm Event session may sit idle before the
// server expires it — informs the keepalive cadence for an event-session pool. Default gap 25s;
// override via HISTORIAN_EVENT_IDLE_SECONDS. NEVER asserted (a break is the finding, not a failure).
[Fact]
[Trait("Category", "LiveSpike")]
public void ReusedEventSession_IdleSweep_BestEffort()
{
if (!TryGetEnv(out string host, out string sandboxTag)) return;
HistorianClientOptions options = BuildOptions(host);
var orchestrator = new HistorianGrpcEventWriteOrchestrator(options);
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = orchestrator.OpenAndRegisterEventSession(connection, CancellationToken.None);
bool first = orchestrator.SendEventOnSession(connection, session, BuildSandboxEvent(sandboxTag, attempt: 0), CancellationToken.None);
_output.WriteLine($"idle-sweep first send ok={first}");
int idleSec = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_EVENT_IDLE_SECONDS"), out int parsed) && parsed > 0
? parsed
: 25;
_output.WriteLine($"idle-sweep: sleeping {idleSec}s before the second send...");
Thread.Sleep(TimeSpan.FromSeconds(idleSec));
try
{
bool second = orchestrator.SendEventOnSession(connection, session, BuildSandboxEvent(sandboxTag, attempt: 1), CancellationToken.None);
_output.WriteLine($"idle {idleSec}s -> second send ok={second} (session {(second ? "SURVIVED" : "rejected")} the idle gap)");
}
catch (Exception ex)
{
_output.WriteLine($"idle {idleSec}s -> second send BROKE ({ex.GetType().Name}: {ex.Message}) — session expired while idle");
}
// No assertion: idle-expiry timing is a LOGGED finding for the keepalive cadence, not a gate.
}
// --- helpers ---
// Build a send event that targets the sandbox identity ONLY. The CM_EVENT send buffer carries no
// per-tag routing field (it registers against the CM_EVENT system tag), so we stamp the sandbox tag
// NAME into SourceName + Type and a marker Property so the appended event is unambiguously
// attributable to the sandbox — never a production tag. A fresh Id/timestamps per attempt.
private static HistorianEvent BuildSandboxEvent(string sandboxTag, int attempt)
{
DateTime now = DateTime.UtcNow;
return new HistorianEvent(
Id: Guid.NewGuid(),
EventTimeUtc: now.AddSeconds(-attempt),
ReceivedTimeUtc: now,
Type: sandboxTag,
SourceName: sandboxTag,
Namespace: "HistGW.EventReuseSpike",
RevisionVersion: 0,
Properties: new Dictionary<string, object?>
{
["SpikeAttempt"] = attempt.ToString(System.Globalization.CultureInfo.InvariantCulture),
["SpikeMarker"] = "B0b-event-session-reuse",
});
}
private static bool TryGetEnv(out string host, out string sandboxTag)
{
host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST") ?? "";
sandboxTag = Environment.GetEnvironmentVariable("HISTORIAN_EVENT_SANDBOX_TAG") ?? "";
return !string.IsNullOrWhiteSpace(host)
&& !string.IsNullOrWhiteSpace(sandboxTag)
&& !string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER"))
&& !string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_PASSWORD"));
}
private static (DateTime StartUtc, DateTime EndUtc) LastSevenDays()
{
DateTime end = DateTime.UtcNow;
return (end - TimeSpan.FromDays(7), end);
}
// requestTimeoutOverride: when set, forces RequestTimeout (used by the read-after-send probe to give
// each GetNext RPC a short deadline). null preserves the env-driven default for the send/idle methods.
private static HistorianClientOptions BuildOptions(string host, TimeSpan? requestTimeoutOverride = null)
{
string? user = Environment.GetEnvironmentVariable("HISTORIAN_USER");
string? password = Environment.GetEnvironmentVariable("HISTORIAN_PASSWORD");
bool explicitCreds = !string.IsNullOrEmpty(user);
int port = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_GRPC_PORT"), out int parsed)
? parsed
: HistorianClientOptions.DefaultGrpcPort;
bool tls = string.Equals(Environment.GetEnvironmentVariable("HISTORIAN_GRPC_TLS"), "true", StringComparison.OrdinalIgnoreCase);
TimeSpan timeout = requestTimeoutOverride
?? (int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_GRPC_TIMEOUT"), out int secs) && secs > 0
? TimeSpan.FromSeconds(secs)
: new HistorianClientOptions { Host = host }.RequestTimeout);
return new HistorianClientOptions
{
Host = host,
Port = port,
Transport = HistorianTransport.RemoteGrpc,
GrpcUseTls = tls,
AllowUntrustedServerCertificate = tls,
ServerDnsIdentity = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_DNSID"),
IntegratedSecurity = !explicitCreds,
UserName = user ?? string.Empty,
Password = password ?? string.Empty,
RequestTimeout = timeout,
Compression = true
};
}
}
@@ -0,0 +1,53 @@
using System;
using System.Buffers.Binary;
using AVEVA.Historian.Client.Wcf;
using Xunit;
namespace AVEVA.Historian.Client.Tests;
/// <summary>
/// Golden-byte coverage for the 2023 R2 gRPC event-SEND registration buffers, pinned against a live
/// native capture (<c>capture-send-event</c> scenario, 2026-06-23). The send itself rides
/// <c>HistoryService.AddStreamValues</c> with the same "OS" buffer the WCF path uses
/// (<see cref="HistorianEventWriteProtocol"/>, already golden-tested in
/// <c>WcfEventWriteProtocolTests</c>); what is gRPC-specific is the CM_EVENT registration the event
/// connection performs first (RegisterTags + the 86-byte gRPC EnsureTags). These fixtures are the raw
/// bytes the native client sent on the wire — they carry no identity (CM_EVENT / "AnE Event" /
/// constant tag + event-type GUIDs / a registration FILETIME).
/// </summary>
public class GrpcEventSendProtocolTests
{
// GrpcHistoryClient.RegisterTags.tagInfos captured from the native event connection: the packet
// header 50 67 02 00 + count(1) + the 16-byte CM_EVENT tag GUID.
private const string CapturedRegisterTagsHex =
"506702000100000045813b35f05d464da253871aef49b321";
// GrpcHistoryClient.EnsureTags.tagInfos captured from the native event connection (86 bytes): the
// 8-byte EnsureTags header + CM_EVENT CTagMetadata + a registration FILETIME + the …e01f2f27
// event-type GUID.
private const string CapturedEnsureTagsHex =
"4e670300010000000386000545813b35f05d464da253871aef49b321090800434d5f4556454e54090900416e45204576656e7402020100000001000000004e18d6bd4503dd0142ae595fb63b604791a5ab0be01f2f27";
[Fact]
public void BuildRegisterCmEventInputBuffer_MatchesNativeGrpcCapture()
{
byte[] expected = Convert.FromHexString(CapturedRegisterTagsHex);
byte[] actual = HistorianEventRegistrationProtocol.BuildRegisterCmEventInputBuffer();
Assert.Equal(expected, actual);
}
[Fact]
public void SerializeCmEventEnsureTagsGrpc_MatchesNativeGrpcCapture()
{
byte[] expected = Convert.FromHexString(CapturedEnsureTagsHex);
Assert.Equal(86, expected.Length);
// The only run-varying field is the registration FILETIME (the 8 bytes immediately before the
// trailing 16-byte event-type GUID). Feed the captured time back so the comparison is exact.
long filetime = BinaryPrimitives.ReadInt64LittleEndian(expected.AsSpan(expected.Length - 24, 8));
DateTime createdUtc = DateTime.FromFileTimeUtc(filetime);
byte[] actual = HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc(createdUtc);
Assert.Equal(expected, actual);
}
}
@@ -0,0 +1,103 @@
using AVEVA.Historian.Client.Grpc;
using GrpcHistory = ArchestrA.Grpc.Contract.History;
using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval;
using GrpcStatus = ArchestrA.Grpc.Contract.Status;
using GrpcTransaction = ArchestrA.Grpc.Contract.Transaction;
using Xunit.Abstractions;
namespace AVEVA.Historian.Client.Tests;
/// <summary>
/// Live evidence test (C3a): reads the four unauthenticated <c>GetInterfaceVersion</c> RPCs from a
/// real 2023 R2 Historian over gRPC and asserts the accepted version set. These RPCs run before any
/// credential exchange, so no HISTORIAN_USER / HISTORIAN_PASSWORD is required — only a reachable host.
///
/// Skips silently when <c>HISTORIAN_GRPC_HOST</c> is absent (offline / CI). The captured integers
/// are recorded in <c>docs/reverse-engineering/grpc-interface-versions.md</c> and close the C3a
/// "2023 R2 gRPC server-version integers not yet captured" gap.
/// </summary>
public sealed class GrpcInterfaceVersionEvidenceTests
{
private readonly ITestOutputHelper _output;
public GrpcInterfaceVersionEvidenceTests(ITestOutputHelper output) => _output = output;
[Fact]
public void GrpcInterfaceVersions_LiveServer_MatchAcceptedSet()
{
string host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST") ?? "";
if (string.IsNullOrWhiteSpace(host))
{
_output.WriteLine("SKIP: HISTORIAN_GRPC_HOST is not set — no live historian available.");
return;
}
HistorianClientOptions options = BuildOptions(host);
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
DateTime deadline = DateTime.UtcNow.Add(TimeSpan.FromSeconds(10));
GrpcHistory.GetInterfaceVersionResponse history =
new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel)
.GetInterfaceVersion(new GrpcHistory.GetInterfaceVersionRequest(), connection.Metadata, deadline, default);
GrpcRetrieval.GetRetrievalInterfaceVersionResponse retrieval =
new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel)
.GetRetrievalInterfaceVersion(new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), connection.Metadata, deadline, default);
GrpcStatus.GetStatusInterfaceVersionResponse status =
new GrpcStatus.StatusService.StatusServiceClient(connection.Channel)
.GetStatusInterfaceVersion(new GrpcStatus.GetStatusInterfaceVersionRequest(), connection.Metadata, deadline, default);
GrpcTransaction.GetTransactionInterfaceVersionResponse transaction =
new GrpcTransaction.TransactionService.TransactionServiceClient(connection.Channel)
.GetTransactionInterfaceVersion(new GrpcTransaction.GetTransactionInterfaceVersionRequest(), connection.Metadata, deadline, default);
_output.WriteLine($"History UiVersion={history.UiVersion} UiError={history.UiError}");
_output.WriteLine($"Retrieval UiVersion={retrieval.UiVersion} UiError={retrieval.UiError}");
_output.WriteLine($"Status UiVersion={status.UiVersion} UiError={status.UiError}");
// Note: Transaction response fields are named Error/Version (not UiError/UiVersion) per the proto.
_output.WriteLine($"Transaction Version={transaction.Version} Error={transaction.Error}");
// History: accepted set is {11 (2020 WCF), 12 (2023 R2 gRPC)}.
Assert.Equal(0u, history.UiError);
Assert.Contains(history.UiVersion, new uint[] { 11u, 12u });
// Retrieval: 4 on both 2020 WCF and 2023 R2 gRPC.
Assert.Equal(0u, retrieval.UiError);
Assert.Equal(4u, retrieval.UiVersion);
// Transaction: 2 (confirmed by live gRPC capture — see grpc-interface-versions.md).
// NOTE: the Transaction response proto uses Error/Version (not UiError/UiVersion).
Assert.Equal(0u, transaction.Error);
Assert.Equal(2u, transaction.Version);
// Status: reachability-only — assert UiError==0 only; UiVersion is not value-gated
// (observed 4 on 2023 R2 gRPC, 0 on 2020 WCF).
Assert.Equal(0u, status.UiError);
}
private static HistorianClientOptions BuildOptions(string host)
{
string? user = Environment.GetEnvironmentVariable("HISTORIAN_USER");
string? password = Environment.GetEnvironmentVariable("HISTORIAN_PASSWORD");
bool explicitCreds = !string.IsNullOrEmpty(user);
int port = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_GRPC_PORT"), out int parsed)
? parsed
: HistorianClientOptions.DefaultGrpcPort;
bool tls = string.Equals(Environment.GetEnvironmentVariable("HISTORIAN_GRPC_TLS"), "true", StringComparison.OrdinalIgnoreCase);
return new HistorianClientOptions
{
Host = host,
Port = port,
Transport = HistorianTransport.RemoteGrpc,
GrpcUseTls = tls,
AllowUntrustedServerCertificate = tls,
ServerDnsIdentity = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_DNSID"),
IntegratedSecurity = !explicitCreds,
UserName = user ?? string.Empty,
Password = password ?? string.Empty,
};
}
}
@@ -0,0 +1,231 @@
using System.Diagnostics;
using AVEVA.Historian.Client.Grpc;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
using Xunit.Abstractions;
namespace AVEVA.Historian.Client.Tests;
/// <summary>
/// SPIKE (pending.md A1): does the 2023 R2 historian honor REUSING one authenticated session across
/// many operations — the precondition for handshake amortization? Env-gated exactly like
/// <see cref="HistorianGrpcIntegrationTests"/> (silent skip without HISTORIAN_GRPC_HOST +
/// HISTORIAN_TEST_TAG + HISTORIAN_USER). READ-ONLY — no writes, safe against live data. Reuse
/// validity is ASSERTED; latency numbers are LOGGED only (no flaky perf gates).
/// </summary>
public sealed class HandshakeReuseSpikeTests
{
private const int ReuseOps = 5;
private readonly ITestOutputHelper _output;
public HandshakeReuseSpikeTests(ITestOutputHelper output) => _output = output;
// (1) REUSE VALIDITY: one session, N reads on the same clientHandle. If the server rejects handle
// reuse, read #2+ throws -> RED finding. All succeed -> GREEN finding.
[Fact]
public void ReusedSession_RunsManyReads_AllSucceed()
{
if (!TryGetEnv(out string host, out string tag)) return;
HistorianClientOptions options = BuildOptions(host);
(DateTime startUtc, DateTime endUtc) = LastSevenDays();
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
var swHandshake = Stopwatch.StartNew();
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, CancellationToken.None);
swHandshake.Stop();
_output.WriteLine($"open-session (handshake) = {swHandshake.ElapsedMilliseconds} ms");
var orchestrator = new HistorianGrpcReadOrchestrator(options);
for (int i = 0; i < ReuseOps; i++)
{
var sw = Stopwatch.StartNew();
List<HistorianSample> rows = orchestrator.RunRawQueryOnSession(
connection, session.ClientHandle, tag, startUtc, endUtc, maxValues: 8, CancellationToken.None);
sw.Stop();
_output.WriteLine($"reused-read[{i}] = {sw.ElapsedMilliseconds} ms, rows={rows.Count}");
Assert.NotEmpty(rows);
Assert.All(rows, r => Assert.Equal(tag, r.TagName));
}
}
// (2) WIN MAGNITUDE: full per-call path (fresh Create+handshake+query) vs amortized (one handshake,
// N reused reads). Logged only — quantifies whether amortization is worth the refactor.
[Fact]
public async Task ReusedSession_VsPerCallPath_LogsLatencyDelta()
{
if (!TryGetEnv(out string host, out string tag)) return;
HistorianClientOptions options = BuildOptions(host);
(DateTime startUtc, DateTime endUtc) = LastSevenDays();
HistorianClient client = new(options);
var swPerCall = Stopwatch.StartNew();
for (int i = 0; i < ReuseOps; i++)
{
await foreach (HistorianSample _ in client.ReadRawAsync(tag, startUtc, endUtc, 8, CancellationToken.None)) { }
}
swPerCall.Stop();
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, CancellationToken.None);
var orchestrator = new HistorianGrpcReadOrchestrator(options);
var swAmortized = Stopwatch.StartNew();
for (int i = 0; i < ReuseOps; i++)
{
orchestrator.RunRawQueryOnSession(connection, session.ClientHandle, tag, startUtc, endUtc, 8, CancellationToken.None);
}
swAmortized.Stop();
_output.WriteLine($"per-call ({ReuseOps} ops) = {swPerCall.ElapsedMilliseconds} ms");
_output.WriteLine($"amortized ({ReuseOps} ops) = {swAmortized.ElapsedMilliseconds} ms");
_output.WriteLine($"saving over {ReuseOps} ops = {swPerCall.ElapsedMilliseconds - swAmortized.ElapsedMilliseconds} ms");
}
// (3) EXPIRY SWEEP: reuse after idle gaps. Default bounded [0s, 30s]; longer tiers opt-in via
// HISTORIAN_REUSE_IDLE_SECONDS="0,30,120,600". Rethrows at the tier where reuse first breaks.
[Fact]
public void ReusedSession_IdleSweep_SurfacesExpiryTier()
{
if (!TryGetEnv(out string host, out string tag)) return;
HistorianClientOptions options = BuildOptions(host);
(DateTime startUtc, DateTime endUtc) = LastSevenDays();
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, CancellationToken.None);
var orchestrator = new HistorianGrpcReadOrchestrator(options);
foreach (int delaySec in ParseIdleSweep())
{
if (delaySec > 0) Thread.Sleep(TimeSpan.FromSeconds(delaySec));
try
{
List<HistorianSample> rows = orchestrator.RunRawQueryOnSession(
connection, session.ClientHandle, tag, startUtc, endUtc, 8, CancellationToken.None);
_output.WriteLine($"idle {delaySec,4}s -> OK (rows={rows.Count})");
}
catch (Exception ex)
{
_output.WriteLine($"idle {delaySec,4}s -> BROKE ({ex.GetType().Name})");
throw;
}
}
}
// (A) WRITE REUSE VALIDITY: one externally-opened 0x401 (write-enabled) session, N writes on it via
// RunWriteOnSession — NO Create()/handshake per write. If the server rejects reusing a write session,
// write #2 throws -> RED finding. Both succeed -> GREEN (write-reuse is sound). Bounded writes to the
// sandbox tag ONLY; latency LOGGED, success ASSERTED.
[Fact]
public void WriteEnabledSession_RunsTwoWrites_AllSucceed()
{
if (!TryGetWriteEnv(out string host, out string sandboxTag)) return;
HistorianClientOptions options = BuildOptions(host);
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = OpenWriteSession(options, connection);
var writer = new HistorianGrpcHistoricalWriteOrchestrator(options);
for (int i = 0; i < 2; i++)
{
var sw = Stopwatch.StartNew();
bool ok = writer.RunWriteOnSession(connection, session, sandboxTag,
new[] { new HistorianHistoricalValue(DateTime.UtcNow.AddSeconds(-i), 1.0 + i, OpcQuality: 192) },
CancellationToken.None);
sw.Stop();
_output.WriteLine($"reused-write[{i}] = {sw.ElapsedMilliseconds} ms, ok={ok}");
Assert.True(ok);
}
}
// (B) READ-ON-WRITE-SESSION PROBE: can a 0x401 (write-enabled) session ALSO serve a raw read? Decides
// the pool shape — ONE-KIND (a single write-enabled session serves both reads and writes) vs TWO-KIND
// (separate read 0x402 / write 0x401 sessions). The kind is LOGGED, never asserted; any failure is
// swallowed (a rejection is itself the finding, not a test failure). READ-ONLY here.
[Fact]
public void WriteEnabledSession_AlsoServesRead_RecordsKind()
{
if (!TryGetWriteEnv(out string host, out string sandboxTag)) return;
HistorianClientOptions options = BuildOptions(host);
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = OpenWriteSession(options, connection);
(DateTime startUtc, DateTime endUtc) = LastSevenDays();
try
{
List<HistorianSample> rows = new HistorianGrpcReadOrchestrator(options)
.RunRawQueryOnSession(connection, session.ClientHandle, sandboxTag, startUtc, endUtc, 8, CancellationToken.None);
_output.WriteLine($"read-on-0x401 -> OK (rows={rows.Count}) => ONE-KIND pool (write-enabled serves reads)");
}
catch (Exception ex)
{
_output.WriteLine($"read-on-0x401 -> FAILED ({ex.GetType().Name}) => TWO-KIND pool (separate read/write)");
}
}
// --- helpers ---
private static bool TryGetWriteEnv(out string host, out string sandboxTag)
{
host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST") ?? "";
sandboxTag = Environment.GetEnvironmentVariable("HISTORIAN_WRITE_SANDBOX_TAG") ?? "";
return !string.IsNullOrWhiteSpace(host)
&& !string.IsNullOrWhiteSpace(sandboxTag)
&& !string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER"));
}
private static HistorianGrpcHandshake.Session OpenWriteSession(HistorianClientOptions o, HistorianGrpcConnection c)
=> HistorianGrpcHandshake.OpenSession(c, o, CancellationToken.None,
connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode);
private static bool TryGetEnv(out string host, out string tag)
{
host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST") ?? "";
tag = Environment.GetEnvironmentVariable("HISTORIAN_TEST_TAG") ?? "";
return !string.IsNullOrWhiteSpace(host)
&& !string.IsNullOrWhiteSpace(tag)
&& !string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER"));
}
private static (DateTime StartUtc, DateTime EndUtc) LastSevenDays()
{
DateTime end = DateTime.UtcNow;
return (end - TimeSpan.FromDays(7), end);
}
private static int[] ParseIdleSweep()
{
string? raw = Environment.GetEnvironmentVariable("HISTORIAN_REUSE_IDLE_SECONDS");
if (string.IsNullOrWhiteSpace(raw)) return [0, 30];
return raw.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries)
.Select(s => int.Parse(s, System.Globalization.CultureInfo.InvariantCulture)).ToArray();
}
private static HistorianClientOptions BuildOptions(string host)
{
string? user = Environment.GetEnvironmentVariable("HISTORIAN_USER");
string? password = Environment.GetEnvironmentVariable("HISTORIAN_PASSWORD");
bool explicitCreds = !string.IsNullOrEmpty(user);
int port = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_GRPC_PORT"), out int parsed)
? parsed
: HistorianClientOptions.DefaultGrpcPort;
bool tls = string.Equals(Environment.GetEnvironmentVariable("HISTORIAN_GRPC_TLS"), "true", StringComparison.OrdinalIgnoreCase);
// Optional per-call deadline override (seconds) for slow/remote boxes — heavier aggregate
// modes over a tunnelled link can exceed the 30s default. Falls back to the SDK default.
TimeSpan timeout = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_GRPC_TIMEOUT"), out int secs) && secs > 0
? TimeSpan.FromSeconds(secs)
: new HistorianClientOptions { Host = host }.RequestTimeout;
return new HistorianClientOptions
{
Host = host,
Port = port,
Transport = HistorianTransport.RemoteGrpc,
GrpcUseTls = tls,
AllowUntrustedServerCertificate = tls,
ServerDnsIdentity = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_DNSID"),
IntegratedSecurity = !explicitCreds,
UserName = user ?? string.Empty,
Password = password ?? string.Empty,
RequestTimeout = timeout,
Compression = true // the stock client always advertises grpc gzip request encoding
};
}
}
@@ -90,6 +90,69 @@ public sealed class HistorianEventRowProtocolTests
Assert.Equal(500, evt.Properties["priority"]);
}
[Fact]
public void Parse_Version11GrpcHeader_ParsesRowsIdenticalToV9()
{
// 2023 R2 gRPC returns the event-row buffer with container version 11; the per-row layout is
// byte-identical to the WCF v9 format. The parser must accept both (verified against a captured
// stock-client read of 50 Alarm.Set/Alarm.Clear rows whose header began 0B00 .. 1E000000 0700).
DateTime t1 = new(2026, 6, 23, 13, 34, 14, DateTimeKind.Utc);
DateTime t2 = t1.AddSeconds(10);
byte[] header = BuildHeader(2u, HistorianEventRowProtocol.EventRowProtocolVersionGrpc); // version 11
byte[] buffer = Concat(header, BuildRow(t1, "Alarm.Set", []), BuildRow(t2, "Alarm.Clear", []));
IReadOnlyList<HistorianEvent> events = HistorianEventRowProtocol.Parse(buffer);
Assert.Equal(2, events.Count);
Assert.Equal("Alarm.Set", events[0].Type);
Assert.Equal(t1, events[0].EventTimeUtc);
Assert.Equal("Alarm.Clear", events[1].Type);
Assert.Equal(t2, events[1].EventTimeUtc);
}
// Verification against the PROVIDED 2023 R2 client: parse the real GetNextEventQueryResultBuffer
// result the stock client received (50 events), proving our read path decodes genuine gRPC event
// data. The capture carries customer identity so it is gitignored — point HISTORIAN_EVENT_CAPTURE_NDJSON
// at the captured ndjson to run; the test skips cleanly otherwise (no fixture committed).
[Fact]
public void Parse_RealStockClientCapture_DecodesAllEvents()
{
string? ndjson = Environment.GetEnvironmentVariable("HISTORIAN_EVENT_CAPTURE_NDJSON");
if (string.IsNullOrWhiteSpace(ndjson) || !File.Exists(ndjson))
{
return; // gated: no capture available
}
byte[]? resultBuffer = null;
foreach (string line in File.ReadLines(ndjson))
{
if (!line.Contains("GetNextEventQueryResultBuffer.result.out")) continue;
int i = line.IndexOf("\"Base64\":\"", StringComparison.Ordinal);
if (i < 0) continue;
i += "\"Base64\":\"".Length;
int j = line.IndexOf('"', i);
resultBuffer = Convert.FromBase64String(line.Substring(i, j - i));
break;
}
Assert.NotNull(resultBuffer);
ushort version = BinaryPrimitives.ReadUInt16LittleEndian(resultBuffer.AsSpan(0, 2));
uint rowCount = BinaryPrimitives.ReadUInt32LittleEndian(resultBuffer.AsSpan(2, 4));
Assert.Equal(HistorianEventRowProtocol.EventRowProtocolVersionGrpc, version); // real gRPC buffer is v11
IReadOnlyList<HistorianEvent> events = HistorianEventRowProtocol.Parse(resultBuffer);
// Our parser decodes every row the stock client received.
Assert.Equal((int)rowCount, events.Count);
Assert.All(events, e =>
{
Assert.False(string.IsNullOrEmpty(e.Type));
Assert.NotEqual(default, e.EventTimeUtc);
});
// Sanitized cross-check: only the generic AVEVA event types (no customer fields asserted).
Assert.All(events, e => Assert.Contains(e.Type, new[] { "Alarm.Set", "Alarm.Clear" }));
}
[Fact]
public void Parse_UnknownTypeMarker_KeepsRawBytesInPropertyBag()
{
@@ -120,11 +183,15 @@ public sealed class HistorianEventRowProtocolTests
Assert.Equal("Alarm.Set", events[0].Type);
}
private static byte[] BuildHeader(uint rowCount)
private static byte[] BuildHeader(uint rowCount) => BuildHeader(rowCount, HistorianEventRowProtocol.EventRowProtocolVersion);
private static byte[] BuildHeader(uint rowCount, ushort version)
{
byte[] header = new byte[6];
BinaryPrimitives.WriteUInt16LittleEndian(header.AsSpan(0, 2), HistorianEventRowProtocol.EventRowProtocolVersion);
// version(2) + rowCount(4) + the single buffer-level header field (0x1E). Rows are markerless.
byte[] header = new byte[10];
BinaryPrimitives.WriteUInt16LittleEndian(header.AsSpan(0, 2), version);
BinaryPrimitives.WriteUInt32LittleEndian(header.AsSpan(2, 4), rowCount);
BinaryPrimitives.WriteUInt32LittleEndian(header.AsSpan(6, 4), HistorianEventRowProtocol.BufferHeaderField);
return header;
}
@@ -141,13 +208,13 @@ public sealed class HistorianEventRowProtocolTests
propertyBlockSize += propertyBlocks[i].Length;
}
byte[] row = new byte[4 + 2 + 8 + 16 + eventTypeBytes.Length + 2 + propertyBlockSize];
// Markerless row: rowFormat(2) + filetime(8) + 8×UInt16 slots(16) + type + propCount + props.
byte[] row = new byte[2 + 8 + 16 + eventTypeBytes.Length + 2 + propertyBlockSize];
Span<byte> span = row;
BinaryPrimitives.WriteUInt32LittleEndian(span[..4], HistorianEventRowProtocol.RowMarker);
BinaryPrimitives.WriteUInt16LittleEndian(span.Slice(4, 2), HistorianEventRowProtocol.RowFormatV9);
BinaryPrimitives.WriteInt64LittleEndian(span.Slice(6, 8), eventTimeUtc.ToFileTimeUtc());
BinaryPrimitives.WriteUInt16LittleEndian(span[..2], HistorianEventRowProtocol.RowFormat);
BinaryPrimitives.WriteInt64LittleEndian(span.Slice(2, 8), eventTimeUtc.ToFileTimeUtc());
// 16 bytes of zeroed slot ushorts left as-is.
int eventTypeOffset = 4 + 2 + 8 + 16;
int eventTypeOffset = 2 + 8 + 16;
eventTypeBytes.CopyTo(span[eventTypeOffset..]);
BinaryPrimitives.WriteUInt16LittleEndian(span.Slice(eventTypeOffset + eventTypeBytes.Length, 2), propertyCount);
int cursor = eventTypeOffset + eventTypeBytes.Length + 2;
@@ -0,0 +1,111 @@
using AVEVA.Historian.Client.Grpc;
using AVEVA.Historian.Client.Models;
using Xunit.Abstractions;
namespace AVEVA.Historian.Client.Tests;
/// <summary>
/// Live end-to-end round-trip for <see cref="HistorianEventSession"/> (the v8 EVENT sibling of
/// <see cref="HistorianSessionRoundTripTests"/>): open ONE reusable event session, SendEvent on it
/// TWICE (no re-handshake/re-register between sends), ping once, dispose. Env-gated exactly like
/// <see cref="EventSessionReuseSpikeTests"/> (silent early-return skip without HISTORIAN_GRPC_HOST +
/// HISTORIAN_USER + HISTORIAN_PASSWORD + HISTORIAN_EVENT_SANDBOX_TAG). Every send targets the sandbox
/// identity ONLY (carried as the event SourceName/Type), never a production tag.
/// </summary>
public sealed class HistorianEventSessionRoundTripTests
{
private readonly ITestOutputHelper _output;
public HistorianEventSessionRoundTripTests(ITestOutputHelper output) => _output = output;
[Fact]
public async Task EventSession_SendTwicePing_AllOnOneSession()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST");
string? sandboxTag = Environment.GetEnvironmentVariable("HISTORIAN_EVENT_SANDBOX_TAG");
if (string.IsNullOrWhiteSpace(host) || string.IsNullOrWhiteSpace(sandboxTag)
|| string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER"))
|| string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_PASSWORD")))
{
return; // skip — env not configured
}
HistorianClientOptions options = BuildOptions(host);
await using var client = new HistorianClient(options);
await using HistorianEventSession session = await client.OpenEventSessionAsync(CancellationToken.None);
// 1) send TWICE on the SAME (open + CM_EVENT-registered) session — the 2nd skips ECDH+register.
for (int i = 0; i < 2; i++)
{
bool sent = await session.SendEventAsync(BuildSandboxEvent(sandboxTag, attempt: i), CancellationToken.None);
_output.WriteLine($"{i + 1}) reused-send[{i}] -> ok={sent}");
Assert.True(sent, $"reused v8 Event session send[{i}] should be accepted (AddStreamValues BSuccess).");
}
// 2) ping on the SAME session — must not throw.
await session.PingAsync(CancellationToken.None);
_output.WriteLine("3) ping -> ok");
// 3) prove the keepalive op actually RETURNS DATA on the v8 Event handle (not just no-throw):
// issue the same underlying GetSystemParameter the ping uses, directly against the event
// session's connection + ClientHandle, and assert it yields a non-empty value.
string? keepalive = HistorianGrpcStatusClient.GetSystemParameterOnSession(
session.Connection, session.Session.ClientHandle, options, "HistorianVersion", CancellationToken.None);
Assert.False(string.IsNullOrEmpty(keepalive));
_output.WriteLine($"4) keepalive GetSystemParameter on event handle -> '{keepalive}'");
_output.WriteLine("event-session round-trip OK (two sends + ping + verified keepalive on one session)");
}
// Build a send event that targets the sandbox identity ONLY (mirrors EventSessionReuseSpikeTests.
// BuildSandboxEvent): the CM_EVENT send buffer carries no per-tag routing field, so the sandbox tag
// NAME is stamped into SourceName + Type + a marker Property so the appended event is unambiguously
// attributable to the sandbox — never a production tag. A fresh Id/timestamps per attempt.
private static HistorianEvent BuildSandboxEvent(string sandboxTag, int attempt)
{
DateTime now = DateTime.UtcNow;
return new HistorianEvent(
Id: Guid.NewGuid(),
EventTimeUtc: now.AddSeconds(-attempt),
ReceivedTimeUtc: now,
Type: sandboxTag,
SourceName: sandboxTag,
Namespace: "HistGW.EventSessionRoundTrip",
RevisionVersion: 0,
Properties: new Dictionary<string, object?>
{
["RoundTripAttempt"] = attempt.ToString(System.Globalization.CultureInfo.InvariantCulture),
["RoundTripMarker"] = "B1-event-session-roundtrip",
});
}
// verbatim copy of BuildOptions from HistorianSessionRoundTripTests / EventSessionReuseSpikeTests
private static HistorianClientOptions BuildOptions(string host)
{
string? user = Environment.GetEnvironmentVariable("HISTORIAN_USER");
string? password = Environment.GetEnvironmentVariable("HISTORIAN_PASSWORD");
bool explicitCreds = !string.IsNullOrEmpty(user);
int port = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_GRPC_PORT"), out int parsed)
? parsed
: HistorianClientOptions.DefaultGrpcPort;
bool tls = string.Equals(Environment.GetEnvironmentVariable("HISTORIAN_GRPC_TLS"), "true", StringComparison.OrdinalIgnoreCase);
TimeSpan timeout = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_GRPC_TIMEOUT"), out int secs) && secs > 0
? TimeSpan.FromSeconds(secs)
: new HistorianClientOptions { Host = host }.RequestTimeout;
return new HistorianClientOptions
{
Host = host,
Port = port,
Transport = HistorianTransport.RemoteGrpc,
GrpcUseTls = tls,
AllowUntrustedServerCertificate = tls,
ServerDnsIdentity = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_DNSID"),
IntegratedSecurity = !explicitCreds,
UserName = user ?? string.Empty,
Password = password ?? string.Empty,
RequestTimeout = timeout,
Compression = true
};
}
}
@@ -18,26 +18,31 @@ namespace AVEVA.Historian.Client.Tests;
public sealed class HistorianGrpcHandshakeRoutingTests
{
[Fact]
public void Handshake_UsesValidateClientCredential_NotExchangeKey()
public void Handshake_TokenLoop_UsesValidateClientCredential_NotExchangeKey()
{
// The auth token loop lives in the shared handshake helper (reused by the read, status,
// and future browse/metadata gRPC paths).
HashSet<string> calledMethods = CollectCalledMethodNames(
"AVEVA.Historian.Client.Grpc.HistorianGrpcHandshake");
// The auth token loop is a compiler-generated lambda (nested closure) passed to RunTokenRounds.
// It MUST carry the SSPI/Negotiate token via StorageService.ValidateClientCredential, never via
// HistoryService.ExchangeKey (the earlier regression). ExchangeKey is now legitimately called
// directly in OpenSession for the SEPARATE v8 event-connection key exchange (ECDH) — that is
// allowed, so the guardrail scopes the no-ExchangeKey rule to the token-loop closures only.
HashSet<string> tokenLoopCalls = CollectCalledMethodNames(
"AVEVA.Historian.Client.Grpc.HistorianGrpcHandshake", nestedClosuresOnly: true);
Assert.Contains("ValidateClientCredential", calledMethods);
Assert.DoesNotContain("ExchangeKey", calledMethods);
Assert.Contains("ValidateClientCredential", tokenLoopCalls);
Assert.DoesNotContain("ExchangeKey", tokenLoopCalls);
}
private static HashSet<string> CollectCalledMethodNames(string typeFullName)
private static HashSet<string> CollectCalledMethodNames(string typeFullName, bool nestedClosuresOnly = false)
{
Assembly sdk = typeof(HistorianClientOptions).Assembly;
Type orchestrator = sdk.GetType(typeFullName, throwOnError: true)!;
Module module = orchestrator.Module;
// The orchestrator type plus its compiler-generated nested types (lambda closures).
IEnumerable<Type> types = new[] { orchestrator }
.Concat(orchestrator.GetNestedTypes(BindingFlags.Public | BindingFlags.NonPublic));
// The compiler-generated nested types (lambda closures) — optionally with the orchestrator type
// itself. The token loop lives inside a closure; ExchangeKey (event key exchange) lives in the
// OpenSession body, so scoping to closures isolates the token-loop routing.
IEnumerable<Type> nested = orchestrator.GetNestedTypes(BindingFlags.Public | BindingFlags.NonPublic);
IEnumerable<Type> types = nestedClosuresOnly ? nested : new[] { orchestrator }.Concat(nested);
var names = new HashSet<string>(StringComparer.Ordinal);
foreach (Type t in types)
@@ -233,6 +233,37 @@ public sealed class HistorianGrpcIntegrationTests
Assert.Contains(samples, s => s.NumericValue is { } v && Math.Abs(v - expected) < 0.01);
}
[Fact]
public async Task SendEventAsync_OverGrpc_AcceptsEvent()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST");
// Gated additionally on a dedicated opt-in so this WRITE test never runs by accident — it
// appends a clearly-marked test event to the server's event history. Captured 2026-06-23:
// the gRPC event send rides HistoryService.AddStreamValues with the same "OS" buffer the WCF
// path uses (HistorianEventWriteProtocol), on a v8 Event session + CM_EVENT registration.
if (string.IsNullOrWhiteSpace(host)
|| string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER"))
|| Environment.GetEnvironmentVariable("HISTORIAN_GRPC_EVENT_SEND") is null)
{
return;
}
HistorianClient client = new(BuildOptions(host));
var evt = new HistorianEvent(
Id: Guid.NewGuid(),
EventTimeUtc: DateTime.UtcNow,
ReceivedTimeUtc: DateTime.UtcNow,
Type: "SdkSendProbe",
SourceName: "SdkSendProbe",
Namespace: "SdkCapture",
RevisionVersion: 0,
Properties: new Dictionary<string, object?> { ["SdkProbeProp"] = "SdkProbeValue" });
bool sent = await client.SendEventAsync(evt, CancellationToken.None);
Assert.True(sent, "gRPC SendEvent should be accepted by the server (AddStreamValues BSuccess).");
}
[Fact]
public async Task ReadAggregateAsync_OverGrpc_ReturnsTimeWeightedAverageRows()
{
@@ -487,13 +518,19 @@ public sealed class HistorianGrpcIntegrationTests
// Plan #2: ReadEvents over gRPC. The chain runs end-to-end and StartEventQuery succeeds
// (no InvalidOperationException), but — confirmed live 2026-06-22 — GetNextEventQueryResultBuffer
// LONG-POLLS when the query has no rows: the gRPC server blocks to the deadline instead of
// returning the synchronous 5-byte code-85 terminal the 2020 WCF op returns. The idle dev box
// holds no events, so the orchestrator reaches its no-data terminal with zero rows and (rather
// than assert a possibly-false "no events" empty) throws ProtocolEvidenceMissingException.
// This pins that current reality and that the chain stays BOUNDED (no multi-minute hang) via
// the short registration + poll deadlines. Flip to asserting parsed rows once an event-bearing
// 2023 R2 server is available. (Set a small HISTORIAN_GRPC_TIMEOUT to keep this snappy.)
// LONG-POLLS and returns zero rows: the gRPC server blocks to the deadline instead of
// returning the synchronous 5-byte code-85 terminal the 2020 WCF op returns, so the orchestrator
// reaches its no-data terminal with zero rows and (rather than assert a possibly-false "no events"
// empty) throws ProtocolEvidenceMissingException.
//
// IMPORTANT (2026-06-22): the zero rows are NOT "no events on the server". Verified against the
// live 2023 R2 box, which holds 19,356 events in the last 30 days (SQL ground truth via the INSQL
// linked server, Runtime.dbo.Events). The EMPTY-FILTER gRPC event query simply does not match
// them. So the gate here is the empty-filter request shape (filter / namespace / event-tag
// registration), NOT data availability — this is capture-gated (needs a native gRPC event-query
// capture), not server-gated. Flip to asserting parsed rows once that capture lands and the
// request is corrected. The chain stays BOUNDED (no multi-minute hang) via the short
// registration + poll deadlines. (Set a small HISTORIAN_GRPC_TIMEOUT to keep this snappy.)
HistorianClient client = new(BuildOptions(host));
DateTime endUtc = DateTime.UtcNow;
@@ -509,6 +546,42 @@ public sealed class HistorianGrpcIntegrationTests
});
}
[Fact]
public async Task EventReadDiagnostic_OverGrpc_PrintsJourney()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST");
if (string.IsNullOrWhiteSpace(host) || string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER")))
{
return;
}
if (Environment.GetEnvironmentVariable("HISTORIAN_GRPC_EVENT_DIAG") is null)
{
return; // opt-in diagnostic only
}
var orch = new AVEVA.Historian.Client.Grpc.HistorianGrpcEventOrchestrator(BuildOptions(host));
var events = new List<HistorianEvent>();
string outcome;
try
{
await foreach (HistorianEvent evt in orch.ReadEventsAsync(DateTime.UtcNow.AddDays(-90), DateTime.UtcNow, null, CancellationToken.None))
{
events.Add(evt);
if (events.Count >= 3) { break; }
}
outcome = $"OK events={events.Count}";
}
catch (Exception ex)
{
outcome = $"{ex.GetType().Name}: {ex.Message}";
}
throw new Xunit.Sdk.XunitException(
$"[DIAG] channel={orch.EventChannelMode} outcome={outcome} | events={events.Count} | LastResultLen={orch.LastResultBufferLength} " +
$"| ResultHex={orch.LastResultBufferHex} | Reg=[{orch.RegistrationDiag}] " +
$"| v8open={AVEVA.Historian.Client.Grpc.HistorianGrpcHandshake.LastEventOpenRequestHex}");
}
[Fact]
public async Task GetConnectionStatusAsync_OverGrpc_ReportsConnected()
{
@@ -531,6 +604,85 @@ public sealed class HistorianGrpcIntegrationTests
Assert.False(status.ConnectedToStoreForward);
}
[Fact]
public async Task DeleteTagExtendedProperties_OverGrpc_ProbeMultiplexedChannel()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST");
// DESTRUCTIVE: reuses the write sandbox-tag gate so it never mutates a server by accident.
string? sandbox = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_WRITE_SANDBOX_TAG");
if (string.IsNullOrWhiteSpace(host) || string.IsNullOrWhiteSpace(sandbox)
|| string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER")))
{
return;
}
// Out-of-scope opportunistic probe (grpc-tooling-completion.md). DelTep is server-blocked on the
// 2020 WCF transport because the server resolves the property from a per-CONNECTION working set
// that WCF's separate per-service channels can't populate (prime + delete land on different
// connections). gRPC builds every service client on ONE shared channel, so this probe runs the
// native GetTgByNm -> GetTepByNm -> DelTep sequence over a single write-enabled session to see
// whether the multiplexed channel satisfies the working-set check.
//
// RESULT — live 2026-06-22 (2023 R2 server, History iface 12): the multiplexed channel does NOT
// lift the wall. BOTH primes succeed on the shared channel (GetTgByNm returns the tag identity,
// GetTepByNm returns the property page), yet DelTep is still rejected with native error code=1
// (the 5-byte buffer's byte0=132 is the universal 0x84 error marker, not a code) and the property
// survives. So DelTep stays server-blocked on BOTH transports — the working set the server
// consults is populated by something the SDK can't reproduce even over one connection (likely the
// native client's in-process registration object, not the wire session). This test PINS that
// negative: it asserts the probe is rejected and the property survives. If a future server or
// registration change lifts the wall, the inverted assertion flips and we notice.
HistorianClientOptions options = BuildOptions(host);
HistorianClient client = new(options);
const string propName = "GrpcDelProbe";
try
{
try { await client.DeleteTagAsync(sandbox!, CancellationToken.None); } catch { /* clean slate */ }
Assert.True(await client.EnsureTagAsync(
new HistorianTagDefinition { TagName = sandbox!, DataType = HistorianDataType.Float, EngineeringUnit = "u", MaxEU = 100 },
CancellationToken.None), "EnsureTags should create the sandbox tag.");
Assert.True(await client.AddTagExtendedPropertyAsync(sandbox!, propName, "todelete", CancellationToken.None),
"AddTagExtendedProperties should seed the property to delete.");
IReadOnlyList<HistorianTagExtendedProperty> before = await client.GetTagExtendedPropertiesAsync(sandbox!, CancellationToken.None);
Assert.Contains(before, p => string.Equals(p.Name, propName, StringComparison.OrdinalIgnoreCase));
HistorianGrpcTagWriteOrchestrator orchestrator = new(options);
HistorianGrpcTagWriteOrchestrator.DeleteTagExtendedPropertiesProbeResult probe =
await orchestrator.ProbeDeleteTagExtendedPropertiesAsync(sandbox!, [propName], CancellationToken.None);
// The primes must succeed (proving the single channel DID load the working set) — otherwise the
// negative below would be inconclusive (a prime failure, not a delete-path block).
Assert.True(probe.TagInfoPrimeBytes > 0, "GetTgByNm prime should return the tag identity on the shared channel.");
Assert.True(probe.ExtPropPrimePages > 0, "GetTepByNm prime should return the property page on the shared channel.");
IReadOnlyList<HistorianTagExtendedProperty> after = await client.GetTagExtendedPropertiesAsync(sandbox!, CancellationToken.None);
bool stillPresent = after.Any(p => string.Equals(p.Name, propName, StringComparison.OrdinalIgnoreCase));
// Pinned negative: the multiplexed channel does not lift the working-set wall.
Assert.False(probe.Accepted,
$"Unexpected: DelTep over gRPC was ACCEPTED — the multiplexed channel may now lift the wall. " +
$"Promote DeleteTagExtendedProperties to a public op. Probe={DescribeProbe(probe, stillPresent)}");
Assert.True(stillPresent,
$"Unexpected: property was removed despite DelTep reporting failure. Probe={DescribeProbe(probe, stillPresent)}");
}
finally
{
for (int i = 0; i < 5; i++)
{
try { await client.DeleteTagAsync(sandbox!, CancellationToken.None); } catch { /* best-effort */ }
await Task.Delay(TimeSpan.FromSeconds(1));
}
}
}
private static string DescribeProbe(
HistorianGrpcTagWriteOrchestrator.DeleteTagExtendedPropertiesProbeResult probe, bool stillPresent)
=> $"Accepted={probe.Accepted} StillPresent={stillPresent} TgPrimeBytes={probe.TagInfoPrimeBytes} " +
$"TepPrimePages={probe.ExtPropPrimePages} Err={probe.ErrorDescription}";
private static HistorianClientOptions BuildOptions(string host)
{
string? user = Environment.GetEnvironmentVariable("HISTORIAN_USER");
@@ -557,7 +709,8 @@ public sealed class HistorianGrpcIntegrationTests
IntegratedSecurity = !explicitCreds,
UserName = user ?? string.Empty,
Password = password ?? string.Empty,
RequestTimeout = timeout
RequestTimeout = timeout,
Compression = true // the stock client always advertises grpc gzip request encoding
};
}
}
@@ -71,8 +71,9 @@ public sealed class HistorianServerVersionGateTests
[Fact]
public void Validate_VerificationDisabled_NeverThrows()
{
// A wildly wrong version is tolerated when the operator opts out (e.g. bringing up a
// 2023 R2 gRPC server whose reported integers have not yet been captured).
// A wildly wrong version is tolerated when the operator opts out. The 2023 R2 gRPC
// integers are now captured live (see docs/reverse-engineering/grpc-interface-versions.md)
// and all accepted; this opt-out is a safety valve for some future, not-yet-captured value.
HistorianServerVersionGate.Validate(HistorianServiceInterface.History, 999u, Options(verify: false));
HistorianServerVersionGate.Validate(HistorianServiceInterface.Retrieval, 0u, Options(verify: false));
}
@@ -0,0 +1,102 @@
using AVEVA.Historian.Client.Models;
using Xunit.Abstractions;
namespace AVEVA.Historian.Client.Tests;
/// <summary>
/// Live end-to-end round-trip for <see cref="HistorianSession"/>: open → write → read → status →
/// ping → dispose — ALL on ONE reused session (no re-handshake). Env-gated exactly like
/// <see cref="HandshakeReuseSpikeTests"/> (silent skip without HISTORIAN_GRPC_HOST +
/// HISTORIAN_WRITE_SANDBOX_TAG + HISTORIAN_USER). Bounded write to the sandbox tag only.
/// </summary>
public sealed class HistorianSessionRoundTripTests
{
private readonly ITestOutputHelper _output;
public HistorianSessionRoundTripTests(ITestOutputHelper output) => _output = output;
[Fact]
public async Task Session_WriteReadStatusPing_AllOnOneSession()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST");
string? sandboxTag = Environment.GetEnvironmentVariable("HISTORIAN_WRITE_SANDBOX_TAG");
if (string.IsNullOrWhiteSpace(host) || string.IsNullOrWhiteSpace(sandboxTag)
|| string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER")))
{
return; // skip — env not configured
}
HistorianClientOptions options = BuildOptions(host);
await using var client = new HistorianClient(options);
await using HistorianSession session = await client.OpenSessionAsync(HistorianSessionKind.WriteEnabled, CancellationToken.None);
// 1) write on the session
bool wrote = await session.AddHistoricalValuesAsync(sandboxTag,
new[] { new HistorianHistoricalValue(DateTime.UtcNow, 42.0, OpcQuality: 192) },
CancellationToken.None);
Assert.True(wrote);
_output.WriteLine("1) write -> ok");
// 2) read on the SAME session
DateTime end = DateTime.UtcNow, start = end - TimeSpan.FromDays(7);
var rows = new List<HistorianSample>();
await foreach (HistorianSample s in session.ReadRawAsync(sandboxTag, start, end, 8, CancellationToken.None))
rows.Add(s);
_output.WriteLine($"2) session read rows={rows.Count}");
Assert.NotEmpty(rows);
// 3) status + ping on the SAME session
string? version = await session.GetSystemParameterAsync("HistorianVersion", CancellationToken.None);
Assert.False(string.IsNullOrWhiteSpace(version));
_output.WriteLine($"3) HistorianVersion={version}");
await session.PingAsync(CancellationToken.None); // must not throw
_output.WriteLine("4) ping -> ok");
// 5) metadata + browse on the SAME session (no re-handshake)
HistorianTagMetadata? meta = await session.GetTagMetadataAsync(sandboxTag, CancellationToken.None);
Assert.NotNull(meta);
_output.WriteLine("5) metadata -> ok");
List<string> browsed = [];
await foreach (string n in session.BrowseTagNamesAsync("*", CancellationToken.None))
{
browsed.Add(n);
if (browsed.Count >= 5) break;
}
Assert.NotEmpty(browsed);
_output.WriteLine($"6) browse rows={browsed.Count}");
_output.WriteLine("session round-trip OK (write+read+status+ping+metadata+browse on one session)");
}
// verbatim copy of BuildOptions from HandshakeReuseSpikeTests
private static HistorianClientOptions BuildOptions(string host)
{
string? user = Environment.GetEnvironmentVariable("HISTORIAN_USER");
string? password = Environment.GetEnvironmentVariable("HISTORIAN_PASSWORD");
bool explicitCreds = !string.IsNullOrEmpty(user);
int port = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_GRPC_PORT"), out int parsed)
? parsed
: HistorianClientOptions.DefaultGrpcPort;
bool tls = string.Equals(Environment.GetEnvironmentVariable("HISTORIAN_GRPC_TLS"), "true", StringComparison.OrdinalIgnoreCase);
TimeSpan timeout = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_GRPC_TIMEOUT"), out int secs) && secs > 0
? TimeSpan.FromSeconds(secs)
: new HistorianClientOptions { Host = host }.RequestTimeout;
return new HistorianClientOptions
{
Host = host,
Port = port,
Transport = HistorianTransport.RemoteGrpc,
GrpcUseTls = tls,
AllowUntrustedServerCertificate = tls,
ServerDnsIdentity = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_DNSID"),
IntegratedSecurity = !explicitCreds,
UserName = user ?? string.Empty,
Password = password ?? string.Empty,
RequestTimeout = timeout,
Compression = true
};
}
}
@@ -62,6 +62,21 @@ public sealed class HistorianTagWriteProtocolTests
+ "09120052657465737453646B5772697465496E7432"
+ "FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF"
+ "09180053444B2077726974652D52452073616E64626F78207461670904004D44415302010100000001E8030000549B0E36EDDBDC011A030904007465737410270000000000000000F03FFE00")]
// Int8/UInt8 (H1 un-gate): prefix is the captured Double buffer; only the data-type code byte differs.
[InlineData(
AVEVA.Historian.Client.Models.HistorianDataType.Int8,
"RetestSdkWriteDouble", 0x01dcdbed24988f3aL,
"4E6703000100000004C6021900000000000000000000000000000000"
+ "09140052657465737453646B5772697465446F75626C65"
+ "FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF"
+ "09180053444B2077726974652D52452073616E64626F78207461670904004D44415302010100000001E80300003A8F9824EDDBDC011A030904007465737410270000000000000000F03FFE00")]
[InlineData(
AVEVA.Historian.Client.Models.HistorianDataType.UInt8,
"RetestSdkWriteDouble", 0x01dcdbed24988f3aL,
"4E6703000100000004C6023900000000000000000000000000000000"
+ "09140052657465737453646B5772697465446F75626C65"
+ "FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF"
+ "09180053444B2077726974652D52452073616E64626F78207461670904004D44415302010100000001E80300003A8F9824EDDBDC011A030904007465737410270000000000000000F03FFE00")]
public void SerializeAnalogCTagMetadata_PerDataType_MatchesCapturedNativeBytes(
AVEVA.Historian.Client.Models.HistorianDataType dataType,
string tagName,
@@ -249,6 +264,9 @@ public sealed class HistorianTagWriteProtocolTests
() => HistorianTagWriteProtocol.GetAnalogDataTypeCode(AVEVA.Historian.Client.Models.HistorianDataType.SingleByteString));
Assert.Throws<ProtocolEvidenceMissingException>(
() => HistorianTagWriteProtocol.GetAnalogDataTypeCode(AVEVA.Historian.Client.Models.HistorianDataType.Int1));
// UInt1 re-gated: historian creates a degenerate UInt1 analog tag (null type descriptor) — see pending notes.
Assert.Throws<ProtocolEvidenceMissingException>(
() => HistorianTagWriteProtocol.GetAnalogDataTypeCode(AVEVA.Historian.Client.Models.HistorianDataType.UInt1));
}
[Fact]
@@ -0,0 +1,22 @@
using System.Reflection;
using AVEVA.Historian.Client.Grpc;
using Xunit;
namespace AVEVA.Historian.Client.Tests;
public class TagClientOnSessionSeamTests
{
[Theory]
[InlineData("BrowseTagNamesOnSession")]
[InlineData("GetTagInfosRawOnSession")]
[InlineData("GetTagMetadataOnSession")]
public void TagClient_ExposesOnSessionSeam(string name)
{
MethodInfo? m = typeof(HistorianGrpcTagClient).GetMethod(
name, BindingFlags.NonPublic | BindingFlags.Static | BindingFlags.Public);
Assert.NotNull(m);
ParameterInfo[] ps = m!.GetParameters();
Assert.Equal("HistorianGrpcConnection", ps[0].ParameterType.Name);
Assert.Equal("Session", ps[1].ParameterType.Name);
}
}
@@ -73,6 +73,33 @@ public sealed class WcfEventQueryProtocolTests
Assert.Equal("6b955b02087047a3199a8c74f3eee85c3b49aaa29b05de12eff2dd536f2da0d5", attempt.RequestSha256);
}
[Fact]
public void Version6EmptyFilterMatchesCapturedGrpcEnvelope()
{
// Captured 2026-06-22 from the stock 2023 R2 client (docs/reverse-engineering/grpc-event-query-capture.md):
// the v6 StartEventQuery request is byte-identical to the v5 buffer except byte 0 (version 6) and a
// 5-byte trailing zero pad (70 vs 65 bytes). The 2023 R2 server returns event rows only for v6.
DateTime start = new DateTime(2026, 4, 25, 14, 39, 36, 800, DateTimeKind.Utc).AddTicks(1646);
DateTime end = new DateTime(2026, 5, 2, 14, 39, 36, 800, DateTimeKind.Utc).AddTicks(1646);
byte[] v5 = HistorianEventQueryProtocol.CreateStartEventQueryAttempts(start, end, 3)[0].RequestBuffer;
HistorianEventQueryAttempt v6Attempt = Assert.Single(
HistorianEventQueryProtocol.CreateStartEventQueryAttempts(start, end, 3, filter: null, version: 6));
byte[] v6 = v6Attempt.RequestBuffer;
Assert.Equal("native-empty-filter-version6", v6Attempt.Name);
Assert.Equal(6, v6Attempt.Version);
Assert.Equal(70, v6.Length);
Assert.Equal([0x06, 0x00], v6[..2]);
// v6 == v5 with byte 0 -> 6 and 5 trailing zero bytes appended.
byte[] expected = new byte[70];
Array.Copy(v5, expected, v5.Length);
expected[0] = 0x06;
Assert.Equal(expected, v6);
Assert.Equal([0x00, 0x00, 0x00, 0x00, 0x00], v6[^5..]);
}
[Fact]
public void NativeEmptyFilterAttemptMatchesDecompiledSaveOrder()
{
@@ -0,0 +1,164 @@
using System.Runtime.Versioning;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
using Xunit.Abstractions;
namespace AVEVA.Historian.Client.Tests;
/// <remarks>
/// C2 live diagnostic spike (pending.md C2): does the managed WCF event-read path return rows
/// against an event-bearing 2023 R2 historian? gRPC event reads are a proven server-side dead-end
/// (docs/reverse-engineering/grpc-event-query-capture.md); the WCF transport is C2's only listed
/// unblock but is itself unproven (the orchestrator documents native code 76 / 85 on this server).
///
/// Drives <see cref="HistorianWcfEventOrchestrator"/> directly over RemoteTcpIntegrated and dumps the
/// full native chain. It NEVER fails the suite (skip+log diagnostic) and is inert off Windows — the
/// GREEN/RED call is by reading the printed "events observed" count + native return codes.
///
/// Gated by <c>HISTORIAN_WCF_EVENT_HOST</c> (independent of the gRPC live vars so it never runs by
/// accident). Optional: <c>HISTORIAN_WCF_EVENT_PORT</c> (default 32568), <c>HISTORIAN_WCF_EVENT_USER</c>
/// + <c>HISTORIAN_WCF_EVENT_PASSWORD</c> (absent =&gt; IntegratedSecurity), <c>HISTORIAN_WCF_EVENT_SPN</c>
/// (Kerberos SPN override; the default is the LocalPipe identity and will not authenticate remotely),
/// <c>HISTORIAN_WCF_EVENT_DAYS</c> (lookback window, default 90 — the live event store held 71,332
/// events in -90d).
///
/// Run from the Windows capture rig over VPN:
/// dotnet test --filter "FullyQualifiedName~WcfEventReadSpike" -l "console;verbosity=detailed"
/// </remarks>
[SupportedOSPlatform("windows")]
public sealed class WcfEventReadSpikeTests
{
private readonly ITestOutputHelper _output;
public WcfEventReadSpikeTests(ITestOutputHelper output) => _output = output;
[Fact]
public async Task WcfEventRead_DiagnosticDump_AgainstRemoteHistorian()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_HOST");
bool certificate = string.Equals(
Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_TRANSPORT"), "certificate", StringComparison.OrdinalIgnoreCase);
// The certificate (TLS) transport and NegotiateAuthentication-based app auth are cross-platform;
// the integrated (Windows SSPI transport-security) transport is Windows-only. Skip when the gate
// is unconfigured, or when an integrated run is requested off Windows.
if (string.IsNullOrWhiteSpace(host) || (!certificate && !OperatingSystem.IsWindows()))
{
return;
}
int port = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_PORT"), out int p)
? p : HistorianClientOptions.DefaultPort; // 32568
int days = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_DAYS"), out int d)
? d : 90;
string? user = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_USER");
string? password = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_PASSWORD");
string? spn = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_SPN");
string? dnsId = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_DNSID");
string? via = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_VIA");
bool bypassVersion = string.Equals(
Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_NOVERSIONCHECK"), "1", StringComparison.Ordinal);
int timeoutSec = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_TIMEOUT_SEC"), out int ts) && ts > 0
? ts : 30;
string? connModeRaw = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_CONNMODE");
uint? connModeOverride = null;
if (!string.IsNullOrWhiteSpace(connModeRaw))
{
string hex = connModeRaw.StartsWith("0x", StringComparison.OrdinalIgnoreCase) ? connModeRaw[2..] : connModeRaw;
if (uint.TryParse(hex, System.Globalization.NumberStyles.HexNumber, null, out uint cm)) { connModeOverride = cm; }
}
bool allowUntrusted = string.Equals(
Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_ALLOW_UNTRUSTED"), "1", StringComparison.Ordinal);
// Certificate transport carries no Windows transport credential (cert-validated TLS channel);
// integrated derives one from the logged-in identity unless an explicit DOMAIN\user is supplied.
// App-level ValidateClientCredential still uses UserName/Password (or the process identity).
bool integrated = !certificate && string.IsNullOrEmpty(user);
HistorianClientOptions options = new()
{
Host = host,
Port = port,
Transport = certificate ? HistorianTransport.RemoteTcpCertificate : HistorianTransport.RemoteTcpIntegrated,
IntegratedSecurity = integrated,
UserName = user ?? string.Empty,
Password = password ?? string.Empty,
TargetSpn = string.IsNullOrWhiteSpace(spn) ? "NT SERVICE\\aahClientAccessPoint" : spn,
ServerDnsIdentity = string.IsNullOrWhiteSpace(dnsId) ? null : dnsId,
AllowUntrustedServerCertificate = allowUntrusted,
ConnectViaAddress = string.IsNullOrWhiteSpace(via) ? null : via,
VerifyServerInterfaceVersion = !bypassVersion,
ConnectTimeout = TimeSpan.FromSeconds(timeoutSec),
RequestTimeout = TimeSpan.FromSeconds(timeoutSec),
EventReadConnectionModeOverride = connModeOverride,
};
HistorianWcfEventOrchestrator orchestrator = new(options);
DateTime endUtc = DateTime.UtcNow;
DateTime startUtc = endUtc - TimeSpan.FromDays(days);
int budgetSec = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_BUDGET_SEC"), out int bgs) && bgs > 0
? bgs : 60;
int observed = 0;
bool hasFirstEvent = false;
string outcome = "completed";
try
{
// Race the read against an overall budget. The chain is synchronous WCF, so a stuck call
// can't be token-cancelled — but the orchestrator's static/instance diagnostics are set as it
// progresses, so on a budget timeout we still dump them to see which phase (auth / CM_EVENT
// registration / query) is hanging. The abandoned task keeps running in the background harness.
using var budget = new CancellationTokenSource(TimeSpan.FromSeconds(budgetSec));
Task<int> readTask = Task.Run(async () =>
{
int n = 0;
await foreach (HistorianEvent evt in orchestrator.ReadEventsAsync(startUtc, endUtc, filter: null, budget.Token))
{
n++;
_ = evt; // event identity intentionally NOT logged (sanitized)
}
return n;
});
Task finished = await Task.WhenAny(readTask, Task.Delay(TimeSpan.FromSeconds(budgetSec + 5)));
if (finished == readTask)
{
observed = await readTask;
hasFirstEvent = observed > 0;
}
else
{
outcome = $"TIMED OUT after {budgetSec}s (chain still running — see return codes for phase reached)";
}
}
catch (Exception ex)
{
// Default: type name only (sanitized). Opt into messages for live binding/protocol debugging
// via HISTORIAN_WCF_EVENT_VERBOSE=1 — binding/protocol errors may carry the endpoint host
// (already known to the operator) but never credentials; still off by default.
outcome = string.Equals(Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_VERBOSE"), "1", StringComparison.Ordinal)
? $"threw {ex.GetType().FullName} :: {ex.Message} | inner={ex.InnerException?.GetType().FullName} :: {ex.InnerException?.Message}"
: $"threw {ex.GetType().Name}";
}
// Sanitized diagnostic dump — counts, native return codes, buffer lengths, sha256 ONLY.
_output.WriteLine($"[C2 WCF spike] transport: {options.Transport} (integratedSec={integrated}, allowUntrusted={allowUntrusted}, connMode={(connModeOverride.HasValue ? "0x" + connModeOverride.Value.ToString("X") : "default-0x402")})");
_output.WriteLine($"[C2 WCF spike] outcome: {outcome}");
_output.WriteLine($"[C2 WCF spike] events observed: {observed}");
_output.WriteLine($"[C2 WCF spike] hasFirstEvent: {hasFirstEvent}");
_output.WriteLine($"[C2 WCF spike] LastUpdC3ReturnCode: {HistorianWcfEventOrchestrator.LastUpdC3ReturnCode}");
_output.WriteLine($"[C2 WCF spike] LastRTag2ReturnCode: {HistorianWcfEventOrchestrator.LastRTag2ReturnCode}");
_output.WriteLine($"[C2 WCF spike] LastAddReturnCode(EnsT2): {HistorianWcfEventOrchestrator.LastAddReturnCode}");
_output.WriteLine($"[C2 WCF spike] LastAddOutputLength: {HistorianWcfEventOrchestrator.LastAddOutputLength}");
_output.WriteLine($"[C2 WCF spike] LastEnsT2PayloadSha256: {HistorianWcfEventOrchestrator.LastEnsT2PayloadSha256}");
_output.WriteLine($"[C2 WCF spike] LastResultBufferLength: {orchestrator.LastResultBufferLength}");
// Contract-safe: LastErrorBufferDescription is DescribeNativeError's STRUCTURED formatting of the
// 5-byte native error buffer ("type=N code=M (0xHEX)" / "<short>") — never freeform server text,
// FQDN, SPN, or credentials. The code value (e.g. 76 / 85) is the RED-case signal this spike exists
// to capture, so it is dumped in full rather than reduced to a length.
_output.WriteLine($"[C2 WCF spike] LastErrorBufferDescription: {orchestrator.LastErrorBufferDescription}");
_output.WriteLine($"[C2 WCF spike] window days: {days}");
// Diagnostic: NEVER fails the suite. GREEN = observed > 0; RED = observed == 0 (read the output).
Assert.True(observed >= 0, "diagnostic always passes; the GREEN/RED signal is the printed count");
}
}
@@ -21,6 +21,11 @@ public sealed class WcfHistoricalWriteProtocolTests
"4f4e0100380000002e00" + "ca9735f7f841b244b56f9c14ccfeac32" + "b09bc72fd701dd01" + "c000" + "c0100100" + "104b59f3e701dd01" + "0000000039300000")]
[InlineData(HistorianDataType.UInt4, 305419896d,
"4f4e0100380000002e00" + "e7ae22d8e4cc65439ebd8bcb09402974" + "602d6663d701dd01" + "c000" + "c0100100" + "498af726e801dd01" + "0000000078563412")]
// Int8/UInt8 (H1 un-gate): prefix is the captured Double buffer; only the value bytes differ. Live round-trip confirms (gateway HistorianTypeRoundTripTests).
[InlineData(HistorianDataType.Int8, 100000d,
"4f4e01003c0000003200" + "7f970bb0b5a7344bb7bf6899ff06d027" + "c07d5f23d701dd01" + "c000" + "c0100100" + "08eff1e6e701dd01" + "00000000a086010000000000")]
[InlineData(HistorianDataType.UInt8, 100001d,
"4f4e01003c0000003200" + "7f970bb0b5a7344bb7bf6899ff06d027" + "c07d5f23d701dd01" + "c000" + "c0100100" + "08eff1e6e701dd01" + "00000000a186010000000000")]
public void SerializeAddStreamValuesBuffer_MatchesCapturedNativeBuffer(HistorianDataType dataType, double value, string capturedHex)
{
byte[] captured = Convert.FromHexString(capturedHex);
@@ -47,6 +52,10 @@ public sealed class WcfHistoricalWriteProtocolTests
Assert.Throws<ProtocolEvidenceMissingException>(() =>
HistorianHistoricalWriteProtocol.SerializeAddStreamValuesBuffer(
Guid.NewGuid(), HistorianDataType.SingleByteString, DateTime.UtcNow, 1.0, DateTime.UtcNow));
// UInt1 re-gated: historian creates a degenerate UInt1 analog tag (null type descriptor) — see pending notes.
Assert.Throws<ProtocolEvidenceMissingException>(() =>
HistorianHistoricalWriteProtocol.SerializeAddStreamValuesBuffer(
Guid.NewGuid(), HistorianDataType.UInt1, DateTime.UtcNow, 1.0, DateTime.UtcNow));
}
[Fact]
@@ -5,6 +5,70 @@ namespace AVEVA.Historian.Client.Tests;
public sealed class WcfOpen2ProtocolTests
{
[Fact]
public void Rc4MatchesStandardTestVector()
{
// Standard RC4 test vector (key "Key", plaintext "Plaintext" -> BBF316E8D940AF0AD3). Pins the
// RC4 used by the v8 credential token = RC4(password-UTF16LE, key=MD5(SHA256(ECDH secret))).
byte[] actual = HistorianNativeHandshake.Rc4(
System.Text.Encoding.ASCII.GetBytes("Key"),
System.Text.Encoding.ASCII.GetBytes("Plaintext"));
Assert.Equal(Convert.FromHexString("BBF316E8D940AF0AD3"), actual);
}
[Fact]
public void CredentialToken_IsRc4OfPasswordKeyedByMd5OfClientKey()
{
// token = RC4(password-UTF16LE, key = MD5(clientKey)). Verify against an independently computed
// reference for a fixed clientKey + password.
byte[] clientKey = Enumerable.Range(0, 32).Select(i => (byte)i).ToArray();
byte[] token = HistorianNativeHandshake.BuildExchangeKeyCredentialToken(clientKey, "Pw1!");
byte[] rc4Key = System.Security.Cryptography.MD5.HashData(clientKey);
byte[] expected = HistorianNativeHandshake.Rc4(rc4Key, System.Text.Encoding.Unicode.GetBytes("Pw1!"));
Assert.Equal(expected, token);
Assert.Equal("Pw1!".Length * 2, token.Length); // UTF-16LE length
}
[Fact]
public void Version8EventSerializerReproducesCapturedNativeStructure()
{
// Field sizes chosen to match the live 2023 R2 Event-connection capture (302 bytes):
// username=12, token=26, serverNode=15, clientNode=32, datasource=16.
// See docs/reverse-engineering/grpc-event-query-capture.md.
var clientKey = new Guid("11223344-5566-7788-99aa-bbccddeeff00");
byte[] actual = HistorianOpen2Protocol.SerializeNativeOpenConnectionVersion8(
new HistorianClientCommonInfo(
FormatVersion: 3,
ServerNodeName: new string('S', 15),
ClientNodeName: new string('C', 32),
ProcessId: 0x11223344,
HcalVersion: 18,
ProcessName: string.Empty,
Proxy: string.Empty,
DataSourceId: new string('D', 16),
ShardId: new Guid(Enumerable.Repeat((byte)0xFF, 16).ToArray()),
ClientVersion: 999_999,
ClientTimestamp: 0x01DD02426F9B6F6C,
ClientDllVersion: string.Empty),
clientKey,
connectionType: 1, // Event
connectionFlag: 1, // Event
userName: new string('U', 12),
credentialToken: new byte[26]);
Assert.Equal(302, actual.Length); // exact native length for these field sizes
Assert.Equal(0x08, actual[0]); // format version 8
Assert.Equal(0xF0, actual[1]); // marker
Assert.Equal(0x01, actual[21]); // marker
Assert.Equal(clientKey.ToByteArray(), actual[22..38]);
Assert.Equal(0x04, actual[94]); // ClientType
Assert.Equal(0x01, actual[95]); // ConnectionType = Event
Assert.Equal(0x01, actual[96]); // type flag = Event
Assert.Equal([0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF], actual[270..286]); // ShardId
}
[Fact]
public void LegacyVersion1SerializerMatchesDecompiledSaveOpenConnectionParamsLayout()
{
@@ -90,8 +90,12 @@ namespace AVEVA.Historian.Grpc2023CaptureHarness
return CaptureWrite(managedDll, args);
case "delete-tag":
return DeleteTag(managedDll, args);
case "capture-event":
return CaptureEvent(managedDll, args);
case "capture-send-event":
return CaptureSendEvent(managedDll, args);
default:
Console.Error.WriteLine($"Unknown scenario '{scenario}'. Supported: load-check, connect, capture-write, delete-tag.");
Console.Error.WriteLine($"Unknown scenario '{scenario}'. Supported: load-check, connect, capture-write, delete-tag, capture-event, capture-send-event.");
return 1;
}
}
@@ -114,7 +118,7 @@ namespace AVEVA.Historian.Grpc2023CaptureHarness
Type tagStatusType = Req(asm, "ArchestrA.HistorianTagStatus");
Type tagStatusListType = Req(asm, "ArchestrA.HistorianTagStatusList");
string server = GetOption(args, "--server") ?? "WONDER-SQL-VD03";
string server = GetOption(args, "--server") ?? Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST") ?? "localhost";
int port = int.TryParse(GetOption(args, "--port"), out int p) ? p : 32565;
string certName = GetOption(args, "--cert") ?? server;
string tagName = GetOption(args, "--tag") ?? "SdkM3CaptureSandbox";
@@ -201,7 +205,7 @@ namespace AVEVA.Historian.Grpc2023CaptureHarness
/// SendValues (the actual wire push) only runs with --commit. Run with --grpc-rewrite pointing
/// at the instrumented copy and AVEVA_HISTORIAN_RE_CAPTURE set to the output file.
/// Usage: capture-write --tag SdkM3CaptureSandbox [--create] [--commit]
/// [--server WONDER-SQL-VD03] [--port 32565] [--cert WONDER-SQL-VD03] [--value 123.0]
/// [--server <host>] [--port 32565] [--cert <host>] [--value 123.0]
/// </summary>
private static int CaptureWrite(string managedDll, string[] args)
{
@@ -221,7 +225,7 @@ namespace AVEVA.Historian.Grpc2023CaptureHarness
Type listType = Req(asm, "ArchestrA.HistorianDataValueList");
Type categoryEnum = Req(asm, "ArchestrA.HistorianDataCategory");
string server = GetOption(args, "--server") ?? "WONDER-SQL-VD03";
string server = GetOption(args, "--server") ?? Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST") ?? "localhost";
int port = int.TryParse(GetOption(args, "--port"), out int p) ? p : 32565;
string certName = GetOption(args, "--cert") ?? server;
string tagName = GetOption(args, "--tag") ?? "SdkM3CaptureSandbox";
@@ -429,12 +433,310 @@ namespace AVEVA.Historian.Grpc2023CaptureHarness
result = m.Invoke(target, a);
}
/// <summary>
/// Drives the native 2023 R2 client through a read-only gRPC EVENT query so the IL-rewritten
/// GrpcRetrievalClient dumps the uncaptured event buffers: StartEventQuery.requestBuffer (the
/// empty-filter request shape our SDK's CreateNativeEmptyFilterAttempt is being compared against)
/// and GetNextEventQueryResultBuffer.result (the row buffer — proves rows flow when driven right).
///
/// CRITICAL: the connection is opened with ConnectionType=Event (NOT Process). CreateEventQuery()
/// returns null unless IsEventConnectionRequested() — the native event read runs on ConnectionIndex 1,
/// a separate connection from the process/data path. This is the prime suspect for why the SDK's
/// gRPC empty-filter query returns zero rows despite the server holding events.
///
/// Sequence: OpenConnection(Event, read-only, gRPC) -> CreateEventQuery() ->
/// EventQueryArgs{StartDateTime,EndDateTime,EventCount} -> EventQuery.StartQuery(args) ->
/// loop EventQuery.MoveNext()/QueryResult -> EventQuery.EndQuery() -> CloseConnection.
/// Run with --grpc-rewrite pointing at the instrumented Archestra.Historian.GrpcClient.dll and
/// AVEVA_HISTORIAN_RE_CAPTURE set to the output NDJSON. Read-only — non-destructive.
/// Usage: capture-event [--server <host>] [--port 32565] [--cert <host>]
/// [--lookback-hours 720] [--max-events 50] [--integrated]
/// </summary>
private static int CaptureEvent(string managedDll, string[] args)
{
Assembly asm = Assembly.LoadFrom(managedDll);
Type accessType = Req(asm, "ArchestrA.HistorianAccess");
Type connArgsType = Req(asm, "ArchestrA.HistorianConnectionArgs");
Type connModeType = Req(asm, "ArchestrA.HistorianConnectionMode");
Type connTypeType = Req(asm, "ArchestrA.HistorianConnectionType");
Type errorType = Req(asm, "ArchestrA.HistorianAccessError");
Type statusType = Req(asm, "ArchestrA.HistorianConnectionStatus");
Type certInfoType = Req(asm, "ArchestrA.CertificateInfo");
Type secModeType = Req(asm, "ArchestrA.HistorianSecurityMode");
Type eventQueryType = Req(asm, "ArchestrA.EventQuery");
Type eventArgsType = Req(asm, "ArchestrA.EventQueryArgs");
string server = GetOption(args, "--server") ?? Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST") ?? "localhost";
int port = int.TryParse(GetOption(args, "--port"), out int p) ? p : 32565;
string certName = GetOption(args, "--cert") ?? server;
int lookbackHours = int.TryParse(GetOption(args, "--lookback-hours"), out int lh) ? lh : 720;
int maxEvents = int.TryParse(GetOption(args, "--max-events"), out int me) ? me : 50;
bool integrated = args.Contains("--integrated");
string? user = Environment.GetEnvironmentVariable("HISTORIAN_USER");
string? password = Environment.GetEnvironmentVariable("HISTORIAN_PASSWORD");
if (!integrated && string.IsNullOrEmpty(user))
{
Console.Error.WriteLine("Set HISTORIAN_USER/HISTORIAN_PASSWORD or pass --integrated.");
return 1;
}
object connArgs = Activator.CreateInstance(connArgsType)!;
SetProp(connArgs, "ServerName", server);
SetProp(connArgs, "TcpPort", checked((ushort)port));
SetProp(connArgs, "ConnectionMode", Enum.Parse(connModeType, "Historian")); // 2 = gRPC
SetProp(connArgs, "ConnectionType", Enum.Parse(connTypeType, "Event")); // EVENT connection
SetProp(connArgs, "ReadOnly", true);
SetProp(connArgs, "IntegratedSecurity", integrated);
SetProp(connArgs, "AllowUnTrustedConnection", true);
if (!integrated)
{
SetProp(connArgs, "UserName", user!);
SetProp(connArgs, "Password", password ?? string.Empty);
}
object certInfo = Activator.CreateInstance(certInfoType)!;
TrySetProp(certInfo, "CertificateName", certName);
TrySetProp(certInfo, "SecurityMode", Enum.Parse(secModeType, "TransportCertificate"));
TrySetProp(connArgs, "SecurityInfo", certInfo);
object access = Activator.CreateInstance(accessType)!;
object error = Activator.CreateInstance(errorType)!;
object?[] openArgs = { connArgs, error };
Console.WriteLine($"OpenConnection: server={server} port={port} mode=Historian type=Event cert={certName} integrated={integrated} readonly=true");
bool opened;
try
{
opened = (bool)accessType.GetMethod("OpenConnection", new[] { connArgsType, errorType.MakeByRefType() })!
.Invoke(access, openArgs)!;
}
catch (TargetInvocationException tie)
{
Console.Error.WriteLine($"OpenConnection threw: {tie.InnerException?.GetType().Name}: {tie.InnerException?.Message}");
return 2;
}
Console.WriteLine($"OpenConnection returned: {opened} err={DescribeError(openArgs[1])}");
if (!opened) { return 2; }
try
{
// Let the event connection (ConnectionIndex 1) come up.
MethodInfo getStatus = accessType.GetMethod("GetConnectionStatus", new[] { statusType.MakeByRefType() })
?? accessType.GetMethods().First(m => m.Name == "GetConnectionStatus" && m.GetParameters().Length == 1);
for (int i = 0; i < 10; i++)
{
object?[] sArgs = { null };
getStatus.Invoke(access, sArgs);
if (ReadBoolProp(sArgs[0], "ConnectedToServer") || !ReadBoolProp(sArgs[0], "Pending")) break;
System.Threading.Thread.Sleep(500);
}
// CreateEventQuery() is non-null only when the connection is an event connection.
MethodInfo createEventQuery = accessType.GetMethod("CreateEventQuery", Type.EmptyTypes)
?? accessType.GetMethods().First(m => m.Name == "CreateEventQuery" && m.GetParameters().Length == 0);
object? eventQuery = createEventQuery.Invoke(access, null);
Console.WriteLine($"CreateEventQuery: {(eventQuery == null ? "NULL (event connection not established!)" : "ok")}");
if (eventQuery == null) { return 3; }
// Build EventQueryArgs over the populated window. Times in UTC.
object eventArgs = Activator.CreateInstance(eventArgsType)!;
DateTime endUtc = DateTime.UtcNow;
DateTime startUtc = endUtc.AddHours(-lookbackHours);
TrySetProp(eventArgs, "StartDateTime", DateTime.SpecifyKind(startUtc, DateTimeKind.Utc));
TrySetProp(eventArgs, "EndDateTime", DateTime.SpecifyKind(endUtc, DateTimeKind.Utc));
TrySetProp(eventArgs, "EventCount", checked((uint)maxEvents));
Console.WriteLine($"EventQueryArgs: start={startUtc:o} end={endUtc:o} eventCount={maxEvents}");
// StartQuery -> triggers GrpcRetrievalClient.StartEventQuery (requestBuffer CAPTURED).
MethodInfo startQuery = eventQueryType.GetMethods()
.First(m => m.Name == "StartQuery" && m.GetParameters().Length == 2);
object?[] startArgs = { eventArgs, Activator.CreateInstance(errorType) };
bool started = (bool)startQuery.Invoke(eventQuery, startArgs)!;
Console.WriteLine($"StartQuery: {started} err={DescribeError(startArgs[1])}");
// Poll rows -> triggers GetNextEventQueryResultBuffer (result buffer CAPTURED).
MethodInfo moveNext = eventQueryType.GetMethods()
.First(m => m.Name == "MoveNext" && m.GetParameters().Length == 1);
PropertyInfo? queryResult = eventQueryType.GetProperty("QueryResult");
int rows = 0;
while (rows < maxEvents)
{
object?[] mnArgs = { Activator.CreateInstance(errorType) };
bool more;
try { more = (bool)moveNext.Invoke(eventQuery, mnArgs)!; }
catch (TargetInvocationException tie)
{
Console.WriteLine($"MoveNext threw: {tie.InnerException?.GetType().Name}: {tie.InnerException?.Message}");
break;
}
if (!more)
{
Console.WriteLine($"MoveNext: end after {rows} row(s) err={DescribeError(mnArgs[0])}");
break;
}
rows++;
if (rows <= 3 && queryResult != null)
{
// Print only the event TYPE + time (non-identity) to confirm rows flow.
object? res = queryResult.GetValue(eventQuery);
string typ = res?.GetType().GetProperty("Type")?.GetValue(res)?.ToString() ?? "?";
object? t = res?.GetType().GetProperty("EventTime")?.GetValue(res);
Console.WriteLine($" row {rows}: Type={typ} EventTime={t}");
}
}
Console.WriteLine($"Rows iterated: {rows}");
MethodInfo? endQuery = eventQueryType.GetMethods()
.FirstOrDefault(m => m.Name == "EndQuery" && m.GetParameters().Length == 1);
if (endQuery != null)
{
object?[] eqArgs = { Activator.CreateInstance(errorType) };
endQuery.Invoke(eventQuery, eqArgs);
}
Console.WriteLine(rows > 0 ? "CAPTURE-EVENT: PASS (rows flowed)" : "CAPTURE-EVENT: request captured (zero rows)");
return 0;
}
finally
{
try
{
MethodInfo? close = accessType.GetMethod("CloseConnection", new[] { errorType.MakeByRefType() });
if (close != null) close.Invoke(access, new object?[] { Activator.CreateInstance(errorType) });
}
catch { /* best-effort */ }
}
}
/// <summary>
/// Drives the native 2023 R2 client through an event SEND so the IL-rewritten GrpcClient dumps
/// the AddStreamValues.btValues (the event VTQ storage-sample buffer — resolves whether a gRPC
/// event send uses the "OS" or "ON" outer signature) AND the Event-connection EnsureTags.btTagInfos
/// (the 83-vs-86-byte CM_EVENT registration byte-diff). Opens a WRITE-ENABLED Event connection,
/// builds a clearly-marked test HistorianEvent, calls AddStreamedValue, then CloseConnection to
/// flush the queued event onto the wire. WRITES a real test event into the server's event history.
/// Run with --grpc-rewrite pointing at the instrumented copy and AVEVA_HISTORIAN_RE_CAPTURE set.
/// Usage: capture-send-event [--server <host>] [--port 32565] [--cert <host>]
/// [--event-type SdkCaptureProbe] [--flush-seconds 6]
/// </summary>
private static int CaptureSendEvent(string managedDll, string[] args)
{
Assembly asm = Assembly.LoadFrom(managedDll);
Type accessType = Req(asm, "ArchestrA.HistorianAccess");
Type connArgsType = Req(asm, "ArchestrA.HistorianConnectionArgs");
Type connModeType = Req(asm, "ArchestrA.HistorianConnectionMode");
Type connTypeType = Req(asm, "ArchestrA.HistorianConnectionType");
Type errorType = Req(asm, "ArchestrA.HistorianAccessError");
Type eventType = Req(asm, "ArchestrA.HistorianEvent");
Type propTypeEnum = Req(asm, "ArchestrA.HistorianEventPropertyType");
Type certInfoType = Req(asm, "ArchestrA.CertificateInfo");
Type secModeType = Req(asm, "ArchestrA.HistorianSecurityMode");
string server = GetOption(args, "--server") ?? Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST") ?? "localhost";
int port = int.TryParse(GetOption(args, "--port"), out int p) ? p : 32565;
string certName = GetOption(args, "--cert") ?? server;
string evtTypeName = GetOption(args, "--event-type") ?? "SdkCaptureProbe";
int flushSeconds = int.TryParse(GetOption(args, "--flush-seconds"), out int fs) ? fs : 6;
string? user = Environment.GetEnvironmentVariable("HISTORIAN_USER");
string? password = Environment.GetEnvironmentVariable("HISTORIAN_PASSWORD");
if (string.IsNullOrEmpty(user))
{
Console.Error.WriteLine("Set HISTORIAN_USER/HISTORIAN_PASSWORD.");
return 1;
}
if (string.IsNullOrWhiteSpace(Environment.GetEnvironmentVariable("AVEVA_HISTORIAN_RE_CAPTURE")))
{
string defaultCap = Path.GetFullPath(Path.Combine(
"artifacts", "reverse-engineering", "grpc-event-capture", "send-event-capture.ndjson"));
Environment.SetEnvironmentVariable("AVEVA_HISTORIAN_RE_CAPTURE", defaultCap);
}
Console.WriteLine($"Capture sink: {Environment.GetEnvironmentVariable("AVEVA_HISTORIAN_RE_CAPTURE")}");
object connArgs = Activator.CreateInstance(connArgsType)!;
SetProp(connArgs, "ServerName", server);
SetProp(connArgs, "TcpPort", checked((ushort)port));
SetProp(connArgs, "ConnectionMode", Enum.Parse(connModeType, "Historian"));
SetProp(connArgs, "ConnectionType", Enum.Parse(connTypeType, "Event")); // EVENT connection
SetProp(connArgs, "ReadOnly", false); // WRITE-enabled
SetProp(connArgs, "IntegratedSecurity", false);
SetProp(connArgs, "AllowUnTrustedConnection", true);
SetProp(connArgs, "UserName", user);
SetProp(connArgs, "Password", password ?? string.Empty);
object certInfo = Activator.CreateInstance(certInfoType)!;
TrySetProp(certInfo, "CertificateName", certName);
TrySetProp(certInfo, "SecurityMode", Enum.Parse(secModeType, "TransportCertificate"));
TrySetProp(connArgs, "SecurityInfo", certInfo);
object access = Activator.CreateInstance(accessType)!;
object?[] openArgs = { connArgs, Activator.CreateInstance(errorType) };
Console.WriteLine($"OpenConnection: server={server} port={port} type=Event readonly=false");
bool opened;
try
{
opened = (bool)accessType.GetMethod("OpenConnection", new[] { connArgsType, errorType.MakeByRefType() })!
.Invoke(access, openArgs)!;
}
catch (TargetInvocationException tie)
{
Console.Error.WriteLine($"OpenConnection threw: {tie.InnerException?.GetType().Name}: {tie.InnerException?.Message}");
return 2;
}
Console.WriteLine($"OpenConnection returned: {opened} err={DescribeError(openArgs[1])}");
if (!opened) { return 2; }
try
{
// Build a clearly-marked test event. Required: Type (≤32 chars), Id, EventTime.
object evt = Activator.CreateInstance(eventType)!;
SetProp(evt, "Type", evtTypeName);
TrySetProp(evt, "Id", Guid.NewGuid());
TrySetProp(evt, "EventTime", DateTime.SpecifyKind(DateTime.UtcNow, DateTimeKind.Utc));
TrySetProp(evt, "Namespace", "SdkCapture");
TrySetProp(evt, "Source", "SdkCaptureProbe");
// One string property to exercise the property-bag framing.
MethodInfo? addProp = eventType.GetMethods().FirstOrDefault(m =>
m.Name == "AddProperty" && m.GetParameters().Length == 4);
if (addProp != null)
{
try
{
object strEnum = Enum.Parse(propTypeEnum, "String", true);
object?[] apArgs = { "SdkProbeProp", "SdkProbeValue", strEnum, Activator.CreateInstance(errorType) };
addProp.Invoke(evt, apArgs);
Console.WriteLine($"AddProperty: err={DescribeError(apArgs[3])}");
}
catch (Exception ex) { Console.WriteLine($"AddProperty skipped: {ex.GetType().Name}"); }
}
MethodInfo addStreamed = accessType.GetMethods().First(m =>
m.Name == "AddStreamedValue" && m.GetParameters().Length == 2
&& m.GetParameters()[0].ParameterType == eventType);
object?[] asArgs = { evt, Activator.CreateInstance(errorType) };
bool sent = (bool)addStreamed.Invoke(access, asArgs)!;
Console.WriteLine($"AddStreamedValue({evtTypeName}): {sent} err={DescribeError(asArgs[1])}");
// Let the native delivery queue flush the event onto the wire (AddStreamValues).
System.Threading.Thread.Sleep(flushSeconds * 1000);
Console.WriteLine(sent ? "CAPTURE-SEND-EVENT: AddStreamedValue accepted (buffer captured on flush)" : "CAPTURE-SEND-EVENT: AddStreamedValue rejected");
return sent ? 0 : 3;
}
finally
{
try
{
// CloseConnection flushes any remaining queued values before teardown.
MethodInfo? close = accessType.GetMethod("CloseConnection", new[] { errorType.MakeByRefType() });
if (close != null) close.Invoke(access, new object?[] { Activator.CreateInstance(errorType) });
}
catch { /* best-effort */ }
}
}
/// <summary>
/// Read-only gRPC connect probe: opens a 2023 R2 Historian (mode=Historian) connection via the
/// native client and reports the resulting connection status. Proves the mixed-mode client can
/// reach the live server over gRPC from this box — the foundation for the write-capture step.
/// Reads creds from HISTORIAN_USER / HISTORIAN_PASSWORD (explicit) or uses IntegratedSecurity.
/// Usage: connect --server WONDER-SQL-VD03 [--port 32565] [--cert WONDER-SQL-VD03] [--integrated]
/// Usage: connect --server <host> [--port 32565] [--cert <host>] [--integrated]
/// </summary>
private static int Connect(string managedDll, string[] args)
{
@@ -448,7 +750,7 @@ namespace AVEVA.Historian.Grpc2023CaptureHarness
Type certInfoType = Req(asm, "ArchestrA.CertificateInfo");
Type secModeType = Req(asm, "ArchestrA.HistorianSecurityMode");
string server = GetOption(args, "--server") ?? "WONDER-SQL-VD03";
string server = GetOption(args, "--server") ?? Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST") ?? "localhost";
int port = int.TryParse(GetOption(args, "--port"), out int p) ? p : 32565;
string certName = GetOption(args, "--cert") ?? server;
bool integrated = args.Contains("--integrated");
@@ -1387,10 +1387,14 @@ static int InstrumentGrpcNonStream(string[] args)
ModuleDefMD module = ModuleDefMD.Load(sourcePath);
MemberRefUser logByteArray = CreateLogByteArrayRef(module);
MemberRefUser logString = CreateLogStringRef(module);
MemberRefUser logUInt32 = CreateLogUInt32Ref(module);
// Cast a wide net: instrument EVERY byte[]-input method on every Grpc*Client type, so whichever
// path the native non-streamed write actually drives (History/Transaction RegisterTags +
// AddNonStreamValues, or a Storage-service route) is captured. Phase = "<Type>.<Method>.<param>".
// Also captures string (strHandle) and uint (uiHandle / queryRequestType) inputs so the event-read
// handle fields are visible, not just byte[] params.
var instrumented = new List<object>();
foreach (TypeDef type in module.GetTypes()
.Where(t => t.Name.String.StartsWith("Grpc", StringComparison.Ordinal) && t.Name.String.EndsWith("Client", StringComparison.Ordinal)))
@@ -1431,6 +1435,34 @@ static int InstrumentGrpcNonStream(string[] args)
});
}
// ENTRY: log string (strHandle) and uint (uiHandle / queryRequestType / count) inputs.
foreach (dnlib.DotNet.Parameter scalarParam in method.Parameters
.Where(p => !p.IsHiddenThisParameter && (p.Type.FullName == "System.String" || p.Type.FullName == "System.UInt32"))
.ToArray())
{
bool isString = scalarParam.Type.FullName == "System.String";
string phase = $"{type.Name}.{method.Name}.{scalarParam.Name}.{(isString ? "str" : "u32")}";
Instruction[] injected =
[
Instruction.Create(OpCodes.Ldstr, phase),
Instruction.Create(OpCodes.Ldarg, scalarParam),
Instruction.Create(OpCodes.Call, isString ? logString : logUInt32),
];
foreach (Instruction instruction in injected.Reverse())
{
method.Body.Instructions.Insert(0, instruction);
}
method.Body.MaxStack = (ushort)Math.Max((int)method.Body.MaxStack, 8);
instrumented.Add(new
{
Type = type.Name.String,
Method = method.Name.String,
Phase = phase,
Direction = "in",
Token = "0x" + method.MDToken.Raw.ToString("X8"),
});
}
// EXIT: log out/ref byte[] responses ("System.Byte[]&") before each ret. ldarg loads the
// managed pointer; ldind.ref dereferences it to the byte[]. (RPC wrappers set the out
// param right before a single ret, so branch-to-ret skew is not a concern here.)