181 Commits

Author SHA1 Message Date
dohertj2 f0a1b04b34 Merge pull request 'feat(wcf): C2 spike + ConnectViaAddress/connmode — WCF transport viable, rows server-gated' (#1) from feat/c2-wcf-event-spike into main 2026-06-26 06:48:01 -04:00
Joseph Doherty f2297315b9 docs(c2): correct the WCF finding — transport+auth viable, row-retrieval server-gated
Overturns the earlier wrong "WCF not served on 2023 R2" conclusion (that was a
test error: wrong port/transport for the reverse tunnel). Corrected: the cert
(TLS) transport + NegotiateAuthentication auth reach the 2023 R2 historian
cross-platform; the 0x501 event connection mode makes CM_EVENT RegisterTags
succeed; yet StartEventQuery returns a 0-row buffer + long-polls over a window
that has events. Registration and window ruled out -> the same server-side
per-connection row gate as gRPC. Event reads stay server-gated over BOTH
transports; not client-fixable. Evidence doc rewritten; gRPC + WCF orchestrator
gating messages corrected.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-26 04:41:21 -04:00
Joseph Doherty de8d5e91ce feat(wcf): EventReadConnectionModeOverride + cross-platform/bounded C2 spike
Live investigation (direct from a VPN host to the 2023 R2 historian's real WCF
port) showed the certificate transport + NegotiateAuthentication auth work
cross-platform, and that the event-read chain needs the 0x501 event connection
mode for CM_EVENT RegisterTags to succeed (0x402/0x401 fail). Even with
registration succeeding over a window that has events, StartEventQuery returns a
0-row header and long-polls — the same server-side per-connection row gate proven
for gRPC. Adds: EventReadConnectionModeOverride (diagnostic), and spike knobs —
cross-platform cert gate, version-check bypass, per-call timeout, overall budget
with phase-diagnostic dump, connection-mode override.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-26 04:38:58 -04:00
Joseph Doherty 8777c0b816 fix(wcf): set Via via CreateChannel(address, via) — ClientViaBehavior absent in .NET WCF
ClientViaBehavior is a .NET Framework type not present in the System.ServiceModel
client libraries. Use the portable ChannelFactory.CreateChannel(EndpointAddress, Uri)
overload instead, via a CreateChannel helper applied at the history-open and
retrieval-query sites (the critical event path). Fixes the build break in 954b9cc.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 20:37:38 -04:00
Joseph Doherty 954b9cc9cc feat(wcf): add ConnectViaAddress (WCF Via) for tunneled historian access + wire into C2 spike
When the historian is reached through a port-forward whose local port differs
from the server's real service port, WCF's server-side AddressFilter rejects the
message (To = tunnel port != server port). ConnectViaAddress lets the channel
connect to the tunnel while addressing the SOAP To the real Host/Port endpoint.
Applied in HistorianWcfClientCredentialsHelper.Configure (the critical event
factories already call it). The C2 spike reads HISTORIAN_WCF_EVENT_VIA.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 20:35:46 -04:00
Joseph Doherty 7992e43908 test(c2): make WCF spike transport-selectable (integrated|certificate) + opt-in verbose
The first live run used the wrong port (32568 direct vs the 42568 WCF tunnel) and
hardcoded RemoteTcpIntegrated; via the tunnel the error advanced from socket-RST
to ProtocolException (binding/security mismatch). Add HISTORIAN_WCF_EVENT_TRANSPORT
(certificate), _DNSID, _ALLOW_UNTRUSTED, and an opt-in _VERBOSE for live binding
diagnosis. Default output stays sanitized; still Windows-only, never fails the suite.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 20:18:23 -04:00
Joseph Doherty 64c9793b91 docs(c2): correct event-read gating messages — WCF not served on 2023 R2
The gRPC ReadEvents throws no longer advise "use the WCF transport for event
reads": that path is moot on 2023 R2 (net.tcp is reset at the framing layer,
live-disproven 2026-06-25). Messages now state event reads are auth-solved but
server-gated over gRPC and have no WCF fallback on 2023 R2, citing the two
evidence docs. WCF orchestrator remarks scoped to legacy 2020 historians; row
layout noted as decoded. String/comment only; throw behavior unchanged.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 17:11:19 -04:00
Joseph Doherty f1c57f7149 docs(c2): record WCF event-read spike live result (RED — transport not served on 2023 R2)
WCF net.tcp (RemoteTcpIntegrated) against the live 2023 R2 historian is reset
at the socket-write/framing layer before any auth — both the event spike and a
basic Probe/ReadRaw throw the identical CommunicationException/SocketException
("forcibly closed by the remote host"). The 2023 R2 box does not serve the
legacy WCF transport; C2's "route via WCF" unblock is moot on this server class.
Sanitized: counts + native return codes + buffer lengths only.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 16:58:56 -04:00
Joseph Doherty 85f3bd4b4e test(c2): env-gated WCF event-read diagnostic spike (RemoteTcpIntegrated)
Drives HistorianWcfEventOrchestrator over RemoteTcpIntegrated and dumps the
native chain (UpdC3/RTag2/EnsT2 return codes, result-buffer length, row count)
to settle whether WCF event reads return rows on an event-bearing 2023 R2 box.
Windows-only, gated by HISTORIAN_WCF_EVENT_HOST, never fails the suite, inert
off Windows. Sanitized: counts + return codes + buffer lengths + sha256 only.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 16:40:21 -04:00
Joseph Doherty 5a7a28872b Merge feat/c1-numeric-writes: Int8/UInt8 live writes + C3a evidence (UInt1 server-blocked)
Int8/UInt8 analog tag-create + historical-value write encoders un-gated and
live-proven; golden-pinned. UInt1 attempted but re-gated — the historian stores a
degenerate UInt1 analog tag. 2023 R2 gRPC interface-version integers captured (C3a).

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 16:07:24 -04:00
Joseph Doherty 100b44a365 docs(write): drop UInt1 from value-layout table; mark Int8/UInt8 live-proven
Removes the stale UInt1 → UInt8(1) entry from the EncodeNativeValue layout
table (UInt1 is re-gated; the prose already said so). Int8/UInt8 layout note
updated from "pending" to live-proven.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 15:42:50 -04:00
Joseph Doherty aa8ca2f6ad docs: Int8/UInt8 analog writes supported (live-proven); UInt1 server-degenerate
EnsureTagAsync + AddHistoricalValues now cover Int8/UInt8 (codes 0x19/0x39, native
LE int64/uint64 value). UInt1 documented as not-supported: the server stores a
degenerate analog tag (GetTagInfo stub, type byte 0x00).

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 15:34:34 -04:00
Joseph Doherty 95b924cdbe fix(write): re-gate UInt1 — historian creates degenerate UInt1 analog tags
Live evidence: EnsureTags(UInt1) returns success but the server stores a
degenerate tag (descriptor type byte 0x00, no GUID/name), so GetTagInfo
truncates and writes fail. Not client-fixable on the analog path. Int8/UInt8
stay GREEN (live-proven). UInt1 reverts to ProtocolEvidenceMissingException
(fail-closed); golden rows removed; negative-gate tests added. Removed the
one-off capture diagnostic.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 15:27:04 -04:00
Joseph Doherty aa56d2d81b docs(re): correct Status interface-version comments (4 on gRPC, not 0)
Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 15:03:22 -04:00
Joseph Doherty dea9107b4b docs(re): capture + record 2023 R2 gRPC interface-version integers (C3a)
Env-gated live evidence test reads History/Retrieval/Transaction/Status
GetInterfaceVersion over gRPC; integers recorded in grpc-interface-versions.md.
Stale not-yet-captured comment fixed; gate XML-doc notes live confirmation.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 14:58:45 -04:00
Joseph Doherty 79bb1d9e06 test(write): golden-pin UInt1/Int8/UInt8 tag-create + value buffers
New InlineData rows derived from the captured Double baselines (type-code /
value bytes swapped). Negative-gate tests retained for still-gated types.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 14:49:21 -04:00
Joseph Doherty 43c2587498 feat(write): un-gate UInt1/Int8/UInt8 tag-create + historical-value encoders
GetAnalogDataTypeCode gains UInt1=0x08/Int8=0x19/UInt8=0x39 (read-side
MapDataType already maps these); EncodeNativeValue gains 1-byte UInt1 +
8-byte LE Int8/UInt8 (mirrors the captured Double value layout). Value API
stays double; 2^53 exact-magnitude ceiling documented. Golden tests + live
round-trip follow.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 14:38:27 -04:00
Joseph Doherty e04eb539f7 Merge feat/amortization-broadening: event-reuse spike + HistorianEventSession + browse/metadata seams
Track A (tag-client …OnSession seams + HistorianSession browse/metadata), the GREEN
event-session reuse spike (B0), and the HistorianEventSession primitive. Live-validated.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 12:58:45 -04:00
Joseph Doherty 2687b2b6d2 feat: HistorianEventSession primitive + OpenEventSessionAsync (v8 Event reuse)
Reusable v8 Event session wrapping the B0a seams (OpenAndRegisterEventSession once +
SendEventOnSession per op) with a GetSystemParameter keepalive; idempotent dispose.
Mirrors HistorianSession (v6 sibling). pending.md A1 broadening, Stage B1.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 11:54:09 -04:00
Joseph Doherty 5f949a86e2 docs(spike): event-session reuse spike results — GREEN
v8 Event session reuses across SendEvent (~10-16x amortization); register-once
sufficient; session survived 25s idle; event reads stay gated (C2). Live-validated
against wonder-sql-vd03. Gate decision: GREEN -> HistorianGateway Stage B1 (separate
event-session pool for SendEvent) warranted.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 11:34:34 -04:00
Joseph Doherty 81da404c5d test(spike): bound event read-after-send so the spike can't hang on C2 long-poll
The Q3 read-after-send probe (ReusedEventSession_ServesReadAfterSend_BestEffort)
long-polled GetNext to the no-data terminal with no tight bound and ran past the
5-min suite timeout on the live run. Bound it two ways: a read-only options copy
with a 5s RequestTimeout (so each GetNext RPC deadlines fast) and an 8s
CancellationToken passed as ct. Either fuse returns the method in ~10s; the
timeout/cancellation is logged as the expected C2-gated outcome (still no assert).
Other three spike methods unchanged.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 11:32:09 -04:00
Joseph Doherty 777a7700b4 test(spike): event-session reuse spike harness (env-gated, B0b)
Opens one v8 Event session and measures SendEvent reuse (register-once, send-many)
+ best-effort read-after-send + optional idle sweep. Skips offline; run live in B0c
to gate event amortization (pending.md A1 broadening, Stage B0).

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 11:02:34 -04:00
Joseph Doherty dc4141e718 feat(grpc): event-on-session seam for the reuse spike (SendEvent[+ReadEvents])
Extract SendEventOnSession (and best-effort RunEventQueryOnSession) so the B0b spike
can run multiple event ops on one already-opened v8 Event session. RegisterCmEventTag
made independently callable. Behaviour-preserving (pending.md A1 broadening, Stage B0).

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 10:39:40 -04:00
Joseph Doherty 81aff03748 feat: HistorianSession browse + metadata on the reused session (+ env-gated round-trip)
Browse mirrors ReadRawAsync (collect-then-yield); metadata mirrors ReadAtTimeAsync
(unary). Delegates to the HistorianGrpcTagClient …OnSession seams so a leased session
browses + reads metadata without re-handshaking (pending.md A1 broadening).

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 10:28:44 -04:00
Joseph Doherty 2f689cbe71 feat(grpc): …OnSession seams on HistorianGrpcTagClient (browse + metadata)
DRY split mirroring HistorianGrpcReadOrchestrator.RunRawQueryOnSession: browse +
GetTagInfos(metadata) gain externally-supplied connection+session seams; per-call
wrappers delegate. Behaviour-preserving (pending.md A1 broadening).

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 10:23:40 -04:00
Joseph Doherty be60d0b8d9 test: HistorianSession end-to-end round-trip (write+read+status+ping on one session, env-gated)
Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 03:09:18 -04:00
Joseph Doherty 15da8516ba feat: HistorianClient.OpenSessionAsync(kind) — open a reusable HistorianSession 2026-06-25 03:07:02 -04:00
Joseph Doherty d2b00fcda5 fix: idempotent HistorianSession.DisposeAsync + upfront cancellation check 2026-06-25 03:05:10 -04:00
Joseph Doherty ad2c02eb42 feat: HistorianSession primitive — reusable authenticated session over the …OnSession seams 2026-06-25 02:59:36 -04:00
Joseph Doherty d42019f481 feat(grpc): HistorianSessionKind + …OnSession seams (read/status/tag-write) 2026-06-25 02:52:32 -04:00
Joseph Doherty a0b5d35e48 docs: write-spike results — write-reuse GREEN, ONE-KIND pool confirmed 2026-06-25 02:11:10 -04:00
Joseph Doherty 9909921e87 test(grpc): write-reuse + read-on-write-session probe (one-vs-two-kind decider) 2026-06-25 02:08:23 -04:00
Joseph Doherty f8db01fd7f feat(grpc): add RunWriteOnSession seam for write-reuse spike 2026-06-25 02:04:52 -04:00
Joseph Doherty 3849f17746 docs: handshake-reuse spike results + verdict (GREEN, idle timeout ~20-25s) 2026-06-25 01:19:50 -04:00
Joseph Doherty 899f9ccf6b test(grpc): env-gated handshake-reuse spike (validity + latency + idle sweep) 2026-06-25 01:10:19 -04:00
Joseph Doherty 96f10372de feat(grpc): add RunRawQueryOnSession seam for handshake-reuse spike 2026-06-25 01:06:03 -04:00
Joseph Doherty 7d8be48d89 Merge handoff-sendevent-persistence: record SendEvent gRPC persistence finding
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 17:36:36 -04:00
Joseph Doherty f2c442bbaf handoff: record SendEvent gRPC persistence finding
Item 3: the gRPC read-back against the live 2023 R2 server proves the SDK-sent
event PERSISTS (independently read back from event history), resolving the older
WCF/M2 caveat ("server accepts AddS2 but the local dev box does not persist to
v_AlarmEventHistory2") as an environment artifact, not an SDK gap. Also retire the
stale "fresh native capture (SendEvent gRPC framing)" next-step note — SendEvent
over gRPC is now shipped + live-validated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 17:35:09 -04:00
Joseph Doherty 32d508eed4 Merge doc-sendevent-claudemd: document SendEventAsync in CLAUDE.md
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 17:30:27 -04:00
Joseph Doherty df9751f066 CLAUDE.md: document SendEventAsync (both transports)
Add SendEventAsync to the write surface: M2 event-send, now on both transports.
WCF runs Open2 event-mode -> CM_EVENT registration -> AddS2; gRPC runs v8 Event
OpenConnection -> registration -> HistoryService.AddStreamValues. Both carry the
same "OS" event VTQ buffer (no distinct event RPC, not the "ON" historical buffer);
gRPC path live-validated end-to-end. Only original events with string properties.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 15:55:13 -04:00
Joseph Doherty 5ad1adb429 Merge handoff-refresh-counts: SendEvent over gRPC + 2023 R2 binary-dive verdicts
- SendEvent over gRPC SHIPPED + live-validated (HistorianGrpcEventWriteOrchestrator;
  reuses the WCF "OS" event buffer verbatim on HistoryService.AddStreamValues).
- 2023 R2 stock-client binary dive: sharpened every pending item to evidence-based
  verdicts; DeleteTagExtendedProperties confirmed walled via native capture.
- Event registration (RegisterTags/EnsureTags) golden-tested vs live capture.
- Handoff date/test-count refresh; revision-probe scope comment fix.
331 offline tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 15:52:56 -04:00
Joseph Doherty dd57d212f8 DeleteTagExtendedProperties: confirm walled via native capture (do not ship)
Tested the binary-dive "use deleteFromServer=true" hypothesis directly against
the native client (local 2020 WCF box, Capture-DeleteTagExtendedProperties.ps1
cross-session sync trick). Result: the native DeleteTagExtendedPropertiesByName
with deleteFromServer=true returns Success=true, but the property is re-fetchable
and re-deletable across repeated fresh sessions — it is NEVER durably removed.

So the native client itself only performs an optimistic client-side cache delete
the server does not durably honor (the HCAL cache-sync model the decompile shows).
This supersedes the earlier "code=1, prop survives" note (that was the same-session
sync-gate failure; with proper cross-session sync it returns Success yet still does
not durably delete). A managed DeleteTagExtendedPropertiesAsync would return a
misleading success, so it correctly stays unshipped. Handoff item 7 updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 15:44:53 -04:00
Joseph Doherty afc7c4bf96 SendEvent over gRPC: implement + live-validate (was capture-gated)
Captured the native 2023 R2 client's gRPC event send (new capture-send-event
harness scenario): it rides HistoryService.AddStreamValues with the SAME "OS"
(0x534F) storage-sample buffer the WCF path already uses (HistorianEventWrite-
Protocol) — confirming "no distinct RPC", and that it is NOT the historical
write's "ON" buffer. Diffed the write-enabled vs read-only v8 Event open: byte-
identical apart from per-session crypto, so the existing OpenSession event path
is reused unchanged.

So SendEvent-over-gRPC was pure assembly of proven parts:
- HistorianGrpcEventWriteOrchestrator = v8 Event open + CM_EVENT registration
  (UpdC3/RegisterTags/EnsureTags) + AddStreamValues(OS buffer).
- HistorianClient.SendEventAsync now routes to it for RemoteGrpc (WCF otherwise).

Live-validated end-to-end against the 2023 R2 server: pure-managed SDK send →
AddStreamValues BSuccess=true → the event reads back from the server (markers
confirmed in returned event rows). The native gRPC RegisterTags(24B) +
EnsureTags(86B) byte-match our serializers (new GrpcEventSendProtocolTests
golden, closing the 83-vs-86 EnsureTags question). Gated live test
SendEventAsync_OverGrpc_AcceptsEvent (opt-in HISTORIAN_GRPC_EVENT_SEND=1).
331 offline tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 15:37:22 -04:00
Joseph Doherty ae536bb4b8 Record 2023 R2 binary-dive verdicts; fix revision-probe scope comment
Mined the full decompiled stock 2023 R2 managed client as the oracle for every
still-pending gRPC item. Governing fact: ArchestrA.HistorianAccess is a C++/CLI
shim into native HistorianClient; the managed Grpc*Client wrappers have zero
call sites, so buffer-building/dispatch for the pending items is native (absent
from the binaries). Sharpened verdicts into handoff.md:

- Items 4/5/6 + OpenStorageConnection: hard-confirmed walled, with real reasons
  (SQL gated out client-side via IsManagedHistorian; no Revision RPC in the gRPC
  contract; LoadBlocks response is a native blob behind the storage console handle).
- Items 3 (SendEvent) and 7 (DeleteTEP): moved from vague to precise, LOCAL-box
  capturable targets (PackToVtq btValues / DeleteTagExtendedPropertiesByName
  BtInput with deleteFromServer=true).

Also correct the HistorianGrpcRevisionProbe doc comment: it probes the
non-streamed ORIGINAL-insert path (AddNonStreamValues), a distinct capability
from revision EDITS (native AddRevisionValues trio, no gRPC RPC) — the prior
comment conflated them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 15:13:11 -04:00
Joseph Doherty cac81c7e5c handoff: refresh date + offline test count after event-parser merge
Bring the doc header current with the merged event-row parser fix: bump
"Last updated" to 2026-06-23 and correct the Build And Test known-good
result from 321/321 to the actual 328/328 (build clean 0/0), noting the
+7 are the HistorianEventRowProtocolTests golden + gated-capture coverage.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 14:40:57 -04:00
Joseph Doherty 0037ab8ca9 Merge handoff-event-parser-update: record event-row parser fix + mark gRPC event angles exhausted
Docs-only. Updates handoff.md item 1 to reflect that all four next-session angles for the
gRPC event zero-rows are tested and ruled out (transport, metadata/cert, topology, data
store) and that the parse path is verified against the provided client with a latent
multi-row bug fixed. Corrects two historical event-row-layout skeletons that described
0x1E as a per-row marker (it is a one-time buffer header field; rows are markerless).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 14:13:39 -04:00
Joseph Doherty 6a67a8366c handoff: record the event-row parser fix + mark the gRPC event angles exhausted
Updated the live-blocker item (1) to reflect that all four next-session angles for the
gRPC event zero-rows are now tested and ruled out (transport, metadata/cert, topology,
data store) and that the parse path is verified against the provided client with a latent
multi-row bug fixed. Corrected the two historical event-row-layout skeletons that wrongly
described 0x1E as a per-row marker: it is a one-time buffer header field and rows are
markerless (which is why the old parser returned only the first row of any multi-row
buffer, on WCF too).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 14:12:45 -04:00
Joseph Doherty 6faf8a5f30 Merge grpc-event-decompile-confirm: stock-client decompile + frame capture + event-parser fix
Three steps verifying the gRPC event read against the provided 2023 R2 client:

- Decompiled the stock managed client (Archestra.Historian.GrpcClient, HistorianAccess):
  confirms no hidden client-side difference. The stock client is gRPC-Web/HTTP-1.1 (same
  transport as ours), m_metadata is gzip-only, the ClientInterceptor is a no-op, and it
  presents no TLS client cert. The event-query orchestration lives in the native C++ core.
- Captured decrypted HTTP/1.1 frames of a native capture-event (50 rows) vs our SDK (0
  rows) through a TLS-terminating tee proxy. Found the native splits services across 5
  connections and runs the event query on a dedicated RetrievalService connection; tested
  replicating that (HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL) -> still 0 rows, so the server
  correlates by session handle, not connection. Topology is not the gate.
- Verified the parse path against the provided client's real result buffer (50 events) and
  fixed a latent bug: the event-row buffer is version + rowCount + a one-time 0x1E header
  field then MARKERLESS rows; the parser wrongly treated 0x1E as a per-row marker and
  decoded only the first row of any multi-row buffer. This also affected the shipped WCF
  event read (identical v9 header). Fixed to a 10-byte header + markerless rows, accepting
  version 9 (WCF) and 11 (gRPC). The real 50-row buffer now decodes to exactly 50 events.

Net: every client-side angle for the gRPC zero-rows is exhausted (payload, transport,
metadata/cert, topology, data store) -> the gate is a server-internal per-connection
retrieval working-set. The parse path is now verified against real 2023 R2 event data on
both transports, and WCF event reads now correctly return all rows of a multi-row buffer.
gRPC event-row retrieval stays auth-solved / parse-verified / retrieval-server-gated.
328 offline tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 14:08:10 -04:00
Joseph Doherty 8f4a188f78 Event-row parser: verify against the provided 2023 R2 client; fix latent multi-row bug
Used the provided stock client as an oracle to verify the event read path. The
capture-event harness returns 50 real events, and the instrument-grpc-nonstream rewrite
captured the exact GetNextEventQueryResultBuffer.result buffer (63,192 bytes, version
0x0B=11, rowCount 50 = 25 Alarm.Set + 25 Alarm.Clear). Feeding that real buffer through
HistorianEventRowProtocol.Parse exposed a latent parser bug.

The real buffer layout is: version(2) + rowCount(4) + headerField(4, =0x1E) followed by
MARKERLESS rows (rowFormat(2)=7 + filetime(8) + 8x u16 slots + compact-ascii type +
propCount + props). The parser wrongly treated the one-time 0x1E field as a per-row
marker and re-consumed [marker+format] for every row, so it decoded only the FIRST row
of any multi-row buffer and stopped. This is not gRPC-specific: the captured WCF v9
buffer has the identical 0900 <rowCount> 1E000000 0700 header, so the shipped WCF event
read had the same latent multi-row truncation.

Fix: read a 10-byte buffer header (skip the 0x1E field once) and parse markerless rows;
accept container version 9 (WCF) and 11 (gRPC), mirroring the interface-version gate that
accepts History 11 and 12.

Verified: the real 50-row buffer now decodes to exactly 50 events, ending cleanly at
end-of-buffer (Parse_RealStockClientCapture_DecodesAllEvents, gated on
HISTORIAN_EVENT_CAPTURE_NDJSON so it skips without the gitignored capture), plus a
synthetic v11 golden test. 328 offline tests pass.

The parse path is now verified against the provided client's real event data on both
transports; the only remaining gap for gRPC events is the server delivering rows to our
connection (the documented retrieval-server-gate).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 14:05:35 -04:00
Joseph Doherty 8199dde452 gRPC events: capture decrypted HTTP/1.1 frames native vs ours — topology found + tested null
Pursued hypothesis #3 (connection-frame capture). Built a TLS-terminating tee proxy
(artifacts/.../httpcap/, gitignored: self-signed server cert, forwards through the
loopback tunnel, logs decrypted HTTP/1.1 + gRPC-Web both directions) and ran a native
capture-event (returns 50 rows) and our SDK diagnostic (0 rows) through the SAME
proxy/upstream for a clean A/B.

Findings:
- The stock client is gRPC-Web/HTTP-1.1 (alpn empty), and clientCert=none on every
  connection — confirming (with the decompile) that hypothesis #2 (TLS client cert) is
  moot: the native presents no client cert.
- Connection topology differs: the native opens 5 TLS connections, one per service, and
  runs the event query (StartEventQuery/GetNext/EndEventQuery) on a DEDICATED
  RetrievalService connection, separate from the HistoryService connection that opened and
  registered the session. Our SDK collapses every service onto one connection. (Matches the
  decompile: the stock client has a separate GrpcClientBase per service.)
- Framing differs benignly: native uses content-length + Expect: 100-continue; SDK uses
  transfer-encoding: chunked. The server accepts both (StartEventQuery returns a valid
  handle), so framing is not the gate. No hidden header on either side.

Tested the topology hypothesis with a new env-gated switch
(HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL=1): run StartEventQuery/GetNext/EndEventQuery on a
dedicated RetrievalService connection (no re-handshake, reusing the session handle —
mirroring native conn4), registration staying on the main connection. Result: still
0B00000000001E000000 (0 rows), QH=1063. Splitting the event query onto its own connection
does not make rows flow — the server correlates by session handle, not connection, so
topology is not the row-scoping gate.

Every angle is now exhausted (payload, transport, metadata/interceptor/cert, topology,
data store). The gate is a server-internal per-connection retrieval working-set in the
native HistorianClient C++ core, unreachable from a pure-managed client. Conclusion
unchanged: auth-solved / retrieval-server-gated; ReadEventsAsync over gRPC keeps the
no-row throw, event reads use WCF. 56 offline gRPC/event tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 13:43:21 -04:00
Joseph Doherty 6cf4dd13fe gRPC events: decompile the stock managed client — confirms no hidden client-side difference
Closed the blind spot the zero-rows conclusion had leaned on: prior cycles used wire
capture (instrument-grpc-nonstream hooks only byte[] params), blind to gRPC metadata,
interceptors, channel options. Read the stock managed source directly
(histsdk-2023r2-analysis/decompiled/Archestra.Historian.GrpcClient + HistorianAccess;
the pure-managed assemblies decompile cleanly though mixed-mode aahClientManaged crashes
ILSpy).

Findings:
- GrpcClientBase.InitializeBase uses GrpcWebHandler (GrpcWebMode, HttpVersion 1.1) — the
  stock client speaks gRPC-Web over HTTP/1.1, the SAME transport as our SDK. This corrects
  the premise of hypothesis #1: there was never a native Grpc.Core HTTP/2 path to differ
  from; the stock client returning 50 rows is itself gRPC-Web. The HTTP/2 disproof's
  conclusion stands and is reinforced (identical transport on both sides).
- m_metadata on every RPC (incl. StartEventQuery/GetNextEventQueryResultBuffer) is only
  grpc-internal-encoding-request: gzip — exactly our header set. The ClientInterceptor is
  a no-op (empty LogCall). So the "invisible per-connection metadata/header" blind spot is
  confirmed empty — no hidden client-side identity the byte[] capture missed.
- CreateEventQuery/StartQuery/MoveNext are not in managed code; the managed
  GrpcRetrievalClient.StartEventQuery is a thin one-RPC stub. The query logic lives in the
  native C++/CLI HistorianClient core — consistent with the working-set being native/server-side.

Every client-controllable layer is now confirmed identical by reading the stock source,
not just by wire match: request bytes, transport, channel options, gRPC metadata,
interceptor. The remaining difference is below the managed surface / server-side.
Conclusion unchanged: gRPC event-row retrieval is auth-solved / retrieval-server-gated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 13:20:39 -04:00
Joseph Doherty e091965d59 Merge grpc-event-http2-disproof: gRPC event row retrieval — transport + SQL angles exhausted
Pursued the two server-side/connection hypotheses from grpc-event-query-capture.md
for the gRPC event zero-rows blocker:

- #1 transport (DISPROVEN): native HTTP/2 (no GrpcWebHandler) returns the byte-identical
  version-11 rowCount-0 terminal as gRPC-Web; the full v8 chain (ExchangeKey auth,
  CM_EVENT registration, StartEventQuery) runs end-to-end over both. Added
  HistorianGrpcChannelFactory.CreateHttp2 + HISTORIAN_GRPC_EVENT_HTTP2 switch.
- #4 SQL ground truth (ANSWERED): the event store has no per-connection column; the
  rich Events view is served live by the Historian engine via the INSQL OLE DB
  provider. Same engine + same window: 71,332 events via INSQL vs 0 via gRPC for our
  connection. The data is global/unscoped — the gate is the gRPC RetrievalService's
  per-connection in-process execution state, not data scoping or transport.

Three independent angles now exhausted (client payload byte-identical, transport
HTTP/2 == gRPC-Web, data store global). gRPC event-row retrieval stands documented as
auth-solved / retrieval-server-gated; ReadEventsAsync over gRPC keeps the no-row throw
and event reads use WCF. 326 offline tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 13:10:32 -04:00
Joseph Doherty f19eb3b821 gRPC events: answer hypothesis #4 (SQL ground truth) — event store is global, not connection-scoped
Pursued the server-side SQL angle for the gRPC event zero-rows. Built a read-only
SOCKS5-relay + Microsoft.Data.SqlClient probe (gitignored, artifacts/.../sqlschema/)
and dumped the live Runtime event schema. Findings:

- No per-connection / per-client / per-session column exists anywhere in the event
  store. The only scoping-like columns on Events/EventHistory/snapshots are event
  content (Source_* origin, User_* acker, Provider_NodeName, SourceServer replication).
- The rich Events view is not a relational table — it is served live by the Historian
  engine via the INSQL OLE DB provider (linked servers INSQL/INSQLD; encrypted remote
  view). The SQL EventHistory base table holds only 168 rows / 1 tag.
- Decisive: for the SAME -90d..now window the gRPC StartEventQuery diagnostic returned
  0 rows, the Events view via INSQL returns 71,332 events (most recent Alarm.Set firing
  seconds before the probe). Same engine, same window — INSQL serves the data, gRPC
  withholds it from our connection.

So there is nothing in the data to scope by: the zero-row gate is the gRPC
RetrievalService's per-connection in-process execution state, not data scoping or
transport (the same class of wall as DeleteTagExtendedProperties). Combined with the
transport disproof, three independent angles are now exhausted — client payload
(byte-identical), transport (HTTP/2 == gRPC-Web), and data store (global, unscoped).
gRPC event-row retrieval stands documented as auth-solved / retrieval-server-gated;
ReadEventsAsync over gRPC keeps the no-row throw and event reads use WCF.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 13:05:19 -04:00
Joseph Doherty b0388e7a40 gRPC events: disprove transport hypothesis (native HTTP/2 also returns zero rows)
Tested grpc-event-query-capture.md's leading next-session hypothesis — that the
native client's Grpc.Core HTTP/2 transport (vs our Grpc.Net.Client + GrpcWebHandler
gRPC-Web) is why event reads return zero rows. Added HistorianGrpcChannelFactory
.CreateHttp2 (plain HTTP/2 over SocketsHttpHandler, no gRPC-Web wrap) and an
HISTORIAN_GRPC_EVENT_HTTP2 switch on the event orchestrator (event path only; reads
stay gRPC-Web).

Live side-by-side against the event-bearing 2023 R2 server, everything else held
constant: the full v8 chain (ExchangeKey auth, CM_EVENT RegisterTags/EnsureTags=True,
StartEventQuery with a valid handle) runs end-to-end over BOTH native HTTP/2 and
gRPC-Web, and the server returns the byte-identical version-11 rowCount-0 terminal
(0B00000000001E000000) on both transports. Transport choice makes no difference —
the leading hypothesis is disproven and the zero-row scoping sits above the gRPC
transport layer.

Also confirmed the native capture-event harness queries a 30-day historical window
(returns 50 rows), so the native read is connection-scoped historical retrieval,
not a live subscription.

CreateHttp2 + the env switch + the EventChannelMode diagnostic are retained for
further connection-level probing. 44 offline tests pass; orchestrator stays on the
no-row throw.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 12:55:09 -04:00
Joseph Doherty 1161c40fd3 Merge docs-event-server-angle: server-side/connection angle documented for next session
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 12:42:48 -04:00
Joseph Doherty 88287a8c66 docs(grpc-events): document the server-side/connection angle for next session
Records the row-retrieval pickup now that the v8 ExchangeKey auth is solved and the
gap is proven connection-level (not client payload):
- grpc-event-query-capture.md: a "NEXT SESSION — the server-side / connection angle"
  section — what's already proven (don't redo), the in-place tooling, and ordered,
  testable hypotheses (HTTP/2 vs gRPC-Web transport [leading], TLS client cert,
  HTTP/2 frame capture, SQL event-store scoping).
- handoff.md item 1: updated to "v8 auth solved; row retrieval connection-gated",
  pointing at the NEXT SESSION section; the "to move any item" summary updated.

Doc-only; sanitization scan clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 12:37:22 -04:00
Joseph Doherty 9a25fa4ef7 Merge grpc-event-v8: v8 Event-connection ExchangeKey auth (ECDH+SHA256+MD5-keyed RC4) live-verified; row retrieval proven connection-gated
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 12:33:26 -04:00
Joseph Doherty 0921e21bdb feat(grpc-events): handle-capture cycle — event-row gap proven NOT a client payload issue
Extends the instrument-grpc rewrite to log string (strHandle) + uint (uiHandle /
queryRequestType) params, not just byte[], and captures our SDK's live v8
openParameters for a byte-diff against the native.

Result of the exhaustive comparison (all live-confirmed via the opt-in
EventReadDiagnostic test):
- StartEventQuery request: byte-identical to the native (v6 layout)
- v8 OpenConnection openParameters: byte-identical to the native (302B) once
  ClientNodeName matches — every control byte/ConnectionType/token/ShardId
- handle usage identical: ExchangeKey->contextKey, registration->storage GUID
  (strHandle), query->client uint (uiHandle); handles valid (RTag/EnsT=True)
- queryRequestType=3, registration order, gzip metadata header — all match
- window has events (native returns 50 now); eventCount not it

Every observable client-side byte matches the native, yet the server scopes 0
events to our connection. The event RPCs succeed over our transport and return a
valid EMPTY result (not a transport error), so this is a connection/server-level
difference (session affinity tied to the native Grpc.Core HTTP/2 connection or a
connection identity used to scope events) — invisible to and unfixable by client
payload matching. Needs server-side insight, not more wire RE.

Added opt-in diagnostics (RegistrationDiag, LastResultBufferHex,
LastEventOpenRequestHex). 326/326 offline; gated test still pins the no-row throw.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 12:31:33 -04:00
Joseph Doherty 32ae301050 feat(grpc-events): native-match event registration + skip ValidateClientCredential; diagnostics
Continues closing the event-row gap after the v8 ExchangeKey/RC4 auth breakthrough.

- HistorianGrpcHandshake: the v8 EVENT path skips StorageService.ValidateClientCredential
  (the native event connection authenticates purely via ExchangeKey + the RC4 token;
  running the Negotiate loop establishes a different session scope).
- HistorianGrpcEventOrchestrator.RegisterCmEventTag: simplified to the exact native
  gRPC event sequence (UpdateClientStatus -> RegisterTags -> EnsureTags -> GetHistorianInfo
  -> GetSystemParameter x7), dropping the 2020-WCF-era cross-service GetV probes and
  params-before-register that the gRPC event flow does not use. eventCount 5 -> 100.
- Opt-in diagnostics (RegistrationDiag, LastResultBufferHex/LastErrorBufferHex; gated
  EventReadDiagnostic test) for the continued investigation.

STATUS: auth + StartEventQuery + registration all succeed live (RTag/EnsT=True, valid
query handle), but GetNext returns version-11 rowCount-0 while the native returns 50 for
a BYTE-IDENTICAL request. Every observable wire element matches the native. The remaining
unknown is the string/uint HANDLE field VALUES the native uses per event RPC — the
instrument-grpc capture logged only byte[] params, not the handle fields. Next: extend
the IL rewrite to log strHandle/uiHandle/queryRequestType, re-capture, and match.
326/326 offline; gated test still pins the no-row throw.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 12:12:35 -04:00
Joseph Doherty 6d0f5c4b8f feat(grpc-events): implement aahCryptV2 token — v8 ExchangeKey auth now passes live
Implements the reverse-engineered v8 credential token in pure managed code and
wires the full event-connection auth chain. Live result: the v8 OpenConnection
now AUTHENTICATES against the 2023 R2 server (past the 132/171 AuthenticationFailed
wall) — the crypto is solved.

- HistorianNativeHandshake.DeriveExchangeKeyClientKey: client key = SHA256(ECDH
  shared secret) via ECDiffieHellman.DeriveKeyFromHash(SHA256), matching the native
  ECDiffieHellmanCng{Hash,SHA256}.DeriveKeyMaterial.
- BuildExchangeKeyCredentialToken + Rc4: token = RC4(password-UTF16LE, key=MD5(clientKey)).
  Reproduces a live-captured token EXACTLY (verified offline) — the native
  HistorianCrypto.NRC4_V2.aahCryptV2 scheme (MD5-keyed RC4). Pure managed; nothing
  AVEVA shipped. RC4 pinned by the standard test vector.
- OpenSession(eventConnection:true): ExchangeKey -> derive client key -> token ->
  v8 OpenConnection with ConnectionType=Event + the token. Orchestrator re-armed.
- HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc: the 86-byte native gRPC
  CM_EVENT EnsureTags (8-byte header + ...2f27 event-type GUID), replacing the
  2020 WCF 83-byte CTagMetadata on the gRPC event registration.

Goldens: RC4 standard vector + token construction. 326/326 offline.

KNOWN REMAINING: the event query still returns zero rows (GetNext yields a 10-byte
zero-row buffer). Auth + StartEventQuery succeed; the query-layer detail (vs the
native row-returning capture) is the last step. Gated test still pins the no-row
throw; opt-in diagnostic (HISTORIAN_GRPC_EVENT_DIAG) surfaces the journey.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 11:46:00 -04:00
Joseph Doherty c45f1a957b docs(grpc-events): token scheme fully RE'd via dnlib — aahCryptV2 (MD5-keyed RC4 + prefix)
Loaded dnlib in PowerShell (ILSpy crashes on the mixed-mode assembly) and scanned
the IL to recover the entire v8 token construction:

- <Module>::CHistoryConnectionGrpc.GetClientKey drives the ECDH: ECDiffieHellmanCng
  {KeyDerivationFunction=Hash, HashAlgorithm=SHA256, KeySize=256} -> ExchangeKey ->
  CngKey.Import(serverPub, EccPublicBlob) -> DeriveKeyMaterial = SHA256(shared secret),
  the 32-byte client key.
- aahClientCommon.CClientBase.ConfigureOpenConnection (the lone GetClientKey caller)
  builds the 26-byte token via HistorianCrypto.NRC4_V2.aahCryptV2 = a custom MD5-keyed
  RC4 stream cipher with a version prefix:
    * body/HashData = MD5 (verified by the round constants 0xd76aa478... + shifts 7/12/17/22)
    * prepare_key = RC4 KSA from a 16-byte MD5 key
    * enc_buffer = MD5 -> key, then rc4encrypt; enc prepends PrefixV2/InnerPrefixV2
      (the constant 0x8e token marker)
  So token = prefix + RC4(plaintext, key=MD5(keyMaterial)), keyMaterial tied to the
  SHA256(ECDH secret) client key. 100% reproducible in pure managed code (RC4+MD5).

Remaining (next cycle): read ConfigureOpenConnection's exact key/plaintext/prefix bytes,
implement aahCryptV2 managed-side, set the v8 token, live-test. Frida CNG + dnlib are
the RE path; nothing AVEVA is shipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 11:21:55 -04:00
Joseph Doherty b2ac35b98e docs(grpc-events): trace the ExchangeKey token crypto — KDF=SHA256(secret); token construction localized
Frida-hooked Windows CNG (scripts/frida/aahclientmanaged-cng-exchangekey.js) during
a real native ExchangeKey to recover the token derivation:

- The ECDH + KDF are standard CNG driven by managed System.Security.Cryptography
  .ECDiffieHellmanCng: NCryptSecretAgreement (P-256) -> NCryptDeriveKey(KDF=HASH,
  SHA256, 32 bytes). So the derived key = SHA256(ECDH shared secret).
- "ECK1" is the standard CNG BCRYPT_ECCPUBLIC_BLOB magic (P-256), confirming our
  BuildExchangeKeyClientHello wire format.
- The 26-byte token (constant 0x8e marker) is a custom construction over the
  derived key: a 528-candidate offline cracker (HMAC/SHA/AES-GCM/CBC/CTR over the
  derived key x request slices x creds) found no match, and it matches none of the
  traced hash digests. It is built in aahClientManaged's C++/CLI <Module> code
  between the DeriveKeyMaterial call and the openParameters assembly.

Next: ILSpy cannot decompile the mixed-mode assembly (crashes, exit 70); use dnlib
(IL-level) to dump the <Module> method referencing DeriveKeyMaterial and read the
post-derive token construction. 2 of 3 layers cleared (key exchange + client key);
the 3rd (token) is localized, pending dnlib extraction. Orchestrator stays on v6.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 11:11:21 -04:00
Joseph Doherty 3fd522fa10 docs(grpc-events): Path B — ExchangeKey ECDH clears 2 of 3 layers
Records that the pure-managed P-256 ExchangeKey works (cleared the v8 client-key
check; error advanced to 132/171 AuthenticationFailed). The remaining layer is the
26-byte credential-token KDF, which requires recovering the native key derivation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 10:31:37 -04:00
Joseph Doherty d67f6f5e96 feat(grpc-events): ExchangeKey ECDH (Path B) — clears the v8 client-key check
Implements HistoryService.ExchangeKey as a pure-managed P-256 ECDH key exchange
and wires it ahead of the v8 Event OpenConnection.

- HistorianNativeHandshake.BuildExchangeKeyClientHello / DeriveExchangeKeySecret:
  .NET ECDiffieHellman (nistP256); wire format "ECK1" + u32(32) + X(32) + Y(32),
  decoded from the live capture. No native AVEVA dependency.
- HistorianGrpcHandshake.OpenSession(eventConnection: true): runs ExchangeKey on
  the context-key handle before the v8 OpenConnection.
- Guardrail HistorianGrpcHandshakeRoutingTests scoped to the token-loop closure:
  still pins that the Negotiate token loop routes to ValidateClientCredential (not
  ExchangeKey), while allowing the legitimate ExchangeKey call in OpenSession.

Live result: ExchangeKey succeeds (server accepts our public key) and the v8
OpenConnection error advances from 132/34 "Failed to get client key" to 132/171
AuthenticationFailed — the ECDH cleared the client-key layer. The remaining
blocker is the 26-byte v8 credential token, which must be derived from the ECDH
shared secret (token KDF, not yet recovered). Orchestrator stays on v6 (set
eventConnection: true to re-arm once the KDF lands). 323/323 offline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 10:31:37 -04:00
Joseph Doherty 7284fdc976 docs(grpc-events): Path A disproven — v8 OpenConnection coupled to ExchangeKey
Records the full v8 openParameters byte map, the ECDH ExchangeKey finding, and
the Path A live result: the v8 OpenConnection on a ValidateClientCredential
session is rejected with native 132/34 "EstablishConnection Failed to get client
key". The v8 path requires the client key established by HistoryService.ExchangeKey
(ECDH), so the next route is Path B — implement ExchangeKey ("ECK1" + 64-byte
P-256 point) via .NET ECDiffieHellman, then reissue the v8 open.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 09:46:07 -04:00
Joseph Doherty 0b1e9d0a7f feat(grpc-events): v8 OpenConnection serializer + native error decode (Path A disproven)
Builds the native 2023 R2 version-8 OpenConnection format, which (unlike v6)
carries a ConnectionType byte (Event vs Process) — required because the 2023 R2
server returns event rows only on an Event connection.

- HistorianOpen2Protocol.SerializeNativeOpenConnectionVersion8: reproduces the
  302-byte v8 layout decoded from a live capture (version 8, markers, client-key
  GUID, username HString, length-prefixed credential token, ClientType /
  ConnectionType / flag / constant word / compact metadata / two empty strings;
  the tail reuses WriteClientCommonInfo). Golden-tested
  (Version8EventSerializerReproducesCapturedNativeStructure).
- HistorianNativeHandshake.BuildEventOpenConnectionVersion8Request: ConnectionType=
  Event, zeroed credential token (mirroring how v6 zeros its credential block and
  relies on the separate ValidateClientCredential handshake).
- HistorianGrpcHandshake.OpenSession: optional eventConnection switch; the
  OpenConnection failure path now decodes the native error (type/code/ASCII).

Path A (reuse ValidateClientCredential + zeroed token) was live-tested and
DISPROVEN: the server parses the v8 buffer but rejects it at the auth check with
native 132/34 "EstablishConnection Failed to get client key" — the v8 path looks
up the client key in the registry HistoryService.ExchangeKey (ECDH) populates,
not the one ValidateClientCredential does. The event orchestrator is therefore
reverted to the v6 session (gated test still pins the no-row throw). The v8
serializer/builder are retained for Path B (implement ExchangeKey). 323/323
offline tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 09:45:52 -04:00
Joseph Doherty ea85ea248d Merge handoff-event-gate-refresh: event-row item reflects captured + v8 gate
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 09:15:03 -04:00
Joseph Doherty 876cbc5d94 docs(handoff): refresh event-row item — captured + v8 connection-type gate
Item 1 is no longer capture-gated (the capture is done, merged 8ad160b). Update it
to: the capture-event run read 50 events from the stock client; the v6 request is
shipped; the remaining gate is the native v8 Event-type OpenConnection (a
ConnectionType field the SDK's v6 Open2 format lacks), a scoped RE+impl follow-on.
Points at docs/reverse-engineering/grpc-event-query-capture.md. Also corrected the
"to move any item" summary so item 1 no longer reads as needing a fresh capture.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 09:06:05 -04:00
Joseph Doherty 8ad160b843 Merge grpc-event-v6-capture: v6 StartEventQuery request + capture-event tooling; v8 connection-type gate diagnosed
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-23 09:01:37 -04:00
Joseph Doherty c6752804ee docs(grpc-events): event-query capture finding + v8 connection-type gate
Records the 2026-06-22 capture of the stock 2023 R2 gRPC event read and the
diagnosis of why row retrieval is gated:

1. The working StartEventQuery request is version 6 (vs the SDK's v5) — shipped in
   the companion code commit.
2. Rows additionally require an EVENT-type connection. Decoding the captured
   OpenConnection.openParameters (native format v8) shows a ConnectionType byte
   (Event=01 / Process=02) right after ClientType — a field the SDK's v6 Open2
   format does not have (it writes ClientType then ConnectionMode back-to-back).
   So the v6 buffer the SDK sends (accepted for reads) cannot mark the connection
   as Event, and the 2023 R2 server returns event rows only on an Event
   connection. The native client also used the ExchangeKey cert auth path.

Conclusion: making event rows flow over gRPC requires the SDK to emit the native
v8 OpenConnection format with ConnectionType=Event (a larger RE+implementation
effort), scoped as a follow-on. v6 is retained as the captured-correct request.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 10:41:29 -04:00
Joseph Doherty dbb5c99c53 feat(grpc-events): v6 StartEventQuery request + capture-event harness scenario
Captured the stock 2023 R2 client doing a gRPC event read (50 rows flowed) to
resolve the open "gRPC event ROW retrieval returns zero rows" item. Two captured
differences from our SDK's path; this lands the first (necessary) one plus the
capture tooling.

- HistorianEventQueryProtocol.CreateStartEventQueryAttempts: add a `version`
  parameter (default 5 = the 2020 WCF format, unchanged). The gRPC event
  orchestrator now opts into version 6 — the leading `06` plus a 5-byte trailing
  zero pad — which is the envelope the stock 2023 R2 client sends. The two
  buffers are otherwise byte-identical (filter block, UTC string, metadata
  namespace). Golden test Version6EmptyFilterMatchesCapturedGrpcEnvelope pins it.
- Grpc2023CaptureHarness: new `capture-event` scenario drives HistorianAccess
  over an Event-type gRPC connection (CreateEventQuery -> EventQueryArgs ->
  StartQuery -> MoveNext) so the wide-net instrument-grpc-nonstream rewrite dumps
  StartEventQuery.requestBuffer + the row result. Hostname defaults sanitized to
  HISTORIAN_GRPC_HOST / "localhost" (removed hardcoded server name).

NECESSARY BUT NOT SUFFICIENT: live validation shows v6 alone does not make rows
flow — the read also requires an Event-type connection, which our SDK's v6 Open2
format cannot express (see the companion docs commit). The gated
ReadEventsAsync_OverGrpc_* test correctly still pins the no-row throw. 322/322
offline tests pass; WCF event path unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 10:41:15 -04:00
Joseph Doherty d9051ba890 Merge fix-event-gate-characterization: gRPC event-row item is capture-gated (SQL ground truth)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 08:12:31 -04:00
Joseph Doherty 73f66cbf27 docs(handoff): re-characterize gRPC event-row gate — capture-gated, not server-gated
Live SQL ground truth (user-authorized one-time read via SOCKS->SQL relay)
disproves the gate on the open gRPC event-row item. The live 2023 R2 server
IS event-bearing — Runtime.dbo.Events holds 19,356 rows in the last 30 days
(90,944 in 365) — yet the empty-filter gRPC event query still returns zero rows
and long-polls to the deadline over that same window.

So GetNextEventQueryResultBuffer returning nothing is NOT "no events on the
server"; the empty-filter request shape (filter / namespace / event-tag
registration) doesn't match existing rows. The remaining work is a fresh native
gRPC event-query capture of the stock client, not access to a different server.

- handoff.md: rewrite open-item #1 with the SQL numbers + capture-gated framing;
  update the "to move any item" summary to match.
- HistorianGrpcIntegrationTests: correct the event-read test comment (drop the
  false "idle dev box holds no events" rationale; document 19k-events-yet-zero-rows).

No behavior change (test edit is comment-only). Sanitization scan clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 08:11:27 -04:00
Joseph Doherty 941e11d292 Merge docs-handoff-refresh: handoff doc reflects exhausted roadmap
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 07:54:29 -04:00
Joseph Doherty 8b966f3d80 docs(handoff): refresh to current state — roadmap exhausted
The handoff doc was anchored at 2026-05-04 (read-path blocker era). Add a
"Current Status (2026-06-22)" section at the top summarizing the full shipped
read/write/config/client-side surface across WCF + gRPC, the 8 remaining gated
items (event-row retrieval, active-SF magnitude, SendEvent capture, SQL wall,
R4.2 edits, ReadBlocks, the disproven DelTep gRPC probe, deferred-by-design),
and what would unblock each. Reframe the old "Active Blocker" as a historical
read-path record; fix the stale 55/55 test count to 321/321 offline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 07:54:29 -04:00
Joseph Doherty c88260c973 Merge grpc-deltep-probe: DelTep multiplexed-channel probe (disproven)
gRPC's single shared channel does NOT lift the WCF per-connection working-set
wall for DeleteTagExtendedProperties — probed live 2026-06-22, primes succeed
but the delete is still rejected (native code=1). Stays server-blocked on both
transports, unshipped; pinned by a gated negative test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 07:06:39 -04:00
Joseph Doherty b3417c2f6a docs(grpc): record DelTep multiplexed-channel probe as disproven
README transport matrix + grpc-tooling-completion.md §Out-of-scope: the gRPC
multiplexed-channel hypothesis for DeleteTagExtendedProperties was probed live
2026-06-22 and disproven — primes succeed on the shared channel but DelTep is
still rejected (native code=1), property survives. Stays server-blocked on both
transports, not shipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 06:55:05 -04:00
Joseph Doherty 2bd86e4e83 probe(grpc): DeleteTagExtendedProperties multiplexed-channel — disproven
Adds an internal RE probe (HistorianGrpcTagWriteOrchestrator.
ProbeDeleteTagExtendedPropertiesAsync) testing whether gRPC's single shared
channel lifts the WCF per-connection working-set wall that blocks DelTep.

Live result (2023 R2, History iface 12): both GetTgByNm + GetTepByNm primes
succeed on the one shared channel, yet DelTep is still rejected (native code=1)
and the property survives. So the working set is populated by the native
client's in-process registration state, not the wire session — neither WCF's
per-service channels nor gRPC's shared channel reproduce it. DelTep stays
server-blocked on BOTH transports and remains unshipped.

Gated negative test DeleteTagExtendedProperties_OverGrpc_ProbeMultiplexedChannel
pins this (primes succeed, delete rejected, prop survives) and flips if a future
server/registration lifts the wall. Comment in HistorianClient records the
probe. 321 offline tests pass; live test passes bounded at ~11s.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 06:55:05 -04:00
Joseph Doherty 32cb5152a6 Merge docs-roadmap-exhausted: mark HCAL roadmap exhausted
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 06:17:09 -04:00
Joseph Doherty df28bcfa53 docs(roadmap): mark HCAL roadmap exhausted; remaining items are all gated
M0/M1/M2/M3 done + live-verified; M4 R4.1/R4.3(idle)/R4.4 merged to main; the
grpc-tooling-completion plan is fully executed. Add a top-of-file status banner
enumerating the only remaining items and why each is gated (infra-gated event-row
retrieval + active-SF magnitude; capture-gated SendEvent; server-walled SQL +
revision edits; out-of-scope ReadBlocks / DeleteTagExtendedProperties). Nothing
left is a pure code task. README transport matrix stays authoritative per-op.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 06:12:04 -04:00
Joseph Doherty a714aa1bff Merge: ext-prop read parser fix + gRPC GetConnectionStatus + SQL-prime result
Fixes the multi-property GetTagExtendedProperties parser (uint16 flags trailer,
captured live), ships GetConnectionStatus over gRPC (plan #5), and records the
negative plan-#4 SQL RegisterTags-prime result.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 06:04:15 -04:00
Joseph Doherty ecf446965a docs(grpc): matrix + plan reflect ext-prop fix, SQL prime result, ConnStatus
- README transport matrix: GetTagExtendedProperties notes the multi-property parser
  fix; AddTagExtendedProperties read-back now round-trips; GetConnectionStatus gRPC
  -> live-verified; ExecuteSqlCommand notes the RegisterTags prime does not help.
  Refresh the closing production-pattern guidance.
- grpc-tooling-completion.md: mark #5 (ConnStatus) done, #4 (SQL prime) negative, and
  the #1 ext-prop read-back follow-up done.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 06:03:59 -04:00
Joseph Doherty 8984dac1ed test(grpc): multi-group ext-prop golden + ConnStatus + tightened write read-back
- WcfTagExtendedPropertyProtocolTests: add a multi-group golden test mirroring the
  live capture (one group per property + uint16 flags trailer) that the old parser
  failed; correct the synthetic builder to the uint16-flags trailer.
- HistorianGrpcIntegrationTests: add GetConnectionStatusAsync_OverGrpc_ReportsConnected
  (plan #5); tighten the write-lifecycle read-back to a hard assert now that the parser
  is fixed; make sandbox cleanup generous best-effort (rename is async + the browse view
  is eventually consistent, so a hard absence assert was racy).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 06:03:59 -04:00
Joseph Doherty 3525653c2b fix(grpc): extended-property read parser + GetConnectionStatus over gRPC
- HistorianTagExtendedPropertyProtocol.ParseResponse: fix the multi-property/
  multi-group response shape captured live from the 2023 R2 server. The server
  returns one group per property (the tag name repeats), each propertyCount=1, and
  a uint16 searchability-flags trailer per property (0x0003 built-in, 0x0001 user-
  added) — NOT the single-byte group trailer the old model assumed, which drifted
  one byte per group and threw "expected 0x09 found 0x01" on any buffer with more
  than one property. Now reads the per-property uint16 trailer (tolerates a legacy
  1-byte tail). Fixes read-back on both WCF and gRPC. Adds GetTagExtendedPropertiesRaw
  for future captures.
- HistorianGrpcStatusClient.GetConnectionStatusAsync (plan #5): synthesize connection
  status from a measured gRPC handshake (OpenConnection yielding a storage-session
  GUID => connected), mirroring the WCF synthesize-from-probe approach. Routed in
  Historian2020ProtocolDialect on UseGrpc (the WCF path used the MDAS binding, which
  can't reach the gRPC port).
- HistorianGrpcSqlClient: record the negative plan-#4 result — a HistoryService.
  RegisterTags prime does NOT clear the server-side CSrvDbConnection fault (tried live
  on both 0x402/0x401); the op stays bounded behind ProtocolEvidenceMissingException.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 06:03:38 -04:00
Joseph Doherty 000f4120d5 Merge: gRPC ReadEvents (bounded) + live-verified gRPC tag-config writes
ReadEvents routed over gRPC (tooled, bounded long-poll handling); tag-config
writes (EnsureTag/DeleteTag/RenameTags/AddTagExtendedProperties) live-verified
against the 2023 R2 server via a self-cleaning sandbox lifecycle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 05:19:59 -04:00
Joseph Doherty 27e969f86d docs(grpc): transport matrix + plan reflect ReadEvents + live-verified writes
- README transport matrix: gRPC writes (EnsureTag/DeleteTag/RenameTags/
  AddTagExtendedProperties) flip to live-verified; note the async-rename retry and
  the extended-property read-back parser gap. ReadEvents gRPC -> tooled-but-bounded
  (StartEventQuery works, GetNext long-polls, throws on no-row pending an
  event-bearing server). Refresh the closing production-pattern guidance.
- grpc-tooling-completion.md: mark items #1 (writes, done) and #2 (ReadEvents,
  tooled/bounded) with the live outcomes and follow-ups.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 04:58:44 -04:00
Joseph Doherty 274466c050 test(grpc): live-verify gRPC writes + pin bounded ReadEvents behavior
- TagWriteLifecycle_OverGrpc_*: live-verified the gRPC tag-config write surface
  (EnsureTags create, AddTagExtendedProperties, StartJob rename, DeleteTags) against
  a self-cleaning synthetic sandbox tag. Hardened for the live server: pre-clean both
  names for a clean slate, retry the async StartJob rename (transiently rejectable
  right after create), tolerate the known extended-property read-back parser gap
  (value marker 0x01 vs compact-string 0x09 — a read-side gap, not a write failure),
  and assert no litter remains after cleanup (TagExistsAsync). Two consecutive clean
  passes.
- ReadEventsAsync_OverGrpc_*: pins the current bounded reality — StartEventQuery
  succeeds but GetNextEventQueryResultBuffer long-polls on no data, so the bounded
  read throws ProtocolEvidenceMissingException on the idle dev box (no hang).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 04:58:44 -04:00
Joseph Doherty f1fd3691ba feat(grpc): route ReadEvents over gRPC + extract shared CM_EVENT registration
Adds HistorianGrpcEventOrchestrator: opens a read-only gRPC session, replays the
CM_EVENT registration (UpdateClientStatus -> 6 GetSystemParameter -> RegisterTags
-> cross-service version probes -> EnsureTags), then StartEventQuery -> loop
GetNextEventQueryResultBuffer -> EndEventQuery, reusing the WCF query builder and
row parser verbatim. Routed in Historian2020ProtocolDialect on UseGrpc.

The captured registration buffers (CmEventTagId, UpdC3 blob, RTag2 buffer, GETHI
builder, pre-register param list, native-error decode) are extracted into a shared
HistorianEventRegistrationProtocol so the WCF and gRPC paths can't drift; the WCF
orchestrator is refactored onto it with no behavior change.

Live finding (2026-06-22): the chain runs and StartEventQuery succeeds, but the
gRPC server long-polls GetNextEventQueryResultBuffer on no data (it blocks to the
deadline instead of returning the synchronous 5-byte code-85 terminal the WCF op
returns). Per-call gRPC-Web deadlines proved unreliable over a tunnel, so the read
is hard-bounded by an overall linked-CTS budget (<=30s; gRPC honors token
cancellation). On the no-row path it throws ProtocolEvidenceMissingException rather
than assert a false-empty list. Row-level retrieval awaits an event-bearing 2023 R2
server (the dev box holds no events).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 04:58:25 -04:00
Joseph Doherty 2d69f2860e Merge grpc-config-ops: tool the WCF-only config ops over gRPC + completion plan
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 01:30:04 -04:00
Joseph Doherty 7e8bb07df3 docs(grpc): add gRPC tooling completion plan
Self-contained plan for finishing gRPC surface parity: live-verify the
sandbox-gated writes, port ReadEvents (CM_EVENT registration state machine),
SendEvent (capture-blocked), the SQL server-wall stretch, and optional
GetConnectionStatus. Includes the proven reuse pattern and live-verification setup
so it survives context compaction.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 01:30:04 -04:00
Joseph Doherty e7a6cf1989 docs(grpc): reflect newly-tooled config ops in the transport matrix
- GetRuntimeParameter / GetTagExtendedProperties now live-verified over gRPC
- ExecuteSqlCommand marked server-walled (new legend state)
- tag-config writes marked sandbox-gated (new legend state)
- document the HISTORIAN_GRPC_WRITE_SANDBOX_TAG live-test gate
- rewrite the matrix summary to reflect what was learned tooling the config ops

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 01:26:34 -04:00
Joseph Doherty 0780cec9a7 test(grpc): live + gated coverage for the gRPC config ops
- GetRuntimeParameterAsync_OverGrpc_ReturnsValue (live)
- GetTagExtendedPropertiesAsync_OverGrpc_DoesNotThrow (live; empty for system tags)
- ExecuteSqlCommandAsync_OverGrpc_IsServerWalled (live; pins the captured wall)
- TagWriteLifecycle_OverGrpc_CreatesAddsPropRenamesDeletes — DESTRUCTIVE, gated on
  HISTORIAN_GRPC_WRITE_SANDBOX_TAG; self-cleaning create->addprop->verify->rename->delete

Full gRPC live suite 19/19 green against a real 2023 R2 server (write lifecycle
skips without the sandbox tag); 317 offline green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 01:26:33 -04:00
Joseph Doherty ef68016c7a feat(grpc): tool the WCF-only config ops over the gRPC transport
Wire the config operations that previously only worked over WCF onto RemoteGrpc,
reusing the proven 2020 byte serializers verbatim inside the protobuf bytes fields
(keyed by the Open2 session handle). Live-verified against a real 2023 R2 server
where noted.

Read ops (live-verified):
- GetRuntimeParameterAsync -> StatusService.GetRuntimeParameter (GETRP serializer)
- GetTagExtendedPropertiesAsync -> RetrievalService.GetTagExtendedPropertiesFromName
  (GetTepByNm serializer + sequence paging; page-0 FillBufferFromVector is the
  benign no-data terminator, matched to the WCF break-and-return-empty semantics)

Server-walled (bounded with captured evidence):
- ExecuteSqlCommandAsync -> RetrievalService.ExecuteSqlCommand. The request rides
  the RPC but the server-side CSrvDbConnection.ExecuteSqlCommand faults
  (IndexOutOfRange / native err 38) on a DB-connection precondition the pure
  managed gRPC session doesn't establish (same class as OpenStorageConnection).
  Surfaced as ProtocolEvidenceMissingException.

Write ops (tooled + routed, sandbox-gated — not run destructively live):
- EnsureTagAsync / DeleteTagAsync / RenameTagsAsync / AddTagExtendedPropertiesAsync
  via HistoryService.EnsureTags / DeleteTags / StartJob / AddTagExtendedProperties
  on a write-enabled (0x401) session, reusing the WCF golden serializers. The WCF
  priming discovery-dance is omitted (the M3 gRPC write probe worked without it);
  add it first if a live sandbox run is rejected.

Routed in Historian2020ProtocolDialect / HistorianClient on the RemoteGrpc branch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 01:26:33 -04:00
Joseph Doherty 035d8a92f2 Merge grpc-test-doc-parity: gRPC live-test parity + transport-matrix docs + v12 gate-note fix
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 00:40:52 -04:00
Joseph Doherty b80ac07942 docs(grpc): transport matrix + correct the obsolete v12 version-gate note
Document the WCF-vs-gRPC surface and fix a stale claim.

- README: add a "Transport matrix (WCF vs gRPC)" section with a per-operation
  table. Mark the config ops the gRPC server exposes-but-untooled with a distinct
  legend state (recovered RPC + bytes buffer named) vs genuinely unavailable, so
  "not tooled" is not conflated with "not possible".
- README: document the gRPC live-test env vars (HISTORIAN_GRPC_HOST/_PORT/_TLS/
  _DNSID/_TIMEOUT/_WRITE_SANDBOX_TAG) and refresh the Status section (test count
  + the live-verified gRPC surface).
- The "gRPC requires VerifyServerInterfaceVersion=false against a v12 server" note
  was obsolete: the gate already accepts History 11 AND 12 (AcceptedVersions), and
  the live gRPC suite runs with the default verification on. Corrected in the
  README, CLAUDE.md, and the HistorianServerVersionGate docstring.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 00:40:13 -04:00
Joseph Doherty 6d8a7d48f8 tests(grpc): live-verify aggregate + at-time over gRPC at WCF parity
The gRPC integration suite was missing live coverage for ReadAggregateAsync and
ReadAtTimeAsync, the two tooled gRPC reads the WCF suite already exercises. Add
them so the full tooled gRPC surface is live-tested like WCF.

- ReadAggregateAsync_OverGrpc_ReturnsTimeWeightedAverageRows
- ReadAggregateAsync_OverGrpc_AcceptsRetrievalMode (Min/MaxWithTime, BestFit)
- ReadAtTimeAsync_OverGrpc_ReturnsRequestedTimestamps

The aggregate tests self-calibrate their window from a real raw sample
(SeedAggregateWindowAsync): the interpolating modes (TimeWeightedAverage /
Min/MaxWithTime) do a slow bounding-value scan and return empty when the window
has no raw data, so a fixed "last N hours" window blows the per-call deadline
against an idle server. Anchoring the window where data actually exists keeps the
scan cheap and returns rows on idle or live servers alike. Adds an optional
HISTORIAN_GRPC_TIMEOUT knob (per-call deadline override) for slow links.

Full tooled gRPC surface now live-green: 15/15 gRPC integration tests pass
against a real 2023 R2 server; 313 offline tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-22 00:40:13 -04:00
Joseph Doherty e45c615a79 docs: record R4.3 measured idle-state status in hcal-roadmap
Update the M4 table row, one-glance status line, and M4 narrative note to
reflect R4.3: measured idle-state GetStoreForwardStatusAsync SHIPPED over
gRPC; active-SF magnitude + R4.2 revision edits stay deferred behind the
shared D2 storage-engine console-pipe wall.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 23:20:51 -04:00
Joseph Doherty 9db2864f70 Merge re/r4.3-sf-status-measured: R4.3 measured idle-state store-forward status
gRPC GetStoreForwardStatusAsync now contacts the server (measured idle-state,
ErrorOccurred on failure) instead of blind synthesis; active-SF magnitude
stays D2-gated. Includes the grpc-sf-status-probe RE tool + re-scope doc.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 23:17:35 -04:00
Joseph Doherty 53a9c87114 R4.3: measured idle-state GetStoreForwardStatusAsync over gRPC
Route GetStoreForwardStatusAsync to a gRPC path that actually contacts the
server (StatusService.GetHistorianConsoleStatus) instead of synthesizing an
all-false result blind. On a reachable/normal server it returns the
not-storing baseline but MEASURED; when the server is unreachable or the
console-status call fails it reports ErrorOccurred with the underlying error
(the old synthesis never contacted the server). The active-SF buffer
magnitude (Storing/Pending/DataStored) stays false because it lives behind
the D2 storage-engine console wall.

Non-gRPC transports keep the synthesized fallback. Live-verified against the
2023 R2 server; gated integration test
GetStoreForwardStatusAsync_OverGrpc_ReturnsMeasuredIdleState added. README
operation table updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 23:14:17 -04:00
Joseph Doherty c2d8fb9bc8 R4.3: gRPC store-forward status probe + re-scope
Add HistorianGrpcStoreForwardStatusProbe and the `grpc-sf-status-probe` CLI
command. The idle-baseline run against the live 2023 R2 server resolves the
plan's §9.3 handle question: the direct StorageService SF pull RPCs
(GetSFParameter / GetRemainingSnapshotsSize) require the OpenStorageConnection
console handle and are D2-gated (err 132, identical under read-only and
write-enabled sessions), while StatusService.GetHistorianConsoleStatus IS
reachable on the session string handle (=3 at idle).

Records the gRPC re-scope and the idle-baseline findings in
docs/plans/store-forward-cache-reverse-engineering.md §9. The probe writes
nothing and releases any console session immediately.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 23:14:05 -04:00
Joseph Doherty f840af5873 Merge re/m4-redundancy: R4.4 client-side multi-historian redundancy
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 22:46:58 -04:00
Joseph Doherty 60b3673f01 M4 R4.4: client-side multi-historian redundancy
Adds AVEVA.Historian.Client.Redundancy — HistorianRedundantClient orchestrates
N single-historian members (IHistorianMember; default HistorianClientMember
over HistorianClient) as one logical client. Pure client-side, no server-side
redundancy protocol, no RE.

- Reads fail over to the next member in priority order. Streaming reads only
  fail over BEFORE the first row is observed; a mid-stream failure propagates
  (failing over mid-stream would risk duplicated/skipped rows).
- Writes fan out: WriteFanout AllMembers | PreferredOnly, with All | Any ack
  policy, returning a per-member HistorianRedundantWriteResult.
- Per-member health: FailureThreshold demotes a failing member out of the
  preferred pool; a background watchdog (PeriodicTimer) + CheckHealthAsync
  re-probe and restore recovered members. GetStatus() snapshot + ActiveMember.
- Composes with R4.1: back a member's writes with a HistorianStoreForwardWriter
  so a down member buffers and replays on recovery — the pragmatic client-side
  equivalent of native ReSyncTags.

14 unit tests (no server): failover order, mid-stream no-failover, all-fail
aggregation, probe-any-up, fan-out ack policies, PreferredOnly, soft reject,
health demotion + CheckHealthAsync restore, watchdog recovery. Full suite 307
green. Roadmap R4.4 marked shipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 22:46:10 -04:00
Joseph Doherty a9000ec06a Merge re/m4-store-forward-outbox: R4.1 pragmatic store-and-forward outbox
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 22:38:35 -04:00
Joseph Doherty dd2aec3b8b M4 R4.1: pragmatic store-and-forward durable outbox
Adds AVEVA.Historian.Client.StoreForward — a client-side store-and-forward
layer over the historian write surface (AddHistoricalValuesAsync /
SendEventAsync). Producers enqueue writes; the writer persists them and
replays on reconnect so a transient disconnect never drops data. This is the
roadmap's recommended pragmatic outbox, NOT a bit-faithful reimplementation of
AVEVA's native SF cache (that stays deferred) — pure managed, no RE.

- HistorianOutboxEntry / HistorianOutboxEntryKind: buffered-write envelope
- IHistorianOutboxStore + InMemoryHistorianOutboxStore (tests) +
  FileHistorianOutboxStore (crash-durable: atomic temp+move JSON per entry,
  FIFO by filename sequence that resumes past on-disk max, corrupt-file
  quarantine). OutboxJson normalizes event object? properties off JsonElement.
- IHistorianWriteSink + HistorianClientWriteSink (HistorianClient-backed)
- HistorianStoreForwardWriter: enqueue, single-flight FIFO FlushAsync with
  head-of-line blocking, optional MaxDeliveryAttempts dead-lettering,
  DropOldest/Reject overflow policy, background drain loop (retry on reconnect),
  GetStatusAsync snapshot mirroring server SF Pending/Storing/ErrorOccurred.

12 unit tests (no server): durability-across-restart, reconnect-drain, FIFO
order/head-of-line, dead-letter, overflow policies, background auto-drain.
Full suite 293 green. Roadmap R4.1 marked shipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 22:35:30 -04:00
Joseph Doherty a91f126287 docs(hcal-roadmap): M3 R3.2 ships all 5 analog types, not Float-only
R3.2 and the one-glance table still read "Float-only"; the shipped
AddHistoricalValuesAsync covers Float/Double/Int2/Int4/UInt4 (golden-tested
+ live write/read-back). Correct both lines to match code + tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 22:20:25 -04:00
Joseph Doherty 7b5d27a8d3 Merge re/m3-value-types: AddHistoricalValuesAsync Double + Int support
Extends the historical-write surface from Float to all five analog types (Float/Double/Int2/Int4/
UInt4), each captured live + golden-tested + write/read-back validated through the pure-managed SDK.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:48:43 -04:00
Joseph Doherty d1e96f48de M3 R3.2: AddHistoricalValuesAsync supports Double + Int (Int2/Int4/UInt4)
Extended the historical-write serializer from Float-only to all five analog types EnsureTagAsync
supports. Captured each type's "ON" buffer live from the native client (sandbox tag per type,
written + captured + deleted):

- The 4-byte value descriptor (C0 10 01 00) is CONSTANT across types — it does not encode the type.
- The value is u32(0) + native-width value, width by the tag's declared type:
  Float->float32, Double->double64, Int2->int16, Int4->int32, UInt4->uint32.

HistorianHistoricalWriteProtocol.SerializeAddStreamValuesBuffer now takes the HistorianDataType and
encodes accordingly (unsupported types throw ProtocolEvidenceMissingException). The orchestrator
resolves the type from the tag-info NativeDataTypeDescriptor via MapDataType. Harness capture-write
gained --data-type. Golden-tested against all five live captures + the gated write/read-back test
validated each type end-to-end through the pure-managed SDK; 281 unit tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:48:29 -04:00
Joseph Doherty d527784def M3 capture harness: add delete-tag scenario (sandbox cleanup)
delete-tag drives the native client's DeleteTags (the clean-delete path, unlike the SDK's WCF
DelT which can leave the row). Primes the write session with AddTag first (DeleteTags on a fresh
connection returns UnknownClient(51) until the client is registered). Used to remove the capture
sandbox tag SdkM3CaptureSandbox from the live server (DeleteTags returned success).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:34:38 -04:00
Joseph Doherty 3cc02e3ed0 Merge re/m3-osc-correction: M3 historical writes SHIPPED (AddHistoricalValuesAsync over gRPC)
Reverse-engineered + shipped the SDK's first historical/backfill write capability:
- Corrected the M3 path (OpenStorageConnection dead-end -> the write rides
  HistoryService.AddStreamValues, NOT AddNonStreamValues/TransactionService).
- Built a net481 capture harness + IL-rewrite to capture the native 2023 R2 'ON' write buffer.
- HistorianHistoricalWriteProtocol ('ON' serializer, golden-tested) +
  HistorianGrpcHistoricalWriteOrchestrator + public AddHistoricalValuesAsync.
- Live-validated: pure-managed SDK wrote a value and read it back over gRPC.
275 unit tests pass; gated live write/read-back test green. Float-only, gRPC-only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:29:25 -04:00
Joseph Doherty 273103f882 M3 R3.2: instrument-grpc-nonstream also captures out/ref byte[] responses
Extends the IL-rewrite to log out (byref) byte[] params at method exit (ldarg + ldind.ref), not
just byte[] inputs. This captured GetTagInfosFromName's response, which located the per-tag GUID at
offset 8 = exactly where ParseTagInfoRecord reads "typeId" — proving the SDK already parses the GUID
AddStreamValues needs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:25:24 -04:00
Joseph Doherty dafafa0c98 M3 R3.2 SHIPPED: docs — AddHistoricalValuesAsync recorded in roadmap, plan, and CLAUDE.md surface
Marks M3 historical writes SHIPPED + live-validated across the roadmap (R3.2/R3.3/one-glance),
revision-write-path.md §"R3.1 CAPTURED", and the CLAUDE.md Required SDK Surface (the new write op,
gRPC-only, AddStreamValues "ON" path, Float-only, distinct from the still-blocked AddS2 streaming
path).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:24:37 -04:00
Joseph Doherty aa36e58d58 M3 R3.2 SHIPPED: AddHistoricalValuesAsync — historical backfill writes over gRPC (live-validated)
Public HistorianClient.AddHistoricalValuesAsync(tag, values) inserts non-streamed original
(backfill) values for an existing tag over the 2023 R2 gRPC front door. The pure-managed SDK
wrote a value and read it back live (gated test AddHistoricalValuesAsync_OverGrpc_WritesAndReadsBack
PASSED against the real server).

- HistorianGrpcHistoricalWriteOrchestrator: write-enabled (0x401) session ->
  RetrievalService.GetTagInfosFromName (resolves the per-tag GUID = the tag-info TypeId, and
  registers the tag on the session) -> HistoryService.AddStreamValues("ON" buffer) per sample.
- HistorianHistoricalValue (public record: TimestampUtc, Value, OpcQuality=192).
- gRPC-only: non-RemoteGrpc transports throw ProtocolEvidenceMissingException (the 2020 WCF
  non-streamed write is architecturally blocked, D2).
- Float value encoding only (the captured type); other types rejected by the serializer.

275 unit tests pass; the new gated live write/read-back test is green against the 2023 R2 server.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:23:08 -04:00
Joseph Doherty 85f0c2f0fa M3 R3.2: HistorianHistoricalWriteProtocol — the "ON" AddStreamValues serializer (golden-validated)
Managed serializer for the historical-write "ON" buffer, byte-for-byte matching the live capture
(WcfHistoricalWriteProtocolTests golden-tests the exact 56-byte native buffer). Layout: "ON"(0x4E4F)
+ count + lengths + 16B tag GUID + sample FILETIME + u16 quality(192) + 4B descriptor + received
FILETIME + value. Value encoding (Float, captured): an 8-byte slot = u32(0) + float32(value) — the
4-byte float in the high dword, NOT a double. The 16B tag GUID is the per-tag GUID the SDK already
parses as ParseTagInfoRecord's "typeId" (confirmed: it appears at offset 8 of GetTagInfosFromName's
response = where typeId is read, and in EnsureTags' response + the "ON" buffer).

Only the Float encoding is captured; other types rejected until captured. Next: gRPC orchestrator
(write-enabled session -> EnsureTags -> resolve tag GUID -> AddStreamValues) + public
AddHistoricalValuesAsync + live write/read-back.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:17:14 -04:00
Joseph Doherty 0e78d638d0 M3 R3.1: document the captured + validated AddStreamValues "ON" write path
revision-write-path.md §"R3.1 CAPTURED" + roadmap R3.1/R3.2/one-glance now record the validated
finding: the historical write is HistoryService.AddStreamValues ("ON" storage-sample buffer, AddS2
"OS" family) + EnsureTags, not AddNonStreamValues/TransactionService. Includes the decoded 56-byte
"ON" buffer layout, the working priming/batch sequence, the tag-GUID keying, and that the D2 cache
gate does not block the primed 2023 R2 client. Remaining work to ship AddHistoricalValuesAsync is
the managed "ON" serializer (adapt HistorianEventWriteProtocol) + gRPC orchestrator wiring.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:04:17 -04:00
Joseph Doherty 9bcfffb365 M3 R3.1 CAPTURED: native non-streamed write rides HistoryService.AddStreamValues ("ON" buffer)
Drove the native 2023 R2 client through a committed non-streamed (historical backfill) write to a
sandbox tag, with the IL-rewritten managed gRPC client dumping every byte[] payload. Read the value
back over gRPC = end-to-end validated.

Key discovery: the M3 historical write does NOT use AddNonStreamValues/TransactionService (the
roadmap's assumption from the static decompile). The native client routes it over
HistoryService.AddStreamValues with an "ON" storage-sample buffer (structurally the AddS2 "OS"
family), plus EnsureTags for registration:

  AddStreamValues.values (56B): "ON"(0x4E4F) + u16 count=1 + u32 totalLen + u16 payloadLen +
    16B tag GUID + FILETIME(sample) + u16 quality=192 + u32 type/desc + FILETIME(received) +
    8B double value.
  EnsureTags.tagInfos (144B): the analog CTagMetadata the SDK's EnsureTagAsync already builds
    (0x4E marker ... fe 00 trailer).

Tooling that produced it:
- instrument-grpc-nonstream now instruments EVERY byte[]-input method on every Grpc*Client
  (45 methods) so the real wire path surfaces regardless of assumptions.
- harness pre-loads the instrumented GrpcClient by identity (LoadFrom context reuses an
  already-loaded assembly before sibling-probing, so the rewrite wins over the bin original);
  capture-write sequence fixed to Begin -> Add -> SendValues -> End (End-before-Send = err 160
  InvalidBatchId); GetTagInfoByName(cache:false) + resync wait resolves the server key; cache
  gate (D2's 129) does NOT block the primed 2023 R2 client.

Buffers captured to gitignored artifacts/. Next: build the "ON" AddStreamValues serializer in
src/ (adapt the existing AddS2 "OS" serializer) + EnsureTags + ship AddHistoricalValuesAsync.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:03:08 -04:00
Joseph Doherty d5c04cd410 M3 R3.1 capture: add capture-write scenario (drives non-streamed write, no run yet)
The capture-write harness scenario drives the native 2023 R2 client through a non-streamed
(historical backfill) write so the IL-rewritten GrpcHistoryClient dumps RegisterTags.tagInfos +
AddNonStreamValues.inBuff to the capture NDJSON. Sequence: open write-enabled gRPC -> (optional
--create) AddTag sandbox -> GetTagInfoByName (real TagKey + primes the per-connection cache, the
gate mitigation) -> CreateHistorianDataValueList(NonStreamedOriginal) -> NonStreamedValuesBegin ->
AddNonStreamedValue -> AddNonStreamedValuesEnd -> SendValues (the wire push; only with --commit).

Not yet run — the actual write to the live server awaits explicit confirmation. Built clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 19:24:15 -04:00
Joseph Doherty c1f263ef83 M3 R3.1 capture: instrument-grpc-nonstream IL-rewrite + harness --grpc-rewrite loading
Adds the dnlib instrument command + harness wiring to capture the two non-streamed-write
buffers from the native 2023 R2 client:

- `instrument-grpc-nonstream <GrpcClient.dll> [out]` injects CaptureLogger.LogByteArray at the
  entry of GrpcHistoryClient.RegisterTags (byte[] tagInfos) and AddNonStreamValues (byte[] inBuff),
  writing the rewrite to docs/reverse-engineering/dnlib-write-copy/grpc2023 (gitignored — derived
  AVEVA binary). dnlib preserves the AVEVA public-key identity so aahClientManaged still binds the
  rewritten copy under the LoadFrom context (no SN re-verification).
- harness `--grpc-rewrite <dir>` probes that dir first, so the instrumented GrpcClient.dll +
  ReverseInstrumentation.dll load ahead of the originals. load-check confirms the rewritten
  strong-named copy binds (HistorianConnectionMode.Historian=2; GrpcHistoryClient RegisterTags +
  AddNonStreamValues present).

Next: capture-write scenario (open write-enabled -> sandbox tag -> read-prime -> AddNonStreamedValue),
which dumps tagInfos + inBuff to the capture NDJSON. Prod write — confirm before running.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 19:20:54 -04:00
Joseph Doherty ce8576bd6e M3 R3.1 capture: read-only gRPC connect scenario — LIVE-VERIFIED
Added the `connect` scenario to the 2023 R2 capture harness and ran it read-only against the
live server. The native mixed-mode client connects end-to-end over gRPC from this box:

  OpenConnection -> True (ErrorCode=Success)
  ConnectedToServer        = True
  ConnectedToServerStorage = True   <-- native client HAS the storage-engine session
  ConnectedToStoreForward  = False

Connection args that work (HistorianConnectionArgs): ServerName, TcpPort=32565,
ConnectionMode=Historian (gRPC), ConnectionType=Process, ReadOnly=true, IntegratedSecurity=false,
UserName/Password (explicit), AllowUnTrustedConnection=true, SecurityInfo=CertificateInfo{
SecurityMode=TransportCertificate, CertificateName=WONDER-SQL-VD03 } (the https:// host over the
loopback tunnel). Creds from HISTORIAN_USER/HISTORIAN_PASSWORD.

Significance: ConnectedToServerStorage=True means the native client establishes the storage
session the pure-managed SDK couldn't — so a write driven through it should route
AddNonStreamValues with a live storage session, and the cache-gate mitigation (read-first)
is promising. Next: IL-rewrite Archestra.Historian.GrpcClient.dll + a write-enabled run to
capture the RegisterTags btTagInfos + AddNonStreamValues btInput (prod write; per-action auth).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 19:12:42 -04:00
Joseph Doherty 328748c0ae M3 R3.1 capture: scaffold net481 x64 harness + load-check (PASS)
First increment of the native-2023R2-gRPC capture (docs/plans/revision-write-path.md
§"R3.1 capture plan"). Loads the mixed-mode aahClientManaged.dll by path (sibling resolver
over bin/ + msi-extract/.../Bin/x64) and reflects the connection API — no live contact.

load-check result on this box (net481 x64):
- aahClientManaged.dll loads cleanly (no missing VC++ runtime / no BadImageFormat) — confirms
  the self-contained mixed-mode assembly runs without an AVEVA install.
- HistorianConnectionMode.Historian = 2 (the 2023 R2 gRPC mode; ClassicHistorian = 1 = legacy)
  — the value the live-connect step sets on HistorianConnectionArgs.ConnectionMode.
- GrpcHistoryClient resolves with RegisterTags + AddNonStreamValues present — the IL-rewrite
  capture targets are reachable.

Standalone net481 project (not in Histsdk.slnx, like NativeTraceHarness). Next: read-only
gRPC connect + tag read (first live step, per-action auth), then IL-rewrite + write capture.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 19:01:12 -04:00
Joseph Doherty 222eed9c02 M3 R3.1: durable capture plan — drive native 2023 R2 gRPC client + IL-rewrite byte[] payloads
Records the feasibility-verified plan to capture the two remaining buffers (regular-tag
RegisterTags btTagInfos + AddNonStreamValues btInput):

- 2023 R2 aahClientManaged.dll is self-contained mixed-mode C++/CLI (only Windows + VC++
  runtime native imports) — loadable in a net481 x64 process, no AVEVA install needed.
- gRPC routes through the managed Archestra.Historian.GrpcClient.dll, so the byte[] payloads
  are capturable by IL-rewriting GrpcHistoryClient.RegisterTags / AddNonStreamValues (dnlib,
  the instrument-wcf-writemessage pattern; rewrite a copy, never the originals).
- Connection is reflection-drivable: HistorianAccess.OpenConnection(HistorianConnectionArgs)
  with ConnectionMode=HistorianConnectionMode.Historian (the gRPC mode), TcpPort=32565, cert.
- gRPC runtime deps (Grpc.Net.Client / Grpc.Core.Api / Google.Protobuf / ...) are present in
  msi-extract/ArchestrA/Toolkits/Bin/x64.
- Risk: the C++ AddNonStreamedValue TagNotFoundInCache(129) gate (the 2020 D2 blocker) may
  block btInput; mitigation = read the tag first. RegisterTags is emitted before that gate.

Build order documented (read-only connect -> IL-rewrite -> write capture -> serializer ->
commit+read-back -> AddHistoricalValuesAsync), each live step gated on per-action auth.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 18:58:38 -04:00
Joseph Doherty 57b9506d01 M3 R3.1: OpenStorageConnection is a dead end (error 85); precondition is front-door RegisterTags
Live-probed StorageService.OpenStorageConnection against the 2023 R2 server over a
write-enabled (0x401) session. Every attempt — sweeping ConnectionMode (0x401/0x402/0x1),
StorageSessionId-in (Open2-GUID / empty), and FreeDiskSpace — returns the IDENTICAL native
error type=4 code=85 ("session not registered"), so it's a structural refusal, not a bad
field value.

Decode (two corroborating facts):
- Error 85 is the same code the event read returns before RegisterTags2 (see
  HistorianWcfEventOrchestrator) — a generic "session not registered for this op".
- The 2023 R2 decompile shows OpenStorageConnection lives on a SEPARATE GrpcStorageClient
  (the storage engine's SF/snapshot channel, own port + service identity); HistorianAccess
  drives non-streamed writes through the native C++ HistorianClient, never this op.

So the roadmap's mapped "missing console session" step was wrong. The real non-streamed-write
precondition is the front-door HistoryService.RegisterTags (RTag2-family) for the target tag —
which is exactly why the R3.1 batch failed at AddNonStreamValues (no tag registered ->
StoreNonStreamValues had no route). Matches the original 2020-WCF D2 hypothesis.

Remaining (both need a native gRPC capture; do not guess bytes): the regular-tag RegisterTags
btTagInfos (only CM_EVENT's tag-GUID form is known) and the AddNonStreamValues btInput.

- HistorianGrpcStorageConnectionProbe + grpc-open-storage-connection CLI (opens nothing
  persistent; CloseStorageConnection on success)
- corrected revision-write-path.md §R3.1 follow-up + hcal-roadmap R3.1/R3.2 rows
- gated regression test pinning the error-85 refusal

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 18:51:16 -04:00
Joseph Doherty 78cb689bdf Merge re/m3-grpc-nonstreamed-write: M3 non-streamed write reachable over gRPC (Begin/End live; sequence mapped)
The D2 storage-engine-pipe wall is WCF-transport-specific. On 2023 R2 gRPC,
TransactionService.AddNonStreamValuesBegin/End round-trips live (write-enabled
session, Open2 storage GUID as strHandle). Live decode + static mining mapped the
full sequence: the remaining insert needs StorageService.OpenStorageConnection
(+ RegisterTags) then a btInput decode — a focused follow-up. Revision EDITS
(R4.2) stay pipe-only even on gRPC.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 18:21:01 -04:00
Joseph Doherty 1a08dd9ec2 M3: roadmap reflects mapped non-streamed sequence (OpenStorageConnection follow-up)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 18:20:35 -04:00
Joseph Doherty ac28679a1f M3 R3.1: map the required non-streamed write sequence (OpenStorageConnection is the missing step)
Static decompile mining of the 2023 R2 client corroborates the live R3.1 error:
the AddNonStreamValues failure is the missing StorageService.OpenStorageConnection,
which creates exactly the \.\pipe\aahStorageEngine\console,sid(...) session named
in the server error. Mapped the full native sequence:

  HistoryService.OpenConnection (have) -> StorageService.OpenStorageConnection
  (MISSING) -> StorageService.RegisterTags -> AddNonStreamValuesBegin (works) ->
  AddNonStreamValues(btInput) (fails - no console session) -> End(commit).

Two hard parts remain, each a live-production decode loop with no static shortcut:
(1) reproduce the 12-arg OpenStorageConnection handshake (several args inferred);
(2) decode the AddNonStreamValues btInput (C++-built, absent from decompiles; only
the 44-byte packed HISTORIAN_VALUE2 is known). Documented in revision-write-path.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 18:12:40 -04:00
Joseph Doherty 8fbb868813 M3 R3.1 decode: AddNonStreamValues reaches server StoreNonStreamValues (storage-engine console pipe)
Empirically decoded the AddNonStreamValues btInput framing against the live 2023
R2 server (grpc-nonstream-decode command + ProbeNonStreamedBuffersAsync driver).
Every transaction rolled back (bCommit=false) — no data written.

Finding: the btInput is assembled native-C++-side (not in any decompile), so 6
evidence-based framings (44-54B, packed HISTORIAN_VALUE2 variants) were probed.
All 6 returned the IDENTICAL server error while an empty buffer returned a
different InvalidParameter — so non-empty buffers pass parameter validation into
CHistStorageConnection::StoreNonStreamValues, which routes to the
\.\pipe\aahStorageEngine\console pipe server-side. Identical-across-framings =>
the blocker is NOT the btInput layout but a missing storage-engine console
session / tag-registration precondition for the connection.

Next step (untested): StorageService.OpenStorageConnection + tag registration
(RegisterTags/AddTagidPairs/AddShardTagids) before AddNonStreamValues, then
commit + read-back on a sandbox tag. Documented in revision-write-path.md
(R3.1 decode section); raw artifact gitignored.

272 unit tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 18:08:27 -04:00
Joseph Doherty 23798db1ef M3 probe: non-streamed write transaction reachable over 2023 R2 gRPC (Begin/End live-verified)
The D2 storage-engine-pipe wall is WCF-transport-specific. On the 2023 R2 gRPC
front door, TransactionService is a first-class service AND the gateway to the
storage engine, so the Open2 storage-session GUID (uppercase) is accepted
directly as strHandle with no legacy pipe.

Live-verified against the real 2023 R2 server over a write-enabled (0x401) gRPC
session: AddNonStreamValuesBegin returns a real strTransactionId, and
AddNonStreamValuesEnd(bCommit=false) discards it cleanly (no data written). On
2020 WCF the same op returns UnknownClient(51) for every handle + priming chain.

- HistorianGrpcRevisionProbe + grpc-revision-probe CLI command + gated test
  NonStreamedWriteTransaction_OverGrpc_BeginsAndDiscards (live pass).
- HistorianGrpcHandshake.OpenSession gains an optional connectionMode param
  (default read-only 0x402; pass 0x401 for write-enabled) — non-breaking.
- Docs: revision-write-path.md "the wall is gone" section; roadmap M3 section,
  R3.1-R3.3 rows, one-glance table, and status note updated honestly.

Not yet shipped: the AddNonStreamValues btInput VTQ buffer is uncaptured (never
guess wire bytes), so no value-commit is implemented. Scope is non-streamed
ORIGINAL backfill; revision EDITS (R4.2) remain pipe-only even on gRPC.

272 unit tests pass; sanitization scan clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 17:51:17 -04:00
Joseph Doherty 04ea0b9a1f R1.3 GetServerTimeZoneAsync over gRPC (live-verified); R1.4 bounded out on gRPC
Live-probed both R1.3 and R1.4 against a real 2023 R2 server over the gRPC
StatusService; implemented the one that carries an evidence-backed value.

R1.3 GetServerTimeZoneAsync — SHIPPED:
- StatusService.GetSystemTimeZoneName(uiHandle) returns the real server zone
  over RemoteGrpc (the 2020 WCF op is a client-side stub returning empty).
- HistorianGrpcStatusClient.GetSystemTimeZoneNameAsync -> dialect routing ->
  public HistorianClient.GetServerTimeZoneAsync. Non-gRPC transports fail
  closed with ProtocolEvidenceMissingException (no empty-string lie).
- Golden message-shape unit test + non-gRPC guardrail unit test + gated live
  test. 271 unit tests pass.

R1.4 GetHistorianInfoAsync (EventStorageMode) — bounded out on gRPC too:
- gRPC GetHistorianInfo is the same named-value query as 2020 WCF (only
  HistorianVersion resolves); EventStorageMode + 7 variants fail on both
  GetHistorianInfo and GetSystemParameter. The 518-byte struct is filled by a
  native vtable+648 HCAL call, not the gRPC op (per the 2023 R2 decompile), so
  the field is never on the wire. Not shipped on any transport. Closes the
  roadmap's open "build against a live 2023 R2 server" caveat.

Also correct the stale M3 roadmap section: D2 already proved
Transaction.AddNonStreamValues* rides the storage-engine pipe (STransactPipeClient2
-> aaStorageEngine), not WCF — same wall as R4.2 — so M3-over-WCF is blocked, not
"the path that is NOT the gated cache push".

Docs: hcal-roadmap.md, wcf-historian-info.md, wcf-status-localhost.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 17:24:10 -04:00
Joseph Doherty 25aff409dc Merge re/grpc-2023r2-handshake: M0 gRPC parity (probe/system-param/metadata/browse) + handshake fix 2026-06-21 16:32:02 -04:00
Joseph Doherty d23722ea73 Merge re/r1.10-rename-tags: RenameTagsAsync via History StartJob
# Conflicts:
#	docs/plans/hcal-capability-matrix.md
#	docs/plans/hcal-roadmap.md
#	src/AVEVA.Historian.Client/Wcf/HistorianWcfTagWriteOrchestrator.cs
#	tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs
#	tools/AVEVA.Historian.NativeTraceHarness/Program.cs
2026-06-21 16:31:44 -04:00
Joseph Doherty 4de222c950 Merge re/r1.4-gethi-finding: R1.1 ExecuteSqlCommand + R1.4 GetHistorianInfo (bounded)
# Conflicts:
#	docs/plans/hcal-roadmap.md
#	src/AVEVA.Historian.Client/HistorianClient.cs
#	src/AVEVA.Historian.Client/Protocol/Historian2020ProtocolDialect.cs
#	tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs
#	tools/AVEVA.Historian.NativeTraceHarness/Program.cs
2026-06-21 16:18:49 -04:00
Joseph Doherty 85ff1b48df R0.1 browse over gRPC SHIPPED — QueryTag cracked, M0 gRPC parity complete
Wires HistorianClient.BrowseTagNamesAsync over gRPC (Transport==RemoteGrpc) via
Grpc/HistorianGrpcTagClient.BrowseTagNamesAsync: StartTagQuery(OData) -> paged
QueryTag -> EndTagQuery. Live-verified against a real 2023 R2 server (returns Sys* tags).

QueryTag packet-id recovered WITHOUT native disassembly: a .rdata packet-descriptor
table in aahClientManaged.dll lists {0x6751,1}=StartTagQuery immediately followed by
{0x6752,1}=QueryTag (found via pefile byte-scan of .rdata), confirmed live.

Wire format (live-verified):
- request btRequest = u16 0x6752 + u16 version(1) + u16 queryType(1=names) + u32 startIndex + u32 count
- response btResonse = u32 count + per-name(u32 charCount + UTF-16LE) + trailer (NextIndex/metadata, ignored)
- new HistorianTagQueryProtocol.ParseTagNameQueryPage tolerates the trailer
- GlobToODataFilter translates the SDK glob filter to OData (Pre*->startswith, *suf->endswith,
  *mid*->contains, exact->eq); the 2023 R2 metadata-server parses filters as OData.

Replaces the earlier RE probe helpers with the shipped browse path. Adds golden-byte
(BuildQueryTagRequest) + 8 glob-translation unit tests + gated live browse test.
226 unit tests pass; 5/5 live gRPC tests pass (read, probe, system-param, metadata, browse).

Milestone 0 (full gRPC parity) is complete.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 16:01:15 -04:00
Joseph Doherty 630295bd18 docs: QueryTag native-RE attempt — lightweight tooling insufficient, needs Ghidra
Recorded the native-disassembly attempt on aahClientManaged.dll (mixed-mode):
ilspycmd cannot decompile it; capstone byte-search can't locate the StartTagQuery
0x6751 marker (not a plain immediate — it's an .rdata constant loaded RIP-relative,
the .text "51 67 00 00" hits are coincidental jump-table data). Managed metadata
gives QueryTag field semantics but not the binary packet-id. Finishing QueryTag
needs Ghidra/IDA xref analysis or the live IL-rewrite capture.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 15:46:54 -04:00
Joseph Doherty 4c9f0d476c docs: QueryTag error = InvalidPacketId (72); needs native aahClient.dll RE
Deepened the R0.1 browse finding. QueryTag's constant rejection decodes to
ArchestrA.CloudHistorian.Contract.ErrorCode.InvalidPacketId (72): the btRequest needs
a QueryTag-specific packet-id header (the generic 0x6751/v1 header StartTagQuery accepts
is rejected). The semantic fields are known from CloudHistorian.Contract
(QueryHandle/QueryType/StartIndex/TagCount request; TagNames[]+TagMetadataBuffer response),
but the binary packet framing lives in native aahClient.dll — aahClientManaged.dll is
mixed-mode (ilspycmd cannot decompile it) and no managed assembly builds the buffer.
Finishing QueryTag needs native RE (Ghidra/IDA) or a live gRPC capture of the stock client.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 15:16:19 -04:00
Joseph Doherty 26ef5e5645 R0.1 browse probe: StartTagQuery over gRPC takes an OData filter (live)
Probes the 2023 R2 gRPC browse path and records the finding. The front door does
NOT hit the 2020 WCF metadata-server-pipe wall.

- RetrievalService.StartTagQuery is cracked: the server (CMdServer::StartActiveTagnamesQuery
  over \.\pipe\aahMetadataServer\console) parses the filter as OData. startswith()/
  contains()/eq/empty succeed and return the 8-byte (queryHandle, tagCount); SQL-LIKE "%"
  and glob "*" fail with "ODataFilter: bad token". Live: 220 Sys* tags counted.
- QueryTag (paging) remains: every guessed btRequest returns a constant native error
  type 4 / code 72 (content-independent) -> framing needs a native capture, not guessing.

Adds RE probe helpers Grpc/HistorianGrpcTagClient.ProbeStartTagQuery + ProbeTagQuerySequence,
a gated StartTagQuery_OverGrpc_AcceptsODataFilter test, and the finding doc
docs/reverse-engineering/grpc-tag-query-odata.md. Browse is not yet wired (QueryTag open).

217 unit tests pass; 5/5 live gRPC tests pass. No tag names/identities committed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 14:58:12 -04:00
Joseph Doherty 0e19adae68 gRPC M0 R0.2: tag metadata over gRPC (GetTagInfosFromName, live-verified)
Routes HistorianClient.GetTagMetadataAsync over gRPC when Transport==RemoteGrpc,
via the new Grpc/HistorianGrpcTagClient calling RetrievalService.GetTagInfosFromName
(the plural string-handle metadata op).

- String handle = the Open2 storage-session GUID formatted uppercase (the format
  that resolves the native string-handle path); threaded out of the shared handshake
  via a new HistorianGrpcHandshake.Session { ClientHandle, StorageSessionId, StringHandle }.
- Request btTagNames = uint count + per-name(uint charCount + UTF-16LE) — golden-byte
  unit-tested (BuildTagNamesBuffer).
- Response btTagInfos = uint count + CTagMetadata records — decoded by the existing
  HistorianTagQueryProtocol.ParseGetTagInfoResponse; data type via the shared MapDataType.

The 2020 WCF string-handle wall does NOT apply on the gRPC front door, as the
string-handle-wall RE note predicted. LIVE-VERIFIED against a real 2023 R2 server:
GetTagMetadataAsync returns the requested tag with a valid decoded data type.

216 unit tests pass. Captured framing confirmed live then discarded; no tag names
or identities committed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 14:35:52 -04:00
Joseph Doherty b0703ebf80 docs: R0.3 live-verified; correct the auth-blocker note (harness quote bug)
R0.3 system-param over gRPC is now LIVE-VERIFIED against the real 2023 R2 server
(returned HistorianVersion), alongside the re-confirmed read chain and probe.

The apparent NTLM round-1 SEC_E_LOGON_DENIED "blocker" was a test-harness
credential-parsing bug, not a server/account/SDK issue: the gitignored creds
file stores quoted values and the env-setup must strip surrounding quotes before
exporting HISTORIAN_USER/PASSWORD. With quotes stripped, the NAM domain account
authenticates and the full chain passes. The round-failure diagnostic added
during the hunt (HistorianNativeHandshake.DescribeError) is kept.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 14:29:12 -04:00
Joseph Doherty c4b8d0dde4 gRPC M0: probe (R0.4, live-verified) + system-param (R0.3) + shared handshake
Roadmap docs/plans/hcal-roadmap.md, milestone M0 (gRPC parity for the DONE
surface). Now unblocked for live verification by a reachable 2023 R2 server.

- R0.4 Probe over gRPC: new HistorianGrpcProbe calls History/Retrieval/Status
  GetInterfaceVersion (unauthenticated). ProbeAsync routes over gRPC when
  Transport==RemoteGrpc. LIVE-VERIFIED against a real 2023 R2 server — needs no
  credentials (runs before the auth loop), so it works despite the auth blocker.

- R0.3 System parameter over gRPC: new HistorianGrpcStatusClient calls
  StatusService.GetSystemParameter over the authenticated session; routed in the
  dialect. Built + unit-tested (request/response field mapping pinned).
  Live-verification pending an auth fix (see below).

- Extracted the proven auth handshake from HistorianGrpcReadOrchestrator into
  shared Grpc/HistorianGrpcHandshake (reused by read + status + future
  browse/metadata). Repointed the IL structural guardrail test to it.

- Diagnostics: round-failure now decodes the native server error + hex/ASCII
  preview (HistorianNativeHandshake.DescribeError). This surfaced the live auth
  blocker as SEC_E_LOGON_DENIED (0x8009030C) at NTLM round 1 — framing is correct,
  the credential did not validate. Probable cause: stale file password or NAM-domain
  NTLM restriction (Kerberos/RDP works, NTLM denied; no SPN path over the tunnel).

216 unit tests pass; live gRPC probe passes. Sanitization scan clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 13:32:04 -04:00
Joseph Doherty 22e9c5e5f8 gRPC 2023 R2: fix auth handshake op routing + accept History v12
First live-verified gRPC read against a real 2023 R2 Historian. The handshake
previously failed at round 0 (cred-independent) because the SSPI/Negotiate token
loop was routed to HistoryService.ExchangeKey. ExchangeKey is a separate
key-exchange/cert-path op, not the Negotiate loop — the token loop belongs on
StorageService.ValidateClientCredential, which kept the 2020 inBuff/outBuff token
framing the SDK's WrapValidateClientCredentialToken/TryRead helpers already build.

Captured + diffed against the recovered 2023 R2 protobuf contract and the
decompiled stock client; routing the loop to ValidateClientCredential completes
the full chain (ValidateClientCredential x N -> OpenConnection -> StartQuery ->
GetNextQueryResultBuffer) and returns rows.

- HistorianGrpcReadOrchestrator: token loop now calls
  StorageService.ValidateClientCredential(Handle, InBuff); corrected the op-map
  doc comment (was asserting the wrong ExchangeKey mapping).
- HistorianServerVersionGate: accept History interface version 12 alongside 11.
  Live server reports History=12, Retrieval=4, Storage=4; the buffers are
  byte-identical (a live read returns rows), so 12 is buffer-compatible. Retrieval
  stays pinned at 4 (matches). New AcceptedVersions() supports multi-version gates.
- New HistorianGrpcHandshakeRoutingTests: IL-level structural guardrail that
  disassembles the orchestrator (incl. lambda closures) and asserts the handshake
  invokes ValidateClientCredential and never ExchangeKey — fails if the regression
  returns.
- Updated gate tests + CLAUDE.md gRPC op-map.

240 unit tests pass on the full stack; 210 on this branch's base. The byte
payloads remain the proven 2020 protocol.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 12:34:04 -04:00
Joseph Doherty c1b1b3d23b R1.11 DelTep capture + R1.3/R1.4/R1.12/R1.13 bounded out
DelTep (extended-property delete) — wire format captured + serializer
golden-proven, but live delete is server-blocked and NOT exposed publicly:
- Captured the DelTep inBuff via a cross-session trick (harness add-tep gains
  --tep-skip-add + read-for-sync before --tep-delete; Capture-DeleteTagExtended
  Properties.ps1 / decode-del-tep-capture.py). Layout = same group framing as
  AddTEx but property-name-only (no 0x43 value) + 0x00 group trailer.
- SerializeDeleteRequest + 4 golden tests pin the server-accepted buffer.
- A decisive experiment shows SDK-added properties ARE deletable (the native
  client read-syncs and deletes one), so SDK-add is complete; the SDK's own
  DelTep is rejected by CHistStorage::DeleteTagExtendedProperties even with
  byte-identical inBuff, matching mode/handle, GetTgByNm+GetTepByNm prime, open
  channel, and 60s retries. Root cause: the native multiplexes services over one
  connection (per-connection working set); the SDK's per-service WCF channels
  don't reproduce it. Kept as documented-but-blocked internal orchestrator path;
  no public HistorianClient delete API.

Bounded out with evidence (no code; docs + roadmap + probe):
- R1.12 localized-property write — no op on 2020 (mirror of R1.6); no
  *LocalizedPropert*/TagLocalized* symbol in any current/*.dll.
- R1.13 non-analog tag create — GATED; native AddTag rejects every non-analog
  type client-side (ValidationFailed, before any WCF op): SingleByteString,
  DoubleByteString, Int1 all fail, Float works. No Discrete type in the native
  enum, no TagType setter. No wire request to capture.
- R1.3 timezone + R1.4 EventStorageMode — re-confirmed 2023R2/gRPC-only from
  the Runtime DB schema (no timezone param, no EventStorageMode anywhere) and a
  parameter-op probe (GetSystemParameter + GETRP return null/throw for every
  candidate; only HistorianVersion works).

238 unit tests pass; full solution builds with 0 warnings.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 11:26:21 -04:00
Joseph Doherty 08b950caee R1.11 AddTagExtendedPropertiesAsync: extended-property write via AddTEx
Adds user-defined extended properties to an existing tag via the 2020 WCF
AddTEx (AddTagExtendedProperties) op. Write-enabled connection + uppercase
storage-session GUID handle; reuses the write orchestrator open/priming chain.

The AddTEx inBuff is the exact inverse of the R1.5 GetTepByNm read-response
framing, so the serializer mirrors the read parser:
  uint32 groupCount + 0x01(group) + [0x09+u16+ASCII tag] + uint32 propCount
  + per prop{ 0x02 + [0x09+u16+ASCII name] + 0x43 VT_BSTR + u16 payloadLen
  + u16 charCount + UTF-16 value } + 0x01(group trailer) + 0x00(terminator).
The trailing 0x00 is required — without it inBuff is one byte short and the
server throws SErrorException in CHistStorage::AddTagExtendedProperties. The
golden fixture pins the clean inBuff the live server accepted (dumped via
AVEVA_HISTORIAN_TEP_DUMP); read-back verified via R1.5. String (0x43) values only.

Delete (DelTep) is deferred: the native DeleteTagExtendedPropertiesByName does a
client-side sync check and returns err 229 for a just-added property, so the
DelTep request never reaches the wire and its inBuff can't be captured yet.

Shipped: HistorianClient.AddTagExtendedPropertiesAsync/AddTagExtendedPropertyAsync;
HistorianTagExtendedPropertyProtocol.SerializeAddRequest; orchestrator path;
golden WcfTagExtendedPropertyWriteProtocolTests (4); gated live write/read-back test;
native-harness `add-tep` scenario + Capture-AddTagExtendedProperties.ps1 +
decode-add-tep-capture.py. Doc: wcf-add-tag-extended-properties.md. 233 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 01:43:19 -04:00
Joseph Doherty bc353df8c4 R1.10 RenameTagsAsync: async tag rename via History StartJob (StJb)
Tag rename has no dedicated WCF op — the (old,new) name batch rides the
generic History StartJob (StJb) job buffer; the server returns a job id and
applies renames asynchronously. Handle is the uppercase storage-session GUID,
Open2 in write mode; reuses the write orchestrator's open+priming chain.

jobBuffer layout (decoded + server-validated): byte[7] zero prefix + uint32
pairCount + per pair (uint32 oldCharCount + UTF-16 oldName + uint32
newCharCount + UTF-16 newName), order (old,new). The raw instrument capture
mangles the final byte with MDAS chunk markers (the R1.1 lesson), so the golden
fixture pins the CLEAN byte[] the SDK handed the channel (dumped via
AVEVA_HISTORIAN_RENAME_DUMP) — the exact buffer the live server accepted and
renamed with.

Gated server-side by the AllowRenameTags system parameter (default 0): when
disabled the native client rejects pre-wire (err 132); the managed SDK surfaces
it as StartJob=false -> Accepted=false. Enabling needs a Historian config
reload, not just a storage-engine restart.

Shipped: HistorianClient.RenameTagAsync/RenameTagsAsync -> HistorianTagRenameResult;
HistorianTagRenameProtocol; orchestrator RenameTags/SendStartJobRename; golden
WcfTagRenameProtocolTests (4, pins server-accepted buffer); gated live test
RenameTagsAsync_AgainstLocalHistorian_RenamesSandboxTag (passed end-to-end).
Native-harness `rename` scenario + Capture-RenameTags.ps1 + decode-rename-capture.py.
Doc: docs/reverse-engineering/wcf-rename-tags.md. 213 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 01:18:41 -04:00
Joseph Doherty fbd839077b R1.4 GetHistorianInfo: bounded out on 2020 WCF (named-value-only, no struct)
Captured the native HistorianAccess.GetHistorianInfo(out HistorianInfo, out err)
and decoded the wire: over 2020 WCF, GETHI is a named-value query whose only
working key is "HistorianVersion" (response ~30 bytes = the version string).
Probed 7 storage-mode key names -> all ok=False/err. The 518-byte HISTORIAN_INFO
struct + EventStorageMode@514 is the 2023R2 HCAL-native/gRPC model (confirmed
from the decompiled 2023R2 source); on 2020 the native client derives the mode
outside the WCF wire.

Version is already exposed (ProbeAsync/GetRuntimeParameterAsync), so no hollow
GetHistorianInfoAsync is shipped (same disposition as R1.3 timezone). This
completes the reachable 2020-WCF M1 read surface; remaining M1 = config writes
(gated on explicit request) or gRPC/2023R2-only items.

RE aids kept: harness `historian-info` scenario, Capture-HistorianInfo.ps1,
decode-historian-info-capture.py, and StringHandleProbeDiagnosticTests
.GETHI_CandidateInfoNames (asserts the named-value-only finding; gated).
Docs: wcf-historian-info.md (new) + roadmap/matrix/wall-doc updates. 230 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 23:42:27 -04:00
Joseph Doherty 1a539882d0 R1.1 ExecuteSqlCommandAsync (ExeC + GetR, NRBF DataTable, no BinaryFormatter)
Ship SQL command execution over the 2020 WCF aa/Retr/ExeC + aa/Retr/GetR ops:
HistorianClient.ExecuteSqlCommandAsync(sql) -> HistorianSqlResult (columns +
typed rows). String-handle ops reached with the Open2 storage-session GUID
formatted uppercase (the handle format that unlocked GETRP/GETHI).

Chain: Retr.GetV prime -> ExeC(handle, sql, option=0, ref queryHandle) ->
GetR loop. Key gotcha captured: GetR returns FALSE even on success -- the byte
stream is in pResultBuff regardless; false just signals the final page. So the
orchestrator consumes the buffer first, then stops on a false result / empty page.

GetR's pResultBuff is an NRBF-serialized System.Data.DataTable
(SerializationFormat.Xml: members XmlSchema (XSD) + XmlDiffGram (rows)).
BinaryFormatter is removed from .NET 10, so the stream is decoded read-only with
the System.Formats.Nrbf package (NrbfDecoder) + XDocument -- no BinaryFormatter,
no code execution. Values are typed per the XSD type, falling back to string.

Adds: HistorianSqlResult / HistorianSqlColumn / HistorianSqlExecuteOption models,
HistorianSqlResultProtocol (NRBF + diffgram parser), HistorianWcfSqlClient
(ExeC/GetR orchestration with an AVEVA_HISTORIAN_SQL_DUMP diagnostic), dialect +
public API. Golden WcfSqlResultProtocolTests pinned to the real clean GetR stream
for the benign "SELECT 1 AS ProbeValue" (no sensitive data); gated live tests
(single cell + multi-column/multi-row/NULL). Doc: wcf-exec-sql.md; roadmap R1.1
DONE; wall doc + memory updated (incl. the QTB-server-side nuance). 229 tests green.

Note: a raw instrument-wcf capture corrupts a large pResultBuff with MDAS
transport chunk markers (0x9F); the clean contract-level byte[] is dumped via the
AVEVA_HISTORIAN_SQL_DUMP env var for the golden fixture.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 23:16:06 -04:00
Joseph Doherty 108220c36b R1.5 GetTagExtendedPropertiesAsync (GetTepByNm) + R1.6 closed (no op)
Ship tag extended-property reads over the 2020 WCF aa/Retr/GetTepByNm op:
HistorianClient.GetTagExtendedPropertiesAsync(tag) -> name/value pairs.

String-handle op reached with the Open2 storage-session GUID formatted
uppercase (same format that unlocked GETRP/GETHI/ExeC). Routed via the
name-based native path (GetTagExtendedPropertiesByName, server-fetch flag),
not the index-based TagQuery path.

Evidence-backed findings from the capture:
- GetTepByNm (and GetTgByNm) succeed with the uppercase handle -- further
  validates the resolved string-handle wall.
- QTB (StartTagQuery) does NOT punch through: captured uppercase, it still
  fails server-side (CMdServer::StartActiveTagnamesQuery over the
  aahMetadataServer pipe) -- a metadata-server blocker, not handle format.
- R1.6 (localized properties) has NO distinct op (only error-message/UI-text
  localization in the managed client); collapses into R1.5. Closed, not throwing.

Wire format (golden-pinned, synthetic bytes -- no dev tag names committed):
- request tagNames = uint count + per-name(uint charCount + UTF-16)
- response = uint tagCount + per-tag(marker + compact-ASCII name +
  uint propCount + per-prop(marker + compact-ASCII name + 0x43 VT_BSTR value)
  + trailer); sequence-paged.

Adds: HistorianTagExtendedProperty model, HistorianTagExtendedPropertyProtocol
(codec), HistorianWcfTagExtendedPropertyClient (orchestration), dialect +
public API; golden WcfTagExtendedPropertyProtocolTests (4) + gated live test
(HISTORIAN_TEP_TAG). Tooling: Capture-TagExtendedProperties.ps1,
decode-tag-properties-capture.py, harness tag-extended-properties scenario.
Docs: wcf-tag-extended-properties.md; roadmap R1.5 DONE / R1.6 collapsed;
wall doc + memory updated with the QTB-server-side nuance. 228 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 22:52:07 -04:00
Joseph Doherty 4da5287d01 R1.2 GetRuntimeParameter + string-handle wall RESOLVED (handle-format bug)
Execute HCAL roadmap R1.2 (GetRuntimeParameterAsync) end-to-end, and in doing so
discover that the "string-handle wall" blocking R1.1/R1.4/R1.5/R1.6 was a handle
FORMAT bug, not a missing native session/filter registration.

R1.2 (shipped, live-verified):
- Captured native GetRuntimeParameter -> WCF op aa/Stat/GETRP (string-handle op,
  GETHI's shape), via scripts/Capture-RuntimeParam.ps1 + instrument-wcf-{write,read}message.
- HistorianRuntimeParameterProtocol serializes pRequestBuff (54 67 01 00 + uint
  nameCount + per-name uint charCount + UTF-16) and parses pResponseBuff (version +
  uint resultCount + CRetVariant 0x43 VT_BSTR + uint16 len + uint16 charCount + UTF-16).
- IStatusServiceContract2.GetRuntimeParameter (GETRP) op; HistorianWcfStatusClient
  passes the Open2 storage-session GUID as the string handle, UPPERCASE.
- Public HistorianClient.GetRuntimeParameterAsync(name) via the dialect.
- Golden WcfRuntimeParameterProtocolTests + gated live test; returns HistorianVersion.

String-handle wall RESOLVED (proven, public APIs deferred):
- The Open2 storage GUID works as the string handle when sent UPPERCASE
  (ToString("D").ToUpperInvariant()); earlier "blocked" probes used lowercase.
- Live-probed GETHI (R1.4) -> returns data; ExeC (R1.1) -> Retr.GetV prime -> ExeC ->
  GetR returns a BinaryFormatter-serialized .NET DataTable. Gated
  StringHandleProbeDiagnosticTests + scripts/Capture-ExecSql.ps1 + exec-sql harness scenario.
- Docs flipped: wcf-string-handle-wall.md RESOLVED banner; roadmap R1.1/R1.4 reachable,
  R1.5/R1.6 likely; wcf-status-localhost.md GETRP section.
- R1.1/R1.4 public APIs NOT shipped: ExeC needs a GetR paging loop + a BinaryFormatter-
  stream parser (BinaryFormatter is removed from .NET 10); GETHI full-info struct needs
  its own capture.

223 unit tests pass; gated live tests green against the local 2020 Historian.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 22:10:31 -04:00
Joseph Doherty 6d470eab4a R1.7: server-side event filters — ReadEventsAsync(HistorianEventFilter), live-honored
Roadmap M1 R1.7. Filters are set on the native EventQuery object via
AddEventFilter(property, HistorianComparisionType, value) — NOT EventQueryArgs
(time/count/order only). Found via a new harness --dump-type-members command.

Captured the native filtered StartEventQuery pRequestBuff (Capture-EventFilter.ps1 +
harness --event-filter knob) and diffed Equal(0) vs Contains(12) to isolate the
operator field. Filter block (decoded byte-for-byte):
  ushort 0 + uint filterCount + uint condCount + uint nameLen + name(UTF-16) +
  uint 1 + ushort op + uint 1 + value(0x09-LEN-0x00 compact-ASCII) + byte 0

The filter is REAL, not inert (unlike the analog-summary knobs): a non-matching
predicate returns 0 events; Type=Equal=User.Write returns only User.Write events.
Verified live via both the native harness and the SDK.

- HistorianClient.ReadEventsAsync(start, end, HistorianEventFilter, ct) overload
- HistorianEventFilter + HistorianEventComparison (18 ops, ordinals = native)
- Filter encoding in HistorianEventQueryProtocol (empty-filter path unchanged)
- Golden-byte tests (block match, op field, empty-filter regression) + gated live test

Single string-valued predicate only; multi-filter (OR) / multi-condition (AND via
AddEventFilterCondition) framing is partially captured and not shipped. 216 unit
tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 18:32:03 -04:00
Joseph Doherty f1e23a3a02 M2: implement SendEventAsync — event-send rides WCF AddS2, not the storage pipe
Roadmap Milestone 2 (event sending). Capture disproved the assumption that event
delivery uses the non-WCF storage-engine pipe (which would block it like revision
writes): a native AddStreamedValue(HistorianEvent) leaves over WCF as AddS2
(IHistoryServiceContract2.AddStreamValues2). CM_EVENT is a built-in registered tag,
so the 129 TagNotFoundInCache gate that blocks AddS2 for user tags does not apply.

- R2.1: NativeTraceHarness "event-send" scenario + Capture-EventSend.ps1; two
  captures diffed to separate constant framing from value-dependent fields.
- R2.2: HistorianEventWriteProtocol serializes the AddS2 pBuf (storage sample buffer
  wrapping the event VTQ) — golden-byte tested. Decoded "OS" sig + length fields +
  CM_EVENT tag id + EventTime/ReceivedTime FILETIMEs + Opc 192 + 0x118D descriptor +
  event Id + Namespace + EventType + version 5 + typed property bag.
- R2.3/R2.4: HistorianWcfEventOrchestrator.SendEventAsync (Open2 event-mode 0x501 ->
  reuse CM_EVENT RTag2/EnsT2 -> AddStreamValues2) + HistorianClient.SendEventAsync.
- R2.5: gated live test; server accepts the AddS2 (success, empty error buffer).

Server requires delivered byte[].Length == declared packet length (uint32@0x04); the
native relies on the MDAS encoder adding a pad byte, so the SDK emits an explicit
trailing 0x00 (else AddS2 rejects with "CValuStream buffer size vs packet length
mismatch"). Original events only (RevisionVersion=0) with string properties; other
property types + revision/update/delete throw ProtocolEvidenceMissingException.

Caveat (documented): accepted events are not persisted on the local dev box; the
native client behaves identically (event ingestion pipeline inactive) — not an SDK
gap. 212 unit tests pass; 16/16 event tests pass live.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 18:00:52 -04:00
Joseph Doherty 1a7519c803 RE: resolve R1.8/R1.9 analog/state summary via request+response capture
Captured the native StartQuery2 pRequestBuff and the GetNextQueryResultBuffer2
response (instrument-wcf-writemessage + chained instrument-wcf-readmessage) and
decoded both against AnalogSummaryHistory SQL ground truth. Conclusion: the rich
multi-aggregate analog/state summary struct is NOT delivered over the 2020 WCF
binary protocol — the response is the ordinary version-9 row buffer the existing
aggregate parser already handles, carrying one value per cycle selected by
RetrievalMode (QueryType 5-8), not ValueSelector (inert on this path). So
"analog summary" == the existing ReadAggregateAsync; no new src/ code warranted.

Tooling (tools/ + scripts/ only, nothing in src/):
- NativeTraceHarness: drive summary knobs via --value-selector /
  --aggregation-type / --max-states (uint16) / --filter
- Capture-SummaryRequest.ps1: repeatable instrument+stage+matrix capture,
  -WithResponse chains the ReadMessage hook
- decode-summary-capture.py: StartQuery2 request diff vs baseline
- decode-summary-response.py: response decode vs SQL ground truth

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 17:01:42 -04:00
Joseph Doherty 362fcb0ef4 Merge feat/r1.8-r1.9-summary-scope: summary-query scoping + native-capture finding
Scoped R1.8/R1.9 (analog/state summary). Reachable on 2020 WCF via the proven
uint-handle StartQuery2; all decode targets located in aahClientManaged.dll
(CAnalogSummaryValue.UnpackFromValueBuffer + struct field sets). Live probing
showed the server accepts SummaryType 1/2/4/5 but returns 0 rows with the
obvious params — the summary config is native-side (AutoSummaryParameters
trailer / native QueryType), not recoverable from managed metadata. Next step
is a native pRequestBuff capture via instrument-wcf-writemessage.

Adds the enum-dump RE CLI command. No guessed code in src/; 208 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 16:12:58 -04:00
Joseph Doherty 34e352ba28 R1.8/R1.9: empirical summary-query probe + enum-dump RE command
Pushed on recovering the summary query params. Findings:

- Added `enum-dump` to the RE CLI (dumps a managed enum's literal members).
  Confirmed INSQL_QUERYTYPE / HISTORIAN_SUMMARYTYPE are value__-only in the
  managed metadata — their named members are native-side constants, so they
  can't be recovered statically. Same for CColumnNameMap.LoadColumnNameMap
  (column->bit built from native string/const data, not IL ldstr/ldc).

- Live-probed StartQuery2 against SysTimeSec sweeping QueryType/SummaryType/
  ColumnSelectorFlags. The server ACCEPTS SummaryType 1/2/4/5 (returns a valid
  version-9 buffer) but yields 0 rows; column flags don't change that;
  QueryType 15/16 don't exist. So a summary query is NOT Full+SummaryType+
  flags — the config lives in the AutoSummaryParameters trailer (currently
  zeroed) and/or a native summary QueryType.

Conclusion recorded in the plan: the request shape needs a NATIVE capture
(instrument-wcf-writemessage on a real summary query), not managed-metadata
recovery or blind probing. Decode targets remain located. No guessed code in
src/; probe scaffolding removed. 208 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 16:11:35 -04:00
Joseph Doherty 085f01123c docs: scope R1.8/R1.9 summary queries (decode targets located)
Established that analog/state-summary queries are reachable on 2020 WCF — they
ride the proven uint-handle StartQuery2 path, and the request serializer already
carries QueryType/SummaryType/ColumnSelectorFlags. Located every decode target in
aahClientManaged.dll:

- CAnalogSummaryValue.UnpackFromValueBuffer (0x06000394) — row decoder
- CAnalogSummaryValue/Struct fields — Min/Max/First/Last/ValueCount/TimeGood/
  Integral/IntegralOfSquares (+ per-field DateTimes, LinearIntegral)
- CStateSummaryStruct — MinContained/MaxContained/TotalContained/PartialStart/
  PartialEnd/StateEntryCount
- QueryColumnSelector.Select{Analog,State,NonSummary}Columns — column flags
- INSQL_QUERYTYPE / HISTORIAN_SUMMARYTYPE — the query/summary enum values

UnpackFromValueBuffer is reader-call-based (no literal offsets), so a correct
parser needs a captured real buffer. Per project discipline no guessed summary
code was added to src/. New plan doc lays out the recover-params -> live-capture
-> decode -> implement+test path. Roadmap R1.8/R1.9 marked scoped/ready.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 15:53:59 -04:00
Joseph Doherty 4786ca9594 Merge feat/m1a-findings-r1.4-gethi: map the 2020 WCF string-handle wall
Probed M1 read items R1.1/R1.3/R1.4 live against the local 2020 server and
found them blocked, then documented the structural boundary (uint-handle ops
work; string-handle ops require unmapped native session registration). No
half-implementations shipped; 208 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 15:34:19 -04:00
Joseph Doherty 84ec175f76 docs: map the 2020 WCF string-handle wall (R1.4 GETHI blocked)
Probed R1.4 GetHistorianInfo (GETHI) live against the local 2020 server.
GETHI returns native error type 4 / code 1 for the exact native request shape
across 5 handle formats (storage GUID, context GUID, uint decimal/X8/0x-hex)
even with Stat.GetV ×2 priming. Its result is discarded (TryRun) in the only
place it's used, so it was never actually verified to return data managed-side.

This confirms a structural boundary on the 2020 WCF surface: ops taking a uint
client handle work (the proven read/browse/metadata/status/event surface);
ops taking a string GUID handle (ExeC, QTB, QTG, GETHI, GetTepByNm, ...) are
blocked behind an unmapped native session/filter registration. Every remaining
M1 *read* item (R1.1/R1.4/R1.5/R1.6) is string-handle -> all gated on that one
RE target. Reachable uint-handle items: R1.7 event filters, R1.8/R1.9 summary
modes.

New: docs/reverse-engineering/wcf-string-handle-wall.md (full dichotomy + table).
Roadmap R1.4/R1.5/R1.6 struck through; reachable items re-pointed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 15:32:15 -04:00
Joseph Doherty 2246fdd395 docs: reclassify M1a R1.1/R1.3 as blocked on 2020 WCF
Live-probed both against the local Historian 2020 (WCF):

- R1.3 GetServerTimeZoneAsync: Status.GetSystemTimeZoneName returns rc=0 with
  an empty value under a real authenticated handle — a client-side stub in the
  GetServerTime family. gRPC/2023R2-only. Reverted the implementation.
- R1.1 ExecuteSqlCommandAsync: Retrieval.ExeC returns native error type 4 /
  code 51 (InvalidParameter); the contract-3 string-handle ops require an
  unmapped native session/filter registration step (the StartTagQuery wall).

Adds an M1a re-classification note steering future work toward proven
uint-handle / already-wired ops (R1.4 GETHI next) over string-handle ops.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 15:22:53 -04:00
Joseph Doherty 5e28f8d92e Merge feat/grpc-transport-2023r2: gRPC transport + R0.6 gate + CW-1 capture pipeline
- 2023 R2 RemoteGrpc transport reusing the proven native byte payloads (unit-tested;
  not yet live-verified — no 2023 R2 server on the local box, which is 2020/WCF).
- R0.6 fail-closed server interface-version gate (Hist=11, Retr=4, Trx=2; evidence-based).
- CW-1 reusable capture -> sanitize -> golden-fixture pipeline + capture-tag-info CLI.
- docs/plans: imported gRPC/HCAL analysis + roadmap, progress-marked.
- Event reads live-verified + test hardened to assert well-formed parsed events.

169 non-live unit tests green; 27/27 integration tests green against the local 2020 server.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 14:54:42 -04:00
Joseph Doherty 7d5aeaeb06 Strengthen live event-read test: assert well-formed parsed events
ReadEventsAsync verified to return real, parsed events against the local 2020
server (e.g. User.Write with 18 properties) — the row parser (HistorianEventRowProtocol
v9) is wired and works. The prior test only asserted NotNull with a stale "row
format not yet decoded" comment.

- Renamed to ReadEventsAsync_AgainstLocalHistorian_ReturnsWellFormedEvents.
- Widened the window to 30 days (robust against a quiet recent window).
- Asserts NotEmpty + per-event well-formedness (non-empty Type, non-null
  Properties, EventTimeUtc within the queried window) — matching the ReadRawAsync
  test's NotEmpty style.
- Documents the known limitation: enumeration stops at the first benign
  `type=4 code=85` soft-terminal, so this verifies parsing correctness rather than
  exhaustive retrieval (draining all rows needs the code-85 decode, a capture task).

Passes live (1 event over 30 days). Non-live unit suite unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 15:12:41 -04:00
Joseph Doherty cf5a66e046 docs/plans: mark R0.6 + CW-1 done; note 2020-only live-verification constraint
The local Historian is 2020 (WCF/32568); the 2023 R2 gRPC endpoint (32565) is
absent, so M0 gRPC routing can be unit-tested but not live-verified here.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:57:38 -04:00
Joseph Doherty fa9cde3e2f CW-1: reusable capture -> sanitize -> golden-fixture pipeline
Adds the highest-leverage reverse-engineering primitive from the roadmap: one
path to turn a live operation buffer into a committable golden fixture. Unblocks
every capture-tier item (R0.5, R1.x, R2.1).

- ProtocolCaptureSanitizer: redacts identity-bearing values (host, tag, user,
  machine) from a native buffer in BOTH ASCII and UTF-16LE, overwriting in place
  with an 'X' fill so length and every field offset are preserved (keeps the
  fixture useful for byte-layout RE). ASCII-letter matching is case-insensitive;
  secrets < 3 chars are skipped to avoid collision corruption. AssertNoSecretsRemain
  is a fail-closed safety net that refuses to emit if any value survives.
- ProtocolFixtureWriter: serializes a capture to fixtures/protocol/<op>/<name>.json
  with sanitized hex, length, SHA-256 of the sanitized bytes, and a scrub report.
  Timestamps are passed in (deterministic / testable).
- capture-tag-info CLI command: captures a live GetTagInfoFromName response and
  writes the fixture. The same native bytes ride inside 2023 R2 gRPC
  GetTagInfosFromName, so the fixture is transport-agnostic.
- 11 unit tests for the sanitizer/writer (test project now references the RE tool).
- First real fixture: get-tag-info/analog-*.json — a 98-byte Int4 CTagMetadata
  buffer captured live from the local Historian 2020 server, tag name redacted,
  verified to contain no identity (descriptor 03 c3 00 31 = Int4, as documented).

180 non-live unit tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:56:48 -04:00
Joseph Doherty 6b892b69ba R0.6: fail-closed server interface-version gate at connect
Turns the previously-discarded GetInterfaceVersion result into a connect-time
version pin. The native buffers carried in the WCF/MDAS body (and in the 2023 R2
gRPC bytes fields) are framed per native interface version; parsing them against
an unexpected version risks silent misinterpretation, so we throw rather than
best-effort parse.

- HistorianServerVersionGate + HistorianServiceInterface: evidence-based
  supported versions discovered from a live Historian 2020 server (product
  20.0.000) via the wcf-probe command — History=11, Retrieval=4, Transaction=2.
  Status' GetInterfaceVersion returns 0, so Status is reachability-only.
- HistorianClientOptions.VerifyServerInterfaceVersion (default true) — bypass
  knob for bringing up a server whose reported integers aren't yet captured
  (e.g. a 2023 R2 gRPC endpoint carrying the same proven 2020 buffers).
- Wired into both transports' connect paths: WCF history (auth-chain helper) +
  retrieval (read orchestrator), and gRPC history + retrieval.
- Mismatch throws ProtocolEvidenceMissingException naming reported/expected
  version and the bypass knob.

10 new unit tests (198 total green). Verified the gate does not regress the
proven WCF read path: a live read against the local 2020 server reaches past the
gate (Retr=4 matches) — the only live failures are a pre-existing environmental
read timeout (OperationCanceledException), identical with and without this change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:48:52 -04:00
Joseph Doherty a530ae0f10 docs/plans: import 2023 R2 gRPC analysis + HCAL reimpl roadmap
Version-control the planning docs alongside the code they describe:
- grpc-transport.md     — 2023 R2 gRPC transport analysis (sanitized source path)
- hcal-capability-matrix.md — HistorianAccess surface x gRPC ops x histsdk status x feasibility tiers
- hcal-roadmap.md       — ordered build plan M0-M4 + cross-cutting workstreams
- histevents.md         — how a HistorianEvent reaches the DB (client->wire->server)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:28:34 -04:00
Joseph Doherty 1e9a87fce9 Add 2023 R2 gRPC transport (RemoteGrpc) reusing native byte payloads
Stands up HistorianTransport.RemoteGrpc end-to-end for the read path,
built on the recovered 2023 R2 gRPC contract (gRPC-Web/HTTP-1.1, port
32565, gzip). The opaque protobuf `bytes` fields carry the SAME native
binary payloads as the 2020 WCF/MDAS path, so the proven serializers and
parsers are reused unchanged.

- Grpc/Protos/*.proto: 6 protoc-validated contracts recovered from
  embedded FileDescriptors (authoritative, not guessed).
- Grpc/HistorianGrpcChannelFactory: GrpcWebHandler/HTTP-1.1 channel,
  ResolvePort/ResolveAddress, optional TLS + gzip.
- Grpc/HistorianGrpcReadOrchestrator: mirrors the WCF read chain over
  gRPC; auth uses HistoryService.ExchangeKey (the gRPC ValCl op).
- Wcf/HistorianNativeHandshake: transport-agnostic Open2 request builder
  + SSPI/Negotiate token loop + response decode, shared by WCF and gRPC.
- Op map (2020 -> gRPC): ValCl->ExchangeKey, Open2->OpenConnection,
  StartQuery2->StartQuery, GetNextQueryResultBuffer2->GetNextQueryResultBuffer.
- HistorianClientOptions: DefaultGrpcPort=32565, GrpcUseTls.
- csproj: Google.Protobuf, Grpc.Net.Client(.Web), Grpc.Tools codegen.

Not yet live-verified against a 2023 R2 server: ExchangeKey is the first
thing to revisit if a live server rejects the handshake; the inner byte
payloads are the proven 2020 protocol. Gated live test via
HISTORIAN_GRPC_HOST. 188 unit tests green; build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:27:47 -04:00
Joseph Doherty 5efa767721 HistorianWcfTagClient: respect options.Transport for cert variant
Browse/metadata previously hardcoded the Windows transport binding +
/Hist-Integrated endpoint regardless of options.Transport. That meant
RemoteTcpCertificate clients (notably anything from Linux, where the
Windows transport security isn't available in WCF) couldn't use these
ops even when other reads worked.

Made the binding+endpoint selection conditional:
- LocalPipe / RemoteTcpIntegrated: keep existing /Hist-Integrated +
  Windows transport binding (the legacy Open2-V1 buffer carries its
  own auth blob and only validates against this transport).
- RemoteTcpCertificate: switch to /HistCert + cert binding, propagating
  options.ServerDnsIdentity. Cert-binding callers also opt out of
  setting Windows credentials on the factory.

Cert-validation override (HistorianWcfClientCredentialsHelper.Configure)
now applies on both the history and retrieval factories.

Confirmed locally: 178/178 tests pass on Windows. Linux verification
still pending — gated tests don't exercise the path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:07:13 -04:00
Joseph Doherty 8a553423ed D2: definitive conclusion — revision-write requires non-WCF storage-engine pipe
IL walk of the native wrapper:

  HistorianAccess.AddRevisionValuesBegin (private, token 0x06006175)
    -> CClientCommon.AddNonStreamValuesBegin
       -> CClient.AddNonStreamValuesBegin (8-instr overload)
          -> CClient.TransactionBegin
             -> CHistStorageConnection.StartTransaction (token 0x06001FDD)
                -> CStorageEngineConsoleClient.StartTransaction

CStorageEngineConsoleClient is built on STransactPipeClient2 +
SCrtMemFile — a shared-memory + named-pipe transport to
aaStorageEngine.exe, completely separate from WCF.

The WCF ITransactionServiceContract2.AddNonStreamValuesBegin2 op is a
server-side relay that requires a pre-existing storage-engine pipe
session for the client. Without that pipe session, the WCF relay returns
UnknownClient (51) — and there's no way to establish the pipe session
via WCF.

D2 is unimplementable as a pure-managed-WCF SDK. The native wrapper
itself depends on the C++ shared-memory channel; replicating that from
managed code would require implementing the storage-engine pipe
protocol, which is a major undertaking and out of scope.

The ITransactionServiceContract2 declaration in our contracts file
stays as documentation; no public API or orchestrator added.
HistorianWcfRevisionOrchestrator remains as an internal probe /
regression check — re-run the probe test if anyone believes the
architecture has changed.

178/178 tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 02:59:29 -04:00
Joseph Doherty 6b385441c1 D2 follow-up: RTag2 doesn't cascade client identity to Trx
Tested hypothesis (1) from the plan: add RTag2(CM_EVENT tag id) to the
priming chain before AddNonStreamValuesBegin2.

Result:
- RTag2 itself succeeds: returns 25-byte response
  (01000000000100000001EE39C30EDCDC010100000000000000), no error buffer.
- But AddNonStreamValuesBegin2 still fails with the same
  04 33 00 00 00 (UnknownClient = 51) for all four handle formats.

So RTag2 on /Hist isn't the cross-service registration trigger we need
for /Trx. Plan doc updated with the result + next-session ordered
probes (try IStorageServiceContract, then IL walk CClientCommon,
then server-side decompile as last resort).

Probe orchestrator now also performs the RTag2 step so the test gives
one-shot diagnostic visibility of both calls.

178/178 tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 02:54:52 -04:00
Joseph Doherty b40e6948e2 D2 (new path): SDK-direct WCF revision orchestrator + probe
Implemented HistorianWcfRevisionOrchestrator that talks WCF directly
to /Trx, bypassing the native wrapper entirely. Probes
AddNonStreamValuesBegin2 against the live local Historian and surfaces
what the server returns. Internal-only API; no public surface added —
the path isn't viable yet.

Findings (live test against localhost):

-  The wire path is reachable. After moving from V1 (uint handle, no
  errorBuffer) to V2 (string handle GUID, out errorBuffer), the server
  recognizes the call (no ContractFilter mismatch, no exception).
-  Server processes the call and returns a structured 5-byte error
  buffer: 04 33 00 00 00 = type 4 (CustomError) + code 51
  (UnknownClient).
-  Tried four handle formats (contextKey upper/lower, storageSessionId
  upper, ClientHandle as decimal string) — all return the same
  UnknownClient.
-  Adding the full priming chain (Stat.GetV ×2, Stat.GETHI ×2, UpdC3,
  6× Stat.GetSystemParameter, AllowRenameTags, Trx.GetV, Stat.GetV,
  Retr.GetV) — same result.

ITransactionServiceContract2 has no Validate/Register/Open op of its
own. The client-with-Trx registration must happen via some cross-
service side effect we haven't isolated.

Important takeaway: the wire-format mismatch is solved (contract method
names + parameter shapes match what the server expects). The remaining
gap is a single missing initialization step. Documented in
docs/plans/revision-write-path.md as concrete next-session steps.

178/178 tests pass (one new probe test added). Probe is gated on
HISTORIAN_HOST=localhost.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 02:51:26 -04:00
Joseph Doherty b5e5f5485b D2: gate is in the C++ HistorianClient, not the managed wrapper
Direct HistorianAccess.AddNonStreamedValue (the 4-param overload that
bypasses HistorianDataValueList and goes straight to
HistorianClient.AddNonStreamedValueAsync) ALSO fails with 129
TagNotFoundInCache against SysTimeSec, even with validate=false.

So the cache check is inside the native C++ HistorianClient's
per-connection tag list — there's no managed-callable bypass.

Critical insight discovered: the SDK doesn't use the C++ HistorianClient
at all. It talks WCF directly. The cache gate that blocks the native
wrapper may not block a managed WCF client because the gate is enforced
by aahClientManaged, not by the WCF server.

This shifts the recommendation for any future D2 attempt from "wrap the
native API" (which is genuinely blocked) to "implement the wire path
directly on top of the existing ITransactionServiceContract methods and
test against the live server" (unverified but plausibly viable). The
harness can't help with that path — the wrapper itself is the blocker
we'd be bypassing.

177/177 tests still pass; harness gains --write-revision-direct flag
for further probing of the native-wrapper path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 02:34:02 -04:00
Joseph Doherty 3af8a13059 D2 (revision-write): probe SysTimeSec — same gate, narrower scope
Extended the harness with --write-revision-target-tag <name> (overrides
the value's TagKey via SQL lookup) and --write-revision-skip-validate
(passes false to AddNonStreamedValue's `validate` boolean). Added
--write-revision-commit gate so the harness validates without actually
calling SendValues by default — important when targeting system tags.

Probed SysTimeSec (wwTagKey=12, server-cache-resident system tag):
- AddNonStreamedValue: ErrorCode=TagNotFoundInCache (129) — same failure
- With validate=false: same failure (the cache check is intrinsic, not
  gated by the boolean)

Conclusion: the gate is per-(client-session, tag), not per-server-cache.
Even tags the SERVER cache knows about are rejected because the LIBRARY
maintains a separate per-connection tag list that AddNonStreamedValue
checks. That list isn't populated by knowing the wwTagKey alone — it
needs whatever mechanism (RegisterTags2 / read flow side effect / IO
server registration) that we haven't reverse-engineered.

The revision-write path remains architecturally blocked for managed
clients. Plan doc updated with the SysTimeSec finding.

177/177 tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 02:27:58 -04:00
Joseph Doherty 2feb56d52c D2 (revision-write): empirically blocked by same gate as AddS2
Drove the revision-write flow via reflection in the native trace harness
(--write-revision-values) to see whether it bypasses the AddS2
architectural blocker. It doesn't.

Findings:
- HistorianAccess.CreateHistorianDataValueList(NonStreamedOriginal) succeeds
- HistorianDataValueList.NonStreamedValuesBegin() succeeds (batchID 0->1)
- HistorianDataValueList.AddNonStreamedValue(value, validate=true, out err)
  FAILS with ErrorCode=TagNotFoundInCache (129) — same client-side
  validation gate that blocks AddS2
- AddNonStreamedValuesEnd() returns void; SendValues() returns true
  with Success because the list is empty (no value was ever added)
- No AddNonStreamValues* WCF calls reach the wire

Conclusion: the revision-write path requires the tag to be in the
library's runtime tag cache, which is only populated by configured
IO server / Application Server pipelines, not by HistorianAccess.AddTag.
This matches the architectural blocker documented for AddS2 and means
no public WriteRevisionsAsync / BeginRevisionAsync should be added to
the SDK — the path is unreachable for client-created sandbox tags.

The Wcf/Contracts/ITransactionServiceContract methods (AddNonStream-
ValuesBegin/AddNonStreamValues/AddNonStreamValuesEnd) remain declared
for completeness; no orchestrator or public surface is added.

The harness extension is preserved as a deterministic reproducer for
the blocker: re-run --write-revision-values to verify the gate any
time. docs/plans/revision-write-path.md updated with the empirical
finding plus the original plan retained as historical context.

177/177 tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 01:45:48 -04:00
Joseph Doherty f4709ff143 Speculative-items sweep: IntegralDivisor, cert tests, D3/D1/D2 findings
Plan: docs/plans/speculative-items-sweep.md (also covers parallelism +
findings).

Implemented:
- C3: HistorianTagDefinition.IntegralDivisor (default 1.0). Wire bytes
  flip per the captured native serializer; live probe shows the server
  stores IntegralDivisor on EngineeringUnit (shared) rather than per-tag,
  so the value is accepted on the wire but doesn't visibly persist for
  the test EU. Documented in the property's doc-comment.
- E: HistorianWcfCertOptionTests (5 tests) covering AllowUntrustedServer-
  Certificate validator installation + ServerDnsIdentity propagation
  through CreateEndpointAddress and CreateBindingPair.

Investigated + documented (deferred):
- D3: Discrete/String/Int1/Int8/UInt8 EnsT2 root cause — server-side
  ValidationFailed: "Transaction validation failed". Native AddTag's
  validator rejects non-analog types; not a wire-format issue. To unlock,
  need to capture a working native flow via a different code path
  (likely SMC's tag-import path or AddTagExtendedProperties carrying
  type-specific metadata). Defer until a customer asks.
- D1: AddTagExtendedProperties feasibility — managed surface confirmed
  (ArchestrA.HistorianAccess.AddTagExtendedProperties + WCF op
  AddTagExtendedPropertyGroups). Cost estimated at 1-2 days of focused
  RE work due to CTagExtendedPropertyGroup payload complexity. Defer.
- D2: AddRevisionValuesBegin/Value/End — sub-plan written at
  docs/plans/revision-write-path.md with 5-step capture sequence,
  workstream estimates, and risk register. Implementation deferred.

177/177 tests pass (was 172; +5 cert tests + 1 IntegralDivisor unit
test, harness probe results not committed).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 00:11:40 -04:00
Joseph Doherty 549995e4a9 CLAUDE.md: cross-platform cert-binding verified end-to-end
With AllowUntrustedServerCertificate=true + ServerDnsIdentity="localhost",
all four representative read calls (ReadRawAsync, GetSystemParameterAsync,
BrowseTagNamesAsync, GetTagMetadataAsync) succeed from a Debian 13 client
against the Windows Historian over RemoteTcpCertificate with explicit
Windows credentials and NegotiateAuthentication via GSSAPI/NTLM.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 23:09:21 -04:00
Joseph Doherty 7502575204 Add HistorianClientOptions.ServerDnsIdentity for cert-binding overrides
When the server cert's CN/SAN doesn't match the URL host (typical for
installer-generated AVEVA Historian certs that claim DNS=localhost
even when reached over a LAN IP), WCF rejects the channel with
"Identity check failed for outgoing message". Set ServerDnsIdentity
to whatever the cert claims (often "localhost") to satisfy the check.
The endpoint address for the cert binding is constructed with a
DnsEndpointIdentity when the option is non-null.

Default null. Pairs with AllowUntrustedServerCertificate so a Linux
client can talk to a self-signed dev Historian over RemoteTcpCertificate.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 23:08:33 -04:00
Joseph Doherty d3e5bf09b6 Add HistorianClientOptions.AllowUntrustedServerCertificate
When true, the SDK's WCF channel factories accept the server's X.509
certificate without chain validation. Intended for connecting to
development / on-prem Historians whose /HistCert endpoint presents an
installer-generated self-signed cert that isn't in the local trust
store. Particularly relevant on Linux: .NET WCF on Linux does its own
X509Chain validation that doesn't honor the system CA bundle, so even
after `update-ca-certificates` succeeds the cert binding still rejects
the server. With this option set, custom certificate validator accepts
any cert and revocation checking is disabled.

Default false. Centralized in HistorianWcfClientCredentialsHelper.Configure
and applied at every ChannelFactory<T> instantiation in the WCF layer
(no-op when the option is false). 171/171 Windows tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 23:05:32 -04:00
Joseph Doherty 92d4110142 CLAUDE.md: cross-platform end-to-end verified
Verified from a Debian 13 client (.NET 10.0.203) against the Windows
Historian using explicit Windows credentials and NegotiateAuthentication
via GSSAPI/NTLM:

- GetTagMetadataAsync: returns correct fields for SysTimeSec
- BrowseTagNamesAsync: returns SysTimeHour, SysTimeMin, SysTimeSec
- ProbeAsync: works over both transports

Calls that touch the cert-transport binding directly (ReadRawAsync,
GetSystemParameterAsync) still fail at X509 chain validation despite
update-ca-certificates. .NET WCF on Linux uses its own X509Chain plumbing
rather than the system CA bundle. Documented as a follow-up rather than
fixed in this pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 22:53:10 -04:00
Joseph Doherty 8607f5d530 CLAUDE.md: document cross-platform status
Verified 2026-05-04:
- SDK builds on Debian 13 with .NET 10 SDK 10.0.203
- ProbeAsync over RemoteTcpCertificate works from Linux
- RemoteTcpIntegrated fails on Linux due to a WCF-level limitation
  (NetTcpBinding + Windows TcpClientCredentialType is BCL-Windows-only),
  not a HistorianSspiClient issue
- Authenticated reads over the cert path with explicit creds are wired
  but await live verification

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 22:29:45 -04:00
Joseph Doherty b8280a1465 Drop SupportedOSPlatform gates; SDK now runs on Linux
The dialect / orchestrators were defensively gated on Windows because
HistorianSspiClient previously P/Invoked InitializeSecurityContextW. With
that replaced by NegotiateAuthentication (cross-platform), the gates are
unnecessary. Removed them from:

- Historian2020ProtocolDialect (4 read paths + 3 status helpers)
- HistorianClient.EnsureTagAsync / DeleteTagAsync
- HistorianWcf{Auth,Read,Event,Status,TagWrite}Orchestrator/Helper
- HistorianWcf{HistAddressing,MessageCapture}Behavior
- HistorianWcfBindingFactory (with #pragma on the Named-Pipe builder
  which still requires Windows at the BCL level)

Runtime constraint: LocalPipe and RemoteTcpIntegrated transports still
require Windows because NetNamedPipeBinding and the Windows transport
security binding are Windows-only at the BCL level. RemoteTcpCertificate
is now usable from Linux, and ProbeAsync is verified working from a
Debian client (10.100.0.35) against the Windows Historian (10.100.0.48).

171/171 tests still pass on Windows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 22:27:57 -04:00
Joseph Doherty 7e4d713eb3 Cross-platform NegotiateAuthentication; StorageType field; docs polish
HistorianSspiClient rewritten on top of System.Net.Security.NegotiateAuthentication
in place of P/Invoke into secur32.dll's InitializeSecurityContextW. The class
keeps the same Next() / Dispose() / two-constructor surface so callers don't
change. RequiredProtectionLevel=EncryptAndSign + AllowedImpersonationLevel=
Identification produces a request-flag set equivalent to the captured native
0x2081C / 0x81C bitmasks (still preserved as constants for the existing unit
tests). Removes the only Windows P/Invoke in the production SDK; the
[SupportedOSPlatform("windows")] gating elsewhere stays in place pending a
separate sweep.

HistorianStorageType (Cyclic = 1, Delta = 2):
Captured 2026-05-04 via --write-storage-type on the harness. Delta differs
from Cyclic in three places — header byte 10 (0x02 -> 0x06), flag-block
byte 1 (0x01 -> 0x02), and 4 zero bytes inserted after StorageRate before
the FILETIME. Server persists Tag.StorageType=1/2 accordingly. Plumbed
through HistorianTagDefinition.StorageType + serializer + orchestrator + 2
new tests (golden bytes + live SQL persistence verification).

Docs polish:
CLAUDE.md no longer claims "no P/Invoke" (HistorianSspiClient is the one
allowed P/Invoke surface); updated test count to 169+; AGENTS.md Required
SDK Surface and Repository Layout brought up to date with the live state
including the write surface; handoff.md "not a git working tree" obsolete
note removed.

171/171 tests pass with the NegotiateAuthentication replacement (was 169;
+2 new tests for StorageType).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 22:19:37 -04:00
Joseph Doherty 5ce62a5900 Wire ApplyScaling, StorageRate; close out write-commands plan
ApplyScaling (HistorianTagDefinition.ApplyScaling):
The EnsT2 trailer's second byte controls server-side scaling — `FE 00`
mirrors MinRaw to MinEU and sets AnalogTag.Scaling=0; `FE 01` persists
distinct MinRaw/MaxRaw and sets Scaling=1. Decoded by toggling
set_ApplyScaling on the native harness and capturing the wire bytes for
both values with identical inputs. The earlier docs claimed
EnsureTagAsync needed a follow-up "UpdateTags" call; the WCF surface has
no such operation — toggling that one byte is the whole fix.

StorageRate (HistorianTagDefinition.StorageRateMs):
Serializer accepts a non-default rate, validated empirically against
the live server which only accepts quantized values
(1000/5000/10000/60000/300000 ms).

EnsureTagAsync upsert semantics:
Second call on the same tag name with different fields succeeds and
updates Description, MinEU, MaxEU, MinRaw, MaxRaw, Scaling in place
(verified by direct SQL inspection in a live test).

Plan + doc closeout:
write-commands-reverse-engineering.md rewritten as a current-state
plan with three workstreams (A doc closeout / B idempotency / C1
StorageRate) and a parallelism table; prior phase notes preserved as
appendix. handoff.md, implementation-status.md, wcf-contract-evidence.md,
README.md updated to remove "writes are out of scope" / non-existent
UpdateTags references and document the actual EnsT2 wire format
including the `FE xx` trailer.

Reverse-engineering harness gains --write-apply-scaling and a SQL
post-check that prints the persisted AnalogTag bounds so future RE
sessions can verify wire→DB causality without leaving the harness.

169/169 tests pass (was 165; +4 new tests covering ApplyScaling,
StorageRate golden bytes, StorageRate live persistence, and
EnsureTagAsync upsert semantics).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 22:04:27 -04:00
Joseph Doherty a175c6e5a0 CLAUDE.md: mark RemoteTcp transports as live-verified
Both RemoteTcpIntegrated (full read surface + status helpers, 9 tests) and
RemoteTcpCertificate (Probe only) now pass against the host's own LAN IP,
exercising the MdasNetTcpWindows / MdasNetTcpCertificate binding branches
and SSPI/TLS handshake against a hostname rather than the loopback fast
path. True off-box verification still blocked on Windows-only
InitializeSecurityContextW in HistorianSspiClient.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 18:38:16 -04:00
Joseph Doherty b05063b195 Document the trailing 35 bytes of GetNextQueryResultBuffer rows
Investigation finding only — no behavior change, no new public fields.

Captured a fresh GetNextQueryResultBufferResponse for SysTimeSec via
instrument-wcf-readmessage and compared against the canonical 4-row
OtOpcUaParityTest_001.Counter fixture. Trailing-block structure is
tag-independent:

  bytes 0-2   constant 0x00 0x00 0x01 (sample-format marker)
  bytes 3-10  Int64 FILETIME UTC (duplicate of startTime for raw rows;
              already used by the aggregate parser as the interval start)
  bytes 11-18 zeros (likely end-time slot — populated by aggregate variants)
  bytes 19-26 varies row-to-row even with identical Quality/Value;
              looks like a storage block sequence ID or snapshot offset
  bytes 27,29 flag bytes (0/1 and 0/4 observed); semantics undecoded
  bytes 28, 30-34 zeros

None of bytes 19-34 have a clear user-facing meaning; they appear to be
server-internal storage metadata. Updated the
TryParseGetNextQueryResultBufferRows remarks block with the byte map and
a note that surfacing them as new HistorianSample fields should wait
until a customer actually asks. CLAUDE.md "Remaining gaps" entry updated
to reflect the new partial decode.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 15:35:48 -04:00
Joseph Doherty 7288f39f5d Add Set-HistorianCredentials.ps1 for DPAPI-encrypted credential persistence
Drops a small helper for the gated live-integration tests that need
HISTORIAN_USER + HISTORIAN_PASSWORD set in the process environment
(currently GetTagMetadataAsync_ExplicitCredentials_AgainstLocalHistorian).

Three modes:
  - Default: prompts for username (default <COMPUTERNAME>\<USERNAME>) and
    password (silent), saves to %USERPROFILE%\.histsdk\credentials.xml via
    Export-Clixml. The SecureString inside the PSCredential is DPAPI-encrypted
    and decryptable only by the same Windows user account on the same machine.
  - -Load: reads the saved credential and exports HISTORIAN_USER +
    HISTORIAN_PASSWORD into the current PowerShell session's environment.
  - -Clear: deletes the saved credential file.

Also accepts -Path to override the storage location (e.g. for keeping
multiple credential sets side by side) and -UserName to skip the username
prompt for password-only re-saves.

Stored under the user profile, never inside the repo, so it cannot be
committed accidentally. The file format is plain Export-Clixml — no custom
encoding shenanigans.

Live-verified locally: -Load + dotnet test passes the previously-skipped
GetTagMetadataAsync_ExplicitCredentials_AgainstLocalHistorian test against
the local Historian with IntegratedSecurity=false.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 15:22:34 -04:00
Joseph Doherty f32fd57874 Remove dead dialect methods; unblock explicit-creds tag-metadata path
Two cleanups from the post-EnsureTagAsync punch list — both isolated, no
protocol discovery required.

#89 dead code in Historian2020ProtocolDialect:
  - BrowseTagNamesAsync and GetTagMetadataAsync on the dialect both threw
    ProtocolEvidenceMissingException, but HistorianClient routes those calls
    directly to HistorianWcfTagClient — the dialect overrides were never
    reached. Removed both methods. ReadBlocksAsync stays (it's a deliberate
    guardrailed entry on the public surface).

#90 explicit-creds tag-metadata path:
  - HistorianWcfTagClient.WcfRetrievalSession.ValidateSupportedAuth threw
    ProtocolEvidenceMissingException whenever IntegratedSecurity=false AND
    UserName/Password were supplied. But the surrounding code already wires
    those creds through ApplyWindowsCredential ->
    factory.Credentials.Windows.ClientCredential — the validator was just
    being conservative about an untested combination.
  - Inverted the check: now only rejects the no-auth-at-all combination
    (IntegratedSecurity=false + no UserName + no Password). The other three
    valid auth shapes pass through to WCF.

Tests: 161 -> 163 (+2). New unit test verifies the no-auth case still
throws; new gated live integration test
GetTagMetadataAsync_ExplicitCredentials_AgainstLocalHistorian exercises the
explicit-creds path when HISTORIAN_USER+HISTORIAN_PASSWORD are set, skips
cleanly otherwise.

CLAUDE.md updated: removed the two now-resolved entries from "Remaining
gaps"; explicit-creds line refined to note the live-verification env-var
requirement.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 15:04:51 -04:00
182 changed files with 25931 additions and 1485 deletions
+6
View File
@@ -29,3 +29,9 @@ Thumbs.db
# Test droppings # Test droppings
*.coverage *.coverage
coverage.cobertura.xml coverage.cobertura.xml
# Live 2023 R2 server credentials — never commit
wonder-sql-vd03.txt
# Reverse-engineering IL-rewrite output: derived AVEVA binaries, never commit
docs/reverse-engineering/dnlib-write-copy/
+73 -57
View File
@@ -27,12 +27,21 @@ a P/Invoke shim as the primary solution; it is useful only as an analysis aid.
## Repository Layout ## Repository Layout
This workspace is an SDK investigation folder, not a full application repo. This workspace is a full Git repo (origin: gitea.dohertylan.com) with the
shipping SDK under `src/`, tests under `tests/`, RE tooling under `tools/`,
and decoded protocol notes under `docs/`. See `CLAUDE.md` for the
authoritative architecture overview.
- `instructions.md` - source planning document and decision record. - `instructions.md` - source planning document and decision record.
- `src\AVEVA.Historian.Client\` - the production managed SDK (pure .NET 10,
no native AVEVA references).
- `tests\AVEVA.Historian.Client.Tests\` - unit + gated live integration tests.
- `tools\` - reverse-engineering tooling (CLI, native trace harness,
WCF capture server, IL-rewrite instrumentation helper).
- `docs\reverse-engineering\` - sanitized RE evidence and decoded notes.
- `current\` - the seven DLLs the existing sidecar links against today. - `current\` - the seven DLLs the existing sidecar links against today.
- `aveva-install-x64\` - full 64-bit AVEVA Historian client-side DLL set. - `aveva-install-x64\` and `aveva-install-x86\` - full AVEVA Historian
- `aveva-install-x86\` - full 32-bit AVEVA Historian client-side DLL set. client-side DLL sets for cross-version reference.
Use `current\` first because it represents the deployed sidecar dependency set. Use `current\` first because it represents the deployed sidecar dependency set.
Use `aveva-install-*` to compare architecture-specific behavior and locate Use `aveva-install-*` to compare architecture-specific behavior and locate
@@ -40,22 +49,44 @@ adjacent client APIs.
## Required SDK Surface ## Required SDK Surface
Keep the managed SDK narrowly scoped to the operations used in production: The shipping public surface (all live-verified against `localhost`
see `CLAUDE.md` "Required SDK Surface" for the authoritative list and
caveats):
- `ReadRawAsync(tag, startUtc, endUtc, maxValues)` Reads:
- `ReadAggregateAsync(tag, startUtc, endUtc, mode, interval)`
- `ReadAtTimeAsync(tag, timestampsUtc)`
- `ReadEventsAsync(startUtc, endUtc)`
- `ProbeAsync()`
The existing alarm-event write path is dormant. Do not implement write-back - `ProbeAsync`
unless a new requirement is supplied. - `ReadRawAsync`
- `ReadAggregateAsync`
- `ReadAtTimeAsync`
- `ReadEventsAsync`
- `BrowseTagNamesAsync`
- `GetTagMetadataAsync`
- Status helpers: `GetConnectionStatusAsync`, `GetStoreForwardStatusAsync`,
`GetSystemParameterAsync`
Writes (added 2026-05-04 by explicit request):
- `EnsureTagAsync` for analog Float / Double / Int2 / Int4 / UInt4
(with optional `ApplyScaling=true` for distinct MinRaw/MaxRaw and
optional `StorageRateMs` for non-default sampling).
- `DeleteTagAsync`.
`AddS2` (write samples) is architecturally blocked — the server's
runtime cache only ingests from configured IOServer / Application Server
pipelines. Do not extend write support without an explicit new request.
## Reverse-Engineering Workflow ## Reverse-Engineering Workflow
The bulk of the original RE workflow has been executed and is now backed
by `docs/reverse-engineering/` evidence. The notes below are the durable
process in case new captures are needed (e.g., for a new Historian version
or a new write op).
### 1. Managed Wrapper Analysis ### 1. Managed Wrapper Analysis
Use dnSpy or ILSpy on `current\aahClientManaged.dll`. Use dnSpy / ILSpy / the in-repo `dnlib-method` CLI on
`current\aahClientManaged.dll`.
Document: Document:
@@ -66,8 +97,8 @@ Document:
- Returned sample/event models, quality fields, timestamp handling, and error - Returned sample/event models, quality fields, timestamp handling, and error
propagation. propagation.
Prefer producing small Markdown notes under a future `docs\reverse-engineering\` Sanitized notes go under `docs\reverse-engineering\` (the folder exists and
folder rather than relying on memory. is the canonical home for committed RE evidence).
### 2. Native ABI Mapping ### 2. Native ABI Mapping
@@ -137,34 +168,12 @@ newer Historian versions.
### 5. Managed Implementation Shape ### 5. Managed Implementation Shape
When implementation starts, use this project shape unless the real repo dictates The implementation has landed and is the authoritative reference. See
otherwise: `CLAUDE.md` "Code Architecture" for the actual layout. The original
abstract shape is preserved as historical context only.
```text Key design rule still in force: keep protocol parsing isolated from transport
src/AVEVA.Historian.Client/ I/O so captured frames can be tested without a live Historian.
AVEVA.Historian.Client.csproj
HistorianClient.cs
HistorianClientOptions.cs
Models/
HistorianSample.cs
HistorianAggregateSample.cs
HistorianEvent.cs
RetrievalMode.cs
Protocol/
HistorianConnection.cs
HistorianFrame.cs
HistorianMessageType.cs
HistorianProtocolReader.cs
HistorianProtocolWriter.cs
Transport/
TcpHistorianTransport.cs
ClusterEndpointPicker.cs
Internal/
BackoffPolicy.cs
```
Keep protocol parsing isolated from transport I/O so captured frames can be
tested without a live Historian.
## Testing Expectations ## Testing Expectations
@@ -188,27 +197,34 @@ Integration tests must skip cleanly when these values are not configured.
## Constraints ## Constraints
- Keep the final SDK pure managed .NET 10. - Keep the final SDK managed .NET 10. The single P/Invoke surface allowed
- Avoid adding native runtime dependencies to the production SDK. in production is `HistorianSspiClient` (Windows SSPI for integrated
- Avoid broad API design. Implement only the operations listed above. auth); do not add unrelated P/Invokes.
- Treat AVEVA protocol details as version-sensitive; document assumptions. - Avoid adding native runtime dependencies to the production SDK. No
reference to `aahClientManaged.dll` / `aahClient.dll` from `src/`.
- Avoid broad API design. Implement only the operations listed in
"Required SDK Surface".
- Treat AVEVA protocol details as version-sensitive; document assumptions
in `docs/reverse-engineering/`.
- Do not redistribute AVEVA binaries. - Do not redistribute AVEVA binaries.
- Do not commit credentials, proprietary captures, or customer data. - Do not commit credentials, proprietary captures, or customer data.
- Do not delete or overwrite DLLs in `current\` or `aveva-install-*`. - Do not delete or overwrite DLLs in `current\` or `aveva-install-*`.
## Definition of Done ## Definition of Done
For the reverse-engineering phase: Both the RE phase and the SDK phase are **met** as of 2026-05-04:
- Managed wrapper public surface and native entry points are documented. - Managed wrapper public surface and native entry points are documented in
- Required query flows have sanitized captures or byte-level notes. `docs/reverse-engineering/`.
- Message framing, request fields, response fields, and error frames are - Required query flows have sanitized captures + byte-level notes; golden
described well enough to implement parser tests. fixtures live under `fixtures/protocol/`.
- Message framing, request/response/error layouts are decoded sufficiently
for round-trip parser tests.
- The shipping SDK implements the Required SDK Surface (reads + writes).
- 169 unit + live integration tests pass.
- Local consumers can replace the sidecar without `aahClientManaged.dll` or
`aahClient.dll` at runtime.
For the SDK phase: Future RE work (e.g., new Historian version, additional write ops) should
follow the same workflow above; new evidence updates `docs/reverse-engineering/`
- The managed client implements the required read-only surface. and the relevant plan file under `docs/plans/`.
- Unit tests cover protocol parse/build behavior.
- Integration tests can validate against a configured live Historian.
- The SDK can replace the existing sidecar call sites without requiring
`aahClientManaged.dll` or `aahClient.dll` at runtime.
+44 -11
View File
@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
## Mission ## Mission
Build a fully managed .NET 10 replacement for AVEVA Historian's `aahClientManaged` / `aahClient.dll` stack by reverse-engineering the proprietary binary protocol. The production SDK under `src/AVEVA.Historian.Client/` must remain pure managed .NET 10 — no P/Invoke, no native AVEVA runtime dependency, no REST. Tools under `tools/` and scripts under `scripts/` are reverse-engineering aids only. Build a fully managed .NET 10 replacement for AVEVA Historian's `aahClientManaged` / `aahClient.dll` stack by reverse-engineering the proprietary binary protocol. The production SDK under `src/AVEVA.Historian.Client/` has no native AVEVA runtime dependency and no REST surface. The one P/Invoke is into Windows SSPI (`HistorianSspiClient``InitializeSecurityContextW`) for integrated-auth NTLM/Negotiate token generation; this gates the SDK to Windows-only execution today. See the `RemoteTcpCertificate` transport for a Windows-free auth path. Tools under `tools/` and scripts under `scripts/` are reverse-engineering aids only.
Read `AGENTS.md` (standing constraints), `instructions.md` (decision record), and `docs/reverse-engineering/handoff.md` (current evidence + active blocker) before starting non-trivial work. The handoff doc is the entry point — it tracks the live blocker, next pickup steps, and the canonical list of primary reference docs. Read `AGENTS.md` (standing constraints), `instructions.md` (decision record), and `docs/reverse-engineering/handoff.md` (current evidence + active blocker) before starting non-trivial work. The handoff doc is the entry point — it tracks the live blocker, next pickup steps, and the canonical list of primary reference docs.
@@ -18,10 +18,12 @@ Reads (the original required surface, all working live as of 2026-05-04):
Writes (added 2026-05-04 by explicit user request — do not extend further without one): Writes (added 2026-05-04 by explicit user request — do not extend further without one):
- `EnsureTagAsync` for analog types: Float, Double, Int2, Int4, UInt4 (live-verified end-to-end). Other types (SingleByteString/DoubleByteString/Int1/Int8/UInt8) fail at native AddTag — likely require a different path and are intentionally not supported. `MinEU`/`MaxEU` round-trip correctly into the DB; `MinRaw`/`MaxRaw` are sent on the wire but the server mirrors them to MinEU/MaxEU when ApplyScaling=false (verified against native — server quirk, not SDK bug). - `EnsureTagAsync` for analog types: Float, Double, Int2, Int4, UInt4, **Int8, UInt8** (all live-verified end-to-end; `Int8`/`UInt8` added 2026-06-25 — same analog `CTagMetadata` layout, type codes `0x19`/`0x39`). **`UInt1` is NOT supported**: the server accepts `EnsureTags(UInt1)` but stores a *degenerate* analog tag (`GetTagInfosFromName` returns a 31-byte stub — descriptor type byte `0x00`, no GUID), so the write fails on the `GetTagInfo` path; re-gated fail-closed. SingleByteString/DoubleByteString and special/event types require a different (non-analog) path and are intentionally not supported. `MinEU`/`MaxEU`/`MinRaw`/`MaxRaw` all round-trip into the DB. By default `ApplyScaling=false` and the server mirrors MinRaw→MinEU and sets `AnalogTag.Scaling=0`; set `ApplyScaling=true` on the definition to persist distinct raw bounds with `AnalogTag.Scaling=1`. The wire encoding is the trailer's second byte (`FE 00` vs `FE 01`).
- `DeleteTagAsync` - `DeleteTagAsync`
- `AddHistoricalValuesAsync` (added 2026-06-21 by explicit user request — M3 historical/backfill writes). **gRPC-only** (`HistorianTransport.RemoteGrpc`); non-gRPC transports throw `ProtocolEvidenceMissingException`. Reverse-engineered by capturing the native 2023 R2 client: the historical write rides `HistoryService.AddStreamValues` with an "ON" storage-sample buffer (`HistorianHistoricalWriteProtocol`, golden-tested), NOT the TransactionService `AddNonStreamValues` path the static decompile suggested. Orchestrator (`HistorianGrpcHistoricalWriteOrchestrator`): write-enabled session → `GetTagInfosFromName` (resolves the per-tag GUID = the tag-info `TypeId`, and maps the data type via `MapDataType`) → `AddStreamValues`. Tag must pre-exist (`EnsureTagAsync`). Supports the analog types `EnsureTagAsync` does — **Float, Double, Int2, Int4, UInt4, Int8, UInt8** (all captured live + golden-tested + write/read-back validated; `Int8`/`UInt8` added 2026-06-25, value = native-width LE int64/uint64). The 4-byte value descriptor is constant (`C0 10 01 00`); the value is `u32(0) + native-width value` (float32 / double64 / int16 / int32 / uint32 / int64 / uint64) selected by the tag's declared type. Other tag types throw `ProtocolEvidenceMissingException` (incl. `UInt1` — server-degenerate, see `EnsureTagAsync` above). Live-validated end-to-end against the 2023 R2 server. The D2/`AddS2` cache gate (err 129) does NOT block the primed 2023 R2 client. See `docs/plans/revision-write-path.md` §"R3.1 CAPTURED".
- `SendEventAsync` (M2 event-send; added by explicit user request). Appends a single `HistorianEvent` to the built-in `CM_EVENT` tag, readable back via `ReadEventsAsync` / `v_AlarmEventHistory2`. Works on **both transports**, routed by `HistorianClientOptions.Transport`: WCF runs Open2 event-mode (`0x501`) → CM_EVENT registration (RTag2 + EnsT2) → `AddS2` (`AddStreamValues2`); gRPC (`RemoteGrpc`, `HistorianGrpcEventWriteOrchestrator`, added 2026-06-23) runs the v8 Event `OpenConnection` (ExchangeKey ECDH) → CM_EVENT registration → `HistoryService.AddStreamValues`. **Both carry the same `"OS"` (0x534F) event VTQ buffer** (`HistorianEventWriteProtocol`, the managed `PackToVtq` equivalent) — there is NO distinct event-send RPC and it is NOT the historical write's `"ON"` buffer (captured live from the native 2023 R2 client; the write-enabled Event open is byte-identical to the read-only one). The gRPC path is **live-validated end-to-end** (send → `BSuccess` → event reads back from the server). Only **original events** (`RevisionVersion = 0`) with **string-valued properties** have a captured encoding; revision/update/delete events and non-string property values throw `ProtocolEvidenceMissingException`. Registration buffers are golden-tested against the live capture (`GrpcEventSendProtocolTests`); gated live test `SendEventAsync_OverGrpc_AcceptsEvent` (opt-in `HISTORIAN_GRPC_EVENT_SEND=1`).
`AddS2` (write samples) is architecturally blocked — server cache only ingests from configured IOServers/ApplicationServer pipelines. Do not add write-samples support. `AddS2` (streaming process-sample writes for user tags) remains architecturally blocked — the server cache only ingests from configured IOServers/ApplicationServer pipelines. Do not add streaming write-samples support. (`AddHistoricalValuesAsync` is the distinct *non-streamed original/backfill* path and is supported.)
Methods without protocol evidence currently throw `ProtocolEvidenceMissingException` from `Historian2020ProtocolDialect`. Do not stub fake behavior — leave them throwing until evidence supports an implementation. Methods without protocol evidence currently throw `ProtocolEvidenceMissingException` from `Historian2020ProtocolDialect`. Do not stub fake behavior — leave them throwing until evidence supports an implementation.
@@ -71,6 +73,7 @@ Three layered subsystems, intentionally decoupled so protocol parsing can be uni
- **`Wcf/`** — managed WCF/MDAS layer. The Historian uses Net.TCP on port `32568` with a custom `application/x-mdas` content type wrapping a binary SOAP 1.2 / WS-Addressing 1.0 envelope. `MdasMessageEncoder` + `MdasMessageEncodingBindingElement` implement that wrapper. `HistorianWcfBindingFactory` produces three flavors: plain MDAS, MDAS+Windows transport (used for `/Hist-Integrated`), and MDAS+certificate (used for `/HistCert`). Service paths live in `HistorianWcfServiceNames`. WCF data contracts (`Wcf/Contracts/`) are reproduced from server-side static analysis and are versioned per native interface (e.g., `IRetrievalServiceContract2..4`). - **`Wcf/`** — managed WCF/MDAS layer. The Historian uses Net.TCP on port `32568` with a custom `application/x-mdas` content type wrapping a binary SOAP 1.2 / WS-Addressing 1.0 envelope. `MdasMessageEncoder` + `MdasMessageEncodingBindingElement` implement that wrapper. `HistorianWcfBindingFactory` produces three flavors: plain MDAS, MDAS+Windows transport (used for `/Hist-Integrated`), and MDAS+certificate (used for `/HistCert`). Service paths live in `HistorianWcfServiceNames`. WCF data contracts (`Wcf/Contracts/`) are reproduced from server-side static analysis and are versioned per native interface (e.g., `IRetrievalServiceContract2..4`).
- **`Protocol/`** — binary frame layer (`HistorianFrameReader`/`Writer`, `HistorianBinaryPrimitives`, `HistorianMessageType`). `Historian2020ProtocolDialect` is the version-anchored bridge between `HistorianClient` and the frame layer; methods without sufficient evidence throw `ProtocolEvidenceMissingException` rather than guessing wire bytes. - **`Protocol/`** — binary frame layer (`HistorianFrameReader`/`Writer`, `HistorianBinaryPrimitives`, `HistorianMessageType`). `Historian2020ProtocolDialect` is the version-anchored bridge between `HistorianClient` and the frame layer; methods without sufficient evidence throw `ProtocolEvidenceMissingException` rather than guessing wire bytes.
- **`Transport/`** — pluggable `IHistorianTransport` (default: TCP). Tests inject a fake transport. - **`Transport/`** — pluggable `IHistorianTransport` (default: TCP). Tests inject a fake transport.
- **`Grpc/`** — 2023 R2 gRPC transport (`HistorianTransport.RemoteGrpc`). The recovered protobuf contract lives in `Grpc/Protos/*.proto` and is compiled to client stubs at build time by `Grpc.Tools`. `HistorianGrpcChannelFactory` builds a gRPC-Web/HTTP-1.1 channel (default port `32565`, optional TLS, gzip) matching the stock 2023 R2 client. `HistorianGrpcReadOrchestrator` mirrors `HistorianWcfReadOrchestrator` but over gRPC: it reuses the exact native serializers/parsers — the same Open2 buffer, SSPI/NTLM tokens, and `DataQueryRequest`/result buffers travel inside protobuf `bytes` fields. The 2020→gRPC op map: `Hist.ValCl``StorageService.ValidateClientCredential` (the SSPI/Negotiate token loop), `Hist.Open2``HistoryService.OpenConnection`, `Retr.StartQuery2``RetrievalService.StartQuery`, `Retr.GetNextQueryResultBuffer2``RetrievalService.GetNextQueryResultBuffer`. The transport-agnostic handshake (Open2 request builder + SSPI token loop + response decode) is shared via `Wcf/HistorianNativeHandshake`. **Live-verified 2026-06-21 against a real 2023 R2 server** (interface versions History=12, Retrieval=4, Storage=4): the full read chain returns rows. NOTE: `HistoryService.ExchangeKey` is a SEPARATE key-exchange/cert-path op, NOT the Negotiate loop — an earlier revision wrongly routed the token loop there and it was rejected at round 0 regardless of credentials; the loop belongs on `StorageService.ValidateClientCredential` (which kept the 2020 inBuff/outBuff token framing). The byte payloads are the proven 2020 protocol and transfer unchanged; only the History interface integer differs (12 vs 11) and is buffer-compatible. The version gate now accepts BOTH 11 and 12 for History (`HistorianServerVersionGate.AcceptedVersions`), so a v12 server passes with the default `VerifyServerInterfaceVersion=true` — no opt-out needed (the earlier requirement to set it false is obsolete). Gated live test: set `HISTORIAN_GRPC_HOST` (+ `HISTORIAN_TEST_TAG`, optional `HISTORIAN_GRPC_PORT`/`HISTORIAN_GRPC_TLS`/`HISTORIAN_GRPC_DNSID`); reach the live 2023 R2 box via [[reference_2023r2_live_server_access]].
- **`Models/`** — public DTOs and enums (`HistorianSample`, `RetrievalMode`, etc.). `HistorianDataValue` represents the discriminated value type. - **`Models/`** — public DTOs and enums (`HistorianSample`, `RetrievalMode`, etc.). `HistorianDataValue` represents the discriminated value type.
`InternalsVisibleTo` exposes internals to the test assembly and the reverse-engineering tool. `InternalsVisibleTo` exposes internals to the test assembly and the reverse-engineering tool.
@@ -83,21 +86,51 @@ The original blocker — `Open2` reaching server logic but `Retr.StartQuery2` re
2. Native SSPI request flags — round 0 = `0x2081C` (adds `IDENTIFY` + `REPLAY_DETECT` + `SEQUENCE_DETECT`); rounds 1+ = `0x81C`. Without `REPLAY_DETECT|SEQUENCE_DETECT`, NTLM MIC generation is skipped and `AcceptSecurityContext` rejects round 1. Implemented in `HistorianSspiClient` via P/Invoke `InitializeSecurityContextW`. 2. Native SSPI request flags — round 0 = `0x2081C` (adds `IDENTIFY` + `REPLAY_DETECT` + `SEQUENCE_DETECT`); rounds 1+ = `0x81C`. Without `REPLAY_DETECT|SEQUENCE_DETECT`, NTLM MIC generation is skipped and `AcceptSecurityContext` rejects round 1. Implemented in `HistorianSspiClient` via P/Invoke `InitializeSecurityContextW`.
3. Cross-service version probes (`Trx/GetV`, `Stat/GetV`, `Retr/GetV`) between RTag2 and EnsT2 in the event flow — required to register the client with each service's session table. 3. Cross-service version probes (`Trx/GetV`, `Stat/GetV`, `Retr/GetV`) between RTag2 and EnsT2 in the event flow — required to register the client with each service's session table.
End-to-end chain working from a pure managed .NET 10 client: `Hist.GetV → Hist.ValCl × N → Hist.Open2 → /Retr.GetV → Retr.IsOriginalAllowed → Retr.StartQuery2 → loop Retr.GetNextQueryResultBuffer2`. 23 live integration tests against `localhost` cover all required reads + the two write ops. End-to-end chain working from a pure managed .NET 10 client: `Hist.GetV → Hist.ValCl × N → Hist.Open2 → /Retr.GetV → Retr.IsOriginalAllowed → Retr.StartQuery2 → loop Retr.GetNextQueryResultBuffer2`. 169 unit + live integration tests against `localhost` cover all required reads, the two write ops, and the `RemoteTcpIntegrated` / `RemoteTcpCertificate` transports.
### Write-path notes (added 2026-05-04) ### Write-path notes (added 2026-05-04)
`EnsureTagAsync` and `DeleteTagAsync` chain follow the same pattern as reads but require Open2 with `NativeIntegratedWriteEnabledConnectionMode = 0x401` (Process | Write | IntegratedSecurity) — the read-path's `0x402` (read-only) makes the server return err 132 `OperationNotEnabled` silently. The analog Float `CTagMetadata` payload is 144 bytes with a leading `0x4E` marker byte and a 2-byte trailer `FE 00`. See `docs/reverse-engineering/handoff.md` and the `WriteDiag` env-gated diagnostic helper in `HistorianWcfTagWriteOrchestrator` for capture details. `EnsureTagAsync` and `DeleteTagAsync` chain follow the same pattern as reads but require Open2 with `NativeIntegratedWriteEnabledConnectionMode = 0x401` (Process | Write | IntegratedSecurity) — the read-path's `0x402` (read-only) makes the server return err 132 `OperationNotEnabled` silently. The analog Float `CTagMetadata` payload is 144 bytes with a leading `0x4E` marker byte and a 2-byte trailer `FE xx` where the second byte is the ApplyScaling flag (`00` for false / `01` for true). The `IHistoryServiceContract2` surface has no `UpdateTags` operation — distinct MinRaw/MaxRaw persistence is achieved entirely by toggling that one byte in the EnsT2 payload, not via a follow-up call. See `docs/reverse-engineering/handoff.md` and the `WriteDiag` env-gated diagnostic helper in `HistorianWcfTagWriteOrchestrator` for capture details.
### Cross-platform status (verified 2026-05-04)
The SDK builds and runs on Linux (Debian 13, .NET 10 SDK 10.0.203). `HistorianSspiClient` was rewritten on top of `System.Net.Security.NegotiateAuthentication` so the only remaining Windows-only surface is in WCF itself:
-**Build** — clean on Linux (no platform-specific compile errors after the
P/Invoke removal).
-**`ProbeAsync` over `RemoteTcpCertificate`** from a Debian client
(10.100.0.35) against the Windows Historian (10.100.0.48) — TLS handshake
succeeds, server returns its version.
- ⚠️ **`RemoteTcpIntegrated`** fails on Linux at the WCF transport layer
(`SecurityNegotiationException → AuthenticationException`). `NetTcpBinding`
with `SecurityMode.Transport` + `TcpClientCredentialType.Windows` requires
Windows-only auth code in WCF that isn't ported to .NET on Linux. This is
a hard WCF limitation, not a `HistorianSspiClient` issue. The
`HistorianWcfBindingFactory.CreateMdasNetNamedPipeBinding` and
`CreateMdasNetTcpWindowsBinding` methods carry a `#pragma warning disable
CA1416` documenting this.
-**Authenticated WCF calls via NegotiateAuthentication GSSAPI/NTLM**
from Linux — verified end-to-end with explicit credentials:
`GetTagMetadataAsync` returned correct fields, `BrowseTagNamesAsync`
returned matching tags. Confirms the SDK's auth chain (Open2 → ValCl × N
→ service call) works cross-platform.
-**Cert-binding calls from Linux** verified end-to-end with the two
new `HistorianClientOptions` knobs: `AllowUntrustedServerCertificate=true`
(skips X509 chain validation — needed because .NET WCF on Linux ignores
the system CA bundle) plus `ServerDnsIdentity="localhost"` (matches the
installer-generated cert's DNS claim when reaching the server by IP).
`ReadRawAsync`, `GetSystemParameterAsync`, `BrowseTagNamesAsync`, and
`GetTagMetadataAsync` all succeed from Debian 13 against the Windows
Historian over `RemoteTcpCertificate` with explicit Windows credentials.
### Remaining gaps ### Remaining gaps
Smaller, isolated items — none block the production read surface: Smaller, isolated items — none block the production read surface:
- Remote TCP transports (`RemoteTcpIntegrated`, `RemoteTcpCertificate`) untested against an actual remote Historian (tests skip without `HISTORIAN_REMOTE_TCP_HOST`). - Remote TCP transports verified by pointing `HISTORIAN_REMOTE_TCP_HOST` (and `HISTORIAN_REMOTE_TCPCERT_HOST` for the cert variant) at the host's own LAN IP — exercises the `MdasNetTcpWindows` / `MdasNetTcpCertificate` binding branches and SSPI/TLS handshake against a hostname rather than the loopback fast path. `RemoteTcpIntegrated`: 9 tests (Probe + full read surface + status helpers). `RemoteTcpCertificate`: Probe only; deeper coverage awaits an explicit-creds setup. True off-box verification (e.g. Linux client) would require porting `HistorianSspiClient` off `InitializeSecurityContextW` to managed `NegotiateAuthentication` + GSSAPI.
- Explicit username/password tag-metadata path (`HistorianWcfTagClient` line ~357) throws — only integrated security wired for that op. - Explicit username/password tag-metadata path is wired (validator only blocks no-auth-at-all), but live-verification requires `HISTORIAN_USER`+`HISTORIAN_PASSWORD` set; gated test `GetTagMetadataAsync_ExplicitCredentials_AgainstLocalHistorian` skips otherwise.
- `Historian2020ProtocolDialect.GetTagInfoByName/GetTagInfos` throws — currently dead code; `GetTagMetadataAsync` works through the WCF tag client instead. - Per-row trailing 35 bytes of `GetNextQueryResultBuffer` are now mapped (see `HistorianDataQueryProtocol.TryParseGetNextQueryResultBufferRows` doc comment) — bytes 3-10 = duplicate FILETIME (already used by aggregate parser), bytes 0-2 + 19-34 = server-internal sample/storage metadata with no clear user-facing meaning. No new public fields added; revisit if a customer asks for storage metadata exposure.
- Per-row trailing ~24 bytes of `GetNextQueryResultBuffer` are not decoded (likely per-sample value/source/state metadata). - (No remaining gaps in the write surface — `ApplyScaling` is now wired, see Required SDK Surface above.)
- `EnsureTagAsync` distinct `MinRaw`/`MaxRaw` persistence requires `ApplyScaling=true` + a follow-up `UpdateTags` call — not yet wired (no API user has asked).
### Tools Layer ### Tools Layer
@@ -122,5 +155,5 @@ Unit tests are golden-byte and round-trip oriented — `WcfDataQueryProtocolTest
- Never commit credentials, hostnames, user names, customer tag names, or raw packet captures. Use placeholders in docs. - Never commit credentials, hostnames, user names, customer tag names, or raw packet captures. Use placeholders in docs.
- Run a sanitization scan after touching auth/capture docs (the rg pattern is in handoff.md "Next Pickup Steps"). - Run a sanitization scan after touching auth/capture docs (the rg pattern is in handoff.md "Next Pickup Steps").
- Production code under `src/` must remain pure managed .NET 10 with no native AVEVA reference. Reverse-engineering harnesses under `tools/` may reference native binaries. - Production code under `src/` must remain pure managed .NET 10 with no native AVEVA reference. The one allowed P/Invoke is into the Windows SSPI surface (`HistorianSspiClient`) for integrated-auth tokens; do not add unrelated P/Invokes. Reverse-engineering harnesses under `tools/` may reference native binaries.
- This workspace IS a Git working tree (origin: gitea.dohertylan.com). Use normal git workflow; the prior note about "no working tree, track via timestamps" is obsolete. - This workspace IS a Git working tree (origin: gitea.dohertylan.com). Use normal git workflow; the prior note about "no working tree, track via timestamps" is obsolete.
+183 -10
View File
@@ -5,22 +5,27 @@ production SDK has no dependency on `aahClientManaged.dll`, `aahClient.dll`, or
any other AVEVA native runtime — the wire protocol is reverse-engineered and any other AVEVA native runtime — the wire protocol is reverse-engineered and
re-implemented in C#. re-implemented in C#.
Read-only by design. The required surface (per [`CLAUDE.md`](CLAUDE.md)): The supported surface (per [`CLAUDE.md`](CLAUDE.md)):
| Operation | Status | | Operation | Status |
|---|---| |---|---|
| `ProbeAsync` | live-verified | | `ProbeAsync` | live-verified |
| `ReadRawAsync` | live-verified | | `ReadRawAsync` | live-verified |
| `ReadAggregateAsync` | live-verified (TimeWeightedAverage; other modes need fixtures) | | `ReadAggregateAsync` | live-verified across all 16 retrieval modes |
| `ReadAtTimeAsync` | live-verified | | `ReadAtTimeAsync` | live-verified |
| `ReadEventsAsync` | live-verified (typed event + 31-property property bag) | | `ReadEventsAsync` | live-verified (typed event + 31-property property bag) |
| `BrowseTagNamesAsync` | live-verified | | `BrowseTagNamesAsync` | live-verified |
| `GetTagMetadataAsync` | live-verified for 17 distinct native data-type codes | | `GetTagMetadataAsync` | live-verified for 17 distinct native data-type codes |
| `GetConnectionStatusAsync` | synthesized from authenticated probe (matches native semantic) | | `GetConnectionStatusAsync` | synthesized from authenticated probe (matches native semantic) |
| `GetStoreForwardStatusAsync` | synthesized defaults (no SF sidecar to probe) | | `GetStoreForwardStatusAsync` | gRPC: measured idle-state (live-verified — contacts server, reports `ErrorOccurred` when unreachable; active-SF magnitude is D2-gated). Non-gRPC: synthesized defaults |
| `GetSystemParameterAsync` | live-verified via `Stat/GetSystemParameter` | | `GetSystemParameterAsync` | live-verified via `Stat/GetSystemParameter` |
| `EnsureTagAsync` | live-verified for analog Float/Double/Int2/Int4/UInt4; `ApplyScaling=true` persists distinct MinRaw/MaxRaw |
| `DeleteTagAsync` | live-verified |
Out of scope: write-back, store-forward write, configuration changes. Out of scope: writing samples (`AddS2` is architecturally blocked — the server's
runtime cache only ingests from configured IOServer / Application Server
pipelines), store-forward write, configuration changes, discrete/string tag
creation (native `AddTag` rejects them).
## Quick start ## Quick start
@@ -51,6 +56,143 @@ auth) and `RemoteTcpCertificate` (server-cert TLS) are now live-verified for
`ProbeAsync`; `RemoteTcpIntegrated` is additionally live-verified for the full `ProbeAsync`; `RemoteTcpIntegrated` is additionally live-verified for the full
read / browse / metadata / event / status surface. read / browse / metadata / event / status surface.
## Transport matrix (WCF vs gRPC)
The SDK speaks two wire protocols. **WCF** is the 2020 binary MDAS protocol over
Net.TCP `32568` (transports `LocalPipe`, `RemoteTcpIntegrated`,
`RemoteTcpCertificate`); **gRPC** is the 2023 R2 HCAL transport over HTTP/2
`32565` (transport `RemoteGrpc`). The core read chain reuses the *same* native
Open2 buffers and SSPI tokens on both — on gRPC they simply ride inside protobuf
`bytes` fields — so reads are at parity. The surfaces diverge at the edges.
Legend: ✅ tooled + live-verified · ⚠️ tooled, partial/synthesized ·
🧪 tooled + routed but **sandbox-gated** (mutates server state, not yet run
destructively against a live box) · 🔌 **the gRPC server exposes the RPC
(recovered in `Grpc/Protos/*.proto`) but the SDK doesn't drive it yet** —
untooled/uncaptured, *not* a protocol gap · ⛔ tooled but **server-walled** (the
request rides the RPC but the server faults on an unmet precondition) ·
❌ unavailable on that transport.
| Operation | WCF | gRPC | Notes |
|---|:---:|:---:|---|
| `ProbeAsync` | ✅ | ✅ | |
| `ReadRawAsync` | ✅ | ✅ | |
| `ReadAggregateAsync` | ✅ | ✅ | all 16 retrieval modes |
| `ReadAtTimeAsync` | ✅ | ✅ | |
| `BrowseTagNamesAsync` | ✅ | ✅ | |
| `GetTagMetadataAsync` | ✅ | ✅ | |
| `GetSystemParameterAsync` | ✅ | ✅ | |
| `AddHistoricalValuesAsync` | ❌ | ✅ | historical/backfill writes ride `HistoryService.AddStreamValues`; non-gRPC throws `ProtocolEvidenceMissingException` |
| `GetServerTimeZoneAsync` | ❌ | ✅ | 2020 `GetSystemTimeZoneName` is a client-side stub (empty); WCF throws |
| `GetStoreForwardStatusAsync` | ⚠️ | ✅ | gRPC contacts the server (measured idle-state, reports `ErrorOccurred`); WCF returns synthesized all-false. Active-SF magnitude is D2-gated on both |
| `GetRuntimeParameterAsync` | ✅ | ✅ | tooled + live-verified over gRPC (`StatusService.GetRuntimeParameter`, the 2020 `GETRP` buffers ride unchanged) |
| `GetTagExtendedPropertiesAsync` | ✅ | ✅ | tooled + live-verified over gRPC (`RetrievalService.GetTagExtendedPropertiesFromName`, the `GetTepByNm` buffers ride unchanged). The shared parser now handles the live multi-property response shape (one group per property + a uint16 searchability-flags trailer), fixed 2026-06-22 |
| `ExecuteSqlCommandAsync` | ✅ | ⛔ | gRPC request rides `RetrievalService.ExecuteSqlCommand`, but the server-side `CSrvDbConnection.ExecuteSqlCommand` faults (`IndexOutOfRange`, native err 38) — an unmet DB-connection precondition. A `HistoryService.RegisterTags` prime does **not** clear it (tried live 2026-06-22, both 0x402/0x401). Bounded behind `ProtocolEvidenceMissingException`. Use WCF |
| `ReadEventsAsync` | ✅ | ⚠️ | tooled + routed over gRPC: the full CM_EVENT registration replay (`UpdateClientStatus``RegisterTags``EnsureTags` + discovery probes) runs and `StartEventQuery` succeeds, but `GetNextEventQueryResultBuffer` **long-polls** on no data (it blocks to the deadline rather than returning the synchronous 5-byte code-85 terminal the WCF op gives). The read is **hard-bounded** (≤30s) and throws `ProtocolEvidenceMissingException` on the no-row path rather than assert a false empty. Row-level retrieval is **not yet live-verified** — the dev box holds no events; pending a capture against an event-bearing 2023 R2 server. Use WCF for event reads |
| `SendEventAsync` | ✅ | 🔌 | rides `AddStreamValues` family; no distinct event-send RPC, framing uncaptured over gRPC |
| `EnsureTagAsync` / `DeleteTagAsync` / `RenameTagsAsync` | ✅ | ✅ | live-verified 2026-06-22 over gRPC (`HistoryService.EnsureTags` / `DeleteTags` / `StartJob`, write-enabled 0x401 session, WCF serializers reused) via a self-cleaning sandbox-tag lifecycle. Rename is an async StartJob — transiently rejectable right after create, so callers should retry |
| `AddTagExtendedPropertiesAsync` | ✅ | ✅ | live-verified 2026-06-22 over gRPC (`HistoryService.AddTagExtendedProperties`, write-enabled session); a written prop now round-trips through `GetTagExtendedPropertiesAsync` (the multi-property parser fix above). `DeleteTagExtendedProperties` stays unshipped: probed over gRPC 2026-06-22 (prime `GetTgByNm`+`GetTepByNm` then `DelTep`, all on the one shared channel) — the server still rejects the delete (native code=1) and the property survives, so gRPC's multiplexed channel does **not** lift the WCF per-connection working-set wall |
| `GetConnectionStatusAsync` | ✅ | ✅ | live-verified 2026-06-22 over gRPC — measured from the handshake (`OpenConnection` yields a storage-session GUID ⇒ connected). No dedicated RPC on either transport; store-forward connectivity stays false (D2-gated) |
| `ReadBlocksAsync` | ❌ | ❌ | `StartBlockRetrievalQuery` never captured on either transport — throws `ProtocolEvidenceMissingException` |
In short: **WCF is the broad, mature surface** (every config write, events, SQL,
and all reads), while **gRPC is the narrower *tooled* surface** — but the 2023 R2
gRPC *contract* is actually a **superset** of WCF. The recovered config RPCs carry
the **same opaque `bytes` buffers the existing WCF serializers already emit**,
keyed by the same `strHandle`/`uiHandle` session handle the read path obtains —
confirmed by tooling the read-side config ops (`GetRuntimeParameter`,
`GetTagExtendedProperties`) live: the WCF buffers ride the gRPC RPC unchanged and
the server accepts them. Two caveats surfaced when capturing the rest: `ExecuteSqlCommand`
is **server-walled** (the front-door `CSrvDbConnection` faults on a DB-connection
precondition the managed session doesn't establish — the same *class* of wall as
`OpenStorageConnection`), and `ReadEvents` is now tooled over gRPC (the CM_EVENT
registration state machine is ported and `StartEventQuery` succeeds) but its row
retrieval is not yet live-verified: the gRPC server long-polls
`GetNextEventQueryResultBuffer` on no data instead of returning the WCF code-85
terminal, so on the idle dev box the bounded read throws
`ProtocolEvidenceMissingException` rather than fabricate an empty result —
confirming rows awaits an event-bearing 2023 R2 server. The remaining 🔌 row
(`SendEventAsync`) is a **capture-and-wire** item (route the existing serializer
into a gRPC orchestrator + live-capture), not protocol-discovery — but per
"capture first, never guess wire bytes" it stays untooled until verified live. The
natural production pattern today: `RemoteGrpc` now covers reads,
`AddHistoricalValuesAsync`, the tag-config writes (create/delete/rename/extended
properties, including read-back), and connection status — all live-verified. Use
WCF for SQL (server-walled on gRPC) and event reads/sends (gRPC event rows are
long-poll-blocked pending an event-bearing server).
> A 2023 R2 server reports History interface version 12 (vs. 11 on 2020). The
> connect-time version gate accepts both — they are byte-compatible — so gRPC
> works against a v12 server with the default `VerifyServerInterfaceVersion=true`;
> no opt-out is required.
## Resilience subsystems (M4)
Two **pure client-side** subsystems layered on top of `HistorianClient`. They use
only the public SDK surface — no extra reverse-engineering, no server-side
protocol — and are fully unit-tested without a live server.
### Store-and-forward (`AVEVA.Historian.Client.StoreForward`)
A durable local outbox that buffers writes when the Historian is unreachable and
replays them on reconnect. `HistorianStoreForwardWriter` wraps a `HistorianClient`
(or any `IHistorianWriteSink`) and persists each enqueued write to an
`IHistorianOutboxStore``FileHistorianOutboxStore` (crash-durable, atomic
JSON-per-entry, FIFO by filename sequence, corrupt-file quarantine) or
`InMemoryHistorianOutboxStore`. A background drain loop retries with FIFO
head-of-line ordering, optional `MaxDeliveryAttempts` dead-lettering, and a
`DropOldest`/`Reject` overflow policy.
```csharp
using AVEVA.Historian.Client.StoreForward;
await using HistorianStoreForwardWriter writer = new(
client,
new FileHistorianOutboxStore(@"C:\ProgramData\histsdk\outbox"));
await writer.StartAsync(); // background drain on reconnect
await writer.EnqueueHistoricalValuesAsync("MyTag",
[new HistorianHistoricalValue(DateTime.UtcNow, 42.0)]); // returns immediately, durable
HistorianStoreForwardStatusSnapshot status = await writer.GetStatusAsync();
// status.PendingCount / .Storing / .ErrorOccurred — mirrors the server SF semantics
```
This is a pragmatic outbox, **not** the bit-faithful native SF cache
(`Forward*Snapshot` decode), which stays deferred.
### Multi-historian redundancy (`AVEVA.Historian.Client.Redundancy`)
`HistorianRedundantClient` fronts N members as one logical client. Reads fail over
to the next healthy member in priority order (streaming reads fail over only
*before the first row* — mid-stream failures propagate to avoid duplicated/skipped
rows); writes fan out (`AllMembers`/`PreferredOnly`) under an `All`/`Any`
acknowledgement policy and return a per-member `HistorianRedundantWriteResult`. A
failure-threshold demotion + background watchdog restores recovered members.
```csharp
using AVEVA.Historian.Client.Redundancy;
await using HistorianRedundantClient cluster = new(
[
new HistorianClientMember("primary", primaryClient),
new HistorianClientMember("secondary", secondaryClient),
]);
await cluster.StartAsync(); // health watchdog
await foreach (HistorianSample s in cluster.ReadRawAsync("MyTag", startUtc, endUtc, 100))
{
// served by the first healthy member; transparently fails over on connect
}
HistorianRedundantWriteResult w = await cluster.AddHistoricalValuesAsync("MyTag",
[new HistorianHistoricalValue(DateTime.UtcNow, 42.0)]); // fanned out across members
```
Backing a member's writes with a `HistorianStoreForwardWriter` gives the pragmatic
equivalent of native ReSyncTags: a down member buffers locally and replays on
recovery.
## Build & test ## Build & test
```powershell ```powershell
@@ -75,6 +217,29 @@ $env:HISTORIAN_USER, $env:HISTORIAN_PASSWORD
$env:HISTORIAN_TAG_FILTER = 'Sys*' # or any LIKE-pattern $env:HISTORIAN_TAG_FILTER = 'Sys*' # or any LIKE-pattern
``` ```
The 2023 R2 `RemoteGrpc` transport has its own gated live suite
(`HistorianGrpcIntegrationTests`) covering the full tooled gRPC surface — probe,
raw / aggregate (incl. multiple retrieval modes) / at-time reads, browse,
metadata, system-parameter, server time-zone, and measured store-forward status.
It skips cleanly unless `HISTORIAN_GRPC_HOST` is set:
```powershell
$env:HISTORIAN_GRPC_HOST = 'my-2023r2-host' # gates the gRPC suite
$env:HISTORIAN_TEST_TAG = 'SysTimeSec'
$env:HISTORIAN_USER, $env:HISTORIAN_PASSWORD # required for the authenticated ops
# Optional:
$env:HISTORIAN_GRPC_PORT = '32565' # default 32565
$env:HISTORIAN_GRPC_TLS = 'true' # gRPC over TLS
$env:HISTORIAN_GRPC_DNSID = 'my-2023r2-host' # cert DNS name when connecting by IP
$env:HISTORIAN_GRPC_TIMEOUT = '120' # per-call deadline (s); raise for slow links
$env:HISTORIAN_WRITE_SANDBOX_TAG = 'MyFloatTag' # gates the AddHistoricalValues write test
$env:HISTORIAN_GRPC_WRITE_SANDBOX_TAG = 'SandboxTag' # gates the DESTRUCTIVE tag create/rename/delete lifecycle test
```
The aggregate tests self-calibrate their query window from a real raw sample, so
they pass against an idle 2023 R2 server (no recent collection) as well as a
live-collecting one.
## Repository layout ## Repository layout
``` ```
@@ -160,9 +325,17 @@ property dictionary → Retr.EndEventQuery → Hist.Close2
## Status ## Status
124 unit + live integration tests pass (`dotnet test --logger "console;verbosity=minimal"`). 313 unit + live integration tests pass (`dotnet test --logger "console;verbosity=minimal"`).
Full read-only SDK surface verified end-to-end against both a local Historian Full SDK surface — reads, browse, metadata, status, plus the two write ops
(`LocalPipe`) and a remote Historian (`RemoteTcpIntegrated` over Net.TCP with (`EnsureTagAsync` / `DeleteTagAsync`) — verified end-to-end against both a
Windows transport auth). `RemoteTcpCertificate` ProbeAsync is live-verified; local Historian (`LocalPipe`) and a remote Historian (`RemoteTcpIntegrated`
the other ops over the certificate transport plus the explicit-credentials over Net.TCP with Windows transport auth). `RemoteTcpCertificate` ProbeAsync
path await live verification. is live-verified; deeper coverage over the cert transport plus the
explicit-credentials path await additional verification.
The 2023 R2 `RemoteGrpc` transport's full tooled surface is live-verified against
a real 2023 R2 server: probe, raw / aggregate (TimeWeightedAverage +
Minimum/MaximumWithTime + BestFit) / at-time reads, browse, metadata,
system-parameter, server time-zone, and measured store-forward status — plus the
gRPC-only `AddHistoricalValuesAsync` backfill write. The remaining gRPC config ops
are exposed by the server but untooled (see the transport matrix above).
+183
View File
@@ -0,0 +1,183 @@
# gRPC Tooling Completion Plan
Status as of 2026-06-22. Tracks the remaining work to finish tooling the AVEVA
Historian SDK's `RemoteGrpc` (2023 R2) transport so it reaches WCF surface parity.
Self-contained for pickup after context compaction.
## Where things stand
The gRPC transport already tools: probe, raw/aggregate/at-time reads, browse,
metadata, system-parameter, server time-zone, measured store-forward status,
`AddHistoricalValues` backfill write, **and** (newest, branch `grpc-config-ops`,
3 commits, NOT yet merged — `main` = `035d8a9`):
- `GetRuntimeParameterAsync` — ✅ live-verified
- `GetTagExtendedPropertiesAsync` (read) — ✅ live-verified
- `ExecuteSqlCommandAsync` — ⛔ server-walled, bounded behind `ProtocolEvidenceMissingException`
- `EnsureTag` / `DeleteTag` / `RenameTags` / `AddTagExtendedProperties` — 🧪 tooled + routed, sandbox-gated, **not yet run destructively live**
- `ReadEventsAsync` — ⚠️ tooled + routed 2026-06-22 (item #2 below): chain runs, `StartEventQuery` succeeds, but `GetNextEventQueryResultBuffer` long-polls on no data; hard-bounded (≤30s) and throws `ProtocolEvidenceMissingException` on the no-row path. Row retrieval pending an event-bearing server.
Test baseline: 317 offline green, 19 gRPC-live green. Relevant memory:
`project_grpc_config_ops_tooling`, `project_m0_grpc_parity`,
`project_roadmap_exhausted_2020wcf`, `reference_2023r2_live_server_access`,
`reference_wonder_sql_vd03_credentials`.
## Proven pattern (reuse for everything below)
A WCF config op is tooled over gRPC by reusing its **existing byte serializer/parser
verbatim** inside the protobuf `bytes` fields, keyed by the Open2 session handle:
- `HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);`
- `HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, ct[, connectionMode]);`
- `session.StringHandle` = uppercase Open2 GUID → **string-handle** ops (Retrieval/Status/History string-handle RPCs).
- `session.ClientHandle` = transient `uint`**uint-handle** ops (StartQuery, DeleteTags, GetNext*).
- write ops pass `connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode` (0x401).
- Call `new <Service>.<Service>Client(connection.Channel).<Rpc>(request, connection.Metadata, DateTime.UtcNow.Add(options.RequestTimeout), ct)`.
- Check `response.Status?.BSuccess`; decode error via `response.Status?.BtError` (hex = native byte0 0x84 + LE u32 code, often followed by facility/file/message ASCII — this decode cracked the SQL + extended-prop cases).
- The gRPC RetrievalService string-handle ops do NOT need the WCF `Retr.GetV` prime.
Proto field-name reference and WCF serializer signatures: see the mapping captured
in `project_grpc_config_ops_tooling` memory and `Grpc/Protos/*.proto`.
## Remaining items (priority order)
### 1. Live-verify the write ops — ✅ DONE 2026-06-22
**Outcome:** ran the gated lifecycle against a synthetic sandbox tag (`ZZ_SdkGrpcWriteProbe`); the
writes flip 🧪→✅. `EnsureTags` (create), `AddTagExtendedProperties`, `StartJob` rename, and
`DeleteTags` all succeed live over gRPC (write-enabled 0x401 session, WCF serializers reused) — NO
priming discovery-dance needed. Two findings: (a) **rename** is an async StartJob that the server can
transiently reject right after the create commits and on target-name collision — the test now
pre-cleans both names and retries rename (4×); callers should likewise retry. (b) **reading a written
extended property back** via `GetTagExtendedPropertiesAsync` hits a shared-parser evidence gap (value
marker `0x01` where the parser expects compact-string `0x09`) — a read-side gap, not a write failure;
the test tolerates it. Lifecycle test is self-cleaning and best-effort cleans up (rename is async +
the browse/metadata view is eventually consistent, so a hard absence assert would be racy).
**Read-side follow-up DONE 2026-06-22:** captured the live `GetTagExtendedPropertiesFromName` bytes
and fixed the parser — the response is one group per property (tag name repeats) with a **uint16
searchability-flags trailer** per property (e.g. `0x0003` built-in, `0x0001` user-added), NOT the
1-byte group trailer the old model assumed (which drifted one byte per group → `0x09`-vs-`0x01`). A
written prop now round-trips end-to-end live; golden multi-group test added.
_Original notes:_
- **Goal:** flip the 🧪 writes to ✅ by running the gated lifecycle test against a sandbox tag.
- **How:** set `HISTORIAN_GRPC_WRITE_SANDBOX_TAG` to a throwaway name and run
`TagWriteLifecycle_OverGrpc_CreatesAddsPropRenamesDeletes` against the live 2023 R2 box.
- **Risk/gotcha:** if any write is rejected, the first fix is to add the WCF write
**priming discovery-dance** (`HistorianWcfTagWriteOrchestrator.RunWritePriming`:
UpdC3 + 6 `GetSystemParameter` + `AllowRenameTags` + Trx/Stat/Retr `GetV`) to
`HistorianGrpcTagWriteOrchestrator` over the gRPC StatusService/HistoryService.
Rename also needs server `AllowRenameTags` enabled. Needs explicit user OK to
mutate the shared server (they previously chose "no live mutate").
- **Files:** `tests/.../HistorianGrpcIntegrationTests.cs` (run only),
`src/.../Grpc/HistorianGrpcTagWriteOrchestrator.cs` (priming only if rejected).
### 2. ReadEvents over gRPC (heaviest read op) — ✅ TOOLED 2026-06-22 (rows pending event-bearing server)
**Outcome:** `ReadEventsAsync` is routed over gRPC (`HistorianGrpcEventOrchestrator`). The CM_EVENT
registration replay (`UpdateClientStatus`→6 `GetSystemParameter``RegisterTags`→cross-service version
probes→`EnsureTags`, captured buffers shared with WCF via `HistorianEventRegistrationProtocol`) runs
and **`StartEventQuery` succeeds live**. The blocker that remains is server behavior, not the port:
`GetNextEventQueryResultBuffer` **long-polls** when the query has no rows — it blocks to the call
deadline instead of returning the synchronous 5-byte type=4 code=85 terminal the 2020 WCF op returns.
Per-call gRPC-Web deadlines proved unreliable over the tunnel (a 4s-deadline chain still ran >90s), so
the read is hard-bounded by an **overall linked-CTS budget** (≤30s, scaled to `RequestTimeout`); gRPC
honors token cancellation. On the no-row path the orchestrator throws `ProtocolEvidenceMissingException`
rather than assert a false-empty list. The idle dev box holds no events, so **row-level retrieval is
not yet live-verified** — flip the gated test
`ReadEventsAsync_OverGrpc_StartsQueryButRowRetrievalIsLongPollBlocked` to assert parsed rows once an
event-bearing 2023 R2 server is available (and consider whether the long-poll needs a "fetch historical
then stop" request flag the native client may set). README row is ⚠️.
_Original notes (still the reference for the registration replay):_
- **Goal:** route `ReadEventsAsync` over gRPC.
- **RPCs (exist):** `RetrievalService.StartEventQuery` (`uiHandle`, `uiQueryRequestType`,
`btRequest`) → `{Status, uiQueryHandle, btResonse}`; `GetNextEventQueryResultBuffer`
(`uiHandle`, `uiQueryHandle`) → `{Status, btResult}`; `EndEventQuery`.
- **Reuse:** `HistorianEventQueryProtocol.CreateStartEventQueryAttempts(...)` for the
request buffer (`QueryRequestTypeEvent`), `HistorianEventRowProtocol.Parse(...)` for rows.
- **The hard part — port the CM_EVENT registration state machine.** Without it,
`GetNextEventQueryResultBuffer` returns native error type=4 **code=85**. WCF does this
in `HistorianWcfEventOrchestrator.AddCmEventTagViaAddT`: UpdC3 → 6 system params →
`RegisterTags2` (CM_EVENT tag id `353b8145-5df0-4d46-a253-871aef49b321`, 24-byte
RTag2 buffer) → cross-service `GetV``EnsureTags2` (CM_EVENT CTagMetadata via
`HistorianAddTagsProtocol.SerializeCmEventCTagMetadata`). gRPC equivalents:
`HistoryService.RegisterTags`, `HistoryService.EnsureTags`,
`HistoryService.UpdateClientStatus`, `StatusService.GetSystemParameter`.
- **Approach:** new `Grpc/HistorianGrpcEventOrchestrator`. Open a read-only session,
replay the registration over gRPC (RegisterTags + EnsureTags + the discovery calls),
then run StartEventQuery → loop GetNextEventQueryResultBuffer → EndEventQuery, parsing
rows. Route in `Historian2020ProtocolDialect.ReadEventsAsync` on `UseGrpc`.
- **Verify:** live (read-only, safe) against the 2023 R2 box; dev box may return no
rows (env) — assert "no error 85 + chain completes," mirror the WCF event test.
- **Risk:** medium-high. Registration may need exact call ordering; capture the error
buffer (hex+ASCII) at each step if code 85 persists.
### 3. SendEvent over gRPC
- **Goal:** route `SendEventAsync` over gRPC.
- **Blocker:** no distinct event-send RPC; WCF rides `AddStreamValues2` (the
`HistorianEventWriteProtocol.SerializeAddStreamValuesBuffer` VTQ). The gRPC framing is
**uncaptured** — needs a native-client gRPC capture before implementing (per
"capture first, never guess"). Depends on #2 (same CM_EVENT registration).
- **Risk:** high / blocked on capture. Lowest priority.
### 4. (Stretch) SQL server-wall investigation — ❌ RegisterTags prime does NOT help (2026-06-22)
- `ExecuteSqlCommand` over gRPC faults server-side in `CSrvDbConnection.ExecuteSqlCommand`
(IndexOutOfRange / native err 38). Tried the `HistoryService.RegisterTags`-family prime
before `ExecuteSqlCommand` on both read-only (0x402) and write-enabled (0x401) sessions:
it does **not** clear the wall — `RegisterTags` itself returned false and `ExecuteSqlCommand`
faulted with the identical native-38 error (decoded buffer: `...CSrvDbConnection.ExecuteSqlCommand
... System.IndexOutOfRangeException`). So unlike OpenStorageConnection, the SQL DB-connection
context is NOT established by the RegisterTags family. The op stays bounded behind
`ProtocolEvidenceMissingException`; use WCF for SQL. Remaining avenues are deeper (reproduce
the server-side DB connection-string/index setup the native client triggers) — low priority.
### 5. GetConnectionStatus over gRPC — ✅ DONE 2026-06-22
- `HistorianGrpcStatusClient.GetConnectionStatusAsync` synthesizes the status from a measured
gRPC handshake (OpenConnection yielding a storage-session GUID ⇒ connected), mirroring the WCF
synthesize-from-probe approach. Routed in `Historian2020ProtocolDialect` on `UseGrpc` (the WCF
path used the MDAS binding, which can't reach the gRPC port). Live-verified; store-forward
connectivity stays false (D2-gated). Gated test `GetConnectionStatusAsync_OverGrpc_ReportsConnected`.
### Out of scope
- `ReadBlocks` (`StartBlockRetrievalQuery`) — never captured on either transport; leave
throwing `ProtocolEvidenceMissingException`.
- `DeleteTagExtendedProperties` — ❌ **PROBED 2026-06-22, multiplexed-channel hypothesis DISPROVEN.**
The WCF block (server resolves the property from a per-connection working set the SDK's separate
per-service channels can't populate) is NOT lifted by gRPC. The probe
(`HistorianGrpcTagWriteOrchestrator.ProbeDeleteTagExtendedPropertiesAsync`) runs the native
`GetTgByNm``GetTepByNm``DelTep` sequence over ONE write-enabled (0x401) session on gRPC's
single shared channel. Live against the 2023 R2 server (History iface 12): both primes succeed on the
shared channel (`TgPrimeBytes=98`, `TepPrimePages=1`) yet `DelTep` is still rejected with native
**code=1** (the 5-byte error buffer's byte0=132 is the universal `0x84` marker, not a code) and the
property survives. Conclusion: the working set the server consults is populated by something the SDK
can't reproduce even over one connection — most likely the native client's in-process registration
object, not the wire session. Stays server-blocked on BOTH transports; not shipped publicly. Pinned
by the gated negative test `DeleteTagExtendedProperties_OverGrpc_ProbeMultiplexedChannel` (flips if a
future server/registration lifts the wall).
## Live verification setup (every live run)
Tunnel to `WONDER-SQL-VD03` must be up (gRPC `localhost:32565`, TLS, cert CN
`WONDER-SQL-VD03`; hosts entry present). Creds in gitignored `wonder-sql-vd03.txt`
(**QUOTED, colon-delimited** — strip quotes; use the `domainusername`/`domainpassword`
NAM domain account, which works for Historian gRPC; `wonderapp` does NOT). Env:
```
HISTORIAN_GRPC_HOST=wonder-sql-vd03 HISTORIAN_GRPC_PORT=32565
HISTORIAN_GRPC_TLS=true HISTORIAN_GRPC_DNSID=WONDER-SQL-VD03
HISTORIAN_USER=<domain user> HISTORIAN_PASSWORD=<domain pass>
HISTORIAN_TEST_TAG=SysTimeSec
# writes only, destructive: HISTORIAN_GRPC_WRITE_SANDBOX_TAG=<throwaway>
# slow links: HISTORIAN_GRPC_TIMEOUT=120
```
Run a subset: `dotnet test ./Histsdk.slnx --no-build --filter "FullyQualifiedName~<name>"`.
Aggregate tests self-calibrate their window from a real raw sample (the box is idle/
not-collecting). Sanitization scan before any commit:
`wonder-sql-vd03|zimmer|nam\\|dohertj2|ADOBuild` over commit-safe files.
## Standing constraints
- Never commit credentials/hostnames/customer tag names/raw captures — placeholders only.
- `src/` stays pure managed .NET 10 (one allowed P/Invoke: SSPI). Never modify `current/`
or `aveva-install-*/`.
- Commit only when asked; branch first if on `main`; required footers
(Co-Authored-By + Claude-Session). Capture wire bytes before implementing — never guess.
+265
View File
@@ -0,0 +1,265 @@
# AVEVA Historian SDK 2023 R2 — gRPC Transport Analysis
**Scope:** Documents the new gRPC transport that 2023 R2 adds to the Historian
Client Access Layer (HCAL). Kept deliberately **separate** from the main
`histsdk` reverse-engineering docs — this is 2023 R2 evidence, not the 2020/WCF
protocol the production SDK currently targets.
**Source:** the 2023 R2 `HistorianSDK` installer (**not installed**).
`SDKSetup.msi` was laid out with `msiexec /a` (administrative extract, no
registration) into a local `msi-extract` staging dir, then the managed
assemblies were decompiled with `ilspycmd`.
**Assembly versions analysed:** `2023.1219.4004.5`
(`Archestra.Grpc.Contract.dll`, `Archestra.Historian.GrpcClient.dll`,
`aahClientManaged.dll`).
---
## 1. Headline finding
The 2023 R2 gRPC transport is a **transport swap, not a protocol redesign.**
Every gRPC request/response wraps the **same opaque native binary buffers** that
the 2020/WCF-MDAS path already carries — `OpenConnection3` v6 buffer, NTLM/SSPI
`ValCl` tokens, `DataQueryRequest`, `GetNextQueryResultBuffer` row buffers, the
`Status` err blob, etc. — inside protobuf `bytes` fields.
Concrete proof, from `Archestra.Historian.GrpcClient`:
```csharp
// History/OpenConnection — same byte[] openParameters the WCF path built
OpenConnectionRequest request = new OpenConnectionRequest {
BtConnectionRequest = ByteString.CopyFrom(openParameters)
};
// Retrieval/StartQuery — same queryType + DataQueryRequest bytes + handle
StartQueryRequest request = new StartQueryRequest {
UiHandle = handle,
UiQueryRequestType = queryRequestType,
BtRequestBuffer = ByteString.CopyFrom(requestBuffer)
};
```
**Implication for the `histsdk` project:** all of the hard-won payload
serializers (`HistorianOpen2Protocol`, `HistorianDataQueryProtocol`,
`HistorianEventRowProtocol`, the SSPI `ValCl` token framing, the EnsT2
`CTagMetadata` layout) transfer **unchanged**. Only the envelope around them
changes: protobuf-over-gRPC instead of binary-SOAP-over-`application/x-mdas`.
The WCF `[MessageParameter(Name=…)]` guessing that dominated the 2020 work is
gone — field names and numbers are explicit in the protobuf contract.
---
## 2. Transport stack
From `GrpcClientBase.InitializeBase(target, portNumber, securedConnection, certificateName, trusted)`:
| Aspect | Value / behaviour |
|---|---|
| Library | `Grpc.Net.Client` + **`Grpc.Net.Client.Web`** (`GrpcWebHandler`) |
| Mode | **gRPC-Web**, `GrpcWebMode.GrpcWeb` (binary `application/grpc-web`, **not** `-text`) |
| HTTP version | **HTTP/1.1** (`GrpcWebHandler.HttpVersion = new Version(1,1)`) — *not* HTTP/2 |
| Address | `http://{target}:{port}` insecure, or `https://{certificateName}:{port}` secure |
| Inner handler | `HttpClientHandler` with custom `ServerCertificateCustomValidationCallback` |
| Compression | gzip on by default; request header `grpc-internal-encoding-request: gzip`; custom `CustomCompressionProvider` / `CustomGZipStream` used for bandwidth accounting |
| Default timeout | 60 s per call (`m_timeoutInSeconds = 60`, sent as gRPC deadline) |
| Interceptor | `ClientInterceptor` (logging hook, currently a no-op `LogCall`) |
Because it is **gRPC-Web over HTTP/1.1**, the transport is proxy/firewall
friendly and does not require HTTP/2 negotiation — note `HistorianConnectionArgs.ProxyServer`
(e.g. `http://host:9480`) in the public API.
### Port
- **Default port `32565`** — `HistorianConnectionArgs.TcpPort`, *"the TCP port
of the Historian Client Access Point."* (Note this differs from the 2020 WCF
port `32568` the production SDK uses.)
- All services reach the **same host:port**; gRPC multiplexes by service path
(`/HistoryService/OpenConnection`, `/RetrievalService/StartQuery`, …).
### Channel topology
Five service stubs grouped into four wrapper clients, each constructing its own
`GrpcChannel` to the same endpoint:
| Wrapper (`GrpcClientBase` subclass) | gRPC service stub(s) |
|---|---|
| `GrpcHistoryClient` | `HistoryService` + `TransactionService` (one channel) |
| `GrpcRetrievalClient` | `RetrievalService` |
| `GrpcStatusClient` | `StatusService` |
| `GrpcStorageClient` | `StorageService` |
---
## 3. Authentication model (unchanged in substance)
Auth is **still the native session handshake**, carried over gRPC instead of
WCF. There is **no per-call bearer/auth token in gRPC metadata** — the only
metadata sent is the gzip-encoding hint. Methods pass `m_metadata` (gzip) or
`null`; neither carries credentials. The server keys the session off the
`handle` GUID established by the handshake, exactly as the 2020 path does.
Handshake operations (same byte payloads as 2020):
- `HistoryService.GetInterfaceVersion` → version probe.
- `StorageService.ValidateClientCredential { string Handle; bytes InBuff }`
`{ Status; bytes OutBuff }`. **`InBuff`/`OutBuff` carry the NTLM/SSPI
tokens** — same multi-round Negotiate exchange, same field names the 2020
`ildasm` revealed (`inBuff`/`outBuff`), now first-class protobuf fields.
- `HistoryService.ExchangeKey { string StrHandle; bytes BtInput }`
`{ Status; bytes BtOutput }` (key-exchange / cert path).
- `HistoryService.OpenConnection { bytes BtConnectionRequest }`
`{ Status; bytes BtConnectionResponse }` — same `OpenConnection3` v6
request buffer in, same 42-byte session blob out.
Public-API security knobs (`aahClientManaged.xml`):
- `HistorianConnectionArgs.ConnectionMode` — *"whether GRPC connection to the
Historian Server. **Default is true** (GRPC)."* This is the master switch
selecting gRPC vs legacy.
- `HistorianSecurityMode`: `None`, `Disabled`, `TransportWindows`
(Windows creds), `TransportCertificate` (server cert).
- `AllowUnTrustedConnection` → maps to the `trusted` arg; when false the client
bypasses X509 chain validation (`ValidateServerCertificate` returns true
early). Equivalent to the production SDK's `AllowUntrustedServerCertificate`.
- `AuthenticationMode` default `HistorianNative`; `CertificateInfo.CertificateName`
supplies the `https://{certificateName}:{port}` SNI/host identity.
---
## 4. gRPC service surface (full RPC list)
All methods are unary (`MethodType 0`). Names map 1:1 onto the 2020 WCF
operations the production SDK already understands.
### HistoryService (`/HistoryService/…`)
`GetInterfaceVersion`, `ExchangeKey`, `OpenConnection`, `CloseConnection`,
`UpdateClientStatus`, `RegisterTags`, `EnsureTags`, `AddStreamValues`,
`AddTagExtendedPropertyGroups`, `AddTagExtendedProperties`, `StartJob`,
`GetJobStatus`, `DeleteTagExtendedProperties`, `DeleteTags`,
`AddTagLocalizedProperties`, `DeleteTagLocalizedProperties`
### RetrievalService (`/RetrievalService/…`)
`GetRetrievalInterfaceVersion`, `StartQuery`, `GetNextQueryResultBuffer`,
`EndQuery`, `GetShardTagidsByTagnameAndSource`, `GetTagInfosFromName`,
`GetTagExtendedPropertiesFromName`, `ExecuteSqlCommand`, `StartEventQuery`,
`GetNextEventQueryResultBuffer`, `EndEventQuery`, `StartTagQuery`, `QueryTag`,
`EndTagQuery`, `GetTagLocalizedPropertiesFromName`
### StatusService (`/StatusService/…`)
`GetStatusInterfaceVersion`, `GetSystemParameter`, `SendInfo`, `RequestInfo`,
`DeleteInfo`, `GetHistorianInfo`, `StartProcess`, `StopProcess`, `PingServer`,
`PingPipe`, `ConfigureAutoStartProcess`, `GetHistorianConsoleStatus`,
`GetRuntimeParameter`, `GetSystemTimeZoneName`, `SetHistorianConsoleStatus`,
`CanUpdateAreaHierarchy`, `UpdateAreaHierarchy`, `UpdateObjectHierarchy`
### StorageService (`/StorageService/…`)
`GetInterfaceVersion`, `OpenStorageConnection`, `OpenStorageConnection2`,
`CloseStorageConnection`, `Ping`, `AddTags`, `RegisterTags`, `AddStreamValues`,
`AddStreamValues2`, `GetTagIds`, `GetTags`, `FlushMetadata`, `FlushData`,
`LoadBlocks`, `GetSnapshots`, `StartQuerySnapshot`, `NextQuerySnapshot`,
`EndSnapshot`, `Stop`, `ClearTagidPairs`, `AddTagidPairs`, `GetSFParameter`,
`SetSFParameter`, `SendSnapshotBegin`, `SendSnapshotEnd`, `SendSnapshot`,
`DeleteSnapshot`, `ClearShardTagids`, `AddShardTagids`, `SplitUnknownShards`,
`GetRemainingSnapshotsSize`, `DeleteTags`, `OpenStorageConnection2`,
`ValidateClientCredential`, `GetInfo`
### TransactionService (`/TransactionService/…`)
`ForwardSnapshot`, `ForwardSnapshotBegin`, `ForwardSnapshotEnd`,
`GetTransactionInterfaceVersion`, `AddNonStreamValuesBegin`,
`AddNonStreamValues`, `AddNonStreamValuesEnd`
> A separate `ArchestrA.CloudHistorian.Contract` assembly defines a parallel
> cloud-ingest contract (`AddHistorianValues`, `CreateTags`, `EnqueueTagDataPacket`,
> `Enqueue…`, etc.) used by `aahCloudConfigurator` / `online.wonderware.com`.
> Out of scope here; noted for completeness.
---
## 5. Representative message shapes (protobuf field numbers)
The universal result wrapper and the auth/query messages — note how thin they
are; the real structure lives inside the `bytes` fields.
```proto
// Common result wrapper (ArchestrA.Grpc.Contract.RequestStatus)
message Status {
bool bSuccess = 1; // success flag (replaces WCF return-bool)
bytes btError = 2; // native error buffer (same type/code blob as WCF err)
}
// HistoryService
message OpenConnectionRequest { bytes btConnectionRequest = 1; } // OpenConnection3 v6 buffer
message OpenConnectionResponse { Status status = 1; bytes btConnectionResponse = 2; } // 42-byte session blob
message ExchangeKeyRequest { string strHandle = 1; bytes btInput = 2; }
// StorageService — Negotiate/NTLM handshake
message ValidateClientCredentialRequest { string handle = 1; bytes inBuff = 2; }
message ValidateClientCredentialResponse { Status status = 1; bytes outBuff = 2; }
// RetrievalService
message StartQueryRequest {
uint32 uiHandle = 1;
uint32 uiQueryRequestType = 2; // RetrievalMode → QueryType, same mapping as 2020
bytes btRequestBuffer = 3; // DataQueryRequest bytes, byte-identical to WCF
}
```
Across the contract the recurring pattern is `{ Status status; bytes <payload> }`
for responses and `{ [string handle][uint …] bytes <payload> }` for requests.
**The full canonical IDL has been recovered.** All six `.proto` files were
rendered from the embedded `FileDescriptor`s and protoc-validated — see
`../out/proto/*.proto`, the portable `../out/archestra_grpc.fileset.pb`
(`FileDescriptorSet`), and the per-message field dump `../out/grpc-contract-dump.md`.
`../out/README.md` explains the contract quirks (global proto package, cross-file
name collisions, all-unary RPCs) and the 2020→gRPC read-path mapping. Regenerate
with the `protodump/` tool.
---
## 6. What this means for histsdk (if a gRPC transport is ever added)
This is **not** a request to implement anything — recording the path:
1. Add a transport enum value (e.g. `RemoteGrpc`) alongside `LocalPipe` /
`RemoteTcpIntegrated` / `RemoteTcpCertificate`.
2. Reference `Grpc.Net.Client` + `Grpc.Net.Client.Web`; build a
`GrpcChannel.ForAddress("http(s)://host:32565", { HttpHandler =
GrpcWebHandler(GrpcWeb, HttpClientHandler), … })` with HTTP/1.1.
3. Reuse **every existing payload serializer unchanged** — feed the same byte
buffers into the protobuf `bytes` fields instead of MDAS bodies. The
orchestrator call order (`GetV → ValCl×N → Open2 → Retr.GetV →
IsOriginalAllowed → StartQuery → GetNextQueryResultBuffer…`) is identical.
4. Auth: still the SSPI/Negotiate token loop via `ValidateClientCredential`,
carried in `inBuff`/`outBuff`. No per-call gRPC auth metadata needed.
5. Biggest win: **no WCF `[MessageParameter]` reverse-engineering** — the
protobuf field numbers are authoritative and stable.
Caveat: this is the **2023 R2** server contract. The production SDK targets a
2020-era server; whether that server exposes the gRPC HCAP endpoint at all is a
server-version question, not a client one. Treat this as forward-looking.
---
## 7. Artifacts (all under the separate analysis folder, none committed to histsdk)
```
histsdk-2023r2-analysis/
msi-extract/ # msiexec /a layout of SDKSetup.msi
bin/ # copied key assemblies + aahClientManaged.xml
decompiled/
Archestra.Grpc.Contract/ # full protobuf contract (services + messages)
Archestra.Historian.GrpcClient/ # transport wrappers (channel/auth/calls)
ArchestrA.CloudHistorian.Contract/ # cloud-ingest contract (out of scope)
protodump/ # .NET 10 tool: descriptor graph -> .proto / dump
out/
proto/*.proto # recovered, protoc-validated IDL (6 files)
archestra_grpc.fileset.pb # portable FileDescriptorSet (grpcurl/buf/protoc)
grpc-contract-dump.md # per-message field dump + service tables
README.md # artifact guide + contract quirks + read-path map
docs/grpc-transport.md # this file
```
gRPC redist proof in the installer:
`Redist/HistorianSDK 2023 R2/x64/{GRPCCore,GRPCNetClient,HistorianGRPCClient,HistorianGRPCContract,Protobuf}.msm`
plus shipped `Grpc.Net.Client*.dll`, `Grpc.Core.Api.dll`, `Google.Protobuf.dll`.
+166
View File
@@ -0,0 +1,166 @@
# HCAL → modern-.NET reimplementation — capability matrix
Feasibility map for a clean managed-.NET client that replaces the AVEVA Historian
SDK (`aahClientManaged` / HCAL). Grounded in: the real `ArchestrA.HistorianAccess`
public surface (`aahClientManaged.xml`), the recovered **2023 R2 gRPC contract**, the
existing **histsdk** reimplementation, and the event/storage analysis in
[`histevents.md`](histevents.md).
## Legend
**Status (histsdk today)** — ✅ implemented + live-verified · 🟗 partial · ⬜ not yet
**Feasibility tier**
| Tier | Meaning | Effort |
|---|---|---|
| **DONE** | already working in histsdk | 0 |
| **TRIVIAL** | gRPC op known, payload already decoded or empty | XS (hrs) |
| **CAPTURE** | one instrument-and-capture of a native payload, then serialize + golden-byte test | S (days) |
| **BOUNDED** | gRPC op exists; decode one proprietary `bytes` payload | SM |
| **HARD** | whole subsystem to reimplement | L (weeks) |
| **GATED** | blocked server-side — client effort doesn't unblock it | n/a |
Effort = incremental work on top of histsdk's existing infrastructure (auth chain,
transport, frame/byte primitives, test harness). All non-DONE items assume the
**gRPC transport** as the foundation (clean protobuf envelope; only the inner byte
blob needs RE).
---
## 1. Connection & session
| Capability | HCAL API | 2023 R2 gRPC op | histsdk | Tier | Notes |
|---|---|---|---|---|---|
| Probe / version | `TestConnection`, GetV | `*Service.GetInterfaceVersion` | ✅ | DONE | |
| Open connection (Process) | `OpenConnection` | `History.OpenConnection` (+ `ExchangeKey` auth) | ✅ | DONE | full auth chain works |
| Open connection (Event) | `OpenConnection` (Event type) | `History.OpenConnection` event mode | 🟗 | TRIVIAL | read path already opens it; flag = ConnectionType.Event |
| Close connection | `CloseConnection` | `History.CloseConnection` | ✅ | DONE | |
| Connection status | `GetConnectionStatus` | `Status.GetHistorianConsoleStatus` | ✅ | DONE | |
| Open/close **storage** connection | `OpenStorageConnection`, `CloseStorageConnection` | `Storage.OpenStorageConnection2` | ⬜ | BOUNDED | needed for any data-write path; storage-engine session |
## 2. Reads — process data
| Capability | HCAL API | 2023 R2 gRPC op | histsdk | Tier | Notes |
|---|---|---|---|---|---|
| Raw / full history | `CreateHistoryQuery` → Start/MoveNext/End | `Retrieval.StartQuery``GetNextQueryResultBuffer``EndQuery` | ✅ | DONE | row buffer parsed |
| Aggregate (interp/avg/min/max/…) | `CreateHistoryQuery` (RetrievalMode) | same | ✅ | DONE | all 15 RetrievalModes mapped |
| At-time / value-at | (interp window) | same | ✅ | DONE | |
| Analog summary | `CreateAnalogSummaryQuery` | `Retrieval.StartQuery` (summary mode) | 🟗 | BOUNDED | mode variant of existing query |
| State summary | `CreateStateSummaryQuery` | `Retrieval.StartQuery` (state mode) | ⬜ | BOUNDED | extra row layout to decode |
| Block read | `ReadBlocks` | `Storage.LoadBlocks` | ⬜ | BOUNDED | low-level; rarely needed |
## 3. Reads — events
| Capability | HCAL API | 2023 R2 gRPC op | histsdk | Tier | Notes |
|---|---|---|---|---|---|
| Event query | `CreateEventQuery` → Start/MoveNext/End | `Retrieval.StartEventQuery``GetNextEventQueryResultBuffer``EndEventQuery` | ✅ | DONE | rows + typed property bag parsed; CM_EVENT registration done |
| Event filters | `EventQuery.AddEventFilter` / `AddEventFilterCondition` | filter bytes in StartEventQuery request | ⬜ | BOUNDED | encode filter predicate into request buffer |
## 4. Browse & metadata
| Capability | HCAL API | 2023 R2 gRPC op | histsdk | Tier | Notes |
|---|---|---|---|---|---|
| Tag name browse | `CreateTagQuery``GetTagNames` | `Retrieval.StartTagQuery`/`QueryTag` (or LikeTagnames) | ✅ | DONE | wildcard works |
| Tag metadata | `GetTagInfoByName`, `TagQuery.GetTagInfo` | `Retrieval.GetTagInfosFromName` | ✅ | DONE | |
| Extended properties (read) | `GetTagExtendedPropertiesByName` | `Retrieval.GetTagExtendedPropertiesFromName` | ⬜ | BOUNDED | TEP buffer decode |
| Localized properties (read) | `GetTagLocalizedPropertiesByName` | `Retrieval.GetTagLocalizedPropertiesFromName` | ⬜ | BOUNDED | |
| SQL passthrough | `ExecuteSqlCommand` | `Retrieval.ExecuteSqlCommand` | ⬜ | TRIVIAL | thin string-in / status-out |
## 5. Tag configuration (writes)
| Capability | HCAL API | 2023 R2 gRPC op | histsdk | Tier | Notes |
|---|---|---|---|---|---|
| Create analog tag | `AddTag` | `History.EnsureTags` (EnsT2) | ✅ | DONE | Float/Double/Int2/Int4/UInt2/UInt4 + scaling |
| Create string/discrete tag | `AddTag` | `History.EnsureTags` | ⬜ | GATED/BOUNDED | native AddTag rejects these types server-side; needs different metadata path |
| Delete tag(s) | `DeleteTags` | `History.DeleteTags` | ✅ | DONE | |
| Rename tag(s) | `RenameTags` | (History op) | ⬜ | BOUNDED | `AllowRenameTags` param already probed |
| Add/Delete extended properties | `AddTagExtendedProperties`, `DeleteTagExtendedPropertiesByName` | `History.AddTagExtendedProperties` / `DeleteTagExtendedProperties` | 🟗 | BOUNDED | **Add DONE** (`AddTagExtendedPropertiesAsync`, AddTEx; inBuff = inverse of R1.5 read framing + trailing `01 00`). Delete (DelTep) deferred — native sync gate (err 229) blocks capturing its inBuff. See `wcf-add-tag-extended-properties.md` |
| Add/Delete localized properties | `AddTagLocalizedProperties`, `DeleteTagLocalizedPropertiesByName` | `History.AddTagLocalizedProperties` / `DeleteTagLocalizedProperties` | ⬜ | BOUNDED | |
## 6. Data writes — values
| Capability | HCAL API | 2023 R2 gRPC op | histsdk | Tier | Notes |
|---|---|---|---|---|---|
| Stream process values | `AddStreamedValue(HistorianDataValue)` | `Storage.AddStreamValues` | ⬜ | **GATED** | runtime cache only ingests from IOServer/AppServer pipelines (`129 Tag not found in cache`). Not a client bug |
| Stream **events** | `AddStreamedValue(HistorianEvent)` | `Storage.AddStreamValues` (event VTQ) | ⬜ | **CAPTURE** | full path mapped; need `CCommonArchestraEventValue::PackToVtq` blob bytes. See histevents.md |
| Non-streamed / historical insert | `AddNonStreamedValue`, `SendNonStreamedValues` | `Transaction.AddNonStreamValues(Begin/End)` | ⬜ | BOUNDED | explicit original-data insert via Transaction svc; verify ingest permission on target |
| Versioned streamed value | `AddVersionedStreamedValue` | `Storage.AddStreamValues2` | ⬜ | CAPTURE | revision flag on the VTQ |
## 7. Revisions / edits (modify stored data)
| Capability | HCAL API | 2023 R2 gRPC op | histsdk | Tier | Notes |
|---|---|---|---|---|---|
| Insert/update/delete revision values | `AddRevisionValue(s)`, `AddRevisionValuesBegin/End` | (storage-engine / transaction path) | ⬜ | HARD | prior RE: revision-write needs the non-WCF **storage-engine pipe** (`STransactPipeClient2`), not the WCF/gRPC surface |
| Event update/delete (revise) | `HistorianEvent.Update/.Delete` | `UpdateEventStatus` (+ revised VTQ) | ⬜ | CAPTURE | RevisionVersion + Update/Delete flags in the event VTQ |
## 8. Status & system info
| Capability | HCAL API | 2023 R2 gRPC op | histsdk | Tier | Notes |
|---|---|---|---|---|---|
| System parameter | `GetSystemParameter` | `Status.GetSystemParameter` | ✅ | DONE | |
| Runtime parameter | `GetRuntimeParameter` | `Status.GetRuntimeParameter` | ⬜ | TRIVIAL | same shape as GetSystemParameter |
| Historian info | `GetHistorianInfo` | `Status.GetHistorianInfo` | 🟗 | BOUNDED | **2020 WCF = version-only** (GETHI is a named-value query; `EventStorageMode` not on the wire). 518-byte struct + `EventStorageMode`@514 is gRPC/2023R2-only. See `wcf-historian-info.md` |
| Server timezone | `GetSystemTimeZoneInfo` | `Status.GetSystemTimeZoneName` | ⬜ | TRIVIAL | |
| Historization status | `GetHistorizationStatus` | `Status` op | ⬜ | BOUNDED | |
| Store-and-forward status | `GetStoreForwardStatus` | (push events / pull GETHI) | 🟗 | HARD | currently synthesized; real read needs duplex push or a decoded pull endpoint — see store-forward plan |
## 9. Store-and-forward (offline buffering)
| Capability | HCAL API | 2023 R2 gRPC op | histsdk | Tier | Notes |
|---|---|---|---|---|---|
| SF buffering + replay | (implicit on write conns) | `Storage`/`Transaction` `*Snapshot` + `Forward*Snapshot` | ⬜ | HARD | full subsystem: local cache format, snapshot framing, recovery log, forward-on-reconnect. Pragmatic alt: a simpler local queue, not bit-faithful SF |
| Event SF | (event conn) | `Forward**Event**SnapshotBegin/…/End` | ⬜ | HARD | dedicated event-snapshot SF stream |
| SF parameters | Get/Set SFP | `Storage.GetSFParameter`/`SetSFParameter` | ⬜ | BOUNDED | |
## 10. Redundancy / multi-historian
| Capability | HCAL API | 2023 R2 gRPC op | histsdk | Tier | Notes |
|---|---|---|---|---|---|
| Tiered/redundant access, failover | `MultiHistorianAccess.*` (OpenConnectionToAll, AddSecondaries, partner watchdog, ReSyncTags) | N×single-historian sessions + client logic | ⬜ | HARD | mostly client-side orchestration over §1–§6; build last |
| Replication config | (server `aahReplication`) | — | ⬜ | GATED | server-side concern |
---
## Roll-up & recommended cut line
**Phase 0 — already DONE (✅):** probe · open/close · raw+aggregate+at-time reads ·
event reads · tag browse · tag metadata · system parameter · connection status ·
create/delete analog tag. This is a usable modern client **today**.
**Phase 1 — TRIVIAL/BOUNDED, high value (SM each):** ExecuteSqlCommand ·
runtime parameter · server timezone · extended/localized property read · event
filters · summary/state-summary queries · rename tags · ext/localized property
writes · GetHistorianInfo. Each is "gRPC op exists, decode one buffer, golden-byte
test." Knocks out most of the remaining read/config surface.
**Phase 2 — CAPTURE (one native capture each, S):** **event sending** (the headline
gap — fully mapped, one `PackToVtq` capture away) · versioned/non-streamed value
writes. Now feasible locally since the Historian is installed.
**Defer / simplify (HARD):** store-and-forward (do a pragmatic local queue instead of
bit-faithful SF) · revision/edit writes (separate storage-engine pipe) · multi-
historian redundancy (client orchestration, build last).
**Won't unblock from the client (GATED):** streaming **process-sample** writes
(`AddS2`) — server cache only ingests from IOServer/AppServer pipelines; confirm your
ingestion model rather than chasing this. Non-analog tag creation likely needs a
distinct server path.
## Cross-cutting realities (apply to every non-DONE row)
- **Inner payloads stay proprietary** even under gRPC — the `bytes` fields carry
native VTQ / CTagMetadata / event-value formats. These are **version-sensitive**;
pin to the server version probed at connect and fail closed on mismatch.
- **Validation needs a live Historian** — now available locally, which is what makes
the CAPTURE-tier items practical.
- **Support tradeoff** — you take on maintenance across Historian versions in exchange
for shedding the stock SDK's bugs (mixed-mode marshaling, WCF quirks, global state)
for the surface you cover.
## Bottom line
A modern-.NET HCAL replacement is **feasible and ~6070% done** for a typical
read+browse+config+event-read workload. The remaining high-value surface is mostly
**BOUNDED/CAPTURE** (incremental, well-understood), with only store-and-forward,
revision-edit, and redundancy being genuine **HARD** subsystems — and one true wall
(**GATED** process-sample writes) that no client can remove.
+362
View File
@@ -0,0 +1,362 @@
# HCAL modern-.NET client — implementation roadmap
Ordered, actionable plan to grow **histsdk** from "reads + basic config" into a broad
HCAL replacement, built on the **2023 R2 gRPC transport**. Derived from
[`hcal-capability-matrix.md`](hcal-capability-matrix.md); event details in
[`histevents.md`](histevents.md).
> Move to the repo's `docs/plans/` when execution starts. Each work item lands as: a
> protocol serializer/parser + golden-byte unit test + an env-gated live integration
> test against the local Historian.
## Status: roadmap exhausted (2026-06-22)
The reachable surface is **complete**. M0/M1/M2/M3 are done and live-verified; M4 R4.1 (store-forward
outbox), R4.3 (measured idle-state SF status), and R4.4 (redundancy) are shipped and **merged to
`main`**; and the follow-on `grpc-tooling-completion.md` plan is fully executed (writes, ReadEvents,
GetConnectionStatus, the extended-property read-parser fix — all done or bounded-out). The transport
matrix in the top-level `README.md` is the authoritative per-operation status.
**Nothing left is a pure code task** — every remaining item is gated:
- **Infra-gated** (needs a different live server): gRPC event *row* retrieval (`StartEventQuery`
succeeds but `GetNextEventQueryResultBuffer` long-polls — needs an **event-bearing** 2023 R2 server);
R4.3 active-SF *magnitude* (needs an **SF-active** server / the D2 storage-engine console handle).
- **Capture-gated** ("capture before guessing wire bytes"): `SendEvent` over gRPC (no distinct RPC;
framing uncaptured).
- **Architecturally walled** (no client-side fix): `ExecuteSqlCommand` over gRPC (server-side
`CSrvDbConnection` fault; a `RegisterTags` prime does not clear it); R4.2 revision *edits*
(storage-engine-pipe-only on both transports).
- **Out of scope until a demand signal**: `ReadBlocks` (`StartBlockRetrievalQuery`, never captured);
`DeleteTagExtendedProperties` (server-blocked on WCF).
## Progress (updated 2026-06-22)
-**R0.6 version gate**`HistorianServerVersionGate` + `HistorianClientOptions.VerifyServerInterfaceVersion`;
fail-closed on connect, wired into both WCF and gRPC paths. Supported versions are
evidence-based (Hist=11/12, Retr=4, Trx=2; Status reachability-only), captured from the
live server. History 12 (2023 R2 gRPC) accepted alongside 11 (buffer-compatible).
-**CW-1 capture pipeline**`ProtocolCaptureSanitizer` + `ProtocolFixtureWriter` +
`capture-tag-info` CLI command; produces sanitized `fixtures/protocol/<op>/` golden files.
11 unit tests. First fixture: `get-tag-info/analog-*.json`.
-**gRPC auth handshake (read chain)** — LIVE-VERIFIED 2026-06-21 against a real 2023 R2
server: `ReadRawAsync` over `RemoteGrpc` returns rows. Token loop routes to
`StorageService.ValidateClientCredential`. Shared handshake extracted to
`Grpc/HistorianGrpcHandshake` for reuse by the status/browse/metadata paths.
-**R0.4 Probe over gRPC**`Grpc/HistorianGrpcProbe` (History/Retrieval/Status
`GetInterfaceVersion`); `ProbeAsync` routes over gRPC when `Transport==RemoteGrpc`.
**LIVE-VERIFIED 2026-06-21** (no credentials required — runs before the auth loop).
-**R0.3 System parameter over gRPC**`Grpc/HistorianGrpcStatusClient.GetSystemParameterAsync`
(`StatusService.GetSystemParameter`); routed in the dialect. Built + unit-tested + **LIVE-VERIFIED
2026-06-21** against a real 2023 R2 server (returned `HistorianVersion`). Code path is the proven
handshake + a single string-in/string-out RPC.
-**R0.2 Tag metadata over gRPC**`Grpc/HistorianGrpcTagClient.GetTagMetadataAsync`
(`RetrievalService.GetTagInfosFromName`, the plural **string-handle** op). `GetTagMetadataAsync`
routes over gRPC when `Transport==RemoteGrpc`. Request `btTagNames` = `uint count + per-name(uint
charCount + UTF-16LE)` (golden-byte unit-tested); response `btTagInfos` = `uint count + CTagMetadata`
records (reuses `ParseGetTagInfoResponse`); string handle = uppercase Open2 storage GUID. The 2020
WCF string-handle wall does **not** apply on the gRPC front door (as predicted). **LIVE-VERIFIED
2026-06-21** — `GetTagMetadataAsync` returned the requested tag + a valid data type.
-**R0.1 Browse over gRPC** — DONE, **LIVE-VERIFIED 2026-06-21**.
`HistorianClient.BrowseTagNamesAsync` routes over gRPC via
`Grpc/HistorianGrpcTagClient.BrowseTagNamesAsync`: StartTagQuery(**OData** filter) → paged
**QueryTag** (`btRequest` = `u16 0x6752 + u16 1 + u16 queryType + u32 startIndex + u32 count`) →
EndTagQuery; response = `u32 count + per-name(u32 charCount + UTF-16LE) + trailer`. The SDK glob
filter is translated by `GlobToODataFilter` (`Pre*``startswith`, `*suf``endswith`, `*mid*`
`contains`, exact→`eq`). The QueryTag packet-id `0x6752` was recovered from a `.rdata`
packet-descriptor table (`{0x6751,1}`=StartTagQuery, `{0x6752,1}`=QueryTag) — no Ghidra needed.
Golden-byte + glob unit tests + gated live test. Full finding:
`docs/reverse-engineering/grpc-tag-query-odata.md`.
> ✅ **Milestone 0 (gRPC parity) is COMPLETE** — probe, system-param, metadata, and browse all run
> over `RemoteGrpc` and are live-verified against a real 2023 R2 server, alongside the read chain.
> ️ **Auth note (2026-06-21, resolved):** an apparent NTLM round-1 `SEC_E_LOGON_DENIED` blocker
> turned out to be a **test-harness credential-parsing bug**, not a server/account/SDK issue — the
> gitignored creds file stores **quoted** values (`"nam\user"`, `"pass"`), and the env-setup must
> **strip surrounding quotes** before exporting `HISTORIAN_USER`/`HISTORIAN_PASSWORD`. With quotes
> stripped, the domain account authenticates and the full read + system-param + probe chain passes
> live. The round-failure diagnostic added during the hunt is kept
> (`HistorianNativeHandshake.DescribeError` decodes the native error + hex/ASCII preview).
> ⚠️ **Live-verification constraint:** the local Historian is **2020** (WCF, port 32568) — the
> 2023 R2 gRPC endpoint (32565) is absent. M0's gRPC routing (R0.1R0.4) can be built and
> golden-byte/unit-tested here but **cannot be live-verified** without an actual 2023 R2 server.
> Treat gRPC ops as unverified until then; the byte payloads remain the proven 2020 protocol.
> 🔬 **M1a re-classification (2026-06-20).** Two "trivial" items were live-probed against the
> 2020 WCF server and found **not deliverable here**, both for evidence-backed reasons:
> - **R1.3 `GetServerTimeZoneAsync`** — `Status.GetSystemTimeZoneName` is a client-side *stub*
> on 2020 (rc=0, empty value), same family as `GetServerTime`. gRPC/2023R2-only.
> - **R1.1 `ExecuteSqlCommandAsync`** — `ExeC` returns native error 51 (InvalidParameter);
> the contract-3 string-handle ops require an unmapped native session/filter registration
> step (the `StartTagQuery` wall).
>
> Takeaway: the M1a "cheap surface" is *cheap only on the 2023 R2 gRPC front door*. On 2020 WCF
> the boundary is the **handle type** (see the string-handle wall note under §1b and
> `docs/reverse-engineering/wcf-string-handle-wall.md`): **`uint`-handle ops work, `string`-handle
> ops are blocked.** GETHI/GetTepByNm were probed and confirmed blocked (not, as first guessed,
> reachable). The reachable **`uint`-handle** items are now **DONE**: ~~R1.8/R1.9 StartQuery
> summary/state modes~~ (resolved = existing `ReadAggregateAsync`) and ~~R1.7 event filters~~
> (✅ 2026-06-20 — `ReadEventsAsync(…, HistorianEventFilter)`, live-honored). M2 event send is
> also done (✅ WCF `AddS2`). **R1.2 `GetRuntimeParameterAsync` is also done** (✅ 2026-06-20,
> `aa/Stat/GETRP`, live-verified) — notably a *string-handle* op that punches through the wall
> using the Open2 storage-session GUID as an **uppercase** string handle, which proved the
> GETHI/ExeC failures were a handle-*format* issue rather than a missing native registration.
> **Follow-up done:** R1.1 `ExecuteSqlCommandAsync` shipped; R1.5 extended-property read shipped
> (R1.6 collapsed into it — no distinct localized op). **R1.4 `GetHistorianInfo` bounded out on
> 2020 WCF** — GETHI there is a named-value query (only `HistorianVersion`); `EventStorageMode` is
> 2023R2-gRPC-only (see `wcf-historian-info.md`). Net: the **reachable 2020-WCF M1 read surface is
> complete**; what remains is config *writes* (M1c — gated on an explicit user request) and the
> gRPC/2023R2-only items (R1.3 timezone, R1.4 EventStorageMode — need a live 2023 R2 server).
>
> **Update 2026-06-21 (live 2023 R2 gRPC probe — both closed):** **R1.3 SHIPPED on gRPC** —
> `GetServerTimeZoneAsync` returns a real zone ("Eastern Daylight Time") via
> `StatusService.GetSystemTimeZoneName`; non-gRPC path fails closed
> (`ProtocolEvidenceMissingException`). **R1.4 bounded out on gRPC too** — `GetHistorianInfo` is
> named-value-only on the gRPC wire as well, `EventStorageMode` resolves under no name on either
> `GetHistorianInfo` or `GetSystemParameter`, and the 518-byte struct is C++-HCAL-internal (filled
> via native vtable+648, not the gRPC op). So **no gRPC/2023R2-specific reads remain open** — the
> entire M1 read surface (2020 WCF + 2023 R2 gRPC) is now closed.
## Guiding principles
1. **gRPC-first.** New ops go on the `RemoteGrpc` transport (clean protobuf envelope);
the inner `bytes` blob is the only thing to RE. Keep WCF as the legacy/Windows path.
2. **Two tests per op, always.** A golden-byte test (deterministic, no server) **and** a
gated live test (`HISTORIAN_GRPC_HOST` / `HISTORIAN_HOST`). No op is "done" without both.
3. **Version-pin, fail closed.** Read server version at connect; gate every byte
serializer on it; throw `ProtocolEvidenceMissingException` on mismatch — never
best-effort parse.
4. **Capture once, encode forever.** For CAPTURE-tier items, instrument one native call,
save a sanitized fixture under `fixtures/protocol/`, then implement against the fixture.
5. **Ship per milestone.** Each milestone is independently releasable.
Effort: **S** ≈ days · **M** ≈ ~1 week · **L** ≈ weeks. Estimates are incremental on
histsdk's existing infra (auth chain, transport, frame primitives, test harness).
---
## Milestone 0 — Foundation: full gRPC parity for the DONE surface (M)
*Goal: everything already working over WCF also works over `RemoteGrpc`, so the whole
read/browse/status surface is Windows-free and the gRPC stack is the default path.*
| ID | Work | gRPC op | Files | Verify | Effort |
|---|---|---|---|---|---|
| R0.1 | Route browse over gRPC | `Retrieval.StartTagQuery`/`QueryTag` or `GetTagInfosFromName` | `Grpc/HistorianGrpcReadOrchestrator` (+ new `…GrpcBrowseClient`), `Historian2020ProtocolDialect` | browse tags live over gRPC | S |
| R0.2 | Route tag metadata over gRPC | `Retrieval.GetTagInfosFromName` | dialect + grpc client | metadata matches WCF result | S |
| R0.3 | Route status/system-param over gRPC | `Status.GetSystemParameter`, `Status.GetHistorianConsoleStatus` | new `Grpc/HistorianGrpcStatusClient` | system param + conn status live | S |
| R0.4 | Probe over gRPC | `*.GetInterfaceVersion` | grpc clients | `ProbeAsync` Windows-free | XS |
| R0.5 | **Capture harness for gRPC payloads** | n/a | reuse `instrument-wcf-*` tooling (same byte blobs) + add a `grpc-call-dump` helper | dump any request/response `bytes` to a fixture | S |
| R0.6 | **Version gate** | server version at connect | `HistorianClientOptions`, orchestrators | mismatched version → throws | S |
**Acceptance:** the entire Phase-0 capability set runs end-to-end over `RemoteGrpc`
(incl. Linux), no WCF on the path. 188+ unit tests green; live gRPC integration suite green.
---
## Milestone 1 — Cheap surface completion (TRIVIAL/BOUNDED) (ML total)
*Goal: knock out the remaining read/config surface. Order = ascending payload difficulty.*
### 1a. Trivial (XSS each, no new payload format)
| ID | Capability | gRPC op | Notes |
|---|---|---|---|
| ~~R1.1~~ | ~~`ExecuteSqlCommandAsync`~~ | `Retrieval.ExecuteSqlCommand` (`ExeC`+`GetR`) | ✅ **DONE (2026-06-20), live-verified.** `ExecuteSqlCommandAsync(sql)``HistorianSqlResult` (columns + typed rows). String-handle op via the uppercase storage GUID. Chain: `Retr.GetV` prime → `ExeC(handle, sql, option=0, ref queryHandle)``GetR` loop (note: `GetR` returns **false even on success** — the stream is in `pResultBuff` regardless; false = final page). `GetR`'s `pResultBuff` is an **NRBF-serialized `DataTable`** (`SerializationFormat.Xml`: members `XmlSchema` + `XmlDiffGram`). BinaryFormatter is gone from .NET 10, so it's decoded read-only with `System.Formats.Nrbf` + `XDocument` (no BinaryFormatter). Shipped: `HistorianSqlResult`/`HistorianSqlColumn`/`HistorianSqlExecuteOption`, `HistorianSqlResultProtocol`, `HistorianWcfSqlClient`, golden `WcfSqlResultProtocolTests`, gated live tests. See `docs/reverse-engineering/wcf-exec-sql.md`. |
| ~~R1.2~~ | ~~`GetRuntimeParameterAsync`~~ | `Status.GetRuntimeParameter` (`aa/Stat/GETRP`) | ✅ **DONE (2026-06-20), live-verified.** Captured (`scripts/Capture-RuntimeParam.ps1`): GETRP is a **`string`-handle** op (GETHI's shape), but reachable from the managed client using the Open2 storage-session GUID as an **uppercase** string handle (`ToString("D").ToUpperInvariant()`). Returns `HistorianVersion` = `20,0,000,000` live. pRequestBuff = `54 67 01 00` + uint nameCount + per-name(uint charCount + UTF-16); pResponseBuff = version + uint resultCount + CRetVariant(`0x43` VT_BSTR + uint16 len + uint16 charCount + UTF-16). Single string-valued param only (multi-name framing inferred, not captured). Shipped: `HistorianClient.GetRuntimeParameterAsync(name)`; golden `WcfRuntimeParameterProtocolTests`. **Note:** GETRP punching through the string-handle wall with the uppercase storage GUID is a strong lead that GETHI/ExeC may be a handle-*format* issue — see `wcf-string-handle-wall.md` §Update. |
| ~~R1.3~~ | `GetServerTimeZoneAsync` | `Status.GetSystemTimeZoneName` | ✅ **DONE on gRPC (2026-06-21), LIVE-VERIFIED** against the real 2023 R2 server — returns `"Eastern Daylight Time"`. `HistorianClient.GetServerTimeZoneAsync` routes over `RemoteGrpc` (`HistorianGrpcStatusClient.GetSystemTimeZoneNameAsync`, `uiHandle`-in/string-out, no buffer). The 2020 WCF op stays a client-side stub (rc=0, empty), so the non-gRPC path **throws `ProtocolEvidenceMissingException`** (fail-closed) rather than return an empty string. Golden message-shape + non-gRPC guardrail unit tests + gated live test. (2020-only routes — per-block `HistoryBlock.TimeZoneOffset`, SQL via R1.1 — remain DST-specific and are not this op.) |
> ✅ **String-handle "wall" RESOLVED (2026-06-20) — it was a handle-FORMAT bug.** R1.4/R1.5/R1.6
> (and R1.1) take a **`string` GUID handle**; the earlier "code 1/51 blocked" verdict came from
> passing the Open2 storage GUID in .NET's default **lowercase**. Sent **uppercase**
> (`storageSessionId.ToString("D").ToUpperInvariant()`) the same handle works: **GETRP** (R1.2,
> shipped), **GETHI** (R1.4) and **ExeC** (R1.1) are all live-verified reachable, and **R1.5
> `GetTepByNm`** is now **shipped + live-verified** (`GetTagExtendedPropertiesAsync`). **R1.6 has no
> distinct op** (collapses into R1.5). Note: `QTB` (StartTagQuery) does **not** punch through — it
> fails *server-side* (`CMdServer::StartActiveTagnamesQuery` over the `aahMetadataServer` pipe),
> independent of handle format, so the index-based property/query paths stay blocked here. Full
> analysis: `docs/reverse-engineering/wcf-string-handle-wall.md` (RESOLVED banner) and
> `docs/reverse-engineering/wcf-tag-extended-properties.md`.
> R1.8/R1.9 (StartQuery summary/state modes) are `uint`-handle and were already reachable.
### 1b. Bounded (decode one `bytes` payload; SM each)
| ID | Capability | gRPC op | Payload to decode | Depends |
|---|---|---|---|---|
| ~~R1.4~~ | `GetHistorianInfoAsync` | `Status.GetHistorianInfo` (`GETHI`) | ⛔ **BOUNDED OUT — now confirmed on the 2023 R2 gRPC front door too (2026-06-21, live-probed).** The motivating field `EventStorageMode` is **not on the wire on either transport.** Live gRPC probe against the real 2023 R2 server: `GetHistorianInfo` is a **named-value** query exactly like 2020 WCF — only `HistorianVersion` resolves (→ `"23,1,000,000"` + `02 00 01 00` trailer); `EventStorageMode` + 7 name variants fail (`success=false`) on **both** `GetHistorianInfo` **and** `GetSystemParameter`. The 518-byte `HISTORIAN_INFO` struct (mode@514) is the **C++ HCAL in-memory model** (managed `HistorianAccess.GetHistorianInfo` fills it via a native **vtable+648** call, not the gRPC op — verified in the 2023 R2 decompile), derived outside the wire. The only wire-reachable field (version) is already shipped (`ProbeAsync`/`GetSystemParameterAsync`/`GetRuntimeParameterAsync`), so a struct API would be hollow + misleading. **Closes the prior "build against a live 2023 R2 server" caveat — done, and there is nothing to ship.** See `docs/reverse-engineering/wcf-historian-info.md`. | uppercase string handle |
| ~~R1.5~~ | Extended-property **read** | `Retrieval.GetTagExtendedPropertiesFromName` (`GetTepByNm`) | ✅ **DONE (2026-06-20), live-verified.** `GetTagExtendedPropertiesAsync(tag)` → name/value pairs. String-handle op via the uppercase storage GUID; name-based path (`GetTagExtendedPropertiesByName`, not the QTB-gated TagQuery path). Request `tagNames` = `uint count` + per-name(`uint charCount`+UTF-16); response = `uint tagCount` + per-tag(marker + compact-ASCII name + `uint propCount` + per-prop(marker + compact-ASCII name + `0x43` VT_BSTR value) + trailer). Sequence-paged. Shipped: `HistorianTagExtendedPropertyProtocol`, golden `WcfTagExtendedPropertyProtocolTests`, gated live test. See `docs/reverse-engineering/wcf-tag-extended-properties.md`. | uppercase string handle |
| ~~R1.6~~ | Localized-property **read** | (no op) | ⛔ **No distinct op on 2020 — collapses into R1.5.** There is no `GetTagLocalizedPropertiesFromName`/`GetTlpByNm` or `GetTagLocalizedPropertiesByName` in `current/aahClientManaged.dll`; the only "localized" surfaces are error-message/UI-text localization. Extended properties (R1.5) are the user-defined tag-property read surface. Closed, not throwing. | — |
| ~~R1.7~~ | Event **filters** | filter bytes in `Retrieval.StartEventQuery` | ✅ **DONE (2026-06-20), live-honored.** `ReadEventsAsync(start, end, HistorianEventFilter)`. The filter rides `StartEventQuery`'s `pRequestBuff` (captured via `EventQuery.AddEventFilter` + instrument-wcf-writemessage; Equal vs Contains diffed to isolate the op). Filter block: `ushort 0 + uint filterCount + uint condCount + uint nameLen + name(UTF-16) + uint 1 + ushort op + uint 1 + value(0x09-len-0x00 compact-ASCII) + byte 0`. **REAL, not inert** (a non-matching predicate returns 0 events; matching returns the subset). Single string-valued predicate only; multi-filter (OR) / multi-condition (AND via `AddEventFilterCondition`) framing not yet fully captured. See `HistorianEventFilter`, golden `WcfEventQueryProtocolTests`. | — |
| ~~R1.8~~ | Analog-summary query | `Retrieval.StartQuery` (summary mode) | ✅ **RESOLVED (2026-06-21) — no new code; == existing `ReadAggregateAsync`.** Request + response both captured (`scripts/Capture-SummaryRequest.ps1 -WithResponse`): the `GetNextQueryResultBuffer2` response is the **ordinary version-9 row buffer** the raw/aggregate parser already handles (decoded 7 rows = SQL ground truth exactly). There is **no rich `CAnalogSummaryValue` struct on the wire** — each row carries a *single* value selected by `RetrievalMode`/QueryType (Integral→8, TimeWeightedAverage→5, …), not an all-aggregates-in-one row; `ValueSelector`/`AggregationType`/`MaxStates` are **inert** on the WCF retrieval path (they configure the SQL provider, not this query). The all-aggregates-at-once shape is the SQL/OLEDB provider's, or the gRPC front door — not 2020 WCF binary. Plan + capture evidence: [`r1.8-r1.9-summary-queries.md`](r1.8-r1.9-summary-queries.md). | — |
| ~~R1.9~~ | State-summary query | `Retrieval.StartQuery` (state mode) | ✅ **RESOLVED (2026-06-21) — same finding as R1.8.** State-summary is the **same `StartQuery2` request** (only `MaxStates`/defaults differ on the wire); the response carries no distinct `CStateSummaryStruct` on the 2020 WCF binary path. Covered by the existing aggregate read; no new `src/` code warranted. Plan: [`r1.8-r1.9-summary-queries.md`](r1.8-r1.9-summary-queries.md). | — |
### 1c. Bounded config writes (SM each)
| ID | Capability | gRPC op | Payload | Notes |
|---|---|---|---|---|
| R1.10 | `RenameTagsAsync` | History rename op | rename request buffer | `AllowRenameTags` already probed |
| ~~R1.11~~ | Extended-property **write** | `History.AddTagExtendedProperties` (AddTEx) | ✅ **Add DONE (2026-06-21), live-verified.** `AddTagExtendedPropertiesAsync`/`AddTagExtendedPropertyAsync` (write mode, uppercase handle). inBuff = exact inverse of the R1.5 read framing (`uint32 groupCount + 0x01 + compact-ASCII tag + uint32 propCount + per prop[0x02 + compact-ASCII name + 0x43 VT_BSTR value] + 0x01 trailer + 0x00 terminator`); the trailing `0x00` is required or the server throws. Golden `WcfTagExtendedPropertyWriteProtocolTests` + gated live write/read-back test. **Delete (DelTep): wire format CAPTURED + serializer golden-proven (2026-06-21), but live delete is server-blocked and NOT shipped.** Captured via a two-session trick (add in Run A → fresh-session read-sync → delete in Run B, past the native err-229 client gate); inBuff = same group framing as Add but property-name-only and a `0x00` group trailer. A decisive experiment shows SDK-added properties ARE deletable (the native client deletes one), so SDK-add is complete; the SDK's own DelTep is rejected (`SErrorException` in `CHistStorage::DeleteTagExtendedProperties`) despite matching mode/handle/inBuff + GetTgByNm/GetTepByNm prime + open channel + 60s retries. Root cause: the native multiplexes services over ONE connection (per-connection working set), which the SDK's per-service WCF channels don't reproduce — needs transport-level multiplexing. See `docs/reverse-engineering/wcf-add-tag-extended-properties.md` §Delete. |
| ~~R1.12~~ | Localized-property **write** | (no op) | ⛔ **No distinct op on 2020 — closed (mirror of R1.6).** A symbol sweep of `current/*.dll` finds no `AddTagLocalizedProperties` / `DeleteTagLocalizedProperties` / any `*LocalizedPropert*` / `TagLocalized*`; only UI/error-text localization (`GetLocalizedText`/`GetLocalizedMessage`/`LocalizedResourcesDir`). Localized properties are a 2023 R2/gRPC concept. Closed, not throwing. See `docs/reverse-engineering/wcf-tag-extended-properties.md` §R1.12. | 2026-06-21 |
| ~~R1.13~~ | Non-analog tag create (string/discrete) | `History.EnsureTags` | distinct CTagMetadata variant | ⛔ **GATED — bounded out (2026-06-21, live-probed).** Native `AddTag` rejects every non-analog type **client-side** (`ErrorCode=ValidationFailed` / "Transaction validation failed", before any WCF op): SingleByteString, DoubleByteString, **and Int1** all fail; Float (control) succeeds. The native `HistorianDataType` enum has **no Discrete/Boolean** and no Int8/UInt8 (SDK-only extensions); `HistorianTag` has **no TagType setter** (type is data-type-derived). So no non-analog wire request is ever emitted → nothing to capture/implement. String/discrete create goes via a different subsystem (config editor / SQL), not this client's AddTag. `EnsureTagAsync` stays analog-only. See `docs/reverse-engineering/wcf-non-analog-tag-create.md`. |
**Acceptance:** read + browse + metadata + system/status + property R/W + summaries +
event-filtered reads + rename all live-verified over gRPC.
---
## Milestone 2 — Event sending (CAPTURE) (SM) ← headline gap
*Goal: `SendEventAsync(HistorianEvent)`. Path fully mapped in histevents.md; one capture away.*
> ✅ **DONE (2026-06-20) — `HistorianClient.SendEventAsync(HistorianEvent)` shipped and
> live-accepted over 2020 WCF.** The headline assumption — that event delivery would ride the
> non-WCF storage-engine pipe (and so be blocked like revision writes) — was **disproved by
> capture**: a native `AddStreamedValue(HistorianEvent)` leaves over WCF as **`AddS2`
> (`IHistoryServiceContract2.AddStreamValues2`)**. CM_EVENT is a built-in registered tag, so the
> `129 TagNotFoundInCache` gate that blocks `AddS2` for user tags does **not** apply to events.
> The full managed chain (Open2 event-mode **0x501** → CM_EVENT RTag2/EnsT2 → AddS2) is accepted
> by the server (`AddS2` returns success, empty error buffer). See the event-send field map under
> §"Event-send wire format" in `histevents.md` and `HistorianEventWriteProtocol`.
>
> ⚠️ **Persistence caveat (environment, not SDK):** on the local dev Historian, accepted events
> are **not persisted** to the queryable store (`v_AlarmEventHistory2` latest stays at the
> pre-test date; count only ages down). The **native** client exhibits the identical behaviour
> (its `AddS2` also returns success but nothing lands), so this is the box's event-ingestion
> pipeline not being active — not an SDK protocol gap. The SDK emits byte-equivalent `AddS2`
> (golden-tested). Full send→store→read-back round-trip awaits a Historian with an active event
> storage pipeline.
| ID | Work | Status |
|---|---|---|
| R2.1 | Capture the event value blob | ✅ `scripts/Capture-EventSend.ps1` (event-send harness scenario + instrument-wcf-{write,read}message); two captures diffed to separate constant framing from value fields. Decisive finding: event-send = WCF `AddS2`, not storage pipe. |
| R2.2 | `HistorianEventWriteProtocol` | ✅ Serializes the `AddS2` pBuf (storage sample buffer wrapping the event VTQ): "OS" sig + sampleCount + length fields + CM_EVENT tag id + EventTime FILETIME + OpcQuality + opaque descriptor + event Id + ReceivedTime FILETIME + Namespace + EventType + version + typed property bag (string props reuse the read parser's `0x43` encoding). Golden-byte test pins capture A. |
| R2.3 | Event write orchestrator | ✅ `HistorianWcfEventOrchestrator.SendEventAsync`: Open2 (0x501) → reuse CM_EVENT RTag2/EnsT2 registration → `AddStreamValues2(handle, pBuf, out err)` on the same /Hist channel + storage-session handle. |
| R2.4 | Public API | ✅ `HistorianClient.SendEventAsync(HistorianEvent)`. Original events only (RevisionVersion=0) with string-valued properties; other property types + revision/update/delete throw `ProtocolEvidenceMissingException` until captured. |
| R2.5 | Round-trip test | ✅ Golden-byte on R2.2 + gated live test `SendEventAsync_AgainstLocalHistorian_AcceptedByServer` (asserts server acceptance; SQL read-back best-effort given the persistence caveat). |
**Acceptance:** an event sent from histsdk is accepted by the historian over WCF with a
byte-correct `AddS2` (✅). Appears-and-reads-back is environment-gated on event persistence (see caveat).
---
## Milestone 3 — Historical / non-streamed value writes (BOUNDED) (M)
*Goal: insert original historical VTQs (backfill), the path that is NOT the gated cache push.*
> ✅ **gRPC UNLOCK (2026-06-21, LIVE-VERIFIED): the transaction lifecycle is REACHABLE over the
> 2023 R2 gRPC front door.** The `grpc-revision-probe` opened a **write-enabled** (`0x401`) gRPC
> session and drove `TransactionService.AddNonStreamValuesBegin(storage-GUID **uppercase**)` →
> real `strTransactionId` → `AddNonStreamValuesEnd(bCommit=false)` (discarded, no data written).
> Where 2020 WCF returns `UnknownClient (51)`, the gRPC `TransactionService` is itself the gateway
> to the storage engine, so the Open2 session GUID is accepted directly — **no legacy pipe**. This
> answers the M3-over-gRPC question below: **yes**, the non-streamed *original* write transaction is
> reachable from the pure-managed SDK. **Not yet shipped:** the `AddNonStreamValues` `btInput` VTQ
> buffer must be captured before any value-commit (never guess wire bytes); revision *edits* (R4.2)
> remain pipe-only even on gRPC. Full detail + decompile basis:
> [`revision-write-path.md`](revision-write-path.md) §"2023 R2 gRPC — the wall is gone".
>
> ⛔ **BLOCKED on 2020 WCF — re-confirmed by the D2 probe (2026-05-05), see
> [`revision-write-path.md`](revision-write-path.md).** The premise above ("the path that is NOT
> the gated cache push") was **disproved** *on WCF*: R3.1's op
> (`Transaction.AddNonStreamValuesBegin/AddNonStreamValues/End`) is the **same**
> `ITransactionServiceContract2.AddNonStreamValuesBegin2` D2 probed, and over WCF it returns
> `04 33 00 00 00` = `UnknownClient (51)` for every handle format **and** the full priming chain
> (Stat/Hist/Retr/Trx GetV + UpdC3 + 6× GetSystemParameter + RTag2). Root cause (IL-walk:
> `CClient.TransactionBegin` → `CHistStorageConnection.StartTransaction` →
> `CStorageEngineConsoleClient.StartTransaction`): the real transaction rides a **shared-memory +
> named-pipe** channel (`STransactPipeClient2` + `SCrtMemFile`) to `aaStorageEngine.exe`, separate
> from WCF. The WCF Trx op is a server-side **relay** that requires a pre-existing storage-engine
> pipe session, which no WCF op can establish. So **M3 over 2020 WCF is unimplementable as a
> pure-managed SDK** — same architectural wall as R4.2 (revisions) and the `AddS2` cache gate.
>
> **Only remaining lever:** the **2023 R2 gRPC front door** (HCAL-native, no legacy storage-engine
> pipe). Whether the gRPC services expose a non-streamed/revision write that bypasses the pipe is
> **untested** — it needs the live 2023 R2 server + a native gRPC capture of the write op, then
> decode/implement. Treat as on-demand (no current demand signal); the WCF path is closed.
| ID | Work | gRPC op | Status |
|---|---|---|---|
| R3.1 | Decode non-streamed VTQ packet | `History.AddStreamValues` ("ON" buffer) + `EnsureTags` | ✅ **CAPTURED + VALIDATED 2026-06-21.** Drove the native 2023 R2 client through a committed historical write (sandbox tag) with the IL-rewritten gRPC client dumping every `byte[]`; the value **read back over gRPC**. The path is **NOT** `AddNonStreamValues`/TransactionService — it's **`HistoryService.AddStreamValues`** with an **"ON" storage-sample buffer** (AddS2 "OS" family) + `EnsureTags`. Buffer decoded: `"ON"(0x4E4F) + u16 count + u32 totalLen + u16 payloadLen + 16B tag GUID + FILETIME + u16 quality + u32 type + FILETIME + 8B double`. D2 cache gate does NOT block the primed 2023 R2 client. See [`revision-write-path.md`](revision-write-path.md) §"R3.1 CAPTURED". |
| R3.2 | `AddHistoricalValuesAsync` | `History.AddStreamValues` ("ON") + `EnsureTags` | ✅ **SHIPPED + LIVE-VALIDATED 2026-06-21.** `HistorianClient.AddHistoricalValuesAsync(tag, values)` over `RemoteGrpc`: write-enabled session → `GetTagInfosFromName` (resolves the per-tag GUID = tag-info `TypeId`) → `HistoryService.AddStreamValues` ("ON" buffer, golden-tested). The pure-managed SDK wrote a value and read it back live. All five analog types captured + golden-tested + live write/read-back validated — **Float/Double/Int2/Int4/UInt4** (value = `u32(0) + native-width value`, descriptor `C0 10 01 00` constant; width selected from the tag's declared type); other types throw. gRPC-only (non-gRPC throws). |
| R3.3 | Ingest-permission validation | confirm the target accepts original-data insert (distinct from `AddS2` cache wall) | ✅ **confirmed** — the D2/AddS2 cache gate (err 129) does NOT block the primed 2023 R2 client; the historical write commits and reads back |
**Acceptance:** historical points inserted and read back. **WCF path closed (D2).** gRPC path:
**transaction lifecycle proven (Begin/End live) + full sequence mapped**; the remaining insert is a
focused follow-up — reproduce `StorageService.OpenStorageConnection` (+ `RegisterTags`), then decode
the `btInput` VTQ buffer, each a live-production probe loop.
---
## Milestone 4 — HARD subsystems (deferred / optional) (L each)
Only if the use case demands them. Each is a real subsystem, not an op.
| ID | Capability | Approach | Risk |
|---|---|---|---|
| R4.1 | Store-and-forward | ✅ **SHIPPED (2026-06-21) — pragmatic durable outbox.** `AVEVA.Historian.Client.StoreForward`: `HistorianStoreForwardWriter` buffers historical-value + event writes to an `IHistorianOutboxStore` (`FileHistorianOutboxStore` = crash-durable atomic JSON-per-entry, FIFO by filename sequence, corrupt-file quarantine; `InMemoryHistorianOutboxStore` for tests) and replays them through an `IHistorianWriteSink` (default `HistorianClientWriteSink`). Background drain loop retries on reconnect; FIFO head-of-line blocking with optional `MaxDeliveryAttempts` dead-lettering; `DropOldest`/`Reject` overflow policy; `GetStatusAsync` snapshot (Pending/Storing/ErrorOccurred mirrors the server SF semantics). 12 unit tests (durability-across-restart, reconnect-drain, head-of-line order, dead-letter, overflow, background loop). **NOT** the bit-faithful native SF cache (`Forward*Snapshot` decode) — that stays deferred; pure client-side, no RE. | high; consider "good enough" |
| R4.2 | Revision / edit writes | `AddRevisionValue(s)` go via the **non-WCF storage-engine pipe** (`STransactPipeClient2`) — separate transport RE | high |
| R4.3 | Real store-forward **status** | ⚠️ **PARTIAL — measured idle-state SHIPPED (2026-06-21, gRPC); active-SF magnitude D2-blocked.** Re-scoped against the recovered 2023 R2 gRPC contract (the old "duplex push vs pull" risk is gone — `StorageService` exposes SF state as plain *pull* RPCs). Idle-baseline probe (`grpc-sf-status-probe`) against the live 2023 R2 server resolved the open handle question: the direct SF pull RPCs (`GetSFParameter` / `GetRemainingSnapshotsSize`) require the `OpenStorageConnection` storage-engine **console handle** and are **D2-gated** (same wall as R4.2 revisions), so `Storing`/`Pending`/`DataStored` magnitude is unreachable from a pure managed client. But `StatusService.GetHistorianConsoleStatus` IS reachable on the session string handle, so `GetStoreForwardStatusAsync` over gRPC now returns a **measured** idle-state — it actually contacts the server and reports `ErrorOccurred` when unreachable (vs the old blind all-false synthesis), live-verified + gated test. Non-gRPC keeps the synthesized fallback. Active-SF magnitude (path b) stays deferred behind D2 + needs an invasive force-SF capture to decode the console-status enum. See `docs/plans/store-forward-cache-reverse-engineering.md` §9. | medium (idle done; magnitude D2-blocked) |
| R4.4 | Multi-historian / redundancy | ✅ **SHIPPED (2026-06-21) — client-side orchestration.** `AVEVA.Historian.Client.Redundancy`: `HistorianRedundantClient` fronts N `IHistorianMember`s (default `HistorianClientMember` over `HistorianClient`) as one logical client. Reads fail over to the next member in priority order — streaming reads only fail over *before the first row* (mid-stream failures propagate to avoid dup/gap); writes fan out (`AllMembers`/`PreferredOnly`) with `All`/`Any` ack policy returning a per-member `HistorianRedundantWriteResult`. Per-member health (`FailureThreshold` demotion) + background watchdog (`CheckHealthAsync`/`PeriodicTimer`) restores recovered members; `GetStatus()` snapshot. Composes with R4.1: back a member's writes with a `HistorianStoreForwardWriter` for the pragmatic ReSyncTags equivalent (down member buffers + replays). 14 unit tests (failover order, mid-stream no-failover, ack policies, fanout modes, watchdog recovery, all-fail aggregation). Pure client-side, no server-side redundancy protocol, no RE. | medium |
---
## Won't-do from the client (GATED)
- **Streaming process-sample writes** (`AddStreamedValue(HistorianDataValue)` / `AddS2`):
runtime cache only ingests from configured IOServer/AppServer pipelines. Confirm your
ingestion architecture instead of pursuing this.
---
## Cross-cutting workstreams (run alongside all milestones)
- **CW-1 Capture tooling** (enables R0.5, R1.x, R2.1): one reusable "call op → dump
request/response `bytes` → sanitized fixture" path. Highest leverage — do first.
- **CW-2 Version compatibility:** matrix of tested Historian versions; serializers keyed
by version; CI gate.
- **CW-3 Cross-platform CI:** run the gRPC suite on Linux/macOS (transport is portable;
explicit-cred auth path).
- **CW-4 Fixtures discipline:** every new op ships a `fixtures/protocol/<op>/` golden file;
sanitize hostnames/tags/GUIDs before commit.
- **CW-5 Public API shape:** keep the modern surface (async, `IAsyncEnumerable`,
cancellation, options record, DI-friendly) consistent as the surface grows.
---
## Sequencing (critical path)
```
CW-1 capture tooling ─┐
M0 gRPC parity ───────┼─→ M1 cheap surface ─→ M2 event send ─→ M3 historical writes ─→ (M4 optional)
R0.6 version gate ────┘
```
Recommended first sprint: **CW-1 + M0 (R0.1R0.6)** → a fully Windows-free, version-safe
gRPC client at today's capability. Second sprint: **M1a + M2** (cheap wins + the headline
event-send). M3/M4 as demand dictates.
> **Status 2026-06-21:** sprints 1 + 2 are **complete** (M0 gRPC parity, the reachable M1 surface,
> and M2 event-send all shipped + live-verified; remaining M1 items are evidence-bounded-out). The
> reachable surface on the **available 2020 WCF infrastructure is exhausted**. **M3 update
> (2026-06-21):** with the live 2023 R2 server, the **M3 non-streamed write transaction is now
> proven reachable over gRPC** — `TransactionService.AddNonStreamValuesBegin/End` round-trips live
> (the D2 storage-engine-pipe wall is WCF-only). The remaining M3 work is bounded and concrete:
> capture the `AddNonStreamValues` `btInput` VTQ buffer → golden-tested serializer → real
> commit+read-back → public `AddHistoricalValuesAsync`. The other levers are unchanged: R4.2 revision
> *edits* stay pipe-only even on gRPC, and M4 (SF / redundancy) is a HARD deferred subsystem.
>
> **M4 update (2026-06-21):** R4.1 store-and-forward, R4.4 redundancy, and R4.3 *measured idle-state*
> SF status are all SHIPPED (pragmatic, client-side). What remains deferred sits behind the **D2
> storage-engine console-pipe wall**: R4.2 revision edits and the R4.3 *active-SF magnitude*
> (`Storing`/`Pending`/`DataStored`) — the SF pull RPCs that carry it need the console handle the
> managed client can't obtain. Decoding the active-SF console-status enum additionally needs an
> invasive force-SF capture on a sacrificial Historian.
## One-glance status
| Milestone | Tier | Effort | Value | When |
|---|---|---|---|---|
| M0 gRPC parity + capture tooling | foundation | M | unblocks everything, Windows-free | ✅ **done** |
| M1 cheap surface | TRIVIAL/BOUNDED | ML | most remaining read/config | ✅ **done** (reachable surface; rest bounded out) |
| M2 event send | CAPTURE | SM | headline write capability | ✅ **done** |
| M3 historical writes | BOUNDED | M | backfill | ✅ **SHIPPED + LIVE-VALIDATED (2026-06-21)**`AddHistoricalValuesAsync` over gRPC = `HistoryService.AddStreamValues` ("ON" buffer) + tag-GUID resolve. Pure-managed SDK write read back live. All 5 analog types (Float/Double/Int2/Int4/UInt4). WCF still blocked (D2) |
| M4 SF / revisions / redundancy | HARD | L×N | parity completeness | **R4.1 store-and-forward + R4.4 redundancy + R4.3 measured idle-state SF status SHIPPED** (client-side, 2026-06-21); R4.2 revisions + R4.3 active-SF magnitude deferred behind the same D2 storage-engine-pipe wall (R4.3 magnitude also needs an SF-active server capture) |
+338
View File
@@ -0,0 +1,338 @@
# How a HistorianEvent reaches the Historian DB files
Living analysis doc. Traces an event end-to-end: client API → wire → server
storage backend (SQL **Database** vs history **Blocks** `.dat`) → read-back.
Evidence base: 2023 R2 `aahClientManaged.dll` (decompiled `ArchestrA.HistorianAccess`,
`HistorianEvent`, `HistorianEventPropertyType`), native `aahStorage.exe` (string
analysis), the recovered gRPC + CloudHistorian contracts, and the histsdk read-side
reverse-engineering (CM_EVENT registration + event-row parser).
Status legend: ✅ proven (from binary) · 🔶 strong inference · ❓ open.
---
## TL;DR
An event is **not a distinct wire message**. The client turns each `HistorianEvent`
into a `HistorianDataValue` of type `Event` against the built-in **`CM_EVENT`** tag,
marshals it into a native VTQ, and **streams it like any tag value** on a dedicated
*Event* connection. Events are batched into an opaque serialized **event data packet**
and delivered (with their own store-and-forward "event snapshot" path). On the server
they are persisted into **one of two backends, chosen by a server-configured
`EventStorageMode`**: a SQL **Database**, or the history **Blocks** (`.dat`) files.
```
HistorianEvent (Type + typed property bag)
└─(AddStreamedValue)→ HistorianDataValue{Type=Event, TagKey=CM_EVENT, EventTime, Q=192}
└─ HistorianEvent.PackToVtq → CCommonArchestraEventValue::PackToVtq (native value blob)
└─ HISTORIAN_VALUE2 (44B; blob ptr @+33) → HistorianClient.AddHistorianValue → queue
└─ flush → EnqueueEventDataPacket{ byte[] SerializedBytes } (batched VTQs)
└─ SERVER: aahEventStorage.exe (InSQLEventSystem)
per-client event-tag pipeline → recovery-log WAL → backend:
• Blocks → elastic snapshot → frozen → history .dat (Circular/Permanent)
• Database → ArchestrAEvents.EventStorage.Contract assembly → SQL (A2ALMDB)
→ EventReplication (redundant historians)
└─ (offline) → store-and-forward → ForwardEventSnapshotBegin/…/End on reconnect
read-back: Retr.StartEventQuery / SQL provider views (Events, v_AlarmEventHistory2, v_EventSnapshot)
```
---
## 1. The `HistorianEvent` object ✅
Decompiled `ArchestrA.HistorianEvent` — a structured header plus a **typed property bag**:
- **Header fields:** `ID`/`Id` (Guid), `Type`/`EventType` (string, e.g. `"Alarm.Set"`,
`"User.Write"`), `EventTime` (DateTime), `ReceivedTime`, `Severity` (ushort),
`Priority` (ushort), `IsAlarm`, `IsSilenced`, `System`, `Source`, `Source_Name`,
`Area`, `Namespace`, `DisplayText`.
- **Revision fields:** `RevisionVersion` (ushort), `Delete` (bool), `Update` (bool) —
events are revisable (see §4 UpdateEventStatus).
- **Property bag:** `AddProperty(name, value, HistorianEventPropertyType, …)` with typed
overloads. `HistorianEventPropertyType` (alphabetical enum):
`Blob, Boolean, Byte, Date, DateTime, Decimal, Double, Duration, Float, Guid, Hex,
Int, Integer, Long, Short, String, Time, UnsignedByte, UnsignedInt, UnsignedLong,
UnsignedShort, Undefined`.
These map onto the wire property-bag the histsdk **read** parser already decodes
(`HistorianEventRowProtocol`): typemarkers `0x02` Boolean, `0x10` Guid, `0x18` FILETIME,
`0x31` Int32, `0x43` UTF-16 string, … — i.e. the write enum and the read typemarkers are
two views of the same typed-value format. The event-send serialization is the inverse of
that read parser.
---
## 2. Client send path — an event becomes a streamed VTQ ✅
From decompiled `ArchestrA.HistorianAccess` (line refs into the decompile):
1. **Open an Event connection.** `HistorianConnectionArgs.ConnectionType =
HistorianConnectionType.Event`, `ReadOnly = false` (sample `Step10.SendEvents`).
2. **Default event tag.** `CreateDefaultEventTag()` (`:3006`) registers tag `CM_EVENT` /
"AnE Event" / `TagDataType = Event` and stores `eventTagHandle`. Same CM_EVENT
registration histsdk reverse-engineered (RTag2 + EnsT2; tag id
`353b8145-5df0-4d46-a253-871aef49b321`).
3. **Wrap as VTQ.** `AddStreamedValue(HistorianEvent)` (`:3123`):
```csharp
historianDataValue.objValue = historianEvent; // header + property bag
historianDataValue.DataValueType = HistorianDataType.Event;
historianDataValue.TagKey = eventTagHandle; // CM_EVENT
historianDataValue.StartDateTime = historianEvent.EventTime;
historianDataValue.OpcQuality = 192;
return AddStreamedValue((ConnectionIndex)1, historianDataValue, false, out error); // 1=Event
```
4. **Marshal + queue.** The private `AddStreamedValue` (`:3173`):
- builds a 44-byte native `HISTORIAN_VALUE2` (`InitBlockUnaligned(…,0,44)`),
- `HistorianAccessUtil.ConvertManagedStructToUnmanagedStruct(value, &HV2, bVersioned…)`
— its `case HistorianDataType.Event` (`HistorianAccessUtil:89`) calls
**`HistorianEvent.PackToVtq(out byte[])`** to produce the event value blob, whose pointer
is placed at `HISTORIAN_VALUE2+33` (freed after send; offset 33 is the value-union pointer
used for Event/String types),
- `HistorianClient.AddHistorianValue(client, &HV2, &err)` (`:3209`) queues the VTQ into
the native delivery buffer and returns immediately.
So an event uses the **same streaming machinery as a process value**; only `DataValueType`
(`Event`) and the target tag (`CM_EVENT`) differ.
### 2a. Event value serialization — `HistorianEvent.PackToVtq` ✅/🔶
`HistorianEvent.PackToVtq` (`HistorianEvent:1392`) populates a native
**`CCommonArchestraEventStruct`** then hands it to the **native** packer
`CCommonArchestraEventValue::PackToVtq(…, 192, 192, vtq)` (Q=192), associated with the
built-in `EVENT_TAGID` / `EVENT_TAGNAME` (`CTagMetadata.CommonArchestraEvent`). The actual
byte layout is produced in C++ — **not visible in managed code** — so pinning exact write
bytes needs a wire/IL capture, exactly as the read side did. But the **field set + order**
the managed code writes into the struct is now known:
```
SetReceivedTime (uint64 FILETIME, from UniqueTime.GetUniqueFileTime — unique/monotonic)
SetEventType (wchar* string, e.g. "Alarm.Set")
SetEventTime (uint64 FILETIME, from EventTime)
SetId (GUID)
SetRevisionVersion (uint16)
SetIsUpdate (bool) ← revision flags
SetIsDelete (bool)
Namespace (string, trimmed, non-printable-validated)
…then the typed property bag: Dictionary<string, Tuple<HistorianEventPropertyType, object>>
```
This matches `HistorianEventRowProtocol` on read: the property bag is name→(type,value) with
the same typed-value encoding (typemarkers `0x02/0x10/0x18/0x31/0x43/…`). So a managed
event-send serializer is tractable: emit the header struct fields above, then the typed
property bag in the read parser's format. The remaining unknown is only the exact native
framing offsets — best obtained by capturing one `PackToVtq` output, then golden-byte testing.
---
## 3. Event transport / delivery pipeline ✅ (CloudHistorian + gRPC contracts)
Events have a **dedicated, batched** connection + delivery pipeline, distinct from tag data
but structurally parallel:
| Stage | Event op | Tag-data analogue |
|---|---|---|
| Open connection | `OpenEventConnection2 { byte[] ClientInfo } → { byte[] ServerInfo }` | OpenConnection |
| Send batch | `EnqueueEventDataPacket { byte[] SerializedBytes }` | `EnqueueTagDataPacket { byte[] SerializedBytes }` |
| Store-and-forward | `ForwardEventSnapshotBegin / ForwardEventSnapshot / ForwardEventSnapshotEnd` | `ForwardSnapshot…` |
| Revise | `UpdateEventStatus` | (revision write) |
Key point: the **event data packet is an opaque serialized byte buffer** (`SerializedBytes`,
DataMember `d`) — the queued event VTQs batched together, exactly the same envelope shape as
the tag data packet. On-prem this is what the storage-streaming op (`AddStreamValues`)
carries; in the cloud variant it is `EnqueueEventDataPacket`.
Validation surfaced via error codes: `InvalidAlarmEventPropertyLength=212`,
`AlarmEventPropertyHasNonPrintableChar=214`, `AlarmEventPropertyHasInvalidSpecialChar=215`,
`AlarmEventPropertyNameIsAReservedName=216` — the server validates alarm/event property
names + values on ingest.
Offline → events spool to the **store-and-forward** cache and replay as **event snapshots**
(`ForwardEventSnapshot*`) on reconnect — a separate SF stream from tag-data snapshots.
---
## 4. Revisions / updates ✅
`HistorianEvent.Update` / `.Delete` / `.RevisionVersion` + the contract's
`UpdateEventStatus` op mean events are not write-once: an event can be re-sent to update or
delete a previously stored event (e.g. alarm acknowledge/clear), bumping `RevisionVersion`.
---
## 5. The storage-backend switch — `EventStorageMode` ✅
The client reads the server's event-storage backend from `HISTORIAN_INFO` **byte offset 514**
(`HistorianAccess` `:5715`):
```csharp
EventStorageMode = (info[514] == -1) ? Unsupported
: (info[514] == 0) ? Database // SQL Server
: Blocks; // history .dat blocks
```
`HistorianEventStorageMode ∈ { Database, Blocks, Unsupported }`. The destination is a
**server** decision; the client streams the same VTQ regardless.
---
## 6. Server side — where it lands ✅ (confirmed on the live local install)
The server-side event component is **`aahEventStorage.exe`** (service `InSQLEventSystem`,
"AVEVA Historian Event System"; plus `aahEventSvc.exe`), at
`…\Wonderware\Historian\x64\aahEventStorage.exe`. Its string table maps the full pipeline:
```
event packets / forwarded snapshots → per-client "event tag pipeline" → batch enqueue
→ Event Storage Recovery Log (WAL; "enqueuing N events to log",
path SystemParameter EventStorageLogPath = C:\Historian\Data\Logs\EventStorage)
→ persist to the active backend:
Block Storage ("Enabled Block Storage for events") → history .dat blocks
Database ("storing N events in database") → SQL via loaded managed
assembly ArchestrAEvents.EventStorage.Contract.EventStorageDatabaseConnection
(e.g. ";Initial Catalog=A2ALMDB;Integrated Security=true;Encrypt=True;…")
→ also fed to EventReplication (aahReplication.exe) for redundant historians
```
So persistence is **pluggable** (a loadable connection assembly) and dual-mode, guarded by a
recovery log. Which backend is live depends on configuration (the `EventStorageMode` of §5).
### This historian = **Block storage** (verified)
- `C:\Historian\Data\Circular` holds **527 `.dat` history blocks** (`Permanent` empty); the
EventStorage recovery log dir exists. `aahEventStorage` logs `"Enabled Block Storage for
events"`.
- SDK-shape alarm/events are present and retrievable: `Runtime.dbo.v_AlarmEventHistory2`
returns 224 rows over the last 30 days.
- `A2ALMDB` (the System-Platform alarm DB the connection string references) is **not present**
here — that path is only used when integrated with AVEVA System Platform alarming. Absent
it, ArchestrA events land in **blocks**, exactly as `aahStorage.exe` advertises (`"Stores
ArchestrA Event Data"`, snapshot→block).
### The SQL surface is **provider-backed views, not physical tables** ✅
In `Runtime`, the rich event objects are **views with NULL `OBJECT_DEFINITION`** — i.e. the
historian's OLE DB History provider exposes them as virtual/extension tables that read the
block store, *not* stored T-SQL:
- `Events`, `v_EventHistory`, `v_EventSnapshot`, `v_EventStringSnapshot`, **`v_AlarmEventHistory2`**
(columns: `EventStampUTC`, `AlarmState`, `TagName`, `Description`, `Area`, `Type`, `Value`,
`Priority`, `Category`, `Provider`, `Operator`, `DomainName`, `UserFullName`, `MilliSec`, …)
— these are the read-back of the SDK alarm/event property bag.
So `SELECT … FROM Events` (and `v_AlarmEventHistory2`) is **the provider reading the block
store**, which is why the handoff could query events even though they live in `.dat` blocks.
### Database-mode physical store
When events ARE stored in SQL (Database mode / A2ALMDB integration), the writer is the loaded
`ArchestrAEvents.EventStorage.Contract` connection assembly doing batched inserts ("storing N
events in database", "creating event storage database connection role"). The exact table
schema there is the A2ALMDB alarm schema (not present on this box to dump).
### Read-back ✅
Uniform regardless of backend: `Retr.StartEventQuery` → `GetNextEventQueryResultBuffer`
(provider) surfaces events from wherever they were stored — so histsdk `ReadEventsAsync` is
mode-agnostic. Engine filter note: `"EventTime filtering can only be specified through
StartDateTime and EndDateTime"`.
## 6b. Two different "event" subsystems — don't conflate ✅
| | Classic event **detectors** | ArchestrA **alarms/events** (the SDK path) |
|---|---|---|
| What | server-side detectors watching tag conditions | client-streamed `HistorianEvent` (alarms, user events) |
| Config/store | `Runtime.dbo._EventTag` (TagName, DetectorTypeKey, DetectorString, Action*, ScanRate, Edge, Priority) | CM_EVENT / CommonArchestraEvent tag |
| History | **physical** `BASE TABLE Runtime.dbo.EventHistory` (`EventLogKey, TagName, DateTime, DetectDateTime, Edge`); 30 rows | block store (or A2ALMDB), surfaced via `v_AlarmEventHistory2` / `v_EventSnapshot` |
| Source | evaluated by the server | sent via `AddStreamedValue(HistorianEvent)` |
`AddStreamedValue(HistorianEvent)` feeds the **right column** (ArchestrA alarms/events) — it is
**not** the classic `EventHistory` detector log.
---
## 7. Relationship to histsdk
- histsdk implements event **reads** only (`ReadEventsAsync` via `StartEventQuery`); its
CM_EVENT EnsT2/RTag2 dance is read-subscription registration.
- Event **writing** is unimplemented but viable. Chain to replicate: Event-type connection →
register CM_EVENT (done) → serialize `HistorianEvent` (header + typed property bag) into the
event-VTQ value blob (inverse of `HistorianEventRowProtocol`) → batch into an event data
packet → stream via `AddStreamValues` (2023 R2 gRPC: `StorageService.AddStreamValues`).
---
## Event-send wire format ✅ (captured 2026-06-20)
`AddStreamedValue(HistorianEvent)` leaves the client over **WCF as `AddS2`
(`IHistoryServiceContract2.AddStreamValues2(string handle, byte[] pBuf, out errorBuffer)`)** —
**not** the storage-engine pipe. (Captured with the NativeTraceHarness `event-send` scenario +
instrument-wcf-writemessage; two events diffed to separate constant framing from value fields.)
CM_EVENT is a built-in registered tag, so the `129 TagNotFoundInCache` gate that blocks `AddS2`
for user tags doesn't apply. Open2 uses event connection-mode **0x501** (vs 0x402 read / 0x401
write). The `pBuf` is a storage **sample buffer** wrapping the event VTQ:
```
0x00 UInt16 0x534F "OS"
0x02 UInt16 sampleCount = 1
0x04 UInt32 packet length = delivered byte[].Length (= valueBlob.Length + 11)
0x08 UInt16 valueBlob.Length + 1
0x0A valueBlob:
+0x00 GUID CM_EVENT tag id (353b8145-5df0-4d46-a253-871aef49b321)
+0x10 Int64 EventTime FILETIME UTC (floored to ms — the VTQ timestamp)
+0x18 UInt16 OpcQuality = 192
+0x1A UInt16 192
+0x1C UInt16 0x118D (opaque CCommonArchestraEventValue descriptor — constant)
+0x1E GUID event Id
+0x2E Int64 ReceivedTime FILETIME UTC (full 100ns; native uses a unique/monotonic time)
+0x36 compact-ASCII string Namespace
+.... compact-ASCII string EventType
+.... UInt16 eventStructVersion = 5
+.... UInt16 propertyCount
+.... propertyCount × { compact-ASCII name; value: UInt8 typeMarker, UInt8 len, UInt8 status, len×bytes }
trailing: 1 pad byte so byte[].Length == packet length (UInt32 @0x04). The native relies on
the MDAS encoder adding this byte (non-deterministic value); the SDK emits 0x00.
```
Property values reuse `HistorianEventRowProtocol`'s typed encoding; only `0x43` (UTF-16 string)
is captured on the write side so far. Implemented in `HistorianEventWriteProtocol` (golden-tested)
and shipped as `HistorianClient.SendEventAsync`. **Server accepts the `AddS2` (success, empty
error buffer); on-box persistence is environment-gated (the native client behaves identically).**
## Open threads
- ⚠️ Event **persistence**: accepted `AddS2` events do not land in `v_AlarmEventHistory2` on the
local dev box (native client identical) — the event storage/ingestion pipeline isn't active
here. Needs a Historian with active event storage to verify send→store→read-back end-to-end.
- 🔶 Non-string event property write encodings (bool/int/filetime/guid/double) and revision/
update/delete event sends (RevisionVersion≠0, IsUpdate/IsDelete) are **not captured**; the SDK
throws `ProtocolEvidenceMissingException` for them. One capture each to extend.
- ❓ `EnqueueEventDataPacket.SerializedBytes` packet framing (header + N event VTQs batched).
- ✅ Database-mode store: server writer is `aahEventStorage.exe` loading the managed
`ArchestrAEvents.EventStorage.Contract` connection assembly; SQL retrieval surface is the
provider-backed `Events` / `v_AlarmEventHistory2` / `v_EventSnapshot` views (NULL T-SQL def).
This box runs **Block storage** (A2ALMDB absent). A2ALMDB physical schema still un-dumped (needs
a System-Platform-integrated box).
- ✅ `ArchestraEvent` vs `CommonArchestraEvent`: send path packs **CommonArchestraEvent** via
`EVENT_TAGID`; both are event-tag schemas in `CTagMetadata` (the server stores either).
- ❓ `UpdateEventStatus` wire payload for `Update`/`Delete` revisions.
- ❓ `EventStorage` recovery-log (`C:\Historian\Data\Logs\EventStorage`) on-disk format (WAL).
- 🔶 Decompile `ArchestrAEvents.EventStorage.Contract.dll` (managed) for the exact DB insert
contract/schema — locate it (not under `…\Wonderware\Historian`; check GAC / Framework\Bin).
---
## Changelog
- Rev 4 (live local install): confirmed server side. `aahEventStorage.exe` (`InSQLEventSystem`)
is the event store engine — per-client event-tag pipeline → recovery-log WAL → Block storage
OR SQL (loadable `ArchestrAEvents.EventStorage.Contract` assembly) → EventReplication. This box
uses **Block storage** (527 `.dat` in `C:\Historian\Data\Circular`; A2ALMDB absent). SQL
`Events`/`v_AlarmEventHistory2`/`v_EventSnapshot` are **provider-backed views over the blocks**
(NULL `OBJECT_DEFINITION`), not physical tables — `v_AlarmEventHistory2` (224 rows/30d) is the
SDK-event read surface. Distinguished the classic event-detector subsystem (`_EventTag` →
physical `EventHistory`) from the ArchestrA alarm/event path (the SDK's target).
- Rev 3: event value serialization pinned to native `CCommonArchestraEventValue::PackToVtq`
via managed `HistorianEvent.PackToVtq`; documented the `CCommonArchestraEventStruct` field
set/order (ReceivedTime, EventType, EventTime, Id, RevisionVersion, IsUpdate/IsDelete,
Namespace, typed property bag) and the path to a managed send serializer.
- Rev 2: HistorianEvent structure + HistorianEventPropertyType enum; client marshaling
(HISTORIAN_VALUE2 / ConvertManagedStructToUnmanagedStruct / AddHistorianValue); dedicated
event pipeline (OpenEventConnection2 / EnqueueEventDataPacket / ForwardEventSnapshot /
store-and-forward); revisions (Update/Delete/UpdateEventStatus); Blocks-mode clarified
(events = generic VTQ snapshots, no event-specific block code).
- Rev 1: client send path, EventStorageMode switch, Blocks/Database backends, read-back.
+166
View File
@@ -0,0 +1,166 @@
# R1.8 / R1.9 — Analog-summary & State-summary queries (implementation plan)
**Status (2026-06-21): RESOLVED by request + response capture. Conclusion: the rich
multi-aggregate analog/state summary struct is NOT delivered over the 2020 WCF binary protocol.
The per-cycle aggregate values it would expose are ALREADY shipped via `ReadAggregateAsync`
(RetrievalMode → QueryType 58). No new `src/` code is warranted for R1.8/R1.9 on 2020 WCF.**
## RESOLVED — what the response capture proved (2026-06-21)
The request side was recovered first (table further down), then the `GetNextQueryResultBuffer2`
**response** was captured (`instrument-wcf-readmessage`, both hooks chained) and decoded against
`AnalogSummaryHistory` SQL ground truth for `SysTimeSec` over a 6 h window / 1 h cycle. Findings:
1. **The response is the ordinary version-9 row buffer** — same layout the existing raw/aggregate
parser (`TryParseGetNextQueryResultBufferAggregateRows`) already handles: `uint16 version=9`,
`uint32 rowCount`, then per-row `tagKey + nameLen + name + ValueCount + cycleEnd FILETIME +
quality + OpcQuality + Value(double) + PercentGood(double) + trailer(cycleStart FILETIME …)`.
The captured 7-row buffer decoded with `Value=31.0`, `PercentGood=100.0`, `ValueCount=1`,
`OpcQuality=192` — matching the SQL row exactly.
2. **There is NO rich `CAnalogSummaryValue` struct on the wire.** Each row carries a *single*
value, not Min+Max+First+Last+Avg+Integral together. The all-aggregates-in-one-row shape that
`CAnalogSummaryValue` / `AnalogSummaryHistory` represents is the **SQL/OLEDB provider's** shape,
not the binary `StartQuery2` retrieval's.
3. **The single value is selected by `RetrievalMode` (QueryType), not by `ValueSelector`.** Proven
against the same constant tag where only the *kind* of aggregate distinguishes the result:
- `RetrievalMode=Integral` (QueryType 8) → `Value = 111600.0` (= SQL `Integral`) ✓
- `RetrievalMode=TimeWeightedAverage` (QueryType 5) → `Value = 31.0` (= SQL `Average`) ✓
- `Cyclic` (QueryType 0) **+ `ValueSelector=Integral`** → `Value = 31.0` (selector **ignored**;
the request byte `ValueSelector@0x59=0x04` was confirmed sent, yet the cyclic value came back).
So `ValueSelector` / `AggregationType` / `MaxStates` are **inert on the WCF retrieval path**
they configure the SQL provider's summary tables, not this binary query.
4. **Resolution unit is correct in the SDK.** The wire `Resolution` is 100 ns ticks (= ms × 10000).
`SerializeFullHistoryRequest` writes `TimeSpan.Ticks`, which the golden test
`SerializerMatchesInstrumentedNativeTimeWeightedAverageRequest` already verifies byte-for-byte
against native (`FromMinutes(1)``600000000`). No bug.
**Therefore:** "analog summary" over 2020 WCF == the existing aggregate read. To get Min, Max,
Average and Integral for a cycle you issue the corresponding `RetrievalMode` queries
(`MinimumWithTime` / `MaximumWithTime` / `TimeWeightedAverage` / `Integral`), each returning that
one aggregate per cycle — all already implemented, mapped (QueryType 58) and golden-tested in
`ReadAggregateAsync`. **R1.8/R1.9 need no new protocol code on this server.** A genuine
all-aggregates-at-once summary would require the gRPC front door or the SQL provider, neither of
which is the 2020 WCF binary path.
Capture/decode tooling is committed and repeatable: `scripts/Capture-SummaryRequest.ps1`
(`-WithResponse` chains ReadMessage), `scripts/decode-summary-capture.py` (request diff),
`scripts/decode-summary-response.py <config>` (response decode vs SQL ground truth). Raw captures
live under `artifacts/reverse-engineering/instrumented-wcf-writemessage-summary/` (gitignored).
---
_Original scoping notes below remain for context. They led to the capture; the conclusion above supersedes their "ready to implement" framing._
Unlike the M1 *read* items gated by the [string-handle wall](../reverse-engineering/wcf-string-handle-wall.md),
summary queries ride the **proven `uint`-handle `StartQuery2`** path — the same call the working
raw/aggregate reads use. So they are genuinely reachable here; the only work is (a) the right
request parameters and (b) decoding the summary row buffer.
## What's already in place
`HistorianDataQueryRequest` + `SerializeFullHistoryRequest`
(`Wcf/HistorianDataQueryProtocol.cs`) already serialize every field a summary query needs:
`QueryType` (INSQL_QUERYTYPE), `SummaryType` (HISTORIAN_SUMMARYTYPE), `AggregationType`,
`ColumnSelectorFlags`, `Resolution`. Normal reads send `SummaryType=0` and
`ColumnSelectorFlags=0x0000_8182_0007_82FF`. A summary query is the **same request with summary
values in those three fields**, then a different row parser on the result buffer.
## Decode targets recovered from `current/aahClientManaged.dll`
Found via `methods … Summary` + `dnlib-method`:
| Native artifact | Token | Use |
|---|---|---|
| `CAnalogSummaryValue.UnpackFromValueBuffer` | `0x06000394` | **the analog-summary row decoder** — a chain of buffer-reader calls (not literal offsets), so decode empirically against a captured buffer |
| `CAnalogSummaryValue.PackToVtq` | `0x06000395` | inverse (for a future write path) |
| `CAnalogSummaryValue` setters | `0x0600038A92` | wire field set: **StartDateTime, Min, Max, First, Last, ValueCount, TimeGood, Integral, IntegralOfSquares** |
| `CAnalogSummaryStruct` setters | `0x0600036977` | fuller field set: adds **MinDateTime, MaxDateTime, FirstDateTime, LastDateTime, FirstNullDateTime, LastNullFlag, LinearIntegral** |
| `CStateSummaryStruct` setters | `0x0600039BA0` | **state-summary fields: MinContained, MaxContained, TotalContained, PartialStart, PartialEnd, StateEntryCount** |
| `QueryColumnSelector.SelectAnalogSummaryColumns` | `0x0600004B` | builds `ColumnSelectorFlags` for analog summary via `CColumnNameMap.GetColumnFlag(name)` per column |
| `QueryColumnSelector.SelectStateSummaryColumns` | `0x0600004C` | same, state summary |
| `QueryColumnSelector.SelectNonSummaryColumns` | `0x0600004D` | the default (matches the `0x…82FF` flags reads already send) |
| `CTypeMetadata.IsAnalogSummary` / `IsStateSummary` | `0x060001A4/A5` | server-side type gating |
| `INSQL_QUERYTYPE` / `HISTORIAN_SUMMARYTYPE` | enums `0200013F` / `02000191` | the `QueryType` / `SummaryType` values to send |
## Native request capture (2026-06-21) — request shape RECOVERED
The earlier blind probing (sweeping `SummaryType`/`ColumnSelectorFlags` over the managed
serializer) was the wrong lever: it returned 0-row buffers because the managed `SummaryType`
field is **not** how the native client encodes a summary. A real capture settled it.
**Capture pipeline (now repeatable):** `scripts/Capture-SummaryRequest.ps1` IL-rewrites a copy
of `aahClientManaged.dll` (`instrument-wcf-writemessage`), stages it alongside the strong-named
`ReverseInstrumentation` logger, then drives the `NativeTraceHarness` history scenario through a
candidate matrix while logging every outgoing MDAS body. `scripts/decode-summary-capture.py`
extracts the `Retr/StartQuery2` `pRequestBuff` from each and diffs the summary candidates against
a tag-matched `baseline-full`. The harness now exposes `--value-selector` / `--aggregation-type`
/ `--max-states` / `--filter` so the native `HistoryQueryArgs` summary knobs can be driven.
**There is no separate "summary" QueryType or `SummaryType` field.** A summary is an ordinary
`StartQuery2` request (`QueryType` = the chosen `RetrievalMode`, e.g. `Cyclic`=0) with three
things set: the **ValueSelector** byte, the **AggregationType** byte, a non-zero **Resolution**
(which fills the previously-zeroed `AutoSummaryParameters` trailer), and — for state summary —
the **MaxStates** field. The server then returns analog- vs state-summary rows based on the tag
type plus these fields. Offsets below are **into the StartQuery2 `pRequestBuff`** (229-byte
`SysTimeSec` baseline; verified byte-for-byte against the native client):
| Offset | Field | Type | Evidence |
|---|---|---|---|
| `0x01` | QueryType | uint32 LE | Full→`02`, Cyclic→`00` (matches the verified `RetrievalMode``QueryType` map) |
| `0x1D` | Resolution | float64 LE | `36e9` ticks → `00 00 00 D0 88 C3 20 42` = `0x4220C388D0000000` (1 h). Zero for non-summary reads |
| `0x32` | Timezone | len-prefixed UTF-16 | `"UTC"` |
| `0x49` | Filter | len-prefixed UTF-16 | `"NoFilter"` default; driven by `--filter` |
| `0x59` | **ValueSelector** | byte | baseline `01` (Auto); `--value-selector Minimum``06`, `Maximum``07`, `Average``08` — exact `HistorianValueSelector` values |
| `0x5B` | **AggregationType** | byte | baseline `03`; `--aggregation-type Average``02` — exact `HistorianAggregationType` values |
| `~0x5F` | ColumnSelectorFlags | bytes | `FF 82 07 00 82 81` — matches the `0x0000_8182_0007_82FF` reads already send; **unchanged** by summary |
| `0x6B` | Tag name | len-prefixed UTF-16 | `count, "SysTimeSec"` |
| after tag | **MaxStates** | uint16 LE | the `01`-default byte after the tag block; `--max-states 10``0A` (state summary, R1.9) |
| `~0xAA` | **AutoSummaryParameters** | block | zero for plain reads; `80 1E 08 6B 47 01` when Resolution set (identical across analog *and* state) — the resolution-derived cycle block |
State summary (R1.9) is the **same request** with `MaxStates` > 0 (the analog `ValueSelector`/
`AggregationType` bytes stay at their `01`/`03` defaults); the analog-vs-state distinction on the
wire is which of those fields is non-default, plus the tag type. Note `MaxStates` is a **UInt16**
on `HistoryQueryArgs` (passing UInt32 throws) — the harness casts accordingly.
Raw captures live under `artifacts/reverse-engineering/instrumented-wcf-writemessage-summary/`
(gitignored). Re-run with `scripts/Capture-SummaryRequest.ps1` (analog: `SysTimeSec`; state:
`-TagName SysPulse`, the local discrete tag).
## Open questions (only the row layout remains)
1. ~~**Request params.**~~ **DONE** — see the table above. ValueSelector @ `0x59`,
AggregationType @ `0x5B`, Resolution @ `0x1D` (→ AutoSummaryParameters @ `~0xAA`),
MaxStates after the tag block. No new QueryType/SummaryType ordinal involved.
2. **Row layout (next concrete step).** Capture the `GetNextQueryResultBuffer2` *response* for an
analog summary of `SysTimeSec` over a multi-hour window with a 1 h resolution — instrument
`ReadMessage` (`instrument-wcf-readmessage`, symmetric to the WriteMessage capture already
wired here) and decode against the `CAnalogSummaryValue` field set
(StartDateTime + Min/Max/First/Last/ValueCount/TimeGood/Integral/IntegralOfSquares). The
request side is no longer a blocker.
## Implementation steps (per the project's two-tests discipline)
1. Add request params to `HistorianDataQueryRequest` builders (a `BuildAnalogSummaryRequest` /
`BuildStateSummaryRequest` alongside `BuildAggregateQueryRequest`).
2. **Live-probe** `SysTimeSec` via a gated diagnostic; sanitize the response into
`fixtures/protocol/analog-summary/` using the CW-1 pipeline.
3. Write `TryParseGetNextQueryResultBufferAnalogSummaryRows` (+ state variant) against the fixture.
4. Public API: `ReadAnalogSummaryAsync` / `ReadStateSummaryAsync` returning new models
`HistorianAnalogSummary` (Min/Max/First/Last/Avg=Integral÷TimeGood/ValueCount/…) and
`HistorianStateSummary` (per-state contained/partial/entry-count). Reuse `RunQuery` plumbing.
5. Golden-byte test on the parser + gated live test on `localhost` (assert non-empty, fields sane).
## State of play
The **request side is fully recovered** from real bytes (table above) — the managed
`HistorianDataQueryRequest` builder can now set `ValueSelector`/`AggregationType`/`Resolution`
(+ `MaxStates` for state) against ground truth rather than guesses. What remains is the
**response row layout**: `CAnalogSummaryValue.UnpackFromValueBuffer` is reader-call-based (no
literal offset table), so the parser needs a captured real *response* buffer to decode against
(step 2 in Open questions — `instrument-wcf-readmessage`, already wired alongside the WriteMessage
capture). Per project rule ("never guess wire bytes; leave throwing until evidence supports it")
no summary code is in `src/` yet — that lands once the response fixture exists.
+675
View File
@@ -0,0 +1,675 @@
# Plan: Revision-Write Path (`AddRevisionValuesBegin/Value/End`)
Status: **WCF: ARCHITECTURALLY BLOCKED (verified 2026-05-05).** **gRPC (2023 R2): the
non-streamed-original transaction is REACHABLE — Begin/End round-trip LIVE-VERIFIED 2026-06-21.**
Same root cause on WCF as `AddS2`: the `TransactionService` relay needs a pre-existing
storage-engine *pipe* session no WCF op can create. The 2023 R2 gRPC front door removes that wall
(see the §"2023 R2 gRPC — the wall is gone" section immediately below); the legacy WCF analysis is
preserved unchanged after it.
## 2023 R2 gRPC — the wall is gone (non-streamed original writes), LIVE-VERIFIED 2026-06-21
The whole D2 WCF blocker was: `ITransactionServiceContract2.AddNonStreamValuesBegin2` returns
`04 33 00 00 00` = `UnknownClient (51)` because the server-side Trx relay requires a storage-engine
pipe session (`STransactPipeClient2``aaStorageEngine.exe`) that no WCF op establishes. On the
**2023 R2 gRPC** transport that relay is replaced by a first-class `TransactionService` gRPC
service, and the gRPC server is itself the gateway to the storage engine — so the client passes the
**HistoryService Open2 storage-session GUID** straight in as `strHandle` and the transaction opens.
**Live probe (`grpc-revision-probe` CLI command / `HistorianGrpcRevisionProbe`):** against the real
2023 R2 server (History iface 12), over a **write-enabled** (`0x401`) gRPC session —
| step | result |
|---|---|
| `HistoryService.OpenConnection` (write-enabled `0x401`) | ✅ `OpenSucceeded`, client handle + storage GUID returned |
| `TransactionService.GetTransactionInterfaceVersion` | ✅ error 0, **version 2** |
| `TransactionService.AddNonStreamValuesBegin(strHandle = storage GUID **UPPERCASE**)` | ✅ **`BeginSucceeded`** — returns a real `strTransactionId` (e.g. `…-FE0A-4822-…`) on the **first** handle format tried |
| `TransactionService.AddNonStreamValuesEnd(handle, txId, bCommit=**false**)` | ✅ `EndDiscardSucceeded` — transaction discarded, **no data written** |
So the answer to the roadmap's open M3-over-gRPC question ("does the 2023 R2 gRPC front door expose
a non-streamed write that bypasses the legacy storage-engine pipe?") is **YES** — Begin/End is
reachable from the pure-managed SDK with no pipe, no native wrapper. The probe is committed as the
`grpc-revision-probe` CLI command + the gated test
`HistorianGrpcIntegrationTests.NonStreamedWriteTransaction_OverGrpc_BeginsAndDiscards`; re-run any
time to confirm the path is still open.
### Decompile basis (handle + op group)
`Archestra.Historian.GrpcClient.GrpcHistoryClient` drives the identical three-phase sequence
(`AddNonStreamValuesBegin(strHandle) → strTransactionId`; `AddNonStreamValues(strHandle,
strTransactionId, btInput)`; `AddNonStreamValuesEnd(strHandle, strTransactionId, bCommit)`), passing
the Open2 session GUID as `strHandle`. `btInput` is the **same opaque native VTQ buffer** the 2020
path uses. Proto: `src/AVEVA.Historian.Client/Grpc/Protos/TransactionService.proto`.
### What is proven vs. what remains (do NOT ship yet)
-**Proven:** the transaction lifecycle (Begin → End/rollback) is reachable over gRPC. The D2
architectural wall is specific to the WCF transport.
-**Not yet captured:** the `AddNonStreamValues` **`btInput` VTQ buffer byte layout**. Per project
discipline ("never guess wire bytes; capture first") no value-commit is implemented. The next step
to actually *ship* M3 (`AddHistoricalValuesAsync`) is to capture the native gRPC `AddNonStreamValues`
`btInput` (or decode the `GrpcHistoryClient` serializer), build a golden-tested serializer, then do a
real `bCommit=true` write + SQL read-back against a sandbox tag created by `EnsureTagAsync`.
- 🔒 **Scope:** this is **non-streamed ORIGINAL backfill** (`HistorianDataCategory.NonStreamedOriginal`
`TransactionService.AddNonStreamValues*`). **Revision EDITS** (`AddRevisionValue(s)` /
`RevisionInsert*`, the R4.2 path) are NOT on the gRPC contract even in 2023 R2 — the capability
matrix confirms they still ride the storage-engine pipe. The gRPC unlock here is original backfill,
not after-the-fact edits.
### R3.1 decode probe (2026-06-21): `AddNonStreamValues` reaches the server-side storage-engine console pipe
The `btInput` VTQ buffer is assembled in native C++ (`SendNonStreamedValues(batchID)` → a vtable
call after values are pooled via native `AddNonStreamedValueAsync(&HISTORIAN_VALUE2)`) and is **not
visible in any decompile** — only the 44-byte packed `HISTORIAN_VALUE2` struct is (TagKey@0,
FILETIME@4, OpcQuality@20, Type@24=7 numeric, value@33, bVersioned@41, VersionStatus@42). So the
framing was probed empirically against the live server with `grpc-nonstream-decode` (every
transaction `bCommit=false` → rolled back, nothing written; tag key from `SysTimeSec`).
**Result — the failure is NOT a buffer-format problem:** six different framings (4454 bytes:
count-prefixed packed struct, struct-only, version+count, OS-wrapped) all returned the **identical**
`AddNonStreamValues` error, while an empty buffer returned a *different* error (`04 01 00 00 00`,
InvalidParameter). The shared error is a nested `SError` whose detail strings are decisive:
```
aahClientAccessPoint::CHistStorageConnection::StoreNonStreamValues::StoreNonStreamValues
\\.\pipe\aahStorageEngine\console,sid(<server storage-engine session GUID>)
```
So non-empty buffers get **past parameter validation into `StoreNonStreamValues`**, which routes to
the **`aahStorageEngine` console named pipe** server-side (the same storage engine as D2 — but the
gRPC *server* now holds the pipe, not the client). Because the error is identical across every
framing, the blocker is **not** the `btInput` layout — it is a **missing storage-engine console
session / tag-registration precondition** for the connection.
**Required call sequence (mapped from the 2023 R2 decompile, corroborates the error above):** the
missing precondition is **`StorageService.OpenStorageConnection`** — it creates exactly the
`\\.\pipe\aahStorageEngine\console,sid(...)` console session named in the failure. The native
non-streamed write path is:
```
HistoryService.OpenConnection (✅ have it — the Open2 handshake)
→ StorageService.OpenStorageConnection (⛔ MISSING — opens the console sid session; SEPARATE
storage session, returns its own uint handle + new GUID)
→ StorageService.RegisterTags (register the tag→storage mapping for the session)
→ TransactionService.AddNonStreamValuesBegin (✅ works)
→ TransactionService.AddNonStreamValues(btInput) (⛔ currently fails here — no console session yet)
→ TransactionService.AddNonStreamValuesEnd(bCommit=true)
→ StorageService.CloseStorageConnection / HistoryService.CloseConnection
```
`OpenStorageConnection` (gRPC `StorageService`) takes 12 args — HostName, EnginePath
(`\\.\pipe\aahStorageEngine\console`), FreeDiskSpace, ProcessName, ProcessId, UserName, Password(+len),
ClientType, ClientVersion, ConnectionMode, ConnectionTimeout, StorageSessionId(in/out) — and returns a
**new** storage `Handle` (uint) + a **new** StorageSessionId GUID (distinct from the Open2 GUID).
**Two hard parts remain, each a separate live-production decode loop (no static shortcut):**
1. **Reproduce the `OpenStorageConnection` handshake** — several of the 12 args are only inferable from
the decompile (ProcessId, ClientType/Version, ConnectionMode, the password-bytes framing), so the
exact values must be confirmed against the live server.
2. **Decode the `AddNonStreamValues` `btInput`** — built in C++ (`SendNonStreamedValues` vtable call),
**absent from every decompile**; only the 44-byte packed `HISTORIAN_VALUE2` struct is known. Must be
decoded empirically once the console session exists (the batch-1 identical-error result could not
distinguish framings precisely *because* there was no session — with a session, framings should
diverge and the correct one becomes findable).
Raw decode artifact: `artifacts/reverse-engineering/grpc-nonstream-decode/batch1-decode.txt`
(gitignored). Probe command: `grpc-nonstream-decode`; driver:
`HistorianGrpcRevisionProbe.ProbeNonStreamedBuffersAsync` (candidate guess-bytes live in the RE tool,
not `src/`).
### R3.1 follow-up (2026-06-21): `OpenStorageConnection` is the WRONG precondition — error 85 = "session not registered"
The mapped sequence above named `StorageService.OpenStorageConnection` as the missing console-session
step. **A live probe (`grpc-open-storage-connection` CLI / `HistorianGrpcStorageConnectionProbe`)
disproved that.** Against the real 2023 R2 server, over a write-enabled (`0x401`) session, every
`OpenStorageConnection` attempt — sweeping `ConnectionMode` (0x401/0x402/0x1), `StorageSessionId`-in
(Open2-GUID-upper / empty), and `FreeDiskSpace` — returned the **identical** error
`84 55 00 00 00 …09 15 00 "OpenStorageConnection"` = **type 4 (CustomError, 0x80 detail flag), code
`0x55` = 85**, independent of all swept values. So it is a *structural* refusal, not a bad field.
**Decoding the refusal (two corroborating facts):**
1. **Error 85 is the generic "session not registered for this op" code.** The event read path hits the
*same* `type=4 code=85` from `GetNextEventQueryResultBuffer` when the session hasn't registered its
tag first (see `HistorianWcfEventOrchestrator` xmldoc) — the fix there is front-door `RegisterTags2`
(RTag2), NOT a storage connection.
2. **`OpenStorageConnection` is not a front-door client op.** In the 2023 R2 decompile it lives on a
**separate `GrpcStorageClient`** (`Archestra.Historian.GrpcClient`, `GrpcClientBase` with its own
`Initialize(target, port, …)` channel) and the managed `HistorianAccess` non-streamed write goes
through the **native C++ `<Module>.HistorianClient.AddNonStreamedValueAsync`**, never this gRPC op.
The `StorageService` proto is almost entirely snapshots / blocks / SF params / `SendSnapshot`
it is the **storage engine's store-and-forward / snapshot interface** (`HistorianAccess`
documents `OpenStorageConnection`/`CloseStorageConnection` as the SF-snapshot *flush*), reached on
a distinct channel under a service identity. A normal Historian client never opens it on 32565.
**Corrected required sequence — the precondition is front-door tag registration, not a storage conn:**
```
HistoryService.OpenConnection (write-enabled 0x401) ✅ have it
→ HistoryService.RegisterTags(strHandle, btTagInfos = TARGET tag) ⛔ the real missing step
(front door, string handle — the RTag2 family; same op that subscribes the event session)
→ TransactionService.AddNonStreamValuesBegin ✅ works
→ TransactionService.AddNonStreamValues(btInput) ⛔ R3.1 batch failed here precisely
BECAUSE no tag was registered for the session (StoreNonStreamValues had no tag→storage route)
→ TransactionService.AddNonStreamValuesEnd(bCommit)
```
This matches the original 2020-WCF D2 hypothesis ("what populates the session's tag working set is
likely a `RegisterTags2` call") — the gRPC front door does expose that op (`HistoryService.RegisterTags`,
in our `HistoryService.proto`).
**Remaining blockers (both need a native gRPC capture — no static shortcut, do NOT guess bytes):**
1. **`HistoryService.RegisterTags` `btTagInfos` for a *regular analog* tag.** The only known RTag2
buffer is CM_EVENT's (a built-in tag identified by a well-known 16-byte *tag*-GUID,
`0x6750` v2 + count + GUID). Regular tags expose only a uint `tagKey` + a *type*-id GUID via
`GetTagInfo` (see `ParseTagInfoRecord`) — **no per-tag GUID**, so the regular-tag registration
framing (tagKey-based vs tag-GUID-based) is uncaptured.
2. **`AddNonStreamValues` `btInput`** — still C++-built and absent from every decompile (unchanged).
Both require capturing the **native 2023 R2 gRPC client** performing a non-streamed write (it would
emit the exact `RegisterTags` `btTagInfos` + `btInput`), or decoding the C++ serializer. Probe:
`grpc-open-storage-connection` (committed, regression-safe — it opens nothing persistent and
CloseStorageConnections on success). **Status: M3 transaction lifecycle proven; the insert precondition
is now correctly identified as front-door `RegisterTags` (NOT `OpenStorageConnection`); shipping
`AddHistoricalValuesAsync` is blocked on capturing the regular-tag `RegisterTags` `btTagInfos` +
the `AddNonStreamValues` `btInput`.**
### R3.1 capture plan (2026-06-21): drive the native 2023 R2 gRPC client + IL-rewrite the byte[] payloads
Feasibility verified end-to-end against `histsdk-2023r2-analysis/bin`:
- **Self-contained, loadable.** 2023 R2 `aahClientManaged.dll` is a 20 MB **mixed-mode C++/CLI**
assembly whose native imports are only Windows + VC++ runtime (`MSVCP140`/`VCRUNTIME140_1`) — **no
external AVEVA native dependency / no Historian install required** to load it in a `net481` x64
process. The native C++ `HistorianClient` (the `<Module>.HistorianClient.*` globals,
e.g. `AddNonStreamedValueAsync(client, &HISTORIAN_VALUE2, &SError)`) is compiled *into* it and is
what builds `btInput`; it then hands the `byte[]` to the **managed** gRPC client.
- **gRPC routes through managed code → IL-rewrite-able.** `Archestra.Historian.GrpcClient.dll`
(`Grpc.Net`-based) is pure managed; `GrpcHistoryClient` holds both `m_historyClient` and
`m_transactionClient`. Capture targets:
- `GrpcHistoryClient.RegisterTags(string handle, byte[] tagInfos, …)` → dump `tagInfos`
- `GrpcHistoryClient.AddNonStreamValues(string handle, string transactionId, byte[] inBuff, …)` → dump `inBuff`
Use the existing dnlib IL-rewrite tooling (`tools/AVEVA.Historian.ReverseInstrumentation` +
`instrument-wcf-writemessage` pattern), writing rewrites to a copy under
`docs/reverse-engineering/dnlib-write-copy/` — never touch `histsdk-2023r2-analysis/bin` originals.
- **gRPC runtime deps are available.** `Archestra.Historian.GrpcClient.dll` references `Grpc.Net.Client`,
`Grpc.Core.Api`, `Grpc.Net.Client.Web`, `Google.Protobuf`, etc. — the full set is present in
`histsdk-2023r2-analysis/msi-extract/ArchestrA/Toolkits/Bin/x64/` (alongside the 5 core DLLs in
`…/bin/`). Assemble all of them into the harness runtime dir so `Assembly.LoadFrom` + the sibling
resolver can satisfy the gRPC stack.
- **Driving the write (reflection, like `NativeTraceHarness`).** `ArchestrA.HistorianAccess.OpenConnection(HistorianConnectionArgs, out err)`
with `HistorianConnectionArgs { ServerName, TcpPort=32565, ConnectionMode=HistorianConnectionMode.Historian
(the 2023 R2 gRPC mode; `ClassicHistorian`=legacy), ConnectionType=Process, ReadOnly=false,
IntegratedSecurity/UserName/Password, AllowUnTrustedConnection=true, SecurityInfo=cert }`, then
`AddNonStreamedValue(ConnectionIndex.Process, HistorianDataValue, bVersioned:false, out err)`.
- **Cache-gate risk (the D2 blocker).** The C++ `AddNonStreamedValueAsync` has a per-connection
`TagNotFoundInCache (129)` gate that, in the 2020 D2 probe, rejected the value **before any bytes
left the client**. Mitigation to try: **read the target tag first** (populate the per-connection
cache) before `AddNonStreamedValue`. `RegisterTags` is emitted during registration *before* this
gate, so its `tagInfos` is capturable **even if** the gate still blocks `btInput`.
Build order (each live step = prod write, per-action auth): (1) `net481` x64 harness loads the 2023 R2
DLL + opens a **read-only** gRPC connection + reads the tag (proves load+connect, no write); (2)
IL-rewrite `Archestra.Historian.GrpcClient.dll`; (3) write-enabled run → capture `RegisterTags`
`tagInfos` (+ `btInput` if the gate passes); (4) build golden serializer(s) in `src/`; (5) real
`bCommit=true` write + SQL read-back on a sandbox tag → ship `AddHistoricalValuesAsync`.
### R3.1 CAPTURED + VALIDATED (2026-06-21): the write rides `HistoryService.AddStreamValues` ("ON" buffer)
The capture ran end-to-end against the live server (`AVEVA.Historian.Grpc2023CaptureHarness`,
`capture-write` scenario, sandbox tag created by the harness, IL-rewritten `GrpcClient` dumping every
`byte[]`). The committed write **persisted and read back over gRPC** (SDK `ReadRawAsync` returned the
sample) — fully validated.
**The roadmap's assumption was wrong.** The native non-streamed (historical backfill) write does **not**
use `AddNonStreamValues` / the TransactionService at all. The native `HistorianAccess.AddNonStreamedValue
→ SendValues` routes over gRPC as **`HistoryService.AddStreamValues`** carrying an **"ON"
storage-sample buffer** (structurally the AddS2 **"OS"** family — same serializer pattern the SDK already
has in `HistorianEventWriteProtocol`), preceded by **`EnsureTags`** to register the tag:
```
EnsureTags.tagInfos (144B) = the analog CTagMetadata the SDK's EnsureTagAsync already builds
(0x4E marker … fe 00 trailer)
AddStreamValues.values (56B) = "ON" (0x4E4F) + u16 sampleCount(1) + u32 totalLen(56)
+ u16 payloadLen(46) + 16B tag GUID + FILETIME(sample)
+ u16 OpcQuality(192=Good) + u32 type/descriptor
+ FILETIME(received/version) + 8B double value
```
The full priming/write sequence that works from the native client (write-enabled session): `OpenConnection`
`UpdateClientStatus` ×N → `EnsureTags``GetTagInfosFromName` (resolve identity) → `AddStreamValues`
("ON" buffer). Notes: (a) the **D2 cache gate (err 129) does NOT block** the primed 2023 R2 client —
`AddNonStreamedValue` returned success once the session was primed (via `AddTag`/`GetTagInfoByName`) and
the server had assigned the tag key; (b) the value is keyed by a **16-byte tag GUID**, not the uint
`tagKey` (so the SDK serializer needs the tag's GUID, available from EnsureTags/GetTagInfo, not just
`HistorianTagMetadata.Key`); (c) batch lifecycle is `NonStreamedValuesBegin → AddNonStreamedValue →
SendValues → AddNonStreamedValuesEnd` (End-before-Send returns err 160 InvalidBatchId).
**SHIPPED 2026-06-21 — `AddHistoricalValuesAsync`.** `HistorianClient.AddHistoricalValuesAsync(tag, values)`
over `RemoteGrpc`: `HistorianGrpcHistoricalWriteOrchestrator` opens a write-enabled session →
`GetTagInfosFromName` (resolves the per-tag GUID = the tag-info record's `TypeId`) →
`HistoryService.AddStreamValues` ("ON" buffer from `HistorianHistoricalWriteProtocol`, golden-tested) per
sample. The pure-managed SDK wrote a value and read it back live (gated test
`AddHistoricalValuesAsync_OverGrpc_WritesAndReadsBack`). **All five analog types captured + validated**
(Float/Double/Int2/Int4/UInt4): the 4-byte value descriptor `C0 10 01 00` is **constant across types**;
the value is `u32(0) + native-width value` (float32 / double64 / int16 / int32 / uint32) selected by the
tag's declared type (the orchestrator maps it from the tag-info `NativeDataTypeDescriptor`). gRPC-only.
Capture artifacts (gitignored): `artifacts/reverse-engineering/grpc-nonstream-capture/cap-*.ndjson`.
---
## Legacy WCF analysis (preserved — still accurate for the 2020 WCF transport)
Status (WCF only): **ARCHITECTURALLY BLOCKED — verified 2026-05-05.** Same root
cause as `AddS2`: client-side cache rejects values for tags that
weren't registered through a configured IO server / Application Server
pipeline. Documented below; implementation deferred until / unless that
prerequisite is removed.
## Empirical finding (2026-05-05)
The native trace harness was extended with `--write-revision-values` to
drive the revision flow:
1. `HistorianAccess.CreateHistorianDataValueList(HistorianDataCategory.NonStreamedOriginal)`
succeeds — list is bound to the live `HistorianClient*` via
`GetClient(ConnectionIndex.Process)`.
2. `HistorianDataValueList.NonStreamedValuesBegin()` succeeds — list
batchID transitions 0 → 1.
3. `HistorianDataValueList.AddNonStreamedValue(value, validate=true, out error)`
**fails** with `ErrorCode=TagNotFoundInCache (129)`,
`ErrorDescription="error = 129 (Tag not found in cache)"` — the value
is never added to the list (`Count` stays 0).
4. `HistorianDataValueList.AddNonStreamedValuesEnd()` returns void.
5. `HistorianAccess.SendValues(list, out error)` returns `true` with
`ErrorCode=Success`**but** no wire bytes left the client because
the list is empty. (Inspecting captured WriteMessage stream confirms
no `AddNonStreamValues*` Trx call appears.)
The validation that rejects the value is the same gate that blocks
`AddStreamedValue` (`AddS2`): the library's local tag cache only knows
about tags that were:
- Auto-populated from a configured IO server / Application Server pipeline, or
- Read via the existing read flow (which hits the cache as a side effect)
Tags created via `HistorianAccess.AddTag` populate `Runtime.dbo.Tag` but
are not added to the in-memory cache that AddStreamedValue /
AddNonStreamedValue consult. So writes from a managed client to a
client-created tag fail at the validation gate before any wire bytes
flow.
## Conclusion
The revision-write path **does not bypass the AddS2 blocker** — it
shares the same `TagNotFoundInCache` precondition.
### Follow-up probe (2026-05-05): SysTimeSec
To narrow the gate's scope, the harness was extended with
`--write-revision-target-tag <name>` (overrides the value's TagKey via
SQL lookup). Probed `SysTimeSec` (an auto-populated system tag whose
wwTagKey=12 is well-known in the runtime cache):
```
AddNonStreamedValue (TagKey=12 SysTimeSec):
Result=False
ErrorCode=TagNotFoundInCache
ErrorDescription="error = 129 (Tag not found in cache)"
```
Same failure. Then probed with `--write-revision-skip-validate` to set
the `validate` boolean to false on `AddNonStreamedValue` — same
`TagNotFoundInCache` failure. The cache check is intrinsic to the
function, not gated by the `validate` parameter.
So the gate is **per-(client-session, tag)**, not per-(server-cache, tag):
- Server-side, `SysTimeSec` IS in the runtime cache (it's auto-populated).
- Client-side, the managed library has its own per-connection tag list
that AddNonStreamedValue checks. That list is NOT populated by simply
knowing the wwTagKey — something else (likely a `RegisterTags2` call
during connection open, or the read flow as a side effect, or
IO-server-driven registration) populates it.
The harness opens with `ReadOnly=false` for the write scenario, which
may suppress the read-flow side effect that would otherwise populate
the local cache. Without further RE on what populates the local cache,
no path is reachable for a managed client to write either streaming or
revision values.
### Cache gate is inside the native C++ HistorianClient
Followup probe (2026-05-05) tested the **direct** public overload
`HistorianAccess.AddNonStreamedValue(ConnectionIndex, HistorianDataValue, bool validate, ref error)`
which bypasses the `HistorianDataValueList` layer entirely and goes
straight to `HistorianClient.AddNonStreamedValueAsync` (a C++ method).
Even with `validate=false` and `TagKey=12 (SysTimeSec)`, the call
fails: `ErrorCode=TagNotFoundInCache (129)`.
So the gate isn't bypassed by:
1. Using a real wwTagKey from SQL
2. Targeting a server-cache-resident tag (SysTimeSec)
3. Setting `validate=false` on AddNonStreamedValue
4. Bypassing the `HistorianDataValueList` layer (calling the direct
`HistorianAccess.AddNonStreamedValue` overload)
The check is inside the **native C++ `HistorianClient`'s per-connection
tag cache**, not in the managed wrapper. No managed-callable path exists
to populate that cache.
### Critical insight: the SDK doesn't use the C++ HistorianClient
The SDK's production code talks **WCF directly** — no C++ HistorianClient
instance, no per-connection local cache to gate against. The cache check
is enforced by the `aahClientManaged.dll` wrapper, not by the WCF server.
This means the SDK could **plausibly** implement the revision-write
path against the existing
`ITransactionServiceContract.AddNonStreamValuesBegin/AddNonStreamValues/AddNonStreamValuesEnd`
contract methods and have the server accept it directly — bypassing the
gate that blocks the native wrapper.
**Unverified assumptions:**
- The server may have its own cache requirement that mirrors the
C++ wrapper's. If yes, the SDK is also blocked. If no, the SDK
can write where the wrapper can't.
- The server may require `RTag2` (RegisterTags2) to be called per-tag
before AddNonStreamValues — that's a known WCF op, already declared
in `IHistoryServiceContract2`, used by the existing event flow. The
SDK could call it.
- The server may require an IO-server-style registration that's not
exposable over the WCF surface at all.
**Recommendation:** if D2 is ever pursued, do it as a **direct
WCF-level implementation in the SDK**, NOT as a wrapper over the
native HistorianAccess methods. The harness can no longer help (the
wrapper itself is gated). Test paths against the live server by
calling the contract methods directly and observing what the server
returns. If `AddNonStreamValues` succeeds without registration, the
path is implementable. If it fails with a server-side cache error,
try `RTag2` first. If it still fails, the path is genuinely blocked
server-side.
### SDK-direct probe results (2026-05-05)
`HistorianWcfRevisionOrchestrator` wires up the priming chain + a probe
of `ITransactionServiceContract2.AddNonStreamValuesBegin2(string handle, out string transactionId, out byte[] errorBuffer)`.
Live test against `localhost`:
-`OpenSucceeded: True` — Hist auth chain + Open2 still work end-to-end
- ✅ Trx channel opens, `Trx.GetV` returns interface version 2
- ✅ Wire path is recognized — server processes the call (no
`ActionNotSupportedException` after switching from the abbreviated
`AddNonS2B` to the default action name)
- ❌ Server returns structured error `04 33 00 00 00` =
type 4 (CustomError) + code 51 (`UnknownClient`) for all four handle
formats tried (contextKey GUID upper, storageSessionId upper, contextKey
lower, ClientHandle as string)
- ❌ Adding the full priming chain (Stat.GetV ×2, Stat.GETHI ×2, UpdC3,
6× Stat.GetSystemParameter, AllowRenameTags, Trx.GetV, Stat.GetV,
Retr.GetV) doesn't change the result — Trx still rejects with
`UnknownClient`
`ITransactionServiceContract2` exposes only `GetV`, `ForwardSnapshot*`,
and `AddNonStreamValues*`. There is no `ValidateClient`, `RegisterClient`,
or `Open` on Trx. So the client-with-Trx registration must happen via
some cross-service side effect we haven't identified.
**Important takeaway:** the wire path works at the WCF protocol layer.
We're past the "is this even reachable" question. The remaining gap is
finding what populates Trx's session table — likely:
1. `RTag2` on /Hist with a tag whose registration cascades to Trx
2. Some `IStorageServiceContract` op that we haven't tried
3. An aspect of the C++ HistorianClient initialization that doesn't
show up in the IL we've inspected (e.g., the
`aahClientCommon.CClientCommon` calls during InitializeProxy)
A future session that wants to push further should try (in order):
1.**DONE 2026-05-05.** Add `RTag2(CM_EVENT tag id)` to the priming
chain — confirmed `RTag2` itself succeeds (returns 25-byte response),
but `AddNonStreamValuesBegin2` still fails with `UnknownClient`.
So RTag2 doesn't cascade client identity to Trx.
2. ⚠️ **OBVIATED 2026-05-05** by finding (3): `IStorageServiceContract`
ops aren't the missing piece either, because the missing piece isn't
on the WCF surface at all.
3.**DONE 2026-05-05** — IL walk of `aahClientCommon.CClientCommon.AddNonStreamValuesBegin`
`aahClientCommon.CClient.AddNonStreamValuesBegin`
`aahClientCommon.CClient.TransactionBegin`
reveals the chain ultimately invokes
**`aahClientCommon.CHistStorageConnection.StartTransaction`** (token
`0x06001FDD`) which calls **`CStorageEngineConsoleClient.StartTransaction`**.
`CStorageEngineConsoleClient` is built on `STransactPipeClient2` +
`SCrtMemFile` — a **shared-memory + named-pipe** transport to the
storage engine, completely separate from WCF.
### Definitive architectural conclusion (2026-05-05)
The revision-write path uses **two transports in tandem**:
1. WCF (`/Hist`, `/Retr`, `/Stat`, `/Trx`) — what our SDK speaks
2. **Shared-memory + named-pipe to `aaStorageEngine.exe`** — what
`CStorageEngineConsoleClient` speaks; the SDK doesn't (and would be
a major project to implement)
The WCF `ITransactionServiceContract2.AddNonStreamValuesBegin2` op we
were probing is a server-side relay that requires a pre-existing
storage-engine pipe session for the client. That session is established
via the pipe channel, not WCF. Without the pipe-side session, the WCF
relay returns `UnknownClient (51)` — and there's no way to establish
the pipe-side session via WCF.
**D2 is unimplementable as a pure-managed-WCF SDK.** The native wrapper
itself depends on the C++ shared-memory channel; to replicate that
behavior from a managed client would require implementing the whole
storage-engine pipe protocol, which is out of scope and probably
not viable without deeper RE of `aaStorageEngine.exe` itself.
The WCF `ITransactionServiceContract2` declaration in our contracts
file is left in place — it's correct as a contract — but no
orchestrator or public surface should be added on top of it. The
`HistorianWcfRevisionOrchestrator` in `src/AVEVA.Historian.Client/Wcf/`
remains as an internal probe / regression check; if anyone ever
believes the architecture has changed, re-run the probe test to
verify the gate still holds.
### Current state of the SDK-direct probe
`HistorianWcfRevisionOrchestrator.ProbeBeginAsync` does:
```
Open2 (write-enabled, 0x401)
→ priming (Stat.GetV ×2, Stat.GETHI ×2, UpdC3, 6× GetSystemParameter,
AllowRenameTags, Trx.GetV, Stat.GetV, Retr.GetV)
→ RTag2(CM_EVENT tag id) // succeeds
→ Trx.GetInterfaceVersion // succeeds, returns version 2
→ Trx.AddNonStreamValuesBegin2 ×4 // all four handle formats fail with
// 04 33 00 00 00 (UnknownClient 51)
```
The probe is committed as a gated test
(`HistorianWcfRevisionProbeTests.AddNonStreamValuesBegin_ProbeReturnsServerResult`)
that can be re-run any time to verify the gate is still where we think
it is, or to test future priming additions.
## Decision
Do **not** add public `WriteRevisionsAsync` / `BeginRevisionAsync` to
the SDK. The contract methods already exist in
`Wcf/Contracts/ITransactionServiceContract.cs`
(`AddNonStreamValuesBegin/AddNonStreamValues/AddNonStreamValuesEnd`)
for completeness, but the orchestrator and public surface stay absent.
Revisit if either of these changes:
1. AVEVA documents (or a customer demonstrates) a code path that
bypasses the cache validation for client-created tags.
2. The SDK's mission expands to include data correction for tags that
ARE in the runtime cache (i.e., tags managed by a real IO server),
in which case the harness extension below provides a starting point.
## Harness diagnostic (preserved)
The `--write-revision-values` flag in
`tools/AVEVA.Historian.NativeTraceHarness/Program.cs` reproduces the
above failure deterministically. Re-run it any time to verify the
blocker still holds:
```powershell
dotnet run --no-build --project tools\AVEVA.Historian.NativeTraceHarness -- `
--scenario write `
--write-sandbox-tag RetestSdkWriteRevSandbox `
--write-data-type Float `
--write-skip-add-tag --write-skip-add-value `
--write-revision-values
```
Look for the `AddNonStreamedValue` row's `ErrorCode` field in the JSON
output.
## Original plan (preserved for context if the blocker ever lifts)
## Context
The Historian's "revision write" path is the documented mechanism for
editing historized data after the fact (replaces the inferred
`ModifyData` / `DeleteData` use cases that don't exist as WCF ops). Native
managed surface (per Phase 1 findings of the write-commands plan):
| Public method | Token | Purpose |
|---|---|---|
| `ArchestrA.HistorianAccess.AddRevisionValuesBegin` | `0x06006175` | Open a revision-edit transaction |
| `ArchestrA.HistorianAccess.AddRevisionValue` | `0x06006176` | Append a value to the open transaction |
| `ArchestrA.HistorianAccess.AddRevisionValuesEnd` | `0x06006177` | Commit the transaction |
| `ArchestrA.HistorianAccess.AddRevisionValues` | `0x0600617F` | Single-shot variant |
| `ArchestrA.HistorianAccess.AddVersionedStreamedValue` | `0x0600616F` | Push one versioned value (related path) |
WCF surface is unknown — likely a new op group on `IHistoryServiceContract2`
or `IRetrievalServiceContract4` or a new contract.
## Goal
Public SDK API:
```csharp
public Task<HistorianRevisionTransaction> BeginRevisionAsync(string tag, CancellationToken ct);
// On the returned transaction:
public Task AddRevisionValueAsync(HistorianSampleEdit sample, CancellationToken ct);
public Task<bool> CommitAsync(CancellationToken ct);
// IDisposable / IAsyncDisposable for cancellation rollback if such a thing exists
```
Or a single batch convenience:
```csharp
public Task<bool> WriteRevisionsAsync(string tag, IReadOnlyList<HistorianSampleEdit> samples, CancellationToken ct);
```
The choice depends on the wire shape — if Begin/Value/End requires the
caller to maintain a server handle between calls, the disposable
transaction is necessary; if it's stateless, the batch convenience is fine.
## Workstreams
### A. Static analysis (1-2 hours)
- Inspect IL for the four managed public methods to identify the
underlying `CHistoryConnectionWCF.*` calls and their server-side WCF
contract methods.
- Add the contract methods to `Wcf/Contracts/IHistoryServiceContract2.cs`
(or a new contract if appropriate) with `[OperationContract(Name = "...")]`
+ `[MessageParameter]` attributes once names are known.
### B. Native harness extension (2-3 hours)
- Add `--scenario revision-write` to the harness.
- Refer to existing `--scenario write` plumbing for the AddTag wrapper
pattern.
- Sequence:
1. Open connection (probably write-enabled mode `0x401`)
2. AddTag for sandbox tag (re-uses existing harness flow)
3. AddStreamedValue for the initial sample (currently blocked
architecturally per Phase 2 findings — but may not be required if
the revision path operates directly on the historian engine state)
4. AddRevisionValuesBegin / AddRevisionValue × N / AddRevisionValuesEnd
5. Read back via existing read path; verify the samples reflect the
edits
### C. Wire capture (1 hour)
- Same `instrument-wcf-writemessage` + `instrument-wcf-readmessage`
IL-rewrite tooling already used for EnsT2 / DelT.
- Capture both Begin/Value/End and the single-shot AddRevisionValues
variant for byte-level diff.
### D. Decode + managed serializer (4-6 hours)
- Walk the captured InBuff bytes against the native serializer IL.
- The Begin payload likely seeds a server-side transaction handle that
Value calls reference. Look for an `out`-returned handle in the Begin
response.
- Value payload structure is likely similar to `AddS2`'s pBuf
(uint16 version + uint32 sampleCount + N × {tagId, FILETIME, quality,
typed value bytes}) but may include a per-sample revision/version field.
### E. Public API + tests (4-6 hours)
- New types: `HistorianSampleEdit` (sample + reason/version metadata),
`HistorianRevisionTransaction` (disposable handle).
- Public methods on `HistorianClient` per the Goal section.
- Unit tests: golden-byte fixtures for Begin/Value/End/Commit payloads.
- Live integration tests: write a known sample, edit it via the
revision path, read back and assert the new value appears.
## Risks
- **Server-cache prerequisite.** If the historian's revision path
also requires the tag to be "live in the runtime cache" (the same
blocker that killed `AddS2`), the entire path may be unimplementable
for the same architectural reason.
- **State across calls.** Begin/Value/End may store transaction state
on the server keyed by the WCF session GUID. WCF's session model
needs to be configured to keep the same channel alive across all
three calls — which is a different lifecycle from the existing
one-call-per-channel pattern in the SDK orchestrators.
- **Concurrent edits.** Server may reject concurrent revision
transactions on the same tag — needs probing.
- **Time bounds.** Revision likely respects the same `RealTimeWindow`
/ `FutureTimeThreshold` system parameters as `AddS2`. Out-of-window
edits silently drop or error — needs probing.
## Success Criteria
- Public `BeginRevisionAsync` (or batch variant) live-verified against
a sandbox tag created by `EnsureTagAsync`.
- Round-trip test: write initial value → revise it → read back → verify
the revised value persists in `History` extension table via SQL.
- Golden-byte fixtures for Begin / Value / End / Commit captured against
the sandbox tag.
- Decision documented for whether the `AddRevisionValues` single-shot
variant is exposed in addition to the Begin/Value/End sequence.
## Dependencies
- Existing analog write surface (`EnsureTagAsync`) — done.
- `AddS2` is **not** a prerequisite; the revision path may be an
independent code path that bypasses the runtime-cache gate. If it
doesn't, this plan is blocked the same way `AddS2` is.
## Out of scope
- Editing event tags. Events come from AVEVA AnE; the SDK only reads
them.
- Bulk schema changes. Forbidden over the wire per the Historian's
architecture.
## Trigger to start
A customer-driven request, or a real need to expose historical data
correction in the SDK's API. Without one, this remains the most
substantive remaining write-path workstream but isn't worth the 1-2
days of focused work speculatively.
+134
View File
@@ -0,0 +1,134 @@
# Plan: Speculative Items Sweep (2026-05-04)
The five items I previously called out as speculative / deferred. Goal:
exercise the cheap ones, investigate the medium ones to feasibility, and
write sub-plans for anything too big to ship in one push.
## Items
| ID | Item | Effort | Touches |
|---|---|---|---|
| C3 | Expose `IntegralDivisor` on `HistorianTagDefinition` | small | `HistorianTagDefinition.cs`, `HistorianTagWriteProtocol.cs`, orchestrator, tests |
| E | Unit tests for `AllowUntrustedServerCertificate` / `ServerDnsIdentity` | small | new test file under `tests/` |
| D3 | Root-cause Discrete/String/Int1/Int8/UInt8 EnsT2 failure | medium (investigation) | native harness, possibly serializer |
| D1 | Capture wire bytes for `AddTagExtendedProperties` | medium (capture + decode) | native harness, possibly new serializer + public API |
| D2 | Implement `AddRevisionValuesBegin/Value/End` (revision-write path) | large | new orchestrator + 3 new public APIs |
## Parallelism
Concurrency-safe groupings (each pair is independent at the file level):
- **C3 ↔ E** — C3 touches `HistorianTagDefinition.cs` + `HistorianTagWriteProtocol.cs` + orchestrator + integration tests; E adds a new test file + might add a small unit-test util. No file overlap.
- **D3 ↔ D1** — Both touch the native trace harness Program.cs, so they conflict if done concurrently. Sequence them.
- **C3/E ↔ D3/D1** — No file overlap; can run concurrently with the harness work.
- **D2** stands alone (different code paths entirely).
In a single-agent session, the order is:
1. C3 (small, predictable) — land first
2. E (small, predictable) — land second
3. D3 (investigation; documents findings whether or not implementation is possible)
4. D1 (investigation + capture; same pattern)
5. D2 — write a focused sub-plan; do NOT implement in this sweep
## Success Criteria
- C3: `HistorianTagDefinition.IntegralDivisor` (default 1.0) plumbed through serializer; unit test asserts non-default value flips the wire bytes; live test asserts the value persists in `Tag.IntegralDivisor` (or wherever it lands in SQL).
- E: 2-3 unit tests asserting `HistorianWcfClientCredentialsHelper.Configure` and `HistorianWcfBindingFactory.CreateEndpointAddress` honor the new options.
- D3: documented root cause + decision (workable path / not workable / requires further capture). If a workable path emerges quickly, also implement.
- D1: documented evidence summary + decision (worth implementing / defer / requires customer ask).
- D2: `docs/plans/revision-write-path.md` (or similar) with the 5-step capture sequence + open questions.
## Findings
### C3 — IntegralDivisor (executed 2026-05-05)
Plumbed through serializer + orchestrator; default 1.0. Unit test verifies
the 8-byte double immediately preceding the trailer flips with a non-default
value. Live probe: server accepts the wire bytes but `IntegralDivisor`
appears to be stored on `EngineeringUnit` (shared across all tags using that
EU) rather than per-tag, and the EU's stored value didn't change for the
test. Documented in the property's doc-comment. No live integration test
added (nothing to assert in SQL).
### E — Cert option unit tests (executed 2026-05-05)
Added `HistorianWcfCertOptionTests` (5 tests) covering:
- `Configure` is a no-op when `AllowUntrustedServerCertificate=false`
- `Configure` installs the accept-any validator + `RevocationMode.NoCheck`
when the option is true
- `CreateEndpointAddress` with no DNS identity returns an address with
`Identity == null`
- `CreateEndpointAddress` with a DNS identity attaches a `DnsEndpointIdentity`
- `CreateBindingPair(RemoteTcpCertificate)` propagates `ServerDnsIdentity`
to the History endpoint (and not to the Retrieval endpoint, which uses
plain MdasNetTcp without TLS)
### D3 — Discrete/String/Int1/Int8/UInt8 EnsT2 root cause (investigated 2026-05-05)
Probed each unsupported type via the native trace harness with
`--write-data-type {Type}`. Result for SingleByteString and Int1 (others
truncated in the same output):
- `HistorianAccess.AddTag` returns `Success=false`, `TagKey=0`
- Error: `ErrorCode=ValidationFailed`, `ErrorType=CustomError`,
`ErrorDescription="Transaction validation failed"`
**Conclusion:** The failure is **server-side**, not wire-format. The
`/Hist.EnsT2` server-side validator rejects non-analog types when invoked
through the `AddTag → EnsT2` code path. To unlock these types from the SDK
we'd need to:
1. Capture a successful native creation of a discrete/string tag via some
other mechanism (likely SMC's tag-import path or a different WCF op
like `AddTagExtendedProperties` carrying the discrete/string-specific
metadata)
2. Diff the working native flow against the failing one to see what
ancillary fields the validator expects (TagType vs CDataType, separate
StorageType, IO-server pre-registration, etc.)
**Decision:** defer until a customer asks. The native AVEVA wrapper itself
cannot create these tags via `AddTag` from a managed client — implementing
this would require RE work on a path the wrapper doesn't exercise, which
is much higher risk than the existing analog write surface.
### D1 — AddTagExtendedProperties feasibility (investigated 2026-05-05)
Managed surface confirmed. Native API:
- Public managed entry point: `ArchestrA.HistorianAccess.AddTagExtendedProperties`
(token `0x0600619B`, 140 IL instructions, 6 locals)
- WCF op: `CHistoryConnectionWCF.AddTagExtendedPropertyGroups`
(token `0x0600405C`)
- Underlying contract method: `IHistoryServiceContract2.AddTagExtendedProperties`
(already declared in our reproduced contracts)
- Managed input type: `HistorianTagExtendedPropertyGroup` wrapping the native
`CTagExtendedPropertyGroup` C++ class. Built from a `std::vector<CTagExtendedPropertyGroup>`
(visible in the IL locals). Property group structure not yet decoded.
**Decision:** defer implementation. Cost estimate:
1. Reflect-construct `HistorianTagExtendedPropertyGroup` via the native
harness (probably 2-4 hours — these C++/CLI types often have hidden
constructor requirements that surface only at runtime).
2. Call `AddTagExtendedProperties` with a sandbox group; capture wire bytes
via `instrument-wcf-writemessage` (1 hour).
3. Decode the `CTagExtendedPropertyGroup` payload — this is its own struct
that needs walking field-by-field against the native serializer IL
(token `0x06002038`, `CHistStorage.AddTagExtendedProperties`) (3-6 hours).
4. Implement managed `HistorianTagExtendedPropertyGroup` model + serializer
+ public `AddTagExtendedPropertiesAsync` API + tests (4-6 hours).
Total: 1-2 days of focused work. Defer until a customer asks for tag
extended properties or the analog write surface needs them as a
prerequisite.
### D2 — AddRevisionValuesBegin/Value/End
Sub-plan deferred to a dedicated session — see
`docs/plans/revision-write-path.md` (created in this sweep).
## Out of scope for this sweep
Refactoring `HistorianWcfTagClient` to respect `options.Transport` for browse / metadata (i.e., let it use cert binding from Windows). Worth doing but not part of the speculative-items list.
@@ -1,6 +1,18 @@
# Store/Forward Cache Reverse-Engineering Plan # Store/Forward Cache Reverse-Engineering Plan
Last updated: 2026-05-04 Last updated: 2026-06-21
> **2026-06-21 R4.3 re-scope — read this first.** The original plan below
> (2026-05-04) was written against the 2020 Net.TCP/WCF transport, before the
> 2023 R2 gRPC transport existed. Its single biggest open risk — *"is SF state
> readable via a one-shot pull, or only via a duplex push contract we'd have to
> add?"* (Q1/Q2 + §3 Step 3 + Risk 4) — is now **answered: pull, no duplex**.
> The recovered gRPC `StorageService` contract exposes SF state as plain
> request/response RPCs. The current R4.3 scope and recommended path are in
> §9 ("2026-06-21 gRPC re-scope"); the 2020-WCF body below is retained as
> background, not the recommended route.
Original last-updated: 2026-05-04
This document plans the reverse-engineering effort needed to replace the This document plans the reverse-engineering effort needed to replace the
synthesized `GetStoreForwardStatusAsync` in synthesized `GetStoreForwardStatusAsync` in
@@ -499,3 +511,202 @@ Explicitly not part of this plan:
- Anything in the - Anything in the
`aahClientCommon.CSFConnection.StartStoreforward` / `aahClientCommon.CSFConnection.StartStoreforward` /
`SetStorageStopped` / `SetTagSynchronized` write surface. `SetStorageStopped` / `SetTagSynchronized` write surface.
## 9. 2026-06-21 gRPC re-scope (current R4.3 plan)
This supersedes the recommended *route* in §2/§3/§4. The deliverable
(§1) and success criteria (§6) are unchanged. What changed is the
transport and the resolved architecture risk.
### 9.1 What the recovered gRPC contract already gives us
The 2023 R2 contract under `src/AVEVA.Historian.Client/Grpc/Protos/`
exposes SF state through **first-class pull RPCs** on `StorageService`
(`StorageService.proto`) — no duplex/callback contract, no native
`HISTORIAN_STORAGE_STATUS` C-struct decode:
- `GetSFParameter(uint32 Handle, string ParameterName)
→ (Status status, string ParamaterValue)` — the direct analogue of the
already-shipped `GetSystemParameter`/`GetRuntimeParameter` string-keyed
pulls. This is the primary SF-state lever: a name→value read.
- `GetRemainingSnapshotsSize(uint32 Handle)
→ (Status status, uint64 SnapshotSize)` — the pending-buffer magnitude
in one call. Non-zero ⇒ data is queued (`Pending`/`DataStored=true`);
zero ⇒ drained. The cleanest single signal for the idle-vs-active split.
- `GetInfo(string Request) → (Status status, bytes info)` — generic
server info blob; a fallback if a named SF key lives here instead of in
`GetSFParameter`.
- `OpenStorageConnectionResponse.ServerStatus` (field 5) and the
`GetSnapshots`/`StartQuerySnapshot` family — secondary signals.
`SetSFParameter` exists too but is **out of scope** (read-only mission, §8).
The `TransactionService.ForwardSnapshot{,Begin,End}` RPCs are the SF
cache *replay/transfer* path (write-side), **not** a status read — also
out of scope here; they belong to the deferred bit-faithful SF cache work,
not to `GetStoreForwardStatusAsync`.
### 9.2 Plumbing that already exists (reuse, don't rebuild)
- `HistorianGrpcHandshake.OpenSession` — authenticated gRPC session
(`ValidateClientCredential` NTLM loop + Open2) yielding `ClientHandle`
(uint) + storage-session GUID. Live-verified against the 2023 R2 box.
- `HistorianGrpcStorageConnectionProbe` — already constructs a
`StorageService.StorageServiceClient`, primes `GetInterfaceVersion`, and
calls `OpenStorageConnection`/`CloseStorageConnection`. The SF-status
probe is a near-clone that swaps the `OpenStorageConnection` body for
`GetSFParameter`/`GetRemainingSnapshotsSize` calls.
- `HistorianGrpcChannelFactory` / `HistorianGrpcConnection` — channel,
metadata, deadlines.
### 9.3 The one open risk that survives: which `Handle`?
`GetSFParameter`/`GetRemainingSnapshotsSize` both take `uint32 Handle`.
Unknown: do they accept the **session `ClientHandle`** (from
`OpenSession`, which is cheap and unblocked), or do they require the
**storage console `Handle`** returned by `OpenStorageConnection` — which
is the D2 wall (`OpenStorageConnection` routes to the
`\\.\pipe\aahStorageEngine\console` session and is the same storage-engine
pipe that blocks revision writes)? See
[[project_roadmap_exhausted_2020wcf]] and `HistorianGrpcStorageConnectionProbe`
header.
- **Best case:** these read-only status RPCs accept the session
`ClientHandle` (status reads shouldn't need a console writer session).
Then R4.3-over-gRPC is unblocked end-to-end and is a small, shippable
feature.
- **Worst case:** they require the `OpenStorageConnection` `Handle` ⇒
R4.3 inherits the D2 storage-engine-pipe wall and stays blocked on the
same root cause as R4.2. Either way the probe answers it in one run.
### 9.4 Discovery steps (execution order)
1. **Add `grpc-sf-status-probe` to `tools/AVEVA.Historian.ReverseEngineering`**
(mirror `HistorianGrpcStorageConnectionProbe`). Against the live 2023 R2
server it:
- opens an authenticated session, gets `ClientHandle`;
- calls `GetRemainingSnapshotsSize(ClientHandle)` and reports
`status.bSuccess` + `SnapshotSize` + any error buffer;
- sweeps `GetSFParameter(ClientHandle, name)` over a candidate
name list (`Status`, `Storing`, `Pending`, `DataStored`,
`SF.Status`, `StoreForwardStatus`, `Forward`, `CacheSize`,
`ErrorOccurred`, plus any names surfaced by Workstream A's IL of
`ConvertUnmanagedSFStorageStatusToManagedStorageStatus`);
- records which names the server accepts and the returned values.
- If every call fails with an auth/handle-shaped error, retry once
with the `OpenStorageConnection` `Handle` to disambiguate §9.3.
2. **Idle baseline first** — run against the server with SF *not* active.
Establishes the "no SF / drained" response shape (expected:
`SnapshotSize=0`, parameter reads succeed-with-defaults or
return a "not configured" sentinel). This alone may be enough to ship
an honest idle-state implementation that is strictly better than
today's hardcoded all-false synthesis (it would be *measured* false).
3. **Active-SF capture** — only if step 2 proves the read works and we
need the active-state fixtures. Force SF on the sacrificial Historian
VM (stop Runtime DB writer; let the queue spill to SF), re-run the
probe, capture the non-zero/`Storing=true` response. This is the one
invasive step and the gate on full success criteria §6.16.3.
4. **Map + implement** — add `GrpcGetStoreForwardStatus` to the gRPC
read orchestrator, map the probed fields onto
`HistorianStoreForwardStatus`, route `GetStoreForwardStatusAsync`
to it when `Transport == RemoteGrpc` (keep the synthesized fallback
for non-gRPC transports and for the "no SF configured" sentinel).
Add golden-byte fixtures (idle + active) and
`WcfStoreForwardStatusProtocolTests`-style parse tests. Gate the live
integration test on `HISTORIAN_GRPC_HOST`.
### 9.5 Effort / feasibility summary
- **Risk collapsed:** pull-vs-push (the old plan's worst risk) is settled
— it's a pull. No duplex WCF/gRPC callback contract.
- **No native struct decode:** `GetSFParameter` returns a *string*; we
skip the `HISTORIAN_STORAGE_STATUS` C-layout RE entirely (Workstream
A.2 / D.1 become "nice-to-have for field names", not blocking).
- **Reuses shipped plumbing:** session open + `StorageServiceClient` +
channel already exist and are live-verified.
- **Remaining unknowns are empirical, one probe-run each:** (a) the
accepted parameter-name vocabulary, (b) which `Handle` the status RPCs
want (§9.3 — the only thing that could re-block it), (c) the
active-SF response shape (needs the invasive force-SF step).
- **Net:** Step 12 are low-risk and could land a *measured* idle-state
`GetStoreForwardStatusAsync` over gRPC quickly. Steps 34 (full
success criteria) still need the sacrificial-VM force-SF capture and
are gated on §9.3 not landing on the D2 wall.
### 9.6 Out of scope (unchanged from §8, restated for gRPC)
`SetSFParameter`, `ForwardSnapshot*` (SF replay/transfer), the on-disk
cache file format, and redundant-partner SF aggregation all remain out of
scope. R4.3 is read-only status, gRPC-first.
### 9.7 Idle-baseline run — RESULTS (2026-06-21)
Built `HistorianGrpcStoreForwardStatusProbe` + the `grpc-sf-status-probe`
CLI command and ran it against the **live 2023 R2 server** with the
historian in its **idle / not-actively-storing** state (storage interface
v4, authenticated session opened OK). Tested both read-only (`0x402`) and
write-enabled (`0x401`) sessions. Findings, with the §9.3 handle question
**resolved**:
1. **Direct `StorageService` SF pull RPCs are D2-gated — confirmed the
§9.3 worst-case branch.**
- `GetRemainingSnapshotsSize(session.ClientHandle)` →
`bSuccess=false`, error buffer `04 84 00 00 00` (= status `0x84` /
**132 `OperationNotEnabled`**). **Identical under `0x401` and
`0x402`** — so it is NOT the read/write connection-mode gate; the
History-session `ClientHandle` is simply not a valid handle for this
op's handle-space.
- `GetSFParameter(session.ClientHandle, <name>)` → server-side
`RpcException(Unknown, "Exception was thrown by handler")` for **all
16** candidate names, both session modes.
- These two ops need the **`OpenStorageConnection` console handle**,
and `OpenStorageConnection` itself fails with the storage-engine
console error (`84 55 00 00 00 01 02 00 09 15 00`
+ ASCII `"OpenStorageConnection"`) — the **D2 storage-engine-pipe
wall**, the same root cause that blocks R4.2 revision writes. We
cannot obtain the console handle, so these two SF RPCs are
unreachable from a pure managed client. See
[[project_roadmap_exhausted_2020wcf]].
2. **One reachable session-handle lever found:**
`StatusService.GetHistorianConsoleStatus(strHandle)` **SUCCEEDS** with
the session string handle (uppercase Open2 GUID) — no console handle
needed — and returns `uiConsoleStatus = 3` at idle. This is the only
SF-adjacent signal reachable from the managed client. **Its enum
semantics are unknown** (3 = presumably "running/normal"); whether it
shifts when SF is actively storing is the open question.
3. `StatusService.GetHistorianInfo(strHandle, btRequest)` → `bSuccess=
false` for every `btRequest` candidate (empty / `u32(0)` / ascii+utf16
`"StoreForward"`); its request framing is not yet known. Lower-yield
than `GetHistorianConsoleStatus`; revisit only if needed.
**Net idle-baseline conclusion.** R4.3's clean direct route
(`GetSFParameter` / `GetRemainingSnapshotsSize`) is **blocked behind the
D2 storage-engine console pipe**, exactly like R4.2 — a pure managed
client cannot open the console session those ops require. The *only*
reachable SF-adjacent signal is `GetHistorianConsoleStatus` → a status
uint. Two paths forward:
- **(a) Ship a measured idle-state only. — SHIPPED + LIVE-VERIFIED 2026-06-21.**
`HistorianGrpcStatusClient.GetStoreForwardStatusAsync` opens a session,
calls `GetHistorianConsoleStatus`, and returns
`HistorianStoreForwardStatus` all-false but *measured*: it actually
contacts the server and reports `ErrorOccurred=true` (with the underlying
error) when the server is unreachable / the console-status call fails —
strictly better than the blind hardcoded synthesis, which never contacts
the server. Routed via `Historian2020ProtocolDialect.GetStoreForwardStatusAsync`
when `Transport == RemoteGrpc` (non-gRPC keeps the synthesized fallback).
Gated live test `HistorianGrpcIntegrationTests.GetStoreForwardStatusAsync_OverGrpc_ReturnsMeasuredIdleState`
passes against the real 2023 R2 server. `Storing`/`Pending`/`DataStored`
magnitude is intentionally NOT surfaced — it lives behind the D2 wall (see
path (b)).
- **(b) Full success criteria (§6) stay blocked** on the D2 console-pipe
wall. Decoding the active-SF `uiConsoleStatus` value and any
`GetSystemParameter` SF keys still needs the invasive force-SF capture
on a sacrificial Historian — and even then `Storing`/`DataStored`
magnitude is only available via the D2-gated `GetRemainingSnapshotsSize`.
Probe code: `src/AVEVA.Historian.Client/Grpc/HistorianGrpcStoreForwardStatusProbe.cs`,
CLI `grpc-sf-status-probe <host> [port] [--tls] [--dnsid <n>] [--write-session]`.
Writes nothing; releases any console session immediately.
+160 -671
View File
@@ -1,241 +1,133 @@
# Plan: Reverse-Engineering Write Commands # Plan: Reverse-Engineering Write Commands
Status: **PHASE 2 PARTIALLY EXECUTED on 2026-05-04** — write-scenario ## Status (2026-05-04, post-ApplyScaling landing)
harness extension built and captured the full EnsT2(Float) wire byte
sequence against a real sandbox tag. AddS2 is blocked client-side by
"Tag not added to server" (error 168) — the native `AddStreamedValue`
refuses to send because the tag isn't in the server's session cache,
even though `EnsT2` created it in the Runtime DB. AddS2 wire bytes
**not yet captured**; needs a separate session to resolve the
post-EnsT2 registration prereq (likely RTag2 with the analog tag GUID,
mirroring the event flow's RTag2(CmEventTagId)).
## Phase 2 results Phase 2 is **complete**. The write surface that the server actually
supports for managed clients is implemented and live-verified:
**Sandbox tag created** in Runtime DB: `RetestSdkWriteSandbox`, - `EnsureTagAsync` for analog tags (Float, Double, Int2, Int4, UInt4),
wwTagKey=240, DateCreated=2026-05-04 07:49:50. Single dedicated tag with optional `ApplyScaling=true` for distinct MinRaw/MaxRaw
per safety §1; no other tags touched. persistence (`AnalogTag.Scaling=1`).
- `DeleteTagAsync` for any tag created via the SDK.
**`tools/AVEVA.Historian.NativeTraceHarness/Program.cs` extended** with Architecturally blocked / out of scope:
`--scenario write`:
- New args: `--write-sandbox-tag <name>` (default - **`AddS2` (write samples)** — server's runtime cache only ingests
`RetestSdkWriteSandbox`; refuses any name that doesn't start with from configured IOServers / Application Server pipelines, not from
`RetestSdkWrite`), `--write-value <numeric>` (default 42.5), client-only `AddTag` flows. The native wrapper hits the same wall;
`--write-data-type <name>` (default Float), `--write-delete-after` this is a server architecture decision, not a protocol gap.
(best-effort cleanup). - **Discrete / String / Int1 / Int8 / UInt8 tag types** — fail at
- Toggles `ConnectionArgs.ReadOnly` to false when scenario is `write` native `HistorianAccess.AddTag` before any wire bytes leave the
(otherwise the connection rejects writes with error 132 "Operation client. Likely require a different code path (`AddTagExtendedProperties`
is not enabled"). or pre-population via SMC); not investigated.
- Calls `ArchestrA.HistorianAccess.AddTag` (drives `EnsT2` on the wire),
then `ArchestrA.HistorianAccess.AddStreamedValue` (would drive
`AddS2` but currently aborts client-side at error 168).
- Resolves the actual `wwTagKey` via SQL when `AddTag` returns 0
because the tag already exists from a prior session.
- Public `AddStreamedValue` overload selector: instance method whose
signature is `(HistorianDataValue, …, HistorianAccessError&)`
picks the simplest dispatcher that's actually reflectable (the
4-param impl is private and not visible to reflection).
**Captures landed** at This plan covers the residual workstreams.
`artifacts/reverse-engineering/instrumented-wcf-writemessage-writes/bothmessage-write-capture-latest.ndjson`
(46 records: 23 outgoing + 23 incoming). Same priming chain as the
event flow:
``` ## Workstreams
Hist.GetV → Hist.GetI ×2 → Hist.ValCl ×2 → Hist.Open2 →
Stat.GetV ×2 → Stat.GETHI ×2 → Hist.UpdC3 →
Stat.GetSystemParameter ×7 → Trx.GetV → Stat.GetV → Retr.GetV →
Hist.EnsT2(Float) → Hist.Close2
```
No `RTag2`. The chain identical to the event flow except the EnsT2 ### A. Documentation closeout
payload is the analog CTagMetadata instead of the event one, and there
is NO RTag2 between Open2 and EnsT2 (events used RTag2 to register
`CmEventTagId`).
**Native EnsT2(Float) request body** (record 42, 322 bytes total; the Status docs across the repo still describe write commands as
146-byte CTagMetadata `InBuff` payload is the new evidence target): "in progress" or speculate about a non-existent `UpdateTags`
operation. Closing those out so future agents don't re-walk the
same dead ends.
```text | Step | File | Action |
67 03 00 01 00 00 00 04 C6 02 01 00 00 00 00 00 |---|---|---|
00 00 00 00 00 00 00 00 00 00 00 09 15 00 52 65 | A1 | `docs/reverse-engineering/handoff.md` | Add a "Write-flow status" note marking EnsT2/DelT live; remove stale write-blocker callouts |
74 65 73 74 53 64 6B 57 72 69 74 65 53 61 6E 64 | A2 | `docs/reverse-engineering/implementation-status.md` | Flip EnsT2 / DelT rows from "out of scope" to "implemented"; add ApplyScaling row |
62 6F 78 FF FF FF FF FF FF FF FF FF FF FF FF FF | A3 | `docs/reverse-engineering/wcf-contract-evidence.md` | Add evidence rows for `EnsT2(analog)` and `DelT` pointing at the captured fixtures |
FF FF FF 09 18 00 53 44 4B 20 77 72 69 74 65 2D | A4 | `README.md` | Operation-status table reflects the two write ops |
52 45 20 73 61 6E 64 62 6F 78 20 74 61 67 09 04
00 4D 44 41 53 02 01 01 00 00 00 01 E8 03 00 00
D6 00 0E 4F BC DB DC 01 1A 03 09 04 00 74 65 73
74 10 27 00 00 00 00 00 00 00 00 F0 3F FE 00 01
01 01
```
Visible fields (still being decoded against the ### B. EnsT2 idempotency / update behavior
`CTagUtil.ConvertTagMetadataToHistorianTag` IL at token `0x060055CE`):
- `09 15 00 RetestSdkWriteSandbox` (compact ASCII tag name, len 21) We don't currently know what happens when `EnsureTagAsync` is called
- 16 bytes of `FF` — possibly a placeholder/sentinel for `CommonArchestraEventTypeId`-equivalent that's not used for analog against a tag name that already exists with different fields. Three
- `09 18 00 SDK write-RE sandbox tag` (compact ASCII description, len 24) plausible outcomes: server errors, server silently updates, server
- `09 04 00 MDAS` (compact ASCII metadata provider) no-ops. This affects how callers should think about the API
- `09 04 00 test` (compact ASCII engineering unit) (create-only vs upsert).
- `0E 4F BC DB DC 01 1A 03` byte-pattern looks like an Int64 FILETIME (date-created ~2026)
- `10 27 00 00` = uint32 0x2710 = 10000 (storage-related)
- `00 00 00 00 00 00 F0 3F` = double 1.0 (likely IntegralDivisor or similar scaling)
- `FE 00 01 01 01` = trailer (matches event tag's `2F 27 01 01 01` shape)
**Decoder script** at `scripts/decode-write-capture.py` for the next | Step | Action |
session. |---|---|
| B1 | Add a live integration test that calls `EnsureTagAsync` twice on the same tag name with different `MinEU/MaxEU/Description`; query SQL after each call to capture observed behavior |
| B2 | Document the observed contract in `HistorianTagDefinition` doc-comment and (if surprising) in CLAUDE.md |
## Phase 2 follow-on findings (2026-05-04, second pass) ### C. Expose currently-hardcoded CTagMetadata fields
**The AddS2 prereq is architectural, not protocol-level.** Three follow-up The serializer hardcodes `StorageRate=1000ms`, `StorageType=Cyclic`,
attempts to trigger AddS2 from the sandbox harness all hit a client-side `IntegralDivisor=1.0`, and a few flag-block bytes. The server accepts
gate before any AddS2 byte reaches the wire: those defaults so existing tests pass, but customers building tags
with non-default rates can't currently express that.
1. **TagKey synthetic→real override.** First attempt used the placeholder | Step | Field | Effort | Notes |
`TagKey=10000000` returned by `HistorianAccess.AddTag`. Native |---|---|---|---|
`AddStreamedValue` refused with error 168 "Tag not added to server". | C1 | `StorageRate` (uint32 ms) | small — wire field is already at a known offset, just plumb a parameter through | Default stays 1000ms |
The harness now ALWAYS resolves the real `wwTagKey` from | C2 | `StorageType` (Cyclic / Delta) | medium — need a comparison capture to find which byte in the flag block encodes it | Deferred unless customer asks |
`Runtime.dbo.Tag` after AddTag (logged as `TagKeyOverride: Synthetic→RealFromSql`). | C3 | `IntegralDivisor` (double) | small — wire field already known | Deferred unless customer asks |
Result: error code shifts to **129 "Tag not found in cache"**.
2. **Server-cache settle wait.** Inserted up to 8s sleep between AddTag and C1 is the only one I'm executing in this round. C2/C3 are listed
AddStreamedValue (configurable via `--write-resync-wait-seconds`). The for completeness; pick them up when there's a concrete request.
wait period contains 2× UpdC3 + 2× Trx/GetV keep-alives but no
server-side cache update — error 129 persists.
3. **Fresh process / fresh connection.** Skipped AddTag entirely ### D. Deferred — no current evidence or customer ask
(`--write-skip-add-tag`) and ran AddStreamedValue alone against the
already-existing sandbox tag. New native client instance, new
client-side cache, new server session. **Same error 129 — no AddS2
bytes sent on wire.** Capture confirms 44 records ending in Close2.
**Interpretation.** The Historian engine's runtime tag cache only | ID | Item | Why deferred |
ingests tags from configured IOServers / Application Server data pipelines, |---|---|---|
not from `HistorianAccess.AddTag`-only client flows. `HistorianAccess.AddTag` | D1 | `AddTagExtendedProperties` / `DeleteTagExtendedProperties` | No wire captures yet; no customer ask |
populates `Runtime.dbo.Tag` (we confirmed wwTagKey=240 was created) but | D2 | `AddRevisionValuesBegin/Value/End` (revision-write path) | Multi-step capture needed against an existing historized tag; complex; no customer ask |
does not register the tag with the live cache that `AddStreamedValue` | D3 | Discrete/String/Int1/Int8/UInt8 EnsT2 root cause | Native `AddTag` fails for these — likely requires an entirely different code path; would need a fresh capture and IL walkthrough |
checks. That registration happens server-side when an upstream data
producer (an OPC driver, the AnE event subsystem, the Application Server
attribute store, etc.) claims the tag.
For SDK purposes this means **`WriteValueAsync` cannot be implemented as ## Parallelism
a generic client API against this server architecture.** The SDK's writeable
surface is realistically:
-`EnsureTagAsync` (drives EnsT2 — 146-byte payload captured) | Track | Files touched | Conflicts with |
-`DeleteTagAsync` (drives DelT — not yet captured but should be straightforward) |---|---|---|
-`WriteValueAsync` — won't work as designed; the server gates the | A1 | `docs/reverse-engineering/handoff.md` | none |
data path on tags being live in its in-memory cache | A2 | `docs/reverse-engineering/implementation-status.md` | none |
-`WriteRevisionAsync``HistorianAccess.AddRevisionValuesBegin/Value/End` | A3 | `docs/reverse-engineering/wcf-contract-evidence.md` | none |
may use a different code path (intended for editing existing historized | A4 | `README.md` | none |
data); needs a separate capture against an existing tag with stored history | B | `tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs`, `src/AVEVA.Historian.Client/Models/HistorianTagDefinition.cs` | C1 (same `HistorianTagDefinition` file) |
| C1 | `src/AVEVA.Historian.Client/Models/HistorianTagDefinition.cs`, `src/AVEVA.Historian.Client/Wcf/HistorianTagWriteProtocol.cs`, `src/AVEVA.Historian.Client/Wcf/HistorianWcfTagWriteOrchestrator.cs`, `tests/AVEVA.Historian.Client.Tests/HistorianTagWriteProtocolTests.cs`, `tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs` | B (same `HistorianTagDefinition` and integration-test file) |
Phase 2 effective deliverables: **Concurrency-safe groupings:**
- ✅ NativeTraceHarness `--scenario write` extension - A1A4 are pairwise independent and pairwise independent of B and C1 → can be assigned to four agents in parallel.
- ✅ EnsT2(Float) 146-byte CTagMetadata wire bytes - B and C1 both touch `HistorianTagDefinition.cs`. Sequence them: B first (it just adds a doc-comment) then C1 (which adds a field).
- ✅ Sandbox tag `RetestSdkWriteSandbox` in Runtime DB (wwTagKey=240)
- ⏸ AddS2 — blocked architecturally; **not just a protocol gap**
- ⏸ DelT — not yet captured (need `--write-delete-after` run)
- ⏸ Revision write path — separate capture needed against a historized
tag
## Phase 3 partial (2026-05-04) — EnsureTagAsync live, DeleteTagAsync partial In a single-agent execution (this session), the order is: A* batched
edits → B → C1 → build/test → commit.
`HistorianTagWriteProtocol` + `HistorianWcfTagWriteOrchestrator` + ## Success Criteria
`HistorianClient.EnsureTagAsync`/`DeleteTagAsync` landed:
- `HistorianTagDefinition` public model (TagName/Description/EngineeringUnit/ - A1A4: documentation reflects the actual current write surface and
DataType/MinEU/MaxEU; only `Float` data type currently supported live). removes references to non-existent operations (`UpdateTags`).
- `HistorianTagWriteProtocol.SerializeAnalogCTagMetadata` — produces 146-byte - B: a passing live test that asserts the observed double-EnsT2
payload byte-for-byte identical to the captured native EnsT2(Float) request. behavior, plus a doc-comment update on `HistorianTagDefinition`.
- `HistorianTagWriteProtocol.SerializeDeleteTagNames` — `[ushort 0x6751, - C1: `HistorianTagDefinition.StorageRateMs` field exposed, default
ushort 1, uint count, per-tag (uint charCount + UTF-16 chars)]`. preserves existing wire output, golden test for a non-default rate,
- `HistorianWcfTagWriteOrchestrator` — both EnsT2 and DelT run the full live test that creates a tag with a non-default rate and asserts
Stat-priming chain captured for the analog flow (UpdC3 + Stat.GetV ×3 + via SQL that `Tag.CurrentEditorUserKey` (or the storage-rate column,
Stat.GETHI ×2 + 7× GetSystemParameter + Trx.GetV + Retr.GetV). TBD) reflects the value.
- New tag-origin marker `0xC7` added to `MapDataType` (SDK-created tags have - All 165+ tests still pass; no regression in existing live tests.
byte 1 = 0xC7, distinct from 0xCF system / 0xC3 MDAS-routed).
Golden-byte tests (5): EnsT2(Float) byte-for-byte match against the captured ---
146-byte fixture; DelT(single tag) byte-for-byte; DelT(multi-tag); empty list
throws; different-inputs-produce-different-bytes.
Live integration test ## Appendix: Prior Phase Notes
(`EnsureTagAsync_AndDeleteTagAsync_RoundTrip_AgainstLocalHistorian`,
gated by `HISTORIAN_WRITE_SANDBOX_TAG=RetestSdkWriteSandbox`): EnsureTagAsync
followed by GetTagMetadataAsync confirms the sandbox tag is created in
the Runtime DB. Test passes 130/130 in the full suite.
**Known DelT gap.** SDK's DeleteTagAsync currently returns true but the The historical phase logs that drove the implementation are preserved
server-side cascading deletion does not always complete — the row remains below for context. They describe the path from "write surface
in `Runtime.dbo.Tag` even after the call returns. The captured native flow's unknown" to "write surface implemented and live-verified" as it
DelT removes the tag cleanly (verified via the harness with unfolded through 2026-05-04. Anything in this appendix that contradicts
`--write-delete-after`), so something the native code does between or the current Status section above is superseded.
around the WCF DelT call is missing from our orchestrator. The harness
cleanup path remains the documented workaround for sandbox housekeeping.
## DelT investigation findings (2026-05-04) ### Phase 1 findings (recorded, not implementing)
Investigation step 1 — wire-byte parity check: the captured native DelT #### §3.4 ModifyData/DeleteData — ELIMINATED FROM SCOPE
request sends `ref` input values `statusSize=1` + `status=null` (encoded as
`.nil` on the wire). My SDK was passing `statusSize=0` + `status=[]` (empty
byte array). Updated SDK to send the native-matching values.
Investigation step 2 — verified DelT still doesn't work standalone: with
the ref-input fix, DelT now returns `false` (not `true`-and-no-effect).
Tag continues to persist in `Runtime.dbo.Tag`. So the wire-byte parity
fix moved the symptom but didn't resolve the root cause.
Investigation step 3 — discovered EnsureTagAsync is **also** silently
broken: byte-for-byte wire matches captured native EnsT2 (golden test
passes), but the call returns false and does NOT create the tag in the
DB. The earlier "EnsureTagAsync round-trip test passing" was relying on
the persistent tag from the broken DelT — a false positive.
Two distinct issues remain:
- EnsT2 silently fails server-side (returns false; no tag created)
- DelT returns false even with native-matching wire bytes; needs deeper
investigation (likely the SDK's WCF channel state vs the native
HistorianAccess instance state)
Diagnostic tooling for next session: write a custom
`IClientMessageInspector` for the SDK's WCF channel that captures
outgoing DelT bytes to a file. Compare byte-for-byte against the
captured native DelT (offset by offset, not just per-field) to isolate
the difference.
## Phase 2 remaining work (revised — narrower scope)
1. Decode the 146-byte EnsT2(Float) CTagMetadata against the IL of
`CTagUtil.ConvertTagMetadataToHistorianTag` (token `0x060055CE`),
then implement `HistorianAddTagsProtocol.SerializeAnalogCTagMetadata`.
Same approach for discrete/string variants — capture each by passing
`--write-data-type Discrete` / `String` to the harness.
2. Capture DelT wire bytes by running the harness with
`--write-delete-after`.
3. Implement public `EnsureTagAsync` + `DeleteTagAsync` only. **Drop
`WriteValueAsync` from this plan.**
4. (Stretch) probe `AddRevisionValuesBegin/Value/End` against a tag that
IS in the server cache (e.g., SysTimeSec) to see whether the revision
path bypasses the cache check.
`WriteValueAsync` is now an OPEN QUESTION: is the only viable path for
client-driven writes the AVEVA REST API or the Application Server SDK?
File a separate plan for that investigation if SDK consumers actually
need data-write support.
## Phase 1 findings (recorded here, not implementing)
### §3.4 ModifyData/DeleteData — ELIMINATED FROM SCOPE
`methods aahClientManaged.dll` returns no managed wrapper for any of: `methods aahClientManaged.dll` returns no managed wrapper for any of:
`EditValue`, `ModifyValue`, `EditData`, `DeleteData`, `ModifyData`, `EditValue`, `ModifyValue`, `EditData`, `DeleteData`, `ModifyData`,
`OverwriteData`. Per the plan's §3.4 disposition rule, this op is `OverwriteData`. Per the plan's §3.4 disposition rule, this op is
REST-only / SMC-only and remains out of scope for the SDK. REST-only / SMC-only and remains out of scope for the SDK.
### §4.a Native serializers identified (token IDs for future Phase 2) #### §4.a Native serializers identified
The wrapper does have managed-public write API: The wrapper does have managed-public write API:
@@ -246,494 +138,91 @@ The wrapper does have managed-public write API:
| `ArchestrA.HistorianAccess.AddNonStreamedValue` | `0x0600618F/90` (2 overloads) | Pushes one timestamped value, non-stream mode | | `ArchestrA.HistorianAccess.AddNonStreamedValue` | `0x0600618F/90` (2 overloads) | Pushes one timestamped value, non-stream mode |
| `ArchestrA.HistorianAccess.DeleteTags` | `0x060061A4` | Removes tags (drives `DelT`) | | `ArchestrA.HistorianAccess.DeleteTags` | `0x060061A4` | Removes tags (drives `DelT`) |
| `ArchestrA.HistorianAccess.AddVersionedStreamedValue` | `0x0600616F` | Pushes versioned value (rev edit) | | `ArchestrA.HistorianAccess.AddVersionedStreamedValue` | `0x0600616F` | Pushes versioned value (rev edit) |
| `ArchestrA.HistorianAccess.AddRevisionValuesBegin/Value/End/AddRevisionValues` | `0x06006175-77, 0x0600617F` | Multi-row revision write (replaces `ModifyData` use case) | | `ArchestrA.HistorianAccess.AddRevisionValuesBegin/Value/End/AddRevisionValues` | `0x06006175-77, 0x0600617F` | Multi-row revision write |
So even though the engine doesn't expose `ModifyData` over WCF, the Native serializer for `EnsT2`:
**revision-write path** (`AddRevisionValuesBegin → AddRevisionValue * N → `<Module>.CTagUtil.ConvertTagMetadataToHistorianTag` at token
AddRevisionValuesEnd`) covers the bulk-modify use case. This is a NEW `0x060055CE`. WCF wrapper for `AddS2`:
discovery worth folding into the Phase 2 scope. `<Module>.CHistoryConnectionWCF.AddStreamValuesToHistorian` at token
`0x0600404C`.
Native serializer for `EnsT2(analog/discrete/string)`: ### Phase 2 follow-on findings (2026-05-04, second pass)
**`<Module>.CTagUtil.ConvertTagMetadataToHistorianTag`** at token
`0x060055CE` (412 IL instructions, 10 locals). Calls `CTagMetadata.GetUnit`,
`GetMessage0`, `GetMessage1`, `GetMaxLength`, `GetMinRaw`, `GetMaxRaw`,
`GetMinEU`, `GetMaxEU`, `GetIntegralDivisor`, `GetDefaultTagRate`,
`GetRolloverValue`, plus `CDataType.IsAnalog/IsWideString/GetRawType/
GetTagType` — i.e. every field the analog `CTagMetadata` shape would
need is wired through this method. Decoding it line-by-line **OR**
capturing live wire bytes against a sandbox tag are the two ways
forward.
WCF wrapper for `AddS2`: **`<Module>.CHistoryConnectionWCF.AddStreamValuesToHistorian`** **The AddS2 prereq is architectural, not protocol-level.** Three
at token `0x0600404C`. Confirms the on-wire shape is follow-up attempts to trigger AddS2 from the sandbox harness all hit
`IHistoryServiceContract2.AddStreamValues2(string handle, byte[] pBuf, a client-side gate before any AddS2 byte reaches the wire:
out byte[] errorBuffer)` — matches our existing contract. Handle is
the same Open2 v6 session GUID we already extract.
### Phase 2 chicken-and-egg resolved 1. TagKey synthetic→real override: even with the real `wwTagKey` the
server returns error 129 "Tag not found in cache".
2. Server-cache settle wait of 8s: error 129 persists.
3. Fresh process / fresh connection (skip AddTag): error 129; no AddS2
bytes sent on wire.
Per §5 ordering: §3.1 (EnsT2) must come before §3.2 (AddS2) because The Historian engine's runtime tag cache only ingests tags from
AddS2 needs an existing tag. The sandbox tag itself is created BY configured IOServers / Application Server pipelines, not from
the first §3.1 EnsT2 test. So the very first write-flow run creates `HistorianAccess.AddTag`-only flows. `WriteValueAsync` cannot be
`RetestSdkWriteSandbox`. No SMC required — the chain is closed. implemented as a generic client API against this server architecture.
### Open question (was §8.6) answered ### Phase 2 results (write captures + EnsureTagAsync/DeleteTagAsync)
The wrapper exposes `AddStreamedValue` AND `AddNonStreamedValue`. EnsT2 + DelT priming chain captured (no `RTag2` between Open2 and
The latter is the documented path for backfilling values older than EnsT2):
`RealTimeWindow`. So the SDK should expose both modes, not just
`AddStreamedValue`. Update the success criteria for `AddS2`
accordingly.
### Phase 2 next steps (NOT EXECUTED in this session)
1. Extend `tools/AVEVA.Historian.NativeTraceHarness/Program.cs` with a
`--scenario write` that calls `HistorianAccess.AddTag` (creating
`RetestSdkWriteSandbox` if absent) followed by
`HistorianAccess.AddStreamedValue`. New args:
`--write-sandbox-tag <name>` (default: `RetestSdkWriteSandbox`),
`--write-value <numeric>`, `--write-data-type analog|discrete|string`.
2. Run the harness with `instrument-wcf-writemessage` +
`instrument-wcf-readmessage` instrumented copies of
`aahClientManaged.dll` to capture the full write flow.
3. Decode `EnsT2(analog)` `InBuff` bytes against the IL of
`CTagUtil.ConvertTagMetadataToHistorianTag` (token `0x060055CE`).
4. Decode `AddS2` `pBuf` bytes against the IL of
`CHistoryConnectionWCF.AddStreamValuesToHistorian` (token
`0x0600404C`).
5. Implement `WriteValueAsync`, `EnsureTagAsync`, `DeleteTagAsync` per
§4.e of the original plan; live tests gated by
`HISTORIAN_WRITE_SANDBOX_TAG`.
Phase 2 was deferred because (a) it requires extending the harness
(non-trivial scaffolding) and (b) per safety §1, even sandbox-tag
writes warrant explicit operator approval before the first run. The
operator decides whether to proceed; if yes, the instructions above
are executable as-is.
---
Original plan content below.
## 1. Goal
"Write commands work" means the production SDK at
`src/AVEVA.Historian.Client/` performs these operations end-to-end
against a live AVEVA Historian, with parsed responses, golden-byte
unit tests, and gated live integration tests.
In scope:
1. **`AddS2` (`IHistoryServiceContract2.AddStreamValues2`)** — push
one or more timestamped samples for an existing historized tag.
Primary use case: an OPC UA driver pushing values to the
Historian.
2. **`EnsT2` (`IHistoryServiceContract2.EnsureTags2`) for
analog/discrete/string data tags** — partially decoded for the
`CM_EVENT` AnE-event tag in
`src/AVEVA.Historian.Client/Wcf/HistorianAddTagsProtocol.cs`. The
`CTagMetadata` byte layout for `CDataType` ∈ {1, 2, 3, 4} is the
new evidence target.
3. **`DelT` (`IHistoryServiceContract2.DeleteTags`)** — needed for
safe sandbox cleanup during RE.
4. **`ModifyData` / `DeleteData`** — only if §3.4 method discovery
confirms a managed WCF op exists.
Out of scope: tag-extended-properties (`AddTEx` / `DelTep`),
`ExKey`, `SetSFP`, snapshot send (`SendSnapshotBegin/End/Snapshot`),
tag-id-pair maintenance, shard splits, flush ops, all
`IStorageServiceContract` writes (engine-internal — see §6.d), event
writes (events come from AVEVA AnE, we only read them), schema
changes (forbidden over the wire).
## 2. Safety Constraints
The Runtime DB is production data even on `localhost`. `AddS2`
writes are persistent — they go to compressed history blocks and
cannot be removed through any client-facing surface.
Hard rules:
1. **Single dedicated sandbox tag.** Add env var
`HISTORIAN_WRITE_SANDBOX_TAG = "RetestSdkWriteSandbox"`. Live
write tests refuse to run when unset, even when other
`HISTORIAN_*` vars are set.
2. **Never write to** any tag named in `HISTORIAN_TEST_TAG`,
`HISTORIAN_TAG_FILTER`, the docs, the test fixtures, or the
captured RE ndjson. The read fixture
`OtOpcUaParityTest_001.Counter` is OFF-LIMITS for writes.
3. **Documented rollback.** Every write session records its time
window to
`artifacts/reverse-engineering/write-sandbox-window-<stamp>.json`
so SQL `SELECT * FROM History WHERE wwTagKey = ? AND DateTime
BETWEEN @s AND @e` can identify exactly which rows the session
inserted. Tag rollback is via decoded `DelT` (§3.3) once
available, or manually via System Management Console until then.
4. **Time bounds on writes.** Every `AddS2` test uses
`DateTime.UtcNow` ± a small offset, so writes always land inside
the live `RealTimeWindow` / `FutureTimeThreshold` system
parameters and cannot accidentally overwrite older blocks.
5. **No customer / corporate hosts.** `localhost` only.
6. **Sanitization scan after every session:**
`rg -n "(?i)(password|credential|secret|token|<known-sensitive-host>|<known-sensitive-machine>|<known-sensitive-user>)" docs\reverse-engineering scripts tools docs\plans`.
Soft rules:
- Use a separate captures dir
(`artifacts/reverse-engineering/instrumented-wcf-writemessage-writes/`)
so write captures don't contaminate the existing read/event
ndjson.
- New integration tests follow the existing gating pattern in
`tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs`
(`Skip = ...` when env var unset).
## 3. Discovery Workstreams
### 3.1 EnsT2 for analog/discrete/string tags (priority 1)
- WCF op: `aa/Hist/EnsT2`.
- Contract:
`src/AVEVA.Historian.Client/Wcf/Contracts/IHistoryServiceContract2.cs:82-89`,
already declared with `[MessageParameter(Name = "InBuff" / "OutBuff")]`.
- Existing code: `HistorianAddTagsProtocol.SerializeCmEventCTagMetadata`
builds the `CDataType=5` (event) shape.
- Missing: the `CTagMetadata` byte layout for `CDataType ∈ {1, 2,
3, 4}` (analog double, discrete, string, analog int per the
type-code table in `data-query-request-ctor-il-latest.txt`);
whether the optional-mask `0x0086` and the 5-byte trailer
`2F 27 01 01 01` change per type; analog engineering-units / range
/ deadband fields (likely populate the bytes that are zero in the
event-tag fixture).
### 3.2 AddS2 stream values (priority 1)
- WCF op: `aa/Hist/AddS2`.
- Contract:
`src/AVEVA.Historian.Client/Wcf/Contracts/IHistoryServiceContract2.cs:75-80`,
already has `[MessageParameter(Name = "pBuf")]`. **Audit
requirement:** verify against `ildasm aahClientAccessPoint.exe`
that `Handle` and `errorBuffer` parameter names also match — the
handoff's parameter-name-mismatch class has bitten ~30 ops.
- Missing: entire `pBuf` byte layout (likely `UInt16 version + UInt32
sampleCount + N × {tagId GUID, FILETIME, qualityByte, value typed
by CDataType}`); whether `Handle` is the same Open2 v6 session GUID
as `UpdC3`/`RTag2`/`EnsT2`; the auth-chain prereqs (event flow
needed Stat priming + Trx/Stat/Retr `GetV` between RTag2 and EnsT2;
writes may have a different chain); success vs error response
shape.
### 3.3 DelT tag deletion (priority 2 — needed for safe RE)
- WCF op: `aa/Hist/DelT`.
- Contract:
`src/AVEVA.Historian.Client/Wcf/Contracts/IHistoryServiceContract2.cs:21-30`.
- Missing: `tagNames` byte layout (likely length-prefixed
compact-ASCII per the handoff convention); whether server refuses
to delete tags with stored history or cascades; whether `DelT` is
sufficient to fully unregister or leaves orphan rows in
`Runtime.dbo.Tag`.
### 3.4 ModifyData / DeleteData (priority 3 — exists?)
No corresponding WCF op is currently declared. **First step:** static
inspection to confirm any managed wrapper exists.
```powershell
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- methods current\aahClientManaged.dll EditValue
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- methods current\aahClientManaged.dll ModifyValue
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- methods current\aahClientManaged.dll EditData
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- methods current\aahClientManaged.dll DeleteData
```
If no managed wrapper exists, this op is REST-only / SMC-only —
mark as **out of scope** in this doc. Otherwise decode like
§3.1/§3.2.
Parallelism: 3.1 and 3.3 can be developed in parallel because the
operator can create the sandbox tag manually via SMC while SDK code
is being written. 3.2 cannot meaningfully proceed until 3.1 (or the
manual tag) exists. 3.4 method discovery is cheap and may eliminate
its own scope.
## 4. RE Steps in Execution Order
For each workstream above, run these five steps. Mirrors the read
+ event flows that recovered the existing protocol.
### 4.a Static method discovery
Find the native serializer:
```powershell
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- methods current\aahClientManaged.dll AddS
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- methods current\aahClientManaged.dll EnsureTag
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- methods current\aahClientManaged.dll DeleteTag
```
Dump IL for each method of interest:
```powershell
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- dnlib-method --instructions current\aahClientManaged.dll <Type::Method>
```
Save sanitized excerpts to
`docs/reverse-engineering/dnlib-<op>-il-latest.txt`.
### 4.b Wire-byte capture for the request
Same IL-rewrite tooling that captured the 27 outgoing event calls:
```powershell
$captureDir = "artifacts\reverse-engineering\instrumented-wcf-writemessage-writes"
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- instrument-wcf-writemessage current\aahClientManaged.dll "$captureDir\aahClientManaged.dll"
Copy-Item -Force "$captureDir\aahClientManaged.dll" "$captureDir\current-copy\aahClientManaged.dll"
$env:AVEVA_HISTORIAN_RE_CAPTURE = (Resolve-Path $captureDir).Path + "\writemessage-capture-write-latest.ndjson"
```
A new harness scenario `--scenario write` needs to be added to
`tools/AVEVA.Historian.NativeTraceHarness` to drive the native
wrapper's `AddStreamValues2` against the sandbox tag. Suggested
new args: `--write-sandbox-tag`, `--write-value`.
### 4.c Wire-byte capture for the response
Symmetric `instrument-wcf-readmessage`:
```powershell
dotnet run --no-build --project tools\AVEVA.Historian.ReverseEngineering -- instrument-wcf-readmessage current\aahClientManaged.dll "$captureDir\aahClientManaged.dll"
```
The success response for `AddS2` is just `<AddS2Result>true</…>` +
empty `errorBuffer`. **Capture at least one negative case** (write
to non-existent tag, or write with malformed CDataType) so the
orchestrator can surface diagnostics like
`HistorianWcfEventOrchestrator.LastErrorBufferDescription`.
### 4.d Decode against IL
Strip SOAP/MDAS envelope; align byte offsets against the native
serializer IL from 4.a (the `ldc.i4 / call WriteByte` sequence
makes field order and constants explicit); cross-reference the
`CDataType` table from `data-query-request-ctor-il-latest.txt` to
interpret typed value bytes; write a parser-and-builder pair and
verify against the captured bytes before committing.
### 4.e Implement managed serializer + tests
New code under `src/AVEVA.Historian.Client/Wcf/`:
- `HistorianAddStreamValuesProtocol.cs` — `Serialize(...)` returns
`byte[] pBuf`, mirroring `HistorianAddTagsProtocol`.
- Extend (or split) `HistorianAddTagsProtocol` for the analog /
discrete / string `EnsT2` shapes.
- `HistorianWcfWriteOrchestrator.cs` — chains `Hist.GetV →
Hist.ValCl × 2 → Hist.Open2 → UpdC3 → priming chain (TBD per
§3.2) → AddS2 loop → Close2`.
Public surface on `HistorianClient`:
- `WriteValueAsync(tag, value, timestampUtc, quality)`
- `WriteValuesAsync(IReadOnlyList<HistorianSampleWrite>)`
- `EnsureTagAsync(HistorianTagDefinition)`
- `DeleteTagAsync(string tagName)`
Until evidence supports each path, throw
`ProtocolEvidenceMissingException` (mirrors the existing read
guardrail).
Unit tests under `tests/AVEVA.Historian.Client.Tests/Wcf/`:
- `WcfAddStreamValuesProtocolTests` — golden-byte tests for one
analog, one discrete, one string write.
- `WcfEnsureTagsProtocolTests` — golden-byte tests for the
analog/discrete/string `CTagMetadata` shapes.
- Extend `ProtocolGuardrailTests` so any not-yet-implemented write
path still throws `ProtocolEvidenceMissingException`.
Live integration tests in `HistorianClientIntegrationTests.cs`,
gated on `HISTORIAN_WRITE_SANDBOX_TAG`:
`WriteValueAsync_WithinDocumentedWindow_PersistsToHistorianDb`
writes a unique value, reads it back via `ReadRawAsync`, and
verifies via direct `sqlcmd` to the History extension table.
## 5. Order of Operations
``` ```
3.4 method discovery (cheap; may eliminate scope) Hist.GetV → Hist.GetI ×2 → Hist.ValCl ×2 → Hist.Open2 →
Stat.GetV ×2 → Stat.GETHI ×2 → Hist.UpdC3 →
Stat.GetSystemParameter ×7 → Trx.GetV → Stat.GetV → Retr.GetV →
3.1 EnsT2 (analog/discrete/string) ──► sandbox tag exists Hist.EnsT2 → Hist.Close2
├─────────────────────────────┐
▼ ▼
3.2 AddS2 (priority 1) 3.3 DelT (sandbox cleanup)
3.4 ModifyData/DeleteData (only if 3.4 confirmed scope)
public surface, golden-byte tests, integration tests
``` ```
3.2 is the headline win and depends only on 3.1 (or a manually Open2 with `NativeIntegratedWriteEnabledConnectionMode = 0x401`
created sandbox tag). 3.3 must land before any commit that (Process | Write | IntegratedSecurity) is required — read-mode
programmatically creates new tags; until then, manual SMC deletion `0x402` makes the server return err 132 `OperationNotEnabled`
is the documented rollback. silently. The analog Float `CTagMetadata` payload is 144 bytes with a
leading `0x4E` marker byte and a 2-byte trailer `FE xx` where the
second byte is the ApplyScaling flag (`00` = false, `01` = true).
## 6. Risks and Mitigations ### ApplyScaling resolution (2026-05-04)
### 6.a Auth chain may differ for writes Earlier docs claimed "MinRaw is mirrored to MinEU — server quirk,
not SDK bug". That conclusion was based on tests that always set
ApplyScaling=false on the native side. Re-running with
`set_ApplyScaling(true)` on the harness and capturing wire bytes for
both values revealed:
Reads use `Hist.Open2(ConnectionMode = 0x402)`. Events use the same - ApplyScaling=false → trailer = `FE 00` → server mirrors MinRaw→MinEU,
`0x402` plus a Stat-priming chain. Writes may need a different sets `AnalogTag.Scaling=0`
mode (the handoff notes `0x501` was an unverified guess for - ApplyScaling=true → trailer = `FE 01` → server persists distinct
events; writes may legitimately need `0x401` or another value). MinRaw/MaxRaw, sets `AnalogTag.Scaling=1`
Mitigation: capture the *full* WriteMessage sequence for a native The `IHistoryServiceContract2` surface has **no `UpdateTags`
write session (not just `AddS2`) to see what `Open2` payload and operation**. Distinct MinRaw/MaxRaw persistence is achieved entirely
priming calls the native wrapper sends. by toggling that one byte in the EnsT2 payload. The SDK now exposes
this via `HistorianTagDefinition.ApplyScaling`.
### 6.b Server-side session-table requirement Capture artifacts:
`artifacts/reverse-engineering/apply-scaling-experiment/enst2-applyscaling-{false,true}.ndjson`.
Writes may require `RTag2` after `EnsT2` and before `AddS2` (the ### Original goal section (preserved for historical reference)
event flow needs `RTag2(CmEventTagId)`). The "tag identifier" the
server returns from `EnsT2` may differ from the GUID the client
seeded.
Mitigation: capture the analog `EnsT2` `OutBuff` (event flow's was "Write commands work" was originally defined as the four ops:
a 45-byte echo) and verify whether subsequent `AddS2` payloads `EnsT2`, `AddS2`, `DelT`, and `ModifyData/DeleteData`. The realized
reference the client-seeded GUID, the server-returned GUID, or a scope is `EnsT2 + DelT` only. AddS2 is permanently blocked by
numeric `wwTagKey`. SQL ground truth: `SELECT TagName, wwTagKey server architecture; ModifyData/DeleteData were eliminated by
FROM Tag WHERE TagName = '...'`. static analysis (no managed wrapper exists). The
`AddRevisionValuesBegin/Value/End` chain remains a stretch goal
(item D2 in the current plan) — it was never investigated because
no SDK consumer has asked for revision writes.
### 6.c Silent-success failure mode ### Original safety rules (still applicable)
`AddS2` may return `true` but no row appears in the History - Single dedicated sandbox tag per RE session, name must start with
extension table — the engine silently drops samples outside the `RetestSdkWrite`.
`FutureTimeThreshold` / `RealTimeWindow` system parameters (which - Never write to any tag named in `HISTORIAN_TEST_TAG`,
the event flow now reads). `HISTORIAN_TAG_FILTER`, the docs, or the captured RE ndjson.
- Time bounds on writes: every test uses `DateTime.UtcNow` so writes
Mitigation: always write at `DateTime.UtcNow`; cross-check with land inside the live `RealTimeWindow` / `FutureTimeThreshold`.
SQL after every test: - `localhost` only; no customer / corporate hosts.
- Sanitization scan after every session.
```sql - Write captures live in `artifacts/reverse-engineering/instrumented-wcf-writemessage-writes/`
SELECT TOP 5 DateTime, Value, QualityDetail (gitignored).
FROM History
WHERE wwTagKey = (SELECT wwTagKey FROM Tag WHERE TagName = @sandbox)
AND DateTime BETWEEN @windowStart AND @windowEnd
ORDER BY DateTime DESC;
```
Surface `FutureTimeThreshold` / `RealTimeWindow` via existing
`GetSystemParameterAsync` so failures are diagnosable.
### 6.d Storage service vs History service
`IStorageServiceContract` also exposes `AddT/AddS/AddS2/DelT`. The
working hypothesis is that `/Hist` is client-facing and `/Stor` is
engine-internal, but it's not yet verified.
Mitigation: the WriteMessage capture (§4.b) shows the actual
service path on the wire. If it goes to `/Stor`, update the
orchestrator. Do NOT preemptively implement against both.
### 6.e Parameter-name mismatches
Handoff already flagged `EnsT`, `EnsT2`, `RTag2`, `ExKey`, `StJb`,
`GtJb` for the same `inBuff`/`inputBuffer` mismatch class that
broke reads for weeks. Until each is audited against the server
contract, requests bind to null and the server NREs.
Mitigation: before the first write WriteMessage capture, run an
`ildasm` audit against `aahClientAccessPoint.exe` for the exact
parameter names of `EnsT2`, `AddS2`, and `DelT`, and reconcile
against the existing `[MessageParameter]` attributes.
### 6.f Customer-data exposure in capture files
Write captures contain the sandbox tag name and any value the test
wrote. Not secrets, but noise.
Mitigation: keep all
`instrumented-wcf-writemessage-writes/` artifacts under
`artifacts/` (already gitignored). Sanitize tag names to
`<sandbox-tag>` before committing decoded bytes into
`docs/reverse-engineering/`.
## 7. Success Criteria
Per op:
- **`EnsT2(analog)`**: `EnsureTagAsync(new HistorianTagDefinition {
Name = sandbox, DataType = Analog })` returns success;
`sqlcmd -E -S . -d Runtime -Q "SELECT TagName FROM Tag WHERE
TagName = '...'"` returns one row.
- **`EnsT2(discrete, string)`**: same shape with corresponding
`DataType`; SQL check uses `DiscreteTag` / `StringTag` view.
- **`AddS2`**: `WriteValueAsync(sandbox, 42.0, DateTime.UtcNow)`
returns success; `ReadRawAsync` returns the value;
`SELECT TOP 1 Value FROM History WHERE wwTagKey = ? AND DateTime
BETWEEN ? AND ?` returns the same value.
- **`DelT`**: `DeleteTagAsync(sandbox)` returns success and SQL
returns zero rows from `Tag`.
- **`ModifyData` / `DeleteData`**: deferred until §3.4 method
discovery confirms scope.
Cross-cutting:
- All new code in `src/AVEVA.Historian.Client/` is pure managed
.NET 10. No new P/Invoke beyond the existing `HistorianSspiClient`.
- Every new op has a golden-byte unit test.
- `dotnet test .\Histsdk.slnx --no-build --logger
"console;verbosity=minimal"` passes 100%.
- With `HISTORIAN_HOST=localhost`,
`HISTORIAN_WRITE_SANDBOX_TAG=RetestSdkWriteSandbox` set, write
integration tests pass and leave zero residue (test `Dispose`
calls `DelT` for cleanup).
- Sanitization scan returns no real secrets.
- `CLAUDE.md` "Required SDK Surface" updated to add the new write
ops — this is a SCOPE CHANGE that must land *alongside* the
evidence, not before. Do not update the SDK surface doc until
3.1 + 3.2 are at least live-test-green.
## 8. Open Questions
1. Does `AddS2` go through `/Hist` or `/Stor` on the wire?
2. Does the sandbox tag need pre-configuration via System
Management Console once before `EnsT2` will accept it from a
client (e.g. for `Storage` / `wwDomain` rows the wire protocol
may not be able to populate)?
3. What `ConnectionMode` does the native wrapper use for write
sessions — `0x402` (read mode reused), `0x401`, or something
else?
4. Does `EnsT2(analog)` require any optional Archestra
engineering-units fields, or are they purely cosmetic? Affects
how minimal `HistorianTagDefinition` can be.
5. Server-side throttles on writes (max samples per AddS2, max
calls per second) — need to surface as batching guidance?
6. What does the server return when `AddS2` is called with a
timestamp older than the tag's earliest stored block? Some
historians silently drop, some error, some accept-and-overwrite.
7. Does the SDK expose write quality as the same
`HistorianSample.Quality` enum used on reads, or a smaller
subset (good/bad)?
8. Is there a managed-side `DelT` path at all? If
`aahClientManaged` only exposes deletion via SMC, §3.3 is
"manual SMC only" and must be documented as such.
## 9. Docs To Update Once Each Workstream Lands
- `CLAUDE.md` "Required SDK Surface" — add `WriteValueAsync`,
`EnsureTagAsync`, `DeleteTagAsync` once 3.1+3.2+3.3 land.
- `AGENTS.md` "Required SDK Surface" — same; update the "alarm-event
write path is dormant" note.
- `docs/reverse-engineering/handoff.md` — add a "Write-flow prereqs"
section symmetric to the existing "Event-flow prereqs".
- `docs/reverse-engineering/wcf-contract-evidence.md` — add evidence
rows for `EnsT2(analog/discrete/string)`, `AddS2`, `DelT`.
- `docs/reverse-engineering/implementation-status.md` — flip
status from "out of scope" to "implemented".
- `README.md` — operation status table.
@@ -0,0 +1,111 @@
# Event-session reuse spike — live results
> **Question:** does the 2023 R2 historian honor REUSING one authenticated **v8 Event**
> session (ECDH `ExchangeKey` → RC4 token → `ConnectionType=Event`, then `RegisterCmEventTag`)
> across multiple `SendEvent` ops, instead of the per-op open+register the SDK does today?
> This is the precondition for amortizing the EVENT path (HistorianGateway `pending.md` A1
> broadening, Stage B0 / B1).
>
> **Verdict: GREEN — a v8 Event session reuses across sends, register-once is sufficient,
> and the amortization is ~1016×. Event READS stay gated (C2) and are not a reuse signal.**
**Date:** 2026-06-25
**Branch:** `feat/amortization-broadening`
**Server:** live 2023 R2 (`wonder-sql-vd03`), RemoteGrpc transport.
**Sandbox identity:** `HISTORIAN_EVENT_SANDBOX_TAG=HistGW.LiveTest.EventSpike` — the CM_EVENT send
buffer has **no per-tag routing field** (it registers against a fixed system tag), so the sandbox
value is stamped into the event `Type`/`SourceName`/`Namespace` + a `SpikeMarker` property as an
**identity marker**; no real tag is written or overwritten.
**Harness:** `tests/AVEVA.Historian.Client.Tests/EventSessionReuseSpikeTests.cs` driving the B0a
seams `HistorianGrpcEventWriteOrchestrator.OpenAndRegisterEventSession` (open v8 Event session +
`RegisterCmEventTag` ONCE) and `SendEventOnSession` (send only — no open/register).
---
## 1. Send reuse — GREEN
`ReusedEventSession_SendsTwice_SecondSkipsHandshake` **passed** (both runs): one
`OpenAndRegisterEventSession` then **two `SendEventOnSession` on the same v8 Event session** — both
accepted (`AddStreamValues` `BSuccess=true`).
```
open+register (ECDH handshake + RegisterCmEventTag) = 242 ms (run 1: 350 ms)
registration diag: RTag=True; EnsT=True
reused-send[0] = 23 ms, ok=True
reused-send[1] = 22 ms, ok=True
```
The server accepts the same v8 Event client handle across back-to-back sends. The session handle is
an immutable `readonly record struct (uint ClientHandle, Guid StorageSessionId)`; the send is
stateless on the client side (each call reserializes a fresh `"OS"` buffer), so nothing per-op is
baked into the handle.
## 2. Amortization — ~1016×
The open+register (P-256 ECDH `ExchangeKey` → RC4 credential token → v8 `OpenConnection`
`RegisterCmEventTag`) costs ~242350 ms and is paid **once**; a reused send is ~22 ms. So over a
burst of N sends the per-send cost collapses from ~(265 ms open + 22 ms) to ~22 ms — a ~1016× win
on the send path, same shape as the v6 read/write amortization (`handshake-reuse-spike-results.md`).
## 3. Register-once is sufficient — GREEN
`ReusedEventSession_RegisterOnce_ThenSendMany` **passed**: `RegisterCmEventTag` run **once** (inside
`OpenAndRegisterEventSession`), then **three** sends, all accepted.
```
register-once send[0] = 25 ms, ok=True
register-once send[1] = 22 ms, ok=True
register-once send[2] = 22 ms, ok=True
```
CM_EVENT registration is **session-scoped, not per-send** — the server holds the registration for the
session's lifetime. A reuse pool registers once per warm session, not per op.
## 4. Idle tolerance — survived ≥25 s (best-effort, single sample)
`ReusedEventSession_IdleSweep_BestEffort` (log-only): after a send, a **25 s idle gap**, then another
send — **the second send succeeded** (`session SURVIVED the idle gap`). Notable: the v6 read session
idle-expires at a ≥25 s gap (`handshake-reuse-spike-results.md` §3), but this v8 Event session
survived 25 s. This is a single-sample best-effort observation — a keepalive should still ping under
the ~20 s floor for safety margin until the v8 Event idle boundary is characterized more finely.
## 5. Read-after-send — GATED (C2), not a reuse signal
`ReusedEventSession_ServesReadAfterSend_BestEffort` (log-only, hard-bounded by a 5 s gRPC deadline +
an 8 s cancellation): the read-after-send on the same session **did not return data** — it cancels at
the bound:
```
read-after-send -> swallowed (RpcException Cancelled / OperationCanceled)
=> read gated/unverified over gRPC (expected)
```
This matches the pre-existing C2 gate: event **reads** over gRPC long-poll `GetNext` to a no-data
terminal and are unverified. So the spike did **not** prove a one-session-serves-both-kinds property
for reads — `SendEvent` is the only trustworthy reuse signal. (An unbounded read hung the first run;
the harness now bounds it so the spike is a clean, re-runnable record.)
---
## 6. Implications for Stage B1 (the event-pool build)
GREEN → a **separate event-session pool** (the approved B1 approach) is warranted and high-value:
1. **Amortize `SendEvent` through a bounded event-session pool.** Open+register a v8 Event session
once per warm session; lease it per send op (exclusive, like the v6 pool); reuse across a burst.
~1016× on the send path.
2. **Keep the event pool SEPARATE from the v6 pool** (B1, as approved) — different auth (ECDH/v8),
heavier re-handshake on drop, and its own idle characteristics.
3. **`ReadEvents` stays PER-CALL / gated (C2).** Reads are unverified over gRPC regardless of reuse,
so the event pool amortizes **sends only**; `ReadEvents` is unaffected by B1 and stays on the
per-call path. (This refines the design's "route SendEvent + ReadEvents through the pool": only
`SendEvent` is routed; `ReadEvents` remains per-call because it is gated, not because of reuse.)
4. **Keepalive:** ping the warm event session under the idle floor. The cheap keepalive op for the
event channel is TBD in B1 (the v6 pool uses `GetSystemParameter`; the event session's equivalent
warm-touch needs picking — likely a no-op send or a lightweight event-channel status op).
5. **Reactive re-auth:** on an expiry-looking failure, evict + full v8 re-handshake (heavier than the
v6 re-auth — one ECDH + register penalty).
**Gate decision: GREEN → HistorianGateway A1 Stage B1 (a bounded `HistorianEventSessionPool` for
`SendEvent`, default-on, parallel to the v6 `HistorianSessionPool`) is warranted and earns its own
re-planned design + plan.**
@@ -0,0 +1,505 @@
# gRPC event-query capture (2026-06-22) — the StartEventQuery request that returns rows
Captured the stock 2023 R2 client performing a **gRPC event read** that returns rows, to resolve
the open item "gRPC event ROW retrieval returns zero rows" (handoff §Current Status item 1). This
closes the capture-gate: the working request shape is now known.
## How it was captured
`tools/AVEVA.Historian.Grpc2023CaptureHarness` gained a `capture-event` scenario. It loads the
self-contained mixed-mode 2023 R2 `aahClientManaged.dll` and drives `HistorianAccess`:
```
OpenConnection(ConnectionMode=Historian /*gRPC*/, ConnectionType=Event, ReadOnly=true)
-> CreateEventQuery() // NON-null only on an Event connection
-> EventQueryArgs { StartDateTime, EndDateTime, EventCount }
-> EventQuery.StartQuery(args) // => GrpcRetrievalClient.StartEventQuery(requestBuffer)
-> loop EventQuery.MoveNext() / QueryResult// => GrpcRetrievalClient.GetNextEventQueryResultBuffer
-> EventQuery.EndQuery() -> CloseConnection
```
The existing wide-net `instrument-grpc-nonstream` IL rewrite (every `Grpc*Client` `byte[]` method)
already covers `GrpcRetrievalClient.StartEventQuery.requestBuffer` (entry) and
`GetNextEventQueryResultBuffer.result` (exit) — no new instrument command was needed. Run read-only
(non-destructive) against the live 2023 R2 server over the loopback tunnel; the rewrite + capture
NDJSON stay under `artifacts/reverse-engineering/grpc-event-capture/` (gitignored — the result
buffer carries event identity data).
Result: **50 events returned over gRPC** (Alarm.Set / Alarm.Clear rows), proving the path works when
driven through an Event connection.
## Two findings
### 1. The event read needs an **Event-type connection** (`ConnectionIndex 1`)
`HistorianAccess.CreateEventQuery()` returns `null` unless `IsEventConnectionRequested()` — i.e. the
connection was opened with `ConnectionType=Event`, which the native client routes to a *separate*
connection (ConnectionIndex 1) from the process/data path. The full captured pre-query sequence on
that connection: `OpenConnection``ExchangeKey``UpdateClientStatus``RegisterTags`(CM_EVENT) →
`EnsureTags`(CM_EVENT) → `GetHistorianInfo` + 7×`GetSystemParameter` (Stat priming) →
`StartEventQuery``GetNextEventQueryResultBuffer` (rows) → `EndEventQuery``CloseConnection`.
### 2. The working `StartEventQuery` request is **version 6**, not 5
Our SDK's `HistorianEventQueryProtocol.CreateNativeFilterAttempt` builds a **version-5** empty-filter
buffer; the stock 2023 R2 client sends **version 6**. Diffed byte-for-byte (same query window +
eventCount), the two buffers are **identical except**:
- **byte 0: version `06` vs `05`**
- **5 additional trailing zero bytes** (stock = 70 bytes, SDK v5 = 65 bytes)
The server returns rows for v6 and **zero rows for v5** (the v5 request is *accepted*
`StartEventQuery` succeeds and yields a query handle — but `GetNextEventQueryResultBuffer` then
matches nothing). Everything else is shared: the two query-window FILETIMEs, `UInt32 eventCount`,
the `UInt32 65536` buffer hint, the `"UTC"` `HistorianString`, and the `01 01000001000001 0000`
metadata-namespace block.
Captured v6 request layout (70 bytes; the FILETIMEs below are just the harness query window — no
identity data):
```
[0..1] UInt16 version = 6 // SDK currently sends 5
[2..9] Int64 startUtc (FILETIME)
[10..17] Int64 endUtc (FILETIME)
[18..21] UInt32 eventCount
[22..25] UInt32 0
[26..27] UInt16 0
[28..29] UInt16 1
[30..36] 7 bytes 0 // empty-filter block
[37..40] UInt32 65536 // buffer-size hint
[41..50] HistorianString "UTC" (UInt32 len=3 + UTF-16LE)
[51..60] 01 01 00 00 01 00 00 01 00 00 // metadata-namespace block (marker + 3 empty)
[61..69] 9 bytes 0 // terminal (SDK v5 writes only 4 here)
```
## Fix part 1 — v6 request (DONE, necessary)
`HistorianEventQueryProtocol.CreateStartEventQueryAttempts` gained a `version` parameter (default 5 =
WCF/2020; the gRPC orchestrator passes 6). v6 emits the leading `06` and the 5-byte trailing pad. The
WCF path is unchanged (v5). Golden test `Version6EmptyFilterMatchesCapturedGrpcEnvelope` pins the
envelope; 322/322 offline tests pass.
## Fix part 2 — EVENT connection (the remaining gate, NOT yet implemented)
Live validation 2026-06-22: with the orchestrator now sending v6 against the event-bearing live
server, `GetNextEventQueryResultBuffer` **still long-polls and returns zero rows** (the gated test
still throws). So **v6 is necessary but not sufficient** — the read also requires an **Event-type
connection**, which our SDK does not open.
Isolated by diffing the captured `OpenConnection.openParameters` (302 bytes, native format v8) for a
**Process** connection (`connect` scenario) vs the **Event** connection (`capture-event`): aside from
the per-session auth GUID/credential-hash regions ([22..37], [68..93], which vary between any two
sessions), the connection differs in **two clean structural bytes**:
| offset | Process | Event |
|--------|---------|-------|
| 95 | `02` | `01` |
| 96 | `00` | `01` |
These correspond to `HistorianConnectionType` (Process vs Event; the native event path runs on
`ConnectionIndex 1`). The problem: our SDK opens the session with the **2020 OpenConnection3 v6**
buffer (`HistorianNativeHandshake.BuildOpenConnection3Request`, `connectionMode 0x402`), which the
2023 R2 server accepts for reads but which carries no event-connection-type marker. `connectionMode`
is NOT the discriminator (2020 WCF event reads work with `0x402`); the native client distinguishes
event vs process via this separate `ConnectionType` field in its v8 `openParameters`.
### Diagnosis (2026-06-22): the v6 Open2 format cannot express an event connection
Decoded the native `openParameters` (302 bytes): **byte 0 = `08` (format version 8)**, then a
context GUID, username, a 26-byte session-derived region ([68..93]), machine/client-node/datasource
strings, and at **[94] `ClientType=04`** immediately followed by **[95] `ConnectionType`
(`01`=Event / `02`=Process)** + **[96] a flag (`01`/`00`)**, then the rest.
Our SDK builds the **v6** buffer (`HistorianOpen2Protocol.SerializeNativeOpenConnection3Version6`,
byte 0 = `06`): it writes `ClientType` (1 byte) **immediately followed by `ConnectionMode` (uint)**
there is **no `ConnectionType` byte at all**. The v8 format *inserts* `ConnectionType` (+flag) between
`ClientType` and the rest. So the v6 buffer the SDK sends (accepted by the 2023 R2 server for *reads*)
structurally cannot mark the connection as Event, and the server returns event rows only for an Event
connection.
Two further obstacles to simply emitting v8:
- the native client authenticated via **`ExchangeKey`** (cert path; 72-byte `btInput`/`btOutput` in
the capture) whereas the SDK's gRPC handshake uses **`ValidateClientCredential`** (Negotiate). The
v8 `openParameters` [68..93] region is session-derived and tied to that auth flow.
- `ConnectionMode` is NOT the lever (2020 WCF event reads work at `0x402`); `ConnectionType` is a
distinct field that only exists from format v8.
Also confirmed a secondary format gap: the native gRPC `EnsureTags` CM_EVENT payload is **86 bytes**
vs the SDK's `SerializeCmEventCTagMetadata` **83 bytes** (a 3-byte 2023 R2 bump, parallel to the
event-query v5→v6). This is likely benign on its own (CM_EVENT pre-exists; 2020 EnsT2 returns
benign-false yet events flow) but should be matched if the event open is ever rebuilt.
**Conclusion — the event-connection gate is NOT a tweak.** Making event rows flow over gRPC requires
the SDK to emit the native **v8 `OpenConnection` format** with `ConnectionType=Event` (a 302-byte
buffer whose layout differs from the v6 buffer and includes a session-derived auth region), and
likely to adopt the `ExchangeKey` cert auth path. That is a substantial RE+implementation effort
comparable to the original Open2 work — scoped as a follow-on, not a quick fix. Until then the gated
`ReadEventsAsync_OverGrpc_*` test correctly still pins the no-row throw, and **v6 (part 1) is retained
as the captured-correct request format** for when the open is rebuilt.
Capture artifacts (gitignored): `artifacts/reverse-engineering/grpc-event-capture/`
`event-capture.ndjson` (Event), `process-connect-2.ndjson` (Process).
## v8 `openParameters` fully decoded (2026-06-23) + the ECDH ExchangeKey finding
Full byte map of the native Event-connection `openParameters` (302 bytes; identity values
redacted — they are session-specific and sit in the gitignored capture):
```
[0] byte 0x08 format version = 8
[1] byte 0xf0 constant marker
[2..20] 19 × 0x00
[21] byte 0x01 constant marker
[22..37] 16B GUID per-session client key
[38..41] u32 username length (chars)
[42..N] UTF-16 username (HistorianString)
[..+1] u16 credential-token length (= 26 in the capture)
[..] 26B token ECDH-derived credential token <-- see below
[94] byte 0x04 ClientType (= our NativeClientType 4)
[95] byte ConnectionType 01 = Event / 02 = Process <-- THE GATE
[96] byte flag 01 (Event) / 00 (Process)
[97..] control bytes (0x03 ... small region, not fully named)
[~114..117]u32 FormatVersion=3
[..] HistorianString machine/server node name
[..] HistorianString client node name "(<ver>)"
[..] u32 session-variable (process-ish)
[..] u32 / zeros
[..] u32 datasource len
[..] UTF-16 datasource id e.g. "2023.1219.4004.5"
[270..285] 16 × 0xff ShardId (all-FF = unset; our v6 sends Empty)
[286..289] u32 client/hcal version int
[290..297] i64 FILETIME ClientTimestamp
[298..301] u32 0
```
The tail (`FormatVersion` → machine → clientNode → datasource → ShardId → version → timestamp)
is the **same `ClientCommonInfo` our v6 already emits**. The new/different parts are: version byte,
the `[1]`/`[21]` markers, the GUID position, the **26-byte credential token** (vs v6's fixed-size
block), the **`ConnectionType` byte**, and ShardId=FF.
**The auth is ECDH, not Negotiate.** The capture's `ExchangeKey` buffers begin `45 43 4b 31` =
ASCII **`"ECK1"`** + a 64-byte EC public-key point — a Diffie-Hellman key exchange — and the 26-byte
`openParameters` token is derived from it. `HistorianSecurityMode` offers only `Disabled` / `None` /
`TransportCertificate`; the harness used `TransportCertificate`, which is what drives the ECDH
`ExchangeKey`. There is **no TLS+Negotiate mode** on the native client (it couples TLS with the cert
ECDH path), so a Negotiate-auth v8 capture cannot be produced from the native client.
**Key de-risking insight:** our SDK's v6 `OpenConnection` sends a **fully zeroed** 1026-byte
credential block (`credentialBlock: new byte[1026]`) and reads still work — because authentication is
actually carried by the separate `StorageService.ValidateClientCredential` (Negotiate) handshake, not
by the bytes inside `openParameters`. By analogy the v8 `[68..93]` token may likewise be **ignorable**
once `ValidateClientCredential` has run. So the first build hypothesis (cheapest, read-only to test):
> Reuse the SDK's existing `ValidateClientCredential` handshake, then send a **v8 `OpenConnection`
> with `ConnectionType=Event` and a zeroed credential token**, and see whether the 2023 R2 server
> returns event rows.
If that works, the ECDH ExchangeKey RE is unnecessary. If it fails, the fallback is full reproduction
of the ECDH `ExchangeKey` handshake (curve/KDF/cipher) — a much larger crypto-RE effort. Build path:
add `SerializeNativeOpenConnectionVersion8(connectionType)` to `HistorianOpen2Protocol`, wire the gRPC
event handshake to use it (events only; reads stay on v6), live-test (non-destructive). Full hex in
the gitignored capture.
### Path A built + live-tested 2026-06-23 — DISPROVEN (v8 is coupled to ExchangeKey)
Built `HistorianOpen2Protocol.SerializeNativeOpenConnectionVersion8` (golden-tested,
`Version8EventSerializerReproducesCapturedNativeStructure` — reproduces the captured 302-byte
structure exactly) + `HistorianNativeHandshake.BuildEventOpenConnectionVersion8Request` (zeroed
credential token) + an `eventConnection` switch on `HistorianGrpcHandshake.OpenSession`, and live-ran
the event read against the server. Result: the v8 `OpenConnection` was **parsed by the server** (got
past the byte format) but **rejected at the auth check** with native error
```
type=132 code=34 "aahHcapLib::HistoryService::EstablishConnection — Failed to get client key"
```
i.e. `EstablishConnection` could not find a server-side **client key** for our session. In the v6
path that key is established by `StorageService.ValidateClientCredential` (which is why v6 reads
work); the v8 path looks it up in the registry that **`HistoryService.ExchangeKey` (ECDH)** populates,
and there is **no `ValidateClientCredential` on `HistoryService`** in the gRPC contract. So the server
branches on the OpenConnection version: v6 accepts the Negotiate-established key, **v8 requires the
ExchangeKey-established key**. The zeroed-token hypothesis is therefore disproven — not because of the
token bytes, but because the whole v8 path is gated on `ExchangeKey` having run first.
**Status:** the v8 serializer/builder are correct and retained (golden-tested), plus the
`OpenConnection` failure now decodes the native error (type/code/ASCII). The event orchestrator is
reverted to the v6 session (gated test still pins the no-row throw). The remaining route is **Path B:
implement `HistoryService.ExchangeKey`** — `"ECK1"` + a 64-byte EC public-key point (P-256 X‖Y, by the
size) — using .NET `ECDiffieHellman`, establish the client key, then reissue the v8 `OpenConnection`.
Open question for Path B: whether merely *completing* the ECDH key agreement registers the client key
(so the zeroed openParameters token still rides through), or whether the token must also be derived
from the shared secret (full KDF/cipher RE).
### Path B started 2026-06-23 — ExchangeKey ECDH works; cleared 2 of 3 layers
Implemented `HistoryService.ExchangeKey` as a **pure-managed P-256 ECDH** key exchange
(`HistorianNativeHandshake.BuildExchangeKeyClientHello` / `DeriveExchangeKeySecret`, .NET
`ECDiffieHellman` over `nistP256`; wire format `"ECK1" + u32(32) + X(32) + Y(32)`) and wired it into
`HistorianGrpcHandshake.OpenSession(eventConnection: true)` ahead of the v8 `OpenConnection`,
on the same context-key handle. Live result against the server: the **`ExchangeKey` RPC succeeds**
(the server accepted our public key), and the v8 `OpenConnection` error **moved one layer deeper**:
```
Path A (no ExchangeKey): 132/34 "Failed to get client key"
Path B (ExchangeKey ECDH): 132/171 AuthenticationFailed "EstablishConnection — Authentication failed"
```
So the ECDH cleared the client-key check; the remaining blocker is **authentication**: the 26-byte
v8 credential token must be a *valid* value derived from the ECDH shared secret (not zeros).
### Token crypto traced 2026-06-23 (Frida → Windows CNG) — KDF found, token construction still open
Hooked Windows CNG (`bcrypt.dll`/`ncrypt.dll`) while the native harness ran a real ExchangeKey
(`scripts/frida/aahclientmanaged-cng-exchangekey.js` + `artifacts/.../cng-trace.py`). Findings:
- **The ECDH + KDF are standard CNG, driven by managed `System.Security.Cryptography.ECDiffieHellmanCng`**
(backtrace top frame = `System.Core.ni.dll`; the caller is aahClientManaged's C++/CLI `<Module>`):
`NCryptSecretAgreement` (P-256) → `NCryptDeriveKey(KDF=HASH, HASH_ALGORITHM=SHA256, 32 bytes)`. So the
derived key = **SHA256(ECDH shared secret)** — exactly `ECDiffieHellmanCng{ KeyDerivationFunction=Hash,
HashAlgorithm=SHA256 }.DeriveKeyMaterial(...)`. Our managed `DeriveExchangeKeySecret` should switch to
this (SHA256 of the raw agreement) to match.
- **`"ECK1"` is NOT AVEVA-custom** — it is the standard Windows CNG `BCRYPT_ECCPUBLIC_BLOB` magic for
P-256 (`NCryptExportKey`/`ImportKey` emit exactly `ECK1 + len(32) + X(32) + Y(32)`), confirming our
`BuildExchangeKeyClientHello` wire format is correct.
- **The 26-byte token is a custom construction that is not yet reproduced.** Correlated one run's
derived key (`SHA256(secret)`) with that run's token (from the IL openParameters capture): a
528-candidate offline cracker (HMAC/SHA/AES-GCM/CBC/CTR over the derived key × request slices ×
creds) found **no match**, and the token matches **none** of the traced hash digests. The token
starts with a constant `0x8e` marker in both captured runs (so it is structured, not raw cipher
output). It is built in managed code between the `DeriveKeyMaterial` call and the openParameters
assembly.
**dnlib IL extraction 2026-06-23 — the token scheme is fully reverse-engineered.** ILSpy can't
decompile the mixed-mode assembly (crashes), but loading `dnlib` in PowerShell and scanning the IL
recovered the whole construction:
- **`<Module>::CHistoryConnectionGrpc.GetClientKey`** is the ECDH driver: `new ECDiffieHellmanCng()`
`KeyDerivationFunction = Hash`, `HashAlgorithm = SHA256`, `KeySize = 256`
`GrpcHistoryClient.ExchangeKey(strHandle, ourPubKey.ToByteArray(), out serverPub, out err)`
`CngKey.Import(serverPub, CngKeyBlobFormat.EccPublicBlob)`**`DeriveKeyMaterial`** = the 32-byte
client key = **`SHA256(ECDH shared secret)`**. (So our managed side should derive the key the same
way — `ECDiffieHellman` raw agreement then SHA256, or equivalently `DeriveKeyFromHash(..., SHA256)`.)
- **The 26-byte token is built by `aahClientCommon.CClientBase.ConfigureOpenConnection`** (the lone
caller of `GetClientKey`) using the **`HistorianCrypto.NRC4_V2.aahCryptV2`** scheme — a custom
**MD5-keyed RC4 stream cipher with a version prefix**:
- `aahCryptV2.body`/`HashData` = **MD5** (verified: the IL loads MD5 round constants `0xd76aa478`
and rotates 7/12/17/22).
- `aahCryptV2.prepare_key` = standard **RC4 KSA** seeding the 256-byte S-box from a **16-byte (MD5)**
key (`std.array<unsigned char,16>`).
- `aahCryptV2.enc_buffer` = `MD5(...)` → key, then **`rc4encrypt`** the body; `enc` prepends a
scheme **prefix** (`NRC4_V2.PrefixV2` / `InnerPrefixV2`) — the constant `0x8e` token marker.
- `from_GUID` keys the cipher from a GUID string.
So the token = `prefix + RC4(plaintext, key = MD5(keyMaterial))`, where the key material ties back to
the `SHA256(ECDH secret)` client key. **This is 100% reproducible in pure managed code** (RC4 + MD5
are ~40 lines; nothing AVEVA ships).
**Remaining to finish (next cycle):** read `ConfigureOpenConnection`'s exact wiring (which value is
MD5'd for the RC4 key, what plaintext is encrypted, the exact prefix bytes — a little more dnlib IL),
implement `aahCryptV2` (RC4+MD5+prefix) managed-side, set the v8 token = that, and live-test
(non-destructive). The offline correlation data (one run's derived key + token + openParameters) is
captured under `artifacts/.../` to validate the managed reproduction before going live.
### Token implemented + auth WORKS live (2026-06-23); row retrieval still 0 — proven NOT a payload issue
`token = RC4(password-UTF16LE, key = MD5(SHA256(ECDH secret)))` was implemented in pure managed C#
(`HistorianNativeHandshake.BuildExchangeKeyCredentialToken` + `Rc4`; client key via
`DeriveKeyFromHash(SHA256)`), golden-tested (RC4 standard vector + token construction), and
**live-verified**: the v8 `OpenConnection` now **authenticates** against the 2023 R2 server (past the
`132/171 AuthenticationFailed` wall). Auth is solved.
The event **query** still returns `version-11 rowCount-0` while the native returns 50 for an
**identical** request. Exhaustively ruled out as the cause (all confirmed live, opt-in
`EventReadDiagnostic` test + the IL rewrite extended to log string/uint handle fields):
- `StartEventQuery` request: **byte-identical** to the native (v6 layout)
- v8 `OpenConnection` `openParameters`: **byte-identical** to the native (302 bytes) once ClientNodeName
is matched — every control byte, ConnectionType, token framing, ShardId, etc.
- Handle usage: identical — `ExchangeKey`→contextKey, registration→storage-session GUID (`strHandle`),
query→client uint (`uiHandle`); our parsed handles are valid (registration `RTag/EnsT=True`, valid
`queryHandle`)
- `queryRequestType = 3`, registration sequence/order, gzip metadata header — all match
- window (events exist; native returns 50 *now*), eventCount — not it
So **every observable client-side byte matches the native**, yet the server scopes 0 events to our
connection. The event RPCs succeed over our transport and return a valid *empty* result (not a
transport error), so it is **not a payload or transport-incompatibility issue** — it is a
connection/server-level difference (e.g. session affinity tied to the native `Grpc.Core` HTTP/2
connection or a connection-identity the server uses to scope events) that is **invisible to, and
unfixable by, client payload matching.** Closing it needs server-side insight or a different angle
(e.g. compare the full HTTP/2 connection setup / TLS identity), not more wire-payload RE.
**Shipped this effort:** the complete ExchangeKey crypto (ECDH + SHA256 + MD5-keyed RC4 token) — the
hard wall — pure managed, golden-tested, auth live-verified. Orchestrator stays on the no-row throw;
gated test unchanged.
### NEXT SESSION — the server-side / connection angle (row retrieval pickup)
Client payloads are exhausted (byte-identical to the native, proven above). The next investigation is
**connection-level**, not wire-payload. Pursue in roughly this order; each is concrete and testable.
**Already proven — do NOT redo:** auth works (ExchangeKey ECDH + RC4 token, live-verified); v8
`openParameters`, all handles (str/uint), `StartEventQuery` request, registration (`RTag/EnsT=True` +
order), `queryRequestType=3`, gzip header — all byte-match the native. Events exist (native returns 50
*now*). The event RPCs succeed over our transport and return a valid version-11 **rowCount-0** (not a
transport error). So the server scopes 0 events to *our* connection specifically.
**Tooling already in place:** opt-in diagnostic test `EventReadDiagnostic_OverGrpc_PrintsJourney`
(env `HISTORIAN_GRPC_EVENT_DIAG=1`, prints registration outcomes, handles, result hex, v8 buffer);
the `capture-event` harness scenario (native, returns rows); `instrument-grpc-nonstream` now logs
string/uint handle fields too; the CNG Frida hook. Live recipe: set `HISTORIAN_GRPC_HOST`/`_PORT
32565`/`_TLS true`/`_DNSID` to the 2023 R2 server + domain creds (strip quotes); reach the box per the
live-server access reference.
1. ~~**Transport: native `Grpc.Core` HTTP/2 vs our `Grpc.Net.Client` + `GrpcWebHandler` (gRPC-Web).**~~
**DISPROVEN 2026-06-23.** Built `HistorianGrpcChannelFactory.CreateHttp2` (plain HTTP/2 over a
`SocketsHttpHandler`, no `GrpcWebHandler` wrap, ALPN `h2` to the TLS server) and wired it into the
event orchestrator behind `HISTORIAN_GRPC_EVENT_HTTP2=1` (event path only; reads stay gRPC-Web). Live
side-by-side against the event-bearing server, **everything else held constant**:
| channel | auth | registration | queryHandle | result buffer |
|---------|------|--------------|-------------|---------------|
| `http2` (native HTTP/2) | ✓ | `RTag=True EnsT=True` | 1057 | `0B00000000001E000000` |
| `grpc-web` (default) | ✓ | `RTag=True EnsT=True` | 1058 | `0B00000000001E000000` |
The complete v8 chain — ExchangeKey ECDH auth, CM_EVENT `RegisterTags`/`EnsureTags`, `StartEventQuery`
(valid handle) — runs end-to-end over **plain native HTTP/2**, and the server returns the
**byte-identical** version-11 (`0x0B`) rowCount-0 terminal on both transports. So gRPC-Web vs native
HTTP/2 is **not** the discriminator — the zero-row scoping is identical regardless of transport. The
`CreateHttp2` factory + the `HISTORIAN_GRPC_EVENT_HTTP2` switch + the `EventChannelMode` diagnostic are
retained for future connection-level probing. This eliminates the leading hypothesis and tightens the
conclusion: the server scopes 0 events to our connection at a layer **above** the gRPC transport.
2. ~~**TLS client identity / certificate.**~~ **DISPROVEN 2026-06-23 (decompile + capture).** The stock
client's `GrpcClientBase.InitializeBase` creates a bare `HttpClientHandler` and sets only
`ServerCertificateCustomValidationCallback` — it **never adds a client certificate**. The TLS-tee
capture (below) confirms `clientCert=none` on every native connection. So the native presents no client
cert; this is not the gate.
3. ~~**HTTP/2-level / connection-frame capture.**~~ **DONE 2026-06-23 — topology difference found, tested,
NULL.** Built a TLS-terminating tee proxy (`artifacts/.../httpcap/`, gitignored: self-signed server
cert, forwards through the loopback tunnel, logs decrypted HTTP/1.1 + gRPC-Web both ways) and ran a
**native `capture-event` (returns 50 rows) and our SDK diagnostic (0 rows) through the same
proxy/upstream**. Note: the stock client is gRPC-Web/HTTP-1.1 (not HTTP/2 — `alpn` empty), so the
capture is HTTP/1.1 framing. Findings:
- **Connection topology differs.** The native opens **5 TLS connections, one per service**
`HistoryService` (ExchangeKey/OpenConnection/Register/EnsureTags), `StatusService` (×2), and
**`RetrievalService` (the event query: GetRetrievalInterfaceVersion → StartEventQuery → GetNext →
EndEventQuery) on its own dedicated connection**. Our SDK collapses **every service onto one
connection**. (Matches the decompile: stock has a separate `GrpcClientBase` per service.)
- **Framing differs** (benign): native uses `content-length` + `Expect: 100-continue`; SDK uses
`transfer-encoding: chunked`. The server accepts both (our `StartEventQuery` returns a valid handle),
so framing is not the gate. No extra/hidden header on either side; `clientCert=none` throughout.
- **TESTED the topology hypothesis (`HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL=1`):** ran
`StartEventQuery`/`GetNext`/`EndEventQuery` on a **dedicated RetrievalService connection** (no
re-handshake, reusing the session handle — exactly mirroring native conn4), registration staying on
the main connection. **Result: still `0B00000000001E000000` (0 rows), `QH=1063`.** Splitting the
event query onto its own connection — the one concrete structural difference the capture revealed —
**does not make rows flow.** So the server correlates by session handle, not by connection, and the
topology is **not** the row-scoping gate. The `CreateHttp2`/`SPLIT_CHANNEL` switches + the
`httpcap` proxy are retained as diagnostics.
4. ~~**Server-side ground truth.**~~ **ANSWERED 2026-06-23 (DISPROVES the data-scoping premise).** Via
the SOCKS→SQL relay (read-only; `artifacts/.../sqlschema/`, gitignored), dumped the full event schema
on the live `Runtime` DB. Findings:
- **No per-connection / per-client / per-session column exists anywhere in the event store.** The only
"scoping-like" columns on `Events`/`EventHistory`/snapshots are event *content*`Source_*` (event
origin area/object/PV), `User_*` (who acknowledged), `Provider_NodeName` (alarm provider node),
`SourceServer`/`SourceTag` (cross-server replication). None is "which client connection requested
this."
- **The rich `Events` view is not a relational table — it is served live by the Historian engine via
the `INSQL` OLE DB provider** (`sys.servers` shows linked servers `INSQL` + `INSQLD`;
`OBJECT_DEFINITION('dbo.Events')` is `NULL` = encrypted remote view). The Historian's own
`EventHistory` base table holds just 168 rows / 1 tag (the internal event-tag detector log); the
alarm/event journal the gRPC query reads lives in the engine, surfaced through INSQL.
- **Decisive: same engine, same `-90d..now` window, two paths diverge.** The `Events` view (via INSQL)
returns **71,332 events** for that window — most recent `Alarm.Set` firing seconds before the probe
(live, every few seconds) — while gRPC `StartEventQuery` for **our** connection returns **0**. The
data is global, abundant, recent, and identical-window-addressable; the engine simply does not hand
it to our gRPC connection.
→ There is **nothing in the data to scope by**, so the zero-row gate is **not** data scoping. It is the
gRPC RetrievalService's **per-connection in-process execution state** — the same class of wall as
`DeleteTagExtendedProperties` (server-side native in-process working-set, not reconstructable from
byte-identical wire requests). Reproduce: `artifacts/.../sqlschema/` (Program.cs = SOCKS5 relay +
`Microsoft.Data.SqlClient`; authenticate with the server's SQL login, not the domain Historian acct —
creds in the gitignored creds file).
### Stock managed client decompiled (2026-06-23) — confirms no hidden client-side difference
Closing the gap that prior cycles left: the zero-rows conclusion had leaned on **wire capture**
(`instrument-grpc-nonstream`, which only hooks `byte[]` params on `Grpc*Client` methods) — blind to gRPC
metadata/headers, interceptors, channel options, and any non-`byte[]` call. Read the **stock managed
client source directly** (`histsdk-2023r2-analysis/decompiled/Archestra.Historian.GrpcClient` +
`HistorianAccess`; the pure-managed assemblies decompile cleanly even though the mixed-mode
`aahClientManaged.dll` crashes ILSpy). Findings:
- **`GrpcClientBase.InitializeBase` builds the same channel we do.** `GrpcWebHandler((GrpcWebMode)0,
HttpClientHandler)` with `HttpVersion = 1.1` — i.e. **the stock client speaks gRPC-Web over HTTP/1.1,
the same transport as our SDK.** This *corrects the premise of hypothesis #1*: there was never a native
`Grpc.Core` HTTP/2 path to differ from — the stock client that returns 50 rows is itself gRPC-Web. The
HTTP/2 disproof's *conclusion* stands (and is reinforced: identical transport on both sides).
- **`m_metadata` passed to every RPC (incl. `StartEventQuery`/`GetNextEventQueryResultBuffer`) is only
`grpc-internal-encoding-request: gzip`** — exactly our header set. No connection-id, session token, or
auth header rides in gRPC metadata. The **`ClientInterceptor` is a no-op** (`LogCall` is empty; both
unary overloads just invoke the continuation). So the "invisible per-connection metadata/header" blind
spot is **confirmed empty** — there is no hidden client-side identity the `byte[]` capture missed.
- **The event-read query orchestration is genuinely not in managed code.** `CreateEventQuery` /
`EventQuery.StartQuery` / `MoveNext` are not in the managed `HistorianAccess`; the managed
`GrpcRetrievalClient.StartEventQuery` is a thin one-RPC stub. The query logic lives in the native
C++/CLI `HistorianClient` core (the mixed-mode part ILSpy can't decompile) — consistent with the
working-set being native/server-side, not a managed step we could read and replicate.
So **every client-controllable layer is now confirmed identical by reading the stock source**, not just
by wire match: request bytes, transport, channel options, gRPC metadata, interceptor. The remaining
difference is below the managed surface (native core) / server-side.
**Conclusion (after #1#4 + stock client decompiled + TLS-tee capture).** Every angle is now exhausted:
- **client payload** — byte-identical (IL capture + decompile);
- **transport** — stock client is *also* gRPC-Web/HTTP-1.1; native HTTP/2 makes no difference, both 0 rows;
- **client metadata/interceptor/channel** — decompiled: identical gzip-only header, no-op interceptor, no
client cert; the TLS-tee capture confirms no hidden header and `clientCert=none`;
- **connection topology** — the native splits services across 5 connections and queries on a dedicated
RetrievalService connection; replicating that (`SPLIT_CHANNEL`) still returns 0 rows → the server
correlates by session handle, not connection;
- **data store** — global, unscoped; 71,332 events the engine serves via INSQL but withholds from our
gRPC connection.
The gate is a **server-internal per-connection retrieval working-set** that a pure-managed client cannot
reconstruct by matching wire bytes, transport, metadata, topology, or data — and the establishing logic is
in the native `HistorianClient` C++ core, not in any decompilable managed step or observable on the wire.
**gRPC event-row retrieval stands documented as auth-solved / retrieval-server-gated**; `ReadEventsAsync`
over gRPC keeps the honest no-row throw, and event reads use the WCF transport. Diagnostics retained for
any future server-side investigation: the `httpcap` TLS-tee proxy, the `CreateHttp2` / `SPLIT_CHANNEL`
switches, the `EventReadDiagnostic` test, and the `capture-event` harness (native, returns rows).
### Verify the parse path against the provided client's real data (2026-06-23) — found + fixed a latent bug
Used the provided 2023 R2 client as an **oracle**: the `capture-event` harness returns 50 real events
(verified live + through the `httpcap` proxy), and the `instrument-grpc-nonstream` rewrite captured the
exact `GetNextEventQueryResultBuffer.result` buffer the stock client received — **63,192 bytes, version
`0x0B` (11), rowCount 50** (25 `Alarm.Set` + 25 `Alarm.Clear`). Fed that real buffer through our
`HistorianEventRowProtocol.Parse` to verify the read path decodes genuine gRPC event data, and it
**exposed a latent parser bug**:
- The real row buffer is `version(2) + rowCount(4) + headerField(4, =0x1E)` then **markerless rows**
(`rowFormat(2)=7 + filetime(8) + 8×u16 slots + compact-ascii type + propCount + props`). Our parser
wrongly treated the one-time `0x1E` field as a **per-row marker** and re-consumed `[marker+format]`
every row — so it parsed only the **first** row of any multi-row buffer and stopped. This is **not
gRPC-specific**: the captured **WCF v9** buffer has the identical `0900 <rowCount> 1E000000 0700 …`
header, so the shipped WCF event read had the same latent multi-row truncation.
- **Fix:** read a 10-byte buffer header (skip the `0x1E` field once) and parse markerless rows; accept
container version **9 (WCF) and 11 (gRPC)**. Verified: the real 50-row buffer now decodes to exactly 50
events, ending cleanly at end-of-buffer (`Parse_RealStockClientCapture_DecodesAllEvents`, gated on
`HISTORIAN_EVENT_CAPTURE_NDJSON`); plus a synthetic v11 golden test. 328 offline tests pass.
So the **parse path is now verified against the provided client's real event data** — the one remaining
gap is strictly the server delivering rows to our gRPC connection (the working-set gate above). If that
were ever opened, the decoded events would now flow through correctly on both transports.
**2 of 3 layers cleared** (key exchange + client key); the 3rd (token construction) is localized to a
specific managed method, pending dnlib extraction. ExchangeKey + the v8 serializer are committed; the
orchestrator stays on v6 (set `eventConnection: true` to re-arm once the token construction lands). The
token-loop routing guardrail (`HistorianGrpcHandshakeRoutingTests`) was scoped to the closure so the
legitimate ExchangeKey call is allowed while still pinning that the Negotiate token loop never routes
there.
@@ -0,0 +1,45 @@
# 2023 R2 gRPC Interface-Version Integers (C3a)
**Captured:** 2026-06-25
**Transport:** 2023 R2 gRPC (h2c, unauthenticated `GetInterfaceVersion` RPCs — no credentials required)
**Status:** LIVE — integers captured from a real AVEVA Historian 2023 R2 server.
## Captured Values
| Service | UiVersion / Version | UiError / Error | Notes |
|---------------|---------------------|-----------------|----------------------------------------------------|
| History | 12 | 0 | Matches `HistoryInterfaceVersionGrpc2023R2 = 12` |
| Retrieval | 4 | 0 | Matches `RetrievalInterfaceVersion = 4` (unchanged from 2020) |
| Transaction | 2 | 0 | Matches `TransactionInterfaceVersion = 2` (unchanged from 2020) |
| Status | 4 | 0 | Reachability-only; version integer is not version-gated (see note) |
> **Field-name note:** The History, Retrieval, and Status proto responses use `UiError`/`UiVersion` fields.
> The Transaction response uses `Error`/`Version` (different naming convention in the proto). Both are
> captured correctly; the table uses a unified column header for readability.
> **Status note:** `StatusService.GetStatusInterfaceVersion` returned UiVersion=4, UiError=0 on the live
> 2023 R2 server. This differs from the historical 0 observed on 2020 WCF — both are reachability-only.
> Status is classified as reachability-only: its version integer carries no semantic meaning for the
> SDK's byte serializers, so its UiVersion is not gated and not asserted in tests.
## Evidence Test
`tests/AVEVA.Historian.Client.Tests/GrpcInterfaceVersionEvidenceTests.cs`
`GrpcInterfaceVersions_LiveServer_MatchAcceptedSet` reads these four RPCs live and asserts:
- `history.UiError == 0` and `history.UiVersion ∈ {11, 12}`
- `retrieval.UiError == 0` and `retrieval.UiVersion == 4`
- `transaction.Error == 0` and `transaction.Version == 2`
- `status.UiError == 0` (version not asserted)
The test skips silently when `HISTORIAN_GRPC_HOST` is absent.
## Gap Closed
This document closes the **C3a** gap: "2023 R2 gRPC server-version integers not yet captured."
Prior to this capture, the `HistorianServerVersionGate` accepted History=12, Retrieval=4, and
Transaction=2 on the basis that they were inferred/expected-to-be-unchanged. All four integers are
now confirmed from a live 2023 R2 server over the gRPC transport; no widening of `AcceptedVersions`
is required (all captured values were already accepted).
The 2020 WCF baseline (History=11, Retrieval=4, Transaction=2) was captured earlier via the
`wcf-probe` command and is documented in `wcf-probe-remote-latest.json` and `wcf-contract-evidence.md`.
@@ -0,0 +1,57 @@
# R0.1 browse over gRPC — StartTagQuery takes an OData filter (2026-06-21)
Live-probed `RetrievalService.StartTagQuery` / `QueryTag` against a real **2023 R2** server over the
gRPC front door (string-handle = uppercase Open2 storage GUID). Key result: **browse is feasible on
2023 R2** — the 2020 WCF "metadata-server pipe" wall does **not** block here.
## StartTagQuery — CRACKED
`StartTagQuery(strHandle, btRequest)` where `btRequest` = the native
`marker(26449) + version(1) + WriteHistorianString(filter)` buffer
(`HistorianTagQueryProtocol.CreateStartTagQueryAttempt`). The server runs
`CMdServer::StartTagQuery::StartActiveTagnamesQuery` over `\\.\pipe\aahMetadataServer\console` and
**parses the filter string as OData** (not SQL-LIKE). Swept filters:
| filter | result |
|---|---|
| `startswith(TagName,'Sys')` | ✅ success, 8-byte response |
| `contains(TagName,'Sys')` | ✅ success |
| `TagName eq 'SysTimeSec'` | ✅ success |
| `` (empty) | ✅ success (all tags) |
| `Sys*` / `*` | ❌ `ODataFilter ... bad token` |
| `TagName like 'Sys%'` / `Name like 'Sys%'` | ❌ rejected |
Success response `btResponse` is the 8-byte `(queryHandle:uint, tagCount:uint)` pair
(`ParseStartTagQueryResponse`). Live: `startswith(TagName,'Sys')` → tagCount = 220.
**Implication for the public API:** browse must translate the SDK's glob filter to OData —
`*` → empty, `Pre*` → `startswith(TagName,'Pre')`, `*sub*` → `contains(TagName,'sub')`,
exact → `TagName eq '...'`. (Escaping single-quotes in names still TBD.)
## QueryTag — CRACKED (2026-06-21), browse SHIPPED
`QueryTag(strHandle, uiQueryHandle, btRequest)` is the paging call that returns the tag-name rows.
The blocker was the packet id: every guessed `btRequest` returned native error **type 4 / code 72 =
`InvalidPacketId`** (`ArchestrA.CloudHistorian.Contract.ErrorCode.InvalidPacketId`). The generic
`0x6751` header that StartTagQuery accepts is the **wrong** id for QueryTag.
**How it was found (no Ghidra needed):** a `.rdata` **packet-descriptor table** in
`aahClientManaged.dll` lists consecutive `{uint marker, uint version}` entries —
`{0x6751, 1}` (StartTagQuery) immediately followed by **`{0x6752, 1}`** (the paired op). Found by
`pefile` byte-scan of `.rdata` for `51 67 00 00` and dumping the surrounding dwords. Testing `0x6752`
live confirmed it.
**QueryTag wire format (live-verified):**
- request `btRequest` = `u16 marker(0x6752) + u16 version(1) + u16 queryType + u32 startIndex + u32 count`
— `queryType = 1` returns tag-name rows (`queryType = 0` returns an empty/count-only page).
- response `btResonse` = `u32 count + per-name(u32 charCount + UTF-16LE) + trailer`
(the trailer is the CloudHistorian `NextIndex`/`TagMetadataBuffer` region — ignored by
`HistorianTagQueryProtocol.ParseTagNameQueryPage`).
- Semantic fields match `ArchestrA.CloudHistorian.Contract.QueryTagRequest`
(`QueryType/StartIndex/TagCount`; the QueryHandle travels in the protobuf `uiQueryHandle`).
**Browse is shipped:** `HistorianClient.BrowseTagNamesAsync` routes over gRPC when
`Transport==RemoteGrpc` via `Grpc/HistorianGrpcTagClient.BrowseTagNamesAsync`
(StartTagQuery(OData) → paged QueryTag(0x6752) → EndTagQuery), with the SDK glob filter translated by
`GlobToODataFilter`. Golden-byte + glob unit tests and a gated live test
(`BrowseTagNamesAsync_OverGrpc_ReturnsSystemTags`) cover it. **M0 gRPC parity is complete.**
+244 -24
View File
@@ -1,6 +1,193 @@
# AVEVA Historian Managed Driver Handoff # AVEVA Historian Managed Driver Handoff
Last updated: 2026-05-04 (event-flow prereqs) Last updated: 2026-06-23 (event-row parser fix merged; roadmap still exhausted — no actionable pure-code tasks remain)
> **Current status supersedes the historical blocker narrative below.** The
> sections from "Active Blocker" onward are a preserved reverse-engineering
> record of how the 2020 WCF read/write/event paths were cracked (2026-05-04).
> They are kept for provenance; they are **not** the live state. Start with
> "Current Status" immediately below.
## Current Status (2026-06-22) — roadmap exhausted
`docs/plans/hcal-roadmap.md` reachable surface is **complete**, and every plan
under `docs/plans/` is either DONE or has only gated items left. There are
**zero actionable pure-code tasks remaining**. Memory anchor:
`project_roadmap_exhausted_2020wcf`.
**Shipped + live-verified across both transports:**
- **Reads** (WCF + gRPC): `ProbeAsync`, `ReadRawAsync`, `ReadAggregateAsync`,
`ReadAtTimeAsync`, `ReadEventsAsync`, `BrowseTagNamesAsync`,
`GetTagMetadataAsync`, status helpers (`GetConnectionStatusAsync`,
`GetStoreForwardStatusAsync`, `GetSystemParameterAsync`).
- **Writes**: `EnsureTagAsync` (analog Float/Double/Int2/Int4/UInt4, `ApplyScaling`),
`DeleteTagAsync`, `RenameTagsAsync`, `AddTagExtendedPropertiesAsync`, and the
M3 historical/backfill `AddHistoricalValuesAsync` (**gRPC-only**, all five
analog types golden-tested + live write/read-back).
- **Config reads** (mostly gRPC): `GetRuntimeParameterAsync`,
`GetTagExtendedPropertiesAsync`, `ExecuteSqlCommandAsync` (WCF; gRPC
server-walled), `GetServerTimeZoneAsync` (gRPC-only).
- **Client-side**: M4 R4.1 store-and-forward outbox, R4.4 redundancy,
R4.3 measured-idle SF status.
The 2023 R2 **gRPC transport** (`HistorianTransport.RemoteGrpc`, port 32565)
reuses the proven 2020 WCF byte serializers/parsers unchanged inside protobuf
`bytes` fields, keyed by the Open2 session handle. Live-verified against a real
2023 R2 server (History interface v12) — see `reference_2023r2_live_server_access`.
**Everything still open is gated — none is a pure-code task:**
1. **gRPC event ROW retrieval** (`ReadEventsAsync` #2) — **AUTH-SOLVED / PARSE-VERIFIED /
RETRIEVAL-SERVER-GATED (every client-side angle exhausted 2026-06-23, merged `6faf8a5`).**
The v8 OpenConnection crypto wall is fully cracked + live-verified: the event connection
authenticates via **`HistoryService.ExchangeKey` (P-256 ECDH) → client key =
`SHA256(shared secret)` → credential token = `RC4(password-UTF16LE, key=MD5(client key))`**
(the native `HistorianCrypto.NRC4_V2.aahCryptV2` MD5-keyed RC4 scheme). RE'd via Frida CNG
hooks + dnlib IL extraction + an offline cracker; implemented pure-managed, golden-tested,
auth live-PASSES. The `StartEventQuery` v6 request and the Event-type v8 `OpenConnection`
(`ConnectionType=Event`) are shipped. **BUT** the query still returns **rowCount-0** while
the native returns 50 for a byte-identical request — and **all four next-session angles are
now tested and ruled out** (`grpc-event-query-capture.md`):
- **transport** — the stock client is *also* gRPC-Web/HTTP-1.1 (decompiled); plain native
HTTP/2 (`CreateHttp2`) returns the same 0 rows;
- **client metadata/cert** — decompiled + TLS-tee captured: gzip-only metadata, no-op
interceptor, **no TLS client cert** on either side;
- **connection topology** — the native splits services across 5 connections and queries on
a dedicated RetrievalService connection; replicating that (`HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL=1`)
still returns 0 rows → the server correlates by **session handle, not connection**;
- **data store** — via SOCKS→SQL: the event store is global/unscoped (no per-connection
column); the `Events` view (served by the engine via the INSQL provider) returns 71,332
events for the same window the gRPC query gets 0.
So the gate is a **server-internal per-connection retrieval working-set** in the native
`HistorianClient` C++ core — not reconstructable from a pure-managed client. **PARSE PATH NOW
VERIFIED + a latent bug FIXED:** fed the provided stock client's real captured result buffer
(63,192 B, 50 events) through `HistorianEventRowProtocol.Parse` — it exposed that the parser
treated the one-time `0x1E` buffer header field as a per-row marker, decoding only the **first
row of any multi-row buffer**. This also hit the **shipped WCF event read** (identical
`0900 <rowCount> 1E000000 0700` header). Fixed to a 10-byte buffer header + **markerless rows**,
accepting container version 9 (WCF) and 11 (gRPC); the real 50-row buffer now decodes to exactly
50 events (`Parse_RealStockClientCapture_DecodesAllEvents`, gated on `HISTORIAN_EVENT_CAPTURE_NDJSON`).
So if the server gate ever opens, decoded events flow through correctly on both transports; until
then the orchestrator stays on the no-row throw (`eventConnection: true` wired; opt-in
`EventReadDiagnostic` test, `HISTORIAN_GRPC_EVENT_DIAG=1`). Diagnostics retained: the `httpcap`
TLS-tee proxy, `CreateHttp2`/`SPLIT_CHANNEL` switches, the `sqlschema` SOCKS→SQL probe, the
`capture-event` harness (native, returns rows).
2. **R4.3 active-SF magnitude** — needs an **SF-active server** (D2 storage-engine
console handle).
3. **SendEvent over gRPC** — ✅ **SHIPPED + LIVE-VALIDATED 2026-06-23.** `SendEventAsync`
now routes over `RemoteGrpc` (`HistorianGrpcEventWriteOrchestrator`). Captured the native
client live (`capture-send-event` harness scenario): the send rides
`HistoryService.AddStreamValues` with the **same "OS" (0x534F) buffer the WCF path uses**
(`HistorianEventWriteProtocol` — "no distinct RPC" confirmed true), on a v8 Event session +
CM_EVENT registration. The write-enabled Event open is **byte-identical** to the read-only one
(diffed live — only per-session crypto differs), so the existing event-open path is reused
unchanged. End-to-end: pure-managed SDK send → `BSuccess=true` → event read back from the live
server (markers `SdkSendProbe`/`SdkCaptureProbe` confirmed in returned rows). Golden-tested
(`GrpcEventSendProtocolTests`) + gated live test (`SendEventAsync_OverGrpc_AcceptsEvent`,
opt-in `HISTORIAN_GRPC_EVENT_SEND=1`).
**PERSISTENCE RESOLVED:** the earlier WCF/M2 caveat was "server accepts AddS2 but the local
dev box does NOT persist the event to `v_AlarmEventHistory2`" — we couldn't tell if that was an
SDK gap or the environment. The gRPC read-back against the live 2023 R2 server **proves the event
persists** (sent via the pure-managed SDK, then independently read back from the server's event
history). So non-persistence on the local box was purely the dev-box event-ingestion environment,
**not an SDK gap** — the event-send path (WCF and gRPC, same "OS" buffer) is durably correct.
4. **ExecuteSqlCommand over gRPC****server-walled** (`CSrvDbConnection`;
RegisterTags prime doesn't help). Use WCF for SQL.
5. **R4.2 revision EDITS** — storage-engine-pipe-only on BOTH transports (the D2 wall).
6. **ReadBlocks** (`StartBlockRetrievalQuery`) — never captured on either transport.
7. **DeleteTagExtendedProperties** — server-blocked; **confirmed walled by capturing the NATIVE
client 2026-06-23.** The agent's "use `deleteFromServer=true`" angle is moot: the native
`HistorianAccess.DeleteTagExtendedPropertiesByName(...,deleteFromServer:true)`, driven with the
cross-session sync trick that gets it past the client-side err-229 sync gate
(`Capture-DeleteTagExtendedProperties.ps1`), returns **`Success=true` / ErrorCode=Success** —
yet across repeated fresh sessions the property is **re-fetchable and re-deletable every time**,
i.e. it is **never durably removed**. So the native client itself only performs an optimistic
client-side cache delete; the server does not durably honor it (matches the HCAL cache-sync
model the decompile shows). Shipping a `DeleteTagExtendedPropertiesAsync` would return a
misleading success while the property persists, so it correctly stays **unshipped**. (Earlier
gRPC multiplexed-channel hypothesis also PROBED + DISPROVEN 2026-06-22, merge `c88260c`; pinned
by `DeleteTagExtendedProperties_OverGrpc_ProbeMultiplexedChannel`.)
8. **Deferred-by-design** items (`write-commands` D1D3, non-analog tag create,
etc.) — bounded out until an explicit customer/user demand signal.
To move any remaining item you need a **server-side / connection-level angle**
(item 1 — v8 event auth is solved; row retrieval is connection-gated, see the
NEXT SESSION section of `grpc-event-query-capture.md`), a **different server**
(SF-active for item 2), or a **demand signal** to unlock a deferred item.
(SendEvent over gRPC — formerly item 3 — is now SHIPPED + live-validated.)
Live-server gRPC probe recipe: set
`HISTORIAN_GRPC_HOST`/`_PORT 32565`/`_TLS true`/`_DNSID` + domain creds (strip
quotes — `reference_wonder_sql_vd03_credentials`) and run the gated
`HistorianGrpcIntegrationTests`.
### 2023 R2 stock-client binary dive (2026-06-23) — sharpened verdicts
Re-read the full decompiled stock 2023 R2 managed client
(`histsdk-2023r2-analysis/decompiled/`: `Archestra.Historian.GrpcClient`,
`ArchestrA.HistorianAccess`, `Archestra.Grpc.Contract`, `HistorianEvent`,
`HistorianAccessUtil`) as the oracle for every still-pending item. **Governing
fact:** `ArchestrA.HistorianAccess.dll` is a C++/CLI mixed assembly — every
data/config/write method is a thin shim into native `<Module>.HistorianClient.*`,
and the managed `Grpc*Client` wrappers are instantiated by **nothing** in the
decompiled set (`new Grpc*Client(` → zero call sites). So the buffer-building and
RPC-dispatch sequencing for these items lives in native C++ not present in the
binaries. That confirms the "gated" calls were not from missing managed steps —
with these refinements:
- **Item 1 (gRPC event rows)** — **confirmed native/server-side.** Stock event
call graph is provably identical to ours (transport, per-service channels,
gzip-only metadata, CM_EVENT registration, v8 ECDH Event-open, `StartEventQuery`
request bytes). `EventQuery.StartQuery`/`MoveNext` dispatch straight into native
`HistorianClient.StartEventQuery`/`GetNextRow`; the query orchestration that
would differ is native and not on the wire. One untested low-effort check
remains: byte-diff a captured **Event-connection** EnsureTags/RegisterTags
against our replay (the 83-vs-86-byte EnsT gap was never actually compared).
- **Item 3 (SendEvent over gRPC)** — ✅ **SHIPPED + LIVE-VALIDATED 2026-06-23** (was
"capturable"). RPC confirmed = `HistoryService.AddStreamValues` (the "no distinct RPC"
note is TRUE). The `btValues` VTQ buffer turned out to be already-owned: our M2
`HistorianEventWriteProtocol.SerializeAddStreamValuesBuffer` ("OS" buffer, decoded from
the WCF event-send) is the transport-independent `PackToVtq` equivalent and the gRPC send
uses it **verbatim** (live capture: sig `OS`/0x534F, CM_EVENT GUID, identical framing — NOT
the historical write's "ON" buffer). The write-enabled Event open is byte-identical to the
read-only one (live diff). So SendEvent-over-gRPC was pure assembly:
`HistorianGrpcEventWriteOrchestrator` = existing v8 Event open + existing CM_EVENT
registration + `AddStreamValues`(OS buffer). End-to-end live-validated (send → `BSuccess`
→ read back from the live server). Golden-tested + gated live test.
- **Item 4 (ExecuteSql over gRPC)** — **confirmed walled + explained.** The stock
client gates SQL **out client-side**: `HistorianAccess.ExecuteSqlCommand` returns
`OperationNotSupported` when `IsManagedHistorian(node)` or `!IsProcessConnectionRequested()`
(decompile ~:6198/:6214) and never sends the RPC. SQL-over-gRPC is unsupported by
design on a managed/gRPC historian; our `ProtocolEvidenceMissingException` is correct.
- **Item 5 (R4.2 revision edits)** — **confirmed HARD.** There is **no Revision RPC
in the gRPC contract** (zero "Revision" message types); the stock client reaches a
revision edit only via the native `HistorianClient.AddRevisionValuesBegin/AddRevisionValue/
AddRevisionValuesEnd` transaction trio over the storage-engine channel. NOTE: this is
a **distinct capability** from `AddNonStreamValues` (non-streamed original insert) —
`HistorianGrpcRevisionProbe` probes the latter; its doc comment was corrected to say so.
- **Item 6 (ReadBlocks/LoadBlocks)** — `LoadBlocks` request is a trivial
handle+sequence cursor but the `historyBlocks` response is a native blob with no
managed decoder, and it needs the D2-blocked `OpenStorageConnection` console handle.
Walled.
- **Item 7 (DeleteTagExtendedProperties)** — **capture done 2026-06-23; CONFIRMED WALLED, don't
ship.** RPC + string handle are correct; ADD/DELETE are structurally identical and neither uses
`StartJob`. The `deleteFromServer`-flag hypothesis is now tested and moot: the native
`DeleteTagExtendedPropertiesByName(...,deleteFromServer:true)`, driven past the err-229 client
sync gate with the cross-session trick (`Capture-DeleteTagExtendedProperties.ps1`), returns
`Success=true` — yet the property is **re-fetchable + re-deletable across repeated fresh sessions
(never durably removed)**. So the native client only does an optimistic client-side cache delete
the server doesn't durably honor (the HCAL cache-sync model). Shipping
`DeleteTagExtendedPropertiesAsync` would return a misleading success, so it stays unshipped.
- **SF/snapshot/shard/ForwardSnapshot ops** — only `Get/SetSFParameter` are managed-built
(typed strings); all others carry opaque native buffers and need the storage console
handle. Walled / tooling-internal.
**Net:** 3 items hard-confirmed walled with real explanations (4, 5, 6 + OpenStorageConnection),
and **2 moved to a precise, local-box-capturable target**: **SendEvent** (`PackToVtq` output)
and **DeleteTEP** (`BtInput` with `deleteFromServer=true`). Both need native instrumentation of
`aahClientManaged.dll` (Frida / IL-rewrite — repo tooling exists under
`tools/AVEVA.Historian.NativeTraceHarness` + `scripts/frida/`), not a special server.
## Project Direction ## Project Direction
@@ -12,7 +199,7 @@ Do not pivot to REST or a P/Invoke production shim unless the project
requirements change. Native and P/Invoke tools in this repo are reverse requirements change. Native and P/Invoke tools in this repo are reverse
engineering aids only. engineering aids only.
Required production surface remains narrowly scoped: Required production surface (all live-verified):
- `ProbeAsync` - `ProbeAsync`
- `ReadRawAsync` - `ReadRawAsync`
@@ -21,8 +208,23 @@ Required production surface remains narrowly scoped:
- `ReadEventsAsync` - `ReadEventsAsync`
- `BrowseTagNamesAsync` - `BrowseTagNamesAsync`
- `GetTagMetadataAsync` - `GetTagMetadataAsync`
- Status helpers: `GetConnectionStatusAsync`, `GetStoreForwardStatusAsync`, `GetSystemParameterAsync`
Writes are out of scope for the current pass. Write surface (added 2026-05-04 by explicit user request — see
`docs/plans/write-commands-reverse-engineering.md` Status section):
- `EnsureTagAsync` for analog Float / Double / Int2 / Int4 / UInt4
(with optional `ApplyScaling=true` for distinct MinRaw / MaxRaw
persistence — server sets `AnalogTag.Scaling=1` when the EnsT2
trailer's second byte is `0x01` instead of `0x00`).
- `DeleteTagAsync`.
`AddS2` (write samples) is **architecturally blocked** — server
runtime cache only ingests from configured IOServers / Application
Server pipelines. Discrete / String / Int1 / Int8 / UInt8 EnsT2 fail
at native `AddTag` and are unsupported. There is no `UpdateTags`
operation on the WCF surface; the misnomer in earlier write-up
drafts has been removed.
## Repository Map ## Repository Map
@@ -58,13 +260,18 @@ dotnet build .\Histsdk.slnx --no-restore
dotnet test .\Histsdk.slnx --no-build --logger "console;verbosity=minimal" dotnet test .\Histsdk.slnx --no-build --logger "console;verbosity=minimal"
``` ```
Current known-good result: Current known-good result (2026-06-23):
- Build succeeds. - Build succeeds (0 warnings / 0 errors).
- Unit tests pass: 55/55. - Offline tests pass: 328/328 (live gRPC/integration tests skip cleanly without
their env vars). Gated live tests add to this when `HISTORIAN_*` /
`HISTORIAN_GRPC_*` are set. The +7 over the prior 321 are the event-row parser
fix's golden + gated-capture coverage (`HistorianEventRowProtocolTests`:
markerless multi-row, the v11 gRPC header, and the 50-event stock-client capture).
The repository folder is not currently a Git working tree in this checkout, so The workspace is a Git working tree (origin: gitea.dohertylan.com). Use
use file timestamps or your own external backup if you need change tracking. normal git workflow for change tracking; the prior "no working tree, use
timestamps" note is obsolete.
## Environment Variables ## Environment Variables
@@ -211,7 +418,10 @@ Negative evidence:
- Running the same managed `ValCl` path through .NET Framework also fails, so - Running the same managed `ValCl` path through .NET Framework also fails, so
this is not just a .NET 10 WCF behavior difference. this is not just a .NET 10 WCF behavior difference.
## Active Blocker ## Historical: Read-Path Blocker (resolved 2026-05-04)
> Preserved RE record. This was the *original* active blocker; it is long
> resolved and is not the live state — see "Current Status" at the top.
**Resolved on `2026-05-04`.** The previous blocker — managed `ValCl` **Resolved on `2026-05-04`.** The previous blocker — managed `ValCl`
rejected by the server — had two causes, both now fixed: rejected by the server — had two causes, both now fixed:
@@ -935,27 +1145,35 @@ record 5 of `instrumented-wcf-readmessage/readmessage-capture-event-latest.ndjso
### Event-row parser ### Event-row parser
`Wcf/HistorianEventRowProtocol.Parse(ReadOnlySpan<byte>)` parses the > **CORRECTED 2026-06-23 (merged `6faf8a5`).** The skeleton below mis-read the `0x1E`
version-9 row buffer: > as a *per-row* marker. Verifying the parser against the provided stock client's real
> 50-event buffer proved `0x1E` is a **one-time buffer-level header field**, and the rows
> are **markerless** — so the original parser silently returned only the **first** row of
> any multi-row buffer (on WCF too). The corrected layout and behaviour are below.
`Wcf/HistorianEventRowProtocol.Parse(ReadOnlySpan<byte>)` parses the row buffer
(container version 9 for WCF, 11 for 2023 R2 gRPC — both accepted):
```text ```text
UInt16 version = 9 UInt16 version = 9 (WCF) | 11 (gRPC)
UInt32 rowCount UInt32 rowCount
N rows, each: UInt32 headerField = 0x1E // ONE buffer-level field, NOT a per-row marker
UInt32 rowMarker = 0x1E N rows, each (MARKERLESS):
UInt16 rowFormat = 7 UInt16 rowFormat = 7
Int64 filetimeUtc (event time) Int64 filetimeUtc (event time)
UInt16 × 8 fieldOffsets (opaque — purpose not fully decoded) UInt16 × 8 fieldOffsets (opaque — purpose not fully decoded)
Property bag (sequence of name=value pairs; first name is the event type) UInt16 propertyCount
Property bag (propertyCount × name=value pairs; first field is the event type)
``` ```
The parser extracts `EventTimeUtc` and `Type` (the first compact-ASCII-string The parser reads the 10-byte buffer header (skipping the `0x1E` field once), then walks
in the property bag) for each row, and seeks forward to the next row by each markerless row by length: `rowFormat(2) + filetime(8) + 8×UInt16 slots + compact-ASCII
scanning for the next `1E 00 00 00 07 00` marker. Property-bag value type + propertyCount + propertyCount × (name + value)`. Value encoding **is** implemented
encoding is partially decoded (compact ASCII `09 LEN 00 …`, UTF-16 strings (compact ASCII `09 LEN 00 …`, Boolean `0x02`, GUID `0x10`, FILETIME `0x18`, Int32 `0x31`,
`43 UInt32 LEN × UInt16`, integers with markers in the `0x880x8B` range, UTF-16 `0x43`; unknown markers preserve raw bytes). Verified against the provided client's
8-byte FILETIMEs) but **value parsing is intentionally not implemented yet** real buffer: `Parse_RealStockClientCapture_DecodesAllEvents` decodes all 50 events (25
— it requires more reverse-engineering and would need sanitized fixtures. Alarm.Set + 25 Alarm.Clear) to end-of-buffer (gated on `HISTORIAN_EVENT_CAPTURE_NDJSON`),
plus a synthetic v11 golden test.
5 unit tests in `HistorianEventRowProtocolTests.cs` cover empty buffer, 5 unit tests in `HistorianEventRowProtocolTests.cs` cover empty buffer,
zero-row, wrong-version, two-row synthetic, and missing-marker. Test count zero-row, wrong-version, two-row synthetic, and missing-marker. Test count
@@ -1053,10 +1271,12 @@ Typemarker dispatch:
Unknown markers preserve the raw `length` value bytes as a `byte[]` in Unknown markers preserve the raw `length` value bytes as a `byte[]` in
the property dictionary. the property dictionary.
Each row layout (refines the earlier skeleton): Each row layout (**corrected 2026-06-23** — see the "Event-row parser" note above; the
`0x1E` is a one-time buffer header field, NOT a per-row marker, and rows are markerless):
```text ```text
UInt32 rowMarker = 0x1E buffer header: UInt16 version (9|11) + UInt32 rowCount + UInt32 headerField (0x1E)
each row (markerless):
UInt16 rowFormat = 7 UInt16 rowFormat = 7
Int64 eventTimeUtcFiletime Int64 eventTimeUtcFiletime
UInt16 × 8 // purpose unclear UInt16 × 8 // purpose unclear
@@ -0,0 +1,126 @@
# Handshake / session-reuse spike — live results
> **Question:** does the 2023 R2 historian honor REUSING one authenticated session
> (channel + `OpenConnection` client handle) across multiple operations, instead of
> the per-operation Create+handshake the SDK does today? This is the precondition for
> "handshake amortization" (HistorianGateway `pending.md` A1).
>
> **Verdict: GREEN — reuse works and the win is large — but the server idle-expires a
> session in ~2025 s, so a reuse pool must keep sessions warm.**
**Date:** 2026-06-25
**Branch:** `spike/handshake-reuse`
**Server:** live 2023 R2 (`wonder-sql-vd03`), RemoteGrpc transport, read-only test tag.
**Harness:** `tests/AVEVA.Historian.Client.Tests/HandshakeReuseSpikeTests.cs` driving the new
internal seam `HistorianGrpcReadOrchestrator.RunRawQueryOnSession(connection, clientHandle, …)`
(runs a raw query against an externally-supplied, already-authenticated connection + handle —
no Create, no handshake).
---
## 1. Reuse validity — GREEN
`ReusedSession_RunsManyReads_AllSucceed` **passed**: one `HistorianGrpcChannelFactory.Create`
+ one `HistorianGrpcHandshake.OpenSession`, then **5 consecutive `RunRawQueryOnSession` reads on
the same `ClientHandle`** — all returned rows.
```
open-session (handshake) = 325 ms
reused-read[0] = 96 ms, rows=8
reused-read[1] = 101 ms, rows=8
reused-read[2] = 179 ms, rows=8
reused-read[3] = 92 ms, rows=8
reused-read[4] = 95 ms, rows=8
```
The server accepts the same client handle across back-to-back `StartQuery`/`GetNextQueryResultBuffer`/
`EndQuery` cycles. Per-query handles are opened/closed each op; the **session** handle is the reused
artifact.
## 2. Win magnitude — large (~4.7×)
`ReusedSession_VsPerCallPath_LogsLatencyDelta` (logged, not asserted):
```
per-call (5 ops) = 2626 ms # fresh Create + full handshake + query, ×5
amortized (5 ops) = 561 ms # one handshake + 5 reused reads
saving over 5 ops = 2065 ms
```
The handshake (`GetInterfaceVersion``ValidateClientCredential` NTLM token loop →
`OpenConnection`, ~325 ms) dominates per-call cost. Amortized, a read is ~110 ms vs ~525 ms
per-call. **Amortization is clearly worth the refactor for any burst of activity.**
## 3. Expiry — idle timeout ~2025 s (NOT an absolute TTL)
`ReusedSession_IdleSweep_SurfacesExpiryTier` rethrows at the first idle gap the server rejects.
Coarse sweep `[0, 30]`: `idle 0s → OK`, `idle 30s → BROKE`.
Fine sweep `[0,5,10,15,20,25,30]`:
```
idle 0s -> OK (rows=8)
idle 5s -> OK
idle 10s -> OK
idle 15s -> OK
idle 20s -> OK # session age here ≈ 50 s cumulative, still alive
idle 25s -> BROKE (InvalidOperationException: gRPC StartQuery (raw) failed, errorLen=5)
```
**Key inference — it's an idle timeout, not a fixed session lifetime.** The reads at gaps of
5/10/15/20 s kept succeeding even though the cumulative session age reached ~50 s by the 20 s-gap
read. The session only died after a **≥25 s idle gap**. So a session survives indefinitely as long
as operations are spaced under ~20 s apart; a quiet gap of ≥25 s invalidates it.
Expired-session failure mode on the wire: `StartQuery` returns `BSuccess=false` with a 5-byte error
buffer, surfaced by the SDK as `InvalidOperationException: gRPC StartQuery (raw) failed
(errorLen=5)`.
---
## 4. Implications for Phase 1 (the full amortization refactor)
A reuse pool is viable and high-value, with two requirements driven by §3:
1. **Keep sessions warm.** Ping each pooled session well under the ~20 s idle floor (e.g. a
~1015 s keepalive — a cheap handle-using op such as `GetSystemParameter`) so a steady-state
session never crosses the idle timeout. Without a keepalive, amortization only helps within a
<~20 s activity burst.
2. **Reactive re-auth on expiry.** Treat `StartQuery failed (errorLen=5)` (and the equivalent on
other handle ops) as an expired-session signal: evict the session and re-handshake on next use
(one handshake penalty). In HistorianGateway this maps onto the existing
`IHistorianConnectionPool.ReportFaulted` eviction seam.
**Concurrency note (unchanged guidance):** lease a session exclusively per-op from a bounded pool —
this validity test only exercised *sequential* reuse, so concurrent use of one handle (esp.
streaming cursors) remains unproven and should be avoided by exclusive leasing.
**Gate decision:** GREEN → HistorianGateway A1 Phase 1 (HistorianSession primitive + orchestrator
acquire/execute split + re-vendor + leased-session pool with keepalive) is warranted and earns its
own design + plan.
---
## 5. Write-spike addendum (Phase 1 Stage 0) — 2026-06-25
Extends the harness to the write path via the `RunWriteOnSession` seam on
`HistorianGrpcHistoricalWriteOrchestrator`. Read + bounded writes to `HISTORIAN_WRITE_SANDBOX_TAG`
only.
```
reused-write[0] = 377 ms, ok=True
reused-write[1] = 76 ms, ok=True # 2nd write reuses the same 0x401 session — no handshake
read-on-0x401 -> OK (rows=3) # a WRITE-enabled session ALSO serves reads
```
**Findings:**
- **Write-session reuse — GREEN.** Two historical writes on one reused `0x401` (write-enabled)
session both succeed; the 2nd skips the Create+handshake.
- **One-kind pool — CONFIRMED.** A `0x401` session served a `StartQuery` read (`session.ClientHandle`)
successfully. So a single **write-enabled** session serves both reads and writes — the gateway pool
needs **one session kind**, not two. (`0x401` "unlocks write capability" and is a superset of the
`0x402` read-only mode, as the vendored comment hinted.)
**Decision for Phase 1 Stage 3:** the gateway always opens `WriteEnabled` sessions; the
`HistorianSessionPool` is a **single warm pool** (no per-kind keying). `HistorianSessionKind` still
exists upstream for API clarity, but the gateway uses only `WriteEnabled`.
@@ -3,13 +3,23 @@
## Completed ## Completed
- Production SDK targets `net10.0` and has no AVEVA binary references. - Production SDK targets `net10.0` and has no AVEVA binary references.
- Public API now includes the intended parity surface: - Public API includes the full intended parity surface:
- TCP probe - TCP probe
- raw, aggregate, at-time, and block history reads - raw, aggregate, at-time, and block history reads
- event reads - event reads
- tag browse and metadata calls - tag browse and metadata calls
- connection, store-forward, and system-parameter status calls - connection, store-forward, and system-parameter status calls
- write-back intentionally remains out of scope for this read-only SDK pass - **`EnsureTagAsync`** for analog Float/Double/Int2/Int4/UInt4 with
optional `ApplyScaling=true` for distinct MinRaw/MaxRaw persistence
(live-verified end-to-end against `localhost`; SQL post-check confirms
`AnalogTag.Scaling=1` and distinct raw bounds when the flag is set)
- **`DeleteTagAsync`** (live-verified)
- **AddS2 (write samples) is architecturally blocked** — server runtime
cache only ingests from configured IOServers / Application Server
pipelines, not from `HistorianAccess.AddTag`-only flows. Three
independent reproduction attempts confirmed the same
`129 "Tag not found in cache"` failure even with the real wwTagKey,
fresh sessions, and 8s settle waits. Not a protocol gap.
- Internal protocol scaffolding exists: - Internal protocol scaffolding exists:
- `HistorianConnection` - `HistorianConnection`
- `HistorianFrameReader` - `HistorianFrameReader`
@@ -0,0 +1,131 @@
# Extended-property write over 2020 WCF — AddTEx (HCAL R1.11)
**Status: ✅ Add DONE + live-verified (2026-06-21). Delete (DelTep) deferred — see below.**
`HistorianClient.AddTagExtendedPropertiesAsync` / `AddTagExtendedPropertyAsync` writes user-defined
extended properties onto an existing tag via the 2020 WCF **`AddTEx`** (AddTagExtendedProperties) op,
and they read back via the R1.5 `GetTagExtendedPropertiesAsync` path. Verified end-to-end from the
pure-managed .NET 10 client against the local 2020 Historian (create tag → add property → read back →
delete tag).
## The op
```
bool AddTagExtendedProperties(string handle, byte[] inBuff, out byte[] errorBuffer) // AddTEx
```
On `IHistoryServiceContract2` (History service). Requires a **write-enabled** connection (Open2 mode
`0x401`) and the uppercase storage-session GUID handle — the SDK reuses the write orchestrator's
open + priming chain (the same one used by EnsT2/DelT). The tag is referenced by name inside `inBuff`;
no extra per-connection tag registration was needed (the server resolves it).
## The inBuff — the exact inverse of the R1.5 read response
The native `AddTagExtendedProperties(TagExtendedPropertyGroupList, out err)` packs its groups into the
`AddTEx` `inBuff` with the **same framing the R1.5 `GetTepByNm` response uses**, so the write serializer
is the inverse of `HistorianTagExtendedPropertyProtocol.ParseResponse`:
```
uint32 groupCount (= 1)
byte 0x01 (group marker)
0x09 + uint16 byteLen + ASCII tagName (compact-ASCII string)
uint32 propertyCount
repeated propertyCount times:
byte 0x02 (property marker)
0x09 + uint16 byteLen + ASCII propertyName
0x43 + uint16 payloadLen + uint16 charCount + UTF-16LE value (VT_BSTR variant; payloadLen = 2 + charCount*2)
byte 0x01 (group trailer)
byte 0x00 (buffer terminator)
```
⚠️ **The trailing `0x01 0x00`** matters: the group trailer is `0x01` (as in the read parser) **plus a
final `0x00` buffer terminator**. Omitting the `0x00` makes `inBuff` one byte short and the server throws
`SErrorException` in `aahClientAccessPoint::CHistStorage::AddTagExtendedProperties` (AddTEx returns
false). The read parser tolerates the extra byte because it only consumes one trailing byte per group.
Only the string (`0x43` VT_BSTR) value variant is evidence-backed (matching the read path). The raw
instrument capture mangles the final byte with MDAS chunk markers, so the golden fixture pins the
**clean** byte[] the SDK handed the channel (dumped via `AVEVA_HISTORIAN_TEP_DUMP`) — the exact buffer
the live server accepted.
## Delete (DelTep) — wire format captured + serializer proven; live delete server-blocked
**Status (2026-06-21): the `DelTep` wire format is captured and decoded, the serializer is
golden-verified against a server-accepted buffer, and SDK-added properties are confirmed deletable —
but the SDK's own delete is rejected server-side and is therefore NOT exposed publicly.** This is a
much deeper result than the earlier "couldn't capture the inBuff" deferral.
### Capturing it: the cross-session trick
The native `DeleteTagExtendedPropertiesByName(tag, propertyNames, deleteFromServer, out err)` performs
a **client-side sync check** and returns error **229 ("Tag extended property not synchronized with
server")** when deleting a *just-added* property — so a same-session add→delete never reaches the wire.
`AddTEx` success does **not** mark the local cache entry as server-synchronized; only a *server fetch*
(`GetTepByNm`) does. So the capture (`scripts/Capture-DeleteTagExtendedProperties.ps1`) runs two
separate harness processes against one instrumented DLL:
- **Run A**: `add-tep` creates the sandbox tag and adds the property (now server-synced).
- **Run B**: a fresh process opens a new connection, fetches the property
(`GetTagExtendedPropertiesByName`, which seeds the local cache as synced), then deletes it — so
`DeleteTagExtendedPropertiesByName` passes the client gate and `DelTep` reaches the wire.
### The inBuff — same group framing as Add, names only
```
uint32 groupCount (= 1)
byte 0x01 (group marker)
0x09 + uint16 byteLen + ASCII tagName
uint32 propertyCount
repeated propertyCount times:
byte 0x02 (property marker) + 0x09 + uint16 byteLen + ASCII propertyName ← NO value variant
byte 0x00 (group trailer) ← 0x00 for delete, vs 0x01 for add
byte 0x00 (buffer terminator)
```
The native `deleteFromServer` argument is **not** in the buffer — it is the client-side flag that
decides whether the wire op fires at all (`true` ⇒ a `DelTep` call). `HistorianTagExtendedProperty
Protocol.SerializeDeleteRequest` produces this exactly; `WcfTagExtendedPropertyWriteProtocolTests`
pins the server-accepted bytes.
### Why the SDK delete is server-blocked
The SDK's `DelTep` is rejected by the server with `SErrorException` in
`aahClientAccessPoint::CHistStorage::DeleteTagExtendedProperties`, even though:
- the **inBuff is byte-identical** to the server-accepted native capture (golden-verified);
- the **Open2 connection mode matches** the native (`0x401`, confirmed from the capture at offset
0x4a4);
- the **handles match** (uppercase storage-session GUID for `DelTep`/`GetTepByNm`, uint client handle
for `GetTgByNm`);
- the SDK first **primes the session** with `GetTgByNm` (tag identity, returns 140 bytes of tag info)
and `GetTepByNm` (returns the property), keeping the Retrieval prime channel **open across** the
`DelTep` call;
- retried with backoff for **60 s** (ruling out a storage-tier sync delay).
A decisive experiment localizes the gap: an **SDK-added** property *is* deletable — the native client
read-syncs and deletes it (`Success:true`). So the SDK's **add is complete**; only the SDK's **delete
session** is the problem. The native client multiplexes Hist/Retr/Stat/Trx over **one connection**
under a single `HistorianAccess` session, so its `GetTepByNm` populates a **per-connection working set**
that the same-connection `DelTep` consults. The SDK uses **separate WCF channels per service** (the
proven read pattern), so the borrowed-GUID Retrieval prime doesn't satisfy that server-side check.
Reproducing it requires transport-level connection multiplexing — a substantial change beyond this op.
The investigated-but-blocked orchestration is kept (internal
`HistorianWcfTagWriteOrchestrator.DeleteTagExtendedPropertiesAsync`, the `PrimeThenDelete` helper) for
follow-up, but `HistorianClient` deliberately exposes **no** public delete to avoid a silently-failing
write API. Capture/decode tooling: `scripts/Capture-DeleteTagExtendedProperties.ps1` +
`scripts/decode-del-tep-capture.py`, harness `add-tep` scenario with `--tep-skip-add` / `--tep-delete`.
## Shipped surface
- `HistorianClient.AddTagExtendedPropertiesAsync(tag, IReadOnlyList<HistorianTagExtendedProperty>)` and
`AddTagExtendedPropertyAsync(tag, name, value)`.
- `HistorianTagExtendedPropertyProtocol.SerializeAddRequest` (the inBuff serializer; lives beside the
R1.5 read parser); orchestrator path in `HistorianWcfTagWriteOrchestrator`.
- Golden `WcfTagExtendedPropertyWriteProtocolTests` (pins the server-accepted buffer + layout); gated
live test `AddTagExtendedPropertiesAsync_AgainstLocalHistorian_WritesAndReadsBack`.
## Capture / decode tooling
`scripts/Capture-AddTagExtendedProperties.ps1` (native-harness `add-tep` scenario +
instrument-wcf-{write,read}message; sandbox-guarded create→add→[optional delete]) and
`scripts/decode-add-tep-capture.py`.
@@ -159,5 +159,33 @@ the earlier speculative raw-frame layer.
handle `0`. See `wcf-status-localhost.md`. handle `0`. See `wcf-status-localhost.md`.
- Query request and response byte-buffer layouts are still proprietary payloads - Query request and response byte-buffer layouts are still proprietary payloads
inside WCF operations such as `StartQuery` and `GetNextQueryResultBuffer`. inside WCF operations such as `StartQuery` and `GetNextQueryResultBuffer`.
- Write payload layouts remain out of scope until read/query payloads are - Write payload layouts decoded for the two supported ops:
decoded and fixture-backed. - `Hist.EnsT2(analog)` 144-byte `CTagMetadata` `InBuff` payload —
leading `0x4E` marker, fixed 10-byte signature, 1-byte CDataType
discriminator (`0x01` Float / `0x21` Double / `0x09` UInt2 / `0x11`
UInt4 / `0x29` Int2 / `0x31` Int4), 16 zero placeholder bytes,
compact-ASCII tag name, 16 bytes of `0xFF`, compact-ASCII description,
compact-ASCII `MDAS`, 7-byte flag block, uint32 storage rate,
int64 FILETIME, scaling block (compact `1A 03` for default
0/100/0/100 ranges OR `1F 00` + 4 doubles MinEU/MaxEU/MinRaw/MaxRaw
for explicit), compact-ASCII engineering unit, uint32 `0x2710`
constant, double 1.0 (IntegralDivisor), 2-byte trailer `FE xx`
where `xx` is the ApplyScaling flag (`0x00` false / `0x01` true).
Live-verified: with `0x01` the server persists distinct
MinRaw/MaxRaw and sets `AnalogTag.Scaling=1`; with `0x00` it
mirrors MinRaw to MinEU. Captured fixtures live at
`artifacts/reverse-engineering/instrumented-wcf-writemessage-writes/`
(default ranges) and
`artifacts/reverse-engineering/apply-scaling-experiment/` (both
ApplyScaling values for the same input ranges). Connection mode is
`0x401` (Process | Write | IntegratedSecurity) — the read-mode
`0x402` makes the server return err 132 silently.
- `Hist.DelT` `tagNames` byte buffer — `ushort 0x6751`, `ushort 1`,
`uint32 tagCount`, then per tag `uint32 charCount + UTF-16-LE chars`.
Decoded via wire capture against the sandbox tag.
- `Hist.AddS2` (write samples) is architecturally blocked — server
runtime cache requires IOServer / Application Server pipeline
registration, not just a `Tag` row in `Runtime.dbo`. Three
reproduction attempts (real wwTagKey, fresh session, 8s settle
wait) confirmed `129 "Tag not found in cache"` is the gate. No
AddS2 wire bytes leave the client.
@@ -0,0 +1,75 @@
# WCF event-read spike — live result (2026-06-25/26): transport+auth viable, row-retrieval server-gated
Settles the open question behind **C2** ("event reads over gRPC are gated; the only listed unblock is
*route event reads via WCF*"). The gRPC event-read path is a proven server-side dead-end
(`grpc-event-query-capture.md`: auth fully solved, every client-controllable layer byte-matched to the
stock client, yet the server scopes 0 rows to our connection). This spike resolved the **WCF** leg.
> **Correction to an earlier draft of this doc.** A first pass concluded "the 2023 R2 historian does not
> serve the legacy WCF transport (connection reset at framing)." **That was a test error, not a server
> fact.** It connected to the historian's real WCF port `32568` *directly* and used the Windows-integrated
> transport. In this environment the historian is reached through a **reverse SSH tunnel** (local
> `42568` → historian `32568`), and integrated/Kerberos auth does not work through that tunnel. The
> socket-RST was the tunnel/transport mismatch, not an absent listener. Corrected below.
## What was run
A Windows-only-by-default, env-gated diagnostic (`tests/AVEVA.Historian.Client.Tests/WcfEventReadSpikeTests.cs`)
drives `HistorianWcfEventOrchestrator.ReadEventsAsync` directly. The decisive run was **cross-platform,
direct** (no tunnel): from the VPN-holding host straight to the historian's real WCF endpoint
`net.tcp://<historian>:32568/HistCert`, using the **certificate transport** (`RemoteTcpCertificate`,
TLS, `AllowUntrustedServerCertificate`) and `NegotiateAuthentication` (cross-platform, explicit domain
credentials). The SDK's interface-version gate was bypassed (`VerifyServerInterfaceVersion=false`) —
the 2023 R2 WCF **History interface reports version 13** (this SDK's serializers target 11/12).
## Result — transport+auth viable; row-retrieval server-gated (sanitized)
Progression of the live errors as the addressing/transport was corrected:
| attempt | error |
|---|---|
| direct `:32568`, integrated | `SocketException` "forcibly closed" (wrong port + transport for the tunnel) |
| tunnel `:42568`, integrated | `ProtocolException` at the security UpgradeResponse (integrated can't negotiate through the tunnel) |
| tunnel `:42568`, certificate | reached the WCF dispatcher → `AddressFilter` mismatch (tunnel rewrites the port) |
| **direct `:32568`, certificate, cross-platform** | **past auth**`ProtocolEvidenceMissingException`: History interface version **13** |
| + `VerifyServerInterfaceVersion=false` | **full chain runs**; query returns a 10-byte **0-row** header, then `GetNext` long-polls |
Connection-mode experiment (certificate transport, direct, version-bypassed, a 1-day window that holds
events), comparing the native OpenConnection mode used for the event-read chain:
| connMode | RegisterTags (RTag2) | EnsureTags (EnsT2) | result buffer | events |
|---|---|---|---|---|
| `0x501` (event) | **0 — success** | 1 (benign-false, as in the 2020 flow) | 10 bytes (0-row header) | **0** |
| `0x401` (write) | 1 (fail) | 1 | 10 bytes | 0 |
| `0x402` (read-only, default) | 1 (fail) | 1 | 10 bytes | 0 |
## Conclusion
1. **WCF transport + auth ARE viable on 2023 R2.** The certificate (TLS) transport negotiates and the
`NegotiateAuthentication` app-level handshake authenticates — **cross-platform** (proven from a
non-Windows VPN host). The earlier "WCF not served" conclusion was wrong. (Integrated/Windows
transport security is not usable through the reverse tunnel — `net.tcp` Kerberos does not tunnel.)
2. **The event-read chain needs the `0x501` event connection mode.** With it, CM_EVENT `RegisterTags`
**succeeds** (it fails on `0x402`/`0x401`). `EnsureTags` returns false, but that is documented as
benign in the 2020 flow that *did* return rows.
3. **Row retrieval is server-gated — same as gRPC.** Even with auth solved and `RegisterTags` succeeding,
over a window that holds events, `StartEventQuery` succeeds but `GetNextEventQueryResultBuffer` returns
a **0-row** header (10 bytes) and long-polls. Registration and window are ruled out as the cause; the
server simply does not scope event rows to a managed connection. This is the **identical** server-side
per-connection retrieval working-set gate proven for gRPC in `grpc-event-query-capture.md`.
**Therefore event reads do not return rows on the 2023 R2 historian over either transport** — gRPC
(retrieval-server-gated) and WCF (transport+auth work, but the same server-side row gate). The only
remaining theoretical unblock is server-side (AVEVA exposing event-row retrieval to a managed
connection) — not client-fixable. **C2 stays closed won't-fix**, for this (corrected) reason.
## SDK additions from this investigation (retained, build-clean, golden where applicable)
- `HistorianClientOptions.ConnectViaAddress` — WCF `Via` (connect to a tunnel/proxy while addressing the
SOAP `To` the real endpoint), so a port-forward whose local port differs from the server's real port
satisfies the server-side WCF AddressFilter.
- `HistorianClientOptions.EventReadConnectionModeOverride` — diagnostic override of the event-read
OpenConnection mode (the `0x501` finding above).
- The C2 spike is now transport-selectable (integrated|certificate), cross-platform for the cert
transport, bounded (per-call timeout + overall budget with a phase-diagnostic dump), and version-gate
bypassable. Output stays sanitized (counts, native return codes, buffer lengths, sha256).
+86
View File
@@ -0,0 +1,86 @@
# SQL command execution over 2020 WCF — ExeC + GetR (HCAL R1.1)
**Status: ✅ DONE + live-verified (2026-06-20).** `HistorianClient.ExecuteSqlCommandAsync(sql)` runs a
SQL command against the Historian over the 2020 WCF ops `aa/Retr/ExeC` (ExecuteSqlCommand) +
`aa/Retr/GetR` (GetRecordSetByteStream) and returns the record set as a `HistorianSqlResult` (the
managed equivalent of the native `DataTable`). Live-verified end-to-end from the pure-managed .NET 10
client against the local 2020 Historian (single-cell, multi-column/multi-row, and NULL cases).
## The ops
Both are on the Retrieval service (`IRetrievalServiceContract3`), **string-handle** ops reached with
the Open2 storage-session GUID formatted `storageSessionId.ToString("D").ToUpperInvariant()` (the same
uppercase handle that unlocked GETRP/GETHI; see `wcf-string-handle-wall.md`). `Retr.GetV` is primed
first.
```
bool ExecuteSqlCommand(string handle, string command, uint option,
ref uint queryHandle, out int retValue, out uint errorSize, out byte[] errorBuffer)
bool GetRecordSetByteStream(string handle, uint queryHandle, ref uint sequence,
out uint resultSize, out byte[] pResultBuff, out uint errorSize, out byte[] errorBuffer)
```
- **`command`** is sent as a plain string (MDAS-encoded ASCII), e.g. `SELECT 1 AS ProbeValue`.
- **`option`** = `HistorianSqlExecuteOption` (`ExecuteRecord=0` is the captured/proven record-set path).
- **`ExeC`** returns the assigned `queryHandle` (and `retValue`); pass it to `GetR`.
- **`GetR` returns `false` even on success** — the byte stream is in `pResultBuff` regardless; a
`false` result just signals the final page. So the orchestrator always consumes `pResultBuff`, then
stops on a `false` result or an empty page. (`sequence` is the paging cursor; small record sets
return everything in one call.)
## The result stream — NRBF-wrapped DataTable (no BinaryFormatter)
`GetR`'s `pResultBuff` is a **.NET Remoting Binary Format (NRBF)** stream wrapping a
`System.Data.DataTable` serialized with `SerializationFormat.Xml`. Stream shape (captured):
```
SerializationHeader (00 01 00 00 00 FF FF FF FF 01 00 00 00 00 00 00 00)
BinaryLibrary (0C): "System.Data, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
ClassWithMembersAndTypes (05): System.Data.DataTable, members:
DataTable.RemotingVersion -> System.Version object
XmlSchema -> BinaryObjectString (XSD: column names + xs types)
XmlDiffGram -> BinaryObjectString (ADO.NET diffgram: the rows)
System.Version object (_Major/_Minor/_Build/_Revision)
MessageEnd (0B)
```
**BinaryFormatter is removed from .NET 10**, so the stream is decoded read-only with
`System.Formats.Nrbf.NrbfDecoder` (the sanctioned successor — it parses records without instantiating
or executing any payload type; added as a managed `PackageReference`). The two embedded XML strings are
then parsed with `XDocument`:
- **XmlSchema** → columns (name + XSD type) under the `msdata:MainDataTable` element's sequence.
- **XmlDiffGram** → rows (each row element under the dataset; cells are child elements or attributes
matching column names). NULL cells are simply absent → parsed as `null`.
Values are typed per the XSD type (`xs:int`→int, `xs:string`→string, `xs:dateTime`→DateTime, …),
falling back to the raw string for unrecognized types. Only the `SerializationFormat.Xml` DataTable
shape is evidence-backed; a stream whose root is not a DataTable class record, or that lacks the two
XML members, throws `ProtocolEvidenceMissingException`.
## Capture / decode tooling
`scripts/Capture-ExecSql.ps1` (NativeTraceHarness `exec-sql` scenario + instrument-wcf-{write,read}message)
captures the ExeC/GetR exchange. ⚠️ A **raw** instrument-wcf capture interleaves MDAS transport chunk
markers (`0x9F`/`0x9E`) into a large `pResultBuff`, so raw byte-slicing yields a corrupted NRBF stream.
The **clean** contract-level byte[] (what the WCF channel reassembles) is dumped via the
`AVEVA_HISTORIAN_SQL_DUMP` env var on `HistorianWcfSqlClient` — that is the golden fixture in
`WcfSqlResultProtocolTests` (the benign `SELECT 1 AS ProbeValue`, no sensitive data).
## Shipped surface
- `HistorianClient.ExecuteSqlCommandAsync(command, option = ExecuteRecord)``HistorianSqlResult`
(`Columns` name+type, `Rows` typed values, `ReturnValue`).
- Models `HistorianSqlResult` / `HistorianSqlColumn` / `HistorianSqlExecuteOption`;
`HistorianSqlResultProtocol` (NRBF + diffgram parser); `HistorianWcfSqlClient` (ExeC/GetR
orchestration); golden `WcfSqlResultProtocolTests`; gated live tests
(`ExecuteSqlCommandAsync_AgainstLocalHistorian_ReturnsRecordSet` and `_MultiColumnMultiRow`).
## Scope notes
- `ExecuteRecord` (record set) is the evidence-backed path. `ExecuteNonQuery`/`ExecuteScalar`/
`ExecuteRecordDirect` are accepted via the option enum but their non-record-set return shapes are not
separately captured — a non-record result yields empty `Columns`/`Rows` with the `ReturnValue` set.
- The command is whatever the Historian's SQL surface accepts (it routes to the Runtime DB). No
client-side SQL validation is performed.
@@ -0,0 +1,81 @@
# GetHistorianInfo over 2020 WCF — GETHI is named-value-only (HCAL R1.4)
**Status: ⛔ Bounded out on BOTH 2020 WCF and 2023 R2 gRPC (2026-06-20; gRPC live-confirmed
2026-06-21).** `GetHistorianInfoAsync` is **not shipped on any transport**: the one field that
motivates it — `EventStorageMode` — is **not on the wire** on either transport (it lives only in
the C++ HCAL's in-memory 518-byte struct, filled via a native vtable+648 call — see the §gRPC
conclusion below). The version field GETHI *does* return is already exposed (`ProbeAsync`,
`GetRuntimeParameterAsync("HistorianVersion")`), so there is nothing new to ship. Note: R1.3
(`GetServerTimeZone`) — once paired with this as "2023R2-only" — **diverged**: it returns a real
value over gRPC and **shipped** 2026-06-21 (`GetServerTimeZoneAsync`); R1.4 did not.
## What the capture showed
`scripts/Capture-HistorianInfo.ps1` drives the native `HistorianAccess.GetHistorianInfo(out
HistorianInfo, out error)` through the instrumented (`instrument-wcf-{write,read}message`)
`current/aahClientManaged.dll`. The native call **succeeds** and returns
`EventStorageMode = Blocks`, `ServerVersion = 20,0,000,000`, no error.
But the wire tells a different story (`scripts/decode-historian-info-capture.py`):
- The only `GETHI` op on the wire is **`aa/Stat/GETHI(handle, pRequestBuff)`** with
`pRequestBuff = 53 67 02 00` (sig `0x6753` + version `2`) `+ uint charCount(16) + UTF-16
"HistorianVersion"` — i.e. the **named-value request**, identical to the GETRP/version shape.
- Its response `pResponseBuff` is **~30 bytes**: `uint charCount(12) + UTF-16 "20,0,000,000"`
(+ a `02 00 01 00` trailer). **Just the version** — not a 518-byte struct.
- The post-GETHI ops in the same capture are `Hist/UpdC3` + a run of `Stat/GetSystemParameter`
(`AllowOriginals`, `HistorianPartner`, `HistorianVersion`, `MaxCyclicStorageTimeout`,
`RealTimeWindow`, `FutureTimeThreshold`, `AllowRenameTags`). **None carries a storage-mode
value.** So the native wrapper's `EventStorageMode` is derived by the C++ HCAL **outside the
WCF wire**, not fetched over it.
## Probe: does GETHI expose storage mode under any name?
`StringHandleProbeDiagnosticTests.GETHI_CandidateInfoNames_AgainstLocalHistorian` (gated on
`HISTORIAN_HOST=localhost`) issues GETHI for `HistorianVersion` plus seven storage-mode name
guesses. Result on the live 2020 server:
| GETHI parameter name | result |
|---|---|
| `HistorianVersion` | **ok=True**, respLen=32 (version) |
| `EventStorageMode`, `EventStorageType`, `StorageType`, `HistorianEventStorageMode`, `EventStorage`, `StorageMode`, `HistorianInfo` | **ok=False**, errLen=5, empty |
So GETHI on 2020 WCF is a strict named-value lookup with exactly one known-good key
(`HistorianVersion`). There is no storage-mode key, no full-struct request.
## Why the 518-byte struct doesn't apply here
The 2023 R2 decompiled `ArchestrA.HistorianAccess.GetHistorianInfo` (analysis folder) allocates
a **518-byte `HISTORIAN_INFO`** struct, pre-inits `int32 @514` to `-1`, calls native HCAL
(vtable+648) which fills it, then reads version (UTF-16 @0) + `EventStorageMode` (`@514`:
`-1`=Unsupported, `0`=Database, else=Blocks). That is the **HCAL-native / 2023R2 gRPC**
front-door model (`StatusService.GetHistorianInfo` returns `bytes btHistorianInfo`). On **2020
WCF** that struct is never marshaled across the wire — only the version named-value is. The
native client's `EventStorageMode` therefore comes from C++-internal state the managed WCF
replay cannot observe or reproduce.
## Conclusion / where it lands
- **2020 WCF:** `GetHistorianInfoAsync` would add nothing over existing surface (version only) and
could not report a real `EventStorageMode` — so it is intentionally **not shipped** (no hollow
`Unsupported`-returning API; project discipline: don't ship misleading behavior).
- **2023 R2 gRPC — LIVE-PROBED 2026-06-21, also bounded out.** The earlier expectation that
`Status.GetHistorianInfo` returns the full 518-byte `btHistorianInfo` over gRPC was **wrong**. On
the real 2023 R2 server (History iface 12), the gRPC `GetHistorianInfo` is the
**same named-value query** as 2020 WCF: only `HistorianVersion` resolves (→ `"23,1,000,000"` +
`02 00 01 00` trailer); `EventStorageMode` and seven name variants return `success=false` on
**both** `GetHistorianInfo` and `GetSystemParameter`. The 518-byte struct is **not on the gRPC
wire** — the 2023 R2 decompile confirms managed `HistorianAccess.GetHistorianInfo` fills it via a
**native vtable+648 HCAL call** (`IClientCommon*` + offset 648), not the gRPC op, so
`EventStorageMode` is derived inside the C++ HCAL outside the wire on gRPC exactly as on WCF.
**Conclusion: `GetHistorianInfoAsync` is not shipped on any transport** (the only wire-reachable
field, version, is already exposed). No `HistorianInfo` / `HistorianEventStorageMode` public type
was added. Probe: the (now-deleted) `GrpcStatusInfoProbeTests`; raw dump under
`artifacts/reverse-engineering/grpc-status-info-probe/` (gitignored).
## Tooling kept as RE aids
- `tools/AVEVA.Historian.NativeTraceHarness` `historian-info` scenario (drives the native call).
- `scripts/Capture-HistorianInfo.ps1` + `scripts/decode-historian-info-capture.py`.
- `StringHandleProbeDiagnosticTests.GETHI_CandidateInfoNames_AgainstLocalHistorian` (locks the
named-value-only finding; gated).
@@ -0,0 +1,67 @@
# Non-analog tag create over 2020 WCF — GATED (HCAL R1.13)
**Status: ⛔ bounded out (2026-06-21). No non-analog (string / discrete / wide-integer) tag-create
path is reachable on 2020 — the native managed client rejects every non-analog type *client-side*,
before any WCF op, so there is no wire format to capture and nothing to implement against.**
`EnsureTagAsync` stays analog-only (Float, Double, Int2, UInt2, Int4, UInt4); unsupported types throw
`ProtocolEvidenceMissingException` from `HistorianTagWriteProtocol.GetAnalogDataTypeCode`.
## What R1.13 asked for
Create string / discrete (non-analog) tags via `History.EnsureTags`, with a distinct `CTagMetadata`
variant. The roadmap flagged it "⚠ native AddTag rejected some types — confirm server path first;
may be GATED."
## Findings (live-probed against the local 2020 Historian)
1. **No discrete/boolean data type exists.** The native `ArchestrA.HistorianDataType` enum
(`current/aahClientManaged.dll`, dumped via `enum-dump`) has exactly 12 members: `Int1, Int2,
UInt2, Int4, UInt4, Float, Double, SingleByteString, DoubleByteString, Event, Structure`. There is
no `Discrete`/`Boolean`, and no `Int8`/`UInt8`/`UInt1`/`Guid`/`FileTime` (those are SDK-only
extensions in `Models/HistorianDataType`, recovered from the C++ `CDataType` predicate IL — they
are not settable on the managed `HistorianTag`).
2. **Tag type is data-type-derived, not separately settable.** `ArchestrA.HistorianTag`
(`--dump-type-members`) has **no** `TagType` property — only `TagDataType`. It does carry the
discrete/string-shaped fields (`MessageOn`/`MessageOff`, `RolloverValue`, dead-band/interpolation),
and the type exposes `ValidateAnalog*`, `ValidateDiscreteGeneralProperties`, and
`ValidateDiscreteAndStringStorageProperties` — but the analog-vs-discrete-vs-string decision is made
internally from the data type, with no way to request "discrete."
3. **Native AddTag rejects every non-analog type client-side.** Driving the native
`HistorianAccess.AddTag(HistorianTag, …)` (harness `write` scenario, `--write-data-type`) against
the live server:
| Data type | AddTag.Success | ErrorCode | ErrorType |
|---|---|---|---|
| SingleByteString | **false** | ValidationFailed | CustomError |
| DoubleByteString | **false** | ValidationFailed | CustomError |
| Int1 | **false** | ValidationFailed | CustomError |
| Int8 / UInt8 | n/a | *not in the native enum* | — |
| Float (control) | true | Success | — |
The error — `ErrorType=CustomError`, `ErrorCode=ValidationFailed`, `ErrorDescription="Transaction
validation failed"` — is raised by the client's own `Validate*` chain **before any WCF message is
sent** (the wrapper even auto-populates discrete defaults `MessageOn=ON`/`MessageOff=OFF`, then
fails validation). So the native client never emits a non-analog `EnsT2`/`AddTag` request.
## Why it's not deliverable here
Because the native client refuses non-analog types client-side, **no wire request exists to
reverse-engineer** — there is no captured `CTagMetadata` variant for string or discrete tags, and the
SDK does not guess wire bytes. The rejection is not string-specific: `Int1` (a non-string integer
outside the analog set `{Float, Double, Int2, UInt2, Int4, UInt4}`) fails identically, so the boundary
is "the analog set" rather than "strings only." Creating string/discrete tags on 2020 evidently goes
through a different subsystem (e.g. the configuration editor / SQL config path), not this client's
`AddTag`. R1.13 is closed as GATED, consistent with the mission note that these types "fail at native
AddTag — likely require a different path and are intentionally not supported."
## Probe commands (read-only / sandbox-guarded)
```
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- enum-dump current\aahClientManaged.dll HistorianDataType
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- dnlib-method current\aahClientManaged.dll HistorianTag.ValidateDataType
# native AddTag probe (sandbox tag must start with RetestSdkWrite; --write-skip-add-value avoids the blocked value path):
dotnet run --project tools\AVEVA.Historian.NativeTraceHarness -- --scenario write --server-name localhost --tcp-port 32568 \
--write-sandbox-tag RetestSdkWriteNADoubleByteString --write-data-type DoubleByteString --write-skip-add-value
```
@@ -0,0 +1,82 @@
# Tag rename over 2020 WCF — StartJob (StJb) rename job (HCAL R1.10)
**Status: ✅ DONE + live-verified (2026-06-21).** `HistorianClient.RenameTagsAsync` /
`RenameTagAsync` renames tags by submitting an asynchronous rename **job** to the Historian. Decoded
from an instrumented native `RenameTags` capture and verified end-to-end from the pure-managed .NET 10
client against the local 2020 Historian (sandbox tag created → renamed → new name visible → cleaned up).
## The op — rename rides the generic job framework
There is **no dedicated rename WCF operation**. The native `RenameTags(Tuple<string,string>[] pairs,
ref HistorianTagRenameStatus, out error)` packs the batch into the generic History **`StartJob`**
(`StJb`) buffer; the server returns a job id and applies the renames in the background. The native
client then polls `GetJobStatus` (`GtJb`) until the job reports done.
```
bool StartJob(string handle, byte[] jobBuffer, out string jobId, out byte[] errorBuffer) // StJb
bool GetJobStatus(string handle, string jobId, out byte[] jobStatus, out byte[] errorBuffer) // GtJb
```
Both already existed in `IHistoryServiceContract2`. `StartJob` takes a **string handle** = the Open2
storage-session GUID formatted `storageSessionId.ToString("D").ToUpperInvariant()` (the same uppercase
handle used by the other string-handle ops). The connection must be **write-enabled** (Open2 mode
`0x401`); the SDK reuses the write orchestrator's open + priming chain.
## The rename jobBuffer (decoded + server-validated)
```
byte[7] reserved / job-descriptor prefix (all zero in every capture)
uint32 pairCount
repeated pairCount times:
uint32 oldNameCharCount + UTF-16LE oldName (the tag being renamed)
uint32 newNameCharCount + UTF-16LE newName (its new name)
```
Char counts are UTF-16 code-unit counts. Pair order is **(old, new)**. ⚠️ A **raw** instrument
capture mangles the buffer's final byte with MDAS chunk markers (`9E`/`9F`) — the same hazard noted
for R1.1. So the golden fixture is the **clean** byte[] the SDK hands the WCF channel, dumped via the
`AVEVA_HISTORIAN_RENAME_DUMP` env hook on `HistorianWcfTagWriteOrchestrator`. That exact buffer was
accepted by the live server and the tag was renamed, so it is server-validated, not hand-stitched.
## Server gate — `AllowRenameTags`
Rename is gated by the **`AllowRenameTags`** system parameter (default **0/disabled**). When disabled,
the **native** client library rejects the call *before the wire* (`error 132 OperationNotEnabled`,
component `aahClientCommon::CClientCommon::RenameTags`); the managed SDK has no such pre-check, so a
disabled gate surfaces as `StartJob` returning false (reported as `Accepted = false`).
To enable for testing: `EXEC Runtime.dbo.aaSystemParameterUpdate @name='AllowRenameTags', @value=1`
**and reload the Historian config** — the running services cache system parameters, so the value only
takes effect after the Historian reloads (a Historian restart; a storage-engine-only restart is **not**
enough — the value is served from the `InSQLConfiguration` cache). Restore to `0` when done.
## Async completion
`RenameTagsAsync` submits the job and returns `HistorianTagRenameResult { Accepted, JobId, PairCount,
Error }`. The renames apply asynchronously server-side (observed: the native `GetTagRenameStatus` went
`Pending=true``false` within ~1.5 s for a single rename on the local box). The SDK does **not** poll
`GtJb` for completion: only the *pending* `jobstatus` buffer (6 zero bytes) was captured — the
done-state encoding was not, so polling is intentionally left out rather than guessing it. The gated
live test confirms completion by polling the new name's metadata after submission.
## Shipped surface
- `HistorianClient.RenameTagAsync(old, new)` / `RenameTagsAsync(IReadOnlyList<(string,string)>)`
`HistorianTagRenameResult`.
- `HistorianTagRenameProtocol.SerializeRenameJob` (the jobBuffer serializer);
`HistorianWcfTagWriteOrchestrator.RenameTags`/`SendStartJobRename` (open → write-priming → StJb).
- Golden `WcfTagRenameProtocolTests` (pins the server-accepted buffer + layout); gated live test
`RenameTagsAsync_AgainstLocalHistorian_RenamesSandboxTag` (needs `HISTORIAN_HOST=localhost`,
`HISTORIAN_RENAME_SANDBOX=RetestSdkWrite…`, and `AllowRenameTags` enabled).
## Capture / decode tooling
`scripts/Capture-RenameTags.ps1` (native-harness `rename` scenario + instrument-wcf-{write,read}message;
sandbox-guarded create→rename→cleanup) and `scripts/decode-rename-capture.py`.
## Scope notes
- String-valued, original tag renames only. The multi-pair batch framing is captured (count-prefixed)
and unit-tested; the live test exercises a single pair.
- `RenameSourceTags` (replication/source-server rename) is **not** shipped — different op signature
(adds a source-server string), not captured.
@@ -26,9 +26,92 @@ Observed sanitized localhost results:
- `GetSystemParameter(handle: 0, "Version")` returns `false` with no error - `GetSystemParameter(handle: 0, "Version")` returns `false` with no error
buffer. buffer.
Re-tested 2026-06-20 with a **real authenticated client handle** (full Open2 auth
chain), not `handle: 0`:
- `GetSystemParameter(handle, "HistorianVersion")` → real version string (works;
shipped as `GetSystemParameterAsync`).
- `GetSystemTimeZoneName(handle)` → return code `0x00000000` (success) but an
**empty value string**. Same channel/handle that makes `GetSystemParameter`
return real data, so this is the op's own behavior, not an auth/marshalling
gap. `GetSystemTimeZoneName` is a member of the `GetServerTime` stub family:
the 2020 WCF path returns success without producing a value (the native client
computes the zone locally). It only becomes a real round-trip on the 2023 R2
gRPC front door (`Status.GetSystemTimeZoneName`), which is absent on this box.
Interpretation: Interpretation:
- `Stat` endpoint routing is confirmed, but status operations that require a - `Stat` endpoint routing is confirmed, but status operations that require a
real client handle are not usable until managed session open is solved. real client handle are not usable until managed session open is solved.
- `GetServerTime` should not be promoted into the public SDK as a real server - `GetServerTime` should not be promoted into the public SDK as a real server
time call from this WCF path; native evidence shows it is a no-op stub here. time call from this WCF path; native evidence shows it is a no-op stub here.
- **`GetServerTimeZoneAsync` (roadmap R1.3) is NOT a trivial WCF op on 2020** — it
is a stub returning empty. Do not ship it over the 2020 WCF transport. Deliver
it only against a live 2023 R2 gRPC server. Reclassified in `docs/plans/hcal-roadmap.md`.
## GETRP / GetRuntimeParameter (roadmap R1.2) — DONE, live-verified 2026-06-20
Captured the native `HistorianAccess.GetRuntimeParameter(List<string>, out List<object>)`
WCF traffic with `scripts/Capture-RuntimeParam.ps1` (instrument-wcf-{write,read}message).
Findings:
- The WCF op is **`aa/Stat/GETRP`** — `bool GETRP(string handle, byte[] pRequestBuff,
out byte[] pResponseBuff, out byte[] errorBuffer)`, i.e. the **same string-handle +
request/response-buffer shape as GETHI**, *not* the simple `GetSystemParameter(uint, string)`
shape the roadmap originally assumed.
- The `string handle` is the **Open2 storage-session GUID** (the value
`ParseOpenConnectionResponse` reads from `outBuff[5..21]`), sent **UPPERCASE, dash-separated,
no braces** (`ToString("D").ToUpperInvariant()`).
- Unlike GETHI (which the earlier probe found blocked), **GETRP succeeds from the pure-managed
client** with that handle: `GetRuntimeParameter("HistorianVersion")``20,0,000,000`.
- `pRequestBuff` = `54 67 01 00` (sig+version) + uint nameCount + per name(uint charCount +
UTF-16LE). `pResponseBuff` = version(1) + uint resultCount + CRetVariant(`0x43` VT_BSTR +
uint16 payloadLen + uint16 charCount + UTF-16LE).
Shipped as `HistorianClient.GetRuntimeParameterAsync(name)`. See
`HistorianRuntimeParameterProtocol`, golden `WcfRuntimeParameterProtocolTests`, and the
handle-format lead in `wcf-string-handle-wall.md` §Update (retry GETHI/ExeC uppercased).
## R1.3 timezone + R1.4 EventStorageMode — re-confirmed bounded out (2026-06-21)
Both were already classified 2023R2/gRPC-only; re-verified from two *fresh* angles that corroborate it
more strongly than the original op-level probes:
- **Runtime DB schema** (`Runtime.dbo`, the server's own source of truth): the `SystemParameter` table
has **no** timezone parameter and **no `EventStorageMode`** (only `EventStorageDuration` /
`EventStorageLogPath`). The server timezone exists only as **per-block storage artifacts**
(`HistoryBlock.TimeZoneOffset` = e.g. 240 min, `wwTimeZone` = e.g. "Eastern Daylight Time") and a
`TimeZone` reference/lookup table; `StorageShard.TimeZoneId` is NULL. So the timezone is a
DST-specific, SQL-only, OS-derived value, not a clean server-config field exposed by any op.
- **Parameter-op probe** (`StringHandleProbeDiagnosticTests.TimezoneAndStorageMode_ParameterProbe`):
`GetSystemParameter` and `GetRuntimeParameter` (GETRP) were asked for every timezone candidate
(`TimeZone`/`ServerTimeZone`/`SystemTimeZone`/`TimeZoneName`/`SystemTimeZoneName`/`TimeStampRule`/
`ServerTime`) and every storage-mode candidate (`EventStorageMode`/`StorageMode`/`EventStorage`/
`EventStorageDuration`). **All returned null (GetSystemParameter) or threw
`ProtocolEvidenceMissingException` (GETRP — non-string/empty response)**; only the `HistorianVersion`
control returned a value (`20,0,000,000`). Note: `TimeStampRule`/`EventStorageDuration` *do* exist in
the `SystemParameter` table yet `GetSystemParameterAsync` returns null for them — the shipped op only
surfaces a whitelisted subset (a possible future widening, unrelated to R1.3/R1.4).
Conclusion: **R1.3 `GetServerTimeZoneAsync` and R1.4 `GetHistorianInfoAsync` (EventStorageMode) are not
deliverable as server ops on 2020.** The only 2020 route to the timezone is a SQL read of
`HistoryBlock`/`TimeZone` via `ExecuteSqlCommand` (R1.1) — a DST-specific value over a different
mechanism than the roadmap's `Status.GetSystemTimeZoneName`. `EventStorageMode` has no 2020
representation at all (it is a 2023 R2 event-storage-architecture field). Deliver both only against a
live 2023 R2 gRPC server.
## Resolution against the live 2023 R2 gRPC server (2026-06-21) — the two diverged
Both ops were taken to the real 2023 R2 box (History iface 12) over the gRPC
StatusService:
- **R1.3 `GetServerTimeZoneAsync` — SHIPPED.** `StatusService.GetSystemTimeZoneName(uiHandle)`
returns the real Windows zone name **"Eastern Daylight Time"** (the 2020 stub returned empty).
`HistorianClient.GetServerTimeZoneAsync` routes over `RemoteGrpc`; the non-gRPC transports throw
`ProtocolEvidenceMissingException` (fail-closed, no empty-string lie). Golden message-shape +
non-gRPC guardrail unit tests + gated live test.
- **R1.4 `GetHistorianInfoAsync` (`EventStorageMode`) — bounded out on gRPC too.** Over gRPC,
`GetHistorianInfo` is the **same named-value query** as 2020 WCF (only `HistorianVersion`
resolves); `EventStorageMode` + 7 variants fail on both `GetHistorianInfo` and
`GetSystemParameter`. The 518-byte struct is C++-HCAL-internal (native vtable+648), not on the
wire. Not shipped on any transport. See `wcf-historian-info.md`.
@@ -0,0 +1,138 @@
# The 2020 WCF string-handle wall (2026-06-20)
> ## ✅✅ RESOLVED (2026-06-20): the "wall" was a handle-FORMAT bug, not a registration wall.
>
> The string-handle ops are reachable from the pure-managed client after all. The Open2
> storage-session GUID must be passed as the `string handle` **UPPERCASE, dash-separated,
> no braces** — `storageSessionId.ToString("D").ToUpperInvariant()`. The earlier probes that
> "proved" the wall passed the GUID in .NET's default **lowercase** `ToString("D")`, which the
> server's session table does not match. Live-verified end-to-end against the local 2020 server:
> - **GETRP** (R1.2) → returns the runtime `HistorianVersion` (shipped).
> - **GETHI** (R1.4) → `returned=True`, returns the version buffer (`0C000000` + UTF-16 "20,0,000,000").
> - **ExeC** (R1.1) → `returned=True`, `Retr.GetV` prime + `ExeC("SELECT 1 AS ProbeValue", option=0)`
> yields `queryHandle`, then `GetR(handle, queryHandle, sequence=0)` returns a 1232-byte result =
> a **BinaryFormatter-serialized .NET DataTable** (stream header `…System.Data, Version=4.0.0.0…`).
>
> Probes: gated `StringHandleProbeDiagnosticTests` (GETHI + ExeC). Captures:
> `scripts/Capture-RuntimeParam.ps1`, `scripts/Capture-ExecSql.ps1`. The handle for ExeC/GetR is the
> **same** Open2 storage-session GUID (confirmed = `outBuff[5..21]`). The original analysis below is
> retained for history; treat its "blocked" conclusions as **superseded** — the only missing piece
> was the uppercase format.
>
> **Update 2026-06-20 — R1.5 `GetTepByNm` shipped; QTB nuance.** `GetTagExtendedPropertiesFromName`
> (`GetTepByNm`) is now **shipped + live-verified** with the uppercase handle
> (`GetTagExtendedPropertiesAsync`; see `wcf-tag-extended-properties.md`). It confirms the
> string-handle Retrieval family is reachable (and `GetTgByNm`/GetTagInfosFromName was observed
> succeeding alongside it). **But not every string-handle op is just a format fix:** `QTB`
> (`StartTagQuery`) was captured being sent with a correctly-**uppercase** handle and still failed
> with `error 1` *server-side* (`CMdServer::StartTagQuery::StartActiveTagnamesQuery` over
> `\\.\pipe\aahMetadataServer\console`). So QTB/QTG (the active-tagnames query family) are blocked by
> the metadata server, not the handle format — distinct from the handle-format wall. **R1.6
> (localized properties) has no distinct op** and collapses into R1.5.
---
Live-probing the local **Historian 2020** (WCF, port 32568) for HCAL roadmap M1
surfaced a clean structural boundary on what the pure-managed client can call. It
explains why R1.1/R1.4/R1.5 all fail and identifies the single RE target that
unblocks the rest of the M1 read surface.
> ⚠️ **Superseded — see the RESOLVED banner above.** The boundary below is real *only* when the
> handle is sent lowercase. With the uppercased storage GUID the string-handle ops succeed.
## The dichotomy
Retrieval/Status/History ops split by the **type of their first (handle) parameter**:
| Handle type | Examples | Status on 2020 WCF |
|---|---|---|
| **`uint` client handle** (Open2 output) | `StartQuery2`, `GetNextQueryResultBuffer2`, `IsOriginalAllowed`, `GetTagInfosFromName`/`GetTagInfoFromName` (GetTgByNm), `GetSystemParameter`, `StartEventQuery`, `GetNextEventQueryResultBuffer`, `RegisterTags2`, `EnsureTags2`, `UpdateClientStatus3` | ✅ **work** — the proven read/browse/metadata/status-param/event/write surface |
| **`string` GUID handle** | `ExecuteSqlCommand` (ExeC), `StartTagQuery` (QTB), `QueryTag` (QTG), `GetHistorianInfo` (GETHI), `GetTagExtendedPropertiesFromName` (GetTepByNm), `GetTagInfosFromName2` (GetTgByNm2), `GetTagidsByTagnameAndSource` | ⛔ **blocked** — native error type 4, code **51 (InvalidParameter)** or **1 (Failure)** |
## Evidence (this probe + prior notes)
- **ExeC** → type 4 / code 51 for every handle variant (storageGuid, contextGuid).
Matches `implementation-status.md` ~982 / ~1404 ("StartTagQuery depends on earlier
native session/filter registration … do not wire through guessed calls").
- **GETHI** (`HistorianVersion` param query — the *exact* native request shape from
`BuildGetHistorianInfoRequest`, with `Stat.GetV ×2` priming) → type 4 / code **1**
for all five handle formats tried: storage-session GUID, context GUID, uint as
decimal, uint as `X8` hex, uint as `0x`-hex. In the only place GETHI is used (the
event-priming chain) its result is wrapped in `TryRun` and **discarded**, so there
was never evidence it actually returns data from the managed client.
- **GetTepByNm / QTB / QTG / GetTgByNm2** all take a `string handle` → same family.
## Why
The string-handle ops are keyed off a **native-side session/filter registration**
that the C++ client performs but the managed replay does not reproduce. The uint
client handle is the Open2 session token the server already trusts; the string GUID
handle indexes a *different* per-service registration table that stays empty unless
the native priming is replicated faithfully. `Stat.GetV ×2` alone is insufficient.
## Consequence for the roadmap
Every remaining **M1 read** item is a string-handle op:
- R1.1 `ExecuteSqlCommandAsync` (ExeC) — blocked
- R1.4 `GetHistorianInfoAsync` (GETHI) — blocked
- R1.5 extended-property read (GetTepByNm) — blocked (string handle, confirmed)
- R1.6 localized-property read — same family
So **M1 read-surface completion on 2020 WCF is gated entirely behind one RE target:
the native session/filter registration for string-handle ops.** Reverse-engineer it
once and the whole family unlocks. Until then, the alternatives are:
1. **RE the registration** — instrument the native `CRetrievalConnectionWCF` /
`CStatusConnectionWCF` priming between Open2 and the first successful string-handle
call (capture-tier; the highest-leverage single RE task for M1).
2. **2023 R2 gRPC server** — these ops are first-class on the gRPC front door, where
the handle/envelope differs and the registration wall may not apply.
Do **not** ship any string-handle op via guessed calls (project discipline:
"leave them throwing until evidence supports an implementation").
## ⚠️ Update (2026-06-20): GETRP punches through — the wall is not absolute
Roadmap **R1.2 `GetRuntimeParameterAsync`** turned out to be a **`string`-handle op**
(`aa/Stat/GETRP(string handle, byte[] pRequestBuff) → (bool, byte[] pResponseBuff,
byte[] errorBuffer)`) — the **same shape as GETHI**, and in the same native session it
uses the **same handle GUID** as GETHI (confirmed: the GUID equals the Open2 `outBuff`
storage-session id at `[5..21]`, the value the managed `ParseOpenConnectionResponse`
already extracts as `StorageSessionId`).
Yet GETRP **works from the pure-managed client** — live-verified, returns the runtime
`HistorianVersion` value `20,0,000,000`. The only material difference from the failed
GETHI probe is the **handle string format**: the native client sends the GUID
**UPPERCASE, dash-separated, no braces** (format example
`XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX`, all hex upper), i.e.
`storageSessionId.ToString("D").ToUpperInvariant()`. `.NET Guid.ToString("D")` is
lowercase, so a probe that passed the GUID without upcasing would not byte-match what
the server's session table is keyed on.
**Implication — CONFIRMED, the wall is largely a handle-format bug.** The follow-up was done:
GETHI and **ExeC both return data with the uppercased storage-session GUID**.
- **R1.1 `ExecuteSqlCommandAsync` (ExeC + GetR) — SHIPPED + live-verified (2026-06-20).**
`ExecuteSqlCommandAsync(sql)``HistorianSqlResult`. `Retr.GetV` prime → `ExeC(handle,
sql, option=0, ref queryHandle)` → `GetR` loop. Note: **`GetR` returns false even on
success** (the byte stream is in `pResultBuff` regardless; false = final page). `pResultBuff`
is an **NRBF `DataTable`** (`SerializationFormat.Xml`: `XmlSchema` + `XmlDiffGram`), decoded
read-only with `System.Formats.Nrbf` + `XDocument` (BinaryFormatter is gone from .NET 10).
Shipped: `HistorianSqlResultProtocol`, `HistorianWcfSqlClient`, golden `WcfSqlResultProtocolTests`,
gated live tests. See `docs/reverse-engineering/wcf-exec-sql.md`.
- **GETHI (R1.4)** returns data with the uppercase handle, **but only the named `HistorianVersion`
value** — over 2020 WCF GETHI is a named-value query (the only working key), *not* a full-struct
read. `EventStorageMode` (the 518-byte-struct `@514` field) is **not on the 2020 WCF wire**; it is
the 2023R2 HCAL-native/gRPC model. So R1.4 is **bounded out on WCF / gRPC-2023R2-only** and the
public API is intentionally not shipped. Full analysis: `docs/reverse-engineering/wcf-historian-info.md`.
So the "wall" collapses to the handle **format** for the Retrieval/Status string-handle ops.
**Exception — QTB/QTG:** `StartTagQuery` does *not* punch through; captured with a correctly
uppercase handle it still fails `error 1` **server-side** (`CMdServer::StartActiveTagnamesQuery`
over `\\.\pipe\aahMetadataServer\console`) — a metadata-server blocker, independent of handle
format. Name-based ops route around it.
See `HistorianRuntimeParameterProtocol`, `IStatusServiceContract2.GetRuntimeParameter`,
golden `WcfRuntimeParameterProtocolTests`, and capture tooling
`scripts/Capture-RuntimeParam.ps1` + `scripts/decode-runtime-param-capture.py`.
@@ -0,0 +1,116 @@
# Tag extended properties over 2020 WCF — GetTepByNm (HCAL R1.5)
**Status: ✅ DONE + live-verified (2026-06-20).** `HistorianClient.GetTagExtendedPropertiesAsync(tag)`
reads a tag's extended (user-defined) properties over the 2020 WCF op
`aa/Retr/GetTagExtendedPropertiesFromName` (`GetTepByNm`). Live-verified end-to-end from the
pure-managed .NET 10 client against the local 2020 Historian.
## The op
`GetTepByNm` is on the Retrieval service (`IRetrievalServiceContract4`):
```
bool GetTagExtendedPropertiesFromName(
string handle, // Open2 storage-session GUID, UPPERCASE dash-no-braces
byte[] tagNames, // [MessageParameter pRequestBuff-style]
ref uint sequence, // paging cursor (0 on first call)
out byte[] tagExtendedProperties, // result buffer
out byte[] errorBuffer)
```
It is a **string-handle** op — reachable from the managed client because the handle is the Open2
storage-session GUID formatted `storageSessionId.ToString("D").ToUpperInvariant()` (the same handle
format that unlocked GETRP/GETHI/ExeC; see `wcf-string-handle-wall.md`). The Retrieval service
version handshake (`Retr.GetV`) is primed first, as the native client does.
## Why the name-based path (not the TagQuery path)
There are two managed entry points:
- **Index-based** `TagQuery.GetTagExtendedPropertyInfo(start, count, …)` — requires a prior
`StartTagQuery` (`QTB`). On this 2020 box **QTB fails server-side** (`error 1` from
`CMdServer::StartTagQuery::StartActiveTagnamesQuery` over `\\.\pipe\aahMetadataServer\console`),
so this path is dead here regardless of handle format.
- **Name-based** `HistorianAccess.GetTagExtendedPropertiesByName(tagName, fetchFromServer, …)`
issues `GetTepByNm` directly with the tag name in `tagNames`, no QTB needed. Its second arg
forces a **server fetch** when true; when false the C++ client reads a local cache and returns
`error 41 (Requested item not found)` without any WCF round-trip. The SDK reproduces the
name-based path.
## Wire format (captured)
`scripts/Capture-TagExtendedProperties.ps1` (NativeTraceHarness `tag-extended-properties` scenario +
instrument-wcf-{write,read}message) → decode with `scripts/decode-tag-properties-capture.py`.
Golden-pinned in `WcfTagExtendedPropertyProtocolTests`.
### Request — `tagNames` buffer
```
uint32 count
per name: uint32 charCount + UTF-16LE chars
```
(For one tag: `01 00 00 00` + `LL 00 00 00` + UTF-16 name.)
### Response — `tagExtendedProperties` buffer
```
uint32 tagCount
per tag:
byte groupMarker (observed 0x01)
0x09 + uint16 byteLen + ASCII tagName (compact-ASCII string)
uint32 propertyCount
per property:
byte propMarker (observed 0x02 — likely the value type)
0x09 + uint16 byteLen + ASCII propertyName
0x43 + uint16 payloadLen + uint16 charCount + UTF-16LE value (VT_BSTR CRetVariant)
byte trailingMarker (observed 0x01)
```
`payloadLen` counts the `charCount` field (2 bytes) + the UTF-16 value bytes. Only the string value
variant (`0x43`) is evidence-backed; other variant types throw `ProtocolEvidenceMissingException`
(same discipline as GETRP). The single-byte `0x01`/`0x02`/`0x01` markers are pinned as observed
constants from a single capture; their full semantics are not independently disambiguated.
Example (sanitized — real capture used a dev tag/value):
```
tag "Reactor.Temp1" → property "Location" = "Plant/AreaA"
```
### Paging
`GetTepByNm` is sequence-paged like `GetNextQueryResultBuffer`: call with `sequence = 0`, parse the
buffer, then re-call with the returned `sequence`. A small result returns everything on the first
call; the next call returns an empty/`nil` buffer (with a benign `CClientUtil::FillBufferFromVector`
terminator) — that is the stop signal. The SDK loops until the buffer carries no rows.
## R1.6 (localized properties) — no distinct op on 2020
There is **no** `GetTagLocalizedPropertiesFromName` / `GetTlpByNm` op or
`GetTagLocalizedPropertiesByName` method in `current/aahClientManaged.dll` — the only "localized"
surfaces are `ClientApp.GetLocalizedText` and `SMessageTextMap.GetLocalizedMessage` (error-message /
UI-text localization), not tag properties. So R1.6 **collapses into R1.5**: extended properties
(`GetTepByNm`) are the user-defined tag-property read surface on 2020. R1.6 is closed as
"no separate op," not left throwing.
## R1.12 (localized-property write) — no distinct op on 2020 (mirror of R1.6)
Symmetric to R1.6 on the write side: there is **no** `AddTagLocalizedProperties` /
`DeleteTagLocalizedProperties` (or any `*LocalizedPropert*` / `TagLocalized*`) symbol in **any**
`current/*.dll`. A full symbol sweep of the shipped client DLLs surfaces only `GetLocalizedText`,
`GetLocalizedMessage`, and `LocalizedResourcesDir` — all UI/error-message-text localization, not tag
data. (The sweep does find the real write op `AddTagExtendedProperties` and the whole `AddTag*`
family, so the absence of a localized op is a true negative, not a grep miss.) R1.12 is therefore
**closed as "no separate op"** — the same conclusion as R1.6's read side. Extended-property write
(R1.11 `AddTEx`) is the user-defined tag-property write surface on 2020; localized properties are a
2023 R2 / gRPC-only concept. Not left throwing.
## Shipped surface
- `HistorianClient.GetTagExtendedPropertiesAsync(tag)``IReadOnlyList<HistorianTagExtendedProperty>`
(`Name`/`Value` pairs; empty when the tag has none).
- `HistorianTagExtendedPropertyProtocol` (serializer/parser), `HistorianWcfTagExtendedPropertyClient`
(orchestration), golden `WcfTagExtendedPropertyProtocolTests`, gated live
`GetTagExtendedPropertiesAsync_AgainstLocalHistorian_ReturnsProperties` (set `HISTORIAN_TEP_TAG`
to a tag with extended properties).
@@ -0,0 +1,18 @@
{
"op": "get-tag-info",
"capturedUtc": "2026-06-19T18:55:46.5988258Z",
"notes": "RetrievalService.GetTagInfoFromName response (CTagMetadata buffer); identical bytes on 2023 R2 gRPC GetTagInfosFromName.",
"request": null,
"response": {
"length": 98,
"sha256": "cdda36baa869355b52ccb4be2735ccacfa2da69f0cafe62e88b807f1a05089fd",
"hex": "03c3003184228c4058e1874a984b3dbecbe0aa42ee000000091d0058585858585858585858585858585858585858585858585858585858580904004d44415302030102000000d057f49465d8dc010a0000000000000024400000000000002440fe00",
"redactions": [
{
"secret": "tag",
"asciiMatches": 1,
"utf16Matches": 0
}
]
}
}
@@ -0,0 +1,104 @@
<#
.SYNOPSIS
Captures the native AVEVA client's AddTagExtendedProperties / DeleteTagExtendedProperties wire
traffic (HCAL roadmap R1.11) so the AddTEx / DelTep inBuff layout can be decoded, not guessed.
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness's `add-tep` scenario against the live Historian with
an IL-rewritten copy of aahClientManaged.dll whose ClientMessageEncoder.WriteMessage AND
ReadMessage are instrumented. The harness opens a WRITE-enabled connection, creates a sandbox tag
(RetestSdkWrite...), and calls AddTagExtendedProperties(TagExtendedPropertyGroupList, out err) with
one string property (and DeleteTagExtendedPropertiesByName when -Delete).
Decode with scripts/decode-add-tep-capture.py: the WCF.WriteMessage.Body whose op is AddTEx carries
the inBuff (tag name + property name/value); DelTep carries the delete inBuff (tag + property names).
SAFETY: sandbox-guarded the tag MUST start with 'RetestSdkWrite'. Default run leaves the tag +
property in place (unless -Delete); pass -Delete to also capture DelTep and remove the property.
.NOTES
Artifacts are diagnostic and gitignored. Sanitize before copying into docs/.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
[string]$TepTag = "RetestSdkWriteTepTag",
[string]$PropName = "SdkTestProp",
[string]$PropValue = "SdkTestValue",
[switch]$Delete,
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
if (-not $TepTag.StartsWith("RetestSdkWrite")) { throw "-TepTag must start with 'RetestSdkWrite' (sandbox guard)." }
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-add-tep"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "add-tep-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing add-tep ($TepTag : $PropName=$PropValue) ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "add-tep",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--tep-tag", $TepTag,
"--tep-name", $PropName,
"--tep-value", $PropValue,
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
if ($Delete) { $harnessArgs += "--tep-delete" }
$harnessJson = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} catch {
Write-Host " (add-tep raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
Write-Host "Harness output (AddTagExtendedProperties / Rows):" -ForegroundColor Cyan
$harnessJson | Select-Object -Last 24
Write-Host "`nDecode with: python scripts\decode-add-tep-capture.py" -ForegroundColor Cyan
@@ -0,0 +1,111 @@
<#
.SYNOPSIS
Captures the native AVEVA client's DeleteTagExtendedProperties (DelTep) wire traffic (HCAL R1.11
delete half) using the CROSS-SESSION trick so the delete passes the client-side sync gate.
.DESCRIPTION
DeleteTagExtendedPropertiesByName does a CLIENT-SIDE sync check and returns err 229 ("Tag extended
property not synchronized with server") for any property the local cache doesn't see as
server-synchronized so a just-added property can't be deleted in the same session and its DelTep
inBuff never reaches the wire. This script captures it in two SEPARATE harness processes (= two
sessions) against one instrumented aahClientManaged.dll:
Run A: add-tep (create sandbox tag + AddTagExtendedProperties) -> property now server-synced
Run B: add-tep --tep-skip-create --tep-skip-add --tep-delete -> fresh connection: fetch the
property from the server (seeds the local cache as SYNCED), then
DeleteTagExtendedPropertiesByName, which now reaches the wire as DelTep.
The capture file is cleared BETWEEN the two runs so it contains only Run B (the GetTepByNm
read-for-sync + the DelTep delete). Decode with scripts/decode-del-tep-capture.py.
SAFETY: sandbox-guarded the tag MUST start with 'RetestSdkWrite'. The run leaves the sandbox tag
in place (property removed); delete the tag afterwards with the supported aaDeleteTag proc.
.NOTES
Artifacts are diagnostic and gitignored. Sanitize before copying into docs/.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
[string]$TepTag = "RetestSdkWriteDelTepSdk",
[string]$PropName = "SdkDelProp",
[string]$PropValue = "SdkDelValue",
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
if (-not $TepTag.StartsWith("RetestSdkWrite")) { throw "-TepTag must start with 'RetestSdkWrite' (sandbox guard)." }
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-del-tep"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "del-tep-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
function Invoke-Harness([string[]]$extraArgs, [string]$label) {
Write-Host "== $label ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "add-tep",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--tep-tag", $TepTag,
"--tep-name", $PropName,
"--tep-value", $PropValue,
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
) + $extraArgs
$json = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$json = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} finally { $ErrorActionPreference = $prevEap }
$json | Select-Object -Last 20
}
# Run A: create the sandbox tag + add the property (server-synced afterwards).
Invoke-Harness @() "Run A: create + AddTagExtendedProperties ($TepTag : $PropName=$PropValue)"
# Clear the capture so the file contains only Run B (read-for-sync + DelTep).
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
# Run B: FRESH session — fetch (sync the local cache) then DeleteTagExtendedPropertiesByName.
Invoke-Harness @("--tep-skip-create", "--tep-skip-add", "--tep-delete") "Run B: fresh session -> read-for-sync -> DelTep"
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary (Run B only) ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
Write-Host "`nDecode with: python scripts\decode-del-tep-capture.py" -ForegroundColor Cyan
+101
View File
@@ -0,0 +1,101 @@
<#
.SYNOPSIS
Captures the native client's StartEventQuery request bytes WITH and WITHOUT an event filter
(HCAL roadmap R1.7) so the filter-predicate encoding can be decoded against the empty-filter
baseline instead of guessed.
.DESCRIPTION
Drives the NativeTraceHarness `event` scenario against the live Historian under an
IL-rewritten aahClientManaged.dll whose ClientMessageEncoder.WriteMessage is instrumented to
log every outgoing MDAS body. Runs twice:
- baseline : no filter (the known empty-filter StartEventQuery)
- filtered : EventQuery.AddEventFilter("Area", Equal, "RetestFilterArea") before StartQuery
Diff the two StartEventQuery request buffers (scripts/decode-event-filter-capture.py) to read
off the exact filter-block bytes (property name / comparison op / value) the native client
emits, then implement the managed predicate against that.
.NOTES
Artifacts are diagnostic and gitignored. Sanitize before copying into docs/. Never commit raw
capture NDJSON, credentials, hostnames, or customer tag names.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
[int]$LookbackMinutes = 43200,
# Property:Op:Value (Op = a HistorianComparisionType name, e.g. Equal/Contains/GreaterThan).
[string]$Filter = "Area:Equal:RetestFilterArea",
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-event-filter"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
$matrix = @(
@{ Name = "baseline"; Args = @() },
@{ Name = "filtered"; Args = @("--event-filter", $Filter) }
)
foreach ($cfg in $matrix) {
$capturePath = Join-Path $captureDir "event-filter-capture-$($cfg.Name)-latest.ndjson"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing: $($cfg.Name) ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "event",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--lookback-minutes", "$LookbackMinutes",
"--max-rows", "1",
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
) + $cfg.Args
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
& dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1 | Out-Null
} catch {
Write-Host " ($($cfg.Name) raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host " -> $recCount records -> $capturePath"
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
Write-Host "`nDecode with: python scripts\decode-event-filter-capture.py" -ForegroundColor Cyan
+110
View File
@@ -0,0 +1,110 @@
<#
.SYNOPSIS
Captures the native AVEVA client's event-SEND wire traffic (HCAL roadmap R2.1) to
determine whether AddStreamedValue(HistorianEvent) rides the WCF MDAS path (capturable
+ implementable as a pure-managed-WCF SDK op) or the storage-engine shared-memory pipe
(like revision writes which would block M2 as a WCF SDK).
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness's `event-send` scenario against the live
Historian with an IL-rewritten copy of aahClientManaged.dll whose
ClientMessageEncoder.WriteMessage AND ReadMessage are instrumented to log every MDAS
body (the same pipeline that produced every other proven request/response shape). The
harness opens an Event connection (ReadOnly=false), builds a clearly-marked test
HistorianEvent, calls AddStreamedValue(HistorianEvent), then CloseStorageConnection to
flush the queued event onto the wire.
Decode with scripts/decode-event-send-capture.py: if a StartStorage/AddStreamValues/
EnqueueEventDataPacket body appears on WCF.WriteMessage.Body, M2 is viable over WCF and
the body carries the PackToVtq event value blob to decode (R2.2). If NOTHING event-shaped
appears on the WCF path even though the native AddStreamedValue returned success, the
delivery used the storage-engine pipe and M2 is architecturally blocked over WCF the
same conclusion as the revision-write path (docs/plans/revision-write-path.md).
.NOTES
Writes a real (clearly-marked) test event into the historian's event history. Artifacts
are diagnostic and gitignored. Sanitize before copying anything into docs/ never commit
raw capture NDJSON, credentials, hostnames, or customer tag names.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
[string]$EventType = "User.Write",
[int]$FlushSeconds = 6,
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-event-send"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "event-send-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
# Chain via a distinct intermediate file (reading+writing the same path drops the second
# hook on the mixed-mode native image). Final dll carries both hooks with distinct Phase
# strings: WCF.WriteMessage.Body and WCF.ReadMessage.Body.
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing event-send ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "event-send",
"--event-send-confirm",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--event-type", $EventType,
"--event-send-flush-seconds", "$FlushSeconds",
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
$harnessJson = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} catch {
Write-Host " (event-send raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
Write-Host "Harness output (look for AddStreamedEvent Success / ErrorCode):" -ForegroundColor Cyan
$harnessJson | Select-Object -Last 60
Write-Host "`nDecode with: python scripts\decode-event-send-capture.py" -ForegroundColor Cyan
+91
View File
@@ -0,0 +1,91 @@
<#
.SYNOPSIS
Captures the native AVEVA client's ExecuteSqlCommand wire traffic (HCAL roadmap R1.1) so the
Retr.ExeC + Retr.GetR string-handle SQL surface (op names, handle format, command/option
encoding, Retr priming, GetR result byte stream) can be decoded instead of guessed.
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness `exec-sql` scenario against the live Historian
with an IL-rewritten copy of aahClientManaged.dll whose ClientMessageEncoder.WriteMessage AND
ReadMessage are instrumented to log every MDAS body. Read-only benign query.
.NOTES
Artifacts are diagnostic and gitignored. Sanitize before copying into docs/ -- never commit raw
capture NDJSON, credentials, hostnames, or customer tag names.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
[string]$Sql = "SELECT 1 AS ProbeValue",
[string]$SqlOption = "ExecuteRecord",
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-exec-sql"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "exec-sql-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing exec-sql ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "exec-sql",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--sql", $Sql,
"--sql-option", $SqlOption,
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
$harnessJson = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} catch {
Write-Host " (exec-sql raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
$harnessJson | Select-Object -Last 6
+102
View File
@@ -0,0 +1,102 @@
<#
.SYNOPSIS
Captures the native AVEVA client's GetHistorianInfo wire traffic (HCAL roadmap R1.4)
so the WCF GETHI request that returns the FULL HISTORIAN_INFO struct can be decoded
instead of guessed.
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness's `historian-info` scenario against the live
Historian with an IL-rewritten copy of aahClientManaged.dll whose
ClientMessageEncoder.WriteMessage AND ReadMessage are instrumented to log every MDAS body
(the same pipeline that produced every other proven request/response shape). The harness
opens a normal authenticated process connection and calls
HistorianAccess.GetHistorianInfo(out HistorianInfo, out err).
Decode with scripts/decode-historian-info-capture.py: locate the WCF.WriteMessage.Body
whose op is GETHI -> that is the GetHistorianInfo request; read off the leading string
handle and the pRequestBuff layout (distinct from the named-value "HistorianVersion"
request). The paired WCF.ReadMessage.Body is the pResponseBuff = the 518-byte
HISTORIAN_INFO struct (version string @0 UTF-16 null-terminated, EventStorageMode int32 @514).
.NOTES
Read-only status call; no data is written. Artifacts are diagnostic and gitignored.
Sanitize before copying anything into docs/ -- never commit raw capture NDJSON,
credentials, hostnames, or customer tag names.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-historian-info"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "historian-info-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
# Chain via a distinct intermediate file (reading+writing the same path drops the second
# hook on the mixed-mode native image). Final dll carries both hooks with distinct Phase
# strings: WCF.WriteMessage.Body and WCF.ReadMessage.Body.
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing historian-info ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "historian-info",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
$harnessJson = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} catch {
Write-Host " (historian-info raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
Write-Host "Harness output (GetHistorianInfoReturned / HistorianInfo):" -ForegroundColor Cyan
$harnessJson | Select-Object -Last 24
Write-Host "`nDecode with: python scripts\decode-historian-info-capture.py" -ForegroundColor Cyan
+133
View File
@@ -0,0 +1,133 @@
<#
.SYNOPSIS
Captures the native AVEVA client's RenameTags wire traffic (HCAL roadmap R1.10) so the
StJb (StartJob) rename jobBuffer + GtJb (GetJobStatus) response can be decoded instead of guessed.
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness's `rename` scenario against the live Historian
with an IL-rewritten copy of aahClientManaged.dll whose ClientMessageEncoder.WriteMessage AND
ReadMessage are instrumented to log every MDAS body. The harness opens a WRITE-enabled
connection, creates a sandbox source tag (RetestSdkWrite...), calls
HistorianAccess.RenameTags([(from,to)], ref status, out err), and polls GetTagRenameStatus.
Rename maps to the generic job framework: StJb(handle, jobBuffer) -> jobId, then
GtJb(handle, jobId) -> jobStatus. Decode with scripts/decode-rename-capture.py: find the
WCF.WriteMessage.Body whose op is StJb -> its jobBuffer carries the (old,new) name pairs; the
paired ReadMessage carries the jobId; the GtJb request/response carry the status.
SAFETY: sandbox-guarded both names MUST start with 'RetestSdkWrite'. The default run renames
RetestSdkWriteRenameSrc -> RetestSdkWriteRenameDst and (unless -SkipCleanup) deletes the
destination tag afterward via a second harness pass.
.NOTES
Artifacts are diagnostic and gitignored. Sanitize before copying anything into docs/ --
never commit raw capture NDJSON, credentials, hostnames, or customer tag names.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
[string]$RenameFrom = "RetestSdkWriteRenameSrc",
[string]$RenameTo = "RetestSdkWriteRenameDst",
[switch]$SkipCleanup,
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
if (-not $RenameFrom.StartsWith("RetestSdkWrite") -or -not $RenameTo.StartsWith("RetestSdkWrite")) {
throw "Both -RenameFrom and -RenameTo must start with 'RetestSdkWrite' (sandbox guard)."
}
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-rename"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "rename-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing rename ($RenameFrom -> $RenameTo) ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "rename",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--rename-from", $RenameFrom,
"--rename-to", $RenameTo,
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
$harnessJson = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} catch {
Write-Host " (rename raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
Write-Host "Harness output (RenameTagsReturned / Rows):" -ForegroundColor Cyan
$harnessJson | Select-Object -Last 30
# Best-effort cleanup: delete the destination sandbox tag so reruns start clean.
if (-not $SkipCleanup) {
Write-Host "`n== Cleanup: deleting $RenameTo ==" -ForegroundColor Cyan
$cleanupArgs = @(
"--scenario", "write",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--write-sandbox-tag", $RenameTo,
"--write-skip-add-tag",
"--write-skip-add-value",
"--write-delete-after",
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
try {
$ErrorActionPreference = "Continue"
& dotnet run --no-build -c $Configuration --project $harnessProj -- @cleanupArgs 2>&1 | Select-Object -Last 4
} catch {
Write-Host " (cleanup raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = "Stop"
}
}
Write-Host "`nDecode with: python scripts\decode-rename-capture.py" -ForegroundColor Cyan
+105
View File
@@ -0,0 +1,105 @@
<#
.SYNOPSIS
Captures the native AVEVA client's GetRuntimeParameter wire traffic (HCAL roadmap R1.2)
so the WCF op name, handle type (uint vs the string-handle wall), and the
btRequest/btResponse buffer format can be decoded instead of guessed.
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness's `runtime-param` scenario against the live
Historian with an IL-rewritten copy of aahClientManaged.dll whose
ClientMessageEncoder.WriteMessage AND ReadMessage are instrumented to log every MDAS body
(the same pipeline that produced every other proven request/response shape). The harness
opens a normal authenticated process connection and calls
HistorianAccess.GetRuntimeParameter(List<string> names, out List<object> results, out err).
Decode with scripts/decode-runtime-param-capture.py: locate the WCF.WriteMessage.Body
whose op carries the parameter name(s) -> that is the GetRuntimeParameter request; read
off the SOAP action / op name, the leading handle param, and the btRequest layout. The
paired WCF.ReadMessage.Body is the btResponse (the CRetVariant value list).
.NOTES
Read-only status call; no data is written. Artifacts are diagnostic and gitignored.
Sanitize before copying anything into docs/ -- never commit raw capture NDJSON,
credentials, hostnames, or customer tag names.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
# Semicolon-separated runtime parameter names. HistorianVersion is a known-good name
# (returns the server version string) so the response decode has a real value.
[string]$Names = "HistorianVersion",
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-runtime-param"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "runtime-param-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
# Chain via a distinct intermediate file (reading+writing the same path drops the second
# hook on the mixed-mode native image). Final dll carries both hooks with distinct Phase
# strings: WCF.WriteMessage.Body and WCF.ReadMessage.Body.
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing runtime-param ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "runtime-param",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--runtime-param-names", $Names,
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
$harnessJson = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} catch {
Write-Host " (runtime-param raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
Write-Host "Harness output (GetRuntimeParameterReturned / Results):" -ForegroundColor Cyan
$harnessJson | Select-Object -Last 20
Write-Host "`nDecode with: python scripts\decode-runtime-param-capture.py" -ForegroundColor Cyan
+150
View File
@@ -0,0 +1,150 @@
<#
.SYNOPSIS
Captures the native AVEVA client's StartQuery2 request bytes for analog/state
summary queries (HCAL roadmap R1.8/R1.9) so the managed SDK's summary request
shape can be decoded against ground truth instead of guessed.
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness against the live Historian with an
IL-rewritten copy of aahClientManaged.dll whose ClientMessageEncoder.WriteMessage
is instrumented to log every outgoing MDAS body (the same pipeline that produced
every other proven request shape). For each candidate HistoryQueryArgs config it
writes a per-config NDJSON capture under
artifacts/reverse-engineering/instrumented-wcf-writemessage-summary/ (gitignored).
The default matrix is:
- baseline-full : RetrievalMode=Full (the known-good non-summary request)
- analog-avg : RetrievalMode=Cyclic + ValueSelector=Average + Resolution
- analog-min : RetrievalMode=Cyclic + ValueSelector=Minimum + Resolution
- analog-agg-avg : RetrievalMode=Cyclic + AggregationType=Average + Resolution
- state-summary : RetrievalMode=Cyclic + MaxStates>0 + Resolution
Diff any candidate against baseline-full (scripts/decode-summary-capture.py) to read
off the exact QueryType / SummaryType / AutoSummaryParameters bytes the native client
sets for a summary, then implement the managed request against that.
.NOTES
Artifacts are diagnostic. Sanitize before copying anything into docs/ never commit
raw capture NDJSON, credentials, hostnames, or customer tag names.
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
# SysTimeSec is the local data-bearing system tag (OtOpcUaParityTest_001.Counter is stale/empty).
[string]$TagName = "SysTimeSec",
[int]$LookbackMinutes = 240,
[int]$MaxRows = 4,
# 1-hour summary cycle in 100ns ticks (1h = 36,000,000,000 ticks).
[uint64]$ResolutionTicks = 36000000000,
[string]$Configuration = "Debug",
# Restrict the run to a single named config from the matrix (default: run all).
[string]$OnlyConfig = "",
# Also instrument ReadMessage so each capture includes the incoming WCF response bodies
# (the GetNextQueryResultBuffer2 pResultBuff summary rows). Decoded by decode-summary-response.py.
[switch]$WithResponse
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-writemessage-summary"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage$(if ($WithResponse) { ' + ReadMessage' }) ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
if ($WithResponse) {
# Chain via a distinct intermediate file (reading+writing the same path drops the second
# hook on the mixed-mode native image). Final dll carries both hooks with distinct Phase
# strings: WCF.WriteMessage.Body and WCF.ReadMessage.Body.
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
} else {
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $instrDll | Out-Null
}
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
# Mirror current/ into current-copy, then overwrite the managed dll with the instrumented
# build and drop the strong-named logger assembly alongside it so the injected call binds.
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
# Candidate matrix: name + harness arg list. Summary configs all use Cyclic + a resolution;
# the differentiator is which summary knob is set.
$matrix = @(
@{ Name = "baseline-full"; Args = @("--retrieval-mode", "Full") },
@{ Name = "analog-avg"; Args = @("--retrieval-mode", "Cyclic", "--value-selector", "Average", "--resolution-ticks", "$ResolutionTicks") },
@{ Name = "analog-min"; Args = @("--retrieval-mode", "Cyclic", "--value-selector", "Minimum", "--resolution-ticks", "$ResolutionTicks") },
@{ Name = "analog-max"; Args = @("--retrieval-mode", "Cyclic", "--value-selector", "Maximum", "--resolution-ticks", "$ResolutionTicks") },
@{ Name = "analog-integral"; Args = @("--retrieval-mode", "Cyclic", "--value-selector", "Integral", "--resolution-ticks", "$ResolutionTicks") },
@{ Name = "mode-integral"; Args = @("--retrieval-mode", "Integral", "--resolution-ticks", "$ResolutionTicks") },
@{ Name = "mode-twavg"; Args = @("--retrieval-mode", "TimeWeightedAverage", "--resolution-ticks", "$ResolutionTicks") },
@{ Name = "analog-agg-avg"; Args = @("--retrieval-mode", "Cyclic", "--aggregation-type", "Average", "--resolution-ticks", "$ResolutionTicks") },
@{ Name = "state-summary"; Args = @("--retrieval-mode", "Cyclic", "--max-states", "10", "--resolution-ticks", "$ResolutionTicks") }
)
if ($OnlyConfig) { $matrix = $matrix | Where-Object { $_.Name -eq $OnlyConfig } }
if (-not $matrix) { throw "No matrix entry named '$OnlyConfig'." }
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
$summary = @()
foreach ($cfg in $matrix) {
$name = $cfg.Name
$capturePath = Join-Path $captureDir "summary-capture-$name-latest.ndjson"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing: $name ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "history",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--tag", $TagName,
"--lookback-minutes", "$LookbackMinutes",
"--max-rows", "$MaxRows",
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
) + $cfg.Args
# Don't let a single config that errors (e.g. state summary on an analog tag) abort the
# whole matrix, and don't treat dotnet's stderr noise as a terminating error.
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
& dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1 | Out-Null
} catch {
Write-Host " (config '$name' raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host " -> $recCount records -> $capturePath"
$summary += [pscustomobject]@{ Config = $name; Records = $recCount; Capture = $capturePath }
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
$summary | Format-Table -AutoSize
Write-Host "Decode with: python scripts\decode-summary-capture.py" -ForegroundColor Cyan
+104
View File
@@ -0,0 +1,104 @@
<#
.SYNOPSIS
Captures the native AVEVA client's GetTagExtendedPropertiesFromName (GetTepByNm) wire traffic
(HCAL roadmap R1.5) so the WCF op name, the string-handle format, the tagNames request buffer,
and the extended-property response buffer can be decoded instead of guessed.
.DESCRIPTION
Drives the .NET-Framework NativeTraceHarness's tag scenario with --retrieve-extended-properties,
which flips TagQueryArgs.RetrieveTagExtendedPropertyInfo and then calls
TagQuery.GetTagExtendedPropertyInfo(start, count, out TagExtendedPropertyGroupList, out err)
the managed method that issues the GetTepByNm op. An IL-rewritten copy of aahClientManaged.dll
logs every MDAS body (ClientMessageEncoder.WriteMessage + ReadMessage), the same pipeline that
produced every other proven request/response shape.
Decode with scripts/decode-tag-properties-capture.py: locate the WCF.WriteMessage.Body whose op
is aa/Retr/GetTepByNm -> that is the request (string handle + tagNames buffer + sequence). The
paired WCF.ReadMessage.Body is the extended-property response buffer.
.NOTES
Read-only metadata call; no data is written. Artifacts are diagnostic and gitignored.
Sanitize before copying anything into docs/ -- never commit raw capture NDJSON, credentials,
hostnames, or customer tag names. SysTimeSec is a built-in system tag (safe to name).
#>
[CmdletBinding()]
param(
[string]$ServerName = "localhost",
[int]$TcpPort = 32568,
# A tag (or wildcard) to query extended properties for. SysTimeSec is a built-in system tag
# present on every Historian; override with a real tag that carries extended properties for a
# richer response decode.
[string]$Tag = "SysTimeSec",
[string]$Configuration = "Debug"
)
$ErrorActionPreference = "Stop"
$repoRoot = Split-Path -Parent $PSScriptRoot
Set-Location $repoRoot
$reProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseEngineering\AVEVA.Historian.ReverseEngineering.csproj"
$harnessProj = Join-Path $repoRoot "tools\AVEVA.Historian.NativeTraceHarness\AVEVA.Historian.NativeTraceHarness.csproj"
$instrProj = Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\AVEVA.Historian.ReverseInstrumentation.csproj"
$captureDir = Join-Path $repoRoot "artifacts\reverse-engineering\instrumented-wcf-tag-extended-properties"
$currentCopy = Join-Path $captureDir "current-copy"
$instrDll = Join-Path $captureDir "aahClientManaged.dll"
$capturePath = Join-Path $captureDir "tag-extended-properties-capture-latest.ndjson"
Write-Host "== Building tooling ($Configuration) ==" -ForegroundColor Cyan
dotnet build $reProj -c $Configuration --nologo -v q | Out-Null
dotnet build $instrProj -c $Configuration --nologo -v q | Out-Null
dotnet build $harnessProj -c $Configuration --nologo -v q | Out-Null
$instrSourceDll = Get-ChildItem -Recurse (Join-Path $repoRoot "tools\AVEVA.Historian.ReverseInstrumentation\bin\$Configuration") `
-Filter "AVEVA.Historian.ReverseInstrumentation.dll" | Select-Object -First 1 -ExpandProperty FullName
if (-not $instrSourceDll) { throw "ReverseInstrumentation.dll not found under bin\$Configuration." }
Write-Host "== Instrumenting WriteMessage + ReadMessage ==" -ForegroundColor Cyan
New-Item -ItemType Directory -Force -Path $captureDir | Out-Null
# Chain via a distinct intermediate file (reading+writing the same path drops the second hook on
# the mixed-mode native image). Final dll carries both hooks: WCF.WriteMessage.Body + WCF.ReadMessage.Body.
$writeOnly = Join-Path $captureDir "aahClientManaged.write.dll"
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-writemessage (Join-Path $repoRoot "current\aahClientManaged.dll") $writeOnly | Out-Null
dotnet run --no-build -c $Configuration --project $reProj -- `
instrument-wcf-readmessage $writeOnly $instrDll | Out-Null
Write-Host "== Staging current-copy ==" -ForegroundColor Cyan
robocopy (Join-Path $repoRoot "current") $currentCopy /MIR /NJH /NJS /NDL /NP /NC /NS | Out-Null
Copy-Item -Force $instrDll (Join-Path $currentCopy "aahClientManaged.dll")
Copy-Item -Force $instrSourceDll (Join-Path $currentCopy "AVEVA.Historian.ReverseInstrumentation.dll")
$harnessDll = Join-Path $currentCopy "aahClientManaged.dll"
if (Test-Path $capturePath) { Remove-Item -Force $capturePath }
$env:AVEVA_HISTORIAN_RE_CAPTURE = $capturePath
Write-Host "== Capturing tag-extended-properties ==" -ForegroundColor Green
$harnessArgs = @(
"--scenario", "tag-extended-properties",
"--server-name", $ServerName,
"--tcp-port", "$TcpPort",
"--tag", $Tag,
"--current-dir", $currentCopy,
"--managed-dll-path", $harnessDll
)
$harnessJson = $null
try {
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$harnessJson = & dotnet run --no-build -c $Configuration --project $harnessProj -- @harnessArgs 2>&1
} catch {
Write-Host " (tag-extended-properties raised: $($_.Exception.Message))" -ForegroundColor Yellow
} finally {
$ErrorActionPreference = $prevEap
}
Remove-Item Env:\AVEVA_HISTORIAN_RE_CAPTURE -ErrorAction SilentlyContinue
$recCount = if (Test-Path $capturePath) { (Get-Content $capturePath | Where-Object { $_.Trim() }).Count } else { 0 }
Write-Host "`n== Capture summary ==" -ForegroundColor Cyan
Write-Host " -> $recCount records -> $capturePath"
Write-Host "Harness output (TagExtendedProperties Success / Groups):" -ForegroundColor Cyan
$harnessJson | Select-Object -Last 24
Write-Host "`nDecode with: python scripts\decode-tag-properties-capture.py" -ForegroundColor Cyan
+117
View File
@@ -0,0 +1,117 @@
<#
.SYNOPSIS
Prompts for Historian credentials and persists them to an encrypted file under the
current user's profile, or loads previously-saved credentials into the current
session's HISTORIAN_USER and HISTORIAN_PASSWORD environment variables.
.DESCRIPTION
Persistence uses Windows DPAPI via Export-Clixml. The resulting XML can only be
decrypted by the same user account on the same machine. The file is saved
outside the repository (default: $env:USERPROFILE\.histsdk\credentials.xml) so
it cannot be accidentally committed.
The integration tests in this repo read HISTORIAN_USER and HISTORIAN_PASSWORD
from the process environment. Run this script with -Load before launching
`dotnet test` to inject the saved credentials into the current session.
.PARAMETER Load
Read previously-saved credentials and set $env:HISTORIAN_USER + $env:HISTORIAN_PASSWORD
in the current PowerShell session. The variables persist for the lifetime of
this session only - re-run with -Load in a new shell.
.PARAMETER Clear
Delete the saved credentials file. Subsequent -Load calls will fail until
credentials are re-saved.
.PARAMETER Path
Override the default storage path. Useful for keeping multiple credential sets
side-by-side (e.g. local-vs-remote).
.PARAMETER UserName
Skip the username prompt and use the supplied value. Password is still prompted
interactively. Convenient for scripted re-saves where only the password rotates.
.EXAMPLE
PS> .\scripts\Set-HistorianCredentials.ps1
Prompts for username + password, saves both to %USERPROFILE%\.histsdk\credentials.xml.
.EXAMPLE
PS> .\scripts\Set-HistorianCredentials.ps1 -Load
Loads the saved credentials into HISTORIAN_USER and HISTORIAN_PASSWORD for this session.
.EXAMPLE
PS> .\scripts\Set-HistorianCredentials.ps1 -Clear
Deletes the saved credentials file.
#>
[CmdletBinding(DefaultParameterSetName = 'Save')]
param(
[Parameter(ParameterSetName = 'Load')]
[switch]$Load,
[Parameter(ParameterSetName = 'Clear')]
[switch]$Clear,
[string]$Path = (Join-Path $env:USERPROFILE '.histsdk\credentials.xml'),
[Parameter(ParameterSetName = 'Save')]
[string]$UserName
)
$ErrorActionPreference = 'Stop'
function ConvertTo-PlainText {
param([Parameter(Mandatory)][securestring]$SecureString)
$bstr = [Runtime.InteropServices.Marshal]::SecureStringToBSTR($SecureString)
try { [Runtime.InteropServices.Marshal]::PtrToStringBSTR($bstr) }
finally { [Runtime.InteropServices.Marshal]::ZeroFreeBSTR($bstr) }
}
if ($Clear) {
if (Test-Path $Path) {
Remove-Item -LiteralPath $Path -Force
Write-Host "Deleted $Path"
} else {
Write-Host "Nothing to clear (file does not exist): $Path"
}
return
}
if ($Load) {
if (-not (Test-Path $Path)) {
throw "No saved credentials at $Path. Run this script without -Load to save them first."
}
$cred = Import-Clixml -LiteralPath $Path
if ($cred -isnot [System.Management.Automation.PSCredential]) {
throw "File at $Path is not a PSCredential - re-save with this script."
}
$env:HISTORIAN_USER = $cred.UserName
$env:HISTORIAN_PASSWORD = ConvertTo-PlainText $cred.Password
Write-Host "Loaded credentials for '$($cred.UserName)' from $Path."
Write-Host "HISTORIAN_USER and HISTORIAN_PASSWORD are set for this PowerShell session."
return
}
# Save mode (default).
$dir = Split-Path -Parent $Path
if (-not (Test-Path $dir)) {
New-Item -ItemType Directory -Force -Path $dir | Out-Null
}
if ([string]::IsNullOrWhiteSpace($UserName)) {
$suggested = "$env:COMPUTERNAME\$env:USERNAME"
$UserName = Read-Host "Historian user [$suggested]"
if ([string]::IsNullOrWhiteSpace($UserName)) {
$UserName = $suggested
}
}
$securePassword = Read-Host "Historian password for '$UserName'" -AsSecureString
$cred = New-Object System.Management.Automation.PSCredential($UserName, $securePassword)
$cred | Export-Clixml -LiteralPath $Path
Write-Host ""
Write-Host "Saved credentials for '$UserName' to $Path"
Write-Host "(DPAPI-encrypted; decryptable only by '$env:USERNAME' on '$env:COMPUTERNAME'.)"
Write-Host ""
Write-Host "Run '.\scripts\Set-HistorianCredentials.ps1 -Load' in any new session to inject"
Write-Host "the credentials into HISTORIAN_USER and HISTORIAN_PASSWORD."
+115
View File
@@ -0,0 +1,115 @@
"""Decode the AddTagExtendedProperties / DeleteTagExtendedProperties WCF inBuff (HCAL R1.11).
Reads the capture produced by scripts/Capture-AddTagExtendedProperties.ps1 and locates the AddTEx /
DelTep WriteMessage bodies by the sandbox tag + property name/value, then dumps the inBuff bytes so
the framing (tag name, property count, per-property name + value markers) can be read off. Compare to
the R1.5 read-response encoding in HistorianTagExtendedPropertyProtocol.
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPDIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-add-tep"
CAP = CAPDIR / "add-tep-capture-latest.ndjson"
TAG = "RetestSdkWriteTepTag"
PROP = "SdkTestProp"
VALUE = "SdkTestValue"
OP_ADD = b"AddTEx"
OP_DEL = b"DelTep"
def hexdump(label, buf, base=0):
print(f"=== {label}: {len(buf)} bytes ===")
for off in range(0, len(buf), 16):
c = buf[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {base + off:04X} {hp:<48} |{ap}|")
print()
def ascii_strings(buf, minlen=3):
out, cur, start = [], [], 0
for i, x in enumerate(buf):
if 32 <= x < 127:
if not cur:
start = i
cur.append(chr(x))
else:
if len(cur) >= minlen:
out.append((start, "".join(cur)))
cur = []
if len(cur) >= minlen:
out.append((start, "".join(cur)))
return out
def u16_strings(buf, minlen=3):
out, i = [], 0
while i < len(buf) - 1:
j, chars = i, []
while j < len(buf) - 1 and 32 <= buf[j] < 127 and buf[j + 1] == 0:
chars.append(chr(buf[j]))
j += 2
if len(chars) >= minlen:
out.append((i, "".join(chars)))
i = j
else:
i += 1
return out
def main() -> int:
if not CAP.exists():
print(f"Missing capture: {CAP}\nRun scripts/Capture-AddTagExtendedProperties.ps1 first.")
return 1
records = []
for line in CAP.open(encoding="utf-8-sig"):
if line.strip():
records.append(json.loads(line))
tag_a, prop_a, val_a = TAG.encode("ascii"), PROP.encode("ascii"), VALUE.encode("ascii")
tag_u, prop_u, val_u = TAG.encode("utf-16-le"), PROP.encode("utf-16-le"), VALUE.encode("utf-16-le")
print(f"== {len(records)} MDAS bodies captured ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
flags = []
if OP_ADD in body:
flags.append("AddTEx")
if OP_DEL in body:
flags.append("DelTep")
if prop_a in body or prop_u in body:
flags.append("PROP")
if val_a in body or val_u in body:
flags.append("VALUE")
print(f" [{idx:02d}] {rec.get('Phase'):26s} len={len(body):5d} {','.join(flags)}")
def dump(op):
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
if rec.get("Phase") == "WCF.WriteMessage.Body" and op in body:
hexdump(f"[{idx}] {op.decode()} WriteMessage", body)
print(" UTF-16 strings:")
for off, s in u16_strings(body):
print(f" 0x{off:04X} {s!r}")
print(" ASCII strings:")
for off, s in ascii_strings(body):
print(f" 0x{off:04X} {s!r}")
print()
print("\n== AddTEx request(s) ==")
dump(OP_ADD)
print("\n== DelTep request(s) ==")
dump(OP_DEL)
return 0
if __name__ == "__main__":
sys.exit(main())
+105
View File
@@ -0,0 +1,105 @@
"""Decode the DeleteTagExtendedProperties (DelTep) WCF inBuff (HCAL R1.11 delete half).
Reads the Run-B capture produced by scripts/Capture-DeleteTagExtendedProperties.ps1 and dumps the
DelTep WriteMessage body so the delete-request framing (tag name + property names + the
delete-from-server flag) can be read off and compared to the AddTEx serializer.
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPDIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-del-tep"
CAP = CAPDIR / "del-tep-capture-latest.ndjson"
TAG = "RetestSdkWriteDelTepSdk"
PROP = "SdkDelProp"
OP_DEL = b"DelTep"
OP_GET = b"GetTepByNm"
def hexdump(label, buf, base=0):
print(f"=== {label}: {len(buf)} bytes ===")
for off in range(0, len(buf), 16):
c = buf[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {base + off:04X} {hp:<48} |{ap}|")
print()
def ascii_strings(buf, minlen=3):
out, cur, start = [], [], 0
for i, x in enumerate(buf):
if 32 <= x < 127:
if not cur:
start = i
cur.append(chr(x))
else:
if len(cur) >= minlen:
out.append((start, "".join(cur)))
cur = []
if len(cur) >= minlen:
out.append((start, "".join(cur)))
return out
def u16_strings(buf, minlen=3):
out, i = [], 0
while i < len(buf) - 1:
j, chars = i, []
while j < len(buf) - 1 and 32 <= buf[j] < 127 and buf[j + 1] == 0:
chars.append(chr(buf[j]))
j += 2
if len(chars) >= minlen:
out.append((i, "".join(chars)))
i = j
else:
i += 1
return out
def main() -> int:
if not CAP.exists():
print(f"Missing capture: {CAP}\nRun scripts/Capture-DeleteTagExtendedProperties.ps1 first.")
return 1
records = []
for line in CAP.open(encoding="utf-8-sig"):
if line.strip():
records.append(json.loads(line))
print(f"== {len(records)} MDAS bodies captured (Run B) ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
flags = []
if OP_DEL in body:
flags.append("DelTep")
if OP_GET in body:
flags.append("GetTepByNm")
if TAG.encode("ascii") in body or TAG.encode("utf-16-le") in body:
flags.append("TAG")
if PROP.encode("ascii") in body or PROP.encode("utf-16-le") in body:
flags.append("PROP")
print(f" [{idx:02d}] {rec.get('Phase'):26s} len={len(body):5d} {','.join(flags)}")
print("\n== DelTep request(s) ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
if rec.get("Phase") == "WCF.WriteMessage.Body" and OP_DEL in body:
hexdump(f"[{idx}] DelTep WriteMessage", body)
print(" UTF-16 strings:")
for off, s in u16_strings(body):
print(f" 0x{off:04X} {s!r}")
print(" ASCII strings:")
for off, s in ascii_strings(body):
print(f" 0x{off:04X} {s!r}")
print()
return 0
if __name__ == "__main__":
sys.exit(main())
+113
View File
@@ -0,0 +1,113 @@
"""Decode the StartEventQuery filter-block encoding (HCAL R1.7).
Extracts the `pRequestBuff` from the StartEventQuery WriteMessage body in the baseline
(no filter) and filtered captures produced by scripts/Capture-EventFilter.ps1, dumps both,
and marks where they diverge so the filter predicate (property name / comparison op / value)
can be read off the empty-filter baseline.
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPDIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-event-filter"
PARAM = b"pRequestBuff"
OP = b"StartEventQuery"
def extract_request(path):
if not path.exists():
return None
for line in path.open(encoding="utf-8-sig"):
if not line.strip():
continue
rec = json.loads(line)
if rec.get("Phase") != "WCF.WriteMessage.Body":
continue
body = base64.b64decode(rec["Base64"])
if OP not in body:
continue
i = body.find(PARAM)
if i < 0:
continue
i += len(PARAM)
for s in range(i, min(i + 16, len(body))):
m = body[s]
if m == 0x9E:
return body[s + 2:s + 2 + body[s + 1]]
if m == 0x9F:
n = int.from_bytes(body[s + 1:s + 3], "little")
return body[s + 3:s + 3 + n]
if m == 0xA0:
n = int.from_bytes(body[s + 1:s + 3], "little")
return body[s + 3:s + 3 + n]
return None
def hexdump(label, buf):
print(f"=== {label}: {len(buf)} bytes ===")
for off in range(0, len(buf), 16):
c = buf[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {off:04X} {hp:<48} |{ap}|")
print()
def main() -> int:
base = extract_request(CAPDIR / "event-filter-capture-baseline-latest.ndjson")
filt = extract_request(CAPDIR / "event-filter-capture-filtered-latest.ndjson")
if base is None or filt is None:
print("Missing capture(s). Run scripts/Capture-EventFilter.ps1 first.")
print(f" baseline: {'ok' if base is not None else 'MISSING'}")
print(f" filtered: {'ok' if filt is not None else 'MISSING'}")
return 1
hexdump("baseline (no filter) pRequestBuff", base)
hexdump("filtered pRequestBuff", filt)
# First divergence offset.
n = min(len(base), len(filt))
div = next((i for i in range(n) if base[i] != filt[i]), n)
print(f"== First divergence at offset 0x{div:04X} (lenBase={len(base)} lenFilt={len(filt)}) ==")
print(" Filtered bytes from divergence (the inserted filter block):")
tail = filt[div:]
for off in range(0, len(tail), 16):
c = tail[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {div + off:04X} {hp:<48} |{ap}|")
print("\n== Strings in filtered buffer ==")
for enc, label in ((b"ascii", "ASCII"), (None, "UTF-16LE")):
if enc == b"ascii":
cur, start = [], 0
for i, x in enumerate(filt):
if 32 <= x < 127:
if not cur:
start = i
cur.append(chr(x))
else:
if len(cur) >= 3:
print(f" {label} 0x{start:04X} {''.join(cur)!r}")
cur = []
else:
i = 0
while i < len(filt) - 1:
j, chars = i, []
while j < len(filt) - 1 and 32 <= filt[j] < 127 and filt[j + 1] == 0:
chars.append(chr(filt[j]))
j += 2
if len(chars) >= 3:
print(f" {label} 0x{i:04X} {''.join(chars)!r}")
i = j
else:
i += 1
return 0
if __name__ == "__main__":
sys.exit(main())
+94
View File
@@ -0,0 +1,94 @@
"""Decide whether native event-send (HCAL R2.1) rides WCF or the storage-engine pipe.
Reads the both-hooks capture produced by scripts/Capture-EventSend.ps1 and, for every
outgoing WCF.WriteMessage.Body, tries to recognise the SOAP action / operation name. It
then renders a verdict:
* If a storage/event delivery op (AddStreamValues / EnqueueEventDataPacket /
OpenEventConnection / StartStorage / AddS2 / AddStreamValues2) appears on the WRITE path,
event-send is a WCF op M2 is implementable over WCF and that body carries the
PackToVtq event value blob to decode (R2.2).
* If NO such op appears on the WRITE path, the queued event was delivered via the
storage-engine shared-memory pipe (not WCF) M2 is architecturally blocked as a
pure-managed-WCF SDK, the same conclusion as the revision-write path.
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPTURE = (REPO_ROOT / "artifacts" / "reverse-engineering"
/ "instrumented-wcf-event-send" / "event-send-capture-latest.ndjson")
# Operation-name markers we care about. The MDAS binary SOAP body carries the action /
# operation name as readable text (ASCII and/or UTF-16LE). We scan for both encodings.
EVENT_OR_STORAGE_OPS = [
"AddStreamValues2", "AddStreamValues", "EnqueueEventDataPacket", "OpenEventConnection2",
"OpenEventConnection", "StartStorage", "AddS2", "ForwardEventSnapshot", "AddStreamedValue",
"AddNonStreamValues",
]
# Other ops we expect to see on a healthy event-send connection (auth/open/registration),
# printed for context so a "no event op" result is clearly "delivery left WCF", not "nothing ran".
KNOWN_OPS = [
"GetV", "ValCl", "Open2", "OpenConnection", "GETHI", "GetSystemParameter",
"UpdC3", "UpdateClientStatus3", "RTag2", "RegisterTags2", "EnsT2", "EnsureTags2",
"IsOriginalAllowed", "StartQuery", "GetInterfaceVersion",
]
def find_ops(body, candidates):
hits = []
for op in candidates:
a = op.encode("ascii")
u = op.encode("utf-16-le")
if a in body or u in body:
hits.append(op)
return hits
def main() -> int:
if not CAPTURE.exists():
print(f"Capture not found: {CAPTURE}")
print("Run: scripts/Capture-EventSend.ps1")
return 1
with CAPTURE.open(encoding="utf-8-sig") as fh:
records = [json.loads(line) for line in fh if line.strip()]
writes = [r for r in records if r.get("Phase") == "WCF.WriteMessage.Body"]
reads = [r for r in records if r.get("Phase") == "WCF.ReadMessage.Body"]
print(f"Records: {len(records)} (write={len(writes)} read={len(reads)})\n")
event_write_hits = []
print("== Outgoing WCF.WriteMessage.Body ops ==")
for i, r in enumerate(writes):
body = base64.b64decode(r["Base64"])
known = find_ops(body, KNOWN_OPS)
event = find_ops(body, EVENT_OR_STORAGE_OPS)
if event:
event_write_hits.extend(event)
label = ", ".join(event + known) or "<no recognized op>"
flag = " <<< EVENT/STORAGE OP" if event else ""
print(f" write[{i:02d}] {len(body):6d}B {label}{flag}")
print("\n== Verdict ==")
if event_write_hits:
uniq = sorted(set(event_write_hits))
print(f" EVENT/STORAGE op(s) on the WCF WRITE path: {uniq}")
print(" => event-send IS a WCF op. M2 viable over WCF; decode the PackToVtq value")
print(" blob in that body for R2.2.")
return 0
print(" NO event/storage delivery op on the WCF WRITE path.")
print(" => the queued event did NOT leave via WCF. If the native AddStreamedValue")
print(" returned success (see harness JSON), delivery used the storage-engine")
print(" shared-memory pipe — M2 is blocked as a pure-managed-WCF SDK, same as the")
print(" revision-write path (docs/plans/revision-write-path.md).")
return 0
if __name__ == "__main__":
sys.exit(main())
+114
View File
@@ -0,0 +1,114 @@
"""Decode the AddS2 (AddStreamValues2) pBuf event-VTQ blob captured for event-send (R2.2).
Extracts the `pBuf` parameter from the AddS2 WriteMessage body in the event-send capture
and hex-dumps it, annotating windows that match the known test event so the
HistorianEvent.PackToVtq framing can be read off and inverted into a managed serializer.
Known test event (from scripts/Capture-EventSend.ps1 defaults):
Type="User.Write", Namespace="RetestSdkEventSend",
properties: Source="RetestSdkEventSend", TestMarker="histsdk-R2.1-capture"
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import struct
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPTURE = (REPO_ROOT / "artifacts" / "reverse-engineering"
/ "instrumented-wcf-event-send" / "event-send-capture-latest.ndjson")
PARAM = b"pBuf"
ADDS2 = b"AddS2"
def extract_param(body, param):
i = body.find(param)
if i < 0:
return None
i += len(param)
# Skip the closing of the element name / attributes until a binary length marker.
# MDAS length markers: 0x9E (1-byte len), 0x9F (2-byte len), 0xA0 (2-byte len+1).
for scan in range(i, min(i + 16, len(body))):
marker = body[scan]
if marker == 0x9E:
length = body[scan + 1]
return body[scan + 2:scan + 2 + length]
if marker == 0x9F:
length = int.from_bytes(body[scan + 1:scan + 3], "little")
return body[scan + 3:scan + 3 + length]
if marker == 0xA0:
length = int.from_bytes(body[scan + 1:scan + 3], "little")
return body[scan + 3:scan + 3 + length + 1]
return None
def main() -> int:
if not CAPTURE.exists():
print(f"Capture not found: {CAPTURE}")
return 1
with CAPTURE.open(encoding="utf-8-sig") as fh:
records = [json.loads(line) for line in fh if line.strip()]
body = None
for r in records:
if r.get("Phase") != "WCF.WriteMessage.Body":
continue
b = base64.b64decode(r["Base64"])
if ADDS2 in b:
body = b
break
if body is None:
print("No AddS2 WriteMessage body found.")
return 2
pbuf = extract_param(body, PARAM)
if pbuf is None:
print("Found AddS2 body but could not extract pBuf. Full body hex dump:")
pbuf = body
print(f"pBuf: {len(pbuf)} bytes\n")
for off in range(0, len(pbuf), 16):
chunk = pbuf[off:off + 16]
hp = " ".join(f"{c:02X}" for c in chunk)
ap = "".join(chr(c) if 32 <= c < 127 else "." for c in chunk)
print(f" {off:04X} {hp:<48} |{ap}|")
print("\n== ASCII strings (len>=3) ==")
cur = []
start = 0
for i, c in enumerate(pbuf):
if 32 <= c < 127:
if not cur:
start = i
cur.append(chr(c))
else:
if len(cur) >= 3:
print(f" 0x{start:04X} {''.join(cur)!r}")
cur = []
if len(cur) >= 3:
print(f" 0x{start:04X} {''.join(cur)!r}")
print("\n== UTF-16LE strings (len>=3) ==")
i = 0
while i < len(pbuf) - 1:
j = i
chars = []
while j < len(pbuf) - 1 and 32 <= pbuf[j] < 127 and pbuf[j + 1] == 0:
chars.append(chr(pbuf[j]))
j += 2
if len(chars) >= 3:
print(f" 0x{i:04X} {''.join(chars)!r}")
i = j
else:
i += 1
return 0
if __name__ == "__main__":
sys.exit(main())
+142
View File
@@ -0,0 +1,142 @@
"""Decode the GetHistorianInfo (GETHI) WCF request/response (HCAL R1.4).
Reads the chained WriteMessage+ReadMessage capture produced by
scripts/Capture-HistorianInfo.ps1 and locates the GetHistorianInfo exchange. The goal is
to learn (a) the pRequestBuff that returns the FULL HISTORIAN_INFO struct (distinct from the
named-value "HistorianVersion" request) and (b) the response struct layout: the analysis
folder says it's 518 bytes with the version string (UTF-16, null-terminated) at offset 0 and
EventStorageMode (int32) at offset 514.
We flag candidate bodies by the GETHI op action, by the server version value, and by a
response length near 518, then dump bytes + the int32 at offset 514 so the layout can be
read off directly.
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import struct
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPDIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-historian-info"
CAP = CAPDIR / "historian-info-capture-latest.ndjson"
# The GETHI op action (WS-Addressing) the native client sends. The server version value is
# version-shaped, not secret; used only to locate the response.
OP_ASCII = b"GetHistorianInfo"
OP_GETHI = b"GETHI"
VERSION = "20,0,000,000"
VERSION_U16 = VERSION.encode("utf-16-le")
VERSION_ASCII = VERSION.encode("ascii")
def hexdump(label, buf, base=0):
print(f"=== {label}: {len(buf)} bytes ===")
for off in range(0, len(buf), 16):
c = buf[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {base + off:04X} {hp:<48} |{ap}|")
print()
def ascii_strings(buf, minlen=3):
out, cur, start = [], [], 0
for i, x in enumerate(buf):
if 32 <= x < 127:
if not cur:
start = i
cur.append(chr(x))
else:
if len(cur) >= minlen:
out.append((start, "".join(cur)))
cur = []
if len(cur) >= minlen:
out.append((start, "".join(cur)))
return out
def u16_strings(buf, minlen=3):
out, i = [], 0
while i < len(buf) - 1:
j, chars = i, []
while j < len(buf) - 1 and 32 <= buf[j] < 127 and buf[j + 1] == 0:
chars.append(chr(buf[j]))
j += 2
if len(chars) >= minlen:
out.append((i, "".join(chars)))
i = j
else:
i += 1
return out
def main() -> int:
if not CAP.exists():
print(f"Missing capture: {CAP}\nRun scripts/Capture-HistorianInfo.ps1 first.")
return 1
records = []
for line in CAP.open(encoding="utf-8-sig"):
if line.strip():
records.append(json.loads(line))
print(f"== {len(records)} MDAS bodies captured ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
flags = []
if OP_ASCII in body or OP_GETHI in body:
flags.append("GETHI-OP")
if VERSION_U16 in body or VERSION_ASCII in body:
flags.append("VERSION")
# A ~518-byte embedded struct is the tell for the full-info response.
if 500 <= len(body) <= 4096:
flags.append(f"len={len(body)}")
print(f" [{idx:02d}] {rec.get('Phase'):26s} len={len(body):5d} {','.join(flags)}")
def find(predicate):
hits = []
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
if predicate(rec, body):
hits.append((idx, rec, body))
return hits
print("\n== Request candidate(s): WriteMessage bodies tagged GETHI-OP ==")
for idx, rec, body in find(lambda r, b: r.get("Phase") == "WCF.WriteMessage.Body"
and (OP_ASCII in b or OP_GETHI in b)):
hexdump(f"[{idx}] WriteMessage", body)
print(" UTF-16 strings:")
for off, s in u16_strings(body):
print(f" 0x{off:04X} {s!r}")
print(" ASCII strings:")
for off, s in ascii_strings(body):
print(f" 0x{off:04X} {s!r}")
print()
print("\n== Response candidate(s): ReadMessage bodies carrying VERSION ==")
for idx, rec, body in find(lambda r, b: r.get("Phase") == "WCF.ReadMessage.Body"
and (VERSION_U16 in b or VERSION_ASCII in b)):
hexdump(f"[{idx}] ReadMessage", body)
print(" UTF-16 strings:")
for off, s in u16_strings(body):
print(f" 0x{off:04X} {s!r}")
# The analysis folder pins EventStorageMode @ offset 514 (int32) inside the
# 518-byte struct. The struct is embedded in the MDAS body at some base; scan for
# a plausible version@0 run and print the int32 514 bytes after each candidate base.
print(" Candidate struct decodes (version@base, int32 @ base+514):")
for base_off, s in u16_strings(body):
if any(ch.isdigit() for ch in s) and "," in s:
idx514 = base_off + 514
if idx514 + 4 <= len(body):
mode = struct.unpack_from("<i", body, idx514)[0]
print(f" base=0x{base_off:04X} version={s!r} int32@+514={mode}")
print()
return 0
if __name__ == "__main__":
sys.exit(main())
+129
View File
@@ -0,0 +1,129 @@
"""Decode the RenameTags WCF request/response (HCAL R1.10).
Reads the chained WriteMessage+ReadMessage capture produced by scripts/Capture-RenameTags.ps1
and locates the rename exchange. Rename maps to the generic job framework:
StJb (StartJob): WriteMessage carries op "StJb" + a string handle + the rename jobBuffer
(the (old,new) name pairs). ReadMessage carries the returned jobId string.
GtJb (GetJobStatus): WriteMessage carries op "GtJb" + handle + jobId. ReadMessage carries
the job-status buffer.
We flag bodies by the StJb/GtJb op and by the sandbox names, then dump the buffers so the
jobBuffer layout (batch count + old/new UTF-16 framing) can be read off directly.
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPDIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-rename"
CAP = CAPDIR / "rename-capture-latest.ndjson"
# Sandbox names used by the default capture run (not secret).
FROM = "RetestSdkWriteRenameSrc"
TO = "RetestSdkWriteRenameDst"
OP_STJB = b"StJb"
OP_GTJB = b"GtJb"
def hexdump(label, buf, base=0):
print(f"=== {label}: {len(buf)} bytes ===")
for off in range(0, len(buf), 16):
c = buf[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {base + off:04X} {hp:<48} |{ap}|")
print()
def ascii_strings(buf, minlen=3):
out, cur, start = [], [], 0
for i, x in enumerate(buf):
if 32 <= x < 127:
if not cur:
start = i
cur.append(chr(x))
else:
if len(cur) >= minlen:
out.append((start, "".join(cur)))
cur = []
if len(cur) >= minlen:
out.append((start, "".join(cur)))
return out
def u16_strings(buf, minlen=3):
out, i = [], 0
while i < len(buf) - 1:
j, chars = i, []
while j < len(buf) - 1 and 32 <= buf[j] < 127 and buf[j + 1] == 0:
chars.append(chr(buf[j]))
j += 2
if len(chars) >= minlen:
out.append((i, "".join(chars)))
i = j
else:
i += 1
return out
def main() -> int:
if not CAP.exists():
print(f"Missing capture: {CAP}\nRun scripts/Capture-RenameTags.ps1 first.")
return 1
records = []
for line in CAP.open(encoding="utf-8-sig"):
if line.strip():
records.append(json.loads(line))
from_u16, to_u16 = FROM.encode("utf-16-le"), TO.encode("utf-16-le")
from_a, to_a = FROM.encode("ascii"), TO.encode("ascii")
print(f"== {len(records)} MDAS bodies captured ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
flags = []
if OP_STJB in body:
flags.append("StJb")
if OP_GTJB in body:
flags.append("GtJb")
if from_u16 in body or from_a in body:
flags.append("FROM")
if to_u16 in body or to_a in body:
flags.append("TO")
print(f" [{idx:02d}] {rec.get('Phase'):26s} len={len(body):5d} {','.join(flags)}")
def find(predicate):
hits = []
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
if predicate(rec, body):
hits.append((idx, rec, body))
return hits
print("\n== StJb request(s): WriteMessage bodies tagged StJb ==")
for idx, rec, body in find(lambda r, b: r.get("Phase") == "WCF.WriteMessage.Body" and OP_STJB in b):
hexdump(f"[{idx}] StJb WriteMessage", body)
print(" UTF-16 strings:")
for off, s in u16_strings(body):
print(f" 0x{off:04X} {s!r}")
print(" ASCII strings:")
for off, s in ascii_strings(body):
print(f" 0x{off:04X} {s!r}")
print()
print("\n== StJb / GtJb response(s) + GtJb request(s) ==")
for idx, rec, body in find(lambda r, b: (OP_STJB in b or OP_GTJB in b) and r.get("Phase") == "WCF.ReadMessage.Body"):
hexdump(f"[{idx}] {rec.get('Phase')}", body)
print(" strings:", [s for _, s in ascii_strings(body)][:16])
print()
return 0
if __name__ == "__main__":
sys.exit(main())
+128
View File
@@ -0,0 +1,128 @@
"""Decode the GetRuntimeParameter WCF request/response (HCAL R1.2).
Reads the chained WriteMessage+ReadMessage capture produced by
scripts/Capture-RuntimeParam.ps1 and locates the GetRuntimeParameter exchange by
searching every MDAS body for the parameter name (UTF-16) on the request side and the
returned value on the response side. Dumps the surrounding bytes so the op name, the
leading handle parameter, and the btRequest/btResponse buffer layout can be read off.
Output is diagnostic. Sanitize before copying into docs/.
"""
import base64
import json
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPDIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-runtime-param"
CAP = CAPDIR / "runtime-param-capture-latest.ndjson"
# Markers we expect on the wire for the default "HistorianVersion" capture.
NAME = "HistorianVersion"
NAME_U16 = NAME.encode("utf-16-le")
NAME_ASCII = NAME.encode("ascii")
VALUE = "20,0,000,000" # server runtime "HistorianVersion" value (version-shaped, not secret)
VALUE_U16 = VALUE.encode("utf-16-le")
VALUE_ASCII = VALUE.encode("ascii")
def hexdump(label, buf, base=0):
print(f"=== {label}: {len(buf)} bytes ===")
for off in range(0, len(buf), 16):
c = buf[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {base + off:04X} {hp:<48} |{ap}|")
print()
def ascii_strings(buf, minlen=3):
out, cur, start = [], [], 0
for i, x in enumerate(buf):
if 32 <= x < 127:
if not cur:
start = i
cur.append(chr(x))
else:
if len(cur) >= minlen:
out.append((start, "".join(cur)))
cur = []
if len(cur) >= minlen:
out.append((start, "".join(cur)))
return out
def u16_strings(buf, minlen=3):
out, i = [], 0
while i < len(buf) - 1:
j, chars = i, []
while j < len(buf) - 1 and 32 <= buf[j] < 127 and buf[j + 1] == 0:
chars.append(chr(buf[j]))
j += 2
if len(chars) >= minlen:
out.append((i, "".join(chars)))
i = j
else:
i += 1
return out
def main() -> int:
if not CAP.exists():
print(f"Missing capture: {CAP}\nRun scripts/Capture-RuntimeParam.ps1 first.")
return 1
records = []
for line in CAP.open(encoding="utf-8-sig"):
if line.strip():
records.append(json.loads(line))
print(f"== {len(records)} MDAS bodies captured ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
flags = []
if NAME_U16 in body or NAME_ASCII in body:
flags.append("NAME")
if VALUE_U16 in body or VALUE_ASCII in body:
flags.append("VALUE")
# The WS-Addressing action is the most reliable op label; show any string that
# looks like an op (contains a slash or is short and capitalized).
print(f" [{idx:02d}] {rec.get('Phase'):26s} len={len(body):5d} {','.join(flags)}")
def find(predicate):
hits = []
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
if predicate(rec, body):
hits.append((idx, rec, body))
return hits
print("\n== Request candidate(s): WriteMessage bodies containing the NAME ==")
for idx, rec, body in find(lambda r, b: r.get("Phase") == "WCF.WriteMessage.Body"
and (NAME_U16 in b or NAME_ASCII in b)):
hexdump(f"[{idx}] WriteMessage", body)
print(" UTF-16 strings:")
for off, s in u16_strings(body):
print(f" 0x{off:04X} {s!r}")
print(" ASCII strings:")
for off, s in ascii_strings(body):
print(f" 0x{off:04X} {s!r}")
print()
print("\n== Response candidate(s): ReadMessage bodies containing the VALUE ==")
for idx, rec, body in find(lambda r, b: r.get("Phase") == "WCF.ReadMessage.Body"
and (VALUE_U16 in b or VALUE_ASCII in b)):
hexdump(f"[{idx}] ReadMessage", body)
print(" UTF-16 strings:")
for off, s in u16_strings(body):
print(f" 0x{off:04X} {s!r}")
print(" ASCII strings:")
for off, s in ascii_strings(body):
print(f" 0x{off:04X} {s!r}")
print()
return 0
if __name__ == "__main__":
sys.exit(main())
+115
View File
@@ -0,0 +1,115 @@
"""Decoder for the analog/state summary request capture (HCAL roadmap R1.8/R1.9).
Reads the per-config NDJSON captures produced by scripts/Capture-SummaryRequest.ps1
under artifacts/reverse-engineering/instrumented-wcf-writemessage-summary/, extracts
the Retr/StartQuery2 `pRequestBuff` payload from each, hex-dumps it, and diffs every
summary candidate against the baseline-full request so the differing bytes (the native
QueryType / SummaryType / AutoSummaryParameters fields) stand out.
Output is diagnostic. The only printed strings are the SDK-chosen system tag name and
protocol field markers sanitize before copying any of it into docs/.
"""
import base64
import json
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPTURE_DIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-writemessage-summary"
ACTION = b"aa/Retr/StartQuery2"
PARAM = b"pRequestBuff"
def extract_request_buffer(records):
"""Return the pRequestBuff bytes from the first StartQuery2 write record, or None."""
for rec in records:
if rec.get("Phase") != "WCF.WriteMessage.Body":
continue
body = base64.b64decode(rec["Base64"])
if ACTION not in body:
continue
i = body.find(PARAM)
if i < 0:
continue
i += len(PARAM)
marker = body[i]
# MDAS length markers (same scheme as the write decoder).
if marker == 0x9E:
length = body[i + 1]
return body[i + 2:i + 2 + length]
if marker == 0x9F:
length = int.from_bytes(body[i + 1:i + 3], "little")
return body[i + 3:i + 3 + length]
if marker == 0xA0:
length = int.from_bytes(body[i + 1:i + 3], "little")
return body[i + 3:i + 3 + length + 1]
return None
return None
def hexdump(payload, diff_against=None):
for off in range(0, len(payload), 16):
chunk = payload[off:off + 16]
cells = []
for j, c in enumerate(chunk):
mark = ""
if diff_against is not None:
k = off + j
if k >= len(diff_against) or diff_against[k] != c:
mark = "*"
cells.append(f"{c:02X}{mark}")
hp = " ".join(cells)
ap = "".join(chr(c) if 32 <= c < 127 else "." for c in chunk)
print(f" {off:04X} {hp:<56} |{ap}|")
def load(path):
with path.open(encoding="utf-8-sig") as fh:
return [json.loads(line) for line in fh if line.strip()]
def main() -> int:
if not CAPTURE_DIR.exists():
print(f"Capture dir not found: {CAPTURE_DIR}")
print("Run scripts/Capture-SummaryRequest.ps1 first.")
return 1
captures = sorted(CAPTURE_DIR.glob("summary-capture-*-latest.ndjson"))
if not captures:
print(f"No capture files in {CAPTURE_DIR}")
return 1
buffers = {}
for path in captures:
name = path.stem.replace("summary-capture-", "").replace("-latest", "")
records = load(path)
buf = extract_request_buffer(records)
buffers[name] = buf
status = f"{len(buf)} bytes" if buf else "<no StartQuery2 request found>"
print(f"{name:<18} records={len(records):>3} pRequestBuff={status}")
baseline = buffers.get("baseline-full")
print()
if not baseline:
print("No baseline-full request buffer captured; cannot diff. Dumping each raw.")
for name, buf in buffers.items():
if buf:
print(f"\n== {name} pRequestBuff ({len(buf)} bytes) ==")
hexdump(buf)
return 0
print(f"== baseline-full pRequestBuff ({len(baseline)} bytes) ==")
hexdump(baseline)
for name, buf in buffers.items():
if name == "baseline-full" or not buf:
continue
print(f"\n== {name} pRequestBuff ({len(buf)} bytes) — '*' marks bytes differing from baseline ==")
hexdump(buf, diff_against=baseline)
return 0
if __name__ == "__main__":
sys.exit(main())
+123
View File
@@ -0,0 +1,123 @@
"""Decode the GetNextQueryResultBuffer2 *response* for an analog summary (HCAL R1.8).
Reads the both-hooks capture produced by
scripts/Capture-SummaryRequest.ps1 -OnlyConfig analog-avg -WithResponse
finds the ReadMessage record carrying GetNextQueryResultBuffer2Response, extracts the
`pResultBuff` payload, hex-dumps it, and annotates every 8-byte window that decodes to a
known ground-truth value (the AnalogSummaryHistory row for SysTimeSec) so the field offsets
of CAnalogSummaryValue can be read off directly.
Output is diagnostic; the only printed strings are the SDK-chosen system tag name and field
markers. Sanitize before copying into docs/.
"""
import base64
import json
import struct
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
# Config name (analog-avg / analog-min / analog-max / …) selectable via argv[1].
CONFIG = sys.argv[1] if len(sys.argv) > 1 else "analog-avg"
CAPTURE = (REPO_ROOT / "artifacts" / "reverse-engineering"
/ "instrumented-wcf-writemessage-summary" / f"summary-capture-{CONFIG}-latest.ndjson")
RESP = b"GetNextQueryResultBuffer2Response"
PARAM = b"pResultBuff"
# Ground-truth values from AnalogSummaryHistory(SysTimeSec, 1h cycle) — used to label offsets.
KNOWN_DOUBLES = {
31.0: "31.0 (First/Last/Average)",
100.0: "100.0 (PercentGood)",
0.031: "0.031 (Integral)",
111600.0: "111600.0 (Integral, full-cycle)",
1.0: "1.0 (ValueCount as double?)",
}
KNOWN_U32 = {
1: "ValueCount=1",
192: "OPCQuality=192",
100: "PercentGood=100",
9: "version=9",
}
def extract_param(body, param):
i = body.find(param)
if i < 0:
return None
i += len(param)
marker = body[i]
if marker == 0x9E:
length = body[i + 1]
return body[i + 2:i + 2 + length]
if marker == 0x9F:
length = int.from_bytes(body[i + 1:i + 3], "little")
return body[i + 3:i + 3 + length]
if marker == 0xA0:
length = int.from_bytes(body[i + 1:i + 3], "little")
return body[i + 3:i + 3 + length + 1]
return None
def main() -> int:
if not CAPTURE.exists():
print(f"Capture not found: {CAPTURE}")
print("Run: scripts/Capture-SummaryRequest.ps1 -OnlyConfig analog-avg -WithResponse")
return 1
with CAPTURE.open(encoding="utf-8-sig") as fh:
records = [json.loads(line) for line in fh if line.strip()]
payload = None
for rec in records:
if rec.get("Phase") != "WCF.ReadMessage.Body":
continue
body = base64.b64decode(rec["Base64"])
if RESP not in body:
continue
payload = extract_param(body, PARAM)
break
if payload is None:
print("No GetNextQueryResultBuffer2Response / pResultBuff found in capture.")
return 2
print(f"pResultBuff: {len(payload)} bytes")
if len(payload) >= 6:
version = int.from_bytes(payload[0:2], "little")
row_count = int.from_bytes(payload[2:6], "little")
print(f" header: version={version} rowCount={row_count}")
print()
# Annotated hex dump.
for off in range(0, len(payload), 16):
chunk = payload[off:off + 16]
hp = " ".join(f"{c:02X}" for c in chunk)
ap = "".join(chr(c) if 32 <= c < 127 else "." for c in chunk)
print(f" {off:04X} {hp:<48} |{ap}|")
# Scan every 8-byte window for known doubles, and every 4-byte window for known u32s.
print("\n== Known-value hits (offset -> field) ==")
for off in range(0, len(payload) - 7):
val = struct.unpack_from("<d", payload, off)[0]
for known, label in KNOWN_DOUBLES.items():
if val == known or (known != 0 and abs(val - known) < 1e-9 * max(1.0, abs(known))):
print(f" 0x{off:04X} double {val!r:>14} -> {label}")
for off in range(0, len(payload) - 3):
val = int.from_bytes(payload[off:off + 4], "little")
if val in KNOWN_U32:
print(f" 0x{off:04X} uint32 {val:>14} -> {KNOWN_U32[val]}")
# FILETIME windows (plausible 2026 timestamps: 0x01DC.. high dword).
print("\n== Plausible FILETIME windows (Int64, year ~2020-2030) ==")
for off in range(0, len(payload) - 7):
ft = int.from_bytes(payload[off:off + 8], "little")
# FILETIME for 2020-01-01 ~= 0x01D5BF.. ; 2030 ~= 0x01E5.. — gate by high word.
if 0x01D5_0000_0000_0000 <= ft <= 0x01E6_0000_0000_0000:
print(f" 0x{off:04X} filetime 0x{ft:016X}")
return 0
if __name__ == "__main__":
sys.exit(main())
+75
View File
@@ -0,0 +1,75 @@
"""Decode the GetTagExtendedPropertiesFromName (GetTepByNm) WCF request/response (HCAL R1.5).
Reads the chained WriteMessage+ReadMessage capture produced by
scripts/Capture-TagExtendedProperties.ps1, locates the aa/Retr/GetTepByNm exchange, and
dumps the tagNames request buffer + tagExtendedProperties response buffer so the op name,
the uppercase string handle, the tagNames layout, and the extended-property response layout
can be read off.
Request tagNames buffer:
uint32 count + per name: uint32 charCount + UTF-16LE chars.
Response tagExtendedProperties buffer:
uint32 tagCount
per tag: byte marker(0x01) + compact-ASCII tagName(0x09 + uint16 len + ascii)
uint32 propCount
per prop: byte marker(0x02) + compact-ASCII propName(0x09 + uint16 len + ascii)
value: 0x43 (VT_BSTR) + uint16 payloadLen + uint16 charCount + UTF-16LE
trailing byte(0x01)
Output is diagnostic. Sanitize before copying into docs/ (tag names / values are dev data).
"""
import base64
import json
import re
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
CAPDIR = REPO_ROOT / "artifacts" / "reverse-engineering" / "instrumented-wcf-tag-extended-properties"
DEFAULT_CAP = CAPDIR / "tep-localized-capture.ndjson"
ACTION = re.compile(rb"aa/[A-Za-z0-9]+/[A-Za-z0-9_]+")
def hexdump(label, buf):
print(f"=== {label}: {len(buf)} bytes ===")
for off in range(0, len(buf), 16):
c = buf[off:off + 16]
hp = " ".join(f"{x:02X}" for x in c)
ap = "".join(chr(x) if 32 <= x < 127 else "." for x in c)
print(f" {off:04X} {hp:<48} |{ap}|")
print()
def main() -> int:
cap = Path(sys.argv[1]) if len(sys.argv) > 1 else DEFAULT_CAP
if not cap.exists():
print(f"Missing capture: {cap}\nRun scripts/Capture-TagExtendedProperties.ps1 -Localized first.")
return 1
records = [json.loads(l) for l in cap.open(encoding="utf-8-sig") if l.strip()]
print(f"== {len(records)} MDAS bodies captured ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
acts = sorted({m.decode() for m in ACTION.findall(body)})
flag = " <== GetTepByNm" if any("Tep" in a for a in acts) else ""
print(f" [{idx:02d}] {rec.get('Phase'):24s} len={len(body):5d} {acts}{flag}")
print("\n== GetTepByNm request(s) [WriteMessage] ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
if rec.get("Phase") == "WCF.WriteMessage.Body" and b"GetTepByNm" in body:
hexdump(f"[{idx}] request", body)
print("\n== GetTepByNm response(s) [ReadMessage] ==")
for idx, rec in enumerate(records):
body = base64.b64decode(rec["Base64"])
if rec.get("Phase") == "WCF.ReadMessage.Body" and b"GetTepByNmResponse" in body:
hexdump(f"[{idx}] response", body)
return 0
if __name__ == "__main__":
sys.exit(main())
@@ -0,0 +1,173 @@
// Frida hook for the native ExchangeKey credential-token crypto (Windows CNG / bcrypt.dll).
// Traces the ECDH secret agreement, the KDF (with its parameter list), symmetric-key import, and
// encrypt/hash so the 26-byte v8 credential-token derivation can be reconstructed in managed code.
// Reverse-engineering aid only — observes the native client; nothing is shipped from here.
'use strict';
function resolve(modName, fnName) {
let m = null;
try { m = Process.getModuleByName(modName); } catch (e) {
try { m = Module.load(modName); } catch (e2) { return null; }
}
try { return m.findExportByName(fnName); } catch (e) { return null; }
}
function dump(label, ptr, len) {
if (ptr.isNull() || len <= 0) { console.log(label + ' <empty>'); return; }
const n = Math.min(len, 256);
console.log(label + ' (' + len + ' bytes)\n' + hexdump(ptr, { length: n, header: false, ansi: false }));
}
function hook(modName, fnName, onEnter, onLeave) {
const addr = resolve(modName, fnName);
if (!addr) { console.log('[skip] ' + modName + '!' + fnName + ' not found'); return; }
Interceptor.attach(addr, { onEnter: onEnter, onLeave: onLeave });
console.log('[hooked] ' + modName + '!' + fnName);
}
// BCryptOpenAlgorithmProvider(phAlgorithm, pszAlgId, pszImplementation, dwFlags) — names every algo used.
hook('bcrypt.dll', 'BCryptOpenAlgorithmProvider', function (a) {
console.log('[OpenAlgorithmProvider] algId=' + (a[1].isNull() ? '?' : a[1].readUtf16String()));
});
// BCryptSecretAgreement(hPrivKey, hPubKey, *phAgreedSecret, flags)
hook('bcrypt.dll', 'BCryptSecretAgreement', function (a) {
console.log('[SecretAgreement] hPriv=' + a[0] + ' hPub=' + a[1]);
});
// Decode a BCryptBufferDesc parameter list (used by BCryptDeriveKey) into (type -> bytes).
function dumpParamList(pParamList) {
if (pParamList.isNull()) { console.log(' paramList <null>'); return; }
const cBuffers = pParamList.add(4).readU32(); // ULONG ulVersion; ULONG cBuffers;
const pBuffers = pParamList.add(8).readPointer(); // BCryptBuffer* pBuffers;
const names = { 0: 'HASH_ALGORITHM', 1: 'SECRET_PREPEND', 2: 'SECRET_APPEND', 3: 'HMAC_KEY',
4: 'TLS_PRF_LABEL', 5: 'TLS_PRF_SEED', 6: 'SECRET_HANDLE', 8: 'SP80056A_CONCAT',
0xD: 'LABEL', 0xE: 'CONTEXT', 0xF: 'SALT', 0x10: 'ITERATION_COUNT' };
console.log(' paramList cBuffers=' + cBuffers);
for (let i = 0; i < cBuffers; i++) {
const b = pBuffers.add(i * 16); // { ULONG cbBuffer; ULONG BufferType; PVOID pvBuffer; }
const cb = b.readU32();
const type = b.add(4).readU32();
const pv = b.add(8).readPointer();
const tn = names[type] || ('0x' + type.toString(16));
if (type === 0 || type === 4 || type === 0xD) { // string-ish (hash alg name / label)
console.log(' [' + tn + '] ' + (pv.isNull() ? '?' : pv.readUtf16String()));
} else {
dump(' [' + tn + ']', pv, cb);
}
}
}
// BCryptDeriveKey(hSecret, pwszKDF, *pParamList, pbDerivedKey, cbDerivedKey, *pcbResult, flags)
hook('bcrypt.dll', 'BCryptDeriveKey', function (a) {
this.kdf = a[1].isNull() ? '?' : a[1].readUtf16String();
this.outKey = a[3]; this.pcb = a[5];
console.log('[DeriveKey] KDF=' + this.kdf + ' cbDerivedKey=' + a[4].toInt32());
dumpParamList(a[2]);
}, function () {
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
dump('[DeriveKey] derived', this.outKey, n);
});
hook('bcrypt.dll', 'BCryptDeriveKeyPBKDF2', function (a) {
console.log('[PBKDF2] cbPassword=' + a[2].toInt32() + ' cbSalt=' + a[4].toInt32() + ' iter=' + a[5]);
dump(' password', a[1], a[2].toInt32());
dump(' salt', a[3], a[4].toInt32());
});
// BCryptGenerateSymmetricKey(hAlg, *phKey, pbKeyObject, cbKeyObject, pbSecret, cbSecret, flags) — the actual key bytes.
hook('bcrypt.dll', 'BCryptGenerateSymmetricKey', function (a) {
dump('[GenerateSymmetricKey] keyBytes', a[4], a[5].toInt32());
});
// BCryptEncrypt(hKey, pbIn, cbIn, *pPad, pbIV, cbIV, pbOut, cbOut, *pcbResult, flags)
hook('bcrypt.dll', 'BCryptEncrypt', function (a) {
this.out = a[6]; this.pcb = a[8];
dump('[Encrypt] plaintext', a[1], a[2].toInt32());
dump('[Encrypt] IV', a[4], a[5].toInt32());
}, function () {
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
dump('[Encrypt] ciphertext', this.out, n);
});
// Hash path (in case the token is a keyed hash rather than a cipher).
hook('bcrypt.dll', 'BCryptHashData', function (a) {
dump('[HashData] input', a[1], a[2].toInt32());
});
hook('bcrypt.dll', 'BCryptFinishHash', function (a) {
this.out = a[1]; this.cb = a[2].toInt32();
}, function () {
dump('[FinishHash] digest', this.out, this.cb);
});
// ---- NCrypt (CNG key-storage layer) — the likely home of the ECDH ExchangeKey + token crypto ----
// NCryptSecretAgreement(hPrivKey, hPubKey, *phAgreedSecret, dwFlags)
hook('ncrypt.dll', 'NCryptSecretAgreement', function (a) {
console.log('[NCryptSecretAgreement] hPriv=' + a[0] + ' hPub=' + a[1]);
console.log(' backtrace (addr -> module+offset):');
Thread.backtrace(this.context, Backtracer.ACCURATE).slice(0, 14).forEach(function (addr) {
const m = Process.findModuleByAddress(addr);
if (m) {
console.log(' ' + addr + ' ' + m.name + '+0x' + addr.sub(m.base).toString(16));
} else {
console.log(' ' + addr + ' <JIT/unknown>');
}
});
});
// NCryptDeriveKey(hSharedSecret, pwszKDF, *pParameterList, pbDerivedKey, cbDerivedKey, *pcbResult, dwFlags)
hook('ncrypt.dll', 'NCryptDeriveKey', function (a) {
this.kdf = a[1].isNull() ? '?' : a[1].readUtf16String();
this.outKey = a[3]; this.pcb = a[5];
console.log('[NCryptDeriveKey] KDF=' + this.kdf + ' cbDerivedKey=' + a[4].toInt32());
dumpParamList(a[2]);
}, function () {
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
dump('[NCryptDeriveKey] derived', this.outKey, n);
});
// NCryptEncrypt(hKey, pbInput, cbInput, *pPaddingInfo, pbOutput, cbOutput, *pcbResult, dwFlags)
hook('ncrypt.dll', 'NCryptEncrypt', function (a) {
this.out = a[4]; this.pcb = a[6];
dump('[NCryptEncrypt] plaintext', a[1], a[2].toInt32());
}, function () {
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
dump('[NCryptEncrypt] ciphertext', this.out, n);
});
// NCryptImportKey(hProvider, hImportKey, pszBlobType, *pParameterList, *phKey, pbData, cbData, dwFlags)
hook('ncrypt.dll', 'NCryptImportKey', function (a) {
console.log('[NCryptImportKey] blobType=' + (a[2].isNull() ? '?' : a[2].readUtf16String()));
dump(' blob', a[5], a[6].toInt32());
});
// NCryptExportKey(hKey, hExportKey, pszBlobType, *pParameterList, pbOutput, cbOutput, *pcbResult, dwFlags)
hook('ncrypt.dll', 'NCryptExportKey', function (a) {
this.blobType = a[2].isNull() ? '?' : a[2].readUtf16String();
this.out = a[4]; this.pcb = a[6];
}, function () {
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
console.log('[NCryptExportKey] blobType=' + this.blobType);
dump(' blob', this.out, n);
});
hook('ncrypt.dll', 'NCryptOpenStorageProvider', function (a) {
console.log('[NCryptOpenStorageProvider] ' + (a[1].isNull() ? '?' : a[1].readUtf16String()));
});
// BCrypt EC key operations (in case the ECDH is bcrypt but uses import/export rather than DeriveKey).
hook('bcrypt.dll', 'BCryptImportKeyPair', function (a) {
console.log('[BCryptImportKeyPair] blobType=' + (a[2].isNull() ? '?' : a[2].readUtf16String()) + ' cb=' + a[5].toInt32());
dump(' blob', a[4], a[5].toInt32());
});
hook('bcrypt.dll', 'BCryptExportKey', function (a) {
this.blobType = a[2].isNull() ? '?' : a[2].readUtf16String();
this.out = a[3]; this.pcb = a[5];
}, function () {
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
console.log('[BCryptExportKey] blobType=' + this.blobType);
dump(' blob', this.out, n);
});
console.log('=== CNG ExchangeKey crypto hooks installed ===');
@@ -7,11 +7,29 @@
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<PackageReference Include="System.Formats.Nrbf" Version="10.0.7" />
<PackageReference Include="System.Security.Cryptography.Xml" Version="10.0.7" /> <PackageReference Include="System.Security.Cryptography.Xml" Version="10.0.7" />
<PackageReference Include="System.ServiceModel.NetNamedPipe" Version="10.0.652802" /> <PackageReference Include="System.ServiceModel.NetNamedPipe" Version="10.0.652802" />
<PackageReference Include="System.ServiceModel.NetTcp" Version="10.0.652802" /> <PackageReference Include="System.ServiceModel.NetTcp" Version="10.0.652802" />
</ItemGroup> </ItemGroup>
<!-- 2023 R2 gRPC transport (RemoteGrpc). Pure-managed: Grpc.Net.Client +
Google.Protobuf. Grpc.Tools is build-only (PrivateAssets=all) and
generates the client stubs from the recovered contract under Grpc/Protos. -->
<ItemGroup>
<PackageReference Include="Google.Protobuf" Version="3.24.4" />
<PackageReference Include="Grpc.Net.Client" Version="2.58.0" />
<PackageReference Include="Grpc.Net.Client.Web" Version="2.58.0" />
<PackageReference Include="Grpc.Tools" Version="2.59.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>
<Protobuf Include="Grpc\Protos\*.proto" GrpcServices="Client" ProtoRoot="Grpc\Protos" />
</ItemGroup>
<ItemGroup> <ItemGroup>
<AssemblyAttribute Include="System.Runtime.CompilerServices.InternalsVisibleToAttribute"> <AssemblyAttribute Include="System.Runtime.CompilerServices.InternalsVisibleToAttribute">
<_Parameter1>AVEVA.Historian.Client.Tests</_Parameter1> <_Parameter1>AVEVA.Historian.Client.Tests</_Parameter1>
@@ -0,0 +1,137 @@
using System.Net.Http;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;
using Grpc.Core;
using Grpc.Net.Client;
using Grpc.Net.Client.Web;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// Builds a <see cref="GrpcChannel"/> for the 2023 R2 Historian Client Access Point,
/// replicating the stock <c>Archestra.Historian.GrpcClient.GrpcClientBase.InitializeBase</c>
/// transport shape: gRPC-Web (binary) over HTTP/1.1, optional TLS with an
/// untrusted-certificate bypass, and gzip request encoding.
/// </summary>
internal static class HistorianGrpcChannelFactory
{
/// <summary>
/// Resolves the effective gRPC port: when the caller left <see cref="HistorianClientOptions.Port"/>
/// at the WCF default (32568), the 2023 R2 gRPC default (32565) is substituted; otherwise the
/// explicit value is honoured.
/// </summary>
internal static int ResolvePort(HistorianClientOptions options) =>
options.Port == HistorianClientOptions.DefaultPort ? HistorianClientOptions.DefaultGrpcPort : options.Port;
/// <summary>
/// Builds the channel address. TLS uses <c>https://{ServerDnsIdentity|Host}:{port}</c> (the
/// DNS-identity override lets the URL match the server certificate name when connecting by IP);
/// plaintext uses <c>http://{Host}:{port}</c>.
/// </summary>
internal static string ResolveAddress(HistorianClientOptions options)
{
int port = ResolvePort(options);
if (options.GrpcUseTls)
{
string tlsHost = !string.IsNullOrEmpty(options.ServerDnsIdentity) ? options.ServerDnsIdentity! : options.Host;
return $"https://{tlsHost}:{port}";
}
return $"http://{options.Host}:{port}";
}
public static HistorianGrpcConnection Create(HistorianClientOptions options)
{
string address = ResolveAddress(options);
var httpHandler = new HttpClientHandler();
if (options.AllowUntrustedServerCertificate)
{
httpHandler.ServerCertificateCustomValidationCallback = AcceptAnyCertificate;
}
// gRPC-Web binary mode over HTTP/1.1 — matches the stock client (GrpcWebMode.GrpcWeb,
// HttpVersion 1.1). The 2023 R2 HCAP endpoint speaks gRPC-Web, not bare HTTP/2 gRPC.
var webHandler = new GrpcWebHandler(GrpcWebMode.GrpcWeb, httpHandler)
{
HttpVersion = new Version(1, 1)
};
var channelOptions = new GrpcChannelOptions
{
HttpHandler = webHandler
};
GrpcChannel channel = GrpcChannel.ForAddress(address, channelOptions);
return new HistorianGrpcConnection(channel, BuildMetadata(options));
}
/// <summary>
/// Builds an event-path channel that speaks <b>native HTTP/2 gRPC</b> (no <see cref="GrpcWebHandler"/>
/// wrap) — the leading hypothesis for why gRPC-Web event reads return zero rows while the native
/// <c>Grpc.Core</c> HTTP/2 client returns rows for a byte-identical request. The stock 2023 R2 client
/// uses native <c>Grpc.Core</c> (HTTP/2); reads happen to work over gRPC-Web too, but the
/// connection-scoped event query may require a true HTTP/2 connection. Over TLS this depends on the
/// server negotiating the <c>h2</c> ALPN protocol; <see cref="SocketsHttpHandler"/> is pinned to
/// HTTP/2 exact so the channel does not silently fall back to HTTP/1.1.
/// </summary>
public static HistorianGrpcConnection CreateHttp2(HistorianClientOptions options)
{
string address = ResolveAddress(options);
var socketsHandler = new SocketsHttpHandler
{
EnableMultipleHttp2Connections = true
};
if (options.AllowUntrustedServerCertificate && options.GrpcUseTls)
{
socketsHandler.SslOptions = new SslClientAuthenticationOptions
{
RemoteCertificateValidationCallback = (_, _, _, _) => true
};
}
var channelOptions = new GrpcChannelOptions
{
HttpHandler = socketsHandler
};
// GrpcChannel over a SocketsHttpHandler already issues requests as HTTP/2 with
// RequestVersionExact (no GrpcWebHandler means no HTTP/1.1 fallback to mask a failed h2
// negotiation — it surfaces instead).
GrpcChannel channel = GrpcChannel.ForAddress(address, channelOptions);
return new HistorianGrpcConnection(channel, BuildMetadata(options));
}
private static Metadata BuildMetadata(HistorianClientOptions options)
{
// The stock client always advertises gzip request encoding; honour the option so
// bandwidth-limited links can disable it.
var metadata = new Metadata();
if (options.Compression)
{
metadata.Add("grpc-internal-encoding-request", "gzip");
}
return metadata;
}
private static bool AcceptAnyCertificate(
HttpRequestMessage request,
X509Certificate2? certificate,
X509Chain? chain,
SslPolicyErrors errors) => true;
}
/// <summary>A live gRPC channel plus the per-call metadata header set.</summary>
internal sealed class HistorianGrpcConnection(GrpcChannel channel, Metadata metadata) : IDisposable
{
public GrpcChannel Channel { get; } = channel;
public Metadata Metadata { get; } = metadata;
public void Dispose() => Channel.Dispose();
}
@@ -0,0 +1,452 @@
using System.Runtime.CompilerServices;
using Google.Protobuf;
using Grpc.Core;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
using GrpcHistory = ArchestrA.Grpc.Contract.History;
using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval;
using GrpcStatus = ArchestrA.Grpc.Contract.Status;
using GrpcTransaction = ArchestrA.Grpc.Contract.Transaction;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// 2023 R2 gRPC event-read orchestrator. Mirrors <see cref="HistorianWcfEventOrchestrator"/> over the
/// gRPC transport: the same CM_EVENT registration sequence and the same event request/row buffers
/// travel inside protobuf <c>bytes</c> fields, reusing the proven WCF serializers/parsers verbatim.
///
/// Operation mapping (2020 WCF → 2023 R2 gRPC):
/// Hist.UpdateClientStatus3 → HistoryService.UpdateClientStatus
/// Hist.RegisterTags2 → HistoryService.RegisterTags
/// Hist.EnsureTags2 → HistoryService.EnsureTags
/// Stat.GetHistorianInfo → StatusService.GetHistorianInfo
/// Stat.GetSystemParameter → StatusService.GetSystemParameter
/// Retr.StartEventQuery → RetrievalService.StartEventQuery
/// Retr.GetNextEventQueryResultBuffer (loop) → RetrievalService.GetNextEventQueryResultBuffer
/// Retr.EndEventQuery → RetrievalService.EndEventQuery
///
/// <para>
/// The CM_EVENT registration replay (<see cref="RegisterCmEventTag"/>) is the hard part: without it
/// the server returns native error type=4 code=85 from GetNextEventQueryResultBuffer. The captured
/// registration buffers are shared with the WCF path via
/// <see cref="HistorianEventRegistrationProtocol"/> so the two transports cannot drift. The gRPC
/// RetrievalService event ops do NOT need the WCF <c>Retr.GetV</c>/<c>IsOriginalAllowed</c> prime
/// (the read path proved the front-door session is sufficient over gRPC).
/// </para>
/// <para>
/// <b>Live status — server-gated (settled 2026-06-25):</b> the chain runs end-to-end and
/// <c>StartEventQuery</c> succeeds, but <c>GetNextEventQueryResultBuffer</c> <b>long-polls</b> to the
/// no-data terminal (instead of the synchronous 5-byte code-85 terminal the 2020 WCF op returns); a
/// poll-deadline expiry is treated as that terminal (see the loop). This is <b>not</b> an empty-box
/// artifact: the live 2023 R2 server holds tens of thousands of events yet scopes <b>0 rows</b> to a
/// managed connection. Every client-controllable layer was byte-matched to the stock client that returns
/// rows (see <c>docs/reverse-engineering/grpc-event-query-capture.md</c>) — the gate is a server-internal
/// per-connection retrieval working-set, <b>not client-fixable</b>. The legacy WCF transport is not a
/// fallback on 2023 R2 (<c>docs/reverse-engineering/wcf-event-read-spike-results.md</c>). Tooled +
/// completes cleanly, but proven NOT to return rows over a managed connection.
/// </para>
/// </summary>
internal sealed class HistorianGrpcEventOrchestrator
{
private readonly HistorianClientOptions _options;
public HistorianGrpcEventOrchestrator(HistorianClientOptions options)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
}
/// <summary>Diagnostic: length of the most recent event-row result buffer the server sent.</summary>
public int LastResultBufferLength { get; private set; }
/// <summary>Diagnostic: type+code description of the most recent error/terminal buffer.</summary>
public string LastErrorBufferDescription { get; private set; } = string.Empty;
/// <summary>Diagnostic: which transport the event channel used (<c>grpc-web</c> or <c>http2</c>).</summary>
public string EventChannelMode { get; private set; } = string.Empty;
/// <summary>Diagnostic: hex of the most recent result buffer (first 48 bytes).</summary>
public string LastResultBufferHex { get; private set; } = string.Empty;
/// <summary>Diagnostic: hex of the most recent GetNext error buffer.</summary>
public string LastErrorBufferHex { get; private set; } = string.Empty;
public async IAsyncEnumerable<HistorianEvent> ReadEventsAsync(
DateTime startUtc,
DateTime endUtc,
HistorianEventFilter? filter,
[EnumeratorCancellation] CancellationToken cancellationToken)
{
if (!_options.IntegratedSecurity && string.IsNullOrEmpty(_options.UserName))
{
throw new ProtocolEvidenceMissingException(
"Managed gRPC event flow currently requires IntegratedSecurity or an explicit UserName + Password.");
}
cancellationToken.ThrowIfCancellationRequested();
// Hard overall cap. The per-call gRPC-Web deadlines are NOT honored reliably over a tunnelled
// link (observed live 2026-06-22: a chain with 4s per-call deadlines still ran >90s because the
// server stalls several registration RPCs and long-polls GetNext). gRPC DOES honor token
// cancellation, so a linked CTS firing at OverallBudget bounds the whole read deterministically.
// A budget timeout on the unverified no-row path is surfaced as ProtocolEvidenceMissing, not as
// a raw cancellation, so callers get the same honest "not row-verified over gRPC" signal.
using CancellationTokenSource linked = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
linked.CancelAfter(OverallBudget);
IReadOnlyList<HistorianEvent> events;
try
{
events = await Task.Run(
() => RunEventChain(startUtc, endUtc, filter, linked.Token),
linked.Token).ConfigureAwait(false);
}
catch (Exception ex) when (IsBudgetCancellation(ex, linked, cancellationToken))
{
throw new ProtocolEvidenceMissingException(
$"ReadEvents over gRPC did not return rows within {OverallBudget.TotalSeconds:0}s: StartEventQuery " +
"succeeds but GetNextEventQueryResultBuffer long-polls to the no-data terminal. Event-row retrieval is " +
"auth-solved but SERVER-GATED on 2023 R2 over both transports — the server scopes 0 rows to a managed " +
"connection (gRPC: docs/reverse-engineering/grpc-event-query-capture.md). The WCF transport reaches the " +
"2023 R2 historian (certificate transport + auth work, CM_EVENT registration succeeds on the 0x501 event " +
"connection) but hits the SAME server-side row gate — 0-row buffer + long-poll (see " +
"docs/reverse-engineering/wcf-event-read-spike-results.md). Not client-fixable on either transport.");
}
foreach (HistorianEvent evt in events)
{
cancellationToken.ThrowIfCancellationRequested();
yield return evt;
}
}
/// <summary>
/// Hard wall-clock budget for the entire gRPC event read. Bounds the chain deterministically since
/// per-call gRPC-Web deadlines are unreliable over a tunnel. Scaled off the request timeout but
/// capped so a long default timeout cannot make the (currently row-unverified) read stall for minutes.
/// </summary>
private TimeSpan OverallBudget
{
get
{
TimeSpan cap = TimeSpan.FromSeconds(30);
return _options.RequestTimeout < cap ? _options.RequestTimeout : cap;
}
}
/// <summary>
/// True when an exception was caused by the overall-budget linked CTS firing (not by the caller's
/// own cancellation). The budget surfaces either as an <see cref="OperationCanceledException"/>
/// (Task.Run / token checks) or a gRPC <see cref="RpcException"/> with
/// <see cref="StatusCode.Cancelled"/> from an in-flight RPC.
/// </summary>
private static bool IsBudgetCancellation(Exception ex, CancellationTokenSource linked, CancellationToken caller)
{
if (caller.IsCancellationRequested || !linked.IsCancellationRequested)
{
return false;
}
return ex is OperationCanceledException
|| (ex is RpcException rpc && rpc.StatusCode is StatusCode.Cancelled or StatusCode.DeadlineExceeded);
}
private List<HistorianEvent> RunEventChain(DateTime startUtc, DateTime endUtc, HistorianEventFilter? filter, CancellationToken cancellationToken)
{
// Hypothesis #1 (server-side/connection angle, grpc-event-query-capture.md): the native client
// uses Grpc.Core native HTTP/2, while our default channel wraps gRPC-Web over HTTP/1.1. Reads
// work over gRPC-Web, but the connection-scoped event query may require a true HTTP/2 connection.
// Opt in via HISTORIAN_GRPC_EVENT_HTTP2=1 to use a plain HTTP/2 channel for the event path only.
bool useHttp2 = string.Equals(
Environment.GetEnvironmentVariable("HISTORIAN_GRPC_EVENT_HTTP2"), "1", StringComparison.Ordinal);
EventChannelMode = useHttp2 ? "http2" : "grpc-web";
using HistorianGrpcConnection connection = useHttp2
? HistorianGrpcChannelFactory.CreateHttp2(_options)
: HistorianGrpcChannelFactory.Create(_options);
// Event reads need an Event-type (v8) connection. OpenSession(eventConnection: true) runs the
// full v8 path: HistoryService.ExchangeKey (P-256 ECDH) -> client key = SHA256(secret) -> v8
// OpenConnection with ConnectionType=Event and the credential token RC4(password, MD5(clientKey)).
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken, eventConnection: true);
RegisterCmEventTag(connection, session, cancellationToken);
List<HistorianEvent> events = RunEventQueryOnSession(connection, session, startUtc, endUtc, filter, cancellationToken);
// Honest no-data handling: when the query returns real rows, hand them back. When it instead
// reaches the no-data terminal with ZERO rows (the gRPC server long-polls GetNext rather than
// returning the WCF code-85 terminal), we cannot distinguish "genuinely no events in range"
// from "the CM_EVENT registration replay didn't fully land over gRPC" — so we refuse to return
// a possibly-false empty list and surface the gated state instead. Proven server-gated: the live
// 2023 R2 server holds tens of thousands of events yet scopes 0 to a managed connection
// (grpc-event-query-capture.md). WCF reaches the same historian (cert transport + auth work,
// CM_EVENT registers on the 0x501 event connection) but hits the SAME row gate — not a fallback
// (wcf-event-read-spike-results.md).
if (events.Count == 0)
{
throw new ProtocolEvidenceMissingException(
"ReadEvents over gRPC: the chain completes and StartEventQuery succeeds, but " +
"GetNextEventQueryResultBuffer returns no rows (it long-polls to the no-data terminal " +
$"after the CM_EVENT registration replay; last={LastErrorBufferDescription}). Event-row retrieval is " +
"auth-solved but SERVER-GATED on 2023 R2 over both transports — the server scopes 0 rows to a managed " +
"connection (gRPC: docs/reverse-engineering/grpc-event-query-capture.md; WCF reaches the historian and " +
"registers on the 0x501 event connection yet hits the same row gate: " +
"docs/reverse-engineering/wcf-event-read-spike-results.md). Not client-fixable on either transport.");
}
return events;
}
private DateTime Deadline() => DateTime.UtcNow.Add(_options.RequestTimeout);
/// <summary>
/// Deadline for the GetNextEventQueryResultBuffer long-poll. Bounded to at most 10s (or the
/// configured <see cref="HistorianClientOptions.RequestTimeout"/> if shorter) so the no-data
/// terminal — a deadline expiry over gRPC — is reached promptly instead of stalling the read for
/// the full request timeout. When rows are available the server returns them well before this.
/// </summary>
private DateTime EventPollDeadline()
{
TimeSpan cap = TimeSpan.FromSeconds(10);
TimeSpan poll = _options.RequestTimeout < cap ? _options.RequestTimeout : cap;
return DateTime.UtcNow.Add(poll);
}
/// <summary>
/// Deadline for the best-effort registration RPCs. Bounded to at most 5s: several of these
/// (RegisterTags / EnsureTags / GetHistorianInfo) <b>stall server-side</b> on the remote 2023 R2
/// box (observed live 2026-06-22) and only return at their deadline, so an unbounded
/// <see cref="RequestTimeout"/> would make the registration phase dominate the read. They are
/// swallowed via <see cref="TryRun"/> regardless of outcome.
/// </summary>
private DateTime RegistrationDeadline()
{
TimeSpan cap = TimeSpan.FromSeconds(5);
TimeSpan d = _options.RequestTimeout < cap ? _options.RequestTimeout : cap;
return DateTime.UtcNow.Add(d);
}
/// <summary>
/// Replays the native event-tag registration sequence (UpdC3 → 6 system params → RTag2 → 1 more
/// system param → cross-service GetV probes → EnsT2) over the gRPC services. Best-effort: each
/// call is wrapped so an individual rejection on this server does not abort the chain — the goal
/// is to drive the server-side session into the state StartEventQuery / GetNextEventQueryResultBuffer
/// expect. Buffers come from <see cref="HistorianEventRegistrationProtocol"/>.
/// </summary>
private void RegisterCmEventTag(HistorianGrpcConnection connection, HistorianGrpcHandshake.Session session, CancellationToken cancellationToken)
{
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel);
// Native 2023 R2 gRPC event-connection registration sequence (captured order):
// UpdateClientStatus -> RegisterTags(CM_EVENT) -> EnsureTags(CM_EVENT) -> GetHistorianInfo
// -> GetSystemParameter x7. (StartEventQuery follows in RunEventQuery.) The 2020-WCF-era extra
// probes (cross-service GetV, params-before-register) are NOT in the gRPC event flow.
byte[] clientStatus = HistorianEventRegistrationProtocol.BuildUpdateClientStatusBlob();
TryRun(() => historyClient.UpdateClientStatus(
new GrpcHistory.UpdateClientStatusRequest { StrHandle = session.StringHandle, BtClientStatus = ByteString.CopyFrom(clientStatus) },
connection.Metadata, RegistrationDeadline(), cancellationToken));
byte[] registerBuffer = HistorianEventRegistrationProtocol.BuildRegisterCmEventInputBuffer();
try
{
GrpcHistory.RegisterTagsResponse rt = historyClient.RegisterTags(
new GrpcHistory.RegisterTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(registerBuffer) },
connection.Metadata, RegistrationDeadline(), cancellationToken);
RegistrationDiag += $"RTag={rt.Status?.BSuccess} e={Convert.ToHexString(rt.Status?.BtError?.ToByteArray() ?? [])}; ";
}
catch (Exception ex) { RegistrationDiag += $"RTag=EX:{ex.GetType().Name}; "; }
// gRPC CM_EVENT EnsureTags uses the 86-byte native format (8-byte header + the …2f27 event-type
// GUID), NOT the 2020 WCF CTagMetadata.
byte[] payload = HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc(DateTime.UtcNow);
try
{
GrpcHistory.EnsureTagsResponse et = historyClient.EnsureTags(
new GrpcHistory.EnsureTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(payload), ElementCount = 1 },
connection.Metadata, RegistrationDeadline(), cancellationToken);
RegistrationDiag += $"EnsT={et.Status?.BSuccess} e={Convert.ToHexString(et.Status?.BtError?.ToByteArray() ?? [])} out={Convert.ToHexString(et.BtTagStatus?.ToByteArray() ?? [])}; ";
}
catch (Exception ex) { RegistrationDiag += $"EnsT=EX:{ex.GetType().Name}; "; }
byte[] historianVersionRequest = HistorianEventRegistrationProtocol.BuildGetHistorianInfoRequest("HistorianVersion");
TryRun(() => statusClient.GetHistorianInfo(
new GrpcStatus.GetHistorianInfoRequest { StrHandle = session.StringHandle, BtRequest = ByteString.CopyFrom(historianVersionRequest) },
connection.Metadata, RegistrationDeadline(), cancellationToken));
string[] eventParams = ["AllowOriginals", "HistorianPartner", "HistorianVersion", "MaxCyclicStorageTimeout", "RealTimeWindow", "FutureTimeThreshold", "AllowRenameTags"];
foreach (string parameterName in eventParams)
{
TryRun(() => statusClient.GetSystemParameter(
new GrpcStatus.GetSystemParameterRequest { UiHandle = session.ClientHandle, StrParameterName = parameterName },
connection.Metadata, RegistrationDeadline(), cancellationToken));
}
}
/// <summary>Diagnostic: outcomes of the key CM_EVENT registration RPCs.</summary>
public string RegistrationDiag { get; private set; } = string.Empty;
// Spike seam (pending.md A1 broadening, Stage B0b): run ONLY the event query (StartEventQuery →
// GetNext loop → EndEventQuery) against an EXTERNALLY-supplied, already-opened + CM_EVENT-registered
// v8 Event connection + session — NO Create()/OpenSession/RegisterCmEventTag here. The per-call
// RunEventChain delegates to this so the per-call read and the B0b reuse spike share one query
// implementation (DRY). NOTE: event reads are otherwise GATED (C2) — the gRPC server long-polls
// GetNext to the no-data terminal and row-level retrieval is not yet verified over gRPC (see class
// remarks); the SEND seam is the spike's primary reuse signal. The split-channel opt-in
// (HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL) is preserved inside, unchanged.
internal List<HistorianEvent> RunEventQueryOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
DateTime startUtc,
DateTime endUtc,
HistorianEventFilter? filter,
CancellationToken cancellationToken)
{
// HTTP/2-frame capture (grpc-event-query-capture.md #3) showed the stock client runs the event
// query on a DEDICATED RetrievalService TLS connection, separate from the HistoryService
// connection that opened+registered the session (correlated only by the session handle); our SDK
// collapses every service onto one connection. Opt in via HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL=1 to
// run StartEventQuery/GetNext/EndEventQuery on their own connection (mirrors native conn4: no
// re-handshake, just the existing handle), to test whether topology is the row-scoping gate.
bool splitChannel = string.Equals(
Environment.GetEnvironmentVariable("HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL"), "1", StringComparison.Ordinal);
HistorianGrpcConnection rconn = splitChannel ? HistorianGrpcChannelFactory.Create(_options) : connection;
try
{
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(rconn.Channel);
GrpcRetrieval.GetRetrievalInterfaceVersionResponse retrievalVersion = retrievalClient.GetRetrievalInterfaceVersion(
new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), rconn.Metadata, Deadline(), cancellationToken);
HistorianServerVersionGate.Validate(HistorianServiceInterface.Retrieval, retrievalVersion.UiVersion, _options);
// Version 6 envelope: the stock 2023 R2 client sends v6 (the WCF path's v5 request is accepted
// here but is the legacy format). NECESSARY but not alone sufficient — live validation 2026-06-22
// showed rows still don't flow on v6 because the read also requires an EVENT-type connection
// (the stock client opens ConnectionType=Event; our OpenSession opens a Process-style 0x402
// session). See docs/reverse-engineering/grpc-event-query-capture.md "remaining gate".
IReadOnlyList<HistorianEventQueryAttempt> attempts = HistorianEventQueryProtocol.CreateStartEventQueryAttempts(
startUtc.ToUniversalTime(),
endUtc.ToUniversalTime(),
eventCount: 100,
filter,
version: 6);
byte[] requestBuffer = attempts[0].RequestBuffer;
GrpcRetrieval.StartEventQueryResponse startResponse = retrievalClient.StartEventQuery(
new GrpcRetrieval.StartEventQueryRequest
{
UiHandle = session.ClientHandle,
UiQueryRequestType = HistorianEventQueryProtocol.QueryRequestTypeEvent,
BtRequest = ByteString.CopyFrom(requestBuffer)
},
rconn.Metadata,
Deadline(),
cancellationToken);
byte[] startError = startResponse.Status?.BtError?.ToByteArray() ?? [];
if (!(startResponse.Status?.BSuccess ?? false))
{
throw new InvalidOperationException(
$"gRPC StartEventQuery failed (errorLen={startError.Length}, error5={HistorianEventRegistrationProtocol.DescribeNativeError(startError)}).");
}
uint queryHandle = startResponse.UiQueryHandle;
RegistrationDiag += $"QH={queryHandle} clientH={session.ClientHandle} strH={session.StringHandle}; ";
try
{
List<HistorianEvent> events = [];
while (true)
{
cancellationToken.ThrowIfCancellationRequested();
GrpcRetrieval.GetNextEventQueryResultBufferResponse nextResponse;
try
{
nextResponse = retrievalClient.GetNextEventQueryResultBuffer(
new GrpcRetrieval.GetNextEventQueryResultBufferRequest { UiHandle = session.ClientHandle, UiQueryHandle = queryHandle },
rconn.Metadata,
EventPollDeadline(),
cancellationToken);
}
catch (RpcException ex) when (ex.StatusCode == StatusCode.DeadlineExceeded)
{
// No-data terminal. Over gRPC the 2023 R2 server LONG-POLLS GetNextEventQueryResultBuffer
// when the query has no (more) rows to hand back, rather than returning the 5-byte
// type=4 code=85 terminal the 2020 WCF op returns synchronously. A poll-deadline
// expiry is therefore the gRPC equivalent of that soft terminal: stop reading and
// return whatever rows were already collected. (Confirmed live 2026-06-22: the chain
// runs and StartEventQuery succeeds, but GetNext blocks to the deadline on the idle
// dev box, which holds no events.) See class remarks.
LastErrorBufferDescription = "GetNext long-poll deadline (no-data terminal)";
return events;
}
byte[] resultBuffer = nextResponse.BtResult?.ToByteArray() ?? [];
byte[] errorBuffer = nextResponse.Status?.BtError?.ToByteArray() ?? [];
bool nextSuccess = nextResponse.Status?.BSuccess ?? false;
LastResultBufferLength = resultBuffer.Length;
LastErrorBufferDescription = HistorianEventRegistrationProtocol.DescribeNativeError(errorBuffer);
LastResultBufferHex = Convert.ToHexString(resultBuffer.Length <= 48 ? resultBuffer : resultBuffer[..48]);
LastErrorBufferHex = Convert.ToHexString(errorBuffer);
// Any 5-byte type=4 error is a soft terminal (code 30 NoMoreData is canonical; code
// 85 / 0x55 is the missing-registration signal seen on early runs). Mirror the WCF
// orchestrator: stop reading and surface the diagnostic rather than throw.
if (errorBuffer.Length == 5 && errorBuffer[0] == 4)
{
return events;
}
if (!nextSuccess)
{
throw new InvalidOperationException(
$"gRPC GetNextEventQueryResultBuffer failed (errorLen={errorBuffer.Length}, error5={HistorianEventRegistrationProtocol.DescribeNativeError(errorBuffer)}).");
}
if (resultBuffer.Length > 0)
{
events.AddRange(HistorianEventRowProtocol.Parse(resultBuffer));
}
if (resultBuffer.Length == 0 && errorBuffer.Length == 0)
{
return events;
}
}
}
finally
{
EndEventQuerySafely(retrievalClient, rconn, session.ClientHandle, queryHandle);
}
}
finally
{
if (splitChannel) { rconn.Dispose(); }
}
}
private void EndEventQuerySafely(
GrpcRetrieval.RetrievalService.RetrievalServiceClient client,
HistorianGrpcConnection connection,
uint clientHandle,
uint queryHandle)
{
try
{
client.EndEventQuery(
new GrpcRetrieval.EndEventQueryRequest { UiHandle = clientHandle, UiQueryHandle = queryHandle },
connection.Metadata,
Deadline(),
CancellationToken.None);
}
catch
{
// Best-effort cleanup; the read result is already collected.
}
}
private static void TryRun(Action action)
{
try { action(); }
catch { }
}
}
@@ -0,0 +1,165 @@
using Google.Protobuf;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
using GrpcHistory = ArchestrA.Grpc.Contract.History;
using GrpcStatus = ArchestrA.Grpc.Contract.Status;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// 2023 R2 gRPC orchestrator for the event SEND (<see cref="HistorianClient.SendEventAsync"/>).
/// Captured live from the native 2023 R2 client (<c>capture-send-event</c> scenario,
/// 2026-06-23): an event send rides <c>HistoryService.AddStreamValues</c> with the SAME
/// <c>"OS"</c> (0x534F) storage-sample buffer the WCF AddS2 path uses
/// (<see cref="HistorianEventWriteProtocol"/>) — NOT a distinct event RPC and NOT the historical
/// write's <c>"ON"</c> buffer. The native client's write-enabled Event <c>OpenConnection</c>
/// request is byte-identical to the read-only event open (the ReadOnly arg does not change the v8
/// open buffer; diffed live — only the per-session client key + credential token differ), so the
/// existing <see cref="HistorianGrpcHandshake.OpenSession"/> event path is reused unchanged. The
/// chain on a single Event session:
/// <list type="number">
/// <item>OpenConnection (v8 Event, ExchangeKey ECDH auth) → string storage handle</item>
/// <item>CM_EVENT registration: UpdateClientStatus → RegisterTags → EnsureTags (the same
/// buffers the gRPC event READ replays — verified byte-identical to the capture)</item>
/// <item><c>HistoryService.AddStreamValues</c>(strHandle, "OS" event buffer)</item>
/// </list>
/// Only original events (<see cref="HistorianEvent.RevisionVersion"/> = 0) with string-valued
/// properties have a captured encoding; others throw <see cref="ProtocolEvidenceMissingException"/>
/// from <see cref="HistorianEventWriteProtocol"/>.
/// </summary>
internal sealed class HistorianGrpcEventWriteOrchestrator
{
private readonly HistorianClientOptions _options;
public HistorianGrpcEventWriteOrchestrator(HistorianClientOptions options)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
}
/// <summary>Diagnostic: type+code description of the most recent AddStreamValues error buffer.</summary>
public string LastSendErrorDescription { get; private set; } = string.Empty;
/// <summary>Diagnostic: outcomes of the CM_EVENT registration RPCs.</summary>
public string RegistrationDiag { get; private set; } = string.Empty;
public Task<bool> SendEventAsync(HistorianEvent evt, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(evt);
if (!_options.IntegratedSecurity && string.IsNullOrEmpty(_options.UserName))
{
throw new ProtocolEvidenceMissingException(
"Managed gRPC event send currently requires IntegratedSecurity or an explicit UserName + Password.");
}
if (evt.RevisionVersion != 0)
{
throw new ProtocolEvidenceMissingException(
"Only original events (RevisionVersion = 0) have a captured send encoding; " +
"revision/update/delete event sends are not yet supported.");
}
return Task.Run(() => Run(evt, cancellationToken), cancellationToken);
}
private bool Run(HistorianEvent evt, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
// The event SEND uses the same v8 Event connection as the event READ. The write-enabled
// open buffer is byte-identical to the read-only one (verified live), so OpenSession's
// event path is reused unchanged. Per-call: open + register + send on a fresh session.
HistorianGrpcHandshake.Session session = OpenAndRegisterEventSession(connection, cancellationToken);
return SendEventOnSession(connection, session, evt, cancellationToken);
}
// Spike seam (pending.md A1 broadening, Stage B0b): open a v8 Event connection and drive the
// CM_EVENT registration ONCE, returning the warm (connection, session). The per-call Run() uses
// it for a single send; the B0b reuse spike calls it once and then issues MULTIPLE
// SendEventOnSession ops against the returned session to measure whether a v8 Event session can
// be reused across sends (it has NEVER been proven reusable — that is exactly what B0b measures).
// The caller owns the connection's lifetime (dispose it).
internal HistorianGrpcHandshake.Session OpenAndRegisterEventSession(
HistorianGrpcConnection connection,
CancellationToken cancellationToken)
{
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(
connection, _options, cancellationToken, eventConnection: true);
RegisterCmEventTag(connection, session, cancellationToken);
return session;
}
// Spike seam (pending.md A1 broadening, Stage B0b): perform ONLY the event send against an
// EXTERNALLY-supplied, already-opened + CM_EVENT-registered v8 Event connection + session —
// i.e. NO Create(), NO OpenSession(eventConnection:true), NO RegisterCmEventTag inside it. The
// per-call Run() path delegates here so the per-call send and the B0b reuse-spike send share one
// implementation (DRY) and stay byte-identical. The spike drives this repeatedly on one warm
// session to measure whether the server honors a reused v8 Event session for multiple sends.
internal bool SendEventOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
HistorianEvent evt,
CancellationToken cancellationToken)
{
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
byte[] pBuf = HistorianEventWriteProtocol.SerializeAddStreamValuesBuffer(evt, DateTime.UtcNow);
GrpcHistory.AddStreamValuesResponse response = historyClient.AddStreamValues(
new GrpcHistory.AddStreamValuesRequest
{
StrHandle = session.StringHandle,
BtValues = ByteString.CopyFrom(pBuf),
},
connection.Metadata,
DateTime.UtcNow.Add(_options.RequestTimeout),
cancellationToken);
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
LastSendErrorDescription = HistorianEventRegistrationProtocol.DescribeNativeError(error);
return response.Status?.BSuccess ?? false;
}
/// <summary>
/// Replays the CM_EVENT registration the native event connection performs before a send:
/// UpdateClientStatus → RegisterTags(CM_EVENT) → EnsureTags(CM_EVENT). The buffers are shared
/// with the gRPC event READ path (<see cref="HistorianEventRegistrationProtocol"/> +
/// <see cref="HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc"/>) and were verified
/// byte-identical to the live capture. Best-effort: an individual rejection does not abort the
/// send (the server may already hold CM_EVENT registered for the session).
/// </summary>
private void RegisterCmEventTag(HistorianGrpcConnection connection, HistorianGrpcHandshake.Session session, CancellationToken cancellationToken)
{
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
DateTime Deadline() => DateTime.UtcNow.Add(_options.RequestTimeout);
byte[] clientStatus = HistorianEventRegistrationProtocol.BuildUpdateClientStatusBlob();
try
{
historyClient.UpdateClientStatus(
new GrpcHistory.UpdateClientStatusRequest { StrHandle = session.StringHandle, BtClientStatus = ByteString.CopyFrom(clientStatus) },
connection.Metadata, Deadline(), cancellationToken);
}
catch { /* best-effort */ }
byte[] registerBuffer = HistorianEventRegistrationProtocol.BuildRegisterCmEventInputBuffer();
try
{
GrpcHistory.RegisterTagsResponse rt = historyClient.RegisterTags(
new GrpcHistory.RegisterTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(registerBuffer) },
connection.Metadata, Deadline(), cancellationToken);
RegistrationDiag += $"RTag={rt.Status?.BSuccess}; ";
}
catch (Exception ex) { RegistrationDiag += $"RTag=EX:{ex.GetType().Name}; "; }
byte[] payload = HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc(DateTime.UtcNow);
try
{
GrpcHistory.EnsureTagsResponse et = historyClient.EnsureTags(
new GrpcHistory.EnsureTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(payload), ElementCount = 1 },
connection.Metadata, Deadline(), cancellationToken);
RegistrationDiag += $"EnsT={et.Status?.BSuccess}; ";
}
catch (Exception ex) { RegistrationDiag += $"EnsT=EX:{ex.GetType().Name}; "; }
}
}
@@ -0,0 +1,149 @@
using System.Security.Cryptography;
using Google.Protobuf;
using Grpc.Core;
using AVEVA.Historian.Client.Wcf;
using GrpcHistory = ArchestrA.Grpc.Contract.History;
using GrpcStorage = ArchestrA.Grpc.Contract.Storage;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// Shared 2023 R2 gRPC authentication handshake. Opens an authenticated History session over an
/// existing <see cref="HistorianGrpcConnection"/> and returns the transient client handle used by
/// the Retrieval/Status services. Extracted from <see cref="HistorianGrpcReadOrchestrator"/> so the
/// read, status, and (future) browse/metadata gRPC paths all drive the identical chain:
/// <c>HistoryService.GetInterfaceVersion → StorageService.ValidateClientCredential (token loop) →
/// HistoryService.OpenConnection</c>. The byte payloads (OpenConnection3 v6 request, NTLM token
/// framing) are the proven 2020 protocol and transfer unchanged inside protobuf <c>bytes</c> fields.
///
/// See <see cref="HistorianGrpcReadOrchestrator"/> for the op-routing rationale (the Negotiate loop
/// belongs on StorageService.ValidateClientCredential, NOT HistoryService.ExchangeKey).
/// </summary>
internal static class HistorianGrpcHandshake
{
/// <summary>Diagnostic: hex of the most recent v8 event-connection OpenConnection request.</summary>
internal static string LastEventOpenRequestHex { get; private set; } = string.Empty;
/// <summary>
/// The handles produced by a successful OpenConnection. <see cref="ClientHandle"/> is the
/// transient <c>uint</c> session token used by StartQuery/GetSystemParameter and the other
/// uint-handle ops. <see cref="StorageSessionId"/> is the storage-session GUID used (formatted
/// uppercase via <see cref="StringHandle"/>) by the string-handle ops
/// (GetTagInfosFromName, GetTagExtendedPropertiesFromName, ExecuteSqlCommand, ...).
/// </summary>
internal readonly record struct Session(uint ClientHandle, Guid StorageSessionId)
{
/// <summary>The storage GUID in the uppercase "D" form the native string-handle ops require.</summary>
public string StringHandle => StorageSessionId.ToString("D").ToUpperInvariant();
}
/// <summary>Convenience overload for callers that only need the uint client handle.</summary>
public static uint OpenAuthenticatedConnection(
HistorianGrpcConnection connection,
HistorianClientOptions options,
CancellationToken cancellationToken)
=> OpenSession(connection, options, cancellationToken).ClientHandle;
/// <param name="connectionMode">
/// The native Open2 connection mode. Defaults to read-only (<c>0x402</c>); pass
/// <see cref="HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode"/>
/// (<c>0x401</c>) for write-enabled sessions (e.g. the non-streamed/revision Transaction path,
/// which the read-only mode silently rejects with err 132 OperationNotEnabled).
/// </param>
public static Session OpenSession(
HistorianGrpcConnection connection,
HistorianClientOptions options,
CancellationToken cancellationToken,
uint connectionMode = HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode,
bool eventConnection = false)
{
DateTime Deadline() => DateTime.UtcNow.Add(options.RequestTimeout);
Guid contextKey = Guid.NewGuid();
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
GrpcHistory.GetInterfaceVersionResponse historyVersion = historyClient.GetInterfaceVersion(
new GrpcHistory.GetInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken);
HistorianServerVersionGate.Validate(HistorianServiceInterface.History, historyVersion.UiVersion, options);
// The v6 (read/write) path authenticates via StorageService.ValidateClientCredential (Negotiate).
// The v8 EVENT path authenticates entirely via ExchangeKey (ECDH) + the RC4 credential token —
// the native client does NOT run ValidateClientCredential for an event connection, and doing so
// establishes a different session scope under which the event query returns zero rows. So skip it.
if (!eventConnection)
{
var storageClient = new GrpcStorage.StorageService.StorageServiceClient(connection.Channel);
HistorianNativeHandshake.RunTokenRounds(
(handle, wrapped, _) =>
{
GrpcStorage.ValidateClientCredentialResponse response = storageClient.ValidateClientCredential(
new GrpcStorage.ValidateClientCredentialRequest { Handle = handle, InBuff = ByteString.CopyFrom(wrapped) },
connection.Metadata,
Deadline(),
cancellationToken);
byte[] serverOutput = response.OutBuff?.ToByteArray() ?? [];
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
bool success = response.Status?.BSuccess ?? false;
return new HistorianNativeHandshake.TokenExchangeResult(success, serverOutput, error);
},
contextKey,
options,
cancellationToken);
}
// Event reads require an Event-type connection (ConnectionType=Event), which only the native
// v8 OpenConnection format carries — the v6 buffer has no such field. The v8 path authenticates
// via HistoryService.ExchangeKey (P-256 ECDH): the shared secret -> SHA256 = the client key, and
// the v8 credential token = RC4(password-UTF16LE, key=MD5(clientKey)) (the native HistorianCrypto
// aahCryptV2 scheme). The server shares the secret and RC4-decrypts the token to validate the
// password. See docs/reverse-engineering/grpc-event-query-capture.md.
byte[] eventToken = [];
if (eventConnection)
{
using ECDiffieHellman ecdh = ECDiffieHellman.Create(ECCurve.NamedCurves.nistP256);
byte[] clientHello = HistorianNativeHandshake.BuildExchangeKeyClientHello(ecdh);
string xkHandle = contextKey.ToString("D").ToUpperInvariant();
GrpcHistory.ExchangeKeyResponse xk = historyClient.ExchangeKey(
new GrpcHistory.ExchangeKeyRequest { StrHandle = xkHandle, BtInput = ByteString.CopyFrom(clientHello) },
connection.Metadata,
Deadline(),
cancellationToken);
if (!(xk.Status?.BSuccess ?? false))
{
byte[] xkErr = xk.Status?.BtError?.ToByteArray() ?? [];
HistorianNativeError? xkDecoded = HistorianOpen2Protocol.TryReadNativeError(xkErr);
string xkAscii = new(xkErr.Where(b => b is >= 0x20 and < 0x7F).Select(b => (char)b).ToArray());
throw new InvalidOperationException(
$"gRPC ExchangeKey failed (errorLen={xkErr.Length}, native={xkDecoded?.Type}/{xkDecoded?.Code}, ascii='{xkAscii}').");
}
byte[] clientKey = HistorianNativeHandshake.DeriveExchangeKeyClientKey(ecdh, xk.BtOutput?.ToByteArray() ?? []);
eventToken = HistorianNativeHandshake.BuildExchangeKeyCredentialToken(clientKey, options.Password);
}
byte[] open2Request = eventConnection
? HistorianNativeHandshake.BuildEventOpenConnectionVersion8Request(contextKey, options.UserName, eventToken)
: HistorianNativeHandshake.BuildOpenConnection3Request(options.Host, contextKey, connectionMode);
if (eventConnection) { LastEventOpenRequestHex = Convert.ToHexString(open2Request); }
GrpcHistory.OpenConnectionResponse open2 = historyClient.OpenConnection(
new GrpcHistory.OpenConnectionRequest { BtConnectionRequest = ByteString.CopyFrom(open2Request) },
connection.Metadata,
Deadline(),
cancellationToken);
byte[] open2Response = open2.BtConnectionResponse?.ToByteArray() ?? [];
if (!(open2.Status?.BSuccess ?? false))
{
byte[] err = open2.Status?.BtError?.ToByteArray() ?? [];
HistorianNativeError? decoded = HistorianOpen2Protocol.TryReadNativeError(err);
string ascii = new(err.Where(b => b is >= 0x20 and < 0x7F).Select(b => (char)b).ToArray());
throw new InvalidOperationException(
$"gRPC OpenConnection failed (errorLen={err.Length}, responseLen={open2Response.Length}, " +
$"native={decoded?.Type}/{decoded?.Code}{(decoded?.Name is { } n ? $" {n}" : "")}, ascii='{ascii}').");
}
(uint clientHandle, Guid storageSessionId) = HistorianNativeHandshake.ParseOpenConnectionResponse(open2Response);
return new Session(clientHandle, storageSessionId);
}
}
@@ -0,0 +1,130 @@
using Google.Protobuf;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
using GrpcHistory = ArchestrA.Grpc.Contract.History;
using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// 2023 R2 gRPC orchestrator for the M3 historical (non-streamed original / backfill) value write.
/// Captured live from the native client (see <c>docs/plans/revision-write-path.md</c> §"R3.1
/// CAPTURED"): the historical write rides <c>HistoryService.AddStreamValues</c> with an "ON"
/// storage-sample buffer (<see cref="HistorianHistoricalWriteProtocol"/>), NOT the TransactionService
/// <c>AddNonStreamValues</c> path. The chain on a single write-enabled (<c>0x401</c>) session:
/// <list type="number">
/// <item>OpenConnection (write-enabled) → string storage handle</item>
/// <item><c>RetrievalService.GetTagInfosFromName</c> → the per-tag GUID (parsed as the tag-info
/// record's <c>TypeId</c>) and registers the tag on the session</item>
/// <item><c>HistoryService.AddStreamValues</c>(strHandle, "ON" buffer) per sample</item>
/// </list>
/// The tag must already exist (create it with <c>EnsureTagAsync</c> first). Only the Float value
/// encoding is captured; other tag types are rejected by the serializer until captured.
/// </summary>
internal sealed class HistorianGrpcHistoricalWriteOrchestrator
{
private readonly HistorianClientOptions _options;
public HistorianGrpcHistoricalWriteOrchestrator(HistorianClientOptions options)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
}
public Task<bool> AddHistoricalValuesAsync(
string tag,
IReadOnlyList<HistorianHistoricalValue> values,
CancellationToken cancellationToken)
=> Task.Run(() => Run(tag, values, cancellationToken), cancellationToken);
private bool Run(string tag, IReadOnlyList<HistorianHistoricalValue> values, CancellationToken cancellationToken)
{
if (values.Count == 0)
{
return true;
}
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(
connection, _options, cancellationToken,
connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode);
return RunWriteOnSession(connection, session, tag, values, cancellationToken);
}
// Spike/Phase-1 seam (A1): run the historical write against an EXTERNALLY-supplied, already-
// authenticated write-enabled (0x401) connection + session — NO Create()/handshake here. Run()
// delegates so the per-call path and the reuse path share one write implementation (DRY).
internal bool RunWriteOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
string tag,
IReadOnlyList<HistorianHistoricalValue> values,
CancellationToken cancellationToken)
{
if (values.Count == 0)
{
return true;
}
string handle = session.StringHandle;
DateTime Deadline() => DateTime.UtcNow.Add(_options.RequestTimeout);
// Resolve the per-tag GUID (and register the tag on this write session) via
// GetTagInfosFromName. The 16-byte GUID the "ON" buffer needs is the tag-info record's TypeId.
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
GrpcRetrieval.GetTagInfosFromNameResponse tagInfoResponse = retrievalClient.GetTagInfosFromName(
new GrpcRetrieval.GetTagInfosFromNameRequest
{
StrHandle = handle,
BtTagNames = ByteString.CopyFrom(HistorianGrpcTagClient.BuildTagNamesBuffer([tag])),
UiSequence = 0,
},
connection.Metadata, Deadline(), cancellationToken);
if (!(tagInfoResponse.Status?.BSuccess ?? false))
{
byte[] error = tagInfoResponse.Status?.BtError?.ToByteArray() ?? [];
throw new InvalidOperationException(
$"gRPC GetTagInfosFromName failed for tag '{tag}' (errorLen={error.Length}); does the tag exist?");
}
byte[] tagInfos = tagInfoResponse.BtTagInfos?.ToByteArray() ?? [];
IReadOnlyList<HistorianTagInfoResponse> parsed = HistorianTagQueryProtocol.ParseGetTagInfoResponse(tagInfos);
if (parsed.Count == 0)
{
throw new InvalidOperationException($"Tag '{tag}' not found on the server.");
}
Guid tagGuid = parsed[0].TypeId;
HistorianDataType dataType = HistorianWcfTagClient.MapDataType(parsed[0].NativeDataTypeDescriptor);
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
foreach (HistorianHistoricalValue value in values)
{
cancellationToken.ThrowIfCancellationRequested();
byte[] buffer = HistorianHistoricalWriteProtocol.SerializeAddStreamValuesBuffer(
tagGuid,
dataType,
value.TimestampUtc,
value.Value,
DateTime.UtcNow,
value.OpcQuality);
GrpcHistory.AddStreamValuesResponse response = historyClient.AddStreamValues(
new GrpcHistory.AddStreamValuesRequest
{
StrHandle = handle,
BtValues = ByteString.CopyFrom(buffer),
},
connection.Metadata, Deadline(), cancellationToken);
if (!(response.Status?.BSuccess ?? false))
{
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
throw new InvalidOperationException(
$"gRPC AddStreamValues failed for tag '{tag}' (errorLen={error.Length}).");
}
}
return true;
}
}
@@ -0,0 +1,48 @@
using Grpc.Core;
using GrpcHistory = ArchestrA.Grpc.Contract.History;
using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval;
using GrpcStatus = ArchestrA.Grpc.Contract.Status;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// 2023 R2 gRPC reachability probe (roadmap item R0.4). Mirrors <see cref="Wcf.HistorianWcfProbe"/>
/// over the gRPC transport: it calls the unauthenticated <c>GetInterfaceVersion</c> RPC on the
/// History, Retrieval, and Status services and applies the same success criteria. No credentials
/// are required — these RPCs run before the SSPI/Negotiate token loop — so the probe works even
/// when authentication is unavailable.
/// </summary>
internal static class HistorianGrpcProbe
{
public static async Task<bool> ProbeAsync(HistorianClientOptions options, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(options);
cancellationToken.ThrowIfCancellationRequested();
return await Task.Run(() => Probe(options, cancellationToken), cancellationToken).ConfigureAwait(false);
}
private static bool Probe(HistorianClientOptions options, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
DateTime deadline = DateTime.UtcNow.Add(options.ConnectTimeout > TimeSpan.Zero ? options.ConnectTimeout : TimeSpan.FromSeconds(5));
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
GrpcHistory.GetInterfaceVersionResponse history = historyClient.GetInterfaceVersion(
new GrpcHistory.GetInterfaceVersionRequest(), connection.Metadata, deadline, cancellationToken);
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
GrpcRetrieval.GetRetrievalInterfaceVersionResponse retrieval = retrievalClient.GetRetrievalInterfaceVersion(
new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), connection.Metadata, deadline, cancellationToken);
var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel);
GrpcStatus.GetStatusInterfaceVersionResponse status = statusClient.GetStatusInterfaceVersion(
new GrpcStatus.GetStatusInterfaceVersionRequest(), connection.Metadata, deadline, cancellationToken);
return history.UiError == 0
&& history.UiVersion > 0
&& retrieval.UiError == 0
&& retrieval.UiVersion > 0
&& status.UiError == 0;
}
}
@@ -0,0 +1,382 @@
using System.Runtime.CompilerServices;
using Google.Protobuf;
using Grpc.Core;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// 2023 R2 gRPC read orchestrator. Mirrors <see cref="HistorianWcfReadOrchestrator"/> over the
/// gRPC transport: the same native binary buffers travel inside protobuf <c>bytes</c> fields,
/// and the same serializers/parsers (<see cref="HistorianNativeHandshake"/>,
/// <see cref="HistorianDataQueryProtocol"/>) are reused unchanged.
///
/// Operation mapping (2020 WCF → 2023 R2 gRPC):
/// Hist.GetInterfaceVersion → HistoryService.GetInterfaceVersion
/// Hist.ValidateClientCredential (loop) → StorageService.ValidateClientCredential (loop)
/// Hist.OpenConnection2 → HistoryService.OpenConnection
/// Retr.StartQuery2 → RetrievalService.StartQuery
/// Retr.GetNextQueryResultBuffer2 (loop) → RetrievalService.GetNextQueryResultBuffer (loop)
/// Retr.EndQuery2 → RetrievalService.EndQuery
///
/// LIVE-VERIFIED 2026-06-21 against a real 2023 R2 server (interface versions: History=12,
/// Retrieval=4, Storage=4). The SSPI/Negotiate token loop maps to
/// <c>StorageService.ValidateClientCredential(Handle, InBuff)→(status, OutBuff)</c> — the op that
/// kept the 2020 inBuff/outBuff token framing. The gRPC HistoryService dropped
/// ValidateClientCredential and gained <c>ExchangeKey</c>, but ExchangeKey is a SEPARATE
/// key-exchange/cert-path op, NOT the Negotiate loop: feeding it an NTLM token is rejected at
/// round 0 regardless of credentials. An earlier revision wrongly routed the loop to ExchangeKey;
/// routing it to StorageService.ValidateClientCredential completes the full read chain. The byte
/// payloads (OpenConnection3 v6, token framing, DataQueryRequest, row buffers) are the proven 2020
/// protocol and transfer unchanged — only the History interface integer differs (12 vs the 2020
/// value 11), and that version is buffer-compatible (a live read returns rows).
/// </summary>
internal sealed class HistorianGrpcReadOrchestrator
{
private const ushort StartQueryRequestType = HistorianDataQueryProtocol.QueryRequestTypeData;
private readonly HistorianClientOptions _options;
public HistorianGrpcReadOrchestrator(HistorianClientOptions options)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
}
public async IAsyncEnumerable<HistorianSample> ReadRawAsync(
string tag,
DateTime startUtc,
DateTime endUtc,
int maxValues,
[EnumeratorCancellation] CancellationToken cancellationToken)
{
ValidateAuth();
cancellationToken.ThrowIfCancellationRequested();
IReadOnlyList<HistorianSample> rows = await Task.Run(
() => RunRawChain(tag, startUtc, endUtc, maxValues, cancellationToken), cancellationToken).ConfigureAwait(false);
foreach (HistorianSample sample in rows)
{
cancellationToken.ThrowIfCancellationRequested();
yield return sample;
}
}
public async IAsyncEnumerable<HistorianAggregateSample> ReadAggregateAsync(
string tag,
DateTime startUtc,
DateTime endUtc,
RetrievalMode mode,
TimeSpan interval,
[EnumeratorCancellation] CancellationToken cancellationToken)
{
ValidateAuth();
cancellationToken.ThrowIfCancellationRequested();
IReadOnlyList<HistorianAggregateSample> rows = await Task.Run(
() => RunAggregateChain(tag, startUtc, endUtc, mode, interval, cancellationToken), cancellationToken).ConfigureAwait(false);
foreach (HistorianAggregateSample sample in rows)
{
cancellationToken.ThrowIfCancellationRequested();
yield return sample;
}
}
public Task<IReadOnlyList<HistorianSample>> ReadAtTimeAsync(
string tag,
IReadOnlyList<DateTime> timestampsUtc,
CancellationToken cancellationToken)
{
ValidateAuth();
cancellationToken.ThrowIfCancellationRequested();
return Task.Run<IReadOnlyList<HistorianSample>>(() => RunAtTimeChain(tag, timestampsUtc, cancellationToken), cancellationToken);
}
private void ValidateAuth()
{
if (!_options.IntegratedSecurity && string.IsNullOrEmpty(_options.UserName))
{
throw new ProtocolEvidenceMissingException(
"Managed gRPC read flow currently requires IntegratedSecurity or an explicit UserName + Password.");
}
}
// Spike/Phase-1 seam (pending.md A1): run a raw query against an EXTERNALLY-supplied, already-
// authenticated connection + client handle — i.e. NO Create()/handshake here. RunRawChain delegates
// to this so the per-call path and the reuse path share one query implementation (DRY). The handshake
// reuse-probe test drives this directly to measure whether the server honors a reused session.
internal List<HistorianSample> RunRawQueryOnSession(
HistorianGrpcConnection connection,
uint clientHandle,
string tag,
DateTime startUtc,
DateTime endUtc,
int maxValues,
CancellationToken cancellationToken)
{
HistorianDataQueryRequest request = HistorianWcfReadOrchestrator.BuildDataQueryRequest(tag, startUtc, endUtc, maxValues);
return RunQuery(connection, clientHandle, request, maxValues, cancellationToken);
}
private List<HistorianSample> RunRawChain(string tag, DateTime startUtc, DateTime endUtc, int maxValues, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
uint clientHandle = OpenAuthenticatedConnection(connection, cancellationToken);
return RunRawQueryOnSession(connection, clientHandle, tag, startUtc, endUtc, maxValues, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): run an aggregate query against an EXTERNALLY-supplied,
// already-authenticated connection + client handle — i.e. NO Create()/handshake here.
// RunAggregateChain delegates to this so the per-call path and the reuse path share one query
// implementation (DRY).
internal List<HistorianAggregateSample> RunAggregateQueryOnSession(
HistorianGrpcConnection connection,
uint clientHandle,
string tag,
DateTime startUtc,
DateTime endUtc,
RetrievalMode mode,
TimeSpan interval,
CancellationToken ct)
{
return RunAggregateQuery(connection, clientHandle, tag, startUtc, endUtc, mode, interval, ct);
}
private List<HistorianAggregateSample> RunAggregateChain(
string tag, DateTime startUtc, DateTime endUtc, RetrievalMode mode, TimeSpan interval, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
uint clientHandle = OpenAuthenticatedConnection(connection, cancellationToken);
return RunAggregateQueryOnSession(connection, clientHandle, tag, startUtc, endUtc, mode, interval, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): run an at-time query against an EXTERNALLY-supplied,
// already-authenticated connection + client handle — i.e. NO Create()/handshake here.
// RunAtTimeChain delegates to this so the per-call path and the reuse path share one
// implementation (DRY).
internal List<HistorianSample> RunAtTimeOnSession(
HistorianGrpcConnection connection,
uint clientHandle,
string tag,
IReadOnlyList<DateTime> timestampsUtc,
CancellationToken ct)
{
if (timestampsUtc.Count == 0)
{
return [];
}
List<HistorianSample> results = new(timestampsUtc.Count);
foreach (DateTime ts in timestampsUtc)
{
ct.ThrowIfCancellationRequested();
DateTime tsUtc = ts.ToUniversalTime();
List<HistorianAggregateSample> aggregates = RunAggregateQuery(
connection,
clientHandle,
tag,
tsUtc - TimeSpan.FromTicks(1),
tsUtc + TimeSpan.FromTicks(1),
RetrievalMode.Interpolated,
TimeSpan.FromTicks(2),
ct);
if (aggregates.Count == 0)
{
continue;
}
HistorianAggregateSample chosen = aggregates[0];
results.Add(new HistorianSample(
TagName: chosen.TagName,
TimestampUtc: tsUtc,
NumericValue: chosen.Value,
StringValue: null,
Quality: chosen.Quality,
QualityDetail: chosen.QualityDetail,
OpcQuality: chosen.OpcQuality,
PercentGood: 100));
}
return results;
}
private List<HistorianSample> RunAtTimeChain(string tag, IReadOnlyList<DateTime> timestampsUtc, CancellationToken cancellationToken)
{
if (timestampsUtc.Count == 0)
{
return [];
}
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
uint clientHandle = OpenAuthenticatedConnection(connection, cancellationToken);
return RunAtTimeOnSession(connection, clientHandle, tag, timestampsUtc, cancellationToken);
}
private uint OpenAuthenticatedConnection(HistorianGrpcConnection connection, CancellationToken cancellationToken)
=> HistorianGrpcHandshake.OpenAuthenticatedConnection(connection, _options, cancellationToken);
private List<HistorianSample> RunQuery(
HistorianGrpcConnection connection,
uint clientHandle,
HistorianDataQueryRequest request,
int maxValues,
CancellationToken cancellationToken)
{
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
GrpcRetrieval.GetRetrievalInterfaceVersionResponse retrievalVersion = retrievalClient.GetRetrievalInterfaceVersion(
new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), null, Deadline(), cancellationToken);
HistorianServerVersionGate.Validate(HistorianServiceInterface.Retrieval, retrievalVersion.UiVersion, _options);
byte[] requestBuffer = HistorianDataQueryProtocol.SerializeFullHistoryRequest(request);
uint queryHandle = StartQuery(retrievalClient, clientHandle, requestBuffer, "raw", cancellationToken);
try
{
List<HistorianSample> samples = [];
while (true)
{
cancellationToken.ThrowIfCancellationRequested();
(byte[] resultBuffer, byte[] errorBuffer) = GetNextResultBuffer(retrievalClient, clientHandle, queryHandle, "raw", cancellationToken);
if (!HistorianDataQueryProtocol.TryParseGetNextQueryResultBufferRows(resultBuffer, errorBuffer, out IReadOnlyList<HistorianSample> rows, out bool hasMoreData))
{
throw new InvalidOperationException($"gRPC GetNextQueryResultBuffer returned an unparsable result buffer (length={resultBuffer.Length}).");
}
foreach (HistorianSample sample in rows)
{
samples.Add(sample);
if (samples.Count >= maxValues)
{
return samples;
}
}
if (!hasMoreData)
{
return samples;
}
}
}
finally
{
EndQuerySafely(retrievalClient, clientHandle, queryHandle);
}
}
private List<HistorianAggregateSample> RunAggregateQuery(
HistorianGrpcConnection connection,
uint clientHandle,
string tag,
DateTime startUtc,
DateTime endUtc,
RetrievalMode mode,
TimeSpan interval,
CancellationToken cancellationToken)
{
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
GrpcRetrieval.GetRetrievalInterfaceVersionResponse retrievalVersion = retrievalClient.GetRetrievalInterfaceVersion(
new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), null, Deadline(), cancellationToken);
HistorianServerVersionGate.Validate(HistorianServiceInterface.Retrieval, retrievalVersion.UiVersion, _options);
HistorianDataQueryRequest request = HistorianWcfReadOrchestrator.BuildAggregateQueryRequest(tag, startUtc, endUtc, mode, interval);
byte[] requestBuffer = HistorianDataQueryProtocol.SerializeFullHistoryRequest(request);
uint queryHandle = StartQuery(retrievalClient, clientHandle, requestBuffer, $"aggregate {mode}", cancellationToken);
try
{
List<HistorianAggregateSample> samples = [];
while (true)
{
cancellationToken.ThrowIfCancellationRequested();
(byte[] resultBuffer, byte[] errorBuffer) = GetNextResultBuffer(retrievalClient, clientHandle, queryHandle, $"aggregate {mode}", cancellationToken);
if (!HistorianDataQueryProtocol.TryParseGetNextQueryResultBufferAggregateRows(
resultBuffer, errorBuffer, mode, interval, out IReadOnlyList<HistorianAggregateSample> rows, out bool hasMoreData))
{
throw new InvalidOperationException($"gRPC GetNextQueryResultBuffer (aggregate {mode}) returned an unparsable buffer (length={resultBuffer.Length}).");
}
samples.AddRange(rows);
if (!hasMoreData)
{
return samples;
}
}
}
finally
{
EndQuerySafely(retrievalClient, clientHandle, queryHandle);
}
}
private uint StartQuery(
GrpcRetrieval.RetrievalService.RetrievalServiceClient client,
uint clientHandle,
byte[] requestBuffer,
string label,
CancellationToken cancellationToken)
{
GrpcRetrieval.StartQueryResponse response = client.StartQuery(
new GrpcRetrieval.StartQueryRequest
{
UiHandle = clientHandle,
UiQueryRequestType = StartQueryRequestType,
BtRequestBuffer = ByteString.CopyFrom(requestBuffer)
},
null,
Deadline(),
cancellationToken);
if (!(response.Status?.BSuccess ?? false))
{
byte[] err = response.Status?.BtError?.ToByteArray() ?? [];
throw new InvalidOperationException($"gRPC StartQuery ({label}) failed (errorLen={err.Length}).");
}
return response.UiQueryHandle;
}
private (byte[] ResultBuffer, byte[] ErrorBuffer) GetNextResultBuffer(
GrpcRetrieval.RetrievalService.RetrievalServiceClient client,
uint clientHandle,
uint queryHandle,
string label,
CancellationToken cancellationToken)
{
GrpcRetrieval.GetNextQueryResultBufferResponse response = client.GetNextQueryResultBuffer(
new GrpcRetrieval.GetNextQueryResultBufferRequest { UiHandle = clientHandle, UiQueryHandle = queryHandle },
null,
Deadline(),
cancellationToken);
byte[] errorBuffer = response.Status?.BtError?.ToByteArray() ?? [];
if (!(response.Status?.BSuccess ?? false))
{
throw new InvalidOperationException($"gRPC GetNextQueryResultBuffer ({label}) failed (errorLen={errorBuffer.Length}).");
}
byte[] resultBuffer = response.BtQueryResult?.ToByteArray() ?? [];
return (resultBuffer, errorBuffer);
}
private void EndQuerySafely(GrpcRetrieval.RetrievalService.RetrievalServiceClient client, uint clientHandle, uint queryHandle)
{
try
{
client.EndQuery(
new GrpcRetrieval.EndQueryRequest { UiHandle = clientHandle, UiQueryHandle = queryHandle },
null,
Deadline(),
CancellationToken.None);
}
catch
{
// Best-effort cleanup; the read result is already collected.
}
}
private DateTime Deadline() => DateTime.UtcNow.Add(_options.RequestTimeout);
}
@@ -0,0 +1,279 @@
using Google.Protobuf;
using AVEVA.Historian.Client.Wcf;
using GrpcTransaction = ArchestrA.Grpc.Contract.Transaction;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// Live probe for the M3 (historical / non-streamed original-value write) path over the 2023 R2
/// gRPC front door. On 2020 WCF this op group is architecturally blocked: the
/// <c>ITransactionServiceContract2.AddNonStreamValuesBegin2</c> relay returns
/// <c>UnknownClient (51)</c> because it requires a pre-existing storage-engine pipe session
/// (<c>STransactPipeClient2</c> → <c>aaStorageEngine.exe</c>) that no WCF op can establish — see
/// <c>docs/plans/revision-write-path.md</c> (the D2 finding).
///
/// The 2023 R2 decompile shows the native gRPC client driving the SAME op group over
/// <c>TransactionService.AddNonStreamValuesBegin/AddNonStreamValues/AddNonStreamValuesEnd</c> and
/// passing the HistoryService Open2 session GUID directly as <c>strHandle</c> — i.e. the gRPC
/// server is the gateway to the storage engine, so the client never touches the legacy pipe. This
/// probe tests whether the SDK's pure-managed handshake can reproduce that: it opens a
/// write-enabled session and calls <c>AddNonStreamValuesBegin</c>, surfacing whatever the server
/// returns. It writes NO data — if Begin succeeds it immediately calls <c>AddNonStreamValuesEnd</c>
/// with <c>bCommit=false</c> to discard the transaction.
///
/// <para><b>Scope note (corrected 2026-06-23 after a 2023 R2 binary re-read).</b> Despite the type
/// name, this probes the <i>non-streamed ORIGINAL / backfill insert</i> capability
/// (<c>AddNonStreamValues</c>), which is a <b>distinct capability from a revision EDIT</b>
/// (overwriting an existing historized value with a new revision). The stock high-level client
/// reaches a revision edit via a separate native transaction trio
/// <c>HistorianClient.AddRevisionValuesBegin/AddRevisionValue/AddRevisionValuesEnd</c>
/// (<c>ArchestrA.HistorianAccess.AddRevisionValues</c>, REVISION_MODE ∈ InsertLatest/UpdateSingle/
/// UpdateMultiple). That trio has <b>NO corresponding RPC in the gRPC contract</b> (no "Revision"
/// message type exists in <c>Archestra.Grpc.Contract</c>) — it rides the native storage-engine
/// transaction channel only. So R4.2 revision edits remain unreachable over gRPC regardless of this
/// probe's outcome; this probe neither covers nor unblocks them.</para>
/// </summary>
internal sealed class HistorianGrpcRevisionProbe
{
private readonly HistorianClientOptions _options;
public HistorianGrpcRevisionProbe(HistorianClientOptions options)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
}
public Task<HistorianGrpcRevisionProbeResult> ProbeBeginAsync(CancellationToken cancellationToken)
=> Task.Run(() => ProbeBegin(cancellationToken), cancellationToken);
/// <summary>
/// Empirical-decode driver for the <c>AddNonStreamValues</c> <c>btInput</c> buffer (R3.1). For
/// each candidate buffer it opens a fresh transaction, sends the buffer, records the server's
/// accept/reject, and ALWAYS ends with <c>bCommit=false</c> (rollback) so nothing persists.
/// The candidate buffers are supplied by the caller (the RE tool) — this method does not invent
/// wire bytes, it just reports what the live server says about each. Safe against a real tag key
/// because every transaction is discarded.
/// </summary>
public Task<IReadOnlyList<HistorianGrpcNonStreamedCandidateResult>> ProbeNonStreamedBuffersAsync(
IReadOnlyList<(string Label, byte[] Buffer)> candidates,
CancellationToken cancellationToken)
=> Task.Run<IReadOnlyList<HistorianGrpcNonStreamedCandidateResult>>(
() => ProbeNonStreamedBuffers(candidates, cancellationToken), cancellationToken);
private List<HistorianGrpcNonStreamedCandidateResult> ProbeNonStreamedBuffers(
IReadOnlyList<(string Label, byte[] Buffer)> candidates,
CancellationToken cancellationToken)
{
var results = new List<HistorianGrpcNonStreamedCandidateResult>();
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(
connection, _options, cancellationToken,
connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode);
var transactionClient = new GrpcTransaction.TransactionService.TransactionServiceClient(connection.Channel);
string handle = session.StringHandle;
DateTime Deadline() => DateTime.UtcNow.Add(_options.RequestTimeout);
// Prime the Transaction service session table.
try
{
transactionClient.GetTransactionInterfaceVersion(
new GrpcTransaction.GetTransactionInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken);
}
catch { /* version prime is best-effort */ }
foreach ((string label, byte[] buffer) in candidates)
{
var candidate = new HistorianGrpcNonStreamedCandidateResult { Label = label, BufferLength = buffer.Length };
string? transactionId = null;
try
{
GrpcTransaction.AddNonStreamValuesBeginResponse begin = transactionClient.AddNonStreamValuesBegin(
new GrpcTransaction.AddNonStreamValuesBeginRequest { StrHandle = handle },
connection.Metadata, Deadline(), cancellationToken);
if (!(begin.Status?.BSuccess ?? false) || string.IsNullOrEmpty(begin.StrTransactionId))
{
candidate.BeginFailed = true;
byte[] be = begin.Status?.BtError?.ToByteArray() ?? [];
candidate.AddErrorHex = be.Length == 0 ? null : Convert.ToHexString(be);
results.Add(candidate);
continue;
}
transactionId = begin.StrTransactionId;
GrpcTransaction.AddNonStreamValuesResponse add = transactionClient.AddNonStreamValues(
new GrpcTransaction.AddNonStreamValuesRequest
{
StrHandle = handle,
StrTransactionId = transactionId,
BtInput = ByteString.CopyFrom(buffer),
},
connection.Metadata, Deadline(), cancellationToken);
candidate.AddSucceeded = add.Status?.BSuccess ?? false;
byte[] ae = add.Status?.BtError?.ToByteArray() ?? [];
candidate.AddErrorHex = ae.Length == 0 ? null : Convert.ToHexString(ae);
}
catch (Exception ex)
{
candidate.Exception = $"{ex.GetType().Name}: {ex.Message}";
}
finally
{
// Always roll back — bCommit=false writes nothing.
if (!string.IsNullOrEmpty(transactionId))
{
try
{
transactionClient.AddNonStreamValuesEnd(
new GrpcTransaction.AddNonStreamValuesEndRequest
{
StrHandle = handle,
StrTransactionId = transactionId,
BCommit = false,
},
connection.Metadata, Deadline(), cancellationToken);
}
catch { /* rollback best-effort */ }
}
}
results.Add(candidate);
}
return results;
}
private HistorianGrpcRevisionProbeResult ProbeBegin(CancellationToken cancellationToken)
{
var result = new HistorianGrpcRevisionProbeResult();
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(
connection,
_options,
cancellationToken,
connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode);
result.OpenSucceeded = true;
result.ClientHandle = session.ClientHandle;
result.StorageSessionId = session.StorageSessionId;
var transactionClient = new GrpcTransaction.TransactionService.TransactionServiceClient(connection.Channel);
DateTime Deadline() => DateTime.UtcNow.Add(_options.RequestTimeout);
// Register the client with the Transaction service's session table (matches the
// cross-service GetV priming the WCF write path uses).
try
{
GrpcTransaction.GetTransactionInterfaceVersionResponse version = transactionClient.GetTransactionInterfaceVersion(
new GrpcTransaction.GetTransactionInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken);
result.TrxInterfaceVersionError = version.Error;
result.TrxInterfaceVersion = version.Version;
}
catch (Exception ex)
{
result.TrxInterfaceVersionException = $"{ex.GetType().Name}: {ex.Message}";
}
// The decompiled native client passes the Open2 storage-session GUID (string) as strHandle.
// Try that first (uppercase "D" form, as the other string-handle ops require), then a couple
// of fallbacks mirroring the WCF probe, so a wrong-format rejection is distinguishable from a
// genuine server-side block.
foreach ((string label, string handle) in new[]
{
("storageSessionId-upper", session.StringHandle),
("storageSessionId-lower", session.StorageSessionId.ToString("D")),
("clientHandle-as-string", session.ClientHandle.ToString()),
})
{
var attempt = new HistorianGrpcRevisionBeginAttempt { HandleLabel = label, HandleSent = handle };
try
{
GrpcTransaction.AddNonStreamValuesBeginResponse begin = transactionClient.AddNonStreamValuesBegin(
new GrpcTransaction.AddNonStreamValuesBeginRequest { StrHandle = handle },
connection.Metadata, Deadline(), cancellationToken);
attempt.Succeeded = begin.Status?.BSuccess ?? false;
attempt.TransactionId = begin.StrTransactionId;
byte[] error = begin.Status?.BtError?.ToByteArray() ?? [];
attempt.ErrorHex = error.Length == 0 ? null : Convert.ToHexString(error);
result.BeginAttempts.Add(attempt);
if (attempt.Succeeded && !string.IsNullOrEmpty(attempt.TransactionId))
{
result.BeginSucceeded = true;
result.BeginTransactionId = attempt.TransactionId;
// Discard immediately — bCommit=false writes nothing. This keeps the probe
// read-only against the live (production) server.
try
{
GrpcTransaction.AddNonStreamValuesEndResponse end = transactionClient.AddNonStreamValuesEnd(
new GrpcTransaction.AddNonStreamValuesEndRequest
{
StrHandle = handle,
StrTransactionId = attempt.TransactionId,
BCommit = false,
},
connection.Metadata, Deadline(), cancellationToken);
result.EndDiscardSucceeded = end.Status?.BSuccess ?? false;
byte[] endError = end.Status?.BtError?.ToByteArray() ?? [];
result.EndDiscardErrorHex = endError.Length == 0 ? null : Convert.ToHexString(endError);
}
catch (Exception ex)
{
result.EndDiscardException = $"{ex.GetType().Name}: {ex.Message}";
}
break;
}
}
catch (Exception ex)
{
attempt.Exception = $"{ex.GetType().Name}: {ex.Message}";
result.BeginAttempts.Add(attempt);
}
}
return result;
}
}
internal sealed class HistorianGrpcRevisionProbeResult
{
public bool OpenSucceeded { get; set; }
public uint ClientHandle { get; set; }
public Guid StorageSessionId { get; set; }
public uint? TrxInterfaceVersionError { get; set; }
public uint? TrxInterfaceVersion { get; set; }
public string? TrxInterfaceVersionException { get; set; }
public bool BeginSucceeded { get; set; }
public string? BeginTransactionId { get; set; }
public bool EndDiscardSucceeded { get; set; }
public string? EndDiscardErrorHex { get; set; }
public string? EndDiscardException { get; set; }
public List<HistorianGrpcRevisionBeginAttempt> BeginAttempts { get; } = new();
}
internal sealed class HistorianGrpcRevisionBeginAttempt
{
public string HandleLabel { get; set; } = "";
public string HandleSent { get; set; } = "";
public bool Succeeded { get; set; }
public string? TransactionId { get; set; }
public string? ErrorHex { get; set; }
public string? Exception { get; set; }
}
internal sealed class HistorianGrpcNonStreamedCandidateResult
{
public string Label { get; set; } = "";
public int BufferLength { get; set; }
public bool BeginFailed { get; set; }
public bool AddSucceeded { get; set; }
public string? AddErrorHex { get; set; }
public string? Exception { get; set; }
}
@@ -0,0 +1,118 @@
using Google.Protobuf;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Protocol;
using AVEVA.Historian.Client.Wcf;
using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// Executes SQL commands over the 2023 R2 gRPC transport (HCAL R1.1), mirroring
/// <see cref="HistorianWcfSqlClient"/>'s two-op <c>ExeC</c>/<c>GetR</c> flow. The 2020 WCF path uses a
/// dedicated <c>GetRecordSetByteStream</c> op; the gRPC front door has no such RPC, so the NRBF
/// recordset stream would be fetched through the generic <c>RetrievalService.GetNextQueryResultBuffer</c>
/// keyed by the query handle <c>ExecuteSqlCommand</c> returns. <c>ExecuteSqlCommand</c> takes the
/// uppercase string session handle; the result-buffer fetch takes the transient <c>uint</c> client
/// handle (both come from the one Open2 session).
/// <para>
/// <b>SERVER-WALLED (captured 2026-06-22).</b> The 2023 R2 front-door
/// <c>RetrievalService.ExecuteSqlCommand</c> faults server-side before returning a query handle:
/// the response carries native error 38 wrapping a managed
/// <c>System.IndexOutOfRangeException ... at aahClientAccessPoint.CSrvDbConnection.ExecuteSqlCommand</c>.
/// This is a server-side <c>CSrvDbConnection</c> (SQL DB-connection) precondition that the pure
/// managed gRPC session does not establish — the same class of wall as
/// <c>StorageService.OpenStorageConnection</c>. Priming <c>Retr.GetV</c> does not clear it, and
/// <b>a <c>HistoryService.RegisterTags</c> prime does NOT clear it either</b> (tried live 2026-06-22 on
/// both read-only <c>0x402</c> and write-enabled <c>0x401</c> sessions: <c>RegisterTags</c> itself
/// returned false and <c>ExecuteSqlCommand</c> faulted with the identical native-38 IndexOutOfRange) —
/// so unlike the OpenStorageConnection wall, the SQL DB-connection context is not established by the
/// RegisterTags family. The request framing here is the captured/expected shape; the op stays bounded
/// behind <see cref="ProtocolEvidenceMissingException"/> until the DB-connection registration is
/// reproduced. Use the WCF transport for SQL.
/// </para>
/// </summary>
internal static class HistorianGrpcSqlClient
{
// GetNextQueryResultBuffer is byte-stream-paged; a small record set returns in one page. Runaway guard.
private const int MaxPages = 4096;
public static Task<HistorianSqlResult> ExecuteSqlCommandAsync(
HistorianClientOptions options,
string command,
HistorianSqlExecuteOption option,
CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(command);
return Task.Run(() => ExecuteSqlCommand(options, command, option, cancellationToken), cancellationToken);
}
private static HistorianSqlResult ExecuteSqlCommand(
HistorianClientOptions options,
string command,
HistorianSqlExecuteOption option,
CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
DateTime Deadline() => DateTime.UtcNow.Add(options.RequestTimeout);
// Prime the Retrieval service version handshake (Retr.GetV) before the string-handle SQL op, as
// the native WCF SQL path does — the server-side ExecuteSqlCommand otherwise faults.
retrievalClient.GetRetrievalInterfaceVersion(
new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken);
GrpcRetrieval.ExecuteSqlCommandResponse exec = retrievalClient.ExecuteSqlCommand(
new GrpcRetrieval.ExecuteSqlCommandRequest
{
StrHandle = session.StringHandle,
StrCommand = command,
UiOption = (uint)option,
UiQueryHandle = 0
},
connection.Metadata,
Deadline(),
cancellationToken);
if (!(exec.Status?.BSuccess ?? false))
{
// Captured 2026-06-22: the server-side CSrvDbConnection.ExecuteSqlCommand throws
// IndexOutOfRange (native error 38) — a DB-connection precondition the pure managed gRPC
// session doesn't establish. Surface the SDK's evidence-missing signal rather than a raw
// server fault. See the class remarks.
throw new ProtocolEvidenceMissingException(
"ExecuteSqlCommand over gRPC: server-side CSrvDbConnection.ExecuteSqlCommand faults " +
"(IndexOutOfRange / native error 38) — an unmet DB-connection precondition (gRPC transport). Use WCF.");
}
int returnValue = exec.IRetValue;
uint queryHandle = exec.UiQueryHandle;
using MemoryStream accumulated = new();
for (int page = 0; page < MaxPages; page++)
{
cancellationToken.ThrowIfCancellationRequested();
GrpcRetrieval.GetNextQueryResultBufferResponse buffer = retrievalClient.GetNextQueryResultBuffer(
new GrpcRetrieval.GetNextQueryResultBufferRequest { UiHandle = session.ClientHandle, UiQueryHandle = queryHandle },
connection.Metadata,
Deadline(),
cancellationToken);
byte[] resultBuffer = buffer.BtQueryResult?.ToByteArray() ?? [];
// GetR is false-even-on-success: the final page returns false with the data still in the
// buffer, so always consume the buffer first, then stop on a false status or an empty page.
if (resultBuffer.Length > 0)
{
accumulated.Write(resultBuffer, 0, resultBuffer.Length);
}
if (!(buffer.Status?.BSuccess ?? false) || resultBuffer.Length == 0)
{
break;
}
}
return HistorianSqlResultProtocol.Parse(accumulated.ToArray(), returnValue);
}
}
@@ -0,0 +1,277 @@
using Google.Protobuf;
using Grpc.Core;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
using GrpcStatus = ArchestrA.Grpc.Contract.Status;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// 2023 R2 gRPC status client (roadmap item R0.3). Mirrors
/// <see cref="Wcf.HistorianWcfStatusClient"/> over the gRPC transport: it opens an authenticated
/// History session via <see cref="HistorianGrpcHandshake"/> and queries the StatusService for the
/// resulting client handle. <c>GetSystemParameter</c> carries the parameter name as a protobuf
/// string and returns the value string directly — there is no opaque native buffer to decode.
/// </summary>
internal static class HistorianGrpcStatusClient
{
public static Task<string?> GetSystemParameterAsync(
HistorianClientOptions options,
string parameterName,
CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(parameterName);
return Task.Run(() => GetSystemParameter(options, parameterName, cancellationToken), cancellationToken);
}
private static string? GetSystemParameter(HistorianClientOptions options, string parameterName, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
uint clientHandle = HistorianGrpcHandshake.OpenAuthenticatedConnection(connection, options, cancellationToken);
return GetSystemParameterOnSession(connection, clientHandle, options, parameterName, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): run GetSystemParameter against an EXTERNALLY-supplied,
// already-authenticated connection + client handle — NO Create()/handshake here. GetSystemParameter
// delegates so the per-call path and the reuse path share one RPC implementation (DRY).
internal static string? GetSystemParameterOnSession(
HistorianGrpcConnection connection,
uint clientHandle,
HistorianClientOptions options,
string parameterName,
CancellationToken cancellationToken)
{
var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel);
GrpcStatus.GetSystemParameterResponse response = statusClient.GetSystemParameter(
new GrpcStatus.GetSystemParameterRequest { UiHandle = clientHandle, StrParameterName = parameterName },
connection.Metadata,
DateTime.UtcNow.Add(options.RequestTimeout),
cancellationToken);
return (response.Status?.BSuccess ?? false) ? response.StrParameterValue : null;
}
/// <summary>
/// Returns a <em>measured</em> store-forward status over the 2023 R2 gRPC transport (R4.3).
/// <para>
/// Unlike the non-gRPC <see cref="Wcf.HistorianWcfStatusClient"/> path — which synthesizes an
/// all-false result <em>without contacting the server</em> — this opens an authenticated session
/// and calls <c>StatusService.GetHistorianConsoleStatus</c>, the only SF-adjacent signal reachable
/// from a pure managed client. (The direct <c>StorageService.GetSFParameter</c> /
/// <c>GetRemainingSnapshotsSize</c> RPCs that carry the SF buffer magnitude require the
/// <c>OpenStorageConnection</c> storage-engine console handle, which is gated behind the D2
/// storage-engine-pipe wall and is unobtainable here — see
/// <c>docs/plans/store-forward-cache-reverse-engineering.md</c> §9.7.)
/// </para>
/// <para>
/// Semantics: a successful console-status read means the server is reachable and its storage
/// console is reporting normally ⇒ the not-storing baseline (all flags false), but now
/// <em>measured</em> rather than blindly assumed. If the server cannot be reached/authenticated,
/// or the console-status call itself fails, <see cref="HistorianStoreForwardStatus.ErrorOccurred"/>
/// is set with the underlying error. The active-SF state (<see cref="HistorianStoreForwardStatus.Storing"/>
/// / <see cref="HistorianStoreForwardStatus.Pending"/> / <see cref="HistorianStoreForwardStatus.DataStored"/>
/// magnitude) is NOT observable from this signal and remains false; populating it requires the
/// D2-gated storage-console path.
/// </para>
/// </summary>
public static Task<HistorianStoreForwardStatus> GetStoreForwardStatusAsync(
HistorianClientOptions options,
CancellationToken cancellationToken)
=> Task.Run(() => GetStoreForwardStatus(options, cancellationToken), cancellationToken);
private static HistorianStoreForwardStatus GetStoreForwardStatus(HistorianClientOptions options, CancellationToken cancellationToken)
{
HistorianStoreForwardStatus NotStoring(bool errorOccurred, string? error) => new(
ServerName: options.Host,
Pending: false,
ErrorOccurred: errorOccurred,
Error: error,
DataStored: false,
Storing: false,
ConnectionKind: HistorianConnectionKind.Process);
try
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
return GetStoreForwardStatusOnSession(connection, session.StringHandle, options, NotStoring, cancellationToken);
}
catch (OperationCanceledException)
{
throw;
}
catch (Exception ex)
{
// Server unreachable / auth failed — genuinely measured: report it instead of a silent all-false.
return NotStoring(errorOccurred: true, error: $"{ex.GetType().Name}: {ex.Message}");
}
}
// Spike/Phase-1 seam (pending.md A1): run GetHistorianConsoleStatus against an EXTERNALLY-supplied,
// already-authenticated connection + string handle — NO Create()/handshake here. GetStoreForwardStatus
// delegates so the per-call path and the reuse path share one RPC implementation (DRY). The
// unreachable/auth-failure try/catch (which must also cover the handshake) stays with the per-call
// method; this seam runs only the RPC + result mapping against the supplied session.
internal static HistorianStoreForwardStatus GetStoreForwardStatusOnSession(
HistorianGrpcConnection connection,
string stringHandle,
HistorianClientOptions options,
Func<bool, string?, HistorianStoreForwardStatus> notStoring,
CancellationToken cancellationToken)
{
var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel);
GrpcStatus.GetHistorianConsoleStatusResponse response = statusClient.GetHistorianConsoleStatus(
new GrpcStatus.GetHistorianConsoleStatusRequest { StrHandle = stringHandle },
connection.Metadata,
DateTime.UtcNow.Add(options.RequestTimeout),
cancellationToken);
if (response.Status?.BSuccess ?? false)
{
// Measured: server reachable, storage console reporting normally → not-storing baseline.
return notStoring(false, null);
}
byte[] err = response.Status?.BtError?.ToByteArray() ?? [];
string detail = err.Length == 0 ? "GetHistorianConsoleStatus returned failure." : Convert.ToHexString(err);
return notStoring(true, $"GetHistorianConsoleStatus failed: {detail}");
}
/// <summary>
/// Returns a <em>measured</em> connection status over the 2023 R2 gRPC transport (plan #5). Mirrors
/// <see cref="Wcf.HistorianWcfStatusClient"/>'s synthesize-from-handshake approach: it opens an
/// authenticated session and reports <see cref="HistorianConnectionStatus.ConnectedToServer"/> /
/// <see cref="HistorianConnectionStatus.ConnectedToServerStorage"/> from whether the handshake
/// (GetInterfaceVersion → ValidateClientCredential token loop → OpenConnection, which yields the
/// storage-session GUID) succeeds. There is no dedicated connection-status RPC on either transport.
/// Store-forward connectivity is not observable here (D2-gated) and stays false.
/// </summary>
public static Task<HistorianConnectionStatus> GetConnectionStatusAsync(
HistorianClientOptions options,
CancellationToken cancellationToken)
=> Task.Run(() => GetConnectionStatus(options, cancellationToken), cancellationToken);
private static HistorianConnectionStatus GetConnectionStatus(HistorianClientOptions options, CancellationToken cancellationToken)
{
bool connected;
string? error = null;
try
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
// A successful OpenConnection yields a non-empty storage-session GUID — proof the server and
// its storage session are reachable, the gRPC analog of the WCF handshake probe.
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
(connected, error) = EvaluateConnectionStatusOnSession(connection, session);
}
catch (OperationCanceledException)
{
throw;
}
catch (Exception ex)
{
connected = false;
error = $"{ex.GetType().Name}: {ex.Message}";
}
return new HistorianConnectionStatus(
ServerName: options.Host,
Pending: false,
ErrorOccurred: !connected,
Error: error,
ConnectedToServer: connected,
ConnectedToServerStorage: connected,
ConnectedToStoreForward: false,
ConnectionKind: HistorianConnectionKind.Process);
}
// Spike/Phase-1 seam (pending.md A1): evaluate connection status against an EXTERNALLY-supplied,
// already-authenticated connection + session — NO Create()/handshake here. GetConnectionStatus
// delegates so the per-call path and the reuse path share one evaluation (DRY). Unlike the other
// status seams there is no follow-on RPC: connectivity is derived entirely from the handshake's own
// storage-session GUID (a successful OpenConnection yields a non-empty GUID). The unreachable/auth
// try/catch (which must also cover the handshake) stays with the per-call method.
internal static (bool Connected, string? Error) EvaluateConnectionStatusOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session)
{
bool connected = session.StorageSessionId != Guid.Empty;
string? error = connected ? null : "OpenConnection returned an empty storage-session handle.";
return (connected, error);
}
/// <summary>
/// Reads the Historian server's system time-zone name (roadmap item R1.3,
/// <c>StatusService.GetSystemTimeZoneName</c>). Unlike the 2020 WCF surface — where the native
/// <c>GetSystemTimeZoneName</c> is a client-side stub that returns an empty string — the 2023 R2
/// gRPC front door returns the real Windows time-zone display name (live-verified:
/// "Eastern Daylight Time"). Takes the transient <c>uint</c> client handle; the response carries
/// the value as a protobuf string with no opaque buffer to decode.
/// </summary>
public static Task<string?> GetSystemTimeZoneNameAsync(
HistorianClientOptions options,
CancellationToken cancellationToken)
=> Task.Run(() => GetSystemTimeZoneName(options, cancellationToken), cancellationToken);
private static string? GetSystemTimeZoneName(HistorianClientOptions options, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
uint clientHandle = HistorianGrpcHandshake.OpenAuthenticatedConnection(connection, options, cancellationToken);
var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel);
GrpcStatus.GetSystemTimeZoneNameResponse response = statusClient.GetSystemTimeZoneName(
new GrpcStatus.GetSystemTimeZoneNameRequest { UiHandle = clientHandle },
connection.Metadata,
DateTime.UtcNow.Add(options.RequestTimeout),
cancellationToken);
if (!(response.Status?.BSuccess ?? false))
{
return null;
}
string? value = response.StrSystemTimeZoneName;
return string.IsNullOrEmpty(value) ? null : value;
}
/// <summary>
/// Reads a Historian runtime parameter over gRPC (<c>StatusService.GetRuntimeParameter</c>).
/// The request/response byte buffers are the proven 2020 <c>GETRP</c> wire format
/// (<see cref="HistorianRuntimeParameterProtocol"/>) carried unchanged inside the protobuf
/// <c>btRequest</c>/<c>btResponse</c> fields; the op keys on the uppercase string session handle.
/// </summary>
public static Task<string?> GetRuntimeParameterAsync(
HistorianClientOptions options,
string parameterName,
CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(parameterName);
return Task.Run(() => GetRuntimeParameter(options, parameterName, cancellationToken), cancellationToken);
}
private static string? GetRuntimeParameter(HistorianClientOptions options, string parameterName, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
byte[] request = HistorianRuntimeParameterProtocol.SerializeRequest(parameterName);
var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel);
GrpcStatus.GetRuntimeParameterResponse response = statusClient.GetRuntimeParameter(
new GrpcStatus.GetRuntimeParameterRequest
{
StrHandle = session.StringHandle,
BtRequest = ByteString.CopyFrom(request)
},
connection.Metadata,
DateTime.UtcNow.Add(options.RequestTimeout),
cancellationToken);
if (!(response.Status?.BSuccess ?? false))
{
return null;
}
byte[] responseBuffer = response.BtResponse?.ToByteArray() ?? [];
return HistorianRuntimeParameterProtocol.ParseSingleStringResult(responseBuffer);
}
}
@@ -0,0 +1,222 @@
using System.Diagnostics;
using System.Text;
using Google.Protobuf;
using AVEVA.Historian.Client.Wcf;
using GrpcStorage = ArchestrA.Grpc.Contract.Storage;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// Live probe for the M3 follow-up step that the R3.1 decode pinned as the missing precondition:
/// <c>StorageService.OpenStorageConnection</c>. The R3.1 finding (see
/// <c>docs/plans/revision-write-path.md</c> §R3.1) was that <c>AddNonStreamValues</c> reaches the
/// server-side <c>CHistStorageConnection::StoreNonStreamValues</c>, which routes to the
/// <c>\\.\pipe\aahStorageEngine\console,sid(...)</c> named pipe and fails for lack of a console
/// session. <c>OpenStorageConnection</c> is the op that creates exactly that console <c>sid</c>
/// session (returning its own <c>uint</c> handle + a NEW storage-session GUID, distinct from the
/// Open2 session).
///
/// Unlike <c>AddNonStreamValues</c>, this op has NO opaque <c>btInput</c> buffer — all 12 request
/// fields are typed protobuf fields (see <c>StorageService.proto</c>). So there are no wire bytes to
/// guess; the only unknowns are the VALUES for a handful of inferable fields (ConnectionMode, the
/// in/out StorageSessionId, FreeDiskSpace, credential framing). This probe sweeps a small matrix of
/// those and reports the server's response for each, so one live run reveals which combination the
/// storage engine accepts. It writes NO historical data — on success it immediately calls
/// <c>CloseStorageConnection</c> to release the console session it opened.
/// </summary>
internal sealed class HistorianGrpcStorageConnectionProbe
{
// Native client identity constants, mirrored from HistorianNativeHandshake so the storage
// engine sees the same client fingerprint the Open2 handshake presented.
private const uint NativeClientType = 4;
private const uint NativeClientVersionInt = 999_999;
private const string EngineConsolePath = @"\\.\pipe\aahStorageEngine\console";
private readonly HistorianClientOptions _options;
public HistorianGrpcStorageConnectionProbe(HistorianClientOptions options)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
}
public Task<HistorianGrpcOpenStorageConnectionResult> ProbeAsync(CancellationToken cancellationToken)
=> Task.Run(() => Probe(cancellationToken), cancellationToken);
private HistorianGrpcOpenStorageConnectionResult Probe(CancellationToken cancellationToken)
{
var result = new HistorianGrpcOpenStorageConnectionResult();
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(
connection, _options, cancellationToken,
connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode);
result.OpenSucceeded = true;
result.ClientHandle = session.ClientHandle;
result.StorageSessionId = session.StorageSessionId;
var storageClient = new GrpcStorage.StorageService.StorageServiceClient(connection.Channel);
DateTime Deadline() => DateTime.UtcNow.Add(_options.RequestTimeout);
// Prime the Storage service's interface-version / session table (matches the cross-service
// GetV priming the other write paths use).
try
{
GrpcStorage.GetInterfaceVersionResponse version = storageClient.GetInterfaceVersion(
new GrpcStorage.GetInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken);
result.StorageInterfaceVersion = version.UiVersion;
result.StorageInterfaceVersionError = version.UiError;
}
catch (Exception ex)
{
result.StorageInterfaceVersionException = $"{ex.GetType().Name}: {ex.Message}";
}
Process current = Process.GetCurrentProcess();
string machineName = Environment.MachineName;
string processName = string.IsNullOrEmpty(current.ProcessName) ? "AVEVA.Historian.Client" : current.ProcessName;
uint processId = checked((uint)current.Id);
string upperGuid = session.StringHandle;
// Password framing: the gRPC session is already NTLM-authenticated (ValidateClientCredential),
// so attempt 1 sends no credential (rely on the authenticated channel). If the storage engine
// demands its own credential we'll see an auth-shaped error and add a credential-bearing
// attempt next iteration. For explicit creds we still try UTF-16LE password bytes as a probe.
byte[] emptyPwd = [];
// Sweep the genuinely-uncertain fields. Order = most-likely-correct first; stop at first
// success. ConnectionMode 0x401 = write-enabled (Process|Write|IntegratedSecurity), the same
// mode Open2 used for the write session. StorageSessionId-in: the native client threads the
// Open2 storage GUID through here (in/out); empty-string is the "create fresh" fallback.
var attempts = new List<(string Label, uint ConnectionMode, string SessionIdIn, uint FreeDiskSpace, byte[] Password)>
{
("mode=0x401, sid=open2-upper", 0x401, upperGuid, 0u, emptyPwd),
("mode=0x401, sid=empty", 0x401, string.Empty, 0u, emptyPwd),
("mode=0x402, sid=open2-upper", 0x402, upperGuid, 0u, emptyPwd),
("mode=0x1, sid=open2-upper", 0x1, upperGuid, 0u, emptyPwd),
("mode=0x401, sid=open2, disk=big", 0x401, upperGuid, 0xFFFFFFFFu, emptyPwd),
};
foreach ((string label, uint mode, string sidIn, uint freeDisk, byte[] pwd) in attempts)
{
var attempt = new HistorianGrpcOpenStorageConnectionAttempt
{
Label = label,
ConnectionMode = mode,
SessionIdIn = sidIn,
};
try
{
var request = new GrpcStorage.OpenStorageConnectionRequest
{
HostName = machineName,
EnginePath = EngineConsolePath,
FreeDiskSpace = freeDisk,
ProcessName = processName,
ProcessId = processId,
UserName = _options.IntegratedSecurity ? string.Empty : _options.UserName,
Password = ByteString.CopyFrom(pwd),
PwdLength = (uint)pwd.Length,
ClientType = NativeClientType,
ClientVersion = NativeClientVersionInt,
ConnectionMode = mode,
ConnectionTimeout = (uint)Math.Max(1, _options.RequestTimeout.TotalMilliseconds),
StorageSessionId = sidIn,
};
GrpcStorage.OpenStorageConnectionResponse response = storageClient.OpenStorageConnection(
request, connection.Metadata, Deadline(), cancellationToken);
attempt.Succeeded = response.Status?.BSuccess ?? false;
attempt.NewHandle = response.Handle;
attempt.NewStorageSessionId = response.StorageSessionId;
attempt.ServerStatus = response.ServerStatus;
attempt.ConnectionTime = response.ConnectionTime;
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
attempt.ErrorHex = error.Length == 0 ? null : Convert.ToHexString(error);
attempt.ErrorPreview = DescribeError(error);
result.Attempts.Add(attempt);
if (attempt.Succeeded)
{
result.OpenStorageSucceeded = true;
result.AcceptedAttempt = label;
result.NewStorageHandle = response.Handle;
result.NewStorageSessionId = response.StorageSessionId;
// Release the console session immediately — this probe persists nothing.
try
{
GrpcStorage.CloseStorageConnectionResponse close = storageClient.CloseStorageConnection(
new GrpcStorage.CloseStorageConnectionRequest { Handle = response.Handle },
connection.Metadata, Deadline(), cancellationToken);
result.CloseSucceeded = close.Status?.BSuccess ?? false;
}
catch (Exception ex)
{
result.CloseException = $"{ex.GetType().Name}: {ex.Message}";
}
break;
}
}
catch (Exception ex)
{
attempt.Exception = $"{ex.GetType().Name}: {ex.Message}";
result.Attempts.Add(attempt);
}
}
return result;
}
/// <summary>Short printable preview of a server error buffer (status codes/messages, no secrets).</summary>
private static string? DescribeError(byte[] error)
{
if (error.Length == 0)
{
return null;
}
ReadOnlySpan<byte> preview = error.AsSpan(0, Math.Min(error.Length, 96));
var sb = new StringBuilder(preview.Length);
foreach (byte b in preview)
{
sb.Append(b is >= 0x20 and < 0x7F ? (char)b : '.');
}
return sb.ToString();
}
}
internal sealed class HistorianGrpcOpenStorageConnectionResult
{
public bool OpenSucceeded { get; set; }
public uint ClientHandle { get; set; }
public Guid StorageSessionId { get; set; }
public uint? StorageInterfaceVersion { get; set; }
public uint? StorageInterfaceVersionError { get; set; }
public string? StorageInterfaceVersionException { get; set; }
public bool OpenStorageSucceeded { get; set; }
public string? AcceptedAttempt { get; set; }
public uint NewStorageHandle { get; set; }
public string? NewStorageSessionId { get; set; }
public bool CloseSucceeded { get; set; }
public string? CloseException { get; set; }
public List<HistorianGrpcOpenStorageConnectionAttempt> Attempts { get; } = new();
}
internal sealed class HistorianGrpcOpenStorageConnectionAttempt
{
public string Label { get; set; } = "";
public uint ConnectionMode { get; set; }
public string SessionIdIn { get; set; } = "";
public bool Succeeded { get; set; }
public uint NewHandle { get; set; }
public string? NewStorageSessionId { get; set; }
public uint ServerStatus { get; set; }
public ulong ConnectionTime { get; set; }
public string? ErrorHex { get; set; }
public string? ErrorPreview { get; set; }
public string? Exception { get; set; }
}
@@ -0,0 +1,364 @@
using System.Text;
using Google.Protobuf;
using AVEVA.Historian.Client.Wcf;
using GrpcStorage = ArchestrA.Grpc.Contract.Storage;
using GrpcStatus = ArchestrA.Grpc.Contract.Status;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// R4.3 discovery probe (see <c>docs/plans/store-forward-cache-reverse-engineering.md</c> §9).
/// Reads store-forward (SF) state from the 2023 R2 <c>StorageService</c> via the recovered PULL
/// RPCs — no duplex/callback contract, no native <c>HISTORIAN_STORAGE_STATUS</c> struct decode:
/// <list type="bullet">
/// <item><c>GetRemainingSnapshotsSize(Handle) → uint64 SnapshotSize</c> — the pending-buffer
/// magnitude in one call (non-zero ⇒ data queued).</item>
/// <item><c>GetSFParameter(Handle, ParameterName) → string</c> — the string-keyed SF state read,
/// the analogue of the already-shipped <c>GetSystemParameter</c>.</item>
/// </list>
/// The one surviving unknown (§9.3) is which <c>uint Handle</c> these RPCs want: the cheap session
/// <c>ClientHandle</c> (unblocked) or the <c>OpenStorageConnection</c> console handle (the D2
/// storage-engine-pipe wall). This probe tries the session handle FIRST and, only if those calls
/// fail handle-shaped, falls back to opening a storage console session to disambiguate — releasing
/// it immediately. It writes NOTHING.
/// </summary>
internal sealed class HistorianGrpcStoreForwardStatusProbe
{
/// <summary>Candidate SF parameter names swept through <c>GetSFParameter</c>. Derived from the
/// managed <c>HistorianStoreForwardStatus</c> fields + the native SF getter vocabulary; the
/// server reveals which it accepts.</summary>
private static readonly string[] CandidateParameterNames =
[
"Status", "Storing", "Pending", "DataStored", "ErrorOccurred", "Error",
"SFStatus", "SF.Status", "StoreForward", "StoreForwardStatus", "Forward",
"ForwardingStatus", "CacheSize", "SnapshotSize", "RemainingSize", "Enabled",
];
private readonly HistorianClientOptions _options;
private readonly bool _writeEnabledSession;
public HistorianGrpcStoreForwardStatusProbe(HistorianClientOptions options, bool writeEnabledSession = false)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
_writeEnabledSession = writeEnabledSession;
}
public Task<HistorianGrpcStoreForwardStatusProbeResult> ProbeAsync(CancellationToken cancellationToken)
=> Task.Run(() => Probe(cancellationToken), cancellationToken);
private HistorianGrpcStoreForwardStatusProbeResult Probe(CancellationToken cancellationToken)
{
var result = new HistorianGrpcStoreForwardStatusProbeResult();
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
// The idle probe found GetRemainingSnapshotsSize returns err 132 OperationNotEnabled under a
// read-only session — the same 0x402-vs-0x401 gate the write paths flip. So allow opening the
// session write-enabled to confirm the op succeeds when enabled.
uint connectionMode = _writeEnabledSession
? HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode
: HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode;
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(
connection, _options, cancellationToken, connectionMode);
result.OpenSucceeded = true;
result.WriteEnabledSession = _writeEnabledSession;
result.ClientHandle = session.ClientHandle;
result.StringHandle = session.StringHandle;
var storageClient = new GrpcStorage.StorageService.StorageServiceClient(connection.Channel);
DateTime Deadline() => DateTime.UtcNow.Add(_options.RequestTimeout);
// Session-handle StatusService route (the old plan's Q2): GetHistorianConsoleStatus +
// GetHistorianInfo take the STRING handle, so they're NOT gated on the OpenStorageConnection
// console handle (the D2 wall). This is the most promising idle-baseline lever.
ProbeStatusService(result, connection, Deadline, session, cancellationToken);
// Prime the Storage service's interface-version / session table.
try
{
GrpcStorage.GetInterfaceVersionResponse version = storageClient.GetInterfaceVersion(
new GrpcStorage.GetInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken);
result.StorageInterfaceVersion = version.UiVersion;
result.StorageInterfaceVersionError = version.UiError;
}
catch (Exception ex)
{
result.StorageInterfaceVersionException = $"{ex.GetType().Name}: {ex.Message}";
}
// Phase 1: try the cheap session ClientHandle (best case — status reads shouldn't need a
// console writer session).
result.SessionHandleAttempt = QueryWithHandle(
storageClient, connection, Deadline, session.ClientHandle, "session.ClientHandle", cancellationToken);
// Phase 2 (disambiguation, §9.3): only if every Phase-1 call failed, try the
// OpenStorageConnection console handle to learn whether SF reads are gated on the D2 wall.
if (!result.SessionHandleAttempt.AnySucceeded)
{
TryConsoleHandleFallback(result, storageClient, connection, Deadline, session, cancellationToken);
}
return result;
}
private static HistorianGrpcSfHandleAttempt QueryWithHandle(
GrpcStorage.StorageService.StorageServiceClient storageClient,
HistorianGrpcConnection connection,
Func<DateTime> deadline,
uint handle,
string handleLabel,
CancellationToken cancellationToken)
{
var attempt = new HistorianGrpcSfHandleAttempt { HandleLabel = handleLabel, Handle = handle };
// GetRemainingSnapshotsSize — the single cleanest pending/idle signal.
try
{
GrpcStorage.GetRemainingSnapshotsSizeResponse resp = storageClient.GetRemainingSnapshotsSize(
new GrpcStorage.GetRemainingSnapshotsSizeRequest { Handle = handle },
connection.Metadata, deadline(), cancellationToken);
byte[] err = resp.Status?.BtError?.ToByteArray() ?? [];
attempt.RemainingSnapshotsSizeSucceeded = resp.Status?.BSuccess ?? false;
attempt.RemainingSnapshotsSize = resp.SnapshotSize;
attempt.RemainingSnapshotsSizeError = DescribeError(err);
attempt.RemainingSnapshotsSizeErrorHex = err.Length == 0 ? null : Convert.ToHexString(err);
}
catch (Exception ex)
{
attempt.RemainingSnapshotsSizeException = $"{ex.GetType().Name}: {ex.Message}";
}
// Sweep GetSFParameter over the candidate name vocabulary.
foreach (string name in CandidateParameterNames)
{
var pr = new HistorianGrpcSfParameterResult { Name = name };
try
{
GrpcStorage.GetSFParameterResponse resp = storageClient.GetSFParameter(
new GrpcStorage.GetSFParameterRequest { Handle = handle, ParameterName = name },
connection.Metadata, deadline(), cancellationToken);
pr.Succeeded = resp.Status?.BSuccess ?? false;
pr.Value = resp.ParamaterValue;
pr.Error = DescribeError(resp.Status?.BtError?.ToByteArray() ?? []);
}
catch (Exception ex)
{
pr.Exception = $"{ex.GetType().Name}: {ex.Message}";
}
attempt.Parameters.Add(pr);
}
return attempt;
}
private void TryConsoleHandleFallback(
HistorianGrpcStoreForwardStatusProbeResult result,
GrpcStorage.StorageService.StorageServiceClient storageClient,
HistorianGrpcConnection connection,
Func<DateTime> deadline,
HistorianGrpcHandshake.Session session,
CancellationToken cancellationToken)
{
result.ConsoleHandleFallbackAttempted = true;
try
{
var request = new GrpcStorage.OpenStorageConnectionRequest
{
HostName = Environment.MachineName,
EnginePath = @"\\.\pipe\aahStorageEngine\console",
FreeDiskSpace = 0,
ProcessName = "AVEVA.Historian.Client",
ProcessId = (uint)Environment.ProcessId,
UserName = _options.IntegratedSecurity ? string.Empty : _options.UserName,
Password = ByteString.Empty,
PwdLength = 0,
ClientType = 4,
ClientVersion = 999_999,
ConnectionMode = HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode,
ConnectionTimeout = (uint)Math.Max(1, _options.RequestTimeout.TotalMilliseconds),
StorageSessionId = session.StringHandle,
};
GrpcStorage.OpenStorageConnectionResponse open = storageClient.OpenStorageConnection(
request, connection.Metadata, deadline(), cancellationToken);
byte[] openErr = open.Status?.BtError?.ToByteArray() ?? [];
result.OpenStorageConnectionSucceeded = open.Status?.BSuccess ?? false;
result.OpenStorageConnectionError = DescribeError(openErr);
result.OpenStorageConnectionErrorHex = openErr.Length == 0 ? null : Convert.ToHexString(openErr);
if (result.OpenStorageConnectionSucceeded)
{
result.ConsoleHandleAttempt = QueryWithHandle(
storageClient, connection, deadline, open.Handle, "OpenStorageConnection.Handle", cancellationToken);
try
{
storageClient.CloseStorageConnection(
new GrpcStorage.CloseStorageConnectionRequest { Handle = open.Handle },
connection.Metadata, deadline(), cancellationToken);
}
catch (Exception ex)
{
result.CloseStorageConnectionException = $"{ex.GetType().Name}: {ex.Message}";
}
}
}
catch (Exception ex)
{
result.OpenStorageConnectionException = $"{ex.GetType().Name}: {ex.Message}";
}
}
private void ProbeStatusService(
HistorianGrpcStoreForwardStatusProbeResult result,
HistorianGrpcConnection connection,
Func<DateTime> deadline,
HistorianGrpcHandshake.Session session,
CancellationToken cancellationToken)
{
var status = new HistorianGrpcSfStatusServiceProbe();
var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel);
string strHandle = session.StringHandle;
// GetHistorianConsoleStatus(strHandle) → uiConsoleStatus. The "console" is the storage-engine
// console where SF lives; this uint status may encode the SF/storing state.
try
{
GrpcStatus.GetHistorianConsoleStatusResponse resp = statusClient.GetHistorianConsoleStatus(
new GrpcStatus.GetHistorianConsoleStatusRequest { StrHandle = strHandle },
connection.Metadata, deadline(), cancellationToken);
byte[] err = resp.Status?.BtError?.ToByteArray() ?? [];
status.ConsoleStatusSucceeded = resp.Status?.BSuccess ?? false;
status.ConsoleStatusValue = resp.UiConsoleStatus;
status.ConsoleStatusError = DescribeError(err);
status.ConsoleStatusErrorHex = err.Length == 0 ? null : Convert.ToHexString(err);
}
catch (Exception ex)
{
status.ConsoleStatusException = $"{ex.GetType().Name}: {ex.Message}";
}
// GetHistorianInfo(strHandle, btRequest) → btHistorianInfo. btRequest framing is unknown; try a
// small set of candidates and report whichever the server accepts + the returned blob (hex).
var infoCandidates = new List<(string Label, byte[] Request)>
{
("empty", []),
("u32(0)", [0, 0, 0, 0]),
("ascii:StoreForward", Encoding.ASCII.GetBytes("StoreForward")),
("utf16:StoreForward", Encoding.Unicode.GetBytes("StoreForward")),
};
foreach ((string label, byte[] request) in infoCandidates)
{
var info = new HistorianGrpcSfHistorianInfoResult { Label = label };
try
{
GrpcStatus.GetHistorianInfoResponse resp = statusClient.GetHistorianInfo(
new GrpcStatus.GetHistorianInfoRequest { StrHandle = strHandle, BtRequest = ByteString.CopyFrom(request) },
connection.Metadata, deadline(), cancellationToken);
byte[] blob = resp.BtHistorianInfo?.ToByteArray() ?? [];
byte[] err = resp.Status?.BtError?.ToByteArray() ?? [];
info.Succeeded = resp.Status?.BSuccess ?? false;
info.InfoLength = blob.Length;
info.InfoHex = blob.Length == 0 ? null : Convert.ToHexString(blob.AsSpan(0, Math.Min(blob.Length, 256)));
info.InfoText = DescribeError(blob);
info.Error = DescribeError(err);
}
catch (Exception ex)
{
info.Exception = $"{ex.GetType().Name}: {ex.Message}";
}
status.HistorianInfo.Add(info);
}
result.StatusService = status;
}
/// <summary>Short printable preview of a server error buffer (status text only, no secrets).</summary>
private static string? DescribeError(byte[] error)
{
if (error.Length == 0)
{
return null;
}
ReadOnlySpan<byte> preview = error.AsSpan(0, Math.Min(error.Length, 96));
var sb = new StringBuilder(preview.Length);
foreach (byte b in preview)
{
sb.Append(b is >= 0x20 and < 0x7F ? (char)b : '.');
}
return sb.ToString();
}
}
internal sealed class HistorianGrpcStoreForwardStatusProbeResult
{
public bool OpenSucceeded { get; set; }
public bool WriteEnabledSession { get; set; }
public uint ClientHandle { get; set; }
public string? StringHandle { get; set; }
public uint? StorageInterfaceVersion { get; set; }
public uint? StorageInterfaceVersionError { get; set; }
public string? StorageInterfaceVersionException { get; set; }
public HistorianGrpcSfStatusServiceProbe? StatusService { get; set; }
public HistorianGrpcSfHandleAttempt? SessionHandleAttempt { get; set; }
public bool ConsoleHandleFallbackAttempted { get; set; }
public bool OpenStorageConnectionSucceeded { get; set; }
public string? OpenStorageConnectionError { get; set; }
public string? OpenStorageConnectionErrorHex { get; set; }
public string? OpenStorageConnectionException { get; set; }
public HistorianGrpcSfHandleAttempt? ConsoleHandleAttempt { get; set; }
public string? CloseStorageConnectionException { get; set; }
}
internal sealed class HistorianGrpcSfStatusServiceProbe
{
public bool ConsoleStatusSucceeded { get; set; }
public uint ConsoleStatusValue { get; set; }
public string? ConsoleStatusError { get; set; }
public string? ConsoleStatusErrorHex { get; set; }
public string? ConsoleStatusException { get; set; }
public List<HistorianGrpcSfHistorianInfoResult> HistorianInfo { get; } = new();
}
internal sealed class HistorianGrpcSfHistorianInfoResult
{
public string Label { get; set; } = "";
public bool Succeeded { get; set; }
public int InfoLength { get; set; }
public string? InfoHex { get; set; }
public string? InfoText { get; set; }
public string? Error { get; set; }
public string? Exception { get; set; }
}
internal sealed class HistorianGrpcSfHandleAttempt
{
public string HandleLabel { get; set; } = "";
public uint Handle { get; set; }
public bool RemainingSnapshotsSizeSucceeded { get; set; }
public ulong RemainingSnapshotsSize { get; set; }
public string? RemainingSnapshotsSizeError { get; set; }
public string? RemainingSnapshotsSizeErrorHex { get; set; }
public string? RemainingSnapshotsSizeException { get; set; }
public List<HistorianGrpcSfParameterResult> Parameters { get; } = new();
/// <summary>True when any pull RPC (size or a parameter) returned bSuccess for this handle.</summary>
public bool AnySucceeded =>
RemainingSnapshotsSizeSucceeded || Parameters.Exists(static p => p.Succeeded);
}
internal sealed class HistorianGrpcSfParameterResult
{
public string Name { get; set; } = "";
public bool Succeeded { get; set; }
public string? Value { get; set; }
public string? Error { get; set; }
public string? Exception { get; set; }
}
@@ -0,0 +1,429 @@
using System.Text;
using Google.Protobuf;
using Grpc.Core;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// 2023 R2 gRPC tag-metadata + browse client (roadmap items R0.2 metadata, R0.1 browse).
/// Browse drives <c>StartTagQuery</c> (OData filter) → paged <c>QueryTag</c> → <c>EndTagQuery</c>
/// (see <see cref="BrowseTagNamesAsync"/> and <c>docs/reverse-engineering/grpc-tag-query-odata.md</c>).
/// Unlike the WCF singular
/// <c>GetTagInfoFromName</c> (a <c>uint</c>-handle op), the gRPC front door exposes the plural
/// <c>RetrievalService.GetTagInfosFromName</c> — a <b>string-handle</b> op keyed off the Open2
/// storage-session GUID (uppercase). The request <c>btTagNames</c> buffer and response
/// <c>btTagInfos</c> buffer carry the proven native encodings:
/// <list type="bullet">
/// <item>request <c>btTagNames</c> = <c>uint count</c> + per-name(<c>uint charCount</c> + UTF-16LE)</item>
/// <item>response <c>btTagInfos</c> = <c>uint tagCount</c> + per-tag CTagMetadata record
/// (the same record <see cref="HistorianTagQueryProtocol.ParseGetTagInfoResponse"/> decodes)</item>
/// </list>
/// The string-handle "wall" that blocks this op family on the 2020 WCF transport does not apply on
/// the gRPC front door (different envelope/registration) — see
/// <c>docs/reverse-engineering/wcf-string-handle-wall.md</c>.
/// </summary>
internal static class HistorianGrpcTagClient
{
public static Task<HistorianTagMetadata?> GetTagMetadataAsync(
HistorianClientOptions options,
string tag,
CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tag);
return Task.Run(() => GetTagMetadata(options, tag, cancellationToken), cancellationToken);
}
private static HistorianTagMetadata? GetTagMetadata(HistorianClientOptions options, string tag, CancellationToken cancellationToken)
{
byte[] tagInfos = GetTagInfosRaw(options, [tag], cancellationToken);
return ParseTagMetadata(tagInfos);
}
// Spike/Phase-1 seam (pending.md A1): resolve tag metadata against an EXTERNALLY-supplied,
// already-authenticated connection + session — i.e. NO Create()/handshake here. The per-call
// GetTagMetadata and this seam share the parse tail (ParseTagMetadata) so neither duplicates the
// decode logic (DRY).
internal static HistorianTagMetadata? GetTagMetadataOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
string tag,
HistorianClientOptions options,
CancellationToken cancellationToken)
{
byte[] tagInfos = GetTagInfosRawOnSession(connection, session, [tag], options, cancellationToken);
return ParseTagMetadata(tagInfos);
}
// Shared parse tail for both the per-call GetTagMetadata and the reuse-path GetTagMetadataOnSession.
private static HistorianTagMetadata? ParseTagMetadata(byte[] tagInfos)
{
if (tagInfos.Length < 4)
{
return null;
}
IReadOnlyList<HistorianTagInfoResponse> parsed = HistorianTagQueryProtocol.ParseGetTagInfoResponse(tagInfos);
if (parsed.Count == 0)
{
return null;
}
HistorianTagInfoResponse info = parsed[0];
return new HistorianTagMetadata(
Name: info.TagName,
Key: info.TagKey,
DataType: HistorianWcfTagClient.MapDataType(info.NativeDataTypeDescriptor),
Description: info.Description ?? info.MetadataProvider,
EngineeringUnit: info.EngineeringUnit ?? string.Empty,
MinRaw: info.MinEU,
MaxRaw: info.MaxEU);
}
/// <summary>
/// Issues a single <c>GetTagInfosFromName</c> call and returns the raw native <c>btTagInfos</c>
/// response buffer. Internal so reverse-engineering probes can capture the framing.
/// </summary>
internal static byte[] GetTagInfosRaw(HistorianClientOptions options, IReadOnlyList<string> tags, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
return GetTagInfosRawOnSession(connection, session, tags, options, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): issue GetTagInfosFromName against an EXTERNALLY-supplied,
// already-authenticated connection + session — i.e. NO Create()/handshake here. GetTagInfosRaw
// delegates to this so the per-call path and the reuse path share one query implementation (DRY).
internal static byte[] GetTagInfosRawOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
IReadOnlyList<string> tags,
HistorianClientOptions options,
CancellationToken cancellationToken)
{
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
byte[] requestBuffer = BuildTagNamesBuffer(tags);
GrpcRetrieval.GetTagInfosFromNameResponse response = retrievalClient.GetTagInfosFromName(
new GrpcRetrieval.GetTagInfosFromNameRequest
{
StrHandle = session.StringHandle,
BtTagNames = ByteString.CopyFrom(requestBuffer),
UiSequence = 0
},
connection.Metadata,
DateTime.UtcNow.Add(options.RequestTimeout),
cancellationToken);
if (!(response.Status?.BSuccess ?? false))
{
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
throw new InvalidOperationException($"gRPC GetTagInfosFromName failed (errorLen={error.Length}).");
}
return response.BtTagInfos?.ToByteArray() ?? [];
}
// GetTagExtendedPropertiesFromName is sequence-paged; a single tag returns everything on page 0
// and an empty/false buffer next. The cap is a runaway guard (mirrors the WCF path).
private const int MaxExtendedPropertyPages = 64;
/// <summary>
/// Reads a tag's extended (user-defined) properties over gRPC
/// (<c>RetrievalService.GetTagExtendedPropertiesFromName</c>, a string-handle op). The request
/// <c>btTagNames</c> and response <c>btTeps</c> buffers are the proven 2020 <c>GetTepByNm</c> wire
/// format (<see cref="HistorianTagExtendedPropertyProtocol"/>) carried unchanged; paging follows
/// the same sequence loop as the WCF path.
/// </summary>
public static Task<IReadOnlyList<HistorianTagExtendedProperty>> GetTagExtendedPropertiesAsync(
HistorianClientOptions options,
string tag,
CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tag);
return Task.Run(() => GetTagExtendedProperties(options, tag, cancellationToken), cancellationToken);
}
// No …OnSession seam: extended-properties browse stays per-call (not amortized through the session
// pool — out of A1-broadening scope). Add a seam here only if the pool ever needs to route it.
/// <summary>
/// Issues a single page-0 <c>GetTagExtendedPropertiesFromName</c> call and returns the raw native
/// <c>btTeps</c> response buffer (empty when the server reports no rows / non-success). Internal so
/// reverse-engineering probes can capture the framing.
/// </summary>
internal static byte[] GetTagExtendedPropertiesRaw(HistorianClientOptions options, string tag, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
byte[] tagNames = HistorianTagExtendedPropertyProtocol.SerializeRequest(tag);
GrpcRetrieval.GetTagExtendedPropertiesFromNameResponse response = retrievalClient.GetTagExtendedPropertiesFromName(
new GrpcRetrieval.GetTagExtendedPropertiesFromNameRequest
{
StrHandle = session.StringHandle,
BtTagNames = ByteString.CopyFrom(tagNames),
UiSequence = 0
},
connection.Metadata,
DateTime.UtcNow.Add(options.RequestTimeout),
cancellationToken);
return (response.Status?.BSuccess ?? false) ? response.BtTeps?.ToByteArray() ?? [] : [];
}
private static IReadOnlyList<HistorianTagExtendedProperty> GetTagExtendedProperties(
HistorianClientOptions options, string tag, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
byte[] tagNames = HistorianTagExtendedPropertyProtocol.SerializeRequest(tag);
List<HistorianTagExtendedProperty> properties = [];
uint sequence = 0;
for (int page = 0; page < MaxExtendedPropertyPages; page++)
{
cancellationToken.ThrowIfCancellationRequested();
GrpcRetrieval.GetTagExtendedPropertiesFromNameResponse response = retrievalClient.GetTagExtendedPropertiesFromName(
new GrpcRetrieval.GetTagExtendedPropertiesFromNameRequest
{
StrHandle = session.StringHandle,
BtTagNames = ByteString.CopyFrom(tagNames),
UiSequence = sequence
},
connection.Metadata,
DateTime.UtcNow.Add(options.RequestTimeout),
cancellationToken);
if (!(response.Status?.BSuccess ?? false))
{
// A non-success terminates paging. The server signals "no more rows" with a
// CClientUtil::FillBufferFromVector marker (live-confirmed) — including on page 0 when
// the tag has no user-defined properties, which is a legitimate empty result, not an
// error. This mirrors the WCF path, which also breaks (returns empty) rather than throws.
break;
}
IReadOnlyList<HistorianTagExtendedPropertyRow> rows =
HistorianTagExtendedPropertyProtocol.ParseResponse(response.BtTeps?.ToByteArray() ?? []);
if (rows.Count == 0)
{
break;
}
foreach (HistorianTagExtendedPropertyRow row in rows)
{
properties.Add(new HistorianTagExtendedProperty(row.PropertyName, row.Value));
}
sequence = response.UiSequence;
}
return properties;
}
// QueryTag (browse paging) request framing, recovered from the .rdata packet-descriptor table
// in aahClientManaged.dll (entries {0x6751,1}=StartTagQuery, {0x6752,1}=QueryTag) and confirmed
// live: btRequest = u16 marker(0x6752) + u16 version(1) + u16 queryType + u32 startIndex + u32 count.
private const ushort QueryTagPacketMarker = 0x6752;
private const ushort TagQueryHeaderVersion = 1;
private const ushort QueryTagModeNames = 1; // queryType 1 returns tag-name rows
private const uint BrowsePageSize = 1000;
/// <summary>
/// Browses tag names over gRPC (roadmap item R0.1). Drives
/// <c>StartTagQuery</c> (OData filter) → paged <c>QueryTag</c> → <c>EndTagQuery</c> on the
/// RetrievalService. The 2023 R2 metadata-server parses the filter as <b>OData</b>, so the SDK's
/// glob filter is translated via <see cref="GlobToODataFilter"/>. Each QueryTag page returns
/// <c>uint count + per-name(uint charCount + UTF-16LE)</c>, decoded by
/// <see cref="HistorianTagQueryProtocol.ParseGetLikeTagNamesResponse"/>.
/// </summary>
public static async IAsyncEnumerable<string> BrowseTagNamesAsync(
HistorianClientOptions options,
string filter,
[System.Runtime.CompilerServices.EnumeratorCancellation] CancellationToken cancellationToken)
{
IReadOnlyList<string> names = await Task.Run(() => BrowseTagNames(options, filter, cancellationToken), cancellationToken).ConfigureAwait(false);
foreach (string name in names)
{
cancellationToken.ThrowIfCancellationRequested();
yield return name;
}
}
private static List<string> BrowseTagNames(HistorianClientOptions options, string filter, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, options, cancellationToken);
return BrowseTagNamesOnSession(connection, session, filter, options, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): drive StartTagQuery → paged QueryTag → EndTagQuery against an
// EXTERNALLY-supplied, already-authenticated connection + session — i.e. NO Create()/handshake here.
// BrowseTagNames delegates to this so the per-call path and the reuse path share one browse
// implementation (DRY).
internal static List<string> BrowseTagNamesOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
string filter,
HistorianClientOptions options,
CancellationToken cancellationToken)
{
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
DateTime Deadline() => DateTime.UtcNow.Add(options.RequestTimeout);
byte[] startRequest = HistorianTagQueryProtocol.CreateStartTagQueryAttempt(GlobToODataFilter(filter)).RequestBuffer;
GrpcRetrieval.StartTagQueryResponse start = retrievalClient.StartTagQuery(
new GrpcRetrieval.StartTagQueryRequest { StrHandle = session.StringHandle, BtRequest = ByteString.CopyFrom(startRequest) },
connection.Metadata, Deadline(), cancellationToken);
if (!(start.Status?.BSuccess ?? false))
{
byte[] error = start.Status?.BtError?.ToByteArray() ?? [];
throw new InvalidOperationException($"gRPC StartTagQuery failed (errorLen={error.Length}).");
}
HistorianTagQueryStartResponse parsed = HistorianTagQueryProtocol.ParseStartTagQueryResponse(start.BtResponse?.ToByteArray() ?? []);
List<string> names = new(checked((int)parsed.TagCount));
try
{
uint startIndex = 0;
while (names.Count < parsed.TagCount)
{
cancellationToken.ThrowIfCancellationRequested();
uint page = Math.Min(BrowsePageSize, parsed.TagCount - (uint)names.Count);
GrpcRetrieval.QueryTagResponse query = retrievalClient.QueryTag(
new GrpcRetrieval.QueryTagRequest
{
StrHandle = session.StringHandle,
UiQueryHandle = parsed.QueryHandle,
BtRequest = ByteString.CopyFrom(BuildQueryTagRequest(QueryTagModeNames, startIndex, page))
},
connection.Metadata, Deadline(), cancellationToken);
if (!(query.Status?.BSuccess ?? false))
{
byte[] error = query.Status?.BtError?.ToByteArray() ?? [];
throw new InvalidOperationException($"gRPC QueryTag failed (errorLen={error.Length}).");
}
IReadOnlyList<string> pageNames = HistorianTagQueryProtocol.ParseTagNameQueryPage(query.BtResonse?.ToByteArray() ?? []);
if (pageNames.Count == 0)
{
break;
}
names.AddRange(pageNames);
startIndex += (uint)pageNames.Count;
}
}
finally
{
try
{
retrievalClient.EndTagQuery(
new GrpcRetrieval.EndTagQueryRequest { StrHandle = session.StringHandle, UiQueryHandle = parsed.QueryHandle },
connection.Metadata, Deadline(), CancellationToken.None);
}
catch { /* best-effort cleanup */ }
}
return names;
}
/// <summary>Builds the QueryTag paging request: u16 marker(0x6752) + u16 version + u16 queryType + u32 startIndex + u32 count.</summary>
internal static byte[] BuildQueryTagRequest(ushort queryType, uint startIndex, uint count)
{
using MemoryStream stream = new();
using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true);
writer.Write(QueryTagPacketMarker);
writer.Write(TagQueryHeaderVersion);
writer.Write(queryType);
writer.Write(startIndex);
writer.Write(count);
return stream.ToArray();
}
/// <summary>
/// Translates the SDK's glob filter (<c>*</c> wildcard) into the OData filter the 2023 R2
/// metadata-server's <c>StartActiveTagnamesQuery</c> expects. Single-quotes are OData-escaped.
/// <list type="bullet">
/// <item><c>*</c> / empty → no filter (all tags)</item>
/// <item><c>Pre*</c> → <c>startswith(TagName,'Pre')</c></item>
/// <item><c>*suf</c> → <c>endswith(TagName,'suf')</c></item>
/// <item><c>*mid*</c> → <c>contains(TagName,'mid')</c></item>
/// <item><c>a*b</c> → <c>startswith(TagName,'a') and endswith(TagName,'b')</c></item>
/// <item><c>Exact</c> → <c>TagName eq 'Exact'</c></item>
/// </list>
/// </summary>
internal static string GlobToODataFilter(string filter)
{
if (string.IsNullOrEmpty(filter) || filter == "*")
{
return string.Empty;
}
static string Esc(string s) => s.Replace("'", "''");
bool starStart = filter.StartsWith('*');
bool starEnd = filter.EndsWith('*');
string core = filter.Trim('*');
if (core.Length == 0)
{
return string.Empty; // "**" etc.
}
if (filter.IndexOf('*') < 0)
{
return $"TagName eq '{Esc(filter)}'";
}
if (starStart && starEnd && !core.Contains('*'))
{
return $"contains(TagName,'{Esc(core)}')";
}
if (starEnd && !core.Contains('*') && !starStart)
{
return $"startswith(TagName,'{Esc(core)}')";
}
if (starStart && !core.Contains('*') && !starEnd)
{
return $"endswith(TagName,'{Esc(core)}')";
}
// Internal wildcard(s): anchor on the prefix before the first '*' and the suffix after the last.
string prefix = filter[..filter.IndexOf('*')];
string suffix = filter[(filter.LastIndexOf('*') + 1)..];
List<string> parts = [];
if (prefix.Length > 0)
{
parts.Add($"startswith(TagName,'{Esc(prefix)}')");
}
if (suffix.Length > 0)
{
parts.Add($"endswith(TagName,'{Esc(suffix)}')");
}
return parts.Count > 0 ? string.Join(" and ", parts) : string.Empty;
}
/// <summary>Builds the native tag-names request buffer: uint count + per-name(uint charCount + UTF-16LE).</summary>
internal static byte[] BuildTagNamesBuffer(IReadOnlyList<string> tags)
{
using MemoryStream stream = new();
using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true);
writer.Write((uint)tags.Count);
foreach (string tag in tags)
{
writer.Write((uint)tag.Length);
if (tag.Length > 0)
{
writer.Write(Encoding.Unicode.GetBytes(tag));
}
}
return stream.ToArray();
}
}
@@ -0,0 +1,352 @@
using Google.Protobuf;
using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf;
using GrpcHistory = ArchestrA.Grpc.Contract.History;
using GrpcRetrieval = ArchestrA.Grpc.Contract.Retrieval;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// Tag-configuration write ops over the 2023 R2 gRPC transport, mirroring
/// <see cref="HistorianWcfTagWriteOrchestrator"/>. Each op opens a <b>write-enabled</b> Open2 session
/// (<c>0x401</c>) and reuses the proven 2020 byte serializers verbatim inside the protobuf
/// <c>bytes</c> fields:
/// <list type="bullet">
/// <item><see cref="EnsureTagAsync"/> → <c>HistoryService.EnsureTags</c> (string handle,
/// <c>btTagInfos</c> = <see cref="HistorianTagWriteProtocol.SerializeAnalogCTagMetadata"/>)</item>
/// <item><see cref="DeleteTagAsync"/> → <c>HistoryService.DeleteTags</c> (uint handle,
/// <c>btTagnames</c> = <see cref="HistorianTagWriteProtocol.SerializeDeleteTagNames"/>)</item>
/// <item><see cref="RenameTagsAsync"/> → <c>HistoryService.StartJob</c> (string handle,
/// <c>btInput</c> = <see cref="HistorianTagRenameProtocol.SerializeRenameJob"/>)</item>
/// <item><see cref="AddTagExtendedPropertiesAsync"/> → <c>HistoryService.AddTagExtendedProperties</c>
/// (string handle, <c>btTeps</c> = <see cref="HistorianTagExtendedPropertyProtocol.SerializeAddRequest"/>)</item>
/// </list>
/// <para>
/// <b>Tooled but not yet live-verified.</b> The request framing reuses the WCF serializers proven on
/// the 2020 transport, and the read-side config ops confirm WCF config buffers ride the gRPC RPC
/// unchanged — but these mutate server state (create/delete/rename tags, write properties), so they
/// are gated behind a sandbox-tag in the integration tests and have not been run destructively against
/// a shared live server. The WCF path additionally runs a priming "discovery dance" (UpdC3 + system
/// parameters + cross-service GetV) before the write; the gRPC front door established the equivalent
/// session state in the M3 non-streamed-write probe without it, so it is omitted here pending live
/// confirmation. If a live run is rejected, that priming is the first thing to add.
/// </para>
/// </summary>
internal sealed class HistorianGrpcTagWriteOrchestrator
{
private const uint WriteEnabledConnectionMode = HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode;
private readonly HistorianClientOptions _options;
public HistorianGrpcTagWriteOrchestrator(HistorianClientOptions options)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
}
public Task<bool> EnsureTagAsync(HistorianTagDefinition definition, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(definition);
ArgumentException.ThrowIfNullOrWhiteSpace(definition.TagName, nameof(definition));
// Surface unsupported (non-analog) types early, exactly as the WCF path does.
_ = HistorianTagWriteProtocol.GetAnalogDataTypeCode(definition.DataType);
return Task.Run(() => EnsureTag(definition, cancellationToken), cancellationToken);
}
private bool EnsureTag(HistorianTagDefinition definition, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken, WriteEnabledConnectionMode);
return EnsureTagOnSession(connection, session, definition, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): run EnsureTags against an EXTERNALLY-supplied, already-
// authenticated write-enabled (0x401) connection + session — NO Create()/handshake here. EnsureTag
// delegates so the per-call path and the reuse path share one op implementation (DRY).
internal bool EnsureTagOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
HistorianTagDefinition definition,
CancellationToken cancellationToken)
{
byte[] payload = HistorianTagWriteProtocol.SerializeAnalogCTagMetadata(
tagName: definition.TagName,
description: definition.Description,
engineeringUnit: definition.EngineeringUnit,
dateCreatedUtc: DateTime.UtcNow,
dataType: definition.DataType,
minEU: definition.MinEU,
maxEU: definition.MaxEU,
minRaw: definition.MinRaw,
maxRaw: definition.MaxRaw,
storageRateMs: definition.StorageRateMs,
applyScaling: definition.ApplyScaling,
storageType: definition.StorageType,
integralDivisor: definition.IntegralDivisor);
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
GrpcHistory.EnsureTagsResponse response = historyClient.EnsureTags(
new GrpcHistory.EnsureTagsRequest
{
StrHandle = session.StringHandle,
BtTagInfos = ByteString.CopyFrom(payload),
ElementCount = 1
},
connection.Metadata,
DateTime.UtcNow.Add(_options.RequestTimeout),
cancellationToken);
return response.Status?.BSuccess ?? false;
}
public Task<bool> DeleteTagAsync(string tagName, CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tagName);
return Task.Run(() => DeleteTag(tagName, cancellationToken), cancellationToken);
}
private bool DeleteTag(string tagName, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken, WriteEnabledConnectionMode);
return DeleteTagOnSession(connection, session, tagName, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): run DeleteTags against an EXTERNALLY-supplied, already-
// authenticated write-enabled (0x401) connection + session — NO Create()/handshake here. DeleteTag
// delegates so the per-call path and the reuse path share one op implementation (DRY).
internal bool DeleteTagOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
string tagName,
CancellationToken cancellationToken)
{
// DeleteTags takes the transient uint client handle (not the string handle), per the WCF wire capture.
byte[] tagNames = HistorianTagWriteProtocol.SerializeDeleteTagNames([tagName]);
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
GrpcHistory.DeleteTagsResponse response = historyClient.DeleteTags(
new GrpcHistory.DeleteTagsRequest
{
UiHandle = session.ClientHandle,
BtTagnames = ByteString.CopyFrom(tagNames)
},
connection.Metadata,
DateTime.UtcNow.Add(_options.RequestTimeout),
cancellationToken);
return response.Status?.BSuccess ?? false;
}
public Task<bool> AddTagExtendedPropertiesAsync(
string tagName, IReadOnlyList<HistorianTagExtendedProperty> properties, CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tagName);
ArgumentNullException.ThrowIfNull(properties);
if (properties.Count == 0)
{
throw new ArgumentException("At least one extended property is required.", nameof(properties));
}
return Task.Run(() => AddTagExtendedProperties(tagName, properties, cancellationToken), cancellationToken);
}
private bool AddTagExtendedProperties(
string tagName, IReadOnlyList<HistorianTagExtendedProperty> properties, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken, WriteEnabledConnectionMode);
return AddTagExtendedPropertiesOnSession(connection, session, tagName, properties, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): run AddTagExtendedProperties against an EXTERNALLY-supplied,
// already-authenticated write-enabled (0x401) connection + session — NO Create()/handshake here.
// AddTagExtendedProperties delegates so the per-call path and the reuse path share one op
// implementation (DRY).
internal bool AddTagExtendedPropertiesOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
string tagName,
IReadOnlyList<HistorianTagExtendedProperty> properties,
CancellationToken cancellationToken)
{
byte[] inBuff = HistorianTagExtendedPropertyProtocol.SerializeAddRequest(tagName, properties);
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
GrpcHistory.AddTagExtendedPropertiesResponse response = historyClient.AddTagExtendedProperties(
new GrpcHistory.AddTagExtendedPropertiesRequest
{
StrHandle = session.StringHandle,
BtTeps = ByteString.CopyFrom(inBuff)
},
connection.Metadata,
DateTime.UtcNow.Add(_options.RequestTimeout),
cancellationToken);
return response.Status?.BSuccess ?? false;
}
/// <summary>Outcome of the <see cref="ProbeDeleteTagExtendedPropertiesAsync"/> single-channel delete probe.</summary>
/// <param name="Accepted">True if the server's <c>DelTep</c> returned success.</param>
/// <param name="ErrorDescription">Decoded native error (byte0 0x84 + LE code + facility/file/message) when rejected.</param>
/// <param name="TagInfoPrimeBytes">Bytes returned by the GetTgByNm prime (tag-identity working-set load).</param>
/// <param name="ExtPropPrimePages">GetTepByNm prime pages that returned success (extended-property working-set load).</param>
internal readonly record struct DeleteTagExtendedPropertiesProbeResult(
bool Accepted, string? ErrorDescription, int TagInfoPrimeBytes, int ExtPropPrimePages);
/// <summary>
/// <b>Reverse-engineering probe (not a public op).</b> Tests whether <c>DelTep</c>
/// (DeleteTagExtendedProperties) — server-blocked on the 2020 WCF transport — succeeds over gRPC.
/// The WCF failure is structural: the server's <c>CHistStorage::DeleteTagExtendedProperties</c>
/// resolves each property from a <i>per-connection working set</i> the native client populates by
/// multiplexing <c>GetTgByNm</c> + <c>GetTepByNm</c> + <c>DelTep</c> over ONE physical connection.
/// The WCF SDK uses a separate channel per service, so the prime and the delete never share a
/// connection and the working set is empty at delete time (SErrorException). Over gRPC every service
/// client is built on the SAME <see cref="HistorianGrpcConnection.Channel"/>, so this probe runs the
/// identical native sequence — GetTgByNm prime, GetTepByNm prime, then DelTep — on one write-enabled
/// (0x401) session/channel, to see whether the multiplexed channel satisfies the working-set check.
/// Returns the decoded outcome rather than throwing so the caller can record a positive or negative
/// result. See docs/reverse-engineering/wcf-add-tag-extended-properties.md §Delete.
/// </summary>
internal Task<DeleteTagExtendedPropertiesProbeResult> ProbeDeleteTagExtendedPropertiesAsync(
string tagName, IReadOnlyList<string> propertyNames, CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tagName);
ArgumentNullException.ThrowIfNull(propertyNames);
if (propertyNames.Count == 0)
{
throw new ArgumentException("At least one property name is required.", nameof(propertyNames));
}
return Task.Run(() => ProbeDeleteTagExtendedProperties(tagName, propertyNames, cancellationToken), cancellationToken);
}
private DeleteTagExtendedPropertiesProbeResult ProbeDeleteTagExtendedProperties(
string tagName, IReadOnlyList<string> propertyNames, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken, WriteEnabledConnectionMode);
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
DateTime Deadline() => DateTime.UtcNow.Add(_options.RequestTimeout);
// Prime 1 — GetTgByNm: load the tag identity into the storage session's working set (same channel).
int tagInfoPrimeBytes = 0;
try
{
GrpcRetrieval.GetTagInfosFromNameResponse tg = retrievalClient.GetTagInfosFromName(
new GrpcRetrieval.GetTagInfosFromNameRequest
{
StrHandle = session.StringHandle,
BtTagNames = ByteString.CopyFrom(HistorianGrpcTagClient.BuildTagNamesBuffer([tagName])),
UiSequence = 0
},
connection.Metadata, Deadline(), cancellationToken);
tagInfoPrimeBytes = tg.BtTagInfos?.Length ?? 0;
}
catch
{
// Best-effort prime; the delete still runs so the error buffer is captured.
}
// Prime 2 — GetTepByNm: load the tag's extended properties into the working set (same channel),
// exactly as the native register->read->delete sequence does on its single connection.
int extPropPrimePages = 0;
byte[] tepRequest = HistorianTagExtendedPropertyProtocol.SerializeRequest(tagName);
uint sequence = 0;
for (int page = 0; page < 64; page++)
{
cancellationToken.ThrowIfCancellationRequested();
GrpcRetrieval.GetTagExtendedPropertiesFromNameResponse tep = retrievalClient.GetTagExtendedPropertiesFromName(
new GrpcRetrieval.GetTagExtendedPropertiesFromNameRequest
{
StrHandle = session.StringHandle,
BtTagNames = ByteString.CopyFrom(tepRequest),
UiSequence = sequence
},
connection.Metadata, Deadline(), cancellationToken);
if (!(tep.Status?.BSuccess ?? false))
{
break;
}
extPropPrimePages++;
if (HistorianTagExtendedPropertyProtocol.ParseResponse(tep.BtTeps?.ToByteArray() ?? []).Count == 0)
{
break;
}
sequence = tep.UiSequence;
}
// DelTep on the SAME channel/session, while the priming reads are part of the same working set.
byte[] inBuff = HistorianTagExtendedPropertyProtocol.SerializeDeleteRequest(tagName, propertyNames);
GrpcHistory.DeleteTagExtendedPropertiesResponse delete = historyClient.DeleteTagExtendedProperties(
new GrpcHistory.DeleteTagExtendedPropertiesRequest
{
StrHandle = session.StringHandle,
BtInput = ByteString.CopyFrom(inBuff)
},
connection.Metadata, Deadline(), cancellationToken);
bool accepted = delete.Status?.BSuccess ?? false;
string? errorDescription = accepted
? null
: HistorianEventRegistrationProtocol.DescribeNativeError(delete.Status?.BtError?.ToByteArray() ?? []);
return new DeleteTagExtendedPropertiesProbeResult(accepted, errorDescription, tagInfoPrimeBytes, extPropPrimePages);
}
public Task<HistorianTagRenameResult> RenameTagsAsync(
IReadOnlyList<(string OldName, string NewName)> pairs, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(pairs);
if (pairs.Count == 0)
{
throw new ArgumentException("At least one (old,new) name pair is required.", nameof(pairs));
}
foreach ((string oldName, string newName) in pairs)
{
ArgumentException.ThrowIfNullOrWhiteSpace(oldName, nameof(pairs));
ArgumentException.ThrowIfNullOrWhiteSpace(newName, nameof(pairs));
}
return Task.Run(() => RenameTags(pairs, cancellationToken), cancellationToken);
}
private HistorianTagRenameResult RenameTags(IReadOnlyList<(string OldName, string NewName)> pairs, CancellationToken cancellationToken)
{
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken, WriteEnabledConnectionMode);
return RenameTagsOnSession(connection, session, pairs, cancellationToken);
}
// Spike/Phase-1 seam (pending.md A1): run StartJob (rename) against an EXTERNALLY-supplied, already-
// authenticated write-enabled (0x401) connection + session — NO Create()/handshake here. RenameTags
// delegates so the per-call path and the reuse path share one op implementation (DRY).
internal HistorianTagRenameResult RenameTagsOnSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
IReadOnlyList<(string OldName, string NewName)> pairs,
CancellationToken cancellationToken)
{
byte[] jobBuffer = HistorianTagRenameProtocol.SerializeRenameJob(pairs);
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
GrpcHistory.StartJobResponse response = historyClient.StartJob(
new GrpcHistory.StartJobRequest
{
StrHandle = session.StringHandle,
BtInput = ByteString.CopyFrom(jobBuffer)
},
connection.Metadata,
DateTime.UtcNow.Add(_options.RequestTimeout),
cancellationToken);
bool ok = response.Status?.BSuccess ?? false;
Guid parsedJobId = Guid.Empty;
if (!string.IsNullOrWhiteSpace(response.StrJobid))
{
Guid.TryParse(response.StrJobid.Trim().Trim('$', '{', '}'), out parsedJobId);
}
return new HistorianTagRenameResult
{
Accepted = ok,
JobId = parsedJobId,
PairCount = pairs.Count,
Error = ok ? null : "Server rejected the rename job (StartJob returned false). Check that the 'AllowRenameTags' system parameter is enabled.",
};
}
}
@@ -0,0 +1,209 @@
// Recovered from HistoryService.proto (AVEVA Historian SDK 2023 R2, Archestra.Grpc.Contract).
// Reconstructed from the embedded protobuf FileDescriptor; field numbers are authoritative.
syntax = "proto3";
import "Status.proto";
option csharp_namespace = "ArchestrA.Grpc.Contract.History";
message CreateTagResponse {
bool bSuccess = 1;
bytes tagid = 2;
}
message GetInterfaceVersionRequest {
}
message GetInterfaceVersionResponse {
uint32 uiError = 1;
uint32 uiVersion = 2;
}
message OpenConnectionRequest {
bytes btConnectionRequest = 1;
}
message OpenConnectionResponse {
.Status status = 1;
bytes btConnectionResponse = 2;
}
message CloseConnectionRequest {
string strHandle = 1;
}
message CloseConnectionResponse {
.Status status = 1;
}
message UpdateClientStatusRequest {
string strHandle = 1;
bytes btClientStatus = 2;
}
message UpdateClientStatusResponse {
.Status status = 1;
bytes btServerStatus = 2;
}
message RegisterTagsRequest {
string strHandle = 1;
bytes btTagInfos = 2;
}
message RegisterTagsResponse {
.Status status = 1;
bytes btTagStatus = 2;
}
message EnsureTagsRequest {
string strHandle = 1;
bytes btTagInfos = 2;
uint32 elementCount = 3;
}
message EnsureTagsResponse {
.Status status = 1;
bytes btTagStatus = 2;
}
message AddStreamValuesRequest {
string strHandle = 1;
bytes btValues = 2;
}
message AddStreamValuesResponse {
.Status status = 1;
}
message TagExtendedProperty {
enum TagExtendedPropertyDataType {
String = 0;
Int16 = 1;
Int32 = 2;
Int64 = 3;
Double = 4;
Boolean = 5;
DateTimeOffset = 6;
Guid = 7;
Geography = 8;
Geometry = 9;
}
string PropertyName = 1;
.TagExtendedProperty.TagExtendedPropertyDataType type = 2;
bytes value = 3;
bool Facetable = 4;
bool Searchable = 5;
bool SubstringSearchable = 6;
}
message TagExtendedPropertyGroup {
string tagname = 1;
repeated .TagExtendedProperty TagExtendedProperties = 2;
}
message AddTagExtendedPropertyRequest {
string strHandle = 1;
repeated .TagExtendedPropertyGroup TagExtendedPropertyGroups = 2;
}
message AddTagExtendedPropertyResponse {
.Status status = 1;
}
message ExchangeKeyRequest {
string strHandle = 1;
bytes btInput = 2;
}
message ExchangeKeyResponse {
.Status status = 1;
bytes btOutput = 2;
}
message StartJobRequest {
string strHandle = 1;
bytes btInput = 2;
}
message StartJobResponse {
.Status status = 1;
string strJobid = 2;
}
message GetJobStatusRequest {
string strHandle = 1;
string strJobid = 2;
}
message GetJobStatusResponse {
.Status status = 1;
bytes btJobStatus = 2;
}
message AddTagExtendedPropertiesRequest {
string strHandle = 1;
bytes btTeps = 2;
}
message AddTagExtendedPropertiesResponse {
.Status status = 1;
}
message DeleteTagExtendedPropertiesRequest {
string strHandle = 1;
bytes btInput = 2;
}
message DeleteTagExtendedPropertiesResponse {
.Status status = 1;
}
message DeleteTagsRequest {
uint32 uiHandle = 1;
bytes btTagnames = 2;
}
message DeleteTagsResponse {
.Status status = 1;
bytes btDeleteTagStatus = 2;
}
message AddTagLocalizedPropertiesRequest {
string strHandle = 1;
bytes btInput = 2;
}
message AddTagLocalizedPropertiesResponse {
.Status status = 1;
}
message DeleteTagLocalizedPropertiesRequest {
string strHandle = 1;
bytes btInput = 2;
}
message DeleteTagLocalizedPropertiesResponse {
.Status status = 1;
}
service HistoryService {
rpc GetInterfaceVersion (.GetInterfaceVersionRequest) returns (.GetInterfaceVersionResponse);
rpc ExchangeKey (.ExchangeKeyRequest) returns (.ExchangeKeyResponse);
rpc OpenConnection (.OpenConnectionRequest) returns (.OpenConnectionResponse);
rpc CloseConnection (.CloseConnectionRequest) returns (.CloseConnectionResponse);
rpc UpdateClientStatus (.UpdateClientStatusRequest) returns (.UpdateClientStatusResponse);
rpc RegisterTags (.RegisterTagsRequest) returns (.RegisterTagsResponse);
rpc EnsureTags (.EnsureTagsRequest) returns (.EnsureTagsResponse);
rpc AddStreamValues (.AddStreamValuesRequest) returns (.AddStreamValuesResponse);
rpc AddTagExtendedPropertyGroups (.AddTagExtendedPropertyRequest) returns (.AddTagExtendedPropertyResponse);
rpc AddTagExtendedProperties (.AddTagExtendedPropertiesRequest) returns (.AddTagExtendedPropertiesResponse);
rpc StartJob (.StartJobRequest) returns (.StartJobResponse);
rpc GetJobStatus (.GetJobStatusRequest) returns (.GetJobStatusResponse);
rpc DeleteTagExtendedProperties (.DeleteTagExtendedPropertiesRequest) returns (.DeleteTagExtendedPropertiesResponse);
rpc DeleteTags (.DeleteTagsRequest) returns (.DeleteTagsResponse);
rpc AddTagLocalizedProperties (.AddTagLocalizedPropertiesRequest) returns (.AddTagLocalizedPropertiesResponse);
rpc DeleteTagLocalizedProperties (.DeleteTagLocalizedPropertiesRequest) returns (.DeleteTagLocalizedPropertiesResponse);
}
@@ -0,0 +1,186 @@
// Recovered from RetrievalService.proto (AVEVA Historian SDK 2023 R2, Archestra.Grpc.Contract).
// Reconstructed from the embedded protobuf FileDescriptor; field numbers are authoritative.
syntax = "proto3";
import "Status.proto";
option csharp_namespace = "ArchestrA.Grpc.Contract.Retrieval";
message GetRetrievalInterfaceVersionRequest {
}
message GetRetrievalInterfaceVersionResponse {
uint32 uiError = 1;
uint32 uiVersion = 2;
}
message StartQueryRequest {
uint32 uiHandle = 1;
uint32 uiQueryRequestType = 2;
bytes btRequestBuffer = 3;
}
message StartQueryResponse {
.Status status = 1;
uint32 uiQueryHandle = 2;
bytes btResponseBuffer = 3;
}
message GetNextQueryResultBufferRequest {
uint32 uiHandle = 1;
uint32 uiQueryHandle = 2;
}
message GetNextQueryResultBufferResponse {
.Status status = 1;
bytes btQueryResult = 2;
}
message EndQueryRequest {
uint32 uiHandle = 1;
uint32 uiQueryHandle = 2;
}
message EndQueryResponse {
.Status status = 1;
}
message GetShardTagidsByTagnameAndSourceRequest {
string strHandle = 1;
bytes btTagnameAndSource = 2;
}
message GetShardTagidsByTagnameAndSourceResponse {
.Status status = 1;
bytes btShardTagids = 2;
}
message GetTagInfosFromNameRequest {
string strHandle = 1;
bytes btTagNames = 2;
uint32 uiSequence = 3;
}
message GetTagInfosFromNameResponse {
.Status status = 1;
bytes btTagInfos = 2;
uint32 uiSequence = 3;
}
message GetTagExtendedPropertiesFromNameRequest {
string strHandle = 1;
bytes btTagNames = 2;
uint32 uiSequence = 3;
}
message GetTagExtendedPropertiesFromNameResponse {
.Status status = 1;
bytes btTeps = 2;
uint32 uiSequence = 3;
}
message ExecuteSqlCommandRequest {
string strHandle = 1;
string StrCommand = 2;
uint32 uiOption = 3;
uint32 uiQueryHandle = 4;
}
message ExecuteSqlCommandResponse {
.Status status = 1;
int32 iRetValue = 2;
uint32 uiQueryHandle = 3;
}
message StartEventQueryRequest {
uint32 uiHandle = 1;
uint32 uiQueryRequestType = 2;
bytes btRequest = 3;
uint32 uiQueryHandle = 4;
}
message StartEventQueryResponse {
.Status status = 1;
uint32 uiQueryHandle = 2;
bytes btResonse = 3;
}
message GetNextEventQueryResultBufferRequest {
uint32 uiHandle = 1;
uint32 uiQueryHandle = 2;
}
message GetNextEventQueryResultBufferResponse {
.Status status = 1;
bytes btResult = 2;
}
message EndEventQueryRequest {
uint32 uiHandle = 1;
uint32 uiQueryHandle = 2;
}
message EndEventQueryResponse {
.Status status = 1;
}
message StartTagQueryRequest {
string strHandle = 1;
bytes btRequest = 2;
}
message StartTagQueryResponse {
.Status status = 1;
bytes btResponse = 2;
}
message QueryTagRequest {
string strHandle = 1;
uint32 uiQueryHandle = 2;
bytes btRequest = 3;
}
message QueryTagResponse {
.Status status = 1;
bytes btResonse = 2;
}
message EndTagQueryRequest {
string strHandle = 1;
uint32 uiQueryHandle = 2;
}
message EndTagQueryResponse {
.Status status = 1;
}
message GetTagLocalizedPropertiesFromNameRequest {
string strHandle = 1;
bytes btTagNames = 2;
uint32 uiSequence = 3;
}
message GetTagLocalizedPropertiesFromNameResponse {
.Status status = 1;
uint32 uiSequence = 2;
bytes btOutBuffer = 3;
}
service RetrievalService {
rpc GetRetrievalInterfaceVersion (.GetRetrievalInterfaceVersionRequest) returns (.GetRetrievalInterfaceVersionResponse);
rpc StartQuery (.StartQueryRequest) returns (.StartQueryResponse);
rpc GetNextQueryResultBuffer (.GetNextQueryResultBufferRequest) returns (.GetNextQueryResultBufferResponse);
rpc EndQuery (.EndQueryRequest) returns (.EndQueryResponse);
rpc GetShardTagidsByTagnameAndSource (.GetShardTagidsByTagnameAndSourceRequest) returns (.GetShardTagidsByTagnameAndSourceResponse);
rpc GetTagInfosFromName (.GetTagInfosFromNameRequest) returns (.GetTagInfosFromNameResponse);
rpc GetTagExtendedPropertiesFromName (.GetTagExtendedPropertiesFromNameRequest) returns (.GetTagExtendedPropertiesFromNameResponse);
rpc ExecuteSqlCommand (.ExecuteSqlCommandRequest) returns (.ExecuteSqlCommandResponse);
rpc StartEventQuery (.StartEventQueryRequest) returns (.StartEventQueryResponse);
rpc GetNextEventQueryResultBuffer (.GetNextEventQueryResultBufferRequest) returns (.GetNextEventQueryResultBufferResponse);
rpc EndEventQuery (.EndEventQueryRequest) returns (.EndEventQueryResponse);
rpc StartTagQuery (.StartTagQueryRequest) returns (.StartTagQueryResponse);
rpc QueryTag (.QueryTagRequest) returns (.QueryTagResponse);
rpc EndTagQuery (.EndTagQueryRequest) returns (.EndTagQueryResponse);
rpc GetTagLocalizedPropertiesFromName (.GetTagLocalizedPropertiesFromNameRequest) returns (.GetTagLocalizedPropertiesFromNameResponse);
}
@@ -0,0 +1,12 @@
// Recovered from Status.proto (AVEVA Historian SDK 2023 R2, Archestra.Grpc.Contract).
// Reconstructed from the embedded protobuf FileDescriptor; field numbers are authoritative.
syntax = "proto3";
option csharp_namespace = "ArchestrA.Grpc.Contract.RequestStatus";
message Status {
bool bSuccess = 1;
bytes btError = 2;
}
@@ -0,0 +1,215 @@
// Recovered from StatusService.proto (AVEVA Historian SDK 2023 R2, Archestra.Grpc.Contract).
// Reconstructed from the embedded protobuf FileDescriptor; field numbers are authoritative.
syntax = "proto3";
import "Status.proto";
option csharp_namespace = "ArchestrA.Grpc.Contract.Status";
message GetStatusInterfaceVersionRequest {
}
message GetStatusInterfaceVersionResponse {
uint32 uiError = 1;
uint32 uiVersion = 2;
}
message GetSystemParameterRequest {
uint32 uiHandle = 1;
string strParameterName = 2;
}
message GetSystemParameterResponse {
.Status status = 1;
string strParameterValue = 2;
}
message SendInfoRequest {
string strHandle = 1;
string strPipeName = 2;
uint32 uiOption = 3;
bytes btReqBuff = 4;
string strInfoID = 5;
}
message SendInfoResponse {
.Status status = 1;
string strInfoID = 2;
bytes btRespBuff = 3;
}
message RequestInfoRequest {
string strHandle = 1;
string strInfoID = 2;
uint32 uiOffset = 3;
}
message RequestInfoResponse {
.Status status = 1;
bytes btRespBuff = 2;
}
message DeleteInfoRequest {
string strHandle = 1;
string strInfoID = 2;
}
message DeleteInfoResponse {
.Status status = 1;
}
message GetHistorianInfoRequest {
string strHandle = 1;
bytes btRequest = 2;
}
message GetHistorianInfoResponse {
.Status status = 1;
bytes btHistorianInfo = 2;
}
message StartProcessRequest {
string strHandle = 1;
string strPipeName = 2;
string strPath = 3;
string strAuguments = 4;
uint32 uiKeepAliveInterval = 5;
uint32 uiKeepAliveMethod = 6;
}
message StartProcessResponse {
.Status status = 1;
}
message StopProcessRequest {
string strHandle = 1;
string StrPipeName = 2;
}
message StopProcessResponse {
.Status status = 1;
}
message PingServerRequest {
string strHandle = 1;
string strPipeName = 2;
uint32 uiTimeout = 3;
}
message PingServerResponse {
.Status status = 1;
}
message PingPipeRequest {
string strHandle = 1;
string strPipeName = 2;
}
message PingPipeResponse {
.Status status = 1;
}
message ConfigureAutoStartProcessRequest {
string strHandle = 1;
string strPipeName = 2;
string strPath = 3;
string strAuguments = 4;
uint32 uiKeepAliveInterval = 5;
uint32 uiKeepAliveMethod = 6;
uint32 uiStartupFlags = 7;
}
message ConfigureAutoStartProcessResponse {
.Status status = 1;
}
message GetHistorianConsoleStatusRequest {
string strHandle = 1;
}
message GetHistorianConsoleStatusResponse {
.Status status = 1;
uint32 uiConsoleStatus = 2;
}
message GetRuntimeParameterRequest {
string strHandle = 1;
bytes btRequest = 2;
}
message GetRuntimeParameterResponse {
.Status status = 1;
bytes btResponse = 2;
}
message GetSystemTimeZoneNameRequest {
uint32 uiHandle = 1;
}
message GetSystemTimeZoneNameResponse {
.Status status = 1;
string strSystemTimeZoneName = 2;
}
message SetHistorianConsoleStatusRequest {
string strHandle = 1;
uint32 uiStatus = 2;
uint32 uiOption = 3;
}
message SetHistorianConsoleStatusResponse {
.Status status = 1;
}
message CanUpdateAreaHierarchyRequest {
uint32 uiHandle = 1;
}
message CanUpdateAreaHierarchyResponse {
.Status status = 1;
bool canUpdate = 2;
}
message UpdateAreaHierarchyRequest {
uint32 uiHandle = 1;
string guid = 2;
uint32 sequence = 3;
bytes buffer = 4;
}
message UpdateAreaHierarchyResponse {
.Status status = 1;
}
message UpdateObjectHierarchyRequest {
uint32 uiHandle = 1;
string guid = 2;
uint32 sequence = 3;
bytes buffer = 4;
}
message UpdateObjectHierarchyResponse {
.Status status = 1;
}
service StatusService {
rpc GetStatusInterfaceVersion (.GetStatusInterfaceVersionRequest) returns (.GetStatusInterfaceVersionResponse);
rpc GetSystemParameter (.GetSystemParameterRequest) returns (.GetSystemParameterResponse);
rpc SendInfo (.SendInfoRequest) returns (.SendInfoResponse);
rpc RequestInfo (.RequestInfoRequest) returns (.RequestInfoResponse);
rpc DeleteInfo (.DeleteInfoRequest) returns (.DeleteInfoResponse);
rpc GetHistorianInfo (.GetHistorianInfoRequest) returns (.GetHistorianInfoResponse);
rpc StartProcess (.StartProcessRequest) returns (.StartProcessResponse);
rpc StopProcess (.StopProcessRequest) returns (.StopProcessResponse);
rpc PingServer (.PingServerRequest) returns (.PingServerResponse);
rpc PingPipe (.PingPipeRequest) returns (.PingPipeResponse);
rpc ConfigureAutoStartProcess (.ConfigureAutoStartProcessRequest) returns (.ConfigureAutoStartProcessResponse);
rpc GetHistorianConsoleStatus (.GetHistorianConsoleStatusRequest) returns (.GetHistorianConsoleStatusResponse);
rpc GetRuntimeParameter (.GetRuntimeParameterRequest) returns (.GetRuntimeParameterResponse);
rpc GetSystemTimeZoneName (.GetSystemTimeZoneNameRequest) returns (.GetSystemTimeZoneNameResponse);
rpc SetHistorianConsoleStatus (.SetHistorianConsoleStatusRequest) returns (.SetHistorianConsoleStatusResponse);
rpc CanUpdateAreaHierarchy (.CanUpdateAreaHierarchyRequest) returns (.CanUpdateAreaHierarchyResponse);
rpc UpdateAreaHierarchy (.UpdateAreaHierarchyRequest) returns (.UpdateAreaHierarchyResponse);
rpc UpdateObjectHierarchy (.UpdateObjectHierarchyRequest) returns (.UpdateObjectHierarchyResponse);
}
@@ -0,0 +1,417 @@
// Recovered from StorageService.proto (AVEVA Historian SDK 2023 R2, Archestra.Grpc.Contract).
// Reconstructed from the embedded protobuf FileDescriptor; field numbers are authoritative.
syntax = "proto3";
import "Status.proto";
option csharp_namespace = "ArchestrA.Grpc.Contract.Storage";
message GetInterfaceVersionRequest {
}
message GetInterfaceVersionResponse {
uint32 uiError = 1;
uint32 uiVersion = 2;
}
message OpenStorageConnectionRequest {
string HostName = 1;
string EnginePath = 2;
uint32 FreeDiskSpace = 3;
string ProcessName = 4;
uint32 ProcessId = 5;
string UserName = 6;
bytes Password = 7;
uint32 PwdLength = 8;
uint32 ClientType = 9;
uint32 ClientVersion = 10;
uint32 ConnectionMode = 11;
uint32 ConnectionTimeout = 12;
string StorageSessionId = 13;
}
message OpenStorageConnectionResponse {
.Status status = 1;
string StorageSessionId = 2;
uint32 Handle = 3;
uint64 ConnectionTime = 4;
uint32 ServerStatus = 5;
}
message CloseStorageConnectionRequest {
uint32 Handle = 1;
}
message CloseStorageConnectionResponse {
.Status status = 1;
}
message PingRequest {
uint32 Handle = 1;
}
message PingResponse {
.Status status = 1;
uint32 OutByteCount = 2;
bytes OutBuff = 3;
}
message AddTagsRequest {
uint32 Handle = 1;
uint32 ElementCount = 2;
uint32 InByteCount = 3;
bytes InBuff = 4;
}
message AddTagsResponse {
.Status status = 1;
uint32 OutByteCount = 2;
bytes OutBuff = 3;
}
message RegisterTagsRequest {
uint32 Handle = 1;
uint32 ElementCount = 2;
uint32 InByteCount = 3;
bytes InBuff = 4;
}
message RegisterTagsResponse {
.Status status = 1;
uint32 OutByteCount = 2;
bytes OutBuff = 3;
}
message AddStreamValuesRequest {
uint32 Handle = 1;
uint32 Size = 2;
bytes Buffer = 3;
}
message AddStreamValuesResponse {
.Status status = 1;
}
message GetTagIdsRequest {
uint32 Handle = 1;
uint32 Sequence = 2;
}
message GetTagIdsResponse {
.Status status = 1;
uint32 Sequence = 2;
uint32 Size = 3;
bytes TagIds = 4;
}
message GetTagsRequest {
uint32 Handle = 1;
uint32 TagIdsSize = 2;
bytes TagIds = 3;
uint32 Sequence = 4;
}
message GetTagsResponse {
.Status status = 1;
uint32 Sequence = 2;
uint32 TagInfosSize = 3;
bytes TagInfos = 4;
}
message FlushMetadataRequest {
uint32 Handle = 1;
uint32 TagIdsSize = 2;
bytes TagIds = 3;
}
message FlushMetadataResponse {
.Status status = 1;
}
message FlushDataRequest {
uint32 Handle = 1;
}
message FlushDataResponse {
.Status status = 1;
}
message LoadBlocksRequest {
uint32 Handle = 1;
uint32 Sequence = 2;
}
message LoadBlocksResponse {
.Status status = 1;
uint32 Sequence = 2;
uint32 HistoryBlockSize = 3;
bytes HistoryBlocks = 4;
}
message GetSnapshotsRequest {
uint32 Handle = 1;
uint64 BlockStartTime = 2;
uint32 Sequence = 3;
}
message GetSnapshotsResponse {
.Status status = 1;
uint32 Sequence = 2;
uint32 SnapshotSize = 3;
bytes Snapshot = 4;
}
message StartQuerySnapshotRequest {
uint32 Handle = 1;
uint64 BlockStartTime = 2;
uint32 SnapshotInfoSize = 3;
bytes SnapshotInfo = 4;
uint32 SnapshotQueryId = 5;
}
message StartQuerySnapshotResponse {
.Status status = 1;
uint32 SnapshotQueryId = 2;
}
message NextQuerySnapshotRequest {
uint32 Handle = 1;
uint32 SnapshotQueryId = 2;
uint32 Sequence = 3;
}
message NextQuerySnapshotResponse {
.Status status = 1;
uint32 Sequence = 2;
uint32 SnapshotSize = 3;
bytes Snapshot = 4;
}
message EndSnapshotRequest {
uint32 Handle = 1;
uint32 SnapshotQueryId = 2;
uint64 BlockStartTime = 3;
uint32 SnapshotInfoSize = 4;
bytes SnapshotInfo = 5;
bool IsDeleteSnapshot = 6;
}
message EndSnapshotResponse {
.Status status = 1;
}
message StopRequest {
uint32 Handle = 1;
}
message StopResponse {
.Status status = 1;
}
message ClearTagidPairsRequest {
uint32 Handle = 1;
}
message ClearTagidPairsResponse {
.Status status = 1;
}
message AddTagidPairsRequest {
uint32 Handle = 1;
uint32 ElementCount = 2;
uint32 InByteCount = 3;
bytes InBuff = 4;
}
message AddTagidPairsResponse {
.Status status = 1;
}
message GetSFParameterRequest {
uint32 Handle = 1;
string ParameterName = 2;
}
message GetSFParameterResponse {
.Status status = 1;
string ParamaterValue = 2;
}
message SetSFParameterRequest {
uint32 Handle = 1;
string ParamaterName = 2;
string ParamaterValue = 3;
}
message SetSFParameterResponse {
.Status status = 1;
}
message SendSnapshotBeginRequest {
uint32 Handle = 1;
uint64 TotalSize = 2;
uint64 StartTime = 3;
uint64 EndTime = 4;
string StorageSessionId = 5;
}
message SendSnapshotBeginResponse {
.Status status = 1;
string StorageSessionId = 2;
uint32 QueryId = 3;
}
message SendSnapshotEndRequest {
uint32 Handle = 1;
string StorageSessionId = 2;
uint32 QueryId = 3;
uint32 TimeRangeSize = 4;
bytes TimeRangeBytes = 5;
}
message SendSnapshotEndResponse {
.Status status = 1;
}
message SendSnapshotRequest {
uint32 Handle = 1;
string StorageSessionId = 2;
uint32 QueryId = 3;
uint32 Size = 4;
uint64 SnapShotChunkOffset = 5;
bytes Buffer = 6;
}
message SendSnapshotResponse {
.Status status = 1;
}
message DeleteSnapshotRequest {
uint32 Handle = 1;
uint64 StartTime = 2;
uint32 SnapshotInfoSize = 3;
bytes SnapshotInfo = 4;
}
message DeleteSnapshotResponse {
.Status status = 1;
}
message AddStreamValues2Request {
uint32 Handle = 1;
string ShardId = 2;
bytes Buffer = 3;
}
message AddStreamValues2Response {
.Status status = 1;
}
message ClearShardTagidsRequest {
uint32 Handle = 1;
}
message ClearShardTagidsResponse {
.Status status = 1;
}
message AddShardTagidsRequest {
uint32 Handle = 1;
bytes Buffer = 2;
}
message AddShardTagidsResponse {
.Status status = 1;
}
message SplitUnknownShardsRequest {
uint32 Handle = 1;
}
message SplitUnknownShardsResponse {
.Status status = 1;
}
message GetRemainingSnapshotsSizeRequest {
uint32 Handle = 1;
}
message GetRemainingSnapshotsSizeResponse {
.Status status = 1;
uint64 SnapshotSize = 2;
}
message DeleteTagsRequest {
uint32 Handle = 1;
bytes Buffer = 2;
}
message DeleteTagsResponse {
.Status status = 1;
}
message OpenStorageConnection2Request {
bytes InParameters = 1;
}
message OpenStorageConnection2Response {
.Status status = 1;
bytes OutParmaters = 2;
}
message ValidateClientCredentialRequest {
string Handle = 1;
bytes InBuff = 2;
}
message ValidateClientCredentialResponse {
.Status status = 1;
bytes OutBuff = 2;
}
message GetInfoRequest {
string Request = 1;
}
message GetInfoResponse {
.Status status = 1;
bytes info = 2;
}
service StorageService {
rpc GetInterfaceVersion (.GetInterfaceVersionRequest) returns (.GetInterfaceVersionResponse);
rpc OpenStorageConnection (.OpenStorageConnectionRequest) returns (.OpenStorageConnectionResponse);
rpc CloseStorageConnection (.CloseStorageConnectionRequest) returns (.CloseStorageConnectionResponse);
rpc Ping (.PingRequest) returns (.PingResponse);
rpc AddTags (.AddTagsRequest) returns (.AddTagsResponse);
rpc RegisterTags (.RegisterTagsRequest) returns (.RegisterTagsResponse);
rpc AddStreamValues (.AddStreamValuesRequest) returns (.AddStreamValuesResponse);
rpc GetTagIds (.GetTagIdsRequest) returns (.GetTagIdsResponse);
rpc GetTags (.GetTagsRequest) returns (.GetTagsResponse);
rpc FlushMetadata (.FlushMetadataRequest) returns (.FlushMetadataResponse);
rpc FlushData (.FlushDataRequest) returns (.FlushDataResponse);
rpc LoadBlocks (.LoadBlocksRequest) returns (.LoadBlocksResponse);
rpc GetSnapshots (.GetSnapshotsRequest) returns (.GetSnapshotsResponse);
rpc StartQuerySnapshot (.StartQuerySnapshotRequest) returns (.StartQuerySnapshotResponse);
rpc NextQuerySnapshot (.NextQuerySnapshotRequest) returns (.NextQuerySnapshotResponse);
rpc EndSnapshot (.EndSnapshotRequest) returns (.EndSnapshotResponse);
rpc Stop (.StopRequest) returns (.StopResponse);
rpc ClearTagidPairs (.ClearTagidPairsRequest) returns (.ClearTagidPairsResponse);
rpc AddTagidPairs (.AddTagidPairsRequest) returns (.AddTagidPairsResponse);
rpc GetSFParameter (.GetSFParameterRequest) returns (.GetSFParameterResponse);
rpc SetSFParameter (.SetSFParameterRequest) returns (.SetSFParameterResponse);
rpc SendSnapshotBegin (.SendSnapshotBeginRequest) returns (.SendSnapshotBeginResponse);
rpc SendSnapshotEnd (.SendSnapshotEndRequest) returns (.SendSnapshotEndResponse);
rpc SendSnapshot (.SendSnapshotRequest) returns (.SendSnapshotResponse);
rpc DeleteSnapshot (.DeleteSnapshotRequest) returns (.DeleteSnapshotResponse);
rpc AddStreamValues2 (.AddStreamValues2Request) returns (.AddStreamValues2Response);
rpc ClearShardTagids (.ClearShardTagidsRequest) returns (.ClearShardTagidsResponse);
rpc AddShardTagids (.AddShardTagidsRequest) returns (.AddShardTagidsResponse);
rpc SplitUnknownShards (.SplitUnknownShardsRequest) returns (.SplitUnknownShardsResponse);
rpc GetRemainingSnapshotsSize (.GetRemainingSnapshotsSizeRequest) returns (.GetRemainingSnapshotsSizeResponse);
rpc DeleteTags (.DeleteTagsRequest) returns (.DeleteTagsResponse);
rpc OpenStorageConnection2 (.OpenStorageConnection2Request) returns (.OpenStorageConnection2Response);
rpc ValidateClientCredential (.ValidateClientCredentialRequest) returns (.ValidateClientCredentialResponse);
rpc GetInfo (.GetInfoRequest) returns (.GetInfoResponse);
}
@@ -0,0 +1,92 @@
// Recovered from TransactionService.proto (AVEVA Historian SDK 2023 R2, Archestra.Grpc.Contract).
// Reconstructed from the embedded protobuf FileDescriptor; field numbers are authoritative.
syntax = "proto3";
import "Status.proto";
option csharp_namespace = "ArchestrA.Grpc.Contract.Transaction";
message ForwardSnapshotRequest {
string strHandle = 1;
string strSessionID = 2;
uint32 queryID = 3;
uint64 snapShotChunkOffset = 4;
bytes btInput = 5;
}
message ForwardSnapshotResponse {
.Status status = 1;
}
message ForwardSnapshotBeginRequest {
string strHandle = 1;
uint64 totalSize = 2;
uint64 startTime = 3;
uint64 endTime = 4;
}
message ForwardSnapshotBeginResponse {
string strSessionID = 1;
uint32 queryID = 2;
.Status status = 3;
}
message ForwardSnapshotEndRequest {
string strHandle = 1;
string strSessionID = 2;
uint32 queryID = 3;
bytes timeRange = 4;
}
message ForwardSnapshotEndResponse {
bytes tagIds = 1;
.Status status = 2;
}
message GetTransactionInterfaceVersionRequest {
}
message GetTransactionInterfaceVersionResponse {
uint32 error = 1;
uint32 version = 2;
}
message AddNonStreamValuesBeginRequest {
string strHandle = 1;
}
message AddNonStreamValuesBeginResponse {
.Status status = 1;
string strTransactionId = 2;
}
message AddNonStreamValuesRequest {
string strHandle = 1;
string strTransactionId = 2;
bytes btInput = 3;
}
message AddNonStreamValuesResponse {
.Status status = 1;
}
message AddNonStreamValuesEndRequest {
string strHandle = 1;
string strTransactionId = 2;
bool bCommit = 3;
}
message AddNonStreamValuesEndResponse {
.Status status = 1;
}
service TransactionService {
rpc ForwardSnapshot (.ForwardSnapshotRequest) returns (.ForwardSnapshotResponse);
rpc ForwardSnapshotBegin (.ForwardSnapshotBeginRequest) returns (.ForwardSnapshotBeginResponse);
rpc ForwardSnapshotEnd (.ForwardSnapshotEndRequest) returns (.ForwardSnapshotEndResponse);
rpc GetTransactionInterfaceVersion (.GetTransactionInterfaceVersionRequest) returns (.GetTransactionInterfaceVersionResponse);
rpc AddNonStreamValuesBegin (.AddNonStreamValuesBeginRequest) returns (.AddNonStreamValuesBeginResponse);
rpc AddNonStreamValues (.AddNonStreamValuesRequest) returns (.AddNonStreamValuesResponse);
rpc AddNonStreamValuesEnd (.AddNonStreamValuesEndRequest) returns (.AddNonStreamValuesEndResponse);
}
+269 -12
View File
@@ -25,7 +25,9 @@ public sealed class HistorianClient : IAsyncDisposable
public async Task<bool> ProbeAsync(CancellationToken cancellationToken = default) public async Task<bool> ProbeAsync(CancellationToken cancellationToken = default)
{ {
return await HistorianWcfProbe.ProbeAsync(_options, cancellationToken).ConfigureAwait(false); return _options.Transport == HistorianTransport.RemoteGrpc
? await Grpc.HistorianGrpcProbe.ProbeAsync(_options, cancellationToken).ConfigureAwait(false)
: await HistorianWcfProbe.ProbeAsync(_options, cancellationToken).ConfigureAwait(false);
} }
public IAsyncEnumerable<HistorianSample> ReadRawAsync( public IAsyncEnumerable<HistorianSample> ReadRawAsync(
@@ -90,19 +92,90 @@ public sealed class HistorianClient : IAsyncDisposable
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
ValidateTimeRange(startUtc, endUtc); ValidateTimeRange(startUtc, endUtc);
return _protocol.ReadEventsAsync(startUtc, endUtc, cancellationToken); return _protocol.ReadEventsAsync(startUtc, endUtc, filter: null, cancellationToken);
}
/// <summary>
/// Reads events in the time window, server-filtered by a single predicate
/// (<paramref name="filter"/>) — e.g. <c>Type Equal "User.Write"</c> or
/// <c>Area Contains "Tank"</c>. The historian applies the filter and returns only matching
/// events. Filtering is a real server-side operation (live-verified: a non-matching predicate
/// returns zero events). Single string-valued predicates only; see <see cref="HistorianEventFilter"/>.
/// </summary>
public IAsyncEnumerable<HistorianEvent> ReadEventsAsync(
DateTime startUtc,
DateTime endUtc,
HistorianEventFilter filter,
CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(filter);
ValidateTimeRange(startUtc, endUtc);
return _protocol.ReadEventsAsync(startUtc, endUtc, filter, cancellationToken);
}
/// <summary>
/// Sends a single <see cref="HistorianEvent"/> to the Historian's built-in CM_EVENT tag.
/// Over WCF this runs Open2 event mode → CM_EVENT registration → AddS2; over the 2023 R2
/// <see cref="HistorianTransport.RemoteGrpc"/> transport it runs the captured-equivalent
/// v8 Event OpenConnection → CM_EVENT registration → <c>HistoryService.AddStreamValues</c>
/// with the same "OS" event buffer (live-captured 2026-06-23 — the send rides the same RPC
/// and buffer as the WCF path, not a distinct event RPC). The event is appended to the
/// historian's event history and is readable back via <see cref="ReadEventsAsync"/> /
/// the <c>v_AlarmEventHistory2</c> view. Only original events
/// (<see cref="HistorianEvent.RevisionVersion"/> = 0) with string-valued properties are
/// supported; other property value types and revision/update/delete events throw
/// <see cref="ProtocolEvidenceMissingException"/> until their wire encoding is captured.
/// </summary>
public Task<bool> SendEventAsync(HistorianEvent historianEvent, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(historianEvent);
return _options.Transport == HistorianTransport.RemoteGrpc
? new Grpc.HistorianGrpcEventWriteOrchestrator(_options).SendEventAsync(historianEvent, cancellationToken)
: new HistorianWcfEventOrchestrator(_options).SendEventAsync(historianEvent, cancellationToken);
}
/// <summary>
/// Inserts historical (non-streamed original / backfill) values for an existing tag. Captured
/// live from the native 2023 R2 client: the write rides <c>HistoryService.AddStreamValues</c>
/// (an "ON" storage-sample buffer) over the gRPC front door — see
/// <c>docs/plans/revision-write-path.md</c> §"R3.1 CAPTURED". Only the
/// <see cref="HistorianTransport.RemoteGrpc"/> transport is supported (the 2020 WCF path is
/// architecturally blocked — D2); other transports throw
/// <see cref="ProtocolEvidenceMissingException"/>. The tag must already exist
/// (create it with <see cref="EnsureTagAsync"/>). Value encoding is captured for Float tags.
/// </summary>
public Task<bool> AddHistoricalValuesAsync(
string tag,
IReadOnlyList<HistorianHistoricalValue> values,
CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tag);
ArgumentNullException.ThrowIfNull(values);
if (_options.Transport != HistorianTransport.RemoteGrpc)
{
throw new ProtocolEvidenceMissingException(
"AddHistoricalValuesAsync is only supported over the 2023 R2 RemoteGrpc transport; the 2020 WCF " +
"non-streamed write is architecturally blocked (see docs/plans/revision-write-path.md, D2).");
}
return new Grpc.HistorianGrpcHistoricalWriteOrchestrator(_options).AddHistoricalValuesAsync(tag, values, cancellationToken);
} }
public IAsyncEnumerable<string> BrowseTagNamesAsync(string filter = "*", CancellationToken cancellationToken = default) public IAsyncEnumerable<string> BrowseTagNamesAsync(string filter = "*", CancellationToken cancellationToken = default)
{ {
ArgumentException.ThrowIfNullOrWhiteSpace(filter); ArgumentException.ThrowIfNullOrWhiteSpace(filter);
return HistorianWcfTagClient.BrowseTagNamesAsync(_options, filter, cancellationToken); return _options.Transport == HistorianTransport.RemoteGrpc
? Grpc.HistorianGrpcTagClient.BrowseTagNamesAsync(_options, filter, cancellationToken)
: HistorianWcfTagClient.BrowseTagNamesAsync(_options, filter, cancellationToken);
} }
public Task<HistorianTagMetadata?> GetTagMetadataAsync(string tag, CancellationToken cancellationToken = default) public Task<HistorianTagMetadata?> GetTagMetadataAsync(string tag, CancellationToken cancellationToken = default)
{ {
ArgumentException.ThrowIfNullOrWhiteSpace(tag); ArgumentException.ThrowIfNullOrWhiteSpace(tag);
return HistorianWcfTagClient.GetTagMetadataAsync(_options, tag, cancellationToken); return _options.Transport == HistorianTransport.RemoteGrpc
? Grpc.HistorianGrpcTagClient.GetTagMetadataAsync(_options, tag, cancellationToken)
: HistorianWcfTagClient.GetTagMetadataAsync(_options, tag, cancellationToken);
} }
public Task<HistorianConnectionStatus> GetConnectionStatusAsync(CancellationToken cancellationToken = default) public Task<HistorianConnectionStatus> GetConnectionStatusAsync(CancellationToken cancellationToken = default)
@@ -121,6 +194,99 @@ public sealed class HistorianClient : IAsyncDisposable
return _protocol.GetSystemParameterAsync(name, cancellationToken); return _protocol.GetSystemParameterAsync(name, cancellationToken);
} }
/// <summary>
/// Reads the Historian server's system time-zone name (e.g. "Eastern Daylight Time").
/// <para>
/// Only the 2023 R2 <see cref="HistorianTransport.RemoteGrpc"/> front door exposes a real value;
/// the 2020 WCF <c>GetSystemTimeZoneName</c> is a client-side stub, so this throws
/// <see cref="ProtocolEvidenceMissingException"/> on the non-gRPC transports. Returns null when a
/// gRPC server reports no value.
/// </para>
/// </summary>
public Task<string?> GetServerTimeZoneAsync(CancellationToken cancellationToken = default)
{
return _protocol.GetServerTimeZoneAsync(cancellationToken);
}
/// <summary>
/// Reads a named Historian <em>runtime</em> parameter (the live server state surface,
/// distinct from the configuration <see cref="GetSystemParameterAsync"/>). Returns the
/// string value, or null when the server reports no value. Single string-valued parameters
/// only (the evidence-backed surface); see <c>HistorianRuntimeParameterProtocol</c>.
/// </summary>
public Task<string?> GetRuntimeParameterAsync(string name, CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(name);
return _protocol.GetRuntimeParameterAsync(name, cancellationToken);
}
/// <summary>
/// Reads the extended (user-defined) properties attached to a tag via the 2020 WCF
/// <c>GetTepByNm</c> op. Returns the property name/value pairs (empty when the tag has none).
/// String-valued properties only (the evidence-backed surface); other value variants throw
/// <see cref="ProtocolEvidenceMissingException"/>. See
/// <c>HistorianTagExtendedPropertyProtocol</c>.
/// </summary>
public Task<IReadOnlyList<HistorianTagExtendedProperty>> GetTagExtendedPropertiesAsync(string tag, CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tag);
return _protocol.GetTagExtendedPropertiesAsync(tag, cancellationToken);
}
/// <summary>
/// Adds (or updates) extended (user-defined) properties on an existing tag via the 2020 WCF
/// <c>AddTEx</c> (AddTagExtendedProperties) op. Requires a write-enabled connection. String-valued
/// properties only (the evidence-backed surface). The new properties are read back via
/// <see cref="GetTagExtendedPropertiesAsync"/>. See <c>HistorianTagExtendedPropertyProtocol</c>.
/// </summary>
public Task<bool> AddTagExtendedPropertiesAsync(string tag, IReadOnlyList<HistorianTagExtendedProperty> properties, CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tag);
ArgumentNullException.ThrowIfNull(properties);
return _options.Transport == HistorianTransport.RemoteGrpc
? new Grpc.HistorianGrpcTagWriteOrchestrator(_options).AddTagExtendedPropertiesAsync(tag, properties, cancellationToken)
: new HistorianWcfTagWriteOrchestrator(_options).AddTagExtendedPropertiesAsync(tag, properties, cancellationToken);
}
/// <summary>Convenience overload of <see cref="AddTagExtendedPropertiesAsync"/> for a single
/// string-valued property.</summary>
public Task<bool> AddTagExtendedPropertyAsync(string tag, string name, string value, CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tag);
ArgumentException.ThrowIfNullOrWhiteSpace(name);
return AddTagExtendedPropertiesAsync(tag, [new HistorianTagExtendedProperty(name, value ?? string.Empty)], cancellationToken);
}
// Extended-property DELETE (DelTep) is intentionally NOT exposed publicly. Its wire format is
// captured and the serializer (HistorianTagExtendedPropertyProtocol.SerializeDeleteRequest) is
// golden-verified against a server-accepted buffer, but the SDK cannot yet make the 2020 server
// accept the delete: the server's CHistStorage::DeleteTagExtendedProperties consults a
// per-connection working set that the native client populates by multiplexing GetTepByNm and
// DelTep over ONE connection, which the SDK's per-service WCF channels don't reproduce. The gRPC
// transport — where every service client shares ONE channel — was probed 2026-06-22 to test that
// multiplexing hypothesis (GetTgByNm + GetTepByNm prime then DelTep on one write-enabled session,
// HistorianGrpcTagWriteOrchestrator.ProbeDeleteTagExtendedPropertiesAsync): both primes succeed on
// the shared channel yet the server STILL rejects the delete (native code=1), so gRPC does not lift
// the wall either. The working set is evidently populated by the native client's in-process
// registration state, not the wire session. See the documented-but-blocked path in
// HistorianWcfTagWriteOrchestrator and docs/reverse-engineering/wcf-add-tag-extended-properties.md §Delete.
/// <summary>
/// Executes a SQL command against the Historian over the WCF <c>ExeC</c>/<c>GetR</c> ops and
/// returns the record set as a <see cref="HistorianSqlResult"/> (the managed equivalent of the
/// native <c>DataTable</c>). The record-set path (<see cref="HistorianSqlExecuteOption.ExecuteRecord"/>,
/// the default) is the evidence-backed surface; the result is decoded from the server's
/// NRBF-serialized DataTable without BinaryFormatter. See <c>HistorianSqlResultProtocol</c>.
/// </summary>
public Task<HistorianSqlResult> ExecuteSqlCommandAsync(
string command,
HistorianSqlExecuteOption option = HistorianSqlExecuteOption.ExecuteRecord,
CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(command);
return _protocol.ExecuteSqlCommandAsync(command, option, cancellationToken);
}
/// <summary> /// <summary>
/// Creates or updates the named tag in the Historian Runtime database via /// Creates or updates the named tag in the Historian Runtime database via
/// <c>EnsureTags2</c>. Currently only <see cref="HistorianDataType.Float"/> is /// <c>EnsureTags2</c>. Currently only <see cref="HistorianDataType.Float"/> is
@@ -132,11 +298,9 @@ public sealed class HistorianClient : IAsyncDisposable
public Task<bool> EnsureTagAsync(HistorianTagDefinition definition, CancellationToken cancellationToken = default) public Task<bool> EnsureTagAsync(HistorianTagDefinition definition, CancellationToken cancellationToken = default)
{ {
ArgumentNullException.ThrowIfNull(definition); ArgumentNullException.ThrowIfNull(definition);
if (!OperatingSystem.IsWindows()) return _options.Transport == HistorianTransport.RemoteGrpc
{ ? new Grpc.HistorianGrpcTagWriteOrchestrator(_options).EnsureTagAsync(definition, cancellationToken)
throw new ProtocolEvidenceMissingException("EnsureTagAsync requires Windows for the SSPI auth path"); : new HistorianWcfTagWriteOrchestrator(_options).EnsureTagAsync(definition, cancellationToken);
}
return new HistorianWcfTagWriteOrchestrator(_options).EnsureTagAsync(definition, cancellationToken);
} }
/// <summary> /// <summary>
@@ -150,11 +314,104 @@ public sealed class HistorianClient : IAsyncDisposable
public Task<bool> DeleteTagAsync(string tagName, CancellationToken cancellationToken = default) public Task<bool> DeleteTagAsync(string tagName, CancellationToken cancellationToken = default)
{ {
ArgumentException.ThrowIfNullOrWhiteSpace(tagName); ArgumentException.ThrowIfNullOrWhiteSpace(tagName);
if (!OperatingSystem.IsWindows()) return _options.Transport == HistorianTransport.RemoteGrpc
? new Grpc.HistorianGrpcTagWriteOrchestrator(_options).DeleteTagAsync(tagName, cancellationToken)
: new HistorianWcfTagWriteOrchestrator(_options).DeleteTagAsync(tagName, cancellationToken);
}
/// <summary>
/// Renames one tag, submitting an asynchronous rename job via the History <c>StartJob</c> (StJb)
/// operation. Convenience wrapper over <see cref="RenameTagsAsync"/> for a single (old,new) pair.
/// Requires the server's <c>AllowRenameTags</c> system parameter to be enabled.
/// </summary>
public Task<HistorianTagRenameResult> RenameTagAsync(string oldName, string newName, CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(oldName);
ArgumentException.ThrowIfNullOrWhiteSpace(newName);
return RenameTagsAsync([(oldName, newName)], cancellationToken);
}
/// <summary>
/// Renames a batch of tags. Each pair is (current name, new name). Rename is an asynchronous
/// server-side job: the batch is submitted via the History <c>StartJob</c> (StJb) operation and
/// the returned <see cref="HistorianTagRenameResult"/> reports whether the server accepted/queued
/// the job (and its job id); the renames apply in the background. The server's
/// <c>AllowRenameTags</c> system parameter must be enabled or the server rejects the job. See
/// <c>docs/reverse-engineering/wcf-rename-tags.md</c>.
/// </summary>
public Task<HistorianTagRenameResult> RenameTagsAsync(IReadOnlyList<(string OldName, string NewName)> pairs, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(pairs);
return _options.Transport == HistorianTransport.RemoteGrpc
? new Grpc.HistorianGrpcTagWriteOrchestrator(_options).RenameTagsAsync(pairs, cancellationToken)
: new HistorianWcfTagWriteOrchestrator(_options).RenameTagsAsync(pairs, cancellationToken);
}
/// <summary>
/// Opens a reusable authenticated <see cref="HistorianSession"/> over the 2023 R2 gRPC transport.
/// The caller owns the session and must dispose it. Reusing the session across ops amortizes the auth
/// handshake; the server idle-expires it in ~20-25s, so keep it warm (HistorianSession.PingAsync) or
/// re-open. RemoteGrpc only.
/// </summary>
public async Task<HistorianSession> OpenSessionAsync(HistorianSessionKind kind, CancellationToken cancellationToken = default)
{
if (_options.Transport != HistorianTransport.RemoteGrpc)
{ {
throw new ProtocolEvidenceMissingException("DeleteTagAsync requires Windows for the SSPI auth path"); throw new ProtocolEvidenceMissingException(
"HistorianSession is only supported over the 2023 R2 RemoteGrpc transport.");
} }
return new HistorianWcfTagWriteOrchestrator(_options).DeleteTagAsync(tagName, cancellationToken);
return await Task.Run(() =>
{
uint mode = kind == HistorianSessionKind.WriteEnabled
? HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode
: HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode;
Grpc.HistorianGrpcConnection connection = Grpc.HistorianGrpcChannelFactory.Create(_options);
try
{
Grpc.HistorianGrpcHandshake.Session session =
Grpc.HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken, connectionMode: mode);
return new HistorianSession(connection, session, _options, kind);
}
catch
{
connection.Dispose(); // don't leak the channel if the handshake fails
throw;
}
}, cancellationToken).ConfigureAwait(false);
}
/// <summary>
/// Opens a reusable v8 EVENT session (ECDH + RegisterCmEventTag ONCE) over the 2023 R2 gRPC
/// transport. The caller owns the session and must dispose it. Reusing the session across sends
/// amortizes the ECDH+register cost (~10-16×, spike-proven); the server idle-expires it in ~25s,
/// so keep it warm (HistorianEventSession.PingAsync) or re-open. For SendEvent amortization only —
/// event reads are gated (C2) and not exposed here. RemoteGrpc only.
/// </summary>
public async Task<HistorianEventSession> OpenEventSessionAsync(CancellationToken cancellationToken = default)
{
if (_options.Transport != HistorianTransport.RemoteGrpc)
{
throw new ProtocolEvidenceMissingException(
"HistorianEventSession is only supported over the 2023 R2 RemoteGrpc transport.");
}
return await Task.Run(() =>
{
Grpc.HistorianGrpcConnection connection = Grpc.HistorianGrpcChannelFactory.Create(_options);
try
{
var orch = new Grpc.HistorianGrpcEventWriteOrchestrator(_options);
Grpc.HistorianGrpcHandshake.Session session = orch.OpenAndRegisterEventSession(connection, cancellationToken);
return new HistorianEventSession(connection, session, _options);
}
catch
{
connection.Dispose(); // don't leak the channel if the handshake fails
throw;
}
}, cancellationToken).ConfigureAwait(false);
} }
public ValueTask DisposeAsync() public ValueTask DisposeAsync()
@@ -6,6 +6,9 @@ public sealed class HistorianClientOptions
{ {
public const int DefaultPort = 32568; public const int DefaultPort = 32568;
/// <summary>Default TCP port of the 2023 R2 Historian Client Access Point gRPC endpoint.</summary>
public const int DefaultGrpcPort = 32565;
public required string Host { get; init; } public required string Host { get; init; }
public int Port { get; init; } = DefaultPort; public int Port { get; init; } = DefaultPort;
@@ -27,4 +30,67 @@ public sealed class HistorianClientOptions
public HistorianTransport Transport { get; init; } = HistorianTransport.LocalPipe; public HistorianTransport Transport { get; init; } = HistorianTransport.LocalPipe;
public string TargetSpn { get; init; } = @"NT SERVICE\aahClientAccessPoint"; public string TargetSpn { get; init; } = @"NT SERVICE\aahClientAccessPoint";
/// <summary>
/// When true, the WCF channel factories used by the SDK accept the server's
/// X.509 certificate without chain validation. Useful when connecting to a
/// development / on-prem Historian whose <c>/HistCert</c> endpoint presents an
/// installer-generated self-signed cert that isn't in the local trust store
/// (notably .NET WCF on Linux ignores the system CA bundle for its own
/// X509Chain checks). Default false; do not enable in production where the
/// server's identity matters.
/// </summary>
public bool AllowUntrustedServerCertificate { get; init; }
/// <summary>
/// Overrides the expected DNS identity in the endpoint address — set this to
/// whatever DNS name the server's certificate actually claims (often
/// <c>localhost</c> on installer-generated AVEVA Historian certificates) when
/// connecting via IP address or a hostname that doesn't match the cert SAN/CN.
/// Without this override WCF rejects the channel with
/// "Identity check failed for outgoing message". Has no effect on transports
/// that don't validate a server certificate.
/// </summary>
public string? ServerDnsIdentity { get; init; }
/// <summary>
/// Optional WCF "Via" address (e.g. <c>net.tcp://host:42568</c>). When set, the SDK's WCF
/// channel factories <b>connect</b> to this address while still addressing the SOAP message
/// <c>To</c> the logical endpoint built from <see cref="Host"/>/<see cref="Port"/>. Use this when
/// the Historian is reached through a port-forwarding tunnel or proxy whose local port differs
/// from the server's real service port: point <see cref="Host"/>/<see cref="Port"/> at the
/// server's real endpoint (so the server's WCF AddressFilter matches) and set this to the tunnel
/// endpoint. Has no effect on the gRPC transport. Default null (connect == address).
/// </summary>
public string? ConnectViaAddress { get; init; }
/// <summary>
/// Diagnostic override for the native OpenConnection mode the WCF event-read chain uses (default
/// <c>0x402</c>, read-only process). Set to e.g. <c>0x501</c> (event) or <c>0x401</c> (write-enabled)
/// to probe whether CM_EVENT registration / event-row retrieval needs a different connection type on a
/// 2023 R2 server. Null = the default read-only process mode. Intended for protocol investigation.
/// </summary>
public uint? EventReadConnectionModeOverride { get; init; }
/// <summary>
/// For <see cref="HistorianTransport.RemoteGrpc"/>: when true the channel uses TLS
/// (<c>https://</c>); when false it uses plaintext (<c>http://</c>). Matches the stock
/// 2023 R2 client's <c>securedConnection</c> flag. The TLS host is taken from
/// <see cref="ServerDnsIdentity"/> when set (to match the server certificate's name),
/// otherwise <see cref="Host"/>. When <see cref="AllowUntrustedServerCertificate"/> is
/// true the server certificate chain is not validated. Default false.
/// </summary>
public bool GrpcUseTls { get; init; }
/// <summary>
/// When true (default) the SDK verifies, at connect time, that the Historian server
/// reports the native interface versions its byte serializers were built against
/// (History=11, Retrieval=4, Transaction=2 — evidence from a live AVEVA Historian 2020
/// server). A mismatch throws <see cref="ProtocolEvidenceMissingException"/> rather than
/// risk misparsing version-framed native buffers. Set false only when you have
/// independently confirmed wire compatibility with a different server version — e.g.
/// when bringing up a 2023 R2 gRPC server whose reported interface integers have not yet
/// been captured. See <see cref="HistorianServerVersionGate"/>.
/// </summary>
public bool VerifyServerInterfaceVersion { get; init; } = true;
} }
@@ -0,0 +1,66 @@
using AVEVA.Historian.Client.Grpc;
using AVEVA.Historian.Client.Models;
namespace AVEVA.Historian.Client;
/// <summary>A live, reusable authenticated v8 EVENT session: holds one event gRPC connection + one
/// open+registered Event handle and runs SendEvent on it WITHOUT re-handshaking. Reuse amortizes the
/// ECDH+register cost (~10-16×, spike-proven). SendEvent only — event READS are gated (C2) and stay
/// per-call. Keep in sync with <see cref="HistorianSession"/> (the v6 sibling).</summary>
public sealed class HistorianEventSession : IAsyncDisposable
{
private readonly HistorianGrpcConnection _connection;
private readonly HistorianGrpcHandshake.Session _session;
private readonly HistorianClientOptions _options;
private int _disposed;
internal HistorianEventSession(
HistorianGrpcConnection connection, HistorianGrpcHandshake.Session session, HistorianClientOptions options)
{
_connection = connection;
_session = session;
_options = options;
}
/// <summary>Exposes the held event gRPC connection for internal callers (e.g. the round-trip test
/// verifying the keepalive op directly). Not part of the public surface.</summary>
internal HistorianGrpcConnection Connection => _connection;
/// <summary>Exposes the held open+registered Event session handle for internal callers (e.g. the
/// round-trip test verifying the keepalive op directly). Not part of the public surface.</summary>
internal HistorianGrpcHandshake.Session Session => _session;
/// <summary>Sends one event on the held (open+registered) v8 Event session.</summary>
public Task<bool> SendEventAsync(HistorianEvent evt, CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(evt);
if (evt.RevisionVersion != 0)
{
throw new ProtocolEvidenceMissingException(
"Only original events (RevisionVersion = 0) have a captured send encoding; " +
"revision/update/delete event sends are not yet supported.");
}
var orch = new HistorianGrpcEventWriteOrchestrator(_options);
return Task.Run(() => orch.SendEventOnSession(_connection, _session, evt, ct), ct);
}
/// <summary>Keepalive via a lightweight <c>GetSystemParameter</c> status read on the event session's
/// <see cref="HistorianGrpcHandshake.Session.ClientHandle"/> (the same status op the native pre-query
/// sequence issues against an authenticated Event session), under the server idle floor. Mirrors
/// <see cref="HistorianSession.PingAsync"/>. The op's effectiveness on a v8 Event handle is
/// live-verified by the round-trip test.</summary>
public Task PingAsync(CancellationToken ct = default)
=> Task.Run(() => HistorianGrpcStatusClient.GetSystemParameterOnSession(
_connection, _session.ClientHandle, _options, "HistorianVersion", ct), ct);
/// <summary>Disposes the underlying event connection (idempotent).</summary>
public ValueTask DisposeAsync()
{
if (Interlocked.Exchange(ref _disposed, 1) == 0)
{
_connection.Dispose();
}
return ValueTask.CompletedTask;
}
}
@@ -0,0 +1,122 @@
namespace AVEVA.Historian.Client;
/// <summary>
/// Identifies a versioned native Historian service interface whose reported interface
/// version is validated at connect time by <see cref="HistorianServerVersionGate"/>.
/// </summary>
internal enum HistorianServiceInterface
{
History,
Retrieval,
Status,
Transaction
}
/// <summary>
/// Fail-closed check (roadmap item R0.6) that a Historian server reports the native
/// interface version this SDK's byte serializers were built against.
///
/// The opaque native buffers carried inside the WCF/MDAS message body — and, on 2023 R2,
/// inside the gRPC <c>bytes</c> fields — are framed per native interface version. Parsing
/// them against an unexpected version risks silent misinterpretation, so per the
/// "version-pin, fail closed" principle this throws <see cref="ProtocolEvidenceMissingException"/>
/// rather than best-effort parsing.
///
/// Supported versions are evidence-based, discovered from a live AVEVA Historian 2020
/// server (product 20.0.000) via the reverse-engineering <c>wcf-probe</c> command:
/// <list type="bullet">
/// <item>History (<c>Hist</c>) interface version = 11</item>
/// <item>Retrieval (<c>Retr</c>) interface version = 4</item>
/// <item>Transaction (<c>Trx</c>) interface version = 2</item>
/// </list>
/// The Status (<c>Stat</c>) service's <c>GetInterfaceVersion</c> is not a real version (0 on
/// 2020 WCF, 4 on 2023 R2 gRPC) — it carries no meaning for the byte serializers either way — so
/// the Status interface is validated for reachability only, never value.
///
/// A 2023 R2 gRPC server reports History interface version 12 even though it carries the
/// same proven 2020 native buffers. That value is captured and accepted (see
/// <see cref="AcceptedVersions"/>), so a v12 server passes with the default
/// <see cref="HistorianClientOptions.VerifyServerInterfaceVersion"/>=<see langword="true"/>;
/// the opt-out is only a safety valve for some future, not-yet-captured interface integer.
/// </summary>
internal static class HistorianServerVersionGate
{
public const uint HistoryInterfaceVersion = 11;
public const uint RetrievalInterfaceVersion = 4;
public const uint TransactionInterfaceVersion = 2;
/// <summary>
/// The 2023 R2 gRPC HistoryService reports interface version 12. It is buffer-compatible with
/// the 2020 version 11 — the OpenConnection3 v6 / token / DataQueryRequest / row buffers are
/// byte-identical — confirmed by a live end-to-end gRPC read against a real 2023 R2 server
/// (2026-06-21). So both 11 and 12 are accepted for History.
///
/// Retrieval=4, Transaction=2, and Status UiError=0 are now confirmed captured live over the
/// 2023 R2 gRPC transport (2026-06-25, unauthenticated GetInterfaceVersion RPCs); see
/// <c>docs/reverse-engineering/grpc-interface-versions.md</c>. All captured values were already
/// accepted — no widening of <see cref="AcceptedVersions"/> was required.
/// </summary>
public const uint HistoryInterfaceVersionGrpc2023R2 = 12;
/// <summary>
/// True when the service interface reports a meaningful version that should be matched.
/// Status is reachability-only (its <c>GetInterfaceVersion</c> is not a real version —
/// 0 on 2020 WCF, 4 on 2023 R2 gRPC).
/// </summary>
public static bool IsValueGated(HistorianServiceInterface service) => service switch
{
HistorianServiceInterface.History => true,
HistorianServiceInterface.Retrieval => true,
HistorianServiceInterface.Transaction => true,
HistorianServiceInterface.Status => false,
_ => false
};
/// <summary>The canonical interface version this SDK's serializers target for a value-gated service.</summary>
public static uint ExpectedVersion(HistorianServiceInterface service) => service switch
{
HistorianServiceInterface.History => HistoryInterfaceVersion,
HistorianServiceInterface.Retrieval => RetrievalInterfaceVersion,
HistorianServiceInterface.Transaction => TransactionInterfaceVersion,
_ => throw new ArgumentOutOfRangeException(nameof(service), service, "Service interface is not value-gated.")
};
/// <summary>
/// All interface versions accepted for a value-gated service. Usually a single value, but
/// History accepts both the 2020 value (11) and the buffer-compatible 2023 R2 gRPC value (12).
/// </summary>
public static uint[] AcceptedVersions(HistorianServiceInterface service) => service switch
{
HistorianServiceInterface.History => [HistoryInterfaceVersion, HistoryInterfaceVersionGrpc2023R2],
HistorianServiceInterface.Retrieval => [RetrievalInterfaceVersion],
HistorianServiceInterface.Transaction => [TransactionInterfaceVersion],
_ => throw new ArgumentOutOfRangeException(nameof(service), service, "Service interface is not value-gated.")
};
/// <summary>
/// Throws <see cref="ProtocolEvidenceMissingException"/> when version verification is enabled
/// and the server's reported interface version differs from the version this SDK targets.
/// No-op when <see cref="HistorianClientOptions.VerifyServerInterfaceVersion"/> is
/// <see langword="false"/>, when the service is not value-gated (Status), or on a match.
/// </summary>
public static void Validate(HistorianServiceInterface service, uint reportedVersion, HistorianClientOptions options)
{
ArgumentNullException.ThrowIfNull(options);
if (!options.VerifyServerInterfaceVersion || !IsValueGated(service))
{
return;
}
uint[] accepted = AcceptedVersions(service);
if (Array.IndexOf(accepted, reportedVersion) >= 0)
{
return;
}
string acceptedList = string.Join(", ", accepted);
throw new ProtocolEvidenceMissingException(
$"{service} interface version {reportedVersion} (this SDK's serializers target version {acceptedList}); " +
$"set {nameof(HistorianClientOptions)}.{nameof(HistorianClientOptions.VerifyServerInterfaceVersion)}=false to bypass at your own risk");
}
}
@@ -0,0 +1,214 @@
using System.Runtime.CompilerServices;
using AVEVA.Historian.Client.Grpc;
using AVEVA.Historian.Client.Models;
namespace AVEVA.Historian.Client;
/// <summary>A live, reusable authenticated Historian session: holds one gRPC connection + one
/// OpenConnection handle and runs ops on them WITHOUT re-handshaking. Reuse across ops amortizes the
/// auth handshake. Idle-expires server-side in ~20-25s — callers keep it warm (PingAsync) or re-open.
/// Reads, browse/metadata, historical-write, tag-write and status; events are NOT exposed (separate channel+auth).</summary>
public sealed class HistorianSession : IAsyncDisposable
{
private readonly HistorianGrpcConnection _connection;
private readonly HistorianGrpcHandshake.Session _session;
private readonly HistorianClientOptions _options;
private int _disposed;
/// <summary>Whether this session was opened read-only or write-enabled (the Open2 connection mode).</summary>
public HistorianSessionKind Kind { get; }
internal HistorianSession(
HistorianGrpcConnection connection,
HistorianGrpcHandshake.Session session,
HistorianClientOptions options,
HistorianSessionKind kind)
{
_connection = connection;
_session = session;
_options = options;
Kind = kind;
}
// --- reads (mirror the orchestrators' async-enumerable wrapping; call the …OnSession seams,
// which take the transient uint client handle) ---
/// <summary>Raw values for <paramref name="tag"/> in [<paramref name="startUtc"/>,
/// <paramref name="endUtc"/>], capped at <paramref name="maxValues"/>, on the held session.</summary>
public async IAsyncEnumerable<HistorianSample> ReadRawAsync(
string tag,
DateTime startUtc,
DateTime endUtc,
int maxValues,
[EnumeratorCancellation] CancellationToken ct = default)
{
ct.ThrowIfCancellationRequested();
var orch = new HistorianGrpcReadOrchestrator(_options);
IReadOnlyList<HistorianSample> rows = await Task.Run(
() => orch.RunRawQueryOnSession(_connection, _session.ClientHandle, tag, startUtc, endUtc, maxValues, ct), ct)
.ConfigureAwait(false);
foreach (HistorianSample sample in rows)
{
ct.ThrowIfCancellationRequested();
yield return sample;
}
}
/// <summary>Aggregate values for <paramref name="tag"/> over [<paramref name="startUtc"/>,
/// <paramref name="endUtc"/>] in <paramref name="interval"/> buckets, using the supplied
/// retrieval <paramref name="mode"/>, on the held session.</summary>
public async IAsyncEnumerable<HistorianAggregateSample> ReadAggregateAsync(
string tag,
DateTime startUtc,
DateTime endUtc,
RetrievalMode mode,
TimeSpan interval,
[EnumeratorCancellation] CancellationToken ct = default)
{
ct.ThrowIfCancellationRequested();
var orch = new HistorianGrpcReadOrchestrator(_options);
IReadOnlyList<HistorianAggregateSample> rows = await Task.Run(
() => orch.RunAggregateQueryOnSession(_connection, _session.ClientHandle, tag, startUtc, endUtc, mode, interval, ct), ct)
.ConfigureAwait(false);
foreach (HistorianAggregateSample sample in rows)
{
ct.ThrowIfCancellationRequested();
yield return sample;
}
}
/// <summary>Interpolated values for <paramref name="tag"/> at each of the supplied
/// <paramref name="timestampsUtc"/>, on the held session.</summary>
public Task<IReadOnlyList<HistorianSample>> ReadAtTimeAsync(
string tag,
IReadOnlyList<DateTime> timestampsUtc,
CancellationToken ct = default)
{
var orch = new HistorianGrpcReadOrchestrator(_options);
return Task.Run<IReadOnlyList<HistorianSample>>(
() => orch.RunAtTimeOnSession(_connection, _session.ClientHandle, tag, timestampsUtc, ct), ct);
}
// --- browse / metadata (call the …OnSession seams, which take the full Session for the string handle) ---
/// <summary>Browses tag names matching <paramref name="filter"/> on the held session.</summary>
public async IAsyncEnumerable<string> BrowseTagNamesAsync(
string filter = "*",
[EnumeratorCancellation] CancellationToken ct = default)
{
ct.ThrowIfCancellationRequested();
List<string> names = await Task.Run(
() => HistorianGrpcTagClient.BrowseTagNamesOnSession(_connection, _session, filter, _options, ct), ct)
.ConfigureAwait(false);
foreach (string name in names)
{
ct.ThrowIfCancellationRequested();
yield return name;
}
}
/// <summary>Reads metadata for <paramref name="tag"/> on the held session (null if unknown).</summary>
public Task<HistorianTagMetadata?> GetTagMetadataAsync(string tag, CancellationToken ct = default)
=> Task.Run(() => HistorianGrpcTagClient.GetTagMetadataOnSession(_connection, _session, tag, _options, ct), ct);
// --- writes (the …OnSession seams take the full Session, since the historical write keys on the
// string handle + tag GUID and the tag-config ops mix string/uint handles) ---
/// <summary>Writes historical/backfill values for <paramref name="tag"/> on the held (write-enabled) session.</summary>
public Task<bool> AddHistoricalValuesAsync(
string tag,
IReadOnlyList<HistorianHistoricalValue> values,
CancellationToken ct = default)
{
var orch = new HistorianGrpcHistoricalWriteOrchestrator(_options);
return Task.Run(() => orch.RunWriteOnSession(_connection, _session, tag, values, ct), ct);
}
/// <summary>Ensures the tag described by <paramref name="definition"/> exists on the held (write-enabled) session.</summary>
public Task<bool> EnsureTagAsync(HistorianTagDefinition definition, CancellationToken ct = default)
{
var orch = new HistorianGrpcTagWriteOrchestrator(_options);
return Task.Run(() => orch.EnsureTagOnSession(_connection, _session, definition, ct), ct);
}
/// <summary>Deletes <paramref name="tagName"/> on the held (write-enabled) session.</summary>
public Task<bool> DeleteTagAsync(string tagName, CancellationToken ct = default)
{
var orch = new HistorianGrpcTagWriteOrchestrator(_options);
return Task.Run(() => orch.DeleteTagOnSession(_connection, _session, tagName, ct), ct);
}
/// <summary>Renames the supplied (old,new) tag-name <paramref name="pairs"/> on the held (write-enabled) session.</summary>
public Task<HistorianTagRenameResult> RenameTagsAsync(
IReadOnlyList<(string OldName, string NewName)> pairs,
CancellationToken ct = default)
{
var orch = new HistorianGrpcTagWriteOrchestrator(_options);
return Task.Run(() => orch.RenameTagsOnSession(_connection, _session, pairs, ct), ct);
}
/// <summary>Adds extended <paramref name="properties"/> to <paramref name="tagName"/> on the held (write-enabled) session.</summary>
public Task<bool> AddTagExtendedPropertiesAsync(
string tagName,
IReadOnlyList<HistorianTagExtendedProperty> properties,
CancellationToken ct = default)
{
var orch = new HistorianGrpcTagWriteOrchestrator(_options);
return Task.Run(() => orch.AddTagExtendedPropertiesOnSession(_connection, _session, tagName, properties, ct), ct);
}
// --- status + keepalive (the status seams are static; handle shape differs per op) ---
/// <summary>Reads the named system parameter (e.g. "HistorianVersion") on the held session.</summary>
public Task<string?> GetSystemParameterAsync(string parameterName, CancellationToken ct = default)
=> Task.Run(() => HistorianGrpcStatusClient.GetSystemParameterOnSession(
_connection, _session.ClientHandle, _options, parameterName, ct), ct);
/// <summary>Reports connection status derived from the held session's storage handle (no follow-on RPC).</summary>
public Task<HistorianConnectionStatus> GetConnectionStatusAsync(CancellationToken ct = default)
=> Task.Run(() =>
{
(bool connected, string? error) = HistorianGrpcStatusClient.EvaluateConnectionStatusOnSession(_connection, _session);
return new HistorianConnectionStatus(
ServerName: _options.Host,
Pending: false,
ErrorOccurred: !connected,
Error: error,
ConnectedToServer: connected,
ConnectedToServerStorage: connected,
ConnectedToStoreForward: false,
ConnectionKind: HistorianConnectionKind.Process);
}, ct);
/// <summary>Reports a measured store-forward status (GetHistorianConsoleStatus) on the held session.</summary>
public Task<HistorianStoreForwardStatus> GetStoreForwardStatusAsync(CancellationToken ct = default)
=> Task.Run(() =>
{
HistorianStoreForwardStatus NotStoring(bool errorOccurred, string? error) => new(
ServerName: _options.Host,
Pending: false,
ErrorOccurred: errorOccurred,
Error: error,
DataStored: false,
Storing: false,
ConnectionKind: HistorianConnectionKind.Process);
return HistorianGrpcStatusClient.GetStoreForwardStatusOnSession(
_connection, _session.StringHandle, _options, NotStoring, ct);
}, ct);
/// <summary>Keeps the session warm against the server's ~20-25s idle expiry by issuing a cheap
/// system-parameter read. Call periodically (or before a latency-sensitive op) to avoid re-handshaking.</summary>
public Task PingAsync(CancellationToken ct = default) => GetSystemParameterAsync("HistorianVersion", ct);
/// <summary>Disposes the underlying gRPC connection (closes the channel). The server-side session
/// also idle-expires on its own; this releases the local channel resources immediately.</summary>
public ValueTask DisposeAsync()
{
if (Interlocked.Exchange(ref _disposed, 1) == 0)
{
_connection.Dispose();
}
return ValueTask.CompletedTask;
}
}
@@ -0,0 +1,5 @@
namespace AVEVA.Historian.Client;
/// <summary>Connection mode for an authenticated session. WriteEnabled (0x401) is a superset that
/// also serves reads (live-verified 2026-06-25); ReadOnly (0x402) is read-only.</summary>
public enum HistorianSessionKind { ReadOnly, WriteEnabled }
@@ -4,5 +4,12 @@ public enum HistorianTransport
{ {
LocalPipe = 0, LocalPipe = 0,
RemoteTcpIntegrated = 1, RemoteTcpIntegrated = 1,
RemoteTcpCertificate = 2 RemoteTcpCertificate = 2,
/// <summary>
/// 2023 R2 gRPC transport (Historian Client Access Point gRPC-Web endpoint, default
/// TCP port 32565). Carries the same native binary payloads as the WCF transports inside
/// protobuf <c>bytes</c> fields. See <c>Grpc/HistorianGrpcReadOrchestrator</c>.
/// </summary>
RemoteGrpc = 3
} }
@@ -0,0 +1,40 @@
namespace AVEVA.Historian.Client.Models;
/// <summary>
/// Comparison operator for a <see cref="HistorianEventFilter"/>. Values mirror the native
/// <c>ArchestrA.HistorianComparisionType</c> ordinals and travel on the wire as a UInt16.
/// </summary>
public enum HistorianEventComparison : ushort
{
Equal = 0,
NotEqual = 1,
LessThan = 2,
NotLessThan = 3,
GreaterThan = 4,
NotGreaterThan = 5,
LessThanEqual = 6,
NotLessThanEqual = 7,
GreaterThanEqual = 8,
NotGreaterThanEqual = 9,
Begins = 10,
NotBegins = 11,
Contains = 12,
NotContains = 13,
Exists = 14,
NotExists = 15,
EndWith = 16,
NotEndWith = 17,
}
/// <summary>
/// A single server-side event filter predicate: <c>PropertyName Comparison Value</c>
/// (e.g. <c>Type Equal "User.Write"</c>, <c>Area Contains "Tank"</c>). Applied to
/// <c>ReadEventsAsync</c>; the server returns only events whose named property satisfies the
/// comparison. For <see cref="HistorianEventComparison.Exists"/> /
/// <see cref="HistorianEventComparison.NotExists"/> the value is ignored but still required by
/// the wire format (pass any non-null string).
/// </summary>
public sealed record HistorianEventFilter(
string PropertyName,
HistorianEventComparison Comparison,
string Value);
@@ -0,0 +1,11 @@
namespace AVEVA.Historian.Client.Models;
/// <summary>
/// A single historical (backfill) value to insert via
/// <see cref="HistorianClient.AddHistoricalValuesAsync"/>. The historian stores the value against
/// the tag at <paramref name="TimestampUtc"/> as original (non-streamed) data.
/// </summary>
/// <param name="TimestampUtc">The value timestamp (UTC). Treated as UTC if unspecified-kind.</param>
/// <param name="Value">The numeric value. Captured/supported for Float tags today.</param>
/// <param name="OpcQuality">OPC quality; defaults to 192 (good).</param>
public sealed record HistorianHistoricalValue(DateTime TimestampUtc, double Value, ushort OpcQuality = 192);
@@ -0,0 +1,21 @@
namespace AVEVA.Historian.Client.Models;
/// <summary>
/// Execution mode for <c>ExecuteSqlCommandAsync</c>, mirroring the native
/// <c>HistorianSqlExecuteOption</c> enum passed to the <c>ExeC</c> op. Only
/// <see cref="ExecuteRecord"/> (the record-set path) is evidence-backed end-to-end.
/// </summary>
public enum HistorianSqlExecuteOption
{
/// <summary>Execute and return a record set (the captured/proven path).</summary>
ExecuteRecord = 0,
/// <summary>Execute without returning a record set; the result carries only the return value.</summary>
ExecuteNonQuery = 1,
/// <summary>Execute and return a single scalar value.</summary>
ExecuteScalar = 2,
/// <summary>Execute a record set directly (server-side direct path).</summary>
ExecuteRecordDirect = 3,
}
@@ -0,0 +1,35 @@
namespace AVEVA.Historian.Client.Models;
/// <summary>
/// A column in a <see cref="HistorianSqlResult"/>: its name and the XSD type from the result-set
/// schema (e.g. <c>xs:int</c>, <c>xs:string</c>, <c>xs:dateTime</c>).
/// </summary>
public sealed record HistorianSqlColumn(string Name, string SchemaType);
/// <summary>
/// The result of <c>ExecuteSqlCommandAsync</c> — the managed equivalent of the <c>DataTable</c>
/// the native client returns. Columns carry name + XSD type; each row is a list of values aligned
/// to <see cref="Columns"/> (typed per the schema where the XSD type is recognized, otherwise the
/// raw string; <c>null</c> for a SQL NULL / absent cell).
/// </summary>
public sealed class HistorianSqlResult
{
public HistorianSqlResult(
IReadOnlyList<HistorianSqlColumn> columns,
IReadOnlyList<IReadOnlyList<object?>> rows,
int returnValue)
{
Columns = columns;
Rows = rows;
ReturnValue = returnValue;
}
/// <summary>The result-set columns, in order.</summary>
public IReadOnlyList<HistorianSqlColumn> Columns { get; }
/// <summary>The rows; each row's values align positionally to <see cref="Columns"/>.</summary>
public IReadOnlyList<IReadOnlyList<object?>> Rows { get; }
/// <summary>The native <c>retValue</c> from <c>ExeC</c> (e.g. rows affected for non-queries).</summary>
public int ReturnValue { get; }
}
@@ -0,0 +1,19 @@
namespace AVEVA.Historian.Client.Models;
/// <summary>
/// Storage strategy for historized samples. Maps to <c>Tag.StorageType</c> in the
/// Runtime DB. Values match the captured native enum and the server-persisted
/// integer column.
/// </summary>
public enum HistorianStorageType
{
/// <summary>
/// Sample on a fixed cadence (see <c>HistorianTagDefinition.StorageRateMs</c>).
/// </summary>
Cyclic = 1,
/// <summary>
/// Sample only on value change (with optional value/time/rate deadbands).
/// </summary>
Delta = 2,
}
@@ -6,6 +6,11 @@ namespace AVEVA.Historian.Client.Models;
/// String/Int1/Int8/UInt8 types failed at native AddTag — likely require a different /// String/Int1/Int8/UInt8 types failed at native AddTag — likely require a different
/// path and are intentionally not supported. MinEU/MaxEU/MinRaw/MaxRaw are now encoded /// path and are intentionally not supported. MinEU/MaxEU/MinRaw/MaxRaw are now encoded
/// into the wire payload (see <c>HistorianTagWriteProtocol</c>). /// into the wire payload (see <c>HistorianTagWriteProtocol</c>).
///
/// Semantics: <c>EnsureTagAsync</c> is an upsert. Calling it twice on the same
/// <see cref="TagName"/> with different fields succeeds both times; the second call
/// updates Description, MinEU, MaxEU, MinRaw, MaxRaw, and AnalogTag.Scaling on the
/// existing row (verified 2026-05-04 by direct SQL inspection after sequential calls).
/// </summary> /// </summary>
public sealed record HistorianTagDefinition public sealed record HistorianTagDefinition
{ {
@@ -28,18 +33,52 @@ public sealed record HistorianTagDefinition
public double MaxEU { get; init; } = 100.0; public double MaxEU { get; init; } = 100.0;
/// <summary> /// <summary>
/// Raw lower bound (pre-scaling). Default 0. Note: with ApplyScaling=false (the /// Raw lower bound (pre-scaling). Default 0. Persisted distinctly only when
/// only path the SDK currently exposes), the server appears to mirror MinRaw to /// <see cref="ApplyScaling"/> is true; with ApplyScaling=false the server mirrors
/// MinEU on EnsureTags2 verified 2026-05-04 against both native and managed /// this to MinEU on EnsureTags2 (verified 2026-05-04 against both native and
/// clients with the same input. The value is sent on the wire but not persisted /// managed clients).
/// independently. To set distinct raw bounds, ApplyScaling=true plus a follow-up
/// UpdateTags call would be required (not yet wired).
/// </summary> /// </summary>
public double MinRaw { get; init; } public double MinRaw { get; init; }
/// <summary> /// <summary>
/// Raw upper bound (pre-scaling). Default 100. See <see cref="MinRaw"/> for the /// Raw upper bound (pre-scaling). Default 100. See <see cref="MinRaw"/> for the
/// server-side mirror caveat with ApplyScaling=false. /// ApplyScaling caveat.
/// </summary> /// </summary>
public double MaxRaw { get; init; } = 100.0; public double MaxRaw { get; init; } = 100.0;
/// <summary>
/// When true, the server persists <see cref="MinRaw"/> / <see cref="MaxRaw"/> as
/// distinct values from <see cref="MinEU"/> / <see cref="MaxEU"/> and sets
/// <c>AnalogTag.Scaling</c> = 1. When false (default), the server mirrors MinRaw
/// to MinEU and MaxRaw to MaxEU and sets <c>AnalogTag.Scaling</c> = 0.
/// </summary>
public bool ApplyScaling { get; init; }
/// <summary>
/// Storage rate in milliseconds. Default 1000ms. The server only accepts
/// quantized values (observed valid set: 1000, 5000, 10000, 60000, 300000) —
/// non-quantized values cause <see cref="HistorianClient.EnsureTagAsync"/> to
/// return false.
/// </summary>
public uint StorageRateMs { get; init; } = 1000u;
/// <summary>
/// Storage strategy. Default <see cref="HistorianStorageType.Cyclic"/> samples
/// on the configured <see cref="StorageRateMs"/> cadence. <see cref="HistorianStorageType.Delta"/>
/// samples only on value change. The server persists this to <c>Tag.StorageType</c>
/// (Cyclic = 1, Delta = 2).
/// </summary>
public HistorianStorageType StorageType { get; init; } = HistorianStorageType.Cyclic;
/// <summary>
/// Divisor applied when storing integral values for trend integration. Default 1.0.
/// Wire bytes flip correctly per the captured native serializer, but live testing
/// 2026-05-05 showed the server stores <c>IntegralDivisor</c> on
/// <c>EngineeringUnit</c> (shared across all tags using that EU) rather than
/// per-tag — so a non-default value sent here is accepted on the wire but does
/// not visibly persist in <c>EngineeringUnit.IntegralDivisor</c> for the test
/// EU. Exposed for completeness and forward-compatibility; check your server's
/// behavior before relying on it.
/// </summary>
public double IntegralDivisor { get; init; } = 1.0;
} }
@@ -0,0 +1,11 @@
namespace AVEVA.Historian.Client.Models;
/// <summary>
/// A single extended (user-defined) property attached to a Historian tag — a name/value pair
/// returned by <c>GetTagExtendedPropertiesAsync</c>. Extended properties are stored separately
/// from the core tag metadata (server table <c>_TagExtendedProperty</c>) and are exposed over the
/// 2020 WCF <c>aa/Retr/GetTepByNm</c> op.
/// </summary>
/// <param name="Name">The property name (e.g., <c>Location</c>).</param>
/// <param name="Value">The property value as a string (the wire format is a VT_BSTR variant).</param>
public sealed record HistorianTagExtendedProperty(string Name, string Value);
@@ -0,0 +1,29 @@
namespace AVEVA.Historian.Client.Models;
/// <summary>
/// Result of <see cref="HistorianClient.RenameTagsAsync"/>. Tag rename on the Historian is an
/// asynchronous server-side <em>job</em>: the client submits the rename batch via the History
/// <c>StartJob</c> (StJb) operation and the server returns a job id, then applies the renames in the
/// background (the native client polls <c>GetJobStatus</c>/<c>GtJb</c> until the job reports done).
///
/// <para><see cref="Accepted"/> reflects whether the server accepted and queued the job. The renames
/// themselves complete asynchronously (observed: well under a couple of seconds for a small batch on
/// the local server). The server gate <c>AllowRenameTags</c> must be enabled, or the native client
/// library rejects the call before it reaches the wire (error 132 OperationNotEnabled).</para>
/// </summary>
public sealed record HistorianTagRenameResult
{
/// <summary>True when the server accepted and queued the rename job (StartJob returned success
/// with an empty error buffer).</summary>
public required bool Accepted { get; init; }
/// <summary>The server-assigned job id for the submitted rename batch (the <c>strJobid</c>
/// returned by StartJob). <see cref="Guid.Empty"/> if the server returned no parseable id.</summary>
public Guid JobId { get; init; }
/// <summary>Number of (old,new) name pairs submitted in the batch.</summary>
public int PairCount { get; init; }
/// <summary>Server error text when <see cref="Accepted"/> is false; otherwise null.</summary>
public string? Error { get; init; }
}
@@ -1,4 +1,4 @@
using System.Runtime.InteropServices; using AVEVA.Historian.Client.Grpc;
using AVEVA.Historian.Client.Models; using AVEVA.Historian.Client.Models;
using AVEVA.Historian.Client.Wcf; using AVEVA.Historian.Client.Wcf;
@@ -13,55 +13,28 @@ internal sealed class Historian2020ProtocolDialect
_options = options ?? throw new ArgumentNullException(nameof(options)); _options = options ?? throw new ArgumentNullException(nameof(options));
} }
private bool UseGrpc => _options.Transport == HistorianTransport.RemoteGrpc;
public IAsyncEnumerable<HistorianSample> ReadRawAsync(string tag, DateTime startUtc, DateTime endUtc, int maxValues, CancellationToken cancellationToken) public IAsyncEnumerable<HistorianSample> ReadRawAsync(string tag, DateTime startUtc, DateTime endUtc, int maxValues, CancellationToken cancellationToken)
{ {
if (!RuntimeInformation.IsOSPlatform(OSPlatform.Windows)) return UseGrpc
{ ? new HistorianGrpcReadOrchestrator(_options).ReadRawAsync(tag, startUtc, endUtc, maxValues, cancellationToken)
return Missing<HistorianSample>("StartDataRetrievalQuery/Full requires Windows for the SSPI path", cancellationToken); : new HistorianWcfReadOrchestrator(_options).ReadRawAsync(tag, startUtc, endUtc, maxValues, cancellationToken);
}
return ReadRawWindowsAsync(tag, startUtc, endUtc, maxValues, cancellationToken);
}
private IAsyncEnumerable<HistorianSample> ReadRawWindowsAsync(string tag, DateTime startUtc, DateTime endUtc, int maxValues, CancellationToken cancellationToken)
{
#pragma warning disable CA1416 // Validated by RuntimeInformation.IsOSPlatform check above.
HistorianWcfReadOrchestrator orchestrator = new(_options);
return orchestrator.ReadRawAsync(tag, startUtc, endUtc, maxValues, cancellationToken);
#pragma warning restore CA1416
} }
public IAsyncEnumerable<HistorianAggregateSample> ReadAggregateAsync(string tag, DateTime startUtc, DateTime endUtc, RetrievalMode mode, TimeSpan interval, CancellationToken cancellationToken) public IAsyncEnumerable<HistorianAggregateSample> ReadAggregateAsync(string tag, DateTime startUtc, DateTime endUtc, RetrievalMode mode, TimeSpan interval, CancellationToken cancellationToken)
{ {
if (!RuntimeInformation.IsOSPlatform(OSPlatform.Windows)) return UseGrpc
{ ? new HistorianGrpcReadOrchestrator(_options).ReadAggregateAsync(tag, startUtc, endUtc, mode, interval, cancellationToken)
return Missing<HistorianAggregateSample>($"StartDataRetrievalQuery/{mode} requires Windows for the SSPI path", cancellationToken); : new HistorianWcfReadOrchestrator(_options).ReadAggregateAsync(tag, startUtc, endUtc, mode, interval, cancellationToken);
}
return ReadAggregateWindowsAsync(tag, startUtc, endUtc, mode, interval, cancellationToken);
}
private IAsyncEnumerable<HistorianAggregateSample> ReadAggregateWindowsAsync(string tag, DateTime startUtc, DateTime endUtc, RetrievalMode mode, TimeSpan interval, CancellationToken cancellationToken)
{
#pragma warning disable CA1416
HistorianWcfReadOrchestrator orchestrator = new(_options);
return orchestrator.ReadAggregateAsync(tag, startUtc, endUtc, mode, interval, cancellationToken);
#pragma warning restore CA1416
} }
public Task<IReadOnlyList<HistorianSample>> ReadAtTimeAsync(string tag, IReadOnlyList<DateTime> timestampsUtc, CancellationToken cancellationToken) public Task<IReadOnlyList<HistorianSample>> ReadAtTimeAsync(string tag, IReadOnlyList<DateTime> timestampsUtc, CancellationToken cancellationToken)
{ {
cancellationToken.ThrowIfCancellationRequested(); cancellationToken.ThrowIfCancellationRequested();
return UseGrpc
if (!RuntimeInformation.IsOSPlatform(OSPlatform.Windows)) ? new HistorianGrpcReadOrchestrator(_options).ReadAtTimeAsync(tag, timestampsUtc, cancellationToken)
{ : new HistorianWcfReadOrchestrator(_options).ReadAtTimeAsync(tag, timestampsUtc, cancellationToken);
throw new ProtocolEvidenceMissingException("StartDataRetrievalQuery/Interpolated at-time requires Windows for the SSPI path");
}
#pragma warning disable CA1416
HistorianWcfReadOrchestrator orchestrator = new(_options);
return orchestrator.ReadAtTimeAsync(tag, timestampsUtc, cancellationToken);
#pragma warning restore CA1416
} }
public IAsyncEnumerable<HistorianBlock> ReadBlocksAsync(string tag, DateTime startUtc, DateTime endUtc, CancellationToken cancellationToken) public IAsyncEnumerable<HistorianBlock> ReadBlocksAsync(string tag, DateTime startUtc, DateTime endUtc, CancellationToken cancellationToken)
@@ -69,64 +42,86 @@ internal sealed class Historian2020ProtocolDialect
return Missing<HistorianBlock>("StartBlockRetrievalQuery", cancellationToken); return Missing<HistorianBlock>("StartBlockRetrievalQuery", cancellationToken);
} }
public IAsyncEnumerable<HistorianEvent> ReadEventsAsync(DateTime startUtc, DateTime endUtc, CancellationToken cancellationToken) public IAsyncEnumerable<HistorianEvent> ReadEventsAsync(DateTime startUtc, DateTime endUtc, HistorianEventFilter? filter, CancellationToken cancellationToken)
{ {
if (!RuntimeInformation.IsOSPlatform(OSPlatform.Windows)) return UseGrpc
{ ? new HistorianGrpcEventOrchestrator(_options).ReadEventsAsync(startUtc, endUtc, filter, cancellationToken)
return Missing<HistorianEvent>("StartEventDataRetrievalQuery requires Windows for the SSPI path", cancellationToken); : new HistorianWcfEventOrchestrator(_options).ReadEventsAsync(startUtc, endUtc, filter, cancellationToken);
}
return ReadEventsWindowsAsync(startUtc, endUtc, cancellationToken);
}
private IAsyncEnumerable<HistorianEvent> ReadEventsWindowsAsync(DateTime startUtc, DateTime endUtc, CancellationToken cancellationToken)
{
#pragma warning disable CA1416
HistorianWcfEventOrchestrator orchestrator = new(_options);
return orchestrator.ReadEventsAsync(startUtc, endUtc, cancellationToken);
#pragma warning restore CA1416
}
public IAsyncEnumerable<string> BrowseTagNamesAsync(string filter, CancellationToken cancellationToken)
{
return Missing<string>("StartLikeTagNameSearch/GetLikeTagnames", cancellationToken);
}
public Task<HistorianTagMetadata?> GetTagMetadataAsync(string tag, CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
throw new ProtocolEvidenceMissingException("GetTagInfoByName/GetTagInfos");
} }
public Task<HistorianConnectionStatus> GetConnectionStatusAsync(CancellationToken cancellationToken) public Task<HistorianConnectionStatus> GetConnectionStatusAsync(CancellationToken cancellationToken)
{ {
cancellationToken.ThrowIfCancellationRequested(); cancellationToken.ThrowIfCancellationRequested();
if (!OperatingSystem.IsWindows())
{ // Over gRPC (2023 R2) the status is measured from the gRPC handshake (the WCF synthesize-from-
throw new ProtocolEvidenceMissingException("GetConnectionStatus on non-Windows"); // probe path uses the MDAS binding, which can't reach the gRPC port). Non-gRPC stays on WCF.
} return UseGrpc
return Wcf.HistorianWcfStatusClient.GetConnectionStatusAsync(_options, cancellationToken); ? HistorianGrpcStatusClient.GetConnectionStatusAsync(_options, cancellationToken)
: Wcf.HistorianWcfStatusClient.GetConnectionStatusAsync(_options, cancellationToken);
} }
public Task<HistorianStoreForwardStatus> GetStoreForwardStatusAsync(CancellationToken cancellationToken) public Task<HistorianStoreForwardStatus> GetStoreForwardStatusAsync(CancellationToken cancellationToken)
{ {
cancellationToken.ThrowIfCancellationRequested(); cancellationToken.ThrowIfCancellationRequested();
if (!OperatingSystem.IsWindows())
{ // Over gRPC (2023 R2) we return a MEASURED idle-state: the client actually contacts the server
throw new ProtocolEvidenceMissingException("GetStoreForwardStatus on non-Windows"); // (GetHistorianConsoleStatus) and reports ErrorOccurred when unreachable. The active-SF buffer
} // magnitude lives behind the D2 storage-engine console wall and stays false. Non-gRPC transports
return Wcf.HistorianWcfStatusClient.GetStoreForwardStatusAsync(_options, cancellationToken); // keep the synthesized all-false (no SF sidecar to probe). See R4.3 §9.7.
return UseGrpc
? HistorianGrpcStatusClient.GetStoreForwardStatusAsync(_options, cancellationToken)
: Wcf.HistorianWcfStatusClient.GetStoreForwardStatusAsync(_options, cancellationToken);
} }
public Task<string?> GetSystemParameterAsync(string name, CancellationToken cancellationToken) public Task<string?> GetSystemParameterAsync(string name, CancellationToken cancellationToken)
{ {
cancellationToken.ThrowIfCancellationRequested(); cancellationToken.ThrowIfCancellationRequested();
ArgumentException.ThrowIfNullOrWhiteSpace(name); ArgumentException.ThrowIfNullOrWhiteSpace(name);
if (!OperatingSystem.IsWindows()) return UseGrpc
? HistorianGrpcStatusClient.GetSystemParameterAsync(_options, name, cancellationToken)
: Wcf.HistorianWcfStatusClient.GetSystemParameterAsync(_options, name, cancellationToken);
}
public Task<string?> GetServerTimeZoneAsync(CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
// 2023 R2 gRPC returns the real server time-zone name; the 2020 WCF
// GetSystemTimeZoneName is a client-side stub (empty value), so there is no evidence-backed
// value to return on that transport — fail closed rather than hand back an empty string.
if (!UseGrpc)
{ {
throw new ProtocolEvidenceMissingException("GetSystemParameter on non-Windows"); throw new ProtocolEvidenceMissingException("GetSystemTimeZoneName (2020 WCF stub — gRPC/2023R2 only)");
} }
return Wcf.HistorianWcfStatusClient.GetSystemParameterAsync(_options, name, cancellationToken);
return HistorianGrpcStatusClient.GetSystemTimeZoneNameAsync(_options, cancellationToken);
}
public Task<string?> GetRuntimeParameterAsync(string name, CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
ArgumentException.ThrowIfNullOrWhiteSpace(name);
return UseGrpc
? HistorianGrpcStatusClient.GetRuntimeParameterAsync(_options, name, cancellationToken)
: Wcf.HistorianWcfStatusClient.GetRuntimeParameterAsync(_options, name, cancellationToken);
}
public Task<IReadOnlyList<Models.HistorianTagExtendedProperty>> GetTagExtendedPropertiesAsync(string tag, CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
ArgumentException.ThrowIfNullOrWhiteSpace(tag);
return UseGrpc
? Grpc.HistorianGrpcTagClient.GetTagExtendedPropertiesAsync(_options, tag, cancellationToken)
: Wcf.HistorianWcfTagExtendedPropertyClient.GetTagExtendedPropertiesAsync(_options, tag, cancellationToken);
}
public Task<HistorianSqlResult> ExecuteSqlCommandAsync(string command, HistorianSqlExecuteOption option, CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
ArgumentException.ThrowIfNullOrWhiteSpace(command);
return UseGrpc
? Grpc.HistorianGrpcSqlClient.ExecuteSqlCommandAsync(_options, command, option, cancellationToken)
: Wcf.HistorianWcfSqlClient.ExecuteSqlCommandAsync(_options, command, option, cancellationToken);
} }
private static async IAsyncEnumerable<T> Missing<T>( private static async IAsyncEnumerable<T> Missing<T>(
@@ -0,0 +1,18 @@
namespace AVEVA.Historian.Client.Redundancy;
/// <summary>
/// Thrown by a <see cref="HistorianRedundantClient"/> read when every member failed the operation.
/// The per-member failures are aggregated in <see cref="Exception.InnerException"/> (an
/// <see cref="AggregateException"/>).
/// </summary>
public sealed class HistorianAllMembersFailedException : Exception
{
public HistorianAllMembersFailedException(string operation, IReadOnlyList<Exception> failures)
: base($"All historian members failed the '{operation}' operation.", new AggregateException(failures))
{
Operation = operation;
}
/// <summary>The orchestrated operation that failed across all members.</summary>
public string Operation { get; }
}
@@ -0,0 +1,49 @@
using AVEVA.Historian.Client.Models;
namespace AVEVA.Historian.Client.Redundancy;
/// <summary>
/// Default <see cref="IHistorianMember"/> adapter over a <see cref="HistorianClient"/>. For durable
/// redundant writes, pass a member whose write methods enqueue to an R4.1
/// <c>HistorianStoreForwardWriter</c> instead — then a member that is down buffers its writes and
/// replays them on recovery (the pragmatic client-side equivalent of native ReSyncTags).
/// </summary>
public sealed class HistorianClientMember : IHistorianMember
{
private readonly HistorianClient _client;
public HistorianClientMember(string name, HistorianClient client)
{
ArgumentException.ThrowIfNullOrWhiteSpace(name);
Name = name;
_client = client ?? throw new ArgumentNullException(nameof(client));
}
public string Name { get; }
public Task<bool> ProbeAsync(CancellationToken cancellationToken) => _client.ProbeAsync(cancellationToken);
public IAsyncEnumerable<HistorianSample> ReadRawAsync(string tag, DateTime startUtc, DateTime endUtc, int maxValues, CancellationToken cancellationToken) =>
_client.ReadRawAsync(tag, startUtc, endUtc, maxValues, cancellationToken);
public IAsyncEnumerable<HistorianAggregateSample> ReadAggregateAsync(string tag, DateTime startUtc, DateTime endUtc, RetrievalMode mode, TimeSpan interval, CancellationToken cancellationToken) =>
_client.ReadAggregateAsync(tag, startUtc, endUtc, mode, interval, cancellationToken);
public Task<IReadOnlyList<HistorianSample>> ReadAtTimeAsync(string tag, IReadOnlyList<DateTime> timestampsUtc, CancellationToken cancellationToken) =>
_client.ReadAtTimeAsync(tag, timestampsUtc, cancellationToken);
public IAsyncEnumerable<HistorianEvent> ReadEventsAsync(DateTime startUtc, DateTime endUtc, CancellationToken cancellationToken) =>
_client.ReadEventsAsync(startUtc, endUtc, cancellationToken);
public IAsyncEnumerable<string> BrowseTagNamesAsync(string filter, CancellationToken cancellationToken) =>
_client.BrowseTagNamesAsync(filter, cancellationToken);
public Task<HistorianTagMetadata?> GetTagMetadataAsync(string tag, CancellationToken cancellationToken) =>
_client.GetTagMetadataAsync(tag, cancellationToken);
public Task<bool> AddHistoricalValuesAsync(string tag, IReadOnlyList<HistorianHistoricalValue> values, CancellationToken cancellationToken) =>
_client.AddHistoricalValuesAsync(tag, values, cancellationToken);
public Task<bool> SendEventAsync(HistorianEvent historianEvent, CancellationToken cancellationToken) =>
_client.SendEventAsync(historianEvent, cancellationToken);
}
@@ -0,0 +1,34 @@
namespace AVEVA.Historian.Client.Redundancy;
/// <summary>A point-in-time health view of one cluster member.</summary>
public sealed record HistorianMemberStatus
{
public required string Name { get; init; }
/// <summary>True when the member is currently in the healthy pool (preferred for routing).</summary>
public required bool IsHealthy { get; init; }
/// <summary>Consecutive failed operations since the last success.</summary>
public required int ConsecutiveFailures { get; init; }
/// <summary>The most recent operation error, if any.</summary>
public string? LastError { get; init; }
/// <summary>When this member last completed an operation successfully (UTC).</summary>
public DateTime? LastSuccessUtc { get; init; }
/// <summary>When this member was last probed/exercised (UTC).</summary>
public DateTime? LastCheckUtc { get; init; }
}
/// <summary>A snapshot of the whole cluster: the active (preferred-healthy) member plus every member's health.</summary>
public sealed record HistorianClusterStatus
{
public required IReadOnlyList<HistorianMemberStatus> Members { get; init; }
/// <summary>The name of the member reads currently prefer (first healthy in priority order), or null when all are down.</summary>
public string? ActiveMember { get; init; }
/// <summary>True when at least one member is healthy.</summary>
public bool AnyHealthy => Members.Any(m => m.IsHealthy);
}
@@ -0,0 +1,47 @@
namespace AVEVA.Historian.Client.Redundancy;
/// <summary>Tuning for <see cref="HistorianRedundantClient"/>.</summary>
public sealed record HistorianRedundancyOptions
{
/// <summary>
/// Consecutive failed operations before a member drops out of the healthy routing pool. Default 1
/// (demote on first failure; the watchdog restores it). Reads still fail over on every failure
/// regardless — this only governs which member is <em>tried first</em>.
/// </summary>
public int FailureThreshold { get; init; } = 1;
/// <summary>Which members a write is sent to. Default <see cref="HistorianWriteFanout.AllMembers"/>.</summary>
public HistorianWriteFanout WriteFanout { get; init; } = HistorianWriteFanout.AllMembers;
/// <summary>What counts as an overall write success. Default <see cref="HistorianWriteAcknowledgement.All"/>.</summary>
public HistorianWriteAcknowledgement WriteAcknowledgement { get; init; } = HistorianWriteAcknowledgement.All;
/// <summary>
/// When true, <see cref="HistorianRedundantClient.StartAsync"/> runs a watchdog loop that probes
/// members on <see cref="WatchdogInterval"/> to restore health after recovery. Default true.
/// </summary>
public bool RunWatchdog { get; init; } = true;
/// <summary>How often the watchdog probes members. Default 15s.</summary>
public TimeSpan WatchdogInterval { get; init; } = TimeSpan.FromSeconds(15);
}
/// <summary>Which members a redundant write targets.</summary>
public enum HistorianWriteFanout
{
/// <summary>Write to every member (client-side replication). The default redundancy posture.</summary>
AllMembers = 0,
/// <summary>Write only to the preferred healthy member (rely on server-side replication).</summary>
PreferredOnly = 1,
}
/// <summary>What makes a fan-out write "succeed" overall.</summary>
public enum HistorianWriteAcknowledgement
{
/// <summary>Every targeted member must accept the write.</summary>
All = 0,
/// <summary>At least one targeted member must accept the write.</summary>
Any = 1,
}
@@ -0,0 +1,364 @@
using System.Runtime.CompilerServices;
using AVEVA.Historian.Client.Models;
namespace AVEVA.Historian.Client.Redundancy;
/// <summary>
/// R4.4 client-side multi-historian redundancy: orchestrates N single-historian
/// <see cref="IHistorianMember"/>s as one logical client. Reads fail over to the next member when one
/// is down; writes fan out per the configured <see cref="HistorianWriteFanout"/> /
/// <see cref="HistorianWriteAcknowledgement"/> policy; a watchdog restores members to the healthy
/// pool after they recover.
/// <para>
/// Member order is priority order — the first is the preferred primary. This is pure client-side
/// orchestration (no server-side redundancy protocol). For durable writes to a member that is down,
/// back that member's writes with an R4.1 <c>HistorianStoreForwardWriter</c> so they buffer and
/// replay on recovery.
/// </para>
/// </summary>
public sealed class HistorianRedundantClient : IAsyncDisposable
{
private readonly IReadOnlyList<MemberState> _members;
private readonly HistorianRedundancyOptions _options;
private CancellationTokenSource? _watchdogCts;
private Task? _watchdogTask;
public HistorianRedundantClient(IReadOnlyList<IHistorianMember> members, HistorianRedundancyOptions? options = null)
{
ArgumentNullException.ThrowIfNull(members);
if (members.Count == 0)
{
throw new ArgumentException("At least one member is required.", nameof(members));
}
_options = options ?? new HistorianRedundancyOptions();
ArgumentOutOfRangeException.ThrowIfNegativeOrZero(_options.FailureThreshold);
_members = members.Select(m => new MemberState(m, _options.FailureThreshold)).ToList();
}
// ---- reads (failover) ----------------------------------------------------------------
/// <summary>True when any member is reachable.</summary>
public async Task<bool> ProbeAsync(CancellationToken cancellationToken = default)
{
foreach (MemberState member in OrderedCandidates())
{
cancellationToken.ThrowIfCancellationRequested();
try
{
if (await member.Member.ProbeAsync(cancellationToken).ConfigureAwait(false))
{
member.MarkSuccess(DateTime.UtcNow);
return true;
}
member.MarkFailure("Probe returned false.", DateTime.UtcNow);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (Exception ex)
{
member.MarkFailure(ex.Message, DateTime.UtcNow);
}
}
return false;
}
public IAsyncEnumerable<HistorianSample> ReadRawAsync(string tag, DateTime startUtc, DateTime endUtc, int maxValues, CancellationToken cancellationToken = default) =>
StreamWithFailoverAsync(nameof(ReadRawAsync), (m, c) => m.ReadRawAsync(tag, startUtc, endUtc, maxValues, c), cancellationToken);
public IAsyncEnumerable<HistorianAggregateSample> ReadAggregateAsync(string tag, DateTime startUtc, DateTime endUtc, RetrievalMode mode, TimeSpan interval, CancellationToken cancellationToken = default) =>
StreamWithFailoverAsync(nameof(ReadAggregateAsync), (m, c) => m.ReadAggregateAsync(tag, startUtc, endUtc, mode, interval, c), cancellationToken);
public IAsyncEnumerable<HistorianEvent> ReadEventsAsync(DateTime startUtc, DateTime endUtc, CancellationToken cancellationToken = default) =>
StreamWithFailoverAsync(nameof(ReadEventsAsync), (m, c) => m.ReadEventsAsync(startUtc, endUtc, c), cancellationToken);
public IAsyncEnumerable<string> BrowseTagNamesAsync(string filter = "*", CancellationToken cancellationToken = default) =>
StreamWithFailoverAsync(nameof(BrowseTagNamesAsync), (m, c) => m.BrowseTagNamesAsync(filter, c), cancellationToken);
public Task<IReadOnlyList<HistorianSample>> ReadAtTimeAsync(string tag, IReadOnlyList<DateTime> timestampsUtc, CancellationToken cancellationToken = default) =>
ExecuteWithFailoverAsync(nameof(ReadAtTimeAsync), (m, c) => m.ReadAtTimeAsync(tag, timestampsUtc, c), cancellationToken);
public Task<HistorianTagMetadata?> GetTagMetadataAsync(string tag, CancellationToken cancellationToken = default) =>
ExecuteWithFailoverAsync(nameof(GetTagMetadataAsync), (m, c) => m.GetTagMetadataAsync(tag, c), cancellationToken);
// ---- writes (fan-out) ----------------------------------------------------------------
public Task<HistorianRedundantWriteResult> AddHistoricalValuesAsync(string tag, IReadOnlyList<HistorianHistoricalValue> values, CancellationToken cancellationToken = default) =>
FanOutWriteAsync((m, c) => m.AddHistoricalValuesAsync(tag, values, c), cancellationToken);
public Task<HistorianRedundantWriteResult> SendEventAsync(HistorianEvent historianEvent, CancellationToken cancellationToken = default) =>
FanOutWriteAsync((m, c) => m.SendEventAsync(historianEvent, c), cancellationToken);
// ---- status --------------------------------------------------------------------------
/// <summary>A snapshot of every member's health and the currently preferred (active) member.</summary>
public HistorianClusterStatus GetStatus()
{
List<HistorianMemberStatus> members = _members.Select(m => m.Snapshot()).ToList();
string? active = _members.FirstOrDefault(m => m.IsHealthy)?.Member.Name;
return new HistorianClusterStatus { Members = members, ActiveMember = active };
}
// ---- watchdog ------------------------------------------------------------------------
/// <summary>
/// Starts the watchdog loop (no-op when <see cref="HistorianRedundancyOptions.RunWatchdog"/> is
/// false, or already started). The loop probes members every
/// <see cref="HistorianRedundancyOptions.WatchdogInterval"/> to restore health after recovery.
/// </summary>
public Task StartAsync(CancellationToken cancellationToken = default)
{
if (!_options.RunWatchdog || _watchdogTask is not null)
{
return Task.CompletedTask;
}
_watchdogCts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
_watchdogTask = Task.Run(() => RunWatchdogAsync(_watchdogCts.Token), CancellationToken.None);
return Task.CompletedTask;
}
/// <summary>Stops the watchdog loop (if running).</summary>
public async Task StopAsync()
{
if (_watchdogCts is null || _watchdogTask is null)
{
return;
}
await _watchdogCts.CancelAsync().ConfigureAwait(false);
try
{
await _watchdogTask.ConfigureAwait(false);
}
catch (OperationCanceledException)
{
}
finally
{
_watchdogCts.Dispose();
_watchdogCts = null;
_watchdogTask = null;
}
}
/// <summary>Probes every member once now, updating health. Returns the resulting cluster status.</summary>
public async Task<HistorianClusterStatus> CheckHealthAsync(CancellationToken cancellationToken = default)
{
foreach (MemberState member in _members)
{
await ProbeMemberAsync(member, cancellationToken).ConfigureAwait(false);
}
return GetStatus();
}
private async Task RunWatchdogAsync(CancellationToken cancellationToken)
{
try
{
using var timer = new PeriodicTimer(_options.WatchdogInterval);
while (await timer.WaitForNextTickAsync(cancellationToken).ConfigureAwait(false))
{
foreach (MemberState member in _members)
{
if (cancellationToken.IsCancellationRequested)
{
return;
}
// Only spend probes on members that need recovering.
if (!member.IsHealthy)
{
await ProbeMemberAsync(member, cancellationToken).ConfigureAwait(false);
}
}
}
}
catch (OperationCanceledException)
{
}
}
private static async Task ProbeMemberAsync(MemberState member, CancellationToken cancellationToken)
{
try
{
if (await member.Member.ProbeAsync(cancellationToken).ConfigureAwait(false))
{
member.MarkSuccess(DateTime.UtcNow);
}
else
{
member.MarkFailure("Probe returned false.", DateTime.UtcNow);
}
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (Exception ex)
{
member.MarkFailure(ex.Message, DateTime.UtcNow);
}
}
// ---- orchestration core --------------------------------------------------------------
private async Task<T> ExecuteWithFailoverAsync<T>(string operation, Func<IHistorianMember, CancellationToken, Task<T>> op, CancellationToken cancellationToken)
{
var failures = new List<Exception>();
foreach (MemberState member in OrderedCandidates())
{
cancellationToken.ThrowIfCancellationRequested();
try
{
T result = await op(member.Member, cancellationToken).ConfigureAwait(false);
member.MarkSuccess(DateTime.UtcNow);
return result;
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (Exception ex)
{
member.MarkFailure(ex.Message, DateTime.UtcNow);
failures.Add(ex);
}
}
throw new HistorianAllMembersFailedException(operation, failures);
}
private async IAsyncEnumerable<T> StreamWithFailoverAsync<T>(
string operation,
Func<IHistorianMember, CancellationToken, IAsyncEnumerable<T>> op,
[EnumeratorCancellation] CancellationToken cancellationToken)
{
var failures = new List<Exception>();
IReadOnlyList<MemberState> candidates = OrderedCandidates();
for (int i = 0; i < candidates.Count; i++)
{
MemberState member = candidates[i];
cancellationToken.ThrowIfCancellationRequested();
IAsyncEnumerator<T> enumerator = op(member.Member, cancellationToken).GetAsyncEnumerator(cancellationToken);
bool failedBeforeYield = false;
try
{
bool yieldedAny = false;
while (true)
{
try
{
if (!await enumerator.MoveNextAsync().ConfigureAwait(false))
{
break;
}
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (Exception ex)
{
member.MarkFailure(ex.Message, DateTime.UtcNow);
failures.Add(ex);
// Failover is only safe before any row has been observed; a mid-stream failure
// would risk duplicated or skipped rows, so propagate it instead.
if (yieldedAny)
{
throw;
}
failedBeforeYield = true;
break;
}
yieldedAny = true;
yield return enumerator.Current;
}
}
finally
{
await enumerator.DisposeAsync().ConfigureAwait(false);
}
if (failedBeforeYield)
{
continue; // try the next member
}
member.MarkSuccess(DateTime.UtcNow);
yield break;
}
throw new HistorianAllMembersFailedException(operation, failures);
}
private async Task<HistorianRedundantWriteResult> FanOutWriteAsync(Func<IHistorianMember, CancellationToken, Task<bool>> op, CancellationToken cancellationToken)
{
IReadOnlyList<MemberState> targets = _options.WriteFanout == HistorianWriteFanout.PreferredOnly
? OrderedCandidates().Take(1).ToList()
: _members;
HistorianMemberWriteOutcome[] outcomes = await Task.WhenAll(targets.Select(async member =>
{
try
{
bool accepted = await op(member.Member, cancellationToken).ConfigureAwait(false);
if (accepted)
{
member.MarkSuccess(DateTime.UtcNow);
return new HistorianMemberWriteOutcome { Member = member.Member.Name, Accepted = true };
}
member.MarkFailure("Member did not accept the write.", DateTime.UtcNow);
return new HistorianMemberWriteOutcome { Member = member.Member.Name, Accepted = false, Error = "Member did not accept the write." };
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (Exception ex)
{
member.MarkFailure(ex.Message, DateTime.UtcNow);
return new HistorianMemberWriteOutcome { Member = member.Member.Name, Accepted = false, Error = ex.Message };
}
})).ConfigureAwait(false);
bool succeeded = _options.WriteAcknowledgement == HistorianWriteAcknowledgement.Any
? outcomes.Any(o => o.Accepted)
: outcomes.All(o => o.Accepted);
return new HistorianRedundantWriteResult { Outcomes = outcomes, Succeeded = succeeded };
}
/// <summary>Members in priority order, healthy first, so reads prefer a known-good member but still
/// fall back to a currently-unhealthy one as a last resort.</summary>
private IReadOnlyList<MemberState> OrderedCandidates()
{
var healthy = new List<MemberState>(_members.Count);
var unhealthy = new List<MemberState>();
foreach (MemberState member in _members)
{
(member.IsHealthy ? healthy : unhealthy).Add(member);
}
healthy.AddRange(unhealthy);
return healthy;
}
public async ValueTask DisposeAsync()
{
await StopAsync().ConfigureAwait(false);
}
}
@@ -0,0 +1,31 @@
namespace AVEVA.Historian.Client.Redundancy;
/// <summary>The per-member outcome of a fan-out write to one cluster member.</summary>
public sealed record HistorianMemberWriteOutcome
{
public required string Member { get; init; }
/// <summary>True when this member accepted the write.</summary>
public required bool Accepted { get; init; }
/// <summary>The delivery error, if this member did not accept the write.</summary>
public string? Error { get; init; }
}
/// <summary>The aggregate result of a redundant (fan-out) write across the targeted members.</summary>
public sealed record HistorianRedundantWriteResult
{
public required IReadOnlyList<HistorianMemberWriteOutcome> Outcomes { get; init; }
/// <summary>
/// Whether the write succeeded overall under the configured
/// <see cref="HistorianWriteAcknowledgement"/> policy.
/// </summary>
public required bool Succeeded { get; init; }
/// <summary>The members that accepted the write.</summary>
public IEnumerable<HistorianMemberWriteOutcome> Accepted => Outcomes.Where(o => o.Accepted);
/// <summary>The members that rejected or failed the write.</summary>
public IEnumerable<HistorianMemberWriteOutcome> Failed => Outcomes.Where(o => !o.Accepted);
}
@@ -0,0 +1,33 @@
using AVEVA.Historian.Client.Models;
namespace AVEVA.Historian.Client.Redundancy;
/// <summary>
/// One historian the <see cref="HistorianRedundantClient"/> orchestrates. Exposes the read/write
/// subset the cluster coordinates (failover reads, fan-out writes). Abstracted from
/// <see cref="HistorianClient"/> so redundancy logic is unit-testable without a server; the default
/// adapter is <see cref="HistorianClientMember"/>.
/// </summary>
public interface IHistorianMember
{
/// <summary>A stable, human-readable name for this member (used in status + diagnostics).</summary>
string Name { get; }
Task<bool> ProbeAsync(CancellationToken cancellationToken);
IAsyncEnumerable<HistorianSample> ReadRawAsync(string tag, DateTime startUtc, DateTime endUtc, int maxValues, CancellationToken cancellationToken);
IAsyncEnumerable<HistorianAggregateSample> ReadAggregateAsync(string tag, DateTime startUtc, DateTime endUtc, RetrievalMode mode, TimeSpan interval, CancellationToken cancellationToken);
Task<IReadOnlyList<HistorianSample>> ReadAtTimeAsync(string tag, IReadOnlyList<DateTime> timestampsUtc, CancellationToken cancellationToken);
IAsyncEnumerable<HistorianEvent> ReadEventsAsync(DateTime startUtc, DateTime endUtc, CancellationToken cancellationToken);
IAsyncEnumerable<string> BrowseTagNamesAsync(string filter, CancellationToken cancellationToken);
Task<HistorianTagMetadata?> GetTagMetadataAsync(string tag, CancellationToken cancellationToken);
Task<bool> AddHistoricalValuesAsync(string tag, IReadOnlyList<HistorianHistoricalValue> values, CancellationToken cancellationToken);
Task<bool> SendEventAsync(HistorianEvent historianEvent, CancellationToken cancellationToken);
}
@@ -0,0 +1,78 @@
namespace AVEVA.Historian.Client.Redundancy;
/// <summary>
/// Mutable per-member health used by <see cref="HistorianRedundantClient"/> to route reads and
/// fan out writes. Thread-safe: ops update it from multiple call sites and the watchdog loop.
/// </summary>
internal sealed class MemberState
{
private readonly Lock _lock = new();
private readonly int _failureThreshold;
private bool _isHealthy = true;
private int _consecutiveFailures;
private string? _lastError;
private DateTime? _lastSuccessUtc;
private DateTime? _lastCheckUtc;
public MemberState(IHistorianMember member, int failureThreshold)
{
Member = member;
_failureThreshold = failureThreshold;
}
public IHistorianMember Member { get; }
public bool IsHealthy
{
get
{
lock (_lock)
{
return _isHealthy;
}
}
}
public void MarkSuccess(DateTime utc)
{
lock (_lock)
{
_consecutiveFailures = 0;
_isHealthy = true;
_lastError = null;
_lastSuccessUtc = utc;
_lastCheckUtc = utc;
}
}
public void MarkFailure(string? error, DateTime utc)
{
lock (_lock)
{
_consecutiveFailures++;
_lastError = error;
_lastCheckUtc = utc;
if (_consecutiveFailures >= _failureThreshold)
{
_isHealthy = false;
}
}
}
public HistorianMemberStatus Snapshot()
{
lock (_lock)
{
return new HistorianMemberStatus
{
Name = Member.Name,
IsHealthy = _isHealthy,
ConsecutiveFailures = _consecutiveFailures,
LastError = _lastError,
LastSuccessUtc = _lastSuccessUtc,
LastCheckUtc = _lastCheckUtc,
};
}
}
}

Some files were not shown because too many files have changed in this diff Show More