Captured the native 2023 R2 client's gRPC event send (new capture-send-event
harness scenario): it rides HistoryService.AddStreamValues with the SAME "OS"
(0x534F) storage-sample buffer the WCF path already uses (HistorianEventWrite-
Protocol) — confirming "no distinct RPC", and that it is NOT the historical
write's "ON" buffer. Diffed the write-enabled vs read-only v8 Event open: byte-
identical apart from per-session crypto, so the existing OpenSession event path
is reused unchanged.
So SendEvent-over-gRPC was pure assembly of proven parts:
- HistorianGrpcEventWriteOrchestrator = v8 Event open + CM_EVENT registration
(UpdC3/RegisterTags/EnsureTags) + AddStreamValues(OS buffer).
- HistorianClient.SendEventAsync now routes to it for RemoteGrpc (WCF otherwise).
Live-validated end-to-end against the 2023 R2 server: pure-managed SDK send →
AddStreamValues BSuccess=true → the event reads back from the server (markers
confirmed in returned event rows). The native gRPC RegisterTags(24B) +
EnsureTags(86B) byte-match our serializers (new GrpcEventSendProtocolTests
golden, closing the 83-vs-86 EnsureTags question). Gated live test
SendEventAsync_OverGrpc_AcceptsEvent (opt-in HISTORIAN_GRPC_EVENT_SEND=1).
331 offline tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Mined the full decompiled stock 2023 R2 managed client as the oracle for every
still-pending gRPC item. Governing fact: ArchestrA.HistorianAccess is a C++/CLI
shim into native HistorianClient; the managed Grpc*Client wrappers have zero
call sites, so buffer-building/dispatch for the pending items is native (absent
from the binaries). Sharpened verdicts into handoff.md:
- Items 4/5/6 + OpenStorageConnection: hard-confirmed walled, with real reasons
(SQL gated out client-side via IsManagedHistorian; no Revision RPC in the gRPC
contract; LoadBlocks response is a native blob behind the storage console handle).
- Items 3 (SendEvent) and 7 (DeleteTEP): moved from vague to precise, LOCAL-box
capturable targets (PackToVtq btValues / DeleteTagExtendedPropertiesByName
BtInput with deleteFromServer=true).
Also correct the HistorianGrpcRevisionProbe doc comment: it probes the
non-streamed ORIGINAL-insert path (AddNonStreamValues), a distinct capability
from revision EDITS (native AddRevisionValues trio, no gRPC RPC) — the prior
comment conflated them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Used the provided stock client as an oracle to verify the event read path. The
capture-event harness returns 50 real events, and the instrument-grpc-nonstream rewrite
captured the exact GetNextEventQueryResultBuffer.result buffer (63,192 bytes, version
0x0B=11, rowCount 50 = 25 Alarm.Set + 25 Alarm.Clear). Feeding that real buffer through
HistorianEventRowProtocol.Parse exposed a latent parser bug.
The real buffer layout is: version(2) + rowCount(4) + headerField(4, =0x1E) followed by
MARKERLESS rows (rowFormat(2)=7 + filetime(8) + 8x u16 slots + compact-ascii type +
propCount + props). The parser wrongly treated the one-time 0x1E field as a per-row
marker and re-consumed [marker+format] for every row, so it decoded only the FIRST row
of any multi-row buffer and stopped. This is not gRPC-specific: the captured WCF v9
buffer has the identical 0900 <rowCount> 1E000000 0700 header, so the shipped WCF event
read had the same latent multi-row truncation.
Fix: read a 10-byte buffer header (skip the 0x1E field once) and parse markerless rows;
accept container version 9 (WCF) and 11 (gRPC), mirroring the interface-version gate that
accepts History 11 and 12.
Verified: the real 50-row buffer now decodes to exactly 50 events, ending cleanly at
end-of-buffer (Parse_RealStockClientCapture_DecodesAllEvents, gated on
HISTORIAN_EVENT_CAPTURE_NDJSON so it skips without the gitignored capture), plus a
synthetic v11 golden test. 328 offline tests pass.
The parse path is now verified against the provided client's real event data on both
transports; the only remaining gap for gRPC events is the server delivering rows to our
connection (the documented retrieval-server-gate).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Pursued hypothesis #3 (connection-frame capture). Built a TLS-terminating tee proxy
(artifacts/.../httpcap/, gitignored: self-signed server cert, forwards through the
loopback tunnel, logs decrypted HTTP/1.1 + gRPC-Web both directions) and ran a native
capture-event (returns 50 rows) and our SDK diagnostic (0 rows) through the SAME
proxy/upstream for a clean A/B.
Findings:
- The stock client is gRPC-Web/HTTP-1.1 (alpn empty), and clientCert=none on every
connection — confirming (with the decompile) that hypothesis #2 (TLS client cert) is
moot: the native presents no client cert.
- Connection topology differs: the native opens 5 TLS connections, one per service, and
runs the event query (StartEventQuery/GetNext/EndEventQuery) on a DEDICATED
RetrievalService connection, separate from the HistoryService connection that opened and
registered the session. Our SDK collapses every service onto one connection. (Matches the
decompile: the stock client has a separate GrpcClientBase per service.)
- Framing differs benignly: native uses content-length + Expect: 100-continue; SDK uses
transfer-encoding: chunked. The server accepts both (StartEventQuery returns a valid
handle), so framing is not the gate. No hidden header on either side.
Tested the topology hypothesis with a new env-gated switch
(HISTORIAN_GRPC_EVENT_SPLIT_CHANNEL=1): run StartEventQuery/GetNext/EndEventQuery on a
dedicated RetrievalService connection (no re-handshake, reusing the session handle —
mirroring native conn4), registration staying on the main connection. Result: still
0B00000000001E000000 (0 rows), QH=1063. Splitting the event query onto its own connection
does not make rows flow — the server correlates by session handle, not connection, so
topology is not the row-scoping gate.
Every angle is now exhausted (payload, transport, metadata/interceptor/cert, topology,
data store). The gate is a server-internal per-connection retrieval working-set in the
native HistorianClient C++ core, unreachable from a pure-managed client. Conclusion
unchanged: auth-solved / retrieval-server-gated; ReadEventsAsync over gRPC keeps the
no-row throw, event reads use WCF. 56 offline gRPC/event tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Tested grpc-event-query-capture.md's leading next-session hypothesis — that the
native client's Grpc.Core HTTP/2 transport (vs our Grpc.Net.Client + GrpcWebHandler
gRPC-Web) is why event reads return zero rows. Added HistorianGrpcChannelFactory
.CreateHttp2 (plain HTTP/2 over SocketsHttpHandler, no gRPC-Web wrap) and an
HISTORIAN_GRPC_EVENT_HTTP2 switch on the event orchestrator (event path only; reads
stay gRPC-Web).
Live side-by-side against the event-bearing 2023 R2 server, everything else held
constant: the full v8 chain (ExchangeKey auth, CM_EVENT RegisterTags/EnsureTags=True,
StartEventQuery with a valid handle) runs end-to-end over BOTH native HTTP/2 and
gRPC-Web, and the server returns the byte-identical version-11 rowCount-0 terminal
(0B00000000001E000000) on both transports. Transport choice makes no difference —
the leading hypothesis is disproven and the zero-row scoping sits above the gRPC
transport layer.
Also confirmed the native capture-event harness queries a 30-day historical window
(returns 50 rows), so the native read is connection-scoped historical retrieval,
not a live subscription.
CreateHttp2 + the env switch + the EventChannelMode diagnostic are retained for
further connection-level probing. 44 offline tests pass; orchestrator stays on the
no-row throw.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Extends the instrument-grpc rewrite to log string (strHandle) + uint (uiHandle /
queryRequestType) params, not just byte[], and captures our SDK's live v8
openParameters for a byte-diff against the native.
Result of the exhaustive comparison (all live-confirmed via the opt-in
EventReadDiagnostic test):
- StartEventQuery request: byte-identical to the native (v6 layout)
- v8 OpenConnection openParameters: byte-identical to the native (302B) once
ClientNodeName matches — every control byte/ConnectionType/token/ShardId
- handle usage identical: ExchangeKey->contextKey, registration->storage GUID
(strHandle), query->client uint (uiHandle); handles valid (RTag/EnsT=True)
- queryRequestType=3, registration order, gzip metadata header — all match
- window has events (native returns 50 now); eventCount not it
Every observable client-side byte matches the native, yet the server scopes 0
events to our connection. The event RPCs succeed over our transport and return a
valid EMPTY result (not a transport error), so this is a connection/server-level
difference (session affinity tied to the native Grpc.Core HTTP/2 connection or a
connection identity used to scope events) — invisible to and unfixable by client
payload matching. Needs server-side insight, not more wire RE.
Added opt-in diagnostics (RegistrationDiag, LastResultBufferHex,
LastEventOpenRequestHex). 326/326 offline; gated test still pins the no-row throw.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Continues closing the event-row gap after the v8 ExchangeKey/RC4 auth breakthrough.
- HistorianGrpcHandshake: the v8 EVENT path skips StorageService.ValidateClientCredential
(the native event connection authenticates purely via ExchangeKey + the RC4 token;
running the Negotiate loop establishes a different session scope).
- HistorianGrpcEventOrchestrator.RegisterCmEventTag: simplified to the exact native
gRPC event sequence (UpdateClientStatus -> RegisterTags -> EnsureTags -> GetHistorianInfo
-> GetSystemParameter x7), dropping the 2020-WCF-era cross-service GetV probes and
params-before-register that the gRPC event flow does not use. eventCount 5 -> 100.
- Opt-in diagnostics (RegistrationDiag, LastResultBufferHex/LastErrorBufferHex; gated
EventReadDiagnostic test) for the continued investigation.
STATUS: auth + StartEventQuery + registration all succeed live (RTag/EnsT=True, valid
query handle), but GetNext returns version-11 rowCount-0 while the native returns 50 for
a BYTE-IDENTICAL request. Every observable wire element matches the native. The remaining
unknown is the string/uint HANDLE field VALUES the native uses per event RPC — the
instrument-grpc capture logged only byte[] params, not the handle fields. Next: extend
the IL rewrite to log strHandle/uiHandle/queryRequestType, re-capture, and match.
326/326 offline; gated test still pins the no-row throw.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Implements the reverse-engineered v8 credential token in pure managed code and
wires the full event-connection auth chain. Live result: the v8 OpenConnection
now AUTHENTICATES against the 2023 R2 server (past the 132/171 AuthenticationFailed
wall) — the crypto is solved.
- HistorianNativeHandshake.DeriveExchangeKeyClientKey: client key = SHA256(ECDH
shared secret) via ECDiffieHellman.DeriveKeyFromHash(SHA256), matching the native
ECDiffieHellmanCng{Hash,SHA256}.DeriveKeyMaterial.
- BuildExchangeKeyCredentialToken + Rc4: token = RC4(password-UTF16LE, key=MD5(clientKey)).
Reproduces a live-captured token EXACTLY (verified offline) — the native
HistorianCrypto.NRC4_V2.aahCryptV2 scheme (MD5-keyed RC4). Pure managed; nothing
AVEVA shipped. RC4 pinned by the standard test vector.
- OpenSession(eventConnection:true): ExchangeKey -> derive client key -> token ->
v8 OpenConnection with ConnectionType=Event + the token. Orchestrator re-armed.
- HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc: the 86-byte native gRPC
CM_EVENT EnsureTags (8-byte header + ...2f27 event-type GUID), replacing the
2020 WCF 83-byte CTagMetadata on the gRPC event registration.
Goldens: RC4 standard vector + token construction. 326/326 offline.
KNOWN REMAINING: the event query still returns zero rows (GetNext yields a 10-byte
zero-row buffer). Auth + StartEventQuery succeed; the query-layer detail (vs the
native row-returning capture) is the last step. Gated test still pins the no-row
throw; opt-in diagnostic (HISTORIAN_GRPC_EVENT_DIAG) surfaces the journey.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Implements HistoryService.ExchangeKey as a pure-managed P-256 ECDH key exchange
and wires it ahead of the v8 Event OpenConnection.
- HistorianNativeHandshake.BuildExchangeKeyClientHello / DeriveExchangeKeySecret:
.NET ECDiffieHellman (nistP256); wire format "ECK1" + u32(32) + X(32) + Y(32),
decoded from the live capture. No native AVEVA dependency.
- HistorianGrpcHandshake.OpenSession(eventConnection: true): runs ExchangeKey on
the context-key handle before the v8 OpenConnection.
- Guardrail HistorianGrpcHandshakeRoutingTests scoped to the token-loop closure:
still pins that the Negotiate token loop routes to ValidateClientCredential (not
ExchangeKey), while allowing the legitimate ExchangeKey call in OpenSession.
Live result: ExchangeKey succeeds (server accepts our public key) and the v8
OpenConnection error advances from 132/34 "Failed to get client key" to 132/171
AuthenticationFailed — the ECDH cleared the client-key layer. The remaining
blocker is the 26-byte v8 credential token, which must be derived from the ECDH
shared secret (token KDF, not yet recovered). Orchestrator stays on v6 (set
eventConnection: true to re-arm once the KDF lands). 323/323 offline.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Builds the native 2023 R2 version-8 OpenConnection format, which (unlike v6)
carries a ConnectionType byte (Event vs Process) — required because the 2023 R2
server returns event rows only on an Event connection.
- HistorianOpen2Protocol.SerializeNativeOpenConnectionVersion8: reproduces the
302-byte v8 layout decoded from a live capture (version 8, markers, client-key
GUID, username HString, length-prefixed credential token, ClientType /
ConnectionType / flag / constant word / compact metadata / two empty strings;
the tail reuses WriteClientCommonInfo). Golden-tested
(Version8EventSerializerReproducesCapturedNativeStructure).
- HistorianNativeHandshake.BuildEventOpenConnectionVersion8Request: ConnectionType=
Event, zeroed credential token (mirroring how v6 zeros its credential block and
relies on the separate ValidateClientCredential handshake).
- HistorianGrpcHandshake.OpenSession: optional eventConnection switch; the
OpenConnection failure path now decodes the native error (type/code/ASCII).
Path A (reuse ValidateClientCredential + zeroed token) was live-tested and
DISPROVEN: the server parses the v8 buffer but rejects it at the auth check with
native 132/34 "EstablishConnection Failed to get client key" — the v8 path looks
up the client key in the registry HistoryService.ExchangeKey (ECDH) populates,
not the one ValidateClientCredential does. The event orchestrator is therefore
reverted to the v6 session (gated test still pins the no-row throw). The v8
serializer/builder are retained for Path B (implement ExchangeKey). 323/323
offline tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Captured the stock 2023 R2 client doing a gRPC event read (50 rows flowed) to
resolve the open "gRPC event ROW retrieval returns zero rows" item. Two captured
differences from our SDK's path; this lands the first (necessary) one plus the
capture tooling.
- HistorianEventQueryProtocol.CreateStartEventQueryAttempts: add a `version`
parameter (default 5 = the 2020 WCF format, unchanged). The gRPC event
orchestrator now opts into version 6 — the leading `06` plus a 5-byte trailing
zero pad — which is the envelope the stock 2023 R2 client sends. The two
buffers are otherwise byte-identical (filter block, UTC string, metadata
namespace). Golden test Version6EmptyFilterMatchesCapturedGrpcEnvelope pins it.
- Grpc2023CaptureHarness: new `capture-event` scenario drives HistorianAccess
over an Event-type gRPC connection (CreateEventQuery -> EventQueryArgs ->
StartQuery -> MoveNext) so the wide-net instrument-grpc-nonstream rewrite dumps
StartEventQuery.requestBuffer + the row result. Hostname defaults sanitized to
HISTORIAN_GRPC_HOST / "localhost" (removed hardcoded server name).
NECESSARY BUT NOT SUFFICIENT: live validation shows v6 alone does not make rows
flow — the read also requires an Event-type connection, which our SDK's v6 Open2
format cannot express (see the companion docs commit). The gated
ReadEventsAsync_OverGrpc_* test correctly still pins the no-row throw. 322/322
offline tests pass; WCF event path unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Adds an internal RE probe (HistorianGrpcTagWriteOrchestrator.
ProbeDeleteTagExtendedPropertiesAsync) testing whether gRPC's single shared
channel lifts the WCF per-connection working-set wall that blocks DelTep.
Live result (2023 R2, History iface 12): both GetTgByNm + GetTepByNm primes
succeed on the one shared channel, yet DelTep is still rejected (native code=1)
and the property survives. So the working set is populated by the native
client's in-process registration state, not the wire session — neither WCF's
per-service channels nor gRPC's shared channel reproduce it. DelTep stays
server-blocked on BOTH transports and remains unshipped.
Gated negative test DeleteTagExtendedProperties_OverGrpc_ProbeMultiplexedChannel
pins this (primes succeed, delete rejected, prop survives) and flips if a future
server/registration lifts the wall. Comment in HistorianClient records the
probe. 321 offline tests pass; live test passes bounded at ~11s.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
- HistorianTagExtendedPropertyProtocol.ParseResponse: fix the multi-property/
multi-group response shape captured live from the 2023 R2 server. The server
returns one group per property (the tag name repeats), each propertyCount=1, and
a uint16 searchability-flags trailer per property (0x0003 built-in, 0x0001 user-
added) — NOT the single-byte group trailer the old model assumed, which drifted
one byte per group and threw "expected 0x09 found 0x01" on any buffer with more
than one property. Now reads the per-property uint16 trailer (tolerates a legacy
1-byte tail). Fixes read-back on both WCF and gRPC. Adds GetTagExtendedPropertiesRaw
for future captures.
- HistorianGrpcStatusClient.GetConnectionStatusAsync (plan #5): synthesize connection
status from a measured gRPC handshake (OpenConnection yielding a storage-session
GUID => connected), mirroring the WCF synthesize-from-probe approach. Routed in
Historian2020ProtocolDialect on UseGrpc (the WCF path used the MDAS binding, which
can't reach the gRPC port).
- HistorianGrpcSqlClient: record the negative plan-#4 result — a HistoryService.
RegisterTags prime does NOT clear the server-side CSrvDbConnection fault (tried live
on both 0x402/0x401); the op stays bounded behind ProtocolEvidenceMissingException.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Adds HistorianGrpcEventOrchestrator: opens a read-only gRPC session, replays the
CM_EVENT registration (UpdateClientStatus -> 6 GetSystemParameter -> RegisterTags
-> cross-service version probes -> EnsureTags), then StartEventQuery -> loop
GetNextEventQueryResultBuffer -> EndEventQuery, reusing the WCF query builder and
row parser verbatim. Routed in Historian2020ProtocolDialect on UseGrpc.
The captured registration buffers (CmEventTagId, UpdC3 blob, RTag2 buffer, GETHI
builder, pre-register param list, native-error decode) are extracted into a shared
HistorianEventRegistrationProtocol so the WCF and gRPC paths can't drift; the WCF
orchestrator is refactored onto it with no behavior change.
Live finding (2026-06-22): the chain runs and StartEventQuery succeeds, but the
gRPC server long-polls GetNextEventQueryResultBuffer on no data (it blocks to the
deadline instead of returning the synchronous 5-byte code-85 terminal the WCF op
returns). Per-call gRPC-Web deadlines proved unreliable over a tunnel, so the read
is hard-bounded by an overall linked-CTS budget (<=30s; gRPC honors token
cancellation). On the no-row path it throws ProtocolEvidenceMissingException rather
than assert a false-empty list. Row-level retrieval awaits an event-bearing 2023 R2
server (the dev box holds no events).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Wire the config operations that previously only worked over WCF onto RemoteGrpc,
reusing the proven 2020 byte serializers verbatim inside the protobuf bytes fields
(keyed by the Open2 session handle). Live-verified against a real 2023 R2 server
where noted.
Read ops (live-verified):
- GetRuntimeParameterAsync -> StatusService.GetRuntimeParameter (GETRP serializer)
- GetTagExtendedPropertiesAsync -> RetrievalService.GetTagExtendedPropertiesFromName
(GetTepByNm serializer + sequence paging; page-0 FillBufferFromVector is the
benign no-data terminator, matched to the WCF break-and-return-empty semantics)
Server-walled (bounded with captured evidence):
- ExecuteSqlCommandAsync -> RetrievalService.ExecuteSqlCommand. The request rides
the RPC but the server-side CSrvDbConnection.ExecuteSqlCommand faults
(IndexOutOfRange / native err 38) on a DB-connection precondition the pure
managed gRPC session doesn't establish (same class as OpenStorageConnection).
Surfaced as ProtocolEvidenceMissingException.
Write ops (tooled + routed, sandbox-gated — not run destructively live):
- EnsureTagAsync / DeleteTagAsync / RenameTagsAsync / AddTagExtendedPropertiesAsync
via HistoryService.EnsureTags / DeleteTags / StartJob / AddTagExtendedProperties
on a write-enabled (0x401) session, reusing the WCF golden serializers. The WCF
priming discovery-dance is omitted (the M3 gRPC write probe worked without it);
add it first if a live sandbox run is rejected.
Routed in Historian2020ProtocolDialect / HistorianClient on the RemoteGrpc branch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Document the WCF-vs-gRPC surface and fix a stale claim.
- README: add a "Transport matrix (WCF vs gRPC)" section with a per-operation
table. Mark the config ops the gRPC server exposes-but-untooled with a distinct
legend state (recovered RPC + bytes buffer named) vs genuinely unavailable, so
"not tooled" is not conflated with "not possible".
- README: document the gRPC live-test env vars (HISTORIAN_GRPC_HOST/_PORT/_TLS/
_DNSID/_TIMEOUT/_WRITE_SANDBOX_TAG) and refresh the Status section (test count
+ the live-verified gRPC surface).
- The "gRPC requires VerifyServerInterfaceVersion=false against a v12 server" note
was obsolete: the gate already accepts History 11 AND 12 (AcceptedVersions), and
the live gRPC suite runs with the default verification on. Corrected in the
README, CLAUDE.md, and the HistorianServerVersionGate docstring.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Route GetStoreForwardStatusAsync to a gRPC path that actually contacts the
server (StatusService.GetHistorianConsoleStatus) instead of synthesizing an
all-false result blind. On a reachable/normal server it returns the
not-storing baseline but MEASURED; when the server is unreachable or the
console-status call fails it reports ErrorOccurred with the underlying error
(the old synthesis never contacted the server). The active-SF buffer
magnitude (Storing/Pending/DataStored) stays false because it lives behind
the D2 storage-engine console wall.
Non-gRPC transports keep the synthesized fallback. Live-verified against the
2023 R2 server; gated integration test
GetStoreForwardStatusAsync_OverGrpc_ReturnsMeasuredIdleState added. README
operation table updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Add HistorianGrpcStoreForwardStatusProbe and the `grpc-sf-status-probe` CLI
command. The idle-baseline run against the live 2023 R2 server resolves the
plan's §9.3 handle question: the direct StorageService SF pull RPCs
(GetSFParameter / GetRemainingSnapshotsSize) require the OpenStorageConnection
console handle and are D2-gated (err 132, identical under read-only and
write-enabled sessions), while StatusService.GetHistorianConsoleStatus IS
reachable on the session string handle (=3 at idle).
Records the gRPC re-scope and the idle-baseline findings in
docs/plans/store-forward-cache-reverse-engineering.md §9. The probe writes
nothing and releases any console session immediately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Adds AVEVA.Historian.Client.Redundancy — HistorianRedundantClient orchestrates
N single-historian members (IHistorianMember; default HistorianClientMember
over HistorianClient) as one logical client. Pure client-side, no server-side
redundancy protocol, no RE.
- Reads fail over to the next member in priority order. Streaming reads only
fail over BEFORE the first row is observed; a mid-stream failure propagates
(failing over mid-stream would risk duplicated/skipped rows).
- Writes fan out: WriteFanout AllMembers | PreferredOnly, with All | Any ack
policy, returning a per-member HistorianRedundantWriteResult.
- Per-member health: FailureThreshold demotes a failing member out of the
preferred pool; a background watchdog (PeriodicTimer) + CheckHealthAsync
re-probe and restore recovered members. GetStatus() snapshot + ActiveMember.
- Composes with R4.1: back a member's writes with a HistorianStoreForwardWriter
so a down member buffers and replays on recovery — the pragmatic client-side
equivalent of native ReSyncTags.
14 unit tests (no server): failover order, mid-stream no-failover, all-fail
aggregation, probe-any-up, fan-out ack policies, PreferredOnly, soft reject,
health demotion + CheckHealthAsync restore, watchdog recovery. Full suite 307
green. Roadmap R4.4 marked shipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Adds AVEVA.Historian.Client.StoreForward — a client-side store-and-forward
layer over the historian write surface (AddHistoricalValuesAsync /
SendEventAsync). Producers enqueue writes; the writer persists them and
replays on reconnect so a transient disconnect never drops data. This is the
roadmap's recommended pragmatic outbox, NOT a bit-faithful reimplementation of
AVEVA's native SF cache (that stays deferred) — pure managed, no RE.
- HistorianOutboxEntry / HistorianOutboxEntryKind: buffered-write envelope
- IHistorianOutboxStore + InMemoryHistorianOutboxStore (tests) +
FileHistorianOutboxStore (crash-durable: atomic temp+move JSON per entry,
FIFO by filename sequence that resumes past on-disk max, corrupt-file
quarantine). OutboxJson normalizes event object? properties off JsonElement.
- IHistorianWriteSink + HistorianClientWriteSink (HistorianClient-backed)
- HistorianStoreForwardWriter: enqueue, single-flight FIFO FlushAsync with
head-of-line blocking, optional MaxDeliveryAttempts dead-lettering,
DropOldest/Reject overflow policy, background drain loop (retry on reconnect),
GetStatusAsync snapshot mirroring server SF Pending/Storing/ErrorOccurred.
12 unit tests (no server): durability-across-restart, reconnect-drain, FIFO
order/head-of-line, dead-letter, overflow policies, background auto-drain.
Full suite 293 green. Roadmap R4.1 marked shipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Extended the historical-write serializer from Float-only to all five analog types EnsureTagAsync
supports. Captured each type's "ON" buffer live from the native client (sandbox tag per type,
written + captured + deleted):
- The 4-byte value descriptor (C0 10 01 00) is CONSTANT across types — it does not encode the type.
- The value is u32(0) + native-width value, width by the tag's declared type:
Float->float32, Double->double64, Int2->int16, Int4->int32, UInt4->uint32.
HistorianHistoricalWriteProtocol.SerializeAddStreamValuesBuffer now takes the HistorianDataType and
encodes accordingly (unsupported types throw ProtocolEvidenceMissingException). The orchestrator
resolves the type from the tag-info NativeDataTypeDescriptor via MapDataType. Harness capture-write
gained --data-type. Golden-tested against all five live captures + the gated write/read-back test
validated each type end-to-end through the pure-managed SDK; 281 unit tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Public HistorianClient.AddHistoricalValuesAsync(tag, values) inserts non-streamed original
(backfill) values for an existing tag over the 2023 R2 gRPC front door. The pure-managed SDK
wrote a value and read it back live (gated test AddHistoricalValuesAsync_OverGrpc_WritesAndReadsBack
PASSED against the real server).
- HistorianGrpcHistoricalWriteOrchestrator: write-enabled (0x401) session ->
RetrievalService.GetTagInfosFromName (resolves the per-tag GUID = the tag-info TypeId, and
registers the tag on the session) -> HistoryService.AddStreamValues("ON" buffer) per sample.
- HistorianHistoricalValue (public record: TimestampUtc, Value, OpcQuality=192).
- gRPC-only: non-RemoteGrpc transports throw ProtocolEvidenceMissingException (the 2020 WCF
non-streamed write is architecturally blocked, D2).
- Float value encoding only (the captured type); other types rejected by the serializer.
275 unit tests pass; the new gated live write/read-back test is green against the 2023 R2 server.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Managed serializer for the historical-write "ON" buffer, byte-for-byte matching the live capture
(WcfHistoricalWriteProtocolTests golden-tests the exact 56-byte native buffer). Layout: "ON"(0x4E4F)
+ count + lengths + 16B tag GUID + sample FILETIME + u16 quality(192) + 4B descriptor + received
FILETIME + value. Value encoding (Float, captured): an 8-byte slot = u32(0) + float32(value) — the
4-byte float in the high dword, NOT a double. The 16B tag GUID is the per-tag GUID the SDK already
parses as ParseTagInfoRecord's "typeId" (confirmed: it appears at offset 8 of GetTagInfosFromName's
response = where typeId is read, and in EnsureTags' response + the "ON" buffer).
Only the Float encoding is captured; other types rejected until captured. Next: gRPC orchestrator
(write-enabled session -> EnsureTags -> resolve tag GUID -> AddStreamValues) + public
AddHistoricalValuesAsync + live write/read-back.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Live-probed StorageService.OpenStorageConnection against the 2023 R2 server over a
write-enabled (0x401) session. Every attempt — sweeping ConnectionMode (0x401/0x402/0x1),
StorageSessionId-in (Open2-GUID / empty), and FreeDiskSpace — returns the IDENTICAL native
error type=4 code=85 ("session not registered"), so it's a structural refusal, not a bad
field value.
Decode (two corroborating facts):
- Error 85 is the same code the event read returns before RegisterTags2 (see
HistorianWcfEventOrchestrator) — a generic "session not registered for this op".
- The 2023 R2 decompile shows OpenStorageConnection lives on a SEPARATE GrpcStorageClient
(the storage engine's SF/snapshot channel, own port + service identity); HistorianAccess
drives non-streamed writes through the native C++ HistorianClient, never this op.
So the roadmap's mapped "missing console session" step was wrong. The real non-streamed-write
precondition is the front-door HistoryService.RegisterTags (RTag2-family) for the target tag —
which is exactly why the R3.1 batch failed at AddNonStreamValues (no tag registered ->
StoreNonStreamValues had no route). Matches the original 2020-WCF D2 hypothesis.
Remaining (both need a native gRPC capture; do not guess bytes): the regular-tag RegisterTags
btTagInfos (only CM_EVENT's tag-GUID form is known) and the AddNonStreamValues btInput.
- HistorianGrpcStorageConnectionProbe + grpc-open-storage-connection CLI (opens nothing
persistent; CloseStorageConnection on success)
- corrected revision-write-path.md §R3.1 follow-up + hcal-roadmap R3.1/R3.2 rows
- gated regression test pinning the error-85 refusal
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Empirically decoded the AddNonStreamValues btInput framing against the live 2023
R2 server (grpc-nonstream-decode command + ProbeNonStreamedBuffersAsync driver).
Every transaction rolled back (bCommit=false) — no data written.
Finding: the btInput is assembled native-C++-side (not in any decompile), so 6
evidence-based framings (44-54B, packed HISTORIAN_VALUE2 variants) were probed.
All 6 returned the IDENTICAL server error while an empty buffer returned a
different InvalidParameter — so non-empty buffers pass parameter validation into
CHistStorageConnection::StoreNonStreamValues, which routes to the
\.\pipe\aahStorageEngine\console pipe server-side. Identical-across-framings =>
the blocker is NOT the btInput layout but a missing storage-engine console
session / tag-registration precondition for the connection.
Next step (untested): StorageService.OpenStorageConnection + tag registration
(RegisterTags/AddTagidPairs/AddShardTagids) before AddNonStreamValues, then
commit + read-back on a sandbox tag. Documented in revision-write-path.md
(R3.1 decode section); raw artifact gitignored.
272 unit tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
The D2 storage-engine-pipe wall is WCF-transport-specific. On the 2023 R2 gRPC
front door, TransactionService is a first-class service AND the gateway to the
storage engine, so the Open2 storage-session GUID (uppercase) is accepted
directly as strHandle with no legacy pipe.
Live-verified against the real 2023 R2 server over a write-enabled (0x401) gRPC
session: AddNonStreamValuesBegin returns a real strTransactionId, and
AddNonStreamValuesEnd(bCommit=false) discards it cleanly (no data written). On
2020 WCF the same op returns UnknownClient(51) for every handle + priming chain.
- HistorianGrpcRevisionProbe + grpc-revision-probe CLI command + gated test
NonStreamedWriteTransaction_OverGrpc_BeginsAndDiscards (live pass).
- HistorianGrpcHandshake.OpenSession gains an optional connectionMode param
(default read-only 0x402; pass 0x401 for write-enabled) — non-breaking.
- Docs: revision-write-path.md "the wall is gone" section; roadmap M3 section,
R3.1-R3.3 rows, one-glance table, and status note updated honestly.
Not yet shipped: the AddNonStreamValues btInput VTQ buffer is uncaptured (never
guess wire bytes), so no value-commit is implemented. Scope is non-streamed
ORIGINAL backfill; revision EDITS (R4.2) remain pipe-only even on gRPC.
272 unit tests pass; sanitization scan clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Live-probed both R1.3 and R1.4 against a real 2023 R2 server over the gRPC
StatusService; implemented the one that carries an evidence-backed value.
R1.3 GetServerTimeZoneAsync — SHIPPED:
- StatusService.GetSystemTimeZoneName(uiHandle) returns the real server zone
over RemoteGrpc (the 2020 WCF op is a client-side stub returning empty).
- HistorianGrpcStatusClient.GetSystemTimeZoneNameAsync -> dialect routing ->
public HistorianClient.GetServerTimeZoneAsync. Non-gRPC transports fail
closed with ProtocolEvidenceMissingException (no empty-string lie).
- Golden message-shape unit test + non-gRPC guardrail unit test + gated live
test. 271 unit tests pass.
R1.4 GetHistorianInfoAsync (EventStorageMode) — bounded out on gRPC too:
- gRPC GetHistorianInfo is the same named-value query as 2020 WCF (only
HistorianVersion resolves); EventStorageMode + 7 variants fail on both
GetHistorianInfo and GetSystemParameter. The 518-byte struct is filled by a
native vtable+648 HCAL call, not the gRPC op (per the 2023 R2 decompile), so
the field is never on the wire. Not shipped on any transport. Closes the
roadmap's open "build against a live 2023 R2 server" caveat.
Also correct the stale M3 roadmap section: D2 already proved
Transaction.AddNonStreamValues* rides the storage-engine pipe (STransactPipeClient2
-> aaStorageEngine), not WCF — same wall as R4.2 — so M3-over-WCF is blocked, not
"the path that is NOT the gated cache push".
Docs: hcal-roadmap.md, wcf-historian-info.md, wcf-status-localhost.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Probes the 2023 R2 gRPC browse path and records the finding. The front door does
NOT hit the 2020 WCF metadata-server-pipe wall.
- RetrievalService.StartTagQuery is cracked: the server (CMdServer::StartActiveTagnamesQuery
over \.\pipe\aahMetadataServer\console) parses the filter as OData. startswith()/
contains()/eq/empty succeed and return the 8-byte (queryHandle, tagCount); SQL-LIKE "%"
and glob "*" fail with "ODataFilter: bad token". Live: 220 Sys* tags counted.
- QueryTag (paging) remains: every guessed btRequest returns a constant native error
type 4 / code 72 (content-independent) -> framing needs a native capture, not guessing.
Adds RE probe helpers Grpc/HistorianGrpcTagClient.ProbeStartTagQuery + ProbeTagQuerySequence,
a gated StartTagQuery_OverGrpc_AcceptsODataFilter test, and the finding doc
docs/reverse-engineering/grpc-tag-query-odata.md. Browse is not yet wired (QueryTag open).
217 unit tests pass; 5/5 live gRPC tests pass. No tag names/identities committed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Routes HistorianClient.GetTagMetadataAsync over gRPC when Transport==RemoteGrpc,
via the new Grpc/HistorianGrpcTagClient calling RetrievalService.GetTagInfosFromName
(the plural string-handle metadata op).
- String handle = the Open2 storage-session GUID formatted uppercase (the format
that resolves the native string-handle path); threaded out of the shared handshake
via a new HistorianGrpcHandshake.Session { ClientHandle, StorageSessionId, StringHandle }.
- Request btTagNames = uint count + per-name(uint charCount + UTF-16LE) — golden-byte
unit-tested (BuildTagNamesBuffer).
- Response btTagInfos = uint count + CTagMetadata records — decoded by the existing
HistorianTagQueryProtocol.ParseGetTagInfoResponse; data type via the shared MapDataType.
The 2020 WCF string-handle wall does NOT apply on the gRPC front door, as the
string-handle-wall RE note predicted. LIVE-VERIFIED against a real 2023 R2 server:
GetTagMetadataAsync returns the requested tag with a valid decoded data type.
216 unit tests pass. Captured framing confirmed live then discarded; no tag names
or identities committed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Roadmap docs/plans/hcal-roadmap.md, milestone M0 (gRPC parity for the DONE
surface). Now unblocked for live verification by a reachable 2023 R2 server.
- R0.4 Probe over gRPC: new HistorianGrpcProbe calls History/Retrieval/Status
GetInterfaceVersion (unauthenticated). ProbeAsync routes over gRPC when
Transport==RemoteGrpc. LIVE-VERIFIED against a real 2023 R2 server — needs no
credentials (runs before the auth loop), so it works despite the auth blocker.
- R0.3 System parameter over gRPC: new HistorianGrpcStatusClient calls
StatusService.GetSystemParameter over the authenticated session; routed in the
dialect. Built + unit-tested (request/response field mapping pinned).
Live-verification pending an auth fix (see below).
- Extracted the proven auth handshake from HistorianGrpcReadOrchestrator into
shared Grpc/HistorianGrpcHandshake (reused by read + status + future
browse/metadata). Repointed the IL structural guardrail test to it.
- Diagnostics: round-failure now decodes the native server error + hex/ASCII
preview (HistorianNativeHandshake.DescribeError). This surfaced the live auth
blocker as SEC_E_LOGON_DENIED (0x8009030C) at NTLM round 1 — framing is correct,
the credential did not validate. Probable cause: stale file password or NAM-domain
NTLM restriction (Kerberos/RDP works, NTLM denied; no SPN path over the tunnel).
216 unit tests pass; live gRPC probe passes. Sanitization scan clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
First live-verified gRPC read against a real 2023 R2 Historian. The handshake
previously failed at round 0 (cred-independent) because the SSPI/Negotiate token
loop was routed to HistoryService.ExchangeKey. ExchangeKey is a separate
key-exchange/cert-path op, not the Negotiate loop — the token loop belongs on
StorageService.ValidateClientCredential, which kept the 2020 inBuff/outBuff token
framing the SDK's WrapValidateClientCredentialToken/TryRead helpers already build.
Captured + diffed against the recovered 2023 R2 protobuf contract and the
decompiled stock client; routing the loop to ValidateClientCredential completes
the full chain (ValidateClientCredential x N -> OpenConnection -> StartQuery ->
GetNextQueryResultBuffer) and returns rows.
- HistorianGrpcReadOrchestrator: token loop now calls
StorageService.ValidateClientCredential(Handle, InBuff); corrected the op-map
doc comment (was asserting the wrong ExchangeKey mapping).
- HistorianServerVersionGate: accept History interface version 12 alongside 11.
Live server reports History=12, Retrieval=4, Storage=4; the buffers are
byte-identical (a live read returns rows), so 12 is buffer-compatible. Retrieval
stays pinned at 4 (matches). New AcceptedVersions() supports multi-version gates.
- New HistorianGrpcHandshakeRoutingTests: IL-level structural guardrail that
disassembles the orchestrator (incl. lambda closures) and asserts the handshake
invokes ValidateClientCredential and never ExchangeKey — fails if the regression
returns.
- Updated gate tests + CLAUDE.md gRPC op-map.
240 unit tests pass on the full stack; 210 on this branch's base. The byte
payloads remain the proven 2020 protocol.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
DelTep (extended-property delete) — wire format captured + serializer
golden-proven, but live delete is server-blocked and NOT exposed publicly:
- Captured the DelTep inBuff via a cross-session trick (harness add-tep gains
--tep-skip-add + read-for-sync before --tep-delete; Capture-DeleteTagExtended
Properties.ps1 / decode-del-tep-capture.py). Layout = same group framing as
AddTEx but property-name-only (no 0x43 value) + 0x00 group trailer.
- SerializeDeleteRequest + 4 golden tests pin the server-accepted buffer.
- A decisive experiment shows SDK-added properties ARE deletable (the native
client read-syncs and deletes one), so SDK-add is complete; the SDK's own
DelTep is rejected by CHistStorage::DeleteTagExtendedProperties even with
byte-identical inBuff, matching mode/handle, GetTgByNm+GetTepByNm prime, open
channel, and 60s retries. Root cause: the native multiplexes services over one
connection (per-connection working set); the SDK's per-service WCF channels
don't reproduce it. Kept as documented-but-blocked internal orchestrator path;
no public HistorianClient delete API.
Bounded out with evidence (no code; docs + roadmap + probe):
- R1.12 localized-property write — no op on 2020 (mirror of R1.6); no
*LocalizedPropert*/TagLocalized* symbol in any current/*.dll.
- R1.13 non-analog tag create — GATED; native AddTag rejects every non-analog
type client-side (ValidationFailed, before any WCF op): SingleByteString,
DoubleByteString, Int1 all fail, Float works. No Discrete type in the native
enum, no TagType setter. No wire request to capture.
- R1.3 timezone + R1.4 EventStorageMode — re-confirmed 2023R2/gRPC-only from
the Runtime DB schema (no timezone param, no EventStorageMode anywhere) and a
parameter-op probe (GetSystemParameter + GETRP return null/throw for every
candidate; only HistorianVersion works).
238 unit tests pass; full solution builds with 0 warnings.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Adds user-defined extended properties to an existing tag via the 2020 WCF
AddTEx (AddTagExtendedProperties) op. Write-enabled connection + uppercase
storage-session GUID handle; reuses the write orchestrator open/priming chain.
The AddTEx inBuff is the exact inverse of the R1.5 GetTepByNm read-response
framing, so the serializer mirrors the read parser:
uint32 groupCount + 0x01(group) + [0x09+u16+ASCII tag] + uint32 propCount
+ per prop{ 0x02 + [0x09+u16+ASCII name] + 0x43 VT_BSTR + u16 payloadLen
+ u16 charCount + UTF-16 value } + 0x01(group trailer) + 0x00(terminator).
The trailing 0x00 is required — without it inBuff is one byte short and the
server throws SErrorException in CHistStorage::AddTagExtendedProperties. The
golden fixture pins the clean inBuff the live server accepted (dumped via
AVEVA_HISTORIAN_TEP_DUMP); read-back verified via R1.5. String (0x43) values only.
Delete (DelTep) is deferred: the native DeleteTagExtendedPropertiesByName does a
client-side sync check and returns err 229 for a just-added property, so the
DelTep request never reaches the wire and its inBuff can't be captured yet.
Shipped: HistorianClient.AddTagExtendedPropertiesAsync/AddTagExtendedPropertyAsync;
HistorianTagExtendedPropertyProtocol.SerializeAddRequest; orchestrator path;
golden WcfTagExtendedPropertyWriteProtocolTests (4); gated live write/read-back test;
native-harness `add-tep` scenario + Capture-AddTagExtendedProperties.ps1 +
decode-add-tep-capture.py. Doc: wcf-add-tag-extended-properties.md. 233 tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Tag rename has no dedicated WCF op — the (old,new) name batch rides the
generic History StartJob (StJb) job buffer; the server returns a job id and
applies renames asynchronously. Handle is the uppercase storage-session GUID,
Open2 in write mode; reuses the write orchestrator's open+priming chain.
jobBuffer layout (decoded + server-validated): byte[7] zero prefix + uint32
pairCount + per pair (uint32 oldCharCount + UTF-16 oldName + uint32
newCharCount + UTF-16 newName), order (old,new). The raw instrument capture
mangles the final byte with MDAS chunk markers (the R1.1 lesson), so the golden
fixture pins the CLEAN byte[] the SDK handed the channel (dumped via
AVEVA_HISTORIAN_RENAME_DUMP) — the exact buffer the live server accepted and
renamed with.
Gated server-side by the AllowRenameTags system parameter (default 0): when
disabled the native client rejects pre-wire (err 132); the managed SDK surfaces
it as StartJob=false -> Accepted=false. Enabling needs a Historian config
reload, not just a storage-engine restart.
Shipped: HistorianClient.RenameTagAsync/RenameTagsAsync -> HistorianTagRenameResult;
HistorianTagRenameProtocol; orchestrator RenameTags/SendStartJobRename; golden
WcfTagRenameProtocolTests (4, pins server-accepted buffer); gated live test
RenameTagsAsync_AgainstLocalHistorian_RenamesSandboxTag (passed end-to-end).
Native-harness `rename` scenario + Capture-RenameTags.ps1 + decode-rename-capture.py.
Doc: docs/reverse-engineering/wcf-rename-tags.md. 213 tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Ship SQL command execution over the 2020 WCF aa/Retr/ExeC + aa/Retr/GetR ops:
HistorianClient.ExecuteSqlCommandAsync(sql) -> HistorianSqlResult (columns +
typed rows). String-handle ops reached with the Open2 storage-session GUID
formatted uppercase (the handle format that unlocked GETRP/GETHI).
Chain: Retr.GetV prime -> ExeC(handle, sql, option=0, ref queryHandle) ->
GetR loop. Key gotcha captured: GetR returns FALSE even on success -- the byte
stream is in pResultBuff regardless; false just signals the final page. So the
orchestrator consumes the buffer first, then stops on a false result / empty page.
GetR's pResultBuff is an NRBF-serialized System.Data.DataTable
(SerializationFormat.Xml: members XmlSchema (XSD) + XmlDiffGram (rows)).
BinaryFormatter is removed from .NET 10, so the stream is decoded read-only with
the System.Formats.Nrbf package (NrbfDecoder) + XDocument -- no BinaryFormatter,
no code execution. Values are typed per the XSD type, falling back to string.
Adds: HistorianSqlResult / HistorianSqlColumn / HistorianSqlExecuteOption models,
HistorianSqlResultProtocol (NRBF + diffgram parser), HistorianWcfSqlClient
(ExeC/GetR orchestration with an AVEVA_HISTORIAN_SQL_DUMP diagnostic), dialect +
public API. Golden WcfSqlResultProtocolTests pinned to the real clean GetR stream
for the benign "SELECT 1 AS ProbeValue" (no sensitive data); gated live tests
(single cell + multi-column/multi-row/NULL). Doc: wcf-exec-sql.md; roadmap R1.1
DONE; wall doc + memory updated (incl. the QTB-server-side nuance). 229 tests green.
Note: a raw instrument-wcf capture corrupts a large pResultBuff with MDAS
transport chunk markers (0x9F); the clean contract-level byte[] is dumped via the
AVEVA_HISTORIAN_SQL_DUMP env var for the golden fixture.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Ship tag extended-property reads over the 2020 WCF aa/Retr/GetTepByNm op:
HistorianClient.GetTagExtendedPropertiesAsync(tag) -> name/value pairs.
String-handle op reached with the Open2 storage-session GUID formatted
uppercase (same format that unlocked GETRP/GETHI/ExeC). Routed via the
name-based native path (GetTagExtendedPropertiesByName, server-fetch flag),
not the index-based TagQuery path.
Evidence-backed findings from the capture:
- GetTepByNm (and GetTgByNm) succeed with the uppercase handle -- further
validates the resolved string-handle wall.
- QTB (StartTagQuery) does NOT punch through: captured uppercase, it still
fails server-side (CMdServer::StartActiveTagnamesQuery over the
aahMetadataServer pipe) -- a metadata-server blocker, not handle format.
- R1.6 (localized properties) has NO distinct op (only error-message/UI-text
localization in the managed client); collapses into R1.5. Closed, not throwing.
Wire format (golden-pinned, synthetic bytes -- no dev tag names committed):
- request tagNames = uint count + per-name(uint charCount + UTF-16)
- response = uint tagCount + per-tag(marker + compact-ASCII name +
uint propCount + per-prop(marker + compact-ASCII name + 0x43 VT_BSTR value)
+ trailer); sequence-paged.
Adds: HistorianTagExtendedProperty model, HistorianTagExtendedPropertyProtocol
(codec), HistorianWcfTagExtendedPropertyClient (orchestration), dialect +
public API; golden WcfTagExtendedPropertyProtocolTests (4) + gated live test
(HISTORIAN_TEP_TAG). Tooling: Capture-TagExtendedProperties.ps1,
decode-tag-properties-capture.py, harness tag-extended-properties scenario.
Docs: wcf-tag-extended-properties.md; roadmap R1.5 DONE / R1.6 collapsed;
wall doc + memory updated with the QTB-server-side nuance. 228 tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Execute HCAL roadmap R1.2 (GetRuntimeParameterAsync) end-to-end, and in doing so
discover that the "string-handle wall" blocking R1.1/R1.4/R1.5/R1.6 was a handle
FORMAT bug, not a missing native session/filter registration.
R1.2 (shipped, live-verified):
- Captured native GetRuntimeParameter -> WCF op aa/Stat/GETRP (string-handle op,
GETHI's shape), via scripts/Capture-RuntimeParam.ps1 + instrument-wcf-{write,read}message.
- HistorianRuntimeParameterProtocol serializes pRequestBuff (54 67 01 00 + uint
nameCount + per-name uint charCount + UTF-16) and parses pResponseBuff (version +
uint resultCount + CRetVariant 0x43 VT_BSTR + uint16 len + uint16 charCount + UTF-16).
- IStatusServiceContract2.GetRuntimeParameter (GETRP) op; HistorianWcfStatusClient
passes the Open2 storage-session GUID as the string handle, UPPERCASE.
- Public HistorianClient.GetRuntimeParameterAsync(name) via the dialect.
- Golden WcfRuntimeParameterProtocolTests + gated live test; returns HistorianVersion.
String-handle wall RESOLVED (proven, public APIs deferred):
- The Open2 storage GUID works as the string handle when sent UPPERCASE
(ToString("D").ToUpperInvariant()); earlier "blocked" probes used lowercase.
- Live-probed GETHI (R1.4) -> returns data; ExeC (R1.1) -> Retr.GetV prime -> ExeC ->
GetR returns a BinaryFormatter-serialized .NET DataTable. Gated
StringHandleProbeDiagnosticTests + scripts/Capture-ExecSql.ps1 + exec-sql harness scenario.
- Docs flipped: wcf-string-handle-wall.md RESOLVED banner; roadmap R1.1/R1.4 reachable,
R1.5/R1.6 likely; wcf-status-localhost.md GETRP section.
- R1.1/R1.4 public APIs NOT shipped: ExeC needs a GetR paging loop + a BinaryFormatter-
stream parser (BinaryFormatter is removed from .NET 10); GETHI full-info struct needs
its own capture.
223 unit tests pass; gated live tests green against the local 2020 Historian.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Roadmap M1 R1.7. Filters are set on the native EventQuery object via
AddEventFilter(property, HistorianComparisionType, value) — NOT EventQueryArgs
(time/count/order only). Found via a new harness --dump-type-members command.
Captured the native filtered StartEventQuery pRequestBuff (Capture-EventFilter.ps1 +
harness --event-filter knob) and diffed Equal(0) vs Contains(12) to isolate the
operator field. Filter block (decoded byte-for-byte):
ushort 0 + uint filterCount + uint condCount + uint nameLen + name(UTF-16) +
uint 1 + ushort op + uint 1 + value(0x09-LEN-0x00 compact-ASCII) + byte 0
The filter is REAL, not inert (unlike the analog-summary knobs): a non-matching
predicate returns 0 events; Type=Equal=User.Write returns only User.Write events.
Verified live via both the native harness and the SDK.
- HistorianClient.ReadEventsAsync(start, end, HistorianEventFilter, ct) overload
- HistorianEventFilter + HistorianEventComparison (18 ops, ordinals = native)
- Filter encoding in HistorianEventQueryProtocol (empty-filter path unchanged)
- Golden-byte tests (block match, op field, empty-filter regression) + gated live test
Single string-valued predicate only; multi-filter (OR) / multi-condition (AND via
AddEventFilterCondition) framing is partially captured and not shipped. 216 unit
tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Joseph Doherty