Adds HistorianGrpcEventOrchestrator: opens a read-only gRPC session, replays the
CM_EVENT registration (UpdateClientStatus -> 6 GetSystemParameter -> RegisterTags
-> cross-service version probes -> EnsureTags), then StartEventQuery -> loop
GetNextEventQueryResultBuffer -> EndEventQuery, reusing the WCF query builder and
row parser verbatim. Routed in Historian2020ProtocolDialect on UseGrpc.
The captured registration buffers (CmEventTagId, UpdC3 blob, RTag2 buffer, GETHI
builder, pre-register param list, native-error decode) are extracted into a shared
HistorianEventRegistrationProtocol so the WCF and gRPC paths can't drift; the WCF
orchestrator is refactored onto it with no behavior change.
Live finding (2026-06-22): the chain runs and StartEventQuery succeeds, but the
gRPC server long-polls GetNextEventQueryResultBuffer on no data (it blocks to the
deadline instead of returning the synchronous 5-byte code-85 terminal the WCF op
returns). Per-call gRPC-Web deadlines proved unreliable over a tunnel, so the read
is hard-bounded by an overall linked-CTS budget (<=30s; gRPC honors token
cancellation). On the no-row path it throws ProtocolEvidenceMissingException rather
than assert a false-empty list. Row-level retrieval awaits an event-bearing 2023 R2
server (the dev box holds no events).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Self-contained plan for finishing gRPC surface parity: live-verify the
sandbox-gated writes, port ReadEvents (CM_EVENT registration state machine),
SendEvent (capture-blocked), the SQL server-wall stretch, and optional
GetConnectionStatus. Includes the proven reuse pattern and live-verification setup
so it survives context compaction.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
- GetRuntimeParameter / GetTagExtendedProperties now live-verified over gRPC
- ExecuteSqlCommand marked server-walled (new legend state)
- tag-config writes marked sandbox-gated (new legend state)
- document the HISTORIAN_GRPC_WRITE_SANDBOX_TAG live-test gate
- rewrite the matrix summary to reflect what was learned tooling the config ops
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
- GetRuntimeParameterAsync_OverGrpc_ReturnsValue (live)
- GetTagExtendedPropertiesAsync_OverGrpc_DoesNotThrow (live; empty for system tags)
- ExecuteSqlCommandAsync_OverGrpc_IsServerWalled (live; pins the captured wall)
- TagWriteLifecycle_OverGrpc_CreatesAddsPropRenamesDeletes — DESTRUCTIVE, gated on
HISTORIAN_GRPC_WRITE_SANDBOX_TAG; self-cleaning create->addprop->verify->rename->delete
Full gRPC live suite 19/19 green against a real 2023 R2 server (write lifecycle
skips without the sandbox tag); 317 offline green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Wire the config operations that previously only worked over WCF onto RemoteGrpc,
reusing the proven 2020 byte serializers verbatim inside the protobuf bytes fields
(keyed by the Open2 session handle). Live-verified against a real 2023 R2 server
where noted.
Read ops (live-verified):
- GetRuntimeParameterAsync -> StatusService.GetRuntimeParameter (GETRP serializer)
- GetTagExtendedPropertiesAsync -> RetrievalService.GetTagExtendedPropertiesFromName
(GetTepByNm serializer + sequence paging; page-0 FillBufferFromVector is the
benign no-data terminator, matched to the WCF break-and-return-empty semantics)
Server-walled (bounded with captured evidence):
- ExecuteSqlCommandAsync -> RetrievalService.ExecuteSqlCommand. The request rides
the RPC but the server-side CSrvDbConnection.ExecuteSqlCommand faults
(IndexOutOfRange / native err 38) on a DB-connection precondition the pure
managed gRPC session doesn't establish (same class as OpenStorageConnection).
Surfaced as ProtocolEvidenceMissingException.
Write ops (tooled + routed, sandbox-gated — not run destructively live):
- EnsureTagAsync / DeleteTagAsync / RenameTagsAsync / AddTagExtendedPropertiesAsync
via HistoryService.EnsureTags / DeleteTags / StartJob / AddTagExtendedProperties
on a write-enabled (0x401) session, reusing the WCF golden serializers. The WCF
priming discovery-dance is omitted (the M3 gRPC write probe worked without it);
add it first if a live sandbox run is rejected.
Routed in Historian2020ProtocolDialect / HistorianClient on the RemoteGrpc branch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Document the WCF-vs-gRPC surface and fix a stale claim.
- README: add a "Transport matrix (WCF vs gRPC)" section with a per-operation
table. Mark the config ops the gRPC server exposes-but-untooled with a distinct
legend state (recovered RPC + bytes buffer named) vs genuinely unavailable, so
"not tooled" is not conflated with "not possible".
- README: document the gRPC live-test env vars (HISTORIAN_GRPC_HOST/_PORT/_TLS/
_DNSID/_TIMEOUT/_WRITE_SANDBOX_TAG) and refresh the Status section (test count
+ the live-verified gRPC surface).
- The "gRPC requires VerifyServerInterfaceVersion=false against a v12 server" note
was obsolete: the gate already accepts History 11 AND 12 (AcceptedVersions), and
the live gRPC suite runs with the default verification on. Corrected in the
README, CLAUDE.md, and the HistorianServerVersionGate docstring.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
The gRPC integration suite was missing live coverage for ReadAggregateAsync and
ReadAtTimeAsync, the two tooled gRPC reads the WCF suite already exercises. Add
them so the full tooled gRPC surface is live-tested like WCF.
- ReadAggregateAsync_OverGrpc_ReturnsTimeWeightedAverageRows
- ReadAggregateAsync_OverGrpc_AcceptsRetrievalMode (Min/MaxWithTime, BestFit)
- ReadAtTimeAsync_OverGrpc_ReturnsRequestedTimestamps
The aggregate tests self-calibrate their window from a real raw sample
(SeedAggregateWindowAsync): the interpolating modes (TimeWeightedAverage /
Min/MaxWithTime) do a slow bounding-value scan and return empty when the window
has no raw data, so a fixed "last N hours" window blows the per-call deadline
against an idle server. Anchoring the window where data actually exists keeps the
scan cheap and returns rows on idle or live servers alike. Adds an optional
HISTORIAN_GRPC_TIMEOUT knob (per-call deadline override) for slow links.
Full tooled gRPC surface now live-green: 15/15 gRPC integration tests pass
against a real 2023 R2 server; 313 offline tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
gRPC GetStoreForwardStatusAsync now contacts the server (measured idle-state,
ErrorOccurred on failure) instead of blind synthesis; active-SF magnitude
stays D2-gated. Includes the grpc-sf-status-probe RE tool + re-scope doc.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Route GetStoreForwardStatusAsync to a gRPC path that actually contacts the
server (StatusService.GetHistorianConsoleStatus) instead of synthesizing an
all-false result blind. On a reachable/normal server it returns the
not-storing baseline but MEASURED; when the server is unreachable or the
console-status call fails it reports ErrorOccurred with the underlying error
(the old synthesis never contacted the server). The active-SF buffer
magnitude (Storing/Pending/DataStored) stays false because it lives behind
the D2 storage-engine console wall.
Non-gRPC transports keep the synthesized fallback. Live-verified against the
2023 R2 server; gated integration test
GetStoreForwardStatusAsync_OverGrpc_ReturnsMeasuredIdleState added. README
operation table updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Add HistorianGrpcStoreForwardStatusProbe and the `grpc-sf-status-probe` CLI
command. The idle-baseline run against the live 2023 R2 server resolves the
plan's §9.3 handle question: the direct StorageService SF pull RPCs
(GetSFParameter / GetRemainingSnapshotsSize) require the OpenStorageConnection
console handle and are D2-gated (err 132, identical under read-only and
write-enabled sessions), while StatusService.GetHistorianConsoleStatus IS
reachable on the session string handle (=3 at idle).
Records the gRPC re-scope and the idle-baseline findings in
docs/plans/store-forward-cache-reverse-engineering.md §9. The probe writes
nothing and releases any console session immediately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Adds AVEVA.Historian.Client.Redundancy — HistorianRedundantClient orchestrates
N single-historian members (IHistorianMember; default HistorianClientMember
over HistorianClient) as one logical client. Pure client-side, no server-side
redundancy protocol, no RE.
- Reads fail over to the next member in priority order. Streaming reads only
fail over BEFORE the first row is observed; a mid-stream failure propagates
(failing over mid-stream would risk duplicated/skipped rows).
- Writes fan out: WriteFanout AllMembers | PreferredOnly, with All | Any ack
policy, returning a per-member HistorianRedundantWriteResult.
- Per-member health: FailureThreshold demotes a failing member out of the
preferred pool; a background watchdog (PeriodicTimer) + CheckHealthAsync
re-probe and restore recovered members. GetStatus() snapshot + ActiveMember.
- Composes with R4.1: back a member's writes with a HistorianStoreForwardWriter
so a down member buffers and replays on recovery — the pragmatic client-side
equivalent of native ReSyncTags.
14 unit tests (no server): failover order, mid-stream no-failover, all-fail
aggregation, probe-any-up, fan-out ack policies, PreferredOnly, soft reject,
health demotion + CheckHealthAsync restore, watchdog recovery. Full suite 307
green. Roadmap R4.4 marked shipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Adds AVEVA.Historian.Client.StoreForward — a client-side store-and-forward
layer over the historian write surface (AddHistoricalValuesAsync /
SendEventAsync). Producers enqueue writes; the writer persists them and
replays on reconnect so a transient disconnect never drops data. This is the
roadmap's recommended pragmatic outbox, NOT a bit-faithful reimplementation of
AVEVA's native SF cache (that stays deferred) — pure managed, no RE.
- HistorianOutboxEntry / HistorianOutboxEntryKind: buffered-write envelope
- IHistorianOutboxStore + InMemoryHistorianOutboxStore (tests) +
FileHistorianOutboxStore (crash-durable: atomic temp+move JSON per entry,
FIFO by filename sequence that resumes past on-disk max, corrupt-file
quarantine). OutboxJson normalizes event object? properties off JsonElement.
- IHistorianWriteSink + HistorianClientWriteSink (HistorianClient-backed)
- HistorianStoreForwardWriter: enqueue, single-flight FIFO FlushAsync with
head-of-line blocking, optional MaxDeliveryAttempts dead-lettering,
DropOldest/Reject overflow policy, background drain loop (retry on reconnect),
GetStatusAsync snapshot mirroring server SF Pending/Storing/ErrorOccurred.
12 unit tests (no server): durability-across-restart, reconnect-drain, FIFO
order/head-of-line, dead-letter, overflow policies, background auto-drain.
Full suite 293 green. Roadmap R4.1 marked shipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
R3.2 and the one-glance table still read "Float-only"; the shipped
AddHistoricalValuesAsync covers Float/Double/Int2/Int4/UInt4 (golden-tested
+ live write/read-back). Correct both lines to match code + tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Extends the historical-write surface from Float to all five analog types (Float/Double/Int2/Int4/
UInt4), each captured live + golden-tested + write/read-back validated through the pure-managed SDK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Extended the historical-write serializer from Float-only to all five analog types EnsureTagAsync
supports. Captured each type's "ON" buffer live from the native client (sandbox tag per type,
written + captured + deleted):
- The 4-byte value descriptor (C0 10 01 00) is CONSTANT across types — it does not encode the type.
- The value is u32(0) + native-width value, width by the tag's declared type:
Float->float32, Double->double64, Int2->int16, Int4->int32, UInt4->uint32.
HistorianHistoricalWriteProtocol.SerializeAddStreamValuesBuffer now takes the HistorianDataType and
encodes accordingly (unsupported types throw ProtocolEvidenceMissingException). The orchestrator
resolves the type from the tag-info NativeDataTypeDescriptor via MapDataType. Harness capture-write
gained --data-type. Golden-tested against all five live captures + the gated write/read-back test
validated each type end-to-end through the pure-managed SDK; 281 unit tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
delete-tag drives the native client's DeleteTags (the clean-delete path, unlike the SDK's WCF
DelT which can leave the row). Primes the write session with AddTag first (DeleteTags on a fresh
connection returns UnknownClient(51) until the client is registered). Used to remove the capture
sandbox tag SdkM3CaptureSandbox from the live server (DeleteTags returned success).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Reverse-engineered + shipped the SDK's first historical/backfill write capability:
- Corrected the M3 path (OpenStorageConnection dead-end -> the write rides
HistoryService.AddStreamValues, NOT AddNonStreamValues/TransactionService).
- Built a net481 capture harness + IL-rewrite to capture the native 2023 R2 'ON' write buffer.
- HistorianHistoricalWriteProtocol ('ON' serializer, golden-tested) +
HistorianGrpcHistoricalWriteOrchestrator + public AddHistoricalValuesAsync.
- Live-validated: pure-managed SDK wrote a value and read it back over gRPC.
275 unit tests pass; gated live write/read-back test green. Float-only, gRPC-only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Extends the IL-rewrite to log out (byref) byte[] params at method exit (ldarg + ldind.ref), not
just byte[] inputs. This captured GetTagInfosFromName's response, which located the per-tag GUID at
offset 8 = exactly where ParseTagInfoRecord reads "typeId" — proving the SDK already parses the GUID
AddStreamValues needs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Marks M3 historical writes SHIPPED + live-validated across the roadmap (R3.2/R3.3/one-glance),
revision-write-path.md §"R3.1 CAPTURED", and the CLAUDE.md Required SDK Surface (the new write op,
gRPC-only, AddStreamValues "ON" path, Float-only, distinct from the still-blocked AddS2 streaming
path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Public HistorianClient.AddHistoricalValuesAsync(tag, values) inserts non-streamed original
(backfill) values for an existing tag over the 2023 R2 gRPC front door. The pure-managed SDK
wrote a value and read it back live (gated test AddHistoricalValuesAsync_OverGrpc_WritesAndReadsBack
PASSED against the real server).
- HistorianGrpcHistoricalWriteOrchestrator: write-enabled (0x401) session ->
RetrievalService.GetTagInfosFromName (resolves the per-tag GUID = the tag-info TypeId, and
registers the tag on the session) -> HistoryService.AddStreamValues("ON" buffer) per sample.
- HistorianHistoricalValue (public record: TimestampUtc, Value, OpcQuality=192).
- gRPC-only: non-RemoteGrpc transports throw ProtocolEvidenceMissingException (the 2020 WCF
non-streamed write is architecturally blocked, D2).
- Float value encoding only (the captured type); other types rejected by the serializer.
275 unit tests pass; the new gated live write/read-back test is green against the 2023 R2 server.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Managed serializer for the historical-write "ON" buffer, byte-for-byte matching the live capture
(WcfHistoricalWriteProtocolTests golden-tests the exact 56-byte native buffer). Layout: "ON"(0x4E4F)
+ count + lengths + 16B tag GUID + sample FILETIME + u16 quality(192) + 4B descriptor + received
FILETIME + value. Value encoding (Float, captured): an 8-byte slot = u32(0) + float32(value) — the
4-byte float in the high dword, NOT a double. The 16B tag GUID is the per-tag GUID the SDK already
parses as ParseTagInfoRecord's "typeId" (confirmed: it appears at offset 8 of GetTagInfosFromName's
response = where typeId is read, and in EnsureTags' response + the "ON" buffer).
Only the Float encoding is captured; other types rejected until captured. Next: gRPC orchestrator
(write-enabled session -> EnsureTags -> resolve tag GUID -> AddStreamValues) + public
AddHistoricalValuesAsync + live write/read-back.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
revision-write-path.md §"R3.1 CAPTURED" + roadmap R3.1/R3.2/one-glance now record the validated
finding: the historical write is HistoryService.AddStreamValues ("ON" storage-sample buffer, AddS2
"OS" family) + EnsureTags, not AddNonStreamValues/TransactionService. Includes the decoded 56-byte
"ON" buffer layout, the working priming/batch sequence, the tag-GUID keying, and that the D2 cache
gate does not block the primed 2023 R2 client. Remaining work to ship AddHistoricalValuesAsync is
the managed "ON" serializer (adapt HistorianEventWriteProtocol) + gRPC orchestrator wiring.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Drove the native 2023 R2 client through a committed non-streamed (historical backfill) write to a
sandbox tag, with the IL-rewritten managed gRPC client dumping every byte[] payload. Read the value
back over gRPC = end-to-end validated.
Key discovery: the M3 historical write does NOT use AddNonStreamValues/TransactionService (the
roadmap's assumption from the static decompile). The native client routes it over
HistoryService.AddStreamValues with an "ON" storage-sample buffer (structurally the AddS2 "OS"
family), plus EnsureTags for registration:
AddStreamValues.values (56B): "ON"(0x4E4F) + u16 count=1 + u32 totalLen + u16 payloadLen +
16B tag GUID + FILETIME(sample) + u16 quality=192 + u32 type/desc + FILETIME(received) +
8B double value.
EnsureTags.tagInfos (144B): the analog CTagMetadata the SDK's EnsureTagAsync already builds
(0x4E marker ... fe 00 trailer).
Tooling that produced it:
- instrument-grpc-nonstream now instruments EVERY byte[]-input method on every Grpc*Client
(45 methods) so the real wire path surfaces regardless of assumptions.
- harness pre-loads the instrumented GrpcClient by identity (LoadFrom context reuses an
already-loaded assembly before sibling-probing, so the rewrite wins over the bin original);
capture-write sequence fixed to Begin -> Add -> SendValues -> End (End-before-Send = err 160
InvalidBatchId); GetTagInfoByName(cache:false) + resync wait resolves the server key; cache
gate (D2's 129) does NOT block the primed 2023 R2 client.
Buffers captured to gitignored artifacts/. Next: build the "ON" AddStreamValues serializer in
src/ (adapt the existing AddS2 "OS" serializer) + EnsureTags + ship AddHistoricalValuesAsync.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
The capture-write harness scenario drives the native 2023 R2 client through a non-streamed
(historical backfill) write so the IL-rewritten GrpcHistoryClient dumps RegisterTags.tagInfos +
AddNonStreamValues.inBuff to the capture NDJSON. Sequence: open write-enabled gRPC -> (optional
--create) AddTag sandbox -> GetTagInfoByName (real TagKey + primes the per-connection cache, the
gate mitigation) -> CreateHistorianDataValueList(NonStreamedOriginal) -> NonStreamedValuesBegin ->
AddNonStreamedValue -> AddNonStreamedValuesEnd -> SendValues (the wire push; only with --commit).
Not yet run — the actual write to the live server awaits explicit confirmation. Built clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Adds the dnlib instrument command + harness wiring to capture the two non-streamed-write
buffers from the native 2023 R2 client:
- `instrument-grpc-nonstream <GrpcClient.dll> [out]` injects CaptureLogger.LogByteArray at the
entry of GrpcHistoryClient.RegisterTags (byte[] tagInfos) and AddNonStreamValues (byte[] inBuff),
writing the rewrite to docs/reverse-engineering/dnlib-write-copy/grpc2023 (gitignored — derived
AVEVA binary). dnlib preserves the AVEVA public-key identity so aahClientManaged still binds the
rewritten copy under the LoadFrom context (no SN re-verification).
- harness `--grpc-rewrite <dir>` probes that dir first, so the instrumented GrpcClient.dll +
ReverseInstrumentation.dll load ahead of the originals. load-check confirms the rewritten
strong-named copy binds (HistorianConnectionMode.Historian=2; GrpcHistoryClient RegisterTags +
AddNonStreamValues present).
Next: capture-write scenario (open write-enabled -> sandbox tag -> read-prime -> AddNonStreamedValue),
which dumps tagInfos + inBuff to the capture NDJSON. Prod write — confirm before running.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Added the `connect` scenario to the 2023 R2 capture harness and ran it read-only against the
live server. The native mixed-mode client connects end-to-end over gRPC from this box:
OpenConnection -> True (ErrorCode=Success)
ConnectedToServer = True
ConnectedToServerStorage = True <-- native client HAS the storage-engine session
ConnectedToStoreForward = False
Connection args that work (HistorianConnectionArgs): ServerName, TcpPort=32565,
ConnectionMode=Historian (gRPC), ConnectionType=Process, ReadOnly=true, IntegratedSecurity=false,
UserName/Password (explicit), AllowUnTrustedConnection=true, SecurityInfo=CertificateInfo{
SecurityMode=TransportCertificate, CertificateName=WONDER-SQL-VD03 } (the https:// host over the
loopback tunnel). Creds from HISTORIAN_USER/HISTORIAN_PASSWORD.
Significance: ConnectedToServerStorage=True means the native client establishes the storage
session the pure-managed SDK couldn't — so a write driven through it should route
AddNonStreamValues with a live storage session, and the cache-gate mitigation (read-first)
is promising. Next: IL-rewrite Archestra.Historian.GrpcClient.dll + a write-enabled run to
capture the RegisterTags btTagInfos + AddNonStreamValues btInput (prod write; per-action auth).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
First increment of the native-2023R2-gRPC capture (docs/plans/revision-write-path.md
§"R3.1 capture plan"). Loads the mixed-mode aahClientManaged.dll by path (sibling resolver
over bin/ + msi-extract/.../Bin/x64) and reflects the connection API — no live contact.
load-check result on this box (net481 x64):
- aahClientManaged.dll loads cleanly (no missing VC++ runtime / no BadImageFormat) — confirms
the self-contained mixed-mode assembly runs without an AVEVA install.
- HistorianConnectionMode.Historian = 2 (the 2023 R2 gRPC mode; ClassicHistorian = 1 = legacy)
— the value the live-connect step sets on HistorianConnectionArgs.ConnectionMode.
- GrpcHistoryClient resolves with RegisterTags + AddNonStreamValues present — the IL-rewrite
capture targets are reachable.
Standalone net481 project (not in Histsdk.slnx, like NativeTraceHarness). Next: read-only
gRPC connect + tag read (first live step, per-action auth), then IL-rewrite + write capture.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Records the feasibility-verified plan to capture the two remaining buffers (regular-tag
RegisterTags btTagInfos + AddNonStreamValues btInput):
- 2023 R2 aahClientManaged.dll is self-contained mixed-mode C++/CLI (only Windows + VC++
runtime native imports) — loadable in a net481 x64 process, no AVEVA install needed.
- gRPC routes through the managed Archestra.Historian.GrpcClient.dll, so the byte[] payloads
are capturable by IL-rewriting GrpcHistoryClient.RegisterTags / AddNonStreamValues (dnlib,
the instrument-wcf-writemessage pattern; rewrite a copy, never the originals).
- Connection is reflection-drivable: HistorianAccess.OpenConnection(HistorianConnectionArgs)
with ConnectionMode=HistorianConnectionMode.Historian (the gRPC mode), TcpPort=32565, cert.
- gRPC runtime deps (Grpc.Net.Client / Grpc.Core.Api / Google.Protobuf / ...) are present in
msi-extract/ArchestrA/Toolkits/Bin/x64.
- Risk: the C++ AddNonStreamedValue TagNotFoundInCache(129) gate (the 2020 D2 blocker) may
block btInput; mitigation = read the tag first. RegisterTags is emitted before that gate.
Build order documented (read-only connect -> IL-rewrite -> write capture -> serializer ->
commit+read-back -> AddHistoricalValuesAsync), each live step gated on per-action auth.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Live-probed StorageService.OpenStorageConnection against the 2023 R2 server over a
write-enabled (0x401) session. Every attempt — sweeping ConnectionMode (0x401/0x402/0x1),
StorageSessionId-in (Open2-GUID / empty), and FreeDiskSpace — returns the IDENTICAL native
error type=4 code=85 ("session not registered"), so it's a structural refusal, not a bad
field value.
Decode (two corroborating facts):
- Error 85 is the same code the event read returns before RegisterTags2 (see
HistorianWcfEventOrchestrator) — a generic "session not registered for this op".
- The 2023 R2 decompile shows OpenStorageConnection lives on a SEPARATE GrpcStorageClient
(the storage engine's SF/snapshot channel, own port + service identity); HistorianAccess
drives non-streamed writes through the native C++ HistorianClient, never this op.
So the roadmap's mapped "missing console session" step was wrong. The real non-streamed-write
precondition is the front-door HistoryService.RegisterTags (RTag2-family) for the target tag —
which is exactly why the R3.1 batch failed at AddNonStreamValues (no tag registered ->
StoreNonStreamValues had no route). Matches the original 2020-WCF D2 hypothesis.
Remaining (both need a native gRPC capture; do not guess bytes): the regular-tag RegisterTags
btTagInfos (only CM_EVENT's tag-GUID form is known) and the AddNonStreamValues btInput.
- HistorianGrpcStorageConnectionProbe + grpc-open-storage-connection CLI (opens nothing
persistent; CloseStorageConnection on success)
- corrected revision-write-path.md §R3.1 follow-up + hcal-roadmap R3.1/R3.2 rows
- gated regression test pinning the error-85 refusal
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
The D2 storage-engine-pipe wall is WCF-transport-specific. On 2023 R2 gRPC,
TransactionService.AddNonStreamValuesBegin/End round-trips live (write-enabled
session, Open2 storage GUID as strHandle). Live decode + static mining mapped the
full sequence: the remaining insert needs StorageService.OpenStorageConnection
(+ RegisterTags) then a btInput decode — a focused follow-up. Revision EDITS
(R4.2) stay pipe-only even on gRPC.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Static decompile mining of the 2023 R2 client corroborates the live R3.1 error:
the AddNonStreamValues failure is the missing StorageService.OpenStorageConnection,
which creates exactly the \.\pipe\aahStorageEngine\console,sid(...) session named
in the server error. Mapped the full native sequence:
HistoryService.OpenConnection (have) -> StorageService.OpenStorageConnection
(MISSING) -> StorageService.RegisterTags -> AddNonStreamValuesBegin (works) ->
AddNonStreamValues(btInput) (fails - no console session) -> End(commit).
Two hard parts remain, each a live-production decode loop with no static shortcut:
(1) reproduce the 12-arg OpenStorageConnection handshake (several args inferred);
(2) decode the AddNonStreamValues btInput (C++-built, absent from decompiles; only
the 44-byte packed HISTORIAN_VALUE2 is known). Documented in revision-write-path.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Empirically decoded the AddNonStreamValues btInput framing against the live 2023
R2 server (grpc-nonstream-decode command + ProbeNonStreamedBuffersAsync driver).
Every transaction rolled back (bCommit=false) — no data written.
Finding: the btInput is assembled native-C++-side (not in any decompile), so 6
evidence-based framings (44-54B, packed HISTORIAN_VALUE2 variants) were probed.
All 6 returned the IDENTICAL server error while an empty buffer returned a
different InvalidParameter — so non-empty buffers pass parameter validation into
CHistStorageConnection::StoreNonStreamValues, which routes to the
\.\pipe\aahStorageEngine\console pipe server-side. Identical-across-framings =>
the blocker is NOT the btInput layout but a missing storage-engine console
session / tag-registration precondition for the connection.
Next step (untested): StorageService.OpenStorageConnection + tag registration
(RegisterTags/AddTagidPairs/AddShardTagids) before AddNonStreamValues, then
commit + read-back on a sandbox tag. Documented in revision-write-path.md
(R3.1 decode section); raw artifact gitignored.
272 unit tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
The D2 storage-engine-pipe wall is WCF-transport-specific. On the 2023 R2 gRPC
front door, TransactionService is a first-class service AND the gateway to the
storage engine, so the Open2 storage-session GUID (uppercase) is accepted
directly as strHandle with no legacy pipe.
Live-verified against the real 2023 R2 server over a write-enabled (0x401) gRPC
session: AddNonStreamValuesBegin returns a real strTransactionId, and
AddNonStreamValuesEnd(bCommit=false) discards it cleanly (no data written). On
2020 WCF the same op returns UnknownClient(51) for every handle + priming chain.
- HistorianGrpcRevisionProbe + grpc-revision-probe CLI command + gated test
NonStreamedWriteTransaction_OverGrpc_BeginsAndDiscards (live pass).
- HistorianGrpcHandshake.OpenSession gains an optional connectionMode param
(default read-only 0x402; pass 0x401 for write-enabled) — non-breaking.
- Docs: revision-write-path.md "the wall is gone" section; roadmap M3 section,
R3.1-R3.3 rows, one-glance table, and status note updated honestly.
Not yet shipped: the AddNonStreamValues btInput VTQ buffer is uncaptured (never
guess wire bytes), so no value-commit is implemented. Scope is non-streamed
ORIGINAL backfill; revision EDITS (R4.2) remain pipe-only even on gRPC.
272 unit tests pass; sanitization scan clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Live-probed both R1.3 and R1.4 against a real 2023 R2 server over the gRPC
StatusService; implemented the one that carries an evidence-backed value.
R1.3 GetServerTimeZoneAsync — SHIPPED:
- StatusService.GetSystemTimeZoneName(uiHandle) returns the real server zone
over RemoteGrpc (the 2020 WCF op is a client-side stub returning empty).
- HistorianGrpcStatusClient.GetSystemTimeZoneNameAsync -> dialect routing ->
public HistorianClient.GetServerTimeZoneAsync. Non-gRPC transports fail
closed with ProtocolEvidenceMissingException (no empty-string lie).
- Golden message-shape unit test + non-gRPC guardrail unit test + gated live
test. 271 unit tests pass.
R1.4 GetHistorianInfoAsync (EventStorageMode) — bounded out on gRPC too:
- gRPC GetHistorianInfo is the same named-value query as 2020 WCF (only
HistorianVersion resolves); EventStorageMode + 7 variants fail on both
GetHistorianInfo and GetSystemParameter. The 518-byte struct is filled by a
native vtable+648 HCAL call, not the gRPC op (per the 2023 R2 decompile), so
the field is never on the wire. Not shipped on any transport. Closes the
roadmap's open "build against a live 2023 R2 server" caveat.
Also correct the stale M3 roadmap section: D2 already proved
Transaction.AddNonStreamValues* rides the storage-engine pipe (STransactPipeClient2
-> aaStorageEngine), not WCF — same wall as R4.2 — so M3-over-WCF is blocked, not
"the path that is NOT the gated cache push".
Docs: hcal-roadmap.md, wcf-historian-info.md, wcf-status-localhost.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Recorded the native-disassembly attempt on aahClientManaged.dll (mixed-mode):
ilspycmd cannot decompile it; capstone byte-search can't locate the StartTagQuery
0x6751 marker (not a plain immediate — it's an .rdata constant loaded RIP-relative,
the .text "51 67 00 00" hits are coincidental jump-table data). Managed metadata
gives QueryTag field semantics but not the binary packet-id. Finishing QueryTag
needs Ghidra/IDA xref analysis or the live IL-rewrite capture.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Deepened the R0.1 browse finding. QueryTag's constant rejection decodes to
ArchestrA.CloudHistorian.Contract.ErrorCode.InvalidPacketId (72): the btRequest needs
a QueryTag-specific packet-id header (the generic 0x6751/v1 header StartTagQuery accepts
is rejected). The semantic fields are known from CloudHistorian.Contract
(QueryHandle/QueryType/StartIndex/TagCount request; TagNames[]+TagMetadataBuffer response),
but the binary packet framing lives in native aahClient.dll — aahClientManaged.dll is
mixed-mode (ilspycmd cannot decompile it) and no managed assembly builds the buffer.
Finishing QueryTag needs native RE (Ghidra/IDA) or a live gRPC capture of the stock client.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Probes the 2023 R2 gRPC browse path and records the finding. The front door does
NOT hit the 2020 WCF metadata-server-pipe wall.
- RetrievalService.StartTagQuery is cracked: the server (CMdServer::StartActiveTagnamesQuery
over \.\pipe\aahMetadataServer\console) parses the filter as OData. startswith()/
contains()/eq/empty succeed and return the 8-byte (queryHandle, tagCount); SQL-LIKE "%"
and glob "*" fail with "ODataFilter: bad token". Live: 220 Sys* tags counted.
- QueryTag (paging) remains: every guessed btRequest returns a constant native error
type 4 / code 72 (content-independent) -> framing needs a native capture, not guessing.
Adds RE probe helpers Grpc/HistorianGrpcTagClient.ProbeStartTagQuery + ProbeTagQuerySequence,
a gated StartTagQuery_OverGrpc_AcceptsODataFilter test, and the finding doc
docs/reverse-engineering/grpc-tag-query-odata.md. Browse is not yet wired (QueryTag open).
217 unit tests pass; 5/5 live gRPC tests pass. No tag names/identities committed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Routes HistorianClient.GetTagMetadataAsync over gRPC when Transport==RemoteGrpc,
via the new Grpc/HistorianGrpcTagClient calling RetrievalService.GetTagInfosFromName
(the plural string-handle metadata op).
- String handle = the Open2 storage-session GUID formatted uppercase (the format
that resolves the native string-handle path); threaded out of the shared handshake
via a new HistorianGrpcHandshake.Session { ClientHandle, StorageSessionId, StringHandle }.
- Request btTagNames = uint count + per-name(uint charCount + UTF-16LE) — golden-byte
unit-tested (BuildTagNamesBuffer).
- Response btTagInfos = uint count + CTagMetadata records — decoded by the existing
HistorianTagQueryProtocol.ParseGetTagInfoResponse; data type via the shared MapDataType.
The 2020 WCF string-handle wall does NOT apply on the gRPC front door, as the
string-handle-wall RE note predicted. LIVE-VERIFIED against a real 2023 R2 server:
GetTagMetadataAsync returns the requested tag with a valid decoded data type.
216 unit tests pass. Captured framing confirmed live then discarded; no tag names
or identities committed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
R0.3 system-param over gRPC is now LIVE-VERIFIED against the real 2023 R2 server
(returned HistorianVersion), alongside the re-confirmed read chain and probe.
The apparent NTLM round-1 SEC_E_LOGON_DENIED "blocker" was a test-harness
credential-parsing bug, not a server/account/SDK issue: the gitignored creds
file stores quoted values and the env-setup must strip surrounding quotes before
exporting HISTORIAN_USER/PASSWORD. With quotes stripped, the NAM domain account
authenticates and the full chain passes. The round-failure diagnostic added
during the hunt (HistorianNativeHandshake.DescribeError) is kept.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Roadmap docs/plans/hcal-roadmap.md, milestone M0 (gRPC parity for the DONE
surface). Now unblocked for live verification by a reachable 2023 R2 server.
- R0.4 Probe over gRPC: new HistorianGrpcProbe calls History/Retrieval/Status
GetInterfaceVersion (unauthenticated). ProbeAsync routes over gRPC when
Transport==RemoteGrpc. LIVE-VERIFIED against a real 2023 R2 server — needs no
credentials (runs before the auth loop), so it works despite the auth blocker.
- R0.3 System parameter over gRPC: new HistorianGrpcStatusClient calls
StatusService.GetSystemParameter over the authenticated session; routed in the
dialect. Built + unit-tested (request/response field mapping pinned).
Live-verification pending an auth fix (see below).
- Extracted the proven auth handshake from HistorianGrpcReadOrchestrator into
shared Grpc/HistorianGrpcHandshake (reused by read + status + future
browse/metadata). Repointed the IL structural guardrail test to it.
- Diagnostics: round-failure now decodes the native server error + hex/ASCII
preview (HistorianNativeHandshake.DescribeError). This surfaced the live auth
blocker as SEC_E_LOGON_DENIED (0x8009030C) at NTLM round 1 — framing is correct,
the credential did not validate. Probable cause: stale file password or NAM-domain
NTLM restriction (Kerberos/RDP works, NTLM denied; no SPN path over the tunnel).
216 unit tests pass; live gRPC probe passes. Sanitization scan clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Joseph Doherty