Commit Graph

98 Commits

Author SHA1 Message Date
Joseph Doherty e45c615a79 docs: record R4.3 measured idle-state status in hcal-roadmap
Update the M4 table row, one-glance status line, and M4 narrative note to
reflect R4.3: measured idle-state GetStoreForwardStatusAsync SHIPPED over
gRPC; active-SF magnitude + R4.2 revision edits stay deferred behind the
shared D2 storage-engine console-pipe wall.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 23:20:51 -04:00
Joseph Doherty 9db2864f70 Merge re/r4.3-sf-status-measured: R4.3 measured idle-state store-forward status
gRPC GetStoreForwardStatusAsync now contacts the server (measured idle-state,
ErrorOccurred on failure) instead of blind synthesis; active-SF magnitude
stays D2-gated. Includes the grpc-sf-status-probe RE tool + re-scope doc.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 23:17:35 -04:00
Joseph Doherty 53a9c87114 R4.3: measured idle-state GetStoreForwardStatusAsync over gRPC
Route GetStoreForwardStatusAsync to a gRPC path that actually contacts the
server (StatusService.GetHistorianConsoleStatus) instead of synthesizing an
all-false result blind. On a reachable/normal server it returns the
not-storing baseline but MEASURED; when the server is unreachable or the
console-status call fails it reports ErrorOccurred with the underlying error
(the old synthesis never contacted the server). The active-SF buffer
magnitude (Storing/Pending/DataStored) stays false because it lives behind
the D2 storage-engine console wall.

Non-gRPC transports keep the synthesized fallback. Live-verified against the
2023 R2 server; gated integration test
GetStoreForwardStatusAsync_OverGrpc_ReturnsMeasuredIdleState added. README
operation table updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 23:14:17 -04:00
Joseph Doherty c2d8fb9bc8 R4.3: gRPC store-forward status probe + re-scope
Add HistorianGrpcStoreForwardStatusProbe and the `grpc-sf-status-probe` CLI
command. The idle-baseline run against the live 2023 R2 server resolves the
plan's §9.3 handle question: the direct StorageService SF pull RPCs
(GetSFParameter / GetRemainingSnapshotsSize) require the OpenStorageConnection
console handle and are D2-gated (err 132, identical under read-only and
write-enabled sessions), while StatusService.GetHistorianConsoleStatus IS
reachable on the session string handle (=3 at idle).

Records the gRPC re-scope and the idle-baseline findings in
docs/plans/store-forward-cache-reverse-engineering.md §9. The probe writes
nothing and releases any console session immediately.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 23:14:05 -04:00
Joseph Doherty f840af5873 Merge re/m4-redundancy: R4.4 client-side multi-historian redundancy
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 22:46:58 -04:00
Joseph Doherty 60b3673f01 M4 R4.4: client-side multi-historian redundancy
Adds AVEVA.Historian.Client.Redundancy — HistorianRedundantClient orchestrates
N single-historian members (IHistorianMember; default HistorianClientMember
over HistorianClient) as one logical client. Pure client-side, no server-side
redundancy protocol, no RE.

- Reads fail over to the next member in priority order. Streaming reads only
  fail over BEFORE the first row is observed; a mid-stream failure propagates
  (failing over mid-stream would risk duplicated/skipped rows).
- Writes fan out: WriteFanout AllMembers | PreferredOnly, with All | Any ack
  policy, returning a per-member HistorianRedundantWriteResult.
- Per-member health: FailureThreshold demotes a failing member out of the
  preferred pool; a background watchdog (PeriodicTimer) + CheckHealthAsync
  re-probe and restore recovered members. GetStatus() snapshot + ActiveMember.
- Composes with R4.1: back a member's writes with a HistorianStoreForwardWriter
  so a down member buffers and replays on recovery — the pragmatic client-side
  equivalent of native ReSyncTags.

14 unit tests (no server): failover order, mid-stream no-failover, all-fail
aggregation, probe-any-up, fan-out ack policies, PreferredOnly, soft reject,
health demotion + CheckHealthAsync restore, watchdog recovery. Full suite 307
green. Roadmap R4.4 marked shipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 22:46:10 -04:00
Joseph Doherty a9000ec06a Merge re/m4-store-forward-outbox: R4.1 pragmatic store-and-forward outbox
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 22:38:35 -04:00
Joseph Doherty dd2aec3b8b M4 R4.1: pragmatic store-and-forward durable outbox
Adds AVEVA.Historian.Client.StoreForward — a client-side store-and-forward
layer over the historian write surface (AddHistoricalValuesAsync /
SendEventAsync). Producers enqueue writes; the writer persists them and
replays on reconnect so a transient disconnect never drops data. This is the
roadmap's recommended pragmatic outbox, NOT a bit-faithful reimplementation of
AVEVA's native SF cache (that stays deferred) — pure managed, no RE.

- HistorianOutboxEntry / HistorianOutboxEntryKind: buffered-write envelope
- IHistorianOutboxStore + InMemoryHistorianOutboxStore (tests) +
  FileHistorianOutboxStore (crash-durable: atomic temp+move JSON per entry,
  FIFO by filename sequence that resumes past on-disk max, corrupt-file
  quarantine). OutboxJson normalizes event object? properties off JsonElement.
- IHistorianWriteSink + HistorianClientWriteSink (HistorianClient-backed)
- HistorianStoreForwardWriter: enqueue, single-flight FIFO FlushAsync with
  head-of-line blocking, optional MaxDeliveryAttempts dead-lettering,
  DropOldest/Reject overflow policy, background drain loop (retry on reconnect),
  GetStatusAsync snapshot mirroring server SF Pending/Storing/ErrorOccurred.

12 unit tests (no server): durability-across-restart, reconnect-drain, FIFO
order/head-of-line, dead-letter, overflow policies, background auto-drain.
Full suite 293 green. Roadmap R4.1 marked shipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 22:35:30 -04:00
Joseph Doherty a91f126287 docs(hcal-roadmap): M3 R3.2 ships all 5 analog types, not Float-only
R3.2 and the one-glance table still read "Float-only"; the shipped
AddHistoricalValuesAsync covers Float/Double/Int2/Int4/UInt4 (golden-tested
+ live write/read-back). Correct both lines to match code + tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 22:20:25 -04:00
Joseph Doherty 7b5d27a8d3 Merge re/m3-value-types: AddHistoricalValuesAsync Double + Int support
Extends the historical-write surface from Float to all five analog types (Float/Double/Int2/Int4/
UInt4), each captured live + golden-tested + write/read-back validated through the pure-managed SDK.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:48:43 -04:00
Joseph Doherty d1e96f48de M3 R3.2: AddHistoricalValuesAsync supports Double + Int (Int2/Int4/UInt4)
Extended the historical-write serializer from Float-only to all five analog types EnsureTagAsync
supports. Captured each type's "ON" buffer live from the native client (sandbox tag per type,
written + captured + deleted):

- The 4-byte value descriptor (C0 10 01 00) is CONSTANT across types — it does not encode the type.
- The value is u32(0) + native-width value, width by the tag's declared type:
  Float->float32, Double->double64, Int2->int16, Int4->int32, UInt4->uint32.

HistorianHistoricalWriteProtocol.SerializeAddStreamValuesBuffer now takes the HistorianDataType and
encodes accordingly (unsupported types throw ProtocolEvidenceMissingException). The orchestrator
resolves the type from the tag-info NativeDataTypeDescriptor via MapDataType. Harness capture-write
gained --data-type. Golden-tested against all five live captures + the gated write/read-back test
validated each type end-to-end through the pure-managed SDK; 281 unit tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:48:29 -04:00
Joseph Doherty d527784def M3 capture harness: add delete-tag scenario (sandbox cleanup)
delete-tag drives the native client's DeleteTags (the clean-delete path, unlike the SDK's WCF
DelT which can leave the row). Primes the write session with AddTag first (DeleteTags on a fresh
connection returns UnknownClient(51) until the client is registered). Used to remove the capture
sandbox tag SdkM3CaptureSandbox from the live server (DeleteTags returned success).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:34:38 -04:00
Joseph Doherty 3cc02e3ed0 Merge re/m3-osc-correction: M3 historical writes SHIPPED (AddHistoricalValuesAsync over gRPC)
Reverse-engineered + shipped the SDK's first historical/backfill write capability:
- Corrected the M3 path (OpenStorageConnection dead-end -> the write rides
  HistoryService.AddStreamValues, NOT AddNonStreamValues/TransactionService).
- Built a net481 capture harness + IL-rewrite to capture the native 2023 R2 'ON' write buffer.
- HistorianHistoricalWriteProtocol ('ON' serializer, golden-tested) +
  HistorianGrpcHistoricalWriteOrchestrator + public AddHistoricalValuesAsync.
- Live-validated: pure-managed SDK wrote a value and read it back over gRPC.
275 unit tests pass; gated live write/read-back test green. Float-only, gRPC-only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:29:25 -04:00
Joseph Doherty 273103f882 M3 R3.2: instrument-grpc-nonstream also captures out/ref byte[] responses
Extends the IL-rewrite to log out (byref) byte[] params at method exit (ldarg + ldind.ref), not
just byte[] inputs. This captured GetTagInfosFromName's response, which located the per-tag GUID at
offset 8 = exactly where ParseTagInfoRecord reads "typeId" — proving the SDK already parses the GUID
AddStreamValues needs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:25:24 -04:00
Joseph Doherty dafafa0c98 M3 R3.2 SHIPPED: docs — AddHistoricalValuesAsync recorded in roadmap, plan, and CLAUDE.md surface
Marks M3 historical writes SHIPPED + live-validated across the roadmap (R3.2/R3.3/one-glance),
revision-write-path.md §"R3.1 CAPTURED", and the CLAUDE.md Required SDK Surface (the new write op,
gRPC-only, AddStreamValues "ON" path, Float-only, distinct from the still-blocked AddS2 streaming
path).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:24:37 -04:00
Joseph Doherty aa36e58d58 M3 R3.2 SHIPPED: AddHistoricalValuesAsync — historical backfill writes over gRPC (live-validated)
Public HistorianClient.AddHistoricalValuesAsync(tag, values) inserts non-streamed original
(backfill) values for an existing tag over the 2023 R2 gRPC front door. The pure-managed SDK
wrote a value and read it back live (gated test AddHistoricalValuesAsync_OverGrpc_WritesAndReadsBack
PASSED against the real server).

- HistorianGrpcHistoricalWriteOrchestrator: write-enabled (0x401) session ->
  RetrievalService.GetTagInfosFromName (resolves the per-tag GUID = the tag-info TypeId, and
  registers the tag on the session) -> HistoryService.AddStreamValues("ON" buffer) per sample.
- HistorianHistoricalValue (public record: TimestampUtc, Value, OpcQuality=192).
- gRPC-only: non-RemoteGrpc transports throw ProtocolEvidenceMissingException (the 2020 WCF
  non-streamed write is architecturally blocked, D2).
- Float value encoding only (the captured type); other types rejected by the serializer.

275 unit tests pass; the new gated live write/read-back test is green against the 2023 R2 server.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:23:08 -04:00
Joseph Doherty 85f0c2f0fa M3 R3.2: HistorianHistoricalWriteProtocol — the "ON" AddStreamValues serializer (golden-validated)
Managed serializer for the historical-write "ON" buffer, byte-for-byte matching the live capture
(WcfHistoricalWriteProtocolTests golden-tests the exact 56-byte native buffer). Layout: "ON"(0x4E4F)
+ count + lengths + 16B tag GUID + sample FILETIME + u16 quality(192) + 4B descriptor + received
FILETIME + value. Value encoding (Float, captured): an 8-byte slot = u32(0) + float32(value) — the
4-byte float in the high dword, NOT a double. The 16B tag GUID is the per-tag GUID the SDK already
parses as ParseTagInfoRecord's "typeId" (confirmed: it appears at offset 8 of GetTagInfosFromName's
response = where typeId is read, and in EnsureTags' response + the "ON" buffer).

Only the Float encoding is captured; other types rejected until captured. Next: gRPC orchestrator
(write-enabled session -> EnsureTags -> resolve tag GUID -> AddStreamValues) + public
AddHistoricalValuesAsync + live write/read-back.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:17:14 -04:00
Joseph Doherty 0e78d638d0 M3 R3.1: document the captured + validated AddStreamValues "ON" write path
revision-write-path.md §"R3.1 CAPTURED" + roadmap R3.1/R3.2/one-glance now record the validated
finding: the historical write is HistoryService.AddStreamValues ("ON" storage-sample buffer, AddS2
"OS" family) + EnsureTags, not AddNonStreamValues/TransactionService. Includes the decoded 56-byte
"ON" buffer layout, the working priming/batch sequence, the tag-GUID keying, and that the D2 cache
gate does not block the primed 2023 R2 client. Remaining work to ship AddHistoricalValuesAsync is
the managed "ON" serializer (adapt HistorianEventWriteProtocol) + gRPC orchestrator wiring.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:04:17 -04:00
Joseph Doherty 9bcfffb365 M3 R3.1 CAPTURED: native non-streamed write rides HistoryService.AddStreamValues ("ON" buffer)
Drove the native 2023 R2 client through a committed non-streamed (historical backfill) write to a
sandbox tag, with the IL-rewritten managed gRPC client dumping every byte[] payload. Read the value
back over gRPC = end-to-end validated.

Key discovery: the M3 historical write does NOT use AddNonStreamValues/TransactionService (the
roadmap's assumption from the static decompile). The native client routes it over
HistoryService.AddStreamValues with an "ON" storage-sample buffer (structurally the AddS2 "OS"
family), plus EnsureTags for registration:

  AddStreamValues.values (56B): "ON"(0x4E4F) + u16 count=1 + u32 totalLen + u16 payloadLen +
    16B tag GUID + FILETIME(sample) + u16 quality=192 + u32 type/desc + FILETIME(received) +
    8B double value.
  EnsureTags.tagInfos (144B): the analog CTagMetadata the SDK's EnsureTagAsync already builds
    (0x4E marker ... fe 00 trailer).

Tooling that produced it:
- instrument-grpc-nonstream now instruments EVERY byte[]-input method on every Grpc*Client
  (45 methods) so the real wire path surfaces regardless of assumptions.
- harness pre-loads the instrumented GrpcClient by identity (LoadFrom context reuses an
  already-loaded assembly before sibling-probing, so the rewrite wins over the bin original);
  capture-write sequence fixed to Begin -> Add -> SendValues -> End (End-before-Send = err 160
  InvalidBatchId); GetTagInfoByName(cache:false) + resync wait resolves the server key; cache
  gate (D2's 129) does NOT block the primed 2023 R2 client.

Buffers captured to gitignored artifacts/. Next: build the "ON" AddStreamValues serializer in
src/ (adapt the existing AddS2 "OS" serializer) + EnsureTags + ship AddHistoricalValuesAsync.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 21:03:08 -04:00
Joseph Doherty d5c04cd410 M3 R3.1 capture: add capture-write scenario (drives non-streamed write, no run yet)
The capture-write harness scenario drives the native 2023 R2 client through a non-streamed
(historical backfill) write so the IL-rewritten GrpcHistoryClient dumps RegisterTags.tagInfos +
AddNonStreamValues.inBuff to the capture NDJSON. Sequence: open write-enabled gRPC -> (optional
--create) AddTag sandbox -> GetTagInfoByName (real TagKey + primes the per-connection cache, the
gate mitigation) -> CreateHistorianDataValueList(NonStreamedOriginal) -> NonStreamedValuesBegin ->
AddNonStreamedValue -> AddNonStreamedValuesEnd -> SendValues (the wire push; only with --commit).

Not yet run — the actual write to the live server awaits explicit confirmation. Built clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 19:24:15 -04:00
Joseph Doherty c1f263ef83 M3 R3.1 capture: instrument-grpc-nonstream IL-rewrite + harness --grpc-rewrite loading
Adds the dnlib instrument command + harness wiring to capture the two non-streamed-write
buffers from the native 2023 R2 client:

- `instrument-grpc-nonstream <GrpcClient.dll> [out]` injects CaptureLogger.LogByteArray at the
  entry of GrpcHistoryClient.RegisterTags (byte[] tagInfos) and AddNonStreamValues (byte[] inBuff),
  writing the rewrite to docs/reverse-engineering/dnlib-write-copy/grpc2023 (gitignored — derived
  AVEVA binary). dnlib preserves the AVEVA public-key identity so aahClientManaged still binds the
  rewritten copy under the LoadFrom context (no SN re-verification).
- harness `--grpc-rewrite <dir>` probes that dir first, so the instrumented GrpcClient.dll +
  ReverseInstrumentation.dll load ahead of the originals. load-check confirms the rewritten
  strong-named copy binds (HistorianConnectionMode.Historian=2; GrpcHistoryClient RegisterTags +
  AddNonStreamValues present).

Next: capture-write scenario (open write-enabled -> sandbox tag -> read-prime -> AddNonStreamedValue),
which dumps tagInfos + inBuff to the capture NDJSON. Prod write — confirm before running.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 19:20:54 -04:00
Joseph Doherty ce8576bd6e M3 R3.1 capture: read-only gRPC connect scenario — LIVE-VERIFIED
Added the `connect` scenario to the 2023 R2 capture harness and ran it read-only against the
live server. The native mixed-mode client connects end-to-end over gRPC from this box:

  OpenConnection -> True (ErrorCode=Success)
  ConnectedToServer        = True
  ConnectedToServerStorage = True   <-- native client HAS the storage-engine session
  ConnectedToStoreForward  = False

Connection args that work (HistorianConnectionArgs): ServerName, TcpPort=32565,
ConnectionMode=Historian (gRPC), ConnectionType=Process, ReadOnly=true, IntegratedSecurity=false,
UserName/Password (explicit), AllowUnTrustedConnection=true, SecurityInfo=CertificateInfo{
SecurityMode=TransportCertificate, CertificateName=WONDER-SQL-VD03 } (the https:// host over the
loopback tunnel). Creds from HISTORIAN_USER/HISTORIAN_PASSWORD.

Significance: ConnectedToServerStorage=True means the native client establishes the storage
session the pure-managed SDK couldn't — so a write driven through it should route
AddNonStreamValues with a live storage session, and the cache-gate mitigation (read-first)
is promising. Next: IL-rewrite Archestra.Historian.GrpcClient.dll + a write-enabled run to
capture the RegisterTags btTagInfos + AddNonStreamValues btInput (prod write; per-action auth).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 19:12:42 -04:00
Joseph Doherty 328748c0ae M3 R3.1 capture: scaffold net481 x64 harness + load-check (PASS)
First increment of the native-2023R2-gRPC capture (docs/plans/revision-write-path.md
§"R3.1 capture plan"). Loads the mixed-mode aahClientManaged.dll by path (sibling resolver
over bin/ + msi-extract/.../Bin/x64) and reflects the connection API — no live contact.

load-check result on this box (net481 x64):
- aahClientManaged.dll loads cleanly (no missing VC++ runtime / no BadImageFormat) — confirms
  the self-contained mixed-mode assembly runs without an AVEVA install.
- HistorianConnectionMode.Historian = 2 (the 2023 R2 gRPC mode; ClassicHistorian = 1 = legacy)
  — the value the live-connect step sets on HistorianConnectionArgs.ConnectionMode.
- GrpcHistoryClient resolves with RegisterTags + AddNonStreamValues present — the IL-rewrite
  capture targets are reachable.

Standalone net481 project (not in Histsdk.slnx, like NativeTraceHarness). Next: read-only
gRPC connect + tag read (first live step, per-action auth), then IL-rewrite + write capture.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 19:01:12 -04:00
Joseph Doherty 222eed9c02 M3 R3.1: durable capture plan — drive native 2023 R2 gRPC client + IL-rewrite byte[] payloads
Records the feasibility-verified plan to capture the two remaining buffers (regular-tag
RegisterTags btTagInfos + AddNonStreamValues btInput):

- 2023 R2 aahClientManaged.dll is self-contained mixed-mode C++/CLI (only Windows + VC++
  runtime native imports) — loadable in a net481 x64 process, no AVEVA install needed.
- gRPC routes through the managed Archestra.Historian.GrpcClient.dll, so the byte[] payloads
  are capturable by IL-rewriting GrpcHistoryClient.RegisterTags / AddNonStreamValues (dnlib,
  the instrument-wcf-writemessage pattern; rewrite a copy, never the originals).
- Connection is reflection-drivable: HistorianAccess.OpenConnection(HistorianConnectionArgs)
  with ConnectionMode=HistorianConnectionMode.Historian (the gRPC mode), TcpPort=32565, cert.
- gRPC runtime deps (Grpc.Net.Client / Grpc.Core.Api / Google.Protobuf / ...) are present in
  msi-extract/ArchestrA/Toolkits/Bin/x64.
- Risk: the C++ AddNonStreamedValue TagNotFoundInCache(129) gate (the 2020 D2 blocker) may
  block btInput; mitigation = read the tag first. RegisterTags is emitted before that gate.

Build order documented (read-only connect -> IL-rewrite -> write capture -> serializer ->
commit+read-back -> AddHistoricalValuesAsync), each live step gated on per-action auth.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 18:58:38 -04:00
Joseph Doherty 57b9506d01 M3 R3.1: OpenStorageConnection is a dead end (error 85); precondition is front-door RegisterTags
Live-probed StorageService.OpenStorageConnection against the 2023 R2 server over a
write-enabled (0x401) session. Every attempt — sweeping ConnectionMode (0x401/0x402/0x1),
StorageSessionId-in (Open2-GUID / empty), and FreeDiskSpace — returns the IDENTICAL native
error type=4 code=85 ("session not registered"), so it's a structural refusal, not a bad
field value.

Decode (two corroborating facts):
- Error 85 is the same code the event read returns before RegisterTags2 (see
  HistorianWcfEventOrchestrator) — a generic "session not registered for this op".
- The 2023 R2 decompile shows OpenStorageConnection lives on a SEPARATE GrpcStorageClient
  (the storage engine's SF/snapshot channel, own port + service identity); HistorianAccess
  drives non-streamed writes through the native C++ HistorianClient, never this op.

So the roadmap's mapped "missing console session" step was wrong. The real non-streamed-write
precondition is the front-door HistoryService.RegisterTags (RTag2-family) for the target tag —
which is exactly why the R3.1 batch failed at AddNonStreamValues (no tag registered ->
StoreNonStreamValues had no route). Matches the original 2020-WCF D2 hypothesis.

Remaining (both need a native gRPC capture; do not guess bytes): the regular-tag RegisterTags
btTagInfos (only CM_EVENT's tag-GUID form is known) and the AddNonStreamValues btInput.

- HistorianGrpcStorageConnectionProbe + grpc-open-storage-connection CLI (opens nothing
  persistent; CloseStorageConnection on success)
- corrected revision-write-path.md §R3.1 follow-up + hcal-roadmap R3.1/R3.2 rows
- gated regression test pinning the error-85 refusal

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 18:51:16 -04:00
Joseph Doherty 78cb689bdf Merge re/m3-grpc-nonstreamed-write: M3 non-streamed write reachable over gRPC (Begin/End live; sequence mapped)
The D2 storage-engine-pipe wall is WCF-transport-specific. On 2023 R2 gRPC,
TransactionService.AddNonStreamValuesBegin/End round-trips live (write-enabled
session, Open2 storage GUID as strHandle). Live decode + static mining mapped the
full sequence: the remaining insert needs StorageService.OpenStorageConnection
(+ RegisterTags) then a btInput decode — a focused follow-up. Revision EDITS
(R4.2) stay pipe-only even on gRPC.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 18:21:01 -04:00
Joseph Doherty 1a08dd9ec2 M3: roadmap reflects mapped non-streamed sequence (OpenStorageConnection follow-up)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 18:20:35 -04:00
Joseph Doherty ac28679a1f M3 R3.1: map the required non-streamed write sequence (OpenStorageConnection is the missing step)
Static decompile mining of the 2023 R2 client corroborates the live R3.1 error:
the AddNonStreamValues failure is the missing StorageService.OpenStorageConnection,
which creates exactly the \.\pipe\aahStorageEngine\console,sid(...) session named
in the server error. Mapped the full native sequence:

  HistoryService.OpenConnection (have) -> StorageService.OpenStorageConnection
  (MISSING) -> StorageService.RegisterTags -> AddNonStreamValuesBegin (works) ->
  AddNonStreamValues(btInput) (fails - no console session) -> End(commit).

Two hard parts remain, each a live-production decode loop with no static shortcut:
(1) reproduce the 12-arg OpenStorageConnection handshake (several args inferred);
(2) decode the AddNonStreamValues btInput (C++-built, absent from decompiles; only
the 44-byte packed HISTORIAN_VALUE2 is known). Documented in revision-write-path.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 18:12:40 -04:00
Joseph Doherty 8fbb868813 M3 R3.1 decode: AddNonStreamValues reaches server StoreNonStreamValues (storage-engine console pipe)
Empirically decoded the AddNonStreamValues btInput framing against the live 2023
R2 server (grpc-nonstream-decode command + ProbeNonStreamedBuffersAsync driver).
Every transaction rolled back (bCommit=false) — no data written.

Finding: the btInput is assembled native-C++-side (not in any decompile), so 6
evidence-based framings (44-54B, packed HISTORIAN_VALUE2 variants) were probed.
All 6 returned the IDENTICAL server error while an empty buffer returned a
different InvalidParameter — so non-empty buffers pass parameter validation into
CHistStorageConnection::StoreNonStreamValues, which routes to the
\.\pipe\aahStorageEngine\console pipe server-side. Identical-across-framings =>
the blocker is NOT the btInput layout but a missing storage-engine console
session / tag-registration precondition for the connection.

Next step (untested): StorageService.OpenStorageConnection + tag registration
(RegisterTags/AddTagidPairs/AddShardTagids) before AddNonStreamValues, then
commit + read-back on a sandbox tag. Documented in revision-write-path.md
(R3.1 decode section); raw artifact gitignored.

272 unit tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 18:08:27 -04:00
Joseph Doherty 23798db1ef M3 probe: non-streamed write transaction reachable over 2023 R2 gRPC (Begin/End live-verified)
The D2 storage-engine-pipe wall is WCF-transport-specific. On the 2023 R2 gRPC
front door, TransactionService is a first-class service AND the gateway to the
storage engine, so the Open2 storage-session GUID (uppercase) is accepted
directly as strHandle with no legacy pipe.

Live-verified against the real 2023 R2 server over a write-enabled (0x401) gRPC
session: AddNonStreamValuesBegin returns a real strTransactionId, and
AddNonStreamValuesEnd(bCommit=false) discards it cleanly (no data written). On
2020 WCF the same op returns UnknownClient(51) for every handle + priming chain.

- HistorianGrpcRevisionProbe + grpc-revision-probe CLI command + gated test
  NonStreamedWriteTransaction_OverGrpc_BeginsAndDiscards (live pass).
- HistorianGrpcHandshake.OpenSession gains an optional connectionMode param
  (default read-only 0x402; pass 0x401 for write-enabled) — non-breaking.
- Docs: revision-write-path.md "the wall is gone" section; roadmap M3 section,
  R3.1-R3.3 rows, one-glance table, and status note updated honestly.

Not yet shipped: the AddNonStreamValues btInput VTQ buffer is uncaptured (never
guess wire bytes), so no value-commit is implemented. Scope is non-streamed
ORIGINAL backfill; revision EDITS (R4.2) remain pipe-only even on gRPC.

272 unit tests pass; sanitization scan clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 17:51:17 -04:00
Joseph Doherty 04ea0b9a1f R1.3 GetServerTimeZoneAsync over gRPC (live-verified); R1.4 bounded out on gRPC
Live-probed both R1.3 and R1.4 against a real 2023 R2 server over the gRPC
StatusService; implemented the one that carries an evidence-backed value.

R1.3 GetServerTimeZoneAsync — SHIPPED:
- StatusService.GetSystemTimeZoneName(uiHandle) returns the real server zone
  over RemoteGrpc (the 2020 WCF op is a client-side stub returning empty).
- HistorianGrpcStatusClient.GetSystemTimeZoneNameAsync -> dialect routing ->
  public HistorianClient.GetServerTimeZoneAsync. Non-gRPC transports fail
  closed with ProtocolEvidenceMissingException (no empty-string lie).
- Golden message-shape unit test + non-gRPC guardrail unit test + gated live
  test. 271 unit tests pass.

R1.4 GetHistorianInfoAsync (EventStorageMode) — bounded out on gRPC too:
- gRPC GetHistorianInfo is the same named-value query as 2020 WCF (only
  HistorianVersion resolves); EventStorageMode + 7 variants fail on both
  GetHistorianInfo and GetSystemParameter. The 518-byte struct is filled by a
  native vtable+648 HCAL call, not the gRPC op (per the 2023 R2 decompile), so
  the field is never on the wire. Not shipped on any transport. Closes the
  roadmap's open "build against a live 2023 R2 server" caveat.

Also correct the stale M3 roadmap section: D2 already proved
Transaction.AddNonStreamValues* rides the storage-engine pipe (STransactPipeClient2
-> aaStorageEngine), not WCF — same wall as R4.2 — so M3-over-WCF is blocked, not
"the path that is NOT the gated cache push".

Docs: hcal-roadmap.md, wcf-historian-info.md, wcf-status-localhost.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 17:24:10 -04:00
Joseph Doherty 25aff409dc Merge re/grpc-2023r2-handshake: M0 gRPC parity (probe/system-param/metadata/browse) + handshake fix 2026-06-21 16:32:02 -04:00
Joseph Doherty d23722ea73 Merge re/r1.10-rename-tags: RenameTagsAsync via History StartJob
# Conflicts:
#	docs/plans/hcal-capability-matrix.md
#	docs/plans/hcal-roadmap.md
#	src/AVEVA.Historian.Client/Wcf/HistorianWcfTagWriteOrchestrator.cs
#	tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs
#	tools/AVEVA.Historian.NativeTraceHarness/Program.cs
2026-06-21 16:31:44 -04:00
Joseph Doherty 4de222c950 Merge re/r1.4-gethi-finding: R1.1 ExecuteSqlCommand + R1.4 GetHistorianInfo (bounded)
# Conflicts:
#	docs/plans/hcal-roadmap.md
#	src/AVEVA.Historian.Client/HistorianClient.cs
#	src/AVEVA.Historian.Client/Protocol/Historian2020ProtocolDialect.cs
#	tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs
#	tools/AVEVA.Historian.NativeTraceHarness/Program.cs
2026-06-21 16:18:49 -04:00
Joseph Doherty 85ff1b48df R0.1 browse over gRPC SHIPPED — QueryTag cracked, M0 gRPC parity complete
Wires HistorianClient.BrowseTagNamesAsync over gRPC (Transport==RemoteGrpc) via
Grpc/HistorianGrpcTagClient.BrowseTagNamesAsync: StartTagQuery(OData) -> paged
QueryTag -> EndTagQuery. Live-verified against a real 2023 R2 server (returns Sys* tags).

QueryTag packet-id recovered WITHOUT native disassembly: a .rdata packet-descriptor
table in aahClientManaged.dll lists {0x6751,1}=StartTagQuery immediately followed by
{0x6752,1}=QueryTag (found via pefile byte-scan of .rdata), confirmed live.

Wire format (live-verified):
- request btRequest = u16 0x6752 + u16 version(1) + u16 queryType(1=names) + u32 startIndex + u32 count
- response btResonse = u32 count + per-name(u32 charCount + UTF-16LE) + trailer (NextIndex/metadata, ignored)
- new HistorianTagQueryProtocol.ParseTagNameQueryPage tolerates the trailer
- GlobToODataFilter translates the SDK glob filter to OData (Pre*->startswith, *suf->endswith,
  *mid*->contains, exact->eq); the 2023 R2 metadata-server parses filters as OData.

Replaces the earlier RE probe helpers with the shipped browse path. Adds golden-byte
(BuildQueryTagRequest) + 8 glob-translation unit tests + gated live browse test.
226 unit tests pass; 5/5 live gRPC tests pass (read, probe, system-param, metadata, browse).

Milestone 0 (full gRPC parity) is complete.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 16:01:15 -04:00
Joseph Doherty 630295bd18 docs: QueryTag native-RE attempt — lightweight tooling insufficient, needs Ghidra
Recorded the native-disassembly attempt on aahClientManaged.dll (mixed-mode):
ilspycmd cannot decompile it; capstone byte-search can't locate the StartTagQuery
0x6751 marker (not a plain immediate — it's an .rdata constant loaded RIP-relative,
the .text "51 67 00 00" hits are coincidental jump-table data). Managed metadata
gives QueryTag field semantics but not the binary packet-id. Finishing QueryTag
needs Ghidra/IDA xref analysis or the live IL-rewrite capture.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 15:46:54 -04:00
Joseph Doherty 4c9f0d476c docs: QueryTag error = InvalidPacketId (72); needs native aahClient.dll RE
Deepened the R0.1 browse finding. QueryTag's constant rejection decodes to
ArchestrA.CloudHistorian.Contract.ErrorCode.InvalidPacketId (72): the btRequest needs
a QueryTag-specific packet-id header (the generic 0x6751/v1 header StartTagQuery accepts
is rejected). The semantic fields are known from CloudHistorian.Contract
(QueryHandle/QueryType/StartIndex/TagCount request; TagNames[]+TagMetadataBuffer response),
but the binary packet framing lives in native aahClient.dll — aahClientManaged.dll is
mixed-mode (ilspycmd cannot decompile it) and no managed assembly builds the buffer.
Finishing QueryTag needs native RE (Ghidra/IDA) or a live gRPC capture of the stock client.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 15:16:19 -04:00
Joseph Doherty 26ef5e5645 R0.1 browse probe: StartTagQuery over gRPC takes an OData filter (live)
Probes the 2023 R2 gRPC browse path and records the finding. The front door does
NOT hit the 2020 WCF metadata-server-pipe wall.

- RetrievalService.StartTagQuery is cracked: the server (CMdServer::StartActiveTagnamesQuery
  over \.\pipe\aahMetadataServer\console) parses the filter as OData. startswith()/
  contains()/eq/empty succeed and return the 8-byte (queryHandle, tagCount); SQL-LIKE "%"
  and glob "*" fail with "ODataFilter: bad token". Live: 220 Sys* tags counted.
- QueryTag (paging) remains: every guessed btRequest returns a constant native error
  type 4 / code 72 (content-independent) -> framing needs a native capture, not guessing.

Adds RE probe helpers Grpc/HistorianGrpcTagClient.ProbeStartTagQuery + ProbeTagQuerySequence,
a gated StartTagQuery_OverGrpc_AcceptsODataFilter test, and the finding doc
docs/reverse-engineering/grpc-tag-query-odata.md. Browse is not yet wired (QueryTag open).

217 unit tests pass; 5/5 live gRPC tests pass. No tag names/identities committed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 14:58:12 -04:00
Joseph Doherty 0e19adae68 gRPC M0 R0.2: tag metadata over gRPC (GetTagInfosFromName, live-verified)
Routes HistorianClient.GetTagMetadataAsync over gRPC when Transport==RemoteGrpc,
via the new Grpc/HistorianGrpcTagClient calling RetrievalService.GetTagInfosFromName
(the plural string-handle metadata op).

- String handle = the Open2 storage-session GUID formatted uppercase (the format
  that resolves the native string-handle path); threaded out of the shared handshake
  via a new HistorianGrpcHandshake.Session { ClientHandle, StorageSessionId, StringHandle }.
- Request btTagNames = uint count + per-name(uint charCount + UTF-16LE) — golden-byte
  unit-tested (BuildTagNamesBuffer).
- Response btTagInfos = uint count + CTagMetadata records — decoded by the existing
  HistorianTagQueryProtocol.ParseGetTagInfoResponse; data type via the shared MapDataType.

The 2020 WCF string-handle wall does NOT apply on the gRPC front door, as the
string-handle-wall RE note predicted. LIVE-VERIFIED against a real 2023 R2 server:
GetTagMetadataAsync returns the requested tag with a valid decoded data type.

216 unit tests pass. Captured framing confirmed live then discarded; no tag names
or identities committed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 14:35:52 -04:00
Joseph Doherty b0703ebf80 docs: R0.3 live-verified; correct the auth-blocker note (harness quote bug)
R0.3 system-param over gRPC is now LIVE-VERIFIED against the real 2023 R2 server
(returned HistorianVersion), alongside the re-confirmed read chain and probe.

The apparent NTLM round-1 SEC_E_LOGON_DENIED "blocker" was a test-harness
credential-parsing bug, not a server/account/SDK issue: the gitignored creds
file stores quoted values and the env-setup must strip surrounding quotes before
exporting HISTORIAN_USER/PASSWORD. With quotes stripped, the NAM domain account
authenticates and the full chain passes. The round-failure diagnostic added
during the hunt (HistorianNativeHandshake.DescribeError) is kept.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 14:29:12 -04:00
Joseph Doherty c4b8d0dde4 gRPC M0: probe (R0.4, live-verified) + system-param (R0.3) + shared handshake
Roadmap docs/plans/hcal-roadmap.md, milestone M0 (gRPC parity for the DONE
surface). Now unblocked for live verification by a reachable 2023 R2 server.

- R0.4 Probe over gRPC: new HistorianGrpcProbe calls History/Retrieval/Status
  GetInterfaceVersion (unauthenticated). ProbeAsync routes over gRPC when
  Transport==RemoteGrpc. LIVE-VERIFIED against a real 2023 R2 server — needs no
  credentials (runs before the auth loop), so it works despite the auth blocker.

- R0.3 System parameter over gRPC: new HistorianGrpcStatusClient calls
  StatusService.GetSystemParameter over the authenticated session; routed in the
  dialect. Built + unit-tested (request/response field mapping pinned).
  Live-verification pending an auth fix (see below).

- Extracted the proven auth handshake from HistorianGrpcReadOrchestrator into
  shared Grpc/HistorianGrpcHandshake (reused by read + status + future
  browse/metadata). Repointed the IL structural guardrail test to it.

- Diagnostics: round-failure now decodes the native server error + hex/ASCII
  preview (HistorianNativeHandshake.DescribeError). This surfaced the live auth
  blocker as SEC_E_LOGON_DENIED (0x8009030C) at NTLM round 1 — framing is correct,
  the credential did not validate. Probable cause: stale file password or NAM-domain
  NTLM restriction (Kerberos/RDP works, NTLM denied; no SPN path over the tunnel).

216 unit tests pass; live gRPC probe passes. Sanitization scan clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 13:32:04 -04:00
Joseph Doherty 22e9c5e5f8 gRPC 2023 R2: fix auth handshake op routing + accept History v12
First live-verified gRPC read against a real 2023 R2 Historian. The handshake
previously failed at round 0 (cred-independent) because the SSPI/Negotiate token
loop was routed to HistoryService.ExchangeKey. ExchangeKey is a separate
key-exchange/cert-path op, not the Negotiate loop — the token loop belongs on
StorageService.ValidateClientCredential, which kept the 2020 inBuff/outBuff token
framing the SDK's WrapValidateClientCredentialToken/TryRead helpers already build.

Captured + diffed against the recovered 2023 R2 protobuf contract and the
decompiled stock client; routing the loop to ValidateClientCredential completes
the full chain (ValidateClientCredential x N -> OpenConnection -> StartQuery ->
GetNextQueryResultBuffer) and returns rows.

- HistorianGrpcReadOrchestrator: token loop now calls
  StorageService.ValidateClientCredential(Handle, InBuff); corrected the op-map
  doc comment (was asserting the wrong ExchangeKey mapping).
- HistorianServerVersionGate: accept History interface version 12 alongside 11.
  Live server reports History=12, Retrieval=4, Storage=4; the buffers are
  byte-identical (a live read returns rows), so 12 is buffer-compatible. Retrieval
  stays pinned at 4 (matches). New AcceptedVersions() supports multi-version gates.
- New HistorianGrpcHandshakeRoutingTests: IL-level structural guardrail that
  disassembles the orchestrator (incl. lambda closures) and asserts the handshake
  invokes ValidateClientCredential and never ExchangeKey — fails if the regression
  returns.
- Updated gate tests + CLAUDE.md gRPC op-map.

240 unit tests pass on the full stack; 210 on this branch's base. The byte
payloads remain the proven 2020 protocol.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 12:34:04 -04:00
Joseph Doherty c1b1b3d23b R1.11 DelTep capture + R1.3/R1.4/R1.12/R1.13 bounded out
DelTep (extended-property delete) — wire format captured + serializer
golden-proven, but live delete is server-blocked and NOT exposed publicly:
- Captured the DelTep inBuff via a cross-session trick (harness add-tep gains
  --tep-skip-add + read-for-sync before --tep-delete; Capture-DeleteTagExtended
  Properties.ps1 / decode-del-tep-capture.py). Layout = same group framing as
  AddTEx but property-name-only (no 0x43 value) + 0x00 group trailer.
- SerializeDeleteRequest + 4 golden tests pin the server-accepted buffer.
- A decisive experiment shows SDK-added properties ARE deletable (the native
  client read-syncs and deletes one), so SDK-add is complete; the SDK's own
  DelTep is rejected by CHistStorage::DeleteTagExtendedProperties even with
  byte-identical inBuff, matching mode/handle, GetTgByNm+GetTepByNm prime, open
  channel, and 60s retries. Root cause: the native multiplexes services over one
  connection (per-connection working set); the SDK's per-service WCF channels
  don't reproduce it. Kept as documented-but-blocked internal orchestrator path;
  no public HistorianClient delete API.

Bounded out with evidence (no code; docs + roadmap + probe):
- R1.12 localized-property write — no op on 2020 (mirror of R1.6); no
  *LocalizedPropert*/TagLocalized* symbol in any current/*.dll.
- R1.13 non-analog tag create — GATED; native AddTag rejects every non-analog
  type client-side (ValidationFailed, before any WCF op): SingleByteString,
  DoubleByteString, Int1 all fail, Float works. No Discrete type in the native
  enum, no TagType setter. No wire request to capture.
- R1.3 timezone + R1.4 EventStorageMode — re-confirmed 2023R2/gRPC-only from
  the Runtime DB schema (no timezone param, no EventStorageMode anywhere) and a
  parameter-op probe (GetSystemParameter + GETRP return null/throw for every
  candidate; only HistorianVersion works).

238 unit tests pass; full solution builds with 0 warnings.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 11:26:21 -04:00
Joseph Doherty 08b950caee R1.11 AddTagExtendedPropertiesAsync: extended-property write via AddTEx
Adds user-defined extended properties to an existing tag via the 2020 WCF
AddTEx (AddTagExtendedProperties) op. Write-enabled connection + uppercase
storage-session GUID handle; reuses the write orchestrator open/priming chain.

The AddTEx inBuff is the exact inverse of the R1.5 GetTepByNm read-response
framing, so the serializer mirrors the read parser:
  uint32 groupCount + 0x01(group) + [0x09+u16+ASCII tag] + uint32 propCount
  + per prop{ 0x02 + [0x09+u16+ASCII name] + 0x43 VT_BSTR + u16 payloadLen
  + u16 charCount + UTF-16 value } + 0x01(group trailer) + 0x00(terminator).
The trailing 0x00 is required — without it inBuff is one byte short and the
server throws SErrorException in CHistStorage::AddTagExtendedProperties. The
golden fixture pins the clean inBuff the live server accepted (dumped via
AVEVA_HISTORIAN_TEP_DUMP); read-back verified via R1.5. String (0x43) values only.

Delete (DelTep) is deferred: the native DeleteTagExtendedPropertiesByName does a
client-side sync check and returns err 229 for a just-added property, so the
DelTep request never reaches the wire and its inBuff can't be captured yet.

Shipped: HistorianClient.AddTagExtendedPropertiesAsync/AddTagExtendedPropertyAsync;
HistorianTagExtendedPropertyProtocol.SerializeAddRequest; orchestrator path;
golden WcfTagExtendedPropertyWriteProtocolTests (4); gated live write/read-back test;
native-harness `add-tep` scenario + Capture-AddTagExtendedProperties.ps1 +
decode-add-tep-capture.py. Doc: wcf-add-tag-extended-properties.md. 233 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 01:43:19 -04:00
Joseph Doherty bc353df8c4 R1.10 RenameTagsAsync: async tag rename via History StartJob (StJb)
Tag rename has no dedicated WCF op — the (old,new) name batch rides the
generic History StartJob (StJb) job buffer; the server returns a job id and
applies renames asynchronously. Handle is the uppercase storage-session GUID,
Open2 in write mode; reuses the write orchestrator's open+priming chain.

jobBuffer layout (decoded + server-validated): byte[7] zero prefix + uint32
pairCount + per pair (uint32 oldCharCount + UTF-16 oldName + uint32
newCharCount + UTF-16 newName), order (old,new). The raw instrument capture
mangles the final byte with MDAS chunk markers (the R1.1 lesson), so the golden
fixture pins the CLEAN byte[] the SDK handed the channel (dumped via
AVEVA_HISTORIAN_RENAME_DUMP) — the exact buffer the live server accepted and
renamed with.

Gated server-side by the AllowRenameTags system parameter (default 0): when
disabled the native client rejects pre-wire (err 132); the managed SDK surfaces
it as StartJob=false -> Accepted=false. Enabling needs a Historian config
reload, not just a storage-engine restart.

Shipped: HistorianClient.RenameTagAsync/RenameTagsAsync -> HistorianTagRenameResult;
HistorianTagRenameProtocol; orchestrator RenameTags/SendStartJobRename; golden
WcfTagRenameProtocolTests (4, pins server-accepted buffer); gated live test
RenameTagsAsync_AgainstLocalHistorian_RenamesSandboxTag (passed end-to-end).
Native-harness `rename` scenario + Capture-RenameTags.ps1 + decode-rename-capture.py.
Doc: docs/reverse-engineering/wcf-rename-tags.md. 213 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-21 01:18:41 -04:00
Joseph Doherty fbd839077b R1.4 GetHistorianInfo: bounded out on 2020 WCF (named-value-only, no struct)
Captured the native HistorianAccess.GetHistorianInfo(out HistorianInfo, out err)
and decoded the wire: over 2020 WCF, GETHI is a named-value query whose only
working key is "HistorianVersion" (response ~30 bytes = the version string).
Probed 7 storage-mode key names -> all ok=False/err. The 518-byte HISTORIAN_INFO
struct + EventStorageMode@514 is the 2023R2 HCAL-native/gRPC model (confirmed
from the decompiled 2023R2 source); on 2020 the native client derives the mode
outside the WCF wire.

Version is already exposed (ProbeAsync/GetRuntimeParameterAsync), so no hollow
GetHistorianInfoAsync is shipped (same disposition as R1.3 timezone). This
completes the reachable 2020-WCF M1 read surface; remaining M1 = config writes
(gated on explicit request) or gRPC/2023R2-only items.

RE aids kept: harness `historian-info` scenario, Capture-HistorianInfo.ps1,
decode-historian-info-capture.py, and StringHandleProbeDiagnosticTests
.GETHI_CandidateInfoNames (asserts the named-value-only finding; gated).
Docs: wcf-historian-info.md (new) + roadmap/matrix/wall-doc updates. 230 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 23:42:27 -04:00
Joseph Doherty 1a539882d0 R1.1 ExecuteSqlCommandAsync (ExeC + GetR, NRBF DataTable, no BinaryFormatter)
Ship SQL command execution over the 2020 WCF aa/Retr/ExeC + aa/Retr/GetR ops:
HistorianClient.ExecuteSqlCommandAsync(sql) -> HistorianSqlResult (columns +
typed rows). String-handle ops reached with the Open2 storage-session GUID
formatted uppercase (the handle format that unlocked GETRP/GETHI).

Chain: Retr.GetV prime -> ExeC(handle, sql, option=0, ref queryHandle) ->
GetR loop. Key gotcha captured: GetR returns FALSE even on success -- the byte
stream is in pResultBuff regardless; false just signals the final page. So the
orchestrator consumes the buffer first, then stops on a false result / empty page.

GetR's pResultBuff is an NRBF-serialized System.Data.DataTable
(SerializationFormat.Xml: members XmlSchema (XSD) + XmlDiffGram (rows)).
BinaryFormatter is removed from .NET 10, so the stream is decoded read-only with
the System.Formats.Nrbf package (NrbfDecoder) + XDocument -- no BinaryFormatter,
no code execution. Values are typed per the XSD type, falling back to string.

Adds: HistorianSqlResult / HistorianSqlColumn / HistorianSqlExecuteOption models,
HistorianSqlResultProtocol (NRBF + diffgram parser), HistorianWcfSqlClient
(ExeC/GetR orchestration with an AVEVA_HISTORIAN_SQL_DUMP diagnostic), dialect +
public API. Golden WcfSqlResultProtocolTests pinned to the real clean GetR stream
for the benign "SELECT 1 AS ProbeValue" (no sensitive data); gated live tests
(single cell + multi-column/multi-row/NULL). Doc: wcf-exec-sql.md; roadmap R1.1
DONE; wall doc + memory updated (incl. the QTB-server-side nuance). 229 tests green.

Note: a raw instrument-wcf capture corrupts a large pResultBuff with MDAS
transport chunk markers (0x9F); the clean contract-level byte[] is dumped via the
AVEVA_HISTORIAN_SQL_DUMP env var for the golden fixture.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 23:16:06 -04:00
Joseph Doherty 108220c36b R1.5 GetTagExtendedPropertiesAsync (GetTepByNm) + R1.6 closed (no op)
Ship tag extended-property reads over the 2020 WCF aa/Retr/GetTepByNm op:
HistorianClient.GetTagExtendedPropertiesAsync(tag) -> name/value pairs.

String-handle op reached with the Open2 storage-session GUID formatted
uppercase (same format that unlocked GETRP/GETHI/ExeC). Routed via the
name-based native path (GetTagExtendedPropertiesByName, server-fetch flag),
not the index-based TagQuery path.

Evidence-backed findings from the capture:
- GetTepByNm (and GetTgByNm) succeed with the uppercase handle -- further
  validates the resolved string-handle wall.
- QTB (StartTagQuery) does NOT punch through: captured uppercase, it still
  fails server-side (CMdServer::StartActiveTagnamesQuery over the
  aahMetadataServer pipe) -- a metadata-server blocker, not handle format.
- R1.6 (localized properties) has NO distinct op (only error-message/UI-text
  localization in the managed client); collapses into R1.5. Closed, not throwing.

Wire format (golden-pinned, synthetic bytes -- no dev tag names committed):
- request tagNames = uint count + per-name(uint charCount + UTF-16)
- response = uint tagCount + per-tag(marker + compact-ASCII name +
  uint propCount + per-prop(marker + compact-ASCII name + 0x43 VT_BSTR value)
  + trailer); sequence-paged.

Adds: HistorianTagExtendedProperty model, HistorianTagExtendedPropertyProtocol
(codec), HistorianWcfTagExtendedPropertyClient (orchestration), dialect +
public API; golden WcfTagExtendedPropertyProtocolTests (4) + gated live test
(HISTORIAN_TEP_TAG). Tooling: Capture-TagExtendedProperties.ps1,
decode-tag-properties-capture.py, harness tag-extended-properties scenario.
Docs: wcf-tag-extended-properties.md; roadmap R1.5 DONE / R1.6 collapsed;
wall doc + memory updated with the QTB-server-side nuance. 228 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 22:52:07 -04:00
Joseph Doherty 4da5287d01 R1.2 GetRuntimeParameter + string-handle wall RESOLVED (handle-format bug)
Execute HCAL roadmap R1.2 (GetRuntimeParameterAsync) end-to-end, and in doing so
discover that the "string-handle wall" blocking R1.1/R1.4/R1.5/R1.6 was a handle
FORMAT bug, not a missing native session/filter registration.

R1.2 (shipped, live-verified):
- Captured native GetRuntimeParameter -> WCF op aa/Stat/GETRP (string-handle op,
  GETHI's shape), via scripts/Capture-RuntimeParam.ps1 + instrument-wcf-{write,read}message.
- HistorianRuntimeParameterProtocol serializes pRequestBuff (54 67 01 00 + uint
  nameCount + per-name uint charCount + UTF-16) and parses pResponseBuff (version +
  uint resultCount + CRetVariant 0x43 VT_BSTR + uint16 len + uint16 charCount + UTF-16).
- IStatusServiceContract2.GetRuntimeParameter (GETRP) op; HistorianWcfStatusClient
  passes the Open2 storage-session GUID as the string handle, UPPERCASE.
- Public HistorianClient.GetRuntimeParameterAsync(name) via the dialect.
- Golden WcfRuntimeParameterProtocolTests + gated live test; returns HistorianVersion.

String-handle wall RESOLVED (proven, public APIs deferred):
- The Open2 storage GUID works as the string handle when sent UPPERCASE
  (ToString("D").ToUpperInvariant()); earlier "blocked" probes used lowercase.
- Live-probed GETHI (R1.4) -> returns data; ExeC (R1.1) -> Retr.GetV prime -> ExeC ->
  GetR returns a BinaryFormatter-serialized .NET DataTable. Gated
  StringHandleProbeDiagnosticTests + scripts/Capture-ExecSql.ps1 + exec-sql harness scenario.
- Docs flipped: wcf-string-handle-wall.md RESOLVED banner; roadmap R1.1/R1.4 reachable,
  R1.5/R1.6 likely; wcf-status-localhost.md GETRP section.
- R1.1/R1.4 public APIs NOT shipped: ExeC needs a GetR paging loop + a BinaryFormatter-
  stream parser (BinaryFormatter is removed from .NET 10); GETHI full-info struct needs
  its own capture.

223 unit tests pass; gated live tests green against the local 2020 Historian.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 22:10:31 -04:00
Joseph Doherty 6d470eab4a R1.7: server-side event filters — ReadEventsAsync(HistorianEventFilter), live-honored
Roadmap M1 R1.7. Filters are set on the native EventQuery object via
AddEventFilter(property, HistorianComparisionType, value) — NOT EventQueryArgs
(time/count/order only). Found via a new harness --dump-type-members command.

Captured the native filtered StartEventQuery pRequestBuff (Capture-EventFilter.ps1 +
harness --event-filter knob) and diffed Equal(0) vs Contains(12) to isolate the
operator field. Filter block (decoded byte-for-byte):
  ushort 0 + uint filterCount + uint condCount + uint nameLen + name(UTF-16) +
  uint 1 + ushort op + uint 1 + value(0x09-LEN-0x00 compact-ASCII) + byte 0

The filter is REAL, not inert (unlike the analog-summary knobs): a non-matching
predicate returns 0 events; Type=Equal=User.Write returns only User.Write events.
Verified live via both the native harness and the SDK.

- HistorianClient.ReadEventsAsync(start, end, HistorianEventFilter, ct) overload
- HistorianEventFilter + HistorianEventComparison (18 ops, ordinals = native)
- Filter encoding in HistorianEventQueryProtocol (empty-filter path unchanged)
- Golden-byte tests (block match, op field, empty-filter regression) + gated live test

Single string-valued predicate only; multi-filter (OR) / multi-condition (AND via
AddEventFilterCondition) framing is partially captured and not shipped. 216 unit
tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
2026-06-20 18:32:03 -04:00