Static decompile mining of the 2023 R2 client corroborates the live R3.1 error:
the AddNonStreamValues failure is the missing StorageService.OpenStorageConnection,
which creates exactly the \.\pipe\aahStorageEngine\console,sid(...) session named
in the server error. Mapped the full native sequence:
HistoryService.OpenConnection (have) -> StorageService.OpenStorageConnection
(MISSING) -> StorageService.RegisterTags -> AddNonStreamValuesBegin (works) ->
AddNonStreamValues(btInput) (fails - no console session) -> End(commit).
Two hard parts remain, each a live-production decode loop with no static shortcut:
(1) reproduce the 12-arg OpenStorageConnection handshake (several args inferred);
(2) decode the AddNonStreamValues btInput (C++-built, absent from decompiles; only
the 44-byte packed HISTORIAN_VALUE2 is known). Documented in revision-write-path.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Empirically decoded the AddNonStreamValues btInput framing against the live 2023
R2 server (grpc-nonstream-decode command + ProbeNonStreamedBuffersAsync driver).
Every transaction rolled back (bCommit=false) — no data written.
Finding: the btInput is assembled native-C++-side (not in any decompile), so 6
evidence-based framings (44-54B, packed HISTORIAN_VALUE2 variants) were probed.
All 6 returned the IDENTICAL server error while an empty buffer returned a
different InvalidParameter — so non-empty buffers pass parameter validation into
CHistStorageConnection::StoreNonStreamValues, which routes to the
\.\pipe\aahStorageEngine\console pipe server-side. Identical-across-framings =>
the blocker is NOT the btInput layout but a missing storage-engine console
session / tag-registration precondition for the connection.
Next step (untested): StorageService.OpenStorageConnection + tag registration
(RegisterTags/AddTagidPairs/AddShardTagids) before AddNonStreamValues, then
commit + read-back on a sandbox tag. Documented in revision-write-path.md
(R3.1 decode section); raw artifact gitignored.
272 unit tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
The D2 storage-engine-pipe wall is WCF-transport-specific. On the 2023 R2 gRPC
front door, TransactionService is a first-class service AND the gateway to the
storage engine, so the Open2 storage-session GUID (uppercase) is accepted
directly as strHandle with no legacy pipe.
Live-verified against the real 2023 R2 server over a write-enabled (0x401) gRPC
session: AddNonStreamValuesBegin returns a real strTransactionId, and
AddNonStreamValuesEnd(bCommit=false) discards it cleanly (no data written). On
2020 WCF the same op returns UnknownClient(51) for every handle + priming chain.
- HistorianGrpcRevisionProbe + grpc-revision-probe CLI command + gated test
NonStreamedWriteTransaction_OverGrpc_BeginsAndDiscards (live pass).
- HistorianGrpcHandshake.OpenSession gains an optional connectionMode param
(default read-only 0x402; pass 0x401 for write-enabled) — non-breaking.
- Docs: revision-write-path.md "the wall is gone" section; roadmap M3 section,
R3.1-R3.3 rows, one-glance table, and status note updated honestly.
Not yet shipped: the AddNonStreamValues btInput VTQ buffer is uncaptured (never
guess wire bytes), so no value-commit is implemented. Scope is non-streamed
ORIGINAL backfill; revision EDITS (R4.2) remain pipe-only even on gRPC.
272 unit tests pass; sanitization scan clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Live-probed both R1.3 and R1.4 against a real 2023 R2 server over the gRPC
StatusService; implemented the one that carries an evidence-backed value.
R1.3 GetServerTimeZoneAsync — SHIPPED:
- StatusService.GetSystemTimeZoneName(uiHandle) returns the real server zone
over RemoteGrpc (the 2020 WCF op is a client-side stub returning empty).
- HistorianGrpcStatusClient.GetSystemTimeZoneNameAsync -> dialect routing ->
public HistorianClient.GetServerTimeZoneAsync. Non-gRPC transports fail
closed with ProtocolEvidenceMissingException (no empty-string lie).
- Golden message-shape unit test + non-gRPC guardrail unit test + gated live
test. 271 unit tests pass.
R1.4 GetHistorianInfoAsync (EventStorageMode) — bounded out on gRPC too:
- gRPC GetHistorianInfo is the same named-value query as 2020 WCF (only
HistorianVersion resolves); EventStorageMode + 7 variants fail on both
GetHistorianInfo and GetSystemParameter. The 518-byte struct is filled by a
native vtable+648 HCAL call, not the gRPC op (per the 2023 R2 decompile), so
the field is never on the wire. Not shipped on any transport. Closes the
roadmap's open "build against a live 2023 R2 server" caveat.
Also correct the stale M3 roadmap section: D2 already proved
Transaction.AddNonStreamValues* rides the storage-engine pipe (STransactPipeClient2
-> aaStorageEngine), not WCF — same wall as R4.2 — so M3-over-WCF is blocked, not
"the path that is NOT the gated cache push".
Docs: hcal-roadmap.md, wcf-historian-info.md, wcf-status-localhost.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Recorded the native-disassembly attempt on aahClientManaged.dll (mixed-mode):
ilspycmd cannot decompile it; capstone byte-search can't locate the StartTagQuery
0x6751 marker (not a plain immediate — it's an .rdata constant loaded RIP-relative,
the .text "51 67 00 00" hits are coincidental jump-table data). Managed metadata
gives QueryTag field semantics but not the binary packet-id. Finishing QueryTag
needs Ghidra/IDA xref analysis or the live IL-rewrite capture.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Deepened the R0.1 browse finding. QueryTag's constant rejection decodes to
ArchestrA.CloudHistorian.Contract.ErrorCode.InvalidPacketId (72): the btRequest needs
a QueryTag-specific packet-id header (the generic 0x6751/v1 header StartTagQuery accepts
is rejected). The semantic fields are known from CloudHistorian.Contract
(QueryHandle/QueryType/StartIndex/TagCount request; TagNames[]+TagMetadataBuffer response),
but the binary packet framing lives in native aahClient.dll — aahClientManaged.dll is
mixed-mode (ilspycmd cannot decompile it) and no managed assembly builds the buffer.
Finishing QueryTag needs native RE (Ghidra/IDA) or a live gRPC capture of the stock client.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Probes the 2023 R2 gRPC browse path and records the finding. The front door does
NOT hit the 2020 WCF metadata-server-pipe wall.
- RetrievalService.StartTagQuery is cracked: the server (CMdServer::StartActiveTagnamesQuery
over \.\pipe\aahMetadataServer\console) parses the filter as OData. startswith()/
contains()/eq/empty succeed and return the 8-byte (queryHandle, tagCount); SQL-LIKE "%"
and glob "*" fail with "ODataFilter: bad token". Live: 220 Sys* tags counted.
- QueryTag (paging) remains: every guessed btRequest returns a constant native error
type 4 / code 72 (content-independent) -> framing needs a native capture, not guessing.
Adds RE probe helpers Grpc/HistorianGrpcTagClient.ProbeStartTagQuery + ProbeTagQuerySequence,
a gated StartTagQuery_OverGrpc_AcceptsODataFilter test, and the finding doc
docs/reverse-engineering/grpc-tag-query-odata.md. Browse is not yet wired (QueryTag open).
217 unit tests pass; 5/5 live gRPC tests pass. No tag names/identities committed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Routes HistorianClient.GetTagMetadataAsync over gRPC when Transport==RemoteGrpc,
via the new Grpc/HistorianGrpcTagClient calling RetrievalService.GetTagInfosFromName
(the plural string-handle metadata op).
- String handle = the Open2 storage-session GUID formatted uppercase (the format
that resolves the native string-handle path); threaded out of the shared handshake
via a new HistorianGrpcHandshake.Session { ClientHandle, StorageSessionId, StringHandle }.
- Request btTagNames = uint count + per-name(uint charCount + UTF-16LE) — golden-byte
unit-tested (BuildTagNamesBuffer).
- Response btTagInfos = uint count + CTagMetadata records — decoded by the existing
HistorianTagQueryProtocol.ParseGetTagInfoResponse; data type via the shared MapDataType.
The 2020 WCF string-handle wall does NOT apply on the gRPC front door, as the
string-handle-wall RE note predicted. LIVE-VERIFIED against a real 2023 R2 server:
GetTagMetadataAsync returns the requested tag with a valid decoded data type.
216 unit tests pass. Captured framing confirmed live then discarded; no tag names
or identities committed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
R0.3 system-param over gRPC is now LIVE-VERIFIED against the real 2023 R2 server
(returned HistorianVersion), alongside the re-confirmed read chain and probe.
The apparent NTLM round-1 SEC_E_LOGON_DENIED "blocker" was a test-harness
credential-parsing bug, not a server/account/SDK issue: the gitignored creds
file stores quoted values and the env-setup must strip surrounding quotes before
exporting HISTORIAN_USER/PASSWORD. With quotes stripped, the NAM domain account
authenticates and the full chain passes. The round-failure diagnostic added
during the hunt (HistorianNativeHandshake.DescribeError) is kept.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Roadmap docs/plans/hcal-roadmap.md, milestone M0 (gRPC parity for the DONE
surface). Now unblocked for live verification by a reachable 2023 R2 server.
- R0.4 Probe over gRPC: new HistorianGrpcProbe calls History/Retrieval/Status
GetInterfaceVersion (unauthenticated). ProbeAsync routes over gRPC when
Transport==RemoteGrpc. LIVE-VERIFIED against a real 2023 R2 server — needs no
credentials (runs before the auth loop), so it works despite the auth blocker.
- R0.3 System parameter over gRPC: new HistorianGrpcStatusClient calls
StatusService.GetSystemParameter over the authenticated session; routed in the
dialect. Built + unit-tested (request/response field mapping pinned).
Live-verification pending an auth fix (see below).
- Extracted the proven auth handshake from HistorianGrpcReadOrchestrator into
shared Grpc/HistorianGrpcHandshake (reused by read + status + future
browse/metadata). Repointed the IL structural guardrail test to it.
- Diagnostics: round-failure now decodes the native server error + hex/ASCII
preview (HistorianNativeHandshake.DescribeError). This surfaced the live auth
blocker as SEC_E_LOGON_DENIED (0x8009030C) at NTLM round 1 — framing is correct,
the credential did not validate. Probable cause: stale file password or NAM-domain
NTLM restriction (Kerberos/RDP works, NTLM denied; no SPN path over the tunnel).
216 unit tests pass; live gRPC probe passes. Sanitization scan clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
First live-verified gRPC read against a real 2023 R2 Historian. The handshake
previously failed at round 0 (cred-independent) because the SSPI/Negotiate token
loop was routed to HistoryService.ExchangeKey. ExchangeKey is a separate
key-exchange/cert-path op, not the Negotiate loop — the token loop belongs on
StorageService.ValidateClientCredential, which kept the 2020 inBuff/outBuff token
framing the SDK's WrapValidateClientCredentialToken/TryRead helpers already build.
Captured + diffed against the recovered 2023 R2 protobuf contract and the
decompiled stock client; routing the loop to ValidateClientCredential completes
the full chain (ValidateClientCredential x N -> OpenConnection -> StartQuery ->
GetNextQueryResultBuffer) and returns rows.
- HistorianGrpcReadOrchestrator: token loop now calls
StorageService.ValidateClientCredential(Handle, InBuff); corrected the op-map
doc comment (was asserting the wrong ExchangeKey mapping).
- HistorianServerVersionGate: accept History interface version 12 alongside 11.
Live server reports History=12, Retrieval=4, Storage=4; the buffers are
byte-identical (a live read returns rows), so 12 is buffer-compatible. Retrieval
stays pinned at 4 (matches). New AcceptedVersions() supports multi-version gates.
- New HistorianGrpcHandshakeRoutingTests: IL-level structural guardrail that
disassembles the orchestrator (incl. lambda closures) and asserts the handshake
invokes ValidateClientCredential and never ExchangeKey — fails if the regression
returns.
- Updated gate tests + CLAUDE.md gRPC op-map.
240 unit tests pass on the full stack; 210 on this branch's base. The byte
payloads remain the proven 2020 protocol.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
DelTep (extended-property delete) — wire format captured + serializer
golden-proven, but live delete is server-blocked and NOT exposed publicly:
- Captured the DelTep inBuff via a cross-session trick (harness add-tep gains
--tep-skip-add + read-for-sync before --tep-delete; Capture-DeleteTagExtended
Properties.ps1 / decode-del-tep-capture.py). Layout = same group framing as
AddTEx but property-name-only (no 0x43 value) + 0x00 group trailer.
- SerializeDeleteRequest + 4 golden tests pin the server-accepted buffer.
- A decisive experiment shows SDK-added properties ARE deletable (the native
client read-syncs and deletes one), so SDK-add is complete; the SDK's own
DelTep is rejected by CHistStorage::DeleteTagExtendedProperties even with
byte-identical inBuff, matching mode/handle, GetTgByNm+GetTepByNm prime, open
channel, and 60s retries. Root cause: the native multiplexes services over one
connection (per-connection working set); the SDK's per-service WCF channels
don't reproduce it. Kept as documented-but-blocked internal orchestrator path;
no public HistorianClient delete API.
Bounded out with evidence (no code; docs + roadmap + probe):
- R1.12 localized-property write — no op on 2020 (mirror of R1.6); no
*LocalizedPropert*/TagLocalized* symbol in any current/*.dll.
- R1.13 non-analog tag create — GATED; native AddTag rejects every non-analog
type client-side (ValidationFailed, before any WCF op): SingleByteString,
DoubleByteString, Int1 all fail, Float works. No Discrete type in the native
enum, no TagType setter. No wire request to capture.
- R1.3 timezone + R1.4 EventStorageMode — re-confirmed 2023R2/gRPC-only from
the Runtime DB schema (no timezone param, no EventStorageMode anywhere) and a
parameter-op probe (GetSystemParameter + GETRP return null/throw for every
candidate; only HistorianVersion works).
238 unit tests pass; full solution builds with 0 warnings.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Adds user-defined extended properties to an existing tag via the 2020 WCF
AddTEx (AddTagExtendedProperties) op. Write-enabled connection + uppercase
storage-session GUID handle; reuses the write orchestrator open/priming chain.
The AddTEx inBuff is the exact inverse of the R1.5 GetTepByNm read-response
framing, so the serializer mirrors the read parser:
uint32 groupCount + 0x01(group) + [0x09+u16+ASCII tag] + uint32 propCount
+ per prop{ 0x02 + [0x09+u16+ASCII name] + 0x43 VT_BSTR + u16 payloadLen
+ u16 charCount + UTF-16 value } + 0x01(group trailer) + 0x00(terminator).
The trailing 0x00 is required — without it inBuff is one byte short and the
server throws SErrorException in CHistStorage::AddTagExtendedProperties. The
golden fixture pins the clean inBuff the live server accepted (dumped via
AVEVA_HISTORIAN_TEP_DUMP); read-back verified via R1.5. String (0x43) values only.
Delete (DelTep) is deferred: the native DeleteTagExtendedPropertiesByName does a
client-side sync check and returns err 229 for a just-added property, so the
DelTep request never reaches the wire and its inBuff can't be captured yet.
Shipped: HistorianClient.AddTagExtendedPropertiesAsync/AddTagExtendedPropertyAsync;
HistorianTagExtendedPropertyProtocol.SerializeAddRequest; orchestrator path;
golden WcfTagExtendedPropertyWriteProtocolTests (4); gated live write/read-back test;
native-harness `add-tep` scenario + Capture-AddTagExtendedProperties.ps1 +
decode-add-tep-capture.py. Doc: wcf-add-tag-extended-properties.md. 233 tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Tag rename has no dedicated WCF op — the (old,new) name batch rides the
generic History StartJob (StJb) job buffer; the server returns a job id and
applies renames asynchronously. Handle is the uppercase storage-session GUID,
Open2 in write mode; reuses the write orchestrator's open+priming chain.
jobBuffer layout (decoded + server-validated): byte[7] zero prefix + uint32
pairCount + per pair (uint32 oldCharCount + UTF-16 oldName + uint32
newCharCount + UTF-16 newName), order (old,new). The raw instrument capture
mangles the final byte with MDAS chunk markers (the R1.1 lesson), so the golden
fixture pins the CLEAN byte[] the SDK handed the channel (dumped via
AVEVA_HISTORIAN_RENAME_DUMP) — the exact buffer the live server accepted and
renamed with.
Gated server-side by the AllowRenameTags system parameter (default 0): when
disabled the native client rejects pre-wire (err 132); the managed SDK surfaces
it as StartJob=false -> Accepted=false. Enabling needs a Historian config
reload, not just a storage-engine restart.
Shipped: HistorianClient.RenameTagAsync/RenameTagsAsync -> HistorianTagRenameResult;
HistorianTagRenameProtocol; orchestrator RenameTags/SendStartJobRename; golden
WcfTagRenameProtocolTests (4, pins server-accepted buffer); gated live test
RenameTagsAsync_AgainstLocalHistorian_RenamesSandboxTag (passed end-to-end).
Native-harness `rename` scenario + Capture-RenameTags.ps1 + decode-rename-capture.py.
Doc: docs/reverse-engineering/wcf-rename-tags.md. 213 tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Captured the native HistorianAccess.GetHistorianInfo(out HistorianInfo, out err)
and decoded the wire: over 2020 WCF, GETHI is a named-value query whose only
working key is "HistorianVersion" (response ~30 bytes = the version string).
Probed 7 storage-mode key names -> all ok=False/err. The 518-byte HISTORIAN_INFO
struct + EventStorageMode@514 is the 2023R2 HCAL-native/gRPC model (confirmed
from the decompiled 2023R2 source); on 2020 the native client derives the mode
outside the WCF wire.
Version is already exposed (ProbeAsync/GetRuntimeParameterAsync), so no hollow
GetHistorianInfoAsync is shipped (same disposition as R1.3 timezone). This
completes the reachable 2020-WCF M1 read surface; remaining M1 = config writes
(gated on explicit request) or gRPC/2023R2-only items.
RE aids kept: harness `historian-info` scenario, Capture-HistorianInfo.ps1,
decode-historian-info-capture.py, and StringHandleProbeDiagnosticTests
.GETHI_CandidateInfoNames (asserts the named-value-only finding; gated).
Docs: wcf-historian-info.md (new) + roadmap/matrix/wall-doc updates. 230 tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Ship SQL command execution over the 2020 WCF aa/Retr/ExeC + aa/Retr/GetR ops:
HistorianClient.ExecuteSqlCommandAsync(sql) -> HistorianSqlResult (columns +
typed rows). String-handle ops reached with the Open2 storage-session GUID
formatted uppercase (the handle format that unlocked GETRP/GETHI).
Chain: Retr.GetV prime -> ExeC(handle, sql, option=0, ref queryHandle) ->
GetR loop. Key gotcha captured: GetR returns FALSE even on success -- the byte
stream is in pResultBuff regardless; false just signals the final page. So the
orchestrator consumes the buffer first, then stops on a false result / empty page.
GetR's pResultBuff is an NRBF-serialized System.Data.DataTable
(SerializationFormat.Xml: members XmlSchema (XSD) + XmlDiffGram (rows)).
BinaryFormatter is removed from .NET 10, so the stream is decoded read-only with
the System.Formats.Nrbf package (NrbfDecoder) + XDocument -- no BinaryFormatter,
no code execution. Values are typed per the XSD type, falling back to string.
Adds: HistorianSqlResult / HistorianSqlColumn / HistorianSqlExecuteOption models,
HistorianSqlResultProtocol (NRBF + diffgram parser), HistorianWcfSqlClient
(ExeC/GetR orchestration with an AVEVA_HISTORIAN_SQL_DUMP diagnostic), dialect +
public API. Golden WcfSqlResultProtocolTests pinned to the real clean GetR stream
for the benign "SELECT 1 AS ProbeValue" (no sensitive data); gated live tests
(single cell + multi-column/multi-row/NULL). Doc: wcf-exec-sql.md; roadmap R1.1
DONE; wall doc + memory updated (incl. the QTB-server-side nuance). 229 tests green.
Note: a raw instrument-wcf capture corrupts a large pResultBuff with MDAS
transport chunk markers (0x9F); the clean contract-level byte[] is dumped via the
AVEVA_HISTORIAN_SQL_DUMP env var for the golden fixture.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Ship tag extended-property reads over the 2020 WCF aa/Retr/GetTepByNm op:
HistorianClient.GetTagExtendedPropertiesAsync(tag) -> name/value pairs.
String-handle op reached with the Open2 storage-session GUID formatted
uppercase (same format that unlocked GETRP/GETHI/ExeC). Routed via the
name-based native path (GetTagExtendedPropertiesByName, server-fetch flag),
not the index-based TagQuery path.
Evidence-backed findings from the capture:
- GetTepByNm (and GetTgByNm) succeed with the uppercase handle -- further
validates the resolved string-handle wall.
- QTB (StartTagQuery) does NOT punch through: captured uppercase, it still
fails server-side (CMdServer::StartActiveTagnamesQuery over the
aahMetadataServer pipe) -- a metadata-server blocker, not handle format.
- R1.6 (localized properties) has NO distinct op (only error-message/UI-text
localization in the managed client); collapses into R1.5. Closed, not throwing.
Wire format (golden-pinned, synthetic bytes -- no dev tag names committed):
- request tagNames = uint count + per-name(uint charCount + UTF-16)
- response = uint tagCount + per-tag(marker + compact-ASCII name +
uint propCount + per-prop(marker + compact-ASCII name + 0x43 VT_BSTR value)
+ trailer); sequence-paged.
Adds: HistorianTagExtendedProperty model, HistorianTagExtendedPropertyProtocol
(codec), HistorianWcfTagExtendedPropertyClient (orchestration), dialect +
public API; golden WcfTagExtendedPropertyProtocolTests (4) + gated live test
(HISTORIAN_TEP_TAG). Tooling: Capture-TagExtendedProperties.ps1,
decode-tag-properties-capture.py, harness tag-extended-properties scenario.
Docs: wcf-tag-extended-properties.md; roadmap R1.5 DONE / R1.6 collapsed;
wall doc + memory updated with the QTB-server-side nuance. 228 tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Execute HCAL roadmap R1.2 (GetRuntimeParameterAsync) end-to-end, and in doing so
discover that the "string-handle wall" blocking R1.1/R1.4/R1.5/R1.6 was a handle
FORMAT bug, not a missing native session/filter registration.
R1.2 (shipped, live-verified):
- Captured native GetRuntimeParameter -> WCF op aa/Stat/GETRP (string-handle op,
GETHI's shape), via scripts/Capture-RuntimeParam.ps1 + instrument-wcf-{write,read}message.
- HistorianRuntimeParameterProtocol serializes pRequestBuff (54 67 01 00 + uint
nameCount + per-name uint charCount + UTF-16) and parses pResponseBuff (version +
uint resultCount + CRetVariant 0x43 VT_BSTR + uint16 len + uint16 charCount + UTF-16).
- IStatusServiceContract2.GetRuntimeParameter (GETRP) op; HistorianWcfStatusClient
passes the Open2 storage-session GUID as the string handle, UPPERCASE.
- Public HistorianClient.GetRuntimeParameterAsync(name) via the dialect.
- Golden WcfRuntimeParameterProtocolTests + gated live test; returns HistorianVersion.
String-handle wall RESOLVED (proven, public APIs deferred):
- The Open2 storage GUID works as the string handle when sent UPPERCASE
(ToString("D").ToUpperInvariant()); earlier "blocked" probes used lowercase.
- Live-probed GETHI (R1.4) -> returns data; ExeC (R1.1) -> Retr.GetV prime -> ExeC ->
GetR returns a BinaryFormatter-serialized .NET DataTable. Gated
StringHandleProbeDiagnosticTests + scripts/Capture-ExecSql.ps1 + exec-sql harness scenario.
- Docs flipped: wcf-string-handle-wall.md RESOLVED banner; roadmap R1.1/R1.4 reachable,
R1.5/R1.6 likely; wcf-status-localhost.md GETRP section.
- R1.1/R1.4 public APIs NOT shipped: ExeC needs a GetR paging loop + a BinaryFormatter-
stream parser (BinaryFormatter is removed from .NET 10); GETHI full-info struct needs
its own capture.
223 unit tests pass; gated live tests green against the local 2020 Historian.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Roadmap M1 R1.7. Filters are set on the native EventQuery object via
AddEventFilter(property, HistorianComparisionType, value) — NOT EventQueryArgs
(time/count/order only). Found via a new harness --dump-type-members command.
Captured the native filtered StartEventQuery pRequestBuff (Capture-EventFilter.ps1 +
harness --event-filter knob) and diffed Equal(0) vs Contains(12) to isolate the
operator field. Filter block (decoded byte-for-byte):
ushort 0 + uint filterCount + uint condCount + uint nameLen + name(UTF-16) +
uint 1 + ushort op + uint 1 + value(0x09-LEN-0x00 compact-ASCII) + byte 0
The filter is REAL, not inert (unlike the analog-summary knobs): a non-matching
predicate returns 0 events; Type=Equal=User.Write returns only User.Write events.
Verified live via both the native harness and the SDK.
- HistorianClient.ReadEventsAsync(start, end, HistorianEventFilter, ct) overload
- HistorianEventFilter + HistorianEventComparison (18 ops, ordinals = native)
- Filter encoding in HistorianEventQueryProtocol (empty-filter path unchanged)
- Golden-byte tests (block match, op field, empty-filter regression) + gated live test
Single string-valued predicate only; multi-filter (OR) / multi-condition (AND via
AddEventFilterCondition) framing is partially captured and not shipped. 216 unit
tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Roadmap Milestone 2 (event sending). Capture disproved the assumption that event
delivery uses the non-WCF storage-engine pipe (which would block it like revision
writes): a native AddStreamedValue(HistorianEvent) leaves over WCF as AddS2
(IHistoryServiceContract2.AddStreamValues2). CM_EVENT is a built-in registered tag,
so the 129 TagNotFoundInCache gate that blocks AddS2 for user tags does not apply.
- R2.1: NativeTraceHarness "event-send" scenario + Capture-EventSend.ps1; two
captures diffed to separate constant framing from value-dependent fields.
- R2.2: HistorianEventWriteProtocol serializes the AddS2 pBuf (storage sample buffer
wrapping the event VTQ) — golden-byte tested. Decoded "OS" sig + length fields +
CM_EVENT tag id + EventTime/ReceivedTime FILETIMEs + Opc 192 + 0x118D descriptor +
event Id + Namespace + EventType + version 5 + typed property bag.
- R2.3/R2.4: HistorianWcfEventOrchestrator.SendEventAsync (Open2 event-mode 0x501 ->
reuse CM_EVENT RTag2/EnsT2 -> AddStreamValues2) + HistorianClient.SendEventAsync.
- R2.5: gated live test; server accepts the AddS2 (success, empty error buffer).
Server requires delivered byte[].Length == declared packet length (uint32@0x04); the
native relies on the MDAS encoder adding a pad byte, so the SDK emits an explicit
trailing 0x00 (else AddS2 rejects with "CValuStream buffer size vs packet length
mismatch"). Original events only (RevisionVersion=0) with string properties; other
property types + revision/update/delete throw ProtocolEvidenceMissingException.
Caveat (documented): accepted events are not persisted on the local dev box; the
native client behaves identically (event ingestion pipeline inactive) — not an SDK
gap. 212 unit tests pass; 16/16 event tests pass live.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Captured the native StartQuery2 pRequestBuff and the GetNextQueryResultBuffer2
response (instrument-wcf-writemessage + chained instrument-wcf-readmessage) and
decoded both against AnalogSummaryHistory SQL ground truth. Conclusion: the rich
multi-aggregate analog/state summary struct is NOT delivered over the 2020 WCF
binary protocol — the response is the ordinary version-9 row buffer the existing
aggregate parser already handles, carrying one value per cycle selected by
RetrievalMode (QueryType 5-8), not ValueSelector (inert on this path). So
"analog summary" == the existing ReadAggregateAsync; no new src/ code warranted.
Tooling (tools/ + scripts/ only, nothing in src/):
- NativeTraceHarness: drive summary knobs via --value-selector /
--aggregation-type / --max-states (uint16) / --filter
- Capture-SummaryRequest.ps1: repeatable instrument+stage+matrix capture,
-WithResponse chains the ReadMessage hook
- decode-summary-capture.py: StartQuery2 request diff vs baseline
- decode-summary-response.py: response decode vs SQL ground truth
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Scoped R1.8/R1.9 (analog/state summary). Reachable on 2020 WCF via the proven
uint-handle StartQuery2; all decode targets located in aahClientManaged.dll
(CAnalogSummaryValue.UnpackFromValueBuffer + struct field sets). Live probing
showed the server accepts SummaryType 1/2/4/5 but returns 0 rows with the
obvious params — the summary config is native-side (AutoSummaryParameters
trailer / native QueryType), not recoverable from managed metadata. Next step
is a native pRequestBuff capture via instrument-wcf-writemessage.
Adds the enum-dump RE CLI command. No guessed code in src/; 208 tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pushed on recovering the summary query params. Findings:
- Added `enum-dump` to the RE CLI (dumps a managed enum's literal members).
Confirmed INSQL_QUERYTYPE / HISTORIAN_SUMMARYTYPE are value__-only in the
managed metadata — their named members are native-side constants, so they
can't be recovered statically. Same for CColumnNameMap.LoadColumnNameMap
(column->bit built from native string/const data, not IL ldstr/ldc).
- Live-probed StartQuery2 against SysTimeSec sweeping QueryType/SummaryType/
ColumnSelectorFlags. The server ACCEPTS SummaryType 1/2/4/5 (returns a valid
version-9 buffer) but yields 0 rows; column flags don't change that;
QueryType 15/16 don't exist. So a summary query is NOT Full+SummaryType+
flags — the config lives in the AutoSummaryParameters trailer (currently
zeroed) and/or a native summary QueryType.
Conclusion recorded in the plan: the request shape needs a NATIVE capture
(instrument-wcf-writemessage on a real summary query), not managed-metadata
recovery or blind probing. Decode targets remain located. No guessed code in
src/; probe scaffolding removed. 208 tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Established that analog/state-summary queries are reachable on 2020 WCF — they
ride the proven uint-handle StartQuery2 path, and the request serializer already
carries QueryType/SummaryType/ColumnSelectorFlags. Located every decode target in
aahClientManaged.dll:
- CAnalogSummaryValue.UnpackFromValueBuffer (0x06000394) — row decoder
- CAnalogSummaryValue/Struct fields — Min/Max/First/Last/ValueCount/TimeGood/
Integral/IntegralOfSquares (+ per-field DateTimes, LinearIntegral)
- CStateSummaryStruct — MinContained/MaxContained/TotalContained/PartialStart/
PartialEnd/StateEntryCount
- QueryColumnSelector.Select{Analog,State,NonSummary}Columns — column flags
- INSQL_QUERYTYPE / HISTORIAN_SUMMARYTYPE — the query/summary enum values
UnpackFromValueBuffer is reader-call-based (no literal offsets), so a correct
parser needs a captured real buffer. Per project discipline no guessed summary
code was added to src/. New plan doc lays out the recover-params -> live-capture
-> decode -> implement+test path. Roadmap R1.8/R1.9 marked scoped/ready.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Probed M1 read items R1.1/R1.3/R1.4 live against the local 2020 server and
found them blocked, then documented the structural boundary (uint-handle ops
work; string-handle ops require unmapped native session registration). No
half-implementations shipped; 208 tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Probed R1.4 GetHistorianInfo (GETHI) live against the local 2020 server.
GETHI returns native error type 4 / code 1 for the exact native request shape
across 5 handle formats (storage GUID, context GUID, uint decimal/X8/0x-hex)
even with Stat.GetV ×2 priming. Its result is discarded (TryRun) in the only
place it's used, so it was never actually verified to return data managed-side.
This confirms a structural boundary on the 2020 WCF surface: ops taking a uint
client handle work (the proven read/browse/metadata/status/event surface);
ops taking a string GUID handle (ExeC, QTB, QTG, GETHI, GetTepByNm, ...) are
blocked behind an unmapped native session/filter registration. Every remaining
M1 *read* item (R1.1/R1.4/R1.5/R1.6) is string-handle -> all gated on that one
RE target. Reachable uint-handle items: R1.7 event filters, R1.8/R1.9 summary
modes.
New: docs/reverse-engineering/wcf-string-handle-wall.md (full dichotomy + table).
Roadmap R1.4/R1.5/R1.6 struck through; reachable items re-pointed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Live-probed both against the local Historian 2020 (WCF):
- R1.3 GetServerTimeZoneAsync: Status.GetSystemTimeZoneName returns rc=0 with
an empty value under a real authenticated handle — a client-side stub in the
GetServerTime family. gRPC/2023R2-only. Reverted the implementation.
- R1.1 ExecuteSqlCommandAsync: Retrieval.ExeC returns native error type 4 /
code 51 (InvalidParameter); the contract-3 string-handle ops require an
unmapped native session/filter registration step (the StartTagQuery wall).
Adds an M1a re-classification note steering future work toward proven
uint-handle / already-wired ops (R1.4 GETHI next) over string-handle ops.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- 2023 R2 RemoteGrpc transport reusing the proven native byte payloads (unit-tested;
not yet live-verified — no 2023 R2 server on the local box, which is 2020/WCF).
- R0.6 fail-closed server interface-version gate (Hist=11, Retr=4, Trx=2; evidence-based).
- CW-1 reusable capture -> sanitize -> golden-fixture pipeline + capture-tag-info CLI.
- docs/plans: imported gRPC/HCAL analysis + roadmap, progress-marked.
- Event reads live-verified + test hardened to assert well-formed parsed events.
169 non-live unit tests green; 27/27 integration tests green against the local 2020 server.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ReadEventsAsync verified to return real, parsed events against the local 2020
server (e.g. User.Write with 18 properties) — the row parser (HistorianEventRowProtocol
v9) is wired and works. The prior test only asserted NotNull with a stale "row
format not yet decoded" comment.
- Renamed to ReadEventsAsync_AgainstLocalHistorian_ReturnsWellFormedEvents.
- Widened the window to 30 days (robust against a quiet recent window).
- Asserts NotEmpty + per-event well-formedness (non-empty Type, non-null
Properties, EventTimeUtc within the queried window) — matching the ReadRawAsync
test's NotEmpty style.
- Documents the known limitation: enumeration stops at the first benign
`type=4 code=85` soft-terminal, so this verifies parsing correctness rather than
exhaustive retrieval (draining all rows needs the code-85 decode, a capture task).
Passes live (1 event over 30 days). Non-live unit suite unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The local Historian is 2020 (WCF/32568); the 2023 R2 gRPC endpoint (32565) is
absent, so M0 gRPC routing can be unit-tested but not live-verified here.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the highest-leverage reverse-engineering primitive from the roadmap: one
path to turn a live operation buffer into a committable golden fixture. Unblocks
every capture-tier item (R0.5, R1.x, R2.1).
- ProtocolCaptureSanitizer: redacts identity-bearing values (host, tag, user,
machine) from a native buffer in BOTH ASCII and UTF-16LE, overwriting in place
with an 'X' fill so length and every field offset are preserved (keeps the
fixture useful for byte-layout RE). ASCII-letter matching is case-insensitive;
secrets < 3 chars are skipped to avoid collision corruption. AssertNoSecretsRemain
is a fail-closed safety net that refuses to emit if any value survives.
- ProtocolFixtureWriter: serializes a capture to fixtures/protocol/<op>/<name>.json
with sanitized hex, length, SHA-256 of the sanitized bytes, and a scrub report.
Timestamps are passed in (deterministic / testable).
- capture-tag-info CLI command: captures a live GetTagInfoFromName response and
writes the fixture. The same native bytes ride inside 2023 R2 gRPC
GetTagInfosFromName, so the fixture is transport-agnostic.
- 11 unit tests for the sanitizer/writer (test project now references the RE tool).
- First real fixture: get-tag-info/analog-*.json — a 98-byte Int4 CTagMetadata
buffer captured live from the local Historian 2020 server, tag name redacted,
verified to contain no identity (descriptor 03 c3 00 31 = Int4, as documented).
180 non-live unit tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Turns the previously-discarded GetInterfaceVersion result into a connect-time
version pin. The native buffers carried in the WCF/MDAS body (and in the 2023 R2
gRPC bytes fields) are framed per native interface version; parsing them against
an unexpected version risks silent misinterpretation, so we throw rather than
best-effort parse.
- HistorianServerVersionGate + HistorianServiceInterface: evidence-based
supported versions discovered from a live Historian 2020 server (product
20.0.000) via the wcf-probe command — History=11, Retrieval=4, Transaction=2.
Status' GetInterfaceVersion returns 0, so Status is reachability-only.
- HistorianClientOptions.VerifyServerInterfaceVersion (default true) — bypass
knob for bringing up a server whose reported integers aren't yet captured
(e.g. a 2023 R2 gRPC endpoint carrying the same proven 2020 buffers).
- Wired into both transports' connect paths: WCF history (auth-chain helper) +
retrieval (read orchestrator), and gRPC history + retrieval.
- Mismatch throws ProtocolEvidenceMissingException naming reported/expected
version and the bypass knob.
10 new unit tests (198 total green). Verified the gate does not regress the
proven WCF read path: a live read against the local 2020 server reaches past the
gate (Retr=4 matches) — the only live failures are a pre-existing environmental
read timeout (OperationCanceledException), identical with and without this change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Version-control the planning docs alongside the code they describe:
- grpc-transport.md — 2023 R2 gRPC transport analysis (sanitized source path)
- hcal-capability-matrix.md — HistorianAccess surface x gRPC ops x histsdk status x feasibility tiers
- hcal-roadmap.md — ordered build plan M0-M4 + cross-cutting workstreams
- histevents.md — how a HistorianEvent reaches the DB (client->wire->server)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stands up HistorianTransport.RemoteGrpc end-to-end for the read path,
built on the recovered 2023 R2 gRPC contract (gRPC-Web/HTTP-1.1, port
32565, gzip). The opaque protobuf `bytes` fields carry the SAME native
binary payloads as the 2020 WCF/MDAS path, so the proven serializers and
parsers are reused unchanged.
- Grpc/Protos/*.proto: 6 protoc-validated contracts recovered from
embedded FileDescriptors (authoritative, not guessed).
- Grpc/HistorianGrpcChannelFactory: GrpcWebHandler/HTTP-1.1 channel,
ResolvePort/ResolveAddress, optional TLS + gzip.
- Grpc/HistorianGrpcReadOrchestrator: mirrors the WCF read chain over
gRPC; auth uses HistoryService.ExchangeKey (the gRPC ValCl op).
- Wcf/HistorianNativeHandshake: transport-agnostic Open2 request builder
+ SSPI/Negotiate token loop + response decode, shared by WCF and gRPC.
- Op map (2020 -> gRPC): ValCl->ExchangeKey, Open2->OpenConnection,
StartQuery2->StartQuery, GetNextQueryResultBuffer2->GetNextQueryResultBuffer.
- HistorianClientOptions: DefaultGrpcPort=32565, GrpcUseTls.
- csproj: Google.Protobuf, Grpc.Net.Client(.Web), Grpc.Tools codegen.
Not yet live-verified against a 2023 R2 server: ExchangeKey is the first
thing to revisit if a live server rejects the handshake; the inner byte
payloads are the proven 2020 protocol. Gated live test via
HISTORIAN_GRPC_HOST. 188 unit tests green; build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Browse/metadata previously hardcoded the Windows transport binding +
/Hist-Integrated endpoint regardless of options.Transport. That meant
RemoteTcpCertificate clients (notably anything from Linux, where the
Windows transport security isn't available in WCF) couldn't use these
ops even when other reads worked.
Made the binding+endpoint selection conditional:
- LocalPipe / RemoteTcpIntegrated: keep existing /Hist-Integrated +
Windows transport binding (the legacy Open2-V1 buffer carries its
own auth blob and only validates against this transport).
- RemoteTcpCertificate: switch to /HistCert + cert binding, propagating
options.ServerDnsIdentity. Cert-binding callers also opt out of
setting Windows credentials on the factory.
Cert-validation override (HistorianWcfClientCredentialsHelper.Configure)
now applies on both the history and retrieval factories.
Confirmed locally: 178/178 tests pass on Windows. Linux verification
still pending — gated tests don't exercise the path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
IL walk of the native wrapper:
HistorianAccess.AddRevisionValuesBegin (private, token 0x06006175)
-> CClientCommon.AddNonStreamValuesBegin
-> CClient.AddNonStreamValuesBegin (8-instr overload)
-> CClient.TransactionBegin
-> CHistStorageConnection.StartTransaction (token 0x06001FDD)
-> CStorageEngineConsoleClient.StartTransaction
CStorageEngineConsoleClient is built on STransactPipeClient2 +
SCrtMemFile — a shared-memory + named-pipe transport to
aaStorageEngine.exe, completely separate from WCF.
The WCF ITransactionServiceContract2.AddNonStreamValuesBegin2 op is a
server-side relay that requires a pre-existing storage-engine pipe
session for the client. Without that pipe session, the WCF relay returns
UnknownClient (51) — and there's no way to establish the pipe session
via WCF.
D2 is unimplementable as a pure-managed-WCF SDK. The native wrapper
itself depends on the C++ shared-memory channel; replicating that from
managed code would require implementing the storage-engine pipe
protocol, which is a major undertaking and out of scope.
The ITransactionServiceContract2 declaration in our contracts file
stays as documentation; no public API or orchestrator added.
HistorianWcfRevisionOrchestrator remains as an internal probe /
regression check — re-run the probe test if anyone believes the
architecture has changed.
178/178 tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tested hypothesis (1) from the plan: add RTag2(CM_EVENT tag id) to the
priming chain before AddNonStreamValuesBegin2.
Result:
- RTag2 itself succeeds: returns 25-byte response
(01000000000100000001EE39C30EDCDC010100000000000000), no error buffer.
- But AddNonStreamValuesBegin2 still fails with the same
04 33 00 00 00 (UnknownClient = 51) for all four handle formats.
So RTag2 on /Hist isn't the cross-service registration trigger we need
for /Trx. Plan doc updated with the result + next-session ordered
probes (try IStorageServiceContract, then IL walk CClientCommon,
then server-side decompile as last resort).
Probe orchestrator now also performs the RTag2 step so the test gives
one-shot diagnostic visibility of both calls.
178/178 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implemented HistorianWcfRevisionOrchestrator that talks WCF directly
to /Trx, bypassing the native wrapper entirely. Probes
AddNonStreamValuesBegin2 against the live local Historian and surfaces
what the server returns. Internal-only API; no public surface added —
the path isn't viable yet.
Findings (live test against localhost):
- ✅ The wire path is reachable. After moving from V1 (uint handle, no
errorBuffer) to V2 (string handle GUID, out errorBuffer), the server
recognizes the call (no ContractFilter mismatch, no exception).
- ✅ Server processes the call and returns a structured 5-byte error
buffer: 04 33 00 00 00 = type 4 (CustomError) + code 51
(UnknownClient).
- ❌ Tried four handle formats (contextKey upper/lower, storageSessionId
upper, ClientHandle as decimal string) — all return the same
UnknownClient.
- ❌ Adding the full priming chain (Stat.GetV ×2, Stat.GETHI ×2, UpdC3,
6× Stat.GetSystemParameter, AllowRenameTags, Trx.GetV, Stat.GetV,
Retr.GetV) — same result.
ITransactionServiceContract2 has no Validate/Register/Open op of its
own. The client-with-Trx registration must happen via some cross-
service side effect we haven't isolated.
Important takeaway: the wire-format mismatch is solved (contract method
names + parameter shapes match what the server expects). The remaining
gap is a single missing initialization step. Documented in
docs/plans/revision-write-path.md as concrete next-session steps.
178/178 tests pass (one new probe test added). Probe is gated on
HISTORIAN_HOST=localhost.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Direct HistorianAccess.AddNonStreamedValue (the 4-param overload that
bypasses HistorianDataValueList and goes straight to
HistorianClient.AddNonStreamedValueAsync) ALSO fails with 129
TagNotFoundInCache against SysTimeSec, even with validate=false.
So the cache check is inside the native C++ HistorianClient's
per-connection tag list — there's no managed-callable bypass.
Critical insight discovered: the SDK doesn't use the C++ HistorianClient
at all. It talks WCF directly. The cache gate that blocks the native
wrapper may not block a managed WCF client because the gate is enforced
by aahClientManaged, not by the WCF server.
This shifts the recommendation for any future D2 attempt from "wrap the
native API" (which is genuinely blocked) to "implement the wire path
directly on top of the existing ITransactionServiceContract methods and
test against the live server" (unverified but plausibly viable). The
harness can't help with that path — the wrapper itself is the blocker
we'd be bypassing.
177/177 tests still pass; harness gains --write-revision-direct flag
for further probing of the native-wrapper path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extended the harness with --write-revision-target-tag <name> (overrides
the value's TagKey via SQL lookup) and --write-revision-skip-validate
(passes false to AddNonStreamedValue's `validate` boolean). Added
--write-revision-commit gate so the harness validates without actually
calling SendValues by default — important when targeting system tags.
Probed SysTimeSec (wwTagKey=12, server-cache-resident system tag):
- AddNonStreamedValue: ErrorCode=TagNotFoundInCache (129) — same failure
- With validate=false: same failure (the cache check is intrinsic, not
gated by the boolean)
Conclusion: the gate is per-(client-session, tag), not per-server-cache.
Even tags the SERVER cache knows about are rejected because the LIBRARY
maintains a separate per-connection tag list that AddNonStreamedValue
checks. That list isn't populated by knowing the wwTagKey alone — it
needs whatever mechanism (RegisterTags2 / read flow side effect / IO
server registration) that we haven't reverse-engineered.
The revision-write path remains architecturally blocked for managed
clients. Plan doc updated with the SysTimeSec finding.
177/177 tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drove the revision-write flow via reflection in the native trace harness
(--write-revision-values) to see whether it bypasses the AddS2
architectural blocker. It doesn't.
Findings:
- HistorianAccess.CreateHistorianDataValueList(NonStreamedOriginal) succeeds
- HistorianDataValueList.NonStreamedValuesBegin() succeeds (batchID 0->1)
- HistorianDataValueList.AddNonStreamedValue(value, validate=true, out err)
FAILS with ErrorCode=TagNotFoundInCache (129) — same client-side
validation gate that blocks AddS2
- AddNonStreamedValuesEnd() returns void; SendValues() returns true
with Success because the list is empty (no value was ever added)
- No AddNonStreamValues* WCF calls reach the wire
Conclusion: the revision-write path requires the tag to be in the
library's runtime tag cache, which is only populated by configured
IO server / Application Server pipelines, not by HistorianAccess.AddTag.
This matches the architectural blocker documented for AddS2 and means
no public WriteRevisionsAsync / BeginRevisionAsync should be added to
the SDK — the path is unreachable for client-created sandbox tags.
The Wcf/Contracts/ITransactionServiceContract methods (AddNonStream-
ValuesBegin/AddNonStreamValues/AddNonStreamValuesEnd) remain declared
for completeness; no orchestrator or public surface is added.
The harness extension is preserved as a deterministic reproducer for
the blocker: re-run --write-revision-values to verify the gate any
time. docs/plans/revision-write-path.md updated with the empirical
finding plus the original plan retained as historical context.
177/177 tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plan: docs/plans/speculative-items-sweep.md (also covers parallelism +
findings).
Implemented:
- C3: HistorianTagDefinition.IntegralDivisor (default 1.0). Wire bytes
flip per the captured native serializer; live probe shows the server
stores IntegralDivisor on EngineeringUnit (shared) rather than per-tag,
so the value is accepted on the wire but doesn't visibly persist for
the test EU. Documented in the property's doc-comment.
- E: HistorianWcfCertOptionTests (5 tests) covering AllowUntrustedServer-
Certificate validator installation + ServerDnsIdentity propagation
through CreateEndpointAddress and CreateBindingPair.
Investigated + documented (deferred):
- D3: Discrete/String/Int1/Int8/UInt8 EnsT2 root cause — server-side
ValidationFailed: "Transaction validation failed". Native AddTag's
validator rejects non-analog types; not a wire-format issue. To unlock,
need to capture a working native flow via a different code path
(likely SMC's tag-import path or AddTagExtendedProperties carrying
type-specific metadata). Defer until a customer asks.
- D1: AddTagExtendedProperties feasibility — managed surface confirmed
(ArchestrA.HistorianAccess.AddTagExtendedProperties + WCF op
AddTagExtendedPropertyGroups). Cost estimated at 1-2 days of focused
RE work due to CTagExtendedPropertyGroup payload complexity. Defer.
- D2: AddRevisionValuesBegin/Value/End — sub-plan written at
docs/plans/revision-write-path.md with 5-step capture sequence,
workstream estimates, and risk register. Implementation deferred.
177/177 tests pass (was 172; +5 cert tests + 1 IntegralDivisor unit
test, harness probe results not committed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
With AllowUntrustedServerCertificate=true + ServerDnsIdentity="localhost",
all four representative read calls (ReadRawAsync, GetSystemParameterAsync,
BrowseTagNamesAsync, GetTagMetadataAsync) succeed from a Debian 13 client
against the Windows Historian over RemoteTcpCertificate with explicit
Windows credentials and NegotiateAuthentication via GSSAPI/NTLM.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When the server cert's CN/SAN doesn't match the URL host (typical for
installer-generated AVEVA Historian certs that claim DNS=localhost
even when reached over a LAN IP), WCF rejects the channel with
"Identity check failed for outgoing message". Set ServerDnsIdentity
to whatever the cert claims (often "localhost") to satisfy the check.
The endpoint address for the cert binding is constructed with a
DnsEndpointIdentity when the option is non-null.
Default null. Pairs with AllowUntrustedServerCertificate so a Linux
client can talk to a self-signed dev Historian over RemoteTcpCertificate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When true, the SDK's WCF channel factories accept the server's X.509
certificate without chain validation. Intended for connecting to
development / on-prem Historians whose /HistCert endpoint presents an
installer-generated self-signed cert that isn't in the local trust
store. Particularly relevant on Linux: .NET WCF on Linux does its own
X509Chain validation that doesn't honor the system CA bundle, so even
after `update-ca-certificates` succeeds the cert binding still rejects
the server. With this option set, custom certificate validator accepts
any cert and revocation checking is disabled.
Default false. Centralized in HistorianWcfClientCredentialsHelper.Configure
and applied at every ChannelFactory<T> instantiation in the WCF layer
(no-op when the option is false). 171/171 Windows tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Joseph Doherty