revision-write-path.md §"R3.1 CAPTURED" + roadmap R3.1/R3.2/one-glance now record the validated
finding: the historical write is HistoryService.AddStreamValues ("ON" storage-sample buffer, AddS2
"OS" family) + EnsureTags, not AddNonStreamValues/TransactionService. Includes the decoded 56-byte
"ON" buffer layout, the working priming/batch sequence, the tag-GUID keying, and that the D2 cache
gate does not block the primed 2023 R2 client. Remaining work to ship AddHistoricalValuesAsync is
the managed "ON" serializer (adapt HistorianEventWriteProtocol) + gRPC orchestrator wiring.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Records the feasibility-verified plan to capture the two remaining buffers (regular-tag
RegisterTags btTagInfos + AddNonStreamValues btInput):
- 2023 R2 aahClientManaged.dll is self-contained mixed-mode C++/CLI (only Windows + VC++
runtime native imports) — loadable in a net481 x64 process, no AVEVA install needed.
- gRPC routes through the managed Archestra.Historian.GrpcClient.dll, so the byte[] payloads
are capturable by IL-rewriting GrpcHistoryClient.RegisterTags / AddNonStreamValues (dnlib,
the instrument-wcf-writemessage pattern; rewrite a copy, never the originals).
- Connection is reflection-drivable: HistorianAccess.OpenConnection(HistorianConnectionArgs)
with ConnectionMode=HistorianConnectionMode.Historian (the gRPC mode), TcpPort=32565, cert.
- gRPC runtime deps (Grpc.Net.Client / Grpc.Core.Api / Google.Protobuf / ...) are present in
msi-extract/ArchestrA/Toolkits/Bin/x64.
- Risk: the C++ AddNonStreamedValue TagNotFoundInCache(129) gate (the 2020 D2 blocker) may
block btInput; mitigation = read the tag first. RegisterTags is emitted before that gate.
Build order documented (read-only connect -> IL-rewrite -> write capture -> serializer ->
commit+read-back -> AddHistoricalValuesAsync), each live step gated on per-action auth.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Live-probed StorageService.OpenStorageConnection against the 2023 R2 server over a
write-enabled (0x401) session. Every attempt — sweeping ConnectionMode (0x401/0x402/0x1),
StorageSessionId-in (Open2-GUID / empty), and FreeDiskSpace — returns the IDENTICAL native
error type=4 code=85 ("session not registered"), so it's a structural refusal, not a bad
field value.
Decode (two corroborating facts):
- Error 85 is the same code the event read returns before RegisterTags2 (see
HistorianWcfEventOrchestrator) — a generic "session not registered for this op".
- The 2023 R2 decompile shows OpenStorageConnection lives on a SEPARATE GrpcStorageClient
(the storage engine's SF/snapshot channel, own port + service identity); HistorianAccess
drives non-streamed writes through the native C++ HistorianClient, never this op.
So the roadmap's mapped "missing console session" step was wrong. The real non-streamed-write
precondition is the front-door HistoryService.RegisterTags (RTag2-family) for the target tag —
which is exactly why the R3.1 batch failed at AddNonStreamValues (no tag registered ->
StoreNonStreamValues had no route). Matches the original 2020-WCF D2 hypothesis.
Remaining (both need a native gRPC capture; do not guess bytes): the regular-tag RegisterTags
btTagInfos (only CM_EVENT's tag-GUID form is known) and the AddNonStreamValues btInput.
- HistorianGrpcStorageConnectionProbe + grpc-open-storage-connection CLI (opens nothing
persistent; CloseStorageConnection on success)
- corrected revision-write-path.md §R3.1 follow-up + hcal-roadmap R3.1/R3.2 rows
- gated regression test pinning the error-85 refusal
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Static decompile mining of the 2023 R2 client corroborates the live R3.1 error:
the AddNonStreamValues failure is the missing StorageService.OpenStorageConnection,
which creates exactly the \.\pipe\aahStorageEngine\console,sid(...) session named
in the server error. Mapped the full native sequence:
HistoryService.OpenConnection (have) -> StorageService.OpenStorageConnection
(MISSING) -> StorageService.RegisterTags -> AddNonStreamValuesBegin (works) ->
AddNonStreamValues(btInput) (fails - no console session) -> End(commit).
Two hard parts remain, each a live-production decode loop with no static shortcut:
(1) reproduce the 12-arg OpenStorageConnection handshake (several args inferred);
(2) decode the AddNonStreamValues btInput (C++-built, absent from decompiles; only
the 44-byte packed HISTORIAN_VALUE2 is known). Documented in revision-write-path.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
Empirically decoded the AddNonStreamValues btInput framing against the live 2023
R2 server (grpc-nonstream-decode command + ProbeNonStreamedBuffersAsync driver).
Every transaction rolled back (bCommit=false) — no data written.
Finding: the btInput is assembled native-C++-side (not in any decompile), so 6
evidence-based framings (44-54B, packed HISTORIAN_VALUE2 variants) were probed.
All 6 returned the IDENTICAL server error while an empty buffer returned a
different InvalidParameter — so non-empty buffers pass parameter validation into
CHistStorageConnection::StoreNonStreamValues, which routes to the
\.\pipe\aahStorageEngine\console pipe server-side. Identical-across-framings =>
the blocker is NOT the btInput layout but a missing storage-engine console
session / tag-registration precondition for the connection.
Next step (untested): StorageService.OpenStorageConnection + tag registration
(RegisterTags/AddTagidPairs/AddShardTagids) before AddNonStreamValues, then
commit + read-back on a sandbox tag. Documented in revision-write-path.md
(R3.1 decode section); raw artifact gitignored.
272 unit tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
The D2 storage-engine-pipe wall is WCF-transport-specific. On the 2023 R2 gRPC
front door, TransactionService is a first-class service AND the gateway to the
storage engine, so the Open2 storage-session GUID (uppercase) is accepted
directly as strHandle with no legacy pipe.
Live-verified against the real 2023 R2 server over a write-enabled (0x401) gRPC
session: AddNonStreamValuesBegin returns a real strTransactionId, and
AddNonStreamValuesEnd(bCommit=false) discards it cleanly (no data written). On
2020 WCF the same op returns UnknownClient(51) for every handle + priming chain.
- HistorianGrpcRevisionProbe + grpc-revision-probe CLI command + gated test
NonStreamedWriteTransaction_OverGrpc_BeginsAndDiscards (live pass).
- HistorianGrpcHandshake.OpenSession gains an optional connectionMode param
(default read-only 0x402; pass 0x401 for write-enabled) — non-breaking.
- Docs: revision-write-path.md "the wall is gone" section; roadmap M3 section,
R3.1-R3.3 rows, one-glance table, and status note updated honestly.
Not yet shipped: the AddNonStreamValues btInput VTQ buffer is uncaptured (never
guess wire bytes), so no value-commit is implemented. Scope is non-streamed
ORIGINAL backfill; revision EDITS (R4.2) remain pipe-only even on gRPC.
272 unit tests pass; sanitization scan clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
IL walk of the native wrapper:
HistorianAccess.AddRevisionValuesBegin (private, token 0x06006175)
-> CClientCommon.AddNonStreamValuesBegin
-> CClient.AddNonStreamValuesBegin (8-instr overload)
-> CClient.TransactionBegin
-> CHistStorageConnection.StartTransaction (token 0x06001FDD)
-> CStorageEngineConsoleClient.StartTransaction
CStorageEngineConsoleClient is built on STransactPipeClient2 +
SCrtMemFile — a shared-memory + named-pipe transport to
aaStorageEngine.exe, completely separate from WCF.
The WCF ITransactionServiceContract2.AddNonStreamValuesBegin2 op is a
server-side relay that requires a pre-existing storage-engine pipe
session for the client. Without that pipe session, the WCF relay returns
UnknownClient (51) — and there's no way to establish the pipe session
via WCF.
D2 is unimplementable as a pure-managed-WCF SDK. The native wrapper
itself depends on the C++ shared-memory channel; replicating that from
managed code would require implementing the storage-engine pipe
protocol, which is a major undertaking and out of scope.
The ITransactionServiceContract2 declaration in our contracts file
stays as documentation; no public API or orchestrator added.
HistorianWcfRevisionOrchestrator remains as an internal probe /
regression check — re-run the probe test if anyone believes the
architecture has changed.
178/178 tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tested hypothesis (1) from the plan: add RTag2(CM_EVENT tag id) to the
priming chain before AddNonStreamValuesBegin2.
Result:
- RTag2 itself succeeds: returns 25-byte response
(01000000000100000001EE39C30EDCDC010100000000000000), no error buffer.
- But AddNonStreamValuesBegin2 still fails with the same
04 33 00 00 00 (UnknownClient = 51) for all four handle formats.
So RTag2 on /Hist isn't the cross-service registration trigger we need
for /Trx. Plan doc updated with the result + next-session ordered
probes (try IStorageServiceContract, then IL walk CClientCommon,
then server-side decompile as last resort).
Probe orchestrator now also performs the RTag2 step so the test gives
one-shot diagnostic visibility of both calls.
178/178 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implemented HistorianWcfRevisionOrchestrator that talks WCF directly
to /Trx, bypassing the native wrapper entirely. Probes
AddNonStreamValuesBegin2 against the live local Historian and surfaces
what the server returns. Internal-only API; no public surface added —
the path isn't viable yet.
Findings (live test against localhost):
- ✅ The wire path is reachable. After moving from V1 (uint handle, no
errorBuffer) to V2 (string handle GUID, out errorBuffer), the server
recognizes the call (no ContractFilter mismatch, no exception).
- ✅ Server processes the call and returns a structured 5-byte error
buffer: 04 33 00 00 00 = type 4 (CustomError) + code 51
(UnknownClient).
- ❌ Tried four handle formats (contextKey upper/lower, storageSessionId
upper, ClientHandle as decimal string) — all return the same
UnknownClient.
- ❌ Adding the full priming chain (Stat.GetV ×2, Stat.GETHI ×2, UpdC3,
6× Stat.GetSystemParameter, AllowRenameTags, Trx.GetV, Stat.GetV,
Retr.GetV) — same result.
ITransactionServiceContract2 has no Validate/Register/Open op of its
own. The client-with-Trx registration must happen via some cross-
service side effect we haven't isolated.
Important takeaway: the wire-format mismatch is solved (contract method
names + parameter shapes match what the server expects). The remaining
gap is a single missing initialization step. Documented in
docs/plans/revision-write-path.md as concrete next-session steps.
178/178 tests pass (one new probe test added). Probe is gated on
HISTORIAN_HOST=localhost.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Direct HistorianAccess.AddNonStreamedValue (the 4-param overload that
bypasses HistorianDataValueList and goes straight to
HistorianClient.AddNonStreamedValueAsync) ALSO fails with 129
TagNotFoundInCache against SysTimeSec, even with validate=false.
So the cache check is inside the native C++ HistorianClient's
per-connection tag list — there's no managed-callable bypass.
Critical insight discovered: the SDK doesn't use the C++ HistorianClient
at all. It talks WCF directly. The cache gate that blocks the native
wrapper may not block a managed WCF client because the gate is enforced
by aahClientManaged, not by the WCF server.
This shifts the recommendation for any future D2 attempt from "wrap the
native API" (which is genuinely blocked) to "implement the wire path
directly on top of the existing ITransactionServiceContract methods and
test against the live server" (unverified but plausibly viable). The
harness can't help with that path — the wrapper itself is the blocker
we'd be bypassing.
177/177 tests still pass; harness gains --write-revision-direct flag
for further probing of the native-wrapper path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extended the harness with --write-revision-target-tag <name> (overrides
the value's TagKey via SQL lookup) and --write-revision-skip-validate
(passes false to AddNonStreamedValue's `validate` boolean). Added
--write-revision-commit gate so the harness validates without actually
calling SendValues by default — important when targeting system tags.
Probed SysTimeSec (wwTagKey=12, server-cache-resident system tag):
- AddNonStreamedValue: ErrorCode=TagNotFoundInCache (129) — same failure
- With validate=false: same failure (the cache check is intrinsic, not
gated by the boolean)
Conclusion: the gate is per-(client-session, tag), not per-server-cache.
Even tags the SERVER cache knows about are rejected because the LIBRARY
maintains a separate per-connection tag list that AddNonStreamedValue
checks. That list isn't populated by knowing the wwTagKey alone — it
needs whatever mechanism (RegisterTags2 / read flow side effect / IO
server registration) that we haven't reverse-engineered.
The revision-write path remains architecturally blocked for managed
clients. Plan doc updated with the SysTimeSec finding.
177/177 tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drove the revision-write flow via reflection in the native trace harness
(--write-revision-values) to see whether it bypasses the AddS2
architectural blocker. It doesn't.
Findings:
- HistorianAccess.CreateHistorianDataValueList(NonStreamedOriginal) succeeds
- HistorianDataValueList.NonStreamedValuesBegin() succeeds (batchID 0->1)
- HistorianDataValueList.AddNonStreamedValue(value, validate=true, out err)
FAILS with ErrorCode=TagNotFoundInCache (129) — same client-side
validation gate that blocks AddS2
- AddNonStreamedValuesEnd() returns void; SendValues() returns true
with Success because the list is empty (no value was ever added)
- No AddNonStreamValues* WCF calls reach the wire
Conclusion: the revision-write path requires the tag to be in the
library's runtime tag cache, which is only populated by configured
IO server / Application Server pipelines, not by HistorianAccess.AddTag.
This matches the architectural blocker documented for AddS2 and means
no public WriteRevisionsAsync / BeginRevisionAsync should be added to
the SDK — the path is unreachable for client-created sandbox tags.
The Wcf/Contracts/ITransactionServiceContract methods (AddNonStream-
ValuesBegin/AddNonStreamValues/AddNonStreamValuesEnd) remain declared
for completeness; no orchestrator or public surface is added.
The harness extension is preserved as a deterministic reproducer for
the blocker: re-run --write-revision-values to verify the gate any
time. docs/plans/revision-write-path.md updated with the empirical
finding plus the original plan retained as historical context.
177/177 tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plan: docs/plans/speculative-items-sweep.md (also covers parallelism +
findings).
Implemented:
- C3: HistorianTagDefinition.IntegralDivisor (default 1.0). Wire bytes
flip per the captured native serializer; live probe shows the server
stores IntegralDivisor on EngineeringUnit (shared) rather than per-tag,
so the value is accepted on the wire but doesn't visibly persist for
the test EU. Documented in the property's doc-comment.
- E: HistorianWcfCertOptionTests (5 tests) covering AllowUntrustedServer-
Certificate validator installation + ServerDnsIdentity propagation
through CreateEndpointAddress and CreateBindingPair.
Investigated + documented (deferred):
- D3: Discrete/String/Int1/Int8/UInt8 EnsT2 root cause — server-side
ValidationFailed: "Transaction validation failed". Native AddTag's
validator rejects non-analog types; not a wire-format issue. To unlock,
need to capture a working native flow via a different code path
(likely SMC's tag-import path or AddTagExtendedProperties carrying
type-specific metadata). Defer until a customer asks.
- D1: AddTagExtendedProperties feasibility — managed surface confirmed
(ArchestrA.HistorianAccess.AddTagExtendedProperties + WCF op
AddTagExtendedPropertyGroups). Cost estimated at 1-2 days of focused
RE work due to CTagExtendedPropertyGroup payload complexity. Defer.
- D2: AddRevisionValuesBegin/Value/End — sub-plan written at
docs/plans/revision-write-path.md with 5-step capture sequence,
workstream estimates, and risk register. Implementation deferred.
177/177 tests pass (was 172; +5 cert tests + 1 IntegralDivisor unit
test, harness probe results not committed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Joseph Doherty