222eed9c02
Records the feasibility-verified plan to capture the two remaining buffers (regular-tag RegisterTags btTagInfos + AddNonStreamValues btInput): - 2023 R2 aahClientManaged.dll is self-contained mixed-mode C++/CLI (only Windows + VC++ runtime native imports) — loadable in a net481 x64 process, no AVEVA install needed. - gRPC routes through the managed Archestra.Historian.GrpcClient.dll, so the byte[] payloads are capturable by IL-rewriting GrpcHistoryClient.RegisterTags / AddNonStreamValues (dnlib, the instrument-wcf-writemessage pattern; rewrite a copy, never the originals). - Connection is reflection-drivable: HistorianAccess.OpenConnection(HistorianConnectionArgs) with ConnectionMode=HistorianConnectionMode.Historian (the gRPC mode), TcpPort=32565, cert. - gRPC runtime deps (Grpc.Net.Client / Grpc.Core.Api / Google.Protobuf / ...) are present in msi-extract/ArchestrA/Toolkits/Bin/x64. - Risk: the C++ AddNonStreamedValue TagNotFoundInCache(129) gate (the 2020 D2 blocker) may block btInput; mitigation = read the tag first. RegisterTags is emitted before that gate. Build order documented (read-only connect -> IL-rewrite -> write capture -> serializer -> commit+read-back -> AddHistoricalValuesAsync), each live step gated on per-action auth. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
634 lines
34 KiB
Markdown
634 lines
34 KiB
Markdown
# Plan: Revision-Write Path (`AddRevisionValuesBegin/Value/End`)
|
||
|
||
Status: **WCF: ARCHITECTURALLY BLOCKED (verified 2026-05-05).** **gRPC (2023 R2): the
|
||
non-streamed-original transaction is REACHABLE — Begin/End round-trip LIVE-VERIFIED 2026-06-21.**
|
||
Same root cause on WCF as `AddS2`: the `TransactionService` relay needs a pre-existing
|
||
storage-engine *pipe* session no WCF op can create. The 2023 R2 gRPC front door removes that wall
|
||
(see the §"2023 R2 gRPC — the wall is gone" section immediately below); the legacy WCF analysis is
|
||
preserved unchanged after it.
|
||
|
||
## 2023 R2 gRPC — the wall is gone (non-streamed original writes), LIVE-VERIFIED 2026-06-21
|
||
|
||
The whole D2 WCF blocker was: `ITransactionServiceContract2.AddNonStreamValuesBegin2` returns
|
||
`04 33 00 00 00` = `UnknownClient (51)` because the server-side Trx relay requires a storage-engine
|
||
pipe session (`STransactPipeClient2` → `aaStorageEngine.exe`) that no WCF op establishes. On the
|
||
**2023 R2 gRPC** transport that relay is replaced by a first-class `TransactionService` gRPC
|
||
service, and the gRPC server is itself the gateway to the storage engine — so the client passes the
|
||
**HistoryService Open2 storage-session GUID** straight in as `strHandle` and the transaction opens.
|
||
|
||
**Live probe (`grpc-revision-probe` CLI command / `HistorianGrpcRevisionProbe`):** against the real
|
||
2023 R2 server (History iface 12), over a **write-enabled** (`0x401`) gRPC session —
|
||
|
||
| step | result |
|
||
|---|---|
|
||
| `HistoryService.OpenConnection` (write-enabled `0x401`) | ✅ `OpenSucceeded`, client handle + storage GUID returned |
|
||
| `TransactionService.GetTransactionInterfaceVersion` | ✅ error 0, **version 2** |
|
||
| `TransactionService.AddNonStreamValuesBegin(strHandle = storage GUID **UPPERCASE**)` | ✅ **`BeginSucceeded`** — returns a real `strTransactionId` (e.g. `…-FE0A-4822-…`) on the **first** handle format tried |
|
||
| `TransactionService.AddNonStreamValuesEnd(handle, txId, bCommit=**false**)` | ✅ `EndDiscardSucceeded` — transaction discarded, **no data written** |
|
||
|
||
So the answer to the roadmap's open M3-over-gRPC question ("does the 2023 R2 gRPC front door expose
|
||
a non-streamed write that bypasses the legacy storage-engine pipe?") is **YES** — Begin/End is
|
||
reachable from the pure-managed SDK with no pipe, no native wrapper. The probe is committed as the
|
||
`grpc-revision-probe` CLI command + the gated test
|
||
`HistorianGrpcIntegrationTests.NonStreamedWriteTransaction_OverGrpc_BeginsAndDiscards`; re-run any
|
||
time to confirm the path is still open.
|
||
|
||
### Decompile basis (handle + op group)
|
||
|
||
`Archestra.Historian.GrpcClient.GrpcHistoryClient` drives the identical three-phase sequence
|
||
(`AddNonStreamValuesBegin(strHandle) → strTransactionId`; `AddNonStreamValues(strHandle,
|
||
strTransactionId, btInput)`; `AddNonStreamValuesEnd(strHandle, strTransactionId, bCommit)`), passing
|
||
the Open2 session GUID as `strHandle`. `btInput` is the **same opaque native VTQ buffer** the 2020
|
||
path uses. Proto: `src/AVEVA.Historian.Client/Grpc/Protos/TransactionService.proto`.
|
||
|
||
### What is proven vs. what remains (do NOT ship yet)
|
||
|
||
- ✅ **Proven:** the transaction lifecycle (Begin → End/rollback) is reachable over gRPC. The D2
|
||
architectural wall is specific to the WCF transport.
|
||
- ⛔ **Not yet captured:** the `AddNonStreamValues` **`btInput` VTQ buffer byte layout**. Per project
|
||
discipline ("never guess wire bytes; capture first") no value-commit is implemented. The next step
|
||
to actually *ship* M3 (`AddHistoricalValuesAsync`) is to capture the native gRPC `AddNonStreamValues`
|
||
`btInput` (or decode the `GrpcHistoryClient` serializer), build a golden-tested serializer, then do a
|
||
real `bCommit=true` write + SQL read-back against a sandbox tag created by `EnsureTagAsync`.
|
||
- 🔒 **Scope:** this is **non-streamed ORIGINAL backfill** (`HistorianDataCategory.NonStreamedOriginal`
|
||
→ `TransactionService.AddNonStreamValues*`). **Revision EDITS** (`AddRevisionValue(s)` /
|
||
`RevisionInsert*`, the R4.2 path) are NOT on the gRPC contract even in 2023 R2 — the capability
|
||
matrix confirms they still ride the storage-engine pipe. The gRPC unlock here is original backfill,
|
||
not after-the-fact edits.
|
||
|
||
### R3.1 decode probe (2026-06-21): `AddNonStreamValues` reaches the server-side storage-engine console pipe
|
||
|
||
The `btInput` VTQ buffer is assembled in native C++ (`SendNonStreamedValues(batchID)` → a vtable
|
||
call after values are pooled via native `AddNonStreamedValueAsync(&HISTORIAN_VALUE2)`) and is **not
|
||
visible in any decompile** — only the 44-byte packed `HISTORIAN_VALUE2` struct is (TagKey@0,
|
||
FILETIME@4, OpcQuality@20, Type@24=7 numeric, value@33, bVersioned@41, VersionStatus@42). So the
|
||
framing was probed empirically against the live server with `grpc-nonstream-decode` (every
|
||
transaction `bCommit=false` → rolled back, nothing written; tag key from `SysTimeSec`).
|
||
|
||
**Result — the failure is NOT a buffer-format problem:** six different framings (44–54 bytes:
|
||
count-prefixed packed struct, struct-only, version+count, OS-wrapped) all returned the **identical**
|
||
`AddNonStreamValues` error, while an empty buffer returned a *different* error (`04 01 00 00 00`,
|
||
InvalidParameter). The shared error is a nested `SError` whose detail strings are decisive:
|
||
|
||
```
|
||
aahClientAccessPoint::CHistStorageConnection::StoreNonStreamValues::StoreNonStreamValues
|
||
\\.\pipe\aahStorageEngine\console,sid(<server storage-engine session GUID>)
|
||
```
|
||
|
||
So non-empty buffers get **past parameter validation into `StoreNonStreamValues`**, which routes to
|
||
the **`aahStorageEngine` console named pipe** server-side (the same storage engine as D2 — but the
|
||
gRPC *server* now holds the pipe, not the client). Because the error is identical across every
|
||
framing, the blocker is **not** the `btInput` layout — it is a **missing storage-engine console
|
||
session / tag-registration precondition** for the connection.
|
||
|
||
**Required call sequence (mapped from the 2023 R2 decompile, corroborates the error above):** the
|
||
missing precondition is **`StorageService.OpenStorageConnection`** — it creates exactly the
|
||
`\\.\pipe\aahStorageEngine\console,sid(...)` console session named in the failure. The native
|
||
non-streamed write path is:
|
||
|
||
```
|
||
HistoryService.OpenConnection (✅ have it — the Open2 handshake)
|
||
→ StorageService.OpenStorageConnection (⛔ MISSING — opens the console sid session; SEPARATE
|
||
storage session, returns its own uint handle + new GUID)
|
||
→ StorageService.RegisterTags (register the tag→storage mapping for the session)
|
||
→ TransactionService.AddNonStreamValuesBegin (✅ works)
|
||
→ TransactionService.AddNonStreamValues(btInput) (⛔ currently fails here — no console session yet)
|
||
→ TransactionService.AddNonStreamValuesEnd(bCommit=true)
|
||
→ StorageService.CloseStorageConnection / HistoryService.CloseConnection
|
||
```
|
||
|
||
`OpenStorageConnection` (gRPC `StorageService`) takes 12 args — HostName, EnginePath
|
||
(`\\.\pipe\aahStorageEngine\console`), FreeDiskSpace, ProcessName, ProcessId, UserName, Password(+len),
|
||
ClientType, ClientVersion, ConnectionMode, ConnectionTimeout, StorageSessionId(in/out) — and returns a
|
||
**new** storage `Handle` (uint) + a **new** StorageSessionId GUID (distinct from the Open2 GUID).
|
||
|
||
**Two hard parts remain, each a separate live-production decode loop (no static shortcut):**
|
||
1. **Reproduce the `OpenStorageConnection` handshake** — several of the 12 args are only inferable from
|
||
the decompile (ProcessId, ClientType/Version, ConnectionMode, the password-bytes framing), so the
|
||
exact values must be confirmed against the live server.
|
||
2. **Decode the `AddNonStreamValues` `btInput`** — built in C++ (`SendNonStreamedValues` vtable call),
|
||
**absent from every decompile**; only the 44-byte packed `HISTORIAN_VALUE2` struct is known. Must be
|
||
decoded empirically once the console session exists (the batch-1 identical-error result could not
|
||
distinguish framings precisely *because* there was no session — with a session, framings should
|
||
diverge and the correct one becomes findable).
|
||
|
||
Raw decode artifact: `artifacts/reverse-engineering/grpc-nonstream-decode/batch1-decode.txt`
|
||
(gitignored). Probe command: `grpc-nonstream-decode`; driver:
|
||
`HistorianGrpcRevisionProbe.ProbeNonStreamedBuffersAsync` (candidate guess-bytes live in the RE tool,
|
||
not `src/`).
|
||
|
||
### R3.1 follow-up (2026-06-21): `OpenStorageConnection` is the WRONG precondition — error 85 = "session not registered"
|
||
|
||
The mapped sequence above named `StorageService.OpenStorageConnection` as the missing console-session
|
||
step. **A live probe (`grpc-open-storage-connection` CLI / `HistorianGrpcStorageConnectionProbe`)
|
||
disproved that.** Against the real 2023 R2 server, over a write-enabled (`0x401`) session, every
|
||
`OpenStorageConnection` attempt — sweeping `ConnectionMode` (0x401/0x402/0x1), `StorageSessionId`-in
|
||
(Open2-GUID-upper / empty), and `FreeDiskSpace` — returned the **identical** error
|
||
`84 55 00 00 00 …09 15 00 "OpenStorageConnection"` = **type 4 (CustomError, 0x80 detail flag), code
|
||
`0x55` = 85**, independent of all swept values. So it is a *structural* refusal, not a bad field.
|
||
|
||
**Decoding the refusal (two corroborating facts):**
|
||
1. **Error 85 is the generic "session not registered for this op" code.** The event read path hits the
|
||
*same* `type=4 code=85` from `GetNextEventQueryResultBuffer` when the session hasn't registered its
|
||
tag first (see `HistorianWcfEventOrchestrator` xmldoc) — the fix there is front-door `RegisterTags2`
|
||
(RTag2), NOT a storage connection.
|
||
2. **`OpenStorageConnection` is not a front-door client op.** In the 2023 R2 decompile it lives on a
|
||
**separate `GrpcStorageClient`** (`Archestra.Historian.GrpcClient`, `GrpcClientBase` with its own
|
||
`Initialize(target, port, …)` channel) and the managed `HistorianAccess` non-streamed write goes
|
||
through the **native C++ `<Module>.HistorianClient.AddNonStreamedValueAsync`**, never this gRPC op.
|
||
The `StorageService` proto is almost entirely snapshots / blocks / SF params / `SendSnapshot` —
|
||
it is the **storage engine's store-and-forward / snapshot interface** (`HistorianAccess`
|
||
documents `OpenStorageConnection`/`CloseStorageConnection` as the SF-snapshot *flush*), reached on
|
||
a distinct channel under a service identity. A normal Historian client never opens it on 32565.
|
||
|
||
**Corrected required sequence — the precondition is front-door tag registration, not a storage conn:**
|
||
|
||
```
|
||
HistoryService.OpenConnection (write-enabled 0x401) ✅ have it
|
||
→ HistoryService.RegisterTags(strHandle, btTagInfos = TARGET tag) ⛔ the real missing step
|
||
(front door, string handle — the RTag2 family; same op that subscribes the event session)
|
||
→ TransactionService.AddNonStreamValuesBegin ✅ works
|
||
→ TransactionService.AddNonStreamValues(btInput) ⛔ R3.1 batch failed here precisely
|
||
BECAUSE no tag was registered for the session (StoreNonStreamValues had no tag→storage route)
|
||
→ TransactionService.AddNonStreamValuesEnd(bCommit)
|
||
```
|
||
|
||
This matches the original 2020-WCF D2 hypothesis ("what populates the session's tag working set is
|
||
likely a `RegisterTags2` call") — the gRPC front door does expose that op (`HistoryService.RegisterTags`,
|
||
in our `HistoryService.proto`).
|
||
|
||
**Remaining blockers (both need a native gRPC capture — no static shortcut, do NOT guess bytes):**
|
||
1. **`HistoryService.RegisterTags` `btTagInfos` for a *regular analog* tag.** The only known RTag2
|
||
buffer is CM_EVENT's (a built-in tag identified by a well-known 16-byte *tag*-GUID,
|
||
`0x6750` v2 + count + GUID). Regular tags expose only a uint `tagKey` + a *type*-id GUID via
|
||
`GetTagInfo` (see `ParseTagInfoRecord`) — **no per-tag GUID**, so the regular-tag registration
|
||
framing (tagKey-based vs tag-GUID-based) is uncaptured.
|
||
2. **`AddNonStreamValues` `btInput`** — still C++-built and absent from every decompile (unchanged).
|
||
|
||
Both require capturing the **native 2023 R2 gRPC client** performing a non-streamed write (it would
|
||
emit the exact `RegisterTags` `btTagInfos` + `btInput`), or decoding the C++ serializer. Probe:
|
||
`grpc-open-storage-connection` (committed, regression-safe — it opens nothing persistent and
|
||
CloseStorageConnections on success). **Status: M3 transaction lifecycle proven; the insert precondition
|
||
is now correctly identified as front-door `RegisterTags` (NOT `OpenStorageConnection`); shipping
|
||
`AddHistoricalValuesAsync` is blocked on capturing the regular-tag `RegisterTags` `btTagInfos` +
|
||
the `AddNonStreamValues` `btInput`.**
|
||
|
||
### R3.1 capture plan (2026-06-21): drive the native 2023 R2 gRPC client + IL-rewrite the byte[] payloads
|
||
|
||
Feasibility verified end-to-end against `histsdk-2023r2-analysis/bin`:
|
||
|
||
- **Self-contained, loadable.** 2023 R2 `aahClientManaged.dll` is a 20 MB **mixed-mode C++/CLI**
|
||
assembly whose native imports are only Windows + VC++ runtime (`MSVCP140`/`VCRUNTIME140_1`) — **no
|
||
external AVEVA native dependency / no Historian install required** to load it in a `net481` x64
|
||
process. The native C++ `HistorianClient` (the `<Module>.HistorianClient.*` globals,
|
||
e.g. `AddNonStreamedValueAsync(client, &HISTORIAN_VALUE2, &SError)`) is compiled *into* it and is
|
||
what builds `btInput`; it then hands the `byte[]` to the **managed** gRPC client.
|
||
- **gRPC routes through managed code → IL-rewrite-able.** `Archestra.Historian.GrpcClient.dll`
|
||
(`Grpc.Net`-based) is pure managed; `GrpcHistoryClient` holds both `m_historyClient` and
|
||
`m_transactionClient`. Capture targets:
|
||
- `GrpcHistoryClient.RegisterTags(string handle, byte[] tagInfos, …)` → dump `tagInfos`
|
||
- `GrpcHistoryClient.AddNonStreamValues(string handle, string transactionId, byte[] inBuff, …)` → dump `inBuff`
|
||
Use the existing dnlib IL-rewrite tooling (`tools/AVEVA.Historian.ReverseInstrumentation` +
|
||
`instrument-wcf-writemessage` pattern), writing rewrites to a copy under
|
||
`docs/reverse-engineering/dnlib-write-copy/` — never touch `histsdk-2023r2-analysis/bin` originals.
|
||
- **gRPC runtime deps are available.** `Archestra.Historian.GrpcClient.dll` references `Grpc.Net.Client`,
|
||
`Grpc.Core.Api`, `Grpc.Net.Client.Web`, `Google.Protobuf`, etc. — the full set is present in
|
||
`histsdk-2023r2-analysis/msi-extract/ArchestrA/Toolkits/Bin/x64/` (alongside the 5 core DLLs in
|
||
`…/bin/`). Assemble all of them into the harness runtime dir so `Assembly.LoadFrom` + the sibling
|
||
resolver can satisfy the gRPC stack.
|
||
- **Driving the write (reflection, like `NativeTraceHarness`).** `ArchestrA.HistorianAccess.OpenConnection(HistorianConnectionArgs, out err)`
|
||
with `HistorianConnectionArgs { ServerName, TcpPort=32565, ConnectionMode=HistorianConnectionMode.Historian
|
||
(the 2023 R2 gRPC mode; `ClassicHistorian`=legacy), ConnectionType=Process, ReadOnly=false,
|
||
IntegratedSecurity/UserName/Password, AllowUnTrustedConnection=true, SecurityInfo=cert }`, then
|
||
`AddNonStreamedValue(ConnectionIndex.Process, HistorianDataValue, bVersioned:false, out err)`.
|
||
- **Cache-gate risk (the D2 blocker).** The C++ `AddNonStreamedValueAsync` has a per-connection
|
||
`TagNotFoundInCache (129)` gate that, in the 2020 D2 probe, rejected the value **before any bytes
|
||
left the client**. Mitigation to try: **read the target tag first** (populate the per-connection
|
||
cache) before `AddNonStreamedValue`. `RegisterTags` is emitted during registration *before* this
|
||
gate, so its `tagInfos` is capturable **even if** the gate still blocks `btInput`.
|
||
|
||
Build order (each live step = prod write, per-action auth): (1) `net481` x64 harness loads the 2023 R2
|
||
DLL + opens a **read-only** gRPC connection + reads the tag (proves load+connect, no write); (2)
|
||
IL-rewrite `Archestra.Historian.GrpcClient.dll`; (3) write-enabled run → capture `RegisterTags`
|
||
`tagInfos` (+ `btInput` if the gate passes); (4) build golden serializer(s) in `src/`; (5) real
|
||
`bCommit=true` write + SQL read-back on a sandbox tag → ship `AddHistoricalValuesAsync`.
|
||
|
||
---
|
||
|
||
## Legacy WCF analysis (preserved — still accurate for the 2020 WCF transport)
|
||
|
||
Status (WCF only): **ARCHITECTURALLY BLOCKED — verified 2026-05-05.** Same root
|
||
cause as `AddS2`: client-side cache rejects values for tags that
|
||
weren't registered through a configured IO server / Application Server
|
||
pipeline. Documented below; implementation deferred until / unless that
|
||
prerequisite is removed.
|
||
|
||
## Empirical finding (2026-05-05)
|
||
|
||
The native trace harness was extended with `--write-revision-values` to
|
||
drive the revision flow:
|
||
|
||
1. `HistorianAccess.CreateHistorianDataValueList(HistorianDataCategory.NonStreamedOriginal)`
|
||
succeeds — list is bound to the live `HistorianClient*` via
|
||
`GetClient(ConnectionIndex.Process)`.
|
||
2. `HistorianDataValueList.NonStreamedValuesBegin()` succeeds — list
|
||
batchID transitions 0 → 1.
|
||
3. `HistorianDataValueList.AddNonStreamedValue(value, validate=true, out error)`
|
||
**fails** with `ErrorCode=TagNotFoundInCache (129)`,
|
||
`ErrorDescription="error = 129 (Tag not found in cache)"` — the value
|
||
is never added to the list (`Count` stays 0).
|
||
4. `HistorianDataValueList.AddNonStreamedValuesEnd()` returns void.
|
||
5. `HistorianAccess.SendValues(list, out error)` returns `true` with
|
||
`ErrorCode=Success` — **but** no wire bytes left the client because
|
||
the list is empty. (Inspecting captured WriteMessage stream confirms
|
||
no `AddNonStreamValues*` Trx call appears.)
|
||
|
||
The validation that rejects the value is the same gate that blocks
|
||
`AddStreamedValue` (`AddS2`): the library's local tag cache only knows
|
||
about tags that were:
|
||
|
||
- Auto-populated from a configured IO server / Application Server pipeline, or
|
||
- Read via the existing read flow (which hits the cache as a side effect)
|
||
|
||
Tags created via `HistorianAccess.AddTag` populate `Runtime.dbo.Tag` but
|
||
are not added to the in-memory cache that AddStreamedValue /
|
||
AddNonStreamedValue consult. So writes from a managed client to a
|
||
client-created tag fail at the validation gate before any wire bytes
|
||
flow.
|
||
|
||
## Conclusion
|
||
|
||
The revision-write path **does not bypass the AddS2 blocker** — it
|
||
shares the same `TagNotFoundInCache` precondition.
|
||
|
||
### Follow-up probe (2026-05-05): SysTimeSec
|
||
|
||
To narrow the gate's scope, the harness was extended with
|
||
`--write-revision-target-tag <name>` (overrides the value's TagKey via
|
||
SQL lookup). Probed `SysTimeSec` (an auto-populated system tag whose
|
||
wwTagKey=12 is well-known in the runtime cache):
|
||
|
||
```
|
||
AddNonStreamedValue (TagKey=12 SysTimeSec):
|
||
Result=False
|
||
ErrorCode=TagNotFoundInCache
|
||
ErrorDescription="error = 129 (Tag not found in cache)"
|
||
```
|
||
|
||
Same failure. Then probed with `--write-revision-skip-validate` to set
|
||
the `validate` boolean to false on `AddNonStreamedValue` — same
|
||
`TagNotFoundInCache` failure. The cache check is intrinsic to the
|
||
function, not gated by the `validate` parameter.
|
||
|
||
So the gate is **per-(client-session, tag)**, not per-(server-cache, tag):
|
||
|
||
- Server-side, `SysTimeSec` IS in the runtime cache (it's auto-populated).
|
||
- Client-side, the managed library has its own per-connection tag list
|
||
that AddNonStreamedValue checks. That list is NOT populated by simply
|
||
knowing the wwTagKey — something else (likely a `RegisterTags2` call
|
||
during connection open, or the read flow as a side effect, or
|
||
IO-server-driven registration) populates it.
|
||
|
||
The harness opens with `ReadOnly=false` for the write scenario, which
|
||
may suppress the read-flow side effect that would otherwise populate
|
||
the local cache. Without further RE on what populates the local cache,
|
||
no path is reachable for a managed client to write either streaming or
|
||
revision values.
|
||
|
||
### Cache gate is inside the native C++ HistorianClient
|
||
|
||
Followup probe (2026-05-05) tested the **direct** public overload
|
||
`HistorianAccess.AddNonStreamedValue(ConnectionIndex, HistorianDataValue, bool validate, ref error)`
|
||
which bypasses the `HistorianDataValueList` layer entirely and goes
|
||
straight to `HistorianClient.AddNonStreamedValueAsync` (a C++ method).
|
||
|
||
Even with `validate=false` and `TagKey=12 (SysTimeSec)`, the call
|
||
fails: `ErrorCode=TagNotFoundInCache (129)`.
|
||
|
||
So the gate isn't bypassed by:
|
||
|
||
1. Using a real wwTagKey from SQL
|
||
2. Targeting a server-cache-resident tag (SysTimeSec)
|
||
3. Setting `validate=false` on AddNonStreamedValue
|
||
4. Bypassing the `HistorianDataValueList` layer (calling the direct
|
||
`HistorianAccess.AddNonStreamedValue` overload)
|
||
|
||
The check is inside the **native C++ `HistorianClient`'s per-connection
|
||
tag cache**, not in the managed wrapper. No managed-callable path exists
|
||
to populate that cache.
|
||
|
||
### Critical insight: the SDK doesn't use the C++ HistorianClient
|
||
|
||
The SDK's production code talks **WCF directly** — no C++ HistorianClient
|
||
instance, no per-connection local cache to gate against. The cache check
|
||
is enforced by the `aahClientManaged.dll` wrapper, not by the WCF server.
|
||
|
||
This means the SDK could **plausibly** implement the revision-write
|
||
path against the existing
|
||
`ITransactionServiceContract.AddNonStreamValuesBegin/AddNonStreamValues/AddNonStreamValuesEnd`
|
||
contract methods and have the server accept it directly — bypassing the
|
||
gate that blocks the native wrapper.
|
||
|
||
**Unverified assumptions:**
|
||
|
||
- The server may have its own cache requirement that mirrors the
|
||
C++ wrapper's. If yes, the SDK is also blocked. If no, the SDK
|
||
can write where the wrapper can't.
|
||
- The server may require `RTag2` (RegisterTags2) to be called per-tag
|
||
before AddNonStreamValues — that's a known WCF op, already declared
|
||
in `IHistoryServiceContract2`, used by the existing event flow. The
|
||
SDK could call it.
|
||
- The server may require an IO-server-style registration that's not
|
||
exposable over the WCF surface at all.
|
||
|
||
**Recommendation:** if D2 is ever pursued, do it as a **direct
|
||
WCF-level implementation in the SDK**, NOT as a wrapper over the
|
||
native HistorianAccess methods. The harness can no longer help (the
|
||
wrapper itself is gated). Test paths against the live server by
|
||
calling the contract methods directly and observing what the server
|
||
returns. If `AddNonStreamValues` succeeds without registration, the
|
||
path is implementable. If it fails with a server-side cache error,
|
||
try `RTag2` first. If it still fails, the path is genuinely blocked
|
||
server-side.
|
||
|
||
### SDK-direct probe results (2026-05-05)
|
||
|
||
`HistorianWcfRevisionOrchestrator` wires up the priming chain + a probe
|
||
of `ITransactionServiceContract2.AddNonStreamValuesBegin2(string handle, out string transactionId, out byte[] errorBuffer)`.
|
||
Live test against `localhost`:
|
||
|
||
- ✅ `OpenSucceeded: True` — Hist auth chain + Open2 still work end-to-end
|
||
- ✅ Trx channel opens, `Trx.GetV` returns interface version 2
|
||
- ✅ Wire path is recognized — server processes the call (no
|
||
`ActionNotSupportedException` after switching from the abbreviated
|
||
`AddNonS2B` to the default action name)
|
||
- ❌ Server returns structured error `04 33 00 00 00` =
|
||
type 4 (CustomError) + code 51 (`UnknownClient`) for all four handle
|
||
formats tried (contextKey GUID upper, storageSessionId upper, contextKey
|
||
lower, ClientHandle as string)
|
||
- ❌ Adding the full priming chain (Stat.GetV ×2, Stat.GETHI ×2, UpdC3,
|
||
6× Stat.GetSystemParameter, AllowRenameTags, Trx.GetV, Stat.GetV,
|
||
Retr.GetV) doesn't change the result — Trx still rejects with
|
||
`UnknownClient`
|
||
|
||
`ITransactionServiceContract2` exposes only `GetV`, `ForwardSnapshot*`,
|
||
and `AddNonStreamValues*`. There is no `ValidateClient`, `RegisterClient`,
|
||
or `Open` on Trx. So the client-with-Trx registration must happen via
|
||
some cross-service side effect we haven't identified.
|
||
|
||
**Important takeaway:** the wire path works at the WCF protocol layer.
|
||
We're past the "is this even reachable" question. The remaining gap is
|
||
finding what populates Trx's session table — likely:
|
||
|
||
1. `RTag2` on /Hist with a tag whose registration cascades to Trx
|
||
2. Some `IStorageServiceContract` op that we haven't tried
|
||
3. An aspect of the C++ HistorianClient initialization that doesn't
|
||
show up in the IL we've inspected (e.g., the
|
||
`aahClientCommon.CClientCommon` calls during InitializeProxy)
|
||
|
||
A future session that wants to push further should try (in order):
|
||
|
||
1. ✅ **DONE 2026-05-05.** Add `RTag2(CM_EVENT tag id)` to the priming
|
||
chain — confirmed `RTag2` itself succeeds (returns 25-byte response),
|
||
but `AddNonStreamValuesBegin2` still fails with `UnknownClient`.
|
||
So RTag2 doesn't cascade client identity to Trx.
|
||
2. ⚠️ **OBVIATED 2026-05-05** by finding (3): `IStorageServiceContract`
|
||
ops aren't the missing piece either, because the missing piece isn't
|
||
on the WCF surface at all.
|
||
3. ✅ **DONE 2026-05-05** — IL walk of `aahClientCommon.CClientCommon.AddNonStreamValuesBegin`
|
||
↓ `aahClientCommon.CClient.AddNonStreamValuesBegin`
|
||
↓ `aahClientCommon.CClient.TransactionBegin`
|
||
reveals the chain ultimately invokes
|
||
**`aahClientCommon.CHistStorageConnection.StartTransaction`** (token
|
||
`0x06001FDD`) which calls **`CStorageEngineConsoleClient.StartTransaction`**.
|
||
`CStorageEngineConsoleClient` is built on `STransactPipeClient2` +
|
||
`SCrtMemFile` — a **shared-memory + named-pipe** transport to the
|
||
storage engine, completely separate from WCF.
|
||
|
||
### Definitive architectural conclusion (2026-05-05)
|
||
|
||
The revision-write path uses **two transports in tandem**:
|
||
|
||
1. WCF (`/Hist`, `/Retr`, `/Stat`, `/Trx`) — what our SDK speaks
|
||
2. **Shared-memory + named-pipe to `aaStorageEngine.exe`** — what
|
||
`CStorageEngineConsoleClient` speaks; the SDK doesn't (and would be
|
||
a major project to implement)
|
||
|
||
The WCF `ITransactionServiceContract2.AddNonStreamValuesBegin2` op we
|
||
were probing is a server-side relay that requires a pre-existing
|
||
storage-engine pipe session for the client. That session is established
|
||
via the pipe channel, not WCF. Without the pipe-side session, the WCF
|
||
relay returns `UnknownClient (51)` — and there's no way to establish
|
||
the pipe-side session via WCF.
|
||
|
||
**D2 is unimplementable as a pure-managed-WCF SDK.** The native wrapper
|
||
itself depends on the C++ shared-memory channel; to replicate that
|
||
behavior from a managed client would require implementing the whole
|
||
storage-engine pipe protocol, which is out of scope and probably
|
||
not viable without deeper RE of `aaStorageEngine.exe` itself.
|
||
|
||
The WCF `ITransactionServiceContract2` declaration in our contracts
|
||
file is left in place — it's correct as a contract — but no
|
||
orchestrator or public surface should be added on top of it. The
|
||
`HistorianWcfRevisionOrchestrator` in `src/AVEVA.Historian.Client/Wcf/`
|
||
remains as an internal probe / regression check; if anyone ever
|
||
believes the architecture has changed, re-run the probe test to
|
||
verify the gate still holds.
|
||
|
||
### Current state of the SDK-direct probe
|
||
|
||
`HistorianWcfRevisionOrchestrator.ProbeBeginAsync` does:
|
||
|
||
```
|
||
Open2 (write-enabled, 0x401)
|
||
→ priming (Stat.GetV ×2, Stat.GETHI ×2, UpdC3, 6× GetSystemParameter,
|
||
AllowRenameTags, Trx.GetV, Stat.GetV, Retr.GetV)
|
||
→ RTag2(CM_EVENT tag id) // succeeds
|
||
→ Trx.GetInterfaceVersion // succeeds, returns version 2
|
||
→ Trx.AddNonStreamValuesBegin2 ×4 // all four handle formats fail with
|
||
// 04 33 00 00 00 (UnknownClient 51)
|
||
```
|
||
|
||
The probe is committed as a gated test
|
||
(`HistorianWcfRevisionProbeTests.AddNonStreamValuesBegin_ProbeReturnsServerResult`)
|
||
that can be re-run any time to verify the gate is still where we think
|
||
it is, or to test future priming additions.
|
||
|
||
## Decision
|
||
|
||
Do **not** add public `WriteRevisionsAsync` / `BeginRevisionAsync` to
|
||
the SDK. The contract methods already exist in
|
||
`Wcf/Contracts/ITransactionServiceContract.cs`
|
||
(`AddNonStreamValuesBegin/AddNonStreamValues/AddNonStreamValuesEnd`)
|
||
for completeness, but the orchestrator and public surface stay absent.
|
||
|
||
Revisit if either of these changes:
|
||
|
||
1. AVEVA documents (or a customer demonstrates) a code path that
|
||
bypasses the cache validation for client-created tags.
|
||
2. The SDK's mission expands to include data correction for tags that
|
||
ARE in the runtime cache (i.e., tags managed by a real IO server),
|
||
in which case the harness extension below provides a starting point.
|
||
|
||
## Harness diagnostic (preserved)
|
||
|
||
The `--write-revision-values` flag in
|
||
`tools/AVEVA.Historian.NativeTraceHarness/Program.cs` reproduces the
|
||
above failure deterministically. Re-run it any time to verify the
|
||
blocker still holds:
|
||
|
||
```powershell
|
||
dotnet run --no-build --project tools\AVEVA.Historian.NativeTraceHarness -- `
|
||
--scenario write `
|
||
--write-sandbox-tag RetestSdkWriteRevSandbox `
|
||
--write-data-type Float `
|
||
--write-skip-add-tag --write-skip-add-value `
|
||
--write-revision-values
|
||
```
|
||
|
||
Look for the `AddNonStreamedValue` row's `ErrorCode` field in the JSON
|
||
output.
|
||
|
||
## Original plan (preserved for context if the blocker ever lifts)
|
||
|
||
## Context
|
||
|
||
The Historian's "revision write" path is the documented mechanism for
|
||
editing historized data after the fact (replaces the inferred
|
||
`ModifyData` / `DeleteData` use cases that don't exist as WCF ops). Native
|
||
managed surface (per Phase 1 findings of the write-commands plan):
|
||
|
||
| Public method | Token | Purpose |
|
||
|---|---|---|
|
||
| `ArchestrA.HistorianAccess.AddRevisionValuesBegin` | `0x06006175` | Open a revision-edit transaction |
|
||
| `ArchestrA.HistorianAccess.AddRevisionValue` | `0x06006176` | Append a value to the open transaction |
|
||
| `ArchestrA.HistorianAccess.AddRevisionValuesEnd` | `0x06006177` | Commit the transaction |
|
||
| `ArchestrA.HistorianAccess.AddRevisionValues` | `0x0600617F` | Single-shot variant |
|
||
| `ArchestrA.HistorianAccess.AddVersionedStreamedValue` | `0x0600616F` | Push one versioned value (related path) |
|
||
|
||
WCF surface is unknown — likely a new op group on `IHistoryServiceContract2`
|
||
or `IRetrievalServiceContract4` or a new contract.
|
||
|
||
## Goal
|
||
|
||
Public SDK API:
|
||
|
||
```csharp
|
||
public Task<HistorianRevisionTransaction> BeginRevisionAsync(string tag, CancellationToken ct);
|
||
// On the returned transaction:
|
||
public Task AddRevisionValueAsync(HistorianSampleEdit sample, CancellationToken ct);
|
||
public Task<bool> CommitAsync(CancellationToken ct);
|
||
// IDisposable / IAsyncDisposable for cancellation rollback if such a thing exists
|
||
```
|
||
|
||
Or a single batch convenience:
|
||
|
||
```csharp
|
||
public Task<bool> WriteRevisionsAsync(string tag, IReadOnlyList<HistorianSampleEdit> samples, CancellationToken ct);
|
||
```
|
||
|
||
The choice depends on the wire shape — if Begin/Value/End requires the
|
||
caller to maintain a server handle between calls, the disposable
|
||
transaction is necessary; if it's stateless, the batch convenience is fine.
|
||
|
||
## Workstreams
|
||
|
||
### A. Static analysis (1-2 hours)
|
||
|
||
- Inspect IL for the four managed public methods to identify the
|
||
underlying `CHistoryConnectionWCF.*` calls and their server-side WCF
|
||
contract methods.
|
||
- Add the contract methods to `Wcf/Contracts/IHistoryServiceContract2.cs`
|
||
(or a new contract if appropriate) with `[OperationContract(Name = "...")]`
|
||
+ `[MessageParameter]` attributes once names are known.
|
||
|
||
### B. Native harness extension (2-3 hours)
|
||
|
||
- Add `--scenario revision-write` to the harness.
|
||
- Refer to existing `--scenario write` plumbing for the AddTag wrapper
|
||
pattern.
|
||
- Sequence:
|
||
1. Open connection (probably write-enabled mode `0x401`)
|
||
2. AddTag for sandbox tag (re-uses existing harness flow)
|
||
3. AddStreamedValue for the initial sample (currently blocked
|
||
architecturally per Phase 2 findings — but may not be required if
|
||
the revision path operates directly on the historian engine state)
|
||
4. AddRevisionValuesBegin / AddRevisionValue × N / AddRevisionValuesEnd
|
||
5. Read back via existing read path; verify the samples reflect the
|
||
edits
|
||
|
||
### C. Wire capture (1 hour)
|
||
|
||
- Same `instrument-wcf-writemessage` + `instrument-wcf-readmessage`
|
||
IL-rewrite tooling already used for EnsT2 / DelT.
|
||
- Capture both Begin/Value/End and the single-shot AddRevisionValues
|
||
variant for byte-level diff.
|
||
|
||
### D. Decode + managed serializer (4-6 hours)
|
||
|
||
- Walk the captured InBuff bytes against the native serializer IL.
|
||
- The Begin payload likely seeds a server-side transaction handle that
|
||
Value calls reference. Look for an `out`-returned handle in the Begin
|
||
response.
|
||
- Value payload structure is likely similar to `AddS2`'s pBuf
|
||
(uint16 version + uint32 sampleCount + N × {tagId, FILETIME, quality,
|
||
typed value bytes}) but may include a per-sample revision/version field.
|
||
|
||
### E. Public API + tests (4-6 hours)
|
||
|
||
- New types: `HistorianSampleEdit` (sample + reason/version metadata),
|
||
`HistorianRevisionTransaction` (disposable handle).
|
||
- Public methods on `HistorianClient` per the Goal section.
|
||
- Unit tests: golden-byte fixtures for Begin/Value/End/Commit payloads.
|
||
- Live integration tests: write a known sample, edit it via the
|
||
revision path, read back and assert the new value appears.
|
||
|
||
## Risks
|
||
|
||
- **Server-cache prerequisite.** If the historian's revision path
|
||
also requires the tag to be "live in the runtime cache" (the same
|
||
blocker that killed `AddS2`), the entire path may be unimplementable
|
||
for the same architectural reason.
|
||
- **State across calls.** Begin/Value/End may store transaction state
|
||
on the server keyed by the WCF session GUID. WCF's session model
|
||
needs to be configured to keep the same channel alive across all
|
||
three calls — which is a different lifecycle from the existing
|
||
one-call-per-channel pattern in the SDK orchestrators.
|
||
- **Concurrent edits.** Server may reject concurrent revision
|
||
transactions on the same tag — needs probing.
|
||
- **Time bounds.** Revision likely respects the same `RealTimeWindow`
|
||
/ `FutureTimeThreshold` system parameters as `AddS2`. Out-of-window
|
||
edits silently drop or error — needs probing.
|
||
|
||
## Success Criteria
|
||
|
||
- Public `BeginRevisionAsync` (or batch variant) live-verified against
|
||
a sandbox tag created by `EnsureTagAsync`.
|
||
- Round-trip test: write initial value → revise it → read back → verify
|
||
the revised value persists in `History` extension table via SQL.
|
||
- Golden-byte fixtures for Begin / Value / End / Commit captured against
|
||
the sandbox tag.
|
||
- Decision documented for whether the `AddRevisionValues` single-shot
|
||
variant is exposed in addition to the Begin/Value/End sequence.
|
||
|
||
## Dependencies
|
||
|
||
- Existing analog write surface (`EnsureTagAsync`) — done.
|
||
- `AddS2` is **not** a prerequisite; the revision path may be an
|
||
independent code path that bypasses the runtime-cache gate. If it
|
||
doesn't, this plan is blocked the same way `AddS2` is.
|
||
|
||
## Out of scope
|
||
|
||
- Editing event tags. Events come from AVEVA AnE; the SDK only reads
|
||
them.
|
||
- Bulk schema changes. Forbidden over the wire per the Historian's
|
||
architecture.
|
||
|
||
## Trigger to start
|
||
|
||
A customer-driven request, or a real need to expose historical data
|
||
correction in the SDK's API. Without one, this remains the most
|
||
substantive remaining write-path workstream but isn't worth the 1-2
|
||
days of focused work speculatively.
|