histsdk/docs/plans/revision-write-path.md

# Plan: Revision-Write Path (`AddRevisionValuesBegin/Value/End`)

Status: **ARCHITECTURALLY BLOCKED — verified 2026-05-05.** Same root
cause as `AddS2`: client-side cache rejects values for tags that
weren't registered through a configured IO server / Application Server
pipeline. Documented below; implementation deferred until / unless that
prerequisite is removed.

## Empirical finding (2026-05-05)

The native trace harness was extended with `--write-revision-values` to
drive the revision flow:

1. `HistorianAccess.CreateHistorianDataValueList(HistorianDataCategory.NonStreamedOriginal)`
   succeeds — list is bound to the live `HistorianClient*` via
   `GetClient(ConnectionIndex.Process)`.
2. `HistorianDataValueList.NonStreamedValuesBegin()` succeeds — list
   batchID transitions 0 → 1.
3. `HistorianDataValueList.AddNonStreamedValue(value, validate=true, out error)`
   **fails** with `ErrorCode=TagNotFoundInCache (129)`,
   `ErrorDescription="error = 129 (Tag not found in cache)"` — the value
   is never added to the list (`Count` stays 0).
4. `HistorianDataValueList.AddNonStreamedValuesEnd()` returns void.
5. `HistorianAccess.SendValues(list, out error)` returns `true` with
   `ErrorCode=Success` — **but** no wire bytes left the client because
   the list is empty. (Inspecting captured WriteMessage stream confirms
   no `AddNonStreamValues*` Trx call appears.)

The validation that rejects the value is the same gate that blocks
`AddStreamedValue` (`AddS2`): the library's local tag cache only knows
about tags that were:

- Auto-populated from a configured IO server / Application Server pipeline, or
- Read via the existing read flow (which hits the cache as a side effect)

Tags created via `HistorianAccess.AddTag` populate `Runtime.dbo.Tag` but
are not added to the in-memory cache that AddStreamedValue /
AddNonStreamedValue consult. So writes from a managed client to a
client-created tag fail at the validation gate before any wire bytes
flow.

## Conclusion

The revision-write path **does not bypass the AddS2 blocker** — it
shares the same `TagNotFoundInCache` precondition.

### Follow-up probe (2026-05-05): SysTimeSec

To narrow the gate's scope, the harness was extended with
`--write-revision-target-tag <name>` (overrides the value's TagKey via
SQL lookup). Probed `SysTimeSec` (an auto-populated system tag whose
wwTagKey=12 is well-known in the runtime cache):

```
AddNonStreamedValue (TagKey=12 SysTimeSec):
  Result=False
  ErrorCode=TagNotFoundInCache
  ErrorDescription="error = 129 (Tag not found in cache)"
```

Same failure. Then probed with `--write-revision-skip-validate` to set
the `validate` boolean to false on `AddNonStreamedValue` — same
`TagNotFoundInCache` failure. The cache check is intrinsic to the
function, not gated by the `validate` parameter.

So the gate is **per-(client-session, tag)**, not per-(server-cache, tag):

- Server-side, `SysTimeSec` IS in the runtime cache (it's auto-populated).
- Client-side, the managed library has its own per-connection tag list
  that AddNonStreamedValue checks. That list is NOT populated by simply
  knowing the wwTagKey — something else (likely a `RegisterTags2` call
  during connection open, or the read flow as a side effect, or
  IO-server-driven registration) populates it.

The harness opens with `ReadOnly=false` for the write scenario, which
may suppress the read-flow side effect that would otherwise populate
the local cache. Without further RE on what populates the local cache,
no path is reachable for a managed client to write either streaming or
revision values.

### Cache gate is inside the native C++ HistorianClient

Followup probe (2026-05-05) tested the **direct** public overload
`HistorianAccess.AddNonStreamedValue(ConnectionIndex, HistorianDataValue, bool validate, ref error)`
which bypasses the `HistorianDataValueList` layer entirely and goes
straight to `HistorianClient.AddNonStreamedValueAsync` (a C++ method).

Even with `validate=false` and `TagKey=12 (SysTimeSec)`, the call
fails: `ErrorCode=TagNotFoundInCache (129)`.

So the gate isn't bypassed by:

1. Using a real wwTagKey from SQL
2. Targeting a server-cache-resident tag (SysTimeSec)
3. Setting `validate=false` on AddNonStreamedValue
4. Bypassing the `HistorianDataValueList` layer (calling the direct
   `HistorianAccess.AddNonStreamedValue` overload)

The check is inside the **native C++ `HistorianClient`'s per-connection
tag cache**, not in the managed wrapper. No managed-callable path exists
to populate that cache.

### Critical insight: the SDK doesn't use the C++ HistorianClient

The SDK's production code talks **WCF directly** — no C++ HistorianClient
instance, no per-connection local cache to gate against. The cache check
is enforced by the `aahClientManaged.dll` wrapper, not by the WCF server.

This means the SDK could **plausibly** implement the revision-write
path against the existing
`ITransactionServiceContract.AddNonStreamValuesBegin/AddNonStreamValues/AddNonStreamValuesEnd`
contract methods and have the server accept it directly — bypassing the
gate that blocks the native wrapper.

**Unverified assumptions:**

- The server may have its own cache requirement that mirrors the
  C++ wrapper's. If yes, the SDK is also blocked. If no, the SDK
  can write where the wrapper can't.
- The server may require `RTag2` (RegisterTags2) to be called per-tag
  before AddNonStreamValues — that's a known WCF op, already declared
  in `IHistoryServiceContract2`, used by the existing event flow. The
  SDK could call it.
- The server may require an IO-server-style registration that's not
  exposable over the WCF surface at all.

**Recommendation:** if D2 is ever pursued, do it as a **direct
WCF-level implementation in the SDK**, NOT as a wrapper over the
native HistorianAccess methods. The harness can no longer help (the
wrapper itself is gated). Test paths against the live server by
calling the contract methods directly and observing what the server
returns. If `AddNonStreamValues` succeeds without registration, the
path is implementable. If it fails with a server-side cache error,
try `RTag2` first. If it still fails, the path is genuinely blocked
server-side.

### SDK-direct probe results (2026-05-05)

`HistorianWcfRevisionOrchestrator` wires up the priming chain + a probe
of `ITransactionServiceContract2.AddNonStreamValuesBegin2(string handle, out string transactionId, out byte[] errorBuffer)`.
Live test against `localhost`:

- ✅ `OpenSucceeded: True` — Hist auth chain + Open2 still work end-to-end
- ✅ Trx channel opens, `Trx.GetV` returns interface version 2
- ✅ Wire path is recognized — server processes the call (no
  `ActionNotSupportedException` after switching from the abbreviated
  `AddNonS2B` to the default action name)
- ❌ Server returns structured error `04 33 00 00 00` =
  type 4 (CustomError) + code 51 (`UnknownClient`) for all four handle
  formats tried (contextKey GUID upper, storageSessionId upper, contextKey
  lower, ClientHandle as string)
- ❌ Adding the full priming chain (Stat.GetV ×2, Stat.GETHI ×2, UpdC3,
  6× Stat.GetSystemParameter, AllowRenameTags, Trx.GetV, Stat.GetV,
  Retr.GetV) doesn't change the result — Trx still rejects with
  `UnknownClient`

`ITransactionServiceContract2` exposes only `GetV`, `ForwardSnapshot*`,
and `AddNonStreamValues*`. There is no `ValidateClient`, `RegisterClient`,
or `Open` on Trx. So the client-with-Trx registration must happen via
some cross-service side effect we haven't identified.

**Important takeaway:** the wire path works at the WCF protocol layer.
We're past the "is this even reachable" question. The remaining gap is
finding what populates Trx's session table — likely:

1. `RTag2` on /Hist with a tag whose registration cascades to Trx
2. Some `IStorageServiceContract` op that we haven't tried
3. An aspect of the C++ HistorianClient initialization that doesn't
   show up in the IL we've inspected (e.g., the
   `aahClientCommon.CClientCommon` calls during InitializeProxy)

A future session that wants to push further should try (in order):

1. ✅ **DONE 2026-05-05.** Add `RTag2(CM_EVENT tag id)` to the priming
   chain — confirmed `RTag2` itself succeeds (returns 25-byte response),
   but `AddNonStreamValuesBegin2` still fails with `UnknownClient`.
   So RTag2 doesn't cascade client identity to Trx.
2. ⚠️ **OBVIATED 2026-05-05** by finding (3): `IStorageServiceContract`
   ops aren't the missing piece either, because the missing piece isn't
   on the WCF surface at all.
3. ✅ **DONE 2026-05-05** — IL walk of `aahClientCommon.CClientCommon.AddNonStreamValuesBegin`
   ↓ `aahClientCommon.CClient.AddNonStreamValuesBegin`
   ↓ `aahClientCommon.CClient.TransactionBegin`
   reveals the chain ultimately invokes
   **`aahClientCommon.CHistStorageConnection.StartTransaction`** (token
   `0x06001FDD`) which calls **`CStorageEngineConsoleClient.StartTransaction`**.
   `CStorageEngineConsoleClient` is built on `STransactPipeClient2` +
   `SCrtMemFile` — a **shared-memory + named-pipe** transport to the
   storage engine, completely separate from WCF.

### Definitive architectural conclusion (2026-05-05)

The revision-write path uses **two transports in tandem**:

1. WCF (`/Hist`, `/Retr`, `/Stat`, `/Trx`) — what our SDK speaks
2. **Shared-memory + named-pipe to `aaStorageEngine.exe`** — what
   `CStorageEngineConsoleClient` speaks; the SDK doesn't (and would be
   a major project to implement)

The WCF `ITransactionServiceContract2.AddNonStreamValuesBegin2` op we
were probing is a server-side relay that requires a pre-existing
storage-engine pipe session for the client. That session is established
via the pipe channel, not WCF. Without the pipe-side session, the WCF
relay returns `UnknownClient (51)` — and there's no way to establish
the pipe-side session via WCF.

**D2 is unimplementable as a pure-managed-WCF SDK.** The native wrapper
itself depends on the C++ shared-memory channel; to replicate that
behavior from a managed client would require implementing the whole
storage-engine pipe protocol, which is out of scope and probably
not viable without deeper RE of `aaStorageEngine.exe` itself.

The WCF `ITransactionServiceContract2` declaration in our contracts
file is left in place — it's correct as a contract — but no
orchestrator or public surface should be added on top of it. The
`HistorianWcfRevisionOrchestrator` in `src/AVEVA.Historian.Client/Wcf/`
remains as an internal probe / regression check; if anyone ever
believes the architecture has changed, re-run the probe test to
verify the gate still holds.

### Current state of the SDK-direct probe

`HistorianWcfRevisionOrchestrator.ProbeBeginAsync` does:

```
Open2 (write-enabled, 0x401)
  → priming (Stat.GetV ×2, Stat.GETHI ×2, UpdC3, 6× GetSystemParameter,
             AllowRenameTags, Trx.GetV, Stat.GetV, Retr.GetV)
  → RTag2(CM_EVENT tag id)            // succeeds
  → Trx.GetInterfaceVersion           // succeeds, returns version 2
  → Trx.AddNonStreamValuesBegin2 ×4   // all four handle formats fail with
                                      // 04 33 00 00 00 (UnknownClient 51)
```

The probe is committed as a gated test
(`HistorianWcfRevisionProbeTests.AddNonStreamValuesBegin_ProbeReturnsServerResult`)
that can be re-run any time to verify the gate is still where we think
it is, or to test future priming additions.

## Decision

Do **not** add public `WriteRevisionsAsync` / `BeginRevisionAsync` to
the SDK. The contract methods already exist in
`Wcf/Contracts/ITransactionServiceContract.cs`
(`AddNonStreamValuesBegin/AddNonStreamValues/AddNonStreamValuesEnd`)
for completeness, but the orchestrator and public surface stay absent.

Revisit if either of these changes:

1. AVEVA documents (or a customer demonstrates) a code path that
   bypasses the cache validation for client-created tags.
2. The SDK's mission expands to include data correction for tags that
   ARE in the runtime cache (i.e., tags managed by a real IO server),
   in which case the harness extension below provides a starting point.

## Harness diagnostic (preserved)

The `--write-revision-values` flag in
`tools/AVEVA.Historian.NativeTraceHarness/Program.cs` reproduces the
above failure deterministically. Re-run it any time to verify the
blocker still holds:

```powershell
dotnet run --no-build --project tools\AVEVA.Historian.NativeTraceHarness -- `
  --scenario write `
  --write-sandbox-tag RetestSdkWriteRevSandbox `
  --write-data-type Float `
  --write-skip-add-tag --write-skip-add-value `
  --write-revision-values
```

Look for the `AddNonStreamedValue` row's `ErrorCode` field in the JSON
output.

## Original plan (preserved for context if the blocker ever lifts)

## Context

The Historian's "revision write" path is the documented mechanism for
editing historized data after the fact (replaces the inferred
`ModifyData` / `DeleteData` use cases that don't exist as WCF ops). Native
managed surface (per Phase 1 findings of the write-commands plan):

| Public method | Token | Purpose |
|---|---|---|
| `ArchestrA.HistorianAccess.AddRevisionValuesBegin` | `0x06006175` | Open a revision-edit transaction |
| `ArchestrA.HistorianAccess.AddRevisionValue` | `0x06006176` | Append a value to the open transaction |
| `ArchestrA.HistorianAccess.AddRevisionValuesEnd` | `0x06006177` | Commit the transaction |
| `ArchestrA.HistorianAccess.AddRevisionValues` | `0x0600617F` | Single-shot variant |
| `ArchestrA.HistorianAccess.AddVersionedStreamedValue` | `0x0600616F` | Push one versioned value (related path) |

WCF surface is unknown — likely a new op group on `IHistoryServiceContract2`
or `IRetrievalServiceContract4` or a new contract.

## Goal

Public SDK API:

```csharp
public Task<HistorianRevisionTransaction> BeginRevisionAsync(string tag, CancellationToken ct);
// On the returned transaction:
public Task AddRevisionValueAsync(HistorianSampleEdit sample, CancellationToken ct);
public Task<bool> CommitAsync(CancellationToken ct);
// IDisposable / IAsyncDisposable for cancellation rollback if such a thing exists
```

Or a single batch convenience:

```csharp
public Task<bool> WriteRevisionsAsync(string tag, IReadOnlyList<HistorianSampleEdit> samples, CancellationToken ct);
```

The choice depends on the wire shape — if Begin/Value/End requires the
caller to maintain a server handle between calls, the disposable
transaction is necessary; if it's stateless, the batch convenience is fine.

## Workstreams

### A. Static analysis (1-2 hours)

- Inspect IL for the four managed public methods to identify the
  underlying `CHistoryConnectionWCF.*` calls and their server-side WCF
  contract methods.
- Add the contract methods to `Wcf/Contracts/IHistoryServiceContract2.cs`
  (or a new contract if appropriate) with `[OperationContract(Name = "...")]`
  + `[MessageParameter]` attributes once names are known.

### B. Native harness extension (2-3 hours)

- Add `--scenario revision-write` to the harness.
- Refer to existing `--scenario write` plumbing for the AddTag wrapper
  pattern.
- Sequence:
  1. Open connection (probably write-enabled mode `0x401`)
  2. AddTag for sandbox tag (re-uses existing harness flow)
  3. AddStreamedValue for the initial sample (currently blocked
     architecturally per Phase 2 findings — but may not be required if
     the revision path operates directly on the historian engine state)
  4. AddRevisionValuesBegin / AddRevisionValue × N / AddRevisionValuesEnd
  5. Read back via existing read path; verify the samples reflect the
     edits

### C. Wire capture (1 hour)

- Same `instrument-wcf-writemessage` + `instrument-wcf-readmessage`
  IL-rewrite tooling already used for EnsT2 / DelT.
- Capture both Begin/Value/End and the single-shot AddRevisionValues
  variant for byte-level diff.

### D. Decode + managed serializer (4-6 hours)

- Walk the captured InBuff bytes against the native serializer IL.
- The Begin payload likely seeds a server-side transaction handle that
  Value calls reference. Look for an `out`-returned handle in the Begin
  response.
- Value payload structure is likely similar to `AddS2`'s pBuf
  (uint16 version + uint32 sampleCount + N × {tagId, FILETIME, quality,
  typed value bytes}) but may include a per-sample revision/version field.

### E. Public API + tests (4-6 hours)

- New types: `HistorianSampleEdit` (sample + reason/version metadata),
  `HistorianRevisionTransaction` (disposable handle).
- Public methods on `HistorianClient` per the Goal section.
- Unit tests: golden-byte fixtures for Begin/Value/End/Commit payloads.
- Live integration tests: write a known sample, edit it via the
  revision path, read back and assert the new value appears.

## Risks

- **Server-cache prerequisite.** If the historian's revision path
  also requires the tag to be "live in the runtime cache" (the same
  blocker that killed `AddS2`), the entire path may be unimplementable
  for the same architectural reason.
- **State across calls.** Begin/Value/End may store transaction state
  on the server keyed by the WCF session GUID. WCF's session model
  needs to be configured to keep the same channel alive across all
  three calls — which is a different lifecycle from the existing
  one-call-per-channel pattern in the SDK orchestrators.
- **Concurrent edits.** Server may reject concurrent revision
  transactions on the same tag — needs probing.
- **Time bounds.** Revision likely respects the same `RealTimeWindow`
  / `FutureTimeThreshold` system parameters as `AddS2`. Out-of-window
  edits silently drop or error — needs probing.

## Success Criteria

- Public `BeginRevisionAsync` (or batch variant) live-verified against
  a sandbox tag created by `EnsureTagAsync`.
- Round-trip test: write initial value → revise it → read back → verify
  the revised value persists in `History` extension table via SQL.
- Golden-byte fixtures for Begin / Value / End / Commit captured against
  the sandbox tag.
- Decision documented for whether the `AddRevisionValues` single-shot
  variant is exposed in addition to the Begin/Value/End sequence.

## Dependencies

- Existing analog write surface (`EnsureTagAsync`) — done.
- `AddS2` is **not** a prerequisite; the revision path may be an
  independent code path that bypasses the runtime-cache gate. If it
  doesn't, this plan is blocked the same way `AddS2` is.

## Out of scope

- Editing event tags. Events come from AVEVA AnE; the SDK only reads
  them.
- Bulk schema changes. Forbidden over the wire per the Historian's
  architecture.

## Trigger to start

A customer-driven request, or a real need to expose historical data
correction in the SDK's API. Without one, this remains the most
substantive remaining write-path workstream but isn't worth the 1-2
days of focused work speculatively.