M3 R3.1: document the captured + validated AddStreamValues "ON" write path

revision-write-path.md §"R3.1 CAPTURED" + roadmap R3.1/R3.2/one-glance now record the validated
finding: the historical write is HistoryService.AddStreamValues ("ON" storage-sample buffer, AddS2
"OS" family) + EnsureTags, not AddNonStreamValues/TransactionService. Includes the decoded 56-byte
"ON" buffer layout, the working priming/batch sequence, the tag-GUID keying, and that the D2 cache
gate does not block the primed 2023 R2 client. Remaining work to ship AddHistoricalValuesAsync is
the managed "ON" serializer (adapt HistorianEventWriteProtocol) + gRPC orchestrator wiring.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
This commit is contained in:
Joseph Doherty
2026-06-21 21:04:17 -04:00
parent 9bcfffb365
commit 0e78d638d0
2 changed files with 40 additions and 3 deletions
+3 -3
View File
@@ -254,8 +254,8 @@ byte-correct `AddS2` (✅). Appears-and-reads-back is environment-gated on event
| ID | Work | gRPC op | Status |
|---|---|---|---|
| R3.1 | Decode non-streamed VTQ packet | `Transaction.AddNonStreamValuesBegin/AddNonStreamValues/End` | 🟡 **gRPC Begin/End LIVE-VERIFIED 2026-06-21; precondition CORRECTED 2026-06-21** (WCF still blocked — D2). The earlier "missing console session" step was **disproved live**: `StorageService.OpenStorageConnection` returns `type=4 code=85` ("session not registered") for every param — it's the storage engine's SF/snapshot channel (separate `GrpcStorageClient`/service identity), not a front-door client op. The real precondition is **front-door `HistoryService.RegisterTags`** (RTag2-family) for the target tag — the R3.1 batch failed at `AddNonStreamValues` *because the tag wasn't registered*. Remaining: capture the regular-tag `RegisterTags` `btTagInfos` (only CM_EVENT's tag-GUID form is known) + the `AddNonStreamValues` `btInput`. See [`revision-write-path.md`](revision-write-path.md) §R3.1 follow-up. |
| R3.2 | `AddHistoricalValuesAsync` | batched begin→values→end | 🟡 architecturally unblocked; precondition now correctly identified (front-door `RegisterTags`, not `OpenStorageConnection`); needs a native gRPC capture of the regular-tag `RegisterTags` `btTagInfos` + the `AddNonStreamValues` `btInput`, then a real `bCommit=true` write/read-back |
| R3.1 | Decode non-streamed VTQ packet | `History.AddStreamValues` ("ON" buffer) + `EnsureTags` | **CAPTURED + VALIDATED 2026-06-21.** Drove the native 2023 R2 client through a committed historical write (sandbox tag) with the IL-rewritten gRPC client dumping every `byte[]`; the value **read back over gRPC**. The path is **NOT** `AddNonStreamValues`/TransactionService — it's **`HistoryService.AddStreamValues`** with an **"ON" storage-sample buffer** (AddS2 "OS" family) + `EnsureTags`. Buffer decoded: `"ON"(0x4E4F) + u16 count + u32 totalLen + u16 payloadLen + 16B tag GUID + FILETIME + u16 quality + u32 type + FILETIME + 8B double`. D2 cache gate does NOT block the primed 2023 R2 client. See [`revision-write-path.md`](revision-write-path.md) §"R3.1 CAPTURED". |
| R3.2 | `AddHistoricalValuesAsync` | `History.AddStreamValues` ("ON") + `EnsureTags` | 🟡 **buffers captured + validated**; remaining: build the managed "ON" serializer in `src/` (adapt `HistorianEventWriteProtocol` "OS"), resolve tag GUID, reuse `EnsureTagAsync` CTagMetadata, wire `AddStreamValues` over the gRPC orchestrator, golden-test, then ship |
| R3.3 | Ingest-permission validation | confirm the target accepts original-data insert (distinct from `AddS2` cache wall) | ✅ **distinct on gRPC** — Begin succeeded against a real write-enabled session (the WCF/native cache gate does not apply here) |
**Acceptance:** historical points inserted and read back. **WCF path closed (D2).** gRPC path:
@@ -330,5 +330,5 @@ event-send). M3/M4 as demand dictates.
| M0 gRPC parity + capture tooling | foundation | M | unblocks everything, Windows-free | ✅ **done** |
| M1 cheap surface | TRIVIAL/BOUNDED | ML | most remaining read/config | ✅ **done** (reachable surface; rest bounded out) |
| M2 event send | CAPTURE | SM | headline write capability | ✅ **done** |
| M3 historical writes | BOUNDED | M | backfill | 🟡 **gRPC Begin/End live-verified (2026-06-21); precondition corrected** — front-door `HistoryService.RegisterTags` (not `OpenStorageConnection`, which is the SF-channel dead end / error 85). WCF blocked (D2). Follow-up: capture regular-tag `RegisterTags` `btTagInfos` + `AddNonStreamValues` `btInput` → commit+read-back |
| M3 historical writes | BOUNDED | M | backfill | 🟡 **wire path CAPTURED + VALIDATED (2026-06-21)** — native historical write = `HistoryService.AddStreamValues` ("ON" buffer) + `EnsureTags` (NOT AddNonStreamValues/Transaction; NOT OpenStorageConnection). Committed write read back over gRPC. Remaining: build the "ON" serializer in `src/` + ship `AddHistoricalValuesAsync`. WCF still blocked (D2) |
| M4 SF / revisions / redundancy | HARD | L×N | parity completeness | defer (R4.2 = same pipe wall) |
+37
View File
@@ -213,6 +213,43 @@ IL-rewrite `Archestra.Historian.GrpcClient.dll`; (3) write-enabled run → captu
`tagInfos` (+ `btInput` if the gate passes); (4) build golden serializer(s) in `src/`; (5) real
`bCommit=true` write + SQL read-back on a sandbox tag → ship `AddHistoricalValuesAsync`.
### R3.1 CAPTURED + VALIDATED (2026-06-21): the write rides `HistoryService.AddStreamValues` ("ON" buffer)
The capture ran end-to-end against the live server (`AVEVA.Historian.Grpc2023CaptureHarness`,
`capture-write` scenario, sandbox tag created by the harness, IL-rewritten `GrpcClient` dumping every
`byte[]`). The committed write **persisted and read back over gRPC** (SDK `ReadRawAsync` returned the
sample) — fully validated.
**The roadmap's assumption was wrong.** The native non-streamed (historical backfill) write does **not**
use `AddNonStreamValues` / the TransactionService at all. The native `HistorianAccess.AddNonStreamedValue
→ SendValues` routes over gRPC as **`HistoryService.AddStreamValues`** carrying an **"ON"
storage-sample buffer** (structurally the AddS2 **"OS"** family — same serializer pattern the SDK already
has in `HistorianEventWriteProtocol`), preceded by **`EnsureTags`** to register the tag:
```
EnsureTags.tagInfos (144B) = the analog CTagMetadata the SDK's EnsureTagAsync already builds
(0x4E marker … fe 00 trailer)
AddStreamValues.values (56B) = "ON" (0x4E4F) + u16 sampleCount(1) + u32 totalLen(56)
+ u16 payloadLen(46) + 16B tag GUID + FILETIME(sample)
+ u16 OpcQuality(192=Good) + u32 type/descriptor
+ FILETIME(received/version) + 8B double value
```
The full priming/write sequence that works from the native client (write-enabled session): `OpenConnection`
`UpdateClientStatus` ×N → `EnsureTags``GetTagInfosFromName` (resolve identity) → `AddStreamValues`
("ON" buffer). Notes: (a) the **D2 cache gate (err 129) does NOT block** the primed 2023 R2 client —
`AddNonStreamedValue` returned success once the session was primed (via `AddTag`/`GetTagInfoByName`) and
the server had assigned the tag key; (b) the value is keyed by a **16-byte tag GUID**, not the uint
`tagKey` (so the SDK serializer needs the tag's GUID, available from EnsureTags/GetTagInfo, not just
`HistorianTagMetadata.Key`); (c) batch lifecycle is `NonStreamedValuesBegin → AddNonStreamedValue →
SendValues → AddNonStreamedValuesEnd` (End-before-Send returns err 160 InvalidBatchId).
**Remaining to ship `AddHistoricalValuesAsync`:** build the managed "ON" `AddStreamValues` serializer in
`src/` (adapt `HistorianEventWriteProtocol`'s "OS" builder), resolve the tag GUID, reuse the existing
`EnsureTagAsync` CTagMetadata, wire `HistoryService.AddStreamValues` over the gRPC orchestrator, golden-test
the buffer, then a real write + read-back on a sandbox tag. Capture artifacts (gitignored):
`artifacts/reverse-engineering/grpc-nonstream-capture/captureB4.ndjson`.
---
## Legacy WCF analysis (preserved — still accurate for the 2020 WCF transport)