dohertj2/histsdk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

M2: implement SendEventAsync — event-send rides WCF AddS2, not the storage pipe

Browse Source 
Roadmap Milestone 2 (event sending). Capture disproved the assumption that event
delivery uses the non-WCF storage-engine pipe (which would block it like revision
writes): a native AddStreamedValue(HistorianEvent) leaves over WCF as AddS2
(IHistoryServiceContract2.AddStreamValues2). CM_EVENT is a built-in registered tag,
so the 129 TagNotFoundInCache gate that blocks AddS2 for user tags does not apply.

- R2.1: NativeTraceHarness "event-send" scenario + Capture-EventSend.ps1; two
  captures diffed to separate constant framing from value-dependent fields.
- R2.2: HistorianEventWriteProtocol serializes the AddS2 pBuf (storage sample buffer
  wrapping the event VTQ) — golden-byte tested. Decoded "OS" sig + length fields +
  CM_EVENT tag id + EventTime/ReceivedTime FILETIMEs + Opc 192 + 0x118D descriptor +
  event Id + Namespace + EventType + version 5 + typed property bag.
- R2.3/R2.4: HistorianWcfEventOrchestrator.SendEventAsync (Open2 event-mode 0x501 ->
  reuse CM_EVENT RTag2/EnsT2 -> AddStreamValues2) + HistorianClient.SendEventAsync.
- R2.5: gated live test; server accepts the AddS2 (success, empty error buffer).

Server requires delivered byte[].Length == declared packet length (uint32@0x04); the
native relies on the MDAS encoder adding a pad byte, so the SDK emits an explicit
trailing 0x00 (else AddS2 rejects with "CValuStream buffer size vs packet length
mismatch"). Original events only (RevisionVersion=0) with string properties; other
property types + revision/update/delete throw ProtocolEvidenceMissingException.

Caveat (documented): accepted events are not persisted on the local dev box; the
native client behaves identically (event ingestion pipeline inactive) — not an SDK
gap. 212 unit tests pass; 16/16 event tests pass live.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
This commit is contained in:
Joseph Doherty
2026-06-20 18:00:52 -04:00
parent 1a7519c803
commit f1e23a3a02
11 changed files with 920 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/hcal-roadmap.md
+27 -9
View File
@@ -125,16 +125,34 @@ event-filtered reads + rename all live-verified over gRPC.
*Goal: `SendEventAsync(HistorianEvent)`. Path fully mapped in histevents.md; one capture away.*
| ID | Work | Detail |
|---|---|---|
| R2.1 | Capture the event value blob | Instrument `CCommonArchestraEventValue::PackToVtq` (or dump the VTQ value bytes) on a live `AddStreamedValue(HistorianEvent)`; save sanitized fixture |
| R2.2 | `HistorianEventWriteProtocol` | Serialize header (`ReceivedTime, EventType, EventTime, Id, RevisionVersion, IsUpdate/IsDelete, Namespace`) + typed property bag — **inverse of `HistorianEventRowProtocol`** (reuse typemarkers `0x02/0x10/0x18/0x31/0x43/…`) |
| R2.3 | Event write orchestrator | Open **Event** connection (write mode) → register CM_EVENT (already have) → `Storage.AddStreamValues` with the event VTQ |
| R2.4 | Public API | `HistorianClient.SendEventAsync(HistorianEvent)` (+ `HistorianEvent` model: Type, EventTime, property bag) |
| R2.5 | Round-trip test | Send an event → read it back via `StartEventQuery` / `v_AlarmEventHistory2`; golden-byte on R2.2 |
> ✅ **DONE (2026-06-20) — `HistorianClient.SendEventAsync(HistorianEvent)` shipped and
> live-accepted over 2020 WCF.** The headline assumption — that event delivery would ride the
> non-WCF storage-engine pipe (and so be blocked like revision writes) — was **disproved by
> capture**: a native `AddStreamedValue(HistorianEvent)` leaves over WCF as **`AddS2`
> (`IHistoryServiceContract2.AddStreamValues2`)**. CM_EVENT is a built-in registered tag, so the
> `129 TagNotFoundInCache` gate that blocks `AddS2` for user tags does **not** apply to events.
> The full managed chain (Open2 event-mode **0x501** → CM_EVENT RTag2/EnsT2 → AddS2) is accepted
> by the server (`AddS2` returns success, empty error buffer). See the event-send field map under
> §"Event-send wire format" in `histevents.md` and `HistorianEventWriteProtocol`.
>
> ⚠️ **Persistence caveat (environment, not SDK):** on the local dev Historian, accepted events
> are **not persisted** to the queryable store (`v_AlarmEventHistory2` latest stays at the
> pre-test date; count only ages down). The **native** client exhibits the identical behaviour
> (its `AddS2` also returns success but nothing lands), so this is the box's event-ingestion
> pipeline not being active — not an SDK protocol gap. The SDK emits byte-equivalent `AddS2`
> (golden-tested). Full send→store→read-back round-trip awaits a Historian with an active event
> storage pipeline.
**Acceptance:** an event sent from histsdk appears in the historian and is read back with
matching Type + properties. **Now practical** — Historian is installed locally.
| ID | Work | Status |
|---|---|---|
| R2.1 | Capture the event value blob | ✅ `scripts/Capture-EventSend.ps1` (event-send harness scenario + instrument-wcf-{write,read}message); two captures diffed to separate constant framing from value fields. Decisive finding: event-send = WCF `AddS2`, not storage pipe. |
| R2.2 | `HistorianEventWriteProtocol` | ✅ Serializes the `AddS2` pBuf (storage sample buffer wrapping the event VTQ): "OS" sig + sampleCount + length fields + CM_EVENT tag id + EventTime FILETIME + OpcQuality + opaque descriptor + event Id + ReceivedTime FILETIME + Namespace + EventType + version + typed property bag (string props reuse the read parser's `0x43` encoding). Golden-byte test pins capture A. |
| R2.3 | Event write orchestrator | ✅ `HistorianWcfEventOrchestrator.SendEventAsync`: Open2 (0x501) → reuse CM_EVENT RTag2/EnsT2 registration → `AddStreamValues2(handle, pBuf, out err)` on the same /Hist channel + storage-session handle. |
| R2.4 | Public API | ✅ `HistorianClient.SendEventAsync(HistorianEvent)`. Original events only (RevisionVersion=0) with string-valued properties; other property types + revision/update/delete throw `ProtocolEvidenceMissingException` until captured. |
| R2.5 | Round-trip test | ✅ Golden-byte on R2.2 + gated live test `SendEventAsync_AgainstLocalHistorian_AcceptedByServer` (asserts server acceptance; SQL read-back best-effort given the persistence caveat). |
**Acceptance:** an event sent from histsdk is accepted by the historian over WCF with a
byte-correct `AddS2` (✅). Appears-and-reads-back is environment-gated on event persistence (see caveat).
---
docs/plans/histevents.md
+43 -4
View File
@@ -257,12 +257,51 @@ StartDateTime and EndDateTime"`.
---
## Event-send wire format ✅ (captured 2026-06-20)
`AddStreamedValue(HistorianEvent)` leaves the client over **WCF as `AddS2`
(`IHistoryServiceContract2.AddStreamValues2(string handle, byte[] pBuf, out errorBuffer)`)** —
**not** the storage-engine pipe. (Captured with the NativeTraceHarness `event-send` scenario +
instrument-wcf-writemessage; two events diffed to separate constant framing from value fields.)
CM_EVENT is a built-in registered tag, so the `129 TagNotFoundInCache` gate that blocks `AddS2`
for user tags doesn't apply. Open2 uses event connection-mode **0x501** (vs 0x402 read / 0x401
write). The `pBuf` is a storage **sample buffer** wrapping the event VTQ:
```
0x00 UInt16 0x534F "OS"
0x02 UInt16 sampleCount = 1
0x04 UInt32 packet length = delivered byte[].Length (= valueBlob.Length + 11)
0x08 UInt16 valueBlob.Length + 1
0x0A valueBlob:
+0x00 GUID CM_EVENT tag id (353b8145-5df0-4d46-a253-871aef49b321)
+0x10 Int64 EventTime FILETIME UTC (floored to ms — the VTQ timestamp)
+0x18 UInt16 OpcQuality = 192
+0x1A UInt16 192
+0x1C UInt16 0x118D (opaque CCommonArchestraEventValue descriptor — constant)
+0x1E GUID event Id
+0x2E Int64 ReceivedTime FILETIME UTC (full 100ns; native uses a unique/monotonic time)
+0x36 compact-ASCII string Namespace
+.... compact-ASCII string EventType
+.... UInt16 eventStructVersion = 5
+.... UInt16 propertyCount
+.... propertyCount × { compact-ASCII name; value: UInt8 typeMarker, UInt8 len, UInt8 status, len×bytes }
trailing: 1 pad byte so byte[].Length == packet length (UInt32 @0x04). The native relies on
the MDAS encoder adding this byte (non-deterministic value); the SDK emits 0x00.
```
Property values reuse `HistorianEventRowProtocol`'s typed encoding; only `0x43` (UTF-16 string)
is captured on the write side so far. Implemented in `HistorianEventWriteProtocol` (golden-tested)
and shipped as `HistorianClient.SendEventAsync`. **Server accepts the `AddS2` (success, empty
error buffer); on-box persistence is environment-gated (the native client behaves identically).**
## Open threads
- 🔶 Event value blob: field set/order known (§2a); **exact native framing** still needs one
`CCommonArchestraEventValue::PackToVtq` output capture + golden-byte test (mirror the read-side
`HistorianEventRowProtocol` reverse-engineering). Now feasible locally — the live historian is
installed, so the same instrument-and-capture approach used for reads applies.
- ⚠️ Event **persistence**: accepted `AddS2` events do not land in `v_AlarmEventHistory2` on the
local dev box (native client identical) — the event storage/ingestion pipeline isn't active
here. Needs a Historian with active event storage to verify send→store→read-back end-to-end.
- 🔶 Non-string event property write encodings (bool/int/filetime/guid/double) and revision/
update/delete event sends (RevisionVersion≠0, IsUpdate/IsDelete) are **not captured**; the SDK
throws `ProtocolEvidenceMissingException` for them. One capture each to extend.
- ❓ `EnqueueEventDataPacket.SerializedBytes` packet framing (header + N event VTQs batched).
- ✅ Database-mode store: server writer is `aahEventStorage.exe` loading the managed
`ArchestrAEvents.EventStorage.Contract` connection assembly; SQL retrieval surface is the