dohertj2/histsdk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

R1.3 GetServerTimeZoneAsync over gRPC (live-verified); R1.4 bounded out on gRPC

Browse Source 
Live-probed both R1.3 and R1.4 against a real 2023 R2 server over the gRPC
StatusService; implemented the one that carries an evidence-backed value.

R1.3 GetServerTimeZoneAsync — SHIPPED:
- StatusService.GetSystemTimeZoneName(uiHandle) returns the real server zone
  over RemoteGrpc (the 2020 WCF op is a client-side stub returning empty).
- HistorianGrpcStatusClient.GetSystemTimeZoneNameAsync -> dialect routing ->
  public HistorianClient.GetServerTimeZoneAsync. Non-gRPC transports fail
  closed with ProtocolEvidenceMissingException (no empty-string lie).
- Golden message-shape unit test + non-gRPC guardrail unit test + gated live
  test. 271 unit tests pass.

R1.4 GetHistorianInfoAsync (EventStorageMode) — bounded out on gRPC too:
- gRPC GetHistorianInfo is the same named-value query as 2020 WCF (only
  HistorianVersion resolves); EventStorageMode + 7 variants fail on both
  GetHistorianInfo and GetSystemParameter. The 518-byte struct is filled by a
  native vtable+648 HCAL call, not the gRPC op (per the 2023 R2 decompile), so
  the field is never on the wire. Not shipped on any transport. Closes the
  roadmap's open "build against a live 2023 R2 server" caveat.

Also correct the stale M3 roadmap section: D2 already proved
Transaction.AddNonStreamValues* rides the storage-engine pipe (STransactPipeClient2
-> aaStorageEngine), not WCF — same wall as R4.2 — so M3-over-WCF is blocked, not
"the path that is NOT the gated cache push".

Docs: hcal-roadmap.md, wcf-historian-info.md, wcf-status-localhost.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
This commit is contained in:
Joseph Doherty
2026-06-21 17:24:10 -04:00
parent 25aff409dc
commit 04ea0b9a1f
8 changed files with 199 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/hcal-roadmap.md
+50 -14
View File
@@ -88,6 +88,15 @@ HCAL replacement, built on the **2023 R2 gRPC transport**. Derived from
> 2023R2-gRPC-only (see `wcf-historian-info.md`). Net: the **reachable 2020-WCF M1 read surface is
> complete**; what remains is config *writes* (M1c — gated on an explicit user request) and the
> gRPC/2023R2-only items (R1.3 timezone, R1.4 EventStorageMode — need a live 2023 R2 server).
>
> **Update 2026-06-21 (live 2023 R2 gRPC probe — both closed):** **R1.3 SHIPPED on gRPC** —
> `GetServerTimeZoneAsync` returns a real zone ("Eastern Daylight Time") via
> `StatusService.GetSystemTimeZoneName`; non-gRPC path fails closed
> (`ProtocolEvidenceMissingException`). **R1.4 bounded out on gRPC too** — `GetHistorianInfo` is
> named-value-only on the gRPC wire as well, `EventStorageMode` resolves under no name on either
> `GetHistorianInfo` or `GetSystemParameter`, and the 518-byte struct is C++-HCAL-internal (filled
> via native vtable+648, not the gRPC op). So **no gRPC/2023R2-specific reads remain open** — the
> entire M1 read surface (2020 WCF + 2023 R2 gRPC) is now closed.
## Guiding principles
@@ -135,7 +144,7 @@ read/browse/status surface is Windows-free and the gRPC stack is the default pat
|---|---|---|---|
| ~~R1.1~~ | ~~`ExecuteSqlCommandAsync`~~ | `Retrieval.ExecuteSqlCommand` (`ExeC`+`GetR`) | ✅ **DONE (2026-06-20), live-verified.** `ExecuteSqlCommandAsync(sql)` → `HistorianSqlResult` (columns + typed rows). String-handle op via the uppercase storage GUID. Chain: `Retr.GetV` prime → `ExeC(handle, sql, option=0, ref queryHandle)` → `GetR` loop (note: `GetR` returns **false even on success** — the stream is in `pResultBuff` regardless; false = final page). `GetR`'s `pResultBuff` is an **NRBF-serialized `DataTable`** (`SerializationFormat.Xml`: members `XmlSchema` + `XmlDiffGram`). BinaryFormatter is gone from .NET 10, so it's decoded read-only with `System.Formats.Nrbf` + `XDocument` (no BinaryFormatter). Shipped: `HistorianSqlResult`/`HistorianSqlColumn`/`HistorianSqlExecuteOption`, `HistorianSqlResultProtocol`, `HistorianWcfSqlClient`, golden `WcfSqlResultProtocolTests`, gated live tests. See `docs/reverse-engineering/wcf-exec-sql.md`. |
| ~~R1.2~~ | ~~`GetRuntimeParameterAsync`~~ | `Status.GetRuntimeParameter` (`aa/Stat/GETRP`) | ✅ **DONE (2026-06-20), live-verified.** Captured (`scripts/Capture-RuntimeParam.ps1`): GETRP is a **`string`-handle** op (GETHI's shape), but reachable from the managed client using the Open2 storage-session GUID as an **uppercase** string handle (`ToString("D").ToUpperInvariant()`). Returns `HistorianVersion` = `20,0,000,000` live. pRequestBuff = `54 67 01 00` + uint nameCount + per-name(uint charCount + UTF-16); pResponseBuff = version + uint resultCount + CRetVariant(`0x43` VT_BSTR + uint16 len + uint16 charCount + UTF-16). Single string-valued param only (multi-name framing inferred, not captured). Shipped: `HistorianClient.GetRuntimeParameterAsync(name)`; golden `WcfRuntimeParameterProtocolTests`. **Note:** GETRP punching through the string-handle wall with the uppercase storage GUID is a strong lead that GETHI/ExeC may be a handle-*format* issue — see `wcf-string-handle-wall.md` §Update. |
| ~~R1.3~~ | ~~`GetServerTimeZoneAsync`~~ | `Status.GetSystemTimeZoneName` | ⛔ **gRPC/2023R2-only — re-confirmed 2026-06-21 from 3 angles.** (1) native `GetSystemTimeZoneName` is a stub (rc=0, empty) in the `GetServerTime` family; (2) Runtime DB has no timezone `SystemParameter` — the zone exists only as per-block `HistoryBlock.TimeZoneOffset`/`wwTimeZone` (DST-specific, SQL-only) + a `TimeZone` lookup table, `StorageShard.TimeZoneId`=NULL; (3) `GetSystemParameter` + `GETRP` return null/throw for every timezone candidate (only `HistorianVersion` works). The sole 2020 route is a SQL read via `ExecuteSqlCommand` (R1.1) — DST-specific, different mechanism. Build the real op only against a live 2023 R2 server. See `docs/reverse-engineering/wcf-status-localhost.md`. |
| ~~R1.3~~ | `GetServerTimeZoneAsync` | `Status.GetSystemTimeZoneName` | ✅ **DONE on gRPC (2026-06-21), LIVE-VERIFIED** against the real 2023 R2 server — returns `"Eastern Daylight Time"`. `HistorianClient.GetServerTimeZoneAsync` routes over `RemoteGrpc` (`HistorianGrpcStatusClient.GetSystemTimeZoneNameAsync`, `uiHandle`-in/string-out, no buffer). The 2020 WCF op stays a client-side stub (rc=0, empty), so the non-gRPC path **throws `ProtocolEvidenceMissingException`** (fail-closed) rather than return an empty string. Golden message-shape + non-gRPC guardrail unit tests + gated live test. (2020-only routes — per-block `HistoryBlock.TimeZoneOffset`, SQL via R1.1 — remain DST-specific and are not this op.) |
> ✅ **String-handle "wall" RESOLVED (2026-06-20) — it was a handle-FORMAT bug.** R1.4/R1.5/R1.6
> (and R1.1) take a **`string` GUID handle**; the earlier "code 1/51 blocked" verdict came from
@@ -153,7 +162,7 @@ read/browse/status surface is Windows-free and the gRPC stack is the default pat
### 1b. Bounded (decode one `bytes` payload; S–M each)
| ID | Capability | gRPC op | Payload to decode | Depends |
|---|---|---|---|---|
| ~~R1.4~~ | `GetHistorianInfoAsync` | `Status.GetHistorianInfo` (`GETHI`) | ⛔ **BOUNDED OUT on 2020 (re-confirmed 2026-06-21).** GETHI is a **named-value** query reachable via the uppercase storage GUID, but **only `HistorianVersion` resolves** — the full 518-byte `HISTORIAN_INFO` struct (incl. `EventStorageMode`@514) is the 2023R2 HCAL-native/gRPC model. `EventStorageMode` has **no 2020 representation at all**: not a `SystemParameter` (only `EventStorageDuration`/`EventStorageLogPath`), not a DB column, and `GETRP`/`GetSystemParameter` return null/throw for it. The only 2020-reachable field (version) is already shipped via `GetSystemParameterAsync`/`GetRuntimeParameterAsync`, so a struct API would be hollow + misleading. Build the real op only against a live 2023 R2 server. See `docs/reverse-engineering/wcf-status-localhost.md`. | uppercase string handle |
| ~~R1.4~~ | `GetHistorianInfoAsync` | `Status.GetHistorianInfo` (`GETHI`) | ⛔ **BOUNDED OUT — now confirmed on the 2023 R2 gRPC front door too (2026-06-21, live-probed).** The motivating field `EventStorageMode` is **not on the wire on either transport.** Live gRPC probe against the real 2023 R2 server: `GetHistorianInfo` is a **named-value** query exactly like 2020 WCF — only `HistorianVersion` resolves (→ `"23,1,000,000"` + `02 00 01 00` trailer); `EventStorageMode` + 7 name variants fail (`success=false`) on **both** `GetHistorianInfo` **and** `GetSystemParameter`. The 518-byte `HISTORIAN_INFO` struct (mode@514) is the **C++ HCAL in-memory model** (managed `HistorianAccess.GetHistorianInfo` fills it via a native **vtable+648** call, not the gRPC op — verified in the 2023 R2 decompile), derived outside the wire. The only wire-reachable field (version) is already shipped (`ProbeAsync`/`GetSystemParameterAsync`/`GetRuntimeParameterAsync`), so a struct API would be hollow + misleading. **Closes the prior "build against a live 2023 R2 server" caveat — done, and there is nothing to ship.** See `docs/reverse-engineering/wcf-historian-info.md`. | uppercase string handle |
| ~~R1.5~~ | Extended-property **read** | `Retrieval.GetTagExtendedPropertiesFromName` (`GetTepByNm`) | ✅ **DONE (2026-06-20), live-verified.** `GetTagExtendedPropertiesAsync(tag)` → name/value pairs. String-handle op via the uppercase storage GUID; name-based path (`GetTagExtendedPropertiesByName`, not the QTB-gated TagQuery path). Request `tagNames` = `uint count` + per-name(`uint charCount`+UTF-16); response = `uint tagCount` + per-tag(marker + compact-ASCII name + `uint propCount` + per-prop(marker + compact-ASCII name + `0x43` VT_BSTR value) + trailer). Sequence-paged. Shipped: `HistorianTagExtendedPropertyProtocol`, golden `WcfTagExtendedPropertyProtocolTests`, gated live test. See `docs/reverse-engineering/wcf-tag-extended-properties.md`. | uppercase string handle |
| ~~R1.6~~ | Localized-property **read** | (no op) | ⛔ **No distinct op on 2020 — collapses into R1.5.** There is no `GetTagLocalizedPropertiesFromName`/`GetTlpByNm` or `GetTagLocalizedPropertiesByName` in `current/aahClientManaged.dll`; the only "localized" surfaces are error-message/UI-text localization. Extended properties (R1.5) are the user-defined tag-property read surface. Closed, not throwing. | — |
| ~~R1.7~~ | Event **filters** | filter bytes in `Retrieval.StartEventQuery` | ✅ **DONE (2026-06-20), live-honored.** `ReadEventsAsync(start, end, HistorianEventFilter)`. The filter rides `StartEventQuery`'s `pRequestBuff` (captured via `EventQuery.AddEventFilter` + instrument-wcf-writemessage; Equal vs Contains diffed to isolate the op). Filter block: `ushort 0 + uint filterCount + uint condCount + uint nameLen + name(UTF-16) + uint 1 + ushort op + uint 1 + value(0x09-len-0x00 compact-ASCII) + byte 0`. **REAL, not inert** (a non-matching predicate returns 0 events; matching returns the subset). Single string-valued predicate only; multi-filter (OR) / multi-condition (AND via `AddEventFilterCondition`) framing not yet fully captured. See `HistorianEventFilter`, golden `WcfEventQueryProtocolTests`. | — |
@@ -212,14 +221,33 @@ byte-correct `AddS2` (✅). Appears-and-reads-back is environment-gated on event
*Goal: insert original historical VTQs (backfill), the path that is NOT the gated cache push.*
| ID | Work | gRPC op |
|---|---|---|
| R3.1 | Decode non-streamed VTQ packet | `Transaction.AddNonStreamValuesBegin/AddNonStreamValues/End` |
| R3.2 | `AddHistoricalValuesAsync` | batched begin→values→end |
| R3.3 | Ingest-permission validation | confirm the target accepts original-data insert (distinct from `AddS2` cache wall) |
> ⛔ **BLOCKED on 2020 WCF — re-confirmed by the D2 probe (2026-05-05), see
> [`revision-write-path.md`](revision-write-path.md).** The premise above ("the path that is NOT
> the gated cache push") was **disproved**: R3.1's op
> (`Transaction.AddNonStreamValuesBegin/AddNonStreamValues/End`) is the **same**
> `ITransactionServiceContract2.AddNonStreamValuesBegin2` D2 probed, and over WCF it returns
> `04 33 00 00 00` = `UnknownClient (51)` for every handle format **and** the full priming chain
> (Stat/Hist/Retr/Trx GetV + UpdC3 + 6× GetSystemParameter + RTag2). Root cause (IL-walk:
> `CClient.TransactionBegin` → `CHistStorageConnection.StartTransaction` →
> `CStorageEngineConsoleClient.StartTransaction`): the real transaction rides a **shared-memory +
> named-pipe** channel (`STransactPipeClient2` + `SCrtMemFile`) to `aaStorageEngine.exe`, separate
> from WCF. The WCF Trx op is a server-side **relay** that requires a pre-existing storage-engine
> pipe session, which no WCF op can establish. So **M3 over 2020 WCF is unimplementable as a
> pure-managed SDK** — same architectural wall as R4.2 (revisions) and the `AddS2` cache gate.
>
> **Only remaining lever:** the **2023 R2 gRPC front door** (HCAL-native, no legacy storage-engine
> pipe). Whether the gRPC services expose a non-streamed/revision write that bypasses the pipe is
> **untested** — it needs the live 2023 R2 server + a native gRPC capture of the write op, then
> decode/implement. Treat as on-demand (no current demand signal); the WCF path is closed.
**Acceptance:** historical points inserted and read back. Document clearly where this
differs from (gated) streaming sample writes.
| ID | Work | gRPC op | Status |
|---|---|---|---|
| R3.1 | Decode non-streamed VTQ packet | `Transaction.AddNonStreamValuesBegin/AddNonStreamValues/End` | ⛔ WCF blocked (storage-engine pipe — D2). gRPC: untested |
| R3.2 | `AddHistoricalValuesAsync` | batched begin→values→end | ⛔ gated on R3.1 |
| R3.3 | Ingest-permission validation | confirm the target accepts original-data insert (distinct from `AddS2` cache wall) | ⛔ proven to share the same gate, not distinct |
**Acceptance:** historical points inserted and read back. **WCF path closed (D2);** would require
the gRPC write path (live 2023 R2 server + capture) to reopen.
---
@@ -271,12 +299,20 @@ Recommended first sprint: **CW-1 + M0 (R0.1–R0.6)** → a fully Windows-free,
gRPC client at today's capability. Second sprint: **M1a + M2** (cheap wins + the headline
event-send). M3/M4 as demand dictates.
> **Status 2026-06-21:** sprints 1 + 2 are **complete** (M0 gRPC parity, the reachable M1 surface,
> and M2 event-send all shipped + live-verified; remaining M1 items are evidence-bounded-out). The
> reachable surface on the **available 2020 WCF infrastructure is exhausted** — every remaining
> roadmap item is now either (a) blocked by the storage-engine-pipe architecture (**M3-WCF**, R4.2),
> (b) **gRPC/2023R2-only** and needs the live 2023 R2 server for a native capture (R1.3 timezone,
> R1.4 EventStorageMode, M3/revisions over gRPC), or (c) a HARD deferred subsystem (M4). No further
> work lands without one of: a live-2023R2 capture session, or a customer-demand trigger.
## One-glance status
| Milestone | Tier | Effort | Value | When |
|---|---|---|---|---|
| M0 gRPC parity + capture tooling | foundation | M | unblocks everything, Windows-free | **now** |
| M1 cheap surface | TRIVIAL/BOUNDED | M–L | most remaining read/config | next |
| M2 event send | CAPTURE | S–M | headline write capability | next |
| M3 historical writes | BOUNDED | M | backfill | on demand |
| M4 SF / revisions / redundancy | HARD | L×N | parity completeness | defer |
| M0 gRPC parity + capture tooling | foundation | M | unblocks everything, Windows-free | ✅ **done** |
| M1 cheap surface | TRIVIAL/BOUNDED | M–L | most remaining read/config | ✅ **done** (reachable surface; rest bounded out) |
| M2 event send | CAPTURE | S–M | headline write capability | ✅ **done** |
| M3 historical writes | BOUNDED | M | backfill | ⛔ WCF blocked (D2); gRPC = on-demand + live 2023R2 |
| M4 SF / revisions / redundancy | HARD | L×N | parity completeness | defer (R4.2 = same pipe wall) |
docs/reverse-engineering/wcf-historian-info.md
+21 -8
View File
@@ -1,10 +1,13 @@
# GetHistorianInfo over 2020 WCF — GETHI is named-value-only (HCAL R1.4)
**Status: ⚠ Bounded out on 2020 WCF (2026-06-20).** `GetHistorianInfoAsync` is **not shipped**:
the one field that motivates it — `EventStorageMode` — is **not on the 2020 WCF wire**. The
version field that GETHI *does* return over WCF is already exposed (`ProbeAsync`,
`GetRuntimeParameterAsync("HistorianVersion")`), so there is nothing new to ship here without a
2023 R2 gRPC server. This parallels R1.3 (`GetServerTimeZone`), which is likewise 2023R2-only.
**Status: ⛔ Bounded out on BOTH 2020 WCF and 2023 R2 gRPC (2026-06-20; gRPC live-confirmed
2026-06-21).** `GetHistorianInfoAsync` is **not shipped on any transport**: the one field that
motivates it — `EventStorageMode` — is **not on the wire** on either transport (it lives only in
the C++ HCAL's in-memory 518-byte struct, filled via a native vtable+648 call — see the §gRPC
conclusion below). The version field GETHI *does* return is already exposed (`ProbeAsync`,
`GetRuntimeParameterAsync("HistorianVersion")`), so there is nothing new to ship. Note: R1.3
(`GetServerTimeZone`) — once paired with this as "2023R2-only" — **diverged**: it returns a real
value over gRPC and **shipped** 2026-06-21 (`GetServerTimeZoneAsync`); R1.4 did not.
## What the capture showed
@@ -56,9 +59,19 @@ replay cannot observe or reproduce.
- **2020 WCF:** `GetHistorianInfoAsync` would add nothing over existing surface (version only) and
could not report a real `EventStorageMode` — so it is intentionally **not shipped** (no hollow
`Unsupported`-returning API; project discipline: don't ship misleading behavior).
- **2023 R2 gRPC:** `Status.GetHistorianInfo` returns the full 518-byte `btHistorianInfo`; decode
version@0 + `EventStorageMode`@514 there. Build + verify against a live 2023 R2 server. The
`HistorianInfo` / `HistorianEventStorageMode` public types should land alongside that path.
- **2023 R2 gRPC — LIVE-PROBED 2026-06-21, also bounded out.** The earlier expectation that
`Status.GetHistorianInfo` returns the full 518-byte `btHistorianInfo` over gRPC was **wrong**. On
the real 2023 R2 server (History iface 12), the gRPC `GetHistorianInfo` is the
**same named-value query** as 2020 WCF: only `HistorianVersion` resolves (→ `"23,1,000,000"` +
`02 00 01 00` trailer); `EventStorageMode` and seven name variants return `success=false` on
**both** `GetHistorianInfo` and `GetSystemParameter`. The 518-byte struct is **not on the gRPC
wire** — the 2023 R2 decompile confirms managed `HistorianAccess.GetHistorianInfo` fills it via a
**native vtable+648 HCAL call** (`IClientCommon*` + offset 648), not the gRPC op, so
`EventStorageMode` is derived inside the C++ HCAL outside the wire on gRPC exactly as on WCF.
**Conclusion: `GetHistorianInfoAsync` is not shipped on any transport** (the only wire-reachable
field, version, is already exposed). No `HistorianInfo` / `HistorianEventStorageMode` public type
was added. Probe: the (now-deleted) `GrpcStatusInfoProbeTests`; raw dump under
`artifacts/reverse-engineering/grpc-status-info-probe/` (gitignored).
## Tooling kept as RE aids
docs/reverse-engineering/wcf-status-localhost.md
+16
View File
@@ -99,3 +99,19 @@ deliverable as server ops on 2020.** The only 2020 route to the timezone is a SQ
mechanism than the roadmap's `Status.GetSystemTimeZoneName`. `EventStorageMode` has no 2020
representation at all (it is a 2023 R2 event-storage-architecture field). Deliver both only against a
live 2023 R2 gRPC server.
## Resolution against the live 2023 R2 gRPC server (2026-06-21) — the two diverged
Both ops were taken to the real 2023 R2 box (History iface 12) over the gRPC
StatusService:
- **R1.3 `GetServerTimeZoneAsync` — SHIPPED.** `StatusService.GetSystemTimeZoneName(uiHandle)`
returns the real Windows zone name **"Eastern Daylight Time"** (the 2020 stub returned empty).
`HistorianClient.GetServerTimeZoneAsync` routes over `RemoteGrpc`; the non-gRPC transports throw
`ProtocolEvidenceMissingException` (fail-closed, no empty-string lie). Golden message-shape +
non-gRPC guardrail unit tests + gated live test.
- **R1.4 `GetHistorianInfoAsync` (`EventStorageMode`) — bounded out on gRPC too.** Over gRPC,
`GetHistorianInfo` is the **same named-value query** as 2020 WCF (only `HistorianVersion`
resolves); `EventStorageMode` + 7 variants fail on both `GetHistorianInfo` and
`GetSystemParameter`. The 518-byte struct is C++-HCAL-internal (native vtable+648), not on the
wire. Not shipped on any transport. See `wcf-historian-info.md`.