histsdk/docs/plans/hcal-capability-matrix.md

HCAL → modern-.NET reimplementation — capability matrix

Feasibility map for a clean managed-.NET client that replaces the AVEVA Historian SDK (aahClientManaged / HCAL). Grounded in: the real ArchestrA.HistorianAccess public surface (aahClientManaged.xml), the recovered 2023 R2 gRPC contract, the existing histsdk reimplementation, and the event/storage analysis in histevents.md.

Legend

Status (histsdk today) — ✅ implemented + live-verified · 🟗 partial · ⬜ not yet

Feasibility tier

Tier	Meaning	Effort
DONE	already working in histsdk	0
TRIVIAL	gRPC op known, payload already decoded or empty	XS (hrs)
CAPTURE	one instrument-and-capture of a native payload, then serialize + golden-byte test	S (days)
BOUNDED	gRPC op exists; decode one proprietary bytes payload	S–M
HARD	whole subsystem to reimplement	L (weeks)
GATED	blocked server-side — client effort doesn't unblock it	n/a

Effort = incremental work on top of histsdk's existing infrastructure (auth chain, transport, frame/byte primitives, test harness). All non-DONE items assume the gRPC transport as the foundation (clean protobuf envelope; only the inner byte blob needs RE).

---

1. Connection & session

Capability	HCAL API	2023 R2 gRPC op	histsdk	Tier	Notes
Probe / version	TestConnection, GetV	*Service.GetInterfaceVersion	✅	DONE
Open connection (Process)	OpenConnection	History.OpenConnection (+ ExchangeKey auth)	✅	DONE	full auth chain works
Open connection (Event)	OpenConnection (Event type)	History.OpenConnection event mode	🟗	TRIVIAL	read path already opens it; flag = ConnectionType.Event
Close connection	CloseConnection	History.CloseConnection	✅	DONE
Connection status	GetConnectionStatus	Status.GetHistorianConsoleStatus	✅	DONE
Open/close storage connection	OpenStorageConnection, CloseStorageConnection	Storage.OpenStorageConnection2	⬜	BOUNDED	needed for any data-write path; storage-engine session

2. Reads — process data

Capability	HCAL API	2023 R2 gRPC op	histsdk	Tier	Notes
Raw / full history	CreateHistoryQuery → Start/MoveNext/End	Retrieval.StartQuery→GetNextQueryResultBuffer→EndQuery	✅	DONE	row buffer parsed
Aggregate (interp/avg/min/max/…)	CreateHistoryQuery (RetrievalMode)	same	✅	DONE	all 15 RetrievalModes mapped
At-time / value-at	(interp window)	same	✅	DONE
Analog summary	CreateAnalogSummaryQuery	Retrieval.StartQuery (summary mode)	🟗	BOUNDED	mode variant of existing query
State summary	CreateStateSummaryQuery	Retrieval.StartQuery (state mode)	⬜	BOUNDED	extra row layout to decode
Block read	ReadBlocks	Storage.LoadBlocks	⬜	BOUNDED	low-level; rarely needed

3. Reads — events

Capability	HCAL API	2023 R2 gRPC op	histsdk	Tier	Notes
Event query	CreateEventQuery → Start/MoveNext/End	Retrieval.StartEventQuery→GetNextEventQueryResultBuffer→EndEventQuery	✅	DONE	rows + typed property bag parsed; CM_EVENT registration done
Event filters	EventQuery.AddEventFilter / AddEventFilterCondition	filter bytes in StartEventQuery request	⬜	BOUNDED	encode filter predicate into request buffer

4. Browse & metadata

Capability	HCAL API	2023 R2 gRPC op	histsdk	Tier	Notes
Tag name browse	CreateTagQuery → GetTagNames	Retrieval.StartTagQuery/QueryTag (or LikeTagnames)	✅	DONE	wildcard works
Tag metadata	GetTagInfoByName, TagQuery.GetTagInfo	Retrieval.GetTagInfosFromName	✅	DONE
Extended properties (read)	GetTagExtendedPropertiesByName	Retrieval.GetTagExtendedPropertiesFromName	⬜	BOUNDED	TEP buffer decode
Localized properties (read)	GetTagLocalizedPropertiesByName	Retrieval.GetTagLocalizedPropertiesFromName	⬜	BOUNDED
SQL passthrough	ExecuteSqlCommand	Retrieval.ExecuteSqlCommand	⬜	TRIVIAL	thin string-in / status-out

5. Tag configuration (writes)

Capability	HCAL API	2023 R2 gRPC op	histsdk	Tier	Notes
Create analog tag	AddTag	History.EnsureTags (EnsT2)	✅	DONE	Float/Double/Int2/Int4/UInt2/UInt4 + scaling
Create string/discrete tag	AddTag	History.EnsureTags	⬜	GATED/BOUNDED	native AddTag rejects these types server-side; needs different metadata path
Delete tag(s)	DeleteTags	History.DeleteTags	✅	DONE
Rename tag(s)	RenameTags	(History op)	⬜	BOUNDED	AllowRenameTags param already probed
Add/Delete extended properties	AddTagExtendedProperties, DeleteTagExtendedPropertiesByName	History.AddTagExtendedProperties / DeleteTagExtendedProperties	⬜	BOUNDED	gRPC op + TEP serialize
Add/Delete localized properties	AddTagLocalizedProperties, DeleteTagLocalizedPropertiesByName	History.AddTagLocalizedProperties / DeleteTagLocalizedProperties	⬜	BOUNDED

6. Data writes — values

Capability	HCAL API	2023 R2 gRPC op	histsdk	Tier	Notes
Stream process values	AddStreamedValue(HistorianDataValue)	Storage.AddStreamValues	⬜	GATED	runtime cache only ingests from IOServer/AppServer pipelines (129 Tag not found in cache). Not a client bug
Stream events	AddStreamedValue(HistorianEvent)	Storage.AddStreamValues (event VTQ)	⬜	CAPTURE	full path mapped; need CCommonArchestraEventValue::PackToVtq blob bytes. See histevents.md
Non-streamed / historical insert	AddNonStreamedValue, SendNonStreamedValues	Transaction.AddNonStreamValues(Begin/End)	⬜	BOUNDED	explicit original-data insert via Transaction svc; verify ingest permission on target
Versioned streamed value	AddVersionedStreamedValue	Storage.AddStreamValues2	⬜	CAPTURE	revision flag on the VTQ

7. Revisions / edits (modify stored data)

Capability	HCAL API	2023 R2 gRPC op	histsdk	Tier	Notes
Insert/update/delete revision values	AddRevisionValue(s), AddRevisionValuesBegin/End	(storage-engine / transaction path)	⬜	HARD	prior RE: revision-write needs the non-WCF storage-engine pipe (STransactPipeClient2), not the WCF/gRPC surface
Event update/delete (revise)	HistorianEvent.Update/.Delete	UpdateEventStatus (+ revised VTQ)	⬜	CAPTURE	RevisionVersion + Update/Delete flags in the event VTQ

8. Status & system info

Capability	HCAL API	2023 R2 gRPC op	histsdk	Tier	Notes
System parameter	GetSystemParameter	Status.GetSystemParameter	✅	DONE
Runtime parameter	GetRuntimeParameter	Status.GetRuntimeParameter	⬜	TRIVIAL	same shape as GetSystemParameter
Historian info	GetHistorianInfo	Status.GetHistorianInfo	🟗	BOUNDED	GETHI buffer; partially decoded (incl. EventStorageMode @ offset 514)
Server timezone	GetSystemTimeZoneInfo	Status.GetSystemTimeZoneName	⬜	TRIVIAL
Historization status	GetHistorizationStatus	Status op	⬜	BOUNDED
Store-and-forward status	GetStoreForwardStatus	(push events / pull GETHI)	🟗	HARD	currently synthesized; real read needs duplex push or a decoded pull endpoint — see store-forward plan

9. Store-and-forward (offline buffering)

Capability	HCAL API	2023 R2 gRPC op	histsdk	Tier	Notes
SF buffering + replay	(implicit on write conns)	Storage/Transaction *Snapshot + Forward*Snapshot	⬜	HARD	full subsystem: local cache format, snapshot framing, recovery log, forward-on-reconnect. Pragmatic alt: a simpler local queue, not bit-faithful SF
Event SF	(event conn)	Forward**Event**SnapshotBegin/…/End	⬜	HARD	dedicated event-snapshot SF stream
SF parameters	Get/Set SFP	Storage.GetSFParameter/SetSFParameter	⬜	BOUNDED

10. Redundancy / multi-historian

Capability	HCAL API	2023 R2 gRPC op	histsdk	Tier	Notes
Tiered/redundant access, failover	MultiHistorianAccess.* (OpenConnectionToAll, AddSecondaries, partner watchdog, ReSyncTags)	N×single-historian sessions + client logic	⬜	HARD	mostly client-side orchestration over §1–§6; build last
Replication config	(server aahReplication)	—	⬜	GATED	server-side concern

---

Roll-up & recommended cut line

Phase 0 — already DONE (✅): probe · open/close · raw+aggregate+at-time reads · event reads · tag browse · tag metadata · system parameter · connection status · create/delete analog tag. This is a usable modern client today.

Phase 1 — TRIVIAL/BOUNDED, high value (S–M each): ExecuteSqlCommand · runtime parameter · server timezone · extended/localized property read · event filters · summary/state-summary queries · rename tags · ext/localized property writes · GetHistorianInfo. Each is "gRPC op exists, decode one buffer, golden-byte test." Knocks out most of the remaining read/config surface.

Phase 2 — CAPTURE (one native capture each, S): event sending (the headline gap — fully mapped, one PackToVtq capture away) · versioned/non-streamed value writes. Now feasible locally since the Historian is installed.

Defer / simplify (HARD): store-and-forward (do a pragmatic local queue instead of bit-faithful SF) · revision/edit writes (separate storage-engine pipe) · multi- historian redundancy (client orchestration, build last).

Won't unblock from the client (GATED): streaming process-sample writes (AddS2) — server cache only ingests from IOServer/AppServer pipelines; confirm your ingestion model rather than chasing this. Non-analog tag creation likely needs a distinct server path.

Cross-cutting realities (apply to every non-DONE row)

• Inner payloads stay proprietary even under gRPC — the bytes fields carry native VTQ / CTagMetadata / event-value formats. These are version-sensitive; pin to the server version probed at connect and fail closed on mismatch.
• Validation needs a live Historian — now available locally, which is what makes the CAPTURE-tier items practical.
• Support tradeoff — you take on maintenance across Historian versions in exchange for shedding the stock SDK's bugs (mixed-mode marshaling, WCF quirks, global state) for the surface you cover.

Bottom line

A modern-.NET HCAL replacement is feasible and ~60–70% done for a typical read+browse+config+event-read workload. The remaining high-value surface is mostly BOUNDED/CAPTURE (incremental, well-understood), with only store-and-forward, revision-edit, and redundancy being genuine HARD subsystems — and one true wall (GATED process-sample writes) that no client can remove.