lmxopcua/docs/Historian.md

Historian — server-side OPC UA HistoryRead (Phase C)

Phase C wires server-side OPC UA HistoryRead for authored equipment tags flagged historized. The feature is driver-agnostic: any equipment tag (Galaxy, Modbus, OpcUaClient, or any other driver) can be marked historized; the server dispatches all history reads to the registered IHistorianDataSource — the HistorianGateway read client (GatewayHistorianDataSource, talking gRPC to the external ZB.MOM.WW.HistorianGateway via the ZB.MOM.WW.HistorianGateway.Client package). No EF migration is required; the historian flag rides in the existing schemaless TagConfig JSON blob alongside the Phase B alarm object. (The bespoke Wonderware TCP sidecar backend this replaced was retired — see drivers/Historian.Wonderware.md.)

Design reference: docs/plans/2026-06-14-galaxy-phase-c-historian-design.md.

---

Historized TagConfig schema

A tag is historized by setting the Historize this tag checkbox and optional Historian tagname (override) textbox in the Tag modal on the /uns equipment page Tags tab. These controls work for all drivers — typed editors (Modbus, S7, OpcUaClient, etc.) and the raw-JSON textarea (Galaxy) alike. The controls merge isHistorized / historianTagname into the existing TagConfig JSON blob via the TagHistorizeConfig helper, preserving all other keys byte-stable. Drivers that still use the raw-JSON editor (Galaxy) can also add the fields directly in the textarea.

Fields

Field	Type	Required	Description
isHistorized	bool	yes	Marks the tag historized. Materialises the OPC UA node with Historizing=true and the HistoryRead AccessLevel bit.
historianTagname	string	no	Explicit historian tagname to query. When absent or empty, defaults to the tag's driver FullName (the TagConfig.FullName).

Examples

Default historian tagname (uses FullName = TestMachine_002.TestFloat):

{"FullName":"TestMachine_002.TestFloat","isHistorized":true}

Explicit historian tagname override:

{"FullName":"TestMachine_002.TestFloat","isHistorized":true,"historianTagname":"Plant.Line1.Flow"}

A tag that is both historized and a native alarm:

{"FullName":"TestMachine_002.HiAlarm","isHistorized":true,"alarm":{"alarmType":"OffNormalAlarm","severity":700}}

---

ServerHistorian configuration

The ServerHistorian section in appsettings.json controls the historian read path. Enabled defaults to false; when disabled, the server registers NullHistorianDataSource and all HistoryRead calls on historized nodes return GoodNoData (empty, not an error).

{
  "ServerHistorian": {
    "Enabled": false,
    "Endpoint": "",
    "ApiKey": "",
    "UseTls": true,
    "AllowUntrustedServerCertificate": false,
    "CaCertificatePath": null,
    "CallTimeout": "00:00:30",
    "MaxTieClusterOverfetch": 65536
  }
}

Key	Type	Default	Description
Enabled	bool	false	Enable the live GatewayHistorianDataSource. false → NullHistorianDataSource (empty reads).
Endpoint	string	""	Absolute gateway URI, e.g. https://host:5222. Scheme selects transport (https:// = TLS, http:// = h2c plaintext). Required when Enabled.
ApiKey	string	""	The gateway peppered-HMAC key (histgw_<id>_<secret>) sent as Authorization: Bearer. Required when Enabled. Supply via env ServerHistorian__ApiKey.
UseTls	bool	true	Connect over TLS; must match the Endpoint scheme.
AllowUntrustedServerCertificate	bool	false	Accept a self-signed / untrusted server certificate (dev / on-prem only).
CaCertificatePath	string|null	null	PEM CA file pinning the gateway's TLS chain. Null/empty uses the OS trust store.
CallTimeout	TimeSpan	00:00:30	Per-call deadline applied to each unary gateway read.
MaxTieClusterOverfetch	int	65536	Maximum samples the server will fetch in one shot to page through a tie cluster (multiple samples sharing one SourceTimestamp). A cluster larger than this ceiling fails BadHistoryOperationUnsupported. Raise to handle abnormally large tie clusters; the default covers all normal-data cases.

Do not commit ApiKey to appsettings.json. Set it via the environment variable ServerHistorian__ApiKey, a secrets store, or a deployment-time overlay. The checked-in default is always empty.

Gateway-side prerequisites. The target gateway must run RuntimeDb:Enabled=true (continuous WriteLiveValues) + RuntimeDb:EventReadsEnabled=true (alarm-history ReadEvents), and the API key must carry the scopes historian:read, historian:write, historian:tags:write.

Migration from the Wonderware backend. Rename the old keys: Host/Port → Endpoint (https://host:5222); SharedSecret → ApiKey (env ServerHistorian__ApiKey); ServerCertThumbprint → CaCertificatePath (+ UseTls / AllowUntrustedServerCertificate).

The ServerHistorian section is independent of the AlarmHistorian section (the alarm write path) and the ContinuousHistorization section (driver-value capture). All three target the same gateway — but only ServerHistorian carries the connection (endpoint/key/TLS); the other two source it from there.

---

HistoryRead behavior

Read variants

The server supports all four OPC UA HistoryRead variants:

Variant	Node type	CLI --aggregate
Raw	Historized variable	(omit --aggregate)
Processed	Historized variable	Average, Minimum, Maximum, Total, Count
AtTime	Historized variable	n/a (client supplies exact timestamps)
Events	Equipment folder (event notifier)	n/a

Variable nodes (historized tags) serve Raw, Processed, and AtTime history. Historizing=true and AccessLevels.HistoryRead are set at materialization so any compliant OPC UA client can discover historized capability from the node's attributes.

Equipment-folder event-notifier nodes serve Event history. Every equipment folder that owns at least one alarm condition is already an event notifier; the server registers a sourceName (the equipment id) for each such folder and maps event history reads to the HistorianGateway using that source. (Alarm-history ReadEvents requires the gateway running RuntimeDb:EventReadsEnabled=true.) Event-field projection supports the standard BaseEventType select clauses — EventId, SourceName, Time, ReceiveTime, Message, and Severity; an unsupported select operand returns a null field (spec-conformant).

OpcUaClient driver — upstream passthrough for all four variants

The OpcUaClient driver's IHistoryProvider implementation forwards all four history-read variants (Raw, Processed, AtTime, and Events) to its upstream OPC UA server. For the Events variant it sends a fixed canonical BaseEventType EventFilter selecting the standard six fields (EventId, SourceName, Time, ReceiveTime, Message, Severity) and maps the upstream HistoryEvent onto HistoricalEvent — the same six-field projection the OtOpcUa node-manager itself projects when serving event history. This is a driver-level capability: the OpcUaClient driver acts as a passthrough to whatever historian the upstream server exposes, and is independent of the single server-side IHistorianDataSource backend (GatewayHistorianDataSource / NullHistorianDataSource) that the OtOpcUa node-manager dispatches HistoryRead to for tags on other drivers (Galaxy, Modbus, S7, etc.).

Graceful degradation

Situation	HistoryRead status
Historized node, historian configured and reachable, results found	Good
Historized node, historian configured, time range empty	GoodNoData
Historized node, historian NOT configured (Enabled=false / Null source)	GoodNoData (empty)
Non-historized node	BadHistoryOperationUnsupported
Backend timeout or exception	BadHistoryOperationUnsupported per node; other nodes in the same batch are unaffected

A historized node with no historian configured never returns an error status — it returns empty. This means a deployment can author and publish historized tags before the historian gateway is provisioned, without producing error spikes in connected clients.

Continuation-point paging (Raw)

HistoryRead-Raw is paged server-side. The historian backend is single-shot (it returns up to NumValuesPerNode samples with no continuation point of its own), so the server synthesises paging time-based:

• A page that returns exactly NumValuesPerNode samples (with NumValuesPerNode > 0) MAY have more behind it, so the server stores a resume cursor and returns an opaque continuation point (16 bytes). The client hands it back to fetch the next page.
• A short page (fewer than the cap) is the last page — no continuation point.
• NumValuesPerNode == 0 ("all values, no limit") is never paged; the whole window returns in one shot.
• The resume cursor is tie-safe: the next page resumes from the last returned sample's SourceTimestamp inclusive and drops the boundary samples already emitted, so samples sharing the boundary timestamp are neither duplicated nor skipped.

Oversized tie clusters — within-timestamp paging. When more samples share one SourceTimestamp than the current page cap, the server detects that the cursor has stalled on a tie cluster (the last returned timestamp equals the resume timestamp). It then over-fetches the entire cluster at that single timestamp up to a bounded ceiling controlled by ServerHistorian:MaxTieClusterOverfetch (default 65 536), then serves the cluster NumValuesPerNode samples at a time across successive pages, advancing the cursor one tick past the timestamp once the cluster is fully drained.

Short pages within a cluster still carry a continuation point. A within-cluster page that returns fewer than NumValuesPerNode samples (because the cluster happened to be smaller than the cap, or is the final partial batch) is not the last page if the cluster itself has not been fully emitted — the server retains the continuation point so the client can drain the remainder. Only when the cluster is exhausted and the cursor has advanced past the timestamp does the short-page rule apply.

Cluster larger than MaxTieClusterOverfetch. If the over-fetch itself reaches the ceiling without spanning the full cluster, the node fails loudly with BadHistoryOperationUnsupported and the tag + timestamp + ceiling are logged. Remedies: raise MaxTieClusterOverfetch (or NumValuesPerNode) to cover the full cluster, or investigate the data anomaly (raw samples normally carry strictly increasing distinct timestamps).

For a single tag's raw history a tie cluster larger than the default 65 536 is a severe data anomaly. The ceiling exists to bound server-side memory on pathological data, not to cap normal operation.

Continuation points are bound to the OPC UA session (the SDK's ServerConfiguration.MaxHistoryContinuationPoints cap, default 100, with oldest-eviction; points are disposed when the session closes). Resuming an unknown / evicted / released point returns BadContinuationPointInvalid. releaseContinuationPoints drops the stored cursors without reading data.

Total aggregate

The OPC UA Total aggregate is supported over the HistorianGateway backend. The gateway exposes a native Integral retrieval mode, so Total maps straight to it (HistoryAggregateType.Total → RetrievalMode.Integral) — no client-side scaling. (This replaces the retired Wonderware path, which had no Total column and derived it client-side as time-weighted Average × interval-seconds.) Count is likewise a native gateway mode. Bucket status codes and timestamps are preserved unchanged; empty / null buckets surface as BadNoData.

Known limitations

• Processed and AtTime are single-shot (no continuation points). Unlike Raw, neither ReadProcessedDetails nor ReadAtTimeDetails carries a client count cap (NumValuesPerNode): the Processed bucket count is deterministic (window / interval) and AtTime returns exactly one sample per requested timestamp, so the single-shot backend returns the complete result in one read and there is no "full page ⇒ maybe more" signal to page on. Returning the full result with no continuation point is spec-conformant.
• No modified-value history (HistoryReadModified). Requests for modified values return BadHistoryOperationUnsupported. This is infra-gated, not a server-code gap: the HistorianGateway backend (GatewayHistorianDataSource) exposes only a current-value read path — there is no modified/edited-history surface to source the data from. The server-side override is in place (it cleanly rejects modified reads per node) and IsReadModified is honoured; serving real modified-value history is unblocked only once the gateway grows a modified-read RPC. Until then, rejecting is the correct, spec-conformant behaviour.

Redundancy and authorization

History reads are served from any node — there is no Primary gate. Authorization is the standard OPC UA HistoryRead permission enforced by the SDK through the AccessLevels.HistoryRead bit set at materialization. A session without sufficient permissions receives BadUserAccessDenied from the SDK before the dispatch reaches the historian.

---

Authoring workflow

1. Open the equipment's Tags tab on /uns/equipment/{id}.
2. Create or edit the tag.
   • For typed-editor drivers (Modbus, S7, OpcUaClient, etc.): check the Historize this tag checkbox and, if needed, fill in the Historian tagname (override) textbox.
   • For raw-JSON editors (Galaxy): you can check the same first-class checkboxes (they appear below the JSON textarea), or add "isHistorized":true (and optionally "historianTagname":"...") directly in the textarea.
3. Save and publish. The server rebuilds its address space; the node materialises with Historizing=true and the HistoryRead AccessLevel bit.
4. Confirm with Client.CLI read that the node's Status is Good and that the value is updating. Then issue a historyread to verify the historian connection returns data.

Native-alarm historian opt-out (alarm.historizeToAveva)

A tag carrying a native "alarm" object has a separate historian opt-out for its alarm transitions (distinct from tag-value historization). On the Tag modal, check or uncheck Historize to AVEVA in the alarm section. This maps to alarm.historizeToAveva (bool?) in the TagConfig JSON:

• Absent or true (default) — the alarm's transitions are written to AVEVA Historian via HistorianAdapterActor.
• false — the durable AVEVA write is suppressed for this alarm's transitions. The live /alerts feed and OPC UA condition events are unaffected.

The gate is applied in HistorianAdapterActor using is not false semantics, matching the scripted-alarm HistorizeToAveva posture. See ScriptedAlarms.md §Native driver alarms and AlarmTracking.md §Historian write-back for the full alarm-historian routing.

---

Client.CLI historyread examples

The historyread command reads historical data from any node. Supply start and end times in ISO 8601 UTC form. See docs/Client.CLI.md for the full flag reference.

# Raw history for a historized Galaxy tag (last 24 hours by default)
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z"

# Limit to 100 values
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z" \
  --max 100

# 1-hour average aggregate
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z" \
  --aggregate Average --interval 3600000

# Authenticated read (ReadOnly role or higher required)
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z" \
  -U reader -P password

Supported --aggregate values: Average, Minimum, Maximum, Count, Start, End, StandardDeviation (aliases: avg, min, max, stddev/stdev, first, last). --interval is the processing interval in milliseconds (default 3600000 = 1 hour). The Total aggregate is served by the server's OPC UA HistoryRead endpoint for any conformant client (derived as time-weighted Average × interval-seconds — see "Total aggregate derivation" above), but is not exposed by this bundled CLI.

---

Live /run gate

The live read gate requires a reachable ZB.MOM.WW.HistorianGateway (VPN to wonder-sql-vd03) with the AVEVA Historian behind it healthy. Set ServerHistorian:Enabled=true with the correct Endpoint (https://host:5222) and supply ServerHistorian__ApiKey via the environment, then deploy and publish at least one historized Galaxy tag. The gate is operator-driven — it is not part of the local docker-dev rig. The gateway-backed driver also ships an env-gated live suite (Category=LiveIntegration); see the HISTGW_GATEWAY_ENDPOINT / HISTGW_GATEWAY_APIKEY / HISTGW_TEST_TAG / HISTGW_WRITE_SANDBOX_TAG / HISTGW_ALARM_SOURCE env vars (it skips cleanly when they are absent).

See AlarmHistorian.md for the alarm write path and ServiceHosting.md for the (external) HistorianGateway deployment.

---

---

Cross-driver TagConfig key reference

The TagConfig JSON blob is the single extension surface for per-tag server-side behaviours across all drivers. Keys from multiple phases coexist in the same blob; unknown keys are preserved byte-stable on round-trip by all typed editors.

Historian + alarm keys (Phase B / Phase C)

Key	Type	Phase	Description
isHistorized	bool	C	Marks the tag historized (HistoryRead + Historizing=true).
historianTagname	string	C	Explicit historian tagname override (defaults to FullName).
alarm	object	B	Native alarm definition (alarmType, severity, historizeToAveva, …).

Array keys (Phase 4c)

Key	Type	Phase	Description
isArray	bool	4c	When true and arrayLength >= 1, materialises the node as a 1-D array (ValueRank=OneDimension). Absent or false → scalar.
arrayLength	uint (≥1)	4c	Element count; sets ArrayDimensions. Required when isArray: true.

Cross-driver array coverage matrix

Driver	Read mechanism	Live-verify status
Modbus	Contiguous FC03/FC04 block; String/BitInRegister modes supported	Mac-verifiable (sim 10.100.0.35:5020)
S7	ReadBytesAsync block + per-element decode loop	Unit-proven (fixture down)
AB CIP	libplctag native array read (atomic + UDT member arrays)	Unit-proven (fixture down)
AB Legacy	PCCC multi-element file read via libplctag (cap 256 elements)	Unit-proven (fixture down)
TwinCAT	ADS native array symbol read	Unit-proven (fixture down)

Array writes, multi-dimensional arrays (ValueRank>1), and per-element historization are out of scope — see Uns.md §Array tags.

---

Closed stillpending.md §2 items (Phase 4 / Phase 4b)

The following items from the stillpending.md backlog §2 were closed by earlier phases and are recorded here so future audits don't re-flag them.

Item	Closed by	Commit
Modbus Int64 / UInt64 OPC UA node DataType correction	Phase 4	bd8fee61
HistoryAggregateType.Total — Total aggregate	Phase 4 (derived client-side as time-weighted Average × interval-seconds)	5e27b5f7
Historian poison alarm-event indefinite retry — dead-letter cap (task #437)	Phase 4	fcb38014

---

See also

• docs/plans/2026-06-14-galaxy-phase-c-historian-design.md — full design and implementation notes
• AlarmHistorian.md — alarm write path; drains to the same HistorianGateway (SendEvent)
• AlarmTracking.md — OPC UA Part 9 alarm surface (event history source)
• Client.CLI.md — full historyread flag reference
• ScriptedAlarms.md §"Native driver alarms" — the Phase B alarm object in TagConfig (parallel carrier)
• Uns.md §Array tags — isArray/arrayLength keys, cross-driver coverage, and deferrals