lmxopcua/docs/Historian.md

Historian — server-side OPC UA HistoryRead (Phase C)

Phase C wires server-side OPC UA HistoryRead for authored equipment tags flagged historized. The feature is driver-agnostic: any equipment tag (Galaxy, Modbus, OpcUaClient, or any other driver) can be marked historized; the server dispatches all history reads to the registered IHistorianDataSource — today, the Wonderware sidecar client (WonderwareHistorianClient). No EF migration is required; the historian flag rides in the existing schemaless TagConfig JSON blob alongside the Phase B alarm object.

Design reference: docs/plans/2026-06-14-galaxy-phase-c-historian-design.md.

---

Historized TagConfig schema

A tag is historized by setting the Historize this tag checkbox and optional Historian tagname (override) textbox in the Tag modal on the /uns equipment page Tags tab. These controls work for all drivers — typed editors (Modbus, S7, OpcUaClient, etc.) and the raw-JSON textarea (Galaxy) alike. The controls merge isHistorized / historianTagname into the existing TagConfig JSON blob via the TagHistorizeConfig helper, preserving all other keys byte-stable. Drivers that still use the raw-JSON editor (Galaxy) can also add the fields directly in the textarea.

Fields

Field	Type	Required	Description
isHistorized	bool	yes	Marks the tag historized. Materialises the OPC UA node with Historizing=true and the HistoryRead AccessLevel bit.
historianTagname	string	no	Explicit historian tagname to query. When absent or empty, defaults to the tag's driver FullName (the TagConfig.FullName).

Examples

Default historian tagname (uses FullName = TestMachine_002.TestFloat):

{"FullName":"TestMachine_002.TestFloat","isHistorized":true}

Explicit historian tagname override:

{"FullName":"TestMachine_002.TestFloat","isHistorized":true,"historianTagname":"Plant.Line1.Flow"}

A tag that is both historized and a native alarm:

{"FullName":"TestMachine_002.HiAlarm","isHistorized":true,"alarm":{"alarmType":"OffNormalAlarm","severity":700}}

---

ServerHistorian configuration

The ServerHistorian section in appsettings.json controls the historian read path. Enabled defaults to false; when disabled, the server registers NullHistorianDataSource and all HistoryRead calls on historized nodes return GoodNoData (empty, not an error).

{
  "ServerHistorian": {
    "Enabled": false,
    "Host": "localhost",
    "Port": 32569,
    "UseTls": false,
    "ServerCertThumbprint": "",
    "SharedSecret": "",
    "MaxTieClusterOverfetch": 65536
  }
}

Key	Type	Default	Description
Enabled	bool	false	Enable the live WonderwareHistorianClient. false → NullHistorianDataSource (empty reads).
Host	string	localhost	DNS name or IP of the machine running the historian sidecar.
Port	int	32569	TCP port the sidecar listens on (OTOPCUA_HISTORIAN_TCP_PORT).
UseTls	bool	false	Wrap the TCP connection in TLS.
ServerCertThumbprint	string	—	Optional SHA-1 thumbprint to pin the sidecar's TLS certificate. Leave empty for CA-chain validation.
SharedSecret	string	—	Shared secret token the sidecar expects on every connection. Required when Enabled.
MaxTieClusterOverfetch	int	65536	Maximum samples the server will fetch in one shot to page through a tie cluster (multiple samples sharing one SourceTimestamp). A cluster larger than this ceiling fails BadHistoryOperationUnsupported. Raise to handle abnormally large tie clusters; the default covers all normal-data cases.

Do not commit SharedSecret to appsettings.json. Set it via an environment variable, a secrets store, or a deployment-time overlay. The checked-in default is always empty.

The ServerHistorian section is independent of the AlarmHistorian section (the alarm write path). They share the same Wonderware sidecar process but hold separate client instances and separate SharedSecret values.

---

HistoryRead behavior

Read variants

The server supports all four OPC UA HistoryRead variants:

Variant	Node type	CLI --aggregate
Raw	Historized variable	(omit --aggregate)
Processed	Historized variable	Average, Minimum, Maximum, Total, Count
AtTime	Historized variable	n/a (client supplies exact timestamps)
Events	Equipment folder (event notifier)	n/a

Variable nodes (historized tags) serve Raw, Processed, and AtTime history. Historizing=true and AccessLevels.HistoryRead are set at materialization so any compliant OPC UA client can discover historized capability from the node's attributes.

Equipment-folder event-notifier nodes serve Event history. Every equipment folder that owns at least one alarm condition is already an event notifier; the server registers a sourceName (the equipment id) for each such folder and maps event history reads to the Wonderware historian using that source. Event-field projection supports the standard BaseEventType select clauses — EventId, SourceName, Time, ReceiveTime, Message, and Severity; an unsupported select operand returns a null field (spec-conformant).

OpcUaClient driver — upstream passthrough for all four variants

The OpcUaClient driver's IHistoryProvider implementation forwards all four history-read variants (Raw, Processed, AtTime, and Events) to its upstream OPC UA server. For the Events variant it sends a fixed canonical BaseEventType EventFilter selecting the standard six fields (EventId, SourceName, Time, ReceiveTime, Message, Severity) and maps the upstream HistoryEvent onto HistoricalEvent — the same six-field projection the OtOpcUa node-manager itself projects when serving event history. This is a driver-level capability: the OpcUaClient driver acts as a passthrough to whatever historian the upstream server exposes, and is independent of the single server-side IHistorianDataSource backend (WonderwareHistorianClient / NullHistorianDataSource) that the OtOpcUa node-manager dispatches HistoryRead to for tags on other drivers (Galaxy, Modbus, S7, etc.).

Graceful degradation

Situation	HistoryRead status
Historized node, historian configured and reachable, results found	Good
Historized node, historian configured, time range empty	GoodNoData
Historized node, historian NOT configured (Enabled=false / Null source)	GoodNoData (empty)
Non-historized node	BadHistoryOperationUnsupported
Backend timeout or exception	BadHistoryOperationUnsupported per node; other nodes in the same batch are unaffected

A historized node with no historian configured never returns an error status — it returns empty. This means a deployment can author and publish historized tags before the historian sidecar is provisioned, without producing error spikes in connected clients.

Continuation-point paging (Raw)

HistoryRead-Raw is paged server-side. The historian backend is single-shot (it returns up to NumValuesPerNode samples with no continuation point of its own), so the server synthesises paging time-based:

• A page that returns exactly NumValuesPerNode samples (with NumValuesPerNode > 0) MAY have more behind it, so the server stores a resume cursor and returns an opaque continuation point (16 bytes). The client hands it back to fetch the next page.
• A short page (fewer than the cap) is the last page — no continuation point.
• NumValuesPerNode == 0 ("all values, no limit") is never paged; the whole window returns in one shot.
• The resume cursor is tie-safe: the next page resumes from the last returned sample's SourceTimestamp inclusive and drops the boundary samples already emitted, so samples sharing the boundary timestamp are neither duplicated nor skipped.

Oversized tie clusters — within-timestamp paging. When more samples share one SourceTimestamp than the current page cap, the server detects that the cursor has stalled on a tie cluster (the last returned timestamp equals the resume timestamp). It then over-fetches the entire cluster at that single timestamp up to a bounded ceiling controlled by ServerHistorian:MaxTieClusterOverfetch (default 65 536), then serves the cluster NumValuesPerNode samples at a time across successive pages, advancing the cursor one tick past the timestamp once the cluster is fully drained.

Short pages within a cluster still carry a continuation point. A within-cluster page that returns fewer than NumValuesPerNode samples (because the cluster happened to be smaller than the cap, or is the final partial batch) is not the last page if the cluster itself has not been fully emitted — the server retains the continuation point so the client can drain the remainder. Only when the cluster is exhausted and the cursor has advanced past the timestamp does the short-page rule apply.

Cluster larger than MaxTieClusterOverfetch. If the over-fetch itself reaches the ceiling without spanning the full cluster, the node fails loudly with BadHistoryOperationUnsupported and the tag + timestamp + ceiling are logged. Remedies: raise MaxTieClusterOverfetch (or NumValuesPerNode) to cover the full cluster, or investigate the data anomaly (raw samples normally carry strictly increasing distinct timestamps).

For a single tag's raw history a tie cluster larger than the default 65 536 is a severe data anomaly. The ceiling exists to bound server-side memory on pathological data, not to cap normal operation.

Continuation points are bound to the OPC UA session (the SDK's ServerConfiguration.MaxHistoryContinuationPoints cap, default 100, with oldest-eviction; points are disposed when the session closes). Resuming an unknown / evicted / released point returns BadContinuationPointInvalid. releaseContinuationPoints drops the stored cursors without reading data.

Total aggregate derivation

The OPC UA Total aggregate is supported over the Wonderware backend. Because the Wonderware AnalogSummary query exposes no Total column, the value is derived client-side using the time-integral identity:

Total = time-weighted Average × interval-seconds

The wire request is issued with the Average column; each returned bucket's value is multiplied by interval.TotalSeconds before the result is returned to the OPC UA client. Bucket status codes and timestamps are preserved unchanged. Null (unavailable) Average buckets produce a null Total (BadNoData downstream) — the scaling is not applied.

This derivation is exact for piecewise-constant (step) signals. For continuously varying signals it is an approximation identical to the one Wonderware would apply internally, so the result is consistent with what AVEVA Historian reports for the same window.

Known limitations

• Processed and AtTime are single-shot (no continuation points). Unlike Raw, neither ReadProcessedDetails nor ReadAtTimeDetails carries a client count cap (NumValuesPerNode): the Processed bucket count is deterministic (window / interval) and AtTime returns exactly one sample per requested timestamp, so the single-shot backend returns the complete result in one read and there is no "full page ⇒ maybe more" signal to page on. Returning the full result with no continuation point is spec-conformant.
• No modified-value history (HistoryReadModified). Requests for modified values return BadHistoryOperationUnsupported. This is infra-gated, not a server-code gap: the AVEVA Wonderware historian backend (IHistorianDataSource, the TCP sidecar client) exposes only a current-value read path — there is no modified/edited-history surface to source the data from. The server-side override is in place (it cleanly rejects modified reads per node) and IsReadModified is honoured; serving real modified-value history is unblocked only once the historian client/sidecar grows a modified-read RPC. Until then, rejecting is the correct, spec-conformant behaviour.

Redundancy and authorization

History reads are served from any node — there is no Primary gate. Authorization is the standard OPC UA HistoryRead permission enforced by the SDK through the AccessLevels.HistoryRead bit set at materialization. A session without sufficient permissions receives BadUserAccessDenied from the SDK before the dispatch reaches the historian.

---

Authoring workflow

1. Open the equipment's Tags tab on /uns/equipment/{id}.
2. Create or edit the tag.
   • For typed-editor drivers (Modbus, S7, OpcUaClient, etc.): check the Historize this tag checkbox and, if needed, fill in the Historian tagname (override) textbox.
   • For raw-JSON editors (Galaxy): you can check the same first-class checkboxes (they appear below the JSON textarea), or add "isHistorized":true (and optionally "historianTagname":"...") directly in the textarea.
3. Save and publish. The server rebuilds its address space; the node materialises with Historizing=true and the HistoryRead AccessLevel bit.
4. Confirm with Client.CLI read that the node's Status is Good and that the value is updating. Then issue a historyread to verify the historian connection returns data.

Native-alarm historian opt-out (alarm.historizeToAveva)

A tag carrying a native "alarm" object has a separate historian opt-out for its alarm transitions (distinct from tag-value historization). On the Tag modal, check or uncheck Historize to AVEVA in the alarm section. This maps to alarm.historizeToAveva (bool?) in the TagConfig JSON:

• Absent or true (default) — the alarm's transitions are written to AVEVA Historian via HistorianAdapterActor.
• false — the durable AVEVA write is suppressed for this alarm's transitions. The live /alerts feed and OPC UA condition events are unaffected.

The gate is applied in HistorianAdapterActor using is not false semantics, matching the scripted-alarm HistorizeToAveva posture. See ScriptedAlarms.md §Native driver alarms and AlarmTracking.md §Historian write-back for the full alarm-historian routing.

---

Client.CLI historyread examples

The historyread command reads historical data from any node. Supply start and end times in ISO 8601 UTC form. See docs/Client.CLI.md for the full flag reference.

# Raw history for a historized Galaxy tag (last 24 hours by default)
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z"

# Limit to 100 values
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z" \
  --max 100

# 1-hour average aggregate
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z" \
  --aggregate Average --interval 3600000

# Authenticated read (ReadOnly role or higher required)
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z" \
  -U reader -P password

Supported --aggregate values: Average, Minimum, Maximum, Count, Start, End, StandardDeviation (aliases: avg, min, max, stddev/stdev, first, last). --interval is the processing interval in milliseconds (default 3600000 = 1 hour). The Total aggregate is served by the server's OPC UA HistoryRead endpoint for any conformant client (derived as time-weighted Average × interval-seconds — see "Total aggregate derivation" above), but is not exposed by this bundled CLI.

---

Live /run gate

The live read gate requires the Wonderware historian sidecar running on the WW Historian VM (10.100.0.48) and AVEVA Historian healthy. Set ServerHistorian:Enabled=true with the correct Host, Port, and SharedSecret in appsettings.json (or via environment variables), then deploy and publish at least one historized Galaxy tag. The gate is operator-driven — it is not part of the local docker-dev rig.

See AlarmHistorian.md for the historian sidecar setup and ServiceHosting.md for the sidecar service configuration.

---

---

Cross-driver TagConfig key reference

The TagConfig JSON blob is the single extension surface for per-tag server-side behaviours across all drivers. Keys from multiple phases coexist in the same blob; unknown keys are preserved byte-stable on round-trip by all typed editors.

Historian + alarm keys (Phase B / Phase C)

Key	Type	Phase	Description
isHistorized	bool	C	Marks the tag historized (HistoryRead + Historizing=true).
historianTagname	string	C	Explicit historian tagname override (defaults to FullName).
alarm	object	B	Native alarm definition (alarmType, severity, historizeToAveva, …).

Array keys (Phase 4c)

Key	Type	Phase	Description
isArray	bool	4c	When true and arrayLength >= 1, materialises the node as a 1-D array (ValueRank=OneDimension). Absent or false → scalar.
arrayLength	uint (≥1)	4c	Element count; sets ArrayDimensions. Required when isArray: true.

Cross-driver array coverage matrix

Driver	Read mechanism	Live-verify status
Modbus	Contiguous FC03/FC04 block; String/BitInRegister modes supported	Mac-verifiable (sim 10.100.0.35:5020)
S7	ReadBytesAsync block + per-element decode loop	Unit-proven (fixture down)
AB CIP	libplctag native array read (atomic + UDT member arrays)	Unit-proven (fixture down)
AB Legacy	PCCC multi-element file read via libplctag (cap 256 elements)	Unit-proven (fixture down)
TwinCAT	ADS native array symbol read	Unit-proven (fixture down)

Array writes, multi-dimensional arrays (ValueRank>1), and per-element historization are out of scope — see Uns.md §Array tags.

---

Closed stillpending.md §2 items (Phase 4 / Phase 4b)

The following items from the stillpending.md backlog §2 were closed by earlier phases and are recorded here so future audits don't re-flag them.

Item	Closed by	Commit
Modbus Int64 / UInt64 OPC UA node DataType correction	Phase 4	bd8fee61
HistoryAggregateType.Total — Total aggregate	Phase 4 (derived client-side as time-weighted Average × interval-seconds)	5e27b5f7
Historian poison alarm-event indefinite retry — dead-letter cap (task #437)	Phase 4	fcb38014

---

See also

• docs/plans/2026-06-14-galaxy-phase-c-historian-design.md — full design and implementation notes
• AlarmHistorian.md — alarm write path; shares the same Wonderware sidecar
• AlarmTracking.md — OPC UA Part 9 alarm surface (event history source)
• Client.CLI.md — full historyread flag reference
• ScriptedAlarms.md §"Native driver alarms" — the Phase B alarm object in TagConfig (parallel carrier)
• Uns.md §Array tags — isArray/arrayLength keys, cross-driver coverage, and deferrals