lmxopcua/docs/Historian.md

# Historian — server-side OPC UA HistoryRead (Phase C)

Phase C wires server-side OPC UA **HistoryRead** for authored equipment tags flagged
historized. The feature is driver-agnostic: any equipment tag (Galaxy, Modbus, OpcUaClient,
or any other driver) can be marked historized; the server dispatches all history reads to the
registered `IHistorianDataSource` — today, the Wonderware sidecar client
(`WonderwareHistorianClient`). No EF migration is required; the historian flag rides in the
existing schemaless `TagConfig` JSON blob alongside the Phase B `alarm` object.

Design reference: [docs/plans/2026-06-14-galaxy-phase-c-historian-design.md](plans/2026-06-14-galaxy-phase-c-historian-design.md).

---

## Historized TagConfig schema

A tag is historized by adding fields to its `TagConfig` blob on the `/uns` equipment page
Tags tab (raw-JSON textarea). No separate UI control exists — Galaxy (the primary use case)
already uses the raw-JSON editor.

### Fields

| Field | Type | Required | Description |
|---|---|---|---|
| `isHistorized` | bool | yes | Marks the tag historized. Materialises the OPC UA node with `Historizing=true` and the `HistoryRead` AccessLevel bit. |
| `historianTagname` | string | no | Explicit historian tagname to query. When absent or empty, defaults to the tag's driver `FullName` (the `TagConfig.FullName`). |

### Examples

Default historian tagname (uses `FullName` = `TestMachine_002.TestFloat`):

```json
{"FullName":"TestMachine_002.TestFloat","isHistorized":true}
```

Explicit historian tagname override:

```json
{"FullName":"TestMachine_002.TestFloat","isHistorized":true,"historianTagname":"Plant.Line1.Flow"}
```

A tag that is both historized and a native alarm:

```json
{"FullName":"TestMachine_002.HiAlarm","isHistorized":true,"alarm":{"alarmType":"OffNormalAlarm","severity":700}}
```

---

## ServerHistorian configuration

The `ServerHistorian` section in `appsettings.json` controls the historian read path.
`Enabled` defaults to `false`; when disabled, the server registers `NullHistorianDataSource`
and all HistoryRead calls on historized nodes return `GoodNoData` (empty, not an error).

```json
{
  "ServerHistorian": {
    "Enabled": false,
    "Host": "localhost",
    "Port": 32569,
    "UseTls": false,
    "ServerCertThumbprint": "",
    "SharedSecret": ""
  }
}
```

| Key | Type | Default | Description |
|---|---|---|---|
| `Enabled` | bool | `false` | Enable the live `WonderwareHistorianClient`. `false` → `NullHistorianDataSource` (empty reads). |
| `Host` | string | `localhost` | DNS name or IP of the machine running the historian sidecar. |
| `Port` | int | `32569` | TCP port the sidecar listens on (`OTOPCUA_HISTORIAN_TCP_PORT`). |
| `UseTls` | bool | `false` | Wrap the TCP connection in TLS. |
| `ServerCertThumbprint` | string | — | Optional SHA-1 thumbprint to pin the sidecar's TLS certificate. Leave empty for CA-chain validation. |
| `SharedSecret` | string | — | Shared secret token the sidecar expects on every connection. Required when `Enabled`. |

> **Do not commit `SharedSecret` to `appsettings.json`.** Set it via an environment variable,
> a secrets store, or a deployment-time overlay. The checked-in default is always empty.

The `ServerHistorian` section is independent of the `AlarmHistorian` section (the alarm
write path). They share the same Wonderware sidecar process but hold separate client
instances and separate `SharedSecret` values.

---

## HistoryRead behavior

### Read variants

The server supports all four OPC UA HistoryRead variants:

| Variant | Node type | CLI `--aggregate` |
|---|---|---|
| **Raw** | Historized variable | (omit `--aggregate`) |
| **Processed** | Historized variable | `Average`, `Minimum`, `Maximum`, `Total`, `Count` |
| **AtTime** | Historized variable | n/a (client supplies exact timestamps) |
| **Events** | Equipment folder (event notifier) | n/a |

**Variable nodes** (historized tags) serve Raw, Processed, and AtTime history.
`Historizing=true` and `AccessLevels.HistoryRead` are set at materialization so any compliant
OPC UA client can discover historized capability from the node's attributes.

**Equipment-folder event-notifier nodes** serve Event history. Every equipment folder that
owns at least one alarm condition is already an event notifier; the server registers a
`sourceName` (the equipment id) for each such folder and maps event history reads to the
Wonderware historian using that source. Event-field projection supports the standard
`BaseEventType` select clauses — `EventId`, `SourceName`, `Time`, `ReceiveTime`, `Message`,
and `Severity`; an unsupported select operand returns a null field (spec-conformant).

### Graceful degradation

| Situation | HistoryRead status |
|---|---|
| Historized node, historian configured and reachable, results found | `Good` |
| Historized node, historian configured, time range empty | `GoodNoData` |
| Historized node, historian NOT configured (`Enabled=false` / Null source) | `GoodNoData` (empty) |
| Non-historized node | `BadHistoryOperationUnsupported` |
| Backend timeout or exception | `BadHistoryOperationUnsupported` per node; other nodes in the same batch are unaffected |

A historized node with no historian configured never returns an error status — it returns
empty. This means a deployment can author and publish historized tags before the historian
sidecar is provisioned, without producing error spikes in connected clients.

### Continuation-point paging (Raw)

`HistoryRead-Raw` is paged server-side. The historian backend is single-shot (it returns up to
`NumValuesPerNode` samples with no continuation point of its own), so the server synthesises
paging time-based:

- A page that returns **exactly** `NumValuesPerNode` samples (with `NumValuesPerNode > 0`) MAY
  have more behind it, so the server stores a resume cursor and returns an opaque continuation
  point (16 bytes). The client hands it back to fetch the next page.
- A short page (fewer than the cap) is the last page — no continuation point.
- `NumValuesPerNode == 0` ("all values, no limit") is never paged; the whole window returns in one
  shot.
- The resume cursor is **tie-safe**: the next page resumes from the last returned sample's
  SourceTimestamp *inclusive* and drops the boundary samples already emitted, so samples sharing
  the boundary timestamp are neither duplicated nor skipped.

> **Paging limitation — oversized tie clusters.** The tie-safe cursor is a `(timestamp, skip)`
> pair, and the single-shot backend only accepts `(start, end, cap)` — it cannot skip. So if **more
> samples share one `SourceTimestamp` than `NumValuesPerNode`** (a tie cluster larger than the page
> cap), the cursor cannot advance past that timestamp: every resume re-reads the same first `cap`
> ties. Rather than silently truncate the read to `GoodNoData` (which would permanently drop the
> un-emitted ties), the resume read fails that node **loudly** with
> `BadHistoryOperationUnsupported` and logs the tag + timestamp + cap. The operator's remedy is to
> re-issue the read with a larger `NumValuesPerNode`. For a single tag's raw history this is a data
> anomaly (raw samples normally carry strictly increasing distinct timestamps); a fully cursor-based
> fix that pages *within* a single timestamp is a possible follow-up.

Continuation points are bound to the OPC UA session (the SDK's
`ServerConfiguration.MaxHistoryContinuationPoints` cap, default 100, with oldest-eviction; points
are disposed when the session closes). Resuming an unknown / evicted / released point returns
`BadContinuationPointInvalid`. `releaseContinuationPoints` drops the stored cursors without reading
data.

### Known limitations

- **Processed and AtTime are single-shot** (no continuation points). Unlike Raw, neither
  `ReadProcessedDetails` nor `ReadAtTimeDetails` carries a client count cap (`NumValuesPerNode`):
  the Processed bucket count is deterministic (window / interval) and AtTime returns exactly one
  sample per requested timestamp, so the single-shot backend returns the complete result in one
  read and there is no "full page ⇒ maybe more" signal to page on. Returning the full result with
  no continuation point is spec-conformant.
- **No modified-value history** (`HistoryReadModified`). Requests for modified values return
  `BadHistoryOperationUnsupported`. This is **infra-gated, not a server-code gap**: the AVEVA
  Wonderware historian backend (`IHistorianDataSource`, the TCP sidecar client) exposes only a
  current-value read path — there is no modified/edited-history surface to source the data from. The
  server-side override is in place (it cleanly rejects modified reads per node) and `IsReadModified`
  is honoured; serving real modified-value history is unblocked only once the historian client/sidecar
  grows a modified-read RPC. Until then, rejecting is the correct, spec-conformant behaviour.

### Redundancy and authorization

History reads are served from any node — there is no Primary gate. Authorization is the
standard OPC UA `HistoryRead` permission enforced by the SDK through the `AccessLevels.HistoryRead`
bit set at materialization. A session without sufficient permissions receives
`BadUserAccessDenied` from the SDK before the dispatch reaches the historian.

---

## Authoring workflow

1. Open the equipment's **Tags** tab on `/uns/equipment/{id}`.
2. Create or edit the tag. Because Galaxy uses the raw-JSON editor, add `"isHistorized":true`
   (and optionally `"historianTagname":"..."`) directly in the TagConfig textarea.
3. Save and publish. The server rebuilds its address space; the node materialises with
   `Historizing=true` and the `HistoryRead` AccessLevel bit.
4. Confirm with Client.CLI `read` that the node's `Status` is `Good` and that the value
   is updating. Then issue a `historyread` to verify the historian connection returns data.

---

## Client.CLI historyread examples

The `historyread` command reads historical data from any node. Supply start and end times in
ISO 8601 UTC form. See [docs/Client.CLI.md](Client.CLI.md) for the full flag reference.

```bash
# Raw history for a historized Galaxy tag (last 24 hours by default)
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z"

# Limit to 100 values
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z" \
  --max 100

# 1-hour average aggregate
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z" \
  --aggregate Average --interval 3600000

# Authenticated read (ReadOnly role or higher required)
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z" \
  -U reader -P password
```

Supported `--aggregate` values: `Average`, `Minimum`, `Maximum`, `Count`, `Start`, `End`,
`StandardDeviation` (aliases: `avg`, `min`, `max`, `stddev`/`stdev`, `first`, `last`).
`--interval` is the processing interval in milliseconds (default 3600000 = 1 hour).

---

## Live /run gate

The live read gate requires the Wonderware historian sidecar running on the WW Historian VM
(`10.100.0.48`) and AVEVA Historian healthy. Set `ServerHistorian:Enabled=true` with the
correct `Host`, `Port`, and `SharedSecret` in `appsettings.json` (or via environment
variables), then deploy and publish at least one historized Galaxy tag. The gate is
operator-driven — it is not part of the local docker-dev rig.

See [AlarmHistorian.md](AlarmHistorian.md) for the historian sidecar setup and
[ServiceHosting.md](ServiceHosting.md) for the sidecar service configuration.

---

## See also

- [docs/plans/2026-06-14-galaxy-phase-c-historian-design.md](plans/2026-06-14-galaxy-phase-c-historian-design.md) — full design and implementation notes
- [AlarmHistorian.md](AlarmHistorian.md) — alarm write path; shares the same Wonderware sidecar
- [AlarmTracking.md](AlarmTracking.md) — OPC UA Part 9 alarm surface (event history source)
- [Client.CLI.md](Client.CLI.md) — full `historyread` flag reference
- [ScriptedAlarms.md](ScriptedAlarms.md) §"Native driver alarms" — the Phase B `alarm` object in `TagConfig` (parallel carrier)