# Historian — server-side OPC UA HistoryRead (Phase C)

Phase C wires server-side OPC UA **HistoryRead** for authored equipment tags flagged
historized. The feature is driver-agnostic: any equipment tag (Galaxy, Modbus, OpcUaClient,
or any other driver) can be marked historized; the server dispatches all history reads to the
registered `IHistorianDataSource` — the **HistorianGateway** read client
(`GatewayHistorianDataSource`, talking gRPC to the external `ZB.MOM.WW.HistorianGateway` via the
`ZB.MOM.WW.HistorianGateway.Client` package). No EF migration is required; the historian flag rides in
the existing schemaless `TagConfig` JSON blob alongside the Phase B `alarm` object. (The bespoke
Wonderware TCP sidecar backend this replaced was retired — see
[drivers/Historian.Wonderware.md](drivers/Historian.Wonderware.md).)

Design reference: [docs/plans/2026-06-14-galaxy-phase-c-historian-design.md](plans/2026-06-14-galaxy-phase-c-historian-design.md).

---

## Historized TagConfig schema

A tag is historized by setting the **Historize this tag** checkbox and optional **Historian
tagname (override)** textbox in the Tag modal on the `/uns` equipment page Tags tab. These
controls work for **all drivers** — typed editors (Modbus, S7, OpcUaClient, etc.) and the
raw-JSON textarea (Galaxy) alike. The controls merge `isHistorized` / `historianTagname`
into the existing `TagConfig` JSON blob via the `TagHistorizeConfig` helper, preserving all
other keys byte-stable. Drivers that still use the raw-JSON editor (Galaxy) can also add the
fields directly in the textarea.

### Fields

| Field | Type | Required | Description |
|---|---|---|---|
| `isHistorized` | bool | yes | Marks the tag historized. Materialises the OPC UA node with `Historizing=true` and the `HistoryRead` AccessLevel bit. |
| `historianTagname` | string | no | Explicit historian tagname to query. When absent or empty, defaults to the tag's driver `FullName` (the `TagConfig.FullName`). |

### Examples

Default historian tagname (uses `FullName` = `TestMachine_002.TestFloat`):

```json
{"FullName":"TestMachine_002.TestFloat","isHistorized":true}
```

Explicit historian tagname override:

```json
{"FullName":"TestMachine_002.TestFloat","isHistorized":true,"historianTagname":"Plant.Line1.Flow"}
```

A tag that is both historized and a native alarm:

```json
{"FullName":"TestMachine_002.HiAlarm","isHistorized":true,"alarm":{"alarmType":"OffNormalAlarm","severity":700}}
```

---

## ServerHistorian configuration

The `ServerHistorian` section in `appsettings.json` controls the historian read path.
`Enabled` defaults to `false`; when disabled, the server registers `NullHistorianDataSource`
and all HistoryRead calls on historized nodes return `GoodNoData` (empty, not an error).

```json
{
  "ServerHistorian": {
    "Enabled": false,
    "Endpoint": "",
    "ApiKey": "",
    "UseTls": true,
    "AllowUntrustedServerCertificate": false,
    "CaCertificatePath": null,
    "CallTimeout": "00:00:30",
    "MaxTieClusterOverfetch": 65536
  }
}
```

| Key | Type | Default | Description |
|---|---|---|---|
| `Enabled` | bool | `false` | Enable the live `GatewayHistorianDataSource`. `false` → `NullHistorianDataSource` (empty reads). |
| `Endpoint` | string | `""` | Absolute gateway URI, e.g. `https://host:5222`. Scheme selects transport (`https://` = TLS, `http://` = h2c plaintext). Required when `Enabled`. |
| `ApiKey` | string | `""` | The gateway peppered-HMAC key (`histgw_<id>_<secret>`) sent as `Authorization: Bearer`. Required when `Enabled`. **Supply via env `ServerHistorian__ApiKey`.** |
| `UseTls` | bool | `true` | Connect over TLS; must match the `Endpoint` scheme. |
| `AllowUntrustedServerCertificate` | bool | `false` | Accept a self-signed / untrusted server certificate (dev / on-prem only). |
| `CaCertificatePath` | string\|null | `null` | PEM CA file pinning the gateway's TLS chain. Null/empty uses the OS trust store. |
| `CallTimeout` | TimeSpan | `00:00:30` | Per-call deadline applied to each unary gateway read. |
| `MaxTieClusterOverfetch` | int | `65536` | Maximum samples the server will fetch in one shot to page through a tie cluster (multiple samples sharing one `SourceTimestamp`). A cluster larger than this ceiling fails `BadHistoryOperationUnsupported`. Raise to handle abnormally large tie clusters; the default covers all normal-data cases. |

> **Do not commit `ApiKey` to `appsettings.json`.** Set it via the environment variable
> `ServerHistorian__ApiKey`, a secrets store, or a deployment-time overlay. The checked-in default is
> always empty.

> **Gateway-side prerequisites.** The target gateway must run `RuntimeDb:Enabled=true` (continuous
> `WriteLiveValues`) + `RuntimeDb:EventReadsEnabled=true` (alarm-history `ReadEvents`), and the API key
> must carry the scopes `historian:read`, `historian:write`, `historian:tags:write`.

> **Migration from the Wonderware backend.** Rename the old keys: `Host`/`Port` → `Endpoint`
> (`https://host:5222`); `SharedSecret` → `ApiKey` (env `ServerHistorian__ApiKey`);
> `ServerCertThumbprint` → `CaCertificatePath` (+ `UseTls` / `AllowUntrustedServerCertificate`).

The `ServerHistorian` section is independent of the `AlarmHistorian` section (the alarm
write path) and the `ContinuousHistorization` section (driver-value capture). All three target the
**same** gateway — but only `ServerHistorian` carries the connection (endpoint/key/TLS); the other two
source it from there.

---

## HistoryRead behavior

### Read variants

The server supports all four OPC UA HistoryRead variants:

| Variant | Node type | CLI `--aggregate` |
|---|---|---|
| **Raw** | Historized variable | (omit `--aggregate`) |
| **Processed** | Historized variable | `Average`, `Minimum`, `Maximum`, `Total`, `Count` |
| **AtTime** | Historized variable | n/a (client supplies exact timestamps) |
| **Events** | Equipment folder (event notifier) | n/a |

**Variable nodes** (historized tags) serve Raw, Processed, and AtTime history.
`Historizing=true` and `AccessLevels.HistoryRead` are set at materialization so any compliant
OPC UA client can discover historized capability from the node's attributes.

**Equipment-folder event-notifier nodes** serve Event history. Every equipment folder that
owns at least one alarm condition is already an event notifier; the server registers a
`sourceName` (the equipment id) for each such folder and maps event history reads to the
HistorianGateway using that source. (Alarm-history `ReadEvents` requires the gateway running
`RuntimeDb:EventReadsEnabled=true`.) Event-field projection supports the standard
`BaseEventType` select clauses — `EventId`, `SourceName`, `Time`, `ReceiveTime`, `Message`,
and `Severity`; an unsupported select operand returns a null field (spec-conformant).

### OpcUaClient driver — upstream passthrough for all four variants

The OpcUaClient driver's `IHistoryProvider` implementation forwards **all four** history-read
variants (Raw, Processed, AtTime, and Events) to its upstream OPC UA server. For the Events
variant it sends a fixed canonical `BaseEventType` `EventFilter` selecting the standard six
fields (`EventId`, `SourceName`, `Time`, `ReceiveTime`, `Message`, `Severity`) and maps the
upstream `HistoryEvent` onto `HistoricalEvent` — the same six-field projection the OtOpcUa
node-manager itself projects when serving event history. This is a **driver-level capability**:
the OpcUaClient driver acts as a passthrough to whatever historian the upstream server exposes,
and is independent of the single server-side `IHistorianDataSource` backend
(`GatewayHistorianDataSource` / `NullHistorianDataSource`) that the OtOpcUa node-manager
dispatches HistoryRead to for tags on other drivers (Galaxy, Modbus, S7, etc.).

### Graceful degradation

| Situation | HistoryRead status |
|---|---|
| Historized node, historian configured and reachable, results found | `Good` |
| Historized node, historian configured, time range empty | `GoodNoData` |
| Historized node, historian NOT configured (`Enabled=false` / Null source) | `GoodNoData` (empty) |
| Non-historized node | `BadHistoryOperationUnsupported` |
| Backend timeout or exception | `BadHistoryOperationUnsupported` per node; other nodes in the same batch are unaffected |

A historized node with no historian configured never returns an error status — it returns
empty. This means a deployment can author and publish historized tags before the historian
gateway is provisioned, without producing error spikes in connected clients.

### Continuation-point paging (Raw)

`HistoryRead-Raw` is paged server-side. The historian backend is single-shot (it returns up to
`NumValuesPerNode` samples with no continuation point of its own), so the server synthesises
paging time-based:

- A page that returns **exactly** `NumValuesPerNode` samples (with `NumValuesPerNode > 0`) MAY
  have more behind it, so the server stores a resume cursor and returns an opaque continuation
  point (16 bytes). The client hands it back to fetch the next page.
- A short page (fewer than the cap) is the last page — no continuation point.
- `NumValuesPerNode == 0` ("all values, no limit") is never paged; the whole window returns in one
  shot.
- The resume cursor is **tie-safe**: the next page resumes from the last returned sample's
  SourceTimestamp *inclusive* and drops the boundary samples already emitted, so samples sharing
  the boundary timestamp are neither duplicated nor skipped.

> **Oversized tie clusters — within-timestamp paging.** When more samples share one
> `SourceTimestamp` than the current page cap, the server detects that the cursor has stalled on
> a tie cluster (the last returned timestamp equals the resume timestamp). It then **over-fetches
> the entire cluster** at that single timestamp up to a bounded ceiling controlled by
> `ServerHistorian:MaxTieClusterOverfetch` (default **65 536**), then serves the cluster
> `NumValuesPerNode` samples at a time across successive pages, advancing the cursor one tick past
> the timestamp once the cluster is fully drained.
>
> **Short pages within a cluster still carry a continuation point.** A within-cluster page that
> returns fewer than `NumValuesPerNode` samples (because the cluster happened to be smaller than
> the cap, or is the final partial batch) is not the last page if the cluster itself has not been
> fully emitted — the server retains the continuation point so the client can drain the remainder.
> Only when the cluster is exhausted and the cursor has advanced past the timestamp does the
> short-page rule apply.
>
> **Cluster larger than `MaxTieClusterOverfetch`.** If the over-fetch itself reaches the ceiling
> without spanning the full cluster, the node fails **loudly** with
> `BadHistoryOperationUnsupported` and the tag + timestamp + ceiling are logged. Remedies: raise
> `MaxTieClusterOverfetch` (or `NumValuesPerNode`) to cover the full cluster, or investigate the
> data anomaly (raw samples normally carry strictly increasing distinct timestamps).
>
> For a single tag's raw history a tie cluster larger than the default 65 536 is a severe data
> anomaly. The ceiling exists to bound server-side memory on pathological data, not to cap normal
> operation.

Continuation points are bound to the OPC UA session (the SDK's
`ServerConfiguration.MaxHistoryContinuationPoints` cap, default 100, with oldest-eviction; points
are disposed when the session closes). Resuming an unknown / evicted / released point returns
`BadContinuationPointInvalid`. `releaseContinuationPoints` drops the stored cursors without reading
data.

### Total aggregate

The OPC UA `Total` aggregate is **supported** over the HistorianGateway backend. The gateway exposes a
native **`Integral`** retrieval mode, so `Total` maps straight to it (`HistoryAggregateType.Total →
RetrievalMode.Integral`) — no client-side scaling. (This replaces the retired Wonderware path, which had no
`Total` column and derived it client-side as time-weighted `Average × interval-seconds`.) `Count` is
likewise a native gateway mode. Bucket status codes and timestamps are preserved unchanged; empty / null
buckets surface as `BadNoData`.

### Known limitations

- **Processed and AtTime are single-shot** (no continuation points). Unlike Raw, neither
  `ReadProcessedDetails` nor `ReadAtTimeDetails` carries a client count cap (`NumValuesPerNode`):
  the Processed bucket count is deterministic (window / interval) and AtTime returns exactly one
  sample per requested timestamp, so the single-shot backend returns the complete result in one
  read and there is no "full page ⇒ maybe more" signal to page on. Returning the full result with
  no continuation point is spec-conformant.
- **No modified-value history** (`HistoryReadModified`). Requests for modified values return
  `BadHistoryOperationUnsupported`. This is **infra-gated, not a server-code gap**: the HistorianGateway
  backend (`GatewayHistorianDataSource`) exposes only a current-value read path — there is no
  modified/edited-history surface to source the data from. The server-side override is in place (it cleanly
  rejects modified reads per node) and `IsReadModified` is honoured; serving real modified-value history is
  unblocked only once the gateway grows a modified-read RPC. Until then, rejecting is the correct,
  spec-conformant behaviour.

### Redundancy and authorization

History reads are served from any node — there is no Primary gate. Authorization is the
standard OPC UA `HistoryRead` permission enforced by the SDK through the `AccessLevels.HistoryRead`
bit set at materialization. A session without sufficient permissions receives
`BadUserAccessDenied` from the SDK before the dispatch reaches the historian.

---

## Authoring workflow

1. Open the equipment's **Tags** tab on `/uns/equipment/{id}`.
2. Create or edit the tag.
   - For **typed-editor drivers** (Modbus, S7, OpcUaClient, etc.): check the
     **Historize this tag** checkbox and, if needed, fill in the **Historian tagname
     (override)** textbox.
   - For **raw-JSON editors** (Galaxy): you can check the same first-class checkboxes
     (they appear below the JSON textarea), or add `"isHistorized":true` (and optionally
     `"historianTagname":"..."`) directly in the textarea.
3. Save and publish. The server rebuilds its address space; the node materialises with
   `Historizing=true` and the `HistoryRead` AccessLevel bit.
4. Confirm with Client.CLI `read` that the node's `Status` is `Good` and that the value
   is updating. Then issue a `historyread` to verify the historian connection returns data.

### Native-alarm historian opt-out (`alarm.historizeToAveva`)

A tag carrying a native `"alarm"` object has a **separate** historian opt-out for its
**alarm transitions** (distinct from tag-value historization). On the Tag modal, check or
uncheck **Historize to AVEVA** in the alarm section. This maps to `alarm.historizeToAveva`
(`bool?`) in the `TagConfig` JSON:

- **Absent or `true`** (default) — the alarm's transitions are written to AVEVA Historian
  via `HistorianAdapterActor`.
- **`false`** — the durable AVEVA write is suppressed for this alarm's transitions. The
  live `/alerts` feed and OPC UA condition events are unaffected.

The gate is applied in `HistorianAdapterActor` using `is not false` semantics, matching the
scripted-alarm `HistorizeToAveva` posture. See
[ScriptedAlarms.md §Native driver alarms](ScriptedAlarms.md#native-driver-alarms-equipment-tag-path)
and [AlarmTracking.md §Historian write-back](AlarmTracking.md#historian-write-back-non-galaxy-alarms)
for the full alarm-historian routing.

---

## Client.CLI historyread examples

The `historyread` command reads historical data from any node. Supply start and end times in
ISO 8601 UTC form. See [docs/Client.CLI.md](Client.CLI.md) for the full flag reference.

```bash
# Raw history for a historized Galaxy tag (last 24 hours by default)
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z"

# Limit to 100 values
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z" \
  --max 100

# 1-hour average aggregate
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z" \
  --aggregate Average --interval 3600000

# Authenticated read (ReadOnly role or higher required)
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=EQ-55297329838d/GalaxyTestTag" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z" \
  -U reader -P password
```

Supported `--aggregate` values: `Average`, `Minimum`, `Maximum`, `Count`, `Start`, `End`,
`StandardDeviation` (aliases: `avg`, `min`, `max`, `stddev`/`stdev`, `first`, `last`).
`--interval` is the processing interval in milliseconds (default 3600000 = 1 hour).
The `Total` aggregate is served by the server's OPC UA HistoryRead endpoint for any conformant
client (derived as time-weighted Average × interval-seconds — see "Total aggregate derivation"
above), but is not exposed by this bundled CLI.

---

## Live /run gate

The live read gate requires a reachable `ZB.MOM.WW.HistorianGateway` (VPN to `wonder-sql-vd03`) with the
AVEVA Historian behind it healthy. Set `ServerHistorian:Enabled=true` with the correct `Endpoint`
(`https://host:5222`) and supply `ServerHistorian__ApiKey` via the environment, then deploy and publish at
least one historized Galaxy tag. The gate is operator-driven — it is not part of the local docker-dev rig.
The gateway-backed driver also ships an env-gated live suite (`Category=LiveIntegration`); see the
`HISTGW_GATEWAY_ENDPOINT` / `HISTGW_GATEWAY_APIKEY` / `HISTGW_TEST_TAG` / `HISTGW_WRITE_SANDBOX_TAG` /
`HISTGW_ALARM_SOURCE` env vars (it skips cleanly when they are absent).

See [AlarmHistorian.md](AlarmHistorian.md) for the alarm write path and
[ServiceHosting.md](ServiceHosting.md) for the (external) HistorianGateway deployment.

---

---

## Cross-driver TagConfig key reference

The `TagConfig` JSON blob is the single extension surface for per-tag server-side
behaviours across all drivers.  Keys from multiple phases coexist in the same blob;
unknown keys are preserved byte-stable on round-trip by all typed editors.

### Historian + alarm keys (Phase B / Phase C)

| Key | Type | Phase | Description |
|---|---|---|---|
| `isHistorized` | bool | C | Marks the tag historized (HistoryRead + Historizing=true). |
| `historianTagname` | string | C | Explicit historian tagname override (defaults to `FullName`). |
| `alarm` | object | B | Native alarm definition (`alarmType`, `severity`, `historizeToAveva`, …). |

### Array keys (Phase 4c)

| Key | Type | Phase | Description |
|---|---|---|---|
| `isArray` | bool | 4c | When `true` and `arrayLength >= 1`, materialises the node as a 1-D array (`ValueRank=OneDimension`). Absent or `false` → scalar. |
| `arrayLength` | uint (≥1) | 4c | Element count; sets `ArrayDimensions`. Required when `isArray: true`. |

### Cross-driver array coverage matrix

| Driver | Read mechanism | Live-verify status |
|---|---|---|
| **Modbus** | Contiguous FC03/FC04 block; String/BitInRegister modes supported | Mac-verifiable (sim `10.100.0.35:5020`) |
| **S7** | `ReadBytesAsync` block + per-element decode loop | Unit-proven (fixture down) |
| **AB CIP** | libplctag native array read (atomic + UDT member arrays) | Unit-proven (fixture down) |
| **AB Legacy** | PCCC multi-element file read via libplctag (cap 256 elements) | Unit-proven (fixture down) |
| **TwinCAT** | ADS native array symbol read | Unit-proven (fixture down) |

Array writes, multi-dimensional arrays (ValueRank>1), and per-element historization are out of scope — see [Uns.md §Array tags](Uns.md#array-tags-1-d).

---

## Closed stillpending.md §2 items (Phase 4 / Phase 4b)

The following items from the `stillpending.md` backlog §2 were closed by earlier
phases and are recorded here so future audits don't re-flag them.

| Item | Closed by | Commit |
|---|---|---|
| Modbus Int64 / UInt64 OPC UA node `DataType` correction | Phase 4 | `bd8fee61` |
| `HistoryAggregateType.Total` — `Total` aggregate | Phase 4 (derived client-side as time-weighted Average × interval-seconds) | `5e27b5f7` |
| Historian poison alarm-event indefinite retry — dead-letter cap (task #437) | Phase 4 | `fcb38014` |

---

## See also

- [docs/plans/2026-06-14-galaxy-phase-c-historian-design.md](plans/2026-06-14-galaxy-phase-c-historian-design.md) — full design and implementation notes
- [AlarmHistorian.md](AlarmHistorian.md) — alarm write path; drains to the same HistorianGateway (`SendEvent`)
- [AlarmTracking.md](AlarmTracking.md) — OPC UA Part 9 alarm surface (event history source)
- [Client.CLI.md](Client.CLI.md) — full `historyread` flag reference
- [ScriptedAlarms.md](ScriptedAlarms.md) §"Native driver alarms" — the Phase B `alarm` object in `TagConfig` (parallel carrier)
- [Uns.md §Array tags](Uns.md#array-tags-1-d) — `isArray`/`arrayLength` keys, cross-driver coverage, and deferrals