lmxopcua/docs/Historian.md

# Historian — server-side OPC UA HistoryRead (Phase C)

Phase C wires server-side OPC UA **HistoryRead** for authored equipment tags flagged
historized. The feature is driver-agnostic: any equipment tag (Galaxy, Modbus, OpcUaClient,
or any other driver) can be marked historized; the server dispatches all history reads to the
registered `IHistorianDataSource` — the **HistorianGateway** read client
(`GatewayHistorianDataSource`, talking gRPC to the external `ZB.MOM.WW.HistorianGateway` via the
`ZB.MOM.WW.HistorianGateway.Client` package). No EF migration is required; the historian flag rides in
the existing schemaless `TagConfig` JSON blob alongside the Phase B `alarm` object. (The bespoke
Wonderware TCP sidecar backend this replaced was retired — see
[drivers/Historian.Wonderware.md](drivers/Historian.Wonderware.md).)

Design reference: [docs/plans/2026-06-14-galaxy-phase-c-historian-design.md](plans/2026-06-14-galaxy-phase-c-historian-design.md).

---

## Historized TagConfig schema

A tag is historized by setting the **Historize this tag** checkbox and optional **Historian
tagname (override)** textbox in the Tag modal on the `/uns` equipment page Tags tab. These
controls work for **all drivers** — typed editors (Modbus, S7, OpcUaClient, etc.) and the
raw-JSON textarea (Galaxy) alike. The controls merge `isHistorized` / `historianTagname`
into the existing `TagConfig` JSON blob via the `TagHistorizeConfig` helper, preserving all
other keys byte-stable. Drivers that still use the raw-JSON editor (Galaxy) can also add the
fields directly in the textarea.

### Fields

| Field | Type | Required | Description |
|---|---|---|---|
| `isHistorized` | bool | yes | Marks the tag historized. Materialises the OPC UA node with `Historizing=true` and the `HistoryRead` AccessLevel bit. |
| `historianTagname` | string | no | Explicit historian tagname to query. When absent or empty, defaults to the tag's driver `FullName` (the `TagConfig.FullName`). |

### Examples

Default historian tagname (uses `FullName` = `TestMachine_002.TestFloat`):

```json
{"FullName":"TestMachine_002.TestFloat","isHistorized":true}
```

Explicit historian tagname override:

```json
{"FullName":"TestMachine_002.TestFloat","isHistorized":true,"historianTagname":"Plant.Line1.Flow"}
```

A tag that is both historized and a native alarm:

```json
{"FullName":"TestMachine_002.HiAlarm","isHistorized":true,"alarm":{"alarmType":"OffNormalAlarm","severity":700}}
```

---

## ServerHistorian configuration

The `ServerHistorian` section in `appsettings.json` controls the historian read path.
`Enabled` defaults to `false`; when disabled, the server registers `NullHistorianDataSource`
and all HistoryRead calls on historized nodes return `GoodNoData` (empty, not an error).

```json
{
  "ServerHistorian": {
    "Enabled": false,
    "Endpoint": "",
    "ApiKey": "",
    "UseTls": true,
    "AllowUntrustedServerCertificate": false,
    "CaCertificatePath": null,
    "CallTimeout": "00:00:30",
    "MaxTieClusterOverfetch": 65536,
    "HistoryReadBatchParallelism": 4,
    "MaxConcurrentHistoryReadBatches": 16,
    "HistoryReadDeadline": "00:01:00"
  }
}
```

| Key | Type | Default | Description |
|---|---|---|---|
| `Enabled` | bool | `false` | Enable the live `GatewayHistorianDataSource`. `false` → `NullHistorianDataSource` (empty reads). |
| `Endpoint` | string | `""` | Absolute gateway URI, e.g. `https://host:5222`. Scheme selects transport (`https://` = TLS, `http://` = h2c plaintext). Required when `Enabled`. |
| `ApiKey` | string | `""` | The gateway peppered-HMAC key (`histgw_<id>_<secret>`) sent as `Authorization: Bearer`. Required when `Enabled`. **Supply via env `ServerHistorian__ApiKey`.** |
| `UseTls` | bool | `true` | Connect over TLS; must match the `Endpoint` scheme. |
| `AllowUntrustedServerCertificate` | bool | `false` | Accept a self-signed / untrusted server certificate (dev / on-prem only). |
| `CaCertificatePath` | string\|null | `null` | PEM CA file pinning the gateway's TLS chain. Null/empty uses the OS trust store. |
| `CallTimeout` | TimeSpan | `00:00:30` | Per-call deadline applied to each unary gateway read. A read for a tag the historian does not yet know about — e.g. a freshly-historized tag that has **not** been provisioned (see [Tag auto-provisioning](#tag-auto-provisioning-ensuretags)) — can run to this **full** deadline before the gateway errors, so an unprovisioned-tag HistoryRead can block for ~30 s. Lower this to fail faster, at the cost of truncating legitimately long reads. |
| `MaxTieClusterOverfetch` | int | `65536` | Maximum samples the server will fetch in one shot to page through a tie cluster (multiple samples sharing one `SourceTimestamp`). A cluster larger than this ceiling fails `BadHistoryOperationUnsupported`. Raise to handle abnormally large tie clusters; the default covers all normal-data cases. |
| `HistoryReadBatchParallelism` | int | `4` | Max historized nodes served **concurrently** within one HistoryRead batch (arch-review 03/S3). The SDK's `HistoryRead*` override surface is synchronous, so each arm bridges once at its boundary; this turns per-node serving from sequential (`N × CallTimeout` worst case) into `⌈N / P⌉ × CallTimeout`. Also caps the gateway-load multiplication a single client can command. `≤ 1` serves sequentially. |
| `MaxConcurrentHistoryReadBatches` | int | `16` | Process-wide limit on HistoryRead batches served **concurrently** (arch-review 03/S3). A flood of history clients degrades to `BadTooManyOperations` rather than exhausting the SDK request-thread pool. With `HistoryReadBatchParallelism` this caps total in-flight gateway reads at `MaxConcurrentHistoryReadBatches × HistoryReadBatchParallelism` (default 16 × 4 = 64). A batch that cannot get a slot within `min(HistoryReadDeadline, 5 s)` fails every handle `BadTooManyOperations`. |
| `HistoryReadDeadline` | TimeSpan | `00:01:00` | Per-request wall-clock deadline for a HistoryRead batch (arch-review 03/S3). A node still in flight at the deadline surfaces `BadTimeout`, so one request can never hold an SDK request thread longer than this regardless of node count. Non-positive = unbounded (a `Validate()` warning). |

> **Do not commit `ApiKey` to `appsettings.json`.** Set it via the environment variable
> `ServerHistorian__ApiKey`, a secrets store, or a deployment-time overlay. The checked-in default is
> always empty.

> **Gateway-side prerequisites.** The target gateway must run `RuntimeDb:Enabled=true` (continuous
> `WriteLiveValues`) + `RuntimeDb:EventReadsEnabled=true` (alarm-history `ReadEvents`), and the API key
> must carry the scopes `historian:read`, `historian:write`, `historian:tags:write`.

> **Migration from the Wonderware backend.** Rename the old keys: `Host`/`Port` → `Endpoint`
> (`https://host:5222`); `SharedSecret` → `ApiKey` (env `ServerHistorian__ApiKey`);
> `ServerCertThumbprint` → `CaCertificatePath` (+ `UseTls` / `AllowUntrustedServerCertificate`).

The `ServerHistorian` section is independent of the `AlarmHistorian` section (the alarm
write path) and the `ContinuousHistorization` section (driver-value capture). All three target the
**same** gateway — but only `ServerHistorian` carries the connection (endpoint/key/TLS); the other two
source it from there.

> **Continuous-historization value quality is server-stamped.** The gateway's `WriteLiveValues` SQL path
> **discards** the quality the recorder sends (a server-managed column; 0.2.0 Contracts C-002), so persisted
> quality is stamped gateway-side regardless of what the `ContinuousHistorizationRecorder` provides (it
> hardcodes OPC-DA Good, `192`). Capturing real per-node quality will not persist until the gateway honors
> the field — see `GatewayHistorianValueWriter`'s remarks (archreview 06/C-7).

> ✅ **Continuous value capture is LIVE-PROVEN END-TO-END
> ([#491](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/491) CLOSED — gate run 2026-08-01,
> completed 2026-08-04; see `docs/plans/2026-08-01-continuous-historization-live-gate.md`).** Against a
> real `ZB.MOM.WW.HistorianGateway` talking to the real AVEVA Historian on `wonder-sql-vd03`: deploy →
> `EnsureTags` provisioning → dependency-mux value delta → FasterLog outbox append → drain →
> `WriteLiveValues` → **value read back from the historian**, plus **restart convergence** (the recorder
> re-registers the deployed ref set after a process restart, with no new deployment).
>
> The read-back used two registers carrying **distinct non-zero** constants (`8224`, `120`), each of
> which read back as its own value — a `0`-valued source cannot distinguish a correct value from a
> wrong one.
>
> ⚠️ The gate found a **production-fatal defect**: the recorder threw `NotSupportedException` on the
> **first value it ever appended**, because it touched `Self` after an `await …ConfigureAwait(false)`,
> which discards Akka's ActorContext. Supervision restarted the actor, and since the recorder is seeded
> with an *empty* ref set, the restart discarded every historized ref — historization stopped after one
> value. Fixed; pinned by a regression test whose outbox fake completes on a **pool** thread (a fake
> returning `ValueTask.CompletedTask`, or even `await Task.Yield()`, cannot reproduce it).
>
> The gate also found the two defects below, both since fixed.
>
> **Related, separately open:** `EnsureTags` provisioning fires only for **newly-added** historized
> tags (`AddressSpaceApplier` iterates the added set), so flipping an **existing** tag to
> `"isHistorized": true` never provisions it — see
> `docs/plans/2026-06-27-historian-gateway-integration-issues.md` Issue 5.

---

## ContinuousHistorization configuration

`ContinuousHistorizationOptions` binds the `ContinuousHistorization` section. It carries **only** the
recorder + outbox knobs — the gateway connection (endpoint / key / TLS) is sourced from
`ServerHistorian` (above), never restated here.

| Key | Type | Default | Description |
|---|---|---|---|
| `Enabled` | bool | `false` | `true` **and** a configured `ServerHistorian` wires the FasterLog outbox + gateway value-writer and spawns the `ContinuousHistorizationRecorder`. `false` → no recorder; driver tag values are not historized. |
| `OutboxPath` | string | `""` | **Directory** holding the FasterLog segment + commit files. Required when `Enabled`. In production set an **absolute** path on durable storage — a relative path resolves against the host's working directory. |
| `CommitMode` | string | `PerEntry` | `PerEntry` fsyncs before the append returns (no loss window); `Periodic` batches commits onto a background timer every `CommitIntervalMs`. Parsed case-insensitively; an unrecognized value falls back to `PerEntry`. **See the batch-commit note below — `PerEntry` is one fsync per ingest *burst*, not per value.** |
| `CommitIntervalMs` | int | `100` | Periodic-mode commit cadence (ms); must be `> 0` under `Periodic`, ignored under `PerEntry`. |
| `DrainBatchSize` | int | `1024` | Outbox entries peeked + written per drain pass. **Raised from 64**: the peek is an in-memory scan of the outbox's live index (widening it costs allocation, not I/O), and a pass is now bounded-parallel over `DrainConcurrency` rather than sequential, so a wide batch no longer costs `tags × latency`. At 64 a drain drew down at most 64 entries every `DrainIntervalSeconds` — below the arrival rate of even a modest historized tag set — so any gateway hiccup left the outbox permanently behind, growing toward `Capacity` and shedding oldest samples instead of catching up. |
| `DrainIntervalSeconds` | double | `2` | Steady drain cadence (and the post-success reschedule). |
| `DrainConcurrency` | int | `8` | Historian tags a single drain pass writes **concurrently**. One peeked batch is grouped per historian tag and each group is an independent gateway round-trip, so a sequential pass cost `tags × latency`; this window makes it `⌈tags / concurrency⌉ × latency` while bounding how many round-trips can be outstanding against the gateway at once. `1` restores the fully sequential drain; a value `< 1` is clamped to 1 (with a `Validate()` warning). **The ack is unchanged** — still all-or-nothing after every group has reported, and the #538 per-tag quarantine still keys on "another group wrote this pass". |
| `Capacity` | int | `500000` | Max un-acked outbox entries before the drop-oldest policy sheds the front of the queue; `0` = unbounded. Roughly 50 MB of FasterLog. The #538 **disk backstop** for the one case the per-tag quarantine deliberately cannot resolve — a historian unreachable or refusing *every* write, where nothing proves any particular tag is at fault so the recorder correctly retains. |
| `MaxWriteAttempts` | int | `5` | #538 poison-tag budget: consecutive failed drain passes one historian tag may accumulate before its queued entries are abandoned and the FIFO head is allowed to advance. **Counted only across passes where *other* tags wrote successfully**, so an ordinary gateway outage never advances it. Clamped to at least 1; each abandonment logs a Warning naming the tag. |
| `MinBackoffSeconds` | double | `1` | Initial retry backoff after a failed drain pass. |
| `MaxBackoffSeconds` | double | `30` | Cap on the exponential retry backoff. |

> **`PerEntry` now means one fsync per ingest *burst*, not per value.** The recorder appends a whole
> value burst through `IHistorizationOutbox.AppendBatchAsync`, which enqueues every entry and then
> issues a **single** group commit — so a burst of N values costs one fsync rather than N. The
> durability contract is unchanged: the call does not return until that commit completes, so nothing
> the caller has been told is buffered can be lost to an unclean restart. (The documented residual is
> unchanged too: a throw *before* the commit leaves the enqueued prefix tracked-but-uncommitted —
> served correctly for the life of the process, absent after an unclean restart — exactly the
> enqueue-then-commit window the single-entry path always had, widened from one entry to the batch.)

> **Cross-repo follow-up — a multi-tag batch `WriteLiveValues` on the gateway contract.** The drain's
> unit of work is one gateway round-trip **per historian tag**, because
> `ZB.MOM.WW.HistorianGateway.Client` **0.3.0** exposes no RPC that writes values for more than one tag
> in a call. Bounded parallelism (`DrainConcurrency`) is therefore the ceiling: it overlaps the
> round-trips but cannot remove them, so a wide tag set still pays `⌈tags / concurrency⌉` network
> hops per drain pass. Collapsing that to a single hop needs a **batch/multi-tag `WriteLiveValues`** on
> the `historian_gateway.v1` contract in the `~/Desktop/HistorianGateway` sister repo. Until that
> lands, raising `DrainBatchSize` widens each hop's payload but not the hop count — which is why the
> concurrency window, not the batch width, is what governs drain throughput today.

---

## A poison tag no longer pins the outbox (#538)

`DrainOnceAsync` peeks a batch, writes **every** per-tag group in it, and then acks **all-or-nothing** —
because the outbox's `RemoveAsync` truncates the FIFO head *through* the acked id, so acking one tag's
entries could silently drop older, unwritten entries belonging to another. That much is deliberate and
unchanged.

What was wrong is what happened when a tag failed **permanently**: the head never advanced, so every
**healthy** tag in the batch was re-submitted to the historian on every drain pass, indefinitely —
duplicate samples at identical timestamps, and an outbox that never truncated (observed live at 8.3 MB
from a single unprovisionable tag, with the gateway's `ok` **and** `error` counters climbing together —
the signature of this bug).

The drain now tracks **consecutive failures per historian tag** and abandons a tag's queued entries once
it exceeds `ContinuousHistorization:MaxWriteAttempts` (default 5), letting the head advance. Two
properties matter:

- **A pass where NOTHING wrote abandons nothing.** The counter only advances on a pass where at least one
  *other* tag wrote successfully — that is what distinguishes "this tag is poison" from "the backend is
  down". A count-only quarantine looks identical on the poison-tag case and silently discards the whole
  backlog during an ordinary gateway restart, turning a duplicate-samples bug into a lost-history one.
  Pinned by a falsification-control test (`A_backend_that_fails_every_tag_abandons_nothing`) that goes red
  when the evidence check is removed while the poison-tag test stays green.
- **A tag over budget waits for the others.** Abandonment is deferred until *every* failing tag in the
  batch is over budget, so a tag is never discarded on a pass that could not have advanced the head anyway.

Each abandonment logs a Warning naming the tag and the discarded sample count, and the total rides
`RecorderStatus.AbandonedEntries`. The most common cause is a tag the historian has no definition for —
easy to reach, because `EnsureTags` provisioning runs only for newly-**added** historized tags, so a
changed `historianTagname` is never provisioned (Issue 5 above).

`ContinuousHistorization:Capacity` now also defaults to **500,000** entries rather than unbounded. It is
the disk backstop for the one case the quarantine deliberately cannot resolve — a historian refusing
*every* write, where retaining is correct. Set `0` to restore unbounded retention.

## Only the recording Primary captures (#537)

Both nodes of a redundant pair serve the same address space and see the same dependency-mux fan-out, so
an ungated recorder wrote **every sample twice** (measured: the gateway's write counter advanced twice
per value with both centrals up, and halved when one was stopped).

The recorder now reads a capture gate per value, fed from `PrimaryGatePolicy.ShouldRecordHistorizedValues`
via `IHistorizationRoleView` — the same shape as the alarm store-and-forward sink's `drainGate`, with two
deliberate differences:

- **It gates the CAPTURE, not the drain.** The alarm queue lives in LocalDb and *replicates*, so the
  Secondary already holds the Primary's rows and gating the drain loses nothing. The historization outbox
  is a node-local FasterLog replicating to nobody: gating only its drain would leave the standby appending
  forever to a queue no one empties — unbounded disk, and a flood of stale duplicates the moment it is
  promoted. Entries a node already captured are still drained by that node, so a demotion loses nothing
  recorded.
- **The relevant peer is one that records the same refs**, not one that holds the same rows. A node stands
  down only for a Primary in its own per-cluster mesh. A node carrying no cluster-scoped role cannot
  establish that, so it never stands down: under a fleet-wide election the elected Primary is usually
  running a different cluster's configuration, and deferring to it would end this cluster's historization
  silently — the drain would report success on an empty outbox forever. That is the same scope mismatch
  that once suspended the alarm drain on every node of the docker-dev rig at once.

An unknown role **captures**, matching the alarm drain: a duplicate sample is recoverable, a hole in the
trend is not. The residual cost is a capture gap across a failover, bounded by how long the newly promoted
node takes to see its first redundancy snapshot — inherent to any non-duplicating design. Suppressed
values are counted in `RecorderStatus.SuppressedNotPrimary`, and each gate transition logs once.

---

## Tag auto-provisioning (EnsureTags)

When a deploy adds a historized tag, the server auto-ensures it exists in the historian via the gateway's
`EnsureTags` **before** the first value or HistoryRead lands. `AddressSpaceApplier.Apply()` fires a
**non-blocking, fire-and-forget** `IHistorianProvisioning.EnsureTagsAsync` for each added historized value
tag (with the resolved historian name — the `historianTagname` override when set, else the driver
`FullName`); the hook is wrapped so a faulted, slow, or throwing provisioner can **never** block or fail a
deploy. Non-numeric (`String`/`DateTime`/`Reference`) types are skipped, not provisioned.

**Wiring + gating.** The concrete provisioner is the gateway-backed `GatewayTagProvisioner`, registered by
the Host's `AddHistorianProvisioning` (`GatewayHistorian.CreateProvisioner`) and gated on the **same**
`ServerHistorian:Enabled` flag as the read path — both target the same single gateway. When the section is
disabled (or absent) the applier resolves the no-op `NullHistorianProvisioning` and **provisioning does not
run** (so on a read-only-historian deployment, ensure tags exist by some other means). The Host requires the
API key to carry the `historian:tags:write` scope for `EnsureTags` to succeed.

**Observability.** Every dispatch logs an Information tally —
`historian provisioning completed (dispatched=N, requested=…, ensured=…, skipped=…, failed=…)`. A line
showing `dispatched=N` but `requested=0` indicates the **no-op** provisioner ran (provisioning is not wired
for this node), which is the signal to check that `ServerHistorian:Enabled=true`.

> **Why this matters for HistoryRead latency.** Until a tag is provisioned, the historian does not know it,
> so a HistoryRead on it runs to the full `CallTimeout` (~30 s) before erroring. Auto-provisioning on deploy
> keeps freshly-historized tags from hitting that slow path. See the `CallTimeout` row above.

### Deferred types and the UInt16 fallback (#600 item 4)

Historization is **numeric-analog only** in v1. `HistorianTypeMapper` classifies exactly nine types as
historizable (`Boolean`, `Int16/32/64`, `UInt16/32/64`, `Float32`, `Float64`):

- **Deferred: `String`, `DateTime`, `Reference`.** The historian's SQL analog write path cannot carry them.
  Provisioning skips them (counted in the `skipped=` tally, via `IsHistorizable`), the mapper throws
  `NotSupportedException` for them, and `ContinuousHistorizationRecorder` drops their values and counts them
  in `DroppedNonNumeric`. A tag authored `isHistorized` with one of these types therefore silently records
  nothing; HistoryRead on it returns whatever the historian holds (typically `GoodNoData`).
- **Fallback: `UInt16 -> Uint4`.** The historian's `UInt2` write path is unproven upstream, so UInt16 tags
  are provisioned as the wider `UInt4`. Values are unaffected; only the historian-side storage type differs.
  (`Boolean -> Int2` is the other fallback, below.)

Lifting either needs gateway-side support first; neither is scheduled.

### Boolean historization maps to Int2 (issue #441)

`EnsureTags` is contract-documented (HistorianGateway 0.2.0+) as an **upsert** — *"creates or updates
historian tags according to the given definitions"* — that maps per-tag onto AVEVA's
`HistorianClient.EnsureTagAsync(definition) -> bool`. It is **not idempotent** (not routed through the
idempotent retry pipeline) and **not skip-if-exists**.

`HistorianTypeMapper` maps **`DriverDataType.Boolean → HistorianDataType.Int2`** and Boolean values
historize as a small **0/1 integer**. Int2 is used rather than the semantically-exact `Int1` because the
historian's **`Int1` analog-tag creation path is server-degenerate** — `EnsureTags(Int1)` stores an
unusable stub, so a Boolean tag mapped to `Int1` never becomes historizable. This is an evidence-based
decision from a live probe against the deployed 2023 R2 historian; the root cause and gateway-side
resolution are tracked in [issue #441](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/441) and
HistorianGateway [#11](https://gitea.dohertylan.com/dohertj2/historiangw/issues/11) / PR #16 (which
documents `Int1` as an explicit, evidence-based exclusion). The value-write path is unaffected: the
recorder sends every value as a `double` on `WriteLiveValues` regardless of tag type, so a Boolean
`0.0/1.0` round-trips cleanly into the Int2 tag.

**Observed live (2026-07-13, real 0.2.0 gateway → real AVEVA historian on `wonder-sql-vd03`, probed via
`grpcurl` — archreview R2-06 T12):**
- `EnsureTags` is a **real create-or-update upsert** — re-`EnsureTags` of an existing tag at the same type
  returns `Success=true`.
- An **in-place retype between SUPPORTED types is ACCEPTED** — `Float→Double` on an existing tag returns
  `Success=true`.
- A type sweep confirmed **`Int2/UInt2/Int4/UInt4/Float/Double` all provision `Success=true`; only `Int1`
  fails** (`Success=false, Error="ProtocolEvidenceMissingException"` — the vendored client's "no capture
  evidence for this type" guard → gRPC `Unimplemented`), both for a fresh tag and as a retype target. Mapping
  Boolean to `Int2` sidesteps that gap entirely.

> **Prerequisite for provisioning to land at all:** the provisioner must stamp a **positive `StorageRateMs`**
> — a `0` made `EnsureTags` throw `ArgumentOutOfRangeException` and fail *every* provision (fixed in
> `GatewayTagProvisioner`, PR #439).

**Migration note for pre-existing Boolean tags.** A Boolean tag historian-side as Float (e.g. from the
0.1.0-era default) is asked, on its next deploy, to upsert to Int2. Both are supported analog types, so the
retype is accepted. Provisioning is fire-and-forget and never blocks/fails a deploy; a per-tag rejection (if
any) surfaces only as the tally's `failed` count, and values written to a still-Float tag remain readable
(Float supersets the 0/1 range) pending reconcile. A bulk retype sweep is **not** done in code.

---

## HistoryRead behavior

### Read variants

The server supports all four OPC UA HistoryRead variants:

| Variant | Node type | CLI `--aggregate` |
|---|---|---|
| **Raw** | Historized variable | (omit `--aggregate`) |
| **Processed** | Historized variable | `Average`, `Minimum`, `Maximum`, `Total`, `Count` |
| **AtTime** | Historized variable | n/a (client supplies exact timestamps) |
| **Events** | Equipment folder (event notifier) | n/a |

**Variable nodes** (historized tags) serve Raw, Processed, and AtTime history.
`Historizing=true` and `AccessLevels.HistoryRead` are set at materialization so any compliant
OPC UA client can discover historized capability from the node's attributes.

> **v3 Batch 4 — dual-namespace registration.** A historized raw tag surfaces as **two**
> variable nodes: the Raw-namespace node (`ns=Raw, s=<RawPath>`) and, for each referencing
> equipment, a UNS-namespace node (`ns=UNS, s=<Area>/<Line>/<Equipment>/<EffectiveName>`).
> **Both NodeIds register the SAME historian tagname** (the raw tag's `FullName` /
> `historianTagname`), so HistoryRead against either NodeId resolves to the same tagname and
> returns the same series. The historization intent is single-sourced — the mux's
> `HistorizedTagRef` set stays keyed by RawPath (no doubles), and continuous historization /
> `EnsureTags` provisioning run once per raw tag regardless of how many equipment reference it.

**Equipment-folder event-notifier nodes** serve Event history. Every equipment folder that
owns at least one alarm condition is already an event notifier; the server registers a
`sourceName` (the equipment id) for each such folder and maps event history reads to the
HistorianGateway using that source. (Alarm-history `ReadEvents` requires the gateway running
`RuntimeDb:EventReadsEnabled=true`.) Event-field projection supports the standard
`BaseEventType` select clauses — `EventId`, `SourceName`, `Time`, `ReceiveTime`, `Message`,
and `Severity`; an unsupported select operand returns a null field (spec-conformant).

### No driver-side history (Gitea #655)

History is served **only** by the server-wide `IHistorianDataSource`
(`GatewayHistorianDataSource` / `NullHistorianDataSource`). No driver contributes
history: the OpcUaClient driver does **not** proxy its upstream server's
HistoryRead (the former `IHistoryProvider` passthrough was never dispatched to and
was deleted). On an OpcUaClient tag, `isHistorized` means "OtOpcUa records this
value into the historian from now on", not "forward history reads upstream". A
node without `isHistorized` has no HistoryRead access bit and answers
`BadHistoryOperationUnsupported`.

### Graceful degradation

| Situation | HistoryRead status |
|---|---|
| Historized node, historian configured and reachable, results found | `Good` |
| Historized node, historian configured, time range empty | `GoodNoData` |
| Historized node, historian NOT configured (`Enabled=false` / Null source) | `GoodNoData` (empty) |
| Non-historized node | `BadHistoryOperationUnsupported` |
| Backend timeout or exception | `BadHistoryOperationUnsupported` per node; other nodes in the same batch are unaffected |

A historized node with no historian configured never returns an error status — it returns
empty. This means a deployment can author and publish historized tags before the historian
gateway is provisioned, without producing error spikes in connected clients.

### Continuation-point paging (Raw)

`HistoryRead-Raw` is paged server-side. The historian backend is single-shot (it returns up to
`NumValuesPerNode` samples with no continuation point of its own), so the server synthesises
paging time-based:

- A page that returns **exactly** `NumValuesPerNode` samples (with `NumValuesPerNode > 0`) MAY
  have more behind it, so the server stores a resume cursor and returns an opaque continuation
  point (16 bytes). The client hands it back to fetch the next page.
- A short page (fewer than the cap) is the last page — no continuation point.
- `NumValuesPerNode == 0` ("all values, no limit") is never paged; the whole window returns in one
  shot.
- The resume cursor is **tie-safe**: the next page resumes from the last returned sample's
  SourceTimestamp *inclusive* and drops the boundary samples already emitted, so samples sharing
  the boundary timestamp are neither duplicated nor skipped.

> **Oversized tie clusters — within-timestamp paging.** When more samples share one
> `SourceTimestamp` than the current page cap, the server detects that the cursor has stalled on
> a tie cluster (the last returned timestamp equals the resume timestamp). It then **over-fetches
> the entire cluster** at that single timestamp up to a bounded ceiling controlled by
> `ServerHistorian:MaxTieClusterOverfetch` (default **65 536**), then serves the cluster
> `NumValuesPerNode` samples at a time across successive pages, advancing the cursor one tick past
> the timestamp once the cluster is fully drained.
>
> **Short pages within a cluster still carry a continuation point.** A within-cluster page that
> returns fewer than `NumValuesPerNode` samples (because the cluster happened to be smaller than
> the cap, or is the final partial batch) is not the last page if the cluster itself has not been
> fully emitted — the server retains the continuation point so the client can drain the remainder.
> Only when the cluster is exhausted and the cursor has advanced past the timestamp does the
> short-page rule apply.
>
> **Cluster larger than `MaxTieClusterOverfetch`.** If the over-fetch itself reaches the ceiling
> without spanning the full cluster, the node fails **loudly** with
> `BadHistoryOperationUnsupported` and the tag + timestamp + ceiling are logged. Remedies: raise
> `MaxTieClusterOverfetch` (or `NumValuesPerNode`) to cover the full cluster, or investigate the
> data anomaly (raw samples normally carry strictly increasing distinct timestamps).
>
> For a single tag's raw history a tie cluster larger than the default 65 536 is a severe data
> anomaly. The ceiling exists to bound server-side memory on pathological data, not to cap normal
> operation.

Continuation points are bound to the OPC UA session (the SDK's
`ServerConfiguration.MaxHistoryContinuationPoints` cap, default 100, with oldest-eviction; points
are disposed when the session closes). Resuming an unknown / evicted / released point returns
`BadContinuationPointInvalid`. `releaseContinuationPoints` drops the stored cursors without reading
data.

### Total aggregate

The OPC UA `Total` aggregate is **supported** over the HistorianGateway backend. The gateway exposes a
native **`Integral`** retrieval mode, so `Total` maps straight to it (`HistoryAggregateType.Total →
RetrievalMode.Integral`) — no client-side scaling. (This replaces the retired Wonderware path, which had no
`Total` column and derived it client-side as time-weighted `Average × interval-seconds`.) `Count` is
likewise a native gateway mode. Bucket status codes and timestamps are preserved unchanged; empty / null
buckets surface as `BadNoData`.

### Aggregate set and `BadAggregateNotSupported` (#285)

`HistoryAggregateType` carries the wider Part 13 set (Interpolative, TimeAverage, Start, End, Delta, Range,
Min/MaxActualTime, Start/EndBound, DeltaBounds, DurationGood/Bad, PercentGood/Bad, WorstQuality,
NumberOfTransitions, AnnotationCount, StandardDeviationPopulation, VariancePopulation); `OtOpcUaNodeManager.MapAggregate`
maps each standard aggregate NodeId. The **HistorianGateway** backend only has retrieval modes for the original
five (Average, Minimum, Maximum, Total, Count): for any other member `AggregateModeMapper` throws
`HistoryAggregateNotSupportedException`, which the HistoryRead path turns into `BadAggregateNotSupported` for that
node only (not a generic failure, and not counted as a backend fault). The OPC UA Client driver forwards every
member upstream. A NodeId outside the mapped set (e.g. `StandardDeviationSample`) is still `BadAggregateNotSupported`.

### Known limitations

- **Processed and AtTime are single-shot** (no continuation points). Unlike Raw, neither
  `ReadProcessedDetails` nor `ReadAtTimeDetails` carries a client count cap (`NumValuesPerNode`):
  the Processed bucket count is deterministic (window / interval) and AtTime returns exactly one
  sample per requested timestamp, so the single-shot backend returns the complete result in one
  read and there is no "full page ⇒ maybe more" signal to page on. Returning the full result with
  no continuation point is spec-conformant.
- **No modified-value history** (`HistoryReadModified`). Requests for modified values return
  `BadHistoryOperationUnsupported`. This is **infra-gated, not a server-code gap**: the HistorianGateway
  backend (`GatewayHistorianDataSource`) exposes only a current-value read path — there is no
  modified/edited-history surface to source the data from. The server-side override is in place (it cleanly
  rejects modified reads per node) and `IsReadModified` is honoured; serving real modified-value history is
  unblocked only once the gateway grows a modified-read RPC. Until then, rejecting is the correct,
  spec-conformant behaviour.

### Redundancy and authorization

History reads are served from any node — there is no Primary gate. Authorization is the
standard OPC UA `HistoryRead` permission enforced by the SDK through the `AccessLevels.HistoryRead`
bit set at materialization. A session without sufficient permissions receives
`BadUserAccessDenied` from the SDK before the dispatch reaches the historian.

---

## Authoring workflow

1. Open the equipment's **Tags** tab on `/uns/equipment/{id}`.
2. Create or edit the tag.
   - For **typed-editor drivers** (Modbus, S7, OpcUaClient, etc.): check the
     **Historize this tag** checkbox and, if needed, fill in the **Historian tagname
     (override)** textbox.
   - For **raw-JSON editors** (Galaxy): you can check the same first-class checkboxes
     (they appear below the JSON textarea), or add `"isHistorized":true` (and optionally
     `"historianTagname":"..."`) directly in the textarea.
3. Save and publish. The server rebuilds its address space; the node materialises with
   `Historizing=true` and the `HistoryRead` AccessLevel bit.
4. Confirm with Client.CLI `read` that the node's `Status` is `Good` and that the value
   is updating. Then issue a `historyread` to verify the historian connection returns data.

### Native-alarm historian opt-out (`alarm.historizeToAveva`)

A tag carrying a native `"alarm"` object has a **separate** historian opt-out for its
**alarm transitions** (distinct from tag-value historization). On the Tag modal, check or
uncheck **Historize to AVEVA** in the alarm section. This maps to `alarm.historizeToAveva`
(`bool?`) in the `TagConfig` JSON:

- **Absent or `true`** (default) — the alarm's transitions are written to AVEVA Historian
  via `HistorianAdapterActor`.
- **`false`** — the durable AVEVA write is suppressed for this alarm's transitions. The
  live `/alerts` feed and OPC UA condition events are unaffected.

The gate is applied in `HistorianAdapterActor` using `is not false` semantics, matching the
scripted-alarm `HistorizeToAveva` posture. See
[ScriptedAlarms.md §Native driver alarms](ScriptedAlarms.md#native-driver-alarms-equipment-tag-path)
and [AlarmTracking.md §Historian write-back](AlarmTracking.md#historian-write-back-non-galaxy-alarms)
for the full alarm-historian routing.

---

## Client.CLI historyread examples

The `historyread` command reads historical data from any node. Supply start and end times in
ISO 8601 UTC form. See [docs/Client.CLI.md](Client.CLI.md) for the full flag reference.

> **v3 Batch 4 NodeIds.** A historized tag is readable through **either** of its two NodeIds —
> the Raw-namespace node (`s=<RawPath>`, e.g. `Plant/Modbus/dev1/Speed`) or a referencing
> equipment's UNS-namespace node (`s=<Area>/<Line>/<Equipment>/<EffectiveName>`). Both register
> the same historian tagname, so the returned series is identical. The `ns=N` index is assigned
> at connect time per the namespace URI (`https://zb.com/otopcua/raw` /
> `https://zb.com/otopcua/uns`) — resolve it from the server's `NamespaceArray` rather than
> hard-coding it. The examples below show a Raw-namespace read (`ns=2` illustrative).

```bash
# Raw history for a historized tag via its Raw-namespace NodeId (last 24 hours by default)
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=Plant/Modbus/dev1/Speed" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z"

# Same series via a referencing equipment's UNS-namespace NodeId, limited to 100 values
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=3;s=filling/line1/station1/Speed" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z" \
  --max 100

# 1-hour average aggregate
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=Plant/Modbus/dev1/Speed" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z" \
  --aggregate Average --interval 3600000

# Authenticated read (ReadOnly role or higher required)
otopcua-cli historyread \
  -u opc.tcp://localhost:4840/OtOpcUa \
  -n "ns=2;s=Plant/Modbus/dev1/Speed" \
  --start "2026-06-13T00:00:00Z" --end "2026-06-14T00:00:00Z" \
  -U reader -P password
```

Supported `--aggregate` values: `Average`, `Minimum`, `Maximum`, `Count`, `Start`, `End`,
`StandardDeviation` (aliases: `avg`, `min`, `max`, `stddev`/`stdev`, `first`, `last`).
`--interval` is the processing interval in milliseconds (default 3600000 = 1 hour).
The `Total` aggregate is served by the server's OPC UA HistoryRead endpoint for any conformant
client (derived as time-weighted Average × interval-seconds — see "Total aggregate derivation"
above), but is not exposed by this bundled CLI.

---

## Live /run gate

The live read gate requires a reachable `ZB.MOM.WW.HistorianGateway` (VPN to `wonder-sql-vd03`) with the
AVEVA Historian behind it healthy. Set `ServerHistorian:Enabled=true` with the correct `Endpoint`
(`https://host:5222`) and supply `ServerHistorian__ApiKey` via the environment, then deploy and publish at
least one historized Galaxy tag. The gate is operator-driven — it is not part of the local docker-dev rig.
The gateway-backed driver also ships an env-gated live suite (`Category=LiveIntegration`); see the
`HISTGW_GATEWAY_ENDPOINT` / `HISTGW_GATEWAY_APIKEY` / `HISTGW_TEST_TAG` / `HISTGW_WRITE_SANDBOX_TAG` /
`HISTGW_BOOL_SANDBOX_TAG` / `HISTGW_ALARM_SOURCE` env vars (it skips cleanly when they are absent).
`HISTGW_BOOL_SANDBOX_TAG` (e.g. `HistGW.LiveTest.BoolSandbox`, a **dedicated** writable Boolean tag —
never the shared Float write sandbox) drives the Boolean historization leg
(`EnsureTags_boolean_sandbox_provisions_as_Int2`) and the retype probe
(`EnsureTags_type_change_outcome_is_observable`) that back the
[Boolean historization maps to Int2](#boolean-historization-maps-to-int2-issue-441) decision. Running
the full suite against the gateway (all six tests, none skipped) also discharges 06/U-2's
"full documented run" for the historian suite.

See [AlarmHistorian.md](AlarmHistorian.md) for the alarm write path and
[ServiceHosting.md](ServiceHosting.md) for the (external) HistorianGateway deployment.

---

---

## Cross-driver TagConfig key reference

The `TagConfig` JSON blob is the single extension surface for per-tag server-side
behaviours across all drivers.  Keys from multiple phases coexist in the same blob;
unknown keys are preserved byte-stable on round-trip by all typed editors.

### Historian + alarm keys (Phase B / Phase C)

| Key | Type | Phase | Description |
|---|---|---|---|
| `isHistorized` | bool | C | Marks the tag historized (HistoryRead + Historizing=true). |
| `historianTagname` | string | C | Explicit historian tagname override (defaults to `FullName`). |
| `alarm` | object | B | Native alarm definition (`alarmType`, `severity`, `historizeToAveva`, …). |

### Array keys (Phase 4c)

| Key | Type | Phase | Description |
|---|---|---|---|
| `isArray` | bool | 4c | When `true` and `arrayLength >= 1`, materialises the node as a 1-D array (`ValueRank=OneDimension`). Absent or `false` → scalar. |
| `arrayLength` | uint (≥1) | 4c | Element count; sets `ArrayDimensions`. Required when `isArray: true`. |

### Cross-driver array coverage matrix

| Driver | Read mechanism | Live-verify status |
|---|---|---|
| **Modbus** | Contiguous FC03/FC04 block; String/BitInRegister modes supported | Mac-verifiable (sim `10.100.0.35:5020`) |
| **S7** | `ReadBytesAsync` block + per-element decode loop | Unit-proven (fixture down) |
| **AB CIP** | libplctag native array read (atomic + UDT member arrays) | Unit-proven (fixture down) |
| **AB Legacy** | PCCC multi-element file read via libplctag (cap 256 elements) | Unit-proven (fixture down) |
| **TwinCAT** | Raw-bytes array read + per-element decode (native notifications carry the whole array) | Live-verified on TwinCAT 2.11 (2026-10-07) |

Array writes, multi-dimensional arrays (ValueRank>1), and per-element historization are out of scope — see [Uns.md §Array tags](Uns.md#array-tags-1-d).

---

## Closed backlog §2 items (Phase 4 / Phase 4b)

The following items from the backlog §2 (archived at
`docs/plans/2026-06-18-stillpending-backlog-audit.md`) were closed by earlier
phases and are recorded here so future audits don't re-flag them.

| Item | Closed by | Commit |
|---|---|---|
| Modbus Int64 / UInt64 OPC UA node `DataType` correction | Phase 4 | `bd8fee61` |
| `HistoryAggregateType.Total` — `Total` aggregate | Phase 4 (derived client-side as time-weighted Average × interval-seconds) | `5e27b5f7` |
| Historian poison alarm-event indefinite retry — dead-letter cap (task #437) | Phase 4 | `fcb38014` |

---

## See also

- [docs/plans/2026-06-14-galaxy-phase-c-historian-design.md](plans/2026-06-14-galaxy-phase-c-historian-design.md) — full design and implementation notes
- [AlarmHistorian.md](AlarmHistorian.md) — alarm write path; drains to the same HistorianGateway (`SendEvent`)
- [AlarmTracking.md](AlarmTracking.md) — OPC UA Part 9 alarm surface (event history source)
- [Client.CLI.md](Client.CLI.md) — full `historyread` flag reference
- [ScriptedAlarms.md](ScriptedAlarms.md) §"Native driver alarms" — the Phase B `alarm` object in `TagConfig` (parallel carrier)
- [Uns.md §Array tags](Uns.md#array-tags-1-d) — `isArray`/`arrayLength` keys, cross-driver coverage, and deferrals