fix(historian): read-only historian bindings + ProvisionTags policy; provision OPC UA type names (#899, #898)
v2-ci / unit-tests (push) Has been cancelled
v2-ci / unit-tests (push) Has been cancelled
#899 — EnsureTags is an upsert, so binding a node to an existing AVEVA tag via historianTagname re-typed/re-described it on every add and host start. - TagConfig `historianReadOnly: true` (TagConfigIntent -> RawTagPlan.HistorianReadOnly): HistoryRead from an existing historian tag that OtOpcUa never provisions and never records into (AddressSpaceApplier.IsOwnedHistorizedValue gates both the EnsureTags hook and the recorder feed). Node-inert: surgical, not mirrored on UNS refs, in the RawTagPlan equality/hash. A non-bool value is the deploy error HistorianReadOnlyInvalid. AdminUI: CSV column HistorianReadOnly (strict), TagHistorizeConfig reads via the runtime parse. - ServerHistorian:ProvisionTags = CreateOrUpdate (default, unchanged) | CreateOnly (Gateway only: TagExistsAsync first, existing tags left alone, failed lookup not sent) | Off. Unknown value / CreateOnly on a SQL backend fail host start. - Default for an explicit historianTagname stays OWNED: overrides pin owned history across renames, a read-only default would silently stop provisioning + recording. #898 — the hook parsed Tag.DataType (an OPC UA name) as DriverDataType, so historized Float/Double/Byte/SByte tags were never provisioned. New shared HistorianDataTypeNames.TryResolve: Float->Float32, Double->Float64, Byte/SByte->Int16 (lossless, provisioned as Int2); unmapped names logged at Information. Gateway now skips Int64/UInt64 (HistorianGateway cannot create Int8/UInt8: lossless-cast-or-reject). UPGRADE: the first host start after this provisions (upserts) every existing historized Float/Double/Byte/SByte tag. Mark foreign-tag bindings historianReadOnly, or set ServerHistorian:ProvisionTags=CreateOnly/Off, before upgrading (docs/Historian.md). Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
+58
-2
@@ -69,6 +69,7 @@ fields directly in the textarea.
|
||||
|---|---|---|---|
|
||||
| `isHistorized` | bool | yes | Marks the tag historized. Materialises the OPC UA node with `Historizing=true` and the `HistoryRead` AccessLevel bit. |
|
||||
| `historianTagname` | string | no | Explicit historian tagname to query. When absent or empty, defaults to the tag's driver `FullName` (the `TagConfig.FullName`). |
|
||||
| `historianReadOnly` | bool | no | **Read-only binding (#899).** `true` = the history is read from an **existing** historian tag that OtOpcUa does not own: the node still serves HistoryRead, but OtOpcUa **never provisions** it (no `EnsureTags`, so it is never created, re-typed or re-described) and **never records** captured values into it (no continuous-historization `WriteLiveValues`). Absent/`false` = an owned tag (provisioned and recorded, the behaviour before #899). Must be a JSON boolean — any other value is the deploy error `HistorianReadOnlyInvalid`. CSV column `HistorianReadOnly`. See [Read-only bindings](#read-only-bindings-to-existing-historian-tags-899). |
|
||||
|
||||
### Examples
|
||||
|
||||
@@ -84,6 +85,12 @@ Explicit historian tagname override:
|
||||
{"FullName":"TestMachine_002.TestFloat","isHistorized":true,"historianTagname":"Plant.Line1.Flow"}
|
||||
```
|
||||
|
||||
Read-only binding to an existing AVEVA tag (never provisioned, never written; #899):
|
||||
|
||||
```json
|
||||
{"scriptId":"SC-…","isHistorized":true,"historianTagname":"SysTimeSec","historianReadOnly":true}
|
||||
```
|
||||
|
||||
A tag that is both historized and a native alarm:
|
||||
|
||||
```json
|
||||
@@ -194,6 +201,7 @@ Processed read is `BadAggregateNotSupported` and no history aggregate functions
|
||||
| `MaxConcurrentHistoryReadBatches` | int | `16` | Process-wide limit on HistoryRead (and HistoryUpdate) batches served **concurrently** (arch-review 03/S3). A flood of history clients degrades to `BadTooManyOperations` rather than exhausting the SDK request-thread pool. With `HistoryReadBatchParallelism` this caps total in-flight gateway reads at `MaxConcurrentHistoryReadBatches × HistoryReadBatchParallelism` (default 16 × 4 = 64). A batch that cannot get a slot within `min(HistoryReadDeadline, 5 s)` fails every handle `BadTooManyOperations`. |
|
||||
| `MaxTimeWeightedRawRows` | int | `1000000` | `SqlServer` / `QuestDb` (#671), and the `Gateway` per-sample aggregates (#681 — every per-sample read there streams raw samples, so the cap applies to the window's raw rows; see [Aggregate set](#aggregate-set-and-badaggregatenotsupported-285)). The most raw in-window samples one [time-weighted aggregate](#time-weighted-aggregates-part-13-671) read (TimeAverage, TimeAverage2, **Total**, Total2, Interpolative, and — since #680 — **PercentGood/PercentBad**, since #820 **DurationGood/DurationBad**) may stream into the server-side evaluator — that family is **not** pushed down to SQL. The window query is capped at this + 1 rows; one row past the cap fails **that node** with `BadTooManyOperations` (not counted as a backend fault) instead of holding a pooled connection until `HistoryReadDeadline` and returning `BadTimeout`. Bound rows outside the window do not count; per-sample aggregates are not capped on the SQL backends (pushed down). 1 000 000 ≈ 11.5 days of a 1 Hz tag. Non-positive = unbounded (a `Validate()` warning). `Gateway` (#688): the row budget it asks the gateway for (`cap + 18`) is clamped at the gateway's default `MaxValuesCeiling` (1,000,000) while the cap is at or under it, so the default works against a gateway with `AllowStreamingBeyondCeiling=false`; against a gateway with a *lower* ceiling the refused budget is retried once at the ceiling the gateway names (#691.3), so no change is needed (`ceiling − 18` only saves that round-trip). |
|
||||
| `HistoryReadDeadline` | TimeSpan | `00:01:00` | Per-request wall-clock deadline for a HistoryRead (and HistoryUpdate) batch (arch-review 03/S3). A node still in flight at the deadline surfaces `BadTimeout`, so one request can never hold an SDK request thread longer than this regardless of node count. Non-positive = unbounded (a `Validate()` warning). |
|
||||
| `ProvisionTags` | string | `CreateOrUpdate` | What [tag auto-provisioning](#tag-auto-provisioning-ensuretags) may do (#899). `CreateOrUpdate` = the `EnsureTags` / `MERGE` upsert: create a missing tag, re-type and re-describe an existing one (the behaviour before #899). `CreateOnly` (**Gateway backend only** — any other backend fails host start) = look each tag up first (`GetTagMetadata`, a read) and send only the ones the historian reports **absent**; an existing tag is left exactly as it is (counted `skipped`), and a failed lookup is counted `failed` and **not** sent. `Off` = no provisioning at all; HistoryRead and continuous historization are unaffected. Case-insensitive; an unknown value fails host start. A tag's own `historianReadOnly: true` opts it out under every policy. |
|
||||
| `HistoryUpdateMaxFutureSkew` | TimeSpan | `1.00:00:00` | #674. How far past the server clock a HistoryUpdate (Insert / Replace / Update) item's `SourceTimestamp` may lie; an item beyond `now + skew` is refused **`BadOutOfRange`** by the node manager before any backend sees it (every backend). Guards against a far-future typo (a 2062 timestamp). The live recorder is not bounded. Non-positive disables the bound (no warning — a deliberate choice). See [HistoryUpdate](#historyupdate). |
|
||||
|
||||
**`ServerHistorian:Gateway`** (read only when `Backend=Gateway`): knobs that exist because of the HistorianGateway's wire contract.
|
||||
@@ -521,6 +529,50 @@ tag (with the resolved historian name — the `historianTagname` override when s
|
||||
deploy. On the **Gateway backend**, non-numeric (`String`/`DateTime`/`Reference`) types are skipped, not provisioned;
|
||||
the SqlServer backend provisions every type (below).
|
||||
|
||||
> ⚠️ **Upgrade warning (#898, 2026-11-13).** Before this release, historized tags whose `DataType` is
|
||||
> **`Float`, `Double`, `Byte` or `SByte`** were **never provisioned**: the hook parsed the OPC UA type name the AdminUI
|
||||
> stores as a `DriverDataType` and silently skipped those four. They now resolve (`Float`→Float, `Double`→Double,
|
||||
> `Byte`/`SByte`→`Int2`), so **on the first host start after the upgrade every existing historized Float / Double /
|
||||
> Byte / SByte tag is provisioned** — on the Gateway backend that is an `EnsureTags` **upsert**, which creates the
|
||||
> AVEVA tag if it is missing and **re-types / re-describes it if it exists** (description = the OtOpcUa tag name,
|
||||
> storage rate 1000 ms, integral divisor 1). That is the intended fix for tags OtOpcUa owns, but a tag whose
|
||||
> `historianTagname` names **someone else's** AVEVA tag (a System Platform attribute, `SysTimeSec`, …) will be
|
||||
> rewritten. **Before upgrading**, either mark each such tag `"historianReadOnly": true`, or set
|
||||
> `ServerHistorian:ProvisionTags=CreateOnly` (existing tags are left alone, missing ones still created) or `Off`.
|
||||
> The SqlServer / QuestDB backends `MERGE`/upsert a row in OtOpcUa's own tag table for those tags — harmless. Also
|
||||
> new: the Gateway skips `Int64`/`UInt64` up front (counted `skipped`, previously sent and `failed`), because
|
||||
> HistorianGateway cannot create `Int8`/`UInt8` tags (lossless-cast-or-reject).
|
||||
|
||||
**Data type names (#898).** `Tag.DataType` holds OPC UA built-in names. The hook resolves them with
|
||||
`HistorianDataTypeNames.TryResolve` (Core.Abstractions): `Float`→`Float32`, `Double`→`Float64`, `Byte`/`SByte`→`Int16`
|
||||
(a lossless widening — provisioned as `Int2`, the type Boolean uses; the one-byte `Int1` creation is server-degenerate,
|
||||
#441), the names shared with `DriverDataType` map to that member, and a `DriverDataType` spelling (`Float32`,
|
||||
`Float64`, `Reference`) is still accepted. A name with no historian mapping (`Guid`, `ByteString`) is not provisioned and
|
||||
is logged at **Information** (`not provisioning historized tag … has no historian mapping`) — it used to be Debug only.
|
||||
|
||||
### Read-only bindings to existing historian tags (#899)
|
||||
|
||||
`EnsureTags` is an upsert, so before #899 there was **no way** to expose an existing AVEVA tag's history on an OtOpcUa
|
||||
node without OtOpcUa rewriting that tag — on every add and on every host start (the first plan adds every tag). Two
|
||||
opt-outs now exist:
|
||||
|
||||
- **Per tag — `"historianReadOnly": true`** in `TagConfig` (CSV column `HistorianReadOnly`). The node keeps
|
||||
`Historizing=true` + the HistoryRead bit and reads the named historian tag; the applier never puts it in an
|
||||
`EnsureTags` request and never feeds it to the continuous-historization recorder, so OtOpcUa issues **no write of
|
||||
any kind** for it. Clearing the flag makes the tag owned: the next deploy provisions it (like a historization flip)
|
||||
and starts recording it. The flag is node-inert (a surgical, in-place deploy) and is not mirrored on UNS reference
|
||||
nodes. On the SqlServer / QuestDB backends HistoryUpdate on the node is unaffected (those stores are OtOpcUa's own
|
||||
tables; HistoryUpdate is an explicit, ACL-governed client operation).
|
||||
- **Server-wide — `ServerHistorian:ProvisionTags`** = `CreateOrUpdate` (default) | `CreateOnly` (Gateway) | `Off` —
|
||||
see the [options table](#serverhistorian-configuration). `CreateOnly` still records into existing tags; only
|
||||
`historianReadOnly` stops recording.
|
||||
|
||||
**Why the default did not change.** An explicit `historianTagname` does **not** imply "someone else's tag": the AdminUI's
|
||||
raw-rename warning tells authors to set one to **pin** an owned tag's history across RawPath renames, so overrides on
|
||||
owned tags are common. Making an override read-only by default would silently stop provisioning **and** recording those
|
||||
tags. Ordinary owned tags (with or without an override) therefore behave exactly as before; binding to a foreign tag is an
|
||||
explicit opt-in.
|
||||
|
||||
**Wiring + gating.** The concrete provisioner comes from the selected backend's provider (`CreateProvisioning`):
|
||||
the gateway-backed `GatewayTagProvisioner`, or — for `SqlServer` — `SqlServerTagProvisioner` (one `MERGE … WITH (HOLDLOCK)`
|
||||
upsert per chunk of 400 tags; every data type is provisionable so `skipped` is always 0; a blank or over-400-character
|
||||
@@ -544,8 +596,12 @@ for this node), which is the signal to check that `ServerHistorian:Enabled=true`
|
||||
|
||||
> **Gateway backend only.** The `SqlServer` backend provisions and records every type (`String` and `DateTime` included).
|
||||
|
||||
Historization on the **Gateway** backend is **numeric-analog only**. `HistorianTypeMapper` classifies exactly nine types as
|
||||
historizable (`Boolean`, `Int16/32/64`, `UInt16/32/64`, `Float32`, `Float64`):
|
||||
Historization on the **Gateway** backend is **numeric-analog only**. `HistorianTypeMapper` classifies exactly seven types as
|
||||
historizable (`Boolean`, `Int16/32`, `UInt16/32`, `Float32`, `Float64`; the OPC UA `Byte`/`SByte` arrive as `Int16`):
|
||||
|
||||
- **Not creatable: `Int64`, `UInt64` (#898).** HistorianGateway's type policy is lossless-cast-or-reject and its client
|
||||
cannot create `Int8`/`UInt8` tags; no creatable type holds a 64-bit integer losslessly. Provisioning skips them
|
||||
(`skipped=`) instead of sending a request the gateway fails with `FailedPrecondition`.
|
||||
|
||||
- **Deferred: `String`, `DateTime`, `Reference`.** The historian's SQL analog write path cannot carry them.
|
||||
Provisioning skips them (counted in the `skipped=` tally, via `IsHistorizable`), the mapper throws
|
||||
|
||||
+3
-2
@@ -63,7 +63,8 @@ removed later.
|
||||
- **Import/Export CSV** — a staged flow (upload → parse → **review grid** with per-row
|
||||
verdicts → commit) over an RFC-4180 parser. Columns: the common set
|
||||
(`Name, TagGroupPath, DataType, AccessLevel, WriteIdempotent, PollGroup, IsHistorized,
|
||||
HistorianTagname, IsArray, ArrayLength, Alarm.*, TagConfigJson`) plus the per-driver typed
|
||||
HistorianTagname, HistorianReadOnly, IsArray, ArrayLength, Alarm.*, TagConfigJson`;
|
||||
`HistorianReadOnly` = the #899 read-only historian binding, see [Historian.md](Historian.md)) plus the per-driver typed
|
||||
columns (e.g. Modbus `Region, Address, ModbusDataType, ByteOrder, …`). `TagGroupPath`
|
||||
auto-creates nested groups; typed columns win over `TagConfigJson` on conflict (flagged in
|
||||
the review grid). **Import is all-or-nothing** — any invalid row blocks the whole commit.
|
||||
@@ -90,7 +91,7 @@ removed later.
|
||||
|
||||
Editing a raw **tag's** attributes is no longer a fleet-wide event. A **non-alarm**
|
||||
raw-tag delta whose only differences are the in-place-updatable attributes —
|
||||
`Writable`, `IsHistorized`, `HistorianTagname`, `DataType`, `IsArray`, `ArrayLength`,
|
||||
`Writable`, `IsHistorized`, `HistorianTagname`, `HistorianReadOnly` (#899, node-inert), `DataType`, `IsArray`, `ArrayLength`,
|
||||
`ArrayWritable` (the driver's array-write capability, #610 — see
|
||||
[Uns.md §Array writes](Uns.md#array-writes-610)) — and/or its set of referencing equipment is applied **in place** on the existing Raw-realm
|
||||
node (`ISurgicalAddressSpaceSink.UpdateTagAttributes`), so every client subscription on
|
||||
|
||||
@@ -252,6 +252,7 @@ drivers (Galaxy) alike.
|
||||
|---|---|---|---|
|
||||
| **Historize this tag** checkbox | `isHistorized` | bool | When checked, the OPC UA node materialises with `Historizing=true` and the `HistoryRead` AccessLevel bit set. |
|
||||
| **Historian tagname (override)** textbox | `historianTagname` | string (optional) | Explicit tagname the historian backend will query. When left blank, the server defaults to the tag's driver `FullName`. |
|
||||
| (CSV column `HistorianReadOnly` / raw JSON) | `historianReadOnly` | bool (optional) | #899: the history is read from an **existing** historian tag OtOpcUa does not own — never provisioned (`EnsureTags`) and never recorded into. See [Historian.md](Historian.md#read-only-bindings-to-existing-historian-tags-899). |
|
||||
|
||||
These fields are merged into the `TagConfig` JSON blob via the pure
|
||||
`TagHistorizeConfig` helper, which preserves all other keys byte-stable.
|
||||
|
||||
Reference in New Issue
Block a user