Merge ph-T14 (pluggable historian docs)

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
This commit is contained in:
Joseph Doherty
2026-10-09 13:52:14 -04:00
7 changed files with 471 additions and 61 deletions
+13 -5
View File
@@ -43,8 +43,8 @@ through.
([`../HistorianGateway/CLAUDE.md`](../HistorianGateway/CLAUDE.md)) — single-process
gRPC sidecar exposing the AVEVA Historian read/write API + read-only Galaxy browse
(via the shared `ZB.MOM.WW.GalaxyRepository` lib). Patterned on `mxaccessgw` but with
no COM / no x86 worker. **OtOpcUa's historian backend** (`Driver.Historian.Gateway`)
consumes it as the sole historian path.
no COM / no x86 worker. **OtOpcUa's default historian backend** (`Driver.Historian.Gateway`)
consumes it; `ServerHistorian:Backend=SqlServer` selects a plain SQL Server historian instead (see Historian section).
- **`~/Desktop/ScadaBridge`**
([`../ScadaBridge/CLAUDE.md`](../ScadaBridge/CLAUDE.md)) — the distributed SCADA
platform (Akka.NET hub-and-spoke) whose Data Connection Layer consumes this server's
@@ -1325,9 +1325,9 @@ are no longer supported (pinned by a test). Condition teardown must forget the w
## Historian / HistoryRead
**Backend: HistorianGateway (sole historian backend).** As of the gateway-integration cutover, the
historian read, alarm-write, and continuous-historization paths are all served by the
**`ZB.MOM.WW.HistorianGateway`** sidecar, consumed as the Gitea-feed
**Backend: pluggable — `ServerHistorian:Backend` = `Gateway` (default) or `SqlServer`** (see the subsection below).
As of the gateway-integration cutover, the **Gateway** backend's historian read, alarm-write, and
continuous-historization paths are all served by the **`ZB.MOM.WW.HistorianGateway`** sidecar, consumed as the Gitea-feed
**`ZB.MOM.WW.HistorianGateway.Client`** gRPC package (`historian_gateway.v1`) behind a thin
`IHistorianGatewayClient` seam in `ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway`. **The bespoke
Wonderware TCP/ArchestrA sidecar projects and the vestigial `Historian.Wonderware` driver type were
@@ -1338,6 +1338,14 @@ A tag is historized by adding `"isHistorized": true` to its `TagConfig` JSON blo
raw-JSON textarea on the `/uns` TagModal); an optional `"historianTagname"` field overrides the default
historian tagname, which is the tag's driver `FullName`.
### Pluggable backends, SQL Server and HistoryUpdate (2026-10)
- **Selection:** `ServerHistorian:Backend` (`Gateway` default | `SqlServer`; unknown fails host start). `HistorianBackendRegistry` → `IHistorianBackendProvider` builds all five seams; the gateway keys are only required for `Gateway`. `SqlServer` uses `ServerHistorian:SqlServer:{ConnectionString (required), Schema=hist, CommandTimeout=30s, AutoCreateSchema=true}`.
- **SQL Server** (`Driver.Historian.SqlServer`): `hist.Sample` PK `(TagId, SourceTimeUtc)` — writes are insert-if-absent, so outbox redelivery is idempotent. Stores every value kind with its **real status**; aggregates (13, simple per-sample definitions, not Part 13) are pushed down to T-SQL with bucket math in **100 ns ticks**. Docker-dev recipe, schema and aggregate semantics: `docs/Historian.md` §"SQL Server backend". Integration tests need `HISTORIAN_SQL_TEST_CONNECTION`.
- **Real status fix:** the recorder used to stamp every sample Good (`192`); it now carries the driver's status. ⚠️ On the **Gateway** backend this changes nothing stored: the gateway discards quality (C-002), so AVEVA still stores Good.
- **Comms loss is historized as an outage marker** (`Empty` + `BadNoCommunication`, one per tag per outage) on SqlServer; dropped (`DroppedUnsupportedKind`) on Gateway. Outbox records are **format v2** — drain or delete `ContinuousHistorization:OutboxPath` before downgrading a node.
- **HistoryUpdate** (SqlServer only; Gateway ⇒ `BadHistoryOperationUnsupported`): Insert/Replace/Update/DeleteRaw/DeleteAtTime, gated in both ACL modes — flag off needs the `HistoryUpdate` role from a **Cluster-scoped** grant (+ key re-check), flag on uses `DataPlaneAuthorizer`'s HistoryUpdate operation; `WriteOperate` never implies it. The `HistoryWrite` AccessLevel bit is set only when an updater is wired at materialise time. `docs/Historian.md` §HistoryUpdate, `docs/security.md`.
### Read path (`ServerHistorian` section)
The server dispatches all OPC UA HistoryRead to the registered `IHistorianDataSource` — the
+9
View File
@@ -250,6 +250,15 @@ otopcua-cli historyupdate -u ... -n "..." --mode delete-at-time --time 2026-01-1
| `--type` | `double` (default), `int64`, `bool`, `string`, `datetime` - how `--value` is parsed |
| `--start` / `--end` | Range for `delete-raw` (both required; `--end` later than `--start`) |
Expected per-item codes on the SQL Server historian backend: `insert` ⇒ `GoodEntryInserted` (repeat ⇒ `BadEntryExists`), `replace` ⇒ `GoodEntryReplaced`
(`BadNoEntryExists` if nothing is stored at that time), `update` ⇒ either Good code, `delete-at-time` ⇒ `Good` / `BadNoEntryExists`; `delete-raw` reports one
node-level code (`Good`, or `BadNoData` if the range held nothing). A node that is not historized, or a server on the Gateway historian backend (no
updater, so no `HistoryWrite` access bit), answers `BadHistoryOperationUnsupported`; a `NaN`/`Infinity` double is `BadInvalidArgument`. The session's API key
(`-U apikey -P otua_<keyId>_<secret>`) needs the `HistoryUpdate` permission — via a `Cluster`-scoped grant when `Security:Acl:Enabled` is off, or a scoped
grant on the node when it is on; a key with only `WriteOperate` gets `BadUserAccessDenied`. See [Historian.md §HistoryUpdate](Historian.md#historyupdate) and
[security.md](security.md#historyupdate-historian-writes). `historyread --aggregate` only knows the seven names in the table below; the SQL backend's other
aggregates (Total, Range, Delta, PercentGood, …) need a different client.
`delete-raw` is sent with `IsDeleteModified=false`. Invalid or inconsistent options (unknown mode, unpaired
value/time, unparsable value, options that do not apply to the mode) fail before any connection is made.
+26 -4
View File
@@ -377,9 +377,12 @@ The Galaxy/MxAccess connection settings are **not an `appsettings` section.** Th
> The `OTOPCUA_GALAXY_*` environment variables that v1's in-process `Galaxy.Host` consumed **no longer live in this repo** — they moved into the separately-installed mxaccessgw gateway's own config (see the v1 archive pointer in `docs/README.md` and the Galaxy overview at [`docs/drivers/Galaxy.md`](drivers/Galaxy.md)). The only Galaxy connection secret this repo touches is the gateway API key via `ApiKeySecretRef` above.
### Historian config (HistorianGateway)
### Historian config (`ServerHistorian:Backend` — HistorianGateway or SQL Server)
The historian backend is the external **`ZB.MOM.WW.HistorianGateway`** sidecar, consumed as the
The historian backend is **pluggable**, selected by `ServerHistorian:Backend` (case-insensitive; default `Gateway`;
unknown value ⇒ host start fails listing the allowed values): `Gateway` is the external **`ZB.MOM.WW.HistorianGateway`**
sidecar below, `SqlServer` is a plain SQL Server database (see [`ServerHistorian:SqlServer`](#serverhistoriansqlserver-backendsqlserver)).
The default backend is the gateway, consumed as the
`ZB.MOM.WW.HistorianGateway.Client` gRPC package (the retired Wonderware TCP sidecar is documented at
[`docs/drivers/Historian.Wonderware.md`](drivers/Historian.Wonderware.md)). The OtOpcUa host reads three
appsettings sections — `ServerHistorian` (read path + gateway connection), `ContinuousHistorization`
@@ -394,9 +397,26 @@ key must carry the scopes `historian:read`, `historian:write`, `historian:tags:w
[`docs/Historian.md`](Historian.md) for the full key reference, the migration note (old Wonderware keys →
gateway keys), and the deployment prerequisites.
#### `ServerHistorian:SqlServer` (`Backend=SqlServer`)
Read only when `ServerHistorian:Backend` is `SqlServer`; the gateway keys (`Endpoint`, `ApiKey`, `UseTls`, …) are then not required.
Validated at host start (driver-role nodes) when `ServerHistorian:Enabled` **or** `AlarmHistorian:Enabled` is true.
| Key | Default | Validation / meaning |
|---|---|---|
| `ServerHistorian:Backend` | `Gateway` | `Gateway` or `SqlServer`; anything else fails host start (checked even when the historian is disabled). |
| `ServerHistorian:SqlServer:ConnectionString` | `""` | **Required.** Historian database connection string (not the ConfigDb). Supply via `${secret:OT-HISTORIAN-SQL}` or env `ServerHistorian__SqlServer__ConnectionString`; the database must already exist. |
| `ServerHistorian:SqlServer:Schema` | `hist` | Must match `^[A-Za-z_][A-Za-z0-9_]{0,127}$` (spliced into DDL/DML). |
| `ServerHistorian:SqlServer:CommandTimeout` | `00:00:30` | Must be `> 0`; rounded up to whole seconds. |
| `ServerHistorian:SqlServer:AutoCreateSchema` | `true` | `true` applies the embedded schema script under an app-lock on first use; `false` only verifies `SchemaVersion >= 1` and fails every historian call (health `LastError`, no crash) if behind. |
HistoryUpdate (SqlServer only) additionally needs `ServerHistorian:Enabled=true` and a key whose principal holds the
`HistoryUpdate` permission — see [`docs/security.md`](security.md#historyupdate-historian-writes). Full semantics, schema, aggregates and the
docker-dev recipe: [`docs/Historian.md`](Historian.md#sql-server-backend).
#### `ContinuousHistorization` (recorder + outbox knobs)
Only the knobs; the gateway connection comes from `ServerHistorian`. Full descriptions in
Only the knobs; the backend connection comes from `ServerHistorian`. Outbox records are format v2 — **drain or delete `OutboxPath` before downgrading a node** to a pre-v2 build (see [`docs/Historian.md` §Outbox format v2 and rollback](Historian.md#outbox-format-v2-and-rollback)); undecodable records are skipped and counted by the gauge `otopcua.historian.outbox.unreadable`. Full descriptions in
[`docs/Historian.md` §ContinuousHistorization configuration](Historian.md#continuoushistorization-configuration).
| Key | Default | Meaning |
@@ -498,7 +518,9 @@ historian secret this repo reads from the environment is the gateway API key:
| Variable | Effect / default |
|---|---|
| `ServerHistorian__ApiKey` | The HistorianGateway peppered-HMAC key (`histgw_<id>_<secret>`) sent as `Authorization: Bearer`. Supply via environment — **never commit**. Required when `ServerHistorian:Enabled=true`. |
| `ServerHistorian__ApiKey` | The HistorianGateway peppered-HMAC key (`histgw_<id>_<secret>`) sent as `Authorization: Bearer`. Supply via environment — **never commit**. Required when `ServerHistorian:Enabled=true` **and `Backend=Gateway`**. |
| `ServerHistorian__Backend` | `Gateway` (default) or `SqlServer`. |
| `ServerHistorian__SqlServer__ConnectionString` | The SQL Server historian connection string. Required when `Backend=SqlServer` and the historian (or alarm history) is enabled. Supply via environment / `${secret:}` — **never commit**. |
### Driver integration-test / fixture sim endpoints
+376 -31
View File
@@ -1,4 +1,4 @@
# Historian — server-side OPC UA HistoryRead (Phase C)
# Historian — server-side OPC UA HistoryRead / HistoryUpdate (pluggable backends)
Phase C wires server-side OPC UA **HistoryRead** for authored equipment tags flagged
historized. The feature is driver-agnostic: any equipment tag (Galaxy, Modbus, OpcUaClient,
@@ -10,7 +10,14 @@ the existing schemaless `TagConfig` JSON blob alongside the Phase B `alarm` obje
Wonderware TCP sidecar backend this replaced was retired — see
[drivers/Historian.Wonderware.md](drivers/Historian.Wonderware.md).)
Design reference: [docs/plans/2026-06-14-galaxy-phase-c-historian-design.md](plans/2026-06-14-galaxy-phase-c-historian-design.md).
> **The backend is pluggable (2026-10).** `ServerHistorian:Backend` selects `Gateway` (the default — everything
> described below as "the gateway") or `SqlServer` (a plain SQL Server database the server reads and writes
> itself, which also supports OPC UA **HistoryUpdate**). See [Backend selection](#backend-selection-serverhistorianbackend),
> [SQL Server backend](#sql-server-backend) and [HistoryUpdate](#historyupdate). Wherever this page says
> "the historian" without qualification it means the selected backend; sections that only apply to one backend say so.
Design references: [docs/plans/2026-06-14-galaxy-phase-c-historian-design.md](plans/2026-06-14-galaxy-phase-c-historian-design.md)
(Phase C) and [docs/plans/2026-10-09-pluggable-historian-design.md](plans/2026-10-09-pluggable-historian-design.md) (pluggable backends).
---
@@ -53,6 +60,34 @@ A tag that is both historized and a native alarm:
---
## Backend selection (`ServerHistorian:Backend`)
`ServerHistorian:Backend` (case-insensitive, trimmed; default `Gateway`) picks the provider
(`IHistorianBackendProvider`) that builds all five historian seams — the HistoryRead data source, the continuous-
historization value writer, the alarm-history writer, the tag provisioner and (optionally) the HistoryUpdate
updater. `HistorianBackendRegistry` in the Host owns the name table. An unknown name fails host start with an
`OptionsValidationException` that lists the allowed values (`Gateway`, `SqlServer`), regardless of whether anything
consumes the historian. The existing enable gates (`ServerHistorian:Enabled`, `AlarmHistorian:Enabled`,
`ContinuousHistorization:Enabled`) are unchanged — they simply get their objects from the selected provider.
| | `Gateway` (default) | `SqlServer` |
|---|---|---|
| Storage | AVEVA Historian via the HistorianGateway sidecar | `{Schema}` tables in a SQL Server database you point it at |
| Value kinds recorded | `Double`, `Int64`, `Boolean` (numeric analog only) | all: `Double`, `Int64`, `Boolean`, `String`, `DateTime`, plus `Empty` (a Bad sample with no value) |
| Status code stored | Sent as DA quality but **discarded** by the gateway (C-002); AVEVA stores every sample Good | Stored bit-for-bit (`StatusCode` int column) |
| Comms-loss / driver-Bad publishes | Dropped, counted in `DroppedUnsupportedKind` | Stored as an `Empty` outage-marker sample carrying the Bad code (see [below](#typed-values-real-status-and-outage-markers)) |
| Processed aggregates | Average, Minimum, Maximum, Total, Count | The thirteen in the [aggregate table](#aggregates-processed-reads) |
| HistoryUpdate | No (`BadHistoryOperationUnsupported`) | Yes (Insert / Replace / Update / DeleteRaw / DeleteAtTime) |
| Alarm history | gateway `SendEvent` | `Event` table (insert-if-absent) |
| Provisioning (`EnsureTags`) | numeric types only; others skipped | `MERGE` upsert of every request, all types |
| Connection | `Endpoint` + `ApiKey` + TLS keys (top-level `ServerHistorian:*`) | `ServerHistorian:SqlServer:*` |
Recording stays **Primary-only** (#537) and **at-least-once** with the #538 per-tag quarantine on both backends;
HistoryRead and HistoryUpdate are served from any node (the database is shared). The historian connection string is
**not** a ConfigDb string, and a driver-only node may carry it without breaking the Phase 4 "no ConfigDb" posture.
---
## ServerHistorian configuration
The `ServerHistorian` section in `appsettings.json` controls the historian read path.
@@ -63,6 +98,7 @@ and all HistoryRead calls on historized nodes return `GoodNoData` (empty, not an
{
"ServerHistorian": {
"Enabled": false,
"Backend": "Gateway",
"Endpoint": "",
"ApiKey": "",
"UseTls": true,
@@ -72,24 +108,51 @@ and all HistoryRead calls on historized nodes return `GoodNoData` (empty, not an
"MaxTieClusterOverfetch": 65536,
"HistoryReadBatchParallelism": 4,
"MaxConcurrentHistoryReadBatches": 16,
"HistoryReadDeadline": "00:01:00"
"HistoryReadDeadline": "00:01:00",
"SqlServer": {
"ConnectionString": "",
"Schema": "hist",
"CommandTimeout": "00:00:30",
"AutoCreateSchema": true
}
}
}
```
| Key | Type | Default | Description |
|---|---|---|---|
| `Enabled` | bool | `false` | Enable the live `GatewayHistorianDataSource`. `false` → `NullHistorianDataSource` (empty reads). |
| `Endpoint` | string | `""` | Absolute gateway URI, e.g. `https://host:5222`. Scheme selects transport (`https://` = TLS, `http://` = h2c plaintext). Required when `Enabled`. |
| `ApiKey` | string | `""` | The gateway peppered-HMAC key (`histgw_<id>_<secret>`) sent as `Authorization: Bearer`. Required when `Enabled`. **Supply via env `ServerHistorian__ApiKey`.** |
| `UseTls` | bool | `true` | Connect over TLS; must match the `Endpoint` scheme. |
| `AllowUntrustedServerCertificate` | bool | `false` | Accept a self-signed / untrusted server certificate (dev / on-prem only). |
| `CaCertificatePath` | string\|null | `null` | PEM CA file pinning the gateway's TLS chain. Null/empty uses the OS trust store. |
| `CallTimeout` | TimeSpan | `00:00:30` | Per-call deadline applied to each unary gateway read. A read for a tag the historian does not yet know about — e.g. a freshly-historized tag that has **not** been provisioned (see [Tag auto-provisioning](#tag-auto-provisioning-ensuretags)) — can run to this **full** deadline before the gateway errors, so an unprovisioned-tag HistoryRead can block for ~30 s. Lower this to fail faster, at the cost of truncating legitimately long reads. |
| `MaxTieClusterOverfetch` | int | `65536` | Maximum samples the server will fetch in one shot to page through a tie cluster (multiple samples sharing one `SourceTimestamp`). A cluster larger than this ceiling fails `BadHistoryOperationUnsupported`. Raise to handle abnormally large tie clusters; the default covers all normal-data cases. |
| `Enabled` | bool | `false` | Enable the selected backend's read client. `false` → `NullHistorianDataSource` (empty reads) and no HistoryUpdate. |
| `Backend` | string | `Gateway` | `Gateway` or `SqlServer` (case-insensitive). Unknown ⇒ host start fails. See [Backend selection](#backend-selection-serverhistorianbackend). |
| `Endpoint` | string | `""` | **Gateway backend only.** Absolute gateway URI, e.g. `https://host:5222`. Scheme selects transport (`https://` = TLS, `http://` = h2c plaintext). Required when `Enabled`. |
| `ApiKey` | string | `""` | **Gateway backend only.** The gateway peppered-HMAC key (`histgw_<id>_<secret>`) sent as `Authorization: Bearer`. Required when `Enabled`. **Supply via env `ServerHistorian__ApiKey`.** |
| `UseTls` | bool | `true` | **Gateway backend only.** Connect over TLS; must match the `Endpoint` scheme. |
| `AllowUntrustedServerCertificate` | bool | `false` | **Gateway backend only.** Accept a self-signed / untrusted server certificate (dev / on-prem only). |
| `CaCertificatePath` | string\|null | `null` | **Gateway backend only.** PEM CA file pinning the gateway's TLS chain. Null/empty uses the OS trust store. |
| `CallTimeout` | TimeSpan | `00:00:30` | **Gateway backend only.** Per-call deadline applied to each unary gateway read. A read for a tag the historian does not yet know about — e.g. a freshly-historized tag that has **not** been provisioned (see [Tag auto-provisioning](#tag-auto-provisioning-ensuretags)) — can run to this **full** deadline before the gateway errors, so an unprovisioned-tag HistoryRead can block for ~30 s. Lower this to fail faster, at the cost of truncating legitimately long reads. |
| `MaxTieClusterOverfetch` | int | `65536` | (Node-manager paging; all backends — a `SqlServer` PK rules out tie clusters, so it is moot there.) Maximum samples the server will fetch in one shot to page through a tie cluster (multiple samples sharing one `SourceTimestamp`). A cluster larger than this ceiling fails `BadHistoryOperationUnsupported`. Raise to handle abnormally large tie clusters; the default covers all normal-data cases. |
| `HistoryReadBatchParallelism` | int | `4` | Max historized nodes served **concurrently** within one HistoryRead batch (arch-review 03/S3). The SDK's `HistoryRead*` override surface is synchronous, so each arm bridges once at its boundary; this turns per-node serving from sequential (`N × CallTimeout` worst case) into `⌈N / P⌉ × CallTimeout`. Also caps the gateway-load multiplication a single client can command. `≤ 1` serves sequentially. |
| `MaxConcurrentHistoryReadBatches` | int | `16` | Process-wide limit on HistoryRead batches served **concurrently** (arch-review 03/S3). A flood of history clients degrades to `BadTooManyOperations` rather than exhausting the SDK request-thread pool. With `HistoryReadBatchParallelism` this caps total in-flight gateway reads at `MaxConcurrentHistoryReadBatches × HistoryReadBatchParallelism` (default 16 × 4 = 64). A batch that cannot get a slot within `min(HistoryReadDeadline, 5 s)` fails every handle `BadTooManyOperations`. |
| `HistoryReadDeadline` | TimeSpan | `00:01:00` | Per-request wall-clock deadline for a HistoryRead batch (arch-review 03/S3). A node still in flight at the deadline surfaces `BadTimeout`, so one request can never hold an SDK request thread longer than this regardless of node count. Non-positive = unbounded (a `Validate()` warning). |
| `MaxConcurrentHistoryReadBatches` | int | `16` | Process-wide limit on HistoryRead (and HistoryUpdate) batches served **concurrently** (arch-review 03/S3). A flood of history clients degrades to `BadTooManyOperations` rather than exhausting the SDK request-thread pool. With `HistoryReadBatchParallelism` this caps total in-flight gateway reads at `MaxConcurrentHistoryReadBatches × HistoryReadBatchParallelism` (default 16 × 4 = 64). A batch that cannot get a slot within `min(HistoryReadDeadline, 5 s)` fails every handle `BadTooManyOperations`. |
| `HistoryReadDeadline` | TimeSpan | `00:01:00` | Per-request wall-clock deadline for a HistoryRead (and HistoryUpdate) batch (arch-review 03/S3). A node still in flight at the deadline surfaces `BadTimeout`, so one request can never hold an SDK request thread longer than this regardless of node count. Non-positive = unbounded (a `Validate()` warning). |
**`ServerHistorian:SqlServer`** (read only when `Backend=SqlServer`; bound by `HistorianBackendRegistry.BindSqlServer`):
| Key | Type | Default | Validation | Description |
|---|---|---|---|---|
| `SqlServer:ConnectionString` | string | `""` | **Required** (non-blank) | ADO.NET connection string for the historian database. Never the ConfigDb. Supply via `${secret:OT-HISTORIAN-SQL}` / environment (`ServerHistorian__SqlServer__ConnectionString`) — never commit a real one. The database must already exist; `AutoCreateSchema` creates the *schema and tables*, not the database. |
| `SqlServer:Schema` | string | `hist` | Must match `^[A-Za-z_][A-Za-z0-9_]{0,127}$` (it is spliced into DDL/DML) | SQL schema that holds `SchemaVersion`, `Tag`, `Sample`, `Event` and the `SampleRow`/`TimeRow` table types. |
| `SqlServer:CommandTimeout` | TimeSpan | `00:00:30` | Must be `> 0` | Per-command timeout, rounded up to whole seconds (minimum 1 s). The SQL backend does **not** use `CallTimeout`. |
| `SqlServer:AutoCreateSchema` | bool | `true` | — | `true`: on first use apply any missing embedded script under a `sp_getapplock` (`OtOpcUaHistorianSchema`), so concurrent nodes serialise. `false`: only verify `SchemaVersion >= 1`; if behind, **every** historian call fails with a clear error naming the missing version (surfaced in the read client's health `LastError`, never a host crash). A failed bootstrap is retried on the next call, not latched. |
**Validation rules** (`ServerHistorianOptionsValidator`, driver-role nodes only; admin-only nodes never build historian singletons):
- `Backend` must name a registered backend — checked **always**, with no I/O, even when nothing is enabled.
- Everything else is **consumer-gated**: it runs only when `ServerHistorian:Enabled` **or** `AlarmHistorian:Enabled` is true
(the alarm sink sources its connection from this section even when `ServerHistorian` is off).
- `Backend=Gateway`: `Endpoint` must be an absolute http(s) URI, `ApiKey` non-empty, and `UseTls` must agree with the endpoint scheme.
- `Backend=SqlServer`: the three `SqlServer` rules above. `Endpoint`/`ApiKey`/`UseTls` are **not** required.
- ⚠️ `Backend=SqlServer` with a malformed `Schema` throws `ArgumentException` while the provider is built, so it stops
host start even when no consumer is enabled (the validator is consumer-gated, provider construction is not).
- The alarm-only warning at startup (`AlarmHistorian` on, `ServerHistorian` off) names the selected backend.
> **Do not commit `ApiKey` to `appsettings.json`.** Set it via the environment variable
> `ServerHistorian__ApiKey`, a secrets store, or a deployment-time overlay. The checked-in default is
@@ -108,11 +171,14 @@ write path) and the `ContinuousHistorization` section (driver-value capture). Al
**same** gateway — but only `ServerHistorian` carries the connection (endpoint/key/TLS); the other two
source it from there.
> **Continuous-historization value quality is server-stamped.** The gateway's `WriteLiveValues` SQL path
> **discards** the quality the recorder sends (a server-managed column; 0.2.0 Contracts C-002), so persisted
> quality is stamped gateway-side regardless of what the `ContinuousHistorizationRecorder` provides (it
> hardcodes OPC-DA Good, `192`). Capturing real per-node quality will not persist until the gateway honors
> the field — see `GatewayHistorianValueWriter`'s remarks (archreview 06/C-7).
> **Continuous-historization value quality on the Gateway backend is still server-stamped.** The recorder now
> carries each sample's **real** OPC UA status (it used to hard-code Good `192`, so a Bad sample was historized as
> Good). `GatewayHistorianValueWriter` maps that status to an OPC-DA quality byte with
> `GatewayQualityMapper.ToDaQuality` (the inverse of the read-side table; unknown codes fall back by severity:
> Good ⇒ 192, Uncertain ⇒ 64, Bad ⇒ 0). But the gateway's `WriteLiveValues` SQL path **discards** the quality
> it is sent (a server-managed column; 0.2.0 Contracts C-002), so **AVEVA still stores every recorded value as
> Good** — a pre-existing limitation of that backend, now documented rather than fixed (archreview 06/C-7). The
> `SqlServer` backend stores the real status.
> ✅ **Continuous value capture is LIVE-PROVEN END-TO-END
> ([#491](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/491) CLOSED — gate run 2026-08-01,
@@ -145,12 +211,13 @@ source it from there.
## ContinuousHistorization configuration
`ContinuousHistorizationOptions` binds the `ContinuousHistorization` section. It carries **only** the
recorder + outbox knobs — the gateway connection (endpoint / key / TLS) is sourced from
`ServerHistorian` (above), never restated here.
recorder + outbox knobs — the backend connection (gateway endpoint / key / TLS, or the SQL Server connection
string) is sourced from `ServerHistorian` (above), never restated here. The outbox is a node-local FasterLog
and is registered the same way for every backend.
| Key | Type | Default | Description |
|---|---|---|---|
| `Enabled` | bool | `false` | `true` **and** a configured `ServerHistorian` wires the FasterLog outbox + gateway value-writer and spawns the `ContinuousHistorizationRecorder`. `false` → no recorder; driver tag values are not historized. |
| `Enabled` | bool | `false` | `true` **and** a configured `ServerHistorian` wires the FasterLog outbox + the selected backend's value writer and spawns the `ContinuousHistorizationRecorder`. `false` → no recorder; driver tag values are not historized. |
| `OutboxPath` | string | `""` | **Directory** holding the FasterLog segment + commit files. Required when `Enabled`. In production set an **absolute** path on durable storage — a relative path resolves against the host's working directory. |
| `CommitMode` | string | `PerEntry` | `PerEntry` fsyncs before the append returns (no loss window); `Periodic` batches commits onto a background timer every `CommitIntervalMs`. Parsed case-insensitively; an unrecognized value falls back to `PerEntry`. **See the batch-commit note below — `PerEntry` is one fsync per ingest *burst*, not per value.** |
| `CommitIntervalMs` | int | `100` | Periodic-mode commit cadence (ms); must be `> 0` under `Periodic`, ignored under `PerEntry`. |
@@ -181,6 +248,50 @@ recorder + outbox knobs — the gateway connection (endpoint / key / TLS) is sou
> lands, raising `DrainBatchSize` widens each hop's payload but not the hop count — which is why the
> concurrency window, not the batch width, is what governs drain throughput today.
### Typed values, real status and outage markers
The recorder no longer assumes "a numeric Good sample". Each mux delta is coerced to a typed `HistorianValue`
(`Double` / `Int64` / `Boolean` / `String` / `DateTime` / `Empty`; a `ulong` above `long.MaxValue` becomes `Double`;
arrays and anything else are dropped and counted) and appended with the **real** status the driver published.
A value whose kind is not in the selected backend's `SupportedKinds` is dropped at capture and counted in
`RecorderStatus.DroppedUnsupportedKind` (the field was `DroppedNonNumeric` before the rename; `Empty` is supported
by `SqlServer`, not by `Gateway`).
⚠️ **Comms loss is historized — as an outage marker (design §9.10).** `DriverHostActor.FanCommsLossToDependencyMux`
(#481) fans `(null, BadNoCommunication)` for every value ref of a comms-lost driver to the dependency mux, and the
recorder cannot tell that from a driver's own Bad publish. So:
- **`SqlServer`:** each disconnect stores **one `Empty` / `BadNoCommunication` (`0x80310000`) sample per historized
tag**. A driver's own Bad publish is stored the same way (`Empty` kind plus its own code — e.g. Modbus
`BadCommunicationError` `0x80050000`). This is deliberate: raw reads show the gap, and aggregates and AtTime
interpolation do not bridge it. Cost: one extra row per tag per outage.
- **`Gateway`:** the marker is `Empty`, which that backend does not support, so it is dropped and counted in
`DroppedUnsupportedKind`.
### Outbox format v2 and rollback
Outbox entries are written in format **v2**: `"HV2\0"` magic + version byte `2` + id + tag + value kind + a
kind-specific payload + the 32-bit status + timestamp. Entries left in an outbox by an older build (the fixed
38-bytes-plus-tag layout, always quality 192) are still **read** — as a `Double` with status `Good`. A record that
matches neither layout is **skipped and counted, never thrown**, so a corrupt record cannot jam the drain (this also
holds during the FasterLog recovery scan at start-up); the lifetime count is the gauge
**`otopcua.historian.outbox.unreadable`** (`IHistorizationOutbox.UnreadableCount`). A growing value means
undecodable records are being skipped.
⚠️ **Rollback: drain or delete the outbox before downgrading.** A node running a pre-v2 build cannot read v2
records — its outbox deserializes every record in its constructor and fails to start on the first v2 one. Before
rolling a node back, let the outbox drain to zero (`otopcua.historization.outbox.depth`) or stop the node and
delete the directory named by `ContinuousHistorization:OutboxPath` (accepting the loss of anything un-drained).
Upgrading needs no action.
**Delivery semantics on `SqlServer`.** Writes are *insert-if-absent* on `(TagId, SourceTimeUtc)`: the outbox's
at-least-once redelivery (and both pair nodes writing the same sample in a boot window) converges on one row, and the
first-stored sample wins. **Duplicate timestamps inside one batch collapse to the first.** A `NaN` / `±Infinity` `Double`
cannot be stored in a SQL `float`, so the **value writer stores it as an `Empty` sample keeping its status** — the tag
is not quarantined for one bad reading (HistoryUpdate behaves differently: see [below](#historyupdate)). A duplicate-key
error (2627/2601) rolls the whole statement back, so the writer re-runs the statement (at most twice) instead of
treating the error as success.
---
## A poison tag no longer pins the outbox (#538)
@@ -257,8 +368,11 @@ tag (with the resolved historian name — the `historianTagname` override when s
`FullName`); the hook is wrapped so a faulted, slow, or throwing provisioner can **never** block or fail a
deploy. Non-numeric (`String`/`DateTime`/`Reference`) types are skipped, not provisioned.
**Wiring + gating.** The concrete provisioner is the gateway-backed `GatewayTagProvisioner`, registered by
the Host's `AddHistorianProvisioning` (`GatewayHistorian.CreateProvisioner`) and gated on the **same**
**Wiring + gating.** The concrete provisioner comes from the selected backend's provider (`CreateProvisioning`):
the gateway-backed `GatewayTagProvisioner`, or — for `SqlServer` — `SqlServerTagProvisioner` (one `MERGE … WITH (HOLDLOCK)`
upsert per chunk of 400 tags; every data type is provisionable so `skipped` is always 0; a blank or over-400-character
name fails alone; a null unit/description in a request keeps the stored value). It is registered by the Host's
`AddHistorianProvisioning` and gated on the **same**
`ServerHistorian:Enabled` flag as the read path — both target the same single gateway. When the section is
disabled (or absent) the applier resolves the no-op `NullHistorianProvisioning` and **provisioning does not
run** (so on a read-only-historian deployment, ensure tags exist by some other means). The Host requires the
@@ -275,13 +389,15 @@ for this node), which is the signal to check that `ServerHistorian:Enabled=true`
### Deferred types and the UInt16 fallback (#600 item 4)
Historization is **numeric-analog only** in v1. `HistorianTypeMapper` classifies exactly nine types as
> **Gateway backend only.** The `SqlServer` backend provisions and records every type (`String` and `DateTime` included).
Historization on the **Gateway** backend is **numeric-analog only**. `HistorianTypeMapper` classifies exactly nine types as
historizable (`Boolean`, `Int16/32/64`, `UInt16/32/64`, `Float32`, `Float64`):
- **Deferred: `String`, `DateTime`, `Reference`.** The historian's SQL analog write path cannot carry them.
Provisioning skips them (counted in the `skipped=` tally, via `IsHistorizable`), the mapper throws
`NotSupportedException` for them, and `ContinuousHistorizationRecorder` drops their values and counts them
in `DroppedNonNumeric`. A tag authored `isHistorized` with one of these types therefore silently records
in `DroppedUnsupportedKind` (renamed from `DroppedNonNumeric`). A tag authored `isHistorized` with one of these types therefore silently records
nothing; HistoryRead on it returns whatever the historian holds (typically `GoodNoData`).
- **Fallback: `UInt16 -> Uint4`.** The historian's `UInt2` write path is unproven upstream, so UInt16 tags
are provisioned as the wider `UInt4`. Values are unaffected; only the historian-side storage type differs.
@@ -339,7 +455,7 @@ The server supports all four OPC UA HistoryRead variants:
| Variant | Node type | CLI `--aggregate` |
|---|---|---|
| **Raw** | Historized variable | (omit `--aggregate`) |
| **Processed** | Historized variable | `Average`, `Minimum`, `Maximum`, `Total`, `Count` |
| **Processed** | Historized variable | `Average`, `Minimum`, `Maximum`, `Count`, `Start`, `End`, `StandardDeviation` (the CLI's own list; `SqlServer` serves thirteen aggregates — see the [aggregate table](#aggregates-processed-reads)) |
| **AtTime** | Historized variable | n/a (client supplies exact timestamps) |
| **Events** | Equipment folder (event notifier) | n/a |
@@ -384,6 +500,7 @@ node without `isHistorized` has no HistoryRead access bit and answers
| Historized node, historian NOT configured (`Enabled=false` / Null source) | `GoodNoData` (empty) |
| Non-historized node | `BadHistoryOperationUnsupported` |
| Backend timeout or exception | `BadHistoryOperationUnsupported` per node; other nodes in the same batch are unaffected |
| `Backend=SqlServer`, `AutoCreateSchema=false`, schema missing/behind | every read/write fails with a clear error naming the missing schema version (health `LastError`); reads answer `BadHistoryOperationUnsupported` per node; no host crash |
A historized node with no historian configured never returns an error status — it returns
empty. This means a deployment can author and publish historized tags before the historian
@@ -450,7 +567,9 @@ buckets surface as `BadNoData`.
`HistoryAggregateType` carries the wider Part 13 set (Interpolative, TimeAverage, Start, End, Delta, Range,
Min/MaxActualTime, Start/EndBound, DeltaBounds, DurationGood/Bad, PercentGood/Bad, WorstQuality,
NumberOfTransitions, AnnotationCount, StandardDeviationPopulation, VariancePopulation); `OtOpcUaNodeManager.MapAggregate`
maps each standard aggregate NodeId. The **HistorianGateway** backend only has retrieval modes for the original
maps each standard aggregate NodeId. The **`SqlServer`** backend serves the thirteen listed in the
[aggregate table](#aggregates-processed-reads); everything else (e.g. `Interpolative`, `TimeAverage`,
`StandardDeviationPopulation`) is `BadAggregateNotSupported` for that node. The **HistorianGateway** backend only has retrieval modes for the original
five (Average, Minimum, Maximum, Total, Count): for any other member `AggregateModeMapper` throws
`HistoryAggregateNotSupportedException`, which the HistoryRead path turns into `BadAggregateNotSupported` for that
node only (not a generic failure, and not counted as a backend fault). The OPC UA Client driver forwards every
@@ -474,10 +593,236 @@ member upstream. A NodeId outside the mapped set (e.g. `StandardDeviationSample`
### Redundancy and authorization
History reads are served from any node — there is no Primary gate. Authorization is the
standard OPC UA `HistoryRead` permission enforced by the SDK through the `AccessLevels.HistoryRead`
bit set at materialization. A session without sufficient permissions receives
`BadUserAccessDenied` from the SDK before the dispatch reaches the historian.
History reads are served from any node — there is no Primary gate. The `HistoryRead` AccessLevel bit set at
materialization admits a node to the read dispatch. With `Security:Acl:Enabled=false` (the default) HistoryRead is
**ungated** for any admitted session; with it `true`, the scoped `HistoryRead` grant decides each node
(see [security.md](security.md#data-plane-authorization)). **HistoryUpdate is always gated** — see [HistoryUpdate](#historyupdate).
---
---
## SQL Server backend
Selected with `ServerHistorian:Backend=SqlServer`. Implemented in `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer`
(the project does not reference the Gateway project). All five seams share one connection helper, one schema
bootstrapper and one tag catalog, so the schema is ensured once per process and tag ids are cached once.
### Schema
Created by the embedded idempotent script `Schema/001_initial.sql` (mirrored at
[`docs/historian/sqlserver/001_initial.sql`](historian/sqlserver/001_initial.sql)); `$(Schema)` is the configured schema.
| Object | Purpose |
|---|---|
| `SchemaVersion(Version, AppliedUtc)` | Applied-script ledger. Required version: **1**. |
| `Tag(TagId int identity, Name nvarchar(400) unique, DataType tinyint, EngineeringUnit, Description, CreatedUtc, UpdatedUtc)` | One row per historian tagname. `DataType` is a `DriverDataType`. |
| `Sample(TagId, SourceTimeUtc datetime2(7), ValueKind tinyint, NumValue float, IntValue bigint, StrValue nvarchar(max), StatusCode int, ServerTimeUtc)` | **PK `(TagId, SourceTimeUtc)`** — one value per timestamp (Part 11), which makes redelivery idempotent and rules out tie clusters. `StatusCode` is the uint status stored bit-for-bit. |
| `Event(EventId nvarchar(64) PK nonclustered, SourceName, AlarmId, AlarmName, AlarmTypeName, Severity, EventKind, Message, UserName, Comment, EventTimeUtc, ReceivedTimeUtc)` | Alarm history. Clustered on `(EventTimeUtc, EventId)`; extra index `(SourceName, EventTimeUtc)`. |
| `SampleRow`, `TimeRow` table types | TVPs for batched sample writes/merges, AtTime and delete-at-time. |
Column mapping per `ValueKind`: `Double` → `NumValue`; `Int64` → `IntValue` + `NumValue` (as a double, for aggregates);
`Boolean` → `IntValue` 0/1 + `NumValue` 0/1; `String` → `StrValue`; `DateTime` → `IntValue` (UTC ticks); `Empty` → all NULL.
`ValueKind` codes: Empty 0, Double 1, Int64 2, Boolean 3, String 4, DateTime 5. Timestamps are bound as `datetime2(7)`
everywhere (full 100 ns precision — never a `DataTable`, which would round to 3.33 ms).
**Applying the script by hand** (needed when `AutoCreateSchema=false`): create the database, then
`sqlcmd -S <server> -U <user> -P <pw> -d <db> -I -v Schema=hist -i docs/historian/sqlserver/001_initial.sql`
(`-v Schema=` supplies the `$(Schema)` placeholder; verified against the docker-dev SQL container). Re-running is a no-op.
### Writes
- **Live values** (`IHistorianValueWriter`): resolve `TagId` through a cache, creating the `Tag` row on a miss
(`MERGE … WITH (HOLDLOCK)`; `DataType` from the first non-Empty value's kind, provisioning corrects it later), then
`INSERT … SELECT … WHERE NOT EXISTS` through the `SampleRow` TVP. A null timestamp is stamped now;
`ServerTimeUtc = SYSUTCDATETIME()`. It returns `false` (never throws) on a SQL failure so the recorder's retry/quarantine
owns it. Semantics: [delivery semantics](#outbox-format-v2-and-rollback).
- **Alarm events** (`IAlarmHistorianWriter`): per event, insert-if-absent keyed by a deterministic `EventId` = lowercase-hex
SHA-256 over the length-prefixed fields `EquipmentPath, AlarmId, EventKind, TimestampUtc.Ticks, User, Comment, Message`.
Outcomes: `Ack` on insert or already-present (including a concurrent writer winning a 2627/2601 race); `RetryPlease` on
transient SQL errors (connection, timeout, deadlock 1205 …), bootstrap failures, cancellation and any unclassified error;
`PermanentFail` (dead-letter) only for data errors SQL will reject forever (truncation 2628/8152, conversion, null, range, constraint).
`SourceName` is the event's `EquipmentPath`; `Severity` is the numeric Part 9 value (Low 200, Medium 500, High 700, Critical 900).
- **Tag names** are `nvarchar(400)`. `EnsureTags` fails a longer name individually, but the live-value writer's tag-create
path does **not** reject one (the parameter silently truncates) — keep historian tagnames under 400 characters.
### Raw reads
`ReadRawAsync` returns every sample as stored — outage markers (`Empty` + Bad status) included.
| Request | Window | Order |
|---|---|---|
| `start < end` | `[start, end)` | ascending |
| `end < start` (Part 11 reverse) | `(end, start]` | descending |
| `start == end` | the sample exactly at that time | — |
`NumValuesPerNode == 0` is unbounded (larger values clamp to `int.MaxValue`). An unknown tag is an empty result (`GoodNoData`);
reads **never create** tag rows. The node manager's paging is unchanged; because the PK rules out tie clusters, resuming
"from the last timestamp inclusive and trimming it" is exact.
### AtTime reads
One statement per call: the distinct requested times go in a `TimeRow` TVP and each picks up its previous-or-equal sample `p` and
next sample `n` via two `OUTER APPLY`s. A *usable* sample has severity Good or Uncertain **and** a value (an `Empty` sample is an
outage marker and counts as Bad whatever its code). Per requested time `T` (the result is stamped `T`):
| Case | Result |
|---|---|
| `p` at exactly `T` | the stored value and status, unchanged |
| no `p` (before the first sample), or unknown tag | `BadNoData`, null |
| **`p` is Bad** (`T` lies inside an outage) | **null value carrying `p`'s Bad status code** (info bits stripped) — never better than its source |
| `p` usable, **`n` Bad** | **`p`'s value held** (not interpolated into the outage), `UncertainDataSubNormal` + Interpolated |
| `Double` / `Int64` with a usable `n` | linear interpolation of `NumValue`; `Int64` is computed **through `double`** and rounded to the nearest integer (ties away from zero), keeping the `Int64` type. Status `Good` + Interpolated when both neighbours are Good, else `UncertainDataSubNormal` + Interpolated |
| `Boolean` / `String` / `DateTime`, **or `T` after the last sample** | **stepped**: `p`'s value with `p`'s status + Interpolated |
⚠️ Because `Int64` interpolation goes through `double`, magnitudes beyond 2^53 lose precision.
### Events
`[start, end)` by `EventTimeUtc`; **a reversed window (`end < start`) is swapped and returned ascending**. A null `sourceName` means all
sources, otherwise `SourceName = @source`. Ordered `(EventTimeUtc, EventId)` ascending. A bounded read fetches `maxEvents + 1` rows and
returns a non-null continuation point when the extra row exists; the node manager does not currently issue or resume event
continuation points, so a client-capped read returns only the first page.
### Aggregates (Processed reads)
> **These are simple per-sample definitions, not full OPC UA Part 13.** There is no time-weighting, no interpolated interval-boundary
> value and no partial-interval status. They are pushed down to T-SQL and suit trend-style consumption. Any `HistoryAggregateType` not
> in the table throws `HistoryAggregateNotSupportedException` ⇒ `BadAggregateNotSupported` for that node (e.g. `Interpolative`,
> `TimeAverage`, `StartBound`, `DurationGood`, `StandardDeviationPopulation`).
**Bucketing.** One result per interval of `[start, end)`; bucket *i* = `[start + i·interval, start + (i+1)·interval)`, start-inclusive /
end-exclusive to the tick, the last bucket truncated at `end`. `interval <= 0` ⇒ a single interval `[start, end)`; `start == end` ⇒ no
intervals. The bucket index is computed in **100 ns ticks** (not milliseconds): `DATEDIFF_BIG(microsecond, @start, t) × 10` plus a
sub-microsecond correction from the two tick digits, divided by the interval in ticks. (`DATEDIFF` counts unit *boundaries crossed*, so a
millisecond formulation misplaces samples near a bucket edge, and a nanosecond one overflows past ≈292 years.) A window of more than
**100 000 intervals** is refused before any I/O and reported `BadHistoryOperationUnsupported` for that node. **Reverse windows**
(`end < start`) are swapped, computed forward **anchored at the earlier time**, and returned in **descending** bucket order. Every returned
status carries the historian *Calculated* info bit, `BadNoData` included.
**Sample classes.** *Usable* = severity Good or Uncertain **and** a stored value (not `Empty`). *Good* = usable with severity Good.
*Bad* = not usable.
| Aggregate | Computed over | Value timestamp |
|---|---|---|
| `Average`, `Minimum`, `Maximum`, `Total` | `AVG` / `MIN` / `MAX` / `SUM` of `NumValue` over usable samples | bucket start |
| `Count` | number of usable samples (`Int32`) | bucket start |
| `Range` | `Maximum − Minimum` | bucket start |
| `Start`, `End` | first / last usable sample by time (keeps the sample's type) | bucket start |
| `Delta` | `End − Start` (numeric samples) | bucket start |
| `MinimumActualTime`, `MaximumActualTime` | the usable sample with the smallest / largest `NumValue`; ties ⇒ earliest | **the sample's own time** |
| `PercentGood` | `100 × (Good samples) / (all samples)` | bucket start |
| `PercentBad` | `100 × (Bad samples) / (all samples)` | bucket start |
Status and edge rules:
- An **empty interval** (no samples) ⇒ `BadNoData`. **`Count` — and every value aggregate — over an interval whose samples are all Bad ⇒ `BadNoData`**, not `0`.
- Value aggregates are `Good` when every sample in the bucket is Good and `UncertainDataSubNormal` when the bucket also holds any Bad **or Uncertain** sample.
- `Count`, `Start`, `End`, `PercentGood`, `PercentBad` accept any value kind; the other aggregates (and `Delta`) on a bucket that holds a usable `String`/`DateTime` sample ⇒ `BadAggregateInvalidInputs` for that interval.
- **`PercentGood` / `PercentBad` and Uncertain samples:** Uncertain samples are counted in the denominator but are **neither Good nor Bad**, so `PercentGood + PercentBad < 100` whenever a bucket holds Uncertain samples. Both report status `Good` for any non-empty bucket.
- Everything except `Count` and `Start`/`End` is a `double`; `Int64` samples aggregate as doubles and `Boolean` as 0/1.
### Health
The read client reports `GetHealthSnapshot()` (query/failure counters, consecutive failures, `LastError` — e.g. a missing schema under
`AutoCreateSchema=false`). A SQL error on a read surfaces as `BadHistoryOperationUnsupported` for **that node only**; a cancelled read
(deadline, client abort) is not counted as a backend fault.
### Docker-dev recipe
The docker-dev `sql` container (`otopcua-dev-sql-1`, host port `14330`, `sa` with the committed dev password from
`docker-dev/docker-compose.yml`) already hosts the ConfigDb; give the historian its own database on the same instance.
```bash
# 1. Create the database (the backend creates the schema + tables on first use, not the database).
sqlcmd -S localhost,14330 -U sa -P 'OtOpcUa!Dev123' -C -I -Q "CREATE DATABASE OtOpcUaHistorian"
# 2. Point the central nodes at it. Add to the `environment:` of central-1 AND central-2 in docker-dev/docker-compose.yml
# (or a compose override), rebuild the images (the rig image goes stale), then `up -d`:
# ServerHistorian__Enabled: "true"
# ServerHistorian__Backend: "SqlServer"
# ServerHistorian__SqlServer__ConnectionString: "Server=sql,1433;Database=OtOpcUaHistorian;User Id=sa;Password=OtOpcUa!Dev123;TrustServerCertificate=True"
# ContinuousHistorization__Enabled: "true"
# ContinuousHistorization__OutboxPath: "/app/data/historization-outbox"
# (ServerHistorian__Endpoint / __ApiKey are not used by this backend.)
# 3. Mark a rig tag historized ("isHistorized": true in its TagConfig), deploy, then watch rows land:
sqlcmd -S localhost,14330 -U sa -P 'OtOpcUa!Dev123' -C -I -d OtOpcUaHistorian \
-Q "SELECT t.Name, COUNT(*) n, MAX(s.SourceTimeUtc) last FROM hist.Sample s JOIN hist.Tag t ON t.TagId=s.TagId GROUP BY t.Name"
```
Only the Primary of a pair records, so the row count advances once per published value, not twice. Read it back with
`Client.CLI historyread` and rewrite it with `Client.CLI historyupdate` ([Client.CLI.md](Client.CLI.md)). The password above is the
committed docker-dev credential, not a secret; use `${secret:…}` / environment for anything real.
### Integration tests
`tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests` (bootstrap incl. two concurrent bootstrappers, every kind
round-trip, redelivery, raw / AtTime / aggregate / event reads, every update result code) is gated on the environment variable
**`HISTORIAN_SQL_TEST_CONNECTION`** — a connection string with `CREATE`/`DROP DATABASE` rights. Absent ⇒ every test skips. Present ⇒ each
run creates a throwaway `OtOpcUaHistorian_T_<guid>` database and drops it on dispose.
```bash
export HISTORIAN_SQL_TEST_CONNECTION='Server=localhost,14330;User Id=sa;Password=OtOpcUa!Dev123;TrustServerCertificate=True;Encrypt=False'
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests
```
---
## HistoryUpdate
OPC UA **HistoryUpdate** lets a client insert, replace, update and delete stored raw history. It is available only when the selected
backend provides an updater — **`SqlServer` does; `Gateway` does not**, and an updater is registered only when `ServerHistorian:Enabled=true`.
Updates go straight to the backend (not through the outbox), are **not Primary-gated** (the database is shared), and share the HistoryRead
batch limiter and `HistoryReadDeadline`.
### Operations
| OPC UA request | Server override | Per-item results (`OperationResults`) |
|---|---|---|
| `UpdateDataDetails`, `PerformInsertReplace = Insert` | `HistoryUpdateData` | `GoodEntryInserted`; `BadEntryExists` if a sample already exists at that timestamp |
| …`Replace` | `HistoryUpdateData` | `GoodEntryReplaced`; `BadNoEntryExists` if none exists |
| …`Update` | `HistoryUpdateData` | `GoodEntryInserted` or `GoodEntryReplaced` (upsert) |
| `DeleteRawModifiedDetails` (`IsDeleteModified=false`) | `HistoryDeleteRawModified` | one node-level code: `Good`; `BadNoData` if `[start, end)` held nothing |
| `DeleteAtTimeDetails` | `HistoryDeleteAtTime` | per time: `Good`; `BadNoEntryExists` |
Each backend call is one transaction (`MERGE … OUTPUT $action` against the TVP). The node-level `StatusCode` stays `Good` when item results are
mixed (Part 11 reports item outcomes in `OperationResults`); it is Bad only for whole-node failures.
**Per-item rejections** (answered without touching SQL; other items still proceed): a null item ⇒ `BadInvalidArgument`;
`SourceTimestamp == DateTime.MinValue` ⇒ `BadInvalidTimestamp`; a value the historian cannot type (array, unsupported CLR type) ⇒
`BadTypeMismatch`; and a **`NaN` / `±Infinity` `Double` ⇒ `BadInvalidArgument`** — it cannot live in a SQL `float`, and unlike the live-value
writer (which stores it as `Empty`) an operator's explicit update is rejected rather than silently rewritten. A caller-supplied status code is stored as given.
**Duplicate timestamps inside one request** (compared at 100 ns): *Insert* — the first occurrence is applied, later ones are `BadEntryExists`;
*Replace* / *Update* — the last one wins and every earlier occurrence reports the winner's code; *DeleteAtTime* — the first reports the real
outcome, later ones `BadNoEntryExists`. **Unknown tag:** Insert / Update create the tag row; Replace and the deletes report `BadNoEntryExists`
(DeleteRaw: `BadNoData`) without creating anything.
**Unsupported ⇒ `BadHistoryOperationUnsupported`** (per node): the node is not historized; the backend has no updater (`Gateway`, or
`ServerHistorian:Enabled=false`); `PerformUpdateType.Remove`; `DeleteRawModifiedDetails.IsDeleteModified=true` (no modified-value history is
kept); and the out-of-scope detail types — **event** HistoryUpdate (`UpdateEventDetails` / `DeleteEventDetails`) and `UpdateStructureData` —
which are not overridden, so the SDK base rejects them. A backend exception is `BadInternalError` for that node and a deadline is `BadTimeout`.
No `AuditHistoryUpdateEvent` is emitted.
### The `HistoryWrite` AccessLevel bit
The SDK base admits a node to HistoryUpdate only if its `AccessLevel` carries `HistoryWrite`. The server ORs that bit in **only when the node is
historized *and* an updater is wired at materialise time** (the raw node and every UNS mirror; the surgical `UpdateTagAttributes` path recomputes
it the same way). `OtOpcUaServerHostedService.StartAsync` therefore wires the updater **before** installing the sink, and nodes materialised
earlier are never retro-updated. With `Backend=Gateway` the bit is never set.
### Authorization
HistoryUpdate is gated in **both** ACL modes (rewriting recorded history was never ungated). `WriteOperate` is deliberately **not** reused.
| `Security:Acl:Enabled` | Decision |
|---|---|
| `false` (default) | The session's API key is re-checked first (`AuthorizeKeyUsable`, Gitea #563 — an expired or disabled key loses its activation-time roles), then the session must carry the **`HistoryUpdate` role**. `ApiKeyInterimRoleMapper` confers it **only from a `Cluster`-scoped grant** carrying `NodePermissions.HistoryUpdate`; narrower-scoped grants confer nothing (the same under-grant rule as `WriteOperate`). Denied ⇒ `BadUserAccessDenied`. A `WriteOperate`-only key is denied. |
| `true` | `DataPlaneAuthorizer.Authorize(identity, OpcUaOperation.HistoryUpdate, nodeId, realm)` — the scoped `NodeAcl` grant decides per node and **replaces** the role check (never chained), so a session with no roles succeeds on the strength of a scoped grant. Denied ⇒ `BadUserAccessDenied` for that node. |
`NodePermissions.HistoryUpdate` (bit 12) is authorable in the AdminUI ACL editor and is **not** part of any bundle
(`ReadOnly` / `Operator` / `Engineer` / `Admin`) — it must be granted explicitly. See [security.md](security.md#data-plane-authorization).
---
+6 -5
View File
@@ -11,7 +11,7 @@ OtOpcUa is a multi-driver OPC UA server. The Core (`ZB.MOM.WW.OtOpcUa.Core` + `C
- `IAlarmSource` — driver-emitted OPC UA A&C events
- `IRediscoverable` — driver-initiated address-space rebuild notifications
- `ISkippedTagReporter` — authored tags the driver could not build into its poll table (see [Skipped tags](#skipped-tags) below)
- `IHistorianDataSource` — server-side historian read backend registration (the HistorianGateway backend) (drivers carry no history capability; Gitea #655)
- `IHistorianDataSource` — server-side historian read backend registration (the selected `ServerHistorian:Backend` — HistorianGateway or SQL Server) (drivers carry no history capability; Gitea #655)
Each driver opts into only the capabilities it supports. Every async capability call at the Server dispatch layer goes through `CapabilityInvoker` (`Core/Resilience/CapabilityInvoker.cs`), which wraps it in a Polly pipeline keyed on `(DriverInstanceId, HostName, DriverCapability)`. The `OTOPCUA0001` analyzer enforces the wrap at build time. Drivers themselves never depend on Polly; they just implement the capability interface and let the Core wrap it.
@@ -75,7 +75,8 @@ Three properties worth knowing:
| [MTConnect](MTConnect.md) | `Driver.MTConnect` (+ `.Contracts`) | — | Hand-rolled `System.Xml.Linq` HTTP/XML client against a vendor-neutral MTConnect Agent (`/probe`, `/current`, `/sample`) — no TrakHound dependency (dropped; see the driver doc) | IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IRediscoverable | Read-only by design (no `IWritable`). Production data plane is entirely `ISubscribable`/`OnDataChange` — `IReadable` is CLI/test-only. Browse is free via the Wave-0 universal discovery browser, no bespoke browser project. `IRediscoverable`/`IHostConnectivityProbe` have no server-side consumer yet — a pre-existing fleet-wide gap, not MTConnect-specific |
| [Sql](Sql.md) | `Driver.Sql` (+ `.Browser`, `.Contracts`) | A | `Microsoft.Data.SqlClient` behind an `ISqlDialect` seam (`SqlServerDialect` shipped; Postgres/ODBC deferred) | IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe | Read-only DB-poll driver with a bespoke schema browser. **Credentials are never persisted** — the deploy gate rejects a stored `connectionString` (#498) and catalog identifiers are validated against the live catalog's own spelling (#496). `SupportsOnlineDiscovery => false` by design (the bespoke browser wins). `SqlTagModel.Query` is deferred to P3 and rejected end-to-end by parser, validator, planner and editor. Merged `4ad54037` |
| [Calculation](Calculation.md) | `Driver.Calculation` | A | none — computes from other tags' RawPaths | IDriver, IDependencyConsumer, ISubscribable, IReadable | **Pseudo-driver**, not a wire protocol: signal-level tags computed from other tags via a script, with deploy-time `scriptId`-existence + Tarjan cycle gates. Its probe is parse-only (no backend to reach). `runTimeoutMs` is authorable through `CalculationDriverForm` (the "no typed config form" gap — G-1 — was closed 2026-07-27) |
| [Historian.Gateway](../Historian.md) | `Driver.Historian.Gateway` | — | `ZB.MOM.WW.HistorianGateway.Client` gRPC (`historian_gateway.v1`) | IHistorianDataSource (server-side read backend) + alarm `SendEvent` writer + `WriteLiveValues` recorder + `IHistorianProvisioning` | Not a tag driver — the sole historian backend. Registers `GatewayHistorianDataSource : IHistorianDataSource` for HistoryRead and serves alarm-write + continuous historization through the gateway. No `IDriver`/`ITagDiscovery` surface. (The retired Wonderware sidecar backend it replaced is documented at [Historian.Wonderware.md](Historian.Wonderware.md).) |
| [Historian.Gateway](../Historian.md) | `Driver.Historian.Gateway` | — | `ZB.MOM.WW.HistorianGateway.Client` gRPC (`historian_gateway.v1`) | IHistorianDataSource (server-side read backend) + alarm `SendEvent` writer + `WriteLiveValues` recorder + `IHistorianProvisioning` | Not a tag driver — the default historian backend (the other is `Historian.SqlServer`, below). Registers `GatewayHistorianDataSource : IHistorianDataSource` for HistoryRead and serves alarm-write + continuous historization through the gateway. No `IDriver`/`ITagDiscovery` surface. (The retired Wonderware sidecar backend it replaced is documented at [Historian.Wonderware.md](Historian.Wonderware.md).) |
| [Historian.SqlServer](../Historian.md#sql-server-backend) | `Driver.Historian.SqlServer` | — | `Microsoft.Data.SqlClient` (a plain SQL Server database, schema `hist` by default) | `IHistorianDataSource` + `IHistorianValueWriter` + `IAlarmHistorianWriter` + `IHistorianProvisioning` + `IHistorianDataUpdater` | Not a tag driver — a pluggable historian backend selected with `ServerHistorian:Backend=SqlServer`. Stores typed values with their real status, serves Raw / AtTime / Events / 13 push-down aggregates, and is the only backend that supports OPC UA **HistoryUpdate**. No `IDriver`/`ITagDiscovery` surface. |
## Per-driver documentation
@@ -99,8 +100,8 @@ Three properties worth knowing:
data-plane precedence rules, and a non-trivial known-limitations list need explaining up front:
- [MTConnect.md](MTConnect.md) — Agent config, `FullName`==DataItem `id`, `UNAVAILABLE`→`BadNoCommunication`, CONDITION-as-`String`, browse-via-universal-browser, fixture recipe, deferred write-back
- **Historian.Gateway** (server-side historian backend, not a tag driver) is documented in the main guide:
- [../Historian.md](../Historian.md) — HistorianGateway backend: read-path registration, HistoryRead dispatch, alarm store-and-forward (`SendEvent`), continuous historization (`WriteLiveValues`), `EnsureTags` provisioning, config keys, deployment prerequisites. (The retired Wonderware sidecar backend it replaced: [Historian.Wonderware.md](Historian.Wonderware.md).)
- **Historian.Gateway** and **Historian.SqlServer** (server-side historian backends, not tag drivers) are documented in the main guide:
- [../Historian.md](../Historian.md) — backend selection (`ServerHistorian:Backend`), the SQL Server backend and HistoryUpdate, plus the HistorianGateway backend: read-path registration, HistoryRead dispatch, alarm store-and-forward (`SendEvent`), continuous historization (`WriteLiveValues`), `EnsureTags` provisioning, config keys, deployment prerequisites. (The retired Wonderware sidecar backend it replaced: [Historian.Wonderware.md](Historian.Wonderware.md).)
- The full per-field spec (capability surface, config schema, addressing, data-type maps, connection settings, quirks for every driver) lives in [docs/v2/driver-specs.md](../v2/driver-specs.md). The overview pages above are the short path and are the current reference; that file is the v2-era per-field spec (see its status banner for which drivers it covers).
@@ -121,7 +122,7 @@ Each driver has a dedicated fixture doc that lays out what the integration / uni
## Related cross-driver docs
- [HistoricalDataAccess.md](../v1/HistoricalDataAccess.md) — v1 archive of the retired per-driver history design (aggregate mapping, continuation points). History is now served only by the server-side `IHistorianDataSource` (HistorianGateway); no driver implements a history capability, and nodes without `isHistorized` return `BadHistoryOperationUnsupported`.
- [HistoricalDataAccess.md](../v1/HistoricalDataAccess.md) — v1 archive of the retired per-driver history design (aggregate mapping, continuation points). History is now served only by the server-side `IHistorianDataSource` (the selected historian backend: HistorianGateway or SQL Server); no driver implements a history capability, and nodes without `isHistorized` return `BadHistoryOperationUnsupported`.
- [AlarmTracking.md](../AlarmTracking.md) — `IAlarmSource` event model and filtering. Implemented by Galaxy (native MxAccess alarms, working end-to-end), OPC UA Client, AB CIP, and FOCAS; AB Legacy, Modbus, S7, TwinCAT, and MQTT have no alarm source.
- [Subscriptions.md](../v1/Subscriptions.md) — how the Server multiplexes subscriptions onto `ISubscribable.OnDataChange`.
- [docs/v2/driver-stability.md](../v2/driver-stability.md) — historical design record of the A / B / C tier model. Only `DriverTier` + the `CapabilityPolicy` defaults per tier × capability (`DriverResilienceOptions.GetTierDefaults`) are live; its `MemoryTracking` / recycle / out-of-process isolation machinery was deleted (#522) and every driver runs Tier A.
@@ -80,7 +80,9 @@ It lives in **Runtime** because the seams it returns span `Core.Abstractions` an
- `ServerHistorianOptionsValidator` runs its three gateway checks (Endpoint, ApiKey, UseTls⇔scheme) only when `Backend == Gateway`; backend-specific options are validated there too, consumer-gated (`ServerHistorian:Enabled || AlarmHistorian:Enabled`), never at registry resolution.
- New `SqlServerHistorianOptions` validation: `ConnectionString` required, `Schema` a valid SQL identifier
(`^[A-Za-z_][A-Za-z0-9_]{0,127}$` — it is spliced into DDL/DML), `CommandTimeout > 0`.
- Validators remain driver-block-only (admin-only nodes never build historian singletons).
- Validators remain driver-block-only (admin-only nodes never build historian singletons). *As built:* an invalid `SqlServer:Schema`
also throws `ArgumentException` when the provider is constructed, so `Backend=SqlServer` + a malformed schema stops host start even
when no consumer is enabled.
- The `Program.cs` alarm-only warning names the selected backend instead of "gateway".
- The historian connection string is **not** a ConfigDb string; driver-only nodes may carry it (Phase 4 intact).
@@ -160,14 +162,14 @@ CREATE TABLE {s}.Sample (
CONSTRAINT FK_Sample_Tag FOREIGN KEY (TagId) REFERENCES {s}.Tag(TagId));
CREATE TABLE {s}.Event (
EventId nvarchar(64) NOT NULL PRIMARY KEY, -- deterministic SHA-256 hex of the event payload
EventId nvarchar(64) NOT NULL CONSTRAINT PK_Event PRIMARY KEY NONCLUSTERED, -- deterministic SHA-256 hex of the event payload
SourceName nvarchar(400) NOT NULL, -- AlarmHistorianEvent.EquipmentPath
AlarmId nvarchar(400) NOT NULL, AlarmName nvarchar(400) NOT NULL, AlarmTypeName nvarchar(100) NOT NULL,
Severity int NOT NULL, EventKind nvarchar(50) NOT NULL, Message nvarchar(max) NULL,
UserName nvarchar(200) NULL, Comment nvarchar(max) NULL,
EventTimeUtc datetime2(7) NOT NULL, ReceivedTimeUtc datetime2(7) NOT NULL);
CREATE CLUSTERED INDEX CX_Event_Time ON {s}.Event (EventTimeUtc, EventId); -- as built (§9.6); the sketched IX_Event_Time is replaced by this
CREATE INDEX IX_Event_Source_Time ON {s}.Event (SourceName, EventTimeUtc);
CREATE INDEX IX_Event_Time ON {s}.Event (EventTimeUtc);
CREATE TYPE {s}.SampleRow AS TABLE (
SourceTimeUtc datetime2(7) NOT NULL PRIMARY KEY, ValueKind tinyint NOT NULL,
@@ -196,7 +198,10 @@ next call (not latched).
`ConcurrentDictionary<string,int>` cache, creating the tag row on a miss (`MERGE` on Name; DataType from
the first value's kind, provisioning later corrects it). Insert via `{s}.SampleRow` TVP with
`INSERT … SELECT … WHERE NOT EXISTS`. Returns `true` on success, `false` on a SQL failure (the recorder's
retry/quarantine owns retry). `ServerTimeUtc = SYSUTCDATETIME()`.
retry/quarantine owns retry). `ServerTimeUtc = SYSUTCDATETIME()`. *As built:* duplicate timestamps within one batch
collapse to the **first**; a `NaN`/`±Inf` `Double` is stored as `Empty` keeping its status (it cannot live in a SQL
`float`); a 2627/2601 duplicate-key error rolls the statement back, so the statement is re-run (≤ 2 retries) rather
than treated as success.
- `IHistorianProvisioning.EnsureTagsAsync`: `MERGE` upsert of every request (all types — no numeric
filter on this backend). Per-request failures are counted, never thrown.
- `IAlarmHistorianWriter.WriteBatchAsync`: per event insert-if-absent; outcome `Ack` on insert or
@@ -211,20 +216,25 @@ next call (not latched).
tie clusters cannot occur (PK).
- **AtTime:** one statement, `{s}.TimeRow` TVP + two `OUTER APPLY`s (previous-or-equal, next). Exact ⇒ stored
value + status. Otherwise Double/Int64 linearly interpolated (status `Good` with the historian
*Interpolated* info bits), Boolean/String/DateTime stepped from the previous
sample (status keeps the previous sample's code + Interpolated bit); no previous sample ⇒ `BadNoData`.
*Interpolated* info bits **when both neighbours are Good, else `UncertainDataSubNormal` + Interpolated**; Int64 is
interpolated through `double` and rounded, ties away from zero), Boolean/String/DateTime — and any time **after the last
sample** — stepped from the previous sample (status keeps the previous sample's code + Interpolated bit); no previous
sample ⇒ `BadNoData`.
If the **previous** neighbour is Bad, the requested time lies inside an outage: the result is a null value
carrying that neighbour's Bad code (never better than its source, per the quality contract; corrected after the
T10 review). If the previous neighbour is usable and the **next** is Bad, the previous value is held (stepped,
not interpolated into the outage) with `UncertainDataSubNormal`.
- **Events:** `TOP(@maxEvents)` from `{s}.Event` filtered by `SourceName = @source` (null ⇒ all) and
`EventTimeUtc` range, ordered ascending. `HistoricalEvent(EventId, SourceName, EventTimeUtc,
`EventTimeUtc` range, ordered ascending (a reversed window is swapped and still returned ascending). `HistoricalEvent(EventId, SourceName, EventTimeUtc,
ReceivedTimeUtc, Message, (ushort)Severity)`.
- **Processed (push-down):** bucket `b = DATEDIFF_BIG(millisecond, @start, SourceTimeUtc) / @intervalMs`,
one row per bucket start `@start + b × interval` for every interval in `[start, end)` (empty intervals
emitted as `BadNoData`). Usable sample = status severity bits Good (`00`) or Uncertain (`01`). Every result
carries the historian *Calculated* info bits. An interval with any Bad
sample but ≥1 usable ⇒ `UncertainDataSubNormal`; no usable ⇒ `BadNoData`.
- **Processed (push-down):** bucket `b = elapsedTicks / @intervalTicks` in **100 ns ticks** (*corrected after T10: the original
sketch used `DATEDIFF_BIG(millisecond, …) / @intervalMs`; `DATEDIFF` counts boundaries crossed, so a ms formulation
misplaces samples near a bucket edge and a ns one overflows past ≈292 years. As built:
`DATEDIFF_BIG(microsecond, @start, t) × 10` plus a sub-microsecond tick correction*), one row per bucket start `@start + b × interval` for every interval in `[start, end)` (empty intervals
emitted as `BadNoData`). Usable sample = status severity bits Good (`00`) or Uncertain (`01`) **and a stored value** (an `Empty`
outage marker is Bad whatever its code). Every result carries the historian *Calculated* info bits. An interval with any
**non-Good (Bad or Uncertain)** sample but ≥1 usable ⇒ `UncertainDataSubNormal`; no usable ⇒ `BadNoData` (so `Count` over an
all-Bad interval is `BadNoData`, not 0). A reversed window is swapped, anchored at the earlier time, and returned descending.
| Aggregate | SQL over usable samples in the bucket | Value timestamp |
|---|---|---|
@@ -233,7 +243,7 @@ next call (not latched).
| Delta | End − Start | bucket start |
| Range | Max − Min | bucket start |
| MinimumActualTime / MaximumActualTime | min / max value, ties ⇒ earliest | the sample's time |
| PercentGood / PercentBad | `100 × count(good or bad) / count(*)` over ALL samples | bucket start |
| PercentGood / PercentBad | `100 × count(Good) / count(*)` and `100 × count(Bad) / count(*)` over ALL samples; **Uncertain samples are neither** (so the two can sum to < 100) | bucket start |
Count / Start / End / PercentGood / PercentBad accept any kind; the rest on a String/DateTime tag ⇒
`BadAggregateInvalidInputs` per interval. Any other `HistoryAggregateType` ⇒ throw
@@ -262,7 +272,11 @@ public interface IHistorianDataUpdater
| DeleteAtTime | per time: `Good`; `BadNoEntryExists` |
Each call is one transaction (`MERGE … OUTPUT $action` against the TVP). A value with a null timestamp ⇒
`BadInvalidTimestamp` for that item. Update/Insert on an unknown tag creates the tag row (same as live
`BadInvalidTimestamp` for that item. *As built:* a `NaN`/`±Inf` `Double` ⇒ `BadInvalidArgument` (rejected, not stored as `Empty`
as the live-value writer does); duplicate timestamps in one request — Insert: first applies, later `BadEntryExists`; Replace/Update:
last wins, earlier ones share its code; DeleteAtTime: first reports, later `BadNoEntryExists`. The node manager answers a null item
`BadInvalidArgument`, `MinValue` `BadInvalidTimestamp`, an untypable value `BadTypeMismatch`, a backend throw `BadInternalError`
and a deadline `BadTimeout` (a failure is not "unsupported"). Update/Insert on an unknown tag creates the tag row (same as live
writes); Replace/Delete on an unknown tag ⇒ `BadNoEntryExists` / `BadNoData`.
**Status codes.** Every status this backend produces is a named `const uint` in an internal
+13 -2
View File
@@ -416,7 +416,9 @@ write and each Part 9 method call first asks `DataPlaneAuthorizer.AuthorizeKeyUs
`AclGate: denied … key … is no longer enabled or has expired`), checked *before* the "no gateway ⇒
`BadNotWritable`" availability answer. Things to know:
- **Only these two gates.** Read, Browse, Subscribe and HistoryRead stay ungated with the flag off —
- **Only these two gates — plus HistoryUpdate.** (HistoryUpdate, added with the pluggable historian, is a third gated
operation in both modes; see [HistoryUpdate](#historyupdate-historian-writes). It re-checks the key the same way.)
Read, Browse, Subscribe and HistoryRead stay ungated with the flag off —
that is the flag-off posture, and an expired key's session can still read and keep its subscriptions.
Turn `Security:Acl:Enabled` on to have expiry reach those too.
- **Anonymous is unaffected.** It carries no key id, so there is nothing to expire; its roles stay what
@@ -484,6 +486,7 @@ the seams consult. Same dark-switch discipline as `MeshTransport`, `ConfigSource
| Alarm Confirm | role `AlarmAck` | `AlarmConfirm` |
| Alarm OneShotShelve / TimedShelve / Unshelve | role `AlarmAck` | `AlarmShelve` |
| HistoryRead (all four variants) | **nothing** | `HistoryRead` on each node's scope, decided **per node in the batch** |
| HistoryUpdate (insert / replace / update / delete) | role `HistoryUpdate` (from a `Cluster`-scoped grant) + key re-check | `HistoryUpdate` on that node's scope, decided **per node in the batch** |
| Read (every attribute, not just Value) | nothing | `Read` on that node's scope, decided per node in the batch |
| Browse / TranslateBrowsePath | nothing | `Browse` on the source node, **and** on each returned reference |
| CreateMonitoredItems / SubscribeToEvents | nothing | `Subscribe` on that node's scope, decided per item |
@@ -872,7 +875,15 @@ public enum NodePermissions : int
The three Write tiers map to Galaxy's v1 `SecurityClassification` — `FreeAccess`/`Operate` → `WriteOperate`, `Tune` → `WriteTune`, `Configure` → `WriteConfigure`. `SecuredWrite` / `VerifiedWrite` / `ViewOnly` classifications remain read-only from OPC UA regardless of grant.
`HistoryUpdate` (bit 12) is a **separate** permission from `HistoryRead` so a read-only grant cannot also authorize OPC UA HistoryUpdate (annotation / insert / delete) operations. `TriePermissionEvaluator` maps `OpcUaOperation.HistoryUpdate` to this bit, closing the read⇒update hole that existed when the only history permission was `HistoryRead`. Note: `HistoryUpdate` is **not** included in any composite bundle (`ReadOnly` / `Operator` / `Engineer` / `Admin`) because the HistoryUpdate service surface (the actual insert/replace/delete backend RPC) is not yet implemented — a client calling HistoryUpdate still receives the SDK's default reject regardless of the grant.
`HistoryUpdate` (bit 12) is a **separate** permission from `HistoryRead` so a read-only grant cannot also authorize OPC UA HistoryUpdate operations. `TriePermissionEvaluator` maps `OpcUaOperation.HistoryUpdate` to this bit. It is **not** included in any composite bundle (`ReadOnly` / `Operator` / `Engineer` / `Admin`) — it must be granted explicitly, and it is offered as its own checkbox in the AdminUI ACL editor (`AclEdit.razor`). The HistoryUpdate service itself is now implemented (SQL Server historian backend only) — see the next section.
### HistoryUpdate (historian writes)
HistoryUpdate rewrites recorded history, so unlike HistoryRead it is gated **whatever `Security:Acl:Enabled` says**, and `WriteOperate` (live writes) never implies it.
- **Flag off (default):** the session's API key is re-checked on the call (`DataPlaneAuthorizer.AuthorizeKeyUsable`, Gitea #563 — an expired or disabled key loses its activation-time roles), then the session must carry the **`HistoryUpdate` role** (`OpcUaDataPlaneRoles.HistoryUpdate`). `ApiKeyInterimRoleMapper` confers it **only from a `Cluster`-scoped grant** carrying `NodePermissions.HistoryUpdate` — narrower-scoped grants confer nothing, for the same reason as `WriteOperate` (honouring a flag without honouring its scope would widen it). A key holding only `WriteOperate` is denied `BadUserAccessDenied`.
- **Flag on:** `DataPlaneAuthorizer.Authorize(identity, OpcUaOperation.HistoryUpdate, nodeId, realm)` decides per node and **replaces** the role check (never chained) — a session with no roles succeeds on the strength of a scoped `HistoryUpdate` grant, and a `WriteOperate` grant does not help.
- **Availability is separate from authorization.** A node accepts HistoryUpdate only if it is historized **and** the selected historian backend has an updater when the node materialises (the `HistoryWrite` AccessLevel bit). `Backend=Gateway` has none, so it answers `BadHistoryOperationUnsupported` for every node whatever the grant. Operations, result codes and unsupported cases: [`docs/Historian.md` §HistoryUpdate](Historian.md#historyupdate).
### Evaluator — `PermissionTrie`