docs(historian): fix-wave docs and stale comments

- Historian.md: Empty samples always Bad (writer/updater/read clamp),
  long-name refusal, 5 s bootstrap failure cache, AtTime after now, info-bit
  masking, HistoryUpdate does NOT type-check (M-1 decision), DeleteRaw
  needs both bounds, reverse Raw paging, the historian/historization metric
  prefixes, and provisioning numeric-only lines qualified as Gateway.
- CLAUDE.md numeric-only lines qualified as Gateway backend.
- Stale 'sole historian' lines in DriverLifecycle, ServiceHosting and the
  retired Wonderware stub; Client.CLI exit-code contract.
- ServerHistorianOptions: Endpoint/Backend XML docs untangled, Validate()
  indented; Program.cs comments are backend-neutral; T10 week-test day count.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
This commit is contained in:
Joseph Doherty
2026-10-09 14:33:30 -04:00
parent 7fafeb433c
commit 60a1e7e669
9 changed files with 104 additions and 59 deletions
+50 -12
View File
@@ -278,6 +278,10 @@ holds during the FasterLog recovery scan at start-up); the lifetime count is the
**`otopcua.historian.outbox.unreadable`** (`IHistorizationOutbox.UnreadableCount`). A growing value means
undecodable records are being skipped.
> **Metric prefixes differ.** The outbox's other two gauges are `otopcua.historization.outbox.depth` and
> `otopcua.historization.outbox.dropped`; the unreadable gauge is **`otopcua.historian.outbox.unreadable`**
> (the name the design fixed). All three describe the same outbox — search for both prefixes.
⚠️ **Rollback: drain or delete the outbox before downgrading.** A node running a pre-v2 build cannot read v2
records — its outbox deserializes every record in its constructor and fails to start on the first v2 one. Before
rolling a node back, let the outbox drain to zero (`otopcua.historization.outbox.depth`) or stop the node and
@@ -287,8 +291,12 @@ Upgrading needs no action.
**Delivery semantics on `SqlServer`.** Writes are *insert-if-absent* on `(TagId, SourceTimeUtc)`: the outbox's
at-least-once redelivery (and both pair nodes writing the same sample in a boot window) converges on one row, and the
first-stored sample wins. **Duplicate timestamps inside one batch collapse to the first.** A `NaN` / `±Infinity` `Double`
cannot be stored in a SQL `float`, so the **value writer stores it as an `Empty` sample keeping its status** — the tag
is not quarantined for one bad reading (HistoryUpdate behaves differently: see [below](#historyupdate)). A duplicate-key
cannot be stored in a SQL `float`, so the **value writer stores it as an `Empty` sample** — the tag is not quarantined for
one bad reading (HistoryUpdate behaves differently: see [below](#historyupdate)). ⚠️ **An `Empty` sample is always Bad**
(the quality contract: a stored sample is never served better than "no value"): a non-finite `Double` that arrived with a
Good or Uncertain status is stored as `Empty` + **`BadOutOfRange`** (`0x803C0000`), and any other `Empty` with a non-Bad
status as `Empty` + **`BadNoData`** (`0x809B0000`). An existing Bad code — the comms-loss `BadNoCommunication` marker, a
driver's own Bad publish — is stored as it is. A duplicate-key
error (2627/2601) rolls the whole statement back, so the writer re-runs the statement (at most twice) instead of
treating the error as success.
@@ -362,21 +370,22 @@ values are counted in `RecorderStatus.SuppressedNotPrimary`, and each gate trans
## Tag auto-provisioning (EnsureTags)
When a deploy adds a historized tag, the server auto-ensures it exists in the historian via the gateway's
`EnsureTags` **before** the first value or HistoryRead lands. `AddressSpaceApplier.Apply()` fires a
`EnsureTags` (Gateway) or the `Tag`-row `MERGE` (SqlServer) **before** the first value or HistoryRead lands. `AddressSpaceApplier.Apply()` fires a
**non-blocking, fire-and-forget** `IHistorianProvisioning.EnsureTagsAsync` for each added historized value
tag (with the resolved historian name — the `historianTagname` override when set, else the driver
`FullName`); the hook is wrapped so a faulted, slow, or throwing provisioner can **never** block or fail a
deploy. Non-numeric (`String`/`DateTime`/`Reference`) types are skipped, not provisioned.
deploy. On the **Gateway backend**, non-numeric (`String`/`DateTime`/`Reference`) types are skipped, not provisioned;
the SqlServer backend provisions every type (below).
**Wiring + gating.** The concrete provisioner comes from the selected backend's provider (`CreateProvisioning`):
the gateway-backed `GatewayTagProvisioner`, or — for `SqlServer` — `SqlServerTagProvisioner` (one `MERGE … WITH (HOLDLOCK)`
upsert per chunk of 400 tags; every data type is provisionable so `skipped` is always 0; a blank or over-400-character
name fails alone; a null unit/description in a request keeps the stored value). It is registered by the Host's
`AddHistorianProvisioning` and gated on the **same**
`ServerHistorian:Enabled` flag as the read path — both target the same single gateway. When the section is
`ServerHistorian:Enabled` flag as the read path — both use the one resolved backend. When the section is
disabled (or absent) the applier resolves the no-op `NullHistorianProvisioning` and **provisioning does not
run** (so on a read-only-historian deployment, ensure tags exist by some other means). The Host requires the
API key to carry the `historian:tags:write` scope for `EnsureTags` to succeed.
run** (so on a read-only-historian deployment, ensure tags exist by some other means). On the Gateway backend the
API key must carry the `historian:tags:write` scope for `EnsureTags` to succeed.
**Observability.** Every dispatch logs an Information tally —
`historian provisioning completed (dispatched=N, requested=…, ensured=…, skipped=…, failed=…)`. A line
@@ -530,6 +539,11 @@ paging time-based:
> `NumValuesPerNode` samples at a time across successive pages, advancing the cursor one tick past
> the timestamp once the cluster is fully drained.
>
> **Reverse reads** (`start > end`, newest first) page the same way in the opposite direction: a drained
> cluster steps the cursor one tick **down**, the read ends when that would reach the (excluded) `end`, and
> the over-fetched cluster is sliced in descending order. Before the fix a reverse read with a small
> `NumValuesPerNode` stopped after its first page with `GoodNoData`.
>
> **Short pages within a cluster still carry a continuation point.** A within-cluster page that
> returns fewer than `NumValuesPerNode` samples (because the cluster happened to be smaller than
> the cap, or is the final partial batch) is not the last page if the cluster itself has not been
@@ -643,12 +657,22 @@ everywhere (full 100 ns precision — never a `DataTable`, which would round to
transient SQL errors (connection, timeout, deadlock 1205 …), bootstrap failures, cancellation and any unclassified error;
`PermanentFail` (dead-letter) only for data errors SQL will reject forever (truncation 2628/8152, conversion, null, range, constraint).
`SourceName` is the event's `EquipmentPath`; `Severity` is the numeric Part 9 value (Low 200, Medium 500, High 700, Critical 900).
- **Tag names** are `nvarchar(400)`. `EnsureTags` fails a longer name individually, but the live-value writer's tag-create
path does **not** reject one (the parameter silently truncates) — keep historian tagnames under 400 characters.
- **Tag names** are `nvarchar(400)`, and a longer name is **refused, never truncated** (a truncating parameter would resolve
it onto whichever tag owns its first 400 characters): `EnsureTags` fails it individually, the live-value writer returns
`false` for that tag (a per-tag failure, so the #538 quarantine eventually abandons it — logged as a Warning), the updater
answers `BadInvalidArgument` for every item, and reads treat it as not found (Raw empty, AtTime / Processed `BadNoData`). An
event read filtered by a source name over 400 characters returns no events. Keep historian tagnames under 400 characters.
- **Schema bootstrap failures are cached for 5 s.** With `AutoCreateSchema=false` and the schema missing (or SQL down), every
historian call fails fast with the last bootstrap error for 5 s before SQL is tried again, so a missing schema does not
serialise every write, read and alarm event behind one round-trip each; each retry that fails logs a Warning. If
`sp_releaseapplock` fails after an auto-create, the connection pool is cleared so the session holding the applock is closed
rather than returned to the pool.
### Raw reads
`ReadRawAsync` returns every sample as stored — outage markers (`Empty` + Bad status) included.
`ReadRawAsync` returns every sample as stored — outage markers (`Empty` + Bad status) included. The one exception is defensive:
a row with no value whose stored status is **not** Bad (only a build from before the writer forced `Empty` samples Bad could
have written one) is served with `BadNoData`, on Raw and AtTime alike.
| Request | Window | Order |
|---|---|---|
@@ -674,6 +698,10 @@ outage marker and counts as Bad whatever its code). Per requested time `T` (the
| `p` usable, **`n` Bad** | **`p`'s value held** (not interpolated into the outage), `UncertainDataSubNormal` + Interpolated |
| `Double` / `Int64` with a usable `n` | linear interpolation of `NumValue`; `Int64` is computed **through `double`** and rounded to the nearest integer (ties away from zero), keeping the `Int64` type. Status `Good` + Interpolated when both neighbours are Good, else `UncertainDataSubNormal` + Interpolated |
| `Boolean` / `String` / `DateTime`, **or `T` after the last sample** | **stepped**: `p`'s value with `p`'s status + Interpolated |
| `T` after the last sample **and after now** | `p`'s value held, `UncertainDataSubNormal` + Interpolated (the value is not known to still hold) |
Setting Interpolated (or, in aggregates, Calculated) first clears any historian bit the stored status already carries, so a
result never has both.
⚠️ Because `Int64` interpolation goes through `double`, magnitudes beyond 2^53 lose precision.
@@ -791,8 +819,18 @@ mixed (Part 11 reports item outcomes in `OperationResults`); it is Bad only for
**Per-item rejections** (answered without touching SQL; other items still proceed): a null item ⇒ `BadInvalidArgument`;
`SourceTimestamp == DateTime.MinValue` ⇒ `BadInvalidTimestamp`; a value the historian cannot type (array, unsupported CLR type) ⇒
`BadTypeMismatch`; and a **`NaN` / `±Infinity` `Double` ⇒ `BadInvalidArgument`** — it cannot live in a SQL `float`, and unlike the live-value
writer (which stores it as `Empty`) an operator's explicit update is rejected rather than silently rewritten. A caller-supplied status code is stored as given.
`BadTypeMismatch`; a **`NaN` / `±Infinity` `Double` ⇒ `BadInvalidArgument`** — it cannot live in a SQL `float`, and unlike the live-value
writer (which stores it as `Empty`) an operator's explicit update is rejected rather than silently rewritten; and a **null value with a
Good or Uncertain status ⇒ `BadInvalidArgument`** — an `Empty` sample is always Bad, so a null value is accepted only with a Bad status
(an operator-authored outage marker). A caller-supplied status code is otherwise stored as given.
**No DataType check.** HistoryUpdate does **not** compare a value's type with the node's `DataType`: a value is stored as sent (a `String`
written to a `Double` node's history is stored and later read back as a `String`), exactly as the live recorder stores whatever type the
driver publishes. Only a value the historian cannot represent at all is refused (`BadTypeMismatch`, above). Part 11 §6.8 permits a
server to answer `BadTypeMismatch` for a value of the wrong type; this server deliberately does not.
**`DeleteRawModifiedDetails` needs both bounds:** a `StartTime` or `EndTime` of `DateTime.MinValue` (the wire's "unspecified") is
`BadInvalidArgument` for that node and never reaches the backend.
**Duplicate timestamps inside one request** (compared at 100 ns): *Insert* — the first occurrence is applied, later ones are `BadEntryExists`;
*Replace* / *Update* — the last one wins and every earlier occurrence reports the winner's code; *DeleteAtTime* — the first reports the real