lmxopcua/docs/plans/2026-10-09-pluggable-historian-design.md

# Pluggable historian backends — design

**Date:** 2026-10-09 · **Status:** approved (brainstorm 2026-10-09) · **Plan:** `2026-10-09-pluggable-historian-plan.md`

## 1. Goal and scope

Today the only historian backend is AVEVA/Wonderware, reached through the HistorianGateway sidecar. This
work makes the backend **pluggable**, ships **SQL Server** as the first new backend, and adds the OPC UA
**HistoryUpdate** service so clients can insert / replace / update / delete raw history.

| In scope (this spec) | Later (own spec) | Dropped |
|---|---|---|
| Provider seam + `ServerHistorian:Backend` selection | QuestDB backend (a new provider only) | SQLite (a per-node file needs pair replication) |
| Typed values (not just `double`) end to end through recorder + outbox | | |
| Real status codes recorded (fixes a hard-coded Good) | | |
| SQL Server backend: schema, writes, reads, aggregates push-down, alarm events, provisioning, updates | | |
| HistoryUpdate service + `HistoryUpdate` role / permission + AdminUI checkbox | | |
| Client.CLI `historyupdate` command | | |

Out of scope everywhere: retention/purge, partitioning/compression, Part 11 "modified values"
(`IsReadModified`, `IsDeleteModified`), event HistoryUpdate (`UpdateEventDetails`/`DeleteEventDetails`),
`UpdateStructureData`, `AuditHistoryUpdateEvent` emission, array historization.

Hard constraints: no new NuGet packages and no version changes (`Microsoft.Data.SqlClient` is already
centrally pinned — referencing it from a new project is allowed, adding a new package is not).
Recording stays Primary-only (#537); delivery stays at-least-once with the #538 per-tag quarantine.

## 2. Architecture — the provider seam (approach A)

The five historian seams the rest of the code already consumes stay; a provider builds them.

```csharp
// src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/IHistorianBackendProvider.cs
public interface IHistorianBackendProvider
{
    string Name { get; }                                         // "Gateway" | "SqlServer" | (later) "QuestDb"
    IReadOnlySet<HistorianValueKind> SupportedKinds { get; }     // recorder capture filter
    IHistorianDataSource   CreateDataSource(IServiceProvider sp);
    IHistorianValueWriter  CreateValueWriter(IServiceProvider sp);
    IAlarmHistorianWriter  CreateAlarmWriter(IServiceProvider sp);
    IHistorianProvisioning CreateProvisioning(IServiceProvider sp);
    IHistorianDataUpdater? CreateUpdater(IServiceProvider sp);  // null ⇒ HistoryUpdate unsupported
}
```

It lives in **Runtime** because the seams it returns span `Core.Abstractions` and `Core.AlarmHistorian`.

- `GatewayHistorianProvider` (Driver.Historian.Gateway) wraps today's `GatewayHistorian.Create…` factories;
  `CreateUpdater` returns `null`; `SupportedKinds = {Double, Int64, Boolean}`.
- `SqlServerHistorianProvider` (new project `Driver.Historian.SqlServer`); `SupportedKinds` = all kinds.
- The Host owns a name→provider table (`HistorianBackendRegistry`). `Program.cs`'s four hard-wired
  `GatewayHistorian.Create…` call sites (three, plus the inline `new GatewayHistorianValueWriter(...)`) become calls
  on the selected provider. The existing gates (`AddServerHistorian`, `AddHistorianProvisioning`,
  `AddAlarmHistorian`, the outbox registration, the recorder spawn) are unchanged — they get their objects
  from the provider.
- The outbox (`FasterLogHistorizationOutbox`) stays in the Gateway project (moving it would add a
  FASTER package reference elsewhere); the Host registers it regardless of backend. The SqlServer project
  does **not** reference the Gateway project.
- Rejected: one fat `IHistorian` (forces stubs, churns every narrow consumer); per-capability backend
  selection (YAGNI, doubles config + validation).

### Configuration

```jsonc
"ServerHistorian": {
  "Enabled": true,
  "Backend": "SqlServer",              // "Gateway" (default when absent) | "SqlServer"
  "HistoryReadBatchParallelism": 4, "MaxConcurrentHistoryReadBatches": 16, "HistoryReadDeadline": "00:01:00",
  "Endpoint": "...", "ApiKey": "...",  // gateway keys stay top-level, unchanged (no migration)
  "SqlServer": {
    "ConnectionString": "${secret:OT-HISTORIAN-SQL}",
    "Schema": "hist",
    "CommandTimeout": "00:00:30",
    "AutoCreateSchema": true
  }
}
```

- Unknown `Backend` ⇒ host start fails (`OptionsValidationException` listing the valid names).
- `ServerHistorianOptionsValidator` runs its three gateway checks (Endpoint, ApiKey, UseTls⇔scheme) only when `Backend == Gateway`; backend-specific options are validated there too, consumer-gated (`ServerHistorian:Enabled || AlarmHistorian:Enabled`), never at registry resolution.
- New `SqlServerHistorianOptions` validation: `ConnectionString` required, `Schema` a valid SQL identifier
  (`^[A-Za-z_][A-Za-z0-9_]{0,127}$` — it is spliced into DDL/DML), `CommandTimeout > 0`.
- Validators remain driver-block-only (admin-only nodes never build historian singletons). *As built:* an invalid `SqlServer:Schema`
  also throws `ArgumentException` when the provider is constructed, so `Backend=SqlServer` + a malformed schema stops host start even
  when no consumer is enabled.
- The `Program.cs` alarm-only warning names the selected backend instead of "gateway".
- The historian connection string is **not** a ConfigDb string; driver-only nodes may carry it (Phase 4 intact).

## 3. Typed values

```csharp
// Core.Abstractions/Historian/HistorianValue.cs
public enum HistorianValueKind : byte { Empty = 0, Double = 1, Int64 = 2, Boolean = 3, String = 4, DateTime = 5 }

public readonly record struct HistorianValue(HistorianValueKind Kind, double Number, long Integer, string? Text)
{
    public static HistorianValue Empty { get; }                 // no value (a Bad/Uncertain sample with a null)
    public static HistorianValue FromDouble(double v);
    public static HistorianValue FromInt64(long v);
    public static HistorianValue FromBoolean(bool v);           // Integer = 1/0, Number = 1/0
    public static HistorianValue FromString(string v);
    public static HistorianValue FromDateTime(DateTime utc);    // Integer = Ticks (UTC)
    public static bool TryFromObject(object? value, out HistorianValue result); // the recorder's coercion
    public bool TryGetDouble(out double v);                     // Double/Int64/Boolean ⇒ true
    public object? ToObject();                                  // back to a CLR value for DataValueSnapshot
}
```

`TryFromObject` mapping: `double/float/decimal` ⇒ Double; `sbyte/byte/short/ushort/int/uint/long` ⇒ Int64;
`ulong` ⇒ Int64 when `≤ long.MaxValue` else Double; `bool` ⇒ Boolean; `string` ⇒ String; `DateTime` ⇒
DateTime (converted to UTC); `null` ⇒ Empty; arrays / anything else ⇒ `false` (dropped + metered).

Changed records:

```csharp
public readonly record struct HistorizationValue(DateTime? TimestampUtc, HistorianValue Value, uint StatusCode);
public sealed record HistorizationOutboxEntry(Guid Id, string Tag, HistorianValue Value, uint StatusCode, DateTime TimestampUtc);
```

**Recorder.** Captures `HistorianValue` via `TryFromObject`; records `msg.StatusCode` (the mux already
carries it — the recorder hard-codes `GoodQuality = 192` today, so Bad samples were historized as Good).
A value whose kind is not in the provider's `SupportedKinds` is dropped at capture and counted under
`DroppedUnsupportedKind` (renamed from `DroppedNonNumeric`, same meter semantics for the gateway). `Empty`
is in SQL Server's set (a Bad sample with no value is history worth keeping) and not the gateway's.

**Gateway backend: no wire change in practice.** Samples now carry their real status, mapped to DA quality
inside `GatewayHistorianValueWriter` by a new reverse mapper `GatewayQualityMapper.ToDaQuality(uint)` — the
exact inverse of the existing `Map(byte)` table, falling back by severity (Good ⇒ 192, Uncertain ⇒ 64,
Bad ⇒ 0). The recorder no longer knows about DA quality. But the gateway's `WriteLiveValues` SQL path
**discards** `Quality` (Contracts C-002, server-managed), so AVEVA keeps storing every recorded value as Good —
a pre-existing limitation of that backend, now documented rather than fixed. The SQL Server backend stores
the real status.

**Outbox format v2.** Entries are written `magic "HV2\0"` + `version (1 byte)` + id(16) + tagLen(4) + tag +
kind(1) + kind payload + status(4) + timestamp(8). Payload by kind: Empty none; Double 8; Int64/Boolean/DateTime
8; String len(4)+UTF-8. Legacy entries (no magic, `length == 38 + tagLen@16`) are read as Double with
status `GatewayQualityMapper.Map(quality)` (DA ⇒ StatusCode; legacy entries are always 192 ⇒ Good). An entry
matching neither layout is dropped and counted (`otopcua.historian.outbox.unreadable`), never thrown — a
corrupt entry must not jam the drain. A pinned legacy-bytes test guards the upgrade path.

## 4. SQL Server backend

### Schema (`{Schema}` default `hist`, database e.g. `OtOpcUaHistorian`)

```sql
CREATE TABLE {s}.SchemaVersion (Version int NOT NULL PRIMARY KEY, AppliedUtc datetime2(7) NOT NULL);

CREATE TABLE {s}.Tag (
  TagId int IDENTITY(1,1) NOT NULL PRIMARY KEY,
  Name nvarchar(400) NOT NULL CONSTRAINT UQ_Tag_Name UNIQUE,
  DataType tinyint NOT NULL,                 -- DriverDataType
  EngineeringUnit nvarchar(64) NULL, Description nvarchar(400) NULL,
  CreatedUtc datetime2(7) NOT NULL, UpdatedUtc datetime2(7) NOT NULL);

CREATE TABLE {s}.Sample (
  TagId int NOT NULL, SourceTimeUtc datetime2(7) NOT NULL,
  ValueKind tinyint NOT NULL,                -- HistorianValueKind
  NumValue float NULL, IntValue bigint NULL, StrValue nvarchar(max) NULL,
  StatusCode int NOT NULL,                   -- uint StatusCode stored bit-for-bit (unchecked cast)
  ServerTimeUtc datetime2(7) NOT NULL,
  CONSTRAINT PK_Sample PRIMARY KEY CLUSTERED (TagId, SourceTimeUtc),
  CONSTRAINT FK_Sample_Tag FOREIGN KEY (TagId) REFERENCES {s}.Tag(TagId));

CREATE TABLE {s}.Event (
  EventId nvarchar(64) NOT NULL CONSTRAINT PK_Event PRIMARY KEY NONCLUSTERED, -- deterministic SHA-256 hex of the event payload
  SourceName nvarchar(400) NOT NULL,                  -- AlarmHistorianEvent.EquipmentPath
  AlarmId nvarchar(400) NOT NULL, AlarmName nvarchar(400) NOT NULL, AlarmTypeName nvarchar(100) NOT NULL,
  Severity int NOT NULL, EventKind nvarchar(50) NOT NULL, Message nvarchar(max) NULL,
  UserName nvarchar(200) NULL, Comment nvarchar(max) NULL,
  EventTimeUtc datetime2(7) NOT NULL, ReceivedTimeUtc datetime2(7) NOT NULL);
CREATE CLUSTERED INDEX CX_Event_Time ON {s}.Event (EventTimeUtc, EventId);   -- as built (§9.6); the sketched IX_Event_Time is replaced by this
CREATE INDEX IX_Event_Source_Time ON {s}.Event (SourceName, EventTimeUtc);

CREATE TYPE {s}.SampleRow AS TABLE (
  SourceTimeUtc datetime2(7) NOT NULL PRIMARY KEY, ValueKind tinyint NOT NULL,
  NumValue float NULL, IntValue bigint NULL, StrValue nvarchar(max) NULL, StatusCode int NOT NULL);
CREATE TYPE {s}.TimeRow AS TABLE (T datetime2(7) NOT NULL PRIMARY KEY);
```

Column mapping per kind: Double ⇒ NumValue; Int64 ⇒ IntValue + NumValue (double, for aggregates);
Boolean ⇒ IntValue 0/1 + NumValue 0/1; String ⇒ StrValue; DateTime ⇒ IntValue (ticks); Empty ⇒ all NULL.

The PK enforces Part 11's one-value-per-timestamp and makes outbox redelivery idempotent: live writes are
**insert-if-absent**. Events are insert-if-absent on `EventId` (`AlarmHistorianEvent` carries no id, so
the writer computes `SHA-256(AlarmId|EventKind|TimestampUtc.Ticks|User|Comment|Message)`, hex).

**Bootstrap.** Numbered idempotent scripts (`001_initial.sql`) are embedded resources (and mirrored to
`docs/historian/sqlserver/`). `SqlServerSchemaBootstrapper` runs once per process on first use, lazily,
under `sp_getapplock @Resource='OtOpcUaHistorianSchema', @LockMode='Exclusive', @LockOwner='Session'`.
`AutoCreateSchema=true` creates the schema + applies missing scripts; `false` only checks
`SchemaVersion` and, if behind, every historian call fails with a clear error naming the missing version
(surfaced in `GetHealthSnapshot().LastError`, never a host crash). Bootstrap failure is retried on the
next call (not latched).

### Writes

- `IHistorianValueWriter.WriteLiveValuesAsync(tag, values)`: resolve `TagId` via a
  `ConcurrentDictionary<string,int>` cache, creating the tag row on a miss (`MERGE` on Name; DataType from
  the first value's kind, provisioning later corrects it). Insert via `{s}.SampleRow` TVP with
  `INSERT … SELECT … WHERE NOT EXISTS`. Returns `true` on success, `false` on a SQL failure (the recorder's
  retry/quarantine owns retry). `ServerTimeUtc = SYSUTCDATETIME()`. *As built:* duplicate timestamps within one batch
  collapse to the **first**; a `NaN`/`±Inf` `Double` is stored as `Empty` (it cannot live in a SQL `float`) and, per the
  final-review I-1 ruling, **an `Empty` sample is always Bad**: a non-Bad non-finite value is stored `BadOutOfRange`, any
  other non-Bad `Empty` `BadNoData`, an existing Bad code kept; reads clamp a legacy non-Bad `Empty` row to `BadNoData`;
  tag names over 400 characters are refused, never truncated; a 2627/2601 duplicate-key error rolls the statement back, so the statement is re-run (≤ 2 retries) rather
  than treated as success.
- `IHistorianProvisioning.EnsureTagsAsync`: `MERGE` upsert of every request (all types — no numeric
  filter on this backend). Per-request failures are counted, never thrown.
- `IAlarmHistorianWriter.WriteBatchAsync`: per event insert-if-absent; outcome `Ack` on insert or
  already-present, `RetryPlease` on transient SQL errors (connection/timeout/deadlock 1205), `PermanentFail`
  on data errors (e.g. truncation 2628/8152).

### Reads

- **Raw:** `SELECT TOP(@max) … WHERE TagId=@id AND SourceTimeUtc >= @lo AND SourceTimeUtc < @hi ORDER BY
  SourceTimeUtc ASC` (forward) or `> @lo AND <= @hi … DESC` when `endUtc < startUtc` (Part 11 reverse).
  Unknown tag ⇒ empty result (node manager reports `GoodNoData`). The node manager's paging is unchanged;
  tie clusters cannot occur (PK).
- **AtTime:** one statement, `{s}.TimeRow` TVP + two `OUTER APPLY`s (previous-or-equal, next). Exact ⇒ stored
  value + status. Otherwise Double/Int64 linearly interpolated (status `Good` with the historian
  *Interpolated* info bits **when both neighbours are Good, else `UncertainDataSubNormal` + Interpolated**; Int64 is
  interpolated through `double` and rounded, ties away from zero), Boolean/String/DateTime — and any time **after the last
  sample** — stepped from the previous sample (status keeps the previous sample's code + Interpolated bit); no previous
  sample ⇒ `BadNoData`.
  If the **previous** neighbour is Bad, the requested time lies inside an outage: the result is a null value
  carrying that neighbour's Bad code (never better than its source, per the quality contract; corrected after the
  T10 review). If the previous neighbour is usable and the **next** is Bad, the previous value is held (stepped,
  not interpolated into the outage) with `UncertainDataSubNormal`.
- **Events:** `TOP(@maxEvents)` from `{s}.Event` filtered by `SourceName = @source` (null ⇒ all) and
  `EventTimeUtc` range, ordered ascending (a reversed window is swapped and still returned ascending). `HistoricalEvent(EventId, SourceName, EventTimeUtc,
  ReceivedTimeUtc, Message, (ushort)Severity)`.
- **Processed (push-down):** bucket `b = elapsedTicks / @intervalTicks` in **100 ns ticks** (*corrected after T10: the original
  sketch used `DATEDIFF_BIG(millisecond, …) / @intervalMs`; `DATEDIFF` counts boundaries crossed, so a ms formulation
  misplaces samples near a bucket edge and a ns one overflows past ≈292 years. As built:
  `DATEDIFF_BIG(microsecond, @start, t) × 10` plus a sub-microsecond tick correction*), one row per bucket start `@start + b × interval` for every interval in `[start, end)` (empty intervals
  emitted as `BadNoData`). Usable sample = status severity bits Good (`00`) or Uncertain (`01`) **and a stored value** (an `Empty`
  outage marker is Bad whatever its code). Every result carries the historian *Calculated* info bits. An interval with any
  **non-Good (Bad or Uncertain)** sample but ≥1 usable ⇒ `UncertainDataSubNormal`; no usable ⇒ `BadNoData` (so `Count` over an
  all-Bad interval is `BadNoData`, not 0). A reversed window is swapped, anchored at the earlier time, and returned descending.

  | Aggregate | SQL over usable samples in the bucket | Value timestamp |
  |---|---|---|
  | Average / Minimum / Maximum / Total / Count | `AVG / MIN / MAX / SUM (NumValue)`, `COUNT(*)` | bucket start |
  | Start / End | first / last by `SourceTimeUtc` (`ROW_NUMBER`) | bucket start |
  | Delta | End − Start | bucket start |
  | Range | Max − Min | bucket start |
  | MinimumActualTime / MaximumActualTime | min / max value, ties ⇒ earliest | the sample's time |
  | PercentGood / PercentBad | `100 × count(Good) / count(*)` and `100 × count(Bad) / count(*)` over ALL samples; **Uncertain samples are neither** (so the two can sum to < 100) | bucket start |

  Count / Start / End / PercentGood / PercentBad accept any kind; the rest on a String/DateTime tag ⇒
  `BadAggregateInvalidInputs` per interval. Any other `HistoryAggregateType` ⇒ throw
  `HistoryAggregateNotSupportedException` (⇒ `BadAggregateNotSupported`). These are the simple per-sample
  definitions, not Part 13's full interval-boundary semantics — documented as such.

### Updates (`IHistorianDataUpdater`)

```csharp
public enum HistoryUpdateMode { Insert, Replace, Update }      // Part 11 PerformUpdateType minus Remove
public interface IHistorianDataUpdater
{
    Task<IReadOnlyList<uint>> UpdateDataAsync(string tag, HistoryUpdateMode mode,
        IReadOnlyList<HistorizationValue> values, CancellationToken ct);       // one code per value, in order
    Task<uint> DeleteRawAsync(string tag, DateTime startUtc, DateTime endUtc, CancellationToken ct);
    Task<IReadOnlyList<uint>> DeleteAtTimeAsync(string tag, IReadOnlyList<DateTime> timesUtc, CancellationToken ct);
}
```

| Operation | Per-item result |
|---|---|
| Insert | `GoodEntryInserted`; `BadEntryExists` if a sample exists at that timestamp |
| Replace | `GoodEntryReplaced`; `BadNoEntryExists` if none |
| Update | `GoodEntryInserted` or `GoodEntryReplaced` |
| DeleteRaw | `Good`; `BadNoData` if the range held nothing |
| DeleteAtTime | per time: `Good`; `BadNoEntryExists` |

Each call is one transaction (`MERGE … OUTPUT $action` against the TVP). A value with a null timestamp ⇒
`BadInvalidTimestamp` for that item. *As built:* a `NaN`/`±Inf` `Double` ⇒ `BadInvalidArgument` (rejected, not stored as `Empty`
as the live-value writer does), and so is a null value with a non-Bad status (I-1); no DataType check against the node (M-1,
decided: values are stored as sent); a `DeleteRawModified` with a `MinValue` bound ⇒ `BadInvalidArgument`; duplicate timestamps in one request — Insert: first applies, later `BadEntryExists`; Replace/Update:
last wins, earlier ones share its code; DeleteAtTime: first reports, later `BadNoEntryExists`. The node manager answers a null item
`BadInvalidArgument`, `MinValue` `BadInvalidTimestamp`, an untypable value `BadTypeMismatch`, a backend throw `BadInternalError`
and a deadline `BadTimeout` (a failure is not "unsupported"). Update/Insert on an unknown tag creates the tag row (same as live
writes); Replace/Delete on an unknown tag ⇒ `BadNoEntryExists` / `BadNoData`.

**Status codes.** Every status this backend produces is a named `const uint` in an internal
`SqlHistorianStatusCodes` class (the house rule: `StatusCodeParityTests` guards named constants by reflection
against `Opc.Ua.StatusCodes`, and an inline literal is invisible to it) — register the class with that parity
test. Historian info bits follow Part 4 §7.39: `InfoType = DataValue (0x00000400)` OR-ed with
`HistorianCalculated (0x1)` / `HistorianInterpolated (0x2)`; the constants are checked against the SDK's
`AggregateBits`/`StatusCode` helpers in a unit test rather than trusted from this doc.

## 5. HistoryUpdate in the server

- `OtOpcUaNodeManager` overrides the SDK's protected `HistoryUpdateData`, `HistoryDeleteRawModified`,
  `HistoryDeleteAtTime` (verify exact signatures against the pinned SDK before writing; the read overrides
  at `OtOpcUaNodeManager.cs` ~4928 are the pattern). The node → historian tagname lookup is the existing
  `TryGetHistorizedTagname`, so raw and UNS NodeIds hit the same rows. Not historized ⇒
  `BadHistoryOperationUnsupported`.
- New settable `IHistorianDataUpdater? HistorianDataUpdater` on the node manager, assigned by
  `OtOpcUaServerHostedService` next to `HistorianDataSource` (resolved as optional — `null` when the
  backend has none). Null ⇒ every HistoryUpdate item `BadHistoryOperationUnsupported`.
- AccessLevel: `ComposeAccessLevel(writable, historized)` gains `historyWritable` — the `HistoryWrite` bit
  is OR-ed in when historized **and** an updater is wired (both raw node and UNS mirrors).
- Out-of-scope details (events, structure data, `IsDeleteModified`) ⇒ `BadHistoryOperationUnsupported`.
- Updates go straight to the backend (not the outbox), are not Primary-gated (the database is shared),
  and share the HistoryRead batch limiter + deadline.
- **Authorization.**
  - ACL flag **on**: `DataPlaneAuthorizer.Authorize(identity, OpcUaOperation.HistoryUpdate, nodeId, realm)`
    (`TriePermissionEvaluator` already maps it to `NodePermissions.HistoryUpdate`).
  - ACL flag **off**: new role `OpcUaDataPlaneRoles.HistoryUpdate = "HistoryUpdate"`, conferred by
    `ApiKeyInterimRoleMapper` only from a **Cluster-scoped** grant carrying `NodePermissions.HistoryUpdate`,
    plus `AuthorizeKeyUsable` (#563 key re-check). Deny ⇒ `BadUserAccessDenied`.
  - **Load-bearing test:** an ALLOW — flag on, session with no roles, scoped `HistoryUpdate` grant ⇒ the
    insert lands. And flag off: a `WriteOperate`-only key is DENIED (the role is not reused).
  - `NodePermissions.HistoryUpdate` is added to the AdminUI ACL editor's permission list (`AclEdit.razor`);
    it stays out of every composite bundle.

## 6. Client.CLI

`historyupdate` command: `-u <url> -n <nodeId> --mode insert|replace|update|delete-raw|delete-at-time`,
`--value <v> --time <iso>` (repeatable for insert/replace/update/delete-at-time), `--start/--end` for
delete-raw. Prints one line per item with its status code name. Docs in `docs/Client.CLI.md`.

## 7. Testing

- **Unit:** `HistorianValue` coercion table; outbox v2 round-trip per kind + pinned legacy bytes +
  unreadable-entry drop; recorder typed capture, real status passthrough, `SupportedKinds` filter (incl.
  `Empty`), unchanged #537/#538 behaviour; gateway writer conversion (Good bytes identical to today);
  provider registry + validators; role mapper; node manager HistoryUpdate (fake updater): allow/deny matrix
  for flag on/off, result-code passthrough, unsupported cases, AccessLevel bit.
- **SQL Server integration** (`Driver.Historian.SqlServer.IntegrationTests`): env-gated on
  `HISTORIAN_SQL_TEST_CONNECTION` (a master-capable connection string; absent ⇒ `Assert.Skip`), each run in
  a throwaway database `OtOpcUaHistorian_T_<guid>` dropped on dispose. Default target: the docker-dev
  `sql` container, `Server=localhost,14330;User Id=sa;Password=OtOpcUa!Dev123;TrustServerCertificate=True`.
  Covers bootstrap (idempotent; two concurrent bootstrappers), every kind round-trip, redelivery idempotency,
  raw forward/reverse/limit, AtTime exact/interpolated/stepped/no-prior, each aggregate against hand-computed
  values incl. Bad-sample and empty-interval cases, events, every update/delete result code, unknown tags.
- **Over-the-wire:** one `OpcUaServer.IntegrationTests` test drives HistoryUpdate Insert then HistoryRead
  through a real SDK client against a fake updater/data source.
- **Live gate (docker-dev):** central pair on `Backend=SqlServer` against database `OtOpcUaHistorian` on
  the rig `sql` container; a historized rig tag records rows (Primary only — row count advances once per
  value); Client.CLI `historyread` raw + processed; `historyupdate` insert/replace/delete round-trip;
  stop the Primary ⇒ the Secondary takes over recording into the same table with no duplicate rows; a
  Bad sample is stored with its Bad status. Evidence recorded in `docs/plans/2026-10-09-pluggable-historian-live-gate.md`.

## 8. Docs and index

`docs/Historian.md` (backend selection, SQL schema, aggregate semantics, HistoryUpdate), `docs/Configuration.md`
(`ServerHistorian:Backend` + `SqlServer` subsection), `docs/security.md` (HistoryUpdate role/permission),
`docs/Client.CLI.md`, `CLAUDE.md` Historian section, and the scadaproj `sister-project-history/otopcua.md`
entry (ScadaBridge impact: none unless it issues HistoryUpdate; HistoryRead semantics unchanged on the
gateway backend apart from honest Bad quality on Bad samples).

## 9. Pre-execution review amendments (Fable, 2026-10-09)

These were verified against the code and override anything above that conflicts.

1. **Outbox recovery.** `FasterLogHistorizationOutbox`'s constructor (`RecoverState`) deserializes every record too, so the "unreadable ⇒ drop + count, never throw" rule applies there as well. An unreadable record is indexed under a synthetic id so FIFO truncation can pass it. If the v2 magic matches but structural validation fails, the legacy exact-length test runs before the entry is declared unreadable. `IHistorizationOutbox` gains `UnreadableCount` (default 0).
2. **Validation** is as in §2 (all three gateway checks gated on `Backend`; consumer-gated backend validation). `HistorianUpdaterHolder.None` is always registered on driver nodes; the real updater only when `ServerHistorian:Enabled`.
3. **Timestamps** are bound as `datetime2(7)` everywhere: `SqlDbType.DateTime2` scalar parameters and `SqlDataRecord` TVPs, never `DataTable`, which would round to 3.33 ms `datetime`.
4. **Raw** `maxValuesPerNode == 0` means unbounded. **Events** fetch `maxEvents + 1` and return a non-null continuation point when truncated, as the interface requires.
5. **Concurrency.** Tag creation and sample `MERGE`s use `WITH (HOLDLOCK)`, and `2627/2601` is treated as "already present". Both pair nodes provision on every deploy.
6. **`Event`** has a nonclustered PK on `EventId` and is clustered on `(EventTimeUtc, EventId)`. The EventId hashes length-prefixed fields and includes `EquipmentPath`.
7. **Processed reads** are capped at 100 000 buckets per request.
8. **HistoryUpdate admission.** The `HistoryWrite` AccessLevel bit decides whether the SDK base admits a node to HistoryUpdate at all, so the updater must be wired before any node materialises. `OtOpcUaServerHostedService.StartAsync` assigns it (via the new `OtOpcUaSdkServer.SetHistorianDataUpdater`) before `SetSink`. Nodes materialised earlier keep their AccessLevel (no retro-update).
9. **HistoryUpdate results.** Successful arms set `errors[i] = Good`. A `SourceTimestamp == DateTime.MinValue` is `BadInvalidTimestamp`.
10. **Comms-loss IS historized, as an outage marker** (corrected after the T3 review). `DriverHostActor.FanCommsLossToDependencyMux` (#481) sends `null` + `BadNoCommunication` for every value ref to the mux, so the recorder receives it, and it cannot be told apart from a driver's own Bad publish. On a backend whose `SupportedKinds` include `Empty` (SQL Server), each disconnect stores one `Empty`/`BadNoCommunication` sample per historized tag. This is deliberate: it records the gap, so raw reads show the outage and aggregates and interpolation do not bridge it. On a numeric-only backend (Gateway) it is dropped as an unsupported kind and counted in `DroppedUnsupportedKind`. A driver's own Bad publish is handled the same way (`Empty` kind plus its Bad code, e.g. Modbus `0x80050000`).
11. **Parallel safety.** `SqlServerTagCatalog` belongs to the skeleton task (T8), so the parallel SQL tasks share one.