lmxopcua/docs/plans/2026-10-09-pluggable-historian-design.md

Pluggable historian backends — design

Date: 2026-10-09 · Status: approved (brainstorm 2026-10-09) · Plan: 2026-10-09-pluggable-historian-plan.md

1. Goal and scope

Today the only historian backend is AVEVA/Wonderware, reached through the HistorianGateway sidecar. This work makes the backend pluggable, ships SQL Server as the first new backend, and adds the OPC UA HistoryUpdate service so clients can insert / replace / update / delete raw history.

In scope (this spec)	Later (own spec)	Dropped
Provider seam + ServerHistorian:Backend selection	QuestDB backend (a new provider only)	SQLite (a per-node file needs pair replication)
Typed values (not just double) end to end through recorder + outbox
Real status codes recorded (fixes a hard-coded Good)
SQL Server backend: schema, writes, reads, aggregates push-down, alarm events, provisioning, updates
HistoryUpdate service + HistoryUpdate role / permission + AdminUI checkbox
Client.CLI historyupdate command

Out of scope everywhere: retention/purge, partitioning/compression, Part 11 "modified values" (IsReadModified, IsDeleteModified), event HistoryUpdate (UpdateEventDetails/DeleteEventDetails), UpdateStructureData, AuditHistoryUpdateEvent emission, array historization.

Hard constraints: no new NuGet packages and no version changes (Microsoft.Data.SqlClient is already centrally pinned — referencing it from a new project is allowed, adding a new package is not). Recording stays Primary-only (#537); delivery stays at-least-once with the #538 per-tag quarantine.

2. Architecture — the provider seam (approach A)

The five historian seams the rest of the code already consumes stay; a provider builds them.

// src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/IHistorianBackendProvider.cs
public interface IHistorianBackendProvider
{
    string Name { get; }                                         // "Gateway" | "SqlServer" | (later) "QuestDb"
    IReadOnlySet<HistorianValueKind> SupportedKinds { get; }     // recorder capture filter
    IHistorianDataSource   CreateDataSource(IServiceProvider sp);
    IHistorianValueWriter  CreateValueWriter(IServiceProvider sp);
    IAlarmHistorianWriter  CreateAlarmWriter(IServiceProvider sp);
    IHistorianProvisioning CreateProvisioning(IServiceProvider sp);
    IHistorianDataUpdater? CreateUpdater(IServiceProvider sp);  // null ⇒ HistoryUpdate unsupported
}

It lives in Runtime because the seams it returns span Core.Abstractions and Core.AlarmHistorian.

• GatewayHistorianProvider (Driver.Historian.Gateway) wraps today's GatewayHistorian.Create… factories; CreateUpdater returns null; SupportedKinds = {Double, Int64, Boolean}.
• SqlServerHistorianProvider (new project Driver.Historian.SqlServer); SupportedKinds = all kinds.
• The Host owns a name→provider table (HistorianBackendRegistry). Program.cs's four hard-wired GatewayHistorian.Create… call sites (three, plus the inline new GatewayHistorianValueWriter(...)) become calls on the selected provider. The existing gates (AddServerHistorian, AddHistorianProvisioning, AddAlarmHistorian, the outbox registration, the recorder spawn) are unchanged — they get their objects from the provider.
• The outbox (FasterLogHistorizationOutbox) stays in the Gateway project (moving it would add a FASTER package reference elsewhere); the Host registers it regardless of backend. The SqlServer project does not reference the Gateway project.
• Rejected: one fat IHistorian (forces stubs, churns every narrow consumer); per-capability backend selection (YAGNI, doubles config + validation).

Configuration

"ServerHistorian": {
  "Enabled": true,
  "Backend": "SqlServer",              // "Gateway" (default when absent) | "SqlServer"
  "HistoryReadBatchParallelism": 4, "MaxConcurrentHistoryReadBatches": 16, "HistoryReadDeadline": "00:01:00",
  "Endpoint": "...", "ApiKey": "...",  // gateway keys stay top-level, unchanged (no migration)
  "SqlServer": {
    "ConnectionString": "${secret:OT-HISTORIAN-SQL}",
    "Schema": "hist",
    "CommandTimeout": "00:00:30",
    "AutoCreateSchema": true
  }
}

• Unknown Backend ⇒ host start fails (OptionsValidationException listing the valid names).
• ServerHistorianOptionsValidator runs its three gateway checks (Endpoint, ApiKey, UseTls⇔scheme) only when Backend == Gateway; backend-specific options are validated there too, consumer-gated (ServerHistorian:Enabled || AlarmHistorian:Enabled), never at registry resolution.
• New SqlServerHistorianOptions validation: ConnectionString required, Schema a valid SQL identifier (^[A-Za-z_][A-Za-z0-9_]{0,127}$ — it is spliced into DDL/DML), CommandTimeout > 0.
• Validators remain driver-block-only (admin-only nodes never build historian singletons). As built: an invalid SqlServer:Schema also throws ArgumentException when the provider is constructed, so Backend=SqlServer + a malformed schema stops host start even when no consumer is enabled.
• The Program.cs alarm-only warning names the selected backend instead of "gateway".
• The historian connection string is not a ConfigDb string; driver-only nodes may carry it (Phase 4 intact).

3. Typed values

// Core.Abstractions/Historian/HistorianValue.cs
public enum HistorianValueKind : byte { Empty = 0, Double = 1, Int64 = 2, Boolean = 3, String = 4, DateTime = 5 }

public readonly record struct HistorianValue(HistorianValueKind Kind, double Number, long Integer, string? Text)
{
    public static HistorianValue Empty { get; }                 // no value (a Bad/Uncertain sample with a null)
    public static HistorianValue FromDouble(double v);
    public static HistorianValue FromInt64(long v);
    public static HistorianValue FromBoolean(bool v);           // Integer = 1/0, Number = 1/0
    public static HistorianValue FromString(string v);
    public static HistorianValue FromDateTime(DateTime utc);    // Integer = Ticks (UTC)
    public static bool TryFromObject(object? value, out HistorianValue result); // the recorder's coercion
    public bool TryGetDouble(out double v);                     // Double/Int64/Boolean ⇒ true
    public object? ToObject();                                  // back to a CLR value for DataValueSnapshot
}

TryFromObject mapping: double/float/decimal ⇒ Double; sbyte/byte/short/ushort/int/uint/long ⇒ Int64; ulong ⇒ Int64 when ≤ long.MaxValue else Double; bool ⇒ Boolean; string ⇒ String; DateTime ⇒ DateTime (converted to UTC); null ⇒ Empty; arrays / anything else ⇒ false (dropped + metered).

Changed records:

public readonly record struct HistorizationValue(DateTime? TimestampUtc, HistorianValue Value, uint StatusCode);
public sealed record HistorizationOutboxEntry(Guid Id, string Tag, HistorianValue Value, uint StatusCode, DateTime TimestampUtc);

Recorder. Captures HistorianValue via TryFromObject; records msg.StatusCode (the mux already carries it — the recorder hard-codes GoodQuality = 192 today, so Bad samples were historized as Good). A value whose kind is not in the provider's SupportedKinds is dropped at capture and counted under DroppedUnsupportedKind (renamed from DroppedNonNumeric, same meter semantics for the gateway). Empty is in SQL Server's set (a Bad sample with no value is history worth keeping) and not the gateway's.

Gateway backend: no wire change in practice. Samples now carry their real status, mapped to DA quality inside GatewayHistorianValueWriter by a new reverse mapper GatewayQualityMapper.ToDaQuality(uint) — the exact inverse of the existing Map(byte) table, falling back by severity (Good ⇒ 192, Uncertain ⇒ 64, Bad ⇒ 0). The recorder no longer knows about DA quality. But the gateway's WriteLiveValues SQL path discards Quality (Contracts C-002, server-managed), so AVEVA keeps storing every recorded value as Good — a pre-existing limitation of that backend, now documented rather than fixed. The SQL Server backend stores the real status.

Outbox format v2. Entries are written magic "HV2\0" + version (1 byte) + id(16) + tagLen(4) + tag + kind(1) + kind payload + status(4) + timestamp(8). Payload by kind: Empty none; Double 8; Int64/Boolean/DateTime 8; String len(4)+UTF-8. Legacy entries (no magic, length == 38 + tagLen@16) are read as Double with status GatewayQualityMapper.Map(quality) (DA ⇒ StatusCode; legacy entries are always 192 ⇒ Good). An entry matching neither layout is dropped and counted (otopcua.historian.outbox.unreadable), never thrown — a corrupt entry must not jam the drain. A pinned legacy-bytes test guards the upgrade path.

4. SQL Server backend

Schema ({Schema} default hist, database e.g. OtOpcUaHistorian)

CREATE TABLE {s}.SchemaVersion (Version int NOT NULL PRIMARY KEY, AppliedUtc datetime2(7) NOT NULL);

CREATE TABLE {s}.Tag (
  TagId int IDENTITY(1,1) NOT NULL PRIMARY KEY,
  Name nvarchar(400) NOT NULL CONSTRAINT UQ_Tag_Name UNIQUE,
  DataType tinyint NOT NULL,                 -- DriverDataType
  EngineeringUnit nvarchar(64) NULL, Description nvarchar(400) NULL,
  CreatedUtc datetime2(7) NOT NULL, UpdatedUtc datetime2(7) NOT NULL);

CREATE TABLE {s}.Sample (
  TagId int NOT NULL, SourceTimeUtc datetime2(7) NOT NULL,
  ValueKind tinyint NOT NULL,                -- HistorianValueKind
  NumValue float NULL, IntValue bigint NULL, StrValue nvarchar(max) NULL,
  StatusCode int NOT NULL,                   -- uint StatusCode stored bit-for-bit (unchecked cast)
  ServerTimeUtc datetime2(7) NOT NULL,
  CONSTRAINT PK_Sample PRIMARY KEY CLUSTERED (TagId, SourceTimeUtc),
  CONSTRAINT FK_Sample_Tag FOREIGN KEY (TagId) REFERENCES {s}.Tag(TagId));

CREATE TABLE {s}.Event (
  EventId nvarchar(64) NOT NULL CONSTRAINT PK_Event PRIMARY KEY NONCLUSTERED, -- deterministic SHA-256 hex of the event payload
  SourceName nvarchar(400) NOT NULL,                  -- AlarmHistorianEvent.EquipmentPath
  AlarmId nvarchar(400) NOT NULL, AlarmName nvarchar(400) NOT NULL, AlarmTypeName nvarchar(100) NOT NULL,
  Severity int NOT NULL, EventKind nvarchar(50) NOT NULL, Message nvarchar(max) NULL,
  UserName nvarchar(200) NULL, Comment nvarchar(max) NULL,
  EventTimeUtc datetime2(7) NOT NULL, ReceivedTimeUtc datetime2(7) NOT NULL);
CREATE CLUSTERED INDEX CX_Event_Time ON {s}.Event (EventTimeUtc, EventId);   -- as built (§9.6); the sketched IX_Event_Time is replaced by this
CREATE INDEX IX_Event_Source_Time ON {s}.Event (SourceName, EventTimeUtc);

CREATE TYPE {s}.SampleRow AS TABLE (
  SourceTimeUtc datetime2(7) NOT NULL PRIMARY KEY, ValueKind tinyint NOT NULL,
  NumValue float NULL, IntValue bigint NULL, StrValue nvarchar(max) NULL, StatusCode int NOT NULL);
CREATE TYPE {s}.TimeRow AS TABLE (T datetime2(7) NOT NULL PRIMARY KEY);

Column mapping per kind: Double ⇒ NumValue; Int64 ⇒ IntValue + NumValue (double, for aggregates); Boolean ⇒ IntValue 0/1 + NumValue 0/1; String ⇒ StrValue; DateTime ⇒ IntValue (ticks); Empty ⇒ all NULL.

The PK enforces Part 11's one-value-per-timestamp and makes outbox redelivery idempotent: live writes are insert-if-absent. Events are insert-if-absent on EventId (AlarmHistorianEvent carries no id, so the writer computes SHA-256(AlarmId|EventKind|TimestampUtc.Ticks|User|Comment|Message), hex).

Bootstrap. Numbered idempotent scripts (001_initial.sql) are embedded resources (and mirrored to docs/historian/sqlserver/). SqlServerSchemaBootstrapper runs once per process on first use, lazily, under sp_getapplock @Resource='OtOpcUaHistorianSchema', @LockMode='Exclusive', @LockOwner='Session'. AutoCreateSchema=true creates the schema + applies missing scripts; false only checks SchemaVersion and, if behind, every historian call fails with a clear error naming the missing version (surfaced in GetHealthSnapshot().LastError, never a host crash). Bootstrap failure is retried on the next call (not latched).

Writes

• IHistorianValueWriter.WriteLiveValuesAsync(tag, values): resolve TagId via a ConcurrentDictionary<string,int> cache, creating the tag row on a miss (MERGE on Name; DataType from the first value's kind, provisioning later corrects it). Insert via {s}.SampleRow TVP with INSERT … SELECT … WHERE NOT EXISTS. Returns true on success, false on a SQL failure (the recorder's retry/quarantine owns retry). ServerTimeUtc = SYSUTCDATETIME(). As built: duplicate timestamps within one batch collapse to the first; a NaN/±Inf Double is stored as Empty (it cannot live in a SQL float) and, per the final-review I-1 ruling, an Empty sample is always Bad: a non-Bad non-finite value is stored BadOutOfRange, any other non-Bad Empty BadNoData, an existing Bad code kept; reads clamp a legacy non-Bad Empty row to BadNoData; tag names over 400 characters are refused, never truncated; a 2627/2601 duplicate-key error rolls the statement back, so the statement is re-run (≤ 2 retries) rather than treated as success.
• IHistorianProvisioning.EnsureTagsAsync: MERGE upsert of every request (all types — no numeric filter on this backend). Per-request failures are counted, never thrown.
• IAlarmHistorianWriter.WriteBatchAsync: per event insert-if-absent; outcome Ack on insert or already-present, RetryPlease on transient SQL errors (connection/timeout/deadlock 1205), PermanentFail on data errors (e.g. truncation 2628/8152).

Reads

• Raw: SELECT TOP(@max) … WHERE TagId=@id AND SourceTimeUtc >= @lo AND SourceTimeUtc < @hi ORDER BY SourceTimeUtc ASC (forward) or > @lo AND <= @hi … DESC when endUtc < startUtc (Part 11 reverse). Unknown tag ⇒ empty result (node manager reports GoodNoData). The node manager's paging is unchanged; tie clusters cannot occur (PK).

• AtTime: one statement, {s}.TimeRow TVP + two OUTER APPLYs (previous-or-equal, next). Exact ⇒ stored value + status. Otherwise Double/Int64 linearly interpolated (status Good with the historian Interpolated info bits when both neighbours are Good, else UncertainDataSubNormal + Interpolated; Int64 is interpolated through double and rounded, ties away from zero), Boolean/String/DateTime — and any time after the last sample — stepped from the previous sample (status keeps the previous sample's code + Interpolated bit); no previous sample ⇒ BadNoData. If the previous neighbour is Bad, the requested time lies inside an outage: the result is a null value carrying that neighbour's Bad code (never better than its source, per the quality contract; corrected after the T10 review). If the previous neighbour is usable and the next is Bad, the previous value is held (stepped, not interpolated into the outage) with UncertainDataSubNormal.

• Events: TOP(@maxEvents) from {s}.Event filtered by SourceName = @source (null ⇒ all) and EventTimeUtc range, ordered ascending (a reversed window is swapped and still returned ascending). HistoricalEvent(EventId, SourceName, EventTimeUtc, ReceivedTimeUtc, Message, (ushort)Severity).

• Processed (push-down): bucket b = elapsedTicks / @intervalTicks in 100 ns ticks (corrected after T10: the original sketch used DATEDIFF_BIG(millisecond, …) / @intervalMs; DATEDIFF counts boundaries crossed, so a ms formulation misplaces samples near a bucket edge and a ns one overflows past ≈292 years. As built: DATEDIFF_BIG(microsecond, @start, t) × 10 plus a sub-microsecond tick correction), one row per bucket start @start + b × interval for every interval in [start, end) (empty intervals emitted as BadNoData). Usable sample = status severity bits Good (00) or Uncertain (01) and a stored value (an Empty outage marker is Bad whatever its code). Every result carries the historian Calculated info bits. An interval with any non-Good (Bad or Uncertain) sample but ≥1 usable ⇒ UncertainDataSubNormal; no usable ⇒ BadNoData (so Count over an all-Bad interval is BadNoData, not 0). A reversed window is swapped, anchored at the earlier time, and returned descending.

  Aggregate	SQL over usable samples in the bucket	Value timestamp
  Average / Minimum / Maximum / Total / Count	AVG / MIN / MAX / SUM (NumValue), COUNT(*)	bucket start
  Start / End	first / last by SourceTimeUtc (ROW_NUMBER)	bucket start
  Delta	End − Start	bucket start
  Range	Max − Min	bucket start
  MinimumActualTime / MaximumActualTime	min / max value, ties ⇒ earliest	the sample's time
  PercentGood / PercentBad	100 × count(Good) / count(*) and 100 × count(Bad) / count(*) over ALL samples; Uncertain samples are neither (so the two can sum to < 100)	bucket start

  Count / Start / End / PercentGood / PercentBad accept any kind; the rest on a String/DateTime tag ⇒ BadAggregateInvalidInputs per interval. Any other HistoryAggregateType ⇒ throw HistoryAggregateNotSupportedException (⇒ BadAggregateNotSupported). These are the simple per-sample definitions, not Part 13's full interval-boundary semantics — documented as such.

Updates (IHistorianDataUpdater)

public enum HistoryUpdateMode { Insert, Replace, Update }      // Part 11 PerformUpdateType minus Remove
public interface IHistorianDataUpdater
{
    Task<IReadOnlyList<uint>> UpdateDataAsync(string tag, HistoryUpdateMode mode,
        IReadOnlyList<HistorizationValue> values, CancellationToken ct);       // one code per value, in order
    Task<uint> DeleteRawAsync(string tag, DateTime startUtc, DateTime endUtc, CancellationToken ct);
    Task<IReadOnlyList<uint>> DeleteAtTimeAsync(string tag, IReadOnlyList<DateTime> timesUtc, CancellationToken ct);
}

Operation	Per-item result
Insert	GoodEntryInserted; BadEntryExists if a sample exists at that timestamp
Replace	GoodEntryReplaced; BadNoEntryExists if none
Update	GoodEntryInserted or GoodEntryReplaced
DeleteRaw	Good; BadNoData if the range held nothing
DeleteAtTime	per time: Good; BadNoEntryExists

Each call is one transaction (MERGE … OUTPUT $action against the TVP). A value with a null timestamp ⇒ BadInvalidTimestamp for that item. As built: a NaN/±Inf Double ⇒ BadInvalidArgument (rejected, not stored as Empty as the live-value writer does), and so is a null value with a non-Bad status (I-1); no DataType check against the node (M-1, decided: values are stored as sent); a DeleteRawModified with a MinValue bound ⇒ BadInvalidArgument; duplicate timestamps in one request — Insert: first applies, later BadEntryExists; Replace/Update: last wins, earlier ones share its code; DeleteAtTime: first reports, later BadNoEntryExists. The node manager answers a null item BadInvalidArgument, MinValue BadInvalidTimestamp, an untypable value BadTypeMismatch, a backend throw BadInternalError and a deadline BadTimeout (a failure is not "unsupported"). Update/Insert on an unknown tag creates the tag row (same as live writes); Replace/Delete on an unknown tag ⇒ BadNoEntryExists / BadNoData.

Status codes. Every status this backend produces is a named const uint in an internal SqlHistorianStatusCodes class (the house rule: StatusCodeParityTests guards named constants by reflection against Opc.Ua.StatusCodes, and an inline literal is invisible to it) — register the class with that parity test. Historian info bits follow Part 4 §7.39: InfoType = DataValue (0x00000400) OR-ed with HistorianCalculated (0x1) / HistorianInterpolated (0x2); the constants are checked against the SDK's AggregateBits/StatusCode helpers in a unit test rather than trusted from this doc.

5. HistoryUpdate in the server

• OtOpcUaNodeManager overrides the SDK's protected HistoryUpdateData, HistoryDeleteRawModified, HistoryDeleteAtTime (verify exact signatures against the pinned SDK before writing; the read overrides at OtOpcUaNodeManager.cs ~4928 are the pattern). The node → historian tagname lookup is the existing TryGetHistorizedTagname, so raw and UNS NodeIds hit the same rows. Not historized ⇒ BadHistoryOperationUnsupported.
• New settable IHistorianDataUpdater? HistorianDataUpdater on the node manager, assigned by OtOpcUaServerHostedService next to HistorianDataSource (resolved as optional — null when the backend has none). Null ⇒ every HistoryUpdate item BadHistoryOperationUnsupported.
• AccessLevel: ComposeAccessLevel(writable, historized) gains historyWritable — the HistoryWrite bit is OR-ed in when historized and an updater is wired (both raw node and UNS mirrors).
• Out-of-scope details (events, structure data, IsDeleteModified) ⇒ BadHistoryOperationUnsupported.
• Updates go straight to the backend (not the outbox), are not Primary-gated (the database is shared), and share the HistoryRead batch limiter + deadline.
• Authorization.
  • ACL flag on: DataPlaneAuthorizer.Authorize(identity, OpcUaOperation.HistoryUpdate, nodeId, realm) (TriePermissionEvaluator already maps it to NodePermissions.HistoryUpdate).
  • ACL flag off: new role OpcUaDataPlaneRoles.HistoryUpdate = "HistoryUpdate", conferred by ApiKeyInterimRoleMapper only from a Cluster-scoped grant carrying NodePermissions.HistoryUpdate, plus AuthorizeKeyUsable (#563 key re-check). Deny ⇒ BadUserAccessDenied.
  • Load-bearing test: an ALLOW — flag on, session with no roles, scoped HistoryUpdate grant ⇒ the insert lands. And flag off: a WriteOperate-only key is DENIED (the role is not reused).
  • NodePermissions.HistoryUpdate is added to the AdminUI ACL editor's permission list (AclEdit.razor); it stays out of every composite bundle.

6. Client.CLI

historyupdate command: -u <url> -n <nodeId> --mode insert|replace|update|delete-raw|delete-at-time, --value <v> --time <iso> (repeatable for insert/replace/update/delete-at-time), --start/--end for delete-raw. Prints one line per item with its status code name. Docs in docs/Client.CLI.md.

7. Testing

• Unit: HistorianValue coercion table; outbox v2 round-trip per kind + pinned legacy bytes + unreadable-entry drop; recorder typed capture, real status passthrough, SupportedKinds filter (incl. Empty), unchanged #537/#538 behaviour; gateway writer conversion (Good bytes identical to today); provider registry + validators; role mapper; node manager HistoryUpdate (fake updater): allow/deny matrix for flag on/off, result-code passthrough, unsupported cases, AccessLevel bit.
• SQL Server integration (Driver.Historian.SqlServer.IntegrationTests): env-gated on HISTORIAN_SQL_TEST_CONNECTION (a master-capable connection string; absent ⇒ Assert.Skip), each run in a throwaway database OtOpcUaHistorian_T_<guid> dropped on dispose. Default target: the docker-dev sql container, Server=localhost,14330;User Id=sa;Password=OtOpcUa!Dev123;TrustServerCertificate=True. Covers bootstrap (idempotent; two concurrent bootstrappers), every kind round-trip, redelivery idempotency, raw forward/reverse/limit, AtTime exact/interpolated/stepped/no-prior, each aggregate against hand-computed values incl. Bad-sample and empty-interval cases, events, every update/delete result code, unknown tags.
• Over-the-wire: one OpcUaServer.IntegrationTests test drives HistoryUpdate Insert then HistoryRead through a real SDK client against a fake updater/data source.
• Live gate (docker-dev): central pair on Backend=SqlServer against database OtOpcUaHistorian on the rig sql container; a historized rig tag records rows (Primary only — row count advances once per value); Client.CLI historyread raw + processed; historyupdate insert/replace/delete round-trip; stop the Primary ⇒ the Secondary takes over recording into the same table with no duplicate rows; a Bad sample is stored with its Bad status. Evidence recorded in docs/plans/2026-10-09-pluggable-historian-live-gate.md.

8. Docs and index

docs/Historian.md (backend selection, SQL schema, aggregate semantics, HistoryUpdate), docs/Configuration.md (ServerHistorian:Backend + SqlServer subsection), docs/security.md (HistoryUpdate role/permission), docs/Client.CLI.md, CLAUDE.md Historian section, and the scadaproj sister-project-history/otopcua.md entry (ScadaBridge impact: none unless it issues HistoryUpdate; HistoryRead semantics unchanged on the gateway backend apart from honest Bad quality on Bad samples).

9. Pre-execution review amendments (Fable, 2026-10-09)

These were verified against the code and override anything above that conflicts.

1. Outbox recovery. FasterLogHistorizationOutbox's constructor (RecoverState) deserializes every record too, so the "unreadable ⇒ drop + count, never throw" rule applies there as well. An unreadable record is indexed under a synthetic id so FIFO truncation can pass it. If the v2 magic matches but structural validation fails, the legacy exact-length test runs before the entry is declared unreadable. IHistorizationOutbox gains UnreadableCount (default 0).
2. Validation is as in §2 (all three gateway checks gated on Backend; consumer-gated backend validation). HistorianUpdaterHolder.None is always registered on driver nodes; the real updater only when ServerHistorian:Enabled.
3. Timestamps are bound as datetime2(7) everywhere: SqlDbType.DateTime2 scalar parameters and SqlDataRecord TVPs, never DataTable, which would round to 3.33 ms datetime.
4. Raw maxValuesPerNode == 0 means unbounded. Events fetch maxEvents + 1 and return a non-null continuation point when truncated, as the interface requires.
5. Concurrency. Tag creation and sample MERGEs use WITH (HOLDLOCK), and 2627/2601 is treated as "already present". Both pair nodes provision on every deploy.
6. Event has a nonclustered PK on EventId and is clustered on (EventTimeUtc, EventId). The EventId hashes length-prefixed fields and includes EquipmentPath.
7. Processed reads are capped at 100 000 buckets per request.
8. HistoryUpdate admission. The HistoryWrite AccessLevel bit decides whether the SDK base admits a node to HistoryUpdate at all, so the updater must be wired before any node materialises. OtOpcUaServerHostedService.StartAsync assigns it (via the new OtOpcUaSdkServer.SetHistorianDataUpdater) before SetSink. Nodes materialised earlier keep their AccessLevel (no retro-update).
9. HistoryUpdate results. Successful arms set errors[i] = Good. A SourceTimestamp == DateTime.MinValue is BadInvalidTimestamp.
10. Comms-loss IS historized, as an outage marker (corrected after the T3 review). DriverHostActor.FanCommsLossToDependencyMux (#481) sends null + BadNoCommunication for every value ref to the mux, so the recorder receives it, and it cannot be told apart from a driver's own Bad publish. On a backend whose SupportedKinds include Empty (SQL Server), each disconnect stores one Empty/BadNoCommunication sample per historized tag. This is deliberate: it records the gap, so raw reads show the outage and aggregates and interpolation do not bridge it. On a numeric-only backend (Gateway) it is dropped as an unsupported kind and counted in DroppedUnsupportedKind. A driver's own Bad publish is handled the same way (Empty kind plus its Bad code, e.g. Modbus 0x80050000).
11. Parallel safety. SqlServerTagCatalog belongs to the skeleton task (T8), so the parallel SQL tasks share one.