docs(historian): pluggable backends, SQL Server backend, HistoryUpdate, outbox v2, spec corrections

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
This commit is contained in:
Joseph Doherty
2026-10-09 13:51:56 -04:00
parent 71fe3a11a7
commit cc34daf181
7 changed files with 471 additions and 61 deletions
@@ -80,7 +80,9 @@ It lives in **Runtime** because the seams it returns span `Core.Abstractions` an
- `ServerHistorianOptionsValidator` runs its three gateway checks (Endpoint, ApiKey, UseTls⇔scheme) only when `Backend == Gateway`; backend-specific options are validated there too, consumer-gated (`ServerHistorian:Enabled || AlarmHistorian:Enabled`), never at registry resolution.
- New `SqlServerHistorianOptions` validation: `ConnectionString` required, `Schema` a valid SQL identifier
(`^[A-Za-z_][A-Za-z0-9_]{0,127}$` — it is spliced into DDL/DML), `CommandTimeout > 0`.
- Validators remain driver-block-only (admin-only nodes never build historian singletons).
- Validators remain driver-block-only (admin-only nodes never build historian singletons). *As built:* an invalid `SqlServer:Schema`
also throws `ArgumentException` when the provider is constructed, so `Backend=SqlServer` + a malformed schema stops host start even
when no consumer is enabled.
- The `Program.cs` alarm-only warning names the selected backend instead of "gateway".
- The historian connection string is **not** a ConfigDb string; driver-only nodes may carry it (Phase 4 intact).
@@ -160,14 +162,14 @@ CREATE TABLE {s}.Sample (
CONSTRAINT FK_Sample_Tag FOREIGN KEY (TagId) REFERENCES {s}.Tag(TagId));
CREATE TABLE {s}.Event (
EventId nvarchar(64) NOT NULL PRIMARY KEY, -- deterministic SHA-256 hex of the event payload
EventId nvarchar(64) NOT NULL CONSTRAINT PK_Event PRIMARY KEY NONCLUSTERED, -- deterministic SHA-256 hex of the event payload
SourceName nvarchar(400) NOT NULL, -- AlarmHistorianEvent.EquipmentPath
AlarmId nvarchar(400) NOT NULL, AlarmName nvarchar(400) NOT NULL, AlarmTypeName nvarchar(100) NOT NULL,
Severity int NOT NULL, EventKind nvarchar(50) NOT NULL, Message nvarchar(max) NULL,
UserName nvarchar(200) NULL, Comment nvarchar(max) NULL,
EventTimeUtc datetime2(7) NOT NULL, ReceivedTimeUtc datetime2(7) NOT NULL);
CREATE CLUSTERED INDEX CX_Event_Time ON {s}.Event (EventTimeUtc, EventId); -- as built (§9.6); the sketched IX_Event_Time is replaced by this
CREATE INDEX IX_Event_Source_Time ON {s}.Event (SourceName, EventTimeUtc);
CREATE INDEX IX_Event_Time ON {s}.Event (EventTimeUtc);
CREATE TYPE {s}.SampleRow AS TABLE (
SourceTimeUtc datetime2(7) NOT NULL PRIMARY KEY, ValueKind tinyint NOT NULL,
@@ -196,7 +198,10 @@ next call (not latched).
`ConcurrentDictionary<string,int>` cache, creating the tag row on a miss (`MERGE` on Name; DataType from
the first value's kind, provisioning later corrects it). Insert via `{s}.SampleRow` TVP with
`INSERT … SELECT … WHERE NOT EXISTS`. Returns `true` on success, `false` on a SQL failure (the recorder's
retry/quarantine owns retry). `ServerTimeUtc = SYSUTCDATETIME()`.
retry/quarantine owns retry). `ServerTimeUtc = SYSUTCDATETIME()`. *As built:* duplicate timestamps within one batch
collapse to the **first**; a `NaN`/`±Inf` `Double` is stored as `Empty` keeping its status (it cannot live in a SQL
`float`); a 2627/2601 duplicate-key error rolls the statement back, so the statement is re-run (≤ 2 retries) rather
than treated as success.
- `IHistorianProvisioning.EnsureTagsAsync`: `MERGE` upsert of every request (all types — no numeric
filter on this backend). Per-request failures are counted, never thrown.
- `IAlarmHistorianWriter.WriteBatchAsync`: per event insert-if-absent; outcome `Ack` on insert or
@@ -211,20 +216,25 @@ next call (not latched).
tie clusters cannot occur (PK).
- **AtTime:** one statement, `{s}.TimeRow` TVP + two `OUTER APPLY`s (previous-or-equal, next). Exact ⇒ stored
value + status. Otherwise Double/Int64 linearly interpolated (status `Good` with the historian
*Interpolated* info bits), Boolean/String/DateTime stepped from the previous
sample (status keeps the previous sample's code + Interpolated bit); no previous sample ⇒ `BadNoData`.
*Interpolated* info bits **when both neighbours are Good, else `UncertainDataSubNormal` + Interpolated**; Int64 is
interpolated through `double` and rounded, ties away from zero), Boolean/String/DateTime — and any time **after the last
sample** — stepped from the previous sample (status keeps the previous sample's code + Interpolated bit); no previous
sample ⇒ `BadNoData`.
If the **previous** neighbour is Bad, the requested time lies inside an outage: the result is a null value
carrying that neighbour's Bad code (never better than its source, per the quality contract; corrected after the
T10 review). If the previous neighbour is usable and the **next** is Bad, the previous value is held (stepped,
not interpolated into the outage) with `UncertainDataSubNormal`.
- **Events:** `TOP(@maxEvents)` from `{s}.Event` filtered by `SourceName = @source` (null ⇒ all) and
`EventTimeUtc` range, ordered ascending. `HistoricalEvent(EventId, SourceName, EventTimeUtc,
`EventTimeUtc` range, ordered ascending (a reversed window is swapped and still returned ascending). `HistoricalEvent(EventId, SourceName, EventTimeUtc,
ReceivedTimeUtc, Message, (ushort)Severity)`.
- **Processed (push-down):** bucket `b = DATEDIFF_BIG(millisecond, @start, SourceTimeUtc) / @intervalMs`,
one row per bucket start `@start + b × interval` for every interval in `[start, end)` (empty intervals
emitted as `BadNoData`). Usable sample = status severity bits Good (`00`) or Uncertain (`01`). Every result
carries the historian *Calculated* info bits. An interval with any Bad
sample but ≥1 usable ⇒ `UncertainDataSubNormal`; no usable ⇒ `BadNoData`.
- **Processed (push-down):** bucket `b = elapsedTicks / @intervalTicks` in **100 ns ticks** (*corrected after T10: the original
sketch used `DATEDIFF_BIG(millisecond, …) / @intervalMs`; `DATEDIFF` counts boundaries crossed, so a ms formulation
misplaces samples near a bucket edge and a ns one overflows past ≈292 years. As built:
`DATEDIFF_BIG(microsecond, @start, t) × 10` plus a sub-microsecond tick correction*), one row per bucket start `@start + b × interval` for every interval in `[start, end)` (empty intervals
emitted as `BadNoData`). Usable sample = status severity bits Good (`00`) or Uncertain (`01`) **and a stored value** (an `Empty`
outage marker is Bad whatever its code). Every result carries the historian *Calculated* info bits. An interval with any
**non-Good (Bad or Uncertain)** sample but ≥1 usable ⇒ `UncertainDataSubNormal`; no usable ⇒ `BadNoData` (so `Count` over an
all-Bad interval is `BadNoData`, not 0). A reversed window is swapped, anchored at the earlier time, and returned descending.
| Aggregate | SQL over usable samples in the bucket | Value timestamp |
|---|---|---|
@@ -233,7 +243,7 @@ next call (not latched).
| Delta | End − Start | bucket start |
| Range | Max − Min | bucket start |
| MinimumActualTime / MaximumActualTime | min / max value, ties ⇒ earliest | the sample's time |
| PercentGood / PercentBad | `100 × count(good or bad) / count(*)` over ALL samples | bucket start |
| PercentGood / PercentBad | `100 × count(Good) / count(*)` and `100 × count(Bad) / count(*)` over ALL samples; **Uncertain samples are neither** (so the two can sum to < 100) | bucket start |
Count / Start / End / PercentGood / PercentBad accept any kind; the rest on a String/DateTime tag ⇒
`BadAggregateInvalidInputs` per interval. Any other `HistoryAggregateType` ⇒ throw
@@ -262,7 +272,11 @@ public interface IHistorianDataUpdater
| DeleteAtTime | per time: `Good`; `BadNoEntryExists` |
Each call is one transaction (`MERGE … OUTPUT $action` against the TVP). A value with a null timestamp ⇒
`BadInvalidTimestamp` for that item. Update/Insert on an unknown tag creates the tag row (same as live
`BadInvalidTimestamp` for that item. *As built:* a `NaN`/`±Inf` `Double` ⇒ `BadInvalidArgument` (rejected, not stored as `Empty`
as the live-value writer does); duplicate timestamps in one request — Insert: first applies, later `BadEntryExists`; Replace/Update:
last wins, earlier ones share its code; DeleteAtTime: first reports, later `BadNoEntryExists`. The node manager answers a null item
`BadInvalidArgument`, `MinValue` `BadInvalidTimestamp`, an untypable value `BadTypeMismatch`, a backend throw `BadInternalError`
and a deadline `BadTimeout` (a failure is not "unsupported"). Update/Insert on an unknown tag creates the tag row (same as live
writes); Replace/Delete on an unknown tag ⇒ `BadNoEntryExists` / `BadNoData`.
**Status codes.** Every status this backend produces is a named `const uint` in an internal