Files

T

Joseph Doherty a6643cedf9 docs(historian): fold Fable pre-execution review into design + plan (3 blockers, 12 should-fix, 11 nits)

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe

2026-10-09 12:40:35 -04:00

66 KiB

Raw Blame History

Pluggable Historian Backends Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers-extended-cc:subagent-driven-development (recommended) or superpowers-extended-cc:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Make the historian backend pluggable (ServerHistorian:Backend), carry typed values + real status end to end, ship a SQL Server backend, and add the OPC UA HistoryUpdate service.

Architecture: A Runtime-level IHistorianBackendProvider builds the five historian seams (data source, value writer, alarm writer, provisioning, updater); the Host picks one by name. The existing recorder/outbox/alarm S&F pipelines are unchanged in structure and now carry a typed HistorianValue + uint StatusCode. SQL Server stores samples in a (TagId, SourceTimeUtc)-clustered table and pushes aggregates down to T-SQL.

Tech Stack: .NET 10, Akka.NET, OPC Foundation UA .NET Standard, Microsoft.Data.SqlClient (already pinned centrally), FASTER log (existing), xUnit v3 + Shouldly.

Spec: docs/plans/2026-10-09-pluggable-historian-design.md — the binding authority. Read it before any task.

Global Constraints

No new NuGet packages, no version changes. Directory.Packages.props is not edited by any task. Adding <PackageReference Include="Microsoft.Data.SqlClient" /> (already pinned) to a new project is allowed.
Recording stays Primary-only (#537 capture gate) and at-least-once with the #538 per-tag quarantine and all-or-nothing ack. Do not restructure DrainOnceAsync.
In any Akka actor handler: capture IActorRef self = Self (and Sender) before the first await; never touch Self/Context after an await …ConfigureAwait(false) (#491).
Test fakes for async seams used by actors must complete on a pool thread (await Task.Delay(1).ConfigureAwait(false)), never synchronously and never via Task.Yield().
Every OPC UA status code a new class emits is a named const uint in an internal *StatusCodes class, and that assembly must be visible to tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/StatusCodeParityTests.cs (it scans its bin dir — add a ProjectReference).
Authorization changes need an ALLOW test (a deny-only test passes against a build that never consults the gate).
A regression test must be shown to fail with the production change reverted (falsification) — record the observation in the task report.
The gateway backend's observable behaviour (bytes sent for Good numeric samples, provisioning skips, read paths) does not change.
Commit messages end with Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe. Subagents never push and never dispatch subagents.
macOS zsh: quote globs; tea needs </dev/null.

User decisions (already made):

Approach A (provider seam over the existing five interfaces).
SQLite dropped; SQL Server is the first new backend; QuestDB is a later spec.
"Update" = OPC UA HistoryUpdate exposed to clients (Insert/Replace/Update/DeleteRaw/DeleteAtTime).
Typed values (Double/Int64/Boolean/String/DateTime + Empty), not numeric-only.
Aggregates pushed down per database (T-SQL), launch subset per spec §4.
Flag-off authorization uses a NEW HistoryUpdate role, not WriteOperate.
Gateway config keys stay top-level; new backends get subsections.
Testing target: the docker-dev sql container (otopcua-dev-sql-1, localhost,14330).

Execution model: waves, models, and parallel safety

Model per role

Role	Model	Why
Plan review (before execution)	fable	user-requested
Implementer, mechanical/standard tasks (T1, T4, T5, T7, T8, T9, T11, T12, T13, T14)	sonnet	multi-file but fully specified
Implementer, judgment-heavy tasks (T2 outbox crash boundary, T3 recorder actor, T6 node-manager HistoryUpdate + auth, T10 reads/aggregates)	opus	actor concurrency, auth semantics, SQL/Part 13 semantics
Per-task reviewer	sonnet for sonnet tasks; opus for T2/T3/T6/T10	scale to risk
Fix rounds 4–5 escalation	one tier up (sonnet→opus, opus→fable)	per SDD skill
Final whole-branch review	fable	cross-task integration risk
Live gate (T15)	controller (this session)	needs rig + judgement; never delegated

Always pass model explicitly on every Agent dispatch.

Waves

Wave 1 (serial):   T1  contracts + compile ripple                         [sonnet]
Wave 2 (parallel): T2 outbox v2 [opus] ‖ T3 recorder ‖ T4 gateway provider ‖ T6 server HistoryUpdate ‖ T7 CLI ‖ T8 SQL skeleton+schema
Wave 3 (parallel): T5 registry+Host ‖ T9 SQL writes ‖ T10 SQL reads ‖ T11 SQL updater ‖ T13 over-the-wire test
Wave 4 (serial):   T12 SQL provider + Host registration
Wave 5 (serial):   T14 docs  →  final fable review  →  T15 live gate (controller)

Dependencies: T2,T3,T4,T6,T7,T8 ← T1. T5 ← T4. T9,T10,T11 ← T8. T13 ← T6. T12 ← T5,T9,T10,T11. T14 ← T12,T13. T15 ← T14 + final review.

Parallel-safety rules (binding)

One git worktree per parallel task, created by the controller under .worktrees/ph-T<N> from the integration branch feat/pluggable-historian at the wave's start commit. Run git -C <wt> log -1 and confirm the base SHA in the dispatch (worktrees can start stale — reset first).

File ownership. A task edits only the files its "Files" list names. The shared hot files have exactly one owner per wave:

Shared file	Owner
`ZB.MOM.WW.OtOpcUa.slnx`	T8 (adds both SqlServer projects)
`src/Server/ZB.MOM.WW.OtOpcUa.Host/Program.cs`	T5 (wave 3), then T12 (wave 4)
`.../Host/Configuration/ServerHistorianOptionsValidator.cs`	T5 (wave 3), then T12 (wave 4)
`.../Runtime/Historian/ContinuousHistorizationMetrics.cs`	T5
`.../OpcUaServer/OtOpcUaSdkServer.cs`, `.../Host/OpcUa/OtOpcUaServerHostedService.cs`	T6
`tests/Server/.../Runtime.Tests/Historian/ContinuousHistorizationRecorderTests.cs`	T1 (ripple) then T3
`Core.Abstractions/Historian/IHistorizationOutbox.cs`	T1
`tests/Core/.../Core.Abstractions.Tests/*.csproj`	T8
`src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs`	T6 only
`.../Runtime/Historian/ContinuousHistorizationRecorder.cs`	T1 (ripple) then T3 only
`.../Driver.Historian.SqlServer/*.csproj`	T8 only (T9–T11 add `.cs` files only; csproj uses default globbing + `EmbeddedResource` for `Schema/*.sql` set by T8)

A task that discovers it needs an un-owned shared file stops and reports NEEDS_CONTEXT rather than editing it.

New-file-only parallelism in the SQL project. T9/T10/T11 each create their own files and their own test files; none edits another's. Shared SQL helpers live in T8's SqlHistorianConnection.cs/SampleRowMapper.cs; if a wave-3 task needs a helper change, it adds a private helper in its own file instead.
Merge order at wave end: the controller merges worktree branches into feat/pluggable-historian in ascending task number, running dotnet build ZB.MOM.WW.OtOpcUa.slnx after each merge and the touched test projects after the last. A conflict ⇒ stop and resolve on the integration branch, never inside a worktree.
Integration-test DBs are per-test-run (OtOpcUaHistorian_T_<guid>), so parallel T9/T10/T11 suites against the same sql container cannot collide.
Subagents do not push, merge, or dispatch subagents. Only the controller merges.

Common commands

dotnet build ZB.MOM.WW.OtOpcUa.slnx
dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests
dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests --filter "FullyQualifiedName~Historian"
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests
export HISTORIAN_SQL_TEST_CONNECTION='Server=localhost,14330;User Id=sa;Password=OtOpcUa!Dev123;TrustServerCertificate=True;Encrypt=False'
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests

The rig sql container must be up: docker ps --filter name=otopcua-dev-sql-1. If down: cd docker-dev && docker compose up -d sql.

Task 1: Typed historian contracts + compile ripple

Goal: Introduce HistorianValue, HistorianValueKind, HistoryUpdateMode, IHistorianDataUpdater, IHistorianBackendProvider, change HistorizationValue/HistorizationOutboxEntry to typed+status, and keep the whole solution compiling with no behaviour change.

Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:

Add long UnreadableCount => 0; as a default interface member on IHistorizationOutbox (Core.Abstractions) — T2 overrides it, T5 binds the gauge. (#6)

The record call sites to update are explicit: grep -rn "HistorizationOutboxEntry(\|HistorizationValue(" src tests --include='*.cs' (≈17 sites in 7 files incl. GatewayLiveIntegrationTests.cs, HistorizationOutboxEntrySerializerTests.cs, GatewayHistorianValueWriterTests.cs, ContinuousHistorizationRecorderTests.cs). Every one of them is in T1's Files. Do not stop at an unlisted one. (#3)

Files:

Create: src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/HistorianValue.cs
Create: src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianDataUpdater.cs
Create: src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/IHistorianBackendProvider.cs
Modify: src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianValueWriter.cs (HistorizationValue)
Modify: src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/HistorizationOutboxEntry.cs
Modify: src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorizationOutbox.cs (default member UnreadableCount)
Modify (ripple only): src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationRecorder.cs (lines ~548 entry construction, ~692 drain mapping), src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/GatewayHistorianValueWriter.cs, .../Recorder/HistorizationOutboxEntrySerializer.cs, and every test that constructs these records (find with grep -rn "HistorizationOutboxEntry(\|HistorizationValue(" src tests --include='*.cs').
Test: tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/Historian/HistorianValueTests.cs

Acceptance Criteria:

HistorianValue.TryFromObject implements exactly the spec §3 table; tests cover every row incl. ulong above long.MaxValue ⇒ Double, DateTime local ⇒ UTC ticks, null ⇒ Empty, int[] ⇒ false.
TryGetDouble true for Double/Int64/Boolean only; ToObject round-trips each kind (DateTime returns DateTimeKind.Utc).
Solution builds; all pre-existing historian tests pass unchanged in assertion content (only constructor call sites updated).
Ripple preserves behaviour: recorder still coerces numerics only (via TryFromObject + TryGetDouble, building HistorianValue.FromDouble(numeric)) and still stamps Good (StatusCode = 0); serializer still writes the legacy layout using Value.Number and quality 192; gateway writer maps Value.Number and quality 192.

Verify: dotnet build ZB.MOM.WW.OtOpcUa.slnx && dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests --filter "FullyQualifiedName~HistorianValue" && dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests && dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests --filter "FullyQualifiedName~Historiz" → all green.

Steps:

Step 1: Write HistorianValueTests (failing — type does not exist). Cases:

[Theory]
[InlineData(1.5d, HistorianValueKind.Double)]
[InlineData(1.5f, HistorianValueKind.Double)]
[InlineData(7, HistorianValueKind.Int64)]
[InlineData((short)7, HistorianValueKind.Int64)]
[InlineData((byte)7, HistorianValueKind.Int64)]
[InlineData(7u, HistorianValueKind.Int64)]
[InlineData(7L, HistorianValueKind.Int64)]
[InlineData(true, HistorianValueKind.Boolean)]
[InlineData("x", HistorianValueKind.String)]
public void Maps_clr_values_to_kinds(object input, HistorianValueKind expected)
{
    HistorianValue.TryFromObject(input, out var v).ShouldBeTrue();
    v.Kind.ShouldBe(expected);
}
// plus: decimal 2.5m ⇒ Double 2.5; ulong.MaxValue ⇒ Double; (ulong)5 ⇒ Int64 5;
// null ⇒ Empty (true); new[]{1,2} ⇒ false; new object() ⇒ false;
// DateTime(2026,1,1,0,0,0,Local) ⇒ DateTime kind, Integer == ToUniversalTime().Ticks, ToObject() Kind Utc;
// FromBoolean(true): Integer 1, Number 1, TryGetDouble ⇒ 1; FromString("a").TryGetDouble ⇒ false; Empty.ToObject() ⇒ null.

Step 2: Run, confirm compile failure.
Step 3: Implement HistorianValue.cs:

namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;

/// <summary>The value kinds a historian backend can store. Persisted as a byte (SQL ValueKind, outbox v2) — never renumber.</summary>
public enum HistorianValueKind : byte { Empty = 0, Double = 1, Int64 = 2, Boolean = 3, String = 4, DateTime = 5 }

/// <summary>A typed historized value. Number carries Double (and the double view of Int64/Boolean);
/// Integer carries Int64, Boolean (1/0) and DateTime (UTC ticks); Text carries String.</summary>
public readonly record struct HistorianValue(HistorianValueKind Kind, double Number, long Integer, string? Text)
{
    public static HistorianValue Empty => default;
    public static HistorianValue FromDouble(double v) => new(HistorianValueKind.Double, v, 0, null);
    public static HistorianValue FromInt64(long v) => new(HistorianValueKind.Int64, v, v, null);
    public static HistorianValue FromBoolean(bool v) => new(HistorianValueKind.Boolean, v ? 1d : 0d, v ? 1 : 0, null);
    public static HistorianValue FromString(string v) => new(HistorianValueKind.String, 0d, 0, v ?? throw new ArgumentNullException(nameof(v)));
    public static HistorianValue FromDateTime(DateTime v) =>
        new(HistorianValueKind.DateTime, 0d, (v.Kind == DateTimeKind.Local ? v.ToUniversalTime() : DateTime.SpecifyKind(v, DateTimeKind.Utc)).Ticks, null);

    public static bool TryFromObject(object? value, out HistorianValue result)
    {
        switch (value)
        {
            case null: result = Empty; return true;
            case double d: result = FromDouble(d); return true;
            case float f: result = FromDouble(f); return true;
            case decimal m: result = FromDouble((double)m); return true;
            case sbyte sb: result = FromInt64(sb); return true;
            case byte b: result = FromInt64(b); return true;
            case short s: result = FromInt64(s); return true;
            case ushort us: result = FromInt64(us); return true;
            case int i: result = FromInt64(i); return true;
            case uint ui: result = FromInt64(ui); return true;
            case long l: result = FromInt64(l); return true;
            case ulong ul: result = ul <= long.MaxValue ? FromInt64((long)ul) : FromDouble(ul); return true;
            case bool bo: result = FromBoolean(bo); return true;
            case string str: result = FromString(str); return true;
            case DateTime dt: result = FromDateTime(dt); return true;
            default: result = Empty; return false;   // arrays, references, structures ⇒ not historizable
        }
    }

    public bool TryGetDouble(out double value)
    {
        switch (Kind)
        {
            case HistorianValueKind.Double:
            case HistorianValueKind.Int64:
            case HistorianValueKind.Boolean: value = Number; return true;
            default: value = 0d; return false;
        }
    }

    public object? ToObject() => Kind switch
    {
        HistorianValueKind.Double => Number,
        HistorianValueKind.Int64 => Integer,
        HistorianValueKind.Boolean => Integer != 0,
        HistorianValueKind.String => Text,
        HistorianValueKind.DateTime => new DateTime(Integer, DateTimeKind.Utc),
        _ => null,
    };
}

Step 4: IHistorianDataUpdater.cs — exactly the spec §4 "Updates" interface + HistoryUpdateMode enum, with XML docs naming the per-item status codes.
Step 5: IHistorianBackendProvider.cs in ZB.MOM.WW.OtOpcUa.Runtime.Historian — exactly spec §2.
Step 6: Change the records:

public readonly record struct HistorizationValue(DateTime? TimestampUtc, HistorianValue Value, uint StatusCode);
public sealed record HistorizationOutboxEntry(Guid Id, string Tag, HistorianValue Value, uint StatusCode, DateTime TimestampUtc);

Step 7: Ripple, behaviour-preserving. Recorder line ~527: replace TryCoerceToDouble(msg.Value, out double numeric) with HistorianValue.TryFromObject(msg.Value, out var hv) && hv.TryGetDouble(out double numeric) (same drop semantics — strings/null/DateTime still dropped as non-numeric); entry = new HistorizationOutboxEntry(Guid.NewGuid(), historianName, HistorianValue.FromDouble(numeric), 0u /*Good*/, timestampUtc); delete TryCoerceToDouble. Drain line ~692: new HistorizationValue(e.TimestampUtc, e.Value, e.StatusCode). Serializer: write entry.Value.Number and (ushort)192; read back as FromDouble(value), status 0. Gateway writer: NumericValue = value.Value.Number, Quality = 192. Keep GoodQuality const for now (T3 removes it). Update test call sites.
Step 8: Build + run Verify. Commit: feat(historian): typed HistorianValue contracts, IHistorianDataUpdater, IHistorianBackendProvider (no behaviour change).

{"files":["src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/HistorianValue.cs","src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianDataUpdater.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/IHistorianBackendProvider.cs","src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianValueWriter.cs","src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/HistorizationOutboxEntry.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationRecorder.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/GatewayHistorianValueWriter.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/HistorizationOutboxEntrySerializer.cs"],"acceptanceCriteria":["TryFromObject matches spec table","solution builds, existing historian tests green","no behaviour change"],"modelTier":"standard","model":"sonnet","wave":1}

Task 2: Outbox format v2 with legacy read

Goal: Serialize every HistorianValueKind + uint StatusCode; read pre-upgrade entries; drop (and count) unreadable entries instead of throwing.

Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:

Model: opus (FIFO invariant on the crash boundary). (#1)

FasterLogHistorizationOutbox.RecoverState() (ctor) ALSO deserializes every record. Use TryDeserialize in both RecoverState and PeekBatchAsync. In RecoverState, index an unreadable record in _live/_index under a synthetic Guid.NewGuid() with its address so FIFO truncation can pass it; in Peek skip it (and make sure an unreadable entry at the head is removable so it can't pin the head — e.g. Peek auto-removes leading unreadable entries). UnreadableCount (overrides T1's default member) increments in both. (#1)

Required test: write a raw garbage record straight into the FasterLog directory (open FasterLog directly), restart the outbox, prove the ctor succeeds, the following good entry drains, and the head advances past the garbage. (#1)

Legacy/v2 ambiguity: when the 4-byte magic matches but v2 structural validation fails, fall back to the legacy exact-length test before declaring unreadable. (#18)

Drop the meter bullet: T5 binds the gauge via IHistorizationOutbox.UnreadableCount. (#6)

Files:

Modify: src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/HistorizationOutboxEntrySerializer.cs
Modify: src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/FasterLogHistorizationOutbox.cs (only the deserialize call site in Peek: skip+count unreadable entries; expose long UnreadableCount)
Test: tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests/Recorder/HistorizationOutboxEntrySerializerV2Tests.cs

Acceptance Criteria:

v2 layout exactly: "HV2\0"(4) ver=2(1) id(16) tagLen(i32 LE) tag(UTF-8) kind(1) payload status(u32 LE) ts(i64 LE ToBinary). Payload: Empty none; Double f64; Int64/Boolean/DateTime i64; String i32 len + UTF-8.
Round-trip test per kind incl. empty string, a 70-KB string, NaN, double.NegativeInfinity, long.MinValue, status 0x80310000.
Pinned legacy bytes: a hex literal produced by today's serializer (generate it once from the T1-era Serialize for Id=00000000-0000-0000-0000-000000000001, Tag="t", Value=42.5, Quality=192, ts=2026-01-01T00:00:00Z and paste the bytes into the test) deserializes to Double 42.5, status 0.
TryDeserialize returns false (never throws) for: 3 random bytes, magic with ver=9, magic with truncated payload, legacy-looking bytes whose length disagrees with 38+tagLen. FasterLogHistorizationOutbox skips such an entry on peek, advances past it (it must be removable so it can't pin the head — truncate through its id or log-address as the outbox already does for acked entries), and increments UnreadableCount; a test proves a corrupt entry between two good ones does not block the second.
Meter: ContinuousHistorizationMetrics gets an observable otopcua.historian.outbox.unreadable bound alongside the existing outbox gauges — only if BindOutbox lives in an un-owned file; otherwise report the counter on the outbox and note it for T5. (Owned file check: ContinuousHistorizationMetrics.cs is NOT owned by T2 — expose the property only.)

Verify: dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests --filter "FullyQualifiedName~Outbox" → green; falsification: with the legacy branch deleted the pinned-bytes test fails.

Steps: (1) capture the legacy hex from the current serializer via a throwaway test print, paste into the new test; (2) write failing v2 tests; (3) implement Serialize (v2 always) + TryDeserialize(ReadOnlySpan<byte>, out HistorizationOutboxEntry?) (v2 by magic, else legacy by exact length, else false) and keep a throwing Deserialize wrapper only if other callers need it; (4) wire skip+count in the outbox peek; (5) green; commit feat(historian): outbox format v2 carries typed values + status; legacy entries still read.

{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/HistorizationOutboxEntrySerializer.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/FasterLogHistorizationOutbox.cs"],"acceptanceCriteria":["v2 layout","pinned legacy bytes read","corrupt entry skipped and counted, never pins head"],"modelTier":"frontier","model":"opus","wave":2}

Task 3: Recorder captures typed values with real status, filtered by backend kinds

Goal: The recorder records HistorianValue + msg.StatusCode and drops kinds the backend does not support.

Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:

Add tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Historian/ContinuousHistorizationRecorderTests.cs to Files — lines ~337 and ~1149 assert DroppedNonNumeric and must follow the rename. (#3)

The recorder remarks ("carrying only (TagId, Value, TimestampUtc) and no quality") are stale: DependencyMuxActor.cs:114,150 passes item.StatusCode. Rewrite them. (verified)

Comms-loss watchdog BadNoCommunication stamps are node-only (FanCommsLossToValueNodes never reaches the mux) and are NOT historized — say so in the remarks; only a driver's own Bad publish (usually null value ⇒ Empty) is. (#14)

Files:

Modify: src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationRecorder.cs
Modify: src/Server/ZB.MOM.WW.OtOpcUa.Runtime/ServiceCollectionExtensions.cs only the recorder Props construction (~line 485–520) to pass supportedKinds — resolve IHistorianBackendProvider optionally; absent ⇒ {Double, Int64, Boolean} (today's behaviour).
Test: tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Historian/ContinuousHistorizationRecorderTypedValueTests.cs

Acceptance Criteria:

New ctor/Props parameter IReadOnlySet<HistorianValueKind> supportedKinds (default = {Double, Int64, Boolean} for existing overloads so other tests compile unchanged).
Capture: TryFromObject false ⇒ drop + _droppedUnsupportedKind++; kind ∉ supportedKinds ⇒ same; otherwise entry carries the typed value and msg.StatusCode.
Status field in the recorder status record renamed DroppedNonNumeric → DroppedUnsupportedKind; any OTel instrument name stays unchanged (dashboards). GoodQuality const and its doc paragraph removed; class remarks updated (the mux DOES carry StatusCode — VirtualTagActor.DependencyValueChanged).
Tests (fake writer completes on a pool thread): a Bad (0x80000000) numeric sample is written with status 0x80000000; a string sample is written when kinds include String and dropped+counted when not; null with status BadNoCommunication is written as Empty when kinds include Empty, dropped otherwise; an int[] is always dropped; existing #537 capture-gate and #538 quarantine tests still pass untouched.
Falsification recorded: reverting to the hard-coded Good makes the Bad-status test fail.

Verify: dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests --filter "FullyQualifiedName~ContinuousHistorization" → green.

Steps: failing tests → implement → green → grep for DroppedNonNumeric across src/tests and update readers (AdminUI/metrics) only within Runtime; if a reader exists outside the owned files, report it for T5 instead of editing. Commit fix(historian): record the real status code and typed values; backend-supported kinds gate capture.

{"files":["src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationRecorder.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/ServiceCollectionExtensions.cs"],"acceptanceCriteria":["real status recorded","kind filter","#537/#538 unchanged","pool-thread fake"],"modelTier":"frontier","model":"opus","wave":2}

Task 4: Gateway provider + reverse quality mapping

Goal: GatewayHistorianProvider : IHistorianBackendProvider; the gateway writer maps typed value + status to the gateway's numeric + DA quality.

Files:

Create: src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/GatewayHistorianProvider.cs
Modify: src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Mapping/GatewayQualityMapper.cs (add ToDaQuality(uint))
Modify: src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/GatewayHistorianValueWriter.cs
Test: tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests/GatewayHistorianProviderTests.cs, .../Mapping/GatewayQualityMapperReverseTests.cs

Acceptance Criteria:

Provider: Name = "Gateway", SupportedKinds = {Double, Int64, Boolean}, CreateUpdater returns null; the four Create* delegate to the existing GatewayHistorian.Create… (data source/alarm writer/provisioner) and build GatewayHistorianValueWriter exactly as Program.cs does today (HistorianGatewayClientAdapter.Create(options, loggerFactory)). Options come from the ctor (ServerHistorianOptions).
ToDaQuality(Map(q)) == q for every explicit byte in Map's table; unknown status falls back by severity bits (0x00…⇒192, 0x40…⇒64, 0x80…⇒0).
Writer: a value with TryGetDouble false is skipped (debug log, no throw); Good Double 42.5 produces the same HistorianLiveValue (NumericValue=42.5, Quality=192) as before T1 — asserted in a test.

Verify: dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests → green.

Steps: tests first → implement → green → commit feat(historian): GatewayHistorianProvider; status→DA quality reverse mapping.

{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/GatewayHistorianProvider.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Mapping/GatewayQualityMapper.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/GatewayHistorianValueWriter.cs"],"acceptanceCriteria":["provider shape","reverse mapping inverse","Good bytes unchanged"],"modelTier":"standard","model":"sonnet","wave":2}

Task 5: Backend registry, config, validators, Host wiring

Goal: ServerHistorian:Backend selects a provider; Program.cs builds every historian seam through it.

Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:

Gate all three gateway checks in ServerHistorianOptionsValidator (Endpoint, ApiKey, UseTls⇔scheme) on Backend == Gateway, and the same in ServerHistorianOptions.Validate() warnings. Tests: Backend=SqlServer, Enabled=true, no Endpoint/ApiKey ⇒ passes; control Backend=Gateway, no ApiKey ⇒ fails (unchanged). (#2)

Unknown Backend ⇒ always fail (no I/O). Backend-specific option validation (SqlServer, added by T12) runs inside ServerHistorianOptionsValidator and only when consumed (ServerHistorian:Enabled || AlarmHistorian:Enabled) — the existing consumer-gated posture. HistorianBackendRegistry.Resolve must not validate. (#11)

Register HistorianUpdaterHolder.None unconditionally in the driver block; register the real new HistorianUpdaterHolder(provider.CreateUpdater(sp)) only when ServerHistorian:Enabled (HistoryUpdate gated like HistoryRead; last registration wins). (#11)

Add src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationMetrics.cs to Files: add observable gauge otopcua.historian.outbox.unreadable from IHistorizationOutbox.UnreadableCount in BindOutbox. (#6)

Program.cs is top-level statements and not unit-testable: the "DI graph unchanged" AC is tested as HistorianBackendRegistry.Resolve(gatewayOpts, cfg).CreateDataSource(sp) is GatewayHistorianDataSource (and likewise for the writer/alarm writer/provisioner). Do not try to host-start Program. (#22)

Spec wording fix: there are three GatewayHistorian.Create… call sites (Program.cs ~241/262/273) plus the inline writer (~313). (#16)

Files:

Create: src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs
Modify: src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ServerHistorianOptions.cs (add Backend string, default "Gateway")
Modify: src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/ServerHistorianOptionsValidator.cs
Modify: src/Server/ZB.MOM.WW.OtOpcUa.Host/Program.cs (lines ~215–318 only)
Test: tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/Configuration/HistorianBackendRegistryTests.cs, extend ServerHistorianOptionsValidatorTests

Acceptance Criteria:

Registry: static IHistorianBackendProvider Resolve(ServerHistorianOptions o, IConfiguration cfg); knows "Gateway" now (T12 adds "SqlServer"); case-insensitive; unknown ⇒ OptionsValidationException naming the value and listing valid names. The registry also exposes IReadOnlyCollection<string> KnownBackends used by the validator.
Validator: unknown backend fails; Endpoint required only when backend is Gateway; existing tests unchanged in intent.
Program.cs: resolve the provider once; register it as a singleton IHistorianBackendProvider; replace the 4 GatewayHistorian.Create… lambdas and the inline GatewayHistorianValueWriter with provider.Create…(sp); register IHistorianDataUpdater via provider.CreateUpdater(sp) only when non-null (TryAddSingleton-style factory is not possible for a null — register conditionally by calling it once at registration time is NOT allowed (needs sp); instead register a small HistorianUpdaterHolder(IHistorianDataUpdater? Updater) singleton, and have T6's hosted-service hook read holder.Updater). The alarm-only warning text names serverHistorianOptions.Backend.
Bind the outbox unreadable gauge from T2 in ContinuousHistorizationMetrics.BindOutbox if T2 reported it (this task owns that file this wave).
With Backend absent the Host's DI graph resolves the same concrete types as before (test: build the service collection with a Gateway config and assert IHistorianDataSource resolves to GatewayHistorianDataSource).

Verify: dotnet build ZB.MOM.WW.OtOpcUa.slnx && dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests --filter "FullyQualifiedName~Historian" → green.

{"files":["src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationMetrics.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ServerHistorianOptions.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/ServerHistorianOptionsValidator.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/Program.cs"],"acceptanceCriteria":["unknown backend fails boot","Gateway default resolves identical types","updater holder registered"],"modelTier":"standard","model":"sonnet","wave":3}

Task 6: HistoryUpdate service in the node manager + authorization

Goal: Clients can HistoryUpdate historized nodes when the backend supplies an updater, gated by ACL (flag on) or the new HistoryUpdate role (flag off).

Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:

SDK pinned OPCFoundation.NetStandard.Opc.Ua.* 1.5.378.106. Verified: public HistoryUpdate(OperationContext, Type detailsType, …) runs under Lock, pre-seeds errors[ii] = BadHistoryOperationUnsupported, and admits a variable only if (AccessLevel & AccessLevels.HistoryWrite) != 0; it releases the lock and dispatches to protected HistoryUpdateData / HistoryDeleteRawModified / HistoryDeleteAtTime(ServerSystemContext, IList<…Details>, IList<HistoryUpdateResult>, IList<ServiceResult>, List<NodeHandle>, IDictionary<NodeId,NodeState>). The HistoryWrite bit is therefore dispatch-load-bearing. (#4)

Add src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaSdkServer.cs to Files: SetHistorianDataUpdater(IHistorianDataUpdater?) mirroring SetHistorianDataSource (line ~122). In OtOpcUaServerHostedService.StartAsync assign it immediately after the NodeManager is null guard (~line 164), before _deferredSink.SetSink (~171); clear it in StopAsync next to the data-source clear (~373). Hosted service is registered before AddAkka, so this precedes every deploy apply. (#4, #5)

On success each arm sets errors[handle.Index] = ServiceResult.Good (the base pre-seeds Unsupported; every HistoryRead success path here does the same, e.g. OtOpcUaNodeManager.cs ~5133). Per-node HistoryUpdateResult.StatusCode stays Good when per-item results are mixed; only a whole-node failure (deny / unsupported / not historized) sets the node code. (#13, #21)

UpdateDataDetails.UpdateValues[i].SourceTimestamp is non-nullable: == DateTime.MinValue ⇒ BadInvalidTimestamp for that item. (#21)

At least one test MUST drive the public nm.HistoryUpdate(OperationContext, typeof(UpdateDataDetails), …) entry point and assert the updater is called — it is the test that fails when the AccessLevel bit is missing. Tests that call the protected arm directly do not count toward the ALLOW criterion. (#4)

AclEdit: just add NodePermissions.HistoryUpdate to the PermissionBits array (~161–166); the page renders enum names, there is no label table. (#23)

Files:

Modify: src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs
Modify: src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Security/OpcUaDataPlaneRoles.cs (HistoryUpdate = "HistoryUpdate")
Modify: src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/ApiKeyInterimRoleMapper.cs
Modify: src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs (assign HistorianDataUpdater from an optional HistorianUpdaterHolder — define HistorianUpdaterHolder in Runtime/Historian/HistorianUpdaterHolder.cs in THIS task: public sealed record HistorianUpdaterHolder(IHistorianDataUpdater? Updater) { public static readonly HistorianUpdaterHolder None = new((IHistorianDataUpdater?)null); }, resolve with GetService ⇒ None when absent)
Modify: src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Pages/Clusters/AclEdit.razor (add NodePermissions.HistoryUpdate to the list at ~line 163–166 with label "History update")
Modify: src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Enums/NodePermissions.cs comment only (remove "until the HistoryUpdate service surface is implemented"; still in no bundle)
Test: tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/History/HistoryUpdateTests.cs, tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/OpcUa/ApiKeyInterimRoleMapperTests.cs (extend)

Acceptance Criteria:

Before writing, read the pinned SDK's CustomNodeManager2 and confirm the exact protected signatures for HistoryUpdateData, HistoryDeleteRawModified, HistoryDeleteAtTime (and how HistoryUpdate dispatches by details type); record them in the report. Use DeepWiki (OPCFoundation/UA-.NETStandard) if the source is not on disk.
public IHistorianDataUpdater? HistorianDataUpdater { get; set; } (volatile backing field, like HistorianDataSource).
UpdateDataDetails: PerformUpdateType.Insert/Replace/Update ⇒ HistoryUpdateMode; Remove ⇒ BadHistoryOperationUnsupported. Values: DataValue ⇒ HistorizationValue(SourceTimestamp, TryFromObject(Value.Value) or BadTypeMismatch item, StatusCode.Code). Per-item results returned in OperationResults in order.
DeleteRawModifiedDetails with IsDeleteModified=false ⇒ DeleteRawAsync; true ⇒ unsupported. DeleteAtTimeDetails ⇒ DeleteAtTimeAsync.
Not historized / no updater ⇒ BadHistoryOperationUnsupported. Event/structure details ⇒ unsupported.
Authorization helper AuthorizeHistoryUpdate mirroring AuthorizeHistoryRead + the WriteOperate pattern at ~line 2024–2033: flag on ⇒ _authorizer.Authorize(identity, OpcUaOperation.HistoryUpdate, nodeId, realm); flag off ⇒ AuthorizeKeyUsable(...) AND role HistoryUpdate. Deny ⇒ BadUserAccessDenied.
Batch limiter + deadline shared with HistoryRead (WithHistoryReadBatchLimiter, CreateDeadline).
ComposeAccessLevel(writable, historized, historyWritable) ORs AccessLevels.HistoryWrite when historized && historyWritable; historyWritable = HistorianDataUpdater is not null at materialise time. Raw + UNS mirrors both get it. (Setting the updater after nodes exist does not retro-update AccessLevel — document; the hosted service assigns it before the first deploy applies, verify the ordering and state it.)
ApiKeyInterimRoleMapper adds OpcUaDataPlaneRoles.HistoryUpdate for a Cluster-scoped grant with NodePermissions.HistoryUpdate; a WriteOperate-only grant does NOT confer it.
Tests (fake updater + fake authorizer, following the existing node-manager test harness for HistoryRead): ALLOW flag-on with no roles + scoped grant ⇒ updater called and GoodEntryInserted returned; flag-off with HistoryUpdate role ⇒ allowed; flag-off with only WriteOperate ⇒ BadUserAccessDenied and updater NOT called; expired key ⇒ denied; no updater ⇒ unsupported; per-item codes passed through in order; UNS NodeId resolves to the same tagname as raw.
Falsification recorded for the ALLOW test (break the flag-on branch ⇒ it fails).

Verify: dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests --filter "FullyQualifiedName~HistoryUpdate" && dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests --filter "FullyQualifiedName~ApiKeyInterimRoleMapper" → green.

{"files":["src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs","src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaSdkServer.cs","src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Security/OpcUaDataPlaneRoles.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/ApiKeyInterimRoleMapper.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/HistorianUpdaterHolder.cs","src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Pages/Clusters/AclEdit.razor","src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Enums/NodePermissions.cs"],"acceptanceCriteria":["SDK signatures verified","ALLOW test","WriteOperate does not confer HistoryUpdate","HistoryWrite access bit"],"modelTier":"frontier","model":"opus","wave":2}

Note: T5 references HistorianUpdaterHolder — T5 runs in wave 3 after T6 merges, so the type exists.

Task 7: Client.CLI `historyupdate` command

Goal: A CLI command to drive HistoryUpdate for manual + live-gate testing.

Files:

Create: src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI/Commands/HistoryUpdateCommand.cs (follow the existing historyread command's structure, options parsing, session/auth handling exactly)
Modify: docs/Client.CLI.md (new section)
Test: the Client.CLI test project if one exists for commands (ls tests/Client); follow its pattern for argument parsing tests.

Acceptance Criteria:

Options per spec §6; --mode values insert|replace|update|delete-raw|delete-at-time; --value parsed against --type double|int64|bool|string|datetime (default double); repeated --time/--value pairs; validation errors exit non-zero with a message.
Sends UpdateDataDetails / DeleteRawModifiedDetails(IsDeleteModified=false) / DeleteAtTimeDetails via Session.HistoryUpdate; prints index time StatusCodeName per item plus the overall service result.
Argument-parsing unit tests for each mode (no server needed).

Verify: dotnet build src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI && dotnet test tests/Client --filter "FullyQualifiedName~HistoryUpdate" → green.

{"files":["src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI/Commands/HistoryUpdateCommand.cs","docs/Client.CLI.md"],"acceptanceCriteria":["all five modes","arg parsing tests"],"modelTier":"standard","model":"sonnet","wave":2}

Task 8: SQL Server backend project skeleton, schema, bootstrapper, test fixture

Goal: New projects, embedded DDL, idempotent locked bootstrap, options + validation, shared connection/mapping helpers, and the env-gated integration fixture.

Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:

Move SqlServerTagCatalog.cs here from T9 (get-or-create + read-only lookup), used by T9/T10/T11/T12. Create path: MERGE {s}.Tag WITH (HOLDLOCK) AS t …; on 2627/2601 re-select the TagId. Integration test: two concurrent get-or-create on the same new name ⇒ both return the same id, one row. (#10, #12)

Timestamps: helper AddDateTime2(SqlCommand, name, DateTime) (SqlDbType.DateTime2, scale 7) for every scalar param; TVPs are IEnumerable<SqlDataRecord> with SqlMetaData(…, SqlDbType.DateTime2, 0, 7) — never a DataTable (infers 3.33 ms datetime). Mapper tests use non-millisecond-aligned ticks (…T00:00:00.1234567Z). (#7)

DDL: CREATE SCHEMA must be alone in its batch ⇒ IF SCHEMA_ID(N'$(Schema)') IS NULL EXEC(N'CREATE SCHEMA [$(Schema)]'); substitute $(Schema) before splitting on GO. (#17)

Event: EventId nvarchar(64) NOT NULL CONSTRAINT PK_Event PRIMARY KEY NONCLUSTERED, plus CREATE CLUSTERED INDEX CX_Event_Time ON {s}.Event (EventTimeUtc, EventId) (drop the separate IX_Event_Time). Keep IX_Event_Source_Time. (#19)

Private helper types introduced by wave-3 tasks inside this project must be prefixed with their task (T9…, T10…, T11…) to avoid merge collisions. (#12)

Verified: StatusCodes.* names in SqlHistorianStatusCodes all exist in the pinned SDK, and AggregateBits Calculated=0x1 / Interpolated=0x2 with InfoType 0x400 are correct; keep the SDK-oracle unit test. (#26)

Files:

Create: src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.csproj (refs: Core.Abstractions, Core.AlarmHistorian, Runtime; <PackageReference Include="Microsoft.Data.SqlClient" />, Microsoft.Extensions.Logging.Abstractions; <EmbeddedResource Include="Schema\*.sql" />; InternalsVisibleTo both test projects)
Create: .../SqlServerHistorianOptions.cs (+ Validate() returning errors per spec §2), .../SqlHistorianStatusCodes.cs, .../SqlHistorianConnection.cs (open connection, schema-qualify helper Q(string obj) ⇒ [{schema}].[{obj}], transient-error classifier: 1205, -2, 40613, 40197, 40501, 49918, 4060, 233, 10053, 10054, 10060, 11001, SqlException with IsTransient), .../SampleRowMapper.cs (HistorianValue ⇄ columns per spec §4 + DataTable builders for SampleRow and TimeRow TVPs), .../SqlServerSchemaBootstrapper.cs, .../Schema/001_initial.sql
Create: docs/historian/sqlserver/001_initial.sql (copy of the embedded script)
Create: tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.Tests/ (unit) and tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/ with SqlHistorianFixture.cs
Modify: ZB.MOM.WW.OtOpcUa.slnx (4 new projects), tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj (ProjectReference to the new src project so StatusCodeParityTests scans SqlHistorianStatusCodes)

Acceptance Criteria:

001_initial.sql is the spec §4 DDL with {s} replaced by a $(Schema) token that the bootstrapper substitutes after validating the identifier regex; every statement guarded (IF SCHEMA_ID(...) IS NULL, IF OBJECT_ID(...) IS NULL, IF TYPE_ID(...) IS NULL) so a re-run is a no-op; ends with INSERT SchemaVersion(1, SYSUTCDATETIME()) guarded by IF NOT EXISTS. Split on ^GO$ lines.
SqlServerSchemaBootstrapper.EnsureAsync(ct): lazy, once per instance (a SemaphoreSlim + _ready flag; failure NOT latched), sp_getapplock exclusive session lock with a 30 s timeout around the apply; AutoCreateSchema=false ⇒ only checks MAX(Version) >= 1 and throws HistorianSchemaMissingException("…missing version 1…").
SqlHistorianStatusCodes contains every code the spec names (Good, GoodNoData, GoodEntryInserted, GoodEntryReplaced, BadNoData, BadEntryExists, BadNoEntryExists, BadInvalidTimestamp, BadTypeMismatch, BadAggregateInvalidInputs, BadHistoryOperationUnsupported, UncertainDataSubNormal) as named const uint with values copied from Opc.Ua.StatusCodes — the parity test passes and lists this class.
Historian info-bit constants (InfoTypeDataValue = 0x00000400, HistorianCalculated = 0x1, HistorianInterpolated = 0x2) in a separate HistorianInfoBits static class, with a unit test asserting new Opc.Ua.StatusCode(Good).SetAggregateBits(AggregateBits.Calculated).Code == (Good | InfoTypeDataValue | HistorianCalculated) (adjust to the SDK's actual helper — the test must compare against the SDK, not against this plan).
SqlHistorianFixture (xUnit IAsyncLifetime, collection fixture): reads HISTORIAN_SQL_TEST_CONNECTION; absent ⇒ SkipReason and tests Assert.Skip(SkipReason) (follow tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.IntegrationTests/SqlPollServerFixture.cs); present ⇒ CREATE DATABASE OtOpcUaHistorian_T_<guid:N>, exposes ConnectionString for it, DROP DATABASE (after ALTER … SET SINGLE_USER WITH ROLLBACK IMMEDIATE) on dispose.
Integration tests: bootstrap twice ⇒ second is a no-op and SchemaVersion has one row; two bootstrappers concurrently ⇒ both succeed, one row; AutoCreateSchema=false on an empty DB ⇒ HistorianSchemaMissingException; invalid schema name rejected by Validate() before any SQL runs (unit).
SampleRowMapper unit tests: every kind round-trips through the column triple.

Verify: dotnet build ZB.MOM.WW.OtOpcUa.slnx && dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.Tests && dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests && dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests --filter "FullyQualifiedName~StatusCodeParity" → green (integration with the env var set; also run once WITHOUT it and confirm all skip).

{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerTagCatalog.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/","tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.Tests/","tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/","ZB.MOM.WW.OtOpcUa.slnx","docs/historian/sqlserver/001_initial.sql"],"acceptanceCriteria":["idempotent DDL","applock bootstrap","parity-guarded status consts","env-gated throwaway DB fixture"],"modelTier":"standard","model":"sonnet","wave":2}

Task 9: SQL Server writes — value writer, provisioning, alarm writer

Goal: Live values, tag provisioning and alarm events persist idempotently.

Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:

SqlServerTagCatalog now comes from T8 — do NOT create it; use it. (#12)

Insert-if-absent: catch 2627/2601 and treat as success (the row is durably present), so a concurrent writer never books a #538 failure. (#25)

Provisioning MERGE … WITH (HOLDLOCK); integration test with two concurrent provisioners of the same new tags ⇒ both report Ensured, no Failed. (#10)

EventId: SHA-256 over length-prefixed UTF-8 fields EquipmentPath, AlarmId, EventKind, TimestampUtc.Ticks, User, Comment, Message (null ⇒ length −1). Unit test that "a|b"+"c" and "a"+"b|c" hash differently. (#20)

All timestamp round-trips use non-ms-aligned ticks and assert tick equality. (#7)

Files (create only):

.../SqlServerHistorianValueWriter.cs, .../SqlServerTagProvisioner.cs, .../SqlServerAlarmHistorianWriter.cs
Tests: tests/.../SqlServer.IntegrationTests/WriteTests.cs, ProvisioningTests.cs, AlarmWriterTests.cs; unit tests/.../SqlServer.Tests/AlarmEventIdTests.cs

Acceptance Criteria:

Every public entry point calls bootstrapper.EnsureAsync first.
Value writer: TVP insert-if-absent (INSERT INTO Sample(...) SELECT @tagId, r.* , SYSUTCDATETIME() FROM @rows r WHERE NOT EXISTS (SELECT 1 FROM Sample s WITH (UPDLOCK, HOLDLOCK) WHERE s.TagId=@tagId AND s.SourceTimeUtc=r.SourceTimeUtc)); within-batch duplicate timestamps collapse to the first (de-dup in C# before building the TVP — TVP PK would reject them); null timestamp ⇒ DateTime.UtcNow; returns false (never throws) on SQL errors; logs category only.
Provisioning: one MERGE per batch upserting Name/DataType/EU/Description; returns (Requested, Ensured, Skipped:0, Failed); never throws.
Alarm writer: EventId = lowercase hex SHA-256 of AlarmId|EventKind|TimestampUtc.Ticks|User|Comment|Message (UTF-8, | separator, null ⇒ empty); insert-if-absent; per-event outcome per spec (transient classifier from T8).
Integration: each kind written then read back by direct SQL equals input; writing the same batch twice leaves row count unchanged and returns true; a 10 000-value batch completes < 5 s on the rig; provisioning upsert changes DataType on re-run; same alarm event twice ⇒ one row, both Ack; an over-long AlarmTypeName (> 100) ⇒ PermanentFail.

Verify: dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests --filter "FullyQualifiedName~Write|FullyQualifiedName~Provisioning|FullyQualifiedName~AlarmWriter" → green.

{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerHistorianValueWriter.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerTagProvisioner.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerAlarmHistorianWriter.cs"],"acceptanceCriteria":["idempotent redelivery","never throws","deterministic EventId"],"modelTier":"standard","model":"sonnet","wave":3}

Task 10: SQL Server reads — raw, at-time, events, processed (push-down)

Goal: SqlServerHistorianDataSource : IHistorianDataSource per spec §4 "Reads".

Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:

maxValuesPerNode == 0 is the node manager's unbounded sentinel ⇒ TOP(2147483647); test it. (#8)

Events: TOP(@maxEvents + 1) (maxEvents 0 ⇒ unbounded); when the extra row is present drop it and return a non-null opaque ContinuationPoint (e.g. UTF-8 of EventTimeUtc.Ticks|EventId) — the interface says MUST. Test maxEvents=2 over 3 rows. (#9)

Processed: cap the bucket count at 100 000; above it throw ArgumentOutOfRangeException and document that the node manager surfaces it as a per-node Bad. Check what the node manager's processed path does with a thrown non-aggregate exception and match it. (#24)

Tag lookup uses T8's SqlServerTagCatalog read-only lookup. All timestamp tests non-ms-aligned. (#7, #12)

Files (create only): .../SqlServerHistorianDataSource.cs, .../SqlAggregateQueries.cs (one method per aggregate family returning the SQL text), tests tests/.../SqlServer.IntegrationTests/RawReadTests.cs, AtTimeReadTests.cs, EventReadTests.cs, ProcessedReadTests.cs.

Acceptance Criteria:

Tag lookup is read-only (SELECT TagId FROM Tag WHERE Name=@n, cached on hit only) — reads never create tags.
Raw forward/reverse/limit exactly per spec; DataValueSnapshot built with Value.ToObject(), StatusCode, source + server timestamps (inspect DataValueSnapshot's ctor first).
AtTime: single statement with TimeRow TVP and two OUTER APPLY (SELECT TOP 1 …); interpolation/stepping/no-prior/bad-neighbour rules per spec; status built from SqlHistorianStatusCodes + HistorianInfoBits.
Processed: generates every interval in [start,end) (empty ⇒ BadNoData), the aggregate table in spec §4, Calculated bit on every result, Uncertain when any Bad present, BadAggregateInvalidInputs for numeric aggregates on String/DateTime tags, HistoryAggregateNotSupportedException for the rest. interval <= 0 ⇒ single interval [start,end).
Reverse processed (end < start) ⇒ swap and return descending (document).
GetHealthSnapshot() tracks totals/consecutive failures/last error (the bootstrap error surfaces here).
Integration fixture data: tag A with samples at t0+0s=10, +10s=20, +20s=Bad(30), +30s=40, +40s=50 (Double); tag S strings; B booleans. Hand-computed expectations for each supported aggregate over 20 s intervals from t0 — write them as literal expected values in the test (e.g. interval [t0,t0+20s): Average 15, Min 10, Max 20, Total 30, Count 2, Start 10, End 20, Delta 10, Range 10, PercentGood 100; interval [t0+20s,t0+40s): usable {40}, Average 40 with Uncertain status, PercentBad 50).
Falsification: drop the Bad-sample filter ⇒ the Uncertain interval test fails.

Verify: dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests --filter "FullyQualifiedName~Read" → green.

{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerHistorianDataSource.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlAggregateQueries.cs"],"acceptanceCriteria":["raw fwd/rev","AtTime interp/step","aggregate table with hand-computed literals","unsupported aggregates throw"],"modelTier":"frontier","model":"opus","wave":3}

Task 11: SQL Server updater

Goal: SqlServerHistorianDataUpdater : IHistorianDataUpdater per spec §4 "Updates".

Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:

Use T8's SqlServerTagCatalog for create-on-Insert/Update (remove the "private MERGE" fallback). Sample MERGE … WITH (HOLDLOCK). (#10, #12)

"Null timestamp" means DateTime.MinValue (the node manager maps it to BadInvalidTimestamp before calling, but the updater must still treat MinValue the same defensively). (#21)

Non-ms-aligned ticks in every Replace/DeleteAtTime test. (#7)

Files (create only): .../SqlServerHistorianDataUpdater.cs; tests tests/.../SqlServer.IntegrationTests/UpdateTests.cs.

Acceptance Criteria:

One transaction per call; MERGE Sample AS t USING @rows AS r ON t.TagId=@tagId AND t.SourceTimeUtc=r.SourceTimeUtc with OUTPUT $action, r.SourceTimeUtc; Insert uses WHEN NOT MATCHED THEN INSERT only (matched ⇒ BadEntryExists), Replace WHEN MATCHED THEN UPDATE only (not matched ⇒ BadNoEntryExists), Update both. Results mapped back to the caller's original order (including within-request duplicate timestamps: second occurrence ⇒ BadEntryExists for Insert, last-wins for Replace/Update — document).
Null timestamp ⇒ BadInvalidTimestamp for that item, others proceed.
Delete raw [start,end) ⇒ Good with rows deleted, BadNoData when 0; DeleteAtTime per time.
Unknown tag: Insert/Update create it via T8's SqlServerTagCatalog; Replace/Delete ⇒ BadNoEntryExists/BadNoData.
Integration test per row of the spec's result table + the duplicate/null cases; replaced value is visible via direct SQL with the new status.

Verify: dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests --filter "FullyQualifiedName~Update" → green.

{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerHistorianDataUpdater.cs"],"acceptanceCriteria":["Part 11 result codes","order preserved","single transaction"],"modelTier":"standard","model":"sonnet","wave":3}

Task 12: SQL Server provider + Host registration

Goal: Backend=SqlServer works end to end in the Host.

Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:

SqlServer option validation lives in ServerHistorianOptionsValidator, consumer-gated (see T5 amendment) — add it there (this task owns that file in wave 4), not in Resolve. (#11)

Add src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/ServerHistorianOptionsValidator.cs to Files. One shared SqlServerSchemaBootstrapper + SqlServerTagCatalog per provider. (#11, #12)

Files:

Create: src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerHistorianProvider.cs (one shared bootstrapper + tag catalog instance across the five seams; SupportedKinds = all six kinds)
Modify: src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs (add "SqlServer", binding ServerHistorian:SqlServer to SqlServerHistorianOptions and failing on its Validate() errors), src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj (ProjectReference), Program.cs only if T5 left a seam that needs it
Test: tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/Configuration/HistorianBackendRegistryTests.cs (extend), SqlServer.Tests SqlServerHistorianProviderTests.cs

Acceptance Criteria:

Backend=SqlServer with no ConnectionString ⇒ host-start validation failure naming ServerHistorian:SqlServer:ConnectionString.
DI: IHistorianDataSource ⇒ SqlServerHistorianDataSource, IHistorianValueWriter ⇒ SqlServer writer, HistorianUpdaterHolder.Updater non-null, recorder supportedKinds contains String and Empty.
The Endpoint requirement does not fire for SqlServer.

Verify: dotnet build ZB.MOM.WW.OtOpcUa.slnx && dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests --filter "FullyQualifiedName~Historian" → green.

{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerHistorianProvider.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj"],"acceptanceCriteria":["validation names the key","DI resolves SqlServer types","updater present"],"modelTier":"standard","model":"sonnet","wave":4}

Task 13: Over-the-wire HistoryUpdate test

Goal: Prove HistoryUpdate works through a real SDK client session (the encoding/dispatch the node-manager unit tests mock around).

Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:

Install the fake updater (Nm.HistorianDataUpdater = fake, same slot the rig uses for Nm.Authorizer, cf. AclKeyExpiryEnforcementTests.cs:141-215) before materialising the historized node. Add a control test that materialises first, then installs the updater, and proves HistoryUpdate returns BadHistoryOperationUnsupported (documents the no-retro-update rule). Assert the per-node StatusCode as well as OperationResults. (#4, #13, #15)

Files (create only): tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/HistoryUpdateOverTheWireTests.cs (use the existing integration harness that ConditionMethodEventTests uses; inject an in-memory fake IHistorianDataUpdater + IHistorianDataSource sharing one dictionary).

Acceptance Criteria:

Client HistoryUpdate Insert of 2 values on a historized node ⇒ GoodEntryInserted ×2; HistoryRead raw returns them; Insert again ⇒ BadEntryExists; DeleteAtTime one ⇒ read returns one.
A session without the grant ⇒ BadUserAccessDenied, fake not called.
The node's AccessLevel read over the wire includes HistoryWrite.

Verify: dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests --filter "FullyQualifiedName~HistoryUpdateOverTheWire" → green.

{"files":["tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/HistoryUpdateOverTheWireTests.cs"],"acceptanceCriteria":["insert/read/duplicate/delete over the wire","deny case","HistoryWrite bit visible"],"modelTier":"standard","model":"sonnet","wave":3}

Task 14: Documentation

Goal: Docs match the shipped behaviour.

Files: docs/Historian.md, docs/Configuration.md, docs/security.md, CLAUDE.md (Historian / HistoryRead section: add backend selection, SQL Server, HistoryUpdate, the real-status fix, and the gateway C-002 quality limitation; keep it a short subsection), docs/drivers/README or index if historians are listed there.

Acceptance Criteria:

Every config key in spec §2 documented with default and validation rule.
Aggregate table + "simple per-sample definitions, not full Part 13" caveat.
HistoryUpdate: operations, result codes, authorization (flag on/off), AccessLevel bit, unsupported cases.
Docker-dev recipe for a SQL historian (OtOpcUaHistorian on the sql container) and the integration-test env var.
No claim the docs make is untested — cross-check each against the merged code.

Verify: controller reads the diff; grep -n "Backend" docs/Configuration.md.

{"files":["docs/Historian.md","docs/Configuration.md","docs/security.md","CLAUDE.md"],"acceptanceCriteria":["keys documented","aggregate caveat","HistoryUpdate auth documented"],"modelTier":"standard","model":"sonnet","wave":5}

Final review (fable), then Task 15: Live gate on docker-dev (controller)

Final review: dispatch a fable reviewer over git diff master...feat/pluggable-historian with the spec; it must specifically check #491 (Self after await), #537/#538 unchanged, every new status literal is a named const, ALLOW tests exist, no Directory.Packages.props edit, gateway Good-bytes unchanged. One fix dispatch (opus) + scoped re-review (sonnet), then adjudicate.

Task 15 — Live gate (controller only; not delegated). USER-ORDERED GATE — NON-SKIPPABLE. Close only after each item below is re-validated with captured output.

CREATE DATABASE OtOpcUaHistorian on otopcua-dev-sql-1; set on central-1 and central-2 (compose env): ServerHistorian__Enabled=true, ServerHistorian__Backend=SqlServer, ServerHistorian__SqlServer__ConnectionString=Server=sql,1433;Database=OtOpcUaHistorian;User Id=sa;Password=OtOpcUa!Dev123;TrustServerCertificate=True, ContinuousHistorization__Enabled=true + OutboxPath. Rebuild images (the rig image goes stale), up -d.
Mark a rig tag historized (one numeric, one string) and deploy. Evidence: hist.Sample row count advances by exactly one per published value (Primary-only), status column real.
Client.CLI historyread raw + processed (Average, 10 s) on the raw NodeId AND its UNS NodeId — same rows.
Client.CLI historyupdate insert (new timestamp) ⇒ GoodEntryInserted; repeat ⇒ BadEntryExists; replace ⇒ GoodEntryReplaced; delete-at-time ⇒ Good; each confirmed by historyread and by direct SQL. Use an API key whose principal has a Cluster-scoped HistoryUpdate grant; a key with only WriteOperate ⇒ BadUserAccessDenied.
Stop the Primary container; the Secondary resumes recording into the same table; no duplicate (TagId, SourceTimeUtc) (guaranteed by PK — also check count growth rate stays 1×); restart.
Stop the driver's device (or blackhole) while the driver is still publishing (before #527 trips Reconnecting) ⇒ a row with ValueKind = 0 (Empty) and the driver's Bad code (Modbus: BadCommunicationError 0x80050000). Watchdog BadNoCommunication stamps are node-only and are NOT historized — do not expect them.
Record evidence in docs/plans/2026-10-09-pluggable-historian-live-gate.md; restore the rig config (Backend off) unless the user wants it kept.

{"files":["docs/plans/2026-10-09-pluggable-historian-live-gate.md"],"acceptanceCriteria":["Primary-only one row per value","historyread raw+processed via raw and UNS NodeIds","historyupdate insert/duplicate/replace/delete codes","WriteOperate-only key denied","failover continues recording","Bad sample stored Bad"],"userGate":true,"tags":["user-gate"],"verifyCommand":"Client.CLI historyread/historyupdate against opc.tcp://localhost:4840 + sqlcmd on otopcua-dev-sql-1","modelTier":"frontier","model":"controller","wave":5}

Finish

Merge feat/pluggable-historian → master only with the user's go-ahead; push only with approval. Then update the scadaproj sister-project-history/otopcua.md (on scadaproj main, pushed) and the index row only if the summary changed (driver/backends list). File Gitea follow-ups: QuestDB backend spec; retention/purge; Part 11 modified values; time-weighted aggregates.

66 KiB Raw Blame History Unescape Escape

Pluggable Historian Backends Implementation Plan

Global Constraints

Execution model: waves, models, and parallel safety

Model per role

Waves

Parallel-safety rules (binding)

Common commands

Task 1: Typed historian contracts + compile ripple

Task 2: Outbox format v2 with legacy read

Task 3: Recorder captures typed values with real status, filtered by backend kinds

Task 4: Gateway provider + reverse quality mapping

Task 5: Backend registry, config, validators, Host wiring

Task 6: HistoryUpdate service in the node manager + authorization

Task 7: Client.CLI historyupdate command

Task 8: SQL Server backend project skeleton, schema, bootstrapper, test fixture

Task 9: SQL Server writes — value writer, provisioning, alarm writer

Task 10: SQL Server reads — raw, at-time, events, processed (push-down)

Task 11: SQL Server updater

Task 12: SQL Server provider + Host registration

Task 13: Over-the-wire HistoryUpdate test

Task 14: Documentation

Final review (fable), then Task 15: Live gate on docker-dev (controller)

Finish

66 KiB

Raw Blame History

Task 7: Client.CLI `historyupdate` command