# Pluggable Historian Backends Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:subagent-driven-development (recommended) or superpowers-extended-cc:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Make the historian backend pluggable (`ServerHistorian:Backend`), carry typed values + real status end to end, ship a SQL Server backend, and add the OPC UA HistoryUpdate service. **Architecture:** A Runtime-level `IHistorianBackendProvider` builds the five historian seams (data source, value writer, alarm writer, provisioning, updater); the Host picks one by name. The existing recorder/outbox/alarm S&F pipelines are unchanged in structure and now carry a typed `HistorianValue` + `uint StatusCode`. SQL Server stores samples in a `(TagId, SourceTimeUtc)`-clustered table and pushes aggregates down to T-SQL. **Tech Stack:** .NET 10, Akka.NET, OPC Foundation UA .NET Standard, Microsoft.Data.SqlClient (already pinned centrally), FASTER log (existing), xUnit v3 + Shouldly. **Spec:** `docs/plans/2026-10-09-pluggable-historian-design.md` — the binding authority. Read it before any task. ## Global Constraints - **No new NuGet packages, no version changes.** `Directory.Packages.props` is not edited by any task. Adding `` (already pinned) to a new project is allowed. - Recording stays **Primary-only** (#537 capture gate) and **at-least-once** with the #538 per-tag quarantine and all-or-nothing ack. Do not restructure `DrainOnceAsync`. - In any Akka actor handler: capture `IActorRef self = Self` (and `Sender`) **before** the first `await`; never touch `Self`/`Context` after an `await …ConfigureAwait(false)` (#491). - Test fakes for async seams used by actors must complete on a **pool thread** (`await Task.Delay(1).ConfigureAwait(false)`), never synchronously and never via `Task.Yield()`. - Every OPC UA status code a new class emits is a **named `const uint`** in an internal `*StatusCodes` class, and that assembly must be visible to `tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/StatusCodeParityTests.cs` (it scans its bin dir — add a ProjectReference). - Authorization changes need an **ALLOW** test (a deny-only test passes against a build that never consults the gate). - A regression test must be shown to fail with the production change reverted (falsification) — record the observation in the task report. - The gateway backend's observable behaviour (bytes sent for Good numeric samples, provisioning skips, read paths) does not change. - Commit messages end with `Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe`. Subagents never push and never dispatch subagents. - macOS zsh: quote globs; `tea` needs `` from the integration branch `feat/pluggable-historian` at the wave's start commit. Run `git -C log -1` and confirm the base SHA in the dispatch (worktrees can start stale — reset first). 2. **File ownership.** A task edits only the files its "Files" list names. The shared hot files have exactly one owner per wave: | Shared file | Owner | |---|---| | `ZB.MOM.WW.OtOpcUa.slnx` | T8 (adds both SqlServer projects) | | `src/Server/ZB.MOM.WW.OtOpcUa.Host/Program.cs` | T5 (wave 3), then T12 (wave 4) | | `.../Host/Configuration/ServerHistorianOptionsValidator.cs` | T5 (wave 3), then T12 (wave 4) | | `.../Runtime/Historian/ContinuousHistorizationMetrics.cs` | T5 | | `.../OpcUaServer/OtOpcUaSdkServer.cs`, `.../Host/OpcUa/OtOpcUaServerHostedService.cs` | T6 | | `tests/Server/.../Runtime.Tests/Historian/ContinuousHistorizationRecorderTests.cs` | T1 (ripple) then T3 | | `Core.Abstractions/Historian/IHistorizationOutbox.cs` | T1 | | `tests/Core/.../Core.Abstractions.Tests/*.csproj` | T8 | | `src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs` | T6 only | | `.../Runtime/Historian/ContinuousHistorizationRecorder.cs` | T1 (ripple) then T3 only | | `.../Driver.Historian.SqlServer/*.csproj` | T8 only (T9–T11 add `.cs` files only; csproj uses default globbing + `EmbeddedResource` for `Schema/*.sql` set by T8) | A task that discovers it needs an un-owned shared file stops and reports `NEEDS_CONTEXT` rather than editing it. 3. **New-file-only parallelism in the SQL project.** T9/T10/T11 each create their own files and their own test files; none edits another's. Shared SQL helpers live in T8's `SqlHistorianConnection.cs`/`SampleRowMapper.cs`; if a wave-3 task needs a helper change, it adds a private helper in its own file instead. 4. **Merge order** at wave end: the controller merges worktree branches into `feat/pluggable-historian` in ascending task number, running `dotnet build ZB.MOM.WW.OtOpcUa.slnx` after each merge and the touched test projects after the last. A conflict ⇒ stop and resolve on the integration branch, never inside a worktree. 5. **Integration-test DBs are per-test-run** (`OtOpcUaHistorian_T_`), so parallel T9/T10/T11 suites against the same `sql` container cannot collide. 6. Subagents do not push, merge, or dispatch subagents. Only the controller merges. ### Common commands ```bash dotnet build ZB.MOM.WW.OtOpcUa.slnx dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests --filter "FullyQualifiedName~Historian" dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests export HISTORIAN_SQL_TEST_CONNECTION='Server=localhost,14330;User Id=sa;Password=OtOpcUa!Dev123;TrustServerCertificate=True;Encrypt=False' dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests ``` The rig `sql` container must be up: `docker ps --filter name=otopcua-dev-sql-1`. If down: `cd docker-dev && docker compose up -d sql`. --- ### Task 1: Typed historian contracts + compile ripple **Goal:** Introduce `HistorianValue`, `HistorianValueKind`, `HistoryUpdateMode`, `IHistorianDataUpdater`, `IHistorianBackendProvider`, change `HistorizationValue`/`HistorizationOutboxEntry` to typed+status, and keep the whole solution compiling with **no behaviour change**. > **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:** > - Add `long UnreadableCount => 0;` as a default interface member on `IHistorizationOutbox` (Core.Abstractions) — T2 overrides it, T5 binds the gauge. (#6) > - The record call sites to update are explicit: `grep -rn "HistorizationOutboxEntry(\|HistorizationValue(" src tests --include='*.cs'` (≈17 sites in 7 files incl. `GatewayLiveIntegrationTests.cs`, `HistorizationOutboxEntrySerializerTests.cs`, `GatewayHistorianValueWriterTests.cs`, `ContinuousHistorizationRecorderTests.cs`). Every one of them is in T1's Files. Do not stop at an unlisted one. (#3) **Files:** - Create: `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/HistorianValue.cs` - Create: `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianDataUpdater.cs` - Create: `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/IHistorianBackendProvider.cs` - Modify: `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianValueWriter.cs` (`HistorizationValue`) - Modify: `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/HistorizationOutboxEntry.cs` - Modify: `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorizationOutbox.cs` (default member `UnreadableCount`) - Modify (ripple only): `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationRecorder.cs` (lines ~548 entry construction, ~692 drain mapping), `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/GatewayHistorianValueWriter.cs`, `.../Recorder/HistorizationOutboxEntrySerializer.cs`, and every test that constructs these records (find with `grep -rn "HistorizationOutboxEntry(\|HistorizationValue(" src tests --include='*.cs'`). - Test: `tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/Historian/HistorianValueTests.cs` **Acceptance Criteria:** - [ ] `HistorianValue.TryFromObject` implements exactly the spec §3 table; tests cover every row incl. `ulong` above `long.MaxValue` ⇒ Double, `DateTime` local ⇒ UTC ticks, `null` ⇒ Empty, `int[]` ⇒ false. - [ ] `TryGetDouble` true for Double/Int64/Boolean only; `ToObject` round-trips each kind (`DateTime` returns `DateTimeKind.Utc`). - [ ] Solution builds; all pre-existing historian tests pass unchanged in assertion content (only constructor call sites updated). - [ ] Ripple preserves behaviour: recorder still coerces numerics only (via `TryFromObject` + `TryGetDouble`, building `HistorianValue.FromDouble(numeric)`) and still stamps Good (`StatusCode = 0`); serializer still writes the legacy layout using `Value.Number` and quality `192`; gateway writer maps `Value.Number` and quality `192`. **Verify:** `dotnet build ZB.MOM.WW.OtOpcUa.slnx && dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests --filter "FullyQualifiedName~HistorianValue" && dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests && dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests --filter "FullyQualifiedName~Historiz"` → all green. **Steps:** - [ ] **Step 1: Write `HistorianValueTests` (failing — type does not exist).** Cases: ```csharp [Theory] [InlineData(1.5d, HistorianValueKind.Double)] [InlineData(1.5f, HistorianValueKind.Double)] [InlineData(7, HistorianValueKind.Int64)] [InlineData((short)7, HistorianValueKind.Int64)] [InlineData((byte)7, HistorianValueKind.Int64)] [InlineData(7u, HistorianValueKind.Int64)] [InlineData(7L, HistorianValueKind.Int64)] [InlineData(true, HistorianValueKind.Boolean)] [InlineData("x", HistorianValueKind.String)] public void Maps_clr_values_to_kinds(object input, HistorianValueKind expected) { HistorianValue.TryFromObject(input, out var v).ShouldBeTrue(); v.Kind.ShouldBe(expected); } // plus: decimal 2.5m ⇒ Double 2.5; ulong.MaxValue ⇒ Double; (ulong)5 ⇒ Int64 5; // null ⇒ Empty (true); new[]{1,2} ⇒ false; new object() ⇒ false; // DateTime(2026,1,1,0,0,0,Local) ⇒ DateTime kind, Integer == ToUniversalTime().Ticks, ToObject() Kind Utc; // FromBoolean(true): Integer 1, Number 1, TryGetDouble ⇒ 1; FromString("a").TryGetDouble ⇒ false; Empty.ToObject() ⇒ null. ``` - [ ] **Step 2: Run, confirm compile failure.** - [ ] **Step 3: Implement `HistorianValue.cs`:** ```csharp namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions; /// The value kinds a historian backend can store. Persisted as a byte (SQL ValueKind, outbox v2) — never renumber. public enum HistorianValueKind : byte { Empty = 0, Double = 1, Int64 = 2, Boolean = 3, String = 4, DateTime = 5 } /// A typed historized value. Number carries Double (and the double view of Int64/Boolean); /// Integer carries Int64, Boolean (1/0) and DateTime (UTC ticks); Text carries String. public readonly record struct HistorianValue(HistorianValueKind Kind, double Number, long Integer, string? Text) { public static HistorianValue Empty => default; public static HistorianValue FromDouble(double v) => new(HistorianValueKind.Double, v, 0, null); public static HistorianValue FromInt64(long v) => new(HistorianValueKind.Int64, v, v, null); public static HistorianValue FromBoolean(bool v) => new(HistorianValueKind.Boolean, v ? 1d : 0d, v ? 1 : 0, null); public static HistorianValue FromString(string v) => new(HistorianValueKind.String, 0d, 0, v ?? throw new ArgumentNullException(nameof(v))); public static HistorianValue FromDateTime(DateTime v) => new(HistorianValueKind.DateTime, 0d, (v.Kind == DateTimeKind.Local ? v.ToUniversalTime() : DateTime.SpecifyKind(v, DateTimeKind.Utc)).Ticks, null); public static bool TryFromObject(object? value, out HistorianValue result) { switch (value) { case null: result = Empty; return true; case double d: result = FromDouble(d); return true; case float f: result = FromDouble(f); return true; case decimal m: result = FromDouble((double)m); return true; case sbyte sb: result = FromInt64(sb); return true; case byte b: result = FromInt64(b); return true; case short s: result = FromInt64(s); return true; case ushort us: result = FromInt64(us); return true; case int i: result = FromInt64(i); return true; case uint ui: result = FromInt64(ui); return true; case long l: result = FromInt64(l); return true; case ulong ul: result = ul <= long.MaxValue ? FromInt64((long)ul) : FromDouble(ul); return true; case bool bo: result = FromBoolean(bo); return true; case string str: result = FromString(str); return true; case DateTime dt: result = FromDateTime(dt); return true; default: result = Empty; return false; // arrays, references, structures ⇒ not historizable } } public bool TryGetDouble(out double value) { switch (Kind) { case HistorianValueKind.Double: case HistorianValueKind.Int64: case HistorianValueKind.Boolean: value = Number; return true; default: value = 0d; return false; } } public object? ToObject() => Kind switch { HistorianValueKind.Double => Number, HistorianValueKind.Int64 => Integer, HistorianValueKind.Boolean => Integer != 0, HistorianValueKind.String => Text, HistorianValueKind.DateTime => new DateTime(Integer, DateTimeKind.Utc), _ => null, }; } ``` - [ ] **Step 4: `IHistorianDataUpdater.cs`** — exactly the spec §4 "Updates" interface + `HistoryUpdateMode` enum, with XML docs naming the per-item status codes. - [ ] **Step 5: `IHistorianBackendProvider.cs`** in `ZB.MOM.WW.OtOpcUa.Runtime.Historian` — exactly spec §2. - [ ] **Step 6: Change the records:** ```csharp public readonly record struct HistorizationValue(DateTime? TimestampUtc, HistorianValue Value, uint StatusCode); public sealed record HistorizationOutboxEntry(Guid Id, string Tag, HistorianValue Value, uint StatusCode, DateTime TimestampUtc); ``` - [ ] **Step 7: Ripple, behaviour-preserving.** Recorder line ~527: replace `TryCoerceToDouble(msg.Value, out double numeric)` with `HistorianValue.TryFromObject(msg.Value, out var hv) && hv.TryGetDouble(out double numeric)` (same drop semantics — strings/null/DateTime still dropped as non-numeric); entry = `new HistorizationOutboxEntry(Guid.NewGuid(), historianName, HistorianValue.FromDouble(numeric), 0u /*Good*/, timestampUtc)`; delete `TryCoerceToDouble`. Drain line ~692: `new HistorizationValue(e.TimestampUtc, e.Value, e.StatusCode)`. Serializer: write `entry.Value.Number` and `(ushort)192`; read back as `FromDouble(value)`, status `0`. Gateway writer: `NumericValue = value.Value.Number, Quality = 192`. Keep `GoodQuality` const for now (T3 removes it). Update test call sites. - [ ] **Step 8: Build + run Verify.** Commit: `feat(historian): typed HistorianValue contracts, IHistorianDataUpdater, IHistorianBackendProvider (no behaviour change)`. ```json:metadata {"files":["src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/HistorianValue.cs","src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianDataUpdater.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/IHistorianBackendProvider.cs","src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianValueWriter.cs","src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/HistorizationOutboxEntry.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationRecorder.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/GatewayHistorianValueWriter.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/HistorizationOutboxEntrySerializer.cs"],"acceptanceCriteria":["TryFromObject matches spec table","solution builds, existing historian tests green","no behaviour change"],"modelTier":"standard","model":"sonnet","wave":1} ``` --- ### Task 2: Outbox format v2 with legacy read **Goal:** Serialize every `HistorianValueKind` + `uint StatusCode`; read pre-upgrade entries; drop (and count) unreadable entries instead of throwing. > **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:** > - **Model: opus** (FIFO invariant on the crash boundary). (#1) > - `FasterLogHistorizationOutbox.RecoverState()` (ctor) ALSO deserializes every record. Use `TryDeserialize` in **both** `RecoverState` and `PeekBatchAsync`. In `RecoverState`, index an unreadable record in `_live/_index` under a synthetic `Guid.NewGuid()` with its address so FIFO truncation can pass it; in Peek skip it (and make sure an unreadable entry at the head is removable so it can't pin the head — e.g. Peek auto-removes leading unreadable entries). `UnreadableCount` (overrides T1's default member) increments in both. (#1) > - Required test: write a raw garbage record straight into the FasterLog directory (open `FasterLog` directly), **restart** the outbox, prove the ctor succeeds, the following good entry drains, and the head advances past the garbage. (#1) > - Legacy/v2 ambiguity: when the 4-byte magic matches but v2 structural validation fails, fall back to the legacy exact-length test before declaring unreadable. (#18) > - Drop the meter bullet: T5 binds the gauge via `IHistorizationOutbox.UnreadableCount`. (#6) **Files:** - Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/HistorizationOutboxEntrySerializer.cs` - Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/FasterLogHistorizationOutbox.cs` (only the deserialize call site in Peek: skip+count unreadable entries; expose `long UnreadableCount`) - Test: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests/Recorder/HistorizationOutboxEntrySerializerV2Tests.cs` **Acceptance Criteria:** - [ ] v2 layout exactly: `"HV2\0"`(4) `ver=2`(1) `id`(16) `tagLen`(i32 LE) `tag`(UTF-8) `kind`(1) payload `status`(u32 LE) `ts`(i64 LE `ToBinary`). Payload: Empty none; Double f64; Int64/Boolean/DateTime i64; String i32 len + UTF-8. - [ ] Round-trip test per kind incl. empty string, a 70-KB string, `NaN`, `double.NegativeInfinity`, `long.MinValue`, status `0x80310000`. - [ ] **Pinned legacy bytes:** a hex literal produced by today's serializer (generate it once from the T1-era `Serialize` for `Id=00000000-0000-0000-0000-000000000001, Tag="t", Value=42.5, Quality=192, ts=2026-01-01T00:00:00Z` and paste the bytes into the test) deserializes to `Double 42.5`, status `0`. - [ ] `TryDeserialize` returns false (never throws) for: 3 random bytes, magic with ver=9, magic with truncated payload, legacy-looking bytes whose length disagrees with `38+tagLen`. `FasterLogHistorizationOutbox` skips such an entry on peek, advances past it (it must be removable so it can't pin the head — truncate through its id or log-address as the outbox already does for acked entries), and increments `UnreadableCount`; a test proves a corrupt entry between two good ones does not block the second. - [ ] Meter: `ContinuousHistorizationMetrics` gets an observable `otopcua.historian.outbox.unreadable` bound alongside the existing outbox gauges — **only if** `BindOutbox` lives in an un-owned file; otherwise report the counter on the outbox and note it for T5. (Owned file check: `ContinuousHistorizationMetrics.cs` is NOT owned by T2 — expose the property only.) **Verify:** `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests --filter "FullyQualifiedName~Outbox"` → green; falsification: with the legacy branch deleted the pinned-bytes test fails. **Steps:** (1) capture the legacy hex from the current serializer via a throwaway test print, paste into the new test; (2) write failing v2 tests; (3) implement `Serialize` (v2 always) + `TryDeserialize(ReadOnlySpan, out HistorizationOutboxEntry?)` (v2 by magic, else legacy by exact length, else false) and keep a throwing `Deserialize` wrapper only if other callers need it; (4) wire skip+count in the outbox peek; (5) green; commit `feat(historian): outbox format v2 carries typed values + status; legacy entries still read`. ```json:metadata {"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/HistorizationOutboxEntrySerializer.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/FasterLogHistorizationOutbox.cs"],"acceptanceCriteria":["v2 layout","pinned legacy bytes read","corrupt entry skipped and counted, never pins head"],"modelTier":"frontier","model":"opus","wave":2} ``` --- ### Task 3: Recorder captures typed values with real status, filtered by backend kinds **Goal:** The recorder records `HistorianValue` + `msg.StatusCode` and drops kinds the backend does not support. > **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:** > - Add `tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Historian/ContinuousHistorizationRecorderTests.cs` to Files — lines ~337 and ~1149 assert `DroppedNonNumeric` and must follow the rename. (#3) > - The recorder remarks ("carrying only (TagId, Value, TimestampUtc) and no quality") are stale: `DependencyMuxActor.cs:114,150` passes `item.StatusCode`. Rewrite them. (verified) > - Comms-loss watchdog `BadNoCommunication` stamps are node-only (`FanCommsLossToValueNodes` never reaches the mux) and are NOT historized — say so in the remarks; only a driver's own Bad publish (usually `null` value ⇒ `Empty`) is. (#14) **Files:** - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationRecorder.cs` - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/ServiceCollectionExtensions.cs` **only** the recorder `Props` construction (~line 485–520) to pass `supportedKinds` — resolve `IHistorianBackendProvider` optionally; absent ⇒ `{Double, Int64, Boolean}` (today's behaviour). - Test: `tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Historian/ContinuousHistorizationRecorderTypedValueTests.cs` **Acceptance Criteria:** - [ ] New ctor/Props parameter `IReadOnlySet supportedKinds` (default = `{Double, Int64, Boolean}` for existing overloads so other tests compile unchanged). - [ ] Capture: `TryFromObject` false ⇒ drop + `_droppedUnsupportedKind++`; kind ∉ supportedKinds ⇒ same; otherwise entry carries the typed value and `msg.StatusCode`. - [ ] Status field in the recorder status record renamed `DroppedNonNumeric` → `DroppedUnsupportedKind`; any OTel instrument name stays unchanged (dashboards). `GoodQuality` const and its doc paragraph removed; class remarks updated (the mux DOES carry StatusCode — `VirtualTagActor.DependencyValueChanged`). - [ ] Tests (fake writer completes on a pool thread): a `Bad (0x80000000)` numeric sample is written with status `0x80000000`; a string sample is written when kinds include String and dropped+counted when not; `null` with status `BadNoCommunication` is written as Empty when kinds include Empty, dropped otherwise; an `int[]` is always dropped; existing #537 capture-gate and #538 quarantine tests still pass untouched. - [ ] Falsification recorded: reverting to the hard-coded Good makes the Bad-status test fail. **Verify:** `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests --filter "FullyQualifiedName~ContinuousHistorization"` → green. **Steps:** failing tests → implement → green → grep for `DroppedNonNumeric` across src/tests and update readers (AdminUI/metrics) **only within Runtime**; if a reader exists outside the owned files, report it for T5 instead of editing. Commit `fix(historian): record the real status code and typed values; backend-supported kinds gate capture`. ```json:metadata {"files":["src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationRecorder.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/ServiceCollectionExtensions.cs"],"acceptanceCriteria":["real status recorded","kind filter","#537/#538 unchanged","pool-thread fake"],"modelTier":"frontier","model":"opus","wave":2} ``` --- ### Task 4: Gateway provider + reverse quality mapping **Goal:** `GatewayHistorianProvider : IHistorianBackendProvider`; the gateway writer maps typed value + status to the gateway's numeric + DA quality. **Files:** - Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/GatewayHistorianProvider.cs` - Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Mapping/GatewayQualityMapper.cs` (add `ToDaQuality(uint)`) - Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/GatewayHistorianValueWriter.cs` - Test: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests/GatewayHistorianProviderTests.cs`, `.../Mapping/GatewayQualityMapperReverseTests.cs` **Acceptance Criteria:** - [ ] Provider: `Name = "Gateway"`, `SupportedKinds = {Double, Int64, Boolean}`, `CreateUpdater` returns `null`; the four `Create*` delegate to the existing `GatewayHistorian.Create…` (data source/alarm writer/provisioner) and build `GatewayHistorianValueWriter` exactly as `Program.cs` does today (`HistorianGatewayClientAdapter.Create(options, loggerFactory)`). Options come from the ctor (`ServerHistorianOptions`). - [ ] `ToDaQuality(Map(q)) == q` for every explicit byte in `Map`'s table; unknown status falls back by severity bits (`0x00…`⇒192, `0x40…`⇒64, `0x80…`⇒0). - [ ] Writer: a value with `TryGetDouble` false is skipped (debug log, no throw); Good Double 42.5 produces the same `HistorianLiveValue` (`NumericValue=42.5, Quality=192`) as before T1 — asserted in a test. **Verify:** `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests` → green. **Steps:** tests first → implement → green → commit `feat(historian): GatewayHistorianProvider; status→DA quality reverse mapping`. ```json:metadata {"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/GatewayHistorianProvider.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Mapping/GatewayQualityMapper.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/GatewayHistorianValueWriter.cs"],"acceptanceCriteria":["provider shape","reverse mapping inverse","Good bytes unchanged"],"modelTier":"standard","model":"sonnet","wave":2} ``` --- ### Task 5: Backend registry, config, validators, Host wiring **Goal:** `ServerHistorian:Backend` selects a provider; Program.cs builds every historian seam through it. > **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:** > - Gate **all three** gateway checks in `ServerHistorianOptionsValidator` (Endpoint, ApiKey, UseTls⇔scheme) on `Backend == Gateway`, and the same in `ServerHistorianOptions.Validate()` warnings. Tests: `Backend=SqlServer, Enabled=true, no Endpoint/ApiKey ⇒ passes`; control `Backend=Gateway, no ApiKey ⇒ fails` (unchanged). (#2) > - Unknown `Backend` ⇒ always fail (no I/O). Backend-specific option validation (SqlServer, added by T12) runs inside `ServerHistorianOptionsValidator` and only when consumed (`ServerHistorian:Enabled || AlarmHistorian:Enabled`) — the existing consumer-gated posture. `HistorianBackendRegistry.Resolve` must not validate. (#11) > - Register `HistorianUpdaterHolder.None` **unconditionally** in the driver block; register the real `new HistorianUpdaterHolder(provider.CreateUpdater(sp))` only when `ServerHistorian:Enabled` (HistoryUpdate gated like HistoryRead; last registration wins). (#11) > - Add `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationMetrics.cs` to Files: add observable gauge `otopcua.historian.outbox.unreadable` from `IHistorizationOutbox.UnreadableCount` in `BindOutbox`. (#6) > - Program.cs is top-level statements and not unit-testable: the "DI graph unchanged" AC is tested as `HistorianBackendRegistry.Resolve(gatewayOpts, cfg).CreateDataSource(sp) is GatewayHistorianDataSource` (and likewise for the writer/alarm writer/provisioner). Do not try to host-start Program. (#22) > - Spec wording fix: there are three `GatewayHistorian.Create…` call sites (Program.cs ~241/262/273) plus the inline writer (~313). (#16) **Files:** - Create: `src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs` - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ServerHistorianOptions.cs` (add `Backend` string, default `"Gateway"`) - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/ServerHistorianOptionsValidator.cs` - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Host/Program.cs` (lines ~215–318 only) - Test: `tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/Configuration/HistorianBackendRegistryTests.cs`, extend `ServerHistorianOptionsValidatorTests` **Acceptance Criteria:** - [ ] Registry: `static IHistorianBackendProvider Resolve(ServerHistorianOptions o, IConfiguration cfg)`; knows `"Gateway"` now (T12 adds `"SqlServer"`); case-insensitive; unknown ⇒ `OptionsValidationException` naming the value and listing valid names. The registry also exposes `IReadOnlyCollection KnownBackends` used by the validator. - [ ] Validator: unknown backend fails; `Endpoint` required only when backend is Gateway; existing tests unchanged in intent. - [ ] Program.cs: resolve the provider once; register it as a singleton `IHistorianBackendProvider`; replace the 4 `GatewayHistorian.Create…` lambdas and the inline `GatewayHistorianValueWriter` with `provider.Create…(sp)`; register `IHistorianDataUpdater` via `provider.CreateUpdater(sp)` **only when non-null** (`TryAddSingleton`-style factory is not possible for a null — register conditionally by calling it once at registration time is NOT allowed (needs sp); instead register a small `HistorianUpdaterHolder(IHistorianDataUpdater? Updater)` singleton, and have T6's hosted-service hook read `holder.Updater`). The alarm-only warning text names `serverHistorianOptions.Backend`. - [ ] Bind the outbox unreadable gauge from T2 in `ContinuousHistorizationMetrics.BindOutbox` if T2 reported it (this task owns that file this wave). - [ ] With `Backend` absent the Host's DI graph resolves the same concrete types as before (test: build the service collection with a Gateway config and assert `IHistorianDataSource` resolves to `GatewayHistorianDataSource`). **Verify:** `dotnet build ZB.MOM.WW.OtOpcUa.slnx && dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests --filter "FullyQualifiedName~Historian"` → green. ```json:metadata {"files":["src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationMetrics.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ServerHistorianOptions.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/ServerHistorianOptionsValidator.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/Program.cs"],"acceptanceCriteria":["unknown backend fails boot","Gateway default resolves identical types","updater holder registered"],"modelTier":"standard","model":"sonnet","wave":3} ``` --- ### Task 6: HistoryUpdate service in the node manager + authorization **Goal:** Clients can HistoryUpdate historized nodes when the backend supplies an updater, gated by ACL (flag on) or the new `HistoryUpdate` role (flag off). > **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:** > - SDK pinned `OPCFoundation.NetStandard.Opc.Ua.* 1.5.378.106`. Verified: public `HistoryUpdate(OperationContext, Type detailsType, …)` runs under `Lock`, pre-seeds `errors[ii] = BadHistoryOperationUnsupported`, and admits a variable **only if `(AccessLevel & AccessLevels.HistoryWrite) != 0`**; it releases the lock and dispatches to protected `HistoryUpdateData` / `HistoryDeleteRawModified` / `HistoryDeleteAtTime(ServerSystemContext, IList<…Details>, IList, IList, List, IDictionary)`. The HistoryWrite bit is therefore **dispatch-load-bearing**. (#4) > - Add `src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaSdkServer.cs` to Files: `SetHistorianDataUpdater(IHistorianDataUpdater?)` mirroring `SetHistorianDataSource` (line ~122). In `OtOpcUaServerHostedService.StartAsync` assign it **immediately after the `NodeManager is null` guard (~line 164), before `_deferredSink.SetSink` (~171)**; clear it in `StopAsync` next to the data-source clear (~373). Hosted service is registered before `AddAkka`, so this precedes every deploy apply. (#4, #5) > - On success each arm sets `errors[handle.Index] = ServiceResult.Good` (the base pre-seeds Unsupported; every HistoryRead success path here does the same, e.g. OtOpcUaNodeManager.cs ~5133). Per-node `HistoryUpdateResult.StatusCode` stays Good when per-item results are mixed; only a whole-node failure (deny / unsupported / not historized) sets the node code. (#13, #21) > - `UpdateDataDetails.UpdateValues[i].SourceTimestamp` is non-nullable: `== DateTime.MinValue` ⇒ `BadInvalidTimestamp` for that item. (#21) > - At least one test MUST drive the public `nm.HistoryUpdate(OperationContext, typeof(UpdateDataDetails), …)` entry point and assert the updater is called — it is the test that fails when the AccessLevel bit is missing. Tests that call the protected arm directly do not count toward the ALLOW criterion. (#4) > - AclEdit: just add `NodePermissions.HistoryUpdate` to the `PermissionBits` array (~161–166); the page renders enum names, there is no label table. (#23) **Files:** - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs` - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Security/OpcUaDataPlaneRoles.cs` (`HistoryUpdate = "HistoryUpdate"`) - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/ApiKeyInterimRoleMapper.cs` - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs` (assign `HistorianDataUpdater` from an optional `HistorianUpdaterHolder` — define `HistorianUpdaterHolder` in `Runtime/Historian/HistorianUpdaterHolder.cs` in THIS task: `public sealed record HistorianUpdaterHolder(IHistorianDataUpdater? Updater) { public static readonly HistorianUpdaterHolder None = new((IHistorianDataUpdater?)null); }`, resolve with `GetService` ⇒ None when absent) - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Pages/Clusters/AclEdit.razor` (add `NodePermissions.HistoryUpdate` to the list at ~line 163–166 with label "History update") - Modify: `src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Enums/NodePermissions.cs` comment only (remove "until the HistoryUpdate service surface is implemented"; still in no bundle) - Test: `tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/History/HistoryUpdateTests.cs`, `tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/OpcUa/ApiKeyInterimRoleMapperTests.cs` (extend) **Acceptance Criteria:** - [ ] Before writing, read the pinned SDK's `CustomNodeManager2` and confirm the exact protected signatures for `HistoryUpdateData`, `HistoryDeleteRawModified`, `HistoryDeleteAtTime` (and how `HistoryUpdate` dispatches by details type); record them in the report. Use DeepWiki (`OPCFoundation/UA-.NETStandard`) if the source is not on disk. - [ ] `public IHistorianDataUpdater? HistorianDataUpdater { get; set; }` (volatile backing field, like `HistorianDataSource`). - [ ] `UpdateDataDetails`: `PerformUpdateType.Insert/Replace/Update` ⇒ `HistoryUpdateMode`; `Remove` ⇒ `BadHistoryOperationUnsupported`. Values: `DataValue` ⇒ `HistorizationValue(SourceTimestamp, TryFromObject(Value.Value) or BadTypeMismatch item, StatusCode.Code)`. Per-item results returned in `OperationResults` in order. - [ ] `DeleteRawModifiedDetails` with `IsDeleteModified=false` ⇒ `DeleteRawAsync`; `true` ⇒ unsupported. `DeleteAtTimeDetails` ⇒ `DeleteAtTimeAsync`. - [ ] Not historized / no updater ⇒ `BadHistoryOperationUnsupported`. Event/structure details ⇒ unsupported. - [ ] Authorization helper `AuthorizeHistoryUpdate` mirroring `AuthorizeHistoryRead` + the WriteOperate pattern at ~line 2024–2033: flag on ⇒ `_authorizer.Authorize(identity, OpcUaOperation.HistoryUpdate, nodeId, realm)`; flag off ⇒ `AuthorizeKeyUsable(...)` AND role `HistoryUpdate`. Deny ⇒ `BadUserAccessDenied`. - [ ] Batch limiter + deadline shared with HistoryRead (`WithHistoryReadBatchLimiter`, `CreateDeadline`). - [ ] `ComposeAccessLevel(writable, historized, historyWritable)` ORs `AccessLevels.HistoryWrite` when `historized && historyWritable`; `historyWritable = HistorianDataUpdater is not null` at materialise time. Raw + UNS mirrors both get it. (Setting the updater after nodes exist does not retro-update AccessLevel — document; the hosted service assigns it before the first deploy applies, verify the ordering and state it.) - [ ] `ApiKeyInterimRoleMapper` adds `OpcUaDataPlaneRoles.HistoryUpdate` for a Cluster-scoped grant with `NodePermissions.HistoryUpdate`; a WriteOperate-only grant does NOT confer it. - [ ] Tests (fake updater + fake authorizer, following the existing node-manager test harness for HistoryRead): **ALLOW flag-on with no roles + scoped grant ⇒ updater called and `GoodEntryInserted` returned**; flag-off with `HistoryUpdate` role ⇒ allowed; flag-off with only `WriteOperate` ⇒ `BadUserAccessDenied` and updater NOT called; expired key ⇒ denied; no updater ⇒ unsupported; per-item codes passed through in order; UNS NodeId resolves to the same tagname as raw. - [ ] Falsification recorded for the ALLOW test (break the flag-on branch ⇒ it fails). **Verify:** `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests --filter "FullyQualifiedName~HistoryUpdate" && dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests --filter "FullyQualifiedName~ApiKeyInterimRoleMapper"` → green. ```json:metadata {"files":["src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs","src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaSdkServer.cs","src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Security/OpcUaDataPlaneRoles.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/ApiKeyInterimRoleMapper.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/HistorianUpdaterHolder.cs","src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Pages/Clusters/AclEdit.razor","src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Enums/NodePermissions.cs"],"acceptanceCriteria":["SDK signatures verified","ALLOW test","WriteOperate does not confer HistoryUpdate","HistoryWrite access bit"],"modelTier":"frontier","model":"opus","wave":2} ``` Note: T5 references `HistorianUpdaterHolder` — T5 runs in wave 3 after T6 merges, so the type exists. --- ### Task 7: Client.CLI `historyupdate` command **Goal:** A CLI command to drive HistoryUpdate for manual + live-gate testing. **Files:** - Create: `src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI/Commands/HistoryUpdateCommand.cs` (follow the existing `historyread` command's structure, options parsing, session/auth handling exactly) - Modify: `docs/Client.CLI.md` (new section) - Test: the Client.CLI test project if one exists for commands (`ls tests/Client`); follow its pattern for argument parsing tests. **Acceptance Criteria:** - [ ] Options per spec §6; `--mode` values `insert|replace|update|delete-raw|delete-at-time`; `--value` parsed against `--type double|int64|bool|string|datetime` (default double); repeated `--time/--value` pairs; validation errors exit non-zero with a message. - [ ] Sends `UpdateDataDetails` / `DeleteRawModifiedDetails(IsDeleteModified=false)` / `DeleteAtTimeDetails` via `Session.HistoryUpdate`; prints `index time StatusCodeName` per item plus the overall service result. - [ ] Argument-parsing unit tests for each mode (no server needed). **Verify:** `dotnet build src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI && dotnet test tests/Client --filter "FullyQualifiedName~HistoryUpdate"` → green. ```json:metadata {"files":["src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI/Commands/HistoryUpdateCommand.cs","docs/Client.CLI.md"],"acceptanceCriteria":["all five modes","arg parsing tests"],"modelTier":"standard","model":"sonnet","wave":2} ``` --- ### Task 8: SQL Server backend project skeleton, schema, bootstrapper, test fixture **Goal:** New projects, embedded DDL, idempotent locked bootstrap, options + validation, shared connection/mapping helpers, and the env-gated integration fixture. > **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:** > - **Move `SqlServerTagCatalog.cs` here from T9** (get-or-create + read-only lookup), used by T9/T10/T11/T12. Create path: `MERGE {s}.Tag WITH (HOLDLOCK) AS t …`; on `2627/2601` re-select the TagId. Integration test: two concurrent get-or-create on the same new name ⇒ both return the same id, one row. (#10, #12) > - Timestamps: helper `AddDateTime2(SqlCommand, name, DateTime)` (`SqlDbType.DateTime2`, scale 7) for every scalar param; TVPs are `IEnumerable` with `SqlMetaData(…, SqlDbType.DateTime2, 0, 7)` — **never a `DataTable`** (infers 3.33 ms `datetime`). Mapper tests use non-millisecond-aligned ticks (`…T00:00:00.1234567Z`). (#7) > - DDL: `CREATE SCHEMA` must be alone in its batch ⇒ `IF SCHEMA_ID(N'$(Schema)') IS NULL EXEC(N'CREATE SCHEMA [$(Schema)]')`; substitute `$(Schema)` **before** splitting on `GO`. (#17) > - `Event`: `EventId nvarchar(64) NOT NULL CONSTRAINT PK_Event PRIMARY KEY NONCLUSTERED`, plus `CREATE CLUSTERED INDEX CX_Event_Time ON {s}.Event (EventTimeUtc, EventId)` (drop the separate `IX_Event_Time`). Keep `IX_Event_Source_Time`. (#19) > - Private helper types introduced by wave-3 tasks inside this project must be prefixed with their task (`T9…`, `T10…`, `T11…`) to avoid merge collisions. (#12) > - Verified: `StatusCodes.*` names in `SqlHistorianStatusCodes` all exist in the pinned SDK, and `AggregateBits` Calculated=0x1 / Interpolated=0x2 with InfoType 0x400 are correct; keep the SDK-oracle unit test. (#26) **Files:** - Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.csproj` (refs: Core.Abstractions, Core.AlarmHistorian, Runtime; ``, `Microsoft.Extensions.Logging.Abstractions`; ``; `InternalsVisibleTo` both test projects) - Create: `.../SqlServerHistorianOptions.cs` (+ `Validate()` returning errors per spec §2), `.../SqlHistorianStatusCodes.cs`, `.../SqlHistorianConnection.cs` (open connection, schema-qualify helper `Q(string obj)` ⇒ `[{schema}].[{obj}]`, transient-error classifier: 1205, -2, 40613, 40197, 40501, 49918, 4060, 233, 10053, 10054, 10060, 11001, SqlException with `IsTransient`), `.../SampleRowMapper.cs` (HistorianValue ⇄ columns per spec §4 + `DataTable` builders for `SampleRow` and `TimeRow` TVPs), `.../SqlServerSchemaBootstrapper.cs`, `.../Schema/001_initial.sql` - Create: `docs/historian/sqlserver/001_initial.sql` (copy of the embedded script) - Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.Tests/` (unit) and `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/` with `SqlHistorianFixture.cs` - Modify: `ZB.MOM.WW.OtOpcUa.slnx` (4 new projects), `tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj` (ProjectReference to the new src project so `StatusCodeParityTests` scans `SqlHistorianStatusCodes`) **Acceptance Criteria:** - [ ] `001_initial.sql` is the spec §4 DDL with `{s}` replaced by a `$(Schema)` token that the bootstrapper substitutes after validating the identifier regex; every statement guarded (`IF SCHEMA_ID(...) IS NULL`, `IF OBJECT_ID(...) IS NULL`, `IF TYPE_ID(...) IS NULL`) so a re-run is a no-op; ends with `INSERT SchemaVersion(1, SYSUTCDATETIME())` guarded by `IF NOT EXISTS`. Split on `^GO$` lines. - [ ] `SqlServerSchemaBootstrapper.EnsureAsync(ct)`: lazy, once per instance (a `SemaphoreSlim` + `_ready` flag; failure NOT latched), `sp_getapplock` exclusive session lock with a 30 s timeout around the apply; `AutoCreateSchema=false` ⇒ only checks `MAX(Version) >= 1` and throws `HistorianSchemaMissingException("…missing version 1…")`. - [ ] `SqlHistorianStatusCodes` contains every code the spec names (`Good, GoodNoData, GoodEntryInserted, GoodEntryReplaced, BadNoData, BadEntryExists, BadNoEntryExists, BadInvalidTimestamp, BadTypeMismatch, BadAggregateInvalidInputs, BadHistoryOperationUnsupported, UncertainDataSubNormal`) as named `const uint` with values copied from `Opc.Ua.StatusCodes` — the parity test passes and lists this class. - [ ] Historian info-bit constants (`InfoTypeDataValue = 0x00000400`, `HistorianCalculated = 0x1`, `HistorianInterpolated = 0x2`) in a separate `HistorianInfoBits` static class, with a unit test asserting `new Opc.Ua.StatusCode(Good).SetAggregateBits(AggregateBits.Calculated).Code == (Good | InfoTypeDataValue | HistorianCalculated)` (adjust to the SDK's actual helper — the test must compare against the SDK, not against this plan). - [ ] `SqlHistorianFixture` (xUnit `IAsyncLifetime`, collection fixture): reads `HISTORIAN_SQL_TEST_CONNECTION`; absent ⇒ `SkipReason` and tests `Assert.Skip(SkipReason)` (follow `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.IntegrationTests/SqlPollServerFixture.cs`); present ⇒ `CREATE DATABASE OtOpcUaHistorian_T_`, exposes `ConnectionString` for it, `DROP DATABASE` (after `ALTER … SET SINGLE_USER WITH ROLLBACK IMMEDIATE`) on dispose. - [ ] Integration tests: bootstrap twice ⇒ second is a no-op and `SchemaVersion` has one row; two bootstrappers concurrently ⇒ both succeed, one row; `AutoCreateSchema=false` on an empty DB ⇒ `HistorianSchemaMissingException`; invalid schema name rejected by `Validate()` before any SQL runs (unit). - [ ] `SampleRowMapper` unit tests: every kind round-trips through the column triple. **Verify:** `dotnet build ZB.MOM.WW.OtOpcUa.slnx && dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.Tests && dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests && dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests --filter "FullyQualifiedName~StatusCodeParity"` → green (integration with the env var set; also run once WITHOUT it and confirm all skip). ```json:metadata {"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerTagCatalog.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/","tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.Tests/","tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/","ZB.MOM.WW.OtOpcUa.slnx","docs/historian/sqlserver/001_initial.sql"],"acceptanceCriteria":["idempotent DDL","applock bootstrap","parity-guarded status consts","env-gated throwaway DB fixture"],"modelTier":"standard","model":"sonnet","wave":2} ``` --- ### Task 9: SQL Server writes — value writer, provisioning, alarm writer **Goal:** Live values, tag provisioning and alarm events persist idempotently. > **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:** > - `SqlServerTagCatalog` now comes from T8 — do NOT create it; use it. (#12) > - Insert-if-absent: catch `2627/2601` and treat as success (the row is durably present), so a concurrent writer never books a #538 failure. (#25) > - Provisioning `MERGE … WITH (HOLDLOCK)`; integration test with two concurrent provisioners of the same new tags ⇒ both report Ensured, no Failed. (#10) > - EventId: SHA-256 over **length-prefixed** UTF-8 fields `EquipmentPath, AlarmId, EventKind, TimestampUtc.Ticks, User, Comment, Message` (null ⇒ length −1). Unit test that `"a|b"+"c"` and `"a"+"b|c"` hash differently. (#20) > - All timestamp round-trips use non-ms-aligned ticks and assert tick equality. (#7) **Files (create only):** - `.../SqlServerHistorianValueWriter.cs`, `.../SqlServerTagProvisioner.cs`, `.../SqlServerAlarmHistorianWriter.cs` - Tests: `tests/.../SqlServer.IntegrationTests/WriteTests.cs`, `ProvisioningTests.cs`, `AlarmWriterTests.cs`; unit `tests/.../SqlServer.Tests/AlarmEventIdTests.cs` **Acceptance Criteria:** - [ ] Every public entry point calls `bootstrapper.EnsureAsync` first. - [ ] Value writer: TVP insert-if-absent (`INSERT INTO Sample(...) SELECT @tagId, r.* , SYSUTCDATETIME() FROM @rows r WHERE NOT EXISTS (SELECT 1 FROM Sample s WITH (UPDLOCK, HOLDLOCK) WHERE s.TagId=@tagId AND s.SourceTimeUtc=r.SourceTimeUtc)`); within-batch duplicate timestamps collapse to the first (de-dup in C# before building the TVP — TVP PK would reject them); null timestamp ⇒ `DateTime.UtcNow`; returns false (never throws) on SQL errors; logs category only. - [ ] Provisioning: one `MERGE` per batch upserting Name/DataType/EU/Description; returns `(Requested, Ensured, Skipped:0, Failed)`; never throws. - [ ] Alarm writer: EventId = lowercase hex SHA-256 of `AlarmId|EventKind|TimestampUtc.Ticks|User|Comment|Message` (UTF-8, `|` separator, null ⇒ empty); insert-if-absent; per-event outcome per spec (transient classifier from T8). - [ ] Integration: each kind written then read back by direct SQL equals input; writing the same batch twice leaves row count unchanged and returns true; a 10 000-value batch completes < 5 s on the rig; provisioning upsert changes DataType on re-run; same alarm event twice ⇒ one row, both `Ack`; an over-long `AlarmTypeName` (> 100) ⇒ `PermanentFail`. **Verify:** `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests --filter "FullyQualifiedName~Write|FullyQualifiedName~Provisioning|FullyQualifiedName~AlarmWriter"` → green. ```json:metadata {"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerHistorianValueWriter.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerTagProvisioner.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerAlarmHistorianWriter.cs"],"acceptanceCriteria":["idempotent redelivery","never throws","deterministic EventId"],"modelTier":"standard","model":"sonnet","wave":3} ``` --- ### Task 10: SQL Server reads — raw, at-time, events, processed (push-down) **Goal:** `SqlServerHistorianDataSource : IHistorianDataSource` per spec §4 "Reads". > **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:** > - `maxValuesPerNode == 0` is the node manager's unbounded sentinel ⇒ `TOP(2147483647)`; test it. (#8) > - Events: `TOP(@maxEvents + 1)` (maxEvents 0 ⇒ unbounded); when the extra row is present drop it and return a non-null opaque ContinuationPoint (e.g. UTF-8 of `EventTimeUtc.Ticks|EventId`) — the interface says MUST. Test `maxEvents=2` over 3 rows. (#9) > - Processed: cap the bucket count at 100 000; above it throw `ArgumentOutOfRangeException` and document that the node manager surfaces it as a per-node Bad. Check what the node manager's processed path does with a thrown non-aggregate exception and match it. (#24) > - Tag lookup uses T8's `SqlServerTagCatalog` read-only lookup. All timestamp tests non-ms-aligned. (#7, #12) **Files (create only):** `.../SqlServerHistorianDataSource.cs`, `.../SqlAggregateQueries.cs` (one method per aggregate family returning the SQL text), tests `tests/.../SqlServer.IntegrationTests/RawReadTests.cs`, `AtTimeReadTests.cs`, `EventReadTests.cs`, `ProcessedReadTests.cs`. **Acceptance Criteria:** - [ ] Tag lookup is read-only (`SELECT TagId FROM Tag WHERE Name=@n`, cached on hit only) — reads never create tags. - [ ] Raw forward/reverse/limit exactly per spec; `DataValueSnapshot` built with `Value.ToObject()`, `StatusCode`, source + server timestamps (inspect `DataValueSnapshot`'s ctor first). - [ ] AtTime: single statement with `TimeRow` TVP and two `OUTER APPLY (SELECT TOP 1 …)`; interpolation/stepping/no-prior/bad-neighbour rules per spec; status built from `SqlHistorianStatusCodes` + `HistorianInfoBits`. - [ ] Processed: generates every interval in `[start,end)` (empty ⇒ `BadNoData`), the aggregate table in spec §4, Calculated bit on every result, Uncertain when any Bad present, `BadAggregateInvalidInputs` for numeric aggregates on String/DateTime tags, `HistoryAggregateNotSupportedException` for the rest. `interval <= 0` ⇒ single interval `[start,end)`. - [ ] Reverse processed (`end < start`) ⇒ swap and return descending (document). - [ ] `GetHealthSnapshot()` tracks totals/consecutive failures/last error (the bootstrap error surfaces here). - [ ] Integration fixture data: tag `A` with samples at t0+0s=10, +10s=20, +20s=Bad(30), +30s=40, +40s=50 (Double); tag `S` strings; `B` booleans. Hand-computed expectations for each supported aggregate over 20 s intervals from t0 — write them as literal expected values in the test (e.g. interval [t0,t0+20s): Average 15, Min 10, Max 20, Total 30, Count 2, Start 10, End 20, Delta 10, Range 10, PercentGood 100; interval [t0+20s,t0+40s): usable {40}, Average 40 with Uncertain status, PercentBad 50). - [ ] Falsification: drop the Bad-sample filter ⇒ the Uncertain interval test fails. **Verify:** `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests --filter "FullyQualifiedName~Read"` → green. ```json:metadata {"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerHistorianDataSource.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlAggregateQueries.cs"],"acceptanceCriteria":["raw fwd/rev","AtTime interp/step","aggregate table with hand-computed literals","unsupported aggregates throw"],"modelTier":"frontier","model":"opus","wave":3} ``` --- ### Task 11: SQL Server updater **Goal:** `SqlServerHistorianDataUpdater : IHistorianDataUpdater` per spec §4 "Updates". > **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:** > - Use T8's `SqlServerTagCatalog` for create-on-Insert/Update (remove the "private MERGE" fallback). Sample `MERGE … WITH (HOLDLOCK)`. (#10, #12) > - "Null timestamp" means `DateTime.MinValue` (the node manager maps it to `BadInvalidTimestamp` before calling, but the updater must still treat `MinValue` the same defensively). (#21) > - Non-ms-aligned ticks in every Replace/DeleteAtTime test. (#7) **Files (create only):** `.../SqlServerHistorianDataUpdater.cs`; tests `tests/.../SqlServer.IntegrationTests/UpdateTests.cs`. **Acceptance Criteria:** - [ ] One transaction per call; `MERGE Sample AS t USING @rows AS r ON t.TagId=@tagId AND t.SourceTimeUtc=r.SourceTimeUtc` with `OUTPUT $action, r.SourceTimeUtc`; Insert uses `WHEN NOT MATCHED THEN INSERT` only (matched ⇒ BadEntryExists), Replace `WHEN MATCHED THEN UPDATE` only (not matched ⇒ BadNoEntryExists), Update both. Results mapped back to the caller's original order (including within-request duplicate timestamps: second occurrence ⇒ `BadEntryExists` for Insert, last-wins for Replace/Update — document). - [ ] Null timestamp ⇒ `BadInvalidTimestamp` for that item, others proceed. - [ ] Delete raw `[start,end)` ⇒ `Good` with rows deleted, `BadNoData` when 0; DeleteAtTime per time. - [ ] Unknown tag: Insert/Update create it via T8's `SqlServerTagCatalog`; Replace/Delete ⇒ BadNoEntryExists/BadNoData. - [ ] Integration test per row of the spec's result table + the duplicate/null cases; replaced value is visible via direct SQL with the new status. **Verify:** `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests --filter "FullyQualifiedName~Update"` → green. ```json:metadata {"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerHistorianDataUpdater.cs"],"acceptanceCriteria":["Part 11 result codes","order preserved","single transaction"],"modelTier":"standard","model":"sonnet","wave":3} ``` --- ### Task 12: SQL Server provider + Host registration **Goal:** `Backend=SqlServer` works end to end in the Host. > **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:** > - SqlServer option validation lives in `ServerHistorianOptionsValidator`, consumer-gated (see T5 amendment) — add it there (this task owns that file in wave 4), not in `Resolve`. (#11) > - Add `src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/ServerHistorianOptionsValidator.cs` to Files. One shared `SqlServerSchemaBootstrapper` + `SqlServerTagCatalog` per provider. (#11, #12) **Files:** - Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerHistorianProvider.cs` (one shared bootstrapper + tag catalog instance across the five seams; `SupportedKinds` = all six kinds) - Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs` (add `"SqlServer"`, binding `ServerHistorian:SqlServer` to `SqlServerHistorianOptions` and failing on its `Validate()` errors), `src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj` (ProjectReference), `Program.cs` only if T5 left a seam that needs it - Test: `tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/Configuration/HistorianBackendRegistryTests.cs` (extend), SqlServer.Tests `SqlServerHistorianProviderTests.cs` **Acceptance Criteria:** - [ ] `Backend=SqlServer` with no ConnectionString ⇒ host-start validation failure naming `ServerHistorian:SqlServer:ConnectionString`. - [ ] DI: `IHistorianDataSource` ⇒ `SqlServerHistorianDataSource`, `IHistorianValueWriter` ⇒ SqlServer writer, `HistorianUpdaterHolder.Updater` non-null, recorder `supportedKinds` contains String and Empty. - [ ] The Endpoint requirement does not fire for SqlServer. **Verify:** `dotnet build ZB.MOM.WW.OtOpcUa.slnx && dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests --filter "FullyQualifiedName~Historian"` → green. ```json:metadata {"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerHistorianProvider.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj"],"acceptanceCriteria":["validation names the key","DI resolves SqlServer types","updater present"],"modelTier":"standard","model":"sonnet","wave":4} ``` --- ### Task 13: Over-the-wire HistoryUpdate test **Goal:** Prove HistoryUpdate works through a real SDK client session (the encoding/dispatch the node-manager unit tests mock around). > **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:** > - Install the fake updater (`Nm.HistorianDataUpdater = fake`, same slot the rig uses for `Nm.Authorizer`, cf. `AclKeyExpiryEnforcementTests.cs:141-215`) **before** materialising the historized node. Add a control test that materialises first, then installs the updater, and proves HistoryUpdate returns `BadHistoryOperationUnsupported` (documents the no-retro-update rule). Assert the per-node `StatusCode` as well as `OperationResults`. (#4, #13, #15) **Files (create only):** `tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/HistoryUpdateOverTheWireTests.cs` (use the existing integration harness that `ConditionMethodEventTests` uses; inject an in-memory fake `IHistorianDataUpdater` + `IHistorianDataSource` sharing one dictionary). **Acceptance Criteria:** - [ ] Client HistoryUpdate Insert of 2 values on a historized node ⇒ `GoodEntryInserted` ×2; HistoryRead raw returns them; Insert again ⇒ `BadEntryExists`; DeleteAtTime one ⇒ read returns one. - [ ] A session without the grant ⇒ `BadUserAccessDenied`, fake not called. - [ ] The node's AccessLevel read over the wire includes `HistoryWrite`. **Verify:** `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests --filter "FullyQualifiedName~HistoryUpdateOverTheWire"` → green. ```json:metadata {"files":["tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/HistoryUpdateOverTheWireTests.cs"],"acceptanceCriteria":["insert/read/duplicate/delete over the wire","deny case","HistoryWrite bit visible"],"modelTier":"standard","model":"sonnet","wave":3} ``` --- ### Task 14: Documentation **Goal:** Docs match the shipped behaviour. **Files:** `docs/Historian.md`, `docs/Configuration.md`, `docs/security.md`, `CLAUDE.md` (Historian / HistoryRead section: add backend selection, SQL Server, HistoryUpdate, the real-status fix, and the gateway C-002 quality limitation; keep it a short subsection), `docs/drivers/README` or index if historians are listed there. **Acceptance Criteria:** - [ ] Every config key in spec §2 documented with default and validation rule. - [ ] Aggregate table + "simple per-sample definitions, not full Part 13" caveat. - [ ] HistoryUpdate: operations, result codes, authorization (flag on/off), AccessLevel bit, unsupported cases. - [ ] Docker-dev recipe for a SQL historian (`OtOpcUaHistorian` on the `sql` container) and the integration-test env var. - [ ] No claim the docs make is untested — cross-check each against the merged code. **Verify:** controller reads the diff; `grep -n "Backend" docs/Configuration.md`. ```json:metadata {"files":["docs/Historian.md","docs/Configuration.md","docs/security.md","CLAUDE.md"],"acceptanceCriteria":["keys documented","aggregate caveat","HistoryUpdate auth documented"],"modelTier":"standard","model":"sonnet","wave":5} ``` --- ### Final review (fable), then Task 15: Live gate on docker-dev (controller) **Final review:** dispatch a **fable** reviewer over `git diff master...feat/pluggable-historian` with the spec; it must specifically check #491 (`Self` after await), #537/#538 unchanged, every new status literal is a named const, ALLOW tests exist, no `Directory.Packages.props` edit, gateway Good-bytes unchanged. One fix dispatch (opus) + scoped re-review (sonnet), then adjudicate. **Task 15 — Live gate (controller only; not delegated).** **USER-ORDERED GATE — NON-SKIPPABLE.** Close only after each item below is re-validated with captured output. 1. `CREATE DATABASE OtOpcUaHistorian` on `otopcua-dev-sql-1`; set on central-1 and central-2 (compose env): `ServerHistorian__Enabled=true`, `ServerHistorian__Backend=SqlServer`, `ServerHistorian__SqlServer__ConnectionString=Server=sql,1433;Database=OtOpcUaHistorian;User Id=sa;Password=OtOpcUa!Dev123;TrustServerCertificate=True`, `ContinuousHistorization__Enabled=true` + `OutboxPath`. Rebuild images (the rig image goes stale), `up -d`. 2. Mark a rig tag historized (one numeric, one string) and deploy. Evidence: `hist.Sample` row count advances by exactly one per published value (Primary-only), status column real. 3. `Client.CLI historyread` raw + processed (Average, 10 s) on the raw NodeId AND its UNS NodeId — same rows. 4. `Client.CLI historyupdate` insert (new timestamp) ⇒ `GoodEntryInserted`; repeat ⇒ `BadEntryExists`; replace ⇒ `GoodEntryReplaced`; delete-at-time ⇒ `Good`; each confirmed by `historyread` and by direct SQL. Use an API key whose principal has a Cluster-scoped `HistoryUpdate` grant; a key with only WriteOperate ⇒ `BadUserAccessDenied`. 5. Stop the Primary container; the Secondary resumes recording into the same table; no duplicate `(TagId, SourceTimeUtc)` (guaranteed by PK — also check count growth rate stays 1×); restart. 6. Stop the driver's device (or blackhole) **while the driver is still publishing** (before #527 trips Reconnecting) ⇒ a row with `ValueKind = 0 (Empty)` and the driver's Bad code (Modbus: `BadCommunicationError 0x80050000`). Watchdog `BadNoCommunication` stamps are node-only and are NOT historized — do not expect them. 7. Record evidence in `docs/plans/2026-10-09-pluggable-historian-live-gate.md`; restore the rig config (Backend off) unless the user wants it kept. ```json:metadata {"files":["docs/plans/2026-10-09-pluggable-historian-live-gate.md"],"acceptanceCriteria":["Primary-only one row per value","historyread raw+processed via raw and UNS NodeIds","historyupdate insert/duplicate/replace/delete codes","WriteOperate-only key denied","failover continues recording","Bad sample stored Bad"],"userGate":true,"tags":["user-gate"],"verifyCommand":"Client.CLI historyread/historyupdate against opc.tcp://localhost:4840 + sqlcmd on otopcua-dev-sql-1","modelTier":"frontier","model":"controller","wave":5} ``` --- ## Finish Merge `feat/pluggable-historian` → `master` only with the user's go-ahead; push only with approval. Then update the scadaproj `sister-project-history/otopcua.md` (on scadaproj `main`, pushed) and the index row only if the summary changed (driver/backends list). File Gitea follow-ups: QuestDB backend spec; retention/purge; Part 11 modified values; time-weighted aggregates.