a6643cedf9
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
646 lines
66 KiB
Markdown
646 lines
66 KiB
Markdown
# Pluggable Historian Backends Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:subagent-driven-development (recommended) or superpowers-extended-cc:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** Make the historian backend pluggable (`ServerHistorian:Backend`), carry typed values + real status end to end, ship a SQL Server backend, and add the OPC UA HistoryUpdate service.
|
||
|
||
**Architecture:** A Runtime-level `IHistorianBackendProvider` builds the five historian seams (data source, value writer, alarm writer, provisioning, updater); the Host picks one by name. The existing recorder/outbox/alarm S&F pipelines are unchanged in structure and now carry a typed `HistorianValue` + `uint StatusCode`. SQL Server stores samples in a `(TagId, SourceTimeUtc)`-clustered table and pushes aggregates down to T-SQL.
|
||
|
||
**Tech Stack:** .NET 10, Akka.NET, OPC Foundation UA .NET Standard, Microsoft.Data.SqlClient (already pinned centrally), FASTER log (existing), xUnit v3 + Shouldly.
|
||
|
||
**Spec:** `docs/plans/2026-10-09-pluggable-historian-design.md` — the binding authority. Read it before any task.
|
||
|
||
## Global Constraints
|
||
|
||
- **No new NuGet packages, no version changes.** `Directory.Packages.props` is not edited by any task. Adding `<PackageReference Include="Microsoft.Data.SqlClient" />` (already pinned) to a new project is allowed.
|
||
- Recording stays **Primary-only** (#537 capture gate) and **at-least-once** with the #538 per-tag quarantine and all-or-nothing ack. Do not restructure `DrainOnceAsync`.
|
||
- In any Akka actor handler: capture `IActorRef self = Self` (and `Sender`) **before** the first `await`; never touch `Self`/`Context` after an `await …ConfigureAwait(false)` (#491).
|
||
- Test fakes for async seams used by actors must complete on a **pool thread** (`await Task.Delay(1).ConfigureAwait(false)`), never synchronously and never via `Task.Yield()`.
|
||
- Every OPC UA status code a new class emits is a **named `const uint`** in an internal `*StatusCodes` class, and that assembly must be visible to `tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/StatusCodeParityTests.cs` (it scans its bin dir — add a ProjectReference).
|
||
- Authorization changes need an **ALLOW** test (a deny-only test passes against a build that never consults the gate).
|
||
- A regression test must be shown to fail with the production change reverted (falsification) — record the observation in the task report.
|
||
- The gateway backend's observable behaviour (bytes sent for Good numeric samples, provisioning skips, read paths) does not change.
|
||
- Commit messages end with `Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe`. Subagents never push and never dispatch subagents.
|
||
- macOS zsh: quote globs; `tea` needs `</dev/null`.
|
||
|
||
**User decisions (already made):**
|
||
- Approach A (provider seam over the existing five interfaces).
|
||
- SQLite dropped; SQL Server is the first new backend; QuestDB is a later spec.
|
||
- "Update" = OPC UA HistoryUpdate exposed to clients (Insert/Replace/Update/DeleteRaw/DeleteAtTime).
|
||
- Typed values (Double/Int64/Boolean/String/DateTime + Empty), not numeric-only.
|
||
- Aggregates pushed down per database (T-SQL), launch subset per spec §4.
|
||
- Flag-off authorization uses a NEW `HistoryUpdate` role, not `WriteOperate`.
|
||
- Gateway config keys stay top-level; new backends get subsections.
|
||
- Testing target: the docker-dev `sql` container (`otopcua-dev-sql-1`, `localhost,14330`).
|
||
|
||
---
|
||
|
||
## Execution model: waves, models, and parallel safety
|
||
|
||
### Model per role
|
||
|
||
| Role | Model | Why |
|
||
|---|---|---|
|
||
| Plan review (before execution) | **fable** | user-requested |
|
||
| Implementer, mechanical/standard tasks (T1, T4, T5, T7, T8, T9, T11, T12, T13, T14) | **sonnet** | multi-file but fully specified |
|
||
| Implementer, judgment-heavy tasks (T2 outbox crash boundary, T3 recorder actor, T6 node-manager HistoryUpdate + auth, T10 reads/aggregates) | **opus** | actor concurrency, auth semantics, SQL/Part 13 semantics |
|
||
| Per-task reviewer | **sonnet** for sonnet tasks; **opus** for T2/T3/T6/T10 | scale to risk |
|
||
| Fix rounds 4–5 escalation | one tier up (sonnet→opus, opus→fable) | per SDD skill |
|
||
| Final whole-branch review | **fable** | cross-task integration risk |
|
||
| Live gate (T15) | controller (this session) | needs rig + judgement; never delegated |
|
||
|
||
Always pass `model` explicitly on every Agent dispatch.
|
||
|
||
### Waves
|
||
|
||
```
|
||
Wave 1 (serial): T1 contracts + compile ripple [sonnet]
|
||
Wave 2 (parallel): T2 outbox v2 [opus] ‖ T3 recorder ‖ T4 gateway provider ‖ T6 server HistoryUpdate ‖ T7 CLI ‖ T8 SQL skeleton+schema
|
||
Wave 3 (parallel): T5 registry+Host ‖ T9 SQL writes ‖ T10 SQL reads ‖ T11 SQL updater ‖ T13 over-the-wire test
|
||
Wave 4 (serial): T12 SQL provider + Host registration
|
||
Wave 5 (serial): T14 docs → final fable review → T15 live gate (controller)
|
||
```
|
||
|
||
Dependencies: T2,T3,T4,T6,T7,T8 ← T1. T5 ← T4. T9,T10,T11 ← T8. T13 ← T6. T12 ← T5,T9,T10,T11. T14 ← T12,T13. T15 ← T14 + final review.
|
||
|
||
### Parallel-safety rules (binding)
|
||
|
||
1. **One git worktree per parallel task**, created by the controller under `.worktrees/ph-T<N>` from the integration branch `feat/pluggable-historian` at the wave's start commit. Run `git -C <wt> log -1` and confirm the base SHA in the dispatch (worktrees can start stale — reset first).
|
||
2. **File ownership.** A task edits only the files its "Files" list names. The shared hot files have exactly one owner per wave:
|
||
|
||
| Shared file | Owner |
|
||
|---|---|
|
||
| `ZB.MOM.WW.OtOpcUa.slnx` | T8 (adds both SqlServer projects) |
|
||
| `src/Server/ZB.MOM.WW.OtOpcUa.Host/Program.cs` | T5 (wave 3), then T12 (wave 4) |
|
||
| `.../Host/Configuration/ServerHistorianOptionsValidator.cs` | T5 (wave 3), then T12 (wave 4) |
|
||
| `.../Runtime/Historian/ContinuousHistorizationMetrics.cs` | T5 |
|
||
| `.../OpcUaServer/OtOpcUaSdkServer.cs`, `.../Host/OpcUa/OtOpcUaServerHostedService.cs` | T6 |
|
||
| `tests/Server/.../Runtime.Tests/Historian/ContinuousHistorizationRecorderTests.cs` | T1 (ripple) then T3 |
|
||
| `Core.Abstractions/Historian/IHistorizationOutbox.cs` | T1 |
|
||
| `tests/Core/.../Core.Abstractions.Tests/*.csproj` | T8 |
|
||
| `src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs` | T6 only |
|
||
| `.../Runtime/Historian/ContinuousHistorizationRecorder.cs` | T1 (ripple) then T3 only |
|
||
| `.../Driver.Historian.SqlServer/*.csproj` | T8 only (T9–T11 add `.cs` files only; csproj uses default globbing + `EmbeddedResource` for `Schema/*.sql` set by T8) |
|
||
|
||
A task that discovers it needs an un-owned shared file stops and reports `NEEDS_CONTEXT` rather than editing it.
|
||
3. **New-file-only parallelism in the SQL project.** T9/T10/T11 each create their own files and their own test files; none edits another's. Shared SQL helpers live in T8's `SqlHistorianConnection.cs`/`SampleRowMapper.cs`; if a wave-3 task needs a helper change, it adds a private helper in its own file instead.
|
||
4. **Merge order** at wave end: the controller merges worktree branches into `feat/pluggable-historian` in ascending task number, running `dotnet build ZB.MOM.WW.OtOpcUa.slnx` after each merge and the touched test projects after the last. A conflict ⇒ stop and resolve on the integration branch, never inside a worktree.
|
||
5. **Integration-test DBs are per-test-run** (`OtOpcUaHistorian_T_<guid>`), so parallel T9/T10/T11 suites against the same `sql` container cannot collide.
|
||
6. Subagents do not push, merge, or dispatch subagents. Only the controller merges.
|
||
|
||
### Common commands
|
||
|
||
```bash
|
||
dotnet build ZB.MOM.WW.OtOpcUa.slnx
|
||
dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests
|
||
dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests --filter "FullyQualifiedName~Historian"
|
||
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests
|
||
export HISTORIAN_SQL_TEST_CONNECTION='Server=localhost,14330;User Id=sa;Password=OtOpcUa!Dev123;TrustServerCertificate=True;Encrypt=False'
|
||
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests
|
||
```
|
||
|
||
The rig `sql` container must be up: `docker ps --filter name=otopcua-dev-sql-1`. If down: `cd docker-dev && docker compose up -d sql`.
|
||
|
||
---
|
||
|
||
### Task 1: Typed historian contracts + compile ripple
|
||
|
||
**Goal:** Introduce `HistorianValue`, `HistorianValueKind`, `HistoryUpdateMode`, `IHistorianDataUpdater`, `IHistorianBackendProvider`, change `HistorizationValue`/`HistorizationOutboxEntry` to typed+status, and keep the whole solution compiling with **no behaviour change**.
|
||
|
||
> **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:**
|
||
> - Add `long UnreadableCount => 0;` as a default interface member on `IHistorizationOutbox` (Core.Abstractions) — T2 overrides it, T5 binds the gauge. (#6)
|
||
> - The record call sites to update are explicit: `grep -rn "HistorizationOutboxEntry(\|HistorizationValue(" src tests --include='*.cs'` (≈17 sites in 7 files incl. `GatewayLiveIntegrationTests.cs`, `HistorizationOutboxEntrySerializerTests.cs`, `GatewayHistorianValueWriterTests.cs`, `ContinuousHistorizationRecorderTests.cs`). Every one of them is in T1's Files. Do not stop at an unlisted one. (#3)
|
||
|
||
**Files:**
|
||
- Create: `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/HistorianValue.cs`
|
||
- Create: `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianDataUpdater.cs`
|
||
- Create: `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/IHistorianBackendProvider.cs`
|
||
- Modify: `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianValueWriter.cs` (`HistorizationValue`)
|
||
- Modify: `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/HistorizationOutboxEntry.cs`
|
||
- Modify: `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorizationOutbox.cs` (default member `UnreadableCount`)
|
||
- Modify (ripple only): `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationRecorder.cs` (lines ~548 entry construction, ~692 drain mapping), `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/GatewayHistorianValueWriter.cs`, `.../Recorder/HistorizationOutboxEntrySerializer.cs`, and every test that constructs these records (find with `grep -rn "HistorizationOutboxEntry(\|HistorizationValue(" src tests --include='*.cs'`).
|
||
- Test: `tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/Historian/HistorianValueTests.cs`
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] `HistorianValue.TryFromObject` implements exactly the spec §3 table; tests cover every row incl. `ulong` above `long.MaxValue` ⇒ Double, `DateTime` local ⇒ UTC ticks, `null` ⇒ Empty, `int[]` ⇒ false.
|
||
- [ ] `TryGetDouble` true for Double/Int64/Boolean only; `ToObject` round-trips each kind (`DateTime` returns `DateTimeKind.Utc`).
|
||
- [ ] Solution builds; all pre-existing historian tests pass unchanged in assertion content (only constructor call sites updated).
|
||
- [ ] Ripple preserves behaviour: recorder still coerces numerics only (via `TryFromObject` + `TryGetDouble`, building `HistorianValue.FromDouble(numeric)`) and still stamps Good (`StatusCode = 0`); serializer still writes the legacy layout using `Value.Number` and quality `192`; gateway writer maps `Value.Number` and quality `192`.
|
||
|
||
**Verify:** `dotnet build ZB.MOM.WW.OtOpcUa.slnx && dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests --filter "FullyQualifiedName~HistorianValue" && dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests && dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests --filter "FullyQualifiedName~Historiz"` → all green.
|
||
|
||
**Steps:**
|
||
|
||
- [ ] **Step 1: Write `HistorianValueTests` (failing — type does not exist).** Cases:
|
||
|
||
```csharp
|
||
[Theory]
|
||
[InlineData(1.5d, HistorianValueKind.Double)]
|
||
[InlineData(1.5f, HistorianValueKind.Double)]
|
||
[InlineData(7, HistorianValueKind.Int64)]
|
||
[InlineData((short)7, HistorianValueKind.Int64)]
|
||
[InlineData((byte)7, HistorianValueKind.Int64)]
|
||
[InlineData(7u, HistorianValueKind.Int64)]
|
||
[InlineData(7L, HistorianValueKind.Int64)]
|
||
[InlineData(true, HistorianValueKind.Boolean)]
|
||
[InlineData("x", HistorianValueKind.String)]
|
||
public void Maps_clr_values_to_kinds(object input, HistorianValueKind expected)
|
||
{
|
||
HistorianValue.TryFromObject(input, out var v).ShouldBeTrue();
|
||
v.Kind.ShouldBe(expected);
|
||
}
|
||
// plus: decimal 2.5m ⇒ Double 2.5; ulong.MaxValue ⇒ Double; (ulong)5 ⇒ Int64 5;
|
||
// null ⇒ Empty (true); new[]{1,2} ⇒ false; new object() ⇒ false;
|
||
// DateTime(2026,1,1,0,0,0,Local) ⇒ DateTime kind, Integer == ToUniversalTime().Ticks, ToObject() Kind Utc;
|
||
// FromBoolean(true): Integer 1, Number 1, TryGetDouble ⇒ 1; FromString("a").TryGetDouble ⇒ false; Empty.ToObject() ⇒ null.
|
||
```
|
||
|
||
- [ ] **Step 2: Run, confirm compile failure.**
|
||
- [ ] **Step 3: Implement `HistorianValue.cs`:**
|
||
|
||
```csharp
|
||
namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||
|
||
/// <summary>The value kinds a historian backend can store. Persisted as a byte (SQL ValueKind, outbox v2) — never renumber.</summary>
|
||
public enum HistorianValueKind : byte { Empty = 0, Double = 1, Int64 = 2, Boolean = 3, String = 4, DateTime = 5 }
|
||
|
||
/// <summary>A typed historized value. Number carries Double (and the double view of Int64/Boolean);
|
||
/// Integer carries Int64, Boolean (1/0) and DateTime (UTC ticks); Text carries String.</summary>
|
||
public readonly record struct HistorianValue(HistorianValueKind Kind, double Number, long Integer, string? Text)
|
||
{
|
||
public static HistorianValue Empty => default;
|
||
public static HistorianValue FromDouble(double v) => new(HistorianValueKind.Double, v, 0, null);
|
||
public static HistorianValue FromInt64(long v) => new(HistorianValueKind.Int64, v, v, null);
|
||
public static HistorianValue FromBoolean(bool v) => new(HistorianValueKind.Boolean, v ? 1d : 0d, v ? 1 : 0, null);
|
||
public static HistorianValue FromString(string v) => new(HistorianValueKind.String, 0d, 0, v ?? throw new ArgumentNullException(nameof(v)));
|
||
public static HistorianValue FromDateTime(DateTime v) =>
|
||
new(HistorianValueKind.DateTime, 0d, (v.Kind == DateTimeKind.Local ? v.ToUniversalTime() : DateTime.SpecifyKind(v, DateTimeKind.Utc)).Ticks, null);
|
||
|
||
public static bool TryFromObject(object? value, out HistorianValue result)
|
||
{
|
||
switch (value)
|
||
{
|
||
case null: result = Empty; return true;
|
||
case double d: result = FromDouble(d); return true;
|
||
case float f: result = FromDouble(f); return true;
|
||
case decimal m: result = FromDouble((double)m); return true;
|
||
case sbyte sb: result = FromInt64(sb); return true;
|
||
case byte b: result = FromInt64(b); return true;
|
||
case short s: result = FromInt64(s); return true;
|
||
case ushort us: result = FromInt64(us); return true;
|
||
case int i: result = FromInt64(i); return true;
|
||
case uint ui: result = FromInt64(ui); return true;
|
||
case long l: result = FromInt64(l); return true;
|
||
case ulong ul: result = ul <= long.MaxValue ? FromInt64((long)ul) : FromDouble(ul); return true;
|
||
case bool bo: result = FromBoolean(bo); return true;
|
||
case string str: result = FromString(str); return true;
|
||
case DateTime dt: result = FromDateTime(dt); return true;
|
||
default: result = Empty; return false; // arrays, references, structures ⇒ not historizable
|
||
}
|
||
}
|
||
|
||
public bool TryGetDouble(out double value)
|
||
{
|
||
switch (Kind)
|
||
{
|
||
case HistorianValueKind.Double:
|
||
case HistorianValueKind.Int64:
|
||
case HistorianValueKind.Boolean: value = Number; return true;
|
||
default: value = 0d; return false;
|
||
}
|
||
}
|
||
|
||
public object? ToObject() => Kind switch
|
||
{
|
||
HistorianValueKind.Double => Number,
|
||
HistorianValueKind.Int64 => Integer,
|
||
HistorianValueKind.Boolean => Integer != 0,
|
||
HistorianValueKind.String => Text,
|
||
HistorianValueKind.DateTime => new DateTime(Integer, DateTimeKind.Utc),
|
||
_ => null,
|
||
};
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: `IHistorianDataUpdater.cs`** — exactly the spec §4 "Updates" interface + `HistoryUpdateMode` enum, with XML docs naming the per-item status codes.
|
||
- [ ] **Step 5: `IHistorianBackendProvider.cs`** in `ZB.MOM.WW.OtOpcUa.Runtime.Historian` — exactly spec §2.
|
||
- [ ] **Step 6: Change the records:**
|
||
|
||
```csharp
|
||
public readonly record struct HistorizationValue(DateTime? TimestampUtc, HistorianValue Value, uint StatusCode);
|
||
public sealed record HistorizationOutboxEntry(Guid Id, string Tag, HistorianValue Value, uint StatusCode, DateTime TimestampUtc);
|
||
```
|
||
|
||
- [ ] **Step 7: Ripple, behaviour-preserving.** Recorder line ~527: replace `TryCoerceToDouble(msg.Value, out double numeric)` with `HistorianValue.TryFromObject(msg.Value, out var hv) && hv.TryGetDouble(out double numeric)` (same drop semantics — strings/null/DateTime still dropped as non-numeric); entry = `new HistorizationOutboxEntry(Guid.NewGuid(), historianName, HistorianValue.FromDouble(numeric), 0u /*Good*/, timestampUtc)`; delete `TryCoerceToDouble`. Drain line ~692: `new HistorizationValue(e.TimestampUtc, e.Value, e.StatusCode)`. Serializer: write `entry.Value.Number` and `(ushort)192`; read back as `FromDouble(value)`, status `0`. Gateway writer: `NumericValue = value.Value.Number, Quality = 192`. Keep `GoodQuality` const for now (T3 removes it). Update test call sites.
|
||
- [ ] **Step 8: Build + run Verify.** Commit: `feat(historian): typed HistorianValue contracts, IHistorianDataUpdater, IHistorianBackendProvider (no behaviour change)`.
|
||
|
||
```json:metadata
|
||
{"files":["src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/HistorianValue.cs","src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianDataUpdater.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/IHistorianBackendProvider.cs","src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianValueWriter.cs","src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/HistorizationOutboxEntry.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationRecorder.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/GatewayHistorianValueWriter.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/HistorizationOutboxEntrySerializer.cs"],"acceptanceCriteria":["TryFromObject matches spec table","solution builds, existing historian tests green","no behaviour change"],"modelTier":"standard","model":"sonnet","wave":1}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 2: Outbox format v2 with legacy read
|
||
|
||
**Goal:** Serialize every `HistorianValueKind` + `uint StatusCode`; read pre-upgrade entries; drop (and count) unreadable entries instead of throwing.
|
||
|
||
> **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:**
|
||
> - **Model: opus** (FIFO invariant on the crash boundary). (#1)
|
||
> - `FasterLogHistorizationOutbox.RecoverState()` (ctor) ALSO deserializes every record. Use `TryDeserialize` in **both** `RecoverState` and `PeekBatchAsync`. In `RecoverState`, index an unreadable record in `_live/_index` under a synthetic `Guid.NewGuid()` with its address so FIFO truncation can pass it; in Peek skip it (and make sure an unreadable entry at the head is removable so it can't pin the head — e.g. Peek auto-removes leading unreadable entries). `UnreadableCount` (overrides T1's default member) increments in both. (#1)
|
||
> - Required test: write a raw garbage record straight into the FasterLog directory (open `FasterLog` directly), **restart** the outbox, prove the ctor succeeds, the following good entry drains, and the head advances past the garbage. (#1)
|
||
> - Legacy/v2 ambiguity: when the 4-byte magic matches but v2 structural validation fails, fall back to the legacy exact-length test before declaring unreadable. (#18)
|
||
> - Drop the meter bullet: T5 binds the gauge via `IHistorizationOutbox.UnreadableCount`. (#6)
|
||
|
||
**Files:**
|
||
- Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/HistorizationOutboxEntrySerializer.cs`
|
||
- Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/FasterLogHistorizationOutbox.cs` (only the deserialize call site in Peek: skip+count unreadable entries; expose `long UnreadableCount`)
|
||
- Test: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests/Recorder/HistorizationOutboxEntrySerializerV2Tests.cs`
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] v2 layout exactly: `"HV2\0"`(4) `ver=2`(1) `id`(16) `tagLen`(i32 LE) `tag`(UTF-8) `kind`(1) payload `status`(u32 LE) `ts`(i64 LE `ToBinary`). Payload: Empty none; Double f64; Int64/Boolean/DateTime i64; String i32 len + UTF-8.
|
||
- [ ] Round-trip test per kind incl. empty string, a 70-KB string, `NaN`, `double.NegativeInfinity`, `long.MinValue`, status `0x80310000`.
|
||
- [ ] **Pinned legacy bytes:** a hex literal produced by today's serializer (generate it once from the T1-era `Serialize` for `Id=00000000-0000-0000-0000-000000000001, Tag="t", Value=42.5, Quality=192, ts=2026-01-01T00:00:00Z` and paste the bytes into the test) deserializes to `Double 42.5`, status `0`.
|
||
- [ ] `TryDeserialize` returns false (never throws) for: 3 random bytes, magic with ver=9, magic with truncated payload, legacy-looking bytes whose length disagrees with `38+tagLen`. `FasterLogHistorizationOutbox` skips such an entry on peek, advances past it (it must be removable so it can't pin the head — truncate through its id or log-address as the outbox already does for acked entries), and increments `UnreadableCount`; a test proves a corrupt entry between two good ones does not block the second.
|
||
- [ ] Meter: `ContinuousHistorizationMetrics` gets an observable `otopcua.historian.outbox.unreadable` bound alongside the existing outbox gauges — **only if** `BindOutbox` lives in an un-owned file; otherwise report the counter on the outbox and note it for T5. (Owned file check: `ContinuousHistorizationMetrics.cs` is NOT owned by T2 — expose the property only.)
|
||
|
||
**Verify:** `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests --filter "FullyQualifiedName~Outbox"` → green; falsification: with the legacy branch deleted the pinned-bytes test fails.
|
||
|
||
**Steps:** (1) capture the legacy hex from the current serializer via a throwaway test print, paste into the new test; (2) write failing v2 tests; (3) implement `Serialize` (v2 always) + `TryDeserialize(ReadOnlySpan<byte>, out HistorizationOutboxEntry?)` (v2 by magic, else legacy by exact length, else false) and keep a throwing `Deserialize` wrapper only if other callers need it; (4) wire skip+count in the outbox peek; (5) green; commit `feat(historian): outbox format v2 carries typed values + status; legacy entries still read`.
|
||
|
||
```json:metadata
|
||
{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/HistorizationOutboxEntrySerializer.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/FasterLogHistorizationOutbox.cs"],"acceptanceCriteria":["v2 layout","pinned legacy bytes read","corrupt entry skipped and counted, never pins head"],"modelTier":"frontier","model":"opus","wave":2}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 3: Recorder captures typed values with real status, filtered by backend kinds
|
||
|
||
**Goal:** The recorder records `HistorianValue` + `msg.StatusCode` and drops kinds the backend does not support.
|
||
|
||
> **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:**
|
||
> - Add `tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Historian/ContinuousHistorizationRecorderTests.cs` to Files — lines ~337 and ~1149 assert `DroppedNonNumeric` and must follow the rename. (#3)
|
||
> - The recorder remarks ("carrying only (TagId, Value, TimestampUtc) and no quality") are stale: `DependencyMuxActor.cs:114,150` passes `item.StatusCode`. Rewrite them. (verified)
|
||
> - Comms-loss watchdog `BadNoCommunication` stamps are node-only (`FanCommsLossToValueNodes` never reaches the mux) and are NOT historized — say so in the remarks; only a driver's own Bad publish (usually `null` value ⇒ `Empty`) is. (#14)
|
||
|
||
**Files:**
|
||
- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationRecorder.cs`
|
||
- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/ServiceCollectionExtensions.cs` **only** the recorder `Props` construction (~line 485–520) to pass `supportedKinds` — resolve `IHistorianBackendProvider` optionally; absent ⇒ `{Double, Int64, Boolean}` (today's behaviour).
|
||
- Test: `tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Historian/ContinuousHistorizationRecorderTypedValueTests.cs`
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] New ctor/Props parameter `IReadOnlySet<HistorianValueKind> supportedKinds` (default = `{Double, Int64, Boolean}` for existing overloads so other tests compile unchanged).
|
||
- [ ] Capture: `TryFromObject` false ⇒ drop + `_droppedUnsupportedKind++`; kind ∉ supportedKinds ⇒ same; otherwise entry carries the typed value and `msg.StatusCode`.
|
||
- [ ] Status field in the recorder status record renamed `DroppedNonNumeric` → `DroppedUnsupportedKind`; any OTel instrument name stays unchanged (dashboards). `GoodQuality` const and its doc paragraph removed; class remarks updated (the mux DOES carry StatusCode — `VirtualTagActor.DependencyValueChanged`).
|
||
- [ ] Tests (fake writer completes on a pool thread): a `Bad (0x80000000)` numeric sample is written with status `0x80000000`; a string sample is written when kinds include String and dropped+counted when not; `null` with status `BadNoCommunication` is written as Empty when kinds include Empty, dropped otherwise; an `int[]` is always dropped; existing #537 capture-gate and #538 quarantine tests still pass untouched.
|
||
- [ ] Falsification recorded: reverting to the hard-coded Good makes the Bad-status test fail.
|
||
|
||
**Verify:** `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests --filter "FullyQualifiedName~ContinuousHistorization"` → green.
|
||
|
||
**Steps:** failing tests → implement → green → grep for `DroppedNonNumeric` across src/tests and update readers (AdminUI/metrics) **only within Runtime**; if a reader exists outside the owned files, report it for T5 instead of editing. Commit `fix(historian): record the real status code and typed values; backend-supported kinds gate capture`.
|
||
|
||
```json:metadata
|
||
{"files":["src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationRecorder.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/ServiceCollectionExtensions.cs"],"acceptanceCriteria":["real status recorded","kind filter","#537/#538 unchanged","pool-thread fake"],"modelTier":"frontier","model":"opus","wave":2}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 4: Gateway provider + reverse quality mapping
|
||
|
||
**Goal:** `GatewayHistorianProvider : IHistorianBackendProvider`; the gateway writer maps typed value + status to the gateway's numeric + DA quality.
|
||
|
||
**Files:**
|
||
- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/GatewayHistorianProvider.cs`
|
||
- Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Mapping/GatewayQualityMapper.cs` (add `ToDaQuality(uint)`)
|
||
- Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/GatewayHistorianValueWriter.cs`
|
||
- Test: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests/GatewayHistorianProviderTests.cs`, `.../Mapping/GatewayQualityMapperReverseTests.cs`
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] Provider: `Name = "Gateway"`, `SupportedKinds = {Double, Int64, Boolean}`, `CreateUpdater` returns `null`; the four `Create*` delegate to the existing `GatewayHistorian.Create…` (data source/alarm writer/provisioner) and build `GatewayHistorianValueWriter` exactly as `Program.cs` does today (`HistorianGatewayClientAdapter.Create(options, loggerFactory)`). Options come from the ctor (`ServerHistorianOptions`).
|
||
- [ ] `ToDaQuality(Map(q)) == q` for every explicit byte in `Map`'s table; unknown status falls back by severity bits (`0x00…`⇒192, `0x40…`⇒64, `0x80…`⇒0).
|
||
- [ ] Writer: a value with `TryGetDouble` false is skipped (debug log, no throw); Good Double 42.5 produces the same `HistorianLiveValue` (`NumericValue=42.5, Quality=192`) as before T1 — asserted in a test.
|
||
|
||
**Verify:** `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests` → green.
|
||
|
||
**Steps:** tests first → implement → green → commit `feat(historian): GatewayHistorianProvider; status→DA quality reverse mapping`.
|
||
|
||
```json:metadata
|
||
{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/GatewayHistorianProvider.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Mapping/GatewayQualityMapper.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/Recorder/GatewayHistorianValueWriter.cs"],"acceptanceCriteria":["provider shape","reverse mapping inverse","Good bytes unchanged"],"modelTier":"standard","model":"sonnet","wave":2}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 5: Backend registry, config, validators, Host wiring
|
||
|
||
**Goal:** `ServerHistorian:Backend` selects a provider; Program.cs builds every historian seam through it.
|
||
|
||
> **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:**
|
||
> - Gate **all three** gateway checks in `ServerHistorianOptionsValidator` (Endpoint, ApiKey, UseTls⇔scheme) on `Backend == Gateway`, and the same in `ServerHistorianOptions.Validate()` warnings. Tests: `Backend=SqlServer, Enabled=true, no Endpoint/ApiKey ⇒ passes`; control `Backend=Gateway, no ApiKey ⇒ fails` (unchanged). (#2)
|
||
> - Unknown `Backend` ⇒ always fail (no I/O). Backend-specific option validation (SqlServer, added by T12) runs inside `ServerHistorianOptionsValidator` and only when consumed (`ServerHistorian:Enabled || AlarmHistorian:Enabled`) — the existing consumer-gated posture. `HistorianBackendRegistry.Resolve` must not validate. (#11)
|
||
> - Register `HistorianUpdaterHolder.None` **unconditionally** in the driver block; register the real `new HistorianUpdaterHolder(provider.CreateUpdater(sp))` only when `ServerHistorian:Enabled` (HistoryUpdate gated like HistoryRead; last registration wins). (#11)
|
||
> - Add `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationMetrics.cs` to Files: add observable gauge `otopcua.historian.outbox.unreadable` from `IHistorizationOutbox.UnreadableCount` in `BindOutbox`. (#6)
|
||
> - Program.cs is top-level statements and not unit-testable: the "DI graph unchanged" AC is tested as `HistorianBackendRegistry.Resolve(gatewayOpts, cfg).CreateDataSource(sp) is GatewayHistorianDataSource` (and likewise for the writer/alarm writer/provisioner). Do not try to host-start Program. (#22)
|
||
> - Spec wording fix: there are three `GatewayHistorian.Create…` call sites (Program.cs ~241/262/273) plus the inline writer (~313). (#16)
|
||
|
||
**Files:**
|
||
- Create: `src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs`
|
||
- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ServerHistorianOptions.cs` (add `Backend` string, default `"Gateway"`)
|
||
- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/ServerHistorianOptionsValidator.cs`
|
||
- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Host/Program.cs` (lines ~215–318 only)
|
||
- Test: `tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/Configuration/HistorianBackendRegistryTests.cs`, extend `ServerHistorianOptionsValidatorTests`
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] Registry: `static IHistorianBackendProvider Resolve(ServerHistorianOptions o, IConfiguration cfg)`; knows `"Gateway"` now (T12 adds `"SqlServer"`); case-insensitive; unknown ⇒ `OptionsValidationException` naming the value and listing valid names. The registry also exposes `IReadOnlyCollection<string> KnownBackends` used by the validator.
|
||
- [ ] Validator: unknown backend fails; `Endpoint` required only when backend is Gateway; existing tests unchanged in intent.
|
||
- [ ] Program.cs: resolve the provider once; register it as a singleton `IHistorianBackendProvider`; replace the 4 `GatewayHistorian.Create…` lambdas and the inline `GatewayHistorianValueWriter` with `provider.Create…(sp)`; register `IHistorianDataUpdater` via `provider.CreateUpdater(sp)` **only when non-null** (`TryAddSingleton`-style factory is not possible for a null — register conditionally by calling it once at registration time is NOT allowed (needs sp); instead register a small `HistorianUpdaterHolder(IHistorianDataUpdater? Updater)` singleton, and have T6's hosted-service hook read `holder.Updater`). The alarm-only warning text names `serverHistorianOptions.Backend`.
|
||
- [ ] Bind the outbox unreadable gauge from T2 in `ContinuousHistorizationMetrics.BindOutbox` if T2 reported it (this task owns that file this wave).
|
||
- [ ] With `Backend` absent the Host's DI graph resolves the same concrete types as before (test: build the service collection with a Gateway config and assert `IHistorianDataSource` resolves to `GatewayHistorianDataSource`).
|
||
|
||
**Verify:** `dotnet build ZB.MOM.WW.OtOpcUa.slnx && dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests --filter "FullyQualifiedName~Historian"` → green.
|
||
|
||
```json:metadata
|
||
{"files":["src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ContinuousHistorizationMetrics.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/ServerHistorianOptions.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/ServerHistorianOptionsValidator.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/Program.cs"],"acceptanceCriteria":["unknown backend fails boot","Gateway default resolves identical types","updater holder registered"],"modelTier":"standard","model":"sonnet","wave":3}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 6: HistoryUpdate service in the node manager + authorization
|
||
|
||
**Goal:** Clients can HistoryUpdate historized nodes when the backend supplies an updater, gated by ACL (flag on) or the new `HistoryUpdate` role (flag off).
|
||
|
||
> **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:**
|
||
> - SDK pinned `OPCFoundation.NetStandard.Opc.Ua.* 1.5.378.106`. Verified: public `HistoryUpdate(OperationContext, Type detailsType, …)` runs under `Lock`, pre-seeds `errors[ii] = BadHistoryOperationUnsupported`, and admits a variable **only if `(AccessLevel & AccessLevels.HistoryWrite) != 0`**; it releases the lock and dispatches to protected `HistoryUpdateData` / `HistoryDeleteRawModified` / `HistoryDeleteAtTime(ServerSystemContext, IList<…Details>, IList<HistoryUpdateResult>, IList<ServiceResult>, List<NodeHandle>, IDictionary<NodeId,NodeState>)`. The HistoryWrite bit is therefore **dispatch-load-bearing**. (#4)
|
||
> - Add `src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaSdkServer.cs` to Files: `SetHistorianDataUpdater(IHistorianDataUpdater?)` mirroring `SetHistorianDataSource` (line ~122). In `OtOpcUaServerHostedService.StartAsync` assign it **immediately after the `NodeManager is null` guard (~line 164), before `_deferredSink.SetSink` (~171)**; clear it in `StopAsync` next to the data-source clear (~373). Hosted service is registered before `AddAkka`, so this precedes every deploy apply. (#4, #5)
|
||
> - On success each arm sets `errors[handle.Index] = ServiceResult.Good` (the base pre-seeds Unsupported; every HistoryRead success path here does the same, e.g. OtOpcUaNodeManager.cs ~5133). Per-node `HistoryUpdateResult.StatusCode` stays Good when per-item results are mixed; only a whole-node failure (deny / unsupported / not historized) sets the node code. (#13, #21)
|
||
> - `UpdateDataDetails.UpdateValues[i].SourceTimestamp` is non-nullable: `== DateTime.MinValue` ⇒ `BadInvalidTimestamp` for that item. (#21)
|
||
> - At least one test MUST drive the public `nm.HistoryUpdate(OperationContext, typeof(UpdateDataDetails), …)` entry point and assert the updater is called — it is the test that fails when the AccessLevel bit is missing. Tests that call the protected arm directly do not count toward the ALLOW criterion. (#4)
|
||
> - AclEdit: just add `NodePermissions.HistoryUpdate` to the `PermissionBits` array (~161–166); the page renders enum names, there is no label table. (#23)
|
||
|
||
**Files:**
|
||
- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs`
|
||
- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Security/OpcUaDataPlaneRoles.cs` (`HistoryUpdate = "HistoryUpdate"`)
|
||
- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/ApiKeyInterimRoleMapper.cs`
|
||
- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs` (assign `HistorianDataUpdater` from an optional `HistorianUpdaterHolder` — define `HistorianUpdaterHolder` in `Runtime/Historian/HistorianUpdaterHolder.cs` in THIS task: `public sealed record HistorianUpdaterHolder(IHistorianDataUpdater? Updater) { public static readonly HistorianUpdaterHolder None = new((IHistorianDataUpdater?)null); }`, resolve with `GetService` ⇒ None when absent)
|
||
- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Pages/Clusters/AclEdit.razor` (add `NodePermissions.HistoryUpdate` to the list at ~line 163–166 with label "History update")
|
||
- Modify: `src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Enums/NodePermissions.cs` comment only (remove "until the HistoryUpdate service surface is implemented"; still in no bundle)
|
||
- Test: `tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/History/HistoryUpdateTests.cs`, `tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/OpcUa/ApiKeyInterimRoleMapperTests.cs` (extend)
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] Before writing, read the pinned SDK's `CustomNodeManager2` and confirm the exact protected signatures for `HistoryUpdateData`, `HistoryDeleteRawModified`, `HistoryDeleteAtTime` (and how `HistoryUpdate` dispatches by details type); record them in the report. Use DeepWiki (`OPCFoundation/UA-.NETStandard`) if the source is not on disk.
|
||
- [ ] `public IHistorianDataUpdater? HistorianDataUpdater { get; set; }` (volatile backing field, like `HistorianDataSource`).
|
||
- [ ] `UpdateDataDetails`: `PerformUpdateType.Insert/Replace/Update` ⇒ `HistoryUpdateMode`; `Remove` ⇒ `BadHistoryOperationUnsupported`. Values: `DataValue` ⇒ `HistorizationValue(SourceTimestamp, TryFromObject(Value.Value) or BadTypeMismatch item, StatusCode.Code)`. Per-item results returned in `OperationResults` in order.
|
||
- [ ] `DeleteRawModifiedDetails` with `IsDeleteModified=false` ⇒ `DeleteRawAsync`; `true` ⇒ unsupported. `DeleteAtTimeDetails` ⇒ `DeleteAtTimeAsync`.
|
||
- [ ] Not historized / no updater ⇒ `BadHistoryOperationUnsupported`. Event/structure details ⇒ unsupported.
|
||
- [ ] Authorization helper `AuthorizeHistoryUpdate` mirroring `AuthorizeHistoryRead` + the WriteOperate pattern at ~line 2024–2033: flag on ⇒ `_authorizer.Authorize(identity, OpcUaOperation.HistoryUpdate, nodeId, realm)`; flag off ⇒ `AuthorizeKeyUsable(...)` AND role `HistoryUpdate`. Deny ⇒ `BadUserAccessDenied`.
|
||
- [ ] Batch limiter + deadline shared with HistoryRead (`WithHistoryReadBatchLimiter`, `CreateDeadline`).
|
||
- [ ] `ComposeAccessLevel(writable, historized, historyWritable)` ORs `AccessLevels.HistoryWrite` when `historized && historyWritable`; `historyWritable = HistorianDataUpdater is not null` at materialise time. Raw + UNS mirrors both get it. (Setting the updater after nodes exist does not retro-update AccessLevel — document; the hosted service assigns it before the first deploy applies, verify the ordering and state it.)
|
||
- [ ] `ApiKeyInterimRoleMapper` adds `OpcUaDataPlaneRoles.HistoryUpdate` for a Cluster-scoped grant with `NodePermissions.HistoryUpdate`; a WriteOperate-only grant does NOT confer it.
|
||
- [ ] Tests (fake updater + fake authorizer, following the existing node-manager test harness for HistoryRead): **ALLOW flag-on with no roles + scoped grant ⇒ updater called and `GoodEntryInserted` returned**; flag-off with `HistoryUpdate` role ⇒ allowed; flag-off with only `WriteOperate` ⇒ `BadUserAccessDenied` and updater NOT called; expired key ⇒ denied; no updater ⇒ unsupported; per-item codes passed through in order; UNS NodeId resolves to the same tagname as raw.
|
||
- [ ] Falsification recorded for the ALLOW test (break the flag-on branch ⇒ it fails).
|
||
|
||
**Verify:** `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests --filter "FullyQualifiedName~HistoryUpdate" && dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests --filter "FullyQualifiedName~ApiKeyInterimRoleMapper"` → green.
|
||
|
||
```json:metadata
|
||
{"files":["src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaNodeManager.cs","src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaSdkServer.cs","src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Security/OpcUaDataPlaneRoles.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/ApiKeyInterimRoleMapper.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs","src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/HistorianUpdaterHolder.cs","src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Pages/Clusters/AclEdit.razor","src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Enums/NodePermissions.cs"],"acceptanceCriteria":["SDK signatures verified","ALLOW test","WriteOperate does not confer HistoryUpdate","HistoryWrite access bit"],"modelTier":"frontier","model":"opus","wave":2}
|
||
```
|
||
|
||
Note: T5 references `HistorianUpdaterHolder` — T5 runs in wave 3 after T6 merges, so the type exists.
|
||
|
||
---
|
||
|
||
### Task 7: Client.CLI `historyupdate` command
|
||
|
||
**Goal:** A CLI command to drive HistoryUpdate for manual + live-gate testing.
|
||
|
||
**Files:**
|
||
- Create: `src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI/Commands/HistoryUpdateCommand.cs` (follow the existing `historyread` command's structure, options parsing, session/auth handling exactly)
|
||
- Modify: `docs/Client.CLI.md` (new section)
|
||
- Test: the Client.CLI test project if one exists for commands (`ls tests/Client`); follow its pattern for argument parsing tests.
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] Options per spec §6; `--mode` values `insert|replace|update|delete-raw|delete-at-time`; `--value` parsed against `--type double|int64|bool|string|datetime` (default double); repeated `--time/--value` pairs; validation errors exit non-zero with a message.
|
||
- [ ] Sends `UpdateDataDetails` / `DeleteRawModifiedDetails(IsDeleteModified=false)` / `DeleteAtTimeDetails` via `Session.HistoryUpdate`; prints `index time StatusCodeName` per item plus the overall service result.
|
||
- [ ] Argument-parsing unit tests for each mode (no server needed).
|
||
|
||
**Verify:** `dotnet build src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI && dotnet test tests/Client --filter "FullyQualifiedName~HistoryUpdate"` → green.
|
||
|
||
```json:metadata
|
||
{"files":["src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI/Commands/HistoryUpdateCommand.cs","docs/Client.CLI.md"],"acceptanceCriteria":["all five modes","arg parsing tests"],"modelTier":"standard","model":"sonnet","wave":2}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 8: SQL Server backend project skeleton, schema, bootstrapper, test fixture
|
||
|
||
**Goal:** New projects, embedded DDL, idempotent locked bootstrap, options + validation, shared connection/mapping helpers, and the env-gated integration fixture.
|
||
|
||
> **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:**
|
||
> - **Move `SqlServerTagCatalog.cs` here from T9** (get-or-create + read-only lookup), used by T9/T10/T11/T12. Create path: `MERGE {s}.Tag WITH (HOLDLOCK) AS t …`; on `2627/2601` re-select the TagId. Integration test: two concurrent get-or-create on the same new name ⇒ both return the same id, one row. (#10, #12)
|
||
> - Timestamps: helper `AddDateTime2(SqlCommand, name, DateTime)` (`SqlDbType.DateTime2`, scale 7) for every scalar param; TVPs are `IEnumerable<SqlDataRecord>` with `SqlMetaData(…, SqlDbType.DateTime2, 0, 7)` — **never a `DataTable`** (infers 3.33 ms `datetime`). Mapper tests use non-millisecond-aligned ticks (`…T00:00:00.1234567Z`). (#7)
|
||
> - DDL: `CREATE SCHEMA` must be alone in its batch ⇒ `IF SCHEMA_ID(N'$(Schema)') IS NULL EXEC(N'CREATE SCHEMA [$(Schema)]')`; substitute `$(Schema)` **before** splitting on `GO`. (#17)
|
||
> - `Event`: `EventId nvarchar(64) NOT NULL CONSTRAINT PK_Event PRIMARY KEY NONCLUSTERED`, plus `CREATE CLUSTERED INDEX CX_Event_Time ON {s}.Event (EventTimeUtc, EventId)` (drop the separate `IX_Event_Time`). Keep `IX_Event_Source_Time`. (#19)
|
||
> - Private helper types introduced by wave-3 tasks inside this project must be prefixed with their task (`T9…`, `T10…`, `T11…`) to avoid merge collisions. (#12)
|
||
> - Verified: `StatusCodes.*` names in `SqlHistorianStatusCodes` all exist in the pinned SDK, and `AggregateBits` Calculated=0x1 / Interpolated=0x2 with InfoType 0x400 are correct; keep the SDK-oracle unit test. (#26)
|
||
|
||
**Files:**
|
||
- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.csproj` (refs: Core.Abstractions, Core.AlarmHistorian, Runtime; `<PackageReference Include="Microsoft.Data.SqlClient" />`, `Microsoft.Extensions.Logging.Abstractions`; `<EmbeddedResource Include="Schema\*.sql" />`; `InternalsVisibleTo` both test projects)
|
||
- Create: `.../SqlServerHistorianOptions.cs` (+ `Validate()` returning errors per spec §2), `.../SqlHistorianStatusCodes.cs`, `.../SqlHistorianConnection.cs` (open connection, schema-qualify helper `Q(string obj)` ⇒ `[{schema}].[{obj}]`, transient-error classifier: 1205, -2, 40613, 40197, 40501, 49918, 4060, 233, 10053, 10054, 10060, 11001, SqlException with `IsTransient`), `.../SampleRowMapper.cs` (HistorianValue ⇄ columns per spec §4 + `DataTable` builders for `SampleRow` and `TimeRow` TVPs), `.../SqlServerSchemaBootstrapper.cs`, `.../Schema/001_initial.sql`
|
||
- Create: `docs/historian/sqlserver/001_initial.sql` (copy of the embedded script)
|
||
- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.Tests/` (unit) and `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/` with `SqlHistorianFixture.cs`
|
||
- Modify: `ZB.MOM.WW.OtOpcUa.slnx` (4 new projects), `tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj` (ProjectReference to the new src project so `StatusCodeParityTests` scans `SqlHistorianStatusCodes`)
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] `001_initial.sql` is the spec §4 DDL with `{s}` replaced by a `$(Schema)` token that the bootstrapper substitutes after validating the identifier regex; every statement guarded (`IF SCHEMA_ID(...) IS NULL`, `IF OBJECT_ID(...) IS NULL`, `IF TYPE_ID(...) IS NULL`) so a re-run is a no-op; ends with `INSERT SchemaVersion(1, SYSUTCDATETIME())` guarded by `IF NOT EXISTS`. Split on `^GO$` lines.
|
||
- [ ] `SqlServerSchemaBootstrapper.EnsureAsync(ct)`: lazy, once per instance (a `SemaphoreSlim` + `_ready` flag; failure NOT latched), `sp_getapplock` exclusive session lock with a 30 s timeout around the apply; `AutoCreateSchema=false` ⇒ only checks `MAX(Version) >= 1` and throws `HistorianSchemaMissingException("…missing version 1…")`.
|
||
- [ ] `SqlHistorianStatusCodes` contains every code the spec names (`Good, GoodNoData, GoodEntryInserted, GoodEntryReplaced, BadNoData, BadEntryExists, BadNoEntryExists, BadInvalidTimestamp, BadTypeMismatch, BadAggregateInvalidInputs, BadHistoryOperationUnsupported, UncertainDataSubNormal`) as named `const uint` with values copied from `Opc.Ua.StatusCodes` — the parity test passes and lists this class.
|
||
- [ ] Historian info-bit constants (`InfoTypeDataValue = 0x00000400`, `HistorianCalculated = 0x1`, `HistorianInterpolated = 0x2`) in a separate `HistorianInfoBits` static class, with a unit test asserting `new Opc.Ua.StatusCode(Good).SetAggregateBits(AggregateBits.Calculated).Code == (Good | InfoTypeDataValue | HistorianCalculated)` (adjust to the SDK's actual helper — the test must compare against the SDK, not against this plan).
|
||
- [ ] `SqlHistorianFixture` (xUnit `IAsyncLifetime`, collection fixture): reads `HISTORIAN_SQL_TEST_CONNECTION`; absent ⇒ `SkipReason` and tests `Assert.Skip(SkipReason)` (follow `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.IntegrationTests/SqlPollServerFixture.cs`); present ⇒ `CREATE DATABASE OtOpcUaHistorian_T_<guid:N>`, exposes `ConnectionString` for it, `DROP DATABASE` (after `ALTER … SET SINGLE_USER WITH ROLLBACK IMMEDIATE`) on dispose.
|
||
- [ ] Integration tests: bootstrap twice ⇒ second is a no-op and `SchemaVersion` has one row; two bootstrappers concurrently ⇒ both succeed, one row; `AutoCreateSchema=false` on an empty DB ⇒ `HistorianSchemaMissingException`; invalid schema name rejected by `Validate()` before any SQL runs (unit).
|
||
- [ ] `SampleRowMapper` unit tests: every kind round-trips through the column triple.
|
||
|
||
**Verify:** `dotnet build ZB.MOM.WW.OtOpcUa.slnx && dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.Tests && dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests && dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests --filter "FullyQualifiedName~StatusCodeParity"` → green (integration with the env var set; also run once WITHOUT it and confirm all skip).
|
||
|
||
```json:metadata
|
||
{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerTagCatalog.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/","tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.Tests/","tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/","ZB.MOM.WW.OtOpcUa.slnx","docs/historian/sqlserver/001_initial.sql"],"acceptanceCriteria":["idempotent DDL","applock bootstrap","parity-guarded status consts","env-gated throwaway DB fixture"],"modelTier":"standard","model":"sonnet","wave":2}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 9: SQL Server writes — value writer, provisioning, alarm writer
|
||
|
||
**Goal:** Live values, tag provisioning and alarm events persist idempotently.
|
||
|
||
> **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:**
|
||
> - `SqlServerTagCatalog` now comes from T8 — do NOT create it; use it. (#12)
|
||
> - Insert-if-absent: catch `2627/2601` and treat as success (the row is durably present), so a concurrent writer never books a #538 failure. (#25)
|
||
> - Provisioning `MERGE … WITH (HOLDLOCK)`; integration test with two concurrent provisioners of the same new tags ⇒ both report Ensured, no Failed. (#10)
|
||
> - EventId: SHA-256 over **length-prefixed** UTF-8 fields `EquipmentPath, AlarmId, EventKind, TimestampUtc.Ticks, User, Comment, Message` (null ⇒ length −1). Unit test that `"a|b"+"c"` and `"a"+"b|c"` hash differently. (#20)
|
||
> - All timestamp round-trips use non-ms-aligned ticks and assert tick equality. (#7)
|
||
|
||
**Files (create only):**
|
||
- `.../SqlServerHistorianValueWriter.cs`, `.../SqlServerTagProvisioner.cs`, `.../SqlServerAlarmHistorianWriter.cs`
|
||
- Tests: `tests/.../SqlServer.IntegrationTests/WriteTests.cs`, `ProvisioningTests.cs`, `AlarmWriterTests.cs`; unit `tests/.../SqlServer.Tests/AlarmEventIdTests.cs`
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] Every public entry point calls `bootstrapper.EnsureAsync` first.
|
||
- [ ] Value writer: TVP insert-if-absent (`INSERT INTO Sample(...) SELECT @tagId, r.* , SYSUTCDATETIME() FROM @rows r WHERE NOT EXISTS (SELECT 1 FROM Sample s WITH (UPDLOCK, HOLDLOCK) WHERE s.TagId=@tagId AND s.SourceTimeUtc=r.SourceTimeUtc)`); within-batch duplicate timestamps collapse to the first (de-dup in C# before building the TVP — TVP PK would reject them); null timestamp ⇒ `DateTime.UtcNow`; returns false (never throws) on SQL errors; logs category only.
|
||
- [ ] Provisioning: one `MERGE` per batch upserting Name/DataType/EU/Description; returns `(Requested, Ensured, Skipped:0, Failed)`; never throws.
|
||
- [ ] Alarm writer: EventId = lowercase hex SHA-256 of `AlarmId|EventKind|TimestampUtc.Ticks|User|Comment|Message` (UTF-8, `|` separator, null ⇒ empty); insert-if-absent; per-event outcome per spec (transient classifier from T8).
|
||
- [ ] Integration: each kind written then read back by direct SQL equals input; writing the same batch twice leaves row count unchanged and returns true; a 10 000-value batch completes < 5 s on the rig; provisioning upsert changes DataType on re-run; same alarm event twice ⇒ one row, both `Ack`; an over-long `AlarmTypeName` (> 100) ⇒ `PermanentFail`.
|
||
|
||
**Verify:** `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests --filter "FullyQualifiedName~Write|FullyQualifiedName~Provisioning|FullyQualifiedName~AlarmWriter"` → green.
|
||
|
||
```json:metadata
|
||
{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerHistorianValueWriter.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerTagProvisioner.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerAlarmHistorianWriter.cs"],"acceptanceCriteria":["idempotent redelivery","never throws","deterministic EventId"],"modelTier":"standard","model":"sonnet","wave":3}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 10: SQL Server reads — raw, at-time, events, processed (push-down)
|
||
|
||
**Goal:** `SqlServerHistorianDataSource : IHistorianDataSource` per spec §4 "Reads".
|
||
|
||
> **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:**
|
||
> - `maxValuesPerNode == 0` is the node manager's unbounded sentinel ⇒ `TOP(2147483647)`; test it. (#8)
|
||
> - Events: `TOP(@maxEvents + 1)` (maxEvents 0 ⇒ unbounded); when the extra row is present drop it and return a non-null opaque ContinuationPoint (e.g. UTF-8 of `EventTimeUtc.Ticks|EventId`) — the interface says MUST. Test `maxEvents=2` over 3 rows. (#9)
|
||
> - Processed: cap the bucket count at 100 000; above it throw `ArgumentOutOfRangeException` and document that the node manager surfaces it as a per-node Bad. Check what the node manager's processed path does with a thrown non-aggregate exception and match it. (#24)
|
||
> - Tag lookup uses T8's `SqlServerTagCatalog` read-only lookup. All timestamp tests non-ms-aligned. (#7, #12)
|
||
|
||
**Files (create only):** `.../SqlServerHistorianDataSource.cs`, `.../SqlAggregateQueries.cs` (one method per aggregate family returning the SQL text), tests `tests/.../SqlServer.IntegrationTests/RawReadTests.cs`, `AtTimeReadTests.cs`, `EventReadTests.cs`, `ProcessedReadTests.cs`.
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] Tag lookup is read-only (`SELECT TagId FROM Tag WHERE Name=@n`, cached on hit only) — reads never create tags.
|
||
- [ ] Raw forward/reverse/limit exactly per spec; `DataValueSnapshot` built with `Value.ToObject()`, `StatusCode`, source + server timestamps (inspect `DataValueSnapshot`'s ctor first).
|
||
- [ ] AtTime: single statement with `TimeRow` TVP and two `OUTER APPLY (SELECT TOP 1 …)`; interpolation/stepping/no-prior/bad-neighbour rules per spec; status built from `SqlHistorianStatusCodes` + `HistorianInfoBits`.
|
||
- [ ] Processed: generates every interval in `[start,end)` (empty ⇒ `BadNoData`), the aggregate table in spec §4, Calculated bit on every result, Uncertain when any Bad present, `BadAggregateInvalidInputs` for numeric aggregates on String/DateTime tags, `HistoryAggregateNotSupportedException` for the rest. `interval <= 0` ⇒ single interval `[start,end)`.
|
||
- [ ] Reverse processed (`end < start`) ⇒ swap and return descending (document).
|
||
- [ ] `GetHealthSnapshot()` tracks totals/consecutive failures/last error (the bootstrap error surfaces here).
|
||
- [ ] Integration fixture data: tag `A` with samples at t0+0s=10, +10s=20, +20s=Bad(30), +30s=40, +40s=50 (Double); tag `S` strings; `B` booleans. Hand-computed expectations for each supported aggregate over 20 s intervals from t0 — write them as literal expected values in the test (e.g. interval [t0,t0+20s): Average 15, Min 10, Max 20, Total 30, Count 2, Start 10, End 20, Delta 10, Range 10, PercentGood 100; interval [t0+20s,t0+40s): usable {40}, Average 40 with Uncertain status, PercentBad 50).
|
||
- [ ] Falsification: drop the Bad-sample filter ⇒ the Uncertain interval test fails.
|
||
|
||
**Verify:** `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests --filter "FullyQualifiedName~Read"` → green.
|
||
|
||
```json:metadata
|
||
{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerHistorianDataSource.cs","src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlAggregateQueries.cs"],"acceptanceCriteria":["raw fwd/rev","AtTime interp/step","aggregate table with hand-computed literals","unsupported aggregates throw"],"modelTier":"frontier","model":"opus","wave":3}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 11: SQL Server updater
|
||
|
||
**Goal:** `SqlServerHistorianDataUpdater : IHistorianDataUpdater` per spec §4 "Updates".
|
||
|
||
> **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:**
|
||
> - Use T8's `SqlServerTagCatalog` for create-on-Insert/Update (remove the "private MERGE" fallback). Sample `MERGE … WITH (HOLDLOCK)`. (#10, #12)
|
||
> - "Null timestamp" means `DateTime.MinValue` (the node manager maps it to `BadInvalidTimestamp` before calling, but the updater must still treat `MinValue` the same defensively). (#21)
|
||
> - Non-ms-aligned ticks in every Replace/DeleteAtTime test. (#7)
|
||
|
||
**Files (create only):** `.../SqlServerHistorianDataUpdater.cs`; tests `tests/.../SqlServer.IntegrationTests/UpdateTests.cs`.
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] One transaction per call; `MERGE Sample AS t USING @rows AS r ON t.TagId=@tagId AND t.SourceTimeUtc=r.SourceTimeUtc` with `OUTPUT $action, r.SourceTimeUtc`; Insert uses `WHEN NOT MATCHED THEN INSERT` only (matched ⇒ BadEntryExists), Replace `WHEN MATCHED THEN UPDATE` only (not matched ⇒ BadNoEntryExists), Update both. Results mapped back to the caller's original order (including within-request duplicate timestamps: second occurrence ⇒ `BadEntryExists` for Insert, last-wins for Replace/Update — document).
|
||
- [ ] Null timestamp ⇒ `BadInvalidTimestamp` for that item, others proceed.
|
||
- [ ] Delete raw `[start,end)` ⇒ `Good` with rows deleted, `BadNoData` when 0; DeleteAtTime per time.
|
||
- [ ] Unknown tag: Insert/Update create it via T8's `SqlServerTagCatalog`; Replace/Delete ⇒ BadNoEntryExists/BadNoData.
|
||
- [ ] Integration test per row of the spec's result table + the duplicate/null cases; replaced value is visible via direct SQL with the new status.
|
||
|
||
**Verify:** `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests --filter "FullyQualifiedName~Update"` → green.
|
||
|
||
```json:metadata
|
||
{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerHistorianDataUpdater.cs"],"acceptanceCriteria":["Part 11 result codes","order preserved","single transaction"],"modelTier":"standard","model":"sonnet","wave":3}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 12: SQL Server provider + Host registration
|
||
|
||
**Goal:** `Backend=SqlServer` works end to end in the Host.
|
||
|
||
> **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:**
|
||
> - SqlServer option validation lives in `ServerHistorianOptionsValidator`, consumer-gated (see T5 amendment) — add it there (this task owns that file in wave 4), not in `Resolve`. (#11)
|
||
> - Add `src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/ServerHistorianOptionsValidator.cs` to Files. One shared `SqlServerSchemaBootstrapper` + `SqlServerTagCatalog` per provider. (#11, #12)
|
||
|
||
**Files:**
|
||
- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerHistorianProvider.cs` (one shared bootstrapper + tag catalog instance across the five seams; `SupportedKinds` = all six kinds)
|
||
- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs` (add `"SqlServer"`, binding `ServerHistorian:SqlServer` to `SqlServerHistorianOptions` and failing on its `Validate()` errors), `src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj` (ProjectReference), `Program.cs` only if T5 left a seam that needs it
|
||
- Test: `tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/Configuration/HistorianBackendRegistryTests.cs` (extend), SqlServer.Tests `SqlServerHistorianProviderTests.cs`
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] `Backend=SqlServer` with no ConnectionString ⇒ host-start validation failure naming `ServerHistorian:SqlServer:ConnectionString`.
|
||
- [ ] DI: `IHistorianDataSource` ⇒ `SqlServerHistorianDataSource`, `IHistorianValueWriter` ⇒ SqlServer writer, `HistorianUpdaterHolder.Updater` non-null, recorder `supportedKinds` contains String and Empty.
|
||
- [ ] The Endpoint requirement does not fire for SqlServer.
|
||
|
||
**Verify:** `dotnet build ZB.MOM.WW.OtOpcUa.slnx && dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests --filter "FullyQualifiedName~Historian"` → green.
|
||
|
||
```json:metadata
|
||
{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/SqlServerHistorianProvider.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj"],"acceptanceCriteria":["validation names the key","DI resolves SqlServer types","updater present"],"modelTier":"standard","model":"sonnet","wave":4}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 13: Over-the-wire HistoryUpdate test
|
||
|
||
**Goal:** Prove HistoryUpdate works through a real SDK client session (the encoding/dispatch the node-manager unit tests mock around).
|
||
|
||
> **Review amendments (Fable, 2026-10-09) — binding, override anything below that conflicts:**
|
||
> - Install the fake updater (`Nm.HistorianDataUpdater = fake`, same slot the rig uses for `Nm.Authorizer`, cf. `AclKeyExpiryEnforcementTests.cs:141-215`) **before** materialising the historized node. Add a control test that materialises first, then installs the updater, and proves HistoryUpdate returns `BadHistoryOperationUnsupported` (documents the no-retro-update rule). Assert the per-node `StatusCode` as well as `OperationResults`. (#4, #13, #15)
|
||
|
||
**Files (create only):** `tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/HistoryUpdateOverTheWireTests.cs` (use the existing integration harness that `ConditionMethodEventTests` uses; inject an in-memory fake `IHistorianDataUpdater` + `IHistorianDataSource` sharing one dictionary).
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] Client HistoryUpdate Insert of 2 values on a historized node ⇒ `GoodEntryInserted` ×2; HistoryRead raw returns them; Insert again ⇒ `BadEntryExists`; DeleteAtTime one ⇒ read returns one.
|
||
- [ ] A session without the grant ⇒ `BadUserAccessDenied`, fake not called.
|
||
- [ ] The node's AccessLevel read over the wire includes `HistoryWrite`.
|
||
|
||
**Verify:** `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests --filter "FullyQualifiedName~HistoryUpdateOverTheWire"` → green.
|
||
|
||
```json:metadata
|
||
{"files":["tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/HistoryUpdateOverTheWireTests.cs"],"acceptanceCriteria":["insert/read/duplicate/delete over the wire","deny case","HistoryWrite bit visible"],"modelTier":"standard","model":"sonnet","wave":3}
|
||
```
|
||
|
||
---
|
||
|
||
### Task 14: Documentation
|
||
|
||
**Goal:** Docs match the shipped behaviour.
|
||
|
||
**Files:** `docs/Historian.md`, `docs/Configuration.md`, `docs/security.md`, `CLAUDE.md` (Historian / HistoryRead section: add backend selection, SQL Server, HistoryUpdate, the real-status fix, and the gateway C-002 quality limitation; keep it a short subsection), `docs/drivers/README` or index if historians are listed there.
|
||
|
||
**Acceptance Criteria:**
|
||
- [ ] Every config key in spec §2 documented with default and validation rule.
|
||
- [ ] Aggregate table + "simple per-sample definitions, not full Part 13" caveat.
|
||
- [ ] HistoryUpdate: operations, result codes, authorization (flag on/off), AccessLevel bit, unsupported cases.
|
||
- [ ] Docker-dev recipe for a SQL historian (`OtOpcUaHistorian` on the `sql` container) and the integration-test env var.
|
||
- [ ] No claim the docs make is untested — cross-check each against the merged code.
|
||
|
||
**Verify:** controller reads the diff; `grep -n "Backend" docs/Configuration.md`.
|
||
|
||
```json:metadata
|
||
{"files":["docs/Historian.md","docs/Configuration.md","docs/security.md","CLAUDE.md"],"acceptanceCriteria":["keys documented","aggregate caveat","HistoryUpdate auth documented"],"modelTier":"standard","model":"sonnet","wave":5}
|
||
```
|
||
|
||
---
|
||
|
||
### Final review (fable), then Task 15: Live gate on docker-dev (controller)
|
||
|
||
**Final review:** dispatch a **fable** reviewer over `git diff master...feat/pluggable-historian` with the spec; it must specifically check #491 (`Self` after await), #537/#538 unchanged, every new status literal is a named const, ALLOW tests exist, no `Directory.Packages.props` edit, gateway Good-bytes unchanged. One fix dispatch (opus) + scoped re-review (sonnet), then adjudicate.
|
||
|
||
**Task 15 — Live gate (controller only; not delegated).** **USER-ORDERED GATE — NON-SKIPPABLE.** Close only after each item below is re-validated with captured output.
|
||
|
||
1. `CREATE DATABASE OtOpcUaHistorian` on `otopcua-dev-sql-1`; set on central-1 and central-2 (compose env): `ServerHistorian__Enabled=true`, `ServerHistorian__Backend=SqlServer`, `ServerHistorian__SqlServer__ConnectionString=Server=sql,1433;Database=OtOpcUaHistorian;User Id=sa;Password=OtOpcUa!Dev123;TrustServerCertificate=True`, `ContinuousHistorization__Enabled=true` + `OutboxPath`. Rebuild images (the rig image goes stale), `up -d`.
|
||
2. Mark a rig tag historized (one numeric, one string) and deploy. Evidence: `hist.Sample` row count advances by exactly one per published value (Primary-only), status column real.
|
||
3. `Client.CLI historyread` raw + processed (Average, 10 s) on the raw NodeId AND its UNS NodeId — same rows.
|
||
4. `Client.CLI historyupdate` insert (new timestamp) ⇒ `GoodEntryInserted`; repeat ⇒ `BadEntryExists`; replace ⇒ `GoodEntryReplaced`; delete-at-time ⇒ `Good`; each confirmed by `historyread` and by direct SQL. Use an API key whose principal has a Cluster-scoped `HistoryUpdate` grant; a key with only WriteOperate ⇒ `BadUserAccessDenied`.
|
||
5. Stop the Primary container; the Secondary resumes recording into the same table; no duplicate `(TagId, SourceTimeUtc)` (guaranteed by PK — also check count growth rate stays 1×); restart.
|
||
6. Stop the driver's device (or blackhole) **while the driver is still publishing** (before #527 trips Reconnecting) ⇒ a row with `ValueKind = 0 (Empty)` and the driver's Bad code (Modbus: `BadCommunicationError 0x80050000`). Watchdog `BadNoCommunication` stamps are node-only and are NOT historized — do not expect them.
|
||
7. Record evidence in `docs/plans/2026-10-09-pluggable-historian-live-gate.md`; restore the rig config (Backend off) unless the user wants it kept.
|
||
|
||
```json:metadata
|
||
{"files":["docs/plans/2026-10-09-pluggable-historian-live-gate.md"],"acceptanceCriteria":["Primary-only one row per value","historyread raw+processed via raw and UNS NodeIds","historyupdate insert/duplicate/replace/delete codes","WriteOperate-only key denied","failover continues recording","Bad sample stored Bad"],"userGate":true,"tags":["user-gate"],"verifyCommand":"Client.CLI historyread/historyupdate against opc.tcp://localhost:4840 + sqlcmd on otopcua-dev-sql-1","modelTier":"frontier","model":"controller","wave":5}
|
||
```
|
||
|
||
---
|
||
|
||
## Finish
|
||
|
||
Merge `feat/pluggable-historian` → `master` only with the user's go-ahead; push only with approval. Then update the scadaproj `sister-project-history/otopcua.md` (on scadaproj `main`, pushed) and the index row only if the summary changed (driver/backends list). File Gitea follow-ups: QuestDB backend spec; retention/purge; Part 11 modified values; time-weighted aggregates.
|