From f54ab04e644836c7018c738c339ff0b64b884476 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Sat, 10 Oct 2026 09:09:42 -0400 Subject: [PATCH] docs(historian): fold Fable pre-execution review into QuestDB plan + spec (6 blockers, 15 should-fix, 8 nits) Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe --- .../2026-10-10-questdb-historian-design.md | 11 +- .../2026-10-10-questdb-historian-plan.md | 215 +++++++++++++++++- ...10-10-questdb-historian-plan.md.tasks.json | 20 +- 3 files changed, 228 insertions(+), 18 deletions(-) diff --git a/docs/plans/2026-10-10-questdb-historian-design.md b/docs/plans/2026-10-10-questdb-historian-design.md index c01068be..17574b8c 100644 --- a/docs/plans/2026-10-10-questdb-historian-design.md +++ b/docs/plans/2026-10-10-questdb-historian-design.md @@ -28,7 +28,7 @@ Out of scope: This is extracted from `Driver.Historian.SqlServer` with **no behaviour change**: -- **`StoredSample`.** It holds the time, the value, the stored status and a deleted flag. It classifies a sample as +- **`StoredSample`.** It holds the time, the value, the stored status. It classifies a sample as usable, good, bad or uncertain. It applies the quality-contract clamp: a NULL value carrying a non-Bad status is served as `BadNoData`. It also defines the info-bit masks (`CodeBitsMask`, `SeverityBadBit`, `SeverityMask`). - **`AtTimeEvaluator`.** This is the former `SqlServerHistorianDataSource.AtTimeValue`. It covers interpolation and @@ -83,7 +83,7 @@ three are **WAL tables with `DEDUP UPSERT KEYS`**, so re-sending a row with the ```sql CREATE TABLE IF NOT EXISTS hist_sample ( - ts TIMESTAMP, tag SYMBOL, kind BYTE, num DOUBLE, int LONG, str VARCHAR, + ts TIMESTAMP, tag SYMBOL, kind BYTE, num_value DOUBLE, int_value LONG, str_value VARCHAR, status LONG, deleted BOOLEAN, server_ts TIMESTAMP ) TIMESTAMP(ts) PARTITION BY DAY WAL DEDUP UPSERT KEYS(ts, tag) [TTL n DAYS]; @@ -118,8 +118,8 @@ Column details: 5. Any SQL or transport failure makes the write return `false`, so the recorder retries and the #538 per-tag quarantine still applies. It never throws. -The recorder is Primary-only (#537), so step 2 does not race another live writer. The recorder does not wait for -WAL visibility. +The recorder is Primary-only (#537), so step 2 does not race another live writer. The recorder waits for WAL +visibility, bounded by `VisibilityTimeout` (plan R5). ### 2.3 HistoryUpdate (soft delete) @@ -155,8 +155,7 @@ kept forever. `hist_tag` never expires. Every query filters `NOT deleted`. Every value goes through `StoredSample`, so the quality clamp is the same as SQL Server's. -- **Raw.** A range on `ts`, `ORDER BY ts [DESC]`, `LIMIT n+1` to detect another page. The dedup key forbids - same-instant ties, so the continuation is just the last timestamp. Bounds follow the existing data-source contract. +- **Raw.** A range on `ts`, `ORDER BY ts [DESC]`, `LIMIT @top` (`maxValuesPerNode == 0` ⇒ unbounded), at most `top` samples, `ContinuationPoint: null`; the node manager pages by re-reading from the last timestamp (plan R8). The dedup key forbids same-instant ties. Bounds follow the existing data-source contract. - **AtTime.** Find the previous-or-equal neighbour and the next neighbour of each requested time. The default is `LIMIT 1` seeks on `ts`, batched into chunked statements; T0 picks `ASOF JOIN` for the previous neighbour if it measures better. `AtTimeEvaluator` then decides the result. diff --git a/docs/plans/2026-10-10-questdb-historian-plan.md b/docs/plans/2026-10-10-questdb-historian-plan.md index 7bdf912c..4d4039e4 100644 --- a/docs/plans/2026-10-10-questdb-historian-plan.md +++ b/docs/plans/2026-10-10-questdb-historian-plan.md @@ -91,6 +91,67 @@ These are binding on every task and every reviewer. They are renamed from their `Sql*` names to neutral ones (table in T1). The parity test keys on field names, not class names. +## Review amendments (Fable, 2026-10-10) + +These are binding. They override anything else in this plan that conflicts with them. The full review is +summarised here as B = blocker, S = should-fix, N = nit. + +**Global rulings** +- **R4 — resolution units (B2).** Conformance tests express every sub-second offset in units of `B.ResolutionTicks`: + - `AddTicks(k)` becomes `AddTicks(k * B.ResolutionTicks)`. + - Seeds are at least one resolution unit apart. + - A "near miss" is `+ B.ResolutionTicks`, and the last tick of a bucket is `edge.AddTicks(-B.ResolutionTicks)`. + - The 100k-bucket cap test uses `TimeSpan.FromTicks(B.ResolutionTicks)` intervals over a + `100_000 * B.ResolutionTicks` window. + - Every timestamp assertion, including `.Ticks` comparisons and `ServerTimestampUtc`, goes through `B.Round(..)`. + - Fixture bases (`T0`, `T(n)`) stay deliberately not µs-aligned. + - On SQL Server (`ResolutionTicks = 1`) every rewritten expression evaluates to today's literal. +- **R5 — both write paths wait for WAL visibility (B6).** `QuestDbWalVisibility` is a T3 file. The value writer + (T4) and the updater (T6) each call `WaitAsync` once after writing, bounded by `VisibilityTimeout`. A timeout logs a + Warning and does not change the return value or result codes. Spec §2.2's "The recorder does not wait for WAL + visibility" is replaced by "The recorder waits for WAL visibility, bounded by `VisibilityTimeout`." +- **R6 — microsecond domain at the boundary (B3, B4).** + - `QuestDbConnection.Round(DateTime)` (= `ticks - ticks % 10`, UTC kind) is applied to EVERY inbound timestamp: + - Raw and Events window bounds; + - each AtTime request time; + - DeleteRaw bounds and DeleteAtTime times; + - the Processed window, BEFORE `BucketPlan.Create`. + - Results are stamped with the rounded request time. + - `QuestDbConnection.BindTimestamp(cmd, name, DateTime)` is the ONLY way a DateTime reaches Npgsql. It rounds, and it + never sends a sentinel: `DateTime.MinValue` becomes `MinValue.AddTicks(10)`, and `MaxValue` rounds to `MaxValue − 9` + ticks. Binding a raw DateTime is a review FAIL. + - The Global Constraints "Bucket edges" rule therefore holds exactly. T8 documents that a request start with sub-µs + ticks admits a sample stored up to 9 ticks earlier. +- **R7 — one tag authority (B5).** T3 creates `QuestDbTagCatalog.cs`, mirroring `SqlServerTagCatalog`: + - `Task ExistsAsync(name, ct)`: positive results cached; an invalid name ⇒ false; runs bootstrap `EnsureAsync` + first. + - `Task EnsureAsync(name, byte dataType, ct)`: inserts into `hist_tag` with `ts = 1970-01-01` when absent. + - **The `hist_tag` row is the authority for "known tag".** T4's writer and T6's Insert/Update call `EnsureAsync`. + T5's reads and T6's Replace/DeleteRaw/DeleteAtTime call `ExistsAsync` and never create. + - T4–T6 add no private tag resolver. +- **R8 — Raw paging (S1).** `LIMIT @top`, where `top = maxValuesPerNode == 0 ? int.MaxValue : maxValuesPerNode`. Return + at most `top` samples with `ContinuationPoint: null`, exactly as the SqlServer data source does. The node manager does + the paging (it re-reads from the last timestamp, inclusive). The spec §3 "LIMIT n+1" is wrong and is replaced. +- **R9 — value columns (S3).** In the QuestDB DDL the value columns are `num_value`, `int_value` and `str_value`, to + avoid the reserved word `int`. The `SampleColumns` names in Sql.Common are unchanged. +- **R10 — models (S9, S10).** + - T7 runs on **opus**. It may fix T4/T6 files and the Raw/Events plumbing in `QuestDbHistorianDataSource.cs`. + - A divergence inside `QuestDbAggregateQueries.cs` or the AtTime neighbour logic is reported, not fixed; the + controller dispatches an opus fix. + - T1 runs as **two sequential opus dispatches**: + - **1a:** extract, SqlServer re-point, rename fixes, slnx and parity wiring, moved unit tests. Its gate is the Step 0 + counts, reproduced exactly. + - **1b:** the new direct `Sql.Common.Tests`, each falsified. + - Two commits, one review each. +- **R11 — ownership (B1, N6, S8).** + - `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/**`: T1 in W1, for **mechanical + rename and using fixes only** (no test moved, renamed, re-literalled or deleted; the count stays at baseline), then + T2 in W2. + - `IT/QuestDbConformanceBackend.cs` (seed and read-back only) is owned by **T5**. T7 owns the subclasses + (`QuestDbConformance.cs`). + - `IT/TombstoneTests.Redelivery.cs` is owned by T7 (see the T6 amendment). +- **N4.** `Platforms AnyCPU;x64` goes on both new src csprojs. + --- ## Waves, models and parallel safety @@ -101,7 +162,7 @@ These are binding on every task and every reviewer. | W1 | T1 extract `Sql.Common` | **opus** | | W2 | T2 conformance suite over SqlServer ‖ T3 QuestDB skeleton | sonnet ‖ sonnet | | W3 | T4 writes ‖ T5 reads + aggregates ‖ T6 updater | sonnet ‖ **opus** ‖ sonnet | -| W4 | T7 provider + Host wiring + QuestDB conformance | sonnet | +| W4 | T7 provider + Host wiring + QuestDB conformance | **opus** (R10) | | W5 | T8 docs → **fable** whole-branch review → one fix wave (**opus**) + scoped re-review (sonnet) → T9 live gate | sonnet / fable / opus / controller | Reviewers: @@ -140,6 +201,33 @@ T3 already wired. ### Task 0: QuestDB spike, overlay compose and spec amendments +> **Review amendments (Fable) — binding:** +> - **Q1 is expanded (S2).** Record, with output: +> - the DateTime `Kind`/`NpgsqlDbType` pair that writes `TIMESTAMP` without error (watch for +> `Cannot write DateTime with Kind=UTC to … timestamp without time zone`); +> - whether open → close → reopen on one `NpgsqlDataSource` works without `No Reset On Close=true` (DISCARD ALL); +> - the CLR type returned for `BYTE` (likely `int2`, so use `GetInt16`), `SYMBOL`, `VARCHAR`, `LONG`, `BOOLEAN` and +> `TIMESTAMP`; +> - the parameter typing needed for `$n` in integer arithmetic (`NpgsqlDbType.Bigint`); +> - that a DOUBLE NaN reads as NULL. +> The confirmed keywords go into spec §4 verbatim. +> - **Q3 (S4):** also test `hist_event` with `event_id VARCHAR` as a dedup key. **Fallback:** no dedup on `hist_event`. +> The alarm writer checks `SELECT 1 … WHERE event_id=$1` before inserting, and `ReadEventsAsync` de-duplicates by +> `event_id` client-side. +> - **Q4 fallback (S11):** if `wal_tables()` lacks `writerTxn`/`sequencerTxn`, the visibility wait polls for the written +> key (`WHERE tag=$1 AND ts=$2 AND server_ts=$3`). Record which form is used. +> - **Q5 (S11):** the image must be ≥ 8.2.3, the first release with TTL. Record the floor. +> - **Q7 fallback (S11):** catch a `PostgresException` whose message contains `already exists`, then re-check +> `tables()`. Never swallow blindly. +> - **Q8 fallback (S11):** if neither a window `ORDER BY num` nor the `GROUP BY` expression works, arg-min/arg-max is a +> second statement per node, joined client-side. One-query-per-node is relaxed to two for the ActualTime family only. +> - **Q13 (B4):** bind `DateTime.MinValue`, `DateTime.MaxValue`, `0001-01-01`, `1601-01-01`, +> `9999-12-31T23:59:59.9999999`, and a pre-2000 timestamp with sub-µs ticks, in `WHERE ts >= $1 AND ts < $2`. Record +> what arrives and what reads back. This confirms R6's `BindTimestamp` rule. +> - **Healthcheck (S14):** check whether `curl` exists in the image. If not, use `wget -qO-`, or drop the healthcheck. +> - **Q2 also fixes the page size** that T6 uses for DeleteRaw. + + **Goal:** prove every QuestDB behaviour the design assumes, against a pinned image, before any code is built on it. The findings are written into spec §2–§4. @@ -240,6 +328,24 @@ Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe" ### Task 1: Extract `Driver.Historian.Sql.Common` (no behaviour change) +> **Review amendments (Fable) — binding:** +> - Runs as dispatch **1a** then **1b** (R10). +> - **Files gain** `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/**`, for mechanical +> rename and using fixes only (R11 / B1). This includes replacing `SqlServerSchemaBootstrapper.FailureCacheDuration` +> with the `BootstrapFailureCache` constant. +> - `ProcessedReadTests.Every_enum_member_is_either_served_or_refused` moves to `Sql.Common.Tests` (B2 list). +> - **Step 1 API additions (S6):** +> - `AlarmEventIds.ToNumericSeverity(AlarmSeverity)` (moved from the private SqlServer method); +> - `public sealed class HistorianHealthCounters` (lock-guarded): `RecordSuccess()`, `RecordFailure(Exception)` +> (`OperationCanceledException` is not a failure), and `Snapshot()` returning `HistorianHealthSnapshot` with the +> `ProcessConnectionOpen = successes > 0 && consecutive == 0` rule. `SqlServerHistorianDataSource` uses it too. +> - `public static class EventContinuation { string Encode(DateTime, string eventId) }`, in the existing +> `"{ticks}|{EventId}"` format. +> - `HistoricalEventRows.ClampSeverity(int) → ushort`. +> - `StatusBits.SeverityUncertainBit = 0x40000000u`. +> - Copy `Platforms AnyCPU;x64` into the new csproj (N4). + + **Goal:** move every database-independent piece of `Driver.Historian.SqlServer` into a new shared library. The SQL Server backend then calls it, with byte-identical behaviour. @@ -418,6 +524,40 @@ public sealed class BootstrapFailureCache // the 5 s negative-cache mechanism ### Task 2: Conformance suite (shared source), run against SQL Server +> **Review amendments (Fable) — binding:** +> - **AC bullet 1 becomes:** "Every moved test keeps its method name and its expected VALUES/STATUS literals. Tick +> offsets are rewritten per R4. The report lists each test whose offsets were rewritten." +> - **AC bullet 5 becomes:** "A `virtual` override is allowed only for a difference R4 cannot express. The report names +> it, and the controller rules on it before T7." +> - **Step 2 explicit list (B2). Move:** +> - `RawReadTests` — all except `A_missing_schema_surfaces_in_health_LastError_and_is_not_latched`. The SQL tag-count +> line in `Unknown_tag_is_an_empty_result_and_reads_never_create_tags` becomes `B.TagExistsAsync`. +> - `AtTimeReadTests` — all. +> - `ProcessedReadTests` — all except `Every_enum_member_is_either_served_or_refused`, which went to Sql.Common.Tests +> in T1. +> - `QualityContractTests` — all. Direct status-column reads become `B.ReadBackAsync` status checks. +> - `WriteTests` — `Every_kind_round_trips…`, `Writing_the_same_batch_twice_is_idempotent`, +> `Within_batch_duplicate_timestamps…`, `Non_finite_doubles…`, `Empty_batch_succeeds…` and `Tag_is_created…`. +> - `UpdateTests` — all. +> - `LongNameTests` — all except `An_event_read_filtered_by_a_long_source_matches_nothing`. +> - **Stay in SqlServer:** +> - `A_missing_schema…` +> - `Ten_thousand_values_write_in_under_five_seconds` +> - `Concurrent_overlapping_writers…` +> - `A_SQL_failure_returns_false…` +> - `EventReadTests` +> - `ProvisioningTests` +> - `AlarmWriterTests` +> - `SchemaBootstrapIntegrationTests` +> - **Interface (S13):** add `Task TagExistsAsync(string tag);`. `ReadBackAsync` returns the stored status +> verbatim, with no read-side clamp; say so in its doc comment. +> - **Skip check (S7):** first run with the env var UNSET and confirm the tests are reported Skipped, not Failed. +> Fallback: `ConformanceBase` exposes a lazy, memoised `protected async Task BackendAsync()`, +> disposed in `DisposeAsync`, and each test starts with `var B = await BackendAsync();`. +> - **Verify (N8):** the report lists every moved method name, plus per-file `[Fact]`/`[Theory]` counts before and +> after. + + **Goal:** one backend-neutral suite of hand-computed behaviour tests, compiled into the SqlServer integration project and passing there. T7 later compiles the same suite into the QuestDB project. @@ -492,6 +632,19 @@ public abstract class ConformanceBase : IAsyncLifetime ### Task 3: QuestDB skeleton (options, connection, bootstrapper, fixture) +> **Review amendments (Fable) — binding:** +> - **Files gain** `QuestDbTagCatalog.cs` (R7) and `QuestDbWalVisibility.cs` (R5, moved from T6). +> - `QuestDbConnection` gains `Round(DateTime)` and `BindTimestamp` (R6), plus `BindByte` and `ReadKind`, so the T0 Q1 +> answers live in one file (S2). +> - The DDL uses `num_value`/`int_value`/`str_value` (R9). +> - **New tests:** +> - `BootstrapTests.ExistsAsync_is_false_for_an_unknown_tag_and_true_after_EnsureAsync`; +> - `Wait_returns_once_writerTxn_reaches_the_captured_sequencerTxn` (or the Q4 fallback form); +> - `Timeout_zero_skips_the_wait`; +> - a `QuestDb.Tests` unit test pinning that `BindTimestamp` never emits `DateTime.MinValue`/`MaxValue`. +> - Copy `Platforms AnyCPU;x64` (N4). + + **Goal:** a building `Driver.Historian.QuestDb` project. It has validated options, a pooled Npgsql connection, an idempotent and concurrency-safe schema bootstrapper with TTL, and an env-gated integration fixture. @@ -559,6 +712,16 @@ idempotent and concurrency-safe schema bootstrapper with TTL, and an env-gated i ### Task 4: QuestDB writes (value writer, tag provisioner, alarm writer) +> **Review amendments (Fable) — binding:** +> - The writer calls `QuestDbTagCatalog.EnsureAsync` before inserting (R7). After inserting, it calls +> `QuestDbWalVisibility.WaitAsync`; a timeout does not change the return value (R5). +> - **Provisioner (S5):** read the existing `hist_tag` rows per chunk (`WHERE name IN (…)`), merge +> `COALESCE(request, stored)` for `eng_unit` and `description` client-side, then insert. Pin this with the SqlServer +> `Upsert_creates_then_changes_DataType_on_rerun…` literal. +> - The alarm writer uses `AlarmEventIds.ToNumericSeverity`, plus the Q3 fallback if VARCHAR dedup was refused. +> - Every DateTime is bound via `BindTimestamp` (R6). + + **Goal:** the recorder, provisioning and alarm-history seams write to QuestDB with the same semantics as SQL Server. **Files:** @@ -612,6 +775,22 @@ passes. ### Task 5: QuestDB reads and aggregates +> **Review amendments (Fable) — binding:** +> - **Raw per R8:** `LIMIT @top`, at most `top` samples, `ContinuationPoint: null`. Delete the "LIMIT n+1" text. +> - Every inbound timestamp is rounded via `QuestDbConnection.Round`, and every DateTime is bound via `BindTimestamp` +> (R6). The Processed window is rounded before `BucketPlan.Create`. +> - Tags are resolved through `QuestDbTagCatalog.ExistsAsync` only (R7). +> - **SQL classification (S6):** usable ⇔ `status < {StatusBits.SeverityBadBit}` AND `kind <> 0`; good ⇔ usable AND +> `status < {StatusBits.SeverityUncertainBit}`. The literals are interpolated from the consts, never typed by hand. Note +> that QuestDB's "not Bad" is an unsigned LONG comparison, not T-SQL's `>= 0`. +> - Health uses the shared `HistorianHealthCounters`. Events use `EventContinuation` and +> `HistoricalEventRows.ClampSeverity`. +> - **T5 owns `IT/QuestDbConformanceBackend.cs`** (seed and read-back only; N6). Step 3's throwaway backend IS this file, +> which is kept. Only the throwaway subclasses are deleted. +> - **Extra test (N2):** `ReadTests` includes one seeded Int64-kind row with `num_value = NULL`, proving the kind, not +> the column, decides the value. + + **Goal:** `QuestDbHistorianDataSource` serves Raw, AtTime, Events and Processed. Results are identical to SQL Server's, except for microsecond rounding. @@ -669,6 +848,18 @@ passes. ### Task 6: QuestDB HistoryUpdate (soft delete) with read-your-writes +> **Review amendments (Fable) — binding:** +> - `QuestDbWalVisibility.cs` moves to T3, so T6 consumes it (R5). +> - Insert and Update call `QuestDbTagCatalog.EnsureAsync`; Replace and the deletes call `ExistsAsync` (R7). +> - **Existence read (S15):** one `WHERE tag=$1 AND ts >= $min AND ts <= $max AND NOT deleted` range query, filtered to +> the winners client-side. Never an `IN` list of `n` parameters. +> - **DeleteRaw (S12):** read live rows in `LIMIT` pages of the Q2 batch size, keyed on the last `ts`, and tombstone them +> page by page. `rowsAffected` is the total. +> - Inbound times are rounded and bound via `BindTimestamp` (R6). +> - The TombstoneTests "redelivery after delete" case moves to T7 as +> `IT/TombstoneTests.Redelivery.cs::Tombstone_is_not_resurrected_by_a_writer_redelivery` (S8). T6 keeps the other two. + + **Goal:** `QuestDbHistorianDataUpdater` implements all five operations through insert-only upserts and tombstones. It returns codes identical to SQL Server's, and waits for WAL visibility before returning. @@ -730,6 +921,15 @@ passes. ### Task 7: Provider, Host wiring and the conformance suite on QuestDB +> **Review amendments (Fable) — binding:** +> - The model is **opus** (R10). T7 may fix T4/T6 files and the data-source Raw/Events plumbing. Aggregate and AtTime +> divergences are reported to the controller, not fixed. +> - T5 owns `QuestDbConformanceBackend.cs`; T7 owns the subclasses (N6). +> - T7 adds `IT/TombstoneTests.Redelivery.cs` (S8). +> - **Compose default (S14):** the exact Q1-confirmed connection string, with `Host=questdb;Port=8812` substituted, +> keeping every keyword T0 found necessary. + + **Goal:** `Backend=QuestDb` resolves, validates and builds all five seams. The full conformance suite passes against QuestDB. @@ -782,13 +982,24 @@ QuestDB. on both backends` with the trailer. ```json:metadata -{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianProvider.cs","tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformanceBackend.cs","tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformance.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/ServerHistorianOptionsValidator.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj","docker-dev/docker-compose.yml"],"verifyCommand":"both IT projects green with env set; Host historian filters green","acceptanceCriteria":["provider shape","registry + consumer-gated validator","full conformance suite green on QuestDB","compose env"],"modelTier":"standard","model":"sonnet","wave":4} +{"files":["src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianProvider.cs","tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformanceBackend.cs","tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformance.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/ServerHistorianOptionsValidator.cs","src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj","docker-dev/docker-compose.yml"],"verifyCommand":"both IT projects green with env set; Host historian filters green","acceptanceCriteria":["provider shape","registry + consumer-gated validator","full conformance suite green on QuestDB","compose env"],"modelTier":"frontier","model":"opus","wave":4} ``` --- ### Task 8: Documentation +> **Review amendments (Fable) — binding:** +> - Document these, in addition to the existing list: +> - R6's sub-µs request-start consequence; +> - QuestDB `LONG` NULL = `long.MinValue`, so a stored `FromInt64(long.MinValue)` reads back Empty (N1); +> - TTL is enforced on write commits only, so a quiet table never drops (N3); +> - all write paths wait for WAL visibility (R5). +> - **CLAUDE.md (N5):** check the Historian section's opening paragraph. If it still says +> "HistorianGateway (sole historian backend)", rewrite it to name all three backends. Otherwise add QuestDb to the +> existing "Pluggable backends" subsection. + + **Goal:** the docs describe the QuestDB backend as built, and every claim is checked against the merged code. **Files:** `docs/Historian.md` (a new §"QuestDB backend"), `docs/Configuration.md` (the `ServerHistorian:QuestDb` diff --git a/docs/plans/2026-10-10-questdb-historian-plan.md.tasks.json b/docs/plans/2026-10-10-questdb-historian-plan.md.tasks.json index 5d35edc6..986a4f8e 100644 --- a/docs/plans/2026-10-10-questdb-historian-plan.md.tasks.json +++ b/docs/plans/2026-10-10-questdb-historian-plan.md.tasks.json @@ -5,13 +5,13 @@ "id": 0, "subject": "Task 0: QuestDB spike, overlay compose and spec amendments", "status": "pending", - "description": "**Goal:** prove every QuestDB behaviour the design assumes, against a pinned image, before any code is built on it.\nThe findings are written into spec \u00a72\u2013\u00a74.\n\n**Files:**\n- Create: `docker-dev/docker-compose.questdb.yml`\n- Create: `docs/plans/2026-10-10-questdb-spike.md`\n- Modify: `docs/plans/2026-10-10-questdb-historian-design.md` (\u00a72\u2013\u00a74: replace assumptions with confirmed forms, and mark\n each one \"T0-confirmed\")\n\n**Acceptance Criteria:**\n- [ ] The overlay pins an exact `questdb/questdb:` tag. The tag's release date is at least two weeks before\n 2026-10-10, and is recorded in the spike doc.\n- [ ] The overlay exposes `18812:8812` and `19000:9000`, uses a named volume `otopcua-questdb-data`, and sets the\n container name `otopcua-dev-questdb-1`. It is brought up with\n `docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.questdb.yml up -d questdb`.\n- [ ] The spike doc records a working Npgsql connection string and answers Q1\u2013Q12 below, each with the exact SQL or\n C# that worked and its observed output.\n- [ ] Spec \u00a72.1 has the confirmed DDL, \u00a72.3 the confirmed visibility-wait query, \u00a73 the confirmed bucket-statistics and\n neighbour query forms, and \u00a74 the confirmed connection-string keywords.\n\n**Verify:** `docker exec otopcua-dev-questdb-1 true` exits 0. The spike harness output is pasted into the spike doc.\n\n**Steps:**\n\n- [ ] **Step 1: Create the overlay.**\n\n```yaml\n# docker-dev/docker-compose.questdb.yml \u2014 optional QuestDB fixture for the QuestDB historian backend.\n# Up: docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.questdb.yml up -d questdb\nservices:\n questdb:\n image: questdb/questdb: # T0 pins an exact tag \u2265 2 weeks old\n container_name: otopcua-dev-questdb-1\n ports:\n - \"18812:8812\" # PG wire\n - \"19000:9000\" # web console / REST\n volumes:\n - otopcua-questdb-data:/var/lib/questdb\n healthcheck:\n test: [\"CMD-SHELL\", \"curl -fs http://localhost:9003/status || exit 1\"]\n interval: 10s\n timeout: 5s\n retries: 12\nvolumes:\n otopcua-questdb-data:\n```\n\n- [ ] **Step 2: Write a throwaway C# harness** in the scratchpad, not in the repo. It is a console project that\n references `Npgsql`, restored from the existing central pin via a temporary `Directory.Packages.props` copy, or\n run with `dotnet run` inside a scratch folder under the repo with `ManagePackageVersionsCentrally`. Answer each\n question with real output:\n - **Q1.** The connection-string keywords that work: test `ServerCompatibilityMode=NoTypeLoading` with and without\n it. Also the parameter types needed for `TIMESTAMP` (UTC `DateTime`), `BYTE`, `SYMBOL`, `VARCHAR`, `BOOLEAN` and\n `LONG`.\n - **Q2.** Whether a parameterised multi-row `INSERT INTO t VALUES (\u2026),(\u2026)` works, and the largest row count per\n statement that works reliably. Pick the batch size: \u2264 1000 rows, capped by any parameter limit found.\n - **Q3.** `CREATE TABLE \u2026 WAL DEDUP UPSERT KEYS(ts, tag)`: re-inserting the same key replaces the row. Show\n before and after.\n - **Q4.** The `wal_tables()` columns. After an insert, how to know the transaction we produced and wait until it is\n applied (`writerTxn >= sequencerTxn`). Measure the typical lag over 20 runs.\n - **Q5.** TTL: the syntax at create (`TTL n DAYS`), and `ALTER TABLE \u2026 SET TTL n DAYS`. Where TTL shows up in\n `tables()` metadata.\n - **Q6.** Timestamp precision through Npgsql: a value with 100 ns ticks (`\u20261234567`) comes back as `\u2026123456` (truncated)\n or `\u2026123457` (rounded)? Confirm that our pre-truncation makes the round trip exact.\n - **Q7.** Two connections run `CREATE TABLE IF NOT EXISTS` for the same table at once, 20 times. Record any error\n text, and the catch rule that makes it idempotent.\n - **Q8.** Bucket statistics: one `GROUP BY` over the expression `(cast(ts AS LONG)*10 - $1) / $2`, with\n `sum(CASE WHEN \u2026 THEN 1 ELSE 0 END)`, `avg`, `min`, `max`, `sum`. Plus the first and last usable sample per bucket,\n with its value and time. Try `first()`/`last()` with a filter, or a window `row_number() OVER (PARTITION BY b\n ORDER BY ts)`. Plus arg-min/arg-max time, via a window function or a second statement.\n - **Q9.** AtTime neighbour lookup: chunked `LIMIT 1` seeks (`ts <= t ORDER BY ts DESC LIMIT 1` and\n `ts > t ORDER BY ts LIMIT 1`, combined with `UNION ALL` per time) versus `ASOF JOIN`. Time both for 100 timestamps\n and pick one.\n - **Q10.** VARCHAR and SYMBOL accept a 400-character string, and `SYMBOL` capacity is fine for tens of thousands of\n tags.\n - **Q11.** An existing-timestamp lookup for a batch: `WHERE tag=$1 AND ts >= $2 AND ts <= $3` (a range), or an `IN`\n list. Pick the one that works with parameters.\n - **Q12.** `DROP TABLE IF EXISTS` works for test cleanup, and `tables()` lists the tables with a given prefix.\n\n- [ ] **Step 3: Write the spike doc** with the image tag and date, the connection string, and Q1\u2013Q12 (SQL, output and\n decision for each).\n- [ ] **Step 4: Amend the spec \u00a72\u2013\u00a74** in place with the confirmed forms. If any assumption failed, write the\n replacement design inline and add a ledger ruling. If the failure breaks a user decision, stop and ask.\n- [ ] **Step 5: Commit**\n\n```bash\ngit add docker-dev/docker-compose.questdb.yml docs/plans/2026-10-10-questdb-spike.md docs/plans/2026-10-10-questdb-historian-design.md\ngit commit -m \"spike(historian): QuestDB behaviours confirmed against pinned image; spec \u00a72\u2013\u00a74 amended\n\nClaude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe\"\n```\n\n```json:metadata\n{\"files\":[\"docker-dev/docker-compose.questdb.yml\",\"docs/plans/2026-10-10-questdb-spike.md\",\"docs/plans/2026-10-10-questdb-historian-design.md\"],\"verifyCommand\":\"docker exec otopcua-dev-questdb-1 true\",\"acceptanceCriteria\":[\"pinned image \u22652 weeks old\",\"overlay ports 18812/19000\",\"Q1\u2013Q12 answered with output\",\"spec \u00a72\u2013\u00a74 amended\"],\"modelTier\":\"frontier\",\"model\":\"controller\",\"wave\":0}\n```\n\n---" + "description": "> **Review amendments (Fable) \u2014 binding:**\n> - **Q1 is expanded (S2).** Record, with output:\n> - the DateTime `Kind`/`NpgsqlDbType` pair that writes `TIMESTAMP` without error (watch for\n> `Cannot write DateTime with Kind=UTC to \u2026 timestamp without time zone`);\n> - whether open \u2192 close \u2192 reopen on one `NpgsqlDataSource` works without `No Reset On Close=true` (DISCARD ALL);\n> - the CLR type returned for `BYTE` (likely `int2`, so use `GetInt16`), `SYMBOL`, `VARCHAR`, `LONG`, `BOOLEAN` and\n> `TIMESTAMP`;\n> - the parameter typing needed for `$n` in integer arithmetic (`NpgsqlDbType.Bigint`);\n> - that a DOUBLE NaN reads as NULL.\n> The confirmed keywords go into spec \u00a74 verbatim.\n> - **Q3 (S4):** also test `hist_event` with `event_id VARCHAR` as a dedup key. **Fallback:** no dedup on `hist_event`.\n> The alarm writer checks `SELECT 1 \u2026 WHERE event_id=$1` before inserting, and `ReadEventsAsync` de-duplicates by\n> `event_id` client-side.\n> - **Q4 fallback (S11):** if `wal_tables()` lacks `writerTxn`/`sequencerTxn`, the visibility wait polls for the written\n> key (`WHERE tag=$1 AND ts=$2 AND server_ts=$3`). Record which form is used.\n> - **Q5 (S11):** the image must be \u2265 8.2.3, the first release with TTL. Record the floor.\n> - **Q7 fallback (S11):** catch a `PostgresException` whose message contains `already exists`, then re-check\n> `tables()`. Never swallow blindly.\n> - **Q8 fallback (S11):** if neither a window `ORDER BY num` nor the `GROUP BY` expression works, arg-min/arg-max is a\n> second statement per node, joined client-side. One-query-per-node is relaxed to two for the ActualTime family only.\n> - **Q13 (B4):** bind `DateTime.MinValue`, `DateTime.MaxValue`, `0001-01-01`, `1601-01-01`,\n> `9999-12-31T23:59:59.9999999`, and a pre-2000 timestamp with sub-\u00b5s ticks, in `WHERE ts >= $1 AND ts < $2`. Record\n> what arrives and what reads back. This confirms R6's `BindTimestamp` rule.\n> - **Healthcheck (S14):** check whether `curl` exists in the image. If not, use `wget -qO-`, or drop the healthcheck.\n> - **Q2 also fixes the page size** that T6 uses for DeleteRaw.\n\n\n**Goal:** prove every QuestDB behaviour the design assumes, against a pinned image, before any code is built on it.\nThe findings are written into spec \u00a72\u2013\u00a74.\n\n**Files:**\n- Create: `docker-dev/docker-compose.questdb.yml`\n- Create: `docs/plans/2026-10-10-questdb-spike.md`\n- Modify: `docs/plans/2026-10-10-questdb-historian-design.md` (\u00a72\u2013\u00a74: replace assumptions with confirmed forms, and mark\n each one \"T0-confirmed\")\n\n**Acceptance Criteria:**\n- [ ] The overlay pins an exact `questdb/questdb:` tag. The tag's release date is at least two weeks before\n 2026-10-10, and is recorded in the spike doc.\n- [ ] The overlay exposes `18812:8812` and `19000:9000`, uses a named volume `otopcua-questdb-data`, and sets the\n container name `otopcua-dev-questdb-1`. It is brought up with\n `docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.questdb.yml up -d questdb`.\n- [ ] The spike doc records a working Npgsql connection string and answers Q1\u2013Q12 below, each with the exact SQL or\n C# that worked and its observed output.\n- [ ] Spec \u00a72.1 has the confirmed DDL, \u00a72.3 the confirmed visibility-wait query, \u00a73 the confirmed bucket-statistics and\n neighbour query forms, and \u00a74 the confirmed connection-string keywords.\n\n**Verify:** `docker exec otopcua-dev-questdb-1 true` exits 0. The spike harness output is pasted into the spike doc.\n\n**Steps:**\n\n- [ ] **Step 1: Create the overlay.**\n\n```yaml\n# docker-dev/docker-compose.questdb.yml \u2014 optional QuestDB fixture for the QuestDB historian backend.\n# Up: docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.questdb.yml up -d questdb\nservices:\n questdb:\n image: questdb/questdb: # T0 pins an exact tag \u2265 2 weeks old\n container_name: otopcua-dev-questdb-1\n ports:\n - \"18812:8812\" # PG wire\n - \"19000:9000\" # web console / REST\n volumes:\n - otopcua-questdb-data:/var/lib/questdb\n healthcheck:\n test: [\"CMD-SHELL\", \"curl -fs http://localhost:9003/status || exit 1\"]\n interval: 10s\n timeout: 5s\n retries: 12\nvolumes:\n otopcua-questdb-data:\n```\n\n- [ ] **Step 2: Write a throwaway C# harness** in the scratchpad, not in the repo. It is a console project that\n references `Npgsql`, restored from the existing central pin via a temporary `Directory.Packages.props` copy, or\n run with `dotnet run` inside a scratch folder under the repo with `ManagePackageVersionsCentrally`. Answer each\n question with real output:\n - **Q1.** The connection-string keywords that work: test `ServerCompatibilityMode=NoTypeLoading` with and without\n it. Also the parameter types needed for `TIMESTAMP` (UTC `DateTime`), `BYTE`, `SYMBOL`, `VARCHAR`, `BOOLEAN` and\n `LONG`.\n - **Q2.** Whether a parameterised multi-row `INSERT INTO t VALUES (\u2026),(\u2026)` works, and the largest row count per\n statement that works reliably. Pick the batch size: \u2264 1000 rows, capped by any parameter limit found.\n - **Q3.** `CREATE TABLE \u2026 WAL DEDUP UPSERT KEYS(ts, tag)`: re-inserting the same key replaces the row. Show\n before and after.\n - **Q4.** The `wal_tables()` columns. After an insert, how to know the transaction we produced and wait until it is\n applied (`writerTxn >= sequencerTxn`). Measure the typical lag over 20 runs.\n - **Q5.** TTL: the syntax at create (`TTL n DAYS`), and `ALTER TABLE \u2026 SET TTL n DAYS`. Where TTL shows up in\n `tables()` metadata.\n - **Q6.** Timestamp precision through Npgsql: a value with 100 ns ticks (`\u20261234567`) comes back as `\u2026123456` (truncated)\n or `\u2026123457` (rounded)? Confirm that our pre-truncation makes the round trip exact.\n - **Q7.** Two connections run `CREATE TABLE IF NOT EXISTS` for the same table at once, 20 times. Record any error\n text, and the catch rule that makes it idempotent.\n - **Q8.** Bucket statistics: one `GROUP BY` over the expression `(cast(ts AS LONG)*10 - $1) / $2`, with\n `sum(CASE WHEN \u2026 THEN 1 ELSE 0 END)`, `avg`, `min`, `max`, `sum`. Plus the first and last usable sample per bucket,\n with its value and time. Try `first()`/`last()` with a filter, or a window `row_number() OVER (PARTITION BY b\n ORDER BY ts)`. Plus arg-min/arg-max time, via a window function or a second statement.\n - **Q9.** AtTime neighbour lookup: chunked `LIMIT 1` seeks (`ts <= t ORDER BY ts DESC LIMIT 1` and\n `ts > t ORDER BY ts LIMIT 1`, combined with `UNION ALL` per time) versus `ASOF JOIN`. Time both for 100 timestamps\n and pick one.\n - **Q10.** VARCHAR and SYMBOL accept a 400-character string, and `SYMBOL` capacity is fine for tens of thousands of\n tags.\n - **Q11.** An existing-timestamp lookup for a batch: `WHERE tag=$1 AND ts >= $2 AND ts <= $3` (a range), or an `IN`\n list. Pick the one that works with parameters.\n - **Q12.** `DROP TABLE IF EXISTS` works for test cleanup, and `tables()` lists the tables with a given prefix.\n\n- [ ] **Step 3: Write the spike doc** with the image tag and date, the connection string, and Q1\u2013Q12 (SQL, output and\n decision for each).\n- [ ] **Step 4: Amend the spec \u00a72\u2013\u00a74** in place with the confirmed forms. If any assumption failed, write the\n replacement design inline and add a ledger ruling. If the failure breaks a user decision, stop and ask.\n- [ ] **Step 5: Commit**\n\n```bash\ngit add docker-dev/docker-compose.questdb.yml docs/plans/2026-10-10-questdb-spike.md docs/plans/2026-10-10-questdb-historian-design.md\ngit commit -m \"spike(historian): QuestDB behaviours confirmed against pinned image; spec \u00a72\u2013\u00a74 amended\n\nClaude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe\"\n```\n\n```json:metadata\n{\"files\":[\"docker-dev/docker-compose.questdb.yml\",\"docs/plans/2026-10-10-questdb-spike.md\",\"docs/plans/2026-10-10-questdb-historian-design.md\"],\"verifyCommand\":\"docker exec otopcua-dev-questdb-1 true\",\"acceptanceCriteria\":[\"pinned image \u22652 weeks old\",\"overlay ports 18812/19000\",\"Q1\u2013Q12 answered with output\",\"spec \u00a72\u2013\u00a74 amended\"],\"modelTier\":\"frontier\",\"model\":\"controller\",\"wave\":0}\n```\n\n---" }, { "id": 1, "subject": "Task 1: Extract `Driver.Historian.Sql.Common` (no behaviour cha", "status": "pending", - "description": "**Goal:** move every database-independent piece of `Driver.Historian.SqlServer` into a new shared library. The SQL\nServer backend then calls it, with byte-identical behaviour.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common.csproj`\n- Create in that project: `HistorianStatusCodes.cs`, `StatusBits.cs`, `HistorianInfoBits.cs`, `HistorianDataTypes.cs`,\n `SampleColumns.cs`, `HistorianSampleRow.cs`, `StoredSample.cs`, `BucketRow.cs`, `AggregateFamily.cs`,\n `AggregateEvaluator.cs`, `AtTimeEvaluator.cs`, `BucketPlan.cs`, `LiveWritePreparer.cs`, `HistoryUpdatePlanner.cs`,\n `HistorianTagNames.cs`, `BootstrapFailureCache.cs`, `HistorianSchemaMissingException.cs`, `AlarmEventIds.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common.Tests/` (csproj + tests)\n- Modify: every file in `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/` that contained the moved code\n (`SqlServerHistorianDataSource.cs`, `SqlAggregateQueries.cs`, `SqlHistorianDataTypes.cs` (delete),\n `SqlHistorianStatusCodes.cs` (keep only SQL Server-only codes, or delete if none), `SampleRowMapper.cs`,\n `SqlServerHistorianValueWriter.cs`, `SqlServerHistorianDataUpdater.cs`, `SqlServerTagCatalog.cs`,\n `SqlServerSchemaBootstrapper.cs`, the csproj)\n- Modify: the SqlServer unit tests that tested moved code. Move them into `Sql.Common.Tests` under the same test name.\n- Modify: `ZB.MOM.WW.OtOpcUa.slnx` (add both new projects alphabetically, next to lines 30 and 104)\n- Modify: `tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj` (add a\n ProjectReference to Sql.Common)\n\n**Acceptance Criteria:**\n- [ ] `Sql.Common` has no reference to `Microsoft.Data.SqlClient` or any other database client. Its references are\n Core.Abstractions, Server/Runtime (only if needed for `HistorizationValue`) and\n `Microsoft.Extensions.Logging.Abstractions`.\n- [ ] The public API matches the signatures in Step 1 exactly. T2\u2013T7 depend on them.\n- [ ] The SqlServer integration suite passes with the same test count as before the change, which is recorded in the\n report. The SqlServer unit suite plus the moved tests in `Sql.Common.Tests` has the same total count.\n- [ ] New `Sql.Common.Tests` unit tests cover `AtTimeEvaluator`, `AggregateEvaluator`, `BucketPlan`, `LiveWritePreparer`\n and `HistoryUpdatePlanner` directly, using the literals from the SqlServer integration tests. Each one is\n falsified once.\n- [ ] StatusCodeParity passes, and its constant count does not drop.\n\n**Verify:**\n- `dotnet build ZB.MOM.WW.OtOpcUa.slnx` gives 0 errors.\n- `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common.Tests` passes.\n- `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.Tests` passes.\n- With `HISTORIAN_SQL_TEST_CONNECTION` set, `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests`\n passes with the same count as before.\n- `dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests --filter \"FullyQualifiedName~StatusCodeParity\"`\n passes.\n\n**Steps:**\n\n- [ ] **Step 0: Record the baseline.** Run both SqlServer test projects and the parity filter, and write the exact\n pass counts into the report.\n\n- [ ] **Step 1: Create the shared API.** Namespace: `ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common`. Each moved body is\n transplanted verbatim from the source named in the comment; only names and visibility change.\n\n```csharp\n// HistorianStatusCodes.cs \u2014 from SqlHistorianStatusCodes + SqlHistorianUpdateStatusCodes (all consts, same values)\npublic static class HistorianStatusCodes\n{\n public const uint Good = 0x00000000u, GoodEntryInserted = 0x00A20000u, GoodEntryReplaced = 0x00A30000u;\n public const uint BadNoData = 0x809B0000u, BadEntryExists = 0x809F0000u, BadNoEntryExists = 0x80A00000u;\n public const uint BadInvalidTimestamp = 0x80230000u, BadAggregateInvalidInputs = 0x80D60000u;\n public const uint UncertainDataSubNormal = 0x40A40000u, BadOutOfRange = 0x803C0000u, BadInvalidArgument = 0x80AB0000u;\n}\n// StatusBits.cs / HistorianInfoBits.cs \u2014 moved verbatim from SqlHistorianStatusCodes.cs (made public)\n// HistorianDataTypes.cs \u2014 from SqlHistorianDataTypes: DefaultForEmpty, byte FromKind(HistorianValueKind)\n// SampleColumns.cs \u2014 record struct SampleColumns(byte ValueKind, double? NumValue, long? IntValue, string? StrValue)\n// + static class SampleColumnMapper { ToColumns(HistorianValue); FromColumns(byte,double?,long?,string?);\n// int ToDbStatus(uint); uint FromDbStatus(int); long ToLongStatus(uint) => status; uint FromLongStatus(long) }\n// HistorianSampleRow.cs \u2014 record struct HistorianSampleRow(DateTime TimeUtc, HistorianValue Value, uint StatusCode) (was SqlSampleRow)\npublic readonly record struct StoredSample(DateTime TimeUtc, HistorianValue Value, uint Status)\n{\n public bool IsUsable { get; } // verbatim rule from SqlServerHistorianDataSource\n public bool IsGood { get; }\n /// Applies the quality clamp (NULL value + non-Bad \u21d2 BadNoData) \u2014 from ReadStored line 533.\n public static StoredSample FromStored(DateTime timeUtc, HistorianValue value, uint status);\n}\npublic sealed class BucketRow // fields verbatim from the private BucketRow, as public get/set properties\n{\n public long Bucket { get; set; } public int Total { get; set; } public int Usable { get; set; }\n public int Good { get; set; } public int Bad { get; set; } public int NonNumeric { get; set; }\n public double? Avg { get; set; } public double? Min { get; set; } public double? Max { get; set; } public double? Sum { get; set; }\n public HistorianValue First { get; set; } public HistorianValue Last { get; set; }\n public double? Extreme { get; set; } public DateTime? ExtremeTime { get; set; }\n}\npublic enum AggregateFamily { Simple, StartEnd, ActualTime }\npublic static class AggregateFamilies { public static AggregateFamily? FamilyOf(HistoryAggregateType a); } // from SqlAggregateQueries.FamilyOf\npublic static class AggregateEvaluator\n{\n public static DataValueSnapshot Evaluate(HistoryAggregateType a, BucketRow r, DateTime bucketStart, DateTime now); // verbatim\n public static DataValueSnapshot NoData(DateTime t, DateTime now); // verbatim\n}\npublic static class AtTimeEvaluator\n{\n public static DataValueSnapshot Evaluate(DateTime t, StoredSample? prev, StoredSample? next, DateTime serverTime, DateTime now); // verbatim AtTimeValue\n}\npublic sealed class BucketPlan // from ReadProcessedAsync lines 355-370 and 395-408\n{\n public const int MaxBuckets = 100_000;\n public DateTime StartUtc { get; } public DateTime EndUtc { get; } public bool Reversed { get; }\n public long IntervalTicks { get; } public long BucketCount { get; }\n public long StartUnixTicks => (StartUtc - DateTime.UnixEpoch).Ticks; // for QuestDB's (cast(ts AS LONG)*10 - start)/interval\n /// BucketCount > MaxBuckets (param name \"interval\") \u2014 same as today.\n public static BucketPlan Create(DateTime startUtc, DateTime endUtc, TimeSpan interval);\n public DateTime BucketStart(long index);\n /// One snapshot per bucket in window order; missing buckets get ; reversed if the window was.\n public IReadOnlyList Assemble(IReadOnlyDictionary byBucket, Func empty);\n}\npublic static class LiveWritePreparer // from SqlServerHistorianValueWriter.Prepare, plus a resolution parameter\n{\n /// 1 = keep 100 ns (SQL Server); 10 = truncate to \u00b5s BEFORE de-duplication (QuestDB).\n public static List Prepare(IReadOnlyList values, Func utcNow, long resolutionTicks = 1);\n}\npublic enum UpsertOutcome { Inserted, Replaced }\npublic sealed class UpdatePlan\n{\n public uint?[] Results { get; } // pre-decided codes (invalid ts, NaN, superseded duplicate\u2026); null = decided by the backend\n public IReadOnlyList<(int Index, HistorianSampleRow Row)> Winners { get; } // rows to send, de-duplicated per mode\n}\npublic static class HistoryUpdatePlanner // pre/post passes from SqlServerHistorianDataUpdater.UpdateDataAsync lines 56-108\n{\n public static UpdatePlan Plan(string tag, HistoryUpdateMode mode, IReadOnlyList values, long resolutionTicks = 1);\n /// ticks of each winner that the backend actually wrote \u2192 Inserted/Replaced; absent \u21d2 not written.\n public static IReadOnlyList Complete(UpdatePlan plan, HistoryUpdateMode mode, IReadOnlyDictionary outcomeByTick);\n public static IReadOnlyList DeleteAtTimeResults(IReadOnlyList times, ISet deletedTicks, bool tagKnown);\n public static uint DeleteRawResult(bool tagKnown, long rowsAffected);\n}\npublic static class HistorianTagNames { public const int MaxLength = 400; public static bool IsValid(string? name); } // from SqlServerTagCatalog.IsValidName\npublic sealed class BootstrapFailureCache // the 5 s negative-cache mechanism from SqlServerSchemaBootstrapper\n{\n public BootstrapFailureCache(TimeProvider clock, TimeSpan duration);\n public void ThrowIfCached(); public void Record(Exception ex); public void Clear();\n}\n// HistorianSchemaMissingException.cs \u2014 moved from SqlServerHistorianOptions.cs (public sealed, ctor(string message))\n// AlarmEventIds.cs \u2014 public static string ComputeEventId(...) moved verbatim from SqlServerAlarmHistorianWriter (same signature)\n```\n\n- [ ] **Step 2: Point the SqlServer backend at the shared lib.**\n - Replace each moved body with a call into `Sql.Common`.\n - Delete the now-empty SqlServer classes.\n - Keep `SqlAggregateQueries`' SQL text and `ReadBucket`, now filling the shared `BucketRow`.\n - Keep `SampleRowMapper`'s TVP builders, which take `HistorianSampleRow`.\n- [ ] **Step 3: Move the SqlServer unit tests for the moved code** into `Sql.Common.Tests`, keeping test names and\n literals: `StatusCodeOracleTests`, `ValueWriterPrepareTests`, the column round-trip parts of\n `SampleRowMapperTests`, and the data-type table test. The TVP parts stay in SqlServer.Tests.\n- [ ] **Step 4: Add direct unit tests that did not exist before.** For example:\n\n```csharp\n[Fact] public void AtTime_inside_an_outage_returns_the_bad_neighbours_code_with_no_value()\n{\n var t0 = new DateTime(2026,10,9,8,0,0,DateTimeKind.Utc);\n var prev = new StoredSample(t0, HistorianValue.Empty, 0x80310000u);\n var next = new StoredSample(t0.AddSeconds(10), HistorianValue.FromDouble(5), 0);\n var r = AtTimeEvaluator.Evaluate(t0.AddSeconds(5), prev, next, t0, t0.AddHours(1));\n r.StatusCode.ShouldBe(0x80310000u); r.Value.ShouldBeNull();\n}\n[Fact] public void BucketPlan_rejects_more_than_100k_buckets()\n => Should.Throw(() =>\n BucketPlan.Create(DateTime.UnixEpoch, DateTime.UnixEpoch.AddSeconds(100_001), TimeSpan.FromSeconds(1)));\n[Fact] public void Prepare_truncates_to_microseconds_before_dedup_when_resolution_is_10()\n{\n var t = new DateTime(2026,10,9,8,0,0,DateTimeKind.Utc).AddTicks(1234567);\n var rows = LiveWritePreparer.Prepare([new(t, HistorianValue.FromDouble(1), 0), new(t.AddTicks(3), HistorianValue.FromDouble(2), 0)], () => t, 10);\n rows.Count.ShouldBe(1); rows[0].TimeUtc.Ticks.ShouldBe(t.Ticks - t.Ticks % 10); rows[0].Value.Number.ShouldBe(1);\n}\n```\n\n Add the matching `AggregateEvaluator` cases, using the 8-row ProcessedRead literals from\n `ProcessedReadTests.Numeric_aggregates_over_the_fixture`. Build the `BucketRow`s by hand from tag A's samples.\n- [ ] **Step 5: Wire the solution and parity test.** Add both projects to the slnx, and the ProjectReference to the\n Core.Abstractions.Tests csproj.\n- [ ] **Step 6: Run the Verify commands.** Counts must equal the Step 0 baseline, plus the new direct tests. Falsify\n each new direct test once by reverting the rule it pins.\n- [ ] **Step 7: Commit** with `refactor(historian): extract Driver.Historian.Sql.Common from the SQL Server backend (no\n behaviour change)` and the Claude-Session trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common/\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common.Tests/\",\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.Tests/\",\"ZB.MOM.WW.OtOpcUa.slnx\",\"tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj\"],\"verifyCommand\":\"dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests (env set) \u2014 same count as baseline\",\"acceptanceCriteria\":[\"no DB client ref in Sql.Common\",\"public API exactly as Step 1\",\"SqlServer suites same counts\",\"direct evaluator tests falsified\",\"parity green\"],\"modelTier\":\"frontier\",\"model\":\"opus\",\"wave\":1}\n```\n\n---", + "description": "> **Review amendments (Fable) \u2014 binding:**\n> - Runs as dispatch **1a** then **1b** (R10).\n> - **Files gain** `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/**`, for mechanical\n> rename and using fixes only (R11 / B1). This includes replacing `SqlServerSchemaBootstrapper.FailureCacheDuration`\n> with the `BootstrapFailureCache` constant.\n> - `ProcessedReadTests.Every_enum_member_is_either_served_or_refused` moves to `Sql.Common.Tests` (B2 list).\n> - **Step 1 API additions (S6):**\n> - `AlarmEventIds.ToNumericSeverity(AlarmSeverity)` (moved from the private SqlServer method);\n> - `public sealed class HistorianHealthCounters` (lock-guarded): `RecordSuccess()`, `RecordFailure(Exception)`\n> (`OperationCanceledException` is not a failure), and `Snapshot()` returning `HistorianHealthSnapshot` with the\n> `ProcessConnectionOpen = successes > 0 && consecutive == 0` rule. `SqlServerHistorianDataSource` uses it too.\n> - `public static class EventContinuation { string Encode(DateTime, string eventId) }`, in the existing\n> `\"{ticks}|{EventId}\"` format.\n> - `HistoricalEventRows.ClampSeverity(int) \u2192 ushort`.\n> - `StatusBits.SeverityUncertainBit = 0x40000000u`.\n> - Copy `Platforms AnyCPU;x64` into the new csproj (N4).\n\n\n**Goal:** move every database-independent piece of `Driver.Historian.SqlServer` into a new shared library. The SQL\nServer backend then calls it, with byte-identical behaviour.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common.csproj`\n- Create in that project: `HistorianStatusCodes.cs`, `StatusBits.cs`, `HistorianInfoBits.cs`, `HistorianDataTypes.cs`,\n `SampleColumns.cs`, `HistorianSampleRow.cs`, `StoredSample.cs`, `BucketRow.cs`, `AggregateFamily.cs`,\n `AggregateEvaluator.cs`, `AtTimeEvaluator.cs`, `BucketPlan.cs`, `LiveWritePreparer.cs`, `HistoryUpdatePlanner.cs`,\n `HistorianTagNames.cs`, `BootstrapFailureCache.cs`, `HistorianSchemaMissingException.cs`, `AlarmEventIds.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common.Tests/` (csproj + tests)\n- Modify: every file in `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/` that contained the moved code\n (`SqlServerHistorianDataSource.cs`, `SqlAggregateQueries.cs`, `SqlHistorianDataTypes.cs` (delete),\n `SqlHistorianStatusCodes.cs` (keep only SQL Server-only codes, or delete if none), `SampleRowMapper.cs`,\n `SqlServerHistorianValueWriter.cs`, `SqlServerHistorianDataUpdater.cs`, `SqlServerTagCatalog.cs`,\n `SqlServerSchemaBootstrapper.cs`, the csproj)\n- Modify: the SqlServer unit tests that tested moved code. Move them into `Sql.Common.Tests` under the same test name.\n- Modify: `ZB.MOM.WW.OtOpcUa.slnx` (add both new projects alphabetically, next to lines 30 and 104)\n- Modify: `tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj` (add a\n ProjectReference to Sql.Common)\n\n**Acceptance Criteria:**\n- [ ] `Sql.Common` has no reference to `Microsoft.Data.SqlClient` or any other database client. Its references are\n Core.Abstractions, Server/Runtime (only if needed for `HistorizationValue`) and\n `Microsoft.Extensions.Logging.Abstractions`.\n- [ ] The public API matches the signatures in Step 1 exactly. T2\u2013T7 depend on them.\n- [ ] The SqlServer integration suite passes with the same test count as before the change, which is recorded in the\n report. The SqlServer unit suite plus the moved tests in `Sql.Common.Tests` has the same total count.\n- [ ] New `Sql.Common.Tests` unit tests cover `AtTimeEvaluator`, `AggregateEvaluator`, `BucketPlan`, `LiveWritePreparer`\n and `HistoryUpdatePlanner` directly, using the literals from the SqlServer integration tests. Each one is\n falsified once.\n- [ ] StatusCodeParity passes, and its constant count does not drop.\n\n**Verify:**\n- `dotnet build ZB.MOM.WW.OtOpcUa.slnx` gives 0 errors.\n- `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common.Tests` passes.\n- `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.Tests` passes.\n- With `HISTORIAN_SQL_TEST_CONNECTION` set, `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests`\n passes with the same count as before.\n- `dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests --filter \"FullyQualifiedName~StatusCodeParity\"`\n passes.\n\n**Steps:**\n\n- [ ] **Step 0: Record the baseline.** Run both SqlServer test projects and the parity filter, and write the exact\n pass counts into the report.\n\n- [ ] **Step 1: Create the shared API.** Namespace: `ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common`. Each moved body is\n transplanted verbatim from the source named in the comment; only names and visibility change.\n\n```csharp\n// HistorianStatusCodes.cs \u2014 from SqlHistorianStatusCodes + SqlHistorianUpdateStatusCodes (all consts, same values)\npublic static class HistorianStatusCodes\n{\n public const uint Good = 0x00000000u, GoodEntryInserted = 0x00A20000u, GoodEntryReplaced = 0x00A30000u;\n public const uint BadNoData = 0x809B0000u, BadEntryExists = 0x809F0000u, BadNoEntryExists = 0x80A00000u;\n public const uint BadInvalidTimestamp = 0x80230000u, BadAggregateInvalidInputs = 0x80D60000u;\n public const uint UncertainDataSubNormal = 0x40A40000u, BadOutOfRange = 0x803C0000u, BadInvalidArgument = 0x80AB0000u;\n}\n// StatusBits.cs / HistorianInfoBits.cs \u2014 moved verbatim from SqlHistorianStatusCodes.cs (made public)\n// HistorianDataTypes.cs \u2014 from SqlHistorianDataTypes: DefaultForEmpty, byte FromKind(HistorianValueKind)\n// SampleColumns.cs \u2014 record struct SampleColumns(byte ValueKind, double? NumValue, long? IntValue, string? StrValue)\n// + static class SampleColumnMapper { ToColumns(HistorianValue); FromColumns(byte,double?,long?,string?);\n// int ToDbStatus(uint); uint FromDbStatus(int); long ToLongStatus(uint) => status; uint FromLongStatus(long) }\n// HistorianSampleRow.cs \u2014 record struct HistorianSampleRow(DateTime TimeUtc, HistorianValue Value, uint StatusCode) (was SqlSampleRow)\npublic readonly record struct StoredSample(DateTime TimeUtc, HistorianValue Value, uint Status)\n{\n public bool IsUsable { get; } // verbatim rule from SqlServerHistorianDataSource\n public bool IsGood { get; }\n /// Applies the quality clamp (NULL value + non-Bad \u21d2 BadNoData) \u2014 from ReadStored line 533.\n public static StoredSample FromStored(DateTime timeUtc, HistorianValue value, uint status);\n}\npublic sealed class BucketRow // fields verbatim from the private BucketRow, as public get/set properties\n{\n public long Bucket { get; set; } public int Total { get; set; } public int Usable { get; set; }\n public int Good { get; set; } public int Bad { get; set; } public int NonNumeric { get; set; }\n public double? Avg { get; set; } public double? Min { get; set; } public double? Max { get; set; } public double? Sum { get; set; }\n public HistorianValue First { get; set; } public HistorianValue Last { get; set; }\n public double? Extreme { get; set; } public DateTime? ExtremeTime { get; set; }\n}\npublic enum AggregateFamily { Simple, StartEnd, ActualTime }\npublic static class AggregateFamilies { public static AggregateFamily? FamilyOf(HistoryAggregateType a); } // from SqlAggregateQueries.FamilyOf\npublic static class AggregateEvaluator\n{\n public static DataValueSnapshot Evaluate(HistoryAggregateType a, BucketRow r, DateTime bucketStart, DateTime now); // verbatim\n public static DataValueSnapshot NoData(DateTime t, DateTime now); // verbatim\n}\npublic static class AtTimeEvaluator\n{\n public static DataValueSnapshot Evaluate(DateTime t, StoredSample? prev, StoredSample? next, DateTime serverTime, DateTime now); // verbatim AtTimeValue\n}\npublic sealed class BucketPlan // from ReadProcessedAsync lines 355-370 and 395-408\n{\n public const int MaxBuckets = 100_000;\n public DateTime StartUtc { get; } public DateTime EndUtc { get; } public bool Reversed { get; }\n public long IntervalTicks { get; } public long BucketCount { get; }\n public long StartUnixTicks => (StartUtc - DateTime.UnixEpoch).Ticks; // for QuestDB's (cast(ts AS LONG)*10 - start)/interval\n /// BucketCount > MaxBuckets (param name \"interval\") \u2014 same as today.\n public static BucketPlan Create(DateTime startUtc, DateTime endUtc, TimeSpan interval);\n public DateTime BucketStart(long index);\n /// One snapshot per bucket in window order; missing buckets get ; reversed if the window was.\n public IReadOnlyList Assemble(IReadOnlyDictionary byBucket, Func empty);\n}\npublic static class LiveWritePreparer // from SqlServerHistorianValueWriter.Prepare, plus a resolution parameter\n{\n /// 1 = keep 100 ns (SQL Server); 10 = truncate to \u00b5s BEFORE de-duplication (QuestDB).\n public static List Prepare(IReadOnlyList values, Func utcNow, long resolutionTicks = 1);\n}\npublic enum UpsertOutcome { Inserted, Replaced }\npublic sealed class UpdatePlan\n{\n public uint?[] Results { get; } // pre-decided codes (invalid ts, NaN, superseded duplicate\u2026); null = decided by the backend\n public IReadOnlyList<(int Index, HistorianSampleRow Row)> Winners { get; } // rows to send, de-duplicated per mode\n}\npublic static class HistoryUpdatePlanner // pre/post passes from SqlServerHistorianDataUpdater.UpdateDataAsync lines 56-108\n{\n public static UpdatePlan Plan(string tag, HistoryUpdateMode mode, IReadOnlyList values, long resolutionTicks = 1);\n /// ticks of each winner that the backend actually wrote \u2192 Inserted/Replaced; absent \u21d2 not written.\n public static IReadOnlyList Complete(UpdatePlan plan, HistoryUpdateMode mode, IReadOnlyDictionary outcomeByTick);\n public static IReadOnlyList DeleteAtTimeResults(IReadOnlyList times, ISet deletedTicks, bool tagKnown);\n public static uint DeleteRawResult(bool tagKnown, long rowsAffected);\n}\npublic static class HistorianTagNames { public const int MaxLength = 400; public static bool IsValid(string? name); } // from SqlServerTagCatalog.IsValidName\npublic sealed class BootstrapFailureCache // the 5 s negative-cache mechanism from SqlServerSchemaBootstrapper\n{\n public BootstrapFailureCache(TimeProvider clock, TimeSpan duration);\n public void ThrowIfCached(); public void Record(Exception ex); public void Clear();\n}\n// HistorianSchemaMissingException.cs \u2014 moved from SqlServerHistorianOptions.cs (public sealed, ctor(string message))\n// AlarmEventIds.cs \u2014 public static string ComputeEventId(...) moved verbatim from SqlServerAlarmHistorianWriter (same signature)\n```\n\n- [ ] **Step 2: Point the SqlServer backend at the shared lib.**\n - Replace each moved body with a call into `Sql.Common`.\n - Delete the now-empty SqlServer classes.\n - Keep `SqlAggregateQueries`' SQL text and `ReadBucket`, now filling the shared `BucketRow`.\n - Keep `SampleRowMapper`'s TVP builders, which take `HistorianSampleRow`.\n- [ ] **Step 3: Move the SqlServer unit tests for the moved code** into `Sql.Common.Tests`, keeping test names and\n literals: `StatusCodeOracleTests`, `ValueWriterPrepareTests`, the column round-trip parts of\n `SampleRowMapperTests`, and the data-type table test. The TVP parts stay in SqlServer.Tests.\n- [ ] **Step 4: Add direct unit tests that did not exist before.** For example:\n\n```csharp\n[Fact] public void AtTime_inside_an_outage_returns_the_bad_neighbours_code_with_no_value()\n{\n var t0 = new DateTime(2026,10,9,8,0,0,DateTimeKind.Utc);\n var prev = new StoredSample(t0, HistorianValue.Empty, 0x80310000u);\n var next = new StoredSample(t0.AddSeconds(10), HistorianValue.FromDouble(5), 0);\n var r = AtTimeEvaluator.Evaluate(t0.AddSeconds(5), prev, next, t0, t0.AddHours(1));\n r.StatusCode.ShouldBe(0x80310000u); r.Value.ShouldBeNull();\n}\n[Fact] public void BucketPlan_rejects_more_than_100k_buckets()\n => Should.Throw(() =>\n BucketPlan.Create(DateTime.UnixEpoch, DateTime.UnixEpoch.AddSeconds(100_001), TimeSpan.FromSeconds(1)));\n[Fact] public void Prepare_truncates_to_microseconds_before_dedup_when_resolution_is_10()\n{\n var t = new DateTime(2026,10,9,8,0,0,DateTimeKind.Utc).AddTicks(1234567);\n var rows = LiveWritePreparer.Prepare([new(t, HistorianValue.FromDouble(1), 0), new(t.AddTicks(3), HistorianValue.FromDouble(2), 0)], () => t, 10);\n rows.Count.ShouldBe(1); rows[0].TimeUtc.Ticks.ShouldBe(t.Ticks - t.Ticks % 10); rows[0].Value.Number.ShouldBe(1);\n}\n```\n\n Add the matching `AggregateEvaluator` cases, using the 8-row ProcessedRead literals from\n `ProcessedReadTests.Numeric_aggregates_over_the_fixture`. Build the `BucketRow`s by hand from tag A's samples.\n- [ ] **Step 5: Wire the solution and parity test.** Add both projects to the slnx, and the ProjectReference to the\n Core.Abstractions.Tests csproj.\n- [ ] **Step 6: Run the Verify commands.** Counts must equal the Step 0 baseline, plus the new direct tests. Falsify\n each new direct test once by reverting the rule it pins.\n- [ ] **Step 7: Commit** with `refactor(historian): extract Driver.Historian.Sql.Common from the SQL Server backend (no\n behaviour change)` and the Claude-Session trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common/\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common.Tests/\",\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.Tests/\",\"ZB.MOM.WW.OtOpcUa.slnx\",\"tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj\"],\"verifyCommand\":\"dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests (env set) \u2014 same count as baseline\",\"acceptanceCriteria\":[\"no DB client ref in Sql.Common\",\"public API exactly as Step 1\",\"SqlServer suites same counts\",\"direct evaluator tests falsified\",\"parity green\"],\"modelTier\":\"frontier\",\"model\":\"opus\",\"wave\":1}\n```\n\n---", "blockedBy": [ 0 ] @@ -20,7 +20,7 @@ "id": 2, "subject": "Task 2: Conformance suite (shared source), run against SQL Serv", "status": "pending", - "description": "**Goal:** one backend-neutral suite of hand-computed behaviour tests, compiled into the SqlServer integration project\nand passing there. T7 later compiles the same suite into the QuestDB project.\n\n**Files:**\n- Create: `tests/Drivers/Historian.Conformance/IConformanceBackend.cs`, `ConformanceData.cs`,\n `RawReadConformance.cs`, `AtTimeConformance.cs`, `ProcessedConformance.cs`, `QualityContractConformance.cs`,\n `LiveWriteConformance.cs`, `UpdateConformance.cs`, `TagNameConformance.cs`\n- Modify: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/*.csproj` (add\n `` and a ProjectReference to Sql.Common)\n- Create in the SqlServer IT project: `SqlServerConformanceBackend.cs`, `SqlServerConformance.cs` (one concrete\n subclass per conformance class)\n- Modify in the SqlServer IT project: remove from the original files the generic tests that moved into the suite\n (`RawReadTests`, `AtTimeReadTests`, `ProcessedReadTests`, `QualityContractTests`, and the generic parts of\n `WriteTests`, `UpdateTests` and `LongNameTests`). SQL Server-specific tests stay where they are.\n\n**Acceptance Criteria:**\n- [ ] Every moved test keeps its method name and literal expectations. `git log -p` shows a move, not a rewrite.\n- [ ] The SqlServer IT total count after the change is at least the pre-change count, and the report has the before\n and after count.\n- [ ] Suite code references only `IConformanceBackend` and Core.Abstractions types. There is no `SqlServer*` type and\n no T-SQL in `tests/Drivers/Historian.Conformance`.\n- [ ] Timestamp assertions go through `B.Round(t)`.\n- [ ] Any test that cannot hold for every backend is a `virtual` member with a doc comment naming the difference.\n None should be needed for SQL Server.\n\n**Verify:** with `HISTORIAN_SQL_TEST_CONNECTION` set, `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests`\npasses with a count of at least the baseline. Without the variable, every test skips.\n\n**Steps:**\n\n- [ ] **Step 1: Define the backend contract.**\n\n```csharp\nnamespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Conformance;\npublic interface IConformanceBackend : IAsyncDisposable\n{\n /// 1 for SQL Server (100 ns), 10 for QuestDB (\u00b5s).\n long ResolutionTicks { get; }\n DateTime Round(DateTime t) => new(t.Ticks - t.Ticks % ResolutionTicks, DateTimeKind.Utc);\n IHistorianDataSource CreateDataSource(TimeProvider? clock = null);\n IHistorianValueWriter CreateValueWriter(Func? utcNow = null);\n IHistorianDataUpdater CreateUpdater();\n /// Writes rows directly in the backend's schema (never through the writer), status verbatim.\n Task SeedAsync(string tag, params (DateTime T, HistorianValue V, uint Status)[] samples);\n /// Live (non-deleted) rows for a tag, ascending, read directly from storage.\n Task> ReadBackAsync(string tag);\n}\npublic abstract class ConformanceBase : IAsyncLifetime\n{\n protected IConformanceBackend B { get; private set; } = null!;\n protected abstract Task CreateBackendAsync(); // calls fixture.SkipIfUnavailable()\n public async ValueTask InitializeAsync() => B = await CreateBackendAsync();\n public async ValueTask DisposeAsync() => await B.DisposeAsync();\n}\n```\n\n `ConformanceData` holds `T0`, `At(seconds)`, the status consts and `SeedAAsync(B)`, verbatim from `ReadTestDb`.\n- [ ] **Step 2: Move the generic tests** listed in the T1 inventory (Raw, AtTime, Processed literals, the quality\n contract, live-write redelivery and first-wins, update result codes, tag names over 400 characters) into the\n suite classes. Write each one as an abstract class `RawReadConformance : ConformanceBase` with `[Fact]` methods.\n- [ ] **Step 3: Implement `SqlServerConformanceBackend`.** It reuses `SqlHistorianFixture`, a unique schema and the\n existing seeding SQL from `ReadTestDb`. Add one concrete class per suite:\n `[Collection(SqlHistorianCollection.Name)] public sealed class SqlServerRawReadConformance(SqlHistorianFixture fx) : RawReadConformance`.\n- [ ] **Step 4: Run the tests and compare counts.** Commit with\n `test(historian): backend conformance suite (shared source), run against SQL Server` and the trailer.\n\n```json:metadata\n{\"files\":[\"tests/Drivers/Historian.Conformance/\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/\"],\"verifyCommand\":\"dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests (env set) \u2265 baseline count\",\"acceptanceCriteria\":[\"tests moved by name with same literals\",\"count \u2265 baseline\",\"suite is backend-neutral\",\"Round() for timestamps\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":2}\n```\n\n---", + "description": "> **Review amendments (Fable) \u2014 binding:**\n> - **AC bullet 1 becomes:** \"Every moved test keeps its method name and its expected VALUES/STATUS literals. Tick\n> offsets are rewritten per R4. The report lists each test whose offsets were rewritten.\"\n> - **AC bullet 5 becomes:** \"A `virtual` override is allowed only for a difference R4 cannot express. The report names\n> it, and the controller rules on it before T7.\"\n> - **Step 2 explicit list (B2). Move:**\n> - `RawReadTests` \u2014 all except `A_missing_schema_surfaces_in_health_LastError_and_is_not_latched`. The SQL tag-count\n> line in `Unknown_tag_is_an_empty_result_and_reads_never_create_tags` becomes `B.TagExistsAsync`.\n> - `AtTimeReadTests` \u2014 all.\n> - `ProcessedReadTests` \u2014 all except `Every_enum_member_is_either_served_or_refused`, which went to Sql.Common.Tests\n> in T1.\n> - `QualityContractTests` \u2014 all. Direct status-column reads become `B.ReadBackAsync` status checks.\n> - `WriteTests` \u2014 `Every_kind_round_trips\u2026`, `Writing_the_same_batch_twice_is_idempotent`,\n> `Within_batch_duplicate_timestamps\u2026`, `Non_finite_doubles\u2026`, `Empty_batch_succeeds\u2026` and `Tag_is_created\u2026`.\n> - `UpdateTests` \u2014 all.\n> - `LongNameTests` \u2014 all except `An_event_read_filtered_by_a_long_source_matches_nothing`.\n> - **Stay in SqlServer:**\n> - `A_missing_schema\u2026`\n> - `Ten_thousand_values_write_in_under_five_seconds`\n> - `Concurrent_overlapping_writers\u2026`\n> - `A_SQL_failure_returns_false\u2026`\n> - `EventReadTests`\n> - `ProvisioningTests`\n> - `AlarmWriterTests`\n> - `SchemaBootstrapIntegrationTests`\n> - **Interface (S13):** add `Task TagExistsAsync(string tag);`. `ReadBackAsync` returns the stored status\n> verbatim, with no read-side clamp; say so in its doc comment.\n> - **Skip check (S7):** first run with the env var UNSET and confirm the tests are reported Skipped, not Failed.\n> Fallback: `ConformanceBase` exposes a lazy, memoised `protected async Task BackendAsync()`,\n> disposed in `DisposeAsync`, and each test starts with `var B = await BackendAsync();`.\n> - **Verify (N8):** the report lists every moved method name, plus per-file `[Fact]`/`[Theory]` counts before and\n> after.\n\n\n**Goal:** one backend-neutral suite of hand-computed behaviour tests, compiled into the SqlServer integration project\nand passing there. T7 later compiles the same suite into the QuestDB project.\n\n**Files:**\n- Create: `tests/Drivers/Historian.Conformance/IConformanceBackend.cs`, `ConformanceData.cs`,\n `RawReadConformance.cs`, `AtTimeConformance.cs`, `ProcessedConformance.cs`, `QualityContractConformance.cs`,\n `LiveWriteConformance.cs`, `UpdateConformance.cs`, `TagNameConformance.cs`\n- Modify: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/*.csproj` (add\n `` and a ProjectReference to Sql.Common)\n- Create in the SqlServer IT project: `SqlServerConformanceBackend.cs`, `SqlServerConformance.cs` (one concrete\n subclass per conformance class)\n- Modify in the SqlServer IT project: remove from the original files the generic tests that moved into the suite\n (`RawReadTests`, `AtTimeReadTests`, `ProcessedReadTests`, `QualityContractTests`, and the generic parts of\n `WriteTests`, `UpdateTests` and `LongNameTests`). SQL Server-specific tests stay where they are.\n\n**Acceptance Criteria:**\n- [ ] Every moved test keeps its method name and literal expectations. `git log -p` shows a move, not a rewrite.\n- [ ] The SqlServer IT total count after the change is at least the pre-change count, and the report has the before\n and after count.\n- [ ] Suite code references only `IConformanceBackend` and Core.Abstractions types. There is no `SqlServer*` type and\n no T-SQL in `tests/Drivers/Historian.Conformance`.\n- [ ] Timestamp assertions go through `B.Round(t)`.\n- [ ] Any test that cannot hold for every backend is a `virtual` member with a doc comment naming the difference.\n None should be needed for SQL Server.\n\n**Verify:** with `HISTORIAN_SQL_TEST_CONNECTION` set, `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests`\npasses with a count of at least the baseline. Without the variable, every test skips.\n\n**Steps:**\n\n- [ ] **Step 1: Define the backend contract.**\n\n```csharp\nnamespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Conformance;\npublic interface IConformanceBackend : IAsyncDisposable\n{\n /// 1 for SQL Server (100 ns), 10 for QuestDB (\u00b5s).\n long ResolutionTicks { get; }\n DateTime Round(DateTime t) => new(t.Ticks - t.Ticks % ResolutionTicks, DateTimeKind.Utc);\n IHistorianDataSource CreateDataSource(TimeProvider? clock = null);\n IHistorianValueWriter CreateValueWriter(Func? utcNow = null);\n IHistorianDataUpdater CreateUpdater();\n /// Writes rows directly in the backend's schema (never through the writer), status verbatim.\n Task SeedAsync(string tag, params (DateTime T, HistorianValue V, uint Status)[] samples);\n /// Live (non-deleted) rows for a tag, ascending, read directly from storage.\n Task> ReadBackAsync(string tag);\n}\npublic abstract class ConformanceBase : IAsyncLifetime\n{\n protected IConformanceBackend B { get; private set; } = null!;\n protected abstract Task CreateBackendAsync(); // calls fixture.SkipIfUnavailable()\n public async ValueTask InitializeAsync() => B = await CreateBackendAsync();\n public async ValueTask DisposeAsync() => await B.DisposeAsync();\n}\n```\n\n `ConformanceData` holds `T0`, `At(seconds)`, the status consts and `SeedAAsync(B)`, verbatim from `ReadTestDb`.\n- [ ] **Step 2: Move the generic tests** listed in the T1 inventory (Raw, AtTime, Processed literals, the quality\n contract, live-write redelivery and first-wins, update result codes, tag names over 400 characters) into the\n suite classes. Write each one as an abstract class `RawReadConformance : ConformanceBase` with `[Fact]` methods.\n- [ ] **Step 3: Implement `SqlServerConformanceBackend`.** It reuses `SqlHistorianFixture`, a unique schema and the\n existing seeding SQL from `ReadTestDb`. Add one concrete class per suite:\n `[Collection(SqlHistorianCollection.Name)] public sealed class SqlServerRawReadConformance(SqlHistorianFixture fx) : RawReadConformance`.\n- [ ] **Step 4: Run the tests and compare counts.** Commit with\n `test(historian): backend conformance suite (shared source), run against SQL Server` and the trailer.\n\n```json:metadata\n{\"files\":[\"tests/Drivers/Historian.Conformance/\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/\"],\"verifyCommand\":\"dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests (env set) \u2265 baseline count\",\"acceptanceCriteria\":[\"tests moved by name with same literals\",\"count \u2265 baseline\",\"suite is backend-neutral\",\"Round() for timestamps\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":2}\n```\n\n---", "blockedBy": [ 1 ] @@ -29,7 +29,7 @@ "id": 3, "subject": "Task 3: QuestDB skeleton (options, connection, bootstrapper, fi", "status": "pending", - "description": "**Goal:** a building `Driver.Historian.QuestDb` project. It has validated options, a pooled Npgsql connection, an\nidempotent and concurrency-safe schema bootstrapper with TTL, and an env-gated integration fixture.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.csproj`\n (references Core.Abstractions, Core.AlarmHistorian, Server/Runtime and Sql.Common; packages `Npgsql` and\n `Microsoft.Extensions.Logging.Abstractions`; InternalsVisibleTo the two QuestDb test assemblies)\n- Create: `QuestDbHistorianOptions.cs`, `QuestDbConnection.cs`, `QuestDbTables.cs`, `QuestDbSchemaBootstrapper.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.Tests/` (csproj, `QuestDbHistorianOptionsTests.cs`,\n `QuestDbTablesTests.cs`)\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/` (csproj with the\n conformance-include line and a ProjectReference to Sql.Common, `QuestDbFixture.cs`, `BootstrapTests.cs`)\n- Modify: `ZB.MOM.WW.OtOpcUa.slnx`, and the Core.Abstractions.Tests csproj (ProjectReference to the QuestDb project)\n\n**Acceptance Criteria:**\n- [ ] `QuestDbHistorianOptions` has `ConnectionString` (default `\"\"`), `TablePrefix` (`\"hist_\"`, regex\n `^[A-Za-z0-9_]+$`), `RetentionDays` (`int?`, \u2265 1), `CommandTimeout` (30 s, > 0), `AutoCreateSchema` (`true`) and\n `VisibilityTimeout` (5 s, \u2265 0). `Validate()` returns error strings that name the key\n `ServerHistorian:QuestDb:`.\n- [ ] `QuestDbTables` produces the three `CREATE TABLE IF NOT EXISTS` statements, as confirmed by T0, for a given prefix\n and retention. Unit tests pin the exact text.\n- [ ] The bootstrapper is idempotent and survives two concurrent instances, using the catch rule from T0 Q7. It\n re-applies a changed TTL. It uses `BootstrapFailureCache` for the 5 s negative cache.\n- [ ] `QuestDbFixture` reads `HISTORIAN_QUESTDB_TEST_CONNECTION` and skips when it is absent. `UniquePrefix()` returns\n `\"t\" + 12 hex + \"_\"`. `DropPrefixAsync(prefix)` drops every table with that prefix, found through `tables()`.\n- [ ] `BootstrapTests` cover:\n - tables created;\n - a second run is a no-op;\n - two concurrent bootstrappers both succeed;\n - TTL is visible in metadata when `RetentionDays=7` and absent when unset;\n - `AutoCreateSchema=false` against a missing table throws `HistorianSchemaMissingException` from Sql.Common.\n\n**Verify:**\n- `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.Tests` passes.\n- With `HISTORIAN_QUESTDB_TEST_CONNECTION` set, `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests --filter \"FullyQualifiedName~Bootstrap\"`\n passes.\n- Without the variable, every test skips.\n\n**Steps:**\n- [ ] Step 1: write the options with `Validate()` and its unit tests (red, then green).\n- [ ] Step 2: write `QuestDbTables` DDL builders (use the spec \u00a72.1 *as amended by T0*), with unit tests pinning the\n text for prefix `hist_` with and without TTL.\n- [ ] Step 3: write `QuestDbConnection`.\n - Members:\n - `NpgsqlDataSource` built once;\n - `OpenAsync(ct)`;\n - `CreateCommand(conn, sql)` with `CommandTimeout`;\n - `static bool IsTransient(Exception)`, true for `NpgsqlException` with `IsTransient`, socket and IO errors, and\n timeouts;\n - `static DateTime ReadUtc(NpgsqlDataReader r, int i)`;\n - `string T(string table)`, which returns prefix + table.\n - Use the parameter typing from T0 Q1.\n- [ ] Step 4: write the bootstrapper and fixture. Bring up the overlay\n (`docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.questdb.yml up -d questdb`), then\n write and run `BootstrapTests`.\n- [ ] Step 5: wire the slnx and the parity-test reference, then build the solution.\n- [ ] Step 6: commit `feat(historian): QuestDB backend skeleton \u2014 options, connection, schema bootstrap with TTL` with\n the trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.Tests/\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/\",\"ZB.MOM.WW.OtOpcUa.slnx\"],\"verifyCommand\":\"dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests --filter FullyQualifiedName~Bootstrap (env set)\",\"acceptanceCriteria\":[\"options validate naming keys\",\"DDL pinned by unit tests\",\"bootstrap idempotent + concurrent + TTL\",\"fixture env-gated with unique prefix\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":2}\n```\n\n---", + "description": "> **Review amendments (Fable) \u2014 binding:**\n> - **Files gain** `QuestDbTagCatalog.cs` (R7) and `QuestDbWalVisibility.cs` (R5, moved from T6).\n> - `QuestDbConnection` gains `Round(DateTime)` and `BindTimestamp` (R6), plus `BindByte` and `ReadKind`, so the T0 Q1\n> answers live in one file (S2).\n> - The DDL uses `num_value`/`int_value`/`str_value` (R9).\n> - **New tests:**\n> - `BootstrapTests.ExistsAsync_is_false_for_an_unknown_tag_and_true_after_EnsureAsync`;\n> - `Wait_returns_once_writerTxn_reaches_the_captured_sequencerTxn` (or the Q4 fallback form);\n> - `Timeout_zero_skips_the_wait`;\n> - a `QuestDb.Tests` unit test pinning that `BindTimestamp` never emits `DateTime.MinValue`/`MaxValue`.\n> - Copy `Platforms AnyCPU;x64` (N4).\n\n\n**Goal:** a building `Driver.Historian.QuestDb` project. It has validated options, a pooled Npgsql connection, an\nidempotent and concurrency-safe schema bootstrapper with TTL, and an env-gated integration fixture.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.csproj`\n (references Core.Abstractions, Core.AlarmHistorian, Server/Runtime and Sql.Common; packages `Npgsql` and\n `Microsoft.Extensions.Logging.Abstractions`; InternalsVisibleTo the two QuestDb test assemblies)\n- Create: `QuestDbHistorianOptions.cs`, `QuestDbConnection.cs`, `QuestDbTables.cs`, `QuestDbSchemaBootstrapper.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.Tests/` (csproj, `QuestDbHistorianOptionsTests.cs`,\n `QuestDbTablesTests.cs`)\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/` (csproj with the\n conformance-include line and a ProjectReference to Sql.Common, `QuestDbFixture.cs`, `BootstrapTests.cs`)\n- Modify: `ZB.MOM.WW.OtOpcUa.slnx`, and the Core.Abstractions.Tests csproj (ProjectReference to the QuestDb project)\n\n**Acceptance Criteria:**\n- [ ] `QuestDbHistorianOptions` has `ConnectionString` (default `\"\"`), `TablePrefix` (`\"hist_\"`, regex\n `^[A-Za-z0-9_]+$`), `RetentionDays` (`int?`, \u2265 1), `CommandTimeout` (30 s, > 0), `AutoCreateSchema` (`true`) and\n `VisibilityTimeout` (5 s, \u2265 0). `Validate()` returns error strings that name the key\n `ServerHistorian:QuestDb:`.\n- [ ] `QuestDbTables` produces the three `CREATE TABLE IF NOT EXISTS` statements, as confirmed by T0, for a given prefix\n and retention. Unit tests pin the exact text.\n- [ ] The bootstrapper is idempotent and survives two concurrent instances, using the catch rule from T0 Q7. It\n re-applies a changed TTL. It uses `BootstrapFailureCache` for the 5 s negative cache.\n- [ ] `QuestDbFixture` reads `HISTORIAN_QUESTDB_TEST_CONNECTION` and skips when it is absent. `UniquePrefix()` returns\n `\"t\" + 12 hex + \"_\"`. `DropPrefixAsync(prefix)` drops every table with that prefix, found through `tables()`.\n- [ ] `BootstrapTests` cover:\n - tables created;\n - a second run is a no-op;\n - two concurrent bootstrappers both succeed;\n - TTL is visible in metadata when `RetentionDays=7` and absent when unset;\n - `AutoCreateSchema=false` against a missing table throws `HistorianSchemaMissingException` from Sql.Common.\n\n**Verify:**\n- `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.Tests` passes.\n- With `HISTORIAN_QUESTDB_TEST_CONNECTION` set, `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests --filter \"FullyQualifiedName~Bootstrap\"`\n passes.\n- Without the variable, every test skips.\n\n**Steps:**\n- [ ] Step 1: write the options with `Validate()` and its unit tests (red, then green).\n- [ ] Step 2: write `QuestDbTables` DDL builders (use the spec \u00a72.1 *as amended by T0*), with unit tests pinning the\n text for prefix `hist_` with and without TTL.\n- [ ] Step 3: write `QuestDbConnection`.\n - Members:\n - `NpgsqlDataSource` built once;\n - `OpenAsync(ct)`;\n - `CreateCommand(conn, sql)` with `CommandTimeout`;\n - `static bool IsTransient(Exception)`, true for `NpgsqlException` with `IsTransient`, socket and IO errors, and\n timeouts;\n - `static DateTime ReadUtc(NpgsqlDataReader r, int i)`;\n - `string T(string table)`, which returns prefix + table.\n - Use the parameter typing from T0 Q1.\n- [ ] Step 4: write the bootstrapper and fixture. Bring up the overlay\n (`docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.questdb.yml up -d questdb`), then\n write and run `BootstrapTests`.\n- [ ] Step 5: wire the slnx and the parity-test reference, then build the solution.\n- [ ] Step 6: commit `feat(historian): QuestDB backend skeleton \u2014 options, connection, schema bootstrap with TTL` with\n the trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.Tests/\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/\",\"ZB.MOM.WW.OtOpcUa.slnx\"],\"verifyCommand\":\"dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests --filter FullyQualifiedName~Bootstrap (env set)\",\"acceptanceCriteria\":[\"options validate naming keys\",\"DDL pinned by unit tests\",\"bootstrap idempotent + concurrent + TTL\",\"fixture env-gated with unique prefix\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":2}\n```\n\n---", "blockedBy": [ 1 ] @@ -38,7 +38,7 @@ "id": 4, "subject": "Task 4: QuestDB writes (value writer, tag provisioner, alarm wr", "status": "pending", - "description": "**Goal:** the recorder, provisioning and alarm-history seams write to QuestDB with the same semantics as SQL Server.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianValueWriter.cs`,\n `QuestDbTagProvisioner.cs`, `QuestDbAlarmHistorianWriter.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/WriteTests.cs`,\n `ProvisioningTests.cs`, `AlarmWriterTests.cs`\n\n**Acceptance Criteria:**\n- [ ] The value writer works as follows:\n - It calls `LiveWritePreparer.Prepare(values, utcNow, resolutionTicks: 10)`.\n - It rejects an invalid tag name (`HistorianTagNames.IsValid`) and returns `false`.\n - It reads the existing timestamps for the tag in the batch range, including tombstones, using the T0 Q11 form.\n - It inserts only the rows that are not yet present, in multi-row parameterised `INSERT`s of at most the T0 Q2\n batch size, with `deleted=false` and `server_ts=utcNow()`.\n - It returns `true` with no SQL when every row already existed.\n - Any exception returns `false`. It never throws, and cancellation also returns `false`.\n- [ ] The provisioner upserts `hist_tag` (`ts = 1970-01-01`, data type from `HistorianDataTypes.FromKind`, eng unit,\n description). Both the request field types and the result tally match the SqlServer provisioner's.\n- [ ] The alarm writer works as follows:\n - It uses the same deterministic `event_id` as the SQL Server writer, through `AlarmEventIds.ComputeEventId` from\n Sql.Common.\n - It inserts events through the dedup key `(ts, event_id)`, so a redelivery is idempotent.\n - `Classify` maps an unknown or non-database exception to RetryPlease, and maps permanent errors as the SqlServer\n writer does.\n- [ ] `WriteTests` cover:\n - typed round trip for all six kinds with a non-Good status;\n - a redelivered batch leaves one row and the first value;\n - a tombstoned key is not resurrected by redelivery (seed a tombstone directly);\n - NaN \u21d2 `BadOutOfRange` Empty;\n - a microsecond collision within a batch keeps the first;\n - a stopped port (`Port=1`) returns `false` without throwing.\n- [ ] Each regression test is falsified once.\n\n**Verify:** with `HISTORIAN_QUESTDB_TEST_CONNECTION` set,\n`dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests --filter \"FullyQualifiedName~Write|FullyQualifiedName~Provisioning|FullyQualifiedName~AlarmWriter\"`\npasses.\n\n**Steps:**\n- [ ] Step 1: write `WriteTests` (red).\n- [ ] Step 2: implement the value writer.\n- [ ] Step 3: make `WriteTests` green, then falsify the first-wins, tombstone and NaN tests.\n- [ ] Step 4: do the provisioner, then the alarm writer, each tests-first.\n- [ ] Step 5: commit `feat(historian): QuestDB live writes, provisioning and alarm events` with the trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianValueWriter.cs\",\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbTagProvisioner.cs\",\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbAlarmHistorianWriter.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/WriteTests.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/ProvisioningTests.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/AlarmWriterTests.cs\"],\"verifyCommand\":\"dotnet test \u2026QuestDb.IntegrationTests --filter Write|Provisioning|AlarmWriter (env set)\",\"acceptanceCriteria\":[\"insert-if-absent incl tombstones\",\"never throws\",\"event id deterministic + idempotent\",\"falsified\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":3}\n```\n\n---", + "description": "> **Review amendments (Fable) \u2014 binding:**\n> - The writer calls `QuestDbTagCatalog.EnsureAsync` before inserting (R7). After inserting, it calls\n> `QuestDbWalVisibility.WaitAsync`; a timeout does not change the return value (R5).\n> - **Provisioner (S5):** read the existing `hist_tag` rows per chunk (`WHERE name IN (\u2026)`), merge\n> `COALESCE(request, stored)` for `eng_unit` and `description` client-side, then insert. Pin this with the SqlServer\n> `Upsert_creates_then_changes_DataType_on_rerun\u2026` literal.\n> - The alarm writer uses `AlarmEventIds.ToNumericSeverity`, plus the Q3 fallback if VARCHAR dedup was refused.\n> - Every DateTime is bound via `BindTimestamp` (R6).\n\n\n**Goal:** the recorder, provisioning and alarm-history seams write to QuestDB with the same semantics as SQL Server.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianValueWriter.cs`,\n `QuestDbTagProvisioner.cs`, `QuestDbAlarmHistorianWriter.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/WriteTests.cs`,\n `ProvisioningTests.cs`, `AlarmWriterTests.cs`\n\n**Acceptance Criteria:**\n- [ ] The value writer works as follows:\n - It calls `LiveWritePreparer.Prepare(values, utcNow, resolutionTicks: 10)`.\n - It rejects an invalid tag name (`HistorianTagNames.IsValid`) and returns `false`.\n - It reads the existing timestamps for the tag in the batch range, including tombstones, using the T0 Q11 form.\n - It inserts only the rows that are not yet present, in multi-row parameterised `INSERT`s of at most the T0 Q2\n batch size, with `deleted=false` and `server_ts=utcNow()`.\n - It returns `true` with no SQL when every row already existed.\n - Any exception returns `false`. It never throws, and cancellation also returns `false`.\n- [ ] The provisioner upserts `hist_tag` (`ts = 1970-01-01`, data type from `HistorianDataTypes.FromKind`, eng unit,\n description). Both the request field types and the result tally match the SqlServer provisioner's.\n- [ ] The alarm writer works as follows:\n - It uses the same deterministic `event_id` as the SQL Server writer, through `AlarmEventIds.ComputeEventId` from\n Sql.Common.\n - It inserts events through the dedup key `(ts, event_id)`, so a redelivery is idempotent.\n - `Classify` maps an unknown or non-database exception to RetryPlease, and maps permanent errors as the SqlServer\n writer does.\n- [ ] `WriteTests` cover:\n - typed round trip for all six kinds with a non-Good status;\n - a redelivered batch leaves one row and the first value;\n - a tombstoned key is not resurrected by redelivery (seed a tombstone directly);\n - NaN \u21d2 `BadOutOfRange` Empty;\n - a microsecond collision within a batch keeps the first;\n - a stopped port (`Port=1`) returns `false` without throwing.\n- [ ] Each regression test is falsified once.\n\n**Verify:** with `HISTORIAN_QUESTDB_TEST_CONNECTION` set,\n`dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests --filter \"FullyQualifiedName~Write|FullyQualifiedName~Provisioning|FullyQualifiedName~AlarmWriter\"`\npasses.\n\n**Steps:**\n- [ ] Step 1: write `WriteTests` (red).\n- [ ] Step 2: implement the value writer.\n- [ ] Step 3: make `WriteTests` green, then falsify the first-wins, tombstone and NaN tests.\n- [ ] Step 4: do the provisioner, then the alarm writer, each tests-first.\n- [ ] Step 5: commit `feat(historian): QuestDB live writes, provisioning and alarm events` with the trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianValueWriter.cs\",\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbTagProvisioner.cs\",\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbAlarmHistorianWriter.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/WriteTests.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/ProvisioningTests.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/AlarmWriterTests.cs\"],\"verifyCommand\":\"dotnet test \u2026QuestDb.IntegrationTests --filter Write|Provisioning|AlarmWriter (env set)\",\"acceptanceCriteria\":[\"insert-if-absent incl tombstones\",\"never throws\",\"event id deterministic + idempotent\",\"falsified\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":3}\n```\n\n---", "blockedBy": [ 3 ] @@ -47,7 +47,7 @@ "id": 5, "subject": "Task 5: QuestDB reads and aggregates", "status": "pending", - "description": "**Goal:** `QuestDbHistorianDataSource` serves Raw, AtTime, Events and Processed. Results are identical to SQL Server's,\nexcept for microsecond rounding.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianDataSource.cs`,\n `QuestDbAggregateQueries.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/ReadTests.cs`\n\n**Acceptance Criteria:**\n- [ ] Every query filters `NOT deleted`. Every row goes through `StoredSample.FromStored`.\n- [ ] **Raw:** forward and reverse, `LIMIT n+1`, and the same paging contract and continuation semantics as the\n SqlServer data source (re-read from the last timestamp, inclusive). Unbounded when `maxValuesPerNode == 0`.\n Unknown tag \u21d2 an empty result. The tag is resolved through the `hist_tag` existence or the sample table, so the\n read never creates a tag.\n- [ ] **AtTime:** the result list matches the input in length and order. Neighbours come from the T0 Q9 form, and\n `AtTimeEvaluator.Evaluate` decides each value.\n- [ ] **Processed:**\n - It uses `BucketPlan.Create` and `AggregateFamilies.FamilyOf`.\n - It runs one statement per node, grouped by `(cast(ts AS LONG)*10 - @startUnixTicks) / @intervalTicks`.\n - It fills the shared `BucketRow` with the same usable/good/bad/non-numeric definitions as SQL Server: usable = status\n severity not Bad and kind \u2260 Empty; good = usable and status < 0x40000000; non-numeric = usable and kind in (4, 5).\n - `AggregateEvaluator.Evaluate` produces the values, and `BucketPlan.Assemble` fills empty buckets and handles\n reversal.\n - An unsupported aggregate throws `HistoryAggregateNotSupportedException`.\n- [ ] **Events:** filtered by source and time range, ascending, limited to `maxEvents`. The `ContinuationPoint` is\n non-null when truncated.\n- [ ] Health counters (`GetHealthSnapshot`) are implemented as in the SqlServer data source (lock-guarded success/failure counters plus the last error). A failure fails only that call and is recorded in\n `GetHealthSnapshot`.\n- [ ] `ReadTests` cover only QuestDB-specific behaviour:\n - tombstoned rows are invisible to Raw, AtTime and Processed (seed a tombstone directly);\n - a whole-range Processed read from 1601 to 9999 with interval 0 does not overflow;\n - a reverse paging walk with `maxValuesPerNode=1` returns every row.\n The shared literals run through the conformance suite in T7.\n\n**Verify:** with `HISTORIAN_QUESTDB_TEST_CONNECTION` set,\n`dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests --filter \"FullyQualifiedName~ReadTests\"`\npasses.\n\n**Steps:**\n- [ ] Step 1: write `ReadTests` (red).\n- [ ] Step 2: implement Raw, then AtTime, Events and Processed.\n- [ ] Step 3: temporarily compile the conformance read classes into this task's worktree. Add a throwaway concrete\n subclass and a throwaway backend that seeds through direct INSERTs. Run the Raw, AtTime and Processed\n conformance classes locally and get them green.\n- [ ] Step 4: delete the throwaway files before committing. T7 owns the real ones. Paste the conformance run output\n into the report.\n- [ ] Step 5: falsify the tombstone filter and the bucket expression (an off-by-one start).\n- [ ] Step 6: commit `feat(historian): QuestDB reads and aggregates (pushed down, shared evaluation)` with the trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianDataSource.cs\",\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbAggregateQueries.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/ReadTests.cs\"],\"verifyCommand\":\"dotnet test \u2026QuestDb.IntegrationTests --filter FullyQualifiedName~ReadTests (env set) + throwaway conformance run in report\",\"acceptanceCriteria\":[\"NOT deleted everywhere\",\"bucket expr exact vs SqlServer\",\"shared evaluators used\",\"conformance read classes green locally\"],\"modelTier\":\"frontier\",\"model\":\"opus\",\"wave\":3}\n```\n\n---", + "description": "> **Review amendments (Fable) \u2014 binding:**\n> - **Raw per R8:** `LIMIT @top`, at most `top` samples, `ContinuationPoint: null`. Delete the \"LIMIT n+1\" text.\n> - Every inbound timestamp is rounded via `QuestDbConnection.Round`, and every DateTime is bound via `BindTimestamp`\n> (R6). The Processed window is rounded before `BucketPlan.Create`.\n> - Tags are resolved through `QuestDbTagCatalog.ExistsAsync` only (R7).\n> - **SQL classification (S6):** usable \u21d4 `status < {StatusBits.SeverityBadBit}` AND `kind <> 0`; good \u21d4 usable AND\n> `status < {StatusBits.SeverityUncertainBit}`. The literals are interpolated from the consts, never typed by hand. Note\n> that QuestDB's \"not Bad\" is an unsigned LONG comparison, not T-SQL's `>= 0`.\n> - Health uses the shared `HistorianHealthCounters`. Events use `EventContinuation` and\n> `HistoricalEventRows.ClampSeverity`.\n> - **T5 owns `IT/QuestDbConformanceBackend.cs`** (seed and read-back only; N6). Step 3's throwaway backend IS this file,\n> which is kept. Only the throwaway subclasses are deleted.\n> - **Extra test (N2):** `ReadTests` includes one seeded Int64-kind row with `num_value = NULL`, proving the kind, not\n> the column, decides the value.\n\n\n**Goal:** `QuestDbHistorianDataSource` serves Raw, AtTime, Events and Processed. Results are identical to SQL Server's,\nexcept for microsecond rounding.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianDataSource.cs`,\n `QuestDbAggregateQueries.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/ReadTests.cs`\n\n**Acceptance Criteria:**\n- [ ] Every query filters `NOT deleted`. Every row goes through `StoredSample.FromStored`.\n- [ ] **Raw:** forward and reverse, `LIMIT n+1`, and the same paging contract and continuation semantics as the\n SqlServer data source (re-read from the last timestamp, inclusive). Unbounded when `maxValuesPerNode == 0`.\n Unknown tag \u21d2 an empty result. The tag is resolved through the `hist_tag` existence or the sample table, so the\n read never creates a tag.\n- [ ] **AtTime:** the result list matches the input in length and order. Neighbours come from the T0 Q9 form, and\n `AtTimeEvaluator.Evaluate` decides each value.\n- [ ] **Processed:**\n - It uses `BucketPlan.Create` and `AggregateFamilies.FamilyOf`.\n - It runs one statement per node, grouped by `(cast(ts AS LONG)*10 - @startUnixTicks) / @intervalTicks`.\n - It fills the shared `BucketRow` with the same usable/good/bad/non-numeric definitions as SQL Server: usable = status\n severity not Bad and kind \u2260 Empty; good = usable and status < 0x40000000; non-numeric = usable and kind in (4, 5).\n - `AggregateEvaluator.Evaluate` produces the values, and `BucketPlan.Assemble` fills empty buckets and handles\n reversal.\n - An unsupported aggregate throws `HistoryAggregateNotSupportedException`.\n- [ ] **Events:** filtered by source and time range, ascending, limited to `maxEvents`. The `ContinuationPoint` is\n non-null when truncated.\n- [ ] Health counters (`GetHealthSnapshot`) are implemented as in the SqlServer data source (lock-guarded success/failure counters plus the last error). A failure fails only that call and is recorded in\n `GetHealthSnapshot`.\n- [ ] `ReadTests` cover only QuestDB-specific behaviour:\n - tombstoned rows are invisible to Raw, AtTime and Processed (seed a tombstone directly);\n - a whole-range Processed read from 1601 to 9999 with interval 0 does not overflow;\n - a reverse paging walk with `maxValuesPerNode=1` returns every row.\n The shared literals run through the conformance suite in T7.\n\n**Verify:** with `HISTORIAN_QUESTDB_TEST_CONNECTION` set,\n`dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests --filter \"FullyQualifiedName~ReadTests\"`\npasses.\n\n**Steps:**\n- [ ] Step 1: write `ReadTests` (red).\n- [ ] Step 2: implement Raw, then AtTime, Events and Processed.\n- [ ] Step 3: temporarily compile the conformance read classes into this task's worktree. Add a throwaway concrete\n subclass and a throwaway backend that seeds through direct INSERTs. Run the Raw, AtTime and Processed\n conformance classes locally and get them green.\n- [ ] Step 4: delete the throwaway files before committing. T7 owns the real ones. Paste the conformance run output\n into the report.\n- [ ] Step 5: falsify the tombstone filter and the bucket expression (an off-by-one start).\n- [ ] Step 6: commit `feat(historian): QuestDB reads and aggregates (pushed down, shared evaluation)` with the trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianDataSource.cs\",\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbAggregateQueries.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/ReadTests.cs\"],\"verifyCommand\":\"dotnet test \u2026QuestDb.IntegrationTests --filter FullyQualifiedName~ReadTests (env set) + throwaway conformance run in report\",\"acceptanceCriteria\":[\"NOT deleted everywhere\",\"bucket expr exact vs SqlServer\",\"shared evaluators used\",\"conformance read classes green locally\"],\"modelTier\":\"frontier\",\"model\":\"opus\",\"wave\":3}\n```\n\n---", "blockedBy": [ 3 ] @@ -56,7 +56,7 @@ "id": 6, "subject": "Task 6: QuestDB HistoryUpdate (soft delete) with read-your-writ", "status": "pending", - "description": "**Goal:** `QuestDbHistorianDataUpdater` implements all five operations through insert-only upserts and tombstones. It\nreturns codes identical to SQL Server's, and waits for WAL visibility before returning.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianDataUpdater.cs`,\n `QuestDbWalVisibility.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/UpdateTests.cs`,\n `TombstoneTests.cs`\n\n**Acceptance Criteria:**\n- [ ] `UpdateDataAsync`:\n 1. `HistoryUpdatePlanner.Plan(tag, mode, values, resolutionTicks: 10)` builds the plan.\n 2. It reads which winner timestamps currently have a **live** row, meaning present and not deleted.\n 3. It writes:\n - Insert: only the absent winners;\n - Replace: only the present winners;\n - Update: every winner.\n Each write is an `INSERT` with `deleted=false`.\n 4. It builds the outcome map (present \u21d2 Replaced, absent \u21d2 Inserted, for rows it wrote).\n 5. `HistoryUpdatePlanner.Complete(...)` produces the codes.\n- [ ] `DeleteAtTimeAsync` writes a tombstone for each time with a live row, with `deleted=true`, value columns NULL and\n `status=BadNoData`. Codes come from `HistoryUpdatePlanner.DeleteAtTimeResults`.\n- [ ] `DeleteRawAsync` writes a tombstone for every live row in `[start, end)`. The code comes from\n `HistoryUpdatePlanner.DeleteRawResult`.\n- [ ] On an unknown tag, Insert and Update create the `hist_tag` row; Replace and Delete do not, and return the shared\n unknown-tag codes.\n- [ ] `QuestDbWalVisibility.WaitAsync(table, ct)` works as follows:\n - It polls the T0 Q4 query until the table's applied transaction is at least the sequencer transaction captured\n after our write.\n - It gives up after `VisibilityTimeout` and logs a Warning, without changing the codes. `VisibilityTimeout = 0`\n skips the wait.\n - Every updater operation calls it once, after writing.\n- [ ] `UpdateTests` cover the five operations' codes (QuestDB-specific checks only; the literal codes run through\n conformance in T7) and that a read made right after `UpdateDataAsync` returns sees the change.\n- [ ] `TombstoneTests` cover:\n - delete then Raw read \u21d2 the row is gone;\n - delete, then a `QuestDbHistorianValueWriter` redelivery of the same sample \u21d2 the row stays gone (this needs T4's\n writer: write the test against the interface, and if T4 is not merged yet, seed a tombstone and assert the\n writer's existence query counts it);\n - insert over a tombstone \u21d2 `GoodEntryInserted` and a visible row.\n\n**Verify:** with `HISTORIAN_QUESTDB_TEST_CONNECTION` set,\n`dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests --filter \"FullyQualifiedName~UpdateTests|FullyQualifiedName~TombstoneTests\"`\npasses.\n\n**Steps:**\n- [ ] Step 1: write the tests (red).\n- [ ] Step 2: implement `QuestDbWalVisibility`.\n- [ ] Step 3: implement the updater.\n- [ ] Step 4: get to green. Falsify the \"live row\" filter by dropping `NOT deleted` from the existence check, and the\n visibility wait by setting the timeout to 0 in the read-after-write test.\n- [ ] Step 5: commit `feat(historian): QuestDB HistoryUpdate with soft delete and read-your-writes` with the trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianDataUpdater.cs\",\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbWalVisibility.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/UpdateTests.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/TombstoneTests.cs\"],\"verifyCommand\":\"dotnet test \u2026QuestDb.IntegrationTests --filter UpdateTests|TombstoneTests (env set)\",\"acceptanceCriteria\":[\"insert-only upserts, no UPDATE/DELETE\",\"codes from shared planner\",\"tombstones never resurrected\",\"read-your-writes wait\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":3}\n```\n\n---", + "description": "> **Review amendments (Fable) \u2014 binding:**\n> - `QuestDbWalVisibility.cs` moves to T3, so T6 consumes it (R5).\n> - Insert and Update call `QuestDbTagCatalog.EnsureAsync`; Replace and the deletes call `ExistsAsync` (R7).\n> - **Existence read (S15):** one `WHERE tag=$1 AND ts >= $min AND ts <= $max AND NOT deleted` range query, filtered to\n> the winners client-side. Never an `IN` list of `n` parameters.\n> - **DeleteRaw (S12):** read live rows in `LIMIT` pages of the Q2 batch size, keyed on the last `ts`, and tombstone them\n> page by page. `rowsAffected` is the total.\n> - Inbound times are rounded and bound via `BindTimestamp` (R6).\n> - The TombstoneTests \"redelivery after delete\" case moves to T7 as\n> `IT/TombstoneTests.Redelivery.cs::Tombstone_is_not_resurrected_by_a_writer_redelivery` (S8). T6 keeps the other two.\n\n\n**Goal:** `QuestDbHistorianDataUpdater` implements all five operations through insert-only upserts and tombstones. It\nreturns codes identical to SQL Server's, and waits for WAL visibility before returning.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianDataUpdater.cs`,\n `QuestDbWalVisibility.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/UpdateTests.cs`,\n `TombstoneTests.cs`\n\n**Acceptance Criteria:**\n- [ ] `UpdateDataAsync`:\n 1. `HistoryUpdatePlanner.Plan(tag, mode, values, resolutionTicks: 10)` builds the plan.\n 2. It reads which winner timestamps currently have a **live** row, meaning present and not deleted.\n 3. It writes:\n - Insert: only the absent winners;\n - Replace: only the present winners;\n - Update: every winner.\n Each write is an `INSERT` with `deleted=false`.\n 4. It builds the outcome map (present \u21d2 Replaced, absent \u21d2 Inserted, for rows it wrote).\n 5. `HistoryUpdatePlanner.Complete(...)` produces the codes.\n- [ ] `DeleteAtTimeAsync` writes a tombstone for each time with a live row, with `deleted=true`, value columns NULL and\n `status=BadNoData`. Codes come from `HistoryUpdatePlanner.DeleteAtTimeResults`.\n- [ ] `DeleteRawAsync` writes a tombstone for every live row in `[start, end)`. The code comes from\n `HistoryUpdatePlanner.DeleteRawResult`.\n- [ ] On an unknown tag, Insert and Update create the `hist_tag` row; Replace and Delete do not, and return the shared\n unknown-tag codes.\n- [ ] `QuestDbWalVisibility.WaitAsync(table, ct)` works as follows:\n - It polls the T0 Q4 query until the table's applied transaction is at least the sequencer transaction captured\n after our write.\n - It gives up after `VisibilityTimeout` and logs a Warning, without changing the codes. `VisibilityTimeout = 0`\n skips the wait.\n - Every updater operation calls it once, after writing.\n- [ ] `UpdateTests` cover the five operations' codes (QuestDB-specific checks only; the literal codes run through\n conformance in T7) and that a read made right after `UpdateDataAsync` returns sees the change.\n- [ ] `TombstoneTests` cover:\n - delete then Raw read \u21d2 the row is gone;\n - delete, then a `QuestDbHistorianValueWriter` redelivery of the same sample \u21d2 the row stays gone (this needs T4's\n writer: write the test against the interface, and if T4 is not merged yet, seed a tombstone and assert the\n writer's existence query counts it);\n - insert over a tombstone \u21d2 `GoodEntryInserted` and a visible row.\n\n**Verify:** with `HISTORIAN_QUESTDB_TEST_CONNECTION` set,\n`dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests --filter \"FullyQualifiedName~UpdateTests|FullyQualifiedName~TombstoneTests\"`\npasses.\n\n**Steps:**\n- [ ] Step 1: write the tests (red).\n- [ ] Step 2: implement `QuestDbWalVisibility`.\n- [ ] Step 3: implement the updater.\n- [ ] Step 4: get to green. Falsify the \"live row\" filter by dropping `NOT deleted` from the existence check, and the\n visibility wait by setting the timeout to 0 in the read-after-write test.\n- [ ] Step 5: commit `feat(historian): QuestDB HistoryUpdate with soft delete and read-your-writes` with the trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianDataUpdater.cs\",\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbWalVisibility.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/UpdateTests.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/TombstoneTests.cs\"],\"verifyCommand\":\"dotnet test \u2026QuestDb.IntegrationTests --filter UpdateTests|TombstoneTests (env set)\",\"acceptanceCriteria\":[\"insert-only upserts, no UPDATE/DELETE\",\"codes from shared planner\",\"tombstones never resurrected\",\"read-your-writes wait\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":3}\n```\n\n---", "blockedBy": [ 3 ] @@ -65,7 +65,7 @@ "id": 7, "subject": "Task 7: Provider, Host wiring and the conformance suite on Ques", "status": "pending", - "description": "**Goal:** `Backend=QuestDb` resolves, validates and builds all five seams. The full conformance suite passes against\nQuestDB.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianProvider.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformanceBackend.cs`,\n `QuestDbConformance.cs` (one concrete subclass per conformance class)\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.Tests/QuestDbHistorianProviderTests.cs`\n- Modify:\n - `src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs` (add a `QuestDbBackendName`\n const, `BindQuestDb(IConfiguration)` and a `Factories` entry);\n - `ServerHistorianOptionsValidator.cs` (a QuestDb block before the `if (!options.IsGatewayBackend) return;` line,\n mirroring the SqlServer block);\n - `src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj` (a ProjectReference);\n - `tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/Configuration/HistorianBackendRegistryTests.cs`;\n - `tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/ServerHistorianOptionsValidatorTests.cs`;\n - `docker-dev/docker-compose.yml`: in the central-1 and central-2 `ServerHistorian__*` blocks, add\n `ServerHistorian__QuestDb__ConnectionString: \"${OTOPCUA_HISTORIAN_QUESTDB:-}\"`\n plus a comment line naming the overlay.\n\n**Acceptance Criteria:**\n- [ ] The provider has `Name=\"QuestDb\"`, `SupportedKinds` = all six kinds as a FrozenSet, and one lazily built shared\n connection and bootstrapper across the seams. The constructor does no I/O and no validation, like the SqlServer\n provider. `CreateUpdater` returns the soft-delete updater.\n- [ ] The registry resolves `QuestDb` case-insensitively.\n- [ ] The validator, when the backend is consumed and is `QuestDb`:\n - reports every `QuestDbHistorianOptions.Validate()` error, naming the key;\n - does not require the Gateway keys;\n - ignores an invalid `TablePrefix` when not consumed.\n- [ ] Registry and validator tests mirror the five SqlServer ones (lines 71 and 319\u2013368).\n- [ ] The QuestDB integration project runs the **entire** conformance suite through `QuestDbConformanceBackend`:\n - `ResolutionTicks = 10`;\n - seeding is direct INSERTs into the prefixed tables;\n - `ReadBackAsync` filters `NOT deleted`.\n Every class passes. Any named override has a doc comment and a ledger ruling.\n\n**Verify:**\n- With both environment variables set, the SqlServer and QuestDB integration projects both pass. The report shows the\n conformance class counts side by side.\n- `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests --filter \"FullyQualifiedName~Historian\"` passes.\n- `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests --filter \"FullyQualifiedName~Historian\"` passes.\n\n**Steps:**\n- [ ] Step 1: write the provider unit tests (red), then the provider.\n- [ ] Step 2: write the registry and validator tests (red), then the wiring.\n- [ ] Step 3: write `QuestDbConformanceBackend` and the subclasses, and run the full suite. Fix any divergence in the\n QuestDB code, even though those files belong to T4\u2013T6; this task is the integration owner in W4.\n- [ ] Step 4: make the compose env edit.\n- [ ] Step 5: build the solution, then commit `feat(historian): QuestDb provider + Host wiring; conformance suite green\n on both backends` with the trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianProvider.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformanceBackend.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformance.cs\",\"src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs\",\"src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/ServerHistorianOptionsValidator.cs\",\"src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj\",\"docker-dev/docker-compose.yml\"],\"verifyCommand\":\"both IT projects green with env set; Host historian filters green\",\"acceptanceCriteria\":[\"provider shape\",\"registry + consumer-gated validator\",\"full conformance suite green on QuestDB\",\"compose env\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":4}\n```\n\n---", + "description": "> **Review amendments (Fable) \u2014 binding:**\n> - The model is **opus** (R10). T7 may fix T4/T6 files and the data-source Raw/Events plumbing. Aggregate and AtTime\n> divergences are reported to the controller, not fixed.\n> - T5 owns `QuestDbConformanceBackend.cs`; T7 owns the subclasses (N6).\n> - T7 adds `IT/TombstoneTests.Redelivery.cs` (S8).\n> - **Compose default (S14):** the exact Q1-confirmed connection string, with `Host=questdb;Port=8812` substituted,\n> keeping every keyword T0 found necessary.\n\n\n**Goal:** `Backend=QuestDb` resolves, validates and builds all five seams. The full conformance suite passes against\nQuestDB.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianProvider.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformanceBackend.cs`,\n `QuestDbConformance.cs` (one concrete subclass per conformance class)\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.Tests/QuestDbHistorianProviderTests.cs`\n- Modify:\n - `src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs` (add a `QuestDbBackendName`\n const, `BindQuestDb(IConfiguration)` and a `Factories` entry);\n - `ServerHistorianOptionsValidator.cs` (a QuestDb block before the `if (!options.IsGatewayBackend) return;` line,\n mirroring the SqlServer block);\n - `src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj` (a ProjectReference);\n - `tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/Configuration/HistorianBackendRegistryTests.cs`;\n - `tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/ServerHistorianOptionsValidatorTests.cs`;\n - `docker-dev/docker-compose.yml`: in the central-1 and central-2 `ServerHistorian__*` blocks, add\n `ServerHistorian__QuestDb__ConnectionString: \"${OTOPCUA_HISTORIAN_QUESTDB:-}\"`\n plus a comment line naming the overlay.\n\n**Acceptance Criteria:**\n- [ ] The provider has `Name=\"QuestDb\"`, `SupportedKinds` = all six kinds as a FrozenSet, and one lazily built shared\n connection and bootstrapper across the seams. The constructor does no I/O and no validation, like the SqlServer\n provider. `CreateUpdater` returns the soft-delete updater.\n- [ ] The registry resolves `QuestDb` case-insensitively.\n- [ ] The validator, when the backend is consumed and is `QuestDb`:\n - reports every `QuestDbHistorianOptions.Validate()` error, naming the key;\n - does not require the Gateway keys;\n - ignores an invalid `TablePrefix` when not consumed.\n- [ ] Registry and validator tests mirror the five SqlServer ones (lines 71 and 319\u2013368).\n- [ ] The QuestDB integration project runs the **entire** conformance suite through `QuestDbConformanceBackend`:\n - `ResolutionTicks = 10`;\n - seeding is direct INSERTs into the prefixed tables;\n - `ReadBackAsync` filters `NOT deleted`.\n Every class passes. Any named override has a doc comment and a ledger ruling.\n\n**Verify:**\n- With both environment variables set, the SqlServer and QuestDB integration projects both pass. The report shows the\n conformance class counts side by side.\n- `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests --filter \"FullyQualifiedName~Historian\"` passes.\n- `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests --filter \"FullyQualifiedName~Historian\"` passes.\n\n**Steps:**\n- [ ] Step 1: write the provider unit tests (red), then the provider.\n- [ ] Step 2: write the registry and validator tests (red), then the wiring.\n- [ ] Step 3: write `QuestDbConformanceBackend` and the subclasses, and run the full suite. Fix any divergence in the\n QuestDB code, even though those files belong to T4\u2013T6; this task is the integration owner in W4.\n- [ ] Step 4: make the compose env edit.\n- [ ] Step 5: build the solution, then commit `feat(historian): QuestDb provider + Host wiring; conformance suite green\n on both backends` with the trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianProvider.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformanceBackend.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformance.cs\",\"src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs\",\"src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/ServerHistorianOptionsValidator.cs\",\"src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj\",\"docker-dev/docker-compose.yml\"],\"verifyCommand\":\"both IT projects green with env set; Host historian filters green\",\"acceptanceCriteria\":[\"provider shape\",\"registry + consumer-gated validator\",\"full conformance suite green on QuestDB\",\"compose env\"],\"modelTier\":\"frontier\",\"model\":\"opus\",\"wave\":4}\n```\n\n---", "blockedBy": [ 2, 4, @@ -77,7 +77,7 @@ "id": 8, "subject": "Task 8: Documentation", "status": "pending", - "description": "**Goal:** the docs describe the QuestDB backend as built, and every claim is checked against the merged code.\n\n**Files:** `docs/Historian.md` (a new \u00a7\"QuestDB backend\"), `docs/Configuration.md` (the `ServerHistorian:QuestDb`\ntable), `CLAUDE.md` (one short bullet in the Pluggable backends subsection), `docker-dev/README.md` (the overlay\nrecipe), and the spec (an as-built pass)\n\n**Acceptance Criteria:**\n- [ ] Every `ServerHistorian:QuestDb` key is documented with its default and validation rule.\n- [ ] The QuestDB-specific behaviour is documented:\n - soft delete and tombstones;\n - insert-if-absent including tombstones;\n - microsecond truncation and collisions;\n - WAL visibility and the read-your-writes wait;\n - TTL;\n - the overlay recipe and `HISTORIAN_QUESTDB_TEST_CONNECTION`.\n- [ ] Each statement is cross-checked against the code, and any spec-versus-code disagreement is fixed in the spec\n inline and listed in the report.\n\n**Verify:** the controller reads the diff, and `grep -n \"QuestDb\" docs/Configuration.md` shows the table.\n\n```json:metadata\n{\"files\":[\"docs/Historian.md\",\"docs/Configuration.md\",\"CLAUDE.md\",\"docker-dev/README.md\",\"docs/plans/2026-10-10-questdb-historian-design.md\"],\"verifyCommand\":\"grep -n QuestDb docs/Configuration.md\",\"acceptanceCriteria\":[\"keys documented\",\"QuestDB-specific semantics documented\",\"claims cross-checked\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":5}\n```\n\n---", + "description": "> **Review amendments (Fable) \u2014 binding:**\n> - Document these, in addition to the existing list:\n> - R6's sub-\u00b5s request-start consequence;\n> - QuestDB `LONG` NULL = `long.MinValue`, so a stored `FromInt64(long.MinValue)` reads back Empty (N1);\n> - TTL is enforced on write commits only, so a quiet table never drops (N3);\n> - all write paths wait for WAL visibility (R5).\n> - **CLAUDE.md (N5):** check the Historian section's opening paragraph. If it still says\n> \"HistorianGateway (sole historian backend)\", rewrite it to name all three backends. Otherwise add QuestDb to the\n> existing \"Pluggable backends\" subsection.\n\n\n**Goal:** the docs describe the QuestDB backend as built, and every claim is checked against the merged code.\n\n**Files:** `docs/Historian.md` (a new \u00a7\"QuestDB backend\"), `docs/Configuration.md` (the `ServerHistorian:QuestDb`\ntable), `CLAUDE.md` (one short bullet in the Pluggable backends subsection), `docker-dev/README.md` (the overlay\nrecipe), and the spec (an as-built pass)\n\n**Acceptance Criteria:**\n- [ ] Every `ServerHistorian:QuestDb` key is documented with its default and validation rule.\n- [ ] The QuestDB-specific behaviour is documented:\n - soft delete and tombstones;\n - insert-if-absent including tombstones;\n - microsecond truncation and collisions;\n - WAL visibility and the read-your-writes wait;\n - TTL;\n - the overlay recipe and `HISTORIAN_QUESTDB_TEST_CONNECTION`.\n- [ ] Each statement is cross-checked against the code, and any spec-versus-code disagreement is fixed in the spec\n inline and listed in the report.\n\n**Verify:** the controller reads the diff, and `grep -n \"QuestDb\" docs/Configuration.md` shows the table.\n\n```json:metadata\n{\"files\":[\"docs/Historian.md\",\"docs/Configuration.md\",\"CLAUDE.md\",\"docker-dev/README.md\",\"docs/plans/2026-10-10-questdb-historian-design.md\"],\"verifyCommand\":\"grep -n QuestDb docs/Configuration.md\",\"acceptanceCriteria\":[\"keys documented\",\"QuestDB-specific semantics documented\",\"claims cross-checked\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":5}\n```\n\n---", "blockedBy": [ 7 ] @@ -92,5 +92,5 @@ ] } ], - "lastUpdated": "2026-10-10T12:55:44.872528Z" + "lastUpdated": "2026-10-10T13:09:42.579032+00:00" } \ No newline at end of file