dohertj2/lmxopcua
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b3031dfea9496eb53a18b31aa47c0c5f242e5f66
lmxopcua/docs/plans/2026-10-10-questdb-historian-plan.md.tasks.json
T

96 lines
51 KiB
JSON
Raw Blame History

{
"planPath": "docs/plans/2026-10-10-questdb-historian-plan.md",
"tasks": [
{
"id": 0,
"subject": "Task 0: QuestDB spike, overlay compose and spec amendments",
"status": "pending",
"description": "**Goal:** prove every QuestDB behaviour the design assumes, against a pinned image, before any code is built on it.\nThe findings are written into spec \u00a72\u2013\u00a74.\n\n**Files:**\n- Create: `docker-dev/docker-compose.questdb.yml`\n- Create: `docs/plans/2026-10-10-questdb-spike.md`\n- Modify: `docs/plans/2026-10-10-questdb-historian-design.md` (\u00a72\u2013\u00a74: replace assumptions with confirmed forms, and mark\n each one \"T0-confirmed\")\n\n**Acceptance Criteria:**\n- [ ] The overlay pins an exact `questdb/questdb:<x.y.z>` tag. The tag's release date is at least two weeks before\n 2026-10-10, and is recorded in the spike doc.\n- [ ] The overlay exposes `18812:8812` and `19000:9000`, uses a named volume `otopcua-questdb-data`, and sets the\n container name `otopcua-dev-questdb-1`. It is brought up with\n `docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.questdb.yml up -d questdb`.\n- [ ] The spike doc records a working Npgsql connection string and answers Q1\u2013Q12 below, each with the exact SQL or\n C# that worked and its observed output.\n- [ ] Spec \u00a72.1 has the confirmed DDL, \u00a72.3 the confirmed visibility-wait query, \u00a73 the confirmed bucket-statistics and\n neighbour query forms, and \u00a74 the confirmed connection-string keywords.\n\n**Verify:** `docker exec otopcua-dev-questdb-1 true` exits 0. The spike harness output is pasted into the spike doc.\n\n**Steps:**\n\n- [ ] **Step 1: Create the overlay.**\n\n```yaml\n# docker-dev/docker-compose.questdb.yml \u2014 optional QuestDB fixture for the QuestDB historian backend.\n# Up: docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.questdb.yml up -d questdb\nservices:\n questdb:\n image: questdb/questdb:<PINNED> # T0 pins an exact tag \u2265 2 weeks old\n container_name: otopcua-dev-questdb-1\n ports:\n - \"18812:8812\" # PG wire\n - \"19000:9000\" # web console / REST\n volumes:\n - otopcua-questdb-data:/var/lib/questdb\n healthcheck:\n test: [\"CMD-SHELL\", \"curl -fs http://localhost:9003/status || exit 1\"]\n interval: 10s\n timeout: 5s\n retries: 12\nvolumes:\n otopcua-questdb-data:\n```\n\n- [ ] **Step 2: Write a throwaway C# harness** in the scratchpad, not in the repo. It is a console project that\n references `Npgsql`, restored from the existing central pin via a temporary `Directory.Packages.props` copy, or\n run with `dotnet run` inside a scratch folder under the repo with `ManagePackageVersionsCentrally`. Answer each\n question with real output:\n - **Q1.** The connection-string keywords that work: test `ServerCompatibilityMode=NoTypeLoading` with and without\n it. Also the parameter types needed for `TIMESTAMP` (UTC `DateTime`), `BYTE`, `SYMBOL`, `VARCHAR`, `BOOLEAN` and\n `LONG`.\n - **Q2.** Whether a parameterised multi-row `INSERT INTO t VALUES (\u2026),(\u2026)` works, and the largest row count per\n statement that works reliably. Pick the batch size: \u2264 1000 rows, capped by any parameter limit found.\n - **Q3.** `CREATE TABLE \u2026 WAL DEDUP UPSERT KEYS(ts, tag)`: re-inserting the same key replaces the row. Show\n before and after.\n - **Q4.** The `wal_tables()` columns. After an insert, how to know the transaction we produced and wait until it is\n applied (`writerTxn >= sequencerTxn`). Measure the typical lag over 20 runs.\n - **Q5.** TTL: the syntax at create (`TTL n DAYS`), and `ALTER TABLE \u2026 SET TTL n DAYS`. Where TTL shows up in\n `tables()` metadata.\n - **Q6.** Timestamp precision through Npgsql: a value with 100 ns ticks (`\u20261234567`) comes back as `\u2026123456` (truncated)\n or `\u2026123457` (rounded)? Confirm that our pre-truncation makes the round trip exact.\n - **Q7.** Two connections run `CREATE TABLE IF NOT EXISTS` for the same table at once, 20 times. Record any error\n text, and the catch rule that makes it idempotent.\n - **Q8.** Bucket statistics: one `GROUP BY` over the expression `(cast(ts AS LONG)*10 - $1) / $2`, with\n `sum(CASE WHEN \u2026 THEN 1 ELSE 0 END)`, `avg`, `min`, `max`, `sum`. Plus the first and last usable sample per bucket,\n with its value and time. Try `first()`/`last()` with a filter, or a window `row_number() OVER (PARTITION BY b\n ORDER BY ts)`. Plus arg-min/arg-max time, via a window function or a second statement.\n - **Q9.** AtTime neighbour lookup: chunked `LIMIT 1` seeks (`ts <= t ORDER BY ts DESC LIMIT 1` and\n `ts > t ORDER BY ts LIMIT 1`, combined with `UNION ALL` per time) versus `ASOF JOIN`. Time both for 100 timestamps\n and pick one.\n - **Q10.** VARCHAR and SYMBOL accept a 400-character string, and `SYMBOL` capacity is fine for tens of thousands of\n tags.\n - **Q11.** An existing-timestamp lookup for a batch: `WHERE tag=$1 AND ts >= $2 AND ts <= $3` (a range), or an `IN`\n list. Pick the one that works with parameters.\n - **Q12.** `DROP TABLE IF EXISTS` works for test cleanup, and `tables()` lists the tables with a given prefix.\n\n- [ ] **Step 3: Write the spike doc** with the image tag and date, the connection string, and Q1\u2013Q12 (SQL, output and\n decision for each).\n- [ ] **Step 4: Amend the spec \u00a72\u2013\u00a74** in place with the confirmed forms. If any assumption failed, write the\n replacement design inline and add a ledger ruling. If the failure breaks a user decision, stop and ask.\n- [ ] **Step 5: Commit**\n\n```bash\ngit add docker-dev/docker-compose.questdb.yml docs/plans/2026-10-10-questdb-spike.md docs/plans/2026-10-10-questdb-historian-design.md\ngit commit -m \"spike(historian): QuestDB behaviours confirmed against pinned image; spec \u00a72\u2013\u00a74 amended\n\nClaude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe\"\n```\n\n```json:metadata\n{\"files\":[\"docker-dev/docker-compose.questdb.yml\",\"docs/plans/2026-10-10-questdb-spike.md\",\"docs/plans/2026-10-10-questdb-historian-design.md\"],\"verifyCommand\":\"docker exec otopcua-dev-questdb-1 true\",\"acceptanceCriteria\":[\"pinned image \u22652 weeks old\",\"overlay ports 18812/19000\",\"Q1\u2013Q12 answered with output\",\"spec \u00a72\u2013\u00a74 amended\"],\"modelTier\":\"frontier\",\"model\":\"controller\",\"wave\":0}\n```\n\n---"
},
{
"id": 1,
"subject": "Task 1: Extract `Driver.Historian.Sql.Common` (no behaviour cha",
"status": "pending",
"description": "**Goal:** move every database-independent piece of `Driver.Historian.SqlServer` into a new shared library. The SQL\nServer backend then calls it, with byte-identical behaviour.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common.csproj`\n- Create in that project: `HistorianStatusCodes.cs`, `StatusBits.cs`, `HistorianInfoBits.cs`, `HistorianDataTypes.cs`,\n `SampleColumns.cs`, `HistorianSampleRow.cs`, `StoredSample.cs`, `BucketRow.cs`, `AggregateFamily.cs`,\n `AggregateEvaluator.cs`, `AtTimeEvaluator.cs`, `BucketPlan.cs`, `LiveWritePreparer.cs`, `HistoryUpdatePlanner.cs`,\n `HistorianTagNames.cs`, `BootstrapFailureCache.cs`, `HistorianSchemaMissingException.cs`, `AlarmEventIds.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common.Tests/` (csproj + tests)\n- Modify: every file in `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/` that contained the moved code\n (`SqlServerHistorianDataSource.cs`, `SqlAggregateQueries.cs`, `SqlHistorianDataTypes.cs` (delete),\n `SqlHistorianStatusCodes.cs` (keep only SQL Server-only codes, or delete if none), `SampleRowMapper.cs`,\n `SqlServerHistorianValueWriter.cs`, `SqlServerHistorianDataUpdater.cs`, `SqlServerTagCatalog.cs`,\n `SqlServerSchemaBootstrapper.cs`, the csproj)\n- Modify: the SqlServer unit tests that tested moved code. Move them into `Sql.Common.Tests` under the same test name.\n- Modify: `ZB.MOM.WW.OtOpcUa.slnx` (add both new projects alphabetically, next to lines 30 and 104)\n- Modify: `tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj` (add a\n ProjectReference to Sql.Common)\n\n**Acceptance Criteria:**\n- [ ] `Sql.Common` has no reference to `Microsoft.Data.SqlClient` or any other database client. Its references are\n Core.Abstractions, Server/Runtime (only if needed for `HistorizationValue`) and\n `Microsoft.Extensions.Logging.Abstractions`.\n- [ ] The public API matches the signatures in Step 1 exactly. T2\u2013T7 depend on them.\n- [ ] The SqlServer integration suite passes with the same test count as before the change, which is recorded in the\n report. The SqlServer unit suite plus the moved tests in `Sql.Common.Tests` has the same total count.\n- [ ] New `Sql.Common.Tests` unit tests cover `AtTimeEvaluator`, `AggregateEvaluator`, `BucketPlan`, `LiveWritePreparer`\n and `HistoryUpdatePlanner` directly, using the literals from the SqlServer integration tests. Each one is\n falsified once.\n- [ ] StatusCodeParity passes, and its constant count does not drop.\n\n**Verify:**\n- `dotnet build ZB.MOM.WW.OtOpcUa.slnx` gives 0 errors.\n- `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common.Tests` passes.\n- `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.Tests` passes.\n- With `HISTORIAN_SQL_TEST_CONNECTION` set, `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests`\n passes with the same count as before.\n- `dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests --filter \"FullyQualifiedName~StatusCodeParity\"`\n passes.\n\n**Steps:**\n\n- [ ] **Step 0: Record the baseline.** Run both SqlServer test projects and the parity filter, and write the exact\n pass counts into the report.\n\n- [ ] **Step 1: Create the shared API.** Namespace: `ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common`. Each moved body is\n transplanted verbatim from the source named in the comment; only names and visibility change.\n\n```csharp\n// HistorianStatusCodes.cs \u2014 from SqlHistorianStatusCodes + SqlHistorianUpdateStatusCodes (all consts, same values)\npublic static class HistorianStatusCodes\n{\n public const uint Good = 0x00000000u, GoodEntryInserted = 0x00A20000u, GoodEntryReplaced = 0x00A30000u;\n public const uint BadNoData = 0x809B0000u, BadEntryExists = 0x809F0000u, BadNoEntryExists = 0x80A00000u;\n public const uint BadInvalidTimestamp = 0x80230000u, BadAggregateInvalidInputs = 0x80D60000u;\n public const uint UncertainDataSubNormal = 0x40A40000u, BadOutOfRange = 0x803C0000u, BadInvalidArgument = 0x80AB0000u;\n}\n// StatusBits.cs / HistorianInfoBits.cs \u2014 moved verbatim from SqlHistorianStatusCodes.cs (made public)\n// HistorianDataTypes.cs \u2014 from SqlHistorianDataTypes: DefaultForEmpty, byte FromKind(HistorianValueKind)\n// SampleColumns.cs \u2014 record struct SampleColumns(byte ValueKind, double? NumValue, long? IntValue, string? StrValue)\n// + static class SampleColumnMapper { ToColumns(HistorianValue); FromColumns(byte,double?,long?,string?);\n// int ToDbStatus(uint); uint FromDbStatus(int); long ToLongStatus(uint) => status; uint FromLongStatus(long) }\n// HistorianSampleRow.cs \u2014 record struct HistorianSampleRow(DateTime TimeUtc, HistorianValue Value, uint StatusCode) (was SqlSampleRow)\npublic readonly record struct StoredSample(DateTime TimeUtc, HistorianValue Value, uint Status)\n{\n public bool IsUsable { get; } // verbatim rule from SqlServerHistorianDataSource\n public bool IsGood { get; }\n /// <summary>Applies the quality clamp (NULL value + non-Bad \u21d2 BadNoData) \u2014 from ReadStored line 533.</summary>\n public static StoredSample FromStored(DateTime timeUtc, HistorianValue value, uint status);\n}\npublic sealed class BucketRow // fields verbatim from the private BucketRow, as public get/set properties\n{\n public long Bucket { get; set; } public int Total { get; set; } public int Usable { get; set; }\n public int Good { get; set; } public int Bad { get; set; } public int NonNumeric { get; set; }\n public double? Avg { get; set; } public double? Min { get; set; } public double? Max { get; set; } public double? Sum { get; set; }\n public HistorianValue First { get; set; } public HistorianValue Last { get; set; }\n public double? Extreme { get; set; } public DateTime? ExtremeTime { get; set; }\n}\npublic enum AggregateFamily { Simple, StartEnd, ActualTime }\npublic static class AggregateFamilies { public static AggregateFamily? FamilyOf(HistoryAggregateType a); } // from SqlAggregateQueries.FamilyOf\npublic static class AggregateEvaluator\n{\n public static DataValueSnapshot Evaluate(HistoryAggregateType a, BucketRow r, DateTime bucketStart, DateTime now); // verbatim\n public static DataValueSnapshot NoData(DateTime t, DateTime now); // verbatim\n}\npublic static class AtTimeEvaluator\n{\n public static DataValueSnapshot Evaluate(DateTime t, StoredSample? prev, StoredSample? next, DateTime serverTime, DateTime now); // verbatim AtTimeValue\n}\npublic sealed class BucketPlan // from ReadProcessedAsync lines 355-370 and 395-408\n{\n public const int MaxBuckets = 100_000;\n public DateTime StartUtc { get; } public DateTime EndUtc { get; } public bool Reversed { get; }\n public long IntervalTicks { get; } public long BucketCount { get; }\n public long StartUnixTicks => (StartUtc - DateTime.UnixEpoch).Ticks; // for QuestDB's (cast(ts AS LONG)*10 - start)/interval\n /// <exception cref=\"ArgumentOutOfRangeException\">BucketCount > MaxBuckets (param name \"interval\") \u2014 same as today.</exception>\n public static BucketPlan Create(DateTime startUtc, DateTime endUtc, TimeSpan interval);\n public DateTime BucketStart(long index);\n /// <summary>One snapshot per bucket in window order; missing buckets get <paramref name=\"empty\"/>; reversed if the window was.</summary>\n public IReadOnlyList<DataValueSnapshot> Assemble(IReadOnlyDictionary<long, DataValueSnapshot> byBucket, Func<DateTime, DataValueSnapshot> empty);\n}\npublic static class LiveWritePreparer // from SqlServerHistorianValueWriter.Prepare, plus a resolution parameter\n{\n /// <param name=\"resolutionTicks\">1 = keep 100 ns (SQL Server); 10 = truncate to \u00b5s BEFORE de-duplication (QuestDB).</param>\n public static List<HistorianSampleRow> Prepare(IReadOnlyList<HistorizationValue> values, Func<DateTime> utcNow, long resolutionTicks = 1);\n}\npublic enum UpsertOutcome { Inserted, Replaced }\npublic sealed class UpdatePlan\n{\n public uint?[] Results { get; } // pre-decided codes (invalid ts, NaN, superseded duplicate\u2026); null = decided by the backend\n public IReadOnlyList<(int Index, HistorianSampleRow Row)> Winners { get; } // rows to send, de-duplicated per mode\n}\npublic static class HistoryUpdatePlanner // pre/post passes from SqlServerHistorianDataUpdater.UpdateDataAsync lines 56-108\n{\n public static UpdatePlan Plan(string tag, HistoryUpdateMode mode, IReadOnlyList<HistorizationValue> values, long resolutionTicks = 1);\n /// <param name=\"outcomeByTick\">ticks of each winner that the backend actually wrote \u2192 Inserted/Replaced; absent \u21d2 not written.</param>\n public static IReadOnlyList<uint> Complete(UpdatePlan plan, HistoryUpdateMode mode, IReadOnlyDictionary<long, UpsertOutcome> outcomeByTick);\n public static IReadOnlyList<uint> DeleteAtTimeResults(IReadOnlyList<DateTime> times, ISet<long> deletedTicks, bool tagKnown);\n public static uint DeleteRawResult(bool tagKnown, long rowsAffected);\n}\npublic static class HistorianTagNames { public const int MaxLength = 400; public static bool IsValid(string? name); } // from SqlServerTagCatalog.IsValidName\npublic sealed class BootstrapFailureCache // the 5 s negative-cache mechanism from SqlServerSchemaBootstrapper\n{\n public BootstrapFailureCache(TimeProvider clock, TimeSpan duration);\n public void ThrowIfCached(); public void Record(Exception ex); public void Clear();\n}\n// HistorianSchemaMissingException.cs \u2014 moved from SqlServerHistorianOptions.cs (public sealed, ctor(string message))\n// AlarmEventIds.cs \u2014 public static string ComputeEventId(...) moved verbatim from SqlServerAlarmHistorianWriter (same signature)\n```\n\n- [ ] **Step 2: Point the SqlServer backend at the shared lib.**\n - Replace each moved body with a call into `Sql.Common`.\n - Delete the now-empty SqlServer classes.\n - Keep `SqlAggregateQueries`' SQL text and `ReadBucket`, now filling the shared `BucketRow`.\n - Keep `SampleRowMapper`'s TVP builders, which take `HistorianSampleRow`.\n- [ ] **Step 3: Move the SqlServer unit tests for the moved code** into `Sql.Common.Tests`, keeping test names and\n literals: `StatusCodeOracleTests`, `ValueWriterPrepareTests`, the column round-trip parts of\n `SampleRowMapperTests`, and the data-type table test. The TVP parts stay in SqlServer.Tests.\n- [ ] **Step 4: Add direct unit tests that did not exist before.** For example:\n\n```csharp\n[Fact] public void AtTime_inside_an_outage_returns_the_bad_neighbours_code_with_no_value()\n{\n var t0 = new DateTime(2026,10,9,8,0,0,DateTimeKind.Utc);\n var prev = new StoredSample(t0, HistorianValue.Empty, 0x80310000u);\n var next = new StoredSample(t0.AddSeconds(10), HistorianValue.FromDouble(5), 0);\n var r = AtTimeEvaluator.Evaluate(t0.AddSeconds(5), prev, next, t0, t0.AddHours(1));\n r.StatusCode.ShouldBe(0x80310000u); r.Value.ShouldBeNull();\n}\n[Fact] public void BucketPlan_rejects_more_than_100k_buckets()\n => Should.Throw<ArgumentOutOfRangeException>(() =>\n BucketPlan.Create(DateTime.UnixEpoch, DateTime.UnixEpoch.AddSeconds(100_001), TimeSpan.FromSeconds(1)));\n[Fact] public void Prepare_truncates_to_microseconds_before_dedup_when_resolution_is_10()\n{\n var t = new DateTime(2026,10,9,8,0,0,DateTimeKind.Utc).AddTicks(1234567);\n var rows = LiveWritePreparer.Prepare([new(t, HistorianValue.FromDouble(1), 0), new(t.AddTicks(3), HistorianValue.FromDouble(2), 0)], () => t, 10);\n rows.Count.ShouldBe(1); rows[0].TimeUtc.Ticks.ShouldBe(t.Ticks - t.Ticks % 10); rows[0].Value.Number.ShouldBe(1);\n}\n```\n\n Add the matching `AggregateEvaluator` cases, using the 8-row ProcessedRead literals from\n `ProcessedReadTests.Numeric_aggregates_over_the_fixture`. Build the `BucketRow`s by hand from tag A's samples.\n- [ ] **Step 5: Wire the solution and parity test.** Add both projects to the slnx, and the ProjectReference to the\n Core.Abstractions.Tests csproj.\n- [ ] **Step 6: Run the Verify commands.** Counts must equal the Step 0 baseline, plus the new direct tests. Falsify\n each new direct test once by reverting the rule it pins.\n- [ ] **Step 7: Commit** with `refactor(historian): extract Driver.Historian.Sql.Common from the SQL Server backend (no\n behaviour change)` and the Claude-Session trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common/\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common.Tests/\",\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer/\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.Tests/\",\"ZB.MOM.WW.OtOpcUa.slnx\",\"tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj\"],\"verifyCommand\":\"dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests (env set) \u2014 same count as baseline\",\"acceptanceCriteria\":[\"no DB client ref in Sql.Common\",\"public API exactly as Step 1\",\"SqlServer suites same counts\",\"direct evaluator tests falsified\",\"parity green\"],\"modelTier\":\"frontier\",\"model\":\"opus\",\"wave\":1}\n```\n\n---",
"blockedBy": [
0
]
},
{
"id": 2,
"subject": "Task 2: Conformance suite (shared source), run against SQL Serv",
"status": "pending",
"description": "**Goal:** one backend-neutral suite of hand-computed behaviour tests, compiled into the SqlServer integration project\nand passing there. T7 later compiles the same suite into the QuestDB project.\n\n**Files:**\n- Create: `tests/Drivers/Historian.Conformance/IConformanceBackend.cs`, `ConformanceData.cs`,\n `RawReadConformance.cs`, `AtTimeConformance.cs`, `ProcessedConformance.cs`, `QualityContractConformance.cs`,\n `LiveWriteConformance.cs`, `UpdateConformance.cs`, `TagNameConformance.cs`\n- Modify: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/*.csproj` (add\n `<Compile Include=\"..\\Historian.Conformance\\**\\*.cs\" LinkBase=\"Conformance\" />` and a ProjectReference to Sql.Common)\n- Create in the SqlServer IT project: `SqlServerConformanceBackend.cs`, `SqlServerConformance.cs` (one concrete\n subclass per conformance class)\n- Modify in the SqlServer IT project: remove from the original files the generic tests that moved into the suite\n (`RawReadTests`, `AtTimeReadTests`, `ProcessedReadTests`, `QualityContractTests`, and the generic parts of\n `WriteTests`, `UpdateTests` and `LongNameTests`). SQL Server-specific tests stay where they are.\n\n**Acceptance Criteria:**\n- [ ] Every moved test keeps its method name and literal expectations. `git log -p` shows a move, not a rewrite.\n- [ ] The SqlServer IT total count after the change is at least the pre-change count, and the report has the before\n and after count.\n- [ ] Suite code references only `IConformanceBackend` and Core.Abstractions types. There is no `SqlServer*` type and\n no T-SQL in `tests/Drivers/Historian.Conformance`.\n- [ ] Timestamp assertions go through `B.Round(t)`.\n- [ ] Any test that cannot hold for every backend is a `virtual` member with a doc comment naming the difference.\n None should be needed for SQL Server.\n\n**Verify:** with `HISTORIAN_SQL_TEST_CONNECTION` set, `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests`\npasses with a count of at least the baseline. Without the variable, every test skips.\n\n**Steps:**\n\n- [ ] **Step 1: Define the backend contract.**\n\n```csharp\nnamespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Conformance;\npublic interface IConformanceBackend : IAsyncDisposable\n{\n /// <summary>1 for SQL Server (100 ns), 10 for QuestDB (\u00b5s).</summary>\n long ResolutionTicks { get; }\n DateTime Round(DateTime t) => new(t.Ticks - t.Ticks % ResolutionTicks, DateTimeKind.Utc);\n IHistorianDataSource CreateDataSource(TimeProvider? clock = null);\n IHistorianValueWriter CreateValueWriter(Func<DateTime>? utcNow = null);\n IHistorianDataUpdater CreateUpdater();\n /// <summary>Writes rows directly in the backend's schema (never through the writer), status verbatim.</summary>\n Task SeedAsync(string tag, params (DateTime T, HistorianValue V, uint Status)[] samples);\n /// <summary>Live (non-deleted) rows for a tag, ascending, read directly from storage.</summary>\n Task<IReadOnlyList<(DateTime T, HistorianValue V, uint Status)>> ReadBackAsync(string tag);\n}\npublic abstract class ConformanceBase : IAsyncLifetime\n{\n protected IConformanceBackend B { get; private set; } = null!;\n protected abstract Task<IConformanceBackend> CreateBackendAsync(); // calls fixture.SkipIfUnavailable()\n public async ValueTask InitializeAsync() => B = await CreateBackendAsync();\n public async ValueTask DisposeAsync() => await B.DisposeAsync();\n}\n```\n\n `ConformanceData` holds `T0`, `At(seconds)`, the status consts and `SeedAAsync(B)`, verbatim from `ReadTestDb`.\n- [ ] **Step 2: Move the generic tests** listed in the T1 inventory (Raw, AtTime, Processed literals, the quality\n contract, live-write redelivery and first-wins, update result codes, tag names over 400 characters) into the\n suite classes. Write each one as an abstract class `RawReadConformance : ConformanceBase` with `[Fact]` methods.\n- [ ] **Step 3: Implement `SqlServerConformanceBackend`.** It reuses `SqlHistorianFixture`, a unique schema and the\n existing seeding SQL from `ReadTestDb`. Add one concrete class per suite:\n `[Collection(SqlHistorianCollection.Name)] public sealed class SqlServerRawReadConformance(SqlHistorianFixture fx) : RawReadConformance`.\n- [ ] **Step 4: Run the tests and compare counts.** Commit with\n `test(historian): backend conformance suite (shared source), run against SQL Server` and the trailer.\n\n```json:metadata\n{\"files\":[\"tests/Drivers/Historian.Conformance/\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests/\"],\"verifyCommand\":\"dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests (env set) \u2265 baseline count\",\"acceptanceCriteria\":[\"tests moved by name with same literals\",\"count \u2265 baseline\",\"suite is backend-neutral\",\"Round() for timestamps\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":2}\n```\n\n---",
"blockedBy": [
1
]
},
{
"id": 3,
"subject": "Task 3: QuestDB skeleton (options, connection, bootstrapper, fi",
"status": "pending",
"description": "**Goal:** a building `Driver.Historian.QuestDb` project. It has validated options, a pooled Npgsql connection, an\nidempotent and concurrency-safe schema bootstrapper with TTL, and an env-gated integration fixture.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.csproj`\n (references Core.Abstractions, Core.AlarmHistorian, Server/Runtime and Sql.Common; packages `Npgsql` and\n `Microsoft.Extensions.Logging.Abstractions`; InternalsVisibleTo the two QuestDb test assemblies)\n- Create: `QuestDbHistorianOptions.cs`, `QuestDbConnection.cs`, `QuestDbTables.cs`, `QuestDbSchemaBootstrapper.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.Tests/` (csproj, `QuestDbHistorianOptionsTests.cs`,\n `QuestDbTablesTests.cs`)\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/` (csproj with the\n conformance-include line and a ProjectReference to Sql.Common, `QuestDbFixture.cs`, `BootstrapTests.cs`)\n- Modify: `ZB.MOM.WW.OtOpcUa.slnx`, and the Core.Abstractions.Tests csproj (ProjectReference to the QuestDb project)\n\n**Acceptance Criteria:**\n- [ ] `QuestDbHistorianOptions` has `ConnectionString` (default `\"\"`), `TablePrefix` (`\"hist_\"`, regex\n `^[A-Za-z0-9_]+$`), `RetentionDays` (`int?`, \u2265 1), `CommandTimeout` (30 s, > 0), `AutoCreateSchema` (`true`) and\n `VisibilityTimeout` (5 s, \u2265 0). `Validate()` returns error strings that name the key\n `ServerHistorian:QuestDb:<Key>`.\n- [ ] `QuestDbTables` produces the three `CREATE TABLE IF NOT EXISTS` statements, as confirmed by T0, for a given prefix\n and retention. Unit tests pin the exact text.\n- [ ] The bootstrapper is idempotent and survives two concurrent instances, using the catch rule from T0 Q7. It\n re-applies a changed TTL. It uses `BootstrapFailureCache` for the 5 s negative cache.\n- [ ] `QuestDbFixture` reads `HISTORIAN_QUESTDB_TEST_CONNECTION` and skips when it is absent. `UniquePrefix()` returns\n `\"t\" + 12 hex + \"_\"`. `DropPrefixAsync(prefix)` drops every table with that prefix, found through `tables()`.\n- [ ] `BootstrapTests` cover:\n - tables created;\n - a second run is a no-op;\n - two concurrent bootstrappers both succeed;\n - TTL is visible in metadata when `RetentionDays=7` and absent when unset;\n - `AutoCreateSchema=false` against a missing table throws `HistorianSchemaMissingException` from Sql.Common.\n\n**Verify:**\n- `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.Tests` passes.\n- With `HISTORIAN_QUESTDB_TEST_CONNECTION` set, `dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests --filter \"FullyQualifiedName~Bootstrap\"`\n passes.\n- Without the variable, every test skips.\n\n**Steps:**\n- [ ] Step 1: write the options with `Validate()` and its unit tests (red, then green).\n- [ ] Step 2: write `QuestDbTables` DDL builders (use the spec \u00a72.1 *as amended by T0*), with unit tests pinning the\n text for prefix `hist_` with and without TTL.\n- [ ] Step 3: write `QuestDbConnection`.\n - Members:\n - `NpgsqlDataSource` built once;\n - `OpenAsync(ct)`;\n - `CreateCommand(conn, sql)` with `CommandTimeout`;\n - `static bool IsTransient(Exception)`, true for `NpgsqlException` with `IsTransient`, socket and IO errors, and\n timeouts;\n - `static DateTime ReadUtc(NpgsqlDataReader r, int i)`;\n - `string T(string table)`, which returns prefix + table.\n - Use the parameter typing from T0 Q1.\n- [ ] Step 4: write the bootstrapper and fixture. Bring up the overlay\n (`docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.questdb.yml up -d questdb`), then\n write and run `BootstrapTests`.\n- [ ] Step 5: wire the slnx and the parity-test reference, then build the solution.\n- [ ] Step 6: commit `feat(historian): QuestDB backend skeleton \u2014 options, connection, schema bootstrap with TTL` with\n the trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.Tests/\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/\",\"ZB.MOM.WW.OtOpcUa.slnx\"],\"verifyCommand\":\"dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests --filter FullyQualifiedName~Bootstrap (env set)\",\"acceptanceCriteria\":[\"options validate naming keys\",\"DDL pinned by unit tests\",\"bootstrap idempotent + concurrent + TTL\",\"fixture env-gated with unique prefix\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":2}\n```\n\n---",
"blockedBy": [
1
]
},
{
"id": 4,
"subject": "Task 4: QuestDB writes (value writer, tag provisioner, alarm wr",
"status": "pending",
"description": "**Goal:** the recorder, provisioning and alarm-history seams write to QuestDB with the same semantics as SQL Server.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianValueWriter.cs`,\n `QuestDbTagProvisioner.cs`, `QuestDbAlarmHistorianWriter.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/WriteTests.cs`,\n `ProvisioningTests.cs`, `AlarmWriterTests.cs`\n\n**Acceptance Criteria:**\n- [ ] The value writer works as follows:\n - It calls `LiveWritePreparer.Prepare(values, utcNow, resolutionTicks: 10)`.\n - It rejects an invalid tag name (`HistorianTagNames.IsValid`) and returns `false`.\n - It reads the existing timestamps for the tag in the batch range, including tombstones, using the T0 Q11 form.\n - It inserts only the rows that are not yet present, in multi-row parameterised `INSERT`s of at most the T0 Q2\n batch size, with `deleted=false` and `server_ts=utcNow()`.\n - It returns `true` with no SQL when every row already existed.\n - Any exception returns `false`. It never throws, and cancellation also returns `false`.\n- [ ] The provisioner upserts `hist_tag` (`ts = 1970-01-01`, data type from `HistorianDataTypes.FromKind`, eng unit,\n description). Both the request field types and the result tally match the SqlServer provisioner's.\n- [ ] The alarm writer works as follows:\n - It uses the same deterministic `event_id` as the SQL Server writer, through `AlarmEventIds.ComputeEventId` from\n Sql.Common.\n - It inserts events through the dedup key `(ts, event_id)`, so a redelivery is idempotent.\n - `Classify` maps an unknown or non-database exception to RetryPlease, and maps permanent errors as the SqlServer\n writer does.\n- [ ] `WriteTests` cover:\n - typed round trip for all six kinds with a non-Good status;\n - a redelivered batch leaves one row and the first value;\n - a tombstoned key is not resurrected by redelivery (seed a tombstone directly);\n - NaN \u21d2 `BadOutOfRange` Empty;\n - a microsecond collision within a batch keeps the first;\n - a stopped port (`Port=1`) returns `false` without throwing.\n- [ ] Each regression test is falsified once.\n\n**Verify:** with `HISTORIAN_QUESTDB_TEST_CONNECTION` set,\n`dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests --filter \"FullyQualifiedName~Write|FullyQualifiedName~Provisioning|FullyQualifiedName~AlarmWriter\"`\npasses.\n\n**Steps:**\n- [ ] Step 1: write `WriteTests` (red).\n- [ ] Step 2: implement the value writer.\n- [ ] Step 3: make `WriteTests` green, then falsify the first-wins, tombstone and NaN tests.\n- [ ] Step 4: do the provisioner, then the alarm writer, each tests-first.\n- [ ] Step 5: commit `feat(historian): QuestDB live writes, provisioning and alarm events` with the trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianValueWriter.cs\",\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbTagProvisioner.cs\",\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbAlarmHistorianWriter.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/WriteTests.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/ProvisioningTests.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/AlarmWriterTests.cs\"],\"verifyCommand\":\"dotnet test \u2026QuestDb.IntegrationTests --filter Write|Provisioning|AlarmWriter (env set)\",\"acceptanceCriteria\":[\"insert-if-absent incl tombstones\",\"never throws\",\"event id deterministic + idempotent\",\"falsified\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":3}\n```\n\n---",
"blockedBy": [
3
]
},
{
"id": 5,
"subject": "Task 5: QuestDB reads and aggregates",
"status": "pending",
"description": "**Goal:** `QuestDbHistorianDataSource` serves Raw, AtTime, Events and Processed. Results are identical to SQL Server's,\nexcept for microsecond rounding.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianDataSource.cs`,\n `QuestDbAggregateQueries.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/ReadTests.cs`\n\n**Acceptance Criteria:**\n- [ ] Every query filters `NOT deleted`. Every row goes through `StoredSample.FromStored`.\n- [ ] **Raw:** forward and reverse, `LIMIT n+1`, and the same paging contract and continuation semantics as the\n SqlServer data source (re-read from the last timestamp, inclusive). Unbounded when `maxValuesPerNode == 0`.\n Unknown tag \u21d2 an empty result. The tag is resolved through the `hist_tag` existence or the sample table, so the\n read never creates a tag.\n- [ ] **AtTime:** the result list matches the input in length and order. Neighbours come from the T0 Q9 form, and\n `AtTimeEvaluator.Evaluate` decides each value.\n- [ ] **Processed:**\n - It uses `BucketPlan.Create` and `AggregateFamilies.FamilyOf`.\n - It runs one statement per node, grouped by `(cast(ts AS LONG)*10 - @startUnixTicks) / @intervalTicks`.\n - It fills the shared `BucketRow` with the same usable/good/bad/non-numeric definitions as SQL Server: usable = status\n severity not Bad and kind \u2260 Empty; good = usable and status < 0x40000000; non-numeric = usable and kind in (4, 5).\n - `AggregateEvaluator.Evaluate` produces the values, and `BucketPlan.Assemble` fills empty buckets and handles\n reversal.\n - An unsupported aggregate throws `HistoryAggregateNotSupportedException`.\n- [ ] **Events:** filtered by source and time range, ascending, limited to `maxEvents`. The `ContinuationPoint` is\n non-null when truncated.\n- [ ] Health counters (`GetHealthSnapshot`) are implemented as in the SqlServer data source (lock-guarded success/failure counters plus the last error). A failure fails only that call and is recorded in\n `GetHealthSnapshot`.\n- [ ] `ReadTests` cover only QuestDB-specific behaviour:\n - tombstoned rows are invisible to Raw, AtTime and Processed (seed a tombstone directly);\n - a whole-range Processed read from 1601 to 9999 with interval 0 does not overflow;\n - a reverse paging walk with `maxValuesPerNode=1` returns every row.\n The shared literals run through the conformance suite in T7.\n\n**Verify:** with `HISTORIAN_QUESTDB_TEST_CONNECTION` set,\n`dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests --filter \"FullyQualifiedName~ReadTests\"`\npasses.\n\n**Steps:**\n- [ ] Step 1: write `ReadTests` (red).\n- [ ] Step 2: implement Raw, then AtTime, Events and Processed.\n- [ ] Step 3: temporarily compile the conformance read classes into this task's worktree. Add a throwaway concrete\n subclass and a throwaway backend that seeds through direct INSERTs. Run the Raw, AtTime and Processed\n conformance classes locally and get them green.\n- [ ] Step 4: delete the throwaway files before committing. T7 owns the real ones. Paste the conformance run output\n into the report.\n- [ ] Step 5: falsify the tombstone filter and the bucket expression (an off-by-one start).\n- [ ] Step 6: commit `feat(historian): QuestDB reads and aggregates (pushed down, shared evaluation)` with the trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianDataSource.cs\",\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbAggregateQueries.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/ReadTests.cs\"],\"verifyCommand\":\"dotnet test \u2026QuestDb.IntegrationTests --filter FullyQualifiedName~ReadTests (env set) + throwaway conformance run in report\",\"acceptanceCriteria\":[\"NOT deleted everywhere\",\"bucket expr exact vs SqlServer\",\"shared evaluators used\",\"conformance read classes green locally\"],\"modelTier\":\"frontier\",\"model\":\"opus\",\"wave\":3}\n```\n\n---",
"blockedBy": [
3
]
},
{
"id": 6,
"subject": "Task 6: QuestDB HistoryUpdate (soft delete) with read-your-writ",
"status": "pending",
"description": "**Goal:** `QuestDbHistorianDataUpdater` implements all five operations through insert-only upserts and tombstones. It\nreturns codes identical to SQL Server's, and waits for WAL visibility before returning.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianDataUpdater.cs`,\n `QuestDbWalVisibility.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/UpdateTests.cs`,\n `TombstoneTests.cs`\n\n**Acceptance Criteria:**\n- [ ] `UpdateDataAsync`:\n 1. `HistoryUpdatePlanner.Plan(tag, mode, values, resolutionTicks: 10)` builds the plan.\n 2. It reads which winner timestamps currently have a **live** row, meaning present and not deleted.\n 3. It writes:\n - Insert: only the absent winners;\n - Replace: only the present winners;\n - Update: every winner.\n Each write is an `INSERT` with `deleted=false`.\n 4. It builds the outcome map (present \u21d2 Replaced, absent \u21d2 Inserted, for rows it wrote).\n 5. `HistoryUpdatePlanner.Complete(...)` produces the codes.\n- [ ] `DeleteAtTimeAsync` writes a tombstone for each time with a live row, with `deleted=true`, value columns NULL and\n `status=BadNoData`. Codes come from `HistoryUpdatePlanner.DeleteAtTimeResults`.\n- [ ] `DeleteRawAsync` writes a tombstone for every live row in `[start, end)`. The code comes from\n `HistoryUpdatePlanner.DeleteRawResult`.\n- [ ] On an unknown tag, Insert and Update create the `hist_tag` row; Replace and Delete do not, and return the shared\n unknown-tag codes.\n- [ ] `QuestDbWalVisibility.WaitAsync(table, ct)` works as follows:\n - It polls the T0 Q4 query until the table's applied transaction is at least the sequencer transaction captured\n after our write.\n - It gives up after `VisibilityTimeout` and logs a Warning, without changing the codes. `VisibilityTimeout = 0`\n skips the wait.\n - Every updater operation calls it once, after writing.\n- [ ] `UpdateTests` cover the five operations' codes (QuestDB-specific checks only; the literal codes run through\n conformance in T7) and that a read made right after `UpdateDataAsync` returns sees the change.\n- [ ] `TombstoneTests` cover:\n - delete then Raw read \u21d2 the row is gone;\n - delete, then a `QuestDbHistorianValueWriter` redelivery of the same sample \u21d2 the row stays gone (this needs T4's\n writer: write the test against the interface, and if T4 is not merged yet, seed a tombstone and assert the\n writer's existence query counts it);\n - insert over a tombstone \u21d2 `GoodEntryInserted` and a visible row.\n\n**Verify:** with `HISTORIAN_QUESTDB_TEST_CONNECTION` set,\n`dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests --filter \"FullyQualifiedName~UpdateTests|FullyQualifiedName~TombstoneTests\"`\npasses.\n\n**Steps:**\n- [ ] Step 1: write the tests (red).\n- [ ] Step 2: implement `QuestDbWalVisibility`.\n- [ ] Step 3: implement the updater.\n- [ ] Step 4: get to green. Falsify the \"live row\" filter by dropping `NOT deleted` from the existence check, and the\n visibility wait by setting the timeout to 0 in the read-after-write test.\n- [ ] Step 5: commit `feat(historian): QuestDB HistoryUpdate with soft delete and read-your-writes` with the trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianDataUpdater.cs\",\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbWalVisibility.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/UpdateTests.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/TombstoneTests.cs\"],\"verifyCommand\":\"dotnet test \u2026QuestDb.IntegrationTests --filter UpdateTests|TombstoneTests (env set)\",\"acceptanceCriteria\":[\"insert-only upserts, no UPDATE/DELETE\",\"codes from shared planner\",\"tombstones never resurrected\",\"read-your-writes wait\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":3}\n```\n\n---",
"blockedBy": [
3
]
},
{
"id": 7,
"subject": "Task 7: Provider, Host wiring and the conformance suite on Ques",
"status": "pending",
"description": "**Goal:** `Backend=QuestDb` resolves, validates and builds all five seams. The full conformance suite passes against\nQuestDB.\n\n**Files:**\n- Create: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianProvider.cs`\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformanceBackend.cs`,\n `QuestDbConformance.cs` (one concrete subclass per conformance class)\n- Create: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.Tests/QuestDbHistorianProviderTests.cs`\n- Modify:\n - `src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs` (add a `QuestDbBackendName`\n const, `BindQuestDb(IConfiguration)` and a `Factories` entry);\n - `ServerHistorianOptionsValidator.cs` (a QuestDb block before the `if (!options.IsGatewayBackend) return;` line,\n mirroring the SqlServer block);\n - `src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj` (a ProjectReference);\n - `tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/Configuration/HistorianBackendRegistryTests.cs`;\n - `tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/ServerHistorianOptionsValidatorTests.cs`;\n - `docker-dev/docker-compose.yml`: in the central-1 and central-2 `ServerHistorian__*` blocks, add\n `ServerHistorian__QuestDb__ConnectionString: \"${OTOPCUA_HISTORIAN_QUESTDB:-<T0 connection string with Host=questdb;Port=8812>}\"`\n plus a comment line naming the overlay.\n\n**Acceptance Criteria:**\n- [ ] The provider has `Name=\"QuestDb\"`, `SupportedKinds` = all six kinds as a FrozenSet, and one lazily built shared\n connection and bootstrapper across the seams. The constructor does no I/O and no validation, like the SqlServer\n provider. `CreateUpdater` returns the soft-delete updater.\n- [ ] The registry resolves `QuestDb` case-insensitively.\n- [ ] The validator, when the backend is consumed and is `QuestDb`:\n - reports every `QuestDbHistorianOptions.Validate()` error, naming the key;\n - does not require the Gateway keys;\n - ignores an invalid `TablePrefix` when not consumed.\n- [ ] Registry and validator tests mirror the five SqlServer ones (lines 71 and 319\u2013368).\n- [ ] The QuestDB integration project runs the **entire** conformance suite through `QuestDbConformanceBackend`:\n - `ResolutionTicks = 10`;\n - seeding is direct INSERTs into the prefixed tables;\n - `ReadBackAsync` filters `NOT deleted`.\n Every class passes. Any named override has a doc comment and a ledger ruling.\n\n**Verify:**\n- With both environment variables set, the SqlServer and QuestDB integration projects both pass. The report shows the\n conformance class counts side by side.\n- `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests --filter \"FullyQualifiedName~Historian\"` passes.\n- `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests --filter \"FullyQualifiedName~Historian\"` passes.\n\n**Steps:**\n- [ ] Step 1: write the provider unit tests (red), then the provider.\n- [ ] Step 2: write the registry and validator tests (red), then the wiring.\n- [ ] Step 3: write `QuestDbConformanceBackend` and the subclasses, and run the full suite. Fix any divergence in the\n QuestDB code, even though those files belong to T4\u2013T6; this task is the integration owner in W4.\n- [ ] Step 4: make the compose env edit.\n- [ ] Step 5: build the solution, then commit `feat(historian): QuestDb provider + Host wiring; conformance suite green\n on both backends` with the trailer.\n\n```json:metadata\n{\"files\":[\"src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb/QuestDbHistorianProvider.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformanceBackend.cs\",\"tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/QuestDbConformance.cs\",\"src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/HistorianBackendRegistry.cs\",\"src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/ServerHistorianOptionsValidator.cs\",\"src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj\",\"docker-dev/docker-compose.yml\"],\"verifyCommand\":\"both IT projects green with env set; Host historian filters green\",\"acceptanceCriteria\":[\"provider shape\",\"registry + consumer-gated validator\",\"full conformance suite green on QuestDB\",\"compose env\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":4}\n```\n\n---",
"blockedBy": [
2,
4,
5,
6
]
},
{
"id": 8,
"subject": "Task 8: Documentation",
"status": "pending",
"description": "**Goal:** the docs describe the QuestDB backend as built, and every claim is checked against the merged code.\n\n**Files:** `docs/Historian.md` (a new \u00a7\"QuestDB backend\"), `docs/Configuration.md` (the `ServerHistorian:QuestDb`\ntable), `CLAUDE.md` (one short bullet in the Pluggable backends subsection), `docker-dev/README.md` (the overlay\nrecipe), and the spec (an as-built pass)\n\n**Acceptance Criteria:**\n- [ ] Every `ServerHistorian:QuestDb` key is documented with its default and validation rule.\n- [ ] The QuestDB-specific behaviour is documented:\n - soft delete and tombstones;\n - insert-if-absent including tombstones;\n - microsecond truncation and collisions;\n - WAL visibility and the read-your-writes wait;\n - TTL;\n - the overlay recipe and `HISTORIAN_QUESTDB_TEST_CONNECTION`.\n- [ ] Each statement is cross-checked against the code, and any spec-versus-code disagreement is fixed in the spec\n inline and listed in the report.\n\n**Verify:** the controller reads the diff, and `grep -n \"QuestDb\" docs/Configuration.md` shows the table.\n\n```json:metadata\n{\"files\":[\"docs/Historian.md\",\"docs/Configuration.md\",\"CLAUDE.md\",\"docker-dev/README.md\",\"docs/plans/2026-10-10-questdb-historian-design.md\"],\"verifyCommand\":\"grep -n QuestDb docs/Configuration.md\",\"acceptanceCriteria\":[\"keys documented\",\"QuestDB-specific semantics documented\",\"claims cross-checked\"],\"modelTier\":\"standard\",\"model\":\"sonnet\",\"wave\":5}\n```\n\n---",
"blockedBy": [
7
]
},
{
"id": 9,
"subject": "Task 9: Live gate on docker-dev (controller)",
"status": "pending",
"description": "**Final review:** a **fable** reviewer reads `git diff master...feat/questdb-historian` together with the spec. It must\nreport PASS or FAIL on each of these checks:\n1. SqlServer behaviour is unchanged: identical conformance results, and no deleted SqlServer test.\n2. No QuestDB `UPDATE` or `DELETE` is issued on data.\n3. `NOT deleted` is present in every read path.\n4. Every new status literal is a named const.\n5. No package or version changes.\n6. The bucket expression is exact.\n7. #491 still holds: no Self/Context access after an await.\n\nThen one fix dispatch (opus) and a scoped re-review (sonnet), followed by adjudication.\n\n**Task 9: live gate (controller only).**\n\n**Goal:** prove `Backend=QuestDb` end to end on the docker-dev rig, against the overlay `questdb` service, with captured\noutput for every leg.\n\n> **USER-ORDERED GATE \u2014 NON-SKIPPABLE.** This task was requested by the user in the current conversation. It MUST NOT\n> be closed by walking around it, by declaring it \"verified inline\", or by substituting a cheaper check. Close only\n> after every item in `acceptanceCriteria` has been re-validated independently, with output captured.\n\n**Files:** `docs/plans/2026-10-10-questdb-historian-live-gate.md`\n\n**Acceptance Criteria:** run the six legs of spec \u00a77 on docker-dev with `OTOPCUA_HISTORIAN_BACKEND=QuestDb`. Each leg\nneeds captured output.\n\n**Verify:** the evidence doc has each leg's CLI output plus the direct QuestDB query output.\n\n**Steps:**\n1. Bring up the overlay `questdb`.\n2. Rebuild the central image and recreate the central nodes with `OTOPCUA_HISTORIAN_ENABLED=true`,\n `OTOPCUA_HISTORIAN_BACKEND=QuestDb` and `OTOPCUA_CONTHIST_ENABLED=true`.\n3. Historize FastUInt1, a string calculation tag and `HR5`. Add a UNS reference. Mint the two gate API keys, as in the\n SQL Server gate recipe (`docs/plans/2026-10-09-pluggable-historian-live-gate.md`), and deploy.\n4. Run legs 1\u20136 with direct `psql` or REST queries. QuestDB REST is `curl -G localhost:19000/exec --data-urlencode \"query=\u2026\"`.\n5. Write the evidence doc.\n6. Restore the rig: recreate the centrals with defaults, revert the config DB, redeploy, `down` the overlay, and stop\n the fixtures.\n\n```json:metadata\n{\"files\":[\"docs/plans/2026-10-10-questdb-historian-live-gate.md\"],\"verifyCommand\":\"Client.CLI historyread/historyupdate against opc.tcp://localhost:4840 + QuestDB REST /exec on :19000\",\"acceptanceCriteria\":[\"Primary-only typed recording one row per value\",\"raw+UNS HistoryRead/Processed identical and match QuestDB\",\"HistoryUpdate codes + delete not resurrected by redelivery + WriteOperate-only and anonymous denied\",\"failover 1x\",\"Modbus loss: BadCommunicationError then BadNoCommunication marker then Good\",\"TTL visible in tables() metadata\"],\"userGate\":true,\"tags\":[\"user-gate\"],\"modelTier\":\"frontier\",\"model\":\"controller\",\"wave\":5}\n```\n\n---",
"blockedBy": [
8
]
}
],
"lastUpdated": "2026-10-10T12:55:44.872528Z"
}
Reference in New Issue View Git Blame Copy Permalink