perf(historian): one QuestDB WAL wait per drain pass, with an exponential poll (lmxopcua#746)

The QuestDB live writer waited for WAL visibility after every tag group, and
the wait polled wal_tables() every 10 ms. With DrainConcurrency = 8 that was
up to eight pollers at ~100 queries/s each against a database whose WAL
apply was already behind.

New Runtime seam IHistorianPassWriter (WriteLiveValuesInPassAsync +
FlushAsync). The recorder writes every group of a pass through it and calls
FlushAsync once after all groups report, before the ack, whenever any group
wrote (an un-acked pass is redelivered and its insert-if-absent check must
see this pass's rows). A flush failure does not fail the pass. The QuestDB
writer implements it: in-pass writes only mark a flush as owed; FlushAsync
waits once. The poll now backs off 10 -> 20 -> ... -> 200 ms, never past the
timeout. Direct WriteLiveValuesAsync callers still wait per call.

Also updates the QuestDB cancelled-write integration test to the rethrow
contract from the previous commit.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-10-22 16:48:40 -04:00
parent 7c5cc9c7e5
commit ee314d1b9e
9 changed files with 395 additions and 13 deletions
+9 -2
View File
@@ -2008,8 +2008,15 @@ Because `DEDUP UPSERT` would otherwise let a re-sent sample overwrite the stored
QuestDB applies WAL commits asynchronously, so a row just inserted may not be visible to the next query (measured: up to 7 of 20 inserts invisible
to an immediate re-read). **Every write path waits** — the live writer, the HistoryUpdate updater (including between DeleteRaw pages), the tag
catalog/provisioner and the alarm writer (`QuestDbWalVisibility`): after the INSERT(s) it reads `sequencerTxn` from `wal_tables()` and polls
`writerTxn` about every 10 ms until it catches up, the table is reported `suspended` (Warning, stop), or `VisibilityTimeout` elapses (Warning).
A timeout never changes a result code and never discards data; the rows simply become visible shortly after. Measured: median 0 ms, p95 6 ms.
`writerTxn` — first after 10 ms, then backing off exponentially (20, 40, 80, 160, then every 200 ms) — until it catches up, the table is
reported `suspended` (Warning, stop), or `VisibilityTimeout` elapses (Warning). A timeout never changes a result code and never discards data;
the rows simply become visible shortly after. Measured: median 0 ms, p95 6 ms.
**The live writer waits once per drain pass, not once per tag group.** The recorder writes every group of a pass through the writer's
`IHistorianPassWriter` path and then calls `FlushAsync` once — after every group has reported, before the ack, whenever any group wrote —
so the pass's rows are visible to the next pass's insert-if-absent check. With `DrainConcurrency = 8` the per-group wait used to run up to
eight pollers at ~100 `wal_tables()` queries/s each against a database whose WAL apply was already behind. A flush failure does not fail
the pass (the rows are committed). Direct callers of `WriteLiveValuesAsync` still wait per call.
### HistoryUpdate (soft delete)