fix(historian): QuestDB modified reads keep a landed change whose marker was lost; H2 review NITs (lmxopcua#778)
QuestDB modified-history reads dropped every still-pending trail row that was not the latest at its source time, so V0->V1 (applied marker never written) followed by V1->V2 hid the V0->V1 change for good. A pending row below the latest is now served when the NEXT trail row at that time recorded a different prior value (kind, value, status or the prior's server_ts) - the next change saw what this one wrote; an orphan of a change that never landed leaves the same prior for the next row and stays hidden. QuestDB window functions reject VARCHAR, so the next row's prior comes from a LEFT JOIN on (ts, lead(id)). The updater's comment and marker-failure log no longer claim modified reads "still serve it" unconditionally. Regression tests (QuestDB 10.0.1 throwaway container): marker lost then a replace / delete at the same time; an orphan followed by a marker-lost retry then a later change (orphan still dropped); the random marker-mix fuzz now varies the recorded priors. All four fail on the old filter. NITs: - SqlServerHistorianValueWriter: a SqlException raised while the token is cancelled (SqlClient's "Operation cancelled by user") is rethrown as OperationCanceledException instead of booked as a failed write. Integration test holds a TABLOCKX on Sample and cancels mid-INSERT; it returned false before. - SqlServerHistorianDataUpdater.DeleteRawBatches is an Interlocked counter. - HistoryReadEvents: an event filter whose SDK validation throws (unknown operator) failed with the generic BadUnexpectedError; the node manager now re-throws any failed event-filter validation as BadEventFilterInvalid. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
+11
-6
@@ -599,7 +599,10 @@ Every backend stores the three (SqlServer `AlarmTypeName` / `AlarmName` / `Alarm
|
||||
honoured:** each event is evaluated against it with the SDK's content filter — `OfType` through the server's type tree (a
|
||||
supertype keeps its subtypes), any other operator against the projected fields. Filtering happens after a page's
|
||||
continuation point is taken from the unfiltered page, so paging is unchanged: a filtered page may hold fewer events (even
|
||||
none) and still carry a continuation point. An unsupported select operand returns a null field (spec-conformant).
|
||||
none) and still carry a continuation point. An unsupported select operand returns a null field (spec-conformant). An
|
||||
event filter the SDK's validation rejects (an operand-count mismatch, a dangling or self-referencing element operand, an
|
||||
unknown operator, …) fails the read with `BadEventFilterInvalid` before any read (#778 — an unknown operator used to fail it with the
|
||||
generic `BadUnexpectedError`).
|
||||
|
||||
### No driver-side history (Gitea #655)
|
||||
|
||||
@@ -1128,10 +1131,12 @@ modification time and the Part 11 `HistoryUpdateType`. A HistoryReadRaw with `Is
|
||||
the change: a crash (or failed sample INSERT) in between leaves an **orphan trail row** describing a change that never landed — never a
|
||||
change without its trail row. A failed sample write after its trail rows landed logs a **Warning** naming the tag, the row count and the
|
||||
time range. **Applied marker:** the trail row is written `pending = true` and, once the change has landed, re-upserted `pending = false` on
|
||||
the table's `DEDUP UPSERT KEYS(ts, tag, modified_id)`. A modified read serves a still-`pending` row only when it is the **latest** trail row at
|
||||
its time and the sample there (a tombstone included) was written at or after the modification — a crash between the change and its marker.
|
||||
Otherwise it is the orphan of a change that never landed and is **not served** (it used to be served as a change that happened). Residual:
|
||||
a double fault — a change that landed without its marker, followed by a later change of the same sample — hides the earlier one.
|
||||
the table's `DEDUP UPSERT KEYS(ts, tag, modified_id)`. A modified read serves a still-`pending` row only when the evidence says its change
|
||||
landed (a crash or a failed marker write between the change and its marker): as the **latest** trail row at its time, the sample there (a
|
||||
tombstone included) was written at or after the modification; below a later trail row at that time, that **next row recorded a different
|
||||
prior value** (kind, value, status or the prior's write time — the next change saw what this one wrote). Otherwise it is the orphan of a
|
||||
change that never landed and is **not served** (it used to be served as a change that happened). #778: the second rule is new — a change
|
||||
that landed without its marker used to vanish from modified history as soon as a later change at the same time arrived.
|
||||
- **Rolling upgrade (SqlServer).** The v2 schema is applied by the first node that upgrades. Until its peer is upgraded too, that
|
||||
not-yet-upgraded node still runs the old build, whose HistoryUpdates write **no trail rows** — silent gaps in the audit trail. Upgrade both
|
||||
nodes of a pair together (stop, upgrade, start both); modifications made by an old-build node in the window are never recorded.
|
||||
@@ -2085,7 +2090,7 @@ present **and not deleted**.
|
||||
current value (read by the same existence query), then writes the change. No multi-table transaction exists, so this order is the guarantee:
|
||||
a crash between the two leaves an **orphan trail row**, never an untracked change — the documented deviation from `SqlServer`'s atomic trail.
|
||||
The row is written `pending` and marked applied after the change, so a modified read does not serve the orphan of a change that never
|
||||
landed. See [Modified-value history](#modified-value-history-part-11).
|
||||
landed — while a change that landed but lost its marker stays served (#778). See [Modified-value history](#modified-value-history-part-11).
|
||||
|
||||
The per-item codes come from the shared `HistoryUpdatePlanner`, so they equal `SqlServer`'s (including the duplicate-timestamp-in-one-request rules
|
||||
above). **Every read and aggregate filters `NOT deleted`.** A client Insert at a deleted timestamp re-populates it (the OPC UA intent); only the
|
||||
|
||||
Reference in New Issue
Block a user