fix(historian): QuestDB modified reads keep a landed change whose marker was lost; H2 review NITs (lmxopcua#778)

QuestDB modified-history reads dropped every still-pending trail row that was
not the latest at its source time, so V0->V1 (applied marker never written)
followed by V1->V2 hid the V0->V1 change for good. A pending row below the
latest is now served when the NEXT trail row at that time recorded a different
prior value (kind, value, status or the prior's server_ts) - the next change saw
what this one wrote; an orphan of a change that never landed leaves the same
prior for the next row and stays hidden. QuestDB window functions reject
VARCHAR, so the next row's prior comes from a LEFT JOIN on (ts, lead(id)).
The updater's comment and marker-failure log no longer claim modified reads
"still serve it" unconditionally.

Regression tests (QuestDB 10.0.1 throwaway container): marker lost then a
replace / delete at the same time; an orphan followed by a marker-lost retry
then a later change (orphan still dropped); the random marker-mix fuzz now
varies the recorded priors. All four fail on the old filter.

NITs:
- SqlServerHistorianValueWriter: a SqlException raised while the token is
  cancelled (SqlClient's "Operation cancelled by user") is rethrown as
  OperationCanceledException instead of booked as a failed write. Integration
  test holds a TABLOCKX on Sample and cancels mid-INSERT; it returned false before.
- SqlServerHistorianDataUpdater.DeleteRawBatches is an Interlocked counter.
- HistoryReadEvents: an event filter whose SDK validation throws (unknown
  operator) failed with the generic BadUnexpectedError; the node manager now
  re-throws any failed event-filter validation as BadEventFilterInvalid.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 16:04:44 -05:00
parent c802f0e7ed
commit e790938f94
9 changed files with 245 additions and 37 deletions
@@ -87,6 +87,39 @@ public sealed class NodeManagerHistoryReadEventsProjectionTests : IDisposable
await host.DisposeAsync();
}
[Theory]
[InlineData("unknown operator")]
[InlineData("self-referencing element")]
[InlineData("operand-count mismatch")]
public async Task An_invalid_where_clause_is_BadEventFilterInvalid(string fault)
{
// lmxopcua#778: an unknown operator made the SDK's pre-validation itself throw, failing the read with the generic
// BadUnexpectedError. The structural faults were already BadEventFilterInvalid and must stay so.
var (host, server, nm, notifier) = await BootAsync();
var where = new ContentFilter();
switch (fault)
{
case "unknown operator":
where.Push((FilterOperator)99, new LiteralOperand(new Variant(1)), new LiteralOperand(new Variant(1)));
break;
case "self-referencing element":
where.Push(FilterOperator.Not, new ElementOperand(0));
break;
default:
where.Push(FilterOperator.Equals, new LiteralOperand(new Variant(1)));
break;
}
var ex = Should.Throw<ServiceResultException>(() => ReadStatus(server, nm, notifier, Select("EventId"), where));
ex.StatusCode.ShouldBe(StatusCodes.BadEventFilterInvalid);
// Positive control: the same read with a valid where-clause is served.
var valid = new ContentFilter();
valid.Push(FilterOperator.Equals, new LiteralOperand(new Variant(1)), new LiteralOperand(new Variant(1)));
StatusCode.IsGood(ReadStatus(server, nm, notifier, Select("EventId"), valid).Status).ShouldBeTrue();
await host.DisposeAsync();
}
private static string[] Ids(HistoryEvent h) =>
h.Events.Select(e => System.Text.Encoding.UTF8.GetString((byte[])e.EventFields[0].Value)).ToArray();
@@ -107,6 +140,14 @@ public sealed class NodeManagerHistoryReadEventsProjectionTests : IDisposable
private static HistoryEvent Read(OtOpcUaSdkServer server, OtOpcUaNodeManager nm, NodeId notifier,
SimpleAttributeOperandCollection select, ContentFilter? where)
{
var (status, result) = ReadStatus(server, nm, notifier, select, where);
StatusCode.IsGood(status).ShouldBeTrue($"HistoryRead failed: {status}");
return (HistoryEvent)ExtensionObject.ToEncodeable(result.HistoryData);
}
private static (StatusCode Status, SdkHistoryReadResult Result) ReadStatus(OtOpcUaSdkServer server, OtOpcUaNodeManager nm,
NodeId notifier, SimpleAttributeOperandCollection select, ContentFilter? where)
{
var details = new ReadEventDetails
{
@@ -120,8 +161,7 @@ public sealed class NodeManagerHistoryReadEventsProjectionTests : IDisposable
var errors = new List<ServiceResult> { null! };
nm.HistoryRead(context, details, TimestampsToReturn.Both, releaseContinuationPoints: false,
new List<HistoryReadValueId> { new() { NodeId = notifier } }, results, errors);
StatusCode.IsGood(errors[0].StatusCode).ShouldBeTrue($"HistoryRead failed: {errors[0].StatusCode}");
return (HistoryEvent)ExtensionObject.ToEncodeable(results[0].HistoryData);
return (errors[0].StatusCode, results[0]);
}
private async Task<(OpcUaApplicationHost Host, OtOpcUaSdkServer Server, OtOpcUaNodeManager Nm, NodeId Notifier)> BootAsync()