HistoryReadEvents projected only EventId / SourceName / Time / ReceiveTime /
Message / Severity: a select clause asking for EventType, ConditionName or
SourceNode (what most generic alarm viewers ask for) got nulls, and the
event filter's where-clause (an OfType) was ignored.
- HistoricalEvent gains optional EventTypeName, ConditionName and AlarmId.
SqlServer, QuestDB and Gateway event reads fill them from what their
alarm writers already store (AlarmTypeName / AlarmName / AlarmId; the
gateway event Type and its AlarmName / AlarmId properties).
- The node manager maps the type name to its Part 9 event type (unknown or
non-event => AlarmConditionType), projects ConditionName, and resolves
SourceNode to the materialised condition the alarm id names (the live
events' SourceNode), else null.
- The where-clause is evaluated per event with the SDK content filter
through an IFilterTarget (OfType via the type tree; other operators
against the projected fields), after the page's cursor is taken, so
paging is unchanged.
Tested in the node manager, over a real client session (the Client.CLI has
no `historyread --events` mode yet; filed as a follow-up), and by alarm
writer -> event read round trips on SqlServer and QuestDB.
Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The Gateway backend's gaps against SqlServer/QuestDb were documented
piecemeal. The backend-selection table in docs/Historian.md now also covers
the aggregate set (and the native time-weighted opt-in), emulated reverse
and end-only reads, the client-side events cap, retention and the health
probe.
Continuous historization on Gateway silently loses sample quality (stored
Good), comms-loss outage markers and non-numeric values. Building its live
writer, which only happens when ContinuousHistorization is enabled, now
logs one startup Warning that names the gaps and the alternatives.
The review asked for the matrix in the AdminUI backend picker, but no such
picker exists: the backend is configuration-only. Filed as a follow-up.
Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
FasterLogHistorizationOutbox.PeekBatchAsync held the state lock across the
FasterLog scan. Once a backlog's head ages out of FasterLog's in-memory
window (any multi-minute historian outage), GetNext reads pages from disk
synchronously, and the recorder's appends (which index under the same lock)
blocked on an Akka dispatcher thread once per drain pass, exactly during
outage recovery.
The peek now snapshots the head under the lock, scans unlocked, and re-takes
the lock only when it met undecodable records, to reconcile them with the
index in O(records scanned) and drop one still at the head (re-checked,
since an ack or drop-oldest may have moved it). A test seam pauses the scan
and proves an append completes meanwhile; wrapping the scan in the lock
again fails it.
Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
All three alarm-history writers inserted one event per round trip, serially
(SqlServer also opened a pooled connection per event), so a 10000-event
alarm storm drained at 10000 x RTT while the backlog sat in LocalDb.
- Sql.Common.AlarmBatchWrite: one statement per chunk; a chunk that fails
PermanentFail-class is re-sent event by event so only the poison event
dead-letters; a transient failure retries the chunk; a cancellation never
dead-letters.
- SqlServer: multi-row insert-if-absent over a VALUES set (150 events per
statement, under the 2100-parameter limit), duplicate ids in a batch
written once; a duplicate-key race isolates per event, where it is Ack.
No new table type, so no schema migration (the review suggested a TVP;
the VALUES form gets the same one-round-trip-per-chunk without one).
- QuestDb: multi-row INSERTs of 500 events; pre-1970 events still
dead-letter unsent.
- Gateway: no batch RPC exists, so sends are pipelined, at most 8 in
flight, outcomes kept in input order.
Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Every IHistorianValueWriter caught Exception and returned false, including
OperationCanceledException from the drain's own token. The recorder's guard
("a cancelled pass is not a tag failure") was therefore dead code: at shutdown
mid-pass each cancelled group could be booked as a consecutive failure and,
with a sibling that already wrote, tip a healthy tag toward quarantine.
SqlServer, QuestDb and Gateway writers now rethrow a cancellation of the
caller's token; a cancellation-shaped error on a live token is still an
ordinary failed write.
The Gateway writer relabelled a Local-kind timestamp as UTC instead of
converting it, so the same value could land at two different times on two
backends. Stamped samples now go through LiveWritePreparer (Local -> UTC,
non-finite doubles not sent, duplicate timestamps keep the first); unstamped
samples are still left for the gateway to stamp, in batch order.
Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Every Gateway HistoryRead was preceded by a BrowseTagNames catalog stream
keyed by a glob filter (a name with '*' or '?' streamed every match), and the
4096-entry presence cache evicted an arbitrary entry (Keys.First()) when full,
so a sweep over more tags thrashed it.
The presence check is now one exact unary GetTagMetadata (null = absent).
Only the deployed gateway's known misparse of a missing tag (Internal /
Unknown / InvalidDataException) falls back to the exact-name catalog check;
any other failure propagates and never becomes absence. The positive TTL is 5
minutes (was 60 s); a full cache drops expired entries first, else the one
closest to expiry. Wire tests cover the working, the misparsing and the
refusing gateway.
Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The Gateway historian's ProcessConnectionOpen / EventConnectionOpen were
derived from read outcomes only, so a freshly booted node with a healthy
gateway reported the historian closed until a client read, and a node that
only writes (continuous historization) never flipped it.
The data source now runs a ProbeAsync loop (ServerHistorian:Gateway:
ProbeInterval, default 30 s, first probe at start-up; 0 = off). Each probe,
bounded at 10 s, is recorded as the latest connectivity observation; the
latest observation (probe or read) is what the snapshot reports. Probes never
touch the query/failure counters. DisposeAsync stops the loop before
disposing the client.
Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
AVEVA's native TimeAverage / Total are answered with Good-status wrong values
while the gateway sends the interval in the wrong unit (historiangw#27): one
whole-span value on the earliest interval, the rest absent. Nothing on the
wire marks the answer as degraded.
Both are now refused BadAggregateNotSupported (unsent, not counted as a
backend failure, not advertised in AggregateFunctions) unless
ServerHistorian:Gateway:EnableNativeTimeWeighted is true. The default
Gateway set is the ten raw-evaluated aggregates
(GatewayHistorianDataSource.RawEvaluatedAggregates); ServedAggregates stays
the 12 the backend can serve with the flag on. The native-path tests opt in.
Also documents the ServerHistorian:Gateway sub-section (EventDrainBudget from
the previous commit) and the events drain in docs/Historian.md.
Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
A capped Gateway HistoryReadEvents read broke out of the gRPC stream as soon
as the page was full. Abandoning a gateway stream cancels the RPC, which the
gateway treats as a connection fault: it evicts its pooled historian session,
so the next read reconnects and concurrent reads on that session fail.
The read now keeps consuming the stream past the cap without collecting it,
bounded by ServerHistorian:Gateway:EventDrainBudget (default 10000 rows; 0 =
the old abandon-at-cap). Rows of a foreign source never count toward the
budget before the "more events" signal is seen, so an abandoned stream never
answers a page as complete. A paged-vs-unpaged fuzz (random tie series,
foreign rows, caps 1..500, budgets 0..10000) pins the page contents and the
drain; it found that abandon-before-signal case.
ReadEventsAsync and the native AtTime arm now rethrow a caller cancellation
instead of recording it as a backend failure.
Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
AbLegacyDriver stamped LastSuccessfulRead (init, read, write), the read's sample
timestamps and host-state change times from raw DateTime.UtcNow; only the backoff
path read the injectable Clock. All of them now read Clock(), so
NativePcccBridgeStatusHealthTests drives a manual clock (init at T0, read at T1)
and asserts LastSuccessfulRead == T1 exactly instead of spin-waiting for the wall
clock to pass the init stamp (#767). Without the driver change the 6 tests fail
(init stamp is wall-clock, not T0).
Directory.Build.props sets TreatWarningsAsErrors for every project since #766, so
the 75 per-csproj copies (64 src + 11 tests) are removed and the props comment
says it is now the only place. tools/mem-probe keeps its explicit opt-out.
Release build: 0 warnings, 0 errors; a planted CS0168 still fails the build in a
src project and in a test project that used to carry its own copy.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
- GatewayHistorianDataSource.ReadRawWindowAsync keeps the cap + 1 probe budget
(budgetCap) separate from the client's cap: a strict-ceiling or crowding
refusal now reports Limit = the server cap that applied and advises setting
NumValuesPerNode at or below it (it said "lower NumValuesPerNode" with
Limit = cap + 1 to a client that sent 0).
- Fuzz: the oracle-source over-cap assertion was vacuous (cap 0 never counts);
it now checks no reverse window reached the forward-only shape. Class docs
name the post-floor cap set (3 ... 1000) and distinguish NumValuesPerNode
1 / 2 from the smallest server cap.
- HistoryPaging.IsFullPage doc says it judges the REQUEST cap (page + ties +
lookahead), which is finite under any limit; AdvertiseMaxReturnDataValues
swallows any exception (returns false for the caller to log), as its doc
promised.
Refs #736
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review R08:1.1: an uncapped HistoryRead-Raw (NumValuesPerNode = 0) read a
tag's whole window into memory on every backend (SQL TOP(int.MaxValue),
QuestDB LIMIT, Gateway max_values = int.MaxValue + a second snapshot copy).
New ServerHistorian:MaxRawValuesPerRead (default 10 000; <= 0 unbounded with
a Validate() warning; 1 raised to 2). Each backend data source takes it,
reports it as IHistorianDataSource.MaxRawValuesPerRead (the Gateway: min
with a strict gateway's learned ceiling), so the node manager's existing
RawPageLimit paging serves N = 0 and N > cap as pages of at most the cap
with a continuation point. Independently, a backend raw read asking for 0
or more than the cap reads cap + 1 rows and is REFUSED
(HistoryReadLimitExceededException => BadTooManyOperations) when the window
holds more - never answered truncated, since the server's paging reads a
short answer as the last page. Bound through the Host registry, the
SqlServer/QuestDb providers and GatewayHistorian.CreateDataSource.
Tests: options (default/binding/warnings), provider binding, Gateway unit +
through a real SDK node manager (uncapped -> 999/998/998/605 + CPs), and a
conformance case for the SQL backends.
Refs #736
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Clears CommentChecker findings in src/ (3,280 -> 209, the remainder being false
positives like PLC-5/IEEE-754/M0.0) by adding missing summaries/params/returns,
normalising inheritdoc usage, and removing Gitea/review tracking IDs while keeping
the rationale they annotated. Comment-only: comment-stripped sources are identical
to the parent commit.
The #687/F4 crowding re-read doubled its room up to 8 re-reads with no cap, so a pathological single-ms
cluster on a streaming gateway was streamed several times over for one page. The room is now capped at
ServerHistorian:MaxTieClusterOverfetch (plumbed into GatewayHistorianDataSource as maxPreWindowRows,
default 65536); past it, or once a read has already seen more, the read is BadTooManyOperations with
PreWindowRows set.
Item 1: IHistorianDataSource.MaxRawValuesPerRead (Gateway: learned ceiling - 16 = L; 0 elsewhere, guard-tested).
RawPageLimit plans every raw page on top of #692's lookahead: with L > 0 the read (page P + boundary ties k
+ lookahead) is <= L, P = min(N, L - k - 1) (uncapped: L - k - 1); no room => straight to the tie-cluster
over-fetch (never an uncapped or underflowing request). One re-plan + re-read when the read taught a new
limit or was refused for crowding pre-window rows (HistoryReadLimitExceededException.PreWindowRows): the
page shrinks by them. Over-fetch bound min(MaxTieClusterOverfetch, L - 1); slice size never 0.
Review fixes: F1 a window starting at/before the epoch counts AVEVA's synthetic row as pre-window (a
ceiling-truncated read was reported complete); F2 the page path disables ReverseRawReader's own re-run
(cold end-only ceiling 20/N=3000 cost 3,023 reads, now <= 40); F3 crowded look-back re-read around, not
refused; F4 the #687 crowding re-read room at least doubles (up to 8 re-reads), so a cluster of hundreds
just before a resume point no longer fails on a default gateway; a crowding refusal that leaves no room
is BadTooManyOperations, never a tick-by-tick cursor.
Item 2: ServerHistorian:EventReadHorizon (default 00:00:00 = exact, unchanged). When set, every fresh
HistoryReadEvents window's earlier bound is raised to now - horizon (both directions, all backends).
Lossy by design; documented.
Item 3 (native TimeAverage/Total re-check) stays blocked on historiangw#27.
A time inside a gap of a bursty tag costs a 64-row jump read landing in the next
burst plus a bisection back across the gap (~6 reads), so the 64 + 4/time cap
refused a 1 h/day 1 Hz tag asked at noon on 30/100/300 consecutive days
(186/606/1,806 reads) that AVEVA's own AtTime served. The cap is a runaway guard
(HistoryReadDeadline bounds wall clock): now 64 + 16 per distinct time.
Regression tests for x30/x100/x300 (each <= 7 reads a time, values checked).
Docs: DateTime-hinted nodes are stepped but keep the gateway's Double (the
Gateway never historizes DateTime nodes; the SQL backends return DateTime).
H1: the walk-back reset its width after a full segment and discarded a full
whole-remainder read, so a dense tag that stopped reporting weeks ago gave up
after 65 reads (BadTooManyOperations). It now gallops (x4, whole-remainder read
after two empties) and then BISECTS once any stored row is known below — a full
segment's (incl. a full whole-remainder read's) last row, or an earlier window
read's last row: (mid, hi] empty => hi = mid; full => known = its last row;
complete with rows => done; one density-aimed segment after a full read until an
aimed segment comes back empty; below 2 ms the whole stretch from the known row
(never a 100 ns step). Stopped 1 h..365 d with 5 y history: 6-18 reads at
1/min, 21-25 at 1 Hz, 25-31 at 10 Hz.
M1: a jump read asks for 64 rows (1,000 times over 10 y: ~66k rows, was ~1M).
L1: per-request read cap 64 + 4 x distinct times with a clear message; a strict
ceiling below 4 rows is refused naming the ceiling.
L2: DateTime hints are stepped. L3: under an integer hint a NaN/inf or
out-of-Int64-range value is unusable, never saturated Good.
L5: conformance Int64 AtTime asserts the type. L4: docs name the backend
differences. Nit: the window read end is clamped to now + 1 day.
Reviewer harnesses (Scratch693Fuzz/Cost, not committed): 0 differential diffs
over 2,400 cases.
Review follow-up: on the Gateway every value is a Double (AVEVA stores a Boolean
as a 0/1 Int2), so the raw-evaluated AtTime interpolated a Boolean node to 0.5.
- IHistorianDataSource gains a default-interface overload
ReadAtTimeAsync(fullReference, timestamps, TypeCode? valueType, ct) that
forwards to the existing one (SqlServer/QuestDB/Null/fakes unchanged).
- OtOpcUaNodeManager.HistoryReadAtTime passes the node's scalar DataType
(AtTimeValueType: built-in type -> TypeCode; arrays/other types => null);
Raw and UNS NodeIds alike.
- GatewayAtTimeReader converts each raw row to the hinted kind BEFORE the shared
AtTimeEvaluator runs (Boolean = value != 0 => stepped; integers => Int64 =>
interpolated, rounded half away from zero, returned as the node's integer
type; floats Double), so the evaluator's own type branch decides.
- AtTimeEvaluator boxed every interpolated Int64 as a Double (`cond ? (long)x :
v` widens to double); its "keeping its type" test passed because Shouldly
ShouldBe compares numbers across types. Fixed (affects SqlServer/QuestDB too)
and pinned with ShouldBeOfType.
Gateway HistoryRead-AtTime passed AVEVA's own interpolation through: live it
answered the previous value as Good at a stored NULL's own time, null + Bad at
a Good sample right after a NULL, a drifted 99.99999999944 for a stored 100,
and no Interpolated bit. It is now served from raw gateway reads and evaluated
with Sql.Common's AtTimeEvaluator, as the SQL backends do (Part 11 §6.5.4).
GatewayAtTimeReader: one <=1,024-row forward window read [first-1ms, last+1ms]
for the batch, a walk-back (density-seeded, x4 growth, whole-remainder shortcut,
bisects dense history) for the first time's prev, and an 18-row probe for the
last time's next; truncated window reads continue or jump with a density-sized
margin. All bounds whole ms; a row counts only strictly inside its request
(AVEVA's boundary row never a sample); Bad rows value-less (#690); never past
now + 1 day; strict-gateway ceiling learned (#691.3) without re-probing.
10 dense times = 1 read (exact) / 3 (interpolated); 10 hourly = 12.
Kill switch ServerHistorian:GatewayAtTimeFromRaw (default true; false = the
old AVEVA passthrough). Known limit: the wire carries no data type, so numeric
tags interpolate as Double (Boolean/integer nodes are not stepped/rounded as on
the SQL backends).
GatewayHistorianDataSource.ReadEventsAsync sized its result list to
maxEvents, which is int.MaxValue for NumValuesPerNode = 0: every
uncapped Gateway events read (forward on master, and the new walked
reverse one) threw 'Array dimensions exceeded' and counted as a backend
failure. Capacity is bounded at 1024. The events walk delegate moved to
ReverseEventsReader so it is tested over the real Gateway source
(forward uncapped, reverse uncapped, reverse capped).
A reverse or end-only capped events read on a backend without
SupportsReverseEvents (Gateway, Null) was refused
BadHistoryOperationUnsupported (explicit reverse) or answered with the
OLDEST N ascending (end-only, #675). It is now served newest-first by
the same ReverseWalk as raw reads: page-sized ms-aligned segments,
never answered from a read the source flagged truncated, ties sorted by
EventId (the gateway orders by event time only - the fake shuffles
them per call), one extra event walked to signal 'more', uncapped = one
forward read reversed, and the width + proven-empty stretch carried in
the events cursor. The oversized-cluster over-fetch keeps exactly the
events at T in the same order. A tie cluster over the budget is
BadTooManyOperations. Docs note the gateway's EventReadMaxRows must be
unlimited (it truncates silently).
With StartTime at or before the epoch, RequestStartFor has no margin left, so
AVEVA's synthetic no-data/boundary row is stamped exactly at StartTime and was
kept: raw reads returned a fabricated Bad sample at 1970-01-01 and the per-sample
aggregates put it in bucket 0 (Start/End fabricated, Min/Max status Uncertain).
IsWindowSample and the aggregate loop now drop any row with t <= ProbeFloor
(RawAggregateReader.IsAfterFloor), matching the #686 walk-back's assumption.
- M1/M2: when the 1 ms pre-window margin (boundary row + stored samples, e.g. a
>16-sample tie cluster at start-1ms / lastTs-1ms) uses up the cap + 16 budget,
re-read ONCE asking for cap + m + 16 (m = pre-window rows seen); fail
BadTooManyOperations only if the re-read is still short because of them. The
raw budget is no longer clamped at the gateway default ceiling (AVEVA always
emits the boundary row, so page sizes 999,985-1,000,000 always came back
short); the #688 clamp stays on the per-sample aggregate path only.
- Merge trap: NumValuesPerNode 0 => int.MaxValue, never through the clamp.
- L3: the forward end is exclusive ([start, end), the IHistorianDataSource
contract); a point read (start == end, the tie-cluster over-fetch) inclusive.
- RequestStartFor never goes before the Unix epoch (#686's walk-back floor).
- L4 (docs): an Uncertain native row for an all-NULL interval is unverified live.
The default MaxTimeWeightedRawRows (1,000,000) + 18 exceeded the gateway's
default MaxValuesCeiling (1,000,000), so with AllowStreamingBeyondCeiling=false
every Gateway per-sample aggregate read was refused. The allowance is now clamped
at 1,000,000 while the cap is at or under it (RawAggregateReader.
ClampToGatewayCeiling); a cap above it keeps its allowance. Raw reads are not
clamped (rebased onto #686).
An AVEVA Full raw read leads with a row stamped at the query start (the value in
effect, Quality=133, or a no-data NULL row); it was returned as a stored sample.
Raw reads now use the #681 classification (RawAggregateReader.RequestStartFor /
IsWindowSample — one authority, by timestamp, since a stored NULL looks the same
on the wire): request from 1 ms early with NumValuesPerNode + 16 rows, keep rows
at or after StartTime, drain the whole stream. A full page still holds exactly
the cap, so #669 paging/resume/tie-cluster reads are unchanged; a margin that
swallows the allowance fails the node rather than ending paging early.
ReturnBounds is not plumbed on any backend (documented).
AVEVA answers one extra row for the interval before the window; it was mapped as
a result stamped before StartTime (N+1 values). Rows whose interval lies outside
the requested span (either direction) are now dropped after the drained stream.
An Uncertain (opc 64-191) interval kept no value (opc >= 192 was the only data
test); it now carries its value as UncertainDataSubNormal | Calculated.
Docs: AVEVA's TWA excludes NULL-gap time while the SQL backends bridge it;
end-to-end per-interval values also need historiangw#27.
1. Uncapped reverse raw read on a forward-only backend = ONE forward read of the
window, filtered and reversed in memory (was a segment-by-segment walk).
2. Segment width is aimed at ~half the row budget from the density just observed
(overflow: from the truncated read's own sample span) and carried page to page
in HistoryContinuationState.SegmentWidthTicks.
3. The walk starts no later than now + 1 day and never reads before the epoch.
4. Request bounds are rounded outward to the millisecond, so a row AVEVA
fabricates at a rounded-up query start cannot land inside the segment.
5. Budget M = max(1, MaxTieClusterOverfetch); reads ask M + 2 rows, completeness is
'short OR last row past the segment', and a read crowded by rows before the
segment is re-read once (<= 2M + 4). A cluster of exactly M, neighbouring
clusters each within M, and M <= 1 are all served; only one timestamp over M
is refused. ServerHistorianOptions text updated.
6. The raw and events tie-path oversized-cluster refusal is BadTooManyOperations
(was BadHistoryOperationUnsupported), consistent with the walk.
7. GatewayHistorianDataSource.ReadRawAsync: a caller cancellation is not a backend
failure; NumValuesPerNode = 0 is sent as int.MaxValue (the client package and
the gateway reject max_values <= 0, so every uncapped Gateway raw read failed);
a MaxValuesCeiling refusal is a HistoryReadLimitExceededException.
8. Tests: upper-edge filter, epoch floor, look-ahead, ms alignment, exact/adjacent
clusters, minimal budget, carried density, and the walk over the real Gateway
source + FakeHistorianGatewayClient (OpcUaServer internals visible to it).
#669 turned an end-only capped HistoryRead-Raw into a reverse read (start > end)
for every backend; the gateway rejects start > end (InvalidArgument), so on the
default Gateway backend every end-only capped and every explicit reverse raw read
failed the node and was counted as a backend health failure.
- IHistorianDataSource.SupportsReverseRaw (default false; SqlServer/QuestDB/Null
true, Gateway false; guard test pins that every production backend declares it).
- OtOpcUaNodeManager sends a reverse window only to a backend that declares it;
otherwise ReverseRawReader walks (lo, hi] back in bounded forward segment reads
(1 min, x4 while sparse; each <= MaxTieClusterOverfetch + 1 rows; a full read is
re-read narrower, never answered truncated; filtered on sample timestamps, which
also drops AVEVA's fabricated query-start rows; stops at the Unix epoch). A
one-timestamp tie cluster over the bound => BadTooManyOperations.
- ServeRawPagedAsync maps HistoryReadLimitExceededException to BadTooManyOperations.
- GatewayHistorianDataSource.ReadRawAsync refuses start > end unsent, without
recording a health failure.
- #682: ServedAggregates (ten raw-evaluated per-sample + TimeAverage/Total) replaces the placeholder
AggregateModeMapper.Served; NativeAggregates is derived from the mapper's switch. The Gateway
expectations in AggregateModeMapperTests, HistorianBackendRegistryTests and
AdvertisedAggregateFunctionsWireTests move to the 12-member set (the wire tests' 'not served by
Gateway' example is now TimeAverage2).
- #684: RawBucketAccumulator applies the SQL backends' Good-only value columns (Avg/Min/Max/Sum and
the ActualTime extreme over Good numeric samples), UncertainMin/UncertainMax over Uncertain numeric
samples, and GoodNonNumeric. The parity oracle carries the #684 InMemoryRows and the DenseUncertain /
UncertainBeyond / UncertainText datasets.
Average, Count, Minimum, Maximum, Range, Start, End, Delta and Minimum/MaximumActualTime are evaluated
on the Gateway backend from forward ReadRaw reads with BucketPlan + AggregateEvaluator, so Gateway
matches SqlServer/QuestDB exactly (values, status bits, interval-start stamps, #677 reverse).
- Window read [start - 1 ms, end] so AVEVA's initial-value row never lands in a bucket; a pre-window
row proves earlier data only with a non-Bad OPC quality (histsdk puts a value on every NULL row).
- Earlier data otherwise from a 2-row [1970, start) probe; later data from a 2-row [end, now+1d]
look-ahead; both ignore rows stamped at their query start/end.
- No gateway stream is ever abandoned (a client-side cancel makes the gateway evict its shared
historian session); past ServerHistorian:MaxTimeWeightedRawRows the bounded stream is drained, then
BadTooManyOperations. The gateway's max_values-ceiling InvalidArgument maps to the same limit; a
caller cancellation is not a backend fault.
- TimeAverage/Total stay AVEVA-native (no prior-sample read for exact Part 13 bounds), now stamped with
the interval start (PENDING the live leg: histsdk's tail+43 start_time field is unverified) and
carrying the Calculated bits.
- Served set: GatewayHistorianDataSource.ServedAggregates (12). Live AVEVA leg open (VPN).
IHistorianDataSource.SupportedAggregates (default: every HistoryAggregateType)
is the one authority for both ServerCapabilities/AggregateFunctions (i=2997) and
HistoryServerCapabilities/AggregateFunctions (i=11201) and for the up-front
BadAggregateNotSupported in HistoryRead-Processed. Gateway declares
AggregateModeMapper.Served (4), SqlServer/QuestDB AggregateFamilies.Served (17),
Null none. Only the folder references are trimmed; the SDK's calculator
factories stay so live AggregateFilter subscriptions keep working.
Co-Authored-By: Claude <noreply@anthropic.com>
- IHistorianDataSource.SupportsReverseEvents (default false; SqlServer/QuestDB true; Gateway/Null declare false).
The node manager checks it BEFORE calling the source: an end-only capped events read becomes reverse only on
a supporting backend (Gateway, the default, keeps the pre-#675 forward [MinValue, EndTime) paging), and an
explicit reverse read on a non-supporting backend is BadHistoryOperationUnsupported without a call.
- Removed the broad catch (NotSupportedException) in the events path: a NotSupportedException from inside a
backend is now logged as a backend fault (Error), not an 'unsupported' Warning.
- Null declares the reverse overload itself; guard test: every production IHistorianDataSource declares both
the reverse overload and the capability (positive-control scan of all four backends).
- Conformance: capability reported + oversized reverse tie cluster (forward over-fetch reversed == reverse
read order; capped reverse read stopping inside the cluster flags truncation) on SqlServer + QuestDB.
- docs/Historian.md: end-only capped events read is backward on SqlServer/QuestDB, forward on Gateway.
- BucketPlan: reverse processed reads anchored at the later time for every
aggregate: window (end, start], partial interval earliest, results stamped
with the later edge, latest first (SDK CreateSlice/GetTimestamp/backward
GetValues). SqlServer adds @bucketAdjustTicks + flips the window predicate;
QuestDB binds BucketBaseUnixTicks. IndexOf is the C# statement of the math.
- Partial bit on the interval the tag's data starts inside (SDK m_startOfData),
every aggregate but Average; per-sample reads fetch the earliest sample.
- Node manager: non-default client AggregateConfiguration =>
BadAggregateConfigurationRejected per node (ServerAggregateConfiguration,
pinned to the SDK AggregateManager defaults).
- GatewayTagProvisioner sends IntegralDivisor = 1.
- SDK parity oracle (Opc.Ua.Server calculators) in unit + both backends'
conformance suites, forward and reverse; remaining divergences documented.
- docs/Historian.md + CLAUDE.md release note (client-visible).
- IHistorianDataSource.ReadEventsAsync(..., bool reverse, ...) overload: reverse = Part 11 window (lo, hi]
newest-first by (time, id) DESC. Default forwards forward reads and throws NotSupportedException for
reverse (Null); Gateway refuses explicitly (ascending-only stream). Server answers
BadHistoryOperationUnsupported instead of silently returning the earliest N ascending.
- SqlServer / QuestDB serve reverse natively (ORDER BY time DESC, id DESC + TOP/LIMIT).
- Node manager: reverse event reads page with continuation points; direction stored in the cursor
(HistoryContinuationState.Reverse) so a resume continues in the cursor's direction; reverse tie clusters
over-fetched forward and sliced from the newest end (SliceTieCluster(reverse:)); end-only reads with a
cap page backward from EndTime (mirrors raw).
- Resumed event pages over-fetch by the boundary skip, so every non-final page is a full cap page.
- Tests: node-manager reverse paging (ties, clusters, coarse resolution, overfetch bound, resume
direction both ways, end-only, unsupported backend); shared EventReadConformance on SqlServer +
QuestDB; over-the-wire reverse paging; offline Gateway pin of the UTC DateTime.MaxValue bound through
the published client's proto conversion (in-process gRPC capture) and FILETIME.
- docs/Historian.md event paging section.
- A DateTime.MinValue start or end on DeleteRawModified is refused
BadInvalidArgument before the backend.
- Tests: anonymous without the HistoryUpdate role denied (with it allowed),
UpdateTagAttributes enabling historization sets HistoryWrite, a throwing
updater is BadInternalError, the MinValue bounds case; corrected the
test helper's comment on how the SDK base seeds unadmitted slots.
- GatewayHistorianValueWriter remarks describe the recorder's real status
capture (and that it does not persist on the gateway).
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
- I-1: an Empty sample is always Bad. The writer stores a non-Bad NaN/Inf as
Empty+BadOutOfRange and any other non-Bad Empty as BadNoData; the updater
refuses a non-Bad Empty with BadInvalidArgument; reads clamp a legacy
non-Bad Empty row to BadNoData (Raw and AtTime).
- Tag/source names over 400 chars are refused, never truncated: writer
per-tag failure, updater BadInvalidArgument, reads not-found, events none.
- HistorianInfoBits masks the historian bits before setting one.
- AtTime past now, stepped from the last sample => UncertainDataSubNormal.
- Bootstrapper: logger, 5 s negative cache, ClearPool + Warning when
sp_releaseapplock fails.
- Provider builds its shared state lazily (an invalid Schema no longer
throws at Resolve); Kinds is a FrozenSet; Gateway reuses the recorder's
DefaultSupportedKinds.
- Cleanup: SqlHistorianDataTypes in its own file (updater dup removed),
dead Invalidate and unused status consts deleted, named severity masks,
alarm writer brace.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Joseph Doherty