Review 07 P7 (rest): adjacent words on one data file were never coalesced
(one round trip per tag) although the array path proves the transport can read
a span. Within a device pass, scalar tags on one N/B/F/L file within 8
elements of each other (block <= 200 data bytes, under the 236-byte typed-read
limit) now ride one multi-element read and are decoded from their offsets with
exactly the CLR types the single-tag path returns (int / float / bool).
Only shapes the single path decodes identically are coalesced: indirect,
sub-element, array, file-number-less (S:/I:/O:) and width-mismatched tags keep
the single path. A block the device refuses (an authored element past the end
of the file, a native PLC-5 non-N file) falls back to per-tag reads for its
members and its shape is remembered; a comms fault fails the members together.
New driver-config key coalesceReads (default true) with an AdminUI switch
(Razor change: AbLegacyDriverForm, for the Wave 4 live check). Adds an
env-gated ab_server parity case (coalesced == per-tag values).
Refs #738
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 07 P7 (High): reads walked every reference of every device serially
with no backoff, so a dead device cost Timeout per tag per poll (50 tags x 2 s
= 100 s) and starved every healthy device long enough to trip the comms-loss
watchdog against a reachable one.
- ReadAsync buckets references by device and reads up to 8 devices
concurrently; each device's tags are still read in order.
- Each device has a ConnectionBackoff (1 s doubling to 30 s). Two consecutive
comms faults mark it down: the rest of that pass and every later pass fail
fast with BadCommunicationError and touch no wire until the window expires or
any answer (data path or probe) resets it. One timing-out tag does not
blacken its answering siblings.
- Cold connects are single-flight per device (ConnectGate), replacing the
per-key creation locks; a cached handle never touches the gate.
- A status/exception proving the device ANSWERED with a deterministic refusal
(not found, not allowed, out of bounds, type/size mismatch) is per-tag Bad,
keeps its handle, warns once per tag and advances LastSuccessfulRead instead
of evicting and degrading. Writes classify the same way.
- Health is written once per read: Degraded if any device faulted, Healthy if
anything read Good, LastSuccessfulRead advanced when any device answered.
Adds an env-gated ab_server case with a blackholed sibling device.
Refs #738
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 07 U10 (user decision: implement array encode now). Both runtimes
encoded a scalar for an array tag: an int[] threw InvalidCastException into
BadTypeMismatch + Degraded, a scalar wrote element 0 natively, and libplctag's
SetInt32(0, ..) re-sent the stale last-read buffer for elements 1..n-1.
IAbLegacyTagRuntime gains EncodeArray; both runtimes write every element at
the element width (Int/AnalogInt 2 bytes, Long/Float 4). The driver converts
the written value before any runtime exists: each element is converted, not
exact-matched (ScadaBridge writes int[] to Int16 nodes), a length other than
the tag's element count is BadOutOfRange, an overflowing element is
BadOutOfRange, a non-array or non-numeric payload is BadTypeMismatch, and a Bit
array (bit-packed words; an encode would clobber neighbour bits) is
BadNotSupported. None of these touch health or the wire. Adds an env-gated
ab_server round trip for an N-file array write.
Refs #738
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 07 H16: an array of String/Timer/Counter/Control passed the mapper, so
every poll created a handle (one init read), the decode threw
NotSupportedException, the generic catch evicted the handle and latched
Degraded. FromTagConfig now rejects that combination (the skip carries the
reason, Inspect/deploy gate and the AdminUI model state it), and a runtime
NotSupportedException on read or write is BadNotSupported: handle kept, no
health write, one Warning per tag and operation. Write refusals (native PLC-5
and status-bit writes, libplctag translated addresses) now warn once per tag too.
Refs #738
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 07 L7: each index value an indirect tag (N7:[N7:0]) swept through left a
per-runtime and a creation SemaphoreSlim in the device's gate tables until
shutdown. A moved index now drops the stale resolution's gates along with its
handle, and sweeps any other stale resolution of the same tag a concurrent
reader cached, so the tables stay bounded by the authored set.
Refs #738
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 07 H13: _devices and _tagsByRawPath were plain dictionaries cleared and
refilled in place while the retained poll loop, the actor's GetHostStatuses and
server reads enumerated them. Both are now built locally and published by
reference under a lifecycle lock; teardown swaps in empty tables.
Lane A contract: ShutdownAsync and Dispose each retire a disjoint set of device
states, so Dispose running concurrently with a still-joining ShutdownAsync is
safe. A lifecycle epoch makes an InitializeAsync that the runtime abandoned
(overlapping the next ShutdownAsync) refuse to publish its tables, start probe
loops or write health. A runtime cached on an already-retired device state by an
in-flight read is disposed instead of leaked, and late reads/writes no longer
overwrite shutdown's health. Per-key semaphores are no longer disposed under
callers that may still hold them.
Refs #738
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review fix round 2 for lane F:
- N1: the init platform seed no longer runs inline under the init token. Galaxy's Tier-A
Initialize budget is 5 s, and a cold gateway DiscoverHierarchy can spend all of it, so the
OperationCanceledException rethrow failed init, opened the Initialize breaker and cost a full
rebuild per retry. InitializeAsync now only resolves the platform source (no network) and
queues a background seed tied to a per-runtime CTS that the teardown cancels. A result that
lands after its runtime was torn down is discarded.
- N2: the seed worker serves the runtime that is live when each pass STARTS, so a request made
under a new runtime while a stale pass is in flight always gets a pass of its own.
- N3: SubscribeAsync and the subscribe-once read capture the runtime generation before
SubscribeBulk and register only if it is unchanged, atomically with the teardown's registry
Clear (_registryGate). On a mismatch they release the handles and fail the call. Re-subscribe
Rebind uses the same gate, and the production rebuild clears the registry unconditionally.
- PerPlatformProbeWatcher re-checks its stop after the SubscribeBulk await, atomically with
the record and publish. A pass that straddles StopAsync releases the handles it acquired
instead of writing the probe entry back into a cleared registry.
Refs #735
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The writer concurrency fake parks on its release signal (the 300 ms delay could release the
gate early on a starved pool), the supervisor test's cross-thread hold flag is Volatile, and
the EffectiveTimeout doc now states the 1 s floor the code and tests implement.
Refs #735
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review fix round 1 for lane F:
- The ScanState probe watcher is seeded at InitializeAsync from a platform-only
DiscoverHierarchy query (IGalaxyPlatformSource / GatewayGalaxyPlatformSource over the owned
repository client) and re-seeded on every deploy-watcher rediscovery. The runtime never
calls DiscoverAsync on a live driver, so per-platform host status was unreachable in
production. A failed fetch logs a Warning and never fails init; re-seeds coalesce so a
deploy during an in-flight pass still gets a pass that starts after it.
- EnsureEventPumpStarted refuses while the runtime is stopping, BEFORE reusing the live pump
(that pump belongs to the dying subscriber), and the production rebuild disposes any stale
pump before wiring the new subscriber.
- TeardownRuntimeAsync closes the MX session under the caller's token linked with a 5 s
SessionCloseDeadline, so a blackholed gateway cannot hold DriverInstanceActor.PostStop for 30 s.
- A runtime generation counter, bumped by every teardown, is re-checked before each Rebind
in ResubscribeAllAsync, so a replay-gap re-snapshot that straddles a teardown cannot
resurrect handles into the cleared registry.
Refs #735
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The SDK's DisposeAsync closes with no token, i.e. under the 30 s DefaultCallTimeout, so a
blackholed gateway parked the driver's teardown for the whole of it. CloseAsync(ct) runs the
close under the caller's token, drops the channel on a cancelled/failed close (the gateway
reaps the session on its idle lease) and leaves the session re-openable. A close followed by
the final dispose closes the gateway session once.
Refs #735
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- GatewayHistorianDataSource.ReadRawWindowAsync keeps the cap + 1 probe budget
(budgetCap) separate from the client's cap: a strict-ceiling or crowding
refusal now reports Limit = the server cap that applied and advises setting
NumValuesPerNode at or below it (it said "lower NumValuesPerNode" with
Limit = cap + 1 to a client that sent 0).
- Fuzz: the oracle-source over-cap assertion was vacuous (cap 0 never counts);
it now checks no reverse window reached the forward-only shape. Class docs
name the post-floor cap set (3 ... 1000) and distinguish NumValuesPerNode
1 / 2 from the smallest server cap.
- HistoryPaging.IsFullPage doc says it judges the REQUEST cap (page + ties +
lookahead), which is finite under any limit; AdvertiseMaxReturnDataValues
swallows any exception (returns false for the caller to log), as its doc
promised.
Refs #736
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 2026-10-21 G-5, G-6, G-8 ("wire them all").
- G-5: GatewayGalaxyHierarchySource now pages the DiscoverHierarchy RPC
itself (DiscoverHierarchyRawAsync) at Repository.DiscoverPageSize. The
mxaccessgw client's typed DiscoverHierarchyAsync overloads hard-code a
5000-row page and DiscoverHierarchyOptions carries no page size, so the
authored knob reached nothing; the raw request takes page_size on the
wire, so it is honoured here (non-positive → 5000). The loop keeps the
client's repeated-page-token guard.
- G-6: GalaxyDriverProbe reads ProbeTimeoutSeconds from the config and
runs under min(caller budget, configured cap) — the knob can shorten a
probe, never extend it past the host's budget. The TCP preflight now
shares that deadline instead of hanging on the caller's token alone.
- G-8: GatewayGalaxyDataWriter runs a batch as one lane per distinct tag,
concurrently and bounded at 8 (MaxConcurrentWriteLanes); same-tag
writes keep their authored order inside one lane and results land at
their request's index. Acks stay serial by design.
- GalaxyDriverForm: help text now says what the two knobs actually do.
Refs #735
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 2026-10-21 G-4, G-7, G-11, G-10.
- G-4: PerPlatformProbeWatcher publishes its probe handles into the
driver's SubscriptionRegistry under the reserved ProbeSubscriptionId
(-1), so the EventPump's fan-out finally reaches the watcher — every
ScanState event used to be dropped as "stale after unsubscribe" and
per-platform host status never updated. ResubscribeAllAsync delegates
the reserved entry to the watcher's new ResubscribeAsync, so the probe
survives a reconnect; StopAsync removes the entry. Probe traffic is
routed by the reserved handle and never reaches the public OnDataChange.
The watcher is also built on the seam path (subscriber only), which is
what makes the wiring testable without a gateway.
- G-7: GalaxyDiscoverer.DiscoverAsync returns the hierarchy it fetched and
the driver selects its $WinPlatform/$AppEngine probe targets from that
same sweep instead of asking the gateway for the whole galaxy twice.
- G-11: OnPumpDataChange stamps LastSuccessfulRead from inbound stream
data, throttled (1 s default), so the comms-loss watchdog has honest
liveness evidence under a Degraded supervisor.
- G-10: an EventPump dispatch-loop exit that is not a cancellation now
reports a stream fault so the supervisor recreates the pump, instead of
leaving a deaf pump reporting Healthy.
Refs #735
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
GalaxyDriver.ShutdownAsync was a no-op: the MX session, both owned gRPC
channels, the EventPump, the alarm feed, the DeployWatcher, the probe
watcher and the reconnect supervisor were released only by DisposeAsync,
which the host never calls — one leaked gateway session per respawn,
removal and /raw browse, and a watchdog Shutdown→Initialize that neither
tore down nor rebuilt anything (review 2026-10-21 G-1, G-2, G-9, G-3).
- ShutdownAsync and DisposeAsync share one ordered TeardownRuntimeAsync
(supervisor first, then deploy/probe watchers, pump, alarm feed,
session, channels); owned seams are nulled so the next InitializeAsync
rebuilds; injected seams survive a shutdown and drop only on the final
dispose. Lifecycle calls are serialised behind a gate.
- A failed production connect releases its partial runtime (no orphaned
client per watchdog retry) and lands Faulted before and after.
- StartDeployWatcherAsync stops the previous watcher before starting a
replacement, and now starts on a production instance's FIRST
InitializeAsync (the old guard returned whenever discovery had not run
here, which on a runtime instance is always).
- ReconnectSupervisor releases each cycle's CancellationTokenSource when
its loop exits and gains an awaitable StopAsync; PerPlatformProbeWatcher
gains an awaitable StopAsync.
- EventPump treats any exception under cancellation as a clean stop, so a
teardown never wakes the supervisor.
- Internal seams for tests: an IGalaxyDeployWatchSource ctor parameter and
a session-open override forwarded to the owned GalaxyMxSession.
Refs #735
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review R08:1.1: an uncapped HistoryRead-Raw (NumValuesPerNode = 0) read a
tag's whole window into memory on every backend (SQL TOP(int.MaxValue),
QuestDB LIMIT, Gateway max_values = int.MaxValue + a second snapshot copy).
New ServerHistorian:MaxRawValuesPerRead (default 10 000; <= 0 unbounded with
a Validate() warning; 1 raised to 2). Each backend data source takes it,
reports it as IHistorianDataSource.MaxRawValuesPerRead (the Gateway: min
with a strict gateway's learned ceiling), so the node manager's existing
RawPageLimit paging serves N = 0 and N > cap as pages of at most the cap
with a continuation point. Independently, a backend raw read asking for 0
or more than the cap reads cap + 1 rows and is REFUSED
(HistoryReadLimitExceededException => BadTooManyOperations) when the window
holds more - never answered truncated, since the server's paging reads a
short answer as the last page. Bound through the Host registry, the
SqlServer/QuestDb providers and GatewayHistorian.CreateDataSource.
Tests: options (default/binding/warnings), provider binding, Gateway unit +
through a real SDK node manager (uncapped -> 999/998/998/605 + CPs), and a
conformance case for the SQL backends.
Refs #736
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Clears CommentChecker findings in src/ (3,280 -> 209, the remainder being false
positives like PLC-5/IEEE-754/M0.0) by adding missing summaries/params/returns,
normalising inheritdoc usage, and removing Gitea/review tracking IDs while keeping
the rationale they annotated. Comment-only: comment-stripped sources are identical
to the parent commit.
The #687/F4 crowding re-read doubled its room up to 8 re-reads with no cap, so a pathological single-ms
cluster on a streaming gateway was streamed several times over for one page. The room is now capped at
ServerHistorian:MaxTieClusterOverfetch (plumbed into GatewayHistorianDataSource as maxPreWindowRows,
default 65536); past it, or once a read has already seen more, the read is BadTooManyOperations with
PreWindowRows set.
Item 1: IHistorianDataSource.MaxRawValuesPerRead (Gateway: learned ceiling - 16 = L; 0 elsewhere, guard-tested).
RawPageLimit plans every raw page on top of #692's lookahead: with L > 0 the read (page P + boundary ties k
+ lookahead) is <= L, P = min(N, L - k - 1) (uncapped: L - k - 1); no room => straight to the tie-cluster
over-fetch (never an uncapped or underflowing request). One re-plan + re-read when the read taught a new
limit or was refused for crowding pre-window rows (HistoryReadLimitExceededException.PreWindowRows): the
page shrinks by them. Over-fetch bound min(MaxTieClusterOverfetch, L - 1); slice size never 0.
Review fixes: F1 a window starting at/before the epoch counts AVEVA's synthetic row as pre-window (a
ceiling-truncated read was reported complete); F2 the page path disables ReverseRawReader's own re-run
(cold end-only ceiling 20/N=3000 cost 3,023 reads, now <= 40); F3 crowded look-back re-read around, not
refused; F4 the #687 crowding re-read room at least doubles (up to 8 re-reads), so a cluster of hundreds
just before a resume point no longer fails on a default gateway; a crowding refusal that leaves no room
is BadTooManyOperations, never a tick-by-tick cursor.
Item 2: ServerHistorian:EventReadHorizon (default 00:00:00 = exact, unchanged). When set, every fresh
HistoryReadEvents window's earlier bound is raised to now - horizon (both directions, all backends).
Lossy by design; documented.
Item 3 (native TimeAverage/Total re-check) stays blocked on historiangw#27.
A time inside a gap of a bursty tag costs a 64-row jump read landing in the next
burst plus a bisection back across the gap (~6 reads), so the 64 + 4/time cap
refused a 1 h/day 1 Hz tag asked at noon on 30/100/300 consecutive days
(186/606/1,806 reads) that AVEVA's own AtTime served. The cap is a runaway guard
(HistoryReadDeadline bounds wall clock): now 64 + 16 per distinct time.
Regression tests for x30/x100/x300 (each <= 7 reads a time, values checked).
Docs: DateTime-hinted nodes are stepped but keep the gateway's Double (the
Gateway never historizes DateTime nodes; the SQL backends return DateTime).
H1: the walk-back reset its width after a full segment and discarded a full
whole-remainder read, so a dense tag that stopped reporting weeks ago gave up
after 65 reads (BadTooManyOperations). It now gallops (x4, whole-remainder read
after two empties) and then BISECTS once any stored row is known below — a full
segment's (incl. a full whole-remainder read's) last row, or an earlier window
read's last row: (mid, hi] empty => hi = mid; full => known = its last row;
complete with rows => done; one density-aimed segment after a full read until an
aimed segment comes back empty; below 2 ms the whole stretch from the known row
(never a 100 ns step). Stopped 1 h..365 d with 5 y history: 6-18 reads at
1/min, 21-25 at 1 Hz, 25-31 at 10 Hz.
M1: a jump read asks for 64 rows (1,000 times over 10 y: ~66k rows, was ~1M).
L1: per-request read cap 64 + 4 x distinct times with a clear message; a strict
ceiling below 4 rows is refused naming the ceiling.
L2: DateTime hints are stepped. L3: under an integer hint a NaN/inf or
out-of-Int64-range value is unusable, never saturated Good.
L5: conformance Int64 AtTime asserts the type. L4: docs name the backend
differences. Nit: the window read end is clamped to now + 1 day.
Reviewer harnesses (Scratch693Fuzz/Cost, not committed): 0 differential diffs
over 2,400 cases.
Review follow-up: on the Gateway every value is a Double (AVEVA stores a Boolean
as a 0/1 Int2), so the raw-evaluated AtTime interpolated a Boolean node to 0.5.
- IHistorianDataSource gains a default-interface overload
ReadAtTimeAsync(fullReference, timestamps, TypeCode? valueType, ct) that
forwards to the existing one (SqlServer/QuestDB/Null/fakes unchanged).
- OtOpcUaNodeManager.HistoryReadAtTime passes the node's scalar DataType
(AtTimeValueType: built-in type -> TypeCode; arrays/other types => null);
Raw and UNS NodeIds alike.
- GatewayAtTimeReader converts each raw row to the hinted kind BEFORE the shared
AtTimeEvaluator runs (Boolean = value != 0 => stepped; integers => Int64 =>
interpolated, rounded half away from zero, returned as the node's integer
type; floats Double), so the evaluator's own type branch decides.
- AtTimeEvaluator boxed every interpolated Int64 as a Double (`cond ? (long)x :
v` widens to double); its "keeping its type" test passed because Shouldly
ShouldBe compares numbers across types. Fixed (affects SqlServer/QuestDB too)
and pinned with ShouldBeOfType.
Gateway HistoryRead-AtTime passed AVEVA's own interpolation through: live it
answered the previous value as Good at a stored NULL's own time, null + Bad at
a Good sample right after a NULL, a drifted 99.99999999944 for a stored 100,
and no Interpolated bit. It is now served from raw gateway reads and evaluated
with Sql.Common's AtTimeEvaluator, as the SQL backends do (Part 11 §6.5.4).
GatewayAtTimeReader: one <=1,024-row forward window read [first-1ms, last+1ms]
for the batch, a walk-back (density-seeded, x4 growth, whole-remainder shortcut,
bisects dense history) for the first time's prev, and an 18-row probe for the
last time's next; truncated window reads continue or jump with a density-sized
margin. All bounds whole ms; a row counts only strictly inside its request
(AVEVA's boundary row never a sample); Bad rows value-less (#690); never past
now + 1 day; strict-gateway ceiling learned (#691.3) without re-probing.
10 dense times = 1 read (exact) / 3 (interpolated); 10 hourly = 12.
Kill switch ServerHistorian:GatewayAtTimeFromRaw (default true; false = the
old AVEVA passthrough). Known limit: the wire carries no data type, so numeric
tags interpolate as Double (Boolean/integer nodes are not stepped/rounded as on
the SQL backends).
GatewayHistorianDataSource.ReadEventsAsync sized its result list to
maxEvents, which is int.MaxValue for NumValuesPerNode = 0: every
uncapped Gateway events read (forward on master, and the new walked
reverse one) threw 'Array dimensions exceeded' and counted as a backend
failure. Capacity is bounded at 1024. The events walk delegate moved to
ReverseEventsReader so it is tested over the real Gateway source
(forward uncapped, reverse uncapped, reverse capped).
A reverse or end-only capped events read on a backend without
SupportsReverseEvents (Gateway, Null) was refused
BadHistoryOperationUnsupported (explicit reverse) or answered with the
OLDEST N ascending (end-only, #675). It is now served newest-first by
the same ReverseWalk as raw reads: page-sized ms-aligned segments,
never answered from a read the source flagged truncated, ties sorted by
EventId (the gateway orders by event time only - the fake shuffles
them per call), one extra event walked to signal 'more', uncapped = one
forward read reversed, and the width + proven-empty stretch carried in
the events cursor. The oversized-cluster over-fetch keeps exactly the
events at T in the same order. A tie cluster over the budget is
BadTooManyOperations. Docs note the gateway's EventReadMaxRows must be
unlimited (it truncates silently).
With StartTime at or before the epoch, RequestStartFor has no margin left, so
AVEVA's synthetic no-data/boundary row is stamped exactly at StartTime and was
kept: raw reads returned a fabricated Bad sample at 1970-01-01 and the per-sample
aggregates put it in bucket 0 (Start/End fabricated, Min/Max status Uncertain).
IsWindowSample and the aggregate loop now drop any row with t <= ProbeFloor
(RawAggregateReader.IsAfterFloor), matching the #686 walk-back's assumption.
- M1/M2: when the 1 ms pre-window margin (boundary row + stored samples, e.g. a
>16-sample tie cluster at start-1ms / lastTs-1ms) uses up the cap + 16 budget,
re-read ONCE asking for cap + m + 16 (m = pre-window rows seen); fail
BadTooManyOperations only if the re-read is still short because of them. The
raw budget is no longer clamped at the gateway default ceiling (AVEVA always
emits the boundary row, so page sizes 999,985-1,000,000 always came back
short); the #688 clamp stays on the per-sample aggregate path only.
- Merge trap: NumValuesPerNode 0 => int.MaxValue, never through the clamp.
- L3: the forward end is exclusive ([start, end), the IHistorianDataSource
contract); a point read (start == end, the tie-cluster over-fetch) inclusive.
- RequestStartFor never goes before the Unix epoch (#686's walk-back floor).
- L4 (docs): an Uncertain native row for an all-NULL interval is unverified live.
The default MaxTimeWeightedRawRows (1,000,000) + 18 exceeded the gateway's
default MaxValuesCeiling (1,000,000), so with AllowStreamingBeyondCeiling=false
every Gateway per-sample aggregate read was refused. The allowance is now clamped
at 1,000,000 while the cap is at or under it (RawAggregateReader.
ClampToGatewayCeiling); a cap above it keeps its allowance. Raw reads are not
clamped (rebased onto #686).
An AVEVA Full raw read leads with a row stamped at the query start (the value in
effect, Quality=133, or a no-data NULL row); it was returned as a stored sample.
Raw reads now use the #681 classification (RawAggregateReader.RequestStartFor /
IsWindowSample — one authority, by timestamp, since a stored NULL looks the same
on the wire): request from 1 ms early with NumValuesPerNode + 16 rows, keep rows
at or after StartTime, drain the whole stream. A full page still holds exactly
the cap, so #669 paging/resume/tie-cluster reads are unchanged; a margin that
swallows the allowance fails the node rather than ending paging early.
ReturnBounds is not plumbed on any backend (documented).
AVEVA answers one extra row for the interval before the window; it was mapped as
a result stamped before StartTime (N+1 values). Rows whose interval lies outside
the requested span (either direction) are now dropped after the drained stream.
An Uncertain (opc 64-191) interval kept no value (opc >= 192 was the only data
test); it now carries its value as UncertainDataSubNormal | Calculated.
Docs: AVEVA's TWA excludes NULL-gap time while the SQL backends bridge it;
end-to-end per-interval values also need historiangw#27.
1. Uncapped reverse raw read on a forward-only backend = ONE forward read of the
window, filtered and reversed in memory (was a segment-by-segment walk).
2. Segment width is aimed at ~half the row budget from the density just observed
(overflow: from the truncated read's own sample span) and carried page to page
in HistoryContinuationState.SegmentWidthTicks.
3. The walk starts no later than now + 1 day and never reads before the epoch.
4. Request bounds are rounded outward to the millisecond, so a row AVEVA
fabricates at a rounded-up query start cannot land inside the segment.
5. Budget M = max(1, MaxTieClusterOverfetch); reads ask M + 2 rows, completeness is
'short OR last row past the segment', and a read crowded by rows before the
segment is re-read once (<= 2M + 4). A cluster of exactly M, neighbouring
clusters each within M, and M <= 1 are all served; only one timestamp over M
is refused. ServerHistorianOptions text updated.
6. The raw and events tie-path oversized-cluster refusal is BadTooManyOperations
(was BadHistoryOperationUnsupported), consistent with the walk.
7. GatewayHistorianDataSource.ReadRawAsync: a caller cancellation is not a backend
failure; NumValuesPerNode = 0 is sent as int.MaxValue (the client package and
the gateway reject max_values <= 0, so every uncapped Gateway raw read failed);
a MaxValuesCeiling refusal is a HistoryReadLimitExceededException.
8. Tests: upper-edge filter, epoch floor, look-ahead, ms alignment, exact/adjacent
clusters, minimal budget, carried density, and the walk over the real Gateway
source + FakeHistorianGatewayClient (OpcUaServer internals visible to it).
#669 turned an end-only capped HistoryRead-Raw into a reverse read (start > end)
for every backend; the gateway rejects start > end (InvalidArgument), so on the
default Gateway backend every end-only capped and every explicit reverse raw read
failed the node and was counted as a backend health failure.
- IHistorianDataSource.SupportsReverseRaw (default false; SqlServer/QuestDB/Null
true, Gateway false; guard test pins that every production backend declares it).
- OtOpcUaNodeManager sends a reverse window only to a backend that declares it;
otherwise ReverseRawReader walks (lo, hi] back in bounded forward segment reads
(1 min, x4 while sparse; each <= MaxTieClusterOverfetch + 1 rows; a full read is
re-read narrower, never answered truncated; filtered on sample timestamps, which
also drops AVEVA's fabricated query-start rows; stops at the Unix epoch). A
one-timestamp tie cluster over the bound => BadTooManyOperations.
- ServeRawPagedAsync maps HistoryReadLimitExceededException to BadTooManyOperations.
- GatewayHistorianDataSource.ReadRawAsync refuses start > end unsent, without
recording a health failure.
The window-bounded pick joins (4a3b1256) let the optimizer choose, under
understated statistics, a nested-loops RANGE seek over the window with the pick
time as a residual, once per bucket (O(buckets x window rows)); the cached plan
was then reused for wide reads: gate 9 1d@1min 20 s, here 47 s / 3.6M reads.
Each pick join now carries WITH (FORCESEEK (PK_Sample (TagId, SourceTimeUtc))):
an exact-key seek on both columns per bucket under any statistics, and no
hash/merge join over the tag's history can exist (forcing one is error 8622), so
the window bound is dropped from the join. Output identical (EXCEPT both ways,
fwd/rev, incl. a mixed Double/Int64/Boolean/String/DateTime/outage tag).
Measured (6M-row tag + 86k mixed-kinds tag; ms / logical reads), understated
stats: 1d@1min 47,223/3.6M -> 253/25.6k; mixed 1d@1min 20,164/1.66M -> 137/24k;
accurate stats: 1d@1min 121 -> 160 ms, 1d@1h 79 -> 117 ms. Rejected: OUTER
APPLY TOP(1) FORCESEEK (identical), OPTION (RECOMPILE) (still 20 s under the bad
stats, and a compile per read).
StartEndJoinBackTests rewritten: understated statistics + compile-small/reuse-wide
(18,645 reads; the merged query read 1,645,845 / 23 s and failed it), accurate
statistics with 300k history rows either side (22,616), forced HASH/MERGE
refused (8622 — also failed against the merged query).
Docs: perf table + explanation; SDK-parity N2 (*ActualTime Calculated bit) and
N3 (Good String => BadAggregateInvalidInputs, SDK skips it).
- SqlServer StartEnd join-back: every pick join also carries the window predicate
(forward >= @start AND < @end, reverse > @start AND <= @end). Keyed on the pick
time alone a hash/merge/adaptive join read the tag's whole history (review:
43,657 vs 361 logical reads), and parameter sniffing reused that plan for every
tag. Forced OPTION (HASH JOIN), 1d@1h on 6M rows: 152,524 -> 6,756 reads.
- Usable picks (x/y) re-check the exact u predicate in the join, so a concurrent
HistoryUpdate Replace that turned the picked sample Bad/Empty reads as absent
(BadNoData), never a Bad row's value under Delta.
- StartEndJoinBackTests (integration): 300k history rows either side of a 1 h
window, forced HASH and MERGE joins, results identical to the unhinted plan and
Start/End correct, session logical reads < 1,000 (183 bounded; 8,055 with the
bound removed — the test fails then).
- TagIndexTests: the refused-ADD-INDEX test asserts the Warning naming the ALTER.
- Historian.md: re-measured table incl. long-history windows (output identical by
EXCEPT both ways in all 7 cases) and the forced-join read counts.
Start/End/Delta: one GROUP BY computes the counts and each bucket's four pick
times (MIN/MAX SourceTimeUtc, any quality and usable); each pick's columns are
then fetched by a seek on the clustered key (TagId, SourceTimeUtc). Replaces the
windowed ROW_NUMBER form (live-gate 8 F1), whose four window sorts spilled.
Output identical (EXCEPT both ways empty, 6M rows, forward + reverse incl. a
sub-microsecond reverse start). 6M rows / 30 days, avg of 3 x 2 rounds:
30d@1h 18.6-31.7 s -> 3.4 s (reverse 18.1-29.9 s -> 3.4-5.2 s)
7d@1d 5.2-6.1 s -> 0.8 s
1d@1min 0.31-0.40 s -> 0.16 s (1d@7min reverse 0.46 -> 0.12 s)
The seeks are separate reads within one statement: under READ COMMITTED a
HistoryUpdate delete between the aggregate and the seek nulls that pick; the
null-tolerant reader (2ceed561) makes that bucket's Start/End/Delta BadNoData,
transiently. Unit test pins it; documented in the query remarks + Historian.md.
QuestDB single-scan StartEnd unchanged.
- Bootstrapper: an ALTER ... ADD INDEX refused up front (e.g. no DDL rights)
logs a Warning and lets the bootstrap succeed (the index is optional); a
WAL-apply failure remains unobserved, like the TTL ALTER — documented.
Integration test (VARCHAR tag => refused ALTER) fails with the catch removed.
- Historian.md: a dominant tag can read ~20% slower on long windows (single-
threaded index scan vs parallel JIT filter); O3 writes rewrite index files;
sample_modified.tag / event.source deliberately not indexed, and why.
- CLAUDE.md: the concurrent-HistoryUpdate-delete caveat on the range-first order.
QuestDB: {p}sample.tag is now SYMBOL INDEX. The processed-read data-range
lookups (EarliestSample / LatestSample) were partition scans of the unindexed
symbol; on 53.6M rows / 120 tags / 61 daily partitions the indexed seeks are
1-1.5 ms warm / 2-4 ms cold whatever the tag's history (unindexed: up to 11 ms
warm, 14 ms cold; cost grew with the table). The bootstrapper adds the index to a
pre-#683 table (ALTER ... ADD INDEX, a WAL txn, idempotent, ~3 s for 68M rows);
AutoCreateSchema=false only logs the statement. Disk +~12%; write apply time
within noise; main per-sample/raw/time-weighted queries unchanged or faster.
Rejected by measurement/analysis: bounded existence probes (still full scans
when a side is absent), LATEST ON (slowest unindexed), per-tag first/last side
table (cannot track TTL drops or deletes INSERT-only).
Both backends now read the data range before the bucket statement: an append
racing the read reaches the buckets but not the range, so it can no longer
produce a Good Count of 0 for an interval that has samples.
Tests: TagIndexTests (new-table index, bound-param seek uses the index,
tombstones excluded / outage marker counted, pre-#683 migration with identical
processed results across 10 aggregates x 4 windows x both directions).
Joseph Doherty