- NodeManagerHistoryReadRawCapFuzzTests: 10k seeds (env OTOPCUA_PAGING_FUZZ_ITERATIONS)
through a real SDK node manager against wire-faithful fakes of SqlServer / QuestDB /
Gateway that enforce the backend half of the cap; random series with ties, random
windows (empty, on/past a tie, between samples, forward/reverse/end-only, bounds),
N in {0,1,2,small,huge}, cap in {off,3..1000}, tie budget {3,8,65536}. Pages must
concatenate to the cap-off one-shot answer (checked against the series), terminate
within values + 2 (+ one empty step per oversized boundary cluster), never exceed
the cap, never ask the backend for 0 / over-cap; refusals only past the budget.
- HistoryPaging.AdvertiseMaxReturnDataValues writes HistoryServerCapabilities/
MaxReturnDataValues (i=11273), read back through the SDK master node manager.
- MinMaxRawValuesPerRead raised to 3: at 2 a resumed page has no room (value + its
boundary tie + lookahead) and every value cost an empty page + CP (seed 15).
- RawPageLimit rows pin the default-cap arithmetic (9 999 first page, 9 998 resumed).
Fuzz findings recorded as follow-ups: a bounded read whose data fits one page
returns cap + 1 (end bound fitted against N, not the cap; node manager, F2);
the Gateway walk-back's cost on a sparse series under a tiny tie budget is
pre-existing (seed 320: 82 reads uncapped, 93 capped).
Refs #736
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
A failed SDK start (port in use, PKI unwritable, bad OpcUa:* override) was one Error line: the
node stayed /health/ready 200, joined its mesh, could be elected driver Primary and acked deploys
through the Null sink while every client got connection-refused.
- IOpcUaEndpointState / OpcUaEndpointState: the start outcome is a fact the hosted service
maintains (NotStarted / Started / Failed / Stopped + reason).
- OpcUaEndpointHealthCheck ("opcua-endpoint", Ready + Active) is Unhealthy unless Started;
registered by AddOtOpcUaHealth(hasAdmin, hasDriver) on driver-role nodes.
- A driver-only node rethrows the start failure (Program.cs's fatal handler exits non-zero so
the supervisor restarts it); a fused node keeps its AdminUI up but is visibly not ready.
- StopAsync detaches the sinks and then STOPS the SDK (the port closes, clients get
connection-refused rather than a stale, inert endpoint); DisposeAsync is idempotent with it.
- The ServiceLevel calculator / redundancy election live in Cluster + Runtime (not this lane);
the fact is exposed for the Wave 3 follow-up to consume.
Tests: port-in-use ⇒ state Failed, probe Unhealthy, real /health/ready and /health/active 503
(positive control: 200 once Started); driver-only rethrows; StopAsync closes the port and
DisposeAsync twice is a no-op; the probe registration is on both tiers for driver nodes only.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
DriverInstanceDiagnostics.Name carries the string driver-instance id
(what /hosts, telemetry and the artifact key on) and DriverInstanceId
the DB row Guid; the wire record has no slot for the authored display
name. The write-cap test asserts entry order per slot pair instead of
completion order of two concurrent writes.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
R05:10 — the namespace deny-list covered System.Threading.Tasks but not
System.Threading, so SemaphoreSlim / Monitor / SpinWait / Mutex /
EventWaitHandle could park a script's pool thread past its budget (or touch a
system-global kernel object); System.Type.GetType(string) / GetMethod* /
InvokeMember and System.Runtime.CompilerServices.RuntimeHelpers resolved and
reach a forbidden type without naming it syntactically.
ForbiddenTypeAnalyzer now denies the whole System.Threading namespace with a
type-granular allow-list (CancellationToken, Interlocked, Volatile — the
sync-free primitives), adds a member deny-list for System.Type's reflection
entry points (GetType/GetMethod(s)/InvokeMember/GetProperty(ies)/GetField(s)/
GetMember(s)/GetConstructor(s)/GetEvent(s)/GetDefaultMembers; instance
object.GetType() stays allowed) and puts RuntimeHelpers on the type deny-list.
Thread/ThreadPool/Timer leave the type list — the prefix rule reaches them.
R05:11 — the allocation budget landed with E1's ScriptRunGuard
(Scripting:MaxAllocBytesPerRun, default 64 MB): sampled around each run,
breach quarantines. It is observation after the fact; a pre-emptive OOM guard
still needs an out-of-process runner (follow-up issue in the lane report).
The Monaco editor shares the analyzer, so DiagnoseTests pins every new form
as an OTSCRIPT_FORBIDDEN marker and every allow-listed form as clean.
Refs #734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
DriverHostActor wraps the health publisher it hands its children in a
ChildHealthLedger (last snapshot per driver id, forwarded to the real
publisher), so GetDiagnostics now answers with each child's row id,
authored name, last-published state, device counts (host statuses when
the driver probes hosts, else one device) and last-change time — instead
of Guid.Empty / "Spawned" / 0 / now. Entries are forgotten on StopChild
and operator Restart.
IFleetDiagnosticsClient gains its first consumer: the new
/fleet/diagnostics page (node selector + ?node= query, revision,
running-from-cache notice, per-driver table). FleetDiagnosticsClient
logs the degrade-to-empty path at Warning.
Review ref: 02/U1.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 2026-10-21 G-5, G-6, G-8 ("wire them all").
- G-5: GatewayGalaxyHierarchySource now pages the DiscoverHierarchy RPC
itself (DiscoverHierarchyRawAsync) at Repository.DiscoverPageSize. The
mxaccessgw client's typed DiscoverHierarchyAsync overloads hard-code a
5000-row page and DiscoverHierarchyOptions carries no page size, so the
authored knob reached nothing; the raw request takes page_size on the
wire, so it is honoured here (non-positive → 5000). The loop keeps the
client's repeated-page-token guard.
- G-6: GalaxyDriverProbe reads ProbeTimeoutSeconds from the config and
runs under min(caller budget, configured cap) — the knob can shorten a
probe, never extend it past the host's budget. The TCP preflight now
shares that deadline instead of hanging on the caller's token alone.
- G-8: GatewayGalaxyDataWriter runs a batch as one lane per distinct tag,
concurrently and bounded at 8 (MaxConcurrentWriteLanes); same-tag
writes keep their authored order inside one lane and results land at
their request's index. Acks stay serial by design.
- GalaxyDriverForm: help text now says what the two knobs actually do.
Refs #735
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
ReceiveAsync suspended the mailbox for the whole of every handler, so one Test Connect against a
black-holed PLC (60 s clamp) or one large deploy compose stalled every other admin operation
fleet-wide — AdminUI alarm acks/shelves, driver restarts, the next deploy.
- TestDriverConnect runs the probe on the thread pool and pipes a TestDriverConnectResult
straight to the requester (every path yields a result, never Status.Failure).
- StartDeployment serialises only the in-flight check (DB row + an actor-local compose-in-flight
flag, because the Deployment row is written at the END of the compose); the snapshot read,
validation, gzip and row write run off-actor and return as DeployComposed /
DeployComposeFailed, where the coordinator is told and the requester answered.
Tests: an alarm ack completes in < 1 s while a 60 s probe is pending and while a compose is
parked on a gated DbContext (positive controls: both still complete afterwards); a second
StartDeployment during an in-flight compose is refused.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
A timed-out script is abandoned, not stopped, and every seam re-ran it on the
next trigger, leaking one pool thread per trigger (R03:1.1, R05:2, R07:L1).
A shared ScriptRunGuard, keyed per script instance, now:
- refuses a run while that script's previous orphan is still executing
(Bad "previous evaluation still running" + otopcua.script.run_refused);
- quarantines after 3 consecutive non-completions or an allocation-budget
breach (sampled with GC.GetAllocatedBytesForCurrentThread; default 64 MB):
no further dispatch, Bad quality, one Error, otopcua.script.quarantined,
cleared on source change / redeploy / reset;
- starts the budget clock when the pool picks the run up, so starvation is
no longer charged to the script (R07:P5), and cancels the evaluator token
on timeout.
Wired into RoslynVirtualTagEvaluator, CalculationEvaluator (new async
EvaluateAsync; the driver's pump and timer ticks now await the run behind a
SemaphoreSlim gate) and ScriptedAlarmEngine (a non-completing predicate
holds state and annotates the condition BadInternalError). ScriptingOptions
(Scripting:MaxAllocBytesPerRun, Scripting:QuarantineAfterConsecutiveTimeouts)
carries the knobs; the Calculation driver config accepts the same two keys.
Tests: guard unit suite (incl. the 30-trigger runaway leaving one orphan with
a healthy sibling as positive control), per-seam refusal/quarantine/reset
tests. The S12 race test now uses one vtag id per loop: twelve concurrent
evaluations of ONE id is a shape production never produces and the guard
refuses by design.
Refs #734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
CallTimeoutSeconds was bound, defaulted and documented but consumed
nowhere. TelemetryDialSupervisor now arms a per-generation deadline on
every dial: a dial that has delivered no event by the deadline is
cancelled and treated as a transport failure (reconnect with a failure
streak). A live node replays its snapshots on subscribe, so this bites
only a dial that never connects (a black-holed host, where the socket
connect has no timeout). An established stream is deliberately not
bounded — the wire has no heartbeat, keepalive pings own its liveness.
Review ref: 02/U2.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
StartDeployment read the whole config DB twice with no transaction — DraftSnapshotFactory for
the validator, then ConfigComposer for the artifact, plus clusters/nodes and Script.SourceCode
again in between. An AdminUI save landing between the two shipped an artifact the DraftValidator
never saw.
- DraftSnapshot carries ServerClusters, ClusterNodes and ApiKeys; DraftSnapshotFactory reads
every table inside one SNAPSHOT transaction on a relational provider (no-op on InMemory or
under an ambient transaction).
- ConfigComposer.Compose(DraftSnapshot) is pure over the snapshot (ordinal-sorted keys);
SnapshotAndFlattenAsync(db) delegates to it, byte-identical.
- AdminOperationsActor validates topology, estimates compile cost and composes the artifact
from the one snapshot it holds.
- Migration EnableSnapshotIsolation: ALTER DATABASE CURRENT SET ALLOW_SNAPSHOT_ISOLATION ON
(suppressTransaction; no model change).
Tests: the SQL Server snapshot read ignores a row committed mid-read (positive control: a later
read sees it); an edit injected between the Equipment and Tags reads is absent from the shipped
artifact and rejected by the next deploy; Compose(snapshot) == SnapshotAndFlattenAsync(db).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- DriverHostActor.StopChild removes the driver's _lastReprovisionUtc stamp
(and its deferred-reprovision timer): the map grew per removed id and a
re-added driver inherited a backoff it never earned (02/L2).
- Stale now registers explicit drops for AttributeValueBatchPublished,
AttributeAlarmPublished, ArtifactForApply and ApplyWedged instead of
letting them hit Unhandled (02/H5).
- DbHealthProbeActor runs its SELECT on the thread pool and pipes the
outcome back (single-flight), with a 2 s command timeout on relational
providers, so a hung SQL Server no longer pins the actor thread and the
cached GetStatus answer stays reachable (02/P7).
- TelemetryLocalHub eviction tombstones are timestamped and swept past a
10 min TTL on the next Evict; a TimeProvider ctor is the test seam (02/L4).
- Tests: ValuePublishes.Single helper replaces the per-value
AttributeValuePublished in CalculationDependencyFlowTests (02/U4, test
half; the production receive is owned by Lane E).
- R05/8 audit: PollGroupEngine.Unsubscribe has no production caller; every
driver already uses UnsubscribeAsync (no change).
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 2026-10-21 G-4, G-7, G-11, G-10.
- G-4: PerPlatformProbeWatcher publishes its probe handles into the
driver's SubscriptionRegistry under the reserved ProbeSubscriptionId
(-1), so the EventPump's fan-out finally reaches the watcher — every
ScanState event used to be dropped as "stale after unsubscribe" and
per-platform host status never updated. ResubscribeAllAsync delegates
the reserved entry to the watcher's new ResubscribeAsync, so the probe
survives a reconnect; StopAsync removes the entry. Probe traffic is
routed by the reserved handle and never reaches the public OnDataChange.
The watcher is also built on the seam path (subscriber only), which is
what makes the wiring testable without a gateway.
- G-7: GalaxyDiscoverer.DiscoverAsync returns the hierarchy it fetched and
the driver selects its $WinPlatform/$AppEngine probe targets from that
same sweep instead of asking the gateway for the whole galaxy twice.
- G-11: OnPumpDataChange stamps LastSuccessfulRead from inbound stream
data, throttled (1 s default), so the comms-loss watchdog has honest
liveness evidence under a Degraded supervisor.
- G-10: an EventPump dispatch-loop exit that is not a cancellation now
reports a stream fault so the supervisor recreates the pump, instead of
leaving a deaf pump reporting Healthy.
Refs #735
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
WriteAttribute was a ReceiveAsync, suspending the child's mailbox for the
whole device round-trip (up to 5 s): writes serialised, the second of two
operator writes to a slow device missed the host's 8 s Ask and was
reverted although the device took it, and every data change queued behind
the write. Writes are now a synchronous Receive that launches the device
call on the thread pool (sender captured, reply from the task, slot freed
by a self-sent WriteCompleted) under a per-driver cap (default 4; excess
writes queue in order). Queued writes fail fast on every exit from
Connected. Subscribe/unsubscribe stay ReceiveAsync.
Review ref: 02/P2.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
GalaxyDriver.ShutdownAsync was a no-op: the MX session, both owned gRPC
channels, the EventPump, the alarm feed, the DeployWatcher, the probe
watcher and the reconnect supervisor were released only by DisposeAsync,
which the host never calls — one leaked gateway session per respawn,
removal and /raw browse, and a watchdog Shutdown→Initialize that neither
tore down nor rebuilt anything (review 2026-10-21 G-1, G-2, G-9, G-3).
- ShutdownAsync and DisposeAsync share one ordered TeardownRuntimeAsync
(supervisor first, then deploy/probe watchers, pump, alarm feed,
session, channels); owned seams are nulled so the next InitializeAsync
rebuilds; injected seams survive a shutdown and drop only on the final
dispose. Lifecycle calls are serialised behind a gate.
- A failed production connect releases its partial runtime (no orphaned
client per watchdog retry) and lands Faulted before and after.
- StartDeployWatcherAsync stops the previous watcher before starting a
replacement, and now starts on a production instance's FIRST
InitializeAsync (the old guard returned whenever discovery had not run
here, which on a runtime instance is always).
- ReconnectSupervisor releases each cycle's CancellationTokenSource when
its loop exits and gains an awaitable StopAsync; PerPlatformProbeWatcher
gains an awaitable StopAsync.
- EventPump treats any exception under cancellation as a clean stop, so a
teardown never wakes the supervisor.
- Internal seams for tests: an IGalaxyDeployWatchSource ctor parameter and
a session-open override forwarded to the owned GalaxyMxSession.
Refs #735
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review R08:1.1: an uncapped HistoryRead-Raw (NumValuesPerNode = 0) read a
tag's whole window into memory on every backend (SQL TOP(int.MaxValue),
QuestDB LIMIT, Gateway max_values = int.MaxValue + a second snapshot copy).
New ServerHistorian:MaxRawValuesPerRead (default 10 000; <= 0 unbounded with
a Validate() warning; 1 raised to 2). Each backend data source takes it,
reports it as IHistorianDataSource.MaxRawValuesPerRead (the Gateway: min
with a strict gateway's learned ceiling), so the node manager's existing
RawPageLimit paging serves N = 0 and N > cap as pages of at most the cap
with a continuation point. Independently, a backend raw read asking for 0
or more than the cap reads cap + 1 rows and is REFUSED
(HistoryReadLimitExceededException => BadTooManyOperations) when the window
holds more - never answered truncated, since the server's paging reads a
short answer as the last page. Bound through the Host registry, the
SqlServer/QuestDb providers and GatewayHistorian.CreateDataSource.
Tests: options (default/binding/warnings), provider binding, Gateway unit +
through a real SDK node manager (uncapped -> 999/998/998/605 + CPs), and a
conformance case for the SQL backends.
Refs #736
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The Reconnecting retry timer started a new ShutdownAsync + InitializeAsync
pair on every tick whether or not the previous attempt had finished; the
generation token only discarded the older RESULT, so with an Initialize
bound longer than the cadence the teardown ran underneath an attempt still
inside the driver (the dispose-before-overwrite race). An _initInFlight
gate, cleared by the piped InitializeSucceeded/Failed, skips a tick while
an attempt is running; the pre-reconnect teardown is bounded by the
shutdown timeout; PostStop cancels an in-flight attempt before shutting
the driver down.
Review ref: 02/H2.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
DriverInstanceActor.PostStop now bounds IDriver.ShutdownAsync by a
shutdownTimeout (default 5 s, logged on overrun) and then disposes the
driver (IAsyncDisposable preferred, else IDisposable) — but only on a
real stop. PreRestart flags the restart so the PostStop it triggers
skips the dispose, because a supervision restart re-uses the same
driver object (Props closure) and the host re-initialises it.
DiscoveryDriverBrowser disposes its throwaway capture/probe driver
after the bounded shutdown, including after a failed or timed-out one.
Review refs: 02/L1, 02/P4, 06/G-1 (host context), 07/L2.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
A Deployment row left in Dispatching with no NodeDeploymentState rows (HandleDispatch threw
before its SaveChanges, or the process died between the AdminOps row write and the
coordinator's Tell) was recovered with an empty expected set; the deadline then hit the
0 == 0 "race" branch and returned without touching the row, so every later StartDeployment was
refused AnotherDeploymentInFlight until someone hand-edited SQL.
- PreStart: a recovered row with no expected-ack rows is a lost dispatch. Inside the apply
deadline it is re-driven through HandleDispatch (bounded: CreatedAtUtc never moves); past it
the row is closed TimedOut with a reason naming the condition.
- HandleDispatch is idempotent on a re-dispatch (rows already seeded ARE the expected set).
- HandleDeadline takes the race branch only when _expectedAcks.Count > 0.
- The PreStart recovery read projects ArtifactBlob away (R03:3.5).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The #687/F4 crowding re-read doubled its room up to 8 re-reads with no cap, so a pathological single-ms
cluster on a streaming gateway was streamed several times over for one page. The room is now capped at
ServerHistorian:MaxTieClusterOverfetch (plumbed into GatewayHistorianDataSource as maxPreWindowRows,
default 65536); past it, or once a read has already seen more, the read is BadTooManyOperations with
PreWindowRows set.
Item 1: IHistorianDataSource.MaxRawValuesPerRead (Gateway: learned ceiling - 16 = L; 0 elsewhere, guard-tested).
RawPageLimit plans every raw page on top of #692's lookahead: with L > 0 the read (page P + boundary ties k
+ lookahead) is <= L, P = min(N, L - k - 1) (uncapped: L - k - 1); no room => straight to the tie-cluster
over-fetch (never an uncapped or underflowing request). One re-plan + re-read when the read taught a new
limit or was refused for crowding pre-window rows (HistoryReadLimitExceededException.PreWindowRows): the
page shrinks by them. Over-fetch bound min(MaxTieClusterOverfetch, L - 1); slice size never 0.
Review fixes: F1 a window starting at/before the epoch counts AVEVA's synthetic row as pre-window (a
ceiling-truncated read was reported complete); F2 the page path disables ReverseRawReader's own re-run
(cold end-only ceiling 20/N=3000 cost 3,023 reads, now <= 40); F3 crowded look-back re-read around, not
refused; F4 the #687 crowding re-read room at least doubles (up to 8 re-reads), so a cluster of hundreds
just before a resume point no longer fails on a default gateway; a crowding refusal that leaves no room
is BadTooManyOperations, never a tick-by-tick cursor.
Item 2: ServerHistorian:EventReadHorizon (default 00:00:00 = exact, unchanged). When set, every fresh
HistoryReadEvents window's earlier bound is raised to now - horizon (both directions, all backends).
Lossy by design; documented.
Item 3 (native TimeAverage/Total re-check) stays blocked on historiangw#27.
A time inside a gap of a bursty tag costs a 64-row jump read landing in the next
burst plus a bisection back across the gap (~6 reads), so the 64 + 4/time cap
refused a 1 h/day 1 Hz tag asked at noon on 30/100/300 consecutive days
(186/606/1,806 reads) that AVEVA's own AtTime served. The cap is a runaway guard
(HistoryReadDeadline bounds wall clock): now 64 + 16 per distinct time.
Regression tests for x30/x100/x300 (each <= 7 reads a time, values checked).
Docs: DateTime-hinted nodes are stepped but keep the gateway's Double (the
Gateway never historizes DateTime nodes; the SQL backends return DateTime).
H1: the walk-back reset its width after a full segment and discarded a full
whole-remainder read, so a dense tag that stopped reporting weeks ago gave up
after 65 reads (BadTooManyOperations). It now gallops (x4, whole-remainder read
after two empties) and then BISECTS once any stored row is known below — a full
segment's (incl. a full whole-remainder read's) last row, or an earlier window
read's last row: (mid, hi] empty => hi = mid; full => known = its last row;
complete with rows => done; one density-aimed segment after a full read until an
aimed segment comes back empty; below 2 ms the whole stretch from the known row
(never a 100 ns step). Stopped 1 h..365 d with 5 y history: 6-18 reads at
1/min, 21-25 at 1 Hz, 25-31 at 10 Hz.
M1: a jump read asks for 64 rows (1,000 times over 10 y: ~66k rows, was ~1M).
L1: per-request read cap 64 + 4 x distinct times with a clear message; a strict
ceiling below 4 rows is refused naming the ceiling.
L2: DateTime hints are stepped. L3: under an integer hint a NaN/inf or
out-of-Int64-range value is unusable, never saturated Good.
L5: conformance Int64 AtTime asserts the type. L4: docs name the backend
differences. Nit: the window read end is clamped to now + 1 day.
Reviewer harnesses (Scratch693Fuzz/Cost, not committed): 0 differential diffs
over 2,400 cases.
F1 (pre-existing): SliceTieCluster continued a forward read while next <= endUtc, but the
forward window is end-exclusive; an oversized cluster one step below EndTime resumed AT
EndTime, the resume became a start == end point read and returned the EndTime samples as
data (microsecond: EndTime 8.1 us truncated to 8.0). Continue only while next is strictly
before EndTime truncated to the backend resolution.
Tests: A_tie_cluster_just_below_the_end_time_never_resumes_onto_the_end_time (native +
microsecond; both fail on the old rule) and
A_cursor_inside_a_tie_cluster_over_fetches_without_a_normal_read_first (pins the
WithinTieCluster shortcut by per-page read count; fails with the shortcut disabled).
Docs: the oversized-cluster exception to 'every page full / no lookahead => last' (F2),
and the N + skip + 1 strict-ceiling edge note referencing #694 (F4).
Review follow-up: on the Gateway every value is a Double (AVEVA stores a Boolean
as a 0/1 Int2), so the raw-evaluated AtTime interpolated a Boolean node to 0.5.
- IHistorianDataSource gains a default-interface overload
ReadAtTimeAsync(fullReference, timestamps, TypeCode? valueType, ct) that
forwards to the existing one (SqlServer/QuestDB/Null/fakes unchanged).
- OtOpcUaNodeManager.HistoryReadAtTime passes the node's scalar DataType
(AtTimeValueType: built-in type -> TypeCode; arrays/other types => null);
Raw and UNS NodeIds alike.
- GatewayAtTimeReader converts each raw row to the hinted kind BEFORE the shared
AtTimeEvaluator runs (Boolean = value != 0 => stepped; integers => Int64 =>
interpolated, rounded half away from zero, returned as the node's integer
type; floats Double), so the evaluator's own type branch decides.
- AtTimeEvaluator boxed every interpolated Int64 as a Double (`cond ? (long)x :
v` widens to double); its "keeping its type" test passed because Shouldly
ShouldBe compares numbers across types. Fixed (affects SqlServer/QuestDB too)
and pinned with ShouldBeOfType.
A resumed raw page asked the backend for only N rows from the boundary timestamp
(inclusive), then trimmed the boundary ties already emitted: it netted N - skip values,
and at N = 1 nothing — the empty "full" page was taken for a stalled tie cluster and a
MaxTieClusterOverfetch + 1 (65,537-row) over-fetch stepped one tick, returning an EMPTY
page with a continuation point between every value (8 values = 17 pages / 25 reads),
forward and reverse, with or without ReturnBounds, on every backend.
Each page's read now asks for N + boundarySkip + 1 rows: the ties it trims, the page and
one lookahead row. The lookahead row is the 'more data' signal, so an exact multiple ends
on its last full page (no trailing empty page). The over-fetch fires only when every row
past the trimmed ties is still on the boundary (a genuine cluster larger than the page);
a cursor left inside a cluster is flagged WithinTieCluster and over-fetches directly.
A page that never leaves the boundary adds the prior skip to its cursor.
Low items: BoundedReadHasAnythingAsync carries its walk hint and, when there is no data,
the end bound it found (HistoryEndBound.Resolved + DataExhausted), so the next page reads
nothing; HistoryBounds.IsPlaceholder is an explicit marker (ConditionalWeakTable over the
NotFound instances) instead of status+null shape, so a stored Bad_BoundNotFound row is data.
Tests: NodeManagerHistoryReadPageFillTests (N=1 every shape/direction/bounds, tie clusters
straddling boundaries at N=1/2/3, exact-multiple, carried end bound + walk hint, stored
placeholder-shaped row). Existing paging tests updated where they encoded the short pages.
Gateway HistoryRead-AtTime passed AVEVA's own interpolation through: live it
answered the previous value as Good at a stored NULL's own time, null + Bad at
a Good sample right after a NULL, a drifted 99.99999999944 for a stored 100,
and no Interpolated bit. It is now served from raw gateway reads and evaluated
with Sql.Common's AtTimeEvaluator, as the SQL backends do (Part 11 §6.5.4).
GatewayAtTimeReader: one <=1,024-row forward window read [first-1ms, last+1ms]
for the batch, a walk-back (density-seeded, x4 growth, whole-remainder shortcut,
bisects dense history) for the first time's prev, and an 18-row probe for the
last time's next; truncated window reads continue or jump with a density-sized
margin. All bounds whole ms; a row counts only strictly inside its request
(AVEVA's boundary row never a sample); Bad rows value-less (#690); never past
now + 1 day; strict-gateway ceiling learned (#691.3) without re-probing.
10 dense times = 1 read (exact) / 3 (interpolated); 10 hourly = 12.
Kill switch ServerHistorian:GatewayAtTimeFromRaw (default true; false = the
old AVEVA passthrough). Known limit: the wire carries no data type, so numeric
tags interpolate as Double (Boolean/integer nodes are not stepped/rounded as on
the SQL backends).
1. The at-or-after bound read on a forward-only backend (Gateway) ends at
now + ReverseRawReader.LookAhead instead of DateTime.MaxValue (the gateway
only checks start <= end, so a year-9999 end reached AVEVA); a bound beyond
the horizon is not found without a read. SqlServer/QuestDB keep MaxValue:
they bind it safely and may hold samples further ahead. The Gateway fake now
refuses far-future ends, so the missing-end-bound test is no longer vacuous.
3. No data and no bound found (whole read empty) => GoodNoData, no values —
the pre-bounds answer (6.5.3.2: success only if one or both bounds exist).
A one-value page whose start bound is missing first checks there is data or
a found end bound.
4. ServerHistorian:HonorReturnBounds (default true); false ignores ReturnBounds
as before #691, including the modified-read Bad_InvalidArgument.
The start bound is the stored prior sample at its own time (walked back), never
the synthetic boundary row stamped at the query start; a stored NULL is a Bad
bound; reverse end bound walks back too.
The node manager composes the start/end bounds from each backend's existing raw
reads (one authority, HistoryBounds): at-or-before = a capped reverse read
(native, or ReverseRawReader's walk-back on a forward-only backend — never
AVEVA's synthetic boundary row), at-or-after = a capped forward read. Bounds
count toward NumValuesPerNode, the start bound leads the first page, the end
bound ends the last (carried in the continuation point). Missing bound =
Bad_BoundNotFound (null, at the bound time; one-sided reads ±1 s from the
previous value, Table 1 footnotes a/b). A modified read with ReturnBounds is
Bad_InvalidArgument (Part 11 6.5.3.3).
The .NET SDK's ReadRawModifiedDetails defaults ReturnBounds to TRUE; the
existing tests built details without setting it and now set it false.
Oracle: every row of Part 11 Table 1 over three backend shapes, plus paging
concatenation, ties, Bad bounds, microsecond truncation; conformance pins the
lookup primitives on SqlServer + QuestDB.
A tie cluster's truncated read left the width at one tick; once the gap
above it was under two milliseconds nothing floored it, so segHi
stepped down one tick per read: 30 rows at one ms 0.3 s below hi with a
carried 1 s width took 10,006 identical reads (S8). (a) The top-anchored
width is floored at the granularity, Min kept; (b) a segment no wider
than the granularity is the unit that escalates to
MaxTieClusterOverfetch and then refuses - per millisecond, exact for the
ms-resolution Gateway (documented). Both are needed: the floor alone
loops on crowded reads with a small tie budget. S8 is a repo test
(4 shapes, reads <= 30, 0 repeated requests); random-data tests now
measure the refusal unit as rows in any 1 ms window. Reviewer harness,
5,000 seeds x default/MSONLY/CROWD: worst page 70 reads (default/MSONLY),
63 (CROWD) at M=65536, p99.9 27-37; no page over 500 reads at any tie
budget.
The null source has no history in either direction; reporting false
made the server walk an empty reverse events read back through three
empty forward reads. It now answers the reverse overload with the empty
result itself (guard test unchanged: it still declares both members).
Bisecting the gap above a truncated read halved below the 1 ms request
granularity (requests are ms-rounded, so it re-sent identical requests)
and bisected the WHOLE gap even when the burst ended right above the
row it saw: a 1,026-row burst ending 100 days ago took 46 reads. After
an empty top-anchored segment the walk now gallops up from the known
row ((known + d, top], d x4, following each truncated read), then
bisects the bracket, with halving floored at the granularity (the Min
kept; a crowded gallop falls back to top-anchored halving). Burst: 18
reads, 0 repeated requests; dense tag stopped 2 days ago: 26 (gallop
alone crawled: 94). Walk fuzzed at 4,000 seeds locally (400 in-tree).
Docs: measured counts; CLAUDE.md gateway prerequisites list
RuntimeDb:EventReadMaxRows.
GatewayHistorianDataSource.ReadEventsAsync sized its result list to
maxEvents, which is int.MaxValue for NumValuesPerNode = 0: every
uncapped Gateway events read (forward on master, and the new walked
reverse one) threw 'Array dimensions exceeded' and counted as a backend
failure. Capacity is bounded at 1024. The events walk delegate moved to
ReverseEventsReader so it is tested over the real Gateway source
(forward uncapped, reverse uncapped, reverse capped).
A reverse or end-only capped events read on a backend without
SupportsReverseEvents (Gateway, Null) was refused
BadHistoryOperationUnsupported (explicit reverse) or answered with the
OLDEST N ascending (end-only, #675). It is now served newest-first by
the same ReverseWalk as raw reads: page-sized ms-aligned segments,
never answered from a read the source flagged truncated, ties sorted by
EventId (the gateway orders by event time only - the fake shuffles
them per call), one extra event walked to signal 'more', uncapped = one
forward read reversed, and the width + proven-empty stretch carried in
the events cursor. The oversized-cluster over-fetch keeps exactly the
events at T in the same order. A tie cluster over the budget is
BadTooManyOperations. Docs note the gateway's EventReadMaxRows must be
unlimited (it truncates silently).
The last pages of a deep reverse page-through on a forward-only backend
each walked down to 1970 one x4 growth step at a time (~14-16 reads,
~2.5 s live). After two empty segments the walk now asks once per page
for the whole remainder at the page budget (a truncated answer is
discarded and the walk carries on), and the stretch the page's last
complete read proved empty rides the continuation point
(HistoryContinuationState.EmptyFromTicks/EmptyToTicks, session-held, so
no versioning) so the next page steps over it. Fuzz now pages with the
walk's own hints and with true random empty stretches.
Joseph Doherty