RefreshAsync now tells "the store answered: no such secret" (revocation - drop the refreshed key AND the
lazily cached one) apart from "the read failed" (keep the previous key). Store reads fan out through
Parallel.ForEachAsync capped at MaxConcurrentRefreshReads = 8 instead of one read per cluster at once.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The one-snapshot refactor re-sorted every collection in memory with an ordinal comparer, while the old
composer's SQL ORDER BY used the database's case-insensitive collation; mixed-case keys therefore re-hashed
an unchanged configuration (spurious drift fleet-wide, a "new" revision on a no-op redeploy). The snapshot
factory now reads every composed table ORDER BY its natural key server-side and Compose serialises in the
order given. Pinned byte-for-byte against a verbatim replica of the old composer on the real SQL Server.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
I7: a Dispatching row committed after the coordinator's PreStart whose Tell never arrived had no owner and
refused every later deploy. The in-flight check (now fully off the admin singleton's thread, serialised by the
actor-local compose flag alone) closes a Dispatching row with zero NodeDeploymentState rows that is past the
apply deadline as TimedOut (optimistic on RowVersion) and proceeds.
HandleDispatch ignores a dispatch for a Sealed/PartiallyFailed/TimedOut row, so a late or duplicated dispatch
cannot flip it back. Coordinator status/reason writes attach a key+RowVersion stub instead of loading the
multi-MB ArtifactBlob; proven on the real SQL Server (blob untouched, seal lands).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
SQL Server's DbUpdateException for one duplicate names every entry in the statement batch, so detaching
ex.Entries dropped the whole flush. On a row-level rejection the writer now asks the database which of the
batch's EventIds it already holds, drops exactly those, re-saves the rest, and falls back to row-by-row only
if a non-duplicate row is still refused. Fresh context + entities per attempt. Pinned against the real
central SQL Server (new SqlServerTestDatabase harness) and a faithful whole-batch interceptor.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The window used to count every terminal row, so 50+ failed deploys pushed the last good artifact out of it.
Now the newest KeepSealed (>= 1) Sealed rows keep their blob, failed rows are prunable unless referenced in
the ReferencedDays window. Pinned: 60 TimedOut + 1 old Sealed -> Sealed retained, 60 TimedOut emptied.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Coordinator ruling (I6): until ServiceLevel consumes IOpcUaEndpointState (F-C1), the probe must not sit on
the Active tier of a fused node, because Traefik routes the AdminUI on /health/active. Driver-only nodes keep
it on Ready + Active; admin-only nodes do not register it. Pinned per role.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
ServerAuthenticationStateProvider throws InvalidOperationException from GetAuthenticationStateAsync in any
scope where no circuit set its state (minimal-API requests, hosted-service scopes). The accessor now treats
that as "no circuit identity" and falls back to the HTTP principal. Pinned with the real provider on both
sides (unset -> HTTP fallback; set -> circuit principal wins).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The hand-rolled bearer handler is replaced by AddJwtBearer (Microsoft.AspNetCore.Authentication.JwtBearer
10.0.7, pinned on the same servicing line as the other AspNetCore packages). Validation parameters come
from JwtTokenService.BuildValidationParameters, which now pins ValidAlgorithms = HS256 for both the
bearer scheme and TryValidate; MapInboundClaims is off so the minted Role/zb:username keys resolve.
/auth/token requires the cookie scheme, so a bearer can no longer mint a fresh token for itself.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- RunWriteAsync wraps the invoker call in WaitAsync(writeTimeout) (Props,
default 10 s = 2x the 5 s write cap): a token-ignoring hung write now
answers BadTimeout, releases its slot and tag, and its orphan outcome
is observed and discarded - later writes to that tag reach the device.
- WriteCompleted carries the launching incarnation (from the process-wide
generation source); a late completion from a previous incarnation after
a supervision restart is ignored instead of freeing the new
incarnation's slot/tag.
- Test: a hung token-ignoring tag-a write is abandoned and a later tag-a
write reaches the device while the abandoned one is still inside it.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- GatewayHistorianDataSource.ReadRawWindowAsync keeps the cap + 1 probe budget
(budgetCap) separate from the client's cap: a strict-ceiling or crowding
refusal now reports Limit = the server cap that applied and advises setting
NumValuesPerNode at or below it (it said "lower NumValuesPerNode" with
Limit = cap + 1 to a client that sent 0).
- Fuzz: the oracle-source over-cap assertion was vacuous (cap 0 never counts);
it now checks no reverse window reached the forward-only shape. Class docs
name the post-floor cap set (3 ... 1000) and distinguish NumValuesPerNode
1 / 2 from the smallest server cap.
- HistoryPaging.IsFullPage doc says it judges the REQUEST cap (page + ties +
lookahead), which is finite under any limit; AdvertiseMaxReturnDataValues
swallows any exception (returns false for the caller to log), as its doc
promised.
Refs #736
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
BoundedApiKeyStore's negative cache was keyed by key id, so a flood of DISTINCT well-formed
otua_<random>_x credentials missed it every time and cost one SQL query per attempt under the
SDK's SessionManager-wide lock, no credential needed (R03:3.3).
- IApiKeyStore.ListKeyIdsAsync (default null = unsupported; ConfigDbApiKeyStore lists every
ApiKeyId): the bounded store keeps a known-id set refreshed every StoreCacheSeconds
(single-flight, failed refresh keeps the previous set) and refuses an unknown id without a store
read ({outcome=unknown_id}).
- A process-wide miss budget (token bucket, Security:ApiKeys:MissBurst=100 /
MissRefillPerSecond=20, ApiKeyStoreGuardOptions) bounds lookups that reach the store and miss
while no set is available; a hit refunds its token. Per-endpoint accounting needs the
OpcUaServer impersonation seam to carry the client address — follow-up.
- docs/security.md: the new knobs, and the known-cluster gate's documented fail-open window at
central boot (R03:4.2) plus the async PSK refresh (R03:3.4).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- Writes: a second write to a tag already inside the driver waits in the
queue behind it (_tagsInFlight) while other tags still run under the
cap of 4; queued writes are stamped and failed BadTimeout at dequeue
once older than the 8 s host Ask budget (Props queuedWriteBudget)
instead of being written late.
- Initialise attempts are bounded pessimistically (initializeTimeout,
default 30 s = 2x Tier C) on top of the invoker's token-only timeout:
an overrun is logged as abandoned, reported InitializeFailed so the
retry loop continues, and its late result is dropped by generation.
Generations come from a process-wide Interlocked counter so a prior
incarnation's late result cannot collide (F6).
- PostStop shares one shutdownTimeout budget across shutdown + dispose.
- TelemetryDialSupervisor cancels the establishment deadline on
StreamStopped; FleetDiagnosticsClient builds the ActorSelection inside
the try (a malformed node degrades to "unavailable"); the diagnostics
page labels the column "Last health publish".
- Tests: same-tag serialisation with a distinct-tag positive control;
stale queued write BadTimeout; token-ignoring hung init -> later
attempt connects; stopwatch assertion demoted to a ceiling.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- MeshGrpcCommandDialer.RetainOnly (IMeshCommandDialer, default no-op): CentralCommunicationActor
hands the dialer the current endpoint set on every contact refresh, so a renamed host or moved
port releases its cached GrpcChannel and idle HTTP/2 connection instead of living for the
process (R03:1.3).
- IMeshPskProvider.RefreshAsync (default = the old InvalidateAll): MeshPskProvider pre-resolves
every known cluster's key concurrently, bounded like the sync path, and swaps an immutable
snapshot in atomically; a failed read keeps that cluster's previous key. The actor calls it from
ContactsLoaded (after SetKnownClusters, off-actor via PipeTo) instead of InvalidateAll, so the
first ack per cluster per ContactRefreshSeconds no longer blocks a gRPC thread on a 3 s-bounded
store read or gets denied past the bound (R03:3.4).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
AuditWriterActor documented per-row dedup on the UX_ConfigAuditLog_EventId index but flushed one
statement batch: a single duplicate EventId (any at-least-once producer across a singleton
hand-over) threw for all 500 rows (R03:2.6). SaveChanges now detaches the entries a
DbUpdateException names (2601/2627 logged at Debug as the expected duplicate, anything else at
Warning) and re-saves, bounded by the batch size; a failure naming no entry stays a whole-batch
failure.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Every deploy stored the whole gzip'd fleet snapshot in Deployment.ArtifactBlob forever (R03:1.2).
DeploymentRetentionActor (admin singleton, proxy-less) sweeps every Deployments:Retention:
IntervalMinutes (60): terminal rows (Sealed / PartiallyFailed / TimedOut) older than the newest
KeepSealed (50) and not referenced by a NodeDeploymentState started/applied within ReferencedDays
(30) get their blob EMPTIED — never deleted, never NULLed: the column stays NOT NULL and every
reader already treats zero bytes as "no artifact" (#485), so no schema migration and no reader
change. In-flight rows are never touched; MaxRowsPerSweep (200) bounds one sweep.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- ApiKeyOptionsValidator (ValidateOnStart on any driver/admin node): a blank or placeholder
Security:ApiKeys:Pepper fails host start instead of rejecting every key after a green boot (R03:2.3).
- JWT is finished and opt-in by key (R03:2.4): with Security:Jwt:SigningKey set, a Bearer scheme
(JwtBearerAuthenticationHandler over JwtTokenService.TryValidate / BuildValidationParameters, fail
closed) accepts the tokens /auth/token mints; the default scheme is a cookie-or-bearer policy scheme
and the fallback policy names it, so /api/* and every RequireAuthorization endpoint take a browser
session OR a machine token. JwtOptions are validated at start (JwtOptionsValidator). Without a key,
/auth/token is not mapped (404, not the former constructor 500) and no bearer scheme exists.
A direct AddJwtBearer needs Microsoft.AspNetCore.Authentication.JwtBearer in
Directory.Packages.props (not this lane's file) — the handler is a drop-in to swap.
- AuthenticationStateAuditActorAccessor is the registered IAuditActorAccessor: the Blazor circuit's
principal first, HttpContext as the fallback (R03:2.5). HttpAuditActorAccessor shares its resolver.
- CookieAuthenticationStateProvider (unregistered, fire-and-forget ping loop) is deleted (R03:1.5/2.5).
- /auth/login logs the LDAP exception at Warning before answering 503 (R03:2.7).
- Admin operations reply with a fixed message + correlation id instead of ex.Message (R03:4.1).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- NodeManagerHistoryReadRawCapFuzzTests: 10k seeds (env OTOPCUA_PAGING_FUZZ_ITERATIONS)
through a real SDK node manager against wire-faithful fakes of SqlServer / QuestDB /
Gateway that enforce the backend half of the cap; random series with ties, random
windows (empty, on/past a tie, between samples, forward/reverse/end-only, bounds),
N in {0,1,2,small,huge}, cap in {off,3..1000}, tie budget {3,8,65536}. Pages must
concatenate to the cap-off one-shot answer (checked against the series), terminate
within values + 2 (+ one empty step per oversized boundary cluster), never exceed
the cap, never ask the backend for 0 / over-cap; refusals only past the budget.
- HistoryPaging.AdvertiseMaxReturnDataValues writes HistoryServerCapabilities/
MaxReturnDataValues (i=11273), read back through the SDK master node manager.
- MinMaxRawValuesPerRead raised to 3: at 2 a resumed page has no room (value + its
boundary tie + lookahead) and every value cost an empty page + CP (seed 15).
- RawPageLimit rows pin the default-cap arithmetic (9 999 first page, 9 998 resumed).
Fuzz findings recorded as follow-ups: a bounded read whose data fits one page
returns cap + 1 (end bound fitted against N, not the cap; node manager, F2);
the Gateway walk-back's cost on a sparse series under a tiny tie budget is
pre-existing (seed 320: 82 reads uncapped, 93 capped).
Refs #736
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
A failed SDK start (port in use, PKI unwritable, bad OpcUa:* override) was one Error line: the
node stayed /health/ready 200, joined its mesh, could be elected driver Primary and acked deploys
through the Null sink while every client got connection-refused.
- IOpcUaEndpointState / OpcUaEndpointState: the start outcome is a fact the hosted service
maintains (NotStarted / Started / Failed / Stopped + reason).
- OpcUaEndpointHealthCheck ("opcua-endpoint", Ready + Active) is Unhealthy unless Started;
registered by AddOtOpcUaHealth(hasAdmin, hasDriver) on driver-role nodes.
- A driver-only node rethrows the start failure (Program.cs's fatal handler exits non-zero so
the supervisor restarts it); a fused node keeps its AdminUI up but is visibly not ready.
- StopAsync detaches the sinks and then STOPS the SDK (the port closes, clients get
connection-refused rather than a stale, inert endpoint); DisposeAsync is idempotent with it.
- The ServiceLevel calculator / redundancy election live in Cluster + Runtime (not this lane);
the fact is exposed for the Wave 3 follow-up to consume.
Tests: port-in-use ⇒ state Failed, probe Unhealthy, real /health/ready and /health/active 503
(positive control: 200 once Started); driver-only rethrows; StopAsync closes the port and
DisposeAsync twice is a no-op; the probe registration is on both tiers for driver nodes only.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
DriverInstanceDiagnostics.Name carries the string driver-instance id
(what /hosts, telemetry and the artifact key on) and DriverInstanceId
the DB row Guid; the wire record has no slot for the authored display
name. The write-cap test asserts entry order per slot pair instead of
completion order of two concurrent writes.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
DriverHostActor wraps the health publisher it hands its children in a
ChildHealthLedger (last snapshot per driver id, forwarded to the real
publisher), so GetDiagnostics now answers with each child's row id,
authored name, last-published state, device counts (host statuses when
the driver probes hosts, else one device) and last-change time — instead
of Guid.Empty / "Spawned" / 0 / now. Entries are forgotten on StopChild
and operator Restart.
IFleetDiagnosticsClient gains its first consumer: the new
/fleet/diagnostics page (node selector + ?node= query, revision,
running-from-cache notice, per-driver table). FleetDiagnosticsClient
logs the degrade-to-empty path at Warning.
Review ref: 02/U1.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
ReceiveAsync suspended the mailbox for the whole of every handler, so one Test Connect against a
black-holed PLC (60 s clamp) or one large deploy compose stalled every other admin operation
fleet-wide — AdminUI alarm acks/shelves, driver restarts, the next deploy.
- TestDriverConnect runs the probe on the thread pool and pipes a TestDriverConnectResult
straight to the requester (every path yields a result, never Status.Failure).
- StartDeployment serialises only the in-flight check (DB row + an actor-local compose-in-flight
flag, because the Deployment row is written at the END of the compose); the snapshot read,
validation, gzip and row write run off-actor and return as DeployComposed /
DeployComposeFailed, where the coordinator is told and the requester answered.
Tests: an alarm ack completes in < 1 s while a 60 s probe is pending and while a compose is
parked on a gated DbContext (positive controls: both still complete afterwards); a second
StartDeployment during an in-flight compose is refused.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
CallTimeoutSeconds was bound, defaulted and documented but consumed
nowhere. TelemetryDialSupervisor now arms a per-generation deadline on
every dial: a dial that has delivered no event by the deadline is
cancelled and treated as a transport failure (reconnect with a failure
streak). A live node replays its snapshots on subscribe, so this bites
only a dial that never connects (a black-holed host, where the socket
connect has no timeout). An established stream is deliberately not
bounded — the wire has no heartbeat, keepalive pings own its liveness.
Review ref: 02/U2.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
StartDeployment read the whole config DB twice with no transaction — DraftSnapshotFactory for
the validator, then ConfigComposer for the artifact, plus clusters/nodes and Script.SourceCode
again in between. An AdminUI save landing between the two shipped an artifact the DraftValidator
never saw.
- DraftSnapshot carries ServerClusters, ClusterNodes and ApiKeys; DraftSnapshotFactory reads
every table inside one SNAPSHOT transaction on a relational provider (no-op on InMemory or
under an ambient transaction).
- ConfigComposer.Compose(DraftSnapshot) is pure over the snapshot (ordinal-sorted keys);
SnapshotAndFlattenAsync(db) delegates to it, byte-identical.
- AdminOperationsActor validates topology, estimates compile cost and composes the artifact
from the one snapshot it holds.
- Migration EnableSnapshotIsolation: ALTER DATABASE CURRENT SET ALLOW_SNAPSHOT_ISOLATION ON
(suppressTransaction; no model change).
Tests: the SQL Server snapshot read ignores a row committed mid-read (positive control: a later
read sees it); an edit injected between the Equipment and Tags reads is absent from the shipped
artifact and rejected by the next deploy; Compose(snapshot) == SnapshotAndFlattenAsync(db).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- DriverHostActor.StopChild removes the driver's _lastReprovisionUtc stamp
(and its deferred-reprovision timer): the map grew per removed id and a
re-added driver inherited a backoff it never earned (02/L2).
- Stale now registers explicit drops for AttributeValueBatchPublished,
AttributeAlarmPublished, ArtifactForApply and ApplyWedged instead of
letting them hit Unhandled (02/H5).
- DbHealthProbeActor runs its SELECT on the thread pool and pipes the
outcome back (single-flight), with a 2 s command timeout on relational
providers, so a hung SQL Server no longer pins the actor thread and the
cached GetStatus answer stays reachable (02/P7).
- TelemetryLocalHub eviction tombstones are timestamped and swept past a
10 min TTL on the next Evict; a TimeProvider ctor is the test seam (02/L4).
- Tests: ValuePublishes.Single helper replaces the per-value
AttributeValuePublished in CalculationDependencyFlowTests (02/U4, test
half; the production receive is owned by Lane E).
- R05/8 audit: PollGroupEngine.Unsubscribe has no production caller; every
driver already uses UnsubscribeAsync (no change).
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
WriteAttribute was a ReceiveAsync, suspending the child's mailbox for the
whole device round-trip (up to 5 s): writes serialised, the second of two
operator writes to a slow device missed the host's 8 s Ask and was
reverted although the device took it, and every data change queued behind
the write. Writes are now a synchronous Receive that launches the device
call on the thread pool (sender captured, reply from the task, slot freed
by a self-sent WriteCompleted) under a per-driver cap (default 4; excess
writes queue in order). Queued writes fail fast on every exit from
Connected. Subscribe/unsubscribe stay ReceiveAsync.
Review ref: 02/P2.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review R08:1.1: an uncapped HistoryRead-Raw (NumValuesPerNode = 0) read a
tag's whole window into memory on every backend (SQL TOP(int.MaxValue),
QuestDB LIMIT, Gateway max_values = int.MaxValue + a second snapshot copy).
New ServerHistorian:MaxRawValuesPerRead (default 10 000; <= 0 unbounded with
a Validate() warning; 1 raised to 2). Each backend data source takes it,
reports it as IHistorianDataSource.MaxRawValuesPerRead (the Gateway: min
with a strict gateway's learned ceiling), so the node manager's existing
RawPageLimit paging serves N = 0 and N > cap as pages of at most the cap
with a continuation point. Independently, a backend raw read asking for 0
or more than the cap reads cap + 1 rows and is REFUSED
(HistoryReadLimitExceededException => BadTooManyOperations) when the window
holds more - never answered truncated, since the server's paging reads a
short answer as the last page. Bound through the Host registry, the
SqlServer/QuestDb providers and GatewayHistorian.CreateDataSource.
Tests: options (default/binding/warnings), provider binding, Gateway unit +
through a real SDK node manager (uncapped -> 999/998/998/605 + CPs), and a
conformance case for the SQL backends.
Refs #736
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The Reconnecting retry timer started a new ShutdownAsync + InitializeAsync
pair on every tick whether or not the previous attempt had finished; the
generation token only discarded the older RESULT, so with an Initialize
bound longer than the cadence the teardown ran underneath an attempt still
inside the driver (the dispose-before-overwrite race). An _initInFlight
gate, cleared by the piped InitializeSucceeded/Failed, skips a tick while
an attempt is running; the pre-reconnect teardown is bounded by the
shutdown timeout; PostStop cancels an in-flight attempt before shutting
the driver down.
Review ref: 02/H2.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
DriverInstanceActor.PostStop now bounds IDriver.ShutdownAsync by a
shutdownTimeout (default 5 s, logged on overrun) and then disposes the
driver (IAsyncDisposable preferred, else IDisposable) — but only on a
real stop. PreRestart flags the restart so the PostStop it triggers
skips the dispose, because a supervision restart re-uses the same
driver object (Props closure) and the host re-initialises it.
DiscoveryDriverBrowser disposes its throwaway capture/probe driver
after the bounded shutdown, including after a failed or timed-out one.
Review refs: 02/L1, 02/P4, 06/G-1 (host context), 07/L2.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
A Deployment row left in Dispatching with no NodeDeploymentState rows (HandleDispatch threw
before its SaveChanges, or the process died between the AdminOps row write and the
coordinator's Tell) was recovered with an empty expected set; the deadline then hit the
0 == 0 "race" branch and returned without touching the row, so every later StartDeployment was
refused AnotherDeploymentInFlight until someone hand-edited SQL.
- PreStart: a recovered row with no expected-ack rows is a lost dispatch. Inside the apply
deadline it is re-driven through HandleDispatch (bounded: CreatedAtUtc never moves); past it
the row is closed TimedOut with a reason naming the condition.
- HandleDispatch is idempotent on a re-dispatch (rows already seeded ARE the expected set).
- HandleDeadline takes the race branch only when _expectedAcks.Count > 0.
- The PreStart recovery read projects ArtifactBlob away (R03:3.5).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The #687/F4 crowding re-read doubled its room up to 8 re-reads with no cap, so a pathological single-ms
cluster on a streaming gateway was streamed several times over for one page. The room is now capped at
ServerHistorian:MaxTieClusterOverfetch (plumbed into GatewayHistorianDataSource as maxPreWindowRows,
default 65536); past it, or once a read has already seen more, the read is BadTooManyOperations with
PreWindowRows set.
Item 1: IHistorianDataSource.MaxRawValuesPerRead (Gateway: learned ceiling - 16 = L; 0 elsewhere, guard-tested).
RawPageLimit plans every raw page on top of #692's lookahead: with L > 0 the read (page P + boundary ties k
+ lookahead) is <= L, P = min(N, L - k - 1) (uncapped: L - k - 1); no room => straight to the tie-cluster
over-fetch (never an uncapped or underflowing request). One re-plan + re-read when the read taught a new
limit or was refused for crowding pre-window rows (HistoryReadLimitExceededException.PreWindowRows): the
page shrinks by them. Over-fetch bound min(MaxTieClusterOverfetch, L - 1); slice size never 0.
Review fixes: F1 a window starting at/before the epoch counts AVEVA's synthetic row as pre-window (a
ceiling-truncated read was reported complete); F2 the page path disables ReverseRawReader's own re-run
(cold end-only ceiling 20/N=3000 cost 3,023 reads, now <= 40); F3 crowded look-back re-read around, not
refused; F4 the #687 crowding re-read room at least doubles (up to 8 re-reads), so a cluster of hundreds
just before a resume point no longer fails on a default gateway; a crowding refusal that leaves no room
is BadTooManyOperations, never a tick-by-tick cursor.
Item 2: ServerHistorian:EventReadHorizon (default 00:00:00 = exact, unchanged). When set, every fresh
HistoryReadEvents window's earlier bound is raised to now - horizon (both directions, all backends).
Lossy by design; documented.
Item 3 (native TimeAverage/Total re-check) stays blocked on historiangw#27.
A time inside a gap of a bursty tag costs a 64-row jump read landing in the next
burst plus a bisection back across the gap (~6 reads), so the 64 + 4/time cap
refused a 1 h/day 1 Hz tag asked at noon on 30/100/300 consecutive days
(186/606/1,806 reads) that AVEVA's own AtTime served. The cap is a runaway guard
(HistoryReadDeadline bounds wall clock): now 64 + 16 per distinct time.
Regression tests for x30/x100/x300 (each <= 7 reads a time, values checked).
Docs: DateTime-hinted nodes are stepped but keep the gateway's Double (the
Gateway never historizes DateTime nodes; the SQL backends return DateTime).
H1: the walk-back reset its width after a full segment and discarded a full
whole-remainder read, so a dense tag that stopped reporting weeks ago gave up
after 65 reads (BadTooManyOperations). It now gallops (x4, whole-remainder read
after two empties) and then BISECTS once any stored row is known below — a full
segment's (incl. a full whole-remainder read's) last row, or an earlier window
read's last row: (mid, hi] empty => hi = mid; full => known = its last row;
complete with rows => done; one density-aimed segment after a full read until an
aimed segment comes back empty; below 2 ms the whole stretch from the known row
(never a 100 ns step). Stopped 1 h..365 d with 5 y history: 6-18 reads at
1/min, 21-25 at 1 Hz, 25-31 at 10 Hz.
M1: a jump read asks for 64 rows (1,000 times over 10 y: ~66k rows, was ~1M).
L1: per-request read cap 64 + 4 x distinct times with a clear message; a strict
ceiling below 4 rows is refused naming the ceiling.
L2: DateTime hints are stepped. L3: under an integer hint a NaN/inf or
out-of-Int64-range value is unusable, never saturated Good.
L5: conformance Int64 AtTime asserts the type. L4: docs name the backend
differences. Nit: the window read end is clamped to now + 1 day.
Reviewer harnesses (Scratch693Fuzz/Cost, not committed): 0 differential diffs
over 2,400 cases.
Joseph Doherty