The Modbus lane (#741 U3) gave the driver String[] and BitInRegister[] holding-register array encoders, and
moved address resolution to ModbusTagAddressResolution (keys first, a bit region coerced to Bool, strict key
reads), but DriverArrayWriteSupport still listed both array shapes read-only (follow-up F1), so a client
write was BadNotWritable before it reached the driver.
Modbus row now mirrors the driver end to end:
- Keys shape only (region + address both present). A string-only addressString tag is resolved by the
driver with its own family through a parser Core.Abstractions cannot reference, so it stays read-only
(pinned: the conservative direction).
- Strict reads as the resolver: address an integer 0-65535; a present region / dataType / byteOrder /
stringByteOrder string must name a member.
- Coils: any dataType (coerced to Bool), <= 1968 coils (FC15).
- HoldingRegisters (FC16, never Bool): numeric, String ((stringLength+1)/2 registers, stringLength >= 1),
BitInRegister (1 register, bitIndex 0-15, span ending <= 65535), all within maxRegistersPerWrite.
FC23 is opt-in and never changes the answer.
- Elements convert (ScadaBridge int[] -> Int16 stays writable); only a String / DateTime node over a
numeric or bit element is left out, since such an element never converts.
Tests: the parity test's PendingF1 exemption and its guard are gone; the parity matrix is now region x
element type x node type against the real WriteAsync with a spec-conformant device (FC15 > 1968 refused),
plus shape edges at every cap boundary (numeric, String, BitInRegister, coil), strict key reads, FC23
on/off, string-only, the ScadaBridge int[] shape and U3 wire bytes. Core.Abstractions row tests and a
composer test (String[] and BitInRegister[] writable; over-cap, no-stringLength and string-only read-only).
Docs: table XML doc, Uns.md, modbus-addressing.md, drivers/Modbus.md, data-plane rules.
Refs #741
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
N3: a keyless blob carrying the addressResolved marker was served from its
string by the driver but saved as HR[0] by the editor model / CSV path. The
precedence rule now lives in ONE pure resolver in Driver.Modbus.Contracts
(ModbusTagAddressResolution.Classify/Resolve): keys when region+address are
both present, else a string-only addressString resolved with the driver's
family (refused when it cannot parse), else refused. The marker never decides.
Consumers that now ask it instead of restating it: the driver's tag reader
(ModbusTagDefinitionFactory.FromTagConfig), the deploy gate (Inspect /
DescribeRejection), the editor model (AddressShape / IsStringOnly /
HasKeyAddress / ResolveWith, ToJson stamping, Validate), the CSV export cells
and the editor hint. The model writes keys only when it loaded them or a field
was authored - never defaults it did not resolve (also closes two divergences
the new matrix found: malformed-string+marker and non-string addressString were
saved as HR[0]).
One table-driven test (ModbusAddressAuthorityAgreementTests) runs 15 persisted
shapes through driver, deploy gate, model load/save/Validate and the real CSV
export->import under all four families; breaking the CSV export or reverting
the model to the marker rule fails it.
Minors: m1 CSV address cells keep the string+marker while it still describes
them (lossless unchanged round trip), drop it when contradicted, and the review
grid reports the drop; m2 the pre-lane test uses the real export residual; m3
unmarked string-only + typed-cell case added; m4 stale docs/summary and the
unused factory parameter removed.
Refs #741
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
2: only a PUBACKed (QoS >= 1) publish advances LastSuccessfulRead; a QoS 0 Sparkplug
command completing means only that the bytes left the local socket.
3: a no-verdict SUBSCRIBE caused by the caller's own cancellation now throws
OperationCanceledException (refs still Bad + pending), so the breaker no longer
counts a pipeline timeout as a failure.
4: the rebirth wait is the window capped at 15 s, below the AdminUI 20 s per-call
timeout.
6: a 10-topic test pins the 8-in-flight publish bound. 7: partial-level wildcards and
$share filters are pinned never to confirm. 8: the intended init-vs-shutdown
supersede is commented at the re-check.
Refs #744
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
One stray duplicate row used to fail every tag in a key-value group or none of them,
depending on whether an unrelated key was absent. Duplicated keys are now refused
per key: only that key's tags publish BadConfigurationError and the rest of the group
reads normally. The key-value cap reads up to 2x the distinct keys (+1 to detect
excess), so a few stray duplicates are attributed per key and only a gross excess
(history/events table) fails the whole group. Duplicated-key refusals count as
answered refusals for health. The row-cap, duplicate-key and multi-row warnings now
use the per-plan limiter (m-3); the reader-wide limiter is removed. Unit tests pin
both the all-keys-present and absent-key cases with Good controls; live SQL Server
gate added (23/23).
Refs #744
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
N2: pre-#741 rows carry region/address beside an addressString with no marker
(the old driver never read the string, the old model preserved it as an
unknown key). The r1 rule resolved those from the string - a silent remap or
BadNodeIdUnknown on upgrade. The driver now loads from region+address whenever
both are present (classic, editor-pinned, pre-lane); only a blob with NO keys
is string-only and resolved through the parser. This is safe because the model
no longer stamps keys onto a string-only tag. The model distinguishes
string-only / legacy-string-beside-keys / pinned, and the editor hint says so.
Test matrix over every persisted shape (keys only, pre-lane keys+string,
pre-lane stamped HR/0+string, string only, marker+keys, marker no keys) under
every family; the pre-lane row also through a model save and a CSV
export->import.
n5: a typed CSV address cell is an authored address - it drops addressString
and the marker (new CsvTypedColumn.ClearsKeys) instead of being discarded.
n6: Validate skips the region/dataType/byteOrder enum checks on a string-only
tag (the driver ignores them). n7: docs/v2/modbus-addressing.md restates the
key-presence rule.
Refs #741
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The Wave-2 lanes gave AbCip, AbLegacy and TwinCAT real whole-array write
encoders (review 07 U10/U9), but the capability table still listed all three
as scalar-only, so every array node materialised read-only and a client write
was BadNotWritable before it reached the driver.
Rows added, each listing only what WriteAsync encodes end to end:
- AbCip: SInt/USInt/Int/UInt/DInt/UDInt/Real/Dt/LInt/ULInt/LReal, declared
length, not a safetyTag. BOOL/STRING/Structure arrays stay read-only.
- AbLegacy: Int/AnalogInt on N/B/A, Long on L, Float on F (file-width match),
plain <file><n>:<word> address, <= 256 elements; not /bit, .sub, indirect,
S:/I:/O:, nor native PLC-5.
- TwinCAT: every codec element type, declared length; not a BOOL bit-in-word
path.
Elements convert in all three encoders, so the rows never key on an exact
node DataType (ScadaBridge int[] -> Int16 stays writable); they leave out
only node types whose elements never convert (DateTime/String over numeric,
Boolean on AbLegacy, STRING under DateTime on TwinCAT: culture-dependent).
Tests: driver-level parity for each driver (every element type x node type
plus shape cases, against the real write path), the AbLegacy parity tests
un-skipped, Core.Abstractions row tests, a composer test (three drivers
writable, refused shapes read-only), and the existing AbCip negative cases
moved to a BOOL array. Docs: Uns.md table, per-driver docs, data-plane rules.
Refs #737#738#739
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
N1: the m2 precedence trusted region/address whenever the keys were present,
but the AdminUI model stamps defaulted HoldingRegisters/0 on every save, so a
string-only tag (CSV import, or any editor toggle) silently became HR[0] -
reads AND writes. The resolved keys are now authoritative only when the editor
marked them ("addressResolved": true, written by Resolve); an unmarked
addressString is always resolved through the parser with the driver's family.
The model no longer stamps address fields for a string-only tag, CSV export
leaves its address cells blank, and the editor hint distinguishes string-only
from pinned tags. Tests: CSV import and editor save of a string-only tag, a
stamped-default blob, and the pinned positive control across families.
n1: docs/v2/modbus-addressing.md updated (FC23 opt-in, maxReadGap semantics,
coalescing scope, addressString/family rules, coil coercion, refusals).
n2: form round-trip tests for UseFC23ForMultiRegisterWrites /
DisableReadCoalescing. n4: the editor keeps unresolved address text when the
parent echoes back its own emit. n3: noted, no change.
Refs #741
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
I-1: a PUT/GET-disabled CPU (every read denied) stayed Faulted with
LastSuccessfulRead frozen, so the comms-loss watchdog reconnected and
re-subscribed it every 90 s forever. Both access-denied arms now also record
an answer: Faulted is kept (worst state wins), the stamp advances, and the
sticky write fault still clears only on a real success. Pinned end to end in
Runtime.Tests: the real S7Driver under the real DriverInstanceActor watchdog
is not reconnected across 2x the threshold; the control (every read timing
out) is.
Minors:
- m-1: injectable pass clock (UtcNow); "LastSuccessfulRead advanced" is now
asserted strictly, including the write-refusal case.
- m-2: the editor uses S7TagShapeRules.IsByteAnchored instead of its own list.
- m-3: a release that races an init publish also cancels the probe loop the
publish started after the release took its snapshot.
- m-4: a retained poll failing during a lifecycle operation writes no health.
- m-5: a multi-var chunk that fell back memoises its members refused alone.
- m-7: access-denied needs S7.Net's exact wording and no transport fault in
the chain.
- m-8: sync teardown clears the write-side Degraded owner; a sync Dispose
racing a running shutdown is tested.
Refs #742
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Key-value and where-pair plans had no row limit, so a selector pointed at a history
or events table materialised every matching row on every poll. ISqlDialect gains
RowLimitPrefix/RowLimitSuffix(n) (SQL Server TOP (n), Postgres LIMIT n, Oracle FETCH
FIRST n ROWS ONLY); plans carry a RowCap (distinct keys, or 1) and ask for RowCap+1
rows; the reader also stops client-side at RowCap+1. Getting the extra row proves the
selector ambiguous: the group publishes BadConfigurationError with a rate-limited
warning. The reader reports those answered refusals so the driver counts them as
answer evidence, and any refusal-only poll now restores Healthy after a comms fault.
Refs #744
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
WriteAsync published one awaited QoS 1 round-trip per request, so 50 writes to a
stalled broker cost 50 publish deadlines. Different topics now publish concurrently
(at most 8 in flight); same-topic writes stay in request order. A caller cancel still
fills every slot. A successful publish advances LastSuccessfulRead. MiniBroker now
frames packets properly and records/PUBACKs PUBLISHes.
Refs #744
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The ingestor refuses array metrics and unmapped types, yet DescribeMetric still stated
their binding tuple, so a browse-commit produced a tag stuck at
BadWaitingForInitialData forever. Those leaves now state no address (the commit
refuses them) and say why in the descriptor line.
Refs #744
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The authored window was persisted and consumed by nothing. It now (clamped 1-120 s)
extends the Sparkplug OpenBudget, bounds a passive UntilStable-style settle on open
(return after a 1 s quiet spell in NBIRTH/DBIRTH traffic), and bounds how long an
operator rebirth request waits for the births it asked for before returning, so the
re-rendered tree carries them. A caller deadline during the wait still reports the
publish.
Refs #744
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- The background probe reported the host Stopped whenever a read pass held
_gate past the probe deadline: contention is the connection being used,
not lost. A contended tick now gives no verdict (counted for tests); a
real probe failure still reports Stopped. A tick that races a teardown no
longer reopens a connection on an uninitialised driver.
- S7DriverProbe deserialised S7DriverOptions directly, whose Timeout is a
TimeSpan no config names, so "timeoutMs" was ignored and every handshake
waited the whole Test Connect budget. The handshake and test read are now
bounded by timeoutMs, capped by the caller's timeout.
Refs #742
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
A span's block is the union of its members' ranges, so one member that runs
past the DB's real end (or over-declares a STRING) made the CPU refuse the
whole block, and every neighbour within it was Bad on every poll. The
planner's stated invariant (a grouped read cannot fault where per-tag reads
would succeed) was false; the multi-var path already fell back, spans did not.
- A refused multi-member span hands its members back; they are read one at a
time for that pass, bypassing the multi-var pass.
- _noGroup remembers the members refused on their own (or every member when
none was) and keeps them out of later plans, so the rest group again. It is
cleared by every initialize publish. Non-refusal span failures are
unchanged (every member stamped).
- The once-per-tag refusal Warning memo (R07:H4) is now also cleared by the
publish, and shutdown resets the write-side Degraded owner, as documented.
- Live gate: MW10 + MD254 (2 bytes past the 256-byte M area) merged into one
refused block; MW10 keeps reading 4097 on every pass.
Refs #742
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
No read or write loop checked _plcDead between items: on a frozen-but-
established peer the first op blew its deadline and marked the handle dead,
and every remaining span / tag / write still went to the wire and could wait
its own deadline under _gate (a 20-span pass ~100 s), with Degraded and the
comms-loss watchdog's clock deferred until the pass ended.
- The span loop, the multi-var chunk loop, the per-tag loop and the write
loop stamp BadCommunicationError without I/O once _plcDead is set.
- A multi-var chunk that dies on a transport loss (socket / IO / timeout /
ConnectionError) marks the handle dead instead of falling back to one
doomed per-tag read per member. The framing class is deliberately excluded
there: snap7/S7.Net report a multi-var with one refused item as
ReadData(WrongNumberReceivedBytes) over a healthy stream.
- Live gate via an in-test freezable TCP proxy (counts request frames, not
seconds). Mutation shows S7.Net 0.20 already self-closes on a cancelled op,
so the gate pins the end-to-end property; the driver guard itself is proven
by the unit tests (an op that ignores its token keeps the socket open).
Refs #742
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
A failed SUBSCRIBE (refusal or no broker verdict) used to drop its topic from every
retry set while SubscribeAsync returned a live handle, leaving tags dark behind a
Healthy connection until a redeploy. Failed topics now join a pending set retried by
the next SubscribeAsync and by every reconnect (not counted toward the reconnect's
total-failure verdict, so an ACL-denied topic cannot reconnect-loop). A no-verdict
subscribe degrades its refs then throws so the actor's reconcile re-subscribes. A
subscribe with no transport attached is recorded pending, never established (M-11).
MiniBroker now answers SUBSCRIBE.
Refs #744
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
A deterministic PLC refusal (object does not exist, address out of range,
type inconsistent...) degraded driver health, and an all-refused driver (a
wrong DB number) sat Degraded with LastSuccessfulRead frozen, so the 90 s
comms-loss watchdog reconnected it forever against a reachable PLC.
- IsRefusal / IsAccessDenied classify over the whole exception chain. Live
probe on the snap7 fixture: S7.Net's READ paths raise the item error as a
BARE Exception ("Received error from PLC: ..."), its write paths as
PlcException(WriteData). The old PlcException-only match missed the read
shape entirely, so a real read access-denied was BadCommunicationError +
Degraded instead of the Faulted PUT/GET alert; the Test Connect probe had
the same blind spot.
- A refusal (read or write) is BadDeviceFailure with a once-per-tag Warning,
no health write and no reopen; it counts as answered, so it advances
LastSuccessfulRead and restores Healthy after a comms fault. An undecodable
buffer (S7ValueRangeException) also counts as answered.
- A write batch the CPU answered without a failure advances
LastSuccessfulRead and retires a Degraded the write side itself published.
- Tests that used ReadData as their generic "device fault" now use SendData
so their Degraded leg is still exercised.
Refs #742
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review m-4. The status-code paragraph still traced the value path
through the deleted per-value AttributeValuePublished. It now reads
AttributeValueBatchPublished -> AttributeValueBatchUpdate ->
IOpcUaAddressSpaceSink.WriteValues, which is what OpcUaPublishActor
calls.
Refs #731#734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review m-2. A Calculation driver's own maxAllocBytesPerRun /
quarantineAfterConsecutiveTimeouts used to be clamped silently by
ScriptRunGuard. ResolveScripting now sanitizes the layered options
itself and returns the adjustments, and the factory logs one Warning
per clamped value, naming the driver. docs/ScriptEditor.md "Runtime
safety" gains the key table: defaults, the 1 MB allocation floor, the
1000 caps, NaN and disable semantics, and the per-driver override.
docs/Configuration.md has no Scripting section, so these keys have no
other home.
Refs #731#734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The init-time tag guards THREW: one mis-authored tag (DB1.DBZ0, DB1.DBW0 typed
Float64, T5 typed Int32, a writable Bool array) faulted the whole driver and the
actor retried the same init every 10 s forever, darkening every tag on the PLC.
Neither the editor nor the deploy gate checked any of it.
- S7AddressParser moves to S7.Contracts (same namespace) and a new
S7TagShapeRules there is the single authority on servable shapes.
- S7TagDefinitionFactory.FromTagConfig applies it, so the AdminUI editor, the
deploy gate (Inspect, with the specific reason) and the driver init refuse
the same tags. Init now skips a refused tag (ISkippedTagReporter) instead of
throwing; the throwing RejectUnsupportedTagConfigs is gone.
- New rules on top of the old guards: a narrow type at the wrong access width
(every read failed) and a Bool array not starting at bit 0 (U4: DB1.DBX0.3
silently read bit 0 of bytes 0..n).
- AdminUI test fixtures that used addresses the driver could never serve
(DB1.DBS0, a browse "SomeAddress", Int32/Float32 on DBW) now use real ones.
Refs #742
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
It had no production sender: every driver data change is already a
batch, and the comms-loss fan was converted earlier. DependencyMuxActor
drops its per-value receive, DriverInstanceActor drops the record, and
DependencyMuxActorTests' 13 sends move to the ValuePublishes.Single
one-item batch (same semantics through the batch path). XML docs and
the living docs (DriverLifecycle, VirtualTags, ScriptEditor) are
reworded; historical plan docs are left as written.
Refs #731#734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
ScriptContext.CancellationToken is backed by an AsyncLocal that
ScriptEvaluator.RunAsync scopes around the compiled delegate, so it is
exactly the token ScriptRunGuard cancels on timeout. A polling script
can now end its own abandoned run and free the slot. AsyncLocal rather
than a field because the scripted-alarm engine reuses one context per
alarm: an orphan keeps seeing its own cancelled token. The scoping
entry point is internal (IVT to Core.Scripting) so a script cannot
replace its own token.
Editor parity: ScriptAnalysisService dot-completion now walks the
receiver's base chain (instance members only, stopping before
System.Object). Before this, a member declared only on ScriptContext
compiled and published but never completed. DiagnoseTests pins the
member as allow-listed; CompletionTests pins the completion.
Refs #731#734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Ruling refinement on I1. Only timeouts are strikes, so a never-returning
while(true), which times out once and is then only refused, was never
quarantined, and that defeated E1. A refusal now quarantines immediately
when the orphan it was refused behind has been executing for longer than
ScriptingOptions.StuckOrphanQuarantineFactor x the script's timeout
(default 10, so 20 s at the 2 s default; 0 or less disables it). It uses
the same quarantine path: one Error, otopcua.script.quarantined with
reason=stuck, Bad, lifted on source change or redeploy. A young orphan
(a short overrun) is still a plain refusal.
The guard stamps the run's start (Stopwatch timestamp) when the pool
picks it up and clears it when the run really finishes.
Tests: the 30-trigger while(true) regression (paced) now ends
quarantined with reason "stuck", still with exactly one orphan and a
healthy sibling. Positive control: a 2x overrun is refused, not
quarantined, and evaluates again once the orphan completes. A factor of
0 disables the rule. Docs (ScriptEditor "Runtime safety", VirtualTags)
and the seam remarks are updated.
Refs #734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- ScriptLogRateLimiter.LimitPerWindow. The ScriptLogTopicSink and
VirtualTagActor summaries now print the configured limit, not
DefaultLimitPerWindow. Tests assert "limit N/window" for a non-default
N.
- CalculationDriver.Dispose cancels, then takes _evalGate (bounded 2 s)
and disposes the gate and _evalCts. The pump and timer ticks enter the
gate through TryEnterEvalGateAsync, which stops cleanly once the
driver is disposed. Test: Dispose with a 6 s runaway run in flight
returns promptly, disposes both, and a late trigger is harmless.
- ScriptedAlarmEngine captures _shutdownToken once and disposes
_shutdownCts in Dispose when every drained task has completed. A task
stuck past the grace leaves the CTS to the GC. Lifecycle tests assert
the disposal.
- DraftValidator: CalculationNoEffectiveTrigger is skipped for a
change-triggered tag whose scriptId does not resolve. That case is
already CalculationScriptMissing or CalculationScriptNotFound, and
there are no reads to count. Both existing tests now assert the
absence (red before the change).
- docs/VirtualTags.md: System.Threading is denied as a whole namespace,
with an allow-list; the type list is now accurate (Unsafe,
RuntimeHelpers, and the ForbiddenMembers on System.Type); the resource
limits section covers the run guard and the allocation budget.
docs/ScriptEditor.md: accurate deny-list and a new "Runtime safety"
section covering timeout, refusal, quarantine after 3 timeouts,
allocation budget, lifting, and rate-limit summaries that appear only
on the source's next entry after a burst then quiet.
Refs #734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The hand-rolled bearer handler is replaced by AddJwtBearer (Microsoft.AspNetCore.Authentication.JwtBearer
10.0.7, pinned on the same servicing line as the other AspNetCore packages). Validation parameters come
from JwtTokenService.BuildValidationParameters, which now pins ValidAlgorithms = HS256 for both the
bearer scheme and TryValidate; MapInboundClaims is off so the minted Role/zb:username keys resolve.
/auth/token requires the cookie scheme, so a bearer can no longer mint a fresh token for itself.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
BoundedApiKeyStore's negative cache was keyed by key id, so a flood of DISTINCT well-formed
otua_<random>_x credentials missed it every time and cost one SQL query per attempt under the
SDK's SessionManager-wide lock, no credential needed (R03:3.3).
- IApiKeyStore.ListKeyIdsAsync (default null = unsupported; ConfigDbApiKeyStore lists every
ApiKeyId): the bounded store keeps a known-id set refreshed every StoreCacheSeconds
(single-flight, failed refresh keeps the previous set) and refuses an unknown id without a store
read ({outcome=unknown_id}).
- A process-wide miss budget (token bucket, Security:ApiKeys:MissBurst=100 /
MissRefillPerSecond=20, ApiKeyStoreGuardOptions) bounds lookups that reach the store and miss
while no set is available; a hit refunds its token. Per-endpoint accounting needs the
OpcUaServer impersonation seam to carry the client address — follow-up.
- docs/security.md: the new knobs, and the known-cluster gate's documented fail-open window at
central boot (R03:4.2) plus the async PSK refresh (R03:3.4).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The #687/F4 crowding re-read doubled its room up to 8 re-reads with no cap, so a pathological single-ms
cluster on a streaming gateway was streamed several times over for one page. The room is now capped at
ServerHistorian:MaxTieClusterOverfetch (plumbed into GatewayHistorianDataSource as maxPreWindowRows,
default 65536); past it, or once a read has already seen more, the read is BadTooManyOperations with
PreWindowRows set.
Item 1: IHistorianDataSource.MaxRawValuesPerRead (Gateway: learned ceiling - 16 = L; 0 elsewhere, guard-tested).
RawPageLimit plans every raw page on top of #692's lookahead: with L > 0 the read (page P + boundary ties k
+ lookahead) is <= L, P = min(N, L - k - 1) (uncapped: L - k - 1); no room => straight to the tie-cluster
over-fetch (never an uncapped or underflowing request). One re-plan + re-read when the read taught a new
limit or was refused for crowding pre-window rows (HistoryReadLimitExceededException.PreWindowRows): the
page shrinks by them. Over-fetch bound min(MaxTieClusterOverfetch, L - 1); slice size never 0.
Review fixes: F1 a window starting at/before the epoch counts AVEVA's synthetic row as pre-window (a
ceiling-truncated read was reported complete); F2 the page path disables ReverseRawReader's own re-run
(cold end-only ceiling 20/N=3000 cost 3,023 reads, now <= 40); F3 crowded look-back re-read around, not
refused; F4 the #687 crowding re-read room at least doubles (up to 8 re-reads), so a cluster of hundreds
just before a resume point no longer fails on a default gateway; a crowding refusal that leaves no room
is BadTooManyOperations, never a tick-by-tick cursor.
Item 2: ServerHistorian:EventReadHorizon (default 00:00:00 = exact, unchanged). When set, every fresh
HistoryReadEvents window's earlier bound is raised to now - horizon (both directions, all backends).
Lossy by design; documented.
Item 3 (native TimeAverage/Total re-check) stays blocked on historiangw#27.
A time inside a gap of a bursty tag costs a 64-row jump read landing in the next
burst plus a bisection back across the gap (~6 reads), so the 64 + 4/time cap
refused a 1 h/day 1 Hz tag asked at noon on 30/100/300 consecutive days
(186/606/1,806 reads) that AVEVA's own AtTime served. The cap is a runaway guard
(HistoryReadDeadline bounds wall clock): now 64 + 16 per distinct time.
Regression tests for x30/x100/x300 (each <= 7 reads a time, values checked).
Docs: DateTime-hinted nodes are stepped but keep the gateway's Double (the
Gateway never historizes DateTime nodes; the SQL backends return DateTime).
H1: the walk-back reset its width after a full segment and discarded a full
whole-remainder read, so a dense tag that stopped reporting weeks ago gave up
after 65 reads (BadTooManyOperations). It now gallops (x4, whole-remainder read
after two empties) and then BISECTS once any stored row is known below — a full
segment's (incl. a full whole-remainder read's) last row, or an earlier window
read's last row: (mid, hi] empty => hi = mid; full => known = its last row;
complete with rows => done; one density-aimed segment after a full read until an
aimed segment comes back empty; below 2 ms the whole stretch from the known row
(never a 100 ns step). Stopped 1 h..365 d with 5 y history: 6-18 reads at
1/min, 21-25 at 1 Hz, 25-31 at 10 Hz.
M1: a jump read asks for 64 rows (1,000 times over 10 y: ~66k rows, was ~1M).
L1: per-request read cap 64 + 4 x distinct times with a clear message; a strict
ceiling below 4 rows is refused naming the ceiling.
L2: DateTime hints are stepped. L3: under an integer hint a NaN/inf or
out-of-Int64-range value is unusable, never saturated Good.
L5: conformance Int64 AtTime asserts the type. L4: docs name the backend
differences. Nit: the window read end is clamped to now + 1 day.
Reviewer harnesses (Scratch693Fuzz/Cost, not committed): 0 differential diffs
over 2,400 cases.
F1 (pre-existing): SliceTieCluster continued a forward read while next <= endUtc, but the
forward window is end-exclusive; an oversized cluster one step below EndTime resumed AT
EndTime, the resume became a start == end point read and returned the EndTime samples as
data (microsecond: EndTime 8.1 us truncated to 8.0). Continue only while next is strictly
before EndTime truncated to the backend resolution.
Tests: A_tie_cluster_just_below_the_end_time_never_resumes_onto_the_end_time (native +
microsecond; both fail on the old rule) and
A_cursor_inside_a_tie_cluster_over_fetches_without_a_normal_read_first (pins the
WithinTieCluster shortcut by per-page read count; fails with the shortcut disabled).
Docs: the oversized-cluster exception to 'every page full / no lookahead => last' (F2),
and the N + skip + 1 strict-ceiling edge note referencing #694 (F4).
Review follow-up: on the Gateway every value is a Double (AVEVA stores a Boolean
as a 0/1 Int2), so the raw-evaluated AtTime interpolated a Boolean node to 0.5.
- IHistorianDataSource gains a default-interface overload
ReadAtTimeAsync(fullReference, timestamps, TypeCode? valueType, ct) that
forwards to the existing one (SqlServer/QuestDB/Null/fakes unchanged).
- OtOpcUaNodeManager.HistoryReadAtTime passes the node's scalar DataType
(AtTimeValueType: built-in type -> TypeCode; arrays/other types => null);
Raw and UNS NodeIds alike.
- GatewayAtTimeReader converts each raw row to the hinted kind BEFORE the shared
AtTimeEvaluator runs (Boolean = value != 0 => stepped; integers => Int64 =>
interpolated, rounded half away from zero, returned as the node's integer
type; floats Double), so the evaluator's own type branch decides.
- AtTimeEvaluator boxed every interpolated Int64 as a Double (`cond ? (long)x :
v` widens to double); its "keeping its type" test passed because Shouldly
ShouldBe compares numbers across types. Fixed (affects SqlServer/QuestDB too)
and pinned with ShouldBeOfType.
Joseph Doherty