The driver logged and surfaced the raw AgentUri (which may carry user:pass@) as
HostName (the /hosts key), in ~20 log lines and in exception messages that reach
DriverHealth.LastError, and the client's request-URI messages (timeouts, stream
end, not-a-stream, framing) embedded it too - while the Test-Connect probe
deliberately redacted it. New MTConnectUriRedaction (user-info, and for the
configured string its query, removed; a clean URI is unchanged so host keys do
not move); the driver computes its display form once per options document.
Refs #743
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The connectivity probe parsed the full /probe device model every 5 s and discarded
it. It now calls a new IMTConnectAgentClient.PingAsync: one GET /probe read with
ResponseHeadersRead and a status check; the body is dropped unread. Same failure
surface as ProbeAsync (the default implementation delegates to it). The probe-tick
tests now count pings and pin that no tick issues a full /probe; the shutdown
test's absence assertion gained a positive control.
Refs #743
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Writes were one POST per slot although the Agent's PUT surface accepts several
id=value pairs. The slots surviving the local refusals now go out as one
WriteObservationsAsync(device, pairs) per device (new IMTConnectAgentClient method;
the default loops the single-pair call). A batch closes at another device or a
repeated DataItem so same-item writes keep request order; a refused multi-slot
request is retried per slot so one refused DataItem costs only its own slot.
Health contract: any Agent answer (accepted or refused) advances
LastSuccessfulRead with the state following the read / stream flags.
Refs #743
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Subscription throughput was hard-capped at SampleCount / SampleIntervalMs with no
lag detection: a machine producing more fell behind the ring buffer forever,
re-baselining every buffer-length with every value backlog-stale.
- The parser carries the header's optional lastSequence; each chunk measures
lag = lastSequence + 1 - nextSequence and whether it was saturated (full count).
- Three consecutive saturated chunks with a positive (or unknown) lag reopen the
stream at the cursor reached with count doubled, up to 16x SampleCount (max
65536), through a new IMTConnectAgentClient.SampleAsync(from, count, ct) overload
(default implementation ignores count). A raised stream that delivers nothing
steps back to the last good count; at the ceiling one Warning per pump.
- MTConnectDriver implements IDriverDiagnostics: sampleCount, sampleLag,
sampleSaturatedChunks, sampleCountRaises on the /hosts bag. The cap is
documented on SampleCount.
Refs #743
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- T-4: every ReadAsync batch fetched and parsed the whole device /current and
built a fresh index even while the /sample pump held a live one. A read is now
answered from the pump's index (read only - the pump stays its single writer)
while the current-generation pump is running, has delivered since it started and
is not degraded; /current stays the fallback, and its per-batch index is built
over a tag map frozen once per binding instead of per call. A served read clears
the read-path degradation a failed /current left.
- T-2: the /current fallback checks the answer's instanceId; a change (an Agent
restart seen by a browse-only driver) drops the cached /probe model and raises
rediscovery once per distinct id, only while the read's session is still installed.
- The pump also acts on no verdict (Degrade / latch) once its session generation
is gone, so an abandoned pump cannot write health over the new session (T-1 follow-through).
Refs #743
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
A pump the lifecycle retired or abandoned that finished after the restart
installed the new session published its stale cursor onto it (PublishAgentCursor
only checked for a null or identical session), so the next pump read its first
chunk as an Agent restart: index cleared, a Bad flicker on every subscribed tag,
a needless re-baseline. The pump now carries the session generation it was
started against; the cursor publish is checked under _subscriptionLock (where the
session install and generation bump are atomic) and a superseded pump also stops
folding late chunks / re-baselines into the new session's index.
Refs #743
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- AttributesAsync was a stub returning nothing; it now reads NodeClass /
DisplayName / DataType / ValueRank / UserAccessLevel in one request and maps them
through UpstreamAttributeMapping, the same mapping discovery registers with (moved
into the shared .Session leaf; the driver forwards to it). Objects have no value
attributes.
- Certificate auth was refused "in v1". The browse now builds its identity through
OpcUaUserIdentityFactory (shared .Session leaf, the driver forwards to it),
including the secret: resolution of Password / UserCertificatePassword through the
host's ISecretResolver (optional ctor param, DI-injected). Fail-closed: a secret:
ref with no store throws instead of being sent as the literal (the browse used to
send a secret: Password verbatim). The identity is built before any network I/O.
Refs #743
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The knob was authored by the AdminUI and parsed, but consumed by nothing. It now
caps the probe deadline (shorter of the caller's budget and the config; never
extends it; 1 s floor), the same rule as the Galaxy probe.
Refs #743
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Health contract: a write response (per-node refusals included) is the server
answering, so it restores Healthy and advances LastSuccessfulRead; a write with no
reply (transport failure / timeout) writes Degraded with the stamp preserved; a
caller-cancelled write touches nothing. A good keep-alive with no outage or dead
subscription latched now clears a Degraded left by a failed read/write instead of
only moving the stamp (a subscription-only driver otherwise stayed Degraded until
its next re-initialise). All writes are generation-stamped.
Refs #743
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
One SemaphoreSlim serialised reads, writes, subscribes, acks and the whole recursive
discovery browse, so a stalled write (up to the operation timeout) blocked every
read and subscription create, and a discovery browse would block the data plane for
its full duration. Reads/acks keep the read gate; writes and DiscoverAsync are
ungated (the SDK session is safe for concurrent service calls, the host bounds write
concurrency); subscription management takes its own gate. Dispose drains both
(DisposeAsync's drain no longer over-releases on a timed-out wait).
Refs #743
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
AcknowledgeAsync swallowed every non-cancellation transport failure (and returned
normally when the session was lost while the ack queued), so the host logged the
Part 9 ack as applied while upstream still showed it unacknowledged. Both now throw
(ServiceResultException with the transport status / BadNotConnected). A reply,
including per-ack refusals, still returns normally and counts as the server answering.
Refs #743
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Ruling refinement on I1. Only timeouts are strikes, so a never-returning
while(true), which times out once and is then only refused, was never
quarantined, and that defeated E1. A refusal now quarantines immediately
when the orphan it was refused behind has been executing for longer than
ScriptingOptions.StuckOrphanQuarantineFactor x the script's timeout
(default 10, so 20 s at the 2 s default; 0 or less disables it). It uses
the same quarantine path: one Error, otopcua.script.quarantined with
reason=stuck, Bad, lifted on source change or redeploy. A young orphan
(a short overrun) is still a plain refusal.
The guard stamps the run's start (Stopwatch timestamp) when the pool
picks it up and clears it when the run really finishes.
Tests: the 30-trigger while(true) regression (paced) now ends
quarantined with reason "stuck", still with exactly one orphan and a
healthy sibling. Positive control: a 2x overrun is refused, not
quarantined, and evaluates again once the orphan completes. A factor of
0 disables the rule. Docs (ScriptEditor "Runtime safety", VirtualTags)
and the seam remarks are updated.
Refs #734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- ScriptLogRateLimiter.LimitPerWindow. The ScriptLogTopicSink and
VirtualTagActor summaries now print the configured limit, not
DefaultLimitPerWindow. Tests assert "limit N/window" for a non-default
N.
- CalculationDriver.Dispose cancels, then takes _evalGate (bounded 2 s)
and disposes the gate and _evalCts. The pump and timer ticks enter the
gate through TryEnterEvalGateAsync, which stops cleanly once the
driver is disposed. Test: Dispose with a 6 s runaway run in flight
returns promptly, disposes both, and a late trigger is harmless.
- ScriptedAlarmEngine captures _shutdownToken once and disposes
_shutdownCts in Dispose when every drained task has completed. A task
stuck past the grace leaves the CTS to the GC. Lifecycle tests assert
the disposal.
- DraftValidator: CalculationNoEffectiveTrigger is skipped for a
change-triggered tag whose scriptId does not resolve. That case is
already CalculationScriptMissing or CalculationScriptNotFound, and
there are no reads to count. Both existing tests now assert the
absence (red before the change).
- docs/VirtualTags.md: System.Threading is denied as a whole namespace,
with an allow-list; the type list is now accurate (Unsafe,
RuntimeHelpers, and the ForbiddenMembers on System.Type); the resource
limits section covers the run guard and the allocation budget.
docs/ScriptEditor.md: accurate deny-list and a new "Runtime safety"
section covering timeout, refusal, quarantine after 3 timeouts,
allocation budget, lifting, and rate-limit summaries that appear only
on the source's next entry after a burst then quiet.
Refs #734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The hand-rolled bearer handler is replaced by AddJwtBearer (Microsoft.AspNetCore.Authentication.JwtBearer
10.0.7, pinned on the same servicing line as the other AspNetCore packages). Validation parameters come
from JwtTokenService.BuildValidationParameters, which now pins ValidAlgorithms = HS256 for both the
bearer scheme and TryValidate; MapInboundClaims is off so the minted Role/zb:username keys resolve.
/auth/token requires the cookie scheme, so a bearer can no longer mint a fresh token for itself.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
BoundedApiKeyStore's negative cache was keyed by key id, so a flood of DISTINCT well-formed
otua_<random>_x credentials missed it every time and cost one SQL query per attempt under the
SDK's SessionManager-wide lock, no credential needed (R03:3.3).
- IApiKeyStore.ListKeyIdsAsync (default null = unsupported; ConfigDbApiKeyStore lists every
ApiKeyId): the bounded store keeps a known-id set refreshed every StoreCacheSeconds
(single-flight, failed refresh keeps the previous set) and refuses an unknown id without a store
read ({outcome=unknown_id}).
- A process-wide miss budget (token bucket, Security:ApiKeys:MissBurst=100 /
MissRefillPerSecond=20, ApiKeyStoreGuardOptions) bounds lookups that reach the store and miss
while no set is available; a hit refunds its token. Per-endpoint accounting needs the
OpcUaServer impersonation seam to carry the client address — follow-up.
- docs/security.md: the new knobs, and the known-cluster gate's documented fail-open window at
central boot (R03:4.2) plus the async PSK refresh (R03:3.4).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The #687/F4 crowding re-read doubled its room up to 8 re-reads with no cap, so a pathological single-ms
cluster on a streaming gateway was streamed several times over for one page. The room is now capped at
ServerHistorian:MaxTieClusterOverfetch (plumbed into GatewayHistorianDataSource as maxPreWindowRows,
default 65536); past it, or once a read has already seen more, the read is BadTooManyOperations with
PreWindowRows set.
Item 1: IHistorianDataSource.MaxRawValuesPerRead (Gateway: learned ceiling - 16 = L; 0 elsewhere, guard-tested).
RawPageLimit plans every raw page on top of #692's lookahead: with L > 0 the read (page P + boundary ties k
+ lookahead) is <= L, P = min(N, L - k - 1) (uncapped: L - k - 1); no room => straight to the tie-cluster
over-fetch (never an uncapped or underflowing request). One re-plan + re-read when the read taught a new
limit or was refused for crowding pre-window rows (HistoryReadLimitExceededException.PreWindowRows): the
page shrinks by them. Over-fetch bound min(MaxTieClusterOverfetch, L - 1); slice size never 0.
Review fixes: F1 a window starting at/before the epoch counts AVEVA's synthetic row as pre-window (a
ceiling-truncated read was reported complete); F2 the page path disables ReverseRawReader's own re-run
(cold end-only ceiling 20/N=3000 cost 3,023 reads, now <= 40); F3 crowded look-back re-read around, not
refused; F4 the #687 crowding re-read room at least doubles (up to 8 re-reads), so a cluster of hundreds
just before a resume point no longer fails on a default gateway; a crowding refusal that leaves no room
is BadTooManyOperations, never a tick-by-tick cursor.
Item 2: ServerHistorian:EventReadHorizon (default 00:00:00 = exact, unchanged). When set, every fresh
HistoryReadEvents window's earlier bound is raised to now - horizon (both directions, all backends).
Lossy by design; documented.
Item 3 (native TimeAverage/Total re-check) stays blocked on historiangw#27.
A time inside a gap of a bursty tag costs a 64-row jump read landing in the next
burst plus a bisection back across the gap (~6 reads), so the 64 + 4/time cap
refused a 1 h/day 1 Hz tag asked at noon on 30/100/300 consecutive days
(186/606/1,806 reads) that AVEVA's own AtTime served. The cap is a runaway guard
(HistoryReadDeadline bounds wall clock): now 64 + 16 per distinct time.
Regression tests for x30/x100/x300 (each <= 7 reads a time, values checked).
Docs: DateTime-hinted nodes are stepped but keep the gateway's Double (the
Gateway never historizes DateTime nodes; the SQL backends return DateTime).
H1: the walk-back reset its width after a full segment and discarded a full
whole-remainder read, so a dense tag that stopped reporting weeks ago gave up
after 65 reads (BadTooManyOperations). It now gallops (x4, whole-remainder read
after two empties) and then BISECTS once any stored row is known below — a full
segment's (incl. a full whole-remainder read's) last row, or an earlier window
read's last row: (mid, hi] empty => hi = mid; full => known = its last row;
complete with rows => done; one density-aimed segment after a full read until an
aimed segment comes back empty; below 2 ms the whole stretch from the known row
(never a 100 ns step). Stopped 1 h..365 d with 5 y history: 6-18 reads at
1/min, 21-25 at 1 Hz, 25-31 at 10 Hz.
M1: a jump read asks for 64 rows (1,000 times over 10 y: ~66k rows, was ~1M).
L1: per-request read cap 64 + 4 x distinct times with a clear message; a strict
ceiling below 4 rows is refused naming the ceiling.
L2: DateTime hints are stepped. L3: under an integer hint a NaN/inf or
out-of-Int64-range value is unusable, never saturated Good.
L5: conformance Int64 AtTime asserts the type. L4: docs name the backend
differences. Nit: the window read end is clamped to now + 1 day.
Reviewer harnesses (Scratch693Fuzz/Cost, not committed): 0 differential diffs
over 2,400 cases.
F1 (pre-existing): SliceTieCluster continued a forward read while next <= endUtc, but the
forward window is end-exclusive; an oversized cluster one step below EndTime resumed AT
EndTime, the resume became a start == end point read and returned the EndTime samples as
data (microsecond: EndTime 8.1 us truncated to 8.0). Continue only while next is strictly
before EndTime truncated to the backend resolution.
Tests: A_tie_cluster_just_below_the_end_time_never_resumes_onto_the_end_time (native +
microsecond; both fail on the old rule) and
A_cursor_inside_a_tie_cluster_over_fetches_without_a_normal_read_first (pins the
WithinTieCluster shortcut by per-page read count; fails with the shortcut disabled).
Docs: the oversized-cluster exception to 'every page full / no lookahead => last' (F2),
and the N + skip + 1 strict-ceiling edge note referencing #694 (F4).
Review follow-up: on the Gateway every value is a Double (AVEVA stores a Boolean
as a 0/1 Int2), so the raw-evaluated AtTime interpolated a Boolean node to 0.5.
- IHistorianDataSource gains a default-interface overload
ReadAtTimeAsync(fullReference, timestamps, TypeCode? valueType, ct) that
forwards to the existing one (SqlServer/QuestDB/Null/fakes unchanged).
- OtOpcUaNodeManager.HistoryReadAtTime passes the node's scalar DataType
(AtTimeValueType: built-in type -> TypeCode; arrays/other types => null);
Raw and UNS NodeIds alike.
- GatewayAtTimeReader converts each raw row to the hinted kind BEFORE the shared
AtTimeEvaluator runs (Boolean = value != 0 => stepped; integers => Int64 =>
interpolated, rounded half away from zero, returned as the node's integer
type; floats Double), so the evaluator's own type branch decides.
- AtTimeEvaluator boxed every interpolated Int64 as a Double (`cond ? (long)x :
v` widens to double); its "keeping its type" test passed because Shouldly
ShouldBe compares numbers across types. Fixed (affects SqlServer/QuestDB too)
and pinned with ShouldBeOfType.
Gateway HistoryRead-AtTime passed AVEVA's own interpolation through: live it
answered the previous value as Good at a stored NULL's own time, null + Bad at
a Good sample right after a NULL, a drifted 99.99999999944 for a stored 100,
and no Interpolated bit. It is now served from raw gateway reads and evaluated
with Sql.Common's AtTimeEvaluator, as the SQL backends do (Part 11 §6.5.4).
GatewayAtTimeReader: one <=1,024-row forward window read [first-1ms, last+1ms]
for the batch, a walk-back (density-seeded, x4 growth, whole-remainder shortcut,
bisects dense history) for the first time's prev, and an 18-row probe for the
last time's next; truncated window reads continue or jump with a density-sized
margin. All bounds whole ms; a row counts only strictly inside its request
(AVEVA's boundary row never a sample); Bad rows value-less (#690); never past
now + 1 day; strict-gateway ceiling learned (#691.3) without re-probing.
10 dense times = 1 read (exact) / 3 (interpolated); 10 hourly = 12.
Kill switch ServerHistorian:GatewayAtTimeFromRaw (default true; false = the
old AVEVA passthrough). Known limit: the wire carries no data type, so numeric
tags interpolate as Double (Boolean/integer nodes are not stepped/rounded as on
the SQL backends).
A tie cluster's truncated read left the width at one tick; once the gap
above it was under two milliseconds nothing floored it, so segHi
stepped down one tick per read: 30 rows at one ms 0.3 s below hi with a
carried 1 s width took 10,006 identical reads (S8). (a) The top-anchored
width is floored at the granularity, Min kept; (b) a segment no wider
than the granularity is the unit that escalates to
MaxTieClusterOverfetch and then refuses - per millisecond, exact for the
ms-resolution Gateway (documented). Both are needed: the floor alone
loops on crowded reads with a small tie budget. S8 is a repo test
(4 shapes, reads <= 30, 0 repeated requests); random-data tests now
measure the refusal unit as rows in any 1 ms window. Reviewer harness,
5,000 seeds x default/MSONLY/CROWD: worst page 70 reads (default/MSONLY),
63 (CROWD) at M=65536, p99.9 27-37; no page over 500 reads at any tie
budget.
Bisecting the gap above a truncated read halved below the 1 ms request
granularity (requests are ms-rounded, so it re-sent identical requests)
and bisected the WHOLE gap even when the burst ended right above the
row it saw: a 1,026-row burst ending 100 days ago took 46 reads. After
an empty top-anchored segment the walk now gallops up from the known
row ((known + d, top], d x4, following each truncated read), then
bisects the bracket, with halving floored at the granularity (the Min
kept; a crowded gallop falls back to top-anchored halving). Burst: 18
reads, 0 repeated requests; dense tag stopped 2 days ago: 26 (gallop
alone crawled: 94). Walk fuzzed at 4,000 seeds locally (400 in-tree).
Docs: measured counts; CLAUDE.md gateway prerequisites list
RuntimeDb:EventReadMaxRows.
GatewayHistorianDataSource.ReadEventsAsync sized its result list to
maxEvents, which is int.MaxValue for NumValuesPerNode = 0: every
uncapped Gateway events read (forward on master, and the new walked
reverse one) threw 'Array dimensions exceeded' and counted as a backend
failure. Capacity is bounded at 1024. The events walk delegate moved to
ReverseEventsReader so it is tested over the real Gateway source
(forward uncapped, reverse uncapped, reverse capped).
A reverse or end-only capped events read on a backend without
SupportsReverseEvents (Gateway, Null) was refused
BadHistoryOperationUnsupported (explicit reverse) or answered with the
OLDEST N ascending (end-only, #675). It is now served newest-first by
the same ReverseWalk as raw reads: page-sized ms-aligned segments,
never answered from a read the source flagged truncated, ties sorted by
EventId (the gateway orders by event time only - the fake shuffles
them per call), one extra event walked to signal 'more', uncapped = one
forward read reversed, and the width + proven-empty stretch carried in
the events cursor. The oversized-cluster over-fetch keeps exactly the
events at T in the same order. A tie cluster over the budget is
BadTooManyOperations. Docs note the gateway's EventReadMaxRows must be
unlimited (it truncates silently).
The last pages of a deep reverse page-through on a forward-only backend
each walked down to 1970 one x4 growth step at a time (~14-16 reads,
~2.5 s live). After two empty segments the walk now asks once per page
for the whole remainder at the page budget (a truncated answer is
discarded and the walk carries on), and the stretch the page's last
complete read proved empty rides the continuation point
(HistoryContinuationState.EmptyFromTicks/EmptyToTicks, session-held, so
no versioning) so the next page steps over it. Fuzz now pages with the
walk's own hints and with true random empty stretches.
ReverseRawReader aimed every segment at half of MaxTieClusterOverfetch
(~32k rows by default) and carried that width page to page, so a paged
'latest N' read on a dense Gateway tag fetched ~32,770 rows per page
(2-5 s live) to return N. The walk now lives in a generic ReverseWalk
core: segments are aimed at ~N+8 rows (page budget 2N+16), a discovery
segment (first without a hint / after an empty one) may read 1,024, and
the tie budget is only the ceiling a single-timestamp cluster escalates
to. A truncated read never re-enters the stretch below its newest row
until the gap above it is bisected. Tests: per-page row counts on a
dense tag (node manager + walk), randomized exactness fuzz (walk and
node-manager paging), dense-tag-below-a-long-gap read bound.
Joseph Doherty