M1: the time-weighted family (incl. Total, formerly a pushed-down SUM) streams
raw window rows into the C# evaluator. New shared knob
ServerHistorian:MaxTimeWeightedRawRows (default 1,000,000; <=0 unbounded with a
Validate warning) flows through HistorianBackendRegistry into both providers.
The window query is capped at cap+1 (SQL Server TOP (@maxRows), QuestDB LIMIT $4);
one row past the cap throws the new Core.Abstractions
HistoryReadLimitExceededException, which the node manager maps to
BadTooManyOperations for that node. Not counted as a backend health failure;
bound rows are not counted; per-sample aggregates are not capped.
M2: reverse Interpolative is anchored at the later time (Part 13): points at
End - i*interval, emitted latest first. 10:00->09:00 @1h yields 10:00.
L1: Partial bit (0x4, = Opc.Ua.AggregateBits.Partial) on non-Bad results of a
truncated last interval (time-weighted family; per-sample family is a follow-up).
L2/L3: aligned with the SDK AggregateCalculator.GetTimeBasedStatusCode
(decompiled from Opc.Ua.Server 1.5.378.106): TreatUncertainAsBad=false counts
Uncertain time (Uncertain samples, held-into-outage, extrapolation) as Good
time; data but no non-Bad time => generic Bad (0x80000000); BadNoData only
when there is no data at all. Consequence: a live tag's latest interval reads
Good (documented).
L5: release note no longer claims Total matches the Gateway Integral exactly
(AVEVA's divisor is unverified); it notes Total is now evaluated in C# and
subject to the cap.
Tests: unit (reverse Interpolative x3, Partial, Bad vs BadNoData incl. an
open-ended outage, cap helpers, SDK bit oracles); conformance (reverse
Interpolative, raw-row cap at/over/0, bounds not counted, health unaffected);
node manager BadTooManyOperations; options default + warning. Each new
behaviour falsified by mutation (aggregator x9, data-source cap checks on
both backends against the real DBs, node-manager catch).
New AggregateFamily.TimeWeighted serving TimeAverage, TimeAverage2, Total,
Total2 and Interpolative, evaluated by the shared Sql.Common
TimeWeightedAggregator for both SQL backends (identical results by
construction). HistoryAggregateType gains TimeAverage2 + Total2 (appended);
OtOpcUaNodeManager.MapAggregate maps their NodeIds. Gateway's mapper already
refuses unknown defined members => BadAggregateNotSupported per node.
Semantics (server defaults, client AggregateConfiguration ignored):
TreatUncertainAsBad=false, PercentDataBad=PercentDataGood=100,
UseSlopedExtrapolation=false. Sloped for Double/Int64, stepped for Boolean;
a usable sample followed by a Bad one is held up to the outage (AtTime rule);
stepped Uncertain extrapolation past the last sample. TimeAverage/Total use
interpolated bounding (bridge outages); TimeAverage2/Total2 exclude Bad time.
Bounding values: latest usable + latest any before start, first any + first
usable at/after end (four TOP 1 / LIMIT 1 seeks).
CLIENT-VISIBLE: Total on SqlServer/QuestDb was SUM of samples; it is now the
time-weighted integral (TimeAverage x interval seconds), matching Gateway's
Integral. Release note in docs/Historian.md.
Trade-off (deviation from the preferred hybrid pushdown, allowed by the
design): the time-weighted family is NOT pushed down. Each read streams the
window's rows plus the four bounds (SQL Server: one five-result-set batch;
QuestDB: bounds UNION ALL + window select) into the C# evaluator. Network is
O(samples in window), memory O(buckets). Writing the per-segment
interpolation/edge splitting twice in two SQL dialects and keeping them
bit-identical was judged the bigger risk; this also makes the QuestDB
lag/lead spike moot.
Tests: 35 hand-computed TimeWeightedAggregator unit cases (each of 10
implementation mutations goes red); served-aggregate pin 13 -> 17;
conformance sparse fixture (intervals wider and narrower than sample gaps)
+ an outage straddling both window edges (each of the four SQL seeks
disabled in turn goes red); node-manager NodeId mapping test.
M1: a subscription that dies while the session lives (outage longer than
the subscription lifetime but shorter than the session timeout) used to
end Healthy with fresh keep-alive stamps and frozen values. PublishError
BadNoSubscription / BadSubscriptionIdInvalid naming a tracked
subscription, or a tracked subscription whose SDK PublishingStopped
persists one more keep-alive window + 5 s across good keep-alives, now
declares it lost: Degraded, outage published once, and nothing advances
LastSuccessfulRead until the session is released, so the #527 watchdog
re-initialises and re-subscribes. Both detectors wire-tested.
M2: in-place recovery is bound to the session whose keep-alive recovered;
one overtaken by a re-initialise no longer re-arms the new session's
outage latch or degrades its health (ReadCoreAsync expectedSession).
L1 CAS health updates; L2 re-check before each recovery publish; L3 skip
CloseSession on a lost session; L4 SDK log forwarding rate-limited
(20/min); L5 per-driver SDK LoggerFactory disposed with the driver.
Wire test asserts the read stamp stays frozen while the server is down.
A bad keep-alive now degrades health (LastSuccessfulRead preserved) and,
once per outage, publishes BadNoCommunication (value retained) for every
subscribed ref plus a quality-only annotation on subscribed alarm
conditions. Good keep-alives and data/event notifications advance
LastSuccessfulRead, so an idle-but-alive subscription never looks dead
to the #527 watchdog. A keep-alive that recovers on the same session
re-reads the Bad refs and republishes them Good; a session that does not
come back is recovered by the watchdog's ShutdownAsync + InitializeAsync
+ re-subscribe.
Deletes the SessionReconnectHandler wiring (it retried forever in SDK
1.5.378, cloned subscriptions the driver never re-tracked and kept health
Healthy), OnReconnectComplete, TryRearmReconnect and the reconnect_*
counters (new keepalive_recovered). Teardown of a lost session skips
wire deletes and bounds CloseSession. SDK warnings reach the driver log.
Wire test reworked: restart => Bad, then watchdog-style reinit => Good.
A SIGKILLed Primary stays an unreachable-but-not-downed member until
auto-down, and the redundancy singleton usually dies with it, so the
standby's cached snapshot kept naming it Primary and nobody captured for
~30 s. DriverHostActor now holds its own ReachabilityEvent (+MemberUp)
subscription, caches the last snapshot, and excludes unreachable peers
from the historization peer-is-Primary predicate only, republishing just
the capture verdict on a reachability change. Alarm drain and the Primary
data-plane verdict are unchanged. Gated by
ContinuousHistorization:CaptureWhilePrimaryUnreachable (default true).
Tests: message-driven cases with real ReachabilityEvents (incl. a
falsification control on the drain/data-plane verdicts) plus a real
two-node cluster crash test with downing disabled.
A capped events read now pages: the source's truncation flag triggers a
session-bound cursor (time inclusive + tie skip), oversized tie clusters
are paged within via a bounded half-open over-fetch, and release/unknown/
cross-kind points behave as for Raw.
IHistorianDataSource.TimestampResolutionTicks (default 1; QuestDB 10). A drained tie
cluster now steps off T by the resolution, so a forward one-value-per-page read on a
microsecond backend no longer re-reads the same sample forever (C1). QuestDB AtTime
results carry the client's original request time (M3); regression for the NEXT seek
skipping a tombstone (M1). Conformance backend wires the real writer and updater.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
- A DateTime.MinValue start or end on DeleteRawModified is refused
BadInvalidArgument before the backend.
- Tests: anonymous without the HistoryUpdate role denied (with it allowed),
UpdateTagAttributes enabling historization sets HistoryWrite, a throwing
updater is BadInternalError, the MinValue bounds case; corrected the
test helper's comment on how the SDK base seeds unadmitted slots.
- GatewayHistorianValueWriter remarks describe the recorder's real status
capture (and that it does not persist on the gateway).
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
The DroppedUnsupportedKind counter still counts every value; only the Debug
line is latched (cleared by the next captured value), so a Gateway-backend
comms loss no longer logs one line per tag.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
A reverse read (start > end) with a small NumValuesPerNode resumed through
the tie-cluster path, which drained the cluster and stepped FORWARD to
T + 1 tick - outside a reverse window - so the read ended with GoodNoData
after one sample. SliceTieCluster now steps down to T - 1 tick for a
reverse read (terminal at the excluded end), and the node manager slices
the over-fetched cluster in the read's own direction so the skip count is
not applied from the wrong end. Forward paging is unchanged.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
- OtOpcUaNodeManager overrides HistoryUpdateData / HistoryDeleteRawModified / HistoryDeleteAtTime,
block-bridging to the optional IHistorianDataUpdater under the shared HistoryRead batch limiter +
deadline. Insert/Replace/Update map to HistoryUpdateMode; Remove, IsDeleteModified, events and
structure data are BadHistoryOperationUnsupported. Per-item codes stitched back in request order
(MinValue SourceTimestamp => BadInvalidTimestamp, untypeable value => BadTypeMismatch); success
clears the base's pre-seeded Unsupported error.
- HistoryWrite AccessLevel bit OR-ed in for historized nodes when an updater is wired (raw + UNS);
it is dispatch-load-bearing in the SDK base. OtOpcUaSdkServer.SetHistorianDataUpdater; the hosted
service assigns it from an optional HistorianUpdaterHolder before SetSink, clears it on stop.
- AuthorizeHistoryUpdate: ACL on => scoped OpcUaOperation.HistoryUpdate decision (replaces the role
check); ACL off => #563 key re-check + new OpcUaDataPlaneRoles.HistoryUpdate role, conferred by
ApiKeyInterimRoleMapper only from a Cluster-scoped HistoryUpdate grant (WriteOperate does not).
- AdminUI ACL editor offers NodePermissions.HistoryUpdate.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Review round 1: the #481 FanCommsLossToDependencyMux fan-out DOES reach the recorder. Per the
controller ruling it is historized as an Empty/BadNoCommunication outage marker when the backend
supports Empty and counted in DroppedUnsupportedKind otherwise; remarks and the counter doc now say
so. DefaultSupportedKinds is a FrozenSet. The drop-leg test now runs under the default (gateway)
kinds, and the Empty leg pins status 0x80310000.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
The recorder now appends the typed HistorianValue and the mux's own StatusCode instead of a
double + hard-coded Good, and drops any kind not in the backend's SupportedKinds (resolved
optionally from IHistorianBackendProvider; absent => {Double, Int64, Boolean}, today's behaviour).
RecorderStatus.DroppedNonNumeric -> DroppedUnsupportedKind; GoodQuality const removed; stale
"mux carries no quality" remarks rewritten (comms-loss node stamps are not historized).
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Since #516 a DriverConfig change is stop + respawn, so DriverSpawnPlan.ToApplyDelta
was always empty and the in-place path had no producer. #663 and #664 were both
latent bugs in it. This deletes it:
- host: ApplyChildDelta, _pendingDelta, HandleApplyResult and HandleDeltaApplied
- plan: DriverSpawnPlan.ToApplyDelta
- actor: the ApplyDelta, ApplyResult and DeltaApplied messages, the Connected
reinit handler and the mid-connect AdoptConfigDuringInit
- tests: the actor tests that drove the seam
IDriver.ReinitializeAsync and each driver's implementation and unit tests stay.
Comments and docs that cited AdoptConfigDuringInit as the reason a driver must
release prior state at init now cite the IDriver contract and overlapping
retries, which still apply.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
AdoptConfigDuringInit re-runs InitializeAsync(newConfig) and acks success, but
Calculation and Galaxy ignored the JSON there: Calculation kept its old tag
set, and Galaxy connected on its old endpoint. Calculation now rebuilds its tag
table from a config with a body (sharing AdoptConfig with ReinitializeAsync and
stopping any running timer scheduler first). Galaxy, which cannot hot-swap its
options, refuses a non-equivalent one the way ReinitializeAsync does. The actor
now sends DeltaApplied once the adopted init succeeds, so a dependency
consumer's mux interest is re-registered. Unreachable from deploys since #516.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
A Connected ApplyDelta called ReinitializeAsync and kept the actor's data and
native-alarm handles, though every driver's reinit drops the driver-side
subscriptions behind them (#630). The reconcile only re-subscribes a missing
handle, so the driver went silent until a reconnect. Release both at the
driver before the reinit and re-subscribe both in a finally. Unreachable from
deploys since #516; pinned by a two-case test, each case failing against a
different wrong fix.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
- Reconcile retries of a failing alarm subscribe back off (skip 2^(n-1)-1
ticks, cap 7 => >= every 4 min at the 30 s tick) so a hanging subscribe
cannot hold the mailbox every tick; the stack is logged once per streak.
- The unreachable handler re-attach logs at Warning (it only fires on a
regression).
- OpcUaClient: a failed SubscribeAlarmsAsync detaches its notification
handler and bounds the session cleanup to 5 s under _gate;
OnEventNotification drops events from subscriptions not registered in
_alarmSubscriptions, so a stray that survives cleanup cannot deliver.
- The retry test also covers 3 failures (crosses the backoff); its tick
positive control documents its dependence on the stub's changing health.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
A data re-subscribe (every deploy's bulk SetDesiredSubscriptions push to a
kept driver) dropped the prior data subscription through the full
DetachSubscription, which also unhooked OnAlarmEvent and forgot the alarm
handle. On any deploy that did not respawn the driver, the alarm
subscription kept logging "established" while every event reached nobody
until a reconnect or respawn. The data path now uses DetachDataSubscription;
only a lost connection and PostStop detach both.
The #488 health-poll reconcile only looked at data, so an alarm subscribe
that failed on (re)connect gated the feed for the life of the connection.
ReconcileAlarmSubscription now re-attaches a missing handler and
re-subscribes a stale/missing alarm handle every tick.
OpcUaClient removes a half-built alarm subscription from its session when
SubscribeAlarmsAsync throws, so the new retries cannot accumulate.
Tests: actor-level data-resubscribe + reconcile-retry tests and an
end-to-end NativeAlarmFeedSurvivesDeployTests through DriverHostActor; all
three fail against the unfixed actor.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
After an OpcUaClient respawn an upstream inactive+unacked condition went out as
Retrigger+Clear (duplicate Activated+Cleared rows, Active flicker), and a
first-sight inactive+acked one as Unspecified (an Activated row for an inactive
alarm; a retained-Active host condition never cleared).
- New AlarmTransitionKind.InactiveUnacknowledged: one projection, rows only for
what changed (Activated+Cleared from acked, Cleared from active+unacked, none
when already inactive+unacked).
- OpcUaClient uses it for moves into inactive+unacked; first sight of an
inactive condition is InactiveUnacknowledged or Clear+Acknowledge, flagged.
- Projector: a flagged Acknowledge on an already-acked condition writes no row.
- E2E tests through DriverHostActor.ForwardNativeAlarm (red-proven).
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Item 2 — controller-discovered UDT members are writable on both transports once
browse-committed and authored ReadWrite (writability is the operator's: the commit
lands AccessLevel Read with no writable key). The write names the member (one CIP
symbolic segment per member), its own type, its own bytes. BIT members are supported:
Z-35162 (read-only probe) exposes a UDT BIT member as a 1-byte 0xC1 BOOL, so the
controller applies the bit and the hidden host's other bits are never in the request.
- AbCipDriver: pre-write width check on BOTH transports (libplctag patched part of
its read buffer and sent it all): BadTypeMismatch, Bool over a wider value
BadNotSupported; nothing sent, handle kept, health untouched, warn once.
- A runtime NotSupportedException is a refusal: no longer latches Degraded.
Item 3 — whole-UDT writes refused BadNotSupported before any runtime exists, by
decision (a structure write overwrites hidden hosts + padding), documented.
ab_server patch 0003: serve member paths from dotted flat tags (UdtSeg.*), and
accept libplctag's pad byte on 1-byte writes (stock ab_server refused every
libplctag BOOL/SINT write with 0x03). New AbCipUdtMemberWriteTests (both
transports), AbCipDiscoveredMemberWriteTests, native request-byte theory, and a
browse-commit mapper pin. Docs: AbCip.md Member writes / Whole-UDT writes / Known
gaps, fixture docs, CLAUDE.md AB section.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
M1 drop-out is a no-row Unspecified+IsRestatement update (projector case); M2 ack judged over every declared
limit; L1 ActiveAcknowledged for an already-acked new limit; L2 Bad Acked/severity carried/skipped; L3 host
end-to-end test; L4 BOOL member-by-member tests + mixed ALMD/ALMA; L5 detection requires no plain Acked.
ALMD active bit is InAlarm per Rockwell's ALMD reference; InFaulted is input health.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
- AlarmTransitionKind.ActiveAcknowledged (appended): active AND acked applied as ONE projection.
Galaxy ActiveAcked snapshot entries and OpcUaClient inactive->(active,acked) use it; the
projector writes 'Activated' only if not already active, 'Acknowledged' only if not acked.
- AlarmEventArgs.IsRestatement (optional, default false): set only by Galaxy snapshot entries
and OpcUaClient first sight. A flagged Retrigger on an Active+unacked condition with the same
message and severity, or a flagged Clear on an inactive condition, writes no /alerts row.
- NativeAlarmProjector reports the history rows per transition; ForwardNativeAlarm writes one
row per entry. Unflagged sources (FOCAS/MTConnect/AbCip, live Galaxy) are unchanged.
- Tests: projector rules each with a must-write control; OpcUaClient transition table;
Galaxy feed/driver mapping; Galaxy + OpcUaClient end-to-end through DriverHostActor.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
HistoryRead is served server-side by IHistorianDataSource; nothing ever dispatched
to a driver's IHistoryProvider. Removes IHistoryProvider, the OpcUaClient history
passthrough, DriverCapability.HistoryRead (+ resilience tier rows and AdminUI row),
HistoryReadStatusException and its node-manager catches, and the tests that only
covered them. Shared result types and HistoryAggregateNotSupportedException stay
(moved to Historian/HistoryReadContracts.cs). A stored ResilienceConfig still
carrying a HistoryRead key parses (diagnostic only; new test).
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
CentralCommunicationActor: while the PSK provider is unresolved or the first
ClusterNode load failed, re-arm RefreshContacts every 2 s (max 15) until the
first publish; Information log on first publish; 'NEVER be published' Error
now only after the retry budget is exhausted. Corrected false comments.
Tests: late provider (Grpc+Dps), failed first load, retries stop, bounded budget;
ControlPlaneStarterOrderTests back to the default 60 s refresh.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
The tests probe a free port (bind 0, release) and start the SDK server later; under a
highly parallel run another test can take the port in between and the SDK fails with
'Failed to establish tcp listener sockets for Ipv4 and IPv6.'. OpcUaHostStart.StartAsync
builds a fresh host + server on a fresh port per attempt (up to 5) when the failure is a
bind failure; all 45 start sites in OpcUaServer.Tests, OpcUaServer.IntegrationTests and
Runtime.Tests use it. Per-attempt GUID ApplicationUris are hoisted out of the host factory
because a changed URI against a reused PKI store fails with 'Cannot access private key'.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
R1: NativeAlarmProjector treats Raise-on-Active as a re-statement only when the message matches what
the condition shows; a different alarm appearing during a respawn un-acks and writes its alerts row.
FOCAS condition messages are a deterministic function of the matched set (tested across fresh driver
instances and reordered lists); Runtime E2E now covers both the same alarm and a new one during a
respawn. Residual (source message determinism) documented in AlarmTracking.md.
R2: a non-OK cnc_rdalmmsg2 return is thrown as FocasWireException{Rc} and classified as a CNC
refusal: conditions get a quality-only BadNotSupported once, one Warning, polling continues, and
driver health is untouched (no 90 s watchdog reconnect loop on a production control). Only transport
faults degrade (falsification control in the test).
R3: alarm-path health is tracked apart from the value path and combined in GetHealth: worst state,
later LastSuccessfulRead. A good value read can no longer mask a failing alarm read (no per-poll
/hosts flip) and a good alarm read cannot mask a value Degraded.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
DriverHostActorNativeAlarmAckRoutingTests: Applied does not mean the child DriverInstanceActor is
Connected, and an ack reaching a still-Connecting child is dropped by design — the single
positive-control ack was itself dropped (4/20 under DOTNET_PROCESSOR_COUNT=1 x8, still failing
with the presence budget dilated to 45 s, so not a budget problem). SpawnHostAndApply now waits
for SubscribeAlarmsAsync, which only the Connected state calls. ScriptedAlarmEngine
OnEvent_emission_happens_outside_evalGate: 2 s blocking poll -> 15 s (5/20 under starvation).
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
MAJOR-1: a deploy that changes a driver's config respawns it, and the fresh instance's empty told
record re-sent Raise for every still-active alarm, un-acking acknowledged conditions and writing a
duplicate "Activated" alerts row. NativeAlarmProjector now treats Raise on an already-active
condition as a re-statement (keeps Acked; ForwardNativeAlarm skips the alerts row); only Retrigger
(or inactive->active) un-acks. Sources whose upstream ack state is authoritative send Retrigger for
an unacked re-statement: OpcUaClient on first sight (no prior told), Galaxy for an unacked active
snapshot entry. Runtime E2E drives a FRESH FOCAS driver after a redeploy and asserts the ack survives.
MAJOR-2: a failed FOCAS alarm read annotates each subscribed condition BadNoCommunication once per
outage (#626 quality-only path) with one Warning (MINOR-6), lifted to Good on recovery; failed alarm
reads degrade driver health with LastSuccessfulRead frozen (successful ones advance it), so an
alarm-only driver trips the #527 comms-loss watchdog.
MINOR-3 CLAUDE.md #630 bullet; MINOR-4 split line; MINOR-5 a non-empty cnc_rdalmmsg2 payload that
fits neither stride throws; MINOR-7 value-tag + parsed-address tables are built and swapped, never
cleared in place under the retained poll engine. Docs corrected (FOCAS.md, AlarmTracking.md, CLAUDE.md).
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Joseph Doherty