The SDK memoizes INodeManager3.ValidateEventRolePermissions per (item, event type,
SourceNode). #826 kept it fresh by reflecting into the private
MonitoredNode2.InvalidatePermissionCaches (plus a detach/re-attach fallback). Since #855
condition events never take that channel, and the search for #878 found nothing
production-raised that does except a condition's SDK method audits, which reach it only
with auditing on, SignAndEncrypt and a SecurityAdmin session (nothing grants that role).
Under enforcement the channel verdict is now a flat deny for items on our nodes, so no
cached verdict depends on a grant, and InvalidateFolderEventVerdicts, the reflection,
the fallback, its counter and the guard test are removed. Method audits still reach
Server-object items, decided per event.
Tests: FolderEventVerdictInvalidationTests -> FolderEventVerdictFreshnessTests; new
revoked-grant test (positive control: a still-granted session) and method-audit test
(SecurityAdmin mapped for the test; positive controls: Server-object item, flag off);
the plain-event test runs each mode on a fresh server. Mutation-checked.
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
A notifier folder read one event source, its NodeId identifier (a RawPath or an EquipmentId). AVEVA stores a
Galaxy object's A&E history under the object's tag_name (Source_Object), so AVEVA's own alarm and event
history reached no production folder. Its lower-case property keys were also missed, so every row read
Severity 1 with no ConditionName.
Sources. A folder's event history now reads its own source plus the Galaxy object of every native condition
that fires on it live: those parented under it and those WireAlarmNotifiers wired to it. Live and history use
the same membership. The object comes from the condition's Galaxy alarm reference, its raw tag's attributeRef,
up to the first '.':
- The composer reads it with the driver's own parser (GalaxyAttributeRef, shared with GalaxyDriver), only for
a GalaxyMxGateway tag with an alarm (RawTagPlan.NativeAlarmReference, part of plan equality).
- The applier binds it after materialising the condition (IOpcUaAddressSpaceSink.BindNativeAlarmHistory,
forwarded by the deferred sink). A rebuild or the condition's removal forgets it.
Each object is read with the #882 rule: its own events and <Object>.<Attribute> rows, never a prefix sibling.
Fan-out. historiangw's ReadEvents filter is a single predicate, so there is one request per source; the wire
contract is unchanged. EventSourceFanOut merges the reads as one source:
- order is (time, source position, arrival), reversed for a reverse read; ties are never re-sorted by EventId,
so the boundary skip stays stable;
- duplicates by EventId are dropped and the merge is cut to the cap;
- a truncated source bounds the page at the time it stopped.
Paging, continuation points (bound to the whole source set), the walked-back reverse read and the tie-cluster
over-fetch are unchanged. A folder with one source is read exactly as before.
ACL. Every source is read inside the folder's own HistoryRead, after AuthorizeHistoryRead (#826). A denied
folder reads nothing and fails closed.
Mapping (EventMapper, property names now case-insensitive):
- Severity: an AVEVA row's priority goes through GalaxyAlarmPriority, the table live Galaxy alarms use
(MxAccessSeverityMapper now delegates to it), so history matches live. Otherwise severity (OtOpcUa's own
rows, which AVEVA stores lower-cased), clamped. AVEVA's own 1-4 severity is ignored. The table ranks a
higher priority as more severe, which inverts AVEVA's convention: filed as #887.
- An AVEVA row whose source_conditionvariable names a native condition that fires on the folder takes that
condition's SourceNode and live ConditionName. Otherwise ConditionName is alarmname or the condition
variable's attribute.
- alarm_id/alarmid map to AlarmId.
Tests:
- GalaxyEquipmentEventHistoryWireTests (4, real SDK session, production Gateway backend, AVEVA-shaped fake
gateway): equipment EqA backed by objects Oven_007 + OvenDoor_07. Covers object and attribute events from
both objects, the prefix sibling excluded, cross-object and in-object ties, forward paging at 1-4 a page,
reverse whole and paged, SourceNode/ConditionName/Severity, the raw device folder, ACL deny with no backend
call plus a granted positive control, and a rebuild forgetting the bindings.
- EventSourceFanOutTests (8), EventMapper AVEVA cases (11), GalaxyAttributeRefTests (16), a composer test and
an applier ordering test.
- Mutation-checked, 17 mutants, all killed. Covered: parent-only membership, no scope, a case-sensitive
condition match, no condition rewrite, no truncation bound, EventId tie order, no sort, no cap, no stop at
the bound, unreversed source order, no dedupe, no '.' boundary, priority ignored, case-sensitive keys, no
applier bind, no deferred forward, and a composer that ignores the driver type. The forget-on-rematerialise
guard survives as belt-and-braces: the rebuild clear covers it.
- Live, read-only, no WinRM, vd03 HistorianGateway ea9f21c: equipment folder EQ-886-ZipsenLeft (objects
DevWinPlatform + Left_001). It read 0 events before and 92 now, equal to the unfiltered read's rows for both
objects. 1/7-event pages and reverse whole/5-event pages agree, and 73 rows resolve to their live condition.
diff_runs vs the #882 run: 221/226 identical. The 5 that differ are expected: DevWinPlatform Severity
1 -> 750/1000, and #881 capabilities.
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
Every Boolean under Server/ServerCapabilities/HistoryServerCapabilities read
false (the SDK default) even though data and event history were served, so a
spec-following client skipped HistoryRead entirely.
Each backend now declares what it serves through its own interface:
- IHistorianDataSource.ReadCapabilities (Data / Events / ServerTimestamps):
Gateway Data|Events; SqlServer + QuestDb add ServerTimestamps (they store
ServerTimeUtc, the Gateway stamps read time); Null declares None. A
reflection guard requires every production source to declare it itself.
- IHistorianDataUpdater.Capabilities (required member): SqlServer and QuestDb
declare Insert/Replace/Update/DeleteRaw/DeleteAtTime; the Gateway provider
has no updater.
The node manager writes the flags on every source/updater assignment
(AdvertisedHistoryCapabilities), and the same declarations drive dispatch:
an update the updater does not declare is BadHistoryOperationUnsupported
without reaching it, the HistoryWrite AccessLevel bit follows a non-empty
declaration, and event-history notifier promotion follows the Events flag.
Event and annotation update flags stay false (nothing dispatches them).
Tests: booted-server wire test per backend + Null + a SqlServer->Gateway
rewire pairing on one session; dispatch gating unit test; the #801 live
gate now asserts the Gateway flags (not run here).
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
- The filter-group registry holds groups weakly (the filter->group table is
the only strong reference), sweeps dead entries amortised and counts only
live shapes against the 4096 cap; filters past the cap are counted on the
new meter otopcua.opcua.condition_events.filter_groups_unshared.
- A group validates its where clause too (restored, never-validated filters
evaluate like validated ones).
- Test: value-equal filters cost one projection per locale set per report.
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
- A filter group owns a deep clone of its first filter, validated once and
evaluated under the group's lock: no item-owned filter is mutated or raced
(Validate mutates a select clause).
- The projection is computed in the SDK's order: where clause first, fields
only for an item that will queue.
- Unambiguous locale key; counter for filters given a private group past the
registry cap; DeliveryCache getter returns its own allocation.
- UseProjectedEventDelivery=false also asks the role and grant checks per
item, so the equivalence test compares against the pre-optimisation path.
- Equivalence test: same select list under three where clauses, a filter
modified mid-run, a session with other locales, an ungranted anonymous
Server-object item, per-rig counter; plus a small overflowing queue with
DiscardOldest true and false.
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
Condition events delivered to many event items (folder and Server-object)
no longer pay the SDK's per-item QueueEvent(IFilterTarget) work under the
node-manager Lock: the role-permission verdict is shared per (session,
identity), the #826 grant check per identity per hop, and the where clause +
select-clause projection per (value-equal EventFilter, locales); each item
gets its own EventFieldList over the shared read-only fields. The SDK's
QueueEvent body is reproduced under the item's own lock (three private SDK
members, guard-tested); anything outside that shape falls back to the SDK.
The per-report duplicate check replaces the SDK's O(queue) scan.
ProjectedEventDeliveryWireTests compares both paths over the wire.
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
- RefreshDirectItemsWithCut keys its "decided off Lock" set per (item,
condition), so a condition one item's walk saw is still decided for another.
- The deferral warning no longer blames slow folder subscribers (no folder of
ours defers since #855).
- FolderEventFloodWireTests' removal test now asserts the folder item got
nothing and the Server object exactly the pre-stall event (it passed with
the post-stall liveness re-check deleted).
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
- Both plain ConditionRefresh paths (Server object and folder) skip a
condition a deploy removed or replaced between the off-Lock walk and the
queueing (the SDK base did not either; only the cut path checked).
- A failure delivering at one notifier hop is logged and the walk continues
to the remaining hops; a failure deciding the Server-object items withholds
only them.
- Test: a condition removed during a plain refresh (folder + Server object).
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
- QueueFrozenReplay's verdict refuses a condition a deploy removed or replaced
since the cut (Server-object and folder), so the cut path cannot replay a
ghost or judge a replaced scripted alarm on its old folder.
- DeliverToNotifierItems resolves the #826 decision's item-independent part
(live condition, notifier check, scopes) once per hop; only the grant and
SDK role checks stay per item.
- A deferred delivery's drain starts on the pool, never inline under Lock.
- Test: a condition removed between the cut and its replay (folder + Server
object).
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
- QueueFrozenReplay re-asks each item's verdict right before each entry's gate
hold (Server-object and folder cut paths): a grant revoked between the
pre-Lock verdicts and the replay no longer lets the replay through. A denied
entry is also removed from the cut's re-sent set.
- ReportEventWithoutBlocking logs and returns on a delivery that faulted
inline instead of booking it as a deferral episode (which ran the drain and
ClearStaleRefreshCuts under the event-report gate while Lock was held).
- OnConditionReportEvent isolates failures per Server-object item.
- Tests: revocation between cut and replay (folder + Server object), throwing
hop not booked as a deferral; production-overload sampling inside Lock.
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
Condition events reaching one of our notifiers no longer go through the SDK
MonitoredNode2 channel: the walk serves the attached items itself, per item and
per event (audit guard, uncached INodeManager3 verdict, item gate/cut,
QueueEvent), as the Server-object path does. No notifier of ours defers any
more; the #838 deferral queue and the #854 cut stay as the safety net and
folder items share the cut (RefreshDirectItems, flag on and off). An event
whose walk suspends is marked deferred before it suspends.
Tests produce deferral with a notifier-stall probe instead of holding a folder
item's SDK lock (which would now block the reporter). New
FolderConditionRefreshWireTests; design note in docs/plans.
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
Review of 1622d07f:
1. Perf: every i=2253 refresh ran the whole notifier walk + permission checks +
QueueEvent under Lock. Now, with nothing deferred (the common case), the refresh is
the SDK-shaped walk off Lock (Lock only to copy the root notifiers), kept only if no
deferral episode existed at any moment of it (count 0 before and after, episode
unchanged); otherwise the cut path also runs and its later replay supersedes. The
cut path takes ACL/role verdicts off Lock, then one short Lock hold: watermark +
frozen snapshots of Retain conditions from _alarmConditions (no tree walk) + per-item
cut and "replay pending" marker; the replay is queued OFF Lock, one event per hold of
the item gate. While pending, a condition event newer than the cut marks its
condition so the older frozen entry is dropped rather than following it. A
concurrent fast-path refresh cancels a pending cut replay of the same item.
Measured, 3000 retained conditions, longest Lock wait seen by another thread during
one refresh: 1622d07f 95-106 ms idle / 102-127 ms deferring; now 0-3 ms idle /
23-32 ms deferring.
2. ACL: a retained condition omitted from the replay for want of a grant is never
withheld (RefreshCut.RetainedAtCut); only Retain=true events of a condition omitted
because it was NOT retained are.
3. One gate/cut state per item for its lifetime (ConditionalWeakTable keyed by the
item): an unsubscribe + resubscribe between refresh and drain keeps the cut.
4. Cuts carry their deferral episode; ClearStaleRefreshCuts runs when the count
returns to zero, and a stale-episode cut is inert.
5. Dispose guard for the custom refresh (ObjectDisposedException once disposing;
Dispose waits for refreshes in flight).
6. Docs: a refresh waits at most for one in-progress QueueEvent on the item.
Tests: resubscribe variants, ACL-grant-lands test, pending-replay probe test,
dispose test, 3000-condition Lock-hold measurement with a probe pinning that neither
path queues its replay under Lock; stress invariant tightened (never backwards outside a
refresh; each refresh ends on the newest delivered state) and tolerant of the SDK's own
BadRefreshInProgress overlap refusal. Each new test fails against its targeted mutation.
While a folder subscriber's SDK channel is full, a condition's later events wait in
its #838 deferral queue and reach i=2253 items only when the drain gets to them, off
Lock. A ConditionRefresh in that window replayed the current state, then the older
deferred events followed RefreshEnd (stale state, or a condition the refresh had just
dropped re-added until the newest drained).
- Condition events are reported as ConditionEventSnapshot (an InstanceStateSnapshot
carrying a report sequence, condition NodeId and Retain); events queued behind a
deferred report are marked Deferred.
- Server-object refresh (ConditionRefresh and ConditionRefresh2, flag on AND off) now
runs in RefreshServerObjectItems: watermark, "anything deferred?", the replay and its
queueing in ONE Lock hold, so the replay is exactly the state at the cut. Per item it
queues the replay and installs a RefreshCut under the item's gate.
- OnConditionReportEvent holds that gate across "superseded? -> QueueEvent" for
deferred events only. Withheld from that item alone: events at/below the cut whose
condition the replay re-sent, or Retain=true events of a condition it omitted; an
older Retain=false event of an omitted condition is still delivered. Queues, other
Server-object items and folder items are untouched; refresh never waits for a drain.
- Lock order Lock -> item gate -> SDK item lock, same as a synchronous report.
- Meter otopcua.opcua.condition_events.superseded.
- Folder items keep a documented residual (SDK channel backlog + memoised per-item
verdict leave no per-event hook).
Tests: ConditionRefreshDeferralWireTests (deterministic overload by holding the folder
item's SDK lock; 6 variants fail before the fix; probe-driven gate test fails with the
gate removed; concurrent reports/refreshes/stall stress test fails before the fix).
#838 review of 91e28e87:
1. Ghost events. The deferred report ran the SDK walk off Lock with no
liveness check, so an event waiting behind a full channel when a deploy
removed its condition still reached i=2253 subscribers after the NodeDeleted
(a ghost Retain alarm for ScadaBridge) and folder items. Delivery now walks
the SDK path itself (OnReportEvent, OnReportEventAsync, inverse notifiers,
then the Parent as BaseInstanceState does) re-checking before EVERY hop that
the condition is the live instance; OnConditionReportEvent checks the same,
which also stops a non-live scripted condition being admitted through its
old equipment folder (#808). A guard test pins that only NodeState and
BaseInstanceState define the walk. New wire test removes the condition in
the same Lock hold as a flood; the flag-off variant fails without the checks.
2. Back-pressure. Deferral is per condition (one condition's events stay
ordered; others are not held up). Overflow policy: coalesce, newest wins --
past 1024 waiting per condition or 65,536 total, the oldest not-yet-started
event is discarded (a condition event carries full state). Warning on
entering deferral (<=1/min), Information on draining, meters
otopcua.opcua.condition_events.deferred / .discarded{reason}.
3. Verified by decompiling SDK 2.0.1: the notifier list is snapshotted under the
node's own lock, MonitoredNode2.Dispose completes its writer (pending writes
fail with ChannelClosedException, swallowed), and the consumer catches
per-event exceptions -- no fix needed; documented.
4. ConditionRefresh does not flush deferrals; documented as a residual.
#844 review: DescribeCredential canonicalises the key id (trim, lowercase;
non-hex ids fold into one identity) and the username, since ConfigDbApiKeyStore
matches with SQL Server's case-insensitive, trailing-space-blind `=`.
#838(a): each notifier's SDK MonitoredNode2 drains a bounded channel (4096,
FullMode=Wait) and the synchronous NodeState.ReportEvent blocks when it is
full. Condition events are reported under Lock, and the delivery task takes
Lock on a verdict-cache miss (the SDK's own GetNodeMetadata, flag off too, and
#826's IsNotifierEventVisibleTo). A burst of >4096 events under one Lock hold
therefore deadlocked the node manager -- FolderEventFloodWireTests stalled at
exactly 4097 events with enforcement on AND off (so it predates #826).
ReportConditionEvent now goes through ReportEventWithoutBlocking:
ReportEventAsync completes inline while channels have room (unchanged normal
path); on overflow the report continues asynchronously holding no thread or
lock, and later reports queue behind it so events keep report order. The
deferred work holds only frozen snapshots and drains once Lock is released;
client delivery stays bounded by each item's queue.
#838(b): documented in docs/security.md that Part 3 ReceiveEvents applies to
the event's EventType and SourceNode (the SDK check, kept first), not to the
notifier; our decision is layered on top and non-condition events on a folder
item fail closed under enforcement. The reflection item stays open.
1. Orphaned subscriptions (session closed without deleting subs) have Session == null, so the public
MonitoredNode2.InvalidatePermissionCacheForSession never reached their items: a cached allow survived a
deploy, the moved alarm's events were queued and delivered on TransferSubscriptions. Each notifier's
cache is now cleared whole through the SDK's private MonitoredNode2.InvalidatePermissionCaches
(reflection, guard-tested; fallback = per-session drop + Remove/Add of session-less items).
2. The verdict is decided on the condition LIVE under the event's NodeId, not the event's own instance:
the SDK key has no instance identity, so a replaced instance's late event cached a deny its successor
inherited until the next invalidation.
3. Invalidation is skipped while enforcement is off (forced across an on/off authorizer transition) and is
one O(cache) clear per node instead of a whole-cache scan per item.
4. TagReferencingEquipment carries alarm-bearing raw tags only (TagConfigIntent.Alarm); the snapshot adopts
an OrdinalIgnoreCase set instead of copying it again.
Tests (each fails with its fix reverted): FolderEventVerdictInvalidationTests (orphan + deploy + transfer;
replaced instance's late event; ACL-off skip; SDK private-method guard), DeploymentArtifactScopePathTests
(alarm-bearing projection).
Event items on an equipment / raw device folder were fed by the SDK's MonitoredNode2 fan-out with no ACL
decision, so after a deploy moved a scripted alarm (or dropped the reference projecting a native alarm into
an equipment) the OLD folder's subscribers kept receiving it - EventId included - until the rebuild landed,
indefinitely after a failed apply.
- OtOpcUaNodeManager re-implements INodeManager3 so ValidateEventRolePermissions (the SDK's per-event,
per-item hook; base not virtual) adds IsNotifierEventVisibleTo: live condition instance, notifier still a
legitimate notifier of it, and the #586/#808 Server-object rule (IsConditionVisibleTo over
ConditionVisibilityScopes).
- The SDK memoizes that verdict per (item, event type, SourceNode): InvalidateFolderEventVerdicts runs on the
authority/authorizer setters, RevalidateMonitoredItems (deploy + timer) and CompleteAddressSpaceApply.
- ConditionRefresh on direct items goes through RefreshDirectItems (base loop + the same decision).
- New generation field TagReferencingEquipment (RawPath -> EquipmentIds, from the artifact's
UnsTagReferences via AclScopePathResolver.ResolveTagRawPath): a native alarm's wired equipment folder only
counts (folder delivery AND Server-object scopes) while the current generation still references it.
- Secondary #808 edge: UNS variables wired UNS->Raw Organizes by AddReference are tracked and always placed
as reference variables (an Area named like its equipment's id no longer narrows them).
Tests: UnsFolderScopeWireTests (moved alarm on a folder item + ConditionRefresh + rebuild; area named like
equipment id), NativeAlarmFolderEventAclWireTests (dropped reference: folder + Server object + refresh +
restore), DeploymentArtifactScopePathTests (projection). Rig/event tests now supply tag references.
Review of af5c9744:
1. UNS reference variables (s=Area/Line/Equip/Eff) of a moved or renamed
equipment stayed readable/writable under the OLD names' grants: they now
resolve as ReferenceVariable and are authorized at their split path only
while it equals their live equipment folder's placement (live chain ==
generation chain) + one name; otherwise denied (Read and Write).
2. TryPlaceScriptedCondition now also requires the owner folder's live chain
to match the generation (AuthorizeScriptedCondition gains ownerFolderChain),
so a moved equipment's alarm/children/commands follow its folder.
3. Identifiers containing NUL are denied before the decision cache, keeping
client ids disjoint from the NUL-prefixed resolved-scope keys (a client
could otherwise poison a real folder's cached verdict for its key shard).
4. An id no live node holds is Unknown: placed at [\0unknown] so only
Cluster / Namespace-uns grants decide it (a removed opaque id can no longer
split onto a same-named narrow grant; narrow grantees get no NodeDeleted for
removed UNS folders/vtags/conditions — documented); 4-segment
reference-shaped ids keep their split.
5. A wire test drives placements from DeploymentArtifact.ParseDataPlaneAuthority.
Baseline updates: #797 owner tests (unit + wire) and Sdk2RepeatedAlarmAuthority
now install folder scopes; Sdk2HistoryAuthority's UNS alias used the RAW path
as its UNS id (never minted in production) and now carries EqA/HistoryAlias.
Each new guard was falsified (reverting it fails its test).
UNS Area/Line/Equipment folders are minted under their opaque entity ids
(s=UA-/UL-/EQ-...), virtual tags under s=EQ-.../Name and scripted conditions
(+ #629 children) under s=<ScriptedAlarmId>[/...], while every UNS grant is
planted at a NAME path. DataPlaneAuthorizer split those ids, so with
Security:Acl:Enabled a narrow (Area/Line/Equipment) grant could not Browse,
Read, subscribe or HistoryRead them, and Server-object delivery never reached
it — so a narrow key never held the EventId Ack/Confirm/AddComment need.
- The generation carries UnsFolderScopes (folderId -> path + id chain), built
by DeploymentArtifact from the same AclScopePathResolver as the grants; ids
two entity kinds share are dropped; plumbed through IDataPlaneAuthoritySink.
- OtOpcUaNodeManager.ResolveUnsAclSubject reports live facts (folder id
ancestry / scripted owner binding); attached by the Authorizer setter.
- Evaluate places the subject against the CURRENT generation before the
decision cache, which now caches the walk keyed on the resolved scope.
Folders require live chain == generation chain (a moved equipment is denied
under old and new line until the rebuild). Owned nodes (vtags, conditions,
children) get owner path + a reserved segment, so Tag grants and their
Browse transit never expose them. Conditions share TryPlaceScriptedCondition
with the #797 command gate.
- Server-object visibility of a scripted condition is decided on the
condition alone, so a moved alarm cannot reach its old owner there.
Tests: UnsFolderScopeWireTests (opaque ids, end-to-end browse -> EventId ->
Acknowledge on an Equipment grant; sibling denied; Tag-only grant; moved alarm
and moved equipment stale windows). Rig tests now install folder scopes
explicitly (ServerEventRig.FolderScopes); the #797 recreate test deliberately
asserts the moved alarm is invisible to the Cluster observer in the window.
Scripted condition NodeIds are opaque ScriptedAlarmIds, so splitting them as a
topology path could never match an Equipment grant: a narrow equipment grant was
denied while a Cluster grant passed.
- OtOpcUaNodeManager binds each LIVE scripted condition instance to the
EquipmentId it was materialised under (only when it hangs under that folder);
dropped on remove / drop-recreate / equipment-subtree removal / rebuild; a
same-id re-materialise under another equipment re-parents and re-binds.
- DataPlaneAuthoritySnapshot carries EquipmentScopePaths (EquipmentId ->
[uns, Area, Line, Equipment]) decoded from the artifact by the same
AclScopePathResolver call that plants Equipment grants.
- DataPlaneAuthorizer.AuthorizeScriptedCondition re-reads key, grant and
generation per call (uncached) and evaluates at the owner path; unbound,
replaced or unplaceable owners deny under every grant. No id-split or
realm-root fallback.
- Native conditions (RawPath) and the ACL-off role gate are unchanged.
NodeIds and AlarmCommand.AlarmId are unchanged.
Sdk2OpaqueScriptedScopeBaselineTests deliberately flipped: it pinned the denial
and now pins owner-grant allow / sibling deny / cluster allow. Follow-up #808
(folder ids + Server-object event visibility for narrow UNS grants).
QuestDB modified-history reads dropped every still-pending trail row that was
not the latest at its source time, so V0->V1 (applied marker never written)
followed by V1->V2 hid the V0->V1 change for good. A pending row below the
latest is now served when the NEXT trail row at that time recorded a different
prior value (kind, value, status or the prior's server_ts) - the next change saw
what this one wrote; an orphan of a change that never landed leaves the same
prior for the next row and stays hidden. QuestDB window functions reject
VARCHAR, so the next row's prior comes from a LEFT JOIN on (ts, lead(id)).
The updater's comment and marker-failure log no longer claim modified reads
"still serve it" unconditionally.
Regression tests (QuestDB 10.0.1 throwaway container): marker lost then a
replace / delete at the same time; an orphan followed by a marker-lost retry
then a later change (orphan still dropped); the random marker-mix fuzz now
varies the recorded priors. All four fail on the old filter.
NITs:
- SqlServerHistorianValueWriter: a SqlException raised while the token is
cancelled (SqlClient's "Operation cancelled by user") is rethrown as
OperationCanceledException instead of booked as a failed write. Integration
test holds a TABLOCKX on Sample and cancels mid-INSERT; it returned false before.
- SqlServerHistorianDataUpdater.DeleteRawBatches is an Interlocked counter.
- HistoryReadEvents: an event filter whose SDK validation throws (unknown
operator) failed with the generic BadUnexpectedError; the node manager now
re-throws any failed event-filter validation as BadEventFilterInvalid.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
HistoryReadEvents projected only EventId / SourceName / Time / ReceiveTime /
Message / Severity: a select clause asking for EventType, ConditionName or
SourceNode (what most generic alarm viewers ask for) got nulls, and the
event filter's where-clause (an OfType) was ignored.
- HistoricalEvent gains optional EventTypeName, ConditionName and AlarmId.
SqlServer, QuestDB and Gateway event reads fill them from what their
alarm writers already store (AlarmTypeName / AlarmName / AlarmId; the
gateway event Type and its AlarmName / AlarmId properties).
- The node manager maps the type name to its Part 9 event type (unknown or
non-event => AlarmConditionType), projects ConditionName, and resolves
SourceNode to the materialised condition the alarm id names (the live
events' SourceNode), else null.
- The where-clause is evaluated per event with the SDK content filter
through an IFilterTarget (OfType via the type tree; other operators
against the projected fields), after the page's cursor is taken, so
paging is unchanged.
Tested in the node manager, over a real client session (the Client.CLI has
no `historyread --events` mode yet; filed as a follow-up), and by alarm
writer -> event read round trips on SqlServer and QuestDB.
Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The native-ack path is fire-and-forget after the Part 9 Acknowledge returns, so
a Galaxy ack refused for lack of a credential (#749), or rejected by MXAccess,
left the client with Good and the condition showing Acknowledged while Galaxy
was never acked. Both options from the issue, layered:
1. Synchronous refusal for what the driver can decide without I/O. New optional
IAlarmAcknowledgePreflight (Core.Abstractions); GalaxyDriver refuses an
anonymous session when the driver has no AlarmAck fallback account.
DriverHostActor publishes a condition NodeId -> owning-driver snapshot into
NativeAlarmAckPreflightView (lock-free, volatile); the node manager's new
NativeAlarmAckPreflight seam, consulted after the authz and Primary gates,
returns BadUserAccessDenied with the reason - nothing routed, applied or
reported. A keyed session is not pre-checked (its identity lives in the
Secrets store, which must not be read under the SDK lock).
2. Revert for everything else. RouteAlarmAck carries the condition NodeId;
DriverInstanceActor replies AlarmAckFailed on a refusal, a backend failure,
a driver without an alarm source, or an ack dropped while connecting /
reconnecting. DriverHostActor re-projects the condition unacknowledged via
NativeAlarmProjector.TryRevertAcknowledge, but only while the local ack is
still pending - any driver transition projected since then wins. A disabled
or unspawned driver is not reverted (unchanged).
GatewayGalaxyAlarmAcknowledger now throws DriverOperationRefusedException on a
non-zero native hresult (wrong password, no ack permission) instead of only
warning, so that case reverts too; as a refusal it neither retries nor trips
the breaker.
Tests (each fails with its fix mutated out): projector revert/supersede,
DriverHostActor refused + failed ack -> unacked AlarmStateUpdate (successful
ack control), preflight view snapshot, node-manager preflight unit test,
over-the-wire ConditionMethodEventTests (BadUserAccessDenied, no route, no
event; positive control), Galaxy preflight + hresult refusal.
Docs: Galaxy.md, AlarmTracking.md, ScriptedAlarms.md, .claude/rules/alarms.md.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
mxaccessgw 488649c+ acknowledges by AuthenticateUser as the operator and
writing <alarm>.AckMsg, so AcknowledgeAlarmRequest now needs a real Galaxy
operator_user + operator_domain + operator_password. The OPC UA data plane is
API-key/anonymous and carries no Galaxy password, so the driver resolves the
account per ack:
1. the session's API key's Galaxy ack identity, set on AdminUI /apikeys and
held ONLY in the encrypted Secrets store
(otopcua/apikeys/<keyId>/galaxy-ack-{user,password}); a key with a user but
no password secret is refused, never silently downgraded;
2. else the driver instance's AlarmAck fallback account (DriverConfig
FallbackUser/FallbackDomain/FallbackPasswordSecretRef; secret:/${secret:}/
env:/file:/dev: only — a literal is refused);
3. else DriverOperationRefusedException: nothing is sent, never an empty
operator_user. Refusals are excluded from the resilience retry/breaker.
- Pins ZB.MOM.WW.MxGateway.Client/Contracts 0.2.1 -> 0.3.0 (vendored into
nuget-packages/ from the Gitea feed, per the local-mxgw source mapping).
- The session ApiKeyId (not a secret) rides NativeAlarmAck ->
RouteNativeAlarmAck -> RouteAlarmAck -> AlarmAcknowledgeRequest; the password
is resolved in the driver and never travels through actor messages.
- The OPC UA principal + key id are appended to the AckMsg comment
("[OPC UA: <principal>, key <id>]") and logged, so a shared fallback
account does not lose who acknowledged.
- Password never logged: GalaxyAckCredential.ToString omits it, the
acknowledger scrubs it from every gateway message it echoes and withholds
any exception that echoes it; audit rows carry only the key id.
- AdminUI: /apikeys "Ack identity" editor (set/remove; delete-key clears it),
Galaxy driver form gains the fallback section.
- Synchronous OPC UA status for a refusal is follow-up #760 (the native ack
path is fire-and-forget after the Part 9 method returns).
Tests: request mapping (password populated), password absent from logs and
exception text (mutation-checked), key-mapped / fallback / refused paths at
resolver and driver level, breaker non-trip, key-id propagation through node
manager and DriverHostActor, AdminUI service + form round-trip.
Closes#749
C1 (critical): B6 computes the plan off the pinned thread while B1 made writes
non-creating, so a value/status/alarm write for a node the in-flight plan adds
was skipped, and change-driven drivers never republish it (BadWaitingForInitialData
indefinitely; on a full rebuild every re-subscribe value landed on discarded
nodes). OpcUaPublishActor now applies writes immediately AND remembers the
latest per (realm, id) while a plan is in flight or queued (value+status
compose; alarm state/quality coalesce to the latest, which the node's delta
gate makes inert on an existing condition), replays it after each apply before
answering, and drops it once the pipeline is idle. Bounded (250k keys by
default; beyond it new keys are not held, logged once).
I2: the data-plane-authority parse runs in its own try inside the plan task; a
throw is logged and never degrades the address-space apply.
I3: the SDK's DeleteMonitoredItems releases an item's component-cache
reference on a Good delete; a tombstoned item's reference was already
released, so the delete took one from the re-created node's entry. Released
items are remembered and the orphan delete takes one back first. Event items
never held a reference (EventManager + SubscribeToEvents), so the three
event-item releases are gone.
I4: direct tests for the HistoryUpdate override's per-request updater check
and the HistoryReadEvents override's source guard (both reached by restoring
the bit after a reset - the B7 race window); fixed the false remark.
Minors: the MaterialiseAlarmCondition kind-swap tombstones the dropped
instance; a bytes-less rebuild arriving mid-plan queues behind it (replies in
order); a PlanReady/PlanFailed from a previous actor incarnation is ignored and
its requester answered degraded; the tie-cluster end bound of an uncapped read
fits against the backend ceiling (defence in depth - guarded by a sweep test);
the lock-time benchmark stays in the default run, headroom noted.
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Lane H1 follow-through: SyncAdvertisedAggregates now also calls
HistoryPaging.AdvertiseMaxReturnDataValues(Server, source
.MaxRawValuesPerRead) so HistoryServerCapabilities/MaxReturnDataValues
(i=11273) follows the historian source on every assignment (the Null
default advertises 0 = no limit); a false return logs a Warning (the cap
is enforced by HistoryRead regardless).
Test: NodeManagerMaxReturnDataValuesSyncTests reads i=11273 back through
the SDK master node manager after setting a capped source (7000, 250)
and after resetting to the Null source (0).
Refs #732#736
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
H1 fuzz finding (routed to Lane B because AppendEndBoundAsync lives in
OtOpcUaNodeManager.cs): a bounded HistoryRead-Raw page sized BELOW
NumValuesPerNode by the backend's row ceiling (RawPageLimit.Plan) took
the end bound whenever its data fitted, because the fit was judged
against NumValuesPerNode instead of the page's effective cap — a full
ceiling-sized page came back with cap + 1 values. The bound is now
fitted against the page allowance (the ceiling-adjusted data cap plus
the start bound a first page already carries); when the page is full
the bound goes to the next page through the continuation point.
Test: NodeManagerHistoryReadBoundsTests
.A_full_ceiling_sized_page_carries_the_end_bound_on_the_next_page
(ceiling 4 ⇒ 3 data rows; first page = start bound + 3 with a CP, second
page = the end bound alone; control: no ceiling ⇒ one page of 5).
Refs #732#736
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 01 U3. EnsureFolderIsEventNotifier granted the EventNotifiers
.HistoryRead bit (and registered the event-history source) only when a
real HistorianDataSource was wired at promotion time, and EnsureVariable
composed the AccessLevels.HistoryWrite bit from whether a
HistorianDataUpdater existed at materialise time — so an assignment
after the first apply left existing folders without event history and
existing nodes refused by the SDK's HistoryUpdate admission until a full
rebuild, and a source reset to Null never took the bit back.
The two setters now re-evaluate: HistorianDataSource walks every promoted
folder (bit + source registration, both directions), HistorianDataUpdater
recomposes the AccessLevel of every historized variable (Writable read
back off the node's own CurrentWrite bit). The Host's boot ordering is no
longer load-bearing.
Tests: NodeManagerCapabilityRefreshTests (late source lights the bit on
an already-promoted folder and a reset takes it back, live-event
subscription untouched; late updater grants HistoryWrite to historized
nodes only and a reset takes it back, authored Writable preserved).
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 01 L3/H5/H4/P6/P4.
- L3/H5 OnEquipmentTagWrite's dispatch is factored into DispatchDeviceWrite:
a gateway that throws synchronously is refused inline with
BadDeviceFailure (it used to escape the handler under the SDK Lock), and
a faulted/cancelled dispatch task has its exception observed and logged
and carries the real reason into the revert/audit event instead of
surfacing as UnobservedTaskException at finalization.
- H4 every timestamp the manager stamps (condition quality, timed-shelve
expiry, method-transition events, carry-over, revert, revalidation)
reads the injected Clock; RemainingShelveMs takes the clock reading.
- P6 OpcUa:Quotas:* (OpcUaQuotaOptions on OpcUaApplicationHostOptions):
Min/MaxRequestThreadCount, MaxQueuedRequestCount with the former
hard-coded 5/100/200 as defaults, and the TransportQuotas members as
nullable overrides over the SDK defaults.
- P4 the SDK line exposes IHistoryReadAsyncNodeManager but CustomNodeManager2
offers no async virtual to override (adopting it means re-implementing
the base dispatcher), so the lever is the admission cap:
MaxConcurrentHistoryReadBatches defaults to clamp(2*ProcessorCount, 4, 16).
The Host still overrides it from ServerHistorianOptions (default 16) —
follow-up F3.
Tests: NodeManagerWriteDispatchTests (sync throw refused; faulted task
reverts; success leaves the value; clock-stamped revert),
OpcUaApplicationHostQuotaTests.
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 01 P1/P2/P3/P5 — work done under the manager-wide SDK Lock on the
pinned publish thread, where every Read/Browse/Publish waits.
- P1 RebuildAddressSpace detaches the shared root's top-level children
once instead of every leaf from its parent (NodeState.RemoveChild is a
linear scan + RemoveAt, ~k²/2 compares per k-child folder); root
notifiers are cleared as a list instead of one O(F) RemoveRootNotifier
each; ForgetPredefinedSubtree / TombstoneMonitoredSubtree are iterative
with one child buffer.
- P2 RemoveEquipmentSubtree walks a parent→children index maintained by
EnsureFolder/EnsureVariable/MaterialiseAlarmCondition (keyed by the
parent id the caller named) instead of prefix-scanning six maps per
removed equipment; a surviving condition's wiring into a removed folder
is found through the folder's own notifier list; descendants are not
detached from a folder that is itself being discarded.
- P3 LowerAllEventsMonitoring(condition) replaces the full O(C) reconcile
on RemoveAlarmConditionNode / RemoveEquipmentSubtree / the kind-swap.
- P5 AlarmNotifierWiring.Folders is a reference-identity HashSet and an
already-wired pair skips the SDK's linear AddNotifier dedup.
Benchmark (NodeManagerLockTimeBenchmarkTests, M-series Mac under load,
three runs each): rebuild of a 10k-tag folder 75–150 ms → 5–6 ms;
removing 200 equipment (250 signals each) from a 50k-node space
471–681 ms → 58–64 ms; all-events bookkeeping visits for 10 condition
removals over 50 conditions 900 → 10. The two timing legs assert < 1 s;
the P3 leg asserts the visit count (machine-independent).
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 01 L2. The SDK keys its fan-out lists by NodeId and its component
cache hands back the CACHED NodeState for a NodeId it already holds, so a
rebuild or a surgical remove+re-add left a NEW item on the same NodeId —
from any session — bound to the DEAD node: created Good, fed its stale
initial value, never again, until every session had deleted its items.
Every removal path (RebuildAddressSpace, RemoveVariableNode,
RemoveAlarmConditionNode incl. the condition's instance-scoped children,
RemoveEquipmentSubtree) now queues BadNodeIdUnknown to the node's data
items, drops the MonitoredNodes entry, releases each item's component
cache reference and forgets the _liveItems/_eventItems entries. Items
the ACL pass stood down (on no fan-out list) are swept by NodeId. A later
client delete of an orphaned item is completed by the DeleteMonitoredItem
override / the SubscribeToEvents unsubscribe arm instead of answering
BadMonitoredItemIdInvalid or NRE-ing in MonitoredNode2.Remove. The
revalidation restore re-resolves the node by NodeId and no longer
re-attaches to Handle.Node blindly. Timestamps now come from the injected
Clock (UtcNow).
Tests: StaleMonitoredNodeWireTests (real SDK, three legs: rebuild,
surgical remove+re-add, second session; each keeps the old item alive,
proves the new item receives a value written after its creation, the old
one gets BadNodeIdUnknown, and its delete succeeds — all three fail with
the tombstone neutered), NodeManagerStaleMonitoredNodeTests (stood-down
item swept on surgical remove and rebuild; sibling removal leaves it
restorable).
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 01 L1/L4/H2. WriteValue/WriteValues are non-creating (an unknown
id is skipped, counted and Debug-logged, as WriteStatuses always was), so
a driver publish racing a surgical remove can no longer resurrect the id
as a flat BaseDataType phantom under the shared root — nor hand a later
re-add of the same tag the phantom's parent/DataType/writability through
EnsureVariable's early return. WriteAlarmCondition for an unmaterialised
id is a logged no-op instead of minting the legacy bool[2] placeholder
(AddPredefinedNode is AddOrUpdate, so it replaced the SDK lookup entry of
any folder sharing the id). The applier no longer writes a condition
state against every removed equipment id (a folder, never a condition).
EnsureFolder/EnsureVariable/MaterialiseAlarmCondition refuse an id already
held by another node kind in the same realm. CreateVariable is deleted.
Tests: NodeManagerPhantomNodeTests (publish racing a remove leaves no
node; the re-add gets the authored shape and is live; unknown alarm id
is a no-op; the guard in all three directions, realm-qualified); the
sink / batch-write / write-statuses tests now declare their variables.
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Clears CommentChecker findings in src/ (3,280 -> 209, the remainder being false
positives like PLC-5/IEEE-754/M0.0) by adding missing summaries/params/returns,
normalising inheritdoc usage, and removing Gitea/review tracking IDs while keeping
the rationale they annotated. Comment-only: comment-stripped sources are identical
to the parent commit.
Item 1: IHistorianDataSource.MaxRawValuesPerRead (Gateway: learned ceiling - 16 = L; 0 elsewhere, guard-tested).
RawPageLimit plans every raw page on top of #692's lookahead: with L > 0 the read (page P + boundary ties k
+ lookahead) is <= L, P = min(N, L - k - 1) (uncapped: L - k - 1); no room => straight to the tie-cluster
over-fetch (never an uncapped or underflowing request). One re-plan + re-read when the read taught a new
limit or was refused for crowding pre-window rows (HistoryReadLimitExceededException.PreWindowRows): the
page shrinks by them. Over-fetch bound min(MaxTieClusterOverfetch, L - 1); slice size never 0.
Review fixes: F1 a window starting at/before the epoch counts AVEVA's synthetic row as pre-window (a
ceiling-truncated read was reported complete); F2 the page path disables ReverseRawReader's own re-run
(cold end-only ceiling 20/N=3000 cost 3,023 reads, now <= 40); F3 crowded look-back re-read around, not
refused; F4 the #687 crowding re-read room at least doubles (up to 8 re-reads), so a cluster of hundreds
just before a resume point no longer fails on a default gateway; a crowding refusal that leaves no room
is BadTooManyOperations, never a tick-by-tick cursor.
Item 2: ServerHistorian:EventReadHorizon (default 00:00:00 = exact, unchanged). When set, every fresh
HistoryReadEvents window's earlier bound is raised to now - horizon (both directions, all backends).
Lossy by design; documented.
Item 3 (native TimeAverage/Total re-check) stays blocked on historiangw#27.
Review follow-up: on the Gateway every value is a Double (AVEVA stores a Boolean
as a 0/1 Int2), so the raw-evaluated AtTime interpolated a Boolean node to 0.5.
- IHistorianDataSource gains a default-interface overload
ReadAtTimeAsync(fullReference, timestamps, TypeCode? valueType, ct) that
forwards to the existing one (SqlServer/QuestDB/Null/fakes unchanged).
- OtOpcUaNodeManager.HistoryReadAtTime passes the node's scalar DataType
(AtTimeValueType: built-in type -> TypeCode; arrays/other types => null);
Raw and UNS NodeIds alike.
- GatewayAtTimeReader converts each raw row to the hinted kind BEFORE the shared
AtTimeEvaluator runs (Boolean = value != 0 => stepped; integers => Int64 =>
interpolated, rounded half away from zero, returned as the node's integer
type; floats Double), so the evaluator's own type branch decides.
- AtTimeEvaluator boxed every interpolated Int64 as a Double (`cond ? (long)x :
v` widens to double); its "keeping its type" test passed because Shouldly
ShouldBe compares numbers across types. Fixed (affects SqlServer/QuestDB too)
and pinned with ShouldBeOfType.
A resumed raw page asked the backend for only N rows from the boundary timestamp
(inclusive), then trimmed the boundary ties already emitted: it netted N - skip values,
and at N = 1 nothing — the empty "full" page was taken for a stalled tie cluster and a
MaxTieClusterOverfetch + 1 (65,537-row) over-fetch stepped one tick, returning an EMPTY
page with a continuation point between every value (8 values = 17 pages / 25 reads),
forward and reverse, with or without ReturnBounds, on every backend.
Each page's read now asks for N + boundarySkip + 1 rows: the ties it trims, the page and
one lookahead row. The lookahead row is the 'more data' signal, so an exact multiple ends
on its last full page (no trailing empty page). The over-fetch fires only when every row
past the trimmed ties is still on the boundary (a genuine cluster larger than the page);
a cursor left inside a cluster is flagged WithinTieCluster and over-fetches directly.
A page that never leaves the boundary adds the prior skip to its cursor.
Low items: BoundedReadHasAnythingAsync carries its walk hint and, when there is no data,
the end bound it found (HistoryEndBound.Resolved + DataExhausted), so the next page reads
nothing; HistoryBounds.IsPlaceholder is an explicit marker (ConditionalWeakTable over the
NotFound instances) instead of status+null shape, so a stored Bad_BoundNotFound row is data.
Tests: NodeManagerHistoryReadPageFillTests (N=1 every shape/direction/bounds, tie clusters
straddling boundaries at N=1/2/3, exact-multiple, carried end bound + walk hint, stored
placeholder-shaped row). Existing paging tests updated where they encoded the short pages.