Commit Graph

69 Commits

Author SHA1 Message Date
Joseph Doherty e303165444 chore(deps): LocalDb 0.3.4 -> 0.3.7 (#889, scadaproj#53)
v2-ci / unit-tests (push) Waiting to run
Pins ZB.MOM.WW.LocalDb and .Replication at 0.3.7 (published 2026-11-12).
Wire-compatible; no code change needed.

- 0.3.6 (scadaproj#36): the library sync service enforces
  LocalDb:Replication:ApiKey itself. LocalDbSyncAuthInterceptor is kept for
  its refuse-every-stream-when-blank rule; doc comment updated. docker-dev
  site-a pair: both nodes now read one variable,
  OTOPCUA_SITE_A_LOCALDB_SYNC_KEY (same dev default as before), so an
  override can never split the pair.
- 0.3.7 (scadaproj#16): a tombstone is held until the peer has it, so a
  node offline past TombstoneRetention no longer brings a pruned
  deployment back. The ledger grows while a peer is down; optional
  TombstoneHardCap (documented in Configuration.md) bounds it.

Runbook, Configuration.md and the cluster-topology rule updated.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 13:44:55 -05:00
Joseph Doherty fa811feb3d fix(opcua): drop the reflective folder-event verdict invalidation; deny the SDK channel under enforcement (#878)
v2-ci / unit-tests (push) Waiting to run
The SDK memoizes INodeManager3.ValidateEventRolePermissions per (item, event type,
SourceNode). #826 kept it fresh by reflecting into the private
MonitoredNode2.InvalidatePermissionCaches (plus a detach/re-attach fallback). Since #855
condition events never take that channel, and the search for #878 found nothing
production-raised that does except a condition's SDK method audits, which reach it only
with auditing on, SignAndEncrypt and a SecurityAdmin session (nothing grants that role).

Under enforcement the channel verdict is now a flat deny for items on our nodes, so no
cached verdict depends on a grant, and InvalidateFolderEventVerdicts, the reflection,
the fallback, its counter and the guard test are removed. Method audits still reach
Server-object items, decided per event.

Tests: FolderEventVerdictInvalidationTests -> FolderEventVerdictFreshnessTests; new
revoked-grant test (positive control: a still-granted session) and method-audit test
(SecurityAdmin mapped for the test; positive controls: Server-object item, flag off);
the plain-event test runs each mode on a fresh server. Mutation-checked.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 11:45:54 -05:00
Joseph Doherty bcde93056a perf(opcua): #870 ring-buffer event monitored-item queue
Since #855 condition events are queued to folder event items under the node
manager Lock. At a full DiscardOldest queue the SDK's EventQueueHandler
dequeues the head of EventMonitoredItemQueue, a List.RemoveAt(0) shift of the
whole queue (~1 us/item at QueueSize 1000, ~10 us at 10000).

OtOpcUaSdkServer.CreateMonitoredItemQueueFactory now wraps the SDK factory in
RingEventQueueFactory: event queues are RingEventMonitoredItemQueue (O(1)
discard), data-change queues, durable requests and disposal go to the SDK
factory unchanged. A factory set through StandardServer.MonitoredItemQueueFactory
is used as-is.

The SDK queue is storage only (decompiled 2.0.1): DiscardOldest, overflow-event
positions and discarded/overflow counters live in EventQueueHandler, which is
unchanged. The ring reproduces QueueSize 0 initial + BadInternalError enqueue,
drop-oldest at Count == QueueSize, FIFO dequeue, SetQueueSize keep-newest /
keep-oldest, the newest-1000 duplicate window via IsEventRaisedFor, no-op
Dispose. RingEventMonitoredItemQueueParityTests drive both queues, and the SDK
EventQueueHandler over both factories, with the same random sequences.

FolderEventDeliveryPerfTests gains Ring_vs_sdk_event_queue (A/B in one process
via the internal OtOpcUaSdkServer.RingEventQueues switch).

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 15:15:45 -05:00
Joseph Doherty eba016096c Merge branch 'master' into worktree-agent-a720278a57ae41792
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:26:05 -05:00
Joseph Doherty b421c7ca91 fix(runtime): #869 review round 3 — key a native-ack revert to the ack that failed
A pending local ack can outlive its own delivery (sources that send no
Acknowledge back; a Clear now only carries it through), so a later failed
re-ack — or a stale Galaxy confirmation timing out after the alarm re-occurred
and was acked again — could revert an ack it does not own. MarkAcknowledged
now issues a sequence (0 when the ack changed nothing), carried on
RouteAlarmAck and echoed on AlarmAckFailed; TryRevertAcknowledge reverts only
on a matching sequence.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:25:58 -05:00
Joseph Doherty 5001e40690 fix(runtime): #869 review round 2 — a carry-through transition no longer settles a pending local ack
NativeAlarmProjector dropped the pending local ack on EVERY driver transition,
so an operator ack followed by the alarm returning to normal (a Clear carrying
the optimistic ack) inside the confirmation window made a later "not
confirmed" report a no-op — the condition stayed inactive+acked while Galaxy
showed UNACK_RTN. Only a transition that states the ack (Acknowledge,
ActiveAcknowledged, or an un-acking Raise/Retrigger/InactiveUnacknowledged)
now settles it; a Clear / same-message Raise / Unspecified update leaves it
revertible, and the revert projects inactive+unacked.

Tests: projector + host end-to-end (ack, Clear, confirmation fails -> inactive
+ unacked); tightened the confirmed-ack positive controls; pinned that the
confirmation receives the same request instance as AcknowledgeAsync.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:19:52 -05:00
Joseph Doherty c9e5c8dcf7 Merge #855 folder event items: per-item delivery under the node manager, ConditionRefresh cut matches the Server object
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:07:21 -05:00
Joseph Doherty 4e492e6550 fix(galaxy): confirm unconfirmed native alarm acks against Galaxy's acknowledged transition (#869)
mxaccessgw answers AcknowledgeAlarm OK both when the AckMsg write was
applied and when it was only sent (no OnWriteComplete; the reply then
carries a diagnostic). EvaluateReply treated both as success, so the H6
live gate saw a condition go Acknowledged while Galaxy never applied it.

- EvaluateReply returns Applied (no diagnostic) or Unconfirmed (diagnostic)
  and logs the gateway diagnostic on every branch that carries one.
- GalaxyAckConfirmationTracker watches the alarm feed from before the RPC
  and, for an unconfirmed reply, waits up to AlarmAck.ConfirmTimeoutMs
  (default 10 s, 0 = old behaviour) for an Acknowledge/ActiveAcknowledged
  transition; on timeout it refuses (BadTimeout + diagnostic).
- New optional IAlarmAcknowledgeConfirmation: DriverInstanceActor awaits it
  after AcknowledgeAsync, off its mailbox and outside the resilience
  pipeline; a failure is the existing #760 AlarmAckFailed revert.
- AdminUI Galaxy form carries the new knob; docs updated.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:03:04 -05:00
Joseph Doherty 8a0032bfdf fix(aggregates): live TimeAverage/Total leave the span before an Uncertain first value out (#821)
When the data starts inside an interval (no start bound) and its first value is
Uncertain, the frozen 1.5.378.182 AverageAggregateCalculator integrated the
uncovered span as a ramp from 0 and divided TimeAverage by the whole interval:
GetRegionsInValueSet (ignoreBadData, sloped) overwrites the BadNoData start
region's status with UncertainDataSubNormal in its one branch without a not-Bad
guard. Live AggregateFilter therefore disagreed with HistoryRead and Part 13
§5.4.3.6/§5.4.3.8 ("No Start Bound: Calculate Partial Interval"); QP: Total
188.96 vs 163.96, TimeAverage 9.448 vs 10.931. SDK 2.0.1 stock does the same.
Owner-approved 2026-11-11.

- New app-owned CoveredTimeAverageAggregateCalculator subclasses the baseline
  and recomputes only TimeAverage/Total, restoring the Bad status of a region
  that starts at a Bad point (only the start bound can). Only such an interval's
  value changes; status, timestamps, type and bits are the baseline's. Stepped
  and TreatUncertainAsBad=true never reached the defect. CompatibilityAggregate-
  Factory routes exactly those two IDs; copied SDK files untouched.
- Golden test: 0 fixture vectors change (every data set has a usable value at
  -10 s), asserted. New sweep over late-start variants of every data set (cut
  +5 s / +25 s, first non-Bad sample Uncertain, all directions/stepping/configs/
  incremental): 150 / 142 of 384 change, each pinned to a baseline-derived
  oracle (untouched baseline's value with that sample Good, its status/bits on
  the variant itself) - never our calculator. Plus a hand-computed case.
- Live-vs-HistoryRead parity (TimeWeightedAggregator) forward + reverse, no
  masks, with discrimination against the pre-#821 baseline; QualityMix now
  compares QP's TimeAverage/Total with the live oracle on both SQL backends;
  booted-runtime facts test expects 163.96 / 10.93.
- README recorded decision, ServerAggregateConfiguration remarks, oracle
  remarks, docs/Historian.md and .claude/rules/historian.md updated.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:30:01 -05:00
Joseph Doherty d9b23e0b95 feat(opcua): per-item delivery for folder event items; folder ConditionRefresh gets the #854 cut (#855)
Condition events reaching one of our notifiers no longer go through the SDK
MonitoredNode2 channel: the walk serves the attached items itself, per item and
per event (audit guard, uncached INodeManager3 verdict, item gate/cut,
QueueEvent), as the Server-object path does. No notifier of ours defers any
more; the #838 deferral queue and the #854 cut stay as the safety net and
folder items share the cut (RefreshDirectItems, flag on and off). An event
whose walk suspends is marked deferred before it suspends.

Tests produce deferral with a notifier-stall probe instead of holding a folder
item's SDK lock (which would now block the reporter). New
FolderConditionRefreshWireTests; design note in docs/plans.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:18:11 -05:00
Joseph Doherty 233e4bab25 Merge #863 TwinCAT Test Connect: drop the AmsNetId-as-IP TCP pre-check; probe via the local AMS router ADS handshake
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:14:58 -05:00
Joseph Doherty fdc9473819 fix(twincat): Test Connect probe no longer dials the AmsNetId as an IP (#863)
TwinCATDriverProbe ran a first-phase TCP check against the AmsNetId's
first four octets on the AMS port (801/851). An AmsNetId is not an IP and
the AMS port is not a TCP port, so Test Connect timed out against every
real runtime (Z35474: dialled 5.118.56.93, an unrelated public host).

The probe now opens no socket of its own: it takes the driver's path,
AdsClient.Connect + ReadStateAsync through the local AMS router, bounded
by the probe budget (Task.Run + WaitAsync), and classifies failures as
no local AMS router / no route or target unreachable / timeout / other.
The degrade-to-green 'TCP reachability only' outcome is gone - with no
TCP check nothing was contacted, so a missing router is reported red.

Tests: the regression test listens on the AmsNetId-derived endpoint and
asserts no connection arrives (fails on the old probe); classification
and budget tests drive the handshake through an internal seam.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:13:55 -05:00
Joseph Doherty 70bc960111 Merge #809 live PercentGood/DurationGood count Good only (Part 13), matching HistoryRead
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:05:51 -05:00
Joseph Doherty ed05d50795 fix(aggregates): live PercentGood/DurationGood count Good time only (#809)
The frozen 1.5.378.182 StatusAggregateCalculator counts Uncertain regions as
Good under TreatUncertainAsBad=false, so live AggregateFilter PercentGood /
DurationGood disagreed with HistoryRead (Part 13 §5.4.3.31/§5.4.3.33, Mantis
0006166) whenever Uncertain data was present. Owner-approved 2026-11-11.

- New app-owned GoodOnlyStatusAggregateCalculator subclasses the baseline and
  replaces only the PercentGood/DurationGood computation (Good regions only);
  CompatibilityAggregateFactory routes exactly those two IDs to it. The copied
  SDK files are untouched; the other 35 aggregates are byte-identical.
- Golden test: the 64 changed vectors (38 PercentGood + 26 DurationGood) are
  exempted exactly — those whose baseline output differs from the same case's
  baseline TreatUncertainAsBad=true twin — and pinned to that twin (baseline
  evidence, not regenerated from the implementation); plus a full Good-only
  pin over all 352 PercentGood/DurationGood vectors and a hand-computed case.
- Live-vs-history parity over Good/Uncertain/Bad mixes (no masks), with a
  discrimination check against the pre-#809 baseline; the booted-runtime facts
  test now expects 8000 ms / 40 %.
- SdkAggregateOracle: Run/RunLive = the registered factory; RunSdkAsIs = the
  untouched baseline; its Part13GoodCalculator duplicate is removed.
- README (recorded decision), ServerAggregateConfiguration remarks,
  docs/Historian.md and .claude/rules/historian.md updated.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:05:34 -05:00
Joseph Doherty e6ec5a0948 docs(opcuaclient): keep server host-name check opt-in; surface it in the Security panel (#837)
Owner decision 2026-11-11: certificateValidation.validateServerHostName stays
opt-in (unset = off). The AdminUI toggle moves from the "Subscriptions, limits &
discovery" panel to the Security panel beside mode/policy, with help text on the
NAT/IP caveat, IPv6 zone ids and the loose first-label match. Binding unchanged
(tri-state, "Default (off)" leaves the key absent). OpcUaClient.md states the
decision; drivers rule updated; source-scan test pins placement + caveats.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 11:57:59 -05:00
Joseph Doherty bdbc47f5fc fix(mtconnect): serve an empty string-typed value as Good "" (#824)
Owner decision 2026-11-11 (Java parity): a genuinely empty observation on a
String/Reference or unauthored item is Good "" instead of BadNoCommunication.
Non-string items (numeric, bool, DateTime, arrays) keep BadNoCommunication -
the Agent supplied no value; BadTypeMismatch would wrongly blame the authored
type. The UNAVAILABLE sentinel is unchanged, and a structured observation with
no text stays BadNotSupported (structured check now precedes the empty check).
Parity matrix, quality-mapping section and drivers rule updated.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 11:56:30 -05:00
Joseph Doherty 4bb9c453a3 test(opcua): event-queue overflow with the per-folder/Server-object ACL active (#838 item 2)
EventQueueOverflowAclWireTests drives >4096 condition events at a folder notifier
item and a Server-object (i=2253) item with Security:Acl enforcement on, a
non-publishing client, and a second session reading a node-manager node throughout:

- denied (EqA-owned scripted) events interleaved with allowed native ones, under one
  Lock hold and per-projection: no deadlock, reads stay bounded, deferral peak stays
  within 2 x (MaxDeferredEventsPerCondition + 1), delivery <= item QueueSize, oldest
  dropped / newest kept in report order, one EventQueueOverflowEventType published ahead
  of the survivors (SDK DiscardOldest, Part 4 5.13.1.5), zero denied events delivered;
- a grant revoked (state.Apply -> revalidation, re-entering Lock) while a deferred tail
  waits behind a full folder channel: neither the tail nor later events reach the
  revoked session's folder or Server-object item; a still-granted session on the same
  channel receives all of it.

Falsified: with enforcement off (and with the revocation removed) all three cases fail.
No production defect found.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 10:35:12 -05:00
Joseph Doherty 03e5145796 Merge #854 ConditionRefresh not overtaken by deferred condition events (Server-object items)
v2-ci / unit-tests (push) Failing after 1h12m23s
2026-11-11 07:39:25 -05:00
Joseph Doherty fe48458a41 fix(opcua): Server-object ConditionRefresh off Lock unless deferring; cut path O(conditions) with replay queued off Lock (#854 review)
Review of 1622d07f:
1. Perf: every i=2253 refresh ran the whole notifier walk + permission checks +
   QueueEvent under Lock. Now, with nothing deferred (the common case), the refresh is
   the SDK-shaped walk off Lock (Lock only to copy the root notifiers), kept only if no
   deferral episode existed at any moment of it (count 0 before and after, episode
   unchanged); otherwise the cut path also runs and its later replay supersedes. The
   cut path takes ACL/role verdicts off Lock, then one short Lock hold: watermark +
   frozen snapshots of Retain conditions from _alarmConditions (no tree walk) + per-item
   cut and "replay pending" marker; the replay is queued OFF Lock, one event per hold of
   the item gate. While pending, a condition event newer than the cut marks its
   condition so the older frozen entry is dropped rather than following it. A
   concurrent fast-path refresh cancels a pending cut replay of the same item.
   Measured, 3000 retained conditions, longest Lock wait seen by another thread during
   one refresh: 1622d07f 95-106 ms idle / 102-127 ms deferring; now 0-3 ms idle /
   23-32 ms deferring.
2. ACL: a retained condition omitted from the replay for want of a grant is never
   withheld (RefreshCut.RetainedAtCut); only Retain=true events of a condition omitted
   because it was NOT retained are.
3. One gate/cut state per item for its lifetime (ConditionalWeakTable keyed by the
   item): an unsubscribe + resubscribe between refresh and drain keeps the cut.
4. Cuts carry their deferral episode; ClearStaleRefreshCuts runs when the count
   returns to zero, and a stale-episode cut is inert.
5. Dispose guard for the custom refresh (ObjectDisposedException once disposing;
   Dispose waits for refreshes in flight).
6. Docs: a refresh waits at most for one in-progress QueueEvent on the item.

Tests: resubscribe variants, ACL-grant-lands test, pending-replay probe test,
dispose test, 3000-condition Lock-hold measurement with a probe pinning that neither
path queues its replay under Lock; stress invariant tightened (never backwards outside a
refresh; each refresh ends on the newest delivered state) and tolerant of the SDK's own
BadRefreshInProgress overlap refusal. Each new test fails against its targeted mutation.
2026-11-11 07:28:08 -05:00
Joseph Doherty 1622d07f2e fix(opcua): ConditionRefresh on a Server-object item is not overtaken by older deferred condition events (#854)
While a folder subscriber's SDK channel is full, a condition's later events wait in
its #838 deferral queue and reach i=2253 items only when the drain gets to them, off
Lock. A ConditionRefresh in that window replayed the current state, then the older
deferred events followed RefreshEnd (stale state, or a condition the refresh had just
dropped re-added until the newest drained).

- Condition events are reported as ConditionEventSnapshot (an InstanceStateSnapshot
  carrying a report sequence, condition NodeId and Retain); events queued behind a
  deferred report are marked Deferred.
- Server-object refresh (ConditionRefresh and ConditionRefresh2, flag on AND off) now
  runs in RefreshServerObjectItems: watermark, "anything deferred?", the replay and its
  queueing in ONE Lock hold, so the replay is exactly the state at the cut. Per item it
  queues the replay and installs a RefreshCut under the item's gate.
- OnConditionReportEvent holds that gate across "superseded? -> QueueEvent" for
  deferred events only. Withheld from that item alone: events at/below the cut whose
  condition the replay re-sent, or Retain=true events of a condition it omitted; an
  older Retain=false event of an omitted condition is still delivered. Queues, other
  Server-object items and folder items are untouched; refresh never waits for a drain.
- Lock order Lock -> item gate -> SDK item lock, same as a synchronous report.
- Meter otopcua.opcua.condition_events.superseded.
- Folder items keep a documented residual (SDK channel backlog + memoised per-item
  verdict leave no per-event hook).

Tests: ConditionRefreshDeferralWireTests (deterministic overload by holding the folder
item's SDK lock; 6 variants fail before the fix; probe-driven gate test fails with the
gate removed; concurrent reports/refreshes/stall stress test fails before the fix).
2026-11-11 06:37:22 -05:00
Joseph Doherty 26349320df Merge #848 #849 #850 credential name claim/pin, env: per-instance rule, merged preflight 2026-11-11 06:36:06 -05:00
Joseph Doherty ea156265a2 docs(security): id-form matching, ASCII-only name form, one-save pin side-step, rename/device-save race (#848 #849 review)
Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-11 06:35:52 -05:00
Joseph Doherty 340d9d26b7 feat(mtconnect): CommsLossConfirmMs AdminUI field, refused-connection fast path reaches native conditions, dedicated live-gate Agent (#852)
- AdminUI MTConnect driver form: "Comms-loss confirm (ms)" in the Reconnect section, always written
  (0 = first refusal, negative = off is never dropped), case-insensitive load, unknown keys preserved,
  advisory at >= 90000 ms (the watchdog fires first). Form output byte-pinned and parsed through
  MTConnectDriver.ParseOptions on the driver side.
- Driver: a confirmed refused-connection outage now annotates every told CONDITION quality-only
  BadNoCommunication (state/ack kept, once per outage, UNAVAILABLE-Bad skipped); the first delivery
  afterwards re-diffs all subscribed conditions so unchanged ones get a quality-only Good.
- docker-dev/docker-compose.mtconnect-gate.yml: throwaway Agent in its adapter's network namespace so a
  stop is a genuine ConnectionRefused (a plain stopped container is a DNS failure = ambiguous).
- Docs: MTConnect.md comms-loss section + live-gate recipe, docker-dev README, drivers.md rule.

Refs #852
2026-11-11 06:23:18 -05:00
Joseph Doherty 7482a88888 fix(opcua): deferred condition events never outlive their condition; per-condition bounded deferral; canonical lockout identity (#838, #844 review)
#838 review of 91e28e87:
1. Ghost events. The deferred report ran the SDK walk off Lock with no
   liveness check, so an event waiting behind a full channel when a deploy
   removed its condition still reached i=2253 subscribers after the NodeDeleted
   (a ghost Retain alarm for ScadaBridge) and folder items. Delivery now walks
   the SDK path itself (OnReportEvent, OnReportEventAsync, inverse notifiers,
   then the Parent as BaseInstanceState does) re-checking before EVERY hop that
   the condition is the live instance; OnConditionReportEvent checks the same,
   which also stops a non-live scripted condition being admitted through its
   old equipment folder (#808). A guard test pins that only NodeState and
   BaseInstanceState define the walk. New wire test removes the condition in
   the same Lock hold as a flood; the flag-off variant fails without the checks.
2. Back-pressure. Deferral is per condition (one condition's events stay
   ordered; others are not held up). Overflow policy: coalesce, newest wins --
   past 1024 waiting per condition or 65,536 total, the oldest not-yet-started
   event is discarded (a condition event carries full state). Warning on
   entering deferral (<=1/min), Information on draining, meters
   otopcua.opcua.condition_events.deferred / .discarded{reason}.
3. Verified by decompiling SDK 2.0.1: the notifier list is snapshotted under the
   node's own lock, MonitoredNode2.Dispose completes its writer (pending writes
   fail with ChannelClosedException, swallowed), and the consumer catches
   per-event exceptions -- no fix needed; documented.
4. ConditionRefresh does not flush deferrals; documented as a residual.

#844 review: DescribeCredential canonicalises the key id (trim, lowercase;
non-hex ids fold into one identity) and the username, since ConfigDbApiKeyStore
matches with SQL Server's case-insensitive, trailing-space-blind `=`.
2026-11-11 04:35:30 -05:00
Joseph Doherty 214813bbc8 Merge #835/#836 review fixes (owner check on unsaved endpoints, credential-use audit) 2026-11-11 04:13:02 -05:00
Joseph Doherty 9072777422 docs(security): #835/#836 review — unsaved-endpoint rule, credential-use audit, accurate residuals, AuthType-gated path rule, directory resolution
Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-11 04:12:40 -05:00
Joseph Doherty 2fc9e02b9a Merge #844 #845 #838 credential lockout per identity, non-blocking condition events 2026-11-11 04:08:13 -05:00
Joseph Doherty 91e28e87be fix(opcua): condition events never block under the node-manager lock; folder-event Part 3 note (#838)
#838(a): each notifier's SDK MonitoredNode2 drains a bounded channel (4096,
FullMode=Wait) and the synchronous NodeState.ReportEvent blocks when it is
full. Condition events are reported under Lock, and the delivery task takes
Lock on a verdict-cache miss (the SDK's own GetNodeMetadata, flag off too, and
#826's IsNotifierEventVisibleTo). A burst of >4096 events under one Lock hold
therefore deadlocked the node manager -- FolderEventFloodWireTests stalled at
exactly 4097 events with enforcement on AND off (so it predates #826).

ReportConditionEvent now goes through ReportEventWithoutBlocking:
ReportEventAsync completes inline while channels have room (unchanged normal
path); on overflow the report continues asynchronously holding no thread or
lock, and later reports queue behind it so events keep report order. The
deferred work holds only frozen snapshots and drains once Lock is released;
client delivery stays bounded by each item's queue.

#838(b): documented in docs/security.md that Part 3 ReceiveEvents applies to
the event's EventType and SourceNode (the SDK check, kept first), not to the
notifier; our decision is layered on top and non-condition events on a folder
item fail closed under enforcement. The reflection item stays open.
2026-11-11 04:06:28 -05:00
Joseph Doherty 87416a230f Merge #835 #836 per-instance credential references, confined user-certificate path 2026-11-11 03:53:12 -05:00
Joseph Doherty 0810e808c9 docs(security): per-instance credential references (#835) and the user-certificate directory (#836)
Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-11 03:52:48 -05:00
Joseph Doherty 19ffb3c902 feat(mtconnect): refused Agent connections mark values BadNoCommunication after a short confirmation; clamp the raised sample count to the Agent's bufferSize (#846)
- A /sample or re-baseline failure whose chain holds SocketError.ConnectionRefused, unbroken for
  CommsLossConfirmMs (new, default 2000; 0 = first refusal; negative = off), publishes every subscribed
  value BadNoCommunication once per outage; the first delivery afterwards republishes all of them from
  the index (unchanged values included). Timeouts/resets/DNS/pause keep the #527 watchdog contract.
- SampleCountCeiling clamps the adaptive raise to the /probe bufferSize (never below SampleCount).
- Docs: MTConnect.md comms-loss section + config key; drivers.md rule.

Refs #846
2026-11-11 03:48:05 -05:00
Joseph Doherty ffba75bb93 fix(opcua): credential lockout counts per credential identity, ignores transient denials (#844, #845)
#844: a success cleared the whole per-client count, so a client holding a valid
key could make four guesses at another key id, sign in with its own, and repeat
forever. Failures are now charged to the (client, credential identity) pair --
IOpcUaUserAuthenticator.DescribeCredential, key:<keyId> for API keys -- and to
the client as a whole. A success clears only its own identity's count; the
per-identity threshold locks that identity; a new per-client cap
(OpcUa:CredentialLockout:MaxFailedAttemptsPerClient, default 2 x
MaxFailedAttempts) that a success never resets bounds spraying across key ids
and locks every credential. Slots are bounded on both counts; a waiter whose
identity has room is not held behind one whose identity has none.

#845: (1) store timeouts, miss-budget refusals, store/authenticator exceptions,
an unset pepper and cancellation are reported Transient
(OpcUaUserAuthResult.Unavailable) and released uncounted, so a ConfigDb stall
can no longer lock every keyed client out; the client still sees the same
opaque denial. (2) a full 4096-entry table defers its next prune until the
earliest entry could age out instead of rescanning under the lock per failure.
(3) ApplicationUri deliberately NOT added to the key: on a None channel it is a
free string, so it would let an attacker reset the count per guess --
documented. (4) wire tests: distinct client certificates on a Sign endpoint are
locked out separately; the ApplicationConfigPath branch keeps the SDK lockout
off (falsified by removing the loaded-branch disable).
2026-11-11 03:47:10 -05:00
Joseph Doherty ddc1f29e3a feat(heidenhain): Part 9 condition class from alarm class/group, ALARM: selectors in the browse picker, unclaimed alarms counted + logged (#839)
- Core: optional AlarmEventArgs.ConditionClass (AlarmConditionClass Base/Process/Maintenance/System/Safety)
  carried by NativeAlarmProjector on AlarmConditionSnapshot and written to ConditionClassId/Name by
  OtOpcUaNodeManager (kept across class-less events, carried across a #641 rebuild).
- Heidenhain: HeidenhainAlarmClasses — HEIDENHAIN DNC-interface class/group names (corroborated by the
  captured iTNC 530 record); class 5 => Safety, group 1/2 => Process, 4/5 => System, else Base.
- DiscoverAsync offers AlarmConditions/ selector leaves (IsAlarm Boolean); browse-commit seeds the alarm object.
- Alarms no selector claims: otopcua.driver.alarms.unmatched counter, /hosts diagnostics, rate-limited Warning.

Refs #839
2026-11-11 03:41:14 -05:00
Joseph Doherty 2314a53842 fix(opcua): parallel UserName activations wait for a slot instead of being refused (#833)
A legitimate consumer opening several keyed sessions at once from one key
(e.g. a ScadaBridge site node behind one NAT IP on a None endpoint) must
never see BadUserAccessDenied for parallelism. BeginAsync now hands out at
most MaxFailedAttempts - recordedFailures verification slots per key;
further activations queue FIFO and are woken as slots free. A waiter is
refused only when the key locks out (never reaching the authenticator) or,
uncounted, after OpcUa:CredentialLockout:SlotWaitSeconds (default 10,
BadTimeout). Cancelled waiters are dequeued; pruning accounts for waiters.
The security property is unchanged: no more than MaxFailedAttempts
unverified guesses in progress-or-failed per window.
2026-11-11 02:43:38 -05:00
Joseph Doherty 61b9adeb9c Merge branch 'master' into fix/832-docker-dev-batch 2026-11-11 02:32:42 -05:00
Joseph Doherty 988927c8ac fix(opcua): credential lockout review fixes (#833)
- Reserve an in-flight slot per client before authenticating (TryBegin /
  Complete / Abandon under one lock); refuse once recorded failures +
  in-flight attempts reach MaxFailedAttempts, so a parallel burst cannot
  pass the check before its first failure is recorded.
- An expired lockout resets the count (LockoutSeconds < FailureWindowSeconds
  relocked on every single failure after the first lockout).
- Validate OpcUa:CredentialLockout at host start: disabled, or both
  durations > 0.
- Log the first refusal of a lockout episode at Information, the rest at
  Debug.
- Document that on None endpoints the key is effectively IP-only (NAT
  neighbours share a UserName count) and a null peer shares one key.
2026-11-11 02:32:40 -05:00
Joseph Doherty 5bfe8386e6 Merge fix/826-folder-event-acl 2026-11-11 02:25:44 -05:00
Joseph Doherty 489664d105 fix(acl): folder event verdict invalidation reaches orphaned items, decides on the live instance, skips ACL-off; alarm-bearing reference map (#826 review)
1. Orphaned subscriptions (session closed without deleting subs) have Session == null, so the public
   MonitoredNode2.InvalidatePermissionCacheForSession never reached their items: a cached allow survived a
   deploy, the moved alarm's events were queued and delivered on TransferSubscriptions. Each notifier's
   cache is now cleared whole through the SDK's private MonitoredNode2.InvalidatePermissionCaches
   (reflection, guard-tested; fallback = per-session drop + Remove/Add of session-less items).
2. The verdict is decided on the condition LIVE under the event's NodeId, not the event's own instance:
   the SDK key has no instance identity, so a replaced instance's late event cached a deny its successor
   inherited until the next invalidation.
3. Invalidation is skipped while enforcement is off (forced across an on/off authorizer transition) and is
   one O(cache) clear per node instead of a whole-cache scan per item.
4. TagReferencingEquipment carries alarm-bearing raw tags only (TagConfigIntent.Alarm); the snapshot adopts
   an OrdinalIgnoreCase set instead of copying it again.

Tests (each fails with its fix reverted): FolderEventVerdictInvalidationTests (orphan + deploy + transfer;
replaced instance's late event; ACL-off skip; SDK private-method guard), DeploymentArtifactScopePathTests
(alarm-bearing projection).
2026-11-11 02:25:24 -05:00
Joseph Doherty e9064dbdf4 Merge fix/815-credential-ref-scoping
# Conflicts:
#	.claude/rules/drivers.md
#	docs/drivers/Heidenhain.md
2026-11-11 02:08:24 -05:00
Joseph Doherty 4563940792 fix(security): warn-only compatibility for deployed drivers + upgrade preflight (#815)
User decision: save, Test Connect, live browse and the deploy gate keep
refusing out-of-namespace credential references, but an ALREADY-DEPLOYED
driver must not fault at the upgrade restart or (Galaxy) at the next
gateway reconnect, since ReopenAsync re-resolves Gateway.ApiKeySecretRef.

- LegacyCredentialReferenceTracker (Core.Abstractions): the one, deprecated,
  call-site-chosen compatibility path. Running Galaxy (InitializeAsync,
  ReopenAsync, ack fallback) and OpcUaClient (session open) pass it and keep
  the pre-#815 semantics (case-insensitive prefixes, Galaxy file:, OpcUaClient
  literal pass-through) with a rate-limited Warning (driver/field/allowed
  form, never the value) and a credential-compat:<field> diagnostics entry
  (both drivers now implement IDriverDiagnostics). Browsers and probes pass
  none and stay hard-fail; Heidenhain has none.
- Deploy-gate message names the migration.
- Upgrade preflight: CredentialReferencePreflightService (admin nodes, once
  after ApplicationStarted, ConfigDb only, resolves nothing) logs a Warning
  per Draft row / Deployed-artifact finding; the /raw banner shows them plus
  Running drivers reporting warn-only use (driver-health diagnostics).
- docs/security.md: exact replacement table (incl. the old form default
  env:MX_API_KEY, file:, un-namespaced secret:, casing changes), the
  ReopenAsync reconnect behaviour, the warn-only window and its removal.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-11 02:07:33 -05:00
Joseph Doherty d3285af6b3 Merge feat/830-heidenhain-followups 2026-11-11 01:55:10 -05:00
Joseph Doherty 598c668108 Merge fix/816-opcuaclient-cert-host 2026-11-11 01:53:35 -05:00
Joseph Doherty 9aff90f884 fix(opcuaclient): never pass checkDomain for a None endpoint; document both SDK check sites (#816 review)
SDK 2.0.1 checks the server host name in TWO places: Session.OpenAsync (after
the chain is trusted) and DefaultSessionFactory.CreateChannelAsync, against the
certificate in the GetEndpoints reply and before any security-mode test. opc-plc
sends a certificate on its None endpoint, so `connect -u
opc.tcp://10.100.0.35:50000 --validate-host-name` was refused on an
unauthenticated channel, where a host check proves nothing.

All three call sites now pass checkDomain only when the selected endpoint's
SecurityMode != None: OpcUaEndpointSelection.CheckDomainFor for the driver
(OpenSessionOnEndpointAsync, so the failover sweep too) and the AdminUI browser;
the same expression in Client.Shared ConnectToEndpointAsync (shared by
failover). New live tests: driver None + flag on vs :50000 reads; browser None +
flag on vs the NAT fixture browses; the secure + flag refusals still hold.

Docs: name both check sites; a zone-qualified IPv6 host never matches (FindDomain
compares the zone too), so link-local needs validateServerHostName=false; the
match is loose (first label alone, subject DC= parts count, no wildcards), which
caveats the CA recommendation.
2026-11-11 01:53:01 -05:00
Joseph Doherty 8e1e73f93c fix(acl): per-event visibility for direct folder event items; generation-checked native wiring (#826)
Event items on an equipment / raw device folder were fed by the SDK's MonitoredNode2 fan-out with no ACL
decision, so after a deploy moved a scripted alarm (or dropped the reference projecting a native alarm into
an equipment) the OLD folder's subscribers kept receiving it - EventId included - until the rebuild landed,
indefinitely after a failed apply.

- OtOpcUaNodeManager re-implements INodeManager3 so ValidateEventRolePermissions (the SDK's per-event,
  per-item hook; base not virtual) adds IsNotifierEventVisibleTo: live condition instance, notifier still a
  legitimate notifier of it, and the #586/#808 Server-object rule (IsConditionVisibleTo over
  ConditionVisibilityScopes).
- The SDK memoizes that verdict per (item, event type, SourceNode): InvalidateFolderEventVerdicts runs on the
  authority/authorizer setters, RevalidateMonitoredItems (deploy + timer) and CompleteAddressSpaceApply.
- ConditionRefresh on direct items goes through RefreshDirectItems (base loop + the same decision).
- New generation field TagReferencingEquipment (RawPath -> EquipmentIds, from the artifact's
  UnsTagReferences via AclScopePathResolver.ResolveTagRawPath): a native alarm's wired equipment folder only
  counts (folder delivery AND Server-object scopes) while the current generation still references it.
- Secondary #808 edge: UNS variables wired UNS->Raw Organizes by AddReference are tracked and always placed
  as reference variables (an Area named like its equipment's id no longer narrows them).

Tests: UnsFolderScopeWireTests (moved alarm on a folder item + ConditionRefresh + rebuild; area named like
equipment id), NativeAlarmFolderEventAclWireTests (dropped reference: folder + Server object + refresh +
restore), DeploymentArtifactScopePathTests (projection). Rig/event tests now supply tag references.
2026-11-11 01:39:06 -05:00
Joseph Doherty 8a6f0e3c84 fix(mtconnect): the Agent refuses count ABOVE bufferSize, not at it (#825)
Live gate against the cppagent 2.7 fixture: count=16384 with bufferSize
16384 streams; 16385 is OUT_OF_RANGE ('count' must be less than 16385).
Warn only when SampleCount > bufferSize, pin both sides of the edge
against the live Agent, and correct the #804 wording that said 'at or
above'.
2026-11-11 01:38:48 -05:00
Joseph Doherty 2b0eed1a8f feat(heidenhain): controller alarms as native Part 9 conditions; rediscovery on identity change (#831)
- ALARM: selectors (HeidenhainAlarmSelector, Contracts): ALARM:*, ALARM:OTHER,
  ALARM:CLASS=<r>[/GROUP=<r>][/NUMBER=<r>] (<r> = n | lo..hi; negative numbers allowed).
  An ALARM: address and the cross-driver alarm object only work together — the factory
  refuses either alone, so editor, save path, CSV and deploy inspector agree. Data type
  fixed Boolean; never a value (ReadAsync BadNotSupported), never in a data subscription.
- HeidenhainDriver implements IAlarmSource: one condition per ALARM: tag at its RawPath,
  driven by the SAME allowlisted R_RI 27/28 walk on the control's single loop, on the new
  Polling.AlarmMs cadence (default 2000 ms, only for controls carrying an ALARM: tag).
  FOCAS #647 transition rules (Raise / Retrigger / Clear), identity = class+group+number+text,
  deterministic display order, told-record on the driver instance, recorded only while a
  handler listens. A failed walk (refusal, overflow, comms loss) is quality-only, never a clear.
- Acknowledge is LOCAL-ONLY: AcknowledgeAsync is a no-op success (FOCAS/MTConnect posture);
  Lsv2ReadOnlyPolicy is unchanged — no ack or write request exists.
- Opt-in per tag, no driver flag (FOCAS #647 / MTConnect #626 precedent); Alarms/* arrays unchanged.
- IRediscoverable: a reconnect whose R_VR identity (model / NC / PLC / options) differs from
  the previous session raises OnRediscoveryNeeded once (advisory re-browse chip). No extra traffic.
- AdminUI: Heidenhain joins NativeAlarmModel's alarm drivers; the typed editor fixes ALARM:
  tags at Boolean and seeds/drops the alarm object with the selector; driver form gains
  "Alarm conditions (ms)". (HeidenhainDriverFormModel also carries the #830 credential-notice change.)

Tests: HeidenhainAlarmTests (grammar, pairing, rules, simulator e2e with the captured iTNC 530
alarm record, local-only ack = zero bytes sent, quality-only outage/refusal, rediscovery),
Runtime.Tests HeidenhainNativeAlarmEndToEndTests (real DriverHostActor condition), a
DraftValidator case, and the NativeAlarmDriverSetGuard now scans the Heidenhain driver.
2026-11-11 01:37:41 -05:00
Joseph Doherty 9742a66a9e docs(security): credential-reference namespaces + migration note (#815)
docs/security.md gains the canonical table of per-driver namespaces, the five
enforcement seams, the out-of-scope cases (appsettings ${secret:}, Sql
connectionStringRef) and the migration steps for existing configs. Galaxy,
OpcUaClient, Heidenhain, Configuration and driver-specs docs, the Galaxy
options XML docs, the drivers rule file and the docker-dev compose comment
point at it. docker-dev's env:GALAXY_MXGW_API_KEY stays valid (allowed legacy
prefix), so no rig change is needed.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-11 01:37:25 -05:00
Joseph Doherty ae91e6b7f7 feat(opcuaclient): opt-in server-certificate host-name check; NAT fixture keeps its PKI (#816, #818)
#816 finding: the SDK checks that the server certificate names the host only
when a session is created with checkDomain:true. The driver, the AdminUI
browser and Client.Shared all called the 8-argument
DefaultSessionFactory.CreateAsync, which forwards checkDomain:false — so no
host-name check ever ran (why a trusted NAT cert for *.invalid browsed fine
over an IP in #813). When enabled the SDK (Session.OpenAsync ->
ValidateDomains -> FindDomain) compares ConfiguredEndpoint.EndpointUrl.IdnHost
— the CONFIGURED host, because OpcUaEndpointSelection/EndpointSelector
rewrote the advertised one — against the cert's DNS/IP SANs; secured
sessions only, and the auto-accept handler accepts a mismatch.

Policy: certificateValidation.validateServerHostName, tri-state. Unset =
OpcUaCertificateValidationOptions.DefaultValidateServerHostName = false, the
pre-#816 behaviour, so nothing changes on upgrade; true = validate the
configured host; false = explicit NAT/port-publish opt-out. Threaded into the
driver, the AdminUI browser (same JSON) and the AdminUI form (a select that
keeps "unset" unset), Client.Shared ConnectionSettings.ValidateServerHostName
and the CLI --validate-host-name. A driver refusal names the opt-out.

#818: the NAT fixture's /app/pki is on the opc-plc-nat-pki volume (verified:
same thumbprint across --force-recreate), its PUBLIC cert is committed as
Docker/nat/opc-plc-nat-server.der, and NatPublishedEndpointBrowseTests run
every secure browse on a throwaway PKI (internal browser seam): refusal with
nothing trusted, positive with the cert trusted (unset/false host check), and
a BadCertificateHostNameInvalid refusal with the check on. The trusted cert is
OPCUA_NAT_SERVER_CERT or the committed one (skip on thumbprint mismatch);
OPCUA_NAT_SERVER_CERT_TRUSTED is retired. CI exports its throwaway NAT
container's public cert. ServerHostNameValidationTests pin the driver against
the plain opc-plc. MIN_EXECUTED not raised (needs a hosted run): +6 in
v2-integration (440 -> 446), documented in scripts/ci/README.md.
2026-11-11 01:33:00 -05:00
Joseph Doherty 1a4d46a702 docs(historian)+infra: shared QuestDB fixture, correct SQL test credential, DurationGood/Bad + #821 docs (#822, #820, #821)
#822: HISTORIAN_SQL_TEST_CONNECTION example now targets the shared host
10.100.0.35,14330 with its committed dev sa password (OtOpcUaDev_2026!); the
docker-dev rig's own localhost,14330 password (OtOpcUa!Dev123) is called out as
a different instance. New labelled (project: lmxopcua) QuestDB fixture stack
tests/.../Historian.QuestDb.IntegrationTests/Docker/docker-compose.yml
(questdb/questdb:10.0.1, PG wire :8812, console :19000), deployed to
/opt/otopcua-questdb on the Docker host; documented in infra/README.md §2a,
docs/Historian.md, CLAUDE.md endpoints and the historian rules.
Also corrects infra/README's stale "only MQTT/NAT carry the label" note.

Docs for #820 (nineteen served aggregates, DurationGood/DurationBad in ms,
Gateway still refuses) and the #821 facts (Part 13 vs frozen live vs stock
SDK 2.0.1, both directions; decision open with #809).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-11 01:29:01 -05:00
Joseph Doherty 40969e5c4b fix(opcua): failed sign-ins no longer lock a client out of anonymous sessions (#833)
OPC Foundation SDK 2.0.1's SessionManager lockout refused EVERY
activation (Anonymous included, before the token is read) from a client
with 5 failed identity validations in 60 s, for 5 minutes, keyed on cert
thumbprint else peer IP. Disable it (MaxFailedAuthenticationAttempts=0
on built and loaded configs) and enforce FailedCredentialLockout in the
identity-registry adapter instead: UserName activations only, keyed on
thumbprint+peer together, same 5/60s/300s defaults, configurable via
OpcUa:CredentialLockout.
2026-11-11 01:23:33 -05:00