Commit Graph

1169 Commits

Author SHA1 Message Date
Joseph Doherty b61f3e0560 fix(galaxy): AVEVA priority 1 is the most urgent: Galaxy alarm Severity follows AVEVA's priority bands, live and history alike (#887)
v2-ci / unit-tests (push) Has been cancelled
GalaxyAlarmPriority, the one table live Galaxy alarms (MxAccessSeverityMapper)
and AVEVA A&E history (EventMapper, #886) map the AVEVA priority through, ranked
a higher number as more severe. AVEVA ranks priority 1 as the most urgent, so a
priority-1 alarm read 250 and a priority-999 system event 1000.

It now uses AVEVA's default priority ranges, most urgent first:

  AVEVA priority  AVEVA severity  bucket    OPC UA Severity
  1-250           1 Critical      Critical  1000
  251-500         2 High          High       750
  501-750         3 Medium        Medium     500
  751-999         4 Low           Low        250
  (history only)  0 not an alarm  -            1

Out-of-range priorities clamp (<= 0 Critical, > 999 Low). Evidence, vd03
Runtime.dbo.Events read-only: priority 1/100 alarms carry AVEVA severity 1,
400/500 severity 2, 600/700 severity 3, and every priority-999 System.* /
User.* row severity 0 with IsAlarm 0.

- Live and history now use the same number. GalaxyDriver passes the table's
  exact value as AlarmEventArgs.OpcUaSeverity. Before, it passed only the bucket,
  which NativeAlarmProjector re-mapped through its generic 200/500/700/900
  ladder, so live and history disagreed even before the direction fix.
- History: a row whose AVEVA severity is 0 (System.Start/Stop/Deploy,
  User.Write...) is not an alarm and reads Severity 1, below every alarm band.
  For alarms AVEVA's 1-4 severity is still never the value: the live feed
  carries only the priority, so it cannot use it, and history must agree.
- Docs: docs/Historian.md gains "Galaxy alarm priority -> Severity
  (lmxopcua#887)"; docs/AlarmTracking.md updated.
- Live re-run of the #801/#886 harness against vd03 HistorianGateway
  ea9f21c (read-only, network-only): 224 of 233 cases identical to the #886
  run, the 9 others differ only in Severity (priority-700 alarm rows 750 ->
  500, 19 system/user events 1000 -> 1). Record: aveva-history-801.md
  "#887 re-run", report aveva-gate-sdk2-887.json.gz.

Client-visible: every OPC UA client that ranks or filters Galaxy alarms by
Severity sees the order reversed (live and HistoryReadEvents). ScadaBridge has
no code that depends on the old values; its own mixed severity directions are
ScadaBridge#129.

Tests updated because they pinned the inverted table:
- Driver.Galaxy.Tests MxAccessSeverityMapperTests (rewritten: edges 1/250/251/
  500/501/750/751/999, 0/-100/int.MinValue/1000/int.MaxValue clamps,
  monotonic-decreasing sweep, equals GalaxyAlarmPriority for -2..1001)
- Driver.Galaxy.Tests GatewayGalaxyAlarmFeedTests (750 and 600 -> Medium)
- Driver.Historian.Gateway.Tests EventMapperTests (band table, priority 999
  -> 250, 1 -> 1000; system events -> 1)
- OpcUaServer.IntegrationTests GalaxyEquipmentEventHistoryWireTests (700 ->
  500, 100 -> 1000, System.Stop -> 1; the fake now stores AVEVA's real 0-4
  severity)

New tests:
- Runtime.Tests NativeAlarmRestatementEndToEndTests
  .Galaxy_live_condition_severity_is_the_AVEVA_priority_band (8 rows: the real
  feed -> real GalaxyDriver -> host -> projector; the same rows EventMapperTests
  pins on the history path)
- EventMapperTests.Aveva_severity_zero_marks_an_event_that_is_not_an_alarm
  (5 rows, incl. the priority-999 alarm positive control)
- GalaxyDriverAlarmSourceTests: OpcUaSeverity passes through

Mutation check, 8 mutants, all killed: the pre-#887 inverted table; the
driver dropping OpcUaSeverity; no not-an-alarm rule; each of the three band
edges off by one; history ignoring priority; a missing AVEVA severity read
as 0.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 12:14:49 -05:00
Joseph Doherty 3a312ffdd1 fix(history): a folder's event history reads the Galaxy objects behind its native conditions (#886)
v2-ci / unit-tests (push) Has been cancelled
A notifier folder read one event source, its NodeId identifier (a RawPath or an EquipmentId). AVEVA stores a
Galaxy object's A&E history under the object's tag_name (Source_Object), so AVEVA's own alarm and event
history reached no production folder. Its lower-case property keys were also missed, so every row read
Severity 1 with no ConditionName.

Sources. A folder's event history now reads its own source plus the Galaxy object of every native condition
that fires on it live: those parented under it and those WireAlarmNotifiers wired to it. Live and history use
the same membership. The object comes from the condition's Galaxy alarm reference, its raw tag's attributeRef,
up to the first '.':
- The composer reads it with the driver's own parser (GalaxyAttributeRef, shared with GalaxyDriver), only for
  a GalaxyMxGateway tag with an alarm (RawTagPlan.NativeAlarmReference, part of plan equality).
- The applier binds it after materialising the condition (IOpcUaAddressSpaceSink.BindNativeAlarmHistory,
  forwarded by the deferred sink). A rebuild or the condition's removal forgets it.
Each object is read with the #882 rule: its own events and <Object>.<Attribute> rows, never a prefix sibling.

Fan-out. historiangw's ReadEvents filter is a single predicate, so there is one request per source; the wire
contract is unchanged. EventSourceFanOut merges the reads as one source:
- order is (time, source position, arrival), reversed for a reverse read; ties are never re-sorted by EventId,
  so the boundary skip stays stable;
- duplicates by EventId are dropped and the merge is cut to the cap;
- a truncated source bounds the page at the time it stopped.
Paging, continuation points (bound to the whole source set), the walked-back reverse read and the tie-cluster
over-fetch are unchanged. A folder with one source is read exactly as before.

ACL. Every source is read inside the folder's own HistoryRead, after AuthorizeHistoryRead (#826). A denied
folder reads nothing and fails closed.

Mapping (EventMapper, property names now case-insensitive):
- Severity: an AVEVA row's priority goes through GalaxyAlarmPriority, the table live Galaxy alarms use
  (MxAccessSeverityMapper now delegates to it), so history matches live. Otherwise severity (OtOpcUa's own
  rows, which AVEVA stores lower-cased), clamped. AVEVA's own 1-4 severity is ignored. The table ranks a
  higher priority as more severe, which inverts AVEVA's convention: filed as #887.
- An AVEVA row whose source_conditionvariable names a native condition that fires on the folder takes that
  condition's SourceNode and live ConditionName. Otherwise ConditionName is alarmname or the condition
  variable's attribute.
- alarm_id/alarmid map to AlarmId.

Tests:
- GalaxyEquipmentEventHistoryWireTests (4, real SDK session, production Gateway backend, AVEVA-shaped fake
  gateway): equipment EqA backed by objects Oven_007 + OvenDoor_07. Covers object and attribute events from
  both objects, the prefix sibling excluded, cross-object and in-object ties, forward paging at 1-4 a page,
  reverse whole and paged, SourceNode/ConditionName/Severity, the raw device folder, ACL deny with no backend
  call plus a granted positive control, and a rebuild forgetting the bindings.
- EventSourceFanOutTests (8), EventMapper AVEVA cases (11), GalaxyAttributeRefTests (16), a composer test and
  an applier ordering test.
- Mutation-checked, 17 mutants, all killed. Covered: parent-only membership, no scope, a case-sensitive
  condition match, no condition rewrite, no truncation bound, EventId tie order, no sort, no cap, no stop at
  the bound, unreversed source order, no dedupe, no '.' boundary, priority ignored, case-sensitive keys, no
  applier bind, no deferred forward, and a composer that ignores the driver type. The forget-on-rematerialise
  guard survives as belt-and-braces: the rebuild clear covers it.
- Live, read-only, no WinRM, vd03 HistorianGateway ea9f21c: equipment folder EQ-886-ZipsenLeft (objects
  DevWinPlatform + Left_001). It read 0 events before and 92 now, equal to the unfiltered read's rows for both
  objects. 1/7-event pages and reverse whole/5-event pages agree, and 73 rows resolve to their live condition.
  diff_runs vs the #882 run: 221/226 identical. The 5 that differ are expected: DevWinPlatform Severity
  1 -> 750/1000, and #881 capabilities.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 11:45:48 -05:00
Joseph Doherty be28aa4432 fix(opcua-client): send UserName passwords in the legacy RSA format so 65-char API keys log in to SDK 1.5 servers (#880)
SDK 2.0.1's UserNameIdentityTokenHandler encrypts a password longer than 64 bytes
(a private RsaEncryptedSecretPasswordThreshold, no setting) as an RsaEncryptedSecret
with EncryptionAlgorithm=null. SDK 1.5.378 (.106 and .182) decrypts only the legacy
format under RSA policies; its SecurityPolicies.Decrypt answers a null algorithm with
BadIdentityTokenInvalid ("Unexpected encryption algorithm") before any key check.
Every production API key is 65 chars, so no SDK 2 OtOpcUa client could use one
against QA (b8b4c18f), a rollback target or an older third-party server.

LegacyRsaUserNameIdentityTokenHandler (Driver.OpcUaClient.Session, plugged in through
the public UserIdentity(IUserIdentityTokenHandler) ctor) always uses the legacy
format, RSA-OAEP(len|password|serverNonce) under the server certificate, under an RSA
policy. That is what SDK 2 sends for passwords of 64 bytes or less and what SDK 1.5
clients always send. The legacy format has no size limit (the plaintext is
length-framed over as many RSA blocks as it needs), and an SDK 2 server reads both
formats. Policy None, the ECC policies and a missing certificate go to the SDK
handler unchanged.

OpcUaUserIdentityFactory.UserName is now the one UserName identity the driver, the
AdminUI browse and Client.Shared (CLI/UI) present. Client.Shared references .Session
for it.

Tests: handler unit tests (all three RSA policies, the 1.5-required algorithm URI,
SDK 2 server-side decrypt, the session Copy() keeps the type, None unchanged, and an
SDK control pinning the 64/65-byte switch); a Client.Shared identity test; and an
SDK2<->SDK2 integration test where a 65-char key is admitted in both formats and a
wrong 65-char key reaches the key check (BadIdentityTokenRejected). The #800 Windows
gate's SDK 2 CLI 65-char case now gates against the 1.5 server too.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 11:12:27 -05:00
Joseph Doherty da948e5fb5 fix(history): a folder's event history includes its object's attribute alarms (#882)
v2-ci / unit-tests (push) Has been cancelled
HistorianGateway answers a per-source ReadEvents (Source_Object = DevWinPlatform) with the
object's own events (SourceName DevWinPlatform) AND its attribute alarms, which it reports as
<Object>.<Attribute> (Source_ProcessVariable, e.g. DevWinPlatform.CPULoadAvg). The Gateway
backend's defensive client-side source filter compared SourceName for equality, so it dropped
the attribute alarms: 70 of the 72 DevWinPlatform events on vd03.

The filter (and the past-the-cap "more events" signal) now uses EventSourceMatch: the source
itself, or <source>.<anything> on the '.' boundary, ordinal. DevWinPlatform never matches
DevWinPlatform2 or DevWinPlatform2.X. This needs no wire change and still makes one request.
Each event keeps its own SourceName. SourceNode stays the condition its stored alarm id names,
which is null for AVEVA rows. The attribute events ride the notifier's own read, so the
per-folder HistoryRead verdict (#826) governs them: a denied folder answers
BadUserAccessDenied before any backend call. The SqlServer and QuestDB backends still match
exactly; they hold only OtOpcUa's own events.

Tests:
- GatewayAttributeEventSourceTests (16): the boundary theory, attribute-in/sibling-out through
  the data source, the drain signal, and an exact attribute-named read.
- AttributeEventHistoryWireTests (2, over a real SDK session with the production Gateway
  backend on an AVEVA-shaped deterministic gateway): object + attribute events with their own
  SourceName, a prefix sibling excluded, paging, and the ACL-on deny with no backend call plus a
  granted positive control.
- Mutation-checked: exact match fails both wire tests; dropping the '.' boundary fails the
  wire sibling check and 6 unit tests.
- Live AVEVA harness re-run against vd03 HistorianGateway ea9f21c (read-only, no WinRM):
  folder DevWinPlatform 2 -> 72 events (whole, 72 1-event pages, reverse); the new #882 check
  passes; 221 of 225 shared cases are identical to the #801 SDK2 baseline, and the 4 that
  differ are exactly the DevWinPlatform event cases. Recorded in aveva-history-801.md.

Follow-up #886: production folders name RawPaths/EquipmentIds, not Galaxy objects, plus the AVEVA
severity/condition property mapping.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 10:34:29 -05:00
Joseph Doherty a28adba2fd test(galaxy): live gate for the stream-attached ack against a real gateway (#877)
v2-ci / unit-tests (push) Failing after 10m42s
Adds GatewayGalaxyStreamAttachedAckLiveTests (skip-gated on MXGW_ENDPOINT +
GALAXY_MXGW_API_KEY, like the other gateway live tests). Proves what the unit
suite cannot: the request flag reaches a real gateway and the sentinel comes back.

- live session advertises stream-events-attached-ack; GatewayGalaxySubscriber reports it
- raw stream: sentinel is the first message when requested (family Unspecified,
  worker_sequence 0, live_after 0) and never sent when not requested (control,
  with a data event as the positive control)
- real GalaxyDriver + EventPump + GatewayGalaxySubscriber behind a pass-through
  recorder: exactly one sentinel, ack observed before the first SubscribeBulk,
  pump StreamAttached completed, sentinel never dispatched nor counted as
  received (galaxy.events.stream_attached=1, received == non-sentinel events),
  no missing-ack warning within the production 1 s budget, and the static
  attribute TestMachine_002.Tagname delivers its initial value
- control: a subscriber that opts out gets no sentinel and the pump never
  reports one, while the #865 fallback still delivers

Run 2026-11-12 against windev 10.100.0.48:5120 (mxaccessgw 447070a5): 4/4.
Mutation checks, each caught: subscriber not requesting the ack (sentinel
count 0); driver skipping the ack wait (wire order became stream | subscribe |
ack, 2/2 runs, so the race is real on a live gateway); pump not intercepting
the sentinel (received=2 vs 1 non-sentinel).

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 08:08:55 -05:00
Joseph Doherty f264977fb0 feat(galaxy): wait on the gateway's stream-attached ack instead of the 1 s best-effort wait (#877)
v2-ci / unit-tests (push) Has been cancelled
mxaccessgw#227 (Contracts/Client 0.4.0) adds an opt-in attach acknowledgement on
StreamEvents: a StreamAttached sentinel MxEvent sent first, only after the
stream's subscriber is registered on the session. This closes the #865 attach
race that waiting for "stream requested" only narrowed.

- Bump vendored MxGateway.Client/Contracts 0.3.0 -> 0.4.0 (nuget-packages/,
  pulled from the Gitea feed; local-mxgw stays the mapped source).
- IGalaxySubscriber.SupportsStreamAttachedAck (default false, so fakes and older
  seams keep the fallback); GatewayGalaxySubscriber reads it from the live
  session's capabilities (stream-events-attached-ack) and passes
  emitStreamAttached accordingly; TracedGalaxySubscriber forwards it.
- EventPump: records AttachAckExpected before the stream opens (published
  before StreamRequested completes); intercepts ev.StreamAttached ahead of the
  ReplayGap check - completes StreamAttached and continues: never dispatched,
  never counted as received, never a resume cursor (worker_sequence 0). The
  signal also completes when the loop ends so no waiter hangs. One pump = one
  stream, so a reconnect's new pump has a fresh signal. New counter
  galaxy.events.stream_attached.
- AwaitCurrentEventStreamAsync: with the capability, waits on StreamAttached
  within the same StreamRequestWait budget (1 s); a missing ack logs a
  rate-limited Warning and proceeds instead of hanging. Without the capability
  the old StreamRequested wait is the fallback.

Tests (StreamAttachedAckTests, 6): ack-gated subscribe gets its initial value;
capability absent uses the fallback; missing ack times out with a warning; the
sentinel is never dispatched; a dead stream releases ack waiters; reconnect
gives each stream a fresh signal. With the ack wait skipped, 3 of them fail.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 07:38:09 -05:00
Joseph Doherty 633f43170a test(galaxy): #872 pin the detected_by rule per source
v2-ci / unit-tests (push) Has been cancelled
The rule fix itself landed in 1a08b4f7 (detail-0 failure rows route on
category); its test only exercised raw_detected_by = 3. Add per-source
coverage for every MxStatusSource value mxaccessgw's MapSource emits
(-1, 0..5, and an out-of-range value) across the failure categories, plus
the live-captured shape the old rule got wrong (OperationalError detected
by the responding automation object, mxaccessgw docs/AlarmProbeFindings.md)
and a RequestingLmx CommunicationError positive control. All 10 new cases
fail against the old 'detail 0 && raw_detected_by != 0 => comms' rule.

Comment in FromMxStatus now cites MapSource and says why NMX (2/3) is not
a transport signal. Item 1 (Warning/Pending) stays open pending a capture.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-12 07:17:53 -05:00
Joseph Doherty 95bde828eb fix(galaxy): #875 Galaxy initialize gets a 20 s budget via per-driver-type capability defaults
Gate F (#761) against the VD03 gateway: Galaxy initialize (session open +
platform seed) took 4–5 s against Tier A's 5 s Initialize timeout, so 10 of 40
restarts failed, paid a 10 s retry and left a killed gateway worker.

DriverFactoryRegistry.Register takes optional capabilityDefaults per driver
type (GetCapabilityDefaults); DriverResilienceOptionsParser layers them over the
tier table and under the per-instance ResilienceConfig; the Host wires the
registry into DriverCapabilityInvokerFactory. Galaxy registers Initialize =
20 s (no retry, breaker 2), under the actor's 30 s pessimistic bound.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 18:44:44 -05:00
Joseph Doherty 1a08b4f746 fix(galaxy): #872 detail-0 failure rows route on category, not detected_by
Every raw detected_by value names an MXAccess layer (requesting/responding
LMX, NMX, automation object); the gateway's own failures arrive as
ProtocolStatus. The 'non-zero source = transport failure' rule made a failure
the responding object detected a comms error. Item 1 of #872 (Warning/Pending
categories) stays open pending a captured row.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 15:15:05 -05:00
Joseph Doherty 875b82175d fix(galaxy): #866 grade OK-status data changes by the item's OPC DA quality
The gateway fills Statuses on every data change (live OnDataChange, value-cache
replay and session snapshot all carry Quality too), and EventPump used the status
row whenever one was present, so Quality was never read: an off-scan, Bad or
Uncertain attribute with an OK status row was published Good. A failure status
row still wins; otherwise the quality byte grades the value (legacy parity:
quality >= 192 Good).

Live, read-only on VD03 (DEV): healthy ScanState and _TestArea attributes stay
Good (0x00000000) via quality 192; the stopped JoePlatform/JoeAppEngine ScanState
now publish BadCommunicationError (0x80050000) instead of generic Bad, with no
"unrecognised quality byte" warning.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 15:14:11 -05:00
Joseph Doherty 71b3b7f35f fix(galaxy): #866 map MxStatus failure details by the MxStatusDetail vocabulary, not as OPC DA quality
The gateway copies MXSTATUS_PROXY.detail verbatim, so a failure row's detail is
always an MxStatusDetail code. FromMxStatus routed any detail <= 0xFF through the
OPC DA quality table: MX_E_PlatformCommunicationError (2) on a stopped platform's
ScanState logged "Unrecognised MXAccess quality byte 0x02" and published generic
Bad (VD03 live capture), MX_E_InvalidObjectId (8) became BadNotConnected,
MX_E_Overflow (20) BadCommunicationError, and a detail in the Uncertain band
published Uncertain for a failure.

Every non-zero detail now maps per the gateway.md vocabulary (timeout,
communication, configuration, type, range, access, initialising), then by
category; the category-fallback warning logs once per (detail, category).

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 15:06:26 -05:00
Joseph Doherty 1059eb7885 test(heidenhain): live gate asserts integer tool leaves keep their runtime type (#867)
Verified read-only on Hermle59 and Hermle70 (iTNC 530): Tool/Number Int64, Tool/Index and Tool/AxisCode Int32.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:36:13 -05:00
Joseph Doherty eba016096c Merge branch 'master' into worktree-agent-a720278a57ae41792
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:26:05 -05:00
Joseph Doherty b5d4e25905 Merge #864/#865 Galaxy host status: per-template platform seed, event stream started before probe registration; registry and reconnect races fixed
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:22:33 -05:00
Joseph Doherty 59d33ab938 test(galaxy): #865 review round 7 - pin the superseded-pump fault filter
Adds an escalated-stream-fault test seam and a test: the live pump's fault
escalates (positive controls before and after), a superseded pump's late
fault after a reopen does not; the absence is asserted only once both
pumps' callbacks are known to have run (HasEnded). Verified to fail with
the filter disabled. The filter's comment now says the identity check does
the work and warns about a future session bump while Healthy.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:20:49 -05:00
Joseph Doherty d80bdf6369 fix(galaxy): #865 review round 5 - a failure during the replay fails the attempt
ReconnectSupervisor swallowed a transport failure reported while Replaying,
so a restarted event stream refused/ended a round trip after the replay
began still let the replay go Healthy over a dead stream (and the next
advise's re-report restarted at attempt 1 with no backoff). A failure
reported during Replaying is now remembered and turns that attempt's
success into a retry with backoff. ReplayAsync's synchronous dead-pump
throw yields to a teardown's cancellation.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:12:33 -05:00
Joseph Doherty 77872b1d1c fix(runtime,galaxy): #869 review round 1 — confirmation off the actor scheduler, stop/async-fault tests
- Start the ack confirmation with Task.Run + ConfigureAwait(false) so it never
  depends on the ReceiveAsync handler's ActorTaskScheduler (the PostStop
  abandon-and-revert report must run after the actor stopped).
- DriverInstanceActorAckConfirmationTests: a confirmation that faults after the
  handler returned is reported; one pending at actor stop is reported; a
  confirmed ack reports nothing (positive control).
- Deterministic tracker matching test (Raise/Clear/Retrigger/other alarm do not
  confirm; case-insensitive match does).
- AdminUI min/max on the confirm-timeout field; Galaxy.md documents the
  reconnect, UNACK_RTN-as-Clear and RPC-budget limits.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:11:49 -05:00
Joseph Doherty 8653960ea3 fix(galaxy): #865 review round 4 - replay fails on a dead stream, backoff-aware wait
- ReplayAsync fails when its restarted pump's stream has already ended, so
  the supervisor backs off and stays Degraded instead of going Healthy over
  a dead stream (whose next advise re-report restarted at attempt 1 with no
  backoff - a tight reconnect loop).
- ReconnectSupervisor.AttemptInProgress (reopen through replay); advises
  wait for a stale pump's replacement only while an attempt is running, not
  for a whole backoff (the state still reads Reopening/Replaying there).
- ReconnectSupervisor: _disposed is set under the state lock and re-checked
  inside ReportTransportFailure, so a late report cannot start a recovery
  loop that StopAsync already missed.
- EventPump.HasEnded is a flag set before StreamRequested is released, and
  a stream that ends/faults on its first MoveNext is marked ended first.
- SubscriptionRegistry fast path: a miss now falls through to the locked
  path (lookup + session + window check atomically) instead of a lock-free
  re-lookup.
- Tests: replay over an immediately-ending stream fails; AttemptInProgress
  across attempt/backoff; ReportTransportFailure after stop; the
  reopen-window test uses a generous wait bound.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:07:26 -05:00
Joseph Doherty 4e492e6550 fix(galaxy): confirm unconfirmed native alarm acks against Galaxy's acknowledged transition (#869)
mxaccessgw answers AcknowledgeAlarm OK both when the AckMsg write was
applied and when it was only sent (no OnWriteComplete; the reply then
carries a diagnostic). EvaluateReply treated both as success, so the H6
live gate saw a condition go Acknowledged while Galaxy never applied it.

- EvaluateReply returns Applied (no diagnostic) or Unconfirmed (diagnostic)
  and logs the gateway diagnostic on every branch that carries one.
- GalaxyAckConfirmationTracker watches the alarm feed from before the RPC
  and, for an unconfirmed reply, waits up to AlarmAck.ConfirmTimeoutMs
  (default 10 s, 0 = old behaviour) for an Acknowledge/ActiveAcknowledged
  transition; on timeout it refuses (BadTimeout + diagnostic).
- New optional IAlarmAcknowledgeConfirmation: DriverInstanceActor awaits it
  after AcknowledgeAsync, off its mailbox and outside the resilience
  pipeline; a failure is the existing #760 AlarmAckFailed revert.
- AdminUI Galaxy form carries the new knob; docs updated.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:03:04 -05:00
Joseph Doherty 1c1705fa77 fix(galaxy): #865 review round 3 - budget-safe stream wait, dead pump, fast-path drop
- StreamRequestWait 1 s (under Tier-A Read 2 s / Subscribe 5 s budgets) and
  a stale pump is only waited on while the supervisor is reopening or
  replaying (or a restart is underway); in a reconnect backoff the advise
  proceeds at once. The no-stream log is a rate-limited Warning while
  Healthy, Debug otherwise.
- A current-session pump whose loop has ended is not a stream: if the
  supervisor already reports Healthy (its fault was swallowed while it was
  recovering) the failure is re-reported to start a new cycle.
- Subscribe/Read no longer start the pump directly (only
  AwaitCurrentEventStreamAsync does, honouring the replay restart flag);
  the teardown refusal is an explicit ThrowIfRuntimeStopping.
- SubscriptionRegistry.ResolveOrPark fast path: the handle lookup and the
  open-window check were not atomic, so a subscribe that registered the
  handle and closed its window in between got its initial value dropped as
  stale (found by a stress run: the first event of a bulk advise lost). It
  re-looks-up before declaring stale. Deterministic test via a seam.
- Tests: first Read on a fresh driver, replay of a data subscription.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:00:19 -05:00
Joseph Doherty 9060ff5561 fix(galaxy): #865 review round 2 - wait for the replay's pump, never race a second stream
Round 1 replaced a stale-session pump from any advise path and disposed the
old one in the background; that could start a second stream while the old
one still held the gateway's single-subscriber slot, and a kept
current-session pump could be dead with no fault re-reported.

- EnsureEventPumpStarted reuses again; only ReplayAsync restarts the pump.
  RestartEventPumpForReplay keeps a current-session pump only while its loop
  is running (EventPump.HasEnded), and flags the restart so no advise path
  starts a competing pump in the dispose gap.
- AwaitCurrentEventStreamAsync (bounded by StreamRequestWait) starts a pump
  only when none exists, WAITS for the replay while the pump is stale or
  restarting, waits for StreamRequested, and returns the session it verified.
  SubscribeAsync (per #861 attempt), read-via-subscribe and the probe hook
  issue and fence the advise under that session, so a reconnect after the
  check is caught as a straddle and retried.
- Read: a deadline (not caller) cancel during the wait falls through to
  per-ref results.
- Tests: reopen-window probe test now asserts the advise waits for the
  replay's single new stream; first Subscribe/Read wait for the stream; the
  fencing tests that advise in the window use a short wait bound; the
  production-path seed test uses an empty galaxy (a real probe advise now
  starts a stream against the fake endpoint).

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:46:15 -05:00
Joseph Doherty cc1a7ec687 fix(galaxy): #865 review round 1 - stale-session pump, data-path stream wait
- EnsureEventPumpStarted replaces a pump created for an older registry
  session (the reopen -> replay window still held the dead session's pump,
  so an advise there lost its initial value and the replay skipped it);
  RestartEventPumpForReplay keeps a pump already on the current session.
- SubscribeAsync, read-via-subscribe and ReplayAsync also await the
  bounded StreamRequested wait before advising, not just the probes.
- TryEnsureEventPumpStarted tolerates a teardown flipping _runtimeStopping
  mid-check; the probe hook honours an already-cancelled seed token.
- StreamRequested is documented as narrowing, not closing, the attach race.
- Tests: reopen-window probe test, EventPump.StreamRequested tests, and the
  FixRound1 fresh-pump assertion now waits on a positive control.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:31:51 -05:00
Joseph Doherty fb44ec2a9c Merge #821 live TimeAverage/Total late-start with Uncertain first sample matches HistoryRead (Part 13 partial interval)
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:30:27 -05:00
Joseph Doherty 8a0032bfdf fix(aggregates): live TimeAverage/Total leave the span before an Uncertain first value out (#821)
When the data starts inside an interval (no start bound) and its first value is
Uncertain, the frozen 1.5.378.182 AverageAggregateCalculator integrated the
uncovered span as a ramp from 0 and divided TimeAverage by the whole interval:
GetRegionsInValueSet (ignoreBadData, sloped) overwrites the BadNoData start
region's status with UncertainDataSubNormal in its one branch without a not-Bad
guard. Live AggregateFilter therefore disagreed with HistoryRead and Part 13
§5.4.3.6/§5.4.3.8 ("No Start Bound: Calculate Partial Interval"); QP: Total
188.96 vs 163.96, TimeAverage 9.448 vs 10.931. SDK 2.0.1 stock does the same.
Owner-approved 2026-11-11.

- New app-owned CoveredTimeAverageAggregateCalculator subclasses the baseline
  and recomputes only TimeAverage/Total, restoring the Bad status of a region
  that starts at a Bad point (only the start bound can). Only such an interval's
  value changes; status, timestamps, type and bits are the baseline's. Stepped
  and TreatUncertainAsBad=true never reached the defect. CompatibilityAggregate-
  Factory routes exactly those two IDs; copied SDK files untouched.
- Golden test: 0 fixture vectors change (every data set has a usable value at
  -10 s), asserted. New sweep over late-start variants of every data set (cut
  +5 s / +25 s, first non-Bad sample Uncertain, all directions/stepping/configs/
  incremental): 150 / 142 of 384 change, each pinned to a baseline-derived
  oracle (untouched baseline's value with that sample Good, its status/bits on
  the variant itself) - never our calculator. Plus a hand-computed case.
- Live-vs-HistoryRead parity (TimeWeightedAggregator) forward + reverse, no
  masks, with discrimination against the pre-#821 baseline; QualityMix now
  compares QP's TimeAverage/Total with the live oracle on both SQL backends;
  booted-runtime facts test expects 163.96 / 10.93.
- README recorded decision, ServerAggregateConfiguration remarks, oracle
  remarks, docs/Historian.md and .claude/rules/historian.md updated.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:30:01 -05:00
Joseph Doherty cbd24d7a01 Merge #867 Heidenhain integer leaves keep their declared type; #868 ConfigPublishCoordinator survives ConfigDb outage at start
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:29:00 -05:00
Joseph Doherty e35f2a4e93 fix(galaxy): probe advises start the event stream first so initial ScanState arrives (#865)
The gateway's StreamEvents with after_worker_sequence = 0 is a fresh stream
and never replays, and the event pump started only on the first
Read/Subscribe, so platform ScanState probes advised before then lost their
initial value (and a driver with no authored tags never got any host status).

PerPlatformProbeWatcher now awaits an ensure-event-stream hook before every
probe SubscribeBulk (seed, discover sync, reconnect ResubscribeStaleAsync).
The driver's hook starts the pump (refused while a teardown is in progress)
and waits, bounded, on the new EventPump.StreamRequested signal - completed
once the stream loop's first MoveNext has issued the StreamEvents RPC, or
when the loop ends - so the advise never precedes the stream request.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:24:08 -05:00
Joseph Doherty c4bc1ef309 fix(galaxy): seed platforms with one template filter per request (#864)
The gateway's shared GalaxyHierarchyProjector ANDs template_chain_contains
entries, so the single request carrying both $WinPlatform and $AppEngine
matched no object and the probe watcher always seeded 0 platforms.
GatewayGalaxyPlatformSource now runs one paged sweep per template and
unions/dedupes by tag name. The tests use a fake gateway that applies the
projector's AND semantics (they fail on the old single request).

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:24:03 -05:00
Joseph Doherty 8e97fc94fd fix(heidenhain): Tool/Number, Tool/Index, Tool/AxisCode publish their declared integer types (#867)
HeidenhainSession.Leaf's `case Lsv2Tool t` switch expression mixed long, int and
double? arms, so C# unified the natural type to double? and Tool/Number (Int64),
Tool/Index and Tool/AxisCode (Int32) were published as Double. Every arm in that
switch (and in the alarm-list, axes and feature-acquire switches in the same file)
is now boxed to object explicitly so no numeric arm can widen another.

New HeidenhainLeafRuntimeTypeTests asserts value.GetType() for every catalog value
leaf against its declared DataType (scalar + array), plus PLC/DATA binding coercion
for Int16/Int32/Int64/Boolean. Verified red on the old code (exactly the three Tool
leaves) and green with the fix.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:20:33 -05:00
Joseph Doherty 233e4bab25 Merge #863 TwinCAT Test Connect: drop the AmsNetId-as-IP TCP pre-check; probe via the local AMS router ADS handshake
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:14:58 -05:00
Joseph Doherty fdc9473819 fix(twincat): Test Connect probe no longer dials the AmsNetId as an IP (#863)
TwinCATDriverProbe ran a first-phase TCP check against the AmsNetId's
first four octets on the AMS port (801/851). An AmsNetId is not an IP and
the AMS port is not a TCP port, so Test Connect timed out against every
real runtime (Z35474: dialled 5.118.56.93, an unrelated public host).

The probe now opens no socket of its own: it takes the driver's path,
AdsClient.Connect + ReadStateAsync through the local AMS router, bounded
by the probe budget (Task.Run + WaitAsync), and classifies failures as
no local AMS router / no route or target unreachable / timeout / other.
The degrade-to-green 'TCP reachability only' outcome is gone - with no
TCP check nothing was contacted, so a missing router is reported red.

Tests: the regression test listens on the AmsNetId-derived endpoint and
asserts no connection arrives (fails on the old probe); classification
and budget tests drive the handshake through an internal seam.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:13:55 -05:00
Joseph Doherty 70bc960111 Merge #809 live PercentGood/DurationGood count Good only (Part 13), matching HistoryRead
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:05:51 -05:00
Joseph Doherty ed05d50795 fix(aggregates): live PercentGood/DurationGood count Good time only (#809)
The frozen 1.5.378.182 StatusAggregateCalculator counts Uncertain regions as
Good under TreatUncertainAsBad=false, so live AggregateFilter PercentGood /
DurationGood disagreed with HistoryRead (Part 13 §5.4.3.31/§5.4.3.33, Mantis
0006166) whenever Uncertain data was present. Owner-approved 2026-11-11.

- New app-owned GoodOnlyStatusAggregateCalculator subclasses the baseline and
  replaces only the PercentGood/DurationGood computation (Good regions only);
  CompatibilityAggregateFactory routes exactly those two IDs to it. The copied
  SDK files are untouched; the other 35 aggregates are byte-identical.
- Golden test: the 64 changed vectors (38 PercentGood + 26 DurationGood) are
  exempted exactly — those whose baseline output differs from the same case's
  baseline TreatUncertainAsBad=true twin — and pinned to that twin (baseline
  evidence, not regenerated from the implementation); plus a full Good-only
  pin over all 352 PercentGood/DurationGood vectors and a hand-computed case.
- Live-vs-history parity over Good/Uncertain/Bad mixes (no masks), with a
  discrimination check against the pre-#809 baseline; the booted-runtime facts
  test now expects 8000 ms / 40 %.
- SdkAggregateOracle: Run/RunLive = the registered factory; RunSdkAsIs = the
  untouched baseline; its Part13GoodCalculator duplicate is removed.
- README (recorded decision), ServerAggregateConfiguration remarks,
  docs/Historian.md and .claude/rules/historian.md updated.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:05:34 -05:00
Joseph Doherty bdbc47f5fc fix(mtconnect): serve an empty string-typed value as Good "" (#824)
Owner decision 2026-11-11 (Java parity): a genuinely empty observation on a
String/Reference or unauthored item is Good "" instead of BadNoCommunication.
Non-string items (numeric, bool, DateTime, arrays) keep BadNoCommunication -
the Agent supplied no value; BadTypeMismatch would wrongly blame the authored
type. The UNAVAILABLE sentinel is unchanged, and a structured observation with
no text stays BadNotSupported (structured check now precedes the empty check).
Parity matrix, quality-mapping section and drivers rule updated.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 11:56:30 -05:00
Joseph Doherty 33e10c2266 fix(galaxy): stranded subscriptions after a no-replay reconnect go Bad + Degraded (#862)
With Reconnect.ReplayOnSessionLost=false a recreated gateway session carries none of
the old advises, so every pre-reconnect subscription went silently deaf: nodes kept
their last value as Good and health returned to Healthy (only a Warning log).

ReplayAsync now records the stranded subscriptions and publishes BadNoCommunication
(StatusCodeMap.BadNoCommunication, 0x80310000; value null — no driver value cache)
for every tag of each, straight to OnDataChange so LastSuccessfulRead is not stamped.
GetHealth reports Degraded naming the setting while any stranded subscription is
still registered; it clears on UnsubscribeAsync (the host's re-subscribe path) or
runtime teardown, and pruned ids never pin it. A subscribe after the reconnect is an
ordinary new-session subscription. With nothing else flowing, the comms-loss
watchdog re-initializes the driver after 90 s (a delayed replay).

Tests: Runtime/StrandedSubscriptionTests (6). Docs: drivers/Galaxy.md, which also
no longer claims the replay uses the gateway's ReplaySubscriptions RPC.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 11:13:19 -05:00
Joseph Doherty a8d3790b09 fix(galaxy): #859 final review — probe replay per platform session, teardown session bump
BLOCKER: the reconnect replay chose work from the registry entries' sessions,
but ONE registry entry holds every probe handle and any probe publish on the
new session (a seed in the reopen→replay gap, or the straddle retry) re-stamps
it as current — the replay then skipped the probe watcher and the old
platforms stayed bound to handle 0 (host status frozen); a publish after the
snapshot made it re-subscribe everything (double advise, leaked handle).
ReplayAsync now excludes the probe entry from its snapshot and calls the new
PerPlatformProbeWatcher.ResubscribeStaleAsync on every reconnect, which
re-subscribes exactly the platforms whose ProbeHandle.Session predates the
current session. The replay-gap path keeps ResubscribeAsync (all platforms).

Also: ClearRegistry (teardown) starts a new registry session, so a late
PurgeParked from a teardown-orphaned call cannot reach the next runtime's
parked values; the read path's unused pump local is gone (call kept).

Tests go through the driver's reopen/replay: the probe old-session-number test
is rewritten that way, plus the seed-in-the-gap scenario (only P and Q are
re-subscribed, R keeps its new-session handle, all three resolve). Both checked
red with the probe replay removed / re-subscribing all platforms.

Refs #859, #861.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 11:07:59 -05:00
Joseph Doherty 133bc07da5 fix(galaxy): #859/#861 third review — read rebind release, probe session fence, reopen fence, pump restart
B-1: the subscribe-once read's cleanup unsubscribes what the registry holds at
removal (the replay may have rebound it to new-session handles), not its
original bindings; nothing when the entry still holds dead old-session
handles. Found while testing it: reads were attached to the pump INSTANCE, so
a read across a reconnect's pump restart never saw the replayed value — reads
now listen on a driver-level ReadDataChange raised for whichever pump is live.

I-A: PerPlatformProbeWatcher records the registry session with each probe
handle. Publishing goes through the new fenced TryUpsert and binds only
current-session handles; removal/stop unsubscribe and purge only those; a
seed reply that straddled a reconnect records nothing and is re-issued (max 3).

I-B: PurgeParked takes the session captured BEFORE the unsubscribe RPC and is
a no-op under the park gate if the session changed.

I-C: ReopenAsync starts a new registry session before AND after the gateway
session swap; ReplayAsync does so only when no reopen ran (injected subscriber
/ tests) and replays only subscriptions still holding older-session handles,
so a subscribe issued after the swap is live and not double-advised.

P-1: ReplayAsync restarts the pump before any early return (no entries used to
leave the dead pump in place). P-2: ReplayOnSessionLost=false logs a Warning;
misleading comment about gateway-side replay fixed.

M-a: TryRebind returns BindOutcome; UnknownId releases the handles just
advised for a subscription removed mid-replay. M-b: the replay/re-snapshot
pass reads the session once and stops if it changes. M-c: parked entries drop
their MxEvent when they leave the parked set. M-d (retry a straddled read)
not done.

Tests (TestWait) for each, each checked red against a mutation of its fix.

Refs #859, #861.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 11:02:55 -05:00
Joseph Doherty d08e900203 fix(galaxy): #859 second review + #861 — window-sequence trim, per-session handles, straddling subscribes
I-2: unclaimed parked events are trimmed by window sequence. Each window gets a
monotonically increasing sequence at Begin, each parked event is stamped with
the newest one, and on every End events stamped before the oldest open
window's sequence are dropped (amortised O(1) via a park-order queue). Under
concurrent reads the open count rarely hit 0, so trailing events used to pile
up to the bound and push genuine early values into overflow.

I-1 / #861: SubscriptionRegistry.StartNewSession() — called FIRST in
ReplayAsync on every reconnect — clears held events, invalidates the reverse
map (subscriptions kept) and bumps a session number. New-session events whose
number an old binding still held now park until the replay's per-subscription
Rebind promotes them, instead of resolving to the wrong tag. TryRegister /
TryRebind refuse a stale session atomically under the park gate: a
SubscribeAsync whose reply straddled a reconnect is re-issued on the new
session (max 3 attempts), a subscribe-once read returns BadCommunicationError,
ResubscribeAllAsync stops if the session changes again. Old-session handle
numbers are never unsubscribed (Remove reports whether the entry belongs to
the current session; the read's late-reply path checks too), and an old
session's pump can no longer resolve through the map (fast path checks the
session).

I-3 (interim): a completed UnsubscribeBulk purges events parked for its
handles (driver + probe watcher). Full fix needs a worker sequence on the
SubscribeBulk reply (mxaccessgw, optional) — documented.

M-1: a resolved handle with a stranded parked list promotes it and resolves
the live event (never parks it). M-2: ResolveOrPark returns the ready events
to replay first, one gate acquisition per event while anything is held.
M-3: SubscribeAsync/read dispose the window before the orphan-release RPC.
M-5: ResolveOrPark remarks document the single-dispatching-pump invariant.

Tests (TestWait): overlapping windows don't accumulate, purge, stranded list,
session fence + map invalidation, stale-session registration refused,
end-to-end replay with swapped handle numbers and event-before-reply,
subscribe and read straddling a reconnect, read deadline closes its window.
Each was checked red against a mutation of the fix it pins.

Refs #859, #861.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 10:41:16 -05:00
Joseph Doherty f89e0cc04c fix(galaxy): #859 review — read path window, atomic promote, session epoch, per-call windows
1. ReadAsync (subscribe-once) now opens the early-event window around its
   SubscribeBulk + registration; an initial value that overtook the reply was
   dropped and the read returned BadTimeout.
2. Register/Rebind write the reverse map and promote parked events in ONE
   critical section under the park gate, and ResolveOrPark resolves under that
   gate whenever anything is held (lock-free fast path only when nothing is),
   so a later live event can no longer overtake the parked initial value. A
   resolved handle that still holds a parked list queues behind it.
3. Held events are fenced by a session epoch: each new EventPump calls
   StartNewEpoch(), discarding parked AND promoted-but-unreplayed events; an
   older pump can neither park nor replay. Item-handle reuse across a reconnect
   can no longer put an old-session value on a new tag.
4. ResubscribeAllAsync opens one window per subscription, not across the whole
   replay.
5. Promoted-but-unreplayed (ready) events count toward MaxParkedEvents.
6. Begin/End of a window run under the park gate, so N->0 discard and 0->1
   open are atomic (no inherited leftovers).
Documented that ParkedEventsReady may run under _registryGate/_syncLock and
relies on the channel's AllowSynchronousContinuations=false.

Tests use TestWait (#858); each new test was checked red against a mutation
of the fix it pins.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 10:26:28 -05:00
Joseph Doherty 1a31e3f584 fix(galaxy): park data-change events that overtake their SubscribeBulk reply (#859)
mxaccessgw gives no reply-before-event ordering: the worker completes the STA
command via a RunContinuationsAsynchronously TCS, so the reply is framed on a
thread-pool continuation while the STA already pumps the OnDataChange Advise
queued (drained/framed by a separate loop), and the reply (unary Invoke) and
the event (StreamEvents) are separate HTTP/2 streams. The driver then also
registers handles concurrently with the pump's dispatch loop. MXAccess's
initial value could therefore be dropped as "stale after unsubscribe",
leaving the node at its pre-subscribe value until the next change.

SubscriptionRegistry gains an early-event window: every SubscribeBulk call
site (SubscribeAsync, ResubscribeAllAsync, PerPlatformProbeWatcher) opens
BeginPendingSubscribe() before the RPC and closes it after registering.
Inside the window unknown-handle events are parked (bounded, 100k); the
Register/Rebind that binds a handle promotes them, and the EventPump replays
them on its single dispatch thread before any later event for that handle
(woken by a sentinel when no further event arrives). Closing the last window
discards unclaimed events, so the stale drop is unchanged outside it.
New counters galaxy.events.early_{parked,replayed,overflow}.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 10:19:24 -05:00
Joseph Doherty 4fdc3708f4 test(galaxy): presence waits survive a starved/loaded runner; absence asserted behind positive controls (#858)
Under heavy parallel load (14 concurrent test processes + 14 CPU burners, with
DOTNET_ThreadPool_ForceMinWorkerThreads=1 as the reproducer) the Galaxy tests
with 1-3 s fixed waits failed 16/70 runs. Every failure was a background loop
started with Task.Run (alarm feed, DeployWatcher, EventPump, ReconnectSupervisor)
not being SCHEDULED within the ceiling - e.g. DeployWatcher's stream had not even
been opened after 2 s. No production race: the code under test was checked for
each (alarm feed dispatch is synchronous in-order, DeployWatcher/ReconnectSupervisor
state is lock- or loop-local, subscribe registers before returning).

- New shared TestWait (UntilAsync / ForAsync / HealthyAsync): 30 s presence
  ceiling, failure reports elapsed ms + an observed-state snapshot (#856 shape;
  GalaxyDriverFixRound1Tests.WaitUntilAsync now delegates to it).
- Converted the short waits in GatewayGalaxyAlarmFeedTests, DeployWatcherTests,
  GalaxyDriverSubscribeTests, GalaxyDriverRawPathKeyingTests (its WaitForAsync
  never threw on timeout), EventPumpStreamFaultTests, EventPumpReplayGapTests,
  ReconnectSupervisorTests; FixRound2 InitBudget 2 s -> 10 s (the parked fetch
  never completes, so any finite bound keeps the verdict).
- Absence by short sleep replaced with positive controls: DeployWatcher bootstrap
  and same-deploy-time (follow with a real change, assert exactly that one raise -
  verified red when suppression is removed), subscribe failed-tag / unsubscribed
  handle (follow with a live handle's event), faulted pump (wait for its
  onStreamFault instead of 100 ms).
- Fixed sleeps that raced scheduling: subscribe-once read tests wait for the
  read's registration (TrackedSubscriptionCountForTests) instead of 50 ms;
  supervisor Dispose test waits for reopen entry instead of 50 ms; the
  already-Healthy WaitForHealthy test asserts synchronous completion instead of a
  wall-clock bound. FakeSubscriber.Map is now a ConcurrentDictionary.

Loaded run (14 parallel x 5, 14 burners, ForceMinWorkerThreads=1):
before 16/70 failed runs; after 0/350 (70 + 210 + 70). Without the thread-pool
override: 0/70 before and after on this host. Serial: 472 passed, 8 skipped.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 10:18:22 -05:00
Joseph Doherty 90de38910d test(galaxy): shutdown stream-cancel signal from the iterator's exit, not a token registration (#856)
Found while load-verifying #856: ShutdownAsync_cancels_the_event_stream_and_releases_the_pump
timed out 7/700 under parallel load. The fake's StreamCancelled was set by a
CancellationToken registration scoped by `using` inside the async iterator.
On Cancel() the newer ReadAllAsync registration runs first and resumes the
iterator on a pool thread, whose `using` can dispose our registration while
Cancel() is still walking the callback list - so it never runs. Signal from
the iterator's finally (when the token is cancelled) instead, which also
proves the stream actually ended. 0/700 after.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 10:00:44 -05:00
Joseph Doherty 988046cbe4 fix(galaxy): WatchedPlatforms cannot show a platform before its probe binding is routable (#856)
PerPlatformProbeWatcher records a platform's handle and publishes the probe
binding into the driver registry as one step under _syncLock, but
WatchedPlatforms read the handle map lock-free - so it could report a
platform one step ahead of the registry Upsert. A ScanState event raised in
that window was dispatched by the EventPump, missed ResolveSubscribers and
was dropped as "stale after unsubscribe", leaving the host status empty.

That is the #856 flake: A_fetch_completing_after_init_populates_host_statuses
waited for WatchedPlatforms, emitted ScanState, and timed out at 10 s on the
final Running wait. Reproduced 11/700 (28 parallel runners x 25 + 14 CPU
burners), every failure at that step with the binding present by the time of
diagnosis and hostStatuses=[] - i.e. the event was dropped. 0/1400 after.

WatchedPlatforms now reads under _syncLock (test seam only; the dispatch path
already checks the map after the registry, the safe order). New
PerPlatformProbeWatcherVisibilityTests spins a reader on another thread and
asserts the registry entry exists the moment the platform is visible
(old code: torn in 1691/2000 iterations). WaitUntilAsync gains a describe
overload; the #856 test's waits now report elapsed time and the observed
seed/binding/pump/host-status chain on timeout.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 10:00:44 -05:00
Joseph Doherty 340d9d26b7 feat(mtconnect): CommsLossConfirmMs AdminUI field, refused-connection fast path reaches native conditions, dedicated live-gate Agent (#852)
- AdminUI MTConnect driver form: "Comms-loss confirm (ms)" in the Reconnect section, always written
  (0 = first refusal, negative = off is never dropped), case-insensitive load, unknown keys preserved,
  advisory at >= 90000 ms (the watchdog fires first). Form output byte-pinned and parsed through
  MTConnectDriver.ParseOptions on the driver side.
- Driver: a confirmed refused-connection outage now annotates every told CONDITION quality-only
  BadNoCommunication (state/ack kept, once per outage, UNAVAILABLE-Bad skipped); the first delivery
  afterwards re-diffs all subscribed conditions so unchanged ones get a quality-only Good.
- docker-dev/docker-compose.mtconnect-gate.yml: throwaway Agent in its adapter's network namespace so a
  stop is a genuine ConnectionRefused (a plain stopped container is a DNS failure = ambiguous).
- Docs: MTConnect.md comms-loss section + live-gate recipe, docker-dev README, drivers.md rule.

Refs #852
2026-11-11 06:23:18 -05:00
Joseph Doherty d1b6cad299 Merge #843 #840 #841 load flakes, ScriptRunGuard slot-release race, DurationGood conformance, CI QuestDB pin
v2-ci / unit-tests (push) Failing after 1h12m19s
2026-11-11 05:10:00 -05:00
Joseph Doherty 271b96a644 fix(security): #835/#836 review — unsaved endpoints never borrow the owner's secrets; audit credential use; path rule follows AuthType
I-1: Test Connect / browse configs may carry unsaved edits (the device modal merges
the stored DriverConfig with the typed device config). The server
(DriverInstanceIdentities.EffectiveOwnerAsync, used by AdminOperationsActor and
BrowserSessionService) honours the named owner only when every endpoint value
(new DriverCredentialRules.EndpointFields: Galaxy Endpoint, OpcUaClient
EndpointUrl/EndpointUrls/DiscoveryUrl, Heidenhain Host/Port) is one that driver
stores; otherwise the owner is null and its instance-named secrets are refused.
Every Test Connect / browse whose config carries an in-scope secret:/env: reference
is recorded (CredentialUseAudit: IAuditWriter event category CredentialUse + a
structured Information line: user, driver, endpoint hosts, outcome).

L-1: the UserCertificatePath rule is skipped only in an object that explicitly sets
AuthType to non-Certificate (string or ordinal); the deploy gate also checks the
runtime-merged driver+device config so the switch cannot be split from the path; the
form no longer persists the hidden field.
L-2: a ConfigDb fault in the browse check is a refusal; RawBrowseModal catches open
failures into its error message.
L-4: a relative UserCertificateDirectory resolves against AppContext.BaseDirectory.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-11 04:12:40 -05:00
Joseph Doherty 87416a230f Merge #835 #836 per-instance credential references, confined user-certificate path 2026-11-11 03:53:12 -05:00
Joseph Doherty 63331b0416 fix(security): per-instance credential references (#835) and confined OpcUaClient user-certificate path (#836)
#835: inside one driver namespace, a secret:<ns>/<X>/... reference whose first
segment names ANOTHER existing same-type driver instance (id or name,
case-insensitive; another-instance match beats an own-name match, so renaming
one's driver after a victim's unlocks nothing) is refused at save
(DriverConfigSaveGate.ValidateAsync), deploy (CredentialReferenceOtherInstance,
Heidenhain included), Test Connect (TestDriverConnect.DriverInstanceId, ConfigDb
read off the mailbox) and live browse (BrowserSessionService, fail-closed without
a ConfigDb). One rule: DriverCredentialPolicy.DescribeOtherInstanceProblems +
DriverInstanceIdentities. Shared names and the driver's own id stay valid; env:
is not instance-scoped. Running drivers are unchanged (warn-only window: the
deploy gate and the upgrade preflight / /raw banner list them).

#836: OpcUaClient UserCertificatePath is a relative path inside
Drivers:OpcUaClient:UserCertificateDirectory (default
<LocalAppData>/OtOpcUa/pki/user-certificates). Syntactic rule (CredentialFilePaths,
a DriverCredentialPolicy path field) at every #815 seam; at load time
ResolveSecretRefsAsync joins, canonicalises and requires lexical AND
link-resolved containment before the PFX password is resolved, and
LoadCertificate refuses an unresolved path. A running driver's pre-#836 absolute
path keeps loading with a rate-limited warning (LegacyCredentialReferenceTracker.ReportPath).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-11 03:52:48 -05:00
Joseph Doherty 7095e128e3 test(historian): shared ProcessedConformance asserts DurationGood/DurationBad on every SQL backend (#840)
#820 served DurationGood/DurationBad (ms) on SqlServer/QuestDB but only
QualityMixConformance asserted them. The shared Processed suite now covers
them like the percentages: hand values over tag A (20000/10000/20000/BadNoData
and 0/10000/0) with statuses + timestamps and Duration = Percent x interval;
the SDK-calculator parity theory (forward/reverse/truncated); outage markers;
the only-Bad interval; string tags; and the time-weighted row cap.
Live: SqlServer.IntegrationTests 211/211 (central SQL), QuestDb.IntegrationTests
249 passed / 1 skipped (shared /opt/otopcua-questdb fixture).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-11 03:50:12 -05:00
Joseph Doherty 19ffb3c902 feat(mtconnect): refused Agent connections mark values BadNoCommunication after a short confirmation; clamp the raised sample count to the Agent's bufferSize (#846)
- A /sample or re-baseline failure whose chain holds SocketError.ConnectionRefused, unbroken for
  CommsLossConfirmMs (new, default 2000; 0 = first refusal; negative = off), publishes every subscribed
  value BadNoCommunication once per outage; the first delivery afterwards republishes all of them from
  the index (unchanged values included). Timeouts/resets/DNS/pause keep the #527 watchdog contract.
- SampleCountCeiling clamps the adaptive raise to the /probe bufferSize (never below SampleCount).
- Docs: MTConnect.md comms-loss section + config key; drivers.md rule.

Refs #846
2026-11-11 03:48:05 -05:00
Joseph Doherty ddc1f29e3a feat(heidenhain): Part 9 condition class from alarm class/group, ALARM: selectors in the browse picker, unclaimed alarms counted + logged (#839)
- Core: optional AlarmEventArgs.ConditionClass (AlarmConditionClass Base/Process/Maintenance/System/Safety)
  carried by NativeAlarmProjector on AlarmConditionSnapshot and written to ConditionClassId/Name by
  OtOpcUaNodeManager (kept across class-less events, carried across a #641 rebuild).
- Heidenhain: HeidenhainAlarmClasses — HEIDENHAIN DNC-interface class/group names (corroborated by the
  captured iTNC 530 record); class 5 => Safety, group 1/2 => Process, 4/5 => System, else Base.
- DiscoverAsync offers AlarmConditions/ selector leaves (IsAlarm Boolean); browse-commit seeds the alarm object.
- Alarms no selector claims: otopcua.driver.alarms.unmatched counter, /hosts diagnostics, rate-limited Warning.

Refs #839
2026-11-11 03:41:14 -05:00