Commit Graph

29 Commits

Author SHA1 Message Date
Joseph Doherty 2238da6504 docs(717): T0 SDK 2.0.1 re-proof — metadata and read projection pass; provenance and first-delivery choice fail as designed
v2-ci / unit-tests (push) Has been cancelled
The throwaway loopback spike (11/11 over the wire) shows SDK 2.0.1 clones the cached NotificationMessage on every
Publish, queued send and Republish, and pools notification payloads, so design §4.6's object-keyed provenance and
first-delivery ledger never match a delivered object (0 hits; Good leaks after demotion). A ledger keyed by
(SubscriptionId, SequenceNumber) plus a provenance token on the clone-copied MonitoredItemNotification.Message helper
passed the same matrix. Per the plan's T0 exit rule this re-opens D4's mechanism; T1+ wait for the owner's go/no-go.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 02:56:54 -05:00
Joseph Doherty 9375c9e287 docs(deploy): DEV vd03 upgraded fea7de0a -> 6c4d2990; #700 Boolean/Int32 typing + native TimeAverage/Total pass on the deployed endpoint
v2-ci / unit-tests (push) Has been cancelled
vd03 runs master 6c4d2990 (#899/#898 read-only historian bindings), self-contained
win-x64, ~15 s downtime, ServiceLevel 250, FOCAS 21/22 Good as before, runner InProcess,
worker isolation re-applied after the folder swap. #898 preflight: 0 historized tags.

#700 second run on the deployed endpoint with historianReadOnly bindings: Boolean
(SysPulse) and Int32 (SysTimeSec) Raw/AtTime typed as the node DataType; native
TimeAverage/Total 9.5/29.5/49.5 and 190/590/990 (= in-process reference), WTC1
all-NULL intervals BadNoData. HistorianGateway /metrics: no EnsureTags/WriteLiveValues
or any write. Temp config + key removed, config back to rev fa9a2e28.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 01:36:30 -05:00
Joseph Doherty 5683d89bda docs(evidence): #700 HistoryRead through the deployed DEV vd03 endpoint (fea7de0a, HG ea9f21c)
v2-ci / unit-tests (push) Has been cancelled
Temporary Calculation-driver nodes bound to existing AVEVA tags (Float ZIPSEN.WTC1,
Byte SysTimeSec/SysPulse, Double missing tag) on vd03, read over OPC UA from the Mac,
then removed; the config is back at its original revision fa9a2e28. Zero historian
writes (gateway metrics: Probe/GetTagMetadata/ReadRaw only). Int32/Boolean bindings
were not deployed because they would trigger an EnsureTags upsert of the existing
AVEVA tag (#899); Float/Double/Byte never provision (#898). Includes the read-only
probe used.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 00:43:38 -05:00
Joseph Doherty ee39c169df fix(historian): refuse aggregates not valid for the node's DataType up front (#897)
v2-ci / unit-tests (push) Has been cancelled
TimeAverage / Total on a Boolean node over the Gateway backend (AVEVA discrete tag, live
SysPulse) went to the gateway since native TimeAverage/Total became default-on (#694). AVEVA
refuses both modes at StartQuery; the gateway reports that only as Internal /
HistorianServerOperationException (the same answer as a historian storage fault), so the
node answered BadHistoryOperationUnsupported and the read counted as a backend failure.

HistoryRead-Processed now checks the node's scalar DataType before calling any backend and
answers BadAggregateNotSupported (Part 11 6.5.4: "If the Aggregate is not valid for the Node
then the StatusCode shall be Bad_AggregateNotSupported"). Nothing is sent, nothing is counted.
- Shared, every backend: HistoryAggregateDataTypes (Core.Abstractions) applies Part 13
  5.4.2.3 Table 50: the numeric-only aggregates (Average, TimeAverage(2), Total(2), Min/Max,
  ActualTime, Range, Delta, DeltaBounds, Interpolative, stddev/variance) on a String or
  DateTime node are refused. Client-visible: the SQL backends used to answer every interval
  BadAggregateInvalidInputs.
- Per backend: IHistorianDataSource.SupportsAggregateFor(aggregate, TypeCode?) (default true;
  every production backend declares it, guard-tested). Gateway: false for native
  TimeAverage/Total on a Boolean node. SqlServer/QuestDb: true, since they serve Boolean as
  0/1 (stepped), so Boolean stays served there, as do the Gateway's raw-evaluated aggregates.

The residual case (a node declared numeric whose AVEVA tag is discrete) cannot be told apart
from a historian fault on the wire and still counts; that needs the gateway to surface
AVEVA's native error code as a deterministic status.

Tests: HistoryAggregateDataTypesTests, guard test, node-manager String/DateTime and per-type
refusals, Gateway capability matrix, DiscreteTagTimeWeightedWireTests (production Gateway
backend over an AVEVA-shaped fake: unsent, 0 health failures, positive controls). Each
mutation (no check, no shared rule, no backend hint, Gateway hint true, shared rule always
true, SQL inheriting the default) fails at least one. Live gate re-run read-only against
AVEVA 2023 R2 via a private HistorianGateway fa9cf31 container: SysPulse TimeAverage on the
wire = BadAggregateNotSupported, health failures unchanged; #801 gate 244 cases pass.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 00:41:33 -05:00
Joseph Doherty 639f511cfc docs(evidence): #694/#700/#746 live AVEVA gate through HistorianGateway dffb5e2 on 4c8e977e
v2-ci / unit-tests (push) Has been cancelled
Read-only. Native TimeAverage/Total forward/reverse, clipped, sub-ms clipped,
fractional-ms and all-NULL intervals vs a step-held raw reference; #700 missing /
existing / typed history, future AtTime, outage statuses, strict-ceiling
stopped-tag reverse (10 calls, bound 12) and lowered/raised limits with the
aggregate cooldown; #746 server-capped events paging (0 abandoned streams vs 81
with the cap removed). Discrete-tag finding filed as #897.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 00:09:45 -05:00
Joseph Doherty fea7de0aef docs(evidence): #799 acceptance runs on master aa60b3e3 - Windows x64 and Linux both fail the unchanged cold RSS cap
v2-ci / unit-tests (push) Waiting to run
Owner scope 2026-11-12 (Windows x64 + Linux, unchanged cycle-3 caps, >=7200 s, SDK 1.5 comparator).

- Linux ARM64 (late-slope, 7200.12 s, 1964 cycles / 491 restarts, shutdown verified, analyzer-validated):
  first rejection RSS at cycle 247 / 900.73 s, 358,682,624 > 358,678,528 (+4 KiB); peak 382.98 MiB vs
  limit 342.06 MiB; late plateau medians 344.4/341.1/341.9 MiB, OLS -1.90 MiB/h. FAIL.
- Windows x64 on windev (same workload, working set as RSS, handle table as descriptors): first rejection RSS
  at cycle 285 / 1130.76 s, 303,505,408 > 302,174,208; peak 319.49 vs limit 288.18 MiB; plateau ~314-315 MiB,
  +0.93 MiB/h. FAIL. Also incomplete: the launcher's wall-clock outer deadline fired early after a +552 s
  w32time step and ended the run at test-phase 7014 s (rejection had already latched); launchers now use a
  Stopwatch.
- SDK 1.5.378.182 comparator (efb9197a publication, Windows, 7201 s): RSS peak +65.8 MiB (never over its cap),
  plateau ~+48 MiB, but the +32 handle cap tripped at cycle 17 (max 674 vs 642).
- Managed flat on both SDK2 runs; growth is the generated-factory JIT burst (+40k/+44k methods) - Linux JIT
  doublemapper ~180 MiB; Windows private bytes grew ~25 MiB of a ~132 MiB working-set rise.
- Retained: b61f3e05 cycle-1 fixture failure (#808 unplaced UNS folder), Windows/SDK1 smoke diagnostics.

No cap raised, no warm baseline, no failed run counted as passing.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 14:51:31 -05:00
Joseph Doherty b61f3e0560 fix(galaxy): AVEVA priority 1 is the most urgent: Galaxy alarm Severity follows AVEVA's priority bands, live and history alike (#887)
v2-ci / unit-tests (push) Has been cancelled
GalaxyAlarmPriority, the one table live Galaxy alarms (MxAccessSeverityMapper)
and AVEVA A&E history (EventMapper, #886) map the AVEVA priority through, ranked
a higher number as more severe. AVEVA ranks priority 1 as the most urgent, so a
priority-1 alarm read 250 and a priority-999 system event 1000.

It now uses AVEVA's default priority ranges, most urgent first:

  AVEVA priority  AVEVA severity  bucket    OPC UA Severity
  1-250           1 Critical      Critical  1000
  251-500         2 High          High       750
  501-750         3 Medium        Medium     500
  751-999         4 Low           Low        250
  (history only)  0 not an alarm  -            1

Out-of-range priorities clamp (<= 0 Critical, > 999 Low). Evidence, vd03
Runtime.dbo.Events read-only: priority 1/100 alarms carry AVEVA severity 1,
400/500 severity 2, 600/700 severity 3, and every priority-999 System.* /
User.* row severity 0 with IsAlarm 0.

- Live and history now use the same number. GalaxyDriver passes the table's
  exact value as AlarmEventArgs.OpcUaSeverity. Before, it passed only the bucket,
  which NativeAlarmProjector re-mapped through its generic 200/500/700/900
  ladder, so live and history disagreed even before the direction fix.
- History: a row whose AVEVA severity is 0 (System.Start/Stop/Deploy,
  User.Write...) is not an alarm and reads Severity 1, below every alarm band.
  For alarms AVEVA's 1-4 severity is still never the value: the live feed
  carries only the priority, so it cannot use it, and history must agree.
- Docs: docs/Historian.md gains "Galaxy alarm priority -> Severity
  (lmxopcua#887)"; docs/AlarmTracking.md updated.
- Live re-run of the #801/#886 harness against vd03 HistorianGateway
  ea9f21c (read-only, network-only): 224 of 233 cases identical to the #886
  run, the 9 others differ only in Severity (priority-700 alarm rows 750 ->
  500, 19 system/user events 1000 -> 1). Record: aveva-history-801.md
  "#887 re-run", report aveva-gate-sdk2-887.json.gz.

Client-visible: every OPC UA client that ranks or filters Galaxy alarms by
Severity sees the order reversed (live and HistoryReadEvents). ScadaBridge has
no code that depends on the old values; its own mixed severity directions are
ScadaBridge#129.

Tests updated because they pinned the inverted table:
- Driver.Galaxy.Tests MxAccessSeverityMapperTests (rewritten: edges 1/250/251/
  500/501/750/751/999, 0/-100/int.MinValue/1000/int.MaxValue clamps,
  monotonic-decreasing sweep, equals GalaxyAlarmPriority for -2..1001)
- Driver.Galaxy.Tests GatewayGalaxyAlarmFeedTests (750 and 600 -> Medium)
- Driver.Historian.Gateway.Tests EventMapperTests (band table, priority 999
  -> 250, 1 -> 1000; system events -> 1)
- OpcUaServer.IntegrationTests GalaxyEquipmentEventHistoryWireTests (700 ->
  500, 100 -> 1000, System.Stop -> 1; the fake now stores AVEVA's real 0-4
  severity)

New tests:
- Runtime.Tests NativeAlarmRestatementEndToEndTests
  .Galaxy_live_condition_severity_is_the_AVEVA_priority_band (8 rows: the real
  feed -> real GalaxyDriver -> host -> projector; the same rows EventMapperTests
  pins on the history path)
- EventMapperTests.Aveva_severity_zero_marks_an_event_that_is_not_an_alarm
  (5 rows, incl. the priority-999 alarm positive control)
- GalaxyDriverAlarmSourceTests: OpcUaSeverity passes through

Mutation check, 8 mutants, all killed: the pre-#887 inverted table; the
driver dropping OpcUaSeverity; no not-an-alarm rule; each of the three band
edges off by one; history ignoring priority; a missing AVEVA severity read
as 0.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 12:14:49 -05:00
Joseph Doherty 3a312ffdd1 fix(history): a folder's event history reads the Galaxy objects behind its native conditions (#886)
v2-ci / unit-tests (push) Has been cancelled
A notifier folder read one event source, its NodeId identifier (a RawPath or an EquipmentId). AVEVA stores a
Galaxy object's A&E history under the object's tag_name (Source_Object), so AVEVA's own alarm and event
history reached no production folder. Its lower-case property keys were also missed, so every row read
Severity 1 with no ConditionName.

Sources. A folder's event history now reads its own source plus the Galaxy object of every native condition
that fires on it live: those parented under it and those WireAlarmNotifiers wired to it. Live and history use
the same membership. The object comes from the condition's Galaxy alarm reference, its raw tag's attributeRef,
up to the first '.':
- The composer reads it with the driver's own parser (GalaxyAttributeRef, shared with GalaxyDriver), only for
  a GalaxyMxGateway tag with an alarm (RawTagPlan.NativeAlarmReference, part of plan equality).
- The applier binds it after materialising the condition (IOpcUaAddressSpaceSink.BindNativeAlarmHistory,
  forwarded by the deferred sink). A rebuild or the condition's removal forgets it.
Each object is read with the #882 rule: its own events and <Object>.<Attribute> rows, never a prefix sibling.

Fan-out. historiangw's ReadEvents filter is a single predicate, so there is one request per source; the wire
contract is unchanged. EventSourceFanOut merges the reads as one source:
- order is (time, source position, arrival), reversed for a reverse read; ties are never re-sorted by EventId,
  so the boundary skip stays stable;
- duplicates by EventId are dropped and the merge is cut to the cap;
- a truncated source bounds the page at the time it stopped.
Paging, continuation points (bound to the whole source set), the walked-back reverse read and the tie-cluster
over-fetch are unchanged. A folder with one source is read exactly as before.

ACL. Every source is read inside the folder's own HistoryRead, after AuthorizeHistoryRead (#826). A denied
folder reads nothing and fails closed.

Mapping (EventMapper, property names now case-insensitive):
- Severity: an AVEVA row's priority goes through GalaxyAlarmPriority, the table live Galaxy alarms use
  (MxAccessSeverityMapper now delegates to it), so history matches live. Otherwise severity (OtOpcUa's own
  rows, which AVEVA stores lower-cased), clamped. AVEVA's own 1-4 severity is ignored. The table ranks a
  higher priority as more severe, which inverts AVEVA's convention: filed as #887.
- An AVEVA row whose source_conditionvariable names a native condition that fires on the folder takes that
  condition's SourceNode and live ConditionName. Otherwise ConditionName is alarmname or the condition
  variable's attribute.
- alarm_id/alarmid map to AlarmId.

Tests:
- GalaxyEquipmentEventHistoryWireTests (4, real SDK session, production Gateway backend, AVEVA-shaped fake
  gateway): equipment EqA backed by objects Oven_007 + OvenDoor_07. Covers object and attribute events from
  both objects, the prefix sibling excluded, cross-object and in-object ties, forward paging at 1-4 a page,
  reverse whole and paged, SourceNode/ConditionName/Severity, the raw device folder, ACL deny with no backend
  call plus a granted positive control, and a rebuild forgetting the bindings.
- EventSourceFanOutTests (8), EventMapper AVEVA cases (11), GalaxyAttributeRefTests (16), a composer test and
  an applier ordering test.
- Mutation-checked, 17 mutants, all killed. Covered: parent-only membership, no scope, a case-sensitive
  condition match, no condition rewrite, no truncation bound, EventId tie order, no sort, no cap, no stop at
  the bound, unreversed source order, no dedupe, no '.' boundary, priority ignored, case-sensitive keys, no
  applier bind, no deferred forward, and a composer that ignores the driver type. The forget-on-rematerialise
  guard survives as belt-and-braces: the rebuild clear covers it.
- Live, read-only, no WinRM, vd03 HistorianGateway ea9f21c: equipment folder EQ-886-ZipsenLeft (objects
  DevWinPlatform + Left_001). It read 0 events before and 92 now, equal to the unfiltered read's rows for both
  objects. 1/7-event pages and reverse whole/5-event pages agree, and 73 rows resolve to their live condition.
  diff_runs vs the #882 run: 221/226 identical. The 5 that differ are expected: DevWinPlatform Severity
  1 -> 750/1000, and #881 capabilities.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 11:45:48 -05:00
Joseph Doherty da948e5fb5 fix(history): a folder's event history includes its object's attribute alarms (#882)
v2-ci / unit-tests (push) Has been cancelled
HistorianGateway answers a per-source ReadEvents (Source_Object = DevWinPlatform) with the
object's own events (SourceName DevWinPlatform) AND its attribute alarms, which it reports as
<Object>.<Attribute> (Source_ProcessVariable, e.g. DevWinPlatform.CPULoadAvg). The Gateway
backend's defensive client-side source filter compared SourceName for equality, so it dropped
the attribute alarms: 70 of the 72 DevWinPlatform events on vd03.

The filter (and the past-the-cap "more events" signal) now uses EventSourceMatch: the source
itself, or <source>.<anything> on the '.' boundary, ordinal. DevWinPlatform never matches
DevWinPlatform2 or DevWinPlatform2.X. This needs no wire change and still makes one request.
Each event keeps its own SourceName. SourceNode stays the condition its stored alarm id names,
which is null for AVEVA rows. The attribute events ride the notifier's own read, so the
per-folder HistoryRead verdict (#826) governs them: a denied folder answers
BadUserAccessDenied before any backend call. The SqlServer and QuestDB backends still match
exactly; they hold only OtOpcUa's own events.

Tests:
- GatewayAttributeEventSourceTests (16): the boundary theory, attribute-in/sibling-out through
  the data source, the drain signal, and an exact attribute-named read.
- AttributeEventHistoryWireTests (2, over a real SDK session with the production Gateway
  backend on an AVEVA-shaped deterministic gateway): object + attribute events with their own
  SourceName, a prefix sibling excluded, paging, and the ACL-on deny with no backend call plus a
  granted positive control.
- Mutation-checked: exact match fails both wire tests; dropping the '.' boundary fails the
  wire sibling check and 6 unit tests.
- Live AVEVA harness re-run against vd03 HistorianGateway ea9f21c (read-only, no WinRM):
  folder DevWinPlatform 2 -> 72 events (whole, 72 1-event pages, reverse); the new #882 check
  passes; 221 of 225 shared cases are identical to the #801 SDK2 baseline, and the 4 that
  differ are exactly the DevWinPlatform event cases. Recorded in aveva-history-801.md.

Follow-up #886: production folders name RawPaths/EquipmentIds, not Galaxy objects, plus the AVEVA
severity/condition property mapping.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 10:34:29 -05:00
Joseph Doherty 008f0b36b2 docs(evidence): #801 real-AVEVA history gate — SDK 2.0.1 identical to 1.5.378.182
v2-ci / unit-tests (push) Waiting to run
Both builds served HistoryRead from the real AVEVA Historian via the DEV
HistorianGateway (vd03, ea9f21c): 225 cases / 3,855 rows / events, 0 differences.
Raw also matches direct SQL on Runtime.dbo.History (6/6) and histgw-cli; simple
aggregates match a recomputation from raw (336/336). Modified, HistoryUpdate,
seven refused aggregates and HistoryServerCapabilities answer as before. No write
reached the historian. Findings (pre-existing, not SDK): all HistoryServerCapabilities
booleans false; AVEVA attribute-alarm events unreachable per source.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 10:02:34 -05:00
Joseph Doherty da4e27003e docs: record completed Linux OPC UA lifecycle diagnostic
v2-ci / unit-tests (pull_request) Waiting to run
2026-11-10 20:28:11 -05:00
Joseph Doherty 1e4b017f6e docs: retain verified Linux rehearsal and diagnostic gate
v2-ci / unit-tests (pull_request) Failing after 1h6m11s
2026-11-10 18:25:37 -05:00
Joseph Doherty b18f97236a docs: record reviewed private temp mount controls
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 18:18:59 -05:00
Joseph Doherty 351edb0a86 docs: retain read-only Linux rehearsal startup failure 2026-11-10 18:13:00 -05:00
Joseph Doherty 626cb4f7f1 docs: record final late-slope controls before artifact preparation
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 18:03:13 -05:00
Joseph Doherty 7023285a3e docs: retain late-slope guard control receipts 2026-11-10 17:54:43 -05:00
Joseph Doherty 041189aa6d docs: close SDK2 ownership evidence review
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 17:35:54 -05:00
Joseph Doherty 327bbdaab9 docs: record reviewed SDK2 ownership regressions and diagnostic design 2026-11-10 17:33:06 -05:00
Joseph Doherty ce21f5db09 Merge master SDK1 ownership repair with SDK2 disposal adaptation 2026-11-10 17:17:00 -05:00
Joseph Doherty efb9197a0f docs: record reviewed SDK1 terminal disposal repair
v2-ci / unit-tests (push) Has been cancelled
2026-11-10 17:05:06 -05:00
Joseph Doherty 2cf7c1d248 docs: locate close control build receipt
v2-ci / unit-tests (pull_request) Waiting to run
2026-11-10 16:40:28 -05:00
Joseph Doherty 0c0209f9a6 docs: record reviewed close ownership and native attribution 2026-11-10 16:39:34 -05:00
Joseph Doherty 331cde68a7 test: record final integrated CI baseline-only failures
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 16:05:13 -05:00
Joseph Doherty 4a04806d22 test: preserve saturation outcome and classify final CI corrections
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 15:46:58 -05:00
Joseph Doherty c6c05a1001 test: record publish smoke with HistorianGateway 1.0 packages 2026-11-10 14:52:08 -05:00
Joseph Doherty 67bc61236a Merge remote-tracking branch 'origin/master' into codex/opcua-sdk-2 2026-11-10 14:40:51 -05:00
Joseph Doherty 230cef4ef0 build: adopt HistorianGateway 1.0 TLS options
v2-ci / unit-tests (push) Has been cancelled
2026-11-10 14:40:30 -05:00
Joseph Doherty b218321bda docs: retain final CI failures and Linux RSS gate evidence
v2-ci / unit-tests (pull_request) Waiting to run
2026-11-10 13:28:40 -05:00
Joseph Doherty aa9af0dcff docs: retain backend benchmark and actual publish evidence 2026-11-10 13:17:13 -05:00