Commit Graph

590 Commits

Author SHA1 Message Date
Joseph Doherty 16ddddb2eb docs(615): G5 signed off by the Fleet Admin; every #615 gate complete, #615 closed
v2-ci / unit-tests (push) Has been cancelled
2026-11-13 08:09:37 -05:00
Joseph Doherty c9bc358dc9 feat(cli): print array values; docs: #615 G9 PASS with otopcua-cli, G8 waived
v2-ci / unit-tests (push) Has been cancelled
G9 ran on an isolated 2-node pair (otopcua-g9, 003c7f89, Mode unset as
deployed) with otopcua-cli as the owner-approved client: A1-A3, A5, A7, A8
(graceful, ~11 s gap), B2 (docker kill, failover <1 s, ~2 s gap, 0 Bad,
survivor 250 at +44 s) and B3 (no failback) pass; A4/A5b unmet under Unset,
met under Warm (#717). G8 (CTT) waived by the owner for GA; G5 is the only
open exit criterion.

The run found the CLI printing every array as System.String[] (ServerArray
unreadable); ValueFormatter now renders [a, b] / hex for read, subscribe and
historyread, with tests. Runbook A4/A5b text updated for the #717 modes.
Evidence: docs/plans/evidence/2026-11-13-615-g9/RESULTS.md.
2026-11-13 07:46:56 -05:00
Joseph Doherty 36eb6d558d docs(717): Mode=Unset re-check after c569f806 (no change); a leaving node reports 0
v2-ci / unit-tests (push) Has been cancelled
T13b: the isolated otopcua-t13b pair with OpcUa:Redundancy:Mode absent, on origin/master f826af8e
(it contains c569f806) and A/B on c569f806^ (687256d6). Every check passes on both builds:
- levels 250/240;
- Good data on both nodes;
- writes, Acknowledge, AddComment and alarm delivery on the Secondary;
- graceful stop: the survivor reaches 250 259 ms after Leaving (pre-fix 270 ms);
- crash: 250 at +43.6 s (pre-fix +42.8 s), with no demotion at auto-down;
- rejoin at 240 with no failback.
The role-decision changes are gated on IsWarm and do not run under Unset.

Plan L5, the design's section 4.3 and the Warm table in Redundancy.md now say a Leaving/Exiting node
reports ServiceLevel 0, not 90 (ComputeWarm returns 0 for any member that is not Up/Joining; T13 note A).

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 06:44:46 -05:00
Joseph Doherty f826af8e8a docs(717): T13 live gate evidence on an isolated Warm pair
v2-ci / unit-tests (push) Waiting to run
Every applicable T13 leg passes on 0c19f26c + c569f806. The leg list covers:
- metadata and ServiceLevel bands;
- standby projection;
- alarm suppression with Standby ConditionRefresh and the promotion refresh;
- the three D4 demotion timings, run live;
- standby write and Part 9 refusal (BadInvalidState);
- graceful and crash failover, with no failback;
- subscription transfer;
- the Mode=Unset regression.

The two crash-path defects the gate found are fixed in c569f806. Evidence, rig files and the
raw-client harness are in docs/plans/evidence/2026-11-13-717-t13/.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 04:39:17 -05:00
Joseph Doherty 687256d68e test(717): deferred Warm coverage - 2-node handover, DataEncoding, deleted/reused items, clone matrix, ACL stand-down, per-counter metrics
v2-ci / unit-tests (push) Has been cancelled
- T3 WarmSingletonHandoverTests: real 2-node cluster, singleton under ClusterSingletonManager on both nodes; the old
  incarnation's held-back publish (older revision, higher sequence) reaches a real Warm OpcUaPublishActor after the
  handover and never demotes the new Primary. Pin: a removed node cannot rejoin from its old ActorSystem; a new one
  rejoins younger (higher up-number).
- T6 DataEncoding: Binary/XML encoded reads of a structured tag projected on the Standby; scalar encoding error kept;
  a DefaultXml item is projected, then promoted through its own encoding (TypeId i=888).
- T7 item deleted between queue and publish; ClientHandle reused by i=2267 on a payload recycled through the SDK 2.0.1
  pool (precondition asserted); WarmStandbyCloneMatrixTests over DataChange+DiagnosticInfos/EventNotificationList/
  StatusChange, clone/cached immutability, encode round-trip, real-pool stale marker.
- T8 ACL-stood-down item receives nothing from demotion or promotion; restored grant reaches it again.
- T10 per-counter MeterListener tests (by instrument instance): warm_transition, warm_role gauge, snapshot_rejected
  (publish actor both reasons + DriverHostActor's ordering gate, first actor test of it), projected_notifications,
  suppressed_alarm_events. Process-wide state tests run in non-parallel collections.
- Every test mutation-checked (26 mutations killed); no product bug found. Plan status updated.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 04:31:11 -05:00
Joseph Doherty f817c77e02 test(717): deferred-event boundary, ACL-filtered promotion refresh, native-ack refusal, effective-URI XML load; docs + T13 overlay
v2-ci / unit-tests (push) Has been cancelled
- Wire: an event reported while Active but deferred (#838) is delivered after demotion; the promotion refresh
  withholds an ACL-denied condition (positive control alongside); ConditionRefresh2 and model-change events on the
  Standby; a native Acknowledge is refused and never routed; Warm validates and advertises the EFFECTIVE
  ApplicationUri from an ApplicationConfigPath XML load.
- docs/Redundancy.md: Warm section, the Unset/None i=11314 behaviour (orphan readable by NodeId under Unset,
  deleted under None), instruments; OpcUaServer.md, Configuration.md, security.md, cluster-topology rule.
- docker-dev/docker-compose.warm.yml for the T13 gate (central pair, Mode=Warm).
- Plan: implementation status and the test gaps left for T13/follow-up.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 03:39:50 -05:00
Joseph Doherty 8667ce58aa docs(717): adopt the T0-proven first-delivery mechanism in spec §4.6/§10 and plan T2/T7/T13
v2-ci / unit-tests (push) Has been cancelled
The decision record is keyed by (SubscriptionId, SequenceNumber); the ordinary-tag marker rides
MonitoredItemNotification.Message, validated by ClientHandle and DataValue. The SDK's cached copy is
never mutated. None deletes the SDK's orphan i=11314. T13 gains transfer and durable-restore legs.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 03:03:56 -05:00
Joseph Doherty 2238da6504 docs(717): T0 SDK 2.0.1 re-proof — metadata and read projection pass; provenance and first-delivery choice fail as designed
v2-ci / unit-tests (push) Has been cancelled
The throwaway loopback spike (11/11 over the wire) shows SDK 2.0.1 clones the cached NotificationMessage on every
Publish, queued send and Republish, and pools notification payloads, so design §4.6's object-keyed provenance and
first-delivery ledger never match a delivered object (0 hits; Good leaks after demotion). A ledger keyed by
(SubscriptionId, SequenceNumber) plus a provenance token on the clone-copied MonitoredItemNotification.Message helper
passed the same matrix. Per the plan's T0 exit rule this re-opens D4's mechanism; T1+ wait for the owner's go/no-go.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 02:56:54 -05:00
Joseph Doherty a42c9d7f7c docs(717): record the owner's 2026-11-13 answers in the Warm redundancy spec and plan
Standby writes and Part 9 methods refused BadInvalidState under Warm, promote while the Primary is
unreachable (OpcUa:Redundancy:PromoteWhilePrimaryUnreachable, default on), standby status code
BadOutOfService, no ConfigDb migration, bands as drafted. ScadaBridge#138 is the filed prerequisite.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 02:56:54 -05:00
Joseph Doherty 9375c9e287 docs(deploy): DEV vd03 upgraded fea7de0a -> 6c4d2990; #700 Boolean/Int32 typing + native TimeAverage/Total pass on the deployed endpoint
v2-ci / unit-tests (push) Has been cancelled
vd03 runs master 6c4d2990 (#899/#898 read-only historian bindings), self-contained
win-x64, ~15 s downtime, ServiceLevel 250, FOCAS 21/22 Good as before, runner InProcess,
worker isolation re-applied after the folder swap. #898 preflight: 0 historized tags.

#700 second run on the deployed endpoint with historianReadOnly bindings: Boolean
(SysPulse) and Int32 (SysTimeSec) Raw/AtTime typed as the node DataType; native
TimeAverage/Total 9.5/29.5/49.5 and 190/590/990 (= in-process reference), WTC1
all-NULL intervals BadNoData. HistorianGateway /metrics: no EnsureTags/WriteLiveValues
or any write. Temp config + key removed, config back to rev fa9a2e28.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 01:36:30 -05:00
Joseph Doherty 5683d89bda docs(evidence): #700 HistoryRead through the deployed DEV vd03 endpoint (fea7de0a, HG ea9f21c)
v2-ci / unit-tests (push) Has been cancelled
Temporary Calculation-driver nodes bound to existing AVEVA tags (Float ZIPSEN.WTC1,
Byte SysTimeSec/SysPulse, Double missing tag) on vd03, read over OPC UA from the Mac,
then removed; the config is back at its original revision fa9a2e28. Zero historian
writes (gateway metrics: Probe/GetTagMetadata/ReadRaw only). Int32/Boolean bindings
were not deployed because they would trigger an EnsureTags upsert of the existing
AVEVA tag (#899); Float/Double/Byte never provision (#898). Includes the read-only
probe used.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 00:43:38 -05:00
Joseph Doherty ee39c169df fix(historian): refuse aggregates not valid for the node's DataType up front (#897)
v2-ci / unit-tests (push) Has been cancelled
TimeAverage / Total on a Boolean node over the Gateway backend (AVEVA discrete tag, live
SysPulse) went to the gateway since native TimeAverage/Total became default-on (#694). AVEVA
refuses both modes at StartQuery; the gateway reports that only as Internal /
HistorianServerOperationException (the same answer as a historian storage fault), so the
node answered BadHistoryOperationUnsupported and the read counted as a backend failure.

HistoryRead-Processed now checks the node's scalar DataType before calling any backend and
answers BadAggregateNotSupported (Part 11 6.5.4: "If the Aggregate is not valid for the Node
then the StatusCode shall be Bad_AggregateNotSupported"). Nothing is sent, nothing is counted.
- Shared, every backend: HistoryAggregateDataTypes (Core.Abstractions) applies Part 13
  5.4.2.3 Table 50: the numeric-only aggregates (Average, TimeAverage(2), Total(2), Min/Max,
  ActualTime, Range, Delta, DeltaBounds, Interpolative, stddev/variance) on a String or
  DateTime node are refused. Client-visible: the SQL backends used to answer every interval
  BadAggregateInvalidInputs.
- Per backend: IHistorianDataSource.SupportsAggregateFor(aggregate, TypeCode?) (default true;
  every production backend declares it, guard-tested). Gateway: false for native
  TimeAverage/Total on a Boolean node. SqlServer/QuestDb: true, since they serve Boolean as
  0/1 (stepped), so Boolean stays served there, as do the Gateway's raw-evaluated aggregates.

The residual case (a node declared numeric whose AVEVA tag is discrete) cannot be told apart
from a historian fault on the wire and still counts; that needs the gateway to surface
AVEVA's native error code as a deterministic status.

Tests: HistoryAggregateDataTypesTests, guard test, node-manager String/DateTime and per-type
refusals, Gateway capability matrix, DiscreteTagTimeWeightedWireTests (production Gateway
backend over an AVEVA-shaped fake: unsent, 0 health failures, positive controls). Each
mutation (no check, no shared rule, no backend hint, Gateway hint true, shared rule always
true, SQL inheriting the default) fails at least one. Live gate re-run read-only against
AVEVA 2023 R2 via a private HistorianGateway fa9cf31 container: SysPulse TimeAverage on the
wire = BadAggregateNotSupported, health failures unchanged; #801 gate 244 cases pass.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 00:41:33 -05:00
Joseph Doherty 639f511cfc docs(evidence): #694/#700/#746 live AVEVA gate through HistorianGateway dffb5e2 on 4c8e977e
v2-ci / unit-tests (push) Has been cancelled
Read-only. Native TimeAverage/Total forward/reverse, clipped, sub-ms clipped,
fractional-ms and all-NULL intervals vs a step-held raw reference; #700 missing /
existing / typed history, future AtTime, outage statuses, strict-ceiling
stopped-tag reverse (10 calls, bound 12) and lowered/raised limits with the
aggregate cooldown; #746 server-capped events paging (0 abandoned streams vs 81
with the cap removed). Discrete-tag finding filed as #897.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 00:09:45 -05:00
Joseph Doherty d9b2381235 docs(#615): G6 PASS, end to end against the live windev Galaxy; evidence + readiness record
v2-ci / unit-tests (push) Has been cancelled
The run covered browse (raw + UNS), read, subscribe, write and readback, and a native alarm
through Part 9 to an ack confirmed in Galaxy and a clear. The run found #895 (fixed in
c971b171) and #896 (open). DeployWatcher was not exercised: there is no headless Galaxy
redeploy.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 23:37:17 -05:00
Joseph Doherty a17ffb721a docs(deploy): QA canary, #890 re-apply, live #794 worker check and #780 runaway drill on vq03/vq04
v2-ci / unit-tests (push) Has been cancelled
- #890: installer at f75ee7d0 re-applied on both nodes with -DenyPaths root, pki, data and
  C:\ProgramData\ZB\kek (the Secrets KEK sits outside the install root); second apply 12/12 unchanged.
- Script-runner canary authored in the AdminUI and deployed (Calculation tag + VirtualTag, 1 s timers):
  Good and advancing on both endpoints; kept on QA as a runner monitor.
- #794 step 5: the live worker token is restricted (SeChangeNotify only; identity/Everyone/Users/RESTRICTED)
  and is denied read and write on pki, data, app, logs and the KEKs; network sockets none. Finding #891.
- #780: QA runaway drill passed (killed at the 4000 ms hard deadline twice, quarantined, siblings Good,
  ServiceLevel 250/240 unchanged); drill removed, redeploy back to the canary revision 82dd0598.
- #780 P4 recorded: InProcess stays the default and the rollback; StuckOrphanQuarantineFactor documented
  as in-process only.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 19:08:50 -05:00
Joseph Doherty f75ee7d089 fix(scripting): ScriptWorker isolation denies the worker every sensitive sibling, not just app\ (#890)
v2-ci / unit-tests (push) Waiting to run
The installer denied NT SERVICE\OtOpcUaScriptWorker only on -InstallRoot. On the wonder layout the node's keys and
secrets sit beside it (E:\ApiInstall\OtOpcUa\pki with the application certificate's private key, data\ with
secrets.db and LocalDb), inheriting BUILTIN\Users:RX. Users is a restricting SID of the worker token, so a worker
could read them.

- Install-ScriptWorkerIsolation.ps1 -DenyPaths <dir>,...: each folder gets the same inherited deny as InstallRoot
  (new SensitiveDirDeny steps). Refuses drive/share roots, script-worker\ and the PE cache; folders must exist;
  a comma/semicolon string works for powershell -File.
- The deny is read AND write (0xD0157: ReadData, WriteData, AppendData, WriteEA, DeleteChild, WriteAttributes,
  Delete, WRITE_DAC, WRITE_OWNER). Traverse/Execute, ReadAttributes, ReadControl and Synchronize stay so the
  loader still reaches script-worker\. The write half stops a worker planting a cert in pki\trusted\certs where
  a data drive lets Users create files in subfolders.
- The list is recorded under the anchor service's key (Parameters\DenyPaths, step DenyPathRecord): a re-run
  removes the deny from folders dropped from the list, -Uninstall removes every recorded one without repeating
  -DenyPaths. -WhatIf/-PlanOnly print every step.
- After an apply, WARN for each protected-DACL item under a denied folder that the inherited deny cannot reach;
  without -DenyPaths, WARN for pki/data/logs/backup/config/secrets siblings of InstallRoot.

Tests:
- Dry-run harness 46 -> 86 assertions (QA layout plan, mask bits, refusals, stale removal, record-driven
  rollback, host account never denied). Mutations (no sibling deny, deny->allow, mask back to ReadData, no
  stale removal, rollback ignoring the record, script-worker not refused) each turn it red.
- Test-ScriptWorkerIsolation.Windows.ps1 (Windows PowerShell 5.1, windev): 34/34, incl. -DenyPaths apply,
  11/11 unchanged re-apply, shorter-list re-apply, uninstall from the record, second uninstall unchanged.
- New InstallerLayoutIsolationTests (OTOPCUA_ISOLATION_INSTALLER_E2E=1, elevated Windows): a scratch wonder
  layout under the real restricted worker token. Before install and after uninstall the token reads pki/data/
  logs/backup/appsettings and plants a trust-list file; installed it can do neither, still reads script-worker\,
  a real worker launched from the layout runs a script, and the host still reads everything. Mutating the
  -DenyPaths denies to a no-op mask turns it red on pki\own\private\otopcua.pfx (reproduces #890).

QA vq03/vq04 are NOT re-applied; the command is in docs/deploy/qa-upgrade-b8b4c18f-to-fea7de0a.md.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 17:31:28 -05:00
Joseph Doherty fea7de0aef docs(evidence): #799 acceptance runs on master aa60b3e3 - Windows x64 and Linux both fail the unchanged cold RSS cap
v2-ci / unit-tests (push) Waiting to run
Owner scope 2026-11-12 (Windows x64 + Linux, unchanged cycle-3 caps, >=7200 s, SDK 1.5 comparator).

- Linux ARM64 (late-slope, 7200.12 s, 1964 cycles / 491 restarts, shutdown verified, analyzer-validated):
  first rejection RSS at cycle 247 / 900.73 s, 358,682,624 > 358,678,528 (+4 KiB); peak 382.98 MiB vs
  limit 342.06 MiB; late plateau medians 344.4/341.1/341.9 MiB, OLS -1.90 MiB/h. FAIL.
- Windows x64 on windev (same workload, working set as RSS, handle table as descriptors): first rejection RSS
  at cycle 285 / 1130.76 s, 303,505,408 > 302,174,208; peak 319.49 vs limit 288.18 MiB; plateau ~314-315 MiB,
  +0.93 MiB/h. FAIL. Also incomplete: the launcher's wall-clock outer deadline fired early after a +552 s
  w32time step and ended the run at test-phase 7014 s (rejection had already latched); launchers now use a
  Stopwatch.
- SDK 1.5.378.182 comparator (efb9197a publication, Windows, 7201 s): RSS peak +65.8 MiB (never over its cap),
  plateau ~+48 MiB, but the +32 handle cap tripped at cycle 17 (max 674 vs 642).
- Managed flat on both SDK2 runs; growth is the generated-factory JIT burst (+40k/+44k methods) - Linux JIT
  doublemapper ~180 MiB; Windows private bytes grew ~25 MiB of a ~132 MiB working-set rise.
- Retained: b61f3e05 cycle-1 fixture failure (#808 unplaced UNS folder), Windows/SDK1 smoke diagnostics.

No cap raised, no warm baseline, no failed run counted as passing.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 14:51:31 -05:00
Joseph Doherty 8ba0e22b27 docs(#780): owner drops the QA shadow week; QA goes straight to the P3 flip
v2-ci / unit-tests (push) Has been cancelled
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 12:40:19 -05:00
Joseph Doherty b61f3e0560 fix(galaxy): AVEVA priority 1 is the most urgent: Galaxy alarm Severity follows AVEVA's priority bands, live and history alike (#887)
v2-ci / unit-tests (push) Has been cancelled
GalaxyAlarmPriority, the one table live Galaxy alarms (MxAccessSeverityMapper)
and AVEVA A&E history (EventMapper, #886) map the AVEVA priority through, ranked
a higher number as more severe. AVEVA ranks priority 1 as the most urgent, so a
priority-1 alarm read 250 and a priority-999 system event 1000.

It now uses AVEVA's default priority ranges, most urgent first:

  AVEVA priority  AVEVA severity  bucket    OPC UA Severity
  1-250           1 Critical      Critical  1000
  251-500         2 High          High       750
  501-750         3 Medium        Medium     500
  751-999         4 Low           Low        250
  (history only)  0 not an alarm  -            1

Out-of-range priorities clamp (<= 0 Critical, > 999 Low). Evidence, vd03
Runtime.dbo.Events read-only: priority 1/100 alarms carry AVEVA severity 1,
400/500 severity 2, 600/700 severity 3, and every priority-999 System.* /
User.* row severity 0 with IsAlarm 0.

- Live and history now use the same number. GalaxyDriver passes the table's
  exact value as AlarmEventArgs.OpcUaSeverity. Before, it passed only the bucket,
  which NativeAlarmProjector re-mapped through its generic 200/500/700/900
  ladder, so live and history disagreed even before the direction fix.
- History: a row whose AVEVA severity is 0 (System.Start/Stop/Deploy,
  User.Write...) is not an alarm and reads Severity 1, below every alarm band.
  For alarms AVEVA's 1-4 severity is still never the value: the live feed
  carries only the priority, so it cannot use it, and history must agree.
- Docs: docs/Historian.md gains "Galaxy alarm priority -> Severity
  (lmxopcua#887)"; docs/AlarmTracking.md updated.
- Live re-run of the #801/#886 harness against vd03 HistorianGateway
  ea9f21c (read-only, network-only): 224 of 233 cases identical to the #886
  run, the 9 others differ only in Severity (priority-700 alarm rows 750 ->
  500, 19 system/user events 1000 -> 1). Record: aveva-history-801.md
  "#887 re-run", report aveva-gate-sdk2-887.json.gz.

Client-visible: every OPC UA client that ranks or filters Galaxy alarms by
Severity sees the order reversed (live and HistoryReadEvents). ScadaBridge has
no code that depends on the old values; its own mixed severity directions are
ScadaBridge#129.

Tests updated because they pinned the inverted table:
- Driver.Galaxy.Tests MxAccessSeverityMapperTests (rewritten: edges 1/250/251/
  500/501/750/751/999, 0/-100/int.MinValue/1000/int.MaxValue clamps,
  monotonic-decreasing sweep, equals GalaxyAlarmPriority for -2..1001)
- Driver.Galaxy.Tests GatewayGalaxyAlarmFeedTests (750 and 600 -> Medium)
- Driver.Historian.Gateway.Tests EventMapperTests (band table, priority 999
  -> 250, 1 -> 1000; system events -> 1)
- OpcUaServer.IntegrationTests GalaxyEquipmentEventHistoryWireTests (700 ->
  500, 100 -> 1000, System.Stop -> 1; the fake now stores AVEVA's real 0-4
  severity)

New tests:
- Runtime.Tests NativeAlarmRestatementEndToEndTests
  .Galaxy_live_condition_severity_is_the_AVEVA_priority_band (8 rows: the real
  feed -> real GalaxyDriver -> host -> projector; the same rows EventMapperTests
  pins on the history path)
- EventMapperTests.Aveva_severity_zero_marks_an_event_that_is_not_an_alarm
  (5 rows, incl. the priority-999 alarm positive control)
- GalaxyDriverAlarmSourceTests: OpcUaSeverity passes through

Mutation check, 8 mutants, all killed: the pre-#887 inverted table; the
driver dropping OpcUaSeverity; no not-an-alarm rule; each of the three band
edges off by one; history ignoring priority; a missing AVEVA severity read
as 0.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 12:14:49 -05:00
Joseph Doherty 3a312ffdd1 fix(history): a folder's event history reads the Galaxy objects behind its native conditions (#886)
v2-ci / unit-tests (push) Has been cancelled
A notifier folder read one event source, its NodeId identifier (a RawPath or an EquipmentId). AVEVA stores a
Galaxy object's A&E history under the object's tag_name (Source_Object), so AVEVA's own alarm and event
history reached no production folder. Its lower-case property keys were also missed, so every row read
Severity 1 with no ConditionName.

Sources. A folder's event history now reads its own source plus the Galaxy object of every native condition
that fires on it live: those parented under it and those WireAlarmNotifiers wired to it. Live and history use
the same membership. The object comes from the condition's Galaxy alarm reference, its raw tag's attributeRef,
up to the first '.':
- The composer reads it with the driver's own parser (GalaxyAttributeRef, shared with GalaxyDriver), only for
  a GalaxyMxGateway tag with an alarm (RawTagPlan.NativeAlarmReference, part of plan equality).
- The applier binds it after materialising the condition (IOpcUaAddressSpaceSink.BindNativeAlarmHistory,
  forwarded by the deferred sink). A rebuild or the condition's removal forgets it.
Each object is read with the #882 rule: its own events and <Object>.<Attribute> rows, never a prefix sibling.

Fan-out. historiangw's ReadEvents filter is a single predicate, so there is one request per source; the wire
contract is unchanged. EventSourceFanOut merges the reads as one source:
- order is (time, source position, arrival), reversed for a reverse read; ties are never re-sorted by EventId,
  so the boundary skip stays stable;
- duplicates by EventId are dropped and the merge is cut to the cap;
- a truncated source bounds the page at the time it stopped.
Paging, continuation points (bound to the whole source set), the walked-back reverse read and the tie-cluster
over-fetch are unchanged. A folder with one source is read exactly as before.

ACL. Every source is read inside the folder's own HistoryRead, after AuthorizeHistoryRead (#826). A denied
folder reads nothing and fails closed.

Mapping (EventMapper, property names now case-insensitive):
- Severity: an AVEVA row's priority goes through GalaxyAlarmPriority, the table live Galaxy alarms use
  (MxAccessSeverityMapper now delegates to it), so history matches live. Otherwise severity (OtOpcUa's own
  rows, which AVEVA stores lower-cased), clamped. AVEVA's own 1-4 severity is ignored. The table ranks a
  higher priority as more severe, which inverts AVEVA's convention: filed as #887.
- An AVEVA row whose source_conditionvariable names a native condition that fires on the folder takes that
  condition's SourceNode and live ConditionName. Otherwise ConditionName is alarmname or the condition
  variable's attribute.
- alarm_id/alarmid map to AlarmId.

Tests:
- GalaxyEquipmentEventHistoryWireTests (4, real SDK session, production Gateway backend, AVEVA-shaped fake
  gateway): equipment EqA backed by objects Oven_007 + OvenDoor_07. Covers object and attribute events from
  both objects, the prefix sibling excluded, cross-object and in-object ties, forward paging at 1-4 a page,
  reverse whole and paged, SourceNode/ConditionName/Severity, the raw device folder, ACL deny with no backend
  call plus a granted positive control, and a rebuild forgetting the bindings.
- EventSourceFanOutTests (8), EventMapper AVEVA cases (11), GalaxyAttributeRefTests (16), a composer test and
  an applier ordering test.
- Mutation-checked, 17 mutants, all killed. Covered: parent-only membership, no scope, a case-sensitive
  condition match, no condition rewrite, no truncation bound, EventId tie order, no sort, no cap, no stop at
  the bound, unreversed source order, no dedupe, no '.' boundary, priority ignored, case-sensitive keys, no
  applier bind, no deferred forward, and a composer that ignores the driver type. The forget-on-rematerialise
  guard survives as belt-and-braces: the rebuild clear covers it.
- Live, read-only, no WinRM, vd03 HistorianGateway ea9f21c: equipment folder EQ-886-ZipsenLeft (objects
  DevWinPlatform + Left_001). It read 0 events before and 92 now, equal to the unfiltered read's rows for both
  objects. 1/7-event pages and reverse whole/5-event pages agree, and 73 rows resolve to their live condition.
  diff_runs vs the #882 run: 221/226 identical. The 5 that differ are expected: DevWinPlatform Severity
  1 -> 750/1000, and #881 capabilities.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 11:45:48 -05:00
Joseph Doherty be28aa4432 fix(opcua-client): send UserName passwords in the legacy RSA format so 65-char API keys log in to SDK 1.5 servers (#880)
SDK 2.0.1's UserNameIdentityTokenHandler encrypts a password longer than 64 bytes
(a private RsaEncryptedSecretPasswordThreshold, no setting) as an RsaEncryptedSecret
with EncryptionAlgorithm=null. SDK 1.5.378 (.106 and .182) decrypts only the legacy
format under RSA policies; its SecurityPolicies.Decrypt answers a null algorithm with
BadIdentityTokenInvalid ("Unexpected encryption algorithm") before any key check.
Every production API key is 65 chars, so no SDK 2 OtOpcUa client could use one
against QA (b8b4c18f), a rollback target or an older third-party server.

LegacyRsaUserNameIdentityTokenHandler (Driver.OpcUaClient.Session, plugged in through
the public UserIdentity(IUserIdentityTokenHandler) ctor) always uses the legacy
format, RSA-OAEP(len|password|serverNonce) under the server certificate, under an RSA
policy. That is what SDK 2 sends for passwords of 64 bytes or less and what SDK 1.5
clients always send. The legacy format has no size limit (the plaintext is
length-framed over as many RSA blocks as it needs), and an SDK 2 server reads both
formats. Policy None, the ECC policies and a missing certificate go to the SDK
handler unchanged.

OpcUaUserIdentityFactory.UserName is now the one UserName identity the driver, the
AdminUI browse and Client.Shared (CLI/UI) present. Client.Shared references .Session
for it.

Tests: handler unit tests (all three RSA policies, the 1.5-required algorithm URI,
SDK 2 server-side decrypt, the session Copy() keeps the type, None unchanged, and an
SDK control pinning the 64/65-byte switch); a Client.Shared identity test; and an
SDK2<->SDK2 integration test where a 65-char key is admitted in both formats and a
wrong 65-char key reaches the key check (BadIdentityTokenRejected). The #800 Windows
gate's SDK 2 CLI 65-char case now gates against the 1.5 server too.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 11:12:27 -05:00
Joseph Doherty da948e5fb5 fix(history): a folder's event history includes its object's attribute alarms (#882)
v2-ci / unit-tests (push) Has been cancelled
HistorianGateway answers a per-source ReadEvents (Source_Object = DevWinPlatform) with the
object's own events (SourceName DevWinPlatform) AND its attribute alarms, which it reports as
<Object>.<Attribute> (Source_ProcessVariable, e.g. DevWinPlatform.CPULoadAvg). The Gateway
backend's defensive client-side source filter compared SourceName for equality, so it dropped
the attribute alarms: 70 of the 72 DevWinPlatform events on vd03.

The filter (and the past-the-cap "more events" signal) now uses EventSourceMatch: the source
itself, or <source>.<anything> on the '.' boundary, ordinal. DevWinPlatform never matches
DevWinPlatform2 or DevWinPlatform2.X. This needs no wire change and still makes one request.
Each event keeps its own SourceName. SourceNode stays the condition its stored alarm id names,
which is null for AVEVA rows. The attribute events ride the notifier's own read, so the
per-folder HistoryRead verdict (#826) governs them: a denied folder answers
BadUserAccessDenied before any backend call. The SqlServer and QuestDB backends still match
exactly; they hold only OtOpcUa's own events.

Tests:
- GatewayAttributeEventSourceTests (16): the boundary theory, attribute-in/sibling-out through
  the data source, the drain signal, and an exact attribute-named read.
- AttributeEventHistoryWireTests (2, over a real SDK session with the production Gateway
  backend on an AVEVA-shaped deterministic gateway): object + attribute events with their own
  SourceName, a prefix sibling excluded, paging, and the ACL-on deny with no backend call plus a
  granted positive control.
- Mutation-checked: exact match fails both wire tests; dropping the '.' boundary fails the
  wire sibling check and 6 unit tests.
- Live AVEVA harness re-run against vd03 HistorianGateway ea9f21c (read-only, no WinRM):
  folder DevWinPlatform 2 -> 72 events (whole, 72 1-event pages, reverse); the new #882 check
  passes; 221 of 225 shared cases are identical to the #801 SDK2 baseline, and the 4 that
  differ are exactly the DevWinPlatform event cases. Recorded in aveva-history-801.md.

Follow-up #886: production folders name RawPaths/EquipmentIds, not Galaxy objects, plus the AVEVA
severity/condition property mapping.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 10:34:29 -05:00
Joseph Doherty 008f0b36b2 docs(evidence): #801 real-AVEVA history gate — SDK 2.0.1 identical to 1.5.378.182
v2-ci / unit-tests (push) Waiting to run
Both builds served HistoryRead from the real AVEVA Historian via the DEV
HistorianGateway (vd03, ea9f21c): 225 cases / 3,855 rows / events, 0 differences.
Raw also matches direct SQL on Runtime.dbo.History (6/6) and histgw-cli; simple
aggregates match a recomputation from raw (336/336). Modified, HistoryUpdate,
seven refused aggregates and HistoryServerCapabilities answer as before. No write
reached the historian. Findings (pre-existing, not SDK): all HistoryServerCapabilities
booleans false; AVEVA attribute-alarm events unreachable per source.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 10:02:34 -05:00
Joseph Doherty c266971d51 docs(scripting): #789 owner accepts the measured Windows-VM budgets (idle p50 ≤ 500 µs, ≤ 50 pts batched at 10k/s)
v2-ci / unit-tests (push) Has been cancelled
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 09:54:22 -05:00
Joseph Doherty d8a000a409 perf(scripting): #789 profile the Windows misses; add the batched paced row; nothing kept in the runner
v2-ci / unit-tests (push) Has been cancelled
Profiled the remaining lmxopcua#789 misses on windev, a VMware VM, using per-thread CPU and context-switch accounting
plus ETW CPU sampling.
- One blocking hand-off costs ~26 us of latency and ~18 us of CPU there, about 10x macOS.
- An idle evaluation makes four hand-offs in a row: host writer -> worker reader -> evaluation thread -> host reader
  -> dispatcher.
- 48% of host CPU at 10k/s is in the kernel: the per-RunAsync NtWriteFile, thread-pool wakes, reads and waits.

Spin-before-block was measured at every hand-off and is not kept. It meets 150 us idle p50 (118 us) only by keeping
about two host cores and four worker cores busy for one stream. At 10k/s it reads +250 points.

Harness: new "paced 10,000/s, batched" row. It uses the same rate and keys, with each 1 ms tick submitted as one
RunBatch call, which is the production seams' shape. It runs after the original four phases, so their rows stay
comparable. On windev it halves host CPU, to +36..+45 points against +71..+84 unbatched. That is still above 25.

Design doc:
- 12.8: the record.
- 14 #15: the owner's decision on the Windows budgets. Recommended: measure on bare-metal Windows, then state the two
  budgets per platform, with the batched row as the gate. The alternative is a shared-memory transport.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 09:52:09 -05:00
Joseph Doherty 464b699e18 docs(scripting): #789 §12.7 — dropping the result-dispatcher hop, measured and not kept
Measured on windev (3 alternated sessions) and macOS: a pool resume in place
of the dispatcher, and an inline-on-reader upper bound, with the harness as
committed and with its paced continuation made synchronous.

- pool: 10k/s host CPU unchanged, idle p50 and idle CPU worse (macOS idle CPU
  x3 from pool spin); only the closed loop gains. Not kept.
- inline (unsafe for every seam; the ceiling for a VirtualTag-only opt-in):
  ~8 points of a core at 10k/s raw and ~50 us idle p50 on windev with a cheap
  continuation, nothing with the harness as committed. Both budgets still
  missed by a wide margin, so the opt-in was not built.

No code change. Records the remaining attribution (harness pool hop ~15
points raw; reader wake + syscalls; worker side untouched).

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 22:28:58 -05:00
Joseph Doherty c56fcc8af7 docs(scripting): #789 §12.6 — review follow-ups and their windev / macOS re-run
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 20:53:57 -05:00
Joseph Doherty d4a2aa6142 docs(scripting): #789 design §12.6 record — Windows blocking pipe I/O, windev before/after
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 20:10:58 -05:00
Joseph Doherty c06769991b docs(plans): #717 Warm redundancy design spec + implementation plan (draft for owner approval)
Spec: per-shape contract (Unset/None/Warm), RedundancySupport + ServerUriArray graph,
ServiceLevel bands, per-service standby behaviour, alarm suppression + ConditionRefresh
(D5), strict first-delivery (D4), transition sequencing, ordered snapshot revisions,
ScadaBridge impact, SDK 2.0.1 re-proof list, open questions. Plan: T0-T13 with wire
tests, docker-dev live gate, rollout/rollback. Docs only.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 18:19:14 -05:00
Joseph Doherty 5ac5520532 docs: #873 SDK 2.0.1 API-key logins need client clock within 5 min ahead of the server (rollout runbook + security guide)
v2-ci / unit-tests (push) Waiting to run
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 18:08:39 -05:00
Joseph Doherty ca0e73fbca fix(opcua): #855 perf review 1 — groups evaluate their own filter clone; lazy fields; tighter equivalence tests
- A filter group owns a deep clone of its first filter, validated once and
  evaluated under the group's lock: no item-owned filter is mutated or raced
  (Validate mutates a select clause).
- The projection is computed in the SDK's order: where clause first, fields
  only for an item that will queue.
- Unambiguous locale key; counter for filters given a private group past the
  registry cap; DeliveryCache getter returns its own allocation.
- UseProjectedEventDelivery=false also asks the role and grant checks per
  item, so the equivalence test compares against the pre-optimisation path.
- Equivalence test: same select list under three where clauses, a filter
  modified mid-run, a session with other locales, an ungranted anonymous
  Server-object item, per-rig counter; plus a small overflowing queue with
  DiscardOldest true and false.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:59:22 -05:00
Joseph Doherty 98d962b24b perf(opcua): #855 — shared per-report projection for condition events under Lock
Condition events delivered to many event items (folder and Server-object)
no longer pay the SDK's per-item QueueEvent(IFilterTarget) work under the
node-manager Lock: the role-permission verdict is shared per (session,
identity), the #826 grant check per identity per hop, and the where clause +
select-clause projection per (value-equal EventFilter, locales); each item
gets its own EventFieldList over the shared read-only fields. The SDK's
QueueEvent body is reproduced under the item's own lock (three private SDK
members, guard-tested); anything outside that shape falls back to the SDK.
The per-report duplicate check replaces the SDK's O(queue) scan.

ProjectedEventDeliveryWireTests compares both paths over the wire.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:42:29 -05:00
Joseph Doherty c9e5c8dcf7 Merge #855 folder event items: per-item delivery under the node manager, ConditionRefresh cut matches the Server object
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:07:21 -05:00
Joseph Doherty 8f30c1e521 docs(script-runner): windev re-measure (#789) and Windows soft-clock test run (#842)
§12.1 gains the 2026-11-11 windev (VMware VM) perf table: idle p50 581 us
(MISS), kill latency after T+G 8-16 ms (PASS), kill both shards 2.12-2.14 s
(MISS by ~6 %), worker RSS 160 MB compiled / 36 MB warm-cache (PASS under the
revised 200 MB budget), host CPU +131 % vs InProcess (MISS, untuned).

It also records the first Windows run of Category=ScriptWorker: 107/0/2 from
C:\build, all 13 WorkerSoftClockTests cases (5 freezing a real worker with
NtSuspendProcess) green with no code change. From a user-profile directory
the restricted-identity isolation test fails because the restricted token
cannot read the worker dll there (profile ACL), which is environment, not a
defect.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:34:07 -05:00
Joseph Doherty d9b23e0b95 feat(opcua): per-item delivery for folder event items; folder ConditionRefresh gets the #854 cut (#855)
Condition events reaching one of our notifiers no longer go through the SDK
MonitoredNode2 channel: the walk serves the attached items itself, per item and
per event (audit guard, uncached INodeManager3 verdict, item gate/cut,
QueueEvent), as the Server-object path does. No notifier of ours defers any
more; the #838 deferral queue and the #854 cut stay as the safety net and
folder items share the cut (RefreshDirectItems, flag on and off). An event
whose walk suspends is marked deferred before it suspends.

Tests produce deferral with a notifier-stall probe instead of holding a folder
item's SDK lock (which would now block the reporter). New
FolderConditionRefreshWireTests; design note in docs/plans.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:18:11 -05:00
Joseph Doherty 93d1809484 docs(script-runner): §8.2 worker idle RSS budget revised to 200 MB + scripts (#789)
The owner revised the per-shard worker idle RSS budget from 80 MB to 200 MB
+ scripts on 2026-11-11: compiles run in the worker, so Roslyn's code and
metadata stay resident (measured 159-187 MB) and no GC setting moves it.
The perf harness now reports both RSS rows against the 200 MB constant with
a PASS/MISS mark; A-3 records what the measurement showed.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:10:59 -05:00
Joseph Doherty ae91e6b7f7 feat(opcuaclient): opt-in server-certificate host-name check; NAT fixture keeps its PKI (#816, #818)
#816 finding: the SDK checks that the server certificate names the host only
when a session is created with checkDomain:true. The driver, the AdminUI
browser and Client.Shared all called the 8-argument
DefaultSessionFactory.CreateAsync, which forwards checkDomain:false — so no
host-name check ever ran (why a trusted NAT cert for *.invalid browsed fine
over an IP in #813). When enabled the SDK (Session.OpenAsync ->
ValidateDomains -> FindDomain) compares ConfiguredEndpoint.EndpointUrl.IdnHost
— the CONFIGURED host, because OpcUaEndpointSelection/EndpointSelector
rewrote the advertised one — against the cert's DNS/IP SANs; secured
sessions only, and the auto-accept handler accepts a mismatch.

Policy: certificateValidation.validateServerHostName, tri-state. Unset =
OpcUaCertificateValidationOptions.DefaultValidateServerHostName = false, the
pre-#816 behaviour, so nothing changes on upgrade; true = validate the
configured host; false = explicit NAT/port-publish opt-out. Threaded into the
driver, the AdminUI browser (same JSON) and the AdminUI form (a select that
keeps "unset" unset), Client.Shared ConnectionSettings.ValidateServerHostName
and the CLI --validate-host-name. A driver refusal names the opt-out.

#818: the NAT fixture's /app/pki is on the opc-plc-nat-pki volume (verified:
same thumbprint across --force-recreate), its PUBLIC cert is committed as
Docker/nat/opc-plc-nat-server.der, and NatPublishedEndpointBrowseTests run
every secure browse on a throwaway PKI (internal browser seam): refusal with
nothing trusted, positive with the cert trusted (unset/false host check), and
a BadCertificateHostNameInvalid refusal with the check on. The trusted cert is
OPCUA_NAT_SERVER_CERT or the committed one (skip on thumbprint mismatch);
OPCUA_NAT_SERVER_CERT_TRUSTED is retired. CI exports its throwaway NAT
container's public cert. ServerHostNameValidationTests pin the driver against
the plain opc-plc. MIN_EXECUTED not raised (needs a hosted run): +6 in
v2-integration (440 -> 446), documented in scripts/ci/README.md.
2026-11-11 01:33:00 -05:00
Joseph Doherty 88c05cc01a docs(opcuaclient): document the shared endpoint-URL rewrite and how to run the NAT tests (#814)
docs/drivers/OpcUaClient.md, docs/plans/2026-05-28-driver-browsers-design.md and
docs/Client.CLI.md: one OpcUaEndpointUrlRewriter for driver, AdminUI browser and
Client CLI/UI; configured host+port win, advertised path kept for opc.tcp; policy/mode
selection and certificate validation unchanged; NAT test recipe against the #813 fixture.
2026-11-10 22:53:01 -05:00
Joseph Doherty da4e27003e docs: record completed Linux OPC UA lifecycle diagnostic
v2-ci / unit-tests (pull_request) Waiting to run
2026-11-10 20:28:11 -05:00
Joseph Doherty 1e4b017f6e docs: retain verified Linux rehearsal and diagnostic gate
v2-ci / unit-tests (pull_request) Failing after 1h6m11s
2026-11-10 18:25:37 -05:00
Joseph Doherty b18f97236a docs: record reviewed private temp mount controls
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 18:18:59 -05:00
Joseph Doherty 351edb0a86 docs: retain read-only Linux rehearsal startup failure 2026-11-10 18:13:00 -05:00
Joseph Doherty 626cb4f7f1 docs: record final late-slope controls before artifact preparation
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 18:03:13 -05:00
Joseph Doherty 7023285a3e docs: retain late-slope guard control receipts 2026-11-10 17:54:43 -05:00
Joseph Doherty fbf73436be docs: record reviewed native import reproduction
v2-ci / unit-tests (pull_request) Waiting to run
2026-11-10 17:42:40 -05:00
Joseph Doherty 041189aa6d docs: close SDK2 ownership evidence review
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 17:35:54 -05:00
Joseph Doherty 327bbdaab9 docs: record reviewed SDK2 ownership regressions and diagnostic design 2026-11-10 17:33:06 -05:00
Joseph Doherty ce21f5db09 Merge master SDK1 ownership repair with SDK2 disposal adaptation 2026-11-10 17:17:00 -05:00
Joseph Doherty efb9197a0f docs: record reviewed SDK1 terminal disposal repair
v2-ci / unit-tests (push) Has been cancelled
2026-11-10 17:05:06 -05:00
Joseph Doherty 2cf7c1d248 docs: locate close control build receipt
v2-ci / unit-tests (pull_request) Waiting to run
2026-11-10 16:40:28 -05:00