Commit Graph

569 Commits

Author SHA1 Message Date
Joseph Doherty da948e5fb5 fix(history): a folder's event history includes its object's attribute alarms (#882)
v2-ci / unit-tests (push) Has been cancelled
HistorianGateway answers a per-source ReadEvents (Source_Object = DevWinPlatform) with the
object's own events (SourceName DevWinPlatform) AND its attribute alarms, which it reports as
<Object>.<Attribute> (Source_ProcessVariable, e.g. DevWinPlatform.CPULoadAvg). The Gateway
backend's defensive client-side source filter compared SourceName for equality, so it dropped
the attribute alarms: 70 of the 72 DevWinPlatform events on vd03.

The filter (and the past-the-cap "more events" signal) now uses EventSourceMatch: the source
itself, or <source>.<anything> on the '.' boundary, ordinal. DevWinPlatform never matches
DevWinPlatform2 or DevWinPlatform2.X. This needs no wire change and still makes one request.
Each event keeps its own SourceName. SourceNode stays the condition its stored alarm id names,
which is null for AVEVA rows. The attribute events ride the notifier's own read, so the
per-folder HistoryRead verdict (#826) governs them: a denied folder answers
BadUserAccessDenied before any backend call. The SqlServer and QuestDB backends still match
exactly; they hold only OtOpcUa's own events.

Tests:
- GatewayAttributeEventSourceTests (16): the boundary theory, attribute-in/sibling-out through
  the data source, the drain signal, and an exact attribute-named read.
- AttributeEventHistoryWireTests (2, over a real SDK session with the production Gateway
  backend on an AVEVA-shaped deterministic gateway): object + attribute events with their own
  SourceName, a prefix sibling excluded, paging, and the ACL-on deny with no backend call plus a
  granted positive control.
- Mutation-checked: exact match fails both wire tests; dropping the '.' boundary fails the
  wire sibling check and 6 unit tests.
- Live AVEVA harness re-run against vd03 HistorianGateway ea9f21c (read-only, no WinRM):
  folder DevWinPlatform 2 -> 72 events (whole, 72 1-event pages, reverse); the new #882 check
  passes; 221 of 225 shared cases are identical to the #801 SDK2 baseline, and the 4 that
  differ are exactly the DevWinPlatform event cases. Recorded in aveva-history-801.md.

Follow-up #886: production folders name RawPaths/EquipmentIds, not Galaxy objects, plus the AVEVA
severity/condition property mapping.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 10:34:29 -05:00
Joseph Doherty 008f0b36b2 docs(evidence): #801 real-AVEVA history gate — SDK 2.0.1 identical to 1.5.378.182
v2-ci / unit-tests (push) Waiting to run
Both builds served HistoryRead from the real AVEVA Historian via the DEV
HistorianGateway (vd03, ea9f21c): 225 cases / 3,855 rows / events, 0 differences.
Raw also matches direct SQL on Runtime.dbo.History (6/6) and histgw-cli; simple
aggregates match a recomputation from raw (336/336). Modified, HistoryUpdate,
seven refused aggregates and HistoryServerCapabilities answer as before. No write
reached the historian. Findings (pre-existing, not SDK): all HistoryServerCapabilities
booleans false; AVEVA attribute-alarm events unreachable per source.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 10:02:34 -05:00
Joseph Doherty c266971d51 docs(scripting): #789 owner accepts the measured Windows-VM budgets (idle p50 ≤ 500 µs, ≤ 50 pts batched at 10k/s)
v2-ci / unit-tests (push) Has been cancelled
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 09:54:22 -05:00
Joseph Doherty d8a000a409 perf(scripting): #789 profile the Windows misses; add the batched paced row; nothing kept in the runner
v2-ci / unit-tests (push) Has been cancelled
Profiled the remaining lmxopcua#789 misses on windev, a VMware VM, using per-thread CPU and context-switch accounting
plus ETW CPU sampling.
- One blocking hand-off costs ~26 us of latency and ~18 us of CPU there, about 10x macOS.
- An idle evaluation makes four hand-offs in a row: host writer -> worker reader -> evaluation thread -> host reader
  -> dispatcher.
- 48% of host CPU at 10k/s is in the kernel: the per-RunAsync NtWriteFile, thread-pool wakes, reads and waits.

Spin-before-block was measured at every hand-off and is not kept. It meets 150 us idle p50 (118 us) only by keeping
about two host cores and four worker cores busy for one stream. At 10k/s it reads +250 points.

Harness: new "paced 10,000/s, batched" row. It uses the same rate and keys, with each 1 ms tick submitted as one
RunBatch call, which is the production seams' shape. It runs after the original four phases, so their rows stay
comparable. On windev it halves host CPU, to +36..+45 points against +71..+84 unbatched. That is still above 25.

Design doc:
- 12.8: the record.
- 14 #15: the owner's decision on the Windows budgets. Recommended: measure on bare-metal Windows, then state the two
  budgets per platform, with the batched row as the gate. The alternative is a shared-memory transport.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 09:52:09 -05:00
Joseph Doherty 464b699e18 docs(scripting): #789 §12.7 — dropping the result-dispatcher hop, measured and not kept
Measured on windev (3 alternated sessions) and macOS: a pool resume in place
of the dispatcher, and an inline-on-reader upper bound, with the harness as
committed and with its paced continuation made synchronous.

- pool: 10k/s host CPU unchanged, idle p50 and idle CPU worse (macOS idle CPU
  x3 from pool spin); only the closed loop gains. Not kept.
- inline (unsafe for every seam; the ceiling for a VirtualTag-only opt-in):
  ~8 points of a core at 10k/s raw and ~50 us idle p50 on windev with a cheap
  continuation, nothing with the harness as committed. Both budgets still
  missed by a wide margin, so the opt-in was not built.

No code change. Records the remaining attribution (harness pool hop ~15
points raw; reader wake + syscalls; worker side untouched).

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 22:28:58 -05:00
Joseph Doherty c56fcc8af7 docs(scripting): #789 §12.6 — review follow-ups and their windev / macOS re-run
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 20:53:57 -05:00
Joseph Doherty d4a2aa6142 docs(scripting): #789 design §12.6 record — Windows blocking pipe I/O, windev before/after
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 20:10:58 -05:00
Joseph Doherty c06769991b docs(plans): #717 Warm redundancy design spec + implementation plan (draft for owner approval)
Spec: per-shape contract (Unset/None/Warm), RedundancySupport + ServerUriArray graph,
ServiceLevel bands, per-service standby behaviour, alarm suppression + ConditionRefresh
(D5), strict first-delivery (D4), transition sequencing, ordered snapshot revisions,
ScadaBridge impact, SDK 2.0.1 re-proof list, open questions. Plan: T0-T13 with wire
tests, docker-dev live gate, rollout/rollback. Docs only.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 18:19:14 -05:00
Joseph Doherty 5ac5520532 docs: #873 SDK 2.0.1 API-key logins need client clock within 5 min ahead of the server (rollout runbook + security guide)
v2-ci / unit-tests (push) Waiting to run
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 18:08:39 -05:00
Joseph Doherty ca0e73fbca fix(opcua): #855 perf review 1 — groups evaluate their own filter clone; lazy fields; tighter equivalence tests
- A filter group owns a deep clone of its first filter, validated once and
  evaluated under the group's lock: no item-owned filter is mutated or raced
  (Validate mutates a select clause).
- The projection is computed in the SDK's order: where clause first, fields
  only for an item that will queue.
- Unambiguous locale key; counter for filters given a private group past the
  registry cap; DeliveryCache getter returns its own allocation.
- UseProjectedEventDelivery=false also asks the role and grant checks per
  item, so the equivalence test compares against the pre-optimisation path.
- Equivalence test: same select list under three where clauses, a filter
  modified mid-run, a session with other locales, an ungranted anonymous
  Server-object item, per-rig counter; plus a small overflowing queue with
  DiscardOldest true and false.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:59:22 -05:00
Joseph Doherty 98d962b24b perf(opcua): #855 — shared per-report projection for condition events under Lock
Condition events delivered to many event items (folder and Server-object)
no longer pay the SDK's per-item QueueEvent(IFilterTarget) work under the
node-manager Lock: the role-permission verdict is shared per (session,
identity), the #826 grant check per identity per hop, and the where clause +
select-clause projection per (value-equal EventFilter, locales); each item
gets its own EventFieldList over the shared read-only fields. The SDK's
QueueEvent body is reproduced under the item's own lock (three private SDK
members, guard-tested); anything outside that shape falls back to the SDK.
The per-report duplicate check replaces the SDK's O(queue) scan.

ProjectedEventDeliveryWireTests compares both paths over the wire.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:42:29 -05:00
Joseph Doherty c9e5c8dcf7 Merge #855 folder event items: per-item delivery under the node manager, ConditionRefresh cut matches the Server object
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:07:21 -05:00
Joseph Doherty 8f30c1e521 docs(script-runner): windev re-measure (#789) and Windows soft-clock test run (#842)
§12.1 gains the 2026-11-11 windev (VMware VM) perf table: idle p50 581 us
(MISS), kill latency after T+G 8-16 ms (PASS), kill both shards 2.12-2.14 s
(MISS by ~6 %), worker RSS 160 MB compiled / 36 MB warm-cache (PASS under the
revised 200 MB budget), host CPU +131 % vs InProcess (MISS, untuned).

It also records the first Windows run of Category=ScriptWorker: 107/0/2 from
C:\build, all 13 WorkerSoftClockTests cases (5 freezing a real worker with
NtSuspendProcess) green with no code change. From a user-profile directory
the restricted-identity isolation test fails because the restricted token
cannot read the worker dll there (profile ACL), which is environment, not a
defect.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:34:07 -05:00
Joseph Doherty d9b23e0b95 feat(opcua): per-item delivery for folder event items; folder ConditionRefresh gets the #854 cut (#855)
Condition events reaching one of our notifiers no longer go through the SDK
MonitoredNode2 channel: the walk serves the attached items itself, per item and
per event (audit guard, uncached INodeManager3 verdict, item gate/cut,
QueueEvent), as the Server-object path does. No notifier of ours defers any
more; the #838 deferral queue and the #854 cut stay as the safety net and
folder items share the cut (RefreshDirectItems, flag on and off). An event
whose walk suspends is marked deferred before it suspends.

Tests produce deferral with a notifier-stall probe instead of holding a folder
item's SDK lock (which would now block the reporter). New
FolderConditionRefreshWireTests; design note in docs/plans.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:18:11 -05:00
Joseph Doherty 93d1809484 docs(script-runner): §8.2 worker idle RSS budget revised to 200 MB + scripts (#789)
The owner revised the per-shard worker idle RSS budget from 80 MB to 200 MB
+ scripts on 2026-11-11: compiles run in the worker, so Roslyn's code and
metadata stay resident (measured 159-187 MB) and no GC setting moves it.
The perf harness now reports both RSS rows against the 200 MB constant with
a PASS/MISS mark; A-3 records what the measurement showed.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:10:59 -05:00
Joseph Doherty ae91e6b7f7 feat(opcuaclient): opt-in server-certificate host-name check; NAT fixture keeps its PKI (#816, #818)
#816 finding: the SDK checks that the server certificate names the host only
when a session is created with checkDomain:true. The driver, the AdminUI
browser and Client.Shared all called the 8-argument
DefaultSessionFactory.CreateAsync, which forwards checkDomain:false — so no
host-name check ever ran (why a trusted NAT cert for *.invalid browsed fine
over an IP in #813). When enabled the SDK (Session.OpenAsync ->
ValidateDomains -> FindDomain) compares ConfiguredEndpoint.EndpointUrl.IdnHost
— the CONFIGURED host, because OpcUaEndpointSelection/EndpointSelector
rewrote the advertised one — against the cert's DNS/IP SANs; secured
sessions only, and the auto-accept handler accepts a mismatch.

Policy: certificateValidation.validateServerHostName, tri-state. Unset =
OpcUaCertificateValidationOptions.DefaultValidateServerHostName = false, the
pre-#816 behaviour, so nothing changes on upgrade; true = validate the
configured host; false = explicit NAT/port-publish opt-out. Threaded into the
driver, the AdminUI browser (same JSON) and the AdminUI form (a select that
keeps "unset" unset), Client.Shared ConnectionSettings.ValidateServerHostName
and the CLI --validate-host-name. A driver refusal names the opt-out.

#818: the NAT fixture's /app/pki is on the opc-plc-nat-pki volume (verified:
same thumbprint across --force-recreate), its PUBLIC cert is committed as
Docker/nat/opc-plc-nat-server.der, and NatPublishedEndpointBrowseTests run
every secure browse on a throwaway PKI (internal browser seam): refusal with
nothing trusted, positive with the cert trusted (unset/false host check), and
a BadCertificateHostNameInvalid refusal with the check on. The trusted cert is
OPCUA_NAT_SERVER_CERT or the committed one (skip on thumbprint mismatch);
OPCUA_NAT_SERVER_CERT_TRUSTED is retired. CI exports its throwaway NAT
container's public cert. ServerHostNameValidationTests pin the driver against
the plain opc-plc. MIN_EXECUTED not raised (needs a hosted run): +6 in
v2-integration (440 -> 446), documented in scripts/ci/README.md.
2026-11-11 01:33:00 -05:00
Joseph Doherty 88c05cc01a docs(opcuaclient): document the shared endpoint-URL rewrite and how to run the NAT tests (#814)
docs/drivers/OpcUaClient.md, docs/plans/2026-05-28-driver-browsers-design.md and
docs/Client.CLI.md: one OpcUaEndpointUrlRewriter for driver, AdminUI browser and
Client CLI/UI; configured host+port win, advertised path kept for opc.tcp; policy/mode
selection and certificate validation unchanged; NAT test recipe against the #813 fixture.
2026-11-10 22:53:01 -05:00
Joseph Doherty da4e27003e docs: record completed Linux OPC UA lifecycle diagnostic
v2-ci / unit-tests (pull_request) Waiting to run
2026-11-10 20:28:11 -05:00
Joseph Doherty 1e4b017f6e docs: retain verified Linux rehearsal and diagnostic gate
v2-ci / unit-tests (pull_request) Failing after 1h6m11s
2026-11-10 18:25:37 -05:00
Joseph Doherty b18f97236a docs: record reviewed private temp mount controls
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 18:18:59 -05:00
Joseph Doherty 351edb0a86 docs: retain read-only Linux rehearsal startup failure 2026-11-10 18:13:00 -05:00
Joseph Doherty 626cb4f7f1 docs: record final late-slope controls before artifact preparation
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 18:03:13 -05:00
Joseph Doherty 7023285a3e docs: retain late-slope guard control receipts 2026-11-10 17:54:43 -05:00
Joseph Doherty fbf73436be docs: record reviewed native import reproduction
v2-ci / unit-tests (pull_request) Waiting to run
2026-11-10 17:42:40 -05:00
Joseph Doherty 041189aa6d docs: close SDK2 ownership evidence review
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 17:35:54 -05:00
Joseph Doherty 327bbdaab9 docs: record reviewed SDK2 ownership regressions and diagnostic design 2026-11-10 17:33:06 -05:00
Joseph Doherty ce21f5db09 Merge master SDK1 ownership repair with SDK2 disposal adaptation 2026-11-10 17:17:00 -05:00
Joseph Doherty efb9197a0f docs: record reviewed SDK1 terminal disposal repair
v2-ci / unit-tests (push) Has been cancelled
2026-11-10 17:05:06 -05:00
Joseph Doherty 2cf7c1d248 docs: locate close control build receipt
v2-ci / unit-tests (pull_request) Waiting to run
2026-11-10 16:40:28 -05:00
Joseph Doherty 0c0209f9a6 docs: record reviewed close ownership and native attribution 2026-11-10 16:39:34 -05:00
Joseph Doherty ed60970fc2 docs: record final review hold and baseline ownership limits
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 16:09:09 -05:00
Joseph Doherty 331cde68a7 test: record final integrated CI baseline-only failures
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 16:05:13 -05:00
Joseph Doherty 4a04806d22 test: preserve saturation outcome and classify final CI corrections
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 15:46:58 -05:00
Joseph Doherty ab35ddd3c2 test: retain completed two-hour client lifecycle evidence
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 14:54:38 -05:00
Joseph Doherty c6c05a1001 test: record publish smoke with HistorianGateway 1.0 packages 2026-11-10 14:52:08 -05:00
Joseph Doherty 67bc61236a Merge remote-tracking branch 'origin/master' into codex/opcua-sdk-2 2026-11-10 14:40:51 -05:00
Joseph Doherty 230cef4ef0 build: adopt HistorianGateway 1.0 TLS options
v2-ci / unit-tests (push) Has been cancelled
2026-11-10 14:40:30 -05:00
Joseph Doherty 12dffb9254 docs: retain reviewed memory investigation and prospective gate limits
v2-ci / unit-tests (pull_request) Waiting to run
2026-11-10 14:21:57 -05:00
Joseph Doherty a7efb39685 docs: record Astra source verdict and preserve failed lifecycle scope
v2-ci / unit-tests (pull_request) Has been cancelled
2026-11-10 13:33:01 -05:00
Joseph Doherty b218321bda docs: retain final CI failures and Linux RSS gate evidence
v2-ci / unit-tests (pull_request) Waiting to run
2026-11-10 13:28:40 -05:00
Joseph Doherty a8ae19b224 docs: reconcile SDK 2 rollout gates with local evidence 2026-11-10 13:28:16 -05:00
Joseph Doherty aa9af0dcff docs: retain backend benchmark and actual publish evidence 2026-11-10 13:17:13 -05:00
Joseph Doherty 780f363f16 docs: document OPC UA 2.0 compatibility and rollout 2026-11-10 11:48:19 -05:00
Joseph Doherty d2076e75c6 build: pin OPC UA SDK 2.0.1 and migration tooling 2026-11-10 09:51:58 -05:00
Joseph Doherty d9984028b7 build: raise Extensions baseline for OPC UA SDK 2.0 2026-11-10 09:51:15 -05:00
Joseph Doherty 56250a88a4 docs: record OPC UA 2.0 migration baseline and decisions 2026-11-10 09:50:10 -05:00
Joseph Doherty ea4a3c9289 docs: add reviewed OPC UA SDK 2.0 upgrade plan
v2-ci / unit-tests (push) Waiting to run
2026-11-10 09:26:19 -05:00
Joseph Doherty 2a8e2981fe feat(scripting): ScriptWorker isolation on Windows: restricted worker identity, pipe ACL, firewall + PeCacheDir installer (lmxopcua#780)
v2-ci / unit-tests (push) Has been cancelled
Owner decisions §14 #5/#6 as built (design §12.5):

- Scripting:Worker:Identity (default unset, Windows only, fail-closed validation): each worker
  starts under a restricted token derived from the host's. Every privilege but SeChangeNotify is
  removed; the restricting SIDs are NT SERVICE\OtOpcUaScriptWorker's SID + Everyone + Users +
  RESTRICTED; the token and default DACLs are host + SYSTEM + identity + RESTRICTED. The worker
  is created suspended, put in its Job Object before it runs, inherits only its stdio pipes, gets
  an ACL'd work dir as TEMP, and runs with DOTNET_EnableDiagnostics=0. The identity is granted on
  the host's window station and desktop, because USER32 init made coreclr.dll fail with
  ERROR_DLL_INIT_FAILED on windev. The logon SID is not used because the host's process DACL
  gives it PROCESS_VM_READ.
- The worker pipe on Windows always has an explicit protected DACL: host account + identity
  only. There is no owner ACE, so an elevated host's Administrators gets no access.
- The worker treats access denied on its parent-pid poll as alive; the Job Object covers that case.
- scripts/install/Install-ScriptWorkerIsolation.ps1 + ScriptWorkerIsolation.psm1 are idempotent
  and support -WhatIf, -Uninstall and -PlanOnly. They create the disabled anchor service, the
  PeCacheDir ACL (protected, no Users, key and markers host-only, pe\ Modify for the identity),
  read+execute on script-worker\, a ReadData deny on the install root, the outbound firewall
  block on the worker exe, and optionally the host-service env.
- Tests: the dry-run harness scripts/install/tests/Test-ScriptWorkerIsolation.ps1 (46
  assertions, mutation-checked); ScriptWorkerIdentityTests; WorkerIsolationTests, which cover
  the pipe ACL, token access and a real restricted worker running a script, all green on windev.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-10 08:46:23 -05:00
Joseph Doherty 0e96dd08a2 fix(scripting): deny System.AppContext and other process-wide mutators in the script sandbox (lmxopcua#792)
v2-ci / unit-tests (push) Waiting to run
ForbiddenTypeAnalyzer now denies System.AppContext, System.Console, GCSettings, ProfileOptimization,
EncodingProvider/CodePagesEncodingProvider, TypeDescriptor and CryptoConfig as types, and the ambient
CultureInfo.{DefaultThreadCurrentCulture,DefaultThreadCurrentUICulture,CurrentCulture,CurrentUICulture}
and TimeZoneInfo.ClearCachedData members (member deny-list now covers properties and member-access reads).
Shadow-mode tests diverge via culture-sensitive string.Compare (worker is InvariantGlobalization) instead of
AppContext.BaseDirectory, with a premise test. Mutation-proved: 21 new cases fail with the analyzer reverted.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-10 07:48:39 -05:00
Joseph Doherty b1eedceacf docs(scripting): §12.4 — Shadow mode as built, its tests, and the docker-dev drill with alarms (lmxopcua#780 P2)
v2-ci / unit-tests (push) Has been cancelled
Shadow on central-1/2 with all three seams: zero mismatches on ordinary VTs / Calc / alarms, counted mismatches on
exactly the divergent (value) and decimal (unsupported_type) scripts, kill -9 of the shadow workers left every
published value Good. OutOfProcess with ScriptedAlarm listed: alarms raise and clear, a runaway predicate is killed
twice and quarantined (BadInternalError) with its siblings Active and Good, kill -9 under an in-flight predicate
freezes it Active + BadResourceUnavailable and it re-evaluates in isolation. Answers A-7 (Modbus UInt16 boxes as
ushort) and A-8 (a decimal VT publishes BadNotSupported in process). Rig restored to revision e221b179.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-10 07:36:11 -05:00