Commit Graph

4953 Commits

Author SHA1 Message Date
Joseph Doherty 88e53f7f84 fix(install): Install-ScriptWorkerIsolation.ps1 works on PowerShell 7 and 5.1, -Uninstall included (#883)
First real run of the #780 installer (windev, #800 gate) failed on both hosts and its rollback failed too:
- PowerShell 7: DirectoryInfo.GetAccessControl/SetAccessControl do not exist on .NET. Use Get-Acl / Set-Acl.
- Windows PowerShell 5.1: an if-expression unrolled the wanted ACE to a scalar ($null on -Remove), so .Count threw under
  StrictMode. Wrap it in @(...).
- Found on the re-run: an apply from 5.1 was not idempotent. 5.1 passes the literal quotes in sc.exe binPath= with legacy
  quoting, so the anchor path was stored unquoted and AnchorService reported 'changed' on every run. sc create/config now
  runs through Invoke-OtOpcUaSc with an explicit command line, identical on both hosts.

Adds scripts/install/tests/Test-ScriptWorkerIsolation.Windows.ps1: a Windows-executed dry run -> apply -> re-apply ->
uninstall -> uninstall test (scratch names, stand-in host service) for either host. The plan-only harness cannot see these.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 11:24:01 -05:00
Joseph Doherty e5b26f3772 test(validation): #880 cross-version matrix evidence, fixed and mutated candidate
v2-ci / unit-tests (push) Has been cancelled
The baseline is 83ed880e (SDK 1.5.378.182). The candidate is the #880 change (on da948e5f), run twice.
Fixed: every SDK 2 client (Shared, OpcUaClient driver, CLI) logs in to the SDK 1.5
server with a 65-char key. A wrong 65-char key gets BadIdentityTokenRejected. In the
control, the SDK default handler is refused by 1.5 with BadIdentityTokenInvalid and
admitted by SDK 2.
Mutation (the factory reverted to the SDK default handler): the SDK 2 Shared connect,
the driver connect and all three CLI calls fail against 1.5 with
BadIdentityTokenInvalid, and the wrong-key case gets Invalid instead of Rejected. This
is #880 reproduced.
In both runs the UNS cases fail against the master server for every client SDK. That
is a fixture gap since #808 and is tracked as #888.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 11:14:15 -05:00
Joseph Doherty 2ad7213046 test(validation): SDK compatibility matrix uses the production 65-char API key, adds #880 format cases
The cross-version matrix (#782) used a 55-char 'otua_matrix_' key, which stayed under
SDK 2's 64-byte RsaEncryptedSecret threshold and hid #880. It now uses the
ApiKeySecret.Generate shape (65 chars, random per run, never recorded). Every Shared
leg adds production-length-API-key-wrong-secret-reaches-key-check, which must get
BadIdentityTokenRejected rather than a format failure. SDK 2 clients also add
sdk2-default-RsaEncryptedSecret-control, the negative control: the SDK's default
handler must be refused with BadIdentityTokenInvalid by a 1.x server and admitted
by an SDK 2 server. The runner passes the server SDK version from ready.json.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 11:12:27 -05:00
Joseph Doherty be28aa4432 fix(opcua-client): send UserName passwords in the legacy RSA format so 65-char API keys log in to SDK 1.5 servers (#880)
SDK 2.0.1's UserNameIdentityTokenHandler encrypts a password longer than 64 bytes
(a private RsaEncryptedSecretPasswordThreshold, no setting) as an RsaEncryptedSecret
with EncryptionAlgorithm=null. SDK 1.5.378 (.106 and .182) decrypts only the legacy
format under RSA policies; its SecurityPolicies.Decrypt answers a null algorithm with
BadIdentityTokenInvalid ("Unexpected encryption algorithm") before any key check.
Every production API key is 65 chars, so no SDK 2 OtOpcUa client could use one
against QA (b8b4c18f), a rollback target or an older third-party server.

LegacyRsaUserNameIdentityTokenHandler (Driver.OpcUaClient.Session, plugged in through
the public UserIdentity(IUserIdentityTokenHandler) ctor) always uses the legacy
format, RSA-OAEP(len|password|serverNonce) under the server certificate, under an RSA
policy. That is what SDK 2 sends for passwords of 64 bytes or less and what SDK 1.5
clients always send. The legacy format has no size limit (the plaintext is
length-framed over as many RSA blocks as it needs), and an SDK 2 server reads both
formats. Policy None, the ECC policies and a missing certificate go to the SDK
handler unchanged.

OpcUaUserIdentityFactory.UserName is now the one UserName identity the driver, the
AdminUI browse and Client.Shared (CLI/UI) present. Client.Shared references .Session
for it.

Tests: handler unit tests (all three RSA policies, the 1.5-required algorithm URI,
SDK 2 server-side decrypt, the session Copy() keeps the type, None unchanged, and an
SDK control pinning the 64/65-byte switch); a Client.Shared identity test; and an
SDK2<->SDK2 integration test where a 65-char key is admitted in both formats and a
wrong 65-char key reaches the key check (BadIdentityTokenRejected). The #800 Windows
gate's SDK 2 CLI 65-char case now gates against the 1.5 server too.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 11:12:27 -05:00
Joseph Doherty 3fff439b11 fix(redundancy): advertise ServiceLevel 0 until the node has a redundancy level (#885)
v2-ci / unit-tests (push) Has been cancelled
The SDK starts Server.ServiceLevel at 255 (full service), and OpcUaPublishActor publishes nothing
until the node's first redundancy snapshot arrives, so a restarting pair node advertised 255 for the
~10-13 s it took to join, out-ranking the real Primary (250) for any client that picks the highest
ServiceLevel.

OtOpcUaSdkServer now writes StartupServiceLevel = 0 in OnNodeManagerStartedAsync: after the SDK's
CreateServerObjectAsync wrote 255 and before the server goes Running, so no session can ever read
255. 0 is the design's existing Down / Detached tier (ServiceLevelCalculator returns 0 for a member
that is not Up/Joining, and a node with no snapshot has no role yet). It is below the SDK 2
session-headroom floor (200), so it stands through session churn before the #884 claim, and the
claim's bind replays any level already requested. Single-node deployments are unaffected beyond
the window: the first snapshot publishes 250 as before.

New Node_with_no_requested_level_advertises_the_startup_level_until_one_is_published: reads 0
before the bind, 0 after a bind with nothing to replay, then 250 once published. Without the
startup write it reads 255 on the first session (mutation-checked). docs/Redundancy.md updated for
#884 and #885.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 11:11:11 -05:00
Joseph Doherty 5e3fb4229d test(redundancy): exercise SessionClosing in the #884 ServiceLevel claim test
v2-ci / unit-tests (push) Has been cancelled
The claim test disposed its sessions, which may leave them to time out rather than raise
SessionClosing, so only the SessionCreated half of the SDK's session-headroom overwrite was
proven. Sessions now close explicitly, a later level (230, inside the SDK's >= 200 overwrite
band for every case, where the old level-10 left the 200 case at 190 and so unexercised) is read
back on a surviving session after another one closes on its own. Probed without the claim: the
close-only read fails (255) in all three cases.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 11:00:59 -05:00
Joseph Doherty f66b9daec0 fix(redundancy): claim Server.ServiceLevel so SDK 2 stops overwriting 250/240 with 255 (#801)
SDK 2.0.1 recomputes Server.ServiceLevel from session headroom on every session create/close and
overwrites any value >= 200 unless a provider has claimed it through
IServerServiceLevelControl.ClaimServiceLevelControl. SdkServiceLevelPublisher never claimed it, so on
the docker-dev pair both nodes read 255 as soon as a client connected while their own metrics showed
250 (Primary) and 240 (Secondary); the Stale tier's 200 was lifted to 255 too. Levels below 200 (100,
0) were untouched, which is why the bootstrap 100 showed briefly.

The publisher now claims the value at construction and writes through the SDK's writer; a server
without the claim (test doubles) keeps the diagnostics-guarded write. New
Published_level_survives_later_sessions (250/240/200) fails without the claim (reads 255) and passes
with it.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 10:55:49 -05:00
Joseph Doherty da948e5fb5 fix(history): a folder's event history includes its object's attribute alarms (#882)
v2-ci / unit-tests (push) Has been cancelled
HistorianGateway answers a per-source ReadEvents (Source_Object = DevWinPlatform) with the
object's own events (SourceName DevWinPlatform) AND its attribute alarms, which it reports as
<Object>.<Attribute> (Source_ProcessVariable, e.g. DevWinPlatform.CPULoadAvg). The Gateway
backend's defensive client-side source filter compared SourceName for equality, so it dropped
the attribute alarms: 70 of the 72 DevWinPlatform events on vd03.

The filter (and the past-the-cap "more events" signal) now uses EventSourceMatch: the source
itself, or <source>.<anything> on the '.' boundary, ordinal. DevWinPlatform never matches
DevWinPlatform2 or DevWinPlatform2.X. This needs no wire change and still makes one request.
Each event keeps its own SourceName. SourceNode stays the condition its stored alarm id names,
which is null for AVEVA rows. The attribute events ride the notifier's own read, so the
per-folder HistoryRead verdict (#826) governs them: a denied folder answers
BadUserAccessDenied before any backend call. The SqlServer and QuestDB backends still match
exactly; they hold only OtOpcUa's own events.

Tests:
- GatewayAttributeEventSourceTests (16): the boundary theory, attribute-in/sibling-out through
  the data source, the drain signal, and an exact attribute-named read.
- AttributeEventHistoryWireTests (2, over a real SDK session with the production Gateway
  backend on an AVEVA-shaped deterministic gateway): object + attribute events with their own
  SourceName, a prefix sibling excluded, paging, and the ACL-on deny with no backend call plus a
  granted positive control.
- Mutation-checked: exact match fails both wire tests; dropping the '.' boundary fails the
  wire sibling check and 6 unit tests.
- Live AVEVA harness re-run against vd03 HistorianGateway ea9f21c (read-only, no WinRM):
  folder DevWinPlatform 2 -> 72 events (whole, 72 1-event pages, reverse); the new #882 check
  passes; 221 of 225 shared cases are identical to the #801 SDK2 baseline, and the 4 that
  differ are exactly the DevWinPlatform event cases. Recorded in aveva-history-801.md.

Follow-up #886: production folders name RawPaths/EquipmentIds, not Galaxy objects, plus the AVEVA
severity/condition property mapping.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 10:34:29 -05:00
Joseph Doherty b27b5b9cc1 fix(history): advertise HistoryServerCapabilities from the wired backend (#881)
v2-ci / unit-tests (push) Has been cancelled
Every Boolean under Server/ServerCapabilities/HistoryServerCapabilities read
false (the SDK default) even though data and event history were served, so a
spec-following client skipped HistoryRead entirely.

Each backend now declares what it serves through its own interface:
- IHistorianDataSource.ReadCapabilities (Data / Events / ServerTimestamps):
  Gateway Data|Events; SqlServer + QuestDb add ServerTimestamps (they store
  ServerTimeUtc, the Gateway stamps read time); Null declares None. A
  reflection guard requires every production source to declare it itself.
- IHistorianDataUpdater.Capabilities (required member): SqlServer and QuestDb
  declare Insert/Replace/Update/DeleteRaw/DeleteAtTime; the Gateway provider
  has no updater.

The node manager writes the flags on every source/updater assignment
(AdvertisedHistoryCapabilities), and the same declarations drive dispatch:
an update the updater does not declare is BadHistoryOperationUnsupported
without reaching it, the HistoryWrite AccessLevel bit follows a non-empty
declaration, and event-history notifier promotion follows the Events flag.
Event and annotation update flags stay false (nothing dispatches them).

Tests: booted-server wire test per backend + Null + a SqlServer->Gateway
rewire pairing on one session; dispatch gating unit test; the #801 live
gate now asserts the Gateway flags (not run here).

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 10:31:29 -05:00
Joseph Doherty ac4c4d40dc test(validation): #800 Windows service gate harness + windev results (SDK 1.5 -> 2.0.1 in place, rollback)
v2-ci / unit-tests (push) Has been cancelled
Adds scripts/validation/opcua-windows-service/: a phased PowerShell gate (Invoke-ServiceGate.ps1)
that installs the known-good build as the real OtOpcUaHost service via Install-Services.ps1, seeds
LocalDb through the production cache API (Seeder, built against the baseline tree), upgrades in
place with config/PKI/LocalDb carried with their ACLs, starts/stops through the SCM, runs a strict
SDK 2 probe matrix (Sign + SignAndEncrypt, valid/wrong/unknown/disabled/anonymous keys, untrusted
client and server), exercises the ScriptWorker isolation, rolls back and cleans up.

windev 2026-11-12 (efb9197a -> 438484fd -> efb9197a): app cert CEF9D91F... reused byte-for-byte
with identical PKI SDDL on every start, all gating checks pass, 0 secrets in logs. Findings filed:
#879 (None endpoint cannot be dropped), #880 (SDK 2 client 65-char key vs SDK 1.5 server:
rollback/QA risk), #883 (Install-ScriptWorkerIsolation.ps1 fails on pwsh 7 and 5.1; runtime
isolation proven with the patch in the evidence directory).

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 10:28:49 -05:00
Joseph Doherty 0f001bccdb test(validation): pair-gate helper for the #801 revocation/expiry/ACL checks on docker-dev
v2-ci / unit-tests (push) Has been cancelled
Console helper that mints and revokes OPC UA API keys through the same IApiKeyAdminService the AdminUI
/apikeys page calls, edits NodeAcl grants in the rig ConfigDb, and holds one OPC UA session open per node
(Client.Shared) with a subscription plus a read/write every 3 s, so a revocation, expiry or grant removal
can be observed landing on an ALREADY-OPEN session on both pair nodes next to a non-revoked control key.
Credentials go to a 0600 file and are never printed. LOCAL docker-dev only.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 10:22:33 -05:00
Joseph Doherty 013d02db95 fix(docker-dev): give each pair node its own ApplicationUri and advertise its partner (#801)
Every docker-dev node served the compiled default ApplicationUri "urn:OtOpcUa" with a one-entry
Server.ServerArray, so the two halves of a redundant pair were indistinguishable by URI and peer
discovery returned nothing. Each node now sets OpcUa__ApplicationUri = urn:OtOpcUa:<service> (its
seeded ClusterNode.ApplicationUri) and OpcUa__PeerApplicationUris__0 = its pair partner. Verified on
the central pair: ServerArray reads [self, partner] on both nodes.

RedundancySupport still reads None by design (#717). The pair's 255/255 ServiceLevel was not a rig
config problem: it is the SDK 2.0.1 session-headroom overwrite filed separately.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 10:22:33 -05:00
Joseph Doherty 008f0b36b2 docs(evidence): #801 real-AVEVA history gate — SDK 2.0.1 identical to 1.5.378.182
v2-ci / unit-tests (push) Waiting to run
Both builds served HistoryRead from the real AVEVA Historian via the DEV
HistorianGateway (vd03, ea9f21c): 225 cases / 3,855 rows / events, 0 differences.
Raw also matches direct SQL on Runtime.dbo.History (6/6) and histgw-cli; simple
aggregates match a recomputation from raw (336/336). Modified, HistoryUpdate,
seven refused aggregates and HistoryServerCapabilities answer as before. No write
reached the historian. Findings (pre-existing, not SDK): all HistoryServerCapabilities
booleans false; AVEVA attribute-alarm events unreachable per source.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 10:02:34 -05:00
Joseph Doherty f607ee7502 test(historian): #801 real-AVEVA history gate over the SDK wire (env-gated)
AvevaGatewayHistoryLiveGateTests boots the server in-process, wires the production
Gateway backend (HistorianBackendRegistry -> GatewayHistorianDataSource) to a real
HistorianGateway and drives HistoryRead over a real SDK session: raw (all, paged,
reverse, bounds), modified, the ten served aggregates and seven refused ones, AtTime,
events (all, 1-event pages, reverse), a missing tag, HistoryServerCapabilities and
node metadata, and a HistoryUpdate that must be refused (sandbox-bound node only).
Every wire answer is compared with the backend's direct answer; the run is dumped as
JSON for cross-SDK and SQL comparison. Skips unless HISTGW_GATEWAY_ENDPOINT and
HISTGW_GATEWAY_APIKEY are set; read-only by construction.

scripts/validation/aveva-history-801: run/compare instructions, the SDK 1.5.378
baseline port patch (API spelling only), and the diff / SQL / aggregate / histgw-cli
comparison scripts.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 10:02:21 -05:00
Joseph Doherty c266971d51 docs(scripting): #789 owner accepts the measured Windows-VM budgets (idle p50 ≤ 500 µs, ≤ 50 pts batched at 10k/s)
v2-ci / unit-tests (push) Has been cancelled
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 09:54:22 -05:00
Joseph Doherty d8a000a409 perf(scripting): #789 profile the Windows misses; add the batched paced row; nothing kept in the runner
v2-ci / unit-tests (push) Has been cancelled
Profiled the remaining lmxopcua#789 misses on windev, a VMware VM, using per-thread CPU and context-switch accounting
plus ETW CPU sampling.
- One blocking hand-off costs ~26 us of latency and ~18 us of CPU there, about 10x macOS.
- An idle evaluation makes four hand-offs in a row: host writer -> worker reader -> evaluation thread -> host reader
  -> dispatcher.
- 48% of host CPU at 10k/s is in the kernel: the per-RunAsync NtWriteFile, thread-pool wakes, reads and waits.

Spin-before-block was measured at every hand-off and is not kept. It meets 150 us idle p50 (118 us) only by keeping
about two host cores and four worker cores busy for one stream. At 10k/s it reads +250 points.

Harness: new "paced 10,000/s, batched" row. It uses the same rate and keys, with each 1 ms tick submitted as one
RunBatch call, which is the production seams' shape. It runs after the original four phases, so their rows stay
comparable. On windev it halves host CPU, to +36..+45 points against +71..+84 unbatched. That is still above 25.

Design doc:
- 12.8: the record.
- 14 #15: the owner's decision on the Windows budgets. Recommended: measure on bare-metal Windows, then state the two
  budgets per platform, with the batched row as the gate. The alternative is a shared-memory transport.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 09:52:09 -05:00
Joseph Doherty 438484fdca test(validation): ScadaBridge consumer matrix — failover suite, UNS subscribe, ConditionRefresh after reconnect (ScadaBridge#125)
v2-ci / unit-tests (push) Has been cancelled
Probe 'failover <primary> <backup> <mode> <dir>' drives ScadaBridge's own DataConnectionActor
(production primary/backup machine, real DataConnectionFactory, default DataConnectionOptions,
FailoverRetryCount 3) against two servers; run.py loses the primary (SIGKILL or SIGSTOP) once the
probe reports a steady primary, and the probe requires Bad quality on loss, "Connected to backup",
the backup's own value, a fresh ConditionRefresh replaying the native condition, and a write via the
backup. rig-failover.sh runs the same probe against the local docker-dev central pair.

Matrix legs gain a UNS subscription case and a Server-object alarm subscription on the reconnected
session whose ConditionRefresh must replay the active condition (11 cases). run.py gains --versions,
--suites, --failover-modes and --failover-loss.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 09:28:01 -05:00
Joseph Doherty e5a03e3cce test(acl): pin the fail-closed rule for non-condition events on folder items (#838 item 3)
v2-ci / unit-tests (push) Waiting to run
A folder event item under enforcement withholds an event that names no live
condition; flag off, the SDK's own type + SourceNode check applies. Positive
controls: the same event reaches the item flag-off and a condition event
reaches it flag-on. Mutation-checked: IsNotifierEventVisibleTo returning true
for a non-condition event fails the test.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 08:54:43 -05:00
Joseph Doherty a28adba2fd test(galaxy): live gate for the stream-attached ack against a real gateway (#877)
v2-ci / unit-tests (push) Failing after 10m42s
Adds GatewayGalaxyStreamAttachedAckLiveTests (skip-gated on MXGW_ENDPOINT +
GALAXY_MXGW_API_KEY, like the other gateway live tests). Proves what the unit
suite cannot: the request flag reaches a real gateway and the sentinel comes back.

- live session advertises stream-events-attached-ack; GatewayGalaxySubscriber reports it
- raw stream: sentinel is the first message when requested (family Unspecified,
  worker_sequence 0, live_after 0) and never sent when not requested (control,
  with a data event as the positive control)
- real GalaxyDriver + EventPump + GatewayGalaxySubscriber behind a pass-through
  recorder: exactly one sentinel, ack observed before the first SubscribeBulk,
  pump StreamAttached completed, sentinel never dispatched nor counted as
  received (galaxy.events.stream_attached=1, received == non-sentinel events),
  no missing-ack warning within the production 1 s budget, and the static
  attribute TestMachine_002.Tagname delivers its initial value
- control: a subscriber that opts out gets no sentinel and the pump never
  reports one, while the #865 fallback still delivers

Run 2026-11-12 against windev 10.100.0.48:5120 (mxaccessgw 447070a5): 4/4.
Mutation checks, each caught: subscriber not requesting the ack (sentinel
count 0); driver skipping the ack wait (wire order became stream | subscribe |
ack, 2/2 runs, so the race is real on a live gateway); pump not intercepting
the sentinel (received=2 vs 1 non-sentinel).

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 08:08:55 -05:00
Joseph Doherty f264977fb0 feat(galaxy): wait on the gateway's stream-attached ack instead of the 1 s best-effort wait (#877)
v2-ci / unit-tests (push) Has been cancelled
mxaccessgw#227 (Contracts/Client 0.4.0) adds an opt-in attach acknowledgement on
StreamEvents: a StreamAttached sentinel MxEvent sent first, only after the
stream's subscriber is registered on the session. This closes the #865 attach
race that waiting for "stream requested" only narrowed.

- Bump vendored MxGateway.Client/Contracts 0.3.0 -> 0.4.0 (nuget-packages/,
  pulled from the Gitea feed; local-mxgw stays the mapped source).
- IGalaxySubscriber.SupportsStreamAttachedAck (default false, so fakes and older
  seams keep the fallback); GatewayGalaxySubscriber reads it from the live
  session's capabilities (stream-events-attached-ack) and passes
  emitStreamAttached accordingly; TracedGalaxySubscriber forwards it.
- EventPump: records AttachAckExpected before the stream opens (published
  before StreamRequested completes); intercepts ev.StreamAttached ahead of the
  ReplayGap check - completes StreamAttached and continues: never dispatched,
  never counted as received, never a resume cursor (worker_sequence 0). The
  signal also completes when the loop ends so no waiter hangs. One pump = one
  stream, so a reconnect's new pump has a fresh signal. New counter
  galaxy.events.stream_attached.
- AwaitCurrentEventStreamAsync: with the capability, waits on StreamAttached
  within the same StreamRequestWait budget (1 s); a missing ack logs a
  rate-limited Warning and proceeds instead of hanging. Without the capability
  the old StreamRequested wait is the fallback.

Tests (StreamAttachedAckTests, 6): ack-gated subscribe gets its initial value;
capability absent uses the fallback; missing ack times out with a warning; the
sentinel is never dispatched; a dead stream releases ack waiters; reconnect
gives each stream a fresh signal. With the ack wait skipped, 3 of them fail.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-12 07:38:09 -05:00
Joseph Doherty 633f43170a test(galaxy): #872 pin the detected_by rule per source
v2-ci / unit-tests (push) Has been cancelled
The rule fix itself landed in 1a08b4f7 (detail-0 failure rows route on
category); its test only exercised raw_detected_by = 3. Add per-source
coverage for every MxStatusSource value mxaccessgw's MapSource emits
(-1, 0..5, and an out-of-range value) across the failure categories, plus
the live-captured shape the old rule got wrong (OperationalError detected
by the responding automation object, mxaccessgw docs/AlarmProbeFindings.md)
and a RequestingLmx CommunicationError positive control. All 10 new cases
fail against the old 'detail 0 && raw_detected_by != 0 => comms' rule.

Comment in FromMxStatus now cites MapSource and says why NMX (2/3) is not
a transport signal. Item 1 (Warning/Pending) stays open pending a capture.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-12 07:17:53 -05:00
Joseph Doherty 135099d722 docs(validation): scadabridge-sdk-compatibility README — #121 fixed, Sign leg, trust controls, ACL switch
v2-ci / unit-tests (push) Waiting to run
Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-12 07:01:24 -05:00
Joseph Doherty ea83fc58b2 test(validation): ScadaBridge consumer matrix — provisioned client identity, Sign leg, trust negative controls (ScadaBridge#121)
Probe 'prepare' provisions the consumer identity via OpcUaApplicationIdentity.EnsureApplicationCertificateAsync
and exports the public DER; runner installs it in the server trusted store before start. Adds Sign
(OPCUA_MATRIX_SERVER_SECURITY=sign), untrusted-client / untrusted-server fail-closed controls per secure
mode, and OPCUA_MATRIX_ACL=off. Autoaccept stays false everywhere.
2026-11-12 07:01:14 -05:00
Joseph Doherty f127f45428 fix(opcua): don't register with a discovery server unless configured (#876)
v2-ci / unit-tests (push) Has been cancelled
OpcUaApplicationHost left ServerConfiguration.MaxRegistrationInterval at the
SDK default (30000) with a null RegistrationEndpoint, so SDK 2.0.1's
StandardServer armed its registration timer against opc.tcp://localhost:4840,
which on an OtOpcUa node is the server itself (no LDS). It answered
BadServiceUnsupported and the SDK retried forever, visible since #874.

Registration is now opt-in: ApplyDiscoveryRegistration sets
MaxRegistrationInterval = 0 (the value that keeps StandardServer from arming
the timer) unless OpcUa:DiscoveryServerUrl is set, in which case the server
registers against that URL every OpcUa:DiscoveryRegistrationIntervalMs
(default 30000) over SignAndEncrypt/Basic256Sha256, like the SDK's own
default endpoint. The validator requires an absolute opc.tcp:// URL and a
positive interval when the URL is set. Documented in Configuration.md and
OpcUaServer.md.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-12 06:29:35 -05:00
Joseph Doherty 182fb353e3 docs(deploy): #793 vd03 close-out — MaxRpm re-pointed (9adc5590), ACL off, ScadaBridge N/A, rollback compatible
v2-ci / unit-tests (push) Has been cancelled
2026-11-12 06:16:20 -05:00
Joseph Doherty 9ee9d0c78b docs(deploy): #793 vd03 record — title at 5fb4b4db, owner-action outcomes, resolved unknowns
v2-ci / unit-tests (push) Has been cancelled
2026-11-12 06:08:51 -05:00
Joseph Doherty 2f0533c4f7 docs(deploy): #793 vd03 post-deploy deploy Applied (e4b4824a)
v2-ci / unit-tests (push) Has been cancelled
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-12 06:03:22 -05:00
Joseph Doherty 3545e50639 docs(deploy): #793 vd03 upgrade to 5fb4b4db executed — record, preflight needs sqlcmd -I
v2-ci / unit-tests (push) Waiting to run
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-12 06:00:13 -05:00
Joseph Doherty 5fb4b4db54 Merge #789 §12.7 dispatcher-hop measurements (no code change kept)
v2-ci / unit-tests (push) Failing after 1h13m31s
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 22:29:36 -05:00
Joseph Doherty 464b699e18 docs(scripting): #789 §12.7 — dropping the result-dispatcher hop, measured and not kept
Measured on windev (3 alternated sessions) and macOS: a pool resume in place
of the dispatcher, and an inline-on-reader upper bound, with the harness as
committed and with its paced continuation made synchronous.

- pool: 10k/s host CPU unchanged, idle p50 and idle CPU worse (macOS idle CPU
  x3 from pool spin); only the closed loop gains. Not kept.
- inline (unsafe for every seam; the ceiling for a VirtualTag-only opt-in):
  ~8 points of a core at 10k/s raw and ~50 us idle p50 on windev with a cheap
  continuation, nothing with the harness as committed. Both budgets still
  missed by a wide margin, so the opt-in was not built.

No code change. Records the remaining attribution (harness pool hop ~15
points raw; reader wake + syscalls; worker side untouched).

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 22:28:58 -05:00
Joseph Doherty 31be927f65 Merge #789 Windows worker pipe I/O off the completion port (+review fixes)
v2-ci / unit-tests (push) Waiting to run
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 20:54:49 -05:00
Joseph Doherty c56fcc8af7 docs(scripting): #789 §12.6 — review follow-ups and their windev / macOS re-run
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 20:53:57 -05:00
Joseph Doherty 03ac57d5d6 fix(scripting): #789 review — reader faults end the worker, write stalls kill it, Cancel off the worker reader
- Host reader: the frame read and its handling are caught separately. Only the
  read's I/O exceptions mean the pipe ended; anything HandleFrame's inline
  completion work throws now kills that worker as ProtocolFault (outstanding
  runs -> worker restarted / BadResourceUnavailable) instead of escaping a
  dedicated thread (process crash) or being swallowed as a clean end (an
  InvalidOperationException used to be, leaving the worker unread until a Crash
  2 s later).
- Worker reader: the same catch-all around HandleFrame (exit ProtocolFault).
- Worker: Cancel() runs on the pool, so a script's blocking
  ctx.CancellationToken callback cannot stop the worker reading its pipe.
- Host write-stall check: ScriptWorkerFrameWriter.WriteInProgressFor; the
  heartbeat monitor kills a worker whose pipe write has been blocked longer than
  max(10 s, silence limit) as KilledUnresponsive, which fails the write and
  releases the producer (seam submit, deploy load, Cancel). Fixes the wrong
  'heartbeat monitor ends it' comment on the pipe buffer quota.
- StopAsync sends Shutdown off the caller so its 1 s budget holds on a full pipe.
- WindowsBlockingPipeStream refuses the async surface (Stream's base versions
  serialise a write behind a waiting read).
- Tests: PipeFaultTests (throwing completion callback -> ProtocolFault and a
  BadResourceUnavailable sibling, shard serves again; a worker that stops reading
  but heartbeats is killed within the bound and the blocked producer returns,
  via the Hello.test_stall_reader_on_diag seam); BlockingPipeStreamTests dispose
  with a write blocked on a full pipe, and with a read and a write both pending;
  Category=ScriptWorker on the protocol, handshake and pipe test classes.
  Mutation checks: no stall kill -> stall test red; catch-all without the fault
  -> callback test red (Crash).

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 20:26:02 -05:00
Joseph Doherty d4a2aa6142 docs(scripting): #789 design §12.6 record — Windows blocking pipe I/O, windev before/after
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 20:10:58 -05:00
Joseph Doherty 82da1ecbc8 perf(scripting): #789 give the Windows worker pipe a 1 MiB buffer quota each way
With a zero quota an overlapped write stays pending until the peer has read the
bytes, so the new blocking writer waited out a peer wake-up per frame (windev
closed loop 54k -> 26k evals/s). With a quota the write completes once buffered.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 19:30:07 -05:00
Joseph Doherty a7fa327e61 test(scripting): #789 BlockingPipeStream duplex, large frame, peer-gone and self-dispose cases
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 19:13:48 -05:00
Joseph Doherty 3c0e5ea4d7 perf(scripting): #789 Windows worker pipe I/O blocks on dedicated threads, off the completion port
A .NET pipe opened Asynchronous completes even its synchronous Read/Write
through the thread pool's completion port, so every frame on Windows cost a
completion packet plus a pool work item (and the pool's spin-wait). New
WindowsBlockingPipeStream issues its own overlapped ReadFile/WriteFile with the
OVERLAPPED.hEvent low bit set (no completion packet) and waits on that event on
the calling thread; reads and writes stay concurrent, unlike a synchronous
handle. ScriptWorkerFrameCodec.BlockingStream picks it on Windows and the pipe
itself on Unix, so both ends now use the dedicated blocking reader + synchronous
combining writer on every OS (UseBlockingIo and the async read loops are gone).

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 19:11:10 -05:00
Joseph Doherty f109e5f29e Merge #875 Galaxy initialize 20 s budget (per-driver-type capability defaults)
v2-ci / unit-tests (push) Successful in 1h14m54s
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 18:44:52 -05:00
Joseph Doherty 95bde828eb fix(galaxy): #875 Galaxy initialize gets a 20 s budget via per-driver-type capability defaults
Gate F (#761) against the VD03 gateway: Galaxy initialize (session open +
platform seed) took 4–5 s against Tier A's 5 s Initialize timeout, so 10 of 40
restarts failed, paid a 10 s retry and left a killed gateway worker.

DriverFactoryRegistry.Register takes optional capabilityDefaults per driver
type (GetCapabilityDefaults); DriverResilienceOptionsParser layers them over the
tier table and under the per-instance ResilienceConfig; the Host wires the
registry into DriverCapabilityInvokerFactory. Galaxy registers Initialize =
20 s (no retry, breaker 2), under the actor's 30 s pessimistic bound.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 18:44:44 -05:00
Joseph Doherty 2be43aa984 Merge #717 Warm redundancy design spec + plan (DRAFT, awaiting owner approval)
v2-ci / unit-tests (push) Has been cancelled
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 18:41:22 -05:00
Joseph Doherty 94c71541a1 Merge #761 gate F record (Galaxy session leak: PASS)
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 18:41:22 -05:00
Joseph Doherty 8d1621e534 docs(review): #761 gate F — 20 Galaxy respawn deploys keep mxgateway.sessions.open flat (VD03 DEV gw)
PASS: gauge baseline 2, +2 with the rig's driver on both centrals, 4 after 20
config deploys (40 respawns; rig opened 39 / closed 37). Final drop after
removal not observed: the Mac's VPN to VD03 dropped right after the settle.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 18:40:52 -05:00
Joseph Doherty c06769991b docs(plans): #717 Warm redundancy design spec + implementation plan (draft for owner approval)
Spec: per-shape contract (Unset/None/Warm), RedundancySupport + ServerUriArray graph,
ServiceLevel bands, per-service standby behaviour, alarm suppression + ConditionRefresh
(D5), strict first-delivery (D4), transition sequencing, ordered snapshot revisions,
ScadaBridge impact, SDK 2.0.1 re-proof list, open questions. Plan: T0-T13 with wire
tests, docker-dev live gate, rollout/rollback. Docs only.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 18:19:14 -05:00
Joseph Doherty 5ac5520532 docs: #873 SDK 2.0.1 API-key logins need client clock within 5 min ahead of the server (rollout runbook + security guide)
v2-ci / unit-tests (push) Waiting to run
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 18:08:39 -05:00
Joseph Doherty 43f8b8e85a Merge #860 #818 CI integration fixtures: busy-port retry, per-run names; floor 448
v2-ci / unit-tests (push) Has been cancelled
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 17:51:57 -05:00
Joseph Doherty 6ad643b3c7 Merge #874 OPC UA SDK server log output reaches the host logger
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 17:51:57 -05:00
Joseph Doherty 71740a5a42 ci: #818 raise v2-integration MIN_EXECUTED 440 -> 448 after the first hosted pass (run 1225: NAT tests executed 9 + 12, 525 total)
Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 17:51:49 -05:00
Joseph Doherty b114779ba5 fix(ci): #860 integration fixtures survive a busy host port and concurrent runs
Hosted v2-integration run 1224 got past the old fixture-host timeout and
failed one step earlier: "Bind for :::19000 failed: port is already
allocated" — the runner's docker daemon is shared. Fixtures now retry on the
next host port when one is taken (opc-plc keeps host == listen port, as its
advertised endpoint requires) and export the ports actually used; container
names carry the run id so a concurrent run's `down` can't remove them, and a
new `logs` subcommand replaces the workflow's hard-coded name list.

Verified locally: with 19000 squatted, QuestDB moved to 18813/19001 and every
fixture reached ready; logs/down found the suffixed containers.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 16:53:14 -05:00
Joseph Doherty 2e948d3139 fix(opcua): #874 route the server's OPC UA SDK log output to the host logger
The host handed the SDK a provider-less telemetry context, so SDK-side
refusals — identity-token decryption (the SDK 2 RsaEncryptedSecret clock
window, #873), certificate validation, secure-channel faults — left no server
log line (found on windev during the #800 Windows validation). The hosted
service now creates one ServerSdkTelemetry context from its ILoggerFactory
(Warning and above, SDK categories, rate-limited 20/min per category) and
shares it between OtOpcUaSdkServer and the ApplicationInstance; released on
dispose. Test: an untrusted client's rejection reaches the captured log.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 16:37:03 -05:00
Joseph Doherty 88698ca042 docs(deploy): vd03 upgrade-readiness pack b8b4c18f -> master (#793)
v2-ci / unit-tests (push) Waiting to run
Read-only ConfigDb preflight (scripts/preflight/vd03-preupgrade-b8b4c18f-configdb-checks.sql)
covering every deploy gate added since the deployed build: VirtualTag/Calculation effective
triggers (#783), VT->VT reads and VirtualTagDependencyCycle (#791), newly forbidden script
APIs (#792, E2), credential-reference namespace / other-instance / reserved-name rules
(#815 #835 #836 #848 #849 #850), MTConnectMultipleDevices, Sql query/write gates (#781),
Heidenhain, UNS effective-name collisions and the FOCAS renames/value changes. Verified on a
throwaway b8b4c18f-schema database (every check fires on seeded rows) and clean against the
docker-dev config.

Upgrade notes (docs/deploy/vd03-upgrade-b8b4c18f-to-master.md): the EnableSnapshotIsolation
migration and how to apply it by hand, the env-var audit (Security:ApiKeys:Pepper is now
required and absent on the box; JWT key validated at start), SDK 2.0.1 / CA KeyUsage, the
ScadaBridge-visible changes, swap/rollback and post-deploy checks.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 16:13:10 -05:00