Joseph Doherty
e5b26f3772
The baseline is83ed880e(SDK 1.5.378.182). The candidate is the #880 change (onda948e5f), run twice. Fixed: every SDK 2 client (Shared, OpcUaClient driver, CLI) logs in to the SDK 1.5 server with a 65-char key. A wrong 65-char key gets BadIdentityTokenRejected. In the control, the SDK default handler is refused by 1.5 with BadIdentityTokenInvalid and admitted by SDK 2. Mutation (the factory reverted to the SDK default handler): the SDK 2 Shared connect, the driver connect and all three CLI calls fail against 1.5 with BadIdentityTokenInvalid, and the wrong-key case gets Invalid instead of Rejected. This is #880 reproduced. In both runs the UNS cases fail against the master server for every client SDK. That is a fixture gap since #808 and is tracked as #888. Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
OPC UA SDK process compatibility
Run from a dedicated candidate checkout with .NET 10, Python 3, PowerShell 7,
and Docker. Supply the unchanged original application's existing host/ and
client/ publish directories. The output directory must not already exist.
./scripts/e2e/test-opcua-sdk-compatibility.ps1 `
-BaselineRoot /absolute/original-checkout `
-BaselineArtifacts /absolute/original-publishes `
-CandidateRoot /absolute/dedicated-candidate-checkout `
-OutputDirectory /absolute/fresh-evidence-directory
The baseline console references the exact published production assemblies; it does not rebuild or edit the original application. The candidate console references the production host, Client.Shared and OPC UA driver projects, and the candidate CLI is separately published. Different SDK versions always run in different processes. Source revisions, SDK assemblies, SHA-256 hashes, commands, per-case results, actual namespace URIs, negotiated Shared security, loaded client SDK assemblies, timestamps and backend request logs are retained.
The required Shared legs are original→candidate, candidate→original and
candidate→candidate. A fourth original→original leg provides a reference for
both the application contract and the small performance sample. Every Shared
leg checks the exact scalar, array and condition browse identities, both
namespaces by URI, typed scalar/Int32[3] read/write, monitored Good quality and
source/server timestamps, the production alarm event filter and native
acknowledgment, and complete raw/event history across continuation points.
Event-history results include timestamp ties, exact IDs/order and positive and
negative OfType predicates. The historian is a deterministic fixture behind
the production server's session-bound continuation store; these results do
not certify any SQL Server, QuestDB or AVEVA backend.
Server, Shared client and driver require trusted Basic256Sha256 SignAndEncrypt peers, with certificate autoaccept disabled. Public certificates are exchanged before sessions open. The test authenticator accepts one exact, random, API-key-shaped credential from the process environment; production ACL, write and condition-method checks use the resulting key identity. This fixture does not replace cryptographic API-key-authenticator coverage.
The credential has the production ApiKeySecret.Generate shape and length,
65 characters (lmxopcua#880). The 55-character key used before then stayed
under SDK 2's 64-byte threshold, so it hid a real failure. SDK 2.0.1's own
UserName handler sends a longer password as an RsaEncryptedSecret, and an
SDK 1.5 server rejects that with BadIdentityTokenInvalid. The production
clients (Shared/CLI and the driver) force the legacy RSA format. Every Shared
leg adds two cases. The first sends a wrong secret of the same length, which
must reach the key check (BadIdentityTokenRejected, not a format failure).
The second runs only on an SDK 2 client and is the negative control: the
SDK's default handler with the valid key must be refused with
BadIdentityTokenInvalid by a 1.x server and admitted by an SDK 2 server.
The Shared client passes the server's SDK version from ready.json for it.
The driver and actual CLI run in a disposable --network none Linux container,
alongside a separate baseline or candidate server process. PKI lives only in
the mounted task directory; no host profile PKI or fleet endpoint is touched.
The driver hardcodes LocalApplicationData/OtOpcUa/pki, so Linux's task-local
XDG_DATA_HOME supplies isolation. The helper refuses a driver store outside
its task root. CLI uses its own task-local canonical store. CLI currently
forces AutoAcceptCertificates=true and exposes no strict-validation flag
(CommandBase.CreateConnectionSettings); the exchanged peers and encrypted
server remain in use, but CLI is not counted as strict client validation.
Driver history is N/A because the application removed that capability (#655);
Shared and CLI execute history reads instead.
Three measured rounds follow ten warmup operations per kind in each round: 50 scalar reads, scalar writes, array reads and array writes per kind. The sample records nearest-rank p50/p95/p99, sequential throughput, client CPU, allocated bytes, GC collections, working set, private bytes, threads and handles. This is a small local diagnostic on a shared development host. It has no production fan-out, reconnect, queue, backpressure, backend or soak performance claim and does not establish Gate G4.
A separately published server helper can be used by other local consumer
checks. Supply a fresh directory and OPCUA_MATRIX_PASSWORD in the process
environment, then launch dotnet ProcessHost.dll server <directory>.
ready.json/endpoint.txt identify the ephemeral loopback port; touching
<directory>/stop shuts it down. Its default is strict SignAndEncrypt only.
OPCUA_MATRIX_SERVER_SECURITY=none is an explicit fixture-only override for
checking an existing unsecured consumer subset; the matrix runner clears that
override. Anonymous is disabled in either mode. Secure clients must place
their public certificates under server-pki/trusted/certs before server start
and trust the resulting server-pki/own/certs certificate.
Explicitly supplied timestamps are asserted. Unset source timestamps are recorded separately as observations, with their original application/SDK wire values. The harness does not change historian algorithms or normalize the unset-timestamp sentinel to manufacture parity.
For the supplementary sustained warmup experiment, use the built consoles from one completed matrix run:
python3 scripts/validation/opcua-sdk-compatibility/performance.py \
--baseline-harness /absolute/run/build-baseline/bin/Release/net10.0/ProcessHost.dll \
--candidate-harness /absolute/run/build-candidate/bin/Release/net10.0/ProcessHost.dll \
--output /absolute/fresh-performance-directory
This runs three fresh baseline/candidate process pairs serially, with 1,000
warmup and 2,000 measured operations per kind in each process. Both server and
client receive DOTNET_TieredCompilation=0 for this experiment only; the
application's shipped configuration is unchanged. The recorded budget is a
15% increase in the median of the three p95 samples for any operation kind.
Exit code 2 means a diagnostic budget regression, while process failures
remain errors. Even an experiment within that budget does not certify the
complete production performance matrix. The allocation-per-operation field
is the average across all four operation kinds in a measured workload.