lmxopcua/scripts/validation/opcua-client-lifecycle/README.md

# Production OPC UA client lifecycle diagnostic

This standalone harness runs the unchanged production `OpcUaClientService`,
`OpcUaClientDriver`, and `OpcUaClientDriverBrowser` in one persistent Linux client
process against a separate process using the production SDK server and node
manager. Both processes load OPC Foundation SDK 2.0.1. The server fixture is a
local copy of the compatibility harness at commit `7b353d7b`; only this directory's
copy has configurable lifetime and atomic public resource snapshots.

Run a short diagnostic from a checkout containing the SDK 2.0.1 changes:

```sh
python3 scripts/validation/opcua-client-lifecycle/run.py \
  --duration 90 --output /tmp/otopcua-client-lifecycle-diagnostic-unique
```

The coordinator must launch and own the full process separately:

```sh
python3 scripts/validation/opcua-client-lifecycle/run.py \
  --full --duration 7200 --output /tmp/otopcua-client-lifecycle-full-unique
```

The output path must not exist. `--full` rejects a duration below 7200 seconds;
`fullTwoHourLifecycle` is true only after the client monotonic clock has actually
elapsed at least 7200 seconds and every final check completes. A diagnostic pass
cannot satisfy that gate. The outer process also has a duration plus 300-second
hard deadline and forcibly removes its own container on timeout.

The launcher publishes this standalone project, then uses the locally cached
official `mcr.microsoft.com/dotnet/sdk:10.0.103-noble` Linux arm64 image with
`--network none`, no host ports and fresh task-owned PKI. `XDG_DATA_HOME` is the
only storage environment override; it never changes `HOME`. A generated
credential stays in the child process environment. The output root is mode 0700
and contains generated private keys: **never commit the entire output directory**.
Commit only the selected redacted JSON/log receipts under `evidence/`.

Preparation invokes bounded failing connections through each product's public
API to create its own application certificate. It does not reflectively invoke
configuration builders or replace the store. Before the live session, public DER
certificates are exchanged explicitly among the three clients and server.
Autoaccept is disabled on both ends. Negotiation must be Basic256Sha256
SignAndEncrypt and API-key-shaped credentials must pass the fixture authenticator
and production authority/ACL path. Existing own DER hashes must remain unchanged
across all loop iterations.

Every cycle performs two Shared connect/read/write/subscribe/unsubscribe/
disconnect rounds on the same object. It performs two driver initialize/read/
write/subscribe/unsubscribe rounds, each followed by a live reinitialize with an
active subscription, then shutdown. It opens, browses and disposes two production
AdminUI browser sessions. Scalar Int32 and Int32[3] writes must roundtrip exactly,
and both Shared and driver scalar data notifications must arrive.

The external harness reads ownership fields for diagnostics only. Fields are
looked up by exact name, with missing/null layouts failing the run. It does not
write reflected fields or change SDK state. For SDK 2.0.1.0 only, each captured
`CertificateManager.m_disposed` boolean must be false while connected and true
after release. An independently created, deliberately disposed SDK manager proves
the before/after probe. All strong manager references are dropped each pass.
Eight released managers are checked per cycle: two Shared, four driver (including
live reinitialization), and two browser. Production session and data subscription
collections must clear, and public Shared connection information and driver health
must return to disconnected/Unknown. Collected weak references to disposed client,
driver, browser and browse-session objects must have no survivors at samples.

After every cycle the server's public session/subscription/monitored-item counts
must return to zero within a bounded settle period. Resource samples capture GC,
managed allocation/retained bytes, RSS/private bytes, global FDs and task/PKI FDs.
Owned-path FD matching canonicalizes absolute paths and uses a path-separator
boundary. Sampling starts cold, then every ten cycles and at completion. RSS and
FD observations require interpretation after warmup; they are not performance
budgets or an automatic G4 acceptance decision.

Artifacts are `result.json`, `source.json`, `command.json`, `build.log`,
`container.log`, and `task/{summary.json,resources.jsonl,sdk-assemblies.json,
client.log,server.log,prepare.log,server/live-resources.json}`. A failure additionally
writes a redacted `task/failure.txt`. The server uses a fixed application URI and
the three client certificate stores are reused for the entire process lifetime.

This is a small local lifecycle gate, not the entire Task 12 gate. It does not
cover certificate revocation, network stalls, node surgical edits/rebuild,
continuation-point abandonment/churn, representative driver fanout/load,
Windows install/publish, ScadaBridge or AVEVA interoperability. It makes no
baseline/candidate performance or full G4 claim. The coordinator's separate server
lifecycle harness supplies complementary SDK session/CP/server mutation coverage.

The checked-in 2026-11-10 diagnostic completed **91.104 seconds**, 68 cycles,
136 Shared rounds, 136 driver rounds (plus 136 live driver reinitializations),
136 browser sessions, and 544 verified released certificate managers. The final
sample had zero surviving weak product objects and zero owned PKI FDs; server
sessions/subscriptions/items returned to zero after every cycle. Warm samples
showed 146–148 total FDs; the final managed heap was 4,737,536 bytes and RSS was
181,018,624 bytes. The deliberate manager before/after control passed and
`--full --duration 90` exited 2 before creating artifacts. This is diagnostic
coverage only; the two-hour gate remains pending until a coordinator-owned full
run completes. Receipts are under `evidence/2026-11-10-diagnostic/`.