Files
lmxopcua/scripts/validation/opcua-lifecycle-memory/native-retention
..

macOS private-key reload native retention — #799

The bounded control identifies configured-directory private-key loading as a trigger for continuing Security::AppleDatabase heap inventory on this machine, while observed SDK certificate-core ownership and tracked .NET temporary-keychain ownership return to zero. No narrow built-in application ownership correction was found that preserves the current raw-session factory, Directory PKI, certificate selection/update behavior and certificate properties. #799 remains open. This is private reproduction/support material, not an approved production mitigation.

Only this new subfolder was changed. No product source, trust policy, SDK version, certificate factory/crypto provider, exportability, key storage format, existing PKI or resource budget was changed. No public upstream issue/message was posted.

Reviewed, executed control

Astra reviewed the actual source and installed-runtime decompile before execution, approved one fixed 200-acquisition run per arm as attribution-only evidence, and afterward independently matched all ten raw heap counts/bytes, checkpoint and completion counts, source hashes, installed crypto hash and eight SDK DLL hashes. The configured-store versus registry distinction below is supported by inspected upstream regression source; that upstream regression test was not executed.

Both arms ran in separate fresh processes on macOS 26.5 arm64, .NET SDK 10.0.403, runtime 10.0.14. A normal SDK ApplicationInstance created one RSA application PKI; its persisted tree was copied into separate private arm roots. Both used the same application URI, public DER identity and private key. Normal SDK configuration validation/certificate checks and teardown were identical. The loop called these public APIs, disposing each returned owner before checkpoints:

Arm Public acquisition, repeated exactly 200 times
Store SecurityConfiguration.FindApplicationCertificateAsync(Basic256Sha256, privateKey: true, ...)
Registry ICertificateRegistry.AcquireApplicationCertificateBySecurityPolicy(Basic256Sha256)

Each arm sampled ready, first acquisition, acquisition 200, application disposed, and after a fixed five-second idle. Every checkpoint used the same full-GC / finalizer / full-GC / 250 ms schedule, then a parent-controlled heap/lsof barrier. Matched initial/first/last RSA sign-and-verify checks confirmed private-key use. No iteration called AsX509Certificate2; no network, server or session was opened. AutoAcceptUntrustedCertificates=false stayed configured; this key-acquisition control makes no additional peer-validation wire claim.

Allocation-stack tracking was enabled in both arms, so its allocation/GC cost is part of these observations. The 180-second child cancellation/checkpoint limit is not a strict total wall deadline: a running heap probe (20 seconds), lsof probe (five seconds), final wait and bounded terminate/kill reap can add bounded overhead. Stdout is a file, with no unbounded communicate or pipe drain. The recorded artifact-creation-to-result-write wall interval was 34.699 seconds (filesystem timestamps, not a monotonic acceptance gate). Ready-to-idle checkpoint intervals were 21.504 seconds store and 9.123 seconds registry, including probe barriers. Both process exits were zero and all children were reaped.

Native inventory and ownership results

Checkpoint Store AppleDatabase Registry AppleDatabase Tracked runtime keychains, each arm Outstanding SDK cores, each arm Temporary keychain files, each arm
Ready 3 3 1 1 1
First acquisition 4 3 1 1 1
Acquisition 200 203 3 1 1 1
Application disposed 203 3 0 0 0
Five-second idle 203 3 0 0 0

Configured PKI FDs and temporary-keychain FDs were zero at every checkpoint. One other task-TMPDIR descriptor remained in each process; it is reported separately and is not labelled a keychain descriptor. Total numeric FDs were 110 ready / 112 final in both arms. Manager disposal was observed through an exact-SDK, read-only m_disposed boolean probe. Store certificate-core counters finished 407 created / 407 disposed; registry finished 7 / 7. These public counters concern underlying certificate cores, not every AddRef wrapper.

The exact installed runtime's internal temporary-keychain dictionary was read under the same dictionary monitor, with a one-second bounded acquisition and exact informational-version/type/field guards. Only its count was read; no native pointer, handle, key or certificate material was enumerated or modified. Its count returning zero establishes no remaining entries in that managed lookup. The installed cleanup code removes the entry before calling native SecKeychainDelete, suppresses delete errors, and then releases the CF handle. The actual installed base SafeKeychainHandle.ReleaseHandle was separately decompiled and does call Interop.CoreFoundation.CFRelease(handle), then zeros its handle. Apple's SecKeychainDelete documentation distinguishes file/keychain deletion from releasing keychain-object memory; this inspection finds the separate CFRelease in the installed runtime, not an obvious missing base-release implementation. These are inspected calls, not native return-status instrumentation. Therefore lookup zero does not prove that native deletion succeeded.

Memory observation Store ready → final idle Registry ready → final idle
Managed live bytes 831,080 → 1,222,192 831,128 → 1,130,616
RSS bytes 87,621,632 → 115,130,368 87,654,400 → 97,927,168
Compiled methods 1,192 → 4,638 1,192 → 2,251

RSS includes different JIT, tracking, GC and allocator effects; this comparison is not an exclusive native-byte attribution or a performance budget. The native class's shallow heap bytes were 2,304 → 155,904 in store and 2,304 → 2,304 in registry (768 bytes per classified object). Shallow class bytes exclude owned allocations and do not explain the entire RSS difference. PrivateMemorySize64 returned zero on this runtime and is not used as a private-memory result.

heap --noContent -s reports allocations classified as the C++ Security::AppleDatabase class. Apple's published file-database source declares that class as a Database subclass with database open/close/delete methods and owned file/modifier state; it is not a file-descriptor counter. The published source is conceptual evidence, not a claim that it is the exact source build of this installed Security.framework. Retaining classified heap objects after observed closure is different from proving an open handle, failed native delete, or the identity of the retaining native owner. See the pinned Apple class definition.

The leaf DER SHA-256 was identical before/after and across arms: fadcc11ec694732b82fe4dc47bb41bce6023d113339e53aca9fee5b079f200b2. After samples and process exits, every seed/arm PKI, TMPDIR and data fixture was removed; cleanup receipt is true. Raw native snapshots stay local and private.

Exact SDK path and public API limits

The inspected SDK checkout is the clean 2.0.1 tag at commit 07e62f72500cc0923a2bab940653c8c1116fa286; the control references immutable actual published SDK 2.0.1 DLLs, with all eight hashes in evidence/2026-11-10/provenance.json.

Source, pinned to the exact SDK commit Relevant behavior
DefaultSessionFactory.CreateChannelAsync Loads a private certificate through the internal overload that defaults registry use to false.
DefaultSessionFactory private CreateAsync Constructs the raw Session with null certificate/chain.
Session.LoadInstanceCertificateAsync Opening that Session independently loads its private certificate; registry preference depends on a channel manager, absent in the current raw factory.
Session.LoadInstanceCertificateEntryAsync Documents configured-store selection for unmanaged callers and registry preference for managed callers.
SecurityConfiguration.FindApplicationCertificateAsync Private-key branch calls registry-less CertificateIdentifierResolver.LoadPrivateKeyAsync.
CertificateIdentifierResolver.LoadPrivateKeyCoreAsync Opens the configured store and asks it to load the private key; returned reference is caller-owned.
DirectoryCertificateStore.LoadPrivateKeyAsync For PFX, macOS storage flags omit EphemeralKeySet and include Exportable, then import through X509CertificateLoader.LoadPkcs12. No public per-store private-key cache/flag setting is offered.
ApplicationConfiguration.ValidateAsync, ApplicationInstance.CheckOrCreateCertificateAsync, ServerBase.OnServerStartingAsync Application validation, normal own-certificate checking and server bring-up also load configured private certificates. Checking disposes its temporary certificate; registry replacement disposes old entries.
Certificate.AsX509Certificate2 Nonvirtual export/reimport copy for exportable private keys. Existing isolated controls demonstrate its PFX retention trigger, but prior sampled real-workload stacks identified directory loading as the dominant sampled import path.

CertificateIdentifier has no Certificate preload/cache property and only a public default constructor. Actual-binary public API inventory is retained in seed-api-inventory.jsonl. RawData carries public encoded certificate material; its setter parses Certificate.FromRawData and derives metadata. It cannot carry the reusable private Certificate owner needed by the raw private-key load path. The exact-tag manager documentation explicitly places materialized ownership in the manager registry.

The public registry acquisition API is reference-counted and supported. However, substituting it in the raw session path changes certificate selection semantics: UnmanagedCertificateLoadHonorsAnExplicitConfiguredStoreReplacementAsync requires a configured replacement to win over an older registry certificate and verifies the registry API is never called. This is inspected upstream test source, not a test run here. A cache/factory rewrite must additionally preserve update/removal visibility, cancellation/fault cleanup, concurrent owners, reconnect and certificate/issuer-chain selection; the B arm proves none of those replacement semantics. ChannelManagerSessionFactory changes architecture and was not used. Overriding/reimplementing channel and session construction is not a narrow application disposal fix.

Production source at ed60970fc294ab171a80cda1b57942a7130b4c39 was inspected: OpcUaApplicationHost assigns configuration ownership before async validation and disposes its ApplicationInstance; Shared transfers manager lifetime to the session adapter and disposes failed configurations; driver/browser release their owned ApplicationInstance after session cleanup. No missing owner was identified in those paths. The store arm already keeps its one application/configuration and registry alive across all 200 acquisitions, so simply lengthening that application owner cannot remove these private-key imports. Existing full-client lifecycle disposal checks are complementary coordinator evidence, not additional executions in this control.

Runtime dependency and upstream-ready boundary

Actual installed System.Security.Cryptography.dll informational version: 10.0.14+0427b9abbead3fa64c6655eb4290c537397131fa. SHA-256: f4957d7218a004a27bf8964e45147df6a88092c6e7344582082840d62c6c579a.

The official .NET v10.0.14 tag resolves to 6aa72daa24f66c01c3d7b6a32519e9d21396deec, different from the installed binary's informational commit. It is not presented as exact installed source. Read-only ILSpy inspection of the actual binary verified the temporary-keychain creation, dictionary/monitor and release behavior used by diagnostics. Paths and hashes of those retained local decompilations are in verification.json.

The public runtime source provides matching context for temporary keychain creation during non-persisted import and unsupported EphemeralKeySet and tracked-keychain release ordering. Microsoft documents that macOS PFX private-key loading requires a keychain and cannot use EphemeralKeySet in its cross-platform cryptography guide. Changing those flags, exportability, storage format or native crypto was not an application mitigation attempted here.

The remaining dependency is an upstream/platform investigation of why the classified native file-database objects remain after observed SDK owner release, tracked runtime owner release and backing-file cleanup, and whether there is a supported correction at that boundary. Native delete status and the retaining C++ owner are still unknown. This control does not identify a specific .NET or Apple defect/fix. An upstream SDK improvement would also need to avoid redundant imports without losing the existing raw configured-store semantics. No supported application-only correction meeting the stated constraints was proven.

Private reproduction and evidence

python3 scripts/validation/opcua-lifecycle-memory/native-retention/run.py \
  --references /tmp/opcua2-publish-hg100-20261110/linux-arm64/host \
  --output /tmp/opcua2-native-reload-FRESH

This builds only the standalone control against existing actual SDK DLLs, never rebuilds product code, requires the exact installed runtime diagnostic layout, and refuses an existing output directory. The fixed repetition count is not configurable. Use a private fresh root and serialize with other ownership/native controls. The checked-in selected receipts are in evidence/2026-11-10/; raw artifacts are /tmp/opcua2-native-reload-20261110a. Raw heap/lsof output and any failed-run private fixture material must not be redistributed wholesale.

No long run, deployment, trust weakening, SDK fork/version change, budget change, public upstream post or #799 closure is authorized or claimed by these results.

A separately reviewed plain native Apple empty-keychain control then executed 200 successful create/delete/CFRelease operations and kept native AppleDatabase at 1 throughout. That negative result rules out empty native create/delete/release alone as sufficient here; it does not identify the private-key/certificate import retaining owner.

The subsequent installed native bridge import control reproduced AppleDatabase growth from 1 to 201 in a measured C process with no managed caller. Native deletes returned zero in that control, all returned CF owners balanced, and keychain files/FDs stayed zero. The installed bridge and Security.framework remain in the path; their respective retaining ownership is unresolved. The native delete statuses of this managed A/B remain unobserved.