Files
lmxopcua/scripts/validation/opcua-lifecycle-memory/LATE_SLOPE.md
T
Joseph Doherty da4e27003e
v2-ci / unit-tests (pull_request) Waiting to run
docs: record completed Linux OPC UA lifecycle diagnostic
2026-11-10 20:28:11 -05:00

7.5 KiB
Raw Blame History

Proposed fixed-duration late-slope diagnostic

Design draft only. No new mode, artifact or long execution is approved by this document. Resolve the current ownership/code gates and review native attribution before deciding whether this experiment supplies a still-needed fact for #799.

The question is whether the canonical Linux process continues to grow well after the observed generated-code step. This is separate from choosing a warmed acceptance baseline, a cross-version performance denominator, or resolving macOS private-key import retention. The existing profile allowance stays 60–1800s.

If approved, add a distinct diagnostic mode fixed at 7200 seconds of canonical cycling after startup, including its cold and qualification-era cycles. It must not claim 7200 seconds after warmup. Keep every normal operation, the active restart every fourth cycle, all logical/ownership checks and actual PKI descriptor checks. Use fresh committed source and immutable self-contained Linux ARM64 .NET10.0.14 artifacts, exact SDK2.0.1 and the landed HG1.0.0 graph. Freeze image, PKI identity, fixture and instrumentation before launch. Root owns the long run.

Keep the original cycle-3 baseline and +64MiB managed / +128MiB RSS / +32 FD caps. Record the sample before asserting; catch only the three resource-cap assertions, latch the first rejection permanently and continue every canonical cycle and restart, with resource sampling. Managed-monotonic, logical/ownership, observer, operation and shutdown failures remain fatal. After final ownership/shutdown checks, emit a diagnostic receipt with acceptance=false and rethrow the original cap rejection. Never emit ordinary complete/two-hour acceptance or convert an earlier rejection to success. A duration completion is distinct from test success.

Prospectively report all quiescent observations, cold rejection, maximum observed quiescent RSS and final values. Predeclare these elapsed-time windows:

Window Quiescent sample elapsed seconds
Early late phase 1800–2400
Middle late phase 3600–4200
Final late phase 6600–7200

Proposed fixed bounds are startup 120s, each complete cycle including its restart 30s, cycle deadline 7230s, shutdown ceiling 60s and container deadline 7440s. The active restart helper has a separate 25s linked deadline; its elapsed time is recorded before post-restart configuration. The final cycle may finish after 7200s, but cannot extend its 30s bound. Validate these bounds with reviewed short controls before committing the protocol; hitting one makes the diagnostic incomplete, not grounds to relax it.

The existing quiescent sample precedes the optional restart. Thus two adjacent bounded cycles allow at most 60s between quiescent samples. Each 600s window must have at least 10 samples, first sample no later than 60s after its start, last sample no earlier than 60s before its end, observed span at least 480s, and no internal gap above 60s. Apply the same 60s maximum gap over the entire 1800–7200s reporting range. Missing coverage is incomplete evidence; it must not become a favorable slope. Unless at least 385 cycles and 96 restarts complete by 1800s, report the elapsed slopes descriptively but withhold attribution to the phase after the previously observed restart milestone. Reaching that count still does not prove factory saturation in this run; no EventPipe observer is attached.

For each window report sample count, first/last/min/max/median RSS and retained managed bytes, FD range, GC committed range, JIT/timer/thread ranges and elapsed coverage. Missing windows are incomplete evidence. Also report ordinary least squares RSS and managed-byte slopes against actual sample elapsed seconds for all observations in 1800–7200s, with units bytes/hour and the exact sample count. Do not choose a favorable endpoint/window/statistic after observing results. These descriptive slopes have no new pass threshold. Quiescent maxima are not continuous process peak measurements, and native MethodSize is not RSS.

Retain certificate allocation StackTrace tracking at the same enabled setting as the earlier canonical saturation experiment and disclose its overhead. Do not attach EventPipe to this proposed run or claim generated-factory readiness from absent counters; earlier traces remain separate source-scoped evidence. This instrumented diagnostic cannot be relabelled as uninstrumented acceptance or as one of the required representative-load comparison runs.

Freeze cycle-phase elapsed timestamps before cleanup and retain shutdown duration separately; no cleanup sample feeds the windows or slopes. The new mode must reach its explicit post-shutdown checks before rethrowing a latched resource cap; the generic profile mode currently throws earlier and must keep its old behavior. Cleanup must attempt both independent owners under their separate deadlines even when one fails or the operation token is canceled. Preserve the first rejection alongside any later cleanup failure; neither error may erase the other. The shutdown ceiling covers 5s to capture manager references, 20s for each owner, 5s to delete the private client directory and 10s to verify shutdown. Any local deadline or overall elapsed miss is fatal and makes the diagnostic incomplete; the container deadline is unchanged.

Before execution, implement and review the proposed setup/last-cycle/cleanup and outer deadlines in the actual harness and launcher; no extension or retry-to-green. Cycle cancellation is cooperative. If a call ignores its token and does not return within the bound, the diagnostic is incomplete; the outer deadline reaps the task-owned container. Never infer a completed bounded cycle from silence. Require negative controls proving a latched first failure cannot disappear, wrong duration/mode is rejected, no ordinary acceptance receipt can be emitted, and malformed/missing sample windows fail the summarizer. Review actual source, control results and every immutable artifact hash with Astra before launch.

The canonical mode is implemented. The short control receipts record three passing helper tests and actual rejection of a 90-second request before setup. The original helper assertion-capture failure is retained; the canonical source is byte-identical across that test-only correction. These controls do not approve a bundle, start the workload or satisfy #799.

The first bundle verified but its separate rehearsal stopped in .NET CLI startup: retained startup failure. The corrected launcher keeps the root read-only and aliases the same private task temp directory at /tmp. The replacement bundle from b18f9723 passed manifest/source/MVID review and its separate 60-second rehearsal completed 16 cycles and four restarts with all three tests passed and cleanup verified. Astra approved one fixed 7,200-second diagnostic on that exact bundle; this approval does not adopt a new acceptance baseline or clear #799/G4.

The fixed run result completed all 7,201.014 seconds of cycling and verified shutdown, with 1,937 cycles, 484 restarts and no cold resource rejection. Peak quiescent RSS was 4.20 MiB below the original cap. The three fixed-window medians and whole late phase fit describe one tracked Linux run. Earlier failures remain, the receipt has acceptance=false, and #799/G4 stay open.