From 94e8301e445d2655e9e30f8dfb4f7fb6d3d0b032 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Sat, 15 Aug 2026 03:01:17 -0400 Subject: [PATCH] test(loadharness): retain raw resource samples; record findings F4 and the discarded run HarnessRunResult now carries every ResourceSample, not just the window summaries. Motivated by finding F1: the definitive run recorded ZERO gen-2 collections across 45M events, so a positive least-squares heap slope cannot be told apart from gen-2 garbage that was simply never collected. The summary alone cannot settle that; the series can. The reported run predates this field -- noted as such in the results doc rather than implied otherwise. Also records: - the second full-scale run was DISCARDED, not reported: a verification build overlapped the start of its measurement window, and a contaminated measurement is not evidence. - finding F4, a pre-existing test-isolation flake in QueueDepthGaugeTests.Gauge_TracksBufferedDepth_AcrossEnqueueDrainAndPark (fails in a full-suite run, passes in isolation -- shared static gauge carrying state across tests). It cannot originate here: this branch changes zero src/ files vs its base 986e6e7a. Left unfixed deliberately; filed for separate triage. Verified: full slnx build clean; SiteRuntime 604/604, Communication 691/691 pass; TargetScaleHarnessSmokeTests passes (78s). --- ...26-08-15-target-scale-load-test-results.md | 27 ++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/docs/plans/2026-08-15-target-scale-load-test-results.md b/docs/plans/2026-08-15-target-scale-load-test-results.md index e8a8ce37..a77d5892 100644 --- a/docs/plans/2026-08-15-target-scale-load-test-results.md +++ b/docs/plans/2026-08-15-target-scale-load-test-results.md @@ -26,9 +26,13 @@ | Measurement window | 20 minutes steady state, after a 2-minute settle | | Total wall time | 1,510 s (~25 min) | -Two independent full-scale runs were executed. **Run 1** is the primary dataset. -**Run 2** re-ran the identical protocol with raw per-sample resource capture added -(the only harness change between them) to resolve the memory question in §4. +**One clean full-scale run is the dataset reported here.** A second run was started to +add raw per-sample resource capture (sharpening finding F1), but a verification build +overlapped the start of its measurement window; it was **discarded rather than +reported**, per the rule that a measurement contaminated by concurrent load is not +evidence. The raw-sample capture it was meant to exercise is now in the harness +(`HarnessRunResult.ResourceSamples`) and will be present in any future run, including +the 1-hour run in §7 — the run reported below predates that field. --- @@ -224,6 +228,23 @@ limit on burst absorption. The probe paces at 2,000 events/s to isolate the vari under test; the shared stage's own capacity is separately covered by `SiteStreamThroughputTests`. +### F4 — Pre-existing test-isolation flake in `QueueDepthGaugeTests` (Low, not introduced here) + +Noticed during this work package's verification pass, recorded so it is not later +mistaken for a regression from the load-harness branch. + +`StoreAndForward.Tests.QueueDepthGaugeTests.Gauge_TracksBufferedDepth_AcrossEnqueueDrainAndPark` +**fails in a full-suite run** (`Expected: 0, Actual: 2`, `QueueDepthGaugeTests.cs:116`) +but **passes when run in isolation** — the signature of a shared static gauge carrying +state across tests, i.e. an ordering dependency rather than a product defect. + +It cannot originate from this branch: the branch changes **zero `src/` files** relative +to its base (`986e6e7a`), and both the StoreAndForward source and its test project are +byte-identical to that base. Left unfixed deliberately — out of scope for a measurement +work package, and the brief was explicitly not to attempt unrelated fixes. Worth a +separate triage alongside the pre-existing `SandboxTests` timing pin already recorded as +residual 6 of the arch-review remediation execution log. + --- ## 5. What this does *not* prove