test(loadharness): retain raw resource samples; record findings F4 and the discarded run
HarnessRunResult now carries every ResourceSample, not just the window summaries.
Motivated by finding F1: the definitive run recorded ZERO gen-2 collections across
45M events, so a positive least-squares heap slope cannot be told apart from
gen-2 garbage that was simply never collected. The summary alone cannot settle
that; the series can. The reported run predates this field -- noted as such in the
results doc rather than implied otherwise.
Also records:
- the second full-scale run was DISCARDED, not reported: a verification build
overlapped the start of its measurement window, and a contaminated measurement
is not evidence.
- finding F4, a pre-existing test-isolation flake in
QueueDepthGaugeTests.Gauge_TracksBufferedDepth_AcrossEnqueueDrainAndPark
(fails in a full-suite run, passes in isolation -- shared static gauge carrying
state across tests). It cannot originate here: this branch changes zero src/
files vs its base 986e6e7a. Left unfixed deliberately; filed for separate triage.
Verified: full slnx build clean; SiteRuntime 604/604, Communication 691/691 pass;
TargetScaleHarnessSmokeTests passes (78s).
This commit is contained in:
@@ -26,9 +26,13 @@
|
||||
| Measurement window | 20 minutes steady state, after a 2-minute settle |
|
||||
| Total wall time | 1,510 s (~25 min) |
|
||||
|
||||
Two independent full-scale runs were executed. **Run 1** is the primary dataset.
|
||||
**Run 2** re-ran the identical protocol with raw per-sample resource capture added
|
||||
(the only harness change between them) to resolve the memory question in §4.
|
||||
**One clean full-scale run is the dataset reported here.** A second run was started to
|
||||
add raw per-sample resource capture (sharpening finding F1), but a verification build
|
||||
overlapped the start of its measurement window; it was **discarded rather than
|
||||
reported**, per the rule that a measurement contaminated by concurrent load is not
|
||||
evidence. The raw-sample capture it was meant to exercise is now in the harness
|
||||
(`HarnessRunResult.ResourceSamples`) and will be present in any future run, including
|
||||
the 1-hour run in §7 — the run reported below predates that field.
|
||||
|
||||
---
|
||||
|
||||
@@ -224,6 +228,23 @@ limit on burst absorption. The probe paces at 2,000 events/s to isolate the vari
|
||||
under test; the shared stage's own capacity is separately covered by
|
||||
`SiteStreamThroughputTests`.
|
||||
|
||||
### F4 — Pre-existing test-isolation flake in `QueueDepthGaugeTests` (Low, not introduced here)
|
||||
|
||||
Noticed during this work package's verification pass, recorded so it is not later
|
||||
mistaken for a regression from the load-harness branch.
|
||||
|
||||
`StoreAndForward.Tests.QueueDepthGaugeTests.Gauge_TracksBufferedDepth_AcrossEnqueueDrainAndPark`
|
||||
**fails in a full-suite run** (`Expected: 0, Actual: 2`, `QueueDepthGaugeTests.cs:116`)
|
||||
but **passes when run in isolation** — the signature of a shared static gauge carrying
|
||||
state across tests, i.e. an ordering dependency rather than a product defect.
|
||||
|
||||
It cannot originate from this branch: the branch changes **zero `src/` files** relative
|
||||
to its base (`986e6e7a`), and both the StoreAndForward source and its test project are
|
||||
byte-identical to that base. Left unfixed deliberately — out of scope for a measurement
|
||||
work package, and the brief was explicitly not to attempt unrelated fixes. Worth a
|
||||
separate triage alongside the pre-existing `SandboxTests` timing pin already recorded as
|
||||
residual 6 of the arch-review remediation execution log.
|
||||
|
||||
---
|
||||
|
||||
## 5. What this does *not* prove
|
||||
|
||||
Reference in New Issue
Block a user