Merge branch 'target-scale-load-test' — deferred register #25 delivered: 375k-subscription target-scale load test with measured evidence (absorbs row 50)

2026-08-15 03:01:51 -04:00
parent fdc76a087e 94e8301e44
commit 40eff637a1
22 changed files with 2996 additions and 18 deletions
@@ -115,9 +115,20 @@ Deliberately not fixed in this program — each has a stated reason, not an over
   library's wire-compatibility guarantee — not a blocker for this program.
 6. **Fragile `SandboxTests` timing pin.** Pre-existing, unrelated to this remediation's changes;
   noted so it isn't mistaken for a regression if it flakes later.
-7. **Target-scale load test (deferred-work register #25).** This program's exit criterion is the
-   live probes above, not #25 — #25 remains the follow-on validation that the moved ceilings hold
-   under real load; schedule separately.
+7. ~~**Target-scale load test (deferred-work register #25).**~~ **RESOLVED 2026-08-15.** The
+   follow-on validation ran: 10 sites x 500 instances x 75 tags = **375,000 live tag
+   subscriptions**, 37,518 updates/s achieved vs 37,500 nominal, **45,021,375 updates** over a
+   20-minute steady-state window. The moved ceilings hold with room to spare — end-to-end tag
+   latency **P99 4.57 ms**, zero dropped events at 100 live subscribers, health report
+   collect+ingest **P99 0.31 ms**, debug view **P99 2.19 ms** with no timeouts, CPU **2.9% of the
+   box**. 11 clean passes, 1 pass with a caveat (F1: 20 minutes with zero gen-2 collections cannot
+   fully settle the leak question; the 1-hour run would), 0 failures. Register **row 50** (S&F
+   drain rate + per-subscriber stream backpressure) was absorbed into the same run and closed with
+   it — S&F drains at **3,533 msg/s**, and a stalled subscriber costs healthy peers **nothing**
+   (100.00%, zero drops). Results: `docs/plans/2026-08-15-target-scale-load-test-results.md`;
+   design: `docs/plans/2026-08-15-target-scale-load-test-design.md`; harness:
+   `tests/ZB.MOM.WW.ScadaBridge.LoadHarness/`. Residual: WP-4's `[xc-3]`/`[xc-4]`/`[xc-5]`/`[xc-8]`
+   are central-cluster/browser concerns outside this harness and remain open scope.
 8. **Playwright 14 pre-existing env failures.** Present on `main` too, rig-state related, not
   introduced by this branch.
 9. **`site_events` retention purge still oplog-visible.** WP3.2's sliced retention DELETE is a row