diff --git a/docs/plans/2026-08-17-perf-followups-r2.md b/docs/plans/2026-08-17-perf-followups-r2.md index c89145ae..3ad01266 100644 --- a/docs/plans/2026-08-17-perf-followups-r2.md +++ b/docs/plans/2026-08-17-perf-followups-r2.md @@ -394,3 +394,196 @@ below is how the next occurrence gets diagnosed. - Implementers: Opus (user preference), including the trivial task for uniformity. - Reviewers judge via `git show `; file-disjoint scheduling makes on-disk fallback acceptable as in round 1. + +--- + +## Outcomes + +*Written at close-out, 2026-08-17, on branch `perf/followups-r2`. All 8 tasks complete. This +round is a **follow-up round**: every item it closes was opened by the round-1 register in +`2026-08-17-perf-followups.md`, and that register is annotated in place with the closing SHAs.* + +### Sweep record + +`dotnet build ZB.MOM.WW.OtOpcUa.slnx` → **0 errors**, 904 warnings — the same two pre-existing +families round 1 recorded (`xUnit1051` cancellation-token advice and the repo's own +`OTOPCUA0001` CapabilityInvoker analyzer firing on test call sites), up 8 from round 1's 896 +because this round's new test call sites are more of the same. No new warning *kind*. + +Full unit sweep — every `*.Tests` project in the solution, run per-project with `--no-build` +against the tree at `580b77ce`. `*.IntegrationTests` are build-only by convention; the S7 suite +ran separately as Task 3's live gate (below). + +**43 / 43 test projects green (exit code 0 on every one).** + +| | Round 1 | **This round** | Δ | +|---|---|---|---| +| Passed | 10 358 | **10 370** | **+12** | +| Failed | 0 | **0** | — | +| Skipped | 19 | **19** | — | +| **Total** | 10 377 | **10 389** | +12 | + +**No test failed, so no isolation re-run was needed, and nothing flaked.** The +12 reconciles +exactly against the three suites this round added tests to, with nothing unaccounted for: + +| Project | Round 1 | This round | Δ | Source | +|---|---|---|---|---| +| `OpcUaServer.Tests` | 476 | **481** | +5 | `SessionAclDecisionCacheTests` 12 → **17** (T1) | +| `Driver.S7.Tests` | 397 | **401** | +4 | `S7SpanReadParityTests` 9 → **13** (T2) | +| `AdminUI.Tests` | 1 537 | **1 540** | +3 | `SqlTestDatabaseReaperTests` (T5) | + +⚠️ **Note the S7 delta is +4, not the +5 the plan estimated** — the plan's task text counted +five *contracts* pinned, which landed as four test methods (one method carries two of them). +T4 added no tests (it reworked 3 of the 5 existing recorder tests in place), and the S7 +**integration** suite's +2 do not appear here because it is build-only in the sweep. + +**All 19 skips are the pre-existing, env-gated ones round 1 recorded, project for project** — +`Core.Tests` (1), `Driver.Galaxy.Tests` (8), `Driver.Historian.Gateway.Tests` (6, the +`LiveIntegration` category needing the `wonder-sql-vd03` VPN), `OpcUaServer.Tests` (1), +`Runtime.Tests` (3). **This round added no skips.** + +The DB-backed suites ran **against a reachable** `10.100.0.35,14330` — executed, not skipped: +`AdminUI.Tests` **1540 passed / 0 skipped** and `Configuration.Tests` **190 / 0 skipped**. That +zero is the load-bearing number for T5: a reaper suite that skipped itself would have proved +nothing about the DDL it runs as `sa`. + +Two verifications outside the sweep proper, because two comment-only fixes (below) landed after +the sweep's build: the solution rebuilt **0 errors**, and `OpcUaServer.Tests` re-ran +**481 / 0 / 1**, unchanged. + +### Per-task outcomes + +| # | Task | Commit(s) | Outcome + deviation | +|---|---|---|---| +| 1 | ACL generation stamp on `Shard` + eviction capacity hint | `1979cf41`, `98d6a190` | Done, **and the review found a second, pre-existing defect that was closed in-round.** The generation is now stamped on `Shard` (ctor + readonly `Generation`); `TryGet` is generation-gated and **non-mutating**; `Set` displaces a mismatched shard and **drops the verdict** if it is still mismatched after the displacement. The eviction capacity hint is gated on the generation matching — a previous generation's fill level is not evidence about this one. `98d6a190` added a monotonic `<=` guard to `InvalidateIfStale` after the reviewer spotted that a stale-generation straggler could roll the stamp *backward* and thrash `ClearAll`. **Falsifications:** removing the stamp check went **deterministically red** on the pin *and* produced **1 357 cross-generation hits** in the stress test; reverting the `<=` guard to `==` rolled the stamp **2 → 1**. 17/17. | +| 2 | S7 cancellation branches + one-bad-decode + shape mismatch | `4f860733`, `120e885b` | Done. The span and multi-var **cancellation reopen** contracts are pinned (hang knobs + entered-TCS + `CreatedPlcCount`), plus one-bad-decode-in-chunk (only that tag goes Bad, **no** fallback) and the short-answer shape mismatch (**whole-chunk** fallback). `120e885b` strengthened the pins to assert the stale handle is **disposed**, not merely replaced. **Falsification:** each `OperationCanceledException` arm was weakened in turn and **exactly the matching test** went red — no arm was covered by a neighbour's test. 13/13. | +| 3 | Timer/Counter live pin — TM/CT area in the `s7_1500` fixture | `fd202d81`, `f797df51` | **LIVE PIN PASSED** vs `10.100.0.35:1102`. Evidence + the fixture quirk below. | +| 4 | Historian recorder tests — bounded watches | `45d67092`, `68461fbf` | Done; three fixed sleeps → bounded watches. Detail below. | +| 5 | Orphaned test-database reaper | `8946644b`, `580b77ce` | Done — **and the code review found a real Critical, fixed pre-merge.** Detail below. | +| 6 | MTConnect `AsStream` ownership comment | `1edfc263` | Done as specified; comment-only, no test. Trivial ⇒ no review. | +| 7 | docker-dev readiness-gated `deploy.sh` + Traefik access log | `a61b3c98`, `9f09a62a` | Done as a **gate plus instrumentation, not a diagnosis** — say so plainly. Detail below. | +| 8 | Close-out — sweep + Outcomes + register annotations | `d54b4fab`, *this commit* | Done. `d54b4fab` carries the two comment-only nits the T1 re-review left for close-out; this commit carries the Outcomes, the sweep record, the round-1 register annotations, and the ledger. | + +### Live pin — S7 Timer/Counter multi-var exclusion (Task 3) + +**2026-08-17, against the `s7_1500` fixture at `10.100.0.35:1102`. PASSED — 4 passed / 1 +env-gated skip.** + +TM and CT areas were added to the fixture profile: **TM0** (`T0 = 0x2456` = 456.0 s, +`T5 = 0x1123` = 12.3 s, `T9`) and **CT0** (`C0 = 4711`, `C7 = 31337`, `C11`). + +- **The claim under test, asserted on real wire shapes:** a batch mixing Timer/Counter tags with + ordinary ones produced **1 block read + 1 multi-var request carrying 3 scalar items + one + 2-byte TM read + one 2-byte CT read**, and `MultiVarItemAreas` **never contains Timer or + Counter**. The round-1 exclusion — added because S7.Net boxes TM/CT as `int` where the + single-read path yields `ushort`, and box parity is load-bearing downstream — now rests on the + wire, not on unit tests alone. +- ⚠️ **Fixture quirk, discovered here and documented with a recaptured live probe transcript:** + python-snap7's server resolves a TM/CT request address as a **BIT address** (it serves byte + `addr >> 3`) where S7.Net and a real CPU send the **device number**. **Device 0 is the fixed + point of both conventions**, which is why value decode is live-pinned at **T0/C0**. +- The device-number leg (T5/C7) is therefore **env-gated on `S7_TIMER_NUMBER_ADDRESSING=1`** — + it is the single skip above, and it is a **ready-made real-CPU gate** rather than a silent + omission. **Not closed by this round:** TM/CT decode at a non-zero device number still needs a + real CPU. + +### Task 4 — what replaced each sleep, and the one place a pass clock does not exist + +Three fixed sleeps became bounded watches on **real observables**, not shorter sleeps: + +- **Shutdown test:** a 25 ms/1 s **negative** watch on the new `RemoveCallCount` and the queue + depth. The 1 s cap is not a guess — it is anchored to the **measured ~150 ms** at which the + falsification tripped, so the cap is roughly 6× the observed signal rather than an arbitrary + round number. `68461fbf` also reduced it to a **single counter read per poll** (two reads per + poll can straddle an increment and compare inconsistent values). +- **Poison-tag test:** a positive watch on `PeekBatchCallCount ≥ 3` — a genuine pass clock. +- ⚠️ **Sequential test: a 25 ms/500 ms negative watch, and this is a disclosed deviation.** A pass + clock is **structurally unavailable** there: the drain is single-flight on `_draining` and the + timer re-arms **only** in `OnDrainResult`, so there is no counter that advances while the + condition under test is false. Verified against the production code and **accepted** rather than + worked around with a fake observable. + +**Falsification:** tripped on `RemoveCallCount` in ~150 ms. 5× stability green. + +### Task 5 — the reaper's Critical, and why the live evidence is shaped the way it is + +The startup reaper drops test databases orphaned **> 24 h** (AdminUI + Configuration prefixes, +per fixture, one-shot `Lazy`, probe-gated on the AdminUI side). + +- ⚠️ **The code review found a real Critical.** The name-shape guard validated only the prefix and + the timestamp, so **any** `..._<14 digits>_` database name on the server would flow + into string-interpolated `ALTER`/`DROP` DDL executed **as `sa`**. Fixed **defense in depth**: + a `Guid.TryParseExact("N")` suffix validation *and* parameterized `QUOTENAME`/`sp_executesql`. + ⚠️ The reviewer's proposed `EXEC('…' + QUOTENAME(@name))` snippet was **itself invalid T-SQL** + (verified live — Msg 102); the accepted fix is not the one that was suggested. +- Also landed: a **3 s reaper `ConnectTimeout`** (a reaper must never be the thing that hangs a + test run), `DatabaseName` minted from the prefix constant rather than re-spelled, the catch + widened to `Exception`, and the reaper tests **joined to the AdminUI SQL collection** — closing + a same-process parallel-collection race that earlier 2-test runs had masked. +- **Live evidence:** planted orphans were reaped **by exactly the right fixture**; non-GUID and + unparseable names were **spared**; the shared server was restored afterwards. 3/3 reaper, + **1540/1540 AdminUI ×3**, 190/190 Configuration. + +### Task 7 — what the docker-dev change does and does not establish + +`deploy.sh` polls `/health/ready` (2 s interval, 120 s cap) and then POSTs **once, un-retried** +(both curls guarded under `set -e`, `--max-time 10/60`), and Traefik now runs `--accesslog`. + +⚠️ **The 404's mechanism is still unconfirmed.** What is *established* is the elimination: the +AdminUI cannot produce it (routes map synchronously before `RunAsync`; the endpoint's own +failure modes are 503/401/422), and Traefik's backend-down answer is 503 `no available server`, +not 404. So a 404 means **no router matched at that instant** — and the access log is how the +**next** occurrence gets named. The gate makes scripted deploys reliable; it does not explain +the bug. + +### Review chain + +7 implementers; **11 review verdicts** (T1 spec + code + delta re-review, T2 spec + code, T3 +spec + code, T4 spec + code, T5 spec + code + re-review, T7 code + re-review); **4 fix rounds** +(T1, T3, T5, T7, plus minors on T2/T4); **1 Critical found and fixed pre-merge** (T5's DDL +injection); **zero findings papered over**. + +Two **falsification-hygiene** lessons from this round, both worth carrying forward: + +- **A falsification must be run against a freshly built binary.** A `when (false)` mutation + combined with a `--no-build` run executed the *stale* binary and reported a vacuous green — + the mutation "passed" without ever having been compiled. +- **A green falsification arm proves nothing until you have seen its red.** T1's `Set`-displacement + behaviour was **unpinned** until a test written specifically for it went red; the existing + tests all passed with the displacement removed. + +### Follow-up register (opened by this round) + +Only genuinely **new or still-open** items. Everything else this round touched is annotated as +closed on the round-1 register. **None blocks the branch.** + +**ACL** + +- **The single-shot CAS can defer stamp publication by one call.** When generation *N* wins the + race for a shared baseline against generation *N+1*, publication of the newer stamp is deferred + one call. **Pre-existing and benign** — it self-heals on the next call and **no verdict is + affected** (a deferred stamp cannot serve a stale grant; the generation gate on `TryGet` is what + decides that, and it is unaffected). Recorded so the next reader does not re-derive it as a bug. + +**S7** + +- **Per-item decode health is last-writer-wins.** Pre-existing; now **pinned and documented** + rather than fixed, because changing it is a behaviour change to how a mixed-outcome chunk + reports driver health, not a test gap. +- **TM/CT decode at a non-zero device number still needs a real CPU.** The test exists and the + env var (`S7_TIMER_NUMBER_ADDRESSING=1`) is ready; python-snap7 cannot serve it (see the + BIT-address quirk above). + +**docker-dev** + +- **The first-POST 404's mechanism is unconfirmed** until the new access log catches an + occurrence. The gate prevents the symptom; the cause is still open. +- **The no-body `POST` `Content-Type` binding is unverified against a live rig** — carried as a + header NOTE in `deploy.sh` rather than silently assumed correct. + +**SQL test infrastructure** + +- **A cross-*process* plant-vs-reap race is inherent to a shared, unlocked server** and is + **accepted**: two test processes racing on the same instance can have one reap what the other + just planted. The 24 h age floor makes it vanishingly unlikely in practice, and the alternative + (a lock table on a shared dev server) costs more than it buys. diff --git a/docs/plans/2026-08-17-perf-followups-r2.md.tasks.json b/docs/plans/2026-08-17-perf-followups-r2.md.tasks.json index c1a9ecd1..7a5cb498 100644 --- a/docs/plans/2026-08-17-perf-followups-r2.md.tasks.json +++ b/docs/plans/2026-08-17-perf-followups-r2.md.tasks.json @@ -1,14 +1,15 @@ { "planPath": "docs/plans/2026-08-17-perf-followups-r2.md", "tasks": [ - {"id": 1, "subject": "Task 1: ACL generation stamp on Shard + eviction capacity hint", "status": "pending"}, - {"id": 2, "subject": "Task 2: S7 cancellation branches + one-bad-decode-in-chunk + shape mismatch", "status": "pending"}, - {"id": 3, "subject": "Task 3: Timer/Counter live pin — TM/CT area in the s7_1500 fixture", "status": "pending"}, - {"id": 4, "subject": "Task 4: Historian recorder tests — bounded watches", "status": "pending"}, - {"id": 5, "subject": "Task 5: Orphaned test-database reaper (AdminUI + Configuration)", "status": "pending"}, - {"id": 6, "subject": "Task 6: MTConnect AsStream ownership comment", "status": "pending"}, - {"id": 7, "subject": "Task 7: docker-dev readiness-gated deploy.sh + Traefik access log", "status": "pending"}, - {"id": 8, "subject": "Task 8: Close-out — sweep + Outcomes + register annotations", "status": "pending", "blockedBy": [1, 2, 3, 4, 5, 6, 7]} + {"id": 1, "subject": "Task 1: ACL generation stamp on Shard + eviction capacity hint", "status": "completed", "note": "1979cf41 + follow-up 98d6a190 (monotonic <= guard on InvalidateIfStale closing the reviewer-found backward-rollback thrash; capacity hint gated on generation match); spec ✅ + code APPROVED + delta re-APPROVED; falsifications: stamp-check removal = 1357 cross-generation hits in stress; Set-displacement was unpinned until its own test; == revert = stamp rolled 2->1. Two cosmetic comment nits left for close-out; known residual (pre-existing, benign): single-shot CAS can defer publication one call when gen3 beats gen4 to the same baseline — self-heals, verdicts unaffected"}, + {"id": 2, "subject": "Task 2: S7 cancellation branches + one-bad-decode-in-chunk + shape mismatch", "status": "completed", "note": "4f860733 + fix 120e885b (Disposed assertion on the stale handle); spec ✅ + code APPROVED; falsification bit precisely on both OCE arms; noticed-not-fixed: per-item decode loop health is last-writer-wins (pinned + documented, pre-existing)"}, + {"id": 3, "subject": "Task 3: Timer/Counter live pin — TM/CT area in the s7_1500 fixture", "status": "completed", "note": "fd202d81 + fix f797df51 (README probe transcript recaptured against the shipped profile; CountingPlcFactory thread-safety remark); GATE: exclusion live-pinned at device 0 vs 10.100.0.35:1102 (snap7 resolves TM/CT addr as BIT address, byte=addr>>3; device 0 is the fixed point of both conventions); T5/C7 device-number test env-gated on S7_TIMER_NUMBER_ADDRESSING=1 as the real-CPU gate; spec ✅ + code review findings fixed; 4 passed / 1 env-gated skip live"}, + {"id": 4, "subject": "Task 4: Historian recorder tests — bounded watches", "status": "completed", "note": "45d67092 + fix 68461fbf (cap 2s->1s with measured ~150ms trip anchor; single counter read per poll); spec ✅ (concurrency-test deviation verified against production and accepted) + code APPROVED; falsification tripped on RemoveCallCount in 150ms; noticed-not-fixed: :1271 ExpectNoMsg is TestKit idiom, left alone"}, + {"id": 5, "subject": "Task 5: Orphaned test-database reaper (AdminUI + Configuration)", "status": "completed", "note": "8946644b + fix 580b77ce; spec ✅; code review found a real Critical (unvalidated suffix into interpolated DROP DDL as sa) — fixed defense-in-depth (Guid.TryParseExact shape guard + parameterized QUOTENAME/sp_executesql; reviewer's EXEC() snippet was invalid T-SQL, verified live Msg 102) + short reaper ConnectTimeout + prefix-constant minting + catch widened + collection-join closing a masked parallel-collection race; re-review APPROVED; live: 3/3 reaper, 1540/1540 AdminUI x3, 190/190 Config, shared server restored; known residual: cross-PROCESS plant-vs-reap race inherent to shared server, accepted"}, + {"id": 6, "subject": "Task 6: MTConnect AsStream ownership comment", "status": "completed", "note": "1edfc263; comment-only, build clean; trivial => no review"}, + {"id": 7, "subject": "Task 7: docker-dev readiness-gated deploy.sh + Traefik access log", "status": "completed", "note": "a61b3c98 + fix 9f09a62a (guarded POST curl under set -e, --max-time 10/60); re-review APPROVED; minor deferred as header NOTE: no-body Content-Type binding unverified against a live rig"}, + {"id": 8, "subject": "Task 8: Close-out — sweep + Outcomes + register annotations", "status": "completed", "blockedBy": [1, 2, 3, 4, 5, 6, 7], "note": "Recorded in the following commit (the docs close-out: Outcomes + sweep record on the r2 plan, append-only 'Closed by (r2)' annotations on the round-1 register, this ledger). Preceded by d54b4fab, the two comment-only nits the T1 re-review deferred here. Build 0 errors / 904 warnings (round-1 families, +8 from this round's new test call sites). Sweep 43/43 projects green: 10370 passed / 0 failed / 19 skipped (10389 total) — +12 over round 1's 10358, reconciling exactly to OpcUaServer.Tests +5 (SessionAclDecisionCacheTests 12->17), Driver.S7.Tests +4 (S7SpanReadParityTests 9->13; the plan's '+5' counted contracts, which landed as 4 methods), AdminUI.Tests +3 (SqlTestDatabaseReaperTests). All 19 skips pre-existing and env-gated, project for project; this round added none. DB-backed suites RAN against 10.100.0.35,14330 — AdminUI 1540/0 skipped, Configuration 190/0 skipped. No failures, no flakes, no isolation re-run needed. S7 integration live vs 10.100.0.35:1102: 4 passed / 1 env-gated skip (S7_TIMER_NUMBER_ADDRESSING device-number leg). Post-edit re-verification: solution rebuild 0 errors, OpcUaServer.Tests 481/0/1 unchanged"} ], - "lastUpdated": "2026-08-17T16:00:00Z" + "closeOut": "Round 2 (perf/followups-r2) complete: 7 implementation tasks + close-out, all 8 green. Every item closed here was opened by the round-1 register, which is annotated in place with closing SHAs. Landed: T1 ACL generation stamp on Shard + eviction capacity hint + a reviewer-found backward-rollback thrash closed in-round (1979cf41, 98d6a190); T2 S7 cancellation-reopen / one-bad-decode / shape-mismatch pins (4f860733, 120e885b); T3 Timer/Counter exclusion LIVE-PINNED on a TM/CT-seeded s7_1500 fixture (fd202d81, f797df51); T4 three historian hope-waits -> bounded watches (45d67092, 68461fbf); T5 orphaned-test-DB reaper, including a real Critical (unvalidated name into interpolated sa DDL) found by review and fixed pre-merge (8946644b, 580b77ce); T6 MTConnect AsStream ownership comment (1edfc263); T7 readiness-gated docker-dev deploy.sh + Traefik access log (a61b3c98, 9f09a62a); T8 close-out (d54b4fab + the docs commit). Review chain: 7 implementers, 11 review verdicts, 4 fix rounds, 1 Critical fixed pre-merge, zero findings papered over. Two falsification-hygiene lessons recorded: a --no-build falsification runs a stale binary and reports a vacuous green, and a falsification arm proves nothing until its red has been seen. Five items stay open in the r2 register, none blocking: the benign ACL single-shot-CAS deferred-publication residual; S7 per-item decode health last-writer-wins; TM/CT decode at a non-zero device number (needs a real CPU, env var ready); the docker-dev 404 mechanism (unconfirmed until the access log catches one) and its unverified no-body POST Content-Type binding; the accepted cross-process plant-vs-reap race on the shared SQL server.", + "lastUpdated": "2026-08-17T18:30:00Z" } diff --git a/docs/plans/2026-08-17-perf-followups.md b/docs/plans/2026-08-17-perf-followups.md index 317119b3..95bc7a20 100644 --- a/docs/plans/2026-08-17-perf-followups.md +++ b/docs/plans/2026-08-17-perf-followups.md @@ -745,36 +745,67 @@ Open items this branch surfaced but deliberately did not close. **None blocks th microseconds and **key revocation/expiry are unaffected** (both are re-checked *above* the cache on every call), so this is a stale *grant walk*, not a stale key. Fix shape if it ever matters: stamp the generation on `Shard` itself, or make the racer wait on the winner. + - **Closed by `1979cf41` + `98d6a190` (r2).** Took the first fix shape — the generation is + stamped on `Shard`, `TryGet` is generation-gated, and `Set` displaces a mismatched shard; + `98d6a190` added the monotonic `<=` guard that stops a stale-generation straggler rolling + the stamp backward. One benign residual remains (see r2's register). - **A fresh `Shard` could take an initial-capacity hint** for a session known to be crawling, to avoid the dictionary's growth rehashes on exactly the workload the per-key ceiling exists for. + - **Closed by `1979cf41` (r2).** The eviction swap seeds the replacement shard with a + capacity hint, gated on the generation matching. **S7** - **No cancellation-branch test on any read path.** Pre-existing, but this branch added a third path (multi-var), so the gap now spans three. + - **Closed by `4f860733` (r2).** The span and multi-var cancellation reopen contracts are + pinned (hang knobs + entered-TCS + `CreatedPlcCount`), and `120e885b` added the assertion + that the stale handle is *disposed*, not merely replaced. - **No integration case for one bad decode inside an otherwise-good chunk** — the per-chunk fallback is unit-tested only. + - **Closed by `4f860733` (r2).** One-bad-decode-in-chunk pinned (only that tag goes Bad, no + fallback), alongside the short-answer shape mismatch (whole-chunk fallback). - **The Timer/Counter exclusion needs a live pin**, which needs a fixture with a TM/CT area. + - **Closed by `fd202d81` (r2).** TM/CT areas added to the `s7_1500` fixture and the exclusion + live-pinned against `10.100.0.35:1102` **at device 0**, the fixed point of snap7's + BIT-address resolution and a real CPU's device-number addressing; the device-number leg + (T5/C7) is env-gated on `S7_TIMER_NUMBER_ADDRESSING=1` as a ready-made real-CPU gate. **Historian** - **Task 3's `Task.Delay(500)` is still a hope-wait** and could become a bounded poll, the same treatment Task 9 gave the durability test. + - **Closed by `45d67092` + `68461fbf` (r2).** All three fixed sleeps became bounded watches on + real observables (`RemoveCallCount`, queue depth, `PeekBatchCallCount`), with the negative + watch's cap anchored to a measured ~150 ms falsification trip. **AdminUI tests** - **A mid-run SQL connectivity blip surfaces as `SqlException` failures, not skips** — the reachability probe is one-shot, taken before the run. + - **Documented wontfix (r2).** The one-shot probe is deliberate — a per-test probe would turn + a genuine mid-run outage into silent skips; verified against the comment at the probe site + and left as-is. - **Orphaned test databases have no reaper.** A killed test process leaves its uniquely-named database behind on `10.100.0.35,14330`. + - **Closed by `8946644b` + `580b77ce` (r2).** A one-shot startup reaper drops >24 h-orphaned + databases per fixture prefix; `580b77ce` fixed a review-found Critical (unvalidated name + into interpolated `sa` DDL) with a GUID-suffix shape guard plus parameterized + `QUOTENAME`/`sp_executesql`. **MTConnect** - **`AsStream` wants a buffer-ownership comment** so a future caller does not have to infer the lifetime from the call site. + - **Closed by `1edfc263` (r2).** The doc comment now states the array-backed aliasing, the + synchronous call-duration lifetime, and that the result holds no reference into the buffer. **docker-dev** - **The first `POST /api/deployments` after boot can 404.** Worth a readiness gate on the deployments endpoint, so scripted deploys do not have to encode a retry. + - **Closed by `a61b3c98` + `9f09a62a` (r2)** — as a *gate plus instrumentation*, not a + diagnosis: `deploy.sh` polls `/health/ready` (2 s/120 s) then POSTs once un-retried, and + Traefik now runs `--accesslog` so the next occurrence names the matching router. **The 404's + mechanism remains unconfirmed** until that log catches one.