ci: job timeout and --blame-hang so a stalled test step fails located, not after 3h of silence (lmxopcua#718)
v2-ci / unit-tests (push) Has been cancelled
v2-ci / unit-tests (push) Has been cancelled
Run 959 (job 3552) was silent from 13:39 for 3h12m and was failed by the server
reaper with no cause. Pre-#707 runs 935/938/941/942/947/959 all ended at 189-193 min;
no run since 76bdca90 exceeds 82 min (healthy ~53). Adds timeout-minutes 120 and
blame-hang (20m, no dump) to the whole-solution test step.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
@@ -33,6 +33,11 @@ jobs:
|
||||
|
||||
unit-tests:
|
||||
runs-on: ubuntu-latest
|
||||
# #718: run 959 sat silent for 3h12m (no log after 13:39) until the server's own ~3h endless-task
|
||||
# reaper failed it, with no hung test named. Every healthy run is ~60 min end to end (test step
|
||||
# ~53 min), so 120 min fails a stall in half the time and cancels the step rather than waiting
|
||||
# for the server reaper.
|
||||
timeout-minutes: 120
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-dotnet@v4
|
||||
@@ -69,6 +74,9 @@ jobs:
|
||||
# — sized on an idle dev box — failed in a long
|
||||
# tail, a different test per run. Dilation only stretches waits: a presence wait still fails if
|
||||
# the message never comes, and absence windows get longer (stricter). Local runs stay at 1.
|
||||
# #718: --blame-hang kills a test host whose single test makes no progress for 20 min and
|
||||
# NAMES that test in the log (no dump: the runner has no createdump handling), so a hang is a
|
||||
# located failure instead of hours of silence. Healthy tests finish in seconds.
|
||||
env:
|
||||
AKKA_TEST_TIMEFACTOR: "3"
|
||||
run: >
|
||||
@@ -76,6 +84,7 @@ jobs:
|
||||
-m:1 -p:TestTfmsInParallel=false --settings scripts/ci/ci.runsettings
|
||||
--filter "Category!=E2E&Category!=LiveIntegration"
|
||||
--logger "trx" --results-directory artifacts/trx
|
||||
--blame-hang --blame-hang-timeout 20m --blame-hang-dump-type none
|
||||
# 07/S-2 fail-on-skip: a whole-solution run that executed ZERO tests is a masked outage
|
||||
# (bad filter, broken discovery), not a pass — fail loudly. Runs even on test failure so
|
||||
# the diagnostic is always emitted.
|
||||
|
||||
@@ -18,6 +18,11 @@ one run is active; an active run is not cancelled, but a newer pending run may
|
||||
supersede an older pending run. The group does not serialize other repositories
|
||||
using the same host.
|
||||
|
||||
Stall guards (#718): the job has `timeout-minutes: 120` (healthy runs take about 60 minutes)
|
||||
and the test step passes `--blame-hang --blame-hang-timeout 20m --blame-hang-dump-type none`, so a
|
||||
test that stops making progress is killed and named in the log instead of leaving hours of silence
|
||||
(run 959 stalled 3h12m until the server reaper failed it).
|
||||
|
||||
To reproduce the bounded test leg after restore and Release build:
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user