ci(v2): whole-solution unit leg + fail-on-skip gate + deflake CLI sleeps (arch-review 07/S-1,S-2,S-4)
S-1: replace the hand-maintained 5-project unit-tests matrix (which silently dropped Client's 388 tests, Analyzers' 31, and every driver + most Core suite) with ONE whole-solution leg — dotnet test ZB.MOM.WW.OtOpcUa.slnx --filter "Category!=E2E&Category!=LiveIntegration". Self-maintaining: a new *.Tests project is covered automatically, matching CLAUDE.md's own guidance. S-2: emit trx + add scripts/ci/assert-not-all-skipped.sh, a fail-on-skip gate that turns 'green CI == everything skipped' into a red build. Wired strict (MIN_EXECUTED=1) on the unit leg; report-only (MIN_EXECUTED=0) on the fixtureless integration leg so its skip tally is VISIBLE without a false red — with a documented follow-up to start the one public-image fixture (opc-plc) as a service and flip it strict. S-4 (paired): the newly-CI'd Client.CLI.Tests had fixed-sleep startup races (await Task.Delay(100/150) before cancelling a background command) that would flake under CI load. Added SubscribeInvoked / SubscribeAlarmsInvoked readiness signals (TaskCompletionSource) to FakeOpcUaClientService and replaced the 11 sleeps across AlarmsCommandTests / SubscribeCommandTests / EventHandlerLifecycleTests with a deterministic await-the-signal (10s timeout guard). Verified: workflow YAML parses; skip-gate proven locally on a real trx (executed=31 => OK), a synthetic all-skipped trx (executed=0 => exit 1 with diagnostic), report-only mode (never fails), multi-file sum, and missing-file (exit 2); Client.CLI.Tests 104/104 green after the deflake. (CI job execution itself is verifiable only on push — nothing pushed.)
This commit is contained in:
Executable
+54
@@ -0,0 +1,54 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# assert-not-all-skipped.sh — arch-review 07/S-2 fail-on-skip guard.
|
||||
#
|
||||
# A CI test leg that ran but executed ZERO tests reports success, so a whole tier
|
||||
# silently skipping (fixtures unreachable, a --filter that matched nothing, a probe
|
||||
# that Assert.Skip'd every case) is indistinguishable from a real pass. This gate
|
||||
# reads the trx result summary and FAILS when fewer than MIN_EXECUTED tests actually
|
||||
# ran across the supplied trx files — turning "green means skipped" into a red build.
|
||||
#
|
||||
# Usage: scripts/ci/assert-not-all-skipped.sh <trx> [<trx> ...]
|
||||
# Env: MIN_EXECUTED minimum executed tests required (default 1).
|
||||
# Set MIN_EXECUTED=0 for a report-only pass (never fails) —
|
||||
# used by the fixtureless integration leg until its fixtures
|
||||
# run as workflow services (S-2 option #2 follow-up).
|
||||
#
|
||||
set -euo pipefail
|
||||
|
||||
min_executed="${MIN_EXECUTED:-1}"
|
||||
|
||||
if [ "$#" -eq 0 ]; then
|
||||
echo "assert-not-all-skipped: no trx files given" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
total_executed=0
|
||||
total_skipped=0
|
||||
seen=0
|
||||
|
||||
for f in "$@"; do
|
||||
if [ ! -f "$f" ]; then
|
||||
echo "assert-not-all-skipped: trx not found: $f" >&2
|
||||
exit 2
|
||||
fi
|
||||
seen=$((seen + 1))
|
||||
# The trx <Counters .../> element in <ResultSummary> carries executed + notExecuted counts.
|
||||
counters="$(grep -oE '<Counters[^>]*/?>' "$f" | head -1)"
|
||||
ex="$(printf '%s' "$counters" | grep -oE 'executed="[0-9]+"' | grep -oE '[0-9]+' || true)"
|
||||
ne="$(printf '%s' "$counters" | grep -oE 'notExecuted="[0-9]+"' | grep -oE '[0-9]+' || true)"
|
||||
ex="${ex:-0}"
|
||||
ne="${ne:-0}"
|
||||
total_executed=$((total_executed + ex))
|
||||
total_skipped=$((total_skipped + ne))
|
||||
echo " $(basename "$f"): executed=${ex} skipped=${ne}"
|
||||
done
|
||||
|
||||
echo "assert-not-all-skipped: files=${seen} executed=${total_executed} skipped=${total_skipped} (min required=${min_executed})"
|
||||
|
||||
if [ "$total_executed" -lt "$min_executed" ]; then
|
||||
echo "::error::CI executed ${total_executed} test(s) (< ${min_executed}) — the tier ran nothing (all skipped?). Treating as FAILURE, not a pass." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "assert-not-all-skipped: OK"
|
||||
Reference in New Issue
Block a user