Joseph Doherty
10b898305f
S-1: replace the hand-maintained 5-project unit-tests matrix (which silently dropped Client's 388 tests, Analyzers' 31, and every driver + most Core suite) with ONE whole-solution leg — dotnet test ZB.MOM.WW.OtOpcUa.slnx --filter "Category!=E2E&Category!=LiveIntegration". Self-maintaining: a new *.Tests project is covered automatically, matching CLAUDE.md's own guidance. S-2: emit trx + add scripts/ci/assert-not-all-skipped.sh, a fail-on-skip gate that turns 'green CI == everything skipped' into a red build. Wired strict (MIN_EXECUTED=1) on the unit leg; report-only (MIN_EXECUTED=0) on the fixtureless integration leg so its skip tally is VISIBLE without a false red — with a documented follow-up to start the one public-image fixture (opc-plc) as a service and flip it strict. S-4 (paired): the newly-CI'd Client.CLI.Tests had fixed-sleep startup races (await Task.Delay(100/150) before cancelling a background command) that would flake under CI load. Added SubscribeInvoked / SubscribeAlarmsInvoked readiness signals (TaskCompletionSource) to FakeOpcUaClientService and replaced the 11 sleeps across AlarmsCommandTests / SubscribeCommandTests / EventHandlerLifecycleTests with a deterministic await-the-signal (10s timeout guard). Verified: workflow YAML parses; skip-gate proven locally on a real trx (executed=31 => OK), a synthetic all-skipped trx (executed=0 => exit 1 with diagnostic), report-only mode (never fails), multi-file sum, and missing-file (exit 2); Client.CLI.Tests 104/104 green after the deflake. (CI job execution itself is verifiable only on push — nothing pushed.)
55 lines
2.1 KiB
Bash
Executable File
55 lines
2.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
#
|
|
# assert-not-all-skipped.sh — arch-review 07/S-2 fail-on-skip guard.
|
|
#
|
|
# A CI test leg that ran but executed ZERO tests reports success, so a whole tier
|
|
# silently skipping (fixtures unreachable, a --filter that matched nothing, a probe
|
|
# that Assert.Skip'd every case) is indistinguishable from a real pass. This gate
|
|
# reads the trx result summary and FAILS when fewer than MIN_EXECUTED tests actually
|
|
# ran across the supplied trx files — turning "green means skipped" into a red build.
|
|
#
|
|
# Usage: scripts/ci/assert-not-all-skipped.sh <trx> [<trx> ...]
|
|
# Env: MIN_EXECUTED minimum executed tests required (default 1).
|
|
# Set MIN_EXECUTED=0 for a report-only pass (never fails) —
|
|
# used by the fixtureless integration leg until its fixtures
|
|
# run as workflow services (S-2 option #2 follow-up).
|
|
#
|
|
set -euo pipefail
|
|
|
|
min_executed="${MIN_EXECUTED:-1}"
|
|
|
|
if [ "$#" -eq 0 ]; then
|
|
echo "assert-not-all-skipped: no trx files given" >&2
|
|
exit 2
|
|
fi
|
|
|
|
total_executed=0
|
|
total_skipped=0
|
|
seen=0
|
|
|
|
for f in "$@"; do
|
|
if [ ! -f "$f" ]; then
|
|
echo "assert-not-all-skipped: trx not found: $f" >&2
|
|
exit 2
|
|
fi
|
|
seen=$((seen + 1))
|
|
# The trx <Counters .../> element in <ResultSummary> carries executed + notExecuted counts.
|
|
counters="$(grep -oE '<Counters[^>]*/?>' "$f" | head -1)"
|
|
ex="$(printf '%s' "$counters" | grep -oE 'executed="[0-9]+"' | grep -oE '[0-9]+' || true)"
|
|
ne="$(printf '%s' "$counters" | grep -oE 'notExecuted="[0-9]+"' | grep -oE '[0-9]+' || true)"
|
|
ex="${ex:-0}"
|
|
ne="${ne:-0}"
|
|
total_executed=$((total_executed + ex))
|
|
total_skipped=$((total_skipped + ne))
|
|
echo " $(basename "$f"): executed=${ex} skipped=${ne}"
|
|
done
|
|
|
|
echo "assert-not-all-skipped: files=${seen} executed=${total_executed} skipped=${total_skipped} (min required=${min_executed})"
|
|
|
|
if [ "$total_executed" -lt "$min_executed" ]; then
|
|
echo "::error::CI executed ${total_executed} test(s) (< ${min_executed}) — the tier ran nothing (all skipped?). Treating as FAILURE, not a pass." >&2
|
|
exit 1
|
|
fi
|
|
|
|
echo "assert-not-all-skipped: OK"
|