Merge branch 'fix/tst-30-runner-docs'
ci / java (push) Successful in 2m11s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m38s
ci / portable (push) Failing after 17m43s

# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
This commit is contained in:
Joseph Doherty
2026-08-07 07:50:21 -04:00
4 changed files with 151 additions and 7 deletions
+33 -5
View File
@@ -453,6 +453,32 @@ runtime because the x86 Worker cannot build on Linux:
`xUnit1030`). Gated `if: github.event_name == 'schedule'`, so it never gates a push. On
failure it opens a Gitea issue via the Actions token, since nobody watches the Actions page.
### Runner capacity is shared and finite
All CI runs on a co-located `gitea-runner` container on docker host `10.100.0.35` with
`maxParallel=1`, and that runner instance is **shared across repos** — it interleaves
`dohertj2/mxaccessgw` and `dohertj2/lmxopcua` jobs on the single slot rather than being
scoped to this repo (`GET /repos/dohertj2/mxaccessgw/actions/runners` returns
`total_count: 0`; the runner is registered at the instance level). Every job in a run
(`portable`, `java`, `windows-x86`) executes serially, so queue latency is additive within
a run and an active `lmxopcua` run blocks `mxaccessgw` entirely — expect ~2030 minutes of
queue depth under cross-repo contention, not a stuck pipeline. This Gitea version (1.26)
also exposes **no run cancel or delete via the API** (`POST .../actions/runs/{id}/cancel`
returns 404, `DELETE .../actions/runs/{id}` returns 400), so a superseded or hung run cannot
be cleared and holds the slot until it finishes or times out. See
`docs/runbooks/TST-30-second-ci-runner.md` for the operator runbook that registers a second
runner to relieve this; until that lands, treat single-runner contention as expected, not a
CI outage.
When queue depth (or the missing-cancel reality) makes waiting impractical, verify a
specific commit out of band instead of waiting behind the queue: run
`CI_SHA=<sha> scripts/ci/run-windev-ci.sh <build|test|live>` from a machine with SSH access
to windev (the same script the SSH-driven `windows-x86`/`nightly-windev` jobs use — see
`scripts/ci/README.md`), or fall back to the manual windev worktree procedure below. This is
the same escape hatch used when the windev tier itself is down — TST-30 generalizes it from
"tier down" to "runner contended": either way, a stuck or slow shared runner should not
block verifying a commit.
The freshness guard `scripts/check-codegen.ps1` runs four checks and fails the build when the
committed client descriptor set (Check 1), the C# `Generated/` (Check 2), the Rust vendored
protos (Check 3), or the Go/Python client bindings (Check 4, IPC-25) no longer match the current
@@ -467,11 +493,13 @@ the x86 build with `CS0246`). See [Client Proto Generation](./ClientProtoGenerat
[Contracts](./Contracts.md).
If the SSH-driven Windows tier is unavailable for infrastructure reasons (windev down, CI
key/secret rotation in flight), fall back to the manual windev worktree procedure as a
degraded mode: on windev, fast-forward an isolated `origin/main` worktree under `C:\build`
(never the dirty Desktop checkout), then run the x86 Worker build and `Worker.Tests`
(`-p:Platform=x86`) there by hand. Do this per merge for worker-touching changes until the
`windows-x86` job is green again.
key/secret rotation in flight) **or** the shared Gitea runner is contended and the queue is
impractical to wait behind (see "Runner capacity is shared and finite" above), fall back to
the manual windev worktree procedure as a degraded mode: on windev, fast-forward an isolated
`origin/main` worktree under `C:\build` (never the dirty Desktop checkout), then run the x86
Worker build and `Worker.Tests` (`-p:Platform=x86`) there by hand. Do this per merge for
worker-touching changes until the `windows-x86` job is green again (tier-down case) or the
queue clears (contention case).
## Related Documentation