ci: drop the nightly-windev schedule; run the deep windev tier on demand
The scheduled nightly-windev job failed most nights for host reasons and filed a Gitea issue each time. Remove the cron trigger and the job, and add a workflow_dispatch trigger whose windev_mode input (test/build/live) drives the windows-x86 job, so the live tier runs on demand - from the Actions page or via scripts/ci/run-windev-ci.sh live - the same way clients are packed and published on demand. Docs updated to match. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
+22
-18
@@ -89,10 +89,12 @@ fake-worker tests cannot validate:
|
||||
when the rig does not drive sample-bearing buffered batches on demand.
|
||||
|
||||
All nine tests are gated by the same `MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`
|
||||
opt-in variable. Opt-in does not mean unscheduled: the `nightly-windev` job runs
|
||||
this suite on windev every night at 06:00 UTC and files a Gitea issue when it goes
|
||||
red (see [Continuous Integration](#continuous-integration)), so the smoke no longer
|
||||
depends on someone remembering to set the variable.
|
||||
opt-in variable. Nothing runs this suite on a schedule; run it on demand on windev with
|
||||
the `live` tier — dispatch the `ci` workflow with `windev_mode: live`, or run
|
||||
`CI_SHA=<sha> scripts/ci/run-windev-ci.sh live` (see
|
||||
[Continuous Integration](#continuous-integration)). The scheduled `nightly-windev` job
|
||||
that used to do this was removed on 2026-11-15 because it failed most nights for host
|
||||
reasons rather than code changes.
|
||||
|
||||
Known coverage gap: the suite reaches all six late-added MXAccess **COM** commands
|
||||
but none of the five **control** commands (`Ping`, `GetSessionState`,
|
||||
@@ -531,8 +533,8 @@ dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj
|
||||
## Running the Gateway Suite on windev
|
||||
|
||||
The gateway suite (`ZB.MOM.WW.MxGateway.Tests`, net10.0/x64) is not part of the CI
|
||||
Windows tier — `windows-x86` and `nightly-windev` run only the x86 Worker build and
|
||||
`Worker.Tests`. It is still run on windev by hand when a change needs Windows
|
||||
Windows tier on a push — `windows-x86` runs only the x86 Worker build and
|
||||
`Worker.Tests` unless it is dispatched by hand with `windev_mode: live`. It is still run on windev by hand when a change needs Windows
|
||||
confirmation, and that run has three Windows-specific characteristics worth knowing
|
||||
before results are interpreted.
|
||||
|
||||
@@ -682,8 +684,7 @@ runtime because the x86 Worker cannot build on Linux:
|
||||
client (IPC-24 deleted the old unconditional churn-revert step, which masked exactly that
|
||||
class). The dev Mac has a Homebrew JDK 17, so `gradle generateProto` can be run there to refresh
|
||||
the Java aggregates when a `.proto` changes.
|
||||
- **`windows-x86`** (Linux runner, per push/PR; skipped on the schedule, where
|
||||
`nightly-windev` runs the same tiers) — builds the **x86 / net48 Worker and
|
||||
- **`windows-x86`** (Linux runner, per push/PR, and on demand via `workflow_dispatch`) — builds the **x86 / net48 Worker and
|
||||
Worker.Tests**, which are Windows-only and out of scope for the Linux jobs. It runs on a
|
||||
Linux runner that always schedules and SSHes to windev (`10.100.0.48`), where
|
||||
`scripts/ci/run-windev-ci.sh` fast-forwards the isolated CI clone `C:\build\mxaccessgw-ci`
|
||||
@@ -694,25 +695,28 @@ runtime because the x86 Worker cannot build on Linux:
|
||||
the tier is down, not the change**: an unreachable windev fails the job (never skips), so
|
||||
if it goes red without a related code change, check the tier — `ssh windev`, read
|
||||
`C:\build\mxaccessgw-ci\ci.log` — before assuming a real test failure.
|
||||
- **`nightly-windev`** (Linux runner, scheduled `0 6 * * *`) — runs, in order, the x86 Worker
|
||||
build, `Worker.Tests`, a full `slnx` build (which surfaces Windows-only analyzer diagnostics
|
||||
such as `xUnit1030`), the net10 gateway suite, and the live-MXAccess smoke
|
||||
(`MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`). The checkout and all five tiers share the windev
|
||||
host lock, and the first nonzero native exit stops the sequence and propagates to the CI job.
|
||||
Gated `if: github.event_name == 'schedule'`, so it never gates a push. On failure it opens a
|
||||
Gitea issue via the Actions token, since nobody watches the Actions page.
|
||||
- **Deep windev tier, on demand** — there is no scheduled run. The `live` mode runs, in order,
|
||||
the x86 Worker build, `Worker.Tests`, a full `slnx` build (which surfaces Windows-only analyzer
|
||||
diagnostics such as `xUnit1030`), the net10 gateway suite, and the live-MXAccess smoke
|
||||
(`MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`). Run it when a change needs it, the same way the clients
|
||||
are packed and published on demand: dispatch the `ci` workflow from the Gitea Actions page with
|
||||
`windev_mode: live`, or run `CI_SHA=<sha> scripts/ci/run-windev-ci.sh live` from a machine with
|
||||
SSH access to windev. The checkout and all five tiers share the windev host lock, and the first
|
||||
nonzero native exit stops the sequence and propagates back. This replaced the `nightly-windev`
|
||||
cron job (removed 2026-11-15), which failed most nights for host reasons and filed a Gitea issue
|
||||
each time.
|
||||
|
||||
The windev host lock is an exclusive file handle on `C:\build\mxaccessgw-ci.ci-lock`, held by
|
||||
the SSH bootstrap for the whole run. It replaced a `New-Item -ItemType Directory` lock that two
|
||||
runs starting in the same second could both take (PowerShell checks for the directory, then
|
||||
creates it), which is how the 2026-09-22 nightly collided with the scheduled `windows-x86` run on
|
||||
creates it), which is how the 2026-09-22 nightly (since removed) collided with the scheduled `windows-x86` run on
|
||||
`.git/index.lock` (mxaccessgw#150). The OS drops the handle when its holder dies, so a killed run
|
||||
leaves no stale lock, and a holder may delete a leftover `index.lock` because no other CI git can
|
||||
be running. Each run also writes a `host (start)` line — and a `host (failure)` line when a tier
|
||||
fails — to `C:\build\mxaccessgw-ci\ci.log` with free commit, physical memory, disk and the
|
||||
number of `testhost` processes, so a host-resource failure can be matched to the host's state.
|
||||
|
||||
When a nightly goes red, sort the cause before touching code: a test that fails with an
|
||||
When a `live` run goes red, sort the cause before touching code: a test that fails with an
|
||||
assertion or test exception is a test or product fault; `OutOfMemoryException` from
|
||||
MSBuild/csc/protoc, `csc.exe exited with code -1073741502` (0xC0000142, a process that could not
|
||||
initialize), a remote exit code with no `ci.log` lines at all, or an SSH `kex_exchange_identification`
|
||||
@@ -743,7 +747,7 @@ second runner.
|
||||
When queue depth (or the missing-cancel reality) makes waiting impractical, verify a
|
||||
specific commit out of band instead of waiting behind the queue: run
|
||||
`CI_SHA=<sha> scripts/ci/run-windev-ci.sh <build|test|live>` from a machine with SSH access
|
||||
to windev (the same script the SSH-driven `windows-x86`/`nightly-windev` jobs use — see
|
||||
to windev (the same script the SSH-driven `windows-x86` job uses — see
|
||||
`scripts/ci/README.md`), or fall back to the manual windev worktree procedure below. This is
|
||||
the same escape hatch used when the windev tier itself is down — TST-30 generalizes it from
|
||||
"tier down" to "runner contended": either way, a stuck or slow shared runner should not
|
||||
|
||||
@@ -228,7 +228,8 @@ days. The SQLite DB must also never be opened with a host-side `sqlite3` while t
|
||||
## 3. Preflight (no changes yet)
|
||||
|
||||
1. Decide the target SHA on `origin/main`; confirm CI is green for it (`windows-x86` job runs the
|
||||
Worker build + `Worker.Tests` on windev; `nightly-windev` runs the live tier,
|
||||
Worker build + `Worker.Tests` on windev; run the live tier on demand with
|
||||
`CI_SHA=<sha> scripts/ci/run-windev-ci.sh live` or a `windev_mode: live` dispatch,
|
||||
`scripts/ci/README.md`).
|
||||
2. Read the current state of the service (read-only):
|
||||
```text
|
||||
|
||||
Reference in New Issue
Block a user