Files
mxaccessgw/scripts/ci/README.md
T
Joseph Doherty e74918ccfb
ci / portable (push) Failing after 10m10s
ci / java (push) Failing after 20m12s
ci / windows-x86 (push) Failing after 22m40s
ci: drop the nightly-windev schedule; run the deep windev tier on demand
The scheduled nightly-windev job failed most nights for host reasons and
filed a Gitea issue each time. Remove the cron trigger and the job, and add
a workflow_dispatch trigger whose windev_mode input (test/build/live) drives
the windows-x86 job, so the live tier runs on demand - from the Actions page
or via scripts/ci/run-windev-ci.sh live - the same way clients are packed and
published on demand. Docs updated to match.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-15 09:42:13 -05:00

6.1 KiB
Raw Blame History

Windows/x86 CI tier (TST-25)

The x86 / net48 Worker and its tests cannot build on a Linux runner, and a native Windows act_runner is broken (host-mode writes each step's script to a path it can't resolve; a runs-on gate with no runner wedges the queue forever). Instead, a Linux CI job SSHes to windev (10.100.0.48) and runs the Worker build/tests there, propagating the remote exit code back so a regression turns the job red.

Pieces

File Runs on Role
run-windev-ci.sh <build|test|live> Linux CI job Writes the SSH key/known-hosts, base64-encodes a bootstrap, SSHes to windev, returns the remote exit code.
windev-worker-ci.ps1 -Sha -Mode windev Locks the CI clone, checks out the SHA, and runs the requested cumulative Windows tiers.
windev.known_hosts committed Pinned windev host keys so StrictHostKeyChecking=yes has no TOFU prompt. Host keys are public; committing is intentional.

Modes are cumulative: build = x86 Worker build; test = build + Worker.Tests; live = test + full-slnx build + the net10 gateway suite + live-MXAccess smoke (MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1). The live order is x86 Worker build, Worker.Tests, full solution build, gateway tests, then live smoke. One host lock covers checkout and every tier, so concurrent runs cannot overlap. The lock is an exclusive (FileShare.None) handle on C:\build\mxaccessgw-ci.ci-lock held by the SSH bootstrap; the OS releases it if the run is killed, so it never goes stale. (It replaced a New-Item -ItemType Directory lock that two runs starting in the same second could both take — mxaccessgw#150.) Runs of SHAs older than that change still use the old directory lock and do not exclude new-style runs; that window closes as branches rebase. each native command is checked immediately and the first nonzero exit aborts the remaining tiers and propagates to CI.

CI jobs (.gitea/workflows/ci.yml): windows-x86 runs test per push/PR. There is no scheduled run. Run the live tier on demand, like the client pack/publish scripts:

# from a machine with SSH access to windev (falls back to the default SSH identity)
WINDEV_SSH_USER=<user> CI_SHA=$(git rev-parse HEAD) scripts/ci/run-windev-ci.sh live

or dispatch the ci workflow from the Gitea Actions page with windev_mode: live. The nightly-windev cron job that used to run live every night and file a Gitea issue on failure was removed on 2026-11-15: it failed most nights for host reasons and the issues were noise.

One-time bring-up (operator — required before this lands on main)

Until these are done, windows-x86 is red on every push (no key → SSH fails). Do them first, then merge.

  1. CI clone — C:\build\mxaccessgw-ci, a fresh clone of the Gitea origin (done during TST-25 verification; recreate with git clone https://gitea.dohertylan.com/dohertj2/mxaccessgw C:\build\mxaccessgw-ci if missing). Never the dirty Desktop checkout.
  2. Dedicated CI key — generate a NEW ed25519 keypair for CI (do not reuse the operator's personal windev key): ssh-keygen -t ed25519 -f ci_windev -N ''. Append ci_windev.pub to the windev CI account's authorized_keys (optionally with a forced command= limiting it to this bootstrap).
  3. Gitea repo secret WINDEV_SSH_KEY — store the private key base64-encoded on a single line: base64 -w0 < ci_windev (macOS: base64 < ci_windev | tr -d '\n'). This is required, not cosmetic — Gitea's secret masker is line-oriented, so a raw multiline PEM is not redacted in the step env: echo and leaks in cleartext; the single-line base64 masks to ***. run-windev-ci.sh auto-decodes it. Host keys are public and come from the committed windev.known_hosts pin, so no WINDEV_SSH_KNOWN_HOSTS secret is wired into ci.yml. run-windev-ci.sh connects as user ci by default; set the WINDEV_SSH_USER variable (not a secret — it is public) to the actual account, and ensure that account exists on windev.
  4. Network precheck — confirm the Gitea runner container (on the traefik docker network, host 10.100.0.35) can reach 10.100.0.48:22: a one-off nc -z -w5 10.100.0.48 22 step. The network was created for gitea name resolution, not LAN egress, so verify.

Verifying it end-to-end (design's acceptance checks)

  • Push a branch → portable, java, windows-x86 all green; windev's clone sits at the pushed SHA.
  • Deliberate red: add a failing [Fact] to Worker.Tests, push, confirm windows-x86 goes red; revert. (Proves exit-code propagation — the whole point.)
  • Unreachable-host red: point at a bogus port / stop sshd, confirm the job fails fast, not hangs.
  • Concurrency: push two branches back-to-back, confirm the second remote run waits on the lock.
  • Nightly (historical — the job was removed 2026-11-15): trigger the schedule path, confirm live runs and a forced failure opens an issue. Done 2026-08-10 (Check 6). Verified two ways: (a) production — every red nightly since 2026-07-17 has auto-filed an issue (#126–#139) authored by the gitea-actions bot, e.g. run 672 → issue #139, with the built-in token masked to *** in the job log; (b) a forced-failure probe on the throwaway branch test/tst25-check6-nightly-issue (run 677 → issue #140, since closed and the branch deleted), which reproduced the job shape with exit 1 in place of the live step and confirmed the if: failure() step fires, the token carries issue-write, and the payload is well-formed. The probe also caught the one defect: ${{ github.server_url }} is the runner-internal http://gitea:3000, so the run link in the issue body was unreachable from a browser — the body now uses the PUBLIC_SERVER_URL job env instead (the API call still targets github.server_url, which is what the job container can resolve).
  • Confirm no key material appears in job logs.

Degraded mode

If the tier is down for infra reasons, fall back to the manual windev worktree procedure (isolated C:\build worktree, dotnet build/test -p:Platform=x86 by hand) per merge for worker-touching changes until windows-x86 is green again. See docs/GatewayTesting.md.