Joseph Doherty
30c92e8e59
Exercising TST-25 acceptance Check 6 end-to-end showed the on-failure issue step works — it has filed an issue on every red nightly since 2026-07-17 (#126-#139) — but that every one of those issues links `http://gitea:3000`, the runner-internal origin Gitea hands the runner. That URL is correct for the issue-creation API call (the job container resolves `gitea` only on the docker network and has no LAN egress to the public origin) and unreachable for anyone clicking out of the issue. Keep `github.server_url` for the API call; add a `PUBLIC_SERVER_URL` job env used only for the browser-facing link in the body. Verified with a forced-failure probe run (677) whose issue (#140) carries a public link that returns 200. Also records Check 6 as done in scripts/ci/README.md and corrects the 2026-07-13 tracking entry that wrote the check off as abandoned.
73 lines
5.0 KiB
Markdown
73 lines
5.0 KiB
Markdown
# Windows/x86 CI tier (TST-25)
|
||
|
||
The x86 / net48 Worker and its tests cannot build on a Linux runner, and a native Windows
|
||
`act_runner` is broken (host-mode writes each step's script to a path it can't resolve; a
|
||
`runs-on` gate with no runner wedges the queue forever). Instead, a Linux CI job SSHes to
|
||
windev (`10.100.0.48`) and runs the Worker build/tests there, propagating the remote exit
|
||
code back so a regression turns the job red.
|
||
|
||
## Pieces
|
||
|
||
| File | Runs on | Role |
|
||
|---|---|---|
|
||
| `run-windev-ci.sh <build\|test\|live>` | Linux CI job | Writes the SSH key/known-hosts, base64-encodes a bootstrap, SSHes to windev, returns the remote exit code. |
|
||
| `windev-worker-ci.ps1 -Sha -Mode` | windev | Locks the CI clone, checks out the SHA, runs the x86 build / `Worker.Tests` / live smoke. |
|
||
| `windev.known_hosts` | committed | Pinned windev host keys so `StrictHostKeyChecking=yes` has no TOFU prompt. Host keys are public; committing is intentional. |
|
||
|
||
Modes are cumulative: `build` = x86 Worker build; `test` = build + `Worker.Tests`;
|
||
`live` = test + full-`slnx` build + live-MXAccess smoke (`MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`).
|
||
|
||
CI jobs (`.gitea/workflows/ci.yml`): `windows-x86` runs `test` per push/PR; `nightly-windev`
|
||
runs `live` on the `0 6 * * *` schedule and opens a Gitea issue on failure.
|
||
|
||
## One-time bring-up (operator — required before this lands on `main`)
|
||
|
||
Until these are done, `windows-x86` is **red on every push** (no key → SSH fails). Do them
|
||
first, then merge.
|
||
|
||
1. **CI clone** — `C:\build\mxaccessgw-ci`, a fresh clone of the Gitea origin (done during
|
||
TST-25 verification; recreate with `git clone https://gitea.dohertylan.com/dohertj2/mxaccessgw C:\build\mxaccessgw-ci` if missing). Never the dirty Desktop checkout.
|
||
2. **Dedicated CI key** — generate a NEW ed25519 keypair for CI (do not reuse the operator's
|
||
personal windev key): `ssh-keygen -t ed25519 -f ci_windev -N ''`. Append `ci_windev.pub`
|
||
to the windev CI account's `authorized_keys` (optionally with a forced `command=` limiting
|
||
it to this bootstrap).
|
||
3. **Gitea repo secret `WINDEV_SSH_KEY`** — store the private key **base64-encoded on a single
|
||
line**: `base64 -w0 < ci_windev` (macOS: `base64 < ci_windev | tr -d '\n'`). This is required,
|
||
not cosmetic — Gitea's secret masker is line-oriented, so a raw multiline PEM is **not**
|
||
redacted in the step `env:` echo and leaks in cleartext; the single-line base64 masks to `***`.
|
||
`run-windev-ci.sh` auto-decodes it. Host keys are public and come from the committed
|
||
`windev.known_hosts` pin, so no `WINDEV_SSH_KNOWN_HOSTS` secret is wired into `ci.yml`.
|
||
`run-windev-ci.sh` connects as user `ci` by default; set the `WINDEV_SSH_USER` **variable**
|
||
(not a secret — it is public) to the actual account, and ensure that account exists on windev.
|
||
4. **Network precheck** — confirm the Gitea runner container (on the `traefik` docker network,
|
||
host `10.100.0.35`) can reach `10.100.0.48:22`: a one-off `nc -z -w5 10.100.0.48 22` step.
|
||
The network was created for gitea name resolution, not LAN egress, so verify.
|
||
5. **Actions token** — the `nightly-windev` on-failure step creates a Gitea issue with the
|
||
built-in token; confirm Actions tokens have issue-write on this repo.
|
||
|
||
## Verifying it end-to-end (design's acceptance checks)
|
||
|
||
- Push a branch → `portable`, `java`, `windows-x86` all green; windev's clone sits at the pushed SHA.
|
||
- Deliberate red: add a failing `[Fact]` to `Worker.Tests`, push, confirm `windows-x86` goes
|
||
red; revert. (Proves exit-code propagation — the whole point.)
|
||
- Unreachable-host red: point at a bogus port / stop sshd, confirm the job fails fast, not hangs.
|
||
- Concurrency: push two branches back-to-back, confirm the second remote run waits on the lock.
|
||
- Nightly: trigger the schedule path, confirm `live` runs and a forced failure opens an issue.
|
||
**Done 2026-08-10** (Check 6). Verified two ways: (a) production — every red nightly since
|
||
2026-07-17 has auto-filed an issue (#126–#139) authored by the `gitea-actions` bot, e.g. run 672
|
||
→ issue #139, with the built-in token masked to `***` in the job log; (b) a forced-failure probe
|
||
on the throwaway branch `test/tst25-check6-nightly-issue` (run 677 → issue #140, since closed and
|
||
the branch deleted), which reproduced the job shape with `exit 1` in place of the live step and
|
||
confirmed the `if: failure()` step fires, the token carries issue-write, and the payload is
|
||
well-formed. The probe also caught the one defect: `${{ github.server_url }}` is the
|
||
runner-internal `http://gitea:3000`, so the run link in the issue body was unreachable from a
|
||
browser — the body now uses the `PUBLIC_SERVER_URL` job env instead (the API call still targets
|
||
`github.server_url`, which is what the job container can resolve).
|
||
- Confirm no key material appears in job logs.
|
||
|
||
## Degraded mode
|
||
|
||
If the tier is down for infra reasons, fall back to the manual windev worktree procedure
|
||
(isolated `C:\build` worktree, `dotnet build`/`test` `-p:Platform=x86` by hand) per merge for
|
||
worker-touching changes until `windows-x86` is green again. See `docs/GatewayTesting.md`.
|