Joseph Doherty
a8f86b5336
TST-24 asked for per-client wire tests against a fake gateway. An audit first corrected the finding's premise: Go, Rust, and Java already had them — bufconn, a loopback tonic server, and InProcessServerBuilder respectively — each already asserting the round trip, the server-observed bearer header, and the ReplayGap sentinel. The two genuine gaps were .NET (every test substituted the transport interface; the test project had no server package at all) and Python (stub monkeypatching everywhere but one opt-in TLS test). Both now serve mxaccess_gateway.v1.MxAccessGateway over a real transport — Kestrel h2c and grpc.aio, each on an ephemeral loopback port — and drive the ordinary public client API against it. Only the gateway's behaviour is canned; the framing, serialization, metadata, and status codes are genuine. Four shapes each: full round trip with every reply field asserted, the authorization header as received by the server (including on the streaming RPC), the ReplayGap sentinel surfaced as the client's typed signal, and a real PERMISSION_DENIED mapping to the typed authorization error. The .NET client was only ever compiled in CI, never tested, so the portable job gains a dotnet test step. Fixes a bug the new tests caught on their first run: Python's connect() built the grpc.aio channel inside asyncio.to_thread, and a grpc.aio channel binds to the event loop current on the constructing thread, so every non-stub connection raised 'There is no current event loop in thread'. No mock-based test could see it, and the test guarding the off-loop behaviour patched create_channel and so asserted the bug. Split resolve_channel_security (blocking TOFU probe, off-loop) from create_channel (on-loop); the guard tests now assert both halves.
199 lines
10 KiB
YAML
199 lines
10 KiB
YAML
# Continuous integration for mxaccessgw (Gitea Actions; origin is Gitea at gitea.dohertylan.com).
|
|
#
|
|
# Scope note: the x86 Worker (src/ZB.MOM.WW.MxGateway.Worker) and Worker.Tests target
|
|
# .NET Framework 4.8 / x86 and need MXAccess COM installed, so they build ONLY on a Windows host.
|
|
# They are out of scope for this Linux `portable` job — the `windows-x86` job below builds and
|
|
# tests them on windev (10.100.0.48) over SSH (see scripts/ci/run-windev-ci.sh), and the nightly
|
|
# `nightly-windev` job additionally runs the live-MXAccess smoke, which is scheduled-only so it
|
|
# never gates a push.
|
|
name: ci
|
|
|
|
on:
|
|
push:
|
|
pull_request:
|
|
schedule:
|
|
# 06:00 UTC nightly: the `nightly-windev` job re-runs the x86 Worker build + full Worker.Tests on
|
|
# windev, then the live-MXAccess smoke (MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1) and a full-slnx build.
|
|
# Push/PR runs skip it — the job is gated `if: github.event_name == 'schedule'`.
|
|
- cron: '0 6 * * *'
|
|
|
|
jobs:
|
|
portable:
|
|
# Any Linux runner: no MXAccess, no x86. Covers the NonWindows solution, gateway fake-worker
|
|
# tests, the codegen/descriptor freshness guards, and the clients that build on Linux.
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- uses: actions/setup-dotnet@v4
|
|
with:
|
|
dotnet-version: '10.0.x'
|
|
|
|
- uses: actions/setup-go@v5
|
|
with:
|
|
go-version: '1.26'
|
|
|
|
- uses: actions/setup-python@v5
|
|
with:
|
|
python-version: '3.12'
|
|
|
|
- uses: dtolnay/rust-toolchain@stable
|
|
with:
|
|
components: clippy, rustfmt
|
|
|
|
# protoc is pinned to 34.1 (see docs/ToolchainLinks.md) so the committed client descriptor
|
|
# regenerates reproducibly. The freshness check normalizes source_code_info, so it tolerates
|
|
# patch drift, but keep the CI toolchain on the pin.
|
|
- name: Install protoc 34.1
|
|
run: |
|
|
curl -sSL -o /tmp/protoc.zip https://github.com/protocolbuffers/protobuf/releases/download/v34.1/protoc-34.1-linux-x86_64.zip
|
|
sudo unzip -o /tmp/protoc.zip -d /usr/local bin/protoc 'include/*'
|
|
protoc --version
|
|
|
|
- name: Build NonWindows solution
|
|
run: dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx -c Release
|
|
|
|
# GitHub-hosted runners ship pwsh; the self-hosted act image does not. Install it as a
|
|
# .NET global tool (the SDK is already set up above) so the codegen check's `shell: pwsh` works.
|
|
- name: Install PowerShell (pwsh)
|
|
run: |
|
|
dotnet tool install --global PowerShell
|
|
echo "$HOME/.dotnet/tools" >> "$GITHUB_PATH"
|
|
|
|
# IPC-25 Check 4 regenerates the Go and Python client bindings and diffs them, so the pinned
|
|
# generators must be present. protoc 34.1 is already installed above; Go and Python are set up
|
|
# above. Pin protoc-gen-go / protoc-gen-go-grpc to match the committed header stamps and grpcio
|
|
# -tools to match the committed _pb2 stamp, or Check 4 false-fails (or masks drift) under churn.
|
|
- name: Install pinned client codegen generators (Check 4)
|
|
run: |
|
|
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
|
|
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.6.2
|
|
echo "$(go env GOPATH)/bin" >> "$GITHUB_PATH"
|
|
python -m pip install 'grpcio-tools==1.80.0'
|
|
|
|
# IPC-01 / IPC-19 / IPC-20 / IPC-25: descriptor set + Contracts/Generated + Go/Python bindings
|
|
# must match the current protos.
|
|
- name: Codegen / descriptor freshness
|
|
shell: pwsh
|
|
run: ./scripts/check-codegen.ps1
|
|
|
|
- name: Gateway fake-worker tests
|
|
run: dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj -c Release --no-build
|
|
|
|
- name: .NET client
|
|
run: dotnet build clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx -c Release
|
|
|
|
# The .NET client was previously only compiled here, so its test project — including
|
|
# MxGatewayClientWireTests, which drives the client against a real loopback gRPC
|
|
# server (TST-24) — never ran in CI. Every other client job already runs its tests.
|
|
- name: .NET client tests
|
|
run: dotnet test clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/ZB.MOM.WW.MxGateway.Client.Tests.csproj -c Release --no-build
|
|
|
|
- name: Go client
|
|
working-directory: clients/go
|
|
run: |
|
|
test -z "$(gofmt -l .)" || (gofmt -l . && echo 'gofmt needed' && exit 1)
|
|
go build ./...
|
|
go test ./...
|
|
|
|
- name: Rust client
|
|
working-directory: clients/rust
|
|
run: |
|
|
cargo fmt --all -- --check
|
|
cargo test --workspace
|
|
cargo clippy --workspace --all-targets -- -D warnings
|
|
|
|
- name: Python client
|
|
working-directory: clients/python
|
|
run: |
|
|
python -m pip install --upgrade pip
|
|
python -m pip install -e ".[dev]"
|
|
python -m pytest
|
|
|
|
java:
|
|
# Java client runs on a JDK-17 Linux runner (the macOS dev box has no JRE). The grpc/protobuf
|
|
# toolchain is fully pinned (clients/java/build.gradle: grpcVersion 1.76.0 / protobufVersion
|
|
# 4.33.1), so a regeneration is byte-identical to the committed aggregates modulo real .proto
|
|
# changes — `Verify generated tree is clean` (git diff) is the true drift gate (IPC-24). The
|
|
# single-file Java aggregates are where message-level proto drift lands, so this job now catches
|
|
# a .proto edited without regenerating and committing the Java client.
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: actions/setup-java@v4
|
|
with:
|
|
distribution: temurin
|
|
java-version: '17'
|
|
# GitHub-hosted runners ship gradle on PATH; the self-hosted act image does not, and the repo
|
|
# has no gradle wrapper. act also can't resolve the gradle/actions monorepo action, so install
|
|
# gradle directly (pinned to 9.5.1, matching the local homebrew build in CLAUDE.md/memory).
|
|
- name: Install Gradle 9.5.1
|
|
run: |
|
|
curl -sSL "https://services.gradle.org/distributions/gradle-9.5.1-bin.zip" -o /tmp/gradle.zip
|
|
sudo unzip -q -d /opt/gradle /tmp/gradle.zip
|
|
echo "/opt/gradle/gradle-9.5.1/bin" >> "$GITHUB_PATH"
|
|
- name: Gradle test
|
|
working-directory: clients/java
|
|
run: gradle test
|
|
- name: Verify generated tree is clean
|
|
# IPC-24: the pinned grpc/protobuf toolchain regenerates byte-identical output, so this
|
|
# git-diff gate now catches message-level proto drift in the single-file Java aggregates
|
|
# (the old unconditional churn-revert step masked exactly that class and was deleted).
|
|
run: git diff --exit-code -- clients/java/src/main/generated
|
|
|
|
windows-x86:
|
|
# TST-25: x86 / net48 Worker build + Worker.Tests, executed on windev (10.100.0.48) over SSH.
|
|
# Runs on a Linux runner so it ALWAYS schedules — act_runner host-mode on Windows is broken (its
|
|
# hostexecutor writes each step's script to a path it cannot resolve, and it cannot stage JS
|
|
# actions), and a `runs-on` gate with no matching runner wedges the queue in "queued" forever.
|
|
# An unreachable windev fails this job RED (loud), never leaves it stuck. A red job can mean the
|
|
# Windows tier is down rather than the change — see docs/GatewayTesting.md § Continuous Integration.
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- name: Worker x86 build + tests on windev
|
|
env:
|
|
# WINDEV_SSH_KEY is stored base64-encoded (single line) so Gitea's line-oriented secret
|
|
# masker redacts it in the step env echo; run-windev-ci.sh decodes it. A raw multiline PEM
|
|
# would leak in cleartext here (TST-25 acceptance: no key material in logs). Known-hosts is
|
|
# public and comes from the committed scripts/ci/windev.known_hosts pin (no secret needed).
|
|
WINDEV_SSH_KEY: ${{ secrets.WINDEV_SSH_KEY }}
|
|
WINDEV_SSH_USER: ${{ vars.WINDEV_SSH_USER }}
|
|
CI_SHA: ${{ github.sha }}
|
|
# `test` = x86 build + Worker.Tests (~50s measured on windev). Demote to `build` only if the
|
|
# test step ever exceeds ~10 min; the x86 build alone still guards the net48/CS0246/x86 class.
|
|
run: ./scripts/ci/run-windev-ci.sh test
|
|
|
|
nightly-windev:
|
|
# TST-25: scheduled deep run — full Worker.Tests + live-MXAccess smoke + full-slnx build on windev.
|
|
# Gated to the schedule event via `if:` (not an unsatisfiable `runs-on`), so push/PR runs mark it
|
|
# skipped, not queued. Never gates a push. On failure it opens a Gitea issue so a red nightly is
|
|
# visible even though nobody watches the Actions page.
|
|
if: github.event_name == 'schedule'
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- name: Full Worker.Tests + live-MXAccess smoke on windev
|
|
env:
|
|
# WINDEV_SSH_KEY is stored base64-encoded (single line) so Gitea's line-oriented secret
|
|
# masker redacts it in the step env echo; run-windev-ci.sh decodes it. A raw multiline PEM
|
|
# would leak in cleartext here (TST-25 acceptance: no key material in logs). Known-hosts is
|
|
# public and comes from the committed scripts/ci/windev.known_hosts pin (no secret needed).
|
|
WINDEV_SSH_KEY: ${{ secrets.WINDEV_SSH_KEY }}
|
|
WINDEV_SSH_USER: ${{ vars.WINDEV_SSH_USER }}
|
|
CI_SHA: ${{ github.sha }}
|
|
run: ./scripts/ci/run-windev-ci.sh live
|
|
- name: Report a red nightly as a Gitea issue
|
|
if: failure()
|
|
run: |
|
|
curl -sSf -X POST \
|
|
-H "Authorization: token ${{ github.token }}" \
|
|
-H "Content-Type: application/json" \
|
|
"${{ github.server_url }}/api/v1/repos/${{ github.repository }}/issues" \
|
|
-d "{\"title\":\"nightly-windev failed on ${{ github.sha }}\",\"body\":\"The scheduled windev Worker + live-MXAccess run failed: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} . A red nightly may mean the Windows tier is down rather than the change — see docs/GatewayTesting.md (Continuous Integration).\"}"
|
|
|
|
# NOTE: there is intentionally no native `windows` runner job. act_runner v0.6.1 host-mode on
|
|
# Windows is broken and Windows containers are impractical for the net48/x86/MXAccess Worker, so the
|
|
# Windows tier runs via the SSH-driven `windows-x86` (per-push) and `nightly-windev` (scheduled)
|
|
# jobs above. Restore a native job only alongside a Windows runner that actually works.
|