ci(codegen): Check 3 sweeps both directions — canonical protos missing a vendored copy now fail

This commit is contained in:
Joseph Doherty
2026-08-17 07:04:49 -04:00
parent 338bc20a0a
commit 711793ffdc
4 changed files with 29 additions and 8 deletions
+7 -5
View File
@@ -28,11 +28,13 @@ tarball, where the rest of the mxaccessgw repo does not exist. The vendored
copies are shipped in the published `.crate` via `Cargo.toml`'s `include` copies are shipped in the published `.crate` via `Cargo.toml`'s `include`
list, which is what makes the crate buildable standalone; they are build list, which is what makes the crate buildable standalone; they are build
inputs only, never a second source of truth. **Refresh rule:** any commit inputs only, never a second source of truth. **Refresh rule:** any commit
that edits a Contracts proto (`mxaccess_gateway.proto`, `mxaccess_worker.proto`, that edits *or adds* a Contracts proto (today `mxaccess_gateway.proto`,
`galaxy_repository.proto`) must copy the changed file(s) into `mxaccess_worker.proto`, `galaxy_repository.proto`) must copy the changed or
`clients/rust/protos/` in that same commit — `scripts/check-codegen.ps1` new file(s) into `clients/rust/protos/` in that same commit —
Check 3 fails the build on byte drift between the vendored copies and the `scripts/check-codegen.ps1` Check 3 fails the build on byte drift between the
canonical Contracts protos. `tonic`/`prost` bindings are generated into vendored copies and the canonical Contracts protos, and equally on a canonical
proto that has no vendored copy (which an in-repo build cannot catch, since
`build.rs` reads the canonical directory here). `tonic`/`prost` bindings are generated into
Cargo build output. `src/generated.rs` declares the Rust modules that Cargo build output. `src/generated.rs` declares the Rust modules that
include those generated files. `src/generated` remains reserved for checked-in include those generated files. `src/generated` remains reserved for checked-in
generator output if the crate later changes to source-tree generation. generator output if the crate later changes to source-tree generation.
+2 -1
View File
@@ -206,7 +206,8 @@ publish` ship them inside the `.crate`, making the crate buildable standalone
with no access to the rest of the mxaccessgw repo. Any Contracts proto change with no access to the rest of the mxaccessgw repo. Any Contracts proto change
must refresh `clients/rust/protos/` in the same commit; `scripts/check-codegen.ps1` must refresh `clients/rust/protos/` in the same commit; `scripts/check-codegen.ps1`
Check 3 byte-compares the vendored copies against the canonical protos and Check 3 byte-compares the vendored copies against the canonical protos and
fails on drift. Because the vendored protos make a standalone build possible, fails on drift — in both directions, so a newly added canonical proto that
was never vendored fails there rather than at a consumer's standalone build. Because the vendored protos make a standalone build possible,
`cargo package`/`cargo publish` run **with** verification (no `--no-verify`) — `cargo package`/`cargo publish` run **with** verification (no `--no-verify`) —
a `cargo package` that cannot build from the vendored tree alone would mean a `cargo package` that cannot build from the vendored tree alone would mean
the vendored copies are stale, and verification is what catches that before the vendored copies are stale, and verification is what catches that before
+5
View File
@@ -689,6 +689,11 @@ committed client descriptor set (Check 1), the C# `Generated/` (Check 2), the Ru
protos (Check 3), or the Go/Python client bindings (Check 4, IPC-25) no longer match the current protos (Check 3), or the Go/Python client bindings (Check 4, IPC-25) no longer match the current
`.proto` sources — the codegen drift class this repo has hit repeatedly (stale client `.proto` sources — the codegen drift class this repo has hit repeatedly (stale client
descriptors, net48 `CS0246` on unregenerated protos, silently stale Go/Python worker bindings). descriptors, net48 `CS0246` on unregenerated protos, silently stale Go/Python worker bindings).
Check 3 sweeps both directions: a vendored copy that drifted from (or has no) canonical proto
fails, and so does a canonical proto with no vendored copy at all. The second direction matters
because `clients/rust/build.rs` prefers the canonical directory whenever it exists, so an
unvendored proto builds fine in-repo and only breaks the standalone crate build a consumer runs
from the published tarball — drift no in-repo build can surface.
Check 4 regenerates the Go and Python bindings with their pinned generators (`protoc-gen-go` Check 4 regenerates the Go and Python bindings with their pinned generators (`protoc-gen-go`
v1.36.11 / `protoc-gen-go-grpc` 1.6.2, `grpcio-tools` 1.80.0) and fails on any diff; a missing v1.36.11 / `protoc-gen-go-grpc` 1.6.2, `grpcio-tools` 1.80.0) and fails on any diff; a missing
generator fails the check rather than skipping it. The **primary** guard for the generator fails the check rather than skipping it. The **primary** guard for the
+15 -2
View File
@@ -11,8 +11,11 @@
# (which breaks the net48 worker build with CS0246 — see docs/Contracts.md). # (which breaks the net48 worker build with CS0246 — see docs/Contracts.md).
# 3. The Rust crate's vendored protos (clients/rust/protos/*.proto — build inputs that make the # 3. The Rust crate's vendored protos (clients/rust/protos/*.proto — build inputs that make the
# crate buildable outside the repo, CLI-02) are byte-identical to the canonical Contracts # crate buildable outside the repo, CLI-02) are byte-identical to the canonical Contracts
# protos. A drift means a .proto was edited without refreshing the vendored copies, which would # protos, and the two directories hold the same set of files. A drift means a .proto was edited
# publish a stale wire contract to crate consumers while the in-repo build stays correct. # without refreshing the vendored copies, which would publish a stale wire contract to crate
# consumers while the in-repo build stays correct. The sweep runs both directions: a canonical
# proto with no vendored copy is just as broken (the published crate cannot build standalone),
# and it is invisible in-repo because build.rs prefers the canonical directory when it exists.
# 4. The committed Go and Python client bindings match a fresh regeneration (IPC-25). The two # 4. The committed Go and Python client bindings match a fresh regeneration (IPC-25). The two
# per-client generate-proto.ps1 scripts pin their generators (protoc-gen-go v1.36.11 / # per-client generate-proto.ps1 scripts pin their generators (protoc-gen-go v1.36.11 /
# protoc-gen-go-grpc v1.6.2 for Go; grpcio-tools 1.80.0 for Python), so a clean checkout # protoc-gen-go-grpc v1.6.2 for Go; grpcio-tools 1.80.0 for Python), so a clean checkout
@@ -88,6 +91,16 @@ try {
$failures.Add("Rust vendored proto drifted from canonical: clients/rust/protos/$($vendored.Name). Refresh it from src/ZB.MOM.WW.MxGateway.Contracts/Protos/$($vendored.Name).") $failures.Add("Rust vendored proto drifted from canonical: clients/rust/protos/$($vendored.Name). Refresh it from src/ZB.MOM.WW.MxGateway.Contracts/Protos/$($vendored.Name).")
} }
} }
# Reverse direction: a canonical proto that was never vendored passes the loop above (it only
# walks the vendored dir) but breaks a standalone crate build, because build.rs falls back to
# clients/rust/protos/ only outside the repo — in-repo it reads the canonical dir and stays green.
foreach ($canonicalProto in Get-ChildItem -Path $canonicalProtoDir -Filter '*.proto' -File) {
$vendoredCounterpart = Join-Path $vendoredProtoDir $canonicalProto.Name
if (-not (Test-Path $vendoredCounterpart)) {
$failures.Add("Canonical proto is not vendored for the Rust crate: $($canonicalProto.Name). Copy src/ZB.MOM.WW.MxGateway.Contracts/Protos/$($canonicalProto.Name) to clients/rust/protos/$($canonicalProto.Name) (and add it to build.rs's input list).")
}
}
} }
catch { catch {
$failures.Add("Rust vendored proto check failed: $($_.Exception.Message)") $failures.Add("Rust vendored proto check failed: $($_.Exception.Message)")