dohertj2/mxaccessgw
1
0
Fork 0
Code Issues 14 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

fix(IPC-23,IPC-24,IPC-25,IPC-32): proto-comment regen wave + codegen-freshness guards

Browse Source 
Proto comments (comment-only, no wire change):
- mxaccess_worker.proto GatewayHello.max_frame_bytes: every worker->gateway frame
  must serialize within the negotiated max; reply builders truncate (IPC-23).
- mxaccess_gateway.proto DrainEventsReply: count-cap + byte-cap, drain-until-empty
  caller contract (IPC-23).
- mxaccess_gateway.proto ReplayGap.oldest_available_sequence: empty-ring value is
  highest-observed+1, oldest-1 resume formula stays valid (GWC-25 deferred amendment).

Regen wave: Contracts/Generated (C# XML doc), rust vendored protos (byte-copy),
Go bindings (worker binding was genuinely stale - lacked MaxFrameBytes entirely),
Python worker _pb2 (real descriptor delta), Java aggregates (javadoc, zero
protobuf-version churn under the pinned toolchain), client descriptor set.

IPC-24: pinned Java toolchain regenerates with no gencode-version churn, so the
unconditional churn-revert step in ci.yml is a fossil - deleted it; git diff is
now a true message-level drift gate for the single-file Java aggregates.

IPC-25: pin protoc-gen-go v1.36.11 / protoc-gen-go-grpc 1.6.2 in the Go generate
script (+ fix a latent pwsh-7 parse bug); add Check 4 to check-codegen.ps1
(regenerate Go+Python bindings, fail on diff, tool-missing fails not skips); add
the pinned-generator installs to the portable CI job.

IPC-32: relabel check-codegen banners 1/4..4/4 (folded into the Check 4 edit).

Docs: ClientProtoGeneration.md, Contracts.md, GatewayTesting.md, build.gradle
checkGeneratedClean caveat. Tracking: IPC-23/24/25/32 -> Done, GWC-25 proto note
resolved, change-log 2026-08-07.
This commit is contained in:
Joseph Doherty
2026-08-07 07:32:22 -04:00
parent 97f79e79ef
commit eacdd2d453
22 changed files with 469 additions and 82 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ClientProtoGeneration.md
+15 -7
View File
@@ -88,15 +88,23 @@ scripts assert the pin and resolve tools from `PATH`:
| Generator | Pinned version | Guard |
|-----------|----------------|-------|
| protoc (descriptor set) | 34.1 | version assertion in `scripts/publish-client-proto-inputs.ps1` |
| `Grpc.Tools` (C# `Generated/`) | 2.80.0 (contracts csproj) | `scripts/check-codegen.ps1` git-diff of `Generated/` |
| `grpcio-tools` (Python) | 1.80.0 (protobuf runtime 6.31.1) | version assertion in `clients/python/generate-proto.ps1` |
| protobuf / grpc-java (Java) | `protobufVersion` / `grpcVersion` in `clients/java/build.gradle` | `checkGeneratedClean` gradle task |
| `Grpc.Tools` (C# `Generated/`) | 2.80.0 (contracts csproj) | `scripts/check-codegen.ps1` git-diff of `Generated/` (Check 2) |
| `protoc-gen-go` (Go) | v1.36.11 | version assertion in `clients/go/generate-proto.ps1`; `check-codegen.ps1` Check 4 |
| `protoc-gen-go-grpc` (Go) | 1.6.2 | version assertion in `clients/go/generate-proto.ps1`; `check-codegen.ps1` Check 4 |
| `grpcio-tools` (Python) | 1.80.0 (protobuf runtime 6.31.1) | version assertion in `clients/python/generate-proto.ps1`; `check-codegen.ps1` Check 4 |
| protobuf / grpc-java (Java) | `protobufVersion` / `grpcVersion` in `clients/java/build.gradle` | CI `git diff --exit-code` over `clients/java/src/main/generated` after `gradle test` (the `checkGeneratedClean` gradle task is the equivalent local check) |
A newer `grpcio-tools` stamps a `GRPC_GENERATED_VERSION` above the pinned grpcio
runtime and breaks Python `pytest`; the Java protobuf plugin rewrites
`MxaccessGateway.java` with spurious protobuf-runtime-version churn on every build
(revert that one file when no `.proto` changed — see
[Gateway Testing](./GatewayTesting.md) "Continuous Integration").
runtime and breaks Python `pytest`, so the Python and Go scripts assert their
generator pins before regenerating. Under the pinned grpc/protobuf toolchain the
Java protobuf plugin regenerates byte-identical output (modulo real `.proto`
changes), so CI enforces Java freshness with a direct `git diff --exit-code -- clients/java/src/main/generated`
step after `gradle test` (which transitively regenerates via `generateProto`); the
`checkGeneratedClean` gradle task is the equivalent check for local/manual use. The
old unconditional churn-revert CI step (which masked message-level drift in the
single-file Java aggregates) was deleted (IPC-24). Go and Python committed
bindings are guarded by `check-codegen.ps1` **Check 4**, which regenerates both
and fails on any diff.
## Output Directories
docs/Contracts.md
+5 -1
View File
@@ -114,7 +114,11 @@ dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csp
`scripts/check-codegen.ps1` enforces this in CI (it force-regenerates and fails on any
`git diff` against the committed `Generated/`) — that regeneration diff in the `portable`
job is the primary guard. The SSH-driven `windows-x86` job's net48 worker build is the
secondary guard (a stale `Generated/` also breaks the x86 build with `CS0246`).
secondary guard (a stale `Generated/` also breaks the x86 build with `CS0246`). The same
script runs four checks in total: the committed client descriptor set (Check 1), the C#
`Generated/` (Check 2), the Rust vendored protos (Check 3), and the Go/Python client bindings
(Check 4) each regenerate and fail on any diff. See
[Client Proto Generation](./ClientProtoGeneration.md) for the pinned generator versions.
Client generation inputs are published through
`clients/proto/proto-inputs.json` and the descriptor set under
docs/GatewayTesting.md
+15 -8
View File
@@ -428,10 +428,13 @@ runtime because the x86 Worker cannot build on Linux:
gateway fake-worker tests, and builds/tests the clients that run on Linux: .NET client
build, Go (`gofmt` + `go build` + `go test`), Rust (`cargo fmt --check` + `cargo test`
+ `cargo clippy -D warnings`), and Python (`pytest`).
- **`java`** (Linux, JDK 17) — `gradle test`. The protobuf gradle plugin rewrites
`MxaccessGateway.java` with spurious protobuf-runtime-version churn on every build, so
when no `.proto` changed the job reverts that one file (`git checkout`) before asserting
the generated tree is clean. The dev Mac has no JRE, so Java verification is CI-only.
- **`java`** (Linux, JDK 17) — `gradle test`, then `git diff --exit-code` over the generated
tree. The grpc/protobuf toolchain is pinned (`clients/java/build.gradle`), so a regeneration
is byte-identical to the committed single-file aggregates modulo real `.proto` changes; the
git-diff is therefore a true drift gate that now catches message-level proto drift in the Java
client (IPC-24 deleted the old unconditional churn-revert step, which masked exactly that
class). The dev Mac has a Homebrew JDK 17, so `gradle generateProto` can be run there to refresh
the Java aggregates when a `.proto` changes.
- **`windows-x86`** (Linux runner, per push/PR) — builds the **x86 / net48 Worker and
Worker.Tests**, which are Windows-only and out of scope for the Linux jobs. It runs on a
Linux runner that always schedules and SSHes to windev (`10.100.0.48`), where
@@ -449,10 +452,14 @@ runtime because the x86 Worker cannot build on Linux:
`xUnit1030`). Gated `if: github.event_name == 'schedule'`, so it never gates a push. On
failure it opens a Gitea issue via the Actions token, since nobody watches the Actions page.
The freshness guard `scripts/check-codegen.ps1` fails the build when the committed
client descriptor set or the C# `Generated/` no longer matches the current `.proto`
sources — the codegen drift class this repo has hit repeatedly (stale client
descriptors, net48 `CS0246` on unregenerated protos). The **primary** guard for the
The freshness guard `scripts/check-codegen.ps1` runs four checks and fails the build when the
committed client descriptor set (Check 1), the C# `Generated/` (Check 2), the Rust vendored
protos (Check 3), or the Go/Python client bindings (Check 4, IPC-25) no longer match the current
`.proto` sources — the codegen drift class this repo has hit repeatedly (stale client
descriptors, net48 `CS0246` on unregenerated protos, silently stale Go/Python worker bindings).
Check 4 regenerates the Go and Python bindings with their pinned generators (`protoc-gen-go`
v1.36.11 / `protoc-gen-go-grpc` 1.6.2, `grpcio-tools` 1.80.0) and fails on any diff; a missing
generator fails the check rather than skipping it. The **primary** guard for the
"regenerate and commit `Generated/`" rule is that regeneration diff in the `portable` job;
the `windows-x86` net48 compile is the **secondary** guard (a stale `Generated/` also breaks
the x86 build with `CS0246`). See [Client Proto Generation](./ClientProtoGeneration.md) and