Joseph Doherty
6060d21995
ClientProtoInputTests.Descriptor_ContainsEveryContractMessageAndField only
compared messages and fields, and only enumerated the gateway/worker
descriptors, so a new enum value, a new RPC, or any galaxy_repository.proto-only
change would not redden the test even though it is documented as the primary
protoc-free CI gate.
Rename to Descriptor_ContainsEveryContractSymbol and extend the reflection walk
on both sides (published protoset and in-process contract) to also collect
enums/enum values ({enumFullName}, {enumFullName}/{valueName}) and
services/methods ({serviceFullName}, {serviceFullName}/{methodName}), and add
GalaxyRepositoryReflection.Descriptor to the enumerated files. The comparison
stays a flat, order-insensitive string-set diff with no protoc dependency.
Update docs/ClientProtoGeneration.md and docs/Contracts.md prose from
"message or field" to the full symbol coverage.
Red-path proof: pointed the test at the pre-IPC-01 stale protoset and confirmed
it failed naming max_frame_bytes, several MxCommandKind/AlarmProviderMode enum
values, MxAccessGateway/StreamAlarms and GalaxyRepository/BrowseChildren, and
the galaxy_repository.v1.* surface; restored the real path and re-ran green.
Flips IPC-27 to Done in the 2026-07-12 remediation tracker and register.
247 lines
11 KiB
Markdown
247 lines
11 KiB
Markdown
# Client Proto Generation
|
|
|
|
This document defines the stable protobuf inputs that official clients use to
|
|
generate language-specific gRPC bindings. The checked-in `.proto` files remain
|
|
the source of truth so clients do not drift from the gateway and worker
|
|
contracts.
|
|
|
|
## Stable Inputs
|
|
|
|
The stable client input manifest is `clients/proto/proto-inputs.json`. It
|
|
records:
|
|
|
|
- the public gateway protocol version,
|
|
- the worker IPC protocol version,
|
|
- the protobuf import root,
|
|
- the public and worker source files,
|
|
- the descriptor set path,
|
|
- golden fixture locations,
|
|
- behavior fixture locations,
|
|
- generated-code output directories for each planned client.
|
|
|
|
The source files listed by the manifest are:
|
|
|
|
- `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto`
|
|
- `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_worker.proto`
|
|
- `src/ZB.MOM.WW.MxGateway.Contracts/Protos/galaxy_repository.proto`
|
|
|
|
`mxaccess_gateway.proto` defines the public gRPC service and shared DTOs.
|
|
`mxaccess_worker.proto` is included in the descriptor because worker-aware
|
|
tests and fake-worker clients need the same command, reply, event, value, and
|
|
status shapes. `galaxy_repository.proto` defines the read-only Galaxy
|
|
Repository browse service used by clients to enumerate the deployed object
|
|
hierarchy and dynamic attributes; see
|
|
[Galaxy Repository Browse](./GalaxyRepository.md).
|
|
|
|
## Protocol Version
|
|
|
|
`GatewayContractInfo.GatewayProtocolVersion` is the public gateway protocol
|
|
version. `OpenSessionReply.gateway_protocol_version` returns the same value so
|
|
clients can compare their generated bindings against the gateway before issuing
|
|
MXAccess commands.
|
|
|
|
`GatewayContractInfo.WorkerProtocolVersion` remains the gateway-to-worker IPC
|
|
protocol version. It is also present in `OpenSessionReply` because parity
|
|
fixtures and fake-worker tests need to know the worker contract used by the
|
|
session.
|
|
|
|
## Descriptor Publishing
|
|
|
|
Run this command after changing either source `.proto` file or the client proto
|
|
manifest, with the **pinned protoc 34.1** (see [Toolchain Links](./ToolchainLinks.md)):
|
|
|
|
```powershell
|
|
pwsh -File scripts/publish-client-proto-inputs.ps1
|
|
```
|
|
|
|
The script writes
|
|
`clients/proto/descriptors/mxaccessgw-client-v1.protoset` with imports and
|
|
source information included. The descriptor is a generated artifact; do not edit
|
|
it by hand. Generating the committed artifact requires the pinned protoc version
|
|
so the checked-in bytes are reproducible; the script fails fast on a version
|
|
mismatch when generating.
|
|
|
|
Use the check mode in CI or before committing:
|
|
|
|
```powershell
|
|
pwsh -File scripts/publish-client-proto-inputs.ps1 -Check
|
|
```
|
|
|
|
`-Check` rebuilds the descriptor and fails when the checked-in descriptor is
|
|
stale. The comparison normalizes both the committed and freshly built descriptor
|
|
through the same protoc with `source_code_info` stripped, so it is tolerant of
|
|
protoc-version encoding drift and does not false-fail across protoc releases
|
|
(it warns, rather than fails, when protoc is off the pin).
|
|
|
|
The gateway test project carries an independent, protoc-free freshness guard:
|
|
`ClientProtoInputTests.Descriptor_ContainsEveryContractSymbol` reflects over the
|
|
in-process contract descriptors — `mxaccess_gateway.proto`, `mxaccess_worker.proto`,
|
|
and `galaxy_repository.proto` — and fails if any contract message, field, enum,
|
|
enum value, service, or method is missing from the committed protoset. This is
|
|
the primary CI gate for descriptor staleness; a red test means "regenerate and
|
|
commit the protoset."
|
|
|
|
### Pinned generator versions
|
|
|
|
Regeneration is reproducible only with the pinned toolchain. Regenerating with a
|
|
different version silently produces incompatible or noisy output, so the per-client
|
|
scripts assert the pin and resolve tools from `PATH`:
|
|
|
|
| Generator | Pinned version | Guard |
|
|
|-----------|----------------|-------|
|
|
| protoc (descriptor set) | 34.1 | version assertion in `scripts/publish-client-proto-inputs.ps1` |
|
|
| `Grpc.Tools` (C# `Generated/`) | 2.80.0 (contracts csproj) | `scripts/check-codegen.ps1` git-diff of `Generated/` (Check 2) |
|
|
| `protoc-gen-go` (Go) | v1.36.11 | version assertion in `clients/go/generate-proto.ps1`; `check-codegen.ps1` Check 4 |
|
|
| `protoc-gen-go-grpc` (Go) | 1.6.2 | version assertion in `clients/go/generate-proto.ps1`; `check-codegen.ps1` Check 4 |
|
|
| `grpcio-tools` (Python) | 1.80.0 (protobuf runtime 6.31.1) | version assertion in `clients/python/generate-proto.ps1`; `check-codegen.ps1` Check 4 |
|
|
| protobuf / grpc-java (Java) | `protobufVersion` / `grpcVersion` in `clients/java/build.gradle` | CI `git diff --exit-code` over `clients/java/src/main/generated` after `gradle test` (the `checkGeneratedClean` gradle task is the equivalent local check) |
|
|
|
|
A newer `grpcio-tools` stamps a `GRPC_GENERATED_VERSION` above the pinned grpcio
|
|
runtime and breaks Python `pytest`, so the Python and Go scripts assert their
|
|
generator pins before regenerating. Under the pinned grpc/protobuf toolchain the
|
|
Java protobuf plugin regenerates byte-identical output (modulo real `.proto`
|
|
changes), so CI enforces Java freshness with a direct `git diff --exit-code -- clients/java/src/main/generated`
|
|
step after `gradle test` (which transitively regenerates via `generateProto`); the
|
|
`checkGeneratedClean` gradle task is the equivalent check for local/manual use. The
|
|
old unconditional churn-revert CI step (which masked message-level drift in the
|
|
single-file Java aggregates) was deleted (IPC-24). Go and Python committed
|
|
bindings are guarded by `check-codegen.ps1` **Check 4**, which regenerates both
|
|
and fails on any diff.
|
|
|
|
## Output Directories
|
|
|
|
The manifest declares these generated-code directories:
|
|
|
|
| Client | Directory |
|
|
|--------|-----------|
|
|
| .NET | `clients/dotnet/generated` |
|
|
| Go | `clients/go/internal/generated` |
|
|
| Rust | `clients/rust/src/generated` |
|
|
| Python | `clients/python/src/zb_mom_ww_mxgateway/generated` |
|
|
| Java | `clients/java/src/main/generated` |
|
|
|
|
Only generator output belongs in these directories. Handwritten client wrappers
|
|
belong in the language-specific source trees created by the client scaffold
|
|
issues.
|
|
|
|
## Language Generation Inputs
|
|
|
|
All generators use `src/ZB.MOM.WW.MxGateway.Contracts/Protos` as the protobuf import
|
|
root. The checked-in descriptor is available when a language build prefers a
|
|
descriptor input, but the `.proto` files remain canonical.
|
|
|
|
Use these commands to regenerate language-specific client bindings:
|
|
|
|
| Client | Command |
|
|
|--------|---------|
|
|
| .NET | `dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj` |
|
|
| Go | `Push-Location clients/go; ./generate-proto.ps1; Pop-Location` |
|
|
| Rust | `Push-Location clients/rust; cargo check --workspace; Pop-Location` |
|
|
| Python | `Push-Location clients/python; ./generate-proto.ps1; Pop-Location` |
|
|
| Java | `Push-Location clients/java; gradle :zb-mom-ww-mxgateway-client:generateProto; Pop-Location` |
|
|
|
|
.NET generation currently runs through the contracts project:
|
|
|
|
```powershell
|
|
dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj
|
|
```
|
|
|
|
Future .NET client projects may either reference `ZB.MOM.WW.MxGateway.Contracts` or
|
|
generate client-local files into `clients/dotnet/generated` with `Grpc.Tools`.
|
|
|
|
Go clients should generate `mxaccess_gateway.proto` and
|
|
`mxaccess_worker.proto` into `clients/go/internal/generated` with
|
|
`protoc-gen-go` and `protoc-gen-go-grpc`. Keep generated packages internal
|
|
unless the wrapper API intentionally exposes raw protobuf messages.
|
|
|
|
The Go scaffold provides a repo-local generation script:
|
|
|
|
```powershell
|
|
clients/go/generate-proto.ps1
|
|
```
|
|
|
|
The script maps both proto files into the internal Go package
|
|
`gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated` because
|
|
the source `.proto` files do not carry Go-specific `go_package` options. This
|
|
keeps language-specific packaging outside the public contract files.
|
|
|
|
Rust clients use `tonic-build` from `clients/rust/build.rs`. The build script
|
|
reads the shared `.proto` files and emits generated `tonic`/`prost` modules
|
|
into Cargo build output. `clients/rust/src/generated.rs` contains the module
|
|
declarations that include those generated files. `clients/rust/src/generated`
|
|
remains reserved for checked-in generator output if the crate later changes to
|
|
source-tree generation, and handwritten wrapper code stays outside that
|
|
directory.
|
|
|
|
Run the Rust workspace checks from `clients/rust`:
|
|
|
|
```powershell
|
|
cargo fmt --all --check
|
|
cargo test --workspace
|
|
cargo check --workspace
|
|
```
|
|
|
|
Python clients should use `grpc_tools.protoc` and write generated modules under
|
|
`clients/python/src/zb_mom_ww_mxgateway/generated` so imports stay separate from
|
|
handwritten async wrappers.
|
|
|
|
The Python scaffold provides a repo-local generation script:
|
|
|
|
```powershell
|
|
clients/python/generate-proto.ps1
|
|
```
|
|
|
|
Java clients use the Gradle protobuf plugin from `clients/java`. The
|
|
`mxgateway-client` project reads the shared `.proto` files and writes generated
|
|
Java protobuf and gRPC sources under `clients/java/src/main/generated`, matching
|
|
the manifest output path. Handwritten client and CLI code stays in the
|
|
`mxgateway-client` and `mxgateway-cli` project source trees.
|
|
|
|
Run the Java workspace checks from `clients/java`:
|
|
|
|
```powershell
|
|
gradle test
|
|
```
|
|
|
|
## Golden Fixtures
|
|
|
|
Golden protobuf JSON fixtures live in `clients/proto/fixtures/golden`. They
|
|
exercise payloads that every language client must parse:
|
|
|
|
- `open-session-reply.ok.json`
|
|
- `register-command-request.json`
|
|
- `on-data-change-event.json`
|
|
|
|
The fixtures use protobuf JSON field names and enum values. Contract tests parse
|
|
them with the generated C# types so schema drift is caught before client
|
|
generation work starts.
|
|
|
|
## Behavior Fixtures
|
|
|
|
Cross-language behavior fixtures live in
|
|
`clients/proto/fixtures/behavior`. The manifest
|
|
`clients/proto/fixtures/behavior/manifest.json` lists command replies, ordered
|
|
event stream samples, value conversion cases, status conversion cases, auth
|
|
error expectations, and timeout/cancel expectations.
|
|
|
|
The behavior fixtures let each generated client wrapper test the same
|
|
expectations without a live gateway. Protobuf message fixtures parse with the
|
|
generated types. Auth and timeout/cancel files describe wrapper behavior above
|
|
the generated transport layer, including credential redaction and the rule that
|
|
client cancellation does not abort an in-flight MXAccess COM call.
|
|
|
|
Run the focused validation script after changing these fixtures:
|
|
|
|
```powershell
|
|
scripts/validate-client-behavior-fixtures.ps1
|
|
```
|
|
|
|
## Related Documentation
|
|
|
|
- [Protobuf Contracts](./Contracts.md)
|
|
- [Client Libraries Detailed Design](./ClientLibrariesDesign.md)
|
|
- [Client Packaging](./ClientPackaging.md)
|
|
- [Client Behavior Fixtures](./ClientBehaviorFixtures.md)
|
|
- [Client Libraries Implementation Plan](./ImplementationPlanClients.md)
|
|
- [Protobuf Style Guide](./style-guides/ProtobufStyleGuide.md)
|