dohertj2/mxaccessgw
1
0
Fork 0
Code Issues 14 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
Files
1c2f3a62c16d010b4a354da84fae780355f0a48c
mxaccessgw/docs/ClientProtoGeneration.md
T
Joseph Doherty eacdd2d453 fix(IPC-23,IPC-24,IPC-25,IPC-32): proto-comment regen wave + codegen-freshness guards 
Proto comments (comment-only, no wire change):
- mxaccess_worker.proto GatewayHello.max_frame_bytes: every worker->gateway frame
  must serialize within the negotiated max; reply builders truncate (IPC-23).
- mxaccess_gateway.proto DrainEventsReply: count-cap + byte-cap, drain-until-empty
  caller contract (IPC-23).
- mxaccess_gateway.proto ReplayGap.oldest_available_sequence: empty-ring value is
  highest-observed+1, oldest-1 resume formula stays valid (GWC-25 deferred amendment).

Regen wave: Contracts/Generated (C# XML doc), rust vendored protos (byte-copy),
Go bindings (worker binding was genuinely stale - lacked MaxFrameBytes entirely),
Python worker _pb2 (real descriptor delta), Java aggregates (javadoc, zero
protobuf-version churn under the pinned toolchain), client descriptor set.

IPC-24: pinned Java toolchain regenerates with no gencode-version churn, so the
unconditional churn-revert step in ci.yml is a fossil - deleted it; git diff is
now a true message-level drift gate for the single-file Java aggregates.

IPC-25: pin protoc-gen-go v1.36.11 / protoc-gen-go-grpc 1.6.2 in the Go generate
script (+ fix a latent pwsh-7 parse bug); add Check 4 to check-codegen.ps1
(regenerate Go+Python bindings, fail on diff, tool-missing fails not skips); add
the pinned-generator installs to the portable CI job.

IPC-32: relabel check-codegen banners 1/4..4/4 (folded into the Check 4 edit).

Docs: ClientProtoGeneration.md, Contracts.md, GatewayTesting.md, build.gradle
checkGeneratedClean caveat. Tracking: IPC-23/24/25/32 -> Done, GWC-25 proto note
resolved, change-log 2026-08-07.
2026-08-07 07:41:18 -04:00

245 lines
10 KiB
Markdown
Raw Blame History

# Client Proto Generation
This document defines the stable protobuf inputs that official clients use to
generate language-specific gRPC bindings. The checked-in `.proto` files remain
the source of truth so clients do not drift from the gateway and worker
contracts.
## Stable Inputs
The stable client input manifest is `clients/proto/proto-inputs.json`. It
records:
- the public gateway protocol version,
- the worker IPC protocol version,
- the protobuf import root,
- the public and worker source files,
- the descriptor set path,
- golden fixture locations,
- behavior fixture locations,
- generated-code output directories for each planned client.
The source files listed by the manifest are:
- `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto`
- `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_worker.proto`
- `src/ZB.MOM.WW.MxGateway.Contracts/Protos/galaxy_repository.proto`
`mxaccess_gateway.proto` defines the public gRPC service and shared DTOs.
`mxaccess_worker.proto` is included in the descriptor because worker-aware
tests and fake-worker clients need the same command, reply, event, value, and
status shapes. `galaxy_repository.proto` defines the read-only Galaxy
Repository browse service used by clients to enumerate the deployed object
hierarchy and dynamic attributes; see
[Galaxy Repository Browse](./GalaxyRepository.md).
## Protocol Version
`GatewayContractInfo.GatewayProtocolVersion` is the public gateway protocol
version. `OpenSessionReply.gateway_protocol_version` returns the same value so
clients can compare their generated bindings against the gateway before issuing
MXAccess commands.
`GatewayContractInfo.WorkerProtocolVersion` remains the gateway-to-worker IPC
protocol version. It is also present in `OpenSessionReply` because parity
fixtures and fake-worker tests need to know the worker contract used by the
session.
## Descriptor Publishing
Run this command after changing either source `.proto` file or the client proto
manifest, with the **pinned protoc 34.1** (see [Toolchain Links](./ToolchainLinks.md)):
```powershell
pwsh -File scripts/publish-client-proto-inputs.ps1
```
The script writes
`clients/proto/descriptors/mxaccessgw-client-v1.protoset` with imports and
source information included. The descriptor is a generated artifact; do not edit
it by hand. Generating the committed artifact requires the pinned protoc version
so the checked-in bytes are reproducible; the script fails fast on a version
mismatch when generating.
Use the check mode in CI or before committing:
```powershell
pwsh -File scripts/publish-client-proto-inputs.ps1 -Check
```
`-Check` rebuilds the descriptor and fails when the checked-in descriptor is
stale. The comparison normalizes both the committed and freshly built descriptor
through the same protoc with `source_code_info` stripped, so it is tolerant of
protoc-version encoding drift and does not false-fail across protoc releases
(it warns, rather than fails, when protoc is off the pin).
The gateway test project carries an independent, protoc-free freshness guard:
`ClientProtoInputTests.Descriptor_ContainsEveryContractMessageAndField` reflects
over the in-process contract descriptors and fails if any contract message or
field is missing from the committed protoset. This is the primary CI gate for
descriptor staleness; a red test means "regenerate and commit the protoset."
### Pinned generator versions
Regeneration is reproducible only with the pinned toolchain. Regenerating with a
different version silently produces incompatible or noisy output, so the per-client
scripts assert the pin and resolve tools from `PATH`:
| Generator | Pinned version | Guard |
|-----------|----------------|-------|
| protoc (descriptor set) | 34.1 | version assertion in `scripts/publish-client-proto-inputs.ps1` |
| `Grpc.Tools` (C# `Generated/`) | 2.80.0 (contracts csproj) | `scripts/check-codegen.ps1` git-diff of `Generated/` (Check 2) |
| `protoc-gen-go` (Go) | v1.36.11 | version assertion in `clients/go/generate-proto.ps1`; `check-codegen.ps1` Check 4 |
| `protoc-gen-go-grpc` (Go) | 1.6.2 | version assertion in `clients/go/generate-proto.ps1`; `check-codegen.ps1` Check 4 |
| `grpcio-tools` (Python) | 1.80.0 (protobuf runtime 6.31.1) | version assertion in `clients/python/generate-proto.ps1`; `check-codegen.ps1` Check 4 |
| protobuf / grpc-java (Java) | `protobufVersion` / `grpcVersion` in `clients/java/build.gradle` | CI `git diff --exit-code` over `clients/java/src/main/generated` after `gradle test` (the `checkGeneratedClean` gradle task is the equivalent local check) |
A newer `grpcio-tools` stamps a `GRPC_GENERATED_VERSION` above the pinned grpcio
runtime and breaks Python `pytest`, so the Python and Go scripts assert their
generator pins before regenerating. Under the pinned grpc/protobuf toolchain the
Java protobuf plugin regenerates byte-identical output (modulo real `.proto`
changes), so CI enforces Java freshness with a direct `git diff --exit-code -- clients/java/src/main/generated`
step after `gradle test` (which transitively regenerates via `generateProto`); the
`checkGeneratedClean` gradle task is the equivalent check for local/manual use. The
old unconditional churn-revert CI step (which masked message-level drift in the
single-file Java aggregates) was deleted (IPC-24). Go and Python committed
bindings are guarded by `check-codegen.ps1` **Check 4**, which regenerates both
and fails on any diff.
## Output Directories
The manifest declares these generated-code directories:
| Client | Directory |
|--------|-----------|
| .NET | `clients/dotnet/generated` |
| Go | `clients/go/internal/generated` |
| Rust | `clients/rust/src/generated` |
| Python | `clients/python/src/zb_mom_ww_mxgateway/generated` |
| Java | `clients/java/src/main/generated` |
Only generator output belongs in these directories. Handwritten client wrappers
belong in the language-specific source trees created by the client scaffold
issues.
## Language Generation Inputs
All generators use `src/ZB.MOM.WW.MxGateway.Contracts/Protos` as the protobuf import
root. The checked-in descriptor is available when a language build prefers a
descriptor input, but the `.proto` files remain canonical.
Use these commands to regenerate language-specific client bindings:
| Client | Command |
|--------|---------|
| .NET | `dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj` |
| Go | `Push-Location clients/go; ./generate-proto.ps1; Pop-Location` |
| Rust | `Push-Location clients/rust; cargo check --workspace; Pop-Location` |
| Python | `Push-Location clients/python; ./generate-proto.ps1; Pop-Location` |
| Java | `Push-Location clients/java; gradle :zb-mom-ww-mxgateway-client:generateProto; Pop-Location` |
.NET generation currently runs through the contracts project:
```powershell
dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj
```
Future .NET client projects may either reference `ZB.MOM.WW.MxGateway.Contracts` or
generate client-local files into `clients/dotnet/generated` with `Grpc.Tools`.
Go clients should generate `mxaccess_gateway.proto` and
`mxaccess_worker.proto` into `clients/go/internal/generated` with
`protoc-gen-go` and `protoc-gen-go-grpc`. Keep generated packages internal
unless the wrapper API intentionally exposes raw protobuf messages.
The Go scaffold provides a repo-local generation script:
```powershell
clients/go/generate-proto.ps1
```
The script maps both proto files into the internal Go package
`gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated` because
the source `.proto` files do not carry Go-specific `go_package` options. This
keeps language-specific packaging outside the public contract files.
Rust clients use `tonic-build` from `clients/rust/build.rs`. The build script
reads the shared `.proto` files and emits generated `tonic`/`prost` modules
into Cargo build output. `clients/rust/src/generated.rs` contains the module
declarations that include those generated files. `clients/rust/src/generated`
remains reserved for checked-in generator output if the crate later changes to
source-tree generation, and handwritten wrapper code stays outside that
directory.
Run the Rust workspace checks from `clients/rust`:
```powershell
cargo fmt --all --check
cargo test --workspace
cargo check --workspace
```
Python clients should use `grpc_tools.protoc` and write generated modules under
`clients/python/src/zb_mom_ww_mxgateway/generated` so imports stay separate from
handwritten async wrappers.
The Python scaffold provides a repo-local generation script:
```powershell
clients/python/generate-proto.ps1
```
Java clients use the Gradle protobuf plugin from `clients/java`. The
`mxgateway-client` project reads the shared `.proto` files and writes generated
Java protobuf and gRPC sources under `clients/java/src/main/generated`, matching
the manifest output path. Handwritten client and CLI code stays in the
`mxgateway-client` and `mxgateway-cli` project source trees.
Run the Java workspace checks from `clients/java`:
```powershell
gradle test
```
## Golden Fixtures
Golden protobuf JSON fixtures live in `clients/proto/fixtures/golden`. They
exercise payloads that every language client must parse:
- `open-session-reply.ok.json`
- `register-command-request.json`
- `on-data-change-event.json`
The fixtures use protobuf JSON field names and enum values. Contract tests parse
them with the generated C# types so schema drift is caught before client
generation work starts.
## Behavior Fixtures
Cross-language behavior fixtures live in
`clients/proto/fixtures/behavior`. The manifest
`clients/proto/fixtures/behavior/manifest.json` lists command replies, ordered
event stream samples, value conversion cases, status conversion cases, auth
error expectations, and timeout/cancel expectations.
The behavior fixtures let each generated client wrapper test the same
expectations without a live gateway. Protobuf message fixtures parse with the
generated types. Auth and timeout/cancel files describe wrapper behavior above
the generated transport layer, including credential redaction and the rule that
client cancellation does not abort an in-flight MXAccess COM call.
Run the focused validation script after changing these fixtures:
```powershell
scripts/validate-client-behavior-fixtures.ps1
```
## Related Documentation
- [Protobuf Contracts](./Contracts.md)
- [Client Libraries Detailed Design](./ClientLibrariesDesign.md)
- [Client Packaging](./ClientPackaging.md)
- [Client Behavior Fixtures](./ClientBehaviorFixtures.md)
- [Client Libraries Implementation Plan](./ImplementationPlanClients.md)
- [Protobuf Style Guide](./style-guides/ProtobufStyleGuide.md)
Reference in New Issue View Git Blame Copy Permalink