Joseph Doherty
44b8e37900
An empty replay ring reported oldest_available_sequence = 0 even when gap was
true. Clients follow the documented after_worker_sequence = oldest - 1 formula,
so an unsigned client computed ulong.MaxValue: the follow-up resume replayed
nothing, reported no gap, and the live filter dropped every subsequent event —
a silently dead stream in the headline detach-and-resume scenario, reachable on
default config once ReplayRetentionSeconds (300) age-evicts the ring.
GWC-25: SessionEventDistributor.RegisterWithReplay's empty-ring branch now
reports _highestSequenceSeen + 1 — the next sequence that can possibly be
delivered — when gap is true, so oldest - 1 lands exactly on the highest
observed sequence and the resume delivers everything newer. Still 0 when there
is no gap, where the field is meaningless and never emitted. Nothing is lost:
the evicted interval was unrecoverable either way, and the sentinel's job is to
say "re-snapshot".
CLI-35: the Python CLI fed every stream item into MessageToDict, which raised on
the ReplayGap dataclass and aborted the command after consuming the stream. A
new _event_row helper renders a gap as {"replayGap": {...}} — the same camelCase
shape the Rust CLI emits — and leaves proto events on the existing path.
CLI-36: the Go CLI formatted result.Event on every row, but the library
deliberately clears Event on a gap, so text mode printed
"0 MX_EVENT_FAMILY_UNSPECIFIED" and JSON mode an empty object, discarding the
resume cursors. The loop now branches on result.IsReplayGap() and renders the
typed row in both modes, counting it toward -limit like any other row. The JSON
row's cursors are typed by hand rather than marshalled with protojson: the
proto3 JSON mapping renders 64-bit integers as strings ("7") while the Rust and
Python CLIs emit numbers (7), so going through protojson would have made Go the
only canonical CLI with a different value type.
Docs in the same change: docs/Sessions.md documents the empty-ring sentinel
value and that oldest - 1 is the universal resume formula in both the retained
and fully-evicted cases; docs/CrossLanguageSmokeMatrix.md gains a per-CLI
gap-rendering table covering both client findings, and records exactly what is
and is not comparable across CLIs (same keys and numeric cursors for Rust/Go/
Python; quoted cursors for .NET/Java; differing key order, whitespace, and
container), so a matrix runner compares parsed values rather than raw bytes.
Tests, all written red first and each reproducing its defect verbatim:
- SessionEventDistributorTests: RegisterWithReplayReportsNextDeliverableSequence
WhenRingEmptiedByAge, ...WithRetentionDisabled, and
ResumeUsingSentinelFormulaAfterEmptyRingGapDeliversLiveEvents.
- GatewayEndToEndReconnectReplayTests.ReconnectAfterFullAgeEvictionResumesWith
SentinelFormula — fake-worker e2e resume walk on a fake clock; the fixture now
takes a retention window and a TimeProvider.
- clients/python test_stream_events_renders_replay_gap.
- clients/go TestRunStreamEventsPrintsReplayGap.
GWC-25's ReplayGap.oldest_available_sequence proto-comment amendment is
deliberately deferred to the later codegen wave (see the tracker change log): it
is comment-only but triggers the full five-client regen fan-out.
151 lines
6.6 KiB
Markdown
151 lines
6.6 KiB
Markdown
# Cross-Language Smoke Matrix
|
|
|
|
The cross-language smoke matrix defines the documented commands used to compare
|
|
official clients against the same live gateway flow. It is a repository
|
|
validation fixture and command reference; normal unit tests validate the matrix
|
|
shape without connecting to a gateway.
|
|
|
|
The matrix lives in
|
|
`clients/proto/fixtures/smoke/cross-language-smoke-matrix.json`.
|
|
|
|
## Scope
|
|
|
|
The matrix covers the supported client languages:
|
|
|
|
- .NET
|
|
- Go
|
|
- Rust
|
|
- Python
|
|
- Java
|
|
|
|
Each client entry defines commands for the same required operation sequence:
|
|
|
|
1. `open-session`
|
|
2. `register`
|
|
3. `add-item`
|
|
4. `advise`
|
|
5. `stream-events`
|
|
6. `close-session`
|
|
|
|
The optional `write` command is documented separately because writing changes
|
|
provider state and should only run when the operator supplies a safe test value.
|
|
|
|
When `stream-events` is resumed with an `after_worker_sequence` cursor that
|
|
predates the oldest event still in the gateway's replay ring, the gateway emits a
|
|
single `ReplayGap` sentinel at the head of the stream. Every client surfaces this
|
|
as a distinct, typed, non-terminal signal (see each client README); the resume
|
|
contract is `after_worker_sequence = oldest_available_sequence - 1`, and it holds
|
|
even when the ring has been emptied entirely by age eviction — the gateway then
|
|
reports the next deliverable sequence rather than `0` (see [Sessions](Sessions.md)).
|
|
The default smoke sequence opens a fresh stream (no cursor) and does not exercise
|
|
the gap path; a resume-with-gap fixture case is tracked separately (TST-24).
|
|
|
|
The CLIs differ in how they *print* that library-level signal. Three of them consume
|
|
the typed gap and emit a dedicated row rather than a degenerate event row; the other
|
|
two hand the raw sentinel `MxEvent` straight to the formatter, so they print the
|
|
sentinel itself, whose `replayGap` field carries the same cursors:
|
|
|
|
| CLI | Text mode | JSON mode |
|
|
|-----|-----------|-----------|
|
|
| `mxgw-rs` (Rust, canonical) | `REPLAY_GAP requested_after=<n> oldest_available=<n>` | `{"replayGap": {"requestedAfterSequence": <n>, "oldestAvailableSequence": <n>}}` as one entry of the `events` array |
|
|
| `mxgw-go` (Go) | `REPLAY_GAP requested_after=<n> oldest_available=<n>` | one `{"replayGap": {"requestedAfterSequence": <n>, "oldestAvailableSequence": <n>}}` line, counted toward `-limit` like any other row |
|
|
| `mxgw-py` (Python) | same JSON dump as `--json` | `{"replayGap": {"requestedAfterSequence": <n>, "oldestAvailableSequence": <n>}}` as one entry of the `events` array |
|
|
| `mxgw-dotnet` (.NET) | the raw sentinel `MxEvent` as protobuf JSON, including its `replayGap` field | same, as one entry of the `events` array |
|
|
| `mxgw-java` (Java) | the sentinel's `worker_sequence` and `family` (`0 MX_EVENT_FAMILY_UNSPECIFIED`) | the raw sentinel `MxEvent` as protobuf JSON, including its `replayGap` field |
|
|
|
|
Rust, Go, and Python emit the same two key names and, deliberately, the same JSON
|
|
value **types**: the cursors are JSON numbers (`7`), not strings. That is why the
|
|
Go CLI types the row by hand instead of marshalling `ReplayGap` with `protojson` —
|
|
the proto3 JSON mapping renders 64-bit integers as strings (`"7"`), which is also
|
|
why the .NET and Java rows, which pass the sentinel through a protobuf JSON
|
|
formatter, carry **quoted** cursors. A matrix runner must therefore compare parsed
|
|
values, not raw bytes, and must not assume the same value type across all five
|
|
CLIs.
|
|
|
|
Two further formatting differences among the three canonical CLIs, none of them
|
|
semantic: Python sorts object keys and uses `", "` / `": "` separators
|
|
(`json.dumps(..., sort_keys=True)`), while Rust and Go emit compact,
|
|
declaration-ordered JSON; and the row sits alone on its own line for Go and for
|
|
Rust's `--jsonl`, but inside an `events` array for Python and for Rust's
|
|
aggregate `--json`.
|
|
|
|
## Integration Gate
|
|
|
|
Cross-language smoke execution is opt-in. Runners should skip the matrix unless
|
|
this variable is set:
|
|
|
|
```powershell
|
|
$env:MXGATEWAY_INTEGRATION = "1"
|
|
```
|
|
|
|
The shared inputs are:
|
|
|
|
| Variable | Default | Purpose |
|
|
|----------|---------|---------|
|
|
| `MXGATEWAY_ENDPOINT` | `localhost:5000` | Gateway endpoint used by client CLIs. |
|
|
| `MXGATEWAY_API_KEY` | Empty | API key source for authenticated gateway deployments. |
|
|
| `MXGATEWAY_TEST_ITEM` | `TestChildObject.TestInt` | MXAccess item used by `add-item`. |
|
|
| `MXGATEWAY_TEST_WRITE_VALUE` | Empty | Enables the optional write step when set by a runner. |
|
|
|
|
The commands in the matrix use `MXGATEWAY_API_KEY` through each CLI's
|
|
`api-key-env` flag. They must not embed bearer tokens or raw API keys.
|
|
|
|
### TLS variant
|
|
|
|
The matrix runs over plaintext (`h2c`) by default. A TLS variant exists but stays
|
|
a manual/opt-in run, consistent with the gate above, because it needs the gateway
|
|
started with an HTTPS endpoint (an `https://` `MXGATEWAY_ENDPOINT`) and each CLI
|
|
switched to its TLS flag (`--tls` / `-tls` / `--plaintext=false` /
|
|
`plaintext=False`). The clients are lenient by default and accept the gateway's
|
|
auto-generated self-signed certificate without extra trust setup, except the Rust
|
|
CLI, which is pin-only and needs `--ca-file` or `--require-certificate-validation`
|
|
(and Python uses trust-on-first-use). See
|
|
[Gateway Configuration — Automatic self-signed certificate](./GatewayConfiguration.md#automatic-self-signed-certificate)
|
|
and each client README for the per-client TLS flags.
|
|
|
|
## JSON Comparison
|
|
|
|
Every command in the matrix requests JSON output. A runner can compare the
|
|
normalized smoke record across languages with these fields:
|
|
|
|
- language,
|
|
- operation,
|
|
- session id,
|
|
- server handle,
|
|
- item handle,
|
|
- event count,
|
|
- event family,
|
|
- worker sequence,
|
|
- protocol status,
|
|
- HRESULT,
|
|
- status arrays,
|
|
- close status.
|
|
|
|
Failure output must include the client language, endpoint, and redacted auth
|
|
context. Auth context identifies the source, such as `MXGATEWAY_API_KEY`, but
|
|
does not include the secret value.
|
|
|
|
## Bundled Smoke Commands
|
|
|
|
Each client also exposes a bundled `smoke` command. Those commands are useful
|
|
for quick local checks, but the full cross-language matrix uses explicit
|
|
operation commands because not every bundled smoke command streams events yet.
|
|
The explicit sequence remains the parity baseline for issue-level validation.
|
|
|
|
## Validation
|
|
|
|
Run the matrix shape tests after changing the smoke matrix:
|
|
|
|
```bash
|
|
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter FullyQualifiedName~CrossLanguageSmokeMatrixTests
|
|
```
|
|
|
|
Live execution remains a separate opt-in step because it depends on a running
|
|
gateway, the installed MXAccess worker path, and provider state.
|
|
|
|
## Related Documentation
|
|
|
|
- [Gateway Testing](./GatewayTesting.md)
|
|
- [Client Libraries Detailed Design](./ClientLibrariesDesign.md)
|
|
- [Client Proto Generation](./ClientProtoGeneration.md)
|