fix(GWC-25,CLI-35,CLI-36): make the empty-ring ReplayGap resumable end to end

An empty replay ring reported oldest_available_sequence = 0 even when gap was
true. Clients follow the documented after_worker_sequence = oldest - 1 formula,
so an unsigned client computed ulong.MaxValue: the follow-up resume replayed
nothing, reported no gap, and the live filter dropped every subsequent event —
a silently dead stream in the headline detach-and-resume scenario, reachable on
default config once ReplayRetentionSeconds (300) age-evicts the ring.

GWC-25: SessionEventDistributor.RegisterWithReplay's empty-ring branch now
reports _highestSequenceSeen + 1 — the next sequence that can possibly be
delivered — when gap is true, so oldest - 1 lands exactly on the highest
observed sequence and the resume delivers everything newer. Still 0 when there
is no gap, where the field is meaningless and never emitted. Nothing is lost:
the evicted interval was unrecoverable either way, and the sentinel's job is to
say "re-snapshot".

CLI-35: the Python CLI fed every stream item into MessageToDict, which raised on
the ReplayGap dataclass and aborted the command after consuming the stream. A
new _event_row helper renders a gap as {"replayGap": {...}} — the same camelCase
shape the Rust CLI emits — and leaves proto events on the existing path.

CLI-36: the Go CLI formatted result.Event on every row, but the library
deliberately clears Event on a gap, so text mode printed
"0 MX_EVENT_FAMILY_UNSPECIFIED" and JSON mode an empty object, discarding the
resume cursors. The loop now branches on result.IsReplayGap() and renders the
typed row in both modes, counting it toward -limit like any other row. The JSON
row's cursors are typed by hand rather than marshalled with protojson: the
proto3 JSON mapping renders 64-bit integers as strings ("7") while the Rust and
Python CLIs emit numbers (7), so going through protojson would have made Go the
only canonical CLI with a different value type.

Docs in the same change: docs/Sessions.md documents the empty-ring sentinel
value and that oldest - 1 is the universal resume formula in both the retained
and fully-evicted cases; docs/CrossLanguageSmokeMatrix.md gains a per-CLI
gap-rendering table covering both client findings, and records exactly what is
and is not comparable across CLIs (same keys and numeric cursors for Rust/Go/
Python; quoted cursors for .NET/Java; differing key order, whitespace, and
container), so a matrix runner compares parsed values rather than raw bytes.

Tests, all written red first and each reproducing its defect verbatim:
- SessionEventDistributorTests: RegisterWithReplayReportsNextDeliverableSequence
  WhenRingEmptiedByAge, ...WithRetentionDisabled, and
  ResumeUsingSentinelFormulaAfterEmptyRingGapDeliversLiveEvents.
- GatewayEndToEndReconnectReplayTests.ReconnectAfterFullAgeEvictionResumesWith
  SentinelFormula — fake-worker e2e resume walk on a fake clock; the fixture now
  takes a retention window and a TimeProvider.
- clients/python test_stream_events_renders_replay_gap.
- clients/go TestRunStreamEventsPrintsReplayGap.

GWC-25's ReplayGap.oldest_available_sequence proto-comment amendment is
deliberately deferred to the later codegen wave (see the tracker change log): it
is comment-only but triggers the full five-client regen fan-out.
This commit is contained in:
Joseph Doherty
2026-08-07 05:37:27 -04:00
parent ead921cace
commit 44b8e37900
12 changed files with 596 additions and 28 deletions
+34 -3
View File
@@ -34,9 +34,40 @@ When `stream-events` is resumed with an `after_worker_sequence` cursor that
predates the oldest event still in the gateway's replay ring, the gateway emits a
single `ReplayGap` sentinel at the head of the stream. Every client surfaces this
as a distinct, typed, non-terminal signal (see each client README); the resume
contract is `after_worker_sequence = oldest_available_sequence - 1`. The default
smoke sequence opens a fresh stream (no cursor) and does not exercise the gap
path; a resume-with-gap fixture case is tracked separately (TST-24).
contract is `after_worker_sequence = oldest_available_sequence - 1`, and it holds
even when the ring has been emptied entirely by age eviction — the gateway then
reports the next deliverable sequence rather than `0` (see [Sessions](Sessions.md)).
The default smoke sequence opens a fresh stream (no cursor) and does not exercise
the gap path; a resume-with-gap fixture case is tracked separately (TST-24).
The CLIs differ in how they *print* that library-level signal. Three of them consume
the typed gap and emit a dedicated row rather than a degenerate event row; the other
two hand the raw sentinel `MxEvent` straight to the formatter, so they print the
sentinel itself, whose `replayGap` field carries the same cursors:
| CLI | Text mode | JSON mode |
|-----|-----------|-----------|
| `mxgw-rs` (Rust, canonical) | `REPLAY_GAP requested_after=<n> oldest_available=<n>` | `{"replayGap": {"requestedAfterSequence": <n>, "oldestAvailableSequence": <n>}}` as one entry of the `events` array |
| `mxgw-go` (Go) | `REPLAY_GAP requested_after=<n> oldest_available=<n>` | one `{"replayGap": {"requestedAfterSequence": <n>, "oldestAvailableSequence": <n>}}` line, counted toward `-limit` like any other row |
| `mxgw-py` (Python) | same JSON dump as `--json` | `{"replayGap": {"requestedAfterSequence": <n>, "oldestAvailableSequence": <n>}}` as one entry of the `events` array |
| `mxgw-dotnet` (.NET) | the raw sentinel `MxEvent` as protobuf JSON, including its `replayGap` field | same, as one entry of the `events` array |
| `mxgw-java` (Java) | the sentinel's `worker_sequence` and `family` (`0 MX_EVENT_FAMILY_UNSPECIFIED`) | the raw sentinel `MxEvent` as protobuf JSON, including its `replayGap` field |
Rust, Go, and Python emit the same two key names and, deliberately, the same JSON
value **types**: the cursors are JSON numbers (`7`), not strings. That is why the
Go CLI types the row by hand instead of marshalling `ReplayGap` with `protojson`
the proto3 JSON mapping renders 64-bit integers as strings (`"7"`), which is also
why the .NET and Java rows, which pass the sentinel through a protobuf JSON
formatter, carry **quoted** cursors. A matrix runner must therefore compare parsed
values, not raw bytes, and must not assume the same value type across all five
CLIs.
Two further formatting differences among the three canonical CLIs, none of them
semantic: Python sorts object keys and uses `", "` / `": "` separators
(`json.dumps(..., sort_keys=True)`), while Rust and Go emit compact,
declaration-ordered JSON; and the row sits alone on its own line for Go and for
Rust's `--jsonl`, but inside an `events` array for Python and for Rust's
aggregate `--json`.
## Integration Gate