Architecture remediation: P2 tier (completeness & polish) #122
@@ -84,6 +84,48 @@ messages. `MxGatewaySession.OpenSessionReply` keeps the raw session-open reply
|
|||||||
available, and command helpers have `*RawAsync` variants when callers need the
|
available, and command helpers have `*RawAsync` variants when callers need the
|
||||||
complete `MxCommandReply`.
|
complete `MxCommandReply`.
|
||||||
|
|
||||||
|
### Event Streaming And Replay Gaps
|
||||||
|
|
||||||
|
`StreamEventsAsync(afterWorkerSequence)` yields raw generated `MxEvent`
|
||||||
|
messages. Passing a non-zero `afterWorkerSequence` resumes a session's event
|
||||||
|
stream after a known worker sequence — this is the reconnect cursor. If that
|
||||||
|
cursor is *stale* — older than the oldest event the gateway still retains in the
|
||||||
|
session replay ring — the events in between were evicted and cannot be replayed.
|
||||||
|
The gateway signals this by emitting a single **replay-gap sentinel** at the head
|
||||||
|
of the resumed stream: an `MxEvent` with its `ReplayGap` field set, `Family`
|
||||||
|
unspecified, and no body. It means "you missed events — discard local state and
|
||||||
|
re-snapshot."
|
||||||
|
|
||||||
|
Rather than force callers to inspect the raw sentinel, the client exposes a
|
||||||
|
typed surface, `StreamEventItemsAsync`, which yields `MxEventStreamItem` values:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
await foreach (MxEventStreamItem item in session.StreamEventItemsAsync(
|
||||||
|
afterWorkerSequence: lastSeenSequence))
|
||||||
|
{
|
||||||
|
if (item.IsReplayGap)
|
||||||
|
{
|
||||||
|
// We missed events: throw away local state and re-snapshot.
|
||||||
|
ReplayGap gap = item.ReplayGap!;
|
||||||
|
// Resume without incurring another gap:
|
||||||
|
lastSeenSequence = gap.OldestAvailableSequence - 1;
|
||||||
|
await ReSnapshotAsync();
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
HandleEvent(item.Event); // normal MXAccess event, IsReplayGap == false
|
||||||
|
lastSeenSequence = item.Event.WorkerSequence;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The typed surface never synthesizes or drops events — it only makes the
|
||||||
|
gateway's own sentinel observable. Normal events pass through with
|
||||||
|
`IsReplayGap == false` and `ReplayGap == null`. The gap is only ever produced by
|
||||||
|
`StreamEvents`; the diagnostic drain path never emits it. If you already consume
|
||||||
|
the raw `StreamEventsAsync` (or the client-level stream), the
|
||||||
|
`AsStreamItemsAsync()` extension projects any `IAsyncEnumerable<MxEvent>` into
|
||||||
|
the same `MxEventStreamItem` surface.
|
||||||
|
|
||||||
For alarms, the client exposes `QueryActiveAlarmsAsync` (one-shot snapshot of
|
For alarms, the client exposes `QueryActiveAlarmsAsync` (one-shot snapshot of
|
||||||
the active alarms the gateway's central monitor currently holds),
|
the active alarms the gateway's central monitor currently holds),
|
||||||
`StreamAlarmsAsync` (server-streaming feed of alarm-state-change messages
|
`StreamAlarmsAsync` (server-streaming feed of alarm-state-change messages
|
||||||
|
|||||||
@@ -215,6 +215,55 @@ public sealed class MxGatewayClientSessionTests
|
|||||||
Assert.Equal("session-fixture", request.SessionId);
|
Assert.Equal("session-fixture", request.SessionId);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Verifies that a reconnect-replay gap sentinel is surfaced as a typed
|
||||||
|
/// <see cref="MxEventStreamItem"/> with the gap populated, while normal
|
||||||
|
/// events pass through unchanged with IsReplayGap false.
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task StreamEventItemsAsync_SurfacesReplayGapSentinel()
|
||||||
|
{
|
||||||
|
FakeGatewayTransport transport = CreateTransport();
|
||||||
|
transport.AddEvent(new MxEvent
|
||||||
|
{
|
||||||
|
SessionId = "session-fixture",
|
||||||
|
ReplayGap = new ReplayGap
|
||||||
|
{
|
||||||
|
RequestedAfterSequence = 5,
|
||||||
|
OldestAvailableSequence = 42,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
transport.AddEvent(new MxEvent
|
||||||
|
{
|
||||||
|
SessionId = "session-fixture",
|
||||||
|
Family = MxEventFamily.OnDataChange,
|
||||||
|
WorkerSequence = 42,
|
||||||
|
});
|
||||||
|
await using MxGatewayClient client = CreateClient(transport);
|
||||||
|
MxGatewaySession session = await client.OpenSessionAsync();
|
||||||
|
|
||||||
|
List<MxEventStreamItem> items = [];
|
||||||
|
await foreach (MxEventStreamItem item in session.StreamEventItemsAsync(afterWorkerSequence: 5))
|
||||||
|
{
|
||||||
|
items.Add(item);
|
||||||
|
}
|
||||||
|
|
||||||
|
Assert.Equal(2, items.Count);
|
||||||
|
|
||||||
|
MxEventStreamItem gap = items[0];
|
||||||
|
Assert.True(gap.IsReplayGap);
|
||||||
|
Assert.NotNull(gap.ReplayGap);
|
||||||
|
Assert.Equal(5UL, gap.ReplayGap!.RequestedAfterSequence);
|
||||||
|
Assert.Equal(42UL, gap.ReplayGap.OldestAvailableSequence);
|
||||||
|
Assert.Same(gap.ReplayGap, gap.Event.ReplayGap);
|
||||||
|
|
||||||
|
MxEventStreamItem normal = items[1];
|
||||||
|
Assert.False(normal.IsReplayGap);
|
||||||
|
Assert.Null(normal.ReplayGap);
|
||||||
|
Assert.Equal(42UL, normal.Event.WorkerSequence);
|
||||||
|
Assert.Equal(MxEventFamily.OnDataChange, normal.Event.Family);
|
||||||
|
}
|
||||||
|
|
||||||
/// <summary>Verifies that close is explicit and idempotent.</summary>
|
/// <summary>Verifies that close is explicit and idempotent.</summary>
|
||||||
[Fact]
|
[Fact]
|
||||||
public async Task CloseAsync_IsExplicitAndIdempotent()
|
public async Task CloseAsync_IsExplicitAndIdempotent()
|
||||||
|
|||||||
@@ -0,0 +1,40 @@
|
|||||||
|
using System.Runtime.CompilerServices;
|
||||||
|
using ZB.MOM.WW.MxGateway.Contracts.Proto;
|
||||||
|
|
||||||
|
namespace ZB.MOM.WW.MxGateway.Client;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Extension methods that project a raw <see cref="MxEvent"/> stream into the
|
||||||
|
/// typed <see cref="MxEventStreamItem"/> surface, making the gateway's
|
||||||
|
/// reconnect-replay gap sentinel observable.
|
||||||
|
/// </summary>
|
||||||
|
public static class MxEventStreamExtensions
|
||||||
|
{
|
||||||
|
/// <summary>
|
||||||
|
/// Projects a raw <see cref="MxEvent"/> stream (e.g.
|
||||||
|
/// <see cref="MxGatewaySession.StreamEventsAsync"/> or
|
||||||
|
/// <see cref="MxGatewayClient.StreamEventsAsync"/>) into typed
|
||||||
|
/// <see cref="MxEventStreamItem"/> values. Normal events pass through with
|
||||||
|
/// <see cref="MxEventStreamItem.IsReplayGap"/> false; the gateway's
|
||||||
|
/// reconnect-replay gap sentinel is surfaced with
|
||||||
|
/// <see cref="MxEventStreamItem.IsReplayGap"/> true and
|
||||||
|
/// <see cref="MxEventStreamItem.ReplayGap"/> populated. The stream is
|
||||||
|
/// forwarded faithfully — no event is synthesized or dropped.
|
||||||
|
/// </summary>
|
||||||
|
/// <param name="source">The raw event stream to wrap.</param>
|
||||||
|
/// <param name="cancellationToken">Cancellation token for the enumeration.</param>
|
||||||
|
/// <returns>The same events, each wrapped as an <see cref="MxEventStreamItem"/>.</returns>
|
||||||
|
public static async IAsyncEnumerable<MxEventStreamItem> AsStreamItemsAsync(
|
||||||
|
this IAsyncEnumerable<MxEvent> source,
|
||||||
|
[EnumeratorCancellation] CancellationToken cancellationToken = default)
|
||||||
|
{
|
||||||
|
ArgumentNullException.ThrowIfNull(source);
|
||||||
|
|
||||||
|
await foreach (MxEvent gatewayEvent in source
|
||||||
|
.WithCancellation(cancellationToken)
|
||||||
|
.ConfigureAwait(false))
|
||||||
|
{
|
||||||
|
yield return MxEventStreamItem.From(gatewayEvent);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,82 @@
|
|||||||
|
using ZB.MOM.WW.MxGateway.Contracts.Proto;
|
||||||
|
|
||||||
|
namespace ZB.MOM.WW.MxGateway.Client;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// One item yielded by the typed event stream. It is either a normal MXAccess
|
||||||
|
/// <see cref="MxEvent"/> or a reconnect-replay <em>gap sentinel</em>.
|
||||||
|
/// </summary>
|
||||||
|
/// <remarks>
|
||||||
|
/// <para>
|
||||||
|
/// The gateway emits a single sentinel <see cref="MxEvent"/> at the head of a
|
||||||
|
/// <c>StreamEvents</c> stream that was resumed via
|
||||||
|
/// <c>StreamEventsRequest.after_worker_sequence</c> when the requested sequence
|
||||||
|
/// predates the oldest event still retained in the session replay ring — i.e.
|
||||||
|
/// events were evicted and cannot be replayed. On that sentinel the
|
||||||
|
/// <c>MxEvent.replay_gap</c> field is set, <see cref="MxEvent.Family"/> is
|
||||||
|
/// <see cref="MxEventFamily.Unspecified"/>, the <c>body</c> oneof is unset, and
|
||||||
|
/// no per-item fields are populated.
|
||||||
|
/// </para>
|
||||||
|
/// <para>
|
||||||
|
/// This wrapper makes that sentinel observable instead of forcing the consumer
|
||||||
|
/// to inspect the raw <see cref="MxEvent"/>. It never synthesizes or swallows an
|
||||||
|
/// event: the gap is exactly the gateway's own sentinel, exposed with
|
||||||
|
/// <see cref="IsReplayGap"/> set and <see cref="ReplayGap"/> populated. Every
|
||||||
|
/// other event flows through unchanged with <see cref="IsReplayGap"/> false.
|
||||||
|
/// </para>
|
||||||
|
/// <para>
|
||||||
|
/// When <see cref="IsReplayGap"/> is <see langword="true"/> the consumer has
|
||||||
|
/// missed events and MUST discard local state and re-snapshot. To resume without
|
||||||
|
/// incurring another gap, reconnect with
|
||||||
|
/// <c>after_worker_sequence = ReplayGap.OldestAvailableSequence - 1</c>.
|
||||||
|
/// </para>
|
||||||
|
/// </remarks>
|
||||||
|
public sealed class MxEventStreamItem
|
||||||
|
{
|
||||||
|
private MxEventStreamItem(MxEvent gatewayEvent, ReplayGap? replayGap)
|
||||||
|
{
|
||||||
|
Event = gatewayEvent;
|
||||||
|
ReplayGap = replayGap;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// The underlying raw <see cref="MxEvent"/>. For a normal event this is the
|
||||||
|
/// MXAccess event itself; for a replay-gap item this is the gateway's
|
||||||
|
/// sentinel event whose only meaningful payload is <see cref="ReplayGap"/>.
|
||||||
|
/// Never <see langword="null"/>.
|
||||||
|
/// </summary>
|
||||||
|
public MxEvent Event { get; }
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// The reconnect-replay gap payload when this item is a gap sentinel;
|
||||||
|
/// otherwise <see langword="null"/>. Read
|
||||||
|
/// <see cref="Contracts.Proto.ReplayGap.RequestedAfterSequence"/> and
|
||||||
|
/// <see cref="Contracts.Proto.ReplayGap.OldestAvailableSequence"/> to learn
|
||||||
|
/// which events were lost.
|
||||||
|
/// </summary>
|
||||||
|
public ReplayGap? ReplayGap { get; }
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// <see langword="true"/> when this item is a reconnect-replay gap sentinel:
|
||||||
|
/// the consumer missed events and must discard local state and re-snapshot.
|
||||||
|
/// Resume without another gap by reconnecting with
|
||||||
|
/// <c>after_worker_sequence = ReplayGap.OldestAvailableSequence - 1</c>.
|
||||||
|
/// </summary>
|
||||||
|
public bool IsReplayGap => ReplayGap is not null;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Wraps a raw stream <see cref="MxEvent"/> as a typed item, classifying it
|
||||||
|
/// as a replay-gap sentinel when <c>MxEvent.replay_gap</c> is present.
|
||||||
|
/// </summary>
|
||||||
|
/// <param name="gatewayEvent">The raw event from the <c>StreamEvents</c> stream.</param>
|
||||||
|
/// <returns>A typed item exposing either the normal event or the replay gap.</returns>
|
||||||
|
internal static MxEventStreamItem From(MxEvent gatewayEvent)
|
||||||
|
{
|
||||||
|
ArgumentNullException.ThrowIfNull(gatewayEvent);
|
||||||
|
|
||||||
|
// For a message-typed proto3 field, presence is a non-null reference.
|
||||||
|
return gatewayEvent.ReplayGap is { } replayGap
|
||||||
|
? new MxEventStreamItem(gatewayEvent, replayGap)
|
||||||
|
: new MxEventStreamItem(gatewayEvent, replayGap: null);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -875,6 +875,34 @@ public sealed class MxGatewaySession : IAsyncDisposable
|
|||||||
cancellationToken);
|
cancellationToken);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Streams events as typed <see cref="MxEventStreamItem"/> values, surfacing
|
||||||
|
/// the gateway's reconnect-replay gap sentinel as an observable, typed signal.
|
||||||
|
/// </summary>
|
||||||
|
/// <remarks>
|
||||||
|
/// When resuming with a stale <paramref name="afterWorkerSequence"/> (older than
|
||||||
|
/// the oldest event still retained in the session replay ring), the gateway emits
|
||||||
|
/// a single gap sentinel at the head of the stream. It arrives here as an item
|
||||||
|
/// with <see cref="MxEventStreamItem.IsReplayGap"/> true and
|
||||||
|
/// <see cref="MxEventStreamItem.ReplayGap"/> populated, meaning the consumer
|
||||||
|
/// missed events and MUST discard local state and re-snapshot. To resume without
|
||||||
|
/// incurring another gap, reconnect with
|
||||||
|
/// <c>afterWorkerSequence = item.ReplayGap.OldestAvailableSequence - 1</c>.
|
||||||
|
/// All other events pass through with <see cref="MxEventStreamItem.IsReplayGap"/>
|
||||||
|
/// false. Use <see cref="StreamEventsAsync"/> when raw generated
|
||||||
|
/// <see cref="MxEvent"/> messages are needed instead.
|
||||||
|
/// </remarks>
|
||||||
|
/// <param name="afterWorkerSequence">The sequence number to stream from. Defaults to 0.</param>
|
||||||
|
/// <param name="cancellationToken">Cancellation token.</param>
|
||||||
|
/// <returns>An async enumerable of typed event items.</returns>
|
||||||
|
public IAsyncEnumerable<MxEventStreamItem> StreamEventItemsAsync(
|
||||||
|
ulong afterWorkerSequence = 0,
|
||||||
|
CancellationToken cancellationToken = default)
|
||||||
|
{
|
||||||
|
return StreamEventsAsync(afterWorkerSequence, cancellationToken)
|
||||||
|
.AsStreamItemsAsync(cancellationToken);
|
||||||
|
}
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// Closes the session and releases resources.
|
/// Closes the session and releases resources.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
|
|||||||
@@ -92,6 +92,44 @@ goroutine cleanup. Raw protobuf messages remain available through the
|
|||||||
`errors.As` for `GatewayError`, `CommandError`, and `MxAccessError`; command
|
`errors.As` for `GatewayError`, `CommandError`, and `MxAccessError`; command
|
||||||
errors preserve the raw reply.
|
errors preserve the raw reply.
|
||||||
|
|
||||||
|
### Reconnect-replay gap
|
||||||
|
|
||||||
|
Each `EventResult` carries exactly one of `Event`, `ReplayGap`, or `Err`. When
|
||||||
|
you resume a stream with `EventsAfter`/`SubscribeEventsAfter` and a non-zero
|
||||||
|
`afterWorkerSequence`, the gateway replays buffered events from that point. If
|
||||||
|
the requested sequence predates the oldest event still retained in its replay
|
||||||
|
ring, it delivers a single **replay-gap sentinel** at the head of the resumed
|
||||||
|
stream: `res.ReplayGap` is non-nil (`res.Event` is nil, `res.IsReplayGap()` is
|
||||||
|
true) and normal events follow it.
|
||||||
|
|
||||||
|
A gap means events were lost, so any locally cached tag/alarm state is now
|
||||||
|
stale. On seeing it, discard your cached state and re-snapshot. To resume
|
||||||
|
without provoking another gap, reconnect from just before the oldest retained
|
||||||
|
sequence:
|
||||||
|
|
||||||
|
```go
|
||||||
|
for res := range events {
|
||||||
|
switch {
|
||||||
|
case res.Err != nil:
|
||||||
|
// terminal: stream ended (see ErrSlowConsumer for the overflow case)
|
||||||
|
return res.Err
|
||||||
|
case res.IsReplayGap():
|
||||||
|
gap := res.ReplayGap
|
||||||
|
log.Printf("replay gap: requested after %d, oldest available %d; re-snapshotting",
|
||||||
|
gap.GetRequestedAfterSequence(), gap.GetOldestAvailableSequence())
|
||||||
|
resnapshot()
|
||||||
|
// to resume cleanly, reconnect with:
|
||||||
|
// session.EventsAfter(ctx, gap.GetOldestAvailableSequence()-1)
|
||||||
|
default:
|
||||||
|
handle(res.Event)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The gateway sets `ReplayGap` only on `StreamEvents` results (never on a fresh,
|
||||||
|
non-resumed stream and never on `DrainEvents`). The client makes the gateway's
|
||||||
|
sentinel typed and observable; it never synthesizes or swallows it.
|
||||||
|
|
||||||
For alarms, the package exposes `Client.QueryActiveAlarms` for one-shot
|
For alarms, the package exposes `Client.QueryActiveAlarms` for one-shot
|
||||||
snapshots, `Client.StreamAlarms` for the server-streaming feed, and
|
snapshots, `Client.StreamAlarms` for the server-streaming feed, and
|
||||||
`Client.AcknowledgeAlarm` to ack an alarm by full reference. The streaming
|
`Client.AcknowledgeAlarm` to ack an alarm by full reference. The streaming
|
||||||
|
|||||||
@@ -200,6 +200,63 @@ func TestEventsSlowConsumerYieldsErrSlowConsumerBeforeClose(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestEventsSurfacesReplayGapSentinelAsTypedSignal(t *testing.T) {
|
||||||
|
fake := &fakeGatewayServer{
|
||||||
|
streamStarted: make(chan struct{}),
|
||||||
|
streamReplayGap: &pb.ReplayGap{
|
||||||
|
RequestedAfterSequence: 5,
|
||||||
|
OldestAvailableSequence: 42,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
client, cleanup := newBufconnClient(t, fake)
|
||||||
|
defer cleanup()
|
||||||
|
session := NewSessionForID(client, "session-1")
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
events, err := session.EventsAfter(ctx, 5)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("EventsAfter() error = %v", err)
|
||||||
|
}
|
||||||
|
<-fake.streamStarted
|
||||||
|
|
||||||
|
// First result must be the typed replay-gap signal, not a normal event.
|
||||||
|
first := <-events
|
||||||
|
if first.Err != nil {
|
||||||
|
t.Fatalf("first result error = %v", first.Err)
|
||||||
|
}
|
||||||
|
if !first.IsReplayGap() {
|
||||||
|
t.Fatalf("first result IsReplayGap() = false, want true")
|
||||||
|
}
|
||||||
|
if first.Event != nil {
|
||||||
|
t.Fatalf("replay-gap result carried a non-nil Event %+v; want the sentinel to clear Event", first.Event)
|
||||||
|
}
|
||||||
|
if got := first.ReplayGap.GetRequestedAfterSequence(); got != 5 {
|
||||||
|
t.Fatalf("ReplayGap.RequestedAfterSequence = %d, want 5", got)
|
||||||
|
}
|
||||||
|
if got := first.ReplayGap.GetOldestAvailableSequence(); got != 42 {
|
||||||
|
t.Fatalf("ReplayGap.OldestAvailableSequence = %d, want 42", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Normal events after the sentinel are unaffected: Event set, ReplayGap nil.
|
||||||
|
second := <-events
|
||||||
|
if second.Err != nil {
|
||||||
|
t.Fatalf("second result error = %v", second.Err)
|
||||||
|
}
|
||||||
|
if second.IsReplayGap() {
|
||||||
|
t.Fatalf("second result IsReplayGap() = true, want false for a normal event")
|
||||||
|
}
|
||||||
|
if second.Event == nil {
|
||||||
|
t.Fatal("second result Event = nil, want a normal event")
|
||||||
|
}
|
||||||
|
if got := second.Event.GetWorkerSequence(); got != 1 {
|
||||||
|
t.Fatalf("normal event worker sequence = %d, want 1", got)
|
||||||
|
}
|
||||||
|
if second.Event.GetFamily() != pb.MxEventFamily_MX_EVENT_FAMILY_ON_DATA_CHANGE {
|
||||||
|
t.Fatalf("normal event family = %s, want ON_DATA_CHANGE", second.Event.GetFamily())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestSessionHelpersBuildCommandsAndExposeRawReply(t *testing.T) {
|
func TestSessionHelpersBuildCommandsAndExposeRawReply(t *testing.T) {
|
||||||
fake := &fakeGatewayServer{
|
fake := &fakeGatewayServer{
|
||||||
invokeReply: &pb.MxCommandReply{
|
invokeReply: &pb.MxCommandReply{
|
||||||
@@ -643,6 +700,7 @@ type fakeGatewayServer struct {
|
|||||||
streamStarted chan struct{}
|
streamStarted chan struct{}
|
||||||
streamDone chan struct{}
|
streamDone chan struct{}
|
||||||
streamEventCount int
|
streamEventCount int
|
||||||
|
streamReplayGap *pb.ReplayGap
|
||||||
invokeReply *pb.MxCommandReply
|
invokeReply *pb.MxCommandReply
|
||||||
invokeRequest *pb.MxCommandRequest
|
invokeRequest *pb.MxCommandRequest
|
||||||
}
|
}
|
||||||
@@ -691,6 +749,16 @@ func (s *fakeGatewayServer) StreamEvents(req *pb.StreamEventsRequest, stream grp
|
|||||||
if s.streamStarted != nil {
|
if s.streamStarted != nil {
|
||||||
close(s.streamStarted)
|
close(s.streamStarted)
|
||||||
}
|
}
|
||||||
|
if s.streamReplayGap != nil {
|
||||||
|
// Emit the reconnect-replay gap sentinel at the head of the resumed
|
||||||
|
// stream: family UNSPECIFIED, body unset, only replay_gap populated.
|
||||||
|
if err := stream.Send(&pb.MxEvent{
|
||||||
|
SessionId: req.GetSessionId(),
|
||||||
|
ReplayGap: s.streamReplayGap,
|
||||||
|
}); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
eventCount := s.streamEventCount
|
eventCount := s.streamEventCount
|
||||||
if eventCount == 0 {
|
if eventCount == 0 {
|
||||||
eventCount = 1
|
eventCount = 1
|
||||||
|
|||||||
@@ -27,14 +27,39 @@ const eventBufferSize = 16
|
|||||||
// non-blockingly on overflow, even when all data slots are full.
|
// non-blockingly on overflow, even when all data slots are full.
|
||||||
const eventBufferReservedSlots = 1
|
const eventBufferReservedSlots = 1
|
||||||
|
|
||||||
// EventResult carries either the next ordered event or a terminal stream error.
|
// EventResult carries the next ordered event, a replay-gap signal, or a
|
||||||
|
// terminal stream error. Exactly one of Event, ReplayGap, or Err is set on any
|
||||||
|
// delivered result.
|
||||||
type EventResult struct {
|
type EventResult struct {
|
||||||
// Event is the next event from the stream when Err is nil.
|
// Event is the next MXAccess event from the stream when both ReplayGap and
|
||||||
|
// Err are nil.
|
||||||
Event *MxEvent
|
Event *MxEvent
|
||||||
|
// ReplayGap, when non-nil, is the gateway's reconnect-replay gap sentinel: it
|
||||||
|
// is delivered at the head of a resumed stream (one opened with a non-zero
|
||||||
|
// after_worker_sequence via EventsAfter/SubscribeEventsAfter) when the
|
||||||
|
// requested sequence predates the oldest event still retained in the replay
|
||||||
|
// ring, so the events in between were lost.
|
||||||
|
//
|
||||||
|
// It is a non-terminal, observable signal — the stream continues with normal
|
||||||
|
// events after it, and Event is nil on a gap result so a gap is never
|
||||||
|
// mistaken for a normal MXAccess event. On seeing a gap the consumer must
|
||||||
|
// discard any locally cached tag/alarm state and re-snapshot. To resume
|
||||||
|
// cleanly (without provoking another gap), reconnect with EventsAfter using
|
||||||
|
// afterWorkerSequence = ReplayGap.GetOldestAvailableSequence() - 1.
|
||||||
|
//
|
||||||
|
// The gateway sets ReplayGap only on StreamEvents results, never on a normal
|
||||||
|
// (non-resumed) stream and never on DrainEvents.
|
||||||
|
ReplayGap *ReplayGap
|
||||||
// Err is the terminal stream error; when non-nil no further results follow.
|
// Err is the terminal stream error; when non-nil no further results follow.
|
||||||
Err error
|
Err error
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// IsReplayGap reports whether this result carries the gateway's reconnect-replay
|
||||||
|
// gap sentinel rather than a normal event or a terminal error.
|
||||||
|
func (r EventResult) IsReplayGap() bool {
|
||||||
|
return r.ReplayGap != nil
|
||||||
|
}
|
||||||
|
|
||||||
// EventSubscription owns a running gateway event stream.
|
// EventSubscription owns a running gateway event stream.
|
||||||
type EventSubscription struct {
|
type EventSubscription struct {
|
||||||
results <-chan EventResult
|
results <-chan EventResult
|
||||||
@@ -736,7 +761,15 @@ func (s *Session) subscribeEventsAfter(ctx context.Context, afterWorkerSequence
|
|||||||
for {
|
for {
|
||||||
event, err := stream.Recv()
|
event, err := stream.Recv()
|
||||||
if err == nil {
|
if err == nil {
|
||||||
if !sendEventResult(streamCtx, results, EventResult{Event: event}, cancelWhenResultBufferFull, cancel) {
|
result := EventResult{Event: event}
|
||||||
|
// The gateway marks a reconnect-replay gap with a sentinel MxEvent
|
||||||
|
// carrying replay_gap (family UNSPECIFIED, body unset). Surface it
|
||||||
|
// as a distinct typed signal rather than a normal event: clear
|
||||||
|
// Event so consumers never process the sentinel as a data change.
|
||||||
|
if gap := event.GetReplayGap(); gap != nil {
|
||||||
|
result = EventResult{ReplayGap: gap}
|
||||||
|
}
|
||||||
|
if !sendEventResult(streamCtx, results, result, cancelWhenResultBufferFull, cancel) {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
continue
|
continue
|
||||||
|
|||||||
@@ -30,6 +30,12 @@ type (
|
|||||||
MxCommand = pb.MxCommand
|
MxCommand = pb.MxCommand
|
||||||
// MxEvent is one ordered event delivered on a session event stream.
|
// MxEvent is one ordered event delivered on a session event stream.
|
||||||
MxEvent = pb.MxEvent
|
MxEvent = pb.MxEvent
|
||||||
|
// ReplayGap is the gateway sentinel payload signalling that a resumed event
|
||||||
|
// stream skipped past the oldest event still retained in the replay ring.
|
||||||
|
// RequestedAfterSequence is the after_worker_sequence the client resumed
|
||||||
|
// from; OldestAvailableSequence is the oldest sequence the gateway can still
|
||||||
|
// replay. See EventResult.ReplayGap for consumption guidance.
|
||||||
|
ReplayGap = pb.ReplayGap
|
||||||
// MxValue is the protobuf representation of an MXAccess value.
|
// MxValue is the protobuf representation of an MXAccess value.
|
||||||
MxValue = pb.MxValue
|
MxValue = pb.MxValue
|
||||||
// Value is an alias for MxValue retained for symmetry with other clients.
|
// Value is an alias for MxValue retained for symmetry with other clients.
|
||||||
|
|||||||
@@ -105,6 +105,40 @@ terminate the stream.
|
|||||||
Canceling a Python task cancels the client-side gRPC call or stream wait. It
|
Canceling a Python task cancels the client-side gRPC call or stream wait. It
|
||||||
does not abort an in-flight MXAccess COM call inside the worker process.
|
does not abort an in-flight MXAccess COM call inside the worker process.
|
||||||
|
|
||||||
|
### Event streaming and reconnect gaps
|
||||||
|
|
||||||
|
`Session.stream_events()` yields an async iterator whose items are either a
|
||||||
|
normal `MxEvent` or a `ReplayGap`. Track the `worker_sequence` of the last event
|
||||||
|
you processed and pass it back as `after_worker_sequence` to resume after a
|
||||||
|
disconnect:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from zb_mom_ww_mxgateway import ReplayGap
|
||||||
|
|
||||||
|
cursor = 0
|
||||||
|
async for item in session.stream_events(after_worker_sequence=cursor):
|
||||||
|
if isinstance(item, ReplayGap):
|
||||||
|
# The gateway dropped events between item.requested_after_sequence and
|
||||||
|
# item.oldest_available_sequence — they are gone from the replay ring.
|
||||||
|
# Discard local tag/alarm state and re-snapshot (e.g. read_bulk /
|
||||||
|
# query_active_alarms), then resume without another gap:
|
||||||
|
cursor = item.resume_after_worker_sequence # oldest_available_sequence - 1
|
||||||
|
continue
|
||||||
|
# Normal MXAccess event.
|
||||||
|
cursor = item.worker_sequence
|
||||||
|
handle(item)
|
||||||
|
```
|
||||||
|
|
||||||
|
`ReplayGap` is the gateway's reconnect-replay gap sentinel made typed and
|
||||||
|
observable. It is delivered only at the head of a stream resumed with a non-zero
|
||||||
|
`after_worker_sequence` when the requested cursor predates the oldest retained
|
||||||
|
event. It is a **non-terminal** signal — the stream continues with normal events
|
||||||
|
after it — and it is never yielded as an `MxEvent`, so it can never be mistaken
|
||||||
|
for a real MXAccess event. The client does not synthesize or swallow it; the
|
||||||
|
gateway only sets it on `StreamEvents` results (never on a fresh stream or a
|
||||||
|
`DrainEvents` reply). `GatewayClient.stream_events_raw` remains the raw protobuf
|
||||||
|
stream (the sentinel arrives there as an `MxEvent` with `replay_gap` set).
|
||||||
|
|
||||||
## Write Semantics And Common Pitfalls
|
## Write Semantics And Common Pitfalls
|
||||||
|
|
||||||
These are MXAccess parity behaviors that surprise new callers. The gateway
|
These are MXAccess parity behaviors that surprise new callers. The gateway
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ from .generated.galaxy_repository_pb2 import (
|
|||||||
GalaxyObject,
|
GalaxyObject,
|
||||||
WatchDeployEventsRequest,
|
WatchDeployEventsRequest,
|
||||||
)
|
)
|
||||||
|
from .events import ReplayGap
|
||||||
from .errors import (
|
from .errors import (
|
||||||
MxAccessError,
|
MxAccessError,
|
||||||
MxGatewayAuthenticationError,
|
MxGatewayAuthenticationError,
|
||||||
@@ -43,6 +44,7 @@ __all__ = [
|
|||||||
"MxGatewayTransportError",
|
"MxGatewayTransportError",
|
||||||
"MxGatewayWorkerError",
|
"MxGatewayWorkerError",
|
||||||
"MxValueView",
|
"MxValueView",
|
||||||
|
"ReplayGap",
|
||||||
"Session",
|
"Session",
|
||||||
"WatchDeployEventsRequest",
|
"WatchDeployEventsRequest",
|
||||||
"__version__",
|
"__version__",
|
||||||
|
|||||||
@@ -0,0 +1,63 @@
|
|||||||
|
"""Typed event-stream signals for the MXAccess Gateway Python client."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from .generated import mxaccess_gateway_pb2 as pb
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class ReplayGap:
|
||||||
|
"""Reconnect-replay gap signal surfaced on a resumed event stream.
|
||||||
|
|
||||||
|
The gateway emits this at the head of a stream resumed with
|
||||||
|
:meth:`Session.stream_events`'s ``after_worker_sequence`` cursor when the
|
||||||
|
requested sequence predates the oldest event still retained in the gateway's
|
||||||
|
replay ring. That means the events between ``requested_after_sequence`` and
|
||||||
|
``oldest_available_sequence`` were dropped from the ring and can no longer be
|
||||||
|
replayed — the client has an unrecoverable hole in its event history.
|
||||||
|
|
||||||
|
``ReplayGap`` is a *non-terminal, observable* signal: the stream keeps
|
||||||
|
delivering normal :class:`~zb_mom_ww_mxgateway.generated.mxaccess_gateway_pb2.MxEvent`
|
||||||
|
values after it. :meth:`Session.stream_events` yields it as a distinct type
|
||||||
|
(never as an ``MxEvent``) so a consumer can branch on
|
||||||
|
``isinstance(item, ReplayGap)`` and never mistake a gap for a real MXAccess
|
||||||
|
event. The client neither synthesizes nor swallows the gateway's sentinel —
|
||||||
|
it only makes that sentinel typed and observable.
|
||||||
|
|
||||||
|
On seeing a gap the consumer must discard any locally cached tag/alarm state
|
||||||
|
and re-snapshot (for example via :meth:`Session.read_bulk` or
|
||||||
|
:meth:`~zb_mom_ww_mxgateway.GatewayClient.query_active_alarms`). To resume
|
||||||
|
the stream without provoking another gap, reconnect with
|
||||||
|
``after_worker_sequence = gap.resume_after_worker_sequence`` (that is,
|
||||||
|
``oldest_available_sequence - 1``) so the next replayed event is the oldest
|
||||||
|
the gateway still retains.
|
||||||
|
|
||||||
|
The gateway sets this only on ``StreamEvents`` results — never on a normal
|
||||||
|
(non-resumed) stream and never on a ``DrainEvents`` reply.
|
||||||
|
"""
|
||||||
|
|
||||||
|
requested_after_sequence: int
|
||||||
|
"""The ``after_worker_sequence`` cursor the resumed stream was opened with."""
|
||||||
|
|
||||||
|
oldest_available_sequence: int
|
||||||
|
"""Oldest worker sequence the gateway can still replay."""
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_proto(cls, gap: pb.ReplayGap) -> "ReplayGap":
|
||||||
|
"""Build a :class:`ReplayGap` from the generated ``ReplayGap`` message."""
|
||||||
|
return cls(
|
||||||
|
requested_after_sequence=gap.requested_after_sequence,
|
||||||
|
oldest_available_sequence=gap.oldest_available_sequence,
|
||||||
|
)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def resume_after_worker_sequence(self) -> int:
|
||||||
|
"""``after_worker_sequence`` to resume the stream without another gap.
|
||||||
|
|
||||||
|
Equal to ``oldest_available_sequence - 1`` so the next event the gateway
|
||||||
|
replays is ``oldest_available_sequence`` — the oldest it still retains.
|
||||||
|
Clamped at ``0`` so it is never negative.
|
||||||
|
"""
|
||||||
|
return max(self.oldest_available_sequence - 1, 0)
|
||||||
@@ -5,6 +5,7 @@ from __future__ import annotations
|
|||||||
from collections.abc import AsyncIterator, Sequence
|
from collections.abc import AsyncIterator, Sequence
|
||||||
|
|
||||||
from .errors import ensure_mxaccess_success
|
from .errors import ensure_mxaccess_success
|
||||||
|
from .events import ReplayGap
|
||||||
from .generated import mxaccess_gateway_pb2 as pb
|
from .generated import mxaccess_gateway_pb2 as pb
|
||||||
from .values import MxValueInput, to_mx_value
|
from .values import MxValueInput, to_mx_value
|
||||||
|
|
||||||
@@ -572,14 +573,57 @@ class Session:
|
|||||||
self,
|
self,
|
||||||
*,
|
*,
|
||||||
after_worker_sequence: int = 0,
|
after_worker_sequence: int = 0,
|
||||||
) -> AsyncIterator[pb.MxEvent]:
|
) -> AsyncIterator[pb.MxEvent | ReplayGap]:
|
||||||
"""Return an async iterator of `MxEvent` messages for this session."""
|
"""Return an async iterator over this session's `MxEvent` stream.
|
||||||
return self.client.stream_events_raw(
|
|
||||||
|
Each yielded item is either a normal :class:`~...MxEvent` or a
|
||||||
|
:class:`ReplayGap`. Branch on ``isinstance(item, ReplayGap)`` — a gap is
|
||||||
|
never delivered as an ``MxEvent`` so it cannot be mistaken for a real
|
||||||
|
MXAccess event.
|
||||||
|
|
||||||
|
Pass a non-zero *after_worker_sequence* to resume a previously observed
|
||||||
|
stream. If that cursor predates the oldest event the gateway still
|
||||||
|
retains in its replay ring, the stream opens with a single
|
||||||
|
:class:`ReplayGap` sentinel (events in the gap were dropped and cannot be
|
||||||
|
replayed), then continues with normal events. On a gap, discard locally
|
||||||
|
cached state, re-snapshot, and — to resume without another gap —
|
||||||
|
reconnect with ``after_worker_sequence = gap.resume_after_worker_sequence``.
|
||||||
|
See :class:`ReplayGap` for the full semantics. The underlying protobuf
|
||||||
|
stream is available raw via ``GatewayClient.stream_events_raw``.
|
||||||
|
"""
|
||||||
|
raw = self.client.stream_events_raw(
|
||||||
pb.StreamEventsRequest(
|
pb.StreamEventsRequest(
|
||||||
session_id=self.session_id,
|
session_id=self.session_id,
|
||||||
after_worker_sequence=after_worker_sequence,
|
after_worker_sequence=after_worker_sequence,
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
return _surface_replay_gaps(raw)
|
||||||
|
|
||||||
|
|
||||||
|
async def _surface_replay_gaps(
|
||||||
|
raw: AsyncIterator[pb.MxEvent],
|
||||||
|
) -> AsyncIterator[pb.MxEvent | ReplayGap]:
|
||||||
|
"""Map the gateway's ``replay_gap`` sentinel event to a typed :class:`ReplayGap`.
|
||||||
|
|
||||||
|
Normal events pass through unchanged. The sentinel (``replay_gap`` set,
|
||||||
|
``family`` unspecified, body unset) is converted to a distinct
|
||||||
|
:class:`ReplayGap` so a consumer can branch on it without inspecting proto
|
||||||
|
presence, and is never yielded as an ``MxEvent``. The sentinel is forwarded
|
||||||
|
faithfully — it is neither dropped nor turned into a normal event.
|
||||||
|
|
||||||
|
Closing this generator (``aclose``) propagates to *raw* so the underlying
|
||||||
|
gRPC call is cancelled, preserving the raw stream's cancel-on-stop contract.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
async for event in raw:
|
||||||
|
if event.HasField("replay_gap"):
|
||||||
|
yield ReplayGap.from_proto(event.replay_gap)
|
||||||
|
else:
|
||||||
|
yield event
|
||||||
|
finally:
|
||||||
|
aclose = getattr(raw, "aclose", None)
|
||||||
|
if aclose is not None:
|
||||||
|
await aclose()
|
||||||
|
|
||||||
|
|
||||||
def _ensure_bulk_size(name: str, count: int) -> None:
|
def _ensure_bulk_size(name: str, count: int) -> None:
|
||||||
|
|||||||
@@ -0,0 +1,106 @@
|
|||||||
|
"""Tests for the typed ReplayGap signal on Session.stream_events (CLI-15)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import AsyncIterator
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from zb_mom_ww_mxgateway import ReplayGap, Session
|
||||||
|
from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeClient:
|
||||||
|
"""Minimal client stub exposing only what Session.stream_events needs."""
|
||||||
|
|
||||||
|
def __init__(self, events: list[pb.MxEvent]) -> None:
|
||||||
|
self._events = events
|
||||||
|
self.last_request: pb.StreamEventsRequest | None = None
|
||||||
|
|
||||||
|
def stream_events_raw(
|
||||||
|
self,
|
||||||
|
request: pb.StreamEventsRequest,
|
||||||
|
) -> AsyncIterator[pb.MxEvent]:
|
||||||
|
self.last_request = request
|
||||||
|
|
||||||
|
async def _gen() -> AsyncIterator[pb.MxEvent]:
|
||||||
|
for event in self._events:
|
||||||
|
yield event
|
||||||
|
|
||||||
|
return _gen()
|
||||||
|
|
||||||
|
|
||||||
|
def _gap_sentinel(*, requested: int, oldest: int) -> pb.MxEvent:
|
||||||
|
return pb.MxEvent(
|
||||||
|
session_id="session-1",
|
||||||
|
family=pb.MX_EVENT_FAMILY_UNSPECIFIED,
|
||||||
|
replay_gap=pb.ReplayGap(
|
||||||
|
requested_after_sequence=requested,
|
||||||
|
oldest_available_sequence=oldest,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _normal_event(worker_sequence: int) -> pb.MxEvent:
|
||||||
|
return pb.MxEvent(
|
||||||
|
session_id="session-1",
|
||||||
|
worker_sequence=worker_sequence,
|
||||||
|
family=pb.MX_EVENT_FAMILY_ON_DATA_CHANGE,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _session(events: list[pb.MxEvent]) -> tuple[Session, _FakeClient]:
|
||||||
|
client = _FakeClient(events)
|
||||||
|
session = Session(client=client, session_id="session-1") # type: ignore[arg-type]
|
||||||
|
return session, client
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_replay_gap_sentinel_surfaces_as_typed_signal() -> None:
|
||||||
|
session, _client = _session(
|
||||||
|
[_gap_sentinel(requested=5, oldest=10), _normal_event(10)],
|
||||||
|
)
|
||||||
|
|
||||||
|
items = [item async for item in session.stream_events(after_worker_sequence=5)]
|
||||||
|
|
||||||
|
assert isinstance(items[0], ReplayGap)
|
||||||
|
assert items[0].requested_after_sequence == 5
|
||||||
|
assert items[0].oldest_available_sequence == 10
|
||||||
|
# Resume cursor is oldest_available_sequence - 1 so the next replayed event
|
||||||
|
# is the oldest the gateway still retains.
|
||||||
|
assert items[0].resume_after_worker_sequence == 9
|
||||||
|
|
||||||
|
# Normal events after the sentinel pass through unchanged as MxEvent.
|
||||||
|
assert isinstance(items[1], pb.MxEvent)
|
||||||
|
assert not items[1].HasField("replay_gap")
|
||||||
|
assert items[1].worker_sequence == 10
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_normal_events_are_unaffected() -> None:
|
||||||
|
session, _client = _session([_normal_event(1), _normal_event(2)])
|
||||||
|
|
||||||
|
items = [item async for item in session.stream_events()]
|
||||||
|
|
||||||
|
assert all(isinstance(item, pb.MxEvent) for item in items)
|
||||||
|
assert [item.worker_sequence for item in items] == [1, 2]
|
||||||
|
assert not any(isinstance(item, ReplayGap) for item in items)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_stream_events_forwards_resume_cursor() -> None:
|
||||||
|
session, client = _session([])
|
||||||
|
|
||||||
|
async for _ in session.stream_events(after_worker_sequence=42):
|
||||||
|
pass
|
||||||
|
|
||||||
|
assert client.last_request is not None
|
||||||
|
assert client.last_request.after_worker_sequence == 42
|
||||||
|
assert client.last_request.session_id == "session-1"
|
||||||
|
|
||||||
|
|
||||||
|
def test_replay_gap_resume_cursor_never_negative() -> None:
|
||||||
|
gap = ReplayGap.from_proto(
|
||||||
|
pb.ReplayGap(requested_after_sequence=0, oldest_available_sequence=0),
|
||||||
|
)
|
||||||
|
assert gap.resume_after_worker_sequence == 0
|
||||||
@@ -137,6 +137,47 @@ redaction. Per-item bulk failures are reported inside each result entry
|
|||||||
`invoke_raw` / `client.invoke_raw` escape hatch performs neither check and
|
`invoke_raw` / `client.invoke_raw` escape hatch performs neither check and
|
||||||
returns the unvalidated reply.
|
returns the unvalidated reply.
|
||||||
|
|
||||||
|
## Event Streaming And Reconnect-Replay Gaps
|
||||||
|
|
||||||
|
`session.events()` / `session.events_after(after_worker_sequence)` (and the
|
||||||
|
lower-level `client.stream_events`) return an `EventStream` that yields
|
||||||
|
`EventItem` values, not bare `MxEvent`s:
|
||||||
|
|
||||||
|
```rust
|
||||||
|
use zb_mom_ww_mxgateway_client::EventItem;
|
||||||
|
|
||||||
|
let mut stream = session.events_after(cursor).await?;
|
||||||
|
while let Some(item) = stream.next().await {
|
||||||
|
match item? {
|
||||||
|
EventItem::Event(event) => { /* apply the MXAccess change */ }
|
||||||
|
EventItem::ReplayGap(gap) => {
|
||||||
|
// Recent history was evicted — discard local state and re-snapshot,
|
||||||
|
// then resume without provoking another gap:
|
||||||
|
let resume = gap.oldest_available_sequence.saturating_sub(1);
|
||||||
|
stream = session.events_after(resume).await?;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Almost every item is a normal `EventItem::Event`. `EventItem::ReplayGap` is a
|
||||||
|
faithful, typed surfacing of the gateway's reconnect-replay gap sentinel — the
|
||||||
|
client does not synthesize it. The gateway emits the sentinel at most once, at
|
||||||
|
the head of a stream **resumed** via `events_after` (`after_worker_sequence`)
|
||||||
|
when the requested sequence is older than the oldest event still retained in the
|
||||||
|
session's replay ring: events in the open interval
|
||||||
|
`(requested_after_sequence, oldest_available_sequence)` were evicted and cannot
|
||||||
|
be replayed. A `ReplayGap` therefore means "you missed events — discard any
|
||||||
|
local state and re-snapshot." To resume without a second gap, reconnect with
|
||||||
|
`events_after(gap.oldest_available_sequence - 1)`, which replays starting at the
|
||||||
|
first still-retained event. A stream opened from the beginning
|
||||||
|
(`session.events()` / `events_after(0)`) never produces a `ReplayGap`.
|
||||||
|
|
||||||
|
`EventItem` provides `as_event()`, `into_event()`, and `replay_gap()` accessors
|
||||||
|
for callers that prefer not to `match`. The `mxgw-cli stream-events` subcommand
|
||||||
|
renders the sentinel as a distinct `REPLAY_GAP …` line (or a `replayGap` JSON
|
||||||
|
object under `--json` / `--jsonl`).
|
||||||
|
|
||||||
## Write Semantics And Common Pitfalls
|
## Write Semantics And Common Pitfalls
|
||||||
|
|
||||||
These are MXAccess parity behaviors that surprise new callers. The gateway
|
These are MXAccess parity behaviors that surprise new callers. The gateway
|
||||||
|
|||||||
@@ -28,8 +28,8 @@ use zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::{
|
|||||||
WriteSecuredBulkEntry,
|
WriteSecuredBulkEntry,
|
||||||
};
|
};
|
||||||
use zb_mom_ww_mxgateway_client::{
|
use zb_mom_ww_mxgateway_client::{
|
||||||
next_correlation_id, ApiKey, ClientOptions, Error, GalaxyClient, GatewayClient, MxValue,
|
next_correlation_id, ApiKey, ClientOptions, Error, EventItem, GalaxyClient, GatewayClient,
|
||||||
MxValueProjection, CLIENT_VERSION, GATEWAY_PROTOCOL_VERSION, WORKER_PROTOCOL_VERSION,
|
MxValue, MxValueProjection, CLIENT_VERSION, GATEWAY_PROTOCOL_VERSION, WORKER_PROTOCOL_VERSION,
|
||||||
};
|
};
|
||||||
|
|
||||||
const MAX_AGGREGATE_EVENTS: usize = 10_000;
|
const MAX_AGGREGATE_EVENTS: usize = 10_000;
|
||||||
@@ -886,11 +886,13 @@ async fn dispatch(command: Command) -> Result<(), Error> {
|
|||||||
let mut events: Vec<Value> = Vec::new();
|
let mut events: Vec<Value> = Vec::new();
|
||||||
let mut event_count = 0usize;
|
let mut event_count = 0usize;
|
||||||
while event_count < max_events {
|
while event_count < max_events {
|
||||||
let Some(event) = stream.next().await else {
|
let Some(item) = stream.next().await else {
|
||||||
break;
|
break;
|
||||||
};
|
};
|
||||||
let event = event?;
|
let item = item?;
|
||||||
event_count += 1;
|
event_count += 1;
|
||||||
|
match item {
|
||||||
|
EventItem::Event(event) => {
|
||||||
if jsonl {
|
if jsonl {
|
||||||
println!("{}", event_to_json(&event));
|
println!("{}", event_to_json(&event));
|
||||||
} else if json {
|
} else if json {
|
||||||
@@ -899,6 +901,30 @@ async fn dispatch(command: Command) -> Result<(), Error> {
|
|||||||
println!("{} {}", event.worker_sequence, event.family);
|
println!("{} {}", event.worker_sequence, event.family);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// Reconnect-replay gap sentinel: recent history was evicted
|
||||||
|
// before this resumed stream could replay it. Render it as a
|
||||||
|
// distinct row so the caller can re-snapshot and resume with
|
||||||
|
// `oldest_available_sequence - 1`.
|
||||||
|
EventItem::ReplayGap(gap) => {
|
||||||
|
let value = json!({
|
||||||
|
"replayGap": {
|
||||||
|
"requestedAfterSequence": gap.requested_after_sequence,
|
||||||
|
"oldestAvailableSequence": gap.oldest_available_sequence,
|
||||||
|
}
|
||||||
|
});
|
||||||
|
if jsonl {
|
||||||
|
println!("{value}");
|
||||||
|
} else if json {
|
||||||
|
events.push(value);
|
||||||
|
} else {
|
||||||
|
println!(
|
||||||
|
"REPLAY_GAP requested_after={} oldest_available={}",
|
||||||
|
gap.requested_after_sequence, gap.oldest_available_sequence
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
if json {
|
if json {
|
||||||
// `eventCount` is preserved for back-compat; `events` carries
|
// `eventCount` is preserved for back-compat; `events` carries
|
||||||
// the per-event detail the cross-language e2e matrix compares.
|
// the per-event detail the cross-language e2e matrix compares.
|
||||||
|
|||||||
+121
-8
@@ -18,7 +18,7 @@ use crate::generated::mxaccess_gateway::v1::mx_access_gateway_client::MxAccessGa
|
|||||||
use crate::generated::mxaccess_gateway::v1::{
|
use crate::generated::mxaccess_gateway::v1::{
|
||||||
AcknowledgeAlarmReply, AcknowledgeAlarmRequest, ActiveAlarmSnapshot, AlarmFeedMessage,
|
AcknowledgeAlarmReply, AcknowledgeAlarmRequest, ActiveAlarmSnapshot, AlarmFeedMessage,
|
||||||
CloseSessionReply, CloseSessionRequest, MxCommandReply, MxCommandRequest, MxEvent,
|
CloseSessionReply, CloseSessionRequest, MxCommandReply, MxCommandRequest, MxEvent,
|
||||||
OpenSessionReply, OpenSessionRequest, QueryActiveAlarmsRequest, StreamAlarmsRequest,
|
OpenSessionReply, OpenSessionRequest, QueryActiveAlarmsRequest, ReplayGap, StreamAlarmsRequest,
|
||||||
StreamEventsRequest,
|
StreamEventsRequest,
|
||||||
};
|
};
|
||||||
use crate::options::{build_tls_config, ClientOptions};
|
use crate::options::{build_tls_config, ClientOptions};
|
||||||
@@ -28,11 +28,120 @@ use crate::session::Session;
|
|||||||
/// [`GatewayClient`] uses internally.
|
/// [`GatewayClient`] uses internally.
|
||||||
pub type RawGatewayClient = MxAccessGatewayClient<InterceptedService<Channel, AuthInterceptor>>;
|
pub type RawGatewayClient = MxAccessGatewayClient<InterceptedService<Channel, AuthInterceptor>>;
|
||||||
|
|
||||||
/// Pinned, boxed [`MxEvent`] stream returned by
|
/// One item yielded by the per-session event stream returned by
|
||||||
/// [`GatewayClient::stream_events`]. Errors are pre-mapped from
|
/// [`GatewayClient::stream_events`].
|
||||||
/// `tonic::Status` to [`Error`]; dropping the stream cancels the call.
|
///
|
||||||
|
/// Almost every item is an ordinary MXAccess event ([`EventItem::Event`]).
|
||||||
|
/// The one exception is the reconnect-replay gap sentinel
|
||||||
|
/// ([`EventItem::ReplayGap`]): the gateway emits it at most once, at the head
|
||||||
|
/// of a stream that was *resumed* via
|
||||||
|
/// [`Session::events_after`](crate::session::Session::events_after)
|
||||||
|
/// (`StreamEventsRequest.after_worker_sequence`) when the requested sequence is
|
||||||
|
/// older than the oldest event still retained in the session replay ring — i.e.
|
||||||
|
/// events were evicted and cannot be replayed.
|
||||||
|
///
|
||||||
|
/// The client does **not** synthesize this signal: it faithfully forwards the
|
||||||
|
/// gateway's sentinel `MxEvent` (whose `replay_gap` field is set), and only
|
||||||
|
/// makes it a distinct, typed variant so consumers can `match` on it instead of
|
||||||
|
/// inspecting a field on a value that otherwise looks like a normal event.
|
||||||
|
///
|
||||||
|
/// # Reacting to a gap
|
||||||
|
///
|
||||||
|
/// A [`EventItem::ReplayGap`] means "you missed events — discard any local
|
||||||
|
/// state and re-snapshot." The events in the open interval
|
||||||
|
/// `(requested_after_sequence, oldest_available_sequence)` are gone. To resume
|
||||||
|
/// the stream without provoking another gap, reconnect with
|
||||||
|
/// [`Session::events_after`](crate::session::Session::events_after) passing
|
||||||
|
/// `oldest_available_sequence - 1`, which replays starting at the first still
|
||||||
|
/// retained event (`oldest_available_sequence`):
|
||||||
|
///
|
||||||
|
/// ```no_run
|
||||||
|
/// # use zb_mom_ww_mxgateway_client::{EventItem, Session};
|
||||||
|
/// # use futures_util::StreamExt;
|
||||||
|
/// # async fn run(session: Session, cursor: u64) -> Result<(), zb_mom_ww_mxgateway_client::Error> {
|
||||||
|
/// let mut stream = session.events_after(cursor).await?;
|
||||||
|
/// while let Some(item) = stream.next().await {
|
||||||
|
/// match item? {
|
||||||
|
/// EventItem::Event(event) => {
|
||||||
|
/// let _ = event; // apply the change
|
||||||
|
/// }
|
||||||
|
/// EventItem::ReplayGap(gap) => {
|
||||||
|
/// // Local state is stale — re-snapshot, then resume without a gap.
|
||||||
|
/// let resume_cursor = gap.oldest_available_sequence.saturating_sub(1);
|
||||||
|
/// stream = session.events_after(resume_cursor).await?;
|
||||||
|
/// }
|
||||||
|
/// }
|
||||||
|
/// }
|
||||||
|
/// # Ok(())
|
||||||
|
/// # }
|
||||||
|
/// ```
|
||||||
|
// The `Event` variant is the hot path (nearly every stream item) and is the
|
||||||
|
// large one; the rare `ReplayGap` sentinel is small. Boxing `Event` to equalize
|
||||||
|
// the variants would add a heap allocation to every streamed event — a
|
||||||
|
// regression versus the prior `Result<MxEvent, Error>` surface, which already
|
||||||
|
// moved `MxEvent` by value. Keep the common path allocation-free.
|
||||||
|
#[allow(clippy::large_enum_variant)]
|
||||||
|
#[derive(Clone, Debug, PartialEq)]
|
||||||
|
pub enum EventItem {
|
||||||
|
/// A normal MXAccess event forwarded from the worker.
|
||||||
|
Event(MxEvent),
|
||||||
|
/// The reconnect-replay gap sentinel — recent event history was evicted
|
||||||
|
/// before this resumed stream could replay it. See [`EventItem`] for how
|
||||||
|
/// to react.
|
||||||
|
ReplayGap(ReplayGap),
|
||||||
|
}
|
||||||
|
|
||||||
|
impl EventItem {
|
||||||
|
/// Classify an incoming `MxEvent` into the typed stream item.
|
||||||
|
///
|
||||||
|
/// A present `replay_gap` promotes the event to [`EventItem::ReplayGap`];
|
||||||
|
/// otherwise it is an ordinary [`EventItem::Event`]. The sentinel is never
|
||||||
|
/// dropped and never surfaced as a normal event.
|
||||||
|
fn from_event(mut event: MxEvent) -> Self {
|
||||||
|
match event.replay_gap.take() {
|
||||||
|
Some(gap) => EventItem::ReplayGap(gap),
|
||||||
|
None => EventItem::Event(event),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Borrow the inner [`MxEvent`] when this item is a normal event, or
|
||||||
|
/// `None` when it is the [`EventItem::ReplayGap`] sentinel.
|
||||||
|
#[must_use]
|
||||||
|
pub fn as_event(&self) -> Option<&MxEvent> {
|
||||||
|
match self {
|
||||||
|
EventItem::Event(event) => Some(event),
|
||||||
|
EventItem::ReplayGap(_) => None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Borrow the [`ReplayGap`] when this item is the reconnect-replay gap
|
||||||
|
/// sentinel, or `None` for a normal event.
|
||||||
|
#[must_use]
|
||||||
|
pub fn replay_gap(&self) -> Option<&ReplayGap> {
|
||||||
|
match self {
|
||||||
|
EventItem::ReplayGap(gap) => Some(gap),
|
||||||
|
EventItem::Event(_) => None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Consume the item and return the inner [`MxEvent`] when it is a normal
|
||||||
|
/// event, or `None` for the [`EventItem::ReplayGap`] sentinel.
|
||||||
|
#[must_use]
|
||||||
|
pub fn into_event(self) -> Option<MxEvent> {
|
||||||
|
match self {
|
||||||
|
EventItem::Event(event) => Some(event),
|
||||||
|
EventItem::ReplayGap(_) => None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Pinned, boxed [`EventItem`] stream returned by
|
||||||
|
/// [`GatewayClient::stream_events`]. Each item is either a normal
|
||||||
|
/// [`EventItem::Event`] or the [`EventItem::ReplayGap`] reconnect-replay
|
||||||
|
/// sentinel. Errors are pre-mapped from `tonic::Status` to [`Error`]; dropping
|
||||||
|
/// the stream cancels the call.
|
||||||
pub type EventStream =
|
pub type EventStream =
|
||||||
std::pin::Pin<Box<dyn futures_core::Stream<Item = Result<MxEvent, Error>> + Send + 'static>>;
|
std::pin::Pin<Box<dyn futures_core::Stream<Item = Result<EventItem, Error>> + Send + 'static>>;
|
||||||
|
|
||||||
/// Pinned, boxed [`ActiveAlarmSnapshot`] stream returned by
|
/// Pinned, boxed [`ActiveAlarmSnapshot`] stream returned by
|
||||||
/// [`GatewayClient::query_active_alarms`]. Errors are pre-mapped from
|
/// [`GatewayClient::query_active_alarms`]. Errors are pre-mapped from
|
||||||
@@ -190,8 +299,12 @@ impl GatewayClient {
|
|||||||
|
|
||||||
/// Open the server-streaming `StreamEvents` RPC.
|
/// Open the server-streaming `StreamEvents` RPC.
|
||||||
///
|
///
|
||||||
/// The returned [`EventStream`] yields `MxEvent` messages as the worker
|
/// The returned [`EventStream`] yields [`EventItem`] values as the worker
|
||||||
/// produces them. Dropping the stream cancels the gRPC call cooperatively.
|
/// produces them: ordinary MXAccess events as [`EventItem::Event`], and the
|
||||||
|
/// gateway's reconnect-replay gap sentinel — set only on resumed streams
|
||||||
|
/// whose requested sequence predates the retained replay history — as
|
||||||
|
/// [`EventItem::ReplayGap`]. Dropping the stream cancels the gRPC call
|
||||||
|
/// cooperatively.
|
||||||
///
|
///
|
||||||
/// # Errors
|
/// # Errors
|
||||||
///
|
///
|
||||||
@@ -201,7 +314,7 @@ impl GatewayClient {
|
|||||||
let mut client = self.inner.clone();
|
let mut client = self.inner.clone();
|
||||||
let response = client.stream_events(self.stream_request(request)).await?;
|
let response = client.stream_events(self.stream_request(request)).await?;
|
||||||
let stream = futures_util::StreamExt::map(response.into_inner(), |result| {
|
let stream = futures_util::StreamExt::map(response.into_inner(), |result| {
|
||||||
result.map_err(Error::from)
|
result.map(EventItem::from_event).map_err(Error::from)
|
||||||
});
|
});
|
||||||
|
|
||||||
Ok(Box::pin(stream))
|
Ok(Box::pin(stream))
|
||||||
|
|||||||
@@ -24,12 +24,14 @@ pub mod version;
|
|||||||
#[doc(inline)]
|
#[doc(inline)]
|
||||||
pub use auth::{ApiKey, AuthInterceptor};
|
pub use auth::{ApiKey, AuthInterceptor};
|
||||||
#[doc(inline)]
|
#[doc(inline)]
|
||||||
pub use client::{AlarmFeedStream, EventStream, GatewayClient};
|
pub use client::{AlarmFeedStream, EventItem, EventStream, GatewayClient};
|
||||||
#[doc(inline)]
|
#[doc(inline)]
|
||||||
pub use error::{CommandError, Error, MxAccessError};
|
pub use error::{CommandError, Error, MxAccessError};
|
||||||
#[doc(inline)]
|
#[doc(inline)]
|
||||||
pub use galaxy::{DeployEventStream, GalaxyClient};
|
pub use galaxy::{DeployEventStream, GalaxyClient};
|
||||||
#[doc(inline)]
|
#[doc(inline)]
|
||||||
|
pub use generated::mxaccess_gateway::v1::ReplayGap;
|
||||||
|
#[doc(inline)]
|
||||||
pub use options::ClientOptions;
|
pub use options::ClientOptions;
|
||||||
#[doc(inline)]
|
#[doc(inline)]
|
||||||
pub use session::{next_correlation_id, Session};
|
pub use session::{next_correlation_id, Session};
|
||||||
|
|||||||
@@ -630,6 +630,13 @@ impl Session {
|
|||||||
|
|
||||||
/// Open the per-session event stream from the beginning.
|
/// Open the per-session event stream from the beginning.
|
||||||
///
|
///
|
||||||
|
/// The returned [`EventStream`] yields [`EventItem`](crate::EventItem)
|
||||||
|
/// values — normal MXAccess events as
|
||||||
|
/// [`EventItem::Event`](crate::EventItem::Event). A stream opened from the
|
||||||
|
/// beginning never produces a
|
||||||
|
/// [`EventItem::ReplayGap`](crate::EventItem::ReplayGap); that sentinel
|
||||||
|
/// only appears on a resumed stream (see [`Session::events_after`]).
|
||||||
|
///
|
||||||
/// # Errors
|
/// # Errors
|
||||||
///
|
///
|
||||||
/// Returns the `tonic::Status` mapped through [`Error::from`] when the
|
/// Returns the `tonic::Status` mapped through [`Error::from`] when the
|
||||||
@@ -642,6 +649,15 @@ impl Session {
|
|||||||
/// `worker_sequence` is greater than `after_worker_sequence`. Pass `0`
|
/// `worker_sequence` is greater than `after_worker_sequence`. Pass `0`
|
||||||
/// to receive every buffered event.
|
/// to receive every buffered event.
|
||||||
///
|
///
|
||||||
|
/// If `after_worker_sequence` predates the oldest event still retained in
|
||||||
|
/// the gateway's replay ring, the stream opens with a single
|
||||||
|
/// [`EventItem::ReplayGap`](crate::EventItem::ReplayGap) sentinel: recent
|
||||||
|
/// history was evicted and cannot be replayed, so the caller must discard
|
||||||
|
/// any local state and re-snapshot. To resume without provoking another
|
||||||
|
/// gap, call this method again with
|
||||||
|
/// `gap.oldest_available_sequence - 1`. See
|
||||||
|
/// [`EventItem`](crate::EventItem) for the full contract.
|
||||||
|
///
|
||||||
/// # Errors
|
/// # Errors
|
||||||
///
|
///
|
||||||
/// Same conditions as [`Session::events`].
|
/// Same conditions as [`Session::events`].
|
||||||
|
|||||||
@@ -27,13 +27,13 @@ use zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::{
|
|||||||
CloseSessionReply, CloseSessionRequest, MxCommandKind, MxCommandReply, MxDataType, MxEvent,
|
CloseSessionReply, CloseSessionRequest, MxCommandKind, MxCommandReply, MxDataType, MxEvent,
|
||||||
MxEventFamily, MxSparseArray, MxSparseElement, MxStatusCategory, MxStatusProxy, MxStatusSource,
|
MxEventFamily, MxSparseArray, MxSparseElement, MxStatusCategory, MxStatusProxy, MxStatusSource,
|
||||||
MxValue, OnAlarmTransitionEvent, OpenSessionReply, OpenSessionRequest, ProtocolStatus,
|
MxValue, OnAlarmTransitionEvent, OpenSessionReply, OpenSessionRequest, ProtocolStatus,
|
||||||
ProtocolStatusCode, QueryActiveAlarmsRequest, RegisterReply, SessionState, StreamAlarmsRequest,
|
ProtocolStatusCode, QueryActiveAlarmsRequest, RegisterReply, ReplayGap, SessionState,
|
||||||
StreamEventsRequest, SubscribeResult, Write2BulkEntry, WriteBulkEntry, WriteCommand,
|
StreamAlarmsRequest, StreamEventsRequest, SubscribeResult, Write2BulkEntry, WriteBulkEntry,
|
||||||
WriteSecured2BulkEntry, WriteSecuredBulkEntry,
|
WriteCommand, WriteSecured2BulkEntry, WriteSecuredBulkEntry,
|
||||||
};
|
};
|
||||||
use zb_mom_ww_mxgateway_client::{
|
use zb_mom_ww_mxgateway_client::{
|
||||||
next_correlation_id, ApiKey, ClientOptions, CommandError, Error, GatewayClient, MxStatus,
|
next_correlation_id, ApiKey, ClientOptions, CommandError, Error, EventItem, GatewayClient,
|
||||||
MxValue as ClientMxValue, MxValueProjection,
|
MxStatus, MxValue as ClientMxValue, MxValueProjection,
|
||||||
};
|
};
|
||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
@@ -128,8 +128,28 @@ async fn event_stream_preserves_order_and_drop_cancels_server_stream() {
|
|||||||
.await
|
.await
|
||||||
.unwrap();
|
.unwrap();
|
||||||
|
|
||||||
assert_eq!(stream.next().await.unwrap().unwrap().worker_sequence, 1);
|
assert_eq!(
|
||||||
assert_eq!(stream.next().await.unwrap().unwrap().worker_sequence, 2);
|
stream
|
||||||
|
.next()
|
||||||
|
.await
|
||||||
|
.unwrap()
|
||||||
|
.unwrap()
|
||||||
|
.as_event()
|
||||||
|
.unwrap()
|
||||||
|
.worker_sequence,
|
||||||
|
1
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
stream
|
||||||
|
.next()
|
||||||
|
.await
|
||||||
|
.unwrap()
|
||||||
|
.unwrap()
|
||||||
|
.as_event()
|
||||||
|
.unwrap()
|
||||||
|
.worker_sequence,
|
||||||
|
2
|
||||||
|
);
|
||||||
|
|
||||||
drop(stream);
|
drop(stream);
|
||||||
for _ in 0..20 {
|
for _ in 0..20 {
|
||||||
@@ -142,6 +162,55 @@ async fn event_stream_preserves_order_and_drop_cancels_server_stream() {
|
|||||||
assert!(state.stream_dropped.load(Ordering::SeqCst));
|
assert!(state.stream_dropped.load(Ordering::SeqCst));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn replay_gap_sentinel_surfaces_as_typed_event_item() {
|
||||||
|
let state = Arc::new(FakeState::default());
|
||||||
|
// Script a resumed stream: the reconnect-replay gap sentinel at the head
|
||||||
|
// (family UNSPECIFIED, no body, `replay_gap` set) followed by a normal
|
||||||
|
// event. The client must promote the sentinel to `EventItem::ReplayGap`
|
||||||
|
// and leave the following event as a normal `EventItem::Event`.
|
||||||
|
*state.stream_events_script.lock().await = Some(vec![
|
||||||
|
MxEvent {
|
||||||
|
replay_gap: Some(ReplayGap {
|
||||||
|
requested_after_sequence: 5,
|
||||||
|
oldest_available_sequence: 42,
|
||||||
|
}),
|
||||||
|
..MxEvent::default()
|
||||||
|
},
|
||||||
|
event(42),
|
||||||
|
]);
|
||||||
|
let endpoint = spawn_fake_gateway(state.clone()).await;
|
||||||
|
let client = GatewayClient::connect(ClientOptions::new(endpoint))
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let mut stream = client
|
||||||
|
.stream_events(StreamEventsRequest {
|
||||||
|
session_id: "session-fixture".to_owned(),
|
||||||
|
after_worker_sequence: 5,
|
||||||
|
})
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
// First item is the typed gap sentinel, not a normal event.
|
||||||
|
let first = stream.next().await.unwrap().unwrap();
|
||||||
|
match &first {
|
||||||
|
EventItem::ReplayGap(gap) => {
|
||||||
|
assert_eq!(gap.requested_after_sequence, 5);
|
||||||
|
assert_eq!(gap.oldest_available_sequence, 42);
|
||||||
|
}
|
||||||
|
EventItem::Event(_) => panic!("expected a ReplayGap sentinel, got a normal event"),
|
||||||
|
}
|
||||||
|
// Accessor helpers reflect the variant.
|
||||||
|
assert!(first.as_event().is_none());
|
||||||
|
assert_eq!(first.replay_gap().unwrap().oldest_available_sequence, 42);
|
||||||
|
|
||||||
|
// The normal event that follows is unaffected.
|
||||||
|
let second = stream.next().await.unwrap().unwrap();
|
||||||
|
assert_eq!(second.as_event().unwrap().worker_sequence, 42);
|
||||||
|
assert!(second.replay_gap().is_none());
|
||||||
|
}
|
||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn acknowledge_alarm_returns_reply_with_native_status() {
|
async fn acknowledge_alarm_returns_reply_with_native_status() {
|
||||||
let state = Arc::new(FakeState::default());
|
let state = Arc::new(FakeState::default());
|
||||||
@@ -672,6 +741,11 @@ struct FakeState {
|
|||||||
/// handler to emit a synthetic ConditionRefresh -> snapshot_complete
|
/// handler to emit a synthetic ConditionRefresh -> snapshot_complete
|
||||||
/// -> transition sequence.
|
/// -> transition sequence.
|
||||||
stream_alarms_script: Mutex<Option<Vec<AlarmFeedMessage>>>,
|
stream_alarms_script: Mutex<Option<Vec<AlarmFeedMessage>>>,
|
||||||
|
/// Optional per-test override that pins the fake's `StreamEvents`
|
||||||
|
/// handler to emit a scripted `MxEvent` sequence (e.g. a `replay_gap`
|
||||||
|
/// sentinel followed by a normal event). When `None`, the handler falls
|
||||||
|
/// back to the default `event(1)` / `event(2)` pair.
|
||||||
|
stream_events_script: Mutex<Option<Vec<MxEvent>>>,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Per-test override for the fake's `Invoke` handler.
|
/// Per-test override for the fake's `Invoke` handler.
|
||||||
@@ -921,9 +995,12 @@ impl MxAccessGateway for FakeGateway {
|
|||||||
&self,
|
&self,
|
||||||
_request: Request<StreamEventsRequest>,
|
_request: Request<StreamEventsRequest>,
|
||||||
) -> Result<Response<Self::StreamEventsStream>, Status> {
|
) -> Result<Response<Self::StreamEventsStream>, Status> {
|
||||||
let (sender, receiver) = mpsc::channel(4);
|
let script = self.state.stream_events_script.lock().await.take();
|
||||||
sender.send(Ok(event(1))).await.unwrap();
|
let events = script.unwrap_or_else(|| vec![event(1), event(2)]);
|
||||||
sender.send(Ok(event(2))).await.unwrap();
|
let (sender, receiver) = mpsc::channel(events.len().max(1));
|
||||||
|
for event in events {
|
||||||
|
sender.send(Ok(event)).await.unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
Ok(Response::new(DropAwareStream {
|
Ok(Response::new(DropAwareStream {
|
||||||
inner: ReceiverStream::new(receiver),
|
inner: ReceiverStream::new(receiver),
|
||||||
|
|||||||
@@ -63,12 +63,29 @@ Goals:
|
|||||||
|
|
||||||
Non-goals for v1:
|
Non-goals for v1:
|
||||||
|
|
||||||
- client-side reconnectable sessions,
|
- automatic client-side session reconnection (the library never transparently
|
||||||
- client-side event replay,
|
re-opens a dropped stream on the consumer's behalf — reconnect timing and
|
||||||
|
policy stay an application concern),
|
||||||
|
- client-side event replay buffering (the gateway owns the replay ring; the
|
||||||
|
client does not retain its own event history),
|
||||||
- client-side command batching,
|
- client-side command batching,
|
||||||
- synthetic MXAccess events,
|
- synthetic MXAccess events,
|
||||||
- hiding MXAccess handles behind opaque client-only handles.
|
- hiding MXAccess handles behind opaque client-only handles.
|
||||||
|
|
||||||
|
The gateway's reconnect-replay *protocol* is, however, consumable from every
|
||||||
|
client: each exposes the `after_worker_sequence` resume cursor on
|
||||||
|
`StreamEvents` and surfaces the gateway's `ReplayGap` sentinel as a distinct,
|
||||||
|
typed, non-terminal signal (CLI-15) so a consumer can detect an evicted-history
|
||||||
|
gap and re-snapshot. Per the no-synthesized-events invariant the client never
|
||||||
|
fabricates or swallows the sentinel — it only makes the gateway's own signal
|
||||||
|
observable. The reaction contract is identical across languages: on a gap,
|
||||||
|
discard cached state, re-snapshot, and resume with
|
||||||
|
`after_worker_sequence = oldest_available_sequence - 1`. The per-language
|
||||||
|
surface follows each idiom (.NET `MxEventStreamItem.IsReplayGap` via
|
||||||
|
`StreamEventItemsAsync`; Go `EventResult.ReplayGap`/`IsReplayGap()`; Rust
|
||||||
|
`EventItem::ReplayGap`; Python yields a typed `ReplayGap` from
|
||||||
|
`stream_events`; Java — pending, tracked with the JDK-17/windows client pass).
|
||||||
|
|
||||||
## Public Client Concepts
|
## Public Client Concepts
|
||||||
|
|
||||||
All languages should expose the same core concepts, using idiomatic naming:
|
All languages should expose the same core concepts, using idiomatic naming:
|
||||||
|
|||||||
@@ -30,6 +30,14 @@ Each client entry defines commands for the same required operation sequence:
|
|||||||
The optional `write` command is documented separately because writing changes
|
The optional `write` command is documented separately because writing changes
|
||||||
provider state and should only run when the operator supplies a safe test value.
|
provider state and should only run when the operator supplies a safe test value.
|
||||||
|
|
||||||
|
When `stream-events` is resumed with an `after_worker_sequence` cursor that
|
||||||
|
predates the oldest event still in the gateway's replay ring, the gateway emits a
|
||||||
|
single `ReplayGap` sentinel at the head of the stream. Every client surfaces this
|
||||||
|
as a distinct, typed, non-terminal signal (see each client README); the resume
|
||||||
|
contract is `after_worker_sequence = oldest_available_sequence - 1`. The default
|
||||||
|
smoke sequence opens a fresh stream (no cursor) and does not exercise the gap
|
||||||
|
path; a resume-with-gap fixture case is tracked separately (TST-24).
|
||||||
|
|
||||||
## Integration Gate
|
## Integration Gate
|
||||||
|
|
||||||
Cross-language smoke execution is opt-in. Runners should skip the matrix unless
|
Cross-language smoke execution is opt-in. Runners should skip the matrix unless
|
||||||
|
|||||||
Reference in New Issue
Block a user