feat(events): opt-in stream-attached acknowledgement on StreamEvents (#227)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m28s
ci / java (push) Failing after 2m21s
ci / portable (push) Failing after 3m33s

A fresh StreamEvents is never replayed, so an initial value published before
the gateway registers the stream's subscriber is lost (lmxopcua#865 —
slow-changing ScanState). Clients had nothing deterministic to wait on.

Contract (additive, proto3):
- StreamEventsRequest.emit_stream_attached = 3 (bool, default false)
- MxEvent.stream_attached = 15 (optional StreamAttached)
- message StreamAttached { uint64 live_after_sequence = 1; }
- OpenSessionReply capability "stream-events-attached-ack"
  (GatewayCapabilities.StreamEventsAttachedAck)

When requested, EventStreamService yields the sentinel as the FIRST message,
after AttachEventSubscriber/AttachEventSubscriberWithReplay has returned, so
receipt implies registration; refused attaches never see it. It precedes the
ReplayGap sentinel and replay on a resume. Shape mirrors ReplayGap: family
UNSPECIFIED, no body, worker_sequence 0.

Opt-in rather than always-on: ScadaBridge's MxGateway adapter stores every
streamed event's worker_sequence as its resume cursor, so an unrequested
sentinel would reset it to 0. Default wire behaviour is unchanged.

.NET client: StreamEventsAsync/StreamEventItemsAsync(after, emitStreamAttached),
MxEventStreamItem.IsStreamAttached, MxGatewaySession.SupportsStreamAttachedAck,
and OpenEventStreamAsync -> MxGatewayEventStream (awaits the sentinel, with
attach timeout; refuses older gateways).

Regenerated Contracts/Generated, the client protoset, Go and Python bindings
(also picks up comment-only drift already on main), synced the Rust vendored
proto and set emit_stream_attached in the Rust StreamEventsRequest literals.
No package versions bumped.

Tests: ordering (sentinel first, then events), race proof (event fanned right
after the sentinel is delivered — mutation-checked: fails when the sentinel is
yielded before registration), resume-with-gap order, refused attach emits no
sentinel, default emits none, contract field numbers, capability, and .NET
client wire tests (await, no-capability, older gateway, timeout).
This commit is contained in:
Joseph Doherty
2026-11-12 07:26:49 -05:00
parent 892bdc1a19
commit afd504d5ae
28 changed files with 2329 additions and 1005 deletions
+40
View File
@@ -137,6 +137,46 @@ gateway's own sentinel observable. Normal events pass through with
`IsReplayGap == false` and `ReplayGap == null`. The gap is only ever produced by
`StreamEvents`; the diagnostic drain path never emits it. If you already consume
the raw `StreamEventsAsync` (or the client-level stream), the
### Waiting For The Stream To Attach
A fresh stream (`afterWorkerSequence == 0`) is never replayed, so an initial value
the session publishes before the gateway has registered the stream is lost. That
matters for an advise whose only interesting event is its first one (a slow-changing
attribute such as `ScanState`). Opening the stream and then waiting an arbitrary
interval only narrows the race. `OpenEventStreamAsync` closes it: it asks the
gateway for the opt-in **stream-attached sentinel** (`emit_stream_attached`, #227)
and completes only when that sentinel arrives, which the gateway sends after it has
registered the stream.
```csharp
await using MxGatewayEventStream stream = await session.OpenEventStreamAsync(
attachTimeout: TimeSpan.FromSeconds(5));
// The stream is attached: an advise issued now cannot lose its initial value.
await session.SubscribeBulkAsync(addresses);
await foreach (MxEvent gatewayEvent in stream.ReadAllAsync(cancellationToken))
{
HandleEvent(gatewayEvent);
}
```
The sentinel is consumed by `OpenEventStreamAsync`; `ReadAllAsync` yields only what
follows it (a replay-gap sentinel on a resumed stream is still yielded). The method
throws `NotSupportedException` without streaming when the session's
`OpenSessionReply` lists capabilities but not `stream-events-attached-ack`
(`SupportsStreamAttachedAck` is `false`), i.e. the gateway predates #227. A session
bound with `AttachSession` carries no capability list, so the request is sent anyway;
an older gateway then answers with a normal event (`NotSupportedException`) or nothing
(`TimeoutException` once `attachTimeout` elapses).
Callers that run their own stream loop can use
`StreamEventsAsync(afterWorkerSequence, emitStreamAttached: true)` or the typed
`StreamEventItemsAsync(afterWorkerSequence, emitStreamAttached: true)` and watch for
`MxEvent.StreamAttached` / `MxEventStreamItem.IsStreamAttached`. The sentinel's
`WorkerSequence` is `0`: do not store it as the resume cursor. Without
`emitStreamAttached` the stream is exactly the pre-#227 shape.
### Caller-owned channels, resuming sessions, diagnostics
`MxGatewayClient.Create(GrpcChannel, MxGatewayClientOptions)` and
@@ -223,6 +223,47 @@ public sealed class MxGatewayClientSessionTests
Assert.Equal("session-fixture", request.SessionId);
}
/// <summary>
/// #227: the default stream call leaves emit_stream_attached unset (the pre-#227 wire shape);
/// the opt-in overload sets it and surfaces the sentinel as a typed item.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task StreamEventItemsAsync_OptIn_SurfacesStreamAttachedSentinel()
{
FakeGatewayTransport transport = CreateTransport();
transport.AddEvent(new MxEvent
{
SessionId = "session-fixture",
StreamAttached = new StreamAttached { LiveAfterSequence = 0 },
});
transport.AddEvent(new MxEvent
{
SessionId = "session-fixture",
Family = MxEventFamily.OnDataChange,
WorkerSequence = 1,
});
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
await foreach (MxEvent _ in session.StreamEventsAsync())
{
}
List<MxEventStreamItem> items = [];
await foreach (MxEventStreamItem item in session.StreamEventItemsAsync(0, emitStreamAttached: true))
{
items.Add(item);
}
Assert.False(transport.StreamEventsCalls[0].Request.EmitStreamAttached);
Assert.True(transport.StreamEventsCalls[1].Request.EmitStreamAttached);
Assert.True(items[0].IsStreamAttached);
Assert.False(items[0].IsReplayGap);
Assert.False(items[1].IsStreamAttached);
Assert.Equal(1UL, items[1].Event.WorkerSequence);
}
/// <summary>
/// Verifies that a reconnect-replay gap sentinel is surfaced as a typed
/// <see cref="MxEventStreamItem"/> with the gap populated, while normal
@@ -130,6 +130,87 @@ public sealed class MxGatewayClientWireTests
Assert.Equal(3ul, server.Service.StreamEventsRequest?.AfterWorkerSequence);
}
/// <summary>
/// #227: <see cref="MxGatewaySession.OpenEventStreamAsync"/> asks for the attach
/// acknowledgement, completes on the sentinel, and yields only the events after it.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task OpenEventStreamAsync_AwaitsTheAttachSentinel_ThenYieldsLaterEvents()
{
await using WireFakeGatewayServer server = await WireFakeGatewayServer.StartAsync(service =>
service.SupportsStreamAttached = true);
await using MxGatewayClient client = server.CreateClient(ApiKey);
MxGatewaySession session = await client.OpenSessionAsync(new OpenSessionRequest());
Assert.True(session.SupportsStreamAttachedAck);
await using MxGatewayEventStream stream = await session.OpenEventStreamAsync(
attachTimeout: TimeSpan.FromSeconds(10));
Assert.True(server.Service.StreamEventsRequest?.EmitStreamAttached);
Assert.Equal(0ul, stream.Attached.LiveAfterSequence);
List<MxEvent> events = [];
await foreach (MxEvent gatewayEvent in stream.ReadAllAsync())
{
events.Add(gatewayEvent);
}
MxEvent only = Assert.Single(events);
Assert.Null(only.StreamAttached);
Assert.Equal(9ul, only.WorkerSequence);
}
/// <summary>
/// #227: a session whose gateway does not advertise the capability fails fast with
/// <see cref="NotSupportedException"/> and never opens a stream.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task OpenEventStreamAsync_OnGatewayWithoutCapability_ThrowsWithoutStreaming()
{
await using WireFakeGatewayServer server = await WireFakeGatewayServer.StartAsync();
await using MxGatewayClient client = server.CreateClient(ApiKey);
MxGatewaySession session = await client.OpenSessionAsync(new OpenSessionRequest());
Assert.False(session.SupportsStreamAttachedAck);
await Assert.ThrowsAsync<NotSupportedException>(() => session.OpenEventStreamAsync());
Assert.Null(server.Service.StreamEventsRequest);
}
/// <summary>
/// #227: a session bound with AttachSession (no capability list) against an older gateway
/// sees a normal event first and is refused rather than handed an unacknowledged stream.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task OpenEventStreamAsync_AttachedSessionOnOlderGateway_RejectsNonSentinelFirstMessage()
{
await using WireFakeGatewayServer server = await WireFakeGatewayServer.StartAsync();
await using MxGatewayClient client = server.CreateClient(ApiKey);
MxGatewaySession session = client.AttachSession(WireFakeGatewayServer.FakeGatewayService.SessionId);
await Assert.ThrowsAsync<NotSupportedException>(
() => session.OpenEventStreamAsync(attachTimeout: TimeSpan.FromSeconds(10)));
Assert.True(server.Service.StreamEventsRequest?.EmitStreamAttached);
}
/// <summary>
/// #227: against a quiet older gateway nothing arrives; the attach timeout bounds the wait.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task OpenEventStreamAsync_WhenNoSentinelArrives_TimesOut()
{
await using WireFakeGatewayServer server = await WireFakeGatewayServer.StartAsync(service =>
service.StallStream = true);
await using MxGatewayClient client = server.CreateClient(ApiKey);
MxGatewaySession session = client.AttachSession(WireFakeGatewayServer.FakeGatewayService.SessionId);
await Assert.ThrowsAsync<TimeoutException>(
() => session.OpenEventStreamAsync(attachTimeout: TimeSpan.FromMilliseconds(200)));
}
/// <summary>
/// Verifies a genuine <c>PERMISSION_DENIED</c> status maps to the typed client
/// exception rather than a bare RpcException.
@@ -155,6 +155,19 @@ internal sealed class WireFakeGatewayServer : IAsyncDisposable
/// </summary>
public ReplayGap? ReplayGap { get; set; }
/// <summary>
/// Gets or sets whether this fake behaves as a #227 gateway: it advertises the
/// attach-acknowledgement capability and, when a request asks for it, writes the
/// stream-attached sentinel first. False models a gateway that predates #227.
/// </summary>
public bool SupportsStreamAttached { get; set; }
/// <summary>
/// Gets or sets whether the stream stalls (until the call is cancelled) instead of
/// writing the canned data-change event — a quiet session on an older gateway.
/// </summary>
public bool StallStream { get; set; }
/// <summary>
/// Gets the last <c>Invoke</c> request the client sent, as decoded from the wire.
/// </summary>
@@ -176,7 +189,7 @@ internal sealed class WireFakeGatewayServer : IAsyncDisposable
ServerCallContext context)
{
Record(context);
return Task.FromResult(new OpenSessionReply
OpenSessionReply reply = new()
{
SessionId = SessionId,
BackendName = "fake-backend",
@@ -185,7 +198,13 @@ internal sealed class WireFakeGatewayServer : IAsyncDisposable
GatewayProtocolVersion = 3,
Capabilities = { "events", "invoke" },
ProtocolStatus = Ok(),
});
};
if (SupportsStreamAttached)
{
reply.Capabilities.Add(ZB.MOM.WW.MxGateway.Contracts.GatewayCapabilities.StreamEventsAttachedAck);
}
return Task.FromResult(reply);
}
/// <inheritdoc />
@@ -230,6 +249,21 @@ internal sealed class WireFakeGatewayServer : IAsyncDisposable
Record(context);
StreamEventsRequest = request;
if (SupportsStreamAttached && request.EmitStreamAttached)
{
// The #227 sentinel shape: family unspecified, body unset, worker_sequence 0.
await responseStream.WriteAsync(new MxEvent
{
SessionId = request.SessionId,
StreamAttached = new StreamAttached { LiveAfterSequence = request.AfterWorkerSequence },
}).ConfigureAwait(false);
}
if (StallStream)
{
await Task.Delay(Timeout.InfiniteTimeSpan, context.CancellationToken).ConfigureAwait(false);
}
if (ReplayGap is not null)
{
// The sentinel shape the gateway emits: family unspecified, body unset,
@@ -12,7 +12,7 @@ public static class MxEventStreamExtensions
{
/// <summary>
/// Projects a raw <see cref="MxEvent"/> stream (e.g.
/// <see cref="MxGatewaySession.StreamEventsAsync"/> or
/// <see cref="MxGatewaySession.StreamEventsAsync(ulong, CancellationToken)"/> or
/// <see cref="MxGatewayClient.StreamEventsAsync"/>) into typed
/// <see cref="MxEventStreamItem"/> values. Normal events pass through with
/// <see cref="MxEventStreamItem.IsReplayGap"/> false; the gateway's
@@ -4,7 +4,8 @@ namespace ZB.MOM.WW.MxGateway.Client;
/// <summary>
/// One item yielded by the typed event stream. It is either a normal MXAccess
/// <see cref="MxEvent"/> or a reconnect-replay <em>gap sentinel</em>.
/// <see cref="MxEvent"/>, a reconnect-replay <em>gap sentinel</em>, or — only on a stream
/// that asked for it — the <em>stream-attached sentinel</em> (#227).
/// </summary>
/// <remarks>
/// <para>
@@ -30,13 +31,21 @@ namespace ZB.MOM.WW.MxGateway.Client;
/// incurring another gap, reconnect with
/// <c>after_worker_sequence = ReplayGap.OldestAvailableSequence - 1</c>.
/// </para>
/// <para>
/// When the stream was opened with <c>emit_stream_attached</c>, its first item is the
/// stream-attached sentinel: <see cref="IsStreamAttached"/> is <see langword="true"/> and
/// <see cref="StreamAttached"/> is populated. It means the gateway has registered the stream,
/// so every event published from then on is delivered. Its <c>worker_sequence</c> is 0 and
/// must not be used as a resume cursor.
/// </para>
/// </remarks>
public sealed class MxEventStreamItem
{
private MxEventStreamItem(MxEvent gatewayEvent, ReplayGap? replayGap)
private MxEventStreamItem(MxEvent gatewayEvent, ReplayGap? replayGap, StreamAttached? streamAttached)
{
Event = gatewayEvent;
ReplayGap = replayGap;
StreamAttached = streamAttached;
}
/// <summary>
@@ -64,6 +73,20 @@ public sealed class MxEventStreamItem
/// </summary>
public bool IsReplayGap => ReplayGap is not null;
/// <summary>
/// The attach acknowledgement when this item is the stream-attached sentinel (#227);
/// otherwise <see langword="null"/>. Only a stream opened with
/// <c>emit_stream_attached</c> carries it, and only as its first item.
/// </summary>
public StreamAttached? StreamAttached { get; }
/// <summary>
/// <see langword="true"/> when this item is the stream-attached sentinel: the gateway has
/// registered the stream, so an advise issued after this point cannot lose its initial value
/// to the attach race.
/// </summary>
public bool IsStreamAttached => StreamAttached is not null;
/// <summary>
/// Wraps a raw stream <see cref="MxEvent"/> as a typed item, classifying it
/// as a replay-gap sentinel when <c>MxEvent.replay_gap</c> is present.
@@ -75,8 +98,6 @@ public sealed class MxEventStreamItem
ArgumentNullException.ThrowIfNull(gatewayEvent);
// For a message-typed proto3 field, presence is a non-null reference.
return gatewayEvent.ReplayGap is { } replayGap
? new MxEventStreamItem(gatewayEvent, replayGap)
: new MxEventStreamItem(gatewayEvent, replayGap: null);
return new MxEventStreamItem(gatewayEvent, gatewayEvent.ReplayGap, gatewayEvent.StreamAttached);
}
}
@@ -123,7 +123,7 @@ public sealed class MxGatewayClient : IAsyncDisposable
/// <summary>
/// Binds a session object to an existing gateway session id without calling
/// OpenSession, so a reconnecting process can resume
/// <see cref="MxGatewaySession.StreamEventsAsync"/> with <c>afterWorkerSequence</c>.
/// <see cref="MxGatewaySession.StreamEventsAsync(ulong, CancellationToken)"/> with <c>afterWorkerSequence</c>.
/// The gateway only allows the API key that opened the session to stream its events
/// or invoke commands on it; any other key is rejected with PermissionDenied. The
/// returned session's <see cref="MxGatewaySession.OpenSessionReply"/> carries only the
@@ -0,0 +1,163 @@
using System.Runtime.CompilerServices;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client;
/// <summary>
/// An event stream the gateway has confirmed attached (#227), returned by
/// <see cref="MxGatewaySession.OpenEventStreamAsync"/>. Its existence is the attach signal:
/// every event the session publishes after it was created reaches <see cref="ReadAllAsync"/>.
/// </summary>
/// <remarks>
/// The stream-attached sentinel is consumed when the stream opens and is not yielded again.
/// A replay-gap sentinel on a resumed stream is a normal stream item and is yielded as-is
/// (inspect <see cref="MxEvent.ReplayGap"/>, or wrap with
/// <see cref="MxEventStreamExtensions.AsStreamItemsAsync"/>). Dispose to end the call.
/// </remarks>
public sealed class MxGatewayEventStream : IAsyncDisposable
{
private readonly IAsyncEnumerator<MxEvent> _enumerator;
private readonly CancellationTokenSource _lifetime;
private int _readStarted;
private int _disposed;
private MxGatewayEventStream(
IAsyncEnumerator<MxEvent> enumerator,
CancellationTokenSource lifetime,
StreamAttached attached)
{
_enumerator = enumerator;
_lifetime = lifetime;
Attached = attached;
}
/// <summary>
/// The gateway's attach acknowledgement. Live events on this stream have a
/// <c>worker_sequence</c> strictly greater than <see cref="StreamAttached.LiveAfterSequence"/>.
/// </summary>
public StreamAttached Attached { get; }
/// <summary>
/// Reads the events that follow the attach acknowledgement. May be enumerated once.
/// </summary>
/// <param name="cancellationToken">Cancels the enumeration and ends the stream.</param>
/// <returns>The remaining events, in gateway order.</returns>
/// <exception cref="InvalidOperationException">The stream was already enumerated.</exception>
public async IAsyncEnumerable<MxEvent> ReadAllAsync(
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
ObjectDisposedException.ThrowIf(Volatile.Read(ref _disposed) != 0, this);
if (Interlocked.Exchange(ref _readStarted, 1) != 0)
{
throw new InvalidOperationException("The event stream can be enumerated only once.");
}
using CancellationTokenRegistration registration = cancellationToken.Register(
static state => ((CancellationTokenSource)state!).Cancel(),
_lifetime);
while (await _enumerator.MoveNextAsync().ConfigureAwait(false))
{
yield return _enumerator.Current;
}
}
/// <summary>Cancels the underlying call and releases it.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
public async ValueTask DisposeAsync()
{
if (Interlocked.Exchange(ref _disposed, 1) != 0)
{
return;
}
await _lifetime.CancelAsync().ConfigureAwait(false);
try
{
await _enumerator.DisposeAsync().ConfigureAwait(false);
}
finally
{
_lifetime.Dispose();
}
}
// Opens the stream and consumes the first message, which must be the stream-attached
// sentinel. Anything else means the gateway ignored the request (it predates #227): the
// stream is closed rather than handed out, because it cannot keep the attach promise.
internal static async Task<MxGatewayEventStream> OpenAsync(
Func<CancellationToken, IAsyncEnumerable<MxEvent>> openStream,
string sessionId,
TimeSpan? attachTimeout,
CancellationToken cancellationToken)
{
CancellationTokenSource lifetime = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
IAsyncEnumerator<MxEvent> enumerator = openStream(lifetime.Token).GetAsyncEnumerator(lifetime.Token);
Task<bool>? first = null;
bool handedOff = false;
try
{
first = enumerator.MoveNextAsync().AsTask();
bool hasFirst;
try
{
hasFirst = attachTimeout is { } timeout
? await first.WaitAsync(timeout, cancellationToken).ConfigureAwait(false)
: await first.WaitAsync(cancellationToken).ConfigureAwait(false);
}
catch (TimeoutException) when (!first.IsCompleted)
{
throw new TimeoutException(
$"The gateway did not acknowledge the event-stream attach for session {sessionId} within {attachTimeout}.");
}
if (!hasFirst)
{
throw new MxGatewayException(
$"The gateway ended the event stream for session {sessionId} before acknowledging the attach.");
}
if (enumerator.Current.StreamAttached is not { } attached)
{
throw new NotSupportedException(
$"The gateway's first event-stream message for session {sessionId} was not the attach "
+ "acknowledgement; the gateway does not support emit_stream_attached.");
}
MxGatewayEventStream stream = new(enumerator, lifetime, attached);
handedOff = true;
return stream;
}
finally
{
if (!handedOff)
{
await lifetime.CancelAsync().ConfigureAwait(false);
// An async iterator cannot be disposed while a MoveNext is in flight (the timeout
// and caller-cancel paths leave one pending), so let the cancelled read unwind first.
// Its outcome is irrelevant: the failure being thrown is what the caller needs.
if (first is not null)
{
try
{
await first.ConfigureAwait(false);
}
catch (Exception)
{
// Swallowed deliberately; see above.
}
}
try
{
await enumerator.DisposeAsync().ConfigureAwait(false);
}
finally
{
lifetime.Dispose();
}
}
}
}
}
@@ -1,4 +1,5 @@
using Grpc.Core;
using ZB.MOM.WW.MxGateway.Contracts;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client;
@@ -1518,7 +1519,7 @@ public sealed class MxGatewaySession : IAsyncDisposable
/// <summary>
/// Drains events queued in the worker through the diagnostic command path. This removes
/// events from the worker queue; it is not a substitute for <see cref="StreamEventsAsync"/>
/// events from the worker queue; it is not a substitute for <see cref="StreamEventsAsync(ulong, CancellationToken)"/>
/// and is not retried automatically. The reply is bounded by server-side caps, so call
/// repeatedly until it returns an empty list.
/// </summary>
@@ -1572,6 +1573,111 @@ public sealed class MxGatewaySession : IAsyncDisposable
cancellationToken);
}
/// <summary>
/// Streams events, optionally asking the gateway for the stream-attached sentinel (#227).
/// </summary>
/// <remarks>
/// With <paramref name="emitStreamAttached"/> set, a gateway that advertises
/// <see cref="SupportsStreamAttachedAck"/> sends one sentinel <see cref="MxEvent"/> first, with
/// <see cref="MxEvent.StreamAttached"/> populated, after it has registered the stream. Consumers
/// that advise items only after seeing it cannot lose an initial value to the attach race. The
/// sentinel's <c>worker_sequence</c> is 0 and must not be used as a resume cursor. Use
/// <see cref="OpenEventStreamAsync"/> to await the sentinel instead of inspecting the stream.
/// </remarks>
/// <param name="afterWorkerSequence">The sequence number to stream from; 0 for a fresh stream.</param>
/// <param name="emitStreamAttached">Whether to ask the gateway for the stream-attached sentinel.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>An async enumerable of events.</returns>
public IAsyncEnumerable<MxEvent> StreamEventsAsync(
ulong afterWorkerSequence,
bool emitStreamAttached,
CancellationToken cancellationToken = default)
{
return _client.StreamEventsAsync(
new StreamEventsRequest
{
SessionId = SessionId,
AfterWorkerSequence = afterWorkerSequence,
EmitStreamAttached = emitStreamAttached,
},
cancellationToken);
}
/// <summary>
/// Typed-item variant of <see cref="StreamEventsAsync(ulong, bool, CancellationToken)"/>: the
/// stream-attached sentinel arrives as an item with <see cref="MxEventStreamItem.IsStreamAttached"/> set.
/// </summary>
/// <param name="afterWorkerSequence">The sequence number to stream from; 0 for a fresh stream.</param>
/// <param name="emitStreamAttached">Whether to ask the gateway for the stream-attached sentinel.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>An async enumerable of typed event items.</returns>
public IAsyncEnumerable<MxEventStreamItem> StreamEventItemsAsync(
ulong afterWorkerSequence,
bool emitStreamAttached,
CancellationToken cancellationToken = default)
{
return StreamEventsAsync(afterWorkerSequence, emitStreamAttached, cancellationToken)
.AsStreamItemsAsync(cancellationToken);
}
/// <summary>
/// <see langword="true"/> when the gateway listed
/// <see cref="GatewayCapabilities.StreamEventsAttachedAck"/> in this session's
/// <see cref="OpenSessionReply"/>, i.e. it honours the stream-attached request (#227).
/// <see langword="false"/> for an older gateway, and also for a session bound with
/// <see cref="MxGatewayClient.AttachSession"/>, whose reply carries no capabilities.
/// </summary>
public bool SupportsStreamAttachedAck =>
OpenSessionReply.Capabilities.Contains(GatewayCapabilities.StreamEventsAttachedAck);
/// <summary>
/// Opens the session's event stream and completes once the gateway confirms it is attached
/// (#227). Advise items after this returns: every event the session publishes from then on
/// reaches the returned stream, so no initial value is lost to the attach race.
/// </summary>
/// <remarks>
/// <para>
/// The returned <see cref="MxGatewayEventStream"/> owns the call; read the remaining events
/// with <see cref="MxGatewayEventStream.ReadAllAsync"/> (the sentinel itself is consumed here)
/// and dispose it to end the stream. <paramref name="cancellationToken"/> governs the whole
/// stream's lifetime, not only the wait; bound the wait with <paramref name="attachTimeout"/>.
/// </para>
/// <para>
/// When the session's reply lists capabilities but not the attach acknowledgement, the gateway
/// predates #227 and this throws <see cref="NotSupportedException"/> without opening a stream.
/// A session bound with <see cref="MxGatewayClient.AttachSession"/> has no capability list, so
/// the request is sent anyway; against an older gateway the first message is then a normal
/// event (or nothing), which ends the stream with <see cref="NotSupportedException"/> or
/// <see cref="TimeoutException"/>.
/// </para>
/// </remarks>
/// <param name="afterWorkerSequence">The sequence number to stream from; 0 for a fresh stream.</param>
/// <param name="attachTimeout">Upper bound on the wait for the sentinel; <see langword="null"/> waits indefinitely.</param>
/// <param name="cancellationToken">Cancels the wait and, afterwards, the stream.</param>
/// <returns>The attached stream.</returns>
/// <exception cref="NotSupportedException">The gateway does not send the attach acknowledgement.</exception>
/// <exception cref="TimeoutException">No sentinel arrived within <paramref name="attachTimeout"/>.</exception>
public async Task<MxGatewayEventStream> OpenEventStreamAsync(
ulong afterWorkerSequence = 0,
TimeSpan? attachTimeout = null,
CancellationToken cancellationToken = default)
{
if (OpenSessionReply.Capabilities.Count > 0 && !SupportsStreamAttachedAck)
{
throw new NotSupportedException(
$"The gateway for session {SessionId} does not advertise "
+ $"'{GatewayCapabilities.StreamEventsAttachedAck}'; it cannot acknowledge an event-stream attach.");
}
return await MxGatewayEventStream
.OpenAsync(
token => StreamEventsAsync(afterWorkerSequence, emitStreamAttached: true, token),
SessionId,
attachTimeout,
cancellationToken)
.ConfigureAwait(false);
}
/// <summary>
/// Streams events as typed <see cref="MxEventStreamItem"/> values, surfacing
/// the gateway's reconnect-replay gap sentinel as an observable, typed signal.
@@ -1586,7 +1692,7 @@ public sealed class MxGatewaySession : IAsyncDisposable
/// incurring another gap, reconnect with
/// <c>afterWorkerSequence = item.ReplayGap.OldestAvailableSequence - 1</c>.
/// All other events pass through with <see cref="MxEventStreamItem.IsReplayGap"/>
/// false. Use <see cref="StreamEventsAsync"/> when raw generated
/// false. Use <see cref="StreamEventsAsync(ulong, CancellationToken)"/> when raw generated
/// <see cref="MxEvent"/> messages are needed instead.
/// </remarks>
/// <param name="afterWorkerSequence">The sequence number to stream from. Defaults to 0.</param>
File diff suppressed because it is too large Load Diff
@@ -93,7 +93,11 @@ func (WorkerState) EnumDescriptor() ([]byte, []int) {
type WorkerFaultCategory int32
const (
WorkerFaultCategory_WORKER_FAULT_CATEGORY_UNSPECIFIED WorkerFaultCategory = 0
WorkerFaultCategory_WORKER_FAULT_CATEGORY_UNSPECIFIED WorkerFaultCategory = 0
// Values 1 and 2 are reserved: never emitted. Bootstrap failures (bad
// arguments, gateway authentication) exit via WorkerExitCode before the pipe
// exists, so there is no channel to carry a fault. Names kept so the numbers
// are not reused.
WorkerFaultCategory_WORKER_FAULT_CATEGORY_INVALID_ARGUMENTS WorkerFaultCategory = 1
WorkerFaultCategory_WORKER_FAULT_CATEGORY_GATEWAY_AUTHENTICATION_FAILED WorkerFaultCategory = 2
WorkerFaultCategory_WORKER_FAULT_CATEGORY_PROTOCOL_MISMATCH WorkerFaultCategory = 3
File diff suppressed because one or more lines are too long
@@ -227,6 +227,8 @@ pub(crate) async fn run(command: Command) -> Result<(), Error> {
.stream_events(StreamEventsRequest {
session_id,
after_worker_sequence,
// #227 attach acknowledgement is opt-in; this stream keeps the default wire shape.
emit_stream_attached: false,
})
.await?;
let mut events: Vec<Value> = Vec::new();
+57 -5
View File
@@ -84,6 +84,20 @@ message CloseSessionReply {
message StreamEventsRequest {
string session_id = 1;
uint64 after_worker_sequence = 2;
// Opt-in attach acknowledgement (mxaccessgw#227). When true, the gateway emits
// one StreamAttached sentinel MxEvent (MxEvent.stream_attached) as the FIRST
// message of this stream, sent only after this stream's subscriber is
// registered on the session. Every event the session publishes after that
// registration is delivered to this stream (subject only to the
// after_worker_sequence filter and the backpressure policy), so a client that
// waits for the sentinel before issuing AddItem/Advise/SubscribeBulk cannot
// lose the initial value to the attach race.
// Default false keeps the pre-#227 wire behaviour: no sentinel. It is opt-in
// because existing consumers treat every streamed MxEvent's worker_sequence as
// a resume cursor and would read the sentinel's 0 as one. A gateway that
// predates this field ignores it and never sends the sentinel; such a gateway
// does not list "stream-events-attached-ack" in OpenSessionReply.capabilities.
bool emit_stream_attached = 3;
}
message MxCommandRequest {
@@ -405,6 +419,9 @@ message AcknowledgeAlarmCommand {
string operator_user = 3;
string operator_node = 4;
string operator_domain = 5;
// Accepted for wire compatibility but NOT propagated to MXAccess: the 8-arg
// AlarmAckByName that carried it returns -55. operator_domain IS used (the
// AuthenticateUser identity is "<operator_domain>\<operator_user>").
string operator_full_name = 6;
// Operator credential used for AuthenticateUser before the AckMsg write.
// Implementations must keep this field out of logs, metrics labels, command
@@ -435,6 +452,9 @@ message AcknowledgeAlarmByNameCommand {
string operator_user = 5;
string operator_node = 6;
string operator_domain = 7;
// Accepted for wire compatibility but NOT propagated to MXAccess: the 8-arg
// AlarmAckByName that carried it returns -55. operator_domain IS used (the
// AuthenticateUser identity is "<operator_domain>\<operator_user>").
string operator_full_name = 8;
// Operator credential used for AuthenticateUser before the AckMsg write.
// Implementations must keep this field out of logs, metrics labels, command
@@ -518,11 +538,15 @@ message WriteSecured2BulkEntry {
// reply returns the cached value WITHOUT modifying the existing
// subscription (was_cached = true).
// - Otherwise the worker takes the snapshot lifecycle itself: AddItem +
// Advise, wait up to `timeout_ms` for the first OnDataChange, then
// UnAdvise + RemoveItem before returning. The session is left exactly
// as it was before the call (was_cached = false).
// Advise, wait for the first OnDataChange, then UnAdvise + RemoveItem
// before returning. The session is left exactly as it was before the
// call (was_cached = false).
//
// `timeout_ms == 0` uses the gateway-configured default (1000 ms).
// `timeout_ms` is one deadline shared by the whole command, not a per-tag
// budget: the worker adds and advises every uncached tag, waits once for all
// of them, then tears them all down. A tag that has not delivered by the
// deadline gets its own was_successful = false result.
// `timeout_ms == 0` uses the worker's default (1000 ms).
message ReadBulkCommand {
int32 server_handle = 1;
repeated string tag_addresses = 2;
@@ -549,6 +573,7 @@ message ShutdownWorkerCommand {
message MxCommandReply {
string session_id = 1;
// Echoes the request's client_correlation_id.
string correlation_id = 2;
MxCommandKind kind = 3;
ProtocolStatus protocol_status = 4;
@@ -790,6 +815,18 @@ message MxEvent {
// Additive (proto3): existing clients that ignore this field continue to
// deserialize the stream unchanged.
optional ReplayGap replay_gap = 14;
// Gateway-synthesized attach acknowledgement (mxaccessgw#227). Set ONLY on the
// single sentinel MxEvent the gateway emits as the FIRST message of a
// StreamEvents stream whose request set emit_stream_attached = true. It is
// sent after the stream's subscriber is registered, so every event the
// session publishes after it reaches this stream. On a resumed stream it
// precedes the ReplayGap sentinel (if any) and the replayed events.
// On that sentinel, `family` is UNSPECIFIED, the `body` oneof is unset,
// `worker_sequence` is 0, and no per-item fields are populated: it is a
// control signal, not an MXAccess event, and its worker_sequence MUST NOT be
// used as a resume cursor. Never set on a stream that did not opt in, and
// ALWAYS unset on events in DrainEventsReply.
optional StreamAttached stream_attached = 15;
oneof body {
OnDataChangeEvent on_data_change = 20;
@@ -827,6 +864,19 @@ message ReplayGap {
uint64 oldest_available_sequence = 2;
}
// Attach acknowledgement carried by a sentinel MxEvent (MxEvent.stream_attached)
// at the head of a StreamEvents stream that requested it
// (StreamEventsRequest.emit_stream_attached). Its arrival means the stream is
// registered on the session: any event published from then on is delivered.
message StreamAttached {
// Live events on this stream are those with worker_sequence strictly greater
// than this value. 0 on a fresh stream (after_worker_sequence = 0). On a
// resumed stream it is the last sequence the replay batch that follows
// covers (or the requested after_worker_sequence when nothing is replayed),
// so it equals the watermark the gateway applies to live events.
uint64 live_after_sequence = 1;
}
enum MxEventFamily {
MX_EVENT_FAMILY_UNSPECIFIED = 0;
MX_EVENT_FAMILY_ON_DATA_CHANGE = 1;
@@ -880,7 +930,7 @@ message OnAlarmTransitionEvent {
// When the alarm originally entered the active state. Preserved across
// acknowledge transitions so the Part 9 condition keeps the original raise
// time. Unset on retrigger from a previously-cleared condition.
// time. Unset when the raise time is unknown (RETRIGGER is reserved and never produced).
google.protobuf.Timestamp original_raise_timestamp = 6;
// When this specific transition occurred (raise time on Raise, ack time on
@@ -930,6 +980,8 @@ enum AlarmTransitionKind {
ALARM_TRANSITION_KIND_RAISE = 1;
ALARM_TRANSITION_KIND_ACKNOWLEDGE = 2;
ALARM_TRANSITION_KIND_CLEAR = 3;
// Reserved: never produced by the gateway or worker. Retained so the numeric
// value is not reused.
ALARM_TRANSITION_KIND_RETRIGGER = 4;
}
+2
View File
@@ -960,6 +960,8 @@ impl Session {
.stream_events(StreamEventsRequest {
session_id: self.id.clone(),
after_worker_sequence,
// #227 attach acknowledgement is opt-in; this stream keeps the default wire shape.
emit_stream_attached: false,
})
.await
}
+4
View File
@@ -128,6 +128,8 @@ async fn event_stream_preserves_order_and_drop_cancels_server_stream() {
.stream_events(StreamEventsRequest {
session_id: "session-fixture".to_owned(),
after_worker_sequence: 0,
// #227 attach acknowledgement is opt-in; this stream keeps the default wire shape.
emit_stream_attached: false,
})
.await
.unwrap();
@@ -192,6 +194,8 @@ async fn replay_gap_sentinel_surfaces_as_typed_event_item() {
.stream_events(StreamEventsRequest {
session_id: "session-fixture".to_owned(),
after_worker_sequence: 5,
// #227 attach acknowledgement is opt-in; this stream keeps the default wire shape.
emit_stream_attached: false,
})
.await
.unwrap();
+1 -1
View File
@@ -335,7 +335,7 @@ The pipeline has four stages:
1. The session is resolved via `sessionManager.TryGetSession`. A miss raises `SessionManagerException(SessionNotFound)` so the handler reports `StatusCode.NotFound`.
2. The attach is owner-scoped: the caller's API key id must match the key that opened the session, or the call fails with `PermissionDenied` before anything is attached or delivered.
3. The call registers a subscriber on the distributor. A fresh stream (`after_worker_sequence == 0`) uses `session.AttachEventSubscriber(...)`. A resume uses `session.AttachEventSubscriberWithReplay(...)`, which snapshots the replay ring for events newer than the cursor and registers the live subscriber under one lock, so the replay-to-live handoff has no gap and no duplicate. In single-subscriber mode (the default) a second attach fails with `EventSubscriberAlreadyActive`; with `Sessions:AllowMultipleEventSubscribers` enabled, an attach beyond `MaxEventSubscribersPerSession` fails with `EventSubscriberLimitReached`. Both map to `ResourceExhausted`. The lease is released in the `finally` block, so the subscriber slot is freed even when the client cancels mid-stream.
4. The call yields events from the lease's bounded channel (capacity `Events:QueueCapacity`). On a resume it first yields a `ReplayGap` sentinel when the cursor predates the oldest retained event, then the replayed batch, then live events filtered to sequences above the last replayed one. Per-session ordering is preserved end-to-end, because the pump is the single writer to each subscriber channel and writes in `WorkerSequence` order.
4. The call yields events from the lease's bounded channel (capacity `Events:QueueCapacity`). When the request set `emit_stream_attached` it first yields the StreamAttached sentinel — after step 3, so the client's receipt of it proves the subscriber is registered (see [Sessions](./Sessions.md#attach-acknowledgement)). On a resume it then yields a `ReplayGap` sentinel when the cursor predates the oldest retained event, then the replayed batch, then live events filtered to sequences above the last replayed one. Per-session ordering is preserved end-to-end, because the pump is the single writer to each subscriber channel and writes in `WorkerSequence` order.
The replay ring is server-side state that outlives a single RPC: with the default `DetachGraceSeconds` and `ReplayRetentionSeconds`, a client that drops can reconnect with its last `worker_sequence` and receive what it missed, as long as the events are still retained.
+11
View File
@@ -282,6 +282,7 @@ The handoff is sealed by a watermark. `RegisterWithReplay` returns `LiveResumeSe
Emit order on a resumed stream:
0. **StreamAttached sentinel (only when requested).** See [Attach acknowledgement](#attach-acknowledgement) below; when asked for, it precedes everything else on any stream, fresh or resumed.
1. **ReplayGap sentinel (only when events were evicted).** If the requested `after_worker_sequence` predates the oldest event still retained — i.e. events in the open interval were dropped by capacity or age eviction and are unrecoverable — the gateway first yields a single sentinel `MxEvent` with `replay_gap` populated (`requested_after_sequence` = the requested watermark, `oldest_available_sequence` = the resume anchor described below). The sentinel carries the session id; its `family` is `UNSPECIFIED`, its `body` oneof is unset, and no per-item fields are populated. It is an explicit, documented control signal — *not* a synthesized MXAccess event — telling the client to discard local state and re-snapshot. A client that wants to resume without another gap should set `after_worker_sequence = oldest_available_sequence - 1` on its next request.
2. **Retained replay batch.** The still-retained events newer than the requested watermark, in ascending `worker_sequence` order.
3. **Live events**, resuming strictly after `LiveResumeSequence`.
@@ -292,6 +293,16 @@ When `after_worker_sequence` is inside the retained window (nothing was evicted)
The ReplayGap sentinel is emitted **only** on the `StreamEvents` server stream and only to the resuming subscriber — it is never fanned to other subscribers and never appears in `DrainEventsReply` (the diagnostic drain path is untouched). Replay retention itself is bounded by `MxGateway:Events:ReplayBufferCapacity` (count), `ReplayRetentionSeconds` (age) and `ReplayBufferMaxBytes` (summed serialized size, so a burst of large-array events cannot pin memory the count cap would allow); whichever binds first evicts oldest-first. Evicted slots are cleared at once, so an evicted event is released immediately rather than when its slot is next reused. See [Configuration](GatewayConfiguration.md).
#### Attach acknowledgement
A fresh stream is not replayed, so an event the session publishes before the gateway registers the stream's subscriber never reaches that stream. A client that advises an item right after issuing `StreamEvents` can lose the item's initial value that way, and for a slow-changing attribute (`ScanState`, a setpoint) the next value may be hours away (lmxopcua#865). The client cannot see when the HTTP/2 request has been processed, so it has nothing to wait on.
`StreamEventsRequest.emit_stream_attached` (#227, default `false`) gives it something. When set, `EventStreamService.StreamEventsAsync` yields one sentinel `MxEvent` with `stream_attached` populated as the **first** message of the stream, and yields it only after `AttachEventSubscriber` / `AttachEventSubscriberWithReplay` has returned — that is, after the subscriber channel is registered on the distributor. Every event the pump fans from that point lands in the channel, and the client cannot receive the sentinel before the server has executed that line, so "sentinel received" implies "registered": an advise issued after it cannot lose its initial value to the attach race. An attach that is refused (owner mismatch, `EventSubscriberAlreadyActive`, limit reached, session not ready) throws before the sentinel, so a refused stream never sees it.
The sentinel carries the session id and `StreamAttached.live_after_sequence` — the watermark live events are filtered at (`0` on a fresh stream, `LiveResumeSequence` on a resume). Like the ReplayGap sentinel its `family` is `UNSPECIFIED`, its `body` oneof is unset, its `worker_sequence` is `0` (so clients must not store it as a resume cursor), and it is a documented control signal rather than a synthesized MXAccess event. It is sent only to the requesting subscriber and never appears in `DrainEventsReply`.
It is opt-in because existing consumers treat every streamed `MxEvent` as data: ScadaBridge's MxGateway adapter stores every event's `worker_sequence` as its reconnect cursor, so an unrequested sentinel would reset that cursor to `0`. A gateway that predates the field ignores it (proto3 unknown field) and never sends the sentinel; gateways that honour it list `stream-events-attached-ack` (`GatewayCapabilities.StreamEventsAttachedAck`) in `OpenSessionReply.capabilities`, so a client can check before it waits.
### Close
`GatewaySession.CloseAsync` is serialized by a per-session `SemaphoreSlim` (`_closeLock`) so only one close runs at a time, but every read/write of `_state` still passes through `_syncRoot` (via `TryBeginClose` and `MarkClosed`). The close path therefore obeys the same lock discipline as `TransitionTo` / `MarkFaulted`: it transitions to `Closing`, asks the worker client to shut down within `ShutdownTimeout`, and on success transitions to `Closed`. `DisposeAsync` waits on `_closeLock` once before disposing the semaphore so an in-flight close's `Release()` cannot race against the dispose. `WorkerClient.ShutdownAsync` is bounded by `ShutdownTimeout` end to end — queuing the `WorkerShutdown` envelope included, since a worker that stopped reading the pipe can leave the bounded outbound channel full with no watchdog left to clear it once the client is `Closing` — so a close never holds `_closeLock` longer than the timeout before reaching the fallback. If `WorkerClient.ShutdownAsync` throws, the session falls back to `IWorkerClient.Kill` (forced close):
+5
View File
@@ -1424,6 +1424,11 @@ Resolved for v1:
- Reconnectable sessions: clients reconnect by re-issuing `StreamEvents` with
`after_worker_sequence`; the gateway replays the retained ring tail and emits
a `ReplayGap` sentinel when events were evicted. See `docs/Sessions.md`.
- Attach acknowledgement (#227): a client that sets
`StreamEventsRequest.emit_stream_attached` receives a `StreamAttached`
sentinel first, once the stream's subscriber is registered, so it can advise
items without racing the first event. Opt-in; advertised as the
`stream-events-attached-ack` capability. See `docs/Sessions.md`.
- Multi-subscriber event fan-out: multiple concurrent `StreamEvents` callers on
the same session are supported; single-subscriber mode uses fail-fast
backpressure, multi-subscriber mode disconnects only the slow consumer. See
@@ -0,0 +1,16 @@
namespace ZB.MOM.WW.MxGateway.Contracts;
/// <summary>
/// Capability strings the gateway advertises in <c>OpenSessionReply.capabilities</c> that a
/// client may branch on. Shared so the gateway and the .NET client spell them identically.
/// </summary>
public static class GatewayCapabilities
{
/// <summary>
/// The gateway honours <c>StreamEventsRequest.emit_stream_attached</c> and sends the
/// <c>MxEvent.stream_attached</c> sentinel first on a stream that asks for it (#227).
/// A gateway that does not list this ignores the request field and never sends it, so a
/// client waiting for the acknowledgement would wait for the first real event instead.
/// </summary>
public const string StreamEventsAttachedAck = "stream-events-attached-ack";
}
File diff suppressed because it is too large Load Diff
@@ -84,6 +84,20 @@ message CloseSessionReply {
message StreamEventsRequest {
string session_id = 1;
uint64 after_worker_sequence = 2;
// Opt-in attach acknowledgement (mxaccessgw#227). When true, the gateway emits
// one StreamAttached sentinel MxEvent (MxEvent.stream_attached) as the FIRST
// message of this stream, sent only after this stream's subscriber is
// registered on the session. Every event the session publishes after that
// registration is delivered to this stream (subject only to the
// after_worker_sequence filter and the backpressure policy), so a client that
// waits for the sentinel before issuing AddItem/Advise/SubscribeBulk cannot
// lose the initial value to the attach race.
// Default false keeps the pre-#227 wire behaviour: no sentinel. It is opt-in
// because existing consumers treat every streamed MxEvent's worker_sequence as
// a resume cursor and would read the sentinel's 0 as one. A gateway that
// predates this field ignores it and never sends the sentinel; such a gateway
// does not list "stream-events-attached-ack" in OpenSessionReply.capabilities.
bool emit_stream_attached = 3;
}
message MxCommandRequest {
@@ -801,6 +815,18 @@ message MxEvent {
// Additive (proto3): existing clients that ignore this field continue to
// deserialize the stream unchanged.
optional ReplayGap replay_gap = 14;
// Gateway-synthesized attach acknowledgement (mxaccessgw#227). Set ONLY on the
// single sentinel MxEvent the gateway emits as the FIRST message of a
// StreamEvents stream whose request set emit_stream_attached = true. It is
// sent after the stream's subscriber is registered, so every event the
// session publishes after it reaches this stream. On a resumed stream it
// precedes the ReplayGap sentinel (if any) and the replayed events.
// On that sentinel, `family` is UNSPECIFIED, the `body` oneof is unset,
// `worker_sequence` is 0, and no per-item fields are populated: it is a
// control signal, not an MXAccess event, and its worker_sequence MUST NOT be
// used as a resume cursor. Never set on a stream that did not opt in, and
// ALWAYS unset on events in DrainEventsReply.
optional StreamAttached stream_attached = 15;
oneof body {
OnDataChangeEvent on_data_change = 20;
@@ -838,6 +864,19 @@ message ReplayGap {
uint64 oldest_available_sequence = 2;
}
// Attach acknowledgement carried by a sentinel MxEvent (MxEvent.stream_attached)
// at the head of a StreamEvents stream that requested it
// (StreamEventsRequest.emit_stream_attached). Its arrival means the stream is
// registered on the session: any event published from then on is delivered.
message StreamAttached {
// Live events on this stream are those with worker_sequence strictly greater
// than this value. 0 on a fresh stream (after_worker_sequence = 0). On a
// resumed stream it is the last sequence the replay batch that follows
// covers (or the requested after_worker_sequence when nothing is replayed),
// so it equals the watermark the gateway applies to live events.
uint64 live_after_sequence = 1;
}
enum MxEventFamily {
MX_EVENT_FAMILY_UNSPECIFIED = 0;
MX_EVENT_FAMILY_ON_DATA_CHANGE = 1;
@@ -135,6 +135,19 @@ public sealed class EventStreamService(
try
{
// Opt-in attach acknowledgement (#227). Emitted FIRST, and only here: by this point
// AttachEventSubscriber / AttachEventSubscriberWithReplay has returned, so this
// subscriber's channel is registered on the session distributor and every event
// the pump fans from now on lands in it (the replay variant also covered the ring
// under the same lock). The client sees the sentinel only after the server reached
// this line, so "sentinel received" implies "registered" — no publish can fall in
// between. Default off so the pre-#227 wire shape is unchanged for existing
// consumers (they treat every streamed worker_sequence as a resume cursor).
if (request.EmitStreamAttached)
{
yield return CreateStreamAttachedSentinel(request.SessionId, afterWorkerSequence);
}
// Emit order for a resume: the ReplayGap sentinel FIRST (only when events were
// evicted), then the still-retained replay batch, then live. The sentinel is an
// explicit documented control signal (not a synthesized MXAccess event) and is
@@ -249,6 +262,16 @@ public sealed class EventStreamService(
}
}
// Builds the opt-in StreamAttached control sentinel (#227): session id plus the live
// watermark, family UNSPECIFIED, no body, worker_sequence 0. Like the ReplayGap sentinel it
// is a documented control signal, not a synthesized MXAccess event.
private static MxEvent CreateStreamAttachedSentinel(string sessionId, ulong liveAfterSequence)
=> new()
{
SessionId = sessionId,
StreamAttached = new StreamAttached { LiveAfterSequence = liveAfterSequence },
};
// Builds the single ReplayGap control sentinel emitted at the head of a resumed
// StreamEvents stream when the requested AfterWorkerSequence predates the oldest event
// still retained (events were evicted). Per the proto contract (MxEvent.replay_gap),
@@ -60,6 +60,7 @@ public sealed class MxAccessGatewayService(
reply.Capabilities.Add("unary-close-session");
reply.Capabilities.Add("unary-invoke");
reply.Capabilities.Add("server-stream-events");
reply.Capabilities.Add(GatewayCapabilities.StreamEventsAttachedAck);
reply.Capabilities.Add("bulk-subscribe-commands");
reply.Capabilities.Add("bulk-read-commands");
reply.Capabilities.Add("bulk-write-commands");
@@ -1669,6 +1669,56 @@ public sealed class ProtobufContractRoundTripTests
Assert.Equal(200UL, parsed.ReplayGap.OldestAvailableSequence);
}
/// <summary>
/// Pins the #227 attach-acknowledgement wire numbers — <c>StreamEventsRequest.emit_stream_attached</c>
/// (3), <c>MxEvent.stream_attached</c> (15), <c>StreamAttached.live_after_sequence</c> (1) — and
/// round-trips the sentinel: body oneof unset, family UNSPECIFIED, presence preserved. Also pins
/// the additive property: a default request serializes exactly as the pre-#227 shape (proto3
/// omits a false bool), and a normal MxEvent parses with <c>stream_attached</c> absent.
/// </summary>
[Fact]
public void StreamAttached_RoundTripsAndPinsFieldNumbers()
{
Assert.Equal(3, StreamEventsRequest.EmitStreamAttachedFieldNumber);
Assert.Equal(15, MxEvent.StreamAttachedFieldNumber);
Assert.Equal(1, StreamAttached.LiveAfterSequenceFieldNumber);
Assert.Equal(
"live_after_sequence",
StreamAttached.Descriptor.FindFieldByNumber(StreamAttached.LiveAfterSequenceFieldNumber).Name);
var sentinel = new MxEvent
{
SessionId = "session-1",
StreamAttached = new StreamAttached { LiveAfterSequence = 42 },
};
var parsed = MxEvent.Parser.ParseFrom(sentinel.ToByteArray());
Assert.Equal(sentinel, parsed);
Assert.Equal(MxEvent.BodyOneofCase.None, parsed.BodyCase);
Assert.Equal(MxEventFamily.Unspecified, parsed.Family);
Assert.Equal(0UL, parsed.WorkerSequence);
Assert.Equal(42UL, parsed.StreamAttached.LiveAfterSequence);
// A sentinel with live_after_sequence 0 still carries presence (message field).
var fresh = MxEvent.Parser.ParseFrom(new MxEvent { StreamAttached = new StreamAttached() }.ToByteArray());
Assert.NotNull(fresh.StreamAttached);
// Normal events never carry it.
var normal = MxEvent.Parser.ParseFrom(new MxEvent
{
Family = MxEventFamily.OnDataChange,
WorkerSequence = 7,
OnDataChange = new OnDataChangeEvent(),
}.ToByteArray());
Assert.Null(normal.StreamAttached);
// Default request is byte-identical to the pre-#227 shape (proto3 omits a false bool).
var legacyRequest = new StreamEventsRequest { SessionId = "s", AfterWorkerSequence = 5 };
var optedIn = new StreamEventsRequest { SessionId = "s", AfterWorkerSequence = 5, EmitStreamAttached = true };
Assert.False(StreamEventsRequest.Parser.ParseFrom(legacyRequest.ToByteArray()).EmitStreamAttached);
Assert.True(optedIn.ToByteArray().Length > legacyRequest.ToByteArray().Length);
Assert.True(StreamEventsRequest.Parser.ParseFrom(optedIn.ToByteArray()).EmitStreamAttached);
}
/// <summary>
/// Pins the wire field number for <c>MxValue.sparse_array_value</c> (19)
/// via the generated constant, pins the <see cref="MxSparseArray"/> and
@@ -609,6 +609,169 @@ public sealed class EventStreamServiceTests
Assert.Equal(new ulong[] { 1, 2, 3 }, events.Select(e => e.WorkerSequence));
Assert.DoesNotContain(events, e => e.ReplayGap is not null);
// #227 is opt-in: a request that does not set emit_stream_attached gets no sentinel.
Assert.DoesNotContain(events, e => e.StreamAttached is not null);
}
/// <summary>
/// #227: with <c>emit_stream_attached</c> set on a fresh stream, the StreamAttached sentinel is
/// the FIRST message (family UNSPECIFIED, no body, worker_sequence 0, live_after_sequence 0),
/// followed by every session event in worker order — including the ones the pump drains only
/// because this attach started it.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task StreamEventsAsync_EmitStreamAttached_FreshStream_SentinelFirstThenEvents()
{
System.Threading.Channels.Channel<WorkerEvent> live =
System.Threading.Channels.Channel.CreateUnbounded<WorkerEvent>();
FakeWorkerClient workerClient = new() { LiveEvents = live };
workerClient.Events.Add(CreateWorkerEvent(1, MxEventFamily.OnDataChange));
workerClient.Events.Add(CreateWorkerEvent(2, MxEventFamily.OnDataChange));
GatewaySession session = CreateReadySession(workerClient);
EventStreamService service = CreateService(new FakeSessionManager(session));
await using IAsyncEnumerator<MxEvent> stream = service
.StreamEventsAsync(CreateRequest(session.SessionId, emitStreamAttached: true), callerKeyId: null, CancellationToken.None)
.GetAsyncEnumerator();
MxEvent attached = await ReadNextAsync(stream);
Assert.NotNull(attached.StreamAttached);
Assert.Equal(0ul, attached.StreamAttached.LiveAfterSequence);
Assert.Equal(MxEventFamily.Unspecified, attached.Family);
Assert.Equal(MxEvent.BodyOneofCase.None, attached.BodyCase);
Assert.Equal(0ul, attached.WorkerSequence);
Assert.Equal(session.SessionId, attached.SessionId);
Assert.Null(attached.ReplayGap);
Assert.Equal(1ul, (await ReadNextAsync(stream)).WorkerSequence);
Assert.Equal(2ul, (await ReadNextAsync(stream)).WorkerSequence);
live.Writer.TryWrite(CreateWorkerEvent(3, MxEventFamily.OnDataChange));
MxEvent liveEvent = await ReadNextAsync(stream);
Assert.Equal(3ul, liveEvent.WorkerSequence);
Assert.Null(liveEvent.StreamAttached);
}
/// <summary>
/// #227 race proof: on a session whose pump is ALREADY running, an event published after the
/// client observes the sentinel — and confirmed FANNED by the pump before the stream is read
/// again — is delivered to the new stream. Fresh attaches are not replayed, so this holds only
/// if the subscriber was registered before the sentinel was yielded. Emitting the sentinel ahead
/// of the registration makes this test time out: the event is fanned while the stream has no
/// subscriber, and the late registration never sees it.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task StreamEventsAsync_EmitStreamAttached_EventPublishedRightAfterSentinel_IsDelivered()
{
System.Threading.Channels.Channel<WorkerEvent> live =
System.Threading.Channels.Channel.CreateUnbounded<WorkerEvent>();
FakeWorkerClient workerClient = new() { LiveEvents = live };
GatewaySession session = CreateReadySession(workerClient);
EventStreamService service = CreateService(new FakeSessionManager(session));
// A gateway-internal observer (the alarm monitor's kind of lease) starts the pump and tells
// the test when an event has been fanned. It does not count against the single-subscriber slot.
using IEventSubscriberLease observer = session.AttachInternalEventSubscriber();
live.Writer.TryWrite(CreateWorkerEvent(1, MxEventFamily.OnDataChange));
Assert.Equal(1ul, (await ReadObserverAsync(observer)).WorkerSequence);
await using IAsyncEnumerator<MxEvent> stream = service
.StreamEventsAsync(CreateRequest(session.SessionId, emitStreamAttached: true), callerKeyId: null, CancellationToken.None)
.GetAsyncEnumerator();
MxEvent attached = await ReadNextAsync(stream);
Assert.NotNull(attached.StreamAttached);
// Published only after the sentinel was observed — the client's "now advise" moment — and
// fully fanned (the observer has it) before the stream is advanced.
live.Writer.TryWrite(CreateWorkerEvent(2, MxEventFamily.OnDataChange));
Assert.Equal(2ul, (await ReadObserverAsync(observer)).WorkerSequence);
MxEvent initialValue = await ReadNextAsync(stream);
Assert.Equal(2ul, initialValue.WorkerSequence);
Assert.Null(initialValue.StreamAttached);
}
private static async Task<MxEvent> ReadObserverAsync(IEventSubscriberLease observer)
=> await observer.Reader.ReadAsync().AsTask().WaitAsync(TestTimeout);
/// <summary>
/// #227 on a resumed stream: the StreamAttached sentinel precedes the ReplayGap sentinel and the
/// replay batch, and reports the live watermark (the last replayed sequence); live follows.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task StreamEventsAsync_EmitStreamAttached_OnResumeWithGap_PrecedesGapAndReplay()
{
System.Threading.Channels.Channel<WorkerEvent> live =
System.Threading.Channels.Channel.CreateUnbounded<WorkerEvent>();
FakeWorkerClient workerClient = new() { LiveEvents = live };
for (ulong sequence = 1; sequence <= 5; sequence++)
{
workerClient.Events.Add(CreateWorkerEvent(sequence, MxEventFamily.OnDataChange));
}
GatewaySession session = CreateReadySession(workerClient, replayBufferCapacity: 3);
EventStreamService service = CreateService(new FakeSessionManager(session));
await PrimeReplayAsync(service, session.SessionId, expectedCount: 5);
await using IAsyncEnumerator<MxEvent> resume = service
.StreamEventsAsync(
CreateRequest(session.SessionId, afterWorkerSequence: 1, emitStreamAttached: true),
callerKeyId: null,
CancellationToken.None)
.GetAsyncEnumerator();
MxEvent attached = await ReadNextAsync(resume);
Assert.NotNull(attached.StreamAttached);
Assert.Null(attached.ReplayGap);
Assert.Equal(5ul, attached.StreamAttached.LiveAfterSequence);
MxEvent gap = await ReadNextAsync(resume);
Assert.NotNull(gap.ReplayGap);
Assert.Null(gap.StreamAttached);
List<ulong> replayed = [];
for (int i = 0; i < 3; i++)
{
replayed.Add((await ReadNextAsync(resume)).WorkerSequence);
}
Assert.Equal(new ulong[] { 3, 4, 5 }, replayed);
live.Writer.TryWrite(CreateWorkerEvent(6, MxEventFamily.OnDataChange));
Assert.Equal(6ul, (await ReadNextAsync(resume)).WorkerSequence);
}
/// <summary>
/// #227: a rejected attach (second subscriber in single-subscriber mode) never emits the
/// sentinel — the sentinel is a promise of registration, so it must not precede a refusal.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task StreamEventsAsync_EmitStreamAttached_RejectedAttach_EmitsNoSentinel()
{
FakeWorkerClient workerClient = new();
GatewaySession session = CreateReadySession(workerClient);
EventStreamService service = CreateService(new FakeSessionManager(session));
await using IAsyncEnumerator<MxEvent> first = service
.StreamEventsAsync(CreateRequest(session.SessionId, emitStreamAttached: true), callerKeyId: null, CancellationToken.None)
.GetAsyncEnumerator();
Assert.NotNull((await ReadNextAsync(first)).StreamAttached);
// Sentinel observed => registered, with no polling: the count is already 1.
Assert.Equal(1, session.ActiveEventSubscriberCount);
await using IAsyncEnumerator<MxEvent> second = service
.StreamEventsAsync(CreateRequest(session.SessionId, emitStreamAttached: true), callerKeyId: null, CancellationToken.None)
.GetAsyncEnumerator();
SessionManagerException exception = await Assert.ThrowsAsync<SessionManagerException>(
async () => await second.MoveNextAsync().AsTask().WaitAsync(TestTimeout));
Assert.Equal(SessionManagerErrorCode.EventSubscriberAlreadyActive, exception.ErrorCode);
}
// Drains the first `expectedCount` events through a throwaway subscriber so the session's
@@ -805,12 +968,16 @@ public sealed class EventStreamServiceTests
return events;
}
private static StreamEventsRequest CreateRequest(string sessionId, ulong afterWorkerSequence = 0)
private static StreamEventsRequest CreateRequest(
string sessionId,
ulong afterWorkerSequence = 0,
bool emitStreamAttached = false)
{
return new StreamEventsRequest
{
SessionId = sessionId,
AfterWorkerSequence = afterWorkerSequence,
EmitStreamAttached = emitStreamAttached,
};
}
@@ -489,6 +489,28 @@ public sealed class MxAccessGatewayServiceTests
Assert.Contains("server-stream-active-alarms", reply.Capabilities);
}
/// <summary>
/// OpenSession advertises the #227 attach-acknowledgement capability, so a client can tell
/// a gateway that honours <c>emit_stream_attached</c> from one that silently ignores it.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task OpenSession_AdvertisesStreamAttachedAckCapability()
{
FakeSessionManager sessionManager = new();
GatewayRequestIdentityAccessor identityAccessor = new();
MxAccessGatewayService service = CreateService(sessionManager, identityAccessor);
using IDisposable identityScope = identityAccessor.Push(CreateIdentity());
OpenSessionReply reply = await service.OpenSession(
new OpenSessionRequest(),
new TestServerCallContext());
Assert.Contains("stream-events-attached-ack", reply.Capabilities);
Assert.Contains(GatewayCapabilities.StreamEventsAttachedAck, reply.Capabilities);
}
/// <summary>
/// Every reply echoes the request's <c>client_correlation_id</c>, including when the caller
/// sent none: the internal worker envelope id never reaches the client.