feat(events): opt-in stream-attached acknowledgement on StreamEvents (#227)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m28s
ci / java (push) Failing after 2m21s
ci / portable (push) Failing after 3m33s

A fresh StreamEvents is never replayed, so an initial value published before
the gateway registers the stream's subscriber is lost (lmxopcua#865 —
slow-changing ScanState). Clients had nothing deterministic to wait on.

Contract (additive, proto3):
- StreamEventsRequest.emit_stream_attached = 3 (bool, default false)
- MxEvent.stream_attached = 15 (optional StreamAttached)
- message StreamAttached { uint64 live_after_sequence = 1; }
- OpenSessionReply capability "stream-events-attached-ack"
  (GatewayCapabilities.StreamEventsAttachedAck)

When requested, EventStreamService yields the sentinel as the FIRST message,
after AttachEventSubscriber/AttachEventSubscriberWithReplay has returned, so
receipt implies registration; refused attaches never see it. It precedes the
ReplayGap sentinel and replay on a resume. Shape mirrors ReplayGap: family
UNSPECIFIED, no body, worker_sequence 0.

Opt-in rather than always-on: ScadaBridge's MxGateway adapter stores every
streamed event's worker_sequence as its resume cursor, so an unrequested
sentinel would reset it to 0. Default wire behaviour is unchanged.

.NET client: StreamEventsAsync/StreamEventItemsAsync(after, emitStreamAttached),
MxEventStreamItem.IsStreamAttached, MxGatewaySession.SupportsStreamAttachedAck,
and OpenEventStreamAsync -> MxGatewayEventStream (awaits the sentinel, with
attach timeout; refuses older gateways).

Regenerated Contracts/Generated, the client protoset, Go and Python bindings
(also picks up comment-only drift already on main), synced the Rust vendored
proto and set emit_stream_attached in the Rust StreamEventsRequest literals.
No package versions bumped.

Tests: ordering (sentinel first, then events), race proof (event fanned right
after the sentinel is delivered — mutation-checked: fails when the sentinel is
yielded before registration), resume-with-gap order, refused attach emits no
sentinel, default emits none, contract field numbers, capability, and .NET
client wire tests (await, no-capability, older gateway, timeout).
This commit is contained in:
Joseph Doherty
2026-11-12 07:26:49 -05:00
parent 892bdc1a19
commit afd504d5ae
28 changed files with 2329 additions and 1005 deletions
@@ -12,7 +12,7 @@ public static class MxEventStreamExtensions
{
/// <summary>
/// Projects a raw <see cref="MxEvent"/> stream (e.g.
/// <see cref="MxGatewaySession.StreamEventsAsync"/> or
/// <see cref="MxGatewaySession.StreamEventsAsync(ulong, CancellationToken)"/> or
/// <see cref="MxGatewayClient.StreamEventsAsync"/>) into typed
/// <see cref="MxEventStreamItem"/> values. Normal events pass through with
/// <see cref="MxEventStreamItem.IsReplayGap"/> false; the gateway's
@@ -4,7 +4,8 @@ namespace ZB.MOM.WW.MxGateway.Client;
/// <summary>
/// One item yielded by the typed event stream. It is either a normal MXAccess
/// <see cref="MxEvent"/> or a reconnect-replay <em>gap sentinel</em>.
/// <see cref="MxEvent"/>, a reconnect-replay <em>gap sentinel</em>, or — only on a stream
/// that asked for it — the <em>stream-attached sentinel</em> (#227).
/// </summary>
/// <remarks>
/// <para>
@@ -30,13 +31,21 @@ namespace ZB.MOM.WW.MxGateway.Client;
/// incurring another gap, reconnect with
/// <c>after_worker_sequence = ReplayGap.OldestAvailableSequence - 1</c>.
/// </para>
/// <para>
/// When the stream was opened with <c>emit_stream_attached</c>, its first item is the
/// stream-attached sentinel: <see cref="IsStreamAttached"/> is <see langword="true"/> and
/// <see cref="StreamAttached"/> is populated. It means the gateway has registered the stream,
/// so every event published from then on is delivered. Its <c>worker_sequence</c> is 0 and
/// must not be used as a resume cursor.
/// </para>
/// </remarks>
public sealed class MxEventStreamItem
{
private MxEventStreamItem(MxEvent gatewayEvent, ReplayGap? replayGap)
private MxEventStreamItem(MxEvent gatewayEvent, ReplayGap? replayGap, StreamAttached? streamAttached)
{
Event = gatewayEvent;
ReplayGap = replayGap;
StreamAttached = streamAttached;
}
/// <summary>
@@ -64,6 +73,20 @@ public sealed class MxEventStreamItem
/// </summary>
public bool IsReplayGap => ReplayGap is not null;
/// <summary>
/// The attach acknowledgement when this item is the stream-attached sentinel (#227);
/// otherwise <see langword="null"/>. Only a stream opened with
/// <c>emit_stream_attached</c> carries it, and only as its first item.
/// </summary>
public StreamAttached? StreamAttached { get; }
/// <summary>
/// <see langword="true"/> when this item is the stream-attached sentinel: the gateway has
/// registered the stream, so an advise issued after this point cannot lose its initial value
/// to the attach race.
/// </summary>
public bool IsStreamAttached => StreamAttached is not null;
/// <summary>
/// Wraps a raw stream <see cref="MxEvent"/> as a typed item, classifying it
/// as a replay-gap sentinel when <c>MxEvent.replay_gap</c> is present.
@@ -75,8 +98,6 @@ public sealed class MxEventStreamItem
ArgumentNullException.ThrowIfNull(gatewayEvent);
// For a message-typed proto3 field, presence is a non-null reference.
return gatewayEvent.ReplayGap is { } replayGap
? new MxEventStreamItem(gatewayEvent, replayGap)
: new MxEventStreamItem(gatewayEvent, replayGap: null);
return new MxEventStreamItem(gatewayEvent, gatewayEvent.ReplayGap, gatewayEvent.StreamAttached);
}
}
@@ -123,7 +123,7 @@ public sealed class MxGatewayClient : IAsyncDisposable
/// <summary>
/// Binds a session object to an existing gateway session id without calling
/// OpenSession, so a reconnecting process can resume
/// <see cref="MxGatewaySession.StreamEventsAsync"/> with <c>afterWorkerSequence</c>.
/// <see cref="MxGatewaySession.StreamEventsAsync(ulong, CancellationToken)"/> with <c>afterWorkerSequence</c>.
/// The gateway only allows the API key that opened the session to stream its events
/// or invoke commands on it; any other key is rejected with PermissionDenied. The
/// returned session's <see cref="MxGatewaySession.OpenSessionReply"/> carries only the
@@ -0,0 +1,163 @@
using System.Runtime.CompilerServices;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client;
/// <summary>
/// An event stream the gateway has confirmed attached (#227), returned by
/// <see cref="MxGatewaySession.OpenEventStreamAsync"/>. Its existence is the attach signal:
/// every event the session publishes after it was created reaches <see cref="ReadAllAsync"/>.
/// </summary>
/// <remarks>
/// The stream-attached sentinel is consumed when the stream opens and is not yielded again.
/// A replay-gap sentinel on a resumed stream is a normal stream item and is yielded as-is
/// (inspect <see cref="MxEvent.ReplayGap"/>, or wrap with
/// <see cref="MxEventStreamExtensions.AsStreamItemsAsync"/>). Dispose to end the call.
/// </remarks>
public sealed class MxGatewayEventStream : IAsyncDisposable
{
private readonly IAsyncEnumerator<MxEvent> _enumerator;
private readonly CancellationTokenSource _lifetime;
private int _readStarted;
private int _disposed;
private MxGatewayEventStream(
IAsyncEnumerator<MxEvent> enumerator,
CancellationTokenSource lifetime,
StreamAttached attached)
{
_enumerator = enumerator;
_lifetime = lifetime;
Attached = attached;
}
/// <summary>
/// The gateway's attach acknowledgement. Live events on this stream have a
/// <c>worker_sequence</c> strictly greater than <see cref="StreamAttached.LiveAfterSequence"/>.
/// </summary>
public StreamAttached Attached { get; }
/// <summary>
/// Reads the events that follow the attach acknowledgement. May be enumerated once.
/// </summary>
/// <param name="cancellationToken">Cancels the enumeration and ends the stream.</param>
/// <returns>The remaining events, in gateway order.</returns>
/// <exception cref="InvalidOperationException">The stream was already enumerated.</exception>
public async IAsyncEnumerable<MxEvent> ReadAllAsync(
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
ObjectDisposedException.ThrowIf(Volatile.Read(ref _disposed) != 0, this);
if (Interlocked.Exchange(ref _readStarted, 1) != 0)
{
throw new InvalidOperationException("The event stream can be enumerated only once.");
}
using CancellationTokenRegistration registration = cancellationToken.Register(
static state => ((CancellationTokenSource)state!).Cancel(),
_lifetime);
while (await _enumerator.MoveNextAsync().ConfigureAwait(false))
{
yield return _enumerator.Current;
}
}
/// <summary>Cancels the underlying call and releases it.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
public async ValueTask DisposeAsync()
{
if (Interlocked.Exchange(ref _disposed, 1) != 0)
{
return;
}
await _lifetime.CancelAsync().ConfigureAwait(false);
try
{
await _enumerator.DisposeAsync().ConfigureAwait(false);
}
finally
{
_lifetime.Dispose();
}
}
// Opens the stream and consumes the first message, which must be the stream-attached
// sentinel. Anything else means the gateway ignored the request (it predates #227): the
// stream is closed rather than handed out, because it cannot keep the attach promise.
internal static async Task<MxGatewayEventStream> OpenAsync(
Func<CancellationToken, IAsyncEnumerable<MxEvent>> openStream,
string sessionId,
TimeSpan? attachTimeout,
CancellationToken cancellationToken)
{
CancellationTokenSource lifetime = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
IAsyncEnumerator<MxEvent> enumerator = openStream(lifetime.Token).GetAsyncEnumerator(lifetime.Token);
Task<bool>? first = null;
bool handedOff = false;
try
{
first = enumerator.MoveNextAsync().AsTask();
bool hasFirst;
try
{
hasFirst = attachTimeout is { } timeout
? await first.WaitAsync(timeout, cancellationToken).ConfigureAwait(false)
: await first.WaitAsync(cancellationToken).ConfigureAwait(false);
}
catch (TimeoutException) when (!first.IsCompleted)
{
throw new TimeoutException(
$"The gateway did not acknowledge the event-stream attach for session {sessionId} within {attachTimeout}.");
}
if (!hasFirst)
{
throw new MxGatewayException(
$"The gateway ended the event stream for session {sessionId} before acknowledging the attach.");
}
if (enumerator.Current.StreamAttached is not { } attached)
{
throw new NotSupportedException(
$"The gateway's first event-stream message for session {sessionId} was not the attach "
+ "acknowledgement; the gateway does not support emit_stream_attached.");
}
MxGatewayEventStream stream = new(enumerator, lifetime, attached);
handedOff = true;
return stream;
}
finally
{
if (!handedOff)
{
await lifetime.CancelAsync().ConfigureAwait(false);
// An async iterator cannot be disposed while a MoveNext is in flight (the timeout
// and caller-cancel paths leave one pending), so let the cancelled read unwind first.
// Its outcome is irrelevant: the failure being thrown is what the caller needs.
if (first is not null)
{
try
{
await first.ConfigureAwait(false);
}
catch (Exception)
{
// Swallowed deliberately; see above.
}
}
try
{
await enumerator.DisposeAsync().ConfigureAwait(false);
}
finally
{
lifetime.Dispose();
}
}
}
}
}
@@ -1,4 +1,5 @@
using Grpc.Core;
using ZB.MOM.WW.MxGateway.Contracts;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client;
@@ -1518,7 +1519,7 @@ public sealed class MxGatewaySession : IAsyncDisposable
/// <summary>
/// Drains events queued in the worker through the diagnostic command path. This removes
/// events from the worker queue; it is not a substitute for <see cref="StreamEventsAsync"/>
/// events from the worker queue; it is not a substitute for <see cref="StreamEventsAsync(ulong, CancellationToken)"/>
/// and is not retried automatically. The reply is bounded by server-side caps, so call
/// repeatedly until it returns an empty list.
/// </summary>
@@ -1572,6 +1573,111 @@ public sealed class MxGatewaySession : IAsyncDisposable
cancellationToken);
}
/// <summary>
/// Streams events, optionally asking the gateway for the stream-attached sentinel (#227).
/// </summary>
/// <remarks>
/// With <paramref name="emitStreamAttached"/> set, a gateway that advertises
/// <see cref="SupportsStreamAttachedAck"/> sends one sentinel <see cref="MxEvent"/> first, with
/// <see cref="MxEvent.StreamAttached"/> populated, after it has registered the stream. Consumers
/// that advise items only after seeing it cannot lose an initial value to the attach race. The
/// sentinel's <c>worker_sequence</c> is 0 and must not be used as a resume cursor. Use
/// <see cref="OpenEventStreamAsync"/> to await the sentinel instead of inspecting the stream.
/// </remarks>
/// <param name="afterWorkerSequence">The sequence number to stream from; 0 for a fresh stream.</param>
/// <param name="emitStreamAttached">Whether to ask the gateway for the stream-attached sentinel.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>An async enumerable of events.</returns>
public IAsyncEnumerable<MxEvent> StreamEventsAsync(
ulong afterWorkerSequence,
bool emitStreamAttached,
CancellationToken cancellationToken = default)
{
return _client.StreamEventsAsync(
new StreamEventsRequest
{
SessionId = SessionId,
AfterWorkerSequence = afterWorkerSequence,
EmitStreamAttached = emitStreamAttached,
},
cancellationToken);
}
/// <summary>
/// Typed-item variant of <see cref="StreamEventsAsync(ulong, bool, CancellationToken)"/>: the
/// stream-attached sentinel arrives as an item with <see cref="MxEventStreamItem.IsStreamAttached"/> set.
/// </summary>
/// <param name="afterWorkerSequence">The sequence number to stream from; 0 for a fresh stream.</param>
/// <param name="emitStreamAttached">Whether to ask the gateway for the stream-attached sentinel.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>An async enumerable of typed event items.</returns>
public IAsyncEnumerable<MxEventStreamItem> StreamEventItemsAsync(
ulong afterWorkerSequence,
bool emitStreamAttached,
CancellationToken cancellationToken = default)
{
return StreamEventsAsync(afterWorkerSequence, emitStreamAttached, cancellationToken)
.AsStreamItemsAsync(cancellationToken);
}
/// <summary>
/// <see langword="true"/> when the gateway listed
/// <see cref="GatewayCapabilities.StreamEventsAttachedAck"/> in this session's
/// <see cref="OpenSessionReply"/>, i.e. it honours the stream-attached request (#227).
/// <see langword="false"/> for an older gateway, and also for a session bound with
/// <see cref="MxGatewayClient.AttachSession"/>, whose reply carries no capabilities.
/// </summary>
public bool SupportsStreamAttachedAck =>
OpenSessionReply.Capabilities.Contains(GatewayCapabilities.StreamEventsAttachedAck);
/// <summary>
/// Opens the session's event stream and completes once the gateway confirms it is attached
/// (#227). Advise items after this returns: every event the session publishes from then on
/// reaches the returned stream, so no initial value is lost to the attach race.
/// </summary>
/// <remarks>
/// <para>
/// The returned <see cref="MxGatewayEventStream"/> owns the call; read the remaining events
/// with <see cref="MxGatewayEventStream.ReadAllAsync"/> (the sentinel itself is consumed here)
/// and dispose it to end the stream. <paramref name="cancellationToken"/> governs the whole
/// stream's lifetime, not only the wait; bound the wait with <paramref name="attachTimeout"/>.
/// </para>
/// <para>
/// When the session's reply lists capabilities but not the attach acknowledgement, the gateway
/// predates #227 and this throws <see cref="NotSupportedException"/> without opening a stream.
/// A session bound with <see cref="MxGatewayClient.AttachSession"/> has no capability list, so
/// the request is sent anyway; against an older gateway the first message is then a normal
/// event (or nothing), which ends the stream with <see cref="NotSupportedException"/> or
/// <see cref="TimeoutException"/>.
/// </para>
/// </remarks>
/// <param name="afterWorkerSequence">The sequence number to stream from; 0 for a fresh stream.</param>
/// <param name="attachTimeout">Upper bound on the wait for the sentinel; <see langword="null"/> waits indefinitely.</param>
/// <param name="cancellationToken">Cancels the wait and, afterwards, the stream.</param>
/// <returns>The attached stream.</returns>
/// <exception cref="NotSupportedException">The gateway does not send the attach acknowledgement.</exception>
/// <exception cref="TimeoutException">No sentinel arrived within <paramref name="attachTimeout"/>.</exception>
public async Task<MxGatewayEventStream> OpenEventStreamAsync(
ulong afterWorkerSequence = 0,
TimeSpan? attachTimeout = null,
CancellationToken cancellationToken = default)
{
if (OpenSessionReply.Capabilities.Count > 0 && !SupportsStreamAttachedAck)
{
throw new NotSupportedException(
$"The gateway for session {SessionId} does not advertise "
+ $"'{GatewayCapabilities.StreamEventsAttachedAck}'; it cannot acknowledge an event-stream attach.");
}
return await MxGatewayEventStream
.OpenAsync(
token => StreamEventsAsync(afterWorkerSequence, emitStreamAttached: true, token),
SessionId,
attachTimeout,
cancellationToken)
.ConfigureAwait(false);
}
/// <summary>
/// Streams events as typed <see cref="MxEventStreamItem"/> values, surfacing
/// the gateway's reconnect-replay gap sentinel as an observable, typed signal.
@@ -1586,7 +1692,7 @@ public sealed class MxGatewaySession : IAsyncDisposable
/// incurring another gap, reconnect with
/// <c>afterWorkerSequence = item.ReplayGap.OldestAvailableSequence - 1</c>.
/// All other events pass through with <see cref="MxEventStreamItem.IsReplayGap"/>
/// false. Use <see cref="StreamEventsAsync"/> when raw generated
/// false. Use <see cref="StreamEventsAsync(ulong, CancellationToken)"/> when raw generated
/// <see cref="MxEvent"/> messages are needed instead.
/// </remarks>
/// <param name="afterWorkerSequence">The sequence number to stream from. Defaults to 0.</param>