feat(events): opt-in stream-attached acknowledgement on StreamEvents (#227)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m28s
ci / java (push) Failing after 2m21s
ci / portable (push) Failing after 3m33s

A fresh StreamEvents is never replayed, so an initial value published before
the gateway registers the stream's subscriber is lost (lmxopcua#865 —
slow-changing ScanState). Clients had nothing deterministic to wait on.

Contract (additive, proto3):
- StreamEventsRequest.emit_stream_attached = 3 (bool, default false)
- MxEvent.stream_attached = 15 (optional StreamAttached)
- message StreamAttached { uint64 live_after_sequence = 1; }
- OpenSessionReply capability "stream-events-attached-ack"
  (GatewayCapabilities.StreamEventsAttachedAck)

When requested, EventStreamService yields the sentinel as the FIRST message,
after AttachEventSubscriber/AttachEventSubscriberWithReplay has returned, so
receipt implies registration; refused attaches never see it. It precedes the
ReplayGap sentinel and replay on a resume. Shape mirrors ReplayGap: family
UNSPECIFIED, no body, worker_sequence 0.

Opt-in rather than always-on: ScadaBridge's MxGateway adapter stores every
streamed event's worker_sequence as its resume cursor, so an unrequested
sentinel would reset it to 0. Default wire behaviour is unchanged.

.NET client: StreamEventsAsync/StreamEventItemsAsync(after, emitStreamAttached),
MxEventStreamItem.IsStreamAttached, MxGatewaySession.SupportsStreamAttachedAck,
and OpenEventStreamAsync -> MxGatewayEventStream (awaits the sentinel, with
attach timeout; refuses older gateways).

Regenerated Contracts/Generated, the client protoset, Go and Python bindings
(also picks up comment-only drift already on main), synced the Rust vendored
proto and set emit_stream_attached in the Rust StreamEventsRequest literals.
No package versions bumped.

Tests: ordering (sentinel first, then events), race proof (event fanned right
after the sentinel is delivered — mutation-checked: fails when the sentinel is
yielded before registration), resume-with-gap order, refused attach emits no
sentinel, default emits none, contract field numbers, capability, and .NET
client wire tests (await, no-capability, older gateway, timeout).
This commit is contained in:
Joseph Doherty
2026-11-12 07:26:49 -05:00
parent 892bdc1a19
commit afd504d5ae
28 changed files with 2329 additions and 1005 deletions
@@ -0,0 +1,16 @@
namespace ZB.MOM.WW.MxGateway.Contracts;
/// <summary>
/// Capability strings the gateway advertises in <c>OpenSessionReply.capabilities</c> that a
/// client may branch on. Shared so the gateway and the .NET client spell them identically.
/// </summary>
public static class GatewayCapabilities
{
/// <summary>
/// The gateway honours <c>StreamEventsRequest.emit_stream_attached</c> and sends the
/// <c>MxEvent.stream_attached</c> sentinel first on a stream that asks for it (#227).
/// A gateway that does not list this ignores the request field and never sends it, so a
/// client waiting for the acknowledgement would wait for the first real event instead.
/// </summary>
public const string StreamEventsAttachedAck = "stream-events-attached-ack";
}
File diff suppressed because it is too large Load Diff
@@ -84,6 +84,20 @@ message CloseSessionReply {
message StreamEventsRequest {
string session_id = 1;
uint64 after_worker_sequence = 2;
// Opt-in attach acknowledgement (mxaccessgw#227). When true, the gateway emits
// one StreamAttached sentinel MxEvent (MxEvent.stream_attached) as the FIRST
// message of this stream, sent only after this stream's subscriber is
// registered on the session. Every event the session publishes after that
// registration is delivered to this stream (subject only to the
// after_worker_sequence filter and the backpressure policy), so a client that
// waits for the sentinel before issuing AddItem/Advise/SubscribeBulk cannot
// lose the initial value to the attach race.
// Default false keeps the pre-#227 wire behaviour: no sentinel. It is opt-in
// because existing consumers treat every streamed MxEvent's worker_sequence as
// a resume cursor and would read the sentinel's 0 as one. A gateway that
// predates this field ignores it and never sends the sentinel; such a gateway
// does not list "stream-events-attached-ack" in OpenSessionReply.capabilities.
bool emit_stream_attached = 3;
}
message MxCommandRequest {
@@ -801,6 +815,18 @@ message MxEvent {
// Additive (proto3): existing clients that ignore this field continue to
// deserialize the stream unchanged.
optional ReplayGap replay_gap = 14;
// Gateway-synthesized attach acknowledgement (mxaccessgw#227). Set ONLY on the
// single sentinel MxEvent the gateway emits as the FIRST message of a
// StreamEvents stream whose request set emit_stream_attached = true. It is
// sent after the stream's subscriber is registered, so every event the
// session publishes after it reaches this stream. On a resumed stream it
// precedes the ReplayGap sentinel (if any) and the replayed events.
// On that sentinel, `family` is UNSPECIFIED, the `body` oneof is unset,
// `worker_sequence` is 0, and no per-item fields are populated: it is a
// control signal, not an MXAccess event, and its worker_sequence MUST NOT be
// used as a resume cursor. Never set on a stream that did not opt in, and
// ALWAYS unset on events in DrainEventsReply.
optional StreamAttached stream_attached = 15;
oneof body {
OnDataChangeEvent on_data_change = 20;
@@ -838,6 +864,19 @@ message ReplayGap {
uint64 oldest_available_sequence = 2;
}
// Attach acknowledgement carried by a sentinel MxEvent (MxEvent.stream_attached)
// at the head of a StreamEvents stream that requested it
// (StreamEventsRequest.emit_stream_attached). Its arrival means the stream is
// registered on the session: any event published from then on is delivered.
message StreamAttached {
// Live events on this stream are those with worker_sequence strictly greater
// than this value. 0 on a fresh stream (after_worker_sequence = 0). On a
// resumed stream it is the last sequence the replay batch that follows
// covers (or the requested after_worker_sequence when nothing is replayed),
// so it equals the watermark the gateway applies to live events.
uint64 live_after_sequence = 1;
}
enum MxEventFamily {
MX_EVENT_FAMILY_UNSPECIFIED = 0;
MX_EVENT_FAMILY_ON_DATA_CHANGE = 1;
@@ -135,6 +135,19 @@ public sealed class EventStreamService(
try
{
// Opt-in attach acknowledgement (#227). Emitted FIRST, and only here: by this point
// AttachEventSubscriber / AttachEventSubscriberWithReplay has returned, so this
// subscriber's channel is registered on the session distributor and every event
// the pump fans from now on lands in it (the replay variant also covered the ring
// under the same lock). The client sees the sentinel only after the server reached
// this line, so "sentinel received" implies "registered" — no publish can fall in
// between. Default off so the pre-#227 wire shape is unchanged for existing
// consumers (they treat every streamed worker_sequence as a resume cursor).
if (request.EmitStreamAttached)
{
yield return CreateStreamAttachedSentinel(request.SessionId, afterWorkerSequence);
}
// Emit order for a resume: the ReplayGap sentinel FIRST (only when events were
// evicted), then the still-retained replay batch, then live. The sentinel is an
// explicit documented control signal (not a synthesized MXAccess event) and is
@@ -249,6 +262,16 @@ public sealed class EventStreamService(
}
}
// Builds the opt-in StreamAttached control sentinel (#227): session id plus the live
// watermark, family UNSPECIFIED, no body, worker_sequence 0. Like the ReplayGap sentinel it
// is a documented control signal, not a synthesized MXAccess event.
private static MxEvent CreateStreamAttachedSentinel(string sessionId, ulong liveAfterSequence)
=> new()
{
SessionId = sessionId,
StreamAttached = new StreamAttached { LiveAfterSequence = liveAfterSequence },
};
// Builds the single ReplayGap control sentinel emitted at the head of a resumed
// StreamEvents stream when the requested AfterWorkerSequence predates the oldest event
// still retained (events were evicted). Per the proto contract (MxEvent.replay_gap),
@@ -60,6 +60,7 @@ public sealed class MxAccessGatewayService(
reply.Capabilities.Add("unary-close-session");
reply.Capabilities.Add("unary-invoke");
reply.Capabilities.Add("server-stream-events");
reply.Capabilities.Add(GatewayCapabilities.StreamEventsAttachedAck);
reply.Capabilities.Add("bulk-subscribe-commands");
reply.Capabilities.Add("bulk-read-commands");
reply.Capabilities.Add("bulk-write-commands");
@@ -1669,6 +1669,56 @@ public sealed class ProtobufContractRoundTripTests
Assert.Equal(200UL, parsed.ReplayGap.OldestAvailableSequence);
}
/// <summary>
/// Pins the #227 attach-acknowledgement wire numbers — <c>StreamEventsRequest.emit_stream_attached</c>
/// (3), <c>MxEvent.stream_attached</c> (15), <c>StreamAttached.live_after_sequence</c> (1) — and
/// round-trips the sentinel: body oneof unset, family UNSPECIFIED, presence preserved. Also pins
/// the additive property: a default request serializes exactly as the pre-#227 shape (proto3
/// omits a false bool), and a normal MxEvent parses with <c>stream_attached</c> absent.
/// </summary>
[Fact]
public void StreamAttached_RoundTripsAndPinsFieldNumbers()
{
Assert.Equal(3, StreamEventsRequest.EmitStreamAttachedFieldNumber);
Assert.Equal(15, MxEvent.StreamAttachedFieldNumber);
Assert.Equal(1, StreamAttached.LiveAfterSequenceFieldNumber);
Assert.Equal(
"live_after_sequence",
StreamAttached.Descriptor.FindFieldByNumber(StreamAttached.LiveAfterSequenceFieldNumber).Name);
var sentinel = new MxEvent
{
SessionId = "session-1",
StreamAttached = new StreamAttached { LiveAfterSequence = 42 },
};
var parsed = MxEvent.Parser.ParseFrom(sentinel.ToByteArray());
Assert.Equal(sentinel, parsed);
Assert.Equal(MxEvent.BodyOneofCase.None, parsed.BodyCase);
Assert.Equal(MxEventFamily.Unspecified, parsed.Family);
Assert.Equal(0UL, parsed.WorkerSequence);
Assert.Equal(42UL, parsed.StreamAttached.LiveAfterSequence);
// A sentinel with live_after_sequence 0 still carries presence (message field).
var fresh = MxEvent.Parser.ParseFrom(new MxEvent { StreamAttached = new StreamAttached() }.ToByteArray());
Assert.NotNull(fresh.StreamAttached);
// Normal events never carry it.
var normal = MxEvent.Parser.ParseFrom(new MxEvent
{
Family = MxEventFamily.OnDataChange,
WorkerSequence = 7,
OnDataChange = new OnDataChangeEvent(),
}.ToByteArray());
Assert.Null(normal.StreamAttached);
// Default request is byte-identical to the pre-#227 shape (proto3 omits a false bool).
var legacyRequest = new StreamEventsRequest { SessionId = "s", AfterWorkerSequence = 5 };
var optedIn = new StreamEventsRequest { SessionId = "s", AfterWorkerSequence = 5, EmitStreamAttached = true };
Assert.False(StreamEventsRequest.Parser.ParseFrom(legacyRequest.ToByteArray()).EmitStreamAttached);
Assert.True(optedIn.ToByteArray().Length > legacyRequest.ToByteArray().Length);
Assert.True(StreamEventsRequest.Parser.ParseFrom(optedIn.ToByteArray()).EmitStreamAttached);
}
/// <summary>
/// Pins the wire field number for <c>MxValue.sparse_array_value</c> (19)
/// via the generated constant, pins the <see cref="MxSparseArray"/> and
@@ -609,6 +609,169 @@ public sealed class EventStreamServiceTests
Assert.Equal(new ulong[] { 1, 2, 3 }, events.Select(e => e.WorkerSequence));
Assert.DoesNotContain(events, e => e.ReplayGap is not null);
// #227 is opt-in: a request that does not set emit_stream_attached gets no sentinel.
Assert.DoesNotContain(events, e => e.StreamAttached is not null);
}
/// <summary>
/// #227: with <c>emit_stream_attached</c> set on a fresh stream, the StreamAttached sentinel is
/// the FIRST message (family UNSPECIFIED, no body, worker_sequence 0, live_after_sequence 0),
/// followed by every session event in worker order — including the ones the pump drains only
/// because this attach started it.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task StreamEventsAsync_EmitStreamAttached_FreshStream_SentinelFirstThenEvents()
{
System.Threading.Channels.Channel<WorkerEvent> live =
System.Threading.Channels.Channel.CreateUnbounded<WorkerEvent>();
FakeWorkerClient workerClient = new() { LiveEvents = live };
workerClient.Events.Add(CreateWorkerEvent(1, MxEventFamily.OnDataChange));
workerClient.Events.Add(CreateWorkerEvent(2, MxEventFamily.OnDataChange));
GatewaySession session = CreateReadySession(workerClient);
EventStreamService service = CreateService(new FakeSessionManager(session));
await using IAsyncEnumerator<MxEvent> stream = service
.StreamEventsAsync(CreateRequest(session.SessionId, emitStreamAttached: true), callerKeyId: null, CancellationToken.None)
.GetAsyncEnumerator();
MxEvent attached = await ReadNextAsync(stream);
Assert.NotNull(attached.StreamAttached);
Assert.Equal(0ul, attached.StreamAttached.LiveAfterSequence);
Assert.Equal(MxEventFamily.Unspecified, attached.Family);
Assert.Equal(MxEvent.BodyOneofCase.None, attached.BodyCase);
Assert.Equal(0ul, attached.WorkerSequence);
Assert.Equal(session.SessionId, attached.SessionId);
Assert.Null(attached.ReplayGap);
Assert.Equal(1ul, (await ReadNextAsync(stream)).WorkerSequence);
Assert.Equal(2ul, (await ReadNextAsync(stream)).WorkerSequence);
live.Writer.TryWrite(CreateWorkerEvent(3, MxEventFamily.OnDataChange));
MxEvent liveEvent = await ReadNextAsync(stream);
Assert.Equal(3ul, liveEvent.WorkerSequence);
Assert.Null(liveEvent.StreamAttached);
}
/// <summary>
/// #227 race proof: on a session whose pump is ALREADY running, an event published after the
/// client observes the sentinel — and confirmed FANNED by the pump before the stream is read
/// again — is delivered to the new stream. Fresh attaches are not replayed, so this holds only
/// if the subscriber was registered before the sentinel was yielded. Emitting the sentinel ahead
/// of the registration makes this test time out: the event is fanned while the stream has no
/// subscriber, and the late registration never sees it.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task StreamEventsAsync_EmitStreamAttached_EventPublishedRightAfterSentinel_IsDelivered()
{
System.Threading.Channels.Channel<WorkerEvent> live =
System.Threading.Channels.Channel.CreateUnbounded<WorkerEvent>();
FakeWorkerClient workerClient = new() { LiveEvents = live };
GatewaySession session = CreateReadySession(workerClient);
EventStreamService service = CreateService(new FakeSessionManager(session));
// A gateway-internal observer (the alarm monitor's kind of lease) starts the pump and tells
// the test when an event has been fanned. It does not count against the single-subscriber slot.
using IEventSubscriberLease observer = session.AttachInternalEventSubscriber();
live.Writer.TryWrite(CreateWorkerEvent(1, MxEventFamily.OnDataChange));
Assert.Equal(1ul, (await ReadObserverAsync(observer)).WorkerSequence);
await using IAsyncEnumerator<MxEvent> stream = service
.StreamEventsAsync(CreateRequest(session.SessionId, emitStreamAttached: true), callerKeyId: null, CancellationToken.None)
.GetAsyncEnumerator();
MxEvent attached = await ReadNextAsync(stream);
Assert.NotNull(attached.StreamAttached);
// Published only after the sentinel was observed — the client's "now advise" moment — and
// fully fanned (the observer has it) before the stream is advanced.
live.Writer.TryWrite(CreateWorkerEvent(2, MxEventFamily.OnDataChange));
Assert.Equal(2ul, (await ReadObserverAsync(observer)).WorkerSequence);
MxEvent initialValue = await ReadNextAsync(stream);
Assert.Equal(2ul, initialValue.WorkerSequence);
Assert.Null(initialValue.StreamAttached);
}
private static async Task<MxEvent> ReadObserverAsync(IEventSubscriberLease observer)
=> await observer.Reader.ReadAsync().AsTask().WaitAsync(TestTimeout);
/// <summary>
/// #227 on a resumed stream: the StreamAttached sentinel precedes the ReplayGap sentinel and the
/// replay batch, and reports the live watermark (the last replayed sequence); live follows.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task StreamEventsAsync_EmitStreamAttached_OnResumeWithGap_PrecedesGapAndReplay()
{
System.Threading.Channels.Channel<WorkerEvent> live =
System.Threading.Channels.Channel.CreateUnbounded<WorkerEvent>();
FakeWorkerClient workerClient = new() { LiveEvents = live };
for (ulong sequence = 1; sequence <= 5; sequence++)
{
workerClient.Events.Add(CreateWorkerEvent(sequence, MxEventFamily.OnDataChange));
}
GatewaySession session = CreateReadySession(workerClient, replayBufferCapacity: 3);
EventStreamService service = CreateService(new FakeSessionManager(session));
await PrimeReplayAsync(service, session.SessionId, expectedCount: 5);
await using IAsyncEnumerator<MxEvent> resume = service
.StreamEventsAsync(
CreateRequest(session.SessionId, afterWorkerSequence: 1, emitStreamAttached: true),
callerKeyId: null,
CancellationToken.None)
.GetAsyncEnumerator();
MxEvent attached = await ReadNextAsync(resume);
Assert.NotNull(attached.StreamAttached);
Assert.Null(attached.ReplayGap);
Assert.Equal(5ul, attached.StreamAttached.LiveAfterSequence);
MxEvent gap = await ReadNextAsync(resume);
Assert.NotNull(gap.ReplayGap);
Assert.Null(gap.StreamAttached);
List<ulong> replayed = [];
for (int i = 0; i < 3; i++)
{
replayed.Add((await ReadNextAsync(resume)).WorkerSequence);
}
Assert.Equal(new ulong[] { 3, 4, 5 }, replayed);
live.Writer.TryWrite(CreateWorkerEvent(6, MxEventFamily.OnDataChange));
Assert.Equal(6ul, (await ReadNextAsync(resume)).WorkerSequence);
}
/// <summary>
/// #227: a rejected attach (second subscriber in single-subscriber mode) never emits the
/// sentinel — the sentinel is a promise of registration, so it must not precede a refusal.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task StreamEventsAsync_EmitStreamAttached_RejectedAttach_EmitsNoSentinel()
{
FakeWorkerClient workerClient = new();
GatewaySession session = CreateReadySession(workerClient);
EventStreamService service = CreateService(new FakeSessionManager(session));
await using IAsyncEnumerator<MxEvent> first = service
.StreamEventsAsync(CreateRequest(session.SessionId, emitStreamAttached: true), callerKeyId: null, CancellationToken.None)
.GetAsyncEnumerator();
Assert.NotNull((await ReadNextAsync(first)).StreamAttached);
// Sentinel observed => registered, with no polling: the count is already 1.
Assert.Equal(1, session.ActiveEventSubscriberCount);
await using IAsyncEnumerator<MxEvent> second = service
.StreamEventsAsync(CreateRequest(session.SessionId, emitStreamAttached: true), callerKeyId: null, CancellationToken.None)
.GetAsyncEnumerator();
SessionManagerException exception = await Assert.ThrowsAsync<SessionManagerException>(
async () => await second.MoveNextAsync().AsTask().WaitAsync(TestTimeout));
Assert.Equal(SessionManagerErrorCode.EventSubscriberAlreadyActive, exception.ErrorCode);
}
// Drains the first `expectedCount` events through a throwaway subscriber so the session's
@@ -805,12 +968,16 @@ public sealed class EventStreamServiceTests
return events;
}
private static StreamEventsRequest CreateRequest(string sessionId, ulong afterWorkerSequence = 0)
private static StreamEventsRequest CreateRequest(
string sessionId,
ulong afterWorkerSequence = 0,
bool emitStreamAttached = false)
{
return new StreamEventsRequest
{
SessionId = sessionId,
AfterWorkerSequence = afterWorkerSequence,
EmitStreamAttached = emitStreamAttached,
};
}
@@ -489,6 +489,28 @@ public sealed class MxAccessGatewayServiceTests
Assert.Contains("server-stream-active-alarms", reply.Capabilities);
}
/// <summary>
/// OpenSession advertises the #227 attach-acknowledgement capability, so a client can tell
/// a gateway that honours <c>emit_stream_attached</c> from one that silently ignores it.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task OpenSession_AdvertisesStreamAttachedAckCapability()
{
FakeSessionManager sessionManager = new();
GatewayRequestIdentityAccessor identityAccessor = new();
MxAccessGatewayService service = CreateService(sessionManager, identityAccessor);
using IDisposable identityScope = identityAccessor.Push(CreateIdentity());
OpenSessionReply reply = await service.OpenSession(
new OpenSessionRequest(),
new TestServerCallContext());
Assert.Contains("stream-events-attached-ack", reply.Capabilities);
Assert.Contains(GatewayCapabilities.StreamEventsAttachedAck, reply.Capabilities);
}
/// <summary>
/// Every reply echoes the request's <c>client_correlation_id</c>, including when the caller
/// sent none: the internal worker envelope id never reaches the client.