fix(comm): stop the duplicate-stream replacement disposing another handler's CTS

Root-caused from a full-suite-load-only failure of
GrpcStreamIntegrationTests.Pipeline_DuplicateCorrelationId_ReplacesStream:
ObjectDisposedException escaping SubscribeInstance.

SubscribeInstance's duplicate-prevention path cancelled AND disposed the
replaced stream's CancellationTokenSource. That CTS belongs to the replaced
handler's own `using var streamCts`, which is still running and still has to
read `streamCts.Token` — the Dispose raced that read. The race is pre-existing
and independent of R2: the same Dispose sat against the same first-token-read
when the handler still used `ReadAllAsync(streamCts.Token)`; it only ever
loses under enough scheduling pressure to land the replacement inside the
first stream's setup window.

Cancel only. The owning handler's `using` still disposes it exactly once on
every exit path, and cancellation is all replacement ever needed. The Cancel
is wrapped for the converse race (owner already finished and disposed),
mirroring CancelAllStreams().

Regression test makes the race deterministic by gating the first stream inside
its setup window — the _activeStreams entry is registered before Subscribe is
called, so the replacement always lands before the first stream reads its
token. Verified fail-before (ObjectDisposedException) / pass-after by
reinstating the Dispose as a negative control.
This commit is contained in:
Joseph Doherty
2026-08-15 03:52:23 -04:00
parent 9b5cb3dd9d
commit e6842c108a
2 changed files with 82 additions and 3 deletions
@@ -353,11 +353,29 @@ public class SiteStreamGrpcServer : SiteStreamService.SiteStreamServiceBase
StatusCode.InvalidArgument, "correlation_id is missing or not a valid identifier"));
}
// Duplicate prevention -- cancel existing stream for this correlationId
// Duplicate prevention -- cancel existing stream for this correlationId.
//
// CANCEL ONLY, never Dispose. The replaced stream's CTS belongs to its own
// handler's `using var streamCts`, which is still running and still reads
// `streamCts.Token` (at the pump call below, and previously at the
// `ReadAllAsync(streamCts.Token)` it replaced). Disposing it from here raced that
// read and surfaced as an unhandled ObjectDisposedException escaping the RPC —
// observed as a full-suite-load-only failure of
// GrpcStreamIntegrationTests.Pipeline_DuplicateCorrelationId_ReplacesStream, and a
// pre-existing hazard (the same Dispose + the same first-token-read relationship
// exist unchanged before R2). Cancellation alone is what replacement needs; the
// owning handler's `using` still disposes it exactly once on every exit path.
if (_activeStreams.TryRemove(correlationId, out var existingEntry))
{
existingEntry.Cts.Cancel();
existingEntry.Cts.Dispose();
try
{
existingEntry.Cts.Cancel();
}
catch (ObjectDisposedException)
{
// Its owner finished and disposed it between the TryRemove and here —
// already terminal, nothing to cancel. Mirrors CancelAllStreams().
}
}
// Check max concurrent streams after duplicate removal.