fix(galaxy): #865 review round 5 - a failure during the replay fails the attempt

ReconnectSupervisor swallowed a transport failure reported while Replaying,
so a restarted event stream refused/ended a round trip after the replay
began still let the replay go Healthy over a dead stream (and the next
advise's re-report restarted at attempt 1 with no backoff). A failure
reported during Replaying is now remembered and turns that attempt's
success into a retry with backoff. ReplayAsync's synchronous dead-pump
throw yields to a teardown's cancellation.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
This commit is contained in:
Joseph Doherty
2026-11-11 13:12:33 -05:00
parent 8653960ea3
commit d80bdf6369
3 changed files with 61 additions and 5 deletions
@@ -49,6 +49,35 @@ public sealed class ReconnectSupervisorTests
sup.CurrentState.ShouldBe(ReconnectSupervisor.State.Reopening, "the backoff still reads Reopening — why the flag exists");
}
/// <summary>
/// #865 review: a transport failure reported WHILE the replay runs (e.g. the restarted event stream is
/// refused a round trip later) must fail that attempt — not go Healthy over a dead transport. The
/// supervisor retries (with backoff) and goes Healthy only after a clean replay. Without the fix the
/// first replay's success went Healthy and replay ran once.
/// </summary>
[Fact]
public async Task A_failure_reported_during_the_replay_fails_that_attempt()
{
ReconnectSupervisor? self = null;
var replays = 0;
using var sup = new ReconnectSupervisor(
reopen: _ => Task.CompletedTask,
replay: _ =>
{
if (Interlocked.Increment(ref replays) == 1)
{
self!.ReportTransportFailure(new IOException("stream refused during replay"));
}
return Task.CompletedTask;
},
options: FastBackoff());
self = sup;
sup.ReportTransportFailure(new IOException("drop"));
await TestWait.HealthyAsync(sup, "the second, clean replay must recover");
Volatile.Read(ref replays).ShouldBe(2, "the faulted replay must not count as recovered");
}
/// <summary>#865: a transport failure reported after StopAsync never starts a recovery loop (it throws ODE).</summary>
[Fact]
public async Task ReportTransportFailure_after_stop_never_starts_a_loop()