Commit Graph

6 Commits

Author SHA1 Message Date
Joseph Doherty 59d33ab938 test(galaxy): #865 review round 7 - pin the superseded-pump fault filter
Adds an escalated-stream-fault test seam and a test: the live pump's fault
escalates (positive controls before and after), a superseded pump's late
fault after a reopen does not; the absence is asserted only once both
pumps' callbacks are known to have run (HasEnded). Verified to fail with
the filter disabled. The filter's comment now says the identity check does
the work and warns about a future session bump while Healthy.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:20:49 -05:00
Joseph Doherty 8653960ea3 fix(galaxy): #865 review round 4 - replay fails on a dead stream, backoff-aware wait
- ReplayAsync fails when its restarted pump's stream has already ended, so
  the supervisor backs off and stays Degraded instead of going Healthy over
  a dead stream (whose next advise re-report restarted at attempt 1 with no
  backoff - a tight reconnect loop).
- ReconnectSupervisor.AttemptInProgress (reopen through replay); advises
  wait for a stale pump's replacement only while an attempt is running, not
  for a whole backoff (the state still reads Reopening/Replaying there).
- ReconnectSupervisor: _disposed is set under the state lock and re-checked
  inside ReportTransportFailure, so a late report cannot start a recovery
  loop that StopAsync already missed.
- EventPump.HasEnded is a flag set before StreamRequested is released, and
  a stream that ends/faults on its first MoveNext is marked ended first.
- SubscriptionRegistry fast path: a miss now falls through to the locked
  path (lookup + session + window check atomically) instead of a lock-free
  re-lookup.
- Tests: replay over an immediately-ending stream fails; AttemptInProgress
  across attempt/backoff; ReportTransportFailure after stop; the
  reopen-window test uses a generous wait bound.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:07:26 -05:00
Joseph Doherty 1c1705fa77 fix(galaxy): #865 review round 3 - budget-safe stream wait, dead pump, fast-path drop
- StreamRequestWait 1 s (under Tier-A Read 2 s / Subscribe 5 s budgets) and
  a stale pump is only waited on while the supervisor is reopening or
  replaying (or a restart is underway); in a reconnect backoff the advise
  proceeds at once. The no-stream log is a rate-limited Warning while
  Healthy, Debug otherwise.
- A current-session pump whose loop has ended is not a stream: if the
  supervisor already reports Healthy (its fault was swallowed while it was
  recovering) the failure is re-reported to start a new cycle.
- Subscribe/Read no longer start the pump directly (only
  AwaitCurrentEventStreamAsync does, honouring the replay restart flag);
  the teardown refusal is an explicit ThrowIfRuntimeStopping.
- SubscriptionRegistry.ResolveOrPark fast path: the handle lookup and the
  open-window check were not atomic, so a subscribe that registered the
  handle and closed its window in between got its initial value dropped as
  stale (found by a stress run: the first event of a bulk advise lost). It
  re-looks-up before declaring stale. Deterministic test via a seam.
- Tests: first Read on a fresh driver, replay of a data subscription.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 13:00:19 -05:00
Joseph Doherty 9060ff5561 fix(galaxy): #865 review round 2 - wait for the replay's pump, never race a second stream
Round 1 replaced a stale-session pump from any advise path and disposed the
old one in the background; that could start a second stream while the old
one still held the gateway's single-subscriber slot, and a kept
current-session pump could be dead with no fault re-reported.

- EnsureEventPumpStarted reuses again; only ReplayAsync restarts the pump.
  RestartEventPumpForReplay keeps a current-session pump only while its loop
  is running (EventPump.HasEnded), and flags the restart so no advise path
  starts a competing pump in the dispose gap.
- AwaitCurrentEventStreamAsync (bounded by StreamRequestWait) starts a pump
  only when none exists, WAITS for the replay while the pump is stale or
  restarting, waits for StreamRequested, and returns the session it verified.
  SubscribeAsync (per #861 attempt), read-via-subscribe and the probe hook
  issue and fence the advise under that session, so a reconnect after the
  check is caught as a straddle and retried.
- Read: a deadline (not caller) cancel during the wait falls through to
  per-ref results.
- Tests: reopen-window probe test now asserts the advise waits for the
  replay's single new stream; first Subscribe/Read wait for the stream; the
  fencing tests that advise in the window use a short wait bound; the
  production-path seed test uses an empty galaxy (a real probe advise now
  starts a stream against the fake endpoint).

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:46:15 -05:00
Joseph Doherty cc1a7ec687 fix(galaxy): #865 review round 1 - stale-session pump, data-path stream wait
- EnsureEventPumpStarted replaces a pump created for an older registry
  session (the reopen -> replay window still held the dead session's pump,
  so an advise there lost its initial value and the replay skipped it);
  RestartEventPumpForReplay keeps a pump already on the current session.
- SubscribeAsync, read-via-subscribe and ReplayAsync also await the
  bounded StreamRequested wait before advising, not just the probes.
- TryEnsureEventPumpStarted tolerates a teardown flipping _runtimeStopping
  mid-check; the probe hook honours an already-cancelled seed token.
- StreamRequested is documented as narrowing, not closing, the attach race.
- Tests: reopen-window probe test, EventPump.StreamRequested tests, and the
  FixRound1 fresh-pump assertion now waits on a positive control.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:31:51 -05:00
Joseph Doherty e35f2a4e93 fix(galaxy): probe advises start the event stream first so initial ScanState arrives (#865)
The gateway's StreamEvents with after_worker_sequence = 0 is a fresh stream
and never replays, and the event pump started only on the first
Read/Subscribe, so platform ScanState probes advised before then lost their
initial value (and a driver with no authored tags never got any host status).

PerPlatformProbeWatcher now awaits an ensure-event-stream hook before every
probe SubscribeBulk (seed, discover sync, reconnect ResubscribeStaleAsync).
The driver's hook starts the pump (refused while a teardown is in progress)
and waits, bounded, on the new EventPump.StreamRequested signal - completed
once the stream loop's first MoveNext has issued the StreamEvents RPC, or
when the loop ends - so the advise never precedes the stream request.

Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
2026-11-11 12:24:08 -05:00