Galaxy: can an MXAccess initial value arrive on the event stream before the driver registers the new item handle? #859

New Issue

2026-11-11T10:01:34-05:00

dohertj2 commented

2026-11-11 10:01:34 -05:00

Possible production gap, unverified — found while fixing #856. The event pump drops an event whose item handle is not (yet) in the subscription registry as "stale after unsubscribe". SubscribeAsync and PerPlatformProbeWatcher register a handle only after the gateway's subscribe reply returns. MXAccess delivers an initial value on subscribe; if mxaccessgw can put that first MxEvent on the stream before the subscribe reply reaches us, the initial value is dropped and the node stays at its pre-subscribe value/quality until the next change (a slow-changing or static attribute would stay stale). Check the gateway's ordering guarantee (subscribe reply vs first OnDataChange for that handle); if not guaranteed, buffer unknown-handle events briefly or register before the reply.

Possible production gap, unverified — found while fixing #856. The event pump drops an event whose item handle is not (yet) in the subscription registry as "stale after unsubscribe". `SubscribeAsync` and `PerPlatformProbeWatcher` register a handle only after the gateway's subscribe reply returns. MXAccess delivers an initial value on subscribe; if mxaccessgw can put that first `MxEvent` on the stream before the subscribe reply reaches us, the initial value is dropped and the node stays at its pre-subscribe value/quality until the next change (a slow-changing or static attribute would stay stale). Check the gateway's ordering guarantee (subscribe reply vs first OnDataChange for that handle); if not guaranteed, buffer unknown-handle events briefly or register before the reply.

dohertj2 added the type:followup priority:p3 cross-repo labels 2026-11-11 10:01:34 -05:00

dohertj2 referenced this issue

2026-11-11 10:27:03 -05:00

Galaxy: a SubscribeAsync still awaiting the old session's reply can register old-session handles after a reconnect #861

dohertj2 referenced this issue

2026-11-11 10:51:51 -05:00

Galaxy: with ReplayOnSessionLost=false a reconnect leaves every subscription silently dead (last Good values, health Healthy) #862

dohertj2 referenced this issue

2026-11-11 11:28:48 -05:00

Flaky: GalaxyDriverFixRound2Tests.A_fetch_completing_after_init_populates_host_statuses (10 s timeout under load) #856

dohertj2 commented

2026-11-11 11:28:50 -05:00

Fixed (1a31e3f5, f89e0cc0, d08e9002, 133bc07d, a8d3790b). Pushed to master in e857135e. The drop was real: mxaccessgw's worker completes the SubscribeBulk reply asynchronously while the STA loop immediately pumps OnDataChange, and the reply (unary) and events (stream) are separate HTTP/2 streams anyway — the first value could beat the reply and was dropped as "stale after unsubscribe" (subscriptions kept the old value until the next change; subscribe-once reads timed out). Fix: pending-subscribe windows park unknown-handle events (bounded, sequence-trimmed), promoted atomically on Register/Rebind and replayed in order on the dispatch thread; reads, subscribes, replay and probe watcher all open windows; UnsubscribeBulk purges parked events. Four independent concurrency reviews; each round's blockers fixed (ordering race, cross-session handle reuse, read cleanup unsubscribing a reused handle, probe replay skip). Also fixed while testing: a read waiting across a reconnect never saw its value (listened on the replaced pump). Residuals: a same-session reused handle can still receive an in-flight stale event (full fix needs a worker sequence on the SubscribeBulk reply in mxaccessgw — optional, documented); a subscribe landing between the two reopen-time session bumps can leak one gateway advise.

Fixed (`1a31e3f5`, `f89e0cc0`, `d08e9002`, `133bc07d`, `a8d3790b`). Pushed to master in `e857135e`. **The drop was real:** mxaccessgw's worker completes the SubscribeBulk reply asynchronously while the STA loop immediately pumps OnDataChange, and the reply (unary) and events (stream) are separate HTTP/2 streams anyway — the first value could beat the reply and was dropped as "stale after unsubscribe" (subscriptions kept the old value until the next change; subscribe-once reads timed out). Fix: pending-subscribe windows park unknown-handle events (bounded, sequence-trimmed), promoted atomically on Register/Rebind and replayed in order on the dispatch thread; reads, subscribes, replay and probe watcher all open windows; UnsubscribeBulk purges parked events. Four independent concurrency reviews; each round's blockers fixed (ordering race, cross-session handle reuse, read cleanup unsubscribing a reused handle, probe replay skip). Also fixed while testing: a read waiting across a reconnect never saw its value (listened on the replaced pump). Residuals: a same-session reused handle can still receive an in-flight stale event (full fix needs a worker sequence on the SubscribeBulk reply in mxaccessgw — optional, documented); a subscribe landing between the two reopen-time session bumps can leak one gateway advise.

dohertj2 closed this issue

2026-11-11 11:28:50 -05:00

dohertj2 referenced this issue

2026-11-11 11:28:50 -05:00

Galaxy: a SubscribeAsync still awaiting the old session's reply can register old-session handles after a reconnect #861

dohertj2 referenced this issue from a commit

2026-11-11 11:28:50 -05:00

fix(galaxy): park data-change events that overtake their SubscribeBulk reply (#859)

dohertj2 referenced this issue from a commit

2026-11-11 11:28:50 -05:00

fix(galaxy): #859 review — read path window, atomic promote, session epoch, per-call windows

dohertj2 referenced this issue from a commit