fix(gateway): bound the session replay ring by bytes as well as count (mxaccessgw#172)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m26s
ci / java (push) Successful in 2m28s
ci / portable (push) Failing after 4m48s

The replay ring was bounded only by ReplayBufferCapacity (1024 events) and
ReplayRetentionSeconds. Event size is unbounded up to the worker frame size,
so a burst of large-array events could pin gigabytes per session while the
count looked small.

New MxGateway:Events:ReplayBufferMaxBytes (default 64 MiB, 0 disables) caps
the summed serialized size (MxEvent.CalculateSize, measured only when the cap
is on). Over the cap the oldest entries are evicted, exactly like the count
cap, so a resuming subscriber sees the existing ReplayGap path; an event bigger
than the whole cap is not retained. Every eviction (count, age or byte) now
clears its slot, so an evicted event is released immediately instead of
staying referenced until the slot is reused.

The 64 MiB default is a judgement call: it never binds for scalar traffic
(1024 x a few hundred bytes) and only trims large-array bursts.

Tests: SessionEventDistributorTests byte-cap eviction, single oversized event,
cap 0 = count only (the first two fail with the cap enforcement removed);
validator rejects a negative value. Docs: GatewayConfiguration.md, Sessions.md,
DesignDecisions.md; appsettings.json carries the key.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 12:38:22 -05:00
parent 7b21f304e9
commit 07badd139e
12 changed files with 186 additions and 17 deletions
+4 -2
View File
@@ -48,6 +48,7 @@ paths, timeouts, queue sizes, enum values, or protocol values are invalid.
"BackpressurePolicy": "FailFast",
"ReplayBufferCapacity": 1024,
"ReplayRetentionSeconds": 300,
"ReplayBufferMaxBytes": 67108864,
"MaxSparseArrayLength": 1000000
},
"Dashboard": {
@@ -196,6 +197,7 @@ All numeric session options must be greater than zero.
| `MxGateway:Events:BackpressurePolicy` | `FailFast` | Per-subscriber event backpressure behavior when a subscriber's bounded event channel overflows. Overflow is isolated to the offending subscriber: it is always disconnected with an `EventQueueOverflow` fault while the session pump and other subscribers keep running. `FailFast` additionally faults the whole session only in the legacy single-subscriber case (the current default mode); with multiple subscribers it degrades to a per-subscriber disconnect so one slow consumer never faults a shared session. `DisconnectSubscriber` disconnects only the slow subscriber in all cases. |
| `MxGateway:Events:ReplayBufferCapacity` | `1024` | Maximum number of events retained per session in the replay ring buffer, used to re-deliver events a returning subscriber missed (reconnect/reattach). The oldest retained event is evicted once this count is exceeded. `0` disables replay retention. |
| `MxGateway:Events:ReplayRetentionSeconds` | `300` | Maximum age, in seconds, of an event retained in the replay ring buffer. Entries older than this are evicted regardless of capacity. `0` disables age-based eviction. |
| `MxGateway:Events:ReplayBufferMaxBytes` | `67108864` (64 MiB) | Maximum summed serialized size, in bytes, of the events retained per session in the replay ring buffer. The count cap alone does not bound memory: one large-array event can approach the worker frame size, so 1024 of them could pin gigabytes per session. When the retained total exceeds this cap the oldest events are evicted first, and an event larger than the whole cap is not retained at all; a resuming subscriber then sees the ordinary `ReplayGap`. The default never binds for scalar traffic and only trims bursts of large events. `0` disables the byte cap. |
| `MxGateway:Events:MaxSparseArrayLength` | `1000000` | Maximum `total_length` a sparse-array write (`MxSparseArray`) may declare. A write above this cap is rejected with `InvalidArgument` before the full array is materialized, guarding against a single write forcing a multi-GB allocation. Must be between `1` and `Array.MaxLength`. |
`QueueCapacity` must be greater than zero and no greater than `int.MaxValue / 2`
@@ -213,8 +215,8 @@ subscribers `FailFast` degrades to a per-subscriber disconnect, matching
healthy subscribers. With `DisconnectSubscriber`, overflow terminates only the
affected stream while the MXAccess session remains active.
`ReplayBufferCapacity` and `ReplayRetentionSeconds` must each be greater than or
equal to zero (either dimension can be disabled with `0`). A returning subscriber
`ReplayBufferCapacity`, `ReplayRetentionSeconds` and `ReplayBufferMaxBytes` must
each be greater than or equal to zero (any dimension can be disabled with `0`). A returning subscriber
that asks for events older than the oldest still-retained event is told it missed
events (a "gap") and must re-snapshot; whatever is still retained is replayed.