perf: add FileStore buffered writes, O(1) state tracking, and eliminate redundant per-publish work

Implement Go-parity background flush loop (coalesce 16KB/8ms) in MsgBlock/FileStore,
replace O(n) GetStateAsync with incremental counters, skip PruneExpired/LoadAsync/
PrunePerSubject when not needed, and bypass RAFT for single-replica streams. Fix counter
tracking bugs in RemoveMsg/EraseMsg/TTL expiry and ObjectDisposedException races in
flush loop disposal. FileStore optimizations verified with 3112/3112 JetStream tests
passing; async publish benchmark remains at ~174 msg/s due to E2E protocol path bottleneck.
This commit is contained in:
Joseph Doherty
2026-03-13 03:11:11 -04:00
parent 37575dc41c
commit 4de691c9c5
30 changed files with 1514 additions and 185 deletions

View File

@@ -122,11 +122,13 @@ public sealed class JetStreamClusterFixture : IAsyncDisposable
/// </summary>
public JetStreamApiResponse UpdateStream(string name, string[] subjects, int replicas, int maxMsgs = 0)
{
// Go: stream update rejects unknown stream names — must exist before update.
if (!_streamManager.TryGet(name, out var existing))
return JetStreamApiResponse.ErrorResponse(404, "stream not found");
// Preserve the existing stream's retention policy so ValidateConfigUpdate
// does not reject the update for changing an immutable field.
var retention = RetentionPolicy.Limits;
if (_streamManager.TryGet(name, out var existing))
retention = existing.Config.Retention;
var retention = existing.Config.Retention;
return _streamManager.CreateOrUpdate(new StreamConfig
{