dohertj2/natsdotnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

perf: add FileStore buffered writes, O(1) state tracking, and eliminate redundant per-publish work

Browse Source 
Implement Go-parity background flush loop (coalesce 16KB/8ms) in MsgBlock/FileStore,
replace O(n) GetStateAsync with incremental counters, skip PruneExpired/LoadAsync/
PrunePerSubject when not needed, and bypass RAFT for single-replica streams. Fix counter
tracking bugs in RemoveMsg/EraseMsg/TTL expiry and ObjectDisposedException races in
flush loop disposal. FileStore optimizations verified with 3112/3112 JetStream tests
passing; async publish benchmark remains at ~174 msg/s due to E2E protocol path bottleneck.
This commit is contained in:
Joseph Doherty
2026-03-13 03:11:11 -04:00
parent 37575dc41c
commit 4de691c9c5
30 changed files with 1514 additions and 185 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
tests/NATS.Server.JetStream.Tests/JetStream/Cluster/JsSuperClusterTests.cs
View File

@@ -407,9 +407,11 @@ public class JsSuperClusterTests
// Stream info returns 3 alternates, sorted by proximity.
await using var cluster = await JetStreamClusterFixture.StartAsync(9);
// In Go, mirrors live in separate clusters (separate jsAccounts) so subjects can overlap.
// Our fixture uses a single StreamManager, so we use distinct subjects per stream.
await cluster.CreateStreamAsync("SOURCE", ["foo", "bar", "baz"], replicas: 3);
await cluster.CreateStreamAsync("MIRROR-1", ["foo", "bar", "baz"], replicas: 1);
await cluster.CreateStreamAsync("MIRROR-2", ["foo", "bar", "baz"], replicas: 2);
await cluster.CreateStreamAsync("MIRROR-1", ["m1foo", "m1bar", "m1baz"], replicas: 1);
await cluster.CreateStreamAsync("MIRROR-2", ["m2foo", "m2bar", "m2baz"], replicas: 2);
// All three streams should exist and be accessible.
var src = await cluster.GetStreamInfoAsync("SOURCE");
@@ -715,7 +717,9 @@ public class JsSuperClusterTests
});
source.Error.ShouldBeNull();
var mirror = await cluster.CreateStreamAsync("MIRROR_AD", ["src.>"], replicas: 1);
// In Go, mirror lives in a separate cluster so subjects can overlap.
// Our fixture uses a single StreamManager, so we use distinct subjects.
var mirror = await cluster.CreateStreamAsync("MIRROR_AD", ["msrc.>"], replicas: 1);
mirror.Error.ShouldBeNull();
// Both source and mirror exist and are accessible.