Replace eager Dictionary<ulong, StoredMessage> with lightweight
Dictionary<ulong, MessageMeta> to eliminate ~200B StoredMessage
allocation per message on the write path.
- Add MessageMeta struct (BlockId, Subject, PayloadLength, HeaderLength,
TimestampNs) — ~40B vs ~200B for StoredMessage
- Add MaterializeMessage(seq) for on-demand reconstruction from blocks
- Update all ~60 _messages references to use _meta
- Methods needing full payload (LoadAsync, ListAsync, etc.) call
MaterializeMessage; metadata-only paths use _meta directly
- Fix MsgBlock.WriteAt to clear stale delete markers on re-write
- Add cached state properties (LastSeq, MessageCount, TotalBytes, FirstSeq)
to IStreamStore/FileStore/MemStore — eliminates GetStateAsync on publish path
- Add Capture(StreamHandle, ...) overload to StreamManager — eliminates
double FindBySubject lookup (once in JetStreamPublisher, once in Capture)
- Remove _messageIndexes dictionary from FileStore write path — all lookups
now use _messages directly, saving ~48B allocation per message
- Add JetStreamPubAckFormatter for hand-rolled UTF-8 success ack formatting —
avoids JsonSerializer overhead on the hot publish path
- Switch flush loop to exponential backoff (1→2→4→8ms) matching Go server
Pub/sub 1:1 (16B) improved from 0.18x to 0.50x, fan-out from 0.18x to 0.44x,
and JetStream durable fetch from 0.13x to 0.64x vs Go. Key changes: replace
.ToArray() copy in SendMessage with pooled buffer handoff, batch multiple small
writes into single WriteAsync via 64KB coalesce buffer in write loop, and remove
profiling Stopwatch instrumentation from ProcessMessage/StreamManager hot paths.
Implement Go-parity background flush loop (coalesce 16KB/8ms) in MsgBlock/FileStore,
replace O(n) GetStateAsync with incremental counters, skip PruneExpired/LoadAsync/
PrunePerSubject when not needed, and bypass RAFT for single-replica streams. Fix counter
tracking bugs in RemoveMsg/EraseMsg/TTL expiry and ObjectDisposedException races in
flush loop disposal. FileStore optimizations verified with 3112/3112 JetStream tests
passing; async publish benchmark remains at ~174 msg/s due to E2E protocol path bottleneck.
Replace HashSet<ulong> _deleted in MsgBlock with SequenceSet — a sorted-range
list that compresses contiguous deletions into (Start, End) intervals. Adds
O(log n) Contains/Add via binary search on range count, matching Go's avl.SequenceSet
semantics with a simpler implementation.
- Add SequenceSet.cs: sorted-range compressed set with Add/Remove/Contains/Count/Clear
and IEnumerable<ulong> in ascending order. Binary search for all O(log n) ops.
- Replace HashSet<ulong> _deleted and _skipSequences in MsgBlock with SequenceSet.
- Add secureErase parameter (default false) to MsgBlock.Delete(): when true, payload
bytes are overwritten with RandomNumberGenerator.Fill() before the delete record is
written, making original content unrecoverable on disk.
- Update FileStore.DeleteInBlock() to propagate secureErase flag.
- Update FileStore.EraseMsg() to use secureErase: true via block layer instead of
delegating to RemoveMsg().
- Add SequenceSetTests.cs: 25 tests covering Add, Remove, Contains, Count, range
compression, gap filling, bridge merges, enumeration, boundary values, round-trip.
- Add FileStoreTombstoneTrackingTests.cs: 12 tests covering SequenceSet tracking in
MsgBlock, tombstone persistence through RebuildIndex recovery, secure erase
payload overwrite verification, and FileStore.EraseMsg integration.
Go reference: filestore.go:5267 (removeMsg), filestore.go:5890 (eraseMsg),
avl/seqset.go (SequenceSet).
- Add AtomicFileWriter static helper: writes to {path}.{random}.tmp, flushes,
then File.Move(overwrite:true) — concurrent-safe via unique temp path per call
- Add _stateWriteLock (SemaphoreSlim 1,1) to FileStore; dispose in both Dispose
and DisposeAsync paths
- Promote WriteStreamState to async WriteStreamStateAsync using AtomicFileWriter
under the write lock; FlushAllPending now returns Task
- Update IStreamStore.FlushAllPending signature to Task; fix MemStore no-op impl
- Fix FileStoreCrashRecoveryTests to await FlushAllPending (3 sync→async tests)
- Add 9 AtomicFileWriterTests covering create, no-tmp-remains, overwrite,
concurrent safety, memory overload, empty data, and large payload
Complete IStreamStore Batch 2 — all remaining interface methods now have
FileStore implementations:
- EncodedStreamState: placeholder returning empty (full codec in Task 9)
- UpdateConfig: no-op placeholder for config hot-reload
- ResetState: no-op (state derived from blocks on construction)
- Delete(bool): now calls Stop() first and matches interface signature
Complete IStreamStore Batch 1 — all core operations now have FileStore
implementations instead of throwing NotSupportedException:
- StoreRawMsg: caller-specified seq/ts for replication/mirroring
- LoadPrevMsg: backward scan for message before given sequence
- Type: returns StorageType.File
- Stop: flush + dispose blocks, reject further writes
14 new tests in FileStoreStreamStoreTests.
Add FlushAllPending() to FileStore — flushes active MsgBlock to disk and
writes a stream.state checkpoint atomically (write-to-temp + rename).
Go ref: filestore.go:5783 (flushPendingWritesUnlocked / writeFullState).
5 new tests in FileStoreCrashRecoveryTests: flush, state file, idempotent,
TTL recovery, truncated block handling.
Add FlushAllPending() to FileStore, fulfilling the IStreamStore interface
contract. The method flushes the active MsgBlock to disk and atomically
writes a stream.state checkpoint using write-to-temp + rename, matching
Go's flushPendingWritesUnlocked / writeFullState pattern.
Add FileStoreCrashRecoveryTests with 5 tests covering:
- FlushAllPending flushes block data to .blk file
- FlushAllPending writes a valid atomic stream.state JSON checkpoint
- FlushAllPending is idempotent (second call overwrites with latest state)
- Recovery prunes messages backdated past the MaxAgeMs cutoff
- Recovery handles a tail-truncated block without throwing
Reference: golang/nats-server/server/filestore.go:5783-5842
Add FileStoreEncryptionTests covering ChaCha20-Poly1305 and AES-GCM
round-trips and wrong-key rejection for the FSV2 AEAD path. Fix
RestorePayload to wrap CryptographicException from AEAD decryption as
InvalidDataException so RecoverBlocks correctly propagates key-mismatch
failures instead of silently swallowing them.
Port Go filestore tombstone/deletion tests, consumer state encode/decode,
consumer file store persistence, and message TTL enforcement. Adds
ConsumerStateCodec and ConsumerFileStore implementations.
17 new tests ported from filestore_test.go.
Add MsgBlock write cache (mirrors Go's msgBlock.cache) to serve reads
for recently-written records without disk I/O; cleared on block seal via
RotateBlock. Add HashWheel-based TTL expiry in FileStore (ExpireFromWheel /
RegisterTtl), replacing the O(n) linear scan on every append with an
O(expired) wheel scan. Implement StoreMsg sync method with per-message TTL
override support. Add 10 tests covering cache hits/eviction, wheel expiry,
retention, StoreMsg seq/ts, per-msg TTL, and recovery re-registration.
Implement PurgeEx, Compact, Truncate, FilteredState, SubjectsState,
SubjectsTotals, State, FastState, GetSeqFromTime on FileStore. Add
MsgBlock.IsDeleted, DeletedSequences, EnumerateNonDeleted. Includes
wildcard subject support via SubjectMatch for all filtered operations.
Replace JSONL persistence with real MsgBlock-based block files (.blk).
FileStore now acts as a block manager that creates, seals, and rotates
MsgBlocks while maintaining an in-memory cache for fast reads/queries.
Key changes:
- AppendAsync writes transformed payloads to MsgBlock via WriteAt
- Block rotation occurs when active block reaches size limit
- Recovery scans .blk files and rebuilds in-memory state from records
- Legacy JSONL migration: existing messages.jsonl data is automatically
converted to block files on first open, then JSONL is deleted
- PurgeAsync disposes and deletes all block files
- RewriteBlocks rebuilds blocks from cache (used by trim/restore)
- InvalidDataException propagates during recovery (wrong encryption key)
MsgBlock.WriteAt added to support explicit sequence numbers and timestamps,
needed when rewriting blocks with non-contiguous sequences (after removes).
Tests updated:
- New FileStoreBlockTests.cs with 9 tests for block-specific behavior
- JetStreamFileStoreCompressionEncryptionParityTests updated to read
FSV1 magic from .blk files instead of messages.jsonl
- JetStreamFileStoreDurabilityParityTests updated to verify .blk files
instead of index.manifest.json
All 3,562 tests pass (3,535 passed + 27 skipped, 0 failures).
