dohertj2/natsdotnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

perf: Phase 2 lazy StoredMessage materialization in FileStore

Browse Source 
Replace eager Dictionary<ulong, StoredMessage> with lightweight
Dictionary<ulong, MessageMeta> to eliminate ~200B StoredMessage
allocation per message on the write path.

- Add MessageMeta struct (BlockId, Subject, PayloadLength, HeaderLength,
  TimestampNs) — ~40B vs ~200B for StoredMessage
- Add MaterializeMessage(seq) for on-demand reconstruction from blocks
- Update all ~60 _messages references to use _meta
- Methods needing full payload (LoadAsync, ListAsync, etc.) call
  MaterializeMessage; metadata-only paths use _meta directly
- Fix MsgBlock.WriteAt to clear stale delete markers on re-write
This commit is contained in:
Joseph Doherty
2026-03-13 15:33:38 -04:00
parent 7404ecdb0e
commit 6e91fda7fd
2 changed files with 365 additions and 237 deletions
Download Patch File Download Diff File Expand all files Collapse all files

598
src/NATS.Server/JetStream/Storage/FileStore.cs
View File

File diff suppressed because it is too large Load Diff

4
src/NATS.Server/JetStream/Storage/MsgBlock.cs
View File

@@ -315,6 +315,10 @@ public sealed class MsgBlock : IDisposable
_index[sequence] = (offset, written); _index[sequence] = (offset, written);
// If this sequence was previously soft-deleted, clear the deletion marker
// so that subsequent Read calls return the new record rather than null.
_deleted.Remove(sequence);
// Go: cache populated lazily on read, not eagerly on write. // Go: cache populated lazily on read, not eagerly on write.
// Reads that miss _cache flush pending buf to disk and decode from there. // Reads that miss _cache flush pending buf to disk and decode from there.