Implements the background loop (Go: monitorCluster) that reads RaftLogEntry
items from a channel and dispatches assignStream, removeStream, assignConsumer,
removeConsumer, and snapshot operations to JetStreamMetaGroup.
- Add five public mutation helpers to JetStreamMetaGroup (AddStreamAssignment,
RemoveStreamAssignment, AddConsumerAssignment, RemoveConsumerAssignment,
ReplaceAllAssignments) used by the monitor path
- JetStreamClusterMonitor: async loop over ChannelReader<RaftLogEntry>, JSON
dispatch, ILogger injection with NullLogger default, malformed entries logged
and skipped rather than aborting the loop
- WaitForProcessedAsync uses Monitor.PulseAll for race-free test synchronisation
with no Task.Delay — satisfies slopwatch SW003/SW004 rules
- 10 new targeted tests all pass; 1101 cluster regression tests unchanged
Implements binary codec for meta-group snapshots: 2-byte little-endian version
header followed by S2-compressed JSON of the stream assignment map. Adds
[JsonObjectCreationHandling(Populate)] to StreamAssignment.Consumers so the
getter-only dictionary is populated in-place during deserialization. 8 tests
covering round-trip, compression ratio, field fidelity, multi-consumer restore,
version rejection, and truncation guard.
Go reference: jetstream_cluster.go:2075-2145 (encodeMetaSnapshot/decodeMetaSnapshot)
Adds BeginJointConsensus, CommitJointConsensus, and CalculateJointQuorum to
RaftNode per Raft paper Section 4. During a joint configuration transition
quorum requires majority from BOTH Cold and Cnew; CommitJointConsensus
finalizes Cnew as the sole active configuration. The existing single-phase
ProposeAddPeerAsync/ProposeRemovePeerAsync are unchanged. Includes 16 new
tests covering flag behaviour, quorum boundaries, idempotent commit, and
backward-compatibility with the existing membership API.
- SyncAsync: remove redundant FlushAsync, use single Flush(flushToDisk:true)
- ComputeCrc: use incremental Crc32.Append to avoid contiguous buffer heap allocation
- Load: cast pos+length to long to guard against int overflow in bounds check
- AppendAsync: delegate to WriteEntryTo (DRY — eliminates duplicated record-building logic)
- Load: extract ParseEntries static helper to eliminate goto pattern with early returns
- Entries: change return type from IEnumerable to IReadOnlyList for index access and Count property
- RaftNode.PersistAsync: remove redundant term.txt write (meta.json now owns term+votedFor)
- RaftWalTests: tighten ShouldBeGreaterThanOrEqualTo(1) -> ShouldBe(1) in truncation/CRC tests;
use .Count property directly on IReadOnlyList instead of .Count() LINQ extension
Implements a binary write-ahead log (RaftWal) for durable RAFT entry
storage, replacing in-memory-only semantics. The WAL uses a magic header
("NWAL" + version), length-prefixed records with per-record CRC32
integrity checking, and CompactAsync with atomic temp-file rename.
Load() tolerates truncated or corrupt tail records for crash safety.
Also fixes RaftNode to persist and reload TermState.VotedFor via a
meta.json file alongside term.txt, ensuring vote durability across
restarts. Falls back gracefully to legacy term.txt when meta.json is
absent.
6 new tests in RaftWalTests: persist/recover, compact, truncation
tolerance, VotedFor round-trip, empty WAL, and CRC corruption.
All 458 Raft tests pass.
Complete IStreamStore Batch 2 — all remaining interface methods now have
FileStore implementations:
- EncodedStreamState: placeholder returning empty (full codec in Task 9)
- UpdateConfig: no-op placeholder for config hot-reload
- ResetState: no-op (state derived from blocks on construction)
- Delete(bool): now calls Stop() first and matches interface signature
Complete IStreamStore Batch 1 — all core operations now have FileStore
implementations instead of throwing NotSupportedException:
- StoreRawMsg: caller-specified seq/ts for replication/mirroring
- LoadPrevMsg: backward scan for message before given sequence
- Type: returns StorageType.File
- Stop: flush + dispose blocks, reject further writes
14 new tests in FileStoreStreamStoreTests.
Add FlushAllPending() to FileStore — flushes active MsgBlock to disk and
writes a stream.state checkpoint atomically (write-to-temp + rename).
Go ref: filestore.go:5783 (flushPendingWritesUnlocked / writeFullState).
5 new tests in FileStoreCrashRecoveryTests: flush, state file, idempotent,
TTL recovery, truncated block handling.
Add FlushAllPending() to FileStore, fulfilling the IStreamStore interface
contract. The method flushes the active MsgBlock to disk and atomically
writes a stream.state checkpoint using write-to-temp + rename, matching
Go's flushPendingWritesUnlocked / writeFullState pattern.
Add FileStoreCrashRecoveryTests with 5 tests covering:
- FlushAllPending flushes block data to .blk file
- FlushAllPending writes a valid atomic stream.state JSON checkpoint
- FlushAllPending is idempotent (second call overwrites with latest state)
- Recovery prunes messages backdated past the MaxAgeMs cutoff
- Recovery handles a tail-truncated block without throwing
Reference: golang/nats-server/server/filestore.go:5783-5842
Code quality review feedback: Should.Throw<Exception> was too broad,
changed to Should.Throw<InvalidDataException> to match the actual
exception type thrown when AEAD decryption fails with wrong key.
Add FileStoreEncryptionTests covering ChaCha20-Poly1305 and AES-GCM
round-trips and wrong-key rejection for the FSV2 AEAD path. Fix
RestorePayload to wrap CryptographicException from AEAD decryption as
InvalidDataException so RecoverBlocks correctly propagates key-mismatch
failures instead of silently swallowing them.
Drop redundant columns now that test_mappings is the single source of truth:
- go_tests: drop dotnet_test, dotnet_file
- dotnet_tests: drop go_test
- VACUUM to reclaim space
Multi-strategy matching:
- Source comment references (// Go: TestXxx): +1,477 mappings
- In-body Go test name references: +249 mappings
- Name similarity with keyword overlap: +368 mappings
- File-context + lower threshold for parity files: +133 mappings
Final state:
- 4,250 / 5,808 dotnet tests mapped (73.2%), up from 2,485 (42.8%)
- 1,558 remaining are genuinely original .NET tests with no Go equivalent
- 59,516 total test_mappings rows
- Fix dotnet_tests scan to handle [Theory]+[InlineData] patterns (was
extracting "InlineData" as test name instead of actual method)
- Rebuild test_mappings with multi-strategy matching: exact path, basename,
class-level, wildcard prefix (ClassName.Prefix*), (N tests) patterns,
fuzzy word overlap, and 31 hand-mapped renames
- All 2,646 mapped Go tests now have entries in test_mappings (was 177 missing)
- 2,485 dotnet tests linked back to Go tests, 5,808 total
- 57,289 total mapping rows
Rewrote structuregaps.md with function-by-function gap analysis
comparing current .NET implementation against Go reference.
Added design doc for Tier 1+2 gap closure across 4 phases:
FileStore+RAFT → JetStream Cluster → Consumer/Stream → Client/MQTT/Config.
StreamManager.Capture now accounts for full message size (subject +
payload + 16-byte overhead) when checking MaxBytes, matching Go's
memStoreMsgSize. PullConsumerEngine uses stream FirstSeq instead of
hardcoded 1 for DeliverAll after purge. Fix 6 tests with Go parity
assertions and updated MaxBytes values.
Add long_running, real_network, test_project, and failing columns to
dotnet_tests table. Set test_project for all 3,491 rows. Mark 9 known
failing tests. Fix slopwatch hook to use absolute path.
Now that MemStore uses Unix epoch timestamps (13a3f81), restore the
original Go MaxAge values that were previously omitted as workarounds:
- JsCluster2: MaxAgeMs=500 (Go: 500ms)
- JsCluster34: MaxAgeMs=5000 (Go: 5s)
DateTime.UtcNow.Ticks * 100L overflows long (Ticks ~6.38e17, x100 =
6.38e19 > long.MaxValue 9.22e18), producing corrupted timestamps that
cause PruneExpiredMessages to immediately expire all messages when
MaxAgeMs is set. Use Unix epoch offset to match Go's time.Now().UnixNano().
Port 36 Go-parity tests covering redelivery, NAK, push consumer
heartbeats, interest retention with wildcards, ack-all with large
first seq, deliver last-per-subject, work-in-progress acks, flow
control stall, KV operations, multi-account isolation, and subject
transforms.
Go refs: TestJetStreamRedeliverAndLateAck, TestJetStreamInterestRetention,
TestJetStreamKVDelete, TestJetStreamMultipleAccountsBasics, and 23+ more.
Port Go client_test.go and server_test.go tests. Cover client connect,
pub/sub, protocol parsing, max payload validation, slow consumer
detection, TLS, auth timeout, server startup/shutdown, version parsing,
and write deadline enforcement.
42 new tests ported from client_test.go and server_test.go.
Port Go consumer_test.go and jetstream_test.go consumer lifecycle tests.
Cover pause/resume state machine, replay rate config, priority pull
requests, max delivery, ephemeral recovery, durable reconnect, and
consumer file store state persistence.
49 new tests ported from consumer_test.go and jetstream_test.go.
Port Go jetstream_test.go storage/recovery/encryption tests. Add stream
recovery stubs, encryption key management, direct-get with time queries,
and subject delete marker handling.
23 new tests ported from jetstream_test.go.
Port Go store_test.go contract tests for IStreamStore interface using
MemStore. Fix CompactInternal to correctly walk forward to find new
FirstSeq before backward cleanup (matching Go behavior).
21 new tests ported from store_test.go.
Port Go filestore tombstone/deletion tests, consumer state encode/decode,
consumer file store persistence, and message TTL enforcement. Adds
ConsumerStateCodec and ConsumerFileStore implementations.
17 new tests ported from filestore_test.go.
Ports 34 Go FileStore tests from filestore_test.go to
FileStoreRecovery2Tests.cs (31 pass, 4 skipped). Tests cover block
recovery, compaction, PSIM indexing, skip-msg handling, TTL expiry,
corrupt index/state detection, and read-only permission checks.
Updates docs/test_parity.db with mapped/skipped status for all 34 tests.
