dohertj2/natsdotnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
307 Commits 2 Branches 0 Tags
17731e2af57179e252a344ead944f0ccb09c8112
Commit Graph

202 Commits

Author SHA1 Message Date
Joseph Doherty
17731e2af5 feat(storage): add binary message record encoding (Go parity) 
Add MessageRecord class with Encode/Decode for the binary wire format
used in JetStream file store blocks. Uses varint-encoded lengths,
XxHash64 checksums, and a flags byte for deletion markers.

Go reference: filestore.go:6720-6724, 8180-8250, 8770-8777

13 tests covering round-trip, headers, checksum validation, corruption
detection, varint encoding, deleted flag, empty/large payloads.
 2026-02-24 12:12:15 -05:00
Joseph Doherty
3e972f217e feat: add FileStore 6-way permutation tests (Go testFileStoreAllPermutations parity) 
Port 20 tests across the {NoCipher, ChaCha, Aes} x {NoCompression, S2Compression}
matrix from Go's testFileStoreAllPermutations (filestore_test.go:55), yielding 120
total Theory executions in FileStorePermutationTests.cs.

Also fix PushFrame.Subject property missing compilation error that was blocking
the test build (JetStreamConsumerDeliveryEdgeTests.cs:119).

Tests covered (each runs 6x):
  Store_and_load_basic                       TestFileStoreBasics:86
  Store_multiple_messages_load_by_sequence   TestFileStoreBasics:86
  LoadLastBySubject_returns_most_recent_for_subject
  Remove_single_message_updates_state        TestFileStoreBasics:129
  Purge_clears_all_messages                  TestFileStorePurge:710
  TrimToMaxMessages_enforces_limit           TestFileStoreMsgLimitBug:518
  Block_rotation_when_exceeding_block_size   TestFileStoreAndRetrieveMultiBlock:1527
  GetState_returns_correct_counts            TestFileStoreBasics:104
  Snapshot_and_restore_round_trip            TestFileStoreSnapshot:1799
  ListAsync_returns_ordered_messages         TestFileStoreTimeStamps:683
  MaxAge_prunes_expired_messages             TestFileStoreAgeLimit:616
  Recovery_after_reopen_preserves_messages   TestFileStoreBasicWriteMsgsAndRestore:181
  Large_payload_store_and_load               64 KiB random payload variant
  Multiple_subjects_filter_by_subject        TestFileStoreBasics multi-subject
  Sequential_writes_maintain_ordering        TestFileStoreSelectNextFirst:304
  Store_creates_files_on_disk                disk-presence variant
  Write_and_read_same_block                  TestFileStoreWriteAndReadSameBlock:1510
  Stored_messages_have_non_decreasing_timestamps TestFileStoreTimeStamps:683
  Remove_out_of_order_collapses_first_seq    TestFileStoreCollapseDmap:1561
  Snapshot_after_removes_preserves_remaining TestFileStoreSnapshot:1904
 2026-02-24 06:43:48 -05:00
Joseph Doherty
6bcd682b76 feat: add NatsRaftTransport with NATS subject routing ($NRG.*) 
Implements RaftSubjects static class with Go's $NRG.* subject constants
and NatsRaftTransport which routes RAFT RPCs over those subjects using
RaftAppendEntryWire / RaftVoteRequestWire encoding. 43 tests cover all
subject patterns, wire encoding fidelity, and transport construction.
 2026-02-24 06:40:41 -05:00
Joseph Doherty
9cc9888bb4 feat: add S2 compression and AEAD encryption for FileStore (Go parity) 
Replace Deflate+XOR with IronSnappy S2 block compression and ChaCha20-Poly1305 / AES-256-GCM
AEAD encryption, matching golang/nats-server/server/filestore.go. Introduces FSV2 envelope
format alongside existing FSV1 for backward compatibility. Adds 55 new tests across
S2CodecTests, AeadEncryptorTests, and FileStoreV2Tests covering all 6 cipher×compression
permutations, tamper detection, and legacy format round-trips.
 2026-02-24 06:29:34 -05:00
Joseph Doherty
2c9683e7aa feat: upgrade JetStreamService to lifecycle orchestrator 
Implements enableJetStream() semantics from golang/nats-server/server/jetstream.go:414-523.

- JetStreamService.StartAsync(): validates config, creates store directory
  (including nested paths via Directory.CreateDirectory), registers all
  $JS.API.> subjects, logs startup stats; idempotent on double-start
- JetStreamService.DisposeAsync(): clears registered subjects, marks not running
- New properties: RegisteredApiSubjects, MaxStreams, MaxConsumers, MaxMemory, MaxStore
- JetStreamOptions: adds MaxStreams and MaxConsumers limits (0 = unlimited)
- FileStoreConfig: removes duplicate StoreCipher/StoreCompression enum declarations
  now that AeadEncryptor.cs owns them; updates defaults to NoCipher/NoCompression
- FileStoreOptions/FileStore: align enum member names with AeadEncryptor.cs
  (NoCipher, NoCompression, S2Compression) to fix cross-task naming conflict
- 13 new tests in JetStreamServiceOrchestrationTests covering all lifecycle paths
 2026-02-24 06:03:46 -05:00
Joseph Doherty
921554f410 feat: define StreamStore/ConsumerStore interfaces from Go store.go 
Port IStreamStore, IConsumerStore, StoreMsg, StreamState, SimpleState,
ConsumerState, FileStoreConfig, StoreCipher, StoreCompression types.
Rename Models.StreamState → ApiStreamState to avoid namespace conflict.
 2026-02-23 21:06:16 -05:00
Joseph Doherty
256daad8e5 feat: port internal data structures from Go (Wave 2) 
- AVL SequenceSet: sparse sequence set with AVL tree, 16 tests
- Subject Tree: Adaptive Radix Tree (ART) with 5 node tiers, 59 tests
- Generic Subject List: trie-based subject matcher, 21 tests
- Time Hash Wheel: O(1) TTL expiration wheel, 8 tests

Total: 106 new tests (1,081 → 1,187 passing)
 2026-02-23 20:56:20 -05:00
Joseph Doherty
636906f545 feat: scaffold namespaces for data structures, FileStore, and RAFT 
Add stub source files for Internal/Avl, Internal/SubjectTree, Internal/Gsl,
Internal/TimeHashWheel, Raft/RaftState, and Raft/IRaftNode, plus empty test
directories for all new namespaces and a Concurrency suite directory.
 2026-02-23 20:42:42 -05:00
Joseph Doherty
61b1a00800 feat: phase C jetstream depth test parity — 34 new tests across 7 subsystems 
Stream lifecycle, publish/ack, consumer delivery, retention policy,
API endpoints, cluster formation, and leader failover tests ported
from Go nats-server reference. 1006 total tests passing.
 2026-02-23 19:55:31 -05:00
Joseph Doherty
7ffee8741f feat: phase A foundation test parity — 64 new tests across 11 subsystems 
Port Go NATS server test behaviors to .NET:
- Client pub/sub (5 tests): simple, no-echo, reply, queue distribution, empty body
- Client UNSUB (4 tests): unsub, auto-unsub max, unsub after auto, disconnect cleanup
- Client headers (3 tests): HPUB/HMSG, server info headers, no-responders 503
- Client lifecycle (3 tests): connect proto, max subscriptions, auth timeout
- Client slow consumer (1 test): pending limit detection and disconnect
- Parser edge cases (3 tests + 2 bug fixes): PUB arg variations, malformed protocol, max control line
- SubList concurrency (13 tests): race on remove/insert/match, large lists, invalid subjects, wildcards
- Server config (4 tests): ephemeral port, server name, name defaults, lame duck
- Route config (3 tests): cluster formation, cross-cluster messaging, reconnect
- Gateway basic (2 tests): cross-cluster forwarding, no echo to origin
- Leaf node basic (2 tests): hub-to-spoke and spoke-to-hub forwarding
- Account import/export (2 tests): stream export/import delivery, isolation

Also fixes NatsParser.ParseSub/ParseUnsub to throw ProtocolViolationException
for short command lines instead of ArgumentOutOfRangeException.

Full suite: 933 passed, 0 failed (up from 869).
 2026-02-23 19:26:30 -05:00
Joseph Doherty
081ff1b0db docs: synchronize strict full go parity evidence and status 2026-02-23 15:03:35 -05:00
Joseph Doherty
1c0fc8fc11 feat: add runtime profiling parity and close config runtime drift 2026-02-23 14:56:27 -05:00
Joseph Doherty
148ff9ebb6 feat: implement jetstream governance runtime parity semantics 2026-02-23 14:54:30 -05:00
Joseph Doherty
0413ff6ae9 feat: implement strict raft consensus and convergence parity 2026-02-23 14:53:18 -05:00
Joseph Doherty
56177a7099 feat: enforce filestore durability and recovery invariants 2026-02-23 14:51:30 -05:00
Joseph Doherty
3896512f84 feat: complete jetstream mirror source strict runtime parity 2026-02-23 14:50:18 -05:00
Joseph Doherty
7bea35aaa8 feat: harden jetstream consumer state machine parity 2026-02-23 14:48:47 -05:00
Joseph Doherty
cdde3c7a1d feat: implement strict retention runtime parity for jetstream 2026-02-23 14:47:24 -05:00
Joseph Doherty
b2312c0dac feat: enforce mqtt auth tls and keepalive semantics 2026-02-23 14:45:37 -05:00
Joseph Doherty
7dcf5776b3 feat: implement mqtt session and qos ack runtime semantics 2026-02-23 14:43:08 -05:00
Joseph Doherty
7faf42c588 feat: implement mqtt packet-level parser and writer 2026-02-23 14:41:23 -05:00
Joseph Doherty
958c4aa8ed feat: harden gateway reply remap and leaf loop transparency 2026-02-23 14:40:07 -05:00
Joseph Doherty
d83b37fec1 feat: make inter-server interest propagation idempotent 2026-02-23 14:38:23 -05:00
Joseph Doherty
6a05308143 feat: enforce account-scoped remote delivery semantics 2026-02-23 14:36:44 -05:00
Joseph Doherty
377ad4a299 feat: complete jetstream deep operational parity closure 2026-02-23 13:43:14 -05:00
Joseph Doherty
2b64d762f6 feat: execute full-repo remaining parity closure plan 2026-02-23 13:08:52 -05:00
Joseph Doherty
b41e6ff320 feat: execute post-baseline jetstream parity plan 2026-02-23 12:11:19 -05:00
Joseph Doherty
8bce096f55 feat: complete final jetstream parity transport and runtime baselines 2026-02-23 11:04:43 -05:00
Joseph Doherty
f46b331921 feat: complete remaining jetstream parity implementation plan 2026-02-23 10:16:16 -05:00
Joseph Doherty
3fea2da2cf Fix merge regressions after jetstream parity merge 2026-02-23 08:56:15 -05:00
Joseph Doherty
a8985ecb1a Merge branch 'codex/jetstream-full-parity-executeplan' into main 
# Conflicts:
#	differences.md
#	docs/plans/2026-02-23-jetstream-full-parity-plan.md
#	src/NATS.Server/Auth/Account.cs
#	src/NATS.Server/Configuration/ConfigProcessor.cs
#	src/NATS.Server/Monitoring/VarzHandler.cs
#	src/NATS.Server/NatsClient.cs
#	src/NATS.Server/NatsOptions.cs
#	src/NATS.Server/NatsServer.cs
 2026-02-23 08:53:44 -05:00
Joseph Doherty
6c83f12e5c feat: add reload semantics for cluster and jetstream options 2026-02-23 06:23:34 -05:00
Joseph Doherty
2aa7265db1 feat: enforce account jetstream limits and jwt tiers 2026-02-23 06:21:51 -05:00
Joseph Doherty
ccbcf759a9 feat: implement jsz and live jetstream monitoring 2026-02-23 06:19:41 -05:00
Joseph Doherty
c87661800d feat: add stream replica groups and leader stepdown 2026-02-23 06:17:30 -05:00
Joseph Doherty
23216d0a48 feat: integrate jetstream meta-group placement 2026-02-23 06:16:01 -05:00
Joseph Doherty
71f7f569b9 Merge branch 'feature/mqtt-connection-type' 2026-02-23 06:15:32 -05:00
Joseph Doherty
3531a87de0 Merge branch 'feature/system-account-types' 
Add SYSTEM and ACCOUNT connection types with InternalClient,
InternalEventSystem, system event publishing, request-reply services,
and cross-account import/export support.
 2026-02-23 06:14:11 -05:00
Joseph Doherty
005600b9b8 feat: implement raft snapshot catchup 2026-02-23 06:13:08 -05:00
Joseph Doherty
ecc4752c07 feat: implement raft log replication and apply 2026-02-23 06:12:18 -05:00
Joseph Doherty
66ec378bdc feat: implement raft election and term state 2026-02-23 06:11:28 -05:00
Joseph Doherty
f1d3c19594 feat: add jetstream mirror and source orchestration 2026-02-23 06:10:41 -05:00
Joseph Doherty
d3aad48096 feat: enforce jetstream ack and redelivery semantics 2026-02-23 06:09:26 -05:00
Joseph Doherty
fecb51095f feat: implement jetstream push delivery and heartbeat 2026-02-23 06:08:14 -05:00
Joseph Doherty
54207e2906 feat: expand mqtt varz monitoring with all Go-compatible fields 2026-02-23 06:07:38 -05:00
Joseph Doherty
9a0de19c2d feat: implement jetstream pull consumer fetch 2026-02-23 06:07:02 -05:00
Joseph Doherty
40b940b1fd feat: add jetstream consumer api lifecycle 2026-02-23 06:06:02 -05:00
Joseph Doherty
6825839191 feat: add jetstream publish preconditions and dedupe 2026-02-23 06:05:01 -05:00
Joseph Doherty
d73e7e2f88 feat: enforce jetstream retention and limits 2026-02-23 06:04:23 -05:00
Joseph Doherty
86283a7f97 feat: add latency tracking for service import request-reply 2026-02-23 06:03:37 -05:00