dohertj2/natsdotnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
290 Commits 2 Branches 0 Tags
4fa0be2281c738993a5a149ae54d8b3a4c553962
Commit Graph

179 Commits

Author SHA1 Message Date
Joseph Doherty
4fa0be2281 feat: add JetStreamClusterFixture for multi-node cluster tests (Go parity) 
Adds a unified JetStreamClusterFixture consolidating the capabilities of all 7
per-suite fixtures (ClusterFormationFixture, ClusterStreamFixture, LeaderFailoverFixture, etc.)
into a single reusable helper for Tasks 6-10. Includes new Go-parity helpers
(WaitOnStreamLeaderAsync, WaitOnConsumerLeaderAsync, GetConsumerLeaderId,
StepDownMetaLeader, SimulateNodeRestart, RemoveNode) matching jetstream_helpers_test.go.
27 smoke tests verify all capabilities pass.
 2026-02-24 07:36:32 -05:00
Joseph Doherty
543b791cb5 feat: port remaining JetStream core tests from Go (edge cases, preconditions, direct get) 
Adds 110 new tests across 5 files covering gaps identified from Go's jetstream_test.go:

- JetStreamStreamEdgeCaseTests (29 tests): max msgs/bytes enforcement, discard
  old/new policies, max msg size, max msgs per subject, sealed/deny-delete/deny-purge
  config, work queue and interest retention, state tracking, CRUD edges.

- JetStreamConsumerDeliveryEdgeTests (25 tests): AckProcessor unit tests (register,
  drop, ack floor, expiry, redelivery), push consumer heartbeat/flow-control frames,
  pull fetch no-wait, batch limit, filter delivery, wildcard filter, ack explicit
  pending tracking, ack-all clearing, work queue pull consumer.

- JetStreamPublishPreconditionTests (21 tests): expected-last-seq match/mismatch,
  duplicate window dedup acceptance/rejection, window expiry allows re-publish,
  PublishPreconditions unit tests (IsDuplicate, Record, TrimOlderThan,
  CheckExpectedLastSeq), pub ack stream/seq fields, sequential writes enforcement.

- JetStreamAccountLimitTests (17 tests): max streams per account (1/3/unlimited),
  slot freed on delete, Account.TryReserveStream/ReleaseStream unit tests,
  JetStreamStreamCount tracking, account info stream/consumer counts, stream names
  sorted, consumer names list, error code 10027 on limit exceeded.

- JetStreamDirectGetTests (18 tests): direct get by sequence (first/middle/last),
  subject preservation, non-existent sequence error, empty stream error, zero seq
  error, multiple independent retrieves, STREAM.MSG.GET API, get-after-delete,
  get-after-purge, memory storage, backend type reporting, consistency between
  direct get and stream msg get.

Go reference: golang/nats-server/server/jetstream_test.go
 2026-02-24 06:47:17 -05:00
Joseph Doherty
3e972f217e feat: add FileStore 6-way permutation tests (Go testFileStoreAllPermutations parity) 
Port 20 tests across the {NoCipher, ChaCha, Aes} x {NoCompression, S2Compression}
matrix from Go's testFileStoreAllPermutations (filestore_test.go:55), yielding 120
total Theory executions in FileStorePermutationTests.cs.

Also fix PushFrame.Subject property missing compilation error that was blocking
the test build (JetStreamConsumerDeliveryEdgeTests.cs:119).

Tests covered (each runs 6x):
  Store_and_load_basic                       TestFileStoreBasics:86
  Store_multiple_messages_load_by_sequence   TestFileStoreBasics:86
  LoadLastBySubject_returns_most_recent_for_subject
  Remove_single_message_updates_state        TestFileStoreBasics:129
  Purge_clears_all_messages                  TestFileStorePurge:710
  TrimToMaxMessages_enforces_limit           TestFileStoreMsgLimitBug:518
  Block_rotation_when_exceeding_block_size   TestFileStoreAndRetrieveMultiBlock:1527
  GetState_returns_correct_counts            TestFileStoreBasics:104
  Snapshot_and_restore_round_trip            TestFileStoreSnapshot:1799
  ListAsync_returns_ordered_messages         TestFileStoreTimeStamps:683
  MaxAge_prunes_expired_messages             TestFileStoreAgeLimit:616
  Recovery_after_reopen_preserves_messages   TestFileStoreBasicWriteMsgsAndRestore:181
  Large_payload_store_and_load               64 KiB random payload variant
  Multiple_subjects_filter_by_subject        TestFileStoreBasics multi-subject
  Sequential_writes_maintain_ordering        TestFileStoreSelectNextFirst:304
  Store_creates_files_on_disk                disk-presence variant
  Write_and_read_same_block                  TestFileStoreWriteAndReadSameBlock:1510
  Stored_messages_have_non_decreasing_timestamps TestFileStoreTimeStamps:683
  Remove_out_of_order_collapses_first_seq    TestFileStoreCollapseDmap:1561
  Snapshot_after_removes_preserves_remaining TestFileStoreSnapshot:1904
 2026-02-24 06:43:48 -05:00
Joseph Doherty
6bcd682b76 feat: add NatsRaftTransport with NATS subject routing ($NRG.*) 
Implements RaftSubjects static class with Go's $NRG.* subject constants
and NatsRaftTransport which routes RAFT RPCs over those subjects using
RaftAppendEntryWire / RaftVoteRequestWire encoding. 43 tests cover all
subject patterns, wire encoding fidelity, and transport construction.
 2026-02-24 06:40:41 -05:00
Joseph Doherty
9cc9888bb4 feat: add S2 compression and AEAD encryption for FileStore (Go parity) 
Replace Deflate+XOR with IronSnappy S2 block compression and ChaCha20-Poly1305 / AES-256-GCM
AEAD encryption, matching golang/nats-server/server/filestore.go. Introduces FSV2 envelope
format alongside existing FSV1 for backward compatibility. Adds 55 new tests across
S2CodecTests, AeadEncryptorTests, and FileStoreV2Tests covering all 6 cipher×compression
permutations, tamper detection, and legacy format round-trips.
 2026-02-24 06:29:34 -05:00
Joseph Doherty
2c9683e7aa feat: upgrade JetStreamService to lifecycle orchestrator 
Implements enableJetStream() semantics from golang/nats-server/server/jetstream.go:414-523.

- JetStreamService.StartAsync(): validates config, creates store directory
  (including nested paths via Directory.CreateDirectory), registers all
  $JS.API.> subjects, logs startup stats; idempotent on double-start
- JetStreamService.DisposeAsync(): clears registered subjects, marks not running
- New properties: RegisteredApiSubjects, MaxStreams, MaxConsumers, MaxMemory, MaxStore
- JetStreamOptions: adds MaxStreams and MaxConsumers limits (0 = unlimited)
- FileStoreConfig: removes duplicate StoreCipher/StoreCompression enum declarations
  now that AeadEncryptor.cs owns them; updates defaults to NoCipher/NoCompression
- FileStoreOptions/FileStore: align enum member names with AeadEncryptor.cs
  (NoCipher, NoCompression, S2Compression) to fix cross-task naming conflict
- 13 new tests in JetStreamServiceOrchestrationTests covering all lifecycle paths
 2026-02-24 06:03:46 -05:00
Joseph Doherty
3ff801865a feat: Waves 3-5 — FileStore, RAFT, JetStream clustering, and concurrency tests 
Add comprehensive Go-parity test coverage across 3 subsystems:
- FileStore: basic CRUD, limits, purge, recovery, subjects, encryption,
  compression, MemStore (161 tests, 24 skipped for not-yet-implemented)
- RAFT: core types, wire format, election, log replication, snapshots
  (95 tests)
- JetStream Clustering: meta controller, stream/consumer replica groups,
  concurrency stress tests (90 tests)

Total: ~346 new test annotations across 17 files (+7,557 lines)
Full suite: 2,606 passing, 0 failures, 27 skipped
 2026-02-23 22:55:41 -05:00
Joseph Doherty
f1353868af feat: Wave 6 batch 2 — accounts/auth, gateways, routes, JetStream API, JetStream cluster tests 
Add comprehensive Go-parity test coverage across 5 subsystems:
- Accounts/Auth: isolation, import/export, auth mechanisms, permissions (82 tests)
- Gateways: connection, forwarding, interest mode, config (106 tests)
- Routes: connection, subscription, forwarding, config validation (78 tests)
- JetStream API: stream/consumer CRUD, pub/sub, features, admin (234 tests)
- JetStream Cluster: streams, consumers, failover, meta (108 tests)

Total: ~608 new test annotations across 22 files (+13,844 lines)
All tests pass individually; suite total: 2,283 passing, 3 skipped
 2026-02-23 22:35:06 -05:00
Joseph Doherty
9554d53bf5 feat: Wave 6 batch 1 — monitoring, config reload, client protocol, MQTT, leaf node tests 
Port 405 new test methods across 5 subsystems for Go parity:
- Monitoring: 102 tests (varz, connz, routez, subsz, stacksz)
- Leaf Nodes: 85 tests (connection, forwarding, loop detection, subject filter, JetStream)
- MQTT Bridge: 86 tests (advanced, auth, retained messages, topic mapping, will messages)
- Client Protocol: 73 tests (connection handling, protocol violations, limits)
- Config Reload: 59 tests (hot reload, option changes, permission updates)

Total: 1,678 tests passing, 0 failures, 3 skipped
 2026-02-23 21:40:29 -05:00
Joseph Doherty
921554f410 feat: define StreamStore/ConsumerStore interfaces from Go store.go 
Port IStreamStore, IConsumerStore, StoreMsg, StreamState, SimpleState,
ConsumerState, FileStoreConfig, StoreCipher, StoreCompression types.
Rename Models.StreamState → ApiStreamState to avoid namespace conflict.
 2026-02-23 21:06:16 -05:00
Joseph Doherty
256daad8e5 feat: port internal data structures from Go (Wave 2) 
- AVL SequenceSet: sparse sequence set with AVL tree, 16 tests
- Subject Tree: Adaptive Radix Tree (ART) with 5 node tiers, 59 tests
- Generic Subject List: trie-based subject matcher, 21 tests
- Time Hash Wheel: O(1) TTL expiration wheel, 8 tests

Total: 106 new tests (1,081 → 1,187 passing)
 2026-02-23 20:56:20 -05:00
Joseph Doherty
636906f545 feat: scaffold namespaces for data structures, FileStore, and RAFT 
Add stub source files for Internal/Avl, Internal/SubjectTree, Internal/Gsl,
Internal/TimeHashWheel, Raft/RaftState, and Raft/IRaftNode, plus empty test
directories for all new namespaces and a Concurrency suite directory.
 2026-02-23 20:42:42 -05:00
Joseph Doherty
553483b6ba feat: phase D protocol surfaces test parity — 75 new tests across MQTT and JWT 
MQTT packet parsing (41 tests), QoS/session delivery (8 tests),
and JWT claim edge cases (43 new tests). All 4 phases complete.
1081 total tests passing, 0 failures.
 2026-02-23 20:06:54 -05:00
Joseph Doherty
61b1a00800 feat: phase C jetstream depth test parity — 34 new tests across 7 subsystems 
Stream lifecycle, publish/ack, consumer delivery, retention policy,
API endpoints, cluster formation, and leader failover tests ported
from Go nats-server reference. 1006 total tests passing.
 2026-02-23 19:55:31 -05:00
Joseph Doherty
28d379e6b7 feat: phase B distributed substrate test parity — 39 new tests across 5 subsystems 
FileStore basics (4), MemStore/retention (10), RAFT election/append (16),
config reload parity (3), monitoring endpoints varz/connz/healthz (6).
972 total tests passing, 0 failures.
 2026-02-23 19:41:30 -05:00
Joseph Doherty
7ffee8741f feat: phase A foundation test parity — 64 new tests across 11 subsystems 
Port Go NATS server test behaviors to .NET:
- Client pub/sub (5 tests): simple, no-echo, reply, queue distribution, empty body
- Client UNSUB (4 tests): unsub, auto-unsub max, unsub after auto, disconnect cleanup
- Client headers (3 tests): HPUB/HMSG, server info headers, no-responders 503
- Client lifecycle (3 tests): connect proto, max subscriptions, auth timeout
- Client slow consumer (1 test): pending limit detection and disconnect
- Parser edge cases (3 tests + 2 bug fixes): PUB arg variations, malformed protocol, max control line
- SubList concurrency (13 tests): race on remove/insert/match, large lists, invalid subjects, wildcards
- Server config (4 tests): ephemeral port, server name, name defaults, lame duck
- Route config (3 tests): cluster formation, cross-cluster messaging, reconnect
- Gateway basic (2 tests): cross-cluster forwarding, no echo to origin
- Leaf node basic (2 tests): hub-to-spoke and spoke-to-hub forwarding
- Account import/export (2 tests): stream export/import delivery, isolation

Also fixes NatsParser.ParseSub/ParseUnsub to throw ProtocolViolationException
for short command lines instead of ArgumentOutOfRangeException.

Full suite: 933 passed, 0 failed (up from 869).
 2026-02-23 19:26:30 -05:00
Joseph Doherty
081ff1b0db docs: synchronize strict full go parity evidence and status 2026-02-23 15:03:35 -05:00
Joseph Doherty
1c0fc8fc11 feat: add runtime profiling parity and close config runtime drift 2026-02-23 14:56:27 -05:00
Joseph Doherty
148ff9ebb6 feat: implement jetstream governance runtime parity semantics 2026-02-23 14:54:30 -05:00
Joseph Doherty
0413ff6ae9 feat: implement strict raft consensus and convergence parity 2026-02-23 14:53:18 -05:00
Joseph Doherty
56177a7099 feat: enforce filestore durability and recovery invariants 2026-02-23 14:51:30 -05:00
Joseph Doherty
3896512f84 feat: complete jetstream mirror source strict runtime parity 2026-02-23 14:50:18 -05:00
Joseph Doherty
7bea35aaa8 feat: harden jetstream consumer state machine parity 2026-02-23 14:48:47 -05:00
Joseph Doherty
cdde3c7a1d feat: implement strict retention runtime parity for jetstream 2026-02-23 14:47:24 -05:00
Joseph Doherty
b2312c0dac feat: enforce mqtt auth tls and keepalive semantics 2026-02-23 14:45:37 -05:00
Joseph Doherty
7dcf5776b3 feat: implement mqtt session and qos ack runtime semantics 2026-02-23 14:43:08 -05:00
Joseph Doherty
7faf42c588 feat: implement mqtt packet-level parser and writer 2026-02-23 14:41:23 -05:00
Joseph Doherty
958c4aa8ed feat: harden gateway reply remap and leaf loop transparency 2026-02-23 14:40:07 -05:00
Joseph Doherty
d83b37fec1 feat: make inter-server interest propagation idempotent 2026-02-23 14:38:23 -05:00
Joseph Doherty
6a05308143 feat: enforce account-scoped remote delivery semantics 2026-02-23 14:36:44 -05:00
Joseph Doherty
ec373d36f6 test: add strict nats capability inventory guardrail 2026-02-23 14:32:55 -05:00
Joseph Doherty
377ad4a299 feat: complete jetstream deep operational parity closure 2026-02-23 13:43:14 -05:00
Joseph Doherty
2b64d762f6 feat: execute full-repo remaining parity closure plan 2026-02-23 13:08:52 -05:00
Joseph Doherty
b41e6ff320 feat: execute post-baseline jetstream parity plan 2026-02-23 12:11:19 -05:00
Joseph Doherty
8bce096f55 feat: complete final jetstream parity transport and runtime baselines 2026-02-23 11:04:43 -05:00
Joseph Doherty
f46b331921 feat: complete remaining jetstream parity implementation plan 2026-02-23 10:16:16 -05:00
Joseph Doherty
a8985ecb1a Merge branch 'codex/jetstream-full-parity-executeplan' into main 
# Conflicts:
#	differences.md
#	docs/plans/2026-02-23-jetstream-full-parity-plan.md
#	src/NATS.Server/Auth/Account.cs
#	src/NATS.Server/Configuration/ConfigProcessor.cs
#	src/NATS.Server/Monitoring/VarzHandler.cs
#	src/NATS.Server/NatsClient.cs
#	src/NATS.Server/NatsOptions.cs
#	src/NATS.Server/NatsServer.cs
 2026-02-23 08:53:44 -05:00
Joseph Doherty
73dd3307ba test: add jetstream integration matrix coverage 2026-02-23 06:25:23 -05:00
Joseph Doherty
264b49f96a test: add go jetstream parity runner 2026-02-23 06:24:41 -05:00
Joseph Doherty
6c83f12e5c feat: add reload semantics for cluster and jetstream options 2026-02-23 06:23:34 -05:00
Joseph Doherty
2aa7265db1 feat: enforce account jetstream limits and jwt tiers 2026-02-23 06:21:51 -05:00
Joseph Doherty
ccbcf759a9 feat: implement jsz and live jetstream monitoring 2026-02-23 06:19:41 -05:00
Joseph Doherty
c87661800d feat: add stream replica groups and leader stepdown 2026-02-23 06:17:30 -05:00
Joseph Doherty
23216d0a48 feat: integrate jetstream meta-group placement 2026-02-23 06:16:01 -05:00
Joseph Doherty
71f7f569b9 Merge branch 'feature/mqtt-connection-type' 2026-02-23 06:15:32 -05:00
Joseph Doherty
3531a87de0 Merge branch 'feature/system-account-types' 
Add SYSTEM and ACCOUNT connection types with InternalClient,
InternalEventSystem, system event publishing, request-reply services,
and cross-account import/export support.
 2026-02-23 06:14:11 -05:00
Joseph Doherty
005600b9b8 feat: implement raft snapshot catchup 2026-02-23 06:13:08 -05:00
Joseph Doherty
ecc4752c07 feat: implement raft log replication and apply 2026-02-23 06:12:18 -05:00
Joseph Doherty
66ec378bdc feat: implement raft election and term state 2026-02-23 06:11:28 -05:00
Joseph Doherty
f1d3c19594 feat: add jetstream mirror and source orchestration 2026-02-23 06:10:41 -05:00