Commit Graph

474 Commits

Author SHA1 Message Date
Joseph Doherty
b9c83d6b3b feat: add route pool size negotiation (Gap 13.3)
Add NegotiatePoolSize static method and NegotiatedPoolSize property to
RouteConnection, and ConfiguredPoolSize / GetEffectivePoolSize to
RouteManager. Includes 14 tests covering negotiation semantics, backward
compatibility (zero means no pooling), default state, and deterministic
pool index computation.
2026-02-25 12:05:57 -05:00
Joseph Doherty
071717dcbf feat: add cluster split handling (Gap 13.5)
Add RemoveRoute, RemoveAllRoutesExcept, RegisterRoute (internal), and
DetectClusterSplit to RouteManager with ClusterSplitResult record, plus
10 tests covering partition detection and route removal behavior.
2026-02-25 12:03:05 -05:00
Joseph Doherty
3192caeab8 feat: add route hash storage for O(1) lookup (Gap 13.4)
Add ConcurrentDictionary<ulong, RouteConnection> _routesByHash with FNV-1a
64-bit hash key, RegisterRouteByHash/UnregisterRouteByHash/GetRouteByHash/
GetRouteByServerId methods, and HashedRouteCount property to RouteManager.
Includes 10 unit tests covering determinism, distinct hashes, CRUD lifecycle,
overwrite semantics, and no-op unregister.
2026-02-25 12:02:35 -05:00
Joseph Doherty
7f3e2e0e0b feat: add account-specific dedicated routes (Gap 13.2)
Add _accountRoutes ConcurrentDictionary to RouteManager with full
CRUD API: RegisterAccountRoute, UnregisterAccountRoute,
GetDedicatedAccountRoute, HasDedicatedRoute,
GetAccountsWithDedicatedRoutes, and DedicatedRouteCount property.
Update GetRouteForAccount to check dedicated routes before falling
back to pool-based selection. Add 10 unit tests in AccountRouteTests.
2026-02-25 12:01:49 -05:00
Joseph Doherty
5fd23571dc feat: add gateway connection registration with state tracking (Gap 11.7)
Adds GatewayConnectionState enum, GatewayRegistration record with atomic
message counters, and a full registry API on GatewayManager: RegisterGateway,
UpdateState, GetRegistration, GetAllRegistrations, UnregisterGateway,
GetConnectedGatewayCount, IncrementMessagesSent, IncrementMessagesReceived.
Covers 11 new tests in GatewayRegistrationTests.cs (all passing).
2026-02-25 11:54:52 -05:00
Joseph Doherty
684254ad86 feat: add gateway command protocol with Go-compatible wire format (Gap 11.6)
Add GatewayCommands static class with wire-format byte sequences (GINFO, GS+, GS-,
GMODE, GMSG, GPING, GPONG) and FormatSub/FormatUnsub/FormatMode/ParseCommandType
helpers matching Go's gateway.go protocol constants. Add GatewayCommandType enum.
10 tests covering all wire formats and command parsing.
2026-02-25 11:54:30 -05:00
Joseph Doherty
dc8d28c222 feat: add reply subject mapping cache with TTL (Gap 11.5)
Add ReplyMapCache to ReplyMapper.cs — an LRU cache with TTL expiration
for gateway reply subject mappings, avoiding repeated string parsing on
the hot path. Includes 10 unit tests covering LRU eviction, TTL expiry,
hit/miss counters, PurgeExpired, and Count.
2026-02-25 11:51:55 -05:00
Joseph Doherty
455a91579a feat: add gateway reconnection with exponential backoff (Gap 11.2)
Adds GatewayReconnectPolicy with exponential backoff and jitter, and
reconnect attempt tracking (GetReconnectAttempts, ResetReconnectAttempts,
ReconnectGatewayAsync) to GatewayManager. 10 new tests cover delay
calculation, cap behaviour, jitter, and per-gateway counter isolation.
2026-02-25 11:51:40 -05:00
Joseph Doherty
d598276807 feat: add account-specific gateway routes (Gap 11.3)
Adds per-account subscription tracking to GatewayConnection via
AddAccountSubscription/RemoveAccountSubscription/GetAccountSubscriptions/
AccountSubscriptionCount, with corresponding SendAccountSubscriptions and
GetAccountSubscriptions helpers on GatewayManager. Covered by 10 new unit tests.
2026-02-25 11:51:09 -05:00
Joseph Doherty
4c53159de8 feat: add runtime logging level changes (Gap 14.5)
Adds LoggingChangeResult and ApplyLoggingChanges to ConfigReloader, deriving
an effective log level from Debug/Trace flags (Trace > Debug > Information) and
reporting flag-level and level-string changes for hot-reload notification.
2026-02-25 11:48:29 -05:00
Joseph Doherty
ff07137ea6 feat: add JetStream config change reload (Gap 14.6)
Add ApplyJetStreamConfigChanges to ConfigReloader for detecting MaxMemoryStore,
MaxFileStore, and Domain changes during hot reload. Add Domain property to
JetStreamOptions. Port 10 unit tests covering all change permutations.
2026-02-25 11:48:22 -05:00
Joseph Doherty
5116aed491 feat: add TLS certificate hot-reload for new connections (Gap 14.3)
Add ReloadTlsCertificates(oldOpts, newOpts) returning TlsReloadResult for
path-based cert comparison and validation during config hot-reload. Add 10
targeted tests covering no-change, path detection, missing file, null transitions,
and success cases.
2026-02-25 11:48:11 -05:00
Joseph Doherty
074ff6b287 feat: add cluster config hot reload (Gap 14.4)
Adds ClusterConfigChangeResult and ApplyClusterConfigChanges to ConfigReloader,
comparing route/gateway/leaf URL sets between old and new NatsOptions and reporting
added/removed routes for connection reconciliation on hot reload.
2026-02-25 11:47:02 -05:00
Joseph Doherty
42e072ad71 feat: add auth change propagation to existing connections (Gap 14.2)
Add PropagateAuthChanges to ConfigReloader that compares Users, Accounts,
and Authorization token between old and new NatsOptions, returning an
AuthChangeResult describing which auth fields changed for connection re-evaluation.
2026-02-25 11:46:28 -05:00
Joseph Doherty
5fea08dda0 feat: complete MQTT session flapper detection (Gap 6.6)
Add FlapperState class and per-client exponential backoff tracking to
MqttSessionStore. New TrackConnectDisconnect(string) overload returns
FlapperState with backoff level and expiry; IsFlapper, GetBackoffMs,
ClearFlapperState, and CheckAndClearStableClients give callers full
visibility and cleanup control. Legacy two-arg overload preserved for
backward compatibility. Ten unit tests cover counting, threshold,
exponential growth, 60s cap, window reset, and stable-client sweep.
2026-02-25 11:42:24 -05:00
Joseph Doherty
96db73d1c6 feat: deliver retained messages on MQTT SUBSCRIBE (Gap 6.5)
Add DeliverRetainedOnSubscribe to MqttRetainedStore, which iterates
GetMatchingRetained and fires a callback with (topic, payload, qos,
retain=true) for each match. Add 11 unit tests covering exact-match,
'+' single-level, '#' multi-level, no-match, cross-level rejection,
callback arguments, return count, and empty-store behaviour.
2026-02-25 11:40:50 -05:00
Joseph Doherty
f069fdc76a feat: add MQTT MaxAckPending flow control (Gap 6.4)
Implements MqttFlowController with per-subscription SemaphoreSlim-based
slot tracking for QoS 1/2 messages. Replaces the previous catch-and-swallow
pattern with an explicit CurrentCount guard on Release. 10 unit tests added.
2026-02-25 11:38:47 -05:00
Joseph Doherty
a44ad4b7fc feat: add MQTT will message delivery on abnormal disconnect (Gap 6.2)
Adds WillMessage class, SetWill/ClearWill/GetWill methods to MqttSessionStore,
PublishWillMessage that dispatches via OnPublish delegate (or tracks as delayed
when DelayIntervalSeconds > 0), and 10 unit tests covering all will message behaviors.
2026-02-25 11:38:43 -05:00
Joseph Doherty
18f0ca0587 feat: add consecutive short-read counter to prevent buffer oscillation (Gap 5.10)
Require 4 consecutive short reads before shrinking AdaptiveReadBuffer, matching
the Go server's readLoop behaviour and preventing buffer size thrashing.
2026-02-25 11:36:12 -05:00
Joseph Doherty
bd2504c8df feat: add SUB permission caching with generation invalidation (Gap 5.8)
Extend PermissionLruCache with SetSub/TryGetSub (internal key prefix "S:")
alongside existing PUB API ("P:" prefix, backward-compatible). Add Invalidate()
and Generation property for generation-based cache invalidation. Add
GenerationId/IncrementGeneration to Account for account-level change signalling.
10 new tests in SubPermissionCacheTests cover all paths.
2026-02-25 11:36:05 -05:00
Joseph Doherty
a6e8088526 test: verify internal client kinds (Gap 5.9) 2026-02-25 11:35:21 -05:00
Joseph Doherty
7e5c6e4fd9 feat: add slow consumer per-kind tracking with account counters (Gap 5.5)
Adds SlowConsumerTracker class for per-ClientKind slow consumer counting
with configurable threshold callbacks, and extends Account with atomic
IncrementSlowConsumers/SlowConsumerCount/ResetSlowConsumerCount members.
Includes 10 unit tests covering concurrency, threshold firing, and reset.
2026-02-25 11:33:58 -05:00
Joseph Doherty
774717d57c feat: add per-client trace delivery and echo control (Gap 5.7)
Implements ClientTraceInfo with TraceMsgDelivery recording and per-client
echo suppression; fixes AccountGoParityTests namespace ambiguity caused by
the new NATS.Server.Tests.Subscriptions test namespace.
2026-02-25 11:32:00 -05:00
Joseph Doherty
bc8f0e63bb feat: add dynamic write buffer pooling with broadcast drain (Gap 5.6)
Enhances OutboundBufferPool with tiered internal pools (512/4096/65536),
RentBuffer/ReturnBuffer raw-array surface, BroadcastDrain coalescing for
fan-out publish, and Interlocked stats counters (RentCount, ReturnCount,
BroadcastCount). Adds 10 DynamicBufferPoolTests covering all new paths.
2026-02-25 11:31:29 -05:00
Joseph Doherty
1a1e99f7d8 feat: add per-account subscription result cache with LRU (Gap 5.4)
Implements RouteResultCache with fixed-capacity LRU eviction and atomic
generation-based invalidation (Go ref: client.go routeCache, maxResultCacheSize=8192).
Fixes AccountGoParityTests namespace ambiguity introduced by new test file.
2026-02-25 11:31:17 -05:00
Joseph Doherty
0e1a39df96 feat: add source/mirror info reporting for monitoring (Gap 4.10)
Add GetMirrorInfo/GetSourceInfo methods to MirrorCoordinator and
SourceCoordinator returning structured MirrorInfoResponse/SourceInfoResponse
records (Name, Lag, Active ms, FilterSubject, Error). Wire both into
StreamManager.GetMirrorInfo and GetSourceInfos for the monitoring API path.
2026-02-25 11:26:34 -05:00
Joseph Doherty
5e49006cfa feat: add stream config update validation (Gap 4.8)
Add ValidateConfigUpdate to StreamManager with immutability rules for storage type,
mirror, sources, and retention policy; sealed stream guard; MaxConsumers decrease
prevention; even-replica rejection; and subject overlap detection against peer streams.
Wire the check into CreateOrUpdate for all update paths. 12 new tests in
ConfigUpdateValidationTests.cs cover all rules including the StreamManager integration test.
2026-02-25 11:25:38 -05:00
Joseph Doherty
79a3ccba4c feat: implement TAR-based stream snapshot with S2 compression (Gap 4.7)
Enhance StreamSnapshotService with CreateTarSnapshotAsync / RestoreTarSnapshotAsync
methods that produce a Snappy-compressed TAR archive (stream.json + messages/*.json).
Add CreateTarSnapshotWithDeadlineAsync for deadline-bounded snapshots, and a
SnapshotRestoreResult record carrying stats. 10 new unit tests in
JetStream/Snapshots/StreamSnapshotTests.cs exercise the full create/restore
round-trip, compression format, empty-stream edge case, and deadline enforcement.
2026-02-25 11:22:25 -05:00
Joseph Doherty
b9f6a8cc0b feat: add cluster-aware pending request tracking for pull consumers (Gap 3.14)
Adds ProposeWaitingRequest, RegisterClusterPending, RemoveClusterPending,
GetClusterPendingRequests, and ClusterPendingCount to PullConsumerEngine,
backed by a ConcurrentDictionary keyed by reply subject. Includes 10 xUnit
tests covering quorum checks, pending tracking, and concurrent access patterns.
2026-02-25 11:21:21 -05:00
Joseph Doherty
a113dd686d feat: complete source consumer API request generation (Gap 4.3)
Add BuildConsumerCreateRequest, BuildConsumerCreateSubject, and
GetDeliverySequence to SourceCoordinator, modelling Go's
setupSourceConsumer/trySetupSourceConsumer. Covers DeliverPolicy
resume-from-sequence, AckPolicy.None, push/flow-control/heartbeat
consumer fields, and the $JS.API.CONSUMER.CREATE.{name} subject format.
2026-02-25 11:21:21 -05:00
Joseph Doherty
aad9cf17e4 feat: add token bucket rate limiter for consumers (Gap 3.13)
Implements TokenBucketRateLimiter with refill-over-time semantics,
TryConsume/EstimateWait/WaitForTokensAsync API, and dynamic rate updates.
12 tests covering all behaviors including SW004-suppressed refill timing test.
2026-02-25 11:15:58 -05:00
Joseph Doherty
778687cf6f feat: add consumer reset to specific sequence (Gap 3.12)
Add ResetToSequence to ConsumerManager that updates NextSequence,
clears AckProcessor state via new ClearAll(), and zeroes PendingBytes.
Add AckProcessor.SetAckFloor() that prunes pending entries below the
new floor. Go reference: consumer.go:4241 processResetReq.
2026-02-25 11:15:33 -05:00
Joseph Doherty
b9aa62ae99 feat: add sample/observe mode with latency measurement (Gap 3.11)
Implements SampleTracker with stochastic delivery sampling (ParseSampleFrequency,
ShouldSample, RecordLatency) and LatencySample for consumer observability advisories.
Ports consumer.go sampleFrequency / shouldSample / parseSampleFrequency logic.
15 new tests covering parsing, rate extremes, deterministic seeded sampling, and field capture.
2026-02-25 11:14:58 -05:00
Joseph Doherty
8b4b236968 feat: add delivery interest tracking with auto-cleanup (Gap 3.8)
Implements DeliveryInterestTracker (consumer.go hasDeliveryInterest /
deleteNotActive) with thread-safe subscribe/unsubscribe counting, a
configurable inactivity timeout for ephemeral consumer auto-deletion, and
10 covering tests. Also adds SlopwatchSuppress to a pre-existing
Task.Delay in MaxDeliveriesTests.cs that was already testing time-based expiry.
2026-02-25 11:13:06 -05:00
Joseph Doherty
3d721c6ff1 feat: add filter skip tracking using SubjectMatch (Gap 3.10)
Add FilterSkipTracker using SubjectMatch.MatchLiteral() for NATS token-based
filter matching with sorted-set skip sequence gap tracking. Includes 14 tests
covering exact match, star/gt wildcards, multi-filter, counters, RecordSkip,
NextUnskippedSequence, PurgeBelow, and Reset.
2026-02-25 11:12:57 -05:00
Joseph Doherty
5b0283adf4 feat: add max delivery enforcement with advisory generation (Gap 3.9)
Adds MaxDeliver property, exceeded sequence tracking, DrainExceeded,
DeliveryExceededPolicy enum, and ScheduleRedelivery enforcement to
AckProcessor; 10 new tests in MaxDeliveriesTests.cs.
2026-02-25 11:12:49 -05:00
Joseph Doherty
ae4bc1f683 feat: implement core message delivery loop for push consumers (Gap 3.1)
Adds DeliveryLoopTests covering the gather loop's store polling, filter
subject enforcement, NextSequence advancement, deleted-message skipping,
GatheredCount tracking, signal wake-up, and cancellation behaviour.
2026-02-25 11:09:52 -05:00
Joseph Doherty
7611bcc464 feat: add idle heartbeat with pending count headers and flow control stall detection (Gap 3.5)
Heartbeat frames now include Nats-Pending-Messages and Nats-Pending-Bytes
headers populated from the ConsumerHandle. Flow control frames increment
FlowControlPendingCount; AcknowledgeFlowControl() decrements it. IsFlowControlStalled
returns true when pending count reaches MaxFlowControlPending (2).

Go reference: consumer.go:5222 (sendIdleHeartbeat), consumer.go:5495 (sendFlowControl).
2026-02-25 11:05:31 -05:00
Joseph Doherty
0acf59f92a feat: wire snapshot/restore API endpoints (Gap 7.4 stub)
Add HandleSnapshotAsync and HandleRestoreAsync with stream-name validation,
chunk metadata (NumChunks, BlkSize) in the response, and richer error codes.
Add StreamManager.Exists helper. Add JetStreamSnapshot.StreamName/NumChunks/BlkSize
fields. Fix AdvisoryEventTests.cs using-directive ordering. Add 12 SnapshotApiTests.
2026-02-25 10:57:01 -05:00
Joseph Doherty
41604df752 feat: wire consumer pause/resume API endpoint (Gap 7.5)
Enhances HandlePause to parse pause_until (RFC3339) for time-bounded
pauses and returns current pause state in the response body. Adds
Paused/PauseUntil properties and PauseResponse factory to
JetStreamApiResponse. Covers 10 new parity tests.
2026-02-25 10:56:22 -05:00
Joseph Doherty
2c52b69c93 feat: add advisory event publication for API operations (Gap 7.6)
Add JetStream advisory subject constants to EventSubjects and a lightweight
AdvisoryPublisher that publishes stream/consumer lifecycle events to
$JS.EVENT.ADVISORY.* subjects without depending on InternalEventSystem
directly (testable via Action delegate injection).
2026-02-25 10:56:11 -05:00
Joseph Doherty
c0d206102d feat: add API rate limiting and request deduplication (Gap 7.3)
Implements ApiRateLimiter with SemaphoreSlim-based concurrency limiting (default 256 slots)
and ConcurrentDictionary dedup cache keyed by request ID with configurable TTL, matching
Go's jetstream_api.go maxConcurrentRequests semaphore and dedup window. Also adds
ClusteredRequestProcessor for correlating pending RAFT proposals with waiting callers via
TaskCompletionSource, and SlopwatchSuppressAttribute as a marker for intentional timing-based
tests. 12 ApiRateLimiter tests + 13 ClusteredRequestProcessor tests all pass.
2026-02-25 10:51:22 -05:00
Joseph Doherty
f6d024c50d feat: add clustered stream/consumer API handlers (Gap 2.12)
Implement HandleClusteredCreateAsync, HandleClusteredUpdateAsync, and
HandleClusteredDeleteAsync on StreamApiHandlers, and HandleClusteredCreateAsync
and HandleClusteredDeleteAsync on ConsumerApiHandlers. These handlers propose
operations to the meta RAFT group (JetStreamMetaGroup) instead of operating on
the local StreamManager/ConsumerManager, matching the Go jsClusteredStreamRequest
and jsClusteredConsumerRequest patterns (jetstream_cluster.go:7620-8265).

Ten tests in ClusteredApiTests.cs verify: stream create proposes to meta group,
duplicate-stream error, not-leader error (code 10003), stream update, stream
delete, not-found-on-delete, consumer create on stream, consumer-on-missing-stream
error, consumer delete, and not-found consumer delete.
2026-02-25 10:43:49 -05:00
Joseph Doherty
d817d6f7a2 feat: implement leader forwarding for JetStream API (Gap 7.1)
Add ILeaderForwarder interface and DefaultLeaderForwarder, update
JetStreamApiRouter with RouteAsync + ForwardedCount so non-leader nodes
can attempt to forward mutating requests to the meta-group leader before
falling back to a NotLeader error response.
2026-02-25 09:53:50 -05:00
Joseph Doherty
aeeb2e6929 feat: add binary assignment codec with golden fixture tests (Gap 2.10)
Implements AssignmentCodec with JSON serialization for StreamAssignment
and ConsumerAssignment, plus Snappy compression helpers for large payloads.
Adds 12 tests covering round-trips, error handling, compression, and a
golden fixture for format stability.
2026-02-25 09:35:19 -05:00
Joseph Doherty
a7c094d6c1 feat: add unsupported asset handling for mixed-version clusters (Gap 2.11)
Add Version property to StreamAssignment and ConsumerAssignment so future-version
entries from newer cluster peers are gracefully skipped rather than applied incorrectly.
ProcessStreamAssignment and ProcessConsumerAssignment now reject entries with
Version > CurrentVersion (1), incrementing SkippedUnsupportedEntries for operator
visibility. Add MetaEntryType.Unknown with a no-op ApplyEntry handler so unrecognised
RAFT entry types never crash the cluster. Version 0 is treated as current for
backward compatibility with pre-versioned assignments.
2026-02-25 09:22:18 -05:00
Joseph Doherty
f69f9b3220 feat: add RaftGroup lifecycle methods (Gap 2.9)
Implement IsMember, SetPreferred, RemovePeer, AddPeer, CreateRaftGroup
factory, IsUnderReplicated, and IsOverReplicated on RaftGroup. Add 22
RaftGroupLifecycleTests covering all helpers and quorum size calculation.
2026-02-25 08:59:36 -05:00
Joseph Doherty
38ae1f6bea feat: add topology-aware placement with tag enforcement (Gap 2.8)
Extends PlacementEngine.SelectPeerGroup with three new capabilities ported
from jetstream_cluster.go:7212 selectPeerGroup:

- JetStreamUniqueTag enforcement: PlacementPolicy.UniqueTag (e.g. "az")
  ensures no two replicas share the same value for a tag with that prefix,
  matching Go's uniqueTagPrefix / checkUniqueTag logic.
- MaxAssetsPerPeer HA limit: peers at or above their asset ceiling are
  deprioritised (moved to fallback), not hard-excluded, so selection still
  succeeds when no preferred peers remain.
- Weighted scoring: candidates sorted by
  score = AvailableStorage - (CurrentAssets * assetCostWeight)
  (DefaultAssetCostWeight = 1 GiB) replacing the raw-storage sort, with a
  custom weight parameter for testing.

10 new tests in TopologyPlacementTests.cs cover all three features and their
edge cases. All 30 PlacementEngine tests continue to pass.
2026-02-25 08:59:18 -05:00
Joseph Doherty
e5f599f770 feat: add peer management with stream reassignment (Gap 2.4)
Add ProcessAddPeer/ProcessRemovePeer to JetStreamMetaGroup for
peer-driven stream reassignment. Includes AddKnownPeer/RemoveKnownPeer
tracked in a HashSet, RemovePeerFromStream, RemapStreamAssignment with
replacement-peer selection from the known pool, and DesiredReplicas on
RaftGroup for under-replication detection. Go ref: jetstream_cluster.go:2290-2439.
2026-02-25 08:53:05 -05:00
Joseph Doherty
0f8f34afaa feat: add entry application pipeline for meta and stream RAFT groups (Gap 2.7)
Extend ApplyEntry in JetStreamMetaGroup with PeerAdd/PeerRemove dispatch
to the existing Task 12 peer management methods. Add StreamMsgOp (Store,
Remove, Purge) and ConsumerOp (Ack, Nak, Deliver, Term, Progress) enums
plus ApplyStreamMsgOp and ApplyConsumerEntry methods to StreamReplicaGroup.
Extend ApplyCommittedEntriesAsync to parse smsg:/centry: command prefixes
and route to the new apply methods. Add MetaEntryType.PeerAdd/PeerRemove
enum values. 35 new tests in EntryApplicationTests.cs, all passing.
2026-02-25 08:38:21 -05:00