Commit Graph

20 Commits

Author SHA1 Message Date
Joseph Doherty
b9f6a8cc0b feat: add cluster-aware pending request tracking for pull consumers (Gap 3.14)
Adds ProposeWaitingRequest, RegisterClusterPending, RemoveClusterPending,
GetClusterPendingRequests, and ClusterPendingCount to PullConsumerEngine,
backed by a ConcurrentDictionary keyed by reply subject. Includes 10 xUnit
tests covering quorum checks, pending tracking, and concurrent access patterns.
2026-02-25 11:21:21 -05:00
Joseph Doherty
aad9cf17e4 feat: add token bucket rate limiter for consumers (Gap 3.13)
Implements TokenBucketRateLimiter with refill-over-time semantics,
TryConsume/EstimateWait/WaitForTokensAsync API, and dynamic rate updates.
12 tests covering all behaviors including SW004-suppressed refill timing test.
2026-02-25 11:15:58 -05:00
Joseph Doherty
778687cf6f feat: add consumer reset to specific sequence (Gap 3.12)
Add ResetToSequence to ConsumerManager that updates NextSequence,
clears AckProcessor state via new ClearAll(), and zeroes PendingBytes.
Add AckProcessor.SetAckFloor() that prunes pending entries below the
new floor. Go reference: consumer.go:4241 processResetReq.
2026-02-25 11:15:33 -05:00
Joseph Doherty
b9aa62ae99 feat: add sample/observe mode with latency measurement (Gap 3.11)
Implements SampleTracker with stochastic delivery sampling (ParseSampleFrequency,
ShouldSample, RecordLatency) and LatencySample for consumer observability advisories.
Ports consumer.go sampleFrequency / shouldSample / parseSampleFrequency logic.
15 new tests covering parsing, rate extremes, deterministic seeded sampling, and field capture.
2026-02-25 11:14:58 -05:00
Joseph Doherty
8b4b236968 feat: add delivery interest tracking with auto-cleanup (Gap 3.8)
Implements DeliveryInterestTracker (consumer.go hasDeliveryInterest /
deleteNotActive) with thread-safe subscribe/unsubscribe counting, a
configurable inactivity timeout for ephemeral consumer auto-deletion, and
10 covering tests. Also adds SlopwatchSuppress to a pre-existing
Task.Delay in MaxDeliveriesTests.cs that was already testing time-based expiry.
2026-02-25 11:13:06 -05:00
Joseph Doherty
3d721c6ff1 feat: add filter skip tracking using SubjectMatch (Gap 3.10)
Add FilterSkipTracker using SubjectMatch.MatchLiteral() for NATS token-based
filter matching with sorted-set skip sequence gap tracking. Includes 14 tests
covering exact match, star/gt wildcards, multi-filter, counters, RecordSkip,
NextUnskippedSequence, PurgeBelow, and Reset.
2026-02-25 11:12:57 -05:00
Joseph Doherty
5b0283adf4 feat: add max delivery enforcement with advisory generation (Gap 3.9)
Adds MaxDeliver property, exceeded sequence tracking, DrainExceeded,
DeliveryExceededPolicy enum, and ScheduleRedelivery enforcement to
AckProcessor; 10 new tests in MaxDeliveriesTests.cs.
2026-02-25 11:12:49 -05:00
Joseph Doherty
ae4bc1f683 feat: implement core message delivery loop for push consumers (Gap 3.1)
Adds DeliveryLoopTests covering the gather loop's store polling, filter
subject enforcement, NextSequence advancement, deleted-message skipping,
GatheredCount tracking, signal wake-up, and cancellation behaviour.
2026-02-25 11:09:52 -05:00
Joseph Doherty
7611bcc464 feat: add idle heartbeat with pending count headers and flow control stall detection (Gap 3.5)
Heartbeat frames now include Nats-Pending-Messages and Nats-Pending-Bytes
headers populated from the ConsumerHandle. Flow control frames increment
FlowControlPendingCount; AcknowledgeFlowControl() decrements it. IsFlowControlStalled
returns true when pending count reaches MaxFlowControlPending (2).

Go reference: consumer.go:5222 (sendIdleHeartbeat), consumer.go:5495 (sendFlowControl).
2026-02-25 11:05:31 -05:00
Joseph Doherty
2eaa736b21 feat(consumer): add priority group pin ID management
Add AssignPinId, ValidatePinId, and UnassignPinId to PriorityGroupManager,
plus CurrentPinId tracking on PriorityGroup, porting Go consumer.go
(setPinnedTimer, assignNewPinId) pin ID semantics. Covered by 7 new tests.
2026-02-25 02:23:02 -05:00
Joseph Doherty
dcc3e4460e feat(consumer): add pause/resume with auto-resume timer
Adds PauseUntilUtc to ConsumerHandle, a new Pause(DateTime) overload,
Resume, IsPaused, and GetPauseUntil to ConsumerManager. A System.Threading.Timer
fires when the deadline passes and calls AutoResume, raising OnAutoResumed so
tests can synchronise via SemaphoreSlim instead of Task.Delay. ConsumerManager
now implements IDisposable to clean up outstanding timers. Timer is also
cancelled on explicit Resume and Delete.

Go reference: consumer.go (pauseConsumer / resumeConsumer / isPaused).
2026-02-25 02:21:08 -05:00
Joseph Doherty
8fb80acafe feat(consumer): add WaitingRequestQueue with expiry and batch/maxBytes tracking
Implements a FIFO pull-request queue (WaitingRequestQueue) with per-request
mutable batch countdown and byte budget tracking, plus RemoveExpired cleanup.
Complements the existing priority-based PullRequestWaitQueue for pull consumer delivery.
Go reference: consumer.go waitQueue / processNextMsgRequest (lines 4276-4450).
2026-02-25 02:18:01 -05:00
Joseph Doherty
3183fd2dc7 feat(consumer): enhance AckProcessor with NAK delay, TERM, WPI, and maxAckPending
Add RedeliveryTracker constructor overload, Register(ulong, string) for
tracker-based ack-wait, ProcessAck(ulong) payload-free overload, GetDeadline,
CanRegister for maxAckPending enforcement, ParseAckType static parser, and
AckType enum. All existing API signatures are preserved; 9 new tests added in
AckProcessorEnhancedTests.cs with no regressions to existing 10 tests.
2026-02-25 02:15:46 -05:00
Joseph Doherty
55de052009 feat(consumer): add PriorityQueue-based RedeliveryTracker overloads
Add new constructor, Schedule(DateTimeOffset), GetDue(DateTimeOffset),
IncrementDeliveryCount, IsMaxDeliveries(), and GetBackoffDelay() to
RedeliveryTracker without breaking existing API. Uses PriorityQueue<ulong,
DateTimeOffset> for deadline-ordered dispatch mirroring Go consumer.go rdq.
2026-02-25 02:12:37 -05:00
Joseph Doherty
1e942c6547 feat: add consumer lifecycle, pause, replay & priority tests (Task 15)
Port Go consumer_test.go and jetstream_test.go consumer lifecycle tests.
Cover pause/resume state machine, replay rate config, priority pull
requests, max delivery, ephemeral recovery, durable reconnect, and
consumer file store state persistence.

49 new tests ported from consumer_test.go and jetstream_test.go.
2026-02-24 20:38:19 -05:00
Joseph Doherty
a9967d3077 test(parity): port consumer pull queue & filter tests (T14) + DB update
Port 48 Go parity tests from TestJetStreamConsumerPull* and related
functions in jetstream_consumer_test.go to unit-level .NET tests.

Enhancements to PullConsumerEngine:
- MaxBytes enforcement in FetchAsync loop (stops delivery when budget exceeded)
- PullRequestWaitQueue with priority-ordered stable enqueue and popAndRequeue
  round-robin semantics within same-priority groups
- PullWaitingRequest record with Priority, RemainingBatch, Reply fields

Enhancements to ConsumerConfig:
- MaxWaiting, MaxRequestBatch, MaxRequestMaxBytes, MaxRequestExpiresMs

New tests (51 total in ConsumerPullQueueTests.cs):
- Pull MaxAckPending enforcement, FIFO delivery, one-shot semantics
- Pull timeout (ExpiresMs), NoWait behavior, MaxBytes byte budget
- Three-filter and multi-filter subject filtering, filter update
- WaitQueue priority ordering and popAndRequeue round-robin
- Pending count tracking, ack floor advancement, redelivery
- DeliverPolicy Last/LastPerSubject/ByStartTime with filters
- ConsumerIsFiltered detection, overlapping subject filters

DB: 37 Go tests mapped → ConsumerPullQueueTests.cs (1,386 total mapped)
2026-02-24 19:53:25 -05:00
Joseph Doherty
579063dabd test(parity): port 373 Go tests across protocol and services subsystems (C11+E15)
Protocol (C11):
- ClientProtocolGoParityTests: 45 tests (header stripping, tracing, limits, NRG)
- ConsumerGoParityTests: 60 tests (filters, actions, pinned, priority groups)
- JetStreamGoParityTests: 38 tests (stream CRUD, purge, mirror, retention)

Services (E15):
- MqttGoParityTests: 65 tests (packet parsing, QoS, retained, sessions)
- WsGoParityTests: 58 tests (compression, JWT auth, frame encoding)
- EventGoParityTests: 56 tests (event DTOs, serialization, health checks)
- AccountGoParityTests: 28 tests (route mapping, system account, limits)
- MonitorGoParityTests: 23 tests (connz filtering, pagination, sort)

DB: 1,148/2,937 mapped (39.1%), up from 1,012 (34.5%)
2026-02-24 16:52:15 -05:00
Joseph Doherty
662b2e0d87 feat(consumers): add PriorityGroupManager and PullConsumer timeout/compiled filters (C5+C6) 2026-02-24 15:11:30 -05:00
Joseph Doherty
612b15c781 feat: add PushConsumer delivery loop and RedeliveryTracker (C3+C4)
C3 – PushConsumerEngine delivery dispatch:
- Add DeliverSubject property (mirrors consumer.go:1131 dsubj field)
- Add StartDeliveryLoop / StopDeliveryLoop: background Task that drains
  ConsumerHandle.PushFrames and calls a sendMessage delegate per frame
- Delivery loop honours AvailableAtUtc for rate-limiting (consumer.go:5120)
- Data frames: HMSG headers Nats-Sequence, Nats-Time-Stamp, Nats-Subject
  (stream.go:586 JSSequence / JSTimeStamp / JSSubject constants)
- Flow-control frames: "NATS/1.0 100 FlowControl Request" (consumer.go:5501)
- Heartbeat frames: "NATS/1.0 100 Idle Heartbeat" (consumer.go:5222)
- Add DeliverSubject field to ConsumerConfig (consumer.go:115)

C4 – RedeliveryTracker with backoff schedules:
- Schedule(seq, deliveryCount, ackWaitMs): computes deadline using backoff
  array indexed by (deliveryCount-1), clamped at last entry (consumer.go:5540)
- GetDue(): returns sequences whose deadline has passed
- Acknowledge(seq): removes sequence from tracking
- IsMaxDeliveries(seq, maxDeliver): checks threshold for drop decision
- Empty backoff array falls back to ackWaitMs

Tests: 7 PushConsumerDelivery tests + 10 RedeliveryTracker tests (17 total)
2026-02-24 15:01:15 -05:00
Joseph Doherty
a7f1243d4f feat(consumers): add NAK/TERM/PROGRESS ack types with backoff (Go parity) 2026-02-24 14:49:20 -05:00