natsnet/docs/plans/2026-02-27-batch-39-consumer-dispatch-design.md

# Batch 39 Consumer Dispatch Design

**Date:** 2026-02-27  
**Batch:** 39 (`Consumer Dispatch`)  
**Scope:** 93 features + 53 unit tests  
**Dependencies:** Batch `38`  
**Go source:** `golang/nats-server/server/consumer.go` (dispatch-heavy region around lines 3925-6706)

## Problem

Batch 39 contains the JetStream consumer dispatch/data-plane logic: pull request queueing, next-message request parsing, pending accounting, heartbeats/flow-control, delivery tracking, redelivery queue semantics, ack reply parsing, stream purge/stop paths, and monitor/signal flow.

If this batch is advanced with placeholders, the runtime behavior for pull consumers, redelivery correctness, and no-interest cleanup will look green in PortTracker while remaining behaviorally incomplete.

## Context Findings

### Required command outputs

- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch show 39 --db porting.db`
  - Status: `pending`
  - Features: `93` (all currently `deferred`)
  - Tests: `53` (all currently `deferred`)
  - Depends on: `38`
  - Go file: `server/consumer.go`
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch list --db porting.db`
  - Confirms dependency chain `38 -> 39`
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db`
  - Overall progress snapshot: `1924/6942 (27.7%)`

Environment note: `dotnet` is not on `PATH` in this shell; use `/usr/local/share/dotnet/dotnet`.

### Batch ownership split

Features by class:

- `NatsConsumer`: 91
- `NextMsgReq`: 1
- `NatsStream`: 1

Tests by class:

- `JetStreamEngineTests`: 34
- `NatsConsumerTests`: 14
- `ConcurrencyTests1`: 2
- `JetStreamClusterTests1`: 1
- `JwtProcessorTests`: 1
- `RouteHandlerTests`: 1

### Current .NET baseline findings

- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.cs` is still a compact lifecycle shell and does not expose most Batch 39 mapped methods.
- `WaitQueue` and `WaitingRequest` exist in `StreamTypes.cs`, but only part of the Go dispatch behavior is represented.
- `ImplBacklog` files for Batch 39-related tests are mostly template placeholders and need full behavioral ports.

## Approach Options

### Approach A: Keep all dispatch work in `NatsConsumer.cs`

- Pros: fewer files and simpler discovery.
- Cons: 90+ methods in one file, poor reviewability, high merge risk, and weaker checkpoint isolation.

### Approach B (Recommended): Dispatch-focused partial decomposition for `NatsConsumer`

- Pros: aligns with Go dispatch domains, improves review boundaries, supports group-by-group verification loops.
- Cons: more files and up-front organization overhead.

### Approach C: Port all 53 tests first, then implement features

- Pros: strict red/green pressure.
- Cons: too many missing APIs initially, broad low-signal red phase, high churn in test scaffolding.

**Decision:** Approach B.

## Proposed Design

### 1. Code Organization

Keep `NatsConsumer` as the central type but split dispatch concerns into partial files:

- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.cs` (core state, shared lock helpers, partial type anchor)
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Dispatch.Waiting.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Dispatch.Pull.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Dispatch.Acks.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Dispatch.Delivery.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Dispatch.Redelivery.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Dispatch.ReplyParsing.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Dispatch.Shutdown.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Consumers.cs` (`DeleteConsumer` mapping)
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StreamTypes.cs` (`WaitQueue`, `WaitingRequest`, `PriorityGroup` helpers)

### 2. Functional Decomposition

- **Waiting queue and request ingress:** pending request map, pin assignment/ttl, request parsing (`ProcessNextMsgReq`, `ProcessResetReq`, `ProcessNextMsgRequest`).
- **Dispatch loop and pending accounting:** `GetNextMsg`, `ProcessWaiting`, `CheckAckFloor`, `LoopAndGatherMsgs`, pending counters.
- **Delivery and flow control:** heartbeat emission, flow-control reply generation, message delivery envelope conversion.
- **Redelivery and ack reply parsing:** pending tracking, redelivery queue operations, ack subject parsing (`ReplyInfo`, `AckReplyInfo`, seq extraction).
- **Lifecycle and stream-interest cleanup:** durable/push/pull helpers, purge/stop/cleanup, stream signals, monitor guard methods.

### 3. Test Design

Replace template tests with behavioral tests tied to mapped IDs:

- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs` (14)
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs` (34)
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs` (2)
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JwtProcessorTests.Impltests.cs` (1)
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RouteHandlerTests.Impltests.cs` (1)
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs` (1)

Also add/expand focused non-backlog unit tests where internal behaviors can be tested deterministically:

- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/NatsConsumerDispatchTests.cs`
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/NatsConsumerTests.cs`

### 4. Verification Model (Design-level)

Batch 39 execution must be evidence-driven:

- Per-feature implementation loop with targeted tests.
- Per-test single-test and class-level verification.
- Stub scan before any status promotion.
- Build and test gates between groups.
- Status updates in max 15-ID chunks.
- Deferred-with-reason instead of stubbing when blocked.

## Feature Groups (max ~20 each)

- **Group A (18): waiting queue + pull request ingress**  
  `696,699,700,701,702,703,704,705,706,707,708,709,710,711,712,713,714,715`
- **Group B (18): dispatch selection + ack floor + inbound loops**  
  `716,717,718,719,720,721,722,723,724,725,726,727,728,729,730,731,732,733`
- **Group C (19): delivery payload + flow control + redelivery foundation**  
  `734,735,736,737,738,739,740,741,742,743,744,745,746,747,748,749,750,751,752`
- **Group D (19): reply parsing + sequence selection + consumer identity helpers**  
  `753,754,755,756,757,758,759,760,761,762,763,764,765,766,767,768,769,770,772`
- **Group E (19): stop/purge/no-interest/monitor signaling path**  
  `774,775,776,777,778,779,780,781,782,783,784,785,786,787,788,789,790,791,792`

## Test Waves

- **Wave T1 (14):** `1230,1232,1251,1261,1265,1267,1273,1277,1283,1284,1285,1286,1339,1370`
- **Wave T2 (17):** `1469,1484,1485,1486,1487,1488,1489,1490,1492,1493,1495,1496,1498,1499,1500,1501,1502`
- **Wave T3 (17):** `1508,1514,1515,1516,1517,1518,1519,1520,1521,1522,1526,1530,1531,1545,1547,1567,1665`
- **Wave T4 (5):** `814,1840,2389,2407,2858`

## Risks and Mitigations

- **Risk:** dispatch loops can mask bugs via timing-sensitive behavior.  
  **Mitigation:** isolate pure logic from timers/network side-effects; unit-test selectors and parsers first.
- **Risk:** placeholder tests could slip through due green-but-empty assertions.  
  **Mitigation:** mandatory stub scan and assertion-quality checks before status updates.
- **Risk:** over-bulk status updates can misclassify unverified IDs.  
  **Mitigation:** enforce max 15 IDs per update and per-task checkpoints.

## Non-Goals

- Executing Batch 39 implementation in this session.
- Re-scoping IDs outside Batch 39.
- Marking any feature/test as verified during planning.