Add batch plans for batches 37-41 (rounds 19-21)
Generated design docs and implementation plans via Codex for: - Batch 37: Stream Messages - Batch 38: Consumer Lifecycle - Batch 39: Consumer Dispatch - Batch 40: MQTT Server/JSA - Batch 41: MQTT Client/IO All plans include mandatory verification protocol and anti-stub guardrails. Updated batches.md with file paths and planned status. All 42 batches (0-41) now have design docs and implementation plans.
This commit is contained in:
Joseph Doherty
165
docs/plans/2026-02-27-batch-38-consumer-lifecycle-design.md
Normal file
165
docs/plans/2026-02-27-batch-38-consumer-lifecycle-design.md
Normal file
@@ -0,0 +1,165 @@
|
||||
# Batch 38 Consumer Lifecycle Design
|
||||
|
||||
**Date:** 2026-02-27
|
||||
**Batch:** 38 (`Consumer Lifecycle`)
|
||||
**Scope:** 96 features + 71 unit tests
|
||||
**Dependencies:** Batches `34`, `36`
|
||||
**Go source:** `golang/nats-server/server/consumer.go` (primary focus lines ~176-3761 for mapped features)
|
||||
|
||||
## Problem
|
||||
|
||||
Batch 38 carries the JetStream consumer control plane and lifecycle core: enum/config normalization, consumer creation/update paths, advisories, delivery-interest checks, ack processing, pending request tracking, store-state persistence, info snapshots, sampling, filtering, and wait-queue mechanics.
|
||||
|
||||
The current .NET surface is far smaller than the mapped scope (`NatsConsumer.cs` is 246 lines vs. `consumer.go` at 6715 lines). If this batch is advanced with placeholders, Batch 39 (`Consumer Dispatch`) will inherit false-green behavior and brittle semantics.
|
||||
|
||||
## Context Findings
|
||||
|
||||
### Required command outputs
|
||||
|
||||
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch show 38 --db porting.db`
|
||||
- Status: `pending`
|
||||
- Features: `96` (IDs `584-684` with mapped gaps)
|
||||
- Tests: `71`
|
||||
- Depends on: `34,36`
|
||||
- Go file: `server/consumer.go`
|
||||
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch list --db porting.db`
|
||||
- Confirms dependency chain includes `34/36 -> 38 -> 39`
|
||||
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db`
|
||||
- Overall progress: `1924/6942 (27.7%)`
|
||||
|
||||
Environment note: `dotnet` is not on shell `PATH`; use `/usr/local/share/dotnet/dotnet`.
|
||||
|
||||
### Dependency readiness
|
||||
|
||||
- `batch ready` currently does **not** include Batch 38.
|
||||
- Dependencies (`34`, `36`) are still `pending` and must be complete before Batch 38 starts.
|
||||
|
||||
### Feature ownership split (from `porting.db`)
|
||||
|
||||
- `NatsConsumer`: 77
|
||||
- `ConsumerAction`: 3
|
||||
- `PriorityPolicy`: 3
|
||||
- `NatsStream`: 3
|
||||
- `WaitingRequest`: 3
|
||||
- `DeliverPolicy`: 1
|
||||
- `AckPolicy`: 1
|
||||
- `ReplayPolicy`: 1
|
||||
- `SubjectTokens`: 1
|
||||
- `NatsServer`: 1
|
||||
- `Account`: 1
|
||||
- `JsPubMsg`: 1
|
||||
|
||||
### Test ownership split (from `porting.db`)
|
||||
|
||||
- `NatsConsumerTests`: 38
|
||||
- `JetStreamEngineTests`: 20
|
||||
- `JetStreamClusterTests1`: 3
|
||||
- `JetStreamClusterTests3`: 3
|
||||
- `JetStreamClusterTests4`: 2
|
||||
- `ConcurrencyTests1`: 2
|
||||
- `AccountTests`: 1
|
||||
- `JetStreamBenchmarks`: 1
|
||||
- `MqttHandlerTests`: 1
|
||||
|
||||
Additional findings:
|
||||
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.cs` currently exposes only a small subset of mapped methods.
|
||||
- `NatsStream` does not currently expose mapped `AddConsumer*` methods.
|
||||
- `Account.CheckNewConsumerConfig` and `NatsServer.HasGatewayInterest` are not present.
|
||||
- `JetStreamClusterTests1/3/4.Impltests.cs` and `JetStreamBenchmarks.Impltests.cs` do not currently exist.
|
||||
- Existing `ImplBacklog` files for this scope heavily use placeholder patterns (`var goFile = ...`, literal-string assertions), requiring replacement with behavioral tests.
|
||||
|
||||
## Approaches
|
||||
|
||||
### Approach A: Expand a single `NatsConsumer.cs` monolith
|
||||
|
||||
- Pros: fewer files.
|
||||
- Cons: weak review boundaries, high merge conflict risk, difficult to validate incrementally across 96 features.
|
||||
|
||||
### Approach B (Recommended): Partial-class decomposition by lifecycle domain + strict verification gates
|
||||
|
||||
- Pros: bounded feature groups, cleaner method ownership, easier per-group verification and anti-stub enforcement.
|
||||
- Cons: requires file reorganization and additional partial files.
|
||||
|
||||
### Approach C: Port all mapped tests first, then backfill production code
|
||||
|
||||
- Pros: forces behavior-driven progress.
|
||||
- Cons: too noisy early because many mapped APIs/types do not yet exist; red phase would be broad and low-signal.
|
||||
|
||||
**Decision:** Approach B.
|
||||
|
||||
## Proposed Design
|
||||
|
||||
### 1. Code organization strategy
|
||||
|
||||
Use partial decomposition for consumer-heavy logic while keeping mapped ownership:
|
||||
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.cs` (core state, constructor, shared locking)
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Config.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Lifecycle.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Advisories.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Acks.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.State.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.WaitQueue.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Consumers.cs` (`AddConsumer*`)
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StreamTypes.ConsumerLifecycle.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StoreTypes.ConsumerPolicies.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.ConsumerConfig.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.ConsumerInterest.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/Internal/SubjectTokens.ConsumerFilters.cs`
|
||||
|
||||
Design intent: keep method surfaces aligned to mapped `dotnet_class` + `dotnet_method` while avoiding one large file.
|
||||
|
||||
### 2. Functional decomposition
|
||||
|
||||
- **Enums and config validation:** consumer actions/policies string+JSON mapping, defaults, config checks.
|
||||
- **Creation and lifecycle wiring:** `AddConsumer*`, assignment, monitor channels, leadership checks.
|
||||
- **Advisories and delivery-interest:** internal subscribe/unsubscribe, advisory publication, gateway interest integration.
|
||||
- **Ack and pending-request flow:** ack reply message parsing/building, `processAck/Nak/Term`, redelivery and pending-request transitions.
|
||||
- **Store state and info path:** read/apply/write state, info snapshots, sampling, filter checks, pull request parsing, wait-queue recycling.
|
||||
|
||||
### 3. Test strategy
|
||||
|
||||
Port mapped tests by class waves and remove template assertions:
|
||||
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs` (38)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs` (20)
|
||||
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs` (3)
|
||||
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs` (3)
|
||||
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests4.Impltests.cs` (2)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs` (2)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AccountTests.Impltests.cs` (1)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs` (1)
|
||||
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamBenchmarks.Impltests.cs` (1; may remain deferred if benchmark-only)
|
||||
|
||||
### 4. Verification architecture
|
||||
|
||||
Adopt Batch-0-style hard verification controls, adapted for feature + test work:
|
||||
|
||||
- Mandatory per-feature loop (evidence required before status promotion).
|
||||
- Mandatory per-test loop (single-test and class-level pass evidence).
|
||||
- Stub detection required after each loop.
|
||||
- Build + targeted test gates required before each status update.
|
||||
- Status updates chunked to max 15 IDs per `batch-update`.
|
||||
- Mandatory checkpoint between every task.
|
||||
- Explicit deferred handling with reason when blocked; no placeholder stubs.
|
||||
|
||||
## Feature Grouping (for implementation plan)
|
||||
|
||||
- Group A (19): `584,585,586,587,588,589,590,591,592,594,595,596,597,598,599,600,601,602,603`
|
||||
- Group B (19): `604,605,606,607,608,610,612,613,614,615,616,617,618,619,620,621,622,623,624`
|
||||
- Group C (19): `625,626,627,629,630,631,632,633,635,636,637,638,639,640,641,642,643,644,645`
|
||||
- Group D (19): `646,647,648,649,650,651,652,653,654,655,656,657,658,659,660,661,662,663,664`
|
||||
- Group E (20): `665,666,667,668,669,670,671,672,673,674,675,676,677,678,679,680,681,682,683,684`
|
||||
|
||||
## Constraints
|
||||
|
||||
- Planning only in this session; no implementation execution.
|
||||
- Must follow `docs/standards/dotnet-standards.md` (xUnit 3 + Shouldly + NSubstitute, nullable, naming conventions).
|
||||
- Batch 38 implementation must start only after Batches 34 and 36 are complete/ready.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Executing Batch 38 code changes in this session.
|
||||
- Re-scoping features/tests outside mapped Batch 38 IDs.
|
||||
- Fabricating benchmark/integration parity via fake-pass unit-test templates.
|
||||
Reference in New Issue
Block a user