dohertj2/natsnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
1916eab00a0f3b5154b0a96a0b525e2a43d7e8a2
natsnet/docs/plans/2026-02-27-batch-38-consumer-lifecycle-design.md
Joseph Doherty 8a126c4932 Add batch plans for batches 37-41 (rounds 19-21) 
Generated design docs and implementation plans via Codex for:
- Batch 37: Stream Messages
- Batch 38: Consumer Lifecycle
- Batch 39: Consumer Dispatch
- Batch 40: MQTT Server/JSA
- Batch 41: MQTT Client/IO

All plans include mandatory verification protocol and anti-stub guardrails.
Updated batches.md with file paths and planned status.
All 42 batches (0-41) now have design docs and implementation plans.
2026-02-27 17:27:51 -05:00

166 lines
8.2 KiB
Markdown
Raw Blame History

# Batch 38 Consumer Lifecycle Design
**Date:** 2026-02-27
**Batch:** 38 (`Consumer Lifecycle`)
**Scope:** 96 features + 71 unit tests
**Dependencies:** Batches `34`, `36`
**Go source:** `golang/nats-server/server/consumer.go` (primary focus lines ~176-3761 for mapped features)
## Problem
Batch 38 carries the JetStream consumer control plane and lifecycle core: enum/config normalization, consumer creation/update paths, advisories, delivery-interest checks, ack processing, pending request tracking, store-state persistence, info snapshots, sampling, filtering, and wait-queue mechanics.
The current .NET surface is far smaller than the mapped scope (`NatsConsumer.cs` is 246 lines vs. `consumer.go` at 6715 lines). If this batch is advanced with placeholders, Batch 39 (`Consumer Dispatch`) will inherit false-green behavior and brittle semantics.
## Context Findings
### Required command outputs
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch show 38 --db porting.db`
- Status: `pending`
- Features: `96` (IDs `584-684` with mapped gaps)
- Tests: `71`
- Depends on: `34,36`
- Go file: `server/consumer.go`
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch list --db porting.db`
- Confirms dependency chain includes `34/36 -> 38 -> 39`
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db`
- Overall progress: `1924/6942 (27.7%)`
Environment note: `dotnet` is not on shell `PATH`; use `/usr/local/share/dotnet/dotnet`.
### Dependency readiness
- `batch ready` currently does **not** include Batch 38.
- Dependencies (`34`, `36`) are still `pending` and must be complete before Batch 38 starts.
### Feature ownership split (from `porting.db`)
- `NatsConsumer`: 77
- `ConsumerAction`: 3
- `PriorityPolicy`: 3
- `NatsStream`: 3
- `WaitingRequest`: 3
- `DeliverPolicy`: 1
- `AckPolicy`: 1
- `ReplayPolicy`: 1
- `SubjectTokens`: 1
- `NatsServer`: 1
- `Account`: 1
- `JsPubMsg`: 1
### Test ownership split (from `porting.db`)
- `NatsConsumerTests`: 38
- `JetStreamEngineTests`: 20
- `JetStreamClusterTests1`: 3
- `JetStreamClusterTests3`: 3
- `JetStreamClusterTests4`: 2
- `ConcurrencyTests1`: 2
- `AccountTests`: 1
- `JetStreamBenchmarks`: 1
- `MqttHandlerTests`: 1
Additional findings:
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.cs` currently exposes only a small subset of mapped methods.
- `NatsStream` does not currently expose mapped `AddConsumer*` methods.
- `Account.CheckNewConsumerConfig` and `NatsServer.HasGatewayInterest` are not present.
- `JetStreamClusterTests1/3/4.Impltests.cs` and `JetStreamBenchmarks.Impltests.cs` do not currently exist.
- Existing `ImplBacklog` files for this scope heavily use placeholder patterns (`var goFile = ...`, literal-string assertions), requiring replacement with behavioral tests.
## Approaches
### Approach A: Expand a single `NatsConsumer.cs` monolith
- Pros: fewer files.
- Cons: weak review boundaries, high merge conflict risk, difficult to validate incrementally across 96 features.
### Approach B (Recommended): Partial-class decomposition by lifecycle domain + strict verification gates
- Pros: bounded feature groups, cleaner method ownership, easier per-group verification and anti-stub enforcement.
- Cons: requires file reorganization and additional partial files.
### Approach C: Port all mapped tests first, then backfill production code
- Pros: forces behavior-driven progress.
- Cons: too noisy early because many mapped APIs/types do not yet exist; red phase would be broad and low-signal.
**Decision:** Approach B.
## Proposed Design
### 1. Code organization strategy
Use partial decomposition for consumer-heavy logic while keeping mapped ownership:
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.cs` (core state, constructor, shared locking)
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Config.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Lifecycle.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Advisories.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Acks.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.State.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.WaitQueue.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Consumers.cs` (`AddConsumer*`)
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StreamTypes.ConsumerLifecycle.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StoreTypes.ConsumerPolicies.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.ConsumerConfig.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.ConsumerInterest.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/Internal/SubjectTokens.ConsumerFilters.cs`
Design intent: keep method surfaces aligned to mapped `dotnet_class` + `dotnet_method` while avoiding one large file.
### 2. Functional decomposition
- **Enums and config validation:** consumer actions/policies string+JSON mapping, defaults, config checks.
- **Creation and lifecycle wiring:** `AddConsumer*`, assignment, monitor channels, leadership checks.
- **Advisories and delivery-interest:** internal subscribe/unsubscribe, advisory publication, gateway interest integration.
- **Ack and pending-request flow:** ack reply message parsing/building, `processAck/Nak/Term`, redelivery and pending-request transitions.
- **Store state and info path:** read/apply/write state, info snapshots, sampling, filter checks, pull request parsing, wait-queue recycling.
### 3. Test strategy
Port mapped tests by class waves and remove template assertions:
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs` (38)
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs` (20)
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs` (3)
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs` (3)
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests4.Impltests.cs` (2)
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs` (2)
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AccountTests.Impltests.cs` (1)
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs` (1)
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamBenchmarks.Impltests.cs` (1; may remain deferred if benchmark-only)
### 4. Verification architecture
Adopt Batch-0-style hard verification controls, adapted for feature + test work:
- Mandatory per-feature loop (evidence required before status promotion).
- Mandatory per-test loop (single-test and class-level pass evidence).
- Stub detection required after each loop.
- Build + targeted test gates required before each status update.
- Status updates chunked to max 15 IDs per `batch-update`.
- Mandatory checkpoint between every task.
- Explicit deferred handling with reason when blocked; no placeholder stubs.
## Feature Grouping (for implementation plan)
- Group A (19): `584,585,586,587,588,589,590,591,592,594,595,596,597,598,599,600,601,602,603`
- Group B (19): `604,605,606,607,608,610,612,613,614,615,616,617,618,619,620,621,622,623,624`
- Group C (19): `625,626,627,629,630,631,632,633,635,636,637,638,639,640,641,642,643,644,645`
- Group D (19): `646,647,648,649,650,651,652,653,654,655,656,657,658,659,660,661,662,663,664`
- Group E (20): `665,666,667,668,669,670,671,672,673,674,675,676,677,678,679,680,681,682,683,684`
## Constraints
- Planning only in this session; no implementation execution.
- Must follow `docs/standards/dotnet-standards.md` (xUnit 3 + Shouldly + NSubstitute, nullable, naming conventions).
- Batch 38 implementation must start only after Batches 34 and 36 are complete/ready.
## Non-Goals
- Executing Batch 38 code changes in this session.
- Re-scoping features/tests outside mapped Batch 38 IDs.
- Fabricating benchmark/integration parity via fake-pass unit-test templates.
Reference in New Issue View Git Blame Copy Permalink