dohertj2/natsnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
c0ec1f3341b2b821e517d94c802917841d092384
natsnet/docs/plans/2026-02-27-batch-38-consumer-lifecycle-design.md
Joseph Doherty 8a126c4932 Add batch plans for batches 37-41 (rounds 19-21) 
Generated design docs and implementation plans via Codex for:
- Batch 37: Stream Messages
- Batch 38: Consumer Lifecycle
- Batch 39: Consumer Dispatch
- Batch 40: MQTT Server/JSA
- Batch 41: MQTT Client/IO

All plans include mandatory verification protocol and anti-stub guardrails.
Updated batches.md with file paths and planned status.
All 42 batches (0-41) now have design docs and implementation plans.
2026-02-27 17:27:51 -05:00

8.2 KiB
Raw Blame History

Batch 38 Consumer Lifecycle Design

Date: 2026-02-27
Batch: 38 (Consumer Lifecycle)
Scope: 96 features + 71 unit tests
Dependencies: Batches 34, 36
Go source: golang/nats-server/server/consumer.go (primary focus lines ~176-3761 for mapped features)

Problem

Batch 38 carries the JetStream consumer control plane and lifecycle core: enum/config normalization, consumer creation/update paths, advisories, delivery-interest checks, ack processing, pending request tracking, store-state persistence, info snapshots, sampling, filtering, and wait-queue mechanics.

The current .NET surface is far smaller than the mapped scope (NatsConsumer.cs is 246 lines vs. consumer.go at 6715 lines). If this batch is advanced with placeholders, Batch 39 (Consumer Dispatch) will inherit false-green behavior and brittle semantics.

Context Findings

Required command outputs

  • /usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch show 38 --db porting.db
    • Status: pending
    • Features: 96 (IDs 584-684 with mapped gaps)
    • Tests: 71
    • Depends on: 34,36
    • Go file: server/consumer.go
  • /usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch list --db porting.db
    • Confirms dependency chain includes 34/36 -> 38 -> 39
  • /usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db
    • Overall progress: 1924/6942 (27.7%)

Environment note: dotnet is not on shell PATH; use /usr/local/share/dotnet/dotnet.

Dependency readiness

  • batch ready currently does not include Batch 38.
  • Dependencies (34, 36) are still pending and must be complete before Batch 38 starts.

Feature ownership split (from porting.db)

  • NatsConsumer: 77
  • ConsumerAction: 3
  • PriorityPolicy: 3
  • NatsStream: 3
  • WaitingRequest: 3
  • DeliverPolicy: 1
  • AckPolicy: 1
  • ReplayPolicy: 1
  • SubjectTokens: 1
  • NatsServer: 1
  • Account: 1
  • JsPubMsg: 1

Test ownership split (from porting.db)

  • NatsConsumerTests: 38
  • JetStreamEngineTests: 20
  • JetStreamClusterTests1: 3
  • JetStreamClusterTests3: 3
  • JetStreamClusterTests4: 2
  • ConcurrencyTests1: 2
  • AccountTests: 1
  • JetStreamBenchmarks: 1
  • MqttHandlerTests: 1

Additional findings:

  • dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.cs currently exposes only a small subset of mapped methods.
  • NatsStream does not currently expose mapped AddConsumer* methods.
  • Account.CheckNewConsumerConfig and NatsServer.HasGatewayInterest are not present.
  • JetStreamClusterTests1/3/4.Impltests.cs and JetStreamBenchmarks.Impltests.cs do not currently exist.
  • Existing ImplBacklog files for this scope heavily use placeholder patterns (var goFile = ..., literal-string assertions), requiring replacement with behavioral tests.

Approaches

Approach A: Expand a single NatsConsumer.cs monolith

  • Pros: fewer files.
  • Cons: weak review boundaries, high merge conflict risk, difficult to validate incrementally across 96 features.

Approach B (Recommended): Partial-class decomposition by lifecycle domain + strict verification gates

  • Pros: bounded feature groups, cleaner method ownership, easier per-group verification and anti-stub enforcement.
  • Cons: requires file reorganization and additional partial files.

Approach C: Port all mapped tests first, then backfill production code

  • Pros: forces behavior-driven progress.
  • Cons: too noisy early because many mapped APIs/types do not yet exist; red phase would be broad and low-signal.

Decision: Approach B.

Proposed Design

1. Code organization strategy

Use partial decomposition for consumer-heavy logic while keeping mapped ownership:

  • Modify: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.cs (core state, constructor, shared locking)
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Config.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Lifecycle.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Advisories.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Acks.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.State.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.WaitQueue.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Consumers.cs (AddConsumer*)
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StreamTypes.ConsumerLifecycle.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StoreTypes.ConsumerPolicies.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.ConsumerConfig.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.ConsumerInterest.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/Internal/SubjectTokens.ConsumerFilters.cs

Design intent: keep method surfaces aligned to mapped dotnet_class + dotnet_method while avoiding one large file.

2. Functional decomposition

  • Enums and config validation: consumer actions/policies string+JSON mapping, defaults, config checks.
  • Creation and lifecycle wiring: AddConsumer*, assignment, monitor channels, leadership checks.
  • Advisories and delivery-interest: internal subscribe/unsubscribe, advisory publication, gateway interest integration.
  • Ack and pending-request flow: ack reply message parsing/building, processAck/Nak/Term, redelivery and pending-request transitions.
  • Store state and info path: read/apply/write state, info snapshots, sampling, filter checks, pull request parsing, wait-queue recycling.

3. Test strategy

Port mapped tests by class waves and remove template assertions:

  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs (38)
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs (20)
  • Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs (3)
  • Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs (3)
  • Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests4.Impltests.cs (2)
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs (2)
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AccountTests.Impltests.cs (1)
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs (1)
  • Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamBenchmarks.Impltests.cs (1; may remain deferred if benchmark-only)

4. Verification architecture

Adopt Batch-0-style hard verification controls, adapted for feature + test work:

  • Mandatory per-feature loop (evidence required before status promotion).
  • Mandatory per-test loop (single-test and class-level pass evidence).
  • Stub detection required after each loop.
  • Build + targeted test gates required before each status update.
  • Status updates chunked to max 15 IDs per batch-update.
  • Mandatory checkpoint between every task.
  • Explicit deferred handling with reason when blocked; no placeholder stubs.

Feature Grouping (for implementation plan)

  • Group A (19): 584,585,586,587,588,589,590,591,592,594,595,596,597,598,599,600,601,602,603
  • Group B (19): 604,605,606,607,608,610,612,613,614,615,616,617,618,619,620,621,622,623,624
  • Group C (19): 625,626,627,629,630,631,632,633,635,636,637,638,639,640,641,642,643,644,645
  • Group D (19): 646,647,648,649,650,651,652,653,654,655,656,657,658,659,660,661,662,663,664
  • Group E (20): 665,666,667,668,669,670,671,672,673,674,675,676,677,678,679,680,681,682,683,684

Constraints

  • Planning only in this session; no implementation execution.
  • Must follow docs/standards/dotnet-standards.md (xUnit 3 + Shouldly + NSubstitute, nullable, naming conventions).
  • Batch 38 implementation must start only after Batches 34 and 36 are complete/ready.

Non-Goals

  • Executing Batch 38 code changes in this session.
  • Re-scoping features/tests outside mapped Batch 38 IDs.
  • Fabricating benchmark/integration parity via fake-pass unit-test templates.
Reference in New Issue View Git Blame Copy Permalink