dohertj2/natsnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
03371eb1949c93e1449fce6f862e4acafa83b833
natsnet/docs/plans/2026-02-27-batch-38-consumer-lifecycle-implementation-plan.md
Joseph Doherty 8a126c4932 Add batch plans for batches 37-41 (rounds 19-21) 
Generated design docs and implementation plans via Codex for:
- Batch 37: Stream Messages
- Batch 38: Consumer Lifecycle
- Batch 39: Consumer Dispatch
- Batch 40: MQTT Server/JSA
- Batch 41: MQTT Client/IO

All plans include mandatory verification protocol and anti-stub guardrails.
Updated batches.md with file paths and planned status.
All 42 batches (0-41) now have design docs and implementation plans.
2026-02-27 17:27:51 -05:00

19 KiB
Raw Blame History

Batch 38 Consumer Lifecycle Implementation Plan

For Codex: REQUIRED SUB-SKILL: Use executeplan to implement this plan task-by-task.

Goal: Port and verify Batch 38 (Consumer Lifecycle) from server/consumer.go so all 96 mapped features and 71 mapped tests are either genuinely implemented/verified or explicitly deferred with concrete blocker notes.

Architecture: Implement Batch 38 in five feature groups (19/19/19/19/20) using NatsConsumer partial decomposition plus supporting updates in NatsStream, StoreTypes, StreamTypes, Account, NatsServer, and SubjectTokens. Execute five test waves with strict per-feature/per-test evidence loops, mandatory stub detection, and checkpoint gates before status promotion.

Tech Stack: .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (porting.db)

Design doc: docs/plans/2026-02-27-batch-38-consumer-lifecycle-design.md

Batch 38 Scope

  • Batch ID: 38
  • Name: Consumer Lifecycle
  • Dependencies: 34,36
  • Go source: golang/nats-server/server/consumer.go
  • Features: 96 (584-684 mapped IDs)
  • Tests: 71

If dotnet is not on PATH, use:

DOTNET=/usr/local/share/dotnet/dotnet

Primary production files (expected):

  • Modify: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Config.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Lifecycle.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Advisories.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Acks.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.State.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.WaitQueue.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Consumers.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StreamTypes.ConsumerLifecycle.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StoreTypes.ConsumerPolicies.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.ConsumerConfig.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.ConsumerInterest.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/Internal/SubjectTokens.ConsumerFilters.cs

Primary test files (expected):

  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/NatsConsumerTests.cs
  • Modify/Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/ConsumerPoliciesTests.cs
  • Modify/Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/ConsumerStateTests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs
  • Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs
  • Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs
  • Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests4.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AccountTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs
  • Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamBenchmarks.Impltests.cs

MANDATORY VERIFICATION PROTOCOL

NON-NEGOTIABLE: Every Batch 38 feature and test must pass this loop. No shortcut updates.

Dependency Preflight (before any status change)

$DOTNET run --project tools/NatsNet.PortTracker -- batch show 34 --db porting.db
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 36 --db porting.db
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 38 --db porting.db
$DOTNET run --project tools/NatsNet.PortTracker -- batch ready --db porting.db

Start only when Batch 38 is ready:

$DOTNET run --project tools/NatsNet.PortTracker -- batch start 38 --db porting.db

Capture baseline:

$DOTNET build dotnet/
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/

What Counts as a Real Port

A feature is real only if all are true:

  1. Mapped method/type exists with behavioral logic (no placeholder body).
  2. Logic consumes real inputs/state (not constants that bypass behavior).
  3. At least one behavioral test exercises the path.
  4. Stub scan and gates are clean.

A test is real only if all are true:

  1. Has Arrange/Act/Assert.
  2. Calls production code in ZB.MOM.NatsNet.Server.*.
  3. Assertions validate behavior derived from Act output/effects.
  4. Contains no template-stub patterns.

Per-Feature Verification Loop (REQUIRED per feature ID)

  1. Inspect mapping and Go context:
$DOTNET run --project tools/NatsNet.PortTracker -- feature show <feature_id> --db porting.db
  1. Claim feature:
$DOTNET run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status stub --db porting.db
  1. Add/adjust focused behavioral test(s) for that feature.
  2. Implement minimal real logic for the mapped method/type.
  3. Run Stub Detection Check.
  4. Run Build Gate.
  5. Run targeted Test Gate for touched classes.
  6. Promote proven feature to complete (or verified only after checkpoint evidence).

Per-Test Verification Loop (REQUIRED per test ID)

  1. Inspect mapping and Go test source:
$DOTNET run --project tools/NatsNet.PortTracker -- test show <test_id> --db porting.db
  1. Claim test:
$DOTNET run --project tools/NatsNet.PortTracker -- test update <test_id> --status stub --db porting.db
  1. Port full behavior (no placeholders).
  2. Run single test:
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~<ClassName>.<MethodName>" --verbosity normal
  1. Confirm Passed: 1, Failed: 0 (never Passed: 0).
  2. Run class-level filter for touched class.
  3. Run Stub Detection Check + Build Gate.
  4. Promote proven test to complete (or verified only after checkpoint evidence).

Stub Detection Check (REQUIRED after every feature/test loop)

changed_files=$(git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests | rg "\\.cs$" || true)
if [ -n "$changed_files" ]; then
  echo "$changed_files" | xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|Assert\\.True\\(true\\)|Assert\\.Pass\\(\\)|var goFile = \"server/|\\.ShouldContain\\(\"Should\"\\)|GetRequiredApiLevel\\(new Dictionary<string, string>\\)\\.ShouldBe\\(string\\.Empty\\)|=>\\s*default;|return\\s+default;|return\\s+null;\\s*$|throw new Exception\\(\"TODO\")"
fi

Any match blocks status promotion until fixed or explicitly deferred.

Build Gate (REQUIRED)

$DOTNET build dotnet/

Run after each feature loop, each test loop, before each batch-update, and at each checkpoint.

Test Gate (REQUIRED)

Minimum targeted gates per touched domain:

$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~JetStream.NatsConsumerTests|FullyQualifiedName~JetStream.WaitQueueTests|FullyQualifiedName~JetStream.ConsumerPoliciesTests|FullyQualifiedName~JetStream.ConsumerStateTests"
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog.NatsConsumerTests|FullyQualifiedName~ImplBacklog.JetStreamEngineTests"
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog.JetStreamClusterTests1|FullyQualifiedName~ImplBacklog.JetStreamClusterTests3|FullyQualifiedName~ImplBacklog.JetStreamClusterTests4"
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog.AccountTests|FullyQualifiedName~ImplBacklog.MqttHandlerTests|FullyQualifiedName~ImplBacklog.ConcurrencyTests1|FullyQualifiedName~ImplBacklog.JetStreamBenchmarks"

Checkpoint/full gate:

$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/

Status Update Protocol (HARD LIMIT: max 15 IDs per batch-update)

  • Never send more than 15 IDs per feature batch-update or test batch-update.
  • Never promote verified without checkpoint evidence.
  • Never update IDs outside active task scope.
  • Blocked items remain deferred with concrete reason.

Use:

$DOTNET run --project tools/NatsNet.PortTracker -- \
  feature batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute

$DOTNET run --project tools/NatsNet.PortTracker -- \
  test batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute

Checkpoint Protocol Between Tasks (REQUIRED)

Before starting the next task:

  1. Run Stub Detection Check on current diff.
  2. Run Build Gate.
  3. Run targeted Test Gate for touched classes.
  4. Run full unit-test gate.
  5. Apply status updates in <=15 ID chunks only.
  6. Commit checkpoint before moving on.

ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)

Forbidden Patterns

Forbidden in mapped Batch 38 feature/test work:

  • throw new NotImplementedException()
  • Empty/no-op mapped method bodies
  • Placeholder comments (TODO, PLACEHOLDER, later)
  • Fake-pass assertions (Assert.True(true), Assert.Pass())
  • Template assertions disconnected from behavior:
    • var goFile = "server/..."
    • "<MethodName>".ShouldContain("Should")
    • GetRequiredApiLevel(new Dictionary<string, string>()).ShouldBe(string.Empty) as a core assertion
  • Constant-return shortcuts for mapped behavior without justification and coverage:
    • => default;
    • return default;
    • return null;
  • Catch-and-ignore blocks used to force green tests

Hard Limits

  • Max feature group size: ~20.
  • Max IDs per feature batch-update: 15.
  • Max IDs per test batch-update: 15.
  • One active feature loop at a time.
  • One active test loop at a time.
  • Mandatory checkpoint between all tasks.
  • No verified promotion without checkpoint evidence.

If You Get Stuck (REQUIRED)

  1. Stop on the current feature/test ID immediately.
  2. Do not write placeholder logic or fake assertions.
  3. Mark the item deferred with a specific blocker reason.
  4. Continue with the next unblocked ID.

Feature deferral:

$DOTNET run --project tools/NatsNet.PortTracker -- \
  feature update <feature_id> --status deferred --override "blocked: <specific reason>" --db porting.db

Test deferral:

$DOTNET run --project tools/NatsNet.PortTracker -- \
  test update <test_id> --status deferred --override "blocked: <specific reason>" --db porting.db

deferred with a concrete reason is correct. Stubbing is not.

Feature Groups (max ~20 each)

Group A (19): enums, subject filters, config defaults/validation, add-consumer and assignment entry

IDs:
584,585,586,587,588,589,590,591,592,594,595,596,597,598,599,600,601,602,603

Group B (19): monitor channels, leadership/advisory path, delivery-interest bridge to server

IDs:
604,605,606,607,608,610,612,613,614,615,616,617,618,619,620,621,622,623,624

Group C (19): delivery-subject/rate-limit/ack ingress and early delivery state transitions

IDs:
625,626,627,629,630,631,632,633,635,636,637,638,639,640,641,642,643,644,645

Group D (19): proposals, ack floor updates, pending-request cluster flow, nak/term/ackWait foundation

IDs:
646,647,648,649,650,651,652,653,654,655,656,657,658,659,660,661,662,663,664

Group E (20): store-state persistence, info snapshots, sampling/filtering, next-request parsing, wait-queue recycle

IDs:
665,666,667,668,669,670,671,672,673,674,675,676,677,678,679,680,681,682,683,684

Test Waves

Wave T1 (19): consumer filters/actions/pinned behavior

IDs:
1220,1221,1222,1223,1224,1225,1226,1228,1239,1240,1241,1242,1243,1244,1248,1250,1259,1260,1263

Wave T2 (19): consumer max-deliver/ack/rate/replay/multi-subject behavior

IDs:
1266,1268,1274,1278,1279,1280,1281,1282,1289,1311,1312,1313,1316,1318,1319,1320,1321,1325,1344

Wave T3 (8): cluster consumer lifecycle consistency

IDs:
812,870,892,1123,1124,1126,1172,1173

Wave T4 (12): core engine consumer lifecycle integration

IDs:
96,1472,1473,1491,1494,1497,1505,1513,1523,1525,1527,1546

Wave T5 (13): perf/race/cross-protocol and benchmark-adjacent coverage

IDs:
749,1557,1558,1559,1561,1624,1647,1670,1671,1764,2212,2385,2388

Task 1: Preflight, Baseline, and Batch Start

Files:

  • Read: docs/plans/2026-02-27-batch-38-consumer-lifecycle-design.md
  • Read: golang/nats-server/server/consumer.go

Steps:

  1. Run dependency preflight for batches 34, 36, 38.
  2. Start Batch 38 only if ready.
  3. Capture baseline build + full unit test.
  4. Record the baseline counts before edits.

Task 2: Implement Feature Group A (19 IDs)

Files:

  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StoreTypes.ConsumerPolicies.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StreamTypes.ConsumerLifecycle.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/Internal/SubjectTokens.ConsumerFilters.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Config.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Consumers.cs
  • Modify/Create tests: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/ConsumerPoliciesTests.cs

Steps:

  1. Execute per-feature loop for each Group A ID.
  2. Keep each status update batch at max 15 IDs.
  3. Run checkpoint protocol.

Task 3: Implement Feature Group B (19 IDs)

Files:

  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Lifecycle.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Advisories.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.ConsumerInterest.cs

Steps:

  1. Execute per-feature loop for each Group B ID.
  2. Validate advisory and delivery-interest paths with focused tests.
  3. Apply status updates in <=15-ID chunks.
  4. Run checkpoint protocol.

Task 4: Implement Feature Group C (19 IDs)

Files:

  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Acks.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.State.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.ConsumerConfig.cs
  • Modify tests: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/NatsConsumerTests.cs

Steps:

  1. Execute per-feature loop for each Group C ID.
  2. Cover ack message creation/parsing and deliver-subject update behavior.
  3. Apply status updates in <=15-ID chunks.
  4. Run checkpoint protocol.

Task 5: Implement Feature Group D (19 IDs)

Files:

  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.State.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Acks.cs
  • Modify/Create tests: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/ConsumerStateTests.cs

Steps:

  1. Execute per-feature loop for each Group D ID.
  2. Verify proposal forwarding, pending-request transitions, and processNak/processTerm.
  3. Apply status updates in <=15-ID chunks.
  4. Run checkpoint protocol.

Task 6: Implement Feature Group E (20 IDs)

Files:

  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.State.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.WaitQueue.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StreamTypes.ConsumerLifecycle.cs
  • Modify tests: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/WaitQueueTests.cs

Steps:

  1. Execute per-feature loop for each Group E ID.
  2. Verify state read/write/info sampling/filter parsing/wait-queue recycle behavior.
  3. Apply status updates in <=15-ID chunks.
  4. Run checkpoint protocol.

Task 7: Port Test Wave T1 (19 IDs)

Files:

  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs

Steps:

  1. Execute per-test loop for each T1 ID.
  2. Replace template assertions with behavioral checks.
  3. Apply status updates in <=15-ID chunks.
  4. Run checkpoint protocol.

Task 8: Port Test Wave T2 (19 IDs)

Files:

  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs

Steps:

  1. Execute per-test loop for each T2 ID.
  2. Validate ack/rate/replay/multi-subject semantics using produced feature behavior.
  3. Apply status updates in <=15-ID chunks.
  4. Run checkpoint protocol.

Task 9: Port Test Wave T3 (8 IDs)

Files:

  • Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs
  • Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs
  • Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests4.Impltests.cs

Steps:

  1. Execute per-test loop for each T3 ID.
  2. Keep cluster-only blockers as deferred with explicit reason if environment-dependent.
  3. Run checkpoint protocol.

Task 10: Port Test Wave T4 (12 IDs)

Files:

  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AccountTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs

Steps:

  1. Execute per-test loop for each T4 ID.
  2. Verify integration behavior for nextReqFromMsg, filtering, ack reply and lifecycle interactions.
  3. Apply status updates in <=15-ID chunks.
  4. Run checkpoint protocol.

Task 11: Port Test Wave T5 (13 IDs)

Files:

  • Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamBenchmarks.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs

Steps:

  1. Execute per-test loop for each T5 ID.
  2. For benchmark-only cases not meaningful as unit tests, defer with explicit benchmark blocker reason.
  3. Apply status updates in <=15-ID chunks.
  4. Run checkpoint protocol.

Task 12: Final Verification, Status Closure, and Batch Completion

Files:

  • Modify: porting.db
  • Generate: reports/current.md

Steps:

  1. Run full stub detection across changed source and backlog tests.
  2. Run full build + full unit-test suite.
  3. Verify all Batch 38 IDs are complete/verified/n_a/deferred(with reason) with no silent stubs.
  4. Run:
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 38 --db porting.db
$DOTNET run --project tools/NatsNet.PortTracker -- report summary --db porting.db
./reports/generate-report.sh
  1. Complete batch:
$DOTNET run --project tools/NatsNet.PortTracker -- batch complete 38 --db porting.db
Reference in New Issue View Git Blame Copy Permalink