Files
natsnet/docs/plans/2026-02-27-batch-34-js-cluster-consumers-implementation-plan.md
Joseph Doherty f8dce79ac0 Add batch plans for batches 31-36 (rounds 16-18)
Generated design docs and implementation plans via Codex for:
- Batch 31: Raft Part 2
- Batch 32: JS Cluster Meta
- Batch 33: JS Cluster Streams
- Batch 34: JS Cluster Consumers
- Batch 35: JS Cluster Remaining
- Batch 36: Stream Lifecycle

All plans include mandatory verification protocol and anti-stub guardrails.
Updated batches.md with file paths and planned status.
2026-02-27 17:01:31 -05:00

19 KiB

Batch 34 JS Cluster Consumers Implementation Plan

For Codex: REQUIRED SUB-SKILL: Use executeplan to implement this plan task-by-task.

Goal: Port and verify Batch 34 (JS Cluster Consumers) from server/jetstream_cluster.go, with auditable status transitions for all 58 features and 160 tests.

Architecture: Implement consumer-cluster behavior in three bounded feature groups (20/20/18) across JetStream, JetStreamCluster, NatsConsumer, NatsServer, and Account. Then port mapped tests in five waves (37/39/33/32/19) with strict per-item verification evidence, anti-stub enforcement, and checkpoint gates between tasks.

Tech Stack: .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (porting.db)

Design doc: docs/plans/2026-02-27-batch-34-js-cluster-consumers-design.md


Batch 34 Scope

  • Batch ID: 34
  • Name: JS Cluster Consumers
  • Dependency: 33
  • Go source: golang/nats-server/server/jetstream_cluster.go
  • Features: 58 (1636-1693)
  • Tests: 160 (IDs listed in wave sections below)

If dotnet is not on PATH, use:

DOTNET=/usr/local/share/dotnet/dotnet

Primary production files (expected touch set):

  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStream.ClusterConsumers.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamCluster.Consumers.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Cluster.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterConsumers.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.JetStream.cs
  • Modify (as needed): dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs
  • Modify (as needed): dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs
  • Modify (as needed): dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.cs
  • Modify (as needed): dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs

Primary test files (expected touch set):

  • Modify/Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs
  • Modify/Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs
  • Modify/Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests4.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ServerOptionsTests.Impltests.cs
  • Modify/Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConfigCheckTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConfigReloaderTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsServerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JwtProcessorTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamJwtTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AuthCalloutTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AuthHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/CertificateStoreWindowsTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AccountTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/WebSocketHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/LeafNodeHandlerTests.Impltests.cs
  • Modify/Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/LeafNodeProxyTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RouteHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/GatewayHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamLeafNodeTests.Impltests.cs
  • Modify/Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamSuperClusterTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MessageTracerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MonitoringHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/EventsHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs

MANDATORY VERIFICATION PROTOCOL

NON-NEGOTIABLE: Every Batch 34 feature/test ID must pass this protocol before promotion.

Dependency Preflight Gate (before any status changes)

$DOTNET run --project tools/NatsNet.PortTracker -- batch show 33 --db porting.db
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 34 --db porting.db
$DOTNET run --project tools/NatsNet.PortTracker -- batch ready --db porting.db

Only when dependency is satisfied and batch is ready:

$DOTNET run --project tools/NatsNet.PortTracker -- batch start 34 --db porting.db

Baseline evidence:

$DOTNET build dotnet/
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/

Per-Feature Verification Loop (REQUIRED per feature ID)

  1. Inspect mapping and Go span:
$DOTNET run --project tools/NatsNet.PortTracker -- feature show <feature_id> --db porting.db
  1. Claim only that feature ID:
$DOTNET run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status stub --db porting.db
  1. Add/update at least one focused test case that fails for the intended behavior.
  2. Run focused test and confirm failure is for the target behavior.
  3. Implement minimal production logic for that feature ID.
  4. Run Stub Detection Check (below).
  5. Run Build Gate.
  6. Run Test Gate (focused/class filters).
  7. If green, mark feature complete (promote to verified only at checkpoint).

Per-Test Verification Loop (REQUIRED per test ID)

  1. Inspect mapping and Go source:
$DOTNET run --project tools/NatsNet.PortTracker -- test show <test_id> --db porting.db
  1. Claim only that test ID:
$DOTNET run --project tools/NatsNet.PortTracker -- test update <test_id> --status stub --db porting.db
  1. Port full Arrange/Act/Assert behavior (no placeholders).
  2. Run single test method:
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~<ClassName>.<MethodName>" --verbosity normal
  1. Confirm summary contains Passed: 1, Failed: 0 (not Passed: 0).
  2. Run class-level filter for the touched class.
  3. Run Stub Detection Check + Build Gate.
  4. If green, mark test complete (promote to verified only at checkpoint).

Stub Detection Check (REQUIRED after every feature/test loop)

Run against changed C# files:

changed_files=$(git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests | rg "\.cs$" || true)
if [ -n "$changed_files" ]; then
  echo "$changed_files" | xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|Assert\.True\(true\)|Assert\.Pass|ShouldBe\(true\)|=>\s*default;|=>\s*null;|return\s+null;\s*$|return\s+0;\s*$|return\s+false;\s*$)"
fi

Any match in mapped methods/tests blocks promotion.

Build Gate (REQUIRED)

Run:

$DOTNET build dotnet/

Required timing:

  • after each feature loop
  • after each test loop
  • before every batch-update
  • at every task checkpoint

Test Gate (REQUIRED)

Minimum gates:

$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~JetStream"
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog"

Plus per-item focused gates:

  • feature loop: nearest impacted class filters
  • test loop: method filter, then class filter

Status Update Protocol (HARD LIMIT: 15 IDs max per batch-update)

  • Never pass more than 15 IDs in one feature batch-update or test batch-update.
  • Never promote verified without loop evidence + checkpoint evidence.
  • Never update IDs outside the active task.
  • For blocked IDs, keep deferred and provide explicit --override reason.

Templates:

$DOTNET run --project tools/NatsNet.PortTracker -- \
  feature batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute

$DOTNET run --project tools/NatsNet.PortTracker -- \
  test batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute

Checkpoint Protocol Between Tasks (REQUIRED)

Before starting the next task:

  1. Run Stub Detection Check.
  2. Run Build Gate.
  3. Run targeted class filters for all touched files.
  4. Run full unit suite:
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
  1. Promote proven IDs only, in <=15 chunks.
  2. Commit checkpoint before moving on.

ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)

These rules apply to both production features and tests.

Forbidden Patterns

  • throw new NotImplementedException() in any mapped Batch 34 method/test.
  • Empty method bodies for mapped features.
  • Placeholder markers in mapped code (TODO, PLACEHOLDER, later).
  • Fake pass assertions (Assert.True(true), Assert.Pass(), one-line tautologies).
  • Constant-return shortcuts that ignore inputs/state (return false;, return 0;, return null;, return string.Empty;).
  • Catch-and-ignore patterns that suppress failures to force green tests.

Hard Limits

  • Feature task groups: max ~20 IDs.
  • feature batch-update: max 15 IDs per call.
  • test batch-update: max 15 IDs per call.
  • One active feature loop at a time.
  • One active test loop at a time.
  • Mandatory checkpoint between all tasks.
  • No verified promotions without checkpoint evidence.

If You Get Stuck (REQUIRED)

  1. Stop on the current ID immediately.
  2. Do not add placeholder logic or fake assertions.
  3. Mark blocked item deferred with explicit reason:
$DOTNET run --project tools/NatsNet.PortTracker -- \
  feature update <feature_id> --status deferred --override "blocked: <specific reason>" --db porting.db

$DOTNET run --project tools/NatsNet.PortTracker -- \
  test update <test_id> --status deferred --override "blocked: <specific reason>" --db porting.db
  1. Continue with the next unblocked ID.

Deferred-with-reason is correct. Stubbed/fake-pass progress is not.


Feature Groups (max ~20 each)

Group A (20): consumer assignment + consumer raft hooks + ack/leader foundations

IDs:
1636,1637,1638,1639,1640,1641,1642,1643,1644,1645,1646,1647,1648,1649,1650,1651,1652,1653,1654,1655

Group B (20): assignment-result processing + peer selection + clustered request prep

IDs:
1656,1657,1658,1659,1660,1661,1662,1663,1664,1665,1666,1667,1668,1669,1670,1671,1672,1673,1674,1675

Group C (18): clustered update/delete/list + encode/decode helpers

IDs:
1676,1677,1678,1679,1680,1681,1682,1683,1684,1685,1686,1687,1688,1689,1690,1691,1692,1693

Test Waves

Wave T1 (37): JetStream cluster consumer core

IDs:
757,761,778,802,813,818,834,881,883,900,911,929,963,964,986,1003,1059,1108,1151,1193,1208,1231,1233,1237,1290,1342,1553,1576,1582,1587,1599,1627,1633,1662,1663,1674,1747

Wave T2 (39): options/config/reload/server validations

IDs:
271,272,273,2515,2534,2535,2536,2543,2548,2554,2558,2559,2562,2564,2568,2570,2573,2574,2575,2577,2578,2579,2580,2581,2582,2584,2591,2592,2594,2595,2596,2723,2740,2763,2765,2777,2785,2889,2892

Wave T3 (33): JWT/auth/cert/account validations

IDs:
103,126,136,139,147,151,155,156,157,1386,1388,1391,1397,1834,1854,1856,1859,1862,1863,1865,1866,1868,1869,1870,1871,1876,1877,1878,1879,1880,1882,1885,1886

Wave T4 (32): websocket/leaf/route/gateway behavior

IDs:
655,1904,1905,1910,1913,1914,1920,1931,1937,1938,1941,1969,1972,1979,2824,2829,2845,2849,3074,3081,3090,3091,3092,3094,3096,3100,3101,3107,3108,3123,3129,3133

IDs:
342,599,1405,1408,1422,1430,1439,1441,1444,1448,2146,2172,2173,2176,2180,2186,2264,2332,2352


Task 1: Preflight and Baseline

Files:

  • Read: docs/plans/2026-02-27-batch-34-js-cluster-consumers-design.md
  • Read: golang/nats-server/server/jetstream_cluster.go

Steps:

  1. Run dependency preflight commands and ensure Batch 34 is startable.
  2. Start Batch 34.
  3. Capture baseline build and test results.
  4. Do not update statuses beyond stub markers before baseline evidence is captured.

Task 2: Implement Feature Group A (20 IDs)

Files:

  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStream.ClusterConsumers.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Cluster.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamCluster.Consumers.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterConsumers.cs

Steps:

  1. Set Group A IDs to stub using two commands (15 + 5 IDs).
  2. Run Per-Feature Verification Loop for each Group A ID.
  3. Execute required gates after every ID.
  4. At checkpoint, promote proven IDs (complete, then verified) in <=15 chunks.

Task 3: Implement Feature Group B (20 IDs)

Files:

  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStream.ClusterConsumers.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamCluster.Consumers.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterConsumers.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.JetStream.cs

Steps:

  1. Set Group B IDs to stub using two commands (15 + 5 IDs).
  2. Run Per-Feature Verification Loop for each Group B ID.
  3. Execute required gates after every ID.
  4. At checkpoint, promote proven IDs in <=15 chunks.

Task 4: Implement Feature Group C (18 IDs)

Files:

  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterConsumers.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamCluster.Consumers.cs
  • Modify/Create: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStream.ClusterConsumers.cs

Steps:

  1. Set Group C IDs to stub using two commands (15 + 3 IDs).
  2. Run Per-Feature Verification Loop for each Group C ID.
  3. Execute required gates after every ID.
  4. At checkpoint, promote proven IDs in <=15 chunks.

Task 5: Port Test Wave T1 (37 IDs)

Files:

  • Modify/Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs
  • Modify/Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs
  • Modify/Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests4.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs

Steps:

  1. Set T1 IDs to stub in <=15 ID chunks.
  2. Run Per-Test Verification Loop for each ID.
  3. Run class filters per touched class.
  4. Execute checkpoint and promote proven IDs in <=15 chunks.

Task 6: Port Test Wave T2 (39 IDs)

Files:

  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ServerOptionsTests.Impltests.cs
  • Modify/Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConfigCheckTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConfigReloaderTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsServerTests.Impltests.cs

Steps:

  1. Set T2 IDs to stub in <=15 ID chunks.
  2. Run Per-Test Verification Loop for each ID.
  3. Run class filters per touched class.
  4. Execute checkpoint and promote proven IDs in <=15 chunks.

Task 7: Port Test Wave T3 (33 IDs)

Files:

  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JwtProcessorTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamJwtTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AuthCalloutTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AuthHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/CertificateStoreWindowsTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AccountTests.Impltests.cs

Steps:

  1. Set T3 IDs to stub in <=15 ID chunks.
  2. Run Per-Test Verification Loop for each ID.
  3. Run class filters per touched class.
  4. Execute checkpoint and promote proven IDs in <=15 chunks.

Task 8: Port Test Wave T4 (32 IDs)

Files:

  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/WebSocketHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/LeafNodeHandlerTests.Impltests.cs
  • Modify/Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/LeafNodeProxyTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RouteHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/GatewayHandlerTests.Impltests.cs

Steps:

  1. Set T4 IDs to stub in <=15 ID chunks.
  2. Run Per-Test Verification Loop for each ID.
  3. Run class filters per touched class.
  4. Execute checkpoint and promote proven IDs in <=15 chunks.

Task 9: Port Test Wave T5 (19 IDs)

Files:

  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamLeafNodeTests.Impltests.cs
  • Modify/Create: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamSuperClusterTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MessageTracerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MonitoringHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/EventsHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs

Steps:

  1. Set T5 IDs to stub in <=15 ID chunks.
  2. Run Per-Test Verification Loop for each ID.
  3. Run class filters per touched class.
  4. Execute checkpoint and promote proven IDs in <=15 chunks.

Task 10: Final Batch Verification and Closure

Files:

  • Modify: porting.db
  • Modify: reports/current.md (via generator)

Steps:

  1. Run final gates (stub scan, build, full unit tests).
  2. Run audits and reconcile status mismatches:
$DOTNET run --project tools/NatsNet.PortTracker -- audit --type features --db porting.db
$DOTNET run --project tools/NatsNet.PortTracker -- audit --type tests --db porting.db
  1. Ensure all Batch 34 IDs are verified or explicit deferred with reason.
  2. Complete batch:
$DOTNET run --project tools/NatsNet.PortTracker -- batch complete 34 --db porting.db
  1. Generate report:
./reports/generate-report.sh