natsnet/docs/plans/2026-02-27-batch-29-jetstream-batching-design.md

Batch 29 JetStream Batching Design

Date: 2026-02-27
Batch: 29 (JetStream Batching)
Scope: 12 features + 3 unit tests
Dependencies: batch 27 (JetStream Core)
Go source: golang/nats-server/server/jetstream_batching.go (+ mapped tests in server/raft_test.go)

Problem

Batch 29 ports JetStream atomic batch internals: batch lifecycle/store setup, staged consistency bookkeeping, apply-path rejection/cleanup, and pre-proposal header validation (checkMsgHeadersPreClusteredProposal). This batch also includes 3 Raft-node behavior tests that depend on batch cleanup correctness.

Context Findings

Required command outputs (captured)

• batch show 29 --db porting.db
  • Batch 29 is pending
  • 12 features + 3 tests are all deferred
  • Dependency: batch 27
  • Go file: server/jetstream_batching.go
• batch list --db porting.db
  • Batch 29 sits after batch 28 and depends on 27
  • Batch 40 (MQTT Server/JSA) depends on 27 as well; keeping 29 high quality prevents later churn in JetStream/Raft behavior
• report summary --db porting.db
  • Features verified: 1271 / 3673
  • Tests verified: 430 / 3257
  • Deferred backlog remains dominant, so no-stub discipline is mandatory

Batch 29 mapped IDs

Features:

• 1508 batching.newBatchGroup
• 1509 getBatchStoreDir
• 1510 newBatchStore
• 1511 batchGroup.readyForCommit
• 1512 batchGroup.cleanup
• 1513 batchGroup.cleanupLocked
• 1514 batchGroup.stopLocked
• 1515 batchStagedDiff.commit
• 1516 batchApply.clearBatchStateLocked
• 1517 batchApply.rejectBatchStateLocked
• 1518 batchApply.rejectBatchState
• 1519 checkMsgHeadersPreClusteredProposal (largest surface, ~423 Go LOC)

Tests:

• 2654 TestNRGMultipleStopsDontPanic -> RaftNodeTests.NRGMultipleStopsDontPanic_ShouldSucceed
• 2674 TestNRGKeepRunningOnServerShutdown -> RaftNodeTests.NRGKeepRunningOnServerShutdown_ShouldSucceed
• 2718 TestNRGInitSingleMemRaftNodeDefaults -> RaftNodeTests.NRGInitSingleMemRaftNodeDefaults_ShouldSucceed

Existing .NET baseline

• dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamBatching.cs exists but is partial and contains stub-like behavior (ReadyForCommit comment indicates stub).
• No current RaftNodeTests file exists under dotnet/tests/...; mapped test targets are not implemented yet.
• JetStream batching test file currently contains deferred placeholders and does not cover Batch 29 mapped Raft tests.

Approaches

Approach A: Single massive JetStreamBatching.cs pass in one shot

• Pros: fewer commits, direct throughput.
• Cons: high defect risk around cross-cutting map/counter/header logic, hard to validate incrementally.

Approach B (Recommended): Two feature waves + one test wave with strict evidence gates

• Wave 1: batch/store lifecycle primitives (1508-1514)
• Wave 2: staged/apply/header semantics (1515-1519)
• Wave 3: mapped Raft tests (2654,2674,2718)
• Pros: manageable review units, easier causality between feature changes and tests, strongest anti-stub control.
• Cons: more status updates/checkpoints.

Approach C: Signature-first fill (compile now, behavior later)

• Pros: quick apparent progress.
• Cons: violates anti-stub goals and creates false tracker progress.

Decision: Approach B.

Proposed Design

1. Component boundaries

• Keep batching logic centered in JetStreamBatching.cs for mapped methods.
• Add narrow helper methods/types only when required to preserve method-level mapping and testability.
• Keep heavy validation (checkMsgHeadersPreClusteredProposal) behaviorally aligned with Go checks: pre-check ordering, duplicate/msg-id checks, counter increment path, expected sequence checks, scheduling/rollup constraints, and discard policy checks.

2. Data and concurrency model

• Preserve lock expectations from Go comments by using existing Lock/ReaderWriterLockSlim conventions.
• Preserve inflight/global counters and cleanup semantics as deterministic state transitions.
• Ensure timer cleanup and commit readiness are race-safe and idempotent.

3. Feature grouping strategy (max ~20)

• Group A (7 features): 1508-1514
  • Batch group creation, store dir/store construction, commit readiness, cleanup and stop paths.
• Group B (5 features): 1515-1519
  • Staged diff commit state, batch apply clear/reject, and full header pre-check function.

4. Test strategy

• Create/port mapped tests in RaftNodeTests (or equivalent mapped class file) with real Arrange/Act/Assert behavior.
• Keep tests deterministic and non-networked where possible; if runtime infrastructure is missing, explicitly defer with reason.
• Add focused unit tests for JetStreamBatching helpers as needed to verify feature behavior before promoting feature status.

5. Status and evidence design

• Status transitions must be evidence-backed: stub -> complete -> verified.
• Chunked updates (max 15 IDs) to prevent bulk unverifiable promotion.
• Checkpoint between tasks: stub scan + build + targeted tests + tracker updates.

Risks and Mitigations

• Risk: 1519 complexity causes partial/placeholder implementation.
  • Mitigation: isolate into dedicated task, require feature-level gates and explicit defer-if-blocked path.
• Risk: Mapped Raft tests need runtime hooks not yet available.
  • Mitigation: mark deferred with exact blocker reason; no fake-pass tests.
• Risk: Tracker drift from actual behavior.
  • Mitigation: per-ID evidence, max-15 update chunk, and post-task checkpoints.

Success Criteria

• All Batch 29 features and tests are either:
  • verified with captured verification evidence, or
  • deferred with explicit blocker reason.
• No new stub patterns introduced in touched production/test files.
• Build and relevant test gates are green at each task checkpoint.

Non-Goals

• Executing implementation in this design doc.
• Refactoring unrelated JetStream or Raft subsystems beyond mapped Batch 29 behavior.
• Broad integration-test harness expansion beyond what Batch 29 requires.