natsnet/docs/plans/2026-02-27-batch-29-jetstream-batching-implementation-plan.md

# Batch 29 JetStream Batching Implementation Plan

> **For Codex:** REQUIRED SUB-SKILL: Use `executeplan` to implement this plan task-by-task.

**Goal:** Port and verify Batch 29 JetStream batching behavior (12 features, 3 tests) from `server/jetstream_batching.go` and mapped `raft_test.go` cases without introducing stubs or unverifiable tracker updates.

**Architecture:** Execute in two feature groups and one test group. Group A ports lifecycle/store primitives (`1508-1514`), Group B ports staged/apply/header checks (`1515-1519`, including the large `checkMsgHeadersPreClusteredProposal` surface), then mapped Raft tests are ported/verified. Every ID follows strict per-item evidence gates before status promotion.

**Tech Stack:** .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (`porting.db`)

**Design doc:** `docs/plans/2026-02-27-batch-29-jetstream-batching-design.md`

---

## Batch 29 Scope

- Batch ID: `29`
- Name: `JetStream Batching`
- Dependency: `27`
- Go source: `golang/nats-server/server/jetstream_batching.go`
- Features: `1508-1519` (12 total)
- Tests: `2654,2674,2718`

Primary implementation files:

- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamBatching.cs`
- Optional split (recommended for readability of feature `1519`):
  - Create/Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamBatching.HeaderChecks.cs`
- Optional supporting updates only if required by compile/runtime behavior:
  - `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs`
  - `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
  - `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/FileStore.cs`

Primary test files:

- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftNodeTests.cs`
- Modify or Create focused batching tests as needed:
  - `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/JetStreamBatchingCoreTests.cs`
- Existing deferred batching integration placeholders (do not convert to fake unit tests):
  - `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/JetStreamBatchingTests.cs`

---

## MANDATORY VERIFICATION PROTOCOL

> **NON-NEGOTIABLE:** Applies to every feature ID and test ID in Batch 29.

### Preflight Dependency Gate (REQUIRED before Task 1 coding)

1. Verify dependency and readiness:
   ```bash
   dotnet run --project tools/NatsNet.PortTracker -- batch show 27 --db porting.db
   dotnet run --project tools/NatsNet.PortTracker -- batch show 29 --db porting.db
   dotnet run --project tools/NatsNet.PortTracker -- batch ready --db porting.db
   ```
2. Start the batch only when dependency checks pass:
   ```bash
   dotnet run --project tools/NatsNet.PortTracker -- batch start 29 --db porting.db
   ```
3. Baseline gates:
   ```bash
   dotnet build dotnet/
   dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
   ```

### Per-Feature Verification Loop (REQUIRED for each feature ID)

For each feature in current task group:

1. Inspect mapping and Go span:
   ```bash
   dotnet run --project tools/NatsNet.PortTracker -- feature show <feature_id> --db porting.db
   sed -n '<go_start>,<go_end>p' golang/nats-server/server/jetstream_batching.go
   ```
2. Mark that feature as `stub` before editing:
   ```bash
   dotnet run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status stub --db porting.db
   ```
3. Implement mapped behavior in .NET (no placeholders).
4. Run **Build Gate**.
5. Run **Test Gate** (focused tests for changed behavior).
6. Run **Stub Detection Check** on touched files.
7. If gates are green and no stubs are detected, promote feature to `complete`:
   ```bash
   dotnet run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status complete --db porting.db
   ```
8. Promote feature IDs to `verified` only after task-level checkpoint passes.

### Per-Test Verification Loop (REQUIRED for each test ID)

1. Inspect mapped test details and Go source:
   ```bash
   dotnet run --project tools/NatsNet.PortTracker -- test show <test_id> --db porting.db
   sed -n '<go_start>,<go_end>p' golang/nats-server/server/raft_test.go
   ```
2. Mark test `stub` before editing:
   ```bash
   dotnet run --project tools/NatsNet.PortTracker -- test update <test_id> --status stub --db porting.db
   ```
3. Implement real Arrange/Act/Assert test calling production code.
4. Run single-test command:
   ```bash
   dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
     --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.JetStream.RaftNodeTests.<MethodName>" \
     --verbosity normal
   ```
5. Confirm summary includes `Passed: 1, Failed: 0` (not `Passed: 0`).
6. Run class-level test gate:
   ```bash
   dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
     --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.JetStream.RaftNodeTests"
   ```
7. Run Stub Detection Check.
8. Promote test to `complete`, then `verified` at checkpoint when evidence is complete.

### Stub Detection Check (REQUIRED after every feature/test loop and each task)

Run against touched code:

```bash
git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests \
  | rg "\.cs$" \
  | xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|Assert\\.True\\(true\\)|Assert\\.Pass|throw new Exception\\(\"TODO\"\\)|=>\\s*default;|=>\\s*null;)"
```

Any match in touched methods means the item is not eligible for `complete` or `verified`.

### Build Gate (REQUIRED)

`dotnet build dotnet/` is mandatory:

- after each feature implementation loop
- after each test implementation loop
- before any `batch-update`
- at every task checkpoint

### Test Gate (REQUIRED)

Minimum per-task gates:

```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~JetStreamBatching"
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~RaftNodeTests"
```

Checkpoint and final gates:

```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
```

### Status Update Protocol (HARD LIMIT: max 15 IDs per batch-update)

- Allowed path: `deferred/not_started -> stub -> complete -> verified`
- Use `batch-update` with **at most 15 IDs** per call.
- Never update IDs outside the current task group.
- Never mark `verified` without corresponding build/test/stub-scan evidence.
- If audit rejects status change, use `--override "<specific reason>"` only with explicit evidence.

Templates:

```bash
dotnet run --project tools/NatsNet.PortTracker -- \
  feature batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute

dotnet run --project tools/NatsNet.PortTracker -- \
  test batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
```

### Checkpoint Protocol Between Tasks (REQUIRED)

At the end of each task before starting the next task:

1. Run Stub Detection Check.
2. Run Build Gate.
3. Run task-relevant Test Gate.
4. Run full unit suite:
   ```bash
   dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
   ```
5. Update status IDs for current task only (max 15 IDs per command).
6. Commit checkpoint changes.

---

## ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)

### Forbidden Patterns (Features + Tests)

Any of these in touched methods means the work is a stub and must not be promoted:

- `throw new NotImplementedException()`
- Empty mapped method bodies
- Placeholder comments: `// TODO`, `// PLACEHOLDER`, `// later`
- Fake-pass assertions: `Assert.True(true)`, `Assert.Pass()`
- Tests that never call production code in `ZB.MOM.NatsNet.Server`
- Constant-return placeholders for non-trivial logic (`return null;`, `return 0;`, `return false;`, `return string.Empty;`) without Go-equivalent behavior
- Catch-all exception swallowing used to force pass behavior

### Hard Limits

- Max feature IDs per group: `~20`
- Max IDs per `feature/test batch-update`: `15`
- One active feature loop at a time
- No `verified` promotion unless build + targeted tests + stub scan are green
- One checkpoint commit per task minimum
- Feature `1519` must not be merged as partial/no-op behavior

### If You Get Stuck (MANDATORY)

1. Stop work on the blocked ID immediately.
2. Do **not** leave placeholder code or fake tests.
3. Mark the blocked item `deferred` with specific reason:
   ```bash
   dotnet run --project tools/NatsNet.PortTracker -- feature update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
   dotnet run --project tools/NatsNet.PortTracker -- test update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
   ```
4. Move to the next unblocked ID in the same task.
5. Deferred-with-reason is correct behavior; stubs are not.

---

## Feature/Test Grouping (max ~20 per group)

### Group A (7 features): Batch lifecycle + store creation

`1508,1509,1510,1511,1512,1513,1514`

### Group B (5 features): Staged/apply state + pre-proposal header checks

`1515,1516,1517,1518,1519`

### Group C (3 tests): Raft-node behavioral tests mapped to batch 29

`2654,2674,2718`

---

### Task 1: Preflight and Batch Start

**Files:**

- Read: `docs/standards/dotnet-standards.md`
- Read: `docs/plans/2026-02-27-batch-29-jetstream-batching-design.md`
- Read: `golang/nats-server/server/jetstream_batching.go`
- Read: `golang/nats-server/server/raft_test.go` (mapped lines)

**Step 1: Dependency/readiness checks**

Run:

```bash
dotnet run --project tools/NatsNet.PortTracker -- batch show 27 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- batch show 29 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- batch ready --db porting.db
```

Expected:

- Batch 29 is startable, dependency 27 satisfied.

**Step 2: Start batch**

Run:

```bash
dotnet run --project tools/NatsNet.PortTracker -- batch start 29 --db porting.db
```

Expected:

- Batch transitions to in-progress.

**Step 3: Baseline build/test**

Run:

```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
```

Expected:

- Known baseline captured before edits.

**Step 4: Checkpoint protocol and commit**

---

### Task 2: Implement Group A Features (`1508-1514`)

**Files:**

- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamBatching.cs`
- Optional modify if needed by compile/runtime wiring:
  - `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs`
  - `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
  - `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/FileStore.cs`
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/JetStreamBatchingCoreTests.cs`

**Step 1: Mark Group A as `stub` (single chunk, 7 IDs <= 15)**

```bash
dotnet run --project tools/NatsNet.PortTracker -- \
  feature batch-update --ids "1508,1509,1510,1511,1512,1513,1514" --set-status stub --db porting.db --execute
```

**Step 2: Execute Per-Feature Verification Loop for each ID in Group A**

Expected:

- Lifecycle/store methods are behaviorally implemented and individually gated.

**Step 3: Task-level gates + checkpoint protocol**

Run:

```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~JetStreamBatching"
```

**Step 4: Promote eligible Group A IDs to `complete` (blocked IDs stay `deferred`)**

```bash
dotnet run --project tools/NatsNet.PortTracker -- \
  feature batch-update --ids "<eligible subset of 1508-1514>" --set-status complete --db porting.db --execute
```

**Step 5: After checkpoint evidence, promote eligible Group A IDs to `verified`**

```bash
dotnet run --project tools/NatsNet.PortTracker -- \
  feature batch-update --ids "<eligible subset up to 7 ids>" --set-status verified --db porting.db --execute
```

---

### Task 3: Implement Group B Features (`1515-1519`)

**Files:**

- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamBatching.cs`
- Optional create/modify to isolate large method:
  - `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamBatching.HeaderChecks.cs`
- Optional supporting updates if required by dependencies:
  - `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs`
  - `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
  - `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamErrors.cs`
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/JetStreamBatchingCoreTests.cs`

**Step 1: Mark Group B as `stub` (single chunk, 5 IDs <= 15)**

```bash
dotnet run --project tools/NatsNet.PortTracker -- \
  feature batch-update --ids "1515,1516,1517,1518,1519" --set-status stub --db porting.db --execute
```

**Step 2: Execute Per-Feature Verification Loop for each ID in Group B**

Notes:

- Treat `1519` as its own micro-loop: implement incrementally but only promote when full gate passes.
- Validate error-path parity (duplicate msg ID, expected sequence checks, schedule/rollup validation, discard-new limits).

**Step 3: Task-level gates + checkpoint protocol**

Run:

```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~JetStreamBatching"
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~RaftTypesTests"
```

**Step 4: Promote eligible Group B IDs to `complete` (blocked IDs stay `deferred`)**

```bash
dotnet run --project tools/NatsNet.PortTracker -- \
  feature batch-update --ids "<eligible subset of 1515-1519>" --set-status complete --db porting.db --execute
```

**Step 5: After checkpoint evidence, promote eligible Group B IDs to `verified`**

```bash
dotnet run --project tools/NatsNet.PortTracker -- \
  feature batch-update --ids "<eligible subset up to 5 ids>" --set-status verified --db porting.db --execute
```

---

### Task 4: Implement Group C Tests (`2654,2674,2718`)

**Files:**

- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftNodeTests.cs`
- Optional modify if helper setup reuse is needed:
  - `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
- Source references:
  - `golang/nats-server/server/raft_test.go`

**Step 1: Mark tests as `stub`**

```bash
dotnet run --project tools/NatsNet.PortTracker -- \
  test batch-update --ids "2654,2674,2718" --set-status stub --db porting.db --execute
```

**Step 2: Execute Per-Test Verification Loop for each test ID**

Expected:

- Each mapped test method is real, discovered, and passing individually.

**Step 3: Task-level gates + checkpoint protocol**

Run:

```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~RaftNodeTests"
```

**Step 4: Promote eligible tests to `complete`**

```bash
dotnet run --project tools/NatsNet.PortTracker -- \
  test batch-update --ids "<eligible subset of 2654,2674,2718>" --set-status complete --db porting.db --execute
```

**Step 5: After checkpoint evidence, promote eligible tests to `verified`**

```bash
dotnet run --project tools/NatsNet.PortTracker -- \
  test batch-update --ids "<eligible subset up to 3 ids>" --set-status verified --db porting.db --execute
```

---

### Task 5: Batch 29 Final Verification and Closeout

**Files:**

- Modify: `porting.db`
- Generate: `reports/current.md` (via report script)

**Step 1: Final mandatory gates**

Run:

```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
dotnet run --project tools/NatsNet.PortTracker -- batch show 29 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db
```

Expected:

- All Batch 29 IDs are `verified` or `deferred` with explicit reasons.

**Step 2: Complete batch**

Run:

```bash
dotnet run --project tools/NatsNet.PortTracker -- batch complete 29 --db porting.db
```

**Step 3: Generate report + commit**

```bash
./reports/generate-report.sh
git add dotnet/src/ZB.MOM.NatsNet.Server/JetStream \
       dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream \
       porting.db reports/
git commit -m "feat(batch29): port jetstream batching and mapped raft tests"
```