Files
natsnet/docs/plans/2026-02-27-batch-31-raft-part-2-implementation-plan.md
Joseph Doherty f8dce79ac0 Add batch plans for batches 31-36 (rounds 16-18)
Generated design docs and implementation plans via Codex for:
- Batch 31: Raft Part 2
- Batch 32: JS Cluster Meta
- Batch 33: JS Cluster Streams
- Batch 34: JS Cluster Consumers
- Batch 35: JS Cluster Remaining
- Batch 36: Stream Lifecycle

All plans include mandatory verification protocol and anti-stub guardrails.
Updated batches.md with file paths and planned status.
2026-02-27 17:01:31 -05:00

445 lines
13 KiB
Markdown

# Batch 31 Raft Part 2 Implementation Plan
> **For Codex:** REQUIRED SUB-SKILL: Use `executeplan` to implement this plan task-by-task.
**Goal:** Port and verify Batch 31 (`Raft Part 2`) Raft behavior from `server/raft.go` with strict evidence gates for both features and tests.
**Architecture:** Implement 53 mapped Raft methods in three dependency-ordered feature groups (max 20 IDs each), then port 19 mapped tests in two waves. Use per-item red/green loops, mandatory stub scans, and chunked tracker updates (`<=15` IDs per batch command) so status moves only with executable proof.
**Tech Stack:** .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (`porting.db`)
**Design doc:** `docs/plans/2026-02-27-batch-31-raft-part-2-design.md`
---
## Batch 31 Scope
- Batch ID: `31`
- Name: `Raft Part 2`
- Dependency: `30`
- Go source: `golang/nats-server/server/raft.go`
- Features: `53`
- Tests: `19`
Primary implementation files:
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
- Optional split (if needed for reviewability):
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Catchup.cs`
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.AppendProcessing.cs`
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Elections.cs`
Primary test files:
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
- Modify (if shared helpers are needed):
- `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
- `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ImpltestsBacklogAssertions.cs`
---
## MANDATORY VERIFICATION PROTOCOL
> **NON-NEGOTIABLE:** Every feature ID and test ID in this batch must pass this protocol.
### Preflight Dependency Gate (before any status change)
1. Confirm dependency and readiness:
```bash
dotnet run --project tools/NatsNet.PortTracker -- batch show 30 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- batch show 31 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- batch ready --db porting.db
```
2. Start only when Batch 31 is ready:
```bash
dotnet run --project tools/NatsNet.PortTracker -- batch start 31 --db porting.db
```
3. Capture baseline build/tests:
```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
```
### Per-Feature Verification Loop (REQUIRED per feature ID)
1. Inspect mapping and Go span:
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature show <feature_id> --db porting.db
sed -n '<go_start>,<go_end>p' golang/nats-server/server/raft.go
```
2. Mark feature as in-progress (`stub` status in tracker only):
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status stub --db porting.db
```
3. Add/adjust a focused failing test for this behavior.
4. Run focused test and confirm it fails for the expected reason.
5. Implement minimum production change to make it pass.
6. Run **Build Gate**.
7. Run **Test Gate** for touched tests.
8. Run **Stub Detection Check**.
9. If all pass, mark feature `complete` (not `verified` yet).
### Per-Test Verification Loop (REQUIRED per test ID)
1. Inspect mapping and Go test span:
```bash
dotnet run --project tools/NatsNet.PortTracker -- test show <test_id> --db porting.db
sed -n '<go_start>,<go_end>p' golang/nats-server/server/raft_test.go
```
2. Mark test as in-progress (`stub` status in tracker only):
```bash
dotnet run --project tools/NatsNet.PortTracker -- test update <test_id> --status stub --db porting.db
```
3. Implement a real Arrange/Act/Assert test that calls production code.
4. Run single-test filter:
```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
--filter "FullyQualifiedName~RaftNodeTests.<MethodName>" \
--verbosity normal
```
5. Confirm discovery + pass (`Passed: 1, Failed: 0`).
6. Run class-level filter for cumulative validation.
7. Run **Stub Detection Check**.
8. Mark test `complete` (promote to `verified` only at checkpoint).
### Stub Detection Check (REQUIRED after each loop and each task)
```bash
git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests \
| rg "\.cs$" \
| xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|Assert\.True\(true\)|Assert\.Pass|=>\s*default;|=>\s*null;|return\s+null;\s*$|return\s+0;\s*$|return\s+false;\s*$)"
```
Any hit in mapped methods/tests blocks promotion.
### Build Gate (REQUIRED)
Run `dotnet build dotnet/`:
- after each feature/test loop,
- before any batch status update,
- at each task checkpoint.
### Test Gate (REQUIRED)
Minimum required gates for touched Raft scope:
```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~RaftTypes"
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~RaftNodeTests"
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog"
```
Checkpoint and final gate:
```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
```
### Status Update Protocol (HARD LIMIT: <=15 IDs per batch-update)
- Allowed flow: `deferred/not_started -> stub -> complete -> verified`
- Never include more than 15 IDs in one `feature/test batch-update` command.
- Never mark `verified` without passing Build Gate + Test Gate + Stub Detection Check.
- Apply updates only for IDs in the active task.
Templates:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
dotnet run --project tools/NatsNet.PortTracker -- \
test batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
```
### Checkpoint Protocol Between Tasks (REQUIRED)
At end of every task before starting the next:
1. Run Stub Detection Check.
2. Run Build Gate.
3. Run focused Test Gate for touched classes.
4. Run full unit test suite.
5. Apply status updates for current task only (`<=15` IDs per command).
6. Commit checkpoint.
---
## ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)
### Forbidden Patterns (Features + Tests)
Any of these in mapped/touched files is disallowed:
- `throw new NotImplementedException()`
- Empty or no-op method bodies for mapped behaviors
- Placeholder comments (`// TODO`, `// PLACEHOLDER`, `// later`)
- Fake-pass assertions (`Assert.True(true)`, `Assert.Pass()`)
- Tests that only assert non-null/default without behavior validation
- Constant-return placeholders for complex logic (`return null;`, `return 0;`, `return false;`, `return string.Empty;`)
- Blanket `catch` blocks that swallow failures to force pass
### Hard Limits
- Max feature IDs per implementation group: `20`
- Max IDs per `feature/test batch-update`: `15`
- One active feature loop at a time
- One active test loop at a time
- No `verified` transition without full gate evidence
- Mandatory checkpoint between tasks
### If You Get Stuck (REQUIRED)
1. Stop work on that ID immediately.
2. Do not leave placeholder code or fake-pass assertions.
3. Mark item `deferred` with explicit reason:
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- test update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
```
4. Add brief note in tracker/comment where needed.
5. Continue with next unblocked ID in the same task.
Deferred-with-reason is valid. Stubs are not.
---
## Feature Groups (max ~20 each)
### Group A (18) - Catchup/snapshot/commit foundations
`2733,2734,2735,2736,2737,2738,2739,2740,2741,2742,2743,2744,2745,2746,2747,2748,2749,2750`
### Group B (18) - Append processing, WAL, peer state
`2751,2752,2753,2754,2755,2756,2758,2759,2760,2761,2765,2766,2767,2768,2769,2772,2776,2777`
### Group C (17) - Vote/RPC/state transitions
`2778,2779,2780,2783,2784,2785,2786,2787,2788,2789,2790,2791,2792,2793,2794,2795,2796`
## Test Waves
### T1 (10) - Election/quorum/state correctness
`2626,2629,2635,2636,2663,2664,2667,2687,2690,2692`
### T2 (9) - Catchup/snapshot/membership and vote retention
`2650,2651,2693,2694,2702,2704,2705,2712,2714`
---
### Task 1: Preflight and Baseline
**Files:**
- Read: `docs/plans/2026-02-27-batch-31-raft-part-2-design.md`
- Read: `golang/nats-server/server/raft.go`
- Read: `golang/nats-server/server/raft_test.go`
**Step 1: Run Preflight Dependency Gate**
Run all preflight commands in protocol.
**Step 2: Start batch and capture baseline**
Run `batch start 31`, `dotnet build`, and full unit tests.
**Step 3: Checkpoint protocol**
Complete checkpoint before feature work.
---
### Task 2: Implement Feature Group A (18)
**Files:**
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Catchup.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
**Step 1: Mark Group A IDs as `stub` in <=15-ID chunks**
Chunk 1:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "2733,2734,2735,2736,2737,2738,2739,2740,2741,2742,2743,2744,2745,2746,2747" --set-status stub --db porting.db --execute
```
Chunk 2:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "2748,2749,2750" --set-status stub --db porting.db --execute
```
**Step 2: Execute Per-Feature Verification Loop for each Group A ID**
**Step 3: Run task checkpoint gates**
**Step 4: Promote eligible Group A IDs to `complete` then `verified` in <=15-ID chunks**
---
### Task 3: Implement Feature Group B (18)
**Files:**
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.AppendProcessing.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
**Step 1: Mark Group B IDs as `stub` in <=15-ID chunks**
Chunk 1:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "2751,2752,2753,2754,2755,2756,2758,2759,2760,2761,2765,2766,2767,2768,2769" --set-status stub --db porting.db --execute
```
Chunk 2:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "2772,2776,2777" --set-status stub --db porting.db --execute
```
**Step 2: Execute Per-Feature Verification Loop for each Group B ID**
**Step 3: Run task checkpoint gates**
**Step 4: Promote eligible Group B IDs to `complete` then `verified` in <=15-ID chunks**
---
### Task 4: Implement Feature Group C (17)
**Files:**
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Elections.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
**Step 1: Mark Group C IDs as `stub` in <=15-ID chunks**
Chunk 1:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "2778,2779,2780,2783,2784,2785,2786,2787,2788,2789,2790,2791,2792,2793,2794" --set-status stub --db porting.db --execute
```
Chunk 2:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "2795,2796" --set-status stub --db porting.db --execute
```
**Step 2: Execute Per-Feature Verification Loop for each Group C ID**
**Step 3: Run task checkpoint gates**
**Step 4: Promote eligible Group C IDs to `complete` then `verified` in <=15-ID chunks**
---
### Task 5: Port Test Wave T1 (10)
**Files:**
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
- Modify (if helpers required): `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ImpltestsBacklogAssertions.cs`
**Step 1: Mark T1 IDs as `stub`**
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
test batch-update --ids "2626,2629,2635,2636,2663,2664,2667,2687,2690,2692" --set-status stub --db porting.db --execute
```
**Step 2: Execute Per-Test Verification Loop for each T1 ID**
**Step 3: Run class-level + checkpoint gates**
**Step 4: Promote eligible T1 IDs to `complete` then `verified`**
---
### Task 6: Port Test Wave T2 (9)
**Files:**
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
**Step 1: Mark T2 IDs as `stub`**
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
test batch-update --ids "2650,2651,2693,2694,2702,2704,2705,2712,2714" --set-status stub --db porting.db --execute
```
**Step 2: Execute Per-Test Verification Loop for each T2 ID**
**Step 3: Run class-level + checkpoint gates**
**Step 4: Promote eligible T2 IDs to `complete` then `verified`**
---
### Task 7: Final Batch 31 Verification and Completion
**Files:**
- Modify: `porting.db`
- Modify: `reports/current.md` (via report script)
**Step 1: Run final mandatory gates**
```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
dotnet run --project tools/NatsNet.PortTracker -- audit --type features --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- audit --type tests --db porting.db
```
**Step 2: Resolve any remaining `stub` statuses**
- Convert to `verified` with evidence, or
- Convert to `deferred` with explicit blocker reason.
**Step 3: Complete the batch**
```bash
dotnet run --project tools/NatsNet.PortTracker -- batch complete 31 --db porting.db
```
**Step 4: Refresh report**
```bash
./reports/generate-report.sh
```