Files
natsnet/docs/plans/2026-02-27-batch-30-raft-part-1-implementation-plan.md
Joseph Doherty c05d93618e Add batch plans for batches 23-30 (rounds 12-15)
Generated design docs and implementation plans via Codex for:
- Batch 23: Routes
- Batch 24: Leaf Nodes
- Batch 25: Gateways
- Batch 26: WebSocket
- Batch 27: JetStream Core
- Batch 28: JetStream API
- Batch 29: JetStream Batching
- Batch 30: Raft Part 1

All plans include mandatory verification protocol and anti-stub guardrails.
Updated batches.md with file paths and planned status.
2026-02-27 16:33:10 -05:00

551 lines
17 KiB
Markdown

# Batch 30 Raft Part 1 Implementation Plan
> **For Codex:** REQUIRED SUB-SKILL: Use `executeplan` to implement this plan task-by-task.
**Goal:** Port and verify Batch 30 (`Raft Part 1`) features from `server/raft.go` with evidence-backed status updates, while processing mapped tests with strict no-stub discipline.
**Architecture:** Implement the 85 features in five Raft-focused groups (max 20 each), progressing from server bootstrap/state setup to follower/leader loops and codec/persistence helpers. Use a dual-track test strategy: direct raft tests first, then mapped transitive regressions by class with explicit defer reasons when infrastructure is unavailable.
**Tech Stack:** .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (`porting.db`)
**Design doc:** `docs/plans/2026-02-27-batch-30-raft-part-1-design.md`
---
## Batch 30 Scope
- Batch ID: `30`
- Name: `Raft Part 1`
- Dependencies: `4,18`
- Go source: `golang/nats-server/server/raft.go`
- Features: `85`
- Tests: `414`
Primary implementation files:
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
- Create/Modify (recommended split for reviewability):
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.ServerIntegration.cs`
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Snapshots.cs`
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.RunLoop.cs`
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Codecs.cs`
- Create/Modify for server-level mapped methods:
- `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.Raft.cs`
- Modify (remove dangling TODO when method exists):
- `dotnet/src/ZB.MOM.NatsNet.Server/Config/ReloadOptions.cs`
Primary test files:
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftNodeCoreTests.cs`
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/Server/NatsServerRaftTests.cs`
- Create/Modify mapped backlog classes as needed for Batch 30 test IDs:
- `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
- existing `ImplBacklog/*.Impltests.cs` files for mapped IDs in this batch
---
## MANDATORY VERIFICATION PROTOCOL
> **NON-NEGOTIABLE:** Applies to every feature ID and test ID touched in Batch 30.
### Preflight Dependency Gate (REQUIRED before Task 1 coding)
1. Verify dependency and readiness:
```bash
dotnet run --project tools/NatsNet.PortTracker -- batch show 4 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- batch show 18 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- batch show 30 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- batch ready --db porting.db
```
2. Start batch only when ready:
```bash
dotnet run --project tools/NatsNet.PortTracker -- batch start 30 --db porting.db
```
3. Capture baseline:
```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
```
### Per-Feature Verification Loop (REQUIRED for each feature ID)
For each feature in active task group:
1. Inspect mapping and Go span:
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature show <feature_id> --db porting.db
sed -n '<go_start>,<go_end>p' golang/nats-server/server/raft.go
```
2. Mark feature `stub` before editing:
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status stub --db porting.db
```
3. Add or update a focused test that fails first for that feature behavior.
4. Run that focused test to confirm red.
5. Implement minimal production code for green.
6. Run Build Gate.
7. Run Test Gate (focused).
8. Run Stub Detection Check.
9. If all gates are green, mark feature `complete`:
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status complete --db porting.db
```
10. Promote to `verified` only at task checkpoint after cumulative evidence is clean.
### Per-Test Verification Loop (REQUIRED for each test ID)
1. Inspect test mapping and Go source:
```bash
dotnet run --project tools/NatsNet.PortTracker -- test show <test_id> --db porting.db
sed -n '<go_start>,<go_end>p' <go_test_file>
```
2. Mark test `stub` before editing:
```bash
dotnet run --project tools/NatsNet.PortTracker -- test update <test_id> --status stub --db porting.db
```
3. Write real Arrange/Act/Assert test that invokes production code.
4. Run single test:
```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
--filter "FullyQualifiedName~<ClassName>.<MethodName>" \
--verbosity normal
```
5. Confirm `Passed: 1, Failed: 0` (if `Passed: 0`, test discovery failed).
6. Run class-level filter for cumulative validation.
7. Run Stub Detection Check.
8. Mark `complete` only after test evidence is real; mark `verified` only at checkpoint.
### Stub Detection Check (REQUIRED after each feature/test loop and each task)
Run against touched C# files:
```bash
git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests \
| rg "\.cs$" \
| xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|Assert\.True\(true\)|Assert\.Pass|=>\s*default;|=>\s*null;|return\s+null;\s*$|return\s+0;\s*$|return\s+false;\s*$)"
```
Any hit in touched mapped methods/tests blocks promotion to `complete` or `verified`.
### Build Gate (REQUIRED)
Run `dotnet build dotnet/`:
- after each feature loop,
- before every status batch update,
- at each task checkpoint.
### Test Gate (REQUIRED)
Minimum focused gates per task:
```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~Raft"
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~NatsServerRaft"
```
For touched ImplBacklog classes:
```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.<ClassName>"
```
Checkpoint and final gates:
```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
```
### Status Update Protocol (HARD LIMIT: max 15 IDs per batch-update)
- Allowed progression: `deferred/not_started -> stub -> complete -> verified`
- Use `feature/test batch-update` with at most `15` IDs per command.
- Never update IDs outside current task/group.
- Never mark `verified` without evidence from build + tests + stub scan.
Templates:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
dotnet run --project tools/NatsNet.PortTracker -- \
test batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
```
### Checkpoint Protocol Between Tasks (REQUIRED)
At the end of every task:
1. Run Stub Detection Check.
2. Run Build Gate.
3. Run focused Test Gate for touched areas.
4. Run full unit test suite.
5. Apply status updates for current task only (max 15 IDs per command).
6. Commit checkpoint.
---
## ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)
### Forbidden Patterns (Features + Tests)
Any of these in touched mapped code is a stub and blocks promotion:
- `throw new NotImplementedException()`
- Empty mapped method bodies for non-trivial behavior
- Placeholder comments (`// TODO`, `// PLACEHOLDER`, `// later`)
- Fake-pass assertions (`Assert.True(true)`, `Assert.Pass()`)
- Tests that do not call production code
- Constant-return placeholders for complex logic (`return null;`, `return 0;`, `return false;`, `return string.Empty;`)
- Catch-all exception swallowing that forces tests to pass
- Reflection-only assertions that never validate observable raft behavior
### Hard Limits
- Max feature IDs per implementation group: `~20`
- Max IDs per `feature/test batch-update`: `15`
- One active feature loop at a time
- One active test loop at a time
- No `verified` status without green build + tests + stub scan evidence
- Mandatory checkpoint after each task before moving forward
### If You Get Stuck
1. Stop the blocked item immediately.
2. Do not leave placeholder implementation or fake-pass tests.
3. Mark blocked item `deferred` with explicit reason:
```bash
dotnet run --project tools/NatsNet.PortTracker -- feature update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- test update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
```
4. Add a short code comment only where needed to explain defer reason in test source.
5. Continue with next unblocked ID in the same group.
Deferred-with-reason is correct. Stubs are not.
---
## Feature Groups (max ~20 each)
### Group A (15)
`2599,2600,2601,2602,2603,2607,2608,2609,2610,2611,2612,2613,2614,2615,2629`
### Group B (14)
`2634,2637,2639,2645,2646,2647,2651,2652,2653,2659,2663,2664,2665,2674`
### Group C (20)
`2683,2684,2685,2686,2687,2688,2689,2690,2691,2692,2693,2694,2695,2696,2697,2698,2701,2702,2703,2704`
### Group D (19)
`2705,2706,2707,2708,2709,2710,2711,2712,2714,2715,2716,2717,2718,2719,2720,2721,2722,2723,2724`
### Group E (17)
`2725,2727,2728,2729,2731,2732,2757,2762,2763,2764,2770,2771,2773,2774,2775,2781,2782`
---
## Test Groups
### T1: Direct Raft tests (19 IDs, highest priority)
`2618,2619,2621,2623,2625,2632,2633,2639,2642,2675,2700,2701,2706,2707,2708,2709,2710,2711,2713`
### T2: Transitive mapped regressions (395 IDs)
- Process class-by-class from mapped ImplBacklog classes (`JetStreamClusterTests*`, `JetStreamSuperClusterTests`, `ConcurrencyTests*`, `MqttHandlerTests`, etc.).
- Replace placeholder tests only when real deterministic verification is possible.
- Otherwise keep `deferred` with explicit reason.
---
### Task 1: Preflight, Batch Start, and Baseline
**Files:**
- Read: `docs/standards/dotnet-standards.md`
- Read: `docs/plans/2026-02-27-batch-30-raft-part-1-design.md`
- Read: `golang/nats-server/server/raft.go`
**Step 1: Run preflight dependency gate**
Run the Preflight Dependency Gate commands.
**Step 2: Start Batch 30**
Run `batch start 30` command.
**Step 3: Capture baseline build and tests**
Run baseline build/test commands.
**Step 4: Checkpoint protocol and commit**
---
### Task 2: Implement Feature Group A (15 features)
**Files:**
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.Raft.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/Config/ReloadOptions.cs`
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/Server/NatsServerRaftTests.cs`
**Step 1: Mark Group A features as `stub` (split into max-15 chunks)**
Chunk 1:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "2599,2600,2601,2602,2603,2607,2608,2609,2610,2611,2612,2613,2614,2615,2629" --set-status stub --db porting.db --execute
```
**Step 2: Execute Per-Feature Verification Loop for each Group A ID**
**Step 3: Run task-level gates and checkpoint protocol**
**Step 4: Promote eligible Group A IDs to `complete` then `verified` in <=15-ID chunks**
---
### Task 3: Implement Feature Group B (14 features)
**Files:**
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Snapshots.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftNodeCoreTests.cs`
**Step 1: Mark Group B as `stub`**
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "2634,2637,2639,2645,2646,2647,2651,2652,2653,2659,2663,2664,2665,2674" --set-status stub --db porting.db --execute
```
**Step 2: Execute Per-Feature Verification Loop for each Group B ID**
**Step 3: Run task-level gates and checkpoint protocol**
**Step 4: Promote eligible Group B IDs to `complete` then `verified`**
---
### Task 4: Implement Feature Group C (20 features)
**Files:**
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.RunLoop.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftNodeCoreTests.cs`
**Step 1: Mark first 15 Group C IDs as `stub`**
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "2683,2684,2685,2686,2687,2688,2689,2690,2691,2692,2693,2694,2695,2696,2697" --set-status stub --db porting.db --execute
```
**Step 2: Mark remaining 5 Group C IDs as `stub`**
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "2698,2701,2702,2703,2704" --set-status stub --db porting.db --execute
```
**Step 3: Execute Per-Feature Verification Loop for all Group C IDs**
**Step 4: Run task-level gates and checkpoint protocol**
**Step 5: Promote eligible Group C IDs to `complete` then `verified` in <=15-ID chunks**
---
### Task 5: Implement Feature Group D (19 features)
**Files:**
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Codecs.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
**Step 1: Mark Group D features `stub` in two chunks**
Chunk 1:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "2705,2706,2707,2708,2709,2710,2711,2712,2714,2715,2716,2717,2718,2719,2720" --set-status stub --db porting.db --execute
```
Chunk 2:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "2721,2722,2723,2724" --set-status stub --db porting.db --execute
```
**Step 2: Execute Per-Feature Verification Loop for all Group D IDs**
**Step 3: Run task-level gates and checkpoint protocol**
**Step 4: Promote eligible Group D IDs to `complete` then `verified`**
---
### Task 6: Implement Feature Group E (17 features)
**Files:**
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.RunLoop.cs`
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Codecs.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftNodeCoreTests.cs`
**Step 1: Mark Group E features `stub` in two chunks**
Chunk 1:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "2725,2727,2728,2729,2731,2732,2757,2762,2763,2764,2770,2771,2773,2774,2775" --set-status stub --db porting.db --execute
```
Chunk 2:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
feature batch-update --ids "2781,2782" --set-status stub --db porting.db --execute
```
**Step 2: Execute Per-Feature Verification Loop for all Group E IDs**
**Step 3: Run task-level gates and checkpoint protocol**
**Step 4: Promote eligible Group E IDs to `complete` then `verified`**
---
### Task 7: Port and Verify T1 Direct Raft Tests (19 tests)
**Files:**
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
**Step 1: Mark T1 tests `stub` in <=15-ID chunks**
Chunk 1:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
test batch-update --ids "2618,2619,2621,2623,2625,2632,2633,2639,2642,2675,2700,2701,2706,2707,2708" --set-status stub --db porting.db --execute
```
Chunk 2:
```bash
dotnet run --project tools/NatsNet.PortTracker -- \
test batch-update --ids "2709,2710,2711,2713" --set-status stub --db porting.db --execute
```
**Step 2: Execute Per-Test Verification Loop for each T1 test ID**
**Step 3: Run class-level and task-level test gates**
**Step 4: Promote eligible T1 tests to `complete` then `verified` in <=15-ID chunks**
---
### Task 8: Process T2 Transitive Mapped Tests (395 tests)
**Files:**
- Modify only touched mapped files under `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/`
**Step 1: Generate T2 working manifest (exclude T1 IDs)**
```bash
sqlite3 -header -column porting.db "
SELECT ut.id, ut.dotnet_class, ut.dotnet_method, ut.go_file, ut.go_line_number
FROM batch_tests bt
JOIN unit_tests ut ON ut.id=bt.test_id
WHERE bt.batch_id=30
AND ut.id NOT IN (2618,2619,2621,2623,2625,2632,2633,2639,2642,2675,2700,2701,2706,2707,2708,2709,2710,2711,2713)
ORDER BY ut.dotnet_class, ut.id;"
```
**Step 2: Work class-by-class using Per-Test Verification Loop**
**Step 3: For blocked tests, apply If-You-Get-Stuck protocol (keep deferred with reason, do not stub)**
**Step 4: Apply status updates in <=15-ID chunks for truly verified tests only**
**Step 5: Run checkpoint protocol after each class**
---
### Task 9: Final Verification, Audit, and Batch Readiness Review
**Files:**
- Modify: `porting.db`
- Optional update: `reports/current.md` (via report script)
**Step 1: Run full gates**
```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
```
**Step 2: Run audit checks**
```bash
dotnet run --project tools/NatsNet.PortTracker -- audit --type features --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- audit --type tests --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- batch show 30 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db
```
**Step 3: Complete batch only if all items satisfy completion criteria**
```bash
dotnet run --project tools/NatsNet.PortTracker -- batch complete 30 --db porting.db
```
If completion criteria fail, keep blocked IDs deferred with explicit reasons and stop; do not force status changes.
---
## Execution Notes
- If `dotnet` is not on PATH in your shell, use `/usr/local/share/dotnet/dotnet` as a drop-in replacement for commands above.
- Do not run implementation from this planning session.