Generated design docs and implementation plans via Codex for: - Batch 31: Raft Part 2 - Batch 32: JS Cluster Meta - Batch 33: JS Cluster Streams - Batch 34: JS Cluster Consumers - Batch 35: JS Cluster Remaining - Batch 36: Stream Lifecycle All plans include mandatory verification protocol and anti-stub guardrails. Updated batches.md with file paths and planned status.
445 lines
13 KiB
Markdown
445 lines
13 KiB
Markdown
# Batch 31 Raft Part 2 Implementation Plan
|
|
|
|
> **For Codex:** REQUIRED SUB-SKILL: Use `executeplan` to implement this plan task-by-task.
|
|
|
|
**Goal:** Port and verify Batch 31 (`Raft Part 2`) Raft behavior from `server/raft.go` with strict evidence gates for both features and tests.
|
|
|
|
**Architecture:** Implement 53 mapped Raft methods in three dependency-ordered feature groups (max 20 IDs each), then port 19 mapped tests in two waves. Use per-item red/green loops, mandatory stub scans, and chunked tracker updates (`<=15` IDs per batch command) so status moves only with executable proof.
|
|
|
|
**Tech Stack:** .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (`porting.db`)
|
|
|
|
**Design doc:** `docs/plans/2026-02-27-batch-31-raft-part-2-design.md`
|
|
|
|
---
|
|
|
|
## Batch 31 Scope
|
|
|
|
- Batch ID: `31`
|
|
- Name: `Raft Part 2`
|
|
- Dependency: `30`
|
|
- Go source: `golang/nats-server/server/raft.go`
|
|
- Features: `53`
|
|
- Tests: `19`
|
|
|
|
Primary implementation files:
|
|
|
|
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
|
|
- Optional split (if needed for reviewability):
|
|
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Catchup.cs`
|
|
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.AppendProcessing.cs`
|
|
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Elections.cs`
|
|
|
|
Primary test files:
|
|
|
|
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
|
- Modify (if shared helpers are needed):
|
|
- `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
|
|
- `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ImpltestsBacklogAssertions.cs`
|
|
|
|
---
|
|
|
|
## MANDATORY VERIFICATION PROTOCOL
|
|
|
|
> **NON-NEGOTIABLE:** Every feature ID and test ID in this batch must pass this protocol.
|
|
|
|
### Preflight Dependency Gate (before any status change)
|
|
|
|
1. Confirm dependency and readiness:
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- batch show 30 --db porting.db
|
|
dotnet run --project tools/NatsNet.PortTracker -- batch show 31 --db porting.db
|
|
dotnet run --project tools/NatsNet.PortTracker -- batch ready --db porting.db
|
|
```
|
|
|
|
2. Start only when Batch 31 is ready:
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- batch start 31 --db porting.db
|
|
```
|
|
|
|
3. Capture baseline build/tests:
|
|
|
|
```bash
|
|
dotnet build dotnet/
|
|
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
|
```
|
|
|
|
### Per-Feature Verification Loop (REQUIRED per feature ID)
|
|
|
|
1. Inspect mapping and Go span:
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- feature show <feature_id> --db porting.db
|
|
sed -n '<go_start>,<go_end>p' golang/nats-server/server/raft.go
|
|
```
|
|
|
|
2. Mark feature as in-progress (`stub` status in tracker only):
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status stub --db porting.db
|
|
```
|
|
|
|
3. Add/adjust a focused failing test for this behavior.
|
|
4. Run focused test and confirm it fails for the expected reason.
|
|
5. Implement minimum production change to make it pass.
|
|
6. Run **Build Gate**.
|
|
7. Run **Test Gate** for touched tests.
|
|
8. Run **Stub Detection Check**.
|
|
9. If all pass, mark feature `complete` (not `verified` yet).
|
|
|
|
### Per-Test Verification Loop (REQUIRED per test ID)
|
|
|
|
1. Inspect mapping and Go test span:
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- test show <test_id> --db porting.db
|
|
sed -n '<go_start>,<go_end>p' golang/nats-server/server/raft_test.go
|
|
```
|
|
|
|
2. Mark test as in-progress (`stub` status in tracker only):
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- test update <test_id> --status stub --db porting.db
|
|
```
|
|
|
|
3. Implement a real Arrange/Act/Assert test that calls production code.
|
|
4. Run single-test filter:
|
|
|
|
```bash
|
|
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
|
|
--filter "FullyQualifiedName~RaftNodeTests.<MethodName>" \
|
|
--verbosity normal
|
|
```
|
|
|
|
5. Confirm discovery + pass (`Passed: 1, Failed: 0`).
|
|
6. Run class-level filter for cumulative validation.
|
|
7. Run **Stub Detection Check**.
|
|
8. Mark test `complete` (promote to `verified` only at checkpoint).
|
|
|
|
### Stub Detection Check (REQUIRED after each loop and each task)
|
|
|
|
```bash
|
|
git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests \
|
|
| rg "\.cs$" \
|
|
| xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|Assert\.True\(true\)|Assert\.Pass|=>\s*default;|=>\s*null;|return\s+null;\s*$|return\s+0;\s*$|return\s+false;\s*$)"
|
|
```
|
|
|
|
Any hit in mapped methods/tests blocks promotion.
|
|
|
|
### Build Gate (REQUIRED)
|
|
|
|
Run `dotnet build dotnet/`:
|
|
|
|
- after each feature/test loop,
|
|
- before any batch status update,
|
|
- at each task checkpoint.
|
|
|
|
### Test Gate (REQUIRED)
|
|
|
|
Minimum required gates for touched Raft scope:
|
|
|
|
```bash
|
|
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~RaftTypes"
|
|
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~RaftNodeTests"
|
|
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog"
|
|
```
|
|
|
|
Checkpoint and final gate:
|
|
|
|
```bash
|
|
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
|
```
|
|
|
|
### Status Update Protocol (HARD LIMIT: <=15 IDs per batch-update)
|
|
|
|
- Allowed flow: `deferred/not_started -> stub -> complete -> verified`
|
|
- Never include more than 15 IDs in one `feature/test batch-update` command.
|
|
- Never mark `verified` without passing Build Gate + Test Gate + Stub Detection Check.
|
|
- Apply updates only for IDs in the active task.
|
|
|
|
Templates:
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- \
|
|
feature batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
|
|
|
|
dotnet run --project tools/NatsNet.PortTracker -- \
|
|
test batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
|
|
```
|
|
|
|
### Checkpoint Protocol Between Tasks (REQUIRED)
|
|
|
|
At end of every task before starting the next:
|
|
|
|
1. Run Stub Detection Check.
|
|
2. Run Build Gate.
|
|
3. Run focused Test Gate for touched classes.
|
|
4. Run full unit test suite.
|
|
5. Apply status updates for current task only (`<=15` IDs per command).
|
|
6. Commit checkpoint.
|
|
|
|
---
|
|
|
|
## ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)
|
|
|
|
### Forbidden Patterns (Features + Tests)
|
|
|
|
Any of these in mapped/touched files is disallowed:
|
|
|
|
- `throw new NotImplementedException()`
|
|
- Empty or no-op method bodies for mapped behaviors
|
|
- Placeholder comments (`// TODO`, `// PLACEHOLDER`, `// later`)
|
|
- Fake-pass assertions (`Assert.True(true)`, `Assert.Pass()`)
|
|
- Tests that only assert non-null/default without behavior validation
|
|
- Constant-return placeholders for complex logic (`return null;`, `return 0;`, `return false;`, `return string.Empty;`)
|
|
- Blanket `catch` blocks that swallow failures to force pass
|
|
|
|
### Hard Limits
|
|
|
|
- Max feature IDs per implementation group: `20`
|
|
- Max IDs per `feature/test batch-update`: `15`
|
|
- One active feature loop at a time
|
|
- One active test loop at a time
|
|
- No `verified` transition without full gate evidence
|
|
- Mandatory checkpoint between tasks
|
|
|
|
### If You Get Stuck (REQUIRED)
|
|
|
|
1. Stop work on that ID immediately.
|
|
2. Do not leave placeholder code or fake-pass assertions.
|
|
3. Mark item `deferred` with explicit reason:
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- feature update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
|
|
dotnet run --project tools/NatsNet.PortTracker -- test update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
|
|
```
|
|
|
|
4. Add brief note in tracker/comment where needed.
|
|
5. Continue with next unblocked ID in the same task.
|
|
|
|
Deferred-with-reason is valid. Stubs are not.
|
|
|
|
---
|
|
|
|
## Feature Groups (max ~20 each)
|
|
|
|
### Group A (18) - Catchup/snapshot/commit foundations
|
|
|
|
`2733,2734,2735,2736,2737,2738,2739,2740,2741,2742,2743,2744,2745,2746,2747,2748,2749,2750`
|
|
|
|
### Group B (18) - Append processing, WAL, peer state
|
|
|
|
`2751,2752,2753,2754,2755,2756,2758,2759,2760,2761,2765,2766,2767,2768,2769,2772,2776,2777`
|
|
|
|
### Group C (17) - Vote/RPC/state transitions
|
|
|
|
`2778,2779,2780,2783,2784,2785,2786,2787,2788,2789,2790,2791,2792,2793,2794,2795,2796`
|
|
|
|
## Test Waves
|
|
|
|
### T1 (10) - Election/quorum/state correctness
|
|
|
|
`2626,2629,2635,2636,2663,2664,2667,2687,2690,2692`
|
|
|
|
### T2 (9) - Catchup/snapshot/membership and vote retention
|
|
|
|
`2650,2651,2693,2694,2702,2704,2705,2712,2714`
|
|
|
|
---
|
|
|
|
### Task 1: Preflight and Baseline
|
|
|
|
**Files:**
|
|
|
|
- Read: `docs/plans/2026-02-27-batch-31-raft-part-2-design.md`
|
|
- Read: `golang/nats-server/server/raft.go`
|
|
- Read: `golang/nats-server/server/raft_test.go`
|
|
|
|
**Step 1: Run Preflight Dependency Gate**
|
|
|
|
Run all preflight commands in protocol.
|
|
|
|
**Step 2: Start batch and capture baseline**
|
|
|
|
Run `batch start 31`, `dotnet build`, and full unit tests.
|
|
|
|
**Step 3: Checkpoint protocol**
|
|
|
|
Complete checkpoint before feature work.
|
|
|
|
---
|
|
|
|
### Task 2: Implement Feature Group A (18)
|
|
|
|
**Files:**
|
|
|
|
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Catchup.cs`
|
|
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
|
|
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
|
|
|
|
**Step 1: Mark Group A IDs as `stub` in <=15-ID chunks**
|
|
|
|
Chunk 1:
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- \
|
|
feature batch-update --ids "2733,2734,2735,2736,2737,2738,2739,2740,2741,2742,2743,2744,2745,2746,2747" --set-status stub --db porting.db --execute
|
|
```
|
|
|
|
Chunk 2:
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- \
|
|
feature batch-update --ids "2748,2749,2750" --set-status stub --db porting.db --execute
|
|
```
|
|
|
|
**Step 2: Execute Per-Feature Verification Loop for each Group A ID**
|
|
|
|
**Step 3: Run task checkpoint gates**
|
|
|
|
**Step 4: Promote eligible Group A IDs to `complete` then `verified` in <=15-ID chunks**
|
|
|
|
---
|
|
|
|
### Task 3: Implement Feature Group B (18)
|
|
|
|
**Files:**
|
|
|
|
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.AppendProcessing.cs`
|
|
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
|
|
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
|
|
|
|
**Step 1: Mark Group B IDs as `stub` in <=15-ID chunks**
|
|
|
|
Chunk 1:
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- \
|
|
feature batch-update --ids "2751,2752,2753,2754,2755,2756,2758,2759,2760,2761,2765,2766,2767,2768,2769" --set-status stub --db porting.db --execute
|
|
```
|
|
|
|
Chunk 2:
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- \
|
|
feature batch-update --ids "2772,2776,2777" --set-status stub --db porting.db --execute
|
|
```
|
|
|
|
**Step 2: Execute Per-Feature Verification Loop for each Group B ID**
|
|
|
|
**Step 3: Run task checkpoint gates**
|
|
|
|
**Step 4: Promote eligible Group B IDs to `complete` then `verified` in <=15-ID chunks**
|
|
|
|
---
|
|
|
|
### Task 4: Implement Feature Group C (17)
|
|
|
|
**Files:**
|
|
|
|
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Elections.cs`
|
|
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
|
|
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
|
|
|
|
**Step 1: Mark Group C IDs as `stub` in <=15-ID chunks**
|
|
|
|
Chunk 1:
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- \
|
|
feature batch-update --ids "2778,2779,2780,2783,2784,2785,2786,2787,2788,2789,2790,2791,2792,2793,2794" --set-status stub --db porting.db --execute
|
|
```
|
|
|
|
Chunk 2:
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- \
|
|
feature batch-update --ids "2795,2796" --set-status stub --db porting.db --execute
|
|
```
|
|
|
|
**Step 2: Execute Per-Feature Verification Loop for each Group C ID**
|
|
|
|
**Step 3: Run task checkpoint gates**
|
|
|
|
**Step 4: Promote eligible Group C IDs to `complete` then `verified` in <=15-ID chunks**
|
|
|
|
---
|
|
|
|
### Task 5: Port Test Wave T1 (10)
|
|
|
|
**Files:**
|
|
|
|
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
|
- Modify (if helpers required): `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ImpltestsBacklogAssertions.cs`
|
|
|
|
**Step 1: Mark T1 IDs as `stub`**
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- \
|
|
test batch-update --ids "2626,2629,2635,2636,2663,2664,2667,2687,2690,2692" --set-status stub --db porting.db --execute
|
|
```
|
|
|
|
**Step 2: Execute Per-Test Verification Loop for each T1 ID**
|
|
|
|
**Step 3: Run class-level + checkpoint gates**
|
|
|
|
**Step 4: Promote eligible T1 IDs to `complete` then `verified`**
|
|
|
|
---
|
|
|
|
### Task 6: Port Test Wave T2 (9)
|
|
|
|
**Files:**
|
|
|
|
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
|
|
|
**Step 1: Mark T2 IDs as `stub`**
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- \
|
|
test batch-update --ids "2650,2651,2693,2694,2702,2704,2705,2712,2714" --set-status stub --db porting.db --execute
|
|
```
|
|
|
|
**Step 2: Execute Per-Test Verification Loop for each T2 ID**
|
|
|
|
**Step 3: Run class-level + checkpoint gates**
|
|
|
|
**Step 4: Promote eligible T2 IDs to `complete` then `verified`**
|
|
|
|
---
|
|
|
|
### Task 7: Final Batch 31 Verification and Completion
|
|
|
|
**Files:**
|
|
|
|
- Modify: `porting.db`
|
|
- Modify: `reports/current.md` (via report script)
|
|
|
|
**Step 1: Run final mandatory gates**
|
|
|
|
```bash
|
|
dotnet build dotnet/
|
|
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
|
dotnet run --project tools/NatsNet.PortTracker -- audit --type features --db porting.db
|
|
dotnet run --project tools/NatsNet.PortTracker -- audit --type tests --db porting.db
|
|
```
|
|
|
|
**Step 2: Resolve any remaining `stub` statuses**
|
|
|
|
- Convert to `verified` with evidence, or
|
|
- Convert to `deferred` with explicit blocker reason.
|
|
|
|
**Step 3: Complete the batch**
|
|
|
|
```bash
|
|
dotnet run --project tools/NatsNet.PortTracker -- batch complete 31 --db porting.db
|
|
```
|
|
|
|
**Step 4: Refresh report**
|
|
|
|
```bash
|
|
./reports/generate-report.sh
|
|
```
|
|
|