Add batch plans for batches 31-36 (rounds 16-18)
Generated design docs and implementation plans via Codex for: - Batch 31: Raft Part 2 - Batch 32: JS Cluster Meta - Batch 33: JS Cluster Streams - Batch 34: JS Cluster Consumers - Batch 35: JS Cluster Remaining - Batch 36: Stream Lifecycle All plans include mandatory verification protocol and anti-stub guardrails. Updated batches.md with file paths and planned status.
This commit is contained in:
Joseph Doherty
@@ -45,12 +45,12 @@
|
||||
| 28 | JetStream API | [design](plans/2026-02-27-batch-28-jetstream-api-design.md) | [plan](plans/2026-02-27-batch-28-jetstream-api-implementation-plan.md) | planned |
|
||||
| 29 | JetStream Batching | [design](plans/2026-02-27-batch-29-jetstream-batching-design.md) | [plan](plans/2026-02-27-batch-29-jetstream-batching-implementation-plan.md) | planned |
|
||||
| 30 | Raft Part 1 | [design](plans/2026-02-27-batch-30-raft-part-1-design.md) | [plan](plans/2026-02-27-batch-30-raft-part-1-implementation-plan.md) | planned |
|
||||
| 31 | Raft Part 2 | | | not_planned |
|
||||
| 32 | JS Cluster Meta | | | not_planned |
|
||||
| 33 | JS Cluster Streams | | | not_planned |
|
||||
| 34 | JS Cluster Consumers | | | not_planned |
|
||||
| 35 | JS Cluster Remaining | | | not_planned |
|
||||
| 36 | Stream Lifecycle | | | not_planned |
|
||||
| 31 | Raft Part 2 | [design](plans/2026-02-27-batch-31-raft-part-2-design.md) | [plan](plans/2026-02-27-batch-31-raft-part-2-implementation-plan.md) | planned |
|
||||
| 32 | JS Cluster Meta | [design](plans/2026-02-27-batch-32-js-cluster-meta-design.md) | [plan](plans/2026-02-27-batch-32-js-cluster-meta-implementation-plan.md) | planned |
|
||||
| 33 | JS Cluster Streams | [design](plans/2026-02-27-batch-33-js-cluster-streams-design.md) | [plan](plans/2026-02-27-batch-33-js-cluster-streams-implementation-plan.md) | planned |
|
||||
| 34 | JS Cluster Consumers | [design](plans/2026-02-27-batch-34-js-cluster-consumers-design.md) | [plan](plans/2026-02-27-batch-34-js-cluster-consumers-implementation-plan.md) | planned |
|
||||
| 35 | JS Cluster Remaining | [design](plans/2026-02-27-batch-35-js-cluster-remaining-design.md) | [plan](plans/2026-02-27-batch-35-js-cluster-remaining-implementation-plan.md) | planned |
|
||||
| 36 | Stream Lifecycle | [design](plans/2026-02-27-batch-36-stream-lifecycle-design.md) | [plan](plans/2026-02-27-batch-36-stream-lifecycle-implementation-plan.md) | planned |
|
||||
| 37 | Stream Messages | | | not_planned |
|
||||
| 38 | Consumer Lifecycle | | | not_planned |
|
||||
| 39 | Consumer Dispatch | | | not_planned |
|
||||
|
||||
127
docs/plans/2026-02-27-batch-31-raft-part-2-design.md
Normal file
127
docs/plans/2026-02-27-batch-31-raft-part-2-design.md
Normal file
@@ -0,0 +1,127 @@
|
||||
# Batch 31 Raft Part 2 Design
|
||||
|
||||
**Date:** 2026-02-27
|
||||
**Batch:** 31 (`Raft Part 2`)
|
||||
**Scope:** 53 features + 19 unit tests
|
||||
**Dependencies:** batch `30` (`Raft Part 1`)
|
||||
**Go source:** `golang/nats-server/server/raft.go`
|
||||
|
||||
## Problem
|
||||
|
||||
Batch 31 covers the second Raft tranche in `raft.go` (roughly lines `3239-5038`), focused on catchup/snapshot transfer, append-entry processing, WAL consistency, quorum tracking, vote request/response handling, and leadership state transitions. The mapped test set (19 tests) is concentrated on candidate/leader transitions, quorum correctness, membership-change edge cases, and snapshot/catchup behavior.
|
||||
|
||||
The design goal is to produce an execution-ready plan that enforces evidence-based status changes and prevents placeholder drift across both production features and tests.
|
||||
|
||||
## Context Findings
|
||||
|
||||
### Required command results
|
||||
|
||||
- `batch show 31 --db porting.db`
|
||||
- Status: `pending`
|
||||
- Features: `53` (currently `deferred`)
|
||||
- Tests: `19` (currently `deferred`)
|
||||
- Depends on: `30`
|
||||
- Go file: `server/raft.go`
|
||||
- `batch list --db porting.db`
|
||||
- Batch 31 is directly gated by Batch 30 and itself gates Batch 32 (`JS Cluster Meta`).
|
||||
- `report summary --db porting.db`
|
||||
- Overall progress: `1924/6942 (27.7%)`
|
||||
- Deferred backlog remains large; verification discipline is required.
|
||||
|
||||
### Feature and source mapping findings
|
||||
|
||||
- Batch 31 feature IDs map in order to `raft.go` methods from:
|
||||
- `sendSnapshotToFollower` through `updateLeader` (`2733-2750`)
|
||||
- `processAppendEntry` through `setWriteErrLocked` (`2751-2777`)
|
||||
- `isClosed` through `switchToLeader` (`2778-2796`)
|
||||
- Existing .NET Raft surface is in:
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
|
||||
- Current comments in `RaftTypes.cs` still describe algorithm methods as stubbed; Batch 31 must replace those gaps with concrete behavior and tests.
|
||||
|
||||
### Test mapping findings
|
||||
|
||||
- All 19 mapped tests are from `server/raft_test.go` and map to `RaftNodeTests` methods.
|
||||
- `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs` does not currently exist, so Batch 31 planning should include creating it.
|
||||
- The mapped tests are behavior-heavy; they cannot be verified using placeholder assertions.
|
||||
|
||||
## Approaches
|
||||
|
||||
### Approach A: Monolithic implementation of all 53 features and 19 tests in one pass
|
||||
|
||||
- Pros: single sweep.
|
||||
- Cons: high regression risk, weak traceability, hard to isolate failures.
|
||||
|
||||
### Approach B (Recommended): Three feature groups (<=20 each) plus two test waves
|
||||
|
||||
- Features are implemented in ordered method clusters, each with strict gates before status updates.
|
||||
- Tests are ported in two behavioral waves (state/quorum first, then snapshot/membership edge cases).
|
||||
- Pros: bounded scope, better failure isolation, cleaner status evidence.
|
||||
- Cons: more checkpoint overhead.
|
||||
|
||||
### Approach C: Test-first across all 19 tests, then fill feature gaps
|
||||
|
||||
- Pros: quickly exposes missing behavior.
|
||||
- Cons: expensive thrash because many tests depend on broad feature slices.
|
||||
|
||||
**Decision:** Approach B.
|
||||
|
||||
## Proposed Design
|
||||
|
||||
### 1. Architecture and file strategy
|
||||
|
||||
- Keep Raft runtime behavior in `JetStream/RaftTypes.cs`, with optional split into partials if file size hurts reviewability:
|
||||
- `RaftTypes.Catchup.cs`
|
||||
- `RaftTypes.AppendProcessing.cs`
|
||||
- `RaftTypes.Elections.cs`
|
||||
- Keep test implementation in dedicated mapped backlog file:
|
||||
- `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
||||
- Reuse existing support types (`IpQueue<T>`, `Channel<T>`, lock + `Interlocked`) and avoid introducing new infra unless required for deterministic testability.
|
||||
|
||||
### 2. Feature slicing (max ~20 per group)
|
||||
|
||||
- **Feature Group A (18): catchup/snapshot/commit foundations**
|
||||
`2733,2734,2735,2736,2737,2738,2739,2740,2741,2742,2743,2744,2745,2746,2747,2748,2749,2750`
|
||||
- **Feature Group B (18): append-entry processing and peer/WAL state**
|
||||
`2751,2752,2753,2754,2755,2756,2758,2759,2760,2761,2765,2766,2767,2768,2769,2772,2776,2777`
|
||||
- **Feature Group C (17): vote/RPC/state transitions**
|
||||
`2778,2779,2780,2783,2784,2785,2786,2787,2788,2789,2790,2791,2792,2793,2794,2795,2796`
|
||||
|
||||
### 3. Test slicing
|
||||
|
||||
- **Test Wave T1 (10): state/quorum/election behavior**
|
||||
`2626,2629,2635,2636,2663,2664,2667,2687,2690,2692`
|
||||
- **Test Wave T2 (9): snapshot/catchup/membership-vote edge cases**
|
||||
`2650,2651,2693,2694,2702,2704,2705,2712,2714`
|
||||
|
||||
### 4. Verification model
|
||||
|
||||
- Enforce per-feature and per-test loops (red/green + stub scan + build/test gates).
|
||||
- Enforce status-update chunking (`<=15` IDs per `feature/test batch-update`).
|
||||
- Enforce checkpoint protocol after every group/wave before proceeding.
|
||||
|
||||
### 5. Stuck-item policy
|
||||
|
||||
- A blocked item is not left as pseudo-implemented.
|
||||
- If blocked, set `deferred` immediately with explicit reason via `--override`, then continue with next unblocked ID.
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
- **Risk:** Batch 30 dependency incomplete blocks execution.
|
||||
**Mitigation:** preflight dependency gate is mandatory; no Batch 31 status updates until Batch 30 is complete/ready.
|
||||
- **Risk:** Large method `processAppendEntry` causes hidden regressions.
|
||||
**Mitigation:** isolate with focused tests per behavior branch plus class-level gates.
|
||||
- **Risk:** fake progress via placeholder methods/tests.
|
||||
**Mitigation:** mandatory anti-stub scans and hard promotion gates.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- All 53 features are either `verified` with evidence or `deferred` with explicit blocker reason.
|
||||
- All 19 tests are either `verified` with execution evidence or `deferred` with explicit blocker reason.
|
||||
- No placeholder/stub patterns in touched production or test code.
|
||||
- Batch-completion readiness is auditable through build/test outputs and chunked status updates.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Executing implementation in this design doc.
|
||||
- Implementing Batch 32+ scope.
|
||||
- Building new distributed integration infrastructure beyond deterministic unit-level needs.
|
||||
@@ -0,0 +1,444 @@
|
||||
# Batch 31 Raft Part 2 Implementation Plan
|
||||
|
||||
> **For Codex:** REQUIRED SUB-SKILL: Use `executeplan` to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Port and verify Batch 31 (`Raft Part 2`) Raft behavior from `server/raft.go` with strict evidence gates for both features and tests.
|
||||
|
||||
**Architecture:** Implement 53 mapped Raft methods in three dependency-ordered feature groups (max 20 IDs each), then port 19 mapped tests in two waves. Use per-item red/green loops, mandatory stub scans, and chunked tracker updates (`<=15` IDs per batch command) so status moves only with executable proof.
|
||||
|
||||
**Tech Stack:** .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (`porting.db`)
|
||||
|
||||
**Design doc:** `docs/plans/2026-02-27-batch-31-raft-part-2-design.md`
|
||||
|
||||
---
|
||||
|
||||
## Batch 31 Scope
|
||||
|
||||
- Batch ID: `31`
|
||||
- Name: `Raft Part 2`
|
||||
- Dependency: `30`
|
||||
- Go source: `golang/nats-server/server/raft.go`
|
||||
- Features: `53`
|
||||
- Tests: `19`
|
||||
|
||||
Primary implementation files:
|
||||
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
|
||||
- Optional split (if needed for reviewability):
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Catchup.cs`
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.AppendProcessing.cs`
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Elections.cs`
|
||||
|
||||
Primary test files:
|
||||
|
||||
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
||||
- Modify (if shared helpers are needed):
|
||||
- `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
|
||||
- `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ImpltestsBacklogAssertions.cs`
|
||||
|
||||
---
|
||||
|
||||
## MANDATORY VERIFICATION PROTOCOL
|
||||
|
||||
> **NON-NEGOTIABLE:** Every feature ID and test ID in this batch must pass this protocol.
|
||||
|
||||
### Preflight Dependency Gate (before any status change)
|
||||
|
||||
1. Confirm dependency and readiness:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch show 30 --db porting.db
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch show 31 --db porting.db
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch ready --db porting.db
|
||||
```
|
||||
|
||||
2. Start only when Batch 31 is ready:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch start 31 --db porting.db
|
||||
```
|
||||
|
||||
3. Capture baseline build/tests:
|
||||
|
||||
```bash
|
||||
dotnet build dotnet/
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
```
|
||||
|
||||
### Per-Feature Verification Loop (REQUIRED per feature ID)
|
||||
|
||||
1. Inspect mapping and Go span:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- feature show <feature_id> --db porting.db
|
||||
sed -n '<go_start>,<go_end>p' golang/nats-server/server/raft.go
|
||||
```
|
||||
|
||||
2. Mark feature as in-progress (`stub` status in tracker only):
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status stub --db porting.db
|
||||
```
|
||||
|
||||
3. Add/adjust a focused failing test for this behavior.
|
||||
4. Run focused test and confirm it fails for the expected reason.
|
||||
5. Implement minimum production change to make it pass.
|
||||
6. Run **Build Gate**.
|
||||
7. Run **Test Gate** for touched tests.
|
||||
8. Run **Stub Detection Check**.
|
||||
9. If all pass, mark feature `complete` (not `verified` yet).
|
||||
|
||||
### Per-Test Verification Loop (REQUIRED per test ID)
|
||||
|
||||
1. Inspect mapping and Go test span:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- test show <test_id> --db porting.db
|
||||
sed -n '<go_start>,<go_end>p' golang/nats-server/server/raft_test.go
|
||||
```
|
||||
|
||||
2. Mark test as in-progress (`stub` status in tracker only):
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- test update <test_id> --status stub --db porting.db
|
||||
```
|
||||
|
||||
3. Implement a real Arrange/Act/Assert test that calls production code.
|
||||
4. Run single-test filter:
|
||||
|
||||
```bash
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
|
||||
--filter "FullyQualifiedName~RaftNodeTests.<MethodName>" \
|
||||
--verbosity normal
|
||||
```
|
||||
|
||||
5. Confirm discovery + pass (`Passed: 1, Failed: 0`).
|
||||
6. Run class-level filter for cumulative validation.
|
||||
7. Run **Stub Detection Check**.
|
||||
8. Mark test `complete` (promote to `verified` only at checkpoint).
|
||||
|
||||
### Stub Detection Check (REQUIRED after each loop and each task)
|
||||
|
||||
```bash
|
||||
git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests \
|
||||
| rg "\.cs$" \
|
||||
| xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|Assert\.True\(true\)|Assert\.Pass|=>\s*default;|=>\s*null;|return\s+null;\s*$|return\s+0;\s*$|return\s+false;\s*$)"
|
||||
```
|
||||
|
||||
Any hit in mapped methods/tests blocks promotion.
|
||||
|
||||
### Build Gate (REQUIRED)
|
||||
|
||||
Run `dotnet build dotnet/`:
|
||||
|
||||
- after each feature/test loop,
|
||||
- before any batch status update,
|
||||
- at each task checkpoint.
|
||||
|
||||
### Test Gate (REQUIRED)
|
||||
|
||||
Minimum required gates for touched Raft scope:
|
||||
|
||||
```bash
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~RaftTypes"
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~RaftNodeTests"
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog"
|
||||
```
|
||||
|
||||
Checkpoint and final gate:
|
||||
|
||||
```bash
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
```
|
||||
|
||||
### Status Update Protocol (HARD LIMIT: <=15 IDs per batch-update)
|
||||
|
||||
- Allowed flow: `deferred/not_started -> stub -> complete -> verified`
|
||||
- Never include more than 15 IDs in one `feature/test batch-update` command.
|
||||
- Never mark `verified` without passing Build Gate + Test Gate + Stub Detection Check.
|
||||
- Apply updates only for IDs in the active task.
|
||||
|
||||
Templates:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
feature batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
|
||||
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
test batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
|
||||
```
|
||||
|
||||
### Checkpoint Protocol Between Tasks (REQUIRED)
|
||||
|
||||
At end of every task before starting the next:
|
||||
|
||||
1. Run Stub Detection Check.
|
||||
2. Run Build Gate.
|
||||
3. Run focused Test Gate for touched classes.
|
||||
4. Run full unit test suite.
|
||||
5. Apply status updates for current task only (`<=15` IDs per command).
|
||||
6. Commit checkpoint.
|
||||
|
||||
---
|
||||
|
||||
## ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)
|
||||
|
||||
### Forbidden Patterns (Features + Tests)
|
||||
|
||||
Any of these in mapped/touched files is disallowed:
|
||||
|
||||
- `throw new NotImplementedException()`
|
||||
- Empty or no-op method bodies for mapped behaviors
|
||||
- Placeholder comments (`// TODO`, `// PLACEHOLDER`, `// later`)
|
||||
- Fake-pass assertions (`Assert.True(true)`, `Assert.Pass()`)
|
||||
- Tests that only assert non-null/default without behavior validation
|
||||
- Constant-return placeholders for complex logic (`return null;`, `return 0;`, `return false;`, `return string.Empty;`)
|
||||
- Blanket `catch` blocks that swallow failures to force pass
|
||||
|
||||
### Hard Limits
|
||||
|
||||
- Max feature IDs per implementation group: `20`
|
||||
- Max IDs per `feature/test batch-update`: `15`
|
||||
- One active feature loop at a time
|
||||
- One active test loop at a time
|
||||
- No `verified` transition without full gate evidence
|
||||
- Mandatory checkpoint between tasks
|
||||
|
||||
### If You Get Stuck (REQUIRED)
|
||||
|
||||
1. Stop work on that ID immediately.
|
||||
2. Do not leave placeholder code or fake-pass assertions.
|
||||
3. Mark item `deferred` with explicit reason:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- feature update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
|
||||
dotnet run --project tools/NatsNet.PortTracker -- test update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
|
||||
```
|
||||
|
||||
4. Add brief note in tracker/comment where needed.
|
||||
5. Continue with next unblocked ID in the same task.
|
||||
|
||||
Deferred-with-reason is valid. Stubs are not.
|
||||
|
||||
---
|
||||
|
||||
## Feature Groups (max ~20 each)
|
||||
|
||||
### Group A (18) - Catchup/snapshot/commit foundations
|
||||
|
||||
`2733,2734,2735,2736,2737,2738,2739,2740,2741,2742,2743,2744,2745,2746,2747,2748,2749,2750`
|
||||
|
||||
### Group B (18) - Append processing, WAL, peer state
|
||||
|
||||
`2751,2752,2753,2754,2755,2756,2758,2759,2760,2761,2765,2766,2767,2768,2769,2772,2776,2777`
|
||||
|
||||
### Group C (17) - Vote/RPC/state transitions
|
||||
|
||||
`2778,2779,2780,2783,2784,2785,2786,2787,2788,2789,2790,2791,2792,2793,2794,2795,2796`
|
||||
|
||||
## Test Waves
|
||||
|
||||
### T1 (10) - Election/quorum/state correctness
|
||||
|
||||
`2626,2629,2635,2636,2663,2664,2667,2687,2690,2692`
|
||||
|
||||
### T2 (9) - Catchup/snapshot/membership and vote retention
|
||||
|
||||
`2650,2651,2693,2694,2702,2704,2705,2712,2714`
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Preflight and Baseline
|
||||
|
||||
**Files:**
|
||||
|
||||
- Read: `docs/plans/2026-02-27-batch-31-raft-part-2-design.md`
|
||||
- Read: `golang/nats-server/server/raft.go`
|
||||
- Read: `golang/nats-server/server/raft_test.go`
|
||||
|
||||
**Step 1: Run Preflight Dependency Gate**
|
||||
|
||||
Run all preflight commands in protocol.
|
||||
|
||||
**Step 2: Start batch and capture baseline**
|
||||
|
||||
Run `batch start 31`, `dotnet build`, and full unit tests.
|
||||
|
||||
**Step 3: Checkpoint protocol**
|
||||
|
||||
Complete checkpoint before feature work.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Implement Feature Group A (18)
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Catchup.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
|
||||
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
|
||||
|
||||
**Step 1: Mark Group A IDs as `stub` in <=15-ID chunks**
|
||||
|
||||
Chunk 1:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
feature batch-update --ids "2733,2734,2735,2736,2737,2738,2739,2740,2741,2742,2743,2744,2745,2746,2747" --set-status stub --db porting.db --execute
|
||||
```
|
||||
|
||||
Chunk 2:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
feature batch-update --ids "2748,2749,2750" --set-status stub --db porting.db --execute
|
||||
```
|
||||
|
||||
**Step 2: Execute Per-Feature Verification Loop for each Group A ID**
|
||||
|
||||
**Step 3: Run task checkpoint gates**
|
||||
|
||||
**Step 4: Promote eligible Group A IDs to `complete` then `verified` in <=15-ID chunks**
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Implement Feature Group B (18)
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.AppendProcessing.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
|
||||
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
|
||||
|
||||
**Step 1: Mark Group B IDs as `stub` in <=15-ID chunks**
|
||||
|
||||
Chunk 1:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
feature batch-update --ids "2751,2752,2753,2754,2755,2756,2758,2759,2760,2761,2765,2766,2767,2768,2769" --set-status stub --db porting.db --execute
|
||||
```
|
||||
|
||||
Chunk 2:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
feature batch-update --ids "2772,2776,2777" --set-status stub --db porting.db --execute
|
||||
```
|
||||
|
||||
**Step 2: Execute Per-Feature Verification Loop for each Group B ID**
|
||||
|
||||
**Step 3: Run task checkpoint gates**
|
||||
|
||||
**Step 4: Promote eligible Group B IDs to `complete` then `verified` in <=15-ID chunks**
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Implement Feature Group C (17)
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.Elections.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/RaftTypes.cs`
|
||||
- Test: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/JetStream/RaftTypesTests.cs`
|
||||
|
||||
**Step 1: Mark Group C IDs as `stub` in <=15-ID chunks**
|
||||
|
||||
Chunk 1:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
feature batch-update --ids "2778,2779,2780,2783,2784,2785,2786,2787,2788,2789,2790,2791,2792,2793,2794" --set-status stub --db porting.db --execute
|
||||
```
|
||||
|
||||
Chunk 2:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
feature batch-update --ids "2795,2796" --set-status stub --db porting.db --execute
|
||||
```
|
||||
|
||||
**Step 2: Execute Per-Feature Verification Loop for each Group C ID**
|
||||
|
||||
**Step 3: Run task checkpoint gates**
|
||||
|
||||
**Step 4: Promote eligible Group C IDs to `complete` then `verified` in <=15-ID chunks**
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Port Test Wave T1 (10)
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
||||
- Modify (if helpers required): `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ImpltestsBacklogAssertions.cs`
|
||||
|
||||
**Step 1: Mark T1 IDs as `stub`**
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
test batch-update --ids "2626,2629,2635,2636,2663,2664,2667,2687,2690,2692" --set-status stub --db porting.db --execute
|
||||
```
|
||||
|
||||
**Step 2: Execute Per-Test Verification Loop for each T1 ID**
|
||||
|
||||
**Step 3: Run class-level + checkpoint gates**
|
||||
|
||||
**Step 4: Promote eligible T1 IDs to `complete` then `verified`**
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Port Test Wave T2 (9)
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
||||
|
||||
**Step 1: Mark T2 IDs as `stub`**
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
test batch-update --ids "2650,2651,2693,2694,2702,2704,2705,2712,2714" --set-status stub --db porting.db --execute
|
||||
```
|
||||
|
||||
**Step 2: Execute Per-Test Verification Loop for each T2 ID**
|
||||
|
||||
**Step 3: Run class-level + checkpoint gates**
|
||||
|
||||
**Step 4: Promote eligible T2 IDs to `complete` then `verified`**
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Final Batch 31 Verification and Completion
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `porting.db`
|
||||
- Modify: `reports/current.md` (via report script)
|
||||
|
||||
**Step 1: Run final mandatory gates**
|
||||
|
||||
```bash
|
||||
dotnet build dotnet/
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
dotnet run --project tools/NatsNet.PortTracker -- audit --type features --db porting.db
|
||||
dotnet run --project tools/NatsNet.PortTracker -- audit --type tests --db porting.db
|
||||
```
|
||||
|
||||
**Step 2: Resolve any remaining `stub` statuses**
|
||||
|
||||
- Convert to `verified` with evidence, or
|
||||
- Convert to `deferred` with explicit blocker reason.
|
||||
|
||||
**Step 3: Complete the batch**
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch complete 31 --db porting.db
|
||||
```
|
||||
|
||||
**Step 4: Refresh report**
|
||||
|
||||
```bash
|
||||
./reports/generate-report.sh
|
||||
```
|
||||
|
||||
137
docs/plans/2026-02-27-batch-32-js-cluster-meta-design.md
Normal file
137
docs/plans/2026-02-27-batch-32-js-cluster-meta-design.md
Normal file
@@ -0,0 +1,137 @@
|
||||
# Batch 32 JS Cluster Meta Design
|
||||
|
||||
**Date:** 2026-02-27
|
||||
**Batch:** 32 (`JS Cluster Meta`)
|
||||
**Scope:** 58 features + 36 unit tests
|
||||
**Dependencies:** batches `27` (`JetStream Core`), `31` (`Raft Part 2`)
|
||||
**Go source:** `golang/nats-server/server/jetstream_cluster.go`
|
||||
|
||||
## Problem
|
||||
|
||||
Batch 32 ports JetStream cluster metadata and control-plane behaviors from `jetstream_cluster.go`, including unsupported assignment handling, cluster leadership/currentness queries, meta-group setup hooks, assignment/health checks, inflight proposal tracking, meta-recovery flags, and orphan detection.
|
||||
|
||||
The mapped tests are distributed across cluster/super-cluster/leaf/mqtt/concurrency/raft Go suites and require non-placeholder behavioral coverage. The design goal is to provide an implementation strategy that prevents fake progress and enforces evidence-based status transitions for both features and tests.
|
||||
|
||||
## Context Findings
|
||||
|
||||
### Required command outputs
|
||||
|
||||
- `batch show 32 --db porting.db`
|
||||
- Status: `pending`
|
||||
- Features: `58` (`deferred`)
|
||||
- Tests: `36` (`deferred`)
|
||||
- Depends on: `27,31`
|
||||
- Go file: `server/jetstream_cluster.go`
|
||||
- `batch list --db porting.db`
|
||||
- Batch 32 is in dependency chain `... -> 31 -> 32 -> 33/35`.
|
||||
- `report summary --db porting.db`
|
||||
- Overall progress: `1924/6942 (27.7%)`
|
||||
|
||||
### Dependency state
|
||||
|
||||
- `batch show 27` status: `pending`
|
||||
- `batch show 31` status: `pending`
|
||||
- `batch ready` does **not** include Batch 32 currently.
|
||||
|
||||
Design implication: Batch 32 execution must start with an explicit dependency gate; no status changes before dependencies are ready.
|
||||
|
||||
### Current .NET codebase state
|
||||
|
||||
- Cluster data types already exist in `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs`.
|
||||
- Most Batch 32 mapped method names are not yet implemented in .NET source.
|
||||
- Backlog tests exist in `ImplBacklog`, but only one cluster class file currently exists:
|
||||
- present: `JetStreamClusterTests2.Impltests.cs`
|
||||
- missing and expected to be created during implementation:
|
||||
- `JetStreamClusterTests1.Impltests.cs`
|
||||
- `JetStreamClusterTests3.Impltests.cs`
|
||||
- `JetStreamClusterTests4.Impltests.cs`
|
||||
- `JetStreamClusterLongTests.Impltests.cs`
|
||||
- `JetStreamSuperClusterTests.Impltests.cs`
|
||||
|
||||
## Approaches
|
||||
|
||||
### Approach A: Monolithic one-pass implementation (all 58 features + 36 tests together)
|
||||
|
||||
- Pros: single pass, less planning overhead.
|
||||
- Cons: high risk of regressions and undetected stubs; weak traceability for status updates.
|
||||
|
||||
### Approach B (Recommended): Three feature groups (<=20 each) + three test waves
|
||||
|
||||
- Implement feature groups in source-order clusters, each with strict build/test/stub gates before status updates.
|
||||
- Port tests in behavior-based waves aligned to cluster-domain breadth.
|
||||
- Pros: bounded risk, clear checkpoints, strong auditability.
|
||||
- Cons: more checkpoint ceremony.
|
||||
|
||||
### Approach C: Test-first all 36 tests before feature porting
|
||||
|
||||
- Pros: surfaces missing behavior early.
|
||||
- Cons: creates heavy thrash because many tests depend on broad feature slices not yet implemented.
|
||||
|
||||
**Decision:** Approach B.
|
||||
|
||||
## Proposed Design
|
||||
|
||||
### 1. Code architecture and file strategy
|
||||
|
||||
- Keep cluster model + cluster helper logic in JetStream cluster files:
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs`
|
||||
- Keep `JetStream`/`JsAccount` cluster-meta behaviors in JetStream engine file(s):
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
|
||||
- optional split if needed for reviewability: `JetStream/JetStreamClusterMeta.cs`
|
||||
- Keep server-facing API entry points in a server partial:
|
||||
- create/modify `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterMeta.cs`
|
||||
- Keep account-facing query helpers in account class:
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs`
|
||||
|
||||
### 2. Feature slicing (max ~20 each)
|
||||
|
||||
- **Feature Group A (20): unsupported assignment + server cluster state basics**
|
||||
IDs: `1520-1539`
|
||||
- **Feature Group B (20): health/leadership queries + clustering enable + assignment checks**
|
||||
IDs: `1540-1559`
|
||||
- **Feature Group C (18): leader/consumer assignment internals + inflight tracking + recovery/orphans**
|
||||
IDs: `1560-1577`
|
||||
|
||||
This keeps each feature task under the required `~20` cap.
|
||||
|
||||
### 3. Test slicing
|
||||
|
||||
- **Wave T1 (13): core cluster behavior (cluster tests 1/2)**
|
||||
IDs: `772,774,775,791,809,810,811,817,853,914,993,1014,1028`
|
||||
- **Wave T2 (12): advanced cluster behavior (cluster tests 3/4 + long)**
|
||||
IDs: `1060,1088,1098,1106,1109,1122,1128,1136,1194,1211,1212,1217`
|
||||
- **Wave T3 (11): cross-domain coverage (leaf/super/mqtt/concurrency/raft hooks)**
|
||||
IDs: `1406,1453,1454,1457,1465,1528,2225,2390,2459,2489,2689`
|
||||
|
||||
### 4. Verification model (features + tests)
|
||||
|
||||
- Mandatory per-feature red/green loop with build + focused tests + stub scan before promotion.
|
||||
- Mandatory per-test loop (single-test pass evidence + class/wave pass evidence).
|
||||
- Status updates only in chunks of `<=15 IDs` per `feature/test batch-update` command.
|
||||
- Task checkpoints between groups/waves with full suite verification.
|
||||
|
||||
### 5. Deferred policy
|
||||
|
||||
If an item is blocked by missing infrastructure or unresolved dependency behavior, explicitly set it to `deferred` with `--override "blocked: <specific reason>"`. Do not leave stubs or fake-pass tests.
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
- **Risk:** dependency batches 27/31 still pending.
|
||||
**Mitigation:** enforce preflight dependency gate before any Batch 32 status transitions.
|
||||
- **Risk:** broad cluster tests encourage placeholder assertions.
|
||||
**Mitigation:** anti-stub guardrails + required assertion quality and per-test evidence.
|
||||
- **Risk:** cross-file method ownership ambiguity (`NatsServer`, `JetStream`, `Account`).
|
||||
**Mitigation:** fixed ownership map in plan and grouped implementation order by type.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- All 58 features are either `verified` with evidence or `deferred` with explicit blocker reason.
|
||||
- All 36 tests are either `verified` with run evidence or `deferred` with explicit blocker reason.
|
||||
- No forbidden stub patterns in touched production/test files.
|
||||
- Batch 32 completion is auditable through build/test outputs and chunked status updates.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Executing Batch 32 implementation in this document.
|
||||
- Porting Batch 33/34/35 behaviors.
|
||||
- Building new distributed integration infrastructure beyond what is needed for deterministic unit/backlog verification.
|
||||
@@ -0,0 +1,391 @@
|
||||
# Batch 32 JS Cluster Meta Implementation Plan
|
||||
|
||||
> **For Codex:** REQUIRED SUB-SKILL: Use `executeplan` to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Port and verify Batch 32 (`JS Cluster Meta`) from `server/jetstream_cluster.go` with strict evidence gates for both features and tests.
|
||||
|
||||
**Architecture:** Implement mapped cluster-meta methods in three source-ordered feature groups (`<=20` IDs each) across `NatsServer`, `JetStream`, `JetStreamCluster`, `RecoveryUpdates`, and `Account`. Then port mapped backlog tests in three waves. Every promotion (`stub -> complete -> verified`) is gated by build/test evidence, stub scans, and chunked tracker updates (`<=15` IDs per update).
|
||||
|
||||
**Tech Stack:** .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (`porting.db`)
|
||||
|
||||
**Design doc:** `docs/plans/2026-02-27-batch-32-js-cluster-meta-design.md`
|
||||
|
||||
---
|
||||
|
||||
## Batch 32 Scope
|
||||
|
||||
- Batch ID: `32`
|
||||
- Name: `JS Cluster Meta`
|
||||
- Dependencies: `27`, `31`
|
||||
- Go source: `golang/nats-server/server/jetstream_cluster.go`
|
||||
- Features: `58`
|
||||
- Tests: `36`
|
||||
|
||||
> If `dotnet` is not available on `PATH`, run commands with `/usr/local/share/dotnet/dotnet` instead.
|
||||
|
||||
Primary source files:
|
||||
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs`
|
||||
- Create/Modify: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterMeta.cs`
|
||||
|
||||
Primary test files:
|
||||
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs`
|
||||
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs`
|
||||
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs`
|
||||
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests4.Impltests.cs`
|
||||
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterLongTests.Impltests.cs`
|
||||
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamSuperClusterTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamLeafNodeTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests2.Impltests.cs`
|
||||
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
||||
|
||||
---
|
||||
|
||||
## MANDATORY VERIFICATION PROTOCOL
|
||||
|
||||
> **NON-NEGOTIABLE:** Every feature ID and test ID in Batch 32 must pass this protocol.
|
||||
|
||||
### Dependency Preflight Gate (before any status change)
|
||||
|
||||
1. Validate dependencies and readiness:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch show 27 --db porting.db
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch show 31 --db porting.db
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch show 32 --db porting.db
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch ready --db porting.db
|
||||
```
|
||||
|
||||
2. Only when Batch 32 is ready, start it:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch start 32 --db porting.db
|
||||
```
|
||||
|
||||
3. Capture clean baseline:
|
||||
|
||||
```bash
|
||||
dotnet build dotnet/
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
```
|
||||
|
||||
### Per-Feature Verification Loop (REQUIRED per feature ID)
|
||||
|
||||
1. Inspect mapped feature and Go span:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- feature show <feature_id> --db porting.db
|
||||
```
|
||||
|
||||
2. Mark only the active feature as `stub` (tracker in-progress marker):
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status stub --db porting.db
|
||||
```
|
||||
|
||||
3. Add/adjust a focused test that fails for the intended behavior.
|
||||
4. Run focused test; confirm expected failure.
|
||||
5. Implement minimal production change for that feature.
|
||||
6. Run **Stub Detection Check** (below).
|
||||
7. Run **Build Gate** (below).
|
||||
8. Run **Test Gate** (below).
|
||||
9. If all pass, mark feature `complete`. Promote to `verified` only at task checkpoint.
|
||||
|
||||
### Stub Detection Check (REQUIRED after each feature loop and each test loop)
|
||||
|
||||
Run against changed C# files only:
|
||||
|
||||
```bash
|
||||
changed_files=$(git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests | rg "\.cs$" || true)
|
||||
if [ -n "$changed_files" ]; then
|
||||
echo "$changed_files" | xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|Assert\.True\(true\)|Assert\.Pass|=>\s*default;|=>\s*null;|return\s+null;\s*$|return\s+0;\s*$|return\s+false;\s*$)"
|
||||
fi
|
||||
```
|
||||
|
||||
Any match in mapped methods/tests blocks promotion.
|
||||
|
||||
### Build Gate (REQUIRED)
|
||||
|
||||
Run `dotnet build dotnet/`:
|
||||
|
||||
- after each feature loop,
|
||||
- after each test loop,
|
||||
- before every status batch-update,
|
||||
- at every checkpoint between tasks.
|
||||
|
||||
### Test Gate (REQUIRED)
|
||||
|
||||
Minimum gates:
|
||||
|
||||
```bash
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~JetStream"
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog"
|
||||
```
|
||||
|
||||
For every mapped backlog test method:
|
||||
|
||||
```bash
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
|
||||
--filter "FullyQualifiedName~<ClassName>.<MethodName>" \
|
||||
--verbosity normal
|
||||
```
|
||||
|
||||
Verify discovery and pass (`Passed: 1, Failed: 0`) before promotion.
|
||||
|
||||
### Status Update Protocol (HARD LIMIT: <=15 IDs per batch-update)
|
||||
|
||||
- Allowed transitions: `deferred/not_started -> stub -> complete -> verified`
|
||||
- Never include more than `15` IDs in one update command.
|
||||
- Never mark `verified` without passing stub scan + build gate + test gate.
|
||||
- Never update IDs outside the currently active task.
|
||||
|
||||
Templates:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
feature batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
|
||||
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
test batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
|
||||
```
|
||||
|
||||
### Checkpoint Protocol Between Tasks (REQUIRED)
|
||||
|
||||
At the end of every task before starting the next:
|
||||
|
||||
1. Run Stub Detection Check.
|
||||
2. Run Build Gate.
|
||||
3. Run targeted Test Gate for touched classes.
|
||||
4. Run full unit tests:
|
||||
|
||||
```bash
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
```
|
||||
|
||||
5. Apply status promotions (`complete`/`verified`) for that task only (`<=15` IDs per command).
|
||||
6. Commit checkpoint.
|
||||
|
||||
---
|
||||
|
||||
## ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)
|
||||
|
||||
### Forbidden Patterns (Features + Tests)
|
||||
|
||||
Any of these in mapped/touched files is disallowed:
|
||||
|
||||
- `throw new NotImplementedException()`
|
||||
- Empty method bodies for mapped feature methods
|
||||
- Placeholder comments (`// TODO`, `// PLACEHOLDER`, `// later`)
|
||||
- Fake-pass assertions (`Assert.True(true)`, `Assert.Pass()`)
|
||||
- Trivial string/self assertions unrelated to production behavior
|
||||
- Unconditional constant returns in mapped methods without state/input usage (`return false;`, `return 0;`, `return null;`, `return string.Empty;`)
|
||||
- Catch-and-ignore blocks that suppress real failures just to pass tests
|
||||
|
||||
### Hard Limits
|
||||
|
||||
- Max features per implementation task group: `20`
|
||||
- Max IDs per `feature batch-update`: `15`
|
||||
- Max IDs per `test batch-update`: `15`
|
||||
- One active feature loop at a time
|
||||
- One active test loop at a time
|
||||
- No `verified` status without checkpoint evidence
|
||||
- Mandatory checkpoint between tasks
|
||||
|
||||
### If You Get Stuck (REQUIRED)
|
||||
|
||||
1. Stop work on that specific ID.
|
||||
2. Do **not** keep or add placeholder/stub code.
|
||||
3. Mark the item `deferred` immediately with explicit reason:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- feature update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
|
||||
dotnet run --project tools/NatsNet.PortTracker -- test update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
|
||||
```
|
||||
|
||||
4. Add a short note in tracker/test comment with concrete blocker details.
|
||||
5. Continue with the next unblocked ID.
|
||||
|
||||
Deferred-with-reason is valid progress. Stubs are not.
|
||||
|
||||
---
|
||||
|
||||
## Feature Groups (max ~20 each)
|
||||
|
||||
### Group A (20) - Unsupported assignments + server/meta state entry points
|
||||
|
||||
IDs:
|
||||
`1520,1521,1522,1523,1524,1525,1526,1527,1528,1529,1530,1531,1532,1533,1534,1535,1536,1537,1538,1539`
|
||||
|
||||
### Group B (20) - Health/leadership checks + clustering enable + assignment checks
|
||||
|
||||
IDs:
|
||||
`1540,1541,1542,1543,1544,1545,1546,1547,1548,1549,1550,1551,1552,1553,1554,1555,1556,1557,1558,1559`
|
||||
|
||||
### Group C (18) - Stream/consumer leadership internals + inflight tracking + recovery/orphans
|
||||
|
||||
IDs:
|
||||
`1560,1561,1562,1563,1564,1565,1566,1567,1568,1569,1570,1571,1572,1573,1574,1575,1576,1577`
|
||||
|
||||
## Test Waves
|
||||
|
||||
### Wave T1 (13) - Core cluster tests (files 1/2)
|
||||
|
||||
IDs:
|
||||
`772,774,775,791,809,810,811,817,853,914,993,1014,1028`
|
||||
|
||||
### Wave T2 (12) - Advanced cluster tests (files 3/4 + long)
|
||||
|
||||
IDs:
|
||||
`1060,1088,1098,1106,1109,1122,1128,1136,1194,1211,1212,1217`
|
||||
|
||||
### Wave T3 (11) - Cross-domain integration-facing backlog tests
|
||||
|
||||
IDs:
|
||||
`1406,1453,1454,1457,1465,1528,2225,2390,2459,2489,2689`
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Preflight and Baseline
|
||||
|
||||
**Files:**
|
||||
|
||||
- Read: `docs/plans/2026-02-27-batch-32-js-cluster-meta-design.md`
|
||||
- Read: `golang/nats-server/server/jetstream_cluster.go`
|
||||
|
||||
**Step 1:** Run dependency preflight gate (27/31/32 + ready list).
|
||||
**Step 2:** Start batch 32 only if ready.
|
||||
**Step 3:** Capture baseline build and full test run.
|
||||
**Step 4:** Apply checkpoint protocol before feature work.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Implement Feature Group A (20)
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs`
|
||||
- Create/Modify: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterMeta.cs`
|
||||
|
||||
**Step 1:** Set Group A to `stub` in two chunks (`15 + 5`).
|
||||
**Step 2:** Execute per-feature verification loop for each ID.
|
||||
**Step 3:** Run checkpoint protocol.
|
||||
**Step 4:** Promote passing IDs to `complete` and `verified` in `<=15` chunks.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Implement Feature Group B (20)
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs`
|
||||
- Create/Modify: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterMeta.cs`
|
||||
|
||||
**Step 1:** Set Group B to `stub` in two chunks (`15 + 5`).
|
||||
**Step 2:** Execute per-feature verification loop for each ID.
|
||||
**Step 3:** Run checkpoint protocol.
|
||||
**Step 4:** Promote passing IDs to `complete` and `verified` in `<=15` chunks.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Implement Feature Group C (18)
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
|
||||
|
||||
**Step 1:** Set Group C to `stub` in two chunks (`15 + 3`).
|
||||
**Step 2:** Execute per-feature verification loop for each ID.
|
||||
**Step 3:** Run checkpoint protocol.
|
||||
**Step 4:** Promote passing IDs to `complete` and `verified` in `<=15` chunks.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Port Test Wave T1 (13)
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs`
|
||||
|
||||
**Step 1:** Set T1 test IDs to `stub` (single chunk of 13).
|
||||
**Step 2:** Execute per-test verification loop for each ID (single-test proof required).
|
||||
**Step 3:** Run class-level filters for `JetStreamClusterTests1` and `JetStreamClusterTests2`.
|
||||
**Step 4:** Run checkpoint protocol and promote passing IDs in `<=15` chunk.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Port Test Wave T2 (12)
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs`
|
||||
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests4.Impltests.cs`
|
||||
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterLongTests.Impltests.cs`
|
||||
|
||||
**Step 1:** Set T2 test IDs to `stub` (single chunk of 12).
|
||||
**Step 2:** Execute per-test verification loop for each ID.
|
||||
**Step 3:** Run class-level filters for touched classes.
|
||||
**Step 4:** Run checkpoint protocol and promote passing IDs.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Port Test Wave T3 (11)
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamSuperClusterTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamLeafNodeTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests2.Impltests.cs`
|
||||
- Create/Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
||||
|
||||
**Step 1:** Set T3 test IDs to `stub` (single chunk of 11).
|
||||
**Step 2:** Execute per-test verification loop for each ID.
|
||||
**Step 3:** Run class-level filters for all touched backlog classes.
|
||||
**Step 4:** Run checkpoint protocol and promote passing IDs.
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Final Batch 32 Verification and Closure
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `porting.db`
|
||||
- Update: `reports/current.md` (generated)
|
||||
|
||||
**Step 1:** Final gates:
|
||||
|
||||
```bash
|
||||
dotnet build dotnet/
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
dotnet run --project tools/NatsNet.PortTracker -- audit --type features --db porting.db
|
||||
dotnet run --project tools/NatsNet.PortTracker -- audit --type tests --db porting.db
|
||||
```
|
||||
|
||||
**Step 2:** Resolve any remaining `stub` statuses:
|
||||
|
||||
- Promote to `verified` with evidence, or
|
||||
- Set to `deferred` with explicit blocker reason.
|
||||
|
||||
**Step 3:** Complete batch:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch complete 32 --db porting.db
|
||||
```
|
||||
|
||||
**Step 4:** Refresh report:
|
||||
|
||||
```bash
|
||||
./reports/generate-report.sh
|
||||
```
|
||||
|
||||
125
docs/plans/2026-02-27-batch-33-js-cluster-streams-design.md
Normal file
125
docs/plans/2026-02-27-batch-33-js-cluster-streams-design.md
Normal file
@@ -0,0 +1,125 @@
|
||||
# Batch 33 JS Cluster Streams Design
|
||||
|
||||
**Date:** 2026-02-27
|
||||
**Batch:** 33 (`JS Cluster Streams`)
|
||||
**Scope:** 58 features + 22 unit tests
|
||||
**Dependency:** batch `32` (`JS Cluster Meta`)
|
||||
**Go source:** `golang/nats-server/server/jetstream_cluster.go`
|
||||
|
||||
## Problem
|
||||
|
||||
Batch 33 ports JetStream cluster stream/consumer assignment execution paths from `server/jetstream_cluster.go`, covering cluster monitoring loops, metadata snapshots, raft-group creation, stream-entry application, leader-change advisories, and stream/consumer create-update-delete flows.
|
||||
|
||||
The mapped tests are spread across JetStream cluster, monitor, JWT, concurrency, and raft suites. The design objective is to define a strict, auditable implementation path that avoids placeholder code and only advances tracker statuses with build/test evidence.
|
||||
|
||||
## Context Findings
|
||||
|
||||
### Required command outputs
|
||||
|
||||
- `batch show 33 --db porting.db`
|
||||
- Status: `pending`
|
||||
- Features: `58` (all `deferred`)
|
||||
- Tests: `22` (all `deferred`)
|
||||
- Depends on: `32`
|
||||
- Go file: `server/jetstream_cluster.go`
|
||||
- `batch list --db porting.db`
|
||||
- Batch chain includes `32 -> 33 -> 34` for JS cluster progression.
|
||||
- `report summary --db porting.db`
|
||||
- Overall progress: `1924/6942 (27.7%)`
|
||||
|
||||
Note: in this environment, `dotnet` is not on `PATH`; use `/usr/local/share/dotnet/dotnet` when needed.
|
||||
|
||||
### Current .NET state relevant to Batch 33
|
||||
|
||||
- Cluster data structures exist in `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs`.
|
||||
- Core types exist in:
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs`
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs`
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.*.cs` (partial server files)
|
||||
- Backlog test coverage is mostly placeholder-level today; `JetStreamClusterTests2.Impltests.cs` is present, while several mapped classes (for example `JetStreamClusterTests3`, `JetStreamClusterLongTests`, `RaftNodeTests`) still need concrete batch coverage.
|
||||
|
||||
## Clarified Constraints
|
||||
|
||||
- Planning only in this session: no implementation execution.
|
||||
- Mandatory guardrails from Batch 0 must be carried forward and adapted to features + tests.
|
||||
- Feature work must be chunked into groups of at most ~20 features.
|
||||
- Status updates must use `batch-update` chunks of max 15 IDs.
|
||||
|
||||
## Approaches
|
||||
|
||||
### Approach A: Monolithic pass (all 58 features + 22 tests)
|
||||
|
||||
- Pros: fewer task boundaries.
|
||||
- Cons: weak traceability and high risk of hidden stubs/regressions.
|
||||
|
||||
### Approach B (Recommended): Three feature groups + three test waves with hard checkpoints
|
||||
|
||||
- Pros: bounded scope per task, stronger verification evidence, easier rollback/debug.
|
||||
- Cons: more command overhead and checkpoint ceremony.
|
||||
|
||||
### Approach C: Test-heavy-first before major feature porting
|
||||
|
||||
- Pros: early behavior signal.
|
||||
- Cons: high churn because many mapped tests depend on stream/consumer cluster plumbing not yet ported.
|
||||
|
||||
**Decision:** Approach B.
|
||||
|
||||
## Proposed Design
|
||||
|
||||
### 1. File ownership model
|
||||
|
||||
- `JetStream` cluster stream orchestration methods in `JetStreamTypes.cs` or a new focused partial file (`JetStream.ClusterStreams.cs`).
|
||||
- `NatsStream` raft/cluster helpers in `NatsStream.cs` or `NatsStream.Cluster.cs`.
|
||||
- `RaftGroup`, `StreamAssignment`, `ConsumerAssignment`, and cluster helpers in `JetStreamClusterTypes.cs` (or focused partials if split improves reviewability).
|
||||
- Server-facing operations and advisories in a new/updated server partial (`NatsServer.JetStreamClusterStreams.cs`).
|
||||
|
||||
### 2. Feature slicing (max ~20 each)
|
||||
|
||||
- **Feature Group A (20 IDs):** cluster monitor + snapshot/recovery primitives
|
||||
IDs: `1578-1597`
|
||||
- **Feature Group B (20 IDs):** meta-entry application + raft-group/stream monitoring + leader-change core
|
||||
IDs: `1598-1617`
|
||||
- **Feature Group C (18 IDs):** advisory + stream assignment/process lifecycle + consumer assignment/process lifecycle
|
||||
IDs: `1618-1635`
|
||||
|
||||
### 3. Test slicing
|
||||
|
||||
- **Test Wave T1 (5 IDs):** cluster long-path + JWT/monitor/concurrency anchors
|
||||
IDs: `1118,1214,1402,2144,2504`
|
||||
- **Test Wave T2 (9 IDs):** raft elections and term behavior (early raft set)
|
||||
IDs: `2616,2620,2622,2624,2627,2628,2630,2631,2634`
|
||||
- **Test Wave T3 (8 IDs):** raft replay/catchup/chain-of-blocks paths
|
||||
IDs: `2637,2638,2652,2657,2670,2671,2698,2699`
|
||||
|
||||
### 4. Verification architecture
|
||||
|
||||
- Per-feature loop: `feature show` -> focused failing test -> minimal implementation -> stub scan -> build gate -> targeted test gate -> status transition.
|
||||
- Per-test loop: `test show` -> Go behavioral port -> single-test run evidence -> class-level run -> status transition.
|
||||
- Checkpoint after every feature group and test wave, including full unit suite run.
|
||||
|
||||
### 5. Deferred handling model
|
||||
|
||||
If blocked by missing dependency behavior/infrastructure, immediately mark item `deferred` with explicit reason via `--override`; do not leave stubs in source or tests.
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
- **Dependency risk:** Batch 32 is prerequisite.
|
||||
**Mitigation:** block all Batch 33 status transitions until dependency preflight confirms readiness.
|
||||
- **Stub-risk in backlog tests:** existing placeholder-style tests can produce false progress.
|
||||
**Mitigation:** required stub scan + assertion-quality checks + single-test execution evidence.
|
||||
- **Ownership ambiguity risk:** methods span `JetStream`, `NatsStream`, `JetStreamCluster`, `NatsServer`.
|
||||
**Mitigation:** explicit file ownership map and grouped tasking by domain.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- All 58 features are either `verified` with evidence or `deferred` with explicit blocker reason.
|
||||
- All 22 tests are either `verified` with evidence or `deferred` with explicit blocker reason.
|
||||
- No forbidden stub patterns in touched files.
|
||||
- Batch progress is auditable from command outputs and chunked status updates.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Executing the implementation in this document.
|
||||
- Extending scope into Batch 34/35.
|
||||
- Building full distributed integration harness beyond mapped unit/backlog verification needs.
|
||||
@@ -0,0 +1,363 @@
|
||||
# Batch 33 JS Cluster Streams Implementation Plan
|
||||
|
||||
> **For Codex:** REQUIRED SUB-SKILL: Use `executeplan` to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Port and verify Batch 33 (`JS Cluster Streams`) from `server/jetstream_cluster.go`, with evidence-based status transitions for all 58 features and 22 tests.
|
||||
|
||||
**Architecture:** Implement stream/consumer cluster logic in three feature groups (`20/20/18`) across `JetStream`, `NatsStream`, `JetStreamCluster` data helpers, and `NatsServer` partials. Then port mapped tests in three waves (`5/9/8`) with method-level execution evidence. Every transition (`deferred/not_started -> stub -> complete -> verified`) is gated by stub scans, build/test gates, and checkpoint review.
|
||||
|
||||
**Tech Stack:** .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (`porting.db`)
|
||||
|
||||
**Design doc:** `docs/plans/2026-02-27-batch-33-js-cluster-streams-design.md`
|
||||
|
||||
---
|
||||
|
||||
## Batch 33 Scope
|
||||
|
||||
- Batch ID: `33`
|
||||
- Name: `JS Cluster Streams`
|
||||
- Dependency: `32`
|
||||
- Go source: `golang/nats-server/server/jetstream_cluster.go`
|
||||
- Features: `58` (`1578-1635`)
|
||||
- Tests: `22` (`1118,1214,1402,2144,2504,2616,2620,2622,2624,2627,2628,2630,2631,2634,2637,2638,2652,2657,2670,2671,2698,2699`)
|
||||
|
||||
If `dotnet` is unavailable on `PATH`, use `/usr/local/share/dotnet/dotnet` in all commands.
|
||||
|
||||
Primary production files (expected touch set):
|
||||
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterStreams.cs`
|
||||
|
||||
Primary test files (expected touch set):
|
||||
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterLongTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamJwtTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MonitoringHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests2.Impltests.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
||||
|
||||
---
|
||||
|
||||
## MANDATORY VERIFICATION PROTOCOL
|
||||
|
||||
> **NON-NEGOTIABLE:** Every Batch 33 feature/test ID must pass this protocol before promotion.
|
||||
|
||||
### Dependency Preflight Gate (before any status changes)
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch show 32 --db porting.db
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch show 33 --db porting.db
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch ready --db porting.db
|
||||
```
|
||||
|
||||
Only if dependency is satisfied and Batch 33 is ready:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch start 33 --db porting.db
|
||||
```
|
||||
|
||||
Capture baseline:
|
||||
|
||||
```bash
|
||||
dotnet build dotnet/
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
```
|
||||
|
||||
### Per-Feature Verification Loop (REQUIRED per feature ID)
|
||||
|
||||
1. Inspect mapping and Go span:
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- feature show <feature_id> --db porting.db
|
||||
```
|
||||
2. Claim only that ID as in-progress marker:
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status stub --db porting.db
|
||||
```
|
||||
3. Add or update at least one focused test that fails for the intended behavior.
|
||||
4. Run focused test and confirm failure is for the target behavior.
|
||||
5. Implement minimal production logic for that feature ID.
|
||||
6. Run `Stub Detection Check`.
|
||||
7. Run `Build Gate`.
|
||||
8. Run `Test Gate` (focused).
|
||||
9. If green, mark that feature `complete` (promotion to `verified` only at checkpoint).
|
||||
|
||||
### Per-Test Verification Loop (REQUIRED per test ID)
|
||||
|
||||
1. Inspect mapping and Go test source:
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- test show <test_id> --db porting.db
|
||||
```
|
||||
2. Claim only that test ID as `stub`.
|
||||
3. Port behavior (Arrange/Act/Assert; no placeholder assertions).
|
||||
4. Run the single test method:
|
||||
```bash
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
|
||||
--filter "FullyQualifiedName~<ClassName>.<MethodName>" --verbosity normal
|
||||
```
|
||||
5. Confirm `Passed: 1, Failed: 0` (not `Passed: 0`).
|
||||
6. Run class-level filter for the touched class.
|
||||
7. Run `Stub Detection Check` + `Build Gate`.
|
||||
8. If green, mark test `complete` (promotion to `verified` only at checkpoint).
|
||||
|
||||
### Stub Detection Check (REQUIRED after each feature/test loop)
|
||||
|
||||
Run only on changed C# files:
|
||||
|
||||
```bash
|
||||
changed_files=$(git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests | rg "\\.cs$" || true)
|
||||
if [ -n "$changed_files" ]; then
|
||||
echo "$changed_files" | xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|Assert\\.True\\(true\\)|Assert\\.Pass|ShouldBe\\(true\\)|=>\\s*default;|=>\\s*null;|return\\s+null;\\s*$|return\\s+0;\\s*$|return\\s+false;\\s*$)"
|
||||
fi
|
||||
```
|
||||
|
||||
Any match in mapped methods/tests blocks promotion.
|
||||
|
||||
### Build Gate (REQUIRED)
|
||||
|
||||
Run `dotnet build dotnet/`:
|
||||
|
||||
- after each feature loop,
|
||||
- after each test loop,
|
||||
- before every `batch-update`,
|
||||
- at every task checkpoint.
|
||||
|
||||
### Test Gate (REQUIRED)
|
||||
|
||||
Minimum gates:
|
||||
|
||||
```bash
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~JetStream"
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog"
|
||||
```
|
||||
|
||||
Plus per-item focused gate:
|
||||
|
||||
- Per feature loop: run nearest impacted backlog or unit class filters.
|
||||
- Per test loop: run single-method filter, then class filter.
|
||||
|
||||
### Status Update Protocol (HARD LIMIT: 15 IDs max per `batch-update`)
|
||||
|
||||
- Never pass more than `15` IDs in one `feature/test batch-update` command.
|
||||
- Never mark `verified` without loop evidence + task checkpoint evidence.
|
||||
- Never update IDs outside the active task.
|
||||
- For blocked IDs, use deferred override with explicit reason.
|
||||
|
||||
Templates:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
feature batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
|
||||
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
test batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
|
||||
```
|
||||
|
||||
### Checkpoint Protocol Between Tasks (REQUIRED)
|
||||
|
||||
At the end of each task before moving on:
|
||||
|
||||
1. Run `Stub Detection Check`.
|
||||
2. Run `Build Gate`.
|
||||
3. Run targeted test filters for touched classes.
|
||||
4. Run full unit suite:
|
||||
```bash
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
```
|
||||
5. Promote only proven IDs (`complete` and/or `verified`) in `<=15` chunks.
|
||||
6. Record evidence and commit checkpoint.
|
||||
|
||||
---
|
||||
|
||||
## ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)
|
||||
|
||||
### Forbidden Patterns (Features + Tests)
|
||||
|
||||
- `throw new NotImplementedException()` in mapped Batch 33 methods.
|
||||
- Empty method body for mapped features.
|
||||
- Placeholder comments in mapped code/tests (`TODO`, `PLACEHOLDER`, `later`).
|
||||
- Fake-pass assertions (`Assert.True(true)`, `Assert.Pass()`, trivial self-string assertions).
|
||||
- Constant-return shortcuts for mapped methods without input/state use (`return false;`, `return 0;`, `return null;`, `return string.Empty;`).
|
||||
- Catch-and-ignore blocks that suppress failures to make tests pass.
|
||||
|
||||
### Hard Limits
|
||||
|
||||
- Feature group size max: `~20` IDs.
|
||||
- `feature batch-update` max IDs: `15`.
|
||||
- `test batch-update` max IDs: `15`.
|
||||
- One active feature loop at a time.
|
||||
- One active test loop at a time.
|
||||
- No `verified` promotion without checkpoint evidence.
|
||||
- Mandatory checkpoint between every task.
|
||||
|
||||
### If You Get Stuck (REQUIRED)
|
||||
|
||||
1. Stop the current ID immediately.
|
||||
2. Do not keep/add stubbed placeholder behavior.
|
||||
3. Mark blocked item `deferred` with explicit reason:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
feature update <feature_id> --status deferred --override "blocked: <specific reason>" --db porting.db
|
||||
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
test update <test_id> --status deferred --override "blocked: <specific reason>" --db porting.db
|
||||
```
|
||||
|
||||
4. Continue with next unblocked ID.
|
||||
|
||||
Deferred with reason is acceptable. Stubbed progress is not.
|
||||
|
||||
---
|
||||
|
||||
## Feature Groups (max ~20 each)
|
||||
|
||||
### Group A (20): monitor/snapshot/recovery primitives
|
||||
|
||||
IDs:
|
||||
`1578,1579,1580,1581,1582,1583,1584,1585,1586,1587,1588,1589,1590,1591,1592,1593,1594,1595,1596,1597`
|
||||
|
||||
### Group B (20): meta entries + raft group + stream monitor/apply + leader change
|
||||
|
||||
IDs:
|
||||
`1598,1599,1600,1601,1602,1603,1604,1605,1606,1607,1608,1609,1610,1611,1612,1613,1614,1615,1616,1617`
|
||||
|
||||
### Group C (18): advisories + stream/consumer assignment lifecycle
|
||||
|
||||
IDs:
|
||||
`1618,1619,1620,1621,1622,1623,1624,1625,1626,1627,1628,1629,1630,1631,1632,1633,1634,1635`
|
||||
|
||||
## Test Waves
|
||||
|
||||
### Wave T1 (5): cluster/jwt/monitor/concurrency anchors
|
||||
|
||||
IDs:
|
||||
`1118,1214,1402,2144,2504`
|
||||
|
||||
### Wave T2 (9): raft election/term/recovery baseline
|
||||
|
||||
IDs:
|
||||
`2616,2620,2622,2624,2627,2628,2630,2631,2634`
|
||||
|
||||
### Wave T3 (8): raft replay/catchup/chain-of-blocks
|
||||
|
||||
IDs:
|
||||
`2637,2638,2652,2657,2670,2671,2698,2699`
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Preflight and Baseline
|
||||
|
||||
**Files:**
|
||||
- Read: `docs/plans/2026-02-27-batch-33-js-cluster-streams-design.md`
|
||||
- Read: `golang/nats-server/server/jetstream_cluster.go`
|
||||
|
||||
**Steps:**
|
||||
1. Run dependency preflight gate and confirm Batch 33 can start.
|
||||
2. Start Batch 33 only when ready.
|
||||
3. Capture baseline build + unit test run.
|
||||
4. Do not change any feature/test status before baseline evidence is captured.
|
||||
|
||||
### Task 2: Implement Feature Group A (20 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Set Group A to `stub` in two commands (`15 + 5` IDs).
|
||||
2. Run Per-Feature Verification Loop for each ID in Group A.
|
||||
3. Apply Stub Detection + Build/Test gates after each ID.
|
||||
4. At checkpoint, promote proven IDs to `complete` then `verified` in `<=15` chunks.
|
||||
|
||||
### Task 3: Implement Feature Group B (20 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterStreams.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Set Group B to `stub` in two commands (`15 + 5` IDs).
|
||||
2. Execute Per-Feature Verification Loop for each Group B ID.
|
||||
3. Run all required gates.
|
||||
4. Execute checkpoint and chunked promotions (`<=15`).
|
||||
|
||||
### Task 4: Implement Feature Group C (18 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterStreams.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Set Group C to `stub` in two commands (`15 + 3` IDs).
|
||||
2. Execute Per-Feature Verification Loop for each Group C ID.
|
||||
3. Run all required gates.
|
||||
4. Execute checkpoint and chunked promotions (`<=15`).
|
||||
|
||||
### Task 5: Port Test Wave T1 (5 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterLongTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamJwtTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MonitoringHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests2.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Set T1 IDs to `stub` (single update command).
|
||||
2. Execute Per-Test Verification Loop for each ID.
|
||||
3. Run class-level filters for each touched class.
|
||||
4. Execute checkpoint and promote passing IDs in `<=15` chunks.
|
||||
|
||||
### Task 6: Port Test Wave T2 (9 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Set T2 IDs to `stub` (single update command).
|
||||
2. Execute Per-Test Verification Loop for each raft ID.
|
||||
3. Run `RaftNodeTests` class filter and confirm pass counts.
|
||||
4. Execute checkpoint and promote proven IDs.
|
||||
|
||||
### Task 7: Port Test Wave T3 (8 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Set T3 IDs to `stub` (single update command).
|
||||
2. Execute Per-Test Verification Loop for each ID.
|
||||
3. Run class filter + full ImplBacklog filter.
|
||||
4. Execute checkpoint and promote proven IDs.
|
||||
|
||||
### Task 8: Final Batch Verification and Closure
|
||||
|
||||
**Files:**
|
||||
- Modify: `porting.db`
|
||||
- Update generated report: `reports/current.md`
|
||||
|
||||
**Steps:**
|
||||
1. Run final gates:
|
||||
```bash
|
||||
dotnet build dotnet/
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
dotnet run --project tools/NatsNet.PortTracker -- audit --type features --db porting.db
|
||||
dotnet run --project tools/NatsNet.PortTracker -- audit --type tests --db porting.db
|
||||
```
|
||||
2. Resolve remaining `stub` items: promote with evidence or defer with reason.
|
||||
3. Complete batch:
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch complete 33 --db porting.db
|
||||
```
|
||||
4. Refresh report:
|
||||
```bash
|
||||
./reports/generate-report.sh
|
||||
```
|
||||
|
||||
149
docs/plans/2026-02-27-batch-34-js-cluster-consumers-design.md
Normal file
149
docs/plans/2026-02-27-batch-34-js-cluster-consumers-design.md
Normal file
@@ -0,0 +1,149 @@
|
||||
# Batch 34 JS Cluster Consumers Design
|
||||
|
||||
**Date:** 2026-02-27
|
||||
**Batch:** 34 (`JS Cluster Consumers`)
|
||||
**Scope:** 58 features + 160 unit tests
|
||||
**Dependency:** batch `33` (`JS Cluster Streams`)
|
||||
**Go source:** `golang/nats-server/server/jetstream_cluster.go`
|
||||
|
||||
## Problem
|
||||
|
||||
Batch 34 ports JetStream cluster consumer operations from `server/jetstream_cluster.go` (lines ~5935-8744), including consumer assignment/inflight reconciliation, replicated ack processing, leader-change handling, peer-group placement logic, clustered stream request handling, and stream/consumer mutation encoding/decoding.
|
||||
|
||||
The mapped test set is broad (160 tests across 29 test classes), so the design must enforce strict evidence gates and avoid fake progress through placeholder implementations.
|
||||
|
||||
## Context Findings
|
||||
|
||||
### Required command outputs
|
||||
|
||||
- `batch show 34 --db porting.db`
|
||||
- Status: `pending`
|
||||
- Features: `58` (all `deferred`)
|
||||
- Tests: `160` (all `deferred`)
|
||||
- Depends on: `33`
|
||||
- Go file: `server/jetstream_cluster.go`
|
||||
- `batch list --db porting.db`
|
||||
- Batch chain includes `33 -> 34 -> 38` for JS cluster consumer progression.
|
||||
- `report summary --db porting.db`
|
||||
- Overall progress: `1924/6942 (27.7%)`
|
||||
|
||||
Environment note: `dotnet` was not on `PATH` in this shell; commands need `/usr/local/share/dotnet/dotnet` fallback.
|
||||
|
||||
### Mapped feature ownership (from `porting.db`)
|
||||
|
||||
- `JetStreamCluster`: 19
|
||||
- `JetStreamEngine`: 13
|
||||
- `NatsServer`: 13
|
||||
- `NatsConsumer`: 7
|
||||
- `SelectPeerError`: 4
|
||||
- `JsAccount`: 1
|
||||
- `Account`: 1
|
||||
|
||||
### Mapped test distribution (top classes)
|
||||
|
||||
- `ServerOptionsTests` (28), `JwtProcessorTests` (20), `WebSocketHandlerTests` (14), `LeafNodeHandlerTests` (11), `JetStreamEngineTests` (11), `JetStreamClusterTests1` (10), plus 23 additional classes.
|
||||
|
||||
## Clarified Constraints
|
||||
|
||||
- Planning only in this session; no implementation execution.
|
||||
- Batch 0 guardrail rigor is mandatory and must be adapted for **features + tests**.
|
||||
- Feature work must be sliced into groups with max ~20 feature IDs.
|
||||
- Status updates must use `feature/test batch-update` chunks of max 15 IDs.
|
||||
- If blocked, mark `deferred` with explicit reason; do not write stubs.
|
||||
|
||||
## Approaches
|
||||
|
||||
### Approach A: Single large implementation pass
|
||||
|
||||
- Pros: low planning overhead.
|
||||
- Cons: poor auditability, high regression/stub risk, hard to isolate failures.
|
||||
|
||||
### Approach B (Recommended): Feature-first 3 groups, then 5 test waves, each with hard checkpoint gates
|
||||
|
||||
- Pros: bounded scope, auditable status transitions, faster root-cause isolation.
|
||||
- Cons: more CLI/test command overhead.
|
||||
|
||||
### Approach C: Test-first across all 160 before feature completion
|
||||
|
||||
- Pros: immediate behavior pressure.
|
||||
- Cons: high churn because many tests depend on not-yet-ported consumer cluster paths.
|
||||
|
||||
**Decision:** Approach B.
|
||||
|
||||
## Proposed Design
|
||||
|
||||
### 1. Architecture and File Ownership
|
||||
|
||||
Production code is split by behavior boundary instead of one monolithic file:
|
||||
|
||||
- `JetStream` consumer orchestration:
|
||||
- expected: `JetStream/JetStream.ClusterConsumers.cs` (create) or `JetStreamTypes.cs` (modify)
|
||||
- `NatsConsumer` cluster hooks:
|
||||
- expected: `JetStream/NatsConsumer.Cluster.cs` (create) or `NatsConsumer.cs` (modify)
|
||||
- `JetStreamCluster` placement + encoding/decoding:
|
||||
- expected: `JetStream/JetStreamCluster.Consumers.cs` (create) or `JetStreamClusterTypes.cs` (modify)
|
||||
- `NatsServer` clustered request/advisory endpoints:
|
||||
- expected: `NatsServer.JetStreamClusterConsumers.cs` (create) as partial server extension
|
||||
- `Account` limits selection helper:
|
||||
- expected: `Accounts/Account.JetStream.cs` (create) or `Accounts/Account.cs` (modify)
|
||||
|
||||
### 2. Feature Slicing (max ~20 IDs each)
|
||||
|
||||
- **Group A (20 IDs):** `1636-1655`
|
||||
Consumer assignment/inflight lookup, consumer raft-node helpers, monitor/apply entries, ack decode, leader advisory primitives.
|
||||
- **Group B (20 IDs):** `1656-1675`
|
||||
Assignment result processors, updates subscription lifecycle, leader-change flow, peer remap/selection foundation, tier/limits checks, base clustered stream request helpers.
|
||||
- **Group C (18 IDs):** `1676-1693`
|
||||
Clustered stream update/delete/purge/restore/list, consumer/message delete requests, and assignment/purge/message encode-decode helpers.
|
||||
|
||||
### 3. Test Slicing
|
||||
|
||||
- **Wave T1 (37 IDs):** JetStream cluster/consumer behavior core (`JetStreamClusterTests1/2/3/4`, `JetStreamEngineTests`, `NatsConsumerTests`)
|
||||
- **Wave T2 (39 IDs):** config/reload/options surface (`ServerOptionsTests`, `ConfigCheckTests`, `ConfigReloaderTests`, `NatsServerTests`)
|
||||
- **Wave T3 (33 IDs):** JWT/auth/cert/account validations (`JwtProcessorTests`, `JetStreamJwtTests`, `AuthCalloutTests`, `AuthHandlerTests`, `CertificateStoreWindowsTests`, `AccountTests`)
|
||||
- **Wave T4 (32 IDs):** transport + route + leaf/websocket (`WebSocketHandlerTests`, `LeafNodeHandlerTests`, `LeafNodeProxyTests`, `RouteHandlerTests`, `GatewayHandlerTests`)
|
||||
- **Wave T5 (19 IDs):** remaining integration-oriented regressions (`MqttHandlerTests`, `JetStreamLeafNodeTests`, `JetStreamSuperClusterTests`, `MessageTracerTests`, `MonitoringHandlerTests`, `EventsHandlerTests`, `JetStreamFileStoreTests`)
|
||||
|
||||
### 4. Verification Model
|
||||
|
||||
- Per-feature loop and per-test loop are mandatory.
|
||||
- Every loop requires:
|
||||
- stub detection scan
|
||||
- build gate
|
||||
- targeted test gate
|
||||
- Checkpoint required between all tasks before any `verified` promotion.
|
||||
- Status transitions are evidence-driven only:
|
||||
- `deferred/not_started -> stub -> complete -> verified`
|
||||
|
||||
### 5. Failure and Deferral Strategy
|
||||
|
||||
If blocked by missing infra/dependency behavior:
|
||||
|
||||
1. Stop the current item.
|
||||
2. Do not introduce placeholder logic or fake-pass tests.
|
||||
3. Mark item `deferred` with explicit reason via `--override`.
|
||||
4. Continue with next unblocked ID.
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
- **Dependency readiness risk (Batch 33):**
|
||||
Mitigation: hard preflight gate before starting Batch 34.
|
||||
- **Wide test blast radius (160 tests / 29 classes):**
|
||||
Mitigation: wave-based execution and strict checkpoints.
|
||||
- **Stub regression risk in ported methods/tests:**
|
||||
Mitigation: non-negotiable anti-stub scans and hard limits.
|
||||
- **Ownership ambiguity across partial classes:**
|
||||
Mitigation: explicit file ownership map and method-to-class grouping.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- All 58 features are `verified` with evidence or `deferred` with explicit blocker reason.
|
||||
- All 160 tests are `verified` with evidence or `deferred` with explicit blocker reason.
|
||||
- No forbidden stub patterns remain in touched production or test files.
|
||||
- Status updates are auditable and chunked (`<=15` IDs per `batch-update` call).
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Executing implementation in this planning session.
|
||||
- Expanding scope beyond Batch 34.
|
||||
- Building new infrastructure outside existing batch-mapped feature/test needs.
|
||||
@@ -0,0 +1,456 @@
|
||||
# Batch 34 JS Cluster Consumers Implementation Plan
|
||||
|
||||
> **For Codex:** REQUIRED SUB-SKILL: Use `executeplan` to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Port and verify Batch 34 (`JS Cluster Consumers`) from `server/jetstream_cluster.go`, with auditable status transitions for all 58 features and 160 tests.
|
||||
|
||||
**Architecture:** Implement consumer-cluster behavior in three bounded feature groups (`20/20/18`) across `JetStream`, `JetStreamCluster`, `NatsConsumer`, `NatsServer`, and `Account`. Then port mapped tests in five waves (`37/39/33/32/19`) with strict per-item verification evidence, anti-stub enforcement, and checkpoint gates between tasks.
|
||||
|
||||
**Tech Stack:** .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (`porting.db`)
|
||||
|
||||
**Design doc:** `docs/plans/2026-02-27-batch-34-js-cluster-consumers-design.md`
|
||||
|
||||
---
|
||||
|
||||
## Batch 34 Scope
|
||||
|
||||
- Batch ID: `34`
|
||||
- Name: `JS Cluster Consumers`
|
||||
- Dependency: `33`
|
||||
- Go source: `golang/nats-server/server/jetstream_cluster.go`
|
||||
- Features: `58` (`1636-1693`)
|
||||
- Tests: `160` (IDs listed in wave sections below)
|
||||
|
||||
If `dotnet` is not on `PATH`, use:
|
||||
|
||||
```bash
|
||||
DOTNET=/usr/local/share/dotnet/dotnet
|
||||
```
|
||||
|
||||
Primary production files (expected touch set):
|
||||
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStream.ClusterConsumers.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamCluster.Consumers.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Cluster.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterConsumers.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.JetStream.cs`
|
||||
- Modify (as needed): `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
|
||||
- Modify (as needed): `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs`
|
||||
- Modify (as needed): `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.cs`
|
||||
- Modify (as needed): `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs`
|
||||
|
||||
Primary test files (expected touch set):
|
||||
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests4.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ServerOptionsTests.Impltests.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConfigCheckTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConfigReloaderTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsServerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JwtProcessorTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamJwtTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AuthCalloutTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AuthHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/CertificateStoreWindowsTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AccountTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/WebSocketHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/LeafNodeHandlerTests.Impltests.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/LeafNodeProxyTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RouteHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/GatewayHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamLeafNodeTests.Impltests.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamSuperClusterTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MessageTracerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MonitoringHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/EventsHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs`
|
||||
|
||||
---
|
||||
|
||||
## MANDATORY VERIFICATION PROTOCOL
|
||||
|
||||
> **NON-NEGOTIABLE:** Every Batch 34 feature/test ID must pass this protocol before promotion.
|
||||
|
||||
### Dependency Preflight Gate (before any status changes)
|
||||
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 33 --db porting.db
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 34 --db porting.db
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch ready --db porting.db
|
||||
```
|
||||
|
||||
Only when dependency is satisfied and batch is ready:
|
||||
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch start 34 --db porting.db
|
||||
```
|
||||
|
||||
Baseline evidence:
|
||||
|
||||
```bash
|
||||
$DOTNET build dotnet/
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
```
|
||||
|
||||
### Per-Feature Verification Loop (REQUIRED per feature ID)
|
||||
|
||||
1. Inspect mapping and Go span:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- feature show <feature_id> --db porting.db
|
||||
```
|
||||
2. Claim only that feature ID:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status stub --db porting.db
|
||||
```
|
||||
3. Add/update at least one focused test case that fails for the intended behavior.
|
||||
4. Run focused test and confirm failure is for the target behavior.
|
||||
5. Implement minimal production logic for that feature ID.
|
||||
6. Run **Stub Detection Check** (below).
|
||||
7. Run **Build Gate**.
|
||||
8. Run **Test Gate** (focused/class filters).
|
||||
9. If green, mark feature `complete` (promote to `verified` only at checkpoint).
|
||||
|
||||
### Per-Test Verification Loop (REQUIRED per test ID)
|
||||
|
||||
1. Inspect mapping and Go source:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- test show <test_id> --db porting.db
|
||||
```
|
||||
2. Claim only that test ID:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- test update <test_id> --status stub --db porting.db
|
||||
```
|
||||
3. Port full Arrange/Act/Assert behavior (no placeholders).
|
||||
4. Run single test method:
|
||||
```bash
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
|
||||
--filter "FullyQualifiedName~<ClassName>.<MethodName>" --verbosity normal
|
||||
```
|
||||
5. Confirm summary contains `Passed: 1, Failed: 0` (not `Passed: 0`).
|
||||
6. Run class-level filter for the touched class.
|
||||
7. Run **Stub Detection Check** + **Build Gate**.
|
||||
8. If green, mark test `complete` (promote to `verified` only at checkpoint).
|
||||
|
||||
### Stub Detection Check (REQUIRED after every feature/test loop)
|
||||
|
||||
Run against changed C# files:
|
||||
|
||||
```bash
|
||||
changed_files=$(git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests | rg "\.cs$" || true)
|
||||
if [ -n "$changed_files" ]; then
|
||||
echo "$changed_files" | xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|Assert\.True\(true\)|Assert\.Pass|ShouldBe\(true\)|=>\s*default;|=>\s*null;|return\s+null;\s*$|return\s+0;\s*$|return\s+false;\s*$)"
|
||||
fi
|
||||
```
|
||||
|
||||
Any match in mapped methods/tests blocks promotion.
|
||||
|
||||
### Build Gate (REQUIRED)
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
$DOTNET build dotnet/
|
||||
```
|
||||
|
||||
Required timing:
|
||||
|
||||
- after each feature loop
|
||||
- after each test loop
|
||||
- before every `batch-update`
|
||||
- at every task checkpoint
|
||||
|
||||
### Test Gate (REQUIRED)
|
||||
|
||||
Minimum gates:
|
||||
|
||||
```bash
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~JetStream"
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog"
|
||||
```
|
||||
|
||||
Plus per-item focused gates:
|
||||
|
||||
- feature loop: nearest impacted class filters
|
||||
- test loop: method filter, then class filter
|
||||
|
||||
### Status Update Protocol (HARD LIMIT: 15 IDs max per `batch-update`)
|
||||
|
||||
- Never pass more than `15` IDs in one `feature batch-update` or `test batch-update`.
|
||||
- Never promote `verified` without loop evidence + checkpoint evidence.
|
||||
- Never update IDs outside the active task.
|
||||
- For blocked IDs, keep `deferred` and provide explicit `--override` reason.
|
||||
|
||||
Templates:
|
||||
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- \
|
||||
feature batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
|
||||
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- \
|
||||
test batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
|
||||
```
|
||||
|
||||
### Checkpoint Protocol Between Tasks (REQUIRED)
|
||||
|
||||
Before starting the next task:
|
||||
|
||||
1. Run **Stub Detection Check**.
|
||||
2. Run **Build Gate**.
|
||||
3. Run targeted class filters for all touched files.
|
||||
4. Run full unit suite:
|
||||
```bash
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
```
|
||||
5. Promote proven IDs only, in `<=15` chunks.
|
||||
6. Commit checkpoint before moving on.
|
||||
|
||||
---
|
||||
|
||||
## ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)
|
||||
|
||||
These rules apply to both production features and tests.
|
||||
|
||||
### Forbidden Patterns
|
||||
|
||||
- `throw new NotImplementedException()` in any mapped Batch 34 method/test.
|
||||
- Empty method bodies for mapped features.
|
||||
- Placeholder markers in mapped code (`TODO`, `PLACEHOLDER`, `later`).
|
||||
- Fake pass assertions (`Assert.True(true)`, `Assert.Pass()`, one-line tautologies).
|
||||
- Constant-return shortcuts that ignore inputs/state (`return false;`, `return 0;`, `return null;`, `return string.Empty;`).
|
||||
- Catch-and-ignore patterns that suppress failures to force green tests.
|
||||
|
||||
### Hard Limits
|
||||
|
||||
- Feature task groups: max `~20` IDs.
|
||||
- `feature batch-update`: max `15` IDs per call.
|
||||
- `test batch-update`: max `15` IDs per call.
|
||||
- One active feature loop at a time.
|
||||
- One active test loop at a time.
|
||||
- Mandatory checkpoint between all tasks.
|
||||
- No `verified` promotions without checkpoint evidence.
|
||||
|
||||
### If You Get Stuck (REQUIRED)
|
||||
|
||||
1. Stop on the current ID immediately.
|
||||
2. Do not add placeholder logic or fake assertions.
|
||||
3. Mark blocked item `deferred` with explicit reason:
|
||||
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- \
|
||||
feature update <feature_id> --status deferred --override "blocked: <specific reason>" --db porting.db
|
||||
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- \
|
||||
test update <test_id> --status deferred --override "blocked: <specific reason>" --db porting.db
|
||||
```
|
||||
|
||||
4. Continue with the next unblocked ID.
|
||||
|
||||
Deferred-with-reason is correct. Stubbed/fake-pass progress is not.
|
||||
|
||||
---
|
||||
|
||||
## Feature Groups (max ~20 each)
|
||||
|
||||
### Group A (20): consumer assignment + consumer raft hooks + ack/leader foundations
|
||||
|
||||
IDs:
|
||||
`1636,1637,1638,1639,1640,1641,1642,1643,1644,1645,1646,1647,1648,1649,1650,1651,1652,1653,1654,1655`
|
||||
|
||||
### Group B (20): assignment-result processing + peer selection + clustered request prep
|
||||
|
||||
IDs:
|
||||
`1656,1657,1658,1659,1660,1661,1662,1663,1664,1665,1666,1667,1668,1669,1670,1671,1672,1673,1674,1675`
|
||||
|
||||
### Group C (18): clustered update/delete/list + encode/decode helpers
|
||||
|
||||
IDs:
|
||||
`1676,1677,1678,1679,1680,1681,1682,1683,1684,1685,1686,1687,1688,1689,1690,1691,1692,1693`
|
||||
|
||||
## Test Waves
|
||||
|
||||
### Wave T1 (37): JetStream cluster consumer core
|
||||
|
||||
IDs:
|
||||
`757,761,778,802,813,818,834,881,883,900,911,929,963,964,986,1003,1059,1108,1151,1193,1208,1231,1233,1237,1290,1342,1553,1576,1582,1587,1599,1627,1633,1662,1663,1674,1747`
|
||||
|
||||
### Wave T2 (39): options/config/reload/server validations
|
||||
|
||||
IDs:
|
||||
`271,272,273,2515,2534,2535,2536,2543,2548,2554,2558,2559,2562,2564,2568,2570,2573,2574,2575,2577,2578,2579,2580,2581,2582,2584,2591,2592,2594,2595,2596,2723,2740,2763,2765,2777,2785,2889,2892`
|
||||
|
||||
### Wave T3 (33): JWT/auth/cert/account validations
|
||||
|
||||
IDs:
|
||||
`103,126,136,139,147,151,155,156,157,1386,1388,1391,1397,1834,1854,1856,1859,1862,1863,1865,1866,1868,1869,1870,1871,1876,1877,1878,1879,1880,1882,1885,1886`
|
||||
|
||||
### Wave T4 (32): websocket/leaf/route/gateway behavior
|
||||
|
||||
IDs:
|
||||
`655,1904,1905,1910,1913,1914,1920,1931,1937,1938,1941,1969,1972,1979,2824,2829,2845,2849,3074,3081,3090,3091,3092,3094,3096,3100,3101,3107,3108,3123,3129,3133`
|
||||
|
||||
### Wave T5 (19): remaining JS-related integration regressions
|
||||
|
||||
IDs:
|
||||
`342,599,1405,1408,1422,1430,1439,1441,1444,1448,2146,2172,2173,2176,2180,2186,2264,2332,2352`
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Preflight and Baseline
|
||||
|
||||
**Files:**
|
||||
- Read: `docs/plans/2026-02-27-batch-34-js-cluster-consumers-design.md`
|
||||
- Read: `golang/nats-server/server/jetstream_cluster.go`
|
||||
|
||||
**Steps:**
|
||||
1. Run dependency preflight commands and ensure Batch 34 is startable.
|
||||
2. Start Batch 34.
|
||||
3. Capture baseline build and test results.
|
||||
4. Do not update statuses beyond `stub` markers before baseline evidence is captured.
|
||||
|
||||
### Task 2: Implement Feature Group A (20 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStream.ClusterConsumers.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsConsumer.Cluster.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamCluster.Consumers.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterConsumers.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Set Group A IDs to `stub` using two commands (`15 + 5` IDs).
|
||||
2. Run Per-Feature Verification Loop for each Group A ID.
|
||||
3. Execute required gates after every ID.
|
||||
4. At checkpoint, promote proven IDs (`complete`, then `verified`) in `<=15` chunks.
|
||||
|
||||
### Task 3: Implement Feature Group B (20 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStream.ClusterConsumers.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamCluster.Consumers.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterConsumers.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.JetStream.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Set Group B IDs to `stub` using two commands (`15 + 5` IDs).
|
||||
2. Run Per-Feature Verification Loop for each Group B ID.
|
||||
3. Execute required gates after every ID.
|
||||
4. At checkpoint, promote proven IDs in `<=15` chunks.
|
||||
|
||||
### Task 4: Implement Feature Group C (18 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterConsumers.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamCluster.Consumers.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStream.ClusterConsumers.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Set Group C IDs to `stub` using two commands (`15 + 3` IDs).
|
||||
2. Run Per-Feature Verification Loop for each Group C ID.
|
||||
3. Execute required gates after every ID.
|
||||
4. At checkpoint, promote proven IDs in `<=15` chunks.
|
||||
|
||||
### Task 5: Port Test Wave T1 (37 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests4.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Set T1 IDs to `stub` in `<=15` ID chunks.
|
||||
2. Run Per-Test Verification Loop for each ID.
|
||||
3. Run class filters per touched class.
|
||||
4. Execute checkpoint and promote proven IDs in `<=15` chunks.
|
||||
|
||||
### Task 6: Port Test Wave T2 (39 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ServerOptionsTests.Impltests.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConfigCheckTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConfigReloaderTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsServerTests.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Set T2 IDs to `stub` in `<=15` ID chunks.
|
||||
2. Run Per-Test Verification Loop for each ID.
|
||||
3. Run class filters per touched class.
|
||||
4. Execute checkpoint and promote proven IDs in `<=15` chunks.
|
||||
|
||||
### Task 7: Port Test Wave T3 (33 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JwtProcessorTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamJwtTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AuthCalloutTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AuthHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/CertificateStoreWindowsTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AccountTests.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Set T3 IDs to `stub` in `<=15` ID chunks.
|
||||
2. Run Per-Test Verification Loop for each ID.
|
||||
3. Run class filters per touched class.
|
||||
4. Execute checkpoint and promote proven IDs in `<=15` chunks.
|
||||
|
||||
### Task 8: Port Test Wave T4 (32 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/WebSocketHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/LeafNodeHandlerTests.Impltests.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/LeafNodeProxyTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RouteHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/GatewayHandlerTests.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Set T4 IDs to `stub` in `<=15` ID chunks.
|
||||
2. Run Per-Test Verification Loop for each ID.
|
||||
3. Run class filters per touched class.
|
||||
4. Execute checkpoint and promote proven IDs in `<=15` chunks.
|
||||
|
||||
### Task 9: Port Test Wave T5 (19 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamLeafNodeTests.Impltests.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamSuperClusterTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MessageTracerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MonitoringHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/EventsHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Set T5 IDs to `stub` in `<=15` ID chunks.
|
||||
2. Run Per-Test Verification Loop for each ID.
|
||||
3. Run class filters per touched class.
|
||||
4. Execute checkpoint and promote proven IDs in `<=15` chunks.
|
||||
|
||||
### Task 10: Final Batch Verification and Closure
|
||||
|
||||
**Files:**
|
||||
- Modify: `porting.db`
|
||||
- Modify: `reports/current.md` (via generator)
|
||||
|
||||
**Steps:**
|
||||
1. Run final gates (`stub scan`, `build`, full unit tests).
|
||||
2. Run audits and reconcile status mismatches:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- audit --type features --db porting.db
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- audit --type tests --db porting.db
|
||||
```
|
||||
3. Ensure all Batch 34 IDs are `verified` or explicit `deferred` with reason.
|
||||
4. Complete batch:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch complete 34 --db porting.db
|
||||
```
|
||||
5. Generate report:
|
||||
```bash
|
||||
./reports/generate-report.sh
|
||||
```
|
||||
|
||||
135
docs/plans/2026-02-27-batch-35-js-cluster-remaining-design.md
Normal file
135
docs/plans/2026-02-27-batch-35-js-cluster-remaining-design.md
Normal file
@@ -0,0 +1,135 @@
|
||||
# Batch 35 JS Cluster Remaining Design
|
||||
|
||||
**Date:** 2026-02-27
|
||||
**Batch:** 35 (`JS Cluster Remaining`)
|
||||
**Scope:** 57 features + 49 unit tests
|
||||
**Dependency:** batch `32` (`JS Cluster Meta`)
|
||||
**Go source:** `golang/nats-server/server/jetstream_cluster.go`
|
||||
|
||||
## Problem
|
||||
|
||||
Batch 35 covers the remaining JetStream cluster behavior in `server/jetstream_cluster.go` (roughly lines `8766-10866`): delete-range and assignment encoding/decoding, stream snapshot/catchup processing, cluster info assembly, catchup throttling counters, and sync subject helpers. It also includes 49 tests, mostly `RaftNodeTests`, that validate catchup/truncation and snapshot correctness.
|
||||
|
||||
The plan must prevent false progress: no placeholder feature ports and no fake-pass tests.
|
||||
|
||||
## Context Findings
|
||||
|
||||
### Required command outputs
|
||||
|
||||
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch show 35 --db porting.db`
|
||||
- Status: `pending`
|
||||
- Features: `57` (all `deferred`)
|
||||
- Tests: `49` (all `deferred`)
|
||||
- Depends on: `32`
|
||||
- Go file: `server/jetstream_cluster.go`
|
||||
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch list --db porting.db`
|
||||
- Confirms chain around this area: `32 -> 33/34/35`.
|
||||
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db`
|
||||
- Overall progress: `1924/6942 (27.7%)`
|
||||
|
||||
Environment note: `dotnet` was not on `PATH` in this shell; use `/usr/local/share/dotnet/dotnet`.
|
||||
|
||||
### Feature ownership distribution (from `porting.db`)
|
||||
|
||||
- `NatsStream`: 27
|
||||
- `JetStreamCluster`: 19
|
||||
- `NatsServer`: 8
|
||||
- `JetStreamEngine`: 3
|
||||
|
||||
### Test distribution (from `porting.db`)
|
||||
|
||||
- `RaftNodeTests`: 42
|
||||
- `JetStreamClusterTests1`: 6
|
||||
- `JetStreamBatchingTests`: 1
|
||||
|
||||
## Constraints and Success Criteria
|
||||
|
||||
- Planning only; no implementation execution in this session.
|
||||
- Reuse Batch 0 rigor, but for **features + tests**.
|
||||
- Feature tasks must be grouped in chunks of max ~20 features.
|
||||
- Status updates must use batch-update chunks of at most 15 IDs.
|
||||
- Blocked work must be marked `deferred` with explicit reason, never stubbed.
|
||||
|
||||
Success means all 57 feature IDs and 49 test IDs are either:
|
||||
- promoted with verification evidence (`complete/verified`), or
|
||||
- kept `deferred` with specific blocker notes.
|
||||
|
||||
## Approaches
|
||||
|
||||
### Approach A: Monolithic pass (all features, then all tests)
|
||||
|
||||
- Pros: simple sequencing.
|
||||
- Cons: high risk, poor auditability, hard to isolate regressions.
|
||||
|
||||
### Approach B (Recommended): Three feature groups plus four test waves with hard gates
|
||||
|
||||
- Pros: bounded scope, clearer rollback points, aligns with max-20 feature grouping and max-15 status updates.
|
||||
- Cons: more command overhead.
|
||||
|
||||
### Approach C: Test-first for all 49 tests before feature completion
|
||||
|
||||
- Pros: immediate behavior pressure.
|
||||
- Cons: high churn because most tests depend on unported catchup/snapshot internals.
|
||||
|
||||
**Decision:** Approach B.
|
||||
|
||||
## Proposed Design
|
||||
|
||||
### 1. Code Organization
|
||||
|
||||
Use behavior-focused files while keeping existing class ownership:
|
||||
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs`
|
||||
- assignment/sync subject encode/decode helpers and cluster utility functions.
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs`
|
||||
- snapshot and catchup state transitions plus inbound cluster message processing.
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
|
||||
- offline/online cluster info and alternate-stream assembly.
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.cs` and/or new partial `NatsServer.JetStreamClusterRemaining.cs`
|
||||
- clustered consumer request path and gcb accounting helpers.
|
||||
|
||||
### 2. Feature Grouping (max ~20)
|
||||
|
||||
- **Group A (20 IDs):** `1694-1713`
|
||||
Delete-range, consumer assignment, stream/batch encode-decode, snapshot support/state capture, clustered inbound message entry point.
|
||||
- **Group B (20 IDs):** `1714-1733`
|
||||
Delete trace, sync request calculation, snapshot delete handling, catchup peer lifecycle, snapshot/catchup processing, stream sync handler, cluster info base methods.
|
||||
- **Group C (17 IDs):** `1734-1750`
|
||||
Cluster info checks, stream alternates/info request handling, gcb accounting/kick channel, run-catchup loop, sync subject helper family.
|
||||
|
||||
### 3. Test Wave Grouping
|
||||
|
||||
- **Wave T1 (7 IDs):** `730,846,847,848,890,891,893` (`JetStreamBatchingTests` + `JetStreamClusterTests1`)
|
||||
- **Wave T2 (14 IDs):** `2640,2641,2643,2644,2645,2646,2647,2648,2649,2653,2655,2656,2658,2659`
|
||||
- **Wave T3 (14 IDs):** `2660,2661,2662,2665,2666,2668,2669,2673,2676,2677,2678,2679,2680,2681`
|
||||
- **Wave T4 (14 IDs):** `2682,2683,2684,2685,2686,2688,2691,2696,2697,2703,2715,2716,2717,2719`
|
||||
|
||||
### 4. Verification Strategy (Design-Level)
|
||||
|
||||
- Every feature follows a per-feature loop with focused test evidence.
|
||||
- Every test follows method-level then class-level verification.
|
||||
- Stub detection runs after each loop and before status promotions.
|
||||
- Build and targeted/full test gates are mandatory before checkpoint status updates.
|
||||
- Checkpoints occur between every task boundary.
|
||||
|
||||
### 5. Deferral Strategy
|
||||
|
||||
If infrastructure or dependency behavior blocks a feature/test:
|
||||
|
||||
1. Stop work on that ID.
|
||||
2. Do not add placeholder implementation/assertions.
|
||||
3. Mark `deferred` with explicit reason via `--override`.
|
||||
4. Continue with next unblocked item.
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
- **Dependency readiness risk (Batch 32):** enforce preflight before `batch start 35`.
|
||||
- **Raft-heavy test concentration:** split `RaftNodeTests` into three equal waves and checkpoint each wave.
|
||||
- **Stub regression under volume:** hard anti-stub scans and strict status chunking (`<=15`).
|
||||
- **Class ownership drift:** keep methods in mapped classes only (`JetStreamCluster`, `NatsStream`, `JetStreamEngine`, `NatsServer`).
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Executing the implementation in this session.
|
||||
- Expanding scope beyond Batch 35 mappings.
|
||||
- Changing batch dependencies/order in PortTracker.
|
||||
@@ -0,0 +1,410 @@
|
||||
# Batch 35 JS Cluster Remaining Implementation Plan
|
||||
|
||||
> **For Codex:** REQUIRED SUB-SKILL: Use `executeplan` to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Port and verify Batch 35 (`JS Cluster Remaining`) from `server/jetstream_cluster.go`, with auditable status transitions for all 57 features and 49 tests.
|
||||
|
||||
**Architecture:** Execute three bounded feature groups (`20/20/17`) mapped to `JetStreamCluster`, `NatsStream`, `JetStreamEngine`, and `NatsServer`, then execute four test waves (`7/14/14/14`) with strict method-level verification evidence. Every promotion is gated by stub scans, build/test gates, and checkpoint protocol between tasks.
|
||||
|
||||
**Tech Stack:** .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (`porting.db`)
|
||||
|
||||
**Design doc:** `docs/plans/2026-02-27-batch-35-js-cluster-remaining-design.md`
|
||||
|
||||
---
|
||||
|
||||
## Batch 35 Scope
|
||||
|
||||
- Batch ID: `35`
|
||||
- Name: `JS Cluster Remaining`
|
||||
- Dependency: `32`
|
||||
- Go source: `golang/nats-server/server/jetstream_cluster.go`
|
||||
- Features: `57` (`1694-1750`)
|
||||
- Tests: `49` (`730,846,847,848,890,891,893,2640,2641,2643,2644,2645,2646,2647,2648,2649,2653,2655,2656,2658,2659,2660,2661,2662,2665,2666,2668,2669,2673,2676,2677,2678,2679,2680,2681,2682,2683,2684,2685,2686,2688,2691,2696,2697,2703,2715,2716,2717,2719`)
|
||||
|
||||
If `dotnet` is not on `PATH`, use:
|
||||
|
||||
```bash
|
||||
DOTNET=/usr/local/share/dotnet/dotnet
|
||||
```
|
||||
|
||||
Primary production files (expected touch set):
|
||||
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
|
||||
- Modify or Create: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterRemaining.cs`
|
||||
- Modify (if not using partial file): `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.cs`
|
||||
|
||||
Primary test files (expected touch set):
|
||||
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamBatchingTests.Impltests.cs`
|
||||
- Modify or Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs`
|
||||
- Modify or Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
||||
|
||||
---
|
||||
|
||||
## MANDATORY VERIFICATION PROTOCOL
|
||||
|
||||
> **NON-NEGOTIABLE:** Every Batch 35 feature/test ID must pass this protocol before promotion.
|
||||
|
||||
### Dependency Preflight Gate (before any status changes)
|
||||
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 32 --db porting.db
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 35 --db porting.db
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch ready --db porting.db
|
||||
```
|
||||
|
||||
Only if Batch 35 is ready:
|
||||
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch start 35 --db porting.db
|
||||
```
|
||||
|
||||
Capture baseline evidence:
|
||||
|
||||
```bash
|
||||
$DOTNET build dotnet/
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
```
|
||||
|
||||
### Per-Feature Verification Loop (REQUIRED per feature ID)
|
||||
|
||||
1. Inspect mapping and Go line span:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- feature show <feature_id> --db porting.db
|
||||
```
|
||||
2. Mark only that feature as in-progress:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status stub --db porting.db
|
||||
```
|
||||
3. Add or update at least one focused test that fails for intended behavior.
|
||||
4. Run focused test; verify failure reason matches target behavior.
|
||||
5. Implement minimal feature logic (no placeholder code).
|
||||
6. Run **Stub Detection Check**.
|
||||
7. Run **Build Gate**.
|
||||
8. Run **Test Gate** (method/class filters for impacted area).
|
||||
9. If green, mark feature `complete`; defer `verified` promotion until task checkpoint.
|
||||
|
||||
### Per-Test Verification Loop (REQUIRED per test ID)
|
||||
|
||||
1. Inspect mapping and Go source:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- test show <test_id> --db porting.db
|
||||
```
|
||||
2. Mark only that test as in-progress:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- test update <test_id> --status stub --db porting.db
|
||||
```
|
||||
3. Port full Arrange/Act/Assert behavior.
|
||||
4. Run single test method:
|
||||
```bash
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
|
||||
--filter "FullyQualifiedName~<ClassName>.<MethodName>" --verbosity normal
|
||||
```
|
||||
5. Confirm summary includes `Passed: 1, Failed: 0` (never `Passed: 0`).
|
||||
6. Run class-level filter for touched class.
|
||||
7. Run **Stub Detection Check** + **Build Gate**.
|
||||
8. If green, mark test `complete`; defer `verified` promotion until task checkpoint.
|
||||
|
||||
### Stub Detection Check (REQUIRED after every feature/test loop)
|
||||
|
||||
Run on changed C# files only:
|
||||
|
||||
```bash
|
||||
changed_files=$(git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests | rg "\\.cs$" || true)
|
||||
if [ -n "$changed_files" ]; then
|
||||
echo "$changed_files" | xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|Assert\\.True\\(true\\)|Assert\\.Pass|ShouldBe\\(true\\)|=>\\s*default;|=>\\s*null;|return\\s+null;\\s*$|return\\s+0;\\s*$|return\\s+false;\\s*$)"
|
||||
fi
|
||||
```
|
||||
|
||||
Any match in mapped methods/tests blocks promotion.
|
||||
|
||||
### Build Gate (REQUIRED)
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
$DOTNET build dotnet/
|
||||
```
|
||||
|
||||
Required timing:
|
||||
|
||||
- after each feature loop
|
||||
- after each test loop
|
||||
- before every `batch-update`
|
||||
- at every checkpoint
|
||||
|
||||
### Test Gate (REQUIRED)
|
||||
|
||||
Minimum gates:
|
||||
|
||||
```bash
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~JetStream"
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~RaftNodeTests"
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog"
|
||||
```
|
||||
|
||||
Per-item focused gates are mandatory:
|
||||
|
||||
- Feature loop: nearest impacted class filters.
|
||||
- Test loop: method filter, then class filter.
|
||||
|
||||
### Status Update Protocol (HARD LIMIT: 15 IDs max per `batch-update`)
|
||||
|
||||
- Never pass more than `15` IDs in one `feature batch-update` or `test batch-update` call.
|
||||
- Never promote `verified` without loop evidence and checkpoint evidence.
|
||||
- Never update IDs outside active task scope.
|
||||
- For blocked items, keep `deferred` and provide explicit `--override` reason.
|
||||
|
||||
Templates:
|
||||
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- \
|
||||
feature batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
|
||||
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- \
|
||||
test batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
|
||||
```
|
||||
|
||||
### Checkpoint Protocol Between Tasks (REQUIRED)
|
||||
|
||||
Before starting the next task:
|
||||
|
||||
1. Run **Stub Detection Check**.
|
||||
2. Run **Build Gate**.
|
||||
3. Run class filters for all touched classes.
|
||||
4. Run full unit suite:
|
||||
```bash
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
```
|
||||
5. Promote only proven IDs in `<=15` chunks.
|
||||
6. Commit checkpoint before next task.
|
||||
|
||||
---
|
||||
|
||||
## ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)
|
||||
|
||||
These apply to features and tests.
|
||||
|
||||
### Forbidden Patterns
|
||||
|
||||
- `throw new NotImplementedException()` in any mapped Batch 35 feature/test.
|
||||
- Empty method bodies for mapped features.
|
||||
- Placeholder comments in mapped paths (`TODO`, `PLACEHOLDER`, `later`).
|
||||
- Fake-pass assertions (`Assert.True(true)`, `Assert.Pass()`, tautological assertions).
|
||||
- Constant-return shortcuts in mapped features that ignore inputs/state (`return false;`, `return 0;`, `return null;`, `return string.Empty;`).
|
||||
- Catch-and-ignore blocks that suppress failures to force green results.
|
||||
|
||||
### Hard Limits
|
||||
|
||||
- Feature task group size max: `~20` IDs.
|
||||
- `feature batch-update` max IDs: `15`.
|
||||
- `test batch-update` max IDs: `15`.
|
||||
- One active feature loop at a time.
|
||||
- One active test loop at a time.
|
||||
- Mandatory checkpoint between every task.
|
||||
- No `verified` promotion without checkpoint evidence.
|
||||
|
||||
### If You Get Stuck (REQUIRED)
|
||||
|
||||
1. Stop on the current ID immediately.
|
||||
2. Do not add placeholder implementation or fake assertions.
|
||||
3. Mark blocked item `deferred` with explicit reason.
|
||||
4. Move to next unblocked ID.
|
||||
|
||||
Feature deferral command:
|
||||
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- \
|
||||
feature update <feature_id> --status deferred --override "blocked: <specific reason>" --db porting.db
|
||||
```
|
||||
|
||||
Test deferral command:
|
||||
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- \
|
||||
test update <test_id> --status deferred --override "blocked: <specific reason>" --db porting.db
|
||||
```
|
||||
|
||||
Deferred with reason is correct behavior. Stubbed/fake-pass progress is not.
|
||||
|
||||
---
|
||||
|
||||
## Feature Groups (max ~20 each)
|
||||
|
||||
### Group A (20): delete-range/assignment codecs + snapshot capability foundations
|
||||
|
||||
IDs:
|
||||
`1694,1695,1696,1697,1698,1699,1700,1701,1702,1703,1704,1705,1706,1707,1708,1709,1710,1711,1712,1713`
|
||||
|
||||
### Group B (20): snapshot delete/catchup peer lifecycle + sync/catchup processing + cluster info base
|
||||
|
||||
IDs:
|
||||
`1714,1715,1716,1717,1718,1719,1720,1721,1722,1723,1724,1725,1726,1727,1728,1729,1730,1731,1732,1733`
|
||||
|
||||
### Group C (17): cluster stream info, gcb controls, runCatchup, and sync subject helpers
|
||||
|
||||
IDs:
|
||||
`1734,1735,1736,1737,1738,1739,1740,1741,1742,1743,1744,1745,1746,1747,1748,1749,1750`
|
||||
|
||||
## Test Waves
|
||||
|
||||
### Wave T1 (7): batching + cluster-meta-recovery/offline strict decoding
|
||||
|
||||
IDs:
|
||||
`730,846,847,848,890,891,893`
|
||||
|
||||
### Wave T2 (14): raft catchup/truncate/snapshot baseline
|
||||
|
||||
IDs:
|
||||
`2640,2641,2643,2644,2645,2646,2647,2648,2649,2653,2655,2656,2658,2659`
|
||||
|
||||
### Wave T3 (14): raft health/quorum/leader-change/snapshot replay
|
||||
|
||||
IDs:
|
||||
`2660,2661,2662,2665,2666,2668,2669,2673,2676,2677,2678,2679,2680,2681`
|
||||
|
||||
### Wave T4 (14): raft startup/rollback/install snapshot and peer replay
|
||||
|
||||
IDs:
|
||||
`2682,2683,2684,2685,2686,2688,2691,2696,2697,2703,2715,2716,2717,2719`
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Preflight and Baseline
|
||||
|
||||
**Files:**
|
||||
- Read: `docs/plans/2026-02-27-batch-35-js-cluster-remaining-design.md`
|
||||
- Read: `golang/nats-server/server/jetstream_cluster.go`
|
||||
|
||||
**Steps:**
|
||||
1. Run dependency preflight commands.
|
||||
2. Start Batch 35 only if dependency checks pass.
|
||||
3. Capture baseline build/test evidence.
|
||||
4. Do not promote any ID before baseline evidence exists.
|
||||
|
||||
### Task 2: Implement Feature Group A (20 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterRemaining.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Move Group A IDs to `stub` in two `batch-update` calls (`15 + 5`).
|
||||
2. Execute Per-Feature Verification Loop for each ID.
|
||||
3. Run mandatory gates after each ID.
|
||||
4. Apply checkpoint protocol.
|
||||
5. Promote proven Group A IDs to `complete/verified` in `<=15` chunks.
|
||||
6. Commit checkpoint.
|
||||
|
||||
### Task 3: Implement Feature Group B (20 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Move Group B IDs to `stub` in two `batch-update` calls (`15 + 5`).
|
||||
2. Execute Per-Feature Verification Loop for each ID.
|
||||
3. Run mandatory gates after each ID.
|
||||
4. Apply checkpoint protocol.
|
||||
5. Promote proven Group B IDs to `complete/verified` in `<=15` chunks.
|
||||
6. Commit checkpoint.
|
||||
|
||||
### Task 4: Implement Feature Group C (17 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterRemaining.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Move Group C IDs to `stub` in two `batch-update` calls (`15 + 2`).
|
||||
2. Execute Per-Feature Verification Loop for each ID.
|
||||
3. Run mandatory gates after each ID.
|
||||
4. Apply checkpoint protocol.
|
||||
5. Promote proven Group C IDs to `complete/verified` in `<=15` chunks.
|
||||
6. Commit checkpoint.
|
||||
|
||||
### Task 5: Port Test Wave T1 (7 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamBatchingTests.Impltests.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Move T1 IDs to `stub` in one call.
|
||||
2. Execute Per-Test Verification Loop for each ID.
|
||||
3. Run class-level filters for touched classes.
|
||||
4. Apply checkpoint protocol.
|
||||
5. Promote proven T1 IDs to `complete/verified` in `<=15` chunks.
|
||||
6. Commit checkpoint.
|
||||
|
||||
### Task 6: Port Test Wave T2 (14 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Move T2 IDs to `stub` in one call.
|
||||
2. Execute Per-Test Verification Loop for each ID.
|
||||
3. Run class-level `RaftNodeTests` filter.
|
||||
4. Apply checkpoint protocol.
|
||||
5. Promote proven T2 IDs to `complete/verified` in `<=15` chunks.
|
||||
6. Commit checkpoint.
|
||||
|
||||
### Task 7: Port Test Wave T3 (14 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Move T3 IDs to `stub` in one call.
|
||||
2. Execute Per-Test Verification Loop for each ID.
|
||||
3. Run class-level `RaftNodeTests` filter.
|
||||
4. Apply checkpoint protocol.
|
||||
5. Promote proven T3 IDs to `complete/verified` in `<=15` chunks.
|
||||
6. Commit checkpoint.
|
||||
|
||||
### Task 8: Port Test Wave T4 (14 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Move T4 IDs to `stub` in one call.
|
||||
2. Execute Per-Test Verification Loop for each ID.
|
||||
3. Run class-level `RaftNodeTests` filter.
|
||||
4. Apply checkpoint protocol.
|
||||
5. Promote proven T4 IDs to `complete/verified` in `<=15` chunks.
|
||||
6. Commit checkpoint.
|
||||
|
||||
### Task 9: Final Batch 35 Closure
|
||||
|
||||
**Files:**
|
||||
- Modify: `porting.db`
|
||||
- Modify: `reports/current.md`
|
||||
|
||||
**Steps:**
|
||||
1. Run final audit checks:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- audit --type features --db porting.db
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- audit --type tests --db porting.db
|
||||
```
|
||||
2. Resolve any audit mismatches (or explicit override reasons).
|
||||
3. Complete batch:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch complete 35 --db porting.db
|
||||
```
|
||||
4. Generate updated report:
|
||||
```bash
|
||||
./reports/generate-report.sh
|
||||
```
|
||||
5. Final regression gate:
|
||||
```bash
|
||||
$DOTNET build dotnet/
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
```
|
||||
6. Commit final Batch 35 closure.
|
||||
156
docs/plans/2026-02-27-batch-36-stream-lifecycle-design.md
Normal file
156
docs/plans/2026-02-27-batch-36-stream-lifecycle-design.md
Normal file
@@ -0,0 +1,156 @@
|
||||
# Batch 36 Stream Lifecycle Design
|
||||
|
||||
**Date:** 2026-02-27
|
||||
**Batch:** 36 (`Stream Lifecycle`)
|
||||
**Scope:** 92 features + 53 unit tests
|
||||
**Dependencies:** Batches `8, 11, 12, 13, 14, 15, 28`
|
||||
**Go source:** `golang/nats-server/server/stream.go` (focus through ~line 4600)
|
||||
|
||||
## Problem
|
||||
|
||||
Batch 36 is the lifecycle/core control plane for JetStream streams: stream creation, assignment, config validation/update, purge/delete flows, mirror/source setup and message processing, and internal subscriptions. This batch is a gating dependency for Batch 37 (`Stream Messages`) and Batch 38 (`Consumer Lifecycle`), so fake progress here creates downstream breakage.
|
||||
|
||||
The current .NET surface is materially under-ported for this area:
|
||||
|
||||
- Batch 36 features are all `deferred`.
|
||||
- Most mapped methods are missing from source classes.
|
||||
- Existing `ImplBacklog` tests for related classes are largely placeholder-style and need replacement with behavior checks.
|
||||
|
||||
## Context Findings
|
||||
|
||||
### Required command outputs
|
||||
|
||||
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch show 36 --db porting.db`
|
||||
- Status: `pending`
|
||||
- Features: `92` (`3196-3293` with gaps)
|
||||
- Tests: `53`
|
||||
- Depends on: `8,11,12,13,14,15,28`
|
||||
- Go file: `server/stream.go`
|
||||
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch list --db porting.db`
|
||||
- Confirms chain: `36 -> 37,38`
|
||||
- `/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db`
|
||||
- Overall progress: `1924/6942 (27.7%)`
|
||||
|
||||
Environment note: `dotnet` is not on `PATH` in this shell. Use `/usr/local/share/dotnet/dotnet` explicitly.
|
||||
|
||||
### Feature ownership split (from `porting.db`)
|
||||
|
||||
- `NatsStream`: 76
|
||||
- `Account`: 4
|
||||
- `PersistModeType`: 3
|
||||
- `StreamSourceInfo`: 2
|
||||
- `StreamSource`: 2
|
||||
- `JsAccount`: 2
|
||||
- `NatsServer`: 1
|
||||
- `InMsg`: 1
|
||||
- `ExternalStream`: 1
|
||||
|
||||
### Test split (from `porting.db`)
|
||||
|
||||
- `JetStreamEngineTests`: 35
|
||||
- `NatsConsumerTests`: 4
|
||||
- `JetStreamClusterTests1`: 3
|
||||
- `JetStreamClusterTests2`: 4
|
||||
- `JetStreamClusterTests3`: 3
|
||||
- `ConcurrencyTests1`: 2
|
||||
- `JetStreamFileStoreTests`: 1
|
||||
- `StorageEngineTests`: 1
|
||||
|
||||
Additional finding: `JetStreamClusterTests1.Impltests.cs` and `JetStreamClusterTests3.Impltests.cs` do not currently exist and must be created.
|
||||
|
||||
## Approaches
|
||||
|
||||
### Approach A: Single-file port in existing files only
|
||||
|
||||
Port all methods directly into existing files (`NatsStream.cs`, `Account.cs`, `JetStreamTypes.cs`, etc.) without structural splits.
|
||||
|
||||
- Pros: minimal file churn.
|
||||
- Cons: high merge risk, hard reviewability, and poor fault isolation for 76 `NatsStream` methods.
|
||||
|
||||
### Approach B (Recommended): Lifecycle-focused partial-file decomposition + staged verification
|
||||
|
||||
Keep mapped owning classes intact but split implementation by concern into new partial files for stream lifecycle/mirror/source/subscription paths. Execute in bounded feature groups and test waves with strict evidence gates.
|
||||
|
||||
- Pros: manageable review units, clearer ownership, better checkpointing and rollback, aligns with anti-stub guardrails.
|
||||
- Cons: requires adding partial declarations where classes are currently non-partial.
|
||||
|
||||
### Approach C: Test-first broad wave before feature completion
|
||||
|
||||
Attempt to port all 53 tests up front to drive implementation.
|
||||
|
||||
- Pros: fast behavior pressure.
|
||||
- Cons: most tests depend on missing lifecycle internals and will churn heavily; expensive red-state noise.
|
||||
|
||||
**Decision:** Approach B.
|
||||
|
||||
## Proposed Design
|
||||
|
||||
### 1. Code organization strategy
|
||||
|
||||
Use concern-oriented partial files while preserving mapped class ownership:
|
||||
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs` (retain core type; convert to `partial`)
|
||||
- Create `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Lifecycle.cs`
|
||||
- Create `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Mirror.cs`
|
||||
- Create `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Source.cs`
|
||||
- Create `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Subscriptions.cs`
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs` (convert to `partial`)
|
||||
- Create `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.StreamLifecycle.cs`
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs` (for `JsAccount`, convert class to `partial` if split)
|
||||
- Create `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JsAccount.StreamLifecycle.cs`
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.cs` or `NatsServer.*.cs` partial for `CheckStreamCfg`
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StoreTypes.cs` and `StreamTypes.cs` for `PersistModeType`, `ExternalStream.Domain`, `StreamSource`/`StreamSourceInfo`, `InMsg`
|
||||
|
||||
### 2. Functional decomposition
|
||||
|
||||
- **Config and creation path:** `PersistModeType`, `ExternalStream.Domain`, `Account.AddStream*`, `StreamSource.ComposeIName/SetIndexName`, `NatsServer.CheckStreamCfg`, `JsAccount.ConfigUpdateCheck`, `NatsStream.Update*`.
|
||||
- **State/advisory path:** leader checks, sequence counters (`CLFS`), created time, create/update/delete advisories, purge/delete/erase behavior.
|
||||
- **Mirror path:** mirror setup/retry/cancel, inbound mirror flow-control handling, lag tracking, retry backoff.
|
||||
- **Source path:** source setup/retry/cancel, shared source message pump, source headers, start-sequence reconstruction.
|
||||
- **Subscription/internal path:** stream/direct/mirror-direct subscriptions, source consumer stop/unsubscribe, batching cleanup, internal subscribe helpers.
|
||||
|
||||
### 3. Test design
|
||||
|
||||
Port test IDs in waves tied to implemented behavior:
|
||||
|
||||
- Wave 1: file store + cluster/meta + consumer API basics (15 tests)
|
||||
- Wave 2-4: `JetStreamEngineTests` heavy stream lifecycle scenarios (12/12/14)
|
||||
|
||||
Test files:
|
||||
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/StorageEngineTests.Impltests.cs`
|
||||
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs`
|
||||
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs`
|
||||
|
||||
### 4. Verification and risk controls (design-level)
|
||||
|
||||
- Per-feature and per-test loops are mandatory before status promotion.
|
||||
- Stub scans are mandatory after each loop and at each checkpoint.
|
||||
- Build and test gates are mandatory before every status batch update.
|
||||
- Status updates are chunked to max 15 IDs.
|
||||
- If blocked: remain `deferred` with explicit reason, never placeholder implementation.
|
||||
|
||||
## Feature Grouping (for implementation plan)
|
||||
|
||||
- Group A (20): IDs `3196-3219` (actual mapped IDs, 20 total)
|
||||
- Group B (20): IDs `3220-3240` (actual mapped IDs, 20 total)
|
||||
- Group C (20): IDs `3241-3261` (actual mapped IDs, 20 total)
|
||||
- Group D (20): IDs `3262-3281` (actual mapped IDs, 20 total)
|
||||
- Group E (12): IDs `3282-3293` (actual mapped IDs, 12 total)
|
||||
|
||||
## Constraints
|
||||
|
||||
- Planning only in this session; no implementation execution.
|
||||
- Must preserve .NET standards (`xUnit 3`, `Shouldly`, `NSubstitute`, nullable, naming conventions).
|
||||
- Must avoid fake-pass tests and placeholder feature bodies.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Executing Batch 36 implementation in this session.
|
||||
- Expanding scope beyond Batch 36 mapped features/tests.
|
||||
- Building new integration infrastructure not required by mapped unit tests.
|
||||
@@ -0,0 +1,410 @@
|
||||
# Batch 36 Stream Lifecycle Implementation Plan
|
||||
|
||||
> **For Codex:** REQUIRED SUB-SKILL: Use `executeplan` to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Port and verify Batch 36 (`Stream Lifecycle`) from `server/stream.go` so all 92 mapped features and 53 mapped tests are either genuinely implemented/verified or explicitly deferred with documented blockers.
|
||||
|
||||
**Architecture:** Implement in five bounded feature groups (`20/20/20/20/12`) aligned to `NatsStream` lifecycle domains (config/update, mirror, source, subscriptions) plus supporting `Account`, `NatsServer`, `JsAccount`, and stream type helpers. Execute four test waves with strict per-test evidence and anti-stub enforcement before any status promotion.
|
||||
|
||||
**Tech Stack:** .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (`porting.db`)
|
||||
|
||||
**Design doc:** `docs/plans/2026-02-27-batch-36-stream-lifecycle-design.md`
|
||||
|
||||
---
|
||||
|
||||
## Batch 36 Scope
|
||||
|
||||
- Batch ID: `36`
|
||||
- Name: `Stream Lifecycle`
|
||||
- Dependencies: `8,11,12,13,14,15,28`
|
||||
- Go source: `golang/nats-server/server/stream.go`
|
||||
- Features: `92` (`3196-3293` mapped IDs)
|
||||
- Tests: `53`
|
||||
|
||||
If `dotnet` is not on `PATH`, use:
|
||||
|
||||
```bash
|
||||
DOTNET=/usr/local/share/dotnet/dotnet
|
||||
```
|
||||
|
||||
Primary production files (expected):
|
||||
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Lifecycle.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Mirror.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Source.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Subscriptions.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StoreTypes.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StreamTypes.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JsAccount.StreamLifecycle.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.StreamLifecycle.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.StreamLifecycle.cs`
|
||||
|
||||
Primary test files (expected):
|
||||
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/StorageEngineTests.Impltests.cs`
|
||||
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs`
|
||||
- Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs`
|
||||
|
||||
---
|
||||
|
||||
## MANDATORY VERIFICATION PROTOCOL
|
||||
|
||||
> **NON-NEGOTIABLE:** Every feature/test in Batch 36 must pass this loop. No shortcut status updates.
|
||||
|
||||
### Dependency Preflight (before any status change)
|
||||
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 8 --db porting.db
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 11 --db porting.db
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 12 --db porting.db
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 13 --db porting.db
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 14 --db porting.db
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 15 --db porting.db
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 28 --db porting.db
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 36 --db porting.db
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch ready --db porting.db
|
||||
```
|
||||
|
||||
Start only when ready:
|
||||
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch start 36 --db porting.db
|
||||
```
|
||||
|
||||
Capture baseline:
|
||||
|
||||
```bash
|
||||
$DOTNET build dotnet/
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
```
|
||||
|
||||
### Per-Feature Verification Loop (REQUIRED per feature ID)
|
||||
|
||||
1. Inspect mapping and Go context:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- feature show <feature_id> --db porting.db
|
||||
```
|
||||
2. Mark claim-in-progress:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status stub --db porting.db
|
||||
```
|
||||
3. Add/adjust focused test(s) for that feature behavior (red first when practical).
|
||||
4. Implement minimal real logic (no placeholder bodies).
|
||||
5. Run **Stub Detection Check** on touched files.
|
||||
6. Run **Build Gate**.
|
||||
7. Run targeted **Test Gate** for affected classes.
|
||||
8. If green, promote feature to `complete` (or `verified` only with checkpoint evidence).
|
||||
9. Record evidence before touching next feature.
|
||||
|
||||
### Per-Test Verification Loop (REQUIRED per test ID)
|
||||
|
||||
1. Inspect mapping and Go test location:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- test show <test_id> --db porting.db
|
||||
```
|
||||
2. Mark claim-in-progress:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- test update <test_id> --status stub --db porting.db
|
||||
```
|
||||
3. Port full Arrange/Act/Assert behavior against production code.
|
||||
4. Run the single test method:
|
||||
```bash
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
|
||||
--filter "FullyQualifiedName~<ClassName>.<MethodName>" --verbosity normal
|
||||
```
|
||||
5. Confirm output shows `Passed: 1, Failed: 0` (never `Passed: 0`).
|
||||
6. Run class-level filter for touched test class.
|
||||
7. Run **Stub Detection Check** + **Build Gate**.
|
||||
8. Promote test to `complete` (or `verified` only with checkpoint evidence).
|
||||
|
||||
### Stub Detection Check (REQUIRED after every loop)
|
||||
|
||||
Run on changed C# files in source + tests:
|
||||
|
||||
```bash
|
||||
changed_files=$(git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests | rg "\\.cs$" || true)
|
||||
if [ -n "$changed_files" ]; then
|
||||
echo "$changed_files" | xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|Assert\\.True\\(true\\)|Assert\\.Pass|ShouldContain\\(\"Should\"\\)|var goFile = \"server/|JetStreamVersioning\\.GetRequiredApiLevel\\(new Dictionary<string, string>\\)\\.ShouldBe\\(string\\.Empty\\)|=>\\s*default;|return\\s+default;|return\\s+null;\\s*$)"
|
||||
fi
|
||||
```
|
||||
|
||||
Any match in mapped Batch 36 methods/tests blocks promotion until resolved.
|
||||
|
||||
### Build Gate (REQUIRED)
|
||||
|
||||
```bash
|
||||
$DOTNET build dotnet/
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
- after every feature loop
|
||||
- after every test loop
|
||||
- before any batch-update
|
||||
- at every task checkpoint
|
||||
|
||||
### Test Gate (REQUIRED)
|
||||
|
||||
Minimum class/domain gates per touched area:
|
||||
|
||||
```bash
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~JetStreamEngineTests"
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~JetStreamClusterTests"
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~NatsConsumerTests"
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~StorageEngineTests|FullyQualifiedName~JetStreamFileStoreTests|FullyQualifiedName~ConcurrencyTests1"
|
||||
```
|
||||
|
||||
Checkpoint/full gate:
|
||||
|
||||
```bash
|
||||
$DOTNET test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
```
|
||||
|
||||
### Status Update Protocol (HARD LIMIT: max 15 IDs per batch-update)
|
||||
|
||||
- Never pass more than `15` IDs per `feature batch-update` or `test batch-update` call.
|
||||
- Never promote `verified` without loop evidence + checkpoint gate.
|
||||
- Never update IDs outside current task scope.
|
||||
- For blocked work, keep `deferred` with explicit reason.
|
||||
|
||||
Templates:
|
||||
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- \
|
||||
feature batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
|
||||
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- \
|
||||
test batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute
|
||||
```
|
||||
|
||||
### Checkpoint Protocol Between Tasks (REQUIRED)
|
||||
|
||||
Before starting the next task:
|
||||
|
||||
1. Run **Stub Detection Check** on current diff.
|
||||
2. Run **Build Gate**.
|
||||
3. Run class filters for all touched classes.
|
||||
4. Run full unit test gate.
|
||||
5. Promote only proven IDs in `<=15` chunks.
|
||||
6. Commit checkpoint before the next task.
|
||||
|
||||
---
|
||||
|
||||
## ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)
|
||||
|
||||
### Forbidden Patterns
|
||||
|
||||
These are forbidden in mapped Batch 36 features/tests:
|
||||
|
||||
- `throw new NotImplementedException()`
|
||||
- Empty or no-op mapped method bodies
|
||||
- Placeholder comments (`TODO`, `PLACEHOLDER`, `later`)
|
||||
- Fake-pass assertions (`Assert.True(true)`, `Assert.Pass()`, tautological string checks)
|
||||
- Template test bodies that only assert literals, including patterns such as:
|
||||
- `var goFile = "server/..."`
|
||||
- `"<MethodName>".ShouldContain("Should")`
|
||||
- `JetStreamVersioning.GetRequiredApiLevel(new Dictionary<string, string>()).ShouldBe(string.Empty)` as the core assertion
|
||||
- Constant-return shortcuts in mapped feature methods that ignore state/input (`return default;`, `return null;`, hardcoded primitives) unless directly required by Go behavior and covered by tests
|
||||
- Catch-and-ignore blocks used to force green tests
|
||||
|
||||
### Hard Limits
|
||||
|
||||
- Feature groups must stay at `~20` IDs max.
|
||||
- `feature batch-update` maximum IDs per call: `15`.
|
||||
- `test batch-update` maximum IDs per call: `15`.
|
||||
- One active feature loop at a time.
|
||||
- One active test loop at a time.
|
||||
- Mandatory checkpoint between every task.
|
||||
- No `verified` promotion without checkpoint evidence.
|
||||
|
||||
### If You Get Stuck (REQUIRED)
|
||||
|
||||
1. Stop on the current ID immediately.
|
||||
2. Do **not** add placeholder code or fake assertions.
|
||||
3. Mark blocked item `deferred` with explicit reason.
|
||||
4. Move to next unblocked ID.
|
||||
|
||||
Feature deferral:
|
||||
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- \
|
||||
feature update <feature_id> --status deferred --override "blocked: <specific reason>" --db porting.db
|
||||
```
|
||||
|
||||
Test deferral:
|
||||
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- \
|
||||
test update <test_id> --status deferred --override "blocked: <specific reason>" --db porting.db
|
||||
```
|
||||
|
||||
`deferred` with a concrete blocker is correct. Stubbed progress is not.
|
||||
|
||||
---
|
||||
|
||||
## Feature Groups (max ~20 each)
|
||||
|
||||
### Group A (20): Stream creation, assignment, leadership primitives
|
||||
|
||||
IDs:
|
||||
`3196,3197,3198,3200,3201,3202,3203,3204,3205,3206,3207,3208,3209,3210,3211,3213,3214,3216,3217,3219`
|
||||
|
||||
### Group B (20): Limits, dedupe, advisories, config validation/update entry points
|
||||
|
||||
IDs:
|
||||
`3220,3221,3222,3223,3224,3225,3226,3227,3228,3229,3230,3231,3232,3233,3234,3235,3237,3238,3239,3240`
|
||||
|
||||
### Group C (20): Update internals, purge/delete, mirror processing/retry foundation
|
||||
|
||||
IDs:
|
||||
`3241,3242,3244,3245,3246,3247,3248,3249,3250,3251,3252,3253,3254,3255,3256,3257,3258,3259,3260,3261`
|
||||
|
||||
### Group D (20): Source consumer setup, flow control, source header/sequence recovery
|
||||
|
||||
IDs:
|
||||
`3262,3263,3264,3265,3266,3267,3268,3269,3270,3271,3272,3273,3274,3275,3276,3277,3278,3279,3280,3281`
|
||||
|
||||
### Group E (12): Direct/mirror-direct subscriptions, internal subscribe/unsubscribe cleanup
|
||||
|
||||
IDs:
|
||||
`3282,3283,3284,3285,3286,3287,3288,3289,3290,3291,3292,3293`
|
||||
|
||||
## Test Waves
|
||||
|
||||
### Wave T1 (15): file-store + cluster + consumer API basics
|
||||
|
||||
IDs:
|
||||
`548,849,869,902,951,953,955,990,1032,1034,1125,1227,1262,1315,1317`
|
||||
|
||||
### Wave T2 (12): stream create/update/publish baseline
|
||||
|
||||
IDs:
|
||||
`1468,1470,1471,1474,1475,1478,1480,1481,1482,1483,1504,1512`
|
||||
|
||||
### Wave T3 (12): limits, perf-sensitive behavior, republish cases
|
||||
|
||||
IDs:
|
||||
`1532,1544,1554,1555,1564,1565,1614,1623,1635,1636,1637,1638`
|
||||
|
||||
### Wave T4 (14): republish tail, direct get, versioning, concurrency/storage
|
||||
|
||||
IDs:
|
||||
`1639,1640,1644,1645,1646,1653,1656,1657,1669,1741,1743,2387,2402,2952`
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Preflight and Baseline
|
||||
|
||||
**Files:**
|
||||
- Read: `docs/plans/2026-02-27-batch-36-stream-lifecycle-design.md`
|
||||
- Read: `golang/nats-server/server/stream.go`
|
||||
|
||||
**Steps:**
|
||||
1. Run dependency preflight commands.
|
||||
2. Start Batch 36 only if ready.
|
||||
3. Capture baseline build/test evidence.
|
||||
4. Do not promote any IDs before baseline evidence exists.
|
||||
|
||||
### Task 2: Implement Feature Group A (20 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StoreTypes.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.StreamLifecycle.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Lifecycle.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Move Group A IDs to `stub` using `15 + 5` update chunks.
|
||||
2. Execute Per-Feature Verification Loop per ID.
|
||||
3. Run mandatory gates after each ID.
|
||||
4. Promote proven IDs (`complete/verified`) in `<=15` chunks.
|
||||
5. Run Checkpoint Protocol.
|
||||
|
||||
### Task 3: Implement Feature Group B (20 IDs) + Test Wave T1 (15 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.StreamLifecycle.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JsAccount.StreamLifecycle.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Lifecycle.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs`
|
||||
- Modify/Create: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Implement Group B features with per-feature loop and gates.
|
||||
2. Port Wave T1 tests with per-test loop and gates.
|
||||
3. Apply status updates in max-15 chunks only.
|
||||
4. Run Checkpoint Protocol.
|
||||
|
||||
### Task 4: Implement Feature Group C (20 IDs) + Test Wave T2 (12 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Lifecycle.cs`
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Mirror.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Implement Group C feature IDs with per-feature loop.
|
||||
2. Port Wave T2 tests.
|
||||
3. Run stub/build/test gates and checkpoint.
|
||||
4. Promote only proven IDs in `<=15` chunks.
|
||||
|
||||
### Task 5: Implement Feature Group D (20 IDs) + Test Wave T3 (12 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Source.cs`
|
||||
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/StreamTypes.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Implement Group D features with per-feature loop.
|
||||
2. Port Wave T3 tests.
|
||||
3. Run mandatory gates and checkpoint.
|
||||
4. Promote only proven IDs in `<=15` chunks.
|
||||
|
||||
### Task 6: Implement Feature Group E (12 IDs) + Test Wave T4 (14 IDs)
|
||||
|
||||
**Files:**
|
||||
- Modify/Create: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/NatsStream.Subscriptions.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/StorageEngineTests.Impltests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Implement Group E feature IDs.
|
||||
2. Port Wave T4 tests.
|
||||
3. Run mandatory gates and checkpoint.
|
||||
4. Promote proven IDs in `<=15` chunks.
|
||||
|
||||
### Task 7: Batch 36 Final Closure
|
||||
|
||||
**Files:**
|
||||
- Modify: `porting.db`
|
||||
- Generate: `reports/current.md` (via report script)
|
||||
|
||||
**Steps:**
|
||||
1. Run final stub scan on all touched source/test files.
|
||||
2. Run full build and full unit suite.
|
||||
3. Validate batch state:
|
||||
```bash
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- batch show 36 --db porting.db
|
||||
$DOTNET run --project tools/NatsNet.PortTracker -- report summary --db porting.db
|
||||
```
|
||||
4. Ensure every Batch 36 ID is `complete/verified/n_a` or `deferred` with explicit blocker notes.
|
||||
5. Generate report:
|
||||
```bash
|
||||
./reports/generate-report.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Execution note:** Planning only. Do not execute this plan in this session.
|
||||
Reference in New Issue
Block a user